annotate mercurial/wireprotov1peer.py @ 48178:f12a19d03d2c

fix: reduce number of tool executions By grouping together (path, ctx) pairs according to the inputs they would provide to fixer tools, we can deduplicate executions of fixer tools to significantly reduce the amount of time spent running slow tools. This change does not handle clean files in the working copy, which could still be deduplicated against the files in the checked out commit. It's a little harder to do that because the filerev is not available in the workingfilectx (and it doesn't exist for added files). Anecdotally, this change makes some real uses cases at Google 10x faster. I think we were originally hesitant to do this because the benefits weren't obvious, and implementing it efficiently is kind of tricky. If we simply memoized the formatter execution function, we would be keeping tons of file content in memory. Also included is a regression test for a corner case that I broke with my first attempt at optimizing this code. Differential Revision: https://phab.mercurial-scm.org/D11280
author Danny Hooper <hooper@google.com>
date Thu, 02 Sep 2021 14:08:45 -0700
parents c424ff4807e6
children a0da5075bca3
Ignore whitespace changes - Everywhere: Within whitespace: At end of lines:
rev   line source
37614
a81d02ea65db wireproto: move version 1 peer functionality to standalone module (API)
Gregory Szorc <gregory.szorc@gmail.com>
parents: 37613
diff changeset
1 # wireprotov1peer.py - Client-side functionality for wire protocol version 1.
11581
4530b3307fb9 protocol: introduce wireproto.py
Matt Mackall <mpm@selenic.com>
parents:
diff changeset
2 #
46819
d4ba4d51f85f contributor: change mentions of mpm to olivia
Rapha?l Gom?s <rgomes@octobus.net>
parents: 46672
diff changeset
3 # Copyright 2005-2010 Olivia Mackall <olivia@selenic.com>
11581
4530b3307fb9 protocol: introduce wireproto.py
Matt Mackall <mpm@selenic.com>
parents:
diff changeset
4 #
4530b3307fb9 protocol: introduce wireproto.py
Matt Mackall <mpm@selenic.com>
parents:
diff changeset
5 # This software may be used and distributed according to the terms of the
4530b3307fb9 protocol: introduce wireproto.py
Matt Mackall <mpm@selenic.com>
parents:
diff changeset
6 # GNU General Public License version 2 or any later version.
4530b3307fb9 protocol: introduce wireproto.py
Matt Mackall <mpm@selenic.com>
parents:
diff changeset
7
25993
0851678be71b wireproto: use absolute_import
Gregory Szorc <gregory.szorc@gmail.com>
parents: 25913
diff changeset
8 from __future__ import absolute_import
0851678be71b wireproto: use absolute_import
Gregory Szorc <gregory.szorc@gmail.com>
parents: 25913
diff changeset
9
37630
e1b32dc4646c wireproto: implement command executor interface for version 1 peers
Gregory Szorc <gregory.szorc@gmail.com>
parents: 37615
diff changeset
10 import sys
37631
2f626233859b wireproto: implement batching on peer executor interface
Gregory Szorc <gregory.szorc@gmail.com>
parents: 37630
diff changeset
11 import weakref
11581
4530b3307fb9 protocol: introduce wireproto.py
Matt Mackall <mpm@selenic.com>
parents:
diff changeset
12
25993
0851678be71b wireproto: use absolute_import
Gregory Szorc <gregory.szorc@gmail.com>
parents: 25913
diff changeset
13 from .i18n import _
43076
2372284d9457 formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents: 42814
diff changeset
14 from .node import bin
43089
c59eb1560c44 py3: manually import getattr where it is needed
Gregory Szorc <gregory.szorc@gmail.com>
parents: 43087
diff changeset
15 from .pycompat import (
c59eb1560c44 py3: manually import getattr where it is needed
Gregory Szorc <gregory.szorc@gmail.com>
parents: 43087
diff changeset
16 getattr,
c59eb1560c44 py3: manually import getattr where it is needed
Gregory Szorc <gregory.szorc@gmail.com>
parents: 43087
diff changeset
17 setattr,
c59eb1560c44 py3: manually import getattr where it is needed
Gregory Szorc <gregory.szorc@gmail.com>
parents: 43087
diff changeset
18 )
25993
0851678be71b wireproto: use absolute_import
Gregory Szorc <gregory.szorc@gmail.com>
parents: 25913
diff changeset
19 from . import (
0851678be71b wireproto: use absolute_import
Gregory Szorc <gregory.szorc@gmail.com>
parents: 25913
diff changeset
20 bundle2,
0851678be71b wireproto: use absolute_import
Gregory Szorc <gregory.szorc@gmail.com>
parents: 25913
diff changeset
21 changegroup as changegroupmod,
0851678be71b wireproto: use absolute_import
Gregory Szorc <gregory.szorc@gmail.com>
parents: 25913
diff changeset
22 encoding,
0851678be71b wireproto: use absolute_import
Gregory Szorc <gregory.szorc@gmail.com>
parents: 25913
diff changeset
23 error,
0851678be71b wireproto: use absolute_import
Gregory Szorc <gregory.szorc@gmail.com>
parents: 25913
diff changeset
24 pushkey as pushkeymod,
30924
48dea083f66d py3: convert the mode argument of os.fdopen to unicodes (1 of 2)
Pulkit Goyal <7895pulkit@gmail.com>
parents: 30914
diff changeset
25 pycompat,
25993
0851678be71b wireproto: use absolute_import
Gregory Szorc <gregory.szorc@gmail.com>
parents: 25913
diff changeset
26 util,
36073
cd6ab329c5c7 wireprototypes: move wire protocol response types to new module
Gregory Szorc <gregory.szorc@gmail.com>
parents: 36071
diff changeset
27 wireprototypes,
25993
0851678be71b wireproto: use absolute_import
Gregory Szorc <gregory.szorc@gmail.com>
parents: 25913
diff changeset
28 )
42813
268662aac075 interfaces: create a new folder for interfaces and move repository.py in it
Pulkit Goyal <pulkit@yandex-team.ru>
parents: 41055
diff changeset
29 from .interfaces import (
268662aac075 interfaces: create a new folder for interfaces and move repository.py in it
Pulkit Goyal <pulkit@yandex-team.ru>
parents: 41055
diff changeset
30 repository,
42814
2c4f656c8e9f interfaceutil: move to interfaces/
Pulkit Goyal <pulkit@yandex-team.ru>
parents: 42813
diff changeset
31 util as interfaceutil,
37810
856f381ad74b interfaceutil: module to stub out zope.interface
Gregory Szorc <gregory.szorc@gmail.com>
parents: 37669
diff changeset
32 )
44060
a61287a95dc3 core: migrate uses of hashlib.sha1 to hashutil.sha1
Augie Fackler <augie@google.com>
parents: 43506
diff changeset
33 from .utils import hashutil
20903
8d477543882b wireproto: introduce an abstractserverproto class
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 20902
diff changeset
34
28883
032c4c2f802a pycompat: switch to util.urlreq/util.urlerr for py3 compat
timeless <timeless@mozdev.org>
parents: 28666
diff changeset
35 urlreq = util.urlreq
032c4c2f802a pycompat: switch to util.urlreq/util.urlerr for py3 compat
timeless <timeless@mozdev.org>
parents: 28666
diff changeset
36
43076
2372284d9457 formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents: 42814
diff changeset
37
47873
c424ff4807e6 wireprotov1peer: update all rpcs to use the new batchable scheme
Valentin Gatien-Baron <valentin.gatienbaron@gmail.com>
parents: 47872
diff changeset
38 def batchable(f):
45942
89a2afe31e82 formating: upgrade to black 20.8b1
Augie Fackler <raf@durin42.com>
parents: 44060
diff changeset
39 """annotation for batchable methods
37615
f3dc8239e3a9 peer: scatter module to the wind (API)
Gregory Szorc <gregory.szorc@gmail.com>
parents: 37614
diff changeset
40
f3dc8239e3a9 peer: scatter module to the wind (API)
Gregory Szorc <gregory.szorc@gmail.com>
parents: 37614
diff changeset
41 Such methods must implement a coroutine as follows:
f3dc8239e3a9 peer: scatter module to the wind (API)
Gregory Szorc <gregory.szorc@gmail.com>
parents: 37614
diff changeset
42
f3dc8239e3a9 peer: scatter module to the wind (API)
Gregory Szorc <gregory.szorc@gmail.com>
parents: 37614
diff changeset
43 @batchable
f3dc8239e3a9 peer: scatter module to the wind (API)
Gregory Szorc <gregory.szorc@gmail.com>
parents: 37614
diff changeset
44 def sample(self, one, two=None):
f3dc8239e3a9 peer: scatter module to the wind (API)
Gregory Szorc <gregory.szorc@gmail.com>
parents: 37614
diff changeset
45 # Build list of encoded arguments suitable for your wire protocol:
46480
05dd091dfa6a wireprotopeer: clarify some variable names now that we allow snake_case
Martin von Zweigbergk <martinvonz@google.com>
parents: 45942
diff changeset
46 encoded_args = [('one', encode(one),), ('two', encode(two),)]
47872
cdad6560e832 wireprotov1peer: simplify the way batchable rpcs are defined
Valentin Gatien-Baron <valentin.gatienbaron@gmail.com>
parents: 46819
diff changeset
47 # Return it, along with a function that will receive the result
cdad6560e832 wireprotov1peer: simplify the way batchable rpcs are defined
Valentin Gatien-Baron <valentin.gatienbaron@gmail.com>
parents: 46819
diff changeset
48 # from the batched request.
cdad6560e832 wireprotov1peer: simplify the way batchable rpcs are defined
Valentin Gatien-Baron <valentin.gatienbaron@gmail.com>
parents: 46819
diff changeset
49 return encoded_args, decode
37615
f3dc8239e3a9 peer: scatter module to the wind (API)
Gregory Szorc <gregory.szorc@gmail.com>
parents: 37614
diff changeset
50
f3dc8239e3a9 peer: scatter module to the wind (API)
Gregory Szorc <gregory.szorc@gmail.com>
parents: 37614
diff changeset
51 The decorator returns a function which wraps this coroutine as a plain
f3dc8239e3a9 peer: scatter module to the wind (API)
Gregory Szorc <gregory.szorc@gmail.com>
parents: 37614
diff changeset
52 method, but adds the original method as an attribute called "batchable",
f3dc8239e3a9 peer: scatter module to the wind (API)
Gregory Szorc <gregory.szorc@gmail.com>
parents: 37614
diff changeset
53 which is used by remotebatch to split the call into separate encoding and
f3dc8239e3a9 peer: scatter module to the wind (API)
Gregory Szorc <gregory.szorc@gmail.com>
parents: 37614
diff changeset
54 decoding phases.
45942
89a2afe31e82 formating: upgrade to black 20.8b1
Augie Fackler <raf@durin42.com>
parents: 44060
diff changeset
55 """
43076
2372284d9457 formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents: 42814
diff changeset
56
37615
f3dc8239e3a9 peer: scatter module to the wind (API)
Gregory Szorc <gregory.szorc@gmail.com>
parents: 37614
diff changeset
57 def plain(*args, **opts):
47872
cdad6560e832 wireprotov1peer: simplify the way batchable rpcs are defined
Valentin Gatien-Baron <valentin.gatienbaron@gmail.com>
parents: 46819
diff changeset
58 encoded_args_or_res, decode = f(*args, **opts)
cdad6560e832 wireprotov1peer: simplify the way batchable rpcs are defined
Valentin Gatien-Baron <valentin.gatienbaron@gmail.com>
parents: 46819
diff changeset
59 if not decode:
46480
05dd091dfa6a wireprotopeer: clarify some variable names now that we allow snake_case
Martin von Zweigbergk <martinvonz@google.com>
parents: 45942
diff changeset
60 return encoded_args_or_res # a local result in this case
37615
f3dc8239e3a9 peer: scatter module to the wind (API)
Gregory Szorc <gregory.szorc@gmail.com>
parents: 37614
diff changeset
61 self = args[0]
f3dc8239e3a9 peer: scatter module to the wind (API)
Gregory Szorc <gregory.szorc@gmail.com>
parents: 37614
diff changeset
62 cmd = pycompat.bytesurl(f.__name__) # ensure cmd is ascii bytestr
47872
cdad6560e832 wireprotov1peer: simplify the way batchable rpcs are defined
Valentin Gatien-Baron <valentin.gatienbaron@gmail.com>
parents: 46819
diff changeset
63 encoded_res = self._submitone(cmd, encoded_args_or_res)
cdad6560e832 wireprotov1peer: simplify the way batchable rpcs are defined
Valentin Gatien-Baron <valentin.gatienbaron@gmail.com>
parents: 46819
diff changeset
64 return decode(encoded_res)
43076
2372284d9457 formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents: 42814
diff changeset
65
37615
f3dc8239e3a9 peer: scatter module to the wind (API)
Gregory Szorc <gregory.szorc@gmail.com>
parents: 37614
diff changeset
66 setattr(plain, 'batchable', f)
39588
f15a587d2dfc wireprotov1peer: forward __name__ of wrapped method in batchable decorator
Augie Fackler <raf@durin42.com>
parents: 38783
diff changeset
67 setattr(plain, '__name__', f.__name__)
37615
f3dc8239e3a9 peer: scatter module to the wind (API)
Gregory Szorc <gregory.szorc@gmail.com>
parents: 37614
diff changeset
68 return plain
f3dc8239e3a9 peer: scatter module to the wind (API)
Gregory Szorc <gregory.szorc@gmail.com>
parents: 37614
diff changeset
69
43076
2372284d9457 formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents: 42814
diff changeset
70
29733
bb04f96df51c wireproto: consolidate code for obtaining "cmds" argument value
Gregory Szorc <gregory.szorc@gmail.com>
parents: 29706
diff changeset
71 def encodebatchcmds(req):
bb04f96df51c wireproto: consolidate code for obtaining "cmds" argument value
Gregory Szorc <gregory.szorc@gmail.com>
parents: 29706
diff changeset
72 """Return a ``cmds`` argument value for the ``batch`` command."""
37612
5e71dea79aae wireproto: move value encoding functions to wireprototypes (API)
Gregory Szorc <gregory.szorc@gmail.com>
parents: 37546
diff changeset
73 escapearg = wireprototypes.escapebatcharg
5e71dea79aae wireproto: move value encoding functions to wireprototypes (API)
Gregory Szorc <gregory.szorc@gmail.com>
parents: 37546
diff changeset
74
29733
bb04f96df51c wireproto: consolidate code for obtaining "cmds" argument value
Gregory Szorc <gregory.szorc@gmail.com>
parents: 29706
diff changeset
75 cmds = []
bb04f96df51c wireproto: consolidate code for obtaining "cmds" argument value
Gregory Szorc <gregory.szorc@gmail.com>
parents: 29706
diff changeset
76 for op, argsdict in req:
29734
62e2e048d068 wireproto: unescape argument names in batch command (BC)
Gregory Szorc <gregory.szorc@gmail.com>
parents: 29733
diff changeset
77 # Old servers didn't properly unescape argument names. So prevent
62e2e048d068 wireproto: unescape argument names in batch command (BC)
Gregory Szorc <gregory.szorc@gmail.com>
parents: 29733
diff changeset
78 # the sending of argument names that may not be decoded properly by
62e2e048d068 wireproto: unescape argument names in batch command (BC)
Gregory Szorc <gregory.szorc@gmail.com>
parents: 29733
diff changeset
79 # servers.
62e2e048d068 wireproto: unescape argument names in batch command (BC)
Gregory Szorc <gregory.szorc@gmail.com>
parents: 29733
diff changeset
80 assert all(escapearg(k) == k for k in argsdict)
62e2e048d068 wireproto: unescape argument names in batch command (BC)
Gregory Szorc <gregory.szorc@gmail.com>
parents: 29733
diff changeset
81
43077
687b865b95ad formatting: byteify all mercurial/ and hgext/ string literals
Augie Fackler <augie@google.com>
parents: 43076
diff changeset
82 args = b','.join(
687b865b95ad formatting: byteify all mercurial/ and hgext/ string literals
Augie Fackler <augie@google.com>
parents: 43076
diff changeset
83 b'%s=%s' % (escapearg(k), escapearg(v))
43106
d783f945a701 py3: finish porting iteritems() to pycompat and remove source transformer
Gregory Szorc <gregory.szorc@gmail.com>
parents: 43089
diff changeset
84 for k, v in pycompat.iteritems(argsdict)
43076
2372284d9457 formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents: 42814
diff changeset
85 )
43077
687b865b95ad formatting: byteify all mercurial/ and hgext/ string literals
Augie Fackler <augie@google.com>
parents: 43076
diff changeset
86 cmds.append(b'%s %s' % (op, args))
29733
bb04f96df51c wireproto: consolidate code for obtaining "cmds" argument value
Gregory Szorc <gregory.szorc@gmail.com>
parents: 29706
diff changeset
87
43077
687b865b95ad formatting: byteify all mercurial/ and hgext/ string literals
Augie Fackler <augie@google.com>
parents: 43076
diff changeset
88 return b';'.join(cmds)
29733
bb04f96df51c wireproto: consolidate code for obtaining "cmds" argument value
Gregory Szorc <gregory.szorc@gmail.com>
parents: 29706
diff changeset
89
43076
2372284d9457 formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents: 42814
diff changeset
90
37631
2f626233859b wireproto: implement batching on peer executor interface
Gregory Szorc <gregory.szorc@gmail.com>
parents: 37630
diff changeset
91 class unsentfuture(pycompat.futures.Future):
2f626233859b wireproto: implement batching on peer executor interface
Gregory Szorc <gregory.szorc@gmail.com>
parents: 37630
diff changeset
92 """A Future variation to represent an unsent command.
2f626233859b wireproto: implement batching on peer executor interface
Gregory Szorc <gregory.szorc@gmail.com>
parents: 37630
diff changeset
93
2f626233859b wireproto: implement batching on peer executor interface
Gregory Szorc <gregory.szorc@gmail.com>
parents: 37630
diff changeset
94 Because we buffer commands and don't submit them immediately, calling
2f626233859b wireproto: implement batching on peer executor interface
Gregory Szorc <gregory.szorc@gmail.com>
parents: 37630
diff changeset
95 ``result()`` on an unsent future could deadlock. Futures for buffered
2f626233859b wireproto: implement batching on peer executor interface
Gregory Szorc <gregory.szorc@gmail.com>
parents: 37630
diff changeset
96 commands are represented by this type, which wraps ``result()`` to
2f626233859b wireproto: implement batching on peer executor interface
Gregory Szorc <gregory.szorc@gmail.com>
parents: 37630
diff changeset
97 call ``sendcommands()``.
2f626233859b wireproto: implement batching on peer executor interface
Gregory Szorc <gregory.szorc@gmail.com>
parents: 37630
diff changeset
98 """
2f626233859b wireproto: implement batching on peer executor interface
Gregory Szorc <gregory.szorc@gmail.com>
parents: 37630
diff changeset
99
2f626233859b wireproto: implement batching on peer executor interface
Gregory Szorc <gregory.szorc@gmail.com>
parents: 37630
diff changeset
100 def result(self, timeout=None):
2f626233859b wireproto: implement batching on peer executor interface
Gregory Szorc <gregory.szorc@gmail.com>
parents: 37630
diff changeset
101 if self.done():
2f626233859b wireproto: implement batching on peer executor interface
Gregory Szorc <gregory.szorc@gmail.com>
parents: 37630
diff changeset
102 return pycompat.futures.Future.result(self, timeout)
2f626233859b wireproto: implement batching on peer executor interface
Gregory Szorc <gregory.szorc@gmail.com>
parents: 37630
diff changeset
103
2f626233859b wireproto: implement batching on peer executor interface
Gregory Szorc <gregory.szorc@gmail.com>
parents: 37630
diff changeset
104 self._peerexecutor.sendcommands()
2f626233859b wireproto: implement batching on peer executor interface
Gregory Szorc <gregory.szorc@gmail.com>
parents: 37630
diff changeset
105
2f626233859b wireproto: implement batching on peer executor interface
Gregory Szorc <gregory.szorc@gmail.com>
parents: 37630
diff changeset
106 # This looks like it will infinitely recurse. However,
2f626233859b wireproto: implement batching on peer executor interface
Gregory Szorc <gregory.szorc@gmail.com>
parents: 37630
diff changeset
107 # sendcommands() should modify __class__. This call serves as a check
2f626233859b wireproto: implement batching on peer executor interface
Gregory Szorc <gregory.szorc@gmail.com>
parents: 37630
diff changeset
108 # on that.
2f626233859b wireproto: implement batching on peer executor interface
Gregory Szorc <gregory.szorc@gmail.com>
parents: 37630
diff changeset
109 return self.result(timeout)
2f626233859b wireproto: implement batching on peer executor interface
Gregory Szorc <gregory.szorc@gmail.com>
parents: 37630
diff changeset
110
43076
2372284d9457 formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents: 42814
diff changeset
111
37810
856f381ad74b interfaceutil: module to stub out zope.interface
Gregory Szorc <gregory.szorc@gmail.com>
parents: 37669
diff changeset
112 @interfaceutil.implementer(repository.ipeercommandexecutor)
37630
e1b32dc4646c wireproto: implement command executor interface for version 1 peers
Gregory Szorc <gregory.szorc@gmail.com>
parents: 37615
diff changeset
113 class peerexecutor(object):
e1b32dc4646c wireproto: implement command executor interface for version 1 peers
Gregory Szorc <gregory.szorc@gmail.com>
parents: 37615
diff changeset
114 def __init__(self, peer):
e1b32dc4646c wireproto: implement command executor interface for version 1 peers
Gregory Szorc <gregory.szorc@gmail.com>
parents: 37615
diff changeset
115 self._peer = peer
e1b32dc4646c wireproto: implement command executor interface for version 1 peers
Gregory Szorc <gregory.szorc@gmail.com>
parents: 37615
diff changeset
116 self._sent = False
e1b32dc4646c wireproto: implement command executor interface for version 1 peers
Gregory Szorc <gregory.szorc@gmail.com>
parents: 37615
diff changeset
117 self._closed = False
e1b32dc4646c wireproto: implement command executor interface for version 1 peers
Gregory Szorc <gregory.szorc@gmail.com>
parents: 37615
diff changeset
118 self._calls = []
37631
2f626233859b wireproto: implement batching on peer executor interface
Gregory Szorc <gregory.szorc@gmail.com>
parents: 37630
diff changeset
119 self._futures = weakref.WeakSet()
2f626233859b wireproto: implement batching on peer executor interface
Gregory Szorc <gregory.szorc@gmail.com>
parents: 37630
diff changeset
120 self._responseexecutor = None
2f626233859b wireproto: implement batching on peer executor interface
Gregory Szorc <gregory.szorc@gmail.com>
parents: 37630
diff changeset
121 self._responsef = None
37630
e1b32dc4646c wireproto: implement command executor interface for version 1 peers
Gregory Szorc <gregory.szorc@gmail.com>
parents: 37615
diff changeset
122
e1b32dc4646c wireproto: implement command executor interface for version 1 peers
Gregory Szorc <gregory.szorc@gmail.com>
parents: 37615
diff changeset
123 def __enter__(self):
e1b32dc4646c wireproto: implement command executor interface for version 1 peers
Gregory Szorc <gregory.szorc@gmail.com>
parents: 37615
diff changeset
124 return self
e1b32dc4646c wireproto: implement command executor interface for version 1 peers
Gregory Szorc <gregory.szorc@gmail.com>
parents: 37615
diff changeset
125
e1b32dc4646c wireproto: implement command executor interface for version 1 peers
Gregory Szorc <gregory.szorc@gmail.com>
parents: 37615
diff changeset
126 def __exit__(self, exctype, excvalee, exctb):
e1b32dc4646c wireproto: implement command executor interface for version 1 peers
Gregory Szorc <gregory.szorc@gmail.com>
parents: 37615
diff changeset
127 self.close()
e1b32dc4646c wireproto: implement command executor interface for version 1 peers
Gregory Szorc <gregory.szorc@gmail.com>
parents: 37615
diff changeset
128
e1b32dc4646c wireproto: implement command executor interface for version 1 peers
Gregory Szorc <gregory.szorc@gmail.com>
parents: 37615
diff changeset
129 def callcommand(self, command, args):
e1b32dc4646c wireproto: implement command executor interface for version 1 peers
Gregory Szorc <gregory.szorc@gmail.com>
parents: 37615
diff changeset
130 if self._sent:
43076
2372284d9457 formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents: 42814
diff changeset
131 raise error.ProgrammingError(
43117
8ff1ecfadcd1 cleanup: join string literals that are already on one line
Martin von Zweigbergk <martinvonz@google.com>
parents: 43106
diff changeset
132 b'callcommand() cannot be used after commands are sent'
43076
2372284d9457 formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents: 42814
diff changeset
133 )
37630
e1b32dc4646c wireproto: implement command executor interface for version 1 peers
Gregory Szorc <gregory.szorc@gmail.com>
parents: 37615
diff changeset
134
e1b32dc4646c wireproto: implement command executor interface for version 1 peers
Gregory Szorc <gregory.szorc@gmail.com>
parents: 37615
diff changeset
135 if self._closed:
43076
2372284d9457 formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents: 42814
diff changeset
136 raise error.ProgrammingError(
43117
8ff1ecfadcd1 cleanup: join string literals that are already on one line
Martin von Zweigbergk <martinvonz@google.com>
parents: 43106
diff changeset
137 b'callcommand() cannot be used after close()'
43076
2372284d9457 formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents: 42814
diff changeset
138 )
37630
e1b32dc4646c wireproto: implement command executor interface for version 1 peers
Gregory Szorc <gregory.szorc@gmail.com>
parents: 37615
diff changeset
139
e1b32dc4646c wireproto: implement command executor interface for version 1 peers
Gregory Szorc <gregory.szorc@gmail.com>
parents: 37615
diff changeset
140 # Commands are dispatched through methods on the peer.
e1b32dc4646c wireproto: implement command executor interface for version 1 peers
Gregory Szorc <gregory.szorc@gmail.com>
parents: 37615
diff changeset
141 fn = getattr(self._peer, pycompat.sysstr(command), None)
e1b32dc4646c wireproto: implement command executor interface for version 1 peers
Gregory Szorc <gregory.szorc@gmail.com>
parents: 37615
diff changeset
142
e1b32dc4646c wireproto: implement command executor interface for version 1 peers
Gregory Szorc <gregory.szorc@gmail.com>
parents: 37615
diff changeset
143 if not fn:
e1b32dc4646c wireproto: implement command executor interface for version 1 peers
Gregory Szorc <gregory.szorc@gmail.com>
parents: 37615
diff changeset
144 raise error.ProgrammingError(
43077
687b865b95ad formatting: byteify all mercurial/ and hgext/ string literals
Augie Fackler <augie@google.com>
parents: 43076
diff changeset
145 b'cannot call command %s: method of same name not available '
687b865b95ad formatting: byteify all mercurial/ and hgext/ string literals
Augie Fackler <augie@google.com>
parents: 43076
diff changeset
146 b'on peer' % command
43076
2372284d9457 formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents: 42814
diff changeset
147 )
37630
e1b32dc4646c wireproto: implement command executor interface for version 1 peers
Gregory Szorc <gregory.szorc@gmail.com>
parents: 37615
diff changeset
148
e1b32dc4646c wireproto: implement command executor interface for version 1 peers
Gregory Szorc <gregory.szorc@gmail.com>
parents: 37615
diff changeset
149 # Commands are either batchable or they aren't. If a command
e1b32dc4646c wireproto: implement command executor interface for version 1 peers
Gregory Szorc <gregory.szorc@gmail.com>
parents: 37615
diff changeset
150 # isn't batchable, we send it immediately because the executor
e1b32dc4646c wireproto: implement command executor interface for version 1 peers
Gregory Szorc <gregory.szorc@gmail.com>
parents: 37615
diff changeset
151 # can no longer accept new commands after a non-batchable command.
37631
2f626233859b wireproto: implement batching on peer executor interface
Gregory Szorc <gregory.szorc@gmail.com>
parents: 37630
diff changeset
152 # If a command is batchable, we queue it for later. But we have
2f626233859b wireproto: implement batching on peer executor interface
Gregory Szorc <gregory.szorc@gmail.com>
parents: 37630
diff changeset
153 # to account for the case of a non-batchable command arriving after
2f626233859b wireproto: implement batching on peer executor interface
Gregory Szorc <gregory.szorc@gmail.com>
parents: 37630
diff changeset
154 # a batchable one and refuse to service it.
2f626233859b wireproto: implement batching on peer executor interface
Gregory Szorc <gregory.szorc@gmail.com>
parents: 37630
diff changeset
155
2f626233859b wireproto: implement batching on peer executor interface
Gregory Szorc <gregory.szorc@gmail.com>
parents: 37630
diff changeset
156 def addcall():
2f626233859b wireproto: implement batching on peer executor interface
Gregory Szorc <gregory.szorc@gmail.com>
parents: 37630
diff changeset
157 f = pycompat.futures.Future()
2f626233859b wireproto: implement batching on peer executor interface
Gregory Szorc <gregory.szorc@gmail.com>
parents: 37630
diff changeset
158 self._futures.add(f)
2f626233859b wireproto: implement batching on peer executor interface
Gregory Szorc <gregory.szorc@gmail.com>
parents: 37630
diff changeset
159 self._calls.append((command, args, fn, f))
2f626233859b wireproto: implement batching on peer executor interface
Gregory Szorc <gregory.szorc@gmail.com>
parents: 37630
diff changeset
160 return f
37630
e1b32dc4646c wireproto: implement command executor interface for version 1 peers
Gregory Szorc <gregory.szorc@gmail.com>
parents: 37615
diff changeset
161
e1b32dc4646c wireproto: implement command executor interface for version 1 peers
Gregory Szorc <gregory.szorc@gmail.com>
parents: 37615
diff changeset
162 if getattr(fn, 'batchable', False):
37631
2f626233859b wireproto: implement batching on peer executor interface
Gregory Szorc <gregory.szorc@gmail.com>
parents: 37630
diff changeset
163 f = addcall()
2f626233859b wireproto: implement batching on peer executor interface
Gregory Szorc <gregory.szorc@gmail.com>
parents: 37630
diff changeset
164
2f626233859b wireproto: implement batching on peer executor interface
Gregory Szorc <gregory.szorc@gmail.com>
parents: 37630
diff changeset
165 # But since we don't issue it immediately, we wrap its result()
2f626233859b wireproto: implement batching on peer executor interface
Gregory Szorc <gregory.szorc@gmail.com>
parents: 37630
diff changeset
166 # to trigger sending so we avoid deadlocks.
2f626233859b wireproto: implement batching on peer executor interface
Gregory Szorc <gregory.szorc@gmail.com>
parents: 37630
diff changeset
167 f.__class__ = unsentfuture
2f626233859b wireproto: implement batching on peer executor interface
Gregory Szorc <gregory.szorc@gmail.com>
parents: 37630
diff changeset
168 f._peerexecutor = self
37630
e1b32dc4646c wireproto: implement command executor interface for version 1 peers
Gregory Szorc <gregory.szorc@gmail.com>
parents: 37615
diff changeset
169 else:
e1b32dc4646c wireproto: implement command executor interface for version 1 peers
Gregory Szorc <gregory.szorc@gmail.com>
parents: 37615
diff changeset
170 if self._calls:
e1b32dc4646c wireproto: implement command executor interface for version 1 peers
Gregory Szorc <gregory.szorc@gmail.com>
parents: 37615
diff changeset
171 raise error.ProgrammingError(
43077
687b865b95ad formatting: byteify all mercurial/ and hgext/ string literals
Augie Fackler <augie@google.com>
parents: 43076
diff changeset
172 b'%s is not batchable and cannot be called on a command '
687b865b95ad formatting: byteify all mercurial/ and hgext/ string literals
Augie Fackler <augie@google.com>
parents: 43076
diff changeset
173 b'executor along with other commands' % command
43076
2372284d9457 formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents: 42814
diff changeset
174 )
37630
e1b32dc4646c wireproto: implement command executor interface for version 1 peers
Gregory Szorc <gregory.szorc@gmail.com>
parents: 37615
diff changeset
175
37631
2f626233859b wireproto: implement batching on peer executor interface
Gregory Szorc <gregory.szorc@gmail.com>
parents: 37630
diff changeset
176 f = addcall()
2f626233859b wireproto: implement batching on peer executor interface
Gregory Szorc <gregory.szorc@gmail.com>
parents: 37630
diff changeset
177
2f626233859b wireproto: implement batching on peer executor interface
Gregory Szorc <gregory.szorc@gmail.com>
parents: 37630
diff changeset
178 # Non-batchable commands can never coexist with another command
2f626233859b wireproto: implement batching on peer executor interface
Gregory Szorc <gregory.szorc@gmail.com>
parents: 37630
diff changeset
179 # in this executor. So send the command immediately.
2f626233859b wireproto: implement batching on peer executor interface
Gregory Szorc <gregory.szorc@gmail.com>
parents: 37630
diff changeset
180 self.sendcommands()
2f626233859b wireproto: implement batching on peer executor interface
Gregory Szorc <gregory.szorc@gmail.com>
parents: 37630
diff changeset
181
37630
e1b32dc4646c wireproto: implement command executor interface for version 1 peers
Gregory Szorc <gregory.szorc@gmail.com>
parents: 37615
diff changeset
182 return f
e1b32dc4646c wireproto: implement command executor interface for version 1 peers
Gregory Szorc <gregory.szorc@gmail.com>
parents: 37615
diff changeset
183
e1b32dc4646c wireproto: implement command executor interface for version 1 peers
Gregory Szorc <gregory.szorc@gmail.com>
parents: 37615
diff changeset
184 def sendcommands(self):
e1b32dc4646c wireproto: implement command executor interface for version 1 peers
Gregory Szorc <gregory.szorc@gmail.com>
parents: 37615
diff changeset
185 if self._sent:
e1b32dc4646c wireproto: implement command executor interface for version 1 peers
Gregory Szorc <gregory.szorc@gmail.com>
parents: 37615
diff changeset
186 return
e1b32dc4646c wireproto: implement command executor interface for version 1 peers
Gregory Szorc <gregory.szorc@gmail.com>
parents: 37615
diff changeset
187
e1b32dc4646c wireproto: implement command executor interface for version 1 peers
Gregory Szorc <gregory.szorc@gmail.com>
parents: 37615
diff changeset
188 if not self._calls:
e1b32dc4646c wireproto: implement command executor interface for version 1 peers
Gregory Szorc <gregory.szorc@gmail.com>
parents: 37615
diff changeset
189 return
e1b32dc4646c wireproto: implement command executor interface for version 1 peers
Gregory Szorc <gregory.szorc@gmail.com>
parents: 37615
diff changeset
190
e1b32dc4646c wireproto: implement command executor interface for version 1 peers
Gregory Szorc <gregory.szorc@gmail.com>
parents: 37615
diff changeset
191 self._sent = True
e1b32dc4646c wireproto: implement command executor interface for version 1 peers
Gregory Szorc <gregory.szorc@gmail.com>
parents: 37615
diff changeset
192
37631
2f626233859b wireproto: implement batching on peer executor interface
Gregory Szorc <gregory.szorc@gmail.com>
parents: 37630
diff changeset
193 # Unhack any future types so caller seens a clean type and to break
2f626233859b wireproto: implement batching on peer executor interface
Gregory Szorc <gregory.szorc@gmail.com>
parents: 37630
diff changeset
194 # cycle between us and futures.
2f626233859b wireproto: implement batching on peer executor interface
Gregory Szorc <gregory.szorc@gmail.com>
parents: 37630
diff changeset
195 for f in self._futures:
2f626233859b wireproto: implement batching on peer executor interface
Gregory Szorc <gregory.szorc@gmail.com>
parents: 37630
diff changeset
196 if isinstance(f, unsentfuture):
2f626233859b wireproto: implement batching on peer executor interface
Gregory Szorc <gregory.szorc@gmail.com>
parents: 37630
diff changeset
197 f.__class__ = pycompat.futures.Future
2f626233859b wireproto: implement batching on peer executor interface
Gregory Szorc <gregory.szorc@gmail.com>
parents: 37630
diff changeset
198 f._peerexecutor = None
2f626233859b wireproto: implement batching on peer executor interface
Gregory Szorc <gregory.szorc@gmail.com>
parents: 37630
diff changeset
199
37630
e1b32dc4646c wireproto: implement command executor interface for version 1 peers
Gregory Szorc <gregory.szorc@gmail.com>
parents: 37615
diff changeset
200 calls = self._calls
e1b32dc4646c wireproto: implement command executor interface for version 1 peers
Gregory Szorc <gregory.szorc@gmail.com>
parents: 37615
diff changeset
201 # Mainly to destroy references to futures.
e1b32dc4646c wireproto: implement command executor interface for version 1 peers
Gregory Szorc <gregory.szorc@gmail.com>
parents: 37615
diff changeset
202 self._calls = None
e1b32dc4646c wireproto: implement command executor interface for version 1 peers
Gregory Szorc <gregory.szorc@gmail.com>
parents: 37615
diff changeset
203
37631
2f626233859b wireproto: implement batching on peer executor interface
Gregory Szorc <gregory.szorc@gmail.com>
parents: 37630
diff changeset
204 # Simple case of a single command. We call it synchronously.
37630
e1b32dc4646c wireproto: implement command executor interface for version 1 peers
Gregory Szorc <gregory.szorc@gmail.com>
parents: 37615
diff changeset
205 if len(calls) == 1:
e1b32dc4646c wireproto: implement command executor interface for version 1 peers
Gregory Szorc <gregory.szorc@gmail.com>
parents: 37615
diff changeset
206 command, args, fn, f = calls[0]
e1b32dc4646c wireproto: implement command executor interface for version 1 peers
Gregory Szorc <gregory.szorc@gmail.com>
parents: 37615
diff changeset
207
e1b32dc4646c wireproto: implement command executor interface for version 1 peers
Gregory Szorc <gregory.szorc@gmail.com>
parents: 37615
diff changeset
208 # Future was cancelled. Ignore it.
e1b32dc4646c wireproto: implement command executor interface for version 1 peers
Gregory Szorc <gregory.szorc@gmail.com>
parents: 37615
diff changeset
209 if not f.set_running_or_notify_cancel():
e1b32dc4646c wireproto: implement command executor interface for version 1 peers
Gregory Szorc <gregory.szorc@gmail.com>
parents: 37615
diff changeset
210 return
e1b32dc4646c wireproto: implement command executor interface for version 1 peers
Gregory Szorc <gregory.szorc@gmail.com>
parents: 37615
diff changeset
211
e1b32dc4646c wireproto: implement command executor interface for version 1 peers
Gregory Szorc <gregory.szorc@gmail.com>
parents: 37615
diff changeset
212 try:
e1b32dc4646c wireproto: implement command executor interface for version 1 peers
Gregory Szorc <gregory.szorc@gmail.com>
parents: 37615
diff changeset
213 result = fn(**pycompat.strkwargs(args))
e1b32dc4646c wireproto: implement command executor interface for version 1 peers
Gregory Szorc <gregory.szorc@gmail.com>
parents: 37615
diff changeset
214 except Exception:
37669
1cb54e6193a6 py3: paper over differences in future exception handling
Augie Fackler <augie@google.com>
parents: 37650
diff changeset
215 pycompat.future_set_exception_info(f, sys.exc_info()[1:])
37630
e1b32dc4646c wireproto: implement command executor interface for version 1 peers
Gregory Szorc <gregory.szorc@gmail.com>
parents: 37615
diff changeset
216 else:
e1b32dc4646c wireproto: implement command executor interface for version 1 peers
Gregory Szorc <gregory.szorc@gmail.com>
parents: 37615
diff changeset
217 f.set_result(result)
e1b32dc4646c wireproto: implement command executor interface for version 1 peers
Gregory Szorc <gregory.szorc@gmail.com>
parents: 37615
diff changeset
218
e1b32dc4646c wireproto: implement command executor interface for version 1 peers
Gregory Szorc <gregory.szorc@gmail.com>
parents: 37615
diff changeset
219 return
e1b32dc4646c wireproto: implement command executor interface for version 1 peers
Gregory Szorc <gregory.szorc@gmail.com>
parents: 37615
diff changeset
220
37631
2f626233859b wireproto: implement batching on peer executor interface
Gregory Szorc <gregory.szorc@gmail.com>
parents: 37630
diff changeset
221 # Batch commands are a bit harder. First, we have to deal with the
2f626233859b wireproto: implement batching on peer executor interface
Gregory Szorc <gregory.szorc@gmail.com>
parents: 37630
diff changeset
222 # @batchable coroutine. That's a bit annoying. Furthermore, we also
2f626233859b wireproto: implement batching on peer executor interface
Gregory Szorc <gregory.szorc@gmail.com>
parents: 37630
diff changeset
223 # need to preserve streaming. i.e. it should be possible for the
2f626233859b wireproto: implement batching on peer executor interface
Gregory Szorc <gregory.szorc@gmail.com>
parents: 37630
diff changeset
224 # futures to resolve as data is coming in off the wire without having
2f626233859b wireproto: implement batching on peer executor interface
Gregory Szorc <gregory.szorc@gmail.com>
parents: 37630
diff changeset
225 # to wait for the final byte of the final response. We do this by
2f626233859b wireproto: implement batching on peer executor interface
Gregory Szorc <gregory.szorc@gmail.com>
parents: 37630
diff changeset
226 # spinning up a thread to read the responses.
2f626233859b wireproto: implement batching on peer executor interface
Gregory Szorc <gregory.szorc@gmail.com>
parents: 37630
diff changeset
227
2f626233859b wireproto: implement batching on peer executor interface
Gregory Szorc <gregory.szorc@gmail.com>
parents: 37630
diff changeset
228 requests = []
2f626233859b wireproto: implement batching on peer executor interface
Gregory Szorc <gregory.szorc@gmail.com>
parents: 37630
diff changeset
229 states = []
2f626233859b wireproto: implement batching on peer executor interface
Gregory Szorc <gregory.szorc@gmail.com>
parents: 37630
diff changeset
230
2f626233859b wireproto: implement batching on peer executor interface
Gregory Szorc <gregory.szorc@gmail.com>
parents: 37630
diff changeset
231 for command, args, fn, f in calls:
2f626233859b wireproto: implement batching on peer executor interface
Gregory Szorc <gregory.szorc@gmail.com>
parents: 37630
diff changeset
232 # Future was cancelled. Ignore it.
2f626233859b wireproto: implement batching on peer executor interface
Gregory Szorc <gregory.szorc@gmail.com>
parents: 37630
diff changeset
233 if not f.set_running_or_notify_cancel():
2f626233859b wireproto: implement batching on peer executor interface
Gregory Szorc <gregory.szorc@gmail.com>
parents: 37630
diff changeset
234 continue
2f626233859b wireproto: implement batching on peer executor interface
Gregory Szorc <gregory.szorc@gmail.com>
parents: 37630
diff changeset
235
2f626233859b wireproto: implement batching on peer executor interface
Gregory Szorc <gregory.szorc@gmail.com>
parents: 37630
diff changeset
236 try:
47872
cdad6560e832 wireprotov1peer: simplify the way batchable rpcs are defined
Valentin Gatien-Baron <valentin.gatienbaron@gmail.com>
parents: 46819
diff changeset
237 encoded_args_or_res, decode = fn.batchable(
43076
2372284d9457 formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents: 42814
diff changeset
238 fn.__self__, **pycompat.strkwargs(args)
2372284d9457 formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents: 42814
diff changeset
239 )
37631
2f626233859b wireproto: implement batching on peer executor interface
Gregory Szorc <gregory.szorc@gmail.com>
parents: 37630
diff changeset
240 except Exception:
37669
1cb54e6193a6 py3: paper over differences in future exception handling
Augie Fackler <augie@google.com>
parents: 37650
diff changeset
241 pycompat.future_set_exception_info(f, sys.exc_info()[1:])
37631
2f626233859b wireproto: implement batching on peer executor interface
Gregory Szorc <gregory.szorc@gmail.com>
parents: 37630
diff changeset
242 return
2f626233859b wireproto: implement batching on peer executor interface
Gregory Szorc <gregory.szorc@gmail.com>
parents: 37630
diff changeset
243
47872
cdad6560e832 wireprotov1peer: simplify the way batchable rpcs are defined
Valentin Gatien-Baron <valentin.gatienbaron@gmail.com>
parents: 46819
diff changeset
244 if not decode:
46480
05dd091dfa6a wireprotopeer: clarify some variable names now that we allow snake_case
Martin von Zweigbergk <martinvonz@google.com>
parents: 45942
diff changeset
245 f.set_result(encoded_args_or_res)
41055
55e8da487b8a wireproto: in batch queries, support queries with immediate responses
Valentin Gatien-Baron <valentin.gatienbaron@gmail.com>
parents: 39588
diff changeset
246 else:
46480
05dd091dfa6a wireprotopeer: clarify some variable names now that we allow snake_case
Martin von Zweigbergk <martinvonz@google.com>
parents: 45942
diff changeset
247 requests.append((command, encoded_args_or_res))
47872
cdad6560e832 wireprotov1peer: simplify the way batchable rpcs are defined
Valentin Gatien-Baron <valentin.gatienbaron@gmail.com>
parents: 46819
diff changeset
248 states.append((command, f, batchable, decode))
37631
2f626233859b wireproto: implement batching on peer executor interface
Gregory Szorc <gregory.szorc@gmail.com>
parents: 37630
diff changeset
249
2f626233859b wireproto: implement batching on peer executor interface
Gregory Szorc <gregory.szorc@gmail.com>
parents: 37630
diff changeset
250 if not requests:
2f626233859b wireproto: implement batching on peer executor interface
Gregory Szorc <gregory.szorc@gmail.com>
parents: 37630
diff changeset
251 return
2f626233859b wireproto: implement batching on peer executor interface
Gregory Szorc <gregory.szorc@gmail.com>
parents: 37630
diff changeset
252
2f626233859b wireproto: implement batching on peer executor interface
Gregory Szorc <gregory.szorc@gmail.com>
parents: 37630
diff changeset
253 # This will emit responses in order they were executed.
2f626233859b wireproto: implement batching on peer executor interface
Gregory Szorc <gregory.szorc@gmail.com>
parents: 37630
diff changeset
254 wireresults = self._peer._submitbatch(requests)
2f626233859b wireproto: implement batching on peer executor interface
Gregory Szorc <gregory.szorc@gmail.com>
parents: 37630
diff changeset
255
2f626233859b wireproto: implement batching on peer executor interface
Gregory Szorc <gregory.szorc@gmail.com>
parents: 37630
diff changeset
256 # The use of a thread pool executor here is a bit weird for something
2f626233859b wireproto: implement batching on peer executor interface
Gregory Szorc <gregory.szorc@gmail.com>
parents: 37630
diff changeset
257 # that only spins up a single thread. However, thread management is
2f626233859b wireproto: implement batching on peer executor interface
Gregory Szorc <gregory.szorc@gmail.com>
parents: 37630
diff changeset
258 # hard and it is easy to encounter race conditions, deadlocks, etc.
2f626233859b wireproto: implement batching on peer executor interface
Gregory Szorc <gregory.szorc@gmail.com>
parents: 37630
diff changeset
259 # concurrent.futures already solves these problems and its thread pool
2f626233859b wireproto: implement batching on peer executor interface
Gregory Szorc <gregory.szorc@gmail.com>
parents: 37630
diff changeset
260 # executor has minimal overhead. So we use it.
2f626233859b wireproto: implement batching on peer executor interface
Gregory Szorc <gregory.szorc@gmail.com>
parents: 37630
diff changeset
261 self._responseexecutor = pycompat.futures.ThreadPoolExecutor(1)
43076
2372284d9457 formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents: 42814
diff changeset
262 self._responsef = self._responseexecutor.submit(
2372284d9457 formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents: 42814
diff changeset
263 self._readbatchresponse, states, wireresults
2372284d9457 formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents: 42814
diff changeset
264 )
37630
e1b32dc4646c wireproto: implement command executor interface for version 1 peers
Gregory Szorc <gregory.szorc@gmail.com>
parents: 37615
diff changeset
265
e1b32dc4646c wireproto: implement command executor interface for version 1 peers
Gregory Szorc <gregory.szorc@gmail.com>
parents: 37615
diff changeset
266 def close(self):
e1b32dc4646c wireproto: implement command executor interface for version 1 peers
Gregory Szorc <gregory.szorc@gmail.com>
parents: 37615
diff changeset
267 self.sendcommands()
e1b32dc4646c wireproto: implement command executor interface for version 1 peers
Gregory Szorc <gregory.szorc@gmail.com>
parents: 37615
diff changeset
268
37631
2f626233859b wireproto: implement batching on peer executor interface
Gregory Szorc <gregory.szorc@gmail.com>
parents: 37630
diff changeset
269 if self._closed:
2f626233859b wireproto: implement batching on peer executor interface
Gregory Szorc <gregory.szorc@gmail.com>
parents: 37630
diff changeset
270 return
2f626233859b wireproto: implement batching on peer executor interface
Gregory Szorc <gregory.szorc@gmail.com>
parents: 37630
diff changeset
271
37630
e1b32dc4646c wireproto: implement command executor interface for version 1 peers
Gregory Szorc <gregory.szorc@gmail.com>
parents: 37615
diff changeset
272 self._closed = True
e1b32dc4646c wireproto: implement command executor interface for version 1 peers
Gregory Szorc <gregory.szorc@gmail.com>
parents: 37615
diff changeset
273
37631
2f626233859b wireproto: implement batching on peer executor interface
Gregory Szorc <gregory.szorc@gmail.com>
parents: 37630
diff changeset
274 if not self._responsef:
2f626233859b wireproto: implement batching on peer executor interface
Gregory Szorc <gregory.szorc@gmail.com>
parents: 37630
diff changeset
275 return
2f626233859b wireproto: implement batching on peer executor interface
Gregory Szorc <gregory.szorc@gmail.com>
parents: 37630
diff changeset
276
2f626233859b wireproto: implement batching on peer executor interface
Gregory Szorc <gregory.szorc@gmail.com>
parents: 37630
diff changeset
277 # We need to wait on our in-flight response and then shut down the
2f626233859b wireproto: implement batching on peer executor interface
Gregory Szorc <gregory.szorc@gmail.com>
parents: 37630
diff changeset
278 # executor once we have a result.
2f626233859b wireproto: implement batching on peer executor interface
Gregory Szorc <gregory.szorc@gmail.com>
parents: 37630
diff changeset
279 try:
2f626233859b wireproto: implement batching on peer executor interface
Gregory Szorc <gregory.szorc@gmail.com>
parents: 37630
diff changeset
280 self._responsef.result()
2f626233859b wireproto: implement batching on peer executor interface
Gregory Szorc <gregory.szorc@gmail.com>
parents: 37630
diff changeset
281 finally:
2f626233859b wireproto: implement batching on peer executor interface
Gregory Szorc <gregory.szorc@gmail.com>
parents: 37630
diff changeset
282 self._responseexecutor.shutdown(wait=True)
2f626233859b wireproto: implement batching on peer executor interface
Gregory Szorc <gregory.szorc@gmail.com>
parents: 37630
diff changeset
283 self._responsef = None
2f626233859b wireproto: implement batching on peer executor interface
Gregory Szorc <gregory.szorc@gmail.com>
parents: 37630
diff changeset
284 self._responseexecutor = None
2f626233859b wireproto: implement batching on peer executor interface
Gregory Szorc <gregory.szorc@gmail.com>
parents: 37630
diff changeset
285
2f626233859b wireproto: implement batching on peer executor interface
Gregory Szorc <gregory.szorc@gmail.com>
parents: 37630
diff changeset
286 # If any of our futures are still in progress, mark them as
2f626233859b wireproto: implement batching on peer executor interface
Gregory Szorc <gregory.szorc@gmail.com>
parents: 37630
diff changeset
287 # errored. Otherwise a result() could wait indefinitely.
2f626233859b wireproto: implement batching on peer executor interface
Gregory Szorc <gregory.szorc@gmail.com>
parents: 37630
diff changeset
288 for f in self._futures:
2f626233859b wireproto: implement batching on peer executor interface
Gregory Szorc <gregory.szorc@gmail.com>
parents: 37630
diff changeset
289 if not f.done():
43076
2372284d9457 formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents: 42814
diff changeset
290 f.set_exception(
2372284d9457 formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents: 42814
diff changeset
291 error.ResponseError(
46672
aa2e38147e8b wireprotov1peer: don't raise internal errors in some cases
Valentin Gatien-Baron <vgatien-baron@janestreet.com>
parents: 46480
diff changeset
292 _(b'unfulfilled batch command response'), None
43076
2372284d9457 formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents: 42814
diff changeset
293 )
2372284d9457 formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents: 42814
diff changeset
294 )
37631
2f626233859b wireproto: implement batching on peer executor interface
Gregory Szorc <gregory.szorc@gmail.com>
parents: 37630
diff changeset
295
2f626233859b wireproto: implement batching on peer executor interface
Gregory Szorc <gregory.szorc@gmail.com>
parents: 37630
diff changeset
296 self._futures = None
2f626233859b wireproto: implement batching on peer executor interface
Gregory Szorc <gregory.szorc@gmail.com>
parents: 37630
diff changeset
297
2f626233859b wireproto: implement batching on peer executor interface
Gregory Szorc <gregory.szorc@gmail.com>
parents: 37630
diff changeset
298 def _readbatchresponse(self, states, wireresults):
2f626233859b wireproto: implement batching on peer executor interface
Gregory Szorc <gregory.szorc@gmail.com>
parents: 37630
diff changeset
299 # Executes in a thread to read data off the wire.
2f626233859b wireproto: implement batching on peer executor interface
Gregory Szorc <gregory.szorc@gmail.com>
parents: 37630
diff changeset
300
47872
cdad6560e832 wireprotov1peer: simplify the way batchable rpcs are defined
Valentin Gatien-Baron <valentin.gatienbaron@gmail.com>
parents: 46819
diff changeset
301 for command, f, batchable, decode in states:
37631
2f626233859b wireproto: implement batching on peer executor interface
Gregory Szorc <gregory.szorc@gmail.com>
parents: 37630
diff changeset
302 # Grab raw result off the wire and teach the internal future
2f626233859b wireproto: implement batching on peer executor interface
Gregory Szorc <gregory.szorc@gmail.com>
parents: 37630
diff changeset
303 # about it.
46672
aa2e38147e8b wireprotov1peer: don't raise internal errors in some cases
Valentin Gatien-Baron <vgatien-baron@janestreet.com>
parents: 46480
diff changeset
304 try:
aa2e38147e8b wireprotov1peer: don't raise internal errors in some cases
Valentin Gatien-Baron <vgatien-baron@janestreet.com>
parents: 46480
diff changeset
305 remoteresult = next(wireresults)
aa2e38147e8b wireprotov1peer: don't raise internal errors in some cases
Valentin Gatien-Baron <vgatien-baron@janestreet.com>
parents: 46480
diff changeset
306 except StopIteration:
aa2e38147e8b wireprotov1peer: don't raise internal errors in some cases
Valentin Gatien-Baron <vgatien-baron@janestreet.com>
parents: 46480
diff changeset
307 # This can happen in particular because next(batchable)
aa2e38147e8b wireprotov1peer: don't raise internal errors in some cases
Valentin Gatien-Baron <vgatien-baron@janestreet.com>
parents: 46480
diff changeset
308 # in the previous iteration can call peer._abort, which
aa2e38147e8b wireprotov1peer: don't raise internal errors in some cases
Valentin Gatien-Baron <vgatien-baron@janestreet.com>
parents: 46480
diff changeset
309 # may close the peer.
aa2e38147e8b wireprotov1peer: don't raise internal errors in some cases
Valentin Gatien-Baron <vgatien-baron@janestreet.com>
parents: 46480
diff changeset
310 f.set_exception(
aa2e38147e8b wireprotov1peer: don't raise internal errors in some cases
Valentin Gatien-Baron <vgatien-baron@janestreet.com>
parents: 46480
diff changeset
311 error.ResponseError(
aa2e38147e8b wireprotov1peer: don't raise internal errors in some cases
Valentin Gatien-Baron <vgatien-baron@janestreet.com>
parents: 46480
diff changeset
312 _(b'unfulfilled batch command response'), None
aa2e38147e8b wireprotov1peer: don't raise internal errors in some cases
Valentin Gatien-Baron <vgatien-baron@janestreet.com>
parents: 46480
diff changeset
313 )
aa2e38147e8b wireprotov1peer: don't raise internal errors in some cases
Valentin Gatien-Baron <vgatien-baron@janestreet.com>
parents: 46480
diff changeset
314 )
aa2e38147e8b wireprotov1peer: don't raise internal errors in some cases
Valentin Gatien-Baron <vgatien-baron@janestreet.com>
parents: 46480
diff changeset
315 else:
aa2e38147e8b wireprotov1peer: don't raise internal errors in some cases
Valentin Gatien-Baron <vgatien-baron@janestreet.com>
parents: 46480
diff changeset
316 try:
47872
cdad6560e832 wireprotov1peer: simplify the way batchable rpcs are defined
Valentin Gatien-Baron <valentin.gatienbaron@gmail.com>
parents: 46819
diff changeset
317 result = decode(remoteresult)
46672
aa2e38147e8b wireprotov1peer: don't raise internal errors in some cases
Valentin Gatien-Baron <vgatien-baron@janestreet.com>
parents: 46480
diff changeset
318 except Exception:
aa2e38147e8b wireprotov1peer: don't raise internal errors in some cases
Valentin Gatien-Baron <vgatien-baron@janestreet.com>
parents: 46480
diff changeset
319 pycompat.future_set_exception_info(f, sys.exc_info()[1:])
aa2e38147e8b wireprotov1peer: don't raise internal errors in some cases
Valentin Gatien-Baron <vgatien-baron@janestreet.com>
parents: 46480
diff changeset
320 else:
aa2e38147e8b wireprotov1peer: don't raise internal errors in some cases
Valentin Gatien-Baron <vgatien-baron@janestreet.com>
parents: 46480
diff changeset
321 f.set_result(result)
37631
2f626233859b wireproto: implement batching on peer executor interface
Gregory Szorc <gregory.szorc@gmail.com>
parents: 37630
diff changeset
322
43076
2372284d9457 formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents: 42814
diff changeset
323
2372284d9457 formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents: 42814
diff changeset
324 @interfaceutil.implementer(
2372284d9457 formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents: 42814
diff changeset
325 repository.ipeercommands, repository.ipeerlegacycommands
2372284d9457 formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents: 42814
diff changeset
326 )
37635
cc8c06835097 wireproto: convert legacy commands to command executor
Gregory Szorc <gregory.szorc@gmail.com>
parents: 37633
diff changeset
327 class wirepeer(repository.peer):
27243
3abee2ba27af wireproto: add docstring for wirepeer
Gregory Szorc <gregory.szorc@gmail.com>
parents: 26911
diff changeset
328 """Client-side interface for communicating with a peer repository.
14622
bd88561afb4b wireproto: add batching support to wirerepository
Peter Arrenbrecht <peter.arrenbrecht@gmail.com>
parents: 14621
diff changeset
329
27243
3abee2ba27af wireproto: add docstring for wirepeer
Gregory Szorc <gregory.szorc@gmail.com>
parents: 26911
diff changeset
330 Methods commonly call wire protocol commands of the same name.
3abee2ba27af wireproto: add docstring for wirepeer
Gregory Szorc <gregory.szorc@gmail.com>
parents: 26911
diff changeset
331
3abee2ba27af wireproto: add docstring for wirepeer
Gregory Szorc <gregory.szorc@gmail.com>
parents: 26911
diff changeset
332 See also httppeer.py and sshpeer.py for protocol-specific
3abee2ba27af wireproto: add docstring for wirepeer
Gregory Szorc <gregory.szorc@gmail.com>
parents: 26911
diff changeset
333 implementations of this interface.
3abee2ba27af wireproto: add docstring for wirepeer
Gregory Szorc <gregory.szorc@gmail.com>
parents: 26911
diff changeset
334 """
43076
2372284d9457 formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents: 42814
diff changeset
335
37630
e1b32dc4646c wireproto: implement command executor interface for version 1 peers
Gregory Szorc <gregory.szorc@gmail.com>
parents: 37615
diff changeset
336 def commandexecutor(self):
e1b32dc4646c wireproto: implement command executor interface for version 1 peers
Gregory Szorc <gregory.szorc@gmail.com>
parents: 37615
diff changeset
337 return peerexecutor(self)
e1b32dc4646c wireproto: implement command executor interface for version 1 peers
Gregory Szorc <gregory.szorc@gmail.com>
parents: 37615
diff changeset
338
37320
39f7d4ee8bcd repository: port peer interfaces to zope.interface
Gregory Szorc <gregory.szorc@gmail.com>
parents: 37295
diff changeset
339 # Begin of ipeercommands interface.
14622
bd88561afb4b wireproto: add batching support to wirerepository
Peter Arrenbrecht <peter.arrenbrecht@gmail.com>
parents: 14621
diff changeset
340
37649
a168799687e5 wireproto: properly call clonebundles command
Gregory Szorc <gregory.szorc@gmail.com>
parents: 37646
diff changeset
341 def clonebundles(self):
43077
687b865b95ad formatting: byteify all mercurial/ and hgext/ string literals
Augie Fackler <augie@google.com>
parents: 43076
diff changeset
342 self.requirecap(b'clonebundles', _(b'clone bundles'))
687b865b95ad formatting: byteify all mercurial/ and hgext/ string literals
Augie Fackler <augie@google.com>
parents: 43076
diff changeset
343 return self._call(b'clonebundles')
37649
a168799687e5 wireproto: properly call clonebundles command
Gregory Szorc <gregory.szorc@gmail.com>
parents: 37646
diff changeset
344
14623
e7c9fdbbb902 wireproto: make a number of commands batchable
Peter Arrenbrecht <peter.arrenbrecht@gmail.com>
parents: 14622
diff changeset
345 @batchable
11586
ddaaaa23bb8f protocol: move basic ssh client commands to wirerepository
Matt Mackall <mpm@selenic.com>
parents: 11585
diff changeset
346 def lookup(self, key):
43077
687b865b95ad formatting: byteify all mercurial/ and hgext/ string literals
Augie Fackler <augie@google.com>
parents: 43076
diff changeset
347 self.requirecap(b'lookup', _(b'look up remote revision'))
47873
c424ff4807e6 wireprotov1peer: update all rpcs to use the new batchable scheme
Valentin Gatien-Baron <valentin.gatienbaron@gmail.com>
parents: 47872
diff changeset
348
c424ff4807e6 wireprotov1peer: update all rpcs to use the new batchable scheme
Valentin Gatien-Baron <valentin.gatienbaron@gmail.com>
parents: 47872
diff changeset
349 def decode(d):
c424ff4807e6 wireprotov1peer: update all rpcs to use the new batchable scheme
Valentin Gatien-Baron <valentin.gatienbaron@gmail.com>
parents: 47872
diff changeset
350 success, data = d[:-1].split(b" ", 1)
c424ff4807e6 wireprotov1peer: update all rpcs to use the new batchable scheme
Valentin Gatien-Baron <valentin.gatienbaron@gmail.com>
parents: 47872
diff changeset
351 if int(success):
c424ff4807e6 wireprotov1peer: update all rpcs to use the new batchable scheme
Valentin Gatien-Baron <valentin.gatienbaron@gmail.com>
parents: 47872
diff changeset
352 return bin(data)
c424ff4807e6 wireprotov1peer: update all rpcs to use the new batchable scheme
Valentin Gatien-Baron <valentin.gatienbaron@gmail.com>
parents: 47872
diff changeset
353 else:
c424ff4807e6 wireprotov1peer: update all rpcs to use the new batchable scheme
Valentin Gatien-Baron <valentin.gatienbaron@gmail.com>
parents: 47872
diff changeset
354 self._abort(error.RepoError(data))
c424ff4807e6 wireprotov1peer: update all rpcs to use the new batchable scheme
Valentin Gatien-Baron <valentin.gatienbaron@gmail.com>
parents: 47872
diff changeset
355
c424ff4807e6 wireprotov1peer: update all rpcs to use the new batchable scheme
Valentin Gatien-Baron <valentin.gatienbaron@gmail.com>
parents: 47872
diff changeset
356 return {b'key': encoding.fromlocal(key)}, decode
11586
ddaaaa23bb8f protocol: move basic ssh client commands to wirerepository
Matt Mackall <mpm@selenic.com>
parents: 11585
diff changeset
357
14623
e7c9fdbbb902 wireproto: make a number of commands batchable
Peter Arrenbrecht <peter.arrenbrecht@gmail.com>
parents: 14622
diff changeset
358 @batchable
11586
ddaaaa23bb8f protocol: move basic ssh client commands to wirerepository
Matt Mackall <mpm@selenic.com>
parents: 11585
diff changeset
359 def heads(self):
47873
c424ff4807e6 wireprotov1peer: update all rpcs to use the new batchable scheme
Valentin Gatien-Baron <valentin.gatienbaron@gmail.com>
parents: 47872
diff changeset
360 def decode(d):
c424ff4807e6 wireprotov1peer: update all rpcs to use the new batchable scheme
Valentin Gatien-Baron <valentin.gatienbaron@gmail.com>
parents: 47872
diff changeset
361 try:
c424ff4807e6 wireprotov1peer: update all rpcs to use the new batchable scheme
Valentin Gatien-Baron <valentin.gatienbaron@gmail.com>
parents: 47872
diff changeset
362 return wireprototypes.decodelist(d[:-1])
c424ff4807e6 wireprotov1peer: update all rpcs to use the new batchable scheme
Valentin Gatien-Baron <valentin.gatienbaron@gmail.com>
parents: 47872
diff changeset
363 except ValueError:
c424ff4807e6 wireprotov1peer: update all rpcs to use the new batchable scheme
Valentin Gatien-Baron <valentin.gatienbaron@gmail.com>
parents: 47872
diff changeset
364 self._abort(error.ResponseError(_(b"unexpected response:"), d))
c424ff4807e6 wireprotov1peer: update all rpcs to use the new batchable scheme
Valentin Gatien-Baron <valentin.gatienbaron@gmail.com>
parents: 47872
diff changeset
365
c424ff4807e6 wireprotov1peer: update all rpcs to use the new batchable scheme
Valentin Gatien-Baron <valentin.gatienbaron@gmail.com>
parents: 47872
diff changeset
366 return {}, decode
11586
ddaaaa23bb8f protocol: move basic ssh client commands to wirerepository
Matt Mackall <mpm@selenic.com>
parents: 11585
diff changeset
367
14623
e7c9fdbbb902 wireproto: make a number of commands batchable
Peter Arrenbrecht <peter.arrenbrecht@gmail.com>
parents: 14622
diff changeset
368 @batchable
13723
e615765fdcc7 wireproto: add known([id]) function
Peter Arrenbrecht <peter.arrenbrecht@gmail.com>
parents: 13722
diff changeset
369 def known(self, nodes):
47873
c424ff4807e6 wireprotov1peer: update all rpcs to use the new batchable scheme
Valentin Gatien-Baron <valentin.gatienbaron@gmail.com>
parents: 47872
diff changeset
370 def decode(d):
c424ff4807e6 wireprotov1peer: update all rpcs to use the new batchable scheme
Valentin Gatien-Baron <valentin.gatienbaron@gmail.com>
parents: 47872
diff changeset
371 try:
c424ff4807e6 wireprotov1peer: update all rpcs to use the new batchable scheme
Valentin Gatien-Baron <valentin.gatienbaron@gmail.com>
parents: 47872
diff changeset
372 return [bool(int(b)) for b in pycompat.iterbytestr(d)]
c424ff4807e6 wireprotov1peer: update all rpcs to use the new batchable scheme
Valentin Gatien-Baron <valentin.gatienbaron@gmail.com>
parents: 47872
diff changeset
373 except ValueError:
c424ff4807e6 wireprotov1peer: update all rpcs to use the new batchable scheme
Valentin Gatien-Baron <valentin.gatienbaron@gmail.com>
parents: 47872
diff changeset
374 self._abort(error.ResponseError(_(b"unexpected response:"), d))
c424ff4807e6 wireprotov1peer: update all rpcs to use the new batchable scheme
Valentin Gatien-Baron <valentin.gatienbaron@gmail.com>
parents: 47872
diff changeset
375
c424ff4807e6 wireprotov1peer: update all rpcs to use the new batchable scheme
Valentin Gatien-Baron <valentin.gatienbaron@gmail.com>
parents: 47872
diff changeset
376 return {b'nodes': wireprototypes.encodelist(nodes)}, decode
13723
e615765fdcc7 wireproto: add known([id]) function
Peter Arrenbrecht <peter.arrenbrecht@gmail.com>
parents: 13722
diff changeset
377
14623
e7c9fdbbb902 wireproto: make a number of commands batchable
Peter Arrenbrecht <peter.arrenbrecht@gmail.com>
parents: 14622
diff changeset
378 @batchable
11586
ddaaaa23bb8f protocol: move basic ssh client commands to wirerepository
Matt Mackall <mpm@selenic.com>
parents: 11585
diff changeset
379 def branchmap(self):
47873
c424ff4807e6 wireprotov1peer: update all rpcs to use the new batchable scheme
Valentin Gatien-Baron <valentin.gatienbaron@gmail.com>
parents: 47872
diff changeset
380 def decode(d):
c424ff4807e6 wireprotov1peer: update all rpcs to use the new batchable scheme
Valentin Gatien-Baron <valentin.gatienbaron@gmail.com>
parents: 47872
diff changeset
381 try:
c424ff4807e6 wireprotov1peer: update all rpcs to use the new batchable scheme
Valentin Gatien-Baron <valentin.gatienbaron@gmail.com>
parents: 47872
diff changeset
382 branchmap = {}
c424ff4807e6 wireprotov1peer: update all rpcs to use the new batchable scheme
Valentin Gatien-Baron <valentin.gatienbaron@gmail.com>
parents: 47872
diff changeset
383 for branchpart in d.splitlines():
c424ff4807e6 wireprotov1peer: update all rpcs to use the new batchable scheme
Valentin Gatien-Baron <valentin.gatienbaron@gmail.com>
parents: 47872
diff changeset
384 branchname, branchheads = branchpart.split(b' ', 1)
c424ff4807e6 wireprotov1peer: update all rpcs to use the new batchable scheme
Valentin Gatien-Baron <valentin.gatienbaron@gmail.com>
parents: 47872
diff changeset
385 branchname = encoding.tolocal(urlreq.unquote(branchname))
c424ff4807e6 wireprotov1peer: update all rpcs to use the new batchable scheme
Valentin Gatien-Baron <valentin.gatienbaron@gmail.com>
parents: 47872
diff changeset
386 branchheads = wireprototypes.decodelist(branchheads)
c424ff4807e6 wireprotov1peer: update all rpcs to use the new batchable scheme
Valentin Gatien-Baron <valentin.gatienbaron@gmail.com>
parents: 47872
diff changeset
387 branchmap[branchname] = branchheads
c424ff4807e6 wireprotov1peer: update all rpcs to use the new batchable scheme
Valentin Gatien-Baron <valentin.gatienbaron@gmail.com>
parents: 47872
diff changeset
388 return branchmap
c424ff4807e6 wireprotov1peer: update all rpcs to use the new batchable scheme
Valentin Gatien-Baron <valentin.gatienbaron@gmail.com>
parents: 47872
diff changeset
389 except TypeError:
c424ff4807e6 wireprotov1peer: update all rpcs to use the new batchable scheme
Valentin Gatien-Baron <valentin.gatienbaron@gmail.com>
parents: 47872
diff changeset
390 self._abort(error.ResponseError(_(b"unexpected response:"), d))
c424ff4807e6 wireprotov1peer: update all rpcs to use the new batchable scheme
Valentin Gatien-Baron <valentin.gatienbaron@gmail.com>
parents: 47872
diff changeset
391
c424ff4807e6 wireprotov1peer: update all rpcs to use the new batchable scheme
Valentin Gatien-Baron <valentin.gatienbaron@gmail.com>
parents: 47872
diff changeset
392 return {}, decode
11586
ddaaaa23bb8f protocol: move basic ssh client commands to wirerepository
Matt Mackall <mpm@selenic.com>
parents: 11585
diff changeset
393
33806
dedab036215d wireproto: use new peer interface
Gregory Szorc <gregory.szorc@gmail.com>
parents: 33767
diff changeset
394 @batchable
dedab036215d wireproto: use new peer interface
Gregory Szorc <gregory.szorc@gmail.com>
parents: 33767
diff changeset
395 def listkeys(self, namespace):
43077
687b865b95ad formatting: byteify all mercurial/ and hgext/ string literals
Augie Fackler <augie@google.com>
parents: 43076
diff changeset
396 if not self.capable(b'pushkey'):
47873
c424ff4807e6 wireprotov1peer: update all rpcs to use the new batchable scheme
Valentin Gatien-Baron <valentin.gatienbaron@gmail.com>
parents: 47872
diff changeset
397 return {}, None
43077
687b865b95ad formatting: byteify all mercurial/ and hgext/ string literals
Augie Fackler <augie@google.com>
parents: 43076
diff changeset
398 self.ui.debug(b'preparing listkeys for "%s"\n' % namespace)
47873
c424ff4807e6 wireprotov1peer: update all rpcs to use the new batchable scheme
Valentin Gatien-Baron <valentin.gatienbaron@gmail.com>
parents: 47872
diff changeset
399
c424ff4807e6 wireprotov1peer: update all rpcs to use the new batchable scheme
Valentin Gatien-Baron <valentin.gatienbaron@gmail.com>
parents: 47872
diff changeset
400 def decode(d):
c424ff4807e6 wireprotov1peer: update all rpcs to use the new batchable scheme
Valentin Gatien-Baron <valentin.gatienbaron@gmail.com>
parents: 47872
diff changeset
401 self.ui.debug(
c424ff4807e6 wireprotov1peer: update all rpcs to use the new batchable scheme
Valentin Gatien-Baron <valentin.gatienbaron@gmail.com>
parents: 47872
diff changeset
402 b'received listkey for "%s": %i bytes\n' % (namespace, len(d))
c424ff4807e6 wireprotov1peer: update all rpcs to use the new batchable scheme
Valentin Gatien-Baron <valentin.gatienbaron@gmail.com>
parents: 47872
diff changeset
403 )
c424ff4807e6 wireprotov1peer: update all rpcs to use the new batchable scheme
Valentin Gatien-Baron <valentin.gatienbaron@gmail.com>
parents: 47872
diff changeset
404 return pushkeymod.decodekeys(d)
c424ff4807e6 wireprotov1peer: update all rpcs to use the new batchable scheme
Valentin Gatien-Baron <valentin.gatienbaron@gmail.com>
parents: 47872
diff changeset
405
c424ff4807e6 wireprotov1peer: update all rpcs to use the new batchable scheme
Valentin Gatien-Baron <valentin.gatienbaron@gmail.com>
parents: 47872
diff changeset
406 return {b'namespace': encoding.fromlocal(namespace)}, decode
11586
ddaaaa23bb8f protocol: move basic ssh client commands to wirerepository
Matt Mackall <mpm@selenic.com>
parents: 11585
diff changeset
407
14623
e7c9fdbbb902 wireproto: make a number of commands batchable
Peter Arrenbrecht <peter.arrenbrecht@gmail.com>
parents: 14622
diff changeset
408 @batchable
11586
ddaaaa23bb8f protocol: move basic ssh client commands to wirerepository
Matt Mackall <mpm@selenic.com>
parents: 11585
diff changeset
409 def pushkey(self, namespace, key, old, new):
43077
687b865b95ad formatting: byteify all mercurial/ and hgext/ string literals
Augie Fackler <augie@google.com>
parents: 43076
diff changeset
410 if not self.capable(b'pushkey'):
47873
c424ff4807e6 wireprotov1peer: update all rpcs to use the new batchable scheme
Valentin Gatien-Baron <valentin.gatienbaron@gmail.com>
parents: 47872
diff changeset
411 return False, None
43077
687b865b95ad formatting: byteify all mercurial/ and hgext/ string literals
Augie Fackler <augie@google.com>
parents: 43076
diff changeset
412 self.ui.debug(b'preparing pushkey for "%s:%s"\n' % (namespace, key))
47873
c424ff4807e6 wireprotov1peer: update all rpcs to use the new batchable scheme
Valentin Gatien-Baron <valentin.gatienbaron@gmail.com>
parents: 47872
diff changeset
413
c424ff4807e6 wireprotov1peer: update all rpcs to use the new batchable scheme
Valentin Gatien-Baron <valentin.gatienbaron@gmail.com>
parents: 47872
diff changeset
414 def decode(d):
c424ff4807e6 wireprotov1peer: update all rpcs to use the new batchable scheme
Valentin Gatien-Baron <valentin.gatienbaron@gmail.com>
parents: 47872
diff changeset
415 d, output = d.split(b'\n', 1)
c424ff4807e6 wireprotov1peer: update all rpcs to use the new batchable scheme
Valentin Gatien-Baron <valentin.gatienbaron@gmail.com>
parents: 47872
diff changeset
416 try:
c424ff4807e6 wireprotov1peer: update all rpcs to use the new batchable scheme
Valentin Gatien-Baron <valentin.gatienbaron@gmail.com>
parents: 47872
diff changeset
417 d = bool(int(d))
c424ff4807e6 wireprotov1peer: update all rpcs to use the new batchable scheme
Valentin Gatien-Baron <valentin.gatienbaron@gmail.com>
parents: 47872
diff changeset
418 except ValueError:
c424ff4807e6 wireprotov1peer: update all rpcs to use the new batchable scheme
Valentin Gatien-Baron <valentin.gatienbaron@gmail.com>
parents: 47872
diff changeset
419 raise error.ResponseError(
c424ff4807e6 wireprotov1peer: update all rpcs to use the new batchable scheme
Valentin Gatien-Baron <valentin.gatienbaron@gmail.com>
parents: 47872
diff changeset
420 _(b'push failed (unexpected response):'), d
c424ff4807e6 wireprotov1peer: update all rpcs to use the new batchable scheme
Valentin Gatien-Baron <valentin.gatienbaron@gmail.com>
parents: 47872
diff changeset
421 )
c424ff4807e6 wireprotov1peer: update all rpcs to use the new batchable scheme
Valentin Gatien-Baron <valentin.gatienbaron@gmail.com>
parents: 47872
diff changeset
422 for l in output.splitlines(True):
c424ff4807e6 wireprotov1peer: update all rpcs to use the new batchable scheme
Valentin Gatien-Baron <valentin.gatienbaron@gmail.com>
parents: 47872
diff changeset
423 self.ui.status(_(b'remote: '), l)
c424ff4807e6 wireprotov1peer: update all rpcs to use the new batchable scheme
Valentin Gatien-Baron <valentin.gatienbaron@gmail.com>
parents: 47872
diff changeset
424 return d
c424ff4807e6 wireprotov1peer: update all rpcs to use the new batchable scheme
Valentin Gatien-Baron <valentin.gatienbaron@gmail.com>
parents: 47872
diff changeset
425
c424ff4807e6 wireprotov1peer: update all rpcs to use the new batchable scheme
Valentin Gatien-Baron <valentin.gatienbaron@gmail.com>
parents: 47872
diff changeset
426 return {
43077
687b865b95ad formatting: byteify all mercurial/ and hgext/ string literals
Augie Fackler <augie@google.com>
parents: 43076
diff changeset
427 b'namespace': encoding.fromlocal(namespace),
687b865b95ad formatting: byteify all mercurial/ and hgext/ string literals
Augie Fackler <augie@google.com>
parents: 43076
diff changeset
428 b'key': encoding.fromlocal(key),
687b865b95ad formatting: byteify all mercurial/ and hgext/ string literals
Augie Fackler <augie@google.com>
parents: 43076
diff changeset
429 b'old': encoding.fromlocal(old),
687b865b95ad formatting: byteify all mercurial/ and hgext/ string literals
Augie Fackler <augie@google.com>
parents: 43076
diff changeset
430 b'new': encoding.fromlocal(new),
47873
c424ff4807e6 wireprotov1peer: update all rpcs to use the new batchable scheme
Valentin Gatien-Baron <valentin.gatienbaron@gmail.com>
parents: 47872
diff changeset
431 }, decode
11586
ddaaaa23bb8f protocol: move basic ssh client commands to wirerepository
Matt Mackall <mpm@selenic.com>
parents: 11585
diff changeset
432
11588
8a1f625e971d protocol: unify stream_out client code
Matt Mackall <mpm@selenic.com>
parents: 11587
diff changeset
433 def stream_out(self):
43077
687b865b95ad formatting: byteify all mercurial/ and hgext/ string literals
Augie Fackler <augie@google.com>
parents: 43076
diff changeset
434 return self._callstream(b'stream_out')
11588
8a1f625e971d protocol: unify stream_out client code
Matt Mackall <mpm@selenic.com>
parents: 11587
diff changeset
435
21646
ce25f465e572 getbundle: declare type of parameters
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 21631
diff changeset
436 def getbundle(self, source, **kwargs):
34739
b880cc11da5d wireproto: bounce kwargs to/from bytes/str as needed
Augie Fackler <augie@google.com>
parents: 34731
diff changeset
437 kwargs = pycompat.byteskwargs(kwargs)
43077
687b865b95ad formatting: byteify all mercurial/ and hgext/ string literals
Augie Fackler <augie@google.com>
parents: 43076
diff changeset
438 self.requirecap(b'getbundle', _(b'look up remote changes'))
13741
b51bf961b3cb wireproto: add getbundle() function
Peter Arrenbrecht <peter.arrenbrecht@gmail.com>
parents: 13726
diff changeset
439 opts = {}
43077
687b865b95ad formatting: byteify all mercurial/ and hgext/ string literals
Augie Fackler <augie@google.com>
parents: 43076
diff changeset
440 bundlecaps = kwargs.get(b'bundlecaps') or set()
43106
d783f945a701 py3: finish porting iteritems() to pycompat and remove source transformer
Gregory Szorc <gregory.szorc@gmail.com>
parents: 43089
diff changeset
441 for key, value in pycompat.iteritems(kwargs):
21646
ce25f465e572 getbundle: declare type of parameters
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 21631
diff changeset
442 if value is None:
ce25f465e572 getbundle: declare type of parameters
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 21631
diff changeset
443 continue
37613
96d735601ca1 wireproto: move gboptsmap to wireprototypes and rename (API)
Gregory Szorc <gregory.szorc@gmail.com>
parents: 37612
diff changeset
444 keytype = wireprototypes.GETBUNDLE_ARGUMENTS.get(key)
21646
ce25f465e572 getbundle: declare type of parameters
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 21631
diff changeset
445 if keytype is None:
34730
6be264009841 wireproto: use a proper exception instead of `assert False`
Augie Fackler <augie@google.com>
parents: 34729
diff changeset
446 raise error.ProgrammingError(
43077
687b865b95ad formatting: byteify all mercurial/ and hgext/ string literals
Augie Fackler <augie@google.com>
parents: 43076
diff changeset
447 b'Unexpectedly None keytype for key %s' % key
43076
2372284d9457 formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents: 42814
diff changeset
448 )
43077
687b865b95ad formatting: byteify all mercurial/ and hgext/ string literals
Augie Fackler <augie@google.com>
parents: 43076
diff changeset
449 elif keytype == b'nodes':
37612
5e71dea79aae wireproto: move value encoding functions to wireprototypes (API)
Gregory Szorc <gregory.szorc@gmail.com>
parents: 37546
diff changeset
450 value = wireprototypes.encodelist(value)
43077
687b865b95ad formatting: byteify all mercurial/ and hgext/ string literals
Augie Fackler <augie@google.com>
parents: 43076
diff changeset
451 elif keytype == b'csv':
687b865b95ad formatting: byteify all mercurial/ and hgext/ string literals
Augie Fackler <augie@google.com>
parents: 43076
diff changeset
452 value = b','.join(value)
687b865b95ad formatting: byteify all mercurial/ and hgext/ string literals
Augie Fackler <augie@google.com>
parents: 43076
diff changeset
453 elif keytype == b'scsv':
687b865b95ad formatting: byteify all mercurial/ and hgext/ string literals
Augie Fackler <augie@google.com>
parents: 43076
diff changeset
454 value = b','.join(sorted(value))
687b865b95ad formatting: byteify all mercurial/ and hgext/ string literals
Augie Fackler <augie@google.com>
parents: 43076
diff changeset
455 elif keytype == b'boolean':
687b865b95ad formatting: byteify all mercurial/ and hgext/ string literals
Augie Fackler <augie@google.com>
parents: 43076
diff changeset
456 value = b'%i' % bool(value)
687b865b95ad formatting: byteify all mercurial/ and hgext/ string literals
Augie Fackler <augie@google.com>
parents: 43076
diff changeset
457 elif keytype != b'plain':
687b865b95ad formatting: byteify all mercurial/ and hgext/ string literals
Augie Fackler <augie@google.com>
parents: 43076
diff changeset
458 raise KeyError(b'unknown getbundle option type %s' % keytype)
21646
ce25f465e572 getbundle: declare type of parameters
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 21631
diff changeset
459 opts[key] = value
43077
687b865b95ad formatting: byteify all mercurial/ and hgext/ string literals
Augie Fackler <augie@google.com>
parents: 43076
diff changeset
460 f = self._callcompressable(b"getbundle", **pycompat.strkwargs(opts))
687b865b95ad formatting: byteify all mercurial/ and hgext/ string literals
Augie Fackler <augie@google.com>
parents: 43076
diff changeset
461 if any((cap.startswith(b'HG2') for cap in bundlecaps)):
24641
60fecc5b14a4 unbundle20: retrieve unbundler instances through a factory function
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 23878
diff changeset
462 return bundle2.getunbundler(self.ui, f)
21069
0a9cae236738 bundle2: allow bundle2 for pulling over the wire
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 21064
diff changeset
463 else:
43077
687b865b95ad formatting: byteify all mercurial/ and hgext/ string literals
Augie Fackler <augie@google.com>
parents: 43076
diff changeset
464 return changegroupmod.cg1unpacker(f, b'UN')
13741
b51bf961b3cb wireproto: add getbundle() function
Peter Arrenbrecht <peter.arrenbrecht@gmail.com>
parents: 13726
diff changeset
465
37646
72e26319f3b8 wireproto: use command executor for unbundle
Gregory Szorc <gregory.szorc@gmail.com>
parents: 37635
diff changeset
466 def unbundle(self, bundle, heads, url):
45942
89a2afe31e82 formating: upgrade to black 20.8b1
Augie Fackler <raf@durin42.com>
parents: 44060
diff changeset
467 """Send cg (a readable file-like object representing the
11592
26e0782b8380 protocol: unify client unbundle support
Matt Mackall <mpm@selenic.com>
parents: 11591
diff changeset
468 changegroup to push, typically a chunkbuffer object) to the
21075
438803e4bd97 bundle2: support for push over the wire
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 21072
diff changeset
469 remote server as a bundle.
438803e4bd97 bundle2: support for push over the wire
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 21072
diff changeset
470
438803e4bd97 bundle2: support for push over the wire
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 21072
diff changeset
471 When pushing a bundle10 stream, return an integer indicating the
32880
4c2a46f89f08 wireproto: update reference to deleted addchangegroup()
Martin von Zweigbergk <martinvonz@google.com>
parents: 32744
diff changeset
472 result of the push (see changegroup.apply()).
21075
438803e4bd97 bundle2: support for push over the wire
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 21072
diff changeset
473
29706
7f6130c7ffe1 wirepeer: rename confusing `source` parameter
Augie Fackler <augie@google.com>
parents: 29590
diff changeset
474 When pushing a bundle20 stream, return a bundle20 stream.
7f6130c7ffe1 wirepeer: rename confusing `source` parameter
Augie Fackler <augie@google.com>
parents: 29590
diff changeset
475
7f6130c7ffe1 wirepeer: rename confusing `source` parameter
Augie Fackler <augie@google.com>
parents: 29590
diff changeset
476 `url` is the url the client thinks it's pushing to, which is
7f6130c7ffe1 wirepeer: rename confusing `source` parameter
Augie Fackler <augie@google.com>
parents: 29590
diff changeset
477 visible to hooks.
45942
89a2afe31e82 formating: upgrade to black 20.8b1
Augie Fackler <raf@durin42.com>
parents: 44060
diff changeset
478 """
11592
26e0782b8380 protocol: unify client unbundle support
Matt Mackall <mpm@selenic.com>
parents: 11591
diff changeset
479
43077
687b865b95ad formatting: byteify all mercurial/ and hgext/ string literals
Augie Fackler <augie@google.com>
parents: 43076
diff changeset
480 if heads != [b'force'] and self.capable(b'unbundlehash'):
37612
5e71dea79aae wireproto: move value encoding functions to wireprototypes (API)
Gregory Szorc <gregory.szorc@gmail.com>
parents: 37546
diff changeset
481 heads = wireprototypes.encodelist(
44060
a61287a95dc3 core: migrate uses of hashlib.sha1 to hashutil.sha1
Augie Fackler <augie@google.com>
parents: 43506
diff changeset
482 [b'hashed', hashutil.sha1(b''.join(sorted(heads))).digest()]
43076
2372284d9457 formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents: 42814
diff changeset
483 )
13942
88f0e41d8802 wireproto: allow unbundle with hashed heads parameter (issue2126)
Shuhei Takahashi <takahashi.shuhei@gmail.com>
parents: 13741
diff changeset
484 else:
37612
5e71dea79aae wireproto: move value encoding functions to wireprototypes (API)
Gregory Szorc <gregory.szorc@gmail.com>
parents: 37546
diff changeset
485 heads = wireprototypes.encodelist(heads)
13942
88f0e41d8802 wireproto: allow unbundle with hashed heads parameter (issue2126)
Shuhei Takahashi <takahashi.shuhei@gmail.com>
parents: 13741
diff changeset
486
43077
687b865b95ad formatting: byteify all mercurial/ and hgext/ string literals
Augie Fackler <augie@google.com>
parents: 43076
diff changeset
487 if util.safehasattr(bundle, b'deltaheader'):
21075
438803e4bd97 bundle2: support for push over the wire
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 21072
diff changeset
488 # this a bundle10, do the old style call sequence
43077
687b865b95ad formatting: byteify all mercurial/ and hgext/ string literals
Augie Fackler <augie@google.com>
parents: 43076
diff changeset
489 ret, output = self._callpush(b"unbundle", bundle, heads=heads)
687b865b95ad formatting: byteify all mercurial/ and hgext/ string literals
Augie Fackler <augie@google.com>
parents: 43076
diff changeset
490 if ret == b"":
687b865b95ad formatting: byteify all mercurial/ and hgext/ string literals
Augie Fackler <augie@google.com>
parents: 43076
diff changeset
491 raise error.ResponseError(_(b'push failed:'), output)
21075
438803e4bd97 bundle2: support for push over the wire
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 21072
diff changeset
492 try:
438803e4bd97 bundle2: support for push over the wire
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 21072
diff changeset
493 ret = int(ret)
438803e4bd97 bundle2: support for push over the wire
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 21072
diff changeset
494 except ValueError:
438803e4bd97 bundle2: support for push over the wire
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 21072
diff changeset
495 raise error.ResponseError(
43077
687b865b95ad formatting: byteify all mercurial/ and hgext/ string literals
Augie Fackler <augie@google.com>
parents: 43076
diff changeset
496 _(b'push failed (unexpected response):'), ret
43076
2372284d9457 formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents: 42814
diff changeset
497 )
11592
26e0782b8380 protocol: unify client unbundle support
Matt Mackall <mpm@selenic.com>
parents: 11591
diff changeset
498
21075
438803e4bd97 bundle2: support for push over the wire
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 21072
diff changeset
499 for l in output.splitlines(True):
43077
687b865b95ad formatting: byteify all mercurial/ and hgext/ string literals
Augie Fackler <augie@google.com>
parents: 43076
diff changeset
500 self.ui.status(_(b'remote: '), l)
21075
438803e4bd97 bundle2: support for push over the wire
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 21072
diff changeset
501 else:
438803e4bd97 bundle2: support for push over the wire
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 21072
diff changeset
502 # bundle2 push. Send a stream, fetch a stream.
43077
687b865b95ad formatting: byteify all mercurial/ and hgext/ string literals
Augie Fackler <augie@google.com>
parents: 43076
diff changeset
503 stream = self._calltwowaystream(b'unbundle', bundle, heads=heads)
24641
60fecc5b14a4 unbundle20: retrieve unbundler instances through a factory function
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 23878
diff changeset
504 ret = bundle2.getunbundler(self.ui, stream)
11592
26e0782b8380 protocol: unify client unbundle support
Matt Mackall <mpm@selenic.com>
parents: 11591
diff changeset
505 return ret
26e0782b8380 protocol: unify client unbundle support
Matt Mackall <mpm@selenic.com>
parents: 11591
diff changeset
506
37320
39f7d4ee8bcd repository: port peer interfaces to zope.interface
Gregory Szorc <gregory.szorc@gmail.com>
parents: 37295
diff changeset
507 # End of ipeercommands interface.
33806
dedab036215d wireproto: use new peer interface
Gregory Szorc <gregory.szorc@gmail.com>
parents: 33767
diff changeset
508
37320
39f7d4ee8bcd repository: port peer interfaces to zope.interface
Gregory Szorc <gregory.szorc@gmail.com>
parents: 37295
diff changeset
509 # Begin of ipeerlegacycommands interface.
33806
dedab036215d wireproto: use new peer interface
Gregory Szorc <gregory.szorc@gmail.com>
parents: 33767
diff changeset
510
dedab036215d wireproto: use new peer interface
Gregory Szorc <gregory.szorc@gmail.com>
parents: 33767
diff changeset
511 def branches(self, nodes):
37612
5e71dea79aae wireproto: move value encoding functions to wireprototypes (API)
Gregory Szorc <gregory.szorc@gmail.com>
parents: 37546
diff changeset
512 n = wireprototypes.encodelist(nodes)
43077
687b865b95ad formatting: byteify all mercurial/ and hgext/ string literals
Augie Fackler <augie@google.com>
parents: 43076
diff changeset
513 d = self._call(b"branches", nodes=n)
33806
dedab036215d wireproto: use new peer interface
Gregory Szorc <gregory.szorc@gmail.com>
parents: 33767
diff changeset
514 try:
37612
5e71dea79aae wireproto: move value encoding functions to wireprototypes (API)
Gregory Szorc <gregory.szorc@gmail.com>
parents: 37546
diff changeset
515 br = [tuple(wireprototypes.decodelist(b)) for b in d.splitlines()]
33806
dedab036215d wireproto: use new peer interface
Gregory Szorc <gregory.szorc@gmail.com>
parents: 33767
diff changeset
516 return br
dedab036215d wireproto: use new peer interface
Gregory Szorc <gregory.szorc@gmail.com>
parents: 33767
diff changeset
517 except ValueError:
43077
687b865b95ad formatting: byteify all mercurial/ and hgext/ string literals
Augie Fackler <augie@google.com>
parents: 43076
diff changeset
518 self._abort(error.ResponseError(_(b"unexpected response:"), d))
33806
dedab036215d wireproto: use new peer interface
Gregory Szorc <gregory.szorc@gmail.com>
parents: 33767
diff changeset
519
dedab036215d wireproto: use new peer interface
Gregory Szorc <gregory.szorc@gmail.com>
parents: 33767
diff changeset
520 def between(self, pairs):
43076
2372284d9457 formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents: 42814
diff changeset
521 batch = 8 # avoid giant requests
33806
dedab036215d wireproto: use new peer interface
Gregory Szorc <gregory.szorc@gmail.com>
parents: 33767
diff changeset
522 r = []
38783
e7aa113b14f7 global: use pycompat.xrange()
Gregory Szorc <gregory.szorc@gmail.com>
parents: 38328
diff changeset
523 for i in pycompat.xrange(0, len(pairs), batch):
43077
687b865b95ad formatting: byteify all mercurial/ and hgext/ string literals
Augie Fackler <augie@google.com>
parents: 43076
diff changeset
524 n = b" ".join(
43076
2372284d9457 formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents: 42814
diff changeset
525 [
43077
687b865b95ad formatting: byteify all mercurial/ and hgext/ string literals
Augie Fackler <augie@google.com>
parents: 43076
diff changeset
526 wireprototypes.encodelist(p, b'-')
43076
2372284d9457 formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents: 42814
diff changeset
527 for p in pairs[i : i + batch]
2372284d9457 formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents: 42814
diff changeset
528 ]
2372284d9457 formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents: 42814
diff changeset
529 )
43077
687b865b95ad formatting: byteify all mercurial/ and hgext/ string literals
Augie Fackler <augie@google.com>
parents: 43076
diff changeset
530 d = self._call(b"between", pairs=n)
33806
dedab036215d wireproto: use new peer interface
Gregory Szorc <gregory.szorc@gmail.com>
parents: 33767
diff changeset
531 try:
43076
2372284d9457 formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents: 42814
diff changeset
532 r.extend(
2372284d9457 formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents: 42814
diff changeset
533 l and wireprototypes.decodelist(l) or []
2372284d9457 formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents: 42814
diff changeset
534 for l in d.splitlines()
2372284d9457 formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents: 42814
diff changeset
535 )
33806
dedab036215d wireproto: use new peer interface
Gregory Szorc <gregory.szorc@gmail.com>
parents: 33767
diff changeset
536 except ValueError:
43077
687b865b95ad formatting: byteify all mercurial/ and hgext/ string literals
Augie Fackler <augie@google.com>
parents: 43076
diff changeset
537 self._abort(error.ResponseError(_(b"unexpected response:"), d))
33806
dedab036215d wireproto: use new peer interface
Gregory Szorc <gregory.szorc@gmail.com>
parents: 33767
diff changeset
538 return r
dedab036215d wireproto: use new peer interface
Gregory Szorc <gregory.szorc@gmail.com>
parents: 33767
diff changeset
539
37635
cc8c06835097 wireproto: convert legacy commands to command executor
Gregory Szorc <gregory.szorc@gmail.com>
parents: 37633
diff changeset
540 def changegroup(self, nodes, source):
37612
5e71dea79aae wireproto: move value encoding functions to wireprototypes (API)
Gregory Szorc <gregory.szorc@gmail.com>
parents: 37546
diff changeset
541 n = wireprototypes.encodelist(nodes)
43077
687b865b95ad formatting: byteify all mercurial/ and hgext/ string literals
Augie Fackler <augie@google.com>
parents: 43076
diff changeset
542 f = self._callcompressable(b"changegroup", roots=n)
687b865b95ad formatting: byteify all mercurial/ and hgext/ string literals
Augie Fackler <augie@google.com>
parents: 43076
diff changeset
543 return changegroupmod.cg1unpacker(f, b'UN')
33806
dedab036215d wireproto: use new peer interface
Gregory Szorc <gregory.szorc@gmail.com>
parents: 33767
diff changeset
544
37635
cc8c06835097 wireproto: convert legacy commands to command executor
Gregory Szorc <gregory.szorc@gmail.com>
parents: 37633
diff changeset
545 def changegroupsubset(self, bases, heads, source):
43077
687b865b95ad formatting: byteify all mercurial/ and hgext/ string literals
Augie Fackler <augie@google.com>
parents: 43076
diff changeset
546 self.requirecap(b'changegroupsubset', _(b'look up remote changes'))
37612
5e71dea79aae wireproto: move value encoding functions to wireprototypes (API)
Gregory Szorc <gregory.szorc@gmail.com>
parents: 37546
diff changeset
547 bases = wireprototypes.encodelist(bases)
5e71dea79aae wireproto: move value encoding functions to wireprototypes (API)
Gregory Szorc <gregory.szorc@gmail.com>
parents: 37546
diff changeset
548 heads = wireprototypes.encodelist(heads)
43076
2372284d9457 formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents: 42814
diff changeset
549 f = self._callcompressable(
43077
687b865b95ad formatting: byteify all mercurial/ and hgext/ string literals
Augie Fackler <augie@google.com>
parents: 43076
diff changeset
550 b"changegroupsubset", bases=bases, heads=heads
43076
2372284d9457 formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents: 42814
diff changeset
551 )
43077
687b865b95ad formatting: byteify all mercurial/ and hgext/ string literals
Augie Fackler <augie@google.com>
parents: 43076
diff changeset
552 return changegroupmod.cg1unpacker(f, b'UN')
33806
dedab036215d wireproto: use new peer interface
Gregory Szorc <gregory.szorc@gmail.com>
parents: 33767
diff changeset
553
37320
39f7d4ee8bcd repository: port peer interfaces to zope.interface
Gregory Szorc <gregory.szorc@gmail.com>
parents: 37295
diff changeset
554 # End of ipeerlegacycommands interface.
33806
dedab036215d wireproto: use new peer interface
Gregory Szorc <gregory.szorc@gmail.com>
parents: 33767
diff changeset
555
dedab036215d wireproto: use new peer interface
Gregory Szorc <gregory.szorc@gmail.com>
parents: 33767
diff changeset
556 def _submitbatch(self, req):
dedab036215d wireproto: use new peer interface
Gregory Szorc <gregory.szorc@gmail.com>
parents: 33767
diff changeset
557 """run batch request <req> on the server
dedab036215d wireproto: use new peer interface
Gregory Szorc <gregory.szorc@gmail.com>
parents: 33767
diff changeset
558
dedab036215d wireproto: use new peer interface
Gregory Szorc <gregory.szorc@gmail.com>
parents: 33767
diff changeset
559 Returns an iterator of the raw responses from the server.
dedab036215d wireproto: use new peer interface
Gregory Szorc <gregory.szorc@gmail.com>
parents: 33767
diff changeset
560 """
36945
4901d1e22b27 peer-request: include more details about batch commands
Boris Feld <boris.feld@octobus.net>
parents: 36840
diff changeset
561 ui = self.ui
43077
687b865b95ad formatting: byteify all mercurial/ and hgext/ string literals
Augie Fackler <augie@google.com>
parents: 43076
diff changeset
562 if ui.debugflag and ui.configbool(b'devel', b'debug.peer-request'):
687b865b95ad formatting: byteify all mercurial/ and hgext/ string literals
Augie Fackler <augie@google.com>
parents: 43076
diff changeset
563 ui.debug(b'devel-peer-request: batched-content\n')
36945
4901d1e22b27 peer-request: include more details about batch commands
Boris Feld <boris.feld@octobus.net>
parents: 36840
diff changeset
564 for op, args in req:
43077
687b865b95ad formatting: byteify all mercurial/ and hgext/ string literals
Augie Fackler <augie@google.com>
parents: 43076
diff changeset
565 msg = b'devel-peer-request: - %s (%d arguments)\n'
36945
4901d1e22b27 peer-request: include more details about batch commands
Boris Feld <boris.feld@octobus.net>
parents: 36840
diff changeset
566 ui.debug(msg % (op, len(args)))
4901d1e22b27 peer-request: include more details about batch commands
Boris Feld <boris.feld@octobus.net>
parents: 36840
diff changeset
567
37612
5e71dea79aae wireproto: move value encoding functions to wireprototypes (API)
Gregory Szorc <gregory.szorc@gmail.com>
parents: 37546
diff changeset
568 unescapearg = wireprototypes.unescapebatcharg
5e71dea79aae wireproto: move value encoding functions to wireprototypes (API)
Gregory Szorc <gregory.szorc@gmail.com>
parents: 37546
diff changeset
569
43077
687b865b95ad formatting: byteify all mercurial/ and hgext/ string literals
Augie Fackler <augie@google.com>
parents: 43076
diff changeset
570 rsp = self._callstream(b"batch", cmds=encodebatchcmds(req))
33806
dedab036215d wireproto: use new peer interface
Gregory Szorc <gregory.szorc@gmail.com>
parents: 33767
diff changeset
571 chunk = rsp.read(1024)
dedab036215d wireproto: use new peer interface
Gregory Szorc <gregory.szorc@gmail.com>
parents: 33767
diff changeset
572 work = [chunk]
dedab036215d wireproto: use new peer interface
Gregory Szorc <gregory.szorc@gmail.com>
parents: 33767
diff changeset
573 while chunk:
43077
687b865b95ad formatting: byteify all mercurial/ and hgext/ string literals
Augie Fackler <augie@google.com>
parents: 43076
diff changeset
574 while b';' not in chunk and chunk:
33806
dedab036215d wireproto: use new peer interface
Gregory Szorc <gregory.szorc@gmail.com>
parents: 33767
diff changeset
575 chunk = rsp.read(1024)
dedab036215d wireproto: use new peer interface
Gregory Szorc <gregory.szorc@gmail.com>
parents: 33767
diff changeset
576 work.append(chunk)
43077
687b865b95ad formatting: byteify all mercurial/ and hgext/ string literals
Augie Fackler <augie@google.com>
parents: 43076
diff changeset
577 merged = b''.join(work)
687b865b95ad formatting: byteify all mercurial/ and hgext/ string literals
Augie Fackler <augie@google.com>
parents: 43076
diff changeset
578 while b';' in merged:
687b865b95ad formatting: byteify all mercurial/ and hgext/ string literals
Augie Fackler <augie@google.com>
parents: 43076
diff changeset
579 one, merged = merged.split(b';', 1)
33806
dedab036215d wireproto: use new peer interface
Gregory Szorc <gregory.szorc@gmail.com>
parents: 33767
diff changeset
580 yield unescapearg(one)
dedab036215d wireproto: use new peer interface
Gregory Szorc <gregory.szorc@gmail.com>
parents: 33767
diff changeset
581 chunk = rsp.read(1024)
dedab036215d wireproto: use new peer interface
Gregory Szorc <gregory.szorc@gmail.com>
parents: 33767
diff changeset
582 work = [merged, chunk]
43077
687b865b95ad formatting: byteify all mercurial/ and hgext/ string literals
Augie Fackler <augie@google.com>
parents: 43076
diff changeset
583 yield unescapearg(b''.join(work))
33806
dedab036215d wireproto: use new peer interface
Gregory Szorc <gregory.szorc@gmail.com>
parents: 33767
diff changeset
584
dedab036215d wireproto: use new peer interface
Gregory Szorc <gregory.szorc@gmail.com>
parents: 33767
diff changeset
585 def _submitone(self, op, args):
34739
b880cc11da5d wireproto: bounce kwargs to/from bytes/str as needed
Augie Fackler <augie@google.com>
parents: 34731
diff changeset
586 return self._call(op, **pycompat.strkwargs(args))
33806
dedab036215d wireproto: use new peer interface
Gregory Szorc <gregory.szorc@gmail.com>
parents: 33767
diff changeset
587
14048
58e58406ed19 wireproto: add test for new optional arg missing on server
Peter Arrenbrecht <peter.arrenbrecht@gmail.com>
parents: 13942
diff changeset
588 def debugwireargs(self, one, two, three=None, four=None, five=None):
13720
9c4e04fe267e debug: add debugwireargs to test argument passing over the wire
Peter Arrenbrecht <peter.arrenbrecht@gmail.com>
parents: 13450
diff changeset
589 # don't pass optional arguments left at their default value
9c4e04fe267e debug: add debugwireargs to test argument passing over the wire
Peter Arrenbrecht <peter.arrenbrecht@gmail.com>
parents: 13450
diff changeset
590 opts = {}
9c4e04fe267e debug: add debugwireargs to test argument passing over the wire
Peter Arrenbrecht <peter.arrenbrecht@gmail.com>
parents: 13450
diff changeset
591 if three is not None:
43506
9f70512ae2cf cleanup: remove pointless r-prefixes on single-quoted strings
Augie Fackler <augie@google.com>
parents: 43117
diff changeset
592 opts['three'] = three
13720
9c4e04fe267e debug: add debugwireargs to test argument passing over the wire
Peter Arrenbrecht <peter.arrenbrecht@gmail.com>
parents: 13450
diff changeset
593 if four is not None:
43506
9f70512ae2cf cleanup: remove pointless r-prefixes on single-quoted strings
Augie Fackler <augie@google.com>
parents: 43117
diff changeset
594 opts['four'] = four
43077
687b865b95ad formatting: byteify all mercurial/ and hgext/ string literals
Augie Fackler <augie@google.com>
parents: 43076
diff changeset
595 return self._call(b'debugwireargs', one=one, two=two, **opts)
13720
9c4e04fe267e debug: add debugwireargs to test argument passing over the wire
Peter Arrenbrecht <peter.arrenbrecht@gmail.com>
parents: 13450
diff changeset
596
20904
3dbe6bcd7f62 wireproto: document protocol specific function of wirepeer
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 20903
diff changeset
597 def _call(self, cmd, **args):
3dbe6bcd7f62 wireproto: document protocol specific function of wirepeer
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 20903
diff changeset
598 """execute <cmd> on the server
3dbe6bcd7f62 wireproto: document protocol specific function of wirepeer
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 20903
diff changeset
599
3dbe6bcd7f62 wireproto: document protocol specific function of wirepeer
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 20903
diff changeset
600 The command is expected to return a simple string.
3dbe6bcd7f62 wireproto: document protocol specific function of wirepeer
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 20903
diff changeset
601
3dbe6bcd7f62 wireproto: document protocol specific function of wirepeer
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 20903
diff changeset
602 returns the server reply as a string."""
3dbe6bcd7f62 wireproto: document protocol specific function of wirepeer
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 20903
diff changeset
603 raise NotImplementedError()
3dbe6bcd7f62 wireproto: document protocol specific function of wirepeer
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 20903
diff changeset
604
3dbe6bcd7f62 wireproto: document protocol specific function of wirepeer
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 20903
diff changeset
605 def _callstream(self, cmd, **args):
3dbe6bcd7f62 wireproto: document protocol specific function of wirepeer
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 20903
diff changeset
606 """execute <cmd> on the server
3dbe6bcd7f62 wireproto: document protocol specific function of wirepeer
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 20903
diff changeset
607
28435
176736afa886 wireproto: document quirk of _callstream between http and ssh
Augie Fackler <augie@google.com>
parents: 27633
diff changeset
608 The command is expected to return a stream. Note that if the
176736afa886 wireproto: document quirk of _callstream between http and ssh
Augie Fackler <augie@google.com>
parents: 27633
diff changeset
609 command doesn't return a stream, _callstream behaves
176736afa886 wireproto: document quirk of _callstream between http and ssh
Augie Fackler <augie@google.com>
parents: 27633
diff changeset
610 differently for ssh and http peers.
20904
3dbe6bcd7f62 wireproto: document protocol specific function of wirepeer
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 20903
diff changeset
611
28435
176736afa886 wireproto: document quirk of _callstream between http and ssh
Augie Fackler <augie@google.com>
parents: 27633
diff changeset
612 returns the server reply as a file like object.
176736afa886 wireproto: document quirk of _callstream between http and ssh
Augie Fackler <augie@google.com>
parents: 27633
diff changeset
613 """
20904
3dbe6bcd7f62 wireproto: document protocol specific function of wirepeer
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 20903
diff changeset
614 raise NotImplementedError()
3dbe6bcd7f62 wireproto: document protocol specific function of wirepeer
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 20903
diff changeset
615
20905
167047ba3cfa wireproto: drop the _decompress method in favor a new call type
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 20904
diff changeset
616 def _callcompressable(self, cmd, **args):
167047ba3cfa wireproto: drop the _decompress method in favor a new call type
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 20904
diff changeset
617 """execute <cmd> on the server
167047ba3cfa wireproto: drop the _decompress method in favor a new call type
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 20904
diff changeset
618
167047ba3cfa wireproto: drop the _decompress method in favor a new call type
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 20904
diff changeset
619 The command is expected to return a stream.
167047ba3cfa wireproto: drop the _decompress method in favor a new call type
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 20904
diff changeset
620
21024
7731a2281cf0 spelling: fixes from spell checker
Mads Kiilerich <madski@unity3d.com>
parents: 20969
diff changeset
621 The stream may have been compressed in some implementations. This
20905
167047ba3cfa wireproto: drop the _decompress method in favor a new call type
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 20904
diff changeset
622 function takes care of the decompression. This is the only difference
167047ba3cfa wireproto: drop the _decompress method in favor a new call type
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 20904
diff changeset
623 with _callstream.
167047ba3cfa wireproto: drop the _decompress method in favor a new call type
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 20904
diff changeset
624
167047ba3cfa wireproto: drop the _decompress method in favor a new call type
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 20904
diff changeset
625 returns the server reply as a file like object.
167047ba3cfa wireproto: drop the _decompress method in favor a new call type
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 20904
diff changeset
626 """
167047ba3cfa wireproto: drop the _decompress method in favor a new call type
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 20904
diff changeset
627 raise NotImplementedError()
167047ba3cfa wireproto: drop the _decompress method in favor a new call type
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 20904
diff changeset
628
20904
3dbe6bcd7f62 wireproto: document protocol specific function of wirepeer
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 20903
diff changeset
629 def _callpush(self, cmd, fp, **args):
3dbe6bcd7f62 wireproto: document protocol specific function of wirepeer
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 20903
diff changeset
630 """execute a <cmd> on server
3dbe6bcd7f62 wireproto: document protocol specific function of wirepeer
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 20903
diff changeset
631
3dbe6bcd7f62 wireproto: document protocol specific function of wirepeer
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 20903
diff changeset
632 The command is expected to be related to a push. Push has a special
3dbe6bcd7f62 wireproto: document protocol specific function of wirepeer
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 20903
diff changeset
633 return method.
3dbe6bcd7f62 wireproto: document protocol specific function of wirepeer
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 20903
diff changeset
634
3dbe6bcd7f62 wireproto: document protocol specific function of wirepeer
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 20903
diff changeset
635 returns the server reply as a (ret, output) tuple. ret is either
3dbe6bcd7f62 wireproto: document protocol specific function of wirepeer
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 20903
diff changeset
636 empty (error) or a stringified int.
3dbe6bcd7f62 wireproto: document protocol specific function of wirepeer
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 20903
diff changeset
637 """
3dbe6bcd7f62 wireproto: document protocol specific function of wirepeer
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 20903
diff changeset
638 raise NotImplementedError()
3dbe6bcd7f62 wireproto: document protocol specific function of wirepeer
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 20903
diff changeset
639
21072
0879352d67d8 wireproto: add a _calltwowaystream method on wirepeer
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 21069
diff changeset
640 def _calltwowaystream(self, cmd, fp, **args):
0879352d67d8 wireproto: add a _calltwowaystream method on wirepeer
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 21069
diff changeset
641 """execute <cmd> on server
0879352d67d8 wireproto: add a _calltwowaystream method on wirepeer
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 21069
diff changeset
642
0879352d67d8 wireproto: add a _calltwowaystream method on wirepeer
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 21069
diff changeset
643 The command will send a stream to the server and get a stream in reply.
0879352d67d8 wireproto: add a _calltwowaystream method on wirepeer
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 21069
diff changeset
644 """
0879352d67d8 wireproto: add a _calltwowaystream method on wirepeer
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 21069
diff changeset
645 raise NotImplementedError()
0879352d67d8 wireproto: add a _calltwowaystream method on wirepeer
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 21069
diff changeset
646
20904
3dbe6bcd7f62 wireproto: document protocol specific function of wirepeer
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 20903
diff changeset
647 def _abort(self, exception):
45942
89a2afe31e82 formating: upgrade to black 20.8b1
Augie Fackler <raf@durin42.com>
parents: 44060
diff changeset
648 """clearly abort the wire protocol connection and raise the exception"""
20904
3dbe6bcd7f62 wireproto: document protocol specific function of wirepeer
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 20903
diff changeset
649 raise NotImplementedError()