mercurial-scm/hg-stable: mercurial/wireprotov1peer.py annotate

annotate mercurial/wireprotov1peer.py @ 50675:60f9602b413e

clonebundles: add support for inline (streaming) clonebundles The idea behind inline clonebundles is to send them through the ssh or https connection to the Mercurial server. We've been using this specifically for streaming clonebundles, although it works for 'regular' clonebundles as well (but is less relevant, since pullbundles exist). We've had this enabled for around 9 months for a part of our users. A few benefits are: - no need to secure an external system, since everything goes through the same Mercurial server - easier scaling (in our case: no risk of inconsistencies between multiple mercurial-server mirrors and nginx clonebundles hosts) Remaining topics/questions right now: - The inline clonebundles don't work for https yet. This is because httppeer doesn't seem to support sending client capabilities. I didn't focus on that as my main goal was to get this working for ssh.

author	Mathias De Mare <mathias.de_mare@nokia.com>
date	Wed, 08 Mar 2023 14:23:43 +0100
parents	b1fb4185e47c
children	2aaabd8f4471

rev	line source
37614 a81d02ea65db wireproto: move version 1 peer functionality to standalone module (API) Gregory Szorc <gregory.szorc@gmail.com> parents: 37613 diff changeset	1 # wireprotov1peer.py - Client-side functionality for wire protocol version 1.
11581 4530b3307fb9 protocol: introduce wireproto.py Matt Mackall <mpm@selenic.com> parents: diff changeset	2 #
46819 d4ba4d51f85f contributor: change mentions of mpm to olivia Rapha?l Gom?s <rgomes@octobus.net> parents: 46711 diff changeset	3 # Copyright 2005-2010 Olivia Mackall <olivia@selenic.com>
11581 4530b3307fb9 protocol: introduce wireproto.py Matt Mackall <mpm@selenic.com> parents: diff changeset	4 #
4530b3307fb9 protocol: introduce wireproto.py Matt Mackall <mpm@selenic.com> parents: diff changeset	5 # This software may be used and distributed according to the terms of the
4530b3307fb9 protocol: introduce wireproto.py Matt Mackall <mpm@selenic.com> parents: diff changeset	6 # GNU General Public License version 2 or any later version.
4530b3307fb9 protocol: introduce wireproto.py Matt Mackall <mpm@selenic.com> parents: diff changeset	7
25993 0851678be71b wireproto: use absolute_import Gregory Szorc <gregory.szorc@gmail.com> parents: 25913 diff changeset	8
37630 e1b32dc4646c wireproto: implement command executor interface for version 1 peers Gregory Szorc <gregory.szorc@gmail.com> parents: 37615 diff changeset	9 import sys
37631 2f626233859b wireproto: implement batching on peer executor interface Gregory Szorc <gregory.szorc@gmail.com> parents: 37630 diff changeset	10 import weakref
11581 4530b3307fb9 protocol: introduce wireproto.py Matt Mackall <mpm@selenic.com> parents: diff changeset	11
48926 a0da5075bca3 cleanup: directly use concurrent.futures instead of via pycompat Augie Fackler <augie@google.com> parents: 47914 diff changeset	12 from concurrent import futures
25993 0851678be71b wireproto: use absolute_import Gregory Szorc <gregory.szorc@gmail.com> parents: 25913 diff changeset	13 from .i18n import _
43076 2372284d9457 formatting: blacken the codebase Augie Fackler <augie@google.com> parents: 42824 diff changeset	14 from .node import bin
43089 c59eb1560c44 py3: manually import getattr where it is needed Gregory Szorc <gregory.szorc@gmail.com> parents: 43087 diff changeset	15 from .pycompat import (
c59eb1560c44 py3: manually import getattr where it is needed Gregory Szorc <gregory.szorc@gmail.com> parents: 43087 diff changeset	16 getattr,
c59eb1560c44 py3: manually import getattr where it is needed Gregory Szorc <gregory.szorc@gmail.com> parents: 43087 diff changeset	17 setattr,
c59eb1560c44 py3: manually import getattr where it is needed Gregory Szorc <gregory.szorc@gmail.com> parents: 43087 diff changeset	18 )
25993 0851678be71b wireproto: use absolute_import Gregory Szorc <gregory.szorc@gmail.com> parents: 25913 diff changeset	19 from . import (
0851678be71b wireproto: use absolute_import Gregory Szorc <gregory.szorc@gmail.com> parents: 25913 diff changeset	20 bundle2,
0851678be71b wireproto: use absolute_import Gregory Szorc <gregory.szorc@gmail.com> parents: 25913 diff changeset	21 changegroup as changegroupmod,
0851678be71b wireproto: use absolute_import Gregory Szorc <gregory.szorc@gmail.com> parents: 25913 diff changeset	22 encoding,
0851678be71b wireproto: use absolute_import Gregory Szorc <gregory.szorc@gmail.com> parents: 25913 diff changeset	23 error,
0851678be71b wireproto: use absolute_import Gregory Szorc <gregory.szorc@gmail.com> parents: 25913 diff changeset	24 pushkey as pushkeymod,
30944 48dea083f66d py3: convert the mode argument of os.fdopen to unicodes (1 of 2) Pulkit Goyal <7895pulkit@gmail.com> parents: 30876 diff changeset	25 pycompat,
25993 0851678be71b wireproto: use absolute_import Gregory Szorc <gregory.szorc@gmail.com> parents: 25913 diff changeset	26 util,
36111 cd6ab329c5c7 wireprototypes: move wire protocol response types to new module Gregory Szorc <gregory.szorc@gmail.com> parents: 36109 diff changeset	27 wireprototypes,
25993 0851678be71b wireproto: use absolute_import Gregory Szorc <gregory.szorc@gmail.com> parents: 25913 diff changeset	28 )
42823 268662aac075 interfaces: create a new folder for interfaces and move repository.py in it Pulkit Goyal <pulkit@yandex-team.ru> parents: 41058 diff changeset	29 from .interfaces import (
268662aac075 interfaces: create a new folder for interfaces and move repository.py in it Pulkit Goyal <pulkit@yandex-team.ru> parents: 41058 diff changeset	30 repository,
42824 2c4f656c8e9f interfaceutil: move to interfaces/ Pulkit Goyal <pulkit@yandex-team.ru> parents: 42823 diff changeset	31 util as interfaceutil,
37810 856f381ad74b interfaceutil: module to stub out zope.interface Gregory Szorc <gregory.szorc@gmail.com> parents: 37669 diff changeset	32 )
44060 a61287a95dc3 core: migrate uses of hashlib.sha1 to hashutil.sha1 Augie Fackler <augie@google.com> parents: 43554 diff changeset	33 from .utils import hashutil
20903 8d477543882b wireproto: introduce an abstractserverproto class Pierre-Yves David <pierre-yves.david@fb.com> parents: 20902 diff changeset	34
28883 032c4c2f802a pycompat: switch to util.urlreq/util.urlerr for py3 compat timeless <timeless@mozdev.org> parents: 28666 diff changeset	35 urlreq = util.urlreq
032c4c2f802a pycompat: switch to util.urlreq/util.urlerr for py3 compat timeless <timeless@mozdev.org> parents: 28666 diff changeset	36
43076 2372284d9457 formatting: blacken the codebase Augie Fackler <augie@google.com> parents: 42824 diff changeset	37
47914 c424ff4807e6 wireprotov1peer: update all rpcs to use the new batchable scheme Valentin Gatien-Baron <valentin.gatienbaron@gmail.com> parents: 47913 diff changeset	38 def batchable(f):
45957 89a2afe31e82 formating: upgrade to black 20.8b1 Augie Fackler <raf@durin42.com> parents: 44060 diff changeset	39 """annotation for batchable methods
37615 f3dc8239e3a9 peer: scatter module to the wind (API) Gregory Szorc <gregory.szorc@gmail.com> parents: 37614 diff changeset	40
f3dc8239e3a9 peer: scatter module to the wind (API) Gregory Szorc <gregory.szorc@gmail.com> parents: 37614 diff changeset	41 Such methods must implement a coroutine as follows:
f3dc8239e3a9 peer: scatter module to the wind (API) Gregory Szorc <gregory.szorc@gmail.com> parents: 37614 diff changeset	42
f3dc8239e3a9 peer: scatter module to the wind (API) Gregory Szorc <gregory.szorc@gmail.com> parents: 37614 diff changeset	43 @batchable
f3dc8239e3a9 peer: scatter module to the wind (API) Gregory Szorc <gregory.szorc@gmail.com> parents: 37614 diff changeset	44 def sample(self, one, two=None):
f3dc8239e3a9 peer: scatter module to the wind (API) Gregory Szorc <gregory.szorc@gmail.com> parents: 37614 diff changeset	45 # Build list of encoded arguments suitable for your wire protocol:
46539 05dd091dfa6a wireprotopeer: clarify some variable names now that we allow snake_case Martin von Zweigbergk <martinvonz@google.com> parents: 45957 diff changeset	46 encoded_args = [('one', encode(one),), ('two', encode(two),)]
47913 cdad6560e832 wireprotov1peer: simplify the way batchable rpcs are defined Valentin Gatien-Baron <valentin.gatienbaron@gmail.com> parents: 46819 diff changeset	47 # Return it, along with a function that will receive the result
cdad6560e832 wireprotov1peer: simplify the way batchable rpcs are defined Valentin Gatien-Baron <valentin.gatienbaron@gmail.com> parents: 46819 diff changeset	48 # from the batched request.
cdad6560e832 wireprotov1peer: simplify the way batchable rpcs are defined Valentin Gatien-Baron <valentin.gatienbaron@gmail.com> parents: 46819 diff changeset	49 return encoded_args, decode
37615 f3dc8239e3a9 peer: scatter module to the wind (API) Gregory Szorc <gregory.szorc@gmail.com> parents: 37614 diff changeset	50
f3dc8239e3a9 peer: scatter module to the wind (API) Gregory Szorc <gregory.szorc@gmail.com> parents: 37614 diff changeset	51 The decorator returns a function which wraps this coroutine as a plain
f3dc8239e3a9 peer: scatter module to the wind (API) Gregory Szorc <gregory.szorc@gmail.com> parents: 37614 diff changeset	52 method, but adds the original method as an attribute called "batchable",
f3dc8239e3a9 peer: scatter module to the wind (API) Gregory Szorc <gregory.szorc@gmail.com> parents: 37614 diff changeset	53 which is used by remotebatch to split the call into separate encoding and
f3dc8239e3a9 peer: scatter module to the wind (API) Gregory Szorc <gregory.szorc@gmail.com> parents: 37614 diff changeset	54 decoding phases.
45957 89a2afe31e82 formating: upgrade to black 20.8b1 Augie Fackler <raf@durin42.com> parents: 44060 diff changeset	55 """
43076 2372284d9457 formatting: blacken the codebase Augie Fackler <augie@google.com> parents: 42824 diff changeset	56
37615 f3dc8239e3a9 peer: scatter module to the wind (API) Gregory Szorc <gregory.szorc@gmail.com> parents: 37614 diff changeset	57 def plain(args, *opts):
47913 cdad6560e832 wireprotov1peer: simplify the way batchable rpcs are defined Valentin Gatien-Baron <valentin.gatienbaron@gmail.com> parents: 46819 diff changeset	58 encoded_args_or_res, decode = f(args, *opts)
cdad6560e832 wireprotov1peer: simplify the way batchable rpcs are defined Valentin Gatien-Baron <valentin.gatienbaron@gmail.com> parents: 46819 diff changeset	59 if not decode:
46539 05dd091dfa6a wireprotopeer: clarify some variable names now that we allow snake_case Martin von Zweigbergk <martinvonz@google.com> parents: 45957 diff changeset	60 return encoded_args_or_res # a local result in this case
37615 f3dc8239e3a9 peer: scatter module to the wind (API) Gregory Szorc <gregory.szorc@gmail.com> parents: 37614 diff changeset	61 self = args[0]
f3dc8239e3a9 peer: scatter module to the wind (API) Gregory Szorc <gregory.szorc@gmail.com> parents: 37614 diff changeset	62 cmd = pycompat.bytesurl(f.__name__) # ensure cmd is ascii bytestr
47913 cdad6560e832 wireprotov1peer: simplify the way batchable rpcs are defined Valentin Gatien-Baron <valentin.gatienbaron@gmail.com> parents: 46819 diff changeset	63 encoded_res = self._submitone(cmd, encoded_args_or_res)
cdad6560e832 wireprotov1peer: simplify the way batchable rpcs are defined Valentin Gatien-Baron <valentin.gatienbaron@gmail.com> parents: 46819 diff changeset	64 return decode(encoded_res)
43076 2372284d9457 formatting: blacken the codebase Augie Fackler <augie@google.com> parents: 42824 diff changeset	65
37615 f3dc8239e3a9 peer: scatter module to the wind (API) Gregory Szorc <gregory.szorc@gmail.com> parents: 37614 diff changeset	66 setattr(plain, 'batchable', f)
39604 f15a587d2dfc wireprotov1peer: forward __name__ of wrapped method in batchable decorator Augie Fackler <raf@durin42.com> parents: 38823 diff changeset	67 setattr(plain, '__name__', f.__name__)
37615 f3dc8239e3a9 peer: scatter module to the wind (API) Gregory Szorc <gregory.szorc@gmail.com> parents: 37614 diff changeset	68 return plain
f3dc8239e3a9 peer: scatter module to the wind (API) Gregory Szorc <gregory.szorc@gmail.com> parents: 37614 diff changeset	69
43076 2372284d9457 formatting: blacken the codebase Augie Fackler <augie@google.com> parents: 42824 diff changeset	70
29745 bb04f96df51c wireproto: consolidate code for obtaining "cmds" argument value Gregory Szorc <gregory.szorc@gmail.com> parents: 29719 diff changeset	71 def encodebatchcmds(req):
bb04f96df51c wireproto: consolidate code for obtaining "cmds" argument value Gregory Szorc <gregory.szorc@gmail.com> parents: 29719 diff changeset	72 """Return a ``cmds`` argument value for the ``batch`` command."""
37612 5e71dea79aae wireproto: move value encoding functions to wireprototypes (API) Gregory Szorc <gregory.szorc@gmail.com> parents: 37546 diff changeset	73 escapearg = wireprototypes.escapebatcharg
5e71dea79aae wireproto: move value encoding functions to wireprototypes (API) Gregory Szorc <gregory.szorc@gmail.com> parents: 37546 diff changeset	74
29745 bb04f96df51c wireproto: consolidate code for obtaining "cmds" argument value Gregory Szorc <gregory.szorc@gmail.com> parents: 29719 diff changeset	75 cmds = []
bb04f96df51c wireproto: consolidate code for obtaining "cmds" argument value Gregory Szorc <gregory.szorc@gmail.com> parents: 29719 diff changeset	76 for op, argsdict in req:
29746 62e2e048d068 wireproto: unescape argument names in batch command (BC) Gregory Szorc <gregory.szorc@gmail.com> parents: 29745 diff changeset	77 # Old servers didn't properly unescape argument names. So prevent
62e2e048d068 wireproto: unescape argument names in batch command (BC) Gregory Szorc <gregory.szorc@gmail.com> parents: 29745 diff changeset	78 # the sending of argument names that may not be decoded properly by
62e2e048d068 wireproto: unescape argument names in batch command (BC) Gregory Szorc <gregory.szorc@gmail.com> parents: 29745 diff changeset	79 # servers.
62e2e048d068 wireproto: unescape argument names in batch command (BC) Gregory Szorc <gregory.szorc@gmail.com> parents: 29745 diff changeset	80 assert all(escapearg(k) == k for k in argsdict)
62e2e048d068 wireproto: unescape argument names in batch command (BC) Gregory Szorc <gregory.szorc@gmail.com> parents: 29745 diff changeset	81
43077 687b865b95ad formatting: byteify all mercurial/ and hgext/ string literals Augie Fackler <augie@google.com> parents: 43076 diff changeset	82 args = b','.join(
49004 f254fc73d956 global: bulk replace simple pycompat.iteritems(x) with x.items() Gregory Szorc <gregory.szorc@gmail.com> parents: 48966 diff changeset	83 b'%s=%s' % (escapearg(k), escapearg(v)) for k, v in argsdict.items()
43076 2372284d9457 formatting: blacken the codebase Augie Fackler <augie@google.com> parents: 42824 diff changeset	84 )
43077 687b865b95ad formatting: byteify all mercurial/ and hgext/ string literals Augie Fackler <augie@google.com> parents: 43076 diff changeset	85 cmds.append(b'%s %s' % (op, args))
29745 bb04f96df51c wireproto: consolidate code for obtaining "cmds" argument value Gregory Szorc <gregory.szorc@gmail.com> parents: 29719 diff changeset	86
43077 687b865b95ad formatting: byteify all mercurial/ and hgext/ string literals Augie Fackler <augie@google.com> parents: 43076 diff changeset	87 return b';'.join(cmds)
29745 bb04f96df51c wireproto: consolidate code for obtaining "cmds" argument value Gregory Szorc <gregory.szorc@gmail.com> parents: 29719 diff changeset	88
43076 2372284d9457 formatting: blacken the codebase Augie Fackler <augie@google.com> parents: 42824 diff changeset	89
48926 a0da5075bca3 cleanup: directly use concurrent.futures instead of via pycompat Augie Fackler <augie@google.com> parents: 47914 diff changeset	90 class unsentfuture(futures.Future):
37631 2f626233859b wireproto: implement batching on peer executor interface Gregory Szorc <gregory.szorc@gmail.com> parents: 37630 diff changeset	91 """A Future variation to represent an unsent command.
2f626233859b wireproto: implement batching on peer executor interface Gregory Szorc <gregory.szorc@gmail.com> parents: 37630 diff changeset	92
2f626233859b wireproto: implement batching on peer executor interface Gregory Szorc <gregory.szorc@gmail.com> parents: 37630 diff changeset	93 Because we buffer commands and don't submit them immediately, calling
2f626233859b wireproto: implement batching on peer executor interface Gregory Szorc <gregory.szorc@gmail.com> parents: 37630 diff changeset	94 ``result()`` on an unsent future could deadlock. Futures for buffered
2f626233859b wireproto: implement batching on peer executor interface Gregory Szorc <gregory.szorc@gmail.com> parents: 37630 diff changeset	95 commands are represented by this type, which wraps ``result()`` to
2f626233859b wireproto: implement batching on peer executor interface Gregory Szorc <gregory.szorc@gmail.com> parents: 37630 diff changeset	96 call ``sendcommands()``.
2f626233859b wireproto: implement batching on peer executor interface Gregory Szorc <gregory.szorc@gmail.com> parents: 37630 diff changeset	97 """
2f626233859b wireproto: implement batching on peer executor interface Gregory Szorc <gregory.szorc@gmail.com> parents: 37630 diff changeset	98
2f626233859b wireproto: implement batching on peer executor interface Gregory Szorc <gregory.szorc@gmail.com> parents: 37630 diff changeset	99 def result(self, timeout=None):
2f626233859b wireproto: implement batching on peer executor interface Gregory Szorc <gregory.szorc@gmail.com> parents: 37630 diff changeset	100 if self.done():
48926 a0da5075bca3 cleanup: directly use concurrent.futures instead of via pycompat Augie Fackler <augie@google.com> parents: 47914 diff changeset	101 return futures.Future.result(self, timeout)
37631 2f626233859b wireproto: implement batching on peer executor interface Gregory Szorc <gregory.szorc@gmail.com> parents: 37630 diff changeset	102
2f626233859b wireproto: implement batching on peer executor interface Gregory Szorc <gregory.szorc@gmail.com> parents: 37630 diff changeset	103 self._peerexecutor.sendcommands()
2f626233859b wireproto: implement batching on peer executor interface Gregory Szorc <gregory.szorc@gmail.com> parents: 37630 diff changeset	104
2f626233859b wireproto: implement batching on peer executor interface Gregory Szorc <gregory.szorc@gmail.com> parents: 37630 diff changeset	105 # This looks like it will infinitely recurse. However,
2f626233859b wireproto: implement batching on peer executor interface Gregory Szorc <gregory.szorc@gmail.com> parents: 37630 diff changeset	106 # sendcommands() should modify __class__. This call serves as a check
2f626233859b wireproto: implement batching on peer executor interface Gregory Szorc <gregory.szorc@gmail.com> parents: 37630 diff changeset	107 # on that.
2f626233859b wireproto: implement batching on peer executor interface Gregory Szorc <gregory.szorc@gmail.com> parents: 37630 diff changeset	108 return self.result(timeout)
2f626233859b wireproto: implement batching on peer executor interface Gregory Szorc <gregory.szorc@gmail.com> parents: 37630 diff changeset	109
43076 2372284d9457 formatting: blacken the codebase Augie Fackler <augie@google.com> parents: 42824 diff changeset	110
37810 856f381ad74b interfaceutil: module to stub out zope.interface Gregory Szorc <gregory.szorc@gmail.com> parents: 37669 diff changeset	111 @interfaceutil.implementer(repository.ipeercommandexecutor)
49037 642e31cb55f0 py3: use class X: instead of class X(object): Gregory Szorc <gregory.szorc@gmail.com> parents: 49004 diff changeset	112 class peerexecutor:
37630 e1b32dc4646c wireproto: implement command executor interface for version 1 peers Gregory Szorc <gregory.szorc@gmail.com> parents: 37615 diff changeset	113 def __init__(self, peer):
e1b32dc4646c wireproto: implement command executor interface for version 1 peers Gregory Szorc <gregory.szorc@gmail.com> parents: 37615 diff changeset	114 self._peer = peer
e1b32dc4646c wireproto: implement command executor interface for version 1 peers Gregory Szorc <gregory.szorc@gmail.com> parents: 37615 diff changeset	115 self._sent = False
e1b32dc4646c wireproto: implement command executor interface for version 1 peers Gregory Szorc <gregory.szorc@gmail.com> parents: 37615 diff changeset	116 self._closed = False
e1b32dc4646c wireproto: implement command executor interface for version 1 peers Gregory Szorc <gregory.szorc@gmail.com> parents: 37615 diff changeset	117 self._calls = []
37631 2f626233859b wireproto: implement batching on peer executor interface Gregory Szorc <gregory.szorc@gmail.com> parents: 37630 diff changeset	118 self._futures = weakref.WeakSet()
2f626233859b wireproto: implement batching on peer executor interface Gregory Szorc <gregory.szorc@gmail.com> parents: 37630 diff changeset	119 self._responseexecutor = None
2f626233859b wireproto: implement batching on peer executor interface Gregory Szorc <gregory.szorc@gmail.com> parents: 37630 diff changeset	120 self._responsef = None
37630 e1b32dc4646c wireproto: implement command executor interface for version 1 peers Gregory Szorc <gregory.szorc@gmail.com> parents: 37615 diff changeset	121
e1b32dc4646c wireproto: implement command executor interface for version 1 peers Gregory Szorc <gregory.szorc@gmail.com> parents: 37615 diff changeset	122 def __enter__(self):
e1b32dc4646c wireproto: implement command executor interface for version 1 peers Gregory Szorc <gregory.szorc@gmail.com> parents: 37615 diff changeset	123 return self
e1b32dc4646c wireproto: implement command executor interface for version 1 peers Gregory Szorc <gregory.szorc@gmail.com> parents: 37615 diff changeset	124
e1b32dc4646c wireproto: implement command executor interface for version 1 peers Gregory Szorc <gregory.szorc@gmail.com> parents: 37615 diff changeset	125 def __exit__(self, exctype, excvalee, exctb):
e1b32dc4646c wireproto: implement command executor interface for version 1 peers Gregory Szorc <gregory.szorc@gmail.com> parents: 37615 diff changeset	126 self.close()
e1b32dc4646c wireproto: implement command executor interface for version 1 peers Gregory Szorc <gregory.szorc@gmail.com> parents: 37615 diff changeset	127
e1b32dc4646c wireproto: implement command executor interface for version 1 peers Gregory Szorc <gregory.szorc@gmail.com> parents: 37615 diff changeset	128 def callcommand(self, command, args):
e1b32dc4646c wireproto: implement command executor interface for version 1 peers Gregory Szorc <gregory.szorc@gmail.com> parents: 37615 diff changeset	129 if self._sent:
43076 2372284d9457 formatting: blacken the codebase Augie Fackler <augie@google.com> parents: 42824 diff changeset	130 raise error.ProgrammingError(
43117 8ff1ecfadcd1 cleanup: join string literals that are already on one line Martin von Zweigbergk <martinvonz@google.com> parents: 43106 diff changeset	131 b'callcommand() cannot be used after commands are sent'
43076 2372284d9457 formatting: blacken the codebase Augie Fackler <augie@google.com> parents: 42824 diff changeset	132 )
37630 e1b32dc4646c wireproto: implement command executor interface for version 1 peers Gregory Szorc <gregory.szorc@gmail.com> parents: 37615 diff changeset	133
e1b32dc4646c wireproto: implement command executor interface for version 1 peers Gregory Szorc <gregory.szorc@gmail.com> parents: 37615 diff changeset	134 if self._closed:
43076 2372284d9457 formatting: blacken the codebase Augie Fackler <augie@google.com> parents: 42824 diff changeset	135 raise error.ProgrammingError(
43117 8ff1ecfadcd1 cleanup: join string literals that are already on one line Martin von Zweigbergk <martinvonz@google.com> parents: 43106 diff changeset	136 b'callcommand() cannot be used after close()'
43076 2372284d9457 formatting: blacken the codebase Augie Fackler <augie@google.com> parents: 42824 diff changeset	137 )
37630 e1b32dc4646c wireproto: implement command executor interface for version 1 peers Gregory Szorc <gregory.szorc@gmail.com> parents: 37615 diff changeset	138
e1b32dc4646c wireproto: implement command executor interface for version 1 peers Gregory Szorc <gregory.szorc@gmail.com> parents: 37615 diff changeset	139 # Commands are dispatched through methods on the peer.
e1b32dc4646c wireproto: implement command executor interface for version 1 peers Gregory Szorc <gregory.szorc@gmail.com> parents: 37615 diff changeset	140 fn = getattr(self._peer, pycompat.sysstr(command), None)
e1b32dc4646c wireproto: implement command executor interface for version 1 peers Gregory Szorc <gregory.szorc@gmail.com> parents: 37615 diff changeset	141
e1b32dc4646c wireproto: implement command executor interface for version 1 peers Gregory Szorc <gregory.szorc@gmail.com> parents: 37615 diff changeset	142 if not fn:
e1b32dc4646c wireproto: implement command executor interface for version 1 peers Gregory Szorc <gregory.szorc@gmail.com> parents: 37615 diff changeset	143 raise error.ProgrammingError(
43077 687b865b95ad formatting: byteify all mercurial/ and hgext/ string literals Augie Fackler <augie@google.com> parents: 43076 diff changeset	144 b'cannot call command %s: method of same name not available '
687b865b95ad formatting: byteify all mercurial/ and hgext/ string literals Augie Fackler <augie@google.com> parents: 43076 diff changeset	145 b'on peer' % command
43076 2372284d9457 formatting: blacken the codebase Augie Fackler <augie@google.com> parents: 42824 diff changeset	146 )
37630 e1b32dc4646c wireproto: implement command executor interface for version 1 peers Gregory Szorc <gregory.szorc@gmail.com> parents: 37615 diff changeset	147
e1b32dc4646c wireproto: implement command executor interface for version 1 peers Gregory Szorc <gregory.szorc@gmail.com> parents: 37615 diff changeset	148 # Commands are either batchable or they aren't. If a command
e1b32dc4646c wireproto: implement command executor interface for version 1 peers Gregory Szorc <gregory.szorc@gmail.com> parents: 37615 diff changeset	149 # isn't batchable, we send it immediately because the executor
e1b32dc4646c wireproto: implement command executor interface for version 1 peers Gregory Szorc <gregory.szorc@gmail.com> parents: 37615 diff changeset	150 # can no longer accept new commands after a non-batchable command.
37631 2f626233859b wireproto: implement batching on peer executor interface Gregory Szorc <gregory.szorc@gmail.com> parents: 37630 diff changeset	151 # If a command is batchable, we queue it for later. But we have
2f626233859b wireproto: implement batching on peer executor interface Gregory Szorc <gregory.szorc@gmail.com> parents: 37630 diff changeset	152 # to account for the case of a non-batchable command arriving after
2f626233859b wireproto: implement batching on peer executor interface Gregory Szorc <gregory.szorc@gmail.com> parents: 37630 diff changeset	153 # a batchable one and refuse to service it.
2f626233859b wireproto: implement batching on peer executor interface Gregory Szorc <gregory.szorc@gmail.com> parents: 37630 diff changeset	154
2f626233859b wireproto: implement batching on peer executor interface Gregory Szorc <gregory.szorc@gmail.com> parents: 37630 diff changeset	155 def addcall():
48926 a0da5075bca3 cleanup: directly use concurrent.futures instead of via pycompat Augie Fackler <augie@google.com> parents: 47914 diff changeset	156 f = futures.Future()
37631 2f626233859b wireproto: implement batching on peer executor interface Gregory Szorc <gregory.szorc@gmail.com> parents: 37630 diff changeset	157 self._futures.add(f)
2f626233859b wireproto: implement batching on peer executor interface Gregory Szorc <gregory.szorc@gmail.com> parents: 37630 diff changeset	158 self._calls.append((command, args, fn, f))
2f626233859b wireproto: implement batching on peer executor interface Gregory Szorc <gregory.szorc@gmail.com> parents: 37630 diff changeset	159 return f
37630 e1b32dc4646c wireproto: implement command executor interface for version 1 peers Gregory Szorc <gregory.szorc@gmail.com> parents: 37615 diff changeset	160
e1b32dc4646c wireproto: implement command executor interface for version 1 peers Gregory Szorc <gregory.szorc@gmail.com> parents: 37615 diff changeset	161 if getattr(fn, 'batchable', False):
37631 2f626233859b wireproto: implement batching on peer executor interface Gregory Szorc <gregory.szorc@gmail.com> parents: 37630 diff changeset	162 f = addcall()
2f626233859b wireproto: implement batching on peer executor interface Gregory Szorc <gregory.szorc@gmail.com> parents: 37630 diff changeset	163
2f626233859b wireproto: implement batching on peer executor interface Gregory Szorc <gregory.szorc@gmail.com> parents: 37630 diff changeset	164 # But since we don't issue it immediately, we wrap its result()
2f626233859b wireproto: implement batching on peer executor interface Gregory Szorc <gregory.szorc@gmail.com> parents: 37630 diff changeset	165 # to trigger sending so we avoid deadlocks.
2f626233859b wireproto: implement batching on peer executor interface Gregory Szorc <gregory.szorc@gmail.com> parents: 37630 diff changeset	166 f.__class__ = unsentfuture
2f626233859b wireproto: implement batching on peer executor interface Gregory Szorc <gregory.szorc@gmail.com> parents: 37630 diff changeset	167 f._peerexecutor = self
37630 e1b32dc4646c wireproto: implement command executor interface for version 1 peers Gregory Szorc <gregory.szorc@gmail.com> parents: 37615 diff changeset	168 else:
e1b32dc4646c wireproto: implement command executor interface for version 1 peers Gregory Szorc <gregory.szorc@gmail.com> parents: 37615 diff changeset	169 if self._calls:
e1b32dc4646c wireproto: implement command executor interface for version 1 peers Gregory Szorc <gregory.szorc@gmail.com> parents: 37615 diff changeset	170 raise error.ProgrammingError(
43077 687b865b95ad formatting: byteify all mercurial/ and hgext/ string literals Augie Fackler <augie@google.com> parents: 43076 diff changeset	171 b'%s is not batchable and cannot be called on a command '
687b865b95ad formatting: byteify all mercurial/ and hgext/ string literals Augie Fackler <augie@google.com> parents: 43076 diff changeset	172 b'executor along with other commands' % command
43076 2372284d9457 formatting: blacken the codebase Augie Fackler <augie@google.com> parents: 42824 diff changeset	173 )
37630 e1b32dc4646c wireproto: implement command executor interface for version 1 peers Gregory Szorc <gregory.szorc@gmail.com> parents: 37615 diff changeset	174
37631 2f626233859b wireproto: implement batching on peer executor interface Gregory Szorc <gregory.szorc@gmail.com> parents: 37630 diff changeset	175 f = addcall()
2f626233859b wireproto: implement batching on peer executor interface Gregory Szorc <gregory.szorc@gmail.com> parents: 37630 diff changeset	176
2f626233859b wireproto: implement batching on peer executor interface Gregory Szorc <gregory.szorc@gmail.com> parents: 37630 diff changeset	177 # Non-batchable commands can never coexist with another command
2f626233859b wireproto: implement batching on peer executor interface Gregory Szorc <gregory.szorc@gmail.com> parents: 37630 diff changeset	178 # in this executor. So send the command immediately.
2f626233859b wireproto: implement batching on peer executor interface Gregory Szorc <gregory.szorc@gmail.com> parents: 37630 diff changeset	179 self.sendcommands()
2f626233859b wireproto: implement batching on peer executor interface Gregory Szorc <gregory.szorc@gmail.com> parents: 37630 diff changeset	180
37630 e1b32dc4646c wireproto: implement command executor interface for version 1 peers Gregory Szorc <gregory.szorc@gmail.com> parents: 37615 diff changeset	181 return f
e1b32dc4646c wireproto: implement command executor interface for version 1 peers Gregory Szorc <gregory.szorc@gmail.com> parents: 37615 diff changeset	182
e1b32dc4646c wireproto: implement command executor interface for version 1 peers Gregory Szorc <gregory.szorc@gmail.com> parents: 37615 diff changeset	183 def sendcommands(self):
e1b32dc4646c wireproto: implement command executor interface for version 1 peers Gregory Szorc <gregory.szorc@gmail.com> parents: 37615 diff changeset	184 if self._sent:
e1b32dc4646c wireproto: implement command executor interface for version 1 peers Gregory Szorc <gregory.szorc@gmail.com> parents: 37615 diff changeset	185 return
e1b32dc4646c wireproto: implement command executor interface for version 1 peers Gregory Szorc <gregory.szorc@gmail.com> parents: 37615 diff changeset	186
e1b32dc4646c wireproto: implement command executor interface for version 1 peers Gregory Szorc <gregory.szorc@gmail.com> parents: 37615 diff changeset	187 if not self._calls:
e1b32dc4646c wireproto: implement command executor interface for version 1 peers Gregory Szorc <gregory.szorc@gmail.com> parents: 37615 diff changeset	188 return
e1b32dc4646c wireproto: implement command executor interface for version 1 peers Gregory Szorc <gregory.szorc@gmail.com> parents: 37615 diff changeset	189
e1b32dc4646c wireproto: implement command executor interface for version 1 peers Gregory Szorc <gregory.szorc@gmail.com> parents: 37615 diff changeset	190 self._sent = True
e1b32dc4646c wireproto: implement command executor interface for version 1 peers Gregory Szorc <gregory.szorc@gmail.com> parents: 37615 diff changeset	191
37631 2f626233859b wireproto: implement batching on peer executor interface Gregory Szorc <gregory.szorc@gmail.com> parents: 37630 diff changeset	192 # Unhack any future types so caller seens a clean type and to break
2f626233859b wireproto: implement batching on peer executor interface Gregory Szorc <gregory.szorc@gmail.com> parents: 37630 diff changeset	193 # cycle between us and futures.
2f626233859b wireproto: implement batching on peer executor interface Gregory Szorc <gregory.szorc@gmail.com> parents: 37630 diff changeset	194 for f in self._futures:
2f626233859b wireproto: implement batching on peer executor interface Gregory Szorc <gregory.szorc@gmail.com> parents: 37630 diff changeset	195 if isinstance(f, unsentfuture):
48926 a0da5075bca3 cleanup: directly use concurrent.futures instead of via pycompat Augie Fackler <augie@google.com> parents: 47914 diff changeset	196 f.__class__ = futures.Future
37631 2f626233859b wireproto: implement batching on peer executor interface Gregory Szorc <gregory.szorc@gmail.com> parents: 37630 diff changeset	197 f._peerexecutor = None
2f626233859b wireproto: implement batching on peer executor interface Gregory Szorc <gregory.szorc@gmail.com> parents: 37630 diff changeset	198
37630 e1b32dc4646c wireproto: implement command executor interface for version 1 peers Gregory Szorc <gregory.szorc@gmail.com> parents: 37615 diff changeset	199 calls = self._calls
e1b32dc4646c wireproto: implement command executor interface for version 1 peers Gregory Szorc <gregory.szorc@gmail.com> parents: 37615 diff changeset	200 # Mainly to destroy references to futures.
e1b32dc4646c wireproto: implement command executor interface for version 1 peers Gregory Szorc <gregory.szorc@gmail.com> parents: 37615 diff changeset	201 self._calls = None
e1b32dc4646c wireproto: implement command executor interface for version 1 peers Gregory Szorc <gregory.szorc@gmail.com> parents: 37615 diff changeset	202
37631 2f626233859b wireproto: implement batching on peer executor interface Gregory Szorc <gregory.szorc@gmail.com> parents: 37630 diff changeset	203 # Simple case of a single command. We call it synchronously.
37630 e1b32dc4646c wireproto: implement command executor interface for version 1 peers Gregory Szorc <gregory.szorc@gmail.com> parents: 37615 diff changeset	204 if len(calls) == 1:
e1b32dc4646c wireproto: implement command executor interface for version 1 peers Gregory Szorc <gregory.szorc@gmail.com> parents: 37615 diff changeset	205 command, args, fn, f = calls[0]
e1b32dc4646c wireproto: implement command executor interface for version 1 peers Gregory Szorc <gregory.szorc@gmail.com> parents: 37615 diff changeset	206
e1b32dc4646c wireproto: implement command executor interface for version 1 peers Gregory Szorc <gregory.szorc@gmail.com> parents: 37615 diff changeset	207 # Future was cancelled. Ignore it.
e1b32dc4646c wireproto: implement command executor interface for version 1 peers Gregory Szorc <gregory.szorc@gmail.com> parents: 37615 diff changeset	208 if not f.set_running_or_notify_cancel():
e1b32dc4646c wireproto: implement command executor interface for version 1 peers Gregory Szorc <gregory.szorc@gmail.com> parents: 37615 diff changeset	209 return
e1b32dc4646c wireproto: implement command executor interface for version 1 peers Gregory Szorc <gregory.szorc@gmail.com> parents: 37615 diff changeset	210
e1b32dc4646c wireproto: implement command executor interface for version 1 peers Gregory Szorc <gregory.szorc@gmail.com> parents: 37615 diff changeset	211 try:
e1b32dc4646c wireproto: implement command executor interface for version 1 peers Gregory Szorc <gregory.szorc@gmail.com> parents: 37615 diff changeset	212 result = fn(**pycompat.strkwargs(args))
e1b32dc4646c wireproto: implement command executor interface for version 1 peers Gregory Szorc <gregory.szorc@gmail.com> parents: 37615 diff changeset	213 except Exception:
37669 1cb54e6193a6 py3: paper over differences in future exception handling Augie Fackler <augie@google.com> parents: 37650 diff changeset	214 pycompat.future_set_exception_info(f, sys.exc_info()[1:])
37630 e1b32dc4646c wireproto: implement command executor interface for version 1 peers Gregory Szorc <gregory.szorc@gmail.com> parents: 37615 diff changeset	215 else:
e1b32dc4646c wireproto: implement command executor interface for version 1 peers Gregory Szorc <gregory.szorc@gmail.com> parents: 37615 diff changeset	216 f.set_result(result)
e1b32dc4646c wireproto: implement command executor interface for version 1 peers Gregory Szorc <gregory.szorc@gmail.com> parents: 37615 diff changeset	217
e1b32dc4646c wireproto: implement command executor interface for version 1 peers Gregory Szorc <gregory.szorc@gmail.com> parents: 37615 diff changeset	218 return
e1b32dc4646c wireproto: implement command executor interface for version 1 peers Gregory Szorc <gregory.szorc@gmail.com> parents: 37615 diff changeset	219
37631 2f626233859b wireproto: implement batching on peer executor interface Gregory Szorc <gregory.szorc@gmail.com> parents: 37630 diff changeset	220 # Batch commands are a bit harder. First, we have to deal with the
2f626233859b wireproto: implement batching on peer executor interface Gregory Szorc <gregory.szorc@gmail.com> parents: 37630 diff changeset	221 # @batchable coroutine. That's a bit annoying. Furthermore, we also
2f626233859b wireproto: implement batching on peer executor interface Gregory Szorc <gregory.szorc@gmail.com> parents: 37630 diff changeset	222 # need to preserve streaming. i.e. it should be possible for the
2f626233859b wireproto: implement batching on peer executor interface Gregory Szorc <gregory.szorc@gmail.com> parents: 37630 diff changeset	223 # futures to resolve as data is coming in off the wire without having
2f626233859b wireproto: implement batching on peer executor interface Gregory Szorc <gregory.szorc@gmail.com> parents: 37630 diff changeset	224 # to wait for the final byte of the final response. We do this by
2f626233859b wireproto: implement batching on peer executor interface Gregory Szorc <gregory.szorc@gmail.com> parents: 37630 diff changeset	225 # spinning up a thread to read the responses.
2f626233859b wireproto: implement batching on peer executor interface Gregory Szorc <gregory.szorc@gmail.com> parents: 37630 diff changeset	226
2f626233859b wireproto: implement batching on peer executor interface Gregory Szorc <gregory.szorc@gmail.com> parents: 37630 diff changeset	227 requests = []
2f626233859b wireproto: implement batching on peer executor interface Gregory Szorc <gregory.szorc@gmail.com> parents: 37630 diff changeset	228 states = []
2f626233859b wireproto: implement batching on peer executor interface Gregory Szorc <gregory.szorc@gmail.com> parents: 37630 diff changeset	229
2f626233859b wireproto: implement batching on peer executor interface Gregory Szorc <gregory.szorc@gmail.com> parents: 37630 diff changeset	230 for command, args, fn, f in calls:
2f626233859b wireproto: implement batching on peer executor interface Gregory Szorc <gregory.szorc@gmail.com> parents: 37630 diff changeset	231 # Future was cancelled. Ignore it.
2f626233859b wireproto: implement batching on peer executor interface Gregory Szorc <gregory.szorc@gmail.com> parents: 37630 diff changeset	232 if not f.set_running_or_notify_cancel():
2f626233859b wireproto: implement batching on peer executor interface Gregory Szorc <gregory.szorc@gmail.com> parents: 37630 diff changeset	233 continue
2f626233859b wireproto: implement batching on peer executor interface Gregory Szorc <gregory.szorc@gmail.com> parents: 37630 diff changeset	234
2f626233859b wireproto: implement batching on peer executor interface Gregory Szorc <gregory.szorc@gmail.com> parents: 37630 diff changeset	235 try:
47913 cdad6560e832 wireprotov1peer: simplify the way batchable rpcs are defined Valentin Gatien-Baron <valentin.gatienbaron@gmail.com> parents: 46819 diff changeset	236 encoded_args_or_res, decode = fn.batchable(
43076 2372284d9457 formatting: blacken the codebase Augie Fackler <augie@google.com> parents: 42824 diff changeset	237 fn.__self__, **pycompat.strkwargs(args)
2372284d9457 formatting: blacken the codebase Augie Fackler <augie@google.com> parents: 42824 diff changeset	238 )
37631 2f626233859b wireproto: implement batching on peer executor interface Gregory Szorc <gregory.szorc@gmail.com> parents: 37630 diff changeset	239 except Exception:
37669 1cb54e6193a6 py3: paper over differences in future exception handling Augie Fackler <augie@google.com> parents: 37650 diff changeset	240 pycompat.future_set_exception_info(f, sys.exc_info()[1:])
37631 2f626233859b wireproto: implement batching on peer executor interface Gregory Szorc <gregory.szorc@gmail.com> parents: 37630 diff changeset	241 return
2f626233859b wireproto: implement batching on peer executor interface Gregory Szorc <gregory.szorc@gmail.com> parents: 37630 diff changeset	242
47913 cdad6560e832 wireprotov1peer: simplify the way batchable rpcs are defined Valentin Gatien-Baron <valentin.gatienbaron@gmail.com> parents: 46819 diff changeset	243 if not decode:
46539 05dd091dfa6a wireprotopeer: clarify some variable names now that we allow snake_case Martin von Zweigbergk <martinvonz@google.com> parents: 45957 diff changeset	244 f.set_result(encoded_args_or_res)
41058 55e8da487b8a wireproto: in batch queries, support queries with immediate responses Valentin Gatien-Baron <valentin.gatienbaron@gmail.com> parents: 39604 diff changeset	245 else:
46539 05dd091dfa6a wireprotopeer: clarify some variable names now that we allow snake_case Martin von Zweigbergk <martinvonz@google.com> parents: 45957 diff changeset	246 requests.append((command, encoded_args_or_res))
47913 cdad6560e832 wireprotov1peer: simplify the way batchable rpcs are defined Valentin Gatien-Baron <valentin.gatienbaron@gmail.com> parents: 46819 diff changeset	247 states.append((command, f, batchable, decode))
37631 2f626233859b wireproto: implement batching on peer executor interface Gregory Szorc <gregory.szorc@gmail.com> parents: 37630 diff changeset	248
2f626233859b wireproto: implement batching on peer executor interface Gregory Szorc <gregory.szorc@gmail.com> parents: 37630 diff changeset	249 if not requests:
2f626233859b wireproto: implement batching on peer executor interface Gregory Szorc <gregory.szorc@gmail.com> parents: 37630 diff changeset	250 return
2f626233859b wireproto: implement batching on peer executor interface Gregory Szorc <gregory.szorc@gmail.com> parents: 37630 diff changeset	251
2f626233859b wireproto: implement batching on peer executor interface Gregory Szorc <gregory.szorc@gmail.com> parents: 37630 diff changeset	252 # This will emit responses in order they were executed.
2f626233859b wireproto: implement batching on peer executor interface Gregory Szorc <gregory.szorc@gmail.com> parents: 37630 diff changeset	253 wireresults = self._peer._submitbatch(requests)
2f626233859b wireproto: implement batching on peer executor interface Gregory Szorc <gregory.szorc@gmail.com> parents: 37630 diff changeset	254
2f626233859b wireproto: implement batching on peer executor interface Gregory Szorc <gregory.szorc@gmail.com> parents: 37630 diff changeset	255 # The use of a thread pool executor here is a bit weird for something
2f626233859b wireproto: implement batching on peer executor interface Gregory Szorc <gregory.szorc@gmail.com> parents: 37630 diff changeset	256 # that only spins up a single thread. However, thread management is
2f626233859b wireproto: implement batching on peer executor interface Gregory Szorc <gregory.szorc@gmail.com> parents: 37630 diff changeset	257 # hard and it is easy to encounter race conditions, deadlocks, etc.
2f626233859b wireproto: implement batching on peer executor interface Gregory Szorc <gregory.szorc@gmail.com> parents: 37630 diff changeset	258 # concurrent.futures already solves these problems and its thread pool
2f626233859b wireproto: implement batching on peer executor interface Gregory Szorc <gregory.szorc@gmail.com> parents: 37630 diff changeset	259 # executor has minimal overhead. So we use it.
48926 a0da5075bca3 cleanup: directly use concurrent.futures instead of via pycompat Augie Fackler <augie@google.com> parents: 47914 diff changeset	260 self._responseexecutor = futures.ThreadPoolExecutor(1)
43076 2372284d9457 formatting: blacken the codebase Augie Fackler <augie@google.com> parents: 42824 diff changeset	261 self._responsef = self._responseexecutor.submit(
2372284d9457 formatting: blacken the codebase Augie Fackler <augie@google.com> parents: 42824 diff changeset	262 self._readbatchresponse, states, wireresults
2372284d9457 formatting: blacken the codebase Augie Fackler <augie@google.com> parents: 42824 diff changeset	263 )
37630 e1b32dc4646c wireproto: implement command executor interface for version 1 peers Gregory Szorc <gregory.szorc@gmail.com> parents: 37615 diff changeset	264
e1b32dc4646c wireproto: implement command executor interface for version 1 peers Gregory Szorc <gregory.szorc@gmail.com> parents: 37615 diff changeset	265 def close(self):
e1b32dc4646c wireproto: implement command executor interface for version 1 peers Gregory Szorc <gregory.szorc@gmail.com> parents: 37615 diff changeset	266 self.sendcommands()
e1b32dc4646c wireproto: implement command executor interface for version 1 peers Gregory Szorc <gregory.szorc@gmail.com> parents: 37615 diff changeset	267
37631 2f626233859b wireproto: implement batching on peer executor interface Gregory Szorc <gregory.szorc@gmail.com> parents: 37630 diff changeset	268 if self._closed:
2f626233859b wireproto: implement batching on peer executor interface Gregory Szorc <gregory.szorc@gmail.com> parents: 37630 diff changeset	269 return
2f626233859b wireproto: implement batching on peer executor interface Gregory Szorc <gregory.szorc@gmail.com> parents: 37630 diff changeset	270
37630 e1b32dc4646c wireproto: implement command executor interface for version 1 peers Gregory Szorc <gregory.szorc@gmail.com> parents: 37615 diff changeset	271 self._closed = True
e1b32dc4646c wireproto: implement command executor interface for version 1 peers Gregory Szorc <gregory.szorc@gmail.com> parents: 37615 diff changeset	272
37631 2f626233859b wireproto: implement batching on peer executor interface Gregory Szorc <gregory.szorc@gmail.com> parents: 37630 diff changeset	273 if not self._responsef:
2f626233859b wireproto: implement batching on peer executor interface Gregory Szorc <gregory.szorc@gmail.com> parents: 37630 diff changeset	274 return
2f626233859b wireproto: implement batching on peer executor interface Gregory Szorc <gregory.szorc@gmail.com> parents: 37630 diff changeset	275
2f626233859b wireproto: implement batching on peer executor interface Gregory Szorc <gregory.szorc@gmail.com> parents: 37630 diff changeset	276 # We need to wait on our in-flight response and then shut down the
2f626233859b wireproto: implement batching on peer executor interface Gregory Szorc <gregory.szorc@gmail.com> parents: 37630 diff changeset	277 # executor once we have a result.
2f626233859b wireproto: implement batching on peer executor interface Gregory Szorc <gregory.szorc@gmail.com> parents: 37630 diff changeset	278 try:
2f626233859b wireproto: implement batching on peer executor interface Gregory Szorc <gregory.szorc@gmail.com> parents: 37630 diff changeset	279 self._responsef.result()
2f626233859b wireproto: implement batching on peer executor interface Gregory Szorc <gregory.szorc@gmail.com> parents: 37630 diff changeset	280 finally:
2f626233859b wireproto: implement batching on peer executor interface Gregory Szorc <gregory.szorc@gmail.com> parents: 37630 diff changeset	281 self._responseexecutor.shutdown(wait=True)
2f626233859b wireproto: implement batching on peer executor interface Gregory Szorc <gregory.szorc@gmail.com> parents: 37630 diff changeset	282 self._responsef = None
2f626233859b wireproto: implement batching on peer executor interface Gregory Szorc <gregory.szorc@gmail.com> parents: 37630 diff changeset	283 self._responseexecutor = None
2f626233859b wireproto: implement batching on peer executor interface Gregory Szorc <gregory.szorc@gmail.com> parents: 37630 diff changeset	284
2f626233859b wireproto: implement batching on peer executor interface Gregory Szorc <gregory.szorc@gmail.com> parents: 37630 diff changeset	285 # If any of our futures are still in progress, mark them as
2f626233859b wireproto: implement batching on peer executor interface Gregory Szorc <gregory.szorc@gmail.com> parents: 37630 diff changeset	286 # errored. Otherwise a result() could wait indefinitely.
2f626233859b wireproto: implement batching on peer executor interface Gregory Szorc <gregory.szorc@gmail.com> parents: 37630 diff changeset	287 for f in self._futures:
2f626233859b wireproto: implement batching on peer executor interface Gregory Szorc <gregory.szorc@gmail.com> parents: 37630 diff changeset	288 if not f.done():
43076 2372284d9457 formatting: blacken the codebase Augie Fackler <augie@google.com> parents: 42824 diff changeset	289 f.set_exception(
2372284d9457 formatting: blacken the codebase Augie Fackler <augie@google.com> parents: 42824 diff changeset	290 error.ResponseError(
46711 aa2e38147e8b wireprotov1peer: don't raise internal errors in some cases Valentin Gatien-Baron <vgatien-baron@janestreet.com> parents: 46539 diff changeset	291 _(b'unfulfilled batch command response'), None
43076 2372284d9457 formatting: blacken the codebase Augie Fackler <augie@google.com> parents: 42824 diff changeset	292 )
2372284d9457 formatting: blacken the codebase Augie Fackler <augie@google.com> parents: 42824 diff changeset	293 )
37631 2f626233859b wireproto: implement batching on peer executor interface Gregory Szorc <gregory.szorc@gmail.com> parents: 37630 diff changeset	294
2f626233859b wireproto: implement batching on peer executor interface Gregory Szorc <gregory.szorc@gmail.com> parents: 37630 diff changeset	295 self._futures = None
2f626233859b wireproto: implement batching on peer executor interface Gregory Szorc <gregory.szorc@gmail.com> parents: 37630 diff changeset	296
2f626233859b wireproto: implement batching on peer executor interface Gregory Szorc <gregory.szorc@gmail.com> parents: 37630 diff changeset	297 def _readbatchresponse(self, states, wireresults):
2f626233859b wireproto: implement batching on peer executor interface Gregory Szorc <gregory.szorc@gmail.com> parents: 37630 diff changeset	298 # Executes in a thread to read data off the wire.
2f626233859b wireproto: implement batching on peer executor interface Gregory Szorc <gregory.szorc@gmail.com> parents: 37630 diff changeset	299
47913 cdad6560e832 wireprotov1peer: simplify the way batchable rpcs are defined Valentin Gatien-Baron <valentin.gatienbaron@gmail.com> parents: 46819 diff changeset	300 for command, f, batchable, decode in states:
37631 2f626233859b wireproto: implement batching on peer executor interface Gregory Szorc <gregory.szorc@gmail.com> parents: 37630 diff changeset	301 # Grab raw result off the wire and teach the internal future
2f626233859b wireproto: implement batching on peer executor interface Gregory Szorc <gregory.szorc@gmail.com> parents: 37630 diff changeset	302 # about it.
46711 aa2e38147e8b wireprotov1peer: don't raise internal errors in some cases Valentin Gatien-Baron <vgatien-baron@janestreet.com> parents: 46539 diff changeset	303 try:
aa2e38147e8b wireprotov1peer: don't raise internal errors in some cases Valentin Gatien-Baron <vgatien-baron@janestreet.com> parents: 46539 diff changeset	304 remoteresult = next(wireresults)
aa2e38147e8b wireprotov1peer: don't raise internal errors in some cases Valentin Gatien-Baron <vgatien-baron@janestreet.com> parents: 46539 diff changeset	305 except StopIteration:
aa2e38147e8b wireprotov1peer: don't raise internal errors in some cases Valentin Gatien-Baron <vgatien-baron@janestreet.com> parents: 46539 diff changeset	306 # This can happen in particular because next(batchable)
aa2e38147e8b wireprotov1peer: don't raise internal errors in some cases Valentin Gatien-Baron <vgatien-baron@janestreet.com> parents: 46539 diff changeset	307 # in the previous iteration can call peer._abort, which
aa2e38147e8b wireprotov1peer: don't raise internal errors in some cases Valentin Gatien-Baron <vgatien-baron@janestreet.com> parents: 46539 diff changeset	308 # may close the peer.
aa2e38147e8b wireprotov1peer: don't raise internal errors in some cases Valentin Gatien-Baron <vgatien-baron@janestreet.com> parents: 46539 diff changeset	309 f.set_exception(
aa2e38147e8b wireprotov1peer: don't raise internal errors in some cases Valentin Gatien-Baron <vgatien-baron@janestreet.com> parents: 46539 diff changeset	310 error.ResponseError(
aa2e38147e8b wireprotov1peer: don't raise internal errors in some cases Valentin Gatien-Baron <vgatien-baron@janestreet.com> parents: 46539 diff changeset	311 _(b'unfulfilled batch command response'), None
aa2e38147e8b wireprotov1peer: don't raise internal errors in some cases Valentin Gatien-Baron <vgatien-baron@janestreet.com> parents: 46539 diff changeset	312 )
aa2e38147e8b wireprotov1peer: don't raise internal errors in some cases Valentin Gatien-Baron <vgatien-baron@janestreet.com> parents: 46539 diff changeset	313 )
aa2e38147e8b wireprotov1peer: don't raise internal errors in some cases Valentin Gatien-Baron <vgatien-baron@janestreet.com> parents: 46539 diff changeset	314 else:
aa2e38147e8b wireprotov1peer: don't raise internal errors in some cases Valentin Gatien-Baron <vgatien-baron@janestreet.com> parents: 46539 diff changeset	315 try:
47913 cdad6560e832 wireprotov1peer: simplify the way batchable rpcs are defined Valentin Gatien-Baron <valentin.gatienbaron@gmail.com> parents: 46819 diff changeset	316 result = decode(remoteresult)
46711 aa2e38147e8b wireprotov1peer: don't raise internal errors in some cases Valentin Gatien-Baron <vgatien-baron@janestreet.com> parents: 46539 diff changeset	317 except Exception:
aa2e38147e8b wireprotov1peer: don't raise internal errors in some cases Valentin Gatien-Baron <vgatien-baron@janestreet.com> parents: 46539 diff changeset	318 pycompat.future_set_exception_info(f, sys.exc_info()[1:])
aa2e38147e8b wireprotov1peer: don't raise internal errors in some cases Valentin Gatien-Baron <vgatien-baron@janestreet.com> parents: 46539 diff changeset	319 else:
aa2e38147e8b wireprotov1peer: don't raise internal errors in some cases Valentin Gatien-Baron <vgatien-baron@janestreet.com> parents: 46539 diff changeset	320 f.set_result(result)
37631 2f626233859b wireproto: implement batching on peer executor interface Gregory Szorc <gregory.szorc@gmail.com> parents: 37630 diff changeset	321
43076 2372284d9457 formatting: blacken the codebase Augie Fackler <augie@google.com> parents: 42824 diff changeset	322
2372284d9457 formatting: blacken the codebase Augie Fackler <augie@google.com> parents: 42824 diff changeset	323 @interfaceutil.implementer(
2372284d9457 formatting: blacken the codebase Augie Fackler <augie@google.com> parents: 42824 diff changeset	324 repository.ipeercommands, repository.ipeerlegacycommands
2372284d9457 formatting: blacken the codebase Augie Fackler <augie@google.com> parents: 42824 diff changeset	325 )
37635 cc8c06835097 wireproto: convert legacy commands to command executor Gregory Szorc <gregory.szorc@gmail.com> parents: 37633 diff changeset	326 class wirepeer(repository.peer):
27243 3abee2ba27af wireproto: add docstring for wirepeer Gregory Szorc <gregory.szorc@gmail.com> parents: 26911 diff changeset	327 """Client-side interface for communicating with a peer repository.
14622 bd88561afb4b wireproto: add batching support to wirerepository Peter Arrenbrecht <peter.arrenbrecht@gmail.com> parents: 14621 diff changeset	328
27243 3abee2ba27af wireproto: add docstring for wirepeer Gregory Szorc <gregory.szorc@gmail.com> parents: 26911 diff changeset	329 Methods commonly call wire protocol commands of the same name.
3abee2ba27af wireproto: add docstring for wirepeer Gregory Szorc <gregory.szorc@gmail.com> parents: 26911 diff changeset	330
3abee2ba27af wireproto: add docstring for wirepeer Gregory Szorc <gregory.szorc@gmail.com> parents: 26911 diff changeset	331 See also httppeer.py and sshpeer.py for protocol-specific
3abee2ba27af wireproto: add docstring for wirepeer Gregory Szorc <gregory.szorc@gmail.com> parents: 26911 diff changeset	332 implementations of this interface.
3abee2ba27af wireproto: add docstring for wirepeer Gregory Szorc <gregory.szorc@gmail.com> parents: 26911 diff changeset	333 """
43076 2372284d9457 formatting: blacken the codebase Augie Fackler <augie@google.com> parents: 42824 diff changeset	334
37630 e1b32dc4646c wireproto: implement command executor interface for version 1 peers Gregory Szorc <gregory.szorc@gmail.com> parents: 37615 diff changeset	335 def commandexecutor(self):
e1b32dc4646c wireproto: implement command executor interface for version 1 peers Gregory Szorc <gregory.szorc@gmail.com> parents: 37615 diff changeset	336 return peerexecutor(self)
e1b32dc4646c wireproto: implement command executor interface for version 1 peers Gregory Szorc <gregory.szorc@gmail.com> parents: 37615 diff changeset	337
37320 39f7d4ee8bcd repository: port peer interfaces to zope.interface Gregory Szorc <gregory.szorc@gmail.com> parents: 37295 diff changeset	338 # Begin of ipeercommands interface.
14622 bd88561afb4b wireproto: add batching support to wirerepository Peter Arrenbrecht <peter.arrenbrecht@gmail.com> parents: 14621 diff changeset	339
37649 a168799687e5 wireproto: properly call clonebundles command Gregory Szorc <gregory.szorc@gmail.com> parents: 37646 diff changeset	340 def clonebundles(self):
43077 687b865b95ad formatting: byteify all mercurial/ and hgext/ string literals Augie Fackler <augie@google.com> parents: 43076 diff changeset	341 self.requirecap(b'clonebundles', _(b'clone bundles'))
687b865b95ad formatting: byteify all mercurial/ and hgext/ string literals Augie Fackler <augie@google.com> parents: 43076 diff changeset	342 return self._call(b'clonebundles')
37649 a168799687e5 wireproto: properly call clonebundles command Gregory Szorc <gregory.szorc@gmail.com> parents: 37646 diff changeset	343
50675 60f9602b413e clonebundles: add support for inline (streaming) clonebundles Mathias De Mare <mathias.de_mare@nokia.com> parents: 50582 diff changeset	344 def _finish_inline_clone_bundle(self, stream):
60f9602b413e clonebundles: add support for inline (streaming) clonebundles Mathias De Mare <mathias.de_mare@nokia.com> parents: 50582 diff changeset	345 pass # allow override for httppeer
60f9602b413e clonebundles: add support for inline (streaming) clonebundles Mathias De Mare <mathias.de_mare@nokia.com> parents: 50582 diff changeset	346
60f9602b413e clonebundles: add support for inline (streaming) clonebundles Mathias De Mare <mathias.de_mare@nokia.com> parents: 50582 diff changeset	347 def get_inline_clone_bundle(self, path):
60f9602b413e clonebundles: add support for inline (streaming) clonebundles Mathias De Mare <mathias.de_mare@nokia.com> parents: 50582 diff changeset	348 stream = self._callstream(b"get_inline_clone_bundle", path=path)
60f9602b413e clonebundles: add support for inline (streaming) clonebundles Mathias De Mare <mathias.de_mare@nokia.com> parents: 50582 diff changeset	349 length = util.uvarintdecodestream(stream)
60f9602b413e clonebundles: add support for inline (streaming) clonebundles Mathias De Mare <mathias.de_mare@nokia.com> parents: 50582 diff changeset	350
60f9602b413e clonebundles: add support for inline (streaming) clonebundles Mathias De Mare <mathias.de_mare@nokia.com> parents: 50582 diff changeset	351 # SSH streams will block if reading more than length
60f9602b413e clonebundles: add support for inline (streaming) clonebundles Mathias De Mare <mathias.de_mare@nokia.com> parents: 50582 diff changeset	352 for chunk in util.filechunkiter(stream, limit=length):
60f9602b413e clonebundles: add support for inline (streaming) clonebundles Mathias De Mare <mathias.de_mare@nokia.com> parents: 50582 diff changeset	353 yield chunk
60f9602b413e clonebundles: add support for inline (streaming) clonebundles Mathias De Mare <mathias.de_mare@nokia.com> parents: 50582 diff changeset	354
60f9602b413e clonebundles: add support for inline (streaming) clonebundles Mathias De Mare <mathias.de_mare@nokia.com> parents: 50582 diff changeset	355 self._finish_inline_clone_bundle(stream)
60f9602b413e clonebundles: add support for inline (streaming) clonebundles Mathias De Mare <mathias.de_mare@nokia.com> parents: 50582 diff changeset	356
14623 e7c9fdbbb902 wireproto: make a number of commands batchable Peter Arrenbrecht <peter.arrenbrecht@gmail.com> parents: 14622 diff changeset	357 @batchable
11586 ddaaaa23bb8f protocol: move basic ssh client commands to wirerepository Matt Mackall <mpm@selenic.com> parents: 11585 diff changeset	358 def lookup(self, key):
43077 687b865b95ad formatting: byteify all mercurial/ and hgext/ string literals Augie Fackler <augie@google.com> parents: 43076 diff changeset	359 self.requirecap(b'lookup', _(b'look up remote revision'))
47914 c424ff4807e6 wireprotov1peer: update all rpcs to use the new batchable scheme Valentin Gatien-Baron <valentin.gatienbaron@gmail.com> parents: 47913 diff changeset	360
c424ff4807e6 wireprotov1peer: update all rpcs to use the new batchable scheme Valentin Gatien-Baron <valentin.gatienbaron@gmail.com> parents: 47913 diff changeset	361 def decode(d):
c424ff4807e6 wireprotov1peer: update all rpcs to use the new batchable scheme Valentin Gatien-Baron <valentin.gatienbaron@gmail.com> parents: 47913 diff changeset	362 success, data = d[:-1].split(b" ", 1)
c424ff4807e6 wireprotov1peer: update all rpcs to use the new batchable scheme Valentin Gatien-Baron <valentin.gatienbaron@gmail.com> parents: 47913 diff changeset	363 if int(success):
c424ff4807e6 wireprotov1peer: update all rpcs to use the new batchable scheme Valentin Gatien-Baron <valentin.gatienbaron@gmail.com> parents: 47913 diff changeset	364 return bin(data)
c424ff4807e6 wireprotov1peer: update all rpcs to use the new batchable scheme Valentin Gatien-Baron <valentin.gatienbaron@gmail.com> parents: 47913 diff changeset	365 else:
c424ff4807e6 wireprotov1peer: update all rpcs to use the new batchable scheme Valentin Gatien-Baron <valentin.gatienbaron@gmail.com> parents: 47913 diff changeset	366 self._abort(error.RepoError(data))
c424ff4807e6 wireprotov1peer: update all rpcs to use the new batchable scheme Valentin Gatien-Baron <valentin.gatienbaron@gmail.com> parents: 47913 diff changeset	367
c424ff4807e6 wireprotov1peer: update all rpcs to use the new batchable scheme Valentin Gatien-Baron <valentin.gatienbaron@gmail.com> parents: 47913 diff changeset	368 return {b'key': encoding.fromlocal(key)}, decode
11586 ddaaaa23bb8f protocol: move basic ssh client commands to wirerepository Matt Mackall <mpm@selenic.com> parents: 11585 diff changeset	369
14623 e7c9fdbbb902 wireproto: make a number of commands batchable Peter Arrenbrecht <peter.arrenbrecht@gmail.com> parents: 14622 diff changeset	370 @batchable
11586 ddaaaa23bb8f protocol: move basic ssh client commands to wirerepository Matt Mackall <mpm@selenic.com> parents: 11585 diff changeset	371 def heads(self):
47914 c424ff4807e6 wireprotov1peer: update all rpcs to use the new batchable scheme Valentin Gatien-Baron <valentin.gatienbaron@gmail.com> parents: 47913 diff changeset	372 def decode(d):
c424ff4807e6 wireprotov1peer: update all rpcs to use the new batchable scheme Valentin Gatien-Baron <valentin.gatienbaron@gmail.com> parents: 47913 diff changeset	373 try:
c424ff4807e6 wireprotov1peer: update all rpcs to use the new batchable scheme Valentin Gatien-Baron <valentin.gatienbaron@gmail.com> parents: 47913 diff changeset	374 return wireprototypes.decodelist(d[:-1])
c424ff4807e6 wireprotov1peer: update all rpcs to use the new batchable scheme Valentin Gatien-Baron <valentin.gatienbaron@gmail.com> parents: 47913 diff changeset	375 except ValueError:
c424ff4807e6 wireprotov1peer: update all rpcs to use the new batchable scheme Valentin Gatien-Baron <valentin.gatienbaron@gmail.com> parents: 47913 diff changeset	376 self._abort(error.ResponseError(_(b"unexpected response:"), d))
c424ff4807e6 wireprotov1peer: update all rpcs to use the new batchable scheme Valentin Gatien-Baron <valentin.gatienbaron@gmail.com> parents: 47913 diff changeset	377
c424ff4807e6 wireprotov1peer: update all rpcs to use the new batchable scheme Valentin Gatien-Baron <valentin.gatienbaron@gmail.com> parents: 47913 diff changeset	378 return {}, decode
11586 ddaaaa23bb8f protocol: move basic ssh client commands to wirerepository Matt Mackall <mpm@selenic.com> parents: 11585 diff changeset	379
14623 e7c9fdbbb902 wireproto: make a number of commands batchable Peter Arrenbrecht <peter.arrenbrecht@gmail.com> parents: 14622 diff changeset	380 @batchable
13723 e615765fdcc7 wireproto: add known([id]) function Peter Arrenbrecht <peter.arrenbrecht@gmail.com> parents: 13722 diff changeset	381 def known(self, nodes):
47914 c424ff4807e6 wireprotov1peer: update all rpcs to use the new batchable scheme Valentin Gatien-Baron <valentin.gatienbaron@gmail.com> parents: 47913 diff changeset	382 def decode(d):
c424ff4807e6 wireprotov1peer: update all rpcs to use the new batchable scheme Valentin Gatien-Baron <valentin.gatienbaron@gmail.com> parents: 47913 diff changeset	383 try:
c424ff4807e6 wireprotov1peer: update all rpcs to use the new batchable scheme Valentin Gatien-Baron <valentin.gatienbaron@gmail.com> parents: 47913 diff changeset	384 return [bool(int(b)) for b in pycompat.iterbytestr(d)]
c424ff4807e6 wireprotov1peer: update all rpcs to use the new batchable scheme Valentin Gatien-Baron <valentin.gatienbaron@gmail.com> parents: 47913 diff changeset	385 except ValueError:
c424ff4807e6 wireprotov1peer: update all rpcs to use the new batchable scheme Valentin Gatien-Baron <valentin.gatienbaron@gmail.com> parents: 47913 diff changeset	386 self._abort(error.ResponseError(_(b"unexpected response:"), d))
c424ff4807e6 wireprotov1peer: update all rpcs to use the new batchable scheme Valentin Gatien-Baron <valentin.gatienbaron@gmail.com> parents: 47913 diff changeset	387
c424ff4807e6 wireprotov1peer: update all rpcs to use the new batchable scheme Valentin Gatien-Baron <valentin.gatienbaron@gmail.com> parents: 47913 diff changeset	388 return {b'nodes': wireprototypes.encodelist(nodes)}, decode
13723 e615765fdcc7 wireproto: add known([id]) function Peter Arrenbrecht <peter.arrenbrecht@gmail.com> parents: 13722 diff changeset	389
14623 e7c9fdbbb902 wireproto: make a number of commands batchable Peter Arrenbrecht <peter.arrenbrecht@gmail.com> parents: 14622 diff changeset	390 @batchable
11586 ddaaaa23bb8f protocol: move basic ssh client commands to wirerepository Matt Mackall <mpm@selenic.com> parents: 11585 diff changeset	391 def branchmap(self):
47914 c424ff4807e6 wireprotov1peer: update all rpcs to use the new batchable scheme Valentin Gatien-Baron <valentin.gatienbaron@gmail.com> parents: 47913 diff changeset	392 def decode(d):
c424ff4807e6 wireprotov1peer: update all rpcs to use the new batchable scheme Valentin Gatien-Baron <valentin.gatienbaron@gmail.com> parents: 47913 diff changeset	393 try:
c424ff4807e6 wireprotov1peer: update all rpcs to use the new batchable scheme Valentin Gatien-Baron <valentin.gatienbaron@gmail.com> parents: 47913 diff changeset	394 branchmap = {}
c424ff4807e6 wireprotov1peer: update all rpcs to use the new batchable scheme Valentin Gatien-Baron <valentin.gatienbaron@gmail.com> parents: 47913 diff changeset	395 for branchpart in d.splitlines():
c424ff4807e6 wireprotov1peer: update all rpcs to use the new batchable scheme Valentin Gatien-Baron <valentin.gatienbaron@gmail.com> parents: 47913 diff changeset	396 branchname, branchheads = branchpart.split(b' ', 1)
c424ff4807e6 wireprotov1peer: update all rpcs to use the new batchable scheme Valentin Gatien-Baron <valentin.gatienbaron@gmail.com> parents: 47913 diff changeset	397 branchname = encoding.tolocal(urlreq.unquote(branchname))
c424ff4807e6 wireprotov1peer: update all rpcs to use the new batchable scheme Valentin Gatien-Baron <valentin.gatienbaron@gmail.com> parents: 47913 diff changeset	398 branchheads = wireprototypes.decodelist(branchheads)
c424ff4807e6 wireprotov1peer: update all rpcs to use the new batchable scheme Valentin Gatien-Baron <valentin.gatienbaron@gmail.com> parents: 47913 diff changeset	399 branchmap[branchname] = branchheads
c424ff4807e6 wireprotov1peer: update all rpcs to use the new batchable scheme Valentin Gatien-Baron <valentin.gatienbaron@gmail.com> parents: 47913 diff changeset	400 return branchmap
c424ff4807e6 wireprotov1peer: update all rpcs to use the new batchable scheme Valentin Gatien-Baron <valentin.gatienbaron@gmail.com> parents: 47913 diff changeset	401 except TypeError:
c424ff4807e6 wireprotov1peer: update all rpcs to use the new batchable scheme Valentin Gatien-Baron <valentin.gatienbaron@gmail.com> parents: 47913 diff changeset	402 self._abort(error.ResponseError(_(b"unexpected response:"), d))
c424ff4807e6 wireprotov1peer: update all rpcs to use the new batchable scheme Valentin Gatien-Baron <valentin.gatienbaron@gmail.com> parents: 47913 diff changeset	403
c424ff4807e6 wireprotov1peer: update all rpcs to use the new batchable scheme Valentin Gatien-Baron <valentin.gatienbaron@gmail.com> parents: 47913 diff changeset	404 return {}, decode
11586 ddaaaa23bb8f protocol: move basic ssh client commands to wirerepository Matt Mackall <mpm@selenic.com> parents: 11585 diff changeset	405
33827 dedab036215d wireproto: use new peer interface Gregory Szorc <gregory.szorc@gmail.com> parents: 33788 diff changeset	406 @batchable
dedab036215d wireproto: use new peer interface Gregory Szorc <gregory.szorc@gmail.com> parents: 33788 diff changeset	407 def listkeys(self, namespace):
43077 687b865b95ad formatting: byteify all mercurial/ and hgext/ string literals Augie Fackler <augie@google.com> parents: 43076 diff changeset	408 if not self.capable(b'pushkey'):
47914 c424ff4807e6 wireprotov1peer: update all rpcs to use the new batchable scheme Valentin Gatien-Baron <valentin.gatienbaron@gmail.com> parents: 47913 diff changeset	409 return {}, None
43077 687b865b95ad formatting: byteify all mercurial/ and hgext/ string literals Augie Fackler <augie@google.com> parents: 43076 diff changeset	410 self.ui.debug(b'preparing listkeys for "%s"\n' % namespace)
47914 c424ff4807e6 wireprotov1peer: update all rpcs to use the new batchable scheme Valentin Gatien-Baron <valentin.gatienbaron@gmail.com> parents: 47913 diff changeset	411
c424ff4807e6 wireprotov1peer: update all rpcs to use the new batchable scheme Valentin Gatien-Baron <valentin.gatienbaron@gmail.com> parents: 47913 diff changeset	412 def decode(d):
c424ff4807e6 wireprotov1peer: update all rpcs to use the new batchable scheme Valentin Gatien-Baron <valentin.gatienbaron@gmail.com> parents: 47913 diff changeset	413 self.ui.debug(
c424ff4807e6 wireprotov1peer: update all rpcs to use the new batchable scheme Valentin Gatien-Baron <valentin.gatienbaron@gmail.com> parents: 47913 diff changeset	414 b'received listkey for "%s": %i bytes\n' % (namespace, len(d))
c424ff4807e6 wireprotov1peer: update all rpcs to use the new batchable scheme Valentin Gatien-Baron <valentin.gatienbaron@gmail.com> parents: 47913 diff changeset	415 )
c424ff4807e6 wireprotov1peer: update all rpcs to use the new batchable scheme Valentin Gatien-Baron <valentin.gatienbaron@gmail.com> parents: 47913 diff changeset	416 return pushkeymod.decodekeys(d)
c424ff4807e6 wireprotov1peer: update all rpcs to use the new batchable scheme Valentin Gatien-Baron <valentin.gatienbaron@gmail.com> parents: 47913 diff changeset	417
c424ff4807e6 wireprotov1peer: update all rpcs to use the new batchable scheme Valentin Gatien-Baron <valentin.gatienbaron@gmail.com> parents: 47913 diff changeset	418 return {b'namespace': encoding.fromlocal(namespace)}, decode
11586 ddaaaa23bb8f protocol: move basic ssh client commands to wirerepository Matt Mackall <mpm@selenic.com> parents: 11585 diff changeset	419
14623 e7c9fdbbb902 wireproto: make a number of commands batchable Peter Arrenbrecht <peter.arrenbrecht@gmail.com> parents: 14622 diff changeset	420 @batchable
11586 ddaaaa23bb8f protocol: move basic ssh client commands to wirerepository Matt Mackall <mpm@selenic.com> parents: 11585 diff changeset	421 def pushkey(self, namespace, key, old, new):
43077 687b865b95ad formatting: byteify all mercurial/ and hgext/ string literals Augie Fackler <augie@google.com> parents: 43076 diff changeset	422 if not self.capable(b'pushkey'):
47914 c424ff4807e6 wireprotov1peer: update all rpcs to use the new batchable scheme Valentin Gatien-Baron <valentin.gatienbaron@gmail.com> parents: 47913 diff changeset	423 return False, None
43077 687b865b95ad formatting: byteify all mercurial/ and hgext/ string literals Augie Fackler <augie@google.com> parents: 43076 diff changeset	424 self.ui.debug(b'preparing pushkey for "%s:%s"\n' % (namespace, key))
47914 c424ff4807e6 wireprotov1peer: update all rpcs to use the new batchable scheme Valentin Gatien-Baron <valentin.gatienbaron@gmail.com> parents: 47913 diff changeset	425
c424ff4807e6 wireprotov1peer: update all rpcs to use the new batchable scheme Valentin Gatien-Baron <valentin.gatienbaron@gmail.com> parents: 47913 diff changeset	426 def decode(d):
c424ff4807e6 wireprotov1peer: update all rpcs to use the new batchable scheme Valentin Gatien-Baron <valentin.gatienbaron@gmail.com> parents: 47913 diff changeset	427 d, output = d.split(b'\n', 1)
c424ff4807e6 wireprotov1peer: update all rpcs to use the new batchable scheme Valentin Gatien-Baron <valentin.gatienbaron@gmail.com> parents: 47913 diff changeset	428 try:
c424ff4807e6 wireprotov1peer: update all rpcs to use the new batchable scheme Valentin Gatien-Baron <valentin.gatienbaron@gmail.com> parents: 47913 diff changeset	429 d = bool(int(d))
c424ff4807e6 wireprotov1peer: update all rpcs to use the new batchable scheme Valentin Gatien-Baron <valentin.gatienbaron@gmail.com> parents: 47913 diff changeset	430 except ValueError:
c424ff4807e6 wireprotov1peer: update all rpcs to use the new batchable scheme Valentin Gatien-Baron <valentin.gatienbaron@gmail.com> parents: 47913 diff changeset	431 raise error.ResponseError(
c424ff4807e6 wireprotov1peer: update all rpcs to use the new batchable scheme Valentin Gatien-Baron <valentin.gatienbaron@gmail.com> parents: 47913 diff changeset	432 _(b'push failed (unexpected response):'), d
c424ff4807e6 wireprotov1peer: update all rpcs to use the new batchable scheme Valentin Gatien-Baron <valentin.gatienbaron@gmail.com> parents: 47913 diff changeset	433 )
c424ff4807e6 wireprotov1peer: update all rpcs to use the new batchable scheme Valentin Gatien-Baron <valentin.gatienbaron@gmail.com> parents: 47913 diff changeset	434 for l in output.splitlines(True):
c424ff4807e6 wireprotov1peer: update all rpcs to use the new batchable scheme Valentin Gatien-Baron <valentin.gatienbaron@gmail.com> parents: 47913 diff changeset	435 self.ui.status(_(b'remote: '), l)
c424ff4807e6 wireprotov1peer: update all rpcs to use the new batchable scheme Valentin Gatien-Baron <valentin.gatienbaron@gmail.com> parents: 47913 diff changeset	436 return d
c424ff4807e6 wireprotov1peer: update all rpcs to use the new batchable scheme Valentin Gatien-Baron <valentin.gatienbaron@gmail.com> parents: 47913 diff changeset	437
c424ff4807e6 wireprotov1peer: update all rpcs to use the new batchable scheme Valentin Gatien-Baron <valentin.gatienbaron@gmail.com> parents: 47913 diff changeset	438 return {
43077 687b865b95ad formatting: byteify all mercurial/ and hgext/ string literals Augie Fackler <augie@google.com> parents: 43076 diff changeset	439 b'namespace': encoding.fromlocal(namespace),
687b865b95ad formatting: byteify all mercurial/ and hgext/ string literals Augie Fackler <augie@google.com> parents: 43076 diff changeset	440 b'key': encoding.fromlocal(key),
687b865b95ad formatting: byteify all mercurial/ and hgext/ string literals Augie Fackler <augie@google.com> parents: 43076 diff changeset	441 b'old': encoding.fromlocal(old),
687b865b95ad formatting: byteify all mercurial/ and hgext/ string literals Augie Fackler <augie@google.com> parents: 43076 diff changeset	442 b'new': encoding.fromlocal(new),
47914 c424ff4807e6 wireprotov1peer: update all rpcs to use the new batchable scheme Valentin Gatien-Baron <valentin.gatienbaron@gmail.com> parents: 47913 diff changeset	443 }, decode
11586 ddaaaa23bb8f protocol: move basic ssh client commands to wirerepository Matt Mackall <mpm@selenic.com> parents: 11585 diff changeset	444
11588 8a1f625e971d protocol: unify stream_out client code Matt Mackall <mpm@selenic.com> parents: 11587 diff changeset	445 def stream_out(self):
43077 687b865b95ad formatting: byteify all mercurial/ and hgext/ string literals Augie Fackler <augie@google.com> parents: 43076 diff changeset	446 return self._callstream(b'stream_out')
11588 8a1f625e971d protocol: unify stream_out client code Matt Mackall <mpm@selenic.com> parents: 11587 diff changeset	447
21646 ce25f465e572 getbundle: declare type of parameters Pierre-Yves David <pierre-yves.david@fb.com> parents: 21631 diff changeset	448 def getbundle(self, source, **kwargs):
34739 b880cc11da5d wireproto: bounce kwargs to/from bytes/str as needed Augie Fackler <augie@google.com> parents: 34731 diff changeset	449 kwargs = pycompat.byteskwargs(kwargs)
43077 687b865b95ad formatting: byteify all mercurial/ and hgext/ string literals Augie Fackler <augie@google.com> parents: 43076 diff changeset	450 self.requirecap(b'getbundle', _(b'look up remote changes'))
13741 b51bf961b3cb wireproto: add getbundle() function Peter Arrenbrecht <peter.arrenbrecht@gmail.com> parents: 13726 diff changeset	451 opts = {}
43077 687b865b95ad formatting: byteify all mercurial/ and hgext/ string literals Augie Fackler <augie@google.com> parents: 43076 diff changeset	452 bundlecaps = kwargs.get(b'bundlecaps') or set()
49004 f254fc73d956 global: bulk replace simple pycompat.iteritems(x) with x.items() Gregory Szorc <gregory.szorc@gmail.com> parents: 48966 diff changeset	453 for key, value in kwargs.items():
21646 ce25f465e572 getbundle: declare type of parameters Pierre-Yves David <pierre-yves.david@fb.com> parents: 21631 diff changeset	454 if value is None:
ce25f465e572 getbundle: declare type of parameters Pierre-Yves David <pierre-yves.david@fb.com> parents: 21631 diff changeset	455 continue
37613 96d735601ca1 wireproto: move gboptsmap to wireprototypes and rename (API) Gregory Szorc <gregory.szorc@gmail.com> parents: 37612 diff changeset	456 keytype = wireprototypes.GETBUNDLE_ARGUMENTS.get(key)
21646 ce25f465e572 getbundle: declare type of parameters Pierre-Yves David <pierre-yves.david@fb.com> parents: 21631 diff changeset	457 if keytype is None:
34730 6be264009841 wireproto: use a proper exception instead of `assert False` Augie Fackler <augie@google.com> parents: 34729 diff changeset	458 raise error.ProgrammingError(
43077 687b865b95ad formatting: byteify all mercurial/ and hgext/ string literals Augie Fackler <augie@google.com> parents: 43076 diff changeset	459 b'Unexpectedly None keytype for key %s' % key
43076 2372284d9457 formatting: blacken the codebase Augie Fackler <augie@google.com> parents: 42824 diff changeset	460 )
43077 687b865b95ad formatting: byteify all mercurial/ and hgext/ string literals Augie Fackler <augie@google.com> parents: 43076 diff changeset	461 elif keytype == b'nodes':
37612 5e71dea79aae wireproto: move value encoding functions to wireprototypes (API) Gregory Szorc <gregory.szorc@gmail.com> parents: 37546 diff changeset	462 value = wireprototypes.encodelist(value)
43077 687b865b95ad formatting: byteify all mercurial/ and hgext/ string literals Augie Fackler <augie@google.com> parents: 43076 diff changeset	463 elif keytype == b'csv':
687b865b95ad formatting: byteify all mercurial/ and hgext/ string literals Augie Fackler <augie@google.com> parents: 43076 diff changeset	464 value = b','.join(value)
687b865b95ad formatting: byteify all mercurial/ and hgext/ string literals Augie Fackler <augie@google.com> parents: 43076 diff changeset	465 elif keytype == b'scsv':
687b865b95ad formatting: byteify all mercurial/ and hgext/ string literals Augie Fackler <augie@google.com> parents: 43076 diff changeset	466 value = b','.join(sorted(value))
687b865b95ad formatting: byteify all mercurial/ and hgext/ string literals Augie Fackler <augie@google.com> parents: 43076 diff changeset	467 elif keytype == b'boolean':
687b865b95ad formatting: byteify all mercurial/ and hgext/ string literals Augie Fackler <augie@google.com> parents: 43076 diff changeset	468 value = b'%i' % bool(value)
687b865b95ad formatting: byteify all mercurial/ and hgext/ string literals Augie Fackler <augie@google.com> parents: 43076 diff changeset	469 elif keytype != b'plain':
687b865b95ad formatting: byteify all mercurial/ and hgext/ string literals Augie Fackler <augie@google.com> parents: 43076 diff changeset	470 raise KeyError(b'unknown getbundle option type %s' % keytype)
21646 ce25f465e572 getbundle: declare type of parameters Pierre-Yves David <pierre-yves.david@fb.com> parents: 21631 diff changeset	471 opts[key] = value
43077 687b865b95ad formatting: byteify all mercurial/ and hgext/ string literals Augie Fackler <augie@google.com> parents: 43076 diff changeset	472 f = self._callcompressable(b"getbundle", **pycompat.strkwargs(opts))
687b865b95ad formatting: byteify all mercurial/ and hgext/ string literals Augie Fackler <augie@google.com> parents: 43076 diff changeset	473 if any((cap.startswith(b'HG2') for cap in bundlecaps)):
24641 60fecc5b14a4 unbundle20: retrieve unbundler instances through a factory function Pierre-Yves David <pierre-yves.david@fb.com> parents: 23878 diff changeset	474 return bundle2.getunbundler(self.ui, f)
21069 0a9cae236738 bundle2: allow bundle2 for pulling over the wire Pierre-Yves David <pierre-yves.david@fb.com> parents: 21064 diff changeset	475 else:
43077 687b865b95ad formatting: byteify all mercurial/ and hgext/ string literals Augie Fackler <augie@google.com> parents: 43076 diff changeset	476 return changegroupmod.cg1unpacker(f, b'UN')
13741 b51bf961b3cb wireproto: add getbundle() function Peter Arrenbrecht <peter.arrenbrecht@gmail.com> parents: 13726 diff changeset	477
37646 72e26319f3b8 wireproto: use command executor for unbundle Gregory Szorc <gregory.szorc@gmail.com> parents: 37635 diff changeset	478 def unbundle(self, bundle, heads, url):
45957 89a2afe31e82 formating: upgrade to black 20.8b1 Augie Fackler <raf@durin42.com> parents: 44060 diff changeset	479 """Send cg (a readable file-like object representing the
11592 26e0782b8380 protocol: unify client unbundle support Matt Mackall <mpm@selenic.com> parents: 11591 diff changeset	480 changegroup to push, typically a chunkbuffer object) to the
21075 438803e4bd97 bundle2: support for push over the wire Pierre-Yves David <pierre-yves.david@fb.com> parents: 21072 diff changeset	481 remote server as a bundle.
438803e4bd97 bundle2: support for push over the wire Pierre-Yves David <pierre-yves.david@fb.com> parents: 21072 diff changeset	482
438803e4bd97 bundle2: support for push over the wire Pierre-Yves David <pierre-yves.david@fb.com> parents: 21072 diff changeset	483 When pushing a bundle10 stream, return an integer indicating the
32898 4c2a46f89f08 wireproto: update reference to deleted addchangegroup() Martin von Zweigbergk <martinvonz@google.com> parents: 32764 diff changeset	484 result of the push (see changegroup.apply()).
21075 438803e4bd97 bundle2: support for push over the wire Pierre-Yves David <pierre-yves.david@fb.com> parents: 21072 diff changeset	485
29719 7f6130c7ffe1 wirepeer: rename confusing `source` parameter Augie Fackler <augie@google.com> parents: 29590 diff changeset	486 When pushing a bundle20 stream, return a bundle20 stream.
7f6130c7ffe1 wirepeer: rename confusing `source` parameter Augie Fackler <augie@google.com> parents: 29590 diff changeset	487
7f6130c7ffe1 wirepeer: rename confusing `source` parameter Augie Fackler <augie@google.com> parents: 29590 diff changeset	488 `url` is the url the client thinks it's pushing to, which is
7f6130c7ffe1 wirepeer: rename confusing `source` parameter Augie Fackler <augie@google.com> parents: 29590 diff changeset	489 visible to hooks.
45957 89a2afe31e82 formating: upgrade to black 20.8b1 Augie Fackler <raf@durin42.com> parents: 44060 diff changeset	490 """
11592 26e0782b8380 protocol: unify client unbundle support Matt Mackall <mpm@selenic.com> parents: 11591 diff changeset	491
43077 687b865b95ad formatting: byteify all mercurial/ and hgext/ string literals Augie Fackler <augie@google.com> parents: 43076 diff changeset	492 if heads != [b'force'] and self.capable(b'unbundlehash'):
37612 5e71dea79aae wireproto: move value encoding functions to wireprototypes (API) Gregory Szorc <gregory.szorc@gmail.com> parents: 37546 diff changeset	493 heads = wireprototypes.encodelist(
44060 a61287a95dc3 core: migrate uses of hashlib.sha1 to hashutil.sha1 Augie Fackler <augie@google.com> parents: 43554 diff changeset	494 [b'hashed', hashutil.sha1(b''.join(sorted(heads))).digest()]
43076 2372284d9457 formatting: blacken the codebase Augie Fackler <augie@google.com> parents: 42824 diff changeset	495 )
13942 88f0e41d8802 wireproto: allow unbundle with hashed heads parameter (issue2126) Shuhei Takahashi <takahashi.shuhei@gmail.com> parents: 13741 diff changeset	496 else:
37612 5e71dea79aae wireproto: move value encoding functions to wireprototypes (API) Gregory Szorc <gregory.szorc@gmail.com> parents: 37546 diff changeset	497 heads = wireprototypes.encodelist(heads)
13942 88f0e41d8802 wireproto: allow unbundle with hashed heads parameter (issue2126) Shuhei Takahashi <takahashi.shuhei@gmail.com> parents: 13741 diff changeset	498
50582 b1fb4185e47c safehasattr: pass attribute name as string instead of bytes Pierre-Yves David <pierre-yves.david@octobus.net> parents: 49292 diff changeset	499 if util.safehasattr(bundle, 'deltaheader'):
21075 438803e4bd97 bundle2: support for push over the wire Pierre-Yves David <pierre-yves.david@fb.com> parents: 21072 diff changeset	500 # this a bundle10, do the old style call sequence
43077 687b865b95ad formatting: byteify all mercurial/ and hgext/ string literals Augie Fackler <augie@google.com> parents: 43076 diff changeset	501 ret, output = self._callpush(b"unbundle", bundle, heads=heads)
687b865b95ad formatting: byteify all mercurial/ and hgext/ string literals Augie Fackler <augie@google.com> parents: 43076 diff changeset	502 if ret == b"":
687b865b95ad formatting: byteify all mercurial/ and hgext/ string literals Augie Fackler <augie@google.com> parents: 43076 diff changeset	503 raise error.ResponseError(_(b'push failed:'), output)
21075 438803e4bd97 bundle2: support for push over the wire Pierre-Yves David <pierre-yves.david@fb.com> parents: 21072 diff changeset	504 try:
438803e4bd97 bundle2: support for push over the wire Pierre-Yves David <pierre-yves.david@fb.com> parents: 21072 diff changeset	505 ret = int(ret)
438803e4bd97 bundle2: support for push over the wire Pierre-Yves David <pierre-yves.david@fb.com> parents: 21072 diff changeset	506 except ValueError:
438803e4bd97 bundle2: support for push over the wire Pierre-Yves David <pierre-yves.david@fb.com> parents: 21072 diff changeset	507 raise error.ResponseError(
43077 687b865b95ad formatting: byteify all mercurial/ and hgext/ string literals Augie Fackler <augie@google.com> parents: 43076 diff changeset	508 _(b'push failed (unexpected response):'), ret
43076 2372284d9457 formatting: blacken the codebase Augie Fackler <augie@google.com> parents: 42824 diff changeset	509 )
11592 26e0782b8380 protocol: unify client unbundle support Matt Mackall <mpm@selenic.com> parents: 11591 diff changeset	510
21075 438803e4bd97 bundle2: support for push over the wire Pierre-Yves David <pierre-yves.david@fb.com> parents: 21072 diff changeset	511 for l in output.splitlines(True):
43077 687b865b95ad formatting: byteify all mercurial/ and hgext/ string literals Augie Fackler <augie@google.com> parents: 43076 diff changeset	512 self.ui.status(_(b'remote: '), l)
21075 438803e4bd97 bundle2: support for push over the wire Pierre-Yves David <pierre-yves.david@fb.com> parents: 21072 diff changeset	513 else:
438803e4bd97 bundle2: support for push over the wire Pierre-Yves David <pierre-yves.david@fb.com> parents: 21072 diff changeset	514 # bundle2 push. Send a stream, fetch a stream.
43077 687b865b95ad formatting: byteify all mercurial/ and hgext/ string literals Augie Fackler <augie@google.com> parents: 43076 diff changeset	515 stream = self._calltwowaystream(b'unbundle', bundle, heads=heads)
24641 60fecc5b14a4 unbundle20: retrieve unbundler instances through a factory function Pierre-Yves David <pierre-yves.david@fb.com> parents: 23878 diff changeset	516 ret = bundle2.getunbundler(self.ui, stream)
11592 26e0782b8380 protocol: unify client unbundle support Matt Mackall <mpm@selenic.com> parents: 11591 diff changeset	517 return ret
26e0782b8380 protocol: unify client unbundle support Matt Mackall <mpm@selenic.com> parents: 11591 diff changeset	518
37320 39f7d4ee8bcd repository: port peer interfaces to zope.interface Gregory Szorc <gregory.szorc@gmail.com> parents: 37295 diff changeset	519 # End of ipeercommands interface.
33827 dedab036215d wireproto: use new peer interface Gregory Szorc <gregory.szorc@gmail.com> parents: 33788 diff changeset	520
37320 39f7d4ee8bcd repository: port peer interfaces to zope.interface Gregory Szorc <gregory.szorc@gmail.com> parents: 37295 diff changeset	521 # Begin of ipeerlegacycommands interface.
33827 dedab036215d wireproto: use new peer interface Gregory Szorc <gregory.szorc@gmail.com> parents: 33788 diff changeset	522
dedab036215d wireproto: use new peer interface Gregory Szorc <gregory.szorc@gmail.com> parents: 33788 diff changeset	523 def branches(self, nodes):
37612 5e71dea79aae wireproto: move value encoding functions to wireprototypes (API) Gregory Szorc <gregory.szorc@gmail.com> parents: 37546 diff changeset	524 n = wireprototypes.encodelist(nodes)
43077 687b865b95ad formatting: byteify all mercurial/ and hgext/ string literals Augie Fackler <augie@google.com> parents: 43076 diff changeset	525 d = self._call(b"branches", nodes=n)
33827 dedab036215d wireproto: use new peer interface Gregory Szorc <gregory.szorc@gmail.com> parents: 33788 diff changeset	526 try:
37612 5e71dea79aae wireproto: move value encoding functions to wireprototypes (API) Gregory Szorc <gregory.szorc@gmail.com> parents: 37546 diff changeset	527 br = [tuple(wireprototypes.decodelist(b)) for b in d.splitlines()]
33827 dedab036215d wireproto: use new peer interface Gregory Szorc <gregory.szorc@gmail.com> parents: 33788 diff changeset	528 return br
dedab036215d wireproto: use new peer interface Gregory Szorc <gregory.szorc@gmail.com> parents: 33788 diff changeset	529 except ValueError:
43077 687b865b95ad formatting: byteify all mercurial/ and hgext/ string literals Augie Fackler <augie@google.com> parents: 43076 diff changeset	530 self._abort(error.ResponseError(_(b"unexpected response:"), d))
33827 dedab036215d wireproto: use new peer interface Gregory Szorc <gregory.szorc@gmail.com> parents: 33788 diff changeset	531
dedab036215d wireproto: use new peer interface Gregory Szorc <gregory.szorc@gmail.com> parents: 33788 diff changeset	532 def between(self, pairs):
43076 2372284d9457 formatting: blacken the codebase Augie Fackler <augie@google.com> parents: 42824 diff changeset	533 batch = 8 # avoid giant requests
33827 dedab036215d wireproto: use new peer interface Gregory Szorc <gregory.szorc@gmail.com> parents: 33788 diff changeset	534 r = []
49292 d44e3c45f0e4 py3: replace `pycompat.xrange` by `range` Manuel Jacob <me@manueljacob.de> parents: 49037 diff changeset	535 for i in range(0, len(pairs), batch):
43077 687b865b95ad formatting: byteify all mercurial/ and hgext/ string literals Augie Fackler <augie@google.com> parents: 43076 diff changeset	536 n = b" ".join(
43076 2372284d9457 formatting: blacken the codebase Augie Fackler <augie@google.com> parents: 42824 diff changeset	537 [
43077 687b865b95ad formatting: byteify all mercurial/ and hgext/ string literals Augie Fackler <augie@google.com> parents: 43076 diff changeset	538 wireprototypes.encodelist(p, b'-')
43076 2372284d9457 formatting: blacken the codebase Augie Fackler <augie@google.com> parents: 42824 diff changeset	539 for p in pairs[i : i + batch]
2372284d9457 formatting: blacken the codebase Augie Fackler <augie@google.com> parents: 42824 diff changeset	540 ]
2372284d9457 formatting: blacken the codebase Augie Fackler <augie@google.com> parents: 42824 diff changeset	541 )
43077 687b865b95ad formatting: byteify all mercurial/ and hgext/ string literals Augie Fackler <augie@google.com> parents: 43076 diff changeset	542 d = self._call(b"between", pairs=n)
33827 dedab036215d wireproto: use new peer interface Gregory Szorc <gregory.szorc@gmail.com> parents: 33788 diff changeset	543 try:
43076 2372284d9457 formatting: blacken the codebase Augie Fackler <augie@google.com> parents: 42824 diff changeset	544 r.extend(
2372284d9457 formatting: blacken the codebase Augie Fackler <augie@google.com> parents: 42824 diff changeset	545 l and wireprototypes.decodelist(l) or []
2372284d9457 formatting: blacken the codebase Augie Fackler <augie@google.com> parents: 42824 diff changeset	546 for l in d.splitlines()
2372284d9457 formatting: blacken the codebase Augie Fackler <augie@google.com> parents: 42824 diff changeset	547 )
33827 dedab036215d wireproto: use new peer interface Gregory Szorc <gregory.szorc@gmail.com> parents: 33788 diff changeset	548 except ValueError:
43077 687b865b95ad formatting: byteify all mercurial/ and hgext/ string literals Augie Fackler <augie@google.com> parents: 43076 diff changeset	549 self._abort(error.ResponseError(_(b"unexpected response:"), d))
33827 dedab036215d wireproto: use new peer interface Gregory Szorc <gregory.szorc@gmail.com> parents: 33788 diff changeset	550 return r
dedab036215d wireproto: use new peer interface Gregory Szorc <gregory.szorc@gmail.com> parents: 33788 diff changeset	551
37635 cc8c06835097 wireproto: convert legacy commands to command executor Gregory Szorc <gregory.szorc@gmail.com> parents: 37633 diff changeset	552 def changegroup(self, nodes, source):
37612 5e71dea79aae wireproto: move value encoding functions to wireprototypes (API) Gregory Szorc <gregory.szorc@gmail.com> parents: 37546 diff changeset	553 n = wireprototypes.encodelist(nodes)
43077 687b865b95ad formatting: byteify all mercurial/ and hgext/ string literals Augie Fackler <augie@google.com> parents: 43076 diff changeset	554 f = self._callcompressable(b"changegroup", roots=n)
687b865b95ad formatting: byteify all mercurial/ and hgext/ string literals Augie Fackler <augie@google.com> parents: 43076 diff changeset	555 return changegroupmod.cg1unpacker(f, b'UN')
33827 dedab036215d wireproto: use new peer interface Gregory Szorc <gregory.szorc@gmail.com> parents: 33788 diff changeset	556
37635 cc8c06835097 wireproto: convert legacy commands to command executor Gregory Szorc <gregory.szorc@gmail.com> parents: 37633 diff changeset	557 def changegroupsubset(self, bases, heads, source):
43077 687b865b95ad formatting: byteify all mercurial/ and hgext/ string literals Augie Fackler <augie@google.com> parents: 43076 diff changeset	558 self.requirecap(b'changegroupsubset', _(b'look up remote changes'))
37612 5e71dea79aae wireproto: move value encoding functions to wireprototypes (API) Gregory Szorc <gregory.szorc@gmail.com> parents: 37546 diff changeset	559 bases = wireprototypes.encodelist(bases)
5e71dea79aae wireproto: move value encoding functions to wireprototypes (API) Gregory Szorc <gregory.szorc@gmail.com> parents: 37546 diff changeset	560 heads = wireprototypes.encodelist(heads)
43076 2372284d9457 formatting: blacken the codebase Augie Fackler <augie@google.com> parents: 42824 diff changeset	561 f = self._callcompressable(
43077 687b865b95ad formatting: byteify all mercurial/ and hgext/ string literals Augie Fackler <augie@google.com> parents: 43076 diff changeset	562 b"changegroupsubset", bases=bases, heads=heads
43076 2372284d9457 formatting: blacken the codebase Augie Fackler <augie@google.com> parents: 42824 diff changeset	563 )
43077 687b865b95ad formatting: byteify all mercurial/ and hgext/ string literals Augie Fackler <augie@google.com> parents: 43076 diff changeset	564 return changegroupmod.cg1unpacker(f, b'UN')
33827 dedab036215d wireproto: use new peer interface Gregory Szorc <gregory.szorc@gmail.com> parents: 33788 diff changeset	565
37320 39f7d4ee8bcd repository: port peer interfaces to zope.interface Gregory Szorc <gregory.szorc@gmail.com> parents: 37295 diff changeset	566 # End of ipeerlegacycommands interface.
33827 dedab036215d wireproto: use new peer interface Gregory Szorc <gregory.szorc@gmail.com> parents: 33788 diff changeset	567
dedab036215d wireproto: use new peer interface Gregory Szorc <gregory.szorc@gmail.com> parents: 33788 diff changeset	568 def _submitbatch(self, req):
dedab036215d wireproto: use new peer interface Gregory Szorc <gregory.szorc@gmail.com> parents: 33788 diff changeset	569 """run batch request <req> on the server
dedab036215d wireproto: use new peer interface Gregory Szorc <gregory.szorc@gmail.com> parents: 33788 diff changeset	570
dedab036215d wireproto: use new peer interface Gregory Szorc <gregory.szorc@gmail.com> parents: 33788 diff changeset	571 Returns an iterator of the raw responses from the server.
dedab036215d wireproto: use new peer interface Gregory Szorc <gregory.szorc@gmail.com> parents: 33788 diff changeset	572 """
36951 4901d1e22b27 peer-request: include more details about batch commands Boris Feld <boris.feld@octobus.net> parents: 36848 diff changeset	573 ui = self.ui
43077 687b865b95ad formatting: byteify all mercurial/ and hgext/ string literals Augie Fackler <augie@google.com> parents: 43076 diff changeset	574 if ui.debugflag and ui.configbool(b'devel', b'debug.peer-request'):
687b865b95ad formatting: byteify all mercurial/ and hgext/ string literals Augie Fackler <augie@google.com> parents: 43076 diff changeset	575 ui.debug(b'devel-peer-request: batched-content\n')
36951 4901d1e22b27 peer-request: include more details about batch commands Boris Feld <boris.feld@octobus.net> parents: 36848 diff changeset	576 for op, args in req:
43077 687b865b95ad formatting: byteify all mercurial/ and hgext/ string literals Augie Fackler <augie@google.com> parents: 43076 diff changeset	577 msg = b'devel-peer-request: - %s (%d arguments)\n'
36951 4901d1e22b27 peer-request: include more details about batch commands Boris Feld <boris.feld@octobus.net> parents: 36848 diff changeset	578 ui.debug(msg % (op, len(args)))
4901d1e22b27 peer-request: include more details about batch commands Boris Feld <boris.feld@octobus.net> parents: 36848 diff changeset	579
37612 5e71dea79aae wireproto: move value encoding functions to wireprototypes (API) Gregory Szorc <gregory.szorc@gmail.com> parents: 37546 diff changeset	580 unescapearg = wireprototypes.unescapebatcharg
5e71dea79aae wireproto: move value encoding functions to wireprototypes (API) Gregory Szorc <gregory.szorc@gmail.com> parents: 37546 diff changeset	581
43077 687b865b95ad formatting: byteify all mercurial/ and hgext/ string literals Augie Fackler <augie@google.com> parents: 43076 diff changeset	582 rsp = self._callstream(b"batch", cmds=encodebatchcmds(req))
33827 dedab036215d wireproto: use new peer interface Gregory Szorc <gregory.szorc@gmail.com> parents: 33788 diff changeset	583 chunk = rsp.read(1024)
dedab036215d wireproto: use new peer interface Gregory Szorc <gregory.szorc@gmail.com> parents: 33788 diff changeset	584 work = [chunk]
dedab036215d wireproto: use new peer interface Gregory Szorc <gregory.szorc@gmail.com> parents: 33788 diff changeset	585 while chunk:
43077 687b865b95ad formatting: byteify all mercurial/ and hgext/ string literals Augie Fackler <augie@google.com> parents: 43076 diff changeset	586 while b';' not in chunk and chunk:
33827 dedab036215d wireproto: use new peer interface Gregory Szorc <gregory.szorc@gmail.com> parents: 33788 diff changeset	587 chunk = rsp.read(1024)
dedab036215d wireproto: use new peer interface Gregory Szorc <gregory.szorc@gmail.com> parents: 33788 diff changeset	588 work.append(chunk)
43077 687b865b95ad formatting: byteify all mercurial/ and hgext/ string literals Augie Fackler <augie@google.com> parents: 43076 diff changeset	589 merged = b''.join(work)
687b865b95ad formatting: byteify all mercurial/ and hgext/ string literals Augie Fackler <augie@google.com> parents: 43076 diff changeset	590 while b';' in merged:
687b865b95ad formatting: byteify all mercurial/ and hgext/ string literals Augie Fackler <augie@google.com> parents: 43076 diff changeset	591 one, merged = merged.split(b';', 1)
33827 dedab036215d wireproto: use new peer interface Gregory Szorc <gregory.szorc@gmail.com> parents: 33788 diff changeset	592 yield unescapearg(one)
dedab036215d wireproto: use new peer interface Gregory Szorc <gregory.szorc@gmail.com> parents: 33788 diff changeset	593 chunk = rsp.read(1024)
dedab036215d wireproto: use new peer interface Gregory Szorc <gregory.szorc@gmail.com> parents: 33788 diff changeset	594 work = [merged, chunk]
43077 687b865b95ad formatting: byteify all mercurial/ and hgext/ string literals Augie Fackler <augie@google.com> parents: 43076 diff changeset	595 yield unescapearg(b''.join(work))
33827 dedab036215d wireproto: use new peer interface Gregory Szorc <gregory.szorc@gmail.com> parents: 33788 diff changeset	596
dedab036215d wireproto: use new peer interface Gregory Szorc <gregory.szorc@gmail.com> parents: 33788 diff changeset	597 def _submitone(self, op, args):
34739 b880cc11da5d wireproto: bounce kwargs to/from bytes/str as needed Augie Fackler <augie@google.com> parents: 34731 diff changeset	598 return self._call(op, **pycompat.strkwargs(args))
33827 dedab036215d wireproto: use new peer interface Gregory Szorc <gregory.szorc@gmail.com> parents: 33788 diff changeset	599
14048 58e58406ed19 wireproto: add test for new optional arg missing on server Peter Arrenbrecht <peter.arrenbrecht@gmail.com> parents: 13942 diff changeset	600 def debugwireargs(self, one, two, three=None, four=None, five=None):
13720 9c4e04fe267e debug: add debugwireargs to test argument passing over the wire Peter Arrenbrecht <peter.arrenbrecht@gmail.com> parents: 13450 diff changeset	601 # don't pass optional arguments left at their default value
9c4e04fe267e debug: add debugwireargs to test argument passing over the wire Peter Arrenbrecht <peter.arrenbrecht@gmail.com> parents: 13450 diff changeset	602 opts = {}
9c4e04fe267e debug: add debugwireargs to test argument passing over the wire Peter Arrenbrecht <peter.arrenbrecht@gmail.com> parents: 13450 diff changeset	603 if three is not None:
43554 9f70512ae2cf cleanup: remove pointless r-prefixes on single-quoted strings Augie Fackler <augie@google.com> parents: 43117 diff changeset	604 opts['three'] = three
13720 9c4e04fe267e debug: add debugwireargs to test argument passing over the wire Peter Arrenbrecht <peter.arrenbrecht@gmail.com> parents: 13450 diff changeset	605 if four is not None:
43554 9f70512ae2cf cleanup: remove pointless r-prefixes on single-quoted strings Augie Fackler <augie@google.com> parents: 43117 diff changeset	606 opts['four'] = four
43077 687b865b95ad formatting: byteify all mercurial/ and hgext/ string literals Augie Fackler <augie@google.com> parents: 43076 diff changeset	607 return self._call(b'debugwireargs', one=one, two=two, **opts)
13720 9c4e04fe267e debug: add debugwireargs to test argument passing over the wire Peter Arrenbrecht <peter.arrenbrecht@gmail.com> parents: 13450 diff changeset	608
20904 3dbe6bcd7f62 wireproto: document protocol specific function of wirepeer Pierre-Yves David <pierre-yves.david@fb.com> parents: 20903 diff changeset	609 def _call(self, cmd, **args):
3dbe6bcd7f62 wireproto: document protocol specific function of wirepeer Pierre-Yves David <pierre-yves.david@fb.com> parents: 20903 diff changeset	610 """execute <cmd> on the server
3dbe6bcd7f62 wireproto: document protocol specific function of wirepeer Pierre-Yves David <pierre-yves.david@fb.com> parents: 20903 diff changeset	611
3dbe6bcd7f62 wireproto: document protocol specific function of wirepeer Pierre-Yves David <pierre-yves.david@fb.com> parents: 20903 diff changeset	612 The command is expected to return a simple string.
3dbe6bcd7f62 wireproto: document protocol specific function of wirepeer Pierre-Yves David <pierre-yves.david@fb.com> parents: 20903 diff changeset	613
3dbe6bcd7f62 wireproto: document protocol specific function of wirepeer Pierre-Yves David <pierre-yves.david@fb.com> parents: 20903 diff changeset	614 returns the server reply as a string."""
3dbe6bcd7f62 wireproto: document protocol specific function of wirepeer Pierre-Yves David <pierre-yves.david@fb.com> parents: 20903 diff changeset	615 raise NotImplementedError()
3dbe6bcd7f62 wireproto: document protocol specific function of wirepeer Pierre-Yves David <pierre-yves.david@fb.com> parents: 20903 diff changeset	616
3dbe6bcd7f62 wireproto: document protocol specific function of wirepeer Pierre-Yves David <pierre-yves.david@fb.com> parents: 20903 diff changeset	617 def _callstream(self, cmd, **args):
3dbe6bcd7f62 wireproto: document protocol specific function of wirepeer Pierre-Yves David <pierre-yves.david@fb.com> parents: 20903 diff changeset	618 """execute <cmd> on the server
3dbe6bcd7f62 wireproto: document protocol specific function of wirepeer Pierre-Yves David <pierre-yves.david@fb.com> parents: 20903 diff changeset	619
28435 176736afa886 wireproto: document quirk of _callstream between http and ssh Augie Fackler <augie@google.com> parents: 27633 diff changeset	620 The command is expected to return a stream. Note that if the
176736afa886 wireproto: document quirk of _callstream between http and ssh Augie Fackler <augie@google.com> parents: 27633 diff changeset	621 command doesn't return a stream, _callstream behaves
176736afa886 wireproto: document quirk of _callstream between http and ssh Augie Fackler <augie@google.com> parents: 27633 diff changeset	622 differently for ssh and http peers.
20904 3dbe6bcd7f62 wireproto: document protocol specific function of wirepeer Pierre-Yves David <pierre-yves.david@fb.com> parents: 20903 diff changeset	623
28435 176736afa886 wireproto: document quirk of _callstream between http and ssh Augie Fackler <augie@google.com> parents: 27633 diff changeset	624 returns the server reply as a file like object.
176736afa886 wireproto: document quirk of _callstream between http and ssh Augie Fackler <augie@google.com> parents: 27633 diff changeset	625 """
20904 3dbe6bcd7f62 wireproto: document protocol specific function of wirepeer Pierre-Yves David <pierre-yves.david@fb.com> parents: 20903 diff changeset	626 raise NotImplementedError()
3dbe6bcd7f62 wireproto: document protocol specific function of wirepeer Pierre-Yves David <pierre-yves.david@fb.com> parents: 20903 diff changeset	627
20905 167047ba3cfa wireproto: drop the _decompress method in favor a new call type Pierre-Yves David <pierre-yves.david@fb.com> parents: 20904 diff changeset	628 def _callcompressable(self, cmd, **args):
167047ba3cfa wireproto: drop the _decompress method in favor a new call type Pierre-Yves David <pierre-yves.david@fb.com> parents: 20904 diff changeset	629 """execute <cmd> on the server
167047ba3cfa wireproto: drop the _decompress method in favor a new call type Pierre-Yves David <pierre-yves.david@fb.com> parents: 20904 diff changeset	630
167047ba3cfa wireproto: drop the _decompress method in favor a new call type Pierre-Yves David <pierre-yves.david@fb.com> parents: 20904 diff changeset	631 The command is expected to return a stream.
167047ba3cfa wireproto: drop the _decompress method in favor a new call type Pierre-Yves David <pierre-yves.david@fb.com> parents: 20904 diff changeset	632
21024 7731a2281cf0 spelling: fixes from spell checker Mads Kiilerich <madski@unity3d.com> parents: 20969 diff changeset	633 The stream may have been compressed in some implementations. This
20905 167047ba3cfa wireproto: drop the _decompress method in favor a new call type Pierre-Yves David <pierre-yves.david@fb.com> parents: 20904 diff changeset	634 function takes care of the decompression. This is the only difference
167047ba3cfa wireproto: drop the _decompress method in favor a new call type Pierre-Yves David <pierre-yves.david@fb.com> parents: 20904 diff changeset	635 with _callstream.
167047ba3cfa wireproto: drop the _decompress method in favor a new call type Pierre-Yves David <pierre-yves.david@fb.com> parents: 20904 diff changeset	636
167047ba3cfa wireproto: drop the _decompress method in favor a new call type Pierre-Yves David <pierre-yves.david@fb.com> parents: 20904 diff changeset	637 returns the server reply as a file like object.
167047ba3cfa wireproto: drop the _decompress method in favor a new call type Pierre-Yves David <pierre-yves.david@fb.com> parents: 20904 diff changeset	638 """
167047ba3cfa wireproto: drop the _decompress method in favor a new call type Pierre-Yves David <pierre-yves.david@fb.com> parents: 20904 diff changeset	639 raise NotImplementedError()
167047ba3cfa wireproto: drop the _decompress method in favor a new call type Pierre-Yves David <pierre-yves.david@fb.com> parents: 20904 diff changeset	640
20904 3dbe6bcd7f62 wireproto: document protocol specific function of wirepeer Pierre-Yves David <pierre-yves.david@fb.com> parents: 20903 diff changeset	641 def _callpush(self, cmd, fp, **args):
3dbe6bcd7f62 wireproto: document protocol specific function of wirepeer Pierre-Yves David <pierre-yves.david@fb.com> parents: 20903 diff changeset	642 """execute a <cmd> on server
3dbe6bcd7f62 wireproto: document protocol specific function of wirepeer Pierre-Yves David <pierre-yves.david@fb.com> parents: 20903 diff changeset	643
3dbe6bcd7f62 wireproto: document protocol specific function of wirepeer Pierre-Yves David <pierre-yves.david@fb.com> parents: 20903 diff changeset	644 The command is expected to be related to a push. Push has a special
3dbe6bcd7f62 wireproto: document protocol specific function of wirepeer Pierre-Yves David <pierre-yves.david@fb.com> parents: 20903 diff changeset	645 return method.
3dbe6bcd7f62 wireproto: document protocol specific function of wirepeer Pierre-Yves David <pierre-yves.david@fb.com> parents: 20903 diff changeset	646
3dbe6bcd7f62 wireproto: document protocol specific function of wirepeer Pierre-Yves David <pierre-yves.david@fb.com> parents: 20903 diff changeset	647 returns the server reply as a (ret, output) tuple. ret is either
3dbe6bcd7f62 wireproto: document protocol specific function of wirepeer Pierre-Yves David <pierre-yves.david@fb.com> parents: 20903 diff changeset	648 empty (error) or a stringified int.
3dbe6bcd7f62 wireproto: document protocol specific function of wirepeer Pierre-Yves David <pierre-yves.david@fb.com> parents: 20903 diff changeset	649 """
3dbe6bcd7f62 wireproto: document protocol specific function of wirepeer Pierre-Yves David <pierre-yves.david@fb.com> parents: 20903 diff changeset	650 raise NotImplementedError()
3dbe6bcd7f62 wireproto: document protocol specific function of wirepeer Pierre-Yves David <pierre-yves.david@fb.com> parents: 20903 diff changeset	651
21072 0879352d67d8 wireproto: add a _calltwowaystream method on wirepeer Pierre-Yves David <pierre-yves.david@fb.com> parents: 21069 diff changeset	652 def _calltwowaystream(self, cmd, fp, **args):
0879352d67d8 wireproto: add a _calltwowaystream method on wirepeer Pierre-Yves David <pierre-yves.david@fb.com> parents: 21069 diff changeset	653 """execute <cmd> on server
0879352d67d8 wireproto: add a _calltwowaystream method on wirepeer Pierre-Yves David <pierre-yves.david@fb.com> parents: 21069 diff changeset	654
0879352d67d8 wireproto: add a _calltwowaystream method on wirepeer Pierre-Yves David <pierre-yves.david@fb.com> parents: 21069 diff changeset	655 The command will send a stream to the server and get a stream in reply.
0879352d67d8 wireproto: add a _calltwowaystream method on wirepeer Pierre-Yves David <pierre-yves.david@fb.com> parents: 21069 diff changeset	656 """
0879352d67d8 wireproto: add a _calltwowaystream method on wirepeer Pierre-Yves David <pierre-yves.david@fb.com> parents: 21069 diff changeset	657 raise NotImplementedError()
0879352d67d8 wireproto: add a _calltwowaystream method on wirepeer Pierre-Yves David <pierre-yves.david@fb.com> parents: 21069 diff changeset	658
20904 3dbe6bcd7f62 wireproto: document protocol specific function of wirepeer Pierre-Yves David <pierre-yves.david@fb.com> parents: 20903 diff changeset	659 def _abort(self, exception):
45957 89a2afe31e82 formating: upgrade to black 20.8b1 Augie Fackler <raf@durin42.com> parents: 44060 diff changeset	660 """clearly abort the wire protocol connection and raise the exception"""
20904 3dbe6bcd7f62 wireproto: document protocol specific function of wirepeer Pierre-Yves David <pierre-yves.david@fb.com> parents: 20903 diff changeset	661 raise NotImplementedError()

Mercurial > public > mercurial-scm > hg-stable

annotate mercurial/wireprotov1peer.py @ 50675:60f9602b413e