mercurial-scm/hg: mercurial/wireprotov2server.py annotate

annotate mercurial/wireprotov2server.py @ 40329:ed55a0077490

wireprotov2: implement command for retrieving raw store files Implementing shallow clone of the changelog is hard. We want the 4.8 release to have a fast implementation of partial clone in wireprotov2. In order to achieve fast, we can't use deltas for transferring changelog and manifestlog data. Per discussions at the 4.8 sprint, this commit implements a somwwhat hacky and likely-to-be-changed-drastically-or-dropped command in wireprotov2 that facilitates access to raw store files, namely the changelog and manifestlog. Using this command, clients can perform a "stream clone" of sorts for just the changelog and manifestlog. This will allow clients to fetch the changelog and manifest revlogs, stream them to disk (which should be fast), then follow up filesdata requests for files revision data for a particular changeset. Differential Revision: https://phab.mercurial-scm.org/D5134

author	Gregory Szorc <gregory.szorc@gmail.com>
date	Tue, 16 Oct 2018 21:31:21 +0200
parents	46a40bce3ae0
children	abbd077965c0

rev	line source
5598 d534ba1c4eb4 separate the wire protocol commands from the user interface commands Dirkjan Ochtman <dirkjan@ochtman.nl> parents: diff changeset	1 # Copyright 21 May 2005 - (c) 2005 Jake Edge <jake@edge2.net>
d534ba1c4eb4 separate the wire protocol commands from the user interface commands Dirkjan Ochtman <dirkjan@ochtman.nl> parents: diff changeset	2 # Copyright 2005-2007 Matt Mackall <mpm@selenic.com>
d534ba1c4eb4 separate the wire protocol commands from the user interface commands Dirkjan Ochtman <dirkjan@ochtman.nl> parents: diff changeset	3 #
8225 46293a0c7e9f updated license to be explicit about GPL version 2 Martin Geisler <mg@lazybytes.net> parents: 8109 diff changeset	4 # This software may be used and distributed according to the terms of the
10263 25e572394f5c Update license to GPLv2+ Matt Mackall <mpm@selenic.com> parents: 9713 diff changeset	5 # GNU General Public License version 2 or any later version.
5598 d534ba1c4eb4 separate the wire protocol commands from the user interface commands Dirkjan Ochtman <dirkjan@ochtman.nl> parents: diff changeset	6
27046 37fcfe52c68c hgweb: use absolute_import Yuya Nishihara <yuya@tcha.org> parents: 20903 diff changeset	7 from __future__ import absolute_import
37fcfe52c68c hgweb: use absolute_import Yuya Nishihara <yuya@tcha.org> parents: 20903 diff changeset	8
40178 46a40bce3ae0 wireprotov2: define and implement "filesdata" command Gregory Szorc <gregory.szorc@gmail.com> parents: 40177 diff changeset	9 import collections
36066 2ad145fbde54 wireprotoserver: add context manager mechanism for redirecting stdio Gregory Szorc <gregory.szorc@gmail.com> parents: 36065 diff changeset	10 import contextlib
40021 c537144fdbef wireprotov2: support response caching Gregory Szorc <gregory.szorc@gmail.com> parents: 39995 diff changeset	11 import hashlib
27046 37fcfe52c68c hgweb: use absolute_import Yuya Nishihara <yuya@tcha.org> parents: 20903 diff changeset	12
35859 1bf5263fe5cc wireprotoserver: move sshserver into module (API) Gregory Szorc <gregory.szorc@gmail.com> parents: 35858 diff changeset	13 from .i18n import _
39630 9c2c77c73f23 wireprotov2: define and implement "changesetdata" command Gregory Szorc <gregory.szorc@gmail.com> parents: 39559 diff changeset	14 from .node import (
39639 0e03e6a44dee wireprotov2: define and implement "filedata" command Gregory Szorc <gregory.szorc@gmail.com> parents: 39637 diff changeset	15 hex,
39630 9c2c77c73f23 wireprotov2: define and implement "changesetdata" command Gregory Szorc <gregory.szorc@gmail.com> parents: 39559 diff changeset	16 nullid,
9c2c77c73f23 wireprotov2: define and implement "changesetdata" command Gregory Szorc <gregory.szorc@gmail.com> parents: 39559 diff changeset	17 )
35856 ef3a24a023ec wireprotoserver: rename hgweb.protocol to wireprotoserver (API) Gregory Szorc <gregory.szorc@gmail.com> parents: 35750 diff changeset	18 from . import (
39630 9c2c77c73f23 wireprotov2: define and implement "changesetdata" command Gregory Szorc <gregory.szorc@gmail.com> parents: 39559 diff changeset	19 discovery,
37546 3a2367e6c6f2 wireproto: move version 2 command handlers to wireprotov2server Gregory Szorc <gregory.szorc@gmail.com> parents: 37545 diff changeset	20 encoding,
34509 e21f274cccea hgweb: in protocol adapter, avoid control reaching end of non-void function Augie Fackler <augie@google.com> parents: 33821 diff changeset	21 error,
40178 46a40bce3ae0 wireprotov2: define and implement "filesdata" command Gregory Szorc <gregory.szorc@gmail.com> parents: 40177 diff changeset	22 match as matchmod,
39811 ae20f52437e9 wireprotov2: advertise recognized path filter prefixes Gregory Szorc <gregory.szorc@gmail.com> parents: 39810 diff changeset	23 narrowspec,
34742 5a9cad0dfddb hgweb: when unpacking args from request form, convert to bytes Augie Fackler <augie@google.com> parents: 34740 diff changeset	24 pycompat,
40329 ed55a0077490 wireprotov2: implement command for retrieving raw store files Gregory Szorc <gregory.szorc@gmail.com> parents: 40178 diff changeset	25 streamclone,
ed55a0077490 wireprotov2: implement command for retrieving raw store files Gregory Szorc <gregory.szorc@gmail.com> parents: 40178 diff changeset	26 util,
37052 8c3c47362934 wireproto: implement basic frame reading and processing Gregory Szorc <gregory.szorc@gmail.com> parents: 37051 diff changeset	27 wireprotoframing,
36073 cd6ab329c5c7 wireprototypes: move wire protocol response types to new module Gregory Szorc <gregory.szorc@gmail.com> parents: 36072 diff changeset	28 wireprototypes,
27046 37fcfe52c68c hgweb: use absolute_import Yuya Nishihara <yuya@tcha.org> parents: 20903 diff changeset	29 )
37810 856f381ad74b interfaceutil: module to stub out zope.interface Gregory Szorc <gregory.szorc@gmail.com> parents: 37784 diff changeset	30 from .utils import (
40021 c537144fdbef wireprotov2: support response caching Gregory Szorc <gregory.szorc@gmail.com> parents: 39995 diff changeset	31 cborutil,
37810 856f381ad74b interfaceutil: module to stub out zope.interface Gregory Szorc <gregory.szorc@gmail.com> parents: 37784 diff changeset	32 interfaceutil,
39837 69b4a5b89dc5 py3: cast exception to bytes Gregory Szorc <gregory.szorc@gmail.com> parents: 39828 diff changeset	33 stringutil,
37810 856f381ad74b interfaceutil: module to stub out zope.interface Gregory Szorc <gregory.szorc@gmail.com> parents: 37784 diff changeset	34 )
35856 ef3a24a023ec wireprotoserver: rename hgweb.protocol to wireprotoserver (API) Gregory Szorc <gregory.szorc@gmail.com> parents: 35750 diff changeset	35
40133 762ef19a07e3 wireprotov2: send protocol settings frame from client Gregory Szorc <gregory.szorc@gmail.com> parents: 40129 diff changeset	36 FRAMINGTYPE = b'application/mercurial-exp-framing-0006'
11595 368cd5325348 protocol: move hgweb protocol support back into protocol.py Matt Mackall <mpm@selenic.com> parents: 11594 diff changeset	37
37644 77c9ee77687c wireproto: rename HTTPV2 so it less like HTTP/2 Gregory Szorc <gregory.szorc@gmail.com> parents: 37557 diff changeset	38 HTTP_WIREPROTO_V2 = wireprototypes.HTTP_WIREPROTO_V2
35976 48a3a9283f09 sshpeer: initial definition and implementation of new SSH protocol Gregory Szorc <gregory.szorc@gmail.com> parents: 35900 diff changeset	39
37784 ee0d5e9d77b2 wireproto: move version 2 commands dict to wireprotov2server Gregory Szorc <gregory.szorc@gmail.com> parents: 37783 diff changeset	40 COMMANDS = wireprototypes.commanddict()
ee0d5e9d77b2 wireproto: move version 2 commands dict to wireprotov2server Gregory Szorc <gregory.szorc@gmail.com> parents: 37783 diff changeset	41
40021 c537144fdbef wireprotov2: support response caching Gregory Szorc <gregory.szorc@gmail.com> parents: 39995 diff changeset	42 # Value inserted into cache key computation function. Change the value to
c537144fdbef wireprotov2: support response caching Gregory Szorc <gregory.szorc@gmail.com> parents: 39995 diff changeset	43 # force new cache keys for every command request. This should be done when
c537144fdbef wireprotov2: support response caching Gregory Szorc <gregory.szorc@gmail.com> parents: 39995 diff changeset	44 # there is a change to how caching works, etc.
c537144fdbef wireprotov2: support response caching Gregory Szorc <gregory.szorc@gmail.com> parents: 39995 diff changeset	45 GLOBAL_CACHE_VERSION = 1
c537144fdbef wireprotov2: support response caching Gregory Szorc <gregory.szorc@gmail.com> parents: 39995 diff changeset	46
37545 93397c4633f6 wireproto: extract HTTP version 2 code to own module Gregory Szorc <gregory.szorc@gmail.com> parents: 37535 diff changeset	47 def handlehttpv2request(rctx, req, res, checkperm, urlparts):
37047 fddcb51b5084 wireproto: define permissions-based routing of HTTPv2 wire protocol Gregory Szorc <gregory.szorc@gmail.com> parents: 37046 diff changeset	48 from .hgweb import common as hgwebcommon
fddcb51b5084 wireproto: define permissions-based routing of HTTPv2 wire protocol Gregory Szorc <gregory.szorc@gmail.com> parents: 37046 diff changeset	49
fddcb51b5084 wireproto: define permissions-based routing of HTTPv2 wire protocol Gregory Szorc <gregory.szorc@gmail.com> parents: 37046 diff changeset	50 # URL space looks like: <permissions>/<command>, where <permission> can
fddcb51b5084 wireproto: define permissions-based routing of HTTPv2 wire protocol Gregory Szorc <gregory.szorc@gmail.com> parents: 37046 diff changeset	51 # be ``ro`` or ``rw`` to signal read-only or read-write, respectively.
fddcb51b5084 wireproto: define permissions-based routing of HTTPv2 wire protocol Gregory Szorc <gregory.szorc@gmail.com> parents: 37046 diff changeset	52
fddcb51b5084 wireproto: define permissions-based routing of HTTPv2 wire protocol Gregory Szorc <gregory.szorc@gmail.com> parents: 37046 diff changeset	53 # Root URL does nothing meaningful... yet.
fddcb51b5084 wireproto: define permissions-based routing of HTTPv2 wire protocol Gregory Szorc <gregory.szorc@gmail.com> parents: 37046 diff changeset	54 if not urlparts:
fddcb51b5084 wireproto: define permissions-based routing of HTTPv2 wire protocol Gregory Szorc <gregory.szorc@gmail.com> parents: 37046 diff changeset	55 res.status = b'200 OK'
fddcb51b5084 wireproto: define permissions-based routing of HTTPv2 wire protocol Gregory Szorc <gregory.szorc@gmail.com> parents: 37046 diff changeset	56 res.headers[b'Content-Type'] = b'text/plain'
fddcb51b5084 wireproto: define permissions-based routing of HTTPv2 wire protocol Gregory Szorc <gregory.szorc@gmail.com> parents: 37046 diff changeset	57 res.setbodybytes(_('HTTP version 2 API handler'))
fddcb51b5084 wireproto: define permissions-based routing of HTTPv2 wire protocol Gregory Szorc <gregory.szorc@gmail.com> parents: 37046 diff changeset	58 return
fddcb51b5084 wireproto: define permissions-based routing of HTTPv2 wire protocol Gregory Szorc <gregory.szorc@gmail.com> parents: 37046 diff changeset	59
fddcb51b5084 wireproto: define permissions-based routing of HTTPv2 wire protocol Gregory Szorc <gregory.szorc@gmail.com> parents: 37046 diff changeset	60 if len(urlparts) == 1:
fddcb51b5084 wireproto: define permissions-based routing of HTTPv2 wire protocol Gregory Szorc <gregory.szorc@gmail.com> parents: 37046 diff changeset	61 res.status = b'404 Not Found'
fddcb51b5084 wireproto: define permissions-based routing of HTTPv2 wire protocol Gregory Szorc <gregory.szorc@gmail.com> parents: 37046 diff changeset	62 res.headers[b'Content-Type'] = b'text/plain'
fddcb51b5084 wireproto: define permissions-based routing of HTTPv2 wire protocol Gregory Szorc <gregory.szorc@gmail.com> parents: 37046 diff changeset	63 res.setbodybytes(_('do not know how to process %s\n') %
fddcb51b5084 wireproto: define permissions-based routing of HTTPv2 wire protocol Gregory Szorc <gregory.szorc@gmail.com> parents: 37046 diff changeset	64 req.dispatchpath)
fddcb51b5084 wireproto: define permissions-based routing of HTTPv2 wire protocol Gregory Szorc <gregory.szorc@gmail.com> parents: 37046 diff changeset	65 return
fddcb51b5084 wireproto: define permissions-based routing of HTTPv2 wire protocol Gregory Szorc <gregory.szorc@gmail.com> parents: 37046 diff changeset	66
fddcb51b5084 wireproto: define permissions-based routing of HTTPv2 wire protocol Gregory Szorc <gregory.szorc@gmail.com> parents: 37046 diff changeset	67 permission, command = urlparts[0:2]
fddcb51b5084 wireproto: define permissions-based routing of HTTPv2 wire protocol Gregory Szorc <gregory.szorc@gmail.com> parents: 37046 diff changeset	68
fddcb51b5084 wireproto: define permissions-based routing of HTTPv2 wire protocol Gregory Szorc <gregory.szorc@gmail.com> parents: 37046 diff changeset	69 if permission not in (b'ro', b'rw'):
fddcb51b5084 wireproto: define permissions-based routing of HTTPv2 wire protocol Gregory Szorc <gregory.szorc@gmail.com> parents: 37046 diff changeset	70 res.status = b'404 Not Found'
fddcb51b5084 wireproto: define permissions-based routing of HTTPv2 wire protocol Gregory Szorc <gregory.szorc@gmail.com> parents: 37046 diff changeset	71 res.headers[b'Content-Type'] = b'text/plain'
fddcb51b5084 wireproto: define permissions-based routing of HTTPv2 wire protocol Gregory Szorc <gregory.szorc@gmail.com> parents: 37046 diff changeset	72 res.setbodybytes(_('unknown permission: %s') % permission)
fddcb51b5084 wireproto: define permissions-based routing of HTTPv2 wire protocol Gregory Szorc <gregory.szorc@gmail.com> parents: 37046 diff changeset	73 return
fddcb51b5084 wireproto: define permissions-based routing of HTTPv2 wire protocol Gregory Szorc <gregory.szorc@gmail.com> parents: 37046 diff changeset	74
37048 fc5e261915b9 wireproto: require POST for all HTTPv2 requests Gregory Szorc <gregory.szorc@gmail.com> parents: 37047 diff changeset	75 if req.method != 'POST':
fc5e261915b9 wireproto: require POST for all HTTPv2 requests Gregory Szorc <gregory.szorc@gmail.com> parents: 37047 diff changeset	76 res.status = b'405 Method Not Allowed'
fc5e261915b9 wireproto: require POST for all HTTPv2 requests Gregory Szorc <gregory.szorc@gmail.com> parents: 37047 diff changeset	77 res.headers[b'Allow'] = b'POST'
fc5e261915b9 wireproto: require POST for all HTTPv2 requests Gregory Szorc <gregory.szorc@gmail.com> parents: 37047 diff changeset	78 res.setbodybytes(_('commands require POST requests'))
fc5e261915b9 wireproto: require POST for all HTTPv2 requests Gregory Szorc <gregory.szorc@gmail.com> parents: 37047 diff changeset	79 return
fc5e261915b9 wireproto: require POST for all HTTPv2 requests Gregory Szorc <gregory.szorc@gmail.com> parents: 37047 diff changeset	80
37047 fddcb51b5084 wireproto: define permissions-based routing of HTTPv2 wire protocol Gregory Szorc <gregory.szorc@gmail.com> parents: 37046 diff changeset	81 # At some point we'll want to use our own API instead of recycling the
fddcb51b5084 wireproto: define permissions-based routing of HTTPv2 wire protocol Gregory Szorc <gregory.szorc@gmail.com> parents: 37046 diff changeset	82 # behavior of version 1 of the wire protocol...
fddcb51b5084 wireproto: define permissions-based routing of HTTPv2 wire protocol Gregory Szorc <gregory.szorc@gmail.com> parents: 37046 diff changeset	83 # TODO return reasonable responses - not responses that overload the
fddcb51b5084 wireproto: define permissions-based routing of HTTPv2 wire protocol Gregory Szorc <gregory.szorc@gmail.com> parents: 37046 diff changeset	84 # HTTP status line message for error reporting.
fddcb51b5084 wireproto: define permissions-based routing of HTTPv2 wire protocol Gregory Szorc <gregory.szorc@gmail.com> parents: 37046 diff changeset	85 try:
fddcb51b5084 wireproto: define permissions-based routing of HTTPv2 wire protocol Gregory Szorc <gregory.szorc@gmail.com> parents: 37046 diff changeset	86 checkperm(rctx, req, 'pull' if permission == b'ro' else 'push')
fddcb51b5084 wireproto: define permissions-based routing of HTTPv2 wire protocol Gregory Szorc <gregory.szorc@gmail.com> parents: 37046 diff changeset	87 except hgwebcommon.ErrorResponse as e:
fddcb51b5084 wireproto: define permissions-based routing of HTTPv2 wire protocol Gregory Szorc <gregory.szorc@gmail.com> parents: 37046 diff changeset	88 res.status = hgwebcommon.statusmessage(e.code, pycompat.bytestr(e))
fddcb51b5084 wireproto: define permissions-based routing of HTTPv2 wire protocol Gregory Szorc <gregory.szorc@gmail.com> parents: 37046 diff changeset	89 for k, v in e.headers:
fddcb51b5084 wireproto: define permissions-based routing of HTTPv2 wire protocol Gregory Szorc <gregory.szorc@gmail.com> parents: 37046 diff changeset	90 res.headers[k] = v
fddcb51b5084 wireproto: define permissions-based routing of HTTPv2 wire protocol Gregory Szorc <gregory.szorc@gmail.com> parents: 37046 diff changeset	91 res.setbodybytes('permission denied')
fddcb51b5084 wireproto: define permissions-based routing of HTTPv2 wire protocol Gregory Szorc <gregory.szorc@gmail.com> parents: 37046 diff changeset	92 return
fddcb51b5084 wireproto: define permissions-based routing of HTTPv2 wire protocol Gregory Szorc <gregory.szorc@gmail.com> parents: 37046 diff changeset	93
37052 8c3c47362934 wireproto: implement basic frame reading and processing Gregory Szorc <gregory.szorc@gmail.com> parents: 37051 diff changeset	94 # We have a special endpoint to reflect the request back at the client.
8c3c47362934 wireproto: implement basic frame reading and processing Gregory Szorc <gregory.szorc@gmail.com> parents: 37051 diff changeset	95 if command == b'debugreflect':
8c3c47362934 wireproto: implement basic frame reading and processing Gregory Szorc <gregory.szorc@gmail.com> parents: 37051 diff changeset	96 _processhttpv2reflectrequest(rctx.repo.ui, rctx.repo, req, res)
8c3c47362934 wireproto: implement basic frame reading and processing Gregory Szorc <gregory.szorc@gmail.com> parents: 37051 diff changeset	97 return
8c3c47362934 wireproto: implement basic frame reading and processing Gregory Szorc <gregory.szorc@gmail.com> parents: 37051 diff changeset	98
37059 bbea991635d0 wireproto: service multiple command requests per HTTP request Gregory Szorc <gregory.szorc@gmail.com> parents: 37057 diff changeset	99 # Extra commands that we handle that aren't really wire protocol
bbea991635d0 wireproto: service multiple command requests per HTTP request Gregory Szorc <gregory.szorc@gmail.com> parents: 37057 diff changeset	100 # commands. Think extra hard before making this hackery available to
bbea991635d0 wireproto: service multiple command requests per HTTP request Gregory Szorc <gregory.szorc@gmail.com> parents: 37057 diff changeset	101 # extension.
bbea991635d0 wireproto: service multiple command requests per HTTP request Gregory Szorc <gregory.szorc@gmail.com> parents: 37057 diff changeset	102 extracommands = {'multirequest'}
bbea991635d0 wireproto: service multiple command requests per HTTP request Gregory Szorc <gregory.szorc@gmail.com> parents: 37057 diff changeset	103
37784 ee0d5e9d77b2 wireproto: move version 2 commands dict to wireprotov2server Gregory Szorc <gregory.szorc@gmail.com> parents: 37783 diff changeset	104 if command not in COMMANDS and command not in extracommands:
37047 fddcb51b5084 wireproto: define permissions-based routing of HTTPv2 wire protocol Gregory Szorc <gregory.szorc@gmail.com> parents: 37046 diff changeset	105 res.status = b'404 Not Found'
fddcb51b5084 wireproto: define permissions-based routing of HTTPv2 wire protocol Gregory Szorc <gregory.szorc@gmail.com> parents: 37046 diff changeset	106 res.headers[b'Content-Type'] = b'text/plain'
fddcb51b5084 wireproto: define permissions-based routing of HTTPv2 wire protocol Gregory Szorc <gregory.szorc@gmail.com> parents: 37046 diff changeset	107 res.setbodybytes(_('unknown wire protocol command: %s\n') % command)
fddcb51b5084 wireproto: define permissions-based routing of HTTPv2 wire protocol Gregory Szorc <gregory.szorc@gmail.com> parents: 37046 diff changeset	108 return
fddcb51b5084 wireproto: define permissions-based routing of HTTPv2 wire protocol Gregory Szorc <gregory.szorc@gmail.com> parents: 37046 diff changeset	109
fddcb51b5084 wireproto: define permissions-based routing of HTTPv2 wire protocol Gregory Szorc <gregory.szorc@gmail.com> parents: 37046 diff changeset	110 repo = rctx.repo
fddcb51b5084 wireproto: define permissions-based routing of HTTPv2 wire protocol Gregory Szorc <gregory.szorc@gmail.com> parents: 37046 diff changeset	111 ui = repo.ui
fddcb51b5084 wireproto: define permissions-based routing of HTTPv2 wire protocol Gregory Szorc <gregory.szorc@gmail.com> parents: 37046 diff changeset	112
fddcb51b5084 wireproto: define permissions-based routing of HTTPv2 wire protocol Gregory Szorc <gregory.szorc@gmail.com> parents: 37046 diff changeset	113 proto = httpv2protocolhandler(req, ui)
fddcb51b5084 wireproto: define permissions-based routing of HTTPv2 wire protocol Gregory Szorc <gregory.szorc@gmail.com> parents: 37046 diff changeset	114
37784 ee0d5e9d77b2 wireproto: move version 2 commands dict to wireprotov2server Gregory Szorc <gregory.szorc@gmail.com> parents: 37783 diff changeset	115 if (not COMMANDS.commandavailable(command, proto)
37059 bbea991635d0 wireproto: service multiple command requests per HTTP request Gregory Szorc <gregory.szorc@gmail.com> parents: 37057 diff changeset	116 and command not in extracommands):
37047 fddcb51b5084 wireproto: define permissions-based routing of HTTPv2 wire protocol Gregory Szorc <gregory.szorc@gmail.com> parents: 37046 diff changeset	117 res.status = b'404 Not Found'
fddcb51b5084 wireproto: define permissions-based routing of HTTPv2 wire protocol Gregory Szorc <gregory.szorc@gmail.com> parents: 37046 diff changeset	118 res.headers[b'Content-Type'] = b'text/plain'
fddcb51b5084 wireproto: define permissions-based routing of HTTPv2 wire protocol Gregory Szorc <gregory.szorc@gmail.com> parents: 37046 diff changeset	119 res.setbodybytes(_('invalid wire protocol command: %s') % command)
fddcb51b5084 wireproto: define permissions-based routing of HTTPv2 wire protocol Gregory Szorc <gregory.szorc@gmail.com> parents: 37046 diff changeset	120 return
fddcb51b5084 wireproto: define permissions-based routing of HTTPv2 wire protocol Gregory Szorc <gregory.szorc@gmail.com> parents: 37046 diff changeset	121
37129 aaabd709df72 wireproto: review fixups Gregory Szorc <gregory.szorc@gmail.com> parents: 37120 diff changeset	122 # TODO consider cases where proxies may add additional Accept headers.
37051 40206e227412 wireproto: define and implement protocol for issuing requests Gregory Szorc <gregory.szorc@gmail.com> parents: 37050 diff changeset	123 if req.headers.get(b'Accept') != FRAMINGTYPE:
37050 37d7a1d18b97 wireproto: define content negotiation for HTTPv2 Gregory Szorc <gregory.szorc@gmail.com> parents: 37048 diff changeset	124 res.status = b'406 Not Acceptable'
37d7a1d18b97 wireproto: define content negotiation for HTTPv2 Gregory Szorc <gregory.szorc@gmail.com> parents: 37048 diff changeset	125 res.headers[b'Content-Type'] = b'text/plain'
37d7a1d18b97 wireproto: define content negotiation for HTTPv2 Gregory Szorc <gregory.szorc@gmail.com> parents: 37048 diff changeset	126 res.setbodybytes(_('client MUST specify Accept header with value: %s\n')
37051 40206e227412 wireproto: define and implement protocol for issuing requests Gregory Szorc <gregory.szorc@gmail.com> parents: 37050 diff changeset	127 % FRAMINGTYPE)
37050 37d7a1d18b97 wireproto: define content negotiation for HTTPv2 Gregory Szorc <gregory.szorc@gmail.com> parents: 37048 diff changeset	128 return
37d7a1d18b97 wireproto: define content negotiation for HTTPv2 Gregory Szorc <gregory.szorc@gmail.com> parents: 37048 diff changeset	129
37052 8c3c47362934 wireproto: implement basic frame reading and processing Gregory Szorc <gregory.szorc@gmail.com> parents: 37051 diff changeset	130 if req.headers.get(b'Content-Type') != FRAMINGTYPE:
37050 37d7a1d18b97 wireproto: define content negotiation for HTTPv2 Gregory Szorc <gregory.szorc@gmail.com> parents: 37048 diff changeset	131 res.status = b'415 Unsupported Media Type'
37d7a1d18b97 wireproto: define content negotiation for HTTPv2 Gregory Szorc <gregory.szorc@gmail.com> parents: 37048 diff changeset	132 # TODO we should send a response with appropriate media type,
37d7a1d18b97 wireproto: define content negotiation for HTTPv2 Gregory Szorc <gregory.szorc@gmail.com> parents: 37048 diff changeset	133 # since client does Accept it.
37d7a1d18b97 wireproto: define content negotiation for HTTPv2 Gregory Szorc <gregory.szorc@gmail.com> parents: 37048 diff changeset	134 res.headers[b'Content-Type'] = b'text/plain'
37d7a1d18b97 wireproto: define content negotiation for HTTPv2 Gregory Szorc <gregory.szorc@gmail.com> parents: 37048 diff changeset	135 res.setbodybytes(_('client MUST send Content-Type header with '
37051 40206e227412 wireproto: define and implement protocol for issuing requests Gregory Szorc <gregory.szorc@gmail.com> parents: 37050 diff changeset	136 'value: %s\n') % FRAMINGTYPE)
37050 37d7a1d18b97 wireproto: define content negotiation for HTTPv2 Gregory Szorc <gregory.szorc@gmail.com> parents: 37048 diff changeset	137 return
37d7a1d18b97 wireproto: define content negotiation for HTTPv2 Gregory Szorc <gregory.szorc@gmail.com> parents: 37048 diff changeset	138
37054 e7a012b60d6e wireproto: implement basic command dispatching for HTTPv2 Gregory Szorc <gregory.szorc@gmail.com> parents: 37053 diff changeset	139 _processhttpv2request(ui, repo, req, res, permission, command, proto)
37046 1cfef5693203 wireproto: support /api/* URL space for exposing APIs Gregory Szorc <gregory.szorc@gmail.com> parents: 36877 diff changeset	140
37052 8c3c47362934 wireproto: implement basic frame reading and processing Gregory Szorc <gregory.szorc@gmail.com> parents: 37051 diff changeset	141 def _processhttpv2reflectrequest(ui, repo, req, res):
8c3c47362934 wireproto: implement basic frame reading and processing Gregory Szorc <gregory.szorc@gmail.com> parents: 37051 diff changeset	142 """Reads unified frame protocol request and dumps out state to client.
8c3c47362934 wireproto: implement basic frame reading and processing Gregory Szorc <gregory.szorc@gmail.com> parents: 37051 diff changeset	143
8c3c47362934 wireproto: implement basic frame reading and processing Gregory Szorc <gregory.szorc@gmail.com> parents: 37051 diff changeset	144 This special endpoint can be used to help debug the wire protocol.
8c3c47362934 wireproto: implement basic frame reading and processing Gregory Szorc <gregory.szorc@gmail.com> parents: 37051 diff changeset	145
8c3c47362934 wireproto: implement basic frame reading and processing Gregory Szorc <gregory.szorc@gmail.com> parents: 37051 diff changeset	146 Instead of routing the request through the normal dispatch mechanism,
8c3c47362934 wireproto: implement basic frame reading and processing Gregory Szorc <gregory.szorc@gmail.com> parents: 37051 diff changeset	147 we instead read all frames, decode them, and feed them into our state
8c3c47362934 wireproto: implement basic frame reading and processing Gregory Szorc <gregory.szorc@gmail.com> parents: 37051 diff changeset	148 tracker. We then dump the log of all that activity back out to the
8c3c47362934 wireproto: implement basic frame reading and processing Gregory Szorc <gregory.szorc@gmail.com> parents: 37051 diff changeset	149 client.
8c3c47362934 wireproto: implement basic frame reading and processing Gregory Szorc <gregory.szorc@gmail.com> parents: 37051 diff changeset	150 """
8c3c47362934 wireproto: implement basic frame reading and processing Gregory Szorc <gregory.szorc@gmail.com> parents: 37051 diff changeset	151 import json
8c3c47362934 wireproto: implement basic frame reading and processing Gregory Szorc <gregory.szorc@gmail.com> parents: 37051 diff changeset	152
8c3c47362934 wireproto: implement basic frame reading and processing Gregory Szorc <gregory.szorc@gmail.com> parents: 37051 diff changeset	153 # Reflection APIs have a history of being abused, accidentally disclosing
8c3c47362934 wireproto: implement basic frame reading and processing Gregory Szorc <gregory.szorc@gmail.com> parents: 37051 diff changeset	154 # sensitive data, etc. So we have a config knob.
8c3c47362934 wireproto: implement basic frame reading and processing Gregory Szorc <gregory.szorc@gmail.com> parents: 37051 diff changeset	155 if not ui.configbool('experimental', 'web.api.debugreflect'):
8c3c47362934 wireproto: implement basic frame reading and processing Gregory Szorc <gregory.szorc@gmail.com> parents: 37051 diff changeset	156 res.status = b'404 Not Found'
8c3c47362934 wireproto: implement basic frame reading and processing Gregory Szorc <gregory.szorc@gmail.com> parents: 37051 diff changeset	157 res.headers[b'Content-Type'] = b'text/plain'
8c3c47362934 wireproto: implement basic frame reading and processing Gregory Szorc <gregory.szorc@gmail.com> parents: 37051 diff changeset	158 res.setbodybytes(_('debugreflect service not available'))
8c3c47362934 wireproto: implement basic frame reading and processing Gregory Szorc <gregory.szorc@gmail.com> parents: 37051 diff changeset	159 return
8c3c47362934 wireproto: implement basic frame reading and processing Gregory Szorc <gregory.szorc@gmail.com> parents: 37051 diff changeset	160
8c3c47362934 wireproto: implement basic frame reading and processing Gregory Szorc <gregory.szorc@gmail.com> parents: 37051 diff changeset	161 # We assume we have a unified framing protocol request body.
8c3c47362934 wireproto: implement basic frame reading and processing Gregory Szorc <gregory.szorc@gmail.com> parents: 37051 diff changeset	162
40129 293835e0fff7 wireprotov2: pass ui into clientreactor and serverreactor Gregory Szorc <gregory.szorc@gmail.com> parents: 40122 diff changeset	163 reactor = wireprotoframing.serverreactor(ui)
37052 8c3c47362934 wireproto: implement basic frame reading and processing Gregory Szorc <gregory.szorc@gmail.com> parents: 37051 diff changeset	164 states = []
8c3c47362934 wireproto: implement basic frame reading and processing Gregory Szorc <gregory.szorc@gmail.com> parents: 37051 diff changeset	165
8c3c47362934 wireproto: implement basic frame reading and processing Gregory Szorc <gregory.szorc@gmail.com> parents: 37051 diff changeset	166 while True:
8c3c47362934 wireproto: implement basic frame reading and processing Gregory Szorc <gregory.szorc@gmail.com> parents: 37051 diff changeset	167 frame = wireprotoframing.readframe(req.bodyfh)
8c3c47362934 wireproto: implement basic frame reading and processing Gregory Szorc <gregory.szorc@gmail.com> parents: 37051 diff changeset	168
8c3c47362934 wireproto: implement basic frame reading and processing Gregory Szorc <gregory.szorc@gmail.com> parents: 37051 diff changeset	169 if not frame:
8c3c47362934 wireproto: implement basic frame reading and processing Gregory Szorc <gregory.szorc@gmail.com> parents: 37051 diff changeset	170 states.append(b'received: <no frame>')
8c3c47362934 wireproto: implement basic frame reading and processing Gregory Szorc <gregory.szorc@gmail.com> parents: 37051 diff changeset	171 break
8c3c47362934 wireproto: implement basic frame reading and processing Gregory Szorc <gregory.szorc@gmail.com> parents: 37051 diff changeset	172
37061 884a0c1604ad wireproto: define attr-based classes for representing frames Gregory Szorc <gregory.szorc@gmail.com> parents: 37059 diff changeset	173 states.append(b'received: %d %d %d %s' % (frame.typeid, frame.flags,
884a0c1604ad wireproto: define attr-based classes for representing frames Gregory Szorc <gregory.szorc@gmail.com> parents: 37059 diff changeset	174 frame.requestid,
884a0c1604ad wireproto: define attr-based classes for representing frames Gregory Szorc <gregory.szorc@gmail.com> parents: 37059 diff changeset	175 frame.payload))
37052 8c3c47362934 wireproto: implement basic frame reading and processing Gregory Szorc <gregory.szorc@gmail.com> parents: 37051 diff changeset	176
37061 884a0c1604ad wireproto: define attr-based classes for representing frames Gregory Szorc <gregory.szorc@gmail.com> parents: 37059 diff changeset	177 action, meta = reactor.onframerecv(frame)
37052 8c3c47362934 wireproto: implement basic frame reading and processing Gregory Szorc <gregory.szorc@gmail.com> parents: 37051 diff changeset	178 states.append(json.dumps((action, meta), sort_keys=True,
8c3c47362934 wireproto: implement basic frame reading and processing Gregory Szorc <gregory.szorc@gmail.com> parents: 37051 diff changeset	179 separators=(', ', ': ')))
8c3c47362934 wireproto: implement basic frame reading and processing Gregory Szorc <gregory.szorc@gmail.com> parents: 37051 diff changeset	180
37056 861e9d37e56e wireproto: buffer output frames when in half duplex mode Gregory Szorc <gregory.szorc@gmail.com> parents: 37055 diff changeset	181 action, meta = reactor.oninputeof()
861e9d37e56e wireproto: buffer output frames when in half duplex mode Gregory Szorc <gregory.szorc@gmail.com> parents: 37055 diff changeset	182 meta['action'] = action
861e9d37e56e wireproto: buffer output frames when in half duplex mode Gregory Szorc <gregory.szorc@gmail.com> parents: 37055 diff changeset	183 states.append(json.dumps(meta, sort_keys=True, separators=(', ',': ')))
861e9d37e56e wireproto: buffer output frames when in half duplex mode Gregory Szorc <gregory.szorc@gmail.com> parents: 37055 diff changeset	184
37052 8c3c47362934 wireproto: implement basic frame reading and processing Gregory Szorc <gregory.szorc@gmail.com> parents: 37051 diff changeset	185 res.status = b'200 OK'
8c3c47362934 wireproto: implement basic frame reading and processing Gregory Szorc <gregory.szorc@gmail.com> parents: 37051 diff changeset	186 res.headers[b'Content-Type'] = b'text/plain'
8c3c47362934 wireproto: implement basic frame reading and processing Gregory Szorc <gregory.szorc@gmail.com> parents: 37051 diff changeset	187 res.setbodybytes(b'\n'.join(states))
8c3c47362934 wireproto: implement basic frame reading and processing Gregory Szorc <gregory.szorc@gmail.com> parents: 37051 diff changeset	188
37054 e7a012b60d6e wireproto: implement basic command dispatching for HTTPv2 Gregory Szorc <gregory.szorc@gmail.com> parents: 37053 diff changeset	189 def _processhttpv2request(ui, repo, req, res, authedperm, reqcommand, proto):
e7a012b60d6e wireproto: implement basic command dispatching for HTTPv2 Gregory Szorc <gregory.szorc@gmail.com> parents: 37053 diff changeset	190 """Post-validation handler for HTTPv2 requests.
e7a012b60d6e wireproto: implement basic command dispatching for HTTPv2 Gregory Szorc <gregory.szorc@gmail.com> parents: 37053 diff changeset	191
e7a012b60d6e wireproto: implement basic command dispatching for HTTPv2 Gregory Szorc <gregory.szorc@gmail.com> parents: 37053 diff changeset	192 Called when the HTTP request contains unified frame-based protocol
e7a012b60d6e wireproto: implement basic command dispatching for HTTPv2 Gregory Szorc <gregory.szorc@gmail.com> parents: 37053 diff changeset	193 frames for evaluation.
e7a012b60d6e wireproto: implement basic command dispatching for HTTPv2 Gregory Szorc <gregory.szorc@gmail.com> parents: 37053 diff changeset	194 """
37056 861e9d37e56e wireproto: buffer output frames when in half duplex mode Gregory Szorc <gregory.szorc@gmail.com> parents: 37055 diff changeset	195 # TODO Some HTTP clients are full duplex and can receive data before
861e9d37e56e wireproto: buffer output frames when in half duplex mode Gregory Szorc <gregory.szorc@gmail.com> parents: 37055 diff changeset	196 # the entire request is transmitted. Figure out a way to indicate support
861e9d37e56e wireproto: buffer output frames when in half duplex mode Gregory Szorc <gregory.szorc@gmail.com> parents: 37055 diff changeset	197 # for that so we can opt into full duplex mode.
40129 293835e0fff7 wireprotov2: pass ui into clientreactor and serverreactor Gregory Szorc <gregory.szorc@gmail.com> parents: 40122 diff changeset	198 reactor = wireprotoframing.serverreactor(ui, deferoutput=True)
37054 e7a012b60d6e wireproto: implement basic command dispatching for HTTPv2 Gregory Szorc <gregory.szorc@gmail.com> parents: 37053 diff changeset	199 seencommand = False
e7a012b60d6e wireproto: implement basic command dispatching for HTTPv2 Gregory Szorc <gregory.szorc@gmail.com> parents: 37053 diff changeset	200
40138 b5bf3dd6ec5b wireprotov2: send content encoded frames from server Gregory Szorc <gregory.szorc@gmail.com> parents: 40133 diff changeset	201 outstream = None
37289 5fadc63ac99f wireproto: explicit API to create outgoing streams Gregory Szorc <gregory.szorc@gmail.com> parents: 37288 diff changeset	202
37054 e7a012b60d6e wireproto: implement basic command dispatching for HTTPv2 Gregory Szorc <gregory.szorc@gmail.com> parents: 37053 diff changeset	203 while True:
e7a012b60d6e wireproto: implement basic command dispatching for HTTPv2 Gregory Szorc <gregory.szorc@gmail.com> parents: 37053 diff changeset	204 frame = wireprotoframing.readframe(req.bodyfh)
e7a012b60d6e wireproto: implement basic command dispatching for HTTPv2 Gregory Szorc <gregory.szorc@gmail.com> parents: 37053 diff changeset	205 if not frame:
e7a012b60d6e wireproto: implement basic command dispatching for HTTPv2 Gregory Szorc <gregory.szorc@gmail.com> parents: 37053 diff changeset	206 break
e7a012b60d6e wireproto: implement basic command dispatching for HTTPv2 Gregory Szorc <gregory.szorc@gmail.com> parents: 37053 diff changeset	207
37061 884a0c1604ad wireproto: define attr-based classes for representing frames Gregory Szorc <gregory.szorc@gmail.com> parents: 37059 diff changeset	208 action, meta = reactor.onframerecv(frame)
37054 e7a012b60d6e wireproto: implement basic command dispatching for HTTPv2 Gregory Szorc <gregory.szorc@gmail.com> parents: 37053 diff changeset	209
e7a012b60d6e wireproto: implement basic command dispatching for HTTPv2 Gregory Szorc <gregory.szorc@gmail.com> parents: 37053 diff changeset	210 if action == 'wantframe':
e7a012b60d6e wireproto: implement basic command dispatching for HTTPv2 Gregory Szorc <gregory.szorc@gmail.com> parents: 37053 diff changeset	211 # Need more data before we can do anything.
e7a012b60d6e wireproto: implement basic command dispatching for HTTPv2 Gregory Szorc <gregory.szorc@gmail.com> parents: 37053 diff changeset	212 continue
e7a012b60d6e wireproto: implement basic command dispatching for HTTPv2 Gregory Szorc <gregory.szorc@gmail.com> parents: 37053 diff changeset	213 elif action == 'runcommand':
40138 b5bf3dd6ec5b wireprotov2: send content encoded frames from server Gregory Szorc <gregory.szorc@gmail.com> parents: 40133 diff changeset	214 # Defer creating output stream because we need to wait for
b5bf3dd6ec5b wireprotov2: send content encoded frames from server Gregory Szorc <gregory.szorc@gmail.com> parents: 40133 diff changeset	215 # protocol settings frames so proper encoding can be applied.
b5bf3dd6ec5b wireprotov2: send content encoded frames from server Gregory Szorc <gregory.szorc@gmail.com> parents: 40133 diff changeset	216 if not outstream:
b5bf3dd6ec5b wireprotov2: send content encoded frames from server Gregory Szorc <gregory.szorc@gmail.com> parents: 40133 diff changeset	217 outstream = reactor.makeoutputstream()
b5bf3dd6ec5b wireprotov2: send content encoded frames from server Gregory Szorc <gregory.szorc@gmail.com> parents: 40133 diff changeset	218
37059 bbea991635d0 wireproto: service multiple command requests per HTTP request Gregory Szorc <gregory.szorc@gmail.com> parents: 37057 diff changeset	219 sentoutput = _httpv2runcommand(ui, repo, req, res, authedperm,
37289 5fadc63ac99f wireproto: explicit API to create outgoing streams Gregory Szorc <gregory.szorc@gmail.com> parents: 37288 diff changeset	220 reqcommand, reactor, outstream,
5fadc63ac99f wireproto: explicit API to create outgoing streams Gregory Szorc <gregory.szorc@gmail.com> parents: 37288 diff changeset	221 meta, issubsequent=seencommand)
37059 bbea991635d0 wireproto: service multiple command requests per HTTP request Gregory Szorc <gregory.szorc@gmail.com> parents: 37057 diff changeset	222
bbea991635d0 wireproto: service multiple command requests per HTTP request Gregory Szorc <gregory.szorc@gmail.com> parents: 37057 diff changeset	223 if sentoutput:
37054 e7a012b60d6e wireproto: implement basic command dispatching for HTTPv2 Gregory Szorc <gregory.szorc@gmail.com> parents: 37053 diff changeset	224 return
e7a012b60d6e wireproto: implement basic command dispatching for HTTPv2 Gregory Szorc <gregory.szorc@gmail.com> parents: 37053 diff changeset	225
37059 bbea991635d0 wireproto: service multiple command requests per HTTP request Gregory Szorc <gregory.szorc@gmail.com> parents: 37057 diff changeset	226 seencommand = True
37054 e7a012b60d6e wireproto: implement basic command dispatching for HTTPv2 Gregory Szorc <gregory.szorc@gmail.com> parents: 37053 diff changeset	227
e7a012b60d6e wireproto: implement basic command dispatching for HTTPv2 Gregory Szorc <gregory.szorc@gmail.com> parents: 37053 diff changeset	228 elif action == 'error':
e7a012b60d6e wireproto: implement basic command dispatching for HTTPv2 Gregory Szorc <gregory.szorc@gmail.com> parents: 37053 diff changeset	229 # TODO define proper error mechanism.
e7a012b60d6e wireproto: implement basic command dispatching for HTTPv2 Gregory Szorc <gregory.szorc@gmail.com> parents: 37053 diff changeset	230 res.status = b'200 OK'
e7a012b60d6e wireproto: implement basic command dispatching for HTTPv2 Gregory Szorc <gregory.szorc@gmail.com> parents: 37053 diff changeset	231 res.headers[b'Content-Type'] = b'text/plain'
e7a012b60d6e wireproto: implement basic command dispatching for HTTPv2 Gregory Szorc <gregory.szorc@gmail.com> parents: 37053 diff changeset	232 res.setbodybytes(meta['message'] + b'\n')
e7a012b60d6e wireproto: implement basic command dispatching for HTTPv2 Gregory Szorc <gregory.szorc@gmail.com> parents: 37053 diff changeset	233 return
e7a012b60d6e wireproto: implement basic command dispatching for HTTPv2 Gregory Szorc <gregory.szorc@gmail.com> parents: 37053 diff changeset	234 else:
e7a012b60d6e wireproto: implement basic command dispatching for HTTPv2 Gregory Szorc <gregory.szorc@gmail.com> parents: 37053 diff changeset	235 raise error.ProgrammingError(
e7a012b60d6e wireproto: implement basic command dispatching for HTTPv2 Gregory Szorc <gregory.szorc@gmail.com> parents: 37053 diff changeset	236 'unhandled action from frame processor: %s' % action)
e7a012b60d6e wireproto: implement basic command dispatching for HTTPv2 Gregory Szorc <gregory.szorc@gmail.com> parents: 37053 diff changeset	237
37056 861e9d37e56e wireproto: buffer output frames when in half duplex mode Gregory Szorc <gregory.szorc@gmail.com> parents: 37055 diff changeset	238 action, meta = reactor.oninputeof()
861e9d37e56e wireproto: buffer output frames when in half duplex mode Gregory Szorc <gregory.szorc@gmail.com> parents: 37055 diff changeset	239 if action == 'sendframes':
861e9d37e56e wireproto: buffer output frames when in half duplex mode Gregory Szorc <gregory.szorc@gmail.com> parents: 37055 diff changeset	240 # We assume we haven't started sending the response yet. If we're
861e9d37e56e wireproto: buffer output frames when in half duplex mode Gregory Szorc <gregory.szorc@gmail.com> parents: 37055 diff changeset	241 # wrong, the response type will raise an exception.
861e9d37e56e wireproto: buffer output frames when in half duplex mode Gregory Szorc <gregory.szorc@gmail.com> parents: 37055 diff changeset	242 res.status = b'200 OK'
861e9d37e56e wireproto: buffer output frames when in half duplex mode Gregory Szorc <gregory.szorc@gmail.com> parents: 37055 diff changeset	243 res.headers[b'Content-Type'] = FRAMINGTYPE
861e9d37e56e wireproto: buffer output frames when in half duplex mode Gregory Szorc <gregory.szorc@gmail.com> parents: 37055 diff changeset	244 res.setbodygen(meta['framegen'])
861e9d37e56e wireproto: buffer output frames when in half duplex mode Gregory Szorc <gregory.szorc@gmail.com> parents: 37055 diff changeset	245 elif action == 'noop':
861e9d37e56e wireproto: buffer output frames when in half duplex mode Gregory Szorc <gregory.szorc@gmail.com> parents: 37055 diff changeset	246 pass
861e9d37e56e wireproto: buffer output frames when in half duplex mode Gregory Szorc <gregory.szorc@gmail.com> parents: 37055 diff changeset	247 else:
861e9d37e56e wireproto: buffer output frames when in half duplex mode Gregory Szorc <gregory.szorc@gmail.com> parents: 37055 diff changeset	248 raise error.ProgrammingError('unhandled action from frame processor: %s'
861e9d37e56e wireproto: buffer output frames when in half duplex mode Gregory Szorc <gregory.szorc@gmail.com> parents: 37055 diff changeset	249 % action)
861e9d37e56e wireproto: buffer output frames when in half duplex mode Gregory Szorc <gregory.szorc@gmail.com> parents: 37055 diff changeset	250
37054 e7a012b60d6e wireproto: implement basic command dispatching for HTTPv2 Gregory Szorc <gregory.szorc@gmail.com> parents: 37053 diff changeset	251 def _httpv2runcommand(ui, repo, req, res, authedperm, reqcommand, reactor,
37289 5fadc63ac99f wireproto: explicit API to create outgoing streams Gregory Szorc <gregory.szorc@gmail.com> parents: 37288 diff changeset	252 outstream, command, issubsequent):
37054 e7a012b60d6e wireproto: implement basic command dispatching for HTTPv2 Gregory Szorc <gregory.szorc@gmail.com> parents: 37053 diff changeset	253 """Dispatch a wire protocol command made from HTTPv2 requests.
e7a012b60d6e wireproto: implement basic command dispatching for HTTPv2 Gregory Szorc <gregory.szorc@gmail.com> parents: 37053 diff changeset	254
e7a012b60d6e wireproto: implement basic command dispatching for HTTPv2 Gregory Szorc <gregory.szorc@gmail.com> parents: 37053 diff changeset	255 The authenticated permission (``authedperm``) along with the original
e7a012b60d6e wireproto: implement basic command dispatching for HTTPv2 Gregory Szorc <gregory.szorc@gmail.com> parents: 37053 diff changeset	256 command from the URL (``reqcommand``) are passed in.
e7a012b60d6e wireproto: implement basic command dispatching for HTTPv2 Gregory Szorc <gregory.szorc@gmail.com> parents: 37053 diff changeset	257 """
e7a012b60d6e wireproto: implement basic command dispatching for HTTPv2 Gregory Szorc <gregory.szorc@gmail.com> parents: 37053 diff changeset	258 # We already validated that the session has permissions to perform the
e7a012b60d6e wireproto: implement basic command dispatching for HTTPv2 Gregory Szorc <gregory.szorc@gmail.com> parents: 37053 diff changeset	259 # actions in ``authedperm``. In the unified frame protocol, the canonical
e7a012b60d6e wireproto: implement basic command dispatching for HTTPv2 Gregory Szorc <gregory.szorc@gmail.com> parents: 37053 diff changeset	260 # command to run is expressed in a frame. However, the URL also requested
e7a012b60d6e wireproto: implement basic command dispatching for HTTPv2 Gregory Szorc <gregory.szorc@gmail.com> parents: 37053 diff changeset	261 # to run a specific command. We need to be careful that the command we
e7a012b60d6e wireproto: implement basic command dispatching for HTTPv2 Gregory Szorc <gregory.szorc@gmail.com> parents: 37053 diff changeset	262 # run doesn't have permissions requirements greater than what was granted
e7a012b60d6e wireproto: implement basic command dispatching for HTTPv2 Gregory Szorc <gregory.szorc@gmail.com> parents: 37053 diff changeset	263 # by ``authedperm``.
e7a012b60d6e wireproto: implement basic command dispatching for HTTPv2 Gregory Szorc <gregory.szorc@gmail.com> parents: 37053 diff changeset	264 #
37059 bbea991635d0 wireproto: service multiple command requests per HTTP request Gregory Szorc <gregory.szorc@gmail.com> parents: 37057 diff changeset	265 # Our rule for this is we only allow one command per HTTP request and
bbea991635d0 wireproto: service multiple command requests per HTTP request Gregory Szorc <gregory.szorc@gmail.com> parents: 37057 diff changeset	266 # that command must match the command in the URL. However, we make
bbea991635d0 wireproto: service multiple command requests per HTTP request Gregory Szorc <gregory.szorc@gmail.com> parents: 37057 diff changeset	267 # an exception for the ``multirequest`` URL. This URL is allowed to
bbea991635d0 wireproto: service multiple command requests per HTTP request Gregory Szorc <gregory.szorc@gmail.com> parents: 37057 diff changeset	268 # execute multiple commands. We double check permissions of each command
bbea991635d0 wireproto: service multiple command requests per HTTP request Gregory Szorc <gregory.szorc@gmail.com> parents: 37057 diff changeset	269 # as it is invoked to ensure there is no privilege escalation.
bbea991635d0 wireproto: service multiple command requests per HTTP request Gregory Szorc <gregory.szorc@gmail.com> parents: 37057 diff changeset	270 # TODO consider allowing multiple commands to regular command URLs
bbea991635d0 wireproto: service multiple command requests per HTTP request Gregory Szorc <gregory.szorc@gmail.com> parents: 37057 diff changeset	271 # iff each command is the same.
37054 e7a012b60d6e wireproto: implement basic command dispatching for HTTPv2 Gregory Szorc <gregory.szorc@gmail.com> parents: 37053 diff changeset	272
e7a012b60d6e wireproto: implement basic command dispatching for HTTPv2 Gregory Szorc <gregory.szorc@gmail.com> parents: 37053 diff changeset	273 proto = httpv2protocolhandler(req, ui, args=command['args'])
e7a012b60d6e wireproto: implement basic command dispatching for HTTPv2 Gregory Szorc <gregory.szorc@gmail.com> parents: 37053 diff changeset	274
37059 bbea991635d0 wireproto: service multiple command requests per HTTP request Gregory Szorc <gregory.szorc@gmail.com> parents: 37057 diff changeset	275 if reqcommand == b'multirequest':
37784 ee0d5e9d77b2 wireproto: move version 2 commands dict to wireprotov2server Gregory Szorc <gregory.szorc@gmail.com> parents: 37783 diff changeset	276 if not COMMANDS.commandavailable(command['command'], proto):
37059 bbea991635d0 wireproto: service multiple command requests per HTTP request Gregory Szorc <gregory.szorc@gmail.com> parents: 37057 diff changeset	277 # TODO proper error mechanism
bbea991635d0 wireproto: service multiple command requests per HTTP request Gregory Szorc <gregory.szorc@gmail.com> parents: 37057 diff changeset	278 res.status = b'200 OK'
bbea991635d0 wireproto: service multiple command requests per HTTP request Gregory Szorc <gregory.szorc@gmail.com> parents: 37057 diff changeset	279 res.headers[b'Content-Type'] = b'text/plain'
bbea991635d0 wireproto: service multiple command requests per HTTP request Gregory Szorc <gregory.szorc@gmail.com> parents: 37057 diff changeset	280 res.setbodybytes(_('wire protocol command not available: %s') %
bbea991635d0 wireproto: service multiple command requests per HTTP request Gregory Szorc <gregory.szorc@gmail.com> parents: 37057 diff changeset	281 command['command'])
bbea991635d0 wireproto: service multiple command requests per HTTP request Gregory Szorc <gregory.szorc@gmail.com> parents: 37057 diff changeset	282 return True
bbea991635d0 wireproto: service multiple command requests per HTTP request Gregory Szorc <gregory.szorc@gmail.com> parents: 37057 diff changeset	283
37129 aaabd709df72 wireproto: review fixups Gregory Szorc <gregory.szorc@gmail.com> parents: 37120 diff changeset	284 # TODO don't use assert here, since it may be elided by -O.
37059 bbea991635d0 wireproto: service multiple command requests per HTTP request Gregory Szorc <gregory.szorc@gmail.com> parents: 37057 diff changeset	285 assert authedperm in (b'ro', b'rw')
37784 ee0d5e9d77b2 wireproto: move version 2 commands dict to wireprotov2server Gregory Szorc <gregory.szorc@gmail.com> parents: 37783 diff changeset	286 wirecommand = COMMANDS[command['command']]
37059 bbea991635d0 wireproto: service multiple command requests per HTTP request Gregory Szorc <gregory.szorc@gmail.com> parents: 37057 diff changeset	287 assert wirecommand.permission in ('push', 'pull')
37054 e7a012b60d6e wireproto: implement basic command dispatching for HTTPv2 Gregory Szorc <gregory.szorc@gmail.com> parents: 37053 diff changeset	288
37059 bbea991635d0 wireproto: service multiple command requests per HTTP request Gregory Szorc <gregory.szorc@gmail.com> parents: 37057 diff changeset	289 if authedperm == b'ro' and wirecommand.permission != 'pull':
bbea991635d0 wireproto: service multiple command requests per HTTP request Gregory Szorc <gregory.szorc@gmail.com> parents: 37057 diff changeset	290 # TODO proper error mechanism
bbea991635d0 wireproto: service multiple command requests per HTTP request Gregory Szorc <gregory.szorc@gmail.com> parents: 37057 diff changeset	291 res.status = b'403 Forbidden'
bbea991635d0 wireproto: service multiple command requests per HTTP request Gregory Szorc <gregory.szorc@gmail.com> parents: 37057 diff changeset	292 res.headers[b'Content-Type'] = b'text/plain'
bbea991635d0 wireproto: service multiple command requests per HTTP request Gregory Szorc <gregory.szorc@gmail.com> parents: 37057 diff changeset	293 res.setbodybytes(_('insufficient permissions to execute '
bbea991635d0 wireproto: service multiple command requests per HTTP request Gregory Szorc <gregory.szorc@gmail.com> parents: 37057 diff changeset	294 'command: %s') % command['command'])
bbea991635d0 wireproto: service multiple command requests per HTTP request Gregory Szorc <gregory.szorc@gmail.com> parents: 37057 diff changeset	295 return True
bbea991635d0 wireproto: service multiple command requests per HTTP request Gregory Szorc <gregory.szorc@gmail.com> parents: 37057 diff changeset	296
bbea991635d0 wireproto: service multiple command requests per HTTP request Gregory Szorc <gregory.szorc@gmail.com> parents: 37057 diff changeset	297 # TODO should we also call checkperm() here? Maybe not if we're going
bbea991635d0 wireproto: service multiple command requests per HTTP request Gregory Szorc <gregory.szorc@gmail.com> parents: 37057 diff changeset	298 # to overhaul that API. The granted scope from the URL check should
bbea991635d0 wireproto: service multiple command requests per HTTP request Gregory Szorc <gregory.szorc@gmail.com> parents: 37057 diff changeset	299 # be good enough.
bbea991635d0 wireproto: service multiple command requests per HTTP request Gregory Szorc <gregory.szorc@gmail.com> parents: 37057 diff changeset	300
bbea991635d0 wireproto: service multiple command requests per HTTP request Gregory Szorc <gregory.szorc@gmail.com> parents: 37057 diff changeset	301 else:
bbea991635d0 wireproto: service multiple command requests per HTTP request Gregory Szorc <gregory.szorc@gmail.com> parents: 37057 diff changeset	302 # Don't allow multiple commands outside of ``multirequest`` URL.
bbea991635d0 wireproto: service multiple command requests per HTTP request Gregory Szorc <gregory.szorc@gmail.com> parents: 37057 diff changeset	303 if issubsequent:
bbea991635d0 wireproto: service multiple command requests per HTTP request Gregory Szorc <gregory.szorc@gmail.com> parents: 37057 diff changeset	304 # TODO proper error mechanism
bbea991635d0 wireproto: service multiple command requests per HTTP request Gregory Szorc <gregory.szorc@gmail.com> parents: 37057 diff changeset	305 res.status = b'200 OK'
bbea991635d0 wireproto: service multiple command requests per HTTP request Gregory Szorc <gregory.szorc@gmail.com> parents: 37057 diff changeset	306 res.headers[b'Content-Type'] = b'text/plain'
bbea991635d0 wireproto: service multiple command requests per HTTP request Gregory Szorc <gregory.szorc@gmail.com> parents: 37057 diff changeset	307 res.setbodybytes(_('multiple commands cannot be issued to this '
bbea991635d0 wireproto: service multiple command requests per HTTP request Gregory Szorc <gregory.szorc@gmail.com> parents: 37057 diff changeset	308 'URL'))
bbea991635d0 wireproto: service multiple command requests per HTTP request Gregory Szorc <gregory.szorc@gmail.com> parents: 37057 diff changeset	309 return True
bbea991635d0 wireproto: service multiple command requests per HTTP request Gregory Szorc <gregory.szorc@gmail.com> parents: 37057 diff changeset	310
bbea991635d0 wireproto: service multiple command requests per HTTP request Gregory Szorc <gregory.szorc@gmail.com> parents: 37057 diff changeset	311 if reqcommand != command['command']:
bbea991635d0 wireproto: service multiple command requests per HTTP request Gregory Szorc <gregory.szorc@gmail.com> parents: 37057 diff changeset	312 # TODO define proper error mechanism
bbea991635d0 wireproto: service multiple command requests per HTTP request Gregory Szorc <gregory.szorc@gmail.com> parents: 37057 diff changeset	313 res.status = b'200 OK'
bbea991635d0 wireproto: service multiple command requests per HTTP request Gregory Szorc <gregory.szorc@gmail.com> parents: 37057 diff changeset	314 res.headers[b'Content-Type'] = b'text/plain'
bbea991635d0 wireproto: service multiple command requests per HTTP request Gregory Szorc <gregory.szorc@gmail.com> parents: 37057 diff changeset	315 res.setbodybytes(_('command in frame must match command in URL'))
bbea991635d0 wireproto: service multiple command requests per HTTP request Gregory Szorc <gregory.szorc@gmail.com> parents: 37057 diff changeset	316 return True
37054 e7a012b60d6e wireproto: implement basic command dispatching for HTTPv2 Gregory Szorc <gregory.szorc@gmail.com> parents: 37053 diff changeset	317
e7a012b60d6e wireproto: implement basic command dispatching for HTTPv2 Gregory Szorc <gregory.szorc@gmail.com> parents: 37053 diff changeset	318 res.status = b'200 OK'
37055 61393f888dfe wireproto: define and implement responses in framing protocol Gregory Szorc <gregory.szorc@gmail.com> parents: 37054 diff changeset	319 res.headers[b'Content-Type'] = FRAMINGTYPE
37054 e7a012b60d6e wireproto: implement basic command dispatching for HTTPv2 Gregory Szorc <gregory.szorc@gmail.com> parents: 37053 diff changeset	320
39559 07b58266bce3 wireprotov2: implement commands as a generator of objects Gregory Szorc <gregory.szorc@gmail.com> parents: 39448 diff changeset	321 try:
40025 b099e6032f38 wireprotov2: server support for sending content redirects Gregory Szorc <gregory.szorc@gmail.com> parents: 40023 diff changeset	322 objs = dispatch(repo, proto, command['command'], command['redirect'])
39559 07b58266bce3 wireprotov2: implement commands as a generator of objects Gregory Szorc <gregory.szorc@gmail.com> parents: 39448 diff changeset	323
07b58266bce3 wireprotov2: implement commands as a generator of objects Gregory Szorc <gregory.szorc@gmail.com> parents: 39448 diff changeset	324 action, meta = reactor.oncommandresponsereadyobjects(
07b58266bce3 wireprotov2: implement commands as a generator of objects Gregory Szorc <gregory.szorc@gmail.com> parents: 39448 diff changeset	325 outstream, command['requestid'], objs)
07b58266bce3 wireprotov2: implement commands as a generator of objects Gregory Szorc <gregory.szorc@gmail.com> parents: 39448 diff changeset	326
39810 0b61d21f05cc wireprotov2: declare command arguments richly Gregory Szorc <gregory.szorc@gmail.com> parents: 39641 diff changeset	327 except error.WireprotoCommandError as e:
0b61d21f05cc wireprotov2: declare command arguments richly Gregory Szorc <gregory.szorc@gmail.com> parents: 39641 diff changeset	328 action, meta = reactor.oncommanderror(
0b61d21f05cc wireprotov2: declare command arguments richly Gregory Szorc <gregory.szorc@gmail.com> parents: 39641 diff changeset	329 outstream, command['requestid'], e.message, e.messageargs)
0b61d21f05cc wireprotov2: declare command arguments richly Gregory Szorc <gregory.szorc@gmail.com> parents: 39641 diff changeset	330
39559 07b58266bce3 wireprotov2: implement commands as a generator of objects Gregory Szorc <gregory.szorc@gmail.com> parents: 39448 diff changeset	331 except Exception as e:
37726 0c184ca594bb wireprotov2: change behavior of error frame Gregory Szorc <gregory.szorc@gmail.com> parents: 37725 diff changeset	332 action, meta = reactor.onservererror(
39559 07b58266bce3 wireprotov2: implement commands as a generator of objects Gregory Szorc <gregory.szorc@gmail.com> parents: 39448 diff changeset	333 outstream, command['requestid'],
39837 69b4a5b89dc5 py3: cast exception to bytes Gregory Szorc <gregory.szorc@gmail.com> parents: 39828 diff changeset	334 _('exception when invoking command: %s') %
69b4a5b89dc5 py3: cast exception to bytes Gregory Szorc <gregory.szorc@gmail.com> parents: 39828 diff changeset	335 stringutil.forcebytestr(e))
37055 61393f888dfe wireproto: define and implement responses in framing protocol Gregory Szorc <gregory.szorc@gmail.com> parents: 37054 diff changeset	336
61393f888dfe wireproto: define and implement responses in framing protocol Gregory Szorc <gregory.szorc@gmail.com> parents: 37054 diff changeset	337 if action == 'sendframes':
61393f888dfe wireproto: define and implement responses in framing protocol Gregory Szorc <gregory.szorc@gmail.com> parents: 37054 diff changeset	338 res.setbodygen(meta['framegen'])
37059 bbea991635d0 wireproto: service multiple command requests per HTTP request Gregory Szorc <gregory.szorc@gmail.com> parents: 37057 diff changeset	339 return True
37056 861e9d37e56e wireproto: buffer output frames when in half duplex mode Gregory Szorc <gregory.szorc@gmail.com> parents: 37055 diff changeset	340 elif action == 'noop':
37129 aaabd709df72 wireproto: review fixups Gregory Szorc <gregory.szorc@gmail.com> parents: 37120 diff changeset	341 return False
37055 61393f888dfe wireproto: define and implement responses in framing protocol Gregory Szorc <gregory.szorc@gmail.com> parents: 37054 diff changeset	342 else:
61393f888dfe wireproto: define and implement responses in framing protocol Gregory Szorc <gregory.szorc@gmail.com> parents: 37054 diff changeset	343 raise error.ProgrammingError('unhandled event from reactor: %s' %
61393f888dfe wireproto: define and implement responses in framing protocol Gregory Szorc <gregory.szorc@gmail.com> parents: 37054 diff changeset	344 action)
37054 e7a012b60d6e wireproto: implement basic command dispatching for HTTPv2 Gregory Szorc <gregory.szorc@gmail.com> parents: 37053 diff changeset	345
37782 99accae4cc59 wireproto: reimplement dispatch() for version 2 server Gregory Szorc <gregory.szorc@gmail.com> parents: 37781 diff changeset	346 def getdispatchrepo(repo, proto, command):
99accae4cc59 wireproto: reimplement dispatch() for version 2 server Gregory Szorc <gregory.szorc@gmail.com> parents: 37781 diff changeset	347 return repo.filtered('served')
99accae4cc59 wireproto: reimplement dispatch() for version 2 server Gregory Szorc <gregory.szorc@gmail.com> parents: 37781 diff changeset	348
40025 b099e6032f38 wireprotov2: server support for sending content redirects Gregory Szorc <gregory.szorc@gmail.com> parents: 40023 diff changeset	349 def dispatch(repo, proto, command, redirect):
40021 c537144fdbef wireprotov2: support response caching Gregory Szorc <gregory.szorc@gmail.com> parents: 39995 diff changeset	350 """Run a wire protocol command.
c537144fdbef wireprotov2: support response caching Gregory Szorc <gregory.szorc@gmail.com> parents: 39995 diff changeset	351
c537144fdbef wireprotov2: support response caching Gregory Szorc <gregory.szorc@gmail.com> parents: 39995 diff changeset	352 Returns an iterable of objects that will be sent to the client.
c537144fdbef wireprotov2: support response caching Gregory Szorc <gregory.szorc@gmail.com> parents: 39995 diff changeset	353 """
37782 99accae4cc59 wireproto: reimplement dispatch() for version 2 server Gregory Szorc <gregory.szorc@gmail.com> parents: 37781 diff changeset	354 repo = getdispatchrepo(repo, proto, command)
99accae4cc59 wireproto: reimplement dispatch() for version 2 server Gregory Szorc <gregory.szorc@gmail.com> parents: 37781 diff changeset	355
40021 c537144fdbef wireprotov2: support response caching Gregory Szorc <gregory.szorc@gmail.com> parents: 39995 diff changeset	356 entry = COMMANDS[command]
c537144fdbef wireprotov2: support response caching Gregory Szorc <gregory.szorc@gmail.com> parents: 39995 diff changeset	357 func = entry.func
c537144fdbef wireprotov2: support response caching Gregory Szorc <gregory.szorc@gmail.com> parents: 39995 diff changeset	358 spec = entry.args
c537144fdbef wireprotov2: support response caching Gregory Szorc <gregory.szorc@gmail.com> parents: 39995 diff changeset	359
37782 99accae4cc59 wireproto: reimplement dispatch() for version 2 server Gregory Szorc <gregory.szorc@gmail.com> parents: 37781 diff changeset	360 args = proto.getargs(spec)
99accae4cc59 wireproto: reimplement dispatch() for version 2 server Gregory Szorc <gregory.szorc@gmail.com> parents: 37781 diff changeset	361
40021 c537144fdbef wireprotov2: support response caching Gregory Szorc <gregory.szorc@gmail.com> parents: 39995 diff changeset	362 # There is some duplicate boilerplate code here for calling the command and
c537144fdbef wireprotov2: support response caching Gregory Szorc <gregory.szorc@gmail.com> parents: 39995 diff changeset	363 # emitting objects. It is either that or a lot of indented code that looks
c537144fdbef wireprotov2: support response caching Gregory Szorc <gregory.szorc@gmail.com> parents: 39995 diff changeset	364 # like a pyramid (since there are a lot of code paths that result in not
c537144fdbef wireprotov2: support response caching Gregory Szorc <gregory.szorc@gmail.com> parents: 39995 diff changeset	365 # using the cacher).
c537144fdbef wireprotov2: support response caching Gregory Szorc <gregory.szorc@gmail.com> parents: 39995 diff changeset	366 callcommand = lambda: func(repo, proto, **pycompat.strkwargs(args))
c537144fdbef wireprotov2: support response caching Gregory Szorc <gregory.szorc@gmail.com> parents: 39995 diff changeset	367
c537144fdbef wireprotov2: support response caching Gregory Szorc <gregory.szorc@gmail.com> parents: 39995 diff changeset	368 # Request is not cacheable. Don't bother instantiating a cacher.
c537144fdbef wireprotov2: support response caching Gregory Szorc <gregory.szorc@gmail.com> parents: 39995 diff changeset	369 if not entry.cachekeyfn:
c537144fdbef wireprotov2: support response caching Gregory Szorc <gregory.szorc@gmail.com> parents: 39995 diff changeset	370 for o in callcommand():
c537144fdbef wireprotov2: support response caching Gregory Szorc <gregory.szorc@gmail.com> parents: 39995 diff changeset	371 yield o
c537144fdbef wireprotov2: support response caching Gregory Szorc <gregory.szorc@gmail.com> parents: 39995 diff changeset	372 return
c537144fdbef wireprotov2: support response caching Gregory Szorc <gregory.szorc@gmail.com> parents: 39995 diff changeset	373
40025 b099e6032f38 wireprotov2: server support for sending content redirects Gregory Szorc <gregory.szorc@gmail.com> parents: 40023 diff changeset	374 if redirect:
b099e6032f38 wireprotov2: server support for sending content redirects Gregory Szorc <gregory.szorc@gmail.com> parents: 40023 diff changeset	375 redirecttargets = redirect[b'targets']
b099e6032f38 wireprotov2: server support for sending content redirects Gregory Szorc <gregory.szorc@gmail.com> parents: 40023 diff changeset	376 redirecthashes = redirect[b'hashes']
b099e6032f38 wireprotov2: server support for sending content redirects Gregory Szorc <gregory.szorc@gmail.com> parents: 40023 diff changeset	377 else:
b099e6032f38 wireprotov2: server support for sending content redirects Gregory Szorc <gregory.szorc@gmail.com> parents: 40023 diff changeset	378 redirecttargets = []
b099e6032f38 wireprotov2: server support for sending content redirects Gregory Szorc <gregory.szorc@gmail.com> parents: 40023 diff changeset	379 redirecthashes = []
b099e6032f38 wireprotov2: server support for sending content redirects Gregory Szorc <gregory.szorc@gmail.com> parents: 40023 diff changeset	380
40021 c537144fdbef wireprotov2: support response caching Gregory Szorc <gregory.szorc@gmail.com> parents: 39995 diff changeset	381 cacher = makeresponsecacher(repo, proto, command, args,
40025 b099e6032f38 wireprotov2: server support for sending content redirects Gregory Szorc <gregory.szorc@gmail.com> parents: 40023 diff changeset	382 cborutil.streamencode,
b099e6032f38 wireprotov2: server support for sending content redirects Gregory Szorc <gregory.szorc@gmail.com> parents: 40023 diff changeset	383 redirecttargets=redirecttargets,
b099e6032f38 wireprotov2: server support for sending content redirects Gregory Szorc <gregory.szorc@gmail.com> parents: 40023 diff changeset	384 redirecthashes=redirecthashes)
40021 c537144fdbef wireprotov2: support response caching Gregory Szorc <gregory.szorc@gmail.com> parents: 39995 diff changeset	385
c537144fdbef wireprotov2: support response caching Gregory Szorc <gregory.szorc@gmail.com> parents: 39995 diff changeset	386 # But we have no cacher. Do default handling.
c537144fdbef wireprotov2: support response caching Gregory Szorc <gregory.szorc@gmail.com> parents: 39995 diff changeset	387 if not cacher:
c537144fdbef wireprotov2: support response caching Gregory Szorc <gregory.szorc@gmail.com> parents: 39995 diff changeset	388 for o in callcommand():
c537144fdbef wireprotov2: support response caching Gregory Szorc <gregory.szorc@gmail.com> parents: 39995 diff changeset	389 yield o
c537144fdbef wireprotov2: support response caching Gregory Szorc <gregory.szorc@gmail.com> parents: 39995 diff changeset	390 return
c537144fdbef wireprotov2: support response caching Gregory Szorc <gregory.szorc@gmail.com> parents: 39995 diff changeset	391
c537144fdbef wireprotov2: support response caching Gregory Szorc <gregory.szorc@gmail.com> parents: 39995 diff changeset	392 with cacher:
c537144fdbef wireprotov2: support response caching Gregory Szorc <gregory.szorc@gmail.com> parents: 39995 diff changeset	393 cachekey = entry.cachekeyfn(repo, proto, cacher, **args)
c537144fdbef wireprotov2: support response caching Gregory Szorc <gregory.szorc@gmail.com> parents: 39995 diff changeset	394
c537144fdbef wireprotov2: support response caching Gregory Szorc <gregory.szorc@gmail.com> parents: 39995 diff changeset	395 # No cache key or the cacher doesn't like it. Do default handling.
c537144fdbef wireprotov2: support response caching Gregory Szorc <gregory.szorc@gmail.com> parents: 39995 diff changeset	396 if cachekey is None or not cacher.setcachekey(cachekey):
c537144fdbef wireprotov2: support response caching Gregory Szorc <gregory.szorc@gmail.com> parents: 39995 diff changeset	397 for o in callcommand():
c537144fdbef wireprotov2: support response caching Gregory Szorc <gregory.szorc@gmail.com> parents: 39995 diff changeset	398 yield o
c537144fdbef wireprotov2: support response caching Gregory Szorc <gregory.szorc@gmail.com> parents: 39995 diff changeset	399 return
c537144fdbef wireprotov2: support response caching Gregory Szorc <gregory.szorc@gmail.com> parents: 39995 diff changeset	400
c537144fdbef wireprotov2: support response caching Gregory Szorc <gregory.szorc@gmail.com> parents: 39995 diff changeset	401 # Serve it from the cache, if possible.
c537144fdbef wireprotov2: support response caching Gregory Szorc <gregory.szorc@gmail.com> parents: 39995 diff changeset	402 cached = cacher.lookup()
c537144fdbef wireprotov2: support response caching Gregory Szorc <gregory.szorc@gmail.com> parents: 39995 diff changeset	403
c537144fdbef wireprotov2: support response caching Gregory Szorc <gregory.szorc@gmail.com> parents: 39995 diff changeset	404 if cached:
c537144fdbef wireprotov2: support response caching Gregory Szorc <gregory.szorc@gmail.com> parents: 39995 diff changeset	405 for o in cached['objs']:
c537144fdbef wireprotov2: support response caching Gregory Szorc <gregory.szorc@gmail.com> parents: 39995 diff changeset	406 yield o
c537144fdbef wireprotov2: support response caching Gregory Szorc <gregory.szorc@gmail.com> parents: 39995 diff changeset	407 return
c537144fdbef wireprotov2: support response caching Gregory Szorc <gregory.szorc@gmail.com> parents: 39995 diff changeset	408
c537144fdbef wireprotov2: support response caching Gregory Szorc <gregory.szorc@gmail.com> parents: 39995 diff changeset	409 # Else call the command and feed its output into the cacher, allowing
c537144fdbef wireprotov2: support response caching Gregory Szorc <gregory.szorc@gmail.com> parents: 39995 diff changeset	410 # the cacher to buffer/mutate objects as it desires.
c537144fdbef wireprotov2: support response caching Gregory Szorc <gregory.szorc@gmail.com> parents: 39995 diff changeset	411 for o in callcommand():
c537144fdbef wireprotov2: support response caching Gregory Szorc <gregory.szorc@gmail.com> parents: 39995 diff changeset	412 for o in cacher.onobject(o):
c537144fdbef wireprotov2: support response caching Gregory Szorc <gregory.szorc@gmail.com> parents: 39995 diff changeset	413 yield o
c537144fdbef wireprotov2: support response caching Gregory Szorc <gregory.szorc@gmail.com> parents: 39995 diff changeset	414
c537144fdbef wireprotov2: support response caching Gregory Szorc <gregory.szorc@gmail.com> parents: 39995 diff changeset	415 for o in cacher.onfinished():
c537144fdbef wireprotov2: support response caching Gregory Szorc <gregory.szorc@gmail.com> parents: 39995 diff changeset	416 yield o
37782 99accae4cc59 wireproto: reimplement dispatch() for version 2 server Gregory Szorc <gregory.szorc@gmail.com> parents: 37781 diff changeset	417
37810 856f381ad74b interfaceutil: module to stub out zope.interface Gregory Szorc <gregory.szorc@gmail.com> parents: 37784 diff changeset	418 @interfaceutil.implementer(wireprototypes.baseprotocolhandler)
37296 78103e4138b1 wireproto: port protocol handler to zope.interface Gregory Szorc <gregory.szorc@gmail.com> parents: 37295 diff changeset	419 class httpv2protocolhandler(object):
37054 e7a012b60d6e wireproto: implement basic command dispatching for HTTPv2 Gregory Szorc <gregory.szorc@gmail.com> parents: 37053 diff changeset	420 def __init__(self, req, ui, args=None):
37047 fddcb51b5084 wireproto: define permissions-based routing of HTTPv2 wire protocol Gregory Szorc <gregory.szorc@gmail.com> parents: 37046 diff changeset	421 self._req = req
fddcb51b5084 wireproto: define permissions-based routing of HTTPv2 wire protocol Gregory Szorc <gregory.szorc@gmail.com> parents: 37046 diff changeset	422 self._ui = ui
37054 e7a012b60d6e wireproto: implement basic command dispatching for HTTPv2 Gregory Szorc <gregory.szorc@gmail.com> parents: 37053 diff changeset	423 self._args = args
37047 fddcb51b5084 wireproto: define permissions-based routing of HTTPv2 wire protocol Gregory Szorc <gregory.szorc@gmail.com> parents: 37046 diff changeset	424
fddcb51b5084 wireproto: define permissions-based routing of HTTPv2 wire protocol Gregory Szorc <gregory.szorc@gmail.com> parents: 37046 diff changeset	425 @property
fddcb51b5084 wireproto: define permissions-based routing of HTTPv2 wire protocol Gregory Szorc <gregory.szorc@gmail.com> parents: 37046 diff changeset	426 def name(self):
37644 77c9ee77687c wireproto: rename HTTPV2 so it less like HTTP/2 Gregory Szorc <gregory.szorc@gmail.com> parents: 37557 diff changeset	427 return HTTP_WIREPROTO_V2
37047 fddcb51b5084 wireproto: define permissions-based routing of HTTPv2 wire protocol Gregory Szorc <gregory.szorc@gmail.com> parents: 37046 diff changeset	428
fddcb51b5084 wireproto: define permissions-based routing of HTTPv2 wire protocol Gregory Szorc <gregory.szorc@gmail.com> parents: 37046 diff changeset	429 def getargs(self, args):
39810 0b61d21f05cc wireprotov2: declare command arguments richly Gregory Szorc <gregory.szorc@gmail.com> parents: 39641 diff changeset	430 # First look for args that were passed but aren't registered on this
0b61d21f05cc wireprotov2: declare command arguments richly Gregory Szorc <gregory.szorc@gmail.com> parents: 39641 diff changeset	431 # command.
0b61d21f05cc wireprotov2: declare command arguments richly Gregory Szorc <gregory.szorc@gmail.com> parents: 39641 diff changeset	432 extra = set(self._args) - set(args)
0b61d21f05cc wireprotov2: declare command arguments richly Gregory Szorc <gregory.szorc@gmail.com> parents: 39641 diff changeset	433 if extra:
0b61d21f05cc wireprotov2: declare command arguments richly Gregory Szorc <gregory.szorc@gmail.com> parents: 39641 diff changeset	434 raise error.WireprotoCommandError(
0b61d21f05cc wireprotov2: declare command arguments richly Gregory Szorc <gregory.szorc@gmail.com> parents: 39641 diff changeset	435 'unsupported argument to command: %s' %
0b61d21f05cc wireprotov2: declare command arguments richly Gregory Szorc <gregory.szorc@gmail.com> parents: 39641 diff changeset	436 ', '.join(sorted(extra)))
0b61d21f05cc wireprotov2: declare command arguments richly Gregory Szorc <gregory.szorc@gmail.com> parents: 39641 diff changeset	437
0b61d21f05cc wireprotov2: declare command arguments richly Gregory Szorc <gregory.szorc@gmail.com> parents: 39641 diff changeset	438 # And look for required arguments that are missing.
0b61d21f05cc wireprotov2: declare command arguments richly Gregory Szorc <gregory.szorc@gmail.com> parents: 39641 diff changeset	439 missing = {a for a in args if args[a]['required']} - set(self._args)
0b61d21f05cc wireprotov2: declare command arguments richly Gregory Szorc <gregory.szorc@gmail.com> parents: 39641 diff changeset	440
0b61d21f05cc wireprotov2: declare command arguments richly Gregory Szorc <gregory.szorc@gmail.com> parents: 39641 diff changeset	441 if missing:
0b61d21f05cc wireprotov2: declare command arguments richly Gregory Szorc <gregory.szorc@gmail.com> parents: 39641 diff changeset	442 raise error.WireprotoCommandError(
0b61d21f05cc wireprotov2: declare command arguments richly Gregory Szorc <gregory.szorc@gmail.com> parents: 39641 diff changeset	443 'missing required arguments: %s' % ', '.join(sorted(missing)))
0b61d21f05cc wireprotov2: declare command arguments richly Gregory Szorc <gregory.szorc@gmail.com> parents: 39641 diff changeset	444
0b61d21f05cc wireprotov2: declare command arguments richly Gregory Szorc <gregory.szorc@gmail.com> parents: 39641 diff changeset	445 # Now derive the arguments to pass to the command, taking into
0b61d21f05cc wireprotov2: declare command arguments richly Gregory Szorc <gregory.szorc@gmail.com> parents: 39641 diff changeset	446 # account the arguments specified by the client.
37054 e7a012b60d6e wireproto: implement basic command dispatching for HTTPv2 Gregory Szorc <gregory.szorc@gmail.com> parents: 37053 diff changeset	447 data = {}
39810 0b61d21f05cc wireprotov2: declare command arguments richly Gregory Szorc <gregory.szorc@gmail.com> parents: 39641 diff changeset	448 for k, meta in sorted(args.items()):
0b61d21f05cc wireprotov2: declare command arguments richly Gregory Szorc <gregory.szorc@gmail.com> parents: 39641 diff changeset	449 # This argument wasn't passed by the client.
0b61d21f05cc wireprotov2: declare command arguments richly Gregory Szorc <gregory.szorc@gmail.com> parents: 39641 diff changeset	450 if k not in self._args:
0b61d21f05cc wireprotov2: declare command arguments richly Gregory Szorc <gregory.szorc@gmail.com> parents: 39641 diff changeset	451 data[k] = meta['default']()
0b61d21f05cc wireprotov2: declare command arguments richly Gregory Szorc <gregory.szorc@gmail.com> parents: 39641 diff changeset	452 continue
0b61d21f05cc wireprotov2: declare command arguments richly Gregory Szorc <gregory.szorc@gmail.com> parents: 39641 diff changeset	453
0b61d21f05cc wireprotov2: declare command arguments richly Gregory Szorc <gregory.szorc@gmail.com> parents: 39641 diff changeset	454 v = self._args[k]
0b61d21f05cc wireprotov2: declare command arguments richly Gregory Szorc <gregory.szorc@gmail.com> parents: 39641 diff changeset	455
0b61d21f05cc wireprotov2: declare command arguments richly Gregory Szorc <gregory.szorc@gmail.com> parents: 39641 diff changeset	456 # Sets may be expressed as lists. Silently normalize.
0b61d21f05cc wireprotov2: declare command arguments richly Gregory Szorc <gregory.szorc@gmail.com> parents: 39641 diff changeset	457 if meta['type'] == 'set' and isinstance(v, list):
0b61d21f05cc wireprotov2: declare command arguments richly Gregory Szorc <gregory.szorc@gmail.com> parents: 39641 diff changeset	458 v = set(v)
0b61d21f05cc wireprotov2: declare command arguments richly Gregory Szorc <gregory.szorc@gmail.com> parents: 39641 diff changeset	459
0b61d21f05cc wireprotov2: declare command arguments richly Gregory Szorc <gregory.szorc@gmail.com> parents: 39641 diff changeset	460 # TODO consider more/stronger type validation.
0b61d21f05cc wireprotov2: declare command arguments richly Gregory Szorc <gregory.szorc@gmail.com> parents: 39641 diff changeset	461
0b61d21f05cc wireprotov2: declare command arguments richly Gregory Szorc <gregory.szorc@gmail.com> parents: 39641 diff changeset	462 data[k] = v
37054 e7a012b60d6e wireproto: implement basic command dispatching for HTTPv2 Gregory Szorc <gregory.szorc@gmail.com> parents: 37053 diff changeset	463
37485 0b7475ea38cf wireproto: port heads command to wire protocol v2 Gregory Szorc <gregory.szorc@gmail.com> parents: 37414 diff changeset	464 return data
37047 fddcb51b5084 wireproto: define permissions-based routing of HTTPv2 wire protocol Gregory Szorc <gregory.szorc@gmail.com> parents: 37046 diff changeset	465
37393 afcfdf53e4b5 wireproto: provide accessors for client capabilities Joerg Sonnenberger <joerg@bec.de> parents: 37296 diff changeset	466 def getprotocaps(self):
afcfdf53e4b5 wireproto: provide accessors for client capabilities Joerg Sonnenberger <joerg@bec.de> parents: 37296 diff changeset	467 # Protocol capabilities are currently not implemented for HTTP V2.
afcfdf53e4b5 wireproto: provide accessors for client capabilities Joerg Sonnenberger <joerg@bec.de> parents: 37296 diff changeset	468 return set()
afcfdf53e4b5 wireproto: provide accessors for client capabilities Joerg Sonnenberger <joerg@bec.de> parents: 37296 diff changeset	469
37414 2d965bfeb8f6 wireproto: allow direct stream processing for unbundle Joerg Sonnenberger <joerg@bec.de> parents: 37393 diff changeset	470 def getpayload(self):
37047 fddcb51b5084 wireproto: define permissions-based routing of HTTPv2 wire protocol Gregory Szorc <gregory.szorc@gmail.com> parents: 37046 diff changeset	471 raise NotImplementedError
fddcb51b5084 wireproto: define permissions-based routing of HTTPv2 wire protocol Gregory Szorc <gregory.szorc@gmail.com> parents: 37046 diff changeset	472
fddcb51b5084 wireproto: define permissions-based routing of HTTPv2 wire protocol Gregory Szorc <gregory.szorc@gmail.com> parents: 37046 diff changeset	473 @contextlib.contextmanager
fddcb51b5084 wireproto: define permissions-based routing of HTTPv2 wire protocol Gregory Szorc <gregory.szorc@gmail.com> parents: 37046 diff changeset	474 def mayberedirectstdio(self):
fddcb51b5084 wireproto: define permissions-based routing of HTTPv2 wire protocol Gregory Szorc <gregory.szorc@gmail.com> parents: 37046 diff changeset	475 raise NotImplementedError
fddcb51b5084 wireproto: define permissions-based routing of HTTPv2 wire protocol Gregory Szorc <gregory.szorc@gmail.com> parents: 37046 diff changeset	476
fddcb51b5084 wireproto: define permissions-based routing of HTTPv2 wire protocol Gregory Szorc <gregory.szorc@gmail.com> parents: 37046 diff changeset	477 def client(self):
fddcb51b5084 wireproto: define permissions-based routing of HTTPv2 wire protocol Gregory Szorc <gregory.szorc@gmail.com> parents: 37046 diff changeset	478 raise NotImplementedError
fddcb51b5084 wireproto: define permissions-based routing of HTTPv2 wire protocol Gregory Szorc <gregory.szorc@gmail.com> parents: 37046 diff changeset	479
fddcb51b5084 wireproto: define permissions-based routing of HTTPv2 wire protocol Gregory Szorc <gregory.szorc@gmail.com> parents: 37046 diff changeset	480 def addcapabilities(self, repo, caps):
37054 e7a012b60d6e wireproto: implement basic command dispatching for HTTPv2 Gregory Szorc <gregory.szorc@gmail.com> parents: 37053 diff changeset	481 return caps
37047 fddcb51b5084 wireproto: define permissions-based routing of HTTPv2 wire protocol Gregory Szorc <gregory.szorc@gmail.com> parents: 37046 diff changeset	482
fddcb51b5084 wireproto: define permissions-based routing of HTTPv2 wire protocol Gregory Szorc <gregory.szorc@gmail.com> parents: 37046 diff changeset	483 def checkperm(self, perm):
fddcb51b5084 wireproto: define permissions-based routing of HTTPv2 wire protocol Gregory Szorc <gregory.szorc@gmail.com> parents: 37046 diff changeset	484 raise NotImplementedError
37546 3a2367e6c6f2 wireproto: move version 2 command handlers to wireprotov2server Gregory Szorc <gregory.szorc@gmail.com> parents: 37545 diff changeset	485
37557 734515aca84d wireproto: define and implement HTTP handshake to upgrade protocol Gregory Szorc <gregory.szorc@gmail.com> parents: 37546 diff changeset	486 def httpv2apidescriptor(req, repo):
734515aca84d wireproto: define and implement HTTP handshake to upgrade protocol Gregory Szorc <gregory.szorc@gmail.com> parents: 37546 diff changeset	487 proto = httpv2protocolhandler(req, repo.ui)
734515aca84d wireproto: define and implement HTTP handshake to upgrade protocol Gregory Szorc <gregory.szorc@gmail.com> parents: 37546 diff changeset	488
734515aca84d wireproto: define and implement HTTP handshake to upgrade protocol Gregory Szorc <gregory.szorc@gmail.com> parents: 37546 diff changeset	489 return _capabilitiesv2(repo, proto)
734515aca84d wireproto: define and implement HTTP handshake to upgrade protocol Gregory Szorc <gregory.szorc@gmail.com> parents: 37546 diff changeset	490
37546 3a2367e6c6f2 wireproto: move version 2 command handlers to wireprotov2server Gregory Szorc <gregory.szorc@gmail.com> parents: 37545 diff changeset	491 def _capabilitiesv2(repo, proto):
3a2367e6c6f2 wireproto: move version 2 command handlers to wireprotov2server Gregory Szorc <gregory.szorc@gmail.com> parents: 37545 diff changeset	492 """Obtain the set of capabilities for version 2 transports.
3a2367e6c6f2 wireproto: move version 2 command handlers to wireprotov2server Gregory Szorc <gregory.szorc@gmail.com> parents: 37545 diff changeset	493
3a2367e6c6f2 wireproto: move version 2 command handlers to wireprotov2server Gregory Szorc <gregory.szorc@gmail.com> parents: 37545 diff changeset	494 These capabilities are distinct from the capabilities for version 1
3a2367e6c6f2 wireproto: move version 2 command handlers to wireprotov2server Gregory Szorc <gregory.szorc@gmail.com> parents: 37545 diff changeset	495 transports.
3a2367e6c6f2 wireproto: move version 2 command handlers to wireprotov2server Gregory Szorc <gregory.szorc@gmail.com> parents: 37545 diff changeset	496 """
3a2367e6c6f2 wireproto: move version 2 command handlers to wireprotov2server Gregory Szorc <gregory.szorc@gmail.com> parents: 37545 diff changeset	497 caps = {
3a2367e6c6f2 wireproto: move version 2 command handlers to wireprotov2server Gregory Szorc <gregory.szorc@gmail.com> parents: 37545 diff changeset	498 'commands': {},
37653 b2fa1591fb44 wireproto: add media type to version 2 capabilities response Gregory Szorc <gregory.szorc@gmail.com> parents: 37644 diff changeset	499 'framingmediatypes': [FRAMINGTYPE],
39811 ae20f52437e9 wireprotov2: advertise recognized path filter prefixes Gregory Szorc <gregory.szorc@gmail.com> parents: 39810 diff changeset	500 'pathfilterprefixes': set(narrowspec.VALID_PREFIXES),
37546 3a2367e6c6f2 wireproto: move version 2 command handlers to wireprotov2server Gregory Szorc <gregory.szorc@gmail.com> parents: 37545 diff changeset	501 }
3a2367e6c6f2 wireproto: move version 2 command handlers to wireprotov2server Gregory Szorc <gregory.szorc@gmail.com> parents: 37545 diff changeset	502
37784 ee0d5e9d77b2 wireproto: move version 2 commands dict to wireprotov2server Gregory Szorc <gregory.szorc@gmail.com> parents: 37783 diff changeset	503 for command, entry in COMMANDS.items():
39812 8e7e822e85ec wireprotov2: expose rich arguments metadata Gregory Szorc <gregory.szorc@gmail.com> parents: 39811 diff changeset	504 args = {}
8e7e822e85ec wireprotov2: expose rich arguments metadata Gregory Szorc <gregory.szorc@gmail.com> parents: 39811 diff changeset	505
8e7e822e85ec wireprotov2: expose rich arguments metadata Gregory Szorc <gregory.szorc@gmail.com> parents: 39811 diff changeset	506 for arg, meta in entry.args.items():
8e7e822e85ec wireprotov2: expose rich arguments metadata Gregory Szorc <gregory.szorc@gmail.com> parents: 39811 diff changeset	507 args[arg] = {
8e7e822e85ec wireprotov2: expose rich arguments metadata Gregory Szorc <gregory.szorc@gmail.com> parents: 39811 diff changeset	508 # TODO should this be a normalized type using CBOR's
8e7e822e85ec wireprotov2: expose rich arguments metadata Gregory Szorc <gregory.szorc@gmail.com> parents: 39811 diff changeset	509 # terminology?
8e7e822e85ec wireprotov2: expose rich arguments metadata Gregory Szorc <gregory.szorc@gmail.com> parents: 39811 diff changeset	510 b'type': meta['type'],
8e7e822e85ec wireprotov2: expose rich arguments metadata Gregory Szorc <gregory.szorc@gmail.com> parents: 39811 diff changeset	511 b'required': meta['required'],
8e7e822e85ec wireprotov2: expose rich arguments metadata Gregory Szorc <gregory.szorc@gmail.com> parents: 39811 diff changeset	512 }
8e7e822e85ec wireprotov2: expose rich arguments metadata Gregory Szorc <gregory.szorc@gmail.com> parents: 39811 diff changeset	513
8e7e822e85ec wireprotov2: expose rich arguments metadata Gregory Szorc <gregory.szorc@gmail.com> parents: 39811 diff changeset	514 if not meta['required']:
8e7e822e85ec wireprotov2: expose rich arguments metadata Gregory Szorc <gregory.szorc@gmail.com> parents: 39811 diff changeset	515 args[arg][b'default'] = meta['default']()
39810 0b61d21f05cc wireprotov2: declare command arguments richly Gregory Szorc <gregory.szorc@gmail.com> parents: 39641 diff changeset	516
39813 c30faea8d02d wireprotov2: advertise set of valid values for requestable fields Gregory Szorc <gregory.szorc@gmail.com> parents: 39812 diff changeset	517 if meta['validvalues']:
c30faea8d02d wireprotov2: advertise set of valid values for requestable fields Gregory Szorc <gregory.szorc@gmail.com> parents: 39812 diff changeset	518 args[arg][b'validvalues'] = meta['validvalues']
c30faea8d02d wireprotov2: advertise set of valid values for requestable fields Gregory Szorc <gregory.szorc@gmail.com> parents: 39812 diff changeset	519
40329 ed55a0077490 wireprotov2: implement command for retrieving raw store files Gregory Szorc <gregory.szorc@gmail.com> parents: 40178 diff changeset	520 # TODO this type of check should be defined in a per-command callback.
ed55a0077490 wireprotov2: implement command for retrieving raw store files Gregory Szorc <gregory.szorc@gmail.com> parents: 40178 diff changeset	521 if (command == b'rawstorefiledata'
ed55a0077490 wireprotov2: implement command for retrieving raw store files Gregory Szorc <gregory.szorc@gmail.com> parents: 40178 diff changeset	522 and not streamclone.allowservergeneration(repo)):
ed55a0077490 wireprotov2: implement command for retrieving raw store files Gregory Szorc <gregory.szorc@gmail.com> parents: 40178 diff changeset	523 continue
ed55a0077490 wireprotov2: implement command for retrieving raw store files Gregory Szorc <gregory.szorc@gmail.com> parents: 40178 diff changeset	524
37546 3a2367e6c6f2 wireproto: move version 2 command handlers to wireprotov2server Gregory Szorc <gregory.szorc@gmail.com> parents: 37545 diff changeset	525 caps['commands'][command] = {
39810 0b61d21f05cc wireprotov2: declare command arguments richly Gregory Szorc <gregory.szorc@gmail.com> parents: 39641 diff changeset	526 'args': args,
37546 3a2367e6c6f2 wireproto: move version 2 command handlers to wireprotov2server Gregory Szorc <gregory.szorc@gmail.com> parents: 37545 diff changeset	527 'permissions': [entry.permission],
3a2367e6c6f2 wireproto: move version 2 command handlers to wireprotov2server Gregory Szorc <gregory.szorc@gmail.com> parents: 37545 diff changeset	528 }
3a2367e6c6f2 wireproto: move version 2 command handlers to wireprotov2server Gregory Szorc <gregory.szorc@gmail.com> parents: 37545 diff changeset	529
40172 30f70d11c224 wireprotov2: advertise recommended batch size for requests Gregory Szorc <gregory.szorc@gmail.com> parents: 40138 diff changeset	530 if entry.extracapabilitiesfn:
30f70d11c224 wireprotov2: advertise recommended batch size for requests Gregory Szorc <gregory.szorc@gmail.com> parents: 40138 diff changeset	531 extracaps = entry.extracapabilitiesfn(repo, proto)
30f70d11c224 wireprotov2: advertise recommended batch size for requests Gregory Szorc <gregory.szorc@gmail.com> parents: 40138 diff changeset	532 caps['commands'][command].update(extracaps)
30f70d11c224 wireprotov2: advertise recommended batch size for requests Gregory Szorc <gregory.szorc@gmail.com> parents: 40138 diff changeset	533
40049 39074a35f7db wireprotov2: always advertise raw repo requirements Gregory Szorc <gregory.szorc@gmail.com> parents: 40025 diff changeset	534 caps['rawrepoformats'] = sorted(repo.requirements &
39074a35f7db wireprotov2: always advertise raw repo requirements Gregory Szorc <gregory.szorc@gmail.com> parents: 40025 diff changeset	535 repo.supportedformats)
37657 23c4ddda7bbe wireproto: expose repository formats via capabilities Gregory Szorc <gregory.szorc@gmail.com> parents: 37653 diff changeset	536
40023 10cf8b116dd8 wireprotov2: advertise redirect targets in capabilities Gregory Szorc <gregory.szorc@gmail.com> parents: 40021 diff changeset	537 targets = getadvertisedredirecttargets(repo, proto)
10cf8b116dd8 wireprotov2: advertise redirect targets in capabilities Gregory Szorc <gregory.szorc@gmail.com> parents: 40021 diff changeset	538 if targets:
10cf8b116dd8 wireprotov2: advertise redirect targets in capabilities Gregory Szorc <gregory.szorc@gmail.com> parents: 40021 diff changeset	539 caps[b'redirect'] = {
10cf8b116dd8 wireprotov2: advertise redirect targets in capabilities Gregory Szorc <gregory.szorc@gmail.com> parents: 40021 diff changeset	540 b'targets': [],
10cf8b116dd8 wireprotov2: advertise redirect targets in capabilities Gregory Szorc <gregory.szorc@gmail.com> parents: 40021 diff changeset	541 b'hashes': [b'sha256', b'sha1'],
10cf8b116dd8 wireprotov2: advertise redirect targets in capabilities Gregory Szorc <gregory.szorc@gmail.com> parents: 40021 diff changeset	542 }
10cf8b116dd8 wireprotov2: advertise redirect targets in capabilities Gregory Szorc <gregory.szorc@gmail.com> parents: 40021 diff changeset	543
10cf8b116dd8 wireprotov2: advertise redirect targets in capabilities Gregory Szorc <gregory.szorc@gmail.com> parents: 40021 diff changeset	544 for target in targets:
10cf8b116dd8 wireprotov2: advertise redirect targets in capabilities Gregory Szorc <gregory.szorc@gmail.com> parents: 40021 diff changeset	545 entry = {
10cf8b116dd8 wireprotov2: advertise redirect targets in capabilities Gregory Szorc <gregory.szorc@gmail.com> parents: 40021 diff changeset	546 b'name': target['name'],
10cf8b116dd8 wireprotov2: advertise redirect targets in capabilities Gregory Szorc <gregory.szorc@gmail.com> parents: 40021 diff changeset	547 b'protocol': target['protocol'],
10cf8b116dd8 wireprotov2: advertise redirect targets in capabilities Gregory Szorc <gregory.szorc@gmail.com> parents: 40021 diff changeset	548 b'uris': target['uris'],
10cf8b116dd8 wireprotov2: advertise redirect targets in capabilities Gregory Szorc <gregory.szorc@gmail.com> parents: 40021 diff changeset	549 }
10cf8b116dd8 wireprotov2: advertise redirect targets in capabilities Gregory Szorc <gregory.szorc@gmail.com> parents: 40021 diff changeset	550
10cf8b116dd8 wireprotov2: advertise redirect targets in capabilities Gregory Szorc <gregory.szorc@gmail.com> parents: 40021 diff changeset	551 for key in ('snirequired', 'tlsversions'):
10cf8b116dd8 wireprotov2: advertise redirect targets in capabilities Gregory Szorc <gregory.szorc@gmail.com> parents: 40021 diff changeset	552 if key in target:
10cf8b116dd8 wireprotov2: advertise redirect targets in capabilities Gregory Szorc <gregory.szorc@gmail.com> parents: 40021 diff changeset	553 entry[key] = target[key]
10cf8b116dd8 wireprotov2: advertise redirect targets in capabilities Gregory Szorc <gregory.szorc@gmail.com> parents: 40021 diff changeset	554
10cf8b116dd8 wireprotov2: advertise redirect targets in capabilities Gregory Szorc <gregory.szorc@gmail.com> parents: 40021 diff changeset	555 caps[b'redirect'][b'targets'].append(entry)
10cf8b116dd8 wireprotov2: advertise redirect targets in capabilities Gregory Szorc <gregory.szorc@gmail.com> parents: 40021 diff changeset	556
37546 3a2367e6c6f2 wireproto: move version 2 command handlers to wireprotov2server Gregory Szorc <gregory.szorc@gmail.com> parents: 37545 diff changeset	557 return proto.addcapabilities(repo, caps)
3a2367e6c6f2 wireproto: move version 2 command handlers to wireprotov2server Gregory Szorc <gregory.szorc@gmail.com> parents: 37545 diff changeset	558
40023 10cf8b116dd8 wireprotov2: advertise redirect targets in capabilities Gregory Szorc <gregory.szorc@gmail.com> parents: 40021 diff changeset	559 def getadvertisedredirecttargets(repo, proto):
10cf8b116dd8 wireprotov2: advertise redirect targets in capabilities Gregory Szorc <gregory.szorc@gmail.com> parents: 40021 diff changeset	560 """Obtain a list of content redirect targets.
10cf8b116dd8 wireprotov2: advertise redirect targets in capabilities Gregory Szorc <gregory.szorc@gmail.com> parents: 40021 diff changeset	561
10cf8b116dd8 wireprotov2: advertise redirect targets in capabilities Gregory Szorc <gregory.szorc@gmail.com> parents: 40021 diff changeset	562 Returns a list containing potential redirect targets that will be
10cf8b116dd8 wireprotov2: advertise redirect targets in capabilities Gregory Szorc <gregory.szorc@gmail.com> parents: 40021 diff changeset	563 advertised in capabilities data. Each dict MUST have the following
10cf8b116dd8 wireprotov2: advertise redirect targets in capabilities Gregory Szorc <gregory.szorc@gmail.com> parents: 40021 diff changeset	564 keys:
10cf8b116dd8 wireprotov2: advertise redirect targets in capabilities Gregory Szorc <gregory.szorc@gmail.com> parents: 40021 diff changeset	565
10cf8b116dd8 wireprotov2: advertise redirect targets in capabilities Gregory Szorc <gregory.szorc@gmail.com> parents: 40021 diff changeset	566 name
10cf8b116dd8 wireprotov2: advertise redirect targets in capabilities Gregory Szorc <gregory.szorc@gmail.com> parents: 40021 diff changeset	567 The name of this redirect target. This is the identifier clients use
10cf8b116dd8 wireprotov2: advertise redirect targets in capabilities Gregory Szorc <gregory.szorc@gmail.com> parents: 40021 diff changeset	568 to refer to a target. It is transferred as part of every command
10cf8b116dd8 wireprotov2: advertise redirect targets in capabilities Gregory Szorc <gregory.szorc@gmail.com> parents: 40021 diff changeset	569 request.
10cf8b116dd8 wireprotov2: advertise redirect targets in capabilities Gregory Szorc <gregory.szorc@gmail.com> parents: 40021 diff changeset	570
10cf8b116dd8 wireprotov2: advertise redirect targets in capabilities Gregory Szorc <gregory.szorc@gmail.com> parents: 40021 diff changeset	571 protocol
10cf8b116dd8 wireprotov2: advertise redirect targets in capabilities Gregory Szorc <gregory.szorc@gmail.com> parents: 40021 diff changeset	572 Network protocol used by this target. Typically this is the string
10cf8b116dd8 wireprotov2: advertise redirect targets in capabilities Gregory Szorc <gregory.szorc@gmail.com> parents: 40021 diff changeset	573 in front of the ``://`` in a URL. e.g. ``https``.
10cf8b116dd8 wireprotov2: advertise redirect targets in capabilities Gregory Szorc <gregory.szorc@gmail.com> parents: 40021 diff changeset	574
10cf8b116dd8 wireprotov2: advertise redirect targets in capabilities Gregory Szorc <gregory.szorc@gmail.com> parents: 40021 diff changeset	575 uris
10cf8b116dd8 wireprotov2: advertise redirect targets in capabilities Gregory Szorc <gregory.szorc@gmail.com> parents: 40021 diff changeset	576 List of representative URIs for this target. Clients can use the
10cf8b116dd8 wireprotov2: advertise redirect targets in capabilities Gregory Szorc <gregory.szorc@gmail.com> parents: 40021 diff changeset	577 URIs to test parsing for compatibility or for ordering preference
10cf8b116dd8 wireprotov2: advertise redirect targets in capabilities Gregory Szorc <gregory.szorc@gmail.com> parents: 40021 diff changeset	578 for which target to use.
10cf8b116dd8 wireprotov2: advertise redirect targets in capabilities Gregory Szorc <gregory.szorc@gmail.com> parents: 40021 diff changeset	579
10cf8b116dd8 wireprotov2: advertise redirect targets in capabilities Gregory Szorc <gregory.szorc@gmail.com> parents: 40021 diff changeset	580 The following optional keys are recognized:
10cf8b116dd8 wireprotov2: advertise redirect targets in capabilities Gregory Szorc <gregory.szorc@gmail.com> parents: 40021 diff changeset	581
10cf8b116dd8 wireprotov2: advertise redirect targets in capabilities Gregory Szorc <gregory.szorc@gmail.com> parents: 40021 diff changeset	582 snirequired
10cf8b116dd8 wireprotov2: advertise redirect targets in capabilities Gregory Szorc <gregory.szorc@gmail.com> parents: 40021 diff changeset	583 Bool indicating if Server Name Indication (SNI) is required to
10cf8b116dd8 wireprotov2: advertise redirect targets in capabilities Gregory Szorc <gregory.szorc@gmail.com> parents: 40021 diff changeset	584 connect to this target.
10cf8b116dd8 wireprotov2: advertise redirect targets in capabilities Gregory Szorc <gregory.szorc@gmail.com> parents: 40021 diff changeset	585
10cf8b116dd8 wireprotov2: advertise redirect targets in capabilities Gregory Szorc <gregory.szorc@gmail.com> parents: 40021 diff changeset	586 tlsversions
10cf8b116dd8 wireprotov2: advertise redirect targets in capabilities Gregory Szorc <gregory.szorc@gmail.com> parents: 40021 diff changeset	587 List of bytes indicating which TLS versions are supported by this
10cf8b116dd8 wireprotov2: advertise redirect targets in capabilities Gregory Szorc <gregory.szorc@gmail.com> parents: 40021 diff changeset	588 target.
10cf8b116dd8 wireprotov2: advertise redirect targets in capabilities Gregory Szorc <gregory.szorc@gmail.com> parents: 40021 diff changeset	589
10cf8b116dd8 wireprotov2: advertise redirect targets in capabilities Gregory Szorc <gregory.szorc@gmail.com> parents: 40021 diff changeset	590 By default, clients reflect the target order advertised by servers
10cf8b116dd8 wireprotov2: advertise redirect targets in capabilities Gregory Szorc <gregory.szorc@gmail.com> parents: 40021 diff changeset	591 and servers will use the first client-advertised target when picking
10cf8b116dd8 wireprotov2: advertise redirect targets in capabilities Gregory Szorc <gregory.szorc@gmail.com> parents: 40021 diff changeset	592 a redirect target. So targets should be advertised in the order the
10cf8b116dd8 wireprotov2: advertise redirect targets in capabilities Gregory Szorc <gregory.szorc@gmail.com> parents: 40021 diff changeset	593 server prefers they be used.
10cf8b116dd8 wireprotov2: advertise redirect targets in capabilities Gregory Szorc <gregory.szorc@gmail.com> parents: 40021 diff changeset	594 """
10cf8b116dd8 wireprotov2: advertise redirect targets in capabilities Gregory Szorc <gregory.szorc@gmail.com> parents: 40021 diff changeset	595 return []
10cf8b116dd8 wireprotov2: advertise redirect targets in capabilities Gregory Szorc <gregory.szorc@gmail.com> parents: 40021 diff changeset	596
40172 30f70d11c224 wireprotov2: advertise recommended batch size for requests Gregory Szorc <gregory.szorc@gmail.com> parents: 40138 diff changeset	597 def wireprotocommand(name, args=None, permission='push', cachekeyfn=None,
30f70d11c224 wireprotov2: advertise recommended batch size for requests Gregory Szorc <gregory.szorc@gmail.com> parents: 40138 diff changeset	598 extracapabilitiesfn=None):
37780 8acd3a9ac4fd wireproto: make version 2 @wireprotocommand an independent function Gregory Szorc <gregory.szorc@gmail.com> parents: 37728 diff changeset	599 """Decorator to declare a wire protocol command.
8acd3a9ac4fd wireproto: make version 2 @wireprotocommand an independent function Gregory Szorc <gregory.szorc@gmail.com> parents: 37728 diff changeset	600
8acd3a9ac4fd wireproto: make version 2 @wireprotocommand an independent function Gregory Szorc <gregory.szorc@gmail.com> parents: 37728 diff changeset	601 ``name`` is the name of the wire protocol command being provided.
8acd3a9ac4fd wireproto: make version 2 @wireprotocommand an independent function Gregory Szorc <gregory.szorc@gmail.com> parents: 37728 diff changeset	602
39810 0b61d21f05cc wireprotov2: declare command arguments richly Gregory Szorc <gregory.szorc@gmail.com> parents: 39641 diff changeset	603 ``args`` is a dict defining arguments accepted by the command. Keys are
0b61d21f05cc wireprotov2: declare command arguments richly Gregory Szorc <gregory.szorc@gmail.com> parents: 39641 diff changeset	604 the argument name. Values are dicts with the following keys:
0b61d21f05cc wireprotov2: declare command arguments richly Gregory Szorc <gregory.szorc@gmail.com> parents: 39641 diff changeset	605
0b61d21f05cc wireprotov2: declare command arguments richly Gregory Szorc <gregory.szorc@gmail.com> parents: 39641 diff changeset	606 ``type``
0b61d21f05cc wireprotov2: declare command arguments richly Gregory Szorc <gregory.szorc@gmail.com> parents: 39641 diff changeset	607 The argument data type. Must be one of the following string
0b61d21f05cc wireprotov2: declare command arguments richly Gregory Szorc <gregory.szorc@gmail.com> parents: 39641 diff changeset	608 literals: ``bytes``, ``int``, ``list``, ``dict``, ``set``,
0b61d21f05cc wireprotov2: declare command arguments richly Gregory Szorc <gregory.szorc@gmail.com> parents: 39641 diff changeset	609 or ``bool``.
0b61d21f05cc wireprotov2: declare command arguments richly Gregory Szorc <gregory.szorc@gmail.com> parents: 39641 diff changeset	610
0b61d21f05cc wireprotov2: declare command arguments richly Gregory Szorc <gregory.szorc@gmail.com> parents: 39641 diff changeset	611 ``default``
0b61d21f05cc wireprotov2: declare command arguments richly Gregory Szorc <gregory.szorc@gmail.com> parents: 39641 diff changeset	612 A callable returning the default value for this argument. If not
0b61d21f05cc wireprotov2: declare command arguments richly Gregory Szorc <gregory.szorc@gmail.com> parents: 39641 diff changeset	613 specified, ``None`` will be the default value.
0b61d21f05cc wireprotov2: declare command arguments richly Gregory Szorc <gregory.szorc@gmail.com> parents: 39641 diff changeset	614
0b61d21f05cc wireprotov2: declare command arguments richly Gregory Szorc <gregory.szorc@gmail.com> parents: 39641 diff changeset	615 ``example``
0b61d21f05cc wireprotov2: declare command arguments richly Gregory Szorc <gregory.szorc@gmail.com> parents: 39641 diff changeset	616 An example value for this argument.
37780 8acd3a9ac4fd wireproto: make version 2 @wireprotocommand an independent function Gregory Szorc <gregory.szorc@gmail.com> parents: 37728 diff changeset	617
39813 c30faea8d02d wireprotov2: advertise set of valid values for requestable fields Gregory Szorc <gregory.szorc@gmail.com> parents: 39812 diff changeset	618 ``validvalues``
c30faea8d02d wireprotov2: advertise set of valid values for requestable fields Gregory Szorc <gregory.szorc@gmail.com> parents: 39812 diff changeset	619 Set of recognized values for this argument.
c30faea8d02d wireprotov2: advertise set of valid values for requestable fields Gregory Szorc <gregory.szorc@gmail.com> parents: 39812 diff changeset	620
37780 8acd3a9ac4fd wireproto: make version 2 @wireprotocommand an independent function Gregory Szorc <gregory.szorc@gmail.com> parents: 37728 diff changeset	621 ``permission`` defines the permission type needed to run this command.
8acd3a9ac4fd wireproto: make version 2 @wireprotocommand an independent function Gregory Szorc <gregory.szorc@gmail.com> parents: 37728 diff changeset	622 Can be ``push`` or ``pull``. These roughly map to read-write and read-only,
8acd3a9ac4fd wireproto: make version 2 @wireprotocommand an independent function Gregory Szorc <gregory.szorc@gmail.com> parents: 37728 diff changeset	623 respectively. Default is to assume command requires ``push`` permissions
8acd3a9ac4fd wireproto: make version 2 @wireprotocommand an independent function Gregory Szorc <gregory.szorc@gmail.com> parents: 37728 diff changeset	624 because otherwise commands not declaring their permissions could modify
8acd3a9ac4fd wireproto: make version 2 @wireprotocommand an independent function Gregory Szorc <gregory.szorc@gmail.com> parents: 37728 diff changeset	625 a repository that is supposed to be read-only.
39559 07b58266bce3 wireprotov2: implement commands as a generator of objects Gregory Szorc <gregory.szorc@gmail.com> parents: 39448 diff changeset	626
40021 c537144fdbef wireprotov2: support response caching Gregory Szorc <gregory.szorc@gmail.com> parents: 39995 diff changeset	627 ``cachekeyfn`` defines an optional callable that can derive the
c537144fdbef wireprotov2: support response caching Gregory Szorc <gregory.szorc@gmail.com> parents: 39995 diff changeset	628 cache key for this request.
c537144fdbef wireprotov2: support response caching Gregory Szorc <gregory.szorc@gmail.com> parents: 39995 diff changeset	629
40172 30f70d11c224 wireprotov2: advertise recommended batch size for requests Gregory Szorc <gregory.szorc@gmail.com> parents: 40138 diff changeset	630 ``extracapabilitiesfn`` defines an optional callable that defines extra
30f70d11c224 wireprotov2: advertise recommended batch size for requests Gregory Szorc <gregory.szorc@gmail.com> parents: 40138 diff changeset	631 command capabilities/parameters that are advertised next to the command
30f70d11c224 wireprotov2: advertise recommended batch size for requests Gregory Szorc <gregory.szorc@gmail.com> parents: 40138 diff changeset	632 in the capabilities data structure describing the server. The callable
30f70d11c224 wireprotov2: advertise recommended batch size for requests Gregory Szorc <gregory.szorc@gmail.com> parents: 40138 diff changeset	633 receives as arguments the repository and protocol objects. It returns
30f70d11c224 wireprotov2: advertise recommended batch size for requests Gregory Szorc <gregory.szorc@gmail.com> parents: 40138 diff changeset	634 a dict of extra fields to add to the command descriptor.
30f70d11c224 wireprotov2: advertise recommended batch size for requests Gregory Szorc <gregory.szorc@gmail.com> parents: 40138 diff changeset	635
39559 07b58266bce3 wireprotov2: implement commands as a generator of objects Gregory Szorc <gregory.szorc@gmail.com> parents: 39448 diff changeset	636 Wire protocol commands are generators of objects to be serialized and
07b58266bce3 wireprotov2: implement commands as a generator of objects Gregory Szorc <gregory.szorc@gmail.com> parents: 39448 diff changeset	637 sent to the client.
07b58266bce3 wireprotov2: implement commands as a generator of objects Gregory Szorc <gregory.szorc@gmail.com> parents: 39448 diff changeset	638
07b58266bce3 wireprotov2: implement commands as a generator of objects Gregory Szorc <gregory.szorc@gmail.com> parents: 39448 diff changeset	639 If a command raises an uncaught exception, this will be translated into
07b58266bce3 wireprotov2: implement commands as a generator of objects Gregory Szorc <gregory.szorc@gmail.com> parents: 39448 diff changeset	640 a command error.
40021 c537144fdbef wireprotov2: support response caching Gregory Szorc <gregory.szorc@gmail.com> parents: 39995 diff changeset	641
c537144fdbef wireprotov2: support response caching Gregory Szorc <gregory.szorc@gmail.com> parents: 39995 diff changeset	642 All commands can opt in to being cacheable by defining a function
c537144fdbef wireprotov2: support response caching Gregory Szorc <gregory.szorc@gmail.com> parents: 39995 diff changeset	643 (``cachekeyfn``) that is called to derive a cache key. This function
c537144fdbef wireprotov2: support response caching Gregory Szorc <gregory.szorc@gmail.com> parents: 39995 diff changeset	644 receives the same arguments as the command itself plus a ``cacher``
c537144fdbef wireprotov2: support response caching Gregory Szorc <gregory.szorc@gmail.com> parents: 39995 diff changeset	645 argument containing the active cacher for the request and returns a bytes
c537144fdbef wireprotov2: support response caching Gregory Szorc <gregory.szorc@gmail.com> parents: 39995 diff changeset	646 containing the key in a cache the response to this command may be cached
c537144fdbef wireprotov2: support response caching Gregory Szorc <gregory.szorc@gmail.com> parents: 39995 diff changeset	647 under.
37780 8acd3a9ac4fd wireproto: make version 2 @wireprotocommand an independent function Gregory Szorc <gregory.szorc@gmail.com> parents: 37728 diff changeset	648 """
8acd3a9ac4fd wireproto: make version 2 @wireprotocommand an independent function Gregory Szorc <gregory.szorc@gmail.com> parents: 37728 diff changeset	649 transports = {k for k, v in wireprototypes.TRANSPORTS.items()
8acd3a9ac4fd wireproto: make version 2 @wireprotocommand an independent function Gregory Szorc <gregory.szorc@gmail.com> parents: 37728 diff changeset	650 if v['version'] == 2}
8acd3a9ac4fd wireproto: make version 2 @wireprotocommand an independent function Gregory Szorc <gregory.szorc@gmail.com> parents: 37728 diff changeset	651
8acd3a9ac4fd wireproto: make version 2 @wireprotocommand an independent function Gregory Szorc <gregory.szorc@gmail.com> parents: 37728 diff changeset	652 if permission not in ('push', 'pull'):
8acd3a9ac4fd wireproto: make version 2 @wireprotocommand an independent function Gregory Szorc <gregory.szorc@gmail.com> parents: 37728 diff changeset	653 raise error.ProgrammingError('invalid wire protocol permission; '
8acd3a9ac4fd wireproto: make version 2 @wireprotocommand an independent function Gregory Szorc <gregory.szorc@gmail.com> parents: 37728 diff changeset	654 'got %s; expected "push" or "pull"' %
8acd3a9ac4fd wireproto: make version 2 @wireprotocommand an independent function Gregory Szorc <gregory.szorc@gmail.com> parents: 37728 diff changeset	655 permission)
8acd3a9ac4fd wireproto: make version 2 @wireprotocommand an independent function Gregory Szorc <gregory.szorc@gmail.com> parents: 37728 diff changeset	656
8acd3a9ac4fd wireproto: make version 2 @wireprotocommand an independent function Gregory Szorc <gregory.szorc@gmail.com> parents: 37728 diff changeset	657 if args is None:
8acd3a9ac4fd wireproto: make version 2 @wireprotocommand an independent function Gregory Szorc <gregory.szorc@gmail.com> parents: 37728 diff changeset	658 args = {}
8acd3a9ac4fd wireproto: make version 2 @wireprotocommand an independent function Gregory Szorc <gregory.szorc@gmail.com> parents: 37728 diff changeset	659
8acd3a9ac4fd wireproto: make version 2 @wireprotocommand an independent function Gregory Szorc <gregory.szorc@gmail.com> parents: 37728 diff changeset	660 if not isinstance(args, dict):
8acd3a9ac4fd wireproto: make version 2 @wireprotocommand an independent function Gregory Szorc <gregory.szorc@gmail.com> parents: 37728 diff changeset	661 raise error.ProgrammingError('arguments for version 2 commands '
8acd3a9ac4fd wireproto: make version 2 @wireprotocommand an independent function Gregory Szorc <gregory.szorc@gmail.com> parents: 37728 diff changeset	662 'must be declared as dicts')
8acd3a9ac4fd wireproto: make version 2 @wireprotocommand an independent function Gregory Szorc <gregory.szorc@gmail.com> parents: 37728 diff changeset	663
39810 0b61d21f05cc wireprotov2: declare command arguments richly Gregory Szorc <gregory.szorc@gmail.com> parents: 39641 diff changeset	664 for arg, meta in args.items():
0b61d21f05cc wireprotov2: declare command arguments richly Gregory Szorc <gregory.szorc@gmail.com> parents: 39641 diff changeset	665 if arg == '*':
0b61d21f05cc wireprotov2: declare command arguments richly Gregory Szorc <gregory.szorc@gmail.com> parents: 39641 diff changeset	666 raise error.ProgrammingError('* argument name not allowed on '
0b61d21f05cc wireprotov2: declare command arguments richly Gregory Szorc <gregory.szorc@gmail.com> parents: 39641 diff changeset	667 'version 2 commands')
0b61d21f05cc wireprotov2: declare command arguments richly Gregory Szorc <gregory.szorc@gmail.com> parents: 39641 diff changeset	668
0b61d21f05cc wireprotov2: declare command arguments richly Gregory Szorc <gregory.szorc@gmail.com> parents: 39641 diff changeset	669 if not isinstance(meta, dict):
0b61d21f05cc wireprotov2: declare command arguments richly Gregory Szorc <gregory.szorc@gmail.com> parents: 39641 diff changeset	670 raise error.ProgrammingError('arguments for version 2 commands '
0b61d21f05cc wireprotov2: declare command arguments richly Gregory Szorc <gregory.szorc@gmail.com> parents: 39641 diff changeset	671 'must declare metadata as a dict')
0b61d21f05cc wireprotov2: declare command arguments richly Gregory Szorc <gregory.szorc@gmail.com> parents: 39641 diff changeset	672
0b61d21f05cc wireprotov2: declare command arguments richly Gregory Szorc <gregory.szorc@gmail.com> parents: 39641 diff changeset	673 if 'type' not in meta:
0b61d21f05cc wireprotov2: declare command arguments richly Gregory Szorc <gregory.szorc@gmail.com> parents: 39641 diff changeset	674 raise error.ProgrammingError('%s argument for command %s does not '
0b61d21f05cc wireprotov2: declare command arguments richly Gregory Szorc <gregory.szorc@gmail.com> parents: 39641 diff changeset	675 'declare type field' % (arg, name))
0b61d21f05cc wireprotov2: declare command arguments richly Gregory Szorc <gregory.szorc@gmail.com> parents: 39641 diff changeset	676
0b61d21f05cc wireprotov2: declare command arguments richly Gregory Szorc <gregory.szorc@gmail.com> parents: 39641 diff changeset	677 if meta['type'] not in ('bytes', 'int', 'list', 'dict', 'set', 'bool'):
0b61d21f05cc wireprotov2: declare command arguments richly Gregory Szorc <gregory.szorc@gmail.com> parents: 39641 diff changeset	678 raise error.ProgrammingError('%s argument for command %s has '
0b61d21f05cc wireprotov2: declare command arguments richly Gregory Szorc <gregory.szorc@gmail.com> parents: 39641 diff changeset	679 'illegal type: %s' % (arg, name,
0b61d21f05cc wireprotov2: declare command arguments richly Gregory Szorc <gregory.szorc@gmail.com> parents: 39641 diff changeset	680 meta['type']))
0b61d21f05cc wireprotov2: declare command arguments richly Gregory Szorc <gregory.szorc@gmail.com> parents: 39641 diff changeset	681
0b61d21f05cc wireprotov2: declare command arguments richly Gregory Szorc <gregory.szorc@gmail.com> parents: 39641 diff changeset	682 if 'example' not in meta:
0b61d21f05cc wireprotov2: declare command arguments richly Gregory Szorc <gregory.szorc@gmail.com> parents: 39641 diff changeset	683 raise error.ProgrammingError('%s argument for command %s does not '
0b61d21f05cc wireprotov2: declare command arguments richly Gregory Szorc <gregory.szorc@gmail.com> parents: 39641 diff changeset	684 'declare example field' % (arg, name))
0b61d21f05cc wireprotov2: declare command arguments richly Gregory Szorc <gregory.szorc@gmail.com> parents: 39641 diff changeset	685
39995 582676acaf6d wireprotov2: derive "required" from presence of default value Gregory Szorc <gregory.szorc@gmail.com> parents: 39864 diff changeset	686 meta['required'] = 'default' not in meta
39810 0b61d21f05cc wireprotov2: declare command arguments richly Gregory Szorc <gregory.szorc@gmail.com> parents: 39641 diff changeset	687
0b61d21f05cc wireprotov2: declare command arguments richly Gregory Szorc <gregory.szorc@gmail.com> parents: 39641 diff changeset	688 meta.setdefault('default', lambda: None)
39813 c30faea8d02d wireprotov2: advertise set of valid values for requestable fields Gregory Szorc <gregory.szorc@gmail.com> parents: 39812 diff changeset	689 meta.setdefault('validvalues', None)
39810 0b61d21f05cc wireprotov2: declare command arguments richly Gregory Szorc <gregory.szorc@gmail.com> parents: 39641 diff changeset	690
37546 3a2367e6c6f2 wireproto: move version 2 command handlers to wireprotov2server Gregory Szorc <gregory.szorc@gmail.com> parents: 37545 diff changeset	691 def register(func):
37784 ee0d5e9d77b2 wireproto: move version 2 commands dict to wireprotov2server Gregory Szorc <gregory.szorc@gmail.com> parents: 37783 diff changeset	692 if name in COMMANDS:
37780 8acd3a9ac4fd wireproto: make version 2 @wireprotocommand an independent function Gregory Szorc <gregory.szorc@gmail.com> parents: 37728 diff changeset	693 raise error.ProgrammingError('%s command already registered '
8acd3a9ac4fd wireproto: make version 2 @wireprotocommand an independent function Gregory Szorc <gregory.szorc@gmail.com> parents: 37728 diff changeset	694 'for version 2' % name)
8acd3a9ac4fd wireproto: make version 2 @wireprotocommand an independent function Gregory Szorc <gregory.szorc@gmail.com> parents: 37728 diff changeset	695
37784 ee0d5e9d77b2 wireproto: move version 2 commands dict to wireprotov2server Gregory Szorc <gregory.szorc@gmail.com> parents: 37783 diff changeset	696 COMMANDS[name] = wireprototypes.commandentry(
40021 c537144fdbef wireprotov2: support response caching Gregory Szorc <gregory.szorc@gmail.com> parents: 39995 diff changeset	697 func, args=args, transports=transports, permission=permission,
40172 30f70d11c224 wireprotov2: advertise recommended batch size for requests Gregory Szorc <gregory.szorc@gmail.com> parents: 40138 diff changeset	698 cachekeyfn=cachekeyfn, extracapabilitiesfn=extracapabilitiesfn)
37780 8acd3a9ac4fd wireproto: make version 2 @wireprotocommand an independent function Gregory Szorc <gregory.szorc@gmail.com> parents: 37728 diff changeset	699
8acd3a9ac4fd wireproto: make version 2 @wireprotocommand an independent function Gregory Szorc <gregory.szorc@gmail.com> parents: 37728 diff changeset	700 return func
37546 3a2367e6c6f2 wireproto: move version 2 command handlers to wireprotov2server Gregory Szorc <gregory.szorc@gmail.com> parents: 37545 diff changeset	701
3a2367e6c6f2 wireproto: move version 2 command handlers to wireprotov2server Gregory Szorc <gregory.szorc@gmail.com> parents: 37545 diff changeset	702 return register
3a2367e6c6f2 wireproto: move version 2 command handlers to wireprotov2server Gregory Szorc <gregory.szorc@gmail.com> parents: 37545 diff changeset	703
40021 c537144fdbef wireprotov2: support response caching Gregory Szorc <gregory.szorc@gmail.com> parents: 39995 diff changeset	704 def makecommandcachekeyfn(command, localversion=None, allargs=False):
c537144fdbef wireprotov2: support response caching Gregory Szorc <gregory.szorc@gmail.com> parents: 39995 diff changeset	705 """Construct a cache key derivation function with common features.
c537144fdbef wireprotov2: support response caching Gregory Szorc <gregory.szorc@gmail.com> parents: 39995 diff changeset	706
c537144fdbef wireprotov2: support response caching Gregory Szorc <gregory.szorc@gmail.com> parents: 39995 diff changeset	707 By default, the cache key is a hash of:
c537144fdbef wireprotov2: support response caching Gregory Szorc <gregory.szorc@gmail.com> parents: 39995 diff changeset	708
c537144fdbef wireprotov2: support response caching Gregory Szorc <gregory.szorc@gmail.com> parents: 39995 diff changeset	709 * The command name.
c537144fdbef wireprotov2: support response caching Gregory Szorc <gregory.szorc@gmail.com> parents: 39995 diff changeset	710 * A global cache version number.
c537144fdbef wireprotov2: support response caching Gregory Szorc <gregory.szorc@gmail.com> parents: 39995 diff changeset	711 * A local cache version number (passed via ``localversion``).
c537144fdbef wireprotov2: support response caching Gregory Szorc <gregory.szorc@gmail.com> parents: 39995 diff changeset	712 * All the arguments passed to the command.
c537144fdbef wireprotov2: support response caching Gregory Szorc <gregory.szorc@gmail.com> parents: 39995 diff changeset	713 * The media type used.
c537144fdbef wireprotov2: support response caching Gregory Szorc <gregory.szorc@gmail.com> parents: 39995 diff changeset	714 * Wire protocol version string.
c537144fdbef wireprotov2: support response caching Gregory Szorc <gregory.szorc@gmail.com> parents: 39995 diff changeset	715 * The repository path.
c537144fdbef wireprotov2: support response caching Gregory Szorc <gregory.szorc@gmail.com> parents: 39995 diff changeset	716 """
c537144fdbef wireprotov2: support response caching Gregory Szorc <gregory.szorc@gmail.com> parents: 39995 diff changeset	717 if not allargs:
c537144fdbef wireprotov2: support response caching Gregory Szorc <gregory.szorc@gmail.com> parents: 39995 diff changeset	718 raise error.ProgrammingError('only allargs=True is currently supported')
c537144fdbef wireprotov2: support response caching Gregory Szorc <gregory.szorc@gmail.com> parents: 39995 diff changeset	719
c537144fdbef wireprotov2: support response caching Gregory Szorc <gregory.szorc@gmail.com> parents: 39995 diff changeset	720 if localversion is None:
c537144fdbef wireprotov2: support response caching Gregory Szorc <gregory.szorc@gmail.com> parents: 39995 diff changeset	721 raise error.ProgrammingError('must set localversion argument value')
c537144fdbef wireprotov2: support response caching Gregory Szorc <gregory.szorc@gmail.com> parents: 39995 diff changeset	722
c537144fdbef wireprotov2: support response caching Gregory Szorc <gregory.szorc@gmail.com> parents: 39995 diff changeset	723 def cachekeyfn(repo, proto, cacher, **args):
c537144fdbef wireprotov2: support response caching Gregory Szorc <gregory.szorc@gmail.com> parents: 39995 diff changeset	724 spec = COMMANDS[command]
c537144fdbef wireprotov2: support response caching Gregory Szorc <gregory.szorc@gmail.com> parents: 39995 diff changeset	725
c537144fdbef wireprotov2: support response caching Gregory Szorc <gregory.szorc@gmail.com> parents: 39995 diff changeset	726 # Commands that mutate the repo can not be cached.
c537144fdbef wireprotov2: support response caching Gregory Szorc <gregory.szorc@gmail.com> parents: 39995 diff changeset	727 if spec.permission == 'push':
c537144fdbef wireprotov2: support response caching Gregory Szorc <gregory.szorc@gmail.com> parents: 39995 diff changeset	728 return None
c537144fdbef wireprotov2: support response caching Gregory Szorc <gregory.szorc@gmail.com> parents: 39995 diff changeset	729
c537144fdbef wireprotov2: support response caching Gregory Szorc <gregory.szorc@gmail.com> parents: 39995 diff changeset	730 # TODO config option to disable caching.
c537144fdbef wireprotov2: support response caching Gregory Szorc <gregory.szorc@gmail.com> parents: 39995 diff changeset	731
c537144fdbef wireprotov2: support response caching Gregory Szorc <gregory.szorc@gmail.com> parents: 39995 diff changeset	732 # Our key derivation strategy is to construct a data structure
c537144fdbef wireprotov2: support response caching Gregory Szorc <gregory.szorc@gmail.com> parents: 39995 diff changeset	733 # holding everything that could influence cacheability and to hash
c537144fdbef wireprotov2: support response caching Gregory Szorc <gregory.szorc@gmail.com> parents: 39995 diff changeset	734 # the CBOR representation of that. Using CBOR seems like it might
c537144fdbef wireprotov2: support response caching Gregory Szorc <gregory.szorc@gmail.com> parents: 39995 diff changeset	735 # be overkill. However, simpler hashing mechanisms are prone to
c537144fdbef wireprotov2: support response caching Gregory Szorc <gregory.szorc@gmail.com> parents: 39995 diff changeset	736 # duplicate input issues. e.g. if you just concatenate two values,
c537144fdbef wireprotov2: support response caching Gregory Szorc <gregory.szorc@gmail.com> parents: 39995 diff changeset	737 # "foo"+"bar" is identical to "fo"+"obar". Using CBOR provides
c537144fdbef wireprotov2: support response caching Gregory Szorc <gregory.szorc@gmail.com> parents: 39995 diff changeset	738 # "padding" between values and prevents these problems.
c537144fdbef wireprotov2: support response caching Gregory Szorc <gregory.szorc@gmail.com> parents: 39995 diff changeset	739
c537144fdbef wireprotov2: support response caching Gregory Szorc <gregory.szorc@gmail.com> parents: 39995 diff changeset	740 # Seed the hash with various data.
c537144fdbef wireprotov2: support response caching Gregory Szorc <gregory.szorc@gmail.com> parents: 39995 diff changeset	741 state = {
c537144fdbef wireprotov2: support response caching Gregory Szorc <gregory.szorc@gmail.com> parents: 39995 diff changeset	742 # To invalidate all cache keys.
c537144fdbef wireprotov2: support response caching Gregory Szorc <gregory.szorc@gmail.com> parents: 39995 diff changeset	743 b'globalversion': GLOBAL_CACHE_VERSION,
c537144fdbef wireprotov2: support response caching Gregory Szorc <gregory.szorc@gmail.com> parents: 39995 diff changeset	744 # More granular cache key invalidation.
c537144fdbef wireprotov2: support response caching Gregory Szorc <gregory.szorc@gmail.com> parents: 39995 diff changeset	745 b'localversion': localversion,
c537144fdbef wireprotov2: support response caching Gregory Szorc <gregory.szorc@gmail.com> parents: 39995 diff changeset	746 # Cache keys are segmented by command.
c537144fdbef wireprotov2: support response caching Gregory Szorc <gregory.szorc@gmail.com> parents: 39995 diff changeset	747 b'command': pycompat.sysbytes(command),
c537144fdbef wireprotov2: support response caching Gregory Szorc <gregory.szorc@gmail.com> parents: 39995 diff changeset	748 # Throw in the media type and API version strings so changes
c537144fdbef wireprotov2: support response caching Gregory Szorc <gregory.szorc@gmail.com> parents: 39995 diff changeset	749 # to exchange semantics invalid cache.
c537144fdbef wireprotov2: support response caching Gregory Szorc <gregory.szorc@gmail.com> parents: 39995 diff changeset	750 b'mediatype': FRAMINGTYPE,
c537144fdbef wireprotov2: support response caching Gregory Szorc <gregory.szorc@gmail.com> parents: 39995 diff changeset	751 b'version': HTTP_WIREPROTO_V2,
c537144fdbef wireprotov2: support response caching Gregory Szorc <gregory.szorc@gmail.com> parents: 39995 diff changeset	752 # So same requests for different repos don't share cache keys.
c537144fdbef wireprotov2: support response caching Gregory Szorc <gregory.szorc@gmail.com> parents: 39995 diff changeset	753 b'repo': repo.root,
c537144fdbef wireprotov2: support response caching Gregory Szorc <gregory.szorc@gmail.com> parents: 39995 diff changeset	754 }
c537144fdbef wireprotov2: support response caching Gregory Szorc <gregory.szorc@gmail.com> parents: 39995 diff changeset	755
c537144fdbef wireprotov2: support response caching Gregory Szorc <gregory.szorc@gmail.com> parents: 39995 diff changeset	756 # The arguments passed to us will have already been normalized.
c537144fdbef wireprotov2: support response caching Gregory Szorc <gregory.szorc@gmail.com> parents: 39995 diff changeset	757 # Default values will be set, etc. This is important because it
c537144fdbef wireprotov2: support response caching Gregory Szorc <gregory.szorc@gmail.com> parents: 39995 diff changeset	758 # means that it doesn't matter if clients send an explicit argument
c537144fdbef wireprotov2: support response caching Gregory Szorc <gregory.szorc@gmail.com> parents: 39995 diff changeset	759 # or rely on the default value: it will all normalize to the same
c537144fdbef wireprotov2: support response caching Gregory Szorc <gregory.szorc@gmail.com> parents: 39995 diff changeset	760 # set of arguments on the server and therefore the same cache key.
c537144fdbef wireprotov2: support response caching Gregory Szorc <gregory.szorc@gmail.com> parents: 39995 diff changeset	761 #
c537144fdbef wireprotov2: support response caching Gregory Szorc <gregory.szorc@gmail.com> parents: 39995 diff changeset	762 # Arguments by their very nature must support being encoded to CBOR.
c537144fdbef wireprotov2: support response caching Gregory Szorc <gregory.szorc@gmail.com> parents: 39995 diff changeset	763 # And the CBOR encoder is deterministic. So we hash the arguments
c537144fdbef wireprotov2: support response caching Gregory Szorc <gregory.szorc@gmail.com> parents: 39995 diff changeset	764 # by feeding the CBOR of their representation into the hasher.
c537144fdbef wireprotov2: support response caching Gregory Szorc <gregory.szorc@gmail.com> parents: 39995 diff changeset	765 if allargs:
c537144fdbef wireprotov2: support response caching Gregory Szorc <gregory.szorc@gmail.com> parents: 39995 diff changeset	766 state[b'args'] = pycompat.byteskwargs(args)
c537144fdbef wireprotov2: support response caching Gregory Szorc <gregory.szorc@gmail.com> parents: 39995 diff changeset	767
c537144fdbef wireprotov2: support response caching Gregory Szorc <gregory.szorc@gmail.com> parents: 39995 diff changeset	768 cacher.adjustcachekeystate(state)
c537144fdbef wireprotov2: support response caching Gregory Szorc <gregory.szorc@gmail.com> parents: 39995 diff changeset	769
c537144fdbef wireprotov2: support response caching Gregory Szorc <gregory.szorc@gmail.com> parents: 39995 diff changeset	770 hasher = hashlib.sha1()
c537144fdbef wireprotov2: support response caching Gregory Szorc <gregory.szorc@gmail.com> parents: 39995 diff changeset	771 for chunk in cborutil.streamencode(state):
c537144fdbef wireprotov2: support response caching Gregory Szorc <gregory.szorc@gmail.com> parents: 39995 diff changeset	772 hasher.update(chunk)
c537144fdbef wireprotov2: support response caching Gregory Szorc <gregory.szorc@gmail.com> parents: 39995 diff changeset	773
c537144fdbef wireprotov2: support response caching Gregory Szorc <gregory.szorc@gmail.com> parents: 39995 diff changeset	774 return pycompat.sysbytes(hasher.hexdigest())
c537144fdbef wireprotov2: support response caching Gregory Szorc <gregory.szorc@gmail.com> parents: 39995 diff changeset	775
c537144fdbef wireprotov2: support response caching Gregory Szorc <gregory.szorc@gmail.com> parents: 39995 diff changeset	776 return cachekeyfn
c537144fdbef wireprotov2: support response caching Gregory Szorc <gregory.szorc@gmail.com> parents: 39995 diff changeset	777
40025 b099e6032f38 wireprotov2: server support for sending content redirects Gregory Szorc <gregory.szorc@gmail.com> parents: 40023 diff changeset	778 def makeresponsecacher(repo, proto, command, args, objencoderfn,
b099e6032f38 wireprotov2: server support for sending content redirects Gregory Szorc <gregory.szorc@gmail.com> parents: 40023 diff changeset	779 redirecttargets, redirecthashes):
40021 c537144fdbef wireprotov2: support response caching Gregory Szorc <gregory.szorc@gmail.com> parents: 39995 diff changeset	780 """Construct a cacher for a cacheable command.
c537144fdbef wireprotov2: support response caching Gregory Szorc <gregory.szorc@gmail.com> parents: 39995 diff changeset	781
c537144fdbef wireprotov2: support response caching Gregory Szorc <gregory.szorc@gmail.com> parents: 39995 diff changeset	782 Returns an ``iwireprotocolcommandcacher`` instance.
c537144fdbef wireprotov2: support response caching Gregory Szorc <gregory.szorc@gmail.com> parents: 39995 diff changeset	783
c537144fdbef wireprotov2: support response caching Gregory Szorc <gregory.szorc@gmail.com> parents: 39995 diff changeset	784 Extensions can monkeypatch this function to provide custom caching
c537144fdbef wireprotov2: support response caching Gregory Szorc <gregory.szorc@gmail.com> parents: 39995 diff changeset	785 backends.
c537144fdbef wireprotov2: support response caching Gregory Szorc <gregory.szorc@gmail.com> parents: 39995 diff changeset	786 """
c537144fdbef wireprotov2: support response caching Gregory Szorc <gregory.szorc@gmail.com> parents: 39995 diff changeset	787 return None
c537144fdbef wireprotov2: support response caching Gregory Szorc <gregory.szorc@gmail.com> parents: 39995 diff changeset	788
40176 41263df08109 wireprotov2: change how revisions are specified to changesetdata Gregory Szorc <gregory.szorc@gmail.com> parents: 40175 diff changeset	789 def resolvenodes(repo, revisions):
41263df08109 wireprotov2: change how revisions are specified to changesetdata Gregory Szorc <gregory.szorc@gmail.com> parents: 40175 diff changeset	790 """Resolve nodes from a revisions specifier data structure."""
41263df08109 wireprotov2: change how revisions are specified to changesetdata Gregory Szorc <gregory.szorc@gmail.com> parents: 40175 diff changeset	791 cl = repo.changelog
41263df08109 wireprotov2: change how revisions are specified to changesetdata Gregory Szorc <gregory.szorc@gmail.com> parents: 40175 diff changeset	792 clhasnode = cl.hasnode
41263df08109 wireprotov2: change how revisions are specified to changesetdata Gregory Szorc <gregory.szorc@gmail.com> parents: 40175 diff changeset	793
41263df08109 wireprotov2: change how revisions are specified to changesetdata Gregory Szorc <gregory.szorc@gmail.com> parents: 40175 diff changeset	794 seen = set()
41263df08109 wireprotov2: change how revisions are specified to changesetdata Gregory Szorc <gregory.szorc@gmail.com> parents: 40175 diff changeset	795 nodes = []
41263df08109 wireprotov2: change how revisions are specified to changesetdata Gregory Szorc <gregory.szorc@gmail.com> parents: 40175 diff changeset	796
41263df08109 wireprotov2: change how revisions are specified to changesetdata Gregory Szorc <gregory.szorc@gmail.com> parents: 40175 diff changeset	797 if not isinstance(revisions, list):
41263df08109 wireprotov2: change how revisions are specified to changesetdata Gregory Szorc <gregory.szorc@gmail.com> parents: 40175 diff changeset	798 raise error.WireprotoCommandError('revisions must be defined as an '
41263df08109 wireprotov2: change how revisions are specified to changesetdata Gregory Szorc <gregory.szorc@gmail.com> parents: 40175 diff changeset	799 'array')
41263df08109 wireprotov2: change how revisions are specified to changesetdata Gregory Szorc <gregory.szorc@gmail.com> parents: 40175 diff changeset	800
41263df08109 wireprotov2: change how revisions are specified to changesetdata Gregory Szorc <gregory.szorc@gmail.com> parents: 40175 diff changeset	801 for spec in revisions:
41263df08109 wireprotov2: change how revisions are specified to changesetdata Gregory Szorc <gregory.szorc@gmail.com> parents: 40175 diff changeset	802 if b'type' not in spec:
41263df08109 wireprotov2: change how revisions are specified to changesetdata Gregory Szorc <gregory.szorc@gmail.com> parents: 40175 diff changeset	803 raise error.WireprotoCommandError(
41263df08109 wireprotov2: change how revisions are specified to changesetdata Gregory Szorc <gregory.szorc@gmail.com> parents: 40175 diff changeset	804 'type key not present in revision specifier')
41263df08109 wireprotov2: change how revisions are specified to changesetdata Gregory Szorc <gregory.szorc@gmail.com> parents: 40175 diff changeset	805
41263df08109 wireprotov2: change how revisions are specified to changesetdata Gregory Szorc <gregory.szorc@gmail.com> parents: 40175 diff changeset	806 typ = spec[b'type']
41263df08109 wireprotov2: change how revisions are specified to changesetdata Gregory Szorc <gregory.szorc@gmail.com> parents: 40175 diff changeset	807
41263df08109 wireprotov2: change how revisions are specified to changesetdata Gregory Szorc <gregory.szorc@gmail.com> parents: 40175 diff changeset	808 if typ == b'changesetexplicit':
41263df08109 wireprotov2: change how revisions are specified to changesetdata Gregory Szorc <gregory.szorc@gmail.com> parents: 40175 diff changeset	809 if b'nodes' not in spec:
41263df08109 wireprotov2: change how revisions are specified to changesetdata Gregory Szorc <gregory.szorc@gmail.com> parents: 40175 diff changeset	810 raise error.WireprotoCommandError(
41263df08109 wireprotov2: change how revisions are specified to changesetdata Gregory Szorc <gregory.szorc@gmail.com> parents: 40175 diff changeset	811 'nodes key not present in changesetexplicit revision '
41263df08109 wireprotov2: change how revisions are specified to changesetdata Gregory Szorc <gregory.szorc@gmail.com> parents: 40175 diff changeset	812 'specifier')
41263df08109 wireprotov2: change how revisions are specified to changesetdata Gregory Szorc <gregory.szorc@gmail.com> parents: 40175 diff changeset	813
41263df08109 wireprotov2: change how revisions are specified to changesetdata Gregory Szorc <gregory.szorc@gmail.com> parents: 40175 diff changeset	814 for node in spec[b'nodes']:
41263df08109 wireprotov2: change how revisions are specified to changesetdata Gregory Szorc <gregory.szorc@gmail.com> parents: 40175 diff changeset	815 if node not in seen:
41263df08109 wireprotov2: change how revisions are specified to changesetdata Gregory Szorc <gregory.szorc@gmail.com> parents: 40175 diff changeset	816 nodes.append(node)
41263df08109 wireprotov2: change how revisions are specified to changesetdata Gregory Szorc <gregory.szorc@gmail.com> parents: 40175 diff changeset	817 seen.add(node)
41263df08109 wireprotov2: change how revisions are specified to changesetdata Gregory Szorc <gregory.szorc@gmail.com> parents: 40175 diff changeset	818
41263df08109 wireprotov2: change how revisions are specified to changesetdata Gregory Szorc <gregory.szorc@gmail.com> parents: 40175 diff changeset	819 elif typ == b'changesetexplicitdepth':
41263df08109 wireprotov2: change how revisions are specified to changesetdata Gregory Szorc <gregory.szorc@gmail.com> parents: 40175 diff changeset	820 for key in (b'nodes', b'depth'):
41263df08109 wireprotov2: change how revisions are specified to changesetdata Gregory Szorc <gregory.szorc@gmail.com> parents: 40175 diff changeset	821 if key not in spec:
41263df08109 wireprotov2: change how revisions are specified to changesetdata Gregory Szorc <gregory.szorc@gmail.com> parents: 40175 diff changeset	822 raise error.WireprotoCommandError(
41263df08109 wireprotov2: change how revisions are specified to changesetdata Gregory Szorc <gregory.szorc@gmail.com> parents: 40175 diff changeset	823 '%s key not present in changesetexplicitdepth revision '
41263df08109 wireprotov2: change how revisions are specified to changesetdata Gregory Szorc <gregory.szorc@gmail.com> parents: 40175 diff changeset	824 'specifier', (key,))
41263df08109 wireprotov2: change how revisions are specified to changesetdata Gregory Szorc <gregory.szorc@gmail.com> parents: 40175 diff changeset	825
41263df08109 wireprotov2: change how revisions are specified to changesetdata Gregory Szorc <gregory.szorc@gmail.com> parents: 40175 diff changeset	826 for rev in repo.revs(b'ancestors(%ln, %d)', spec[b'nodes'],
41263df08109 wireprotov2: change how revisions are specified to changesetdata Gregory Szorc <gregory.szorc@gmail.com> parents: 40175 diff changeset	827 spec[b'depth'] - 1):
41263df08109 wireprotov2: change how revisions are specified to changesetdata Gregory Szorc <gregory.szorc@gmail.com> parents: 40175 diff changeset	828 node = cl.node(rev)
41263df08109 wireprotov2: change how revisions are specified to changesetdata Gregory Szorc <gregory.szorc@gmail.com> parents: 40175 diff changeset	829
41263df08109 wireprotov2: change how revisions are specified to changesetdata Gregory Szorc <gregory.szorc@gmail.com> parents: 40175 diff changeset	830 if node not in seen:
41263df08109 wireprotov2: change how revisions are specified to changesetdata Gregory Szorc <gregory.szorc@gmail.com> parents: 40175 diff changeset	831 nodes.append(node)
41263df08109 wireprotov2: change how revisions are specified to changesetdata Gregory Szorc <gregory.szorc@gmail.com> parents: 40175 diff changeset	832 seen.add(node)
41263df08109 wireprotov2: change how revisions are specified to changesetdata Gregory Szorc <gregory.szorc@gmail.com> parents: 40175 diff changeset	833
41263df08109 wireprotov2: change how revisions are specified to changesetdata Gregory Szorc <gregory.szorc@gmail.com> parents: 40175 diff changeset	834 elif typ == b'changesetdagrange':
41263df08109 wireprotov2: change how revisions are specified to changesetdata Gregory Szorc <gregory.szorc@gmail.com> parents: 40175 diff changeset	835 for key in (b'roots', b'heads'):
41263df08109 wireprotov2: change how revisions are specified to changesetdata Gregory Szorc <gregory.szorc@gmail.com> parents: 40175 diff changeset	836 if key not in spec:
41263df08109 wireprotov2: change how revisions are specified to changesetdata Gregory Szorc <gregory.szorc@gmail.com> parents: 40175 diff changeset	837 raise error.WireprotoCommandError(
41263df08109 wireprotov2: change how revisions are specified to changesetdata Gregory Szorc <gregory.szorc@gmail.com> parents: 40175 diff changeset	838 '%s key not present in changesetdagrange revision '
41263df08109 wireprotov2: change how revisions are specified to changesetdata Gregory Szorc <gregory.szorc@gmail.com> parents: 40175 diff changeset	839 'specifier', (key,))
41263df08109 wireprotov2: change how revisions are specified to changesetdata Gregory Szorc <gregory.szorc@gmail.com> parents: 40175 diff changeset	840
41263df08109 wireprotov2: change how revisions are specified to changesetdata Gregory Szorc <gregory.szorc@gmail.com> parents: 40175 diff changeset	841 if not spec[b'heads']:
41263df08109 wireprotov2: change how revisions are specified to changesetdata Gregory Szorc <gregory.szorc@gmail.com> parents: 40175 diff changeset	842 raise error.WireprotoCommandError(
41263df08109 wireprotov2: change how revisions are specified to changesetdata Gregory Szorc <gregory.szorc@gmail.com> parents: 40175 diff changeset	843 'heads key in changesetdagrange cannot be empty')
41263df08109 wireprotov2: change how revisions are specified to changesetdata Gregory Szorc <gregory.szorc@gmail.com> parents: 40175 diff changeset	844
41263df08109 wireprotov2: change how revisions are specified to changesetdata Gregory Szorc <gregory.szorc@gmail.com> parents: 40175 diff changeset	845 if spec[b'roots']:
41263df08109 wireprotov2: change how revisions are specified to changesetdata Gregory Szorc <gregory.szorc@gmail.com> parents: 40175 diff changeset	846 common = [n for n in spec[b'roots'] if clhasnode(n)]
41263df08109 wireprotov2: change how revisions are specified to changesetdata Gregory Szorc <gregory.szorc@gmail.com> parents: 40175 diff changeset	847 else:
41263df08109 wireprotov2: change how revisions are specified to changesetdata Gregory Szorc <gregory.szorc@gmail.com> parents: 40175 diff changeset	848 common = [nullid]
41263df08109 wireprotov2: change how revisions are specified to changesetdata Gregory Szorc <gregory.szorc@gmail.com> parents: 40175 diff changeset	849
41263df08109 wireprotov2: change how revisions are specified to changesetdata Gregory Szorc <gregory.szorc@gmail.com> parents: 40175 diff changeset	850 for n in discovery.outgoing(repo, common, spec[b'heads']).missing:
41263df08109 wireprotov2: change how revisions are specified to changesetdata Gregory Szorc <gregory.szorc@gmail.com> parents: 40175 diff changeset	851 if n not in seen:
41263df08109 wireprotov2: change how revisions are specified to changesetdata Gregory Szorc <gregory.szorc@gmail.com> parents: 40175 diff changeset	852 nodes.append(n)
41263df08109 wireprotov2: change how revisions are specified to changesetdata Gregory Szorc <gregory.szorc@gmail.com> parents: 40175 diff changeset	853 seen.add(n)
41263df08109 wireprotov2: change how revisions are specified to changesetdata Gregory Szorc <gregory.szorc@gmail.com> parents: 40175 diff changeset	854
41263df08109 wireprotov2: change how revisions are specified to changesetdata Gregory Szorc <gregory.szorc@gmail.com> parents: 40175 diff changeset	855 else:
41263df08109 wireprotov2: change how revisions are specified to changesetdata Gregory Szorc <gregory.szorc@gmail.com> parents: 40175 diff changeset	856 raise error.WireprotoCommandError(
41263df08109 wireprotov2: change how revisions are specified to changesetdata Gregory Szorc <gregory.szorc@gmail.com> parents: 40175 diff changeset	857 'unknown revision specifier type: %s', (typ,))
41263df08109 wireprotov2: change how revisions are specified to changesetdata Gregory Szorc <gregory.szorc@gmail.com> parents: 40175 diff changeset	858
41263df08109 wireprotov2: change how revisions are specified to changesetdata Gregory Szorc <gregory.szorc@gmail.com> parents: 40175 diff changeset	859 return nodes
41263df08109 wireprotov2: change how revisions are specified to changesetdata Gregory Szorc <gregory.szorc@gmail.com> parents: 40175 diff changeset	860
37546 3a2367e6c6f2 wireproto: move version 2 command handlers to wireprotov2server Gregory Szorc <gregory.szorc@gmail.com> parents: 37545 diff changeset	861 @wireprotocommand('branchmap', permission='pull')
3a2367e6c6f2 wireproto: move version 2 command handlers to wireprotov2server Gregory Szorc <gregory.szorc@gmail.com> parents: 37545 diff changeset	862 def branchmapv2(repo, proto):
39559 07b58266bce3 wireprotov2: implement commands as a generator of objects Gregory Szorc <gregory.szorc@gmail.com> parents: 39448 diff changeset	863 yield {encoding.fromlocal(k): v
07b58266bce3 wireprotov2: implement commands as a generator of objects Gregory Szorc <gregory.szorc@gmail.com> parents: 39448 diff changeset	864 for k, v in repo.branchmap().iteritems()}
37546 3a2367e6c6f2 wireproto: move version 2 command handlers to wireprotov2server Gregory Szorc <gregory.szorc@gmail.com> parents: 37545 diff changeset	865
3a2367e6c6f2 wireproto: move version 2 command handlers to wireprotov2server Gregory Szorc <gregory.szorc@gmail.com> parents: 37545 diff changeset	866 @wireprotocommand('capabilities', permission='pull')
3a2367e6c6f2 wireproto: move version 2 command handlers to wireprotov2server Gregory Szorc <gregory.szorc@gmail.com> parents: 37545 diff changeset	867 def capabilitiesv2(repo, proto):
39559 07b58266bce3 wireprotov2: implement commands as a generator of objects Gregory Szorc <gregory.szorc@gmail.com> parents: 39448 diff changeset	868 yield _capabilitiesv2(repo, proto)
37546 3a2367e6c6f2 wireproto: move version 2 command handlers to wireprotov2server Gregory Szorc <gregory.szorc@gmail.com> parents: 37545 diff changeset	869
39810 0b61d21f05cc wireprotov2: declare command arguments richly Gregory Szorc <gregory.szorc@gmail.com> parents: 39641 diff changeset	870 @wireprotocommand(
0b61d21f05cc wireprotov2: declare command arguments richly Gregory Szorc <gregory.szorc@gmail.com> parents: 39641 diff changeset	871 'changesetdata',
0b61d21f05cc wireprotov2: declare command arguments richly Gregory Szorc <gregory.szorc@gmail.com> parents: 39641 diff changeset	872 args={
40176 41263df08109 wireprotov2: change how revisions are specified to changesetdata Gregory Szorc <gregory.szorc@gmail.com> parents: 40175 diff changeset	873 'revisions': {
39810 0b61d21f05cc wireprotov2: declare command arguments richly Gregory Szorc <gregory.szorc@gmail.com> parents: 39641 diff changeset	874 'type': 'list',
40176 41263df08109 wireprotov2: change how revisions are specified to changesetdata Gregory Szorc <gregory.szorc@gmail.com> parents: 40175 diff changeset	875 'example': [{
41263df08109 wireprotov2: change how revisions are specified to changesetdata Gregory Szorc <gregory.szorc@gmail.com> parents: 40175 diff changeset	876 b'type': b'changesetexplicit',
41263df08109 wireprotov2: change how revisions are specified to changesetdata Gregory Szorc <gregory.szorc@gmail.com> parents: 40175 diff changeset	877 b'nodes': [b'abcdef...'],
41263df08109 wireprotov2: change how revisions are specified to changesetdata Gregory Szorc <gregory.szorc@gmail.com> parents: 40175 diff changeset	878 }],
39815 d3d333ab167a wireprotov2: teach changesetdata to fetch ancestors until depth Gregory Szorc <gregory.szorc@gmail.com> parents: 39814 diff changeset	879 },
39810 0b61d21f05cc wireprotov2: declare command arguments richly Gregory Szorc <gregory.szorc@gmail.com> parents: 39641 diff changeset	880 'fields': {
0b61d21f05cc wireprotov2: declare command arguments richly Gregory Szorc <gregory.szorc@gmail.com> parents: 39641 diff changeset	881 'type': 'set',
0b61d21f05cc wireprotov2: declare command arguments richly Gregory Szorc <gregory.szorc@gmail.com> parents: 39641 diff changeset	882 'default': set,
0b61d21f05cc wireprotov2: declare command arguments richly Gregory Szorc <gregory.szorc@gmail.com> parents: 39641 diff changeset	883 'example': {b'parents', b'revision'},
39813 c30faea8d02d wireprotov2: advertise set of valid values for requestable fields Gregory Szorc <gregory.szorc@gmail.com> parents: 39812 diff changeset	884 'validvalues': {b'bookmarks', b'parents', b'phase', b'revision'},
39810 0b61d21f05cc wireprotov2: declare command arguments richly Gregory Szorc <gregory.szorc@gmail.com> parents: 39641 diff changeset	885 },
0b61d21f05cc wireprotov2: declare command arguments richly Gregory Szorc <gregory.szorc@gmail.com> parents: 39641 diff changeset	886 },
0b61d21f05cc wireprotov2: declare command arguments richly Gregory Szorc <gregory.szorc@gmail.com> parents: 39641 diff changeset	887 permission='pull')
40176 41263df08109 wireprotov2: change how revisions are specified to changesetdata Gregory Szorc <gregory.szorc@gmail.com> parents: 40175 diff changeset	888 def changesetdata(repo, proto, revisions, fields):
39636 399ddd3227a4 wireprotov2: add TODOs around extending changesetdata fields Gregory Szorc <gregory.szorc@gmail.com> parents: 39634 diff changeset	889 # TODO look for unknown fields and abort when they can't be serviced.
39813 c30faea8d02d wireprotov2: advertise set of valid values for requestable fields Gregory Szorc <gregory.szorc@gmail.com> parents: 39812 diff changeset	890 # This could probably be validated by dispatcher using validvalues.
39636 399ddd3227a4 wireprotov2: add TODOs around extending changesetdata fields Gregory Szorc <gregory.szorc@gmail.com> parents: 39634 diff changeset	891
39630 9c2c77c73f23 wireprotov2: define and implement "changesetdata" command Gregory Szorc <gregory.szorc@gmail.com> parents: 39559 diff changeset	892 cl = repo.changelog
40176 41263df08109 wireprotov2: change how revisions are specified to changesetdata Gregory Szorc <gregory.szorc@gmail.com> parents: 40175 diff changeset	893 outgoing = resolvenodes(repo, revisions)
39632 c1aacb0d76ff wireprotov2: add phases to "changesetdata" command Gregory Szorc <gregory.szorc@gmail.com> parents: 39630 diff changeset	894 publishing = repo.publishing()
39630 9c2c77c73f23 wireprotov2: define and implement "changesetdata" command Gregory Szorc <gregory.szorc@gmail.com> parents: 39559 diff changeset	895
9c2c77c73f23 wireprotov2: define and implement "changesetdata" command Gregory Szorc <gregory.szorc@gmail.com> parents: 39559 diff changeset	896 if outgoing:
9c2c77c73f23 wireprotov2: define and implement "changesetdata" command Gregory Szorc <gregory.szorc@gmail.com> parents: 39559 diff changeset	897 repo.hook('preoutgoing', throw=True, source='serve')
9c2c77c73f23 wireprotov2: define and implement "changesetdata" command Gregory Szorc <gregory.szorc@gmail.com> parents: 39559 diff changeset	898
9c2c77c73f23 wireprotov2: define and implement "changesetdata" command Gregory Szorc <gregory.szorc@gmail.com> parents: 39559 diff changeset	899 yield {
9c2c77c73f23 wireprotov2: define and implement "changesetdata" command Gregory Szorc <gregory.szorc@gmail.com> parents: 39559 diff changeset	900 b'totalitems': len(outgoing),
9c2c77c73f23 wireprotov2: define and implement "changesetdata" command Gregory Szorc <gregory.szorc@gmail.com> parents: 39559 diff changeset	901 }
9c2c77c73f23 wireprotov2: define and implement "changesetdata" command Gregory Szorc <gregory.szorc@gmail.com> parents: 39559 diff changeset	902
39632 c1aacb0d76ff wireprotov2: add phases to "changesetdata" command Gregory Szorc <gregory.szorc@gmail.com> parents: 39630 diff changeset	903 # The phases of nodes already transferred to the client may have changed
c1aacb0d76ff wireprotov2: add phases to "changesetdata" command Gregory Szorc <gregory.szorc@gmail.com> parents: 39630 diff changeset	904 # since the client last requested data. We send phase-only records
c1aacb0d76ff wireprotov2: add phases to "changesetdata" command Gregory Szorc <gregory.szorc@gmail.com> parents: 39630 diff changeset	905 # for these revisions, if requested.
40175 6c42409691ec wireprotov2: stop sending phase updates for base revisions Gregory Szorc <gregory.szorc@gmail.com> parents: 40172 diff changeset	906 # TODO actually do this. We'll probably want to emit phase heads
6c42409691ec wireprotov2: stop sending phase updates for base revisions Gregory Szorc <gregory.szorc@gmail.com> parents: 40172 diff changeset	907 # in the ancestry set of the outgoing revisions. This will ensure
6c42409691ec wireprotov2: stop sending phase updates for base revisions Gregory Szorc <gregory.szorc@gmail.com> parents: 40172 diff changeset	908 # that phase updates within that set are seen.
6c42409691ec wireprotov2: stop sending phase updates for base revisions Gregory Szorc <gregory.szorc@gmail.com> parents: 40172 diff changeset	909 if b'phase' in fields:
6c42409691ec wireprotov2: stop sending phase updates for base revisions Gregory Szorc <gregory.szorc@gmail.com> parents: 40172 diff changeset	910 pass
39632 c1aacb0d76ff wireprotov2: add phases to "changesetdata" command Gregory Szorc <gregory.szorc@gmail.com> parents: 39630 diff changeset	911
39634 9dffa99f9158 wireprotov2: add bookmarks to "changesetdata" command Gregory Szorc <gregory.szorc@gmail.com> parents: 39632 diff changeset	912 nodebookmarks = {}
9dffa99f9158 wireprotov2: add bookmarks to "changesetdata" command Gregory Szorc <gregory.szorc@gmail.com> parents: 39632 diff changeset	913 for mark, node in repo._bookmarks.items():
9dffa99f9158 wireprotov2: add bookmarks to "changesetdata" command Gregory Szorc <gregory.szorc@gmail.com> parents: 39632 diff changeset	914 nodebookmarks.setdefault(node, set()).add(mark)
9dffa99f9158 wireprotov2: add bookmarks to "changesetdata" command Gregory Szorc <gregory.szorc@gmail.com> parents: 39632 diff changeset	915
39630 9c2c77c73f23 wireprotov2: define and implement "changesetdata" command Gregory Szorc <gregory.szorc@gmail.com> parents: 39559 diff changeset	916 # It is already topologically sorted by revision number.
9c2c77c73f23 wireprotov2: define and implement "changesetdata" command Gregory Szorc <gregory.szorc@gmail.com> parents: 39559 diff changeset	917 for node in outgoing:
9c2c77c73f23 wireprotov2: define and implement "changesetdata" command Gregory Szorc <gregory.szorc@gmail.com> parents: 39559 diff changeset	918 d = {
9c2c77c73f23 wireprotov2: define and implement "changesetdata" command Gregory Szorc <gregory.szorc@gmail.com> parents: 39559 diff changeset	919 b'node': node,
9c2c77c73f23 wireprotov2: define and implement "changesetdata" command Gregory Szorc <gregory.szorc@gmail.com> parents: 39559 diff changeset	920 }
9c2c77c73f23 wireprotov2: define and implement "changesetdata" command Gregory Szorc <gregory.szorc@gmail.com> parents: 39559 diff changeset	921
9c2c77c73f23 wireprotov2: define and implement "changesetdata" command Gregory Szorc <gregory.szorc@gmail.com> parents: 39559 diff changeset	922 if b'parents' in fields:
9c2c77c73f23 wireprotov2: define and implement "changesetdata" command Gregory Szorc <gregory.szorc@gmail.com> parents: 39559 diff changeset	923 d[b'parents'] = cl.parents(node)
9c2c77c73f23 wireprotov2: define and implement "changesetdata" command Gregory Szorc <gregory.szorc@gmail.com> parents: 39559 diff changeset	924
39632 c1aacb0d76ff wireprotov2: add phases to "changesetdata" command Gregory Szorc <gregory.szorc@gmail.com> parents: 39630 diff changeset	925 if b'phase' in fields:
c1aacb0d76ff wireprotov2: add phases to "changesetdata" command Gregory Szorc <gregory.szorc@gmail.com> parents: 39630 diff changeset	926 if publishing:
c1aacb0d76ff wireprotov2: add phases to "changesetdata" command Gregory Szorc <gregory.szorc@gmail.com> parents: 39630 diff changeset	927 d[b'phase'] = b'public'
c1aacb0d76ff wireprotov2: add phases to "changesetdata" command Gregory Szorc <gregory.szorc@gmail.com> parents: 39630 diff changeset	928 else:
c1aacb0d76ff wireprotov2: add phases to "changesetdata" command Gregory Szorc <gregory.szorc@gmail.com> parents: 39630 diff changeset	929 ctx = repo[node]
c1aacb0d76ff wireprotov2: add phases to "changesetdata" command Gregory Szorc <gregory.szorc@gmail.com> parents: 39630 diff changeset	930 d[b'phase'] = ctx.phasestr()
c1aacb0d76ff wireprotov2: add phases to "changesetdata" command Gregory Szorc <gregory.szorc@gmail.com> parents: 39630 diff changeset	931
39634 9dffa99f9158 wireprotov2: add bookmarks to "changesetdata" command Gregory Szorc <gregory.szorc@gmail.com> parents: 39632 diff changeset	932 if b'bookmarks' in fields and node in nodebookmarks:
9dffa99f9158 wireprotov2: add bookmarks to "changesetdata" command Gregory Szorc <gregory.szorc@gmail.com> parents: 39632 diff changeset	933 d[b'bookmarks'] = sorted(nodebookmarks[node])
9dffa99f9158 wireprotov2: add bookmarks to "changesetdata" command Gregory Szorc <gregory.szorc@gmail.com> parents: 39632 diff changeset	934 del nodebookmarks[node]
9dffa99f9158 wireprotov2: add bookmarks to "changesetdata" command Gregory Szorc <gregory.szorc@gmail.com> parents: 39632 diff changeset	935
39814 d059cb669632 wireprotov2: allow multiple fields to follow revision maps Gregory Szorc <gregory.szorc@gmail.com> parents: 39813 diff changeset	936 followingmeta = []
d059cb669632 wireprotov2: allow multiple fields to follow revision maps Gregory Szorc <gregory.szorc@gmail.com> parents: 39813 diff changeset	937 followingdata = []
39630 9c2c77c73f23 wireprotov2: define and implement "changesetdata" command Gregory Szorc <gregory.szorc@gmail.com> parents: 39559 diff changeset	938
9c2c77c73f23 wireprotov2: define and implement "changesetdata" command Gregory Szorc <gregory.szorc@gmail.com> parents: 39559 diff changeset	939 if b'revision' in fields:
9c2c77c73f23 wireprotov2: define and implement "changesetdata" command Gregory Szorc <gregory.szorc@gmail.com> parents: 39559 diff changeset	940 revisiondata = cl.revision(node, raw=True)
39814 d059cb669632 wireprotov2: allow multiple fields to follow revision maps Gregory Szorc <gregory.szorc@gmail.com> parents: 39813 diff changeset	941 followingmeta.append((b'revision', len(revisiondata)))
d059cb669632 wireprotov2: allow multiple fields to follow revision maps Gregory Szorc <gregory.szorc@gmail.com> parents: 39813 diff changeset	942 followingdata.append(revisiondata)
39630 9c2c77c73f23 wireprotov2: define and implement "changesetdata" command Gregory Szorc <gregory.szorc@gmail.com> parents: 39559 diff changeset	943
39636 399ddd3227a4 wireprotov2: add TODOs around extending changesetdata fields Gregory Szorc <gregory.szorc@gmail.com> parents: 39634 diff changeset	944 # TODO make it possible for extensions to wrap a function or register
399ddd3227a4 wireprotov2: add TODOs around extending changesetdata fields Gregory Szorc <gregory.szorc@gmail.com> parents: 39634 diff changeset	945 # a handler to service custom fields.
399ddd3227a4 wireprotov2: add TODOs around extending changesetdata fields Gregory Szorc <gregory.szorc@gmail.com> parents: 39634 diff changeset	946
39814 d059cb669632 wireprotov2: allow multiple fields to follow revision maps Gregory Szorc <gregory.szorc@gmail.com> parents: 39813 diff changeset	947 if followingmeta:
d059cb669632 wireprotov2: allow multiple fields to follow revision maps Gregory Szorc <gregory.szorc@gmail.com> parents: 39813 diff changeset	948 d[b'fieldsfollowing'] = followingmeta
d059cb669632 wireprotov2: allow multiple fields to follow revision maps Gregory Szorc <gregory.szorc@gmail.com> parents: 39813 diff changeset	949
39630 9c2c77c73f23 wireprotov2: define and implement "changesetdata" command Gregory Szorc <gregory.szorc@gmail.com> parents: 39559 diff changeset	950 yield d
9c2c77c73f23 wireprotov2: define and implement "changesetdata" command Gregory Szorc <gregory.szorc@gmail.com> parents: 39559 diff changeset	951
39814 d059cb669632 wireprotov2: allow multiple fields to follow revision maps Gregory Szorc <gregory.szorc@gmail.com> parents: 39813 diff changeset	952 for extra in followingdata:
d059cb669632 wireprotov2: allow multiple fields to follow revision maps Gregory Szorc <gregory.szorc@gmail.com> parents: 39813 diff changeset	953 yield extra
39630 9c2c77c73f23 wireprotov2: define and implement "changesetdata" command Gregory Szorc <gregory.szorc@gmail.com> parents: 39559 diff changeset	954
39634 9dffa99f9158 wireprotov2: add bookmarks to "changesetdata" command Gregory Szorc <gregory.szorc@gmail.com> parents: 39632 diff changeset	955 # If requested, send bookmarks from nodes that didn't have revision
9dffa99f9158 wireprotov2: add bookmarks to "changesetdata" command Gregory Szorc <gregory.szorc@gmail.com> parents: 39632 diff changeset	956 # data sent so receiver is aware of any bookmark updates.
9dffa99f9158 wireprotov2: add bookmarks to "changesetdata" command Gregory Szorc <gregory.szorc@gmail.com> parents: 39632 diff changeset	957 if b'bookmarks' in fields:
9dffa99f9158 wireprotov2: add bookmarks to "changesetdata" command Gregory Szorc <gregory.szorc@gmail.com> parents: 39632 diff changeset	958 for node, marks in sorted(nodebookmarks.iteritems()):
9dffa99f9158 wireprotov2: add bookmarks to "changesetdata" command Gregory Szorc <gregory.szorc@gmail.com> parents: 39632 diff changeset	959 yield {
9dffa99f9158 wireprotov2: add bookmarks to "changesetdata" command Gregory Szorc <gregory.szorc@gmail.com> parents: 39632 diff changeset	960 b'node': node,
9dffa99f9158 wireprotov2: add bookmarks to "changesetdata" command Gregory Szorc <gregory.szorc@gmail.com> parents: 39632 diff changeset	961 b'bookmarks': sorted(marks),
9dffa99f9158 wireprotov2: add bookmarks to "changesetdata" command Gregory Szorc <gregory.szorc@gmail.com> parents: 39632 diff changeset	962 }
9dffa99f9158 wireprotov2: add bookmarks to "changesetdata" command Gregory Szorc <gregory.szorc@gmail.com> parents: 39632 diff changeset	963
39639 0e03e6a44dee wireprotov2: define and implement "filedata" command Gregory Szorc <gregory.szorc@gmail.com> parents: 39637 diff changeset	964 class FileAccessError(Exception):
0e03e6a44dee wireprotov2: define and implement "filedata" command Gregory Szorc <gregory.szorc@gmail.com> parents: 39637 diff changeset	965 """Represents an error accessing a specific file."""
0e03e6a44dee wireprotov2: define and implement "filedata" command Gregory Szorc <gregory.szorc@gmail.com> parents: 39637 diff changeset	966
0e03e6a44dee wireprotov2: define and implement "filedata" command Gregory Szorc <gregory.szorc@gmail.com> parents: 39637 diff changeset	967 def __init__(self, path, msg, args):
0e03e6a44dee wireprotov2: define and implement "filedata" command Gregory Szorc <gregory.szorc@gmail.com> parents: 39637 diff changeset	968 self.path = path
0e03e6a44dee wireprotov2: define and implement "filedata" command Gregory Szorc <gregory.szorc@gmail.com> parents: 39637 diff changeset	969 self.msg = msg
0e03e6a44dee wireprotov2: define and implement "filedata" command Gregory Szorc <gregory.szorc@gmail.com> parents: 39637 diff changeset	970 self.args = args
0e03e6a44dee wireprotov2: define and implement "filedata" command Gregory Szorc <gregory.szorc@gmail.com> parents: 39637 diff changeset	971
0e03e6a44dee wireprotov2: define and implement "filedata" command Gregory Szorc <gregory.szorc@gmail.com> parents: 39637 diff changeset	972 def getfilestore(repo, proto, path):
0e03e6a44dee wireprotov2: define and implement "filedata" command Gregory Szorc <gregory.szorc@gmail.com> parents: 39637 diff changeset	973 """Obtain a file storage object for use with wire protocol.
0e03e6a44dee wireprotov2: define and implement "filedata" command Gregory Szorc <gregory.szorc@gmail.com> parents: 39637 diff changeset	974
0e03e6a44dee wireprotov2: define and implement "filedata" command Gregory Szorc <gregory.szorc@gmail.com> parents: 39637 diff changeset	975 Exists as a standalone function so extensions can monkeypatch to add
0e03e6a44dee wireprotov2: define and implement "filedata" command Gregory Szorc <gregory.szorc@gmail.com> parents: 39637 diff changeset	976 access control.
0e03e6a44dee wireprotov2: define and implement "filedata" command Gregory Szorc <gregory.szorc@gmail.com> parents: 39637 diff changeset	977 """
0e03e6a44dee wireprotov2: define and implement "filedata" command Gregory Szorc <gregory.szorc@gmail.com> parents: 39637 diff changeset	978 # This seems to work even if the file doesn't exist. So catch
0e03e6a44dee wireprotov2: define and implement "filedata" command Gregory Szorc <gregory.szorc@gmail.com> parents: 39637 diff changeset	979 # "empty" files and return an error.
0e03e6a44dee wireprotov2: define and implement "filedata" command Gregory Szorc <gregory.szorc@gmail.com> parents: 39637 diff changeset	980 fl = repo.file(path)
0e03e6a44dee wireprotov2: define and implement "filedata" command Gregory Szorc <gregory.szorc@gmail.com> parents: 39637 diff changeset	981
0e03e6a44dee wireprotov2: define and implement "filedata" command Gregory Szorc <gregory.szorc@gmail.com> parents: 39637 diff changeset	982 if not len(fl):
0e03e6a44dee wireprotov2: define and implement "filedata" command Gregory Szorc <gregory.szorc@gmail.com> parents: 39637 diff changeset	983 raise FileAccessError(path, 'unknown file: %s', (path,))
0e03e6a44dee wireprotov2: define and implement "filedata" command Gregory Szorc <gregory.szorc@gmail.com> parents: 39637 diff changeset	984
0e03e6a44dee wireprotov2: define and implement "filedata" command Gregory Szorc <gregory.szorc@gmail.com> parents: 39637 diff changeset	985 return fl
0e03e6a44dee wireprotov2: define and implement "filedata" command Gregory Szorc <gregory.szorc@gmail.com> parents: 39637 diff changeset	986
40177 41e2633bcd00 wireprotov2: extract file object emission to own function Gregory Szorc <gregory.szorc@gmail.com> parents: 40176 diff changeset	987 def emitfilerevisions(revisions, fields):
41e2633bcd00 wireprotov2: extract file object emission to own function Gregory Szorc <gregory.szorc@gmail.com> parents: 40176 diff changeset	988 for revision in revisions:
41e2633bcd00 wireprotov2: extract file object emission to own function Gregory Szorc <gregory.szorc@gmail.com> parents: 40176 diff changeset	989 d = {
41e2633bcd00 wireprotov2: extract file object emission to own function Gregory Szorc <gregory.szorc@gmail.com> parents: 40176 diff changeset	990 b'node': revision.node,
41e2633bcd00 wireprotov2: extract file object emission to own function Gregory Szorc <gregory.szorc@gmail.com> parents: 40176 diff changeset	991 }
41e2633bcd00 wireprotov2: extract file object emission to own function Gregory Szorc <gregory.szorc@gmail.com> parents: 40176 diff changeset	992
41e2633bcd00 wireprotov2: extract file object emission to own function Gregory Szorc <gregory.szorc@gmail.com> parents: 40176 diff changeset	993 if b'parents' in fields:
41e2633bcd00 wireprotov2: extract file object emission to own function Gregory Szorc <gregory.szorc@gmail.com> parents: 40176 diff changeset	994 d[b'parents'] = [revision.p1node, revision.p2node]
41e2633bcd00 wireprotov2: extract file object emission to own function Gregory Szorc <gregory.szorc@gmail.com> parents: 40176 diff changeset	995
41e2633bcd00 wireprotov2: extract file object emission to own function Gregory Szorc <gregory.szorc@gmail.com> parents: 40176 diff changeset	996 followingmeta = []
41e2633bcd00 wireprotov2: extract file object emission to own function Gregory Szorc <gregory.szorc@gmail.com> parents: 40176 diff changeset	997 followingdata = []
41e2633bcd00 wireprotov2: extract file object emission to own function Gregory Szorc <gregory.szorc@gmail.com> parents: 40176 diff changeset	998
41e2633bcd00 wireprotov2: extract file object emission to own function Gregory Szorc <gregory.szorc@gmail.com> parents: 40176 diff changeset	999 if b'revision' in fields:
41e2633bcd00 wireprotov2: extract file object emission to own function Gregory Szorc <gregory.szorc@gmail.com> parents: 40176 diff changeset	1000 if revision.revision is not None:
41e2633bcd00 wireprotov2: extract file object emission to own function Gregory Szorc <gregory.szorc@gmail.com> parents: 40176 diff changeset	1001 followingmeta.append((b'revision', len(revision.revision)))
41e2633bcd00 wireprotov2: extract file object emission to own function Gregory Szorc <gregory.szorc@gmail.com> parents: 40176 diff changeset	1002 followingdata.append(revision.revision)
41e2633bcd00 wireprotov2: extract file object emission to own function Gregory Szorc <gregory.szorc@gmail.com> parents: 40176 diff changeset	1003 else:
41e2633bcd00 wireprotov2: extract file object emission to own function Gregory Szorc <gregory.szorc@gmail.com> parents: 40176 diff changeset	1004 d[b'deltabasenode'] = revision.basenode
41e2633bcd00 wireprotov2: extract file object emission to own function Gregory Szorc <gregory.szorc@gmail.com> parents: 40176 diff changeset	1005 followingmeta.append((b'delta', len(revision.delta)))
41e2633bcd00 wireprotov2: extract file object emission to own function Gregory Szorc <gregory.szorc@gmail.com> parents: 40176 diff changeset	1006 followingdata.append(revision.delta)
41e2633bcd00 wireprotov2: extract file object emission to own function Gregory Szorc <gregory.szorc@gmail.com> parents: 40176 diff changeset	1007
41e2633bcd00 wireprotov2: extract file object emission to own function Gregory Szorc <gregory.szorc@gmail.com> parents: 40176 diff changeset	1008 if followingmeta:
41e2633bcd00 wireprotov2: extract file object emission to own function Gregory Szorc <gregory.szorc@gmail.com> parents: 40176 diff changeset	1009 d[b'fieldsfollowing'] = followingmeta
41e2633bcd00 wireprotov2: extract file object emission to own function Gregory Szorc <gregory.szorc@gmail.com> parents: 40176 diff changeset	1010
41e2633bcd00 wireprotov2: extract file object emission to own function Gregory Szorc <gregory.szorc@gmail.com> parents: 40176 diff changeset	1011 yield d
41e2633bcd00 wireprotov2: extract file object emission to own function Gregory Szorc <gregory.szorc@gmail.com> parents: 40176 diff changeset	1012
41e2633bcd00 wireprotov2: extract file object emission to own function Gregory Szorc <gregory.szorc@gmail.com> parents: 40176 diff changeset	1013 for extra in followingdata:
41e2633bcd00 wireprotov2: extract file object emission to own function Gregory Szorc <gregory.szorc@gmail.com> parents: 40176 diff changeset	1014 yield extra
41e2633bcd00 wireprotov2: extract file object emission to own function Gregory Szorc <gregory.szorc@gmail.com> parents: 40176 diff changeset	1015
40178 46a40bce3ae0 wireprotov2: define and implement "filesdata" command Gregory Szorc <gregory.szorc@gmail.com> parents: 40177 diff changeset	1016 def makefilematcher(repo, pathfilter):
46a40bce3ae0 wireprotov2: define and implement "filesdata" command Gregory Szorc <gregory.szorc@gmail.com> parents: 40177 diff changeset	1017 """Construct a matcher from a path filter dict."""
46a40bce3ae0 wireprotov2: define and implement "filesdata" command Gregory Szorc <gregory.szorc@gmail.com> parents: 40177 diff changeset	1018
46a40bce3ae0 wireprotov2: define and implement "filesdata" command Gregory Szorc <gregory.szorc@gmail.com> parents: 40177 diff changeset	1019 # Validate values.
46a40bce3ae0 wireprotov2: define and implement "filesdata" command Gregory Szorc <gregory.szorc@gmail.com> parents: 40177 diff changeset	1020 if pathfilter:
46a40bce3ae0 wireprotov2: define and implement "filesdata" command Gregory Szorc <gregory.szorc@gmail.com> parents: 40177 diff changeset	1021 for key in (b'include', b'exclude'):
46a40bce3ae0 wireprotov2: define and implement "filesdata" command Gregory Szorc <gregory.szorc@gmail.com> parents: 40177 diff changeset	1022 for pattern in pathfilter.get(key, []):
46a40bce3ae0 wireprotov2: define and implement "filesdata" command Gregory Szorc <gregory.szorc@gmail.com> parents: 40177 diff changeset	1023 if not pattern.startswith((b'path:', b'rootfilesin:')):
46a40bce3ae0 wireprotov2: define and implement "filesdata" command Gregory Szorc <gregory.szorc@gmail.com> parents: 40177 diff changeset	1024 raise error.WireprotoCommandError(
46a40bce3ae0 wireprotov2: define and implement "filesdata" command Gregory Szorc <gregory.szorc@gmail.com> parents: 40177 diff changeset	1025 '%s pattern must begin with `path:` or `rootfilesin:`; '
46a40bce3ae0 wireprotov2: define and implement "filesdata" command Gregory Szorc <gregory.szorc@gmail.com> parents: 40177 diff changeset	1026 'got %s', (key, pattern))
46a40bce3ae0 wireprotov2: define and implement "filesdata" command Gregory Szorc <gregory.szorc@gmail.com> parents: 40177 diff changeset	1027
46a40bce3ae0 wireprotov2: define and implement "filesdata" command Gregory Szorc <gregory.szorc@gmail.com> parents: 40177 diff changeset	1028 if pathfilter:
46a40bce3ae0 wireprotov2: define and implement "filesdata" command Gregory Szorc <gregory.szorc@gmail.com> parents: 40177 diff changeset	1029 matcher = matchmod.match(repo.root, b'',
46a40bce3ae0 wireprotov2: define and implement "filesdata" command Gregory Szorc <gregory.szorc@gmail.com> parents: 40177 diff changeset	1030 include=pathfilter.get(b'include', []),
46a40bce3ae0 wireprotov2: define and implement "filesdata" command Gregory Szorc <gregory.szorc@gmail.com> parents: 40177 diff changeset	1031 exclude=pathfilter.get(b'exclude', []))
46a40bce3ae0 wireprotov2: define and implement "filesdata" command Gregory Szorc <gregory.szorc@gmail.com> parents: 40177 diff changeset	1032 else:
46a40bce3ae0 wireprotov2: define and implement "filesdata" command Gregory Szorc <gregory.szorc@gmail.com> parents: 40177 diff changeset	1033 matcher = matchmod.match(repo.root, b'')
46a40bce3ae0 wireprotov2: define and implement "filesdata" command Gregory Szorc <gregory.szorc@gmail.com> parents: 40177 diff changeset	1034
46a40bce3ae0 wireprotov2: define and implement "filesdata" command Gregory Szorc <gregory.szorc@gmail.com> parents: 40177 diff changeset	1035 # Requested patterns could include files not in the local store. So
46a40bce3ae0 wireprotov2: define and implement "filesdata" command Gregory Szorc <gregory.szorc@gmail.com> parents: 40177 diff changeset	1036 # filter those out.
46a40bce3ae0 wireprotov2: define and implement "filesdata" command Gregory Szorc <gregory.szorc@gmail.com> parents: 40177 diff changeset	1037 return matchmod.intersectmatchers(repo.narrowmatch(), matcher)
46a40bce3ae0 wireprotov2: define and implement "filesdata" command Gregory Szorc <gregory.szorc@gmail.com> parents: 40177 diff changeset	1038
39810 0b61d21f05cc wireprotov2: declare command arguments richly Gregory Szorc <gregory.szorc@gmail.com> parents: 39641 diff changeset	1039 @wireprotocommand(
0b61d21f05cc wireprotov2: declare command arguments richly Gregory Szorc <gregory.szorc@gmail.com> parents: 39641 diff changeset	1040 'filedata',
0b61d21f05cc wireprotov2: declare command arguments richly Gregory Szorc <gregory.szorc@gmail.com> parents: 39641 diff changeset	1041 args={
0b61d21f05cc wireprotov2: declare command arguments richly Gregory Szorc <gregory.szorc@gmail.com> parents: 39641 diff changeset	1042 'haveparents': {
0b61d21f05cc wireprotov2: declare command arguments richly Gregory Szorc <gregory.szorc@gmail.com> parents: 39641 diff changeset	1043 'type': 'bool',
0b61d21f05cc wireprotov2: declare command arguments richly Gregory Szorc <gregory.szorc@gmail.com> parents: 39641 diff changeset	1044 'default': lambda: False,
0b61d21f05cc wireprotov2: declare command arguments richly Gregory Szorc <gregory.szorc@gmail.com> parents: 39641 diff changeset	1045 'example': True,
0b61d21f05cc wireprotov2: declare command arguments richly Gregory Szorc <gregory.szorc@gmail.com> parents: 39641 diff changeset	1046 },
0b61d21f05cc wireprotov2: declare command arguments richly Gregory Szorc <gregory.szorc@gmail.com> parents: 39641 diff changeset	1047 'nodes': {
0b61d21f05cc wireprotov2: declare command arguments richly Gregory Szorc <gregory.szorc@gmail.com> parents: 39641 diff changeset	1048 'type': 'list',
0b61d21f05cc wireprotov2: declare command arguments richly Gregory Szorc <gregory.szorc@gmail.com> parents: 39641 diff changeset	1049 'example': [b'0123456...'],
0b61d21f05cc wireprotov2: declare command arguments richly Gregory Szorc <gregory.szorc@gmail.com> parents: 39641 diff changeset	1050 },
0b61d21f05cc wireprotov2: declare command arguments richly Gregory Szorc <gregory.szorc@gmail.com> parents: 39641 diff changeset	1051 'fields': {
0b61d21f05cc wireprotov2: declare command arguments richly Gregory Szorc <gregory.szorc@gmail.com> parents: 39641 diff changeset	1052 'type': 'set',
0b61d21f05cc wireprotov2: declare command arguments richly Gregory Szorc <gregory.szorc@gmail.com> parents: 39641 diff changeset	1053 'default': set,
0b61d21f05cc wireprotov2: declare command arguments richly Gregory Szorc <gregory.szorc@gmail.com> parents: 39641 diff changeset	1054 'example': {b'parents', b'revision'},
39813 c30faea8d02d wireprotov2: advertise set of valid values for requestable fields Gregory Szorc <gregory.szorc@gmail.com> parents: 39812 diff changeset	1055 'validvalues': {b'parents', b'revision'},
39810 0b61d21f05cc wireprotov2: declare command arguments richly Gregory Szorc <gregory.szorc@gmail.com> parents: 39641 diff changeset	1056 },
0b61d21f05cc wireprotov2: declare command arguments richly Gregory Szorc <gregory.szorc@gmail.com> parents: 39641 diff changeset	1057 'path': {
0b61d21f05cc wireprotov2: declare command arguments richly Gregory Szorc <gregory.szorc@gmail.com> parents: 39641 diff changeset	1058 'type': 'bytes',
0b61d21f05cc wireprotov2: declare command arguments richly Gregory Szorc <gregory.szorc@gmail.com> parents: 39641 diff changeset	1059 'example': b'foo.txt',
0b61d21f05cc wireprotov2: declare command arguments richly Gregory Szorc <gregory.szorc@gmail.com> parents: 39641 diff changeset	1060 }
0b61d21f05cc wireprotov2: declare command arguments richly Gregory Szorc <gregory.szorc@gmail.com> parents: 39641 diff changeset	1061 },
40021 c537144fdbef wireprotov2: support response caching Gregory Szorc <gregory.szorc@gmail.com> parents: 39995 diff changeset	1062 permission='pull',
c537144fdbef wireprotov2: support response caching Gregory Szorc <gregory.szorc@gmail.com> parents: 39995 diff changeset	1063 # TODO censoring a file revision won't invalidate the cache.
c537144fdbef wireprotov2: support response caching Gregory Szorc <gregory.szorc@gmail.com> parents: 39995 diff changeset	1064 # Figure out a way to take censoring into account when deriving
c537144fdbef wireprotov2: support response caching Gregory Szorc <gregory.szorc@gmail.com> parents: 39995 diff changeset	1065 # the cache key.
c537144fdbef wireprotov2: support response caching Gregory Szorc <gregory.szorc@gmail.com> parents: 39995 diff changeset	1066 cachekeyfn=makecommandcachekeyfn('filedata', 1, allargs=True))
39810 0b61d21f05cc wireprotov2: declare command arguments richly Gregory Szorc <gregory.szorc@gmail.com> parents: 39641 diff changeset	1067 def filedata(repo, proto, haveparents, nodes, fields, path):
40178 46a40bce3ae0 wireprotov2: define and implement "filesdata" command Gregory Szorc <gregory.szorc@gmail.com> parents: 40177 diff changeset	1068 # TODO this API allows access to file revisions that are attached to
46a40bce3ae0 wireprotov2: define and implement "filesdata" command Gregory Szorc <gregory.szorc@gmail.com> parents: 40177 diff changeset	1069 # secret changesets. filesdata does not have this problem. Maybe this
46a40bce3ae0 wireprotov2: define and implement "filesdata" command Gregory Szorc <gregory.szorc@gmail.com> parents: 40177 diff changeset	1070 # API should be deleted?
46a40bce3ae0 wireprotov2: define and implement "filesdata" command Gregory Szorc <gregory.szorc@gmail.com> parents: 40177 diff changeset	1071
39639 0e03e6a44dee wireprotov2: define and implement "filedata" command Gregory Szorc <gregory.szorc@gmail.com> parents: 39637 diff changeset	1072 try:
0e03e6a44dee wireprotov2: define and implement "filedata" command Gregory Szorc <gregory.szorc@gmail.com> parents: 39637 diff changeset	1073 # Extensions may wish to access the protocol handler.
0e03e6a44dee wireprotov2: define and implement "filedata" command Gregory Szorc <gregory.szorc@gmail.com> parents: 39637 diff changeset	1074 store = getfilestore(repo, proto, path)
0e03e6a44dee wireprotov2: define and implement "filedata" command Gregory Szorc <gregory.szorc@gmail.com> parents: 39637 diff changeset	1075 except FileAccessError as e:
0e03e6a44dee wireprotov2: define and implement "filedata" command Gregory Szorc <gregory.szorc@gmail.com> parents: 39637 diff changeset	1076 raise error.WireprotoCommandError(e.msg, e.args)
0e03e6a44dee wireprotov2: define and implement "filedata" command Gregory Szorc <gregory.szorc@gmail.com> parents: 39637 diff changeset	1077
0e03e6a44dee wireprotov2: define and implement "filedata" command Gregory Szorc <gregory.szorc@gmail.com> parents: 39637 diff changeset	1078 # Validate requested nodes.
0e03e6a44dee wireprotov2: define and implement "filedata" command Gregory Szorc <gregory.szorc@gmail.com> parents: 39637 diff changeset	1079 for node in nodes:
0e03e6a44dee wireprotov2: define and implement "filedata" command Gregory Szorc <gregory.szorc@gmail.com> parents: 39637 diff changeset	1080 try:
0e03e6a44dee wireprotov2: define and implement "filedata" command Gregory Szorc <gregory.szorc@gmail.com> parents: 39637 diff changeset	1081 store.rev(node)
0e03e6a44dee wireprotov2: define and implement "filedata" command Gregory Szorc <gregory.szorc@gmail.com> parents: 39637 diff changeset	1082 except error.LookupError:
0e03e6a44dee wireprotov2: define and implement "filedata" command Gregory Szorc <gregory.szorc@gmail.com> parents: 39637 diff changeset	1083 raise error.WireprotoCommandError('unknown file node: %s',
0e03e6a44dee wireprotov2: define and implement "filedata" command Gregory Szorc <gregory.szorc@gmail.com> parents: 39637 diff changeset	1084 (hex(node),))
0e03e6a44dee wireprotov2: define and implement "filedata" command Gregory Szorc <gregory.szorc@gmail.com> parents: 39637 diff changeset	1085
39864 7b752bf08435 wireprotov2server: port to emitrevisions() Gregory Szorc <gregory.szorc@gmail.com> parents: 39837 diff changeset	1086 revisions = store.emitrevisions(nodes,
7b752bf08435 wireprotov2server: port to emitrevisions() Gregory Szorc <gregory.szorc@gmail.com> parents: 39837 diff changeset	1087 revisiondata=b'revision' in fields,
7b752bf08435 wireprotov2server: port to emitrevisions() Gregory Szorc <gregory.szorc@gmail.com> parents: 39837 diff changeset	1088 assumehaveparentrevisions=haveparents)
39639 0e03e6a44dee wireprotov2: define and implement "filedata" command Gregory Szorc <gregory.szorc@gmail.com> parents: 39637 diff changeset	1089
0e03e6a44dee wireprotov2: define and implement "filedata" command Gregory Szorc <gregory.szorc@gmail.com> parents: 39637 diff changeset	1090 yield {
39864 7b752bf08435 wireprotov2server: port to emitrevisions() Gregory Szorc <gregory.szorc@gmail.com> parents: 39837 diff changeset	1091 b'totalitems': len(nodes),
39639 0e03e6a44dee wireprotov2: define and implement "filedata" command Gregory Szorc <gregory.szorc@gmail.com> parents: 39637 diff changeset	1092 }
0e03e6a44dee wireprotov2: define and implement "filedata" command Gregory Szorc <gregory.szorc@gmail.com> parents: 39637 diff changeset	1093
40177 41e2633bcd00 wireprotov2: extract file object emission to own function Gregory Szorc <gregory.szorc@gmail.com> parents: 40176 diff changeset	1094 for o in emitfilerevisions(revisions, fields):
41e2633bcd00 wireprotov2: extract file object emission to own function Gregory Szorc <gregory.szorc@gmail.com> parents: 40176 diff changeset	1095 yield o
39639 0e03e6a44dee wireprotov2: define and implement "filedata" command Gregory Szorc <gregory.szorc@gmail.com> parents: 39637 diff changeset	1096
40178 46a40bce3ae0 wireprotov2: define and implement "filesdata" command Gregory Szorc <gregory.szorc@gmail.com> parents: 40177 diff changeset	1097 def filesdatacapabilities(repo, proto):
46a40bce3ae0 wireprotov2: define and implement "filesdata" command Gregory Szorc <gregory.szorc@gmail.com> parents: 40177 diff changeset	1098 batchsize = repo.ui.configint(
46a40bce3ae0 wireprotov2: define and implement "filesdata" command Gregory Szorc <gregory.szorc@gmail.com> parents: 40177 diff changeset	1099 b'experimental', b'server.filesdata.recommended-batch-size')
46a40bce3ae0 wireprotov2: define and implement "filesdata" command Gregory Szorc <gregory.szorc@gmail.com> parents: 40177 diff changeset	1100 return {
46a40bce3ae0 wireprotov2: define and implement "filesdata" command Gregory Szorc <gregory.szorc@gmail.com> parents: 40177 diff changeset	1101 b'recommendedbatchsize': batchsize,
46a40bce3ae0 wireprotov2: define and implement "filesdata" command Gregory Szorc <gregory.szorc@gmail.com> parents: 40177 diff changeset	1102 }
46a40bce3ae0 wireprotov2: define and implement "filesdata" command Gregory Szorc <gregory.szorc@gmail.com> parents: 40177 diff changeset	1103
46a40bce3ae0 wireprotov2: define and implement "filesdata" command Gregory Szorc <gregory.szorc@gmail.com> parents: 40177 diff changeset	1104 @wireprotocommand(
46a40bce3ae0 wireprotov2: define and implement "filesdata" command Gregory Szorc <gregory.szorc@gmail.com> parents: 40177 diff changeset	1105 'filesdata',
46a40bce3ae0 wireprotov2: define and implement "filesdata" command Gregory Szorc <gregory.szorc@gmail.com> parents: 40177 diff changeset	1106 args={
46a40bce3ae0 wireprotov2: define and implement "filesdata" command Gregory Szorc <gregory.szorc@gmail.com> parents: 40177 diff changeset	1107 'haveparents': {
46a40bce3ae0 wireprotov2: define and implement "filesdata" command Gregory Szorc <gregory.szorc@gmail.com> parents: 40177 diff changeset	1108 'type': 'bool',
46a40bce3ae0 wireprotov2: define and implement "filesdata" command Gregory Szorc <gregory.szorc@gmail.com> parents: 40177 diff changeset	1109 'default': lambda: False,
46a40bce3ae0 wireprotov2: define and implement "filesdata" command Gregory Szorc <gregory.szorc@gmail.com> parents: 40177 diff changeset	1110 'example': True,
46a40bce3ae0 wireprotov2: define and implement "filesdata" command Gregory Szorc <gregory.szorc@gmail.com> parents: 40177 diff changeset	1111 },
46a40bce3ae0 wireprotov2: define and implement "filesdata" command Gregory Szorc <gregory.szorc@gmail.com> parents: 40177 diff changeset	1112 'fields': {
46a40bce3ae0 wireprotov2: define and implement "filesdata" command Gregory Szorc <gregory.szorc@gmail.com> parents: 40177 diff changeset	1113 'type': 'set',
46a40bce3ae0 wireprotov2: define and implement "filesdata" command Gregory Szorc <gregory.szorc@gmail.com> parents: 40177 diff changeset	1114 'default': set,
46a40bce3ae0 wireprotov2: define and implement "filesdata" command Gregory Szorc <gregory.szorc@gmail.com> parents: 40177 diff changeset	1115 'example': {b'parents', b'revision'},
46a40bce3ae0 wireprotov2: define and implement "filesdata" command Gregory Szorc <gregory.szorc@gmail.com> parents: 40177 diff changeset	1116 'validvalues': {b'firstchangeset', b'parents', b'revision'},
46a40bce3ae0 wireprotov2: define and implement "filesdata" command Gregory Szorc <gregory.szorc@gmail.com> parents: 40177 diff changeset	1117 },
46a40bce3ae0 wireprotov2: define and implement "filesdata" command Gregory Szorc <gregory.szorc@gmail.com> parents: 40177 diff changeset	1118 'pathfilter': {
46a40bce3ae0 wireprotov2: define and implement "filesdata" command Gregory Szorc <gregory.szorc@gmail.com> parents: 40177 diff changeset	1119 'type': 'dict',
46a40bce3ae0 wireprotov2: define and implement "filesdata" command Gregory Szorc <gregory.szorc@gmail.com> parents: 40177 diff changeset	1120 'default': lambda: None,
46a40bce3ae0 wireprotov2: define and implement "filesdata" command Gregory Szorc <gregory.szorc@gmail.com> parents: 40177 diff changeset	1121 'example': {b'include': [b'path:tests']},
46a40bce3ae0 wireprotov2: define and implement "filesdata" command Gregory Szorc <gregory.szorc@gmail.com> parents: 40177 diff changeset	1122 },
46a40bce3ae0 wireprotov2: define and implement "filesdata" command Gregory Szorc <gregory.szorc@gmail.com> parents: 40177 diff changeset	1123 'revisions': {
46a40bce3ae0 wireprotov2: define and implement "filesdata" command Gregory Szorc <gregory.szorc@gmail.com> parents: 40177 diff changeset	1124 'type': 'list',
46a40bce3ae0 wireprotov2: define and implement "filesdata" command Gregory Szorc <gregory.szorc@gmail.com> parents: 40177 diff changeset	1125 'example': [{
46a40bce3ae0 wireprotov2: define and implement "filesdata" command Gregory Szorc <gregory.szorc@gmail.com> parents: 40177 diff changeset	1126 b'type': b'changesetexplicit',
46a40bce3ae0 wireprotov2: define and implement "filesdata" command Gregory Szorc <gregory.szorc@gmail.com> parents: 40177 diff changeset	1127 b'nodes': [b'abcdef...'],
46a40bce3ae0 wireprotov2: define and implement "filesdata" command Gregory Szorc <gregory.szorc@gmail.com> parents: 40177 diff changeset	1128 }],
46a40bce3ae0 wireprotov2: define and implement "filesdata" command Gregory Szorc <gregory.szorc@gmail.com> parents: 40177 diff changeset	1129 },
46a40bce3ae0 wireprotov2: define and implement "filesdata" command Gregory Szorc <gregory.szorc@gmail.com> parents: 40177 diff changeset	1130 },
46a40bce3ae0 wireprotov2: define and implement "filesdata" command Gregory Szorc <gregory.szorc@gmail.com> parents: 40177 diff changeset	1131 permission='pull',
46a40bce3ae0 wireprotov2: define and implement "filesdata" command Gregory Szorc <gregory.szorc@gmail.com> parents: 40177 diff changeset	1132 # TODO censoring a file revision won't invalidate the cache.
46a40bce3ae0 wireprotov2: define and implement "filesdata" command Gregory Szorc <gregory.szorc@gmail.com> parents: 40177 diff changeset	1133 # Figure out a way to take censoring into account when deriving
46a40bce3ae0 wireprotov2: define and implement "filesdata" command Gregory Szorc <gregory.szorc@gmail.com> parents: 40177 diff changeset	1134 # the cache key.
46a40bce3ae0 wireprotov2: define and implement "filesdata" command Gregory Szorc <gregory.szorc@gmail.com> parents: 40177 diff changeset	1135 cachekeyfn=makecommandcachekeyfn('filesdata', 1, allargs=True),
46a40bce3ae0 wireprotov2: define and implement "filesdata" command Gregory Szorc <gregory.szorc@gmail.com> parents: 40177 diff changeset	1136 extracapabilitiesfn=filesdatacapabilities)
46a40bce3ae0 wireprotov2: define and implement "filesdata" command Gregory Szorc <gregory.szorc@gmail.com> parents: 40177 diff changeset	1137 def filesdata(repo, proto, haveparents, fields, pathfilter, revisions):
46a40bce3ae0 wireprotov2: define and implement "filesdata" command Gregory Szorc <gregory.szorc@gmail.com> parents: 40177 diff changeset	1138 # TODO This should operate on a repo that exposes obsolete changesets. There
46a40bce3ae0 wireprotov2: define and implement "filesdata" command Gregory Szorc <gregory.szorc@gmail.com> parents: 40177 diff changeset	1139 # is a race between a client making a push that obsoletes a changeset and
46a40bce3ae0 wireprotov2: define and implement "filesdata" command Gregory Szorc <gregory.szorc@gmail.com> parents: 40177 diff changeset	1140 # another client fetching files data for that changeset. If a client has a
46a40bce3ae0 wireprotov2: define and implement "filesdata" command Gregory Szorc <gregory.szorc@gmail.com> parents: 40177 diff changeset	1141 # changeset, it should probably be allowed to access files data for that
46a40bce3ae0 wireprotov2: define and implement "filesdata" command Gregory Szorc <gregory.szorc@gmail.com> parents: 40177 diff changeset	1142 # changeset.
46a40bce3ae0 wireprotov2: define and implement "filesdata" command Gregory Szorc <gregory.szorc@gmail.com> parents: 40177 diff changeset	1143
46a40bce3ae0 wireprotov2: define and implement "filesdata" command Gregory Szorc <gregory.szorc@gmail.com> parents: 40177 diff changeset	1144 cl = repo.changelog
46a40bce3ae0 wireprotov2: define and implement "filesdata" command Gregory Szorc <gregory.szorc@gmail.com> parents: 40177 diff changeset	1145 outgoing = resolvenodes(repo, revisions)
46a40bce3ae0 wireprotov2: define and implement "filesdata" command Gregory Szorc <gregory.szorc@gmail.com> parents: 40177 diff changeset	1146 filematcher = makefilematcher(repo, pathfilter)
46a40bce3ae0 wireprotov2: define and implement "filesdata" command Gregory Szorc <gregory.szorc@gmail.com> parents: 40177 diff changeset	1147
46a40bce3ae0 wireprotov2: define and implement "filesdata" command Gregory Szorc <gregory.szorc@gmail.com> parents: 40177 diff changeset	1148 # Figure out what needs to be emitted.
46a40bce3ae0 wireprotov2: define and implement "filesdata" command Gregory Szorc <gregory.szorc@gmail.com> parents: 40177 diff changeset	1149 changedpaths = set()
46a40bce3ae0 wireprotov2: define and implement "filesdata" command Gregory Szorc <gregory.szorc@gmail.com> parents: 40177 diff changeset	1150 fnodes = collections.defaultdict(set)
46a40bce3ae0 wireprotov2: define and implement "filesdata" command Gregory Szorc <gregory.szorc@gmail.com> parents: 40177 diff changeset	1151
46a40bce3ae0 wireprotov2: define and implement "filesdata" command Gregory Szorc <gregory.szorc@gmail.com> parents: 40177 diff changeset	1152 for node in outgoing:
46a40bce3ae0 wireprotov2: define and implement "filesdata" command Gregory Szorc <gregory.szorc@gmail.com> parents: 40177 diff changeset	1153 ctx = repo[node]
46a40bce3ae0 wireprotov2: define and implement "filesdata" command Gregory Szorc <gregory.szorc@gmail.com> parents: 40177 diff changeset	1154 changedpaths.update(ctx.files())
46a40bce3ae0 wireprotov2: define and implement "filesdata" command Gregory Szorc <gregory.szorc@gmail.com> parents: 40177 diff changeset	1155
46a40bce3ae0 wireprotov2: define and implement "filesdata" command Gregory Szorc <gregory.szorc@gmail.com> parents: 40177 diff changeset	1156 changedpaths = sorted(p for p in changedpaths if filematcher(p))
46a40bce3ae0 wireprotov2: define and implement "filesdata" command Gregory Szorc <gregory.szorc@gmail.com> parents: 40177 diff changeset	1157
46a40bce3ae0 wireprotov2: define and implement "filesdata" command Gregory Szorc <gregory.szorc@gmail.com> parents: 40177 diff changeset	1158 # If ancestors are known, we send file revisions having a linkrev in the
46a40bce3ae0 wireprotov2: define and implement "filesdata" command Gregory Szorc <gregory.szorc@gmail.com> parents: 40177 diff changeset	1159 # outgoing set of changeset revisions.
46a40bce3ae0 wireprotov2: define and implement "filesdata" command Gregory Szorc <gregory.szorc@gmail.com> parents: 40177 diff changeset	1160 if haveparents:
46a40bce3ae0 wireprotov2: define and implement "filesdata" command Gregory Szorc <gregory.szorc@gmail.com> parents: 40177 diff changeset	1161 outgoingclrevs = set(cl.rev(n) for n in outgoing)
46a40bce3ae0 wireprotov2: define and implement "filesdata" command Gregory Szorc <gregory.szorc@gmail.com> parents: 40177 diff changeset	1162
46a40bce3ae0 wireprotov2: define and implement "filesdata" command Gregory Szorc <gregory.szorc@gmail.com> parents: 40177 diff changeset	1163 for path in changedpaths:
46a40bce3ae0 wireprotov2: define and implement "filesdata" command Gregory Szorc <gregory.szorc@gmail.com> parents: 40177 diff changeset	1164 try:
46a40bce3ae0 wireprotov2: define and implement "filesdata" command Gregory Szorc <gregory.szorc@gmail.com> parents: 40177 diff changeset	1165 store = getfilestore(repo, proto, path)
46a40bce3ae0 wireprotov2: define and implement "filesdata" command Gregory Szorc <gregory.szorc@gmail.com> parents: 40177 diff changeset	1166 except FileAccessError as e:
46a40bce3ae0 wireprotov2: define and implement "filesdata" command Gregory Szorc <gregory.szorc@gmail.com> parents: 40177 diff changeset	1167 raise error.WireprotoCommandError(e.msg, e.args)
46a40bce3ae0 wireprotov2: define and implement "filesdata" command Gregory Szorc <gregory.szorc@gmail.com> parents: 40177 diff changeset	1168
46a40bce3ae0 wireprotov2: define and implement "filesdata" command Gregory Szorc <gregory.szorc@gmail.com> parents: 40177 diff changeset	1169 for rev in store:
46a40bce3ae0 wireprotov2: define and implement "filesdata" command Gregory Szorc <gregory.szorc@gmail.com> parents: 40177 diff changeset	1170 linkrev = store.linkrev(rev)
46a40bce3ae0 wireprotov2: define and implement "filesdata" command Gregory Szorc <gregory.szorc@gmail.com> parents: 40177 diff changeset	1171
46a40bce3ae0 wireprotov2: define and implement "filesdata" command Gregory Szorc <gregory.szorc@gmail.com> parents: 40177 diff changeset	1172 if linkrev in outgoingclrevs:
46a40bce3ae0 wireprotov2: define and implement "filesdata" command Gregory Szorc <gregory.szorc@gmail.com> parents: 40177 diff changeset	1173 fnodes[path].add(store.node(rev))
46a40bce3ae0 wireprotov2: define and implement "filesdata" command Gregory Szorc <gregory.szorc@gmail.com> parents: 40177 diff changeset	1174
46a40bce3ae0 wireprotov2: define and implement "filesdata" command Gregory Szorc <gregory.szorc@gmail.com> parents: 40177 diff changeset	1175 # If ancestors aren't known, we walk the manifests and send all
46a40bce3ae0 wireprotov2: define and implement "filesdata" command Gregory Szorc <gregory.szorc@gmail.com> parents: 40177 diff changeset	1176 # encountered file revisions.
46a40bce3ae0 wireprotov2: define and implement "filesdata" command Gregory Szorc <gregory.szorc@gmail.com> parents: 40177 diff changeset	1177 else:
46a40bce3ae0 wireprotov2: define and implement "filesdata" command Gregory Szorc <gregory.szorc@gmail.com> parents: 40177 diff changeset	1178 for node in outgoing:
46a40bce3ae0 wireprotov2: define and implement "filesdata" command Gregory Szorc <gregory.szorc@gmail.com> parents: 40177 diff changeset	1179 mctx = repo[node].manifestctx()
46a40bce3ae0 wireprotov2: define and implement "filesdata" command Gregory Szorc <gregory.szorc@gmail.com> parents: 40177 diff changeset	1180
46a40bce3ae0 wireprotov2: define and implement "filesdata" command Gregory Szorc <gregory.szorc@gmail.com> parents: 40177 diff changeset	1181 for path, fnode in mctx.read().items():
46a40bce3ae0 wireprotov2: define and implement "filesdata" command Gregory Szorc <gregory.szorc@gmail.com> parents: 40177 diff changeset	1182 if filematcher(path):
46a40bce3ae0 wireprotov2: define and implement "filesdata" command Gregory Szorc <gregory.szorc@gmail.com> parents: 40177 diff changeset	1183 fnodes[path].add(fnode)
46a40bce3ae0 wireprotov2: define and implement "filesdata" command Gregory Szorc <gregory.szorc@gmail.com> parents: 40177 diff changeset	1184
46a40bce3ae0 wireprotov2: define and implement "filesdata" command Gregory Szorc <gregory.szorc@gmail.com> parents: 40177 diff changeset	1185 yield {
46a40bce3ae0 wireprotov2: define and implement "filesdata" command Gregory Szorc <gregory.szorc@gmail.com> parents: 40177 diff changeset	1186 b'totalpaths': len(fnodes),
46a40bce3ae0 wireprotov2: define and implement "filesdata" command Gregory Szorc <gregory.szorc@gmail.com> parents: 40177 diff changeset	1187 b'totalitems': sum(len(v) for v in fnodes.values())
46a40bce3ae0 wireprotov2: define and implement "filesdata" command Gregory Szorc <gregory.szorc@gmail.com> parents: 40177 diff changeset	1188 }
46a40bce3ae0 wireprotov2: define and implement "filesdata" command Gregory Szorc <gregory.szorc@gmail.com> parents: 40177 diff changeset	1189
46a40bce3ae0 wireprotov2: define and implement "filesdata" command Gregory Szorc <gregory.szorc@gmail.com> parents: 40177 diff changeset	1190 for path, filenodes in sorted(fnodes.items()):
46a40bce3ae0 wireprotov2: define and implement "filesdata" command Gregory Szorc <gregory.szorc@gmail.com> parents: 40177 diff changeset	1191 try:
46a40bce3ae0 wireprotov2: define and implement "filesdata" command Gregory Szorc <gregory.szorc@gmail.com> parents: 40177 diff changeset	1192 store = getfilestore(repo, proto, path)
46a40bce3ae0 wireprotov2: define and implement "filesdata" command Gregory Szorc <gregory.szorc@gmail.com> parents: 40177 diff changeset	1193 except FileAccessError as e:
46a40bce3ae0 wireprotov2: define and implement "filesdata" command Gregory Szorc <gregory.szorc@gmail.com> parents: 40177 diff changeset	1194 raise error.WireprotoCommandError(e.msg, e.args)
46a40bce3ae0 wireprotov2: define and implement "filesdata" command Gregory Szorc <gregory.szorc@gmail.com> parents: 40177 diff changeset	1195
46a40bce3ae0 wireprotov2: define and implement "filesdata" command Gregory Szorc <gregory.szorc@gmail.com> parents: 40177 diff changeset	1196 yield {
46a40bce3ae0 wireprotov2: define and implement "filesdata" command Gregory Szorc <gregory.szorc@gmail.com> parents: 40177 diff changeset	1197 b'path': path,
46a40bce3ae0 wireprotov2: define and implement "filesdata" command Gregory Szorc <gregory.szorc@gmail.com> parents: 40177 diff changeset	1198 b'totalitems': len(filenodes),
46a40bce3ae0 wireprotov2: define and implement "filesdata" command Gregory Szorc <gregory.szorc@gmail.com> parents: 40177 diff changeset	1199 }
46a40bce3ae0 wireprotov2: define and implement "filesdata" command Gregory Szorc <gregory.szorc@gmail.com> parents: 40177 diff changeset	1200
46a40bce3ae0 wireprotov2: define and implement "filesdata" command Gregory Szorc <gregory.szorc@gmail.com> parents: 40177 diff changeset	1201 revisions = store.emitrevisions(filenodes,
46a40bce3ae0 wireprotov2: define and implement "filesdata" command Gregory Szorc <gregory.szorc@gmail.com> parents: 40177 diff changeset	1202 revisiondata=b'revision' in fields,
46a40bce3ae0 wireprotov2: define and implement "filesdata" command Gregory Szorc <gregory.szorc@gmail.com> parents: 40177 diff changeset	1203 assumehaveparentrevisions=haveparents)
46a40bce3ae0 wireprotov2: define and implement "filesdata" command Gregory Szorc <gregory.szorc@gmail.com> parents: 40177 diff changeset	1204
46a40bce3ae0 wireprotov2: define and implement "filesdata" command Gregory Szorc <gregory.szorc@gmail.com> parents: 40177 diff changeset	1205 for o in emitfilerevisions(revisions, fields):
46a40bce3ae0 wireprotov2: define and implement "filesdata" command Gregory Szorc <gregory.szorc@gmail.com> parents: 40177 diff changeset	1206 yield o
46a40bce3ae0 wireprotov2: define and implement "filesdata" command Gregory Szorc <gregory.szorc@gmail.com> parents: 40177 diff changeset	1207
39810 0b61d21f05cc wireprotov2: declare command arguments richly Gregory Szorc <gregory.szorc@gmail.com> parents: 39641 diff changeset	1208 @wireprotocommand(
0b61d21f05cc wireprotov2: declare command arguments richly Gregory Szorc <gregory.szorc@gmail.com> parents: 39641 diff changeset	1209 'heads',
0b61d21f05cc wireprotov2: declare command arguments richly Gregory Szorc <gregory.szorc@gmail.com> parents: 39641 diff changeset	1210 args={
0b61d21f05cc wireprotov2: declare command arguments richly Gregory Szorc <gregory.szorc@gmail.com> parents: 39641 diff changeset	1211 'publiconly': {
0b61d21f05cc wireprotov2: declare command arguments richly Gregory Szorc <gregory.szorc@gmail.com> parents: 39641 diff changeset	1212 'type': 'bool',
0b61d21f05cc wireprotov2: declare command arguments richly Gregory Szorc <gregory.szorc@gmail.com> parents: 39641 diff changeset	1213 'default': lambda: False,
0b61d21f05cc wireprotov2: declare command arguments richly Gregory Szorc <gregory.szorc@gmail.com> parents: 39641 diff changeset	1214 'example': False,
0b61d21f05cc wireprotov2: declare command arguments richly Gregory Szorc <gregory.szorc@gmail.com> parents: 39641 diff changeset	1215 },
0b61d21f05cc wireprotov2: declare command arguments richly Gregory Szorc <gregory.szorc@gmail.com> parents: 39641 diff changeset	1216 },
0b61d21f05cc wireprotov2: declare command arguments richly Gregory Szorc <gregory.szorc@gmail.com> parents: 39641 diff changeset	1217 permission='pull')
0b61d21f05cc wireprotov2: declare command arguments richly Gregory Szorc <gregory.szorc@gmail.com> parents: 39641 diff changeset	1218 def headsv2(repo, proto, publiconly):
37546 3a2367e6c6f2 wireproto: move version 2 command handlers to wireprotov2server Gregory Szorc <gregory.szorc@gmail.com> parents: 37545 diff changeset	1219 if publiconly:
3a2367e6c6f2 wireproto: move version 2 command handlers to wireprotov2server Gregory Szorc <gregory.szorc@gmail.com> parents: 37545 diff changeset	1220 repo = repo.filtered('immutable')
3a2367e6c6f2 wireproto: move version 2 command handlers to wireprotov2server Gregory Szorc <gregory.szorc@gmail.com> parents: 37545 diff changeset	1221
39559 07b58266bce3 wireprotov2: implement commands as a generator of objects Gregory Szorc <gregory.szorc@gmail.com> parents: 39448 diff changeset	1222 yield repo.heads()
37546 3a2367e6c6f2 wireproto: move version 2 command handlers to wireprotov2server Gregory Szorc <gregory.szorc@gmail.com> parents: 37545 diff changeset	1223
39810 0b61d21f05cc wireprotov2: declare command arguments richly Gregory Szorc <gregory.szorc@gmail.com> parents: 39641 diff changeset	1224 @wireprotocommand(
0b61d21f05cc wireprotov2: declare command arguments richly Gregory Szorc <gregory.szorc@gmail.com> parents: 39641 diff changeset	1225 'known',
0b61d21f05cc wireprotov2: declare command arguments richly Gregory Szorc <gregory.szorc@gmail.com> parents: 39641 diff changeset	1226 args={
0b61d21f05cc wireprotov2: declare command arguments richly Gregory Szorc <gregory.szorc@gmail.com> parents: 39641 diff changeset	1227 'nodes': {
0b61d21f05cc wireprotov2: declare command arguments richly Gregory Szorc <gregory.szorc@gmail.com> parents: 39641 diff changeset	1228 'type': 'list',
0b61d21f05cc wireprotov2: declare command arguments richly Gregory Szorc <gregory.szorc@gmail.com> parents: 39641 diff changeset	1229 'default': list,
0b61d21f05cc wireprotov2: declare command arguments richly Gregory Szorc <gregory.szorc@gmail.com> parents: 39641 diff changeset	1230 'example': [b'deadbeef'],
0b61d21f05cc wireprotov2: declare command arguments richly Gregory Szorc <gregory.szorc@gmail.com> parents: 39641 diff changeset	1231 },
0b61d21f05cc wireprotov2: declare command arguments richly Gregory Szorc <gregory.szorc@gmail.com> parents: 39641 diff changeset	1232 },
0b61d21f05cc wireprotov2: declare command arguments richly Gregory Szorc <gregory.szorc@gmail.com> parents: 39641 diff changeset	1233 permission='pull')
0b61d21f05cc wireprotov2: declare command arguments richly Gregory Szorc <gregory.szorc@gmail.com> parents: 39641 diff changeset	1234 def knownv2(repo, proto, nodes):
37546 3a2367e6c6f2 wireproto: move version 2 command handlers to wireprotov2server Gregory Szorc <gregory.szorc@gmail.com> parents: 37545 diff changeset	1235 result = b''.join(b'1' if n else b'0' for n in repo.known(nodes))
39559 07b58266bce3 wireprotov2: implement commands as a generator of objects Gregory Szorc <gregory.szorc@gmail.com> parents: 39448 diff changeset	1236 yield result
37546 3a2367e6c6f2 wireproto: move version 2 command handlers to wireprotov2server Gregory Szorc <gregory.szorc@gmail.com> parents: 37545 diff changeset	1237
39810 0b61d21f05cc wireprotov2: declare command arguments richly Gregory Szorc <gregory.szorc@gmail.com> parents: 39641 diff changeset	1238 @wireprotocommand(
0b61d21f05cc wireprotov2: declare command arguments richly Gregory Szorc <gregory.szorc@gmail.com> parents: 39641 diff changeset	1239 'listkeys',
0b61d21f05cc wireprotov2: declare command arguments richly Gregory Szorc <gregory.szorc@gmail.com> parents: 39641 diff changeset	1240 args={
0b61d21f05cc wireprotov2: declare command arguments richly Gregory Szorc <gregory.szorc@gmail.com> parents: 39641 diff changeset	1241 'namespace': {
0b61d21f05cc wireprotov2: declare command arguments richly Gregory Szorc <gregory.szorc@gmail.com> parents: 39641 diff changeset	1242 'type': 'bytes',
0b61d21f05cc wireprotov2: declare command arguments richly Gregory Szorc <gregory.szorc@gmail.com> parents: 39641 diff changeset	1243 'example': b'ns',
0b61d21f05cc wireprotov2: declare command arguments richly Gregory Szorc <gregory.szorc@gmail.com> parents: 39641 diff changeset	1244 },
0b61d21f05cc wireprotov2: declare command arguments richly Gregory Szorc <gregory.szorc@gmail.com> parents: 39641 diff changeset	1245 },
0b61d21f05cc wireprotov2: declare command arguments richly Gregory Szorc <gregory.szorc@gmail.com> parents: 39641 diff changeset	1246 permission='pull')
0b61d21f05cc wireprotov2: declare command arguments richly Gregory Szorc <gregory.szorc@gmail.com> parents: 39641 diff changeset	1247 def listkeysv2(repo, proto, namespace):
37546 3a2367e6c6f2 wireproto: move version 2 command handlers to wireprotov2server Gregory Szorc <gregory.szorc@gmail.com> parents: 37545 diff changeset	1248 keys = repo.listkeys(encoding.tolocal(namespace))
3a2367e6c6f2 wireproto: move version 2 command handlers to wireprotov2server Gregory Szorc <gregory.szorc@gmail.com> parents: 37545 diff changeset	1249 keys = {encoding.fromlocal(k): encoding.fromlocal(v)
3a2367e6c6f2 wireproto: move version 2 command handlers to wireprotov2server Gregory Szorc <gregory.szorc@gmail.com> parents: 37545 diff changeset	1250 for k, v in keys.iteritems()}
3a2367e6c6f2 wireproto: move version 2 command handlers to wireprotov2server Gregory Szorc <gregory.szorc@gmail.com> parents: 37545 diff changeset	1251
39559 07b58266bce3 wireprotov2: implement commands as a generator of objects Gregory Szorc <gregory.szorc@gmail.com> parents: 39448 diff changeset	1252 yield keys
37546 3a2367e6c6f2 wireproto: move version 2 command handlers to wireprotov2server Gregory Szorc <gregory.szorc@gmail.com> parents: 37545 diff changeset	1253
39810 0b61d21f05cc wireprotov2: declare command arguments richly Gregory Szorc <gregory.szorc@gmail.com> parents: 39641 diff changeset	1254 @wireprotocommand(
0b61d21f05cc wireprotov2: declare command arguments richly Gregory Szorc <gregory.szorc@gmail.com> parents: 39641 diff changeset	1255 'lookup',
0b61d21f05cc wireprotov2: declare command arguments richly Gregory Szorc <gregory.szorc@gmail.com> parents: 39641 diff changeset	1256 args={
0b61d21f05cc wireprotov2: declare command arguments richly Gregory Szorc <gregory.szorc@gmail.com> parents: 39641 diff changeset	1257 'key': {
0b61d21f05cc wireprotov2: declare command arguments richly Gregory Szorc <gregory.szorc@gmail.com> parents: 39641 diff changeset	1258 'type': 'bytes',
0b61d21f05cc wireprotov2: declare command arguments richly Gregory Szorc <gregory.szorc@gmail.com> parents: 39641 diff changeset	1259 'example': b'foo',
0b61d21f05cc wireprotov2: declare command arguments richly Gregory Szorc <gregory.szorc@gmail.com> parents: 39641 diff changeset	1260 },
0b61d21f05cc wireprotov2: declare command arguments richly Gregory Szorc <gregory.szorc@gmail.com> parents: 39641 diff changeset	1261 },
0b61d21f05cc wireprotov2: declare command arguments richly Gregory Szorc <gregory.szorc@gmail.com> parents: 39641 diff changeset	1262 permission='pull')
37546 3a2367e6c6f2 wireproto: move version 2 command handlers to wireprotov2server Gregory Szorc <gregory.szorc@gmail.com> parents: 37545 diff changeset	1263 def lookupv2(repo, proto, key):
3a2367e6c6f2 wireproto: move version 2 command handlers to wireprotov2server Gregory Szorc <gregory.szorc@gmail.com> parents: 37545 diff changeset	1264 key = encoding.tolocal(key)
3a2367e6c6f2 wireproto: move version 2 command handlers to wireprotov2server Gregory Szorc <gregory.szorc@gmail.com> parents: 37545 diff changeset	1265
3a2367e6c6f2 wireproto: move version 2 command handlers to wireprotov2server Gregory Szorc <gregory.szorc@gmail.com> parents: 37545 diff changeset	1266 # TODO handle exception.
3a2367e6c6f2 wireproto: move version 2 command handlers to wireprotov2server Gregory Szorc <gregory.szorc@gmail.com> parents: 37545 diff changeset	1267 node = repo.lookup(key)
3a2367e6c6f2 wireproto: move version 2 command handlers to wireprotov2server Gregory Szorc <gregory.szorc@gmail.com> parents: 37545 diff changeset	1268
39559 07b58266bce3 wireprotov2: implement commands as a generator of objects Gregory Szorc <gregory.szorc@gmail.com> parents: 39448 diff changeset	1269 yield node
37546 3a2367e6c6f2 wireproto: move version 2 command handlers to wireprotov2server Gregory Szorc <gregory.szorc@gmail.com> parents: 37545 diff changeset	1270
40172 30f70d11c224 wireprotov2: advertise recommended batch size for requests Gregory Szorc <gregory.szorc@gmail.com> parents: 40138 diff changeset	1271 def manifestdatacapabilities(repo, proto):
30f70d11c224 wireprotov2: advertise recommended batch size for requests Gregory Szorc <gregory.szorc@gmail.com> parents: 40138 diff changeset	1272 batchsize = repo.ui.configint(
30f70d11c224 wireprotov2: advertise recommended batch size for requests Gregory Szorc <gregory.szorc@gmail.com> parents: 40138 diff changeset	1273 b'experimental', b'server.manifestdata.recommended-batch-size')
30f70d11c224 wireprotov2: advertise recommended batch size for requests Gregory Szorc <gregory.szorc@gmail.com> parents: 40138 diff changeset	1274
30f70d11c224 wireprotov2: advertise recommended batch size for requests Gregory Szorc <gregory.szorc@gmail.com> parents: 40138 diff changeset	1275 return {
30f70d11c224 wireprotov2: advertise recommended batch size for requests Gregory Szorc <gregory.szorc@gmail.com> parents: 40138 diff changeset	1276 b'recommendedbatchsize': batchsize,
30f70d11c224 wireprotov2: advertise recommended batch size for requests Gregory Szorc <gregory.szorc@gmail.com> parents: 40138 diff changeset	1277 }
30f70d11c224 wireprotov2: advertise recommended batch size for requests Gregory Szorc <gregory.szorc@gmail.com> parents: 40138 diff changeset	1278
39810 0b61d21f05cc wireprotov2: declare command arguments richly Gregory Szorc <gregory.szorc@gmail.com> parents: 39641 diff changeset	1279 @wireprotocommand(
0b61d21f05cc wireprotov2: declare command arguments richly Gregory Szorc <gregory.szorc@gmail.com> parents: 39641 diff changeset	1280 'manifestdata',
0b61d21f05cc wireprotov2: declare command arguments richly Gregory Szorc <gregory.szorc@gmail.com> parents: 39641 diff changeset	1281 args={
0b61d21f05cc wireprotov2: declare command arguments richly Gregory Szorc <gregory.szorc@gmail.com> parents: 39641 diff changeset	1282 'nodes': {
0b61d21f05cc wireprotov2: declare command arguments richly Gregory Szorc <gregory.szorc@gmail.com> parents: 39641 diff changeset	1283 'type': 'list',
0b61d21f05cc wireprotov2: declare command arguments richly Gregory Szorc <gregory.szorc@gmail.com> parents: 39641 diff changeset	1284 'example': [b'0123456...'],
0b61d21f05cc wireprotov2: declare command arguments richly Gregory Szorc <gregory.szorc@gmail.com> parents: 39641 diff changeset	1285 },
0b61d21f05cc wireprotov2: declare command arguments richly Gregory Szorc <gregory.szorc@gmail.com> parents: 39641 diff changeset	1286 'haveparents': {
0b61d21f05cc wireprotov2: declare command arguments richly Gregory Szorc <gregory.szorc@gmail.com> parents: 39641 diff changeset	1287 'type': 'bool',
0b61d21f05cc wireprotov2: declare command arguments richly Gregory Szorc <gregory.szorc@gmail.com> parents: 39641 diff changeset	1288 'default': lambda: False,
0b61d21f05cc wireprotov2: declare command arguments richly Gregory Szorc <gregory.szorc@gmail.com> parents: 39641 diff changeset	1289 'example': True,
0b61d21f05cc wireprotov2: declare command arguments richly Gregory Szorc <gregory.szorc@gmail.com> parents: 39641 diff changeset	1290 },
0b61d21f05cc wireprotov2: declare command arguments richly Gregory Szorc <gregory.szorc@gmail.com> parents: 39641 diff changeset	1291 'fields': {
0b61d21f05cc wireprotov2: declare command arguments richly Gregory Szorc <gregory.szorc@gmail.com> parents: 39641 diff changeset	1292 'type': 'set',
0b61d21f05cc wireprotov2: declare command arguments richly Gregory Szorc <gregory.szorc@gmail.com> parents: 39641 diff changeset	1293 'default': set,
0b61d21f05cc wireprotov2: declare command arguments richly Gregory Szorc <gregory.szorc@gmail.com> parents: 39641 diff changeset	1294 'example': {b'parents', b'revision'},
39813 c30faea8d02d wireprotov2: advertise set of valid values for requestable fields Gregory Szorc <gregory.szorc@gmail.com> parents: 39812 diff changeset	1295 'validvalues': {b'parents', b'revision'},
39810 0b61d21f05cc wireprotov2: declare command arguments richly Gregory Szorc <gregory.szorc@gmail.com> parents: 39641 diff changeset	1296 },
0b61d21f05cc wireprotov2: declare command arguments richly Gregory Szorc <gregory.szorc@gmail.com> parents: 39641 diff changeset	1297 'tree': {
0b61d21f05cc wireprotov2: declare command arguments richly Gregory Szorc <gregory.szorc@gmail.com> parents: 39641 diff changeset	1298 'type': 'bytes',
0b61d21f05cc wireprotov2: declare command arguments richly Gregory Szorc <gregory.szorc@gmail.com> parents: 39641 diff changeset	1299 'example': b'',
0b61d21f05cc wireprotov2: declare command arguments richly Gregory Szorc <gregory.szorc@gmail.com> parents: 39641 diff changeset	1300 },
0b61d21f05cc wireprotov2: declare command arguments richly Gregory Szorc <gregory.szorc@gmail.com> parents: 39641 diff changeset	1301 },
40021 c537144fdbef wireprotov2: support response caching Gregory Szorc <gregory.szorc@gmail.com> parents: 39995 diff changeset	1302 permission='pull',
40172 30f70d11c224 wireprotov2: advertise recommended batch size for requests Gregory Szorc <gregory.szorc@gmail.com> parents: 40138 diff changeset	1303 cachekeyfn=makecommandcachekeyfn('manifestdata', 1, allargs=True),
30f70d11c224 wireprotov2: advertise recommended batch size for requests Gregory Szorc <gregory.szorc@gmail.com> parents: 40138 diff changeset	1304 extracapabilitiesfn=manifestdatacapabilities)
39810 0b61d21f05cc wireprotov2: declare command arguments richly Gregory Szorc <gregory.szorc@gmail.com> parents: 39641 diff changeset	1305 def manifestdata(repo, proto, haveparents, nodes, fields, tree):
39637 c7a7c7e844e5 wireprotov2: define and implement "manifestdata" command Gregory Szorc <gregory.szorc@gmail.com> parents: 39636 diff changeset	1306 store = repo.manifestlog.getstorage(tree)
c7a7c7e844e5 wireprotov2: define and implement "manifestdata" command Gregory Szorc <gregory.szorc@gmail.com> parents: 39636 diff changeset	1307
c7a7c7e844e5 wireprotov2: define and implement "manifestdata" command Gregory Szorc <gregory.szorc@gmail.com> parents: 39636 diff changeset	1308 # Validate the node is known and abort on unknown revisions.
c7a7c7e844e5 wireprotov2: define and implement "manifestdata" command Gregory Szorc <gregory.szorc@gmail.com> parents: 39636 diff changeset	1309 for node in nodes:
c7a7c7e844e5 wireprotov2: define and implement "manifestdata" command Gregory Szorc <gregory.szorc@gmail.com> parents: 39636 diff changeset	1310 try:
c7a7c7e844e5 wireprotov2: define and implement "manifestdata" command Gregory Szorc <gregory.szorc@gmail.com> parents: 39636 diff changeset	1311 store.rev(node)
c7a7c7e844e5 wireprotov2: define and implement "manifestdata" command Gregory Szorc <gregory.szorc@gmail.com> parents: 39636 diff changeset	1312 except error.LookupError:
c7a7c7e844e5 wireprotov2: define and implement "manifestdata" command Gregory Szorc <gregory.szorc@gmail.com> parents: 39636 diff changeset	1313 raise error.WireprotoCommandError(
c7a7c7e844e5 wireprotov2: define and implement "manifestdata" command Gregory Szorc <gregory.szorc@gmail.com> parents: 39636 diff changeset	1314 'unknown node: %s', (node,))
c7a7c7e844e5 wireprotov2: define and implement "manifestdata" command Gregory Szorc <gregory.szorc@gmail.com> parents: 39636 diff changeset	1315
39864 7b752bf08435 wireprotov2server: port to emitrevisions() Gregory Szorc <gregory.szorc@gmail.com> parents: 39837 diff changeset	1316 revisions = store.emitrevisions(nodes,
7b752bf08435 wireprotov2server: port to emitrevisions() Gregory Szorc <gregory.szorc@gmail.com> parents: 39837 diff changeset	1317 revisiondata=b'revision' in fields,
7b752bf08435 wireprotov2server: port to emitrevisions() Gregory Szorc <gregory.szorc@gmail.com> parents: 39837 diff changeset	1318 assumehaveparentrevisions=haveparents)
39637 c7a7c7e844e5 wireprotov2: define and implement "manifestdata" command Gregory Szorc <gregory.szorc@gmail.com> parents: 39636 diff changeset	1319
c7a7c7e844e5 wireprotov2: define and implement "manifestdata" command Gregory Szorc <gregory.szorc@gmail.com> parents: 39636 diff changeset	1320 yield {
39864 7b752bf08435 wireprotov2server: port to emitrevisions() Gregory Szorc <gregory.szorc@gmail.com> parents: 39837 diff changeset	1321 b'totalitems': len(nodes),
39637 c7a7c7e844e5 wireprotov2: define and implement "manifestdata" command Gregory Szorc <gregory.szorc@gmail.com> parents: 39636 diff changeset	1322 }
c7a7c7e844e5 wireprotov2: define and implement "manifestdata" command Gregory Szorc <gregory.szorc@gmail.com> parents: 39636 diff changeset	1323
39864 7b752bf08435 wireprotov2server: port to emitrevisions() Gregory Szorc <gregory.szorc@gmail.com> parents: 39837 diff changeset	1324 for revision in revisions:
39637 c7a7c7e844e5 wireprotov2: define and implement "manifestdata" command Gregory Szorc <gregory.szorc@gmail.com> parents: 39636 diff changeset	1325 d = {
39864 7b752bf08435 wireprotov2server: port to emitrevisions() Gregory Szorc <gregory.szorc@gmail.com> parents: 39837 diff changeset	1326 b'node': revision.node,
39637 c7a7c7e844e5 wireprotov2: define and implement "manifestdata" command Gregory Szorc <gregory.szorc@gmail.com> parents: 39636 diff changeset	1327 }
c7a7c7e844e5 wireprotov2: define and implement "manifestdata" command Gregory Szorc <gregory.szorc@gmail.com> parents: 39636 diff changeset	1328
c7a7c7e844e5 wireprotov2: define and implement "manifestdata" command Gregory Szorc <gregory.szorc@gmail.com> parents: 39636 diff changeset	1329 if b'parents' in fields:
39864 7b752bf08435 wireprotov2server: port to emitrevisions() Gregory Szorc <gregory.szorc@gmail.com> parents: 39837 diff changeset	1330 d[b'parents'] = [revision.p1node, revision.p2node]
39637 c7a7c7e844e5 wireprotov2: define and implement "manifestdata" command Gregory Szorc <gregory.szorc@gmail.com> parents: 39636 diff changeset	1331
39814 d059cb669632 wireprotov2: allow multiple fields to follow revision maps Gregory Szorc <gregory.szorc@gmail.com> parents: 39813 diff changeset	1332 followingmeta = []
d059cb669632 wireprotov2: allow multiple fields to follow revision maps Gregory Szorc <gregory.szorc@gmail.com> parents: 39813 diff changeset	1333 followingdata = []
d059cb669632 wireprotov2: allow multiple fields to follow revision maps Gregory Szorc <gregory.szorc@gmail.com> parents: 39813 diff changeset	1334
39637 c7a7c7e844e5 wireprotov2: define and implement "manifestdata" command Gregory Szorc <gregory.szorc@gmail.com> parents: 39636 diff changeset	1335 if b'revision' in fields:
39864 7b752bf08435 wireprotov2server: port to emitrevisions() Gregory Szorc <gregory.szorc@gmail.com> parents: 39837 diff changeset	1336 if revision.revision is not None:
7b752bf08435 wireprotov2server: port to emitrevisions() Gregory Szorc <gregory.szorc@gmail.com> parents: 39837 diff changeset	1337 followingmeta.append((b'revision', len(revision.revision)))
7b752bf08435 wireprotov2server: port to emitrevisions() Gregory Szorc <gregory.szorc@gmail.com> parents: 39837 diff changeset	1338 followingdata.append(revision.revision)
39637 c7a7c7e844e5 wireprotov2: define and implement "manifestdata" command Gregory Szorc <gregory.szorc@gmail.com> parents: 39636 diff changeset	1339 else:
39864 7b752bf08435 wireprotov2server: port to emitrevisions() Gregory Szorc <gregory.szorc@gmail.com> parents: 39837 diff changeset	1340 d[b'deltabasenode'] = revision.basenode
7b752bf08435 wireprotov2server: port to emitrevisions() Gregory Szorc <gregory.szorc@gmail.com> parents: 39837 diff changeset	1341 followingmeta.append((b'delta', len(revision.delta)))
7b752bf08435 wireprotov2server: port to emitrevisions() Gregory Szorc <gregory.szorc@gmail.com> parents: 39837 diff changeset	1342 followingdata.append(revision.delta)
39814 d059cb669632 wireprotov2: allow multiple fields to follow revision maps Gregory Szorc <gregory.szorc@gmail.com> parents: 39813 diff changeset	1343
d059cb669632 wireprotov2: allow multiple fields to follow revision maps Gregory Szorc <gregory.szorc@gmail.com> parents: 39813 diff changeset	1344 if followingmeta:
d059cb669632 wireprotov2: allow multiple fields to follow revision maps Gregory Szorc <gregory.szorc@gmail.com> parents: 39813 diff changeset	1345 d[b'fieldsfollowing'] = followingmeta
39637 c7a7c7e844e5 wireprotov2: define and implement "manifestdata" command Gregory Szorc <gregory.szorc@gmail.com> parents: 39636 diff changeset	1346
c7a7c7e844e5 wireprotov2: define and implement "manifestdata" command Gregory Szorc <gregory.szorc@gmail.com> parents: 39636 diff changeset	1347 yield d
c7a7c7e844e5 wireprotov2: define and implement "manifestdata" command Gregory Szorc <gregory.szorc@gmail.com> parents: 39636 diff changeset	1348
39814 d059cb669632 wireprotov2: allow multiple fields to follow revision maps Gregory Szorc <gregory.szorc@gmail.com> parents: 39813 diff changeset	1349 for extra in followingdata:
d059cb669632 wireprotov2: allow multiple fields to follow revision maps Gregory Szorc <gregory.szorc@gmail.com> parents: 39813 diff changeset	1350 yield extra
39637 c7a7c7e844e5 wireprotov2: define and implement "manifestdata" command Gregory Szorc <gregory.szorc@gmail.com> parents: 39636 diff changeset	1351
39810 0b61d21f05cc wireprotov2: declare command arguments richly Gregory Szorc <gregory.szorc@gmail.com> parents: 39641 diff changeset	1352 @wireprotocommand(
0b61d21f05cc wireprotov2: declare command arguments richly Gregory Szorc <gregory.szorc@gmail.com> parents: 39641 diff changeset	1353 'pushkey',
0b61d21f05cc wireprotov2: declare command arguments richly Gregory Szorc <gregory.szorc@gmail.com> parents: 39641 diff changeset	1354 args={
0b61d21f05cc wireprotov2: declare command arguments richly Gregory Szorc <gregory.szorc@gmail.com> parents: 39641 diff changeset	1355 'namespace': {
0b61d21f05cc wireprotov2: declare command arguments richly Gregory Szorc <gregory.szorc@gmail.com> parents: 39641 diff changeset	1356 'type': 'bytes',
0b61d21f05cc wireprotov2: declare command arguments richly Gregory Szorc <gregory.szorc@gmail.com> parents: 39641 diff changeset	1357 'example': b'ns',
0b61d21f05cc wireprotov2: declare command arguments richly Gregory Szorc <gregory.szorc@gmail.com> parents: 39641 diff changeset	1358 },
0b61d21f05cc wireprotov2: declare command arguments richly Gregory Szorc <gregory.szorc@gmail.com> parents: 39641 diff changeset	1359 'key': {
0b61d21f05cc wireprotov2: declare command arguments richly Gregory Szorc <gregory.szorc@gmail.com> parents: 39641 diff changeset	1360 'type': 'bytes',
0b61d21f05cc wireprotov2: declare command arguments richly Gregory Szorc <gregory.szorc@gmail.com> parents: 39641 diff changeset	1361 'example': b'key',
0b61d21f05cc wireprotov2: declare command arguments richly Gregory Szorc <gregory.szorc@gmail.com> parents: 39641 diff changeset	1362 },
0b61d21f05cc wireprotov2: declare command arguments richly Gregory Szorc <gregory.szorc@gmail.com> parents: 39641 diff changeset	1363 'old': {
0b61d21f05cc wireprotov2: declare command arguments richly Gregory Szorc <gregory.szorc@gmail.com> parents: 39641 diff changeset	1364 'type': 'bytes',
0b61d21f05cc wireprotov2: declare command arguments richly Gregory Szorc <gregory.szorc@gmail.com> parents: 39641 diff changeset	1365 'example': b'old',
0b61d21f05cc wireprotov2: declare command arguments richly Gregory Szorc <gregory.szorc@gmail.com> parents: 39641 diff changeset	1366 },
0b61d21f05cc wireprotov2: declare command arguments richly Gregory Szorc <gregory.szorc@gmail.com> parents: 39641 diff changeset	1367 'new': {
0b61d21f05cc wireprotov2: declare command arguments richly Gregory Szorc <gregory.szorc@gmail.com> parents: 39641 diff changeset	1368 'type': 'bytes',
0b61d21f05cc wireprotov2: declare command arguments richly Gregory Szorc <gregory.szorc@gmail.com> parents: 39641 diff changeset	1369 'example': 'new',
0b61d21f05cc wireprotov2: declare command arguments richly Gregory Szorc <gregory.szorc@gmail.com> parents: 39641 diff changeset	1370 },
0b61d21f05cc wireprotov2: declare command arguments richly Gregory Szorc <gregory.szorc@gmail.com> parents: 39641 diff changeset	1371 },
0b61d21f05cc wireprotov2: declare command arguments richly Gregory Szorc <gregory.szorc@gmail.com> parents: 39641 diff changeset	1372 permission='push')
37546 3a2367e6c6f2 wireproto: move version 2 command handlers to wireprotov2server Gregory Szorc <gregory.szorc@gmail.com> parents: 37545 diff changeset	1373 def pushkeyv2(repo, proto, namespace, key, old, new):
3a2367e6c6f2 wireproto: move version 2 command handlers to wireprotov2server Gregory Szorc <gregory.szorc@gmail.com> parents: 37545 diff changeset	1374 # TODO handle ui output redirection
39559 07b58266bce3 wireprotov2: implement commands as a generator of objects Gregory Szorc <gregory.szorc@gmail.com> parents: 39448 diff changeset	1375 yield repo.pushkey(encoding.tolocal(namespace),
07b58266bce3 wireprotov2: implement commands as a generator of objects Gregory Szorc <gregory.szorc@gmail.com> parents: 39448 diff changeset	1376 encoding.tolocal(key),
07b58266bce3 wireprotov2: implement commands as a generator of objects Gregory Szorc <gregory.szorc@gmail.com> parents: 39448 diff changeset	1377 encoding.tolocal(old),
07b58266bce3 wireprotov2: implement commands as a generator of objects Gregory Szorc <gregory.szorc@gmail.com> parents: 39448 diff changeset	1378 encoding.tolocal(new))
40329 ed55a0077490 wireprotov2: implement command for retrieving raw store files Gregory Szorc <gregory.szorc@gmail.com> parents: 40178 diff changeset	1379
ed55a0077490 wireprotov2: implement command for retrieving raw store files Gregory Szorc <gregory.szorc@gmail.com> parents: 40178 diff changeset	1380
ed55a0077490 wireprotov2: implement command for retrieving raw store files Gregory Szorc <gregory.szorc@gmail.com> parents: 40178 diff changeset	1381 @wireprotocommand(
ed55a0077490 wireprotov2: implement command for retrieving raw store files Gregory Szorc <gregory.szorc@gmail.com> parents: 40178 diff changeset	1382 'rawstorefiledata',
ed55a0077490 wireprotov2: implement command for retrieving raw store files Gregory Szorc <gregory.szorc@gmail.com> parents: 40178 diff changeset	1383 args={
ed55a0077490 wireprotov2: implement command for retrieving raw store files Gregory Szorc <gregory.szorc@gmail.com> parents: 40178 diff changeset	1384 'files': {
ed55a0077490 wireprotov2: implement command for retrieving raw store files Gregory Szorc <gregory.szorc@gmail.com> parents: 40178 diff changeset	1385 'type': 'list',
ed55a0077490 wireprotov2: implement command for retrieving raw store files Gregory Szorc <gregory.szorc@gmail.com> parents: 40178 diff changeset	1386 'example': [b'changelog', b'manifestlog'],
ed55a0077490 wireprotov2: implement command for retrieving raw store files Gregory Szorc <gregory.szorc@gmail.com> parents: 40178 diff changeset	1387 },
ed55a0077490 wireprotov2: implement command for retrieving raw store files Gregory Szorc <gregory.szorc@gmail.com> parents: 40178 diff changeset	1388 'pathfilter': {
ed55a0077490 wireprotov2: implement command for retrieving raw store files Gregory Szorc <gregory.szorc@gmail.com> parents: 40178 diff changeset	1389 'type': 'list',
ed55a0077490 wireprotov2: implement command for retrieving raw store files Gregory Szorc <gregory.szorc@gmail.com> parents: 40178 diff changeset	1390 'default': lambda: None,
ed55a0077490 wireprotov2: implement command for retrieving raw store files Gregory Szorc <gregory.szorc@gmail.com> parents: 40178 diff changeset	1391 'example': {b'include': [b'path:tests']},
ed55a0077490 wireprotov2: implement command for retrieving raw store files Gregory Szorc <gregory.szorc@gmail.com> parents: 40178 diff changeset	1392 },
ed55a0077490 wireprotov2: implement command for retrieving raw store files Gregory Szorc <gregory.szorc@gmail.com> parents: 40178 diff changeset	1393 },
ed55a0077490 wireprotov2: implement command for retrieving raw store files Gregory Szorc <gregory.szorc@gmail.com> parents: 40178 diff changeset	1394 permission='pull')
ed55a0077490 wireprotov2: implement command for retrieving raw store files Gregory Szorc <gregory.szorc@gmail.com> parents: 40178 diff changeset	1395 def rawstorefiledata(repo, proto, files, pathfilter):
ed55a0077490 wireprotov2: implement command for retrieving raw store files Gregory Szorc <gregory.szorc@gmail.com> parents: 40178 diff changeset	1396 if not streamclone.allowservergeneration(repo):
ed55a0077490 wireprotov2: implement command for retrieving raw store files Gregory Szorc <gregory.szorc@gmail.com> parents: 40178 diff changeset	1397 raise error.WireprotoCommandError(b'stream clone is disabled')
ed55a0077490 wireprotov2: implement command for retrieving raw store files Gregory Szorc <gregory.szorc@gmail.com> parents: 40178 diff changeset	1398
ed55a0077490 wireprotov2: implement command for retrieving raw store files Gregory Szorc <gregory.szorc@gmail.com> parents: 40178 diff changeset	1399 # TODO support dynamically advertising what store files "sets" are
ed55a0077490 wireprotov2: implement command for retrieving raw store files Gregory Szorc <gregory.szorc@gmail.com> parents: 40178 diff changeset	1400 # available. For now, we support changelog, manifestlog, and files.
ed55a0077490 wireprotov2: implement command for retrieving raw store files Gregory Szorc <gregory.szorc@gmail.com> parents: 40178 diff changeset	1401 files = set(files)
ed55a0077490 wireprotov2: implement command for retrieving raw store files Gregory Szorc <gregory.szorc@gmail.com> parents: 40178 diff changeset	1402 allowedfiles = {b'changelog', b'manifestlog'}
ed55a0077490 wireprotov2: implement command for retrieving raw store files Gregory Szorc <gregory.szorc@gmail.com> parents: 40178 diff changeset	1403
ed55a0077490 wireprotov2: implement command for retrieving raw store files Gregory Szorc <gregory.szorc@gmail.com> parents: 40178 diff changeset	1404 unsupported = files - allowedfiles
ed55a0077490 wireprotov2: implement command for retrieving raw store files Gregory Szorc <gregory.szorc@gmail.com> parents: 40178 diff changeset	1405 if unsupported:
ed55a0077490 wireprotov2: implement command for retrieving raw store files Gregory Szorc <gregory.szorc@gmail.com> parents: 40178 diff changeset	1406 raise error.WireprotoCommandError(b'unknown file type: %s',
ed55a0077490 wireprotov2: implement command for retrieving raw store files Gregory Szorc <gregory.szorc@gmail.com> parents: 40178 diff changeset	1407 (b', '.join(sorted(unsupported)),))
ed55a0077490 wireprotov2: implement command for retrieving raw store files Gregory Szorc <gregory.szorc@gmail.com> parents: 40178 diff changeset	1408
ed55a0077490 wireprotov2: implement command for retrieving raw store files Gregory Szorc <gregory.szorc@gmail.com> parents: 40178 diff changeset	1409 with repo.lock():
ed55a0077490 wireprotov2: implement command for retrieving raw store files Gregory Szorc <gregory.szorc@gmail.com> parents: 40178 diff changeset	1410 topfiles = list(repo.store.topfiles())
ed55a0077490 wireprotov2: implement command for retrieving raw store files Gregory Szorc <gregory.szorc@gmail.com> parents: 40178 diff changeset	1411
ed55a0077490 wireprotov2: implement command for retrieving raw store files Gregory Szorc <gregory.szorc@gmail.com> parents: 40178 diff changeset	1412 sendfiles = []
ed55a0077490 wireprotov2: implement command for retrieving raw store files Gregory Szorc <gregory.szorc@gmail.com> parents: 40178 diff changeset	1413 totalsize = 0
ed55a0077490 wireprotov2: implement command for retrieving raw store files Gregory Szorc <gregory.szorc@gmail.com> parents: 40178 diff changeset	1414
ed55a0077490 wireprotov2: implement command for retrieving raw store files Gregory Szorc <gregory.szorc@gmail.com> parents: 40178 diff changeset	1415 # TODO this is a bunch of storage layer interface abstractions because
ed55a0077490 wireprotov2: implement command for retrieving raw store files Gregory Szorc <gregory.szorc@gmail.com> parents: 40178 diff changeset	1416 # it assumes revlogs.
ed55a0077490 wireprotov2: implement command for retrieving raw store files Gregory Szorc <gregory.szorc@gmail.com> parents: 40178 diff changeset	1417 for name, encodedname, size in topfiles:
ed55a0077490 wireprotov2: implement command for retrieving raw store files Gregory Szorc <gregory.szorc@gmail.com> parents: 40178 diff changeset	1418 if b'changelog' in files and name.startswith(b'00changelog'):
ed55a0077490 wireprotov2: implement command for retrieving raw store files Gregory Szorc <gregory.szorc@gmail.com> parents: 40178 diff changeset	1419 pass
ed55a0077490 wireprotov2: implement command for retrieving raw store files Gregory Szorc <gregory.szorc@gmail.com> parents: 40178 diff changeset	1420 elif b'manifestlog' in files and name.startswith(b'00manifest'):
ed55a0077490 wireprotov2: implement command for retrieving raw store files Gregory Szorc <gregory.szorc@gmail.com> parents: 40178 diff changeset	1421 pass
ed55a0077490 wireprotov2: implement command for retrieving raw store files Gregory Szorc <gregory.szorc@gmail.com> parents: 40178 diff changeset	1422 else:
ed55a0077490 wireprotov2: implement command for retrieving raw store files Gregory Szorc <gregory.szorc@gmail.com> parents: 40178 diff changeset	1423 continue
ed55a0077490 wireprotov2: implement command for retrieving raw store files Gregory Szorc <gregory.szorc@gmail.com> parents: 40178 diff changeset	1424
ed55a0077490 wireprotov2: implement command for retrieving raw store files Gregory Szorc <gregory.szorc@gmail.com> parents: 40178 diff changeset	1425 sendfiles.append((b'store', name, size))
ed55a0077490 wireprotov2: implement command for retrieving raw store files Gregory Szorc <gregory.szorc@gmail.com> parents: 40178 diff changeset	1426 totalsize += size
ed55a0077490 wireprotov2: implement command for retrieving raw store files Gregory Szorc <gregory.szorc@gmail.com> parents: 40178 diff changeset	1427
ed55a0077490 wireprotov2: implement command for retrieving raw store files Gregory Szorc <gregory.szorc@gmail.com> parents: 40178 diff changeset	1428 yield {
ed55a0077490 wireprotov2: implement command for retrieving raw store files Gregory Szorc <gregory.szorc@gmail.com> parents: 40178 diff changeset	1429 b'filecount': len(sendfiles),
ed55a0077490 wireprotov2: implement command for retrieving raw store files Gregory Szorc <gregory.szorc@gmail.com> parents: 40178 diff changeset	1430 b'totalsize': totalsize,
ed55a0077490 wireprotov2: implement command for retrieving raw store files Gregory Szorc <gregory.szorc@gmail.com> parents: 40178 diff changeset	1431 }
ed55a0077490 wireprotov2: implement command for retrieving raw store files Gregory Szorc <gregory.szorc@gmail.com> parents: 40178 diff changeset	1432
ed55a0077490 wireprotov2: implement command for retrieving raw store files Gregory Szorc <gregory.szorc@gmail.com> parents: 40178 diff changeset	1433 for location, name, size in sendfiles:
ed55a0077490 wireprotov2: implement command for retrieving raw store files Gregory Szorc <gregory.szorc@gmail.com> parents: 40178 diff changeset	1434 yield {
ed55a0077490 wireprotov2: implement command for retrieving raw store files Gregory Szorc <gregory.szorc@gmail.com> parents: 40178 diff changeset	1435 b'location': location,
ed55a0077490 wireprotov2: implement command for retrieving raw store files Gregory Szorc <gregory.szorc@gmail.com> parents: 40178 diff changeset	1436 b'path': name,
ed55a0077490 wireprotov2: implement command for retrieving raw store files Gregory Szorc <gregory.szorc@gmail.com> parents: 40178 diff changeset	1437 b'size': size,
ed55a0077490 wireprotov2: implement command for retrieving raw store files Gregory Szorc <gregory.szorc@gmail.com> parents: 40178 diff changeset	1438 }
ed55a0077490 wireprotov2: implement command for retrieving raw store files Gregory Szorc <gregory.szorc@gmail.com> parents: 40178 diff changeset	1439
ed55a0077490 wireprotov2: implement command for retrieving raw store files Gregory Szorc <gregory.szorc@gmail.com> parents: 40178 diff changeset	1440 # We have to use a closure for this to ensure the context manager is
ed55a0077490 wireprotov2: implement command for retrieving raw store files Gregory Szorc <gregory.szorc@gmail.com> parents: 40178 diff changeset	1441 # closed only after sending the final chunk.
ed55a0077490 wireprotov2: implement command for retrieving raw store files Gregory Szorc <gregory.szorc@gmail.com> parents: 40178 diff changeset	1442 def getfiledata():
ed55a0077490 wireprotov2: implement command for retrieving raw store files Gregory Szorc <gregory.szorc@gmail.com> parents: 40178 diff changeset	1443 with repo.svfs(name, 'rb', auditpath=False) as fh:
ed55a0077490 wireprotov2: implement command for retrieving raw store files Gregory Szorc <gregory.szorc@gmail.com> parents: 40178 diff changeset	1444 for chunk in util.filechunkiter(fh, limit=size):
ed55a0077490 wireprotov2: implement command for retrieving raw store files Gregory Szorc <gregory.szorc@gmail.com> parents: 40178 diff changeset	1445 yield chunk
ed55a0077490 wireprotov2: implement command for retrieving raw store files Gregory Szorc <gregory.szorc@gmail.com> parents: 40178 diff changeset	1446
ed55a0077490 wireprotov2: implement command for retrieving raw store files Gregory Szorc <gregory.szorc@gmail.com> parents: 40178 diff changeset	1447 yield wireprototypes.indefinitebytestringresponse(
ed55a0077490 wireprotov2: implement command for retrieving raw store files Gregory Szorc <gregory.szorc@gmail.com> parents: 40178 diff changeset	1448 getfiledata())

Mercurial > public > mercurial-scm > hg

annotate mercurial/wireprotov2server.py @ 40329:ed55a0077490