Mercurial > public > mercurial-scm > hg-stable

annotate mercurial/bundle2.py @ 20844:2631204d7305

Find changesets by keywords (author, files, the commit message), revision number or hash, or revset expression.
bundle2: implement the mandatory/advisory logic for parameter Parameter starting with an upper case are mandatory, the one starting with a lower case are advisory and may be ignored.
author Pierre-Yves David <pierre-yves.david@fb.com>
date Wed, 19 Mar 2014 17:53:45 -0700
parents 0641b41b0b49
children 8a6a86c9a5b5
Ignore whitespace changes - Everywhere: Within whitespace: At end of lines:
rev   line source
20801
9c5183cb9bca bundle2: very first version of a bundle2 bundler
Pierre-Yves David <pierre-yves.david@fb.com>
parents:
diff changeset
1 # bundle2.py - generic container format to transmit arbitrary data.
9c5183cb9bca bundle2: very first version of a bundle2 bundler
Pierre-Yves David <pierre-yves.david@fb.com>
parents:
diff changeset
2 #
9c5183cb9bca bundle2: very first version of a bundle2 bundler
Pierre-Yves David <pierre-yves.david@fb.com>
parents:
diff changeset
3 # Copyright 2013 Facebook, Inc.
9c5183cb9bca bundle2: very first version of a bundle2 bundler
Pierre-Yves David <pierre-yves.david@fb.com>
parents:
diff changeset
4 #
9c5183cb9bca bundle2: very first version of a bundle2 bundler
Pierre-Yves David <pierre-yves.david@fb.com>
parents:
diff changeset
5 # This software may be used and distributed according to the terms of the
9c5183cb9bca bundle2: very first version of a bundle2 bundler
Pierre-Yves David <pierre-yves.david@fb.com>
parents:
diff changeset
6 # GNU General Public License version 2 or any later version.
9c5183cb9bca bundle2: very first version of a bundle2 bundler
Pierre-Yves David <pierre-yves.david@fb.com>
parents:
diff changeset
7 """Handling of the new bundle2 format
9c5183cb9bca bundle2: very first version of a bundle2 bundler
Pierre-Yves David <pierre-yves.david@fb.com>
parents:
diff changeset
8
9c5183cb9bca bundle2: very first version of a bundle2 bundler
Pierre-Yves David <pierre-yves.david@fb.com>
parents:
diff changeset
9 The goal of bundle2 is to act as an atomically packet to transmit a set of
9c5183cb9bca bundle2: very first version of a bundle2 bundler
Pierre-Yves David <pierre-yves.david@fb.com>
parents:
diff changeset
10 payloads in an application agnostic way. It consist in a sequence of "parts"
9c5183cb9bca bundle2: very first version of a bundle2 bundler
Pierre-Yves David <pierre-yves.david@fb.com>
parents:
diff changeset
11 that will be handed to and processed by the application layer.
9c5183cb9bca bundle2: very first version of a bundle2 bundler
Pierre-Yves David <pierre-yves.david@fb.com>
parents:
diff changeset
12
9c5183cb9bca bundle2: very first version of a bundle2 bundler
Pierre-Yves David <pierre-yves.david@fb.com>
parents:
diff changeset
13
9c5183cb9bca bundle2: very first version of a bundle2 bundler
Pierre-Yves David <pierre-yves.david@fb.com>
parents:
diff changeset
14 General format architecture
9c5183cb9bca bundle2: very first version of a bundle2 bundler
Pierre-Yves David <pierre-yves.david@fb.com>
parents:
diff changeset
15 ===========================
9c5183cb9bca bundle2: very first version of a bundle2 bundler
Pierre-Yves David <pierre-yves.david@fb.com>
parents:
diff changeset
16
9c5183cb9bca bundle2: very first version of a bundle2 bundler
Pierre-Yves David <pierre-yves.david@fb.com>
parents:
diff changeset
17 The format is architectured as follow
9c5183cb9bca bundle2: very first version of a bundle2 bundler
Pierre-Yves David <pierre-yves.david@fb.com>
parents:
diff changeset
18
9c5183cb9bca bundle2: very first version of a bundle2 bundler
Pierre-Yves David <pierre-yves.david@fb.com>
parents:
diff changeset
19 - magic string
9c5183cb9bca bundle2: very first version of a bundle2 bundler
Pierre-Yves David <pierre-yves.david@fb.com>
parents:
diff changeset
20 - stream level parameters
9c5183cb9bca bundle2: very first version of a bundle2 bundler
Pierre-Yves David <pierre-yves.david@fb.com>
parents:
diff changeset
21 - payload parts (any number)
9c5183cb9bca bundle2: very first version of a bundle2 bundler
Pierre-Yves David <pierre-yves.david@fb.com>
parents:
diff changeset
22 - end of stream marker.
9c5183cb9bca bundle2: very first version of a bundle2 bundler
Pierre-Yves David <pierre-yves.david@fb.com>
parents:
diff changeset
23
20804
db9d3991d2c6 bundle2: support bundling simple parameter
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 20803
diff changeset
24 The current implementation accept some stream level option but no part.
20801
9c5183cb9bca bundle2: very first version of a bundle2 bundler
Pierre-Yves David <pierre-yves.david@fb.com>
parents:
diff changeset
25
9c5183cb9bca bundle2: very first version of a bundle2 bundler
Pierre-Yves David <pierre-yves.david@fb.com>
parents:
diff changeset
26 Details on the Binary format
9c5183cb9bca bundle2: very first version of a bundle2 bundler
Pierre-Yves David <pierre-yves.david@fb.com>
parents:
diff changeset
27 ============================
9c5183cb9bca bundle2: very first version of a bundle2 bundler
Pierre-Yves David <pierre-yves.david@fb.com>
parents:
diff changeset
28
9c5183cb9bca bundle2: very first version of a bundle2 bundler
Pierre-Yves David <pierre-yves.david@fb.com>
parents:
diff changeset
29 All numbers are unsigned and big endian.
9c5183cb9bca bundle2: very first version of a bundle2 bundler
Pierre-Yves David <pierre-yves.david@fb.com>
parents:
diff changeset
30
9c5183cb9bca bundle2: very first version of a bundle2 bundler
Pierre-Yves David <pierre-yves.david@fb.com>
parents:
diff changeset
31 stream level parameters
9c5183cb9bca bundle2: very first version of a bundle2 bundler
Pierre-Yves David <pierre-yves.david@fb.com>
parents:
diff changeset
32 ------------------------
9c5183cb9bca bundle2: very first version of a bundle2 bundler
Pierre-Yves David <pierre-yves.david@fb.com>
parents:
diff changeset
33
9c5183cb9bca bundle2: very first version of a bundle2 bundler
Pierre-Yves David <pierre-yves.david@fb.com>
parents:
diff changeset
34 Binary format is as follow
9c5183cb9bca bundle2: very first version of a bundle2 bundler
Pierre-Yves David <pierre-yves.david@fb.com>
parents:
diff changeset
35
9c5183cb9bca bundle2: very first version of a bundle2 bundler
Pierre-Yves David <pierre-yves.david@fb.com>
parents:
diff changeset
36 :params size: (16 bits integer)
9c5183cb9bca bundle2: very first version of a bundle2 bundler
Pierre-Yves David <pierre-yves.david@fb.com>
parents:
diff changeset
37
9c5183cb9bca bundle2: very first version of a bundle2 bundler
Pierre-Yves David <pierre-yves.david@fb.com>
parents:
diff changeset
38 The total number of Bytes used by the parameters
9c5183cb9bca bundle2: very first version of a bundle2 bundler
Pierre-Yves David <pierre-yves.david@fb.com>
parents:
diff changeset
39
9c5183cb9bca bundle2: very first version of a bundle2 bundler
Pierre-Yves David <pierre-yves.david@fb.com>
parents:
diff changeset
40 :params value: arbitrary number of Bytes
9c5183cb9bca bundle2: very first version of a bundle2 bundler
Pierre-Yves David <pierre-yves.david@fb.com>
parents:
diff changeset
41
9c5183cb9bca bundle2: very first version of a bundle2 bundler
Pierre-Yves David <pierre-yves.david@fb.com>
parents:
diff changeset
42 A blob of `params size` containing the serialized version of all stream level
9c5183cb9bca bundle2: very first version of a bundle2 bundler
Pierre-Yves David <pierre-yves.david@fb.com>
parents:
diff changeset
43 parameters.
9c5183cb9bca bundle2: very first version of a bundle2 bundler
Pierre-Yves David <pierre-yves.david@fb.com>
parents:
diff changeset
44
20809
b93bb639451a bundle2: support for bundling parameter value
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 20808
diff changeset
45 The blob contains a space separated list of parameters. parameter with value
20811
9785c3f8f598 bundle2: urlquote stream parameter name and value
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 20810
diff changeset
46 are stored in the form `<name>=<value>`. Both name and value are urlquoted.
20804
db9d3991d2c6 bundle2: support bundling simple parameter
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 20803
diff changeset
47
20813
8c74b3ce5b70 bundle2: refuse empty parameter name
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 20812
diff changeset
48 Empty name are obviously forbidden.
8c74b3ce5b70 bundle2: refuse empty parameter name
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 20812
diff changeset
49
20844
2631204d7305 bundle2: implement the mandatory/advisory logic for parameter
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 20843
diff changeset
50 Name MUST start with a letter. If this first letter is lower case, the
2631204d7305 bundle2: implement the mandatory/advisory logic for parameter
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 20843
diff changeset
51 parameter is advisory and can be safefly ignored. However when the first
2631204d7305 bundle2: implement the mandatory/advisory logic for parameter
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 20843
diff changeset
52 letter is capital, the parameter is mandatory and the bundling process MUST
2631204d7305 bundle2: implement the mandatory/advisory logic for parameter
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 20843
diff changeset
53 stop if he is not able to proceed it.
20814
8532f5e1b9df bundle2: force the first char of parameter to be an letter.
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 20813
diff changeset
54
20808
4c9130c7a29f bundle2: clarify stream parameter design in the documentation
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 20805
diff changeset
55 Stream parameters use a simple textual format for two main reasons:
20804
db9d3991d2c6 bundle2: support bundling simple parameter
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 20803
diff changeset
56
20808
4c9130c7a29f bundle2: clarify stream parameter design in the documentation
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 20805
diff changeset
57 - Stream level parameters should remains simple and we want to discourage any
4c9130c7a29f bundle2: clarify stream parameter design in the documentation
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 20805
diff changeset
58 crazy usage.
4c9130c7a29f bundle2: clarify stream parameter design in the documentation
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 20805
diff changeset
59 - Textual data allow easy human inspection of a the bundle2 header in case of
4c9130c7a29f bundle2: clarify stream parameter design in the documentation
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 20805
diff changeset
60 troubles.
4c9130c7a29f bundle2: clarify stream parameter design in the documentation
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 20805
diff changeset
61
4c9130c7a29f bundle2: clarify stream parameter design in the documentation
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 20805
diff changeset
62 Any Applicative level options MUST go into a bundle2 part instead.
20801
9c5183cb9bca bundle2: very first version of a bundle2 bundler
Pierre-Yves David <pierre-yves.david@fb.com>
parents:
diff changeset
63
9c5183cb9bca bundle2: very first version of a bundle2 bundler
Pierre-Yves David <pierre-yves.david@fb.com>
parents:
diff changeset
64 Payload part
9c5183cb9bca bundle2: very first version of a bundle2 bundler
Pierre-Yves David <pierre-yves.david@fb.com>
parents:
diff changeset
65 ------------------------
9c5183cb9bca bundle2: very first version of a bundle2 bundler
Pierre-Yves David <pierre-yves.david@fb.com>
parents:
diff changeset
66
9c5183cb9bca bundle2: very first version of a bundle2 bundler
Pierre-Yves David <pierre-yves.david@fb.com>
parents:
diff changeset
67 Binary format is as follow
9c5183cb9bca bundle2: very first version of a bundle2 bundler
Pierre-Yves David <pierre-yves.david@fb.com>
parents:
diff changeset
68
9c5183cb9bca bundle2: very first version of a bundle2 bundler
Pierre-Yves David <pierre-yves.david@fb.com>
parents:
diff changeset
69 :header size: (16 bits inter)
9c5183cb9bca bundle2: very first version of a bundle2 bundler
Pierre-Yves David <pierre-yves.david@fb.com>
parents:
diff changeset
70
9c5183cb9bca bundle2: very first version of a bundle2 bundler
Pierre-Yves David <pierre-yves.david@fb.com>
parents:
diff changeset
71 The total number of Bytes used by the part headers. When the header is empty
9c5183cb9bca bundle2: very first version of a bundle2 bundler
Pierre-Yves David <pierre-yves.david@fb.com>
parents:
diff changeset
72 (size = 0) this is interpreted as the end of stream marker.
9c5183cb9bca bundle2: very first version of a bundle2 bundler
Pierre-Yves David <pierre-yves.david@fb.com>
parents:
diff changeset
73
9c5183cb9bca bundle2: very first version of a bundle2 bundler
Pierre-Yves David <pierre-yves.david@fb.com>
parents:
diff changeset
74 Currently forced to 0 in the current state of the implementation
9c5183cb9bca bundle2: very first version of a bundle2 bundler
Pierre-Yves David <pierre-yves.david@fb.com>
parents:
diff changeset
75 """
9c5183cb9bca bundle2: very first version of a bundle2 bundler
Pierre-Yves David <pierre-yves.david@fb.com>
parents:
diff changeset
76
20802
520df53ad26a bundle2: a very first version of bundle2 unbundler
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 20801
diff changeset
77 import util
20804
db9d3991d2c6 bundle2: support bundling simple parameter
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 20803
diff changeset
78 import struct
20811
9785c3f8f598 bundle2: urlquote stream parameter name and value
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 20810
diff changeset
79 import urllib
20814
8532f5e1b9df bundle2: force the first char of parameter to be an letter.
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 20813
diff changeset
80 import string
20804
db9d3991d2c6 bundle2: support bundling simple parameter
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 20803
diff changeset
81
20802
520df53ad26a bundle2: a very first version of bundle2 unbundler
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 20801
diff changeset
82 import changegroup
20803
88db3e615319 bundle2: make sure the unbundler refuse non bundle2 stream
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 20802
diff changeset
83 from i18n import _
20802
520df53ad26a bundle2: a very first version of bundle2 unbundler
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 20801
diff changeset
84
20804
db9d3991d2c6 bundle2: support bundling simple parameter
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 20803
diff changeset
85 _pack = struct.pack
db9d3991d2c6 bundle2: support bundling simple parameter
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 20803
diff changeset
86 _unpack = struct.unpack
db9d3991d2c6 bundle2: support bundling simple parameter
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 20803
diff changeset
87
20801
9c5183cb9bca bundle2: very first version of a bundle2 bundler
Pierre-Yves David <pierre-yves.david@fb.com>
parents:
diff changeset
88 _magicstring = 'HG20'
9c5183cb9bca bundle2: very first version of a bundle2 bundler
Pierre-Yves David <pierre-yves.david@fb.com>
parents:
diff changeset
89
20804
db9d3991d2c6 bundle2: support bundling simple parameter
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 20803
diff changeset
90 _fstreamparamsize = '>H'
db9d3991d2c6 bundle2: support bundling simple parameter
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 20803
diff changeset
91
20801
9c5183cb9bca bundle2: very first version of a bundle2 bundler
Pierre-Yves David <pierre-yves.david@fb.com>
parents:
diff changeset
92 class bundle20(object):
9c5183cb9bca bundle2: very first version of a bundle2 bundler
Pierre-Yves David <pierre-yves.david@fb.com>
parents:
diff changeset
93 """represent an outgoing bundle2 container
9c5183cb9bca bundle2: very first version of a bundle2 bundler
Pierre-Yves David <pierre-yves.david@fb.com>
parents:
diff changeset
94
20804
db9d3991d2c6 bundle2: support bundling simple parameter
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 20803
diff changeset
95 Use the `addparam` method to add stream level parameter. Then call
db9d3991d2c6 bundle2: support bundling simple parameter
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 20803
diff changeset
96 `getchunks` to retrieve all the binary chunks of datathat compose the
db9d3991d2c6 bundle2: support bundling simple parameter
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 20803
diff changeset
97 bundle2 container.
db9d3991d2c6 bundle2: support bundling simple parameter
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 20803
diff changeset
98
db9d3991d2c6 bundle2: support bundling simple parameter
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 20803
diff changeset
99 This object does not support payload part yet."""
20801
9c5183cb9bca bundle2: very first version of a bundle2 bundler
Pierre-Yves David <pierre-yves.david@fb.com>
parents:
diff changeset
100
20842
938718d72624 bundle2: print debug information during bundling
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 20814
diff changeset
101 def __init__(self, ui):
938718d72624 bundle2: print debug information during bundling
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 20814
diff changeset
102 self.ui = ui
20801
9c5183cb9bca bundle2: very first version of a bundle2 bundler
Pierre-Yves David <pierre-yves.david@fb.com>
parents:
diff changeset
103 self._params = []
9c5183cb9bca bundle2: very first version of a bundle2 bundler
Pierre-Yves David <pierre-yves.david@fb.com>
parents:
diff changeset
104 self._parts = []
9c5183cb9bca bundle2: very first version of a bundle2 bundler
Pierre-Yves David <pierre-yves.david@fb.com>
parents:
diff changeset
105
20804
db9d3991d2c6 bundle2: support bundling simple parameter
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 20803
diff changeset
106 def addparam(self, name, value=None):
db9d3991d2c6 bundle2: support bundling simple parameter
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 20803
diff changeset
107 """add a stream level parameter"""
20813
8c74b3ce5b70 bundle2: refuse empty parameter name
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 20812
diff changeset
108 if not name:
8c74b3ce5b70 bundle2: refuse empty parameter name
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 20812
diff changeset
109 raise ValueError('empty parameter name')
20814
8532f5e1b9df bundle2: force the first char of parameter to be an letter.
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 20813
diff changeset
110 if name[0] not in string.letters:
8532f5e1b9df bundle2: force the first char of parameter to be an letter.
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 20813
diff changeset
111 raise ValueError('non letter first character: %r' % name)
20804
db9d3991d2c6 bundle2: support bundling simple parameter
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 20803
diff changeset
112 self._params.append((name, value))
db9d3991d2c6 bundle2: support bundling simple parameter
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 20803
diff changeset
113
20801
9c5183cb9bca bundle2: very first version of a bundle2 bundler
Pierre-Yves David <pierre-yves.david@fb.com>
parents:
diff changeset
114 def getchunks(self):
20842
938718d72624 bundle2: print debug information during bundling
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 20814
diff changeset
115 self.ui.debug('start emission of %s stream\n' % _magicstring)
20801
9c5183cb9bca bundle2: very first version of a bundle2 bundler
Pierre-Yves David <pierre-yves.david@fb.com>
parents:
diff changeset
116 yield _magicstring
20804
db9d3991d2c6 bundle2: support bundling simple parameter
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 20803
diff changeset
117 param = self._paramchunk()
20842
938718d72624 bundle2: print debug information during bundling
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 20814
diff changeset
118 self.ui.debug('bundle parameter: %s\n' % param)
20804
db9d3991d2c6 bundle2: support bundling simple parameter
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 20803
diff changeset
119 yield _pack(_fstreamparamsize, len(param))
db9d3991d2c6 bundle2: support bundling simple parameter
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 20803
diff changeset
120 if param:
db9d3991d2c6 bundle2: support bundling simple parameter
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 20803
diff changeset
121 yield param
db9d3991d2c6 bundle2: support bundling simple parameter
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 20803
diff changeset
122
20801
9c5183cb9bca bundle2: very first version of a bundle2 bundler
Pierre-Yves David <pierre-yves.david@fb.com>
parents:
diff changeset
123 # no support for parts
9c5183cb9bca bundle2: very first version of a bundle2 bundler
Pierre-Yves David <pierre-yves.david@fb.com>
parents:
diff changeset
124 # to be obviously fixed soon.
9c5183cb9bca bundle2: very first version of a bundle2 bundler
Pierre-Yves David <pierre-yves.david@fb.com>
parents:
diff changeset
125 assert not self._parts
20842
938718d72624 bundle2: print debug information during bundling
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 20814
diff changeset
126 self.ui.debug('end of bundle\n')
20801
9c5183cb9bca bundle2: very first version of a bundle2 bundler
Pierre-Yves David <pierre-yves.david@fb.com>
parents:
diff changeset
127 yield '\0\0'
20802
520df53ad26a bundle2: a very first version of bundle2 unbundler
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 20801
diff changeset
128
20804
db9d3991d2c6 bundle2: support bundling simple parameter
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 20803
diff changeset
129 def _paramchunk(self):
db9d3991d2c6 bundle2: support bundling simple parameter
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 20803
diff changeset
130 """return a encoded version of all stream parameters"""
db9d3991d2c6 bundle2: support bundling simple parameter
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 20803
diff changeset
131 blocks = []
20809
b93bb639451a bundle2: support for bundling parameter value
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 20808
diff changeset
132 for par, value in self._params:
20811
9785c3f8f598 bundle2: urlquote stream parameter name and value
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 20810
diff changeset
133 par = urllib.quote(par)
20809
b93bb639451a bundle2: support for bundling parameter value
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 20808
diff changeset
134 if value is not None:
20811
9785c3f8f598 bundle2: urlquote stream parameter name and value
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 20810
diff changeset
135 value = urllib.quote(value)
20809
b93bb639451a bundle2: support for bundling parameter value
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 20808
diff changeset
136 par = '%s=%s' % (par, value)
b93bb639451a bundle2: support for bundling parameter value
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 20808
diff changeset
137 blocks.append(par)
20804
db9d3991d2c6 bundle2: support bundling simple parameter
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 20803
diff changeset
138 return ' '.join(blocks)
db9d3991d2c6 bundle2: support bundling simple parameter
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 20803
diff changeset
139
20802
520df53ad26a bundle2: a very first version of bundle2 unbundler
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 20801
diff changeset
140 class unbundle20(object):
520df53ad26a bundle2: a very first version of bundle2 unbundler
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 20801
diff changeset
141 """interpret a bundle2 stream
520df53ad26a bundle2: a very first version of bundle2 unbundler
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 20801
diff changeset
142
520df53ad26a bundle2: a very first version of bundle2 unbundler
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 20801
diff changeset
143 (this will eventually yield parts)"""
520df53ad26a bundle2: a very first version of bundle2 unbundler
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 20801
diff changeset
144
20843
0641b41b0b49 bundle2: print debug information during unbundling
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 20842
diff changeset
145 def __init__(self, ui, fp):
0641b41b0b49 bundle2: print debug information during unbundling
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 20842
diff changeset
146 self.ui = ui
20802
520df53ad26a bundle2: a very first version of bundle2 unbundler
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 20801
diff changeset
147 self._fp = fp
20803
88db3e615319 bundle2: make sure the unbundler refuse non bundle2 stream
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 20802
diff changeset
148 header = self._readexact(4)
88db3e615319 bundle2: make sure the unbundler refuse non bundle2 stream
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 20802
diff changeset
149 magic, version = header[0:2], header[2:4]
88db3e615319 bundle2: make sure the unbundler refuse non bundle2 stream
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 20802
diff changeset
150 if magic != 'HG':
88db3e615319 bundle2: make sure the unbundler refuse non bundle2 stream
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 20802
diff changeset
151 raise util.Abort(_('not a Mercurial bundle'))
88db3e615319 bundle2: make sure the unbundler refuse non bundle2 stream
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 20802
diff changeset
152 if version != '20':
88db3e615319 bundle2: make sure the unbundler refuse non bundle2 stream
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 20802
diff changeset
153 raise util.Abort(_('unknown bundle version %s') % version)
20843
0641b41b0b49 bundle2: print debug information during unbundling
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 20842
diff changeset
154 self.ui.debug('start processing of %s stream\n' % header)
20802
520df53ad26a bundle2: a very first version of bundle2 unbundler
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 20801
diff changeset
155
520df53ad26a bundle2: a very first version of bundle2 unbundler
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 20801
diff changeset
156 def _unpack(self, format):
520df53ad26a bundle2: a very first version of bundle2 unbundler
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 20801
diff changeset
157 """unpack this struct format from the stream"""
520df53ad26a bundle2: a very first version of bundle2 unbundler
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 20801
diff changeset
158 data = self._readexact(struct.calcsize(format))
520df53ad26a bundle2: a very first version of bundle2 unbundler
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 20801
diff changeset
159 return _unpack(format, data)
520df53ad26a bundle2: a very first version of bundle2 unbundler
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 20801
diff changeset
160
520df53ad26a bundle2: a very first version of bundle2 unbundler
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 20801
diff changeset
161 def _readexact(self, size):
520df53ad26a bundle2: a very first version of bundle2 unbundler
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 20801
diff changeset
162 """read exactly <size> bytes from the stream"""
520df53ad26a bundle2: a very first version of bundle2 unbundler
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 20801
diff changeset
163 return changegroup.readexactly(self._fp, size)
520df53ad26a bundle2: a very first version of bundle2 unbundler
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 20801
diff changeset
164
520df53ad26a bundle2: a very first version of bundle2 unbundler
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 20801
diff changeset
165 @util.propertycache
520df53ad26a bundle2: a very first version of bundle2 unbundler
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 20801
diff changeset
166 def params(self):
520df53ad26a bundle2: a very first version of bundle2 unbundler
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 20801
diff changeset
167 """dictionnary of stream level parameters"""
20843
0641b41b0b49 bundle2: print debug information during unbundling
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 20842
diff changeset
168 self.ui.debug('reading bundle2 stream parameters\n')
20805
c5aaeca0cfbf bundle2: support for unbundling simple parameter
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 20804
diff changeset
169 params = {}
c5aaeca0cfbf bundle2: support for unbundling simple parameter
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 20804
diff changeset
170 paramssize = self._unpack(_fstreamparamsize)[0]
c5aaeca0cfbf bundle2: support for unbundling simple parameter
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 20804
diff changeset
171 if paramssize:
c5aaeca0cfbf bundle2: support for unbundling simple parameter
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 20804
diff changeset
172 for p in self._readexact(paramssize).split(' '):
20810
47293877b54c bundle2: support for unbundling parameter value
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 20809
diff changeset
173 p = p.split('=', 1)
20812
e2f908773754 bundle2: urlunquote stream parameter name and value during unbundling
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 20811
diff changeset
174 p = [urllib.unquote(i) for i in p]
20810
47293877b54c bundle2: support for unbundling parameter value
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 20809
diff changeset
175 if len(p) < 2:
47293877b54c bundle2: support for unbundling parameter value
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 20809
diff changeset
176 p.append(None)
20844
2631204d7305 bundle2: implement the mandatory/advisory logic for parameter
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 20843
diff changeset
177 self._processparam(*p)
20810
47293877b54c bundle2: support for unbundling parameter value
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 20809
diff changeset
178 params[p[0]] = p[1]
20805
c5aaeca0cfbf bundle2: support for unbundling simple parameter
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 20804
diff changeset
179 return params
20802
520df53ad26a bundle2: a very first version of bundle2 unbundler
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 20801
diff changeset
180
20844
2631204d7305 bundle2: implement the mandatory/advisory logic for parameter
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 20843
diff changeset
181 def _processparam(self, name, value):
2631204d7305 bundle2: implement the mandatory/advisory logic for parameter
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 20843
diff changeset
182 """process a parameter, applying its effect if needed
2631204d7305 bundle2: implement the mandatory/advisory logic for parameter
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 20843
diff changeset
183
2631204d7305 bundle2: implement the mandatory/advisory logic for parameter
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 20843
diff changeset
184 Parameter starting with a lower case letter are advisory and will be
2631204d7305 bundle2: implement the mandatory/advisory logic for parameter
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 20843
diff changeset
185 ignored when unknown. Those starting with an upper case letter are
2631204d7305 bundle2: implement the mandatory/advisory logic for parameter
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 20843
diff changeset
186 mandatory and will this function will raise a KeyError when unknown.
2631204d7305 bundle2: implement the mandatory/advisory logic for parameter
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 20843
diff changeset
187
2631204d7305 bundle2: implement the mandatory/advisory logic for parameter
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 20843
diff changeset
188 Note: no option are currently supported. Any input will be either
2631204d7305 bundle2: implement the mandatory/advisory logic for parameter
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 20843
diff changeset
189 ignored or failing.
2631204d7305 bundle2: implement the mandatory/advisory logic for parameter
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 20843
diff changeset
190 """
2631204d7305 bundle2: implement the mandatory/advisory logic for parameter
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 20843
diff changeset
191 if not name:
2631204d7305 bundle2: implement the mandatory/advisory logic for parameter
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 20843
diff changeset
192 raise ValueError('empty parameter name')
2631204d7305 bundle2: implement the mandatory/advisory logic for parameter
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 20843
diff changeset
193 if name[0] not in string.letters:
2631204d7305 bundle2: implement the mandatory/advisory logic for parameter
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 20843
diff changeset
194 raise ValueError('non letter first character: %r' % name)
2631204d7305 bundle2: implement the mandatory/advisory logic for parameter
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 20843
diff changeset
195 # Some logic will be later added here to try to process the option for
2631204d7305 bundle2: implement the mandatory/advisory logic for parameter
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 20843
diff changeset
196 # a dict of known parameter.
2631204d7305 bundle2: implement the mandatory/advisory logic for parameter
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 20843
diff changeset
197 if name[0].islower():
2631204d7305 bundle2: implement the mandatory/advisory logic for parameter
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 20843
diff changeset
198 self.ui.debug("ignoring unknown parameter %r\n" % name)
2631204d7305 bundle2: implement the mandatory/advisory logic for parameter
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 20843
diff changeset
199 else:
2631204d7305 bundle2: implement the mandatory/advisory logic for parameter
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 20843
diff changeset
200 raise KeyError(name)
2631204d7305 bundle2: implement the mandatory/advisory logic for parameter
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 20843
diff changeset
201
2631204d7305 bundle2: implement the mandatory/advisory logic for parameter
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 20843
diff changeset
202
20802
520df53ad26a bundle2: a very first version of bundle2 unbundler
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 20801
diff changeset
203 def __iter__(self):
520df53ad26a bundle2: a very first version of bundle2 unbundler
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 20801
diff changeset
204 """yield all parts contained in the stream"""
520df53ad26a bundle2: a very first version of bundle2 unbundler
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 20801
diff changeset
205 # make sure param have been loaded
520df53ad26a bundle2: a very first version of bundle2 unbundler
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 20801
diff changeset
206 self.params
20843
0641b41b0b49 bundle2: print debug information during unbundling
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 20842
diff changeset
207 self.ui.debug('start extraction of bundle2 parts\n')
20802
520df53ad26a bundle2: a very first version of bundle2 unbundler
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 20801
diff changeset
208 part = self._readpart()
520df53ad26a bundle2: a very first version of bundle2 unbundler
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 20801
diff changeset
209 while part is not None:
520df53ad26a bundle2: a very first version of bundle2 unbundler
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 20801
diff changeset
210 yield part
520df53ad26a bundle2: a very first version of bundle2 unbundler
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 20801
diff changeset
211 part = self._readpart()
20843
0641b41b0b49 bundle2: print debug information during unbundling
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 20842
diff changeset
212 self.ui.debug('end of bundle2 stream\n')
20802
520df53ad26a bundle2: a very first version of bundle2 unbundler
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 20801
diff changeset
213
520df53ad26a bundle2: a very first version of bundle2 unbundler
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 20801
diff changeset
214 def _readpart(self):
520df53ad26a bundle2: a very first version of bundle2 unbundler
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 20801
diff changeset
215 """return None when an end of stream markers is reach"""
520df53ad26a bundle2: a very first version of bundle2 unbundler
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 20801
diff changeset
216 headersize = self._readexact(2)
520df53ad26a bundle2: a very first version of bundle2 unbundler
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 20801
diff changeset
217 assert headersize == '\0\0'
520df53ad26a bundle2: a very first version of bundle2 unbundler
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 20801
diff changeset
218 return None
520df53ad26a bundle2: a very first version of bundle2 unbundler
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 20801
diff changeset
219
520df53ad26a bundle2: a very first version of bundle2 unbundler
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 20801
diff changeset
220
520df53ad26a bundle2: a very first version of bundle2 unbundler
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 20801
diff changeset
221