Mercurial > public > mercurial-scm > hg-stable

annotate mercurial/bundle2.py @ 35274:1f30cbac34b6

Find changesets by keywords (author, files, the commit message), revision number or hash, or revset expression.
bundle2: add a 'modes' dictionary to the bundle operation This new attribute allows the codes requesting an unbundling to pass important information to individual part handlers. The current target use case is to allow for receiving 'bookmarks' part without directly updating local repository, but just recording the received data instead. This is necessary for pull where the remote bookmarks are processed locally. I expect the concept to be beneficial to other parts in the future. To clarify the bookmark behavior on pull, the remote bookmark value are not just taken -as-is- into the local repository. There is an extra step to detect bookmark divergence. The remote bookmarks data are stored until this processing happens.
author Boris Feld <boris.feld@octobus.net>
date Tue, 17 Oct 2017 15:39:34 +0200
parents f392066d127c
children 496154e41968
Ignore whitespace changes - Everywhere: Within whitespace: At end of lines:
rev   line source
20801
9c5183cb9bca bundle2: very first version of a bundle2 bundler
Pierre-Yves David <pierre-yves.david@fb.com>
parents:
diff changeset
1 # bundle2.py - generic container format to transmit arbitrary data.
9c5183cb9bca bundle2: very first version of a bundle2 bundler
Pierre-Yves David <pierre-yves.david@fb.com>
parents:
diff changeset
2 #
9c5183cb9bca bundle2: very first version of a bundle2 bundler
Pierre-Yves David <pierre-yves.david@fb.com>
parents:
diff changeset
3 # Copyright 2013 Facebook, Inc.
9c5183cb9bca bundle2: very first version of a bundle2 bundler
Pierre-Yves David <pierre-yves.david@fb.com>
parents:
diff changeset
4 #
9c5183cb9bca bundle2: very first version of a bundle2 bundler
Pierre-Yves David <pierre-yves.david@fb.com>
parents:
diff changeset
5 # This software may be used and distributed according to the terms of the
9c5183cb9bca bundle2: very first version of a bundle2 bundler
Pierre-Yves David <pierre-yves.david@fb.com>
parents:
diff changeset
6 # GNU General Public License version 2 or any later version.
9c5183cb9bca bundle2: very first version of a bundle2 bundler
Pierre-Yves David <pierre-yves.david@fb.com>
parents:
diff changeset
7 """Handling of the new bundle2 format
9c5183cb9bca bundle2: very first version of a bundle2 bundler
Pierre-Yves David <pierre-yves.david@fb.com>
parents:
diff changeset
8
9c5183cb9bca bundle2: very first version of a bundle2 bundler
Pierre-Yves David <pierre-yves.david@fb.com>
parents:
diff changeset
9 The goal of bundle2 is to act as an atomically packet to transmit a set of
9c5183cb9bca bundle2: very first version of a bundle2 bundler
Pierre-Yves David <pierre-yves.david@fb.com>
parents:
diff changeset
10 payloads in an application agnostic way. It consist in a sequence of "parts"
9c5183cb9bca bundle2: very first version of a bundle2 bundler
Pierre-Yves David <pierre-yves.david@fb.com>
parents:
diff changeset
11 that will be handed to and processed by the application layer.
9c5183cb9bca bundle2: very first version of a bundle2 bundler
Pierre-Yves David <pierre-yves.david@fb.com>
parents:
diff changeset
12
9c5183cb9bca bundle2: very first version of a bundle2 bundler
Pierre-Yves David <pierre-yves.david@fb.com>
parents:
diff changeset
13
9c5183cb9bca bundle2: very first version of a bundle2 bundler
Pierre-Yves David <pierre-yves.david@fb.com>
parents:
diff changeset
14 General format architecture
9c5183cb9bca bundle2: very first version of a bundle2 bundler
Pierre-Yves David <pierre-yves.david@fb.com>
parents:
diff changeset
15 ===========================
9c5183cb9bca bundle2: very first version of a bundle2 bundler
Pierre-Yves David <pierre-yves.david@fb.com>
parents:
diff changeset
16
9c5183cb9bca bundle2: very first version of a bundle2 bundler
Pierre-Yves David <pierre-yves.david@fb.com>
parents:
diff changeset
17 The format is architectured as follow
9c5183cb9bca bundle2: very first version of a bundle2 bundler
Pierre-Yves David <pierre-yves.david@fb.com>
parents:
diff changeset
18
9c5183cb9bca bundle2: very first version of a bundle2 bundler
Pierre-Yves David <pierre-yves.david@fb.com>
parents:
diff changeset
19 - magic string
9c5183cb9bca bundle2: very first version of a bundle2 bundler
Pierre-Yves David <pierre-yves.david@fb.com>
parents:
diff changeset
20 - stream level parameters
9c5183cb9bca bundle2: very first version of a bundle2 bundler
Pierre-Yves David <pierre-yves.david@fb.com>
parents:
diff changeset
21 - payload parts (any number)
9c5183cb9bca bundle2: very first version of a bundle2 bundler
Pierre-Yves David <pierre-yves.david@fb.com>
parents:
diff changeset
22 - end of stream marker.
9c5183cb9bca bundle2: very first version of a bundle2 bundler
Pierre-Yves David <pierre-yves.david@fb.com>
parents:
diff changeset
23
20856
8a6a86c9a5b5 bundle2: support bundling of empty part (with a type)
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 20844
diff changeset
24 the Binary format
20801
9c5183cb9bca bundle2: very first version of a bundle2 bundler
Pierre-Yves David <pierre-yves.david@fb.com>
parents:
diff changeset
25 ============================
9c5183cb9bca bundle2: very first version of a bundle2 bundler
Pierre-Yves David <pierre-yves.david@fb.com>
parents:
diff changeset
26
21024
7731a2281cf0 spelling: fixes from spell checker
Mads Kiilerich <madski@unity3d.com>
parents: 21020
diff changeset
27 All numbers are unsigned and big-endian.
20801
9c5183cb9bca bundle2: very first version of a bundle2 bundler
Pierre-Yves David <pierre-yves.david@fb.com>
parents:
diff changeset
28
9c5183cb9bca bundle2: very first version of a bundle2 bundler
Pierre-Yves David <pierre-yves.david@fb.com>
parents:
diff changeset
29 stream level parameters
9c5183cb9bca bundle2: very first version of a bundle2 bundler
Pierre-Yves David <pierre-yves.david@fb.com>
parents:
diff changeset
30 ------------------------
9c5183cb9bca bundle2: very first version of a bundle2 bundler
Pierre-Yves David <pierre-yves.david@fb.com>
parents:
diff changeset
31
9c5183cb9bca bundle2: very first version of a bundle2 bundler
Pierre-Yves David <pierre-yves.david@fb.com>
parents:
diff changeset
32 Binary format is as follow
9c5183cb9bca bundle2: very first version of a bundle2 bundler
Pierre-Yves David <pierre-yves.david@fb.com>
parents:
diff changeset
33
23009
90f86ad3d4ff bundle2: change header size and make them signed (new format)
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 23008
diff changeset
34 :params size: int32
20801
9c5183cb9bca bundle2: very first version of a bundle2 bundler
Pierre-Yves David <pierre-yves.david@fb.com>
parents:
diff changeset
35
9c5183cb9bca bundle2: very first version of a bundle2 bundler
Pierre-Yves David <pierre-yves.david@fb.com>
parents:
diff changeset
36 The total number of Bytes used by the parameters
9c5183cb9bca bundle2: very first version of a bundle2 bundler
Pierre-Yves David <pierre-yves.david@fb.com>
parents:
diff changeset
37
9c5183cb9bca bundle2: very first version of a bundle2 bundler
Pierre-Yves David <pierre-yves.david@fb.com>
parents:
diff changeset
38 :params value: arbitrary number of Bytes
9c5183cb9bca bundle2: very first version of a bundle2 bundler
Pierre-Yves David <pierre-yves.david@fb.com>
parents:
diff changeset
39
9c5183cb9bca bundle2: very first version of a bundle2 bundler
Pierre-Yves David <pierre-yves.david@fb.com>
parents:
diff changeset
40 A blob of `params size` containing the serialized version of all stream level
9c5183cb9bca bundle2: very first version of a bundle2 bundler
Pierre-Yves David <pierre-yves.david@fb.com>
parents:
diff changeset
41 parameters.
9c5183cb9bca bundle2: very first version of a bundle2 bundler
Pierre-Yves David <pierre-yves.david@fb.com>
parents:
diff changeset
42
21024
7731a2281cf0 spelling: fixes from spell checker
Mads Kiilerich <madski@unity3d.com>
parents: 21020
diff changeset
43 The blob contains a space separated list of parameters. Parameters with value
20811
9785c3f8f598 bundle2: urlquote stream parameter name and value
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 20810
diff changeset
44 are stored in the form `<name>=<value>`. Both name and value are urlquoted.
20804
db9d3991d2c6 bundle2: support bundling simple parameter
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 20803
diff changeset
45
20813
8c74b3ce5b70 bundle2: refuse empty parameter name
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 20812
diff changeset
46 Empty name are obviously forbidden.
8c74b3ce5b70 bundle2: refuse empty parameter name
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 20812
diff changeset
47
20844
2631204d7305 bundle2: implement the mandatory/advisory logic for parameter
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 20843
diff changeset
48 Name MUST start with a letter. If this first letter is lower case, the
21024
7731a2281cf0 spelling: fixes from spell checker
Mads Kiilerich <madski@unity3d.com>
parents: 21020
diff changeset
49 parameter is advisory and can be safely ignored. However when the first
20844
2631204d7305 bundle2: implement the mandatory/advisory logic for parameter
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 20843
diff changeset
50 letter is capital, the parameter is mandatory and the bundling process MUST
2631204d7305 bundle2: implement the mandatory/advisory logic for parameter
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 20843
diff changeset
51 stop if he is not able to proceed it.
20814
8532f5e1b9df bundle2: force the first char of parameter to be an letter.
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 20813
diff changeset
52
20808
4c9130c7a29f bundle2: clarify stream parameter design in the documentation
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 20805
diff changeset
53 Stream parameters use a simple textual format for two main reasons:
20804
db9d3991d2c6 bundle2: support bundling simple parameter
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 20803
diff changeset
54
21024
7731a2281cf0 spelling: fixes from spell checker
Mads Kiilerich <madski@unity3d.com>
parents: 21020
diff changeset
55 - Stream level parameters should remain simple and we want to discourage any
20808
4c9130c7a29f bundle2: clarify stream parameter design in the documentation
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 20805
diff changeset
56 crazy usage.
21024
7731a2281cf0 spelling: fixes from spell checker
Mads Kiilerich <madski@unity3d.com>
parents: 21020
diff changeset
57 - Textual data allow easy human inspection of a bundle2 header in case of
20808
4c9130c7a29f bundle2: clarify stream parameter design in the documentation
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 20805
diff changeset
58 troubles.
4c9130c7a29f bundle2: clarify stream parameter design in the documentation
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 20805
diff changeset
59
4c9130c7a29f bundle2: clarify stream parameter design in the documentation
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 20805
diff changeset
60 Any Applicative level options MUST go into a bundle2 part instead.
20801
9c5183cb9bca bundle2: very first version of a bundle2 bundler
Pierre-Yves David <pierre-yves.david@fb.com>
parents:
diff changeset
61
9c5183cb9bca bundle2: very first version of a bundle2 bundler
Pierre-Yves David <pierre-yves.david@fb.com>
parents:
diff changeset
62 Payload part
9c5183cb9bca bundle2: very first version of a bundle2 bundler
Pierre-Yves David <pierre-yves.david@fb.com>
parents:
diff changeset
63 ------------------------
9c5183cb9bca bundle2: very first version of a bundle2 bundler
Pierre-Yves David <pierre-yves.david@fb.com>
parents:
diff changeset
64
9c5183cb9bca bundle2: very first version of a bundle2 bundler
Pierre-Yves David <pierre-yves.david@fb.com>
parents:
diff changeset
65 Binary format is as follow
9c5183cb9bca bundle2: very first version of a bundle2 bundler
Pierre-Yves David <pierre-yves.david@fb.com>
parents:
diff changeset
66
23009
90f86ad3d4ff bundle2: change header size and make them signed (new format)
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 23008
diff changeset
67 :header size: int32
20801
9c5183cb9bca bundle2: very first version of a bundle2 bundler
Pierre-Yves David <pierre-yves.david@fb.com>
parents:
diff changeset
68
25507
5bee483705e8 bundle2: clarify in docstring that header size is for a single header
Martin von Zweigbergk <martinvonz@google.com>
parents: 25506
diff changeset
69 The total number of Bytes used by the part header. When the header is empty
20801
9c5183cb9bca bundle2: very first version of a bundle2 bundler
Pierre-Yves David <pierre-yves.david@fb.com>
parents:
diff changeset
70 (size = 0) this is interpreted as the end of stream marker.
9c5183cb9bca bundle2: very first version of a bundle2 bundler
Pierre-Yves David <pierre-yves.david@fb.com>
parents:
diff changeset
71
20856
8a6a86c9a5b5 bundle2: support bundling of empty part (with a type)
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 20844
diff changeset
72 :header:
8a6a86c9a5b5 bundle2: support bundling of empty part (with a type)
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 20844
diff changeset
73
8a6a86c9a5b5 bundle2: support bundling of empty part (with a type)
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 20844
diff changeset
74 The header defines how to interpret the part. It contains two piece of
8a6a86c9a5b5 bundle2: support bundling of empty part (with a type)
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 20844
diff changeset
75 data: the part type, and the part parameters.
8a6a86c9a5b5 bundle2: support bundling of empty part (with a type)
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 20844
diff changeset
76
8a6a86c9a5b5 bundle2: support bundling of empty part (with a type)
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 20844
diff changeset
77 The part type is used to route an application level handler, that can
8a6a86c9a5b5 bundle2: support bundling of empty part (with a type)
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 20844
diff changeset
78 interpret payload.
8a6a86c9a5b5 bundle2: support bundling of empty part (with a type)
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 20844
diff changeset
79
8a6a86c9a5b5 bundle2: support bundling of empty part (with a type)
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 20844
diff changeset
80 Part parameters are passed to the application level handler. They are
8a6a86c9a5b5 bundle2: support bundling of empty part (with a type)
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 20844
diff changeset
81 meant to convey information that will help the application level object to
8a6a86c9a5b5 bundle2: support bundling of empty part (with a type)
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 20844
diff changeset
82 interpret the part payload.
8a6a86c9a5b5 bundle2: support bundling of empty part (with a type)
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 20844
diff changeset
83
8a6a86c9a5b5 bundle2: support bundling of empty part (with a type)
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 20844
diff changeset
84 The binary format of the header is has follow
8a6a86c9a5b5 bundle2: support bundling of empty part (with a type)
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 20844
diff changeset
85
8a6a86c9a5b5 bundle2: support bundling of empty part (with a type)
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 20844
diff changeset
86 :typesize: (one byte)
20877
9e9e3a4e9261 bundle2: part params
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 20876
diff changeset
87
23916
a3f7c781786b bundle2: fix parttype enforcement
Matt Mackall <mpm@selenic.com>
parents: 23868
diff changeset
88 :parttype: alphanumerical part name (restricted to [a-zA-Z0-9_:-]*)
20877
9e9e3a4e9261 bundle2: part params
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 20876
diff changeset
89
20995
e995d104c87f bundle2: add an integer id to part
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 20952
diff changeset
90 :partid: A 32bits integer (unique in the bundle) that can be used to refer
e995d104c87f bundle2: add an integer id to part
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 20952
diff changeset
91 to this part.
e995d104c87f bundle2: add an integer id to part
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 20952
diff changeset
92
20877
9e9e3a4e9261 bundle2: part params
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 20876
diff changeset
93 :parameters:
9e9e3a4e9261 bundle2: part params
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 20876
diff changeset
94
21024
7731a2281cf0 spelling: fixes from spell checker
Mads Kiilerich <madski@unity3d.com>
parents: 21020
diff changeset
95 Part's parameter may have arbitrary content, the binary structure is::
20877
9e9e3a4e9261 bundle2: part params
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 20876
diff changeset
96
9e9e3a4e9261 bundle2: part params
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 20876
diff changeset
97 <mandatory-count><advisory-count><param-sizes><param-data>
9e9e3a4e9261 bundle2: part params
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 20876
diff changeset
98
9e9e3a4e9261 bundle2: part params
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 20876
diff changeset
99 :mandatory-count: 1 byte, number of mandatory parameters
9e9e3a4e9261 bundle2: part params
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 20876
diff changeset
100
9e9e3a4e9261 bundle2: part params
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 20876
diff changeset
101 :advisory-count: 1 byte, number of advisory parameters
9e9e3a4e9261 bundle2: part params
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 20876
diff changeset
102
9e9e3a4e9261 bundle2: part params
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 20876
diff changeset
103 :param-sizes:
9e9e3a4e9261 bundle2: part params
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 20876
diff changeset
104
9e9e3a4e9261 bundle2: part params
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 20876
diff changeset
105 N couple of bytes, where N is the total number of parameters. Each
9e9e3a4e9261 bundle2: part params
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 20876
diff changeset
106 couple contains (<size-of-key>, <size-of-value) for one parameter.
9e9e3a4e9261 bundle2: part params
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 20876
diff changeset
107
9e9e3a4e9261 bundle2: part params
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 20876
diff changeset
108 :param-data:
9e9e3a4e9261 bundle2: part params
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 20876
diff changeset
109
9e9e3a4e9261 bundle2: part params
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 20876
diff changeset
110 A blob of bytes from which each parameter key and value can be
9e9e3a4e9261 bundle2: part params
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 20876
diff changeset
111 retrieved using the list of size couples stored in the previous
9e9e3a4e9261 bundle2: part params
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 20876
diff changeset
112 field.
9e9e3a4e9261 bundle2: part params
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 20876
diff changeset
113
9e9e3a4e9261 bundle2: part params
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 20876
diff changeset
114 Mandatory parameters comes first, then the advisory ones.
20856
8a6a86c9a5b5 bundle2: support bundling of empty part (with a type)
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 20844
diff changeset
115
21607
054fa5176fa7 bundle2: forbid duplicate parameter keys
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 21606
diff changeset
116 Each parameter's key MUST be unique within the part.
054fa5176fa7 bundle2: forbid duplicate parameter keys
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 21606
diff changeset
117
20856
8a6a86c9a5b5 bundle2: support bundling of empty part (with a type)
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 20844
diff changeset
118 :payload:
8a6a86c9a5b5 bundle2: support bundling of empty part (with a type)
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 20844
diff changeset
119
20876
ddd56f3eb786 bundle2: support for bundling and unbundling payload
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 20864
diff changeset
120 payload is a series of `<chunksize><chunkdata>`.
ddd56f3eb786 bundle2: support for bundling and unbundling payload
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 20864
diff changeset
121
23009
90f86ad3d4ff bundle2: change header size and make them signed (new format)
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 23008
diff changeset
122 `chunksize` is an int32, `chunkdata` are plain bytes (as much as
20876
ddd56f3eb786 bundle2: support for bundling and unbundling payload
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 20864
diff changeset
123 `chunksize` says)` The payload part is concluded by a zero size chunk.
ddd56f3eb786 bundle2: support for bundling and unbundling payload
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 20864
diff changeset
124
ddd56f3eb786 bundle2: support for bundling and unbundling payload
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 20864
diff changeset
125 The current implementation always produces either zero or one chunk.
21024
7731a2281cf0 spelling: fixes from spell checker
Mads Kiilerich <madski@unity3d.com>
parents: 21020
diff changeset
126 This is an implementation limitation that will ultimately be lifted.
20891
1c6cd23fc221 bundle2: add some distinction between mandatory and advisory part
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 20890
diff changeset
127
23009
90f86ad3d4ff bundle2: change header size and make them signed (new format)
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 23008
diff changeset
128 `chunksize` can be negative to trigger special case processing. No such
90f86ad3d4ff bundle2: change header size and make them signed (new format)
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 23008
diff changeset
129 processing is in place yet.
90f86ad3d4ff bundle2: change header size and make them signed (new format)
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 23008
diff changeset
130
20891
1c6cd23fc221 bundle2: add some distinction between mandatory and advisory part
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 20890
diff changeset
131 Bundle processing
1c6cd23fc221 bundle2: add some distinction between mandatory and advisory part
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 20890
diff changeset
132 ============================
1c6cd23fc221 bundle2: add some distinction between mandatory and advisory part
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 20890
diff changeset
133
1c6cd23fc221 bundle2: add some distinction between mandatory and advisory part
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 20890
diff changeset
134 Each part is processed in order using a "part handler". Handler are registered
1c6cd23fc221 bundle2: add some distinction between mandatory and advisory part
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 20890
diff changeset
135 for a certain part type.
1c6cd23fc221 bundle2: add some distinction between mandatory and advisory part
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 20890
diff changeset
136
1c6cd23fc221 bundle2: add some distinction between mandatory and advisory part
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 20890
diff changeset
137 The matching of a part to its handler is case insensitive. The case of the
1c6cd23fc221 bundle2: add some distinction between mandatory and advisory part
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 20890
diff changeset
138 part type is used to know if a part is mandatory or advisory. If the Part type
1c6cd23fc221 bundle2: add some distinction between mandatory and advisory part
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 20890
diff changeset
139 contains any uppercase char it is considered mandatory. When no handler is
1c6cd23fc221 bundle2: add some distinction between mandatory and advisory part
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 20890
diff changeset
140 known for a Mandatory part, the process is aborted and an exception is raised.
20892
6fe95448596d bundle2: read the whole bundle from stream on abort
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 20891
diff changeset
141 If the part is advisory and no handler is known, the part is ignored. When the
6fe95448596d bundle2: read the whole bundle from stream on abort
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 20891
diff changeset
142 process is aborted, the full bundle is still read from the stream to keep the
6fe95448596d bundle2: read the whole bundle from stream on abort
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 20891
diff changeset
143 channel usable. But none of the part read from an abort are processed. In the
6fe95448596d bundle2: read the whole bundle from stream on abort
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 20891
diff changeset
144 future, dropping the stream may become an option for channel we do not care to
6fe95448596d bundle2: read the whole bundle from stream on abort
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 20891
diff changeset
145 preserve.
20801
9c5183cb9bca bundle2: very first version of a bundle2 bundler
Pierre-Yves David <pierre-yves.david@fb.com>
parents:
diff changeset
146 """
9c5183cb9bca bundle2: very first version of a bundle2 bundler
Pierre-Yves David <pierre-yves.david@fb.com>
parents:
diff changeset
147
33698
da7c285ec6da bundle2: use modern Python division
Augie Fackler <augie@google.com>
parents: 33693
diff changeset
148 from __future__ import absolute_import, division
25919
8221fefaea08 bundle2: use absolute_import
Gregory Szorc <gregory.szorc@gmail.com>
parents: 25660
diff changeset
149
24026
3daef83a1873 bundle2.unpackermixin: control for underlying file descriptor
Eric Sumner <ericsumner@fb.com>
parents: 23916
diff changeset
150 import errno
35069
241d9caca11e bundle2: use os.SEEK_* constants
Gregory Szorc <gregory.szorc@gmail.com>
parents: 34820
diff changeset
151 import os
25919
8221fefaea08 bundle2: use absolute_import
Gregory Szorc <gregory.szorc@gmail.com>
parents: 25660
diff changeset
152 import re
20814
8532f5e1b9df bundle2: force the first char of parameter to be an letter.
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 20813
diff changeset
153 import string
25919
8221fefaea08 bundle2: use absolute_import
Gregory Szorc <gregory.szorc@gmail.com>
parents: 25660
diff changeset
154 import struct
8221fefaea08 bundle2: use absolute_import
Gregory Szorc <gregory.szorc@gmail.com>
parents: 25660
diff changeset
155 import sys
20804
db9d3991d2c6 bundle2: support bundling simple parameter
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 20803
diff changeset
156
25919
8221fefaea08 bundle2: use absolute_import
Gregory Szorc <gregory.szorc@gmail.com>
parents: 25660
diff changeset
157 from .i18n import _
8221fefaea08 bundle2: use absolute_import
Gregory Szorc <gregory.szorc@gmail.com>
parents: 25660
diff changeset
158 from . import (
35267
dbf868623daf bookmark: add a 'check:bookmarks' bundle2 part
Boris Feld <boris.feld@octobus.net>
parents: 35141
diff changeset
159 bookmarks,
25919
8221fefaea08 bundle2: use absolute_import
Gregory Szorc <gregory.szorc@gmail.com>
parents: 25660
diff changeset
160 changegroup,
8221fefaea08 bundle2: use absolute_import
Gregory Szorc <gregory.szorc@gmail.com>
parents: 25660
diff changeset
161 error,
34820
a95067b1dca6 phase: introduce a new 'check:phases' part
Boris Feld <boris.feld@octobus.net>
parents: 34637
diff changeset
162 node as nodemod,
25919
8221fefaea08 bundle2: use absolute_import
Gregory Szorc <gregory.szorc@gmail.com>
parents: 25660
diff changeset
163 obsolete,
33043
e8c8d81eb864 bundle: add config option to include phases
Martin von Zweigbergk <martinvonz@google.com>
parents: 33042
diff changeset
164 phases,
25919
8221fefaea08 bundle2: use absolute_import
Gregory Szorc <gregory.szorc@gmail.com>
parents: 25660
diff changeset
165 pushkey,
30031
0f6d6fdd3c2a pycompat: provide 'ispy3' constant
Yuya Nishihara <yuya@tcha.org>
parents: 29941
diff changeset
166 pycompat,
25919
8221fefaea08 bundle2: use absolute_import
Gregory Szorc <gregory.szorc@gmail.com>
parents: 25660
diff changeset
167 tags,
8221fefaea08 bundle2: use absolute_import
Gregory Szorc <gregory.szorc@gmail.com>
parents: 25660
diff changeset
168 url,
8221fefaea08 bundle2: use absolute_import
Gregory Szorc <gregory.szorc@gmail.com>
parents: 25660
diff changeset
169 util,
8221fefaea08 bundle2: use absolute_import
Gregory Szorc <gregory.szorc@gmail.com>
parents: 25660
diff changeset
170 )
20802
520df53ad26a bundle2: a very first version of bundle2 unbundler
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 20801
diff changeset
171
28883
032c4c2f802a pycompat: switch to util.urlreq/util.urlerr for py3 compat
timeless <timeless@mozdev.org>
parents: 28672
diff changeset
172 urlerr = util.urlerr
032c4c2f802a pycompat: switch to util.urlreq/util.urlerr for py3 compat
timeless <timeless@mozdev.org>
parents: 28672
diff changeset
173 urlreq = util.urlreq
032c4c2f802a pycompat: switch to util.urlreq/util.urlerr for py3 compat
timeless <timeless@mozdev.org>
parents: 28672
diff changeset
174
20804
db9d3991d2c6 bundle2: support bundling simple parameter
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 20803
diff changeset
175 _pack = struct.pack
db9d3991d2c6 bundle2: support bundling simple parameter
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 20803
diff changeset
176 _unpack = struct.unpack
db9d3991d2c6 bundle2: support bundling simple parameter
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 20803
diff changeset
177
23009
90f86ad3d4ff bundle2: change header size and make them signed (new format)
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 23008
diff changeset
178 _fstreamparamsize = '>i'
90f86ad3d4ff bundle2: change header size and make them signed (new format)
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 23008
diff changeset
179 _fpartheadersize = '>i'
20856
8a6a86c9a5b5 bundle2: support bundling of empty part (with a type)
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 20844
diff changeset
180 _fparttypesize = '>B'
20995
e995d104c87f bundle2: add an integer id to part
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 20952
diff changeset
181 _fpartid = '>I'
23009
90f86ad3d4ff bundle2: change header size and make them signed (new format)
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 23008
diff changeset
182 _fpayloadsize = '>i'
20877
9e9e3a4e9261 bundle2: part params
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 20876
diff changeset
183 _fpartparamcount = '>BB'
9e9e3a4e9261 bundle2: part params
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 20876
diff changeset
184
21001
c93bb6a08fa1 bundle2: support chunk iterator as part data
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 21000
diff changeset
185 preferedchunksize = 4096
c93bb6a08fa1 bundle2: support chunk iterator as part data
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 21000
diff changeset
186
23868
405eacbecc52 bundle2: enforce parttype as alphanumerical
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 23590
diff changeset
187 _parttypeforbidden = re.compile('[^a-zA-Z0-9_:-]')
405eacbecc52 bundle2: enforce parttype as alphanumerical
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 23590
diff changeset
188
25313
8f2c362bcb58 bundle2: introduce a specific function for bundling debug message
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 25181
diff changeset
189 def outdebug(ui, message):
8f2c362bcb58 bundle2: introduce a specific function for bundling debug message
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 25181
diff changeset
190 """debug regarding output stream (bundling)"""
33165
0224820688ac config: register the 'devel.bundle2.debug' config
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 33135
diff changeset
191 if ui.configbool('devel', 'bundle2.debug'):
25336
7c60a42265fb bundle2: hide bundle2 stream debug under a config flag
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 25335
diff changeset
192 ui.debug('bundle2-output: %s\n' % message)
25313
8f2c362bcb58 bundle2: introduce a specific function for bundling debug message
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 25181
diff changeset
193
25318
64dd8f25fcf4 bundle2: introduce a specific function for debug messages while unbundling
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 25317
diff changeset
194 def indebug(ui, message):
64dd8f25fcf4 bundle2: introduce a specific function for debug messages while unbundling
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 25317
diff changeset
195 """debug on input stream (unbundling)"""
33165
0224820688ac config: register the 'devel.bundle2.debug' config
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 33135
diff changeset
196 if ui.configbool('devel', 'bundle2.debug'):
25336
7c60a42265fb bundle2: hide bundle2 stream debug under a config flag
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 25335
diff changeset
197 ui.debug('bundle2-input: %s\n' % message)
25318
64dd8f25fcf4 bundle2: introduce a specific function for debug messages while unbundling
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 25317
diff changeset
198
23868
405eacbecc52 bundle2: enforce parttype as alphanumerical
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 23590
diff changeset
199 def validateparttype(parttype):
405eacbecc52 bundle2: enforce parttype as alphanumerical
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 23590
diff changeset
200 """raise ValueError if a parttype contains invalid character"""
23916
a3f7c781786b bundle2: fix parttype enforcement
Matt Mackall <mpm@selenic.com>
parents: 23868
diff changeset
201 if _parttypeforbidden.search(parttype):
23868
405eacbecc52 bundle2: enforce parttype as alphanumerical
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 23590
diff changeset
202 raise ValueError(parttype)
405eacbecc52 bundle2: enforce parttype as alphanumerical
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 23590
diff changeset
203
20877
9e9e3a4e9261 bundle2: part params
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 20876
diff changeset
204 def _makefpartparamsizes(nbparams):
9e9e3a4e9261 bundle2: part params
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 20876
diff changeset
205 """return a struct format to read part parameter sizes
9e9e3a4e9261 bundle2: part params
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 20876
diff changeset
206
9e9e3a4e9261 bundle2: part params
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 20876
diff changeset
207 The number parameters is variable so we need to build that format
9e9e3a4e9261 bundle2: part params
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 20876
diff changeset
208 dynamically.
9e9e3a4e9261 bundle2: part params
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 20876
diff changeset
209 """
9e9e3a4e9261 bundle2: part params
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 20876
diff changeset
210 return '>'+('BB'*nbparams)
20804
db9d3991d2c6 bundle2: support bundling simple parameter
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 20803
diff changeset
211
20890
ec7fc110faee bundle2: introduce a `parthandler` decorator
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 20889
diff changeset
212 parthandlermapping = {}
20889
deed5edb72de bundle2: first version of a bundle processing
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 20887
diff changeset
213
21623
5b26d82e4e2a bundle2: make it possible to declare params handled by a part handler
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 21622
diff changeset
214 def parthandler(parttype, params=()):
20890
ec7fc110faee bundle2: introduce a `parthandler` decorator
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 20889
diff changeset
215 """decorator that register a function as a bundle2 part handler
ec7fc110faee bundle2: introduce a `parthandler` decorator
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 20889
diff changeset
216
ec7fc110faee bundle2: introduce a `parthandler` decorator
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 20889
diff changeset
217 eg::
ec7fc110faee bundle2: introduce a `parthandler` decorator
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 20889
diff changeset
218
21624
d61066d787c8 bundle2: declare supported parameters for all handlers
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 21623
diff changeset
219 @parthandler('myparttype', ('mandatory', 'param', 'handled'))
20890
ec7fc110faee bundle2: introduce a `parthandler` decorator
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 20889
diff changeset
220 def myparttypehandler(...):
ec7fc110faee bundle2: introduce a `parthandler` decorator
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 20889
diff changeset
221 '''process a part of type "my part".'''
ec7fc110faee bundle2: introduce a `parthandler` decorator
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 20889
diff changeset
222 ...
ec7fc110faee bundle2: introduce a `parthandler` decorator
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 20889
diff changeset
223 """
23868
405eacbecc52 bundle2: enforce parttype as alphanumerical
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 23590
diff changeset
224 validateparttype(parttype)
20890
ec7fc110faee bundle2: introduce a `parthandler` decorator
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 20889
diff changeset
225 def _decorator(func):
20891
1c6cd23fc221 bundle2: add some distinction between mandatory and advisory part
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 20890
diff changeset
226 lparttype = parttype.lower() # enforce lower case matching.
1c6cd23fc221 bundle2: add some distinction between mandatory and advisory part
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 20890
diff changeset
227 assert lparttype not in parthandlermapping
1c6cd23fc221 bundle2: add some distinction between mandatory and advisory part
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 20890
diff changeset
228 parthandlermapping[lparttype] = func
21623
5b26d82e4e2a bundle2: make it possible to declare params handled by a part handler
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 21622
diff changeset
229 func.params = frozenset(params)
20890
ec7fc110faee bundle2: introduce a `parthandler` decorator
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 20889
diff changeset
230 return func
ec7fc110faee bundle2: introduce a `parthandler` decorator
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 20889
diff changeset
231 return _decorator
20889
deed5edb72de bundle2: first version of a bundle processing
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 20887
diff changeset
232
20949
571f2903ff1e bundle2: record processing results in the bundleoperation object
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 20948
diff changeset
233 class unbundlerecords(object):
571f2903ff1e bundle2: record processing results in the bundleoperation object
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 20948
diff changeset
234 """keep record of what happens during and unbundle
571f2903ff1e bundle2: record processing results in the bundleoperation object
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 20948
diff changeset
235
571f2903ff1e bundle2: record processing results in the bundleoperation object
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 20948
diff changeset
236 New records are added using `records.add('cat', obj)`. Where 'cat' is a
21024
7731a2281cf0 spelling: fixes from spell checker
Mads Kiilerich <madski@unity3d.com>
parents: 21020
diff changeset
237 category of record and obj is an arbitrary object.
20949
571f2903ff1e bundle2: record processing results in the bundleoperation object
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 20948
diff changeset
238
571f2903ff1e bundle2: record processing results in the bundleoperation object
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 20948
diff changeset
239 `records['cat']` will return all entries of this category 'cat'.
571f2903ff1e bundle2: record processing results in the bundleoperation object
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 20948
diff changeset
240
571f2903ff1e bundle2: record processing results in the bundleoperation object
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 20948
diff changeset
241 Iterating on the object itself will yield `('category', obj)` tuples
571f2903ff1e bundle2: record processing results in the bundleoperation object
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 20948
diff changeset
242 for all entries.
571f2903ff1e bundle2: record processing results in the bundleoperation object
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 20948
diff changeset
243
571f2903ff1e bundle2: record processing results in the bundleoperation object
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 20948
diff changeset
244 All iterations happens in chronological order.
571f2903ff1e bundle2: record processing results in the bundleoperation object
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 20948
diff changeset
245 """
571f2903ff1e bundle2: record processing results in the bundleoperation object
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 20948
diff changeset
246
571f2903ff1e bundle2: record processing results in the bundleoperation object
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 20948
diff changeset
247 def __init__(self):
571f2903ff1e bundle2: record processing results in the bundleoperation object
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 20948
diff changeset
248 self._categories = {}
571f2903ff1e bundle2: record processing results in the bundleoperation object
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 20948
diff changeset
249 self._sequences = []
20996
ed3c5e18a047 bundle2: add reply awareness to unbundlerecords
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 20995
diff changeset
250 self._replies = {}
20949
571f2903ff1e bundle2: record processing results in the bundleoperation object
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 20948
diff changeset
251
20996
ed3c5e18a047 bundle2: add reply awareness to unbundlerecords
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 20995
diff changeset
252 def add(self, category, entry, inreplyto=None):
20949
571f2903ff1e bundle2: record processing results in the bundleoperation object
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 20948
diff changeset
253 """add a new record of a given category.
571f2903ff1e bundle2: record processing results in the bundleoperation object
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 20948
diff changeset
254
571f2903ff1e bundle2: record processing results in the bundleoperation object
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 20948
diff changeset
255 The entry can then be retrieved in the list returned by
571f2903ff1e bundle2: record processing results in the bundleoperation object
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 20948
diff changeset
256 self['category']."""
571f2903ff1e bundle2: record processing results in the bundleoperation object
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 20948
diff changeset
257 self._categories.setdefault(category, []).append(entry)
571f2903ff1e bundle2: record processing results in the bundleoperation object
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 20948
diff changeset
258 self._sequences.append((category, entry))
20996
ed3c5e18a047 bundle2: add reply awareness to unbundlerecords
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 20995
diff changeset
259 if inreplyto is not None:
ed3c5e18a047 bundle2: add reply awareness to unbundlerecords
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 20995
diff changeset
260 self.getreplies(inreplyto).add(category, entry)
ed3c5e18a047 bundle2: add reply awareness to unbundlerecords
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 20995
diff changeset
261
ed3c5e18a047 bundle2: add reply awareness to unbundlerecords
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 20995
diff changeset
262 def getreplies(self, partid):
23139
e53f6b72a0e4 spelling: fixes from proofreading of spell checker issues
Mads Kiilerich <madski@unity3d.com>
parents: 23067
diff changeset
263 """get the records that are replies to a specific part"""
20996
ed3c5e18a047 bundle2: add reply awareness to unbundlerecords
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 20995
diff changeset
264 return self._replies.setdefault(partid, unbundlerecords())
20949
571f2903ff1e bundle2: record processing results in the bundleoperation object
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 20948
diff changeset
265
571f2903ff1e bundle2: record processing results in the bundleoperation object
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 20948
diff changeset
266 def __getitem__(self, cat):
571f2903ff1e bundle2: record processing results in the bundleoperation object
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 20948
diff changeset
267 return tuple(self._categories.get(cat, ()))
571f2903ff1e bundle2: record processing results in the bundleoperation object
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 20948
diff changeset
268
571f2903ff1e bundle2: record processing results in the bundleoperation object
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 20948
diff changeset
269 def __iter__(self):
571f2903ff1e bundle2: record processing results in the bundleoperation object
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 20948
diff changeset
270 return iter(self._sequences)
571f2903ff1e bundle2: record processing results in the bundleoperation object
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 20948
diff changeset
271
571f2903ff1e bundle2: record processing results in the bundleoperation object
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 20948
diff changeset
272 def __len__(self):
571f2903ff1e bundle2: record processing results in the bundleoperation object
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 20948
diff changeset
273 return len(self._sequences)
571f2903ff1e bundle2: record processing results in the bundleoperation object
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 20948
diff changeset
274
571f2903ff1e bundle2: record processing results in the bundleoperation object
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 20948
diff changeset
275 def __nonzero__(self):
571f2903ff1e bundle2: record processing results in the bundleoperation object
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 20948
diff changeset
276 return bool(self._sequences)
571f2903ff1e bundle2: record processing results in the bundleoperation object
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 20948
diff changeset
277
31483
413b44003462 py3: add __bool__ to every class defining __nonzero__
Gregory Szorc <gregory.szorc@gmail.com>
parents: 30985
diff changeset
278 __bool__ = __nonzero__
413b44003462 py3: add __bool__ to every class defining __nonzero__
Gregory Szorc <gregory.szorc@gmail.com>
parents: 30985
diff changeset
279
20948
329cd74b52bd bundle2: introduce a bundleoperation object
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 20947
diff changeset
280 class bundleoperation(object):
329cd74b52bd bundle2: introduce a bundleoperation object
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 20947
diff changeset
281 """an object that represents a single bundling process
329cd74b52bd bundle2: introduce a bundleoperation object
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 20947
diff changeset
282
329cd74b52bd bundle2: introduce a bundleoperation object
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 20947
diff changeset
283 Its purpose is to carry unbundle-related objects and states.
329cd74b52bd bundle2: introduce a bundleoperation object
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 20947
diff changeset
284
329cd74b52bd bundle2: introduce a bundleoperation object
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 20947
diff changeset
285 A new object should be created at the beginning of each bundle processing.
329cd74b52bd bundle2: introduce a bundleoperation object
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 20947
diff changeset
286 The object is to be returned by the processing function.
329cd74b52bd bundle2: introduce a bundleoperation object
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 20947
diff changeset
287
329cd74b52bd bundle2: introduce a bundleoperation object
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 20947
diff changeset
288 The object has very little content now it will ultimately contain:
329cd74b52bd bundle2: introduce a bundleoperation object
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 20947
diff changeset
289 * an access to the repo the bundle is applied to,
329cd74b52bd bundle2: introduce a bundleoperation object
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 20947
diff changeset
290 * a ui object,
329cd74b52bd bundle2: introduce a bundleoperation object
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 20947
diff changeset
291 * a way to retrieve a transaction to add changes to the repo,
329cd74b52bd bundle2: introduce a bundleoperation object
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 20947
diff changeset
292 * a way to record the result of processing each part,
329cd74b52bd bundle2: introduce a bundleoperation object
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 20947
diff changeset
293 * a way to construct a bundle response when applicable.
329cd74b52bd bundle2: introduce a bundleoperation object
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 20947
diff changeset
294 """
329cd74b52bd bundle2: introduce a bundleoperation object
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 20947
diff changeset
295
24878
e530cde6d115 bundle2: disable ouput capture unless we use http (issue4613 issue4615)
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 24851
diff changeset
296 def __init__(self, repo, transactiongetter, captureoutput=True):
20948
329cd74b52bd bundle2: introduce a bundleoperation object
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 20947
diff changeset
297 self.repo = repo
329cd74b52bd bundle2: introduce a bundleoperation object
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 20947
diff changeset
298 self.ui = repo.ui
20949
571f2903ff1e bundle2: record processing results in the bundleoperation object
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 20948
diff changeset
299 self.records = unbundlerecords()
20997
d7df4b7378ae bundle2: produce a bundle2 reply
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 20996
diff changeset
300 self.reply = None
24878
e530cde6d115 bundle2: disable ouput capture unless we use http (issue4613 issue4615)
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 24851
diff changeset
301 self.captureoutput = captureoutput
33692
f3407d56a6e8 bundle2: add the capability to store hookargs on bundle operation object
Pulkit Goyal <7895pulkit@gmail.com>
parents: 33541
diff changeset
302 self.hookargs = {}
33693
5fc4ddfbe626 bundle2: load hookargs from bundleoperation into transaction when started one
Pulkit Goyal <7895pulkit@gmail.com>
parents: 33692
diff changeset
303 self._gettransaction = transactiongetter
35274
1f30cbac34b6 bundle2: add a 'modes' dictionary to the bundle operation
Boris Feld <boris.feld@octobus.net>
parents: 35270
diff changeset
304 # carries value that can modify part behavior
1f30cbac34b6 bundle2: add a 'modes' dictionary to the bundle operation
Boris Feld <boris.feld@octobus.net>
parents: 35270
diff changeset
305 self.modes = {}
33693
5fc4ddfbe626 bundle2: load hookargs from bundleoperation into transaction when started one
Pulkit Goyal <7895pulkit@gmail.com>
parents: 33692
diff changeset
306
5fc4ddfbe626 bundle2: load hookargs from bundleoperation into transaction when started one
Pulkit Goyal <7895pulkit@gmail.com>
parents: 33692
diff changeset
307 def gettransaction(self):
5fc4ddfbe626 bundle2: load hookargs from bundleoperation into transaction when started one
Pulkit Goyal <7895pulkit@gmail.com>
parents: 33692
diff changeset
308 transaction = self._gettransaction()
5fc4ddfbe626 bundle2: load hookargs from bundleoperation into transaction when started one
Pulkit Goyal <7895pulkit@gmail.com>
parents: 33692
diff changeset
309
33831
b3a41f4d837b bundle2: relax the condition to update transaction.hookargs
Yuya Nishihara <yuya@tcha.org>
parents: 33830
diff changeset
310 if self.hookargs:
33693
5fc4ddfbe626 bundle2: load hookargs from bundleoperation into transaction when started one
Pulkit Goyal <7895pulkit@gmail.com>
parents: 33692
diff changeset
311 # the ones added to the transaction supercede those added
5fc4ddfbe626 bundle2: load hookargs from bundleoperation into transaction when started one
Pulkit Goyal <7895pulkit@gmail.com>
parents: 33692
diff changeset
312 # to the operation.
5fc4ddfbe626 bundle2: load hookargs from bundleoperation into transaction when started one
Pulkit Goyal <7895pulkit@gmail.com>
parents: 33692
diff changeset
313 self.hookargs.update(transaction.hookargs)
5fc4ddfbe626 bundle2: load hookargs from bundleoperation into transaction when started one
Pulkit Goyal <7895pulkit@gmail.com>
parents: 33692
diff changeset
314 transaction.hookargs = self.hookargs
5fc4ddfbe626 bundle2: load hookargs from bundleoperation into transaction when started one
Pulkit Goyal <7895pulkit@gmail.com>
parents: 33692
diff changeset
315
33831
b3a41f4d837b bundle2: relax the condition to update transaction.hookargs
Yuya Nishihara <yuya@tcha.org>
parents: 33830
diff changeset
316 # mark the hookargs as flushed. further attempts to add to
b3a41f4d837b bundle2: relax the condition to update transaction.hookargs
Yuya Nishihara <yuya@tcha.org>
parents: 33830
diff changeset
317 # hookargs will result in an abort.
b3a41f4d837b bundle2: relax the condition to update transaction.hookargs
Yuya Nishihara <yuya@tcha.org>
parents: 33830
diff changeset
318 self.hookargs = None
33693
5fc4ddfbe626 bundle2: load hookargs from bundleoperation into transaction when started one
Pulkit Goyal <7895pulkit@gmail.com>
parents: 33692
diff changeset
319
5fc4ddfbe626 bundle2: load hookargs from bundleoperation into transaction when started one
Pulkit Goyal <7895pulkit@gmail.com>
parents: 33692
diff changeset
320 return transaction
33692
f3407d56a6e8 bundle2: add the capability to store hookargs on bundle operation object
Pulkit Goyal <7895pulkit@gmail.com>
parents: 33541
diff changeset
321
f3407d56a6e8 bundle2: add the capability to store hookargs on bundle operation object
Pulkit Goyal <7895pulkit@gmail.com>
parents: 33541
diff changeset
322 def addhookargs(self, hookargs):
33693
5fc4ddfbe626 bundle2: load hookargs from bundleoperation into transaction when started one
Pulkit Goyal <7895pulkit@gmail.com>
parents: 33692
diff changeset
323 if self.hookargs is None:
33830
1bf5c5507614 bundle2: raise ProgrammingError for invalid call of addhookargs()
Yuya Nishihara <yuya@tcha.org>
parents: 33791
diff changeset
324 raise error.ProgrammingError('attempted to add hookargs to '
1bf5c5507614 bundle2: raise ProgrammingError for invalid call of addhookargs()
Yuya Nishihara <yuya@tcha.org>
parents: 33791
diff changeset
325 'operation after transaction started')
33692
f3407d56a6e8 bundle2: add the capability to store hookargs on bundle operation object
Pulkit Goyal <7895pulkit@gmail.com>
parents: 33541
diff changeset
326 self.hookargs.update(hookargs)
20948
329cd74b52bd bundle2: introduce a bundleoperation object
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 20947
diff changeset
327
20952
b24ee5076b94 bundle2: make it possible have a global transaction for the unbundling
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 20950
diff changeset
328 class TransactionUnavailable(RuntimeError):
b24ee5076b94 bundle2: make it possible have a global transaction for the unbundling
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 20950
diff changeset
329 pass
b24ee5076b94 bundle2: make it possible have a global transaction for the unbundling
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 20950
diff changeset
330
b24ee5076b94 bundle2: make it possible have a global transaction for the unbundling
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 20950
diff changeset
331 def _notransaction():
b24ee5076b94 bundle2: make it possible have a global transaction for the unbundling
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 20950
diff changeset
332 """default method to get a transaction while processing a bundle
b24ee5076b94 bundle2: make it possible have a global transaction for the unbundling
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 20950
diff changeset
333
b24ee5076b94 bundle2: make it possible have a global transaction for the unbundling
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 20950
diff changeset
334 Raise an exception to highlight the fact that no transaction was expected
b24ee5076b94 bundle2: make it possible have a global transaction for the unbundling
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 20950
diff changeset
335 to be created"""
b24ee5076b94 bundle2: make it possible have a global transaction for the unbundling
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 20950
diff changeset
336 raise TransactionUnavailable()
b24ee5076b94 bundle2: make it possible have a global transaction for the unbundling
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 20950
diff changeset
337
33055
18c2489ac96d bundle: make applybundle() delegate v1 bundles to applybundle1()
Martin von Zweigbergk <martinvonz@google.com>
parents: 33052
diff changeset
338 def applybundle(repo, unbundler, tr, source=None, url=None, **kwargs):
26790
28a6c2d72097 bundle2: introduce an "applybundle" function
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 26698
diff changeset
339 # transform me into unbundler.apply() as soon as the freeze is lifted
33055
18c2489ac96d bundle: make applybundle() delegate v1 bundles to applybundle1()
Martin von Zweigbergk <martinvonz@google.com>
parents: 33052
diff changeset
340 if isinstance(unbundler, unbundle20):
18c2489ac96d bundle: make applybundle() delegate v1 bundles to applybundle1()
Martin von Zweigbergk <martinvonz@google.com>
parents: 33052
diff changeset
341 tr.hookargs['bundle2'] = '1'
18c2489ac96d bundle: make applybundle() delegate v1 bundles to applybundle1()
Martin von Zweigbergk <martinvonz@google.com>
parents: 33052
diff changeset
342 if source is not None and 'source' not in tr.hookargs:
18c2489ac96d bundle: make applybundle() delegate v1 bundles to applybundle1()
Martin von Zweigbergk <martinvonz@google.com>
parents: 33052
diff changeset
343 tr.hookargs['source'] = source
18c2489ac96d bundle: make applybundle() delegate v1 bundles to applybundle1()
Martin von Zweigbergk <martinvonz@google.com>
parents: 33052
diff changeset
344 if url is not None and 'url' not in tr.hookargs:
18c2489ac96d bundle: make applybundle() delegate v1 bundles to applybundle1()
Martin von Zweigbergk <martinvonz@google.com>
parents: 33052
diff changeset
345 tr.hookargs['url'] = url
18c2489ac96d bundle: make applybundle() delegate v1 bundles to applybundle1()
Martin von Zweigbergk <martinvonz@google.com>
parents: 33052
diff changeset
346 return processbundle(repo, unbundler, lambda: tr)
18c2489ac96d bundle: make applybundle() delegate v1 bundles to applybundle1()
Martin von Zweigbergk <martinvonz@google.com>
parents: 33052
diff changeset
347 else:
33056
1d2b68952950 bundle: inline applybundle1()
Martin von Zweigbergk <martinvonz@google.com>
parents: 33055
diff changeset
348 # the transactiongetter won't be used, but we might as well set it
1d2b68952950 bundle: inline applybundle1()
Martin von Zweigbergk <martinvonz@google.com>
parents: 33055
diff changeset
349 op = bundleoperation(repo, lambda: tr)
1d2b68952950 bundle: inline applybundle1()
Martin von Zweigbergk <martinvonz@google.com>
parents: 33055
diff changeset
350 _processchangegroup(op, unbundler, tr, source, url, **kwargs)
1d2b68952950 bundle: inline applybundle1()
Martin von Zweigbergk <martinvonz@google.com>
parents: 33055
diff changeset
351 return op
26790
28a6c2d72097 bundle2: introduce an "applybundle" function
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 26698
diff changeset
352
34162
e9e0e1143fc5 bundle2: move part iterator a separate class
Durham Goode <durham@fb.com>
parents: 34116
diff changeset
353 class partiterator(object):
34164
21c2df59a1da bundle2: move exception handling into part iterator
Durham Goode <durham@fb.com>
parents: 34163
diff changeset
354 def __init__(self, repo, op, unbundler):
34163
550343626bb2 bundle2: move part counter to partiterator
Durham Goode <durham@fb.com>
parents: 34162
diff changeset
355 self.repo = repo
34164
21c2df59a1da bundle2: move exception handling into part iterator
Durham Goode <durham@fb.com>
parents: 34163
diff changeset
356 self.op = op
34162
e9e0e1143fc5 bundle2: move part iterator a separate class
Durham Goode <durham@fb.com>
parents: 34116
diff changeset
357 self.unbundler = unbundler
34163
550343626bb2 bundle2: move part counter to partiterator
Durham Goode <durham@fb.com>
parents: 34162
diff changeset
358 self.iterator = None
550343626bb2 bundle2: move part counter to partiterator
Durham Goode <durham@fb.com>
parents: 34162
diff changeset
359 self.count = 0
34265
e71890f27767 bundle2: move processpart stream maintenance into part iterator
Durham Goode <durham@fb.com>
parents: 34256
diff changeset
360 self.current = None
34162
e9e0e1143fc5 bundle2: move part iterator a separate class
Durham Goode <durham@fb.com>
parents: 34116
diff changeset
361
e9e0e1143fc5 bundle2: move part iterator a separate class
Durham Goode <durham@fb.com>
parents: 34116
diff changeset
362 def __enter__(self):
34163
550343626bb2 bundle2: move part counter to partiterator
Durham Goode <durham@fb.com>
parents: 34162
diff changeset
363 def func():
550343626bb2 bundle2: move part counter to partiterator
Durham Goode <durham@fb.com>
parents: 34162
diff changeset
364 itr = enumerate(self.unbundler.iterparts())
550343626bb2 bundle2: move part counter to partiterator
Durham Goode <durham@fb.com>
parents: 34162
diff changeset
365 for count, p in itr:
550343626bb2 bundle2: move part counter to partiterator
Durham Goode <durham@fb.com>
parents: 34162
diff changeset
366 self.count = count
34265
e71890f27767 bundle2: move processpart stream maintenance into part iterator
Durham Goode <durham@fb.com>
parents: 34256
diff changeset
367 self.current = p
34163
550343626bb2 bundle2: move part counter to partiterator
Durham Goode <durham@fb.com>
parents: 34162
diff changeset
368 yield p
35135
db5038525718 bundle2: implement consume() API on unbundlepart
Gregory Szorc <gregory.szorc@gmail.com>
parents: 35134
diff changeset
369 p.consume()
34265
e71890f27767 bundle2: move processpart stream maintenance into part iterator
Durham Goode <durham@fb.com>
parents: 34256
diff changeset
370 self.current = None
34163
550343626bb2 bundle2: move part counter to partiterator
Durham Goode <durham@fb.com>
parents: 34162
diff changeset
371 self.iterator = func()
550343626bb2 bundle2: move part counter to partiterator
Durham Goode <durham@fb.com>
parents: 34162
diff changeset
372 return self.iterator
34162
e9e0e1143fc5 bundle2: move part iterator a separate class
Durham Goode <durham@fb.com>
parents: 34116
diff changeset
373
34164
21c2df59a1da bundle2: move exception handling into part iterator
Durham Goode <durham@fb.com>
parents: 34163
diff changeset
374 def __exit__(self, type, exc, tb):
34163
550343626bb2 bundle2: move part counter to partiterator
Durham Goode <durham@fb.com>
parents: 34162
diff changeset
375 if not self.iterator:
550343626bb2 bundle2: move part counter to partiterator
Durham Goode <durham@fb.com>
parents: 34162
diff changeset
376 return
550343626bb2 bundle2: move part counter to partiterator
Durham Goode <durham@fb.com>
parents: 34162
diff changeset
377
34637
5f79f5f8487a bundle2: immediate exit for ctrl+c (issue5692)
Durham Goode <durham@fb.com>
parents: 34330
diff changeset
378 # Only gracefully abort in a normal exception situation. User aborts
5f79f5f8487a bundle2: immediate exit for ctrl+c (issue5692)
Durham Goode <durham@fb.com>
parents: 34330
diff changeset
379 # like Ctrl+C throw a KeyboardInterrupt which is not a base Exception,
5f79f5f8487a bundle2: immediate exit for ctrl+c (issue5692)
Durham Goode <durham@fb.com>
parents: 34330
diff changeset
380 # and should not gracefully cleanup.
5f79f5f8487a bundle2: immediate exit for ctrl+c (issue5692)
Durham Goode <durham@fb.com>
parents: 34330
diff changeset
381 if isinstance(exc, Exception):
34164
21c2df59a1da bundle2: move exception handling into part iterator
Durham Goode <durham@fb.com>
parents: 34163
diff changeset
382 # Any exceptions seeking to the end of the bundle at this point are
21c2df59a1da bundle2: move exception handling into part iterator
Durham Goode <durham@fb.com>
parents: 34163
diff changeset
383 # almost certainly related to the underlying stream being bad.
21c2df59a1da bundle2: move exception handling into part iterator
Durham Goode <durham@fb.com>
parents: 34163
diff changeset
384 # And, chances are that the exception we're handling is related to
21c2df59a1da bundle2: move exception handling into part iterator
Durham Goode <durham@fb.com>
parents: 34163
diff changeset
385 # getting in that bad state. So, we swallow the seeking error and
21c2df59a1da bundle2: move exception handling into part iterator
Durham Goode <durham@fb.com>
parents: 34163
diff changeset
386 # re-raise the original error.
21c2df59a1da bundle2: move exception handling into part iterator
Durham Goode <durham@fb.com>
parents: 34163
diff changeset
387 seekerror = False
21c2df59a1da bundle2: move exception handling into part iterator
Durham Goode <durham@fb.com>
parents: 34163
diff changeset
388 try:
34637
5f79f5f8487a bundle2: immediate exit for ctrl+c (issue5692)
Durham Goode <durham@fb.com>
parents: 34330
diff changeset
389 if self.current:
5f79f5f8487a bundle2: immediate exit for ctrl+c (issue5692)
Durham Goode <durham@fb.com>
parents: 34330
diff changeset
390 # consume the part content to not corrupt the stream.
35135
db5038525718 bundle2: implement consume() API on unbundlepart
Gregory Szorc <gregory.szorc@gmail.com>
parents: 35134
diff changeset
391 self.current.consume()
34637
5f79f5f8487a bundle2: immediate exit for ctrl+c (issue5692)
Durham Goode <durham@fb.com>
parents: 34330
diff changeset
392
34164
21c2df59a1da bundle2: move exception handling into part iterator
Durham Goode <durham@fb.com>
parents: 34163
diff changeset
393 for part in self.iterator:
21c2df59a1da bundle2: move exception handling into part iterator
Durham Goode <durham@fb.com>
parents: 34163
diff changeset
394 # consume the bundle content
35135
db5038525718 bundle2: implement consume() API on unbundlepart
Gregory Szorc <gregory.szorc@gmail.com>
parents: 35134
diff changeset
395 part.consume()
34164
21c2df59a1da bundle2: move exception handling into part iterator
Durham Goode <durham@fb.com>
parents: 34163
diff changeset
396 except Exception:
21c2df59a1da bundle2: move exception handling into part iterator
Durham Goode <durham@fb.com>
parents: 34163
diff changeset
397 seekerror = True
21c2df59a1da bundle2: move exception handling into part iterator
Durham Goode <durham@fb.com>
parents: 34163
diff changeset
398
21c2df59a1da bundle2: move exception handling into part iterator
Durham Goode <durham@fb.com>
parents: 34163
diff changeset
399 # Small hack to let caller code distinguish exceptions from bundle2
21c2df59a1da bundle2: move exception handling into part iterator
Durham Goode <durham@fb.com>
parents: 34163
diff changeset
400 # processing from processing the old format. This is mostly needed
21c2df59a1da bundle2: move exception handling into part iterator
Durham Goode <durham@fb.com>
parents: 34163
diff changeset
401 # to handle different return codes to unbundle according to the type
21c2df59a1da bundle2: move exception handling into part iterator
Durham Goode <durham@fb.com>
parents: 34163
diff changeset
402 # of bundle. We should probably clean up or drop this return code
21c2df59a1da bundle2: move exception handling into part iterator
Durham Goode <durham@fb.com>
parents: 34163
diff changeset
403 # craziness in a future version.
21c2df59a1da bundle2: move exception handling into part iterator
Durham Goode <durham@fb.com>
parents: 34163
diff changeset
404 exc.duringunbundle2 = True
21c2df59a1da bundle2: move exception handling into part iterator
Durham Goode <durham@fb.com>
parents: 34163
diff changeset
405 salvaged = []
21c2df59a1da bundle2: move exception handling into part iterator
Durham Goode <durham@fb.com>
parents: 34163
diff changeset
406 replycaps = None
21c2df59a1da bundle2: move exception handling into part iterator
Durham Goode <durham@fb.com>
parents: 34163
diff changeset
407 if self.op.reply is not None:
21c2df59a1da bundle2: move exception handling into part iterator
Durham Goode <durham@fb.com>
parents: 34163
diff changeset
408 salvaged = self.op.reply.salvageoutput()
21c2df59a1da bundle2: move exception handling into part iterator
Durham Goode <durham@fb.com>
parents: 34163
diff changeset
409 replycaps = self.op.reply.capabilities
21c2df59a1da bundle2: move exception handling into part iterator
Durham Goode <durham@fb.com>
parents: 34163
diff changeset
410 exc._replycaps = replycaps
21c2df59a1da bundle2: move exception handling into part iterator
Durham Goode <durham@fb.com>
parents: 34163
diff changeset
411 exc._bundle2salvagedoutput = salvaged
21c2df59a1da bundle2: move exception handling into part iterator
Durham Goode <durham@fb.com>
parents: 34163
diff changeset
412
21c2df59a1da bundle2: move exception handling into part iterator
Durham Goode <durham@fb.com>
parents: 34163
diff changeset
413 # Re-raising from a variable loses the original stack. So only use
21c2df59a1da bundle2: move exception handling into part iterator
Durham Goode <durham@fb.com>
parents: 34163
diff changeset
414 # that form if we need to.
21c2df59a1da bundle2: move exception handling into part iterator
Durham Goode <durham@fb.com>
parents: 34163
diff changeset
415 if seekerror:
21c2df59a1da bundle2: move exception handling into part iterator
Durham Goode <durham@fb.com>
parents: 34163
diff changeset
416 raise exc
21c2df59a1da bundle2: move exception handling into part iterator
Durham Goode <durham@fb.com>
parents: 34163
diff changeset
417
34163
550343626bb2 bundle2: move part counter to partiterator
Durham Goode <durham@fb.com>
parents: 34162
diff changeset
418 self.repo.ui.debug('bundle2-input-bundle: %i parts total\n' %
550343626bb2 bundle2: move part counter to partiterator
Durham Goode <durham@fb.com>
parents: 34162
diff changeset
419 self.count)
34162
e9e0e1143fc5 bundle2: move part iterator a separate class
Durham Goode <durham@fb.com>
parents: 34116
diff changeset
420
24851
df0ce98c882f bundle2: also save output when error happens during part processing
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 24849
diff changeset
421 def processbundle(repo, unbundler, transactiongetter=None, op=None):
20889
deed5edb72de bundle2: first version of a bundle processing
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 20887
diff changeset
422 """This function process a bundle, apply effect to/from a repo
deed5edb72de bundle2: first version of a bundle processing
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 20887
diff changeset
423
20947
c33d7bf53812 bundle2: feed a unbundle20 to the `processbundle` function
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 20892
diff changeset
424 It iterates over each part then searches for and uses the proper handling
c33d7bf53812 bundle2: feed a unbundle20 to the `processbundle` function
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 20892
diff changeset
425 code to process the part. Parts are processed in order.
20889
deed5edb72de bundle2: first version of a bundle processing
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 20887
diff changeset
426
20891
1c6cd23fc221 bundle2: add some distinction between mandatory and advisory part
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 20890
diff changeset
427 Unknown Mandatory part will abort the process.
24851
df0ce98c882f bundle2: also save output when error happens during part processing
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 24849
diff changeset
428
df0ce98c882f bundle2: also save output when error happens during part processing
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 24849
diff changeset
429 It is temporarily possible to provide a prebuilt bundleoperation to the
df0ce98c882f bundle2: also save output when error happens during part processing
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 24849
diff changeset
430 function. This is used to ensure output is properly propagated in case of
df0ce98c882f bundle2: also save output when error happens during part processing
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 24849
diff changeset
431 an error during the unbundling. This output capturing part will likely be
df0ce98c882f bundle2: also save output when error happens during part processing
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 24849
diff changeset
432 reworked and this ability will probably go away in the process.
20889
deed5edb72de bundle2: first version of a bundle processing
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 20887
diff changeset
433 """
24851
df0ce98c882f bundle2: also save output when error happens during part processing
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 24849
diff changeset
434 if op is None:
df0ce98c882f bundle2: also save output when error happens during part processing
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 24849
diff changeset
435 if transactiongetter is None:
df0ce98c882f bundle2: also save output when error happens during part processing
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 24849
diff changeset
436 transactiongetter = _notransaction
df0ce98c882f bundle2: also save output when error happens during part processing
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 24849
diff changeset
437 op = bundleoperation(repo, transactiongetter)
20889
deed5edb72de bundle2: first version of a bundle processing
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 20887
diff changeset
438 # todo:
deed5edb72de bundle2: first version of a bundle processing
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 20887
diff changeset
439 # - replace this is a init function soon.
deed5edb72de bundle2: first version of a bundle processing
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 20887
diff changeset
440 # - exception catching
deed5edb72de bundle2: first version of a bundle processing
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 20887
diff changeset
441 unbundler.params
25331
53ca47ef645d bundle2: add generic debug output regarding processed bundle
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 25324
diff changeset
442 if repo.ui.debugflag:
53ca47ef645d bundle2: add generic debug output regarding processed bundle
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 25324
diff changeset
443 msg = ['bundle2-input-bundle:']
53ca47ef645d bundle2: add generic debug output regarding processed bundle
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 25324
diff changeset
444 if unbundler.params:
33135
126eae7dae74 bundle2: add debug info about the number of stream params
Siddharth Agarwal <sid0@fb.com>
parents: 33134
diff changeset
445 msg.append(' %i params' % len(unbundler.params))
33791
119e1c6be1ce bundle2: fix transaction availability detection
Boris Feld <boris.feld@octobus.net>
parents: 33737
diff changeset
446 if op._gettransaction is None or op._gettransaction is _notransaction:
25331
53ca47ef645d bundle2: add generic debug output regarding processed bundle
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 25324
diff changeset
447 msg.append(' no-transaction')
53ca47ef645d bundle2: add generic debug output regarding processed bundle
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 25324
diff changeset
448 else:
53ca47ef645d bundle2: add generic debug output regarding processed bundle
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 25324
diff changeset
449 msg.append(' with-transaction')
53ca47ef645d bundle2: add generic debug output regarding processed bundle
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 25324
diff changeset
450 msg.append('\n')
53ca47ef645d bundle2: add generic debug output regarding processed bundle
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 25324
diff changeset
451 repo.ui.debug(''.join(msg))
34162
e9e0e1143fc5 bundle2: move part iterator a separate class
Durham Goode <durham@fb.com>
parents: 34116
diff changeset
452
34268
f010097c885c bundle2: move part processing to a separate function
Durham Goode <durham@fb.com>
parents: 34267
diff changeset
453 processparts(repo, op, unbundler)
f010097c885c bundle2: move part processing to a separate function
Durham Goode <durham@fb.com>
parents: 34267
diff changeset
454
f010097c885c bundle2: move part processing to a separate function
Durham Goode <durham@fb.com>
parents: 34267
diff changeset
455 return op
f010097c885c bundle2: move part processing to a separate function
Durham Goode <durham@fb.com>
parents: 34267
diff changeset
456
f010097c885c bundle2: move part processing to a separate function
Durham Goode <durham@fb.com>
parents: 34267
diff changeset
457 def processparts(repo, op, unbundler):
34164
21c2df59a1da bundle2: move exception handling into part iterator
Durham Goode <durham@fb.com>
parents: 34163
diff changeset
458 with partiterator(repo, op, unbundler) as parts:
21c2df59a1da bundle2: move exception handling into part iterator
Durham Goode <durham@fb.com>
parents: 34163
diff changeset
459 for part in parts:
21c2df59a1da bundle2: move exception handling into part iterator
Durham Goode <durham@fb.com>
parents: 34163
diff changeset
460 _processpart(op, part)
25332
0cdd1c811705 bundle2: add generic debug output at the end of bundle processing
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 25331
diff changeset
461
33050
f0efd2bffe1e bundle: extract _processchangegroup() method
Martin von Zweigbergk <martinvonz@google.com>
parents: 33049
diff changeset
462 def _processchangegroup(op, cg, tr, source, url, **kwargs):
33461
bb72031f0ea8 changegroup: stop returning and recording added nodes in 'cg.apply'
Boris Feld <boris.feld@octobus.net>
parents: 33460
diff changeset
463 ret = cg.apply(op.repo, tr, source, url, **kwargs)
33050
f0efd2bffe1e bundle: extract _processchangegroup() method
Martin von Zweigbergk <martinvonz@google.com>
parents: 33049
diff changeset
464 op.records.add('changegroup', {
f0efd2bffe1e bundle: extract _processchangegroup() method
Martin von Zweigbergk <martinvonz@google.com>
parents: 33049
diff changeset
465 'return': ret,
f0efd2bffe1e bundle: extract _processchangegroup() method
Martin von Zweigbergk <martinvonz@google.com>
parents: 33049
diff changeset
466 })
f0efd2bffe1e bundle: extract _processchangegroup() method
Martin von Zweigbergk <martinvonz@google.com>
parents: 33049
diff changeset
467 return ret
f0efd2bffe1e bundle: extract _processchangegroup() method
Martin von Zweigbergk <martinvonz@google.com>
parents: 33049
diff changeset
468
34266
07e4170f02f3 bundle2: move handler validation out of processpart
Durham Goode <durham@fb.com>
parents: 34265
diff changeset
469 def _gethandler(op, part):
07e4170f02f3 bundle2: move handler validation out of processpart
Durham Goode <durham@fb.com>
parents: 34265
diff changeset
470 status = 'unknown' # used by debug output
07e4170f02f3 bundle2: move handler validation out of processpart
Durham Goode <durham@fb.com>
parents: 34265
diff changeset
471 try:
07e4170f02f3 bundle2: move handler validation out of processpart
Durham Goode <durham@fb.com>
parents: 34265
diff changeset
472 handler = parthandlermapping.get(part.type)
07e4170f02f3 bundle2: move handler validation out of processpart
Durham Goode <durham@fb.com>
parents: 34265
diff changeset
473 if handler is None:
07e4170f02f3 bundle2: move handler validation out of processpart
Durham Goode <durham@fb.com>
parents: 34265
diff changeset
474 status = 'unsupported-type'
07e4170f02f3 bundle2: move handler validation out of processpart
Durham Goode <durham@fb.com>
parents: 34265
diff changeset
475 raise error.BundleUnknownFeatureError(parttype=part.type)
34284
a48b3db0baba bundle2: stop using %r to quote part names
Augie Fackler <augie@google.com>
parents: 34268
diff changeset
476 indebug(op.ui, 'found a handler for part %s' % part.type)
34266
07e4170f02f3 bundle2: move handler validation out of processpart
Durham Goode <durham@fb.com>
parents: 34265
diff changeset
477 unknownparams = part.mandatorykeys - handler.params
07e4170f02f3 bundle2: move handler validation out of processpart
Durham Goode <durham@fb.com>
parents: 34265
diff changeset
478 if unknownparams:
07e4170f02f3 bundle2: move handler validation out of processpart
Durham Goode <durham@fb.com>
parents: 34265
diff changeset
479 unknownparams = list(unknownparams)
07e4170f02f3 bundle2: move handler validation out of processpart
Durham Goode <durham@fb.com>
parents: 34265
diff changeset
480 unknownparams.sort()
34284
a48b3db0baba bundle2: stop using %r to quote part names
Augie Fackler <augie@google.com>
parents: 34268
diff changeset
481 status = 'unsupported-params (%s)' % ', '.join(unknownparams)
34266
07e4170f02f3 bundle2: move handler validation out of processpart
Durham Goode <durham@fb.com>
parents: 34265
diff changeset
482 raise error.BundleUnknownFeatureError(parttype=part.type,
07e4170f02f3 bundle2: move handler validation out of processpart
Durham Goode <durham@fb.com>
parents: 34265
diff changeset
483 params=unknownparams)
07e4170f02f3 bundle2: move handler validation out of processpart
Durham Goode <durham@fb.com>
parents: 34265
diff changeset
484 status = 'supported'
07e4170f02f3 bundle2: move handler validation out of processpart
Durham Goode <durham@fb.com>
parents: 34265
diff changeset
485 except error.BundleUnknownFeatureError as exc:
07e4170f02f3 bundle2: move handler validation out of processpart
Durham Goode <durham@fb.com>
parents: 34265
diff changeset
486 if part.mandatory: # mandatory parts
07e4170f02f3 bundle2: move handler validation out of processpart
Durham Goode <durham@fb.com>
parents: 34265
diff changeset
487 raise
07e4170f02f3 bundle2: move handler validation out of processpart
Durham Goode <durham@fb.com>
parents: 34265
diff changeset
488 indebug(op.ui, 'ignoring unsupported advisory part %s' % exc)
07e4170f02f3 bundle2: move handler validation out of processpart
Durham Goode <durham@fb.com>
parents: 34265
diff changeset
489 return # skip to part processing
07e4170f02f3 bundle2: move handler validation out of processpart
Durham Goode <durham@fb.com>
parents: 34265
diff changeset
490 finally:
07e4170f02f3 bundle2: move handler validation out of processpart
Durham Goode <durham@fb.com>
parents: 34265
diff changeset
491 if op.ui.debugflag:
07e4170f02f3 bundle2: move handler validation out of processpart
Durham Goode <durham@fb.com>
parents: 34265
diff changeset
492 msg = ['bundle2-input-part: "%s"' % part.type]
07e4170f02f3 bundle2: move handler validation out of processpart
Durham Goode <durham@fb.com>
parents: 34265
diff changeset
493 if not part.mandatory:
07e4170f02f3 bundle2: move handler validation out of processpart
Durham Goode <durham@fb.com>
parents: 34265
diff changeset
494 msg.append(' (advisory)')
07e4170f02f3 bundle2: move handler validation out of processpart
Durham Goode <durham@fb.com>
parents: 34265
diff changeset
495 nbmp = len(part.mandatorykeys)
07e4170f02f3 bundle2: move handler validation out of processpart
Durham Goode <durham@fb.com>
parents: 34265
diff changeset
496 nbap = len(part.params) - nbmp
07e4170f02f3 bundle2: move handler validation out of processpart
Durham Goode <durham@fb.com>
parents: 34265
diff changeset
497 if nbmp or nbap:
07e4170f02f3 bundle2: move handler validation out of processpart
Durham Goode <durham@fb.com>
parents: 34265
diff changeset
498 msg.append(' (params:')
07e4170f02f3 bundle2: move handler validation out of processpart
Durham Goode <durham@fb.com>
parents: 34265
diff changeset
499 if nbmp:
07e4170f02f3 bundle2: move handler validation out of processpart
Durham Goode <durham@fb.com>
parents: 34265
diff changeset
500 msg.append(' %i mandatory' % nbmp)
07e4170f02f3 bundle2: move handler validation out of processpart
Durham Goode <durham@fb.com>
parents: 34265
diff changeset
501 if nbap:
07e4170f02f3 bundle2: move handler validation out of processpart
Durham Goode <durham@fb.com>
parents: 34265
diff changeset
502 msg.append(' %i advisory' % nbmp)
07e4170f02f3 bundle2: move handler validation out of processpart
Durham Goode <durham@fb.com>
parents: 34265
diff changeset
503 msg.append(')')
07e4170f02f3 bundle2: move handler validation out of processpart
Durham Goode <durham@fb.com>
parents: 34265
diff changeset
504 msg.append(' %s\n' % status)
07e4170f02f3 bundle2: move handler validation out of processpart
Durham Goode <durham@fb.com>
parents: 34265
diff changeset
505 op.ui.debug(''.join(msg))
07e4170f02f3 bundle2: move handler validation out of processpart
Durham Goode <durham@fb.com>
parents: 34265
diff changeset
506
07e4170f02f3 bundle2: move handler validation out of processpart
Durham Goode <durham@fb.com>
parents: 34265
diff changeset
507 return handler
07e4170f02f3 bundle2: move handler validation out of processpart
Durham Goode <durham@fb.com>
parents: 34265
diff changeset
508
23008
d3137827016a bundle2: extract processing of part into its own function
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 23001
diff changeset
509 def _processpart(op, part):
d3137827016a bundle2: extract processing of part into its own function
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 23001
diff changeset
510 """process a single part from a bundle
d3137827016a bundle2: extract processing of part into its own function
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 23001
diff changeset
511
d3137827016a bundle2: extract processing of part into its own function
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 23001
diff changeset
512 The part is guaranteed to have been fully consumed when the function exits
d3137827016a bundle2: extract processing of part into its own function
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 23001
diff changeset
513 (even if an exception is raised)."""
34267
cc7b37c90616 bundle2: remove unnecessary try finally
Durham Goode <durham@fb.com>
parents: 34266
diff changeset
514 handler = _gethandler(op, part)
cc7b37c90616 bundle2: remove unnecessary try finally
Durham Goode <durham@fb.com>
parents: 34266
diff changeset
515 if handler is None:
cc7b37c90616 bundle2: remove unnecessary try finally
Durham Goode <durham@fb.com>
parents: 34266
diff changeset
516 return
23008
d3137827016a bundle2: extract processing of part into its own function
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 23001
diff changeset
517
34267
cc7b37c90616 bundle2: remove unnecessary try finally
Durham Goode <durham@fb.com>
parents: 34266
diff changeset
518 # handler is called outside the above try block so that we don't
cc7b37c90616 bundle2: remove unnecessary try finally
Durham Goode <durham@fb.com>
parents: 34266
diff changeset
519 # risk catching KeyErrors from anything other than the
cc7b37c90616 bundle2: remove unnecessary try finally
Durham Goode <durham@fb.com>
parents: 34266
diff changeset
520 # parthandlermapping lookup (any KeyError raised by handler()
cc7b37c90616 bundle2: remove unnecessary try finally
Durham Goode <durham@fb.com>
parents: 34266
diff changeset
521 # itself represents a defect of a different variety).
cc7b37c90616 bundle2: remove unnecessary try finally
Durham Goode <durham@fb.com>
parents: 34266
diff changeset
522 output = None
cc7b37c90616 bundle2: remove unnecessary try finally
Durham Goode <durham@fb.com>
parents: 34266
diff changeset
523 if op.captureoutput and op.reply is not None:
cc7b37c90616 bundle2: remove unnecessary try finally
Durham Goode <durham@fb.com>
parents: 34266
diff changeset
524 op.ui.pushbuffer(error=True, subproc=True)
cc7b37c90616 bundle2: remove unnecessary try finally
Durham Goode <durham@fb.com>
parents: 34266
diff changeset
525 output = ''
cc7b37c90616 bundle2: remove unnecessary try finally
Durham Goode <durham@fb.com>
parents: 34266
diff changeset
526 try:
cc7b37c90616 bundle2: remove unnecessary try finally
Durham Goode <durham@fb.com>
parents: 34266
diff changeset
527 handler(op, part)
23008
d3137827016a bundle2: extract processing of part into its own function
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 23001
diff changeset
528 finally:
34267
cc7b37c90616 bundle2: remove unnecessary try finally
Durham Goode <durham@fb.com>
parents: 34266
diff changeset
529 if output is not None:
cc7b37c90616 bundle2: remove unnecessary try finally
Durham Goode <durham@fb.com>
parents: 34266
diff changeset
530 output = op.ui.popbuffer()
cc7b37c90616 bundle2: remove unnecessary try finally
Durham Goode <durham@fb.com>
parents: 34266
diff changeset
531 if output:
cc7b37c90616 bundle2: remove unnecessary try finally
Durham Goode <durham@fb.com>
parents: 34266
diff changeset
532 outpart = op.reply.newpart('output', data=output,
cc7b37c90616 bundle2: remove unnecessary try finally
Durham Goode <durham@fb.com>
parents: 34266
diff changeset
533 mandatory=False)
cc7b37c90616 bundle2: remove unnecessary try finally
Durham Goode <durham@fb.com>
parents: 34266
diff changeset
534 outpart.addparam(
cc7b37c90616 bundle2: remove unnecessary try finally
Durham Goode <durham@fb.com>
parents: 34266
diff changeset
535 'in-reply-to', pycompat.bytestr(part.id), mandatory=False)
23008
d3137827016a bundle2: extract processing of part into its own function
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 23001
diff changeset
536
21138
f469879d27ec bundle2: extract capabilities decoding
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 21137
diff changeset
537 def decodecaps(blob):
23139
e53f6b72a0e4 spelling: fixes from proofreading of spell checker issues
Mads Kiilerich <madski@unity3d.com>
parents: 23067
diff changeset
538 """decode a bundle2 caps bytes blob into a dictionary
21138
f469879d27ec bundle2: extract capabilities decoding
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 21137
diff changeset
539
f469879d27ec bundle2: extract capabilities decoding
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 21137
diff changeset
540 The blob is a list of capabilities (one per line)
f469879d27ec bundle2: extract capabilities decoding
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 21137
diff changeset
541 Capabilities may have values using a line of the form::
f469879d27ec bundle2: extract capabilities decoding
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 21137
diff changeset
542
f469879d27ec bundle2: extract capabilities decoding
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 21137
diff changeset
543 capability=value1,value2,value3
f469879d27ec bundle2: extract capabilities decoding
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 21137
diff changeset
544
f469879d27ec bundle2: extract capabilities decoding
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 21137
diff changeset
545 The values are always a list."""
f469879d27ec bundle2: extract capabilities decoding
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 21137
diff changeset
546 caps = {}
f469879d27ec bundle2: extract capabilities decoding
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 21137
diff changeset
547 for line in blob.splitlines():
f469879d27ec bundle2: extract capabilities decoding
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 21137
diff changeset
548 if not line:
f469879d27ec bundle2: extract capabilities decoding
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 21137
diff changeset
549 continue
f469879d27ec bundle2: extract capabilities decoding
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 21137
diff changeset
550 if '=' not in line:
f469879d27ec bundle2: extract capabilities decoding
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 21137
diff changeset
551 key, vals = line, ()
f469879d27ec bundle2: extract capabilities decoding
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 21137
diff changeset
552 else:
f469879d27ec bundle2: extract capabilities decoding
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 21137
diff changeset
553 key, vals = line.split('=', 1)
f469879d27ec bundle2: extract capabilities decoding
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 21137
diff changeset
554 vals = vals.split(',')
28883
032c4c2f802a pycompat: switch to util.urlreq/util.urlerr for py3 compat
timeless <timeless@mozdev.org>
parents: 28672
diff changeset
555 key = urlreq.unquote(key)
032c4c2f802a pycompat: switch to util.urlreq/util.urlerr for py3 compat
timeless <timeless@mozdev.org>
parents: 28672
diff changeset
556 vals = [urlreq.unquote(v) for v in vals]
21138
f469879d27ec bundle2: extract capabilities decoding
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 21137
diff changeset
557 caps[key] = vals
f469879d27ec bundle2: extract capabilities decoding
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 21137
diff changeset
558 return caps
f469879d27ec bundle2: extract capabilities decoding
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 21137
diff changeset
559
21139
2b8c82f7f11d bundle2: capabilities encoding
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 21138
diff changeset
560 def encodecaps(caps):
2b8c82f7f11d bundle2: capabilities encoding
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 21138
diff changeset
561 """encode a bundle2 caps dictionary into a bytes blob"""
2b8c82f7f11d bundle2: capabilities encoding
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 21138
diff changeset
562 chunks = []
2b8c82f7f11d bundle2: capabilities encoding
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 21138
diff changeset
563 for ca in sorted(caps):
2b8c82f7f11d bundle2: capabilities encoding
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 21138
diff changeset
564 vals = caps[ca]
28883
032c4c2f802a pycompat: switch to util.urlreq/util.urlerr for py3 compat
timeless <timeless@mozdev.org>
parents: 28672
diff changeset
565 ca = urlreq.quote(ca)
032c4c2f802a pycompat: switch to util.urlreq/util.urlerr for py3 compat
timeless <timeless@mozdev.org>
parents: 28672
diff changeset
566 vals = [urlreq.quote(v) for v in vals]
21139
2b8c82f7f11d bundle2: capabilities encoding
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 21138
diff changeset
567 if vals:
2b8c82f7f11d bundle2: capabilities encoding
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 21138
diff changeset
568 ca = "%s=%s" % (ca, ','.join(vals))
2b8c82f7f11d bundle2: capabilities encoding
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 21138
diff changeset
569 chunks.append(ca)
2b8c82f7f11d bundle2: capabilities encoding
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 21138
diff changeset
570 return '\n'.join(chunks)
2b8c82f7f11d bundle2: capabilities encoding
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 21138
diff changeset
571
28666
ae53ecc47414 bundle: move writebundle() from changegroup.py to bundle2.py (API)
Martin von Zweigbergk <martinvonz@google.com>
parents: 27953
diff changeset
572 bundletypes = {
30361
f81002f736d7 bundle2: use new compression engine API for compression
Gregory Szorc <gregory.szorc@gmail.com>
parents: 30177
diff changeset
573 "": ("", 'UN'), # only when using unbundle on ssh and old http servers
28666
ae53ecc47414 bundle: move writebundle() from changegroup.py to bundle2.py (API)
Martin von Zweigbergk <martinvonz@google.com>
parents: 27953
diff changeset
574 # since the unification ssh accepts a header but there
ae53ecc47414 bundle: move writebundle() from changegroup.py to bundle2.py (API)
Martin von Zweigbergk <martinvonz@google.com>
parents: 27953
diff changeset
575 # is no capability signaling it.
ae53ecc47414 bundle: move writebundle() from changegroup.py to bundle2.py (API)
Martin von Zweigbergk <martinvonz@google.com>
parents: 27953
diff changeset
576 "HG20": (), # special-cased below
30361
f81002f736d7 bundle2: use new compression engine API for compression
Gregory Szorc <gregory.szorc@gmail.com>
parents: 30177
diff changeset
577 "HG10UN": ("HG10UN", 'UN'),
28666
ae53ecc47414 bundle: move writebundle() from changegroup.py to bundle2.py (API)
Martin von Zweigbergk <martinvonz@google.com>
parents: 27953
diff changeset
578 "HG10BZ": ("HG10", 'BZ'),
ae53ecc47414 bundle: move writebundle() from changegroup.py to bundle2.py (API)
Martin von Zweigbergk <martinvonz@google.com>
parents: 27953
diff changeset
579 "HG10GZ": ("HG10GZ", 'GZ'),
ae53ecc47414 bundle: move writebundle() from changegroup.py to bundle2.py (API)
Martin von Zweigbergk <martinvonz@google.com>
parents: 27953
diff changeset
580 }
ae53ecc47414 bundle: move writebundle() from changegroup.py to bundle2.py (API)
Martin von Zweigbergk <martinvonz@google.com>
parents: 27953
diff changeset
581
ae53ecc47414 bundle: move writebundle() from changegroup.py to bundle2.py (API)
Martin von Zweigbergk <martinvonz@google.com>
parents: 27953
diff changeset
582 # hgweb uses this list to communicate its preferred type
ae53ecc47414 bundle: move writebundle() from changegroup.py to bundle2.py (API)
Martin von Zweigbergk <martinvonz@google.com>
parents: 27953
diff changeset
583 bundlepriority = ['HG10GZ', 'HG10BZ', 'HG10UN']
ae53ecc47414 bundle: move writebundle() from changegroup.py to bundle2.py (API)
Martin von Zweigbergk <martinvonz@google.com>
parents: 27953
diff changeset
584
20801
9c5183cb9bca bundle2: very first version of a bundle2 bundler
Pierre-Yves David <pierre-yves.david@fb.com>
parents:
diff changeset
585 class bundle20(object):
9c5183cb9bca bundle2: very first version of a bundle2 bundler
Pierre-Yves David <pierre-yves.david@fb.com>
parents:
diff changeset
586 """represent an outgoing bundle2 container
9c5183cb9bca bundle2: very first version of a bundle2 bundler
Pierre-Yves David <pierre-yves.david@fb.com>
parents:
diff changeset
587
21599
57cd844d7a5b bundle2: have ``newpart`` automatically add the part to the bundle
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 21598
diff changeset
588 Use the `addparam` method to add stream level parameter. and `newpart` to
20856
8a6a86c9a5b5 bundle2: support bundling of empty part (with a type)
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 20844
diff changeset
589 populate it. Then call `getchunks` to retrieve all the binary chunks of
21024
7731a2281cf0 spelling: fixes from spell checker
Mads Kiilerich <madski@unity3d.com>
parents: 21020
diff changeset
590 data that compose the bundle2 container."""
20801
9c5183cb9bca bundle2: very first version of a bundle2 bundler
Pierre-Yves David <pierre-yves.david@fb.com>
parents:
diff changeset
591
24686
e0e28e910fa3 bundle2: rename format, parts and config to final names
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 24648
diff changeset
592 _magicstring = 'HG20'
24640
685639f9430d bundle20: move magic string into the class
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 24071
diff changeset
593
21134
2f8c4fa237f5 bundle2: adds a capabilities attribute on bundler20
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 21133
diff changeset
594 def __init__(self, ui, capabilities=()):
20842
938718d72624 bundle2: print debug information during bundling
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 20814
diff changeset
595 self.ui = ui
20801
9c5183cb9bca bundle2: very first version of a bundle2 bundler
Pierre-Yves David <pierre-yves.david@fb.com>
parents:
diff changeset
596 self._params = []
9c5183cb9bca bundle2: very first version of a bundle2 bundler
Pierre-Yves David <pierre-yves.david@fb.com>
parents:
diff changeset
597 self._parts = []
21136
b6fd496e5c72 bundle2: support for capabilities with values
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 21135
diff changeset
598 self.capabilities = dict(capabilities)
30361
f81002f736d7 bundle2: use new compression engine API for compression
Gregory Szorc <gregory.szorc@gmail.com>
parents: 30177
diff changeset
599 self._compengine = util.compengines.forbundletype('UN')
30757
511a4bf52754 bundle2: allow compression options to be passed to compressor
Gregory Szorc <gregory.szorc@gmail.com>
parents: 30448
diff changeset
600 self._compopts = None
26404
795f02a24b87 bundle2: allow compressed bundle
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 26396
diff changeset
601
30757
511a4bf52754 bundle2: allow compression options to be passed to compressor
Gregory Szorc <gregory.szorc@gmail.com>
parents: 30448
diff changeset
602 def setcompression(self, alg, compopts=None):
26404
795f02a24b87 bundle2: allow compressed bundle
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 26396
diff changeset
603 """setup core part compression to <alg>"""
30448
71b368e3b590 bundle2: equate 'UN' with no compression
Gregory Szorc <gregory.szorc@gmail.com>
parents: 30367
diff changeset
604 if alg in (None, 'UN'):
26404
795f02a24b87 bundle2: allow compressed bundle
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 26396
diff changeset
605 return
30877
aa25989b0658 bundle2: fix assertion that 'compression' hasn't been set
Siddharth Agarwal <sid0@fb.com>
parents: 30757
diff changeset
606 assert not any(n.lower() == 'compression' for n, v in self._params)
26404
795f02a24b87 bundle2: allow compressed bundle
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 26396
diff changeset
607 self.addparam('Compression', alg)
30361
f81002f736d7 bundle2: use new compression engine API for compression
Gregory Szorc <gregory.szorc@gmail.com>
parents: 30177
diff changeset
608 self._compengine = util.compengines.forbundletype(alg)
30757
511a4bf52754 bundle2: allow compression options to be passed to compressor
Gregory Szorc <gregory.szorc@gmail.com>
parents: 30448
diff changeset
609 self._compopts = compopts
20801
9c5183cb9bca bundle2: very first version of a bundle2 bundler
Pierre-Yves David <pierre-yves.david@fb.com>
parents:
diff changeset
610
21900
b8bd97085ec9 bundle2: add a ``bundle20.nbparts`` property
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 21660
diff changeset
611 @property
b8bd97085ec9 bundle2: add a ``bundle20.nbparts`` property
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 21660
diff changeset
612 def nbparts(self):
b8bd97085ec9 bundle2: add a ``bundle20.nbparts`` property
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 21660
diff changeset
613 """total number of parts added to the bundler"""
b8bd97085ec9 bundle2: add a ``bundle20.nbparts`` property
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 21660
diff changeset
614 return len(self._parts)
b8bd97085ec9 bundle2: add a ``bundle20.nbparts`` property
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 21660
diff changeset
615
21597
1daad9dcdba2 bundle2: small doc update on the bundler
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 21186
diff changeset
616 # methods used to defines the bundle2 content
20804
db9d3991d2c6 bundle2: support bundling simple parameter
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 20803
diff changeset
617 def addparam(self, name, value=None):
db9d3991d2c6 bundle2: support bundling simple parameter
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 20803
diff changeset
618 """add a stream level parameter"""
20813
8c74b3ce5b70 bundle2: refuse empty parameter name
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 20812
diff changeset
619 if not name:
34286
047f2af8ceb0 bundle2: make ValueError messages native strings
Augie Fackler <augie@google.com>
parents: 34285
diff changeset
620 raise ValueError(r'empty parameter name')
34287
2bdd55abd56c bundle2: portably grab first byte of part name for letter check
Augie Fackler <augie@google.com>
parents: 34286
diff changeset
621 if name[0:1] not in pycompat.bytestr(string.ascii_letters):
34286
047f2af8ceb0 bundle2: make ValueError messages native strings
Augie Fackler <augie@google.com>
parents: 34285
diff changeset
622 raise ValueError(r'non letter first character: %s' % name)
20804
db9d3991d2c6 bundle2: support bundling simple parameter
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 20803
diff changeset
623 self._params.append((name, value))
db9d3991d2c6 bundle2: support bundling simple parameter
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 20803
diff changeset
624
20856
8a6a86c9a5b5 bundle2: support bundling of empty part (with a type)
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 20844
diff changeset
625 def addpart(self, part):
8a6a86c9a5b5 bundle2: support bundling of empty part (with a type)
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 20844
diff changeset
626 """add a new part to the bundle2 container
8a6a86c9a5b5 bundle2: support bundling of empty part (with a type)
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 20844
diff changeset
627
21024
7731a2281cf0 spelling: fixes from spell checker
Mads Kiilerich <madski@unity3d.com>
parents: 21020
diff changeset
628 Parts contains the actual applicative payload."""
20995
e995d104c87f bundle2: add an integer id to part
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 20952
diff changeset
629 assert part.id is None
e995d104c87f bundle2: add an integer id to part
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 20952
diff changeset
630 part.id = len(self._parts) # very cheap counter
20856
8a6a86c9a5b5 bundle2: support bundling of empty part (with a type)
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 20844
diff changeset
631 self._parts.append(part)
8a6a86c9a5b5 bundle2: support bundling of empty part (with a type)
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 20844
diff changeset
632
21598
1b0dbb91de5b bundle2: add a ``newpart`` method to ``bundle20``
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 21597
diff changeset
633 def newpart(self, typeid, *args, **kwargs):
21602
cc33ae50bab3 bundle2: warn about error during initialization in ``newpart`` docstring
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 21601
diff changeset
634 """create a new part and add it to the containers
cc33ae50bab3 bundle2: warn about error during initialization in ``newpart`` docstring
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 21601
diff changeset
635
cc33ae50bab3 bundle2: warn about error during initialization in ``newpart`` docstring
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 21601
diff changeset
636 As the part is directly added to the containers. For now, this means
cc33ae50bab3 bundle2: warn about error during initialization in ``newpart`` docstring
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 21601
diff changeset
637 that any failure to properly initialize the part after calling
cc33ae50bab3 bundle2: warn about error during initialization in ``newpart`` docstring
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 21601
diff changeset
638 ``newpart`` should result in a failure of the whole bundling process.
cc33ae50bab3 bundle2: warn about error during initialization in ``newpart`` docstring
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 21601
diff changeset
639
cc33ae50bab3 bundle2: warn about error during initialization in ``newpart`` docstring
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 21601
diff changeset
640 You can still fall back to manually create and add if you need better
cc33ae50bab3 bundle2: warn about error during initialization in ``newpart`` docstring
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 21601
diff changeset
641 control."""
21598
1b0dbb91de5b bundle2: add a ``newpart`` method to ``bundle20``
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 21597
diff changeset
642 part = bundlepart(typeid, *args, **kwargs)
21599
57cd844d7a5b bundle2: have ``newpart`` automatically add the part to the bundle
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 21598
diff changeset
643 self.addpart(part)
21598
1b0dbb91de5b bundle2: add a ``newpart`` method to ``bundle20``
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 21597
diff changeset
644 return part
1b0dbb91de5b bundle2: add a ``newpart`` method to ``bundle20``
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 21597
diff changeset
645
21597
1daad9dcdba2 bundle2: small doc update on the bundler
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 21186
diff changeset
646 # methods used to generate the bundle2 stream
20801
9c5183cb9bca bundle2: very first version of a bundle2 bundler
Pierre-Yves David <pierre-yves.david@fb.com>
parents:
diff changeset
647 def getchunks(self):
25322
1f7c0403f8be bundle2: add generic debug output regarding generated bundle
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 25321
diff changeset
648 if self.ui.debugflag:
1f7c0403f8be bundle2: add generic debug output regarding generated bundle
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 25321
diff changeset
649 msg = ['bundle2-output-bundle: "%s",' % self._magicstring]
1f7c0403f8be bundle2: add generic debug output regarding generated bundle
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 25321
diff changeset
650 if self._params:
1f7c0403f8be bundle2: add generic debug output regarding generated bundle
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 25321
diff changeset
651 msg.append(' (%i params)' % len(self._params))
1f7c0403f8be bundle2: add generic debug output regarding generated bundle
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 25321
diff changeset
652 msg.append(' %i parts total\n' % len(self._parts))
1f7c0403f8be bundle2: add generic debug output regarding generated bundle
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 25321
diff changeset
653 self.ui.debug(''.join(msg))
25315
e28245b0da56 bundle2: handle new line in 'outdebug' function
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 25314
diff changeset
654 outdebug(self.ui, 'start emission of %s stream' % self._magicstring)
24640
685639f9430d bundle20: move magic string into the class
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 24071
diff changeset
655 yield self._magicstring
20804
db9d3991d2c6 bundle2: support bundling simple parameter
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 20803
diff changeset
656 param = self._paramchunk()
25315
e28245b0da56 bundle2: handle new line in 'outdebug' function
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 25314
diff changeset
657 outdebug(self.ui, 'bundle parameter: %s' % param)
20804
db9d3991d2c6 bundle2: support bundling simple parameter
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 20803
diff changeset
658 yield _pack(_fstreamparamsize, len(param))
db9d3991d2c6 bundle2: support bundling simple parameter
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 20803
diff changeset
659 if param:
db9d3991d2c6 bundle2: support bundling simple parameter
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 20803
diff changeset
660 yield param
30757
511a4bf52754 bundle2: allow compression options to be passed to compressor
Gregory Szorc <gregory.szorc@gmail.com>
parents: 30448
diff changeset
661 for chunk in self._compengine.compressstream(self._getcorechunk(),
511a4bf52754 bundle2: allow compression options to be passed to compressor
Gregory Szorc <gregory.szorc@gmail.com>
parents: 30448
diff changeset
662 self._compopts):
30367
5925bda42dbd bundle2: use compressstream compression engine API
Gregory Szorc <gregory.szorc@gmail.com>
parents: 30363
diff changeset
663 yield chunk
20802
520df53ad26a bundle2: a very first version of bundle2 unbundler
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 20801
diff changeset
664
20804
db9d3991d2c6 bundle2: support bundling simple parameter
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 20803
diff changeset
665 def _paramchunk(self):
db9d3991d2c6 bundle2: support bundling simple parameter
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 20803
diff changeset
666 """return a encoded version of all stream parameters"""
db9d3991d2c6 bundle2: support bundling simple parameter
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 20803
diff changeset
667 blocks = []
20809
b93bb639451a bundle2: support for bundling parameter value
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 20808
diff changeset
668 for par, value in self._params:
28883
032c4c2f802a pycompat: switch to util.urlreq/util.urlerr for py3 compat
timeless <timeless@mozdev.org>
parents: 28672
diff changeset
669 par = urlreq.quote(par)
20809
b93bb639451a bundle2: support for bundling parameter value
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 20808
diff changeset
670 if value is not None:
28883
032c4c2f802a pycompat: switch to util.urlreq/util.urlerr for py3 compat
timeless <timeless@mozdev.org>
parents: 28672
diff changeset
671 value = urlreq.quote(value)
20809
b93bb639451a bundle2: support for bundling parameter value
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 20808
diff changeset
672 par = '%s=%s' % (par, value)
b93bb639451a bundle2: support for bundling parameter value
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 20808
diff changeset
673 blocks.append(par)
20804
db9d3991d2c6 bundle2: support bundling simple parameter
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 20803
diff changeset
674 return ' '.join(blocks)
db9d3991d2c6 bundle2: support bundling simple parameter
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 20803
diff changeset
675
26396
d90c30801cdf bundle20: extract core payload generation in its own function
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 26395
diff changeset
676 def _getcorechunk(self):
d90c30801cdf bundle20: extract core payload generation in its own function
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 26395
diff changeset
677 """yield chunk for the core part of the bundle
d90c30801cdf bundle20: extract core payload generation in its own function
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 26395
diff changeset
678
d90c30801cdf bundle20: extract core payload generation in its own function
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 26395
diff changeset
679 (all but headers and parameters)"""
d90c30801cdf bundle20: extract core payload generation in its own function
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 26395
diff changeset
680 outdebug(self.ui, 'start of parts')
d90c30801cdf bundle20: extract core payload generation in its own function
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 26395
diff changeset
681 for part in self._parts:
d90c30801cdf bundle20: extract core payload generation in its own function
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 26395
diff changeset
682 outdebug(self.ui, 'bundle part: "%s"' % part.type)
d90c30801cdf bundle20: extract core payload generation in its own function
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 26395
diff changeset
683 for chunk in part.getchunks(ui=self.ui):
d90c30801cdf bundle20: extract core payload generation in its own function
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 26395
diff changeset
684 yield chunk
d90c30801cdf bundle20: extract core payload generation in its own function
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 26395
diff changeset
685 outdebug(self.ui, 'end of bundle')
d90c30801cdf bundle20: extract core payload generation in its own function
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 26395
diff changeset
686 yield _pack(_fpartheadersize, 0)
d90c30801cdf bundle20: extract core payload generation in its own function
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 26395
diff changeset
687
d90c30801cdf bundle20: extract core payload generation in its own function
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 26395
diff changeset
688
24794
21f2e8f48ae1 bundle2: add a 'salvageoutput' method on bundle20
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 24793
diff changeset
689 def salvageoutput(self):
21f2e8f48ae1 bundle2: add a 'salvageoutput' method on bundle20
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 24793
diff changeset
690 """return a list with a copy of all output parts in the bundle
21f2e8f48ae1 bundle2: add a 'salvageoutput' method on bundle20
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 24793
diff changeset
691
21f2e8f48ae1 bundle2: add a 'salvageoutput' method on bundle20
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 24793
diff changeset
692 This is meant to be used during error handling to make sure we preserve
21f2e8f48ae1 bundle2: add a 'salvageoutput' method on bundle20
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 24793
diff changeset
693 server output"""
21f2e8f48ae1 bundle2: add a 'salvageoutput' method on bundle20
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 24793
diff changeset
694 salvaged = []
21f2e8f48ae1 bundle2: add a 'salvageoutput' method on bundle20
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 24793
diff changeset
695 for part in self._parts:
21f2e8f48ae1 bundle2: add a 'salvageoutput' method on bundle20
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 24793
diff changeset
696 if part.type.startswith('output'):
21f2e8f48ae1 bundle2: add a 'salvageoutput' method on bundle20
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 24793
diff changeset
697 salvaged.append(part.copy())
21f2e8f48ae1 bundle2: add a 'salvageoutput' method on bundle20
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 24793
diff changeset
698 return salvaged
21f2e8f48ae1 bundle2: add a 'salvageoutput' method on bundle20
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 24793
diff changeset
699
21f2e8f48ae1 bundle2: add a 'salvageoutput' method on bundle20
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 24793
diff changeset
700
21013
a813caca89b3 bundle2: extract stream/unpack logic in an unpackermixin
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 21005
diff changeset
701 class unpackermixin(object):
a813caca89b3 bundle2: extract stream/unpack logic in an unpackermixin
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 21005
diff changeset
702 """A mixin to extract bytes and struct data from a stream"""
20802
520df53ad26a bundle2: a very first version of bundle2 unbundler
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 20801
diff changeset
703
21013
a813caca89b3 bundle2: extract stream/unpack logic in an unpackermixin
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 21005
diff changeset
704 def __init__(self, fp):
20802
520df53ad26a bundle2: a very first version of bundle2 unbundler
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 20801
diff changeset
705 self._fp = fp
520df53ad26a bundle2: a very first version of bundle2 unbundler
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 20801
diff changeset
706
520df53ad26a bundle2: a very first version of bundle2 unbundler
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 20801
diff changeset
707 def _unpack(self, format):
31862
9bd9e9cb3f69 bundle2: clarify the docstring of unpackermixin methods
Pierre-Yves David <pierre-yves.david@ens-lyon.org>
parents: 31861
diff changeset
708 """unpack this struct format from the stream
9bd9e9cb3f69 bundle2: clarify the docstring of unpackermixin methods
Pierre-Yves David <pierre-yves.david@ens-lyon.org>
parents: 31861
diff changeset
709
9bd9e9cb3f69 bundle2: clarify the docstring of unpackermixin methods
Pierre-Yves David <pierre-yves.david@ens-lyon.org>
parents: 31861
diff changeset
710 This method is meant for internal usage by the bundle2 protocol only.
9bd9e9cb3f69 bundle2: clarify the docstring of unpackermixin methods
Pierre-Yves David <pierre-yves.david@ens-lyon.org>
parents: 31861
diff changeset
711 They directly manipulate the low level stream including bundle2 level
9bd9e9cb3f69 bundle2: clarify the docstring of unpackermixin methods
Pierre-Yves David <pierre-yves.david@ens-lyon.org>
parents: 31861
diff changeset
712 instruction.
9bd9e9cb3f69 bundle2: clarify the docstring of unpackermixin methods
Pierre-Yves David <pierre-yves.david@ens-lyon.org>
parents: 31861
diff changeset
713
9bd9e9cb3f69 bundle2: clarify the docstring of unpackermixin methods
Pierre-Yves David <pierre-yves.david@ens-lyon.org>
parents: 31861
diff changeset
714 Do not use it to implement higher-level logic or methods."""
20802
520df53ad26a bundle2: a very first version of bundle2 unbundler
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 20801
diff changeset
715 data = self._readexact(struct.calcsize(format))
520df53ad26a bundle2: a very first version of bundle2 unbundler
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 20801
diff changeset
716 return _unpack(format, data)
520df53ad26a bundle2: a very first version of bundle2 unbundler
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 20801
diff changeset
717
520df53ad26a bundle2: a very first version of bundle2 unbundler
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 20801
diff changeset
718 def _readexact(self, size):
31862
9bd9e9cb3f69 bundle2: clarify the docstring of unpackermixin methods
Pierre-Yves David <pierre-yves.david@ens-lyon.org>
parents: 31861
diff changeset
719 """read exactly <size> bytes from the stream
9bd9e9cb3f69 bundle2: clarify the docstring of unpackermixin methods
Pierre-Yves David <pierre-yves.david@ens-lyon.org>
parents: 31861
diff changeset
720
9bd9e9cb3f69 bundle2: clarify the docstring of unpackermixin methods
Pierre-Yves David <pierre-yves.david@ens-lyon.org>
parents: 31861
diff changeset
721 This method is meant for internal usage by the bundle2 protocol only.
9bd9e9cb3f69 bundle2: clarify the docstring of unpackermixin methods
Pierre-Yves David <pierre-yves.david@ens-lyon.org>
parents: 31861
diff changeset
722 They directly manipulate the low level stream including bundle2 level
9bd9e9cb3f69 bundle2: clarify the docstring of unpackermixin methods
Pierre-Yves David <pierre-yves.david@ens-lyon.org>
parents: 31861
diff changeset
723 instruction.
9bd9e9cb3f69 bundle2: clarify the docstring of unpackermixin methods
Pierre-Yves David <pierre-yves.david@ens-lyon.org>
parents: 31861
diff changeset
724
9bd9e9cb3f69 bundle2: clarify the docstring of unpackermixin methods
Pierre-Yves David <pierre-yves.david@ens-lyon.org>
parents: 31861
diff changeset
725 Do not use it to implement higher-level logic or methods."""
20802
520df53ad26a bundle2: a very first version of bundle2 unbundler
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 20801
diff changeset
726 return changegroup.readexactly(self._fp, size)
520df53ad26a bundle2: a very first version of bundle2 unbundler
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 20801
diff changeset
727
25640
39f0064a3079 bundle2.getunbundler: rename "header" to "magicstring"
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 25518
diff changeset
728 def getunbundler(ui, fp, magicstring=None):
39f0064a3079 bundle2.getunbundler: rename "header" to "magicstring"
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 25518
diff changeset
729 """return a valid unbundler object for a given magicstring"""
39f0064a3079 bundle2.getunbundler: rename "header" to "magicstring"
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 25518
diff changeset
730 if magicstring is None:
39f0064a3079 bundle2.getunbundler: rename "header" to "magicstring"
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 25518
diff changeset
731 magicstring = changegroup.readexactly(fp, 4)
39f0064a3079 bundle2.getunbundler: rename "header" to "magicstring"
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 25518
diff changeset
732 magic, version = magicstring[0:2], magicstring[2:4]
24648
5cac3accdaa1 unbundle20: allow generic dispatch between unbundlers
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 24642
diff changeset
733 if magic != 'HG':
33134
0aae80d14fed bundle2: add some debugging information to the not-a-bundle error
Siddharth Agarwal <sid0@fb.com>
parents: 33056
diff changeset
734 ui.debug(
0aae80d14fed bundle2: add some debugging information to the not-a-bundle error
Siddharth Agarwal <sid0@fb.com>
parents: 33056
diff changeset
735 "error: invalid magic: %r (version %r), should be 'HG'\n"
0aae80d14fed bundle2: add some debugging information to the not-a-bundle error
Siddharth Agarwal <sid0@fb.com>
parents: 33056
diff changeset
736 % (magic, version))
26587
56b2bcea2529 error: get Abort from 'error' instead of 'util'
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 26565
diff changeset
737 raise error.Abort(_('not a Mercurial bundle'))
24648
5cac3accdaa1 unbundle20: allow generic dispatch between unbundlers
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 24642
diff changeset
738 unbundlerclass = formatmap.get(version)
5cac3accdaa1 unbundle20: allow generic dispatch between unbundlers
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 24642
diff changeset
739 if unbundlerclass is None:
26587
56b2bcea2529 error: get Abort from 'error' instead of 'util'
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 26565
diff changeset
740 raise error.Abort(_('unknown bundle version %s') % version)
24648
5cac3accdaa1 unbundle20: allow generic dispatch between unbundlers
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 24642
diff changeset
741 unbundler = unbundlerclass(ui, fp)
25640
39f0064a3079 bundle2.getunbundler: rename "header" to "magicstring"
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 25518
diff changeset
742 indebug(ui, 'start processing of %s stream' % magicstring)
24642
54e5c239c2d9 unbundle20: move header parsing into the 'getunbundler' function
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 24641
diff changeset
743 return unbundler
24641
60fecc5b14a4 unbundle20: retrieve unbundler instances through a factory function
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 24640
diff changeset
744
21013
a813caca89b3 bundle2: extract stream/unpack logic in an unpackermixin
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 21005
diff changeset
745 class unbundle20(unpackermixin):
a813caca89b3 bundle2: extract stream/unpack logic in an unpackermixin
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 21005
diff changeset
746 """interpret a bundle2 stream
a813caca89b3 bundle2: extract stream/unpack logic in an unpackermixin
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 21005
diff changeset
747
21129
07bcbf326c8d bundle2: use an official iterparts method to unbundle parts
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 21066
diff changeset
748 This class is fed with a binary stream and yields parts through its
07bcbf326c8d bundle2: use an official iterparts method to unbundle parts
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 21066
diff changeset
749 `iterparts` methods."""
21013
a813caca89b3 bundle2: extract stream/unpack logic in an unpackermixin
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 21005
diff changeset
750
26542
b87e4638dabf bundle2: add a way to just forward the bundle2 stream to another user
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 26541
diff changeset
751 _magicstring = 'HG20'
b87e4638dabf bundle2: add a way to just forward the bundle2 stream to another user
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 26541
diff changeset
752
24642
54e5c239c2d9 unbundle20: move header parsing into the 'getunbundler' function
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 24641
diff changeset
753 def __init__(self, ui, fp):
21066
5ecfe76d0d96 bundle2: make header reading optional
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 21062
diff changeset
754 """If header is specified, we do not read it out of the stream."""
21013
a813caca89b3 bundle2: extract stream/unpack logic in an unpackermixin
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 21005
diff changeset
755 self.ui = ui
30363
d045b4091197 bundle2: use compression engines API to obtain decompressor
Gregory Szorc <gregory.szorc@gmail.com>
parents: 30361
diff changeset
756 self._compengine = util.compengines.forbundletype('UN')
26802
42f705f2c02d bundle2: make unbundle.compressed return True when compressed
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 26795
diff changeset
757 self._compressed = None
21013
a813caca89b3 bundle2: extract stream/unpack logic in an unpackermixin
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 21005
diff changeset
758 super(unbundle20, self).__init__(fp)
a813caca89b3 bundle2: extract stream/unpack logic in an unpackermixin
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 21005
diff changeset
759
20802
520df53ad26a bundle2: a very first version of bundle2 unbundler
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 20801
diff changeset
760 @util.propertycache
520df53ad26a bundle2: a very first version of bundle2 unbundler
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 20801
diff changeset
761 def params(self):
21024
7731a2281cf0 spelling: fixes from spell checker
Mads Kiilerich <madski@unity3d.com>
parents: 21020
diff changeset
762 """dictionary of stream level parameters"""
25320
697d8953b04d bundle2: handle new line in 'indebug' function
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 25319
diff changeset
763 indebug(self.ui, 'reading bundle2 stream parameters')
20805
c5aaeca0cfbf bundle2: support for unbundling simple parameter
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 20804
diff changeset
764 params = {}
c5aaeca0cfbf bundle2: support for unbundling simple parameter
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 20804
diff changeset
765 paramssize = self._unpack(_fstreamparamsize)[0]
23011
006a81d07e57 bundle2: detect and disallow a negative chunk size
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 23010
diff changeset
766 if paramssize < 0:
006a81d07e57 bundle2: detect and disallow a negative chunk size
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 23010
diff changeset
767 raise error.BundleValueError('negative bundle param size: %i'
006a81d07e57 bundle2: detect and disallow a negative chunk size
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 23010
diff changeset
768 % paramssize)
20805
c5aaeca0cfbf bundle2: support for unbundling simple parameter
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 20804
diff changeset
769 if paramssize:
26541
d40029b4296e bundle2: split parameter retrieval and processing
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 26404
diff changeset
770 params = self._readexact(paramssize)
d40029b4296e bundle2: split parameter retrieval and processing
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 26404
diff changeset
771 params = self._processallparams(params)
20805
c5aaeca0cfbf bundle2: support for unbundling simple parameter
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 20804
diff changeset
772 return params
20802
520df53ad26a bundle2: a very first version of bundle2 unbundler
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 20801
diff changeset
773
26541
d40029b4296e bundle2: split parameter retrieval and processing
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 26404
diff changeset
774 def _processallparams(self, paramsblock):
d40029b4296e bundle2: split parameter retrieval and processing
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 26404
diff changeset
775 """"""
29591
6215b5537ba5 bundle2: use a sorted dict for holding parameters
Gregory Szorc <gregory.szorc@gmail.com>
parents: 29294
diff changeset
776 params = util.sortdict()
26541
d40029b4296e bundle2: split parameter retrieval and processing
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 26404
diff changeset
777 for p in paramsblock.split(' '):
d40029b4296e bundle2: split parameter retrieval and processing
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 26404
diff changeset
778 p = p.split('=', 1)
28883
032c4c2f802a pycompat: switch to util.urlreq/util.urlerr for py3 compat
timeless <timeless@mozdev.org>
parents: 28672
diff changeset
779 p = [urlreq.unquote(i) for i in p]
26541
d40029b4296e bundle2: split parameter retrieval and processing
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 26404
diff changeset
780 if len(p) < 2:
d40029b4296e bundle2: split parameter retrieval and processing
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 26404
diff changeset
781 p.append(None)
d40029b4296e bundle2: split parameter retrieval and processing
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 26404
diff changeset
782 self._processparam(*p)
d40029b4296e bundle2: split parameter retrieval and processing
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 26404
diff changeset
783 params[p[0]] = p[1]
d40029b4296e bundle2: split parameter retrieval and processing
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 26404
diff changeset
784 return params
d40029b4296e bundle2: split parameter retrieval and processing
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 26404
diff changeset
785
d40029b4296e bundle2: split parameter retrieval and processing
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 26404
diff changeset
786
20844
2631204d7305 bundle2: implement the mandatory/advisory logic for parameter
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 20843
diff changeset
787 def _processparam(self, name, value):
2631204d7305 bundle2: implement the mandatory/advisory logic for parameter
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 20843
diff changeset
788 """process a parameter, applying its effect if needed
2631204d7305 bundle2: implement the mandatory/advisory logic for parameter
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 20843
diff changeset
789
2631204d7305 bundle2: implement the mandatory/advisory logic for parameter
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 20843
diff changeset
790 Parameter starting with a lower case letter are advisory and will be
2631204d7305 bundle2: implement the mandatory/advisory logic for parameter
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 20843
diff changeset
791 ignored when unknown. Those starting with an upper case letter are
2631204d7305 bundle2: implement the mandatory/advisory logic for parameter
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 20843
diff changeset
792 mandatory and will this function will raise a KeyError when unknown.
2631204d7305 bundle2: implement the mandatory/advisory logic for parameter
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 20843
diff changeset
793
2631204d7305 bundle2: implement the mandatory/advisory logic for parameter
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 20843
diff changeset
794 Note: no option are currently supported. Any input will be either
2631204d7305 bundle2: implement the mandatory/advisory logic for parameter
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 20843
diff changeset
795 ignored or failing.
2631204d7305 bundle2: implement the mandatory/advisory logic for parameter
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 20843
diff changeset
796 """
2631204d7305 bundle2: implement the mandatory/advisory logic for parameter
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 20843
diff changeset
797 if not name:
34284
a48b3db0baba bundle2: stop using %r to quote part names
Augie Fackler <augie@google.com>
parents: 34268
diff changeset
798 raise ValueError(r'empty parameter name')
a48b3db0baba bundle2: stop using %r to quote part names
Augie Fackler <augie@google.com>
parents: 34268
diff changeset
799 if name[0:1] not in pycompat.bytestr(string.ascii_letters):
a48b3db0baba bundle2: stop using %r to quote part names
Augie Fackler <augie@google.com>
parents: 34268
diff changeset
800 raise ValueError(r'non letter first character: %s' % name)
26395
4e7b0bf9f0b1 unbundle20: allow registering handlers for stream level parameters
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 26393
diff changeset
801 try:
4e7b0bf9f0b1 unbundle20: allow registering handlers for stream level parameters
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 26393
diff changeset
802 handler = b2streamparamsmap[name.lower()]
4e7b0bf9f0b1 unbundle20: allow registering handlers for stream level parameters
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 26393
diff changeset
803 except KeyError:
34284
a48b3db0baba bundle2: stop using %r to quote part names
Augie Fackler <augie@google.com>
parents: 34268
diff changeset
804 if name[0:1].islower():
a48b3db0baba bundle2: stop using %r to quote part names
Augie Fackler <augie@google.com>
parents: 34268
diff changeset
805 indebug(self.ui, "ignoring unknown parameter %s" % name)
26395
4e7b0bf9f0b1 unbundle20: allow registering handlers for stream level parameters
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 26393
diff changeset
806 else:
4e7b0bf9f0b1 unbundle20: allow registering handlers for stream level parameters
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 26393
diff changeset
807 raise error.BundleUnknownFeatureError(params=(name,))
20844
2631204d7305 bundle2: implement the mandatory/advisory logic for parameter
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 20843
diff changeset
808 else:
26395
4e7b0bf9f0b1 unbundle20: allow registering handlers for stream level parameters
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 26393
diff changeset
809 handler(self, name, value)
20844
2631204d7305 bundle2: implement the mandatory/advisory logic for parameter
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 20843
diff changeset
810
26542
b87e4638dabf bundle2: add a way to just forward the bundle2 stream to another user
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 26541
diff changeset
811 def _forwardchunks(self):
b87e4638dabf bundle2: add a way to just forward the bundle2 stream to another user
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 26541
diff changeset
812 """utility to transfer a bundle2 as binary
b87e4638dabf bundle2: add a way to just forward the bundle2 stream to another user
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 26541
diff changeset
813
b87e4638dabf bundle2: add a way to just forward the bundle2 stream to another user
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 26541
diff changeset
814 This is made necessary by the fact the 'getbundle' command over 'ssh'
b87e4638dabf bundle2: add a way to just forward the bundle2 stream to another user
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 26541
diff changeset
815 have no way to know then the reply end, relying on the bundle to be
b87e4638dabf bundle2: add a way to just forward the bundle2 stream to another user
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 26541
diff changeset
816 interpreted to know its end. This is terrible and we are sorry, but we
b87e4638dabf bundle2: add a way to just forward the bundle2 stream to another user
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 26541
diff changeset
817 needed to move forward to get general delta enabled.
b87e4638dabf bundle2: add a way to just forward the bundle2 stream to another user
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 26541
diff changeset
818 """
b87e4638dabf bundle2: add a way to just forward the bundle2 stream to another user
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 26541
diff changeset
819 yield self._magicstring
b87e4638dabf bundle2: add a way to just forward the bundle2 stream to another user
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 26541
diff changeset
820 assert 'params' not in vars(self)
b87e4638dabf bundle2: add a way to just forward the bundle2 stream to another user
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 26541
diff changeset
821 paramssize = self._unpack(_fstreamparamsize)[0]
b87e4638dabf bundle2: add a way to just forward the bundle2 stream to another user
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 26541
diff changeset
822 if paramssize < 0:
b87e4638dabf bundle2: add a way to just forward the bundle2 stream to another user
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 26541
diff changeset
823 raise error.BundleValueError('negative bundle param size: %i'
b87e4638dabf bundle2: add a way to just forward the bundle2 stream to another user
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 26541
diff changeset
824 % paramssize)
b87e4638dabf bundle2: add a way to just forward the bundle2 stream to another user
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 26541
diff changeset
825 yield _pack(_fstreamparamsize, paramssize)
b87e4638dabf bundle2: add a way to just forward the bundle2 stream to another user
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 26541
diff changeset
826 if paramssize:
b87e4638dabf bundle2: add a way to just forward the bundle2 stream to another user
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 26541
diff changeset
827 params = self._readexact(paramssize)
b87e4638dabf bundle2: add a way to just forward the bundle2 stream to another user
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 26541
diff changeset
828 self._processallparams(params)
b87e4638dabf bundle2: add a way to just forward the bundle2 stream to another user
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 26541
diff changeset
829 yield params
30363
d045b4091197 bundle2: use compression engines API to obtain decompressor
Gregory Szorc <gregory.szorc@gmail.com>
parents: 30361
diff changeset
830 assert self._compengine.bundletype == 'UN'
26542
b87e4638dabf bundle2: add a way to just forward the bundle2 stream to another user
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 26541
diff changeset
831 # From there, payload might need to be decompressed
30363
d045b4091197 bundle2: use compression engines API to obtain decompressor
Gregory Szorc <gregory.szorc@gmail.com>
parents: 30361
diff changeset
832 self._fp = self._compengine.decompressorreader(self._fp)
26542
b87e4638dabf bundle2: add a way to just forward the bundle2 stream to another user
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 26541
diff changeset
833 emptycount = 0
b87e4638dabf bundle2: add a way to just forward the bundle2 stream to another user
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 26541
diff changeset
834 while emptycount < 2:
b87e4638dabf bundle2: add a way to just forward the bundle2 stream to another user
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 26541
diff changeset
835 # so we can brainlessly loop
b87e4638dabf bundle2: add a way to just forward the bundle2 stream to another user
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 26541
diff changeset
836 assert _fpartheadersize == _fpayloadsize
b87e4638dabf bundle2: add a way to just forward the bundle2 stream to another user
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 26541
diff changeset
837 size = self._unpack(_fpartheadersize)[0]
b87e4638dabf bundle2: add a way to just forward the bundle2 stream to another user
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 26541
diff changeset
838 yield _pack(_fpartheadersize, size)
b87e4638dabf bundle2: add a way to just forward the bundle2 stream to another user
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 26541
diff changeset
839 if size:
b87e4638dabf bundle2: add a way to just forward the bundle2 stream to another user
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 26541
diff changeset
840 emptycount = 0
b87e4638dabf bundle2: add a way to just forward the bundle2 stream to another user
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 26541
diff changeset
841 else:
b87e4638dabf bundle2: add a way to just forward the bundle2 stream to another user
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 26541
diff changeset
842 emptycount += 1
b87e4638dabf bundle2: add a way to just forward the bundle2 stream to another user
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 26541
diff changeset
843 continue
b87e4638dabf bundle2: add a way to just forward the bundle2 stream to another user
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 26541
diff changeset
844 if size == flaginterrupt:
b87e4638dabf bundle2: add a way to just forward the bundle2 stream to another user
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 26541
diff changeset
845 continue
b87e4638dabf bundle2: add a way to just forward the bundle2 stream to another user
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 26541
diff changeset
846 elif size < 0:
b87e4638dabf bundle2: add a way to just forward the bundle2 stream to another user
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 26541
diff changeset
847 raise error.BundleValueError('negative chunk size: %i')
b87e4638dabf bundle2: add a way to just forward the bundle2 stream to another user
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 26541
diff changeset
848 yield self._readexact(size)
b87e4638dabf bundle2: add a way to just forward the bundle2 stream to another user
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 26541
diff changeset
849
b87e4638dabf bundle2: add a way to just forward the bundle2 stream to another user
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 26541
diff changeset
850
35137
da91e7309daf bundle2: don't use seekable bundle2 parts by default (issue5691)
Gregory Szorc <gregory.szorc@gmail.com>
parents: 35136
diff changeset
851 def iterparts(self, seekable=False):
20802
520df53ad26a bundle2: a very first version of bundle2 unbundler
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 20801
diff changeset
852 """yield all parts contained in the stream"""
35137
da91e7309daf bundle2: don't use seekable bundle2 parts by default (issue5691)
Gregory Szorc <gregory.szorc@gmail.com>
parents: 35136
diff changeset
853 cls = seekableunbundlepart if seekable else unbundlepart
20802
520df53ad26a bundle2: a very first version of bundle2 unbundler
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 20801
diff changeset
854 # make sure param have been loaded
520df53ad26a bundle2: a very first version of bundle2 unbundler
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 20801
diff changeset
855 self.params
26404
795f02a24b87 bundle2: allow compressed bundle
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 26396
diff changeset
856 # From there, payload need to be decompressed
30363
d045b4091197 bundle2: use compression engines API to obtain decompressor
Gregory Szorc <gregory.szorc@gmail.com>
parents: 30361
diff changeset
857 self._fp = self._compengine.decompressorreader(self._fp)
25320
697d8953b04d bundle2: handle new line in 'indebug' function
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 25319
diff changeset
858 indebug(self.ui, 'start extraction of bundle2 parts')
21014
a6246bba7b9e bundle2: add an unbundle part responsible from unbundling part
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 21013
diff changeset
859 headerblock = self._readpartheader()
a6246bba7b9e bundle2: add an unbundle part responsible from unbundling part
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 21013
diff changeset
860 while headerblock is not None:
35137
da91e7309daf bundle2: don't use seekable bundle2 parts by default (issue5691)
Gregory Szorc <gregory.szorc@gmail.com>
parents: 35136
diff changeset
861 part = cls(self.ui, headerblock, self._fp)
20802
520df53ad26a bundle2: a very first version of bundle2 unbundler
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 20801
diff changeset
862 yield part
35135
db5038525718 bundle2: implement consume() API on unbundlepart
Gregory Szorc <gregory.szorc@gmail.com>
parents: 35134
diff changeset
863 # Ensure part is fully consumed so we can start reading the next
db5038525718 bundle2: implement consume() API on unbundlepart
Gregory Szorc <gregory.szorc@gmail.com>
parents: 35134
diff changeset
864 # part.
db5038525718 bundle2: implement consume() API on unbundlepart
Gregory Szorc <gregory.szorc@gmail.com>
parents: 35134
diff changeset
865 part.consume()
35136
2b72bc88043f bundle2: only seek to beginning of part in bundlerepo
Gregory Szorc <gregory.szorc@gmail.com>
parents: 35135
diff changeset
866
21014
a6246bba7b9e bundle2: add an unbundle part responsible from unbundling part
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 21013
diff changeset
867 headerblock = self._readpartheader()
25320
697d8953b04d bundle2: handle new line in 'indebug' function
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 25319
diff changeset
868 indebug(self.ui, 'end of bundle2 stream')
20802
520df53ad26a bundle2: a very first version of bundle2 unbundler
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 20801
diff changeset
869
21014
a6246bba7b9e bundle2: add an unbundle part responsible from unbundling part
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 21013
diff changeset
870 def _readpartheader(self):
a6246bba7b9e bundle2: add an unbundle part responsible from unbundling part
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 21013
diff changeset
871 """reads a part header size and return the bytes blob
20864
9a75d2559cff bundle2: support unbundling empty part
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 20856
diff changeset
872
21014
a6246bba7b9e bundle2: add an unbundle part responsible from unbundling part
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 21013
diff changeset
873 returns None if empty"""
20864
9a75d2559cff bundle2: support unbundling empty part
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 20856
diff changeset
874 headersize = self._unpack(_fpartheadersize)[0]
23011
006a81d07e57 bundle2: detect and disallow a negative chunk size
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 23010
diff changeset
875 if headersize < 0:
006a81d07e57 bundle2: detect and disallow a negative chunk size
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 23010
diff changeset
876 raise error.BundleValueError('negative part header size: %i'
006a81d07e57 bundle2: detect and disallow a negative chunk size
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 23010
diff changeset
877 % headersize)
25320
697d8953b04d bundle2: handle new line in 'indebug' function
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 25319
diff changeset
878 indebug(self.ui, 'part header size: %i' % headersize)
21014
a6246bba7b9e bundle2: add an unbundle part responsible from unbundling part
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 21013
diff changeset
879 if headersize:
a6246bba7b9e bundle2: add an unbundle part responsible from unbundling part
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 21013
diff changeset
880 return self._readexact(headersize)
a6246bba7b9e bundle2: add an unbundle part responsible from unbundling part
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 21013
diff changeset
881 return None
20864
9a75d2559cff bundle2: support unbundling empty part
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 20856
diff changeset
882
24071
184a2f6f40da bundle2.unbundle20: add compressed() method
Eric Sumner <ericsumner@fb.com>
parents: 24070
diff changeset
883 def compressed(self):
26802
42f705f2c02d bundle2: make unbundle.compressed return True when compressed
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 26795
diff changeset
884 self.params # load params
42f705f2c02d bundle2: make unbundle.compressed return True when compressed
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 26795
diff changeset
885 return self._compressed
20802
520df53ad26a bundle2: a very first version of bundle2 unbundler
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 20801
diff changeset
886
31863
cd7aaf344d83 bundle2: move the 'close' method off the unpackermixin
Pierre-Yves David <pierre-yves.david@ens-lyon.org>
parents: 31862
diff changeset
887 def close(self):
cd7aaf344d83 bundle2: move the 'close' method off the unpackermixin
Pierre-Yves David <pierre-yves.david@ens-lyon.org>
parents: 31862
diff changeset
888 """close underlying file"""
cd7aaf344d83 bundle2: move the 'close' method off the unpackermixin
Pierre-Yves David <pierre-yves.david@ens-lyon.org>
parents: 31862
diff changeset
889 if util.safehasattr(self._fp, 'close'):
cd7aaf344d83 bundle2: move the 'close' method off the unpackermixin
Pierre-Yves David <pierre-yves.david@ens-lyon.org>
parents: 31862
diff changeset
890 return self._fp.close()
cd7aaf344d83 bundle2: move the 'close' method off the unpackermixin
Pierre-Yves David <pierre-yves.david@ens-lyon.org>
parents: 31862
diff changeset
891
24686
e0e28e910fa3 bundle2: rename format, parts and config to final names
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 24648
diff changeset
892 formatmap = {'20': unbundle20}
24648
5cac3accdaa1 unbundle20: allow generic dispatch between unbundlers
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 24642
diff changeset
893
26395
4e7b0bf9f0b1 unbundle20: allow registering handlers for stream level parameters
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 26393
diff changeset
894 b2streamparamsmap = {}
4e7b0bf9f0b1 unbundle20: allow registering handlers for stream level parameters
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 26393
diff changeset
895
4e7b0bf9f0b1 unbundle20: allow registering handlers for stream level parameters
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 26393
diff changeset
896 def b2streamparamhandler(name):
4e7b0bf9f0b1 unbundle20: allow registering handlers for stream level parameters
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 26393
diff changeset
897 """register a handler for a stream level parameter"""
4e7b0bf9f0b1 unbundle20: allow registering handlers for stream level parameters
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 26393
diff changeset
898 def decorator(func):
4e7b0bf9f0b1 unbundle20: allow registering handlers for stream level parameters
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 26393
diff changeset
899 assert name not in formatmap
4e7b0bf9f0b1 unbundle20: allow registering handlers for stream level parameters
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 26393
diff changeset
900 b2streamparamsmap[name] = func
4e7b0bf9f0b1 unbundle20: allow registering handlers for stream level parameters
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 26393
diff changeset
901 return func
4e7b0bf9f0b1 unbundle20: allow registering handlers for stream level parameters
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 26393
diff changeset
902 return decorator
4e7b0bf9f0b1 unbundle20: allow registering handlers for stream level parameters
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 26393
diff changeset
903
26404
795f02a24b87 bundle2: allow compressed bundle
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 26396
diff changeset
904 @b2streamparamhandler('compression')
795f02a24b87 bundle2: allow compressed bundle
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 26396
diff changeset
905 def processcompression(unbundler, param, value):
795f02a24b87 bundle2: allow compressed bundle
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 26396
diff changeset
906 """read compression parameter and install payload decompression"""
30363
d045b4091197 bundle2: use compression engines API to obtain decompressor
Gregory Szorc <gregory.szorc@gmail.com>
parents: 30361
diff changeset
907 if value not in util.compengines.supportedbundletypes:
26404
795f02a24b87 bundle2: allow compressed bundle
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 26396
diff changeset
908 raise error.BundleUnknownFeatureError(params=(param,),
795f02a24b87 bundle2: allow compressed bundle
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 26396
diff changeset
909 values=(value,))
30363
d045b4091197 bundle2: use compression engines API to obtain decompressor
Gregory Szorc <gregory.szorc@gmail.com>
parents: 30361
diff changeset
910 unbundler._compengine = util.compengines.forbundletype(value)
26802
42f705f2c02d bundle2: make unbundle.compressed return True when compressed
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 26795
diff changeset
911 if value is not None:
42f705f2c02d bundle2: make unbundle.compressed return True when compressed
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 26795
diff changeset
912 unbundler._compressed = True
26404
795f02a24b87 bundle2: allow compressed bundle
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 26396
diff changeset
913
21005
3d38ebb586fe bundle2: rename part to bundlepart
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 21004
diff changeset
914 class bundlepart(object):
20856
8a6a86c9a5b5 bundle2: support bundling of empty part (with a type)
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 20844
diff changeset
915 """A bundle2 part contains application level payload
8a6a86c9a5b5 bundle2: support bundling of empty part (with a type)
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 20844
diff changeset
916
8a6a86c9a5b5 bundle2: support bundling of empty part (with a type)
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 20844
diff changeset
917 The part `type` is used to route the part to the application level
8a6a86c9a5b5 bundle2: support bundling of empty part (with a type)
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 20844
diff changeset
918 handler.
21604
c399bf961cb9 bundle2: the ability to set ``data`` attribute of the part is now official
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 21603
diff changeset
919
c399bf961cb9 bundle2: the ability to set ``data`` attribute of the part is now official
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 21603
diff changeset
920 The part payload is contained in ``part.data``. It could be raw bytes or a
21605
f9dabfceb259 bundle2: introduce a ``addparam`` method on part
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 21604
diff changeset
921 generator of byte chunks.
f9dabfceb259 bundle2: introduce a ``addparam`` method on part
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 21604
diff changeset
922
f9dabfceb259 bundle2: introduce a ``addparam`` method on part
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 21604
diff changeset
923 You can add parameters to the part using the ``addparam`` method.
f9dabfceb259 bundle2: introduce a ``addparam`` method on part
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 21604
diff changeset
924 Parameters can be either mandatory (default) or advisory. Remote side
f9dabfceb259 bundle2: introduce a ``addparam`` method on part
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 21604
diff changeset
925 should be able to safely ignore the advisory ones.
f9dabfceb259 bundle2: introduce a ``addparam`` method on part
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 21604
diff changeset
926
f9dabfceb259 bundle2: introduce a ``addparam`` method on part
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 21604
diff changeset
927 Both data and parameters cannot be modified after the generation has begun.
20856
8a6a86c9a5b5 bundle2: support bundling of empty part (with a type)
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 20844
diff changeset
928 """
8a6a86c9a5b5 bundle2: support bundling of empty part (with a type)
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 20844
diff changeset
929
20877
9e9e3a4e9261 bundle2: part params
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 20876
diff changeset
930 def __init__(self, parttype, mandatoryparams=(), advisoryparams=(),
23590
4440c7cc3728 bundle2.bundlepart: make mandatory part flag explicit in API
Eric Sumner <ericsumner@fb.com>
parents: 23586
diff changeset
931 data='', mandatory=True):
23868
405eacbecc52 bundle2: enforce parttype as alphanumerical
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 23590
diff changeset
932 validateparttype(parttype)
20995
e995d104c87f bundle2: add an integer id to part
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 20952
diff changeset
933 self.id = None
20856
8a6a86c9a5b5 bundle2: support bundling of empty part (with a type)
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 20844
diff changeset
934 self.type = parttype
21604
c399bf961cb9 bundle2: the ability to set ``data`` attribute of the part is now official
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 21603
diff changeset
935 self._data = data
21605
f9dabfceb259 bundle2: introduce a ``addparam`` method on part
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 21604
diff changeset
936 self._mandatoryparams = list(mandatoryparams)
f9dabfceb259 bundle2: introduce a ``addparam`` method on part
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 21604
diff changeset
937 self._advisoryparams = list(advisoryparams)
21607
054fa5176fa7 bundle2: forbid duplicate parameter keys
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 21606
diff changeset
938 # checking for duplicated entries
054fa5176fa7 bundle2: forbid duplicate parameter keys
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 21606
diff changeset
939 self._seenparams = set()
054fa5176fa7 bundle2: forbid duplicate parameter keys
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 21606
diff changeset
940 for pname, __ in self._mandatoryparams + self._advisoryparams:
054fa5176fa7 bundle2: forbid duplicate parameter keys
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 21606
diff changeset
941 if pname in self._seenparams:
31652
4dbef666c6c9 bundle2: use ProgrammingError
Jun Wu <quark@fb.com>
parents: 31483
diff changeset
942 raise error.ProgrammingError('duplicated params: %s' % pname)
21607
054fa5176fa7 bundle2: forbid duplicate parameter keys
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 21606
diff changeset
943 self._seenparams.add(pname)
21601
7ff01befc7ec bundle2: track life cycle of parts
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 21600
diff changeset
944 # status of the part's generation:
7ff01befc7ec bundle2: track life cycle of parts
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 21600
diff changeset
945 # - None: not started,
7ff01befc7ec bundle2: track life cycle of parts
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 21600
diff changeset
946 # - False: currently generated,
7ff01befc7ec bundle2: track life cycle of parts
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 21600
diff changeset
947 # - True: generation done.
7ff01befc7ec bundle2: track life cycle of parts
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 21600
diff changeset
948 self._generated = None
23590
4440c7cc3728 bundle2.bundlepart: make mandatory part flag explicit in API
Eric Sumner <ericsumner@fb.com>
parents: 23586
diff changeset
949 self.mandatory = mandatory
20856
8a6a86c9a5b5 bundle2: support bundling of empty part (with a type)
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 20844
diff changeset
950
30906
1f51b4658f21 bundle2: implement a basic __repr__ for bundle2 part
Pierre-Yves David <pierre-yves.david@ens-lyon.org>
parents: 30905
diff changeset
951 def __repr__(self):
1f51b4658f21 bundle2: implement a basic __repr__ for bundle2 part
Pierre-Yves David <pierre-yves.david@ens-lyon.org>
parents: 30905
diff changeset
952 cls = "%s.%s" % (self.__class__.__module__, self.__class__.__name__)
1f51b4658f21 bundle2: implement a basic __repr__ for bundle2 part
Pierre-Yves David <pierre-yves.david@ens-lyon.org>
parents: 30905
diff changeset
953 return ('<%s object at %x; id: %s; type: %s; mandatory: %s>'
1f51b4658f21 bundle2: implement a basic __repr__ for bundle2 part
Pierre-Yves David <pierre-yves.david@ens-lyon.org>
parents: 30905
diff changeset
954 % (cls, id(self), self.id, self.type, self.mandatory))
1f51b4658f21 bundle2: implement a basic __repr__ for bundle2 part
Pierre-Yves David <pierre-yves.david@ens-lyon.org>
parents: 30905
diff changeset
955
24793
2ec894582ea2 bundle2: add a 'copy' method on parts
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 24743
diff changeset
956 def copy(self):
2ec894582ea2 bundle2: add a 'copy' method on parts
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 24743
diff changeset
957 """return a copy of the part
2ec894582ea2 bundle2: add a 'copy' method on parts
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 24743
diff changeset
958
2ec894582ea2 bundle2: add a 'copy' method on parts
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 24743
diff changeset
959 The new part have the very same content but no partid assigned yet.
2ec894582ea2 bundle2: add a 'copy' method on parts
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 24743
diff changeset
960 Parts with generated data cannot be copied."""
2ec894582ea2 bundle2: add a 'copy' method on parts
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 24743
diff changeset
961 assert not util.safehasattr(self.data, 'next')
2ec894582ea2 bundle2: add a 'copy' method on parts
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 24743
diff changeset
962 return self.__class__(self.type, self._mandatoryparams,
2ec894582ea2 bundle2: add a 'copy' method on parts
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 24743
diff changeset
963 self._advisoryparams, self._data, self.mandatory)
2ec894582ea2 bundle2: add a 'copy' method on parts
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 24743
diff changeset
964
21604
c399bf961cb9 bundle2: the ability to set ``data`` attribute of the part is now official
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 21603
diff changeset
965 # methods used to defines the part content
27879
52a4ad62b006 cleanup: use modern @property/@foo.setter property specification
Augie Fackler <augie@google.com>
parents: 27751
diff changeset
966 @property
52a4ad62b006 cleanup: use modern @property/@foo.setter property specification
Augie Fackler <augie@google.com>
parents: 27751
diff changeset
967 def data(self):
52a4ad62b006 cleanup: use modern @property/@foo.setter property specification
Augie Fackler <augie@google.com>
parents: 27751
diff changeset
968 return self._data
52a4ad62b006 cleanup: use modern @property/@foo.setter property specification
Augie Fackler <augie@google.com>
parents: 27751
diff changeset
969
52a4ad62b006 cleanup: use modern @property/@foo.setter property specification
Augie Fackler <augie@google.com>
parents: 27751
diff changeset
970 @data.setter
52a4ad62b006 cleanup: use modern @property/@foo.setter property specification
Augie Fackler <augie@google.com>
parents: 27751
diff changeset
971 def data(self, data):
21604
c399bf961cb9 bundle2: the ability to set ``data`` attribute of the part is now official
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 21603
diff changeset
972 if self._generated is not None:
21618
7568f5c1c801 bundle2: move exception classes into the error module
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 21617
diff changeset
973 raise error.ReadOnlyPartError('part is being generated')
21604
c399bf961cb9 bundle2: the ability to set ``data`` attribute of the part is now official
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 21603
diff changeset
974 self._data = data
c399bf961cb9 bundle2: the ability to set ``data`` attribute of the part is now official
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 21603
diff changeset
975
21605
f9dabfceb259 bundle2: introduce a ``addparam`` method on part
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 21604
diff changeset
976 @property
f9dabfceb259 bundle2: introduce a ``addparam`` method on part
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 21604
diff changeset
977 def mandatoryparams(self):
f9dabfceb259 bundle2: introduce a ``addparam`` method on part
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 21604
diff changeset
978 # make it an immutable tuple to force people through ``addparam``
f9dabfceb259 bundle2: introduce a ``addparam`` method on part
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 21604
diff changeset
979 return tuple(self._mandatoryparams)
f9dabfceb259 bundle2: introduce a ``addparam`` method on part
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 21604
diff changeset
980
f9dabfceb259 bundle2: introduce a ``addparam`` method on part
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 21604
diff changeset
981 @property
f9dabfceb259 bundle2: introduce a ``addparam`` method on part
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 21604
diff changeset
982 def advisoryparams(self):
f9dabfceb259 bundle2: introduce a ``addparam`` method on part
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 21604
diff changeset
983 # make it an immutable tuple to force people through ``addparam``
f9dabfceb259 bundle2: introduce a ``addparam`` method on part
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 21604
diff changeset
984 return tuple(self._advisoryparams)
f9dabfceb259 bundle2: introduce a ``addparam`` method on part
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 21604
diff changeset
985
f9dabfceb259 bundle2: introduce a ``addparam`` method on part
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 21604
diff changeset
986 def addparam(self, name, value='', mandatory=True):
31861
6d055cd6669f bundle2: add documention to 'part.addparams'
Pierre-Yves David <pierre-yves.david@ens-lyon.org>
parents: 31652
diff changeset
987 """add a parameter to the part
6d055cd6669f bundle2: add documention to 'part.addparams'
Pierre-Yves David <pierre-yves.david@ens-lyon.org>
parents: 31652
diff changeset
988
6d055cd6669f bundle2: add documention to 'part.addparams'
Pierre-Yves David <pierre-yves.david@ens-lyon.org>
parents: 31652
diff changeset
989 If 'mandatory' is set to True, the remote handler must claim support
6d055cd6669f bundle2: add documention to 'part.addparams'
Pierre-Yves David <pierre-yves.david@ens-lyon.org>
parents: 31652
diff changeset
990 for this parameter or the unbundling will be aborted.
6d055cd6669f bundle2: add documention to 'part.addparams'
Pierre-Yves David <pierre-yves.david@ens-lyon.org>
parents: 31652
diff changeset
991
6d055cd6669f bundle2: add documention to 'part.addparams'
Pierre-Yves David <pierre-yves.david@ens-lyon.org>
parents: 31652
diff changeset
992 The 'name' and 'value' cannot exceed 255 bytes each.
6d055cd6669f bundle2: add documention to 'part.addparams'
Pierre-Yves David <pierre-yves.david@ens-lyon.org>
parents: 31652
diff changeset
993 """
21605
f9dabfceb259 bundle2: introduce a ``addparam`` method on part
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 21604
diff changeset
994 if self._generated is not None:
21618
7568f5c1c801 bundle2: move exception classes into the error module
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 21617
diff changeset
995 raise error.ReadOnlyPartError('part is being generated')
21607
054fa5176fa7 bundle2: forbid duplicate parameter keys
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 21606
diff changeset
996 if name in self._seenparams:
054fa5176fa7 bundle2: forbid duplicate parameter keys
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 21606
diff changeset
997 raise ValueError('duplicated params: %s' % name)
054fa5176fa7 bundle2: forbid duplicate parameter keys
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 21606
diff changeset
998 self._seenparams.add(name)
21605
f9dabfceb259 bundle2: introduce a ``addparam`` method on part
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 21604
diff changeset
999 params = self._advisoryparams
f9dabfceb259 bundle2: introduce a ``addparam`` method on part
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 21604
diff changeset
1000 if mandatory:
f9dabfceb259 bundle2: introduce a ``addparam`` method on part
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 21604
diff changeset
1001 params = self._mandatoryparams
f9dabfceb259 bundle2: introduce a ``addparam`` method on part
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 21604
diff changeset
1002 params.append((name, value))
f9dabfceb259 bundle2: introduce a ``addparam`` method on part
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 21604
diff changeset
1003
21601
7ff01befc7ec bundle2: track life cycle of parts
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 21600
diff changeset
1004 # methods used to generates the bundle2 stream
25321
b44ee346211f bundle2: add debug output for part generation
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 25320
diff changeset
1005 def getchunks(self, ui):
21601
7ff01befc7ec bundle2: track life cycle of parts
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 21600
diff changeset
1006 if self._generated is not None:
31652
4dbef666c6c9 bundle2: use ProgrammingError
Jun Wu <quark@fb.com>
parents: 31483
diff changeset
1007 raise error.ProgrammingError('part can only be consumed once')
21601
7ff01befc7ec bundle2: track life cycle of parts
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 21600
diff changeset
1008 self._generated = False
25323
21a25fb81d2c bundle2: add generic debug output regarding generated parts
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 25322
diff changeset
1009
21a25fb81d2c bundle2: add generic debug output regarding generated parts
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 25322
diff changeset
1010 if ui.debugflag:
21a25fb81d2c bundle2: add generic debug output regarding generated parts
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 25322
diff changeset
1011 msg = ['bundle2-output-part: "%s"' % self.type]
21a25fb81d2c bundle2: add generic debug output regarding generated parts
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 25322
diff changeset
1012 if not self.mandatory:
21a25fb81d2c bundle2: add generic debug output regarding generated parts
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 25322
diff changeset
1013 msg.append(' (advisory)')
21a25fb81d2c bundle2: add generic debug output regarding generated parts
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 25322
diff changeset
1014 nbmp = len(self.mandatoryparams)
21a25fb81d2c bundle2: add generic debug output regarding generated parts
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 25322
diff changeset
1015 nbap = len(self.advisoryparams)
21a25fb81d2c bundle2: add generic debug output regarding generated parts
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 25322
diff changeset
1016 if nbmp or nbap:
21a25fb81d2c bundle2: add generic debug output regarding generated parts
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 25322
diff changeset
1017 msg.append(' (params:')
21a25fb81d2c bundle2: add generic debug output regarding generated parts
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 25322
diff changeset
1018 if nbmp:
21a25fb81d2c bundle2: add generic debug output regarding generated parts
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 25322
diff changeset
1019 msg.append(' %i mandatory' % nbmp)
21a25fb81d2c bundle2: add generic debug output regarding generated parts
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 25322
diff changeset
1020 if nbap:
21a25fb81d2c bundle2: add generic debug output regarding generated parts
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 25322
diff changeset
1021 msg.append(' %i advisory' % nbmp)
21a25fb81d2c bundle2: add generic debug output regarding generated parts
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 25322
diff changeset
1022 msg.append(')')
21a25fb81d2c bundle2: add generic debug output regarding generated parts
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 25322
diff changeset
1023 if not self.data:
21a25fb81d2c bundle2: add generic debug output regarding generated parts
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 25322
diff changeset
1024 msg.append(' empty payload')
34285
ea7588c11c36 bundle2: update check for a generator to work on Python 3
Augie Fackler <augie@google.com>
parents: 34284
diff changeset
1025 elif (util.safehasattr(self.data, 'next')
ea7588c11c36 bundle2: update check for a generator to work on Python 3
Augie Fackler <augie@google.com>
parents: 34284
diff changeset
1026 or util.safehasattr(self.data, '__next__')):
25323
21a25fb81d2c bundle2: add generic debug output regarding generated parts
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 25322
diff changeset
1027 msg.append(' streamed payload')
21a25fb81d2c bundle2: add generic debug output regarding generated parts
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 25322
diff changeset
1028 else:
21a25fb81d2c bundle2: add generic debug output regarding generated parts
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 25322
diff changeset
1029 msg.append(' %i bytes payload' % len(self.data))
21a25fb81d2c bundle2: add generic debug output regarding generated parts
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 25322
diff changeset
1030 msg.append('\n')
21a25fb81d2c bundle2: add generic debug output regarding generated parts
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 25322
diff changeset
1031 ui.debug(''.join(msg))
21a25fb81d2c bundle2: add generic debug output regarding generated parts
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 25322
diff changeset
1032
20877
9e9e3a4e9261 bundle2: part params
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 20876
diff changeset
1033 #### header
23590
4440c7cc3728 bundle2.bundlepart: make mandatory part flag explicit in API
Eric Sumner <ericsumner@fb.com>
parents: 23586
diff changeset
1034 if self.mandatory:
4440c7cc3728 bundle2.bundlepart: make mandatory part flag explicit in API
Eric Sumner <ericsumner@fb.com>
parents: 23586
diff changeset
1035 parttype = self.type.upper()
4440c7cc3728 bundle2.bundlepart: make mandatory part flag explicit in API
Eric Sumner <ericsumner@fb.com>
parents: 23586
diff changeset
1036 else:
4440c7cc3728 bundle2.bundlepart: make mandatory part flag explicit in API
Eric Sumner <ericsumner@fb.com>
parents: 23586
diff changeset
1037 parttype = self.type.lower()
33737
373ca5103ba8 bundle2: convert ints to strings using pycompat.bytestring()
Augie Fackler <augie@google.com>
parents: 33719
diff changeset
1038 outdebug(ui, 'part %s: "%s"' % (pycompat.bytestr(self.id), parttype))
20877
9e9e3a4e9261 bundle2: part params
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 20876
diff changeset
1039 ## parttype
23590
4440c7cc3728 bundle2.bundlepart: make mandatory part flag explicit in API
Eric Sumner <ericsumner@fb.com>
parents: 23586
diff changeset
1040 header = [_pack(_fparttypesize, len(parttype)),
4440c7cc3728 bundle2.bundlepart: make mandatory part flag explicit in API
Eric Sumner <ericsumner@fb.com>
parents: 23586
diff changeset
1041 parttype, _pack(_fpartid, self.id),
20856
8a6a86c9a5b5 bundle2: support bundling of empty part (with a type)
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 20844
diff changeset
1042 ]
20877
9e9e3a4e9261 bundle2: part params
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 20876
diff changeset
1043 ## parameters
9e9e3a4e9261 bundle2: part params
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 20876
diff changeset
1044 # count
9e9e3a4e9261 bundle2: part params
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 20876
diff changeset
1045 manpar = self.mandatoryparams
9e9e3a4e9261 bundle2: part params
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 20876
diff changeset
1046 advpar = self.advisoryparams
9e9e3a4e9261 bundle2: part params
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 20876
diff changeset
1047 header.append(_pack(_fpartparamcount, len(manpar), len(advpar)))
9e9e3a4e9261 bundle2: part params
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 20876
diff changeset
1048 # size
9e9e3a4e9261 bundle2: part params
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 20876
diff changeset
1049 parsizes = []
9e9e3a4e9261 bundle2: part params
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 20876
diff changeset
1050 for key, value in manpar:
9e9e3a4e9261 bundle2: part params
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 20876
diff changeset
1051 parsizes.append(len(key))
9e9e3a4e9261 bundle2: part params
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 20876
diff changeset
1052 parsizes.append(len(value))
9e9e3a4e9261 bundle2: part params
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 20876
diff changeset
1053 for key, value in advpar:
9e9e3a4e9261 bundle2: part params
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 20876
diff changeset
1054 parsizes.append(len(key))
9e9e3a4e9261 bundle2: part params
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 20876
diff changeset
1055 parsizes.append(len(value))
33698
da7c285ec6da bundle2: use modern Python division
Augie Fackler <augie@google.com>
parents: 33693
diff changeset
1056 paramsizes = _pack(_makefpartparamsizes(len(parsizes) // 2), *parsizes)
20877
9e9e3a4e9261 bundle2: part params
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 20876
diff changeset
1057 header.append(paramsizes)
9e9e3a4e9261 bundle2: part params
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 20876
diff changeset
1058 # key, value
9e9e3a4e9261 bundle2: part params
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 20876
diff changeset
1059 for key, value in manpar:
9e9e3a4e9261 bundle2: part params
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 20876
diff changeset
1060 header.append(key)
9e9e3a4e9261 bundle2: part params
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 20876
diff changeset
1061 header.append(value)
9e9e3a4e9261 bundle2: part params
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 20876
diff changeset
1062 for key, value in advpar:
9e9e3a4e9261 bundle2: part params
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 20876
diff changeset
1063 header.append(key)
9e9e3a4e9261 bundle2: part params
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 20876
diff changeset
1064 header.append(value)
9e9e3a4e9261 bundle2: part params
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 20876
diff changeset
1065 ## finalize header
34256
ab379eed2e31 bundle2: raise a more helpful error if building a bundle part header fails
Augie Fackler <raf@durin42.com>
parents: 34228
diff changeset
1066 try:
ab379eed2e31 bundle2: raise a more helpful error if building a bundle part header fails
Augie Fackler <raf@durin42.com>
parents: 34228
diff changeset
1067 headerchunk = ''.join(header)
ab379eed2e31 bundle2: raise a more helpful error if building a bundle part header fails
Augie Fackler <raf@durin42.com>
parents: 34228
diff changeset
1068 except TypeError:
ab379eed2e31 bundle2: raise a more helpful error if building a bundle part header fails
Augie Fackler <raf@durin42.com>
parents: 34228
diff changeset
1069 raise TypeError(r'Found a non-bytes trying to '
ab379eed2e31 bundle2: raise a more helpful error if building a bundle part header fails
Augie Fackler <raf@durin42.com>
parents: 34228
diff changeset
1070 r'build bundle part header: %r' % header)
25321
b44ee346211f bundle2: add debug output for part generation
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 25320
diff changeset
1071 outdebug(ui, 'header chunk size: %i' % len(headerchunk))
20856
8a6a86c9a5b5 bundle2: support bundling of empty part (with a type)
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 20844
diff changeset
1072 yield _pack(_fpartheadersize, len(headerchunk))
8a6a86c9a5b5 bundle2: support bundling of empty part (with a type)
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 20844
diff changeset
1073 yield headerchunk
20877
9e9e3a4e9261 bundle2: part params
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 20876
diff changeset
1074 ## payload
23067
420a051616ce bundle2: transmit exception during part generation
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 23066
diff changeset
1075 try:
420a051616ce bundle2: transmit exception during part generation
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 23066
diff changeset
1076 for chunk in self._payloadchunks():
25321
b44ee346211f bundle2: add debug output for part generation
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 25320
diff changeset
1077 outdebug(ui, 'payload chunk size: %i' % len(chunk))
23067
420a051616ce bundle2: transmit exception during part generation
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 23066
diff changeset
1078 yield _pack(_fpayloadsize, len(chunk))
420a051616ce bundle2: transmit exception during part generation
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 23066
diff changeset
1079 yield chunk
26144
4bc3707f3e67 bundle2: don't try to recover from a GeneratorExit (issue4785)
Augie Fackler <augie@google.com>
parents: 25919
diff changeset
1080 except GeneratorExit:
4bc3707f3e67 bundle2: don't try to recover from a GeneratorExit (issue4785)
Augie Fackler <augie@google.com>
parents: 25919
diff changeset
1081 # GeneratorExit means that nobody is listening for our
4bc3707f3e67 bundle2: don't try to recover from a GeneratorExit (issue4785)
Augie Fackler <augie@google.com>
parents: 25919
diff changeset
1082 # results anyway, so just bail quickly rather than trying
4bc3707f3e67 bundle2: don't try to recover from a GeneratorExit (issue4785)
Augie Fackler <augie@google.com>
parents: 25919
diff changeset
1083 # to produce an error part.
4bc3707f3e67 bundle2: don't try to recover from a GeneratorExit (issue4785)
Augie Fackler <augie@google.com>
parents: 25919
diff changeset
1084 ui.debug('bundle2-generatorexit\n')
4bc3707f3e67 bundle2: don't try to recover from a GeneratorExit (issue4785)
Augie Fackler <augie@google.com>
parents: 25919
diff changeset
1085 raise
25660
328739ea70c3 global: mass rewrite to use modern exception syntax
Gregory Szorc <gregory.szorc@gmail.com>
parents: 25641
diff changeset
1086 except BaseException as exc:
33708
1d5e497c08b3 py3: convert arbitrary exception object to byte string more reliably
Yuya Nishihara <yuya@tcha.org>
parents: 33702
diff changeset
1087 bexc = util.forcebytestr(exc)
23067
420a051616ce bundle2: transmit exception during part generation
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 23066
diff changeset
1088 # backup exception data for later
25324
deed9c6b12d2 bundle2: add generic debug output regarding generated interruption
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 25323
diff changeset
1089 ui.debug('bundle2-input-stream-interrupt: encoding exception %s'
33702
9f862fe7a8bd bundle2: obtain repr() of exception in a python3-safe way
Augie Fackler <augie@google.com>
parents: 33701
diff changeset
1090 % bexc)
32232
76f9a0009b4b pycompat: extract helper to raise exception with traceback
Yuya Nishihara <yuya@tcha.org>
parents: 32024
diff changeset
1091 tb = sys.exc_info()[2]
33702
9f862fe7a8bd bundle2: obtain repr() of exception in a python3-safe way
Augie Fackler <augie@google.com>
parents: 33701
diff changeset
1092 msg = 'unexpected error: %s' % bexc
24686
e0e28e910fa3 bundle2: rename format, parts and config to final names
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 24648
diff changeset
1093 interpart = bundlepart('error:abort', [('message', msg)],
23590
4440c7cc3728 bundle2.bundlepart: make mandatory part flag explicit in API
Eric Sumner <ericsumner@fb.com>
parents: 23586
diff changeset
1094 mandatory=False)
23067
420a051616ce bundle2: transmit exception during part generation
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 23066
diff changeset
1095 interpart.id = 0
420a051616ce bundle2: transmit exception during part generation
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 23066
diff changeset
1096 yield _pack(_fpayloadsize, -1)
25321
b44ee346211f bundle2: add debug output for part generation
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 25320
diff changeset
1097 for chunk in interpart.getchunks(ui=ui):
23067
420a051616ce bundle2: transmit exception during part generation
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 23066
diff changeset
1098 yield chunk
25321
b44ee346211f bundle2: add debug output for part generation
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 25320
diff changeset
1099 outdebug(ui, 'closing payload chunk')
23067
420a051616ce bundle2: transmit exception during part generation
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 23066
diff changeset
1100 # abort current part payload
420a051616ce bundle2: transmit exception during part generation
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 23066
diff changeset
1101 yield _pack(_fpayloadsize, 0)
32232
76f9a0009b4b pycompat: extract helper to raise exception with traceback
Yuya Nishihara <yuya@tcha.org>
parents: 32024
diff changeset
1102 pycompat.raisewithtb(exc, tb)
21000
4cae06ae1562 bundle2: extract a _payloadchunks method for part
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 20998
diff changeset
1103 # end of payload
25321
b44ee346211f bundle2: add debug output for part generation
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 25320
diff changeset
1104 outdebug(ui, 'closing payload chunk')
21000
4cae06ae1562 bundle2: extract a _payloadchunks method for part
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 20998
diff changeset
1105 yield _pack(_fpayloadsize, 0)
21601
7ff01befc7ec bundle2: track life cycle of parts
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 21600
diff changeset
1106 self._generated = True
21000
4cae06ae1562 bundle2: extract a _payloadchunks method for part
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 20998
diff changeset
1107
4cae06ae1562 bundle2: extract a _payloadchunks method for part
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 20998
diff changeset
1108 def _payloadchunks(self):
4cae06ae1562 bundle2: extract a _payloadchunks method for part
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 20998
diff changeset
1109 """yield chunks of a the part payload
4cae06ae1562 bundle2: extract a _payloadchunks method for part
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 20998
diff changeset
1110
4cae06ae1562 bundle2: extract a _payloadchunks method for part
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 20998
diff changeset
1111 Exists to handle the different methods to provide data to a part."""
20876
ddd56f3eb786 bundle2: support for bundling and unbundling payload
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 20864
diff changeset
1112 # we only support fixed size data now.
ddd56f3eb786 bundle2: support for bundling and unbundling payload
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 20864
diff changeset
1113 # This will be improved in the future.
33699
5ae35a1347fd bundle2: look for __next__ as well as next to identify iterators
Augie Fackler <augie@google.com>
parents: 33698
diff changeset
1114 if (util.safehasattr(self.data, 'next')
5ae35a1347fd bundle2: look for __next__ as well as next to identify iterators
Augie Fackler <augie@google.com>
parents: 33698
diff changeset
1115 or util.safehasattr(self.data, '__next__')):
21001
c93bb6a08fa1 bundle2: support chunk iterator as part data
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 21000
diff changeset
1116 buff = util.chunkbuffer(self.data)
c93bb6a08fa1 bundle2: support chunk iterator as part data
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 21000
diff changeset
1117 chunk = buff.read(preferedchunksize)
c93bb6a08fa1 bundle2: support chunk iterator as part data
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 21000
diff changeset
1118 while chunk:
c93bb6a08fa1 bundle2: support chunk iterator as part data
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 21000
diff changeset
1119 yield chunk
c93bb6a08fa1 bundle2: support chunk iterator as part data
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 21000
diff changeset
1120 chunk = buff.read(preferedchunksize)
c93bb6a08fa1 bundle2: support chunk iterator as part data
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 21000
diff changeset
1121 elif len(self.data):
20876
ddd56f3eb786 bundle2: support for bundling and unbundling payload
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 20864
diff changeset
1122 yield self.data
20802
520df53ad26a bundle2: a very first version of bundle2 unbundler
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 20801
diff changeset
1123
23066
ad144882318d bundle2: add a interrupt mechanism
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 23029
diff changeset
1124
ad144882318d bundle2: add a interrupt mechanism
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 23029
diff changeset
1125 flaginterrupt = -1
ad144882318d bundle2: add a interrupt mechanism
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 23029
diff changeset
1126
ad144882318d bundle2: add a interrupt mechanism
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 23029
diff changeset
1127 class interrupthandler(unpackermixin):
ad144882318d bundle2: add a interrupt mechanism
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 23029
diff changeset
1128 """read one part and process it with restricted capability
ad144882318d bundle2: add a interrupt mechanism
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 23029
diff changeset
1129
ad144882318d bundle2: add a interrupt mechanism
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 23029
diff changeset
1130 This allows to transmit exception raised on the producer size during part
ad144882318d bundle2: add a interrupt mechanism
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 23029
diff changeset
1131 iteration while the consumer is reading a part.
ad144882318d bundle2: add a interrupt mechanism
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 23029
diff changeset
1132
ad144882318d bundle2: add a interrupt mechanism
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 23029
diff changeset
1133 Part processed in this manner only have access to a ui object,"""
ad144882318d bundle2: add a interrupt mechanism
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 23029
diff changeset
1134
ad144882318d bundle2: add a interrupt mechanism
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 23029
diff changeset
1135 def __init__(self, ui, fp):
ad144882318d bundle2: add a interrupt mechanism
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 23029
diff changeset
1136 super(interrupthandler, self).__init__(fp)
ad144882318d bundle2: add a interrupt mechanism
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 23029
diff changeset
1137 self.ui = ui
ad144882318d bundle2: add a interrupt mechanism
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 23029
diff changeset
1138
ad144882318d bundle2: add a interrupt mechanism
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 23029
diff changeset
1139 def _readpartheader(self):
ad144882318d bundle2: add a interrupt mechanism
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 23029
diff changeset
1140 """reads a part header size and return the bytes blob
ad144882318d bundle2: add a interrupt mechanism
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 23029
diff changeset
1141
ad144882318d bundle2: add a interrupt mechanism
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 23029
diff changeset
1142 returns None if empty"""
ad144882318d bundle2: add a interrupt mechanism
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 23029
diff changeset
1143 headersize = self._unpack(_fpartheadersize)[0]
ad144882318d bundle2: add a interrupt mechanism
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 23029
diff changeset
1144 if headersize < 0:
ad144882318d bundle2: add a interrupt mechanism
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 23029
diff changeset
1145 raise error.BundleValueError('negative part header size: %i'
ad144882318d bundle2: add a interrupt mechanism
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 23029
diff changeset
1146 % headersize)
25318
64dd8f25fcf4 bundle2: introduce a specific function for debug messages while unbundling
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 25317
diff changeset
1147 indebug(self.ui, 'part header size: %i\n' % headersize)
23066
ad144882318d bundle2: add a interrupt mechanism
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 23029
diff changeset
1148 if headersize:
ad144882318d bundle2: add a interrupt mechanism
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 23029
diff changeset
1149 return self._readexact(headersize)
ad144882318d bundle2: add a interrupt mechanism
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 23029
diff changeset
1150 return None
ad144882318d bundle2: add a interrupt mechanism
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 23029
diff changeset
1151
ad144882318d bundle2: add a interrupt mechanism
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 23029
diff changeset
1152 def __call__(self):
25335
8f7137a85a0e bundle2: add generic debug output regarding processed interruption
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 25334
diff changeset
1153
8f7137a85a0e bundle2: add generic debug output regarding processed interruption
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 25334
diff changeset
1154 self.ui.debug('bundle2-input-stream-interrupt:'
8f7137a85a0e bundle2: add generic debug output regarding processed interruption
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 25334
diff changeset
1155 ' opening out of band context\n')
25320
697d8953b04d bundle2: handle new line in 'indebug' function
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 25319
diff changeset
1156 indebug(self.ui, 'bundle2 stream interruption, looking for a part.')
23066
ad144882318d bundle2: add a interrupt mechanism
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 23029
diff changeset
1157 headerblock = self._readpartheader()
ad144882318d bundle2: add a interrupt mechanism
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 23029
diff changeset
1158 if headerblock is None:
25320
697d8953b04d bundle2: handle new line in 'indebug' function
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 25319
diff changeset
1159 indebug(self.ui, 'no part found during interruption.')
23066
ad144882318d bundle2: add a interrupt mechanism
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 23029
diff changeset
1160 return
35137
da91e7309daf bundle2: don't use seekable bundle2 parts by default (issue5691)
Gregory Szorc <gregory.szorc@gmail.com>
parents: 35136
diff changeset
1161 part = unbundlepart(self.ui, headerblock, self._fp)
23066
ad144882318d bundle2: add a interrupt mechanism
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 23029
diff changeset
1162 op = interruptoperation(self.ui)
34265
e71890f27767 bundle2: move processpart stream maintenance into part iterator
Durham Goode <durham@fb.com>
parents: 34256
diff changeset
1163 hardabort = False
e71890f27767 bundle2: move processpart stream maintenance into part iterator
Durham Goode <durham@fb.com>
parents: 34256
diff changeset
1164 try:
e71890f27767 bundle2: move processpart stream maintenance into part iterator
Durham Goode <durham@fb.com>
parents: 34256
diff changeset
1165 _processpart(op, part)
e71890f27767 bundle2: move processpart stream maintenance into part iterator
Durham Goode <durham@fb.com>
parents: 34256
diff changeset
1166 except (SystemExit, KeyboardInterrupt):
e71890f27767 bundle2: move processpart stream maintenance into part iterator
Durham Goode <durham@fb.com>
parents: 34256
diff changeset
1167 hardabort = True
e71890f27767 bundle2: move processpart stream maintenance into part iterator
Durham Goode <durham@fb.com>
parents: 34256
diff changeset
1168 raise
e71890f27767 bundle2: move processpart stream maintenance into part iterator
Durham Goode <durham@fb.com>
parents: 34256
diff changeset
1169 finally:
e71890f27767 bundle2: move processpart stream maintenance into part iterator
Durham Goode <durham@fb.com>
parents: 34256
diff changeset
1170 if not hardabort:
35135
db5038525718 bundle2: implement consume() API on unbundlepart
Gregory Szorc <gregory.szorc@gmail.com>
parents: 35134
diff changeset
1171 part.consume()
25335
8f7137a85a0e bundle2: add generic debug output regarding processed interruption
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 25334
diff changeset
1172 self.ui.debug('bundle2-input-stream-interrupt:'
8f7137a85a0e bundle2: add generic debug output regarding processed interruption
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 25334
diff changeset
1173 ' closing out of band context\n')
23066
ad144882318d bundle2: add a interrupt mechanism
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 23029
diff changeset
1174
ad144882318d bundle2: add a interrupt mechanism
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 23029
diff changeset
1175 class interruptoperation(object):
ad144882318d bundle2: add a interrupt mechanism
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 23029
diff changeset
1176 """A limited operation to be use by part handler during interruption
ad144882318d bundle2: add a interrupt mechanism
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 23029
diff changeset
1177
ad144882318d bundle2: add a interrupt mechanism
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 23029
diff changeset
1178 It only have access to an ui object.
ad144882318d bundle2: add a interrupt mechanism
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 23029
diff changeset
1179 """
ad144882318d bundle2: add a interrupt mechanism
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 23029
diff changeset
1180
ad144882318d bundle2: add a interrupt mechanism
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 23029
diff changeset
1181 def __init__(self, ui):
ad144882318d bundle2: add a interrupt mechanism
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 23029
diff changeset
1182 self.ui = ui
ad144882318d bundle2: add a interrupt mechanism
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 23029
diff changeset
1183 self.reply = None
24878
e530cde6d115 bundle2: disable ouput capture unless we use http (issue4613 issue4615)
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 24851
diff changeset
1184 self.captureoutput = False
23066
ad144882318d bundle2: add a interrupt mechanism
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 23029
diff changeset
1185
ad144882318d bundle2: add a interrupt mechanism
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 23029
diff changeset
1186 @property
ad144882318d bundle2: add a interrupt mechanism
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 23029
diff changeset
1187 def repo(self):
31652
4dbef666c6c9 bundle2: use ProgrammingError
Jun Wu <quark@fb.com>
parents: 31483
diff changeset
1188 raise error.ProgrammingError('no repo access from stream interruption')
23066
ad144882318d bundle2: add a interrupt mechanism
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 23029
diff changeset
1189
ad144882318d bundle2: add a interrupt mechanism
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 23029
diff changeset
1190 def gettransaction(self):
ad144882318d bundle2: add a interrupt mechanism
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 23029
diff changeset
1191 raise TransactionUnavailable('no repo access from stream interruption')
ad144882318d bundle2: add a interrupt mechanism
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 23029
diff changeset
1192
35134
8aa43ff9c12c bundle2: implement generic part payload decoder
Gregory Szorc <gregory.szorc@gmail.com>
parents: 35133
diff changeset
1193 def decodepayloadchunks(ui, fh):
8aa43ff9c12c bundle2: implement generic part payload decoder
Gregory Szorc <gregory.szorc@gmail.com>
parents: 35133
diff changeset
1194 """Reads bundle2 part payload data into chunks.
8aa43ff9c12c bundle2: implement generic part payload decoder
Gregory Szorc <gregory.szorc@gmail.com>
parents: 35133
diff changeset
1195
8aa43ff9c12c bundle2: implement generic part payload decoder
Gregory Szorc <gregory.szorc@gmail.com>
parents: 35133
diff changeset
1196 Part payload data consists of framed chunks. This function takes
8aa43ff9c12c bundle2: implement generic part payload decoder
Gregory Szorc <gregory.szorc@gmail.com>
parents: 35133
diff changeset
1197 a file handle and emits those chunks.
8aa43ff9c12c bundle2: implement generic part payload decoder
Gregory Szorc <gregory.szorc@gmail.com>
parents: 35133
diff changeset
1198 """
35138
589eed45a452 bundle2: inline debug logging
Gregory Szorc <gregory.szorc@gmail.com>
parents: 35137
diff changeset
1199 dolog = ui.configbool('devel', 'bundle2.debug')
589eed45a452 bundle2: inline debug logging
Gregory Szorc <gregory.szorc@gmail.com>
parents: 35137
diff changeset
1200 debug = ui.debug
589eed45a452 bundle2: inline debug logging
Gregory Szorc <gregory.szorc@gmail.com>
parents: 35137
diff changeset
1201
35140
764e3ad1cf54 bundle2: inline struct operations
Gregory Szorc <gregory.szorc@gmail.com>
parents: 35139
diff changeset
1202 headerstruct = struct.Struct(_fpayloadsize)
764e3ad1cf54 bundle2: inline struct operations
Gregory Szorc <gregory.szorc@gmail.com>
parents: 35139
diff changeset
1203 headersize = headerstruct.size
764e3ad1cf54 bundle2: inline struct operations
Gregory Szorc <gregory.szorc@gmail.com>
parents: 35139
diff changeset
1204 unpack = headerstruct.unpack
764e3ad1cf54 bundle2: inline struct operations
Gregory Szorc <gregory.szorc@gmail.com>
parents: 35139
diff changeset
1205
35134
8aa43ff9c12c bundle2: implement generic part payload decoder
Gregory Szorc <gregory.szorc@gmail.com>
parents: 35133
diff changeset
1206 readexactly = changegroup.readexactly
35139
1fb0846ad792 bundle2: inline changegroup.readexactly()
Gregory Szorc <gregory.szorc@gmail.com>
parents: 35138
diff changeset
1207 read = fh.read
35134
8aa43ff9c12c bundle2: implement generic part payload decoder
Gregory Szorc <gregory.szorc@gmail.com>
parents: 35133
diff changeset
1208
35140
764e3ad1cf54 bundle2: inline struct operations
Gregory Szorc <gregory.szorc@gmail.com>
parents: 35139
diff changeset
1209 chunksize = unpack(readexactly(fh, headersize))[0]
35134
8aa43ff9c12c bundle2: implement generic part payload decoder
Gregory Szorc <gregory.szorc@gmail.com>
parents: 35133
diff changeset
1210 indebug(ui, 'payload chunk size: %i' % chunksize)
8aa43ff9c12c bundle2: implement generic part payload decoder
Gregory Szorc <gregory.szorc@gmail.com>
parents: 35133
diff changeset
1211
35139
1fb0846ad792 bundle2: inline changegroup.readexactly()
Gregory Szorc <gregory.szorc@gmail.com>
parents: 35138
diff changeset
1212 # changegroup.readexactly() is inlined below for performance.
35134
8aa43ff9c12c bundle2: implement generic part payload decoder
Gregory Szorc <gregory.szorc@gmail.com>
parents: 35133
diff changeset
1213 while chunksize:
8aa43ff9c12c bundle2: implement generic part payload decoder
Gregory Szorc <gregory.szorc@gmail.com>
parents: 35133
diff changeset
1214 if chunksize >= 0:
35139
1fb0846ad792 bundle2: inline changegroup.readexactly()
Gregory Szorc <gregory.szorc@gmail.com>
parents: 35138
diff changeset
1215 s = read(chunksize)
1fb0846ad792 bundle2: inline changegroup.readexactly()
Gregory Szorc <gregory.szorc@gmail.com>
parents: 35138
diff changeset
1216 if len(s) < chunksize:
1fb0846ad792 bundle2: inline changegroup.readexactly()
Gregory Szorc <gregory.szorc@gmail.com>
parents: 35138
diff changeset
1217 raise error.Abort(_('stream ended unexpectedly '
1fb0846ad792 bundle2: inline changegroup.readexactly()
Gregory Szorc <gregory.szorc@gmail.com>
parents: 35138
diff changeset
1218 ' (got %d bytes, expected %d)') %
1fb0846ad792 bundle2: inline changegroup.readexactly()
Gregory Szorc <gregory.szorc@gmail.com>
parents: 35138
diff changeset
1219 (len(s), chunksize))
1fb0846ad792 bundle2: inline changegroup.readexactly()
Gregory Szorc <gregory.szorc@gmail.com>
parents: 35138
diff changeset
1220
1fb0846ad792 bundle2: inline changegroup.readexactly()
Gregory Szorc <gregory.szorc@gmail.com>
parents: 35138
diff changeset
1221 yield s
35134
8aa43ff9c12c bundle2: implement generic part payload decoder
Gregory Szorc <gregory.szorc@gmail.com>
parents: 35133
diff changeset
1222 elif chunksize == flaginterrupt:
8aa43ff9c12c bundle2: implement generic part payload decoder
Gregory Szorc <gregory.szorc@gmail.com>
parents: 35133
diff changeset
1223 # Interrupt "signal" detected. The regular stream is interrupted
8aa43ff9c12c bundle2: implement generic part payload decoder
Gregory Szorc <gregory.szorc@gmail.com>
parents: 35133
diff changeset
1224 # and a bundle2 part follows. Consume it.
8aa43ff9c12c bundle2: implement generic part payload decoder
Gregory Szorc <gregory.szorc@gmail.com>
parents: 35133
diff changeset
1225 interrupthandler(ui, fh)()
8aa43ff9c12c bundle2: implement generic part payload decoder
Gregory Szorc <gregory.szorc@gmail.com>
parents: 35133
diff changeset
1226 else:
8aa43ff9c12c bundle2: implement generic part payload decoder
Gregory Szorc <gregory.szorc@gmail.com>
parents: 35133
diff changeset
1227 raise error.BundleValueError(
8aa43ff9c12c bundle2: implement generic part payload decoder
Gregory Szorc <gregory.szorc@gmail.com>
parents: 35133
diff changeset
1228 'negative payload chunk size: %s' % chunksize)
8aa43ff9c12c bundle2: implement generic part payload decoder
Gregory Szorc <gregory.szorc@gmail.com>
parents: 35133
diff changeset
1229
35139
1fb0846ad792 bundle2: inline changegroup.readexactly()
Gregory Szorc <gregory.szorc@gmail.com>
parents: 35138
diff changeset
1230 s = read(headersize)
1fb0846ad792 bundle2: inline changegroup.readexactly()
Gregory Szorc <gregory.szorc@gmail.com>
parents: 35138
diff changeset
1231 if len(s) < headersize:
1fb0846ad792 bundle2: inline changegroup.readexactly()
Gregory Szorc <gregory.szorc@gmail.com>
parents: 35138
diff changeset
1232 raise error.Abort(_('stream ended unexpectedly '
1fb0846ad792 bundle2: inline changegroup.readexactly()
Gregory Szorc <gregory.szorc@gmail.com>
parents: 35138
diff changeset
1233 ' (got %d bytes, expected %d)') %
1fb0846ad792 bundle2: inline changegroup.readexactly()
Gregory Szorc <gregory.szorc@gmail.com>
parents: 35138
diff changeset
1234 (len(s), chunksize))
1fb0846ad792 bundle2: inline changegroup.readexactly()
Gregory Szorc <gregory.szorc@gmail.com>
parents: 35138
diff changeset
1235
35140
764e3ad1cf54 bundle2: inline struct operations
Gregory Szorc <gregory.szorc@gmail.com>
parents: 35139
diff changeset
1236 chunksize = unpack(s)[0]
35138
589eed45a452 bundle2: inline debug logging
Gregory Szorc <gregory.szorc@gmail.com>
parents: 35137
diff changeset
1237
589eed45a452 bundle2: inline debug logging
Gregory Szorc <gregory.szorc@gmail.com>
parents: 35137
diff changeset
1238 # indebug() inlined for performance.
589eed45a452 bundle2: inline debug logging
Gregory Szorc <gregory.szorc@gmail.com>
parents: 35137
diff changeset
1239 if dolog:
589eed45a452 bundle2: inline debug logging
Gregory Szorc <gregory.szorc@gmail.com>
parents: 35137
diff changeset
1240 debug('bundle2-input: payload chunk size: %i\n' % chunksize)
35134
8aa43ff9c12c bundle2: implement generic part payload decoder
Gregory Szorc <gregory.szorc@gmail.com>
parents: 35133
diff changeset
1241
21014
a6246bba7b9e bundle2: add an unbundle part responsible from unbundling part
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 21013
diff changeset
1242 class unbundlepart(unpackermixin):
a6246bba7b9e bundle2: add an unbundle part responsible from unbundling part
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 21013
diff changeset
1243 """a bundle part read from a bundle"""
a6246bba7b9e bundle2: add an unbundle part responsible from unbundling part
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 21013
diff changeset
1244
a6246bba7b9e bundle2: add an unbundle part responsible from unbundling part
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 21013
diff changeset
1245 def __init__(self, ui, header, fp):
a6246bba7b9e bundle2: add an unbundle part responsible from unbundling part
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 21013
diff changeset
1246 super(unbundlepart, self).__init__(fp)
31889
a02e773008f5 bundle2: move 'seek' and 'tell' methods off the unpackermixin class
Pierre-Yves David <pierre-yves.david@ens-lyon.org>
parents: 31863
diff changeset
1247 self._seekable = (util.safehasattr(fp, 'seek') and
a02e773008f5 bundle2: move 'seek' and 'tell' methods off the unpackermixin class
Pierre-Yves David <pierre-yves.david@ens-lyon.org>
parents: 31863
diff changeset
1248 util.safehasattr(fp, 'tell'))
21014
a6246bba7b9e bundle2: add an unbundle part responsible from unbundling part
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 21013
diff changeset
1249 self.ui = ui
a6246bba7b9e bundle2: add an unbundle part responsible from unbundling part
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 21013
diff changeset
1250 # unbundle state attr
a6246bba7b9e bundle2: add an unbundle part responsible from unbundling part
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 21013
diff changeset
1251 self._headerdata = header
21015
14dd49260246 bundle2: move the fromheader closure into the class itself
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 21014
diff changeset
1252 self._headeroffset = 0
21019
3dc09f831a2e bundle2: lazy unbundle of part payload
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 21016
diff changeset
1253 self._initialized = False
3dc09f831a2e bundle2: lazy unbundle of part payload
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 21016
diff changeset
1254 self.consumed = False
21014
a6246bba7b9e bundle2: add an unbundle part responsible from unbundling part
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 21013
diff changeset
1255 # part data
a6246bba7b9e bundle2: add an unbundle part responsible from unbundling part
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 21013
diff changeset
1256 self.id = None
a6246bba7b9e bundle2: add an unbundle part responsible from unbundling part
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 21013
diff changeset
1257 self.type = None
a6246bba7b9e bundle2: add an unbundle part responsible from unbundling part
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 21013
diff changeset
1258 self.mandatoryparams = None
a6246bba7b9e bundle2: add an unbundle part responsible from unbundling part
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 21013
diff changeset
1259 self.advisoryparams = None
21610
d6056805f8f4 bundle2: introduce a ``params`` dictionary on unbundled parts
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 21609
diff changeset
1260 self.params = None
21612
f221eb0531d9 bundle2: expose mandatory params in a mandatorykeys attribute
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 21611
diff changeset
1261 self.mandatorykeys = ()
21019
3dc09f831a2e bundle2: lazy unbundle of part payload
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 21016
diff changeset
1262 self._readheader()
23585
94b25d71dd0f bundle2.unbundlepart: decouple mandatory from parttype
Eric Sumner <ericsumner@fb.com>
parents: 23439
diff changeset
1263 self._mandatory = None
24036
c7601086338a bundle2.unbundlepart: tell() implementation
Eric Sumner <ericsumner@fb.com>
parents: 24035
diff changeset
1264 self._pos = 0
21014
a6246bba7b9e bundle2: add an unbundle part responsible from unbundling part
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 21013
diff changeset
1265
21015
14dd49260246 bundle2: move the fromheader closure into the class itself
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 21014
diff changeset
1266 def _fromheader(self, size):
14dd49260246 bundle2: move the fromheader closure into the class itself
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 21014
diff changeset
1267 """return the next <size> byte from the header"""
14dd49260246 bundle2: move the fromheader closure into the class itself
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 21014
diff changeset
1268 offset = self._headeroffset
14dd49260246 bundle2: move the fromheader closure into the class itself
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 21014
diff changeset
1269 data = self._headerdata[offset:(offset + size)]
21019
3dc09f831a2e bundle2: lazy unbundle of part payload
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 21016
diff changeset
1270 self._headeroffset = offset + size
21015
14dd49260246 bundle2: move the fromheader closure into the class itself
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 21014
diff changeset
1271 return data
14dd49260246 bundle2: move the fromheader closure into the class itself
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 21014
diff changeset
1272
21016
b477afb1c81e bundle2: move unpackheader closure into the class
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 21015
diff changeset
1273 def _unpackheader(self, format):
b477afb1c81e bundle2: move unpackheader closure into the class
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 21015
diff changeset
1274 """read given format from header
b477afb1c81e bundle2: move unpackheader closure into the class
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 21015
diff changeset
1275
b477afb1c81e bundle2: move unpackheader closure into the class
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 21015
diff changeset
1276 This automatically compute the size of the format to read."""
b477afb1c81e bundle2: move unpackheader closure into the class
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 21015
diff changeset
1277 data = self._fromheader(struct.calcsize(format))
b477afb1c81e bundle2: move unpackheader closure into the class
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 21015
diff changeset
1278 return _unpack(format, data)
b477afb1c81e bundle2: move unpackheader closure into the class
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 21015
diff changeset
1279
21608
3cb96ca90c17 bundle2: introduce an ``_initparams`` method
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 21607
diff changeset
1280 def _initparams(self, mandatoryparams, advisoryparams):
3cb96ca90c17 bundle2: introduce an ``_initparams`` method
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 21607
diff changeset
1281 """internal function to setup all logic related parameters"""
21609
63cc2594ef8a bundle2: make sure unbundled part param are read-only
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 21608
diff changeset
1282 # make it read only to prevent people touching it by mistake.
63cc2594ef8a bundle2: make sure unbundled part param are read-only
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 21608
diff changeset
1283 self.mandatoryparams = tuple(mandatoryparams)
63cc2594ef8a bundle2: make sure unbundled part param are read-only
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 21608
diff changeset
1284 self.advisoryparams = tuple(advisoryparams)
21610
d6056805f8f4 bundle2: introduce a ``params`` dictionary on unbundled parts
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 21609
diff changeset
1285 # user friendly UI
29591
6215b5537ba5 bundle2: use a sorted dict for holding parameters
Gregory Szorc <gregory.szorc@gmail.com>
parents: 29294
diff changeset
1286 self.params = util.sortdict(self.mandatoryparams)
6215b5537ba5 bundle2: use a sorted dict for holding parameters
Gregory Szorc <gregory.szorc@gmail.com>
parents: 29294
diff changeset
1287 self.params.update(self.advisoryparams)
21612
f221eb0531d9 bundle2: expose mandatory params in a mandatorykeys attribute
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 21611
diff changeset
1288 self.mandatorykeys = frozenset(p[0] for p in mandatoryparams)
21608
3cb96ca90c17 bundle2: introduce an ``_initparams`` method
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 21607
diff changeset
1289
21019
3dc09f831a2e bundle2: lazy unbundle of part payload
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 21016
diff changeset
1290 def _readheader(self):
21014
a6246bba7b9e bundle2: add an unbundle part responsible from unbundling part
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 21013
diff changeset
1291 """read the header and setup the object"""
21016
b477afb1c81e bundle2: move unpackheader closure into the class
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 21015
diff changeset
1292 typesize = self._unpackheader(_fparttypesize)[0]
21015
14dd49260246 bundle2: move the fromheader closure into the class itself
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 21014
diff changeset
1293 self.type = self._fromheader(typesize)
25320
697d8953b04d bundle2: handle new line in 'indebug' function
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 25319
diff changeset
1294 indebug(self.ui, 'part type: "%s"' % self.type)
21016
b477afb1c81e bundle2: move unpackheader closure into the class
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 21015
diff changeset
1295 self.id = self._unpackheader(_fpartid)[0]
33737
373ca5103ba8 bundle2: convert ints to strings using pycompat.bytestring()
Augie Fackler <augie@google.com>
parents: 33719
diff changeset
1296 indebug(self.ui, 'part id: "%s"' % pycompat.bytestr(self.id))
23585
94b25d71dd0f bundle2.unbundlepart: decouple mandatory from parttype
Eric Sumner <ericsumner@fb.com>
parents: 23439
diff changeset
1297 # extract mandatory bit from type
94b25d71dd0f bundle2.unbundlepart: decouple mandatory from parttype
Eric Sumner <ericsumner@fb.com>
parents: 23439
diff changeset
1298 self.mandatory = (self.type != self.type.lower())
94b25d71dd0f bundle2.unbundlepart: decouple mandatory from parttype
Eric Sumner <ericsumner@fb.com>
parents: 23439
diff changeset
1299 self.type = self.type.lower()
21014
a6246bba7b9e bundle2: add an unbundle part responsible from unbundling part
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 21013
diff changeset
1300 ## reading parameters
a6246bba7b9e bundle2: add an unbundle part responsible from unbundling part
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 21013
diff changeset
1301 # param count
21016
b477afb1c81e bundle2: move unpackheader closure into the class
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 21015
diff changeset
1302 mancount, advcount = self._unpackheader(_fpartparamcount)
25320
697d8953b04d bundle2: handle new line in 'indebug' function
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 25319
diff changeset
1303 indebug(self.ui, 'part parameters: %i' % (mancount + advcount))
21014
a6246bba7b9e bundle2: add an unbundle part responsible from unbundling part
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 21013
diff changeset
1304 # param size
21016
b477afb1c81e bundle2: move unpackheader closure into the class
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 21015
diff changeset
1305 fparamsizes = _makefpartparamsizes(mancount + advcount)
b477afb1c81e bundle2: move unpackheader closure into the class
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 21015
diff changeset
1306 paramsizes = self._unpackheader(fparamsizes)
21014
a6246bba7b9e bundle2: add an unbundle part responsible from unbundling part
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 21013
diff changeset
1307 # make it a list of couple again
33700
9a323773216c bundle2: work around zip() being lazy in Python 3
Augie Fackler <augie@google.com>
parents: 33699
diff changeset
1308 paramsizes = list(zip(paramsizes[::2], paramsizes[1::2]))
21014
a6246bba7b9e bundle2: add an unbundle part responsible from unbundling part
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 21013
diff changeset
1309 # split mandatory from advisory
a6246bba7b9e bundle2: add an unbundle part responsible from unbundling part
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 21013
diff changeset
1310 mansizes = paramsizes[:mancount]
a6246bba7b9e bundle2: add an unbundle part responsible from unbundling part
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 21013
diff changeset
1311 advsizes = paramsizes[mancount:]
23139
e53f6b72a0e4 spelling: fixes from proofreading of spell checker issues
Mads Kiilerich <madski@unity3d.com>
parents: 23067
diff changeset
1312 # retrieve param value
21014
a6246bba7b9e bundle2: add an unbundle part responsible from unbundling part
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 21013
diff changeset
1313 manparams = []
a6246bba7b9e bundle2: add an unbundle part responsible from unbundling part
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 21013
diff changeset
1314 for key, value in mansizes:
21015
14dd49260246 bundle2: move the fromheader closure into the class itself
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 21014
diff changeset
1315 manparams.append((self._fromheader(key), self._fromheader(value)))
21014
a6246bba7b9e bundle2: add an unbundle part responsible from unbundling part
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 21013
diff changeset
1316 advparams = []
a6246bba7b9e bundle2: add an unbundle part responsible from unbundling part
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 21013
diff changeset
1317 for key, value in advsizes:
21015
14dd49260246 bundle2: move the fromheader closure into the class itself
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 21014
diff changeset
1318 advparams.append((self._fromheader(key), self._fromheader(value)))
21608
3cb96ca90c17 bundle2: introduce an ``_initparams`` method
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 21607
diff changeset
1319 self._initparams(manparams, advparams)
21014
a6246bba7b9e bundle2: add an unbundle part responsible from unbundling part
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 21013
diff changeset
1320 ## part payload
24034
9881a1437799 bundle2.unbundlepart: raise payloadchunks from a closure to a method
Eric Sumner <ericsumner@fb.com>
parents: 24026
diff changeset
1321 self._payloadstream = util.chunkbuffer(self._payloadchunks())
21019
3dc09f831a2e bundle2: lazy unbundle of part payload
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 21016
diff changeset
1322 # we read the data, tell it
3dc09f831a2e bundle2: lazy unbundle of part payload
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 21016
diff changeset
1323 self._initialized = True
3dc09f831a2e bundle2: lazy unbundle of part payload
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 21016
diff changeset
1324
35134
8aa43ff9c12c bundle2: implement generic part payload decoder
Gregory Szorc <gregory.szorc@gmail.com>
parents: 35133
diff changeset
1325 def _payloadchunks(self):
8aa43ff9c12c bundle2: implement generic part payload decoder
Gregory Szorc <gregory.szorc@gmail.com>
parents: 35133
diff changeset
1326 """Generator of decoded chunks in the payload."""
8aa43ff9c12c bundle2: implement generic part payload decoder
Gregory Szorc <gregory.szorc@gmail.com>
parents: 35133
diff changeset
1327 return decodepayloadchunks(self.ui, self._fp)
8aa43ff9c12c bundle2: implement generic part payload decoder
Gregory Szorc <gregory.szorc@gmail.com>
parents: 35133
diff changeset
1328
35135
db5038525718 bundle2: implement consume() API on unbundlepart
Gregory Szorc <gregory.szorc@gmail.com>
parents: 35134
diff changeset
1329 def consume(self):
db5038525718 bundle2: implement consume() API on unbundlepart
Gregory Szorc <gregory.szorc@gmail.com>
parents: 35134
diff changeset
1330 """Read the part payload until completion.
db5038525718 bundle2: implement consume() API on unbundlepart
Gregory Szorc <gregory.szorc@gmail.com>
parents: 35134
diff changeset
1331
db5038525718 bundle2: implement consume() API on unbundlepart
Gregory Szorc <gregory.szorc@gmail.com>
parents: 35134
diff changeset
1332 By consuming the part data, the underlying stream read offset will
db5038525718 bundle2: implement consume() API on unbundlepart
Gregory Szorc <gregory.szorc@gmail.com>
parents: 35134
diff changeset
1333 be advanced to the next part (or end of stream).
db5038525718 bundle2: implement consume() API on unbundlepart
Gregory Szorc <gregory.szorc@gmail.com>
parents: 35134
diff changeset
1334 """
db5038525718 bundle2: implement consume() API on unbundlepart
Gregory Szorc <gregory.szorc@gmail.com>
parents: 35134
diff changeset
1335 if self.consumed:
db5038525718 bundle2: implement consume() API on unbundlepart
Gregory Szorc <gregory.szorc@gmail.com>
parents: 35134
diff changeset
1336 return
db5038525718 bundle2: implement consume() API on unbundlepart
Gregory Szorc <gregory.szorc@gmail.com>
parents: 35134
diff changeset
1337
db5038525718 bundle2: implement consume() API on unbundlepart
Gregory Szorc <gregory.szorc@gmail.com>
parents: 35134
diff changeset
1338 chunk = self.read(32768)
db5038525718 bundle2: implement consume() API on unbundlepart
Gregory Szorc <gregory.szorc@gmail.com>
parents: 35134
diff changeset
1339 while chunk:
db5038525718 bundle2: implement consume() API on unbundlepart
Gregory Szorc <gregory.szorc@gmail.com>
parents: 35134
diff changeset
1340 self._pos += len(chunk)
db5038525718 bundle2: implement consume() API on unbundlepart
Gregory Szorc <gregory.szorc@gmail.com>
parents: 35134
diff changeset
1341 chunk = self.read(32768)
db5038525718 bundle2: implement consume() API on unbundlepart
Gregory Szorc <gregory.szorc@gmail.com>
parents: 35134
diff changeset
1342
21019
3dc09f831a2e bundle2: lazy unbundle of part payload
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 21016
diff changeset
1343 def read(self, size=None):
3dc09f831a2e bundle2: lazy unbundle of part payload
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 21016
diff changeset
1344 """read payload data"""
3dc09f831a2e bundle2: lazy unbundle of part payload
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 21016
diff changeset
1345 if not self._initialized:
3dc09f831a2e bundle2: lazy unbundle of part payload
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 21016
diff changeset
1346 self._readheader()
3dc09f831a2e bundle2: lazy unbundle of part payload
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 21016
diff changeset
1347 if size is None:
3dc09f831a2e bundle2: lazy unbundle of part payload
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 21016
diff changeset
1348 data = self._payloadstream.read()
3dc09f831a2e bundle2: lazy unbundle of part payload
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 21016
diff changeset
1349 else:
3dc09f831a2e bundle2: lazy unbundle of part payload
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 21016
diff changeset
1350 data = self._payloadstream.read(size)
25334
76cba1ecf55f bundle2: add generic debug output regarding processed part payload
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 25333
diff changeset
1351 self._pos += len(data)
21019
3dc09f831a2e bundle2: lazy unbundle of part payload
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 21016
diff changeset
1352 if size is None or len(data) < size:
25334
76cba1ecf55f bundle2: add generic debug output regarding processed part payload
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 25333
diff changeset
1353 if not self.consumed and self._pos:
76cba1ecf55f bundle2: add generic debug output regarding processed part payload
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 25333
diff changeset
1354 self.ui.debug('bundle2-input-part: total payload size %i\n'
76cba1ecf55f bundle2: add generic debug output regarding processed part payload
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 25333
diff changeset
1355 % self._pos)
21019
3dc09f831a2e bundle2: lazy unbundle of part payload
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 21016
diff changeset
1356 self.consumed = True
3dc09f831a2e bundle2: lazy unbundle of part payload
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 21016
diff changeset
1357 return data
3dc09f831a2e bundle2: lazy unbundle of part payload
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 21016
diff changeset
1358
35133
073eec083e25 bundle2: extract logic for seeking bundle2 part into own class
Gregory Szorc <gregory.szorc@gmail.com>
parents: 35069
diff changeset
1359 class seekableunbundlepart(unbundlepart):
073eec083e25 bundle2: extract logic for seeking bundle2 part into own class
Gregory Szorc <gregory.szorc@gmail.com>
parents: 35069
diff changeset
1360 """A bundle2 part in a bundle that is seekable.
073eec083e25 bundle2: extract logic for seeking bundle2 part into own class
Gregory Szorc <gregory.szorc@gmail.com>
parents: 35069
diff changeset
1361
073eec083e25 bundle2: extract logic for seeking bundle2 part into own class
Gregory Szorc <gregory.szorc@gmail.com>
parents: 35069
diff changeset
1362 Regular ``unbundlepart`` instances can only be read once. This class
073eec083e25 bundle2: extract logic for seeking bundle2 part into own class
Gregory Szorc <gregory.szorc@gmail.com>
parents: 35069
diff changeset
1363 extends ``unbundlepart`` to enable bi-directional seeking within the
073eec083e25 bundle2: extract logic for seeking bundle2 part into own class
Gregory Szorc <gregory.szorc@gmail.com>
parents: 35069
diff changeset
1364 part.
073eec083e25 bundle2: extract logic for seeking bundle2 part into own class
Gregory Szorc <gregory.szorc@gmail.com>
parents: 35069
diff changeset
1365
073eec083e25 bundle2: extract logic for seeking bundle2 part into own class
Gregory Szorc <gregory.szorc@gmail.com>
parents: 35069
diff changeset
1366 Bundle2 part data consists of framed chunks. Offsets when seeking
073eec083e25 bundle2: extract logic for seeking bundle2 part into own class
Gregory Szorc <gregory.szorc@gmail.com>
parents: 35069
diff changeset
1367 refer to the decoded data, not the offsets in the underlying bundle2
073eec083e25 bundle2: extract logic for seeking bundle2 part into own class
Gregory Szorc <gregory.szorc@gmail.com>
parents: 35069
diff changeset
1368 stream.
073eec083e25 bundle2: extract logic for seeking bundle2 part into own class
Gregory Szorc <gregory.szorc@gmail.com>
parents: 35069
diff changeset
1369
073eec083e25 bundle2: extract logic for seeking bundle2 part into own class
Gregory Szorc <gregory.szorc@gmail.com>
parents: 35069
diff changeset
1370 To facilitate quickly seeking within the decoded data, instances of this
073eec083e25 bundle2: extract logic for seeking bundle2 part into own class
Gregory Szorc <gregory.szorc@gmail.com>
parents: 35069
diff changeset
1371 class maintain a mapping between offsets in the underlying stream and
073eec083e25 bundle2: extract logic for seeking bundle2 part into own class
Gregory Szorc <gregory.szorc@gmail.com>
parents: 35069
diff changeset
1372 the decoded payload. This mapping will consume memory in proportion
073eec083e25 bundle2: extract logic for seeking bundle2 part into own class
Gregory Szorc <gregory.szorc@gmail.com>
parents: 35069
diff changeset
1373 to the number of chunks within the payload (which almost certainly
073eec083e25 bundle2: extract logic for seeking bundle2 part into own class
Gregory Szorc <gregory.szorc@gmail.com>
parents: 35069
diff changeset
1374 increases in proportion with the size of the part).
073eec083e25 bundle2: extract logic for seeking bundle2 part into own class
Gregory Szorc <gregory.szorc@gmail.com>
parents: 35069
diff changeset
1375 """
073eec083e25 bundle2: extract logic for seeking bundle2 part into own class
Gregory Szorc <gregory.szorc@gmail.com>
parents: 35069
diff changeset
1376 def __init__(self, ui, header, fp):
073eec083e25 bundle2: extract logic for seeking bundle2 part into own class
Gregory Szorc <gregory.szorc@gmail.com>
parents: 35069
diff changeset
1377 # (payload, file) offsets for chunk starts.
073eec083e25 bundle2: extract logic for seeking bundle2 part into own class
Gregory Szorc <gregory.szorc@gmail.com>
parents: 35069
diff changeset
1378 self._chunkindex = []
073eec083e25 bundle2: extract logic for seeking bundle2 part into own class
Gregory Szorc <gregory.szorc@gmail.com>
parents: 35069
diff changeset
1379
073eec083e25 bundle2: extract logic for seeking bundle2 part into own class
Gregory Szorc <gregory.szorc@gmail.com>
parents: 35069
diff changeset
1380 super(seekableunbundlepart, self).__init__(ui, header, fp)
073eec083e25 bundle2: extract logic for seeking bundle2 part into own class
Gregory Szorc <gregory.szorc@gmail.com>
parents: 35069
diff changeset
1381
073eec083e25 bundle2: extract logic for seeking bundle2 part into own class
Gregory Szorc <gregory.szorc@gmail.com>
parents: 35069
diff changeset
1382 def _payloadchunks(self, chunknum=0):
073eec083e25 bundle2: extract logic for seeking bundle2 part into own class
Gregory Szorc <gregory.szorc@gmail.com>
parents: 35069
diff changeset
1383 '''seek to specified chunk and start yielding data'''
073eec083e25 bundle2: extract logic for seeking bundle2 part into own class
Gregory Szorc <gregory.szorc@gmail.com>
parents: 35069
diff changeset
1384 if len(self._chunkindex) == 0:
073eec083e25 bundle2: extract logic for seeking bundle2 part into own class
Gregory Szorc <gregory.szorc@gmail.com>
parents: 35069
diff changeset
1385 assert chunknum == 0, 'Must start with chunk 0'
073eec083e25 bundle2: extract logic for seeking bundle2 part into own class
Gregory Szorc <gregory.szorc@gmail.com>
parents: 35069
diff changeset
1386 self._chunkindex.append((0, self._tellfp()))
073eec083e25 bundle2: extract logic for seeking bundle2 part into own class
Gregory Szorc <gregory.szorc@gmail.com>
parents: 35069
diff changeset
1387 else:
073eec083e25 bundle2: extract logic for seeking bundle2 part into own class
Gregory Szorc <gregory.szorc@gmail.com>
parents: 35069
diff changeset
1388 assert chunknum < len(self._chunkindex), \
073eec083e25 bundle2: extract logic for seeking bundle2 part into own class
Gregory Szorc <gregory.szorc@gmail.com>
parents: 35069
diff changeset
1389 'Unknown chunk %d' % chunknum
073eec083e25 bundle2: extract logic for seeking bundle2 part into own class
Gregory Szorc <gregory.szorc@gmail.com>
parents: 35069
diff changeset
1390 self._seekfp(self._chunkindex[chunknum][1])
073eec083e25 bundle2: extract logic for seeking bundle2 part into own class
Gregory Szorc <gregory.szorc@gmail.com>
parents: 35069
diff changeset
1391
073eec083e25 bundle2: extract logic for seeking bundle2 part into own class
Gregory Szorc <gregory.szorc@gmail.com>
parents: 35069
diff changeset
1392 pos = self._chunkindex[chunknum][0]
35134
8aa43ff9c12c bundle2: implement generic part payload decoder
Gregory Szorc <gregory.szorc@gmail.com>
parents: 35133
diff changeset
1393
8aa43ff9c12c bundle2: implement generic part payload decoder
Gregory Szorc <gregory.szorc@gmail.com>
parents: 35133
diff changeset
1394 for chunk in decodepayloadchunks(self.ui, self._fp):
8aa43ff9c12c bundle2: implement generic part payload decoder
Gregory Szorc <gregory.szorc@gmail.com>
parents: 35133
diff changeset
1395 chunknum += 1
8aa43ff9c12c bundle2: implement generic part payload decoder
Gregory Szorc <gregory.szorc@gmail.com>
parents: 35133
diff changeset
1396 pos += len(chunk)
8aa43ff9c12c bundle2: implement generic part payload decoder
Gregory Szorc <gregory.szorc@gmail.com>
parents: 35133
diff changeset
1397 if chunknum == len(self._chunkindex):
8aa43ff9c12c bundle2: implement generic part payload decoder
Gregory Szorc <gregory.szorc@gmail.com>
parents: 35133
diff changeset
1398 self._chunkindex.append((pos, self._tellfp()))
8aa43ff9c12c bundle2: implement generic part payload decoder
Gregory Szorc <gregory.szorc@gmail.com>
parents: 35133
diff changeset
1399
8aa43ff9c12c bundle2: implement generic part payload decoder
Gregory Szorc <gregory.szorc@gmail.com>
parents: 35133
diff changeset
1400 yield chunk
35133
073eec083e25 bundle2: extract logic for seeking bundle2 part into own class
Gregory Szorc <gregory.szorc@gmail.com>
parents: 35069
diff changeset
1401
073eec083e25 bundle2: extract logic for seeking bundle2 part into own class
Gregory Szorc <gregory.szorc@gmail.com>
parents: 35069
diff changeset
1402 def _findchunk(self, pos):
073eec083e25 bundle2: extract logic for seeking bundle2 part into own class
Gregory Szorc <gregory.szorc@gmail.com>
parents: 35069
diff changeset
1403 '''for a given payload position, return a chunk number and offset'''
073eec083e25 bundle2: extract logic for seeking bundle2 part into own class
Gregory Szorc <gregory.szorc@gmail.com>
parents: 35069
diff changeset
1404 for chunk, (ppos, fpos) in enumerate(self._chunkindex):
073eec083e25 bundle2: extract logic for seeking bundle2 part into own class
Gregory Szorc <gregory.szorc@gmail.com>
parents: 35069
diff changeset
1405 if ppos == pos:
073eec083e25 bundle2: extract logic for seeking bundle2 part into own class
Gregory Szorc <gregory.szorc@gmail.com>
parents: 35069
diff changeset
1406 return chunk, 0
073eec083e25 bundle2: extract logic for seeking bundle2 part into own class
Gregory Szorc <gregory.szorc@gmail.com>
parents: 35069
diff changeset
1407 elif ppos > pos:
073eec083e25 bundle2: extract logic for seeking bundle2 part into own class
Gregory Szorc <gregory.szorc@gmail.com>
parents: 35069
diff changeset
1408 return chunk - 1, pos - self._chunkindex[chunk - 1][0]
073eec083e25 bundle2: extract logic for seeking bundle2 part into own class
Gregory Szorc <gregory.szorc@gmail.com>
parents: 35069
diff changeset
1409 raise ValueError('Unknown chunk')
073eec083e25 bundle2: extract logic for seeking bundle2 part into own class
Gregory Szorc <gregory.szorc@gmail.com>
parents: 35069
diff changeset
1410
24036
c7601086338a bundle2.unbundlepart: tell() implementation
Eric Sumner <ericsumner@fb.com>
parents: 24035
diff changeset
1411 def tell(self):
c7601086338a bundle2.unbundlepart: tell() implementation
Eric Sumner <ericsumner@fb.com>
parents: 24035
diff changeset
1412 return self._pos
c7601086338a bundle2.unbundlepart: tell() implementation
Eric Sumner <ericsumner@fb.com>
parents: 24035
diff changeset
1413
35069
241d9caca11e bundle2: use os.SEEK_* constants
Gregory Szorc <gregory.szorc@gmail.com>
parents: 34820
diff changeset
1414 def seek(self, offset, whence=os.SEEK_SET):
241d9caca11e bundle2: use os.SEEK_* constants
Gregory Szorc <gregory.szorc@gmail.com>
parents: 34820
diff changeset
1415 if whence == os.SEEK_SET:
24037
f0b498cfc5c8 bundle2.unbundlepart: implement seek()
Eric Sumner <ericsumner@fb.com>
parents: 24036
diff changeset
1416 newpos = offset
35069
241d9caca11e bundle2: use os.SEEK_* constants
Gregory Szorc <gregory.szorc@gmail.com>
parents: 34820
diff changeset
1417 elif whence == os.SEEK_CUR:
24037
f0b498cfc5c8 bundle2.unbundlepart: implement seek()
Eric Sumner <ericsumner@fb.com>
parents: 24036
diff changeset
1418 newpos = self._pos + offset
35069
241d9caca11e bundle2: use os.SEEK_* constants
Gregory Szorc <gregory.szorc@gmail.com>
parents: 34820
diff changeset
1419 elif whence == os.SEEK_END:
24037
f0b498cfc5c8 bundle2.unbundlepart: implement seek()
Eric Sumner <ericsumner@fb.com>
parents: 24036
diff changeset
1420 if not self.consumed:
35141
699b2a759319 bundle2: avoid unbound read when seeking
Gregory Szorc <gregory.szorc@gmail.com>
parents: 35140
diff changeset
1421 # Can't use self.consume() here because it advances self._pos.
699b2a759319 bundle2: avoid unbound read when seeking
Gregory Szorc <gregory.szorc@gmail.com>
parents: 35140
diff changeset
1422 chunk = self.read(32768)
699b2a759319 bundle2: avoid unbound read when seeking
Gregory Szorc <gregory.szorc@gmail.com>
parents: 35140
diff changeset
1423 while chunk:
699b2a759319 bundle2: avoid unbound read when seeking
Gregory Szorc <gregory.szorc@gmail.com>
parents: 35140
diff changeset
1424 chunk = self.read(32768)
24037
f0b498cfc5c8 bundle2.unbundlepart: implement seek()
Eric Sumner <ericsumner@fb.com>
parents: 24036
diff changeset
1425 newpos = self._chunkindex[-1][0] - offset
f0b498cfc5c8 bundle2.unbundlepart: implement seek()
Eric Sumner <ericsumner@fb.com>
parents: 24036
diff changeset
1426 else:
f0b498cfc5c8 bundle2.unbundlepart: implement seek()
Eric Sumner <ericsumner@fb.com>
parents: 24036
diff changeset
1427 raise ValueError('Unknown whence value: %r' % (whence,))
f0b498cfc5c8 bundle2.unbundlepart: implement seek()
Eric Sumner <ericsumner@fb.com>
parents: 24036
diff changeset
1428
f0b498cfc5c8 bundle2.unbundlepart: implement seek()
Eric Sumner <ericsumner@fb.com>
parents: 24036
diff changeset
1429 if newpos > self._chunkindex[-1][0] and not self.consumed:
35141
699b2a759319 bundle2: avoid unbound read when seeking
Gregory Szorc <gregory.szorc@gmail.com>
parents: 35140
diff changeset
1430 # Can't use self.consume() here because it advances self._pos.
699b2a759319 bundle2: avoid unbound read when seeking
Gregory Szorc <gregory.szorc@gmail.com>
parents: 35140
diff changeset
1431 chunk = self.read(32768)
699b2a759319 bundle2: avoid unbound read when seeking
Gregory Szorc <gregory.szorc@gmail.com>
parents: 35140
diff changeset
1432 while chunk:
699b2a759319 bundle2: avoid unbound read when seeking
Gregory Szorc <gregory.szorc@gmail.com>
parents: 35140
diff changeset
1433 chunk = self.read(32668)
699b2a759319 bundle2: avoid unbound read when seeking
Gregory Szorc <gregory.szorc@gmail.com>
parents: 35140
diff changeset
1434
24037
f0b498cfc5c8 bundle2.unbundlepart: implement seek()
Eric Sumner <ericsumner@fb.com>
parents: 24036
diff changeset
1435 if not 0 <= newpos <= self._chunkindex[-1][0]:
f0b498cfc5c8 bundle2.unbundlepart: implement seek()
Eric Sumner <ericsumner@fb.com>
parents: 24036
diff changeset
1436 raise ValueError('Offset out of range')
f0b498cfc5c8 bundle2.unbundlepart: implement seek()
Eric Sumner <ericsumner@fb.com>
parents: 24036
diff changeset
1437
f0b498cfc5c8 bundle2.unbundlepart: implement seek()
Eric Sumner <ericsumner@fb.com>
parents: 24036
diff changeset
1438 if self._pos != newpos:
f0b498cfc5c8 bundle2.unbundlepart: implement seek()
Eric Sumner <ericsumner@fb.com>
parents: 24036
diff changeset
1439 chunk, internaloffset = self._findchunk(newpos)
f0b498cfc5c8 bundle2.unbundlepart: implement seek()
Eric Sumner <ericsumner@fb.com>
parents: 24036
diff changeset
1440 self._payloadstream = util.chunkbuffer(self._payloadchunks(chunk))
f0b498cfc5c8 bundle2.unbundlepart: implement seek()
Eric Sumner <ericsumner@fb.com>
parents: 24036
diff changeset
1441 adjust = self.read(internaloffset)
f0b498cfc5c8 bundle2.unbundlepart: implement seek()
Eric Sumner <ericsumner@fb.com>
parents: 24036
diff changeset
1442 if len(adjust) != internaloffset:
26587
56b2bcea2529 error: get Abort from 'error' instead of 'util'
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 26565
diff changeset
1443 raise error.Abort(_('Seek failed\n'))
24037
f0b498cfc5c8 bundle2.unbundlepart: implement seek()
Eric Sumner <ericsumner@fb.com>
parents: 24036
diff changeset
1444 self._pos = newpos
f0b498cfc5c8 bundle2.unbundlepart: implement seek()
Eric Sumner <ericsumner@fb.com>
parents: 24036
diff changeset
1445
31889
a02e773008f5 bundle2: move 'seek' and 'tell' methods off the unpackermixin class
Pierre-Yves David <pierre-yves.david@ens-lyon.org>
parents: 31863
diff changeset
1446 def _seekfp(self, offset, whence=0):
a02e773008f5 bundle2: move 'seek' and 'tell' methods off the unpackermixin class
Pierre-Yves David <pierre-yves.david@ens-lyon.org>
parents: 31863
diff changeset
1447 """move the underlying file pointer
a02e773008f5 bundle2: move 'seek' and 'tell' methods off the unpackermixin class
Pierre-Yves David <pierre-yves.david@ens-lyon.org>
parents: 31863
diff changeset
1448
a02e773008f5 bundle2: move 'seek' and 'tell' methods off the unpackermixin class
Pierre-Yves David <pierre-yves.david@ens-lyon.org>
parents: 31863
diff changeset
1449 This method is meant for internal usage by the bundle2 protocol only.
a02e773008f5 bundle2: move 'seek' and 'tell' methods off the unpackermixin class
Pierre-Yves David <pierre-yves.david@ens-lyon.org>
parents: 31863
diff changeset
1450 They directly manipulate the low level stream including bundle2 level
a02e773008f5 bundle2: move 'seek' and 'tell' methods off the unpackermixin class
Pierre-Yves David <pierre-yves.david@ens-lyon.org>
parents: 31863
diff changeset
1451 instruction.
a02e773008f5 bundle2: move 'seek' and 'tell' methods off the unpackermixin class
Pierre-Yves David <pierre-yves.david@ens-lyon.org>
parents: 31863
diff changeset
1452
a02e773008f5 bundle2: move 'seek' and 'tell' methods off the unpackermixin class
Pierre-Yves David <pierre-yves.david@ens-lyon.org>
parents: 31863
diff changeset
1453 Do not use it to implement higher-level logic or methods."""
a02e773008f5 bundle2: move 'seek' and 'tell' methods off the unpackermixin class
Pierre-Yves David <pierre-yves.david@ens-lyon.org>
parents: 31863
diff changeset
1454 if self._seekable:
a02e773008f5 bundle2: move 'seek' and 'tell' methods off the unpackermixin class
Pierre-Yves David <pierre-yves.david@ens-lyon.org>
parents: 31863
diff changeset
1455 return self._fp.seek(offset, whence)
a02e773008f5 bundle2: move 'seek' and 'tell' methods off the unpackermixin class
Pierre-Yves David <pierre-yves.david@ens-lyon.org>
parents: 31863
diff changeset
1456 else:
a02e773008f5 bundle2: move 'seek' and 'tell' methods off the unpackermixin class
Pierre-Yves David <pierre-yves.david@ens-lyon.org>
parents: 31863
diff changeset
1457 raise NotImplementedError(_('File pointer is not seekable'))
a02e773008f5 bundle2: move 'seek' and 'tell' methods off the unpackermixin class
Pierre-Yves David <pierre-yves.david@ens-lyon.org>
parents: 31863
diff changeset
1458
a02e773008f5 bundle2: move 'seek' and 'tell' methods off the unpackermixin class
Pierre-Yves David <pierre-yves.david@ens-lyon.org>
parents: 31863
diff changeset
1459 def _tellfp(self):
a02e773008f5 bundle2: move 'seek' and 'tell' methods off the unpackermixin class
Pierre-Yves David <pierre-yves.david@ens-lyon.org>
parents: 31863
diff changeset
1460 """return the file offset, or None if file is not seekable
a02e773008f5 bundle2: move 'seek' and 'tell' methods off the unpackermixin class
Pierre-Yves David <pierre-yves.david@ens-lyon.org>
parents: 31863
diff changeset
1461
a02e773008f5 bundle2: move 'seek' and 'tell' methods off the unpackermixin class
Pierre-Yves David <pierre-yves.david@ens-lyon.org>
parents: 31863
diff changeset
1462 This method is meant for internal usage by the bundle2 protocol only.
a02e773008f5 bundle2: move 'seek' and 'tell' methods off the unpackermixin class
Pierre-Yves David <pierre-yves.david@ens-lyon.org>
parents: 31863
diff changeset
1463 They directly manipulate the low level stream including bundle2 level
a02e773008f5 bundle2: move 'seek' and 'tell' methods off the unpackermixin class
Pierre-Yves David <pierre-yves.david@ens-lyon.org>
parents: 31863
diff changeset
1464 instruction.
a02e773008f5 bundle2: move 'seek' and 'tell' methods off the unpackermixin class
Pierre-Yves David <pierre-yves.david@ens-lyon.org>
parents: 31863
diff changeset
1465
a02e773008f5 bundle2: move 'seek' and 'tell' methods off the unpackermixin class
Pierre-Yves David <pierre-yves.david@ens-lyon.org>
parents: 31863
diff changeset
1466 Do not use it to implement higher-level logic or methods."""
a02e773008f5 bundle2: move 'seek' and 'tell' methods off the unpackermixin class
Pierre-Yves David <pierre-yves.david@ens-lyon.org>
parents: 31863
diff changeset
1467 if self._seekable:
a02e773008f5 bundle2: move 'seek' and 'tell' methods off the unpackermixin class
Pierre-Yves David <pierre-yves.david@ens-lyon.org>
parents: 31863
diff changeset
1468 try:
a02e773008f5 bundle2: move 'seek' and 'tell' methods off the unpackermixin class
Pierre-Yves David <pierre-yves.david@ens-lyon.org>
parents: 31863
diff changeset
1469 return self._fp.tell()
a02e773008f5 bundle2: move 'seek' and 'tell' methods off the unpackermixin class
Pierre-Yves David <pierre-yves.david@ens-lyon.org>
parents: 31863
diff changeset
1470 except IOError as e:
a02e773008f5 bundle2: move 'seek' and 'tell' methods off the unpackermixin class
Pierre-Yves David <pierre-yves.david@ens-lyon.org>
parents: 31863
diff changeset
1471 if e.errno == errno.ESPIPE:
a02e773008f5 bundle2: move 'seek' and 'tell' methods off the unpackermixin class
Pierre-Yves David <pierre-yves.david@ens-lyon.org>
parents: 31863
diff changeset
1472 self._seekable = False
a02e773008f5 bundle2: move 'seek' and 'tell' methods off the unpackermixin class
Pierre-Yves David <pierre-yves.david@ens-lyon.org>
parents: 31863
diff changeset
1473 else:
a02e773008f5 bundle2: move 'seek' and 'tell' methods off the unpackermixin class
Pierre-Yves David <pierre-yves.david@ens-lyon.org>
parents: 31863
diff changeset
1474 raise
a02e773008f5 bundle2: move 'seek' and 'tell' methods off the unpackermixin class
Pierre-Yves David <pierre-yves.david@ens-lyon.org>
parents: 31863
diff changeset
1475 return None
a02e773008f5 bundle2: move 'seek' and 'tell' methods off the unpackermixin class
Pierre-Yves David <pierre-yves.david@ens-lyon.org>
parents: 31863
diff changeset
1476
25317
5a5b7046d00e bundle2: add an informative comment to the capability dict
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 25315
diff changeset
1477 # These are only the static capabilities.
5a5b7046d00e bundle2: add an informative comment to the capability dict
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 25315
diff changeset
1478 # Check the 'getrepocaps' function for the rest.
24686
e0e28e910fa3 bundle2: rename format, parts and config to final names
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 24648
diff changeset
1479 capabilities = {'HG20': (),
35268
ad5f2b923b0d push: include a 'check:bookmarks' part when possible
Boris Feld <boris.feld@octobus.net>
parents: 35267
diff changeset
1480 'bookmarks': (),
25493
d8e7b0781ad7 bundle2: convey PushkeyFailed error over the wire
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 25492
diff changeset
1481 'error': ('abort', 'unsupportedcontent', 'pushraced',
d8e7b0781ad7 bundle2: convey PushkeyFailed error over the wire
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 25492
diff changeset
1482 'pushkey'),
24686
e0e28e910fa3 bundle2: rename format, parts and config to final names
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 24648
diff changeset
1483 'listkeys': (),
e0e28e910fa3 bundle2: rename format, parts and config to final names
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 24648
diff changeset
1484 'pushkey': (),
23029
149fc8a44184 bundle2: client side support for a part to import external bundles
Mike Hommey <mh@glandium.org>
parents: 23011
diff changeset
1485 'digests': tuple(sorted(util.DIGESTS.keys())),
24686
e0e28e910fa3 bundle2: rename format, parts and config to final names
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 24648
diff changeset
1486 'remote-changegroup': ('http', 'https'),
25401
d29201352af7 bundle2: part handler for processing .hgtags fnodes mappings
Gregory Szorc <gregory.szorc@gmail.com>
parents: 25336
diff changeset
1487 'hgtagsfnodes': (),
34329
10e162bb9bf5 pull: use 'phase-heads' to retrieve phase information
Boris Feld <boris.feld@octobus.net>
parents: 34328
diff changeset
1488 'phases': ('heads',),
22341
2d16b39601b5 obsmarker: move bundle2caps from the localrepo class to the bundle2 module
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 22340
diff changeset
1489 }
2d16b39601b5 obsmarker: move bundle2caps from the localrepo class to the bundle2 module
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 22340
diff changeset
1490
23439
743736fc7c41 bundle2-push: provide transaction to reply unbundler
Eric Sumner <ericsumner@fb.com>
parents: 23438
diff changeset
1491 def getrepocaps(repo, allowpushback=False):
22342
262c5cc126c1 bundle2: introduce a `getrepocaps` to retrieve the bundle2 caps of a repo
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 22341
diff changeset
1492 """return the bundle2 capabilities for a given repo
262c5cc126c1 bundle2: introduce a `getrepocaps` to retrieve the bundle2 caps of a repo
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 22341
diff changeset
1493
22343
76677a2c1cfd bundle2: advertise the obsmarker part in bundle2 capabilities
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 22342
diff changeset
1494 Exists to allow extensions (like evolution) to mutate the capabilities.
22342
262c5cc126c1 bundle2: introduce a `getrepocaps` to retrieve the bundle2 caps of a repo
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 22341
diff changeset
1495 """
22343
76677a2c1cfd bundle2: advertise the obsmarker part in bundle2 capabilities
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 22342
diff changeset
1496 caps = capabilities.copy()
27953
88609cfa3745 changegroup: fix pulling to treemanifest repo from flat repo (issue5066)
Martin von Zweigbergk <martinvonz@google.com>
parents: 27879
diff changeset
1497 caps['changegroup'] = tuple(sorted(
88609cfa3745 changegroup: fix pulling to treemanifest repo from flat repo (issue5066)
Martin von Zweigbergk <martinvonz@google.com>
parents: 27879
diff changeset
1498 changegroup.supportedincomingversions(repo)))
22953
b1d694d3975e obsolete: add exchange option
Durham Goode <durham@fb.com>
parents: 22661
diff changeset
1499 if obsolete.isenabled(repo, obsolete.exchangeopt):
22343
76677a2c1cfd bundle2: advertise the obsmarker part in bundle2 capabilities
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 22342
diff changeset
1500 supportedformat = tuple('V%i' % v for v in obsolete.formats)
24686
e0e28e910fa3 bundle2: rename format, parts and config to final names
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 24648
diff changeset
1501 caps['obsmarkers'] = supportedformat
23439
743736fc7c41 bundle2-push: provide transaction to reply unbundler
Eric Sumner <ericsumner@fb.com>
parents: 23438
diff changeset
1502 if allowpushback:
24686
e0e28e910fa3 bundle2: rename format, parts and config to final names
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 24648
diff changeset
1503 caps['pushback'] = ()
33222
593ad8df9dd2 configitems: register the 'server.concurrent-push-mode' config
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 33165
diff changeset
1504 cpmode = repo.ui.config('server', 'concurrent-push-mode')
32910
a7851519ea02 check-concurrency: expose the feature as 'concurrent-push-mode'
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 32729
diff changeset
1505 if cpmode == 'check-related':
32729
16ada4cbb1a9 push: add a way to allow concurrent pushes on unrelated heads
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 32549
diff changeset
1506 caps['checkheads'] = ('related',)
34329
10e162bb9bf5 pull: use 'phase-heads' to retrieve phase information
Boris Feld <boris.feld@octobus.net>
parents: 34328
diff changeset
1507 if 'phases' in repo.ui.configlist('devel', 'legacy.exchange'):
10e162bb9bf5 pull: use 'phase-heads' to retrieve phase information
Boris Feld <boris.feld@octobus.net>
parents: 34328
diff changeset
1508 caps.pop('phases')
22343
76677a2c1cfd bundle2: advertise the obsmarker part in bundle2 capabilities
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 22342
diff changeset
1509 return caps
22342
262c5cc126c1 bundle2: introduce a `getrepocaps` to retrieve the bundle2 caps of a repo
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 22341
diff changeset
1510
21644
17755dd8c509 bundle2: introduce a bundle2caps function
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 21628
diff changeset
1511 def bundle2caps(remote):
23139
e53f6b72a0e4 spelling: fixes from proofreading of spell checker issues
Mads Kiilerich <madski@unity3d.com>
parents: 23067
diff changeset
1512 """return the bundle capabilities of a peer as dict"""
24686
e0e28e910fa3 bundle2: rename format, parts and config to final names
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 24648
diff changeset
1513 raw = remote.capable('bundle2')
21644
17755dd8c509 bundle2: introduce a bundle2caps function
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 21628
diff changeset
1514 if not raw and raw != '':
17755dd8c509 bundle2: introduce a bundle2caps function
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 21628
diff changeset
1515 return {}
28883
032c4c2f802a pycompat: switch to util.urlreq/util.urlerr for py3 compat
timeless <timeless@mozdev.org>
parents: 28672
diff changeset
1516 capsblob = urlreq.unquote(remote.capable('bundle2'))
21644
17755dd8c509 bundle2: introduce a bundle2caps function
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 21628
diff changeset
1517 return decodecaps(capsblob)
21014
a6246bba7b9e bundle2: add an unbundle part responsible from unbundling part
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 21013
diff changeset
1518
22344
9829b7948100 bundle2: add a `obsmarkersversion` function to extract supported version
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 22343
diff changeset
1519 def obsmarkersversion(caps):
9829b7948100 bundle2: add a `obsmarkersversion` function to extract supported version
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 22343
diff changeset
1520 """extract the list of supported obsmarkers versions from a bundle2caps dict
9829b7948100 bundle2: add a `obsmarkersversion` function to extract supported version
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 22343
diff changeset
1521 """
24686
e0e28e910fa3 bundle2: rename format, parts and config to final names
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 24648
diff changeset
1522 obscaps = caps.get('obsmarkers', ())
22344
9829b7948100 bundle2: add a `obsmarkersversion` function to extract supported version
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 22343
diff changeset
1523 return [int(c[1:]) for c in obscaps if c.startswith('V')]
9829b7948100 bundle2: add a `obsmarkersversion` function to extract supported version
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 22343
diff changeset
1524
32261
9dc36df78403 bundle: introduce an higher level function to write bundle on disk
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 32232
diff changeset
1525 def writenewbundle(ui, repo, source, filename, bundletype, outgoing, opts,
9dc36df78403 bundle: introduce an higher level function to write bundle on disk
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 32232
diff changeset
1526 vfs=None, compression=None, compopts=None):
9dc36df78403 bundle: introduce an higher level function to write bundle on disk
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 32232
diff changeset
1527 if bundletype.startswith('HG10'):
34116
5ede882c249c changegroup: replace getchangegroup with makechangegroup
Durham Goode <durham@fb.com>
parents: 33906
diff changeset
1528 cg = changegroup.makechangegroup(repo, outgoing, '01', source)
32261
9dc36df78403 bundle: introduce an higher level function to write bundle on disk
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 32232
diff changeset
1529 return writebundle(ui, cg, filename, bundletype, vfs=vfs,
9dc36df78403 bundle: introduce an higher level function to write bundle on disk
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 32232
diff changeset
1530 compression=compression, compopts=compopts)
9dc36df78403 bundle: introduce an higher level function to write bundle on disk
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 32232
diff changeset
1531 elif not bundletype.startswith('HG20'):
9dc36df78403 bundle: introduce an higher level function to write bundle on disk
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 32232
diff changeset
1532 raise error.ProgrammingError('unknown bundle type: %s' % bundletype)
9dc36df78403 bundle: introduce an higher level function to write bundle on disk
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 32232
diff changeset
1533
32549
37d70ba1d9d1 bundle: add an experimental knob to include obsmarkers in bundle
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 32548
diff changeset
1534 caps = {}
37d70ba1d9d1 bundle: add an experimental knob to include obsmarkers in bundle
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 32548
diff changeset
1535 if 'obsolescence' in opts:
37d70ba1d9d1 bundle: add an experimental knob to include obsmarkers in bundle
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 32548
diff changeset
1536 caps['obsmarkers'] = ('V1',)
37d70ba1d9d1 bundle: add an experimental knob to include obsmarkers in bundle
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 32548
diff changeset
1537 bundle = bundle20(ui, caps)
32261
9dc36df78403 bundle: introduce an higher level function to write bundle on disk
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 32232
diff changeset
1538 bundle.setcompression(compression, compopts)
9dc36df78403 bundle: introduce an higher level function to write bundle on disk
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 32232
diff changeset
1539 _addpartsfromopts(ui, repo, bundle, source, outgoing, opts)
9dc36df78403 bundle: introduce an higher level function to write bundle on disk
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 32232
diff changeset
1540 chunkiter = bundle.getchunks()
9dc36df78403 bundle: introduce an higher level function to write bundle on disk
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 32232
diff changeset
1541
9dc36df78403 bundle: introduce an higher level function to write bundle on disk
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 32232
diff changeset
1542 return changegroup.writechunks(ui, chunkiter, filename, vfs=vfs)
9dc36df78403 bundle: introduce an higher level function to write bundle on disk
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 32232
diff changeset
1543
9dc36df78403 bundle: introduce an higher level function to write bundle on disk
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 32232
diff changeset
1544 def _addpartsfromopts(ui, repo, bundler, source, outgoing, opts):
9dc36df78403 bundle: introduce an higher level function to write bundle on disk
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 32232
diff changeset
1545 # We should eventually reconcile this logic with the one behind
9dc36df78403 bundle: introduce an higher level function to write bundle on disk
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 32232
diff changeset
1546 # 'exchange.getbundle2partsgenerator'.
9dc36df78403 bundle: introduce an higher level function to write bundle on disk
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 32232
diff changeset
1547 #
9dc36df78403 bundle: introduce an higher level function to write bundle on disk
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 32232
diff changeset
1548 # The type of input from 'getbundle' and 'writenewbundle' are a bit
9dc36df78403 bundle: introduce an higher level function to write bundle on disk
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 32232
diff changeset
1549 # different right now. So we keep them separated for now for the sake of
9dc36df78403 bundle: introduce an higher level function to write bundle on disk
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 32232
diff changeset
1550 # simplicity.
9dc36df78403 bundle: introduce an higher level function to write bundle on disk
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 32232
diff changeset
1551
9dc36df78403 bundle: introduce an higher level function to write bundle on disk
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 32232
diff changeset
1552 # we always want a changegroup in such bundle
9dc36df78403 bundle: introduce an higher level function to write bundle on disk
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 32232
diff changeset
1553 cgversion = opts.get('cg.version')
9dc36df78403 bundle: introduce an higher level function to write bundle on disk
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 32232
diff changeset
1554 if cgversion is None:
9dc36df78403 bundle: introduce an higher level function to write bundle on disk
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 32232
diff changeset
1555 cgversion = changegroup.safeversion(repo)
34116
5ede882c249c changegroup: replace getchangegroup with makechangegroup
Durham Goode <durham@fb.com>
parents: 33906
diff changeset
1556 cg = changegroup.makechangegroup(repo, outgoing, cgversion, source)
32261
9dc36df78403 bundle: introduce an higher level function to write bundle on disk
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 32232
diff changeset
1557 part = bundler.newpart('changegroup', data=cg.getchunks())
9dc36df78403 bundle: introduce an higher level function to write bundle on disk
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 32232
diff changeset
1558 part.addparam('version', cg.version)
9dc36df78403 bundle: introduce an higher level function to write bundle on disk
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 32232
diff changeset
1559 if 'clcount' in cg.extras:
34228
8e0358024a36 bundles: turn nbchanges int into a bytestr using pycompat.bytestr
Augie Fackler <raf@durin42.com>
parents: 34164
diff changeset
1560 part.addparam('nbchanges', '%d' % cg.extras['clcount'],
32261
9dc36df78403 bundle: introduce an higher level function to write bundle on disk
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 32232
diff changeset
1561 mandatory=False)
33408
e3867c712d51 bundle2: automatically add 'targetphase' parameter in writenewbundle
Boris Feld <boris.feld@octobus.net>
parents: 33407
diff changeset
1562 if opts.get('phases') and repo.revs('%ln and secret()',
e3867c712d51 bundle2: automatically add 'targetphase' parameter in writenewbundle
Boris Feld <boris.feld@octobus.net>
parents: 33407
diff changeset
1563 outgoing.missingheads):
e3867c712d51 bundle2: automatically add 'targetphase' parameter in writenewbundle
Boris Feld <boris.feld@octobus.net>
parents: 33407
diff changeset
1564 part.addparam('targetphase', '%d' % phases.secret, mandatory=False)
32261
9dc36df78403 bundle: introduce an higher level function to write bundle on disk
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 32232
diff changeset
1565
32263
d7f93ebbbbdf bundle: add optional 'tagsfnodecache' data to on disk bundle (issue5543)
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 32262
diff changeset
1566 addparttagsfnodescache(repo, bundler, outgoing)
d7f93ebbbbdf bundle: add optional 'tagsfnodecache' data to on disk bundle (issue5543)
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 32262
diff changeset
1567
32549
37d70ba1d9d1 bundle: add an experimental knob to include obsmarkers in bundle
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 32548
diff changeset
1568 if opts.get('obsolescence', False):
37d70ba1d9d1 bundle: add an experimental knob to include obsmarkers in bundle
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 32548
diff changeset
1569 obsmarkers = repo.obsstore.relevantmarkers(outgoing.missing)
37d70ba1d9d1 bundle: add an experimental knob to include obsmarkers in bundle
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 32548
diff changeset
1570 buildobsmarkerspart(bundler, obsmarkers)
37d70ba1d9d1 bundle: add an experimental knob to include obsmarkers in bundle
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 32548
diff changeset
1571
33043
e8c8d81eb864 bundle: add config option to include phases
Martin von Zweigbergk <martinvonz@google.com>
parents: 33042
diff changeset
1572 if opts.get('phases', False):
e8c8d81eb864 bundle: add config option to include phases
Martin von Zweigbergk <martinvonz@google.com>
parents: 33042
diff changeset
1573 headsbyphase = phases.subsetphaseheads(repo, outgoing.missing)
34326
5779d096a696 phases: move binary encoding into a reusable function
Boris Feld <boris.feld@octobus.net>
parents: 34325
diff changeset
1574 phasedata = phases.binaryencode(headsbyphase)
5779d096a696 phases: move binary encoding into a reusable function
Boris Feld <boris.feld@octobus.net>
parents: 34325
diff changeset
1575 bundler.newpart('phase-heads', data=phasedata)
33043
e8c8d81eb864 bundle: add config option to include phases
Martin von Zweigbergk <martinvonz@google.com>
parents: 33042
diff changeset
1576
32262
6068712cbf03 bundle2: move tagsfnodecache generation in a generic function
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 32261
diff changeset
1577 def addparttagsfnodescache(repo, bundler, outgoing):
6068712cbf03 bundle2: move tagsfnodecache generation in a generic function
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 32261
diff changeset
1578 # we include the tags fnode cache for the bundle changeset
6068712cbf03 bundle2: move tagsfnodecache generation in a generic function
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 32261
diff changeset
1579 # (as an optional parts)
6068712cbf03 bundle2: move tagsfnodecache generation in a generic function
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 32261
diff changeset
1580 cache = tags.hgtagsfnodescache(repo.unfiltered())
6068712cbf03 bundle2: move tagsfnodecache generation in a generic function
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 32261
diff changeset
1581 chunks = []
6068712cbf03 bundle2: move tagsfnodecache generation in a generic function
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 32261
diff changeset
1582
6068712cbf03 bundle2: move tagsfnodecache generation in a generic function
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 32261
diff changeset
1583 # .hgtags fnodes are only relevant for head changesets. While we could
6068712cbf03 bundle2: move tagsfnodecache generation in a generic function
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 32261
diff changeset
1584 # transfer values for all known nodes, there will likely be little to
6068712cbf03 bundle2: move tagsfnodecache generation in a generic function
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 32261
diff changeset
1585 # no benefit.
6068712cbf03 bundle2: move tagsfnodecache generation in a generic function
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 32261
diff changeset
1586 #
6068712cbf03 bundle2: move tagsfnodecache generation in a generic function
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 32261
diff changeset
1587 # We don't bother using a generator to produce output data because
6068712cbf03 bundle2: move tagsfnodecache generation in a generic function
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 32261
diff changeset
1588 # a) we only have 40 bytes per head and even esoteric numbers of heads
6068712cbf03 bundle2: move tagsfnodecache generation in a generic function
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 32261
diff changeset
1589 # consume little memory (1M heads is 40MB) b) we don't want to send the
6068712cbf03 bundle2: move tagsfnodecache generation in a generic function
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 32261
diff changeset
1590 # part if we don't have entries and knowing if we have entries requires
6068712cbf03 bundle2: move tagsfnodecache generation in a generic function
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 32261
diff changeset
1591 # cache lookups.
6068712cbf03 bundle2: move tagsfnodecache generation in a generic function
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 32261
diff changeset
1592 for node in outgoing.missingheads:
6068712cbf03 bundle2: move tagsfnodecache generation in a generic function
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 32261
diff changeset
1593 # Don't compute missing, as this may slow down serving.
6068712cbf03 bundle2: move tagsfnodecache generation in a generic function
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 32261
diff changeset
1594 fnode = cache.getfnode(node, computemissing=False)
6068712cbf03 bundle2: move tagsfnodecache generation in a generic function
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 32261
diff changeset
1595 if fnode is not None:
6068712cbf03 bundle2: move tagsfnodecache generation in a generic function
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 32261
diff changeset
1596 chunks.extend([node, fnode])
6068712cbf03 bundle2: move tagsfnodecache generation in a generic function
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 32261
diff changeset
1597
6068712cbf03 bundle2: move tagsfnodecache generation in a generic function
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 32261
diff changeset
1598 if chunks:
6068712cbf03 bundle2: move tagsfnodecache generation in a generic function
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 32261
diff changeset
1599 bundler.newpart('hgtagsfnodes', data=''.join(chunks))
6068712cbf03 bundle2: move tagsfnodecache generation in a generic function
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 32261
diff changeset
1600
32548
e70d6dbde713 bundle2: move function building obsmarker-part in the bundle2 module
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 32425
diff changeset
1601 def buildobsmarkerspart(bundler, markers):
e70d6dbde713 bundle2: move function building obsmarker-part in the bundle2 module
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 32425
diff changeset
1602 """add an obsmarker part to the bundler with <markers>
e70d6dbde713 bundle2: move function building obsmarker-part in the bundle2 module
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 32425
diff changeset
1603
e70d6dbde713 bundle2: move function building obsmarker-part in the bundle2 module
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 32425
diff changeset
1604 No part is created if markers is empty.
e70d6dbde713 bundle2: move function building obsmarker-part in the bundle2 module
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 32425
diff changeset
1605 Raises ValueError if the bundler doesn't support any known obsmarker format.
e70d6dbde713 bundle2: move function building obsmarker-part in the bundle2 module
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 32425
diff changeset
1606 """
e70d6dbde713 bundle2: move function building obsmarker-part in the bundle2 module
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 32425
diff changeset
1607 if not markers:
e70d6dbde713 bundle2: move function building obsmarker-part in the bundle2 module
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 32425
diff changeset
1608 return None
e70d6dbde713 bundle2: move function building obsmarker-part in the bundle2 module
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 32425
diff changeset
1609
e70d6dbde713 bundle2: move function building obsmarker-part in the bundle2 module
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 32425
diff changeset
1610 remoteversions = obsmarkersversion(bundler.capabilities)
e70d6dbde713 bundle2: move function building obsmarker-part in the bundle2 module
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 32425
diff changeset
1611 version = obsolete.commonversion(remoteversions)
e70d6dbde713 bundle2: move function building obsmarker-part in the bundle2 module
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 32425
diff changeset
1612 if version is None:
e70d6dbde713 bundle2: move function building obsmarker-part in the bundle2 module
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 32425
diff changeset
1613 raise ValueError('bundler does not support common obsmarker format')
e70d6dbde713 bundle2: move function building obsmarker-part in the bundle2 module
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 32425
diff changeset
1614 stream = obsolete.encodemarkers(markers, True, version=version)
e70d6dbde713 bundle2: move function building obsmarker-part in the bundle2 module
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 32425
diff changeset
1615 return bundler.newpart('obsmarkers', data=stream)
e70d6dbde713 bundle2: move function building obsmarker-part in the bundle2 module
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 32425
diff changeset
1616
30757
511a4bf52754 bundle2: allow compression options to be passed to compressor
Gregory Szorc <gregory.szorc@gmail.com>
parents: 30448
diff changeset
1617 def writebundle(ui, cg, filename, bundletype, vfs=None, compression=None,
511a4bf52754 bundle2: allow compression options to be passed to compressor
Gregory Szorc <gregory.szorc@gmail.com>
parents: 30448
diff changeset
1618 compopts=None):
28666
ae53ecc47414 bundle: move writebundle() from changegroup.py to bundle2.py (API)
Martin von Zweigbergk <martinvonz@google.com>
parents: 27953
diff changeset
1619 """Write a bundle file and return its filename.
ae53ecc47414 bundle: move writebundle() from changegroup.py to bundle2.py (API)
Martin von Zweigbergk <martinvonz@google.com>
parents: 27953
diff changeset
1620
ae53ecc47414 bundle: move writebundle() from changegroup.py to bundle2.py (API)
Martin von Zweigbergk <martinvonz@google.com>
parents: 27953
diff changeset
1621 Existing files will not be overwritten.
ae53ecc47414 bundle: move writebundle() from changegroup.py to bundle2.py (API)
Martin von Zweigbergk <martinvonz@google.com>
parents: 27953
diff changeset
1622 If no filename is specified, a temporary file is created.
ae53ecc47414 bundle: move writebundle() from changegroup.py to bundle2.py (API)
Martin von Zweigbergk <martinvonz@google.com>
parents: 27953
diff changeset
1623 bz2 compression can be turned off.
ae53ecc47414 bundle: move writebundle() from changegroup.py to bundle2.py (API)
Martin von Zweigbergk <martinvonz@google.com>
parents: 27953
diff changeset
1624 The bundle file will be deleted in case of errors.
ae53ecc47414 bundle: move writebundle() from changegroup.py to bundle2.py (API)
Martin von Zweigbergk <martinvonz@google.com>
parents: 27953
diff changeset
1625 """
ae53ecc47414 bundle: move writebundle() from changegroup.py to bundle2.py (API)
Martin von Zweigbergk <martinvonz@google.com>
parents: 27953
diff changeset
1626
ae53ecc47414 bundle: move writebundle() from changegroup.py to bundle2.py (API)
Martin von Zweigbergk <martinvonz@google.com>
parents: 27953
diff changeset
1627 if bundletype == "HG20":
ae53ecc47414 bundle: move writebundle() from changegroup.py to bundle2.py (API)
Martin von Zweigbergk <martinvonz@google.com>
parents: 27953
diff changeset
1628 bundle = bundle20(ui)
30757
511a4bf52754 bundle2: allow compression options to be passed to compressor
Gregory Szorc <gregory.szorc@gmail.com>
parents: 30448
diff changeset
1629 bundle.setcompression(compression, compopts)
28666
ae53ecc47414 bundle: move writebundle() from changegroup.py to bundle2.py (API)
Martin von Zweigbergk <martinvonz@google.com>
parents: 27953
diff changeset
1630 part = bundle.newpart('changegroup', data=cg.getchunks())
ae53ecc47414 bundle: move writebundle() from changegroup.py to bundle2.py (API)
Martin von Zweigbergk <martinvonz@google.com>
parents: 27953
diff changeset
1631 part.addparam('version', cg.version)
29593
953839de96ab bundle2: store changeset count when creating file bundles
Gregory Szorc <gregory.szorc@gmail.com>
parents: 29591
diff changeset
1632 if 'clcount' in cg.extras:
34228
8e0358024a36 bundles: turn nbchanges int into a bytestr using pycompat.bytestr
Augie Fackler <raf@durin42.com>
parents: 34164
diff changeset
1633 part.addparam('nbchanges', '%d' % cg.extras['clcount'],
29593
953839de96ab bundle2: store changeset count when creating file bundles
Gregory Szorc <gregory.szorc@gmail.com>
parents: 29591
diff changeset
1634 mandatory=False)
28666
ae53ecc47414 bundle: move writebundle() from changegroup.py to bundle2.py (API)
Martin von Zweigbergk <martinvonz@google.com>
parents: 27953
diff changeset
1635 chunkiter = bundle.getchunks()
ae53ecc47414 bundle: move writebundle() from changegroup.py to bundle2.py (API)
Martin von Zweigbergk <martinvonz@google.com>
parents: 27953
diff changeset
1636 else:
ae53ecc47414 bundle: move writebundle() from changegroup.py to bundle2.py (API)
Martin von Zweigbergk <martinvonz@google.com>
parents: 27953
diff changeset
1637 # compression argument is only for the bundle2 case
ae53ecc47414 bundle: move writebundle() from changegroup.py to bundle2.py (API)
Martin von Zweigbergk <martinvonz@google.com>
parents: 27953
diff changeset
1638 assert compression is None
ae53ecc47414 bundle: move writebundle() from changegroup.py to bundle2.py (API)
Martin von Zweigbergk <martinvonz@google.com>
parents: 27953
diff changeset
1639 if cg.version != '01':
ae53ecc47414 bundle: move writebundle() from changegroup.py to bundle2.py (API)
Martin von Zweigbergk <martinvonz@google.com>
parents: 27953
diff changeset
1640 raise error.Abort(_('old bundle types only supports v1 '
ae53ecc47414 bundle: move writebundle() from changegroup.py to bundle2.py (API)
Martin von Zweigbergk <martinvonz@google.com>
parents: 27953
diff changeset
1641 'changegroups'))
ae53ecc47414 bundle: move writebundle() from changegroup.py to bundle2.py (API)
Martin von Zweigbergk <martinvonz@google.com>
parents: 27953
diff changeset
1642 header, comp = bundletypes[bundletype]
30361
f81002f736d7 bundle2: use new compression engine API for compression
Gregory Szorc <gregory.szorc@gmail.com>
parents: 30177
diff changeset
1643 if comp not in util.compengines.supportedbundletypes:
28666
ae53ecc47414 bundle: move writebundle() from changegroup.py to bundle2.py (API)
Martin von Zweigbergk <martinvonz@google.com>
parents: 27953
diff changeset
1644 raise error.Abort(_('unknown stream compression type: %s')
ae53ecc47414 bundle: move writebundle() from changegroup.py to bundle2.py (API)
Martin von Zweigbergk <martinvonz@google.com>
parents: 27953
diff changeset
1645 % comp)
30361
f81002f736d7 bundle2: use new compression engine API for compression
Gregory Szorc <gregory.szorc@gmail.com>
parents: 30177
diff changeset
1646 compengine = util.compengines.forbundletype(comp)
28666
ae53ecc47414 bundle: move writebundle() from changegroup.py to bundle2.py (API)
Martin von Zweigbergk <martinvonz@google.com>
parents: 27953
diff changeset
1647 def chunkiter():
ae53ecc47414 bundle: move writebundle() from changegroup.py to bundle2.py (API)
Martin von Zweigbergk <martinvonz@google.com>
parents: 27953
diff changeset
1648 yield header
30757
511a4bf52754 bundle2: allow compression options to be passed to compressor
Gregory Szorc <gregory.szorc@gmail.com>
parents: 30448
diff changeset
1649 for chunk in compengine.compressstream(cg.getchunks(), compopts):
30367
5925bda42dbd bundle2: use compressstream compression engine API
Gregory Szorc <gregory.szorc@gmail.com>
parents: 30363
diff changeset
1650 yield chunk
28666
ae53ecc47414 bundle: move writebundle() from changegroup.py to bundle2.py (API)
Martin von Zweigbergk <martinvonz@google.com>
parents: 27953
diff changeset
1651 chunkiter = chunkiter()
ae53ecc47414 bundle: move writebundle() from changegroup.py to bundle2.py (API)
Martin von Zweigbergk <martinvonz@google.com>
parents: 27953
diff changeset
1652
ae53ecc47414 bundle: move writebundle() from changegroup.py to bundle2.py (API)
Martin von Zweigbergk <martinvonz@google.com>
parents: 27953
diff changeset
1653 # parse the changegroup data, otherwise we will block
ae53ecc47414 bundle: move writebundle() from changegroup.py to bundle2.py (API)
Martin von Zweigbergk <martinvonz@google.com>
parents: 27953
diff changeset
1654 # in case of sshrepo because we don't know the end of the stream
ae53ecc47414 bundle: move writebundle() from changegroup.py to bundle2.py (API)
Martin von Zweigbergk <martinvonz@google.com>
parents: 27953
diff changeset
1655 return changegroup.writechunks(ui, chunkiter, filename, vfs=vfs)
ae53ecc47414 bundle: move writebundle() from changegroup.py to bundle2.py (API)
Martin von Zweigbergk <martinvonz@google.com>
parents: 27953
diff changeset
1656
33049
d765ad56081f bundle: make combinechangegroupresults() take a bundleoperation
Martin von Zweigbergk <martinvonz@google.com>
parents: 33048
diff changeset
1657 def combinechangegroupresults(op):
33048
52c7060b707a bundle: move combineresults() from changegroup to bundle2
Martin von Zweigbergk <martinvonz@google.com>
parents: 33047
diff changeset
1658 """logic to combine 0 or more addchangegroup results into one"""
33049
d765ad56081f bundle: make combinechangegroupresults() take a bundleoperation
Martin von Zweigbergk <martinvonz@google.com>
parents: 33048
diff changeset
1659 results = [r.get('return', 0)
d765ad56081f bundle: make combinechangegroupresults() take a bundleoperation
Martin von Zweigbergk <martinvonz@google.com>
parents: 33048
diff changeset
1660 for r in op.records['changegroup']]
33048
52c7060b707a bundle: move combineresults() from changegroup to bundle2
Martin von Zweigbergk <martinvonz@google.com>
parents: 33047
diff changeset
1661 changedheads = 0
52c7060b707a bundle: move combineresults() from changegroup to bundle2
Martin von Zweigbergk <martinvonz@google.com>
parents: 33047
diff changeset
1662 result = 1
52c7060b707a bundle: move combineresults() from changegroup to bundle2
Martin von Zweigbergk <martinvonz@google.com>
parents: 33047
diff changeset
1663 for ret in results:
52c7060b707a bundle: move combineresults() from changegroup to bundle2
Martin von Zweigbergk <martinvonz@google.com>
parents: 33047
diff changeset
1664 # If any changegroup result is 0, return 0
52c7060b707a bundle: move combineresults() from changegroup to bundle2
Martin von Zweigbergk <martinvonz@google.com>
parents: 33047
diff changeset
1665 if ret == 0:
52c7060b707a bundle: move combineresults() from changegroup to bundle2
Martin von Zweigbergk <martinvonz@google.com>
parents: 33047
diff changeset
1666 result = 0
52c7060b707a bundle: move combineresults() from changegroup to bundle2
Martin von Zweigbergk <martinvonz@google.com>
parents: 33047
diff changeset
1667 break
52c7060b707a bundle: move combineresults() from changegroup to bundle2
Martin von Zweigbergk <martinvonz@google.com>
parents: 33047
diff changeset
1668 if ret < -1:
52c7060b707a bundle: move combineresults() from changegroup to bundle2
Martin von Zweigbergk <martinvonz@google.com>
parents: 33047
diff changeset
1669 changedheads += ret + 1
52c7060b707a bundle: move combineresults() from changegroup to bundle2
Martin von Zweigbergk <martinvonz@google.com>
parents: 33047
diff changeset
1670 elif ret > 1:
52c7060b707a bundle: move combineresults() from changegroup to bundle2
Martin von Zweigbergk <martinvonz@google.com>
parents: 33047
diff changeset
1671 changedheads += ret - 1
52c7060b707a bundle: move combineresults() from changegroup to bundle2
Martin von Zweigbergk <martinvonz@google.com>
parents: 33047
diff changeset
1672 if changedheads > 0:
52c7060b707a bundle: move combineresults() from changegroup to bundle2
Martin von Zweigbergk <martinvonz@google.com>
parents: 33047
diff changeset
1673 result = 1 + changedheads
52c7060b707a bundle: move combineresults() from changegroup to bundle2
Martin von Zweigbergk <martinvonz@google.com>
parents: 33047
diff changeset
1674 elif changedheads < 0:
52c7060b707a bundle: move combineresults() from changegroup to bundle2
Martin von Zweigbergk <martinvonz@google.com>
parents: 33047
diff changeset
1675 result = -1 + changedheads
52c7060b707a bundle: move combineresults() from changegroup to bundle2
Martin von Zweigbergk <martinvonz@google.com>
parents: 33047
diff changeset
1676 return result
52c7060b707a bundle: move combineresults() from changegroup to bundle2
Martin von Zweigbergk <martinvonz@google.com>
parents: 33047
diff changeset
1677
33407
39d4e5a66f5a bundle2: support the 'targetphase' parameter for the changegroup part
Boris Feld <boris.feld@octobus.net>
parents: 33252
diff changeset
1678 @parthandler('changegroup', ('version', 'nbchanges', 'treemanifest',
39d4e5a66f5a bundle2: support the 'targetphase' parameter for the changegroup part
Boris Feld <boris.feld@octobus.net>
parents: 33252
diff changeset
1679 'targetphase'))
20998
93a3c5b58635 bundle2: use reply part to return result of addchangegroup
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 20997
diff changeset
1680 def handlechangegroup(op, inpart):
20950
c7ceae0faf69 bundle2: first crude version of bundling changeset with bundle2
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 20949
diff changeset
1681 """apply a changegroup part on the repo
c7ceae0faf69 bundle2: first crude version of bundling changeset with bundle2
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 20949
diff changeset
1682
c7ceae0faf69 bundle2: first crude version of bundling changeset with bundle2
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 20949
diff changeset
1683 This is a very early implementation that will massive rework before being
c7ceae0faf69 bundle2: first crude version of bundling changeset with bundle2
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 20949
diff changeset
1684 inflicted to any end-user.
c7ceae0faf69 bundle2: first crude version of bundling changeset with bundle2
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 20949
diff changeset
1685 """
32948
af31d531dda0 changegroup: let callers pass in transaction to apply() (API)
Martin von Zweigbergk <martinvonz@google.com>
parents: 32910
diff changeset
1686 tr = op.gettransaction()
23170
02e8f9b60052 bundle2: support a "version" argument in `changegroup` part
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 23169
diff changeset
1687 unpackerversion = inpart.params.get('version', '01')
02e8f9b60052 bundle2: support a "version" argument in `changegroup` part
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 23169
diff changeset
1688 # We should raise an appropriate exception here
27751
a40e2f7fe49d changegroup: hide packermap behind methods
Martin von Zweigbergk <martinvonz@google.com>
parents: 27734
diff changeset
1689 cg = changegroup.getunbundler(unpackerversion, inpart, None)
23001
4df9b5e62f70 bundle2: add a comment about addchangegroup source and url
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 22953
diff changeset
1690 # the source and url passed here are overwritten by the one contained in
4df9b5e62f70 bundle2: add a comment about addchangegroup source and url
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 22953
diff changeset
1691 # the transaction.hookargs argument. So 'bundle2' is a placeholder
25518
ca656f3dffd7 bundle2: provide number of changesets information to 'addchangegroup'
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 25507
diff changeset
1692 nbchangesets = None
ca656f3dffd7 bundle2: provide number of changesets information to 'addchangegroup'
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 25507
diff changeset
1693 if 'nbchanges' in inpart.params:
ca656f3dffd7 bundle2: provide number of changesets information to 'addchangegroup'
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 25507
diff changeset
1694 nbchangesets = int(inpart.params.get('nbchanges'))
27734
5c0fd878779c treemanifests: set bundle2 part parameter indicating treemanifest
Martin von Zweigbergk <martinvonz@google.com>
parents: 26829
diff changeset
1695 if ('treemanifest' in inpart.params and
5c0fd878779c treemanifests: set bundle2 part parameter indicating treemanifest
Martin von Zweigbergk <martinvonz@google.com>
parents: 26829
diff changeset
1696 'treemanifest' not in op.repo.requirements):
5c0fd878779c treemanifests: set bundle2 part parameter indicating treemanifest
Martin von Zweigbergk <martinvonz@google.com>
parents: 26829
diff changeset
1697 if len(op.repo.changelog) != 0:
5c0fd878779c treemanifests: set bundle2 part parameter indicating treemanifest
Martin von Zweigbergk <martinvonz@google.com>
parents: 26829
diff changeset
1698 raise error.Abort(_(
5c0fd878779c treemanifests: set bundle2 part parameter indicating treemanifest
Martin von Zweigbergk <martinvonz@google.com>
parents: 26829
diff changeset
1699 "bundle contains tree manifests, but local repo is "
5c0fd878779c treemanifests: set bundle2 part parameter indicating treemanifest
Martin von Zweigbergk <martinvonz@google.com>
parents: 26829
diff changeset
1700 "non-empty and does not use tree manifests"))
5c0fd878779c treemanifests: set bundle2 part parameter indicating treemanifest
Martin von Zweigbergk <martinvonz@google.com>
parents: 26829
diff changeset
1701 op.repo.requirements.add('treemanifest')
5c0fd878779c treemanifests: set bundle2 part parameter indicating treemanifest
Martin von Zweigbergk <martinvonz@google.com>
parents: 26829
diff changeset
1702 op.repo._applyopenerreqs()
5c0fd878779c treemanifests: set bundle2 part parameter indicating treemanifest
Martin von Zweigbergk <martinvonz@google.com>
parents: 26829
diff changeset
1703 op.repo._writerequirements()
33407
39d4e5a66f5a bundle2: support the 'targetphase' parameter for the changegroup part
Boris Feld <boris.feld@octobus.net>
parents: 33252
diff changeset
1704 extrakwargs = {}
39d4e5a66f5a bundle2: support the 'targetphase' parameter for the changegroup part
Boris Feld <boris.feld@octobus.net>
parents: 33252
diff changeset
1705 targetphase = inpart.params.get('targetphase')
39d4e5a66f5a bundle2: support the 'targetphase' parameter for the changegroup part
Boris Feld <boris.feld@octobus.net>
parents: 33252
diff changeset
1706 if targetphase is not None:
39d4e5a66f5a bundle2: support the 'targetphase' parameter for the changegroup part
Boris Feld <boris.feld@octobus.net>
parents: 33252
diff changeset
1707 extrakwargs['targetphase'] = int(targetphase)
33050
f0efd2bffe1e bundle: extract _processchangegroup() method
Martin von Zweigbergk <martinvonz@google.com>
parents: 33049
diff changeset
1708 ret = _processchangegroup(op, cg, tr, 'bundle2', 'bundle2',
33407
39d4e5a66f5a bundle2: support the 'targetphase' parameter for the changegroup part
Boris Feld <boris.feld@octobus.net>
parents: 33252
diff changeset
1709 expectedtotal=nbchangesets, **extrakwargs)
20998
93a3c5b58635 bundle2: use reply part to return result of addchangegroup
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 20997
diff changeset
1710 if op.reply is not None:
23139
e53f6b72a0e4 spelling: fixes from proofreading of spell checker issues
Mads Kiilerich <madski@unity3d.com>
parents: 23067
diff changeset
1711 # This is definitely not the final form of this
20998
93a3c5b58635 bundle2: use reply part to return result of addchangegroup
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 20997
diff changeset
1712 # return. But one need to start somewhere.
24686
e0e28e910fa3 bundle2: rename format, parts and config to final names
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 24648
diff changeset
1713 part = op.reply.newpart('reply:changegroup', mandatory=False)
33701
24b7c75c6441 bundle2: use bytestr() instead of str() to convert part id to bytes
Augie Fackler <augie@google.com>
parents: 33700
diff changeset
1714 part.addparam(
24b7c75c6441 bundle2: use bytestr() instead of str() to convert part id to bytes
Augie Fackler <augie@google.com>
parents: 33700
diff changeset
1715 'in-reply-to', pycompat.bytestr(inpart.id), mandatory=False)
21606
e55888447958 bundle2: update part creators to ``addparam`` when relevant
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 21605
diff changeset
1716 part.addparam('return', '%i' % ret, mandatory=False)
21019
3dc09f831a2e bundle2: lazy unbundle of part payload
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 21016
diff changeset
1717 assert not inpart.read()
20950
c7ceae0faf69 bundle2: first crude version of bundling changeset with bundle2
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 20949
diff changeset
1718
23029
149fc8a44184 bundle2: client side support for a part to import external bundles
Mike Hommey <mh@glandium.org>
parents: 23011
diff changeset
1719 _remotechangegroupparams = tuple(['url', 'size', 'digests'] +
149fc8a44184 bundle2: client side support for a part to import external bundles
Mike Hommey <mh@glandium.org>
parents: 23011
diff changeset
1720 ['digest:%s' % k for k in util.DIGESTS.keys()])
24686
e0e28e910fa3 bundle2: rename format, parts and config to final names
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 24648
diff changeset
1721 @parthandler('remote-changegroup', _remotechangegroupparams)
23029
149fc8a44184 bundle2: client side support for a part to import external bundles
Mike Hommey <mh@glandium.org>
parents: 23011
diff changeset
1722 def handleremotechangegroup(op, inpart):
149fc8a44184 bundle2: client side support for a part to import external bundles
Mike Hommey <mh@glandium.org>
parents: 23011
diff changeset
1723 """apply a bundle10 on the repo, given an url and validation information
149fc8a44184 bundle2: client side support for a part to import external bundles
Mike Hommey <mh@glandium.org>
parents: 23011
diff changeset
1724
149fc8a44184 bundle2: client side support for a part to import external bundles
Mike Hommey <mh@glandium.org>
parents: 23011
diff changeset
1725 All the information about the remote bundle to import are given as
149fc8a44184 bundle2: client side support for a part to import external bundles
Mike Hommey <mh@glandium.org>
parents: 23011
diff changeset
1726 parameters. The parameters include:
149fc8a44184 bundle2: client side support for a part to import external bundles
Mike Hommey <mh@glandium.org>
parents: 23011
diff changeset
1727 - url: the url to the bundle10.
149fc8a44184 bundle2: client side support for a part to import external bundles
Mike Hommey <mh@glandium.org>
parents: 23011
diff changeset
1728 - size: the bundle10 file size. It is used to validate what was
149fc8a44184 bundle2: client side support for a part to import external bundles
Mike Hommey <mh@glandium.org>
parents: 23011
diff changeset
1729 retrieved by the client matches the server knowledge about the bundle.
149fc8a44184 bundle2: client side support for a part to import external bundles
Mike Hommey <mh@glandium.org>
parents: 23011
diff changeset
1730 - digests: a space separated list of the digest types provided as
149fc8a44184 bundle2: client side support for a part to import external bundles
Mike Hommey <mh@glandium.org>
parents: 23011
diff changeset
1731 parameters.
149fc8a44184 bundle2: client side support for a part to import external bundles
Mike Hommey <mh@glandium.org>
parents: 23011
diff changeset
1732 - digest:<digest-type>: the hexadecimal representation of the digest with
149fc8a44184 bundle2: client side support for a part to import external bundles
Mike Hommey <mh@glandium.org>
parents: 23011
diff changeset
1733 that name. Like the size, it is used to validate what was retrieved by
149fc8a44184 bundle2: client side support for a part to import external bundles
Mike Hommey <mh@glandium.org>
parents: 23011
diff changeset
1734 the client matches what the server knows about the bundle.
149fc8a44184 bundle2: client side support for a part to import external bundles
Mike Hommey <mh@glandium.org>
parents: 23011
diff changeset
1735
149fc8a44184 bundle2: client side support for a part to import external bundles
Mike Hommey <mh@glandium.org>
parents: 23011
diff changeset
1736 When multiple digest types are given, all of them are checked.
149fc8a44184 bundle2: client side support for a part to import external bundles
Mike Hommey <mh@glandium.org>
parents: 23011
diff changeset
1737 """
149fc8a44184 bundle2: client side support for a part to import external bundles
Mike Hommey <mh@glandium.org>
parents: 23011
diff changeset
1738 try:
149fc8a44184 bundle2: client side support for a part to import external bundles
Mike Hommey <mh@glandium.org>
parents: 23011
diff changeset
1739 raw_url = inpart.params['url']
149fc8a44184 bundle2: client side support for a part to import external bundles
Mike Hommey <mh@glandium.org>
parents: 23011
diff changeset
1740 except KeyError:
26587
56b2bcea2529 error: get Abort from 'error' instead of 'util'
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 26565
diff changeset
1741 raise error.Abort(_('remote-changegroup: missing "%s" param') % 'url')
23029
149fc8a44184 bundle2: client side support for a part to import external bundles
Mike Hommey <mh@glandium.org>
parents: 23011
diff changeset
1742 parsed_url = util.url(raw_url)
24686
e0e28e910fa3 bundle2: rename format, parts and config to final names
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 24648
diff changeset
1743 if parsed_url.scheme not in capabilities['remote-changegroup']:
26587
56b2bcea2529 error: get Abort from 'error' instead of 'util'
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 26565
diff changeset
1744 raise error.Abort(_('remote-changegroup does not support %s urls') %
23029
149fc8a44184 bundle2: client side support for a part to import external bundles
Mike Hommey <mh@glandium.org>
parents: 23011
diff changeset
1745 parsed_url.scheme)
149fc8a44184 bundle2: client side support for a part to import external bundles
Mike Hommey <mh@glandium.org>
parents: 23011
diff changeset
1746
149fc8a44184 bundle2: client side support for a part to import external bundles
Mike Hommey <mh@glandium.org>
parents: 23011
diff changeset
1747 try:
149fc8a44184 bundle2: client side support for a part to import external bundles
Mike Hommey <mh@glandium.org>
parents: 23011
diff changeset
1748 size = int(inpart.params['size'])
149fc8a44184 bundle2: client side support for a part to import external bundles
Mike Hommey <mh@glandium.org>
parents: 23011
diff changeset
1749 except ValueError:
26587
56b2bcea2529 error: get Abort from 'error' instead of 'util'
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 26565
diff changeset
1750 raise error.Abort(_('remote-changegroup: invalid value for param "%s"')
23029
149fc8a44184 bundle2: client side support for a part to import external bundles
Mike Hommey <mh@glandium.org>
parents: 23011
diff changeset
1751 % 'size')
149fc8a44184 bundle2: client side support for a part to import external bundles
Mike Hommey <mh@glandium.org>
parents: 23011
diff changeset
1752 except KeyError:
26587
56b2bcea2529 error: get Abort from 'error' instead of 'util'
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 26565
diff changeset
1753 raise error.Abort(_('remote-changegroup: missing "%s" param') % 'size')
23029
149fc8a44184 bundle2: client side support for a part to import external bundles
Mike Hommey <mh@glandium.org>
parents: 23011
diff changeset
1754
149fc8a44184 bundle2: client side support for a part to import external bundles
Mike Hommey <mh@glandium.org>
parents: 23011
diff changeset
1755 digests = {}
149fc8a44184 bundle2: client side support for a part to import external bundles
Mike Hommey <mh@glandium.org>
parents: 23011
diff changeset
1756 for typ in inpart.params.get('digests', '').split():
149fc8a44184 bundle2: client side support for a part to import external bundles
Mike Hommey <mh@glandium.org>
parents: 23011
diff changeset
1757 param = 'digest:%s' % typ
149fc8a44184 bundle2: client side support for a part to import external bundles
Mike Hommey <mh@glandium.org>
parents: 23011
diff changeset
1758 try:
149fc8a44184 bundle2: client side support for a part to import external bundles
Mike Hommey <mh@glandium.org>
parents: 23011
diff changeset
1759 value = inpart.params[param]
149fc8a44184 bundle2: client side support for a part to import external bundles
Mike Hommey <mh@glandium.org>
parents: 23011
diff changeset
1760 except KeyError:
26587
56b2bcea2529 error: get Abort from 'error' instead of 'util'
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 26565
diff changeset
1761 raise error.Abort(_('remote-changegroup: missing "%s" param') %
23029
149fc8a44184 bundle2: client side support for a part to import external bundles
Mike Hommey <mh@glandium.org>
parents: 23011
diff changeset
1762 param)
149fc8a44184 bundle2: client side support for a part to import external bundles
Mike Hommey <mh@glandium.org>
parents: 23011
diff changeset
1763 digests[typ] = value
149fc8a44184 bundle2: client side support for a part to import external bundles
Mike Hommey <mh@glandium.org>
parents: 23011
diff changeset
1764
149fc8a44184 bundle2: client side support for a part to import external bundles
Mike Hommey <mh@glandium.org>
parents: 23011
diff changeset
1765 real_part = util.digestchecker(url.open(op.ui, raw_url), size, digests)
149fc8a44184 bundle2: client side support for a part to import external bundles
Mike Hommey <mh@glandium.org>
parents: 23011
diff changeset
1766
32948
af31d531dda0 changegroup: let callers pass in transaction to apply() (API)
Martin von Zweigbergk <martinvonz@google.com>
parents: 32910
diff changeset
1767 tr = op.gettransaction()
25919
8221fefaea08 bundle2: use absolute_import
Gregory Szorc <gregory.szorc@gmail.com>
parents: 25660
diff changeset
1768 from . import exchange
23029
149fc8a44184 bundle2: client side support for a part to import external bundles
Mike Hommey <mh@glandium.org>
parents: 23011
diff changeset
1769 cg = exchange.readbundle(op.repo.ui, real_part, raw_url)
149fc8a44184 bundle2: client side support for a part to import external bundles
Mike Hommey <mh@glandium.org>
parents: 23011
diff changeset
1770 if not isinstance(cg, changegroup.cg1unpacker):
26587
56b2bcea2529 error: get Abort from 'error' instead of 'util'
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 26565
diff changeset
1771 raise error.Abort(_('%s: not a bundle version 1.0') %
23029
149fc8a44184 bundle2: client side support for a part to import external bundles
Mike Hommey <mh@glandium.org>
parents: 23011
diff changeset
1772 util.hidepassword(raw_url))
33050
f0efd2bffe1e bundle: extract _processchangegroup() method
Martin von Zweigbergk <martinvonz@google.com>
parents: 33049
diff changeset
1773 ret = _processchangegroup(op, cg, tr, 'bundle2', 'bundle2')
23029
149fc8a44184 bundle2: client side support for a part to import external bundles
Mike Hommey <mh@glandium.org>
parents: 23011
diff changeset
1774 if op.reply is not None:
23139
e53f6b72a0e4 spelling: fixes from proofreading of spell checker issues
Mads Kiilerich <madski@unity3d.com>
parents: 23067
diff changeset
1775 # This is definitely not the final form of this
23029
149fc8a44184 bundle2: client side support for a part to import external bundles
Mike Hommey <mh@glandium.org>
parents: 23011
diff changeset
1776 # return. But one need to start somewhere.
24686
e0e28e910fa3 bundle2: rename format, parts and config to final names
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 24648
diff changeset
1777 part = op.reply.newpart('reply:changegroup')
33701
24b7c75c6441 bundle2: use bytestr() instead of str() to convert part id to bytes
Augie Fackler <augie@google.com>
parents: 33700
diff changeset
1778 part.addparam(
24b7c75c6441 bundle2: use bytestr() instead of str() to convert part id to bytes
Augie Fackler <augie@google.com>
parents: 33700
diff changeset
1779 'in-reply-to', pycompat.bytestr(inpart.id), mandatory=False)
23029
149fc8a44184 bundle2: client side support for a part to import external bundles
Mike Hommey <mh@glandium.org>
parents: 23011
diff changeset
1780 part.addparam('return', '%i' % ret, mandatory=False)
149fc8a44184 bundle2: client side support for a part to import external bundles
Mike Hommey <mh@glandium.org>
parents: 23011
diff changeset
1781 try:
149fc8a44184 bundle2: client side support for a part to import external bundles
Mike Hommey <mh@glandium.org>
parents: 23011
diff changeset
1782 real_part.validate()
26587
56b2bcea2529 error: get Abort from 'error' instead of 'util'
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 26565
diff changeset
1783 except error.Abort as e:
56b2bcea2529 error: get Abort from 'error' instead of 'util'
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 26565
diff changeset
1784 raise error.Abort(_('bundle at %s is corrupted:\n%s') %
23029
149fc8a44184 bundle2: client side support for a part to import external bundles
Mike Hommey <mh@glandium.org>
parents: 23011
diff changeset
1785 (util.hidepassword(raw_url), str(e)))
149fc8a44184 bundle2: client side support for a part to import external bundles
Mike Hommey <mh@glandium.org>
parents: 23011
diff changeset
1786 assert not inpart.read()
149fc8a44184 bundle2: client side support for a part to import external bundles
Mike Hommey <mh@glandium.org>
parents: 23011
diff changeset
1787
24686
e0e28e910fa3 bundle2: rename format, parts and config to final names
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 24648
diff changeset
1788 @parthandler('reply:changegroup', ('return', 'in-reply-to'))
22548
8a1ae133770a bundle2: rename functions that have the same name
Mike Hommey <mh@glandium.org>
parents: 22390
diff changeset
1789 def handlereplychangegroup(op, inpart):
21611
71b7b3f79a3c bundle2: use the new ``part.params`` dictionary
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 21610
diff changeset
1790 ret = int(inpart.params['return'])
71b7b3f79a3c bundle2: use the new ``part.params`` dictionary
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 21610
diff changeset
1791 replyto = int(inpart.params['in-reply-to'])
71b7b3f79a3c bundle2: use the new ``part.params`` dictionary
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 21610
diff changeset
1792 op.records.add('changegroup', {'return': ret}, replyto)
20950
c7ceae0faf69 bundle2: first crude version of bundling changeset with bundle2
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 20949
diff changeset
1793
35267
dbf868623daf bookmark: add a 'check:bookmarks' bundle2 part
Boris Feld <boris.feld@octobus.net>
parents: 35141
diff changeset
1794 @parthandler('check:bookmarks')
dbf868623daf bookmark: add a 'check:bookmarks' bundle2 part
Boris Feld <boris.feld@octobus.net>
parents: 35141
diff changeset
1795 def handlecheckbookmarks(op, inpart):
dbf868623daf bookmark: add a 'check:bookmarks' bundle2 part
Boris Feld <boris.feld@octobus.net>
parents: 35141
diff changeset
1796 """check location of bookmarks
dbf868623daf bookmark: add a 'check:bookmarks' bundle2 part
Boris Feld <boris.feld@octobus.net>
parents: 35141
diff changeset
1797
dbf868623daf bookmark: add a 'check:bookmarks' bundle2 part
Boris Feld <boris.feld@octobus.net>
parents: 35141
diff changeset
1798 This part is to be used to detect push race regarding bookmark, it
dbf868623daf bookmark: add a 'check:bookmarks' bundle2 part
Boris Feld <boris.feld@octobus.net>
parents: 35141
diff changeset
1799 contains binary encoded (bookmark, node) tuple. If the local state does
dbf868623daf bookmark: add a 'check:bookmarks' bundle2 part
Boris Feld <boris.feld@octobus.net>
parents: 35141
diff changeset
1800 not marks the one in the part, a PushRaced exception is raised
dbf868623daf bookmark: add a 'check:bookmarks' bundle2 part
Boris Feld <boris.feld@octobus.net>
parents: 35141
diff changeset
1801 """
dbf868623daf bookmark: add a 'check:bookmarks' bundle2 part
Boris Feld <boris.feld@octobus.net>
parents: 35141
diff changeset
1802 bookdata = bookmarks.binarydecode(inpart)
dbf868623daf bookmark: add a 'check:bookmarks' bundle2 part
Boris Feld <boris.feld@octobus.net>
parents: 35141
diff changeset
1803
dbf868623daf bookmark: add a 'check:bookmarks' bundle2 part
Boris Feld <boris.feld@octobus.net>
parents: 35141
diff changeset
1804 msgstandard = ('repository changed while pushing - please try again '
dbf868623daf bookmark: add a 'check:bookmarks' bundle2 part
Boris Feld <boris.feld@octobus.net>
parents: 35141
diff changeset
1805 '(bookmark "%s" move from %s to %s)')
dbf868623daf bookmark: add a 'check:bookmarks' bundle2 part
Boris Feld <boris.feld@octobus.net>
parents: 35141
diff changeset
1806 msgmissing = ('repository changed while pushing - please try again '
dbf868623daf bookmark: add a 'check:bookmarks' bundle2 part
Boris Feld <boris.feld@octobus.net>
parents: 35141
diff changeset
1807 '(bookmark "%s" is missing, expected %s)')
dbf868623daf bookmark: add a 'check:bookmarks' bundle2 part
Boris Feld <boris.feld@octobus.net>
parents: 35141
diff changeset
1808 msgexist = ('repository changed while pushing - please try again '
dbf868623daf bookmark: add a 'check:bookmarks' bundle2 part
Boris Feld <boris.feld@octobus.net>
parents: 35141
diff changeset
1809 '(bookmark "%s" set on %s, expected missing)')
dbf868623daf bookmark: add a 'check:bookmarks' bundle2 part
Boris Feld <boris.feld@octobus.net>
parents: 35141
diff changeset
1810 for book, node in bookdata:
dbf868623daf bookmark: add a 'check:bookmarks' bundle2 part
Boris Feld <boris.feld@octobus.net>
parents: 35141
diff changeset
1811 currentnode = op.repo._bookmarks.get(book)
dbf868623daf bookmark: add a 'check:bookmarks' bundle2 part
Boris Feld <boris.feld@octobus.net>
parents: 35141
diff changeset
1812 if currentnode != node:
dbf868623daf bookmark: add a 'check:bookmarks' bundle2 part
Boris Feld <boris.feld@octobus.net>
parents: 35141
diff changeset
1813 if node is None:
dbf868623daf bookmark: add a 'check:bookmarks' bundle2 part
Boris Feld <boris.feld@octobus.net>
parents: 35141
diff changeset
1814 finalmsg = msgexist % (book, nodemod.short(currentnode))
dbf868623daf bookmark: add a 'check:bookmarks' bundle2 part
Boris Feld <boris.feld@octobus.net>
parents: 35141
diff changeset
1815 elif currentnode is None:
dbf868623daf bookmark: add a 'check:bookmarks' bundle2 part
Boris Feld <boris.feld@octobus.net>
parents: 35141
diff changeset
1816 finalmsg = msgmissing % (book, nodemod.short(node))
dbf868623daf bookmark: add a 'check:bookmarks' bundle2 part
Boris Feld <boris.feld@octobus.net>
parents: 35141
diff changeset
1817 else:
dbf868623daf bookmark: add a 'check:bookmarks' bundle2 part
Boris Feld <boris.feld@octobus.net>
parents: 35141
diff changeset
1818 finalmsg = msgstandard % (book, nodemod.short(node),
dbf868623daf bookmark: add a 'check:bookmarks' bundle2 part
Boris Feld <boris.feld@octobus.net>
parents: 35141
diff changeset
1819 nodemod.short(currentnode))
dbf868623daf bookmark: add a 'check:bookmarks' bundle2 part
Boris Feld <boris.feld@octobus.net>
parents: 35141
diff changeset
1820 raise error.PushRaced(finalmsg)
dbf868623daf bookmark: add a 'check:bookmarks' bundle2 part
Boris Feld <boris.feld@octobus.net>
parents: 35141
diff changeset
1821
24686
e0e28e910fa3 bundle2: rename format, parts and config to final names
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 24648
diff changeset
1822 @parthandler('check:heads')
22548
8a1ae133770a bundle2: rename functions that have the same name
Mike Hommey <mh@glandium.org>
parents: 22390
diff changeset
1823 def handlecheckheads(op, inpart):
21060
0bea9db7543b bundle2: add a "check:heads" handler
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 21024
diff changeset
1824 """check that head of the repo did not change
0bea9db7543b bundle2: add a "check:heads" handler
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 21024
diff changeset
1825
0bea9db7543b bundle2: add a "check:heads" handler
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 21024
diff changeset
1826 This is used to detect a push race when using unbundle.
0bea9db7543b bundle2: add a "check:heads" handler
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 21024
diff changeset
1827 This replaces the "heads" argument of unbundle."""
0bea9db7543b bundle2: add a "check:heads" handler
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 21024
diff changeset
1828 h = inpart.read(20)
0bea9db7543b bundle2: add a "check:heads" handler
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 21024
diff changeset
1829 heads = []
0bea9db7543b bundle2: add a "check:heads" handler
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 21024
diff changeset
1830 while len(h) == 20:
0bea9db7543b bundle2: add a "check:heads" handler
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 21024
diff changeset
1831 heads.append(h)
0bea9db7543b bundle2: add a "check:heads" handler
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 21024
diff changeset
1832 h = inpart.read(20)
0bea9db7543b bundle2: add a "check:heads" handler
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 21024
diff changeset
1833 assert not h
26565
ee1bcb9aa6e4 bundle2: add op.gettransaction() to handlers that need the lock
Durham Goode <durham@fb.com>
parents: 26542
diff changeset
1834 # Trigger a transaction so that we are guaranteed to have the lock now.
ee1bcb9aa6e4 bundle2: add op.gettransaction() to handlers that need the lock
Durham Goode <durham@fb.com>
parents: 26542
diff changeset
1835 if op.ui.configbool('experimental', 'bundle2lazylocking'):
ee1bcb9aa6e4 bundle2: add op.gettransaction() to handlers that need the lock
Durham Goode <durham@fb.com>
parents: 26542
diff changeset
1836 op.gettransaction()
29294
077d0535f51f bundle2: don't assume ordering of heads checked after push
Mads Kiilerich <madski@unity3d.com>
parents: 28883
diff changeset
1837 if sorted(heads) != sorted(op.repo.heads()):
21185
5b3717e1a3ea bundle2: add an error message to push race error
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 21184
diff changeset
1838 raise error.PushRaced('repository changed while pushing - '
5b3717e1a3ea bundle2: add an error message to push race error
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 21184
diff changeset
1839 'please try again')
21130
1ff06386217f bundle2: introduce `replycaps` part for on-demand reply
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 21129
diff changeset
1840
32729
16ada4cbb1a9 push: add a way to allow concurrent pushes on unrelated heads
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 32549
diff changeset
1841 @parthandler('check:updated-heads')
16ada4cbb1a9 push: add a way to allow concurrent pushes on unrelated heads
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 32549
diff changeset
1842 def handlecheckupdatedheads(op, inpart):
16ada4cbb1a9 push: add a way to allow concurrent pushes on unrelated heads
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 32549
diff changeset
1843 """check for race on the heads touched by a push
16ada4cbb1a9 push: add a way to allow concurrent pushes on unrelated heads
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 32549
diff changeset
1844
16ada4cbb1a9 push: add a way to allow concurrent pushes on unrelated heads
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 32549
diff changeset
1845 This is similar to 'check:heads' but focus on the heads actually updated
16ada4cbb1a9 push: add a way to allow concurrent pushes on unrelated heads
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 32549
diff changeset
1846 during the push. If other activities happen on unrelated heads, it is
16ada4cbb1a9 push: add a way to allow concurrent pushes on unrelated heads
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 32549
diff changeset
1847 ignored.
16ada4cbb1a9 push: add a way to allow concurrent pushes on unrelated heads
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 32549
diff changeset
1848
16ada4cbb1a9 push: add a way to allow concurrent pushes on unrelated heads
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 32549
diff changeset
1849 This allow server with high traffic to avoid push contention as long as
16ada4cbb1a9 push: add a way to allow concurrent pushes on unrelated heads
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 32549
diff changeset
1850 unrelated parts of the graph are involved."""
16ada4cbb1a9 push: add a way to allow concurrent pushes on unrelated heads
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 32549
diff changeset
1851 h = inpart.read(20)
16ada4cbb1a9 push: add a way to allow concurrent pushes on unrelated heads
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 32549
diff changeset
1852 heads = []
16ada4cbb1a9 push: add a way to allow concurrent pushes on unrelated heads
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 32549
diff changeset
1853 while len(h) == 20:
16ada4cbb1a9 push: add a way to allow concurrent pushes on unrelated heads
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 32549
diff changeset
1854 heads.append(h)
16ada4cbb1a9 push: add a way to allow concurrent pushes on unrelated heads
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 32549
diff changeset
1855 h = inpart.read(20)
16ada4cbb1a9 push: add a way to allow concurrent pushes on unrelated heads
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 32549
diff changeset
1856 assert not h
16ada4cbb1a9 push: add a way to allow concurrent pushes on unrelated heads
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 32549
diff changeset
1857 # trigger a transaction so that we are guaranteed to have the lock now.
16ada4cbb1a9 push: add a way to allow concurrent pushes on unrelated heads
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 32549
diff changeset
1858 if op.ui.configbool('experimental', 'bundle2lazylocking'):
16ada4cbb1a9 push: add a way to allow concurrent pushes on unrelated heads
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 32549
diff changeset
1859 op.gettransaction()
16ada4cbb1a9 push: add a way to allow concurrent pushes on unrelated heads
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 32549
diff changeset
1860
16ada4cbb1a9 push: add a way to allow concurrent pushes on unrelated heads
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 32549
diff changeset
1861 currentheads = set()
16ada4cbb1a9 push: add a way to allow concurrent pushes on unrelated heads
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 32549
diff changeset
1862 for ls in op.repo.branchmap().itervalues():
16ada4cbb1a9 push: add a way to allow concurrent pushes on unrelated heads
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 32549
diff changeset
1863 currentheads.update(ls)
16ada4cbb1a9 push: add a way to allow concurrent pushes on unrelated heads
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 32549
diff changeset
1864
16ada4cbb1a9 push: add a way to allow concurrent pushes on unrelated heads
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 32549
diff changeset
1865 for h in heads:
16ada4cbb1a9 push: add a way to allow concurrent pushes on unrelated heads
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 32549
diff changeset
1866 if h not in currentheads:
16ada4cbb1a9 push: add a way to allow concurrent pushes on unrelated heads
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 32549
diff changeset
1867 raise error.PushRaced('repository changed while pushing - '
16ada4cbb1a9 push: add a way to allow concurrent pushes on unrelated heads
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 32549
diff changeset
1868 'please try again')
16ada4cbb1a9 push: add a way to allow concurrent pushes on unrelated heads
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 32549
diff changeset
1869
34820
a95067b1dca6 phase: introduce a new 'check:phases' part
Boris Feld <boris.feld@octobus.net>
parents: 34637
diff changeset
1870 @parthandler('check:phases')
a95067b1dca6 phase: introduce a new 'check:phases' part
Boris Feld <boris.feld@octobus.net>
parents: 34637
diff changeset
1871 def handlecheckphases(op, inpart):
a95067b1dca6 phase: introduce a new 'check:phases' part
Boris Feld <boris.feld@octobus.net>
parents: 34637
diff changeset
1872 """check that phase boundaries of the repository did not change
a95067b1dca6 phase: introduce a new 'check:phases' part
Boris Feld <boris.feld@octobus.net>
parents: 34637
diff changeset
1873
a95067b1dca6 phase: introduce a new 'check:phases' part
Boris Feld <boris.feld@octobus.net>
parents: 34637
diff changeset
1874 This is used to detect a push race.
a95067b1dca6 phase: introduce a new 'check:phases' part
Boris Feld <boris.feld@octobus.net>
parents: 34637
diff changeset
1875 """
a95067b1dca6 phase: introduce a new 'check:phases' part
Boris Feld <boris.feld@octobus.net>
parents: 34637
diff changeset
1876 phasetonodes = phases.binarydecode(inpart)
a95067b1dca6 phase: introduce a new 'check:phases' part
Boris Feld <boris.feld@octobus.net>
parents: 34637
diff changeset
1877 unfi = op.repo.unfiltered()
a95067b1dca6 phase: introduce a new 'check:phases' part
Boris Feld <boris.feld@octobus.net>
parents: 34637
diff changeset
1878 cl = unfi.changelog
a95067b1dca6 phase: introduce a new 'check:phases' part
Boris Feld <boris.feld@octobus.net>
parents: 34637
diff changeset
1879 phasecache = unfi._phasecache
a95067b1dca6 phase: introduce a new 'check:phases' part
Boris Feld <boris.feld@octobus.net>
parents: 34637
diff changeset
1880 msg = ('repository changed while pushing - please try again '
a95067b1dca6 phase: introduce a new 'check:phases' part
Boris Feld <boris.feld@octobus.net>
parents: 34637
diff changeset
1881 '(%s is %s expected %s)')
a95067b1dca6 phase: introduce a new 'check:phases' part
Boris Feld <boris.feld@octobus.net>
parents: 34637
diff changeset
1882 for expectedphase, nodes in enumerate(phasetonodes):
a95067b1dca6 phase: introduce a new 'check:phases' part
Boris Feld <boris.feld@octobus.net>
parents: 34637
diff changeset
1883 for n in nodes:
a95067b1dca6 phase: introduce a new 'check:phases' part
Boris Feld <boris.feld@octobus.net>
parents: 34637
diff changeset
1884 actualphase = phasecache.phase(unfi, cl.rev(n))
a95067b1dca6 phase: introduce a new 'check:phases' part
Boris Feld <boris.feld@octobus.net>
parents: 34637
diff changeset
1885 if actualphase != expectedphase:
a95067b1dca6 phase: introduce a new 'check:phases' part
Boris Feld <boris.feld@octobus.net>
parents: 34637
diff changeset
1886 finalmsg = msg % (nodemod.short(n),
a95067b1dca6 phase: introduce a new 'check:phases' part
Boris Feld <boris.feld@octobus.net>
parents: 34637
diff changeset
1887 phases.phasenames[actualphase],
a95067b1dca6 phase: introduce a new 'check:phases' part
Boris Feld <boris.feld@octobus.net>
parents: 34637
diff changeset
1888 phases.phasenames[expectedphase])
a95067b1dca6 phase: introduce a new 'check:phases' part
Boris Feld <boris.feld@octobus.net>
parents: 34637
diff changeset
1889 raise error.PushRaced(finalmsg)
a95067b1dca6 phase: introduce a new 'check:phases' part
Boris Feld <boris.feld@octobus.net>
parents: 34637
diff changeset
1890
24686
e0e28e910fa3 bundle2: rename format, parts and config to final names
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 24648
diff changeset
1891 @parthandler('output')
21131
b7435117d951 bundle2: capture remote stdout while unbundling
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 21130
diff changeset
1892 def handleoutput(op, inpart):
b7435117d951 bundle2: capture remote stdout while unbundling
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 21130
diff changeset
1893 """forward output captured on the server to the client"""
b7435117d951 bundle2: capture remote stdout while unbundling
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 21130
diff changeset
1894 for line in inpart.read().splitlines():
29664
ccd436f7db6d bundle2: localize handleoutput remote prompts
Akihiko Odaki <akihiko.odaki.4i@stu.hosei.ac.jp>
parents: 29662
diff changeset
1895 op.ui.status(_('remote: %s\n') % line)
21131
b7435117d951 bundle2: capture remote stdout while unbundling
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 21130
diff changeset
1896
24686
e0e28e910fa3 bundle2: rename format, parts and config to final names
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 24648
diff changeset
1897 @parthandler('replycaps')
21130
1ff06386217f bundle2: introduce `replycaps` part for on-demand reply
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 21129
diff changeset
1898 def handlereplycaps(op, inpart):
1ff06386217f bundle2: introduce `replycaps` part for on-demand reply
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 21129
diff changeset
1899 """Notify that a reply bundle should be created
1ff06386217f bundle2: introduce `replycaps` part for on-demand reply
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 21129
diff changeset
1900
21138
f469879d27ec bundle2: extract capabilities decoding
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 21137
diff changeset
1901 The payload contains the capabilities information for the reply"""
f469879d27ec bundle2: extract capabilities decoding
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 21137
diff changeset
1902 caps = decodecaps(inpart.read())
21130
1ff06386217f bundle2: introduce `replycaps` part for on-demand reply
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 21129
diff changeset
1903 if op.reply is None:
21135
98fbf3adfd83 bundle2: add capabilities support in `replycaps` part
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 21134
diff changeset
1904 op.reply = bundle20(op.ui, caps)
21131
b7435117d951 bundle2: capture remote stdout while unbundling
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 21130
diff changeset
1905
26829
58f1645f72c3 bundle2: attribute remote failures to remote (issue4788)
Gregory Szorc <gregory.szorc@gmail.com>
parents: 26802
diff changeset
1906 class AbortFromPart(error.Abort):
58f1645f72c3 bundle2: attribute remote failures to remote (issue4788)
Gregory Szorc <gregory.szorc@gmail.com>
parents: 26802
diff changeset
1907 """Sub-class of Abort that denotes an error from a bundle2 part."""
58f1645f72c3 bundle2: attribute remote failures to remote (issue4788)
Gregory Szorc <gregory.szorc@gmail.com>
parents: 26802
diff changeset
1908
24686
e0e28e910fa3 bundle2: rename format, parts and config to final names
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 24648
diff changeset
1909 @parthandler('error:abort', ('message', 'hint'))
24741
bb67e52362d6 bundle2: fix names for error part handler
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 24733
diff changeset
1910 def handleerrorabort(op, inpart):
21177
952af771bc17 bundle2: gracefully handle abort during unbundle
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 21176
diff changeset
1911 """Used to transmit abort error over the wire"""
26829
58f1645f72c3 bundle2: attribute remote failures to remote (issue4788)
Gregory Szorc <gregory.szorc@gmail.com>
parents: 26802
diff changeset
1912 raise AbortFromPart(inpart.params['message'],
58f1645f72c3 bundle2: attribute remote failures to remote (issue4788)
Gregory Szorc <gregory.szorc@gmail.com>
parents: 26802
diff changeset
1913 hint=inpart.params.get('hint'))
21183
4345274adc4b bundle2: gracefully handle UnknownPartError during unbundle
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 21179
diff changeset
1914
25493
d8e7b0781ad7 bundle2: convey PushkeyFailed error over the wire
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 25492
diff changeset
1915 @parthandler('error:pushkey', ('namespace', 'key', 'new', 'old', 'ret',
d8e7b0781ad7 bundle2: convey PushkeyFailed error over the wire
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 25492
diff changeset
1916 'in-reply-to'))
d8e7b0781ad7 bundle2: convey PushkeyFailed error over the wire
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 25492
diff changeset
1917 def handleerrorpushkey(op, inpart):
d8e7b0781ad7 bundle2: convey PushkeyFailed error over the wire
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 25492
diff changeset
1918 """Used to transmit failure of a mandatory pushkey over the wire"""
d8e7b0781ad7 bundle2: convey PushkeyFailed error over the wire
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 25492
diff changeset
1919 kwargs = {}
d8e7b0781ad7 bundle2: convey PushkeyFailed error over the wire
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 25492
diff changeset
1920 for name in ('namespace', 'key', 'new', 'old', 'ret'):
d8e7b0781ad7 bundle2: convey PushkeyFailed error over the wire
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 25492
diff changeset
1921 value = inpart.params.get(name)
d8e7b0781ad7 bundle2: convey PushkeyFailed error over the wire
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 25492
diff changeset
1922 if value is not None:
d8e7b0781ad7 bundle2: convey PushkeyFailed error over the wire
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 25492
diff changeset
1923 kwargs[name] = value
d8e7b0781ad7 bundle2: convey PushkeyFailed error over the wire
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 25492
diff changeset
1924 raise error.PushkeyFailed(inpart.params['in-reply-to'], **kwargs)
d8e7b0781ad7 bundle2: convey PushkeyFailed error over the wire
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 25492
diff changeset
1925
24686
e0e28e910fa3 bundle2: rename format, parts and config to final names
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 24648
diff changeset
1926 @parthandler('error:unsupportedcontent', ('parttype', 'params'))
24741
bb67e52362d6 bundle2: fix names for error part handler
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 24733
diff changeset
1927 def handleerrorunsupportedcontent(op, inpart):
21619
292331e906d7 bundle2: rename b2x:error:unknownpart to b2x:error:unsupportedcontent
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 21618
diff changeset
1928 """Used to transmit unknown content error over the wire"""
21622
457492741007 bundle2: support transmission of params error over the wire
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 21620
diff changeset
1929 kwargs = {}
21627
3e8bcc90f07c bundle2: support None parttype in BundleValueError
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 21626
diff changeset
1930 parttype = inpart.params.get('parttype')
3e8bcc90f07c bundle2: support None parttype in BundleValueError
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 21626
diff changeset
1931 if parttype is not None:
3e8bcc90f07c bundle2: support None parttype in BundleValueError
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 21626
diff changeset
1932 kwargs['parttype'] = parttype
21622
457492741007 bundle2: support transmission of params error over the wire
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 21620
diff changeset
1933 params = inpart.params.get('params')
457492741007 bundle2: support transmission of params error over the wire
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 21620
diff changeset
1934 if params is not None:
457492741007 bundle2: support transmission of params error over the wire
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 21620
diff changeset
1935 kwargs['params'] = params.split('\0')
457492741007 bundle2: support transmission of params error over the wire
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 21620
diff changeset
1936
26393
cff70549a959 bundle2: rename error exception class for unsupported feature
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 26269
diff changeset
1937 raise error.BundleUnknownFeatureError(**kwargs)
21186
9f3652e851f8 bundle2: gracefully handle PushRaced error during unbundle
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 21185
diff changeset
1938
24686
e0e28e910fa3 bundle2: rename format, parts and config to final names
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 24648
diff changeset
1939 @parthandler('error:pushraced', ('message',))
24741
bb67e52362d6 bundle2: fix names for error part handler
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 24733
diff changeset
1940 def handleerrorpushraced(op, inpart):
21186
9f3652e851f8 bundle2: gracefully handle PushRaced error during unbundle
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 21185
diff changeset
1941 """Used to transmit push race error over the wire"""
21611
71b7b3f79a3c bundle2: use the new ``part.params`` dictionary
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 21610
diff changeset
1942 raise error.ResponseError(_('push failed:'), inpart.params['message'])
21655
35095f332846 bundle: introduce a listkey handler
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 21644
diff changeset
1943
24686
e0e28e910fa3 bundle2: rename format, parts and config to final names
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 24648
diff changeset
1944 @parthandler('listkeys', ('namespace',))
21655
35095f332846 bundle: introduce a listkey handler
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 21644
diff changeset
1945 def handlelistkeys(op, inpart):
35095f332846 bundle: introduce a listkey handler
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 21644
diff changeset
1946 """retrieve pushkey namespace content stored in a bundle2"""
35095f332846 bundle: introduce a listkey handler
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 21644
diff changeset
1947 namespace = inpart.params['namespace']
35095f332846 bundle: introduce a listkey handler
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 21644
diff changeset
1948 r = pushkey.decodekeys(inpart.read())
35095f332846 bundle: introduce a listkey handler
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 21644
diff changeset
1949 op.records.add('listkeys', (namespace, r))
21660
e87d2a12d41b bundle2: add ``pushkey`` support
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 21655
diff changeset
1950
24686
e0e28e910fa3 bundle2: rename format, parts and config to final names
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 24648
diff changeset
1951 @parthandler('pushkey', ('namespace', 'key', 'old', 'new'))
21660
e87d2a12d41b bundle2: add ``pushkey`` support
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 21655
diff changeset
1952 def handlepushkey(op, inpart):
e87d2a12d41b bundle2: add ``pushkey`` support
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 21655
diff changeset
1953 """process a pushkey request"""
e87d2a12d41b bundle2: add ``pushkey`` support
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 21655
diff changeset
1954 dec = pushkey.decode
e87d2a12d41b bundle2: add ``pushkey`` support
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 21655
diff changeset
1955 namespace = dec(inpart.params['namespace'])
e87d2a12d41b bundle2: add ``pushkey`` support
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 21655
diff changeset
1956 key = dec(inpart.params['key'])
e87d2a12d41b bundle2: add ``pushkey`` support
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 21655
diff changeset
1957 old = dec(inpart.params['old'])
e87d2a12d41b bundle2: add ``pushkey`` support
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 21655
diff changeset
1958 new = dec(inpart.params['new'])
26565
ee1bcb9aa6e4 bundle2: add op.gettransaction() to handlers that need the lock
Durham Goode <durham@fb.com>
parents: 26542
diff changeset
1959 # Grab the transaction to ensure that we have the lock before performing the
ee1bcb9aa6e4 bundle2: add op.gettransaction() to handlers that need the lock
Durham Goode <durham@fb.com>
parents: 26542
diff changeset
1960 # pushkey.
ee1bcb9aa6e4 bundle2: add op.gettransaction() to handlers that need the lock
Durham Goode <durham@fb.com>
parents: 26542
diff changeset
1961 if op.ui.configbool('experimental', 'bundle2lazylocking'):
ee1bcb9aa6e4 bundle2: add op.gettransaction() to handlers that need the lock
Durham Goode <durham@fb.com>
parents: 26542
diff changeset
1962 op.gettransaction()
21660
e87d2a12d41b bundle2: add ``pushkey`` support
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 21655
diff changeset
1963 ret = op.repo.pushkey(namespace, key, old, new)
e87d2a12d41b bundle2: add ``pushkey`` support
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 21655
diff changeset
1964 record = {'namespace': namespace,
e87d2a12d41b bundle2: add ``pushkey`` support
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 21655
diff changeset
1965 'key': key,
e87d2a12d41b bundle2: add ``pushkey`` support
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 21655
diff changeset
1966 'old': old,
e87d2a12d41b bundle2: add ``pushkey`` support
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 21655
diff changeset
1967 'new': new}
e87d2a12d41b bundle2: add ``pushkey`` support
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 21655
diff changeset
1968 op.records.add('pushkey', record)
e87d2a12d41b bundle2: add ``pushkey`` support
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 21655
diff changeset
1969 if op.reply is not None:
24686
e0e28e910fa3 bundle2: rename format, parts and config to final names
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 24648
diff changeset
1970 rpart = op.reply.newpart('reply:pushkey')
33701
24b7c75c6441 bundle2: use bytestr() instead of str() to convert part id to bytes
Augie Fackler <augie@google.com>
parents: 33700
diff changeset
1971 rpart.addparam(
24b7c75c6441 bundle2: use bytestr() instead of str() to convert part id to bytes
Augie Fackler <augie@google.com>
parents: 33700
diff changeset
1972 'in-reply-to', pycompat.bytestr(inpart.id), mandatory=False)
21660
e87d2a12d41b bundle2: add ``pushkey`` support
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 21655
diff changeset
1973 rpart.addparam('return', '%i' % ret, mandatory=False)
25481
6de96cb31176 bundle2: abort when a mandatory pushkey part fails
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 25401
diff changeset
1974 if inpart.mandatory and not ret:
25484
a5192774e925 bundle2: introduce a PushkeyFail error to abort unbundle on pushkey error
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 25481
diff changeset
1975 kwargs = {}
a5192774e925 bundle2: introduce a PushkeyFail error to abort unbundle on pushkey error
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 25481
diff changeset
1976 for key in ('namespace', 'key', 'new', 'old', 'ret'):
a5192774e925 bundle2: introduce a PushkeyFail error to abort unbundle on pushkey error
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 25481
diff changeset
1977 if key in inpart.params:
a5192774e925 bundle2: introduce a PushkeyFail error to abort unbundle on pushkey error
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 25481
diff changeset
1978 kwargs[key] = inpart.params[key]
a5192774e925 bundle2: introduce a PushkeyFail error to abort unbundle on pushkey error
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 25481
diff changeset
1979 raise error.PushkeyFailed(partid=str(inpart.id), **kwargs)
21660
e87d2a12d41b bundle2: add ``pushkey`` support
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 21655
diff changeset
1980
35269
af5507203d01 bookmark: introduce a 'bookmarks' part
Boris Feld <boris.feld@octobus.net>
parents: 35268
diff changeset
1981 @parthandler('bookmarks')
af5507203d01 bookmark: introduce a 'bookmarks' part
Boris Feld <boris.feld@octobus.net>
parents: 35268
diff changeset
1982 def handlebookmark(op, inpart):
af5507203d01 bookmark: introduce a 'bookmarks' part
Boris Feld <boris.feld@octobus.net>
parents: 35268
diff changeset
1983 """transmit bookmark information
af5507203d01 bookmark: introduce a 'bookmarks' part
Boris Feld <boris.feld@octobus.net>
parents: 35268
diff changeset
1984
af5507203d01 bookmark: introduce a 'bookmarks' part
Boris Feld <boris.feld@octobus.net>
parents: 35268
diff changeset
1985 The part contains binary encoded bookmark information. The bookmark
af5507203d01 bookmark: introduce a 'bookmarks' part
Boris Feld <boris.feld@octobus.net>
parents: 35268
diff changeset
1986 information is applied as is to the unbundling repository. Make sure a
af5507203d01 bookmark: introduce a 'bookmarks' part
Boris Feld <boris.feld@octobus.net>
parents: 35268
diff changeset
1987 'check:bookmarks' part is issued earlier to check for race condition in
af5507203d01 bookmark: introduce a 'bookmarks' part
Boris Feld <boris.feld@octobus.net>
parents: 35268
diff changeset
1988 such update.
af5507203d01 bookmark: introduce a 'bookmarks' part
Boris Feld <boris.feld@octobus.net>
parents: 35268
diff changeset
1989
af5507203d01 bookmark: introduce a 'bookmarks' part
Boris Feld <boris.feld@octobus.net>
parents: 35268
diff changeset
1990 This behavior is suitable for pushing. Semantic adjustment will be needed
af5507203d01 bookmark: introduce a 'bookmarks' part
Boris Feld <boris.feld@octobus.net>
parents: 35268
diff changeset
1991 for pull.
af5507203d01 bookmark: introduce a 'bookmarks' part
Boris Feld <boris.feld@octobus.net>
parents: 35268
diff changeset
1992 """
af5507203d01 bookmark: introduce a 'bookmarks' part
Boris Feld <boris.feld@octobus.net>
parents: 35268
diff changeset
1993 changes = bookmarks.binarydecode(inpart)
35270
f392066d127c bookmark: add pushkey hook compatiblity to the bundle2 part
Boris Feld <boris.feld@octobus.net>
parents: 35269
diff changeset
1994
f392066d127c bookmark: add pushkey hook compatiblity to the bundle2 part
Boris Feld <boris.feld@octobus.net>
parents: 35269
diff changeset
1995 tr = op.gettransaction()
f392066d127c bookmark: add pushkey hook compatiblity to the bundle2 part
Boris Feld <boris.feld@octobus.net>
parents: 35269
diff changeset
1996 bookstore = op.repo._bookmarks
f392066d127c bookmark: add pushkey hook compatiblity to the bundle2 part
Boris Feld <boris.feld@octobus.net>
parents: 35269
diff changeset
1997
f392066d127c bookmark: add pushkey hook compatiblity to the bundle2 part
Boris Feld <boris.feld@octobus.net>
parents: 35269
diff changeset
1998 pushkeycompat = op.repo.ui.configbool('server', 'bookmarks-pushkey-compat')
f392066d127c bookmark: add pushkey hook compatiblity to the bundle2 part
Boris Feld <boris.feld@octobus.net>
parents: 35269
diff changeset
1999 if pushkeycompat:
f392066d127c bookmark: add pushkey hook compatiblity to the bundle2 part
Boris Feld <boris.feld@octobus.net>
parents: 35269
diff changeset
2000 allhooks = []
f392066d127c bookmark: add pushkey hook compatiblity to the bundle2 part
Boris Feld <boris.feld@octobus.net>
parents: 35269
diff changeset
2001 for book, node in changes:
f392066d127c bookmark: add pushkey hook compatiblity to the bundle2 part
Boris Feld <boris.feld@octobus.net>
parents: 35269
diff changeset
2002 hookargs = tr.hookargs.copy()
f392066d127c bookmark: add pushkey hook compatiblity to the bundle2 part
Boris Feld <boris.feld@octobus.net>
parents: 35269
diff changeset
2003 hookargs['pushkeycompat'] = '1'
f392066d127c bookmark: add pushkey hook compatiblity to the bundle2 part
Boris Feld <boris.feld@octobus.net>
parents: 35269
diff changeset
2004 hookargs['namespace'] = 'bookmark'
f392066d127c bookmark: add pushkey hook compatiblity to the bundle2 part
Boris Feld <boris.feld@octobus.net>
parents: 35269
diff changeset
2005 hookargs['key'] = book
f392066d127c bookmark: add pushkey hook compatiblity to the bundle2 part
Boris Feld <boris.feld@octobus.net>
parents: 35269
diff changeset
2006 hookargs['old'] = nodemod.hex(bookstore.get(book, ''))
f392066d127c bookmark: add pushkey hook compatiblity to the bundle2 part
Boris Feld <boris.feld@octobus.net>
parents: 35269
diff changeset
2007 hookargs['new'] = nodemod.hex(node if node is not None else '')
f392066d127c bookmark: add pushkey hook compatiblity to the bundle2 part
Boris Feld <boris.feld@octobus.net>
parents: 35269
diff changeset
2008 allhooks.append(hookargs)
f392066d127c bookmark: add pushkey hook compatiblity to the bundle2 part
Boris Feld <boris.feld@octobus.net>
parents: 35269
diff changeset
2009 for hookargs in allhooks:
f392066d127c bookmark: add pushkey hook compatiblity to the bundle2 part
Boris Feld <boris.feld@octobus.net>
parents: 35269
diff changeset
2010 op.repo.hook('prepushkey', throw=True, **hookargs)
f392066d127c bookmark: add pushkey hook compatiblity to the bundle2 part
Boris Feld <boris.feld@octobus.net>
parents: 35269
diff changeset
2011
f392066d127c bookmark: add pushkey hook compatiblity to the bundle2 part
Boris Feld <boris.feld@octobus.net>
parents: 35269
diff changeset
2012 bookstore.applychanges(op.repo, tr, changes)
f392066d127c bookmark: add pushkey hook compatiblity to the bundle2 part
Boris Feld <boris.feld@octobus.net>
parents: 35269
diff changeset
2013
f392066d127c bookmark: add pushkey hook compatiblity to the bundle2 part
Boris Feld <boris.feld@octobus.net>
parents: 35269
diff changeset
2014 if pushkeycompat:
f392066d127c bookmark: add pushkey hook compatiblity to the bundle2 part
Boris Feld <boris.feld@octobus.net>
parents: 35269
diff changeset
2015 def runhook():
f392066d127c bookmark: add pushkey hook compatiblity to the bundle2 part
Boris Feld <boris.feld@octobus.net>
parents: 35269
diff changeset
2016 for hookargs in allhooks:
f392066d127c bookmark: add pushkey hook compatiblity to the bundle2 part
Boris Feld <boris.feld@octobus.net>
parents: 35269
diff changeset
2017 op.repo.hook('prepushkey', **hookargs)
f392066d127c bookmark: add pushkey hook compatiblity to the bundle2 part
Boris Feld <boris.feld@octobus.net>
parents: 35269
diff changeset
2018 op.repo._afterlock(runhook)
35269
af5507203d01 bookmark: introduce a 'bookmarks' part
Boris Feld <boris.feld@octobus.net>
parents: 35268
diff changeset
2019
33043
e8c8d81eb864 bundle: add config option to include phases
Martin von Zweigbergk <martinvonz@google.com>
parents: 33042
diff changeset
2020 @parthandler('phase-heads')
e8c8d81eb864 bundle: add config option to include phases
Martin von Zweigbergk <martinvonz@google.com>
parents: 33042
diff changeset
2021 def handlephases(op, inpart):
e8c8d81eb864 bundle: add config option to include phases
Martin von Zweigbergk <martinvonz@google.com>
parents: 33042
diff changeset
2022 """apply phases from bundle part to repo"""
34327
12c42bcd4133 phases: move the binary decoding function in the phases module
Boris Feld <boris.feld@octobus.net>
parents: 34326
diff changeset
2023 headsbyphase = phases.binarydecode(inpart)
34328
4ef472b975ff bundle2: only grab a transaction when 'phase-heads' affect the repository
Boris Feld <boris.feld@octobus.net>
parents: 34327
diff changeset
2024 phases.updatephases(op.repo.unfiltered(), op.gettransaction, headsbyphase)
33043
e8c8d81eb864 bundle: add config option to include phases
Martin von Zweigbergk <martinvonz@google.com>
parents: 33042
diff changeset
2025
24686
e0e28e910fa3 bundle2: rename format, parts and config to final names
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 24648
diff changeset
2026 @parthandler('reply:pushkey', ('return', 'in-reply-to'))
21660
e87d2a12d41b bundle2: add ``pushkey`` support
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 21655
diff changeset
2027 def handlepushkeyreply(op, inpart):
e87d2a12d41b bundle2: add ``pushkey`` support
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 21655
diff changeset
2028 """retrieve the result of a pushkey request"""
e87d2a12d41b bundle2: add ``pushkey`` support
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 21655
diff changeset
2029 ret = int(inpart.params['return'])
e87d2a12d41b bundle2: add ``pushkey`` support
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 21655
diff changeset
2030 partid = int(inpart.params['in-reply-to'])
e87d2a12d41b bundle2: add ``pushkey`` support
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 21655
diff changeset
2031 op.records.add('pushkey', {'return': ret}, partid)
22336
60786c8a2f70 bundle2: add an obsmarkers part handler
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 21900
diff changeset
2032
24686
e0e28e910fa3 bundle2: rename format, parts and config to final names
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 24648
diff changeset
2033 @parthandler('obsmarkers')
22336
60786c8a2f70 bundle2: add an obsmarkers part handler
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 21900
diff changeset
2034 def handleobsmarker(op, inpart):
60786c8a2f70 bundle2: add an obsmarkers part handler
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 21900
diff changeset
2035 """add a stream of obsmarkers to the repo"""
60786c8a2f70 bundle2: add an obsmarkers part handler
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 21900
diff changeset
2036 tr = op.gettransaction()
24733
c00e4338fa4b obsolete: experimental flag to get debug about obsmarkers exchange
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 24686
diff changeset
2037 markerdata = inpart.read()
33499
0407a51b9d8c codemod: register core configitems using a script
Jun Wu <quark@fb.com>
parents: 33461
diff changeset
2038 if op.ui.config('experimental', 'obsmarkers-exchange-debug'):
24733
c00e4338fa4b obsolete: experimental flag to get debug about obsmarkers exchange
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 24686
diff changeset
2039 op.ui.write(('obsmarker-exchange: %i bytes received\n')
c00e4338fa4b obsolete: experimental flag to get debug about obsmarkers exchange
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 24686
diff changeset
2040 % len(markerdata))
26685
ea390d889d3a bundle2: gracefully skip 'obsmarkers' part if evolution is disabled
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 26587
diff changeset
2041 # The mergemarkers call will crash if marker creation is not enabled.
ea390d889d3a bundle2: gracefully skip 'obsmarkers' part if evolution is disabled
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 26587
diff changeset
2042 # we want to avoid this if the part is advisory.
ea390d889d3a bundle2: gracefully skip 'obsmarkers' part if evolution is disabled
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 26587
diff changeset
2043 if not inpart.mandatory and op.repo.obsstore.readonly:
ea390d889d3a bundle2: gracefully skip 'obsmarkers' part if evolution is disabled
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 26587
diff changeset
2044 op.repo.ui.debug('ignoring obsolescence markers, feature not enabled')
ea390d889d3a bundle2: gracefully skip 'obsmarkers' part if evolution is disabled
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 26587
diff changeset
2045 return
24733
c00e4338fa4b obsolete: experimental flag to get debug about obsmarkers exchange
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 24686
diff changeset
2046 new = op.repo.obsstore.mergemarkers(tr, markerdata)
32124
99515353c72a obsolete: invalidate "volatile" set cache after merging marker
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 32024
diff changeset
2047 op.repo.invalidatevolatilesets()
22337
c380fe290290 obsmarker: write a message with the number of markers added through bundle2
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 22336
diff changeset
2048 if new:
c380fe290290 obsmarker: write a message with the number of markers added through bundle2
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 22336
diff changeset
2049 op.repo.ui.status(_('%i new obsolescence markers\n') % new)
22338
8fcd56095d3b obssmarker: add a bundle2 record with the number of markers added
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 22337
diff changeset
2050 op.records.add('obsmarkers', {'new': new})
22340
394a17de6a2d obsmarker: produce a reply part for markers received through bundle2
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 22338
diff changeset
2051 if op.reply is not None:
24686
e0e28e910fa3 bundle2: rename format, parts and config to final names
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 24648
diff changeset
2052 rpart = op.reply.newpart('reply:obsmarkers')
33701
24b7c75c6441 bundle2: use bytestr() instead of str() to convert part id to bytes
Augie Fackler <augie@google.com>
parents: 33700
diff changeset
2053 rpart.addparam(
24b7c75c6441 bundle2: use bytestr() instead of str() to convert part id to bytes
Augie Fackler <augie@google.com>
parents: 33700
diff changeset
2054 'in-reply-to', pycompat.bytestr(inpart.id), mandatory=False)
22340
394a17de6a2d obsmarker: produce a reply part for markers received through bundle2
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 22338
diff changeset
2055 rpart.addparam('new', '%i' % new, mandatory=False)
394a17de6a2d obsmarker: produce a reply part for markers received through bundle2
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 22338
diff changeset
2056
394a17de6a2d obsmarker: produce a reply part for markers received through bundle2
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 22338
diff changeset
2057
24686
e0e28e910fa3 bundle2: rename format, parts and config to final names
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 24648
diff changeset
2058 @parthandler('reply:obsmarkers', ('new', 'in-reply-to'))
25506
86472038dfd2 bundle2: rename duplicate handlepushkeyreply to handleobsmarkerreply
Martin von Zweigbergk <martinvonz@google.com>
parents: 25493
diff changeset
2059 def handleobsmarkerreply(op, inpart):
22340
394a17de6a2d obsmarker: produce a reply part for markers received through bundle2
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 22338
diff changeset
2060 """retrieve the result of a pushkey request"""
394a17de6a2d obsmarker: produce a reply part for markers received through bundle2
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 22338
diff changeset
2061 ret = int(inpart.params['new'])
394a17de6a2d obsmarker: produce a reply part for markers received through bundle2
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 22338
diff changeset
2062 partid = int(inpart.params['in-reply-to'])
394a17de6a2d obsmarker: produce a reply part for markers received through bundle2
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 22338
diff changeset
2063 op.records.add('obsmarkers', {'new': ret}, partid)
25401
d29201352af7 bundle2: part handler for processing .hgtags fnodes mappings
Gregory Szorc <gregory.szorc@gmail.com>
parents: 25336
diff changeset
2064
d29201352af7 bundle2: part handler for processing .hgtags fnodes mappings
Gregory Szorc <gregory.szorc@gmail.com>
parents: 25336
diff changeset
2065 @parthandler('hgtagsfnodes')
d29201352af7 bundle2: part handler for processing .hgtags fnodes mappings
Gregory Szorc <gregory.szorc@gmail.com>
parents: 25336
diff changeset
2066 def handlehgtagsfnodes(op, inpart):
d29201352af7 bundle2: part handler for processing .hgtags fnodes mappings
Gregory Szorc <gregory.szorc@gmail.com>
parents: 25336
diff changeset
2067 """Applies .hgtags fnodes cache entries to the local repo.
d29201352af7 bundle2: part handler for processing .hgtags fnodes mappings
Gregory Szorc <gregory.szorc@gmail.com>
parents: 25336
diff changeset
2068
d29201352af7 bundle2: part handler for processing .hgtags fnodes mappings
Gregory Szorc <gregory.szorc@gmail.com>
parents: 25336
diff changeset
2069 Payload is pairs of 20 byte changeset nodes and filenodes.
d29201352af7 bundle2: part handler for processing .hgtags fnodes mappings
Gregory Szorc <gregory.szorc@gmail.com>
parents: 25336
diff changeset
2070 """
26565
ee1bcb9aa6e4 bundle2: add op.gettransaction() to handlers that need the lock
Durham Goode <durham@fb.com>
parents: 26542
diff changeset
2071 # Grab the transaction so we ensure that we have the lock at this point.
ee1bcb9aa6e4 bundle2: add op.gettransaction() to handlers that need the lock
Durham Goode <durham@fb.com>
parents: 26542
diff changeset
2072 if op.ui.configbool('experimental', 'bundle2lazylocking'):
ee1bcb9aa6e4 bundle2: add op.gettransaction() to handlers that need the lock
Durham Goode <durham@fb.com>
parents: 26542
diff changeset
2073 op.gettransaction()
25401
d29201352af7 bundle2: part handler for processing .hgtags fnodes mappings
Gregory Szorc <gregory.szorc@gmail.com>
parents: 25336
diff changeset
2074 cache = tags.hgtagsfnodescache(op.repo.unfiltered())
d29201352af7 bundle2: part handler for processing .hgtags fnodes mappings
Gregory Szorc <gregory.szorc@gmail.com>
parents: 25336
diff changeset
2075
d29201352af7 bundle2: part handler for processing .hgtags fnodes mappings
Gregory Szorc <gregory.szorc@gmail.com>
parents: 25336
diff changeset
2076 count = 0
d29201352af7 bundle2: part handler for processing .hgtags fnodes mappings
Gregory Szorc <gregory.szorc@gmail.com>
parents: 25336
diff changeset
2077 while True:
d29201352af7 bundle2: part handler for processing .hgtags fnodes mappings
Gregory Szorc <gregory.szorc@gmail.com>
parents: 25336
diff changeset
2078 node = inpart.read(20)
d29201352af7 bundle2: part handler for processing .hgtags fnodes mappings
Gregory Szorc <gregory.szorc@gmail.com>
parents: 25336
diff changeset
2079 fnode = inpart.read(20)
d29201352af7 bundle2: part handler for processing .hgtags fnodes mappings
Gregory Szorc <gregory.szorc@gmail.com>
parents: 25336
diff changeset
2080 if len(node) < 20 or len(fnode) < 20:
25641
c0bdfe87b245 bundle2: reword debug message for invalid .hgtags data
Gregory Szorc <gregory.szorc@gmail.com>
parents: 25640
diff changeset
2081 op.ui.debug('ignoring incomplete received .hgtags fnodes data\n')
25401
d29201352af7 bundle2: part handler for processing .hgtags fnodes mappings
Gregory Szorc <gregory.szorc@gmail.com>
parents: 25336
diff changeset
2082 break
d29201352af7 bundle2: part handler for processing .hgtags fnodes mappings
Gregory Szorc <gregory.szorc@gmail.com>
parents: 25336
diff changeset
2083 cache.setfnode(node, fnode)
d29201352af7 bundle2: part handler for processing .hgtags fnodes mappings
Gregory Szorc <gregory.szorc@gmail.com>
parents: 25336
diff changeset
2084 count += 1
d29201352af7 bundle2: part handler for processing .hgtags fnodes mappings
Gregory Szorc <gregory.szorc@gmail.com>
parents: 25336
diff changeset
2085
d29201352af7 bundle2: part handler for processing .hgtags fnodes mappings
Gregory Szorc <gregory.szorc@gmail.com>
parents: 25336
diff changeset
2086 cache.write()
d29201352af7 bundle2: part handler for processing .hgtags fnodes mappings
Gregory Szorc <gregory.szorc@gmail.com>
parents: 25336
diff changeset
2087 op.ui.debug('applied %i hgtags fnodes cache entries\n' % count)
33719
db3dc11356ed pushvars: move fb extension pushvars to core
Pulkit Goyal <7895pulkit@gmail.com>
parents: 33708
diff changeset
2088
db3dc11356ed pushvars: move fb extension pushvars to core
Pulkit Goyal <7895pulkit@gmail.com>
parents: 33708
diff changeset
2089 @parthandler('pushvars')
db3dc11356ed pushvars: move fb extension pushvars to core
Pulkit Goyal <7895pulkit@gmail.com>
parents: 33708
diff changeset
2090 def bundle2getvars(op, part):
db3dc11356ed pushvars: move fb extension pushvars to core
Pulkit Goyal <7895pulkit@gmail.com>
parents: 33708
diff changeset
2091 '''unbundle a bundle2 containing shellvars on the server'''
db3dc11356ed pushvars: move fb extension pushvars to core
Pulkit Goyal <7895pulkit@gmail.com>
parents: 33708
diff changeset
2092 # An option to disable unbundling on server-side for security reasons
33856
057d31ceace3 pushvars: add a coreconfigitem for push.pushvars.server
Pulkit Goyal <7895pulkit@gmail.com>
parents: 33831
diff changeset
2093 if op.ui.configbool('push', 'pushvars.server'):
33719
db3dc11356ed pushvars: move fb extension pushvars to core
Pulkit Goyal <7895pulkit@gmail.com>
parents: 33708
diff changeset
2094 hookargs = {}
db3dc11356ed pushvars: move fb extension pushvars to core
Pulkit Goyal <7895pulkit@gmail.com>
parents: 33708
diff changeset
2095 for key, value in part.advisoryparams:
db3dc11356ed pushvars: move fb extension pushvars to core
Pulkit Goyal <7895pulkit@gmail.com>
parents: 33708
diff changeset
2096 key = key.upper()
db3dc11356ed pushvars: move fb extension pushvars to core
Pulkit Goyal <7895pulkit@gmail.com>
parents: 33708
diff changeset
2097 # We want pushed variables to have USERVAR_ prepended so we know
db3dc11356ed pushvars: move fb extension pushvars to core
Pulkit Goyal <7895pulkit@gmail.com>
parents: 33708
diff changeset
2098 # they came from the --pushvar flag.
db3dc11356ed pushvars: move fb extension pushvars to core
Pulkit Goyal <7895pulkit@gmail.com>
parents: 33708
diff changeset
2099 key = "USERVAR_" + key
db3dc11356ed pushvars: move fb extension pushvars to core
Pulkit Goyal <7895pulkit@gmail.com>
parents: 33708
diff changeset
2100 hookargs[key] = value
db3dc11356ed pushvars: move fb extension pushvars to core
Pulkit Goyal <7895pulkit@gmail.com>
parents: 33708
diff changeset
2101 op.addhookargs(hookargs)