mercurial-scm/hg: mercurial/bundle2.py annotate

annotate mercurial/bundle2.py @ 37178:68ee61822182

thirdparty: port zope.interface to relative imports By using relative imports, we're guaranteed to get modules vendored with Mercurial rather than other random modules that might be in sys.path. My editor strips trailing whitespace on save. So some minor source code cleanup was also performed as part of this commit. # no-check-commit because some modified lines have double newlines Differential Revision: https://phab.mercurial-scm.org/D2930

author	Gregory Szorc <gregory.szorc@gmail.com>
date	Wed, 21 Mar 2018 19:52:30 -0700
parents	6f467adf9f05
children	7e906d8a825f

rev	line source
20801 9c5183cb9bca bundle2: very first version of a bundle2 bundler Pierre-Yves David <pierre-yves.david@fb.com> parents: diff changeset	1 # bundle2.py - generic container format to transmit arbitrary data.
9c5183cb9bca bundle2: very first version of a bundle2 bundler Pierre-Yves David <pierre-yves.david@fb.com> parents: diff changeset	2 #
9c5183cb9bca bundle2: very first version of a bundle2 bundler Pierre-Yves David <pierre-yves.david@fb.com> parents: diff changeset	3 # Copyright 2013 Facebook, Inc.
9c5183cb9bca bundle2: very first version of a bundle2 bundler Pierre-Yves David <pierre-yves.david@fb.com> parents: diff changeset	4 #
9c5183cb9bca bundle2: very first version of a bundle2 bundler Pierre-Yves David <pierre-yves.david@fb.com> parents: diff changeset	5 # This software may be used and distributed according to the terms of the
9c5183cb9bca bundle2: very first version of a bundle2 bundler Pierre-Yves David <pierre-yves.david@fb.com> parents: diff changeset	6 # GNU General Public License version 2 or any later version.
9c5183cb9bca bundle2: very first version of a bundle2 bundler Pierre-Yves David <pierre-yves.david@fb.com> parents: diff changeset	7 """Handling of the new bundle2 format
9c5183cb9bca bundle2: very first version of a bundle2 bundler Pierre-Yves David <pierre-yves.david@fb.com> parents: diff changeset	8
9c5183cb9bca bundle2: very first version of a bundle2 bundler Pierre-Yves David <pierre-yves.david@fb.com> parents: diff changeset	9 The goal of bundle2 is to act as an atomically packet to transmit a set of
9c5183cb9bca bundle2: very first version of a bundle2 bundler Pierre-Yves David <pierre-yves.david@fb.com> parents: diff changeset	10 payloads in an application agnostic way. It consist in a sequence of "parts"
9c5183cb9bca bundle2: very first version of a bundle2 bundler Pierre-Yves David <pierre-yves.david@fb.com> parents: diff changeset	11 that will be handed to and processed by the application layer.
9c5183cb9bca bundle2: very first version of a bundle2 bundler Pierre-Yves David <pierre-yves.david@fb.com> parents: diff changeset	12
9c5183cb9bca bundle2: very first version of a bundle2 bundler Pierre-Yves David <pierre-yves.david@fb.com> parents: diff changeset	13
9c5183cb9bca bundle2: very first version of a bundle2 bundler Pierre-Yves David <pierre-yves.david@fb.com> parents: diff changeset	14 General format architecture
9c5183cb9bca bundle2: very first version of a bundle2 bundler Pierre-Yves David <pierre-yves.david@fb.com> parents: diff changeset	15 ===========================
9c5183cb9bca bundle2: very first version of a bundle2 bundler Pierre-Yves David <pierre-yves.david@fb.com> parents: diff changeset	16
9c5183cb9bca bundle2: very first version of a bundle2 bundler Pierre-Yves David <pierre-yves.david@fb.com> parents: diff changeset	17 The format is architectured as follow
9c5183cb9bca bundle2: very first version of a bundle2 bundler Pierre-Yves David <pierre-yves.david@fb.com> parents: diff changeset	18
9c5183cb9bca bundle2: very first version of a bundle2 bundler Pierre-Yves David <pierre-yves.david@fb.com> parents: diff changeset	19 - magic string
9c5183cb9bca bundle2: very first version of a bundle2 bundler Pierre-Yves David <pierre-yves.david@fb.com> parents: diff changeset	20 - stream level parameters
9c5183cb9bca bundle2: very first version of a bundle2 bundler Pierre-Yves David <pierre-yves.david@fb.com> parents: diff changeset	21 - payload parts (any number)
9c5183cb9bca bundle2: very first version of a bundle2 bundler Pierre-Yves David <pierre-yves.david@fb.com> parents: diff changeset	22 - end of stream marker.
9c5183cb9bca bundle2: very first version of a bundle2 bundler Pierre-Yves David <pierre-yves.david@fb.com> parents: diff changeset	23
20856 8a6a86c9a5b5 bundle2: support bundling of empty part (with a type) Pierre-Yves David <pierre-yves.david@fb.com> parents: 20844 diff changeset	24 the Binary format
20801 9c5183cb9bca bundle2: very first version of a bundle2 bundler Pierre-Yves David <pierre-yves.david@fb.com> parents: diff changeset	25 ============================
9c5183cb9bca bundle2: very first version of a bundle2 bundler Pierre-Yves David <pierre-yves.david@fb.com> parents: diff changeset	26
21024 7731a2281cf0 spelling: fixes from spell checker Mads Kiilerich <madski@unity3d.com> parents: 21020 diff changeset	27 All numbers are unsigned and big-endian.
20801 9c5183cb9bca bundle2: very first version of a bundle2 bundler Pierre-Yves David <pierre-yves.david@fb.com> parents: diff changeset	28
9c5183cb9bca bundle2: very first version of a bundle2 bundler Pierre-Yves David <pierre-yves.david@fb.com> parents: diff changeset	29 stream level parameters
9c5183cb9bca bundle2: very first version of a bundle2 bundler Pierre-Yves David <pierre-yves.david@fb.com> parents: diff changeset	30 ------------------------
9c5183cb9bca bundle2: very first version of a bundle2 bundler Pierre-Yves David <pierre-yves.david@fb.com> parents: diff changeset	31
9c5183cb9bca bundle2: very first version of a bundle2 bundler Pierre-Yves David <pierre-yves.david@fb.com> parents: diff changeset	32 Binary format is as follow
9c5183cb9bca bundle2: very first version of a bundle2 bundler Pierre-Yves David <pierre-yves.david@fb.com> parents: diff changeset	33
23009 90f86ad3d4ff bundle2: change header size and make them signed (new format) Pierre-Yves David <pierre-yves.david@fb.com> parents: 23008 diff changeset	34 :params size: int32
20801 9c5183cb9bca bundle2: very first version of a bundle2 bundler Pierre-Yves David <pierre-yves.david@fb.com> parents: diff changeset	35
9c5183cb9bca bundle2: very first version of a bundle2 bundler Pierre-Yves David <pierre-yves.david@fb.com> parents: diff changeset	36 The total number of Bytes used by the parameters
9c5183cb9bca bundle2: very first version of a bundle2 bundler Pierre-Yves David <pierre-yves.david@fb.com> parents: diff changeset	37
9c5183cb9bca bundle2: very first version of a bundle2 bundler Pierre-Yves David <pierre-yves.david@fb.com> parents: diff changeset	38 :params value: arbitrary number of Bytes
9c5183cb9bca bundle2: very first version of a bundle2 bundler Pierre-Yves David <pierre-yves.david@fb.com> parents: diff changeset	39
9c5183cb9bca bundle2: very first version of a bundle2 bundler Pierre-Yves David <pierre-yves.david@fb.com> parents: diff changeset	40 A blob of `params size` containing the serialized version of all stream level
9c5183cb9bca bundle2: very first version of a bundle2 bundler Pierre-Yves David <pierre-yves.david@fb.com> parents: diff changeset	41 parameters.
9c5183cb9bca bundle2: very first version of a bundle2 bundler Pierre-Yves David <pierre-yves.david@fb.com> parents: diff changeset	42
21024 7731a2281cf0 spelling: fixes from spell checker Mads Kiilerich <madski@unity3d.com> parents: 21020 diff changeset	43 The blob contains a space separated list of parameters. Parameters with value
20811 9785c3f8f598 bundle2: urlquote stream parameter name and value Pierre-Yves David <pierre-yves.david@fb.com> parents: 20810 diff changeset	44 are stored in the form `<name>=<value>`. Both name and value are urlquoted.
20804 db9d3991d2c6 bundle2: support bundling simple parameter Pierre-Yves David <pierre-yves.david@fb.com> parents: 20803 diff changeset	45
20813 8c74b3ce5b70 bundle2: refuse empty parameter name Pierre-Yves David <pierre-yves.david@fb.com> parents: 20812 diff changeset	46 Empty name are obviously forbidden.
8c74b3ce5b70 bundle2: refuse empty parameter name Pierre-Yves David <pierre-yves.david@fb.com> parents: 20812 diff changeset	47
20844 2631204d7305 bundle2: implement the mandatory/advisory logic for parameter Pierre-Yves David <pierre-yves.david@fb.com> parents: 20843 diff changeset	48 Name MUST start with a letter. If this first letter is lower case, the
21024 7731a2281cf0 spelling: fixes from spell checker Mads Kiilerich <madski@unity3d.com> parents: 21020 diff changeset	49 parameter is advisory and can be safely ignored. However when the first
20844 2631204d7305 bundle2: implement the mandatory/advisory logic for parameter Pierre-Yves David <pierre-yves.david@fb.com> parents: 20843 diff changeset	50 letter is capital, the parameter is mandatory and the bundling process MUST
2631204d7305 bundle2: implement the mandatory/advisory logic for parameter Pierre-Yves David <pierre-yves.david@fb.com> parents: 20843 diff changeset	51 stop if he is not able to proceed it.
20814 8532f5e1b9df bundle2: force the first char of parameter to be an letter. Pierre-Yves David <pierre-yves.david@fb.com> parents: 20813 diff changeset	52
20808 4c9130c7a29f bundle2: clarify stream parameter design in the documentation Pierre-Yves David <pierre-yves.david@fb.com> parents: 20805 diff changeset	53 Stream parameters use a simple textual format for two main reasons:
20804 db9d3991d2c6 bundle2: support bundling simple parameter Pierre-Yves David <pierre-yves.david@fb.com> parents: 20803 diff changeset	54
21024 7731a2281cf0 spelling: fixes from spell checker Mads Kiilerich <madski@unity3d.com> parents: 21020 diff changeset	55 - Stream level parameters should remain simple and we want to discourage any
20808 4c9130c7a29f bundle2: clarify stream parameter design in the documentation Pierre-Yves David <pierre-yves.david@fb.com> parents: 20805 diff changeset	56 crazy usage.
21024 7731a2281cf0 spelling: fixes from spell checker Mads Kiilerich <madski@unity3d.com> parents: 21020 diff changeset	57 - Textual data allow easy human inspection of a bundle2 header in case of
20808 4c9130c7a29f bundle2: clarify stream parameter design in the documentation Pierre-Yves David <pierre-yves.david@fb.com> parents: 20805 diff changeset	58 troubles.
4c9130c7a29f bundle2: clarify stream parameter design in the documentation Pierre-Yves David <pierre-yves.david@fb.com> parents: 20805 diff changeset	59
4c9130c7a29f bundle2: clarify stream parameter design in the documentation Pierre-Yves David <pierre-yves.david@fb.com> parents: 20805 diff changeset	60 Any Applicative level options MUST go into a bundle2 part instead.
20801 9c5183cb9bca bundle2: very first version of a bundle2 bundler Pierre-Yves David <pierre-yves.david@fb.com> parents: diff changeset	61
9c5183cb9bca bundle2: very first version of a bundle2 bundler Pierre-Yves David <pierre-yves.david@fb.com> parents: diff changeset	62 Payload part
9c5183cb9bca bundle2: very first version of a bundle2 bundler Pierre-Yves David <pierre-yves.david@fb.com> parents: diff changeset	63 ------------------------
9c5183cb9bca bundle2: very first version of a bundle2 bundler Pierre-Yves David <pierre-yves.david@fb.com> parents: diff changeset	64
9c5183cb9bca bundle2: very first version of a bundle2 bundler Pierre-Yves David <pierre-yves.david@fb.com> parents: diff changeset	65 Binary format is as follow
9c5183cb9bca bundle2: very first version of a bundle2 bundler Pierre-Yves David <pierre-yves.david@fb.com> parents: diff changeset	66
23009 90f86ad3d4ff bundle2: change header size and make them signed (new format) Pierre-Yves David <pierre-yves.david@fb.com> parents: 23008 diff changeset	67 :header size: int32
20801 9c5183cb9bca bundle2: very first version of a bundle2 bundler Pierre-Yves David <pierre-yves.david@fb.com> parents: diff changeset	68
25507 5bee483705e8 bundle2: clarify in docstring that header size is for a single header Martin von Zweigbergk <martinvonz@google.com> parents: 25506 diff changeset	69 The total number of Bytes used by the part header. When the header is empty
20801 9c5183cb9bca bundle2: very first version of a bundle2 bundler Pierre-Yves David <pierre-yves.david@fb.com> parents: diff changeset	70 (size = 0) this is interpreted as the end of stream marker.
9c5183cb9bca bundle2: very first version of a bundle2 bundler Pierre-Yves David <pierre-yves.david@fb.com> parents: diff changeset	71
20856 8a6a86c9a5b5 bundle2: support bundling of empty part (with a type) Pierre-Yves David <pierre-yves.david@fb.com> parents: 20844 diff changeset	72 :header:
8a6a86c9a5b5 bundle2: support bundling of empty part (with a type) Pierre-Yves David <pierre-yves.david@fb.com> parents: 20844 diff changeset	73
8a6a86c9a5b5 bundle2: support bundling of empty part (with a type) Pierre-Yves David <pierre-yves.david@fb.com> parents: 20844 diff changeset	74 The header defines how to interpret the part. It contains two piece of
8a6a86c9a5b5 bundle2: support bundling of empty part (with a type) Pierre-Yves David <pierre-yves.david@fb.com> parents: 20844 diff changeset	75 data: the part type, and the part parameters.
8a6a86c9a5b5 bundle2: support bundling of empty part (with a type) Pierre-Yves David <pierre-yves.david@fb.com> parents: 20844 diff changeset	76
8a6a86c9a5b5 bundle2: support bundling of empty part (with a type) Pierre-Yves David <pierre-yves.david@fb.com> parents: 20844 diff changeset	77 The part type is used to route an application level handler, that can
8a6a86c9a5b5 bundle2: support bundling of empty part (with a type) Pierre-Yves David <pierre-yves.david@fb.com> parents: 20844 diff changeset	78 interpret payload.
8a6a86c9a5b5 bundle2: support bundling of empty part (with a type) Pierre-Yves David <pierre-yves.david@fb.com> parents: 20844 diff changeset	79
8a6a86c9a5b5 bundle2: support bundling of empty part (with a type) Pierre-Yves David <pierre-yves.david@fb.com> parents: 20844 diff changeset	80 Part parameters are passed to the application level handler. They are
8a6a86c9a5b5 bundle2: support bundling of empty part (with a type) Pierre-Yves David <pierre-yves.david@fb.com> parents: 20844 diff changeset	81 meant to convey information that will help the application level object to
8a6a86c9a5b5 bundle2: support bundling of empty part (with a type) Pierre-Yves David <pierre-yves.david@fb.com> parents: 20844 diff changeset	82 interpret the part payload.
8a6a86c9a5b5 bundle2: support bundling of empty part (with a type) Pierre-Yves David <pierre-yves.david@fb.com> parents: 20844 diff changeset	83
8a6a86c9a5b5 bundle2: support bundling of empty part (with a type) Pierre-Yves David <pierre-yves.david@fb.com> parents: 20844 diff changeset	84 The binary format of the header is has follow
8a6a86c9a5b5 bundle2: support bundling of empty part (with a type) Pierre-Yves David <pierre-yves.david@fb.com> parents: 20844 diff changeset	85
8a6a86c9a5b5 bundle2: support bundling of empty part (with a type) Pierre-Yves David <pierre-yves.david@fb.com> parents: 20844 diff changeset	86 :typesize: (one byte)
20877 9e9e3a4e9261 bundle2: part params Pierre-Yves David <pierre-yves.david@fb.com> parents: 20876 diff changeset	87
23916 a3f7c781786b bundle2: fix parttype enforcement Matt Mackall <mpm@selenic.com> parents: 23868 diff changeset	88 :parttype: alphanumerical part name (restricted to [a-zA-Z0-9_:-]*)
20877 9e9e3a4e9261 bundle2: part params Pierre-Yves David <pierre-yves.david@fb.com> parents: 20876 diff changeset	89
20995 e995d104c87f bundle2: add an integer id to part Pierre-Yves David <pierre-yves.david@fb.com> parents: 20952 diff changeset	90 :partid: A 32bits integer (unique in the bundle) that can be used to refer
e995d104c87f bundle2: add an integer id to part Pierre-Yves David <pierre-yves.david@fb.com> parents: 20952 diff changeset	91 to this part.
e995d104c87f bundle2: add an integer id to part Pierre-Yves David <pierre-yves.david@fb.com> parents: 20952 diff changeset	92
20877 9e9e3a4e9261 bundle2: part params Pierre-Yves David <pierre-yves.david@fb.com> parents: 20876 diff changeset	93 :parameters:
9e9e3a4e9261 bundle2: part params Pierre-Yves David <pierre-yves.david@fb.com> parents: 20876 diff changeset	94
21024 7731a2281cf0 spelling: fixes from spell checker Mads Kiilerich <madski@unity3d.com> parents: 21020 diff changeset	95 Part's parameter may have arbitrary content, the binary structure is::
20877 9e9e3a4e9261 bundle2: part params Pierre-Yves David <pierre-yves.david@fb.com> parents: 20876 diff changeset	96
9e9e3a4e9261 bundle2: part params Pierre-Yves David <pierre-yves.david@fb.com> parents: 20876 diff changeset	97 <mandatory-count><advisory-count><param-sizes><param-data>
9e9e3a4e9261 bundle2: part params Pierre-Yves David <pierre-yves.david@fb.com> parents: 20876 diff changeset	98
9e9e3a4e9261 bundle2: part params Pierre-Yves David <pierre-yves.david@fb.com> parents: 20876 diff changeset	99 :mandatory-count: 1 byte, number of mandatory parameters
9e9e3a4e9261 bundle2: part params Pierre-Yves David <pierre-yves.david@fb.com> parents: 20876 diff changeset	100
9e9e3a4e9261 bundle2: part params Pierre-Yves David <pierre-yves.david@fb.com> parents: 20876 diff changeset	101 :advisory-count: 1 byte, number of advisory parameters
9e9e3a4e9261 bundle2: part params Pierre-Yves David <pierre-yves.david@fb.com> parents: 20876 diff changeset	102
9e9e3a4e9261 bundle2: part params Pierre-Yves David <pierre-yves.david@fb.com> parents: 20876 diff changeset	103 :param-sizes:
9e9e3a4e9261 bundle2: part params Pierre-Yves David <pierre-yves.david@fb.com> parents: 20876 diff changeset	104
9e9e3a4e9261 bundle2: part params Pierre-Yves David <pierre-yves.david@fb.com> parents: 20876 diff changeset	105 N couple of bytes, where N is the total number of parameters. Each
9e9e3a4e9261 bundle2: part params Pierre-Yves David <pierre-yves.david@fb.com> parents: 20876 diff changeset	106 couple contains (<size-of-key>, <size-of-value) for one parameter.
9e9e3a4e9261 bundle2: part params Pierre-Yves David <pierre-yves.david@fb.com> parents: 20876 diff changeset	107
9e9e3a4e9261 bundle2: part params Pierre-Yves David <pierre-yves.david@fb.com> parents: 20876 diff changeset	108 :param-data:
9e9e3a4e9261 bundle2: part params Pierre-Yves David <pierre-yves.david@fb.com> parents: 20876 diff changeset	109
9e9e3a4e9261 bundle2: part params Pierre-Yves David <pierre-yves.david@fb.com> parents: 20876 diff changeset	110 A blob of bytes from which each parameter key and value can be
9e9e3a4e9261 bundle2: part params Pierre-Yves David <pierre-yves.david@fb.com> parents: 20876 diff changeset	111 retrieved using the list of size couples stored in the previous
9e9e3a4e9261 bundle2: part params Pierre-Yves David <pierre-yves.david@fb.com> parents: 20876 diff changeset	112 field.
9e9e3a4e9261 bundle2: part params Pierre-Yves David <pierre-yves.david@fb.com> parents: 20876 diff changeset	113
9e9e3a4e9261 bundle2: part params Pierre-Yves David <pierre-yves.david@fb.com> parents: 20876 diff changeset	114 Mandatory parameters comes first, then the advisory ones.
20856 8a6a86c9a5b5 bundle2: support bundling of empty part (with a type) Pierre-Yves David <pierre-yves.david@fb.com> parents: 20844 diff changeset	115
21607 054fa5176fa7 bundle2: forbid duplicate parameter keys Pierre-Yves David <pierre-yves.david@fb.com> parents: 21606 diff changeset	116 Each parameter's key MUST be unique within the part.
054fa5176fa7 bundle2: forbid duplicate parameter keys Pierre-Yves David <pierre-yves.david@fb.com> parents: 21606 diff changeset	117
20856 8a6a86c9a5b5 bundle2: support bundling of empty part (with a type) Pierre-Yves David <pierre-yves.david@fb.com> parents: 20844 diff changeset	118 :payload:
8a6a86c9a5b5 bundle2: support bundling of empty part (with a type) Pierre-Yves David <pierre-yves.david@fb.com> parents: 20844 diff changeset	119
20876 ddd56f3eb786 bundle2: support for bundling and unbundling payload Pierre-Yves David <pierre-yves.david@fb.com> parents: 20864 diff changeset	120 payload is a series of `<chunksize><chunkdata>`.
ddd56f3eb786 bundle2: support for bundling and unbundling payload Pierre-Yves David <pierre-yves.david@fb.com> parents: 20864 diff changeset	121
23009 90f86ad3d4ff bundle2: change header size and make them signed (new format) Pierre-Yves David <pierre-yves.david@fb.com> parents: 23008 diff changeset	122 `chunksize` is an int32, `chunkdata` are plain bytes (as much as
20876 ddd56f3eb786 bundle2: support for bundling and unbundling payload Pierre-Yves David <pierre-yves.david@fb.com> parents: 20864 diff changeset	123 `chunksize` says)` The payload part is concluded by a zero size chunk.
ddd56f3eb786 bundle2: support for bundling and unbundling payload Pierre-Yves David <pierre-yves.david@fb.com> parents: 20864 diff changeset	124
ddd56f3eb786 bundle2: support for bundling and unbundling payload Pierre-Yves David <pierre-yves.david@fb.com> parents: 20864 diff changeset	125 The current implementation always produces either zero or one chunk.
21024 7731a2281cf0 spelling: fixes from spell checker Mads Kiilerich <madski@unity3d.com> parents: 21020 diff changeset	126 This is an implementation limitation that will ultimately be lifted.
20891 1c6cd23fc221 bundle2: add some distinction between mandatory and advisory part Pierre-Yves David <pierre-yves.david@fb.com> parents: 20890 diff changeset	127
23009 90f86ad3d4ff bundle2: change header size and make them signed (new format) Pierre-Yves David <pierre-yves.david@fb.com> parents: 23008 diff changeset	128 `chunksize` can be negative to trigger special case processing. No such
90f86ad3d4ff bundle2: change header size and make them signed (new format) Pierre-Yves David <pierre-yves.david@fb.com> parents: 23008 diff changeset	129 processing is in place yet.
90f86ad3d4ff bundle2: change header size and make them signed (new format) Pierre-Yves David <pierre-yves.david@fb.com> parents: 23008 diff changeset	130
20891 1c6cd23fc221 bundle2: add some distinction between mandatory and advisory part Pierre-Yves David <pierre-yves.david@fb.com> parents: 20890 diff changeset	131 Bundle processing
1c6cd23fc221 bundle2: add some distinction between mandatory and advisory part Pierre-Yves David <pierre-yves.david@fb.com> parents: 20890 diff changeset	132 ============================
1c6cd23fc221 bundle2: add some distinction between mandatory and advisory part Pierre-Yves David <pierre-yves.david@fb.com> parents: 20890 diff changeset	133
1c6cd23fc221 bundle2: add some distinction between mandatory and advisory part Pierre-Yves David <pierre-yves.david@fb.com> parents: 20890 diff changeset	134 Each part is processed in order using a "part handler". Handler are registered
1c6cd23fc221 bundle2: add some distinction between mandatory and advisory part Pierre-Yves David <pierre-yves.david@fb.com> parents: 20890 diff changeset	135 for a certain part type.
1c6cd23fc221 bundle2: add some distinction between mandatory and advisory part Pierre-Yves David <pierre-yves.david@fb.com> parents: 20890 diff changeset	136
1c6cd23fc221 bundle2: add some distinction between mandatory and advisory part Pierre-Yves David <pierre-yves.david@fb.com> parents: 20890 diff changeset	137 The matching of a part to its handler is case insensitive. The case of the
1c6cd23fc221 bundle2: add some distinction between mandatory and advisory part Pierre-Yves David <pierre-yves.david@fb.com> parents: 20890 diff changeset	138 part type is used to know if a part is mandatory or advisory. If the Part type
1c6cd23fc221 bundle2: add some distinction between mandatory and advisory part Pierre-Yves David <pierre-yves.david@fb.com> parents: 20890 diff changeset	139 contains any uppercase char it is considered mandatory. When no handler is
1c6cd23fc221 bundle2: add some distinction between mandatory and advisory part Pierre-Yves David <pierre-yves.david@fb.com> parents: 20890 diff changeset	140 known for a Mandatory part, the process is aborted and an exception is raised.
20892 6fe95448596d bundle2: read the whole bundle from stream on abort Pierre-Yves David <pierre-yves.david@fb.com> parents: 20891 diff changeset	141 If the part is advisory and no handler is known, the part is ignored. When the
6fe95448596d bundle2: read the whole bundle from stream on abort Pierre-Yves David <pierre-yves.david@fb.com> parents: 20891 diff changeset	142 process is aborted, the full bundle is still read from the stream to keep the
6fe95448596d bundle2: read the whole bundle from stream on abort Pierre-Yves David <pierre-yves.david@fb.com> parents: 20891 diff changeset	143 channel usable. But none of the part read from an abort are processed. In the
6fe95448596d bundle2: read the whole bundle from stream on abort Pierre-Yves David <pierre-yves.david@fb.com> parents: 20891 diff changeset	144 future, dropping the stream may become an option for channel we do not care to
6fe95448596d bundle2: read the whole bundle from stream on abort Pierre-Yves David <pierre-yves.david@fb.com> parents: 20891 diff changeset	145 preserve.
20801 9c5183cb9bca bundle2: very first version of a bundle2 bundler Pierre-Yves David <pierre-yves.david@fb.com> parents: diff changeset	146 """
9c5183cb9bca bundle2: very first version of a bundle2 bundler Pierre-Yves David <pierre-yves.david@fb.com> parents: diff changeset	147
33672 da7c285ec6da bundle2: use modern Python division Augie Fackler <augie@google.com> parents: 33665 diff changeset	148 from __future__ import absolute_import, division
25919 8221fefaea08 bundle2: use absolute_import Gregory Szorc <gregory.szorc@gmail.com> parents: 25660 diff changeset	149
36964 79b73be4dda5 rev-branch-cache: add a function to generate a part Boris Feld <boris.feld@octobus.net> parents: 36963 diff changeset	150 import collections
24026 3daef83a1873 bundle2.unpackermixin: control for underlying file descriptor Eric Sumner <ericsumner@fb.com> parents: 23916 diff changeset	151 import errno
35046 241d9caca11e bundle2: use os.SEEK_* constants Gregory Szorc <gregory.szorc@gmail.com> parents: 34820 diff changeset	152 import os
25919 8221fefaea08 bundle2: use absolute_import Gregory Szorc <gregory.szorc@gmail.com> parents: 25660 diff changeset	153 import re
20814 8532f5e1b9df bundle2: force the first char of parameter to be an letter. Pierre-Yves David <pierre-yves.david@fb.com> parents: 20813 diff changeset	154 import string
25919 8221fefaea08 bundle2: use absolute_import Gregory Szorc <gregory.szorc@gmail.com> parents: 25660 diff changeset	155 import struct
8221fefaea08 bundle2: use absolute_import Gregory Szorc <gregory.szorc@gmail.com> parents: 25660 diff changeset	156 import sys
20804 db9d3991d2c6 bundle2: support bundling simple parameter Pierre-Yves David <pierre-yves.david@fb.com> parents: 20803 diff changeset	157
25919 8221fefaea08 bundle2: use absolute_import Gregory Szorc <gregory.szorc@gmail.com> parents: 25660 diff changeset	158 from .i18n import _
8221fefaea08 bundle2: use absolute_import Gregory Szorc <gregory.szorc@gmail.com> parents: 25660 diff changeset	159 from . import (
35258 dbf868623daf bookmark: add a 'check:bookmarks' bundle2 part Boris Feld <boris.feld@octobus.net> parents: 35120 diff changeset	160 bookmarks,
25919 8221fefaea08 bundle2: use absolute_import Gregory Szorc <gregory.szorc@gmail.com> parents: 25660 diff changeset	161 changegroup,
36963 9988fc10f49e revbranchcache: add a bundle2 handler for a rbc part Boris Feld <boris.feld@octobus.net> parents: 36428 diff changeset	162 encoding,
25919 8221fefaea08 bundle2: use absolute_import Gregory Szorc <gregory.szorc@gmail.com> parents: 25660 diff changeset	163 error,
34820 a95067b1dca6 phase: introduce a new 'check:phases' part Boris Feld <boris.feld@octobus.net> parents: 34637 diff changeset	164 node as nodemod,
25919 8221fefaea08 bundle2: use absolute_import Gregory Szorc <gregory.szorc@gmail.com> parents: 25660 diff changeset	165 obsolete,
33031 e8c8d81eb864 bundle: add config option to include phases Martin von Zweigbergk <martinvonz@google.com> parents: 33030 diff changeset	166 phases,
25919 8221fefaea08 bundle2: use absolute_import Gregory Szorc <gregory.szorc@gmail.com> parents: 25660 diff changeset	167 pushkey,
30030 0f6d6fdd3c2a pycompat: provide 'ispy3' constant Yuya Nishihara <yuya@tcha.org> parents: 29937 diff changeset	168 pycompat,
35758 b996ddf5963d bundle2: add a 'stream' part handler for stream cloning Boris Feld <boris.feld@octobus.net> parents: 35655 diff changeset	169 streamclone,
25919 8221fefaea08 bundle2: use absolute_import Gregory Szorc <gregory.szorc@gmail.com> parents: 25660 diff changeset	170 tags,
8221fefaea08 bundle2: use absolute_import Gregory Szorc <gregory.szorc@gmail.com> parents: 25660 diff changeset	171 url,
8221fefaea08 bundle2: use absolute_import Gregory Szorc <gregory.szorc@gmail.com> parents: 25660 diff changeset	172 util,
8221fefaea08 bundle2: use absolute_import Gregory Szorc <gregory.szorc@gmail.com> parents: 25660 diff changeset	173 )
37084 f0b6fbea00cf stringutil: bulk-replace call sites to point to new module Yuya Nishihara <yuya@tcha.org> parents: 37005 diff changeset	174 from .utils import (
f0b6fbea00cf stringutil: bulk-replace call sites to point to new module Yuya Nishihara <yuya@tcha.org> parents: 37005 diff changeset	175 stringutil,
f0b6fbea00cf stringutil: bulk-replace call sites to point to new module Yuya Nishihara <yuya@tcha.org> parents: 37005 diff changeset	176 )
20802 520df53ad26a bundle2: a very first version of bundle2 unbundler Pierre-Yves David <pierre-yves.david@fb.com> parents: 20801 diff changeset	177
28883 032c4c2f802a pycompat: switch to util.urlreq/util.urlerr for py3 compat timeless <timeless@mozdev.org> parents: 28672 diff changeset	178 urlerr = util.urlerr
032c4c2f802a pycompat: switch to util.urlreq/util.urlerr for py3 compat timeless <timeless@mozdev.org> parents: 28672 diff changeset	179 urlreq = util.urlreq
032c4c2f802a pycompat: switch to util.urlreq/util.urlerr for py3 compat timeless <timeless@mozdev.org> parents: 28672 diff changeset	180
20804 db9d3991d2c6 bundle2: support bundling simple parameter Pierre-Yves David <pierre-yves.david@fb.com> parents: 20803 diff changeset	181 _pack = struct.pack
db9d3991d2c6 bundle2: support bundling simple parameter Pierre-Yves David <pierre-yves.david@fb.com> parents: 20803 diff changeset	182 _unpack = struct.unpack
db9d3991d2c6 bundle2: support bundling simple parameter Pierre-Yves David <pierre-yves.david@fb.com> parents: 20803 diff changeset	183
23009 90f86ad3d4ff bundle2: change header size and make them signed (new format) Pierre-Yves David <pierre-yves.david@fb.com> parents: 23008 diff changeset	184 _fstreamparamsize = '>i'
90f86ad3d4ff bundle2: change header size and make them signed (new format) Pierre-Yves David <pierre-yves.david@fb.com> parents: 23008 diff changeset	185 _fpartheadersize = '>i'
20856 8a6a86c9a5b5 bundle2: support bundling of empty part (with a type) Pierre-Yves David <pierre-yves.david@fb.com> parents: 20844 diff changeset	186 _fparttypesize = '>B'
20995 e995d104c87f bundle2: add an integer id to part Pierre-Yves David <pierre-yves.david@fb.com> parents: 20952 diff changeset	187 _fpartid = '>I'
23009 90f86ad3d4ff bundle2: change header size and make them signed (new format) Pierre-Yves David <pierre-yves.david@fb.com> parents: 23008 diff changeset	188 _fpayloadsize = '>i'
20877 9e9e3a4e9261 bundle2: part params Pierre-Yves David <pierre-yves.david@fb.com> parents: 20876 diff changeset	189 _fpartparamcount = '>BB'
9e9e3a4e9261 bundle2: part params Pierre-Yves David <pierre-yves.david@fb.com> parents: 20876 diff changeset	190
35793 4fb2bb61597c bundle2: increase payload part chunk size to 32kb Gregory Szorc <gregory.szorc@gmail.com> parents: 35792 diff changeset	191 preferedchunksize = 32768
21001 c93bb6a08fa1 bundle2: support chunk iterator as part data Pierre-Yves David <pierre-yves.david@fb.com> parents: 21000 diff changeset	192
23868 405eacbecc52 bundle2: enforce parttype as alphanumerical Pierre-Yves David <pierre-yves.david@fb.com> parents: 23590 diff changeset	193 _parttypeforbidden = re.compile('[^a-zA-Z0-9_:-]')
405eacbecc52 bundle2: enforce parttype as alphanumerical Pierre-Yves David <pierre-yves.david@fb.com> parents: 23590 diff changeset	194
25313 8f2c362bcb58 bundle2: introduce a specific function for bundling debug message Pierre-Yves David <pierre-yves.david@fb.com> parents: 25181 diff changeset	195 def outdebug(ui, message):
8f2c362bcb58 bundle2: introduce a specific function for bundling debug message Pierre-Yves David <pierre-yves.david@fb.com> parents: 25181 diff changeset	196 """debug regarding output stream (bundling)"""
33159 0224820688ac config: register the 'devel.bundle2.debug' config Pierre-Yves David <pierre-yves.david@octobus.net> parents: 33128 diff changeset	197 if ui.configbool('devel', 'bundle2.debug'):
25336 7c60a42265fb bundle2: hide bundle2 stream debug under a config flag Pierre-Yves David <pierre-yves.david@fb.com> parents: 25335 diff changeset	198 ui.debug('bundle2-output: %s\n' % message)
25313 8f2c362bcb58 bundle2: introduce a specific function for bundling debug message Pierre-Yves David <pierre-yves.david@fb.com> parents: 25181 diff changeset	199
25318 64dd8f25fcf4 bundle2: introduce a specific function for debug messages while unbundling Pierre-Yves David <pierre-yves.david@fb.com> parents: 25317 diff changeset	200 def indebug(ui, message):
64dd8f25fcf4 bundle2: introduce a specific function for debug messages while unbundling Pierre-Yves David <pierre-yves.david@fb.com> parents: 25317 diff changeset	201 """debug on input stream (unbundling)"""
33159 0224820688ac config: register the 'devel.bundle2.debug' config Pierre-Yves David <pierre-yves.david@octobus.net> parents: 33128 diff changeset	202 if ui.configbool('devel', 'bundle2.debug'):
25336 7c60a42265fb bundle2: hide bundle2 stream debug under a config flag Pierre-Yves David <pierre-yves.david@fb.com> parents: 25335 diff changeset	203 ui.debug('bundle2-input: %s\n' % message)
25318 64dd8f25fcf4 bundle2: introduce a specific function for debug messages while unbundling Pierre-Yves David <pierre-yves.david@fb.com> parents: 25317 diff changeset	204
23868 405eacbecc52 bundle2: enforce parttype as alphanumerical Pierre-Yves David <pierre-yves.david@fb.com> parents: 23590 diff changeset	205 def validateparttype(parttype):
405eacbecc52 bundle2: enforce parttype as alphanumerical Pierre-Yves David <pierre-yves.david@fb.com> parents: 23590 diff changeset	206 """raise ValueError if a parttype contains invalid character"""
23916 a3f7c781786b bundle2: fix parttype enforcement Matt Mackall <mpm@selenic.com> parents: 23868 diff changeset	207 if _parttypeforbidden.search(parttype):
23868 405eacbecc52 bundle2: enforce parttype as alphanumerical Pierre-Yves David <pierre-yves.david@fb.com> parents: 23590 diff changeset	208 raise ValueError(parttype)
405eacbecc52 bundle2: enforce parttype as alphanumerical Pierre-Yves David <pierre-yves.david@fb.com> parents: 23590 diff changeset	209
20877 9e9e3a4e9261 bundle2: part params Pierre-Yves David <pierre-yves.david@fb.com> parents: 20876 diff changeset	210 def _makefpartparamsizes(nbparams):
9e9e3a4e9261 bundle2: part params Pierre-Yves David <pierre-yves.david@fb.com> parents: 20876 diff changeset	211 """return a struct format to read part parameter sizes
9e9e3a4e9261 bundle2: part params Pierre-Yves David <pierre-yves.david@fb.com> parents: 20876 diff changeset	212
9e9e3a4e9261 bundle2: part params Pierre-Yves David <pierre-yves.david@fb.com> parents: 20876 diff changeset	213 The number parameters is variable so we need to build that format
9e9e3a4e9261 bundle2: part params Pierre-Yves David <pierre-yves.david@fb.com> parents: 20876 diff changeset	214 dynamically.
9e9e3a4e9261 bundle2: part params Pierre-Yves David <pierre-yves.david@fb.com> parents: 20876 diff changeset	215 """
9e9e3a4e9261 bundle2: part params Pierre-Yves David <pierre-yves.david@fb.com> parents: 20876 diff changeset	216 return '>'+('BB'*nbparams)
20804 db9d3991d2c6 bundle2: support bundling simple parameter Pierre-Yves David <pierre-yves.david@fb.com> parents: 20803 diff changeset	217
20890 ec7fc110faee bundle2: introduce a `parthandler` decorator Pierre-Yves David <pierre-yves.david@fb.com> parents: 20889 diff changeset	218 parthandlermapping = {}
20889 deed5edb72de bundle2: first version of a bundle processing Pierre-Yves David <pierre-yves.david@fb.com> parents: 20887 diff changeset	219
21623 5b26d82e4e2a bundle2: make it possible to declare params handled by a part handler Pierre-Yves David <pierre-yves.david@fb.com> parents: 21622 diff changeset	220 def parthandler(parttype, params=()):
20890 ec7fc110faee bundle2: introduce a `parthandler` decorator Pierre-Yves David <pierre-yves.david@fb.com> parents: 20889 diff changeset	221 """decorator that register a function as a bundle2 part handler
ec7fc110faee bundle2: introduce a `parthandler` decorator Pierre-Yves David <pierre-yves.david@fb.com> parents: 20889 diff changeset	222
ec7fc110faee bundle2: introduce a `parthandler` decorator Pierre-Yves David <pierre-yves.david@fb.com> parents: 20889 diff changeset	223 eg::
ec7fc110faee bundle2: introduce a `parthandler` decorator Pierre-Yves David <pierre-yves.david@fb.com> parents: 20889 diff changeset	224
21624 d61066d787c8 bundle2: declare supported parameters for all handlers Pierre-Yves David <pierre-yves.david@fb.com> parents: 21623 diff changeset	225 @parthandler('myparttype', ('mandatory', 'param', 'handled'))
20890 ec7fc110faee bundle2: introduce a `parthandler` decorator Pierre-Yves David <pierre-yves.david@fb.com> parents: 20889 diff changeset	226 def myparttypehandler(...):
ec7fc110faee bundle2: introduce a `parthandler` decorator Pierre-Yves David <pierre-yves.david@fb.com> parents: 20889 diff changeset	227 '''process a part of type "my part".'''
ec7fc110faee bundle2: introduce a `parthandler` decorator Pierre-Yves David <pierre-yves.david@fb.com> parents: 20889 diff changeset	228 ...
ec7fc110faee bundle2: introduce a `parthandler` decorator Pierre-Yves David <pierre-yves.david@fb.com> parents: 20889 diff changeset	229 """
23868 405eacbecc52 bundle2: enforce parttype as alphanumerical Pierre-Yves David <pierre-yves.david@fb.com> parents: 23590 diff changeset	230 validateparttype(parttype)
20890 ec7fc110faee bundle2: introduce a `parthandler` decorator Pierre-Yves David <pierre-yves.david@fb.com> parents: 20889 diff changeset	231 def _decorator(func):
20891 1c6cd23fc221 bundle2: add some distinction between mandatory and advisory part Pierre-Yves David <pierre-yves.david@fb.com> parents: 20890 diff changeset	232 lparttype = parttype.lower() # enforce lower case matching.
1c6cd23fc221 bundle2: add some distinction between mandatory and advisory part Pierre-Yves David <pierre-yves.david@fb.com> parents: 20890 diff changeset	233 assert lparttype not in parthandlermapping
1c6cd23fc221 bundle2: add some distinction between mandatory and advisory part Pierre-Yves David <pierre-yves.david@fb.com> parents: 20890 diff changeset	234 parthandlermapping[lparttype] = func
21623 5b26d82e4e2a bundle2: make it possible to declare params handled by a part handler Pierre-Yves David <pierre-yves.david@fb.com> parents: 21622 diff changeset	235 func.params = frozenset(params)
20890 ec7fc110faee bundle2: introduce a `parthandler` decorator Pierre-Yves David <pierre-yves.david@fb.com> parents: 20889 diff changeset	236 return func
ec7fc110faee bundle2: introduce a `parthandler` decorator Pierre-Yves David <pierre-yves.david@fb.com> parents: 20889 diff changeset	237 return _decorator
20889 deed5edb72de bundle2: first version of a bundle processing Pierre-Yves David <pierre-yves.david@fb.com> parents: 20887 diff changeset	238
20949 571f2903ff1e bundle2: record processing results in the bundleoperation object Pierre-Yves David <pierre-yves.david@fb.com> parents: 20948 diff changeset	239 class unbundlerecords(object):
571f2903ff1e bundle2: record processing results in the bundleoperation object Pierre-Yves David <pierre-yves.david@fb.com> parents: 20948 diff changeset	240 """keep record of what happens during and unbundle
571f2903ff1e bundle2: record processing results in the bundleoperation object Pierre-Yves David <pierre-yves.david@fb.com> parents: 20948 diff changeset	241
571f2903ff1e bundle2: record processing results in the bundleoperation object Pierre-Yves David <pierre-yves.david@fb.com> parents: 20948 diff changeset	242 New records are added using `records.add('cat', obj)`. Where 'cat' is a
21024 7731a2281cf0 spelling: fixes from spell checker Mads Kiilerich <madski@unity3d.com> parents: 21020 diff changeset	243 category of record and obj is an arbitrary object.
20949 571f2903ff1e bundle2: record processing results in the bundleoperation object Pierre-Yves David <pierre-yves.david@fb.com> parents: 20948 diff changeset	244
571f2903ff1e bundle2: record processing results in the bundleoperation object Pierre-Yves David <pierre-yves.david@fb.com> parents: 20948 diff changeset	245 `records['cat']` will return all entries of this category 'cat'.
571f2903ff1e bundle2: record processing results in the bundleoperation object Pierre-Yves David <pierre-yves.david@fb.com> parents: 20948 diff changeset	246
571f2903ff1e bundle2: record processing results in the bundleoperation object Pierre-Yves David <pierre-yves.david@fb.com> parents: 20948 diff changeset	247 Iterating on the object itself will yield `('category', obj)` tuples
571f2903ff1e bundle2: record processing results in the bundleoperation object Pierre-Yves David <pierre-yves.david@fb.com> parents: 20948 diff changeset	248 for all entries.
571f2903ff1e bundle2: record processing results in the bundleoperation object Pierre-Yves David <pierre-yves.david@fb.com> parents: 20948 diff changeset	249
571f2903ff1e bundle2: record processing results in the bundleoperation object Pierre-Yves David <pierre-yves.david@fb.com> parents: 20948 diff changeset	250 All iterations happens in chronological order.
571f2903ff1e bundle2: record processing results in the bundleoperation object Pierre-Yves David <pierre-yves.david@fb.com> parents: 20948 diff changeset	251 """
571f2903ff1e bundle2: record processing results in the bundleoperation object Pierre-Yves David <pierre-yves.david@fb.com> parents: 20948 diff changeset	252
571f2903ff1e bundle2: record processing results in the bundleoperation object Pierre-Yves David <pierre-yves.david@fb.com> parents: 20948 diff changeset	253 def __init__(self):
571f2903ff1e bundle2: record processing results in the bundleoperation object Pierre-Yves David <pierre-yves.david@fb.com> parents: 20948 diff changeset	254 self._categories = {}
571f2903ff1e bundle2: record processing results in the bundleoperation object Pierre-Yves David <pierre-yves.david@fb.com> parents: 20948 diff changeset	255 self._sequences = []
20996 ed3c5e18a047 bundle2: add reply awareness to unbundlerecords Pierre-Yves David <pierre-yves.david@fb.com> parents: 20995 diff changeset	256 self._replies = {}
20949 571f2903ff1e bundle2: record processing results in the bundleoperation object Pierre-Yves David <pierre-yves.david@fb.com> parents: 20948 diff changeset	257
20996 ed3c5e18a047 bundle2: add reply awareness to unbundlerecords Pierre-Yves David <pierre-yves.david@fb.com> parents: 20995 diff changeset	258 def add(self, category, entry, inreplyto=None):
20949 571f2903ff1e bundle2: record processing results in the bundleoperation object Pierre-Yves David <pierre-yves.david@fb.com> parents: 20948 diff changeset	259 """add a new record of a given category.
571f2903ff1e bundle2: record processing results in the bundleoperation object Pierre-Yves David <pierre-yves.david@fb.com> parents: 20948 diff changeset	260
571f2903ff1e bundle2: record processing results in the bundleoperation object Pierre-Yves David <pierre-yves.david@fb.com> parents: 20948 diff changeset	261 The entry can then be retrieved in the list returned by
571f2903ff1e bundle2: record processing results in the bundleoperation object Pierre-Yves David <pierre-yves.david@fb.com> parents: 20948 diff changeset	262 self['category']."""
571f2903ff1e bundle2: record processing results in the bundleoperation object Pierre-Yves David <pierre-yves.david@fb.com> parents: 20948 diff changeset	263 self._categories.setdefault(category, []).append(entry)
571f2903ff1e bundle2: record processing results in the bundleoperation object Pierre-Yves David <pierre-yves.david@fb.com> parents: 20948 diff changeset	264 self._sequences.append((category, entry))
20996 ed3c5e18a047 bundle2: add reply awareness to unbundlerecords Pierre-Yves David <pierre-yves.david@fb.com> parents: 20995 diff changeset	265 if inreplyto is not None:
ed3c5e18a047 bundle2: add reply awareness to unbundlerecords Pierre-Yves David <pierre-yves.david@fb.com> parents: 20995 diff changeset	266 self.getreplies(inreplyto).add(category, entry)
ed3c5e18a047 bundle2: add reply awareness to unbundlerecords Pierre-Yves David <pierre-yves.david@fb.com> parents: 20995 diff changeset	267
ed3c5e18a047 bundle2: add reply awareness to unbundlerecords Pierre-Yves David <pierre-yves.david@fb.com> parents: 20995 diff changeset	268 def getreplies(self, partid):
23139 e53f6b72a0e4 spelling: fixes from proofreading of spell checker issues Mads Kiilerich <madski@unity3d.com> parents: 23067 diff changeset	269 """get the records that are replies to a specific part"""
20996 ed3c5e18a047 bundle2: add reply awareness to unbundlerecords Pierre-Yves David <pierre-yves.david@fb.com> parents: 20995 diff changeset	270 return self._replies.setdefault(partid, unbundlerecords())
20949 571f2903ff1e bundle2: record processing results in the bundleoperation object Pierre-Yves David <pierre-yves.david@fb.com> parents: 20948 diff changeset	271
571f2903ff1e bundle2: record processing results in the bundleoperation object Pierre-Yves David <pierre-yves.david@fb.com> parents: 20948 diff changeset	272 def __getitem__(self, cat):
571f2903ff1e bundle2: record processing results in the bundleoperation object Pierre-Yves David <pierre-yves.david@fb.com> parents: 20948 diff changeset	273 return tuple(self._categories.get(cat, ()))
571f2903ff1e bundle2: record processing results in the bundleoperation object Pierre-Yves David <pierre-yves.david@fb.com> parents: 20948 diff changeset	274
571f2903ff1e bundle2: record processing results in the bundleoperation object Pierre-Yves David <pierre-yves.david@fb.com> parents: 20948 diff changeset	275 def __iter__(self):
571f2903ff1e bundle2: record processing results in the bundleoperation object Pierre-Yves David <pierre-yves.david@fb.com> parents: 20948 diff changeset	276 return iter(self._sequences)
571f2903ff1e bundle2: record processing results in the bundleoperation object Pierre-Yves David <pierre-yves.david@fb.com> parents: 20948 diff changeset	277
571f2903ff1e bundle2: record processing results in the bundleoperation object Pierre-Yves David <pierre-yves.david@fb.com> parents: 20948 diff changeset	278 def __len__(self):
571f2903ff1e bundle2: record processing results in the bundleoperation object Pierre-Yves David <pierre-yves.david@fb.com> parents: 20948 diff changeset	279 return len(self._sequences)
571f2903ff1e bundle2: record processing results in the bundleoperation object Pierre-Yves David <pierre-yves.david@fb.com> parents: 20948 diff changeset	280
571f2903ff1e bundle2: record processing results in the bundleoperation object Pierre-Yves David <pierre-yves.david@fb.com> parents: 20948 diff changeset	281 def __nonzero__(self):
571f2903ff1e bundle2: record processing results in the bundleoperation object Pierre-Yves David <pierre-yves.david@fb.com> parents: 20948 diff changeset	282 return bool(self._sequences)
571f2903ff1e bundle2: record processing results in the bundleoperation object Pierre-Yves David <pierre-yves.david@fb.com> parents: 20948 diff changeset	283
31476 413b44003462 py3: add __bool__ to every class defining __nonzero__ Gregory Szorc <gregory.szorc@gmail.com> parents: 30965 diff changeset	284 __bool__ = __nonzero__
413b44003462 py3: add __bool__ to every class defining __nonzero__ Gregory Szorc <gregory.szorc@gmail.com> parents: 30965 diff changeset	285
20948 329cd74b52bd bundle2: introduce a bundleoperation object Pierre-Yves David <pierre-yves.david@fb.com> parents: 20947 diff changeset	286 class bundleoperation(object):
329cd74b52bd bundle2: introduce a bundleoperation object Pierre-Yves David <pierre-yves.david@fb.com> parents: 20947 diff changeset	287 """an object that represents a single bundling process
329cd74b52bd bundle2: introduce a bundleoperation object Pierre-Yves David <pierre-yves.david@fb.com> parents: 20947 diff changeset	288
329cd74b52bd bundle2: introduce a bundleoperation object Pierre-Yves David <pierre-yves.david@fb.com> parents: 20947 diff changeset	289 Its purpose is to carry unbundle-related objects and states.
329cd74b52bd bundle2: introduce a bundleoperation object Pierre-Yves David <pierre-yves.david@fb.com> parents: 20947 diff changeset	290
329cd74b52bd bundle2: introduce a bundleoperation object Pierre-Yves David <pierre-yves.david@fb.com> parents: 20947 diff changeset	291 A new object should be created at the beginning of each bundle processing.
329cd74b52bd bundle2: introduce a bundleoperation object Pierre-Yves David <pierre-yves.david@fb.com> parents: 20947 diff changeset	292 The object is to be returned by the processing function.
329cd74b52bd bundle2: introduce a bundleoperation object Pierre-Yves David <pierre-yves.david@fb.com> parents: 20947 diff changeset	293
329cd74b52bd bundle2: introduce a bundleoperation object Pierre-Yves David <pierre-yves.david@fb.com> parents: 20947 diff changeset	294 The object has very little content now it will ultimately contain:
329cd74b52bd bundle2: introduce a bundleoperation object Pierre-Yves David <pierre-yves.david@fb.com> parents: 20947 diff changeset	295 * an access to the repo the bundle is applied to,
329cd74b52bd bundle2: introduce a bundleoperation object Pierre-Yves David <pierre-yves.david@fb.com> parents: 20947 diff changeset	296 * a ui object,
329cd74b52bd bundle2: introduce a bundleoperation object Pierre-Yves David <pierre-yves.david@fb.com> parents: 20947 diff changeset	297 * a way to retrieve a transaction to add changes to the repo,
329cd74b52bd bundle2: introduce a bundleoperation object Pierre-Yves David <pierre-yves.david@fb.com> parents: 20947 diff changeset	298 * a way to record the result of processing each part,
329cd74b52bd bundle2: introduce a bundleoperation object Pierre-Yves David <pierre-yves.david@fb.com> parents: 20947 diff changeset	299 * a way to construct a bundle response when applicable.
329cd74b52bd bundle2: introduce a bundleoperation object Pierre-Yves David <pierre-yves.david@fb.com> parents: 20947 diff changeset	300 """
329cd74b52bd bundle2: introduce a bundleoperation object Pierre-Yves David <pierre-yves.david@fb.com> parents: 20947 diff changeset	301
24878 e530cde6d115 bundle2: disable ouput capture unless we use http (issue4613 issue4615) Pierre-Yves David <pierre-yves.david@fb.com> parents: 24851 diff changeset	302 def __init__(self, repo, transactiongetter, captureoutput=True):
20948 329cd74b52bd bundle2: introduce a bundleoperation object Pierre-Yves David <pierre-yves.david@fb.com> parents: 20947 diff changeset	303 self.repo = repo
329cd74b52bd bundle2: introduce a bundleoperation object Pierre-Yves David <pierre-yves.david@fb.com> parents: 20947 diff changeset	304 self.ui = repo.ui
20949 571f2903ff1e bundle2: record processing results in the bundleoperation object Pierre-Yves David <pierre-yves.david@fb.com> parents: 20948 diff changeset	305 self.records = unbundlerecords()
20997 d7df4b7378ae bundle2: produce a bundle2 reply Pierre-Yves David <pierre-yves.david@fb.com> parents: 20996 diff changeset	306 self.reply = None
24878 e530cde6d115 bundle2: disable ouput capture unless we use http (issue4613 issue4615) Pierre-Yves David <pierre-yves.david@fb.com> parents: 24851 diff changeset	307 self.captureoutput = captureoutput
33664 f3407d56a6e8 bundle2: add the capability to store hookargs on bundle operation object Pulkit Goyal <7895pulkit@gmail.com> parents: 33541 diff changeset	308 self.hookargs = {}
33665 5fc4ddfbe626 bundle2: load hookargs from bundleoperation into transaction when started one Pulkit Goyal <7895pulkit@gmail.com> parents: 33664 diff changeset	309 self._gettransaction = transactiongetter
35265 1f30cbac34b6 bundle2: add a 'modes' dictionary to the bundle operation Boris Feld <boris.feld@octobus.net> parents: 35261 diff changeset	310 # carries value that can modify part behavior
1f30cbac34b6 bundle2: add a 'modes' dictionary to the bundle operation Boris Feld <boris.feld@octobus.net> parents: 35261 diff changeset	311 self.modes = {}
33665 5fc4ddfbe626 bundle2: load hookargs from bundleoperation into transaction when started one Pulkit Goyal <7895pulkit@gmail.com> parents: 33664 diff changeset	312
5fc4ddfbe626 bundle2: load hookargs from bundleoperation into transaction when started one Pulkit Goyal <7895pulkit@gmail.com> parents: 33664 diff changeset	313 def gettransaction(self):
5fc4ddfbe626 bundle2: load hookargs from bundleoperation into transaction when started one Pulkit Goyal <7895pulkit@gmail.com> parents: 33664 diff changeset	314 transaction = self._gettransaction()
5fc4ddfbe626 bundle2: load hookargs from bundleoperation into transaction when started one Pulkit Goyal <7895pulkit@gmail.com> parents: 33664 diff changeset	315
33810 b3a41f4d837b bundle2: relax the condition to update transaction.hookargs Yuya Nishihara <yuya@tcha.org> parents: 33809 diff changeset	316 if self.hookargs:
33665 5fc4ddfbe626 bundle2: load hookargs from bundleoperation into transaction when started one Pulkit Goyal <7895pulkit@gmail.com> parents: 33664 diff changeset	317 # the ones added to the transaction supercede those added
5fc4ddfbe626 bundle2: load hookargs from bundleoperation into transaction when started one Pulkit Goyal <7895pulkit@gmail.com> parents: 33664 diff changeset	318 # to the operation.
5fc4ddfbe626 bundle2: load hookargs from bundleoperation into transaction when started one Pulkit Goyal <7895pulkit@gmail.com> parents: 33664 diff changeset	319 self.hookargs.update(transaction.hookargs)
5fc4ddfbe626 bundle2: load hookargs from bundleoperation into transaction when started one Pulkit Goyal <7895pulkit@gmail.com> parents: 33664 diff changeset	320 transaction.hookargs = self.hookargs
5fc4ddfbe626 bundle2: load hookargs from bundleoperation into transaction when started one Pulkit Goyal <7895pulkit@gmail.com> parents: 33664 diff changeset	321
33810 b3a41f4d837b bundle2: relax the condition to update transaction.hookargs Yuya Nishihara <yuya@tcha.org> parents: 33809 diff changeset	322 # mark the hookargs as flushed. further attempts to add to
b3a41f4d837b bundle2: relax the condition to update transaction.hookargs Yuya Nishihara <yuya@tcha.org> parents: 33809 diff changeset	323 # hookargs will result in an abort.
b3a41f4d837b bundle2: relax the condition to update transaction.hookargs Yuya Nishihara <yuya@tcha.org> parents: 33809 diff changeset	324 self.hookargs = None
33665 5fc4ddfbe626 bundle2: load hookargs from bundleoperation into transaction when started one Pulkit Goyal <7895pulkit@gmail.com> parents: 33664 diff changeset	325
5fc4ddfbe626 bundle2: load hookargs from bundleoperation into transaction when started one Pulkit Goyal <7895pulkit@gmail.com> parents: 33664 diff changeset	326 return transaction
33664 f3407d56a6e8 bundle2: add the capability to store hookargs on bundle operation object Pulkit Goyal <7895pulkit@gmail.com> parents: 33541 diff changeset	327
f3407d56a6e8 bundle2: add the capability to store hookargs on bundle operation object Pulkit Goyal <7895pulkit@gmail.com> parents: 33541 diff changeset	328 def addhookargs(self, hookargs):
33665 5fc4ddfbe626 bundle2: load hookargs from bundleoperation into transaction when started one Pulkit Goyal <7895pulkit@gmail.com> parents: 33664 diff changeset	329 if self.hookargs is None:
33809 1bf5c5507614 bundle2: raise ProgrammingError for invalid call of addhookargs() Yuya Nishihara <yuya@tcha.org> parents: 33770 diff changeset	330 raise error.ProgrammingError('attempted to add hookargs to '
1bf5c5507614 bundle2: raise ProgrammingError for invalid call of addhookargs() Yuya Nishihara <yuya@tcha.org> parents: 33770 diff changeset	331 'operation after transaction started')
33664 f3407d56a6e8 bundle2: add the capability to store hookargs on bundle operation object Pulkit Goyal <7895pulkit@gmail.com> parents: 33541 diff changeset	332 self.hookargs.update(hookargs)
20948 329cd74b52bd bundle2: introduce a bundleoperation object Pierre-Yves David <pierre-yves.david@fb.com> parents: 20947 diff changeset	333
20952 b24ee5076b94 bundle2: make it possible have a global transaction for the unbundling Pierre-Yves David <pierre-yves.david@fb.com> parents: 20950 diff changeset	334 class TransactionUnavailable(RuntimeError):
b24ee5076b94 bundle2: make it possible have a global transaction for the unbundling Pierre-Yves David <pierre-yves.david@fb.com> parents: 20950 diff changeset	335 pass
b24ee5076b94 bundle2: make it possible have a global transaction for the unbundling Pierre-Yves David <pierre-yves.david@fb.com> parents: 20950 diff changeset	336
b24ee5076b94 bundle2: make it possible have a global transaction for the unbundling Pierre-Yves David <pierre-yves.david@fb.com> parents: 20950 diff changeset	337 def _notransaction():
b24ee5076b94 bundle2: make it possible have a global transaction for the unbundling Pierre-Yves David <pierre-yves.david@fb.com> parents: 20950 diff changeset	338 """default method to get a transaction while processing a bundle
b24ee5076b94 bundle2: make it possible have a global transaction for the unbundling Pierre-Yves David <pierre-yves.david@fb.com> parents: 20950 diff changeset	339
b24ee5076b94 bundle2: make it possible have a global transaction for the unbundling Pierre-Yves David <pierre-yves.david@fb.com> parents: 20950 diff changeset	340 Raise an exception to highlight the fact that no transaction was expected
b24ee5076b94 bundle2: make it possible have a global transaction for the unbundling Pierre-Yves David <pierre-yves.david@fb.com> parents: 20950 diff changeset	341 to be created"""
b24ee5076b94 bundle2: make it possible have a global transaction for the unbundling Pierre-Yves David <pierre-yves.david@fb.com> parents: 20950 diff changeset	342 raise TransactionUnavailable()
b24ee5076b94 bundle2: make it possible have a global transaction for the unbundling Pierre-Yves David <pierre-yves.david@fb.com> parents: 20950 diff changeset	343
33043 18c2489ac96d bundle: make applybundle() delegate v1 bundles to applybundle1() Martin von Zweigbergk <martinvonz@google.com> parents: 33040 diff changeset	344 def applybundle(repo, unbundler, tr, source=None, url=None, **kwargs):
26790 28a6c2d72097 bundle2: introduce an "applybundle" function Pierre-Yves David <pierre-yves.david@fb.com> parents: 26698 diff changeset	345 # transform me into unbundler.apply() as soon as the freeze is lifted
33043 18c2489ac96d bundle: make applybundle() delegate v1 bundles to applybundle1() Martin von Zweigbergk <martinvonz@google.com> parents: 33040 diff changeset	346 if isinstance(unbundler, unbundle20):
18c2489ac96d bundle: make applybundle() delegate v1 bundles to applybundle1() Martin von Zweigbergk <martinvonz@google.com> parents: 33040 diff changeset	347 tr.hookargs['bundle2'] = '1'
18c2489ac96d bundle: make applybundle() delegate v1 bundles to applybundle1() Martin von Zweigbergk <martinvonz@google.com> parents: 33040 diff changeset	348 if source is not None and 'source' not in tr.hookargs:
18c2489ac96d bundle: make applybundle() delegate v1 bundles to applybundle1() Martin von Zweigbergk <martinvonz@google.com> parents: 33040 diff changeset	349 tr.hookargs['source'] = source
18c2489ac96d bundle: make applybundle() delegate v1 bundles to applybundle1() Martin von Zweigbergk <martinvonz@google.com> parents: 33040 diff changeset	350 if url is not None and 'url' not in tr.hookargs:
18c2489ac96d bundle: make applybundle() delegate v1 bundles to applybundle1() Martin von Zweigbergk <martinvonz@google.com> parents: 33040 diff changeset	351 tr.hookargs['url'] = url
18c2489ac96d bundle: make applybundle() delegate v1 bundles to applybundle1() Martin von Zweigbergk <martinvonz@google.com> parents: 33040 diff changeset	352 return processbundle(repo, unbundler, lambda: tr)
18c2489ac96d bundle: make applybundle() delegate v1 bundles to applybundle1() Martin von Zweigbergk <martinvonz@google.com> parents: 33040 diff changeset	353 else:
33044 1d2b68952950 bundle: inline applybundle1() Martin von Zweigbergk <martinvonz@google.com> parents: 33043 diff changeset	354 # the transactiongetter won't be used, but we might as well set it
1d2b68952950 bundle: inline applybundle1() Martin von Zweigbergk <martinvonz@google.com> parents: 33043 diff changeset	355 op = bundleoperation(repo, lambda: tr)
1d2b68952950 bundle: inline applybundle1() Martin von Zweigbergk <martinvonz@google.com> parents: 33043 diff changeset	356 _processchangegroup(op, unbundler, tr, source, url, **kwargs)
1d2b68952950 bundle: inline applybundle1() Martin von Zweigbergk <martinvonz@google.com> parents: 33043 diff changeset	357 return op
26790 28a6c2d72097 bundle2: introduce an "applybundle" function Pierre-Yves David <pierre-yves.david@fb.com> parents: 26698 diff changeset	358
34150 e9e0e1143fc5 bundle2: move part iterator a separate class Durham Goode <durham@fb.com> parents: 34101 diff changeset	359 class partiterator(object):
34152 21c2df59a1da bundle2: move exception handling into part iterator Durham Goode <durham@fb.com> parents: 34151 diff changeset	360 def __init__(self, repo, op, unbundler):
34151 550343626bb2 bundle2: move part counter to partiterator Durham Goode <durham@fb.com> parents: 34150 diff changeset	361 self.repo = repo
34152 21c2df59a1da bundle2: move exception handling into part iterator Durham Goode <durham@fb.com> parents: 34151 diff changeset	362 self.op = op
34150 e9e0e1143fc5 bundle2: move part iterator a separate class Durham Goode <durham@fb.com> parents: 34101 diff changeset	363 self.unbundler = unbundler
34151 550343626bb2 bundle2: move part counter to partiterator Durham Goode <durham@fb.com> parents: 34150 diff changeset	364 self.iterator = None
550343626bb2 bundle2: move part counter to partiterator Durham Goode <durham@fb.com> parents: 34150 diff changeset	365 self.count = 0
34258 e71890f27767 bundle2: move processpart stream maintenance into part iterator Durham Goode <durham@fb.com> parents: 34249 diff changeset	366 self.current = None
34150 e9e0e1143fc5 bundle2: move part iterator a separate class Durham Goode <durham@fb.com> parents: 34101 diff changeset	367
e9e0e1143fc5 bundle2: move part iterator a separate class Durham Goode <durham@fb.com> parents: 34101 diff changeset	368 def __enter__(self):
34151 550343626bb2 bundle2: move part counter to partiterator Durham Goode <durham@fb.com> parents: 34150 diff changeset	369 def func():
550343626bb2 bundle2: move part counter to partiterator Durham Goode <durham@fb.com> parents: 34150 diff changeset	370 itr = enumerate(self.unbundler.iterparts())
550343626bb2 bundle2: move part counter to partiterator Durham Goode <durham@fb.com> parents: 34150 diff changeset	371 for count, p in itr:
550343626bb2 bundle2: move part counter to partiterator Durham Goode <durham@fb.com> parents: 34150 diff changeset	372 self.count = count
34258 e71890f27767 bundle2: move processpart stream maintenance into part iterator Durham Goode <durham@fb.com> parents: 34249 diff changeset	373 self.current = p
34151 550343626bb2 bundle2: move part counter to partiterator Durham Goode <durham@fb.com> parents: 34150 diff changeset	374 yield p
35114 db5038525718 bundle2: implement consume() API on unbundlepart Gregory Szorc <gregory.szorc@gmail.com> parents: 35113 diff changeset	375 p.consume()
34258 e71890f27767 bundle2: move processpart stream maintenance into part iterator Durham Goode <durham@fb.com> parents: 34249 diff changeset	376 self.current = None
34151 550343626bb2 bundle2: move part counter to partiterator Durham Goode <durham@fb.com> parents: 34150 diff changeset	377 self.iterator = func()
550343626bb2 bundle2: move part counter to partiterator Durham Goode <durham@fb.com> parents: 34150 diff changeset	378 return self.iterator
34150 e9e0e1143fc5 bundle2: move part iterator a separate class Durham Goode <durham@fb.com> parents: 34101 diff changeset	379
34152 21c2df59a1da bundle2: move exception handling into part iterator Durham Goode <durham@fb.com> parents: 34151 diff changeset	380 def __exit__(self, type, exc, tb):
34151 550343626bb2 bundle2: move part counter to partiterator Durham Goode <durham@fb.com> parents: 34150 diff changeset	381 if not self.iterator:
550343626bb2 bundle2: move part counter to partiterator Durham Goode <durham@fb.com> parents: 34150 diff changeset	382 return
550343626bb2 bundle2: move part counter to partiterator Durham Goode <durham@fb.com> parents: 34150 diff changeset	383
34637 5f79f5f8487a bundle2: immediate exit for ctrl+c (issue5692) Durham Goode <durham@fb.com> parents: 34323 diff changeset	384 # Only gracefully abort in a normal exception situation. User aborts
5f79f5f8487a bundle2: immediate exit for ctrl+c (issue5692) Durham Goode <durham@fb.com> parents: 34323 diff changeset	385 # like Ctrl+C throw a KeyboardInterrupt which is not a base Exception,
5f79f5f8487a bundle2: immediate exit for ctrl+c (issue5692) Durham Goode <durham@fb.com> parents: 34323 diff changeset	386 # and should not gracefully cleanup.
5f79f5f8487a bundle2: immediate exit for ctrl+c (issue5692) Durham Goode <durham@fb.com> parents: 34323 diff changeset	387 if isinstance(exc, Exception):
34152 21c2df59a1da bundle2: move exception handling into part iterator Durham Goode <durham@fb.com> parents: 34151 diff changeset	388 # Any exceptions seeking to the end of the bundle at this point are
21c2df59a1da bundle2: move exception handling into part iterator Durham Goode <durham@fb.com> parents: 34151 diff changeset	389 # almost certainly related to the underlying stream being bad.
21c2df59a1da bundle2: move exception handling into part iterator Durham Goode <durham@fb.com> parents: 34151 diff changeset	390 # And, chances are that the exception we're handling is related to
21c2df59a1da bundle2: move exception handling into part iterator Durham Goode <durham@fb.com> parents: 34151 diff changeset	391 # getting in that bad state. So, we swallow the seeking error and
21c2df59a1da bundle2: move exception handling into part iterator Durham Goode <durham@fb.com> parents: 34151 diff changeset	392 # re-raise the original error.
21c2df59a1da bundle2: move exception handling into part iterator Durham Goode <durham@fb.com> parents: 34151 diff changeset	393 seekerror = False
21c2df59a1da bundle2: move exception handling into part iterator Durham Goode <durham@fb.com> parents: 34151 diff changeset	394 try:
34637 5f79f5f8487a bundle2: immediate exit for ctrl+c (issue5692) Durham Goode <durham@fb.com> parents: 34323 diff changeset	395 if self.current:
5f79f5f8487a bundle2: immediate exit for ctrl+c (issue5692) Durham Goode <durham@fb.com> parents: 34323 diff changeset	396 # consume the part content to not corrupt the stream.
35114 db5038525718 bundle2: implement consume() API on unbundlepart Gregory Szorc <gregory.szorc@gmail.com> parents: 35113 diff changeset	397 self.current.consume()
34637 5f79f5f8487a bundle2: immediate exit for ctrl+c (issue5692) Durham Goode <durham@fb.com> parents: 34323 diff changeset	398
34152 21c2df59a1da bundle2: move exception handling into part iterator Durham Goode <durham@fb.com> parents: 34151 diff changeset	399 for part in self.iterator:
21c2df59a1da bundle2: move exception handling into part iterator Durham Goode <durham@fb.com> parents: 34151 diff changeset	400 # consume the bundle content
35114 db5038525718 bundle2: implement consume() API on unbundlepart Gregory Szorc <gregory.szorc@gmail.com> parents: 35113 diff changeset	401 part.consume()
34152 21c2df59a1da bundle2: move exception handling into part iterator Durham Goode <durham@fb.com> parents: 34151 diff changeset	402 except Exception:
21c2df59a1da bundle2: move exception handling into part iterator Durham Goode <durham@fb.com> parents: 34151 diff changeset	403 seekerror = True
21c2df59a1da bundle2: move exception handling into part iterator Durham Goode <durham@fb.com> parents: 34151 diff changeset	404
21c2df59a1da bundle2: move exception handling into part iterator Durham Goode <durham@fb.com> parents: 34151 diff changeset	405 # Small hack to let caller code distinguish exceptions from bundle2
21c2df59a1da bundle2: move exception handling into part iterator Durham Goode <durham@fb.com> parents: 34151 diff changeset	406 # processing from processing the old format. This is mostly needed
21c2df59a1da bundle2: move exception handling into part iterator Durham Goode <durham@fb.com> parents: 34151 diff changeset	407 # to handle different return codes to unbundle according to the type
21c2df59a1da bundle2: move exception handling into part iterator Durham Goode <durham@fb.com> parents: 34151 diff changeset	408 # of bundle. We should probably clean up or drop this return code
21c2df59a1da bundle2: move exception handling into part iterator Durham Goode <durham@fb.com> parents: 34151 diff changeset	409 # craziness in a future version.
21c2df59a1da bundle2: move exception handling into part iterator Durham Goode <durham@fb.com> parents: 34151 diff changeset	410 exc.duringunbundle2 = True
21c2df59a1da bundle2: move exception handling into part iterator Durham Goode <durham@fb.com> parents: 34151 diff changeset	411 salvaged = []
21c2df59a1da bundle2: move exception handling into part iterator Durham Goode <durham@fb.com> parents: 34151 diff changeset	412 replycaps = None
21c2df59a1da bundle2: move exception handling into part iterator Durham Goode <durham@fb.com> parents: 34151 diff changeset	413 if self.op.reply is not None:
21c2df59a1da bundle2: move exception handling into part iterator Durham Goode <durham@fb.com> parents: 34151 diff changeset	414 salvaged = self.op.reply.salvageoutput()
21c2df59a1da bundle2: move exception handling into part iterator Durham Goode <durham@fb.com> parents: 34151 diff changeset	415 replycaps = self.op.reply.capabilities
21c2df59a1da bundle2: move exception handling into part iterator Durham Goode <durham@fb.com> parents: 34151 diff changeset	416 exc._replycaps = replycaps
21c2df59a1da bundle2: move exception handling into part iterator Durham Goode <durham@fb.com> parents: 34151 diff changeset	417 exc._bundle2salvagedoutput = salvaged
21c2df59a1da bundle2: move exception handling into part iterator Durham Goode <durham@fb.com> parents: 34151 diff changeset	418
21c2df59a1da bundle2: move exception handling into part iterator Durham Goode <durham@fb.com> parents: 34151 diff changeset	419 # Re-raising from a variable loses the original stack. So only use
21c2df59a1da bundle2: move exception handling into part iterator Durham Goode <durham@fb.com> parents: 34151 diff changeset	420 # that form if we need to.
21c2df59a1da bundle2: move exception handling into part iterator Durham Goode <durham@fb.com> parents: 34151 diff changeset	421 if seekerror:
21c2df59a1da bundle2: move exception handling into part iterator Durham Goode <durham@fb.com> parents: 34151 diff changeset	422 raise exc
21c2df59a1da bundle2: move exception handling into part iterator Durham Goode <durham@fb.com> parents: 34151 diff changeset	423
34151 550343626bb2 bundle2: move part counter to partiterator Durham Goode <durham@fb.com> parents: 34150 diff changeset	424 self.repo.ui.debug('bundle2-input-bundle: %i parts total\n' %
550343626bb2 bundle2: move part counter to partiterator Durham Goode <durham@fb.com> parents: 34150 diff changeset	425 self.count)
34150 e9e0e1143fc5 bundle2: move part iterator a separate class Durham Goode <durham@fb.com> parents: 34101 diff changeset	426
24851 df0ce98c882f bundle2: also save output when error happens during part processing Pierre-Yves David <pierre-yves.david@fb.com> parents: 24849 diff changeset	427 def processbundle(repo, unbundler, transactiongetter=None, op=None):
20889 deed5edb72de bundle2: first version of a bundle processing Pierre-Yves David <pierre-yves.david@fb.com> parents: 20887 diff changeset	428 """This function process a bundle, apply effect to/from a repo
deed5edb72de bundle2: first version of a bundle processing Pierre-Yves David <pierre-yves.david@fb.com> parents: 20887 diff changeset	429
20947 c33d7bf53812 bundle2: feed a unbundle20 to the `processbundle` function Pierre-Yves David <pierre-yves.david@fb.com> parents: 20892 diff changeset	430 It iterates over each part then searches for and uses the proper handling
c33d7bf53812 bundle2: feed a unbundle20 to the `processbundle` function Pierre-Yves David <pierre-yves.david@fb.com> parents: 20892 diff changeset	431 code to process the part. Parts are processed in order.
20889 deed5edb72de bundle2: first version of a bundle processing Pierre-Yves David <pierre-yves.david@fb.com> parents: 20887 diff changeset	432
20891 1c6cd23fc221 bundle2: add some distinction between mandatory and advisory part Pierre-Yves David <pierre-yves.david@fb.com> parents: 20890 diff changeset	433 Unknown Mandatory part will abort the process.
24851 df0ce98c882f bundle2: also save output when error happens during part processing Pierre-Yves David <pierre-yves.david@fb.com> parents: 24849 diff changeset	434
df0ce98c882f bundle2: also save output when error happens during part processing Pierre-Yves David <pierre-yves.david@fb.com> parents: 24849 diff changeset	435 It is temporarily possible to provide a prebuilt bundleoperation to the
df0ce98c882f bundle2: also save output when error happens during part processing Pierre-Yves David <pierre-yves.david@fb.com> parents: 24849 diff changeset	436 function. This is used to ensure output is properly propagated in case of
df0ce98c882f bundle2: also save output when error happens during part processing Pierre-Yves David <pierre-yves.david@fb.com> parents: 24849 diff changeset	437 an error during the unbundling. This output capturing part will likely be
df0ce98c882f bundle2: also save output when error happens during part processing Pierre-Yves David <pierre-yves.david@fb.com> parents: 24849 diff changeset	438 reworked and this ability will probably go away in the process.
20889 deed5edb72de bundle2: first version of a bundle processing Pierre-Yves David <pierre-yves.david@fb.com> parents: 20887 diff changeset	439 """
24851 df0ce98c882f bundle2: also save output when error happens during part processing Pierre-Yves David <pierre-yves.david@fb.com> parents: 24849 diff changeset	440 if op is None:
df0ce98c882f bundle2: also save output when error happens during part processing Pierre-Yves David <pierre-yves.david@fb.com> parents: 24849 diff changeset	441 if transactiongetter is None:
df0ce98c882f bundle2: also save output when error happens during part processing Pierre-Yves David <pierre-yves.david@fb.com> parents: 24849 diff changeset	442 transactiongetter = _notransaction
df0ce98c882f bundle2: also save output when error happens during part processing Pierre-Yves David <pierre-yves.david@fb.com> parents: 24849 diff changeset	443 op = bundleoperation(repo, transactiongetter)
20889 deed5edb72de bundle2: first version of a bundle processing Pierre-Yves David <pierre-yves.david@fb.com> parents: 20887 diff changeset	444 # todo:
deed5edb72de bundle2: first version of a bundle processing Pierre-Yves David <pierre-yves.david@fb.com> parents: 20887 diff changeset	445 # - replace this is a init function soon.
deed5edb72de bundle2: first version of a bundle processing Pierre-Yves David <pierre-yves.david@fb.com> parents: 20887 diff changeset	446 # - exception catching
deed5edb72de bundle2: first version of a bundle processing Pierre-Yves David <pierre-yves.david@fb.com> parents: 20887 diff changeset	447 unbundler.params
25331 53ca47ef645d bundle2: add generic debug output regarding processed bundle Pierre-Yves David <pierre-yves.david@fb.com> parents: 25324 diff changeset	448 if repo.ui.debugflag:
53ca47ef645d bundle2: add generic debug output regarding processed bundle Pierre-Yves David <pierre-yves.david@fb.com> parents: 25324 diff changeset	449 msg = ['bundle2-input-bundle:']
53ca47ef645d bundle2: add generic debug output regarding processed bundle Pierre-Yves David <pierre-yves.david@fb.com> parents: 25324 diff changeset	450 if unbundler.params:
33128 126eae7dae74 bundle2: add debug info about the number of stream params Siddharth Agarwal <sid0@fb.com> parents: 33127 diff changeset	451 msg.append(' %i params' % len(unbundler.params))
33770 119e1c6be1ce bundle2: fix transaction availability detection Boris Feld <boris.feld@octobus.net> parents: 33711 diff changeset	452 if op._gettransaction is None or op._gettransaction is _notransaction:
25331 53ca47ef645d bundle2: add generic debug output regarding processed bundle Pierre-Yves David <pierre-yves.david@fb.com> parents: 25324 diff changeset	453 msg.append(' no-transaction')
53ca47ef645d bundle2: add generic debug output regarding processed bundle Pierre-Yves David <pierre-yves.david@fb.com> parents: 25324 diff changeset	454 else:
53ca47ef645d bundle2: add generic debug output regarding processed bundle Pierre-Yves David <pierre-yves.david@fb.com> parents: 25324 diff changeset	455 msg.append(' with-transaction')
53ca47ef645d bundle2: add generic debug output regarding processed bundle Pierre-Yves David <pierre-yves.david@fb.com> parents: 25324 diff changeset	456 msg.append('\n')
53ca47ef645d bundle2: add generic debug output regarding processed bundle Pierre-Yves David <pierre-yves.david@fb.com> parents: 25324 diff changeset	457 repo.ui.debug(''.join(msg))
34150 e9e0e1143fc5 bundle2: move part iterator a separate class Durham Goode <durham@fb.com> parents: 34101 diff changeset	458
34261 f010097c885c bundle2: move part processing to a separate function Durham Goode <durham@fb.com> parents: 34260 diff changeset	459 processparts(repo, op, unbundler)
f010097c885c bundle2: move part processing to a separate function Durham Goode <durham@fb.com> parents: 34260 diff changeset	460
f010097c885c bundle2: move part processing to a separate function Durham Goode <durham@fb.com> parents: 34260 diff changeset	461 return op
f010097c885c bundle2: move part processing to a separate function Durham Goode <durham@fb.com> parents: 34260 diff changeset	462
f010097c885c bundle2: move part processing to a separate function Durham Goode <durham@fb.com> parents: 34260 diff changeset	463 def processparts(repo, op, unbundler):
34152 21c2df59a1da bundle2: move exception handling into part iterator Durham Goode <durham@fb.com> parents: 34151 diff changeset	464 with partiterator(repo, op, unbundler) as parts:
21c2df59a1da bundle2: move exception handling into part iterator Durham Goode <durham@fb.com> parents: 34151 diff changeset	465 for part in parts:
21c2df59a1da bundle2: move exception handling into part iterator Durham Goode <durham@fb.com> parents: 34151 diff changeset	466 _processpart(op, part)
25332 0cdd1c811705 bundle2: add generic debug output at the end of bundle processing Pierre-Yves David <pierre-yves.david@fb.com> parents: 25331 diff changeset	467
33038 f0efd2bffe1e bundle: extract _processchangegroup() method Martin von Zweigbergk <martinvonz@google.com> parents: 33037 diff changeset	468 def _processchangegroup(op, cg, tr, source, url, **kwargs):
33461 bb72031f0ea8 changegroup: stop returning and recording added nodes in 'cg.apply' Boris Feld <boris.feld@octobus.net> parents: 33460 diff changeset	469 ret = cg.apply(op.repo, tr, source, url, **kwargs)
33038 f0efd2bffe1e bundle: extract _processchangegroup() method Martin von Zweigbergk <martinvonz@google.com> parents: 33037 diff changeset	470 op.records.add('changegroup', {
f0efd2bffe1e bundle: extract _processchangegroup() method Martin von Zweigbergk <martinvonz@google.com> parents: 33037 diff changeset	471 'return': ret,
f0efd2bffe1e bundle: extract _processchangegroup() method Martin von Zweigbergk <martinvonz@google.com> parents: 33037 diff changeset	472 })
f0efd2bffe1e bundle: extract _processchangegroup() method Martin von Zweigbergk <martinvonz@google.com> parents: 33037 diff changeset	473 return ret
f0efd2bffe1e bundle: extract _processchangegroup() method Martin von Zweigbergk <martinvonz@google.com> parents: 33037 diff changeset	474
34259 07e4170f02f3 bundle2: move handler validation out of processpart Durham Goode <durham@fb.com> parents: 34258 diff changeset	475 def _gethandler(op, part):
07e4170f02f3 bundle2: move handler validation out of processpart Durham Goode <durham@fb.com> parents: 34258 diff changeset	476 status = 'unknown' # used by debug output
07e4170f02f3 bundle2: move handler validation out of processpart Durham Goode <durham@fb.com> parents: 34258 diff changeset	477 try:
07e4170f02f3 bundle2: move handler validation out of processpart Durham Goode <durham@fb.com> parents: 34258 diff changeset	478 handler = parthandlermapping.get(part.type)
07e4170f02f3 bundle2: move handler validation out of processpart Durham Goode <durham@fb.com> parents: 34258 diff changeset	479 if handler is None:
07e4170f02f3 bundle2: move handler validation out of processpart Durham Goode <durham@fb.com> parents: 34258 diff changeset	480 status = 'unsupported-type'
07e4170f02f3 bundle2: move handler validation out of processpart Durham Goode <durham@fb.com> parents: 34258 diff changeset	481 raise error.BundleUnknownFeatureError(parttype=part.type)
34277 a48b3db0baba bundle2: stop using %r to quote part names Augie Fackler <augie@google.com> parents: 34261 diff changeset	482 indebug(op.ui, 'found a handler for part %s' % part.type)
34259 07e4170f02f3 bundle2: move handler validation out of processpart Durham Goode <durham@fb.com> parents: 34258 diff changeset	483 unknownparams = part.mandatorykeys - handler.params
07e4170f02f3 bundle2: move handler validation out of processpart Durham Goode <durham@fb.com> parents: 34258 diff changeset	484 if unknownparams:
07e4170f02f3 bundle2: move handler validation out of processpart Durham Goode <durham@fb.com> parents: 34258 diff changeset	485 unknownparams = list(unknownparams)
07e4170f02f3 bundle2: move handler validation out of processpart Durham Goode <durham@fb.com> parents: 34258 diff changeset	486 unknownparams.sort()
34277 a48b3db0baba bundle2: stop using %r to quote part names Augie Fackler <augie@google.com> parents: 34261 diff changeset	487 status = 'unsupported-params (%s)' % ', '.join(unknownparams)
34259 07e4170f02f3 bundle2: move handler validation out of processpart Durham Goode <durham@fb.com> parents: 34258 diff changeset	488 raise error.BundleUnknownFeatureError(parttype=part.type,
07e4170f02f3 bundle2: move handler validation out of processpart Durham Goode <durham@fb.com> parents: 34258 diff changeset	489 params=unknownparams)
07e4170f02f3 bundle2: move handler validation out of processpart Durham Goode <durham@fb.com> parents: 34258 diff changeset	490 status = 'supported'
07e4170f02f3 bundle2: move handler validation out of processpart Durham Goode <durham@fb.com> parents: 34258 diff changeset	491 except error.BundleUnknownFeatureError as exc:
07e4170f02f3 bundle2: move handler validation out of processpart Durham Goode <durham@fb.com> parents: 34258 diff changeset	492 if part.mandatory: # mandatory parts
07e4170f02f3 bundle2: move handler validation out of processpart Durham Goode <durham@fb.com> parents: 34258 diff changeset	493 raise
07e4170f02f3 bundle2: move handler validation out of processpart Durham Goode <durham@fb.com> parents: 34258 diff changeset	494 indebug(op.ui, 'ignoring unsupported advisory part %s' % exc)
07e4170f02f3 bundle2: move handler validation out of processpart Durham Goode <durham@fb.com> parents: 34258 diff changeset	495 return # skip to part processing
07e4170f02f3 bundle2: move handler validation out of processpart Durham Goode <durham@fb.com> parents: 34258 diff changeset	496 finally:
07e4170f02f3 bundle2: move handler validation out of processpart Durham Goode <durham@fb.com> parents: 34258 diff changeset	497 if op.ui.debugflag:
07e4170f02f3 bundle2: move handler validation out of processpart Durham Goode <durham@fb.com> parents: 34258 diff changeset	498 msg = ['bundle2-input-part: "%s"' % part.type]
07e4170f02f3 bundle2: move handler validation out of processpart Durham Goode <durham@fb.com> parents: 34258 diff changeset	499 if not part.mandatory:
07e4170f02f3 bundle2: move handler validation out of processpart Durham Goode <durham@fb.com> parents: 34258 diff changeset	500 msg.append(' (advisory)')
07e4170f02f3 bundle2: move handler validation out of processpart Durham Goode <durham@fb.com> parents: 34258 diff changeset	501 nbmp = len(part.mandatorykeys)
07e4170f02f3 bundle2: move handler validation out of processpart Durham Goode <durham@fb.com> parents: 34258 diff changeset	502 nbap = len(part.params) - nbmp
07e4170f02f3 bundle2: move handler validation out of processpart Durham Goode <durham@fb.com> parents: 34258 diff changeset	503 if nbmp or nbap:
07e4170f02f3 bundle2: move handler validation out of processpart Durham Goode <durham@fb.com> parents: 34258 diff changeset	504 msg.append(' (params:')
07e4170f02f3 bundle2: move handler validation out of processpart Durham Goode <durham@fb.com> parents: 34258 diff changeset	505 if nbmp:
07e4170f02f3 bundle2: move handler validation out of processpart Durham Goode <durham@fb.com> parents: 34258 diff changeset	506 msg.append(' %i mandatory' % nbmp)
07e4170f02f3 bundle2: move handler validation out of processpart Durham Goode <durham@fb.com> parents: 34258 diff changeset	507 if nbap:
07e4170f02f3 bundle2: move handler validation out of processpart Durham Goode <durham@fb.com> parents: 34258 diff changeset	508 msg.append(' %i advisory' % nbmp)
07e4170f02f3 bundle2: move handler validation out of processpart Durham Goode <durham@fb.com> parents: 34258 diff changeset	509 msg.append(')')
07e4170f02f3 bundle2: move handler validation out of processpart Durham Goode <durham@fb.com> parents: 34258 diff changeset	510 msg.append(' %s\n' % status)
07e4170f02f3 bundle2: move handler validation out of processpart Durham Goode <durham@fb.com> parents: 34258 diff changeset	511 op.ui.debug(''.join(msg))
07e4170f02f3 bundle2: move handler validation out of processpart Durham Goode <durham@fb.com> parents: 34258 diff changeset	512
07e4170f02f3 bundle2: move handler validation out of processpart Durham Goode <durham@fb.com> parents: 34258 diff changeset	513 return handler
07e4170f02f3 bundle2: move handler validation out of processpart Durham Goode <durham@fb.com> parents: 34258 diff changeset	514
23008 d3137827016a bundle2: extract processing of part into its own function Pierre-Yves David <pierre-yves.david@fb.com> parents: 23001 diff changeset	515 def _processpart(op, part):
d3137827016a bundle2: extract processing of part into its own function Pierre-Yves David <pierre-yves.david@fb.com> parents: 23001 diff changeset	516 """process a single part from a bundle
d3137827016a bundle2: extract processing of part into its own function Pierre-Yves David <pierre-yves.david@fb.com> parents: 23001 diff changeset	517
d3137827016a bundle2: extract processing of part into its own function Pierre-Yves David <pierre-yves.david@fb.com> parents: 23001 diff changeset	518 The part is guaranteed to have been fully consumed when the function exits
d3137827016a bundle2: extract processing of part into its own function Pierre-Yves David <pierre-yves.david@fb.com> parents: 23001 diff changeset	519 (even if an exception is raised)."""
34260 cc7b37c90616 bundle2: remove unnecessary try finally Durham Goode <durham@fb.com> parents: 34259 diff changeset	520 handler = _gethandler(op, part)
cc7b37c90616 bundle2: remove unnecessary try finally Durham Goode <durham@fb.com> parents: 34259 diff changeset	521 if handler is None:
cc7b37c90616 bundle2: remove unnecessary try finally Durham Goode <durham@fb.com> parents: 34259 diff changeset	522 return
23008 d3137827016a bundle2: extract processing of part into its own function Pierre-Yves David <pierre-yves.david@fb.com> parents: 23001 diff changeset	523
34260 cc7b37c90616 bundle2: remove unnecessary try finally Durham Goode <durham@fb.com> parents: 34259 diff changeset	524 # handler is called outside the above try block so that we don't
cc7b37c90616 bundle2: remove unnecessary try finally Durham Goode <durham@fb.com> parents: 34259 diff changeset	525 # risk catching KeyErrors from anything other than the
cc7b37c90616 bundle2: remove unnecessary try finally Durham Goode <durham@fb.com> parents: 34259 diff changeset	526 # parthandlermapping lookup (any KeyError raised by handler()
cc7b37c90616 bundle2: remove unnecessary try finally Durham Goode <durham@fb.com> parents: 34259 diff changeset	527 # itself represents a defect of a different variety).
cc7b37c90616 bundle2: remove unnecessary try finally Durham Goode <durham@fb.com> parents: 34259 diff changeset	528 output = None
cc7b37c90616 bundle2: remove unnecessary try finally Durham Goode <durham@fb.com> parents: 34259 diff changeset	529 if op.captureoutput and op.reply is not None:
cc7b37c90616 bundle2: remove unnecessary try finally Durham Goode <durham@fb.com> parents: 34259 diff changeset	530 op.ui.pushbuffer(error=True, subproc=True)
cc7b37c90616 bundle2: remove unnecessary try finally Durham Goode <durham@fb.com> parents: 34259 diff changeset	531 output = ''
cc7b37c90616 bundle2: remove unnecessary try finally Durham Goode <durham@fb.com> parents: 34259 diff changeset	532 try:
cc7b37c90616 bundle2: remove unnecessary try finally Durham Goode <durham@fb.com> parents: 34259 diff changeset	533 handler(op, part)
23008 d3137827016a bundle2: extract processing of part into its own function Pierre-Yves David <pierre-yves.david@fb.com> parents: 23001 diff changeset	534 finally:
34260 cc7b37c90616 bundle2: remove unnecessary try finally Durham Goode <durham@fb.com> parents: 34259 diff changeset	535 if output is not None:
cc7b37c90616 bundle2: remove unnecessary try finally Durham Goode <durham@fb.com> parents: 34259 diff changeset	536 output = op.ui.popbuffer()
cc7b37c90616 bundle2: remove unnecessary try finally Durham Goode <durham@fb.com> parents: 34259 diff changeset	537 if output:
cc7b37c90616 bundle2: remove unnecessary try finally Durham Goode <durham@fb.com> parents: 34259 diff changeset	538 outpart = op.reply.newpart('output', data=output,
cc7b37c90616 bundle2: remove unnecessary try finally Durham Goode <durham@fb.com> parents: 34259 diff changeset	539 mandatory=False)
cc7b37c90616 bundle2: remove unnecessary try finally Durham Goode <durham@fb.com> parents: 34259 diff changeset	540 outpart.addparam(
cc7b37c90616 bundle2: remove unnecessary try finally Durham Goode <durham@fb.com> parents: 34259 diff changeset	541 'in-reply-to', pycompat.bytestr(part.id), mandatory=False)
23008 d3137827016a bundle2: extract processing of part into its own function Pierre-Yves David <pierre-yves.david@fb.com> parents: 23001 diff changeset	542
21138 f469879d27ec bundle2: extract capabilities decoding Pierre-Yves David <pierre-yves.david@fb.com> parents: 21137 diff changeset	543 def decodecaps(blob):
23139 e53f6b72a0e4 spelling: fixes from proofreading of spell checker issues Mads Kiilerich <madski@unity3d.com> parents: 23067 diff changeset	544 """decode a bundle2 caps bytes blob into a dictionary
21138 f469879d27ec bundle2: extract capabilities decoding Pierre-Yves David <pierre-yves.david@fb.com> parents: 21137 diff changeset	545
f469879d27ec bundle2: extract capabilities decoding Pierre-Yves David <pierre-yves.david@fb.com> parents: 21137 diff changeset	546 The blob is a list of capabilities (one per line)
f469879d27ec bundle2: extract capabilities decoding Pierre-Yves David <pierre-yves.david@fb.com> parents: 21137 diff changeset	547 Capabilities may have values using a line of the form::
f469879d27ec bundle2: extract capabilities decoding Pierre-Yves David <pierre-yves.david@fb.com> parents: 21137 diff changeset	548
f469879d27ec bundle2: extract capabilities decoding Pierre-Yves David <pierre-yves.david@fb.com> parents: 21137 diff changeset	549 capability=value1,value2,value3
f469879d27ec bundle2: extract capabilities decoding Pierre-Yves David <pierre-yves.david@fb.com> parents: 21137 diff changeset	550
f469879d27ec bundle2: extract capabilities decoding Pierre-Yves David <pierre-yves.david@fb.com> parents: 21137 diff changeset	551 The values are always a list."""
f469879d27ec bundle2: extract capabilities decoding Pierre-Yves David <pierre-yves.david@fb.com> parents: 21137 diff changeset	552 caps = {}
f469879d27ec bundle2: extract capabilities decoding Pierre-Yves David <pierre-yves.david@fb.com> parents: 21137 diff changeset	553 for line in blob.splitlines():
f469879d27ec bundle2: extract capabilities decoding Pierre-Yves David <pierre-yves.david@fb.com> parents: 21137 diff changeset	554 if not line:
f469879d27ec bundle2: extract capabilities decoding Pierre-Yves David <pierre-yves.david@fb.com> parents: 21137 diff changeset	555 continue
f469879d27ec bundle2: extract capabilities decoding Pierre-Yves David <pierre-yves.david@fb.com> parents: 21137 diff changeset	556 if '=' not in line:
f469879d27ec bundle2: extract capabilities decoding Pierre-Yves David <pierre-yves.david@fb.com> parents: 21137 diff changeset	557 key, vals = line, ()
f469879d27ec bundle2: extract capabilities decoding Pierre-Yves David <pierre-yves.david@fb.com> parents: 21137 diff changeset	558 else:
f469879d27ec bundle2: extract capabilities decoding Pierre-Yves David <pierre-yves.david@fb.com> parents: 21137 diff changeset	559 key, vals = line.split('=', 1)
f469879d27ec bundle2: extract capabilities decoding Pierre-Yves David <pierre-yves.david@fb.com> parents: 21137 diff changeset	560 vals = vals.split(',')
28883 032c4c2f802a pycompat: switch to util.urlreq/util.urlerr for py3 compat timeless <timeless@mozdev.org> parents: 28672 diff changeset	561 key = urlreq.unquote(key)
032c4c2f802a pycompat: switch to util.urlreq/util.urlerr for py3 compat timeless <timeless@mozdev.org> parents: 28672 diff changeset	562 vals = [urlreq.unquote(v) for v in vals]
21138 f469879d27ec bundle2: extract capabilities decoding Pierre-Yves David <pierre-yves.david@fb.com> parents: 21137 diff changeset	563 caps[key] = vals
f469879d27ec bundle2: extract capabilities decoding Pierre-Yves David <pierre-yves.david@fb.com> parents: 21137 diff changeset	564 return caps
f469879d27ec bundle2: extract capabilities decoding Pierre-Yves David <pierre-yves.david@fb.com> parents: 21137 diff changeset	565
21139 2b8c82f7f11d bundle2: capabilities encoding Pierre-Yves David <pierre-yves.david@fb.com> parents: 21138 diff changeset	566 def encodecaps(caps):
2b8c82f7f11d bundle2: capabilities encoding Pierre-Yves David <pierre-yves.david@fb.com> parents: 21138 diff changeset	567 """encode a bundle2 caps dictionary into a bytes blob"""
2b8c82f7f11d bundle2: capabilities encoding Pierre-Yves David <pierre-yves.david@fb.com> parents: 21138 diff changeset	568 chunks = []
2b8c82f7f11d bundle2: capabilities encoding Pierre-Yves David <pierre-yves.david@fb.com> parents: 21138 diff changeset	569 for ca in sorted(caps):
2b8c82f7f11d bundle2: capabilities encoding Pierre-Yves David <pierre-yves.david@fb.com> parents: 21138 diff changeset	570 vals = caps[ca]
28883 032c4c2f802a pycompat: switch to util.urlreq/util.urlerr for py3 compat timeless <timeless@mozdev.org> parents: 28672 diff changeset	571 ca = urlreq.quote(ca)
032c4c2f802a pycompat: switch to util.urlreq/util.urlerr for py3 compat timeless <timeless@mozdev.org> parents: 28672 diff changeset	572 vals = [urlreq.quote(v) for v in vals]
21139 2b8c82f7f11d bundle2: capabilities encoding Pierre-Yves David <pierre-yves.david@fb.com> parents: 21138 diff changeset	573 if vals:
2b8c82f7f11d bundle2: capabilities encoding Pierre-Yves David <pierre-yves.david@fb.com> parents: 21138 diff changeset	574 ca = "%s=%s" % (ca, ','.join(vals))
2b8c82f7f11d bundle2: capabilities encoding Pierre-Yves David <pierre-yves.david@fb.com> parents: 21138 diff changeset	575 chunks.append(ca)
2b8c82f7f11d bundle2: capabilities encoding Pierre-Yves David <pierre-yves.david@fb.com> parents: 21138 diff changeset	576 return '\n'.join(chunks)
2b8c82f7f11d bundle2: capabilities encoding Pierre-Yves David <pierre-yves.david@fb.com> parents: 21138 diff changeset	577
28666 ae53ecc47414 bundle: move writebundle() from changegroup.py to bundle2.py (API) Martin von Zweigbergk <martinvonz@google.com> parents: 27953 diff changeset	578 bundletypes = {
30351 f81002f736d7 bundle2: use new compression engine API for compression Gregory Szorc <gregory.szorc@gmail.com> parents: 30177 diff changeset	579 "": ("", 'UN'), # only when using unbundle on ssh and old http servers
28666 ae53ecc47414 bundle: move writebundle() from changegroup.py to bundle2.py (API) Martin von Zweigbergk <martinvonz@google.com> parents: 27953 diff changeset	580 # since the unification ssh accepts a header but there
ae53ecc47414 bundle: move writebundle() from changegroup.py to bundle2.py (API) Martin von Zweigbergk <martinvonz@google.com> parents: 27953 diff changeset	581 # is no capability signaling it.
ae53ecc47414 bundle: move writebundle() from changegroup.py to bundle2.py (API) Martin von Zweigbergk <martinvonz@google.com> parents: 27953 diff changeset	582 "HG20": (), # special-cased below
30351 f81002f736d7 bundle2: use new compression engine API for compression Gregory Szorc <gregory.szorc@gmail.com> parents: 30177 diff changeset	583 "HG10UN": ("HG10UN", 'UN'),
28666 ae53ecc47414 bundle: move writebundle() from changegroup.py to bundle2.py (API) Martin von Zweigbergk <martinvonz@google.com> parents: 27953 diff changeset	584 "HG10BZ": ("HG10", 'BZ'),
ae53ecc47414 bundle: move writebundle() from changegroup.py to bundle2.py (API) Martin von Zweigbergk <martinvonz@google.com> parents: 27953 diff changeset	585 "HG10GZ": ("HG10GZ", 'GZ'),
ae53ecc47414 bundle: move writebundle() from changegroup.py to bundle2.py (API) Martin von Zweigbergk <martinvonz@google.com> parents: 27953 diff changeset	586 }
ae53ecc47414 bundle: move writebundle() from changegroup.py to bundle2.py (API) Martin von Zweigbergk <martinvonz@google.com> parents: 27953 diff changeset	587
ae53ecc47414 bundle: move writebundle() from changegroup.py to bundle2.py (API) Martin von Zweigbergk <martinvonz@google.com> parents: 27953 diff changeset	588 # hgweb uses this list to communicate its preferred type
ae53ecc47414 bundle: move writebundle() from changegroup.py to bundle2.py (API) Martin von Zweigbergk <martinvonz@google.com> parents: 27953 diff changeset	589 bundlepriority = ['HG10GZ', 'HG10BZ', 'HG10UN']
ae53ecc47414 bundle: move writebundle() from changegroup.py to bundle2.py (API) Martin von Zweigbergk <martinvonz@google.com> parents: 27953 diff changeset	590
20801 9c5183cb9bca bundle2: very first version of a bundle2 bundler Pierre-Yves David <pierre-yves.david@fb.com> parents: diff changeset	591 class bundle20(object):
9c5183cb9bca bundle2: very first version of a bundle2 bundler Pierre-Yves David <pierre-yves.david@fb.com> parents: diff changeset	592 """represent an outgoing bundle2 container
9c5183cb9bca bundle2: very first version of a bundle2 bundler Pierre-Yves David <pierre-yves.david@fb.com> parents: diff changeset	593
21599 57cd844d7a5b bundle2: have ``newpart`` automatically add the part to the bundle Pierre-Yves David <pierre-yves.david@fb.com> parents: 21598 diff changeset	594 Use the `addparam` method to add stream level parameter. and `newpart` to
20856 8a6a86c9a5b5 bundle2: support bundling of empty part (with a type) Pierre-Yves David <pierre-yves.david@fb.com> parents: 20844 diff changeset	595 populate it. Then call `getchunks` to retrieve all the binary chunks of
21024 7731a2281cf0 spelling: fixes from spell checker Mads Kiilerich <madski@unity3d.com> parents: 21020 diff changeset	596 data that compose the bundle2 container."""
20801 9c5183cb9bca bundle2: very first version of a bundle2 bundler Pierre-Yves David <pierre-yves.david@fb.com> parents: diff changeset	597
24686 e0e28e910fa3 bundle2: rename format, parts and config to final names Pierre-Yves David <pierre-yves.david@fb.com> parents: 24648 diff changeset	598 _magicstring = 'HG20'
24640 685639f9430d bundle20: move magic string into the class Pierre-Yves David <pierre-yves.david@fb.com> parents: 24071 diff changeset	599
21134 2f8c4fa237f5 bundle2: adds a capabilities attribute on bundler20 Pierre-Yves David <pierre-yves.david@fb.com> parents: 21133 diff changeset	600 def __init__(self, ui, capabilities=()):
20842 938718d72624 bundle2: print debug information during bundling Pierre-Yves David <pierre-yves.david@fb.com> parents: 20814 diff changeset	601 self.ui = ui
20801 9c5183cb9bca bundle2: very first version of a bundle2 bundler Pierre-Yves David <pierre-yves.david@fb.com> parents: diff changeset	602 self._params = []
9c5183cb9bca bundle2: very first version of a bundle2 bundler Pierre-Yves David <pierre-yves.david@fb.com> parents: diff changeset	603 self._parts = []
21136 b6fd496e5c72 bundle2: support for capabilities with values Pierre-Yves David <pierre-yves.david@fb.com> parents: 21135 diff changeset	604 self.capabilities = dict(capabilities)
30351 f81002f736d7 bundle2: use new compression engine API for compression Gregory Szorc <gregory.szorc@gmail.com> parents: 30177 diff changeset	605 self._compengine = util.compengines.forbundletype('UN')
30757 511a4bf52754 bundle2: allow compression options to be passed to compressor Gregory Szorc <gregory.szorc@gmail.com> parents: 30439 diff changeset	606 self._compopts = None
35787 a84dbc87dae9 exchange: send bundle2 stream clones uncompressed Gregory Szorc <gregory.szorc@gmail.com> parents: 35783 diff changeset	607 # If compression is being handled by a consumer of the raw
a84dbc87dae9 exchange: send bundle2 stream clones uncompressed Gregory Szorc <gregory.szorc@gmail.com> parents: 35783 diff changeset	608 # data (e.g. the wire protocol), unsetting this flag tells
a84dbc87dae9 exchange: send bundle2 stream clones uncompressed Gregory Szorc <gregory.szorc@gmail.com> parents: 35783 diff changeset	609 # consumers that the bundle is best left uncompressed.
a84dbc87dae9 exchange: send bundle2 stream clones uncompressed Gregory Szorc <gregory.szorc@gmail.com> parents: 35783 diff changeset	610 self.prefercompressed = True
26404 795f02a24b87 bundle2: allow compressed bundle Pierre-Yves David <pierre-yves.david@fb.com> parents: 26396 diff changeset	611
30757 511a4bf52754 bundle2: allow compression options to be passed to compressor Gregory Szorc <gregory.szorc@gmail.com> parents: 30439 diff changeset	612 def setcompression(self, alg, compopts=None):
26404 795f02a24b87 bundle2: allow compressed bundle Pierre-Yves David <pierre-yves.david@fb.com> parents: 26396 diff changeset	613 """setup core part compression to <alg>"""
30439 71b368e3b590 bundle2: equate 'UN' with no compression Gregory Szorc <gregory.szorc@gmail.com> parents: 30357 diff changeset	614 if alg in (None, 'UN'):
26404 795f02a24b87 bundle2: allow compressed bundle Pierre-Yves David <pierre-yves.david@fb.com> parents: 26396 diff changeset	615 return
30915 aa25989b0658 bundle2: fix assertion that 'compression' hasn't been set Siddharth Agarwal <sid0@fb.com> parents: 30757 diff changeset	616 assert not any(n.lower() == 'compression' for n, v in self._params)
26404 795f02a24b87 bundle2: allow compressed bundle Pierre-Yves David <pierre-yves.david@fb.com> parents: 26396 diff changeset	617 self.addparam('Compression', alg)
30351 f81002f736d7 bundle2: use new compression engine API for compression Gregory Szorc <gregory.szorc@gmail.com> parents: 30177 diff changeset	618 self._compengine = util.compengines.forbundletype(alg)
30757 511a4bf52754 bundle2: allow compression options to be passed to compressor Gregory Szorc <gregory.szorc@gmail.com> parents: 30439 diff changeset	619 self._compopts = compopts
20801 9c5183cb9bca bundle2: very first version of a bundle2 bundler Pierre-Yves David <pierre-yves.david@fb.com> parents: diff changeset	620
21900 b8bd97085ec9 bundle2: add a ``bundle20.nbparts`` property Pierre-Yves David <pierre-yves.david@fb.com> parents: 21660 diff changeset	621 @property
b8bd97085ec9 bundle2: add a ``bundle20.nbparts`` property Pierre-Yves David <pierre-yves.david@fb.com> parents: 21660 diff changeset	622 def nbparts(self):
b8bd97085ec9 bundle2: add a ``bundle20.nbparts`` property Pierre-Yves David <pierre-yves.david@fb.com> parents: 21660 diff changeset	623 """total number of parts added to the bundler"""
b8bd97085ec9 bundle2: add a ``bundle20.nbparts`` property Pierre-Yves David <pierre-yves.david@fb.com> parents: 21660 diff changeset	624 return len(self._parts)
b8bd97085ec9 bundle2: add a ``bundle20.nbparts`` property Pierre-Yves David <pierre-yves.david@fb.com> parents: 21660 diff changeset	625
21597 1daad9dcdba2 bundle2: small doc update on the bundler Pierre-Yves David <pierre-yves.david@fb.com> parents: 21186 diff changeset	626 # methods used to defines the bundle2 content
20804 db9d3991d2c6 bundle2: support bundling simple parameter Pierre-Yves David <pierre-yves.david@fb.com> parents: 20803 diff changeset	627 def addparam(self, name, value=None):
db9d3991d2c6 bundle2: support bundling simple parameter Pierre-Yves David <pierre-yves.david@fb.com> parents: 20803 diff changeset	628 """add a stream level parameter"""
20813 8c74b3ce5b70 bundle2: refuse empty parameter name Pierre-Yves David <pierre-yves.david@fb.com> parents: 20812 diff changeset	629 if not name:
34279 047f2af8ceb0 bundle2: make ValueError messages native strings Augie Fackler <augie@google.com> parents: 34278 diff changeset	630 raise ValueError(r'empty parameter name')
34280 2bdd55abd56c bundle2: portably grab first byte of part name for letter check Augie Fackler <augie@google.com> parents: 34279 diff changeset	631 if name[0:1] not in pycompat.bytestr(string.ascii_letters):
34279 047f2af8ceb0 bundle2: make ValueError messages native strings Augie Fackler <augie@google.com> parents: 34278 diff changeset	632 raise ValueError(r'non letter first character: %s' % name)
20804 db9d3991d2c6 bundle2: support bundling simple parameter Pierre-Yves David <pierre-yves.david@fb.com> parents: 20803 diff changeset	633 self._params.append((name, value))
db9d3991d2c6 bundle2: support bundling simple parameter Pierre-Yves David <pierre-yves.david@fb.com> parents: 20803 diff changeset	634
20856 8a6a86c9a5b5 bundle2: support bundling of empty part (with a type) Pierre-Yves David <pierre-yves.david@fb.com> parents: 20844 diff changeset	635 def addpart(self, part):
8a6a86c9a5b5 bundle2: support bundling of empty part (with a type) Pierre-Yves David <pierre-yves.david@fb.com> parents: 20844 diff changeset	636 """add a new part to the bundle2 container
8a6a86c9a5b5 bundle2: support bundling of empty part (with a type) Pierre-Yves David <pierre-yves.david@fb.com> parents: 20844 diff changeset	637
21024 7731a2281cf0 spelling: fixes from spell checker Mads Kiilerich <madski@unity3d.com> parents: 21020 diff changeset	638 Parts contains the actual applicative payload."""
20995 e995d104c87f bundle2: add an integer id to part Pierre-Yves David <pierre-yves.david@fb.com> parents: 20952 diff changeset	639 assert part.id is None
e995d104c87f bundle2: add an integer id to part Pierre-Yves David <pierre-yves.david@fb.com> parents: 20952 diff changeset	640 part.id = len(self._parts) # very cheap counter
20856 8a6a86c9a5b5 bundle2: support bundling of empty part (with a type) Pierre-Yves David <pierre-yves.david@fb.com> parents: 20844 diff changeset	641 self._parts.append(part)
8a6a86c9a5b5 bundle2: support bundling of empty part (with a type) Pierre-Yves David <pierre-yves.david@fb.com> parents: 20844 diff changeset	642
21598 1b0dbb91de5b bundle2: add a ``newpart`` method to ``bundle20`` Pierre-Yves David <pierre-yves.david@fb.com> parents: 21597 diff changeset	643 def newpart(self, typeid, args, *kwargs):
21602 cc33ae50bab3 bundle2: warn about error during initialization in ``newpart`` docstring Pierre-Yves David <pierre-yves.david@fb.com> parents: 21601 diff changeset	644 """create a new part and add it to the containers
cc33ae50bab3 bundle2: warn about error during initialization in ``newpart`` docstring Pierre-Yves David <pierre-yves.david@fb.com> parents: 21601 diff changeset	645
cc33ae50bab3 bundle2: warn about error during initialization in ``newpart`` docstring Pierre-Yves David <pierre-yves.david@fb.com> parents: 21601 diff changeset	646 As the part is directly added to the containers. For now, this means
cc33ae50bab3 bundle2: warn about error during initialization in ``newpart`` docstring Pierre-Yves David <pierre-yves.david@fb.com> parents: 21601 diff changeset	647 that any failure to properly initialize the part after calling
cc33ae50bab3 bundle2: warn about error during initialization in ``newpart`` docstring Pierre-Yves David <pierre-yves.david@fb.com> parents: 21601 diff changeset	648 ``newpart`` should result in a failure of the whole bundling process.
cc33ae50bab3 bundle2: warn about error during initialization in ``newpart`` docstring Pierre-Yves David <pierre-yves.david@fb.com> parents: 21601 diff changeset	649
cc33ae50bab3 bundle2: warn about error during initialization in ``newpart`` docstring Pierre-Yves David <pierre-yves.david@fb.com> parents: 21601 diff changeset	650 You can still fall back to manually create and add if you need better
cc33ae50bab3 bundle2: warn about error during initialization in ``newpart`` docstring Pierre-Yves David <pierre-yves.david@fb.com> parents: 21601 diff changeset	651 control."""
21598 1b0dbb91de5b bundle2: add a ``newpart`` method to ``bundle20`` Pierre-Yves David <pierre-yves.david@fb.com> parents: 21597 diff changeset	652 part = bundlepart(typeid, args, *kwargs)
21599 57cd844d7a5b bundle2: have ``newpart`` automatically add the part to the bundle Pierre-Yves David <pierre-yves.david@fb.com> parents: 21598 diff changeset	653 self.addpart(part)
21598 1b0dbb91de5b bundle2: add a ``newpart`` method to ``bundle20`` Pierre-Yves David <pierre-yves.david@fb.com> parents: 21597 diff changeset	654 return part
1b0dbb91de5b bundle2: add a ``newpart`` method to ``bundle20`` Pierre-Yves David <pierre-yves.david@fb.com> parents: 21597 diff changeset	655
21597 1daad9dcdba2 bundle2: small doc update on the bundler Pierre-Yves David <pierre-yves.david@fb.com> parents: 21186 diff changeset	656 # methods used to generate the bundle2 stream
20801 9c5183cb9bca bundle2: very first version of a bundle2 bundler Pierre-Yves David <pierre-yves.david@fb.com> parents: diff changeset	657 def getchunks(self):
25322 1f7c0403f8be bundle2: add generic debug output regarding generated bundle Pierre-Yves David <pierre-yves.david@fb.com> parents: 25321 diff changeset	658 if self.ui.debugflag:
1f7c0403f8be bundle2: add generic debug output regarding generated bundle Pierre-Yves David <pierre-yves.david@fb.com> parents: 25321 diff changeset	659 msg = ['bundle2-output-bundle: "%s",' % self._magicstring]
1f7c0403f8be bundle2: add generic debug output regarding generated bundle Pierre-Yves David <pierre-yves.david@fb.com> parents: 25321 diff changeset	660 if self._params:
1f7c0403f8be bundle2: add generic debug output regarding generated bundle Pierre-Yves David <pierre-yves.david@fb.com> parents: 25321 diff changeset	661 msg.append(' (%i params)' % len(self._params))
1f7c0403f8be bundle2: add generic debug output regarding generated bundle Pierre-Yves David <pierre-yves.david@fb.com> parents: 25321 diff changeset	662 msg.append(' %i parts total\n' % len(self._parts))
1f7c0403f8be bundle2: add generic debug output regarding generated bundle Pierre-Yves David <pierre-yves.david@fb.com> parents: 25321 diff changeset	663 self.ui.debug(''.join(msg))
25315 e28245b0da56 bundle2: handle new line in 'outdebug' function Pierre-Yves David <pierre-yves.david@fb.com> parents: 25314 diff changeset	664 outdebug(self.ui, 'start emission of %s stream' % self._magicstring)
24640 685639f9430d bundle20: move magic string into the class Pierre-Yves David <pierre-yves.david@fb.com> parents: 24071 diff changeset	665 yield self._magicstring
20804 db9d3991d2c6 bundle2: support bundling simple parameter Pierre-Yves David <pierre-yves.david@fb.com> parents: 20803 diff changeset	666 param = self._paramchunk()
25315 e28245b0da56 bundle2: handle new line in 'outdebug' function Pierre-Yves David <pierre-yves.david@fb.com> parents: 25314 diff changeset	667 outdebug(self.ui, 'bundle parameter: %s' % param)
20804 db9d3991d2c6 bundle2: support bundling simple parameter Pierre-Yves David <pierre-yves.david@fb.com> parents: 20803 diff changeset	668 yield _pack(_fstreamparamsize, len(param))
db9d3991d2c6 bundle2: support bundling simple parameter Pierre-Yves David <pierre-yves.david@fb.com> parents: 20803 diff changeset	669 if param:
db9d3991d2c6 bundle2: support bundling simple parameter Pierre-Yves David <pierre-yves.david@fb.com> parents: 20803 diff changeset	670 yield param
30757 511a4bf52754 bundle2: allow compression options to be passed to compressor Gregory Szorc <gregory.szorc@gmail.com> parents: 30439 diff changeset	671 for chunk in self._compengine.compressstream(self._getcorechunk(),
511a4bf52754 bundle2: allow compression options to be passed to compressor Gregory Szorc <gregory.szorc@gmail.com> parents: 30439 diff changeset	672 self._compopts):
30357 5925bda42dbd bundle2: use compressstream compression engine API Gregory Szorc <gregory.szorc@gmail.com> parents: 30353 diff changeset	673 yield chunk
20802 520df53ad26a bundle2: a very first version of bundle2 unbundler Pierre-Yves David <pierre-yves.david@fb.com> parents: 20801 diff changeset	674
20804 db9d3991d2c6 bundle2: support bundling simple parameter Pierre-Yves David <pierre-yves.david@fb.com> parents: 20803 diff changeset	675 def _paramchunk(self):
db9d3991d2c6 bundle2: support bundling simple parameter Pierre-Yves David <pierre-yves.david@fb.com> parents: 20803 diff changeset	676 """return a encoded version of all stream parameters"""
db9d3991d2c6 bundle2: support bundling simple parameter Pierre-Yves David <pierre-yves.david@fb.com> parents: 20803 diff changeset	677 blocks = []
20809 b93bb639451a bundle2: support for bundling parameter value Pierre-Yves David <pierre-yves.david@fb.com> parents: 20808 diff changeset	678 for par, value in self._params:
28883 032c4c2f802a pycompat: switch to util.urlreq/util.urlerr for py3 compat timeless <timeless@mozdev.org> parents: 28672 diff changeset	679 par = urlreq.quote(par)
20809 b93bb639451a bundle2: support for bundling parameter value Pierre-Yves David <pierre-yves.david@fb.com> parents: 20808 diff changeset	680 if value is not None:
28883 032c4c2f802a pycompat: switch to util.urlreq/util.urlerr for py3 compat timeless <timeless@mozdev.org> parents: 28672 diff changeset	681 value = urlreq.quote(value)
20809 b93bb639451a bundle2: support for bundling parameter value Pierre-Yves David <pierre-yves.david@fb.com> parents: 20808 diff changeset	682 par = '%s=%s' % (par, value)
b93bb639451a bundle2: support for bundling parameter value Pierre-Yves David <pierre-yves.david@fb.com> parents: 20808 diff changeset	683 blocks.append(par)
20804 db9d3991d2c6 bundle2: support bundling simple parameter Pierre-Yves David <pierre-yves.david@fb.com> parents: 20803 diff changeset	684 return ' '.join(blocks)
db9d3991d2c6 bundle2: support bundling simple parameter Pierre-Yves David <pierre-yves.david@fb.com> parents: 20803 diff changeset	685
26396 d90c30801cdf bundle20: extract core payload generation in its own function Pierre-Yves David <pierre-yves.david@fb.com> parents: 26395 diff changeset	686 def _getcorechunk(self):
d90c30801cdf bundle20: extract core payload generation in its own function Pierre-Yves David <pierre-yves.david@fb.com> parents: 26395 diff changeset	687 """yield chunk for the core part of the bundle
d90c30801cdf bundle20: extract core payload generation in its own function Pierre-Yves David <pierre-yves.david@fb.com> parents: 26395 diff changeset	688
d90c30801cdf bundle20: extract core payload generation in its own function Pierre-Yves David <pierre-yves.david@fb.com> parents: 26395 diff changeset	689 (all but headers and parameters)"""
d90c30801cdf bundle20: extract core payload generation in its own function Pierre-Yves David <pierre-yves.david@fb.com> parents: 26395 diff changeset	690 outdebug(self.ui, 'start of parts')
d90c30801cdf bundle20: extract core payload generation in its own function Pierre-Yves David <pierre-yves.david@fb.com> parents: 26395 diff changeset	691 for part in self._parts:
d90c30801cdf bundle20: extract core payload generation in its own function Pierre-Yves David <pierre-yves.david@fb.com> parents: 26395 diff changeset	692 outdebug(self.ui, 'bundle part: "%s"' % part.type)
d90c30801cdf bundle20: extract core payload generation in its own function Pierre-Yves David <pierre-yves.david@fb.com> parents: 26395 diff changeset	693 for chunk in part.getchunks(ui=self.ui):
d90c30801cdf bundle20: extract core payload generation in its own function Pierre-Yves David <pierre-yves.david@fb.com> parents: 26395 diff changeset	694 yield chunk
d90c30801cdf bundle20: extract core payload generation in its own function Pierre-Yves David <pierre-yves.david@fb.com> parents: 26395 diff changeset	695 outdebug(self.ui, 'end of bundle')
d90c30801cdf bundle20: extract core payload generation in its own function Pierre-Yves David <pierre-yves.david@fb.com> parents: 26395 diff changeset	696 yield _pack(_fpartheadersize, 0)
d90c30801cdf bundle20: extract core payload generation in its own function Pierre-Yves David <pierre-yves.david@fb.com> parents: 26395 diff changeset	697
d90c30801cdf bundle20: extract core payload generation in its own function Pierre-Yves David <pierre-yves.david@fb.com> parents: 26395 diff changeset	698
24794 21f2e8f48ae1 bundle2: add a 'salvageoutput' method on bundle20 Pierre-Yves David <pierre-yves.david@fb.com> parents: 24793 diff changeset	699 def salvageoutput(self):
21f2e8f48ae1 bundle2: add a 'salvageoutput' method on bundle20 Pierre-Yves David <pierre-yves.david@fb.com> parents: 24793 diff changeset	700 """return a list with a copy of all output parts in the bundle
21f2e8f48ae1 bundle2: add a 'salvageoutput' method on bundle20 Pierre-Yves David <pierre-yves.david@fb.com> parents: 24793 diff changeset	701
21f2e8f48ae1 bundle2: add a 'salvageoutput' method on bundle20 Pierre-Yves David <pierre-yves.david@fb.com> parents: 24793 diff changeset	702 This is meant to be used during error handling to make sure we preserve
21f2e8f48ae1 bundle2: add a 'salvageoutput' method on bundle20 Pierre-Yves David <pierre-yves.david@fb.com> parents: 24793 diff changeset	703 server output"""
21f2e8f48ae1 bundle2: add a 'salvageoutput' method on bundle20 Pierre-Yves David <pierre-yves.david@fb.com> parents: 24793 diff changeset	704 salvaged = []
21f2e8f48ae1 bundle2: add a 'salvageoutput' method on bundle20 Pierre-Yves David <pierre-yves.david@fb.com> parents: 24793 diff changeset	705 for part in self._parts:
21f2e8f48ae1 bundle2: add a 'salvageoutput' method on bundle20 Pierre-Yves David <pierre-yves.david@fb.com> parents: 24793 diff changeset	706 if part.type.startswith('output'):
21f2e8f48ae1 bundle2: add a 'salvageoutput' method on bundle20 Pierre-Yves David <pierre-yves.david@fb.com> parents: 24793 diff changeset	707 salvaged.append(part.copy())
21f2e8f48ae1 bundle2: add a 'salvageoutput' method on bundle20 Pierre-Yves David <pierre-yves.david@fb.com> parents: 24793 diff changeset	708 return salvaged
21f2e8f48ae1 bundle2: add a 'salvageoutput' method on bundle20 Pierre-Yves David <pierre-yves.david@fb.com> parents: 24793 diff changeset	709
21f2e8f48ae1 bundle2: add a 'salvageoutput' method on bundle20 Pierre-Yves David <pierre-yves.david@fb.com> parents: 24793 diff changeset	710
21013 a813caca89b3 bundle2: extract stream/unpack logic in an unpackermixin Pierre-Yves David <pierre-yves.david@fb.com> parents: 21005 diff changeset	711 class unpackermixin(object):
a813caca89b3 bundle2: extract stream/unpack logic in an unpackermixin Pierre-Yves David <pierre-yves.david@fb.com> parents: 21005 diff changeset	712 """A mixin to extract bytes and struct data from a stream"""
20802 520df53ad26a bundle2: a very first version of bundle2 unbundler Pierre-Yves David <pierre-yves.david@fb.com> parents: 20801 diff changeset	713
21013 a813caca89b3 bundle2: extract stream/unpack logic in an unpackermixin Pierre-Yves David <pierre-yves.david@fb.com> parents: 21005 diff changeset	714 def __init__(self, fp):
20802 520df53ad26a bundle2: a very first version of bundle2 unbundler Pierre-Yves David <pierre-yves.david@fb.com> parents: 20801 diff changeset	715 self._fp = fp
520df53ad26a bundle2: a very first version of bundle2 unbundler Pierre-Yves David <pierre-yves.david@fb.com> parents: 20801 diff changeset	716
520df53ad26a bundle2: a very first version of bundle2 unbundler Pierre-Yves David <pierre-yves.david@fb.com> parents: 20801 diff changeset	717 def _unpack(self, format):
31862 9bd9e9cb3f69 bundle2: clarify the docstring of unpackermixin methods Pierre-Yves David <pierre-yves.david@ens-lyon.org> parents: 31861 diff changeset	718 """unpack this struct format from the stream
9bd9e9cb3f69 bundle2: clarify the docstring of unpackermixin methods Pierre-Yves David <pierre-yves.david@ens-lyon.org> parents: 31861 diff changeset	719
9bd9e9cb3f69 bundle2: clarify the docstring of unpackermixin methods Pierre-Yves David <pierre-yves.david@ens-lyon.org> parents: 31861 diff changeset	720 This method is meant for internal usage by the bundle2 protocol only.
9bd9e9cb3f69 bundle2: clarify the docstring of unpackermixin methods Pierre-Yves David <pierre-yves.david@ens-lyon.org> parents: 31861 diff changeset	721 They directly manipulate the low level stream including bundle2 level
9bd9e9cb3f69 bundle2: clarify the docstring of unpackermixin methods Pierre-Yves David <pierre-yves.david@ens-lyon.org> parents: 31861 diff changeset	722 instruction.
9bd9e9cb3f69 bundle2: clarify the docstring of unpackermixin methods Pierre-Yves David <pierre-yves.david@ens-lyon.org> parents: 31861 diff changeset	723
9bd9e9cb3f69 bundle2: clarify the docstring of unpackermixin methods Pierre-Yves David <pierre-yves.david@ens-lyon.org> parents: 31861 diff changeset	724 Do not use it to implement higher-level logic or methods."""
20802 520df53ad26a bundle2: a very first version of bundle2 unbundler Pierre-Yves David <pierre-yves.david@fb.com> parents: 20801 diff changeset	725 data = self._readexact(struct.calcsize(format))
520df53ad26a bundle2: a very first version of bundle2 unbundler Pierre-Yves David <pierre-yves.david@fb.com> parents: 20801 diff changeset	726 return _unpack(format, data)
520df53ad26a bundle2: a very first version of bundle2 unbundler Pierre-Yves David <pierre-yves.david@fb.com> parents: 20801 diff changeset	727
520df53ad26a bundle2: a very first version of bundle2 unbundler Pierre-Yves David <pierre-yves.david@fb.com> parents: 20801 diff changeset	728 def _readexact(self, size):
31862 9bd9e9cb3f69 bundle2: clarify the docstring of unpackermixin methods Pierre-Yves David <pierre-yves.david@ens-lyon.org> parents: 31861 diff changeset	729 """read exactly <size> bytes from the stream
9bd9e9cb3f69 bundle2: clarify the docstring of unpackermixin methods Pierre-Yves David <pierre-yves.david@ens-lyon.org> parents: 31861 diff changeset	730
9bd9e9cb3f69 bundle2: clarify the docstring of unpackermixin methods Pierre-Yves David <pierre-yves.david@ens-lyon.org> parents: 31861 diff changeset	731 This method is meant for internal usage by the bundle2 protocol only.
9bd9e9cb3f69 bundle2: clarify the docstring of unpackermixin methods Pierre-Yves David <pierre-yves.david@ens-lyon.org> parents: 31861 diff changeset	732 They directly manipulate the low level stream including bundle2 level
9bd9e9cb3f69 bundle2: clarify the docstring of unpackermixin methods Pierre-Yves David <pierre-yves.david@ens-lyon.org> parents: 31861 diff changeset	733 instruction.
9bd9e9cb3f69 bundle2: clarify the docstring of unpackermixin methods Pierre-Yves David <pierre-yves.david@ens-lyon.org> parents: 31861 diff changeset	734
9bd9e9cb3f69 bundle2: clarify the docstring of unpackermixin methods Pierre-Yves David <pierre-yves.david@ens-lyon.org> parents: 31861 diff changeset	735 Do not use it to implement higher-level logic or methods."""
20802 520df53ad26a bundle2: a very first version of bundle2 unbundler Pierre-Yves David <pierre-yves.david@fb.com> parents: 20801 diff changeset	736 return changegroup.readexactly(self._fp, size)
520df53ad26a bundle2: a very first version of bundle2 unbundler Pierre-Yves David <pierre-yves.david@fb.com> parents: 20801 diff changeset	737
25640 39f0064a3079 bundle2.getunbundler: rename "header" to "magicstring" Pierre-Yves David <pierre-yves.david@fb.com> parents: 25518 diff changeset	738 def getunbundler(ui, fp, magicstring=None):
39f0064a3079 bundle2.getunbundler: rename "header" to "magicstring" Pierre-Yves David <pierre-yves.david@fb.com> parents: 25518 diff changeset	739 """return a valid unbundler object for a given magicstring"""
39f0064a3079 bundle2.getunbundler: rename "header" to "magicstring" Pierre-Yves David <pierre-yves.david@fb.com> parents: 25518 diff changeset	740 if magicstring is None:
39f0064a3079 bundle2.getunbundler: rename "header" to "magicstring" Pierre-Yves David <pierre-yves.david@fb.com> parents: 25518 diff changeset	741 magicstring = changegroup.readexactly(fp, 4)
39f0064a3079 bundle2.getunbundler: rename "header" to "magicstring" Pierre-Yves David <pierre-yves.david@fb.com> parents: 25518 diff changeset	742 magic, version = magicstring[0:2], magicstring[2:4]
24648 5cac3accdaa1 unbundle20: allow generic dispatch between unbundlers Pierre-Yves David <pierre-yves.david@fb.com> parents: 24642 diff changeset	743 if magic != 'HG':
33127 0aae80d14fed bundle2: add some debugging information to the not-a-bundle error Siddharth Agarwal <sid0@fb.com> parents: 33044 diff changeset	744 ui.debug(
0aae80d14fed bundle2: add some debugging information to the not-a-bundle error Siddharth Agarwal <sid0@fb.com> parents: 33044 diff changeset	745 "error: invalid magic: %r (version %r), should be 'HG'\n"
0aae80d14fed bundle2: add some debugging information to the not-a-bundle error Siddharth Agarwal <sid0@fb.com> parents: 33044 diff changeset	746 % (magic, version))
26587 56b2bcea2529 error: get Abort from 'error' instead of 'util' Pierre-Yves David <pierre-yves.david@fb.com> parents: 26565 diff changeset	747 raise error.Abort(_('not a Mercurial bundle'))
24648 5cac3accdaa1 unbundle20: allow generic dispatch between unbundlers Pierre-Yves David <pierre-yves.david@fb.com> parents: 24642 diff changeset	748 unbundlerclass = formatmap.get(version)
5cac3accdaa1 unbundle20: allow generic dispatch between unbundlers Pierre-Yves David <pierre-yves.david@fb.com> parents: 24642 diff changeset	749 if unbundlerclass is None:
26587 56b2bcea2529 error: get Abort from 'error' instead of 'util' Pierre-Yves David <pierre-yves.david@fb.com> parents: 26565 diff changeset	750 raise error.Abort(_('unknown bundle version %s') % version)
24648 5cac3accdaa1 unbundle20: allow generic dispatch between unbundlers Pierre-Yves David <pierre-yves.david@fb.com> parents: 24642 diff changeset	751 unbundler = unbundlerclass(ui, fp)
25640 39f0064a3079 bundle2.getunbundler: rename "header" to "magicstring" Pierre-Yves David <pierre-yves.david@fb.com> parents: 25518 diff changeset	752 indebug(ui, 'start processing of %s stream' % magicstring)
24642 54e5c239c2d9 unbundle20: move header parsing into the 'getunbundler' function Pierre-Yves David <pierre-yves.david@fb.com> parents: 24641 diff changeset	753 return unbundler
24641 60fecc5b14a4 unbundle20: retrieve unbundler instances through a factory function Pierre-Yves David <pierre-yves.david@fb.com> parents: 24640 diff changeset	754
21013 a813caca89b3 bundle2: extract stream/unpack logic in an unpackermixin Pierre-Yves David <pierre-yves.david@fb.com> parents: 21005 diff changeset	755 class unbundle20(unpackermixin):
a813caca89b3 bundle2: extract stream/unpack logic in an unpackermixin Pierre-Yves David <pierre-yves.david@fb.com> parents: 21005 diff changeset	756 """interpret a bundle2 stream
a813caca89b3 bundle2: extract stream/unpack logic in an unpackermixin Pierre-Yves David <pierre-yves.david@fb.com> parents: 21005 diff changeset	757
21129 07bcbf326c8d bundle2: use an official iterparts method to unbundle parts Pierre-Yves David <pierre-yves.david@fb.com> parents: 21066 diff changeset	758 This class is fed with a binary stream and yields parts through its
07bcbf326c8d bundle2: use an official iterparts method to unbundle parts Pierre-Yves David <pierre-yves.david@fb.com> parents: 21066 diff changeset	759 `iterparts` methods."""
21013 a813caca89b3 bundle2: extract stream/unpack logic in an unpackermixin Pierre-Yves David <pierre-yves.david@fb.com> parents: 21005 diff changeset	760
26542 b87e4638dabf bundle2: add a way to just forward the bundle2 stream to another user Pierre-Yves David <pierre-yves.david@fb.com> parents: 26541 diff changeset	761 _magicstring = 'HG20'
b87e4638dabf bundle2: add a way to just forward the bundle2 stream to another user Pierre-Yves David <pierre-yves.david@fb.com> parents: 26541 diff changeset	762
24642 54e5c239c2d9 unbundle20: move header parsing into the 'getunbundler' function Pierre-Yves David <pierre-yves.david@fb.com> parents: 24641 diff changeset	763 def __init__(self, ui, fp):
21066 5ecfe76d0d96 bundle2: make header reading optional Pierre-Yves David <pierre-yves.david@fb.com> parents: 21062 diff changeset	764 """If header is specified, we do not read it out of the stream."""
21013 a813caca89b3 bundle2: extract stream/unpack logic in an unpackermixin Pierre-Yves David <pierre-yves.david@fb.com> parents: 21005 diff changeset	765 self.ui = ui
30353 d045b4091197 bundle2: use compression engines API to obtain decompressor Gregory Szorc <gregory.szorc@gmail.com> parents: 30351 diff changeset	766 self._compengine = util.compengines.forbundletype('UN')
26802 42f705f2c02d bundle2: make unbundle.compressed return True when compressed Pierre-Yves David <pierre-yves.david@fb.com> parents: 26795 diff changeset	767 self._compressed = None
21013 a813caca89b3 bundle2: extract stream/unpack logic in an unpackermixin Pierre-Yves David <pierre-yves.david@fb.com> parents: 21005 diff changeset	768 super(unbundle20, self).__init__(fp)
a813caca89b3 bundle2: extract stream/unpack logic in an unpackermixin Pierre-Yves David <pierre-yves.david@fb.com> parents: 21005 diff changeset	769
20802 520df53ad26a bundle2: a very first version of bundle2 unbundler Pierre-Yves David <pierre-yves.david@fb.com> parents: 20801 diff changeset	770 @util.propertycache
520df53ad26a bundle2: a very first version of bundle2 unbundler Pierre-Yves David <pierre-yves.david@fb.com> parents: 20801 diff changeset	771 def params(self):
21024 7731a2281cf0 spelling: fixes from spell checker Mads Kiilerich <madski@unity3d.com> parents: 21020 diff changeset	772 """dictionary of stream level parameters"""
25320 697d8953b04d bundle2: handle new line in 'indebug' function Pierre-Yves David <pierre-yves.david@fb.com> parents: 25319 diff changeset	773 indebug(self.ui, 'reading bundle2 stream parameters')
20805 c5aaeca0cfbf bundle2: support for unbundling simple parameter Pierre-Yves David <pierre-yves.david@fb.com> parents: 20804 diff changeset	774 params = {}
c5aaeca0cfbf bundle2: support for unbundling simple parameter Pierre-Yves David <pierre-yves.david@fb.com> parents: 20804 diff changeset	775 paramssize = self._unpack(_fstreamparamsize)[0]
23011 006a81d07e57 bundle2: detect and disallow a negative chunk size Pierre-Yves David <pierre-yves.david@fb.com> parents: 23010 diff changeset	776 if paramssize < 0:
006a81d07e57 bundle2: detect and disallow a negative chunk size Pierre-Yves David <pierre-yves.david@fb.com> parents: 23010 diff changeset	777 raise error.BundleValueError('negative bundle param size: %i'
006a81d07e57 bundle2: detect and disallow a negative chunk size Pierre-Yves David <pierre-yves.david@fb.com> parents: 23010 diff changeset	778 % paramssize)
20805 c5aaeca0cfbf bundle2: support for unbundling simple parameter Pierre-Yves David <pierre-yves.david@fb.com> parents: 20804 diff changeset	779 if paramssize:
26541 d40029b4296e bundle2: split parameter retrieval and processing Pierre-Yves David <pierre-yves.david@fb.com> parents: 26404 diff changeset	780 params = self._readexact(paramssize)
d40029b4296e bundle2: split parameter retrieval and processing Pierre-Yves David <pierre-yves.david@fb.com> parents: 26404 diff changeset	781 params = self._processallparams(params)
20805 c5aaeca0cfbf bundle2: support for unbundling simple parameter Pierre-Yves David <pierre-yves.david@fb.com> parents: 20804 diff changeset	782 return params
20802 520df53ad26a bundle2: a very first version of bundle2 unbundler Pierre-Yves David <pierre-yves.david@fb.com> parents: 20801 diff changeset	783
26541 d40029b4296e bundle2: split parameter retrieval and processing Pierre-Yves David <pierre-yves.david@fb.com> parents: 26404 diff changeset	784 def _processallparams(self, paramsblock):
d40029b4296e bundle2: split parameter retrieval and processing Pierre-Yves David <pierre-yves.david@fb.com> parents: 26404 diff changeset	785 """"""
29591 6215b5537ba5 bundle2: use a sorted dict for holding parameters Gregory Szorc <gregory.szorc@gmail.com> parents: 29294 diff changeset	786 params = util.sortdict()
26541 d40029b4296e bundle2: split parameter retrieval and processing Pierre-Yves David <pierre-yves.david@fb.com> parents: 26404 diff changeset	787 for p in paramsblock.split(' '):
d40029b4296e bundle2: split parameter retrieval and processing Pierre-Yves David <pierre-yves.david@fb.com> parents: 26404 diff changeset	788 p = p.split('=', 1)
28883 032c4c2f802a pycompat: switch to util.urlreq/util.urlerr for py3 compat timeless <timeless@mozdev.org> parents: 28672 diff changeset	789 p = [urlreq.unquote(i) for i in p]
26541 d40029b4296e bundle2: split parameter retrieval and processing Pierre-Yves David <pierre-yves.david@fb.com> parents: 26404 diff changeset	790 if len(p) < 2:
d40029b4296e bundle2: split parameter retrieval and processing Pierre-Yves David <pierre-yves.david@fb.com> parents: 26404 diff changeset	791 p.append(None)
d40029b4296e bundle2: split parameter retrieval and processing Pierre-Yves David <pierre-yves.david@fb.com> parents: 26404 diff changeset	792 self._processparam(*p)
d40029b4296e bundle2: split parameter retrieval and processing Pierre-Yves David <pierre-yves.david@fb.com> parents: 26404 diff changeset	793 params[p[0]] = p[1]
d40029b4296e bundle2: split parameter retrieval and processing Pierre-Yves David <pierre-yves.david@fb.com> parents: 26404 diff changeset	794 return params
d40029b4296e bundle2: split parameter retrieval and processing Pierre-Yves David <pierre-yves.david@fb.com> parents: 26404 diff changeset	795
d40029b4296e bundle2: split parameter retrieval and processing Pierre-Yves David <pierre-yves.david@fb.com> parents: 26404 diff changeset	796
20844 2631204d7305 bundle2: implement the mandatory/advisory logic for parameter Pierre-Yves David <pierre-yves.david@fb.com> parents: 20843 diff changeset	797 def _processparam(self, name, value):
2631204d7305 bundle2: implement the mandatory/advisory logic for parameter Pierre-Yves David <pierre-yves.david@fb.com> parents: 20843 diff changeset	798 """process a parameter, applying its effect if needed
2631204d7305 bundle2: implement the mandatory/advisory logic for parameter Pierre-Yves David <pierre-yves.david@fb.com> parents: 20843 diff changeset	799
2631204d7305 bundle2: implement the mandatory/advisory logic for parameter Pierre-Yves David <pierre-yves.david@fb.com> parents: 20843 diff changeset	800 Parameter starting with a lower case letter are advisory and will be
2631204d7305 bundle2: implement the mandatory/advisory logic for parameter Pierre-Yves David <pierre-yves.david@fb.com> parents: 20843 diff changeset	801 ignored when unknown. Those starting with an upper case letter are
2631204d7305 bundle2: implement the mandatory/advisory logic for parameter Pierre-Yves David <pierre-yves.david@fb.com> parents: 20843 diff changeset	802 mandatory and will this function will raise a KeyError when unknown.
2631204d7305 bundle2: implement the mandatory/advisory logic for parameter Pierre-Yves David <pierre-yves.david@fb.com> parents: 20843 diff changeset	803
2631204d7305 bundle2: implement the mandatory/advisory logic for parameter Pierre-Yves David <pierre-yves.david@fb.com> parents: 20843 diff changeset	804 Note: no option are currently supported. Any input will be either
2631204d7305 bundle2: implement the mandatory/advisory logic for parameter Pierre-Yves David <pierre-yves.david@fb.com> parents: 20843 diff changeset	805 ignored or failing.
2631204d7305 bundle2: implement the mandatory/advisory logic for parameter Pierre-Yves David <pierre-yves.david@fb.com> parents: 20843 diff changeset	806 """
2631204d7305 bundle2: implement the mandatory/advisory logic for parameter Pierre-Yves David <pierre-yves.david@fb.com> parents: 20843 diff changeset	807 if not name:
34277 a48b3db0baba bundle2: stop using %r to quote part names Augie Fackler <augie@google.com> parents: 34261 diff changeset	808 raise ValueError(r'empty parameter name')
a48b3db0baba bundle2: stop using %r to quote part names Augie Fackler <augie@google.com> parents: 34261 diff changeset	809 if name[0:1] not in pycompat.bytestr(string.ascii_letters):
a48b3db0baba bundle2: stop using %r to quote part names Augie Fackler <augie@google.com> parents: 34261 diff changeset	810 raise ValueError(r'non letter first character: %s' % name)
26395 4e7b0bf9f0b1 unbundle20: allow registering handlers for stream level parameters Pierre-Yves David <pierre-yves.david@fb.com> parents: 26393 diff changeset	811 try:
4e7b0bf9f0b1 unbundle20: allow registering handlers for stream level parameters Pierre-Yves David <pierre-yves.david@fb.com> parents: 26393 diff changeset	812 handler = b2streamparamsmap[name.lower()]
4e7b0bf9f0b1 unbundle20: allow registering handlers for stream level parameters Pierre-Yves David <pierre-yves.david@fb.com> parents: 26393 diff changeset	813 except KeyError:
34277 a48b3db0baba bundle2: stop using %r to quote part names Augie Fackler <augie@google.com> parents: 34261 diff changeset	814 if name[0:1].islower():
a48b3db0baba bundle2: stop using %r to quote part names Augie Fackler <augie@google.com> parents: 34261 diff changeset	815 indebug(self.ui, "ignoring unknown parameter %s" % name)
26395 4e7b0bf9f0b1 unbundle20: allow registering handlers for stream level parameters Pierre-Yves David <pierre-yves.david@fb.com> parents: 26393 diff changeset	816 else:
4e7b0bf9f0b1 unbundle20: allow registering handlers for stream level parameters Pierre-Yves David <pierre-yves.david@fb.com> parents: 26393 diff changeset	817 raise error.BundleUnknownFeatureError(params=(name,))
20844 2631204d7305 bundle2: implement the mandatory/advisory logic for parameter Pierre-Yves David <pierre-yves.david@fb.com> parents: 20843 diff changeset	818 else:
26395 4e7b0bf9f0b1 unbundle20: allow registering handlers for stream level parameters Pierre-Yves David <pierre-yves.david@fb.com> parents: 26393 diff changeset	819 handler(self, name, value)
20844 2631204d7305 bundle2: implement the mandatory/advisory logic for parameter Pierre-Yves David <pierre-yves.david@fb.com> parents: 20843 diff changeset	820
26542 b87e4638dabf bundle2: add a way to just forward the bundle2 stream to another user Pierre-Yves David <pierre-yves.david@fb.com> parents: 26541 diff changeset	821 def _forwardchunks(self):
b87e4638dabf bundle2: add a way to just forward the bundle2 stream to another user Pierre-Yves David <pierre-yves.david@fb.com> parents: 26541 diff changeset	822 """utility to transfer a bundle2 as binary
b87e4638dabf bundle2: add a way to just forward the bundle2 stream to another user Pierre-Yves David <pierre-yves.david@fb.com> parents: 26541 diff changeset	823
b87e4638dabf bundle2: add a way to just forward the bundle2 stream to another user Pierre-Yves David <pierre-yves.david@fb.com> parents: 26541 diff changeset	824 This is made necessary by the fact the 'getbundle' command over 'ssh'
b87e4638dabf bundle2: add a way to just forward the bundle2 stream to another user Pierre-Yves David <pierre-yves.david@fb.com> parents: 26541 diff changeset	825 have no way to know then the reply end, relying on the bundle to be
b87e4638dabf bundle2: add a way to just forward the bundle2 stream to another user Pierre-Yves David <pierre-yves.david@fb.com> parents: 26541 diff changeset	826 interpreted to know its end. This is terrible and we are sorry, but we
b87e4638dabf bundle2: add a way to just forward the bundle2 stream to another user Pierre-Yves David <pierre-yves.david@fb.com> parents: 26541 diff changeset	827 needed to move forward to get general delta enabled.
b87e4638dabf bundle2: add a way to just forward the bundle2 stream to another user Pierre-Yves David <pierre-yves.david@fb.com> parents: 26541 diff changeset	828 """
b87e4638dabf bundle2: add a way to just forward the bundle2 stream to another user Pierre-Yves David <pierre-yves.david@fb.com> parents: 26541 diff changeset	829 yield self._magicstring
b87e4638dabf bundle2: add a way to just forward the bundle2 stream to another user Pierre-Yves David <pierre-yves.david@fb.com> parents: 26541 diff changeset	830 assert 'params' not in vars(self)
b87e4638dabf bundle2: add a way to just forward the bundle2 stream to another user Pierre-Yves David <pierre-yves.david@fb.com> parents: 26541 diff changeset	831 paramssize = self._unpack(_fstreamparamsize)[0]
b87e4638dabf bundle2: add a way to just forward the bundle2 stream to another user Pierre-Yves David <pierre-yves.david@fb.com> parents: 26541 diff changeset	832 if paramssize < 0:
b87e4638dabf bundle2: add a way to just forward the bundle2 stream to another user Pierre-Yves David <pierre-yves.david@fb.com> parents: 26541 diff changeset	833 raise error.BundleValueError('negative bundle param size: %i'
b87e4638dabf bundle2: add a way to just forward the bundle2 stream to another user Pierre-Yves David <pierre-yves.david@fb.com> parents: 26541 diff changeset	834 % paramssize)
b87e4638dabf bundle2: add a way to just forward the bundle2 stream to another user Pierre-Yves David <pierre-yves.david@fb.com> parents: 26541 diff changeset	835 yield _pack(_fstreamparamsize, paramssize)
b87e4638dabf bundle2: add a way to just forward the bundle2 stream to another user Pierre-Yves David <pierre-yves.david@fb.com> parents: 26541 diff changeset	836 if paramssize:
b87e4638dabf bundle2: add a way to just forward the bundle2 stream to another user Pierre-Yves David <pierre-yves.david@fb.com> parents: 26541 diff changeset	837 params = self._readexact(paramssize)
b87e4638dabf bundle2: add a way to just forward the bundle2 stream to another user Pierre-Yves David <pierre-yves.david@fb.com> parents: 26541 diff changeset	838 self._processallparams(params)
b87e4638dabf bundle2: add a way to just forward the bundle2 stream to another user Pierre-Yves David <pierre-yves.david@fb.com> parents: 26541 diff changeset	839 yield params
30353 d045b4091197 bundle2: use compression engines API to obtain decompressor Gregory Szorc <gregory.szorc@gmail.com> parents: 30351 diff changeset	840 assert self._compengine.bundletype == 'UN'
26542 b87e4638dabf bundle2: add a way to just forward the bundle2 stream to another user Pierre-Yves David <pierre-yves.david@fb.com> parents: 26541 diff changeset	841 # From there, payload might need to be decompressed
30353 d045b4091197 bundle2: use compression engines API to obtain decompressor Gregory Szorc <gregory.szorc@gmail.com> parents: 30351 diff changeset	842 self._fp = self._compengine.decompressorreader(self._fp)
26542 b87e4638dabf bundle2: add a way to just forward the bundle2 stream to another user Pierre-Yves David <pierre-yves.david@fb.com> parents: 26541 diff changeset	843 emptycount = 0
b87e4638dabf bundle2: add a way to just forward the bundle2 stream to another user Pierre-Yves David <pierre-yves.david@fb.com> parents: 26541 diff changeset	844 while emptycount < 2:
b87e4638dabf bundle2: add a way to just forward the bundle2 stream to another user Pierre-Yves David <pierre-yves.david@fb.com> parents: 26541 diff changeset	845 # so we can brainlessly loop
b87e4638dabf bundle2: add a way to just forward the bundle2 stream to another user Pierre-Yves David <pierre-yves.david@fb.com> parents: 26541 diff changeset	846 assert _fpartheadersize == _fpayloadsize
b87e4638dabf bundle2: add a way to just forward the bundle2 stream to another user Pierre-Yves David <pierre-yves.david@fb.com> parents: 26541 diff changeset	847 size = self._unpack(_fpartheadersize)[0]
b87e4638dabf bundle2: add a way to just forward the bundle2 stream to another user Pierre-Yves David <pierre-yves.david@fb.com> parents: 26541 diff changeset	848 yield _pack(_fpartheadersize, size)
b87e4638dabf bundle2: add a way to just forward the bundle2 stream to another user Pierre-Yves David <pierre-yves.david@fb.com> parents: 26541 diff changeset	849 if size:
b87e4638dabf bundle2: add a way to just forward the bundle2 stream to another user Pierre-Yves David <pierre-yves.david@fb.com> parents: 26541 diff changeset	850 emptycount = 0
b87e4638dabf bundle2: add a way to just forward the bundle2 stream to another user Pierre-Yves David <pierre-yves.david@fb.com> parents: 26541 diff changeset	851 else:
b87e4638dabf bundle2: add a way to just forward the bundle2 stream to another user Pierre-Yves David <pierre-yves.david@fb.com> parents: 26541 diff changeset	852 emptycount += 1
b87e4638dabf bundle2: add a way to just forward the bundle2 stream to another user Pierre-Yves David <pierre-yves.david@fb.com> parents: 26541 diff changeset	853 continue
b87e4638dabf bundle2: add a way to just forward the bundle2 stream to another user Pierre-Yves David <pierre-yves.david@fb.com> parents: 26541 diff changeset	854 if size == flaginterrupt:
b87e4638dabf bundle2: add a way to just forward the bundle2 stream to another user Pierre-Yves David <pierre-yves.david@fb.com> parents: 26541 diff changeset	855 continue
b87e4638dabf bundle2: add a way to just forward the bundle2 stream to another user Pierre-Yves David <pierre-yves.david@fb.com> parents: 26541 diff changeset	856 elif size < 0:
b87e4638dabf bundle2: add a way to just forward the bundle2 stream to another user Pierre-Yves David <pierre-yves.david@fb.com> parents: 26541 diff changeset	857 raise error.BundleValueError('negative chunk size: %i')
b87e4638dabf bundle2: add a way to just forward the bundle2 stream to another user Pierre-Yves David <pierre-yves.david@fb.com> parents: 26541 diff changeset	858 yield self._readexact(size)
b87e4638dabf bundle2: add a way to just forward the bundle2 stream to another user Pierre-Yves David <pierre-yves.david@fb.com> parents: 26541 diff changeset	859
b87e4638dabf bundle2: add a way to just forward the bundle2 stream to another user Pierre-Yves David <pierre-yves.david@fb.com> parents: 26541 diff changeset	860
35116 da91e7309daf bundle2: don't use seekable bundle2 parts by default (issue5691) Gregory Szorc <gregory.szorc@gmail.com> parents: 35115 diff changeset	861 def iterparts(self, seekable=False):
20802 520df53ad26a bundle2: a very first version of bundle2 unbundler Pierre-Yves David <pierre-yves.david@fb.com> parents: 20801 diff changeset	862 """yield all parts contained in the stream"""
35116 da91e7309daf bundle2: don't use seekable bundle2 parts by default (issue5691) Gregory Szorc <gregory.szorc@gmail.com> parents: 35115 diff changeset	863 cls = seekableunbundlepart if seekable else unbundlepart
20802 520df53ad26a bundle2: a very first version of bundle2 unbundler Pierre-Yves David <pierre-yves.david@fb.com> parents: 20801 diff changeset	864 # make sure param have been loaded
520df53ad26a bundle2: a very first version of bundle2 unbundler Pierre-Yves David <pierre-yves.david@fb.com> parents: 20801 diff changeset	865 self.params
26404 795f02a24b87 bundle2: allow compressed bundle Pierre-Yves David <pierre-yves.david@fb.com> parents: 26396 diff changeset	866 # From there, payload need to be decompressed
30353 d045b4091197 bundle2: use compression engines API to obtain decompressor Gregory Szorc <gregory.szorc@gmail.com> parents: 30351 diff changeset	867 self._fp = self._compengine.decompressorreader(self._fp)
25320 697d8953b04d bundle2: handle new line in 'indebug' function Pierre-Yves David <pierre-yves.david@fb.com> parents: 25319 diff changeset	868 indebug(self.ui, 'start extraction of bundle2 parts')
21014 a6246bba7b9e bundle2: add an unbundle part responsible from unbundling part Pierre-Yves David <pierre-yves.david@fb.com> parents: 21013 diff changeset	869 headerblock = self._readpartheader()
a6246bba7b9e bundle2: add an unbundle part responsible from unbundling part Pierre-Yves David <pierre-yves.david@fb.com> parents: 21013 diff changeset	870 while headerblock is not None:
35116 da91e7309daf bundle2: don't use seekable bundle2 parts by default (issue5691) Gregory Szorc <gregory.szorc@gmail.com> parents: 35115 diff changeset	871 part = cls(self.ui, headerblock, self._fp)
20802 520df53ad26a bundle2: a very first version of bundle2 unbundler Pierre-Yves David <pierre-yves.david@fb.com> parents: 20801 diff changeset	872 yield part
35114 db5038525718 bundle2: implement consume() API on unbundlepart Gregory Szorc <gregory.szorc@gmail.com> parents: 35113 diff changeset	873 # Ensure part is fully consumed so we can start reading the next
db5038525718 bundle2: implement consume() API on unbundlepart Gregory Szorc <gregory.szorc@gmail.com> parents: 35113 diff changeset	874 # part.
db5038525718 bundle2: implement consume() API on unbundlepart Gregory Szorc <gregory.szorc@gmail.com> parents: 35113 diff changeset	875 part.consume()
35115 2b72bc88043f bundle2: only seek to beginning of part in bundlerepo Gregory Szorc <gregory.szorc@gmail.com> parents: 35114 diff changeset	876
21014 a6246bba7b9e bundle2: add an unbundle part responsible from unbundling part Pierre-Yves David <pierre-yves.david@fb.com> parents: 21013 diff changeset	877 headerblock = self._readpartheader()
25320 697d8953b04d bundle2: handle new line in 'indebug' function Pierre-Yves David <pierre-yves.david@fb.com> parents: 25319 diff changeset	878 indebug(self.ui, 'end of bundle2 stream')
20802 520df53ad26a bundle2: a very first version of bundle2 unbundler Pierre-Yves David <pierre-yves.david@fb.com> parents: 20801 diff changeset	879
21014 a6246bba7b9e bundle2: add an unbundle part responsible from unbundling part Pierre-Yves David <pierre-yves.david@fb.com> parents: 21013 diff changeset	880 def _readpartheader(self):
a6246bba7b9e bundle2: add an unbundle part responsible from unbundling part Pierre-Yves David <pierre-yves.david@fb.com> parents: 21013 diff changeset	881 """reads a part header size and return the bytes blob
20864 9a75d2559cff bundle2: support unbundling empty part Pierre-Yves David <pierre-yves.david@fb.com> parents: 20856 diff changeset	882
21014 a6246bba7b9e bundle2: add an unbundle part responsible from unbundling part Pierre-Yves David <pierre-yves.david@fb.com> parents: 21013 diff changeset	883 returns None if empty"""
20864 9a75d2559cff bundle2: support unbundling empty part Pierre-Yves David <pierre-yves.david@fb.com> parents: 20856 diff changeset	884 headersize = self._unpack(_fpartheadersize)[0]
23011 006a81d07e57 bundle2: detect and disallow a negative chunk size Pierre-Yves David <pierre-yves.david@fb.com> parents: 23010 diff changeset	885 if headersize < 0:
006a81d07e57 bundle2: detect and disallow a negative chunk size Pierre-Yves David <pierre-yves.david@fb.com> parents: 23010 diff changeset	886 raise error.BundleValueError('negative part header size: %i'
006a81d07e57 bundle2: detect and disallow a negative chunk size Pierre-Yves David <pierre-yves.david@fb.com> parents: 23010 diff changeset	887 % headersize)
25320 697d8953b04d bundle2: handle new line in 'indebug' function Pierre-Yves David <pierre-yves.david@fb.com> parents: 25319 diff changeset	888 indebug(self.ui, 'part header size: %i' % headersize)
21014 a6246bba7b9e bundle2: add an unbundle part responsible from unbundling part Pierre-Yves David <pierre-yves.david@fb.com> parents: 21013 diff changeset	889 if headersize:
a6246bba7b9e bundle2: add an unbundle part responsible from unbundling part Pierre-Yves David <pierre-yves.david@fb.com> parents: 21013 diff changeset	890 return self._readexact(headersize)
a6246bba7b9e bundle2: add an unbundle part responsible from unbundling part Pierre-Yves David <pierre-yves.david@fb.com> parents: 21013 diff changeset	891 return None
20864 9a75d2559cff bundle2: support unbundling empty part Pierre-Yves David <pierre-yves.david@fb.com> parents: 20856 diff changeset	892
24071 184a2f6f40da bundle2.unbundle20: add compressed() method Eric Sumner <ericsumner@fb.com> parents: 24070 diff changeset	893 def compressed(self):
26802 42f705f2c02d bundle2: make unbundle.compressed return True when compressed Pierre-Yves David <pierre-yves.david@fb.com> parents: 26795 diff changeset	894 self.params # load params
42f705f2c02d bundle2: make unbundle.compressed return True when compressed Pierre-Yves David <pierre-yves.david@fb.com> parents: 26795 diff changeset	895 return self._compressed
20802 520df53ad26a bundle2: a very first version of bundle2 unbundler Pierre-Yves David <pierre-yves.david@fb.com> parents: 20801 diff changeset	896
31863 cd7aaf344d83 bundle2: move the 'close' method off the unpackermixin Pierre-Yves David <pierre-yves.david@ens-lyon.org> parents: 31862 diff changeset	897 def close(self):
cd7aaf344d83 bundle2: move the 'close' method off the unpackermixin Pierre-Yves David <pierre-yves.david@ens-lyon.org> parents: 31862 diff changeset	898 """close underlying file"""
cd7aaf344d83 bundle2: move the 'close' method off the unpackermixin Pierre-Yves David <pierre-yves.david@ens-lyon.org> parents: 31862 diff changeset	899 if util.safehasattr(self._fp, 'close'):
cd7aaf344d83 bundle2: move the 'close' method off the unpackermixin Pierre-Yves David <pierre-yves.david@ens-lyon.org> parents: 31862 diff changeset	900 return self._fp.close()
cd7aaf344d83 bundle2: move the 'close' method off the unpackermixin Pierre-Yves David <pierre-yves.david@ens-lyon.org> parents: 31862 diff changeset	901
24686 e0e28e910fa3 bundle2: rename format, parts and config to final names Pierre-Yves David <pierre-yves.david@fb.com> parents: 24648 diff changeset	902 formatmap = {'20': unbundle20}
24648 5cac3accdaa1 unbundle20: allow generic dispatch between unbundlers Pierre-Yves David <pierre-yves.david@fb.com> parents: 24642 diff changeset	903
26395 4e7b0bf9f0b1 unbundle20: allow registering handlers for stream level parameters Pierre-Yves David <pierre-yves.david@fb.com> parents: 26393 diff changeset	904 b2streamparamsmap = {}
4e7b0bf9f0b1 unbundle20: allow registering handlers for stream level parameters Pierre-Yves David <pierre-yves.david@fb.com> parents: 26393 diff changeset	905
4e7b0bf9f0b1 unbundle20: allow registering handlers for stream level parameters Pierre-Yves David <pierre-yves.david@fb.com> parents: 26393 diff changeset	906 def b2streamparamhandler(name):
4e7b0bf9f0b1 unbundle20: allow registering handlers for stream level parameters Pierre-Yves David <pierre-yves.david@fb.com> parents: 26393 diff changeset	907 """register a handler for a stream level parameter"""
4e7b0bf9f0b1 unbundle20: allow registering handlers for stream level parameters Pierre-Yves David <pierre-yves.david@fb.com> parents: 26393 diff changeset	908 def decorator(func):
4e7b0bf9f0b1 unbundle20: allow registering handlers for stream level parameters Pierre-Yves David <pierre-yves.david@fb.com> parents: 26393 diff changeset	909 assert name not in formatmap
4e7b0bf9f0b1 unbundle20: allow registering handlers for stream level parameters Pierre-Yves David <pierre-yves.david@fb.com> parents: 26393 diff changeset	910 b2streamparamsmap[name] = func
4e7b0bf9f0b1 unbundle20: allow registering handlers for stream level parameters Pierre-Yves David <pierre-yves.david@fb.com> parents: 26393 diff changeset	911 return func
4e7b0bf9f0b1 unbundle20: allow registering handlers for stream level parameters Pierre-Yves David <pierre-yves.david@fb.com> parents: 26393 diff changeset	912 return decorator
4e7b0bf9f0b1 unbundle20: allow registering handlers for stream level parameters Pierre-Yves David <pierre-yves.david@fb.com> parents: 26393 diff changeset	913
26404 795f02a24b87 bundle2: allow compressed bundle Pierre-Yves David <pierre-yves.david@fb.com> parents: 26396 diff changeset	914 @b2streamparamhandler('compression')
795f02a24b87 bundle2: allow compressed bundle Pierre-Yves David <pierre-yves.david@fb.com> parents: 26396 diff changeset	915 def processcompression(unbundler, param, value):
795f02a24b87 bundle2: allow compressed bundle Pierre-Yves David <pierre-yves.david@fb.com> parents: 26396 diff changeset	916 """read compression parameter and install payload decompression"""
30353 d045b4091197 bundle2: use compression engines API to obtain decompressor Gregory Szorc <gregory.szorc@gmail.com> parents: 30351 diff changeset	917 if value not in util.compengines.supportedbundletypes:
26404 795f02a24b87 bundle2: allow compressed bundle Pierre-Yves David <pierre-yves.david@fb.com> parents: 26396 diff changeset	918 raise error.BundleUnknownFeatureError(params=(param,),
795f02a24b87 bundle2: allow compressed bundle Pierre-Yves David <pierre-yves.david@fb.com> parents: 26396 diff changeset	919 values=(value,))
30353 d045b4091197 bundle2: use compression engines API to obtain decompressor Gregory Szorc <gregory.szorc@gmail.com> parents: 30351 diff changeset	920 unbundler._compengine = util.compengines.forbundletype(value)
26802 42f705f2c02d bundle2: make unbundle.compressed return True when compressed Pierre-Yves David <pierre-yves.david@fb.com> parents: 26795 diff changeset	921 if value is not None:
42f705f2c02d bundle2: make unbundle.compressed return True when compressed Pierre-Yves David <pierre-yves.david@fb.com> parents: 26795 diff changeset	922 unbundler._compressed = True
26404 795f02a24b87 bundle2: allow compressed bundle Pierre-Yves David <pierre-yves.david@fb.com> parents: 26396 diff changeset	923
21005 3d38ebb586fe bundle2: rename part to bundlepart Pierre-Yves David <pierre-yves.david@fb.com> parents: 21004 diff changeset	924 class bundlepart(object):
20856 8a6a86c9a5b5 bundle2: support bundling of empty part (with a type) Pierre-Yves David <pierre-yves.david@fb.com> parents: 20844 diff changeset	925 """A bundle2 part contains application level payload
8a6a86c9a5b5 bundle2: support bundling of empty part (with a type) Pierre-Yves David <pierre-yves.david@fb.com> parents: 20844 diff changeset	926
8a6a86c9a5b5 bundle2: support bundling of empty part (with a type) Pierre-Yves David <pierre-yves.david@fb.com> parents: 20844 diff changeset	927 The part `type` is used to route the part to the application level
8a6a86c9a5b5 bundle2: support bundling of empty part (with a type) Pierre-Yves David <pierre-yves.david@fb.com> parents: 20844 diff changeset	928 handler.
21604 c399bf961cb9 bundle2: the ability to set ``data`` attribute of the part is now official Pierre-Yves David <pierre-yves.david@fb.com> parents: 21603 diff changeset	929
c399bf961cb9 bundle2: the ability to set ``data`` attribute of the part is now official Pierre-Yves David <pierre-yves.david@fb.com> parents: 21603 diff changeset	930 The part payload is contained in ``part.data``. It could be raw bytes or a
21605 f9dabfceb259 bundle2: introduce a ``addparam`` method on part Pierre-Yves David <pierre-yves.david@fb.com> parents: 21604 diff changeset	931 generator of byte chunks.
f9dabfceb259 bundle2: introduce a ``addparam`` method on part Pierre-Yves David <pierre-yves.david@fb.com> parents: 21604 diff changeset	932
f9dabfceb259 bundle2: introduce a ``addparam`` method on part Pierre-Yves David <pierre-yves.david@fb.com> parents: 21604 diff changeset	933 You can add parameters to the part using the ``addparam`` method.
f9dabfceb259 bundle2: introduce a ``addparam`` method on part Pierre-Yves David <pierre-yves.david@fb.com> parents: 21604 diff changeset	934 Parameters can be either mandatory (default) or advisory. Remote side
f9dabfceb259 bundle2: introduce a ``addparam`` method on part Pierre-Yves David <pierre-yves.david@fb.com> parents: 21604 diff changeset	935 should be able to safely ignore the advisory ones.
f9dabfceb259 bundle2: introduce a ``addparam`` method on part Pierre-Yves David <pierre-yves.david@fb.com> parents: 21604 diff changeset	936
f9dabfceb259 bundle2: introduce a ``addparam`` method on part Pierre-Yves David <pierre-yves.david@fb.com> parents: 21604 diff changeset	937 Both data and parameters cannot be modified after the generation has begun.
20856 8a6a86c9a5b5 bundle2: support bundling of empty part (with a type) Pierre-Yves David <pierre-yves.david@fb.com> parents: 20844 diff changeset	938 """
8a6a86c9a5b5 bundle2: support bundling of empty part (with a type) Pierre-Yves David <pierre-yves.david@fb.com> parents: 20844 diff changeset	939
20877 9e9e3a4e9261 bundle2: part params Pierre-Yves David <pierre-yves.david@fb.com> parents: 20876 diff changeset	940 def __init__(self, parttype, mandatoryparams=(), advisoryparams=(),
23590 4440c7cc3728 bundle2.bundlepart: make mandatory part flag explicit in API Eric Sumner <ericsumner@fb.com> parents: 23586 diff changeset	941 data='', mandatory=True):
23868 405eacbecc52 bundle2: enforce parttype as alphanumerical Pierre-Yves David <pierre-yves.david@fb.com> parents: 23590 diff changeset	942 validateparttype(parttype)
20995 e995d104c87f bundle2: add an integer id to part Pierre-Yves David <pierre-yves.david@fb.com> parents: 20952 diff changeset	943 self.id = None
20856 8a6a86c9a5b5 bundle2: support bundling of empty part (with a type) Pierre-Yves David <pierre-yves.david@fb.com> parents: 20844 diff changeset	944 self.type = parttype
21604 c399bf961cb9 bundle2: the ability to set ``data`` attribute of the part is now official Pierre-Yves David <pierre-yves.david@fb.com> parents: 21603 diff changeset	945 self._data = data
21605 f9dabfceb259 bundle2: introduce a ``addparam`` method on part Pierre-Yves David <pierre-yves.david@fb.com> parents: 21604 diff changeset	946 self._mandatoryparams = list(mandatoryparams)
f9dabfceb259 bundle2: introduce a ``addparam`` method on part Pierre-Yves David <pierre-yves.david@fb.com> parents: 21604 diff changeset	947 self._advisoryparams = list(advisoryparams)
21607 054fa5176fa7 bundle2: forbid duplicate parameter keys Pierre-Yves David <pierre-yves.david@fb.com> parents: 21606 diff changeset	948 # checking for duplicated entries
054fa5176fa7 bundle2: forbid duplicate parameter keys Pierre-Yves David <pierre-yves.david@fb.com> parents: 21606 diff changeset	949 self._seenparams = set()
054fa5176fa7 bundle2: forbid duplicate parameter keys Pierre-Yves David <pierre-yves.david@fb.com> parents: 21606 diff changeset	950 for pname, __ in self._mandatoryparams + self._advisoryparams:
054fa5176fa7 bundle2: forbid duplicate parameter keys Pierre-Yves David <pierre-yves.david@fb.com> parents: 21606 diff changeset	951 if pname in self._seenparams:
31647 4dbef666c6c9 bundle2: use ProgrammingError Jun Wu <quark@fb.com> parents: 31476 diff changeset	952 raise error.ProgrammingError('duplicated params: %s' % pname)
21607 054fa5176fa7 bundle2: forbid duplicate parameter keys Pierre-Yves David <pierre-yves.david@fb.com> parents: 21606 diff changeset	953 self._seenparams.add(pname)
21601 7ff01befc7ec bundle2: track life cycle of parts Pierre-Yves David <pierre-yves.david@fb.com> parents: 21600 diff changeset	954 # status of the part's generation:
7ff01befc7ec bundle2: track life cycle of parts Pierre-Yves David <pierre-yves.david@fb.com> parents: 21600 diff changeset	955 # - None: not started,
7ff01befc7ec bundle2: track life cycle of parts Pierre-Yves David <pierre-yves.david@fb.com> parents: 21600 diff changeset	956 # - False: currently generated,
7ff01befc7ec bundle2: track life cycle of parts Pierre-Yves David <pierre-yves.david@fb.com> parents: 21600 diff changeset	957 # - True: generation done.
7ff01befc7ec bundle2: track life cycle of parts Pierre-Yves David <pierre-yves.david@fb.com> parents: 21600 diff changeset	958 self._generated = None
23590 4440c7cc3728 bundle2.bundlepart: make mandatory part flag explicit in API Eric Sumner <ericsumner@fb.com> parents: 23586 diff changeset	959 self.mandatory = mandatory
20856 8a6a86c9a5b5 bundle2: support bundling of empty part (with a type) Pierre-Yves David <pierre-yves.david@fb.com> parents: 20844 diff changeset	960
30872 1f51b4658f21 bundle2: implement a basic __repr__ for bundle2 part Pierre-Yves David <pierre-yves.david@ens-lyon.org> parents: 30871 diff changeset	961 def __repr__(self):
1f51b4658f21 bundle2: implement a basic __repr__ for bundle2 part Pierre-Yves David <pierre-yves.david@ens-lyon.org> parents: 30871 diff changeset	962 cls = "%s.%s" % (self.__class__.__module__, self.__class__.__name__)
1f51b4658f21 bundle2: implement a basic __repr__ for bundle2 part Pierre-Yves David <pierre-yves.david@ens-lyon.org> parents: 30871 diff changeset	963 return ('<%s object at %x; id: %s; type: %s; mandatory: %s>'
1f51b4658f21 bundle2: implement a basic __repr__ for bundle2 part Pierre-Yves David <pierre-yves.david@ens-lyon.org> parents: 30871 diff changeset	964 % (cls, id(self), self.id, self.type, self.mandatory))
1f51b4658f21 bundle2: implement a basic __repr__ for bundle2 part Pierre-Yves David <pierre-yves.david@ens-lyon.org> parents: 30871 diff changeset	965
24793 2ec894582ea2 bundle2: add a 'copy' method on parts Pierre-Yves David <pierre-yves.david@fb.com> parents: 24743 diff changeset	966 def copy(self):
2ec894582ea2 bundle2: add a 'copy' method on parts Pierre-Yves David <pierre-yves.david@fb.com> parents: 24743 diff changeset	967 """return a copy of the part
2ec894582ea2 bundle2: add a 'copy' method on parts Pierre-Yves David <pierre-yves.david@fb.com> parents: 24743 diff changeset	968
2ec894582ea2 bundle2: add a 'copy' method on parts Pierre-Yves David <pierre-yves.david@fb.com> parents: 24743 diff changeset	969 The new part have the very same content but no partid assigned yet.
2ec894582ea2 bundle2: add a 'copy' method on parts Pierre-Yves David <pierre-yves.david@fb.com> parents: 24743 diff changeset	970 Parts with generated data cannot be copied."""
2ec894582ea2 bundle2: add a 'copy' method on parts Pierre-Yves David <pierre-yves.david@fb.com> parents: 24743 diff changeset	971 assert not util.safehasattr(self.data, 'next')
2ec894582ea2 bundle2: add a 'copy' method on parts Pierre-Yves David <pierre-yves.david@fb.com> parents: 24743 diff changeset	972 return self.__class__(self.type, self._mandatoryparams,
2ec894582ea2 bundle2: add a 'copy' method on parts Pierre-Yves David <pierre-yves.david@fb.com> parents: 24743 diff changeset	973 self._advisoryparams, self._data, self.mandatory)
2ec894582ea2 bundle2: add a 'copy' method on parts Pierre-Yves David <pierre-yves.david@fb.com> parents: 24743 diff changeset	974
21604 c399bf961cb9 bundle2: the ability to set ``data`` attribute of the part is now official Pierre-Yves David <pierre-yves.david@fb.com> parents: 21603 diff changeset	975 # methods used to defines the part content
27879 52a4ad62b006 cleanup: use modern @property/@foo.setter property specification Augie Fackler <augie@google.com> parents: 27751 diff changeset	976 @property
52a4ad62b006 cleanup: use modern @property/@foo.setter property specification Augie Fackler <augie@google.com> parents: 27751 diff changeset	977 def data(self):
52a4ad62b006 cleanup: use modern @property/@foo.setter property specification Augie Fackler <augie@google.com> parents: 27751 diff changeset	978 return self._data
52a4ad62b006 cleanup: use modern @property/@foo.setter property specification Augie Fackler <augie@google.com> parents: 27751 diff changeset	979
52a4ad62b006 cleanup: use modern @property/@foo.setter property specification Augie Fackler <augie@google.com> parents: 27751 diff changeset	980 @data.setter
52a4ad62b006 cleanup: use modern @property/@foo.setter property specification Augie Fackler <augie@google.com> parents: 27751 diff changeset	981 def data(self, data):
21604 c399bf961cb9 bundle2: the ability to set ``data`` attribute of the part is now official Pierre-Yves David <pierre-yves.david@fb.com> parents: 21603 diff changeset	982 if self._generated is not None:
21618 7568f5c1c801 bundle2: move exception classes into the error module Pierre-Yves David <pierre-yves.david@fb.com> parents: 21617 diff changeset	983 raise error.ReadOnlyPartError('part is being generated')
21604 c399bf961cb9 bundle2: the ability to set ``data`` attribute of the part is now official Pierre-Yves David <pierre-yves.david@fb.com> parents: 21603 diff changeset	984 self._data = data
c399bf961cb9 bundle2: the ability to set ``data`` attribute of the part is now official Pierre-Yves David <pierre-yves.david@fb.com> parents: 21603 diff changeset	985
21605 f9dabfceb259 bundle2: introduce a ``addparam`` method on part Pierre-Yves David <pierre-yves.david@fb.com> parents: 21604 diff changeset	986 @property
f9dabfceb259 bundle2: introduce a ``addparam`` method on part Pierre-Yves David <pierre-yves.david@fb.com> parents: 21604 diff changeset	987 def mandatoryparams(self):
f9dabfceb259 bundle2: introduce a ``addparam`` method on part Pierre-Yves David <pierre-yves.david@fb.com> parents: 21604 diff changeset	988 # make it an immutable tuple to force people through ``addparam``
f9dabfceb259 bundle2: introduce a ``addparam`` method on part Pierre-Yves David <pierre-yves.david@fb.com> parents: 21604 diff changeset	989 return tuple(self._mandatoryparams)
f9dabfceb259 bundle2: introduce a ``addparam`` method on part Pierre-Yves David <pierre-yves.david@fb.com> parents: 21604 diff changeset	990
f9dabfceb259 bundle2: introduce a ``addparam`` method on part Pierre-Yves David <pierre-yves.david@fb.com> parents: 21604 diff changeset	991 @property
f9dabfceb259 bundle2: introduce a ``addparam`` method on part Pierre-Yves David <pierre-yves.david@fb.com> parents: 21604 diff changeset	992 def advisoryparams(self):
f9dabfceb259 bundle2: introduce a ``addparam`` method on part Pierre-Yves David <pierre-yves.david@fb.com> parents: 21604 diff changeset	993 # make it an immutable tuple to force people through ``addparam``
f9dabfceb259 bundle2: introduce a ``addparam`` method on part Pierre-Yves David <pierre-yves.david@fb.com> parents: 21604 diff changeset	994 return tuple(self._advisoryparams)
f9dabfceb259 bundle2: introduce a ``addparam`` method on part Pierre-Yves David <pierre-yves.david@fb.com> parents: 21604 diff changeset	995
f9dabfceb259 bundle2: introduce a ``addparam`` method on part Pierre-Yves David <pierre-yves.david@fb.com> parents: 21604 diff changeset	996 def addparam(self, name, value='', mandatory=True):
31861 6d055cd6669f bundle2: add documention to 'part.addparams' Pierre-Yves David <pierre-yves.david@ens-lyon.org> parents: 31647 diff changeset	997 """add a parameter to the part
6d055cd6669f bundle2: add documention to 'part.addparams' Pierre-Yves David <pierre-yves.david@ens-lyon.org> parents: 31647 diff changeset	998
6d055cd6669f bundle2: add documention to 'part.addparams' Pierre-Yves David <pierre-yves.david@ens-lyon.org> parents: 31647 diff changeset	999 If 'mandatory' is set to True, the remote handler must claim support
6d055cd6669f bundle2: add documention to 'part.addparams' Pierre-Yves David <pierre-yves.david@ens-lyon.org> parents: 31647 diff changeset	1000 for this parameter or the unbundling will be aborted.
6d055cd6669f bundle2: add documention to 'part.addparams' Pierre-Yves David <pierre-yves.david@ens-lyon.org> parents: 31647 diff changeset	1001
6d055cd6669f bundle2: add documention to 'part.addparams' Pierre-Yves David <pierre-yves.david@ens-lyon.org> parents: 31647 diff changeset	1002 The 'name' and 'value' cannot exceed 255 bytes each.
6d055cd6669f bundle2: add documention to 'part.addparams' Pierre-Yves David <pierre-yves.david@ens-lyon.org> parents: 31647 diff changeset	1003 """
21605 f9dabfceb259 bundle2: introduce a ``addparam`` method on part Pierre-Yves David <pierre-yves.david@fb.com> parents: 21604 diff changeset	1004 if self._generated is not None:
21618 7568f5c1c801 bundle2: move exception classes into the error module Pierre-Yves David <pierre-yves.david@fb.com> parents: 21617 diff changeset	1005 raise error.ReadOnlyPartError('part is being generated')
21607 054fa5176fa7 bundle2: forbid duplicate parameter keys Pierre-Yves David <pierre-yves.david@fb.com> parents: 21606 diff changeset	1006 if name in self._seenparams:
054fa5176fa7 bundle2: forbid duplicate parameter keys Pierre-Yves David <pierre-yves.david@fb.com> parents: 21606 diff changeset	1007 raise ValueError('duplicated params: %s' % name)
054fa5176fa7 bundle2: forbid duplicate parameter keys Pierre-Yves David <pierre-yves.david@fb.com> parents: 21606 diff changeset	1008 self._seenparams.add(name)
21605 f9dabfceb259 bundle2: introduce a ``addparam`` method on part Pierre-Yves David <pierre-yves.david@fb.com> parents: 21604 diff changeset	1009 params = self._advisoryparams
f9dabfceb259 bundle2: introduce a ``addparam`` method on part Pierre-Yves David <pierre-yves.david@fb.com> parents: 21604 diff changeset	1010 if mandatory:
f9dabfceb259 bundle2: introduce a ``addparam`` method on part Pierre-Yves David <pierre-yves.david@fb.com> parents: 21604 diff changeset	1011 params = self._mandatoryparams
f9dabfceb259 bundle2: introduce a ``addparam`` method on part Pierre-Yves David <pierre-yves.david@fb.com> parents: 21604 diff changeset	1012 params.append((name, value))
f9dabfceb259 bundle2: introduce a ``addparam`` method on part Pierre-Yves David <pierre-yves.david@fb.com> parents: 21604 diff changeset	1013
21601 7ff01befc7ec bundle2: track life cycle of parts Pierre-Yves David <pierre-yves.david@fb.com> parents: 21600 diff changeset	1014 # methods used to generates the bundle2 stream
25321 b44ee346211f bundle2: add debug output for part generation Pierre-Yves David <pierre-yves.david@fb.com> parents: 25320 diff changeset	1015 def getchunks(self, ui):
21601 7ff01befc7ec bundle2: track life cycle of parts Pierre-Yves David <pierre-yves.david@fb.com> parents: 21600 diff changeset	1016 if self._generated is not None:
31647 4dbef666c6c9 bundle2: use ProgrammingError Jun Wu <quark@fb.com> parents: 31476 diff changeset	1017 raise error.ProgrammingError('part can only be consumed once')
21601 7ff01befc7ec bundle2: track life cycle of parts Pierre-Yves David <pierre-yves.david@fb.com> parents: 21600 diff changeset	1018 self._generated = False
25323 21a25fb81d2c bundle2: add generic debug output regarding generated parts Pierre-Yves David <pierre-yves.david@fb.com> parents: 25322 diff changeset	1019
21a25fb81d2c bundle2: add generic debug output regarding generated parts Pierre-Yves David <pierre-yves.david@fb.com> parents: 25322 diff changeset	1020 if ui.debugflag:
21a25fb81d2c bundle2: add generic debug output regarding generated parts Pierre-Yves David <pierre-yves.david@fb.com> parents: 25322 diff changeset	1021 msg = ['bundle2-output-part: "%s"' % self.type]
21a25fb81d2c bundle2: add generic debug output regarding generated parts Pierre-Yves David <pierre-yves.david@fb.com> parents: 25322 diff changeset	1022 if not self.mandatory:
21a25fb81d2c bundle2: add generic debug output regarding generated parts Pierre-Yves David <pierre-yves.david@fb.com> parents: 25322 diff changeset	1023 msg.append(' (advisory)')
21a25fb81d2c bundle2: add generic debug output regarding generated parts Pierre-Yves David <pierre-yves.david@fb.com> parents: 25322 diff changeset	1024 nbmp = len(self.mandatoryparams)
21a25fb81d2c bundle2: add generic debug output regarding generated parts Pierre-Yves David <pierre-yves.david@fb.com> parents: 25322 diff changeset	1025 nbap = len(self.advisoryparams)
21a25fb81d2c bundle2: add generic debug output regarding generated parts Pierre-Yves David <pierre-yves.david@fb.com> parents: 25322 diff changeset	1026 if nbmp or nbap:
21a25fb81d2c bundle2: add generic debug output regarding generated parts Pierre-Yves David <pierre-yves.david@fb.com> parents: 25322 diff changeset	1027 msg.append(' (params:')
21a25fb81d2c bundle2: add generic debug output regarding generated parts Pierre-Yves David <pierre-yves.david@fb.com> parents: 25322 diff changeset	1028 if nbmp:
21a25fb81d2c bundle2: add generic debug output regarding generated parts Pierre-Yves David <pierre-yves.david@fb.com> parents: 25322 diff changeset	1029 msg.append(' %i mandatory' % nbmp)
21a25fb81d2c bundle2: add generic debug output regarding generated parts Pierre-Yves David <pierre-yves.david@fb.com> parents: 25322 diff changeset	1030 if nbap:
21a25fb81d2c bundle2: add generic debug output regarding generated parts Pierre-Yves David <pierre-yves.david@fb.com> parents: 25322 diff changeset	1031 msg.append(' %i advisory' % nbmp)
21a25fb81d2c bundle2: add generic debug output regarding generated parts Pierre-Yves David <pierre-yves.david@fb.com> parents: 25322 diff changeset	1032 msg.append(')')
21a25fb81d2c bundle2: add generic debug output regarding generated parts Pierre-Yves David <pierre-yves.david@fb.com> parents: 25322 diff changeset	1033 if not self.data:
21a25fb81d2c bundle2: add generic debug output regarding generated parts Pierre-Yves David <pierre-yves.david@fb.com> parents: 25322 diff changeset	1034 msg.append(' empty payload')
34278 ea7588c11c36 bundle2: update check for a generator to work on Python 3 Augie Fackler <augie@google.com> parents: 34277 diff changeset	1035 elif (util.safehasattr(self.data, 'next')
ea7588c11c36 bundle2: update check for a generator to work on Python 3 Augie Fackler <augie@google.com> parents: 34277 diff changeset	1036 or util.safehasattr(self.data, '__next__')):
25323 21a25fb81d2c bundle2: add generic debug output regarding generated parts Pierre-Yves David <pierre-yves.david@fb.com> parents: 25322 diff changeset	1037 msg.append(' streamed payload')
21a25fb81d2c bundle2: add generic debug output regarding generated parts Pierre-Yves David <pierre-yves.david@fb.com> parents: 25322 diff changeset	1038 else:
21a25fb81d2c bundle2: add generic debug output regarding generated parts Pierre-Yves David <pierre-yves.david@fb.com> parents: 25322 diff changeset	1039 msg.append(' %i bytes payload' % len(self.data))
21a25fb81d2c bundle2: add generic debug output regarding generated parts Pierre-Yves David <pierre-yves.david@fb.com> parents: 25322 diff changeset	1040 msg.append('\n')
21a25fb81d2c bundle2: add generic debug output regarding generated parts Pierre-Yves David <pierre-yves.david@fb.com> parents: 25322 diff changeset	1041 ui.debug(''.join(msg))
21a25fb81d2c bundle2: add generic debug output regarding generated parts Pierre-Yves David <pierre-yves.david@fb.com> parents: 25322 diff changeset	1042
20877 9e9e3a4e9261 bundle2: part params Pierre-Yves David <pierre-yves.david@fb.com> parents: 20876 diff changeset	1043 #### header
23590 4440c7cc3728 bundle2.bundlepart: make mandatory part flag explicit in API Eric Sumner <ericsumner@fb.com> parents: 23586 diff changeset	1044 if self.mandatory:
4440c7cc3728 bundle2.bundlepart: make mandatory part flag explicit in API Eric Sumner <ericsumner@fb.com> parents: 23586 diff changeset	1045 parttype = self.type.upper()
4440c7cc3728 bundle2.bundlepart: make mandatory part flag explicit in API Eric Sumner <ericsumner@fb.com> parents: 23586 diff changeset	1046 else:
4440c7cc3728 bundle2.bundlepart: make mandatory part flag explicit in API Eric Sumner <ericsumner@fb.com> parents: 23586 diff changeset	1047 parttype = self.type.lower()
33711 373ca5103ba8 bundle2: convert ints to strings using pycompat.bytestring() Augie Fackler <augie@google.com> parents: 33693 diff changeset	1048 outdebug(ui, 'part %s: "%s"' % (pycompat.bytestr(self.id), parttype))
20877 9e9e3a4e9261 bundle2: part params Pierre-Yves David <pierre-yves.david@fb.com> parents: 20876 diff changeset	1049 ## parttype
23590 4440c7cc3728 bundle2.bundlepart: make mandatory part flag explicit in API Eric Sumner <ericsumner@fb.com> parents: 23586 diff changeset	1050 header = [_pack(_fparttypesize, len(parttype)),
4440c7cc3728 bundle2.bundlepart: make mandatory part flag explicit in API Eric Sumner <ericsumner@fb.com> parents: 23586 diff changeset	1051 parttype, _pack(_fpartid, self.id),
20856 8a6a86c9a5b5 bundle2: support bundling of empty part (with a type) Pierre-Yves David <pierre-yves.david@fb.com> parents: 20844 diff changeset	1052 ]
20877 9e9e3a4e9261 bundle2: part params Pierre-Yves David <pierre-yves.david@fb.com> parents: 20876 diff changeset	1053 ## parameters
9e9e3a4e9261 bundle2: part params Pierre-Yves David <pierre-yves.david@fb.com> parents: 20876 diff changeset	1054 # count
9e9e3a4e9261 bundle2: part params Pierre-Yves David <pierre-yves.david@fb.com> parents: 20876 diff changeset	1055 manpar = self.mandatoryparams
9e9e3a4e9261 bundle2: part params Pierre-Yves David <pierre-yves.david@fb.com> parents: 20876 diff changeset	1056 advpar = self.advisoryparams
9e9e3a4e9261 bundle2: part params Pierre-Yves David <pierre-yves.david@fb.com> parents: 20876 diff changeset	1057 header.append(_pack(_fpartparamcount, len(manpar), len(advpar)))
9e9e3a4e9261 bundle2: part params Pierre-Yves David <pierre-yves.david@fb.com> parents: 20876 diff changeset	1058 # size
9e9e3a4e9261 bundle2: part params Pierre-Yves David <pierre-yves.david@fb.com> parents: 20876 diff changeset	1059 parsizes = []
9e9e3a4e9261 bundle2: part params Pierre-Yves David <pierre-yves.david@fb.com> parents: 20876 diff changeset	1060 for key, value in manpar:
9e9e3a4e9261 bundle2: part params Pierre-Yves David <pierre-yves.david@fb.com> parents: 20876 diff changeset	1061 parsizes.append(len(key))
9e9e3a4e9261 bundle2: part params Pierre-Yves David <pierre-yves.david@fb.com> parents: 20876 diff changeset	1062 parsizes.append(len(value))
9e9e3a4e9261 bundle2: part params Pierre-Yves David <pierre-yves.david@fb.com> parents: 20876 diff changeset	1063 for key, value in advpar:
9e9e3a4e9261 bundle2: part params Pierre-Yves David <pierre-yves.david@fb.com> parents: 20876 diff changeset	1064 parsizes.append(len(key))
9e9e3a4e9261 bundle2: part params Pierre-Yves David <pierre-yves.david@fb.com> parents: 20876 diff changeset	1065 parsizes.append(len(value))
33672 da7c285ec6da bundle2: use modern Python division Augie Fackler <augie@google.com> parents: 33665 diff changeset	1066 paramsizes = _pack(_makefpartparamsizes(len(parsizes) // 2), *parsizes)
20877 9e9e3a4e9261 bundle2: part params Pierre-Yves David <pierre-yves.david@fb.com> parents: 20876 diff changeset	1067 header.append(paramsizes)
9e9e3a4e9261 bundle2: part params Pierre-Yves David <pierre-yves.david@fb.com> parents: 20876 diff changeset	1068 # key, value
9e9e3a4e9261 bundle2: part params Pierre-Yves David <pierre-yves.david@fb.com> parents: 20876 diff changeset	1069 for key, value in manpar:
9e9e3a4e9261 bundle2: part params Pierre-Yves David <pierre-yves.david@fb.com> parents: 20876 diff changeset	1070 header.append(key)
9e9e3a4e9261 bundle2: part params Pierre-Yves David <pierre-yves.david@fb.com> parents: 20876 diff changeset	1071 header.append(value)
9e9e3a4e9261 bundle2: part params Pierre-Yves David <pierre-yves.david@fb.com> parents: 20876 diff changeset	1072 for key, value in advpar:
9e9e3a4e9261 bundle2: part params Pierre-Yves David <pierre-yves.david@fb.com> parents: 20876 diff changeset	1073 header.append(key)
9e9e3a4e9261 bundle2: part params Pierre-Yves David <pierre-yves.david@fb.com> parents: 20876 diff changeset	1074 header.append(value)
9e9e3a4e9261 bundle2: part params Pierre-Yves David <pierre-yves.david@fb.com> parents: 20876 diff changeset	1075 ## finalize header
34249 ab379eed2e31 bundle2: raise a more helpful error if building a bundle part header fails Augie Fackler <raf@durin42.com> parents: 34221 diff changeset	1076 try:
ab379eed2e31 bundle2: raise a more helpful error if building a bundle part header fails Augie Fackler <raf@durin42.com> parents: 34221 diff changeset	1077 headerchunk = ''.join(header)
ab379eed2e31 bundle2: raise a more helpful error if building a bundle part header fails Augie Fackler <raf@durin42.com> parents: 34221 diff changeset	1078 except TypeError:
ab379eed2e31 bundle2: raise a more helpful error if building a bundle part header fails Augie Fackler <raf@durin42.com> parents: 34221 diff changeset	1079 raise TypeError(r'Found a non-bytes trying to '
ab379eed2e31 bundle2: raise a more helpful error if building a bundle part header fails Augie Fackler <raf@durin42.com> parents: 34221 diff changeset	1080 r'build bundle part header: %r' % header)
25321 b44ee346211f bundle2: add debug output for part generation Pierre-Yves David <pierre-yves.david@fb.com> parents: 25320 diff changeset	1081 outdebug(ui, 'header chunk size: %i' % len(headerchunk))
20856 8a6a86c9a5b5 bundle2: support bundling of empty part (with a type) Pierre-Yves David <pierre-yves.david@fb.com> parents: 20844 diff changeset	1082 yield _pack(_fpartheadersize, len(headerchunk))
8a6a86c9a5b5 bundle2: support bundling of empty part (with a type) Pierre-Yves David <pierre-yves.david@fb.com> parents: 20844 diff changeset	1083 yield headerchunk
20877 9e9e3a4e9261 bundle2: part params Pierre-Yves David <pierre-yves.david@fb.com> parents: 20876 diff changeset	1084 ## payload
23067 420a051616ce bundle2: transmit exception during part generation Pierre-Yves David <pierre-yves.david@fb.com> parents: 23066 diff changeset	1085 try:
420a051616ce bundle2: transmit exception during part generation Pierre-Yves David <pierre-yves.david@fb.com> parents: 23066 diff changeset	1086 for chunk in self._payloadchunks():
25321 b44ee346211f bundle2: add debug output for part generation Pierre-Yves David <pierre-yves.david@fb.com> parents: 25320 diff changeset	1087 outdebug(ui, 'payload chunk size: %i' % len(chunk))
23067 420a051616ce bundle2: transmit exception during part generation Pierre-Yves David <pierre-yves.david@fb.com> parents: 23066 diff changeset	1088 yield _pack(_fpayloadsize, len(chunk))
420a051616ce bundle2: transmit exception during part generation Pierre-Yves David <pierre-yves.david@fb.com> parents: 23066 diff changeset	1089 yield chunk
26144 4bc3707f3e67 bundle2: don't try to recover from a GeneratorExit (issue4785) Augie Fackler <augie@google.com> parents: 25919 diff changeset	1090 except GeneratorExit:
4bc3707f3e67 bundle2: don't try to recover from a GeneratorExit (issue4785) Augie Fackler <augie@google.com> parents: 25919 diff changeset	1091 # GeneratorExit means that nobody is listening for our
4bc3707f3e67 bundle2: don't try to recover from a GeneratorExit (issue4785) Augie Fackler <augie@google.com> parents: 25919 diff changeset	1092 # results anyway, so just bail quickly rather than trying
4bc3707f3e67 bundle2: don't try to recover from a GeneratorExit (issue4785) Augie Fackler <augie@google.com> parents: 25919 diff changeset	1093 # to produce an error part.
4bc3707f3e67 bundle2: don't try to recover from a GeneratorExit (issue4785) Augie Fackler <augie@google.com> parents: 25919 diff changeset	1094 ui.debug('bundle2-generatorexit\n')
4bc3707f3e67 bundle2: don't try to recover from a GeneratorExit (issue4785) Augie Fackler <augie@google.com> parents: 25919 diff changeset	1095 raise
25660 328739ea70c3 global: mass rewrite to use modern exception syntax Gregory Szorc <gregory.szorc@gmail.com> parents: 25641 diff changeset	1096 except BaseException as exc:
37084 f0b6fbea00cf stringutil: bulk-replace call sites to point to new module Yuya Nishihara <yuya@tcha.org> parents: 37005 diff changeset	1097 bexc = stringutil.forcebytestr(exc)
23067 420a051616ce bundle2: transmit exception during part generation Pierre-Yves David <pierre-yves.david@fb.com> parents: 23066 diff changeset	1098 # backup exception data for later
25324 deed9c6b12d2 bundle2: add generic debug output regarding generated interruption Pierre-Yves David <pierre-yves.david@fb.com> parents: 25323 diff changeset	1099 ui.debug('bundle2-input-stream-interrupt: encoding exception %s'
33676 9f862fe7a8bd bundle2: obtain repr() of exception in a python3-safe way Augie Fackler <augie@google.com> parents: 33675 diff changeset	1100 % bexc)
32186 76f9a0009b4b pycompat: extract helper to raise exception with traceback Yuya Nishihara <yuya@tcha.org> parents: 32024 diff changeset	1101 tb = sys.exc_info()[2]
33676 9f862fe7a8bd bundle2: obtain repr() of exception in a python3-safe way Augie Fackler <augie@google.com> parents: 33675 diff changeset	1102 msg = 'unexpected error: %s' % bexc
24686 e0e28e910fa3 bundle2: rename format, parts and config to final names Pierre-Yves David <pierre-yves.david@fb.com> parents: 24648 diff changeset	1103 interpart = bundlepart('error:abort', [('message', msg)],
23590 4440c7cc3728 bundle2.bundlepart: make mandatory part flag explicit in API Eric Sumner <ericsumner@fb.com> parents: 23586 diff changeset	1104 mandatory=False)
23067 420a051616ce bundle2: transmit exception during part generation Pierre-Yves David <pierre-yves.david@fb.com> parents: 23066 diff changeset	1105 interpart.id = 0
420a051616ce bundle2: transmit exception during part generation Pierre-Yves David <pierre-yves.david@fb.com> parents: 23066 diff changeset	1106 yield _pack(_fpayloadsize, -1)
25321 b44ee346211f bundle2: add debug output for part generation Pierre-Yves David <pierre-yves.david@fb.com> parents: 25320 diff changeset	1107 for chunk in interpart.getchunks(ui=ui):
23067 420a051616ce bundle2: transmit exception during part generation Pierre-Yves David <pierre-yves.david@fb.com> parents: 23066 diff changeset	1108 yield chunk
25321 b44ee346211f bundle2: add debug output for part generation Pierre-Yves David <pierre-yves.david@fb.com> parents: 25320 diff changeset	1109 outdebug(ui, 'closing payload chunk')
23067 420a051616ce bundle2: transmit exception during part generation Pierre-Yves David <pierre-yves.david@fb.com> parents: 23066 diff changeset	1110 # abort current part payload
420a051616ce bundle2: transmit exception during part generation Pierre-Yves David <pierre-yves.david@fb.com> parents: 23066 diff changeset	1111 yield _pack(_fpayloadsize, 0)
32186 76f9a0009b4b pycompat: extract helper to raise exception with traceback Yuya Nishihara <yuya@tcha.org> parents: 32024 diff changeset	1112 pycompat.raisewithtb(exc, tb)
21000 4cae06ae1562 bundle2: extract a _payloadchunks method for part Pierre-Yves David <pierre-yves.david@fb.com> parents: 20998 diff changeset	1113 # end of payload
25321 b44ee346211f bundle2: add debug output for part generation Pierre-Yves David <pierre-yves.david@fb.com> parents: 25320 diff changeset	1114 outdebug(ui, 'closing payload chunk')
21000 4cae06ae1562 bundle2: extract a _payloadchunks method for part Pierre-Yves David <pierre-yves.david@fb.com> parents: 20998 diff changeset	1115 yield _pack(_fpayloadsize, 0)
21601 7ff01befc7ec bundle2: track life cycle of parts Pierre-Yves David <pierre-yves.david@fb.com> parents: 21600 diff changeset	1116 self._generated = True
21000 4cae06ae1562 bundle2: extract a _payloadchunks method for part Pierre-Yves David <pierre-yves.david@fb.com> parents: 20998 diff changeset	1117
4cae06ae1562 bundle2: extract a _payloadchunks method for part Pierre-Yves David <pierre-yves.david@fb.com> parents: 20998 diff changeset	1118 def _payloadchunks(self):
4cae06ae1562 bundle2: extract a _payloadchunks method for part Pierre-Yves David <pierre-yves.david@fb.com> parents: 20998 diff changeset	1119 """yield chunks of a the part payload
4cae06ae1562 bundle2: extract a _payloadchunks method for part Pierre-Yves David <pierre-yves.david@fb.com> parents: 20998 diff changeset	1120
4cae06ae1562 bundle2: extract a _payloadchunks method for part Pierre-Yves David <pierre-yves.david@fb.com> parents: 20998 diff changeset	1121 Exists to handle the different methods to provide data to a part."""
20876 ddd56f3eb786 bundle2: support for bundling and unbundling payload Pierre-Yves David <pierre-yves.david@fb.com> parents: 20864 diff changeset	1122 # we only support fixed size data now.
ddd56f3eb786 bundle2: support for bundling and unbundling payload Pierre-Yves David <pierre-yves.david@fb.com> parents: 20864 diff changeset	1123 # This will be improved in the future.
33673 5ae35a1347fd bundle2: look for __next__ as well as next to identify iterators Augie Fackler <augie@google.com> parents: 33672 diff changeset	1124 if (util.safehasattr(self.data, 'next')
5ae35a1347fd bundle2: look for __next__ as well as next to identify iterators Augie Fackler <augie@google.com> parents: 33672 diff changeset	1125 or util.safehasattr(self.data, '__next__')):
21001 c93bb6a08fa1 bundle2: support chunk iterator as part data Pierre-Yves David <pierre-yves.david@fb.com> parents: 21000 diff changeset	1126 buff = util.chunkbuffer(self.data)
c93bb6a08fa1 bundle2: support chunk iterator as part data Pierre-Yves David <pierre-yves.david@fb.com> parents: 21000 diff changeset	1127 chunk = buff.read(preferedchunksize)
c93bb6a08fa1 bundle2: support chunk iterator as part data Pierre-Yves David <pierre-yves.david@fb.com> parents: 21000 diff changeset	1128 while chunk:
c93bb6a08fa1 bundle2: support chunk iterator as part data Pierre-Yves David <pierre-yves.david@fb.com> parents: 21000 diff changeset	1129 yield chunk
c93bb6a08fa1 bundle2: support chunk iterator as part data Pierre-Yves David <pierre-yves.david@fb.com> parents: 21000 diff changeset	1130 chunk = buff.read(preferedchunksize)
c93bb6a08fa1 bundle2: support chunk iterator as part data Pierre-Yves David <pierre-yves.david@fb.com> parents: 21000 diff changeset	1131 elif len(self.data):
20876 ddd56f3eb786 bundle2: support for bundling and unbundling payload Pierre-Yves David <pierre-yves.david@fb.com> parents: 20864 diff changeset	1132 yield self.data
20802 520df53ad26a bundle2: a very first version of bundle2 unbundler Pierre-Yves David <pierre-yves.david@fb.com> parents: 20801 diff changeset	1133
23066 ad144882318d bundle2: add a interrupt mechanism Pierre-Yves David <pierre-yves.david@fb.com> parents: 23029 diff changeset	1134
ad144882318d bundle2: add a interrupt mechanism Pierre-Yves David <pierre-yves.david@fb.com> parents: 23029 diff changeset	1135 flaginterrupt = -1
ad144882318d bundle2: add a interrupt mechanism Pierre-Yves David <pierre-yves.david@fb.com> parents: 23029 diff changeset	1136
ad144882318d bundle2: add a interrupt mechanism Pierre-Yves David <pierre-yves.david@fb.com> parents: 23029 diff changeset	1137 class interrupthandler(unpackermixin):
ad144882318d bundle2: add a interrupt mechanism Pierre-Yves David <pierre-yves.david@fb.com> parents: 23029 diff changeset	1138 """read one part and process it with restricted capability
ad144882318d bundle2: add a interrupt mechanism Pierre-Yves David <pierre-yves.david@fb.com> parents: 23029 diff changeset	1139
ad144882318d bundle2: add a interrupt mechanism Pierre-Yves David <pierre-yves.david@fb.com> parents: 23029 diff changeset	1140 This allows to transmit exception raised on the producer size during part
ad144882318d bundle2: add a interrupt mechanism Pierre-Yves David <pierre-yves.david@fb.com> parents: 23029 diff changeset	1141 iteration while the consumer is reading a part.
ad144882318d bundle2: add a interrupt mechanism Pierre-Yves David <pierre-yves.david@fb.com> parents: 23029 diff changeset	1142
ad144882318d bundle2: add a interrupt mechanism Pierre-Yves David <pierre-yves.david@fb.com> parents: 23029 diff changeset	1143 Part processed in this manner only have access to a ui object,"""
ad144882318d bundle2: add a interrupt mechanism Pierre-Yves David <pierre-yves.david@fb.com> parents: 23029 diff changeset	1144
ad144882318d bundle2: add a interrupt mechanism Pierre-Yves David <pierre-yves.david@fb.com> parents: 23029 diff changeset	1145 def __init__(self, ui, fp):
ad144882318d bundle2: add a interrupt mechanism Pierre-Yves David <pierre-yves.david@fb.com> parents: 23029 diff changeset	1146 super(interrupthandler, self).__init__(fp)
ad144882318d bundle2: add a interrupt mechanism Pierre-Yves David <pierre-yves.david@fb.com> parents: 23029 diff changeset	1147 self.ui = ui
ad144882318d bundle2: add a interrupt mechanism Pierre-Yves David <pierre-yves.david@fb.com> parents: 23029 diff changeset	1148
ad144882318d bundle2: add a interrupt mechanism Pierre-Yves David <pierre-yves.david@fb.com> parents: 23029 diff changeset	1149 def _readpartheader(self):
ad144882318d bundle2: add a interrupt mechanism Pierre-Yves David <pierre-yves.david@fb.com> parents: 23029 diff changeset	1150 """reads a part header size and return the bytes blob
ad144882318d bundle2: add a interrupt mechanism Pierre-Yves David <pierre-yves.david@fb.com> parents: 23029 diff changeset	1151
ad144882318d bundle2: add a interrupt mechanism Pierre-Yves David <pierre-yves.david@fb.com> parents: 23029 diff changeset	1152 returns None if empty"""
ad144882318d bundle2: add a interrupt mechanism Pierre-Yves David <pierre-yves.david@fb.com> parents: 23029 diff changeset	1153 headersize = self._unpack(_fpartheadersize)[0]
ad144882318d bundle2: add a interrupt mechanism Pierre-Yves David <pierre-yves.david@fb.com> parents: 23029 diff changeset	1154 if headersize < 0:
ad144882318d bundle2: add a interrupt mechanism Pierre-Yves David <pierre-yves.david@fb.com> parents: 23029 diff changeset	1155 raise error.BundleValueError('negative part header size: %i'
ad144882318d bundle2: add a interrupt mechanism Pierre-Yves David <pierre-yves.david@fb.com> parents: 23029 diff changeset	1156 % headersize)
25318 64dd8f25fcf4 bundle2: introduce a specific function for debug messages while unbundling Pierre-Yves David <pierre-yves.david@fb.com> parents: 25317 diff changeset	1157 indebug(self.ui, 'part header size: %i\n' % headersize)
23066 ad144882318d bundle2: add a interrupt mechanism Pierre-Yves David <pierre-yves.david@fb.com> parents: 23029 diff changeset	1158 if headersize:
ad144882318d bundle2: add a interrupt mechanism Pierre-Yves David <pierre-yves.david@fb.com> parents: 23029 diff changeset	1159 return self._readexact(headersize)
ad144882318d bundle2: add a interrupt mechanism Pierre-Yves David <pierre-yves.david@fb.com> parents: 23029 diff changeset	1160 return None
ad144882318d bundle2: add a interrupt mechanism Pierre-Yves David <pierre-yves.david@fb.com> parents: 23029 diff changeset	1161
ad144882318d bundle2: add a interrupt mechanism Pierre-Yves David <pierre-yves.david@fb.com> parents: 23029 diff changeset	1162 def __call__(self):
25335 8f7137a85a0e bundle2: add generic debug output regarding processed interruption Pierre-Yves David <pierre-yves.david@fb.com> parents: 25334 diff changeset	1163
8f7137a85a0e bundle2: add generic debug output regarding processed interruption Pierre-Yves David <pierre-yves.david@fb.com> parents: 25334 diff changeset	1164 self.ui.debug('bundle2-input-stream-interrupt:'
8f7137a85a0e bundle2: add generic debug output regarding processed interruption Pierre-Yves David <pierre-yves.david@fb.com> parents: 25334 diff changeset	1165 ' opening out of band context\n')
25320 697d8953b04d bundle2: handle new line in 'indebug' function Pierre-Yves David <pierre-yves.david@fb.com> parents: 25319 diff changeset	1166 indebug(self.ui, 'bundle2 stream interruption, looking for a part.')
23066 ad144882318d bundle2: add a interrupt mechanism Pierre-Yves David <pierre-yves.david@fb.com> parents: 23029 diff changeset	1167 headerblock = self._readpartheader()
ad144882318d bundle2: add a interrupt mechanism Pierre-Yves David <pierre-yves.david@fb.com> parents: 23029 diff changeset	1168 if headerblock is None:
25320 697d8953b04d bundle2: handle new line in 'indebug' function Pierre-Yves David <pierre-yves.david@fb.com> parents: 25319 diff changeset	1169 indebug(self.ui, 'no part found during interruption.')
23066 ad144882318d bundle2: add a interrupt mechanism Pierre-Yves David <pierre-yves.david@fb.com> parents: 23029 diff changeset	1170 return
35116 da91e7309daf bundle2: don't use seekable bundle2 parts by default (issue5691) Gregory Szorc <gregory.szorc@gmail.com> parents: 35115 diff changeset	1171 part = unbundlepart(self.ui, headerblock, self._fp)
23066 ad144882318d bundle2: add a interrupt mechanism Pierre-Yves David <pierre-yves.david@fb.com> parents: 23029 diff changeset	1172 op = interruptoperation(self.ui)
34258 e71890f27767 bundle2: move processpart stream maintenance into part iterator Durham Goode <durham@fb.com> parents: 34249 diff changeset	1173 hardabort = False
e71890f27767 bundle2: move processpart stream maintenance into part iterator Durham Goode <durham@fb.com> parents: 34249 diff changeset	1174 try:
e71890f27767 bundle2: move processpart stream maintenance into part iterator Durham Goode <durham@fb.com> parents: 34249 diff changeset	1175 _processpart(op, part)
e71890f27767 bundle2: move processpart stream maintenance into part iterator Durham Goode <durham@fb.com> parents: 34249 diff changeset	1176 except (SystemExit, KeyboardInterrupt):
e71890f27767 bundle2: move processpart stream maintenance into part iterator Durham Goode <durham@fb.com> parents: 34249 diff changeset	1177 hardabort = True
e71890f27767 bundle2: move processpart stream maintenance into part iterator Durham Goode <durham@fb.com> parents: 34249 diff changeset	1178 raise
e71890f27767 bundle2: move processpart stream maintenance into part iterator Durham Goode <durham@fb.com> parents: 34249 diff changeset	1179 finally:
e71890f27767 bundle2: move processpart stream maintenance into part iterator Durham Goode <durham@fb.com> parents: 34249 diff changeset	1180 if not hardabort:
35114 db5038525718 bundle2: implement consume() API on unbundlepart Gregory Szorc <gregory.szorc@gmail.com> parents: 35113 diff changeset	1181 part.consume()
25335 8f7137a85a0e bundle2: add generic debug output regarding processed interruption Pierre-Yves David <pierre-yves.david@fb.com> parents: 25334 diff changeset	1182 self.ui.debug('bundle2-input-stream-interrupt:'
8f7137a85a0e bundle2: add generic debug output regarding processed interruption Pierre-Yves David <pierre-yves.david@fb.com> parents: 25334 diff changeset	1183 ' closing out of band context\n')
23066 ad144882318d bundle2: add a interrupt mechanism Pierre-Yves David <pierre-yves.david@fb.com> parents: 23029 diff changeset	1184
ad144882318d bundle2: add a interrupt mechanism Pierre-Yves David <pierre-yves.david@fb.com> parents: 23029 diff changeset	1185 class interruptoperation(object):
ad144882318d bundle2: add a interrupt mechanism Pierre-Yves David <pierre-yves.david@fb.com> parents: 23029 diff changeset	1186 """A limited operation to be use by part handler during interruption
ad144882318d bundle2: add a interrupt mechanism Pierre-Yves David <pierre-yves.david@fb.com> parents: 23029 diff changeset	1187
ad144882318d bundle2: add a interrupt mechanism Pierre-Yves David <pierre-yves.david@fb.com> parents: 23029 diff changeset	1188 It only have access to an ui object.
ad144882318d bundle2: add a interrupt mechanism Pierre-Yves David <pierre-yves.david@fb.com> parents: 23029 diff changeset	1189 """
ad144882318d bundle2: add a interrupt mechanism Pierre-Yves David <pierre-yves.david@fb.com> parents: 23029 diff changeset	1190
ad144882318d bundle2: add a interrupt mechanism Pierre-Yves David <pierre-yves.david@fb.com> parents: 23029 diff changeset	1191 def __init__(self, ui):
ad144882318d bundle2: add a interrupt mechanism Pierre-Yves David <pierre-yves.david@fb.com> parents: 23029 diff changeset	1192 self.ui = ui
ad144882318d bundle2: add a interrupt mechanism Pierre-Yves David <pierre-yves.david@fb.com> parents: 23029 diff changeset	1193 self.reply = None
24878 e530cde6d115 bundle2: disable ouput capture unless we use http (issue4613 issue4615) Pierre-Yves David <pierre-yves.david@fb.com> parents: 24851 diff changeset	1194 self.captureoutput = False
23066 ad144882318d bundle2: add a interrupt mechanism Pierre-Yves David <pierre-yves.david@fb.com> parents: 23029 diff changeset	1195
ad144882318d bundle2: add a interrupt mechanism Pierre-Yves David <pierre-yves.david@fb.com> parents: 23029 diff changeset	1196 @property
ad144882318d bundle2: add a interrupt mechanism Pierre-Yves David <pierre-yves.david@fb.com> parents: 23029 diff changeset	1197 def repo(self):
31647 4dbef666c6c9 bundle2: use ProgrammingError Jun Wu <quark@fb.com> parents: 31476 diff changeset	1198 raise error.ProgrammingError('no repo access from stream interruption')
23066 ad144882318d bundle2: add a interrupt mechanism Pierre-Yves David <pierre-yves.david@fb.com> parents: 23029 diff changeset	1199
ad144882318d bundle2: add a interrupt mechanism Pierre-Yves David <pierre-yves.david@fb.com> parents: 23029 diff changeset	1200 def gettransaction(self):
ad144882318d bundle2: add a interrupt mechanism Pierre-Yves David <pierre-yves.david@fb.com> parents: 23029 diff changeset	1201 raise TransactionUnavailable('no repo access from stream interruption')
ad144882318d bundle2: add a interrupt mechanism Pierre-Yves David <pierre-yves.david@fb.com> parents: 23029 diff changeset	1202
35113 8aa43ff9c12c bundle2: implement generic part payload decoder Gregory Szorc <gregory.szorc@gmail.com> parents: 35112 diff changeset	1203 def decodepayloadchunks(ui, fh):
8aa43ff9c12c bundle2: implement generic part payload decoder Gregory Szorc <gregory.szorc@gmail.com> parents: 35112 diff changeset	1204 """Reads bundle2 part payload data into chunks.
8aa43ff9c12c bundle2: implement generic part payload decoder Gregory Szorc <gregory.szorc@gmail.com> parents: 35112 diff changeset	1205
8aa43ff9c12c bundle2: implement generic part payload decoder Gregory Szorc <gregory.szorc@gmail.com> parents: 35112 diff changeset	1206 Part payload data consists of framed chunks. This function takes
8aa43ff9c12c bundle2: implement generic part payload decoder Gregory Szorc <gregory.szorc@gmail.com> parents: 35112 diff changeset	1207 a file handle and emits those chunks.
8aa43ff9c12c bundle2: implement generic part payload decoder Gregory Szorc <gregory.szorc@gmail.com> parents: 35112 diff changeset	1208 """
35117 589eed45a452 bundle2: inline debug logging Gregory Szorc <gregory.szorc@gmail.com> parents: 35116 diff changeset	1209 dolog = ui.configbool('devel', 'bundle2.debug')
589eed45a452 bundle2: inline debug logging Gregory Szorc <gregory.szorc@gmail.com> parents: 35116 diff changeset	1210 debug = ui.debug
589eed45a452 bundle2: inline debug logging Gregory Szorc <gregory.szorc@gmail.com> parents: 35116 diff changeset	1211
35119 764e3ad1cf54 bundle2: inline struct operations Gregory Szorc <gregory.szorc@gmail.com> parents: 35118 diff changeset	1212 headerstruct = struct.Struct(_fpayloadsize)
764e3ad1cf54 bundle2: inline struct operations Gregory Szorc <gregory.szorc@gmail.com> parents: 35118 diff changeset	1213 headersize = headerstruct.size
764e3ad1cf54 bundle2: inline struct operations Gregory Szorc <gregory.szorc@gmail.com> parents: 35118 diff changeset	1214 unpack = headerstruct.unpack
764e3ad1cf54 bundle2: inline struct operations Gregory Szorc <gregory.szorc@gmail.com> parents: 35118 diff changeset	1215
35113 8aa43ff9c12c bundle2: implement generic part payload decoder Gregory Szorc <gregory.szorc@gmail.com> parents: 35112 diff changeset	1216 readexactly = changegroup.readexactly
35118 1fb0846ad792 bundle2: inline changegroup.readexactly() Gregory Szorc <gregory.szorc@gmail.com> parents: 35117 diff changeset	1217 read = fh.read
35113 8aa43ff9c12c bundle2: implement generic part payload decoder Gregory Szorc <gregory.szorc@gmail.com> parents: 35112 diff changeset	1218
35119 764e3ad1cf54 bundle2: inline struct operations Gregory Szorc <gregory.szorc@gmail.com> parents: 35118 diff changeset	1219 chunksize = unpack(readexactly(fh, headersize))[0]
35113 8aa43ff9c12c bundle2: implement generic part payload decoder Gregory Szorc <gregory.szorc@gmail.com> parents: 35112 diff changeset	1220 indebug(ui, 'payload chunk size: %i' % chunksize)
8aa43ff9c12c bundle2: implement generic part payload decoder Gregory Szorc <gregory.szorc@gmail.com> parents: 35112 diff changeset	1221
35118 1fb0846ad792 bundle2: inline changegroup.readexactly() Gregory Szorc <gregory.szorc@gmail.com> parents: 35117 diff changeset	1222 # changegroup.readexactly() is inlined below for performance.
35113 8aa43ff9c12c bundle2: implement generic part payload decoder Gregory Szorc <gregory.szorc@gmail.com> parents: 35112 diff changeset	1223 while chunksize:
8aa43ff9c12c bundle2: implement generic part payload decoder Gregory Szorc <gregory.szorc@gmail.com> parents: 35112 diff changeset	1224 if chunksize >= 0:
35118 1fb0846ad792 bundle2: inline changegroup.readexactly() Gregory Szorc <gregory.szorc@gmail.com> parents: 35117 diff changeset	1225 s = read(chunksize)
1fb0846ad792 bundle2: inline changegroup.readexactly() Gregory Szorc <gregory.szorc@gmail.com> parents: 35117 diff changeset	1226 if len(s) < chunksize:
1fb0846ad792 bundle2: inline changegroup.readexactly() Gregory Szorc <gregory.szorc@gmail.com> parents: 35117 diff changeset	1227 raise error.Abort(_('stream ended unexpectedly '
1fb0846ad792 bundle2: inline changegroup.readexactly() Gregory Szorc <gregory.szorc@gmail.com> parents: 35117 diff changeset	1228 ' (got %d bytes, expected %d)') %
1fb0846ad792 bundle2: inline changegroup.readexactly() Gregory Szorc <gregory.szorc@gmail.com> parents: 35117 diff changeset	1229 (len(s), chunksize))
1fb0846ad792 bundle2: inline changegroup.readexactly() Gregory Szorc <gregory.szorc@gmail.com> parents: 35117 diff changeset	1230
1fb0846ad792 bundle2: inline changegroup.readexactly() Gregory Szorc <gregory.szorc@gmail.com> parents: 35117 diff changeset	1231 yield s
35113 8aa43ff9c12c bundle2: implement generic part payload decoder Gregory Szorc <gregory.szorc@gmail.com> parents: 35112 diff changeset	1232 elif chunksize == flaginterrupt:
8aa43ff9c12c bundle2: implement generic part payload decoder Gregory Szorc <gregory.szorc@gmail.com> parents: 35112 diff changeset	1233 # Interrupt "signal" detected. The regular stream is interrupted
8aa43ff9c12c bundle2: implement generic part payload decoder Gregory Szorc <gregory.szorc@gmail.com> parents: 35112 diff changeset	1234 # and a bundle2 part follows. Consume it.
8aa43ff9c12c bundle2: implement generic part payload decoder Gregory Szorc <gregory.szorc@gmail.com> parents: 35112 diff changeset	1235 interrupthandler(ui, fh)()
8aa43ff9c12c bundle2: implement generic part payload decoder Gregory Szorc <gregory.szorc@gmail.com> parents: 35112 diff changeset	1236 else:
8aa43ff9c12c bundle2: implement generic part payload decoder Gregory Szorc <gregory.szorc@gmail.com> parents: 35112 diff changeset	1237 raise error.BundleValueError(
8aa43ff9c12c bundle2: implement generic part payload decoder Gregory Szorc <gregory.szorc@gmail.com> parents: 35112 diff changeset	1238 'negative payload chunk size: %s' % chunksize)
8aa43ff9c12c bundle2: implement generic part payload decoder Gregory Szorc <gregory.szorc@gmail.com> parents: 35112 diff changeset	1239
35118 1fb0846ad792 bundle2: inline changegroup.readexactly() Gregory Szorc <gregory.szorc@gmail.com> parents: 35117 diff changeset	1240 s = read(headersize)
1fb0846ad792 bundle2: inline changegroup.readexactly() Gregory Szorc <gregory.szorc@gmail.com> parents: 35117 diff changeset	1241 if len(s) < headersize:
1fb0846ad792 bundle2: inline changegroup.readexactly() Gregory Szorc <gregory.szorc@gmail.com> parents: 35117 diff changeset	1242 raise error.Abort(_('stream ended unexpectedly '
1fb0846ad792 bundle2: inline changegroup.readexactly() Gregory Szorc <gregory.szorc@gmail.com> parents: 35117 diff changeset	1243 ' (got %d bytes, expected %d)') %
1fb0846ad792 bundle2: inline changegroup.readexactly() Gregory Szorc <gregory.szorc@gmail.com> parents: 35117 diff changeset	1244 (len(s), chunksize))
1fb0846ad792 bundle2: inline changegroup.readexactly() Gregory Szorc <gregory.szorc@gmail.com> parents: 35117 diff changeset	1245
35119 764e3ad1cf54 bundle2: inline struct operations Gregory Szorc <gregory.szorc@gmail.com> parents: 35118 diff changeset	1246 chunksize = unpack(s)[0]
35117 589eed45a452 bundle2: inline debug logging Gregory Szorc <gregory.szorc@gmail.com> parents: 35116 diff changeset	1247
589eed45a452 bundle2: inline debug logging Gregory Szorc <gregory.szorc@gmail.com> parents: 35116 diff changeset	1248 # indebug() inlined for performance.
589eed45a452 bundle2: inline debug logging Gregory Szorc <gregory.szorc@gmail.com> parents: 35116 diff changeset	1249 if dolog:
589eed45a452 bundle2: inline debug logging Gregory Szorc <gregory.szorc@gmail.com> parents: 35116 diff changeset	1250 debug('bundle2-input: payload chunk size: %i\n' % chunksize)
35113 8aa43ff9c12c bundle2: implement generic part payload decoder Gregory Szorc <gregory.szorc@gmail.com> parents: 35112 diff changeset	1251
21014 a6246bba7b9e bundle2: add an unbundle part responsible from unbundling part Pierre-Yves David <pierre-yves.david@fb.com> parents: 21013 diff changeset	1252 class unbundlepart(unpackermixin):
a6246bba7b9e bundle2: add an unbundle part responsible from unbundling part Pierre-Yves David <pierre-yves.david@fb.com> parents: 21013 diff changeset	1253 """a bundle part read from a bundle"""
a6246bba7b9e bundle2: add an unbundle part responsible from unbundling part Pierre-Yves David <pierre-yves.david@fb.com> parents: 21013 diff changeset	1254
a6246bba7b9e bundle2: add an unbundle part responsible from unbundling part Pierre-Yves David <pierre-yves.david@fb.com> parents: 21013 diff changeset	1255 def __init__(self, ui, header, fp):
a6246bba7b9e bundle2: add an unbundle part responsible from unbundling part Pierre-Yves David <pierre-yves.david@fb.com> parents: 21013 diff changeset	1256 super(unbundlepart, self).__init__(fp)
31889 a02e773008f5 bundle2: move 'seek' and 'tell' methods off the unpackermixin class Pierre-Yves David <pierre-yves.david@ens-lyon.org> parents: 31863 diff changeset	1257 self._seekable = (util.safehasattr(fp, 'seek') and
a02e773008f5 bundle2: move 'seek' and 'tell' methods off the unpackermixin class Pierre-Yves David <pierre-yves.david@ens-lyon.org> parents: 31863 diff changeset	1258 util.safehasattr(fp, 'tell'))
21014 a6246bba7b9e bundle2: add an unbundle part responsible from unbundling part Pierre-Yves David <pierre-yves.david@fb.com> parents: 21013 diff changeset	1259 self.ui = ui
a6246bba7b9e bundle2: add an unbundle part responsible from unbundling part Pierre-Yves David <pierre-yves.david@fb.com> parents: 21013 diff changeset	1260 # unbundle state attr
a6246bba7b9e bundle2: add an unbundle part responsible from unbundling part Pierre-Yves David <pierre-yves.david@fb.com> parents: 21013 diff changeset	1261 self._headerdata = header
21015 14dd49260246 bundle2: move the fromheader closure into the class itself Pierre-Yves David <pierre-yves.david@fb.com> parents: 21014 diff changeset	1262 self._headeroffset = 0
21019 3dc09f831a2e bundle2: lazy unbundle of part payload Pierre-Yves David <pierre-yves.david@fb.com> parents: 21016 diff changeset	1263 self._initialized = False
3dc09f831a2e bundle2: lazy unbundle of part payload Pierre-Yves David <pierre-yves.david@fb.com> parents: 21016 diff changeset	1264 self.consumed = False
21014 a6246bba7b9e bundle2: add an unbundle part responsible from unbundling part Pierre-Yves David <pierre-yves.david@fb.com> parents: 21013 diff changeset	1265 # part data
a6246bba7b9e bundle2: add an unbundle part responsible from unbundling part Pierre-Yves David <pierre-yves.david@fb.com> parents: 21013 diff changeset	1266 self.id = None
a6246bba7b9e bundle2: add an unbundle part responsible from unbundling part Pierre-Yves David <pierre-yves.david@fb.com> parents: 21013 diff changeset	1267 self.type = None
a6246bba7b9e bundle2: add an unbundle part responsible from unbundling part Pierre-Yves David <pierre-yves.david@fb.com> parents: 21013 diff changeset	1268 self.mandatoryparams = None
a6246bba7b9e bundle2: add an unbundle part responsible from unbundling part Pierre-Yves David <pierre-yves.david@fb.com> parents: 21013 diff changeset	1269 self.advisoryparams = None
21610 d6056805f8f4 bundle2: introduce a ``params`` dictionary on unbundled parts Pierre-Yves David <pierre-yves.david@fb.com> parents: 21609 diff changeset	1270 self.params = None
21612 f221eb0531d9 bundle2: expose mandatory params in a mandatorykeys attribute Pierre-Yves David <pierre-yves.david@fb.com> parents: 21611 diff changeset	1271 self.mandatorykeys = ()
21019 3dc09f831a2e bundle2: lazy unbundle of part payload Pierre-Yves David <pierre-yves.david@fb.com> parents: 21016 diff changeset	1272 self._readheader()
23585 94b25d71dd0f bundle2.unbundlepart: decouple mandatory from parttype Eric Sumner <ericsumner@fb.com> parents: 23439 diff changeset	1273 self._mandatory = None
24036 c7601086338a bundle2.unbundlepart: tell() implementation Eric Sumner <ericsumner@fb.com> parents: 24035 diff changeset	1274 self._pos = 0
21014 a6246bba7b9e bundle2: add an unbundle part responsible from unbundling part Pierre-Yves David <pierre-yves.david@fb.com> parents: 21013 diff changeset	1275
21015 14dd49260246 bundle2: move the fromheader closure into the class itself Pierre-Yves David <pierre-yves.david@fb.com> parents: 21014 diff changeset	1276 def _fromheader(self, size):
14dd49260246 bundle2: move the fromheader closure into the class itself Pierre-Yves David <pierre-yves.david@fb.com> parents: 21014 diff changeset	1277 """return the next <size> byte from the header"""
14dd49260246 bundle2: move the fromheader closure into the class itself Pierre-Yves David <pierre-yves.david@fb.com> parents: 21014 diff changeset	1278 offset = self._headeroffset
14dd49260246 bundle2: move the fromheader closure into the class itself Pierre-Yves David <pierre-yves.david@fb.com> parents: 21014 diff changeset	1279 data = self._headerdata[offset:(offset + size)]
21019 3dc09f831a2e bundle2: lazy unbundle of part payload Pierre-Yves David <pierre-yves.david@fb.com> parents: 21016 diff changeset	1280 self._headeroffset = offset + size
21015 14dd49260246 bundle2: move the fromheader closure into the class itself Pierre-Yves David <pierre-yves.david@fb.com> parents: 21014 diff changeset	1281 return data
14dd49260246 bundle2: move the fromheader closure into the class itself Pierre-Yves David <pierre-yves.david@fb.com> parents: 21014 diff changeset	1282
21016 b477afb1c81e bundle2: move unpackheader closure into the class Pierre-Yves David <pierre-yves.david@fb.com> parents: 21015 diff changeset	1283 def _unpackheader(self, format):
b477afb1c81e bundle2: move unpackheader closure into the class Pierre-Yves David <pierre-yves.david@fb.com> parents: 21015 diff changeset	1284 """read given format from header
b477afb1c81e bundle2: move unpackheader closure into the class Pierre-Yves David <pierre-yves.david@fb.com> parents: 21015 diff changeset	1285
b477afb1c81e bundle2: move unpackheader closure into the class Pierre-Yves David <pierre-yves.david@fb.com> parents: 21015 diff changeset	1286 This automatically compute the size of the format to read."""
b477afb1c81e bundle2: move unpackheader closure into the class Pierre-Yves David <pierre-yves.david@fb.com> parents: 21015 diff changeset	1287 data = self._fromheader(struct.calcsize(format))
b477afb1c81e bundle2: move unpackheader closure into the class Pierre-Yves David <pierre-yves.david@fb.com> parents: 21015 diff changeset	1288 return _unpack(format, data)
b477afb1c81e bundle2: move unpackheader closure into the class Pierre-Yves David <pierre-yves.david@fb.com> parents: 21015 diff changeset	1289
21608 3cb96ca90c17 bundle2: introduce an ``_initparams`` method Pierre-Yves David <pierre-yves.david@fb.com> parents: 21607 diff changeset	1290 def _initparams(self, mandatoryparams, advisoryparams):
3cb96ca90c17 bundle2: introduce an ``_initparams`` method Pierre-Yves David <pierre-yves.david@fb.com> parents: 21607 diff changeset	1291 """internal function to setup all logic related parameters"""
21609 63cc2594ef8a bundle2: make sure unbundled part param are read-only Pierre-Yves David <pierre-yves.david@fb.com> parents: 21608 diff changeset	1292 # make it read only to prevent people touching it by mistake.
63cc2594ef8a bundle2: make sure unbundled part param are read-only Pierre-Yves David <pierre-yves.david@fb.com> parents: 21608 diff changeset	1293 self.mandatoryparams = tuple(mandatoryparams)
63cc2594ef8a bundle2: make sure unbundled part param are read-only Pierre-Yves David <pierre-yves.david@fb.com> parents: 21608 diff changeset	1294 self.advisoryparams = tuple(advisoryparams)
21610 d6056805f8f4 bundle2: introduce a ``params`` dictionary on unbundled parts Pierre-Yves David <pierre-yves.david@fb.com> parents: 21609 diff changeset	1295 # user friendly UI
29591 6215b5537ba5 bundle2: use a sorted dict for holding parameters Gregory Szorc <gregory.szorc@gmail.com> parents: 29294 diff changeset	1296 self.params = util.sortdict(self.mandatoryparams)
6215b5537ba5 bundle2: use a sorted dict for holding parameters Gregory Szorc <gregory.szorc@gmail.com> parents: 29294 diff changeset	1297 self.params.update(self.advisoryparams)
21612 f221eb0531d9 bundle2: expose mandatory params in a mandatorykeys attribute Pierre-Yves David <pierre-yves.david@fb.com> parents: 21611 diff changeset	1298 self.mandatorykeys = frozenset(p[0] for p in mandatoryparams)
21608 3cb96ca90c17 bundle2: introduce an ``_initparams`` method Pierre-Yves David <pierre-yves.david@fb.com> parents: 21607 diff changeset	1299
21019 3dc09f831a2e bundle2: lazy unbundle of part payload Pierre-Yves David <pierre-yves.david@fb.com> parents: 21016 diff changeset	1300 def _readheader(self):
21014 a6246bba7b9e bundle2: add an unbundle part responsible from unbundling part Pierre-Yves David <pierre-yves.david@fb.com> parents: 21013 diff changeset	1301 """read the header and setup the object"""
21016 b477afb1c81e bundle2: move unpackheader closure into the class Pierre-Yves David <pierre-yves.david@fb.com> parents: 21015 diff changeset	1302 typesize = self._unpackheader(_fparttypesize)[0]
21015 14dd49260246 bundle2: move the fromheader closure into the class itself Pierre-Yves David <pierre-yves.david@fb.com> parents: 21014 diff changeset	1303 self.type = self._fromheader(typesize)
25320 697d8953b04d bundle2: handle new line in 'indebug' function Pierre-Yves David <pierre-yves.david@fb.com> parents: 25319 diff changeset	1304 indebug(self.ui, 'part type: "%s"' % self.type)
21016 b477afb1c81e bundle2: move unpackheader closure into the class Pierre-Yves David <pierre-yves.david@fb.com> parents: 21015 diff changeset	1305 self.id = self._unpackheader(_fpartid)[0]
33711 373ca5103ba8 bundle2: convert ints to strings using pycompat.bytestring() Augie Fackler <augie@google.com> parents: 33693 diff changeset	1306 indebug(self.ui, 'part id: "%s"' % pycompat.bytestr(self.id))
23585 94b25d71dd0f bundle2.unbundlepart: decouple mandatory from parttype Eric Sumner <ericsumner@fb.com> parents: 23439 diff changeset	1307 # extract mandatory bit from type
94b25d71dd0f bundle2.unbundlepart: decouple mandatory from parttype Eric Sumner <ericsumner@fb.com> parents: 23439 diff changeset	1308 self.mandatory = (self.type != self.type.lower())
94b25d71dd0f bundle2.unbundlepart: decouple mandatory from parttype Eric Sumner <ericsumner@fb.com> parents: 23439 diff changeset	1309 self.type = self.type.lower()
21014 a6246bba7b9e bundle2: add an unbundle part responsible from unbundling part Pierre-Yves David <pierre-yves.david@fb.com> parents: 21013 diff changeset	1310 ## reading parameters
a6246bba7b9e bundle2: add an unbundle part responsible from unbundling part Pierre-Yves David <pierre-yves.david@fb.com> parents: 21013 diff changeset	1311 # param count
21016 b477afb1c81e bundle2: move unpackheader closure into the class Pierre-Yves David <pierre-yves.david@fb.com> parents: 21015 diff changeset	1312 mancount, advcount = self._unpackheader(_fpartparamcount)
25320 697d8953b04d bundle2: handle new line in 'indebug' function Pierre-Yves David <pierre-yves.david@fb.com> parents: 25319 diff changeset	1313 indebug(self.ui, 'part parameters: %i' % (mancount + advcount))
21014 a6246bba7b9e bundle2: add an unbundle part responsible from unbundling part Pierre-Yves David <pierre-yves.david@fb.com> parents: 21013 diff changeset	1314 # param size
21016 b477afb1c81e bundle2: move unpackheader closure into the class Pierre-Yves David <pierre-yves.david@fb.com> parents: 21015 diff changeset	1315 fparamsizes = _makefpartparamsizes(mancount + advcount)
b477afb1c81e bundle2: move unpackheader closure into the class Pierre-Yves David <pierre-yves.david@fb.com> parents: 21015 diff changeset	1316 paramsizes = self._unpackheader(fparamsizes)
21014 a6246bba7b9e bundle2: add an unbundle part responsible from unbundling part Pierre-Yves David <pierre-yves.david@fb.com> parents: 21013 diff changeset	1317 # make it a list of couple again
33674 9a323773216c bundle2: work around zip() being lazy in Python 3 Augie Fackler <augie@google.com> parents: 33673 diff changeset	1318 paramsizes = list(zip(paramsizes[::2], paramsizes[1::2]))
21014 a6246bba7b9e bundle2: add an unbundle part responsible from unbundling part Pierre-Yves David <pierre-yves.david@fb.com> parents: 21013 diff changeset	1319 # split mandatory from advisory
a6246bba7b9e bundle2: add an unbundle part responsible from unbundling part Pierre-Yves David <pierre-yves.david@fb.com> parents: 21013 diff changeset	1320 mansizes = paramsizes[:mancount]
a6246bba7b9e bundle2: add an unbundle part responsible from unbundling part Pierre-Yves David <pierre-yves.david@fb.com> parents: 21013 diff changeset	1321 advsizes = paramsizes[mancount:]
23139 e53f6b72a0e4 spelling: fixes from proofreading of spell checker issues Mads Kiilerich <madski@unity3d.com> parents: 23067 diff changeset	1322 # retrieve param value
21014 a6246bba7b9e bundle2: add an unbundle part responsible from unbundling part Pierre-Yves David <pierre-yves.david@fb.com> parents: 21013 diff changeset	1323 manparams = []
a6246bba7b9e bundle2: add an unbundle part responsible from unbundling part Pierre-Yves David <pierre-yves.david@fb.com> parents: 21013 diff changeset	1324 for key, value in mansizes:
21015 14dd49260246 bundle2: move the fromheader closure into the class itself Pierre-Yves David <pierre-yves.david@fb.com> parents: 21014 diff changeset	1325 manparams.append((self._fromheader(key), self._fromheader(value)))
21014 a6246bba7b9e bundle2: add an unbundle part responsible from unbundling part Pierre-Yves David <pierre-yves.david@fb.com> parents: 21013 diff changeset	1326 advparams = []
a6246bba7b9e bundle2: add an unbundle part responsible from unbundling part Pierre-Yves David <pierre-yves.david@fb.com> parents: 21013 diff changeset	1327 for key, value in advsizes:
21015 14dd49260246 bundle2: move the fromheader closure into the class itself Pierre-Yves David <pierre-yves.david@fb.com> parents: 21014 diff changeset	1328 advparams.append((self._fromheader(key), self._fromheader(value)))
21608 3cb96ca90c17 bundle2: introduce an ``_initparams`` method Pierre-Yves David <pierre-yves.david@fb.com> parents: 21607 diff changeset	1329 self._initparams(manparams, advparams)
21014 a6246bba7b9e bundle2: add an unbundle part responsible from unbundling part Pierre-Yves David <pierre-yves.david@fb.com> parents: 21013 diff changeset	1330 ## part payload
24034 9881a1437799 bundle2.unbundlepart: raise payloadchunks from a closure to a method Eric Sumner <ericsumner@fb.com> parents: 24026 diff changeset	1331 self._payloadstream = util.chunkbuffer(self._payloadchunks())
21019 3dc09f831a2e bundle2: lazy unbundle of part payload Pierre-Yves David <pierre-yves.david@fb.com> parents: 21016 diff changeset	1332 # we read the data, tell it
3dc09f831a2e bundle2: lazy unbundle of part payload Pierre-Yves David <pierre-yves.david@fb.com> parents: 21016 diff changeset	1333 self._initialized = True
3dc09f831a2e bundle2: lazy unbundle of part payload Pierre-Yves David <pierre-yves.david@fb.com> parents: 21016 diff changeset	1334
35113 8aa43ff9c12c bundle2: implement generic part payload decoder Gregory Szorc <gregory.szorc@gmail.com> parents: 35112 diff changeset	1335 def _payloadchunks(self):
8aa43ff9c12c bundle2: implement generic part payload decoder Gregory Szorc <gregory.szorc@gmail.com> parents: 35112 diff changeset	1336 """Generator of decoded chunks in the payload."""
8aa43ff9c12c bundle2: implement generic part payload decoder Gregory Szorc <gregory.szorc@gmail.com> parents: 35112 diff changeset	1337 return decodepayloadchunks(self.ui, self._fp)
8aa43ff9c12c bundle2: implement generic part payload decoder Gregory Szorc <gregory.szorc@gmail.com> parents: 35112 diff changeset	1338
35114 db5038525718 bundle2: implement consume() API on unbundlepart Gregory Szorc <gregory.szorc@gmail.com> parents: 35113 diff changeset	1339 def consume(self):
db5038525718 bundle2: implement consume() API on unbundlepart Gregory Szorc <gregory.szorc@gmail.com> parents: 35113 diff changeset	1340 """Read the part payload until completion.
db5038525718 bundle2: implement consume() API on unbundlepart Gregory Szorc <gregory.szorc@gmail.com> parents: 35113 diff changeset	1341
db5038525718 bundle2: implement consume() API on unbundlepart Gregory Szorc <gregory.szorc@gmail.com> parents: 35113 diff changeset	1342 By consuming the part data, the underlying stream read offset will
db5038525718 bundle2: implement consume() API on unbundlepart Gregory Szorc <gregory.szorc@gmail.com> parents: 35113 diff changeset	1343 be advanced to the next part (or end of stream).
db5038525718 bundle2: implement consume() API on unbundlepart Gregory Szorc <gregory.szorc@gmail.com> parents: 35113 diff changeset	1344 """
db5038525718 bundle2: implement consume() API on unbundlepart Gregory Szorc <gregory.szorc@gmail.com> parents: 35113 diff changeset	1345 if self.consumed:
db5038525718 bundle2: implement consume() API on unbundlepart Gregory Szorc <gregory.szorc@gmail.com> parents: 35113 diff changeset	1346 return
db5038525718 bundle2: implement consume() API on unbundlepart Gregory Szorc <gregory.szorc@gmail.com> parents: 35113 diff changeset	1347
db5038525718 bundle2: implement consume() API on unbundlepart Gregory Szorc <gregory.szorc@gmail.com> parents: 35113 diff changeset	1348 chunk = self.read(32768)
db5038525718 bundle2: implement consume() API on unbundlepart Gregory Szorc <gregory.szorc@gmail.com> parents: 35113 diff changeset	1349 while chunk:
db5038525718 bundle2: implement consume() API on unbundlepart Gregory Szorc <gregory.szorc@gmail.com> parents: 35113 diff changeset	1350 self._pos += len(chunk)
db5038525718 bundle2: implement consume() API on unbundlepart Gregory Szorc <gregory.szorc@gmail.com> parents: 35113 diff changeset	1351 chunk = self.read(32768)
db5038525718 bundle2: implement consume() API on unbundlepart Gregory Szorc <gregory.szorc@gmail.com> parents: 35113 diff changeset	1352
21019 3dc09f831a2e bundle2: lazy unbundle of part payload Pierre-Yves David <pierre-yves.david@fb.com> parents: 21016 diff changeset	1353 def read(self, size=None):
3dc09f831a2e bundle2: lazy unbundle of part payload Pierre-Yves David <pierre-yves.david@fb.com> parents: 21016 diff changeset	1354 """read payload data"""
3dc09f831a2e bundle2: lazy unbundle of part payload Pierre-Yves David <pierre-yves.david@fb.com> parents: 21016 diff changeset	1355 if not self._initialized:
3dc09f831a2e bundle2: lazy unbundle of part payload Pierre-Yves David <pierre-yves.david@fb.com> parents: 21016 diff changeset	1356 self._readheader()
3dc09f831a2e bundle2: lazy unbundle of part payload Pierre-Yves David <pierre-yves.david@fb.com> parents: 21016 diff changeset	1357 if size is None:
3dc09f831a2e bundle2: lazy unbundle of part payload Pierre-Yves David <pierre-yves.david@fb.com> parents: 21016 diff changeset	1358 data = self._payloadstream.read()
3dc09f831a2e bundle2: lazy unbundle of part payload Pierre-Yves David <pierre-yves.david@fb.com> parents: 21016 diff changeset	1359 else:
3dc09f831a2e bundle2: lazy unbundle of part payload Pierre-Yves David <pierre-yves.david@fb.com> parents: 21016 diff changeset	1360 data = self._payloadstream.read(size)
25334 76cba1ecf55f bundle2: add generic debug output regarding processed part payload Pierre-Yves David <pierre-yves.david@fb.com> parents: 25333 diff changeset	1361 self._pos += len(data)
21019 3dc09f831a2e bundle2: lazy unbundle of part payload Pierre-Yves David <pierre-yves.david@fb.com> parents: 21016 diff changeset	1362 if size is None or len(data) < size:
25334 76cba1ecf55f bundle2: add generic debug output regarding processed part payload Pierre-Yves David <pierre-yves.david@fb.com> parents: 25333 diff changeset	1363 if not self.consumed and self._pos:
76cba1ecf55f bundle2: add generic debug output regarding processed part payload Pierre-Yves David <pierre-yves.david@fb.com> parents: 25333 diff changeset	1364 self.ui.debug('bundle2-input-part: total payload size %i\n'
76cba1ecf55f bundle2: add generic debug output regarding processed part payload Pierre-Yves David <pierre-yves.david@fb.com> parents: 25333 diff changeset	1365 % self._pos)
21019 3dc09f831a2e bundle2: lazy unbundle of part payload Pierre-Yves David <pierre-yves.david@fb.com> parents: 21016 diff changeset	1366 self.consumed = True
3dc09f831a2e bundle2: lazy unbundle of part payload Pierre-Yves David <pierre-yves.david@fb.com> parents: 21016 diff changeset	1367 return data
3dc09f831a2e bundle2: lazy unbundle of part payload Pierre-Yves David <pierre-yves.david@fb.com> parents: 21016 diff changeset	1368
35112 073eec083e25 bundle2: extract logic for seeking bundle2 part into own class Gregory Szorc <gregory.szorc@gmail.com> parents: 35046 diff changeset	1369 class seekableunbundlepart(unbundlepart):
073eec083e25 bundle2: extract logic for seeking bundle2 part into own class Gregory Szorc <gregory.szorc@gmail.com> parents: 35046 diff changeset	1370 """A bundle2 part in a bundle that is seekable.
073eec083e25 bundle2: extract logic for seeking bundle2 part into own class Gregory Szorc <gregory.szorc@gmail.com> parents: 35046 diff changeset	1371
073eec083e25 bundle2: extract logic for seeking bundle2 part into own class Gregory Szorc <gregory.szorc@gmail.com> parents: 35046 diff changeset	1372 Regular ``unbundlepart`` instances can only be read once. This class
073eec083e25 bundle2: extract logic for seeking bundle2 part into own class Gregory Szorc <gregory.szorc@gmail.com> parents: 35046 diff changeset	1373 extends ``unbundlepart`` to enable bi-directional seeking within the
073eec083e25 bundle2: extract logic for seeking bundle2 part into own class Gregory Szorc <gregory.szorc@gmail.com> parents: 35046 diff changeset	1374 part.
073eec083e25 bundle2: extract logic for seeking bundle2 part into own class Gregory Szorc <gregory.szorc@gmail.com> parents: 35046 diff changeset	1375
073eec083e25 bundle2: extract logic for seeking bundle2 part into own class Gregory Szorc <gregory.szorc@gmail.com> parents: 35046 diff changeset	1376 Bundle2 part data consists of framed chunks. Offsets when seeking
073eec083e25 bundle2: extract logic for seeking bundle2 part into own class Gregory Szorc <gregory.szorc@gmail.com> parents: 35046 diff changeset	1377 refer to the decoded data, not the offsets in the underlying bundle2
073eec083e25 bundle2: extract logic for seeking bundle2 part into own class Gregory Szorc <gregory.szorc@gmail.com> parents: 35046 diff changeset	1378 stream.
073eec083e25 bundle2: extract logic for seeking bundle2 part into own class Gregory Szorc <gregory.szorc@gmail.com> parents: 35046 diff changeset	1379
073eec083e25 bundle2: extract logic for seeking bundle2 part into own class Gregory Szorc <gregory.szorc@gmail.com> parents: 35046 diff changeset	1380 To facilitate quickly seeking within the decoded data, instances of this
073eec083e25 bundle2: extract logic for seeking bundle2 part into own class Gregory Szorc <gregory.szorc@gmail.com> parents: 35046 diff changeset	1381 class maintain a mapping between offsets in the underlying stream and
073eec083e25 bundle2: extract logic for seeking bundle2 part into own class Gregory Szorc <gregory.szorc@gmail.com> parents: 35046 diff changeset	1382 the decoded payload. This mapping will consume memory in proportion
073eec083e25 bundle2: extract logic for seeking bundle2 part into own class Gregory Szorc <gregory.szorc@gmail.com> parents: 35046 diff changeset	1383 to the number of chunks within the payload (which almost certainly
073eec083e25 bundle2: extract logic for seeking bundle2 part into own class Gregory Szorc <gregory.szorc@gmail.com> parents: 35046 diff changeset	1384 increases in proportion with the size of the part).
073eec083e25 bundle2: extract logic for seeking bundle2 part into own class Gregory Szorc <gregory.szorc@gmail.com> parents: 35046 diff changeset	1385 """
073eec083e25 bundle2: extract logic for seeking bundle2 part into own class Gregory Szorc <gregory.szorc@gmail.com> parents: 35046 diff changeset	1386 def __init__(self, ui, header, fp):
073eec083e25 bundle2: extract logic for seeking bundle2 part into own class Gregory Szorc <gregory.szorc@gmail.com> parents: 35046 diff changeset	1387 # (payload, file) offsets for chunk starts.
073eec083e25 bundle2: extract logic for seeking bundle2 part into own class Gregory Szorc <gregory.szorc@gmail.com> parents: 35046 diff changeset	1388 self._chunkindex = []
073eec083e25 bundle2: extract logic for seeking bundle2 part into own class Gregory Szorc <gregory.szorc@gmail.com> parents: 35046 diff changeset	1389
073eec083e25 bundle2: extract logic for seeking bundle2 part into own class Gregory Szorc <gregory.szorc@gmail.com> parents: 35046 diff changeset	1390 super(seekableunbundlepart, self).__init__(ui, header, fp)
073eec083e25 bundle2: extract logic for seeking bundle2 part into own class Gregory Szorc <gregory.szorc@gmail.com> parents: 35046 diff changeset	1391
073eec083e25 bundle2: extract logic for seeking bundle2 part into own class Gregory Szorc <gregory.szorc@gmail.com> parents: 35046 diff changeset	1392 def _payloadchunks(self, chunknum=0):
073eec083e25 bundle2: extract logic for seeking bundle2 part into own class Gregory Szorc <gregory.szorc@gmail.com> parents: 35046 diff changeset	1393 '''seek to specified chunk and start yielding data'''
073eec083e25 bundle2: extract logic for seeking bundle2 part into own class Gregory Szorc <gregory.szorc@gmail.com> parents: 35046 diff changeset	1394 if len(self._chunkindex) == 0:
073eec083e25 bundle2: extract logic for seeking bundle2 part into own class Gregory Szorc <gregory.szorc@gmail.com> parents: 35046 diff changeset	1395 assert chunknum == 0, 'Must start with chunk 0'
073eec083e25 bundle2: extract logic for seeking bundle2 part into own class Gregory Szorc <gregory.szorc@gmail.com> parents: 35046 diff changeset	1396 self._chunkindex.append((0, self._tellfp()))
073eec083e25 bundle2: extract logic for seeking bundle2 part into own class Gregory Szorc <gregory.szorc@gmail.com> parents: 35046 diff changeset	1397 else:
073eec083e25 bundle2: extract logic for seeking bundle2 part into own class Gregory Szorc <gregory.szorc@gmail.com> parents: 35046 diff changeset	1398 assert chunknum < len(self._chunkindex), \
073eec083e25 bundle2: extract logic for seeking bundle2 part into own class Gregory Szorc <gregory.szorc@gmail.com> parents: 35046 diff changeset	1399 'Unknown chunk %d' % chunknum
073eec083e25 bundle2: extract logic for seeking bundle2 part into own class Gregory Szorc <gregory.szorc@gmail.com> parents: 35046 diff changeset	1400 self._seekfp(self._chunkindex[chunknum][1])
073eec083e25 bundle2: extract logic for seeking bundle2 part into own class Gregory Szorc <gregory.szorc@gmail.com> parents: 35046 diff changeset	1401
073eec083e25 bundle2: extract logic for seeking bundle2 part into own class Gregory Szorc <gregory.szorc@gmail.com> parents: 35046 diff changeset	1402 pos = self._chunkindex[chunknum][0]
35113 8aa43ff9c12c bundle2: implement generic part payload decoder Gregory Szorc <gregory.szorc@gmail.com> parents: 35112 diff changeset	1403
8aa43ff9c12c bundle2: implement generic part payload decoder Gregory Szorc <gregory.szorc@gmail.com> parents: 35112 diff changeset	1404 for chunk in decodepayloadchunks(self.ui, self._fp):
8aa43ff9c12c bundle2: implement generic part payload decoder Gregory Szorc <gregory.szorc@gmail.com> parents: 35112 diff changeset	1405 chunknum += 1
8aa43ff9c12c bundle2: implement generic part payload decoder Gregory Szorc <gregory.szorc@gmail.com> parents: 35112 diff changeset	1406 pos += len(chunk)
8aa43ff9c12c bundle2: implement generic part payload decoder Gregory Szorc <gregory.szorc@gmail.com> parents: 35112 diff changeset	1407 if chunknum == len(self._chunkindex):
8aa43ff9c12c bundle2: implement generic part payload decoder Gregory Szorc <gregory.szorc@gmail.com> parents: 35112 diff changeset	1408 self._chunkindex.append((pos, self._tellfp()))
8aa43ff9c12c bundle2: implement generic part payload decoder Gregory Szorc <gregory.szorc@gmail.com> parents: 35112 diff changeset	1409
8aa43ff9c12c bundle2: implement generic part payload decoder Gregory Szorc <gregory.szorc@gmail.com> parents: 35112 diff changeset	1410 yield chunk
35112 073eec083e25 bundle2: extract logic for seeking bundle2 part into own class Gregory Szorc <gregory.szorc@gmail.com> parents: 35046 diff changeset	1411
073eec083e25 bundle2: extract logic for seeking bundle2 part into own class Gregory Szorc <gregory.szorc@gmail.com> parents: 35046 diff changeset	1412 def _findchunk(self, pos):
073eec083e25 bundle2: extract logic for seeking bundle2 part into own class Gregory Szorc <gregory.szorc@gmail.com> parents: 35046 diff changeset	1413 '''for a given payload position, return a chunk number and offset'''
073eec083e25 bundle2: extract logic for seeking bundle2 part into own class Gregory Szorc <gregory.szorc@gmail.com> parents: 35046 diff changeset	1414 for chunk, (ppos, fpos) in enumerate(self._chunkindex):
073eec083e25 bundle2: extract logic for seeking bundle2 part into own class Gregory Szorc <gregory.szorc@gmail.com> parents: 35046 diff changeset	1415 if ppos == pos:
073eec083e25 bundle2: extract logic for seeking bundle2 part into own class Gregory Szorc <gregory.szorc@gmail.com> parents: 35046 diff changeset	1416 return chunk, 0
073eec083e25 bundle2: extract logic for seeking bundle2 part into own class Gregory Szorc <gregory.szorc@gmail.com> parents: 35046 diff changeset	1417 elif ppos > pos:
073eec083e25 bundle2: extract logic for seeking bundle2 part into own class Gregory Szorc <gregory.szorc@gmail.com> parents: 35046 diff changeset	1418 return chunk - 1, pos - self._chunkindex[chunk - 1][0]
073eec083e25 bundle2: extract logic for seeking bundle2 part into own class Gregory Szorc <gregory.szorc@gmail.com> parents: 35046 diff changeset	1419 raise ValueError('Unknown chunk')
073eec083e25 bundle2: extract logic for seeking bundle2 part into own class Gregory Szorc <gregory.szorc@gmail.com> parents: 35046 diff changeset	1420
24036 c7601086338a bundle2.unbundlepart: tell() implementation Eric Sumner <ericsumner@fb.com> parents: 24035 diff changeset	1421 def tell(self):
c7601086338a bundle2.unbundlepart: tell() implementation Eric Sumner <ericsumner@fb.com> parents: 24035 diff changeset	1422 return self._pos
c7601086338a bundle2.unbundlepart: tell() implementation Eric Sumner <ericsumner@fb.com> parents: 24035 diff changeset	1423
35046 241d9caca11e bundle2: use os.SEEK_* constants Gregory Szorc <gregory.szorc@gmail.com> parents: 34820 diff changeset	1424 def seek(self, offset, whence=os.SEEK_SET):
241d9caca11e bundle2: use os.SEEK_* constants Gregory Szorc <gregory.szorc@gmail.com> parents: 34820 diff changeset	1425 if whence == os.SEEK_SET:
24037 f0b498cfc5c8 bundle2.unbundlepart: implement seek() Eric Sumner <ericsumner@fb.com> parents: 24036 diff changeset	1426 newpos = offset
35046 241d9caca11e bundle2: use os.SEEK_* constants Gregory Szorc <gregory.szorc@gmail.com> parents: 34820 diff changeset	1427 elif whence == os.SEEK_CUR:
24037 f0b498cfc5c8 bundle2.unbundlepart: implement seek() Eric Sumner <ericsumner@fb.com> parents: 24036 diff changeset	1428 newpos = self._pos + offset
35046 241d9caca11e bundle2: use os.SEEK_* constants Gregory Szorc <gregory.szorc@gmail.com> parents: 34820 diff changeset	1429 elif whence == os.SEEK_END:
24037 f0b498cfc5c8 bundle2.unbundlepart: implement seek() Eric Sumner <ericsumner@fb.com> parents: 24036 diff changeset	1430 if not self.consumed:
35120 699b2a759319 bundle2: avoid unbound read when seeking Gregory Szorc <gregory.szorc@gmail.com> parents: 35119 diff changeset	1431 # Can't use self.consume() here because it advances self._pos.
699b2a759319 bundle2: avoid unbound read when seeking Gregory Szorc <gregory.szorc@gmail.com> parents: 35119 diff changeset	1432 chunk = self.read(32768)
699b2a759319 bundle2: avoid unbound read when seeking Gregory Szorc <gregory.szorc@gmail.com> parents: 35119 diff changeset	1433 while chunk:
699b2a759319 bundle2: avoid unbound read when seeking Gregory Szorc <gregory.szorc@gmail.com> parents: 35119 diff changeset	1434 chunk = self.read(32768)
24037 f0b498cfc5c8 bundle2.unbundlepart: implement seek() Eric Sumner <ericsumner@fb.com> parents: 24036 diff changeset	1435 newpos = self._chunkindex[-1][0] - offset
f0b498cfc5c8 bundle2.unbundlepart: implement seek() Eric Sumner <ericsumner@fb.com> parents: 24036 diff changeset	1436 else:
f0b498cfc5c8 bundle2.unbundlepart: implement seek() Eric Sumner <ericsumner@fb.com> parents: 24036 diff changeset	1437 raise ValueError('Unknown whence value: %r' % (whence,))
f0b498cfc5c8 bundle2.unbundlepart: implement seek() Eric Sumner <ericsumner@fb.com> parents: 24036 diff changeset	1438
f0b498cfc5c8 bundle2.unbundlepart: implement seek() Eric Sumner <ericsumner@fb.com> parents: 24036 diff changeset	1439 if newpos > self._chunkindex[-1][0] and not self.consumed:
35120 699b2a759319 bundle2: avoid unbound read when seeking Gregory Szorc <gregory.szorc@gmail.com> parents: 35119 diff changeset	1440 # Can't use self.consume() here because it advances self._pos.
699b2a759319 bundle2: avoid unbound read when seeking Gregory Szorc <gregory.szorc@gmail.com> parents: 35119 diff changeset	1441 chunk = self.read(32768)
699b2a759319 bundle2: avoid unbound read when seeking Gregory Szorc <gregory.szorc@gmail.com> parents: 35119 diff changeset	1442 while chunk:
699b2a759319 bundle2: avoid unbound read when seeking Gregory Szorc <gregory.szorc@gmail.com> parents: 35119 diff changeset	1443 chunk = self.read(32668)
699b2a759319 bundle2: avoid unbound read when seeking Gregory Szorc <gregory.szorc@gmail.com> parents: 35119 diff changeset	1444
24037 f0b498cfc5c8 bundle2.unbundlepart: implement seek() Eric Sumner <ericsumner@fb.com> parents: 24036 diff changeset	1445 if not 0 <= newpos <= self._chunkindex[-1][0]:
f0b498cfc5c8 bundle2.unbundlepart: implement seek() Eric Sumner <ericsumner@fb.com> parents: 24036 diff changeset	1446 raise ValueError('Offset out of range')
f0b498cfc5c8 bundle2.unbundlepart: implement seek() Eric Sumner <ericsumner@fb.com> parents: 24036 diff changeset	1447
f0b498cfc5c8 bundle2.unbundlepart: implement seek() Eric Sumner <ericsumner@fb.com> parents: 24036 diff changeset	1448 if self._pos != newpos:
f0b498cfc5c8 bundle2.unbundlepart: implement seek() Eric Sumner <ericsumner@fb.com> parents: 24036 diff changeset	1449 chunk, internaloffset = self._findchunk(newpos)
f0b498cfc5c8 bundle2.unbundlepart: implement seek() Eric Sumner <ericsumner@fb.com> parents: 24036 diff changeset	1450 self._payloadstream = util.chunkbuffer(self._payloadchunks(chunk))
f0b498cfc5c8 bundle2.unbundlepart: implement seek() Eric Sumner <ericsumner@fb.com> parents: 24036 diff changeset	1451 adjust = self.read(internaloffset)
f0b498cfc5c8 bundle2.unbundlepart: implement seek() Eric Sumner <ericsumner@fb.com> parents: 24036 diff changeset	1452 if len(adjust) != internaloffset:
26587 56b2bcea2529 error: get Abort from 'error' instead of 'util' Pierre-Yves David <pierre-yves.david@fb.com> parents: 26565 diff changeset	1453 raise error.Abort(_('Seek failed\n'))
24037 f0b498cfc5c8 bundle2.unbundlepart: implement seek() Eric Sumner <ericsumner@fb.com> parents: 24036 diff changeset	1454 self._pos = newpos
f0b498cfc5c8 bundle2.unbundlepart: implement seek() Eric Sumner <ericsumner@fb.com> parents: 24036 diff changeset	1455
31889 a02e773008f5 bundle2: move 'seek' and 'tell' methods off the unpackermixin class Pierre-Yves David <pierre-yves.david@ens-lyon.org> parents: 31863 diff changeset	1456 def _seekfp(self, offset, whence=0):
a02e773008f5 bundle2: move 'seek' and 'tell' methods off the unpackermixin class Pierre-Yves David <pierre-yves.david@ens-lyon.org> parents: 31863 diff changeset	1457 """move the underlying file pointer
a02e773008f5 bundle2: move 'seek' and 'tell' methods off the unpackermixin class Pierre-Yves David <pierre-yves.david@ens-lyon.org> parents: 31863 diff changeset	1458
a02e773008f5 bundle2: move 'seek' and 'tell' methods off the unpackermixin class Pierre-Yves David <pierre-yves.david@ens-lyon.org> parents: 31863 diff changeset	1459 This method is meant for internal usage by the bundle2 protocol only.
a02e773008f5 bundle2: move 'seek' and 'tell' methods off the unpackermixin class Pierre-Yves David <pierre-yves.david@ens-lyon.org> parents: 31863 diff changeset	1460 They directly manipulate the low level stream including bundle2 level
a02e773008f5 bundle2: move 'seek' and 'tell' methods off the unpackermixin class Pierre-Yves David <pierre-yves.david@ens-lyon.org> parents: 31863 diff changeset	1461 instruction.
a02e773008f5 bundle2: move 'seek' and 'tell' methods off the unpackermixin class Pierre-Yves David <pierre-yves.david@ens-lyon.org> parents: 31863 diff changeset	1462
a02e773008f5 bundle2: move 'seek' and 'tell' methods off the unpackermixin class Pierre-Yves David <pierre-yves.david@ens-lyon.org> parents: 31863 diff changeset	1463 Do not use it to implement higher-level logic or methods."""
a02e773008f5 bundle2: move 'seek' and 'tell' methods off the unpackermixin class Pierre-Yves David <pierre-yves.david@ens-lyon.org> parents: 31863 diff changeset	1464 if self._seekable:
a02e773008f5 bundle2: move 'seek' and 'tell' methods off the unpackermixin class Pierre-Yves David <pierre-yves.david@ens-lyon.org> parents: 31863 diff changeset	1465 return self._fp.seek(offset, whence)
a02e773008f5 bundle2: move 'seek' and 'tell' methods off the unpackermixin class Pierre-Yves David <pierre-yves.david@ens-lyon.org> parents: 31863 diff changeset	1466 else:
a02e773008f5 bundle2: move 'seek' and 'tell' methods off the unpackermixin class Pierre-Yves David <pierre-yves.david@ens-lyon.org> parents: 31863 diff changeset	1467 raise NotImplementedError(_('File pointer is not seekable'))
a02e773008f5 bundle2: move 'seek' and 'tell' methods off the unpackermixin class Pierre-Yves David <pierre-yves.david@ens-lyon.org> parents: 31863 diff changeset	1468
a02e773008f5 bundle2: move 'seek' and 'tell' methods off the unpackermixin class Pierre-Yves David <pierre-yves.david@ens-lyon.org> parents: 31863 diff changeset	1469 def _tellfp(self):
a02e773008f5 bundle2: move 'seek' and 'tell' methods off the unpackermixin class Pierre-Yves David <pierre-yves.david@ens-lyon.org> parents: 31863 diff changeset	1470 """return the file offset, or None if file is not seekable
a02e773008f5 bundle2: move 'seek' and 'tell' methods off the unpackermixin class Pierre-Yves David <pierre-yves.david@ens-lyon.org> parents: 31863 diff changeset	1471
a02e773008f5 bundle2: move 'seek' and 'tell' methods off the unpackermixin class Pierre-Yves David <pierre-yves.david@ens-lyon.org> parents: 31863 diff changeset	1472 This method is meant for internal usage by the bundle2 protocol only.
a02e773008f5 bundle2: move 'seek' and 'tell' methods off the unpackermixin class Pierre-Yves David <pierre-yves.david@ens-lyon.org> parents: 31863 diff changeset	1473 They directly manipulate the low level stream including bundle2 level
a02e773008f5 bundle2: move 'seek' and 'tell' methods off the unpackermixin class Pierre-Yves David <pierre-yves.david@ens-lyon.org> parents: 31863 diff changeset	1474 instruction.
a02e773008f5 bundle2: move 'seek' and 'tell' methods off the unpackermixin class Pierre-Yves David <pierre-yves.david@ens-lyon.org> parents: 31863 diff changeset	1475
a02e773008f5 bundle2: move 'seek' and 'tell' methods off the unpackermixin class Pierre-Yves David <pierre-yves.david@ens-lyon.org> parents: 31863 diff changeset	1476 Do not use it to implement higher-level logic or methods."""
a02e773008f5 bundle2: move 'seek' and 'tell' methods off the unpackermixin class Pierre-Yves David <pierre-yves.david@ens-lyon.org> parents: 31863 diff changeset	1477 if self._seekable:
a02e773008f5 bundle2: move 'seek' and 'tell' methods off the unpackermixin class Pierre-Yves David <pierre-yves.david@ens-lyon.org> parents: 31863 diff changeset	1478 try:
a02e773008f5 bundle2: move 'seek' and 'tell' methods off the unpackermixin class Pierre-Yves David <pierre-yves.david@ens-lyon.org> parents: 31863 diff changeset	1479 return self._fp.tell()
a02e773008f5 bundle2: move 'seek' and 'tell' methods off the unpackermixin class Pierre-Yves David <pierre-yves.david@ens-lyon.org> parents: 31863 diff changeset	1480 except IOError as e:
a02e773008f5 bundle2: move 'seek' and 'tell' methods off the unpackermixin class Pierre-Yves David <pierre-yves.david@ens-lyon.org> parents: 31863 diff changeset	1481 if e.errno == errno.ESPIPE:
a02e773008f5 bundle2: move 'seek' and 'tell' methods off the unpackermixin class Pierre-Yves David <pierre-yves.david@ens-lyon.org> parents: 31863 diff changeset	1482 self._seekable = False
a02e773008f5 bundle2: move 'seek' and 'tell' methods off the unpackermixin class Pierre-Yves David <pierre-yves.david@ens-lyon.org> parents: 31863 diff changeset	1483 else:
a02e773008f5 bundle2: move 'seek' and 'tell' methods off the unpackermixin class Pierre-Yves David <pierre-yves.david@ens-lyon.org> parents: 31863 diff changeset	1484 raise
a02e773008f5 bundle2: move 'seek' and 'tell' methods off the unpackermixin class Pierre-Yves David <pierre-yves.david@ens-lyon.org> parents: 31863 diff changeset	1485 return None
a02e773008f5 bundle2: move 'seek' and 'tell' methods off the unpackermixin class Pierre-Yves David <pierre-yves.david@ens-lyon.org> parents: 31863 diff changeset	1486
25317 5a5b7046d00e bundle2: add an informative comment to the capability dict Pierre-Yves David <pierre-yves.david@fb.com> parents: 25315 diff changeset	1487 # These are only the static capabilities.
5a5b7046d00e bundle2: add an informative comment to the capability dict Pierre-Yves David <pierre-yves.david@fb.com> parents: 25315 diff changeset	1488 # Check the 'getrepocaps' function for the rest.
24686 e0e28e910fa3 bundle2: rename format, parts and config to final names Pierre-Yves David <pierre-yves.david@fb.com> parents: 24648 diff changeset	1489 capabilities = {'HG20': (),
35259 ad5f2b923b0d push: include a 'check:bookmarks' part when possible Boris Feld <boris.feld@octobus.net> parents: 35258 diff changeset	1490 'bookmarks': (),
25493 d8e7b0781ad7 bundle2: convey PushkeyFailed error over the wire Pierre-Yves David <pierre-yves.david@fb.com> parents: 25492 diff changeset	1491 'error': ('abort', 'unsupportedcontent', 'pushraced',
d8e7b0781ad7 bundle2: convey PushkeyFailed error over the wire Pierre-Yves David <pierre-yves.david@fb.com> parents: 25492 diff changeset	1492 'pushkey'),
24686 e0e28e910fa3 bundle2: rename format, parts and config to final names Pierre-Yves David <pierre-yves.david@fb.com> parents: 24648 diff changeset	1493 'listkeys': (),
e0e28e910fa3 bundle2: rename format, parts and config to final names Pierre-Yves David <pierre-yves.david@fb.com> parents: 24648 diff changeset	1494 'pushkey': (),
23029 149fc8a44184 bundle2: client side support for a part to import external bundles Mike Hommey <mh@glandium.org> parents: 23011 diff changeset	1495 'digests': tuple(sorted(util.DIGESTS.keys())),
24686 e0e28e910fa3 bundle2: rename format, parts and config to final names Pierre-Yves David <pierre-yves.david@fb.com> parents: 24648 diff changeset	1496 'remote-changegroup': ('http', 'https'),
25401 d29201352af7 bundle2: part handler for processing .hgtags fnodes mappings Gregory Szorc <gregory.szorc@gmail.com> parents: 25336 diff changeset	1497 'hgtagsfnodes': (),
36968 2090044a288d revbranchcache: advertise and use 'rbc' exchange capability Boris Feld <boris.feld@octobus.net> parents: 36965 diff changeset	1498 'rev-branch-cache': (),
34322 10e162bb9bf5 pull: use 'phase-heads' to retrieve phase information Boris Feld <boris.feld@octobus.net> parents: 34321 diff changeset	1499 'phases': ('heads',),
35763 7eedbd5d4880 streamclone: add support for bundle2 based stream clone Boris Feld <boris.feld@octobus.net> parents: 35758 diff changeset	1500 'stream': ('v2',),
22341 2d16b39601b5 obsmarker: move bundle2caps from the localrepo class to the bundle2 module Pierre-Yves David <pierre-yves.david@fb.com> parents: 22340 diff changeset	1501 }
2d16b39601b5 obsmarker: move bundle2caps from the localrepo class to the bundle2 module Pierre-Yves David <pierre-yves.david@fb.com> parents: 22340 diff changeset	1502
35783 c97639ad6874 bundle2: specify what capabilities will be used for Gregory Szorc <gregory.szorc@gmail.com> parents: 35763 diff changeset	1503 def getrepocaps(repo, allowpushback=False, role=None):
22342 262c5cc126c1 bundle2: introduce a `getrepocaps` to retrieve the bundle2 caps of a repo Pierre-Yves David <pierre-yves.david@fb.com> parents: 22341 diff changeset	1504 """return the bundle2 capabilities for a given repo
262c5cc126c1 bundle2: introduce a `getrepocaps` to retrieve the bundle2 caps of a repo Pierre-Yves David <pierre-yves.david@fb.com> parents: 22341 diff changeset	1505
22343 76677a2c1cfd bundle2: advertise the obsmarker part in bundle2 capabilities Pierre-Yves David <pierre-yves.david@fb.com> parents: 22342 diff changeset	1506 Exists to allow extensions (like evolution) to mutate the capabilities.
35783 c97639ad6874 bundle2: specify what capabilities will be used for Gregory Szorc <gregory.szorc@gmail.com> parents: 35763 diff changeset	1507
c97639ad6874 bundle2: specify what capabilities will be used for Gregory Szorc <gregory.szorc@gmail.com> parents: 35763 diff changeset	1508 The returned value is used for servers advertising their capabilities as
c97639ad6874 bundle2: specify what capabilities will be used for Gregory Szorc <gregory.szorc@gmail.com> parents: 35763 diff changeset	1509 well as clients advertising their capabilities to servers as part of
c97639ad6874 bundle2: specify what capabilities will be used for Gregory Szorc <gregory.szorc@gmail.com> parents: 35763 diff changeset	1510 bundle2 requests. The ``role`` argument specifies which is which.
22342 262c5cc126c1 bundle2: introduce a `getrepocaps` to retrieve the bundle2 caps of a repo Pierre-Yves David <pierre-yves.david@fb.com> parents: 22341 diff changeset	1511 """
35783 c97639ad6874 bundle2: specify what capabilities will be used for Gregory Szorc <gregory.szorc@gmail.com> parents: 35763 diff changeset	1512 if role not in ('client', 'server'):
c97639ad6874 bundle2: specify what capabilities will be used for Gregory Szorc <gregory.szorc@gmail.com> parents: 35763 diff changeset	1513 raise error.ProgrammingError('role argument must be client or server')
c97639ad6874 bundle2: specify what capabilities will be used for Gregory Szorc <gregory.szorc@gmail.com> parents: 35763 diff changeset	1514
22343 76677a2c1cfd bundle2: advertise the obsmarker part in bundle2 capabilities Pierre-Yves David <pierre-yves.david@fb.com> parents: 22342 diff changeset	1515 caps = capabilities.copy()
27953 88609cfa3745 changegroup: fix pulling to treemanifest repo from flat repo (issue5066) Martin von Zweigbergk <martinvonz@google.com> parents: 27879 diff changeset	1516 caps['changegroup'] = tuple(sorted(
88609cfa3745 changegroup: fix pulling to treemanifest repo from flat repo (issue5066) Martin von Zweigbergk <martinvonz@google.com> parents: 27879 diff changeset	1517 changegroup.supportedincomingversions(repo)))
22953 b1d694d3975e obsolete: add exchange option Durham Goode <durham@fb.com> parents: 22661 diff changeset	1518 if obsolete.isenabled(repo, obsolete.exchangeopt):
22343 76677a2c1cfd bundle2: advertise the obsmarker part in bundle2 capabilities Pierre-Yves David <pierre-yves.david@fb.com> parents: 22342 diff changeset	1519 supportedformat = tuple('V%i' % v for v in obsolete.formats)
24686 e0e28e910fa3 bundle2: rename format, parts and config to final names Pierre-Yves David <pierre-yves.david@fb.com> parents: 24648 diff changeset	1520 caps['obsmarkers'] = supportedformat
23439 743736fc7c41 bundle2-push: provide transaction to reply unbundler Eric Sumner <ericsumner@fb.com> parents: 23438 diff changeset	1521 if allowpushback:
24686 e0e28e910fa3 bundle2: rename format, parts and config to final names Pierre-Yves David <pierre-yves.david@fb.com> parents: 24648 diff changeset	1522 caps['pushback'] = ()
33222 593ad8df9dd2 configitems: register the 'server.concurrent-push-mode' config Pierre-Yves David <pierre-yves.david@octobus.net> parents: 33159 diff changeset	1523 cpmode = repo.ui.config('server', 'concurrent-push-mode')
32892 a7851519ea02 check-concurrency: expose the feature as 'concurrent-push-mode' Pierre-Yves David <pierre-yves.david@octobus.net> parents: 32709 diff changeset	1524 if cpmode == 'check-related':
32709 16ada4cbb1a9 push: add a way to allow concurrent pushes on unrelated heads Pierre-Yves David <pierre-yves.david@octobus.net> parents: 32516 diff changeset	1525 caps['checkheads'] = ('related',)
34322 10e162bb9bf5 pull: use 'phase-heads' to retrieve phase information Boris Feld <boris.feld@octobus.net> parents: 34321 diff changeset	1526 if 'phases' in repo.ui.configlist('devel', 'legacy.exchange'):
10e162bb9bf5 pull: use 'phase-heads' to retrieve phase information Boris Feld <boris.feld@octobus.net> parents: 34321 diff changeset	1527 caps.pop('phases')
35790 c1b9eb15a51c bundle2: don't advertise stream bundle2 capability when feature disabled Gregory Szorc <gregory.szorc@gmail.com> parents: 35788 diff changeset	1528
c1b9eb15a51c bundle2: don't advertise stream bundle2 capability when feature disabled Gregory Szorc <gregory.szorc@gmail.com> parents: 35788 diff changeset	1529 # Don't advertise stream clone support in server mode if not configured.
c1b9eb15a51c bundle2: don't advertise stream bundle2 capability when feature disabled Gregory Szorc <gregory.szorc@gmail.com> parents: 35788 diff changeset	1530 if role == 'server':
c1b9eb15a51c bundle2: don't advertise stream bundle2 capability when feature disabled Gregory Szorc <gregory.szorc@gmail.com> parents: 35788 diff changeset	1531 streamsupported = repo.ui.configbool('server', 'uncompressed',
c1b9eb15a51c bundle2: don't advertise stream bundle2 capability when feature disabled Gregory Szorc <gregory.szorc@gmail.com> parents: 35788 diff changeset	1532 untrusted=True)
c1b9eb15a51c bundle2: don't advertise stream bundle2 capability when feature disabled Gregory Szorc <gregory.szorc@gmail.com> parents: 35788 diff changeset	1533 featuresupported = repo.ui.configbool('experimental', 'bundle2.stream')
c1b9eb15a51c bundle2: don't advertise stream bundle2 capability when feature disabled Gregory Szorc <gregory.szorc@gmail.com> parents: 35788 diff changeset	1534
c1b9eb15a51c bundle2: don't advertise stream bundle2 capability when feature disabled Gregory Szorc <gregory.szorc@gmail.com> parents: 35788 diff changeset	1535 if not streamsupported or not featuresupported:
c1b9eb15a51c bundle2: don't advertise stream bundle2 capability when feature disabled Gregory Szorc <gregory.szorc@gmail.com> parents: 35788 diff changeset	1536 caps.pop('stream')
35792 1d118f9f4f57 bundle2: always advertise client support for stream parts Gregory Szorc <gregory.szorc@gmail.com> parents: 35790 diff changeset	1537 # Else always advertise support on client, because payload support
1d118f9f4f57 bundle2: always advertise client support for stream parts Gregory Szorc <gregory.szorc@gmail.com> parents: 35790 diff changeset	1538 # should always be advertised.
35790 c1b9eb15a51c bundle2: don't advertise stream bundle2 capability when feature disabled Gregory Szorc <gregory.szorc@gmail.com> parents: 35788 diff changeset	1539
22343 76677a2c1cfd bundle2: advertise the obsmarker part in bundle2 capabilities Pierre-Yves David <pierre-yves.david@fb.com> parents: 22342 diff changeset	1540 return caps
22342 262c5cc126c1 bundle2: introduce a `getrepocaps` to retrieve the bundle2 caps of a repo Pierre-Yves David <pierre-yves.david@fb.com> parents: 22341 diff changeset	1541
21644 17755dd8c509 bundle2: introduce a bundle2caps function Pierre-Yves David <pierre-yves.david@fb.com> parents: 21628 diff changeset	1542 def bundle2caps(remote):
23139 e53f6b72a0e4 spelling: fixes from proofreading of spell checker issues Mads Kiilerich <madski@unity3d.com> parents: 23067 diff changeset	1543 """return the bundle capabilities of a peer as dict"""
24686 e0e28e910fa3 bundle2: rename format, parts and config to final names Pierre-Yves David <pierre-yves.david@fb.com> parents: 24648 diff changeset	1544 raw = remote.capable('bundle2')
21644 17755dd8c509 bundle2: introduce a bundle2caps function Pierre-Yves David <pierre-yves.david@fb.com> parents: 21628 diff changeset	1545 if not raw and raw != '':
17755dd8c509 bundle2: introduce a bundle2caps function Pierre-Yves David <pierre-yves.david@fb.com> parents: 21628 diff changeset	1546 return {}
28883 032c4c2f802a pycompat: switch to util.urlreq/util.urlerr for py3 compat timeless <timeless@mozdev.org> parents: 28672 diff changeset	1547 capsblob = urlreq.unquote(remote.capable('bundle2'))
21644 17755dd8c509 bundle2: introduce a bundle2caps function Pierre-Yves David <pierre-yves.david@fb.com> parents: 21628 diff changeset	1548 return decodecaps(capsblob)
21014 a6246bba7b9e bundle2: add an unbundle part responsible from unbundling part Pierre-Yves David <pierre-yves.david@fb.com> parents: 21013 diff changeset	1549
22344 9829b7948100 bundle2: add a `obsmarkersversion` function to extract supported version Pierre-Yves David <pierre-yves.david@fb.com> parents: 22343 diff changeset	1550 def obsmarkersversion(caps):
9829b7948100 bundle2: add a `obsmarkersversion` function to extract supported version Pierre-Yves David <pierre-yves.david@fb.com> parents: 22343 diff changeset	1551 """extract the list of supported obsmarkers versions from a bundle2caps dict
9829b7948100 bundle2: add a `obsmarkersversion` function to extract supported version Pierre-Yves David <pierre-yves.david@fb.com> parents: 22343 diff changeset	1552 """
24686 e0e28e910fa3 bundle2: rename format, parts and config to final names Pierre-Yves David <pierre-yves.david@fb.com> parents: 24648 diff changeset	1553 obscaps = caps.get('obsmarkers', ())
22344 9829b7948100 bundle2: add a `obsmarkersversion` function to extract supported version Pierre-Yves David <pierre-yves.david@fb.com> parents: 22343 diff changeset	1554 return [int(c[1:]) for c in obscaps if c.startswith('V')]
9829b7948100 bundle2: add a `obsmarkersversion` function to extract supported version Pierre-Yves David <pierre-yves.david@fb.com> parents: 22343 diff changeset	1555
32221 9dc36df78403 bundle: introduce an higher level function to write bundle on disk Pierre-Yves David <pierre-yves.david@octobus.net> parents: 32186 diff changeset	1556 def writenewbundle(ui, repo, source, filename, bundletype, outgoing, opts,
9dc36df78403 bundle: introduce an higher level function to write bundle on disk Pierre-Yves David <pierre-yves.david@octobus.net> parents: 32186 diff changeset	1557 vfs=None, compression=None, compopts=None):
9dc36df78403 bundle: introduce an higher level function to write bundle on disk Pierre-Yves David <pierre-yves.david@octobus.net> parents: 32186 diff changeset	1558 if bundletype.startswith('HG10'):
34101 5ede882c249c changegroup: replace getchangegroup with makechangegroup Durham Goode <durham@fb.com> parents: 33888 diff changeset	1559 cg = changegroup.makechangegroup(repo, outgoing, '01', source)
32221 9dc36df78403 bundle: introduce an higher level function to write bundle on disk Pierre-Yves David <pierre-yves.david@octobus.net> parents: 32186 diff changeset	1560 return writebundle(ui, cg, filename, bundletype, vfs=vfs,
9dc36df78403 bundle: introduce an higher level function to write bundle on disk Pierre-Yves David <pierre-yves.david@octobus.net> parents: 32186 diff changeset	1561 compression=compression, compopts=compopts)
9dc36df78403 bundle: introduce an higher level function to write bundle on disk Pierre-Yves David <pierre-yves.david@octobus.net> parents: 32186 diff changeset	1562 elif not bundletype.startswith('HG20'):
9dc36df78403 bundle: introduce an higher level function to write bundle on disk Pierre-Yves David <pierre-yves.david@octobus.net> parents: 32186 diff changeset	1563 raise error.ProgrammingError('unknown bundle type: %s' % bundletype)
9dc36df78403 bundle: introduce an higher level function to write bundle on disk Pierre-Yves David <pierre-yves.david@octobus.net> parents: 32186 diff changeset	1564
32516 37d70ba1d9d1 bundle: add an experimental knob to include obsmarkers in bundle Pierre-Yves David <pierre-yves.david@octobus.net> parents: 32515 diff changeset	1565 caps = {}
37d70ba1d9d1 bundle: add an experimental knob to include obsmarkers in bundle Pierre-Yves David <pierre-yves.david@octobus.net> parents: 32515 diff changeset	1566 if 'obsolescence' in opts:
37d70ba1d9d1 bundle: add an experimental knob to include obsmarkers in bundle Pierre-Yves David <pierre-yves.david@octobus.net> parents: 32515 diff changeset	1567 caps['obsmarkers'] = ('V1',)
37d70ba1d9d1 bundle: add an experimental knob to include obsmarkers in bundle Pierre-Yves David <pierre-yves.david@octobus.net> parents: 32515 diff changeset	1568 bundle = bundle20(ui, caps)
32221 9dc36df78403 bundle: introduce an higher level function to write bundle on disk Pierre-Yves David <pierre-yves.david@octobus.net> parents: 32186 diff changeset	1569 bundle.setcompression(compression, compopts)
9dc36df78403 bundle: introduce an higher level function to write bundle on disk Pierre-Yves David <pierre-yves.david@octobus.net> parents: 32186 diff changeset	1570 _addpartsfromopts(ui, repo, bundle, source, outgoing, opts)
9dc36df78403 bundle: introduce an higher level function to write bundle on disk Pierre-Yves David <pierre-yves.david@octobus.net> parents: 32186 diff changeset	1571 chunkiter = bundle.getchunks()
9dc36df78403 bundle: introduce an higher level function to write bundle on disk Pierre-Yves David <pierre-yves.david@octobus.net> parents: 32186 diff changeset	1572
9dc36df78403 bundle: introduce an higher level function to write bundle on disk Pierre-Yves David <pierre-yves.david@octobus.net> parents: 32186 diff changeset	1573 return changegroup.writechunks(ui, chunkiter, filename, vfs=vfs)
9dc36df78403 bundle: introduce an higher level function to write bundle on disk Pierre-Yves David <pierre-yves.david@octobus.net> parents: 32186 diff changeset	1574
9dc36df78403 bundle: introduce an higher level function to write bundle on disk Pierre-Yves David <pierre-yves.david@octobus.net> parents: 32186 diff changeset	1575 def _addpartsfromopts(ui, repo, bundler, source, outgoing, opts):
9dc36df78403 bundle: introduce an higher level function to write bundle on disk Pierre-Yves David <pierre-yves.david@octobus.net> parents: 32186 diff changeset	1576 # We should eventually reconcile this logic with the one behind
9dc36df78403 bundle: introduce an higher level function to write bundle on disk Pierre-Yves David <pierre-yves.david@octobus.net> parents: 32186 diff changeset	1577 # 'exchange.getbundle2partsgenerator'.
9dc36df78403 bundle: introduce an higher level function to write bundle on disk Pierre-Yves David <pierre-yves.david@octobus.net> parents: 32186 diff changeset	1578 #
9dc36df78403 bundle: introduce an higher level function to write bundle on disk Pierre-Yves David <pierre-yves.david@octobus.net> parents: 32186 diff changeset	1579 # The type of input from 'getbundle' and 'writenewbundle' are a bit
9dc36df78403 bundle: introduce an higher level function to write bundle on disk Pierre-Yves David <pierre-yves.david@octobus.net> parents: 32186 diff changeset	1580 # different right now. So we keep them separated for now for the sake of
9dc36df78403 bundle: introduce an higher level function to write bundle on disk Pierre-Yves David <pierre-yves.david@octobus.net> parents: 32186 diff changeset	1581 # simplicity.
9dc36df78403 bundle: introduce an higher level function to write bundle on disk Pierre-Yves David <pierre-yves.david@octobus.net> parents: 32186 diff changeset	1582
37005 66c0ff381cfc bundle: condition the changegroup part when creating a new bundle Boris Feld <boris.feld@octobus.net> parents: 36968 diff changeset	1583 # we might not always want a changegroup in such bundle, for example in
66c0ff381cfc bundle: condition the changegroup part when creating a new bundle Boris Feld <boris.feld@octobus.net> parents: 36968 diff changeset	1584 # stream bundles
66c0ff381cfc bundle: condition the changegroup part when creating a new bundle Boris Feld <boris.feld@octobus.net> parents: 36968 diff changeset	1585 if opts.get('changegroup', True):
66c0ff381cfc bundle: condition the changegroup part when creating a new bundle Boris Feld <boris.feld@octobus.net> parents: 36968 diff changeset	1586 cgversion = opts.get('cg.version')
66c0ff381cfc bundle: condition the changegroup part when creating a new bundle Boris Feld <boris.feld@octobus.net> parents: 36968 diff changeset	1587 if cgversion is None:
66c0ff381cfc bundle: condition the changegroup part when creating a new bundle Boris Feld <boris.feld@octobus.net> parents: 36968 diff changeset	1588 cgversion = changegroup.safeversion(repo)
66c0ff381cfc bundle: condition the changegroup part when creating a new bundle Boris Feld <boris.feld@octobus.net> parents: 36968 diff changeset	1589 cg = changegroup.makechangegroup(repo, outgoing, cgversion, source)
66c0ff381cfc bundle: condition the changegroup part when creating a new bundle Boris Feld <boris.feld@octobus.net> parents: 36968 diff changeset	1590 part = bundler.newpart('changegroup', data=cg.getchunks())
66c0ff381cfc bundle: condition the changegroup part when creating a new bundle Boris Feld <boris.feld@octobus.net> parents: 36968 diff changeset	1591 part.addparam('version', cg.version)
66c0ff381cfc bundle: condition the changegroup part when creating a new bundle Boris Feld <boris.feld@octobus.net> parents: 36968 diff changeset	1592 if 'clcount' in cg.extras:
66c0ff381cfc bundle: condition the changegroup part when creating a new bundle Boris Feld <boris.feld@octobus.net> parents: 36968 diff changeset	1593 part.addparam('nbchanges', '%d' % cg.extras['clcount'],
66c0ff381cfc bundle: condition the changegroup part when creating a new bundle Boris Feld <boris.feld@octobus.net> parents: 36968 diff changeset	1594 mandatory=False)
66c0ff381cfc bundle: condition the changegroup part when creating a new bundle Boris Feld <boris.feld@octobus.net> parents: 36968 diff changeset	1595 if opts.get('phases') and repo.revs('%ln and secret()',
66c0ff381cfc bundle: condition the changegroup part when creating a new bundle Boris Feld <boris.feld@octobus.net> parents: 36968 diff changeset	1596 outgoing.missingheads):
66c0ff381cfc bundle: condition the changegroup part when creating a new bundle Boris Feld <boris.feld@octobus.net> parents: 36968 diff changeset	1597 part.addparam('targetphase', '%d' % phases.secret, mandatory=False)
32221 9dc36df78403 bundle: introduce an higher level function to write bundle on disk Pierre-Yves David <pierre-yves.david@octobus.net> parents: 32186 diff changeset	1598
37167 6f467adf9f05 bundle: add the possibility to bundle a stream v2 part Boris Feld <boris.feld@octobus.net> parents: 37165 diff changeset	1599 if opts.get('streamv2', False):
6f467adf9f05 bundle: add the possibility to bundle a stream v2 part Boris Feld <boris.feld@octobus.net> parents: 37165 diff changeset	1600 addpartbundlestream2(bundler, repo, stream=True)
6f467adf9f05 bundle: add the possibility to bundle a stream v2 part Boris Feld <boris.feld@octobus.net> parents: 37165 diff changeset	1601
37165 6c7a6b04b274 bundlespec: move computing the bundle contentops in parsebundlespec Boris Feld <boris.feld@octobus.net> parents: 37084 diff changeset	1602 if opts.get('tagsfnodescache', True):
6c7a6b04b274 bundlespec: move computing the bundle contentops in parsebundlespec Boris Feld <boris.feld@octobus.net> parents: 37084 diff changeset	1603 addparttagsfnodescache(repo, bundler, outgoing)
6c7a6b04b274 bundlespec: move computing the bundle contentops in parsebundlespec Boris Feld <boris.feld@octobus.net> parents: 37084 diff changeset	1604
6c7a6b04b274 bundlespec: move computing the bundle contentops in parsebundlespec Boris Feld <boris.feld@octobus.net> parents: 37084 diff changeset	1605 if opts.get('revbranchcache', True):
6c7a6b04b274 bundlespec: move computing the bundle contentops in parsebundlespec Boris Feld <boris.feld@octobus.net> parents: 37084 diff changeset	1606 addpartrevbranchcache(repo, bundler, outgoing)
32223 d7f93ebbbbdf bundle: add optional 'tagsfnodecache' data to on disk bundle (issue5543) Pierre-Yves David <pierre-yves.david@octobus.net> parents: 32222 diff changeset	1607
32516 37d70ba1d9d1 bundle: add an experimental knob to include obsmarkers in bundle Pierre-Yves David <pierre-yves.david@octobus.net> parents: 32515 diff changeset	1608 if opts.get('obsolescence', False):
37d70ba1d9d1 bundle: add an experimental knob to include obsmarkers in bundle Pierre-Yves David <pierre-yves.david@octobus.net> parents: 32515 diff changeset	1609 obsmarkers = repo.obsstore.relevantmarkers(outgoing.missing)
37d70ba1d9d1 bundle: add an experimental knob to include obsmarkers in bundle Pierre-Yves David <pierre-yves.david@octobus.net> parents: 32515 diff changeset	1610 buildobsmarkerspart(bundler, obsmarkers)
37d70ba1d9d1 bundle: add an experimental knob to include obsmarkers in bundle Pierre-Yves David <pierre-yves.david@octobus.net> parents: 32515 diff changeset	1611
33031 e8c8d81eb864 bundle: add config option to include phases Martin von Zweigbergk <martinvonz@google.com> parents: 33030 diff changeset	1612 if opts.get('phases', False):
e8c8d81eb864 bundle: add config option to include phases Martin von Zweigbergk <martinvonz@google.com> parents: 33030 diff changeset	1613 headsbyphase = phases.subsetphaseheads(repo, outgoing.missing)
34319 5779d096a696 phases: move binary encoding into a reusable function Boris Feld <boris.feld@octobus.net> parents: 34318 diff changeset	1614 phasedata = phases.binaryencode(headsbyphase)
5779d096a696 phases: move binary encoding into a reusable function Boris Feld <boris.feld@octobus.net> parents: 34318 diff changeset	1615 bundler.newpart('phase-heads', data=phasedata)
33031 e8c8d81eb864 bundle: add config option to include phases Martin von Zweigbergk <martinvonz@google.com> parents: 33030 diff changeset	1616
32222 6068712cbf03 bundle2: move tagsfnodecache generation in a generic function Pierre-Yves David <pierre-yves.david@octobus.net> parents: 32221 diff changeset	1617 def addparttagsfnodescache(repo, bundler, outgoing):
6068712cbf03 bundle2: move tagsfnodecache generation in a generic function Pierre-Yves David <pierre-yves.david@octobus.net> parents: 32221 diff changeset	1618 # we include the tags fnode cache for the bundle changeset
6068712cbf03 bundle2: move tagsfnodecache generation in a generic function Pierre-Yves David <pierre-yves.david@octobus.net> parents: 32221 diff changeset	1619 # (as an optional parts)
6068712cbf03 bundle2: move tagsfnodecache generation in a generic function Pierre-Yves David <pierre-yves.david@octobus.net> parents: 32221 diff changeset	1620 cache = tags.hgtagsfnodescache(repo.unfiltered())
6068712cbf03 bundle2: move tagsfnodecache generation in a generic function Pierre-Yves David <pierre-yves.david@octobus.net> parents: 32221 diff changeset	1621 chunks = []
6068712cbf03 bundle2: move tagsfnodecache generation in a generic function Pierre-Yves David <pierre-yves.david@octobus.net> parents: 32221 diff changeset	1622
6068712cbf03 bundle2: move tagsfnodecache generation in a generic function Pierre-Yves David <pierre-yves.david@octobus.net> parents: 32221 diff changeset	1623 # .hgtags fnodes are only relevant for head changesets. While we could
6068712cbf03 bundle2: move tagsfnodecache generation in a generic function Pierre-Yves David <pierre-yves.david@octobus.net> parents: 32221 diff changeset	1624 # transfer values for all known nodes, there will likely be little to
6068712cbf03 bundle2: move tagsfnodecache generation in a generic function Pierre-Yves David <pierre-yves.david@octobus.net> parents: 32221 diff changeset	1625 # no benefit.
6068712cbf03 bundle2: move tagsfnodecache generation in a generic function Pierre-Yves David <pierre-yves.david@octobus.net> parents: 32221 diff changeset	1626 #
6068712cbf03 bundle2: move tagsfnodecache generation in a generic function Pierre-Yves David <pierre-yves.david@octobus.net> parents: 32221 diff changeset	1627 # We don't bother using a generator to produce output data because
6068712cbf03 bundle2: move tagsfnodecache generation in a generic function Pierre-Yves David <pierre-yves.david@octobus.net> parents: 32221 diff changeset	1628 # a) we only have 40 bytes per head and even esoteric numbers of heads
6068712cbf03 bundle2: move tagsfnodecache generation in a generic function Pierre-Yves David <pierre-yves.david@octobus.net> parents: 32221 diff changeset	1629 # consume little memory (1M heads is 40MB) b) we don't want to send the
6068712cbf03 bundle2: move tagsfnodecache generation in a generic function Pierre-Yves David <pierre-yves.david@octobus.net> parents: 32221 diff changeset	1630 # part if we don't have entries and knowing if we have entries requires
6068712cbf03 bundle2: move tagsfnodecache generation in a generic function Pierre-Yves David <pierre-yves.david@octobus.net> parents: 32221 diff changeset	1631 # cache lookups.
6068712cbf03 bundle2: move tagsfnodecache generation in a generic function Pierre-Yves David <pierre-yves.david@octobus.net> parents: 32221 diff changeset	1632 for node in outgoing.missingheads:
6068712cbf03 bundle2: move tagsfnodecache generation in a generic function Pierre-Yves David <pierre-yves.david@octobus.net> parents: 32221 diff changeset	1633 # Don't compute missing, as this may slow down serving.
6068712cbf03 bundle2: move tagsfnodecache generation in a generic function Pierre-Yves David <pierre-yves.david@octobus.net> parents: 32221 diff changeset	1634 fnode = cache.getfnode(node, computemissing=False)
6068712cbf03 bundle2: move tagsfnodecache generation in a generic function Pierre-Yves David <pierre-yves.david@octobus.net> parents: 32221 diff changeset	1635 if fnode is not None:
6068712cbf03 bundle2: move tagsfnodecache generation in a generic function Pierre-Yves David <pierre-yves.david@octobus.net> parents: 32221 diff changeset	1636 chunks.extend([node, fnode])
6068712cbf03 bundle2: move tagsfnodecache generation in a generic function Pierre-Yves David <pierre-yves.david@octobus.net> parents: 32221 diff changeset	1637
6068712cbf03 bundle2: move tagsfnodecache generation in a generic function Pierre-Yves David <pierre-yves.david@octobus.net> parents: 32221 diff changeset	1638 if chunks:
6068712cbf03 bundle2: move tagsfnodecache generation in a generic function Pierre-Yves David <pierre-yves.david@octobus.net> parents: 32221 diff changeset	1639 bundler.newpart('hgtagsfnodes', data=''.join(chunks))
6068712cbf03 bundle2: move tagsfnodecache generation in a generic function Pierre-Yves David <pierre-yves.david@octobus.net> parents: 32221 diff changeset	1640
36964 79b73be4dda5 rev-branch-cache: add a function to generate a part Boris Feld <boris.feld@octobus.net> parents: 36963 diff changeset	1641 def addpartrevbranchcache(repo, bundler, outgoing):
79b73be4dda5 rev-branch-cache: add a function to generate a part Boris Feld <boris.feld@octobus.net> parents: 36963 diff changeset	1642 # we include the rev branch cache for the bundle changeset
79b73be4dda5 rev-branch-cache: add a function to generate a part Boris Feld <boris.feld@octobus.net> parents: 36963 diff changeset	1643 # (as an optional parts)
79b73be4dda5 rev-branch-cache: add a function to generate a part Boris Feld <boris.feld@octobus.net> parents: 36963 diff changeset	1644 cache = repo.revbranchcache()
79b73be4dda5 rev-branch-cache: add a function to generate a part Boris Feld <boris.feld@octobus.net> parents: 36963 diff changeset	1645 cl = repo.unfiltered().changelog
79b73be4dda5 rev-branch-cache: add a function to generate a part Boris Feld <boris.feld@octobus.net> parents: 36963 diff changeset	1646 branchesdata = collections.defaultdict(lambda: (set(), set()))
79b73be4dda5 rev-branch-cache: add a function to generate a part Boris Feld <boris.feld@octobus.net> parents: 36963 diff changeset	1647 for node in outgoing.missing:
79b73be4dda5 rev-branch-cache: add a function to generate a part Boris Feld <boris.feld@octobus.net> parents: 36963 diff changeset	1648 branch, close = cache.branchinfo(cl.rev(node))
79b73be4dda5 rev-branch-cache: add a function to generate a part Boris Feld <boris.feld@octobus.net> parents: 36963 diff changeset	1649 branchesdata[branch][close].add(node)
79b73be4dda5 rev-branch-cache: add a function to generate a part Boris Feld <boris.feld@octobus.net> parents: 36963 diff changeset	1650
79b73be4dda5 rev-branch-cache: add a function to generate a part Boris Feld <boris.feld@octobus.net> parents: 36963 diff changeset	1651 def generate():
79b73be4dda5 rev-branch-cache: add a function to generate a part Boris Feld <boris.feld@octobus.net> parents: 36963 diff changeset	1652 for branch, (nodes, closed) in sorted(branchesdata.items()):
79b73be4dda5 rev-branch-cache: add a function to generate a part Boris Feld <boris.feld@octobus.net> parents: 36963 diff changeset	1653 utf8branch = encoding.fromlocal(branch)
79b73be4dda5 rev-branch-cache: add a function to generate a part Boris Feld <boris.feld@octobus.net> parents: 36963 diff changeset	1654 yield rbcstruct.pack(len(utf8branch), len(nodes), len(closed))
79b73be4dda5 rev-branch-cache: add a function to generate a part Boris Feld <boris.feld@octobus.net> parents: 36963 diff changeset	1655 yield utf8branch
79b73be4dda5 rev-branch-cache: add a function to generate a part Boris Feld <boris.feld@octobus.net> parents: 36963 diff changeset	1656 for n in sorted(nodes):
79b73be4dda5 rev-branch-cache: add a function to generate a part Boris Feld <boris.feld@octobus.net> parents: 36963 diff changeset	1657 yield n
79b73be4dda5 rev-branch-cache: add a function to generate a part Boris Feld <boris.feld@octobus.net> parents: 36963 diff changeset	1658 for n in sorted(closed):
79b73be4dda5 rev-branch-cache: add a function to generate a part Boris Feld <boris.feld@octobus.net> parents: 36963 diff changeset	1659 yield n
79b73be4dda5 rev-branch-cache: add a function to generate a part Boris Feld <boris.feld@octobus.net> parents: 36963 diff changeset	1660
79b73be4dda5 rev-branch-cache: add a function to generate a part Boris Feld <boris.feld@octobus.net> parents: 36963 diff changeset	1661 bundler.newpart('cache:rev-branch-cache', data=generate())
79b73be4dda5 rev-branch-cache: add a function to generate a part Boris Feld <boris.feld@octobus.net> parents: 36963 diff changeset	1662
37167 6f467adf9f05 bundle: add the possibility to bundle a stream v2 part Boris Feld <boris.feld@octobus.net> parents: 37165 diff changeset	1663 def _formatrequirementsspec(requirements):
6f467adf9f05 bundle: add the possibility to bundle a stream v2 part Boris Feld <boris.feld@octobus.net> parents: 37165 diff changeset	1664 return urlreq.quote(','.join(sorted(requirements)))
6f467adf9f05 bundle: add the possibility to bundle a stream v2 part Boris Feld <boris.feld@octobus.net> parents: 37165 diff changeset	1665
6f467adf9f05 bundle: add the possibility to bundle a stream v2 part Boris Feld <boris.feld@octobus.net> parents: 37165 diff changeset	1666 def _formatrequirementsparams(requirements):
6f467adf9f05 bundle: add the possibility to bundle a stream v2 part Boris Feld <boris.feld@octobus.net> parents: 37165 diff changeset	1667 requirements = _formatrequirementsspec(requirements)
6f467adf9f05 bundle: add the possibility to bundle a stream v2 part Boris Feld <boris.feld@octobus.net> parents: 37165 diff changeset	1668 params = "%s%s" % (urlreq.quote("requirements="), requirements)
6f467adf9f05 bundle: add the possibility to bundle a stream v2 part Boris Feld <boris.feld@octobus.net> parents: 37165 diff changeset	1669 return params
6f467adf9f05 bundle: add the possibility to bundle a stream v2 part Boris Feld <boris.feld@octobus.net> parents: 37165 diff changeset	1670
6f467adf9f05 bundle: add the possibility to bundle a stream v2 part Boris Feld <boris.feld@octobus.net> parents: 37165 diff changeset	1671 def addpartbundlestream2(bundler, repo, **kwargs):
6f467adf9f05 bundle: add the possibility to bundle a stream v2 part Boris Feld <boris.feld@octobus.net> parents: 37165 diff changeset	1672 if not kwargs.get('stream', False):
6f467adf9f05 bundle: add the possibility to bundle a stream v2 part Boris Feld <boris.feld@octobus.net> parents: 37165 diff changeset	1673 return
6f467adf9f05 bundle: add the possibility to bundle a stream v2 part Boris Feld <boris.feld@octobus.net> parents: 37165 diff changeset	1674
6f467adf9f05 bundle: add the possibility to bundle a stream v2 part Boris Feld <boris.feld@octobus.net> parents: 37165 diff changeset	1675 if not streamclone.allowservergeneration(repo):
6f467adf9f05 bundle: add the possibility to bundle a stream v2 part Boris Feld <boris.feld@octobus.net> parents: 37165 diff changeset	1676 raise error.Abort(_('stream data requested but server does not allow '
6f467adf9f05 bundle: add the possibility to bundle a stream v2 part Boris Feld <boris.feld@octobus.net> parents: 37165 diff changeset	1677 'this feature'),
6f467adf9f05 bundle: add the possibility to bundle a stream v2 part Boris Feld <boris.feld@octobus.net> parents: 37165 diff changeset	1678 hint=_('well-behaved clients should not be '
6f467adf9f05 bundle: add the possibility to bundle a stream v2 part Boris Feld <boris.feld@octobus.net> parents: 37165 diff changeset	1679 'requesting stream data from servers not '
6f467adf9f05 bundle: add the possibility to bundle a stream v2 part Boris Feld <boris.feld@octobus.net> parents: 37165 diff changeset	1680 'advertising it; the client may be buggy'))
6f467adf9f05 bundle: add the possibility to bundle a stream v2 part Boris Feld <boris.feld@octobus.net> parents: 37165 diff changeset	1681
6f467adf9f05 bundle: add the possibility to bundle a stream v2 part Boris Feld <boris.feld@octobus.net> parents: 37165 diff changeset	1682 # Stream clones don't compress well. And compression undermines a
6f467adf9f05 bundle: add the possibility to bundle a stream v2 part Boris Feld <boris.feld@octobus.net> parents: 37165 diff changeset	1683 # goal of stream clones, which is to be fast. Communicate the desire
6f467adf9f05 bundle: add the possibility to bundle a stream v2 part Boris Feld <boris.feld@octobus.net> parents: 37165 diff changeset	1684 # to avoid compression to consumers of the bundle.
6f467adf9f05 bundle: add the possibility to bundle a stream v2 part Boris Feld <boris.feld@octobus.net> parents: 37165 diff changeset	1685 bundler.prefercompressed = False
6f467adf9f05 bundle: add the possibility to bundle a stream v2 part Boris Feld <boris.feld@octobus.net> parents: 37165 diff changeset	1686
6f467adf9f05 bundle: add the possibility to bundle a stream v2 part Boris Feld <boris.feld@octobus.net> parents: 37165 diff changeset	1687 filecount, bytecount, it = streamclone.generatev2(repo)
6f467adf9f05 bundle: add the possibility to bundle a stream v2 part Boris Feld <boris.feld@octobus.net> parents: 37165 diff changeset	1688 requirements = _formatrequirementsspec(repo.requirements)
6f467adf9f05 bundle: add the possibility to bundle a stream v2 part Boris Feld <boris.feld@octobus.net> parents: 37165 diff changeset	1689 part = bundler.newpart('stream2', data=it)
6f467adf9f05 bundle: add the possibility to bundle a stream v2 part Boris Feld <boris.feld@octobus.net> parents: 37165 diff changeset	1690 part.addparam('bytecount', '%d' % bytecount, mandatory=True)
6f467adf9f05 bundle: add the possibility to bundle a stream v2 part Boris Feld <boris.feld@octobus.net> parents: 37165 diff changeset	1691 part.addparam('filecount', '%d' % filecount, mandatory=True)
6f467adf9f05 bundle: add the possibility to bundle a stream v2 part Boris Feld <boris.feld@octobus.net> parents: 37165 diff changeset	1692 part.addparam('requirements', requirements, mandatory=True)
6f467adf9f05 bundle: add the possibility to bundle a stream v2 part Boris Feld <boris.feld@octobus.net> parents: 37165 diff changeset	1693
32515 e70d6dbde713 bundle2: move function building obsmarker-part in the bundle2 module Pierre-Yves David <pierre-yves.david@octobus.net> parents: 32387 diff changeset	1694 def buildobsmarkerspart(bundler, markers):
e70d6dbde713 bundle2: move function building obsmarker-part in the bundle2 module Pierre-Yves David <pierre-yves.david@octobus.net> parents: 32387 diff changeset	1695 """add an obsmarker part to the bundler with <markers>
e70d6dbde713 bundle2: move function building obsmarker-part in the bundle2 module Pierre-Yves David <pierre-yves.david@octobus.net> parents: 32387 diff changeset	1696
e70d6dbde713 bundle2: move function building obsmarker-part in the bundle2 module Pierre-Yves David <pierre-yves.david@octobus.net> parents: 32387 diff changeset	1697 No part is created if markers is empty.
e70d6dbde713 bundle2: move function building obsmarker-part in the bundle2 module Pierre-Yves David <pierre-yves.david@octobus.net> parents: 32387 diff changeset	1698 Raises ValueError if the bundler doesn't support any known obsmarker format.
e70d6dbde713 bundle2: move function building obsmarker-part in the bundle2 module Pierre-Yves David <pierre-yves.david@octobus.net> parents: 32387 diff changeset	1699 """
e70d6dbde713 bundle2: move function building obsmarker-part in the bundle2 module Pierre-Yves David <pierre-yves.david@octobus.net> parents: 32387 diff changeset	1700 if not markers:
e70d6dbde713 bundle2: move function building obsmarker-part in the bundle2 module Pierre-Yves David <pierre-yves.david@octobus.net> parents: 32387 diff changeset	1701 return None
e70d6dbde713 bundle2: move function building obsmarker-part in the bundle2 module Pierre-Yves David <pierre-yves.david@octobus.net> parents: 32387 diff changeset	1702
e70d6dbde713 bundle2: move function building obsmarker-part in the bundle2 module Pierre-Yves David <pierre-yves.david@octobus.net> parents: 32387 diff changeset	1703 remoteversions = obsmarkersversion(bundler.capabilities)
e70d6dbde713 bundle2: move function building obsmarker-part in the bundle2 module Pierre-Yves David <pierre-yves.david@octobus.net> parents: 32387 diff changeset	1704 version = obsolete.commonversion(remoteversions)
e70d6dbde713 bundle2: move function building obsmarker-part in the bundle2 module Pierre-Yves David <pierre-yves.david@octobus.net> parents: 32387 diff changeset	1705 if version is None:
e70d6dbde713 bundle2: move function building obsmarker-part in the bundle2 module Pierre-Yves David <pierre-yves.david@octobus.net> parents: 32387 diff changeset	1706 raise ValueError('bundler does not support common obsmarker format')
e70d6dbde713 bundle2: move function building obsmarker-part in the bundle2 module Pierre-Yves David <pierre-yves.david@octobus.net> parents: 32387 diff changeset	1707 stream = obsolete.encodemarkers(markers, True, version=version)
e70d6dbde713 bundle2: move function building obsmarker-part in the bundle2 module Pierre-Yves David <pierre-yves.david@octobus.net> parents: 32387 diff changeset	1708 return bundler.newpart('obsmarkers', data=stream)
e70d6dbde713 bundle2: move function building obsmarker-part in the bundle2 module Pierre-Yves David <pierre-yves.david@octobus.net> parents: 32387 diff changeset	1709
30757 511a4bf52754 bundle2: allow compression options to be passed to compressor Gregory Szorc <gregory.szorc@gmail.com> parents: 30439 diff changeset	1710 def writebundle(ui, cg, filename, bundletype, vfs=None, compression=None,
511a4bf52754 bundle2: allow compression options to be passed to compressor Gregory Szorc <gregory.szorc@gmail.com> parents: 30439 diff changeset	1711 compopts=None):
28666 ae53ecc47414 bundle: move writebundle() from changegroup.py to bundle2.py (API) Martin von Zweigbergk <martinvonz@google.com> parents: 27953 diff changeset	1712 """Write a bundle file and return its filename.
ae53ecc47414 bundle: move writebundle() from changegroup.py to bundle2.py (API) Martin von Zweigbergk <martinvonz@google.com> parents: 27953 diff changeset	1713
ae53ecc47414 bundle: move writebundle() from changegroup.py to bundle2.py (API) Martin von Zweigbergk <martinvonz@google.com> parents: 27953 diff changeset	1714 Existing files will not be overwritten.
ae53ecc47414 bundle: move writebundle() from changegroup.py to bundle2.py (API) Martin von Zweigbergk <martinvonz@google.com> parents: 27953 diff changeset	1715 If no filename is specified, a temporary file is created.
ae53ecc47414 bundle: move writebundle() from changegroup.py to bundle2.py (API) Martin von Zweigbergk <martinvonz@google.com> parents: 27953 diff changeset	1716 bz2 compression can be turned off.
ae53ecc47414 bundle: move writebundle() from changegroup.py to bundle2.py (API) Martin von Zweigbergk <martinvonz@google.com> parents: 27953 diff changeset	1717 The bundle file will be deleted in case of errors.
ae53ecc47414 bundle: move writebundle() from changegroup.py to bundle2.py (API) Martin von Zweigbergk <martinvonz@google.com> parents: 27953 diff changeset	1718 """
ae53ecc47414 bundle: move writebundle() from changegroup.py to bundle2.py (API) Martin von Zweigbergk <martinvonz@google.com> parents: 27953 diff changeset	1719
ae53ecc47414 bundle: move writebundle() from changegroup.py to bundle2.py (API) Martin von Zweigbergk <martinvonz@google.com> parents: 27953 diff changeset	1720 if bundletype == "HG20":
ae53ecc47414 bundle: move writebundle() from changegroup.py to bundle2.py (API) Martin von Zweigbergk <martinvonz@google.com> parents: 27953 diff changeset	1721 bundle = bundle20(ui)
30757 511a4bf52754 bundle2: allow compression options to be passed to compressor Gregory Szorc <gregory.szorc@gmail.com> parents: 30439 diff changeset	1722 bundle.setcompression(compression, compopts)
28666 ae53ecc47414 bundle: move writebundle() from changegroup.py to bundle2.py (API) Martin von Zweigbergk <martinvonz@google.com> parents: 27953 diff changeset	1723 part = bundle.newpart('changegroup', data=cg.getchunks())
ae53ecc47414 bundle: move writebundle() from changegroup.py to bundle2.py (API) Martin von Zweigbergk <martinvonz@google.com> parents: 27953 diff changeset	1724 part.addparam('version', cg.version)
29593 953839de96ab bundle2: store changeset count when creating file bundles Gregory Szorc <gregory.szorc@gmail.com> parents: 29591 diff changeset	1725 if 'clcount' in cg.extras:
34221 8e0358024a36 bundles: turn nbchanges int into a bytestr using pycompat.bytestr Augie Fackler <raf@durin42.com> parents: 34152 diff changeset	1726 part.addparam('nbchanges', '%d' % cg.extras['clcount'],
29593 953839de96ab bundle2: store changeset count when creating file bundles Gregory Szorc <gregory.szorc@gmail.com> parents: 29591 diff changeset	1727 mandatory=False)
28666 ae53ecc47414 bundle: move writebundle() from changegroup.py to bundle2.py (API) Martin von Zweigbergk <martinvonz@google.com> parents: 27953 diff changeset	1728 chunkiter = bundle.getchunks()
ae53ecc47414 bundle: move writebundle() from changegroup.py to bundle2.py (API) Martin von Zweigbergk <martinvonz@google.com> parents: 27953 diff changeset	1729 else:
ae53ecc47414 bundle: move writebundle() from changegroup.py to bundle2.py (API) Martin von Zweigbergk <martinvonz@google.com> parents: 27953 diff changeset	1730 # compression argument is only for the bundle2 case
ae53ecc47414 bundle: move writebundle() from changegroup.py to bundle2.py (API) Martin von Zweigbergk <martinvonz@google.com> parents: 27953 diff changeset	1731 assert compression is None
ae53ecc47414 bundle: move writebundle() from changegroup.py to bundle2.py (API) Martin von Zweigbergk <martinvonz@google.com> parents: 27953 diff changeset	1732 if cg.version != '01':
ae53ecc47414 bundle: move writebundle() from changegroup.py to bundle2.py (API) Martin von Zweigbergk <martinvonz@google.com> parents: 27953 diff changeset	1733 raise error.Abort(_('old bundle types only supports v1 '
ae53ecc47414 bundle: move writebundle() from changegroup.py to bundle2.py (API) Martin von Zweigbergk <martinvonz@google.com> parents: 27953 diff changeset	1734 'changegroups'))
ae53ecc47414 bundle: move writebundle() from changegroup.py to bundle2.py (API) Martin von Zweigbergk <martinvonz@google.com> parents: 27953 diff changeset	1735 header, comp = bundletypes[bundletype]
30351 f81002f736d7 bundle2: use new compression engine API for compression Gregory Szorc <gregory.szorc@gmail.com> parents: 30177 diff changeset	1736 if comp not in util.compengines.supportedbundletypes:
28666 ae53ecc47414 bundle: move writebundle() from changegroup.py to bundle2.py (API) Martin von Zweigbergk <martinvonz@google.com> parents: 27953 diff changeset	1737 raise error.Abort(_('unknown stream compression type: %s')
ae53ecc47414 bundle: move writebundle() from changegroup.py to bundle2.py (API) Martin von Zweigbergk <martinvonz@google.com> parents: 27953 diff changeset	1738 % comp)
30351 f81002f736d7 bundle2: use new compression engine API for compression Gregory Szorc <gregory.szorc@gmail.com> parents: 30177 diff changeset	1739 compengine = util.compengines.forbundletype(comp)
28666 ae53ecc47414 bundle: move writebundle() from changegroup.py to bundle2.py (API) Martin von Zweigbergk <martinvonz@google.com> parents: 27953 diff changeset	1740 def chunkiter():
ae53ecc47414 bundle: move writebundle() from changegroup.py to bundle2.py (API) Martin von Zweigbergk <martinvonz@google.com> parents: 27953 diff changeset	1741 yield header
30757 511a4bf52754 bundle2: allow compression options to be passed to compressor Gregory Szorc <gregory.szorc@gmail.com> parents: 30439 diff changeset	1742 for chunk in compengine.compressstream(cg.getchunks(), compopts):
30357 5925bda42dbd bundle2: use compressstream compression engine API Gregory Szorc <gregory.szorc@gmail.com> parents: 30353 diff changeset	1743 yield chunk
28666 ae53ecc47414 bundle: move writebundle() from changegroup.py to bundle2.py (API) Martin von Zweigbergk <martinvonz@google.com> parents: 27953 diff changeset	1744 chunkiter = chunkiter()
ae53ecc47414 bundle: move writebundle() from changegroup.py to bundle2.py (API) Martin von Zweigbergk <martinvonz@google.com> parents: 27953 diff changeset	1745
ae53ecc47414 bundle: move writebundle() from changegroup.py to bundle2.py (API) Martin von Zweigbergk <martinvonz@google.com> parents: 27953 diff changeset	1746 # parse the changegroup data, otherwise we will block
ae53ecc47414 bundle: move writebundle() from changegroup.py to bundle2.py (API) Martin von Zweigbergk <martinvonz@google.com> parents: 27953 diff changeset	1747 # in case of sshrepo because we don't know the end of the stream
ae53ecc47414 bundle: move writebundle() from changegroup.py to bundle2.py (API) Martin von Zweigbergk <martinvonz@google.com> parents: 27953 diff changeset	1748 return changegroup.writechunks(ui, chunkiter, filename, vfs=vfs)
ae53ecc47414 bundle: move writebundle() from changegroup.py to bundle2.py (API) Martin von Zweigbergk <martinvonz@google.com> parents: 27953 diff changeset	1749
33037 d765ad56081f bundle: make combinechangegroupresults() take a bundleoperation Martin von Zweigbergk <martinvonz@google.com> parents: 33036 diff changeset	1750 def combinechangegroupresults(op):
33036 52c7060b707a bundle: move combineresults() from changegroup to bundle2 Martin von Zweigbergk <martinvonz@google.com> parents: 33035 diff changeset	1751 """logic to combine 0 or more addchangegroup results into one"""
33037 d765ad56081f bundle: make combinechangegroupresults() take a bundleoperation Martin von Zweigbergk <martinvonz@google.com> parents: 33036 diff changeset	1752 results = [r.get('return', 0)
d765ad56081f bundle: make combinechangegroupresults() take a bundleoperation Martin von Zweigbergk <martinvonz@google.com> parents: 33036 diff changeset	1753 for r in op.records['changegroup']]
33036 52c7060b707a bundle: move combineresults() from changegroup to bundle2 Martin von Zweigbergk <martinvonz@google.com> parents: 33035 diff changeset	1754 changedheads = 0
52c7060b707a bundle: move combineresults() from changegroup to bundle2 Martin von Zweigbergk <martinvonz@google.com> parents: 33035 diff changeset	1755 result = 1
52c7060b707a bundle: move combineresults() from changegroup to bundle2 Martin von Zweigbergk <martinvonz@google.com> parents: 33035 diff changeset	1756 for ret in results:
52c7060b707a bundle: move combineresults() from changegroup to bundle2 Martin von Zweigbergk <martinvonz@google.com> parents: 33035 diff changeset	1757 # If any changegroup result is 0, return 0
52c7060b707a bundle: move combineresults() from changegroup to bundle2 Martin von Zweigbergk <martinvonz@google.com> parents: 33035 diff changeset	1758 if ret == 0:
52c7060b707a bundle: move combineresults() from changegroup to bundle2 Martin von Zweigbergk <martinvonz@google.com> parents: 33035 diff changeset	1759 result = 0
52c7060b707a bundle: move combineresults() from changegroup to bundle2 Martin von Zweigbergk <martinvonz@google.com> parents: 33035 diff changeset	1760 break
52c7060b707a bundle: move combineresults() from changegroup to bundle2 Martin von Zweigbergk <martinvonz@google.com> parents: 33035 diff changeset	1761 if ret < -1:
52c7060b707a bundle: move combineresults() from changegroup to bundle2 Martin von Zweigbergk <martinvonz@google.com> parents: 33035 diff changeset	1762 changedheads += ret + 1
52c7060b707a bundle: move combineresults() from changegroup to bundle2 Martin von Zweigbergk <martinvonz@google.com> parents: 33035 diff changeset	1763 elif ret > 1:
52c7060b707a bundle: move combineresults() from changegroup to bundle2 Martin von Zweigbergk <martinvonz@google.com> parents: 33035 diff changeset	1764 changedheads += ret - 1
52c7060b707a bundle: move combineresults() from changegroup to bundle2 Martin von Zweigbergk <martinvonz@google.com> parents: 33035 diff changeset	1765 if changedheads > 0:
52c7060b707a bundle: move combineresults() from changegroup to bundle2 Martin von Zweigbergk <martinvonz@google.com> parents: 33035 diff changeset	1766 result = 1 + changedheads
52c7060b707a bundle: move combineresults() from changegroup to bundle2 Martin von Zweigbergk <martinvonz@google.com> parents: 33035 diff changeset	1767 elif changedheads < 0:
52c7060b707a bundle: move combineresults() from changegroup to bundle2 Martin von Zweigbergk <martinvonz@google.com> parents: 33035 diff changeset	1768 result = -1 + changedheads
52c7060b707a bundle: move combineresults() from changegroup to bundle2 Martin von Zweigbergk <martinvonz@google.com> parents: 33035 diff changeset	1769 return result
52c7060b707a bundle: move combineresults() from changegroup to bundle2 Martin von Zweigbergk <martinvonz@google.com> parents: 33035 diff changeset	1770
33407 39d4e5a66f5a bundle2: support the 'targetphase' parameter for the changegroup part Boris Feld <boris.feld@octobus.net> parents: 33252 diff changeset	1771 @parthandler('changegroup', ('version', 'nbchanges', 'treemanifest',
39d4e5a66f5a bundle2: support the 'targetphase' parameter for the changegroup part Boris Feld <boris.feld@octobus.net> parents: 33252 diff changeset	1772 'targetphase'))
20998 93a3c5b58635 bundle2: use reply part to return result of addchangegroup Pierre-Yves David <pierre-yves.david@fb.com> parents: 20997 diff changeset	1773 def handlechangegroup(op, inpart):
20950 c7ceae0faf69 bundle2: first crude version of bundling changeset with bundle2 Pierre-Yves David <pierre-yves.david@fb.com> parents: 20949 diff changeset	1774 """apply a changegroup part on the repo
c7ceae0faf69 bundle2: first crude version of bundling changeset with bundle2 Pierre-Yves David <pierre-yves.david@fb.com> parents: 20949 diff changeset	1775
c7ceae0faf69 bundle2: first crude version of bundling changeset with bundle2 Pierre-Yves David <pierre-yves.david@fb.com> parents: 20949 diff changeset	1776 This is a very early implementation that will massive rework before being
c7ceae0faf69 bundle2: first crude version of bundling changeset with bundle2 Pierre-Yves David <pierre-yves.david@fb.com> parents: 20949 diff changeset	1777 inflicted to any end-user.
c7ceae0faf69 bundle2: first crude version of bundling changeset with bundle2 Pierre-Yves David <pierre-yves.david@fb.com> parents: 20949 diff changeset	1778 """
32930 af31d531dda0 changegroup: let callers pass in transaction to apply() (API) Martin von Zweigbergk <martinvonz@google.com> parents: 32892 diff changeset	1779 tr = op.gettransaction()
23170 02e8f9b60052 bundle2: support a "version" argument in `changegroup` part Pierre-Yves David <pierre-yves.david@fb.com> parents: 23169 diff changeset	1780 unpackerversion = inpart.params.get('version', '01')
02e8f9b60052 bundle2: support a "version" argument in `changegroup` part Pierre-Yves David <pierre-yves.david@fb.com> parents: 23169 diff changeset	1781 # We should raise an appropriate exception here
27751 a40e2f7fe49d changegroup: hide packermap behind methods Martin von Zweigbergk <martinvonz@google.com> parents: 27734 diff changeset	1782 cg = changegroup.getunbundler(unpackerversion, inpart, None)
23001 4df9b5e62f70 bundle2: add a comment about addchangegroup source and url Pierre-Yves David <pierre-yves.david@fb.com> parents: 22953 diff changeset	1783 # the source and url passed here are overwritten by the one contained in
4df9b5e62f70 bundle2: add a comment about addchangegroup source and url Pierre-Yves David <pierre-yves.david@fb.com> parents: 22953 diff changeset	1784 # the transaction.hookargs argument. So 'bundle2' is a placeholder
25518 ca656f3dffd7 bundle2: provide number of changesets information to 'addchangegroup' Pierre-Yves David <pierre-yves.david@fb.com> parents: 25507 diff changeset	1785 nbchangesets = None
ca656f3dffd7 bundle2: provide number of changesets information to 'addchangegroup' Pierre-Yves David <pierre-yves.david@fb.com> parents: 25507 diff changeset	1786 if 'nbchanges' in inpart.params:
ca656f3dffd7 bundle2: provide number of changesets information to 'addchangegroup' Pierre-Yves David <pierre-yves.david@fb.com> parents: 25507 diff changeset	1787 nbchangesets = int(inpart.params.get('nbchanges'))
27734 5c0fd878779c treemanifests: set bundle2 part parameter indicating treemanifest Martin von Zweigbergk <martinvonz@google.com> parents: 26829 diff changeset	1788 if ('treemanifest' in inpart.params and
5c0fd878779c treemanifests: set bundle2 part parameter indicating treemanifest Martin von Zweigbergk <martinvonz@google.com> parents: 26829 diff changeset	1789 'treemanifest' not in op.repo.requirements):
5c0fd878779c treemanifests: set bundle2 part parameter indicating treemanifest Martin von Zweigbergk <martinvonz@google.com> parents: 26829 diff changeset	1790 if len(op.repo.changelog) != 0:
5c0fd878779c treemanifests: set bundle2 part parameter indicating treemanifest Martin von Zweigbergk <martinvonz@google.com> parents: 26829 diff changeset	1791 raise error.Abort(_(
5c0fd878779c treemanifests: set bundle2 part parameter indicating treemanifest Martin von Zweigbergk <martinvonz@google.com> parents: 26829 diff changeset	1792 "bundle contains tree manifests, but local repo is "
5c0fd878779c treemanifests: set bundle2 part parameter indicating treemanifest Martin von Zweigbergk <martinvonz@google.com> parents: 26829 diff changeset	1793 "non-empty and does not use tree manifests"))
5c0fd878779c treemanifests: set bundle2 part parameter indicating treemanifest Martin von Zweigbergk <martinvonz@google.com> parents: 26829 diff changeset	1794 op.repo.requirements.add('treemanifest')
5c0fd878779c treemanifests: set bundle2 part parameter indicating treemanifest Martin von Zweigbergk <martinvonz@google.com> parents: 26829 diff changeset	1795 op.repo._applyopenerreqs()
5c0fd878779c treemanifests: set bundle2 part parameter indicating treemanifest Martin von Zweigbergk <martinvonz@google.com> parents: 26829 diff changeset	1796 op.repo._writerequirements()
33407 39d4e5a66f5a bundle2: support the 'targetphase' parameter for the changegroup part Boris Feld <boris.feld@octobus.net> parents: 33252 diff changeset	1797 extrakwargs = {}
39d4e5a66f5a bundle2: support the 'targetphase' parameter for the changegroup part Boris Feld <boris.feld@octobus.net> parents: 33252 diff changeset	1798 targetphase = inpart.params.get('targetphase')
39d4e5a66f5a bundle2: support the 'targetphase' parameter for the changegroup part Boris Feld <boris.feld@octobus.net> parents: 33252 diff changeset	1799 if targetphase is not None:
36116 b587a889b97e py3: use raw string for key in *kwargs Gregory Szorc <gregory.szorc@gmail.com>* parents: 35813 diff changeset	1800 extrakwargs[r'targetphase'] = int(targetphase)
33038 f0efd2bffe1e bundle: extract _processchangegroup() method Martin von Zweigbergk <martinvonz@google.com> parents: 33037 diff changeset	1801 ret = _processchangegroup(op, cg, tr, 'bundle2', 'bundle2',
33407 39d4e5a66f5a bundle2: support the 'targetphase' parameter for the changegroup part Boris Feld <boris.feld@octobus.net> parents: 33252 diff changeset	1802 expectedtotal=nbchangesets, **extrakwargs)
20998 93a3c5b58635 bundle2: use reply part to return result of addchangegroup Pierre-Yves David <pierre-yves.david@fb.com> parents: 20997 diff changeset	1803 if op.reply is not None:
23139 e53f6b72a0e4 spelling: fixes from proofreading of spell checker issues Mads Kiilerich <madski@unity3d.com> parents: 23067 diff changeset	1804 # This is definitely not the final form of this
20998 93a3c5b58635 bundle2: use reply part to return result of addchangegroup Pierre-Yves David <pierre-yves.david@fb.com> parents: 20997 diff changeset	1805 # return. But one need to start somewhere.
24686 e0e28e910fa3 bundle2: rename format, parts and config to final names Pierre-Yves David <pierre-yves.david@fb.com> parents: 24648 diff changeset	1806 part = op.reply.newpart('reply:changegroup', mandatory=False)
33675 24b7c75c6441 bundle2: use bytestr() instead of str() to convert part id to bytes Augie Fackler <augie@google.com> parents: 33674 diff changeset	1807 part.addparam(
24b7c75c6441 bundle2: use bytestr() instead of str() to convert part id to bytes Augie Fackler <augie@google.com> parents: 33674 diff changeset	1808 'in-reply-to', pycompat.bytestr(inpart.id), mandatory=False)
21606 e55888447958 bundle2: update part creators to ``addparam`` when relevant Pierre-Yves David <pierre-yves.david@fb.com> parents: 21605 diff changeset	1809 part.addparam('return', '%i' % ret, mandatory=False)
21019 3dc09f831a2e bundle2: lazy unbundle of part payload Pierre-Yves David <pierre-yves.david@fb.com> parents: 21016 diff changeset	1810 assert not inpart.read()
20950 c7ceae0faf69 bundle2: first crude version of bundling changeset with bundle2 Pierre-Yves David <pierre-yves.david@fb.com> parents: 20949 diff changeset	1811
23029 149fc8a44184 bundle2: client side support for a part to import external bundles Mike Hommey <mh@glandium.org> parents: 23011 diff changeset	1812 _remotechangegroupparams = tuple(['url', 'size', 'digests'] +
149fc8a44184 bundle2: client side support for a part to import external bundles Mike Hommey <mh@glandium.org> parents: 23011 diff changeset	1813 ['digest:%s' % k for k in util.DIGESTS.keys()])
24686 e0e28e910fa3 bundle2: rename format, parts and config to final names Pierre-Yves David <pierre-yves.david@fb.com> parents: 24648 diff changeset	1814 @parthandler('remote-changegroup', _remotechangegroupparams)
23029 149fc8a44184 bundle2: client side support for a part to import external bundles Mike Hommey <mh@glandium.org> parents: 23011 diff changeset	1815 def handleremotechangegroup(op, inpart):
149fc8a44184 bundle2: client side support for a part to import external bundles Mike Hommey <mh@glandium.org> parents: 23011 diff changeset	1816 """apply a bundle10 on the repo, given an url and validation information
149fc8a44184 bundle2: client side support for a part to import external bundles Mike Hommey <mh@glandium.org> parents: 23011 diff changeset	1817
149fc8a44184 bundle2: client side support for a part to import external bundles Mike Hommey <mh@glandium.org> parents: 23011 diff changeset	1818 All the information about the remote bundle to import are given as
149fc8a44184 bundle2: client side support for a part to import external bundles Mike Hommey <mh@glandium.org> parents: 23011 diff changeset	1819 parameters. The parameters include:
149fc8a44184 bundle2: client side support for a part to import external bundles Mike Hommey <mh@glandium.org> parents: 23011 diff changeset	1820 - url: the url to the bundle10.
149fc8a44184 bundle2: client side support for a part to import external bundles Mike Hommey <mh@glandium.org> parents: 23011 diff changeset	1821 - size: the bundle10 file size. It is used to validate what was
149fc8a44184 bundle2: client side support for a part to import external bundles Mike Hommey <mh@glandium.org> parents: 23011 diff changeset	1822 retrieved by the client matches the server knowledge about the bundle.
149fc8a44184 bundle2: client side support for a part to import external bundles Mike Hommey <mh@glandium.org> parents: 23011 diff changeset	1823 - digests: a space separated list of the digest types provided as
149fc8a44184 bundle2: client side support for a part to import external bundles Mike Hommey <mh@glandium.org> parents: 23011 diff changeset	1824 parameters.
149fc8a44184 bundle2: client side support for a part to import external bundles Mike Hommey <mh@glandium.org> parents: 23011 diff changeset	1825 - digest:<digest-type>: the hexadecimal representation of the digest with
149fc8a44184 bundle2: client side support for a part to import external bundles Mike Hommey <mh@glandium.org> parents: 23011 diff changeset	1826 that name. Like the size, it is used to validate what was retrieved by
149fc8a44184 bundle2: client side support for a part to import external bundles Mike Hommey <mh@glandium.org> parents: 23011 diff changeset	1827 the client matches what the server knows about the bundle.
149fc8a44184 bundle2: client side support for a part to import external bundles Mike Hommey <mh@glandium.org> parents: 23011 diff changeset	1828
149fc8a44184 bundle2: client side support for a part to import external bundles Mike Hommey <mh@glandium.org> parents: 23011 diff changeset	1829 When multiple digest types are given, all of them are checked.
149fc8a44184 bundle2: client side support for a part to import external bundles Mike Hommey <mh@glandium.org> parents: 23011 diff changeset	1830 """
149fc8a44184 bundle2: client side support for a part to import external bundles Mike Hommey <mh@glandium.org> parents: 23011 diff changeset	1831 try:
149fc8a44184 bundle2: client side support for a part to import external bundles Mike Hommey <mh@glandium.org> parents: 23011 diff changeset	1832 raw_url = inpart.params['url']
149fc8a44184 bundle2: client side support for a part to import external bundles Mike Hommey <mh@glandium.org> parents: 23011 diff changeset	1833 except KeyError:
26587 56b2bcea2529 error: get Abort from 'error' instead of 'util' Pierre-Yves David <pierre-yves.david@fb.com> parents: 26565 diff changeset	1834 raise error.Abort(_('remote-changegroup: missing "%s" param') % 'url')
23029 149fc8a44184 bundle2: client side support for a part to import external bundles Mike Hommey <mh@glandium.org> parents: 23011 diff changeset	1835 parsed_url = util.url(raw_url)
24686 e0e28e910fa3 bundle2: rename format, parts and config to final names Pierre-Yves David <pierre-yves.david@fb.com> parents: 24648 diff changeset	1836 if parsed_url.scheme not in capabilities['remote-changegroup']:
26587 56b2bcea2529 error: get Abort from 'error' instead of 'util' Pierre-Yves David <pierre-yves.david@fb.com> parents: 26565 diff changeset	1837 raise error.Abort(_('remote-changegroup does not support %s urls') %
23029 149fc8a44184 bundle2: client side support for a part to import external bundles Mike Hommey <mh@glandium.org> parents: 23011 diff changeset	1838 parsed_url.scheme)
149fc8a44184 bundle2: client side support for a part to import external bundles Mike Hommey <mh@glandium.org> parents: 23011 diff changeset	1839
149fc8a44184 bundle2: client side support for a part to import external bundles Mike Hommey <mh@glandium.org> parents: 23011 diff changeset	1840 try:
149fc8a44184 bundle2: client side support for a part to import external bundles Mike Hommey <mh@glandium.org> parents: 23011 diff changeset	1841 size = int(inpart.params['size'])
149fc8a44184 bundle2: client side support for a part to import external bundles Mike Hommey <mh@glandium.org> parents: 23011 diff changeset	1842 except ValueError:
26587 56b2bcea2529 error: get Abort from 'error' instead of 'util' Pierre-Yves David <pierre-yves.david@fb.com> parents: 26565 diff changeset	1843 raise error.Abort(_('remote-changegroup: invalid value for param "%s"')
23029 149fc8a44184 bundle2: client side support for a part to import external bundles Mike Hommey <mh@glandium.org> parents: 23011 diff changeset	1844 % 'size')
149fc8a44184 bundle2: client side support for a part to import external bundles Mike Hommey <mh@glandium.org> parents: 23011 diff changeset	1845 except KeyError:
26587 56b2bcea2529 error: get Abort from 'error' instead of 'util' Pierre-Yves David <pierre-yves.david@fb.com> parents: 26565 diff changeset	1846 raise error.Abort(_('remote-changegroup: missing "%s" param') % 'size')
23029 149fc8a44184 bundle2: client side support for a part to import external bundles Mike Hommey <mh@glandium.org> parents: 23011 diff changeset	1847
149fc8a44184 bundle2: client side support for a part to import external bundles Mike Hommey <mh@glandium.org> parents: 23011 diff changeset	1848 digests = {}
149fc8a44184 bundle2: client side support for a part to import external bundles Mike Hommey <mh@glandium.org> parents: 23011 diff changeset	1849 for typ in inpart.params.get('digests', '').split():
149fc8a44184 bundle2: client side support for a part to import external bundles Mike Hommey <mh@glandium.org> parents: 23011 diff changeset	1850 param = 'digest:%s' % typ
149fc8a44184 bundle2: client side support for a part to import external bundles Mike Hommey <mh@glandium.org> parents: 23011 diff changeset	1851 try:
149fc8a44184 bundle2: client side support for a part to import external bundles Mike Hommey <mh@glandium.org> parents: 23011 diff changeset	1852 value = inpart.params[param]
149fc8a44184 bundle2: client side support for a part to import external bundles Mike Hommey <mh@glandium.org> parents: 23011 diff changeset	1853 except KeyError:
26587 56b2bcea2529 error: get Abort from 'error' instead of 'util' Pierre-Yves David <pierre-yves.david@fb.com> parents: 26565 diff changeset	1854 raise error.Abort(_('remote-changegroup: missing "%s" param') %
23029 149fc8a44184 bundle2: client side support for a part to import external bundles Mike Hommey <mh@glandium.org> parents: 23011 diff changeset	1855 param)
149fc8a44184 bundle2: client side support for a part to import external bundles Mike Hommey <mh@glandium.org> parents: 23011 diff changeset	1856 digests[typ] = value
149fc8a44184 bundle2: client side support for a part to import external bundles Mike Hommey <mh@glandium.org> parents: 23011 diff changeset	1857
149fc8a44184 bundle2: client side support for a part to import external bundles Mike Hommey <mh@glandium.org> parents: 23011 diff changeset	1858 real_part = util.digestchecker(url.open(op.ui, raw_url), size, digests)
149fc8a44184 bundle2: client side support for a part to import external bundles Mike Hommey <mh@glandium.org> parents: 23011 diff changeset	1859
32930 af31d531dda0 changegroup: let callers pass in transaction to apply() (API) Martin von Zweigbergk <martinvonz@google.com> parents: 32892 diff changeset	1860 tr = op.gettransaction()
25919 8221fefaea08 bundle2: use absolute_import Gregory Szorc <gregory.szorc@gmail.com> parents: 25660 diff changeset	1861 from . import exchange
23029 149fc8a44184 bundle2: client side support for a part to import external bundles Mike Hommey <mh@glandium.org> parents: 23011 diff changeset	1862 cg = exchange.readbundle(op.repo.ui, real_part, raw_url)
149fc8a44184 bundle2: client side support for a part to import external bundles Mike Hommey <mh@glandium.org> parents: 23011 diff changeset	1863 if not isinstance(cg, changegroup.cg1unpacker):
26587 56b2bcea2529 error: get Abort from 'error' instead of 'util' Pierre-Yves David <pierre-yves.david@fb.com> parents: 26565 diff changeset	1864 raise error.Abort(_('%s: not a bundle version 1.0') %
23029 149fc8a44184 bundle2: client side support for a part to import external bundles Mike Hommey <mh@glandium.org> parents: 23011 diff changeset	1865 util.hidepassword(raw_url))
33038 f0efd2bffe1e bundle: extract _processchangegroup() method Martin von Zweigbergk <martinvonz@google.com> parents: 33037 diff changeset	1866 ret = _processchangegroup(op, cg, tr, 'bundle2', 'bundle2')
23029 149fc8a44184 bundle2: client side support for a part to import external bundles Mike Hommey <mh@glandium.org> parents: 23011 diff changeset	1867 if op.reply is not None:
23139 e53f6b72a0e4 spelling: fixes from proofreading of spell checker issues Mads Kiilerich <madski@unity3d.com> parents: 23067 diff changeset	1868 # This is definitely not the final form of this
23029 149fc8a44184 bundle2: client side support for a part to import external bundles Mike Hommey <mh@glandium.org> parents: 23011 diff changeset	1869 # return. But one need to start somewhere.
24686 e0e28e910fa3 bundle2: rename format, parts and config to final names Pierre-Yves David <pierre-yves.david@fb.com> parents: 24648 diff changeset	1870 part = op.reply.newpart('reply:changegroup')
33675 24b7c75c6441 bundle2: use bytestr() instead of str() to convert part id to bytes Augie Fackler <augie@google.com> parents: 33674 diff changeset	1871 part.addparam(
24b7c75c6441 bundle2: use bytestr() instead of str() to convert part id to bytes Augie Fackler <augie@google.com> parents: 33674 diff changeset	1872 'in-reply-to', pycompat.bytestr(inpart.id), mandatory=False)
23029 149fc8a44184 bundle2: client side support for a part to import external bundles Mike Hommey <mh@glandium.org> parents: 23011 diff changeset	1873 part.addparam('return', '%i' % ret, mandatory=False)
149fc8a44184 bundle2: client side support for a part to import external bundles Mike Hommey <mh@glandium.org> parents: 23011 diff changeset	1874 try:
149fc8a44184 bundle2: client side support for a part to import external bundles Mike Hommey <mh@glandium.org> parents: 23011 diff changeset	1875 real_part.validate()
26587 56b2bcea2529 error: get Abort from 'error' instead of 'util' Pierre-Yves David <pierre-yves.david@fb.com> parents: 26565 diff changeset	1876 except error.Abort as e:
56b2bcea2529 error: get Abort from 'error' instead of 'util' Pierre-Yves David <pierre-yves.david@fb.com> parents: 26565 diff changeset	1877 raise error.Abort(_('bundle at %s is corrupted:\n%s') %
23029 149fc8a44184 bundle2: client side support for a part to import external bundles Mike Hommey <mh@glandium.org> parents: 23011 diff changeset	1878 (util.hidepassword(raw_url), str(e)))
149fc8a44184 bundle2: client side support for a part to import external bundles Mike Hommey <mh@glandium.org> parents: 23011 diff changeset	1879 assert not inpart.read()
149fc8a44184 bundle2: client side support for a part to import external bundles Mike Hommey <mh@glandium.org> parents: 23011 diff changeset	1880
24686 e0e28e910fa3 bundle2: rename format, parts and config to final names Pierre-Yves David <pierre-yves.david@fb.com> parents: 24648 diff changeset	1881 @parthandler('reply:changegroup', ('return', 'in-reply-to'))
22548 8a1ae133770a bundle2: rename functions that have the same name Mike Hommey <mh@glandium.org> parents: 22390 diff changeset	1882 def handlereplychangegroup(op, inpart):
21611 71b7b3f79a3c bundle2: use the new ``part.params`` dictionary Pierre-Yves David <pierre-yves.david@fb.com> parents: 21610 diff changeset	1883 ret = int(inpart.params['return'])
71b7b3f79a3c bundle2: use the new ``part.params`` dictionary Pierre-Yves David <pierre-yves.david@fb.com> parents: 21610 diff changeset	1884 replyto = int(inpart.params['in-reply-to'])
71b7b3f79a3c bundle2: use the new ``part.params`` dictionary Pierre-Yves David <pierre-yves.david@fb.com> parents: 21610 diff changeset	1885 op.records.add('changegroup', {'return': ret}, replyto)
20950 c7ceae0faf69 bundle2: first crude version of bundling changeset with bundle2 Pierre-Yves David <pierre-yves.david@fb.com> parents: 20949 diff changeset	1886
35258 dbf868623daf bookmark: add a 'check:bookmarks' bundle2 part Boris Feld <boris.feld@octobus.net> parents: 35120 diff changeset	1887 @parthandler('check:bookmarks')
dbf868623daf bookmark: add a 'check:bookmarks' bundle2 part Boris Feld <boris.feld@octobus.net> parents: 35120 diff changeset	1888 def handlecheckbookmarks(op, inpart):
dbf868623daf bookmark: add a 'check:bookmarks' bundle2 part Boris Feld <boris.feld@octobus.net> parents: 35120 diff changeset	1889 """check location of bookmarks
dbf868623daf bookmark: add a 'check:bookmarks' bundle2 part Boris Feld <boris.feld@octobus.net> parents: 35120 diff changeset	1890
dbf868623daf bookmark: add a 'check:bookmarks' bundle2 part Boris Feld <boris.feld@octobus.net> parents: 35120 diff changeset	1891 This part is to be used to detect push race regarding bookmark, it
dbf868623daf bookmark: add a 'check:bookmarks' bundle2 part Boris Feld <boris.feld@octobus.net> parents: 35120 diff changeset	1892 contains binary encoded (bookmark, node) tuple. If the local state does
dbf868623daf bookmark: add a 'check:bookmarks' bundle2 part Boris Feld <boris.feld@octobus.net> parents: 35120 diff changeset	1893 not marks the one in the part, a PushRaced exception is raised
dbf868623daf bookmark: add a 'check:bookmarks' bundle2 part Boris Feld <boris.feld@octobus.net> parents: 35120 diff changeset	1894 """
dbf868623daf bookmark: add a 'check:bookmarks' bundle2 part Boris Feld <boris.feld@octobus.net> parents: 35120 diff changeset	1895 bookdata = bookmarks.binarydecode(inpart)
dbf868623daf bookmark: add a 'check:bookmarks' bundle2 part Boris Feld <boris.feld@octobus.net> parents: 35120 diff changeset	1896
dbf868623daf bookmark: add a 'check:bookmarks' bundle2 part Boris Feld <boris.feld@octobus.net> parents: 35120 diff changeset	1897 msgstandard = ('repository changed while pushing - please try again '
dbf868623daf bookmark: add a 'check:bookmarks' bundle2 part Boris Feld <boris.feld@octobus.net> parents: 35120 diff changeset	1898 '(bookmark "%s" move from %s to %s)')
dbf868623daf bookmark: add a 'check:bookmarks' bundle2 part Boris Feld <boris.feld@octobus.net> parents: 35120 diff changeset	1899 msgmissing = ('repository changed while pushing - please try again '
dbf868623daf bookmark: add a 'check:bookmarks' bundle2 part Boris Feld <boris.feld@octobus.net> parents: 35120 diff changeset	1900 '(bookmark "%s" is missing, expected %s)')
dbf868623daf bookmark: add a 'check:bookmarks' bundle2 part Boris Feld <boris.feld@octobus.net> parents: 35120 diff changeset	1901 msgexist = ('repository changed while pushing - please try again '
dbf868623daf bookmark: add a 'check:bookmarks' bundle2 part Boris Feld <boris.feld@octobus.net> parents: 35120 diff changeset	1902 '(bookmark "%s" set on %s, expected missing)')
dbf868623daf bookmark: add a 'check:bookmarks' bundle2 part Boris Feld <boris.feld@octobus.net> parents: 35120 diff changeset	1903 for book, node in bookdata:
dbf868623daf bookmark: add a 'check:bookmarks' bundle2 part Boris Feld <boris.feld@octobus.net> parents: 35120 diff changeset	1904 currentnode = op.repo._bookmarks.get(book)
dbf868623daf bookmark: add a 'check:bookmarks' bundle2 part Boris Feld <boris.feld@octobus.net> parents: 35120 diff changeset	1905 if currentnode != node:
dbf868623daf bookmark: add a 'check:bookmarks' bundle2 part Boris Feld <boris.feld@octobus.net> parents: 35120 diff changeset	1906 if node is None:
dbf868623daf bookmark: add a 'check:bookmarks' bundle2 part Boris Feld <boris.feld@octobus.net> parents: 35120 diff changeset	1907 finalmsg = msgexist % (book, nodemod.short(currentnode))
dbf868623daf bookmark: add a 'check:bookmarks' bundle2 part Boris Feld <boris.feld@octobus.net> parents: 35120 diff changeset	1908 elif currentnode is None:
dbf868623daf bookmark: add a 'check:bookmarks' bundle2 part Boris Feld <boris.feld@octobus.net> parents: 35120 diff changeset	1909 finalmsg = msgmissing % (book, nodemod.short(node))
dbf868623daf bookmark: add a 'check:bookmarks' bundle2 part Boris Feld <boris.feld@octobus.net> parents: 35120 diff changeset	1910 else:
dbf868623daf bookmark: add a 'check:bookmarks' bundle2 part Boris Feld <boris.feld@octobus.net> parents: 35120 diff changeset	1911 finalmsg = msgstandard % (book, nodemod.short(node),
dbf868623daf bookmark: add a 'check:bookmarks' bundle2 part Boris Feld <boris.feld@octobus.net> parents: 35120 diff changeset	1912 nodemod.short(currentnode))
dbf868623daf bookmark: add a 'check:bookmarks' bundle2 part Boris Feld <boris.feld@octobus.net> parents: 35120 diff changeset	1913 raise error.PushRaced(finalmsg)
dbf868623daf bookmark: add a 'check:bookmarks' bundle2 part Boris Feld <boris.feld@octobus.net> parents: 35120 diff changeset	1914
24686 e0e28e910fa3 bundle2: rename format, parts and config to final names Pierre-Yves David <pierre-yves.david@fb.com> parents: 24648 diff changeset	1915 @parthandler('check:heads')
22548 8a1ae133770a bundle2: rename functions that have the same name Mike Hommey <mh@glandium.org> parents: 22390 diff changeset	1916 def handlecheckheads(op, inpart):
21060 0bea9db7543b bundle2: add a "check:heads" handler Pierre-Yves David <pierre-yves.david@fb.com> parents: 21024 diff changeset	1917 """check that head of the repo did not change
0bea9db7543b bundle2: add a "check:heads" handler Pierre-Yves David <pierre-yves.david@fb.com> parents: 21024 diff changeset	1918
0bea9db7543b bundle2: add a "check:heads" handler Pierre-Yves David <pierre-yves.david@fb.com> parents: 21024 diff changeset	1919 This is used to detect a push race when using unbundle.
0bea9db7543b bundle2: add a "check:heads" handler Pierre-Yves David <pierre-yves.david@fb.com> parents: 21024 diff changeset	1920 This replaces the "heads" argument of unbundle."""
0bea9db7543b bundle2: add a "check:heads" handler Pierre-Yves David <pierre-yves.david@fb.com> parents: 21024 diff changeset	1921 h = inpart.read(20)
0bea9db7543b bundle2: add a "check:heads" handler Pierre-Yves David <pierre-yves.david@fb.com> parents: 21024 diff changeset	1922 heads = []
0bea9db7543b bundle2: add a "check:heads" handler Pierre-Yves David <pierre-yves.david@fb.com> parents: 21024 diff changeset	1923 while len(h) == 20:
0bea9db7543b bundle2: add a "check:heads" handler Pierre-Yves David <pierre-yves.david@fb.com> parents: 21024 diff changeset	1924 heads.append(h)
0bea9db7543b bundle2: add a "check:heads" handler Pierre-Yves David <pierre-yves.david@fb.com> parents: 21024 diff changeset	1925 h = inpart.read(20)
0bea9db7543b bundle2: add a "check:heads" handler Pierre-Yves David <pierre-yves.david@fb.com> parents: 21024 diff changeset	1926 assert not h
26565 ee1bcb9aa6e4 bundle2: add op.gettransaction() to handlers that need the lock Durham Goode <durham@fb.com> parents: 26542 diff changeset	1927 # Trigger a transaction so that we are guaranteed to have the lock now.
ee1bcb9aa6e4 bundle2: add op.gettransaction() to handlers that need the lock Durham Goode <durham@fb.com> parents: 26542 diff changeset	1928 if op.ui.configbool('experimental', 'bundle2lazylocking'):
ee1bcb9aa6e4 bundle2: add op.gettransaction() to handlers that need the lock Durham Goode <durham@fb.com> parents: 26542 diff changeset	1929 op.gettransaction()
29294 077d0535f51f bundle2: don't assume ordering of heads checked after push Mads Kiilerich <madski@unity3d.com> parents: 28883 diff changeset	1930 if sorted(heads) != sorted(op.repo.heads()):
21185 5b3717e1a3ea bundle2: add an error message to push race error Pierre-Yves David <pierre-yves.david@fb.com> parents: 21184 diff changeset	1931 raise error.PushRaced('repository changed while pushing - '
5b3717e1a3ea bundle2: add an error message to push race error Pierre-Yves David <pierre-yves.david@fb.com> parents: 21184 diff changeset	1932 'please try again')
21130 1ff06386217f bundle2: introduce `replycaps` part for on-demand reply Pierre-Yves David <pierre-yves.david@fb.com> parents: 21129 diff changeset	1933
32709 16ada4cbb1a9 push: add a way to allow concurrent pushes on unrelated heads Pierre-Yves David <pierre-yves.david@octobus.net> parents: 32516 diff changeset	1934 @parthandler('check:updated-heads')
16ada4cbb1a9 push: add a way to allow concurrent pushes on unrelated heads Pierre-Yves David <pierre-yves.david@octobus.net> parents: 32516 diff changeset	1935 def handlecheckupdatedheads(op, inpart):
16ada4cbb1a9 push: add a way to allow concurrent pushes on unrelated heads Pierre-Yves David <pierre-yves.david@octobus.net> parents: 32516 diff changeset	1936 """check for race on the heads touched by a push
16ada4cbb1a9 push: add a way to allow concurrent pushes on unrelated heads Pierre-Yves David <pierre-yves.david@octobus.net> parents: 32516 diff changeset	1937
16ada4cbb1a9 push: add a way to allow concurrent pushes on unrelated heads Pierre-Yves David <pierre-yves.david@octobus.net> parents: 32516 diff changeset	1938 This is similar to 'check:heads' but focus on the heads actually updated
16ada4cbb1a9 push: add a way to allow concurrent pushes on unrelated heads Pierre-Yves David <pierre-yves.david@octobus.net> parents: 32516 diff changeset	1939 during the push. If other activities happen on unrelated heads, it is
16ada4cbb1a9 push: add a way to allow concurrent pushes on unrelated heads Pierre-Yves David <pierre-yves.david@octobus.net> parents: 32516 diff changeset	1940 ignored.
16ada4cbb1a9 push: add a way to allow concurrent pushes on unrelated heads Pierre-Yves David <pierre-yves.david@octobus.net> parents: 32516 diff changeset	1941
16ada4cbb1a9 push: add a way to allow concurrent pushes on unrelated heads Pierre-Yves David <pierre-yves.david@octobus.net> parents: 32516 diff changeset	1942 This allow server with high traffic to avoid push contention as long as
16ada4cbb1a9 push: add a way to allow concurrent pushes on unrelated heads Pierre-Yves David <pierre-yves.david@octobus.net> parents: 32516 diff changeset	1943 unrelated parts of the graph are involved."""
16ada4cbb1a9 push: add a way to allow concurrent pushes on unrelated heads Pierre-Yves David <pierre-yves.david@octobus.net> parents: 32516 diff changeset	1944 h = inpart.read(20)
16ada4cbb1a9 push: add a way to allow concurrent pushes on unrelated heads Pierre-Yves David <pierre-yves.david@octobus.net> parents: 32516 diff changeset	1945 heads = []
16ada4cbb1a9 push: add a way to allow concurrent pushes on unrelated heads Pierre-Yves David <pierre-yves.david@octobus.net> parents: 32516 diff changeset	1946 while len(h) == 20:
16ada4cbb1a9 push: add a way to allow concurrent pushes on unrelated heads Pierre-Yves David <pierre-yves.david@octobus.net> parents: 32516 diff changeset	1947 heads.append(h)
16ada4cbb1a9 push: add a way to allow concurrent pushes on unrelated heads Pierre-Yves David <pierre-yves.david@octobus.net> parents: 32516 diff changeset	1948 h = inpart.read(20)
16ada4cbb1a9 push: add a way to allow concurrent pushes on unrelated heads Pierre-Yves David <pierre-yves.david@octobus.net> parents: 32516 diff changeset	1949 assert not h
16ada4cbb1a9 push: add a way to allow concurrent pushes on unrelated heads Pierre-Yves David <pierre-yves.david@octobus.net> parents: 32516 diff changeset	1950 # trigger a transaction so that we are guaranteed to have the lock now.
16ada4cbb1a9 push: add a way to allow concurrent pushes on unrelated heads Pierre-Yves David <pierre-yves.david@octobus.net> parents: 32516 diff changeset	1951 if op.ui.configbool('experimental', 'bundle2lazylocking'):
16ada4cbb1a9 push: add a way to allow concurrent pushes on unrelated heads Pierre-Yves David <pierre-yves.david@octobus.net> parents: 32516 diff changeset	1952 op.gettransaction()
16ada4cbb1a9 push: add a way to allow concurrent pushes on unrelated heads Pierre-Yves David <pierre-yves.david@octobus.net> parents: 32516 diff changeset	1953
16ada4cbb1a9 push: add a way to allow concurrent pushes on unrelated heads Pierre-Yves David <pierre-yves.david@octobus.net> parents: 32516 diff changeset	1954 currentheads = set()
16ada4cbb1a9 push: add a way to allow concurrent pushes on unrelated heads Pierre-Yves David <pierre-yves.david@octobus.net> parents: 32516 diff changeset	1955 for ls in op.repo.branchmap().itervalues():
16ada4cbb1a9 push: add a way to allow concurrent pushes on unrelated heads Pierre-Yves David <pierre-yves.david@octobus.net> parents: 32516 diff changeset	1956 currentheads.update(ls)
16ada4cbb1a9 push: add a way to allow concurrent pushes on unrelated heads Pierre-Yves David <pierre-yves.david@octobus.net> parents: 32516 diff changeset	1957
16ada4cbb1a9 push: add a way to allow concurrent pushes on unrelated heads Pierre-Yves David <pierre-yves.david@octobus.net> parents: 32516 diff changeset	1958 for h in heads:
16ada4cbb1a9 push: add a way to allow concurrent pushes on unrelated heads Pierre-Yves David <pierre-yves.david@octobus.net> parents: 32516 diff changeset	1959 if h not in currentheads:
16ada4cbb1a9 push: add a way to allow concurrent pushes on unrelated heads Pierre-Yves David <pierre-yves.david@octobus.net> parents: 32516 diff changeset	1960 raise error.PushRaced('repository changed while pushing - '
16ada4cbb1a9 push: add a way to allow concurrent pushes on unrelated heads Pierre-Yves David <pierre-yves.david@octobus.net> parents: 32516 diff changeset	1961 'please try again')
16ada4cbb1a9 push: add a way to allow concurrent pushes on unrelated heads Pierre-Yves David <pierre-yves.david@octobus.net> parents: 32516 diff changeset	1962
34820 a95067b1dca6 phase: introduce a new 'check:phases' part Boris Feld <boris.feld@octobus.net> parents: 34637 diff changeset	1963 @parthandler('check:phases')
a95067b1dca6 phase: introduce a new 'check:phases' part Boris Feld <boris.feld@octobus.net> parents: 34637 diff changeset	1964 def handlecheckphases(op, inpart):
a95067b1dca6 phase: introduce a new 'check:phases' part Boris Feld <boris.feld@octobus.net> parents: 34637 diff changeset	1965 """check that phase boundaries of the repository did not change
a95067b1dca6 phase: introduce a new 'check:phases' part Boris Feld <boris.feld@octobus.net> parents: 34637 diff changeset	1966
a95067b1dca6 phase: introduce a new 'check:phases' part Boris Feld <boris.feld@octobus.net> parents: 34637 diff changeset	1967 This is used to detect a push race.
a95067b1dca6 phase: introduce a new 'check:phases' part Boris Feld <boris.feld@octobus.net> parents: 34637 diff changeset	1968 """
a95067b1dca6 phase: introduce a new 'check:phases' part Boris Feld <boris.feld@octobus.net> parents: 34637 diff changeset	1969 phasetonodes = phases.binarydecode(inpart)
a95067b1dca6 phase: introduce a new 'check:phases' part Boris Feld <boris.feld@octobus.net> parents: 34637 diff changeset	1970 unfi = op.repo.unfiltered()
a95067b1dca6 phase: introduce a new 'check:phases' part Boris Feld <boris.feld@octobus.net> parents: 34637 diff changeset	1971 cl = unfi.changelog
a95067b1dca6 phase: introduce a new 'check:phases' part Boris Feld <boris.feld@octobus.net> parents: 34637 diff changeset	1972 phasecache = unfi._phasecache
a95067b1dca6 phase: introduce a new 'check:phases' part Boris Feld <boris.feld@octobus.net> parents: 34637 diff changeset	1973 msg = ('repository changed while pushing - please try again '
a95067b1dca6 phase: introduce a new 'check:phases' part Boris Feld <boris.feld@octobus.net> parents: 34637 diff changeset	1974 '(%s is %s expected %s)')
a95067b1dca6 phase: introduce a new 'check:phases' part Boris Feld <boris.feld@octobus.net> parents: 34637 diff changeset	1975 for expectedphase, nodes in enumerate(phasetonodes):
a95067b1dca6 phase: introduce a new 'check:phases' part Boris Feld <boris.feld@octobus.net> parents: 34637 diff changeset	1976 for n in nodes:
a95067b1dca6 phase: introduce a new 'check:phases' part Boris Feld <boris.feld@octobus.net> parents: 34637 diff changeset	1977 actualphase = phasecache.phase(unfi, cl.rev(n))
a95067b1dca6 phase: introduce a new 'check:phases' part Boris Feld <boris.feld@octobus.net> parents: 34637 diff changeset	1978 if actualphase != expectedphase:
a95067b1dca6 phase: introduce a new 'check:phases' part Boris Feld <boris.feld@octobus.net> parents: 34637 diff changeset	1979 finalmsg = msg % (nodemod.short(n),
a95067b1dca6 phase: introduce a new 'check:phases' part Boris Feld <boris.feld@octobus.net> parents: 34637 diff changeset	1980 phases.phasenames[actualphase],
a95067b1dca6 phase: introduce a new 'check:phases' part Boris Feld <boris.feld@octobus.net> parents: 34637 diff changeset	1981 phases.phasenames[expectedphase])
a95067b1dca6 phase: introduce a new 'check:phases' part Boris Feld <boris.feld@octobus.net> parents: 34637 diff changeset	1982 raise error.PushRaced(finalmsg)
a95067b1dca6 phase: introduce a new 'check:phases' part Boris Feld <boris.feld@octobus.net> parents: 34637 diff changeset	1983
24686 e0e28e910fa3 bundle2: rename format, parts and config to final names Pierre-Yves David <pierre-yves.david@fb.com> parents: 24648 diff changeset	1984 @parthandler('output')
21131 b7435117d951 bundle2: capture remote stdout while unbundling Pierre-Yves David <pierre-yves.david@fb.com> parents: 21130 diff changeset	1985 def handleoutput(op, inpart):
b7435117d951 bundle2: capture remote stdout while unbundling Pierre-Yves David <pierre-yves.david@fb.com> parents: 21130 diff changeset	1986 """forward output captured on the server to the client"""
b7435117d951 bundle2: capture remote stdout while unbundling Pierre-Yves David <pierre-yves.david@fb.com> parents: 21130 diff changeset	1987 for line in inpart.read().splitlines():
29851 ccd436f7db6d bundle2: localize handleoutput remote prompts Akihiko Odaki <akihiko.odaki.4i@stu.hosei.ac.jp> parents: 29847 diff changeset	1988 op.ui.status(_('remote: %s\n') % line)
21131 b7435117d951 bundle2: capture remote stdout while unbundling Pierre-Yves David <pierre-yves.david@fb.com> parents: 21130 diff changeset	1989
24686 e0e28e910fa3 bundle2: rename format, parts and config to final names Pierre-Yves David <pierre-yves.david@fb.com> parents: 24648 diff changeset	1990 @parthandler('replycaps')
21130 1ff06386217f bundle2: introduce `replycaps` part for on-demand reply Pierre-Yves David <pierre-yves.david@fb.com> parents: 21129 diff changeset	1991 def handlereplycaps(op, inpart):
1ff06386217f bundle2: introduce `replycaps` part for on-demand reply Pierre-Yves David <pierre-yves.david@fb.com> parents: 21129 diff changeset	1992 """Notify that a reply bundle should be created
1ff06386217f bundle2: introduce `replycaps` part for on-demand reply Pierre-Yves David <pierre-yves.david@fb.com> parents: 21129 diff changeset	1993
21138 f469879d27ec bundle2: extract capabilities decoding Pierre-Yves David <pierre-yves.david@fb.com> parents: 21137 diff changeset	1994 The payload contains the capabilities information for the reply"""
f469879d27ec bundle2: extract capabilities decoding Pierre-Yves David <pierre-yves.david@fb.com> parents: 21137 diff changeset	1995 caps = decodecaps(inpart.read())
21130 1ff06386217f bundle2: introduce `replycaps` part for on-demand reply Pierre-Yves David <pierre-yves.david@fb.com> parents: 21129 diff changeset	1996 if op.reply is None:
21135 98fbf3adfd83 bundle2: add capabilities support in `replycaps` part Pierre-Yves David <pierre-yves.david@fb.com> parents: 21134 diff changeset	1997 op.reply = bundle20(op.ui, caps)
21131 b7435117d951 bundle2: capture remote stdout while unbundling Pierre-Yves David <pierre-yves.david@fb.com> parents: 21130 diff changeset	1998
26829 58f1645f72c3 bundle2: attribute remote failures to remote (issue4788) Gregory Szorc <gregory.szorc@gmail.com> parents: 26802 diff changeset	1999 class AbortFromPart(error.Abort):
58f1645f72c3 bundle2: attribute remote failures to remote (issue4788) Gregory Szorc <gregory.szorc@gmail.com> parents: 26802 diff changeset	2000 """Sub-class of Abort that denotes an error from a bundle2 part."""
58f1645f72c3 bundle2: attribute remote failures to remote (issue4788) Gregory Szorc <gregory.szorc@gmail.com> parents: 26802 diff changeset	2001
24686 e0e28e910fa3 bundle2: rename format, parts and config to final names Pierre-Yves David <pierre-yves.david@fb.com> parents: 24648 diff changeset	2002 @parthandler('error:abort', ('message', 'hint'))
24741 bb67e52362d6 bundle2: fix names for error part handler Pierre-Yves David <pierre-yves.david@fb.com> parents: 24733 diff changeset	2003 def handleerrorabort(op, inpart):
21177 952af771bc17 bundle2: gracefully handle abort during unbundle Pierre-Yves David <pierre-yves.david@fb.com> parents: 21176 diff changeset	2004 """Used to transmit abort error over the wire"""
26829 58f1645f72c3 bundle2: attribute remote failures to remote (issue4788) Gregory Szorc <gregory.szorc@gmail.com> parents: 26802 diff changeset	2005 raise AbortFromPart(inpart.params['message'],
58f1645f72c3 bundle2: attribute remote failures to remote (issue4788) Gregory Szorc <gregory.szorc@gmail.com> parents: 26802 diff changeset	2006 hint=inpart.params.get('hint'))
21183 4345274adc4b bundle2: gracefully handle UnknownPartError during unbundle Pierre-Yves David <pierre-yves.david@fb.com> parents: 21179 diff changeset	2007
25493 d8e7b0781ad7 bundle2: convey PushkeyFailed error over the wire Pierre-Yves David <pierre-yves.david@fb.com> parents: 25492 diff changeset	2008 @parthandler('error:pushkey', ('namespace', 'key', 'new', 'old', 'ret',
d8e7b0781ad7 bundle2: convey PushkeyFailed error over the wire Pierre-Yves David <pierre-yves.david@fb.com> parents: 25492 diff changeset	2009 'in-reply-to'))
d8e7b0781ad7 bundle2: convey PushkeyFailed error over the wire Pierre-Yves David <pierre-yves.david@fb.com> parents: 25492 diff changeset	2010 def handleerrorpushkey(op, inpart):
d8e7b0781ad7 bundle2: convey PushkeyFailed error over the wire Pierre-Yves David <pierre-yves.david@fb.com> parents: 25492 diff changeset	2011 """Used to transmit failure of a mandatory pushkey over the wire"""
d8e7b0781ad7 bundle2: convey PushkeyFailed error over the wire Pierre-Yves David <pierre-yves.david@fb.com> parents: 25492 diff changeset	2012 kwargs = {}
d8e7b0781ad7 bundle2: convey PushkeyFailed error over the wire Pierre-Yves David <pierre-yves.david@fb.com> parents: 25492 diff changeset	2013 for name in ('namespace', 'key', 'new', 'old', 'ret'):
d8e7b0781ad7 bundle2: convey PushkeyFailed error over the wire Pierre-Yves David <pierre-yves.david@fb.com> parents: 25492 diff changeset	2014 value = inpart.params.get(name)
d8e7b0781ad7 bundle2: convey PushkeyFailed error over the wire Pierre-Yves David <pierre-yves.david@fb.com> parents: 25492 diff changeset	2015 if value is not None:
d8e7b0781ad7 bundle2: convey PushkeyFailed error over the wire Pierre-Yves David <pierre-yves.david@fb.com> parents: 25492 diff changeset	2016 kwargs[name] = value
36427 247b473f408e bundle2: *strkwargs love on various kwargs constructions Augie Fackler <augie@google.com>* parents: 36273 diff changeset	2017 raise error.PushkeyFailed(inpart.params['in-reply-to'],
247b473f408e bundle2: *strkwargs love on various kwargs constructions Augie Fackler <augie@google.com>* parents: 36273 diff changeset	2018 **pycompat.strkwargs(kwargs))
25493 d8e7b0781ad7 bundle2: convey PushkeyFailed error over the wire Pierre-Yves David <pierre-yves.david@fb.com> parents: 25492 diff changeset	2019
24686 e0e28e910fa3 bundle2: rename format, parts and config to final names Pierre-Yves David <pierre-yves.david@fb.com> parents: 24648 diff changeset	2020 @parthandler('error:unsupportedcontent', ('parttype', 'params'))
24741 bb67e52362d6 bundle2: fix names for error part handler Pierre-Yves David <pierre-yves.david@fb.com> parents: 24733 diff changeset	2021 def handleerrorunsupportedcontent(op, inpart):
21619 292331e906d7 bundle2: rename b2x:error:unknownpart to b2x:error:unsupportedcontent Pierre-Yves David <pierre-yves.david@fb.com> parents: 21618 diff changeset	2022 """Used to transmit unknown content error over the wire"""
21622 457492741007 bundle2: support transmission of params error over the wire Pierre-Yves David <pierre-yves.david@fb.com> parents: 21620 diff changeset	2023 kwargs = {}
21627 3e8bcc90f07c bundle2: support None parttype in BundleValueError Pierre-Yves David <pierre-yves.david@fb.com> parents: 21626 diff changeset	2024 parttype = inpart.params.get('parttype')
3e8bcc90f07c bundle2: support None parttype in BundleValueError Pierre-Yves David <pierre-yves.david@fb.com> parents: 21626 diff changeset	2025 if parttype is not None:
3e8bcc90f07c bundle2: support None parttype in BundleValueError Pierre-Yves David <pierre-yves.david@fb.com> parents: 21626 diff changeset	2026 kwargs['parttype'] = parttype
21622 457492741007 bundle2: support transmission of params error over the wire Pierre-Yves David <pierre-yves.david@fb.com> parents: 21620 diff changeset	2027 params = inpart.params.get('params')
457492741007 bundle2: support transmission of params error over the wire Pierre-Yves David <pierre-yves.david@fb.com> parents: 21620 diff changeset	2028 if params is not None:
457492741007 bundle2: support transmission of params error over the wire Pierre-Yves David <pierre-yves.david@fb.com> parents: 21620 diff changeset	2029 kwargs['params'] = params.split('\0')
457492741007 bundle2: support transmission of params error over the wire Pierre-Yves David <pierre-yves.david@fb.com> parents: 21620 diff changeset	2030
36427 247b473f408e bundle2: *strkwargs love on various kwargs constructions Augie Fackler <augie@google.com>* parents: 36273 diff changeset	2031 raise error.BundleUnknownFeatureError(**pycompat.strkwargs(kwargs))
21186 9f3652e851f8 bundle2: gracefully handle PushRaced error during unbundle Pierre-Yves David <pierre-yves.david@fb.com> parents: 21185 diff changeset	2032
24686 e0e28e910fa3 bundle2: rename format, parts and config to final names Pierre-Yves David <pierre-yves.david@fb.com> parents: 24648 diff changeset	2033 @parthandler('error:pushraced', ('message',))
24741 bb67e52362d6 bundle2: fix names for error part handler Pierre-Yves David <pierre-yves.david@fb.com> parents: 24733 diff changeset	2034 def handleerrorpushraced(op, inpart):
21186 9f3652e851f8 bundle2: gracefully handle PushRaced error during unbundle Pierre-Yves David <pierre-yves.david@fb.com> parents: 21185 diff changeset	2035 """Used to transmit push race error over the wire"""
21611 71b7b3f79a3c bundle2: use the new ``part.params`` dictionary Pierre-Yves David <pierre-yves.david@fb.com> parents: 21610 diff changeset	2036 raise error.ResponseError(_('push failed:'), inpart.params['message'])
21655 35095f332846 bundle: introduce a listkey handler Pierre-Yves David <pierre-yves.david@fb.com> parents: 21644 diff changeset	2037
24686 e0e28e910fa3 bundle2: rename format, parts and config to final names Pierre-Yves David <pierre-yves.david@fb.com> parents: 24648 diff changeset	2038 @parthandler('listkeys', ('namespace',))
21655 35095f332846 bundle: introduce a listkey handler Pierre-Yves David <pierre-yves.david@fb.com> parents: 21644 diff changeset	2039 def handlelistkeys(op, inpart):
35095f332846 bundle: introduce a listkey handler Pierre-Yves David <pierre-yves.david@fb.com> parents: 21644 diff changeset	2040 """retrieve pushkey namespace content stored in a bundle2"""
35095f332846 bundle: introduce a listkey handler Pierre-Yves David <pierre-yves.david@fb.com> parents: 21644 diff changeset	2041 namespace = inpart.params['namespace']
35095f332846 bundle: introduce a listkey handler Pierre-Yves David <pierre-yves.david@fb.com> parents: 21644 diff changeset	2042 r = pushkey.decodekeys(inpart.read())
35095f332846 bundle: introduce a listkey handler Pierre-Yves David <pierre-yves.david@fb.com> parents: 21644 diff changeset	2043 op.records.add('listkeys', (namespace, r))
21660 e87d2a12d41b bundle2: add ``pushkey`` support Pierre-Yves David <pierre-yves.david@fb.com> parents: 21655 diff changeset	2044
24686 e0e28e910fa3 bundle2: rename format, parts and config to final names Pierre-Yves David <pierre-yves.david@fb.com> parents: 24648 diff changeset	2045 @parthandler('pushkey', ('namespace', 'key', 'old', 'new'))
21660 e87d2a12d41b bundle2: add ``pushkey`` support Pierre-Yves David <pierre-yves.david@fb.com> parents: 21655 diff changeset	2046 def handlepushkey(op, inpart):
e87d2a12d41b bundle2: add ``pushkey`` support Pierre-Yves David <pierre-yves.david@fb.com> parents: 21655 diff changeset	2047 """process a pushkey request"""
e87d2a12d41b bundle2: add ``pushkey`` support Pierre-Yves David <pierre-yves.david@fb.com> parents: 21655 diff changeset	2048 dec = pushkey.decode
e87d2a12d41b bundle2: add ``pushkey`` support Pierre-Yves David <pierre-yves.david@fb.com> parents: 21655 diff changeset	2049 namespace = dec(inpart.params['namespace'])
e87d2a12d41b bundle2: add ``pushkey`` support Pierre-Yves David <pierre-yves.david@fb.com> parents: 21655 diff changeset	2050 key = dec(inpart.params['key'])
e87d2a12d41b bundle2: add ``pushkey`` support Pierre-Yves David <pierre-yves.david@fb.com> parents: 21655 diff changeset	2051 old = dec(inpart.params['old'])
e87d2a12d41b bundle2: add ``pushkey`` support Pierre-Yves David <pierre-yves.david@fb.com> parents: 21655 diff changeset	2052 new = dec(inpart.params['new'])
26565 ee1bcb9aa6e4 bundle2: add op.gettransaction() to handlers that need the lock Durham Goode <durham@fb.com> parents: 26542 diff changeset	2053 # Grab the transaction to ensure that we have the lock before performing the
ee1bcb9aa6e4 bundle2: add op.gettransaction() to handlers that need the lock Durham Goode <durham@fb.com> parents: 26542 diff changeset	2054 # pushkey.
ee1bcb9aa6e4 bundle2: add op.gettransaction() to handlers that need the lock Durham Goode <durham@fb.com> parents: 26542 diff changeset	2055 if op.ui.configbool('experimental', 'bundle2lazylocking'):
ee1bcb9aa6e4 bundle2: add op.gettransaction() to handlers that need the lock Durham Goode <durham@fb.com> parents: 26542 diff changeset	2056 op.gettransaction()
21660 e87d2a12d41b bundle2: add ``pushkey`` support Pierre-Yves David <pierre-yves.david@fb.com> parents: 21655 diff changeset	2057 ret = op.repo.pushkey(namespace, key, old, new)
e87d2a12d41b bundle2: add ``pushkey`` support Pierre-Yves David <pierre-yves.david@fb.com> parents: 21655 diff changeset	2058 record = {'namespace': namespace,
e87d2a12d41b bundle2: add ``pushkey`` support Pierre-Yves David <pierre-yves.david@fb.com> parents: 21655 diff changeset	2059 'key': key,
e87d2a12d41b bundle2: add ``pushkey`` support Pierre-Yves David <pierre-yves.david@fb.com> parents: 21655 diff changeset	2060 'old': old,
e87d2a12d41b bundle2: add ``pushkey`` support Pierre-Yves David <pierre-yves.david@fb.com> parents: 21655 diff changeset	2061 'new': new}
e87d2a12d41b bundle2: add ``pushkey`` support Pierre-Yves David <pierre-yves.david@fb.com> parents: 21655 diff changeset	2062 op.records.add('pushkey', record)
e87d2a12d41b bundle2: add ``pushkey`` support Pierre-Yves David <pierre-yves.david@fb.com> parents: 21655 diff changeset	2063 if op.reply is not None:
24686 e0e28e910fa3 bundle2: rename format, parts and config to final names Pierre-Yves David <pierre-yves.david@fb.com> parents: 24648 diff changeset	2064 rpart = op.reply.newpart('reply:pushkey')
33675 24b7c75c6441 bundle2: use bytestr() instead of str() to convert part id to bytes Augie Fackler <augie@google.com> parents: 33674 diff changeset	2065 rpart.addparam(
24b7c75c6441 bundle2: use bytestr() instead of str() to convert part id to bytes Augie Fackler <augie@google.com> parents: 33674 diff changeset	2066 'in-reply-to', pycompat.bytestr(inpart.id), mandatory=False)
21660 e87d2a12d41b bundle2: add ``pushkey`` support Pierre-Yves David <pierre-yves.david@fb.com> parents: 21655 diff changeset	2067 rpart.addparam('return', '%i' % ret, mandatory=False)
25481 6de96cb31176 bundle2: abort when a mandatory pushkey part fails Pierre-Yves David <pierre-yves.david@fb.com> parents: 25401 diff changeset	2068 if inpart.mandatory and not ret:
25484 a5192774e925 bundle2: introduce a PushkeyFail error to abort unbundle on pushkey error Pierre-Yves David <pierre-yves.david@fb.com> parents: 25481 diff changeset	2069 kwargs = {}
a5192774e925 bundle2: introduce a PushkeyFail error to abort unbundle on pushkey error Pierre-Yves David <pierre-yves.david@fb.com> parents: 25481 diff changeset	2070 for key in ('namespace', 'key', 'new', 'old', 'ret'):
a5192774e925 bundle2: introduce a PushkeyFail error to abort unbundle on pushkey error Pierre-Yves David <pierre-yves.david@fb.com> parents: 25481 diff changeset	2071 if key in inpart.params:
a5192774e925 bundle2: introduce a PushkeyFail error to abort unbundle on pushkey error Pierre-Yves David <pierre-yves.david@fb.com> parents: 25481 diff changeset	2072 kwargs[key] = inpart.params[key]
36428 0b697c650b04 bundle2: part id is an int, use %d to make it bytes Augie Fackler <augie@google.com> parents: 36427 diff changeset	2073 raise error.PushkeyFailed(partid='%d' % inpart.id,
36427 247b473f408e bundle2: *strkwargs love on various kwargs constructions Augie Fackler <augie@google.com>* parents: 36273 diff changeset	2074 **pycompat.strkwargs(kwargs))
21660 e87d2a12d41b bundle2: add ``pushkey`` support Pierre-Yves David <pierre-yves.david@fb.com> parents: 21655 diff changeset	2075
35260 af5507203d01 bookmark: introduce a 'bookmarks' part Boris Feld <boris.feld@octobus.net> parents: 35259 diff changeset	2076 @parthandler('bookmarks')
af5507203d01 bookmark: introduce a 'bookmarks' part Boris Feld <boris.feld@octobus.net> parents: 35259 diff changeset	2077 def handlebookmark(op, inpart):
af5507203d01 bookmark: introduce a 'bookmarks' part Boris Feld <boris.feld@octobus.net> parents: 35259 diff changeset	2078 """transmit bookmark information
af5507203d01 bookmark: introduce a 'bookmarks' part Boris Feld <boris.feld@octobus.net> parents: 35259 diff changeset	2079
35266 496154e41968 bundle2: support a 'records' mode for the 'bookmarks' part Boris Feld <boris.feld@octobus.net> parents: 35265 diff changeset	2080 The part contains binary encoded bookmark information.
496154e41968 bundle2: support a 'records' mode for the 'bookmarks' part Boris Feld <boris.feld@octobus.net> parents: 35265 diff changeset	2081
496154e41968 bundle2: support a 'records' mode for the 'bookmarks' part Boris Feld <boris.feld@octobus.net> parents: 35265 diff changeset	2082 The exact behavior of this part can be controlled by the 'bookmarks' mode
496154e41968 bundle2: support a 'records' mode for the 'bookmarks' part Boris Feld <boris.feld@octobus.net> parents: 35265 diff changeset	2083 on the bundle operation.
35260 af5507203d01 bookmark: introduce a 'bookmarks' part Boris Feld <boris.feld@octobus.net> parents: 35259 diff changeset	2084
35266 496154e41968 bundle2: support a 'records' mode for the 'bookmarks' part Boris Feld <boris.feld@octobus.net> parents: 35265 diff changeset	2085 When mode is 'apply' (the default) the bookmark information is applied as
496154e41968 bundle2: support a 'records' mode for the 'bookmarks' part Boris Feld <boris.feld@octobus.net> parents: 35265 diff changeset	2086 is to the unbundling repository. Make sure a 'check:bookmarks' part is
496154e41968 bundle2: support a 'records' mode for the 'bookmarks' part Boris Feld <boris.feld@octobus.net> parents: 35265 diff changeset	2087 issued earlier to check for push races in such update. This behavior is
496154e41968 bundle2: support a 'records' mode for the 'bookmarks' part Boris Feld <boris.feld@octobus.net> parents: 35265 diff changeset	2088 suitable for pushing.
496154e41968 bundle2: support a 'records' mode for the 'bookmarks' part Boris Feld <boris.feld@octobus.net> parents: 35265 diff changeset	2089
496154e41968 bundle2: support a 'records' mode for the 'bookmarks' part Boris Feld <boris.feld@octobus.net> parents: 35265 diff changeset	2090 When mode is 'records', the information is recorded into the 'bookmarks'
496154e41968 bundle2: support a 'records' mode for the 'bookmarks' part Boris Feld <boris.feld@octobus.net> parents: 35265 diff changeset	2091 records of the bundle operation. This behavior is suitable for pulling.
35260 af5507203d01 bookmark: introduce a 'bookmarks' part Boris Feld <boris.feld@octobus.net> parents: 35259 diff changeset	2092 """
af5507203d01 bookmark: introduce a 'bookmarks' part Boris Feld <boris.feld@octobus.net> parents: 35259 diff changeset	2093 changes = bookmarks.binarydecode(inpart)
35261 f392066d127c bookmark: add pushkey hook compatiblity to the bundle2 part Boris Feld <boris.feld@octobus.net> parents: 35260 diff changeset	2094
35266 496154e41968 bundle2: support a 'records' mode for the 'bookmarks' part Boris Feld <boris.feld@octobus.net> parents: 35265 diff changeset	2095 pushkeycompat = op.repo.ui.configbool('server', 'bookmarks-pushkey-compat')
496154e41968 bundle2: support a 'records' mode for the 'bookmarks' part Boris Feld <boris.feld@octobus.net> parents: 35265 diff changeset	2096 bookmarksmode = op.modes.get('bookmarks', 'apply')
35261 f392066d127c bookmark: add pushkey hook compatiblity to the bundle2 part Boris Feld <boris.feld@octobus.net> parents: 35260 diff changeset	2097
35266 496154e41968 bundle2: support a 'records' mode for the 'bookmarks' part Boris Feld <boris.feld@octobus.net> parents: 35265 diff changeset	2098 if bookmarksmode == 'apply':
496154e41968 bundle2: support a 'records' mode for the 'bookmarks' part Boris Feld <boris.feld@octobus.net> parents: 35265 diff changeset	2099 tr = op.gettransaction()
496154e41968 bundle2: support a 'records' mode for the 'bookmarks' part Boris Feld <boris.feld@octobus.net> parents: 35265 diff changeset	2100 bookstore = op.repo._bookmarks
496154e41968 bundle2: support a 'records' mode for the 'bookmarks' part Boris Feld <boris.feld@octobus.net> parents: 35265 diff changeset	2101 if pushkeycompat:
496154e41968 bundle2: support a 'records' mode for the 'bookmarks' part Boris Feld <boris.feld@octobus.net> parents: 35265 diff changeset	2102 allhooks = []
496154e41968 bundle2: support a 'records' mode for the 'bookmarks' part Boris Feld <boris.feld@octobus.net> parents: 35265 diff changeset	2103 for book, node in changes:
496154e41968 bundle2: support a 'records' mode for the 'bookmarks' part Boris Feld <boris.feld@octobus.net> parents: 35265 diff changeset	2104 hookargs = tr.hookargs.copy()
496154e41968 bundle2: support a 'records' mode for the 'bookmarks' part Boris Feld <boris.feld@octobus.net> parents: 35265 diff changeset	2105 hookargs['pushkeycompat'] = '1'
35811 e35320ce8043 bookmarks: fix pushkey compatibility mode (issue5777) Boris Feld <boris.feld@octobus.net> parents: 35804 diff changeset	2106 hookargs['namespace'] = 'bookmarks'
35266 496154e41968 bundle2: support a 'records' mode for the 'bookmarks' part Boris Feld <boris.feld@octobus.net> parents: 35265 diff changeset	2107 hookargs['key'] = book
496154e41968 bundle2: support a 'records' mode for the 'bookmarks' part Boris Feld <boris.feld@octobus.net> parents: 35265 diff changeset	2108 hookargs['old'] = nodemod.hex(bookstore.get(book, ''))
496154e41968 bundle2: support a 'records' mode for the 'bookmarks' part Boris Feld <boris.feld@octobus.net> parents: 35265 diff changeset	2109 hookargs['new'] = nodemod.hex(node if node is not None else '')
496154e41968 bundle2: support a 'records' mode for the 'bookmarks' part Boris Feld <boris.feld@octobus.net> parents: 35265 diff changeset	2110 allhooks.append(hookargs)
496154e41968 bundle2: support a 'records' mode for the 'bookmarks' part Boris Feld <boris.feld@octobus.net> parents: 35265 diff changeset	2111
496154e41968 bundle2: support a 'records' mode for the 'bookmarks' part Boris Feld <boris.feld@octobus.net> parents: 35265 diff changeset	2112 for hookargs in allhooks:
36273 1ec6fd26f841 bundle2: use pycompat.strkwargs as needed Augie Fackler <augie@google.com> parents: 36116 diff changeset	2113 op.repo.hook('prepushkey', throw=True,
1ec6fd26f841 bundle2: use pycompat.strkwargs as needed Augie Fackler <augie@google.com> parents: 36116 diff changeset	2114 **pycompat.strkwargs(hookargs))
35266 496154e41968 bundle2: support a 'records' mode for the 'bookmarks' part Boris Feld <boris.feld@octobus.net> parents: 35265 diff changeset	2115
496154e41968 bundle2: support a 'records' mode for the 'bookmarks' part Boris Feld <boris.feld@octobus.net> parents: 35265 diff changeset	2116 bookstore.applychanges(op.repo, op.gettransaction(), changes)
496154e41968 bundle2: support a 'records' mode for the 'bookmarks' part Boris Feld <boris.feld@octobus.net> parents: 35265 diff changeset	2117
496154e41968 bundle2: support a 'records' mode for the 'bookmarks' part Boris Feld <boris.feld@octobus.net> parents: 35265 diff changeset	2118 if pushkeycompat:
496154e41968 bundle2: support a 'records' mode for the 'bookmarks' part Boris Feld <boris.feld@octobus.net> parents: 35265 diff changeset	2119 def runhook():
496154e41968 bundle2: support a 'records' mode for the 'bookmarks' part Boris Feld <boris.feld@octobus.net> parents: 35265 diff changeset	2120 for hookargs in allhooks:
36273 1ec6fd26f841 bundle2: use pycompat.strkwargs as needed Augie Fackler <augie@google.com> parents: 36116 diff changeset	2121 op.repo.hook('pushkey', **pycompat.strkwargs(hookargs))
35266 496154e41968 bundle2: support a 'records' mode for the 'bookmarks' part Boris Feld <boris.feld@octobus.net> parents: 35265 diff changeset	2122 op.repo._afterlock(runhook)
496154e41968 bundle2: support a 'records' mode for the 'bookmarks' part Boris Feld <boris.feld@octobus.net> parents: 35265 diff changeset	2123
496154e41968 bundle2: support a 'records' mode for the 'bookmarks' part Boris Feld <boris.feld@octobus.net> parents: 35265 diff changeset	2124 elif bookmarksmode == 'records':
35261 f392066d127c bookmark: add pushkey hook compatiblity to the bundle2 part Boris Feld <boris.feld@octobus.net> parents: 35260 diff changeset	2125 for book, node in changes:
35266 496154e41968 bundle2: support a 'records' mode for the 'bookmarks' part Boris Feld <boris.feld@octobus.net> parents: 35265 diff changeset	2126 record = {'bookmark': book, 'node': node}
496154e41968 bundle2: support a 'records' mode for the 'bookmarks' part Boris Feld <boris.feld@octobus.net> parents: 35265 diff changeset	2127 op.records.add('bookmarks', record)
496154e41968 bundle2: support a 'records' mode for the 'bookmarks' part Boris Feld <boris.feld@octobus.net> parents: 35265 diff changeset	2128 else:
496154e41968 bundle2: support a 'records' mode for the 'bookmarks' part Boris Feld <boris.feld@octobus.net> parents: 35265 diff changeset	2129 raise error.ProgrammingError('unkown bookmark mode: %s' % bookmarksmode)
35260 af5507203d01 bookmark: introduce a 'bookmarks' part Boris Feld <boris.feld@octobus.net> parents: 35259 diff changeset	2130
33031 e8c8d81eb864 bundle: add config option to include phases Martin von Zweigbergk <martinvonz@google.com> parents: 33030 diff changeset	2131 @parthandler('phase-heads')
e8c8d81eb864 bundle: add config option to include phases Martin von Zweigbergk <martinvonz@google.com> parents: 33030 diff changeset	2132 def handlephases(op, inpart):
e8c8d81eb864 bundle: add config option to include phases Martin von Zweigbergk <martinvonz@google.com> parents: 33030 diff changeset	2133 """apply phases from bundle part to repo"""
34320 12c42bcd4133 phases: move the binary decoding function in the phases module Boris Feld <boris.feld@octobus.net> parents: 34319 diff changeset	2134 headsbyphase = phases.binarydecode(inpart)
34321 4ef472b975ff bundle2: only grab a transaction when 'phase-heads' affect the repository Boris Feld <boris.feld@octobus.net> parents: 34320 diff changeset	2135 phases.updatephases(op.repo.unfiltered(), op.gettransaction, headsbyphase)
33031 e8c8d81eb864 bundle: add config option to include phases Martin von Zweigbergk <martinvonz@google.com> parents: 33030 diff changeset	2136
24686 e0e28e910fa3 bundle2: rename format, parts and config to final names Pierre-Yves David <pierre-yves.david@fb.com> parents: 24648 diff changeset	2137 @parthandler('reply:pushkey', ('return', 'in-reply-to'))
21660 e87d2a12d41b bundle2: add ``pushkey`` support Pierre-Yves David <pierre-yves.david@fb.com> parents: 21655 diff changeset	2138 def handlepushkeyreply(op, inpart):
e87d2a12d41b bundle2: add ``pushkey`` support Pierre-Yves David <pierre-yves.david@fb.com> parents: 21655 diff changeset	2139 """retrieve the result of a pushkey request"""
e87d2a12d41b bundle2: add ``pushkey`` support Pierre-Yves David <pierre-yves.david@fb.com> parents: 21655 diff changeset	2140 ret = int(inpart.params['return'])
e87d2a12d41b bundle2: add ``pushkey`` support Pierre-Yves David <pierre-yves.david@fb.com> parents: 21655 diff changeset	2141 partid = int(inpart.params['in-reply-to'])
e87d2a12d41b bundle2: add ``pushkey`` support Pierre-Yves David <pierre-yves.david@fb.com> parents: 21655 diff changeset	2142 op.records.add('pushkey', {'return': ret}, partid)
22336 60786c8a2f70 bundle2: add an obsmarkers part handler Pierre-Yves David <pierre-yves.david@fb.com> parents: 21900 diff changeset	2143
24686 e0e28e910fa3 bundle2: rename format, parts and config to final names Pierre-Yves David <pierre-yves.david@fb.com> parents: 24648 diff changeset	2144 @parthandler('obsmarkers')
22336 60786c8a2f70 bundle2: add an obsmarkers part handler Pierre-Yves David <pierre-yves.david@fb.com> parents: 21900 diff changeset	2145 def handleobsmarker(op, inpart):
60786c8a2f70 bundle2: add an obsmarkers part handler Pierre-Yves David <pierre-yves.david@fb.com> parents: 21900 diff changeset	2146 """add a stream of obsmarkers to the repo"""
60786c8a2f70 bundle2: add an obsmarkers part handler Pierre-Yves David <pierre-yves.david@fb.com> parents: 21900 diff changeset	2147 tr = op.gettransaction()
24733 c00e4338fa4b obsolete: experimental flag to get debug about obsmarkers exchange Pierre-Yves David <pierre-yves.david@fb.com> parents: 24686 diff changeset	2148 markerdata = inpart.read()
33499 0407a51b9d8c codemod: register core configitems using a script Jun Wu <quark@fb.com> parents: 33461 diff changeset	2149 if op.ui.config('experimental', 'obsmarkers-exchange-debug'):
24733 c00e4338fa4b obsolete: experimental flag to get debug about obsmarkers exchange Pierre-Yves David <pierre-yves.david@fb.com> parents: 24686 diff changeset	2150 op.ui.write(('obsmarker-exchange: %i bytes received\n')
c00e4338fa4b obsolete: experimental flag to get debug about obsmarkers exchange Pierre-Yves David <pierre-yves.david@fb.com> parents: 24686 diff changeset	2151 % len(markerdata))
26685 ea390d889d3a bundle2: gracefully skip 'obsmarkers' part if evolution is disabled Pierre-Yves David <pierre-yves.david@fb.com> parents: 26587 diff changeset	2152 # The mergemarkers call will crash if marker creation is not enabled.
ea390d889d3a bundle2: gracefully skip 'obsmarkers' part if evolution is disabled Pierre-Yves David <pierre-yves.david@fb.com> parents: 26587 diff changeset	2153 # we want to avoid this if the part is advisory.
ea390d889d3a bundle2: gracefully skip 'obsmarkers' part if evolution is disabled Pierre-Yves David <pierre-yves.david@fb.com> parents: 26587 diff changeset	2154 if not inpart.mandatory and op.repo.obsstore.readonly:
35466 7906354cbc68 debug: add newlines at the end of three locations that appear to need it Kyle Lippincott <spectral@google.com> parents: 35266 diff changeset	2155 op.repo.ui.debug('ignoring obsolescence markers, feature not enabled\n')
26685 ea390d889d3a bundle2: gracefully skip 'obsmarkers' part if evolution is disabled Pierre-Yves David <pierre-yves.david@fb.com> parents: 26587 diff changeset	2156 return
24733 c00e4338fa4b obsolete: experimental flag to get debug about obsmarkers exchange Pierre-Yves David <pierre-yves.david@fb.com> parents: 24686 diff changeset	2157 new = op.repo.obsstore.mergemarkers(tr, markerdata)
32326 99515353c72a obsolete: invalidate "volatile" set cache after merging marker Pierre-Yves David <pierre-yves.david@octobus.net> parents: 32024 diff changeset	2158 op.repo.invalidatevolatilesets()
22337 c380fe290290 obsmarker: write a message with the number of markers added through bundle2 Pierre-Yves David <pierre-yves.david@fb.com> parents: 22336 diff changeset	2159 if new:
c380fe290290 obsmarker: write a message with the number of markers added through bundle2 Pierre-Yves David <pierre-yves.david@fb.com> parents: 22336 diff changeset	2160 op.repo.ui.status(_('%i new obsolescence markers\n') % new)
22338 8fcd56095d3b obssmarker: add a bundle2 record with the number of markers added Pierre-Yves David <pierre-yves.david@fb.com> parents: 22337 diff changeset	2161 op.records.add('obsmarkers', {'new': new})
22340 394a17de6a2d obsmarker: produce a reply part for markers received through bundle2 Pierre-Yves David <pierre-yves.david@fb.com> parents: 22338 diff changeset	2162 if op.reply is not None:
24686 e0e28e910fa3 bundle2: rename format, parts and config to final names Pierre-Yves David <pierre-yves.david@fb.com> parents: 24648 diff changeset	2163 rpart = op.reply.newpart('reply:obsmarkers')
33675 24b7c75c6441 bundle2: use bytestr() instead of str() to convert part id to bytes Augie Fackler <augie@google.com> parents: 33674 diff changeset	2164 rpart.addparam(
24b7c75c6441 bundle2: use bytestr() instead of str() to convert part id to bytes Augie Fackler <augie@google.com> parents: 33674 diff changeset	2165 'in-reply-to', pycompat.bytestr(inpart.id), mandatory=False)
22340 394a17de6a2d obsmarker: produce a reply part for markers received through bundle2 Pierre-Yves David <pierre-yves.david@fb.com> parents: 22338 diff changeset	2166 rpart.addparam('new', '%i' % new, mandatory=False)
394a17de6a2d obsmarker: produce a reply part for markers received through bundle2 Pierre-Yves David <pierre-yves.david@fb.com> parents: 22338 diff changeset	2167
394a17de6a2d obsmarker: produce a reply part for markers received through bundle2 Pierre-Yves David <pierre-yves.david@fb.com> parents: 22338 diff changeset	2168
24686 e0e28e910fa3 bundle2: rename format, parts and config to final names Pierre-Yves David <pierre-yves.david@fb.com> parents: 24648 diff changeset	2169 @parthandler('reply:obsmarkers', ('new', 'in-reply-to'))
25506 86472038dfd2 bundle2: rename duplicate handlepushkeyreply to handleobsmarkerreply Martin von Zweigbergk <martinvonz@google.com> parents: 25493 diff changeset	2170 def handleobsmarkerreply(op, inpart):
22340 394a17de6a2d obsmarker: produce a reply part for markers received through bundle2 Pierre-Yves David <pierre-yves.david@fb.com> parents: 22338 diff changeset	2171 """retrieve the result of a pushkey request"""
394a17de6a2d obsmarker: produce a reply part for markers received through bundle2 Pierre-Yves David <pierre-yves.david@fb.com> parents: 22338 diff changeset	2172 ret = int(inpart.params['new'])
394a17de6a2d obsmarker: produce a reply part for markers received through bundle2 Pierre-Yves David <pierre-yves.david@fb.com> parents: 22338 diff changeset	2173 partid = int(inpart.params['in-reply-to'])
394a17de6a2d obsmarker: produce a reply part for markers received through bundle2 Pierre-Yves David <pierre-yves.david@fb.com> parents: 22338 diff changeset	2174 op.records.add('obsmarkers', {'new': ret}, partid)
25401 d29201352af7 bundle2: part handler for processing .hgtags fnodes mappings Gregory Szorc <gregory.szorc@gmail.com> parents: 25336 diff changeset	2175
d29201352af7 bundle2: part handler for processing .hgtags fnodes mappings Gregory Szorc <gregory.szorc@gmail.com> parents: 25336 diff changeset	2176 @parthandler('hgtagsfnodes')
d29201352af7 bundle2: part handler for processing .hgtags fnodes mappings Gregory Szorc <gregory.szorc@gmail.com> parents: 25336 diff changeset	2177 def handlehgtagsfnodes(op, inpart):
d29201352af7 bundle2: part handler for processing .hgtags fnodes mappings Gregory Szorc <gregory.szorc@gmail.com> parents: 25336 diff changeset	2178 """Applies .hgtags fnodes cache entries to the local repo.
d29201352af7 bundle2: part handler for processing .hgtags fnodes mappings Gregory Szorc <gregory.szorc@gmail.com> parents: 25336 diff changeset	2179
d29201352af7 bundle2: part handler for processing .hgtags fnodes mappings Gregory Szorc <gregory.szorc@gmail.com> parents: 25336 diff changeset	2180 Payload is pairs of 20 byte changeset nodes and filenodes.
d29201352af7 bundle2: part handler for processing .hgtags fnodes mappings Gregory Szorc <gregory.szorc@gmail.com> parents: 25336 diff changeset	2181 """
26565 ee1bcb9aa6e4 bundle2: add op.gettransaction() to handlers that need the lock Durham Goode <durham@fb.com> parents: 26542 diff changeset	2182 # Grab the transaction so we ensure that we have the lock at this point.
ee1bcb9aa6e4 bundle2: add op.gettransaction() to handlers that need the lock Durham Goode <durham@fb.com> parents: 26542 diff changeset	2183 if op.ui.configbool('experimental', 'bundle2lazylocking'):
ee1bcb9aa6e4 bundle2: add op.gettransaction() to handlers that need the lock Durham Goode <durham@fb.com> parents: 26542 diff changeset	2184 op.gettransaction()
25401 d29201352af7 bundle2: part handler for processing .hgtags fnodes mappings Gregory Szorc <gregory.szorc@gmail.com> parents: 25336 diff changeset	2185 cache = tags.hgtagsfnodescache(op.repo.unfiltered())
d29201352af7 bundle2: part handler for processing .hgtags fnodes mappings Gregory Szorc <gregory.szorc@gmail.com> parents: 25336 diff changeset	2186
d29201352af7 bundle2: part handler for processing .hgtags fnodes mappings Gregory Szorc <gregory.szorc@gmail.com> parents: 25336 diff changeset	2187 count = 0
d29201352af7 bundle2: part handler for processing .hgtags fnodes mappings Gregory Szorc <gregory.szorc@gmail.com> parents: 25336 diff changeset	2188 while True:
d29201352af7 bundle2: part handler for processing .hgtags fnodes mappings Gregory Szorc <gregory.szorc@gmail.com> parents: 25336 diff changeset	2189 node = inpart.read(20)
d29201352af7 bundle2: part handler for processing .hgtags fnodes mappings Gregory Szorc <gregory.szorc@gmail.com> parents: 25336 diff changeset	2190 fnode = inpart.read(20)
d29201352af7 bundle2: part handler for processing .hgtags fnodes mappings Gregory Szorc <gregory.szorc@gmail.com> parents: 25336 diff changeset	2191 if len(node) < 20 or len(fnode) < 20:
25641 c0bdfe87b245 bundle2: reword debug message for invalid .hgtags data Gregory Szorc <gregory.szorc@gmail.com> parents: 25640 diff changeset	2192 op.ui.debug('ignoring incomplete received .hgtags fnodes data\n')
25401 d29201352af7 bundle2: part handler for processing .hgtags fnodes mappings Gregory Szorc <gregory.szorc@gmail.com> parents: 25336 diff changeset	2193 break
d29201352af7 bundle2: part handler for processing .hgtags fnodes mappings Gregory Szorc <gregory.szorc@gmail.com> parents: 25336 diff changeset	2194 cache.setfnode(node, fnode)
d29201352af7 bundle2: part handler for processing .hgtags fnodes mappings Gregory Szorc <gregory.szorc@gmail.com> parents: 25336 diff changeset	2195 count += 1
d29201352af7 bundle2: part handler for processing .hgtags fnodes mappings Gregory Szorc <gregory.szorc@gmail.com> parents: 25336 diff changeset	2196
d29201352af7 bundle2: part handler for processing .hgtags fnodes mappings Gregory Szorc <gregory.szorc@gmail.com> parents: 25336 diff changeset	2197 cache.write()
d29201352af7 bundle2: part handler for processing .hgtags fnodes mappings Gregory Szorc <gregory.szorc@gmail.com> parents: 25336 diff changeset	2198 op.ui.debug('applied %i hgtags fnodes cache entries\n' % count)
33693 db3dc11356ed pushvars: move fb extension pushvars to core Pulkit Goyal <7895pulkit@gmail.com> parents: 33682 diff changeset	2199
36963 9988fc10f49e revbranchcache: add a bundle2 handler for a rbc part Boris Feld <boris.feld@octobus.net> parents: 36428 diff changeset	2200 rbcstruct = struct.Struct('>III')
9988fc10f49e revbranchcache: add a bundle2 handler for a rbc part Boris Feld <boris.feld@octobus.net> parents: 36428 diff changeset	2201
9988fc10f49e revbranchcache: add a bundle2 handler for a rbc part Boris Feld <boris.feld@octobus.net> parents: 36428 diff changeset	2202 @parthandler('cache:rev-branch-cache')
9988fc10f49e revbranchcache: add a bundle2 handler for a rbc part Boris Feld <boris.feld@octobus.net> parents: 36428 diff changeset	2203 def handlerbc(op, inpart):
9988fc10f49e revbranchcache: add a bundle2 handler for a rbc part Boris Feld <boris.feld@octobus.net> parents: 36428 diff changeset	2204 """receive a rev-branch-cache payload and update the local cache
9988fc10f49e revbranchcache: add a bundle2 handler for a rbc part Boris Feld <boris.feld@octobus.net> parents: 36428 diff changeset	2205
9988fc10f49e revbranchcache: add a bundle2 handler for a rbc part Boris Feld <boris.feld@octobus.net> parents: 36428 diff changeset	2206 The payload is a series of data related to each branch
9988fc10f49e revbranchcache: add a bundle2 handler for a rbc part Boris Feld <boris.feld@octobus.net> parents: 36428 diff changeset	2207
9988fc10f49e revbranchcache: add a bundle2 handler for a rbc part Boris Feld <boris.feld@octobus.net> parents: 36428 diff changeset	2208 1) branch name length
9988fc10f49e revbranchcache: add a bundle2 handler for a rbc part Boris Feld <boris.feld@octobus.net> parents: 36428 diff changeset	2209 2) number of open heads
9988fc10f49e revbranchcache: add a bundle2 handler for a rbc part Boris Feld <boris.feld@octobus.net> parents: 36428 diff changeset	2210 3) number of closed heads
9988fc10f49e revbranchcache: add a bundle2 handler for a rbc part Boris Feld <boris.feld@octobus.net> parents: 36428 diff changeset	2211 4) open heads nodes
9988fc10f49e revbranchcache: add a bundle2 handler for a rbc part Boris Feld <boris.feld@octobus.net> parents: 36428 diff changeset	2212 5) closed heads nodes
9988fc10f49e revbranchcache: add a bundle2 handler for a rbc part Boris Feld <boris.feld@octobus.net> parents: 36428 diff changeset	2213 """
9988fc10f49e revbranchcache: add a bundle2 handler for a rbc part Boris Feld <boris.feld@octobus.net> parents: 36428 diff changeset	2214 total = 0
9988fc10f49e revbranchcache: add a bundle2 handler for a rbc part Boris Feld <boris.feld@octobus.net> parents: 36428 diff changeset	2215 rawheader = inpart.read(rbcstruct.size)
9988fc10f49e revbranchcache: add a bundle2 handler for a rbc part Boris Feld <boris.feld@octobus.net> parents: 36428 diff changeset	2216 cache = op.repo.revbranchcache()
9988fc10f49e revbranchcache: add a bundle2 handler for a rbc part Boris Feld <boris.feld@octobus.net> parents: 36428 diff changeset	2217 cl = op.repo.unfiltered().changelog
9988fc10f49e revbranchcache: add a bundle2 handler for a rbc part Boris Feld <boris.feld@octobus.net> parents: 36428 diff changeset	2218 while rawheader:
9988fc10f49e revbranchcache: add a bundle2 handler for a rbc part Boris Feld <boris.feld@octobus.net> parents: 36428 diff changeset	2219 header = rbcstruct.unpack(rawheader)
9988fc10f49e revbranchcache: add a bundle2 handler for a rbc part Boris Feld <boris.feld@octobus.net> parents: 36428 diff changeset	2220 total += header[1] + header[2]
9988fc10f49e revbranchcache: add a bundle2 handler for a rbc part Boris Feld <boris.feld@octobus.net> parents: 36428 diff changeset	2221 utf8branch = inpart.read(header[0])
9988fc10f49e revbranchcache: add a bundle2 handler for a rbc part Boris Feld <boris.feld@octobus.net> parents: 36428 diff changeset	2222 branch = encoding.tolocal(utf8branch)
9988fc10f49e revbranchcache: add a bundle2 handler for a rbc part Boris Feld <boris.feld@octobus.net> parents: 36428 diff changeset	2223 for x in xrange(header[1]):
9988fc10f49e revbranchcache: add a bundle2 handler for a rbc part Boris Feld <boris.feld@octobus.net> parents: 36428 diff changeset	2224 node = inpart.read(20)
9988fc10f49e revbranchcache: add a bundle2 handler for a rbc part Boris Feld <boris.feld@octobus.net> parents: 36428 diff changeset	2225 rev = cl.rev(node)
9988fc10f49e revbranchcache: add a bundle2 handler for a rbc part Boris Feld <boris.feld@octobus.net> parents: 36428 diff changeset	2226 cache.setdata(branch, rev, node, False)
9988fc10f49e revbranchcache: add a bundle2 handler for a rbc part Boris Feld <boris.feld@octobus.net> parents: 36428 diff changeset	2227 for x in xrange(header[2]):
9988fc10f49e revbranchcache: add a bundle2 handler for a rbc part Boris Feld <boris.feld@octobus.net> parents: 36428 diff changeset	2228 node = inpart.read(20)
9988fc10f49e revbranchcache: add a bundle2 handler for a rbc part Boris Feld <boris.feld@octobus.net> parents: 36428 diff changeset	2229 rev = cl.rev(node)
9988fc10f49e revbranchcache: add a bundle2 handler for a rbc part Boris Feld <boris.feld@octobus.net> parents: 36428 diff changeset	2230 cache.setdata(branch, rev, node, True)
9988fc10f49e revbranchcache: add a bundle2 handler for a rbc part Boris Feld <boris.feld@octobus.net> parents: 36428 diff changeset	2231 rawheader = inpart.read(rbcstruct.size)
9988fc10f49e revbranchcache: add a bundle2 handler for a rbc part Boris Feld <boris.feld@octobus.net> parents: 36428 diff changeset	2232 cache.write()
9988fc10f49e revbranchcache: add a bundle2 handler for a rbc part Boris Feld <boris.feld@octobus.net> parents: 36428 diff changeset	2233
33693 db3dc11356ed pushvars: move fb extension pushvars to core Pulkit Goyal <7895pulkit@gmail.com> parents: 33682 diff changeset	2234 @parthandler('pushvars')
db3dc11356ed pushvars: move fb extension pushvars to core Pulkit Goyal <7895pulkit@gmail.com> parents: 33682 diff changeset	2235 def bundle2getvars(op, part):
db3dc11356ed pushvars: move fb extension pushvars to core Pulkit Goyal <7895pulkit@gmail.com> parents: 33682 diff changeset	2236 '''unbundle a bundle2 containing shellvars on the server'''
db3dc11356ed pushvars: move fb extension pushvars to core Pulkit Goyal <7895pulkit@gmail.com> parents: 33682 diff changeset	2237 # An option to disable unbundling on server-side for security reasons
33835 057d31ceace3 pushvars: add a coreconfigitem for push.pushvars.server Pulkit Goyal <7895pulkit@gmail.com> parents: 33810 diff changeset	2238 if op.ui.configbool('push', 'pushvars.server'):
33693 db3dc11356ed pushvars: move fb extension pushvars to core Pulkit Goyal <7895pulkit@gmail.com> parents: 33682 diff changeset	2239 hookargs = {}
db3dc11356ed pushvars: move fb extension pushvars to core Pulkit Goyal <7895pulkit@gmail.com> parents: 33682 diff changeset	2240 for key, value in part.advisoryparams:
db3dc11356ed pushvars: move fb extension pushvars to core Pulkit Goyal <7895pulkit@gmail.com> parents: 33682 diff changeset	2241 key = key.upper()
db3dc11356ed pushvars: move fb extension pushvars to core Pulkit Goyal <7895pulkit@gmail.com> parents: 33682 diff changeset	2242 # We want pushed variables to have USERVAR_ prepended so we know
db3dc11356ed pushvars: move fb extension pushvars to core Pulkit Goyal <7895pulkit@gmail.com> parents: 33682 diff changeset	2243 # they came from the --pushvar flag.
db3dc11356ed pushvars: move fb extension pushvars to core Pulkit Goyal <7895pulkit@gmail.com> parents: 33682 diff changeset	2244 key = "USERVAR_" + key
db3dc11356ed pushvars: move fb extension pushvars to core Pulkit Goyal <7895pulkit@gmail.com> parents: 33682 diff changeset	2245 hookargs[key] = value
db3dc11356ed pushvars: move fb extension pushvars to core Pulkit Goyal <7895pulkit@gmail.com> parents: 33682 diff changeset	2246 op.addhookargs(hookargs)
35758 b996ddf5963d bundle2: add a 'stream' part handler for stream cloning Boris Feld <boris.feld@octobus.net> parents: 35655 diff changeset	2247
35788 b116a66bcc44 bundle2: move version of stream clone into part name Gregory Szorc <gregory.szorc@gmail.com> parents: 35787 diff changeset	2248 @parthandler('stream2', ('requirements', 'filecount', 'bytecount'))
b116a66bcc44 bundle2: move version of stream clone into part name Gregory Szorc <gregory.szorc@gmail.com> parents: 35787 diff changeset	2249 def handlestreamv2bundle(op, part):
35758 b996ddf5963d bundle2: add a 'stream' part handler for stream cloning Boris Feld <boris.feld@octobus.net> parents: 35655 diff changeset	2250
35813 768326377e4d bundle2: fix the formatting of the stream part requirements Boris Feld <boris.feld@octobus.net> parents: 35811 diff changeset	2251 requirements = urlreq.unquote(part.params['requirements']).split(',')
35758 b996ddf5963d bundle2: add a 'stream' part handler for stream cloning Boris Feld <boris.feld@octobus.net> parents: 35655 diff changeset	2252 filecount = int(part.params['filecount'])
b996ddf5963d bundle2: add a 'stream' part handler for stream cloning Boris Feld <boris.feld@octobus.net> parents: 35655 diff changeset	2253 bytecount = int(part.params['bytecount'])
b996ddf5963d bundle2: add a 'stream' part handler for stream cloning Boris Feld <boris.feld@octobus.net> parents: 35655 diff changeset	2254
b996ddf5963d bundle2: add a 'stream' part handler for stream cloning Boris Feld <boris.feld@octobus.net> parents: 35655 diff changeset	2255 repo = op.repo
b996ddf5963d bundle2: add a 'stream' part handler for stream cloning Boris Feld <boris.feld@octobus.net> parents: 35655 diff changeset	2256 if len(repo):
b996ddf5963d bundle2: add a 'stream' part handler for stream cloning Boris Feld <boris.feld@octobus.net> parents: 35655 diff changeset	2257 msg = _('cannot apply stream clone to non empty repository')
b996ddf5963d bundle2: add a 'stream' part handler for stream cloning Boris Feld <boris.feld@octobus.net> parents: 35655 diff changeset	2258 raise error.Abort(msg)
b996ddf5963d bundle2: add a 'stream' part handler for stream cloning Boris Feld <boris.feld@octobus.net> parents: 35655 diff changeset	2259
b996ddf5963d bundle2: add a 'stream' part handler for stream cloning Boris Feld <boris.feld@octobus.net> parents: 35655 diff changeset	2260 repo.ui.debug('applying stream bundle\n')
b996ddf5963d bundle2: add a 'stream' part handler for stream cloning Boris Feld <boris.feld@octobus.net> parents: 35655 diff changeset	2261 streamclone.applybundlev2(repo, part, filecount, bytecount,
b996ddf5963d bundle2: add a 'stream' part handler for stream cloning Boris Feld <boris.feld@octobus.net> parents: 35655 diff changeset	2262 requirements)

Mercurial > public > mercurial-scm > hg

annotate mercurial/bundle2.py @ 37178:68ee61822182