mercurial-scm/hg-stable: mercurial/bundle2.py annotate

annotate mercurial/bundle2.py @ 52928:f863fc99bef9

bundle2-part: provide a "__iter__" pass through Since the bundle2 already come in fixed size chunk, there is few need to go have the chunkbuffer slice and recompose them, we can just yield the chunk that the format provides.

author	Pierre-Yves David <pierre-yves.david@octobus.net>
date	Mon, 27 Jan 2025 18:08:34 +0100
parents	b8da7eed1657
children	2787db338b15

rev	line source
20801 9c5183cb9bca bundle2: very first version of a bundle2 bundler Pierre-Yves David <pierre-yves.david@fb.com> parents: diff changeset	1 # bundle2.py - generic container format to transmit arbitrary data.
9c5183cb9bca bundle2: very first version of a bundle2 bundler Pierre-Yves David <pierre-yves.david@fb.com> parents: diff changeset	2 #
9c5183cb9bca bundle2: very first version of a bundle2 bundler Pierre-Yves David <pierre-yves.david@fb.com> parents: diff changeset	3 # Copyright 2013 Facebook, Inc.
9c5183cb9bca bundle2: very first version of a bundle2 bundler Pierre-Yves David <pierre-yves.david@fb.com> parents: diff changeset	4 #
9c5183cb9bca bundle2: very first version of a bundle2 bundler Pierre-Yves David <pierre-yves.david@fb.com> parents: diff changeset	5 # This software may be used and distributed according to the terms of the
9c5183cb9bca bundle2: very first version of a bundle2 bundler Pierre-Yves David <pierre-yves.david@fb.com> parents: diff changeset	6 # GNU General Public License version 2 or any later version.
9c5183cb9bca bundle2: very first version of a bundle2 bundler Pierre-Yves David <pierre-yves.david@fb.com> parents: diff changeset	7 """Handling of the new bundle2 format
9c5183cb9bca bundle2: very first version of a bundle2 bundler Pierre-Yves David <pierre-yves.david@fb.com> parents: diff changeset	8
9c5183cb9bca bundle2: very first version of a bundle2 bundler Pierre-Yves David <pierre-yves.david@fb.com> parents: diff changeset	9 The goal of bundle2 is to act as an atomically packet to transmit a set of
9c5183cb9bca bundle2: very first version of a bundle2 bundler Pierre-Yves David <pierre-yves.david@fb.com> parents: diff changeset	10 payloads in an application agnostic way. It consist in a sequence of "parts"
9c5183cb9bca bundle2: very first version of a bundle2 bundler Pierre-Yves David <pierre-yves.david@fb.com> parents: diff changeset	11 that will be handed to and processed by the application layer.
9c5183cb9bca bundle2: very first version of a bundle2 bundler Pierre-Yves David <pierre-yves.david@fb.com> parents: diff changeset	12
9c5183cb9bca bundle2: very first version of a bundle2 bundler Pierre-Yves David <pierre-yves.david@fb.com> parents: diff changeset	13
9c5183cb9bca bundle2: very first version of a bundle2 bundler Pierre-Yves David <pierre-yves.david@fb.com> parents: diff changeset	14 General format architecture
9c5183cb9bca bundle2: very first version of a bundle2 bundler Pierre-Yves David <pierre-yves.david@fb.com> parents: diff changeset	15 ===========================
9c5183cb9bca bundle2: very first version of a bundle2 bundler Pierre-Yves David <pierre-yves.david@fb.com> parents: diff changeset	16
9c5183cb9bca bundle2: very first version of a bundle2 bundler Pierre-Yves David <pierre-yves.david@fb.com> parents: diff changeset	17 The format is architectured as follow
9c5183cb9bca bundle2: very first version of a bundle2 bundler Pierre-Yves David <pierre-yves.david@fb.com> parents: diff changeset	18
9c5183cb9bca bundle2: very first version of a bundle2 bundler Pierre-Yves David <pierre-yves.david@fb.com> parents: diff changeset	19 - magic string
9c5183cb9bca bundle2: very first version of a bundle2 bundler Pierre-Yves David <pierre-yves.david@fb.com> parents: diff changeset	20 - stream level parameters
9c5183cb9bca bundle2: very first version of a bundle2 bundler Pierre-Yves David <pierre-yves.david@fb.com> parents: diff changeset	21 - payload parts (any number)
9c5183cb9bca bundle2: very first version of a bundle2 bundler Pierre-Yves David <pierre-yves.david@fb.com> parents: diff changeset	22 - end of stream marker.
9c5183cb9bca bundle2: very first version of a bundle2 bundler Pierre-Yves David <pierre-yves.david@fb.com> parents: diff changeset	23
20856 8a6a86c9a5b5 bundle2: support bundling of empty part (with a type) Pierre-Yves David <pierre-yves.david@fb.com> parents: 20844 diff changeset	24 the Binary format
20801 9c5183cb9bca bundle2: very first version of a bundle2 bundler Pierre-Yves David <pierre-yves.david@fb.com> parents: diff changeset	25 ============================
9c5183cb9bca bundle2: very first version of a bundle2 bundler Pierre-Yves David <pierre-yves.david@fb.com> parents: diff changeset	26
21024 7731a2281cf0 spelling: fixes from spell checker Mads Kiilerich <madski@unity3d.com> parents: 21020 diff changeset	27 All numbers are unsigned and big-endian.
20801 9c5183cb9bca bundle2: very first version of a bundle2 bundler Pierre-Yves David <pierre-yves.david@fb.com> parents: diff changeset	28
9c5183cb9bca bundle2: very first version of a bundle2 bundler Pierre-Yves David <pierre-yves.david@fb.com> parents: diff changeset	29 stream level parameters
9c5183cb9bca bundle2: very first version of a bundle2 bundler Pierre-Yves David <pierre-yves.david@fb.com> parents: diff changeset	30 ------------------------
9c5183cb9bca bundle2: very first version of a bundle2 bundler Pierre-Yves David <pierre-yves.david@fb.com> parents: diff changeset	31
9c5183cb9bca bundle2: very first version of a bundle2 bundler Pierre-Yves David <pierre-yves.david@fb.com> parents: diff changeset	32 Binary format is as follow
9c5183cb9bca bundle2: very first version of a bundle2 bundler Pierre-Yves David <pierre-yves.david@fb.com> parents: diff changeset	33
23009 90f86ad3d4ff bundle2: change header size and make them signed (new format) Pierre-Yves David <pierre-yves.david@fb.com> parents: 23008 diff changeset	34 :params size: int32
20801 9c5183cb9bca bundle2: very first version of a bundle2 bundler Pierre-Yves David <pierre-yves.david@fb.com> parents: diff changeset	35
9c5183cb9bca bundle2: very first version of a bundle2 bundler Pierre-Yves David <pierre-yves.david@fb.com> parents: diff changeset	36 The total number of Bytes used by the parameters
9c5183cb9bca bundle2: very first version of a bundle2 bundler Pierre-Yves David <pierre-yves.david@fb.com> parents: diff changeset	37
9c5183cb9bca bundle2: very first version of a bundle2 bundler Pierre-Yves David <pierre-yves.david@fb.com> parents: diff changeset	38 :params value: arbitrary number of Bytes
9c5183cb9bca bundle2: very first version of a bundle2 bundler Pierre-Yves David <pierre-yves.david@fb.com> parents: diff changeset	39
9c5183cb9bca bundle2: very first version of a bundle2 bundler Pierre-Yves David <pierre-yves.david@fb.com> parents: diff changeset	40 A blob of `params size` containing the serialized version of all stream level
9c5183cb9bca bundle2: very first version of a bundle2 bundler Pierre-Yves David <pierre-yves.david@fb.com> parents: diff changeset	41 parameters.
9c5183cb9bca bundle2: very first version of a bundle2 bundler Pierre-Yves David <pierre-yves.david@fb.com> parents: diff changeset	42
21024 7731a2281cf0 spelling: fixes from spell checker Mads Kiilerich <madski@unity3d.com> parents: 21020 diff changeset	43 The blob contains a space separated list of parameters. Parameters with value
20811 9785c3f8f598 bundle2: urlquote stream parameter name and value Pierre-Yves David <pierre-yves.david@fb.com> parents: 20810 diff changeset	44 are stored in the form `<name>=<value>`. Both name and value are urlquoted.
20804 db9d3991d2c6 bundle2: support bundling simple parameter Pierre-Yves David <pierre-yves.david@fb.com> parents: 20803 diff changeset	45
20813 8c74b3ce5b70 bundle2: refuse empty parameter name Pierre-Yves David <pierre-yves.david@fb.com> parents: 20812 diff changeset	46 Empty name are obviously forbidden.
8c74b3ce5b70 bundle2: refuse empty parameter name Pierre-Yves David <pierre-yves.david@fb.com> parents: 20812 diff changeset	47
20844 2631204d7305 bundle2: implement the mandatory/advisory logic for parameter Pierre-Yves David <pierre-yves.david@fb.com> parents: 20843 diff changeset	48 Name MUST start with a letter. If this first letter is lower case, the
21024 7731a2281cf0 spelling: fixes from spell checker Mads Kiilerich <madski@unity3d.com> parents: 21020 diff changeset	49 parameter is advisory and can be safely ignored. However when the first
20844 2631204d7305 bundle2: implement the mandatory/advisory logic for parameter Pierre-Yves David <pierre-yves.david@fb.com> parents: 20843 diff changeset	50 letter is capital, the parameter is mandatory and the bundling process MUST
2631204d7305 bundle2: implement the mandatory/advisory logic for parameter Pierre-Yves David <pierre-yves.david@fb.com> parents: 20843 diff changeset	51 stop if he is not able to proceed it.
20814 8532f5e1b9df bundle2: force the first char of parameter to be an letter. Pierre-Yves David <pierre-yves.david@fb.com> parents: 20813 diff changeset	52
20808 4c9130c7a29f bundle2: clarify stream parameter design in the documentation Pierre-Yves David <pierre-yves.david@fb.com> parents: 20805 diff changeset	53 Stream parameters use a simple textual format for two main reasons:
20804 db9d3991d2c6 bundle2: support bundling simple parameter Pierre-Yves David <pierre-yves.david@fb.com> parents: 20803 diff changeset	54
21024 7731a2281cf0 spelling: fixes from spell checker Mads Kiilerich <madski@unity3d.com> parents: 21020 diff changeset	55 - Stream level parameters should remain simple and we want to discourage any
20808 4c9130c7a29f bundle2: clarify stream parameter design in the documentation Pierre-Yves David <pierre-yves.david@fb.com> parents: 20805 diff changeset	56 crazy usage.
21024 7731a2281cf0 spelling: fixes from spell checker Mads Kiilerich <madski@unity3d.com> parents: 21020 diff changeset	57 - Textual data allow easy human inspection of a bundle2 header in case of
20808 4c9130c7a29f bundle2: clarify stream parameter design in the documentation Pierre-Yves David <pierre-yves.david@fb.com> parents: 20805 diff changeset	58 troubles.
4c9130c7a29f bundle2: clarify stream parameter design in the documentation Pierre-Yves David <pierre-yves.david@fb.com> parents: 20805 diff changeset	59
4c9130c7a29f bundle2: clarify stream parameter design in the documentation Pierre-Yves David <pierre-yves.david@fb.com> parents: 20805 diff changeset	60 Any Applicative level options MUST go into a bundle2 part instead.
20801 9c5183cb9bca bundle2: very first version of a bundle2 bundler Pierre-Yves David <pierre-yves.david@fb.com> parents: diff changeset	61
9c5183cb9bca bundle2: very first version of a bundle2 bundler Pierre-Yves David <pierre-yves.david@fb.com> parents: diff changeset	62 Payload part
9c5183cb9bca bundle2: very first version of a bundle2 bundler Pierre-Yves David <pierre-yves.david@fb.com> parents: diff changeset	63 ------------------------
9c5183cb9bca bundle2: very first version of a bundle2 bundler Pierre-Yves David <pierre-yves.david@fb.com> parents: diff changeset	64
9c5183cb9bca bundle2: very first version of a bundle2 bundler Pierre-Yves David <pierre-yves.david@fb.com> parents: diff changeset	65 Binary format is as follow
9c5183cb9bca bundle2: very first version of a bundle2 bundler Pierre-Yves David <pierre-yves.david@fb.com> parents: diff changeset	66
23009 90f86ad3d4ff bundle2: change header size and make them signed (new format) Pierre-Yves David <pierre-yves.david@fb.com> parents: 23008 diff changeset	67 :header size: int32
20801 9c5183cb9bca bundle2: very first version of a bundle2 bundler Pierre-Yves David <pierre-yves.david@fb.com> parents: diff changeset	68
25507 5bee483705e8 bundle2: clarify in docstring that header size is for a single header Martin von Zweigbergk <martinvonz@google.com> parents: 25506 diff changeset	69 The total number of Bytes used by the part header. When the header is empty
20801 9c5183cb9bca bundle2: very first version of a bundle2 bundler Pierre-Yves David <pierre-yves.david@fb.com> parents: diff changeset	70 (size = 0) this is interpreted as the end of stream marker.
9c5183cb9bca bundle2: very first version of a bundle2 bundler Pierre-Yves David <pierre-yves.david@fb.com> parents: diff changeset	71
20856 8a6a86c9a5b5 bundle2: support bundling of empty part (with a type) Pierre-Yves David <pierre-yves.david@fb.com> parents: 20844 diff changeset	72 :header:
8a6a86c9a5b5 bundle2: support bundling of empty part (with a type) Pierre-Yves David <pierre-yves.david@fb.com> parents: 20844 diff changeset	73
8a6a86c9a5b5 bundle2: support bundling of empty part (with a type) Pierre-Yves David <pierre-yves.david@fb.com> parents: 20844 diff changeset	74 The header defines how to interpret the part. It contains two piece of
8a6a86c9a5b5 bundle2: support bundling of empty part (with a type) Pierre-Yves David <pierre-yves.david@fb.com> parents: 20844 diff changeset	75 data: the part type, and the part parameters.
8a6a86c9a5b5 bundle2: support bundling of empty part (with a type) Pierre-Yves David <pierre-yves.david@fb.com> parents: 20844 diff changeset	76
8a6a86c9a5b5 bundle2: support bundling of empty part (with a type) Pierre-Yves David <pierre-yves.david@fb.com> parents: 20844 diff changeset	77 The part type is used to route an application level handler, that can
8a6a86c9a5b5 bundle2: support bundling of empty part (with a type) Pierre-Yves David <pierre-yves.david@fb.com> parents: 20844 diff changeset	78 interpret payload.
8a6a86c9a5b5 bundle2: support bundling of empty part (with a type) Pierre-Yves David <pierre-yves.david@fb.com> parents: 20844 diff changeset	79
8a6a86c9a5b5 bundle2: support bundling of empty part (with a type) Pierre-Yves David <pierre-yves.david@fb.com> parents: 20844 diff changeset	80 Part parameters are passed to the application level handler. They are
8a6a86c9a5b5 bundle2: support bundling of empty part (with a type) Pierre-Yves David <pierre-yves.david@fb.com> parents: 20844 diff changeset	81 meant to convey information that will help the application level object to
8a6a86c9a5b5 bundle2: support bundling of empty part (with a type) Pierre-Yves David <pierre-yves.david@fb.com> parents: 20844 diff changeset	82 interpret the part payload.
8a6a86c9a5b5 bundle2: support bundling of empty part (with a type) Pierre-Yves David <pierre-yves.david@fb.com> parents: 20844 diff changeset	83
8a6a86c9a5b5 bundle2: support bundling of empty part (with a type) Pierre-Yves David <pierre-yves.david@fb.com> parents: 20844 diff changeset	84 The binary format of the header is has follow
8a6a86c9a5b5 bundle2: support bundling of empty part (with a type) Pierre-Yves David <pierre-yves.david@fb.com> parents: 20844 diff changeset	85
8a6a86c9a5b5 bundle2: support bundling of empty part (with a type) Pierre-Yves David <pierre-yves.david@fb.com> parents: 20844 diff changeset	86 :typesize: (one byte)
20877 9e9e3a4e9261 bundle2: part params Pierre-Yves David <pierre-yves.david@fb.com> parents: 20876 diff changeset	87
23916 a3f7c781786b bundle2: fix parttype enforcement Matt Mackall <mpm@selenic.com> parents: 23868 diff changeset	88 :parttype: alphanumerical part name (restricted to [a-zA-Z0-9_:-]*)
20877 9e9e3a4e9261 bundle2: part params Pierre-Yves David <pierre-yves.david@fb.com> parents: 20876 diff changeset	89
20995 e995d104c87f bundle2: add an integer id to part Pierre-Yves David <pierre-yves.david@fb.com> parents: 20952 diff changeset	90 :partid: A 32bits integer (unique in the bundle) that can be used to refer
e995d104c87f bundle2: add an integer id to part Pierre-Yves David <pierre-yves.david@fb.com> parents: 20952 diff changeset	91 to this part.
e995d104c87f bundle2: add an integer id to part Pierre-Yves David <pierre-yves.david@fb.com> parents: 20952 diff changeset	92
20877 9e9e3a4e9261 bundle2: part params Pierre-Yves David <pierre-yves.david@fb.com> parents: 20876 diff changeset	93 :parameters:
9e9e3a4e9261 bundle2: part params Pierre-Yves David <pierre-yves.david@fb.com> parents: 20876 diff changeset	94
21024 7731a2281cf0 spelling: fixes from spell checker Mads Kiilerich <madski@unity3d.com> parents: 21020 diff changeset	95 Part's parameter may have arbitrary content, the binary structure is::
20877 9e9e3a4e9261 bundle2: part params Pierre-Yves David <pierre-yves.david@fb.com> parents: 20876 diff changeset	96
9e9e3a4e9261 bundle2: part params Pierre-Yves David <pierre-yves.david@fb.com> parents: 20876 diff changeset	97 <mandatory-count><advisory-count><param-sizes><param-data>
9e9e3a4e9261 bundle2: part params Pierre-Yves David <pierre-yves.david@fb.com> parents: 20876 diff changeset	98
9e9e3a4e9261 bundle2: part params Pierre-Yves David <pierre-yves.david@fb.com> parents: 20876 diff changeset	99 :mandatory-count: 1 byte, number of mandatory parameters
9e9e3a4e9261 bundle2: part params Pierre-Yves David <pierre-yves.david@fb.com> parents: 20876 diff changeset	100
9e9e3a4e9261 bundle2: part params Pierre-Yves David <pierre-yves.david@fb.com> parents: 20876 diff changeset	101 :advisory-count: 1 byte, number of advisory parameters
9e9e3a4e9261 bundle2: part params Pierre-Yves David <pierre-yves.david@fb.com> parents: 20876 diff changeset	102
9e9e3a4e9261 bundle2: part params Pierre-Yves David <pierre-yves.david@fb.com> parents: 20876 diff changeset	103 :param-sizes:
9e9e3a4e9261 bundle2: part params Pierre-Yves David <pierre-yves.david@fb.com> parents: 20876 diff changeset	104
9e9e3a4e9261 bundle2: part params Pierre-Yves David <pierre-yves.david@fb.com> parents: 20876 diff changeset	105 N couple of bytes, where N is the total number of parameters. Each
9e9e3a4e9261 bundle2: part params Pierre-Yves David <pierre-yves.david@fb.com> parents: 20876 diff changeset	106 couple contains (<size-of-key>, <size-of-value) for one parameter.
9e9e3a4e9261 bundle2: part params Pierre-Yves David <pierre-yves.david@fb.com> parents: 20876 diff changeset	107
9e9e3a4e9261 bundle2: part params Pierre-Yves David <pierre-yves.david@fb.com> parents: 20876 diff changeset	108 :param-data:
9e9e3a4e9261 bundle2: part params Pierre-Yves David <pierre-yves.david@fb.com> parents: 20876 diff changeset	109
9e9e3a4e9261 bundle2: part params Pierre-Yves David <pierre-yves.david@fb.com> parents: 20876 diff changeset	110 A blob of bytes from which each parameter key and value can be
9e9e3a4e9261 bundle2: part params Pierre-Yves David <pierre-yves.david@fb.com> parents: 20876 diff changeset	111 retrieved using the list of size couples stored in the previous
9e9e3a4e9261 bundle2: part params Pierre-Yves David <pierre-yves.david@fb.com> parents: 20876 diff changeset	112 field.
9e9e3a4e9261 bundle2: part params Pierre-Yves David <pierre-yves.david@fb.com> parents: 20876 diff changeset	113
9e9e3a4e9261 bundle2: part params Pierre-Yves David <pierre-yves.david@fb.com> parents: 20876 diff changeset	114 Mandatory parameters comes first, then the advisory ones.
20856 8a6a86c9a5b5 bundle2: support bundling of empty part (with a type) Pierre-Yves David <pierre-yves.david@fb.com> parents: 20844 diff changeset	115
21607 054fa5176fa7 bundle2: forbid duplicate parameter keys Pierre-Yves David <pierre-yves.david@fb.com> parents: 21606 diff changeset	116 Each parameter's key MUST be unique within the part.
054fa5176fa7 bundle2: forbid duplicate parameter keys Pierre-Yves David <pierre-yves.david@fb.com> parents: 21606 diff changeset	117
20856 8a6a86c9a5b5 bundle2: support bundling of empty part (with a type) Pierre-Yves David <pierre-yves.david@fb.com> parents: 20844 diff changeset	118 :payload:
8a6a86c9a5b5 bundle2: support bundling of empty part (with a type) Pierre-Yves David <pierre-yves.david@fb.com> parents: 20844 diff changeset	119
20876 ddd56f3eb786 bundle2: support for bundling and unbundling payload Pierre-Yves David <pierre-yves.david@fb.com> parents: 20864 diff changeset	120 payload is a series of `<chunksize><chunkdata>`.
ddd56f3eb786 bundle2: support for bundling and unbundling payload Pierre-Yves David <pierre-yves.david@fb.com> parents: 20864 diff changeset	121
23009 90f86ad3d4ff bundle2: change header size and make them signed (new format) Pierre-Yves David <pierre-yves.david@fb.com> parents: 23008 diff changeset	122 `chunksize` is an int32, `chunkdata` are plain bytes (as much as
20876 ddd56f3eb786 bundle2: support for bundling and unbundling payload Pierre-Yves David <pierre-yves.david@fb.com> parents: 20864 diff changeset	123 `chunksize` says)` The payload part is concluded by a zero size chunk.
ddd56f3eb786 bundle2: support for bundling and unbundling payload Pierre-Yves David <pierre-yves.david@fb.com> parents: 20864 diff changeset	124
ddd56f3eb786 bundle2: support for bundling and unbundling payload Pierre-Yves David <pierre-yves.david@fb.com> parents: 20864 diff changeset	125 The current implementation always produces either zero or one chunk.
21024 7731a2281cf0 spelling: fixes from spell checker Mads Kiilerich <madski@unity3d.com> parents: 21020 diff changeset	126 This is an implementation limitation that will ultimately be lifted.
20891 1c6cd23fc221 bundle2: add some distinction between mandatory and advisory part Pierre-Yves David <pierre-yves.david@fb.com> parents: 20890 diff changeset	127
23009 90f86ad3d4ff bundle2: change header size and make them signed (new format) Pierre-Yves David <pierre-yves.david@fb.com> parents: 23008 diff changeset	128 `chunksize` can be negative to trigger special case processing. No such
90f86ad3d4ff bundle2: change header size and make them signed (new format) Pierre-Yves David <pierre-yves.david@fb.com> parents: 23008 diff changeset	129 processing is in place yet.
90f86ad3d4ff bundle2: change header size and make them signed (new format) Pierre-Yves David <pierre-yves.david@fb.com> parents: 23008 diff changeset	130
20891 1c6cd23fc221 bundle2: add some distinction between mandatory and advisory part Pierre-Yves David <pierre-yves.david@fb.com> parents: 20890 diff changeset	131 Bundle processing
1c6cd23fc221 bundle2: add some distinction between mandatory and advisory part Pierre-Yves David <pierre-yves.david@fb.com> parents: 20890 diff changeset	132 ============================
1c6cd23fc221 bundle2: add some distinction between mandatory and advisory part Pierre-Yves David <pierre-yves.david@fb.com> parents: 20890 diff changeset	133
1c6cd23fc221 bundle2: add some distinction between mandatory and advisory part Pierre-Yves David <pierre-yves.david@fb.com> parents: 20890 diff changeset	134 Each part is processed in order using a "part handler". Handler are registered
1c6cd23fc221 bundle2: add some distinction between mandatory and advisory part Pierre-Yves David <pierre-yves.david@fb.com> parents: 20890 diff changeset	135 for a certain part type.
1c6cd23fc221 bundle2: add some distinction between mandatory and advisory part Pierre-Yves David <pierre-yves.david@fb.com> parents: 20890 diff changeset	136
1c6cd23fc221 bundle2: add some distinction between mandatory and advisory part Pierre-Yves David <pierre-yves.david@fb.com> parents: 20890 diff changeset	137 The matching of a part to its handler is case insensitive. The case of the
1c6cd23fc221 bundle2: add some distinction between mandatory and advisory part Pierre-Yves David <pierre-yves.david@fb.com> parents: 20890 diff changeset	138 part type is used to know if a part is mandatory or advisory. If the Part type
1c6cd23fc221 bundle2: add some distinction between mandatory and advisory part Pierre-Yves David <pierre-yves.david@fb.com> parents: 20890 diff changeset	139 contains any uppercase char it is considered mandatory. When no handler is
1c6cd23fc221 bundle2: add some distinction between mandatory and advisory part Pierre-Yves David <pierre-yves.david@fb.com> parents: 20890 diff changeset	140 known for a Mandatory part, the process is aborted and an exception is raised.
20892 6fe95448596d bundle2: read the whole bundle from stream on abort Pierre-Yves David <pierre-yves.david@fb.com> parents: 20891 diff changeset	141 If the part is advisory and no handler is known, the part is ignored. When the
6fe95448596d bundle2: read the whole bundle from stream on abort Pierre-Yves David <pierre-yves.david@fb.com> parents: 20891 diff changeset	142 process is aborted, the full bundle is still read from the stream to keep the
6fe95448596d bundle2: read the whole bundle from stream on abort Pierre-Yves David <pierre-yves.david@fb.com> parents: 20891 diff changeset	143 channel usable. But none of the part read from an abort are processed. In the
6fe95448596d bundle2: read the whole bundle from stream on abort Pierre-Yves David <pierre-yves.david@fb.com> parents: 20891 diff changeset	144 future, dropping the stream may become an option for channel we do not care to
6fe95448596d bundle2: read the whole bundle from stream on abort Pierre-Yves David <pierre-yves.david@fb.com> parents: 20891 diff changeset	145 preserve.
20801 9c5183cb9bca bundle2: very first version of a bundle2 bundler Pierre-Yves David <pierre-yves.david@fb.com> parents: diff changeset	146 """
9c5183cb9bca bundle2: very first version of a bundle2 bundler Pierre-Yves David <pierre-yves.david@fb.com> parents: diff changeset	147
51901 f4733654f144 typing: add `from __future__ import annotations` to most files Matt Harbison <matt_harbison@yahoo.com> parents: 51900 diff changeset	148 from __future__ import annotations
25919 8221fefaea08 bundle2: use absolute_import Gregory Szorc <gregory.szorc@gmail.com> parents: 25660 diff changeset	149
36970 79b73be4dda5 rev-branch-cache: add a function to generate a part Boris Feld <boris.feld@octobus.net> parents: 36969 diff changeset	150 import collections
24026 3daef83a1873 bundle2.unpackermixin: control for underlying file descriptor Eric Sumner <ericsumner@fb.com> parents: 23916 diff changeset	151 import errno
35069 241d9caca11e bundle2: use os.SEEK_* constants Gregory Szorc <gregory.szorc@gmail.com> parents: 34820 diff changeset	152 import os
25919 8221fefaea08 bundle2: use absolute_import Gregory Szorc <gregory.szorc@gmail.com> parents: 25660 diff changeset	153 import re
20814 8532f5e1b9df bundle2: force the first char of parameter to be an letter. Pierre-Yves David <pierre-yves.david@fb.com> parents: 20813 diff changeset	154 import string
25919 8221fefaea08 bundle2: use absolute_import Gregory Szorc <gregory.szorc@gmail.com> parents: 25660 diff changeset	155 import struct
8221fefaea08 bundle2: use absolute_import Gregory Szorc <gregory.szorc@gmail.com> parents: 25660 diff changeset	156 import sys
51731 6fc31e7bd5db typing: add some type hints for bundle2 capabilities Matt Harbison <matt_harbison@yahoo.com> parents: 51658 diff changeset	157 import typing
20804 db9d3991d2c6 bundle2: support bundling simple parameter Pierre-Yves David <pierre-yves.david@fb.com> parents: 20803 diff changeset	158
25919 8221fefaea08 bundle2: use absolute_import Gregory Szorc <gregory.szorc@gmail.com> parents: 25660 diff changeset	159 from .i18n import _
46114 59fa3890d40a node: import symbols explicitly Joerg Sonnenberger <joerg@bec.de> parents: 45957 diff changeset	160 from .node import (
59fa3890d40a node: import symbols explicitly Joerg Sonnenberger <joerg@bec.de> parents: 45957 diff changeset	161 hex,
59fa3890d40a node: import symbols explicitly Joerg Sonnenberger <joerg@bec.de> parents: 45957 diff changeset	162 short,
59fa3890d40a node: import symbols explicitly Joerg Sonnenberger <joerg@bec.de> parents: 45957 diff changeset	163 )
25919 8221fefaea08 bundle2: use absolute_import Gregory Szorc <gregory.szorc@gmail.com> parents: 25660 diff changeset	164 from . import (
35267 dbf868623daf bookmark: add a 'check:bookmarks' bundle2 part Boris Feld <boris.feld@octobus.net> parents: 35141 diff changeset	165 bookmarks,
25919 8221fefaea08 bundle2: use absolute_import Gregory Szorc <gregory.szorc@gmail.com> parents: 25660 diff changeset	166 changegroup,
36969 9988fc10f49e revbranchcache: add a bundle2 handler for a rbc part Boris Feld <boris.feld@octobus.net> parents: 36458 diff changeset	167 encoding,
25919 8221fefaea08 bundle2: use absolute_import Gregory Szorc <gregory.szorc@gmail.com> parents: 25660 diff changeset	168 error,
8221fefaea08 bundle2: use absolute_import Gregory Szorc <gregory.szorc@gmail.com> parents: 25660 diff changeset	169 obsolete,
33043 e8c8d81eb864 bundle: add config option to include phases Martin von Zweigbergk <martinvonz@google.com> parents: 33042 diff changeset	170 phases,
25919 8221fefaea08 bundle2: use absolute_import Gregory Szorc <gregory.szorc@gmail.com> parents: 25660 diff changeset	171 pushkey,
30031 0f6d6fdd3c2a pycompat: provide 'ispy3' constant Yuya Nishihara <yuya@tcha.org> parents: 29941 diff changeset	172 pycompat,
45392 77b8588dd84e requirements: introduce new requirements related module Pulkit Goyal <7895pulkit@gmail.com> parents: 45391 diff changeset	173 requirements,
45106 a03c177a4679 scmutil: add writereporequirements() and route requires writing through it Pulkit Goyal <7895pulkit@gmail.com> parents: 44406 diff changeset	174 scmutil,
35758 b996ddf5963d bundle2: add a 'stream' part handler for stream cloning Boris Feld <boris.feld@octobus.net> parents: 35655 diff changeset	175 streamclone,
25919 8221fefaea08 bundle2: use absolute_import Gregory Szorc <gregory.szorc@gmail.com> parents: 25660 diff changeset	176 tags,
8221fefaea08 bundle2: use absolute_import Gregory Szorc <gregory.szorc@gmail.com> parents: 25660 diff changeset	177 url,
8221fefaea08 bundle2: use absolute_import Gregory Szorc <gregory.szorc@gmail.com> parents: 25660 diff changeset	178 util,
8221fefaea08 bundle2: use absolute_import Gregory Szorc <gregory.szorc@gmail.com> parents: 25660 diff changeset	179 )
46907 ffd3e823a7e5 urlutil: extract `url` related code from `util` into the new module Pierre-Yves David <pierre-yves.david@octobus.net> parents: 46793 diff changeset	180 from .utils import (
ffd3e823a7e5 urlutil: extract `url` related code from `util` into the new module Pierre-Yves David <pierre-yves.david@octobus.net> parents: 46793 diff changeset	181 stringutil,
ffd3e823a7e5 urlutil: extract `url` related code from `util` into the new module Pierre-Yves David <pierre-yves.david@octobus.net> parents: 46793 diff changeset	182 urlutil,
ffd3e823a7e5 urlutil: extract `url` related code from `util` into the new module Pierre-Yves David <pierre-yves.david@octobus.net> parents: 46793 diff changeset	183 )
47237 19d4802cb304 sidedata: add a 'side-data' repository feature and use it Pierre-Yves David <pierre-yves.david@octobus.net> parents: 47142 diff changeset	184 from .interfaces import repository
20802 520df53ad26a bundle2: a very first version of bundle2 unbundler Pierre-Yves David <pierre-yves.david@fb.com> parents: 20801 diff changeset	185
51731 6fc31e7bd5db typing: add some type hints for bundle2 capabilities Matt Harbison <matt_harbison@yahoo.com> parents: 51658 diff changeset	186 if typing.TYPE_CHECKING:
6fc31e7bd5db typing: add some type hints for bundle2 capabilities Matt Harbison <matt_harbison@yahoo.com> parents: 51658 diff changeset	187 from typing import (
6fc31e7bd5db typing: add some type hints for bundle2 capabilities Matt Harbison <matt_harbison@yahoo.com> parents: 51658 diff changeset	188 Dict,
52709 279e217d6041 typing: lock in the new type annotations detected with the pyupgrade changes Matt Harbison <matt_harbison@yahoo.com> parents: 52670 diff changeset	189 Iterator,
51731 6fc31e7bd5db typing: add some type hints for bundle2 capabilities Matt Harbison <matt_harbison@yahoo.com> parents: 51658 diff changeset	190 List,
6fc31e7bd5db typing: add some type hints for bundle2 capabilities Matt Harbison <matt_harbison@yahoo.com> parents: 51658 diff changeset	191 Optional,
6fc31e7bd5db typing: add some type hints for bundle2 capabilities Matt Harbison <matt_harbison@yahoo.com> parents: 51658 diff changeset	192 Tuple,
6fc31e7bd5db typing: add some type hints for bundle2 capabilities Matt Harbison <matt_harbison@yahoo.com> parents: 51658 diff changeset	193 Union,
6fc31e7bd5db typing: add some type hints for bundle2 capabilities Matt Harbison <matt_harbison@yahoo.com> parents: 51658 diff changeset	194 )
6fc31e7bd5db typing: add some type hints for bundle2 capabilities Matt Harbison <matt_harbison@yahoo.com> parents: 51658 diff changeset	195
6fc31e7bd5db typing: add some type hints for bundle2 capabilities Matt Harbison <matt_harbison@yahoo.com> parents: 51658 diff changeset	196 Capabilities = Dict[bytes, Union[List[bytes], Tuple[bytes, ...]]]
6fc31e7bd5db typing: add some type hints for bundle2 capabilities Matt Harbison <matt_harbison@yahoo.com> parents: 51658 diff changeset	197
28883 032c4c2f802a pycompat: switch to util.urlreq/util.urlerr for py3 compat timeless <timeless@mozdev.org> parents: 28672 diff changeset	198 urlerr = util.urlerr
032c4c2f802a pycompat: switch to util.urlreq/util.urlerr for py3 compat timeless <timeless@mozdev.org> parents: 28672 diff changeset	199 urlreq = util.urlreq
032c4c2f802a pycompat: switch to util.urlreq/util.urlerr for py3 compat timeless <timeless@mozdev.org> parents: 28672 diff changeset	200
20804 db9d3991d2c6 bundle2: support bundling simple parameter Pierre-Yves David <pierre-yves.david@fb.com> parents: 20803 diff changeset	201 _pack = struct.pack
db9d3991d2c6 bundle2: support bundling simple parameter Pierre-Yves David <pierre-yves.david@fb.com> parents: 20803 diff changeset	202 _unpack = struct.unpack
db9d3991d2c6 bundle2: support bundling simple parameter Pierre-Yves David <pierre-yves.david@fb.com> parents: 20803 diff changeset	203
43077 687b865b95ad formatting: byteify all mercurial/ and hgext/ string literals Augie Fackler <augie@google.com> parents: 43076 diff changeset	204 _fstreamparamsize = b'>i'
687b865b95ad formatting: byteify all mercurial/ and hgext/ string literals Augie Fackler <augie@google.com> parents: 43076 diff changeset	205 _fpartheadersize = b'>i'
687b865b95ad formatting: byteify all mercurial/ and hgext/ string literals Augie Fackler <augie@google.com> parents: 43076 diff changeset	206 _fparttypesize = b'>B'
687b865b95ad formatting: byteify all mercurial/ and hgext/ string literals Augie Fackler <augie@google.com> parents: 43076 diff changeset	207 _fpartid = b'>I'
687b865b95ad formatting: byteify all mercurial/ and hgext/ string literals Augie Fackler <augie@google.com> parents: 43076 diff changeset	208 _fpayloadsize = b'>i'
687b865b95ad formatting: byteify all mercurial/ and hgext/ string literals Augie Fackler <augie@google.com> parents: 43076 diff changeset	209 _fpartparamcount = b'>BB'
20877 9e9e3a4e9261 bundle2: part params Pierre-Yves David <pierre-yves.david@fb.com> parents: 20876 diff changeset	210
35793 4fb2bb61597c bundle2: increase payload part chunk size to 32kb Gregory Szorc <gregory.szorc@gmail.com> parents: 35792 diff changeset	211 preferedchunksize = 32768
21001 c93bb6a08fa1 bundle2: support chunk iterator as part data Pierre-Yves David <pierre-yves.david@fb.com> parents: 21000 diff changeset	212
43077 687b865b95ad formatting: byteify all mercurial/ and hgext/ string literals Augie Fackler <augie@google.com> parents: 43076 diff changeset	213 _parttypeforbidden = re.compile(b'[^a-zA-Z0-9_:-]')
23868 405eacbecc52 bundle2: enforce parttype as alphanumerical Pierre-Yves David <pierre-yves.david@fb.com> parents: 23590 diff changeset	214
43076 2372284d9457 formatting: blacken the codebase Augie Fackler <augie@google.com> parents: 42935 diff changeset	215
25313 8f2c362bcb58 bundle2: introduce a specific function for bundling debug message Pierre-Yves David <pierre-yves.david@fb.com> parents: 25181 diff changeset	216 def outdebug(ui, message):
8f2c362bcb58 bundle2: introduce a specific function for bundling debug message Pierre-Yves David <pierre-yves.david@fb.com> parents: 25181 diff changeset	217 """debug regarding output stream (bundling)"""
43077 687b865b95ad formatting: byteify all mercurial/ and hgext/ string literals Augie Fackler <augie@google.com> parents: 43076 diff changeset	218 if ui.configbool(b'devel', b'bundle2.debug'):
687b865b95ad formatting: byteify all mercurial/ and hgext/ string literals Augie Fackler <augie@google.com> parents: 43076 diff changeset	219 ui.debug(b'bundle2-output: %s\n' % message)
25313 8f2c362bcb58 bundle2: introduce a specific function for bundling debug message Pierre-Yves David <pierre-yves.david@fb.com> parents: 25181 diff changeset	220
43076 2372284d9457 formatting: blacken the codebase Augie Fackler <augie@google.com> parents: 42935 diff changeset	221
25318 64dd8f25fcf4 bundle2: introduce a specific function for debug messages while unbundling Pierre-Yves David <pierre-yves.david@fb.com> parents: 25317 diff changeset	222 def indebug(ui, message):
64dd8f25fcf4 bundle2: introduce a specific function for debug messages while unbundling Pierre-Yves David <pierre-yves.david@fb.com> parents: 25317 diff changeset	223 """debug on input stream (unbundling)"""
43077 687b865b95ad formatting: byteify all mercurial/ and hgext/ string literals Augie Fackler <augie@google.com> parents: 43076 diff changeset	224 if ui.configbool(b'devel', b'bundle2.debug'):
687b865b95ad formatting: byteify all mercurial/ and hgext/ string literals Augie Fackler <augie@google.com> parents: 43076 diff changeset	225 ui.debug(b'bundle2-input: %s\n' % message)
25318 64dd8f25fcf4 bundle2: introduce a specific function for debug messages while unbundling Pierre-Yves David <pierre-yves.david@fb.com> parents: 25317 diff changeset	226
43076 2372284d9457 formatting: blacken the codebase Augie Fackler <augie@google.com> parents: 42935 diff changeset	227
23868 405eacbecc52 bundle2: enforce parttype as alphanumerical Pierre-Yves David <pierre-yves.david@fb.com> parents: 23590 diff changeset	228 def validateparttype(parttype):
405eacbecc52 bundle2: enforce parttype as alphanumerical Pierre-Yves David <pierre-yves.david@fb.com> parents: 23590 diff changeset	229 """raise ValueError if a parttype contains invalid character"""
23916 a3f7c781786b bundle2: fix parttype enforcement Matt Mackall <mpm@selenic.com> parents: 23868 diff changeset	230 if _parttypeforbidden.search(parttype):
23868 405eacbecc52 bundle2: enforce parttype as alphanumerical Pierre-Yves David <pierre-yves.david@fb.com> parents: 23590 diff changeset	231 raise ValueError(parttype)
405eacbecc52 bundle2: enforce parttype as alphanumerical Pierre-Yves David <pierre-yves.david@fb.com> parents: 23590 diff changeset	232
43076 2372284d9457 formatting: blacken the codebase Augie Fackler <augie@google.com> parents: 42935 diff changeset	233
20877 9e9e3a4e9261 bundle2: part params Pierre-Yves David <pierre-yves.david@fb.com> parents: 20876 diff changeset	234 def _makefpartparamsizes(nbparams):
9e9e3a4e9261 bundle2: part params Pierre-Yves David <pierre-yves.david@fb.com> parents: 20876 diff changeset	235 """return a struct format to read part parameter sizes
9e9e3a4e9261 bundle2: part params Pierre-Yves David <pierre-yves.david@fb.com> parents: 20876 diff changeset	236
9e9e3a4e9261 bundle2: part params Pierre-Yves David <pierre-yves.david@fb.com> parents: 20876 diff changeset	237 The number parameters is variable so we need to build that format
9e9e3a4e9261 bundle2: part params Pierre-Yves David <pierre-yves.david@fb.com> parents: 20876 diff changeset	238 dynamically.
9e9e3a4e9261 bundle2: part params Pierre-Yves David <pierre-yves.david@fb.com> parents: 20876 diff changeset	239 """
43077 687b865b95ad formatting: byteify all mercurial/ and hgext/ string literals Augie Fackler <augie@google.com> parents: 43076 diff changeset	240 return b'>' + (b'BB' * nbparams)
43076 2372284d9457 formatting: blacken the codebase Augie Fackler <augie@google.com> parents: 42935 diff changeset	241
20804 db9d3991d2c6 bundle2: support bundling simple parameter Pierre-Yves David <pierre-yves.david@fb.com> parents: 20803 diff changeset	242
20890 ec7fc110faee bundle2: introduce a `parthandler` decorator Pierre-Yves David <pierre-yves.david@fb.com> parents: 20889 diff changeset	243 parthandlermapping = {}
20889 deed5edb72de bundle2: first version of a bundle processing Pierre-Yves David <pierre-yves.david@fb.com> parents: 20887 diff changeset	244
43076 2372284d9457 formatting: blacken the codebase Augie Fackler <augie@google.com> parents: 42935 diff changeset	245
21623 5b26d82e4e2a bundle2: make it possible to declare params handled by a part handler Pierre-Yves David <pierre-yves.david@fb.com> parents: 21622 diff changeset	246 def parthandler(parttype, params=()):
20890 ec7fc110faee bundle2: introduce a `parthandler` decorator Pierre-Yves David <pierre-yves.david@fb.com> parents: 20889 diff changeset	247 """decorator that register a function as a bundle2 part handler
ec7fc110faee bundle2: introduce a `parthandler` decorator Pierre-Yves David <pierre-yves.david@fb.com> parents: 20889 diff changeset	248
ec7fc110faee bundle2: introduce a `parthandler` decorator Pierre-Yves David <pierre-yves.david@fb.com> parents: 20889 diff changeset	249 eg::
ec7fc110faee bundle2: introduce a `parthandler` decorator Pierre-Yves David <pierre-yves.david@fb.com> parents: 20889 diff changeset	250
21624 d61066d787c8 bundle2: declare supported parameters for all handlers Pierre-Yves David <pierre-yves.david@fb.com> parents: 21623 diff changeset	251 @parthandler('myparttype', ('mandatory', 'param', 'handled'))
20890 ec7fc110faee bundle2: introduce a `parthandler` decorator Pierre-Yves David <pierre-yves.david@fb.com> parents: 20889 diff changeset	252 def myparttypehandler(...):
ec7fc110faee bundle2: introduce a `parthandler` decorator Pierre-Yves David <pierre-yves.david@fb.com> parents: 20889 diff changeset	253 '''process a part of type "my part".'''
ec7fc110faee bundle2: introduce a `parthandler` decorator Pierre-Yves David <pierre-yves.david@fb.com> parents: 20889 diff changeset	254 ...
ec7fc110faee bundle2: introduce a `parthandler` decorator Pierre-Yves David <pierre-yves.david@fb.com> parents: 20889 diff changeset	255 """
23868 405eacbecc52 bundle2: enforce parttype as alphanumerical Pierre-Yves David <pierre-yves.david@fb.com> parents: 23590 diff changeset	256 validateparttype(parttype)
43076 2372284d9457 formatting: blacken the codebase Augie Fackler <augie@google.com> parents: 42935 diff changeset	257
20890 ec7fc110faee bundle2: introduce a `parthandler` decorator Pierre-Yves David <pierre-yves.david@fb.com> parents: 20889 diff changeset	258 def _decorator(func):
43076 2372284d9457 formatting: blacken the codebase Augie Fackler <augie@google.com> parents: 42935 diff changeset	259 lparttype = parttype.lower() # enforce lower case matching.
20891 1c6cd23fc221 bundle2: add some distinction between mandatory and advisory part Pierre-Yves David <pierre-yves.david@fb.com> parents: 20890 diff changeset	260 assert lparttype not in parthandlermapping
1c6cd23fc221 bundle2: add some distinction between mandatory and advisory part Pierre-Yves David <pierre-yves.david@fb.com> parents: 20890 diff changeset	261 parthandlermapping[lparttype] = func
21623 5b26d82e4e2a bundle2: make it possible to declare params handled by a part handler Pierre-Yves David <pierre-yves.david@fb.com> parents: 21622 diff changeset	262 func.params = frozenset(params)
20890 ec7fc110faee bundle2: introduce a `parthandler` decorator Pierre-Yves David <pierre-yves.david@fb.com> parents: 20889 diff changeset	263 return func
43076 2372284d9457 formatting: blacken the codebase Augie Fackler <augie@google.com> parents: 42935 diff changeset	264
20890 ec7fc110faee bundle2: introduce a `parthandler` decorator Pierre-Yves David <pierre-yves.david@fb.com> parents: 20889 diff changeset	265 return _decorator
20889 deed5edb72de bundle2: first version of a bundle processing Pierre-Yves David <pierre-yves.david@fb.com> parents: 20887 diff changeset	266
43076 2372284d9457 formatting: blacken the codebase Augie Fackler <augie@google.com> parents: 42935 diff changeset	267
49037 642e31cb55f0 py3: use class X: instead of class X(object): Gregory Szorc <gregory.szorc@gmail.com> parents: 49004 diff changeset	268 class unbundlerecords:
20949 571f2903ff1e bundle2: record processing results in the bundleoperation object Pierre-Yves David <pierre-yves.david@fb.com> parents: 20948 diff changeset	269 """keep record of what happens during and unbundle
571f2903ff1e bundle2: record processing results in the bundleoperation object Pierre-Yves David <pierre-yves.david@fb.com> parents: 20948 diff changeset	270
571f2903ff1e bundle2: record processing results in the bundleoperation object Pierre-Yves David <pierre-yves.david@fb.com> parents: 20948 diff changeset	271 New records are added using `records.add('cat', obj)`. Where 'cat' is a
21024 7731a2281cf0 spelling: fixes from spell checker Mads Kiilerich <madski@unity3d.com> parents: 21020 diff changeset	272 category of record and obj is an arbitrary object.
20949 571f2903ff1e bundle2: record processing results in the bundleoperation object Pierre-Yves David <pierre-yves.david@fb.com> parents: 20948 diff changeset	273
571f2903ff1e bundle2: record processing results in the bundleoperation object Pierre-Yves David <pierre-yves.david@fb.com> parents: 20948 diff changeset	274 `records['cat']` will return all entries of this category 'cat'.
571f2903ff1e bundle2: record processing results in the bundleoperation object Pierre-Yves David <pierre-yves.david@fb.com> parents: 20948 diff changeset	275
571f2903ff1e bundle2: record processing results in the bundleoperation object Pierre-Yves David <pierre-yves.david@fb.com> parents: 20948 diff changeset	276 Iterating on the object itself will yield `('category', obj)` tuples
571f2903ff1e bundle2: record processing results in the bundleoperation object Pierre-Yves David <pierre-yves.david@fb.com> parents: 20948 diff changeset	277 for all entries.
571f2903ff1e bundle2: record processing results in the bundleoperation object Pierre-Yves David <pierre-yves.david@fb.com> parents: 20948 diff changeset	278
571f2903ff1e bundle2: record processing results in the bundleoperation object Pierre-Yves David <pierre-yves.david@fb.com> parents: 20948 diff changeset	279 All iterations happens in chronological order.
571f2903ff1e bundle2: record processing results in the bundleoperation object Pierre-Yves David <pierre-yves.david@fb.com> parents: 20948 diff changeset	280 """
571f2903ff1e bundle2: record processing results in the bundleoperation object Pierre-Yves David <pierre-yves.david@fb.com> parents: 20948 diff changeset	281
571f2903ff1e bundle2: record processing results in the bundleoperation object Pierre-Yves David <pierre-yves.david@fb.com> parents: 20948 diff changeset	282 def __init__(self):
571f2903ff1e bundle2: record processing results in the bundleoperation object Pierre-Yves David <pierre-yves.david@fb.com> parents: 20948 diff changeset	283 self._categories = {}
571f2903ff1e bundle2: record processing results in the bundleoperation object Pierre-Yves David <pierre-yves.david@fb.com> parents: 20948 diff changeset	284 self._sequences = []
20996 ed3c5e18a047 bundle2: add reply awareness to unbundlerecords Pierre-Yves David <pierre-yves.david@fb.com> parents: 20995 diff changeset	285 self._replies = {}
20949 571f2903ff1e bundle2: record processing results in the bundleoperation object Pierre-Yves David <pierre-yves.david@fb.com> parents: 20948 diff changeset	286
20996 ed3c5e18a047 bundle2: add reply awareness to unbundlerecords Pierre-Yves David <pierre-yves.david@fb.com> parents: 20995 diff changeset	287 def add(self, category, entry, inreplyto=None):
20949 571f2903ff1e bundle2: record processing results in the bundleoperation object Pierre-Yves David <pierre-yves.david@fb.com> parents: 20948 diff changeset	288 """add a new record of a given category.
571f2903ff1e bundle2: record processing results in the bundleoperation object Pierre-Yves David <pierre-yves.david@fb.com> parents: 20948 diff changeset	289
571f2903ff1e bundle2: record processing results in the bundleoperation object Pierre-Yves David <pierre-yves.david@fb.com> parents: 20948 diff changeset	290 The entry can then be retrieved in the list returned by
571f2903ff1e bundle2: record processing results in the bundleoperation object Pierre-Yves David <pierre-yves.david@fb.com> parents: 20948 diff changeset	291 self['category']."""
571f2903ff1e bundle2: record processing results in the bundleoperation object Pierre-Yves David <pierre-yves.david@fb.com> parents: 20948 diff changeset	292 self._categories.setdefault(category, []).append(entry)
571f2903ff1e bundle2: record processing results in the bundleoperation object Pierre-Yves David <pierre-yves.david@fb.com> parents: 20948 diff changeset	293 self._sequences.append((category, entry))
20996 ed3c5e18a047 bundle2: add reply awareness to unbundlerecords Pierre-Yves David <pierre-yves.david@fb.com> parents: 20995 diff changeset	294 if inreplyto is not None:
ed3c5e18a047 bundle2: add reply awareness to unbundlerecords Pierre-Yves David <pierre-yves.david@fb.com> parents: 20995 diff changeset	295 self.getreplies(inreplyto).add(category, entry)
ed3c5e18a047 bundle2: add reply awareness to unbundlerecords Pierre-Yves David <pierre-yves.david@fb.com> parents: 20995 diff changeset	296
ed3c5e18a047 bundle2: add reply awareness to unbundlerecords Pierre-Yves David <pierre-yves.david@fb.com> parents: 20995 diff changeset	297 def getreplies(self, partid):
23139 e53f6b72a0e4 spelling: fixes from proofreading of spell checker issues Mads Kiilerich <madski@unity3d.com> parents: 23067 diff changeset	298 """get the records that are replies to a specific part"""
20996 ed3c5e18a047 bundle2: add reply awareness to unbundlerecords Pierre-Yves David <pierre-yves.david@fb.com> parents: 20995 diff changeset	299 return self._replies.setdefault(partid, unbundlerecords())
20949 571f2903ff1e bundle2: record processing results in the bundleoperation object Pierre-Yves David <pierre-yves.david@fb.com> parents: 20948 diff changeset	300
571f2903ff1e bundle2: record processing results in the bundleoperation object Pierre-Yves David <pierre-yves.david@fb.com> parents: 20948 diff changeset	301 def __getitem__(self, cat):
571f2903ff1e bundle2: record processing results in the bundleoperation object Pierre-Yves David <pierre-yves.david@fb.com> parents: 20948 diff changeset	302 return tuple(self._categories.get(cat, ()))
571f2903ff1e bundle2: record processing results in the bundleoperation object Pierre-Yves David <pierre-yves.david@fb.com> parents: 20948 diff changeset	303
571f2903ff1e bundle2: record processing results in the bundleoperation object Pierre-Yves David <pierre-yves.david@fb.com> parents: 20948 diff changeset	304 def __iter__(self):
571f2903ff1e bundle2: record processing results in the bundleoperation object Pierre-Yves David <pierre-yves.david@fb.com> parents: 20948 diff changeset	305 return iter(self._sequences)
571f2903ff1e bundle2: record processing results in the bundleoperation object Pierre-Yves David <pierre-yves.david@fb.com> parents: 20948 diff changeset	306
571f2903ff1e bundle2: record processing results in the bundleoperation object Pierre-Yves David <pierre-yves.david@fb.com> parents: 20948 diff changeset	307 def __len__(self):
571f2903ff1e bundle2: record processing results in the bundleoperation object Pierre-Yves David <pierre-yves.david@fb.com> parents: 20948 diff changeset	308 return len(self._sequences)
571f2903ff1e bundle2: record processing results in the bundleoperation object Pierre-Yves David <pierre-yves.david@fb.com> parents: 20948 diff changeset	309
571f2903ff1e bundle2: record processing results in the bundleoperation object Pierre-Yves David <pierre-yves.david@fb.com> parents: 20948 diff changeset	310 def __nonzero__(self):
571f2903ff1e bundle2: record processing results in the bundleoperation object Pierre-Yves David <pierre-yves.david@fb.com> parents: 20948 diff changeset	311 return bool(self._sequences)
571f2903ff1e bundle2: record processing results in the bundleoperation object Pierre-Yves David <pierre-yves.david@fb.com> parents: 20948 diff changeset	312
31483 413b44003462 py3: add __bool__ to every class defining __nonzero__ Gregory Szorc <gregory.szorc@gmail.com> parents: 30985 diff changeset	313 __bool__ = __nonzero__
413b44003462 py3: add __bool__ to every class defining __nonzero__ Gregory Szorc <gregory.szorc@gmail.com> parents: 30985 diff changeset	314
43076 2372284d9457 formatting: blacken the codebase Augie Fackler <augie@google.com> parents: 42935 diff changeset	315
49037 642e31cb55f0 py3: use class X: instead of class X(object): Gregory Szorc <gregory.szorc@gmail.com> parents: 49004 diff changeset	316 class bundleoperation:
20948 329cd74b52bd bundle2: introduce a bundleoperation object Pierre-Yves David <pierre-yves.david@fb.com> parents: 20947 diff changeset	317 """an object that represents a single bundling process
329cd74b52bd bundle2: introduce a bundleoperation object Pierre-Yves David <pierre-yves.david@fb.com> parents: 20947 diff changeset	318
329cd74b52bd bundle2: introduce a bundleoperation object Pierre-Yves David <pierre-yves.david@fb.com> parents: 20947 diff changeset	319 Its purpose is to carry unbundle-related objects and states.
329cd74b52bd bundle2: introduce a bundleoperation object Pierre-Yves David <pierre-yves.david@fb.com> parents: 20947 diff changeset	320
329cd74b52bd bundle2: introduce a bundleoperation object Pierre-Yves David <pierre-yves.david@fb.com> parents: 20947 diff changeset	321 A new object should be created at the beginning of each bundle processing.
329cd74b52bd bundle2: introduce a bundleoperation object Pierre-Yves David <pierre-yves.david@fb.com> parents: 20947 diff changeset	322 The object is to be returned by the processing function.
329cd74b52bd bundle2: introduce a bundleoperation object Pierre-Yves David <pierre-yves.david@fb.com> parents: 20947 diff changeset	323
329cd74b52bd bundle2: introduce a bundleoperation object Pierre-Yves David <pierre-yves.david@fb.com> parents: 20947 diff changeset	324 The object has very little content now it will ultimately contain:
329cd74b52bd bundle2: introduce a bundleoperation object Pierre-Yves David <pierre-yves.david@fb.com> parents: 20947 diff changeset	325 * an access to the repo the bundle is applied to,
329cd74b52bd bundle2: introduce a bundleoperation object Pierre-Yves David <pierre-yves.david@fb.com> parents: 20947 diff changeset	326 * a ui object,
329cd74b52bd bundle2: introduce a bundleoperation object Pierre-Yves David <pierre-yves.david@fb.com> parents: 20947 diff changeset	327 * a way to retrieve a transaction to add changes to the repo,
329cd74b52bd bundle2: introduce a bundleoperation object Pierre-Yves David <pierre-yves.david@fb.com> parents: 20947 diff changeset	328 * a way to record the result of processing each part,
329cd74b52bd bundle2: introduce a bundleoperation object Pierre-Yves David <pierre-yves.david@fb.com> parents: 20947 diff changeset	329 * a way to construct a bundle response when applicable.
329cd74b52bd bundle2: introduce a bundleoperation object Pierre-Yves David <pierre-yves.david@fb.com> parents: 20947 diff changeset	330 """
329cd74b52bd bundle2: introduce a bundleoperation object Pierre-Yves David <pierre-yves.david@fb.com> parents: 20947 diff changeset	331
49876 4188e75af983 bundleoperation: optionnaly record the `remote` that produced the bundle Pierre-Yves David <pierre-yves.david@octobus.net> parents: 49764 diff changeset	332 def __init__(
4188e75af983 bundleoperation: optionnaly record the `remote` that produced the bundle Pierre-Yves David <pierre-yves.david@octobus.net> parents: 49764 diff changeset	333 self,
4188e75af983 bundleoperation: optionnaly record the `remote` that produced the bundle Pierre-Yves David <pierre-yves.david@octobus.net> parents: 49764 diff changeset	334 repo,
4188e75af983 bundleoperation: optionnaly record the `remote` that produced the bundle Pierre-Yves David <pierre-yves.david@octobus.net> parents: 49764 diff changeset	335 transactiongetter,
4188e75af983 bundleoperation: optionnaly record the `remote` that produced the bundle Pierre-Yves David <pierre-yves.david@octobus.net> parents: 49764 diff changeset	336 captureoutput=True,
4188e75af983 bundleoperation: optionnaly record the `remote` that produced the bundle Pierre-Yves David <pierre-yves.david@octobus.net> parents: 49764 diff changeset	337 source=b'',
4188e75af983 bundleoperation: optionnaly record the `remote` that produced the bundle Pierre-Yves David <pierre-yves.david@octobus.net> parents: 49764 diff changeset	338 remote=None,
4188e75af983 bundleoperation: optionnaly record the `remote` that produced the bundle Pierre-Yves David <pierre-yves.david@octobus.net> parents: 49764 diff changeset	339 ):
20948 329cd74b52bd bundle2: introduce a bundleoperation object Pierre-Yves David <pierre-yves.david@fb.com> parents: 20947 diff changeset	340 self.repo = repo
49876 4188e75af983 bundleoperation: optionnaly record the `remote` that produced the bundle Pierre-Yves David <pierre-yves.david@octobus.net> parents: 49764 diff changeset	341 # the peer object who produced this bundle if available
4188e75af983 bundleoperation: optionnaly record the `remote` that produced the bundle Pierre-Yves David <pierre-yves.david@octobus.net> parents: 49764 diff changeset	342 self.remote = remote
20948 329cd74b52bd bundle2: introduce a bundleoperation object Pierre-Yves David <pierre-yves.david@fb.com> parents: 20947 diff changeset	343 self.ui = repo.ui
20949 571f2903ff1e bundle2: record processing results in the bundleoperation object Pierre-Yves David <pierre-yves.david@fb.com> parents: 20948 diff changeset	344 self.records = unbundlerecords()
20997 d7df4b7378ae bundle2: produce a bundle2 reply Pierre-Yves David <pierre-yves.david@fb.com> parents: 20996 diff changeset	345 self.reply = None
24878 e530cde6d115 bundle2: disable ouput capture unless we use http (issue4613 issue4615) Pierre-Yves David <pierre-yves.david@fb.com> parents: 24851 diff changeset	346 self.captureoutput = captureoutput
33692 f3407d56a6e8 bundle2: add the capability to store hookargs on bundle operation object Pulkit Goyal <7895pulkit@gmail.com> parents: 33541 diff changeset	347 self.hookargs = {}
33693 5fc4ddfbe626 bundle2: load hookargs from bundleoperation into transaction when started one Pulkit Goyal <7895pulkit@gmail.com> parents: 33692 diff changeset	348 self._gettransaction = transactiongetter
35274 1f30cbac34b6 bundle2: add a 'modes' dictionary to the bundle operation Boris Feld <boris.feld@octobus.net> parents: 35270 diff changeset	349 # carries value that can modify part behavior
1f30cbac34b6 bundle2: add a 'modes' dictionary to the bundle operation Boris Feld <boris.feld@octobus.net> parents: 35270 diff changeset	350 self.modes = {}
37237 7e906d8a825f bundle2: add 'source' atrribute to bundleoperation class Pulkit Goyal <7895pulkit@gmail.com> parents: 37169 diff changeset	351 self.source = source
33693 5fc4ddfbe626 bundle2: load hookargs from bundleoperation into transaction when started one Pulkit Goyal <7895pulkit@gmail.com> parents: 33692 diff changeset	352
5fc4ddfbe626 bundle2: load hookargs from bundleoperation into transaction when started one Pulkit Goyal <7895pulkit@gmail.com> parents: 33692 diff changeset	353 def gettransaction(self):
5fc4ddfbe626 bundle2: load hookargs from bundleoperation into transaction when started one Pulkit Goyal <7895pulkit@gmail.com> parents: 33692 diff changeset	354 transaction = self._gettransaction()
5fc4ddfbe626 bundle2: load hookargs from bundleoperation into transaction when started one Pulkit Goyal <7895pulkit@gmail.com> parents: 33692 diff changeset	355
33831 b3a41f4d837b bundle2: relax the condition to update transaction.hookargs Yuya Nishihara <yuya@tcha.org> parents: 33830 diff changeset	356 if self.hookargs:
33693 5fc4ddfbe626 bundle2: load hookargs from bundleoperation into transaction when started one Pulkit Goyal <7895pulkit@gmail.com> parents: 33692 diff changeset	357 # the ones added to the transaction supercede those added
5fc4ddfbe626 bundle2: load hookargs from bundleoperation into transaction when started one Pulkit Goyal <7895pulkit@gmail.com> parents: 33692 diff changeset	358 # to the operation.
5fc4ddfbe626 bundle2: load hookargs from bundleoperation into transaction when started one Pulkit Goyal <7895pulkit@gmail.com> parents: 33692 diff changeset	359 self.hookargs.update(transaction.hookargs)
5fc4ddfbe626 bundle2: load hookargs from bundleoperation into transaction when started one Pulkit Goyal <7895pulkit@gmail.com> parents: 33692 diff changeset	360 transaction.hookargs = self.hookargs
5fc4ddfbe626 bundle2: load hookargs from bundleoperation into transaction when started one Pulkit Goyal <7895pulkit@gmail.com> parents: 33692 diff changeset	361
33831 b3a41f4d837b bundle2: relax the condition to update transaction.hookargs Yuya Nishihara <yuya@tcha.org> parents: 33830 diff changeset	362 # mark the hookargs as flushed. further attempts to add to
b3a41f4d837b bundle2: relax the condition to update transaction.hookargs Yuya Nishihara <yuya@tcha.org> parents: 33830 diff changeset	363 # hookargs will result in an abort.
b3a41f4d837b bundle2: relax the condition to update transaction.hookargs Yuya Nishihara <yuya@tcha.org> parents: 33830 diff changeset	364 self.hookargs = None
33693 5fc4ddfbe626 bundle2: load hookargs from bundleoperation into transaction when started one Pulkit Goyal <7895pulkit@gmail.com> parents: 33692 diff changeset	365
5fc4ddfbe626 bundle2: load hookargs from bundleoperation into transaction when started one Pulkit Goyal <7895pulkit@gmail.com> parents: 33692 diff changeset	366 return transaction
33692 f3407d56a6e8 bundle2: add the capability to store hookargs on bundle operation object Pulkit Goyal <7895pulkit@gmail.com> parents: 33541 diff changeset	367
f3407d56a6e8 bundle2: add the capability to store hookargs on bundle operation object Pulkit Goyal <7895pulkit@gmail.com> parents: 33541 diff changeset	368 def addhookargs(self, hookargs):
33693 5fc4ddfbe626 bundle2: load hookargs from bundleoperation into transaction when started one Pulkit Goyal <7895pulkit@gmail.com> parents: 33692 diff changeset	369 if self.hookargs is None:
43076 2372284d9457 formatting: blacken the codebase Augie Fackler <augie@google.com> parents: 42935 diff changeset	370 raise error.ProgrammingError(
43077 687b865b95ad formatting: byteify all mercurial/ and hgext/ string literals Augie Fackler <augie@google.com> parents: 43076 diff changeset	371 b'attempted to add hookargs to '
687b865b95ad formatting: byteify all mercurial/ and hgext/ string literals Augie Fackler <augie@google.com> parents: 43076 diff changeset	372 b'operation after transaction started'
43076 2372284d9457 formatting: blacken the codebase Augie Fackler <augie@google.com> parents: 42935 diff changeset	373 )
33692 f3407d56a6e8 bundle2: add the capability to store hookargs on bundle operation object Pulkit Goyal <7895pulkit@gmail.com> parents: 33541 diff changeset	374 self.hookargs.update(hookargs)
20948 329cd74b52bd bundle2: introduce a bundleoperation object Pierre-Yves David <pierre-yves.david@fb.com> parents: 20947 diff changeset	375
43076 2372284d9457 formatting: blacken the codebase Augie Fackler <augie@google.com> parents: 42935 diff changeset	376
20952 b24ee5076b94 bundle2: make it possible have a global transaction for the unbundling Pierre-Yves David <pierre-yves.david@fb.com> parents: 20950 diff changeset	377 class TransactionUnavailable(RuntimeError):
b24ee5076b94 bundle2: make it possible have a global transaction for the unbundling Pierre-Yves David <pierre-yves.david@fb.com> parents: 20950 diff changeset	378 pass
b24ee5076b94 bundle2: make it possible have a global transaction for the unbundling Pierre-Yves David <pierre-yves.david@fb.com> parents: 20950 diff changeset	379
43076 2372284d9457 formatting: blacken the codebase Augie Fackler <augie@google.com> parents: 42935 diff changeset	380
20952 b24ee5076b94 bundle2: make it possible have a global transaction for the unbundling Pierre-Yves David <pierre-yves.david@fb.com> parents: 20950 diff changeset	381 def _notransaction():
b24ee5076b94 bundle2: make it possible have a global transaction for the unbundling Pierre-Yves David <pierre-yves.david@fb.com> parents: 20950 diff changeset	382 """default method to get a transaction while processing a bundle
b24ee5076b94 bundle2: make it possible have a global transaction for the unbundling Pierre-Yves David <pierre-yves.david@fb.com> parents: 20950 diff changeset	383
b24ee5076b94 bundle2: make it possible have a global transaction for the unbundling Pierre-Yves David <pierre-yves.david@fb.com> parents: 20950 diff changeset	384 Raise an exception to highlight the fact that no transaction was expected
b24ee5076b94 bundle2: make it possible have a global transaction for the unbundling Pierre-Yves David <pierre-yves.david@fb.com> parents: 20950 diff changeset	385 to be created"""
b24ee5076b94 bundle2: make it possible have a global transaction for the unbundling Pierre-Yves David <pierre-yves.david@fb.com> parents: 20950 diff changeset	386 raise TransactionUnavailable()
b24ee5076b94 bundle2: make it possible have a global transaction for the unbundling Pierre-Yves David <pierre-yves.david@fb.com> parents: 20950 diff changeset	387
43076 2372284d9457 formatting: blacken the codebase Augie Fackler <augie@google.com> parents: 42935 diff changeset	388
49876 4188e75af983 bundleoperation: optionnaly record the `remote` that produced the bundle Pierre-Yves David <pierre-yves.david@octobus.net> parents: 49764 diff changeset	389 def applybundle(repo, unbundler, tr, source, url=None, remote=None, **kwargs):
26790 28a6c2d72097 bundle2: introduce an "applybundle" function Pierre-Yves David <pierre-yves.david@fb.com> parents: 26698 diff changeset	390 # transform me into unbundler.apply() as soon as the freeze is lifted
33055 18c2489ac96d bundle: make applybundle() delegate v1 bundles to applybundle1() Martin von Zweigbergk <martinvonz@google.com> parents: 33052 diff changeset	391 if isinstance(unbundler, unbundle20):
43077 687b865b95ad formatting: byteify all mercurial/ and hgext/ string literals Augie Fackler <augie@google.com> parents: 43076 diff changeset	392 tr.hookargs[b'bundle2'] = b'1'
687b865b95ad formatting: byteify all mercurial/ and hgext/ string literals Augie Fackler <augie@google.com> parents: 43076 diff changeset	393 if source is not None and b'source' not in tr.hookargs:
687b865b95ad formatting: byteify all mercurial/ and hgext/ string literals Augie Fackler <augie@google.com> parents: 43076 diff changeset	394 tr.hookargs[b'source'] = source
687b865b95ad formatting: byteify all mercurial/ and hgext/ string literals Augie Fackler <augie@google.com> parents: 43076 diff changeset	395 if url is not None and b'url' not in tr.hookargs:
687b865b95ad formatting: byteify all mercurial/ and hgext/ string literals Augie Fackler <augie@google.com> parents: 43076 diff changeset	396 tr.hookargs[b'url'] = url
49876 4188e75af983 bundleoperation: optionnaly record the `remote` that produced the bundle Pierre-Yves David <pierre-yves.david@octobus.net> parents: 49764 diff changeset	397 return processbundle(
4188e75af983 bundleoperation: optionnaly record the `remote` that produced the bundle Pierre-Yves David <pierre-yves.david@octobus.net> parents: 49764 diff changeset	398 repo, unbundler, lambda: tr, source=source, remote=remote
4188e75af983 bundleoperation: optionnaly record the `remote` that produced the bundle Pierre-Yves David <pierre-yves.david@octobus.net> parents: 49764 diff changeset	399 )
33055 18c2489ac96d bundle: make applybundle() delegate v1 bundles to applybundle1() Martin von Zweigbergk <martinvonz@google.com> parents: 33052 diff changeset	400 else:
33056 1d2b68952950 bundle: inline applybundle1() Martin von Zweigbergk <martinvonz@google.com> parents: 33055 diff changeset	401 # the transactiongetter won't be used, but we might as well set it
49876 4188e75af983 bundleoperation: optionnaly record the `remote` that produced the bundle Pierre-Yves David <pierre-yves.david@octobus.net> parents: 49764 diff changeset	402 op = bundleoperation(repo, lambda: tr, source=source, remote=remote)
33056 1d2b68952950 bundle: inline applybundle1() Martin von Zweigbergk <martinvonz@google.com> parents: 33055 diff changeset	403 _processchangegroup(op, unbundler, tr, source, url, **kwargs)
1d2b68952950 bundle: inline applybundle1() Martin von Zweigbergk <martinvonz@google.com> parents: 33055 diff changeset	404 return op
26790 28a6c2d72097 bundle2: introduce an "applybundle" function Pierre-Yves David <pierre-yves.david@fb.com> parents: 26698 diff changeset	405
43076 2372284d9457 formatting: blacken the codebase Augie Fackler <augie@google.com> parents: 42935 diff changeset	406
49037 642e31cb55f0 py3: use class X: instead of class X(object): Gregory Szorc <gregory.szorc@gmail.com> parents: 49004 diff changeset	407 class partiterator:
34164 21c2df59a1da bundle2: move exception handling into part iterator Durham Goode <durham@fb.com> parents: 34163 diff changeset	408 def __init__(self, repo, op, unbundler):
34163 550343626bb2 bundle2: move part counter to partiterator Durham Goode <durham@fb.com> parents: 34162 diff changeset	409 self.repo = repo
34164 21c2df59a1da bundle2: move exception handling into part iterator Durham Goode <durham@fb.com> parents: 34163 diff changeset	410 self.op = op
34162 e9e0e1143fc5 bundle2: move part iterator a separate class Durham Goode <durham@fb.com> parents: 34116 diff changeset	411 self.unbundler = unbundler
34163 550343626bb2 bundle2: move part counter to partiterator Durham Goode <durham@fb.com> parents: 34162 diff changeset	412 self.iterator = None
550343626bb2 bundle2: move part counter to partiterator Durham Goode <durham@fb.com> parents: 34162 diff changeset	413 self.count = 0
34265 e71890f27767 bundle2: move processpart stream maintenance into part iterator Durham Goode <durham@fb.com> parents: 34256 diff changeset	414 self.current = None
34162 e9e0e1143fc5 bundle2: move part iterator a separate class Durham Goode <durham@fb.com> parents: 34116 diff changeset	415
e9e0e1143fc5 bundle2: move part iterator a separate class Durham Goode <durham@fb.com> parents: 34116 diff changeset	416 def __enter__(self):
34163 550343626bb2 bundle2: move part counter to partiterator Durham Goode <durham@fb.com> parents: 34162 diff changeset	417 def func():
42935 181ee2118a96 bundle2: fix an off-by-one in debug message of number of parts Martin von Zweigbergk <martinvonz@google.com> parents: 42900 diff changeset	418 itr = enumerate(self.unbundler.iterparts(), 1)
34163 550343626bb2 bundle2: move part counter to partiterator Durham Goode <durham@fb.com> parents: 34162 diff changeset	419 for count, p in itr:
550343626bb2 bundle2: move part counter to partiterator Durham Goode <durham@fb.com> parents: 34162 diff changeset	420 self.count = count
34265 e71890f27767 bundle2: move processpart stream maintenance into part iterator Durham Goode <durham@fb.com> parents: 34256 diff changeset	421 self.current = p
34163 550343626bb2 bundle2: move part counter to partiterator Durham Goode <durham@fb.com> parents: 34162 diff changeset	422 yield p
35135 db5038525718 bundle2: implement consume() API on unbundlepart Gregory Szorc <gregory.szorc@gmail.com> parents: 35134 diff changeset	423 p.consume()
34265 e71890f27767 bundle2: move processpart stream maintenance into part iterator Durham Goode <durham@fb.com> parents: 34256 diff changeset	424 self.current = None
43076 2372284d9457 formatting: blacken the codebase Augie Fackler <augie@google.com> parents: 42935 diff changeset	425
34163 550343626bb2 bundle2: move part counter to partiterator Durham Goode <durham@fb.com> parents: 34162 diff changeset	426 self.iterator = func()
550343626bb2 bundle2: move part counter to partiterator Durham Goode <durham@fb.com> parents: 34162 diff changeset	427 return self.iterator
34162 e9e0e1143fc5 bundle2: move part iterator a separate class Durham Goode <durham@fb.com> parents: 34116 diff changeset	428
34164 21c2df59a1da bundle2: move exception handling into part iterator Durham Goode <durham@fb.com> parents: 34163 diff changeset	429 def __exit__(self, type, exc, tb):
34163 550343626bb2 bundle2: move part counter to partiterator Durham Goode <durham@fb.com> parents: 34162 diff changeset	430 if not self.iterator:
550343626bb2 bundle2: move part counter to partiterator Durham Goode <durham@fb.com> parents: 34162 diff changeset	431 return
550343626bb2 bundle2: move part counter to partiterator Durham Goode <durham@fb.com> parents: 34162 diff changeset	432
34637 5f79f5f8487a bundle2: immediate exit for ctrl+c (issue5692) Durham Goode <durham@fb.com> parents: 34330 diff changeset	433 # Only gracefully abort in a normal exception situation. User aborts
5f79f5f8487a bundle2: immediate exit for ctrl+c (issue5692) Durham Goode <durham@fb.com> parents: 34330 diff changeset	434 # like Ctrl+C throw a KeyboardInterrupt which is not a base Exception,
5f79f5f8487a bundle2: immediate exit for ctrl+c (issue5692) Durham Goode <durham@fb.com> parents: 34330 diff changeset	435 # and should not gracefully cleanup.
5f79f5f8487a bundle2: immediate exit for ctrl+c (issue5692) Durham Goode <durham@fb.com> parents: 34330 diff changeset	436 if isinstance(exc, Exception):
34164 21c2df59a1da bundle2: move exception handling into part iterator Durham Goode <durham@fb.com> parents: 34163 diff changeset	437 # Any exceptions seeking to the end of the bundle at this point are
21c2df59a1da bundle2: move exception handling into part iterator Durham Goode <durham@fb.com> parents: 34163 diff changeset	438 # almost certainly related to the underlying stream being bad.
21c2df59a1da bundle2: move exception handling into part iterator Durham Goode <durham@fb.com> parents: 34163 diff changeset	439 # And, chances are that the exception we're handling is related to
21c2df59a1da bundle2: move exception handling into part iterator Durham Goode <durham@fb.com> parents: 34163 diff changeset	440 # getting in that bad state. So, we swallow the seeking error and
21c2df59a1da bundle2: move exception handling into part iterator Durham Goode <durham@fb.com> parents: 34163 diff changeset	441 # re-raise the original error.
21c2df59a1da bundle2: move exception handling into part iterator Durham Goode <durham@fb.com> parents: 34163 diff changeset	442 seekerror = False
21c2df59a1da bundle2: move exception handling into part iterator Durham Goode <durham@fb.com> parents: 34163 diff changeset	443 try:
34637 5f79f5f8487a bundle2: immediate exit for ctrl+c (issue5692) Durham Goode <durham@fb.com> parents: 34330 diff changeset	444 if self.current:
5f79f5f8487a bundle2: immediate exit for ctrl+c (issue5692) Durham Goode <durham@fb.com> parents: 34330 diff changeset	445 # consume the part content to not corrupt the stream.
35135 db5038525718 bundle2: implement consume() API on unbundlepart Gregory Szorc <gregory.szorc@gmail.com> parents: 35134 diff changeset	446 self.current.consume()
34637 5f79f5f8487a bundle2: immediate exit for ctrl+c (issue5692) Durham Goode <durham@fb.com> parents: 34330 diff changeset	447
34164 21c2df59a1da bundle2: move exception handling into part iterator Durham Goode <durham@fb.com> parents: 34163 diff changeset	448 for part in self.iterator:
21c2df59a1da bundle2: move exception handling into part iterator Durham Goode <durham@fb.com> parents: 34163 diff changeset	449 # consume the bundle content
35135 db5038525718 bundle2: implement consume() API on unbundlepart Gregory Szorc <gregory.szorc@gmail.com> parents: 35134 diff changeset	450 part.consume()
34164 21c2df59a1da bundle2: move exception handling into part iterator Durham Goode <durham@fb.com> parents: 34163 diff changeset	451 except Exception:
21c2df59a1da bundle2: move exception handling into part iterator Durham Goode <durham@fb.com> parents: 34163 diff changeset	452 seekerror = True
21c2df59a1da bundle2: move exception handling into part iterator Durham Goode <durham@fb.com> parents: 34163 diff changeset	453
21c2df59a1da bundle2: move exception handling into part iterator Durham Goode <durham@fb.com> parents: 34163 diff changeset	454 # Small hack to let caller code distinguish exceptions from bundle2
21c2df59a1da bundle2: move exception handling into part iterator Durham Goode <durham@fb.com> parents: 34163 diff changeset	455 # processing from processing the old format. This is mostly needed
21c2df59a1da bundle2: move exception handling into part iterator Durham Goode <durham@fb.com> parents: 34163 diff changeset	456 # to handle different return codes to unbundle according to the type
21c2df59a1da bundle2: move exception handling into part iterator Durham Goode <durham@fb.com> parents: 34163 diff changeset	457 # of bundle. We should probably clean up or drop this return code
21c2df59a1da bundle2: move exception handling into part iterator Durham Goode <durham@fb.com> parents: 34163 diff changeset	458 # craziness in a future version.
21c2df59a1da bundle2: move exception handling into part iterator Durham Goode <durham@fb.com> parents: 34163 diff changeset	459 exc.duringunbundle2 = True
21c2df59a1da bundle2: move exception handling into part iterator Durham Goode <durham@fb.com> parents: 34163 diff changeset	460 salvaged = []
21c2df59a1da bundle2: move exception handling into part iterator Durham Goode <durham@fb.com> parents: 34163 diff changeset	461 replycaps = None
21c2df59a1da bundle2: move exception handling into part iterator Durham Goode <durham@fb.com> parents: 34163 diff changeset	462 if self.op.reply is not None:
21c2df59a1da bundle2: move exception handling into part iterator Durham Goode <durham@fb.com> parents: 34163 diff changeset	463 salvaged = self.op.reply.salvageoutput()
21c2df59a1da bundle2: move exception handling into part iterator Durham Goode <durham@fb.com> parents: 34163 diff changeset	464 replycaps = self.op.reply.capabilities
21c2df59a1da bundle2: move exception handling into part iterator Durham Goode <durham@fb.com> parents: 34163 diff changeset	465 exc._replycaps = replycaps
21c2df59a1da bundle2: move exception handling into part iterator Durham Goode <durham@fb.com> parents: 34163 diff changeset	466 exc._bundle2salvagedoutput = salvaged
21c2df59a1da bundle2: move exception handling into part iterator Durham Goode <durham@fb.com> parents: 34163 diff changeset	467
21c2df59a1da bundle2: move exception handling into part iterator Durham Goode <durham@fb.com> parents: 34163 diff changeset	468 # Re-raising from a variable loses the original stack. So only use
21c2df59a1da bundle2: move exception handling into part iterator Durham Goode <durham@fb.com> parents: 34163 diff changeset	469 # that form if we need to.
21c2df59a1da bundle2: move exception handling into part iterator Durham Goode <durham@fb.com> parents: 34163 diff changeset	470 if seekerror:
21c2df59a1da bundle2: move exception handling into part iterator Durham Goode <durham@fb.com> parents: 34163 diff changeset	471 raise exc
21c2df59a1da bundle2: move exception handling into part iterator Durham Goode <durham@fb.com> parents: 34163 diff changeset	472
43076 2372284d9457 formatting: blacken the codebase Augie Fackler <augie@google.com> parents: 42935 diff changeset	473 self.repo.ui.debug(
43077 687b865b95ad formatting: byteify all mercurial/ and hgext/ string literals Augie Fackler <augie@google.com> parents: 43076 diff changeset	474 b'bundle2-input-bundle: %i parts total\n' % self.count
43076 2372284d9457 formatting: blacken the codebase Augie Fackler <augie@google.com> parents: 42935 diff changeset	475 )
2372284d9457 formatting: blacken the codebase Augie Fackler <augie@google.com> parents: 42935 diff changeset	476
34162 e9e0e1143fc5 bundle2: move part iterator a separate class Durham Goode <durham@fb.com> parents: 34116 diff changeset	477
49876 4188e75af983 bundleoperation: optionnaly record the `remote` that produced the bundle Pierre-Yves David <pierre-yves.david@octobus.net> parents: 49764 diff changeset	478 def processbundle(
4188e75af983 bundleoperation: optionnaly record the `remote` that produced the bundle Pierre-Yves David <pierre-yves.david@octobus.net> parents: 49764 diff changeset	479 repo,
4188e75af983 bundleoperation: optionnaly record the `remote` that produced the bundle Pierre-Yves David <pierre-yves.david@octobus.net> parents: 49764 diff changeset	480 unbundler,
4188e75af983 bundleoperation: optionnaly record the `remote` that produced the bundle Pierre-Yves David <pierre-yves.david@octobus.net> parents: 49764 diff changeset	481 transactiongetter=None,
4188e75af983 bundleoperation: optionnaly record the `remote` that produced the bundle Pierre-Yves David <pierre-yves.david@octobus.net> parents: 49764 diff changeset	482 op=None,
4188e75af983 bundleoperation: optionnaly record the `remote` that produced the bundle Pierre-Yves David <pierre-yves.david@octobus.net> parents: 49764 diff changeset	483 source=b'',
4188e75af983 bundleoperation: optionnaly record the `remote` that produced the bundle Pierre-Yves David <pierre-yves.david@octobus.net> parents: 49764 diff changeset	484 remote=None,
4188e75af983 bundleoperation: optionnaly record the `remote` that produced the bundle Pierre-Yves David <pierre-yves.david@octobus.net> parents: 49764 diff changeset	485 ):
20889 deed5edb72de bundle2: first version of a bundle processing Pierre-Yves David <pierre-yves.david@fb.com> parents: 20887 diff changeset	486 """This function process a bundle, apply effect to/from a repo
deed5edb72de bundle2: first version of a bundle processing Pierre-Yves David <pierre-yves.david@fb.com> parents: 20887 diff changeset	487
20947 c33d7bf53812 bundle2: feed a unbundle20 to the `processbundle` function Pierre-Yves David <pierre-yves.david@fb.com> parents: 20892 diff changeset	488 It iterates over each part then searches for and uses the proper handling
c33d7bf53812 bundle2: feed a unbundle20 to the `processbundle` function Pierre-Yves David <pierre-yves.david@fb.com> parents: 20892 diff changeset	489 code to process the part. Parts are processed in order.
20889 deed5edb72de bundle2: first version of a bundle processing Pierre-Yves David <pierre-yves.david@fb.com> parents: 20887 diff changeset	490
20891 1c6cd23fc221 bundle2: add some distinction between mandatory and advisory part Pierre-Yves David <pierre-yves.david@fb.com> parents: 20890 diff changeset	491 Unknown Mandatory part will abort the process.
24851 df0ce98c882f bundle2: also save output when error happens during part processing Pierre-Yves David <pierre-yves.david@fb.com> parents: 24849 diff changeset	492
df0ce98c882f bundle2: also save output when error happens during part processing Pierre-Yves David <pierre-yves.david@fb.com> parents: 24849 diff changeset	493 It is temporarily possible to provide a prebuilt bundleoperation to the
df0ce98c882f bundle2: also save output when error happens during part processing Pierre-Yves David <pierre-yves.david@fb.com> parents: 24849 diff changeset	494 function. This is used to ensure output is properly propagated in case of
df0ce98c882f bundle2: also save output when error happens during part processing Pierre-Yves David <pierre-yves.david@fb.com> parents: 24849 diff changeset	495 an error during the unbundling. This output capturing part will likely be
df0ce98c882f bundle2: also save output when error happens during part processing Pierre-Yves David <pierre-yves.david@fb.com> parents: 24849 diff changeset	496 reworked and this ability will probably go away in the process.
20889 deed5edb72de bundle2: first version of a bundle processing Pierre-Yves David <pierre-yves.david@fb.com> parents: 20887 diff changeset	497 """
24851 df0ce98c882f bundle2: also save output when error happens during part processing Pierre-Yves David <pierre-yves.david@fb.com> parents: 24849 diff changeset	498 if op is None:
df0ce98c882f bundle2: also save output when error happens during part processing Pierre-Yves David <pierre-yves.david@fb.com> parents: 24849 diff changeset	499 if transactiongetter is None:
df0ce98c882f bundle2: also save output when error happens during part processing Pierre-Yves David <pierre-yves.david@fb.com> parents: 24849 diff changeset	500 transactiongetter = _notransaction
49876 4188e75af983 bundleoperation: optionnaly record the `remote` that produced the bundle Pierre-Yves David <pierre-yves.david@octobus.net> parents: 49764 diff changeset	501 op = bundleoperation(
4188e75af983 bundleoperation: optionnaly record the `remote` that produced the bundle Pierre-Yves David <pierre-yves.david@octobus.net> parents: 49764 diff changeset	502 repo,
4188e75af983 bundleoperation: optionnaly record the `remote` that produced the bundle Pierre-Yves David <pierre-yves.david@octobus.net> parents: 49764 diff changeset	503 transactiongetter,
4188e75af983 bundleoperation: optionnaly record the `remote` that produced the bundle Pierre-Yves David <pierre-yves.david@octobus.net> parents: 49764 diff changeset	504 source=source,
4188e75af983 bundleoperation: optionnaly record the `remote` that produced the bundle Pierre-Yves David <pierre-yves.david@octobus.net> parents: 49764 diff changeset	505 remote=remote,
4188e75af983 bundleoperation: optionnaly record the `remote` that produced the bundle Pierre-Yves David <pierre-yves.david@octobus.net> parents: 49764 diff changeset	506 )
20889 deed5edb72de bundle2: first version of a bundle processing Pierre-Yves David <pierre-yves.david@fb.com> parents: 20887 diff changeset	507 # todo:
deed5edb72de bundle2: first version of a bundle processing Pierre-Yves David <pierre-yves.david@fb.com> parents: 20887 diff changeset	508 # - replace this is a init function soon.
deed5edb72de bundle2: first version of a bundle processing Pierre-Yves David <pierre-yves.david@fb.com> parents: 20887 diff changeset	509 # - exception catching
deed5edb72de bundle2: first version of a bundle processing Pierre-Yves David <pierre-yves.david@fb.com> parents: 20887 diff changeset	510 unbundler.params
25331 53ca47ef645d bundle2: add generic debug output regarding processed bundle Pierre-Yves David <pierre-yves.david@fb.com> parents: 25324 diff changeset	511 if repo.ui.debugflag:
43077 687b865b95ad formatting: byteify all mercurial/ and hgext/ string literals Augie Fackler <augie@google.com> parents: 43076 diff changeset	512 msg = [b'bundle2-input-bundle:']
25331 53ca47ef645d bundle2: add generic debug output regarding processed bundle Pierre-Yves David <pierre-yves.david@fb.com> parents: 25324 diff changeset	513 if unbundler.params:
43077 687b865b95ad formatting: byteify all mercurial/ and hgext/ string literals Augie Fackler <augie@google.com> parents: 43076 diff changeset	514 msg.append(b' %i params' % len(unbundler.params))
33791 119e1c6be1ce bundle2: fix transaction availability detection Boris Feld <boris.feld@octobus.net> parents: 33737 diff changeset	515 if op._gettransaction is None or op._gettransaction is _notransaction:
43077 687b865b95ad formatting: byteify all mercurial/ and hgext/ string literals Augie Fackler <augie@google.com> parents: 43076 diff changeset	516 msg.append(b' no-transaction')
25331 53ca47ef645d bundle2: add generic debug output regarding processed bundle Pierre-Yves David <pierre-yves.david@fb.com> parents: 25324 diff changeset	517 else:
43077 687b865b95ad formatting: byteify all mercurial/ and hgext/ string literals Augie Fackler <augie@google.com> parents: 43076 diff changeset	518 msg.append(b' with-transaction')
687b865b95ad formatting: byteify all mercurial/ and hgext/ string literals Augie Fackler <augie@google.com> parents: 43076 diff changeset	519 msg.append(b'\n')
687b865b95ad formatting: byteify all mercurial/ and hgext/ string literals Augie Fackler <augie@google.com> parents: 43076 diff changeset	520 repo.ui.debug(b''.join(msg))
34162 e9e0e1143fc5 bundle2: move part iterator a separate class Durham Goode <durham@fb.com> parents: 34116 diff changeset	521
34268 f010097c885c bundle2: move part processing to a separate function Durham Goode <durham@fb.com> parents: 34267 diff changeset	522 processparts(repo, op, unbundler)
f010097c885c bundle2: move part processing to a separate function Durham Goode <durham@fb.com> parents: 34267 diff changeset	523
f010097c885c bundle2: move part processing to a separate function Durham Goode <durham@fb.com> parents: 34267 diff changeset	524 return op
f010097c885c bundle2: move part processing to a separate function Durham Goode <durham@fb.com> parents: 34267 diff changeset	525
43076 2372284d9457 formatting: blacken the codebase Augie Fackler <augie@google.com> parents: 42935 diff changeset	526
34268 f010097c885c bundle2: move part processing to a separate function Durham Goode <durham@fb.com> parents: 34267 diff changeset	527 def processparts(repo, op, unbundler):
34164 21c2df59a1da bundle2: move exception handling into part iterator Durham Goode <durham@fb.com> parents: 34163 diff changeset	528 with partiterator(repo, op, unbundler) as parts:
21c2df59a1da bundle2: move exception handling into part iterator Durham Goode <durham@fb.com> parents: 34163 diff changeset	529 for part in parts:
21c2df59a1da bundle2: move exception handling into part iterator Durham Goode <durham@fb.com> parents: 34163 diff changeset	530 _processpart(op, part)
25332 0cdd1c811705 bundle2: add generic debug output at the end of bundle processing Pierre-Yves David <pierre-yves.david@fb.com> parents: 25331 diff changeset	531
43076 2372284d9457 formatting: blacken the codebase Augie Fackler <augie@google.com> parents: 42935 diff changeset	532
33050 f0efd2bffe1e bundle: extract _processchangegroup() method Martin von Zweigbergk <martinvonz@google.com> parents: 33049 diff changeset	533 def _processchangegroup(op, cg, tr, source, url, **kwargs):
49878 f1887500f3ec delta-find: add a `delta-reuse-policy` on configuration `path` Pierre-Yves David <pierre-yves.david@octobus.net> parents: 49876 diff changeset	534 if op.remote is not None and op.remote.path is not None:
f1887500f3ec delta-find: add a `delta-reuse-policy` on configuration `path` Pierre-Yves David <pierre-yves.david@octobus.net> parents: 49876 diff changeset	535 remote_path = op.remote.path
f1887500f3ec delta-find: add a `delta-reuse-policy` on configuration `path` Pierre-Yves David <pierre-yves.david@octobus.net> parents: 49876 diff changeset	536 kwargs = kwargs.copy()
f1887500f3ec delta-find: add a `delta-reuse-policy` on configuration `path` Pierre-Yves David <pierre-yves.david@octobus.net> parents: 49876 diff changeset	537 kwargs['delta_base_reuse_policy'] = remote_path.delta_reuse_policy
33461 bb72031f0ea8 changegroup: stop returning and recording added nodes in 'cg.apply' Boris Feld <boris.feld@octobus.net> parents: 33460 diff changeset	538 ret = cg.apply(op.repo, tr, source, url, **kwargs)
45957 89a2afe31e82 formating: upgrade to black 20.8b1 Augie Fackler <raf@durin42.com> parents: 45682 diff changeset	539 op.records.add(
89a2afe31e82 formating: upgrade to black 20.8b1 Augie Fackler <raf@durin42.com> parents: 45682 diff changeset	540 b'changegroup',
89a2afe31e82 formating: upgrade to black 20.8b1 Augie Fackler <raf@durin42.com> parents: 45682 diff changeset	541 {
89a2afe31e82 formating: upgrade to black 20.8b1 Augie Fackler <raf@durin42.com> parents: 45682 diff changeset	542 b'return': ret,
89a2afe31e82 formating: upgrade to black 20.8b1 Augie Fackler <raf@durin42.com> parents: 45682 diff changeset	543 },
89a2afe31e82 formating: upgrade to black 20.8b1 Augie Fackler <raf@durin42.com> parents: 45682 diff changeset	544 )
33050 f0efd2bffe1e bundle: extract _processchangegroup() method Martin von Zweigbergk <martinvonz@google.com> parents: 33049 diff changeset	545 return ret
f0efd2bffe1e bundle: extract _processchangegroup() method Martin von Zweigbergk <martinvonz@google.com> parents: 33049 diff changeset	546
43076 2372284d9457 formatting: blacken the codebase Augie Fackler <augie@google.com> parents: 42935 diff changeset	547
34266 07e4170f02f3 bundle2: move handler validation out of processpart Durham Goode <durham@fb.com> parents: 34265 diff changeset	548 def _gethandler(op, part):
43077 687b865b95ad formatting: byteify all mercurial/ and hgext/ string literals Augie Fackler <augie@google.com> parents: 43076 diff changeset	549 status = b'unknown' # used by debug output
34266 07e4170f02f3 bundle2: move handler validation out of processpart Durham Goode <durham@fb.com> parents: 34265 diff changeset	550 try:
07e4170f02f3 bundle2: move handler validation out of processpart Durham Goode <durham@fb.com> parents: 34265 diff changeset	551 handler = parthandlermapping.get(part.type)
07e4170f02f3 bundle2: move handler validation out of processpart Durham Goode <durham@fb.com> parents: 34265 diff changeset	552 if handler is None:
43077 687b865b95ad formatting: byteify all mercurial/ and hgext/ string literals Augie Fackler <augie@google.com> parents: 43076 diff changeset	553 status = b'unsupported-type'
34266 07e4170f02f3 bundle2: move handler validation out of processpart Durham Goode <durham@fb.com> parents: 34265 diff changeset	554 raise error.BundleUnknownFeatureError(parttype=part.type)
43077 687b865b95ad formatting: byteify all mercurial/ and hgext/ string literals Augie Fackler <augie@google.com> parents: 43076 diff changeset	555 indebug(op.ui, b'found a handler for part %s' % part.type)
34266 07e4170f02f3 bundle2: move handler validation out of processpart Durham Goode <durham@fb.com> parents: 34265 diff changeset	556 unknownparams = part.mandatorykeys - handler.params
07e4170f02f3 bundle2: move handler validation out of processpart Durham Goode <durham@fb.com> parents: 34265 diff changeset	557 if unknownparams:
07e4170f02f3 bundle2: move handler validation out of processpart Durham Goode <durham@fb.com> parents: 34265 diff changeset	558 unknownparams = list(unknownparams)
07e4170f02f3 bundle2: move handler validation out of processpart Durham Goode <durham@fb.com> parents: 34265 diff changeset	559 unknownparams.sort()
43077 687b865b95ad formatting: byteify all mercurial/ and hgext/ string literals Augie Fackler <augie@google.com> parents: 43076 diff changeset	560 status = b'unsupported-params (%s)' % b', '.join(unknownparams)
43076 2372284d9457 formatting: blacken the codebase Augie Fackler <augie@google.com> parents: 42935 diff changeset	561 raise error.BundleUnknownFeatureError(
2372284d9457 formatting: blacken the codebase Augie Fackler <augie@google.com> parents: 42935 diff changeset	562 parttype=part.type, params=unknownparams
2372284d9457 formatting: blacken the codebase Augie Fackler <augie@google.com> parents: 42935 diff changeset	563 )
43077 687b865b95ad formatting: byteify all mercurial/ and hgext/ string literals Augie Fackler <augie@google.com> parents: 43076 diff changeset	564 status = b'supported'
34266 07e4170f02f3 bundle2: move handler validation out of processpart Durham Goode <durham@fb.com> parents: 34265 diff changeset	565 except error.BundleUnknownFeatureError as exc:
43076 2372284d9457 formatting: blacken the codebase Augie Fackler <augie@google.com> parents: 42935 diff changeset	566 if part.mandatory: # mandatory parts
34266 07e4170f02f3 bundle2: move handler validation out of processpart Durham Goode <durham@fb.com> parents: 34265 diff changeset	567 raise
43077 687b865b95ad formatting: byteify all mercurial/ and hgext/ string literals Augie Fackler <augie@google.com> parents: 43076 diff changeset	568 indebug(op.ui, b'ignoring unsupported advisory part %s' % exc)
43076 2372284d9457 formatting: blacken the codebase Augie Fackler <augie@google.com> parents: 42935 diff changeset	569 return # skip to part processing
34266 07e4170f02f3 bundle2: move handler validation out of processpart Durham Goode <durham@fb.com> parents: 34265 diff changeset	570 finally:
07e4170f02f3 bundle2: move handler validation out of processpart Durham Goode <durham@fb.com> parents: 34265 diff changeset	571 if op.ui.debugflag:
43077 687b865b95ad formatting: byteify all mercurial/ and hgext/ string literals Augie Fackler <augie@google.com> parents: 43076 diff changeset	572 msg = [b'bundle2-input-part: "%s"' % part.type]
34266 07e4170f02f3 bundle2: move handler validation out of processpart Durham Goode <durham@fb.com> parents: 34265 diff changeset	573 if not part.mandatory:
43077 687b865b95ad formatting: byteify all mercurial/ and hgext/ string literals Augie Fackler <augie@google.com> parents: 43076 diff changeset	574 msg.append(b' (advisory)')
34266 07e4170f02f3 bundle2: move handler validation out of processpart Durham Goode <durham@fb.com> parents: 34265 diff changeset	575 nbmp = len(part.mandatorykeys)
07e4170f02f3 bundle2: move handler validation out of processpart Durham Goode <durham@fb.com> parents: 34265 diff changeset	576 nbap = len(part.params) - nbmp
07e4170f02f3 bundle2: move handler validation out of processpart Durham Goode <durham@fb.com> parents: 34265 diff changeset	577 if nbmp or nbap:
43077 687b865b95ad formatting: byteify all mercurial/ and hgext/ string literals Augie Fackler <augie@google.com> parents: 43076 diff changeset	578 msg.append(b' (params:')
34266 07e4170f02f3 bundle2: move handler validation out of processpart Durham Goode <durham@fb.com> parents: 34265 diff changeset	579 if nbmp:
43077 687b865b95ad formatting: byteify all mercurial/ and hgext/ string literals Augie Fackler <augie@google.com> parents: 43076 diff changeset	580 msg.append(b' %i mandatory' % nbmp)
34266 07e4170f02f3 bundle2: move handler validation out of processpart Durham Goode <durham@fb.com> parents: 34265 diff changeset	581 if nbap:
43077 687b865b95ad formatting: byteify all mercurial/ and hgext/ string literals Augie Fackler <augie@google.com> parents: 43076 diff changeset	582 msg.append(b' %i advisory' % nbmp)
687b865b95ad formatting: byteify all mercurial/ and hgext/ string literals Augie Fackler <augie@google.com> parents: 43076 diff changeset	583 msg.append(b')')
687b865b95ad formatting: byteify all mercurial/ and hgext/ string literals Augie Fackler <augie@google.com> parents: 43076 diff changeset	584 msg.append(b' %s\n' % status)
687b865b95ad formatting: byteify all mercurial/ and hgext/ string literals Augie Fackler <augie@google.com> parents: 43076 diff changeset	585 op.ui.debug(b''.join(msg))
34266 07e4170f02f3 bundle2: move handler validation out of processpart Durham Goode <durham@fb.com> parents: 34265 diff changeset	586
07e4170f02f3 bundle2: move handler validation out of processpart Durham Goode <durham@fb.com> parents: 34265 diff changeset	587 return handler
07e4170f02f3 bundle2: move handler validation out of processpart Durham Goode <durham@fb.com> parents: 34265 diff changeset	588
43076 2372284d9457 formatting: blacken the codebase Augie Fackler <augie@google.com> parents: 42935 diff changeset	589
23008 d3137827016a bundle2: extract processing of part into its own function Pierre-Yves David <pierre-yves.david@fb.com> parents: 23001 diff changeset	590 def _processpart(op, part):
d3137827016a bundle2: extract processing of part into its own function Pierre-Yves David <pierre-yves.david@fb.com> parents: 23001 diff changeset	591 """process a single part from a bundle
d3137827016a bundle2: extract processing of part into its own function Pierre-Yves David <pierre-yves.david@fb.com> parents: 23001 diff changeset	592
d3137827016a bundle2: extract processing of part into its own function Pierre-Yves David <pierre-yves.david@fb.com> parents: 23001 diff changeset	593 The part is guaranteed to have been fully consumed when the function exits
d3137827016a bundle2: extract processing of part into its own function Pierre-Yves David <pierre-yves.david@fb.com> parents: 23001 diff changeset	594 (even if an exception is raised)."""
34267 cc7b37c90616 bundle2: remove unnecessary try finally Durham Goode <durham@fb.com> parents: 34266 diff changeset	595 handler = _gethandler(op, part)
cc7b37c90616 bundle2: remove unnecessary try finally Durham Goode <durham@fb.com> parents: 34266 diff changeset	596 if handler is None:
cc7b37c90616 bundle2: remove unnecessary try finally Durham Goode <durham@fb.com> parents: 34266 diff changeset	597 return
23008 d3137827016a bundle2: extract processing of part into its own function Pierre-Yves David <pierre-yves.david@fb.com> parents: 23001 diff changeset	598
34267 cc7b37c90616 bundle2: remove unnecessary try finally Durham Goode <durham@fb.com> parents: 34266 diff changeset	599 # handler is called outside the above try block so that we don't
cc7b37c90616 bundle2: remove unnecessary try finally Durham Goode <durham@fb.com> parents: 34266 diff changeset	600 # risk catching KeyErrors from anything other than the
cc7b37c90616 bundle2: remove unnecessary try finally Durham Goode <durham@fb.com> parents: 34266 diff changeset	601 # parthandlermapping lookup (any KeyError raised by handler()
cc7b37c90616 bundle2: remove unnecessary try finally Durham Goode <durham@fb.com> parents: 34266 diff changeset	602 # itself represents a defect of a different variety).
cc7b37c90616 bundle2: remove unnecessary try finally Durham Goode <durham@fb.com> parents: 34266 diff changeset	603 output = None
cc7b37c90616 bundle2: remove unnecessary try finally Durham Goode <durham@fb.com> parents: 34266 diff changeset	604 if op.captureoutput and op.reply is not None:
cc7b37c90616 bundle2: remove unnecessary try finally Durham Goode <durham@fb.com> parents: 34266 diff changeset	605 op.ui.pushbuffer(error=True, subproc=True)
43077 687b865b95ad formatting: byteify all mercurial/ and hgext/ string literals Augie Fackler <augie@google.com> parents: 43076 diff changeset	606 output = b''
34267 cc7b37c90616 bundle2: remove unnecessary try finally Durham Goode <durham@fb.com> parents: 34266 diff changeset	607 try:
cc7b37c90616 bundle2: remove unnecessary try finally Durham Goode <durham@fb.com> parents: 34266 diff changeset	608 handler(op, part)
23008 d3137827016a bundle2: extract processing of part into its own function Pierre-Yves David <pierre-yves.david@fb.com> parents: 23001 diff changeset	609 finally:
34267 cc7b37c90616 bundle2: remove unnecessary try finally Durham Goode <durham@fb.com> parents: 34266 diff changeset	610 if output is not None:
cc7b37c90616 bundle2: remove unnecessary try finally Durham Goode <durham@fb.com> parents: 34266 diff changeset	611 output = op.ui.popbuffer()
cc7b37c90616 bundle2: remove unnecessary try finally Durham Goode <durham@fb.com> parents: 34266 diff changeset	612 if output:
43077 687b865b95ad formatting: byteify all mercurial/ and hgext/ string literals Augie Fackler <augie@google.com> parents: 43076 diff changeset	613 outpart = op.reply.newpart(b'output', data=output, mandatory=False)
34267 cc7b37c90616 bundle2: remove unnecessary try finally Durham Goode <durham@fb.com> parents: 34266 diff changeset	614 outpart.addparam(
43077 687b865b95ad formatting: byteify all mercurial/ and hgext/ string literals Augie Fackler <augie@google.com> parents: 43076 diff changeset	615 b'in-reply-to', pycompat.bytestr(part.id), mandatory=False
43076 2372284d9457 formatting: blacken the codebase Augie Fackler <augie@google.com> parents: 42935 diff changeset	616 )
2372284d9457 formatting: blacken the codebase Augie Fackler <augie@google.com> parents: 42935 diff changeset	617
23008 d3137827016a bundle2: extract processing of part into its own function Pierre-Yves David <pierre-yves.david@fb.com> parents: 23001 diff changeset	618
52670 4cb75772818d pyupgrade: drop the quoting around type annotations Matt Harbison <matt_harbison@yahoo.com> parents: 52669 diff changeset	619 def decodecaps(blob: bytes) -> Capabilities:
23139 e53f6b72a0e4 spelling: fixes from proofreading of spell checker issues Mads Kiilerich <madski@unity3d.com> parents: 23067 diff changeset	620 """decode a bundle2 caps bytes blob into a dictionary
21138 f469879d27ec bundle2: extract capabilities decoding Pierre-Yves David <pierre-yves.david@fb.com> parents: 21137 diff changeset	621
f469879d27ec bundle2: extract capabilities decoding Pierre-Yves David <pierre-yves.david@fb.com> parents: 21137 diff changeset	622 The blob is a list of capabilities (one per line)
f469879d27ec bundle2: extract capabilities decoding Pierre-Yves David <pierre-yves.david@fb.com> parents: 21137 diff changeset	623 Capabilities may have values using a line of the form::
f469879d27ec bundle2: extract capabilities decoding Pierre-Yves David <pierre-yves.david@fb.com> parents: 21137 diff changeset	624
f469879d27ec bundle2: extract capabilities decoding Pierre-Yves David <pierre-yves.david@fb.com> parents: 21137 diff changeset	625 capability=value1,value2,value3
f469879d27ec bundle2: extract capabilities decoding Pierre-Yves David <pierre-yves.david@fb.com> parents: 21137 diff changeset	626
f469879d27ec bundle2: extract capabilities decoding Pierre-Yves David <pierre-yves.david@fb.com> parents: 21137 diff changeset	627 The values are always a list."""
f469879d27ec bundle2: extract capabilities decoding Pierre-Yves David <pierre-yves.david@fb.com> parents: 21137 diff changeset	628 caps = {}
f469879d27ec bundle2: extract capabilities decoding Pierre-Yves David <pierre-yves.david@fb.com> parents: 21137 diff changeset	629 for line in blob.splitlines():
f469879d27ec bundle2: extract capabilities decoding Pierre-Yves David <pierre-yves.david@fb.com> parents: 21137 diff changeset	630 if not line:
f469879d27ec bundle2: extract capabilities decoding Pierre-Yves David <pierre-yves.david@fb.com> parents: 21137 diff changeset	631 continue
43077 687b865b95ad formatting: byteify all mercurial/ and hgext/ string literals Augie Fackler <augie@google.com> parents: 43076 diff changeset	632 if b'=' not in line:
21138 f469879d27ec bundle2: extract capabilities decoding Pierre-Yves David <pierre-yves.david@fb.com> parents: 21137 diff changeset	633 key, vals = line, ()
f469879d27ec bundle2: extract capabilities decoding Pierre-Yves David <pierre-yves.david@fb.com> parents: 21137 diff changeset	634 else:
43077 687b865b95ad formatting: byteify all mercurial/ and hgext/ string literals Augie Fackler <augie@google.com> parents: 43076 diff changeset	635 key, vals = line.split(b'=', 1)
687b865b95ad formatting: byteify all mercurial/ and hgext/ string literals Augie Fackler <augie@google.com> parents: 43076 diff changeset	636 vals = vals.split(b',')
28883 032c4c2f802a pycompat: switch to util.urlreq/util.urlerr for py3 compat timeless <timeless@mozdev.org> parents: 28672 diff changeset	637 key = urlreq.unquote(key)
032c4c2f802a pycompat: switch to util.urlreq/util.urlerr for py3 compat timeless <timeless@mozdev.org> parents: 28672 diff changeset	638 vals = [urlreq.unquote(v) for v in vals]
21138 f469879d27ec bundle2: extract capabilities decoding Pierre-Yves David <pierre-yves.david@fb.com> parents: 21137 diff changeset	639 caps[key] = vals
f469879d27ec bundle2: extract capabilities decoding Pierre-Yves David <pierre-yves.david@fb.com> parents: 21137 diff changeset	640 return caps
f469879d27ec bundle2: extract capabilities decoding Pierre-Yves David <pierre-yves.david@fb.com> parents: 21137 diff changeset	641
43076 2372284d9457 formatting: blacken the codebase Augie Fackler <augie@google.com> parents: 42935 diff changeset	642
21139 2b8c82f7f11d bundle2: capabilities encoding Pierre-Yves David <pierre-yves.david@fb.com> parents: 21138 diff changeset	643 def encodecaps(caps):
2b8c82f7f11d bundle2: capabilities encoding Pierre-Yves David <pierre-yves.david@fb.com> parents: 21138 diff changeset	644 """encode a bundle2 caps dictionary into a bytes blob"""
2b8c82f7f11d bundle2: capabilities encoding Pierre-Yves David <pierre-yves.david@fb.com> parents: 21138 diff changeset	645 chunks = []
2b8c82f7f11d bundle2: capabilities encoding Pierre-Yves David <pierre-yves.david@fb.com> parents: 21138 diff changeset	646 for ca in sorted(caps):
2b8c82f7f11d bundle2: capabilities encoding Pierre-Yves David <pierre-yves.david@fb.com> parents: 21138 diff changeset	647 vals = caps[ca]
28883 032c4c2f802a pycompat: switch to util.urlreq/util.urlerr for py3 compat timeless <timeless@mozdev.org> parents: 28672 diff changeset	648 ca = urlreq.quote(ca)
032c4c2f802a pycompat: switch to util.urlreq/util.urlerr for py3 compat timeless <timeless@mozdev.org> parents: 28672 diff changeset	649 vals = [urlreq.quote(v) for v in vals]
21139 2b8c82f7f11d bundle2: capabilities encoding Pierre-Yves David <pierre-yves.david@fb.com> parents: 21138 diff changeset	650 if vals:
43077 687b865b95ad formatting: byteify all mercurial/ and hgext/ string literals Augie Fackler <augie@google.com> parents: 43076 diff changeset	651 ca = b"%s=%s" % (ca, b','.join(vals))
21139 2b8c82f7f11d bundle2: capabilities encoding Pierre-Yves David <pierre-yves.david@fb.com> parents: 21138 diff changeset	652 chunks.append(ca)
43077 687b865b95ad formatting: byteify all mercurial/ and hgext/ string literals Augie Fackler <augie@google.com> parents: 43076 diff changeset	653 return b'\n'.join(chunks)
21139 2b8c82f7f11d bundle2: capabilities encoding Pierre-Yves David <pierre-yves.david@fb.com> parents: 21138 diff changeset	654
43076 2372284d9457 formatting: blacken the codebase Augie Fackler <augie@google.com> parents: 42935 diff changeset	655
28666 ae53ecc47414 bundle: move writebundle() from changegroup.py to bundle2.py (API) Martin von Zweigbergk <martinvonz@google.com> parents: 27953 diff changeset	656 bundletypes = {
43077 687b865b95ad formatting: byteify all mercurial/ and hgext/ string literals Augie Fackler <augie@google.com> parents: 43076 diff changeset	657 b"": (b"", b'UN'), # only when using unbundle on ssh and old http servers
43076 2372284d9457 formatting: blacken the codebase Augie Fackler <augie@google.com> parents: 42935 diff changeset	658 # since the unification ssh accepts a header but there
2372284d9457 formatting: blacken the codebase Augie Fackler <augie@google.com> parents: 42935 diff changeset	659 # is no capability signaling it.
43077 687b865b95ad formatting: byteify all mercurial/ and hgext/ string literals Augie Fackler <augie@google.com> parents: 43076 diff changeset	660 b"HG20": (), # special-cased below
687b865b95ad formatting: byteify all mercurial/ and hgext/ string literals Augie Fackler <augie@google.com> parents: 43076 diff changeset	661 b"HG10UN": (b"HG10UN", b'UN'),
687b865b95ad formatting: byteify all mercurial/ and hgext/ string literals Augie Fackler <augie@google.com> parents: 43076 diff changeset	662 b"HG10BZ": (b"HG10", b'BZ'),
687b865b95ad formatting: byteify all mercurial/ and hgext/ string literals Augie Fackler <augie@google.com> parents: 43076 diff changeset	663 b"HG10GZ": (b"HG10GZ", b'GZ'),
28666 ae53ecc47414 bundle: move writebundle() from changegroup.py to bundle2.py (API) Martin von Zweigbergk <martinvonz@google.com> parents: 27953 diff changeset	664 }
ae53ecc47414 bundle: move writebundle() from changegroup.py to bundle2.py (API) Martin von Zweigbergk <martinvonz@google.com> parents: 27953 diff changeset	665
ae53ecc47414 bundle: move writebundle() from changegroup.py to bundle2.py (API) Martin von Zweigbergk <martinvonz@google.com> parents: 27953 diff changeset	666 # hgweb uses this list to communicate its preferred type
43077 687b865b95ad formatting: byteify all mercurial/ and hgext/ string literals Augie Fackler <augie@google.com> parents: 43076 diff changeset	667 bundlepriority = [b'HG10GZ', b'HG10BZ', b'HG10UN']
28666 ae53ecc47414 bundle: move writebundle() from changegroup.py to bundle2.py (API) Martin von Zweigbergk <martinvonz@google.com> parents: 27953 diff changeset	668
43076 2372284d9457 formatting: blacken the codebase Augie Fackler <augie@google.com> parents: 42935 diff changeset	669
49037 642e31cb55f0 py3: use class X: instead of class X(object): Gregory Szorc <gregory.szorc@gmail.com> parents: 49004 diff changeset	670 class bundle20:
20801 9c5183cb9bca bundle2: very first version of a bundle2 bundler Pierre-Yves David <pierre-yves.david@fb.com> parents: diff changeset	671 """represent an outgoing bundle2 container
9c5183cb9bca bundle2: very first version of a bundle2 bundler Pierre-Yves David <pierre-yves.david@fb.com> parents: diff changeset	672
21599 57cd844d7a5b bundle2: have ``newpart`` automatically add the part to the bundle Pierre-Yves David <pierre-yves.david@fb.com> parents: 21598 diff changeset	673 Use the `addparam` method to add stream level parameter. and `newpart` to
20856 8a6a86c9a5b5 bundle2: support bundling of empty part (with a type) Pierre-Yves David <pierre-yves.david@fb.com> parents: 20844 diff changeset	674 populate it. Then call `getchunks` to retrieve all the binary chunks of
21024 7731a2281cf0 spelling: fixes from spell checker Mads Kiilerich <madski@unity3d.com> parents: 21020 diff changeset	675 data that compose the bundle2 container."""
20801 9c5183cb9bca bundle2: very first version of a bundle2 bundler Pierre-Yves David <pierre-yves.david@fb.com> parents: diff changeset	676
43077 687b865b95ad formatting: byteify all mercurial/ and hgext/ string literals Augie Fackler <augie@google.com> parents: 43076 diff changeset	677 _magicstring = b'HG20'
24640 685639f9430d bundle20: move magic string into the class Pierre-Yves David <pierre-yves.david@fb.com> parents: 24071 diff changeset	678
52670 4cb75772818d pyupgrade: drop the quoting around type annotations Matt Harbison <matt_harbison@yahoo.com> parents: 52669 diff changeset	679 def __init__(self, ui, capabilities: Optional[Capabilities] = None):
51731 6fc31e7bd5db typing: add some type hints for bundle2 capabilities Matt Harbison <matt_harbison@yahoo.com> parents: 51658 diff changeset	680 if capabilities is None:
6fc31e7bd5db typing: add some type hints for bundle2 capabilities Matt Harbison <matt_harbison@yahoo.com> parents: 51658 diff changeset	681 capabilities = {}
6fc31e7bd5db typing: add some type hints for bundle2 capabilities Matt Harbison <matt_harbison@yahoo.com> parents: 51658 diff changeset	682
20842 938718d72624 bundle2: print debug information during bundling Pierre-Yves David <pierre-yves.david@fb.com> parents: 20814 diff changeset	683 self.ui = ui
20801 9c5183cb9bca bundle2: very first version of a bundle2 bundler Pierre-Yves David <pierre-yves.david@fb.com> parents: diff changeset	684 self._params = []
9c5183cb9bca bundle2: very first version of a bundle2 bundler Pierre-Yves David <pierre-yves.david@fb.com> parents: diff changeset	685 self._parts = []
52670 4cb75772818d pyupgrade: drop the quoting around type annotations Matt Harbison <matt_harbison@yahoo.com> parents: 52669 diff changeset	686 self.capabilities: Capabilities = dict(capabilities)
43077 687b865b95ad formatting: byteify all mercurial/ and hgext/ string literals Augie Fackler <augie@google.com> parents: 43076 diff changeset	687 self._compengine = util.compengines.forbundletype(b'UN')
30757 511a4bf52754 bundle2: allow compression options to be passed to compressor Gregory Szorc <gregory.szorc@gmail.com> parents: 30448 diff changeset	688 self._compopts = None
35787 a84dbc87dae9 exchange: send bundle2 stream clones uncompressed Gregory Szorc <gregory.szorc@gmail.com> parents: 35783 diff changeset	689 # If compression is being handled by a consumer of the raw
a84dbc87dae9 exchange: send bundle2 stream clones uncompressed Gregory Szorc <gregory.szorc@gmail.com> parents: 35783 diff changeset	690 # data (e.g. the wire protocol), unsetting this flag tells
a84dbc87dae9 exchange: send bundle2 stream clones uncompressed Gregory Szorc <gregory.szorc@gmail.com> parents: 35783 diff changeset	691 # consumers that the bundle is best left uncompressed.
a84dbc87dae9 exchange: send bundle2 stream clones uncompressed Gregory Szorc <gregory.szorc@gmail.com> parents: 35783 diff changeset	692 self.prefercompressed = True
26404 795f02a24b87 bundle2: allow compressed bundle Pierre-Yves David <pierre-yves.david@fb.com> parents: 26396 diff changeset	693
30757 511a4bf52754 bundle2: allow compression options to be passed to compressor Gregory Szorc <gregory.szorc@gmail.com> parents: 30448 diff changeset	694 def setcompression(self, alg, compopts=None):
26404 795f02a24b87 bundle2: allow compressed bundle Pierre-Yves David <pierre-yves.david@fb.com> parents: 26396 diff changeset	695 """setup core part compression to <alg>"""
43077 687b865b95ad formatting: byteify all mercurial/ and hgext/ string literals Augie Fackler <augie@google.com> parents: 43076 diff changeset	696 if alg in (None, b'UN'):
26404 795f02a24b87 bundle2: allow compressed bundle Pierre-Yves David <pierre-yves.david@fb.com> parents: 26396 diff changeset	697 return
43077 687b865b95ad formatting: byteify all mercurial/ and hgext/ string literals Augie Fackler <augie@google.com> parents: 43076 diff changeset	698 assert not any(n.lower() == b'compression' for n, v in self._params)
687b865b95ad formatting: byteify all mercurial/ and hgext/ string literals Augie Fackler <augie@google.com> parents: 43076 diff changeset	699 self.addparam(b'Compression', alg)
30361 f81002f736d7 bundle2: use new compression engine API for compression Gregory Szorc <gregory.szorc@gmail.com> parents: 30177 diff changeset	700 self._compengine = util.compengines.forbundletype(alg)
30757 511a4bf52754 bundle2: allow compression options to be passed to compressor Gregory Szorc <gregory.szorc@gmail.com> parents: 30448 diff changeset	701 self._compopts = compopts
20801 9c5183cb9bca bundle2: very first version of a bundle2 bundler Pierre-Yves David <pierre-yves.david@fb.com> parents: diff changeset	702
21900 b8bd97085ec9 bundle2: add a ``bundle20.nbparts`` property Pierre-Yves David <pierre-yves.david@fb.com> parents: 21660 diff changeset	703 @property
b8bd97085ec9 bundle2: add a ``bundle20.nbparts`` property Pierre-Yves David <pierre-yves.david@fb.com> parents: 21660 diff changeset	704 def nbparts(self):
b8bd97085ec9 bundle2: add a ``bundle20.nbparts`` property Pierre-Yves David <pierre-yves.david@fb.com> parents: 21660 diff changeset	705 """total number of parts added to the bundler"""
b8bd97085ec9 bundle2: add a ``bundle20.nbparts`` property Pierre-Yves David <pierre-yves.david@fb.com> parents: 21660 diff changeset	706 return len(self._parts)
b8bd97085ec9 bundle2: add a ``bundle20.nbparts`` property Pierre-Yves David <pierre-yves.david@fb.com> parents: 21660 diff changeset	707
21597 1daad9dcdba2 bundle2: small doc update on the bundler Pierre-Yves David <pierre-yves.david@fb.com> parents: 21186 diff changeset	708 # methods used to defines the bundle2 content
20804 db9d3991d2c6 bundle2: support bundling simple parameter Pierre-Yves David <pierre-yves.david@fb.com> parents: 20803 diff changeset	709 def addparam(self, name, value=None):
db9d3991d2c6 bundle2: support bundling simple parameter Pierre-Yves David <pierre-yves.david@fb.com> parents: 20803 diff changeset	710 """add a stream level parameter"""
20813 8c74b3ce5b70 bundle2: refuse empty parameter name Pierre-Yves David <pierre-yves.david@fb.com> parents: 20812 diff changeset	711 if not name:
38610 d474b3b44d4f bundle2: use ProgrammingError to report bad use of addparam() Yuya Nishihara <yuya@tcha.org> parents: 37837 diff changeset	712 raise error.ProgrammingError(b'empty parameter name')
43536 ea25b4673231 bundle2: more pytype suppressions around bytestring Augie Fackler <augie@google.com> parents: 43131 diff changeset	713 if name[0:1] not in pycompat.bytestr(
ea25b4673231 bundle2: more pytype suppressions around bytestring Augie Fackler <augie@google.com> parents: 43131 diff changeset	714 string.ascii_letters # pytype: disable=wrong-arg-types
ea25b4673231 bundle2: more pytype suppressions around bytestring Augie Fackler <augie@google.com> parents: 43131 diff changeset	715 ):
43076 2372284d9457 formatting: blacken the codebase Augie Fackler <augie@google.com> parents: 42935 diff changeset	716 raise error.ProgrammingError(
2372284d9457 formatting: blacken the codebase Augie Fackler <augie@google.com> parents: 42935 diff changeset	717 b'non letter first character: %s' % name
2372284d9457 formatting: blacken the codebase Augie Fackler <augie@google.com> parents: 42935 diff changeset	718 )
20804 db9d3991d2c6 bundle2: support bundling simple parameter Pierre-Yves David <pierre-yves.david@fb.com> parents: 20803 diff changeset	719 self._params.append((name, value))
db9d3991d2c6 bundle2: support bundling simple parameter Pierre-Yves David <pierre-yves.david@fb.com> parents: 20803 diff changeset	720
20856 8a6a86c9a5b5 bundle2: support bundling of empty part (with a type) Pierre-Yves David <pierre-yves.david@fb.com> parents: 20844 diff changeset	721 def addpart(self, part):
8a6a86c9a5b5 bundle2: support bundling of empty part (with a type) Pierre-Yves David <pierre-yves.david@fb.com> parents: 20844 diff changeset	722 """add a new part to the bundle2 container
8a6a86c9a5b5 bundle2: support bundling of empty part (with a type) Pierre-Yves David <pierre-yves.david@fb.com> parents: 20844 diff changeset	723
21024 7731a2281cf0 spelling: fixes from spell checker Mads Kiilerich <madski@unity3d.com> parents: 21020 diff changeset	724 Parts contains the actual applicative payload."""
20995 e995d104c87f bundle2: add an integer id to part Pierre-Yves David <pierre-yves.david@fb.com> parents: 20952 diff changeset	725 assert part.id is None
43076 2372284d9457 formatting: blacken the codebase Augie Fackler <augie@google.com> parents: 42935 diff changeset	726 part.id = len(self._parts) # very cheap counter
20856 8a6a86c9a5b5 bundle2: support bundling of empty part (with a type) Pierre-Yves David <pierre-yves.david@fb.com> parents: 20844 diff changeset	727 self._parts.append(part)
8a6a86c9a5b5 bundle2: support bundling of empty part (with a type) Pierre-Yves David <pierre-yves.david@fb.com> parents: 20844 diff changeset	728
21598 1b0dbb91de5b bundle2: add a ``newpart`` method to ``bundle20`` Pierre-Yves David <pierre-yves.david@fb.com> parents: 21597 diff changeset	729 def newpart(self, typeid, args, *kwargs):
21602 cc33ae50bab3 bundle2: warn about error during initialization in ``newpart`` docstring Pierre-Yves David <pierre-yves.david@fb.com> parents: 21601 diff changeset	730 """create a new part and add it to the containers
cc33ae50bab3 bundle2: warn about error during initialization in ``newpart`` docstring Pierre-Yves David <pierre-yves.david@fb.com> parents: 21601 diff changeset	731
cc33ae50bab3 bundle2: warn about error during initialization in ``newpart`` docstring Pierre-Yves David <pierre-yves.david@fb.com> parents: 21601 diff changeset	732 As the part is directly added to the containers. For now, this means
cc33ae50bab3 bundle2: warn about error during initialization in ``newpart`` docstring Pierre-Yves David <pierre-yves.david@fb.com> parents: 21601 diff changeset	733 that any failure to properly initialize the part after calling
cc33ae50bab3 bundle2: warn about error during initialization in ``newpart`` docstring Pierre-Yves David <pierre-yves.david@fb.com> parents: 21601 diff changeset	734 ``newpart`` should result in a failure of the whole bundling process.
cc33ae50bab3 bundle2: warn about error during initialization in ``newpart`` docstring Pierre-Yves David <pierre-yves.david@fb.com> parents: 21601 diff changeset	735
cc33ae50bab3 bundle2: warn about error during initialization in ``newpart`` docstring Pierre-Yves David <pierre-yves.david@fb.com> parents: 21601 diff changeset	736 You can still fall back to manually create and add if you need better
cc33ae50bab3 bundle2: warn about error during initialization in ``newpart`` docstring Pierre-Yves David <pierre-yves.david@fb.com> parents: 21601 diff changeset	737 control."""
21598 1b0dbb91de5b bundle2: add a ``newpart`` method to ``bundle20`` Pierre-Yves David <pierre-yves.david@fb.com> parents: 21597 diff changeset	738 part = bundlepart(typeid, args, *kwargs)
21599 57cd844d7a5b bundle2: have ``newpart`` automatically add the part to the bundle Pierre-Yves David <pierre-yves.david@fb.com> parents: 21598 diff changeset	739 self.addpart(part)
21598 1b0dbb91de5b bundle2: add a ``newpart`` method to ``bundle20`` Pierre-Yves David <pierre-yves.david@fb.com> parents: 21597 diff changeset	740 return part
1b0dbb91de5b bundle2: add a ``newpart`` method to ``bundle20`` Pierre-Yves David <pierre-yves.david@fb.com> parents: 21597 diff changeset	741
21597 1daad9dcdba2 bundle2: small doc update on the bundler Pierre-Yves David <pierre-yves.david@fb.com> parents: 21186 diff changeset	742 # methods used to generate the bundle2 stream
52709 279e217d6041 typing: lock in the new type annotations detected with the pyupgrade changes Matt Harbison <matt_harbison@yahoo.com> parents: 52670 diff changeset	743 def getchunks(self) -> Iterator[bytes]:
25322 1f7c0403f8be bundle2: add generic debug output regarding generated bundle Pierre-Yves David <pierre-yves.david@fb.com> parents: 25321 diff changeset	744 if self.ui.debugflag:
43077 687b865b95ad formatting: byteify all mercurial/ and hgext/ string literals Augie Fackler <augie@google.com> parents: 43076 diff changeset	745 msg = [b'bundle2-output-bundle: "%s",' % self._magicstring]
25322 1f7c0403f8be bundle2: add generic debug output regarding generated bundle Pierre-Yves David <pierre-yves.david@fb.com> parents: 25321 diff changeset	746 if self._params:
43077 687b865b95ad formatting: byteify all mercurial/ and hgext/ string literals Augie Fackler <augie@google.com> parents: 43076 diff changeset	747 msg.append(b' (%i params)' % len(self._params))
687b865b95ad formatting: byteify all mercurial/ and hgext/ string literals Augie Fackler <augie@google.com> parents: 43076 diff changeset	748 msg.append(b' %i parts total\n' % len(self._parts))
687b865b95ad formatting: byteify all mercurial/ and hgext/ string literals Augie Fackler <augie@google.com> parents: 43076 diff changeset	749 self.ui.debug(b''.join(msg))
687b865b95ad formatting: byteify all mercurial/ and hgext/ string literals Augie Fackler <augie@google.com> parents: 43076 diff changeset	750 outdebug(self.ui, b'start emission of %s stream' % self._magicstring)
24640 685639f9430d bundle20: move magic string into the class Pierre-Yves David <pierre-yves.david@fb.com> parents: 24071 diff changeset	751 yield self._magicstring
20804 db9d3991d2c6 bundle2: support bundling simple parameter Pierre-Yves David <pierre-yves.david@fb.com> parents: 20803 diff changeset	752 param = self._paramchunk()
43077 687b865b95ad formatting: byteify all mercurial/ and hgext/ string literals Augie Fackler <augie@google.com> parents: 43076 diff changeset	753 outdebug(self.ui, b'bundle parameter: %s' % param)
20804 db9d3991d2c6 bundle2: support bundling simple parameter Pierre-Yves David <pierre-yves.david@fb.com> parents: 20803 diff changeset	754 yield _pack(_fstreamparamsize, len(param))
db9d3991d2c6 bundle2: support bundling simple parameter Pierre-Yves David <pierre-yves.david@fb.com> parents: 20803 diff changeset	755 if param:
db9d3991d2c6 bundle2: support bundling simple parameter Pierre-Yves David <pierre-yves.david@fb.com> parents: 20803 diff changeset	756 yield param
52669 e627cc25b6f3 pyupgrade: rewrite `yield` statements in a loop to `yield from` Matt Harbison <matt_harbison@yahoo.com> parents: 52668 diff changeset	757 yield from self._compengine.compressstream(
43076 2372284d9457 formatting: blacken the codebase Augie Fackler <augie@google.com> parents: 42935 diff changeset	758 self._getcorechunk(), self._compopts
52669 e627cc25b6f3 pyupgrade: rewrite `yield` statements in a loop to `yield from` Matt Harbison <matt_harbison@yahoo.com> parents: 52668 diff changeset	759 )
20802 520df53ad26a bundle2: a very first version of bundle2 unbundler Pierre-Yves David <pierre-yves.david@fb.com> parents: 20801 diff changeset	760
20804 db9d3991d2c6 bundle2: support bundling simple parameter Pierre-Yves David <pierre-yves.david@fb.com> parents: 20803 diff changeset	761 def _paramchunk(self):
db9d3991d2c6 bundle2: support bundling simple parameter Pierre-Yves David <pierre-yves.david@fb.com> parents: 20803 diff changeset	762 """return a encoded version of all stream parameters"""
db9d3991d2c6 bundle2: support bundling simple parameter Pierre-Yves David <pierre-yves.david@fb.com> parents: 20803 diff changeset	763 blocks = []
20809 b93bb639451a bundle2: support for bundling parameter value Pierre-Yves David <pierre-yves.david@fb.com> parents: 20808 diff changeset	764 for par, value in self._params:
28883 032c4c2f802a pycompat: switch to util.urlreq/util.urlerr for py3 compat timeless <timeless@mozdev.org> parents: 28672 diff changeset	765 par = urlreq.quote(par)
20809 b93bb639451a bundle2: support for bundling parameter value Pierre-Yves David <pierre-yves.david@fb.com> parents: 20808 diff changeset	766 if value is not None:
28883 032c4c2f802a pycompat: switch to util.urlreq/util.urlerr for py3 compat timeless <timeless@mozdev.org> parents: 28672 diff changeset	767 value = urlreq.quote(value)
43077 687b865b95ad formatting: byteify all mercurial/ and hgext/ string literals Augie Fackler <augie@google.com> parents: 43076 diff changeset	768 par = b'%s=%s' % (par, value)
20809 b93bb639451a bundle2: support for bundling parameter value Pierre-Yves David <pierre-yves.david@fb.com> parents: 20808 diff changeset	769 blocks.append(par)
43077 687b865b95ad formatting: byteify all mercurial/ and hgext/ string literals Augie Fackler <augie@google.com> parents: 43076 diff changeset	770 return b' '.join(blocks)
20804 db9d3991d2c6 bundle2: support bundling simple parameter Pierre-Yves David <pierre-yves.david@fb.com> parents: 20803 diff changeset	771
26396 d90c30801cdf bundle20: extract core payload generation in its own function Pierre-Yves David <pierre-yves.david@fb.com> parents: 26395 diff changeset	772 def _getcorechunk(self):
d90c30801cdf bundle20: extract core payload generation in its own function Pierre-Yves David <pierre-yves.david@fb.com> parents: 26395 diff changeset	773 """yield chunk for the core part of the bundle
d90c30801cdf bundle20: extract core payload generation in its own function Pierre-Yves David <pierre-yves.david@fb.com> parents: 26395 diff changeset	774
d90c30801cdf bundle20: extract core payload generation in its own function Pierre-Yves David <pierre-yves.david@fb.com> parents: 26395 diff changeset	775 (all but headers and parameters)"""
43077 687b865b95ad formatting: byteify all mercurial/ and hgext/ string literals Augie Fackler <augie@google.com> parents: 43076 diff changeset	776 outdebug(self.ui, b'start of parts')
26396 d90c30801cdf bundle20: extract core payload generation in its own function Pierre-Yves David <pierre-yves.david@fb.com> parents: 26395 diff changeset	777 for part in self._parts:
43077 687b865b95ad formatting: byteify all mercurial/ and hgext/ string literals Augie Fackler <augie@google.com> parents: 43076 diff changeset	778 outdebug(self.ui, b'bundle part: "%s"' % part.type)
52669 e627cc25b6f3 pyupgrade: rewrite `yield` statements in a loop to `yield from` Matt Harbison <matt_harbison@yahoo.com> parents: 52668 diff changeset	779 yield from part.getchunks(ui=self.ui)
43077 687b865b95ad formatting: byteify all mercurial/ and hgext/ string literals Augie Fackler <augie@google.com> parents: 43076 diff changeset	780 outdebug(self.ui, b'end of bundle')
26396 d90c30801cdf bundle20: extract core payload generation in its own function Pierre-Yves David <pierre-yves.david@fb.com> parents: 26395 diff changeset	781 yield _pack(_fpartheadersize, 0)
d90c30801cdf bundle20: extract core payload generation in its own function Pierre-Yves David <pierre-yves.david@fb.com> parents: 26395 diff changeset	782
24794 21f2e8f48ae1 bundle2: add a 'salvageoutput' method on bundle20 Pierre-Yves David <pierre-yves.david@fb.com> parents: 24793 diff changeset	783 def salvageoutput(self):
21f2e8f48ae1 bundle2: add a 'salvageoutput' method on bundle20 Pierre-Yves David <pierre-yves.david@fb.com> parents: 24793 diff changeset	784 """return a list with a copy of all output parts in the bundle
21f2e8f48ae1 bundle2: add a 'salvageoutput' method on bundle20 Pierre-Yves David <pierre-yves.david@fb.com> parents: 24793 diff changeset	785
21f2e8f48ae1 bundle2: add a 'salvageoutput' method on bundle20 Pierre-Yves David <pierre-yves.david@fb.com> parents: 24793 diff changeset	786 This is meant to be used during error handling to make sure we preserve
21f2e8f48ae1 bundle2: add a 'salvageoutput' method on bundle20 Pierre-Yves David <pierre-yves.david@fb.com> parents: 24793 diff changeset	787 server output"""
21f2e8f48ae1 bundle2: add a 'salvageoutput' method on bundle20 Pierre-Yves David <pierre-yves.david@fb.com> parents: 24793 diff changeset	788 salvaged = []
21f2e8f48ae1 bundle2: add a 'salvageoutput' method on bundle20 Pierre-Yves David <pierre-yves.david@fb.com> parents: 24793 diff changeset	789 for part in self._parts:
43077 687b865b95ad formatting: byteify all mercurial/ and hgext/ string literals Augie Fackler <augie@google.com> parents: 43076 diff changeset	790 if part.type.startswith(b'output'):
24794 21f2e8f48ae1 bundle2: add a 'salvageoutput' method on bundle20 Pierre-Yves David <pierre-yves.david@fb.com> parents: 24793 diff changeset	791 salvaged.append(part.copy())
21f2e8f48ae1 bundle2: add a 'salvageoutput' method on bundle20 Pierre-Yves David <pierre-yves.david@fb.com> parents: 24793 diff changeset	792 return salvaged
21f2e8f48ae1 bundle2: add a 'salvageoutput' method on bundle20 Pierre-Yves David <pierre-yves.david@fb.com> parents: 24793 diff changeset	793
21f2e8f48ae1 bundle2: add a 'salvageoutput' method on bundle20 Pierre-Yves David <pierre-yves.david@fb.com> parents: 24793 diff changeset	794
49037 642e31cb55f0 py3: use class X: instead of class X(object): Gregory Szorc <gregory.szorc@gmail.com> parents: 49004 diff changeset	795 class unpackermixin:
21013 a813caca89b3 bundle2: extract stream/unpack logic in an unpackermixin Pierre-Yves David <pierre-yves.david@fb.com> parents: 21005 diff changeset	796 """A mixin to extract bytes and struct data from a stream"""
20802 520df53ad26a bundle2: a very first version of bundle2 unbundler Pierre-Yves David <pierre-yves.david@fb.com> parents: 20801 diff changeset	797
21013 a813caca89b3 bundle2: extract stream/unpack logic in an unpackermixin Pierre-Yves David <pierre-yves.david@fb.com> parents: 21005 diff changeset	798 def __init__(self, fp):
20802 520df53ad26a bundle2: a very first version of bundle2 unbundler Pierre-Yves David <pierre-yves.david@fb.com> parents: 20801 diff changeset	799 self._fp = fp
520df53ad26a bundle2: a very first version of bundle2 unbundler Pierre-Yves David <pierre-yves.david@fb.com> parents: 20801 diff changeset	800
520df53ad26a bundle2: a very first version of bundle2 unbundler Pierre-Yves David <pierre-yves.david@fb.com> parents: 20801 diff changeset	801 def _unpack(self, format):
31862 9bd9e9cb3f69 bundle2: clarify the docstring of unpackermixin methods Pierre-Yves David <pierre-yves.david@ens-lyon.org> parents: 31861 diff changeset	802 """unpack this struct format from the stream
9bd9e9cb3f69 bundle2: clarify the docstring of unpackermixin methods Pierre-Yves David <pierre-yves.david@ens-lyon.org> parents: 31861 diff changeset	803
9bd9e9cb3f69 bundle2: clarify the docstring of unpackermixin methods Pierre-Yves David <pierre-yves.david@ens-lyon.org> parents: 31861 diff changeset	804 This method is meant for internal usage by the bundle2 protocol only.
9bd9e9cb3f69 bundle2: clarify the docstring of unpackermixin methods Pierre-Yves David <pierre-yves.david@ens-lyon.org> parents: 31861 diff changeset	805 They directly manipulate the low level stream including bundle2 level
9bd9e9cb3f69 bundle2: clarify the docstring of unpackermixin methods Pierre-Yves David <pierre-yves.david@ens-lyon.org> parents: 31861 diff changeset	806 instruction.
9bd9e9cb3f69 bundle2: clarify the docstring of unpackermixin methods Pierre-Yves David <pierre-yves.david@ens-lyon.org> parents: 31861 diff changeset	807
9bd9e9cb3f69 bundle2: clarify the docstring of unpackermixin methods Pierre-Yves David <pierre-yves.david@ens-lyon.org> parents: 31861 diff changeset	808 Do not use it to implement higher-level logic or methods."""
20802 520df53ad26a bundle2: a very first version of bundle2 unbundler Pierre-Yves David <pierre-yves.david@fb.com> parents: 20801 diff changeset	809 data = self._readexact(struct.calcsize(format))
520df53ad26a bundle2: a very first version of bundle2 unbundler Pierre-Yves David <pierre-yves.david@fb.com> parents: 20801 diff changeset	810 return _unpack(format, data)
520df53ad26a bundle2: a very first version of bundle2 unbundler Pierre-Yves David <pierre-yves.david@fb.com> parents: 20801 diff changeset	811
520df53ad26a bundle2: a very first version of bundle2 unbundler Pierre-Yves David <pierre-yves.david@fb.com> parents: 20801 diff changeset	812 def _readexact(self, size):
31862 9bd9e9cb3f69 bundle2: clarify the docstring of unpackermixin methods Pierre-Yves David <pierre-yves.david@ens-lyon.org> parents: 31861 diff changeset	813 """read exactly <size> bytes from the stream
9bd9e9cb3f69 bundle2: clarify the docstring of unpackermixin methods Pierre-Yves David <pierre-yves.david@ens-lyon.org> parents: 31861 diff changeset	814
9bd9e9cb3f69 bundle2: clarify the docstring of unpackermixin methods Pierre-Yves David <pierre-yves.david@ens-lyon.org> parents: 31861 diff changeset	815 This method is meant for internal usage by the bundle2 protocol only.
9bd9e9cb3f69 bundle2: clarify the docstring of unpackermixin methods Pierre-Yves David <pierre-yves.david@ens-lyon.org> parents: 31861 diff changeset	816 They directly manipulate the low level stream including bundle2 level
9bd9e9cb3f69 bundle2: clarify the docstring of unpackermixin methods Pierre-Yves David <pierre-yves.david@ens-lyon.org> parents: 31861 diff changeset	817 instruction.
9bd9e9cb3f69 bundle2: clarify the docstring of unpackermixin methods Pierre-Yves David <pierre-yves.david@ens-lyon.org> parents: 31861 diff changeset	818
9bd9e9cb3f69 bundle2: clarify the docstring of unpackermixin methods Pierre-Yves David <pierre-yves.david@ens-lyon.org> parents: 31861 diff changeset	819 Do not use it to implement higher-level logic or methods."""
20802 520df53ad26a bundle2: a very first version of bundle2 unbundler Pierre-Yves David <pierre-yves.david@fb.com> parents: 20801 diff changeset	820 return changegroup.readexactly(self._fp, size)
520df53ad26a bundle2: a very first version of bundle2 unbundler Pierre-Yves David <pierre-yves.david@fb.com> parents: 20801 diff changeset	821
43076 2372284d9457 formatting: blacken the codebase Augie Fackler <augie@google.com> parents: 42935 diff changeset	822
25640 39f0064a3079 bundle2.getunbundler: rename "header" to "magicstring" Pierre-Yves David <pierre-yves.david@fb.com> parents: 25518 diff changeset	823 def getunbundler(ui, fp, magicstring=None):
39f0064a3079 bundle2.getunbundler: rename "header" to "magicstring" Pierre-Yves David <pierre-yves.david@fb.com> parents: 25518 diff changeset	824 """return a valid unbundler object for a given magicstring"""
39f0064a3079 bundle2.getunbundler: rename "header" to "magicstring" Pierre-Yves David <pierre-yves.david@fb.com> parents: 25518 diff changeset	825 if magicstring is None:
39f0064a3079 bundle2.getunbundler: rename "header" to "magicstring" Pierre-Yves David <pierre-yves.david@fb.com> parents: 25518 diff changeset	826 magicstring = changegroup.readexactly(fp, 4)
39f0064a3079 bundle2.getunbundler: rename "header" to "magicstring" Pierre-Yves David <pierre-yves.david@fb.com> parents: 25518 diff changeset	827 magic, version = magicstring[0:2], magicstring[2:4]
43077 687b865b95ad formatting: byteify all mercurial/ and hgext/ string literals Augie Fackler <augie@google.com> parents: 43076 diff changeset	828 if magic != b'HG':
33134 0aae80d14fed bundle2: add some debugging information to the not-a-bundle error Siddharth Agarwal <sid0@fb.com> parents: 33056 diff changeset	829 ui.debug(
43077 687b865b95ad formatting: byteify all mercurial/ and hgext/ string literals Augie Fackler <augie@google.com> parents: 43076 diff changeset	830 b"error: invalid magic: %r (version %r), should be 'HG'\n"
43076 2372284d9457 formatting: blacken the codebase Augie Fackler <augie@google.com> parents: 42935 diff changeset	831 % (magic, version)
2372284d9457 formatting: blacken the codebase Augie Fackler <augie@google.com> parents: 42935 diff changeset	832 )
43077 687b865b95ad formatting: byteify all mercurial/ and hgext/ string literals Augie Fackler <augie@google.com> parents: 43076 diff changeset	833 raise error.Abort(_(b'not a Mercurial bundle'))
24648 5cac3accdaa1 unbundle20: allow generic dispatch between unbundlers Pierre-Yves David <pierre-yves.david@fb.com> parents: 24642 diff changeset	834 unbundlerclass = formatmap.get(version)
5cac3accdaa1 unbundle20: allow generic dispatch between unbundlers Pierre-Yves David <pierre-yves.david@fb.com> parents: 24642 diff changeset	835 if unbundlerclass is None:
43077 687b865b95ad formatting: byteify all mercurial/ and hgext/ string literals Augie Fackler <augie@google.com> parents: 43076 diff changeset	836 raise error.Abort(_(b'unknown bundle version %s') % version)
24648 5cac3accdaa1 unbundle20: allow generic dispatch between unbundlers Pierre-Yves David <pierre-yves.david@fb.com> parents: 24642 diff changeset	837 unbundler = unbundlerclass(ui, fp)
43077 687b865b95ad formatting: byteify all mercurial/ and hgext/ string literals Augie Fackler <augie@google.com> parents: 43076 diff changeset	838 indebug(ui, b'start processing of %s stream' % magicstring)
24642 54e5c239c2d9 unbundle20: move header parsing into the 'getunbundler' function Pierre-Yves David <pierre-yves.david@fb.com> parents: 24641 diff changeset	839 return unbundler
24641 60fecc5b14a4 unbundle20: retrieve unbundler instances through a factory function Pierre-Yves David <pierre-yves.david@fb.com> parents: 24640 diff changeset	840
43076 2372284d9457 formatting: blacken the codebase Augie Fackler <augie@google.com> parents: 42935 diff changeset	841
21013 a813caca89b3 bundle2: extract stream/unpack logic in an unpackermixin Pierre-Yves David <pierre-yves.david@fb.com> parents: 21005 diff changeset	842 class unbundle20(unpackermixin):
a813caca89b3 bundle2: extract stream/unpack logic in an unpackermixin Pierre-Yves David <pierre-yves.david@fb.com> parents: 21005 diff changeset	843 """interpret a bundle2 stream
a813caca89b3 bundle2: extract stream/unpack logic in an unpackermixin Pierre-Yves David <pierre-yves.david@fb.com> parents: 21005 diff changeset	844
21129 07bcbf326c8d bundle2: use an official iterparts method to unbundle parts Pierre-Yves David <pierre-yves.david@fb.com> parents: 21066 diff changeset	845 This class is fed with a binary stream and yields parts through its
07bcbf326c8d bundle2: use an official iterparts method to unbundle parts Pierre-Yves David <pierre-yves.david@fb.com> parents: 21066 diff changeset	846 `iterparts` methods."""
21013 a813caca89b3 bundle2: extract stream/unpack logic in an unpackermixin Pierre-Yves David <pierre-yves.david@fb.com> parents: 21005 diff changeset	847
43077 687b865b95ad formatting: byteify all mercurial/ and hgext/ string literals Augie Fackler <augie@google.com> parents: 43076 diff changeset	848 _magicstring = b'HG20'
26542 b87e4638dabf bundle2: add a way to just forward the bundle2 stream to another user Pierre-Yves David <pierre-yves.david@fb.com> parents: 26541 diff changeset	849
24642 54e5c239c2d9 unbundle20: move header parsing into the 'getunbundler' function Pierre-Yves David <pierre-yves.david@fb.com> parents: 24641 diff changeset	850 def __init__(self, ui, fp):
21066 5ecfe76d0d96 bundle2: make header reading optional Pierre-Yves David <pierre-yves.david@fb.com> parents: 21062 diff changeset	851 """If header is specified, we do not read it out of the stream."""
21013 a813caca89b3 bundle2: extract stream/unpack logic in an unpackermixin Pierre-Yves David <pierre-yves.david@fb.com> parents: 21005 diff changeset	852 self.ui = ui
43077 687b865b95ad formatting: byteify all mercurial/ and hgext/ string literals Augie Fackler <augie@google.com> parents: 43076 diff changeset	853 self._compengine = util.compengines.forbundletype(b'UN')
26802 42f705f2c02d bundle2: make unbundle.compressed return True when compressed Pierre-Yves David <pierre-yves.david@fb.com> parents: 26795 diff changeset	854 self._compressed = None
52668 5cc8deb96b48 pyupgrade: modernize calls to superclass methods Matt Harbison <matt_harbison@yahoo.com> parents: 52665 diff changeset	855 super().__init__(fp)
21013 a813caca89b3 bundle2: extract stream/unpack logic in an unpackermixin Pierre-Yves David <pierre-yves.david@fb.com> parents: 21005 diff changeset	856
20802 520df53ad26a bundle2: a very first version of bundle2 unbundler Pierre-Yves David <pierre-yves.david@fb.com> parents: 20801 diff changeset	857 @util.propertycache
520df53ad26a bundle2: a very first version of bundle2 unbundler Pierre-Yves David <pierre-yves.david@fb.com> parents: 20801 diff changeset	858 def params(self):
21024 7731a2281cf0 spelling: fixes from spell checker Mads Kiilerich <madski@unity3d.com> parents: 21020 diff changeset	859 """dictionary of stream level parameters"""
43077 687b865b95ad formatting: byteify all mercurial/ and hgext/ string literals Augie Fackler <augie@google.com> parents: 43076 diff changeset	860 indebug(self.ui, b'reading bundle2 stream parameters')
20805 c5aaeca0cfbf bundle2: support for unbundling simple parameter Pierre-Yves David <pierre-yves.david@fb.com> parents: 20804 diff changeset	861 params = {}
c5aaeca0cfbf bundle2: support for unbundling simple parameter Pierre-Yves David <pierre-yves.david@fb.com> parents: 20804 diff changeset	862 paramssize = self._unpack(_fstreamparamsize)[0]
23011 006a81d07e57 bundle2: detect and disallow a negative chunk size Pierre-Yves David <pierre-yves.david@fb.com> parents: 23010 diff changeset	863 if paramssize < 0:
43076 2372284d9457 formatting: blacken the codebase Augie Fackler <augie@google.com> parents: 42935 diff changeset	864 raise error.BundleValueError(
43077 687b865b95ad formatting: byteify all mercurial/ and hgext/ string literals Augie Fackler <augie@google.com> parents: 43076 diff changeset	865 b'negative bundle param size: %i' % paramssize
43076 2372284d9457 formatting: blacken the codebase Augie Fackler <augie@google.com> parents: 42935 diff changeset	866 )
20805 c5aaeca0cfbf bundle2: support for unbundling simple parameter Pierre-Yves David <pierre-yves.david@fb.com> parents: 20804 diff changeset	867 if paramssize:
26541 d40029b4296e bundle2: split parameter retrieval and processing Pierre-Yves David <pierre-yves.david@fb.com> parents: 26404 diff changeset	868 params = self._readexact(paramssize)
d40029b4296e bundle2: split parameter retrieval and processing Pierre-Yves David <pierre-yves.david@fb.com> parents: 26404 diff changeset	869 params = self._processallparams(params)
20805 c5aaeca0cfbf bundle2: support for unbundling simple parameter Pierre-Yves David <pierre-yves.david@fb.com> parents: 20804 diff changeset	870 return params
20802 520df53ad26a bundle2: a very first version of bundle2 unbundler Pierre-Yves David <pierre-yves.david@fb.com> parents: 20801 diff changeset	871
52899 b8da7eed1657 pycompat: backed out changeset e02c36478284 Pierre-Yves David <pierre-yves.david@octobus.net> parents: 52892 diff changeset	872 def _processallparams(self, paramsblock):
47031 f38bf44e077f black: make codebase compatible with black v21.4b2 and v20.8b1 Kyle Lippincott <spectral@google.com> parents: 46907 diff changeset	873 """ """
29591 6215b5537ba5 bundle2: use a sorted dict for holding parameters Gregory Szorc <gregory.szorc@gmail.com> parents: 29294 diff changeset	874 params = util.sortdict()
43077 687b865b95ad formatting: byteify all mercurial/ and hgext/ string literals Augie Fackler <augie@google.com> parents: 43076 diff changeset	875 for p in paramsblock.split(b' '):
687b865b95ad formatting: byteify all mercurial/ and hgext/ string literals Augie Fackler <augie@google.com> parents: 43076 diff changeset	876 p = p.split(b'=', 1)
28883 032c4c2f802a pycompat: switch to util.urlreq/util.urlerr for py3 compat timeless <timeless@mozdev.org> parents: 28672 diff changeset	877 p = [urlreq.unquote(i) for i in p]
26541 d40029b4296e bundle2: split parameter retrieval and processing Pierre-Yves David <pierre-yves.david@fb.com> parents: 26404 diff changeset	878 if len(p) < 2:
d40029b4296e bundle2: split parameter retrieval and processing Pierre-Yves David <pierre-yves.david@fb.com> parents: 26404 diff changeset	879 p.append(None)
d40029b4296e bundle2: split parameter retrieval and processing Pierre-Yves David <pierre-yves.david@fb.com> parents: 26404 diff changeset	880 self._processparam(*p)
d40029b4296e bundle2: split parameter retrieval and processing Pierre-Yves David <pierre-yves.david@fb.com> parents: 26404 diff changeset	881 params[p[0]] = p[1]
d40029b4296e bundle2: split parameter retrieval and processing Pierre-Yves David <pierre-yves.david@fb.com> parents: 26404 diff changeset	882 return params
d40029b4296e bundle2: split parameter retrieval and processing Pierre-Yves David <pierre-yves.david@fb.com> parents: 26404 diff changeset	883
20844 2631204d7305 bundle2: implement the mandatory/advisory logic for parameter Pierre-Yves David <pierre-yves.david@fb.com> parents: 20843 diff changeset	884 def _processparam(self, name, value):
2631204d7305 bundle2: implement the mandatory/advisory logic for parameter Pierre-Yves David <pierre-yves.david@fb.com> parents: 20843 diff changeset	885 """process a parameter, applying its effect if needed
2631204d7305 bundle2: implement the mandatory/advisory logic for parameter Pierre-Yves David <pierre-yves.david@fb.com> parents: 20843 diff changeset	886
2631204d7305 bundle2: implement the mandatory/advisory logic for parameter Pierre-Yves David <pierre-yves.david@fb.com> parents: 20843 diff changeset	887 Parameter starting with a lower case letter are advisory and will be
2631204d7305 bundle2: implement the mandatory/advisory logic for parameter Pierre-Yves David <pierre-yves.david@fb.com> parents: 20843 diff changeset	888 ignored when unknown. Those starting with an upper case letter are
2631204d7305 bundle2: implement the mandatory/advisory logic for parameter Pierre-Yves David <pierre-yves.david@fb.com> parents: 20843 diff changeset	889 mandatory and will this function will raise a KeyError when unknown.
2631204d7305 bundle2: implement the mandatory/advisory logic for parameter Pierre-Yves David <pierre-yves.david@fb.com> parents: 20843 diff changeset	890
2631204d7305 bundle2: implement the mandatory/advisory logic for parameter Pierre-Yves David <pierre-yves.david@fb.com> parents: 20843 diff changeset	891 Note: no option are currently supported. Any input will be either
2631204d7305 bundle2: implement the mandatory/advisory logic for parameter Pierre-Yves David <pierre-yves.david@fb.com> parents: 20843 diff changeset	892 ignored or failing.
2631204d7305 bundle2: implement the mandatory/advisory logic for parameter Pierre-Yves David <pierre-yves.david@fb.com> parents: 20843 diff changeset	893 """
2631204d7305 bundle2: implement the mandatory/advisory logic for parameter Pierre-Yves David <pierre-yves.david@fb.com> parents: 20843 diff changeset	894 if not name:
43554 9f70512ae2cf cleanup: remove pointless r-prefixes on single-quoted strings Augie Fackler <augie@google.com> parents: 43543 diff changeset	895 raise ValueError('empty parameter name')
43536 ea25b4673231 bundle2: more pytype suppressions around bytestring Augie Fackler <augie@google.com> parents: 43131 diff changeset	896 if name[0:1] not in pycompat.bytestr(
ea25b4673231 bundle2: more pytype suppressions around bytestring Augie Fackler <augie@google.com> parents: 43131 diff changeset	897 string.ascii_letters # pytype: disable=wrong-arg-types
ea25b4673231 bundle2: more pytype suppressions around bytestring Augie Fackler <augie@google.com> parents: 43131 diff changeset	898 ):
43554 9f70512ae2cf cleanup: remove pointless r-prefixes on single-quoted strings Augie Fackler <augie@google.com> parents: 43543 diff changeset	899 raise ValueError('non letter first character: %s' % name)
26395 4e7b0bf9f0b1 unbundle20: allow registering handlers for stream level parameters Pierre-Yves David <pierre-yves.david@fb.com> parents: 26393 diff changeset	900 try:
4e7b0bf9f0b1 unbundle20: allow registering handlers for stream level parameters Pierre-Yves David <pierre-yves.david@fb.com> parents: 26393 diff changeset	901 handler = b2streamparamsmap[name.lower()]
4e7b0bf9f0b1 unbundle20: allow registering handlers for stream level parameters Pierre-Yves David <pierre-yves.david@fb.com> parents: 26393 diff changeset	902 except KeyError:
34284 a48b3db0baba bundle2: stop using %r to quote part names Augie Fackler <augie@google.com> parents: 34268 diff changeset	903 if name[0:1].islower():
43077 687b865b95ad formatting: byteify all mercurial/ and hgext/ string literals Augie Fackler <augie@google.com> parents: 43076 diff changeset	904 indebug(self.ui, b"ignoring unknown parameter %s" % name)
26395 4e7b0bf9f0b1 unbundle20: allow registering handlers for stream level parameters Pierre-Yves David <pierre-yves.david@fb.com> parents: 26393 diff changeset	905 else:
4e7b0bf9f0b1 unbundle20: allow registering handlers for stream level parameters Pierre-Yves David <pierre-yves.david@fb.com> parents: 26393 diff changeset	906 raise error.BundleUnknownFeatureError(params=(name,))
20844 2631204d7305 bundle2: implement the mandatory/advisory logic for parameter Pierre-Yves David <pierre-yves.david@fb.com> parents: 20843 diff changeset	907 else:
26395 4e7b0bf9f0b1 unbundle20: allow registering handlers for stream level parameters Pierre-Yves David <pierre-yves.david@fb.com> parents: 26393 diff changeset	908 handler(self, name, value)
20844 2631204d7305 bundle2: implement the mandatory/advisory logic for parameter Pierre-Yves David <pierre-yves.david@fb.com> parents: 20843 diff changeset	909
26542 b87e4638dabf bundle2: add a way to just forward the bundle2 stream to another user Pierre-Yves David <pierre-yves.david@fb.com> parents: 26541 diff changeset	910 def _forwardchunks(self):
b87e4638dabf bundle2: add a way to just forward the bundle2 stream to another user Pierre-Yves David <pierre-yves.david@fb.com> parents: 26541 diff changeset	911 """utility to transfer a bundle2 as binary
b87e4638dabf bundle2: add a way to just forward the bundle2 stream to another user Pierre-Yves David <pierre-yves.david@fb.com> parents: 26541 diff changeset	912
b87e4638dabf bundle2: add a way to just forward the bundle2 stream to another user Pierre-Yves David <pierre-yves.david@fb.com> parents: 26541 diff changeset	913 This is made necessary by the fact the 'getbundle' command over 'ssh'
50788 704c3d0878d9 cleanup: typos, formatting Joerg Sonnenberger <joerg@bec.de> parents: 50706 diff changeset	914 have no way to know when the reply ends, relying on the bundle to be
26542 b87e4638dabf bundle2: add a way to just forward the bundle2 stream to another user Pierre-Yves David <pierre-yves.david@fb.com> parents: 26541 diff changeset	915 interpreted to know its end. This is terrible and we are sorry, but we
b87e4638dabf bundle2: add a way to just forward the bundle2 stream to another user Pierre-Yves David <pierre-yves.david@fb.com> parents: 26541 diff changeset	916 needed to move forward to get general delta enabled.
b87e4638dabf bundle2: add a way to just forward the bundle2 stream to another user Pierre-Yves David <pierre-yves.david@fb.com> parents: 26541 diff changeset	917 """
b87e4638dabf bundle2: add a way to just forward the bundle2 stream to another user Pierre-Yves David <pierre-yves.david@fb.com> parents: 26541 diff changeset	918 yield self._magicstring
43421 be384a2052aa py3: don't use bytes with vars() or __dict__ Martin von Zweigbergk <martinvonz@google.com> parents: 43131 diff changeset	919 assert 'params' not in vars(self)
26542 b87e4638dabf bundle2: add a way to just forward the bundle2 stream to another user Pierre-Yves David <pierre-yves.david@fb.com> parents: 26541 diff changeset	920 paramssize = self._unpack(_fstreamparamsize)[0]
b87e4638dabf bundle2: add a way to just forward the bundle2 stream to another user Pierre-Yves David <pierre-yves.david@fb.com> parents: 26541 diff changeset	921 if paramssize < 0:
43076 2372284d9457 formatting: blacken the codebase Augie Fackler <augie@google.com> parents: 42935 diff changeset	922 raise error.BundleValueError(
43077 687b865b95ad formatting: byteify all mercurial/ and hgext/ string literals Augie Fackler <augie@google.com> parents: 43076 diff changeset	923 b'negative bundle param size: %i' % paramssize
43076 2372284d9457 formatting: blacken the codebase Augie Fackler <augie@google.com> parents: 42935 diff changeset	924 )
26542 b87e4638dabf bundle2: add a way to just forward the bundle2 stream to another user Pierre-Yves David <pierre-yves.david@fb.com> parents: 26541 diff changeset	925 if paramssize:
b87e4638dabf bundle2: add a way to just forward the bundle2 stream to another user Pierre-Yves David <pierre-yves.david@fb.com> parents: 26541 diff changeset	926 params = self._readexact(paramssize)
b87e4638dabf bundle2: add a way to just forward the bundle2 stream to another user Pierre-Yves David <pierre-yves.david@fb.com> parents: 26541 diff changeset	927 self._processallparams(params)
42143 29569f2db929 bundle2: handle compression in _forwardchunks Joerg Sonnenberger <joerg@bec.de> parents: 42057 diff changeset	928 # The payload itself is decompressed below, so drop
29569f2db929 bundle2: handle compression in _forwardchunks Joerg Sonnenberger <joerg@bec.de> parents: 42057 diff changeset	929 # the compression parameter passed down to compensate.
29569f2db929 bundle2: handle compression in _forwardchunks Joerg Sonnenberger <joerg@bec.de> parents: 42057 diff changeset	930 outparams = []
43077 687b865b95ad formatting: byteify all mercurial/ and hgext/ string literals Augie Fackler <augie@google.com> parents: 43076 diff changeset	931 for p in params.split(b' '):
687b865b95ad formatting: byteify all mercurial/ and hgext/ string literals Augie Fackler <augie@google.com> parents: 43076 diff changeset	932 k, v = p.split(b'=', 1)
687b865b95ad formatting: byteify all mercurial/ and hgext/ string literals Augie Fackler <augie@google.com> parents: 43076 diff changeset	933 if k.lower() != b'compression':
42143 29569f2db929 bundle2: handle compression in _forwardchunks Joerg Sonnenberger <joerg@bec.de> parents: 42057 diff changeset	934 outparams.append(p)
43077 687b865b95ad formatting: byteify all mercurial/ and hgext/ string literals Augie Fackler <augie@google.com> parents: 43076 diff changeset	935 outparams = b' '.join(outparams)
42143 29569f2db929 bundle2: handle compression in _forwardchunks Joerg Sonnenberger <joerg@bec.de> parents: 42057 diff changeset	936 yield _pack(_fstreamparamsize, len(outparams))
29569f2db929 bundle2: handle compression in _forwardchunks Joerg Sonnenberger <joerg@bec.de> parents: 42057 diff changeset	937 yield outparams
29569f2db929 bundle2: handle compression in _forwardchunks Joerg Sonnenberger <joerg@bec.de> parents: 42057 diff changeset	938 else:
29569f2db929 bundle2: handle compression in _forwardchunks Joerg Sonnenberger <joerg@bec.de> parents: 42057 diff changeset	939 yield _pack(_fstreamparamsize, paramssize)
26542 b87e4638dabf bundle2: add a way to just forward the bundle2 stream to another user Pierre-Yves David <pierre-yves.david@fb.com> parents: 26541 diff changeset	940 # From there, payload might need to be decompressed
30363 d045b4091197 bundle2: use compression engines API to obtain decompressor Gregory Szorc <gregory.szorc@gmail.com> parents: 30361 diff changeset	941 self._fp = self._compengine.decompressorreader(self._fp)
26542 b87e4638dabf bundle2: add a way to just forward the bundle2 stream to another user Pierre-Yves David <pierre-yves.david@fb.com> parents: 26541 diff changeset	942 emptycount = 0
b87e4638dabf bundle2: add a way to just forward the bundle2 stream to another user Pierre-Yves David <pierre-yves.david@fb.com> parents: 26541 diff changeset	943 while emptycount < 2:
b87e4638dabf bundle2: add a way to just forward the bundle2 stream to another user Pierre-Yves David <pierre-yves.david@fb.com> parents: 26541 diff changeset	944 # so we can brainlessly loop
b87e4638dabf bundle2: add a way to just forward the bundle2 stream to another user Pierre-Yves David <pierre-yves.david@fb.com> parents: 26541 diff changeset	945 assert _fpartheadersize == _fpayloadsize
b87e4638dabf bundle2: add a way to just forward the bundle2 stream to another user Pierre-Yves David <pierre-yves.david@fb.com> parents: 26541 diff changeset	946 size = self._unpack(_fpartheadersize)[0]
b87e4638dabf bundle2: add a way to just forward the bundle2 stream to another user Pierre-Yves David <pierre-yves.david@fb.com> parents: 26541 diff changeset	947 yield _pack(_fpartheadersize, size)
b87e4638dabf bundle2: add a way to just forward the bundle2 stream to another user Pierre-Yves David <pierre-yves.david@fb.com> parents: 26541 diff changeset	948 if size:
b87e4638dabf bundle2: add a way to just forward the bundle2 stream to another user Pierre-Yves David <pierre-yves.david@fb.com> parents: 26541 diff changeset	949 emptycount = 0
b87e4638dabf bundle2: add a way to just forward the bundle2 stream to another user Pierre-Yves David <pierre-yves.david@fb.com> parents: 26541 diff changeset	950 else:
b87e4638dabf bundle2: add a way to just forward the bundle2 stream to another user Pierre-Yves David <pierre-yves.david@fb.com> parents: 26541 diff changeset	951 emptycount += 1
b87e4638dabf bundle2: add a way to just forward the bundle2 stream to another user Pierre-Yves David <pierre-yves.david@fb.com> parents: 26541 diff changeset	952 continue
b87e4638dabf bundle2: add a way to just forward the bundle2 stream to another user Pierre-Yves David <pierre-yves.david@fb.com> parents: 26541 diff changeset	953 if size == flaginterrupt:
b87e4638dabf bundle2: add a way to just forward the bundle2 stream to another user Pierre-Yves David <pierre-yves.david@fb.com> parents: 26541 diff changeset	954 continue
b87e4638dabf bundle2: add a way to just forward the bundle2 stream to another user Pierre-Yves David <pierre-yves.david@fb.com> parents: 26541 diff changeset	955 elif size < 0:
43077 687b865b95ad formatting: byteify all mercurial/ and hgext/ string literals Augie Fackler <augie@google.com> parents: 43076 diff changeset	956 raise error.BundleValueError(b'negative chunk size: %i')
26542 b87e4638dabf bundle2: add a way to just forward the bundle2 stream to another user Pierre-Yves David <pierre-yves.david@fb.com> parents: 26541 diff changeset	957 yield self._readexact(size)
b87e4638dabf bundle2: add a way to just forward the bundle2 stream to another user Pierre-Yves David <pierre-yves.david@fb.com> parents: 26541 diff changeset	958
35137 da91e7309daf bundle2: don't use seekable bundle2 parts by default (issue5691) Gregory Szorc <gregory.szorc@gmail.com> parents: 35136 diff changeset	959 def iterparts(self, seekable=False):
20802 520df53ad26a bundle2: a very first version of bundle2 unbundler Pierre-Yves David <pierre-yves.david@fb.com> parents: 20801 diff changeset	960 """yield all parts contained in the stream"""
35137 da91e7309daf bundle2: don't use seekable bundle2 parts by default (issue5691) Gregory Szorc <gregory.szorc@gmail.com> parents: 35136 diff changeset	961 cls = seekableunbundlepart if seekable else unbundlepart
20802 520df53ad26a bundle2: a very first version of bundle2 unbundler Pierre-Yves David <pierre-yves.david@fb.com> parents: 20801 diff changeset	962 # make sure param have been loaded
520df53ad26a bundle2: a very first version of bundle2 unbundler Pierre-Yves David <pierre-yves.david@fb.com> parents: 20801 diff changeset	963 self.params
26404 795f02a24b87 bundle2: allow compressed bundle Pierre-Yves David <pierre-yves.david@fb.com> parents: 26396 diff changeset	964 # From there, payload need to be decompressed
30363 d045b4091197 bundle2: use compression engines API to obtain decompressor Gregory Szorc <gregory.szorc@gmail.com> parents: 30361 diff changeset	965 self._fp = self._compengine.decompressorreader(self._fp)
43077 687b865b95ad formatting: byteify all mercurial/ and hgext/ string literals Augie Fackler <augie@google.com> parents: 43076 diff changeset	966 indebug(self.ui, b'start extraction of bundle2 parts')
21014 a6246bba7b9e bundle2: add an unbundle part responsible from unbundling part Pierre-Yves David <pierre-yves.david@fb.com> parents: 21013 diff changeset	967 headerblock = self._readpartheader()
a6246bba7b9e bundle2: add an unbundle part responsible from unbundling part Pierre-Yves David <pierre-yves.david@fb.com> parents: 21013 diff changeset	968 while headerblock is not None:
35137 da91e7309daf bundle2: don't use seekable bundle2 parts by default (issue5691) Gregory Szorc <gregory.szorc@gmail.com> parents: 35136 diff changeset	969 part = cls(self.ui, headerblock, self._fp)
20802 520df53ad26a bundle2: a very first version of bundle2 unbundler Pierre-Yves David <pierre-yves.david@fb.com> parents: 20801 diff changeset	970 yield part
35135 db5038525718 bundle2: implement consume() API on unbundlepart Gregory Szorc <gregory.szorc@gmail.com> parents: 35134 diff changeset	971 # Ensure part is fully consumed so we can start reading the next
db5038525718 bundle2: implement consume() API on unbundlepart Gregory Szorc <gregory.szorc@gmail.com> parents: 35134 diff changeset	972 # part.
db5038525718 bundle2: implement consume() API on unbundlepart Gregory Szorc <gregory.szorc@gmail.com> parents: 35134 diff changeset	973 part.consume()
35136 2b72bc88043f bundle2: only seek to beginning of part in bundlerepo Gregory Szorc <gregory.szorc@gmail.com> parents: 35135 diff changeset	974
21014 a6246bba7b9e bundle2: add an unbundle part responsible from unbundling part Pierre-Yves David <pierre-yves.david@fb.com> parents: 21013 diff changeset	975 headerblock = self._readpartheader()
43077 687b865b95ad formatting: byteify all mercurial/ and hgext/ string literals Augie Fackler <augie@google.com> parents: 43076 diff changeset	976 indebug(self.ui, b'end of bundle2 stream')
20802 520df53ad26a bundle2: a very first version of bundle2 unbundler Pierre-Yves David <pierre-yves.david@fb.com> parents: 20801 diff changeset	977
21014 a6246bba7b9e bundle2: add an unbundle part responsible from unbundling part Pierre-Yves David <pierre-yves.david@fb.com> parents: 21013 diff changeset	978 def _readpartheader(self):
a6246bba7b9e bundle2: add an unbundle part responsible from unbundling part Pierre-Yves David <pierre-yves.david@fb.com> parents: 21013 diff changeset	979 """reads a part header size and return the bytes blob
20864 9a75d2559cff bundle2: support unbundling empty part Pierre-Yves David <pierre-yves.david@fb.com> parents: 20856 diff changeset	980
21014 a6246bba7b9e bundle2: add an unbundle part responsible from unbundling part Pierre-Yves David <pierre-yves.david@fb.com> parents: 21013 diff changeset	981 returns None if empty"""
20864 9a75d2559cff bundle2: support unbundling empty part Pierre-Yves David <pierre-yves.david@fb.com> parents: 20856 diff changeset	982 headersize = self._unpack(_fpartheadersize)[0]
23011 006a81d07e57 bundle2: detect and disallow a negative chunk size Pierre-Yves David <pierre-yves.david@fb.com> parents: 23010 diff changeset	983 if headersize < 0:
43076 2372284d9457 formatting: blacken the codebase Augie Fackler <augie@google.com> parents: 42935 diff changeset	984 raise error.BundleValueError(
43077 687b865b95ad formatting: byteify all mercurial/ and hgext/ string literals Augie Fackler <augie@google.com> parents: 43076 diff changeset	985 b'negative part header size: %i' % headersize
43076 2372284d9457 formatting: blacken the codebase Augie Fackler <augie@google.com> parents: 42935 diff changeset	986 )
43077 687b865b95ad formatting: byteify all mercurial/ and hgext/ string literals Augie Fackler <augie@google.com> parents: 43076 diff changeset	987 indebug(self.ui, b'part header size: %i' % headersize)
21014 a6246bba7b9e bundle2: add an unbundle part responsible from unbundling part Pierre-Yves David <pierre-yves.david@fb.com> parents: 21013 diff changeset	988 if headersize:
a6246bba7b9e bundle2: add an unbundle part responsible from unbundling part Pierre-Yves David <pierre-yves.david@fb.com> parents: 21013 diff changeset	989 return self._readexact(headersize)
a6246bba7b9e bundle2: add an unbundle part responsible from unbundling part Pierre-Yves David <pierre-yves.david@fb.com> parents: 21013 diff changeset	990 return None
20864 9a75d2559cff bundle2: support unbundling empty part Pierre-Yves David <pierre-yves.david@fb.com> parents: 20856 diff changeset	991
24071 184a2f6f40da bundle2.unbundle20: add compressed() method Eric Sumner <ericsumner@fb.com> parents: 24070 diff changeset	992 def compressed(self):
43076 2372284d9457 formatting: blacken the codebase Augie Fackler <augie@google.com> parents: 42935 diff changeset	993 self.params # load params
26802 42f705f2c02d bundle2: make unbundle.compressed return True when compressed Pierre-Yves David <pierre-yves.david@fb.com> parents: 26795 diff changeset	994 return self._compressed
20802 520df53ad26a bundle2: a very first version of bundle2 unbundler Pierre-Yves David <pierre-yves.david@fb.com> parents: 20801 diff changeset	995
31863 cd7aaf344d83 bundle2: move the 'close' method off the unpackermixin Pierre-Yves David <pierre-yves.david@ens-lyon.org> parents: 31862 diff changeset	996 def close(self):
cd7aaf344d83 bundle2: move the 'close' method off the unpackermixin Pierre-Yves David <pierre-yves.david@ens-lyon.org> parents: 31862 diff changeset	997 """close underlying file"""
50951 d718eddf01d9 safehasattr: drop usage in favor of hasattr Pierre-Yves David <pierre-yves.david@octobus.net> parents: 50916 diff changeset	998 if hasattr(self._fp, 'close'):
31863 cd7aaf344d83 bundle2: move the 'close' method off the unpackermixin Pierre-Yves David <pierre-yves.david@ens-lyon.org> parents: 31862 diff changeset	999 return self._fp.close()
cd7aaf344d83 bundle2: move the 'close' method off the unpackermixin Pierre-Yves David <pierre-yves.david@ens-lyon.org> parents: 31862 diff changeset	1000
43076 2372284d9457 formatting: blacken the codebase Augie Fackler <augie@google.com> parents: 42935 diff changeset	1001
43077 687b865b95ad formatting: byteify all mercurial/ and hgext/ string literals Augie Fackler <augie@google.com> parents: 43076 diff changeset	1002 formatmap = {b'20': unbundle20}
24648 5cac3accdaa1 unbundle20: allow generic dispatch between unbundlers Pierre-Yves David <pierre-yves.david@fb.com> parents: 24642 diff changeset	1003
26395 4e7b0bf9f0b1 unbundle20: allow registering handlers for stream level parameters Pierre-Yves David <pierre-yves.david@fb.com> parents: 26393 diff changeset	1004 b2streamparamsmap = {}
4e7b0bf9f0b1 unbundle20: allow registering handlers for stream level parameters Pierre-Yves David <pierre-yves.david@fb.com> parents: 26393 diff changeset	1005
43076 2372284d9457 formatting: blacken the codebase Augie Fackler <augie@google.com> parents: 42935 diff changeset	1006
26395 4e7b0bf9f0b1 unbundle20: allow registering handlers for stream level parameters Pierre-Yves David <pierre-yves.david@fb.com> parents: 26393 diff changeset	1007 def b2streamparamhandler(name):
4e7b0bf9f0b1 unbundle20: allow registering handlers for stream level parameters Pierre-Yves David <pierre-yves.david@fb.com> parents: 26393 diff changeset	1008 """register a handler for a stream level parameter"""
43076 2372284d9457 formatting: blacken the codebase Augie Fackler <augie@google.com> parents: 42935 diff changeset	1009
26395 4e7b0bf9f0b1 unbundle20: allow registering handlers for stream level parameters Pierre-Yves David <pierre-yves.david@fb.com> parents: 26393 diff changeset	1010 def decorator(func):
4e7b0bf9f0b1 unbundle20: allow registering handlers for stream level parameters Pierre-Yves David <pierre-yves.david@fb.com> parents: 26393 diff changeset	1011 assert name not in formatmap
4e7b0bf9f0b1 unbundle20: allow registering handlers for stream level parameters Pierre-Yves David <pierre-yves.david@fb.com> parents: 26393 diff changeset	1012 b2streamparamsmap[name] = func
4e7b0bf9f0b1 unbundle20: allow registering handlers for stream level parameters Pierre-Yves David <pierre-yves.david@fb.com> parents: 26393 diff changeset	1013 return func
43076 2372284d9457 formatting: blacken the codebase Augie Fackler <augie@google.com> parents: 42935 diff changeset	1014
26395 4e7b0bf9f0b1 unbundle20: allow registering handlers for stream level parameters Pierre-Yves David <pierre-yves.david@fb.com> parents: 26393 diff changeset	1015 return decorator
4e7b0bf9f0b1 unbundle20: allow registering handlers for stream level parameters Pierre-Yves David <pierre-yves.david@fb.com> parents: 26393 diff changeset	1016
43076 2372284d9457 formatting: blacken the codebase Augie Fackler <augie@google.com> parents: 42935 diff changeset	1017
43077 687b865b95ad formatting: byteify all mercurial/ and hgext/ string literals Augie Fackler <augie@google.com> parents: 43076 diff changeset	1018 @b2streamparamhandler(b'compression')
26404 795f02a24b87 bundle2: allow compressed bundle Pierre-Yves David <pierre-yves.david@fb.com> parents: 26396 diff changeset	1019 def processcompression(unbundler, param, value):
795f02a24b87 bundle2: allow compressed bundle Pierre-Yves David <pierre-yves.david@fb.com> parents: 26396 diff changeset	1020 """read compression parameter and install payload decompression"""
30363 d045b4091197 bundle2: use compression engines API to obtain decompressor Gregory Szorc <gregory.szorc@gmail.com> parents: 30361 diff changeset	1021 if value not in util.compengines.supportedbundletypes:
43076 2372284d9457 formatting: blacken the codebase Augie Fackler <augie@google.com> parents: 42935 diff changeset	1022 raise error.BundleUnknownFeatureError(params=(param,), values=(value,))
30363 d045b4091197 bundle2: use compression engines API to obtain decompressor Gregory Szorc <gregory.szorc@gmail.com> parents: 30361 diff changeset	1023 unbundler._compengine = util.compengines.forbundletype(value)
26802 42f705f2c02d bundle2: make unbundle.compressed return True when compressed Pierre-Yves David <pierre-yves.david@fb.com> parents: 26795 diff changeset	1024 if value is not None:
42f705f2c02d bundle2: make unbundle.compressed return True when compressed Pierre-Yves David <pierre-yves.david@fb.com> parents: 26795 diff changeset	1025 unbundler._compressed = True
26404 795f02a24b87 bundle2: allow compressed bundle Pierre-Yves David <pierre-yves.david@fb.com> parents: 26396 diff changeset	1026
43076 2372284d9457 formatting: blacken the codebase Augie Fackler <augie@google.com> parents: 42935 diff changeset	1027
49037 642e31cb55f0 py3: use class X: instead of class X(object): Gregory Szorc <gregory.szorc@gmail.com> parents: 49004 diff changeset	1028 class bundlepart:
20856 8a6a86c9a5b5 bundle2: support bundling of empty part (with a type) Pierre-Yves David <pierre-yves.david@fb.com> parents: 20844 diff changeset	1029 """A bundle2 part contains application level payload
8a6a86c9a5b5 bundle2: support bundling of empty part (with a type) Pierre-Yves David <pierre-yves.david@fb.com> parents: 20844 diff changeset	1030
8a6a86c9a5b5 bundle2: support bundling of empty part (with a type) Pierre-Yves David <pierre-yves.david@fb.com> parents: 20844 diff changeset	1031 The part `type` is used to route the part to the application level
8a6a86c9a5b5 bundle2: support bundling of empty part (with a type) Pierre-Yves David <pierre-yves.david@fb.com> parents: 20844 diff changeset	1032 handler.
21604 c399bf961cb9 bundle2: the ability to set ``data`` attribute of the part is now official Pierre-Yves David <pierre-yves.david@fb.com> parents: 21603 diff changeset	1033
c399bf961cb9 bundle2: the ability to set ``data`` attribute of the part is now official Pierre-Yves David <pierre-yves.david@fb.com> parents: 21603 diff changeset	1034 The part payload is contained in ``part.data``. It could be raw bytes or a
21605 f9dabfceb259 bundle2: introduce a ``addparam`` method on part Pierre-Yves David <pierre-yves.david@fb.com> parents: 21604 diff changeset	1035 generator of byte chunks.
f9dabfceb259 bundle2: introduce a ``addparam`` method on part Pierre-Yves David <pierre-yves.david@fb.com> parents: 21604 diff changeset	1036
f9dabfceb259 bundle2: introduce a ``addparam`` method on part Pierre-Yves David <pierre-yves.david@fb.com> parents: 21604 diff changeset	1037 You can add parameters to the part using the ``addparam`` method.
f9dabfceb259 bundle2: introduce a ``addparam`` method on part Pierre-Yves David <pierre-yves.david@fb.com> parents: 21604 diff changeset	1038 Parameters can be either mandatory (default) or advisory. Remote side
f9dabfceb259 bundle2: introduce a ``addparam`` method on part Pierre-Yves David <pierre-yves.david@fb.com> parents: 21604 diff changeset	1039 should be able to safely ignore the advisory ones.
f9dabfceb259 bundle2: introduce a ``addparam`` method on part Pierre-Yves David <pierre-yves.david@fb.com> parents: 21604 diff changeset	1040
f9dabfceb259 bundle2: introduce a ``addparam`` method on part Pierre-Yves David <pierre-yves.david@fb.com> parents: 21604 diff changeset	1041 Both data and parameters cannot be modified after the generation has begun.
20856 8a6a86c9a5b5 bundle2: support bundling of empty part (with a type) Pierre-Yves David <pierre-yves.david@fb.com> parents: 20844 diff changeset	1042 """
8a6a86c9a5b5 bundle2: support bundling of empty part (with a type) Pierre-Yves David <pierre-yves.david@fb.com> parents: 20844 diff changeset	1043
43076 2372284d9457 formatting: blacken the codebase Augie Fackler <augie@google.com> parents: 42935 diff changeset	1044 def __init__(
2372284d9457 formatting: blacken the codebase Augie Fackler <augie@google.com> parents: 42935 diff changeset	1045 self,
2372284d9457 formatting: blacken the codebase Augie Fackler <augie@google.com> parents: 42935 diff changeset	1046 parttype,
2372284d9457 formatting: blacken the codebase Augie Fackler <augie@google.com> parents: 42935 diff changeset	1047 mandatoryparams=(),
2372284d9457 formatting: blacken the codebase Augie Fackler <augie@google.com> parents: 42935 diff changeset	1048 advisoryparams=(),
43077 687b865b95ad formatting: byteify all mercurial/ and hgext/ string literals Augie Fackler <augie@google.com> parents: 43076 diff changeset	1049 data=b'',
43076 2372284d9457 formatting: blacken the codebase Augie Fackler <augie@google.com> parents: 42935 diff changeset	1050 mandatory=True,
2372284d9457 formatting: blacken the codebase Augie Fackler <augie@google.com> parents: 42935 diff changeset	1051 ):
23868 405eacbecc52 bundle2: enforce parttype as alphanumerical Pierre-Yves David <pierre-yves.david@fb.com> parents: 23590 diff changeset	1052 validateparttype(parttype)
20995 e995d104c87f bundle2: add an integer id to part Pierre-Yves David <pierre-yves.david@fb.com> parents: 20952 diff changeset	1053 self.id = None
20856 8a6a86c9a5b5 bundle2: support bundling of empty part (with a type) Pierre-Yves David <pierre-yves.david@fb.com> parents: 20844 diff changeset	1054 self.type = parttype
21604 c399bf961cb9 bundle2: the ability to set ``data`` attribute of the part is now official Pierre-Yves David <pierre-yves.david@fb.com> parents: 21603 diff changeset	1055 self._data = data
21605 f9dabfceb259 bundle2: introduce a ``addparam`` method on part Pierre-Yves David <pierre-yves.david@fb.com> parents: 21604 diff changeset	1056 self._mandatoryparams = list(mandatoryparams)
f9dabfceb259 bundle2: introduce a ``addparam`` method on part Pierre-Yves David <pierre-yves.david@fb.com> parents: 21604 diff changeset	1057 self._advisoryparams = list(advisoryparams)
21607 054fa5176fa7 bundle2: forbid duplicate parameter keys Pierre-Yves David <pierre-yves.david@fb.com> parents: 21606 diff changeset	1058 # checking for duplicated entries
054fa5176fa7 bundle2: forbid duplicate parameter keys Pierre-Yves David <pierre-yves.david@fb.com> parents: 21606 diff changeset	1059 self._seenparams = set()
054fa5176fa7 bundle2: forbid duplicate parameter keys Pierre-Yves David <pierre-yves.david@fb.com> parents: 21606 diff changeset	1060 for pname, __ in self._mandatoryparams + self._advisoryparams:
054fa5176fa7 bundle2: forbid duplicate parameter keys Pierre-Yves David <pierre-yves.david@fb.com> parents: 21606 diff changeset	1061 if pname in self._seenparams:
43077 687b865b95ad formatting: byteify all mercurial/ and hgext/ string literals Augie Fackler <augie@google.com> parents: 43076 diff changeset	1062 raise error.ProgrammingError(b'duplicated params: %s' % pname)
21607 054fa5176fa7 bundle2: forbid duplicate parameter keys Pierre-Yves David <pierre-yves.david@fb.com> parents: 21606 diff changeset	1063 self._seenparams.add(pname)
21601 7ff01befc7ec bundle2: track life cycle of parts Pierre-Yves David <pierre-yves.david@fb.com> parents: 21600 diff changeset	1064 # status of the part's generation:
7ff01befc7ec bundle2: track life cycle of parts Pierre-Yves David <pierre-yves.david@fb.com> parents: 21600 diff changeset	1065 # - None: not started,
7ff01befc7ec bundle2: track life cycle of parts Pierre-Yves David <pierre-yves.david@fb.com> parents: 21600 diff changeset	1066 # - False: currently generated,
7ff01befc7ec bundle2: track life cycle of parts Pierre-Yves David <pierre-yves.david@fb.com> parents: 21600 diff changeset	1067 # - True: generation done.
7ff01befc7ec bundle2: track life cycle of parts Pierre-Yves David <pierre-yves.david@fb.com> parents: 21600 diff changeset	1068 self._generated = None
23590 4440c7cc3728 bundle2.bundlepart: make mandatory part flag explicit in API Eric Sumner <ericsumner@fb.com> parents: 23586 diff changeset	1069 self.mandatory = mandatory
20856 8a6a86c9a5b5 bundle2: support bundling of empty part (with a type) Pierre-Yves David <pierre-yves.david@fb.com> parents: 20844 diff changeset	1070
30906 1f51b4658f21 bundle2: implement a basic __repr__ for bundle2 part Pierre-Yves David <pierre-yves.david@ens-lyon.org> parents: 30905 diff changeset	1071 def __repr__(self):
44173 74172a234dd3 py3: fully fix bundlepart.__repr__ to return str not bytes Kyle Lippincott <spectral@google.com> parents: 44164 diff changeset	1072 cls = "%s.%s" % (self.__class__.__module__, self.__class__.__name__)
74172a234dd3 py3: fully fix bundlepart.__repr__ to return str not bytes Kyle Lippincott <spectral@google.com> parents: 44164 diff changeset	1073 return '<%s object at %x; id: %s; type: %s; mandatory: %s>' % (
43076 2372284d9457 formatting: blacken the codebase Augie Fackler <augie@google.com> parents: 42935 diff changeset	1074 cls,
2372284d9457 formatting: blacken the codebase Augie Fackler <augie@google.com> parents: 42935 diff changeset	1075 id(self),
2372284d9457 formatting: blacken the codebase Augie Fackler <augie@google.com> parents: 42935 diff changeset	1076 self.id,
2372284d9457 formatting: blacken the codebase Augie Fackler <augie@google.com> parents: 42935 diff changeset	1077 self.type,
2372284d9457 formatting: blacken the codebase Augie Fackler <augie@google.com> parents: 42935 diff changeset	1078 self.mandatory,
2372284d9457 formatting: blacken the codebase Augie Fackler <augie@google.com> parents: 42935 diff changeset	1079 )
30906 1f51b4658f21 bundle2: implement a basic __repr__ for bundle2 part Pierre-Yves David <pierre-yves.david@ens-lyon.org> parents: 30905 diff changeset	1080
24793 2ec894582ea2 bundle2: add a 'copy' method on parts Pierre-Yves David <pierre-yves.david@fb.com> parents: 24743 diff changeset	1081 def copy(self):
2ec894582ea2 bundle2: add a 'copy' method on parts Pierre-Yves David <pierre-yves.david@fb.com> parents: 24743 diff changeset	1082 """return a copy of the part
2ec894582ea2 bundle2: add a 'copy' method on parts Pierre-Yves David <pierre-yves.david@fb.com> parents: 24743 diff changeset	1083
2ec894582ea2 bundle2: add a 'copy' method on parts Pierre-Yves David <pierre-yves.david@fb.com> parents: 24743 diff changeset	1084 The new part have the very same content but no partid assigned yet.
2ec894582ea2 bundle2: add a 'copy' method on parts Pierre-Yves David <pierre-yves.david@fb.com> parents: 24743 diff changeset	1085 Parts with generated data cannot be copied."""
50951 d718eddf01d9 safehasattr: drop usage in favor of hasattr Pierre-Yves David <pierre-yves.david@octobus.net> parents: 50916 diff changeset	1086 assert not hasattr(self.data, 'next')
43076 2372284d9457 formatting: blacken the codebase Augie Fackler <augie@google.com> parents: 42935 diff changeset	1087 return self.__class__(
2372284d9457 formatting: blacken the codebase Augie Fackler <augie@google.com> parents: 42935 diff changeset	1088 self.type,
2372284d9457 formatting: blacken the codebase Augie Fackler <augie@google.com> parents: 42935 diff changeset	1089 self._mandatoryparams,
2372284d9457 formatting: blacken the codebase Augie Fackler <augie@google.com> parents: 42935 diff changeset	1090 self._advisoryparams,
2372284d9457 formatting: blacken the codebase Augie Fackler <augie@google.com> parents: 42935 diff changeset	1091 self._data,
2372284d9457 formatting: blacken the codebase Augie Fackler <augie@google.com> parents: 42935 diff changeset	1092 self.mandatory,
2372284d9457 formatting: blacken the codebase Augie Fackler <augie@google.com> parents: 42935 diff changeset	1093 )
24793 2ec894582ea2 bundle2: add a 'copy' method on parts Pierre-Yves David <pierre-yves.david@fb.com> parents: 24743 diff changeset	1094
21604 c399bf961cb9 bundle2: the ability to set ``data`` attribute of the part is now official Pierre-Yves David <pierre-yves.david@fb.com> parents: 21603 diff changeset	1095 # methods used to defines the part content
27879 52a4ad62b006 cleanup: use modern @property/@foo.setter property specification Augie Fackler <augie@google.com> parents: 27751 diff changeset	1096 @property
52a4ad62b006 cleanup: use modern @property/@foo.setter property specification Augie Fackler <augie@google.com> parents: 27751 diff changeset	1097 def data(self):
52a4ad62b006 cleanup: use modern @property/@foo.setter property specification Augie Fackler <augie@google.com> parents: 27751 diff changeset	1098 return self._data
52a4ad62b006 cleanup: use modern @property/@foo.setter property specification Augie Fackler <augie@google.com> parents: 27751 diff changeset	1099
52a4ad62b006 cleanup: use modern @property/@foo.setter property specification Augie Fackler <augie@google.com> parents: 27751 diff changeset	1100 @data.setter
52a4ad62b006 cleanup: use modern @property/@foo.setter property specification Augie Fackler <augie@google.com> parents: 27751 diff changeset	1101 def data(self, data):
21604 c399bf961cb9 bundle2: the ability to set ``data`` attribute of the part is now official Pierre-Yves David <pierre-yves.david@fb.com> parents: 21603 diff changeset	1102 if self._generated is not None:
43077 687b865b95ad formatting: byteify all mercurial/ and hgext/ string literals Augie Fackler <augie@google.com> parents: 43076 diff changeset	1103 raise error.ReadOnlyPartError(b'part is being generated')
21604 c399bf961cb9 bundle2: the ability to set ``data`` attribute of the part is now official Pierre-Yves David <pierre-yves.david@fb.com> parents: 21603 diff changeset	1104 self._data = data
c399bf961cb9 bundle2: the ability to set ``data`` attribute of the part is now official Pierre-Yves David <pierre-yves.david@fb.com> parents: 21603 diff changeset	1105
21605 f9dabfceb259 bundle2: introduce a ``addparam`` method on part Pierre-Yves David <pierre-yves.david@fb.com> parents: 21604 diff changeset	1106 @property
f9dabfceb259 bundle2: introduce a ``addparam`` method on part Pierre-Yves David <pierre-yves.david@fb.com> parents: 21604 diff changeset	1107 def mandatoryparams(self):
f9dabfceb259 bundle2: introduce a ``addparam`` method on part Pierre-Yves David <pierre-yves.david@fb.com> parents: 21604 diff changeset	1108 # make it an immutable tuple to force people through ``addparam``
f9dabfceb259 bundle2: introduce a ``addparam`` method on part Pierre-Yves David <pierre-yves.david@fb.com> parents: 21604 diff changeset	1109 return tuple(self._mandatoryparams)
f9dabfceb259 bundle2: introduce a ``addparam`` method on part Pierre-Yves David <pierre-yves.david@fb.com> parents: 21604 diff changeset	1110
f9dabfceb259 bundle2: introduce a ``addparam`` method on part Pierre-Yves David <pierre-yves.david@fb.com> parents: 21604 diff changeset	1111 @property
f9dabfceb259 bundle2: introduce a ``addparam`` method on part Pierre-Yves David <pierre-yves.david@fb.com> parents: 21604 diff changeset	1112 def advisoryparams(self):
f9dabfceb259 bundle2: introduce a ``addparam`` method on part Pierre-Yves David <pierre-yves.david@fb.com> parents: 21604 diff changeset	1113 # make it an immutable tuple to force people through ``addparam``
f9dabfceb259 bundle2: introduce a ``addparam`` method on part Pierre-Yves David <pierre-yves.david@fb.com> parents: 21604 diff changeset	1114 return tuple(self._advisoryparams)
f9dabfceb259 bundle2: introduce a ``addparam`` method on part Pierre-Yves David <pierre-yves.david@fb.com> parents: 21604 diff changeset	1115
43077 687b865b95ad formatting: byteify all mercurial/ and hgext/ string literals Augie Fackler <augie@google.com> parents: 43076 diff changeset	1116 def addparam(self, name, value=b'', mandatory=True):
31861 6d055cd6669f bundle2: add documention to 'part.addparams' Pierre-Yves David <pierre-yves.david@ens-lyon.org> parents: 31652 diff changeset	1117 """add a parameter to the part
6d055cd6669f bundle2: add documention to 'part.addparams' Pierre-Yves David <pierre-yves.david@ens-lyon.org> parents: 31652 diff changeset	1118
6d055cd6669f bundle2: add documention to 'part.addparams' Pierre-Yves David <pierre-yves.david@ens-lyon.org> parents: 31652 diff changeset	1119 If 'mandatory' is set to True, the remote handler must claim support
6d055cd6669f bundle2: add documention to 'part.addparams' Pierre-Yves David <pierre-yves.david@ens-lyon.org> parents: 31652 diff changeset	1120 for this parameter or the unbundling will be aborted.
6d055cd6669f bundle2: add documention to 'part.addparams' Pierre-Yves David <pierre-yves.david@ens-lyon.org> parents: 31652 diff changeset	1121
6d055cd6669f bundle2: add documention to 'part.addparams' Pierre-Yves David <pierre-yves.david@ens-lyon.org> parents: 31652 diff changeset	1122 The 'name' and 'value' cannot exceed 255 bytes each.
6d055cd6669f bundle2: add documention to 'part.addparams' Pierre-Yves David <pierre-yves.david@ens-lyon.org> parents: 31652 diff changeset	1123 """
21605 f9dabfceb259 bundle2: introduce a ``addparam`` method on part Pierre-Yves David <pierre-yves.david@fb.com> parents: 21604 diff changeset	1124 if self._generated is not None:
43077 687b865b95ad formatting: byteify all mercurial/ and hgext/ string literals Augie Fackler <augie@google.com> parents: 43076 diff changeset	1125 raise error.ReadOnlyPartError(b'part is being generated')
21607 054fa5176fa7 bundle2: forbid duplicate parameter keys Pierre-Yves David <pierre-yves.david@fb.com> parents: 21606 diff changeset	1126 if name in self._seenparams:
43077 687b865b95ad formatting: byteify all mercurial/ and hgext/ string literals Augie Fackler <augie@google.com> parents: 43076 diff changeset	1127 raise ValueError(b'duplicated params: %s' % name)
21607 054fa5176fa7 bundle2: forbid duplicate parameter keys Pierre-Yves David <pierre-yves.david@fb.com> parents: 21606 diff changeset	1128 self._seenparams.add(name)
21605 f9dabfceb259 bundle2: introduce a ``addparam`` method on part Pierre-Yves David <pierre-yves.david@fb.com> parents: 21604 diff changeset	1129 params = self._advisoryparams
f9dabfceb259 bundle2: introduce a ``addparam`` method on part Pierre-Yves David <pierre-yves.david@fb.com> parents: 21604 diff changeset	1130 if mandatory:
f9dabfceb259 bundle2: introduce a ``addparam`` method on part Pierre-Yves David <pierre-yves.david@fb.com> parents: 21604 diff changeset	1131 params = self._mandatoryparams
f9dabfceb259 bundle2: introduce a ``addparam`` method on part Pierre-Yves David <pierre-yves.david@fb.com> parents: 21604 diff changeset	1132 params.append((name, value))
f9dabfceb259 bundle2: introduce a ``addparam`` method on part Pierre-Yves David <pierre-yves.david@fb.com> parents: 21604 diff changeset	1133
21601 7ff01befc7ec bundle2: track life cycle of parts Pierre-Yves David <pierre-yves.david@fb.com> parents: 21600 diff changeset	1134 # methods used to generates the bundle2 stream
25321 b44ee346211f bundle2: add debug output for part generation Pierre-Yves David <pierre-yves.david@fb.com> parents: 25320 diff changeset	1135 def getchunks(self, ui):
21601 7ff01befc7ec bundle2: track life cycle of parts Pierre-Yves David <pierre-yves.david@fb.com> parents: 21600 diff changeset	1136 if self._generated is not None:
43077 687b865b95ad formatting: byteify all mercurial/ and hgext/ string literals Augie Fackler <augie@google.com> parents: 43076 diff changeset	1137 raise error.ProgrammingError(b'part can only be consumed once')
21601 7ff01befc7ec bundle2: track life cycle of parts Pierre-Yves David <pierre-yves.david@fb.com> parents: 21600 diff changeset	1138 self._generated = False
25323 21a25fb81d2c bundle2: add generic debug output regarding generated parts Pierre-Yves David <pierre-yves.david@fb.com> parents: 25322 diff changeset	1139
21a25fb81d2c bundle2: add generic debug output regarding generated parts Pierre-Yves David <pierre-yves.david@fb.com> parents: 25322 diff changeset	1140 if ui.debugflag:
43077 687b865b95ad formatting: byteify all mercurial/ and hgext/ string literals Augie Fackler <augie@google.com> parents: 43076 diff changeset	1141 msg = [b'bundle2-output-part: "%s"' % self.type]
25323 21a25fb81d2c bundle2: add generic debug output regarding generated parts Pierre-Yves David <pierre-yves.david@fb.com> parents: 25322 diff changeset	1142 if not self.mandatory:
43077 687b865b95ad formatting: byteify all mercurial/ and hgext/ string literals Augie Fackler <augie@google.com> parents: 43076 diff changeset	1143 msg.append(b' (advisory)')
25323 21a25fb81d2c bundle2: add generic debug output regarding generated parts Pierre-Yves David <pierre-yves.david@fb.com> parents: 25322 diff changeset	1144 nbmp = len(self.mandatoryparams)
21a25fb81d2c bundle2: add generic debug output regarding generated parts Pierre-Yves David <pierre-yves.david@fb.com> parents: 25322 diff changeset	1145 nbap = len(self.advisoryparams)
21a25fb81d2c bundle2: add generic debug output regarding generated parts Pierre-Yves David <pierre-yves.david@fb.com> parents: 25322 diff changeset	1146 if nbmp or nbap:
43077 687b865b95ad formatting: byteify all mercurial/ and hgext/ string literals Augie Fackler <augie@google.com> parents: 43076 diff changeset	1147 msg.append(b' (params:')
25323 21a25fb81d2c bundle2: add generic debug output regarding generated parts Pierre-Yves David <pierre-yves.david@fb.com> parents: 25322 diff changeset	1148 if nbmp:
43077 687b865b95ad formatting: byteify all mercurial/ and hgext/ string literals Augie Fackler <augie@google.com> parents: 43076 diff changeset	1149 msg.append(b' %i mandatory' % nbmp)
25323 21a25fb81d2c bundle2: add generic debug output regarding generated parts Pierre-Yves David <pierre-yves.david@fb.com> parents: 25322 diff changeset	1150 if nbap:
43077 687b865b95ad formatting: byteify all mercurial/ and hgext/ string literals Augie Fackler <augie@google.com> parents: 43076 diff changeset	1151 msg.append(b' %i advisory' % nbmp)
687b865b95ad formatting: byteify all mercurial/ and hgext/ string literals Augie Fackler <augie@google.com> parents: 43076 diff changeset	1152 msg.append(b')')
25323 21a25fb81d2c bundle2: add generic debug output regarding generated parts Pierre-Yves David <pierre-yves.david@fb.com> parents: 25322 diff changeset	1153 if not self.data:
43077 687b865b95ad formatting: byteify all mercurial/ and hgext/ string literals Augie Fackler <augie@google.com> parents: 43076 diff changeset	1154 msg.append(b' empty payload')
50951 d718eddf01d9 safehasattr: drop usage in favor of hasattr Pierre-Yves David <pierre-yves.david@octobus.net> parents: 50916 diff changeset	1155 elif hasattr(self.data, 'next') or hasattr(self.data, '__next__'):
43077 687b865b95ad formatting: byteify all mercurial/ and hgext/ string literals Augie Fackler <augie@google.com> parents: 43076 diff changeset	1156 msg.append(b' streamed payload')
25323 21a25fb81d2c bundle2: add generic debug output regarding generated parts Pierre-Yves David <pierre-yves.david@fb.com> parents: 25322 diff changeset	1157 else:
43077 687b865b95ad formatting: byteify all mercurial/ and hgext/ string literals Augie Fackler <augie@google.com> parents: 43076 diff changeset	1158 msg.append(b' %i bytes payload' % len(self.data))
687b865b95ad formatting: byteify all mercurial/ and hgext/ string literals Augie Fackler <augie@google.com> parents: 43076 diff changeset	1159 msg.append(b'\n')
687b865b95ad formatting: byteify all mercurial/ and hgext/ string literals Augie Fackler <augie@google.com> parents: 43076 diff changeset	1160 ui.debug(b''.join(msg))
25323 21a25fb81d2c bundle2: add generic debug output regarding generated parts Pierre-Yves David <pierre-yves.david@fb.com> parents: 25322 diff changeset	1161
20877 9e9e3a4e9261 bundle2: part params Pierre-Yves David <pierre-yves.david@fb.com> parents: 20876 diff changeset	1162 #### header
23590 4440c7cc3728 bundle2.bundlepart: make mandatory part flag explicit in API Eric Sumner <ericsumner@fb.com> parents: 23586 diff changeset	1163 if self.mandatory:
4440c7cc3728 bundle2.bundlepart: make mandatory part flag explicit in API Eric Sumner <ericsumner@fb.com> parents: 23586 diff changeset	1164 parttype = self.type.upper()
4440c7cc3728 bundle2.bundlepart: make mandatory part flag explicit in API Eric Sumner <ericsumner@fb.com> parents: 23586 diff changeset	1165 else:
4440c7cc3728 bundle2.bundlepart: make mandatory part flag explicit in API Eric Sumner <ericsumner@fb.com> parents: 23586 diff changeset	1166 parttype = self.type.lower()
43077 687b865b95ad formatting: byteify all mercurial/ and hgext/ string literals Augie Fackler <augie@google.com> parents: 43076 diff changeset	1167 outdebug(ui, b'part %s: "%s"' % (pycompat.bytestr(self.id), parttype))
20877 9e9e3a4e9261 bundle2: part params Pierre-Yves David <pierre-yves.david@fb.com> parents: 20876 diff changeset	1168 ## parttype
43076 2372284d9457 formatting: blacken the codebase Augie Fackler <augie@google.com> parents: 42935 diff changeset	1169 header = [
2372284d9457 formatting: blacken the codebase Augie Fackler <augie@google.com> parents: 42935 diff changeset	1170 _pack(_fparttypesize, len(parttype)),
2372284d9457 formatting: blacken the codebase Augie Fackler <augie@google.com> parents: 42935 diff changeset	1171 parttype,
2372284d9457 formatting: blacken the codebase Augie Fackler <augie@google.com> parents: 42935 diff changeset	1172 _pack(_fpartid, self.id),
2372284d9457 formatting: blacken the codebase Augie Fackler <augie@google.com> parents: 42935 diff changeset	1173 ]
20877 9e9e3a4e9261 bundle2: part params Pierre-Yves David <pierre-yves.david@fb.com> parents: 20876 diff changeset	1174 ## parameters
9e9e3a4e9261 bundle2: part params Pierre-Yves David <pierre-yves.david@fb.com> parents: 20876 diff changeset	1175 # count
9e9e3a4e9261 bundle2: part params Pierre-Yves David <pierre-yves.david@fb.com> parents: 20876 diff changeset	1176 manpar = self.mandatoryparams
9e9e3a4e9261 bundle2: part params Pierre-Yves David <pierre-yves.david@fb.com> parents: 20876 diff changeset	1177 advpar = self.advisoryparams
9e9e3a4e9261 bundle2: part params Pierre-Yves David <pierre-yves.david@fb.com> parents: 20876 diff changeset	1178 header.append(_pack(_fpartparamcount, len(manpar), len(advpar)))
9e9e3a4e9261 bundle2: part params Pierre-Yves David <pierre-yves.david@fb.com> parents: 20876 diff changeset	1179 # size
9e9e3a4e9261 bundle2: part params Pierre-Yves David <pierre-yves.david@fb.com> parents: 20876 diff changeset	1180 parsizes = []
9e9e3a4e9261 bundle2: part params Pierre-Yves David <pierre-yves.david@fb.com> parents: 20876 diff changeset	1181 for key, value in manpar:
9e9e3a4e9261 bundle2: part params Pierre-Yves David <pierre-yves.david@fb.com> parents: 20876 diff changeset	1182 parsizes.append(len(key))
9e9e3a4e9261 bundle2: part params Pierre-Yves David <pierre-yves.david@fb.com> parents: 20876 diff changeset	1183 parsizes.append(len(value))
9e9e3a4e9261 bundle2: part params Pierre-Yves David <pierre-yves.david@fb.com> parents: 20876 diff changeset	1184 for key, value in advpar:
9e9e3a4e9261 bundle2: part params Pierre-Yves David <pierre-yves.david@fb.com> parents: 20876 diff changeset	1185 parsizes.append(len(key))
9e9e3a4e9261 bundle2: part params Pierre-Yves David <pierre-yves.david@fb.com> parents: 20876 diff changeset	1186 parsizes.append(len(value))
33698 da7c285ec6da bundle2: use modern Python division Augie Fackler <augie@google.com> parents: 33693 diff changeset	1187 paramsizes = _pack(_makefpartparamsizes(len(parsizes) // 2), *parsizes)
20877 9e9e3a4e9261 bundle2: part params Pierre-Yves David <pierre-yves.david@fb.com> parents: 20876 diff changeset	1188 header.append(paramsizes)
9e9e3a4e9261 bundle2: part params Pierre-Yves David <pierre-yves.david@fb.com> parents: 20876 diff changeset	1189 # key, value
9e9e3a4e9261 bundle2: part params Pierre-Yves David <pierre-yves.david@fb.com> parents: 20876 diff changeset	1190 for key, value in manpar:
9e9e3a4e9261 bundle2: part params Pierre-Yves David <pierre-yves.david@fb.com> parents: 20876 diff changeset	1191 header.append(key)
9e9e3a4e9261 bundle2: part params Pierre-Yves David <pierre-yves.david@fb.com> parents: 20876 diff changeset	1192 header.append(value)
9e9e3a4e9261 bundle2: part params Pierre-Yves David <pierre-yves.david@fb.com> parents: 20876 diff changeset	1193 for key, value in advpar:
9e9e3a4e9261 bundle2: part params Pierre-Yves David <pierre-yves.david@fb.com> parents: 20876 diff changeset	1194 header.append(key)
9e9e3a4e9261 bundle2: part params Pierre-Yves David <pierre-yves.david@fb.com> parents: 20876 diff changeset	1195 header.append(value)
9e9e3a4e9261 bundle2: part params Pierre-Yves David <pierre-yves.david@fb.com> parents: 20876 diff changeset	1196 ## finalize header
34256 ab379eed2e31 bundle2: raise a more helpful error if building a bundle part header fails Augie Fackler <raf@durin42.com> parents: 34228 diff changeset	1197 try:
43077 687b865b95ad formatting: byteify all mercurial/ and hgext/ string literals Augie Fackler <augie@google.com> parents: 43076 diff changeset	1198 headerchunk = b''.join(header)
34256 ab379eed2e31 bundle2: raise a more helpful error if building a bundle part header fails Augie Fackler <raf@durin42.com> parents: 34228 diff changeset	1199 except TypeError:
43076 2372284d9457 formatting: blacken the codebase Augie Fackler <augie@google.com> parents: 42935 diff changeset	1200 raise TypeError(
43554 9f70512ae2cf cleanup: remove pointless r-prefixes on single-quoted strings Augie Fackler <augie@google.com> parents: 43543 diff changeset	1201 'Found a non-bytes trying to '
9f70512ae2cf cleanup: remove pointless r-prefixes on single-quoted strings Augie Fackler <augie@google.com> parents: 43543 diff changeset	1202 'build bundle part header: %r' % header
43076 2372284d9457 formatting: blacken the codebase Augie Fackler <augie@google.com> parents: 42935 diff changeset	1203 )
43077 687b865b95ad formatting: byteify all mercurial/ and hgext/ string literals Augie Fackler <augie@google.com> parents: 43076 diff changeset	1204 outdebug(ui, b'header chunk size: %i' % len(headerchunk))
20856 8a6a86c9a5b5 bundle2: support bundling of empty part (with a type) Pierre-Yves David <pierre-yves.david@fb.com> parents: 20844 diff changeset	1205 yield _pack(_fpartheadersize, len(headerchunk))
8a6a86c9a5b5 bundle2: support bundling of empty part (with a type) Pierre-Yves David <pierre-yves.david@fb.com> parents: 20844 diff changeset	1206 yield headerchunk
20877 9e9e3a4e9261 bundle2: part params Pierre-Yves David <pierre-yves.david@fb.com> parents: 20876 diff changeset	1207 ## payload
23067 420a051616ce bundle2: transmit exception during part generation Pierre-Yves David <pierre-yves.david@fb.com> parents: 23066 diff changeset	1208 try:
420a051616ce bundle2: transmit exception during part generation Pierre-Yves David <pierre-yves.david@fb.com> parents: 23066 diff changeset	1209 for chunk in self._payloadchunks():
43077 687b865b95ad formatting: byteify all mercurial/ and hgext/ string literals Augie Fackler <augie@google.com> parents: 43076 diff changeset	1210 outdebug(ui, b'payload chunk size: %i' % len(chunk))
23067 420a051616ce bundle2: transmit exception during part generation Pierre-Yves David <pierre-yves.david@fb.com> parents: 23066 diff changeset	1211 yield _pack(_fpayloadsize, len(chunk))
420a051616ce bundle2: transmit exception during part generation Pierre-Yves David <pierre-yves.david@fb.com> parents: 23066 diff changeset	1212 yield chunk
26144 4bc3707f3e67 bundle2: don't try to recover from a GeneratorExit (issue4785) Augie Fackler <augie@google.com> parents: 25919 diff changeset	1213 except GeneratorExit:
4bc3707f3e67 bundle2: don't try to recover from a GeneratorExit (issue4785) Augie Fackler <augie@google.com> parents: 25919 diff changeset	1214 # GeneratorExit means that nobody is listening for our
4bc3707f3e67 bundle2: don't try to recover from a GeneratorExit (issue4785) Augie Fackler <augie@google.com> parents: 25919 diff changeset	1215 # results anyway, so just bail quickly rather than trying
4bc3707f3e67 bundle2: don't try to recover from a GeneratorExit (issue4785) Augie Fackler <augie@google.com> parents: 25919 diff changeset	1216 # to produce an error part.
43077 687b865b95ad formatting: byteify all mercurial/ and hgext/ string literals Augie Fackler <augie@google.com> parents: 43076 diff changeset	1217 ui.debug(b'bundle2-generatorexit\n')
26144 4bc3707f3e67 bundle2: don't try to recover from a GeneratorExit (issue4785) Augie Fackler <augie@google.com> parents: 25919 diff changeset	1218 raise
25660 328739ea70c3 global: mass rewrite to use modern exception syntax Gregory Szorc <gregory.szorc@gmail.com> parents: 25641 diff changeset	1219 except BaseException as exc:
37087 f0b6fbea00cf stringutil: bulk-replace call sites to point to new module Yuya Nishihara <yuya@tcha.org> parents: 37008 diff changeset	1220 bexc = stringutil.forcebytestr(exc)
23067 420a051616ce bundle2: transmit exception during part generation Pierre-Yves David <pierre-yves.david@fb.com> parents: 23066 diff changeset	1221 # backup exception data for later
43076 2372284d9457 formatting: blacken the codebase Augie Fackler <augie@google.com> parents: 42935 diff changeset	1222 ui.debug(
43077 687b865b95ad formatting: byteify all mercurial/ and hgext/ string literals Augie Fackler <augie@google.com> parents: 43076 diff changeset	1223 b'bundle2-input-stream-interrupt: encoding exception %s' % bexc
43076 2372284d9457 formatting: blacken the codebase Augie Fackler <augie@google.com> parents: 42935 diff changeset	1224 )
32232 76f9a0009b4b pycompat: extract helper to raise exception with traceback Yuya Nishihara <yuya@tcha.org> parents: 32024 diff changeset	1225 tb = sys.exc_info()[2]
43077 687b865b95ad formatting: byteify all mercurial/ and hgext/ string literals Augie Fackler <augie@google.com> parents: 43076 diff changeset	1226 msg = b'unexpected error: %s' % bexc
43076 2372284d9457 formatting: blacken the codebase Augie Fackler <augie@google.com> parents: 42935 diff changeset	1227 interpart = bundlepart(
43077 687b865b95ad formatting: byteify all mercurial/ and hgext/ string literals Augie Fackler <augie@google.com> parents: 43076 diff changeset	1228 b'error:abort', [(b'message', msg)], mandatory=False
43076 2372284d9457 formatting: blacken the codebase Augie Fackler <augie@google.com> parents: 42935 diff changeset	1229 )
23067 420a051616ce bundle2: transmit exception during part generation Pierre-Yves David <pierre-yves.david@fb.com> parents: 23066 diff changeset	1230 interpart.id = 0
420a051616ce bundle2: transmit exception during part generation Pierre-Yves David <pierre-yves.david@fb.com> parents: 23066 diff changeset	1231 yield _pack(_fpayloadsize, -1)
25321 b44ee346211f bundle2: add debug output for part generation Pierre-Yves David <pierre-yves.david@fb.com> parents: 25320 diff changeset	1232 for chunk in interpart.getchunks(ui=ui):
23067 420a051616ce bundle2: transmit exception during part generation Pierre-Yves David <pierre-yves.david@fb.com> parents: 23066 diff changeset	1233 yield chunk
43077 687b865b95ad formatting: byteify all mercurial/ and hgext/ string literals Augie Fackler <augie@google.com> parents: 43076 diff changeset	1234 outdebug(ui, b'closing payload chunk')
23067 420a051616ce bundle2: transmit exception during part generation Pierre-Yves David <pierre-yves.david@fb.com> parents: 23066 diff changeset	1235 # abort current part payload
420a051616ce bundle2: transmit exception during part generation Pierre-Yves David <pierre-yves.david@fb.com> parents: 23066 diff changeset	1236 yield _pack(_fpayloadsize, 0)
32232 76f9a0009b4b pycompat: extract helper to raise exception with traceback Yuya Nishihara <yuya@tcha.org> parents: 32024 diff changeset	1237 pycompat.raisewithtb(exc, tb)
21000 4cae06ae1562 bundle2: extract a _payloadchunks method for part Pierre-Yves David <pierre-yves.david@fb.com> parents: 20998 diff changeset	1238 # end of payload
43077 687b865b95ad formatting: byteify all mercurial/ and hgext/ string literals Augie Fackler <augie@google.com> parents: 43076 diff changeset	1239 outdebug(ui, b'closing payload chunk')
21000 4cae06ae1562 bundle2: extract a _payloadchunks method for part Pierre-Yves David <pierre-yves.david@fb.com> parents: 20998 diff changeset	1240 yield _pack(_fpayloadsize, 0)
21601 7ff01befc7ec bundle2: track life cycle of parts Pierre-Yves David <pierre-yves.david@fb.com> parents: 21600 diff changeset	1241 self._generated = True
21000 4cae06ae1562 bundle2: extract a _payloadchunks method for part Pierre-Yves David <pierre-yves.david@fb.com> parents: 20998 diff changeset	1242
4cae06ae1562 bundle2: extract a _payloadchunks method for part Pierre-Yves David <pierre-yves.david@fb.com> parents: 20998 diff changeset	1243 def _payloadchunks(self):
4cae06ae1562 bundle2: extract a _payloadchunks method for part Pierre-Yves David <pierre-yves.david@fb.com> parents: 20998 diff changeset	1244 """yield chunks of a the part payload
4cae06ae1562 bundle2: extract a _payloadchunks method for part Pierre-Yves David <pierre-yves.david@fb.com> parents: 20998 diff changeset	1245
4cae06ae1562 bundle2: extract a _payloadchunks method for part Pierre-Yves David <pierre-yves.david@fb.com> parents: 20998 diff changeset	1246 Exists to handle the different methods to provide data to a part."""
20876 ddd56f3eb786 bundle2: support for bundling and unbundling payload Pierre-Yves David <pierre-yves.david@fb.com> parents: 20864 diff changeset	1247 # we only support fixed size data now.
ddd56f3eb786 bundle2: support for bundling and unbundling payload Pierre-Yves David <pierre-yves.david@fb.com> parents: 20864 diff changeset	1248 # This will be improved in the future.
50951 d718eddf01d9 safehasattr: drop usage in favor of hasattr Pierre-Yves David <pierre-yves.david@octobus.net> parents: 50916 diff changeset	1249 if hasattr(self.data, 'next') or hasattr(self.data, '__next__'):
21001 c93bb6a08fa1 bundle2: support chunk iterator as part data Pierre-Yves David <pierre-yves.david@fb.com> parents: 21000 diff changeset	1250 buff = util.chunkbuffer(self.data)
c93bb6a08fa1 bundle2: support chunk iterator as part data Pierre-Yves David <pierre-yves.david@fb.com> parents: 21000 diff changeset	1251 chunk = buff.read(preferedchunksize)
c93bb6a08fa1 bundle2: support chunk iterator as part data Pierre-Yves David <pierre-yves.david@fb.com> parents: 21000 diff changeset	1252 while chunk:
c93bb6a08fa1 bundle2: support chunk iterator as part data Pierre-Yves David <pierre-yves.david@fb.com> parents: 21000 diff changeset	1253 yield chunk
c93bb6a08fa1 bundle2: support chunk iterator as part data Pierre-Yves David <pierre-yves.david@fb.com> parents: 21000 diff changeset	1254 chunk = buff.read(preferedchunksize)
c93bb6a08fa1 bundle2: support chunk iterator as part data Pierre-Yves David <pierre-yves.david@fb.com> parents: 21000 diff changeset	1255 elif len(self.data):
20876 ddd56f3eb786 bundle2: support for bundling and unbundling payload Pierre-Yves David <pierre-yves.david@fb.com> parents: 20864 diff changeset	1256 yield self.data
20802 520df53ad26a bundle2: a very first version of bundle2 unbundler Pierre-Yves David <pierre-yves.david@fb.com> parents: 20801 diff changeset	1257
23066 ad144882318d bundle2: add a interrupt mechanism Pierre-Yves David <pierre-yves.david@fb.com> parents: 23029 diff changeset	1258
ad144882318d bundle2: add a interrupt mechanism Pierre-Yves David <pierre-yves.david@fb.com> parents: 23029 diff changeset	1259 flaginterrupt = -1
ad144882318d bundle2: add a interrupt mechanism Pierre-Yves David <pierre-yves.david@fb.com> parents: 23029 diff changeset	1260
43076 2372284d9457 formatting: blacken the codebase Augie Fackler <augie@google.com> parents: 42935 diff changeset	1261
23066 ad144882318d bundle2: add a interrupt mechanism Pierre-Yves David <pierre-yves.david@fb.com> parents: 23029 diff changeset	1262 class interrupthandler(unpackermixin):
ad144882318d bundle2: add a interrupt mechanism Pierre-Yves David <pierre-yves.david@fb.com> parents: 23029 diff changeset	1263 """read one part and process it with restricted capability
ad144882318d bundle2: add a interrupt mechanism Pierre-Yves David <pierre-yves.david@fb.com> parents: 23029 diff changeset	1264
ad144882318d bundle2: add a interrupt mechanism Pierre-Yves David <pierre-yves.david@fb.com> parents: 23029 diff changeset	1265 This allows to transmit exception raised on the producer size during part
ad144882318d bundle2: add a interrupt mechanism Pierre-Yves David <pierre-yves.david@fb.com> parents: 23029 diff changeset	1266 iteration while the consumer is reading a part.
ad144882318d bundle2: add a interrupt mechanism Pierre-Yves David <pierre-yves.david@fb.com> parents: 23029 diff changeset	1267
ad144882318d bundle2: add a interrupt mechanism Pierre-Yves David <pierre-yves.david@fb.com> parents: 23029 diff changeset	1268 Part processed in this manner only have access to a ui object,"""
ad144882318d bundle2: add a interrupt mechanism Pierre-Yves David <pierre-yves.david@fb.com> parents: 23029 diff changeset	1269
ad144882318d bundle2: add a interrupt mechanism Pierre-Yves David <pierre-yves.david@fb.com> parents: 23029 diff changeset	1270 def __init__(self, ui, fp):
52668 5cc8deb96b48 pyupgrade: modernize calls to superclass methods Matt Harbison <matt_harbison@yahoo.com> parents: 52665 diff changeset	1271 super().__init__(fp)
23066 ad144882318d bundle2: add a interrupt mechanism Pierre-Yves David <pierre-yves.david@fb.com> parents: 23029 diff changeset	1272 self.ui = ui
ad144882318d bundle2: add a interrupt mechanism Pierre-Yves David <pierre-yves.david@fb.com> parents: 23029 diff changeset	1273
ad144882318d bundle2: add a interrupt mechanism Pierre-Yves David <pierre-yves.david@fb.com> parents: 23029 diff changeset	1274 def _readpartheader(self):
ad144882318d bundle2: add a interrupt mechanism Pierre-Yves David <pierre-yves.david@fb.com> parents: 23029 diff changeset	1275 """reads a part header size and return the bytes blob
ad144882318d bundle2: add a interrupt mechanism Pierre-Yves David <pierre-yves.david@fb.com> parents: 23029 diff changeset	1276
ad144882318d bundle2: add a interrupt mechanism Pierre-Yves David <pierre-yves.david@fb.com> parents: 23029 diff changeset	1277 returns None if empty"""
ad144882318d bundle2: add a interrupt mechanism Pierre-Yves David <pierre-yves.david@fb.com> parents: 23029 diff changeset	1278 headersize = self._unpack(_fpartheadersize)[0]
ad144882318d bundle2: add a interrupt mechanism Pierre-Yves David <pierre-yves.david@fb.com> parents: 23029 diff changeset	1279 if headersize < 0:
43076 2372284d9457 formatting: blacken the codebase Augie Fackler <augie@google.com> parents: 42935 diff changeset	1280 raise error.BundleValueError(
43077 687b865b95ad formatting: byteify all mercurial/ and hgext/ string literals Augie Fackler <augie@google.com> parents: 43076 diff changeset	1281 b'negative part header size: %i' % headersize
43076 2372284d9457 formatting: blacken the codebase Augie Fackler <augie@google.com> parents: 42935 diff changeset	1282 )
43077 687b865b95ad formatting: byteify all mercurial/ and hgext/ string literals Augie Fackler <augie@google.com> parents: 43076 diff changeset	1283 indebug(self.ui, b'part header size: %i\n' % headersize)
23066 ad144882318d bundle2: add a interrupt mechanism Pierre-Yves David <pierre-yves.david@fb.com> parents: 23029 diff changeset	1284 if headersize:
ad144882318d bundle2: add a interrupt mechanism Pierre-Yves David <pierre-yves.david@fb.com> parents: 23029 diff changeset	1285 return self._readexact(headersize)
ad144882318d bundle2: add a interrupt mechanism Pierre-Yves David <pierre-yves.david@fb.com> parents: 23029 diff changeset	1286 return None
ad144882318d bundle2: add a interrupt mechanism Pierre-Yves David <pierre-yves.david@fb.com> parents: 23029 diff changeset	1287
ad144882318d bundle2: add a interrupt mechanism Pierre-Yves David <pierre-yves.david@fb.com> parents: 23029 diff changeset	1288 def __call__(self):
43076 2372284d9457 formatting: blacken the codebase Augie Fackler <augie@google.com> parents: 42935 diff changeset	1289 self.ui.debug(
43117 8ff1ecfadcd1 cleanup: join string literals that are already on one line Martin von Zweigbergk <martinvonz@google.com> parents: 43115 diff changeset	1290 b'bundle2-input-stream-interrupt: opening out of band context\n'
43076 2372284d9457 formatting: blacken the codebase Augie Fackler <augie@google.com> parents: 42935 diff changeset	1291 )
43077 687b865b95ad formatting: byteify all mercurial/ and hgext/ string literals Augie Fackler <augie@google.com> parents: 43076 diff changeset	1292 indebug(self.ui, b'bundle2 stream interruption, looking for a part.')
23066 ad144882318d bundle2: add a interrupt mechanism Pierre-Yves David <pierre-yves.david@fb.com> parents: 23029 diff changeset	1293 headerblock = self._readpartheader()
ad144882318d bundle2: add a interrupt mechanism Pierre-Yves David <pierre-yves.david@fb.com> parents: 23029 diff changeset	1294 if headerblock is None:
43077 687b865b95ad formatting: byteify all mercurial/ and hgext/ string literals Augie Fackler <augie@google.com> parents: 43076 diff changeset	1295 indebug(self.ui, b'no part found during interruption.')
23066 ad144882318d bundle2: add a interrupt mechanism Pierre-Yves David <pierre-yves.david@fb.com> parents: 23029 diff changeset	1296 return
35137 da91e7309daf bundle2: don't use seekable bundle2 parts by default (issue5691) Gregory Szorc <gregory.szorc@gmail.com> parents: 35136 diff changeset	1297 part = unbundlepart(self.ui, headerblock, self._fp)
23066 ad144882318d bundle2: add a interrupt mechanism Pierre-Yves David <pierre-yves.david@fb.com> parents: 23029 diff changeset	1298 op = interruptoperation(self.ui)
34265 e71890f27767 bundle2: move processpart stream maintenance into part iterator Durham Goode <durham@fb.com> parents: 34256 diff changeset	1299 hardabort = False
e71890f27767 bundle2: move processpart stream maintenance into part iterator Durham Goode <durham@fb.com> parents: 34256 diff changeset	1300 try:
e71890f27767 bundle2: move processpart stream maintenance into part iterator Durham Goode <durham@fb.com> parents: 34256 diff changeset	1301 _processpart(op, part)
e71890f27767 bundle2: move processpart stream maintenance into part iterator Durham Goode <durham@fb.com> parents: 34256 diff changeset	1302 except (SystemExit, KeyboardInterrupt):
e71890f27767 bundle2: move processpart stream maintenance into part iterator Durham Goode <durham@fb.com> parents: 34256 diff changeset	1303 hardabort = True
e71890f27767 bundle2: move processpart stream maintenance into part iterator Durham Goode <durham@fb.com> parents: 34256 diff changeset	1304 raise
e71890f27767 bundle2: move processpart stream maintenance into part iterator Durham Goode <durham@fb.com> parents: 34256 diff changeset	1305 finally:
e71890f27767 bundle2: move processpart stream maintenance into part iterator Durham Goode <durham@fb.com> parents: 34256 diff changeset	1306 if not hardabort:
35135 db5038525718 bundle2: implement consume() API on unbundlepart Gregory Szorc <gregory.szorc@gmail.com> parents: 35134 diff changeset	1307 part.consume()
43076 2372284d9457 formatting: blacken the codebase Augie Fackler <augie@google.com> parents: 42935 diff changeset	1308 self.ui.debug(
43117 8ff1ecfadcd1 cleanup: join string literals that are already on one line Martin von Zweigbergk <martinvonz@google.com> parents: 43115 diff changeset	1309 b'bundle2-input-stream-interrupt: closing out of band context\n'
43076 2372284d9457 formatting: blacken the codebase Augie Fackler <augie@google.com> parents: 42935 diff changeset	1310 )
2372284d9457 formatting: blacken the codebase Augie Fackler <augie@google.com> parents: 42935 diff changeset	1311
23066 ad144882318d bundle2: add a interrupt mechanism Pierre-Yves David <pierre-yves.david@fb.com> parents: 23029 diff changeset	1312
49037 642e31cb55f0 py3: use class X: instead of class X(object): Gregory Szorc <gregory.szorc@gmail.com> parents: 49004 diff changeset	1313 class interruptoperation:
23066 ad144882318d bundle2: add a interrupt mechanism Pierre-Yves David <pierre-yves.david@fb.com> parents: 23029 diff changeset	1314 """A limited operation to be use by part handler during interruption
ad144882318d bundle2: add a interrupt mechanism Pierre-Yves David <pierre-yves.david@fb.com> parents: 23029 diff changeset	1315
ad144882318d bundle2: add a interrupt mechanism Pierre-Yves David <pierre-yves.david@fb.com> parents: 23029 diff changeset	1316 It only have access to an ui object.
ad144882318d bundle2: add a interrupt mechanism Pierre-Yves David <pierre-yves.david@fb.com> parents: 23029 diff changeset	1317 """
ad144882318d bundle2: add a interrupt mechanism Pierre-Yves David <pierre-yves.david@fb.com> parents: 23029 diff changeset	1318
ad144882318d bundle2: add a interrupt mechanism Pierre-Yves David <pierre-yves.david@fb.com> parents: 23029 diff changeset	1319 def __init__(self, ui):
ad144882318d bundle2: add a interrupt mechanism Pierre-Yves David <pierre-yves.david@fb.com> parents: 23029 diff changeset	1320 self.ui = ui
ad144882318d bundle2: add a interrupt mechanism Pierre-Yves David <pierre-yves.david@fb.com> parents: 23029 diff changeset	1321 self.reply = None
24878 e530cde6d115 bundle2: disable ouput capture unless we use http (issue4613 issue4615) Pierre-Yves David <pierre-yves.david@fb.com> parents: 24851 diff changeset	1322 self.captureoutput = False
23066 ad144882318d bundle2: add a interrupt mechanism Pierre-Yves David <pierre-yves.david@fb.com> parents: 23029 diff changeset	1323
ad144882318d bundle2: add a interrupt mechanism Pierre-Yves David <pierre-yves.david@fb.com> parents: 23029 diff changeset	1324 @property
ad144882318d bundle2: add a interrupt mechanism Pierre-Yves David <pierre-yves.david@fb.com> parents: 23029 diff changeset	1325 def repo(self):
43077 687b865b95ad formatting: byteify all mercurial/ and hgext/ string literals Augie Fackler <augie@google.com> parents: 43076 diff changeset	1326 raise error.ProgrammingError(b'no repo access from stream interruption')
23066 ad144882318d bundle2: add a interrupt mechanism Pierre-Yves David <pierre-yves.david@fb.com> parents: 23029 diff changeset	1327
ad144882318d bundle2: add a interrupt mechanism Pierre-Yves David <pierre-yves.david@fb.com> parents: 23029 diff changeset	1328 def gettransaction(self):
43077 687b865b95ad formatting: byteify all mercurial/ and hgext/ string literals Augie Fackler <augie@google.com> parents: 43076 diff changeset	1329 raise TransactionUnavailable(b'no repo access from stream interruption')
23066 ad144882318d bundle2: add a interrupt mechanism Pierre-Yves David <pierre-yves.david@fb.com> parents: 23029 diff changeset	1330
43076 2372284d9457 formatting: blacken the codebase Augie Fackler <augie@google.com> parents: 42935 diff changeset	1331
35134 8aa43ff9c12c bundle2: implement generic part payload decoder Gregory Szorc <gregory.szorc@gmail.com> parents: 35133 diff changeset	1332 def decodepayloadchunks(ui, fh):
8aa43ff9c12c bundle2: implement generic part payload decoder Gregory Szorc <gregory.szorc@gmail.com> parents: 35133 diff changeset	1333 """Reads bundle2 part payload data into chunks.
8aa43ff9c12c bundle2: implement generic part payload decoder Gregory Szorc <gregory.szorc@gmail.com> parents: 35133 diff changeset	1334
8aa43ff9c12c bundle2: implement generic part payload decoder Gregory Szorc <gregory.szorc@gmail.com> parents: 35133 diff changeset	1335 Part payload data consists of framed chunks. This function takes
8aa43ff9c12c bundle2: implement generic part payload decoder Gregory Szorc <gregory.szorc@gmail.com> parents: 35133 diff changeset	1336 a file handle and emits those chunks.
8aa43ff9c12c bundle2: implement generic part payload decoder Gregory Szorc <gregory.szorc@gmail.com> parents: 35133 diff changeset	1337 """
43077 687b865b95ad formatting: byteify all mercurial/ and hgext/ string literals Augie Fackler <augie@google.com> parents: 43076 diff changeset	1338 dolog = ui.configbool(b'devel', b'bundle2.debug')
35138 589eed45a452 bundle2: inline debug logging Gregory Szorc <gregory.szorc@gmail.com> parents: 35137 diff changeset	1339 debug = ui.debug
589eed45a452 bundle2: inline debug logging Gregory Szorc <gregory.szorc@gmail.com> parents: 35137 diff changeset	1340
35140 764e3ad1cf54 bundle2: inline struct operations Gregory Szorc <gregory.szorc@gmail.com> parents: 35139 diff changeset	1341 headerstruct = struct.Struct(_fpayloadsize)
764e3ad1cf54 bundle2: inline struct operations Gregory Szorc <gregory.szorc@gmail.com> parents: 35139 diff changeset	1342 headersize = headerstruct.size
764e3ad1cf54 bundle2: inline struct operations Gregory Szorc <gregory.szorc@gmail.com> parents: 35139 diff changeset	1343 unpack = headerstruct.unpack
764e3ad1cf54 bundle2: inline struct operations Gregory Szorc <gregory.szorc@gmail.com> parents: 35139 diff changeset	1344
35134 8aa43ff9c12c bundle2: implement generic part payload decoder Gregory Szorc <gregory.szorc@gmail.com> parents: 35133 diff changeset	1345 readexactly = changegroup.readexactly
35139 1fb0846ad792 bundle2: inline changegroup.readexactly() Gregory Szorc <gregory.szorc@gmail.com> parents: 35138 diff changeset	1346 read = fh.read
35134 8aa43ff9c12c bundle2: implement generic part payload decoder Gregory Szorc <gregory.szorc@gmail.com> parents: 35133 diff changeset	1347
35140 764e3ad1cf54 bundle2: inline struct operations Gregory Szorc <gregory.szorc@gmail.com> parents: 35139 diff changeset	1348 chunksize = unpack(readexactly(fh, headersize))[0]
43077 687b865b95ad formatting: byteify all mercurial/ and hgext/ string literals Augie Fackler <augie@google.com> parents: 43076 diff changeset	1349 indebug(ui, b'payload chunk size: %i' % chunksize)
35134 8aa43ff9c12c bundle2: implement generic part payload decoder Gregory Szorc <gregory.szorc@gmail.com> parents: 35133 diff changeset	1350
35139 1fb0846ad792 bundle2: inline changegroup.readexactly() Gregory Szorc <gregory.szorc@gmail.com> parents: 35138 diff changeset	1351 # changegroup.readexactly() is inlined below for performance.
35134 8aa43ff9c12c bundle2: implement generic part payload decoder Gregory Szorc <gregory.szorc@gmail.com> parents: 35133 diff changeset	1352 while chunksize:
8aa43ff9c12c bundle2: implement generic part payload decoder Gregory Szorc <gregory.szorc@gmail.com> parents: 35133 diff changeset	1353 if chunksize >= 0:
35139 1fb0846ad792 bundle2: inline changegroup.readexactly() Gregory Szorc <gregory.szorc@gmail.com> parents: 35138 diff changeset	1354 s = read(chunksize)
1fb0846ad792 bundle2: inline changegroup.readexactly() Gregory Szorc <gregory.szorc@gmail.com> parents: 35138 diff changeset	1355 if len(s) < chunksize:
43076 2372284d9457 formatting: blacken the codebase Augie Fackler <augie@google.com> parents: 42935 diff changeset	1356 raise error.Abort(
2372284d9457 formatting: blacken the codebase Augie Fackler <augie@google.com> parents: 42935 diff changeset	1357 _(
43077 687b865b95ad formatting: byteify all mercurial/ and hgext/ string literals Augie Fackler <augie@google.com> parents: 43076 diff changeset	1358 b'stream ended unexpectedly '
687b865b95ad formatting: byteify all mercurial/ and hgext/ string literals Augie Fackler <augie@google.com> parents: 43076 diff changeset	1359 b' (got %d bytes, expected %d)'
43076 2372284d9457 formatting: blacken the codebase Augie Fackler <augie@google.com> parents: 42935 diff changeset	1360 )
2372284d9457 formatting: blacken the codebase Augie Fackler <augie@google.com> parents: 42935 diff changeset	1361 % (len(s), chunksize)
2372284d9457 formatting: blacken the codebase Augie Fackler <augie@google.com> parents: 42935 diff changeset	1362 )
35139 1fb0846ad792 bundle2: inline changegroup.readexactly() Gregory Szorc <gregory.szorc@gmail.com> parents: 35138 diff changeset	1363
1fb0846ad792 bundle2: inline changegroup.readexactly() Gregory Szorc <gregory.szorc@gmail.com> parents: 35138 diff changeset	1364 yield s
35134 8aa43ff9c12c bundle2: implement generic part payload decoder Gregory Szorc <gregory.szorc@gmail.com> parents: 35133 diff changeset	1365 elif chunksize == flaginterrupt:
8aa43ff9c12c bundle2: implement generic part payload decoder Gregory Szorc <gregory.szorc@gmail.com> parents: 35133 diff changeset	1366 # Interrupt "signal" detected. The regular stream is interrupted
8aa43ff9c12c bundle2: implement generic part payload decoder Gregory Szorc <gregory.szorc@gmail.com> parents: 35133 diff changeset	1367 # and a bundle2 part follows. Consume it.
8aa43ff9c12c bundle2: implement generic part payload decoder Gregory Szorc <gregory.szorc@gmail.com> parents: 35133 diff changeset	1368 interrupthandler(ui, fh)()
8aa43ff9c12c bundle2: implement generic part payload decoder Gregory Szorc <gregory.szorc@gmail.com> parents: 35133 diff changeset	1369 else:
8aa43ff9c12c bundle2: implement generic part payload decoder Gregory Szorc <gregory.szorc@gmail.com> parents: 35133 diff changeset	1370 raise error.BundleValueError(
43077 687b865b95ad formatting: byteify all mercurial/ and hgext/ string literals Augie Fackler <augie@google.com> parents: 43076 diff changeset	1371 b'negative payload chunk size: %s' % chunksize
43076 2372284d9457 formatting: blacken the codebase Augie Fackler <augie@google.com> parents: 42935 diff changeset	1372 )
35134 8aa43ff9c12c bundle2: implement generic part payload decoder Gregory Szorc <gregory.szorc@gmail.com> parents: 35133 diff changeset	1373
35139 1fb0846ad792 bundle2: inline changegroup.readexactly() Gregory Szorc <gregory.szorc@gmail.com> parents: 35138 diff changeset	1374 s = read(headersize)
1fb0846ad792 bundle2: inline changegroup.readexactly() Gregory Szorc <gregory.szorc@gmail.com> parents: 35138 diff changeset	1375 if len(s) < headersize:
43076 2372284d9457 formatting: blacken the codebase Augie Fackler <augie@google.com> parents: 42935 diff changeset	1376 raise error.Abort(
43117 8ff1ecfadcd1 cleanup: join string literals that are already on one line Martin von Zweigbergk <martinvonz@google.com> parents: 43115 diff changeset	1377 _(b'stream ended unexpectedly (got %d bytes, expected %d)')
43076 2372284d9457 formatting: blacken the codebase Augie Fackler <augie@google.com> parents: 42935 diff changeset	1378 % (len(s), chunksize)
2372284d9457 formatting: blacken the codebase Augie Fackler <augie@google.com> parents: 42935 diff changeset	1379 )
35139 1fb0846ad792 bundle2: inline changegroup.readexactly() Gregory Szorc <gregory.szorc@gmail.com> parents: 35138 diff changeset	1380
35140 764e3ad1cf54 bundle2: inline struct operations Gregory Szorc <gregory.szorc@gmail.com> parents: 35139 diff changeset	1381 chunksize = unpack(s)[0]
35138 589eed45a452 bundle2: inline debug logging Gregory Szorc <gregory.szorc@gmail.com> parents: 35137 diff changeset	1382
589eed45a452 bundle2: inline debug logging Gregory Szorc <gregory.szorc@gmail.com> parents: 35137 diff changeset	1383 # indebug() inlined for performance.
589eed45a452 bundle2: inline debug logging Gregory Szorc <gregory.szorc@gmail.com> parents: 35137 diff changeset	1384 if dolog:
43077 687b865b95ad formatting: byteify all mercurial/ and hgext/ string literals Augie Fackler <augie@google.com> parents: 43076 diff changeset	1385 debug(b'bundle2-input: payload chunk size: %i\n' % chunksize)
35134 8aa43ff9c12c bundle2: implement generic part payload decoder Gregory Szorc <gregory.szorc@gmail.com> parents: 35133 diff changeset	1386
43076 2372284d9457 formatting: blacken the codebase Augie Fackler <augie@google.com> parents: 42935 diff changeset	1387
21014 a6246bba7b9e bundle2: add an unbundle part responsible from unbundling part Pierre-Yves David <pierre-yves.david@fb.com> parents: 21013 diff changeset	1388 class unbundlepart(unpackermixin):
a6246bba7b9e bundle2: add an unbundle part responsible from unbundling part Pierre-Yves David <pierre-yves.david@fb.com> parents: 21013 diff changeset	1389 """a bundle part read from a bundle"""
a6246bba7b9e bundle2: add an unbundle part responsible from unbundling part Pierre-Yves David <pierre-yves.david@fb.com> parents: 21013 diff changeset	1390
a6246bba7b9e bundle2: add an unbundle part responsible from unbundling part Pierre-Yves David <pierre-yves.david@fb.com> parents: 21013 diff changeset	1391 def __init__(self, ui, header, fp):
52668 5cc8deb96b48 pyupgrade: modernize calls to superclass methods Matt Harbison <matt_harbison@yahoo.com> parents: 52665 diff changeset	1392 super().__init__(fp)
50951 d718eddf01d9 safehasattr: drop usage in favor of hasattr Pierre-Yves David <pierre-yves.david@octobus.net> parents: 50916 diff changeset	1393 self._seekable = hasattr(fp, 'seek') and hasattr(fp, 'tell')
21014 a6246bba7b9e bundle2: add an unbundle part responsible from unbundling part Pierre-Yves David <pierre-yves.david@fb.com> parents: 21013 diff changeset	1394 self.ui = ui
a6246bba7b9e bundle2: add an unbundle part responsible from unbundling part Pierre-Yves David <pierre-yves.david@fb.com> parents: 21013 diff changeset	1395 # unbundle state attr
a6246bba7b9e bundle2: add an unbundle part responsible from unbundling part Pierre-Yves David <pierre-yves.david@fb.com> parents: 21013 diff changeset	1396 self._headerdata = header
21015 14dd49260246 bundle2: move the fromheader closure into the class itself Pierre-Yves David <pierre-yves.david@fb.com> parents: 21014 diff changeset	1397 self._headeroffset = 0
21019 3dc09f831a2e bundle2: lazy unbundle of part payload Pierre-Yves David <pierre-yves.david@fb.com> parents: 21016 diff changeset	1398 self._initialized = False
3dc09f831a2e bundle2: lazy unbundle of part payload Pierre-Yves David <pierre-yves.david@fb.com> parents: 21016 diff changeset	1399 self.consumed = False
21014 a6246bba7b9e bundle2: add an unbundle part responsible from unbundling part Pierre-Yves David <pierre-yves.david@fb.com> parents: 21013 diff changeset	1400 # part data
a6246bba7b9e bundle2: add an unbundle part responsible from unbundling part Pierre-Yves David <pierre-yves.david@fb.com> parents: 21013 diff changeset	1401 self.id = None
a6246bba7b9e bundle2: add an unbundle part responsible from unbundling part Pierre-Yves David <pierre-yves.david@fb.com> parents: 21013 diff changeset	1402 self.type = None
a6246bba7b9e bundle2: add an unbundle part responsible from unbundling part Pierre-Yves David <pierre-yves.david@fb.com> parents: 21013 diff changeset	1403 self.mandatoryparams = None
a6246bba7b9e bundle2: add an unbundle part responsible from unbundling part Pierre-Yves David <pierre-yves.david@fb.com> parents: 21013 diff changeset	1404 self.advisoryparams = None
21610 d6056805f8f4 bundle2: introduce a ``params`` dictionary on unbundled parts Pierre-Yves David <pierre-yves.david@fb.com> parents: 21609 diff changeset	1405 self.params = None
21612 f221eb0531d9 bundle2: expose mandatory params in a mandatorykeys attribute Pierre-Yves David <pierre-yves.david@fb.com> parents: 21611 diff changeset	1406 self.mandatorykeys = ()
21019 3dc09f831a2e bundle2: lazy unbundle of part payload Pierre-Yves David <pierre-yves.david@fb.com> parents: 21016 diff changeset	1407 self._readheader()
23585 94b25d71dd0f bundle2.unbundlepart: decouple mandatory from parttype Eric Sumner <ericsumner@fb.com> parents: 23439 diff changeset	1408 self._mandatory = None
24036 c7601086338a bundle2.unbundlepart: tell() implementation Eric Sumner <ericsumner@fb.com> parents: 24035 diff changeset	1409 self._pos = 0
21014 a6246bba7b9e bundle2: add an unbundle part responsible from unbundling part Pierre-Yves David <pierre-yves.david@fb.com> parents: 21013 diff changeset	1410
21015 14dd49260246 bundle2: move the fromheader closure into the class itself Pierre-Yves David <pierre-yves.david@fb.com> parents: 21014 diff changeset	1411 def _fromheader(self, size):
14dd49260246 bundle2: move the fromheader closure into the class itself Pierre-Yves David <pierre-yves.david@fb.com> parents: 21014 diff changeset	1412 """return the next <size> byte from the header"""
14dd49260246 bundle2: move the fromheader closure into the class itself Pierre-Yves David <pierre-yves.david@fb.com> parents: 21014 diff changeset	1413 offset = self._headeroffset
43076 2372284d9457 formatting: blacken the codebase Augie Fackler <augie@google.com> parents: 42935 diff changeset	1414 data = self._headerdata[offset : (offset + size)]
21019 3dc09f831a2e bundle2: lazy unbundle of part payload Pierre-Yves David <pierre-yves.david@fb.com> parents: 21016 diff changeset	1415 self._headeroffset = offset + size
21015 14dd49260246 bundle2: move the fromheader closure into the class itself Pierre-Yves David <pierre-yves.david@fb.com> parents: 21014 diff changeset	1416 return data
14dd49260246 bundle2: move the fromheader closure into the class itself Pierre-Yves David <pierre-yves.david@fb.com> parents: 21014 diff changeset	1417
21016 b477afb1c81e bundle2: move unpackheader closure into the class Pierre-Yves David <pierre-yves.david@fb.com> parents: 21015 diff changeset	1418 def _unpackheader(self, format):
b477afb1c81e bundle2: move unpackheader closure into the class Pierre-Yves David <pierre-yves.david@fb.com> parents: 21015 diff changeset	1419 """read given format from header
b477afb1c81e bundle2: move unpackheader closure into the class Pierre-Yves David <pierre-yves.david@fb.com> parents: 21015 diff changeset	1420
b477afb1c81e bundle2: move unpackheader closure into the class Pierre-Yves David <pierre-yves.david@fb.com> parents: 21015 diff changeset	1421 This automatically compute the size of the format to read."""
b477afb1c81e bundle2: move unpackheader closure into the class Pierre-Yves David <pierre-yves.david@fb.com> parents: 21015 diff changeset	1422 data = self._fromheader(struct.calcsize(format))
b477afb1c81e bundle2: move unpackheader closure into the class Pierre-Yves David <pierre-yves.david@fb.com> parents: 21015 diff changeset	1423 return _unpack(format, data)
b477afb1c81e bundle2: move unpackheader closure into the class Pierre-Yves David <pierre-yves.david@fb.com> parents: 21015 diff changeset	1424
21608 3cb96ca90c17 bundle2: introduce an ``_initparams`` method Pierre-Yves David <pierre-yves.david@fb.com> parents: 21607 diff changeset	1425 def _initparams(self, mandatoryparams, advisoryparams):
3cb96ca90c17 bundle2: introduce an ``_initparams`` method Pierre-Yves David <pierre-yves.david@fb.com> parents: 21607 diff changeset	1426 """internal function to setup all logic related parameters"""
21609 63cc2594ef8a bundle2: make sure unbundled part param are read-only Pierre-Yves David <pierre-yves.david@fb.com> parents: 21608 diff changeset	1427 # make it read only to prevent people touching it by mistake.
63cc2594ef8a bundle2: make sure unbundled part param are read-only Pierre-Yves David <pierre-yves.david@fb.com> parents: 21608 diff changeset	1428 self.mandatoryparams = tuple(mandatoryparams)
43076 2372284d9457 formatting: blacken the codebase Augie Fackler <augie@google.com> parents: 42935 diff changeset	1429 self.advisoryparams = tuple(advisoryparams)
21610 d6056805f8f4 bundle2: introduce a ``params`` dictionary on unbundled parts Pierre-Yves David <pierre-yves.david@fb.com> parents: 21609 diff changeset	1430 # user friendly UI
29591 6215b5537ba5 bundle2: use a sorted dict for holding parameters Gregory Szorc <gregory.szorc@gmail.com> parents: 29294 diff changeset	1431 self.params = util.sortdict(self.mandatoryparams)
6215b5537ba5 bundle2: use a sorted dict for holding parameters Gregory Szorc <gregory.szorc@gmail.com> parents: 29294 diff changeset	1432 self.params.update(self.advisoryparams)
21612 f221eb0531d9 bundle2: expose mandatory params in a mandatorykeys attribute Pierre-Yves David <pierre-yves.david@fb.com> parents: 21611 diff changeset	1433 self.mandatorykeys = frozenset(p[0] for p in mandatoryparams)
21608 3cb96ca90c17 bundle2: introduce an ``_initparams`` method Pierre-Yves David <pierre-yves.david@fb.com> parents: 21607 diff changeset	1434
21019 3dc09f831a2e bundle2: lazy unbundle of part payload Pierre-Yves David <pierre-yves.david@fb.com> parents: 21016 diff changeset	1435 def _readheader(self):
21014 a6246bba7b9e bundle2: add an unbundle part responsible from unbundling part Pierre-Yves David <pierre-yves.david@fb.com> parents: 21013 diff changeset	1436 """read the header and setup the object"""
21016 b477afb1c81e bundle2: move unpackheader closure into the class Pierre-Yves David <pierre-yves.david@fb.com> parents: 21015 diff changeset	1437 typesize = self._unpackheader(_fparttypesize)[0]
21015 14dd49260246 bundle2: move the fromheader closure into the class itself Pierre-Yves David <pierre-yves.david@fb.com> parents: 21014 diff changeset	1438 self.type = self._fromheader(typesize)
43077 687b865b95ad formatting: byteify all mercurial/ and hgext/ string literals Augie Fackler <augie@google.com> parents: 43076 diff changeset	1439 indebug(self.ui, b'part type: "%s"' % self.type)
21016 b477afb1c81e bundle2: move unpackheader closure into the class Pierre-Yves David <pierre-yves.david@fb.com> parents: 21015 diff changeset	1440 self.id = self._unpackheader(_fpartid)[0]
43077 687b865b95ad formatting: byteify all mercurial/ and hgext/ string literals Augie Fackler <augie@google.com> parents: 43076 diff changeset	1441 indebug(self.ui, b'part id: "%s"' % pycompat.bytestr(self.id))
23585 94b25d71dd0f bundle2.unbundlepart: decouple mandatory from parttype Eric Sumner <ericsumner@fb.com> parents: 23439 diff changeset	1442 # extract mandatory bit from type
43076 2372284d9457 formatting: blacken the codebase Augie Fackler <augie@google.com> parents: 42935 diff changeset	1443 self.mandatory = self.type != self.type.lower()
23585 94b25d71dd0f bundle2.unbundlepart: decouple mandatory from parttype Eric Sumner <ericsumner@fb.com> parents: 23439 diff changeset	1444 self.type = self.type.lower()
21014 a6246bba7b9e bundle2: add an unbundle part responsible from unbundling part Pierre-Yves David <pierre-yves.david@fb.com> parents: 21013 diff changeset	1445 ## reading parameters
a6246bba7b9e bundle2: add an unbundle part responsible from unbundling part Pierre-Yves David <pierre-yves.david@fb.com> parents: 21013 diff changeset	1446 # param count
21016 b477afb1c81e bundle2: move unpackheader closure into the class Pierre-Yves David <pierre-yves.david@fb.com> parents: 21015 diff changeset	1447 mancount, advcount = self._unpackheader(_fpartparamcount)
43077 687b865b95ad formatting: byteify all mercurial/ and hgext/ string literals Augie Fackler <augie@google.com> parents: 43076 diff changeset	1448 indebug(self.ui, b'part parameters: %i' % (mancount + advcount))
21014 a6246bba7b9e bundle2: add an unbundle part responsible from unbundling part Pierre-Yves David <pierre-yves.david@fb.com> parents: 21013 diff changeset	1449 # param size
21016 b477afb1c81e bundle2: move unpackheader closure into the class Pierre-Yves David <pierre-yves.david@fb.com> parents: 21015 diff changeset	1450 fparamsizes = _makefpartparamsizes(mancount + advcount)
b477afb1c81e bundle2: move unpackheader closure into the class Pierre-Yves David <pierre-yves.david@fb.com> parents: 21015 diff changeset	1451 paramsizes = self._unpackheader(fparamsizes)
21014 a6246bba7b9e bundle2: add an unbundle part responsible from unbundling part Pierre-Yves David <pierre-yves.david@fb.com> parents: 21013 diff changeset	1452 # make it a list of couple again
33700 9a323773216c bundle2: work around zip() being lazy in Python 3 Augie Fackler <augie@google.com> parents: 33699 diff changeset	1453 paramsizes = list(zip(paramsizes[::2], paramsizes[1::2]))
21014 a6246bba7b9e bundle2: add an unbundle part responsible from unbundling part Pierre-Yves David <pierre-yves.david@fb.com> parents: 21013 diff changeset	1454 # split mandatory from advisory
a6246bba7b9e bundle2: add an unbundle part responsible from unbundling part Pierre-Yves David <pierre-yves.david@fb.com> parents: 21013 diff changeset	1455 mansizes = paramsizes[:mancount]
a6246bba7b9e bundle2: add an unbundle part responsible from unbundling part Pierre-Yves David <pierre-yves.david@fb.com> parents: 21013 diff changeset	1456 advsizes = paramsizes[mancount:]
23139 e53f6b72a0e4 spelling: fixes from proofreading of spell checker issues Mads Kiilerich <madski@unity3d.com> parents: 23067 diff changeset	1457 # retrieve param value
21014 a6246bba7b9e bundle2: add an unbundle part responsible from unbundling part Pierre-Yves David <pierre-yves.david@fb.com> parents: 21013 diff changeset	1458 manparams = []
a6246bba7b9e bundle2: add an unbundle part responsible from unbundling part Pierre-Yves David <pierre-yves.david@fb.com> parents: 21013 diff changeset	1459 for key, value in mansizes:
21015 14dd49260246 bundle2: move the fromheader closure into the class itself Pierre-Yves David <pierre-yves.david@fb.com> parents: 21014 diff changeset	1460 manparams.append((self._fromheader(key), self._fromheader(value)))
21014 a6246bba7b9e bundle2: add an unbundle part responsible from unbundling part Pierre-Yves David <pierre-yves.david@fb.com> parents: 21013 diff changeset	1461 advparams = []
a6246bba7b9e bundle2: add an unbundle part responsible from unbundling part Pierre-Yves David <pierre-yves.david@fb.com> parents: 21013 diff changeset	1462 for key, value in advsizes:
21015 14dd49260246 bundle2: move the fromheader closure into the class itself Pierre-Yves David <pierre-yves.david@fb.com> parents: 21014 diff changeset	1463 advparams.append((self._fromheader(key), self._fromheader(value)))
21608 3cb96ca90c17 bundle2: introduce an ``_initparams`` method Pierre-Yves David <pierre-yves.david@fb.com> parents: 21607 diff changeset	1464 self._initparams(manparams, advparams)
21014 a6246bba7b9e bundle2: add an unbundle part responsible from unbundling part Pierre-Yves David <pierre-yves.david@fb.com> parents: 21013 diff changeset	1465 ## part payload
24034 9881a1437799 bundle2.unbundlepart: raise payloadchunks from a closure to a method Eric Sumner <ericsumner@fb.com> parents: 24026 diff changeset	1466 self._payloadstream = util.chunkbuffer(self._payloadchunks())
21019 3dc09f831a2e bundle2: lazy unbundle of part payload Pierre-Yves David <pierre-yves.david@fb.com> parents: 21016 diff changeset	1467 # we read the data, tell it
3dc09f831a2e bundle2: lazy unbundle of part payload Pierre-Yves David <pierre-yves.david@fb.com> parents: 21016 diff changeset	1468 self._initialized = True
3dc09f831a2e bundle2: lazy unbundle of part payload Pierre-Yves David <pierre-yves.david@fb.com> parents: 21016 diff changeset	1469
52928 f863fc99bef9 bundle2-part: provide a "__iter__" pass through Pierre-Yves David <pierre-yves.david@octobus.net> parents: 52899 diff changeset	1470 def __iter__(self):
f863fc99bef9 bundle2-part: provide a "__iter__" pass through Pierre-Yves David <pierre-yves.david@octobus.net> parents: 52899 diff changeset	1471 for chunk in self._payloadstream:
f863fc99bef9 bundle2-part: provide a "__iter__" pass through Pierre-Yves David <pierre-yves.david@octobus.net> parents: 52899 diff changeset	1472 self._pos += len(chunk)
f863fc99bef9 bundle2-part: provide a "__iter__" pass through Pierre-Yves David <pierre-yves.david@octobus.net> parents: 52899 diff changeset	1473 yield chunk
f863fc99bef9 bundle2-part: provide a "__iter__" pass through Pierre-Yves David <pierre-yves.david@octobus.net> parents: 52899 diff changeset	1474
35134 8aa43ff9c12c bundle2: implement generic part payload decoder Gregory Szorc <gregory.szorc@gmail.com> parents: 35133 diff changeset	1475 def _payloadchunks(self):
8aa43ff9c12c bundle2: implement generic part payload decoder Gregory Szorc <gregory.szorc@gmail.com> parents: 35133 diff changeset	1476 """Generator of decoded chunks in the payload."""
8aa43ff9c12c bundle2: implement generic part payload decoder Gregory Szorc <gregory.szorc@gmail.com> parents: 35133 diff changeset	1477 return decodepayloadchunks(self.ui, self._fp)
8aa43ff9c12c bundle2: implement generic part payload decoder Gregory Szorc <gregory.szorc@gmail.com> parents: 35133 diff changeset	1478
35135 db5038525718 bundle2: implement consume() API on unbundlepart Gregory Szorc <gregory.szorc@gmail.com> parents: 35134 diff changeset	1479 def consume(self):
db5038525718 bundle2: implement consume() API on unbundlepart Gregory Szorc <gregory.szorc@gmail.com> parents: 35134 diff changeset	1480 """Read the part payload until completion.
db5038525718 bundle2: implement consume() API on unbundlepart Gregory Szorc <gregory.szorc@gmail.com> parents: 35134 diff changeset	1481
db5038525718 bundle2: implement consume() API on unbundlepart Gregory Szorc <gregory.szorc@gmail.com> parents: 35134 diff changeset	1482 By consuming the part data, the underlying stream read offset will
db5038525718 bundle2: implement consume() API on unbundlepart Gregory Szorc <gregory.szorc@gmail.com> parents: 35134 diff changeset	1483 be advanced to the next part (or end of stream).
db5038525718 bundle2: implement consume() API on unbundlepart Gregory Szorc <gregory.szorc@gmail.com> parents: 35134 diff changeset	1484 """
db5038525718 bundle2: implement consume() API on unbundlepart Gregory Szorc <gregory.szorc@gmail.com> parents: 35134 diff changeset	1485 if self.consumed:
db5038525718 bundle2: implement consume() API on unbundlepart Gregory Szorc <gregory.szorc@gmail.com> parents: 35134 diff changeset	1486 return
db5038525718 bundle2: implement consume() API on unbundlepart Gregory Szorc <gregory.szorc@gmail.com> parents: 35134 diff changeset	1487
db5038525718 bundle2: implement consume() API on unbundlepart Gregory Szorc <gregory.szorc@gmail.com> parents: 35134 diff changeset	1488 chunk = self.read(32768)
db5038525718 bundle2: implement consume() API on unbundlepart Gregory Szorc <gregory.szorc@gmail.com> parents: 35134 diff changeset	1489 while chunk:
db5038525718 bundle2: implement consume() API on unbundlepart Gregory Szorc <gregory.szorc@gmail.com> parents: 35134 diff changeset	1490 self._pos += len(chunk)
db5038525718 bundle2: implement consume() API on unbundlepart Gregory Szorc <gregory.szorc@gmail.com> parents: 35134 diff changeset	1491 chunk = self.read(32768)
db5038525718 bundle2: implement consume() API on unbundlepart Gregory Szorc <gregory.szorc@gmail.com> parents: 35134 diff changeset	1492
21019 3dc09f831a2e bundle2: lazy unbundle of part payload Pierre-Yves David <pierre-yves.david@fb.com> parents: 21016 diff changeset	1493 def read(self, size=None):
3dc09f831a2e bundle2: lazy unbundle of part payload Pierre-Yves David <pierre-yves.david@fb.com> parents: 21016 diff changeset	1494 """read payload data"""
3dc09f831a2e bundle2: lazy unbundle of part payload Pierre-Yves David <pierre-yves.david@fb.com> parents: 21016 diff changeset	1495 if not self._initialized:
3dc09f831a2e bundle2: lazy unbundle of part payload Pierre-Yves David <pierre-yves.david@fb.com> parents: 21016 diff changeset	1496 self._readheader()
3dc09f831a2e bundle2: lazy unbundle of part payload Pierre-Yves David <pierre-yves.david@fb.com> parents: 21016 diff changeset	1497 if size is None:
3dc09f831a2e bundle2: lazy unbundle of part payload Pierre-Yves David <pierre-yves.david@fb.com> parents: 21016 diff changeset	1498 data = self._payloadstream.read()
3dc09f831a2e bundle2: lazy unbundle of part payload Pierre-Yves David <pierre-yves.david@fb.com> parents: 21016 diff changeset	1499 else:
3dc09f831a2e bundle2: lazy unbundle of part payload Pierre-Yves David <pierre-yves.david@fb.com> parents: 21016 diff changeset	1500 data = self._payloadstream.read(size)
25334 76cba1ecf55f bundle2: add generic debug output regarding processed part payload Pierre-Yves David <pierre-yves.david@fb.com> parents: 25333 diff changeset	1501 self._pos += len(data)
21019 3dc09f831a2e bundle2: lazy unbundle of part payload Pierre-Yves David <pierre-yves.david@fb.com> parents: 21016 diff changeset	1502 if size is None or len(data) < size:
25334 76cba1ecf55f bundle2: add generic debug output regarding processed part payload Pierre-Yves David <pierre-yves.david@fb.com> parents: 25333 diff changeset	1503 if not self.consumed and self._pos:
43076 2372284d9457 formatting: blacken the codebase Augie Fackler <augie@google.com> parents: 42935 diff changeset	1504 self.ui.debug(
43077 687b865b95ad formatting: byteify all mercurial/ and hgext/ string literals Augie Fackler <augie@google.com> parents: 43076 diff changeset	1505 b'bundle2-input-part: total payload size %i\n' % self._pos
43076 2372284d9457 formatting: blacken the codebase Augie Fackler <augie@google.com> parents: 42935 diff changeset	1506 )
21019 3dc09f831a2e bundle2: lazy unbundle of part payload Pierre-Yves David <pierre-yves.david@fb.com> parents: 21016 diff changeset	1507 self.consumed = True
3dc09f831a2e bundle2: lazy unbundle of part payload Pierre-Yves David <pierre-yves.david@fb.com> parents: 21016 diff changeset	1508 return data
3dc09f831a2e bundle2: lazy unbundle of part payload Pierre-Yves David <pierre-yves.david@fb.com> parents: 21016 diff changeset	1509
43076 2372284d9457 formatting: blacken the codebase Augie Fackler <augie@google.com> parents: 42935 diff changeset	1510
35133 073eec083e25 bundle2: extract logic for seeking bundle2 part into own class Gregory Szorc <gregory.szorc@gmail.com> parents: 35069 diff changeset	1511 class seekableunbundlepart(unbundlepart):
073eec083e25 bundle2: extract logic for seeking bundle2 part into own class Gregory Szorc <gregory.szorc@gmail.com> parents: 35069 diff changeset	1512 """A bundle2 part in a bundle that is seekable.
073eec083e25 bundle2: extract logic for seeking bundle2 part into own class Gregory Szorc <gregory.szorc@gmail.com> parents: 35069 diff changeset	1513
073eec083e25 bundle2: extract logic for seeking bundle2 part into own class Gregory Szorc <gregory.szorc@gmail.com> parents: 35069 diff changeset	1514 Regular ``unbundlepart`` instances can only be read once. This class
073eec083e25 bundle2: extract logic for seeking bundle2 part into own class Gregory Szorc <gregory.szorc@gmail.com> parents: 35069 diff changeset	1515 extends ``unbundlepart`` to enable bi-directional seeking within the
073eec083e25 bundle2: extract logic for seeking bundle2 part into own class Gregory Szorc <gregory.szorc@gmail.com> parents: 35069 diff changeset	1516 part.
073eec083e25 bundle2: extract logic for seeking bundle2 part into own class Gregory Szorc <gregory.szorc@gmail.com> parents: 35069 diff changeset	1517
073eec083e25 bundle2: extract logic for seeking bundle2 part into own class Gregory Szorc <gregory.szorc@gmail.com> parents: 35069 diff changeset	1518 Bundle2 part data consists of framed chunks. Offsets when seeking
073eec083e25 bundle2: extract logic for seeking bundle2 part into own class Gregory Szorc <gregory.szorc@gmail.com> parents: 35069 diff changeset	1519 refer to the decoded data, not the offsets in the underlying bundle2
073eec083e25 bundle2: extract logic for seeking bundle2 part into own class Gregory Szorc <gregory.szorc@gmail.com> parents: 35069 diff changeset	1520 stream.
073eec083e25 bundle2: extract logic for seeking bundle2 part into own class Gregory Szorc <gregory.szorc@gmail.com> parents: 35069 diff changeset	1521
073eec083e25 bundle2: extract logic for seeking bundle2 part into own class Gregory Szorc <gregory.szorc@gmail.com> parents: 35069 diff changeset	1522 To facilitate quickly seeking within the decoded data, instances of this
073eec083e25 bundle2: extract logic for seeking bundle2 part into own class Gregory Szorc <gregory.szorc@gmail.com> parents: 35069 diff changeset	1523 class maintain a mapping between offsets in the underlying stream and
073eec083e25 bundle2: extract logic for seeking bundle2 part into own class Gregory Szorc <gregory.szorc@gmail.com> parents: 35069 diff changeset	1524 the decoded payload. This mapping will consume memory in proportion
073eec083e25 bundle2: extract logic for seeking bundle2 part into own class Gregory Szorc <gregory.szorc@gmail.com> parents: 35069 diff changeset	1525 to the number of chunks within the payload (which almost certainly
073eec083e25 bundle2: extract logic for seeking bundle2 part into own class Gregory Szorc <gregory.szorc@gmail.com> parents: 35069 diff changeset	1526 increases in proportion with the size of the part).
073eec083e25 bundle2: extract logic for seeking bundle2 part into own class Gregory Szorc <gregory.szorc@gmail.com> parents: 35069 diff changeset	1527 """
43076 2372284d9457 formatting: blacken the codebase Augie Fackler <augie@google.com> parents: 42935 diff changeset	1528
35133 073eec083e25 bundle2: extract logic for seeking bundle2 part into own class Gregory Szorc <gregory.szorc@gmail.com> parents: 35069 diff changeset	1529 def __init__(self, ui, header, fp):
073eec083e25 bundle2: extract logic for seeking bundle2 part into own class Gregory Szorc <gregory.szorc@gmail.com> parents: 35069 diff changeset	1530 # (payload, file) offsets for chunk starts.
073eec083e25 bundle2: extract logic for seeking bundle2 part into own class Gregory Szorc <gregory.szorc@gmail.com> parents: 35069 diff changeset	1531 self._chunkindex = []
073eec083e25 bundle2: extract logic for seeking bundle2 part into own class Gregory Szorc <gregory.szorc@gmail.com> parents: 35069 diff changeset	1532
52668 5cc8deb96b48 pyupgrade: modernize calls to superclass methods Matt Harbison <matt_harbison@yahoo.com> parents: 52665 diff changeset	1533 super().__init__(ui, header, fp)
35133 073eec083e25 bundle2: extract logic for seeking bundle2 part into own class Gregory Szorc <gregory.szorc@gmail.com> parents: 35069 diff changeset	1534
073eec083e25 bundle2: extract logic for seeking bundle2 part into own class Gregory Szorc <gregory.szorc@gmail.com> parents: 35069 diff changeset	1535 def _payloadchunks(self, chunknum=0):
073eec083e25 bundle2: extract logic for seeking bundle2 part into own class Gregory Szorc <gregory.szorc@gmail.com> parents: 35069 diff changeset	1536 '''seek to specified chunk and start yielding data'''
073eec083e25 bundle2: extract logic for seeking bundle2 part into own class Gregory Szorc <gregory.szorc@gmail.com> parents: 35069 diff changeset	1537 if len(self._chunkindex) == 0:
43077 687b865b95ad formatting: byteify all mercurial/ and hgext/ string literals Augie Fackler <augie@google.com> parents: 43076 diff changeset	1538 assert chunknum == 0, b'Must start with chunk 0'
35133 073eec083e25 bundle2: extract logic for seeking bundle2 part into own class Gregory Szorc <gregory.szorc@gmail.com> parents: 35069 diff changeset	1539 self._chunkindex.append((0, self._tellfp()))
073eec083e25 bundle2: extract logic for seeking bundle2 part into own class Gregory Szorc <gregory.szorc@gmail.com> parents: 35069 diff changeset	1540 else:
41768 aaad36b88298 cleanup: use () to wrap long lines instead of \ Augie Fackler <augie@google.com> parents: 40416 diff changeset	1541 assert chunknum < len(self._chunkindex), (
43077 687b865b95ad formatting: byteify all mercurial/ and hgext/ string literals Augie Fackler <augie@google.com> parents: 43076 diff changeset	1542 b'Unknown chunk %d' % chunknum
43076 2372284d9457 formatting: blacken the codebase Augie Fackler <augie@google.com> parents: 42935 diff changeset	1543 )
35133 073eec083e25 bundle2: extract logic for seeking bundle2 part into own class Gregory Szorc <gregory.szorc@gmail.com> parents: 35069 diff changeset	1544 self._seekfp(self._chunkindex[chunknum][1])
073eec083e25 bundle2: extract logic for seeking bundle2 part into own class Gregory Szorc <gregory.szorc@gmail.com> parents: 35069 diff changeset	1545
073eec083e25 bundle2: extract logic for seeking bundle2 part into own class Gregory Szorc <gregory.szorc@gmail.com> parents: 35069 diff changeset	1546 pos = self._chunkindex[chunknum][0]
35134 8aa43ff9c12c bundle2: implement generic part payload decoder Gregory Szorc <gregory.szorc@gmail.com> parents: 35133 diff changeset	1547
8aa43ff9c12c bundle2: implement generic part payload decoder Gregory Szorc <gregory.szorc@gmail.com> parents: 35133 diff changeset	1548 for chunk in decodepayloadchunks(self.ui, self._fp):
8aa43ff9c12c bundle2: implement generic part payload decoder Gregory Szorc <gregory.szorc@gmail.com> parents: 35133 diff changeset	1549 chunknum += 1
8aa43ff9c12c bundle2: implement generic part payload decoder Gregory Szorc <gregory.szorc@gmail.com> parents: 35133 diff changeset	1550 pos += len(chunk)
8aa43ff9c12c bundle2: implement generic part payload decoder Gregory Szorc <gregory.szorc@gmail.com> parents: 35133 diff changeset	1551 if chunknum == len(self._chunkindex):
8aa43ff9c12c bundle2: implement generic part payload decoder Gregory Szorc <gregory.szorc@gmail.com> parents: 35133 diff changeset	1552 self._chunkindex.append((pos, self._tellfp()))
8aa43ff9c12c bundle2: implement generic part payload decoder Gregory Szorc <gregory.szorc@gmail.com> parents: 35133 diff changeset	1553
8aa43ff9c12c bundle2: implement generic part payload decoder Gregory Szorc <gregory.szorc@gmail.com> parents: 35133 diff changeset	1554 yield chunk
35133 073eec083e25 bundle2: extract logic for seeking bundle2 part into own class Gregory Szorc <gregory.szorc@gmail.com> parents: 35069 diff changeset	1555
073eec083e25 bundle2: extract logic for seeking bundle2 part into own class Gregory Szorc <gregory.szorc@gmail.com> parents: 35069 diff changeset	1556 def _findchunk(self, pos):
073eec083e25 bundle2: extract logic for seeking bundle2 part into own class Gregory Szorc <gregory.szorc@gmail.com> parents: 35069 diff changeset	1557 '''for a given payload position, return a chunk number and offset'''
073eec083e25 bundle2: extract logic for seeking bundle2 part into own class Gregory Szorc <gregory.szorc@gmail.com> parents: 35069 diff changeset	1558 for chunk, (ppos, fpos) in enumerate(self._chunkindex):
073eec083e25 bundle2: extract logic for seeking bundle2 part into own class Gregory Szorc <gregory.szorc@gmail.com> parents: 35069 diff changeset	1559 if ppos == pos:
073eec083e25 bundle2: extract logic for seeking bundle2 part into own class Gregory Szorc <gregory.szorc@gmail.com> parents: 35069 diff changeset	1560 return chunk, 0
073eec083e25 bundle2: extract logic for seeking bundle2 part into own class Gregory Szorc <gregory.szorc@gmail.com> parents: 35069 diff changeset	1561 elif ppos > pos:
073eec083e25 bundle2: extract logic for seeking bundle2 part into own class Gregory Szorc <gregory.szorc@gmail.com> parents: 35069 diff changeset	1562 return chunk - 1, pos - self._chunkindex[chunk - 1][0]
43077 687b865b95ad formatting: byteify all mercurial/ and hgext/ string literals Augie Fackler <augie@google.com> parents: 43076 diff changeset	1563 raise ValueError(b'Unknown chunk')
35133 073eec083e25 bundle2: extract logic for seeking bundle2 part into own class Gregory Szorc <gregory.szorc@gmail.com> parents: 35069 diff changeset	1564
24036 c7601086338a bundle2.unbundlepart: tell() implementation Eric Sumner <ericsumner@fb.com> parents: 24035 diff changeset	1565 def tell(self):
c7601086338a bundle2.unbundlepart: tell() implementation Eric Sumner <ericsumner@fb.com> parents: 24035 diff changeset	1566 return self._pos
c7601086338a bundle2.unbundlepart: tell() implementation Eric Sumner <ericsumner@fb.com> parents: 24035 diff changeset	1567
35069 241d9caca11e bundle2: use os.SEEK_* constants Gregory Szorc <gregory.szorc@gmail.com> parents: 34820 diff changeset	1568 def seek(self, offset, whence=os.SEEK_SET):
241d9caca11e bundle2: use os.SEEK_* constants Gregory Szorc <gregory.szorc@gmail.com> parents: 34820 diff changeset	1569 if whence == os.SEEK_SET:
24037 f0b498cfc5c8 bundle2.unbundlepart: implement seek() Eric Sumner <ericsumner@fb.com> parents: 24036 diff changeset	1570 newpos = offset
35069 241d9caca11e bundle2: use os.SEEK_* constants Gregory Szorc <gregory.szorc@gmail.com> parents: 34820 diff changeset	1571 elif whence == os.SEEK_CUR:
24037 f0b498cfc5c8 bundle2.unbundlepart: implement seek() Eric Sumner <ericsumner@fb.com> parents: 24036 diff changeset	1572 newpos = self._pos + offset
35069 241d9caca11e bundle2: use os.SEEK_* constants Gregory Szorc <gregory.szorc@gmail.com> parents: 34820 diff changeset	1573 elif whence == os.SEEK_END:
24037 f0b498cfc5c8 bundle2.unbundlepart: implement seek() Eric Sumner <ericsumner@fb.com> parents: 24036 diff changeset	1574 if not self.consumed:
35141 699b2a759319 bundle2: avoid unbound read when seeking Gregory Szorc <gregory.szorc@gmail.com> parents: 35140 diff changeset	1575 # Can't use self.consume() here because it advances self._pos.
699b2a759319 bundle2: avoid unbound read when seeking Gregory Szorc <gregory.szorc@gmail.com> parents: 35140 diff changeset	1576 chunk = self.read(32768)
699b2a759319 bundle2: avoid unbound read when seeking Gregory Szorc <gregory.szorc@gmail.com> parents: 35140 diff changeset	1577 while chunk:
699b2a759319 bundle2: avoid unbound read when seeking Gregory Szorc <gregory.szorc@gmail.com> parents: 35140 diff changeset	1578 chunk = self.read(32768)
24037 f0b498cfc5c8 bundle2.unbundlepart: implement seek() Eric Sumner <ericsumner@fb.com> parents: 24036 diff changeset	1579 newpos = self._chunkindex[-1][0] - offset
f0b498cfc5c8 bundle2.unbundlepart: implement seek() Eric Sumner <ericsumner@fb.com> parents: 24036 diff changeset	1580 else:
43077 687b865b95ad formatting: byteify all mercurial/ and hgext/ string literals Augie Fackler <augie@google.com> parents: 43076 diff changeset	1581 raise ValueError(b'Unknown whence value: %r' % (whence,))
24037 f0b498cfc5c8 bundle2.unbundlepart: implement seek() Eric Sumner <ericsumner@fb.com> parents: 24036 diff changeset	1582
f0b498cfc5c8 bundle2.unbundlepart: implement seek() Eric Sumner <ericsumner@fb.com> parents: 24036 diff changeset	1583 if newpos > self._chunkindex[-1][0] and not self.consumed:
35141 699b2a759319 bundle2: avoid unbound read when seeking Gregory Szorc <gregory.szorc@gmail.com> parents: 35140 diff changeset	1584 # Can't use self.consume() here because it advances self._pos.
699b2a759319 bundle2: avoid unbound read when seeking Gregory Szorc <gregory.szorc@gmail.com> parents: 35140 diff changeset	1585 chunk = self.read(32768)
699b2a759319 bundle2: avoid unbound read when seeking Gregory Szorc <gregory.szorc@gmail.com> parents: 35140 diff changeset	1586 while chunk:
699b2a759319 bundle2: avoid unbound read when seeking Gregory Szorc <gregory.szorc@gmail.com> parents: 35140 diff changeset	1587 chunk = self.read(32668)
699b2a759319 bundle2: avoid unbound read when seeking Gregory Szorc <gregory.szorc@gmail.com> parents: 35140 diff changeset	1588
24037 f0b498cfc5c8 bundle2.unbundlepart: implement seek() Eric Sumner <ericsumner@fb.com> parents: 24036 diff changeset	1589 if not 0 <= newpos <= self._chunkindex[-1][0]:
43077 687b865b95ad formatting: byteify all mercurial/ and hgext/ string literals Augie Fackler <augie@google.com> parents: 43076 diff changeset	1590 raise ValueError(b'Offset out of range')
24037 f0b498cfc5c8 bundle2.unbundlepart: implement seek() Eric Sumner <ericsumner@fb.com> parents: 24036 diff changeset	1591
f0b498cfc5c8 bundle2.unbundlepart: implement seek() Eric Sumner <ericsumner@fb.com> parents: 24036 diff changeset	1592 if self._pos != newpos:
f0b498cfc5c8 bundle2.unbundlepart: implement seek() Eric Sumner <ericsumner@fb.com> parents: 24036 diff changeset	1593 chunk, internaloffset = self._findchunk(newpos)
f0b498cfc5c8 bundle2.unbundlepart: implement seek() Eric Sumner <ericsumner@fb.com> parents: 24036 diff changeset	1594 self._payloadstream = util.chunkbuffer(self._payloadchunks(chunk))
f0b498cfc5c8 bundle2.unbundlepart: implement seek() Eric Sumner <ericsumner@fb.com> parents: 24036 diff changeset	1595 adjust = self.read(internaloffset)
f0b498cfc5c8 bundle2.unbundlepart: implement seek() Eric Sumner <ericsumner@fb.com> parents: 24036 diff changeset	1596 if len(adjust) != internaloffset:
43077 687b865b95ad formatting: byteify all mercurial/ and hgext/ string literals Augie Fackler <augie@google.com> parents: 43076 diff changeset	1597 raise error.Abort(_(b'Seek failed\n'))
24037 f0b498cfc5c8 bundle2.unbundlepart: implement seek() Eric Sumner <ericsumner@fb.com> parents: 24036 diff changeset	1598 self._pos = newpos
f0b498cfc5c8 bundle2.unbundlepart: implement seek() Eric Sumner <ericsumner@fb.com> parents: 24036 diff changeset	1599
31889 a02e773008f5 bundle2: move 'seek' and 'tell' methods off the unpackermixin class Pierre-Yves David <pierre-yves.david@ens-lyon.org> parents: 31863 diff changeset	1600 def _seekfp(self, offset, whence=0):
a02e773008f5 bundle2: move 'seek' and 'tell' methods off the unpackermixin class Pierre-Yves David <pierre-yves.david@ens-lyon.org> parents: 31863 diff changeset	1601 """move the underlying file pointer
a02e773008f5 bundle2: move 'seek' and 'tell' methods off the unpackermixin class Pierre-Yves David <pierre-yves.david@ens-lyon.org> parents: 31863 diff changeset	1602
a02e773008f5 bundle2: move 'seek' and 'tell' methods off the unpackermixin class Pierre-Yves David <pierre-yves.david@ens-lyon.org> parents: 31863 diff changeset	1603 This method is meant for internal usage by the bundle2 protocol only.
a02e773008f5 bundle2: move 'seek' and 'tell' methods off the unpackermixin class Pierre-Yves David <pierre-yves.david@ens-lyon.org> parents: 31863 diff changeset	1604 They directly manipulate the low level stream including bundle2 level
a02e773008f5 bundle2: move 'seek' and 'tell' methods off the unpackermixin class Pierre-Yves David <pierre-yves.david@ens-lyon.org> parents: 31863 diff changeset	1605 instruction.
a02e773008f5 bundle2: move 'seek' and 'tell' methods off the unpackermixin class Pierre-Yves David <pierre-yves.david@ens-lyon.org> parents: 31863 diff changeset	1606
a02e773008f5 bundle2: move 'seek' and 'tell' methods off the unpackermixin class Pierre-Yves David <pierre-yves.david@ens-lyon.org> parents: 31863 diff changeset	1607 Do not use it to implement higher-level logic or methods."""
a02e773008f5 bundle2: move 'seek' and 'tell' methods off the unpackermixin class Pierre-Yves David <pierre-yves.david@ens-lyon.org> parents: 31863 diff changeset	1608 if self._seekable:
a02e773008f5 bundle2: move 'seek' and 'tell' methods off the unpackermixin class Pierre-Yves David <pierre-yves.david@ens-lyon.org> parents: 31863 diff changeset	1609 return self._fp.seek(offset, whence)
a02e773008f5 bundle2: move 'seek' and 'tell' methods off the unpackermixin class Pierre-Yves David <pierre-yves.david@ens-lyon.org> parents: 31863 diff changeset	1610 else:
43077 687b865b95ad formatting: byteify all mercurial/ and hgext/ string literals Augie Fackler <augie@google.com> parents: 43076 diff changeset	1611 raise NotImplementedError(_(b'File pointer is not seekable'))
31889 a02e773008f5 bundle2: move 'seek' and 'tell' methods off the unpackermixin class Pierre-Yves David <pierre-yves.david@ens-lyon.org> parents: 31863 diff changeset	1612
a02e773008f5 bundle2: move 'seek' and 'tell' methods off the unpackermixin class Pierre-Yves David <pierre-yves.david@ens-lyon.org> parents: 31863 diff changeset	1613 def _tellfp(self):
a02e773008f5 bundle2: move 'seek' and 'tell' methods off the unpackermixin class Pierre-Yves David <pierre-yves.david@ens-lyon.org> parents: 31863 diff changeset	1614 """return the file offset, or None if file is not seekable
a02e773008f5 bundle2: move 'seek' and 'tell' methods off the unpackermixin class Pierre-Yves David <pierre-yves.david@ens-lyon.org> parents: 31863 diff changeset	1615
a02e773008f5 bundle2: move 'seek' and 'tell' methods off the unpackermixin class Pierre-Yves David <pierre-yves.david@ens-lyon.org> parents: 31863 diff changeset	1616 This method is meant for internal usage by the bundle2 protocol only.
a02e773008f5 bundle2: move 'seek' and 'tell' methods off the unpackermixin class Pierre-Yves David <pierre-yves.david@ens-lyon.org> parents: 31863 diff changeset	1617 They directly manipulate the low level stream including bundle2 level
a02e773008f5 bundle2: move 'seek' and 'tell' methods off the unpackermixin class Pierre-Yves David <pierre-yves.david@ens-lyon.org> parents: 31863 diff changeset	1618 instruction.
a02e773008f5 bundle2: move 'seek' and 'tell' methods off the unpackermixin class Pierre-Yves David <pierre-yves.david@ens-lyon.org> parents: 31863 diff changeset	1619
a02e773008f5 bundle2: move 'seek' and 'tell' methods off the unpackermixin class Pierre-Yves David <pierre-yves.david@ens-lyon.org> parents: 31863 diff changeset	1620 Do not use it to implement higher-level logic or methods."""
a02e773008f5 bundle2: move 'seek' and 'tell' methods off the unpackermixin class Pierre-Yves David <pierre-yves.david@ens-lyon.org> parents: 31863 diff changeset	1621 if self._seekable:
a02e773008f5 bundle2: move 'seek' and 'tell' methods off the unpackermixin class Pierre-Yves David <pierre-yves.david@ens-lyon.org> parents: 31863 diff changeset	1622 try:
a02e773008f5 bundle2: move 'seek' and 'tell' methods off the unpackermixin class Pierre-Yves David <pierre-yves.david@ens-lyon.org> parents: 31863 diff changeset	1623 return self._fp.tell()
52665 24ee91ba9aa8 pyupgrade: drop usage of py3 aliases for `OSError` Matt Harbison <matt_harbison@yahoo.com> parents: 51933 diff changeset	1624 except OSError as e:
31889 a02e773008f5 bundle2: move 'seek' and 'tell' methods off the unpackermixin class Pierre-Yves David <pierre-yves.david@ens-lyon.org> parents: 31863 diff changeset	1625 if e.errno == errno.ESPIPE:
a02e773008f5 bundle2: move 'seek' and 'tell' methods off the unpackermixin class Pierre-Yves David <pierre-yves.david@ens-lyon.org> parents: 31863 diff changeset	1626 self._seekable = False
a02e773008f5 bundle2: move 'seek' and 'tell' methods off the unpackermixin class Pierre-Yves David <pierre-yves.david@ens-lyon.org> parents: 31863 diff changeset	1627 else:
a02e773008f5 bundle2: move 'seek' and 'tell' methods off the unpackermixin class Pierre-Yves David <pierre-yves.david@ens-lyon.org> parents: 31863 diff changeset	1628 raise
a02e773008f5 bundle2: move 'seek' and 'tell' methods off the unpackermixin class Pierre-Yves David <pierre-yves.david@ens-lyon.org> parents: 31863 diff changeset	1629 return None
a02e773008f5 bundle2: move 'seek' and 'tell' methods off the unpackermixin class Pierre-Yves David <pierre-yves.david@ens-lyon.org> parents: 31863 diff changeset	1630
43076 2372284d9457 formatting: blacken the codebase Augie Fackler <augie@google.com> parents: 42935 diff changeset	1631
25317 5a5b7046d00e bundle2: add an informative comment to the capability dict Pierre-Yves David <pierre-yves.david@fb.com> parents: 25315 diff changeset	1632 # These are only the static capabilities.
5a5b7046d00e bundle2: add an informative comment to the capability dict Pierre-Yves David <pierre-yves.david@fb.com> parents: 25315 diff changeset	1633 # Check the 'getrepocaps' function for the rest.
52670 4cb75772818d pyupgrade: drop the quoting around type annotations Matt Harbison <matt_harbison@yahoo.com> parents: 52669 diff changeset	1634 capabilities: Capabilities = {
43077 687b865b95ad formatting: byteify all mercurial/ and hgext/ string literals Augie Fackler <augie@google.com> parents: 43076 diff changeset	1635 b'HG20': (),
687b865b95ad formatting: byteify all mercurial/ and hgext/ string literals Augie Fackler <augie@google.com> parents: 43076 diff changeset	1636 b'bookmarks': (),
687b865b95ad formatting: byteify all mercurial/ and hgext/ string literals Augie Fackler <augie@google.com> parents: 43076 diff changeset	1637 b'error': (b'abort', b'unsupportedcontent', b'pushraced', b'pushkey'),
687b865b95ad formatting: byteify all mercurial/ and hgext/ string literals Augie Fackler <augie@google.com> parents: 43076 diff changeset	1638 b'listkeys': (),
687b865b95ad formatting: byteify all mercurial/ and hgext/ string literals Augie Fackler <augie@google.com> parents: 43076 diff changeset	1639 b'pushkey': (),
687b865b95ad formatting: byteify all mercurial/ and hgext/ string literals Augie Fackler <augie@google.com> parents: 43076 diff changeset	1640 b'digests': tuple(sorted(util.DIGESTS.keys())),
687b865b95ad formatting: byteify all mercurial/ and hgext/ string literals Augie Fackler <augie@google.com> parents: 43076 diff changeset	1641 b'remote-changegroup': (b'http', b'https'),
687b865b95ad formatting: byteify all mercurial/ and hgext/ string literals Augie Fackler <augie@google.com> parents: 43076 diff changeset	1642 b'hgtagsfnodes': (),
687b865b95ad formatting: byteify all mercurial/ and hgext/ string literals Augie Fackler <augie@google.com> parents: 43076 diff changeset	1643 b'phases': (b'heads',),
687b865b95ad formatting: byteify all mercurial/ and hgext/ string literals Augie Fackler <augie@google.com> parents: 43076 diff changeset	1644 b'stream': (b'v2',),
43076 2372284d9457 formatting: blacken the codebase Augie Fackler <augie@google.com> parents: 42935 diff changeset	1645 }
2372284d9457 formatting: blacken the codebase Augie Fackler <augie@google.com> parents: 42935 diff changeset	1646
22341 2d16b39601b5 obsmarker: move bundle2caps from the localrepo class to the bundle2 module Pierre-Yves David <pierre-yves.david@fb.com> parents: 22340 diff changeset	1647
51731 6fc31e7bd5db typing: add some type hints for bundle2 capabilities Matt Harbison <matt_harbison@yahoo.com> parents: 51658 diff changeset	1648 # TODO: drop the default value for 'role'
52670 4cb75772818d pyupgrade: drop the quoting around type annotations Matt Harbison <matt_harbison@yahoo.com> parents: 52669 diff changeset	1649 def getrepocaps(repo, allowpushback: bool = False, role=None) -> Capabilities:
22342 262c5cc126c1 bundle2: introduce a `getrepocaps` to retrieve the bundle2 caps of a repo Pierre-Yves David <pierre-yves.david@fb.com> parents: 22341 diff changeset	1650 """return the bundle2 capabilities for a given repo
262c5cc126c1 bundle2: introduce a `getrepocaps` to retrieve the bundle2 caps of a repo Pierre-Yves David <pierre-yves.david@fb.com> parents: 22341 diff changeset	1651
22343 76677a2c1cfd bundle2: advertise the obsmarker part in bundle2 capabilities Pierre-Yves David <pierre-yves.david@fb.com> parents: 22342 diff changeset	1652 Exists to allow extensions (like evolution) to mutate the capabilities.
35783 c97639ad6874 bundle2: specify what capabilities will be used for Gregory Szorc <gregory.szorc@gmail.com> parents: 35763 diff changeset	1653
c97639ad6874 bundle2: specify what capabilities will be used for Gregory Szorc <gregory.szorc@gmail.com> parents: 35763 diff changeset	1654 The returned value is used for servers advertising their capabilities as
c97639ad6874 bundle2: specify what capabilities will be used for Gregory Szorc <gregory.szorc@gmail.com> parents: 35763 diff changeset	1655 well as clients advertising their capabilities to servers as part of
c97639ad6874 bundle2: specify what capabilities will be used for Gregory Szorc <gregory.szorc@gmail.com> parents: 35763 diff changeset	1656 bundle2 requests. The ``role`` argument specifies which is which.
22342 262c5cc126c1 bundle2: introduce a `getrepocaps` to retrieve the bundle2 caps of a repo Pierre-Yves David <pierre-yves.david@fb.com> parents: 22341 diff changeset	1657 """
43077 687b865b95ad formatting: byteify all mercurial/ and hgext/ string literals Augie Fackler <augie@google.com> parents: 43076 diff changeset	1658 if role not in (b'client', b'server'):
687b865b95ad formatting: byteify all mercurial/ and hgext/ string literals Augie Fackler <augie@google.com> parents: 43076 diff changeset	1659 raise error.ProgrammingError(b'role argument must be client or server')
35783 c97639ad6874 bundle2: specify what capabilities will be used for Gregory Szorc <gregory.szorc@gmail.com> parents: 35763 diff changeset	1660
22343 76677a2c1cfd bundle2: advertise the obsmarker part in bundle2 capabilities Pierre-Yves David <pierre-yves.david@fb.com> parents: 22342 diff changeset	1661 caps = capabilities.copy()
43077 687b865b95ad formatting: byteify all mercurial/ and hgext/ string literals Augie Fackler <augie@google.com> parents: 43076 diff changeset	1662 caps[b'changegroup'] = tuple(
43076 2372284d9457 formatting: blacken the codebase Augie Fackler <augie@google.com> parents: 42935 diff changeset	1663 sorted(changegroup.supportedincomingversions(repo))
2372284d9457 formatting: blacken the codebase Augie Fackler <augie@google.com> parents: 42935 diff changeset	1664 )
22953 b1d694d3975e obsolete: add exchange option Durham Goode <durham@fb.com> parents: 22661 diff changeset	1665 if obsolete.isenabled(repo, obsolete.exchangeopt):
43077 687b865b95ad formatting: byteify all mercurial/ and hgext/ string literals Augie Fackler <augie@google.com> parents: 43076 diff changeset	1666 supportedformat = tuple(b'V%i' % v for v in obsolete.formats)
687b865b95ad formatting: byteify all mercurial/ and hgext/ string literals Augie Fackler <augie@google.com> parents: 43076 diff changeset	1667 caps[b'obsmarkers'] = supportedformat
23439 743736fc7c41 bundle2-push: provide transaction to reply unbundler Eric Sumner <ericsumner@fb.com> parents: 23438 diff changeset	1668 if allowpushback:
43077 687b865b95ad formatting: byteify all mercurial/ and hgext/ string literals Augie Fackler <augie@google.com> parents: 43076 diff changeset	1669 caps[b'pushback'] = ()
687b865b95ad formatting: byteify all mercurial/ and hgext/ string literals Augie Fackler <augie@google.com> parents: 43076 diff changeset	1670 cpmode = repo.ui.config(b'server', b'concurrent-push-mode')
687b865b95ad formatting: byteify all mercurial/ and hgext/ string literals Augie Fackler <augie@google.com> parents: 43076 diff changeset	1671 if cpmode == b'check-related':
687b865b95ad formatting: byteify all mercurial/ and hgext/ string literals Augie Fackler <augie@google.com> parents: 43076 diff changeset	1672 caps[b'checkheads'] = (b'related',)
687b865b95ad formatting: byteify all mercurial/ and hgext/ string literals Augie Fackler <augie@google.com> parents: 43076 diff changeset	1673 if b'phases' in repo.ui.configlist(b'devel', b'legacy.exchange'):
687b865b95ad formatting: byteify all mercurial/ and hgext/ string literals Augie Fackler <augie@google.com> parents: 43076 diff changeset	1674 caps.pop(b'phases')
35790 c1b9eb15a51c bundle2: don't advertise stream bundle2 capability when feature disabled Gregory Szorc <gregory.szorc@gmail.com> parents: 35788 diff changeset	1675
c1b9eb15a51c bundle2: don't advertise stream bundle2 capability when feature disabled Gregory Szorc <gregory.szorc@gmail.com> parents: 35788 diff changeset	1676 # Don't advertise stream clone support in server mode if not configured.
43077 687b865b95ad formatting: byteify all mercurial/ and hgext/ string literals Augie Fackler <augie@google.com> parents: 43076 diff changeset	1677 if role == b'server':
43076 2372284d9457 formatting: blacken the codebase Augie Fackler <augie@google.com> parents: 42935 diff changeset	1678 streamsupported = repo.ui.configbool(
43077 687b865b95ad formatting: byteify all mercurial/ and hgext/ string literals Augie Fackler <augie@google.com> parents: 43076 diff changeset	1679 b'server', b'uncompressed', untrusted=True
43076 2372284d9457 formatting: blacken the codebase Augie Fackler <augie@google.com> parents: 42935 diff changeset	1680 )
43077 687b865b95ad formatting: byteify all mercurial/ and hgext/ string literals Augie Fackler <augie@google.com> parents: 43076 diff changeset	1681 featuresupported = repo.ui.configbool(b'server', b'bundle2.stream')
35790 c1b9eb15a51c bundle2: don't advertise stream bundle2 capability when feature disabled Gregory Szorc <gregory.szorc@gmail.com> parents: 35788 diff changeset	1682
c1b9eb15a51c bundle2: don't advertise stream bundle2 capability when feature disabled Gregory Szorc <gregory.szorc@gmail.com> parents: 35788 diff changeset	1683 if not streamsupported or not featuresupported:
43077 687b865b95ad formatting: byteify all mercurial/ and hgext/ string literals Augie Fackler <augie@google.com> parents: 43076 diff changeset	1684 caps.pop(b'stream')
35792 1d118f9f4f57 bundle2: always advertise client support for stream parts Gregory Szorc <gregory.szorc@gmail.com> parents: 35790 diff changeset	1685 # Else always advertise support on client, because payload support
1d118f9f4f57 bundle2: always advertise client support for stream parts Gregory Szorc <gregory.szorc@gmail.com> parents: 35790 diff changeset	1686 # should always be advertised.
35790 c1b9eb15a51c bundle2: don't advertise stream bundle2 capability when feature disabled Gregory Szorc <gregory.szorc@gmail.com> parents: 35788 diff changeset	1687
50558 58adcabc295f stream-clone: introduce the notion of an experimental "v3" version Pierre-Yves David <pierre-yves.david@octobus.net> parents: 50553 diff changeset	1688 if repo.ui.configbool(b'experimental', b'stream-v3'):
58adcabc295f stream-clone: introduce the notion of an experimental "v3" version Pierre-Yves David <pierre-yves.david@octobus.net> parents: 50553 diff changeset	1689 if b'stream' in caps:
58adcabc295f stream-clone: introduce the notion of an experimental "v3" version Pierre-Yves David <pierre-yves.david@octobus.net> parents: 50553 diff changeset	1690 caps[b'stream'] += (b'v3-exp',)
58adcabc295f stream-clone: introduce the notion of an experimental "v3" version Pierre-Yves David <pierre-yves.david@octobus.net> parents: 50553 diff changeset	1691
46672 7015b0232c5e exchange: stop advertising rev-branch-cache bundle capability Joerg Sonnenberger <joerg@bec.de> parents: 46571 diff changeset	1692 # b'rev-branch-cache is no longer advertised, but still supported
7015b0232c5e exchange: stop advertising rev-branch-cache bundle capability Joerg Sonnenberger <joerg@bec.de> parents: 46571 diff changeset	1693 # for legacy clients.
7015b0232c5e exchange: stop advertising rev-branch-cache bundle capability Joerg Sonnenberger <joerg@bec.de> parents: 46571 diff changeset	1694
22343 76677a2c1cfd bundle2: advertise the obsmarker part in bundle2 capabilities Pierre-Yves David <pierre-yves.david@fb.com> parents: 22342 diff changeset	1695 return caps
22342 262c5cc126c1 bundle2: introduce a `getrepocaps` to retrieve the bundle2 caps of a repo Pierre-Yves David <pierre-yves.david@fb.com> parents: 22341 diff changeset	1696
43076 2372284d9457 formatting: blacken the codebase Augie Fackler <augie@google.com> parents: 42935 diff changeset	1697
52670 4cb75772818d pyupgrade: drop the quoting around type annotations Matt Harbison <matt_harbison@yahoo.com> parents: 52669 diff changeset	1698 def bundle2caps(remote) -> Capabilities:
23139 e53f6b72a0e4 spelling: fixes from proofreading of spell checker issues Mads Kiilerich <madski@unity3d.com> parents: 23067 diff changeset	1699 """return the bundle capabilities of a peer as dict"""
43077 687b865b95ad formatting: byteify all mercurial/ and hgext/ string literals Augie Fackler <augie@google.com> parents: 43076 diff changeset	1700 raw = remote.capable(b'bundle2')
687b865b95ad formatting: byteify all mercurial/ and hgext/ string literals Augie Fackler <augie@google.com> parents: 43076 diff changeset	1701 if not raw and raw != b'':
21644 17755dd8c509 bundle2: introduce a bundle2caps function Pierre-Yves David <pierre-yves.david@fb.com> parents: 21628 diff changeset	1702 return {}
43077 687b865b95ad formatting: byteify all mercurial/ and hgext/ string literals Augie Fackler <augie@google.com> parents: 43076 diff changeset	1703 capsblob = urlreq.unquote(remote.capable(b'bundle2'))
21644 17755dd8c509 bundle2: introduce a bundle2caps function Pierre-Yves David <pierre-yves.david@fb.com> parents: 21628 diff changeset	1704 return decodecaps(capsblob)
21014 a6246bba7b9e bundle2: add an unbundle part responsible from unbundling part Pierre-Yves David <pierre-yves.david@fb.com> parents: 21013 diff changeset	1705
43076 2372284d9457 formatting: blacken the codebase Augie Fackler <augie@google.com> parents: 42935 diff changeset	1706
52670 4cb75772818d pyupgrade: drop the quoting around type annotations Matt Harbison <matt_harbison@yahoo.com> parents: 52669 diff changeset	1707 def obsmarkersversion(caps: Capabilities):
45957 89a2afe31e82 formating: upgrade to black 20.8b1 Augie Fackler <raf@durin42.com> parents: 45682 diff changeset	1708 """extract the list of supported obsmarkers versions from a bundle2caps dict"""
43077 687b865b95ad formatting: byteify all mercurial/ and hgext/ string literals Augie Fackler <augie@google.com> parents: 43076 diff changeset	1709 obscaps = caps.get(b'obsmarkers', ())
687b865b95ad formatting: byteify all mercurial/ and hgext/ string literals Augie Fackler <augie@google.com> parents: 43076 diff changeset	1710 return [int(c[1:]) for c in obscaps if c.startswith(b'V')]
22344 9829b7948100 bundle2: add a `obsmarkersversion` function to extract supported version Pierre-Yves David <pierre-yves.david@fb.com> parents: 22343 diff changeset	1711
43076 2372284d9457 formatting: blacken the codebase Augie Fackler <augie@google.com> parents: 42935 diff changeset	1712
2372284d9457 formatting: blacken the codebase Augie Fackler <augie@google.com> parents: 42935 diff changeset	1713 def writenewbundle(
2372284d9457 formatting: blacken the codebase Augie Fackler <augie@google.com> parents: 42935 diff changeset	1714 ui,
2372284d9457 formatting: blacken the codebase Augie Fackler <augie@google.com> parents: 42935 diff changeset	1715 repo,
2372284d9457 formatting: blacken the codebase Augie Fackler <augie@google.com> parents: 42935 diff changeset	1716 source,
2372284d9457 formatting: blacken the codebase Augie Fackler <augie@google.com> parents: 42935 diff changeset	1717 filename,
2372284d9457 formatting: blacken the codebase Augie Fackler <augie@google.com> parents: 42935 diff changeset	1718 bundletype,
2372284d9457 formatting: blacken the codebase Augie Fackler <augie@google.com> parents: 42935 diff changeset	1719 outgoing,
2372284d9457 formatting: blacken the codebase Augie Fackler <augie@google.com> parents: 42935 diff changeset	1720 opts,
2372284d9457 formatting: blacken the codebase Augie Fackler <augie@google.com> parents: 42935 diff changeset	1721 vfs=None,
2372284d9457 formatting: blacken the codebase Augie Fackler <augie@google.com> parents: 42935 diff changeset	1722 compression=None,
2372284d9457 formatting: blacken the codebase Augie Fackler <augie@google.com> parents: 42935 diff changeset	1723 compopts=None,
50433 bcf54837241d bundle: prevent implicite bundling of internal changeset Pierre-Yves David <pierre-yves.david@octobus.net> parents: 50426 diff changeset	1724 allow_internal=False,
43076 2372284d9457 formatting: blacken the codebase Augie Fackler <augie@google.com> parents: 42935 diff changeset	1725 ):
43077 687b865b95ad formatting: byteify all mercurial/ and hgext/ string literals Augie Fackler <augie@google.com> parents: 43076 diff changeset	1726 if bundletype.startswith(b'HG10'):
687b865b95ad formatting: byteify all mercurial/ and hgext/ string literals Augie Fackler <augie@google.com> parents: 43076 diff changeset	1727 cg = changegroup.makechangegroup(repo, outgoing, b'01', source)
43076 2372284d9457 formatting: blacken the codebase Augie Fackler <augie@google.com> parents: 42935 diff changeset	1728 return writebundle(
2372284d9457 formatting: blacken the codebase Augie Fackler <augie@google.com> parents: 42935 diff changeset	1729 ui,
2372284d9457 formatting: blacken the codebase Augie Fackler <augie@google.com> parents: 42935 diff changeset	1730 cg,
2372284d9457 formatting: blacken the codebase Augie Fackler <augie@google.com> parents: 42935 diff changeset	1731 filename,
2372284d9457 formatting: blacken the codebase Augie Fackler <augie@google.com> parents: 42935 diff changeset	1732 bundletype,
2372284d9457 formatting: blacken the codebase Augie Fackler <augie@google.com> parents: 42935 diff changeset	1733 vfs=vfs,
2372284d9457 formatting: blacken the codebase Augie Fackler <augie@google.com> parents: 42935 diff changeset	1734 compression=compression,
2372284d9457 formatting: blacken the codebase Augie Fackler <augie@google.com> parents: 42935 diff changeset	1735 compopts=compopts,
2372284d9457 formatting: blacken the codebase Augie Fackler <augie@google.com> parents: 42935 diff changeset	1736 )
43077 687b865b95ad formatting: byteify all mercurial/ and hgext/ string literals Augie Fackler <augie@google.com> parents: 43076 diff changeset	1737 elif not bundletype.startswith(b'HG20'):
687b865b95ad formatting: byteify all mercurial/ and hgext/ string literals Augie Fackler <augie@google.com> parents: 43076 diff changeset	1738 raise error.ProgrammingError(b'unknown bundle type: %s' % bundletype)
32261 9dc36df78403 bundle: introduce an higher level function to write bundle on disk Pierre-Yves David <pierre-yves.david@octobus.net> parents: 32232 diff changeset	1739
50433 bcf54837241d bundle: prevent implicite bundling of internal changeset Pierre-Yves David <pierre-yves.david@octobus.net> parents: 50426 diff changeset	1740 # enforce that no internal phase are to be bundled
bcf54837241d bundle: prevent implicite bundling of internal changeset Pierre-Yves David <pierre-yves.david@octobus.net> parents: 50426 diff changeset	1741 bundled_internal = repo.revs(b"%ln and _internal()", outgoing.ancestorsof)
bcf54837241d bundle: prevent implicite bundling of internal changeset Pierre-Yves David <pierre-yves.david@octobus.net> parents: 50426 diff changeset	1742 if bundled_internal and not allow_internal:
bcf54837241d bundle: prevent implicite bundling of internal changeset Pierre-Yves David <pierre-yves.david@octobus.net> parents: 50426 diff changeset	1743 count = len(repo.revs(b'%ln and _internal()', outgoing.missing))
bcf54837241d bundle: prevent implicite bundling of internal changeset Pierre-Yves David <pierre-yves.david@octobus.net> parents: 50426 diff changeset	1744 msg = "backup bundle would contains %d internal changesets"
bcf54837241d bundle: prevent implicite bundling of internal changeset Pierre-Yves David <pierre-yves.david@octobus.net> parents: 50426 diff changeset	1745 msg %= count
bcf54837241d bundle: prevent implicite bundling of internal changeset Pierre-Yves David <pierre-yves.david@octobus.net> parents: 50426 diff changeset	1746 raise error.ProgrammingError(msg)
bcf54837241d bundle: prevent implicite bundling of internal changeset Pierre-Yves David <pierre-yves.david@octobus.net> parents: 50426 diff changeset	1747
52670 4cb75772818d pyupgrade: drop the quoting around type annotations Matt Harbison <matt_harbison@yahoo.com> parents: 52669 diff changeset	1748 caps: Capabilities = {}
49341 290c29df1915 bundlespec: check the `obsolescence` value before adding the caps Pierre-Yves David <pierre-yves.david@octobus.net> parents: 49067 diff changeset	1749 if opts.get(b'obsolescence', False):
43077 687b865b95ad formatting: byteify all mercurial/ and hgext/ string literals Augie Fackler <augie@google.com> parents: 43076 diff changeset	1750 caps[b'obsmarkers'] = (b'V1',)
51596 c4aab3661f25 bundlespec: rationalize the way we specify stream bundle version Pierre-Yves David <pierre-yves.david@octobus.net> parents: 51467 diff changeset	1751 stream_version = opts.get(b'stream', b"")
c4aab3661f25 bundlespec: rationalize the way we specify stream bundle version Pierre-Yves David <pierre-yves.david@octobus.net> parents: 51467 diff changeset	1752 if stream_version == b"v2":
50552 f2bcb56a1d39 stream-clone: make sure the `stream` capability is set when bundling Pierre-Yves David <pierre-yves.david@octobus.net> parents: 50551 diff changeset	1753 caps[b'stream'] = [b'v2']
51596 c4aab3661f25 bundlespec: rationalize the way we specify stream bundle version Pierre-Yves David <pierre-yves.david@octobus.net> parents: 51467 diff changeset	1754 elif stream_version == b"v3-exp":
50567 f4a540c203d7 stream-clone: support streamv3 on the cli [hg bundle] Arseniy Alekseyev <aalekseyev@janestreet.com> parents: 50566 diff changeset	1755 caps[b'stream'] = [b'v3-exp']
32549 37d70ba1d9d1 bundle: add an experimental knob to include obsmarkers in bundle Pierre-Yves David <pierre-yves.david@octobus.net> parents: 32548 diff changeset	1756 bundle = bundle20(ui, caps)
32261 9dc36df78403 bundle: introduce an higher level function to write bundle on disk Pierre-Yves David <pierre-yves.david@octobus.net> parents: 32232 diff changeset	1757 bundle.setcompression(compression, compopts)
9dc36df78403 bundle: introduce an higher level function to write bundle on disk Pierre-Yves David <pierre-yves.david@octobus.net> parents: 32232 diff changeset	1758 _addpartsfromopts(ui, repo, bundle, source, outgoing, opts)
9dc36df78403 bundle: introduce an higher level function to write bundle on disk Pierre-Yves David <pierre-yves.david@octobus.net> parents: 32232 diff changeset	1759 chunkiter = bundle.getchunks()
9dc36df78403 bundle: introduce an higher level function to write bundle on disk Pierre-Yves David <pierre-yves.david@octobus.net> parents: 32232 diff changeset	1760
9dc36df78403 bundle: introduce an higher level function to write bundle on disk Pierre-Yves David <pierre-yves.david@octobus.net> parents: 32232 diff changeset	1761 return changegroup.writechunks(ui, chunkiter, filename, vfs=vfs)
9dc36df78403 bundle: introduce an higher level function to write bundle on disk Pierre-Yves David <pierre-yves.david@octobus.net> parents: 32232 diff changeset	1762
43076 2372284d9457 formatting: blacken the codebase Augie Fackler <augie@google.com> parents: 42935 diff changeset	1763
32261 9dc36df78403 bundle: introduce an higher level function to write bundle on disk Pierre-Yves David <pierre-yves.david@octobus.net> parents: 32232 diff changeset	1764 def _addpartsfromopts(ui, repo, bundler, source, outgoing, opts):
9dc36df78403 bundle: introduce an higher level function to write bundle on disk Pierre-Yves David <pierre-yves.david@octobus.net> parents: 32232 diff changeset	1765 # We should eventually reconcile this logic with the one behind
9dc36df78403 bundle: introduce an higher level function to write bundle on disk Pierre-Yves David <pierre-yves.david@octobus.net> parents: 32232 diff changeset	1766 # 'exchange.getbundle2partsgenerator'.
9dc36df78403 bundle: introduce an higher level function to write bundle on disk Pierre-Yves David <pierre-yves.david@octobus.net> parents: 32232 diff changeset	1767 #
9dc36df78403 bundle: introduce an higher level function to write bundle on disk Pierre-Yves David <pierre-yves.david@octobus.net> parents: 32232 diff changeset	1768 # The type of input from 'getbundle' and 'writenewbundle' are a bit
9dc36df78403 bundle: introduce an higher level function to write bundle on disk Pierre-Yves David <pierre-yves.david@octobus.net> parents: 32232 diff changeset	1769 # different right now. So we keep them separated for now for the sake of
9dc36df78403 bundle: introduce an higher level function to write bundle on disk Pierre-Yves David <pierre-yves.david@octobus.net> parents: 32232 diff changeset	1770 # simplicity.
9dc36df78403 bundle: introduce an higher level function to write bundle on disk Pierre-Yves David <pierre-yves.david@octobus.net> parents: 32232 diff changeset	1771
37008 66c0ff381cfc bundle: condition the changegroup part when creating a new bundle Boris Feld <boris.feld@octobus.net> parents: 36974 diff changeset	1772 # we might not always want a changegroup in such bundle, for example in
66c0ff381cfc bundle: condition the changegroup part when creating a new bundle Boris Feld <boris.feld@octobus.net> parents: 36974 diff changeset	1773 # stream bundles
43077 687b865b95ad formatting: byteify all mercurial/ and hgext/ string literals Augie Fackler <augie@google.com> parents: 43076 diff changeset	1774 if opts.get(b'changegroup', True):
687b865b95ad formatting: byteify all mercurial/ and hgext/ string literals Augie Fackler <augie@google.com> parents: 43076 diff changeset	1775 cgversion = opts.get(b'cg.version')
37008 66c0ff381cfc bundle: condition the changegroup part when creating a new bundle Boris Feld <boris.feld@octobus.net> parents: 36974 diff changeset	1776 if cgversion is None:
66c0ff381cfc bundle: condition the changegroup part when creating a new bundle Boris Feld <boris.feld@octobus.net> parents: 36974 diff changeset	1777 cgversion = changegroup.safeversion(repo)
66c0ff381cfc bundle: condition the changegroup part when creating a new bundle Boris Feld <boris.feld@octobus.net> parents: 36974 diff changeset	1778 cg = changegroup.makechangegroup(repo, outgoing, cgversion, source)
43077 687b865b95ad formatting: byteify all mercurial/ and hgext/ string literals Augie Fackler <augie@google.com> parents: 43076 diff changeset	1779 part = bundler.newpart(b'changegroup', data=cg.getchunks())
687b865b95ad formatting: byteify all mercurial/ and hgext/ string literals Augie Fackler <augie@google.com> parents: 43076 diff changeset	1780 part.addparam(b'version', cg.version)
687b865b95ad formatting: byteify all mercurial/ and hgext/ string literals Augie Fackler <augie@google.com> parents: 43076 diff changeset	1781 if b'clcount' in cg.extras:
43076 2372284d9457 formatting: blacken the codebase Augie Fackler <augie@google.com> parents: 42935 diff changeset	1782 part.addparam(
43077 687b865b95ad formatting: byteify all mercurial/ and hgext/ string literals Augie Fackler <augie@google.com> parents: 43076 diff changeset	1783 b'nbchanges', b'%d' % cg.extras[b'clcount'], mandatory=False
43076 2372284d9457 formatting: blacken the codebase Augie Fackler <augie@google.com> parents: 42935 diff changeset	1784 )
50426 385a4f8056e5 bundle: include required phases when saving a bundle (issue6794) Jason R. Coombs <jaraco@jaraco.com>, Pierre-Yves David <pierre-yves.david@octobus.net> parents: 49878 diff changeset	1785 if opts.get(b'phases'):
385a4f8056e5 bundle: include required phases when saving a bundle (issue6794) Jason R. Coombs <jaraco@jaraco.com>, Pierre-Yves David <pierre-yves.david@octobus.net> parents: 49878 diff changeset	1786 target_phase = phases.draft
385a4f8056e5 bundle: include required phases when saving a bundle (issue6794) Jason R. Coombs <jaraco@jaraco.com>, Pierre-Yves David <pierre-yves.david@octobus.net> parents: 49878 diff changeset	1787 for head in outgoing.ancestorsof:
385a4f8056e5 bundle: include required phases when saving a bundle (issue6794) Jason R. Coombs <jaraco@jaraco.com>, Pierre-Yves David <pierre-yves.david@octobus.net> parents: 49878 diff changeset	1788 target_phase = max(target_phase, repo[head].phase())
385a4f8056e5 bundle: include required phases when saving a bundle (issue6794) Jason R. Coombs <jaraco@jaraco.com>, Pierre-Yves David <pierre-yves.david@octobus.net> parents: 49878 diff changeset	1789 if target_phase > phases.draft:
385a4f8056e5 bundle: include required phases when saving a bundle (issue6794) Jason R. Coombs <jaraco@jaraco.com>, Pierre-Yves David <pierre-yves.david@octobus.net> parents: 49878 diff changeset	1790 part.addparam(
385a4f8056e5 bundle: include required phases when saving a bundle (issue6794) Jason R. Coombs <jaraco@jaraco.com>, Pierre-Yves David <pierre-yves.david@octobus.net> parents: 49878 diff changeset	1791 b'targetphase',
385a4f8056e5 bundle: include required phases when saving a bundle (issue6794) Jason R. Coombs <jaraco@jaraco.com>, Pierre-Yves David <pierre-yves.david@octobus.net> parents: 49878 diff changeset	1792 b'%d' % target_phase,
385a4f8056e5 bundle: include required phases when saving a bundle (issue6794) Jason R. Coombs <jaraco@jaraco.com>, Pierre-Yves David <pierre-yves.david@octobus.net> parents: 49878 diff changeset	1793 mandatory=False,
385a4f8056e5 bundle: include required phases when saving a bundle (issue6794) Jason R. Coombs <jaraco@jaraco.com>, Pierre-Yves David <pierre-yves.david@octobus.net> parents: 49878 diff changeset	1794 )
47237 19d4802cb304 sidedata: add a 'side-data' repository feature and use it Pierre-Yves David <pierre-yves.david@octobus.net> parents: 47142 diff changeset	1795 if repository.REPO_FEATURE_SIDE_DATA in repo.features:
19d4802cb304 sidedata: add a 'side-data' repository feature and use it Pierre-Yves David <pierre-yves.david@octobus.net> parents: 47142 diff changeset	1796 part.addparam(b'exp-sidedata', b'1')
43077 687b865b95ad formatting: byteify all mercurial/ and hgext/ string literals Augie Fackler <augie@google.com> parents: 43076 diff changeset	1797
51596 c4aab3661f25 bundlespec: rationalize the way we specify stream bundle version Pierre-Yves David <pierre-yves.david@octobus.net> parents: 51467 diff changeset	1798 if opts.get(b'stream', b"") == b"v2":
37169 6f467adf9f05 bundle: add the possibility to bundle a stream v2 part Boris Feld <boris.feld@octobus.net> parents: 37167 diff changeset	1799 addpartbundlestream2(bundler, repo, stream=True)
6f467adf9f05 bundle: add the possibility to bundle a stream v2 part Boris Feld <boris.feld@octobus.net> parents: 37167 diff changeset	1800
51596 c4aab3661f25 bundlespec: rationalize the way we specify stream bundle version Pierre-Yves David <pierre-yves.david@octobus.net> parents: 51467 diff changeset	1801 if opts.get(b'stream', b"") == b"v3-exp":
50567 f4a540c203d7 stream-clone: support streamv3 on the cli [hg bundle] Arseniy Alekseyev <aalekseyev@janestreet.com> parents: 50566 diff changeset	1802 addpartbundlestream2(bundler, repo, stream=True)
f4a540c203d7 stream-clone: support streamv3 on the cli [hg bundle] Arseniy Alekseyev <aalekseyev@janestreet.com> parents: 50566 diff changeset	1803
43077 687b865b95ad formatting: byteify all mercurial/ and hgext/ string literals Augie Fackler <augie@google.com> parents: 43076 diff changeset	1804 if opts.get(b'tagsfnodescache', True):
37167 6c7a6b04b274 bundlespec: move computing the bundle contentops in parsebundlespec Boris Feld <boris.feld@octobus.net> parents: 37087 diff changeset	1805 addparttagsfnodescache(repo, bundler, outgoing)
6c7a6b04b274 bundlespec: move computing the bundle contentops in parsebundlespec Boris Feld <boris.feld@octobus.net> parents: 37087 diff changeset	1806
43077 687b865b95ad formatting: byteify all mercurial/ and hgext/ string literals Augie Fackler <augie@google.com> parents: 43076 diff changeset	1807 if opts.get(b'revbranchcache', True):
37167 6c7a6b04b274 bundlespec: move computing the bundle contentops in parsebundlespec Boris Feld <boris.feld@octobus.net> parents: 37087 diff changeset	1808 addpartrevbranchcache(repo, bundler, outgoing)
32263 d7f93ebbbbdf bundle: add optional 'tagsfnodecache' data to on disk bundle (issue5543) Pierre-Yves David <pierre-yves.david@octobus.net> parents: 32262 diff changeset	1809
43077 687b865b95ad formatting: byteify all mercurial/ and hgext/ string literals Augie Fackler <augie@google.com> parents: 43076 diff changeset	1810 if opts.get(b'obsolescence', False):
51933 8583d138f436 exchange: improve computation of relevant markers for large repos Joerg Sonnenberger <joerg@bec.de> parents: 51901 diff changeset	1811 obsmarkers = repo.obsstore.relevantmarkers(nodes=outgoing.missing)
46165 41d695a08e90 bundle: optional advisory obsolescence parts Joerg Sonnenberger <joerg@bec.de> parents: 46114 diff changeset	1812 buildobsmarkerspart(
41d695a08e90 bundle: optional advisory obsolescence parts Joerg Sonnenberger <joerg@bec.de> parents: 46114 diff changeset	1813 bundler,
41d695a08e90 bundle: optional advisory obsolescence parts Joerg Sonnenberger <joerg@bec.de> parents: 46114 diff changeset	1814 obsmarkers,
41d695a08e90 bundle: optional advisory obsolescence parts Joerg Sonnenberger <joerg@bec.de> parents: 46114 diff changeset	1815 mandatory=opts.get(b'obsolescence-mandatory', True),
41d695a08e90 bundle: optional advisory obsolescence parts Joerg Sonnenberger <joerg@bec.de> parents: 46114 diff changeset	1816 )
32549 37d70ba1d9d1 bundle: add an experimental knob to include obsmarkers in bundle Pierre-Yves David <pierre-yves.david@octobus.net> parents: 32548 diff changeset	1817
43077 687b865b95ad formatting: byteify all mercurial/ and hgext/ string literals Augie Fackler <augie@google.com> parents: 43076 diff changeset	1818 if opts.get(b'phases', False):
33043 e8c8d81eb864 bundle: add config option to include phases Martin von Zweigbergk <martinvonz@google.com> parents: 33042 diff changeset	1819 headsbyphase = phases.subsetphaseheads(repo, outgoing.missing)
34326 5779d096a696 phases: move binary encoding into a reusable function Boris Feld <boris.feld@octobus.net> parents: 34325 diff changeset	1820 phasedata = phases.binaryencode(headsbyphase)
43077 687b865b95ad formatting: byteify all mercurial/ and hgext/ string literals Augie Fackler <augie@google.com> parents: 43076 diff changeset	1821 bundler.newpart(b'phase-heads', data=phasedata)
33043 e8c8d81eb864 bundle: add config option to include phases Martin von Zweigbergk <martinvonz@google.com> parents: 33042 diff changeset	1822
43076 2372284d9457 formatting: blacken the codebase Augie Fackler <augie@google.com> parents: 42935 diff changeset	1823
32262 6068712cbf03 bundle2: move tagsfnodecache generation in a generic function Pierre-Yves David <pierre-yves.david@octobus.net> parents: 32261 diff changeset	1824 def addparttagsfnodescache(repo, bundler, outgoing):
6068712cbf03 bundle2: move tagsfnodecache generation in a generic function Pierre-Yves David <pierre-yves.david@octobus.net> parents: 32261 diff changeset	1825 # we include the tags fnode cache for the bundle changeset
6068712cbf03 bundle2: move tagsfnodecache generation in a generic function Pierre-Yves David <pierre-yves.david@octobus.net> parents: 32261 diff changeset	1826 # (as an optional parts)
6068712cbf03 bundle2: move tagsfnodecache generation in a generic function Pierre-Yves David <pierre-yves.david@octobus.net> parents: 32261 diff changeset	1827 cache = tags.hgtagsfnodescache(repo.unfiltered())
6068712cbf03 bundle2: move tagsfnodecache generation in a generic function Pierre-Yves David <pierre-yves.david@octobus.net> parents: 32261 diff changeset	1828 chunks = []
6068712cbf03 bundle2: move tagsfnodecache generation in a generic function Pierre-Yves David <pierre-yves.david@octobus.net> parents: 32261 diff changeset	1829
6068712cbf03 bundle2: move tagsfnodecache generation in a generic function Pierre-Yves David <pierre-yves.david@octobus.net> parents: 32261 diff changeset	1830 # .hgtags fnodes are only relevant for head changesets. While we could
6068712cbf03 bundle2: move tagsfnodecache generation in a generic function Pierre-Yves David <pierre-yves.david@octobus.net> parents: 32261 diff changeset	1831 # transfer values for all known nodes, there will likely be little to
6068712cbf03 bundle2: move tagsfnodecache generation in a generic function Pierre-Yves David <pierre-yves.david@octobus.net> parents: 32261 diff changeset	1832 # no benefit.
6068712cbf03 bundle2: move tagsfnodecache generation in a generic function Pierre-Yves David <pierre-yves.david@octobus.net> parents: 32261 diff changeset	1833 #
6068712cbf03 bundle2: move tagsfnodecache generation in a generic function Pierre-Yves David <pierre-yves.david@octobus.net> parents: 32261 diff changeset	1834 # We don't bother using a generator to produce output data because
6068712cbf03 bundle2: move tagsfnodecache generation in a generic function Pierre-Yves David <pierre-yves.david@octobus.net> parents: 32261 diff changeset	1835 # a) we only have 40 bytes per head and even esoteric numbers of heads
6068712cbf03 bundle2: move tagsfnodecache generation in a generic function Pierre-Yves David <pierre-yves.david@octobus.net> parents: 32261 diff changeset	1836 # consume little memory (1M heads is 40MB) b) we don't want to send the
6068712cbf03 bundle2: move tagsfnodecache generation in a generic function Pierre-Yves David <pierre-yves.david@octobus.net> parents: 32261 diff changeset	1837 # part if we don't have entries and knowing if we have entries requires
6068712cbf03 bundle2: move tagsfnodecache generation in a generic function Pierre-Yves David <pierre-yves.david@octobus.net> parents: 32261 diff changeset	1838 # cache lookups.
45144 c93dd9d9f1e6 discovery: change users of `outgoing.missingheads` to `outgoing.ancestorsof` Manuel Jacob <me@manueljacob.de> parents: 45117 diff changeset	1839 for node in outgoing.ancestorsof:
32262 6068712cbf03 bundle2: move tagsfnodecache generation in a generic function Pierre-Yves David <pierre-yves.david@octobus.net> parents: 32261 diff changeset	1840 # Don't compute missing, as this may slow down serving.
6068712cbf03 bundle2: move tagsfnodecache generation in a generic function Pierre-Yves David <pierre-yves.david@octobus.net> parents: 32261 diff changeset	1841 fnode = cache.getfnode(node, computemissing=False)
46547 5aac1a1a5beb tagcache: distinguish between invalid and missing entries Matt Harbison <matt_harbison@yahoo.com> parents: 46481 diff changeset	1842 if fnode:
32262 6068712cbf03 bundle2: move tagsfnodecache generation in a generic function Pierre-Yves David <pierre-yves.david@octobus.net> parents: 32261 diff changeset	1843 chunks.extend([node, fnode])
6068712cbf03 bundle2: move tagsfnodecache generation in a generic function Pierre-Yves David <pierre-yves.david@octobus.net> parents: 32261 diff changeset	1844
6068712cbf03 bundle2: move tagsfnodecache generation in a generic function Pierre-Yves David <pierre-yves.david@octobus.net> parents: 32261 diff changeset	1845 if chunks:
51467 780fc16b62e6 bundle2: make the "hgtagsfnodes" part advisory Pierre-Yves David <pierre-yves.david@octobus.net> parents: 50998 diff changeset	1846 bundler.newpart(
780fc16b62e6 bundle2: make the "hgtagsfnodes" part advisory Pierre-Yves David <pierre-yves.david@octobus.net> parents: 50998 diff changeset	1847 b'hgtagsfnodes',
780fc16b62e6 bundle2: make the "hgtagsfnodes" part advisory Pierre-Yves David <pierre-yves.david@octobus.net> parents: 50998 diff changeset	1848 mandatory=False,
780fc16b62e6 bundle2: make the "hgtagsfnodes" part advisory Pierre-Yves David <pierre-yves.david@octobus.net> parents: 50998 diff changeset	1849 data=b''.join(chunks),
780fc16b62e6 bundle2: make the "hgtagsfnodes" part advisory Pierre-Yves David <pierre-yves.david@octobus.net> parents: 50998 diff changeset	1850 )
32262 6068712cbf03 bundle2: move tagsfnodecache generation in a generic function Pierre-Yves David <pierre-yves.david@octobus.net> parents: 32261 diff changeset	1851
43076 2372284d9457 formatting: blacken the codebase Augie Fackler <augie@google.com> parents: 42935 diff changeset	1852
36970 79b73be4dda5 rev-branch-cache: add a function to generate a part Boris Feld <boris.feld@octobus.net> parents: 36969 diff changeset	1853 def addpartrevbranchcache(repo, bundler, outgoing):
79b73be4dda5 rev-branch-cache: add a function to generate a part Boris Feld <boris.feld@octobus.net> parents: 36969 diff changeset	1854 # we include the rev branch cache for the bundle changeset
79b73be4dda5 rev-branch-cache: add a function to generate a part Boris Feld <boris.feld@octobus.net> parents: 36969 diff changeset	1855 # (as an optional parts)
79b73be4dda5 rev-branch-cache: add a function to generate a part Boris Feld <boris.feld@octobus.net> parents: 36969 diff changeset	1856 cache = repo.revbranchcache()
79b73be4dda5 rev-branch-cache: add a function to generate a part Boris Feld <boris.feld@octobus.net> parents: 36969 diff changeset	1857 cl = repo.unfiltered().changelog
79b73be4dda5 rev-branch-cache: add a function to generate a part Boris Feld <boris.feld@octobus.net> parents: 36969 diff changeset	1858 branchesdata = collections.defaultdict(lambda: (set(), set()))
79b73be4dda5 rev-branch-cache: add a function to generate a part Boris Feld <boris.feld@octobus.net> parents: 36969 diff changeset	1859 for node in outgoing.missing:
79b73be4dda5 rev-branch-cache: add a function to generate a part Boris Feld <boris.feld@octobus.net> parents: 36969 diff changeset	1860 branch, close = cache.branchinfo(cl.rev(node))
79b73be4dda5 rev-branch-cache: add a function to generate a part Boris Feld <boris.feld@octobus.net> parents: 36969 diff changeset	1861 branchesdata[branch][close].add(node)
79b73be4dda5 rev-branch-cache: add a function to generate a part Boris Feld <boris.feld@octobus.net> parents: 36969 diff changeset	1862
79b73be4dda5 rev-branch-cache: add a function to generate a part Boris Feld <boris.feld@octobus.net> parents: 36969 diff changeset	1863 def generate():
79b73be4dda5 rev-branch-cache: add a function to generate a part Boris Feld <boris.feld@octobus.net> parents: 36969 diff changeset	1864 for branch, (nodes, closed) in sorted(branchesdata.items()):
79b73be4dda5 rev-branch-cache: add a function to generate a part Boris Feld <boris.feld@octobus.net> parents: 36969 diff changeset	1865 utf8branch = encoding.fromlocal(branch)
79b73be4dda5 rev-branch-cache: add a function to generate a part Boris Feld <boris.feld@octobus.net> parents: 36969 diff changeset	1866 yield rbcstruct.pack(len(utf8branch), len(nodes), len(closed))
79b73be4dda5 rev-branch-cache: add a function to generate a part Boris Feld <boris.feld@octobus.net> parents: 36969 diff changeset	1867 yield utf8branch
52669 e627cc25b6f3 pyupgrade: rewrite `yield` statements in a loop to `yield from` Matt Harbison <matt_harbison@yahoo.com> parents: 52668 diff changeset	1868 yield from sorted(nodes)
e627cc25b6f3 pyupgrade: rewrite `yield` statements in a loop to `yield from` Matt Harbison <matt_harbison@yahoo.com> parents: 52668 diff changeset	1869 yield from sorted(closed)
36970 79b73be4dda5 rev-branch-cache: add a function to generate a part Boris Feld <boris.feld@octobus.net> parents: 36969 diff changeset	1870
43077 687b865b95ad formatting: byteify all mercurial/ and hgext/ string literals Augie Fackler <augie@google.com> parents: 43076 diff changeset	1871 bundler.newpart(b'cache:rev-branch-cache', data=generate(), mandatory=False)
43076 2372284d9457 formatting: blacken the codebase Augie Fackler <augie@google.com> parents: 42935 diff changeset	1872
36970 79b73be4dda5 rev-branch-cache: add a function to generate a part Boris Feld <boris.feld@octobus.net> parents: 36969 diff changeset	1873
37169 6f467adf9f05 bundle: add the possibility to bundle a stream v2 part Boris Feld <boris.feld@octobus.net> parents: 37167 diff changeset	1874 def _formatrequirementsspec(requirements):
43077 687b865b95ad formatting: byteify all mercurial/ and hgext/ string literals Augie Fackler <augie@google.com> parents: 43076 diff changeset	1875 requirements = [req for req in requirements if req != b"shared"]
687b865b95ad formatting: byteify all mercurial/ and hgext/ string literals Augie Fackler <augie@google.com> parents: 43076 diff changeset	1876 return urlreq.quote(b','.join(sorted(requirements)))
37169 6f467adf9f05 bundle: add the possibility to bundle a stream v2 part Boris Feld <boris.feld@octobus.net> parents: 37167 diff changeset	1877
43076 2372284d9457 formatting: blacken the codebase Augie Fackler <augie@google.com> parents: 42935 diff changeset	1878
37169 6f467adf9f05 bundle: add the possibility to bundle a stream v2 part Boris Feld <boris.feld@octobus.net> parents: 37167 diff changeset	1879 def _formatrequirementsparams(requirements):
6f467adf9f05 bundle: add the possibility to bundle a stream v2 part Boris Feld <boris.feld@octobus.net> parents: 37167 diff changeset	1880 requirements = _formatrequirementsspec(requirements)
43077 687b865b95ad formatting: byteify all mercurial/ and hgext/ string literals Augie Fackler <augie@google.com> parents: 43076 diff changeset	1881 params = b"%s%s" % (urlreq.quote(b"requirements="), requirements)
37169 6f467adf9f05 bundle: add the possibility to bundle a stream v2 part Boris Feld <boris.feld@octobus.net> parents: 37167 diff changeset	1882 return params
6f467adf9f05 bundle: add the possibility to bundle a stream v2 part Boris Feld <boris.feld@octobus.net> parents: 37167 diff changeset	1883
43076 2372284d9457 formatting: blacken the codebase Augie Fackler <augie@google.com> parents: 42935 diff changeset	1884
46726 bc2519513ae0 sidedata-exchange: add `wanted_sidedata` and `sidedata_computers` to repos Rapha?l Gom?s <rgomes@octobus.net> parents: 46672 diff changeset	1885 def format_remote_wanted_sidedata(repo):
bc2519513ae0 sidedata-exchange: add `wanted_sidedata` and `sidedata_computers` to repos Rapha?l Gom?s <rgomes@octobus.net> parents: 46672 diff changeset	1886 """Formats a repo's wanted sidedata categories into a bytestring for
bc2519513ae0 sidedata-exchange: add `wanted_sidedata` and `sidedata_computers` to repos Rapha?l Gom?s <rgomes@octobus.net> parents: 46672 diff changeset	1887 capabilities exchange."""
bc2519513ae0 sidedata-exchange: add `wanted_sidedata` and `sidedata_computers` to repos Rapha?l Gom?s <rgomes@octobus.net> parents: 46672 diff changeset	1888 wanted = b""
bc2519513ae0 sidedata-exchange: add `wanted_sidedata` and `sidedata_computers` to repos Rapha?l Gom?s <rgomes@octobus.net> parents: 46672 diff changeset	1889 if repo._wanted_sidedata:
bc2519513ae0 sidedata-exchange: add `wanted_sidedata` and `sidedata_computers` to repos Rapha?l Gom?s <rgomes@octobus.net> parents: 46672 diff changeset	1890 wanted = b','.join(
bc2519513ae0 sidedata-exchange: add `wanted_sidedata` and `sidedata_computers` to repos Rapha?l Gom?s <rgomes@octobus.net> parents: 46672 diff changeset	1891 pycompat.bytestr(c) for c in sorted(repo._wanted_sidedata)
bc2519513ae0 sidedata-exchange: add `wanted_sidedata` and `sidedata_computers` to repos Rapha?l Gom?s <rgomes@octobus.net> parents: 46672 diff changeset	1892 )
bc2519513ae0 sidedata-exchange: add `wanted_sidedata` and `sidedata_computers` to repos Rapha?l Gom?s <rgomes@octobus.net> parents: 46672 diff changeset	1893 return wanted
bc2519513ae0 sidedata-exchange: add `wanted_sidedata` and `sidedata_computers` to repos Rapha?l Gom?s <rgomes@octobus.net> parents: 46672 diff changeset	1894
bc2519513ae0 sidedata-exchange: add `wanted_sidedata` and `sidedata_computers` to repos Rapha?l Gom?s <rgomes@octobus.net> parents: 46672 diff changeset	1895
bc2519513ae0 sidedata-exchange: add `wanted_sidedata` and `sidedata_computers` to repos Rapha?l Gom?s <rgomes@octobus.net> parents: 46672 diff changeset	1896 def read_remote_wanted_sidedata(remote):
bc2519513ae0 sidedata-exchange: add `wanted_sidedata` and `sidedata_computers` to repos Rapha?l Gom?s <rgomes@octobus.net> parents: 46672 diff changeset	1897 sidedata_categories = remote.capable(b'exp-wanted-sidedata')
bc2519513ae0 sidedata-exchange: add `wanted_sidedata` and `sidedata_computers` to repos Rapha?l Gom?s <rgomes@octobus.net> parents: 46672 diff changeset	1898 return read_wanted_sidedata(sidedata_categories)
bc2519513ae0 sidedata-exchange: add `wanted_sidedata` and `sidedata_computers` to repos Rapha?l Gom?s <rgomes@octobus.net> parents: 46672 diff changeset	1899
bc2519513ae0 sidedata-exchange: add `wanted_sidedata` and `sidedata_computers` to repos Rapha?l Gom?s <rgomes@octobus.net> parents: 46672 diff changeset	1900
bc2519513ae0 sidedata-exchange: add `wanted_sidedata` and `sidedata_computers` to repos Rapha?l Gom?s <rgomes@octobus.net> parents: 46672 diff changeset	1901 def read_wanted_sidedata(formatted):
bc2519513ae0 sidedata-exchange: add `wanted_sidedata` and `sidedata_computers` to repos Rapha?l Gom?s <rgomes@octobus.net> parents: 46672 diff changeset	1902 if formatted:
bc2519513ae0 sidedata-exchange: add `wanted_sidedata` and `sidedata_computers` to repos Rapha?l Gom?s <rgomes@octobus.net> parents: 46672 diff changeset	1903 return set(formatted.split(b','))
bc2519513ae0 sidedata-exchange: add `wanted_sidedata` and `sidedata_computers` to repos Rapha?l Gom?s <rgomes@octobus.net> parents: 46672 diff changeset	1904 return set()
bc2519513ae0 sidedata-exchange: add `wanted_sidedata` and `sidedata_computers` to repos Rapha?l Gom?s <rgomes@octobus.net> parents: 46672 diff changeset	1905
bc2519513ae0 sidedata-exchange: add `wanted_sidedata` and `sidedata_computers` to repos Rapha?l Gom?s <rgomes@octobus.net> parents: 46672 diff changeset	1906
37169 6f467adf9f05 bundle: add the possibility to bundle a stream v2 part Boris Feld <boris.feld@octobus.net> parents: 37167 diff changeset	1907 def addpartbundlestream2(bundler, repo, **kwargs):
43554 9f70512ae2cf cleanup: remove pointless r-prefixes on single-quoted strings Augie Fackler <augie@google.com> parents: 43543 diff changeset	1908 if not kwargs.get('stream', False):
37169 6f467adf9f05 bundle: add the possibility to bundle a stream v2 part Boris Feld <boris.feld@octobus.net> parents: 37167 diff changeset	1909 return
6f467adf9f05 bundle: add the possibility to bundle a stream v2 part Boris Feld <boris.feld@octobus.net> parents: 37167 diff changeset	1910
6f467adf9f05 bundle: add the possibility to bundle a stream v2 part Boris Feld <boris.feld@octobus.net> parents: 37167 diff changeset	1911 if not streamclone.allowservergeneration(repo):
50551 cd2a2963b982 stream-clone: upgrade the error message for bad stream request Arseniy Alekseyev <aalekseyev@janestreet.com> parents: 50433 diff changeset	1912 msg = _(b'stream data requested but server does not allow this feature')
cd2a2963b982 stream-clone: upgrade the error message for bad stream request Arseniy Alekseyev <aalekseyev@janestreet.com> parents: 50433 diff changeset	1913 hint = _(b'the client seems buggy')
cd2a2963b982 stream-clone: upgrade the error message for bad stream request Arseniy Alekseyev <aalekseyev@janestreet.com> parents: 50433 diff changeset	1914 raise error.Abort(msg, hint=hint)
50553 3c0da0fa0eed stream-clone: check the version of streaming clone supported by the client Arseniy Alekseyev <aalekseyev@janestreet.com> parents: 50552 diff changeset	1915 if not (b'stream' in bundler.capabilities):
3c0da0fa0eed stream-clone: check the version of streaming clone supported by the client Arseniy Alekseyev <aalekseyev@janestreet.com> parents: 50552 diff changeset	1916 msg = _(
3c0da0fa0eed stream-clone: check the version of streaming clone supported by the client Arseniy Alekseyev <aalekseyev@janestreet.com> parents: 50552 diff changeset	1917 b'stream data requested but supported streaming clone versions were not specified'
3c0da0fa0eed stream-clone: check the version of streaming clone supported by the client Arseniy Alekseyev <aalekseyev@janestreet.com> parents: 50552 diff changeset	1918 )
3c0da0fa0eed stream-clone: check the version of streaming clone supported by the client Arseniy Alekseyev <aalekseyev@janestreet.com> parents: 50552 diff changeset	1919 hint = _(b'the client seems buggy')
3c0da0fa0eed stream-clone: check the version of streaming clone supported by the client Arseniy Alekseyev <aalekseyev@janestreet.com> parents: 50552 diff changeset	1920 raise error.Abort(msg, hint=hint)
50558 58adcabc295f stream-clone: introduce the notion of an experimental "v3" version Pierre-Yves David <pierre-yves.david@octobus.net> parents: 50553 diff changeset	1921 client_supported = set(bundler.capabilities[b'stream'])
58adcabc295f stream-clone: introduce the notion of an experimental "v3" version Pierre-Yves David <pierre-yves.david@octobus.net> parents: 50553 diff changeset	1922 server_supported = set(getrepocaps(repo, role=b'client').get(b'stream', []))
58adcabc295f stream-clone: introduce the notion of an experimental "v3" version Pierre-Yves David <pierre-yves.david@octobus.net> parents: 50553 diff changeset	1923 common_supported = client_supported & server_supported
58adcabc295f stream-clone: introduce the notion of an experimental "v3" version Pierre-Yves David <pierre-yves.david@octobus.net> parents: 50553 diff changeset	1924 if not common_supported:
58adcabc295f stream-clone: introduce the notion of an experimental "v3" version Pierre-Yves David <pierre-yves.david@octobus.net> parents: 50553 diff changeset	1925 msg = _(b'no common supported version with the client: %s; %s')
58adcabc295f stream-clone: introduce the notion of an experimental "v3" version Pierre-Yves David <pierre-yves.david@octobus.net> parents: 50553 diff changeset	1926 str_server = b','.join(sorted(server_supported))
58adcabc295f stream-clone: introduce the notion of an experimental "v3" version Pierre-Yves David <pierre-yves.david@octobus.net> parents: 50553 diff changeset	1927 str_client = b','.join(sorted(client_supported))
58adcabc295f stream-clone: introduce the notion of an experimental "v3" version Pierre-Yves David <pierre-yves.david@octobus.net> parents: 50553 diff changeset	1928 msg %= (str_server, str_client)
58adcabc295f stream-clone: introduce the notion of an experimental "v3" version Pierre-Yves David <pierre-yves.david@octobus.net> parents: 50553 diff changeset	1929 raise error.Abort(msg)
58adcabc295f stream-clone: introduce the notion of an experimental "v3" version Pierre-Yves David <pierre-yves.david@octobus.net> parents: 50553 diff changeset	1930 version = max(common_supported)
37169 6f467adf9f05 bundle: add the possibility to bundle a stream v2 part Boris Feld <boris.feld@octobus.net> parents: 37167 diff changeset	1931
6f467adf9f05 bundle: add the possibility to bundle a stream v2 part Boris Feld <boris.feld@octobus.net> parents: 37167 diff changeset	1932 # Stream clones don't compress well. And compression undermines a
6f467adf9f05 bundle: add the possibility to bundle a stream v2 part Boris Feld <boris.feld@octobus.net> parents: 37167 diff changeset	1933 # goal of stream clones, which is to be fast. Communicate the desire
6f467adf9f05 bundle: add the possibility to bundle a stream v2 part Boris Feld <boris.feld@octobus.net> parents: 37167 diff changeset	1934 # to avoid compression to consumers of the bundle.
6f467adf9f05 bundle: add the possibility to bundle a stream v2 part Boris Feld <boris.feld@octobus.net> parents: 37167 diff changeset	1935 bundler.prefercompressed = False
6f467adf9f05 bundle: add the possibility to bundle a stream v2 part Boris Feld <boris.feld@octobus.net> parents: 37167 diff changeset	1936
40338 af62936c2508 streamclone: new server config and some API changes for narrow stream clones Pulkit Goyal <pulkit@yandex-team.ru> parents: 40147 diff changeset	1937 # get the includes and excludes
43554 9f70512ae2cf cleanup: remove pointless r-prefixes on single-quoted strings Augie Fackler <augie@google.com> parents: 43543 diff changeset	1938 includepats = kwargs.get('includepats')
9f70512ae2cf cleanup: remove pointless r-prefixes on single-quoted strings Augie Fackler <augie@google.com> parents: 43543 diff changeset	1939 excludepats = kwargs.get('excludepats')
40338 af62936c2508 streamclone: new server config and some API changes for narrow stream clones Pulkit Goyal <pulkit@yandex-team.ru> parents: 40147 diff changeset	1940
43076 2372284d9457 formatting: blacken the codebase Augie Fackler <augie@google.com> parents: 42935 diff changeset	1941 narrowstream = repo.ui.configbool(
43077 687b865b95ad formatting: byteify all mercurial/ and hgext/ string literals Augie Fackler <augie@google.com> parents: 43076 diff changeset	1942 b'experimental', b'server.stream-narrow-clones'
43076 2372284d9457 formatting: blacken the codebase Augie Fackler <augie@google.com> parents: 42935 diff changeset	1943 )
40338 af62936c2508 streamclone: new server config and some API changes for narrow stream clones Pulkit Goyal <pulkit@yandex-team.ru> parents: 40147 diff changeset	1944
af62936c2508 streamclone: new server config and some API changes for narrow stream clones Pulkit Goyal <pulkit@yandex-team.ru> parents: 40147 diff changeset	1945 if (includepats or excludepats) and not narrowstream:
43077 687b865b95ad formatting: byteify all mercurial/ and hgext/ string literals Augie Fackler <augie@google.com> parents: 43076 diff changeset	1946 raise error.Abort(_(b'server does not support narrow stream clones'))
40338 af62936c2508 streamclone: new server config and some API changes for narrow stream clones Pulkit Goyal <pulkit@yandex-team.ru> parents: 40147 diff changeset	1947
40398 0ac794e0e285 streamclone: include obsstore file into stream bundle if client can read it Anton Shestakov <av6@dwimlabs.net> parents: 40372 diff changeset	1948 includeobsmarkers = False
0ac794e0e285 streamclone: include obsstore file into stream bundle if client can read it Anton Shestakov <av6@dwimlabs.net> parents: 40372 diff changeset	1949 if repo.obsstore:
0ac794e0e285 streamclone: include obsstore file into stream bundle if client can read it Anton Shestakov <av6@dwimlabs.net> parents: 40372 diff changeset	1950 remoteversions = obsmarkersversion(bundler.capabilities)
40399 4ab6e7b4fe8a streamclone: abort when client needs to handle obsmarkers, but doesn't Anton Shestakov <av6@dwimlabs.net> parents: 40398 diff changeset	1951 if not remoteversions:
43076 2372284d9457 formatting: blacken the codebase Augie Fackler <augie@google.com> parents: 42935 diff changeset	1952 raise error.Abort(
2372284d9457 formatting: blacken the codebase Augie Fackler <augie@google.com> parents: 42935 diff changeset	1953 _(
43077 687b865b95ad formatting: byteify all mercurial/ and hgext/ string literals Augie Fackler <augie@google.com> parents: 43076 diff changeset	1954 b'server has obsolescence markers, but client '
687b865b95ad formatting: byteify all mercurial/ and hgext/ string literals Augie Fackler <augie@google.com> parents: 43076 diff changeset	1955 b'cannot receive them via stream clone'
43076 2372284d9457 formatting: blacken the codebase Augie Fackler <augie@google.com> parents: 42935 diff changeset	1956 )
2372284d9457 formatting: blacken the codebase Augie Fackler <augie@google.com> parents: 42935 diff changeset	1957 )
40399 4ab6e7b4fe8a streamclone: abort when client needs to handle obsmarkers, but doesn't Anton Shestakov <av6@dwimlabs.net> parents: 40398 diff changeset	1958 elif repo.obsstore._version in remoteversions:
40398 0ac794e0e285 streamclone: include obsstore file into stream bundle if client can read it Anton Shestakov <av6@dwimlabs.net> parents: 40372 diff changeset	1959 includeobsmarkers = True
0ac794e0e285 streamclone: include obsstore file into stream bundle if client can read it Anton Shestakov <av6@dwimlabs.net> parents: 40372 diff changeset	1960
50558 58adcabc295f stream-clone: introduce the notion of an experimental "v3" version Pierre-Yves David <pierre-yves.david@octobus.net> parents: 50553 diff changeset	1961 if version == b"v2":
58adcabc295f stream-clone: introduce the notion of an experimental "v3" version Pierre-Yves David <pierre-yves.david@octobus.net> parents: 50553 diff changeset	1962 filecount, bytecount, it = streamclone.generatev2(
58adcabc295f stream-clone: introduce the notion of an experimental "v3" version Pierre-Yves David <pierre-yves.david@octobus.net> parents: 50553 diff changeset	1963 repo, includepats, excludepats, includeobsmarkers
58adcabc295f stream-clone: introduce the notion of an experimental "v3" version Pierre-Yves David <pierre-yves.david@octobus.net> parents: 50553 diff changeset	1964 )
58adcabc295f stream-clone: introduce the notion of an experimental "v3" version Pierre-Yves David <pierre-yves.david@octobus.net> parents: 50553 diff changeset	1965 requirements = streamclone.streamed_requirements(repo)
58adcabc295f stream-clone: introduce the notion of an experimental "v3" version Pierre-Yves David <pierre-yves.david@octobus.net> parents: 50553 diff changeset	1966 requirements = _formatrequirementsspec(requirements)
58adcabc295f stream-clone: introduce the notion of an experimental "v3" version Pierre-Yves David <pierre-yves.david@octobus.net> parents: 50553 diff changeset	1967 part = bundler.newpart(b'stream2', data=it)
58adcabc295f stream-clone: introduce the notion of an experimental "v3" version Pierre-Yves David <pierre-yves.david@octobus.net> parents: 50553 diff changeset	1968 part.addparam(b'bytecount', b'%d' % bytecount, mandatory=True)
58adcabc295f stream-clone: introduce the notion of an experimental "v3" version Pierre-Yves David <pierre-yves.david@octobus.net> parents: 50553 diff changeset	1969 part.addparam(b'filecount', b'%d' % filecount, mandatory=True)
58adcabc295f stream-clone: introduce the notion of an experimental "v3" version Pierre-Yves David <pierre-yves.david@octobus.net> parents: 50553 diff changeset	1970 part.addparam(b'requirements', requirements, mandatory=True)
58adcabc295f stream-clone: introduce the notion of an experimental "v3" version Pierre-Yves David <pierre-yves.david@octobus.net> parents: 50553 diff changeset	1971 elif version == b"v3-exp":
50706 0452af304808 stream-clone: add a v3 version of the protocol Arseniy Alekseyev <aalekseyev@janestreet.com> parents: 50647 diff changeset	1972 it = streamclone.generatev3(
50558 58adcabc295f stream-clone: introduce the notion of an experimental "v3" version Pierre-Yves David <pierre-yves.david@octobus.net> parents: 50553 diff changeset	1973 repo, includepats, excludepats, includeobsmarkers
58adcabc295f stream-clone: introduce the notion of an experimental "v3" version Pierre-Yves David <pierre-yves.david@octobus.net> parents: 50553 diff changeset	1974 )
58adcabc295f stream-clone: introduce the notion of an experimental "v3" version Pierre-Yves David <pierre-yves.david@octobus.net> parents: 50553 diff changeset	1975 requirements = streamclone.streamed_requirements(repo)
58adcabc295f stream-clone: introduce the notion of an experimental "v3" version Pierre-Yves David <pierre-yves.david@octobus.net> parents: 50553 diff changeset	1976 requirements = _formatrequirementsspec(requirements)
50566 116da6bb7e3b stream-clone: add the `-exp` prefix to the bundle part Pierre-Yves David <pierre-yves.david@octobus.net> parents: 50558 diff changeset	1977 part = bundler.newpart(b'stream3-exp', data=it)
50558 58adcabc295f stream-clone: introduce the notion of an experimental "v3" version Pierre-Yves David <pierre-yves.david@octobus.net> parents: 50553 diff changeset	1978 part.addparam(b'requirements', requirements, mandatory=True)
37169 6f467adf9f05 bundle: add the possibility to bundle a stream v2 part Boris Feld <boris.feld@octobus.net> parents: 37167 diff changeset	1979
43076 2372284d9457 formatting: blacken the codebase Augie Fackler <augie@google.com> parents: 42935 diff changeset	1980
46165 41d695a08e90 bundle: optional advisory obsolescence parts Joerg Sonnenberger <joerg@bec.de> parents: 46114 diff changeset	1981 def buildobsmarkerspart(bundler, markers, mandatory=True):
32548 e70d6dbde713 bundle2: move function building obsmarker-part in the bundle2 module Pierre-Yves David <pierre-yves.david@octobus.net> parents: 32425 diff changeset	1982 """add an obsmarker part to the bundler with <markers>
e70d6dbde713 bundle2: move function building obsmarker-part in the bundle2 module Pierre-Yves David <pierre-yves.david@octobus.net> parents: 32425 diff changeset	1983
e70d6dbde713 bundle2: move function building obsmarker-part in the bundle2 module Pierre-Yves David <pierre-yves.david@octobus.net> parents: 32425 diff changeset	1984 No part is created if markers is empty.
e70d6dbde713 bundle2: move function building obsmarker-part in the bundle2 module Pierre-Yves David <pierre-yves.david@octobus.net> parents: 32425 diff changeset	1985 Raises ValueError if the bundler doesn't support any known obsmarker format.
e70d6dbde713 bundle2: move function building obsmarker-part in the bundle2 module Pierre-Yves David <pierre-yves.david@octobus.net> parents: 32425 diff changeset	1986 """
e70d6dbde713 bundle2: move function building obsmarker-part in the bundle2 module Pierre-Yves David <pierre-yves.david@octobus.net> parents: 32425 diff changeset	1987 if not markers:
e70d6dbde713 bundle2: move function building obsmarker-part in the bundle2 module Pierre-Yves David <pierre-yves.david@octobus.net> parents: 32425 diff changeset	1988 return None
e70d6dbde713 bundle2: move function building obsmarker-part in the bundle2 module Pierre-Yves David <pierre-yves.david@octobus.net> parents: 32425 diff changeset	1989
e70d6dbde713 bundle2: move function building obsmarker-part in the bundle2 module Pierre-Yves David <pierre-yves.david@octobus.net> parents: 32425 diff changeset	1990 remoteversions = obsmarkersversion(bundler.capabilities)
e70d6dbde713 bundle2: move function building obsmarker-part in the bundle2 module Pierre-Yves David <pierre-yves.david@octobus.net> parents: 32425 diff changeset	1991 version = obsolete.commonversion(remoteversions)
e70d6dbde713 bundle2: move function building obsmarker-part in the bundle2 module Pierre-Yves David <pierre-yves.david@octobus.net> parents: 32425 diff changeset	1992 if version is None:
43077 687b865b95ad formatting: byteify all mercurial/ and hgext/ string literals Augie Fackler <augie@google.com> parents: 43076 diff changeset	1993 raise ValueError(b'bundler does not support common obsmarker format')
32548 e70d6dbde713 bundle2: move function building obsmarker-part in the bundle2 module Pierre-Yves David <pierre-yves.david@octobus.net> parents: 32425 diff changeset	1994 stream = obsolete.encodemarkers(markers, True, version=version)
46165 41d695a08e90 bundle: optional advisory obsolescence parts Joerg Sonnenberger <joerg@bec.de> parents: 46114 diff changeset	1995 return bundler.newpart(b'obsmarkers', data=stream, mandatory=mandatory)
32548 e70d6dbde713 bundle2: move function building obsmarker-part in the bundle2 module Pierre-Yves David <pierre-yves.david@octobus.net> parents: 32425 diff changeset	1996
43076 2372284d9457 formatting: blacken the codebase Augie Fackler <augie@google.com> parents: 42935 diff changeset	1997
2372284d9457 formatting: blacken the codebase Augie Fackler <augie@google.com> parents: 42935 diff changeset	1998 def writebundle(
2372284d9457 formatting: blacken the codebase Augie Fackler <augie@google.com> parents: 42935 diff changeset	1999 ui, cg, filename, bundletype, vfs=None, compression=None, compopts=None
2372284d9457 formatting: blacken the codebase Augie Fackler <augie@google.com> parents: 42935 diff changeset	2000 ):
28666 ae53ecc47414 bundle: move writebundle() from changegroup.py to bundle2.py (API) Martin von Zweigbergk <martinvonz@google.com> parents: 27953 diff changeset	2001 """Write a bundle file and return its filename.
ae53ecc47414 bundle: move writebundle() from changegroup.py to bundle2.py (API) Martin von Zweigbergk <martinvonz@google.com> parents: 27953 diff changeset	2002
ae53ecc47414 bundle: move writebundle() from changegroup.py to bundle2.py (API) Martin von Zweigbergk <martinvonz@google.com> parents: 27953 diff changeset	2003 Existing files will not be overwritten.
ae53ecc47414 bundle: move writebundle() from changegroup.py to bundle2.py (API) Martin von Zweigbergk <martinvonz@google.com> parents: 27953 diff changeset	2004 If no filename is specified, a temporary file is created.
ae53ecc47414 bundle: move writebundle() from changegroup.py to bundle2.py (API) Martin von Zweigbergk <martinvonz@google.com> parents: 27953 diff changeset	2005 bz2 compression can be turned off.
ae53ecc47414 bundle: move writebundle() from changegroup.py to bundle2.py (API) Martin von Zweigbergk <martinvonz@google.com> parents: 27953 diff changeset	2006 The bundle file will be deleted in case of errors.
ae53ecc47414 bundle: move writebundle() from changegroup.py to bundle2.py (API) Martin von Zweigbergk <martinvonz@google.com> parents: 27953 diff changeset	2007 """
ae53ecc47414 bundle: move writebundle() from changegroup.py to bundle2.py (API) Martin von Zweigbergk <martinvonz@google.com> parents: 27953 diff changeset	2008
43077 687b865b95ad formatting: byteify all mercurial/ and hgext/ string literals Augie Fackler <augie@google.com> parents: 43076 diff changeset	2009 if bundletype == b"HG20":
28666 ae53ecc47414 bundle: move writebundle() from changegroup.py to bundle2.py (API) Martin von Zweigbergk <martinvonz@google.com> parents: 27953 diff changeset	2010 bundle = bundle20(ui)
30757 511a4bf52754 bundle2: allow compression options to be passed to compressor Gregory Szorc <gregory.szorc@gmail.com> parents: 30448 diff changeset	2011 bundle.setcompression(compression, compopts)
43077 687b865b95ad formatting: byteify all mercurial/ and hgext/ string literals Augie Fackler <augie@google.com> parents: 43076 diff changeset	2012 part = bundle.newpart(b'changegroup', data=cg.getchunks())
687b865b95ad formatting: byteify all mercurial/ and hgext/ string literals Augie Fackler <augie@google.com> parents: 43076 diff changeset	2013 part.addparam(b'version', cg.version)
687b865b95ad formatting: byteify all mercurial/ and hgext/ string literals Augie Fackler <augie@google.com> parents: 43076 diff changeset	2014 if b'clcount' in cg.extras:
43076 2372284d9457 formatting: blacken the codebase Augie Fackler <augie@google.com> parents: 42935 diff changeset	2015 part.addparam(
43077 687b865b95ad formatting: byteify all mercurial/ and hgext/ string literals Augie Fackler <augie@google.com> parents: 43076 diff changeset	2016 b'nbchanges', b'%d' % cg.extras[b'clcount'], mandatory=False
43076 2372284d9457 formatting: blacken the codebase Augie Fackler <augie@google.com> parents: 42935 diff changeset	2017 )
28666 ae53ecc47414 bundle: move writebundle() from changegroup.py to bundle2.py (API) Martin von Zweigbergk <martinvonz@google.com> parents: 27953 diff changeset	2018 chunkiter = bundle.getchunks()
ae53ecc47414 bundle: move writebundle() from changegroup.py to bundle2.py (API) Martin von Zweigbergk <martinvonz@google.com> parents: 27953 diff changeset	2019 else:
ae53ecc47414 bundle: move writebundle() from changegroup.py to bundle2.py (API) Martin von Zweigbergk <martinvonz@google.com> parents: 27953 diff changeset	2020 # compression argument is only for the bundle2 case
ae53ecc47414 bundle: move writebundle() from changegroup.py to bundle2.py (API) Martin von Zweigbergk <martinvonz@google.com> parents: 27953 diff changeset	2021 assert compression is None
43077 687b865b95ad formatting: byteify all mercurial/ and hgext/ string literals Augie Fackler <augie@google.com> parents: 43076 diff changeset	2022 if cg.version != b'01':
43076 2372284d9457 formatting: blacken the codebase Augie Fackler <augie@google.com> parents: 42935 diff changeset	2023 raise error.Abort(
43117 8ff1ecfadcd1 cleanup: join string literals that are already on one line Martin von Zweigbergk <martinvonz@google.com> parents: 43115 diff changeset	2024 _(b'old bundle types only supports v1 changegroups')
43076 2372284d9457 formatting: blacken the codebase Augie Fackler <augie@google.com> parents: 42935 diff changeset	2025 )
49764 9be765b82a90 typing: minor tweaks to allow updating to pytype 2022.11.18 Matt Harbison <matt_harbison@yahoo.com> parents: 49341 diff changeset	2026
9be765b82a90 typing: minor tweaks to allow updating to pytype 2022.11.18 Matt Harbison <matt_harbison@yahoo.com> parents: 49341 diff changeset	2027 # HG20 is the case without 2 values to unpack, but is handled above.
9be765b82a90 typing: minor tweaks to allow updating to pytype 2022.11.18 Matt Harbison <matt_harbison@yahoo.com> parents: 49341 diff changeset	2028 # pytype: disable=bad-unpacking
28666 ae53ecc47414 bundle: move writebundle() from changegroup.py to bundle2.py (API) Martin von Zweigbergk <martinvonz@google.com> parents: 27953 diff changeset	2029 header, comp = bundletypes[bundletype]
49764 9be765b82a90 typing: minor tweaks to allow updating to pytype 2022.11.18 Matt Harbison <matt_harbison@yahoo.com> parents: 49341 diff changeset	2030 # pytype: enable=bad-unpacking
9be765b82a90 typing: minor tweaks to allow updating to pytype 2022.11.18 Matt Harbison <matt_harbison@yahoo.com> parents: 49341 diff changeset	2031
30361 f81002f736d7 bundle2: use new compression engine API for compression Gregory Szorc <gregory.szorc@gmail.com> parents: 30177 diff changeset	2032 if comp not in util.compengines.supportedbundletypes:
43077 687b865b95ad formatting: byteify all mercurial/ and hgext/ string literals Augie Fackler <augie@google.com> parents: 43076 diff changeset	2033 raise error.Abort(_(b'unknown stream compression type: %s') % comp)
30361 f81002f736d7 bundle2: use new compression engine API for compression Gregory Szorc <gregory.szorc@gmail.com> parents: 30177 diff changeset	2034 compengine = util.compengines.forbundletype(comp)
43076 2372284d9457 formatting: blacken the codebase Augie Fackler <augie@google.com> parents: 42935 diff changeset	2035
28666 ae53ecc47414 bundle: move writebundle() from changegroup.py to bundle2.py (API) Martin von Zweigbergk <martinvonz@google.com> parents: 27953 diff changeset	2036 def chunkiter():
ae53ecc47414 bundle: move writebundle() from changegroup.py to bundle2.py (API) Martin von Zweigbergk <martinvonz@google.com> parents: 27953 diff changeset	2037 yield header
52669 e627cc25b6f3 pyupgrade: rewrite `yield` statements in a loop to `yield from` Matt Harbison <matt_harbison@yahoo.com> parents: 52668 diff changeset	2038 yield from compengine.compressstream(cg.getchunks(), compopts)
43076 2372284d9457 formatting: blacken the codebase Augie Fackler <augie@google.com> parents: 42935 diff changeset	2039
28666 ae53ecc47414 bundle: move writebundle() from changegroup.py to bundle2.py (API) Martin von Zweigbergk <martinvonz@google.com> parents: 27953 diff changeset	2040 chunkiter = chunkiter()
ae53ecc47414 bundle: move writebundle() from changegroup.py to bundle2.py (API) Martin von Zweigbergk <martinvonz@google.com> parents: 27953 diff changeset	2041
ae53ecc47414 bundle: move writebundle() from changegroup.py to bundle2.py (API) Martin von Zweigbergk <martinvonz@google.com> parents: 27953 diff changeset	2042 # parse the changegroup data, otherwise we will block
ae53ecc47414 bundle: move writebundle() from changegroup.py to bundle2.py (API) Martin von Zweigbergk <martinvonz@google.com> parents: 27953 diff changeset	2043 # in case of sshrepo because we don't know the end of the stream
ae53ecc47414 bundle: move writebundle() from changegroup.py to bundle2.py (API) Martin von Zweigbergk <martinvonz@google.com> parents: 27953 diff changeset	2044 return changegroup.writechunks(ui, chunkiter, filename, vfs=vfs)
ae53ecc47414 bundle: move writebundle() from changegroup.py to bundle2.py (API) Martin von Zweigbergk <martinvonz@google.com> parents: 27953 diff changeset	2045
43076 2372284d9457 formatting: blacken the codebase Augie Fackler <augie@google.com> parents: 42935 diff changeset	2046
33049 d765ad56081f bundle: make combinechangegroupresults() take a bundleoperation Martin von Zweigbergk <martinvonz@google.com> parents: 33048 diff changeset	2047 def combinechangegroupresults(op):
33048 52c7060b707a bundle: move combineresults() from changegroup to bundle2 Martin von Zweigbergk <martinvonz@google.com> parents: 33047 diff changeset	2048 """logic to combine 0 or more addchangegroup results into one"""
43077 687b865b95ad formatting: byteify all mercurial/ and hgext/ string literals Augie Fackler <augie@google.com> parents: 43076 diff changeset	2049 results = [r.get(b'return', 0) for r in op.records[b'changegroup']]
33048 52c7060b707a bundle: move combineresults() from changegroup to bundle2 Martin von Zweigbergk <martinvonz@google.com> parents: 33047 diff changeset	2050 changedheads = 0
52c7060b707a bundle: move combineresults() from changegroup to bundle2 Martin von Zweigbergk <martinvonz@google.com> parents: 33047 diff changeset	2051 result = 1
52c7060b707a bundle: move combineresults() from changegroup to bundle2 Martin von Zweigbergk <martinvonz@google.com> parents: 33047 diff changeset	2052 for ret in results:
52c7060b707a bundle: move combineresults() from changegroup to bundle2 Martin von Zweigbergk <martinvonz@google.com> parents: 33047 diff changeset	2053 # If any changegroup result is 0, return 0
52c7060b707a bundle: move combineresults() from changegroup to bundle2 Martin von Zweigbergk <martinvonz@google.com> parents: 33047 diff changeset	2054 if ret == 0:
52c7060b707a bundle: move combineresults() from changegroup to bundle2 Martin von Zweigbergk <martinvonz@google.com> parents: 33047 diff changeset	2055 result = 0
52c7060b707a bundle: move combineresults() from changegroup to bundle2 Martin von Zweigbergk <martinvonz@google.com> parents: 33047 diff changeset	2056 break
52c7060b707a bundle: move combineresults() from changegroup to bundle2 Martin von Zweigbergk <martinvonz@google.com> parents: 33047 diff changeset	2057 if ret < -1:
52c7060b707a bundle: move combineresults() from changegroup to bundle2 Martin von Zweigbergk <martinvonz@google.com> parents: 33047 diff changeset	2058 changedheads += ret + 1
52c7060b707a bundle: move combineresults() from changegroup to bundle2 Martin von Zweigbergk <martinvonz@google.com> parents: 33047 diff changeset	2059 elif ret > 1:
52c7060b707a bundle: move combineresults() from changegroup to bundle2 Martin von Zweigbergk <martinvonz@google.com> parents: 33047 diff changeset	2060 changedheads += ret - 1
52c7060b707a bundle: move combineresults() from changegroup to bundle2 Martin von Zweigbergk <martinvonz@google.com> parents: 33047 diff changeset	2061 if changedheads > 0:
52c7060b707a bundle: move combineresults() from changegroup to bundle2 Martin von Zweigbergk <martinvonz@google.com> parents: 33047 diff changeset	2062 result = 1 + changedheads
52c7060b707a bundle: move combineresults() from changegroup to bundle2 Martin von Zweigbergk <martinvonz@google.com> parents: 33047 diff changeset	2063 elif changedheads < 0:
52c7060b707a bundle: move combineresults() from changegroup to bundle2 Martin von Zweigbergk <martinvonz@google.com> parents: 33047 diff changeset	2064 result = -1 + changedheads
52c7060b707a bundle: move combineresults() from changegroup to bundle2 Martin von Zweigbergk <martinvonz@google.com> parents: 33047 diff changeset	2065 return result
52c7060b707a bundle: move combineresults() from changegroup to bundle2 Martin von Zweigbergk <martinvonz@google.com> parents: 33047 diff changeset	2066
43076 2372284d9457 formatting: blacken the codebase Augie Fackler <augie@google.com> parents: 42935 diff changeset	2067
2372284d9457 formatting: blacken the codebase Augie Fackler <augie@google.com> parents: 42935 diff changeset	2068 @parthandler(
43131 c17a63eb5d4c sidedata: apply basic but tight security around exchange Pierre-Yves David <pierre-yves.david@octobus.net> parents: 43117 diff changeset	2069 b'changegroup',
c17a63eb5d4c sidedata: apply basic but tight security around exchange Pierre-Yves David <pierre-yves.david@octobus.net> parents: 43117 diff changeset	2070 (
c17a63eb5d4c sidedata: apply basic but tight security around exchange Pierre-Yves David <pierre-yves.david@octobus.net> parents: 43117 diff changeset	2071 b'version',
c17a63eb5d4c sidedata: apply basic but tight security around exchange Pierre-Yves David <pierre-yves.david@octobus.net> parents: 43117 diff changeset	2072 b'nbchanges',
c17a63eb5d4c sidedata: apply basic but tight security around exchange Pierre-Yves David <pierre-yves.david@octobus.net> parents: 43117 diff changeset	2073 b'exp-sidedata',
46726 bc2519513ae0 sidedata-exchange: add `wanted_sidedata` and `sidedata_computers` to repos Rapha?l Gom?s <rgomes@octobus.net> parents: 46672 diff changeset	2074 b'exp-wanted-sidedata',
43131 c17a63eb5d4c sidedata: apply basic but tight security around exchange Pierre-Yves David <pierre-yves.david@octobus.net> parents: 43117 diff changeset	2075 b'treemanifest',
c17a63eb5d4c sidedata: apply basic but tight security around exchange Pierre-Yves David <pierre-yves.david@octobus.net> parents: 43117 diff changeset	2076 b'targetphase',
c17a63eb5d4c sidedata: apply basic but tight security around exchange Pierre-Yves David <pierre-yves.david@octobus.net> parents: 43117 diff changeset	2077 ),
43076 2372284d9457 formatting: blacken the codebase Augie Fackler <augie@google.com> parents: 42935 diff changeset	2078 )
20998 93a3c5b58635 bundle2: use reply part to return result of addchangegroup Pierre-Yves David <pierre-yves.david@fb.com> parents: 20997 diff changeset	2079 def handlechangegroup(op, inpart):
46166 c511fef30290 bundle2: drop some outdated comment Pierre-Yves David <pierre-yves.david@octobus.net> parents: 46165 diff changeset	2080 """apply a changegroup part on the repo"""
39716 b10d145837bc localrepo: extract resolving of opener options to standalone functions Gregory Szorc <gregory.szorc@gmail.com> parents: 38823 diff changeset	2081 from . import localrepo
b10d145837bc localrepo: extract resolving of opener options to standalone functions Gregory Szorc <gregory.szorc@gmail.com> parents: 38823 diff changeset	2082
32948 af31d531dda0 changegroup: let callers pass in transaction to apply() (API) Martin von Zweigbergk <martinvonz@google.com> parents: 32910 diff changeset	2083 tr = op.gettransaction()
43077 687b865b95ad formatting: byteify all mercurial/ and hgext/ string literals Augie Fackler <augie@google.com> parents: 43076 diff changeset	2084 unpackerversion = inpart.params.get(b'version', b'01')
23170 02e8f9b60052 bundle2: support a "version" argument in `changegroup` part Pierre-Yves David <pierre-yves.david@fb.com> parents: 23169 diff changeset	2085 # We should raise an appropriate exception here
27751 a40e2f7fe49d changegroup: hide packermap behind methods Martin von Zweigbergk <martinvonz@google.com> parents: 27734 diff changeset	2086 cg = changegroup.getunbundler(unpackerversion, inpart, None)
23001 4df9b5e62f70 bundle2: add a comment about addchangegroup source and url Pierre-Yves David <pierre-yves.david@fb.com> parents: 22953 diff changeset	2087 # the source and url passed here are overwritten by the one contained in
4df9b5e62f70 bundle2: add a comment about addchangegroup source and url Pierre-Yves David <pierre-yves.david@fb.com> parents: 22953 diff changeset	2088 # the transaction.hookargs argument. So 'bundle2' is a placeholder
25518 ca656f3dffd7 bundle2: provide number of changesets information to 'addchangegroup' Pierre-Yves David <pierre-yves.david@fb.com> parents: 25507 diff changeset	2089 nbchangesets = None
43077 687b865b95ad formatting: byteify all mercurial/ and hgext/ string literals Augie Fackler <augie@google.com> parents: 43076 diff changeset	2090 if b'nbchanges' in inpart.params:
687b865b95ad formatting: byteify all mercurial/ and hgext/ string literals Augie Fackler <augie@google.com> parents: 43076 diff changeset	2091 nbchangesets = int(inpart.params.get(b'nbchanges'))
45558 10284ce3d5ed scmutil: introduce function to check whether repo uses treemanifest or not Pulkit Goyal <7895pulkit@gmail.com> parents: 45392 diff changeset	2092 if b'treemanifest' in inpart.params and not scmutil.istreemanifest(op.repo):
27734 5c0fd878779c treemanifests: set bundle2 part parameter indicating treemanifest Martin von Zweigbergk <martinvonz@google.com> parents: 26829 diff changeset	2093 if len(op.repo.changelog) != 0:
43076 2372284d9457 formatting: blacken the codebase Augie Fackler <augie@google.com> parents: 42935 diff changeset	2094 raise error.Abort(
2372284d9457 formatting: blacken the codebase Augie Fackler <augie@google.com> parents: 42935 diff changeset	2095 _(
43077 687b865b95ad formatting: byteify all mercurial/ and hgext/ string literals Augie Fackler <augie@google.com> parents: 43076 diff changeset	2096 b"bundle contains tree manifests, but local repo is "
687b865b95ad formatting: byteify all mercurial/ and hgext/ string literals Augie Fackler <augie@google.com> parents: 43076 diff changeset	2097 b"non-empty and does not use tree manifests"
43076 2372284d9457 formatting: blacken the codebase Augie Fackler <augie@google.com> parents: 42935 diff changeset	2098 )
2372284d9457 formatting: blacken the codebase Augie Fackler <augie@google.com> parents: 42935 diff changeset	2099 )
45392 77b8588dd84e requirements: introduce new requirements related module Pulkit Goyal <7895pulkit@gmail.com> parents: 45391 diff changeset	2100 op.repo.requirements.add(requirements.TREEMANIFEST_REQUIREMENT)
39716 b10d145837bc localrepo: extract resolving of opener options to standalone functions Gregory Szorc <gregory.szorc@gmail.com> parents: 38823 diff changeset	2101 op.repo.svfs.options = localrepo.resolvestorevfsoptions(
43076 2372284d9457 formatting: blacken the codebase Augie Fackler <augie@google.com> parents: 42935 diff changeset	2102 op.repo.ui, op.repo.requirements, op.repo.features
2372284d9457 formatting: blacken the codebase Augie Fackler <augie@google.com> parents: 42935 diff changeset	2103 )
45106 a03c177a4679 scmutil: add writereporequirements() and route requires writing through it Pulkit Goyal <7895pulkit@gmail.com> parents: 44406 diff changeset	2104 scmutil.writereporequirements(op.repo)
43131 c17a63eb5d4c sidedata: apply basic but tight security around exchange Pierre-Yves David <pierre-yves.david@octobus.net> parents: 43117 diff changeset	2105
33407 39d4e5a66f5a bundle2: support the 'targetphase' parameter for the changegroup part Boris Feld <boris.feld@octobus.net> parents: 33252 diff changeset	2106 extrakwargs = {}
43077 687b865b95ad formatting: byteify all mercurial/ and hgext/ string literals Augie Fackler <augie@google.com> parents: 43076 diff changeset	2107 targetphase = inpart.params.get(b'targetphase')
33407 39d4e5a66f5a bundle2: support the 'targetphase' parameter for the changegroup part Boris Feld <boris.feld@octobus.net> parents: 33252 diff changeset	2108 if targetphase is not None:
43554 9f70512ae2cf cleanup: remove pointless r-prefixes on single-quoted strings Augie Fackler <augie@google.com> parents: 43543 diff changeset	2109 extrakwargs['targetphase'] = int(targetphase)
46726 bc2519513ae0 sidedata-exchange: add `wanted_sidedata` and `sidedata_computers` to repos Rapha?l Gom?s <rgomes@octobus.net> parents: 46672 diff changeset	2110
bc2519513ae0 sidedata-exchange: add `wanted_sidedata` and `sidedata_computers` to repos Rapha?l Gom?s <rgomes@octobus.net> parents: 46672 diff changeset	2111 remote_sidedata = inpart.params.get(b'exp-wanted-sidedata')
bc2519513ae0 sidedata-exchange: add `wanted_sidedata` and `sidedata_computers` to repos Rapha?l Gom?s <rgomes@octobus.net> parents: 46672 diff changeset	2112 extrakwargs['sidedata_categories'] = read_wanted_sidedata(remote_sidedata)
bc2519513ae0 sidedata-exchange: add `wanted_sidedata` and `sidedata_computers` to repos Rapha?l Gom?s <rgomes@octobus.net> parents: 46672 diff changeset	2113
43076 2372284d9457 formatting: blacken the codebase Augie Fackler <augie@google.com> parents: 42935 diff changeset	2114 ret = _processchangegroup(
2372284d9457 formatting: blacken the codebase Augie Fackler <augie@google.com> parents: 42935 diff changeset	2115 op,
2372284d9457 formatting: blacken the codebase Augie Fackler <augie@google.com> parents: 42935 diff changeset	2116 cg,
2372284d9457 formatting: blacken the codebase Augie Fackler <augie@google.com> parents: 42935 diff changeset	2117 tr,
46571 83ac7c91e172 bundle2: pass the operation source down to the changegroup Rapha?l Gom?s <rgomes@octobus.net> parents: 46547 diff changeset	2118 op.source,
43077 687b865b95ad formatting: byteify all mercurial/ and hgext/ string literals Augie Fackler <augie@google.com> parents: 43076 diff changeset	2119 b'bundle2',
43076 2372284d9457 formatting: blacken the codebase Augie Fackler <augie@google.com> parents: 42935 diff changeset	2120 expectedtotal=nbchangesets,
51900 607e94e01851 format: add many "missing" comma Matt Harbison <matt_harbison@yahoo.com> parents: 51762 diff changeset	2121 **extrakwargs,
43076 2372284d9457 formatting: blacken the codebase Augie Fackler <augie@google.com> parents: 42935 diff changeset	2122 )
20998 93a3c5b58635 bundle2: use reply part to return result of addchangegroup Pierre-Yves David <pierre-yves.david@fb.com> parents: 20997 diff changeset	2123 if op.reply is not None:
23139 e53f6b72a0e4 spelling: fixes from proofreading of spell checker issues Mads Kiilerich <madski@unity3d.com> parents: 23067 diff changeset	2124 # This is definitely not the final form of this
20998 93a3c5b58635 bundle2: use reply part to return result of addchangegroup Pierre-Yves David <pierre-yves.david@fb.com> parents: 20997 diff changeset	2125 # return. But one need to start somewhere.
43077 687b865b95ad formatting: byteify all mercurial/ and hgext/ string literals Augie Fackler <augie@google.com> parents: 43076 diff changeset	2126 part = op.reply.newpart(b'reply:changegroup', mandatory=False)
33701 24b7c75c6441 bundle2: use bytestr() instead of str() to convert part id to bytes Augie Fackler <augie@google.com> parents: 33700 diff changeset	2127 part.addparam(
43077 687b865b95ad formatting: byteify all mercurial/ and hgext/ string literals Augie Fackler <augie@google.com> parents: 43076 diff changeset	2128 b'in-reply-to', pycompat.bytestr(inpart.id), mandatory=False
43076 2372284d9457 formatting: blacken the codebase Augie Fackler <augie@google.com> parents: 42935 diff changeset	2129 )
43077 687b865b95ad formatting: byteify all mercurial/ and hgext/ string literals Augie Fackler <augie@google.com> parents: 43076 diff changeset	2130 part.addparam(b'return', b'%i' % ret, mandatory=False)
21019 3dc09f831a2e bundle2: lazy unbundle of part payload Pierre-Yves David <pierre-yves.david@fb.com> parents: 21016 diff changeset	2131 assert not inpart.read()
20950 c7ceae0faf69 bundle2: first crude version of bundling changeset with bundle2 Pierre-Yves David <pierre-yves.david@fb.com> parents: 20949 diff changeset	2132
43076 2372284d9457 formatting: blacken the codebase Augie Fackler <augie@google.com> parents: 42935 diff changeset	2133
2372284d9457 formatting: blacken the codebase Augie Fackler <augie@google.com> parents: 42935 diff changeset	2134 _remotechangegroupparams = tuple(
43077 687b865b95ad formatting: byteify all mercurial/ and hgext/ string literals Augie Fackler <augie@google.com> parents: 43076 diff changeset	2135 [b'url', b'size', b'digests']
687b865b95ad formatting: byteify all mercurial/ and hgext/ string literals Augie Fackler <augie@google.com> parents: 43076 diff changeset	2136 + [b'digest:%s' % k for k in util.DIGESTS.keys()]
43076 2372284d9457 formatting: blacken the codebase Augie Fackler <augie@google.com> parents: 42935 diff changeset	2137 )
2372284d9457 formatting: blacken the codebase Augie Fackler <augie@google.com> parents: 42935 diff changeset	2138
2372284d9457 formatting: blacken the codebase Augie Fackler <augie@google.com> parents: 42935 diff changeset	2139
43077 687b865b95ad formatting: byteify all mercurial/ and hgext/ string literals Augie Fackler <augie@google.com> parents: 43076 diff changeset	2140 @parthandler(b'remote-changegroup', _remotechangegroupparams)
23029 149fc8a44184 bundle2: client side support for a part to import external bundles Mike Hommey <mh@glandium.org> parents: 23011 diff changeset	2141 def handleremotechangegroup(op, inpart):
149fc8a44184 bundle2: client side support for a part to import external bundles Mike Hommey <mh@glandium.org> parents: 23011 diff changeset	2142 """apply a bundle10 on the repo, given an url and validation information
149fc8a44184 bundle2: client side support for a part to import external bundles Mike Hommey <mh@glandium.org> parents: 23011 diff changeset	2143
149fc8a44184 bundle2: client side support for a part to import external bundles Mike Hommey <mh@glandium.org> parents: 23011 diff changeset	2144 All the information about the remote bundle to import are given as
149fc8a44184 bundle2: client side support for a part to import external bundles Mike Hommey <mh@glandium.org> parents: 23011 diff changeset	2145 parameters. The parameters include:
149fc8a44184 bundle2: client side support for a part to import external bundles Mike Hommey <mh@glandium.org> parents: 23011 diff changeset	2146 - url: the url to the bundle10.
149fc8a44184 bundle2: client side support for a part to import external bundles Mike Hommey <mh@glandium.org> parents: 23011 diff changeset	2147 - size: the bundle10 file size. It is used to validate what was
149fc8a44184 bundle2: client side support for a part to import external bundles Mike Hommey <mh@glandium.org> parents: 23011 diff changeset	2148 retrieved by the client matches the server knowledge about the bundle.
149fc8a44184 bundle2: client side support for a part to import external bundles Mike Hommey <mh@glandium.org> parents: 23011 diff changeset	2149 - digests: a space separated list of the digest types provided as
149fc8a44184 bundle2: client side support for a part to import external bundles Mike Hommey <mh@glandium.org> parents: 23011 diff changeset	2150 parameters.
149fc8a44184 bundle2: client side support for a part to import external bundles Mike Hommey <mh@glandium.org> parents: 23011 diff changeset	2151 - digest:<digest-type>: the hexadecimal representation of the digest with
149fc8a44184 bundle2: client side support for a part to import external bundles Mike Hommey <mh@glandium.org> parents: 23011 diff changeset	2152 that name. Like the size, it is used to validate what was retrieved by
149fc8a44184 bundle2: client side support for a part to import external bundles Mike Hommey <mh@glandium.org> parents: 23011 diff changeset	2153 the client matches what the server knows about the bundle.
149fc8a44184 bundle2: client side support for a part to import external bundles Mike Hommey <mh@glandium.org> parents: 23011 diff changeset	2154
149fc8a44184 bundle2: client side support for a part to import external bundles Mike Hommey <mh@glandium.org> parents: 23011 diff changeset	2155 When multiple digest types are given, all of them are checked.
149fc8a44184 bundle2: client side support for a part to import external bundles Mike Hommey <mh@glandium.org> parents: 23011 diff changeset	2156 """
149fc8a44184 bundle2: client side support for a part to import external bundles Mike Hommey <mh@glandium.org> parents: 23011 diff changeset	2157 try:
43077 687b865b95ad formatting: byteify all mercurial/ and hgext/ string literals Augie Fackler <augie@google.com> parents: 43076 diff changeset	2158 raw_url = inpart.params[b'url']
23029 149fc8a44184 bundle2: client side support for a part to import external bundles Mike Hommey <mh@glandium.org> parents: 23011 diff changeset	2159 except KeyError:
43077 687b865b95ad formatting: byteify all mercurial/ and hgext/ string literals Augie Fackler <augie@google.com> parents: 43076 diff changeset	2160 raise error.Abort(_(b'remote-changegroup: missing "%s" param') % b'url')
46907 ffd3e823a7e5 urlutil: extract `url` related code from `util` into the new module Pierre-Yves David <pierre-yves.david@octobus.net> parents: 46793 diff changeset	2161 parsed_url = urlutil.url(raw_url)
43077 687b865b95ad formatting: byteify all mercurial/ and hgext/ string literals Augie Fackler <augie@google.com> parents: 43076 diff changeset	2162 if parsed_url.scheme not in capabilities[b'remote-changegroup']:
43076 2372284d9457 formatting: blacken the codebase Augie Fackler <augie@google.com> parents: 42935 diff changeset	2163 raise error.Abort(
43077 687b865b95ad formatting: byteify all mercurial/ and hgext/ string literals Augie Fackler <augie@google.com> parents: 43076 diff changeset	2164 _(b'remote-changegroup does not support %s urls')
687b865b95ad formatting: byteify all mercurial/ and hgext/ string literals Augie Fackler <augie@google.com> parents: 43076 diff changeset	2165 % parsed_url.scheme
43076 2372284d9457 formatting: blacken the codebase Augie Fackler <augie@google.com> parents: 42935 diff changeset	2166 )
23029 149fc8a44184 bundle2: client side support for a part to import external bundles Mike Hommey <mh@glandium.org> parents: 23011 diff changeset	2167
149fc8a44184 bundle2: client side support for a part to import external bundles Mike Hommey <mh@glandium.org> parents: 23011 diff changeset	2168 try:
43077 687b865b95ad formatting: byteify all mercurial/ and hgext/ string literals Augie Fackler <augie@google.com> parents: 43076 diff changeset	2169 size = int(inpart.params[b'size'])
23029 149fc8a44184 bundle2: client side support for a part to import external bundles Mike Hommey <mh@glandium.org> parents: 23011 diff changeset	2170 except ValueError:
43076 2372284d9457 formatting: blacken the codebase Augie Fackler <augie@google.com> parents: 42935 diff changeset	2171 raise error.Abort(
43077 687b865b95ad formatting: byteify all mercurial/ and hgext/ string literals Augie Fackler <augie@google.com> parents: 43076 diff changeset	2172 _(b'remote-changegroup: invalid value for param "%s"') % b'size'
43076 2372284d9457 formatting: blacken the codebase Augie Fackler <augie@google.com> parents: 42935 diff changeset	2173 )
23029 149fc8a44184 bundle2: client side support for a part to import external bundles Mike Hommey <mh@glandium.org> parents: 23011 diff changeset	2174 except KeyError:
43077 687b865b95ad formatting: byteify all mercurial/ and hgext/ string literals Augie Fackler <augie@google.com> parents: 43076 diff changeset	2175 raise error.Abort(
687b865b95ad formatting: byteify all mercurial/ and hgext/ string literals Augie Fackler <augie@google.com> parents: 43076 diff changeset	2176 _(b'remote-changegroup: missing "%s" param') % b'size'
687b865b95ad formatting: byteify all mercurial/ and hgext/ string literals Augie Fackler <augie@google.com> parents: 43076 diff changeset	2177 )
23029 149fc8a44184 bundle2: client side support for a part to import external bundles Mike Hommey <mh@glandium.org> parents: 23011 diff changeset	2178
149fc8a44184 bundle2: client side support for a part to import external bundles Mike Hommey <mh@glandium.org> parents: 23011 diff changeset	2179 digests = {}
43077 687b865b95ad formatting: byteify all mercurial/ and hgext/ string literals Augie Fackler <augie@google.com> parents: 43076 diff changeset	2180 for typ in inpart.params.get(b'digests', b'').split():
687b865b95ad formatting: byteify all mercurial/ and hgext/ string literals Augie Fackler <augie@google.com> parents: 43076 diff changeset	2181 param = b'digest:%s' % typ
23029 149fc8a44184 bundle2: client side support for a part to import external bundles Mike Hommey <mh@glandium.org> parents: 23011 diff changeset	2182 try:
149fc8a44184 bundle2: client side support for a part to import external bundles Mike Hommey <mh@glandium.org> parents: 23011 diff changeset	2183 value = inpart.params[param]
149fc8a44184 bundle2: client side support for a part to import external bundles Mike Hommey <mh@glandium.org> parents: 23011 diff changeset	2184 except KeyError:
43076 2372284d9457 formatting: blacken the codebase Augie Fackler <augie@google.com> parents: 42935 diff changeset	2185 raise error.Abort(
43077 687b865b95ad formatting: byteify all mercurial/ and hgext/ string literals Augie Fackler <augie@google.com> parents: 43076 diff changeset	2186 _(b'remote-changegroup: missing "%s" param') % param
43076 2372284d9457 formatting: blacken the codebase Augie Fackler <augie@google.com> parents: 42935 diff changeset	2187 )
23029 149fc8a44184 bundle2: client side support for a part to import external bundles Mike Hommey <mh@glandium.org> parents: 23011 diff changeset	2188 digests[typ] = value
149fc8a44184 bundle2: client side support for a part to import external bundles Mike Hommey <mh@glandium.org> parents: 23011 diff changeset	2189
149fc8a44184 bundle2: client side support for a part to import external bundles Mike Hommey <mh@glandium.org> parents: 23011 diff changeset	2190 real_part = util.digestchecker(url.open(op.ui, raw_url), size, digests)
149fc8a44184 bundle2: client side support for a part to import external bundles Mike Hommey <mh@glandium.org> parents: 23011 diff changeset	2191
32948 af31d531dda0 changegroup: let callers pass in transaction to apply() (API) Martin von Zweigbergk <martinvonz@google.com> parents: 32910 diff changeset	2192 tr = op.gettransaction()
25919 8221fefaea08 bundle2: use absolute_import Gregory Szorc <gregory.szorc@gmail.com> parents: 25660 diff changeset	2193 from . import exchange
43076 2372284d9457 formatting: blacken the codebase Augie Fackler <augie@google.com> parents: 42935 diff changeset	2194
23029 149fc8a44184 bundle2: client side support for a part to import external bundles Mike Hommey <mh@glandium.org> parents: 23011 diff changeset	2195 cg = exchange.readbundle(op.repo.ui, real_part, raw_url)
149fc8a44184 bundle2: client side support for a part to import external bundles Mike Hommey <mh@glandium.org> parents: 23011 diff changeset	2196 if not isinstance(cg, changegroup.cg1unpacker):
43076 2372284d9457 formatting: blacken the codebase Augie Fackler <augie@google.com> parents: 42935 diff changeset	2197 raise error.Abort(
46907 ffd3e823a7e5 urlutil: extract `url` related code from `util` into the new module Pierre-Yves David <pierre-yves.david@octobus.net> parents: 46793 diff changeset	2198 _(b'%s: not a bundle version 1.0') % urlutil.hidepassword(raw_url)
43076 2372284d9457 formatting: blacken the codebase Augie Fackler <augie@google.com> parents: 42935 diff changeset	2199 )
46571 83ac7c91e172 bundle2: pass the operation source down to the changegroup Rapha?l Gom?s <rgomes@octobus.net> parents: 46547 diff changeset	2200 ret = _processchangegroup(op, cg, tr, op.source, b'bundle2')
23029 149fc8a44184 bundle2: client side support for a part to import external bundles Mike Hommey <mh@glandium.org> parents: 23011 diff changeset	2201 if op.reply is not None:
23139 e53f6b72a0e4 spelling: fixes from proofreading of spell checker issues Mads Kiilerich <madski@unity3d.com> parents: 23067 diff changeset	2202 # This is definitely not the final form of this
23029 149fc8a44184 bundle2: client side support for a part to import external bundles Mike Hommey <mh@glandium.org> parents: 23011 diff changeset	2203 # return. But one need to start somewhere.
43077 687b865b95ad formatting: byteify all mercurial/ and hgext/ string literals Augie Fackler <augie@google.com> parents: 43076 diff changeset	2204 part = op.reply.newpart(b'reply:changegroup')
33701 24b7c75c6441 bundle2: use bytestr() instead of str() to convert part id to bytes Augie Fackler <augie@google.com> parents: 33700 diff changeset	2205 part.addparam(
43077 687b865b95ad formatting: byteify all mercurial/ and hgext/ string literals Augie Fackler <augie@google.com> parents: 43076 diff changeset	2206 b'in-reply-to', pycompat.bytestr(inpart.id), mandatory=False
43076 2372284d9457 formatting: blacken the codebase Augie Fackler <augie@google.com> parents: 42935 diff changeset	2207 )
43077 687b865b95ad formatting: byteify all mercurial/ and hgext/ string literals Augie Fackler <augie@google.com> parents: 43076 diff changeset	2208 part.addparam(b'return', b'%i' % ret, mandatory=False)
23029 149fc8a44184 bundle2: client side support for a part to import external bundles Mike Hommey <mh@glandium.org> parents: 23011 diff changeset	2209 try:
149fc8a44184 bundle2: client side support for a part to import external bundles Mike Hommey <mh@glandium.org> parents: 23011 diff changeset	2210 real_part.validate()
26587 56b2bcea2529 error: get Abort from 'error' instead of 'util' Pierre-Yves David <pierre-yves.david@fb.com> parents: 26565 diff changeset	2211 except error.Abort as e:
43076 2372284d9457 formatting: blacken the codebase Augie Fackler <augie@google.com> parents: 42935 diff changeset	2212 raise error.Abort(
43077 687b865b95ad formatting: byteify all mercurial/ and hgext/ string literals Augie Fackler <augie@google.com> parents: 43076 diff changeset	2213 _(b'bundle at %s is corrupted:\n%s')
46907 ffd3e823a7e5 urlutil: extract `url` related code from `util` into the new module Pierre-Yves David <pierre-yves.david@octobus.net> parents: 46793 diff changeset	2214 % (urlutil.hidepassword(raw_url), e.message)
43076 2372284d9457 formatting: blacken the codebase Augie Fackler <augie@google.com> parents: 42935 diff changeset	2215 )
23029 149fc8a44184 bundle2: client side support for a part to import external bundles Mike Hommey <mh@glandium.org> parents: 23011 diff changeset	2216 assert not inpart.read()
149fc8a44184 bundle2: client side support for a part to import external bundles Mike Hommey <mh@glandium.org> parents: 23011 diff changeset	2217
43076 2372284d9457 formatting: blacken the codebase Augie Fackler <augie@google.com> parents: 42935 diff changeset	2218
43077 687b865b95ad formatting: byteify all mercurial/ and hgext/ string literals Augie Fackler <augie@google.com> parents: 43076 diff changeset	2219 @parthandler(b'reply:changegroup', (b'return', b'in-reply-to'))
22548 8a1ae133770a bundle2: rename functions that have the same name Mike Hommey <mh@glandium.org> parents: 22390 diff changeset	2220 def handlereplychangegroup(op, inpart):
43077 687b865b95ad formatting: byteify all mercurial/ and hgext/ string literals Augie Fackler <augie@google.com> parents: 43076 diff changeset	2221 ret = int(inpart.params[b'return'])
687b865b95ad formatting: byteify all mercurial/ and hgext/ string literals Augie Fackler <augie@google.com> parents: 43076 diff changeset	2222 replyto = int(inpart.params[b'in-reply-to'])
687b865b95ad formatting: byteify all mercurial/ and hgext/ string literals Augie Fackler <augie@google.com> parents: 43076 diff changeset	2223 op.records.add(b'changegroup', {b'return': ret}, replyto)
687b865b95ad formatting: byteify all mercurial/ and hgext/ string literals Augie Fackler <augie@google.com> parents: 43076 diff changeset	2224
687b865b95ad formatting: byteify all mercurial/ and hgext/ string literals Augie Fackler <augie@google.com> parents: 43076 diff changeset	2225
687b865b95ad formatting: byteify all mercurial/ and hgext/ string literals Augie Fackler <augie@google.com> parents: 43076 diff changeset	2226 @parthandler(b'check:bookmarks')
35267 dbf868623daf bookmark: add a 'check:bookmarks' bundle2 part Boris Feld <boris.feld@octobus.net> parents: 35141 diff changeset	2227 def handlecheckbookmarks(op, inpart):
dbf868623daf bookmark: add a 'check:bookmarks' bundle2 part Boris Feld <boris.feld@octobus.net> parents: 35141 diff changeset	2228 """check location of bookmarks
dbf868623daf bookmark: add a 'check:bookmarks' bundle2 part Boris Feld <boris.feld@octobus.net> parents: 35141 diff changeset	2229
dbf868623daf bookmark: add a 'check:bookmarks' bundle2 part Boris Feld <boris.feld@octobus.net> parents: 35141 diff changeset	2230 This part is to be used to detect push race regarding bookmark, it
dbf868623daf bookmark: add a 'check:bookmarks' bundle2 part Boris Feld <boris.feld@octobus.net> parents: 35141 diff changeset	2231 contains binary encoded (bookmark, node) tuple. If the local state does
dbf868623daf bookmark: add a 'check:bookmarks' bundle2 part Boris Feld <boris.feld@octobus.net> parents: 35141 diff changeset	2232 not marks the one in the part, a PushRaced exception is raised
dbf868623daf bookmark: add a 'check:bookmarks' bundle2 part Boris Feld <boris.feld@octobus.net> parents: 35141 diff changeset	2233 """
46793 6266d19556ad node: introduce nodeconstants class Joerg Sonnenberger <joerg@bec.de> parents: 46726 diff changeset	2234 bookdata = bookmarks.binarydecode(op.repo, inpart)
35267 dbf868623daf bookmark: add a 'check:bookmarks' bundle2 part Boris Feld <boris.feld@octobus.net> parents: 35141 diff changeset	2235
43076 2372284d9457 formatting: blacken the codebase Augie Fackler <augie@google.com> parents: 42935 diff changeset	2236 msgstandard = (
43077 687b865b95ad formatting: byteify all mercurial/ and hgext/ string literals Augie Fackler <augie@google.com> parents: 43076 diff changeset	2237 b'remote repository changed while pushing - please try again '
687b865b95ad formatting: byteify all mercurial/ and hgext/ string literals Augie Fackler <augie@google.com> parents: 43076 diff changeset	2238 b'(bookmark "%s" move from %s to %s)'
43076 2372284d9457 formatting: blacken the codebase Augie Fackler <augie@google.com> parents: 42935 diff changeset	2239 )
2372284d9457 formatting: blacken the codebase Augie Fackler <augie@google.com> parents: 42935 diff changeset	2240 msgmissing = (
43077 687b865b95ad formatting: byteify all mercurial/ and hgext/ string literals Augie Fackler <augie@google.com> parents: 43076 diff changeset	2241 b'remote repository changed while pushing - please try again '
687b865b95ad formatting: byteify all mercurial/ and hgext/ string literals Augie Fackler <augie@google.com> parents: 43076 diff changeset	2242 b'(bookmark "%s" is missing, expected %s)'
43076 2372284d9457 formatting: blacken the codebase Augie Fackler <augie@google.com> parents: 42935 diff changeset	2243 )
2372284d9457 formatting: blacken the codebase Augie Fackler <augie@google.com> parents: 42935 diff changeset	2244 msgexist = (
43077 687b865b95ad formatting: byteify all mercurial/ and hgext/ string literals Augie Fackler <augie@google.com> parents: 43076 diff changeset	2245 b'remote repository changed while pushing - please try again '
687b865b95ad formatting: byteify all mercurial/ and hgext/ string literals Augie Fackler <augie@google.com> parents: 43076 diff changeset	2246 b'(bookmark "%s" set on %s, expected missing)'
43076 2372284d9457 formatting: blacken the codebase Augie Fackler <augie@google.com> parents: 42935 diff changeset	2247 )
35267 dbf868623daf bookmark: add a 'check:bookmarks' bundle2 part Boris Feld <boris.feld@octobus.net> parents: 35141 diff changeset	2248 for book, node in bookdata:
dbf868623daf bookmark: add a 'check:bookmarks' bundle2 part Boris Feld <boris.feld@octobus.net> parents: 35141 diff changeset	2249 currentnode = op.repo._bookmarks.get(book)
dbf868623daf bookmark: add a 'check:bookmarks' bundle2 part Boris Feld <boris.feld@octobus.net> parents: 35141 diff changeset	2250 if currentnode != node:
dbf868623daf bookmark: add a 'check:bookmarks' bundle2 part Boris Feld <boris.feld@octobus.net> parents: 35141 diff changeset	2251 if node is None:
46114 59fa3890d40a node: import symbols explicitly Joerg Sonnenberger <joerg@bec.de> parents: 45957 diff changeset	2252 finalmsg = msgexist % (book, short(currentnode))
35267 dbf868623daf bookmark: add a 'check:bookmarks' bundle2 part Boris Feld <boris.feld@octobus.net> parents: 35141 diff changeset	2253 elif currentnode is None:
46114 59fa3890d40a node: import symbols explicitly Joerg Sonnenberger <joerg@bec.de> parents: 45957 diff changeset	2254 finalmsg = msgmissing % (book, short(node))
35267 dbf868623daf bookmark: add a 'check:bookmarks' bundle2 part Boris Feld <boris.feld@octobus.net> parents: 35141 diff changeset	2255 else:
43076 2372284d9457 formatting: blacken the codebase Augie Fackler <augie@google.com> parents: 42935 diff changeset	2256 finalmsg = msgstandard % (
2372284d9457 formatting: blacken the codebase Augie Fackler <augie@google.com> parents: 42935 diff changeset	2257 book,
46114 59fa3890d40a node: import symbols explicitly Joerg Sonnenberger <joerg@bec.de> parents: 45957 diff changeset	2258 short(node),
59fa3890d40a node: import symbols explicitly Joerg Sonnenberger <joerg@bec.de> parents: 45957 diff changeset	2259 short(currentnode),
43076 2372284d9457 formatting: blacken the codebase Augie Fackler <augie@google.com> parents: 42935 diff changeset	2260 )
35267 dbf868623daf bookmark: add a 'check:bookmarks' bundle2 part Boris Feld <boris.feld@octobus.net> parents: 35141 diff changeset	2261 raise error.PushRaced(finalmsg)
dbf868623daf bookmark: add a 'check:bookmarks' bundle2 part Boris Feld <boris.feld@octobus.net> parents: 35141 diff changeset	2262
43076 2372284d9457 formatting: blacken the codebase Augie Fackler <augie@google.com> parents: 42935 diff changeset	2263
43077 687b865b95ad formatting: byteify all mercurial/ and hgext/ string literals Augie Fackler <augie@google.com> parents: 43076 diff changeset	2264 @parthandler(b'check:heads')
22548 8a1ae133770a bundle2: rename functions that have the same name Mike Hommey <mh@glandium.org> parents: 22390 diff changeset	2265 def handlecheckheads(op, inpart):
21060 0bea9db7543b bundle2: add a "check:heads" handler Pierre-Yves David <pierre-yves.david@fb.com> parents: 21024 diff changeset	2266 """check that head of the repo did not change
0bea9db7543b bundle2: add a "check:heads" handler Pierre-Yves David <pierre-yves.david@fb.com> parents: 21024 diff changeset	2267
0bea9db7543b bundle2: add a "check:heads" handler Pierre-Yves David <pierre-yves.david@fb.com> parents: 21024 diff changeset	2268 This is used to detect a push race when using unbundle.
0bea9db7543b bundle2: add a "check:heads" handler Pierre-Yves David <pierre-yves.david@fb.com> parents: 21024 diff changeset	2269 This replaces the "heads" argument of unbundle."""
0bea9db7543b bundle2: add a "check:heads" handler Pierre-Yves David <pierre-yves.david@fb.com> parents: 21024 diff changeset	2270 h = inpart.read(20)
0bea9db7543b bundle2: add a "check:heads" handler Pierre-Yves David <pierre-yves.david@fb.com> parents: 21024 diff changeset	2271 heads = []
0bea9db7543b bundle2: add a "check:heads" handler Pierre-Yves David <pierre-yves.david@fb.com> parents: 21024 diff changeset	2272 while len(h) == 20:
0bea9db7543b bundle2: add a "check:heads" handler Pierre-Yves David <pierre-yves.david@fb.com> parents: 21024 diff changeset	2273 heads.append(h)
0bea9db7543b bundle2: add a "check:heads" handler Pierre-Yves David <pierre-yves.david@fb.com> parents: 21024 diff changeset	2274 h = inpart.read(20)
0bea9db7543b bundle2: add a "check:heads" handler Pierre-Yves David <pierre-yves.david@fb.com> parents: 21024 diff changeset	2275 assert not h
26565 ee1bcb9aa6e4 bundle2: add op.gettransaction() to handlers that need the lock Durham Goode <durham@fb.com> parents: 26542 diff changeset	2276 # Trigger a transaction so that we are guaranteed to have the lock now.
43077 687b865b95ad formatting: byteify all mercurial/ and hgext/ string literals Augie Fackler <augie@google.com> parents: 43076 diff changeset	2277 if op.ui.configbool(b'experimental', b'bundle2lazylocking'):
26565 ee1bcb9aa6e4 bundle2: add op.gettransaction() to handlers that need the lock Durham Goode <durham@fb.com> parents: 26542 diff changeset	2278 op.gettransaction()
29294 077d0535f51f bundle2: don't assume ordering of heads checked after push Mads Kiilerich <madski@unity3d.com> parents: 28883 diff changeset	2279 if sorted(heads) != sorted(op.repo.heads()):
43076 2372284d9457 formatting: blacken the codebase Augie Fackler <augie@google.com> parents: 42935 diff changeset	2280 raise error.PushRaced(
43117 8ff1ecfadcd1 cleanup: join string literals that are already on one line Martin von Zweigbergk <martinvonz@google.com> parents: 43115 diff changeset	2281 b'remote repository changed while pushing - please try again'
43076 2372284d9457 formatting: blacken the codebase Augie Fackler <augie@google.com> parents: 42935 diff changeset	2282 )
2372284d9457 formatting: blacken the codebase Augie Fackler <augie@google.com> parents: 42935 diff changeset	2283
21130 1ff06386217f bundle2: introduce `replycaps` part for on-demand reply Pierre-Yves David <pierre-yves.david@fb.com> parents: 21129 diff changeset	2284
43077 687b865b95ad formatting: byteify all mercurial/ and hgext/ string literals Augie Fackler <augie@google.com> parents: 43076 diff changeset	2285 @parthandler(b'check:updated-heads')
32729 16ada4cbb1a9 push: add a way to allow concurrent pushes on unrelated heads Pierre-Yves David <pierre-yves.david@octobus.net> parents: 32549 diff changeset	2286 def handlecheckupdatedheads(op, inpart):
16ada4cbb1a9 push: add a way to allow concurrent pushes on unrelated heads Pierre-Yves David <pierre-yves.david@octobus.net> parents: 32549 diff changeset	2287 """check for race on the heads touched by a push
16ada4cbb1a9 push: add a way to allow concurrent pushes on unrelated heads Pierre-Yves David <pierre-yves.david@octobus.net> parents: 32549 diff changeset	2288
16ada4cbb1a9 push: add a way to allow concurrent pushes on unrelated heads Pierre-Yves David <pierre-yves.david@octobus.net> parents: 32549 diff changeset	2289 This is similar to 'check:heads' but focus on the heads actually updated
16ada4cbb1a9 push: add a way to allow concurrent pushes on unrelated heads Pierre-Yves David <pierre-yves.david@octobus.net> parents: 32549 diff changeset	2290 during the push. If other activities happen on unrelated heads, it is
16ada4cbb1a9 push: add a way to allow concurrent pushes on unrelated heads Pierre-Yves David <pierre-yves.david@octobus.net> parents: 32549 diff changeset	2291 ignored.
16ada4cbb1a9 push: add a way to allow concurrent pushes on unrelated heads Pierre-Yves David <pierre-yves.david@octobus.net> parents: 32549 diff changeset	2292
16ada4cbb1a9 push: add a way to allow concurrent pushes on unrelated heads Pierre-Yves David <pierre-yves.david@octobus.net> parents: 32549 diff changeset	2293 This allow server with high traffic to avoid push contention as long as
16ada4cbb1a9 push: add a way to allow concurrent pushes on unrelated heads Pierre-Yves David <pierre-yves.david@octobus.net> parents: 32549 diff changeset	2294 unrelated parts of the graph are involved."""
16ada4cbb1a9 push: add a way to allow concurrent pushes on unrelated heads Pierre-Yves David <pierre-yves.david@octobus.net> parents: 32549 diff changeset	2295 h = inpart.read(20)
16ada4cbb1a9 push: add a way to allow concurrent pushes on unrelated heads Pierre-Yves David <pierre-yves.david@octobus.net> parents: 32549 diff changeset	2296 heads = []
16ada4cbb1a9 push: add a way to allow concurrent pushes on unrelated heads Pierre-Yves David <pierre-yves.david@octobus.net> parents: 32549 diff changeset	2297 while len(h) == 20:
16ada4cbb1a9 push: add a way to allow concurrent pushes on unrelated heads Pierre-Yves David <pierre-yves.david@octobus.net> parents: 32549 diff changeset	2298 heads.append(h)
16ada4cbb1a9 push: add a way to allow concurrent pushes on unrelated heads Pierre-Yves David <pierre-yves.david@octobus.net> parents: 32549 diff changeset	2299 h = inpart.read(20)
16ada4cbb1a9 push: add a way to allow concurrent pushes on unrelated heads Pierre-Yves David <pierre-yves.david@octobus.net> parents: 32549 diff changeset	2300 assert not h
16ada4cbb1a9 push: add a way to allow concurrent pushes on unrelated heads Pierre-Yves David <pierre-yves.david@octobus.net> parents: 32549 diff changeset	2301 # trigger a transaction so that we are guaranteed to have the lock now.
43077 687b865b95ad formatting: byteify all mercurial/ and hgext/ string literals Augie Fackler <augie@google.com> parents: 43076 diff changeset	2302 if op.ui.configbool(b'experimental', b'bundle2lazylocking'):
32729 16ada4cbb1a9 push: add a way to allow concurrent pushes on unrelated heads Pierre-Yves David <pierre-yves.david@octobus.net> parents: 32549 diff changeset	2303 op.gettransaction()
16ada4cbb1a9 push: add a way to allow concurrent pushes on unrelated heads Pierre-Yves David <pierre-yves.david@octobus.net> parents: 32549 diff changeset	2304
16ada4cbb1a9 push: add a way to allow concurrent pushes on unrelated heads Pierre-Yves David <pierre-yves.david@octobus.net> parents: 32549 diff changeset	2305 currentheads = set()
42002 662ffdde5adf branchcache: rename itervalues() to iterheads() Pulkit Goyal <pulkit@yandex-team.ru> parents: 41801 diff changeset	2306 for ls in op.repo.branchmap().iterheads():
32729 16ada4cbb1a9 push: add a way to allow concurrent pushes on unrelated heads Pierre-Yves David <pierre-yves.david@octobus.net> parents: 32549 diff changeset	2307 currentheads.update(ls)
16ada4cbb1a9 push: add a way to allow concurrent pushes on unrelated heads Pierre-Yves David <pierre-yves.david@octobus.net> parents: 32549 diff changeset	2308
16ada4cbb1a9 push: add a way to allow concurrent pushes on unrelated heads Pierre-Yves David <pierre-yves.david@octobus.net> parents: 32549 diff changeset	2309 for h in heads:
16ada4cbb1a9 push: add a way to allow concurrent pushes on unrelated heads Pierre-Yves David <pierre-yves.david@octobus.net> parents: 32549 diff changeset	2310 if h not in currentheads:
43076 2372284d9457 formatting: blacken the codebase Augie Fackler <augie@google.com> parents: 42935 diff changeset	2311 raise error.PushRaced(
43077 687b865b95ad formatting: byteify all mercurial/ and hgext/ string literals Augie Fackler <augie@google.com> parents: 43076 diff changeset	2312 b'remote repository changed while pushing - '
687b865b95ad formatting: byteify all mercurial/ and hgext/ string literals Augie Fackler <augie@google.com> parents: 43076 diff changeset	2313 b'please try again'
43076 2372284d9457 formatting: blacken the codebase Augie Fackler <augie@google.com> parents: 42935 diff changeset	2314 )
2372284d9457 formatting: blacken the codebase Augie Fackler <augie@google.com> parents: 42935 diff changeset	2315
32729 16ada4cbb1a9 push: add a way to allow concurrent pushes on unrelated heads Pierre-Yves David <pierre-yves.david@octobus.net> parents: 32549 diff changeset	2316
43077 687b865b95ad formatting: byteify all mercurial/ and hgext/ string literals Augie Fackler <augie@google.com> parents: 43076 diff changeset	2317 @parthandler(b'check:phases')
34820 a95067b1dca6 phase: introduce a new 'check:phases' part Boris Feld <boris.feld@octobus.net> parents: 34637 diff changeset	2318 def handlecheckphases(op, inpart):
a95067b1dca6 phase: introduce a new 'check:phases' part Boris Feld <boris.feld@octobus.net> parents: 34637 diff changeset	2319 """check that phase boundaries of the repository did not change
a95067b1dca6 phase: introduce a new 'check:phases' part Boris Feld <boris.feld@octobus.net> parents: 34637 diff changeset	2320
a95067b1dca6 phase: introduce a new 'check:phases' part Boris Feld <boris.feld@octobus.net> parents: 34637 diff changeset	2321 This is used to detect a push race.
a95067b1dca6 phase: introduce a new 'check:phases' part Boris Feld <boris.feld@octobus.net> parents: 34637 diff changeset	2322 """
a95067b1dca6 phase: introduce a new 'check:phases' part Boris Feld <boris.feld@octobus.net> parents: 34637 diff changeset	2323 phasetonodes = phases.binarydecode(inpart)
a95067b1dca6 phase: introduce a new 'check:phases' part Boris Feld <boris.feld@octobus.net> parents: 34637 diff changeset	2324 unfi = op.repo.unfiltered()
a95067b1dca6 phase: introduce a new 'check:phases' part Boris Feld <boris.feld@octobus.net> parents: 34637 diff changeset	2325 cl = unfi.changelog
a95067b1dca6 phase: introduce a new 'check:phases' part Boris Feld <boris.feld@octobus.net> parents: 34637 diff changeset	2326 phasecache = unfi._phasecache
43076 2372284d9457 formatting: blacken the codebase Augie Fackler <augie@google.com> parents: 42935 diff changeset	2327 msg = (
43077 687b865b95ad formatting: byteify all mercurial/ and hgext/ string literals Augie Fackler <augie@google.com> parents: 43076 diff changeset	2328 b'remote repository changed while pushing - please try again '
687b865b95ad formatting: byteify all mercurial/ and hgext/ string literals Augie Fackler <augie@google.com> parents: 43076 diff changeset	2329 b'(%s is %s expected %s)'
43076 2372284d9457 formatting: blacken the codebase Augie Fackler <augie@google.com> parents: 42935 diff changeset	2330 )
49004 f254fc73d956 global: bulk replace simple pycompat.iteritems(x) with x.items() Gregory Szorc <gregory.szorc@gmail.com> parents: 48966 diff changeset	2331 for expectedphase, nodes in phasetonodes.items():
34820 a95067b1dca6 phase: introduce a new 'check:phases' part Boris Feld <boris.feld@octobus.net> parents: 34637 diff changeset	2332 for n in nodes:
a95067b1dca6 phase: introduce a new 'check:phases' part Boris Feld <boris.feld@octobus.net> parents: 34637 diff changeset	2333 actualphase = phasecache.phase(unfi, cl.rev(n))
a95067b1dca6 phase: introduce a new 'check:phases' part Boris Feld <boris.feld@octobus.net> parents: 34637 diff changeset	2334 if actualphase != expectedphase:
43076 2372284d9457 formatting: blacken the codebase Augie Fackler <augie@google.com> parents: 42935 diff changeset	2335 finalmsg = msg % (
46114 59fa3890d40a node: import symbols explicitly Joerg Sonnenberger <joerg@bec.de> parents: 45957 diff changeset	2336 short(n),
43076 2372284d9457 formatting: blacken the codebase Augie Fackler <augie@google.com> parents: 42935 diff changeset	2337 phases.phasenames[actualphase],
2372284d9457 formatting: blacken the codebase Augie Fackler <augie@google.com> parents: 42935 diff changeset	2338 phases.phasenames[expectedphase],
2372284d9457 formatting: blacken the codebase Augie Fackler <augie@google.com> parents: 42935 diff changeset	2339 )
34820 a95067b1dca6 phase: introduce a new 'check:phases' part Boris Feld <boris.feld@octobus.net> parents: 34637 diff changeset	2340 raise error.PushRaced(finalmsg)
a95067b1dca6 phase: introduce a new 'check:phases' part Boris Feld <boris.feld@octobus.net> parents: 34637 diff changeset	2341
43076 2372284d9457 formatting: blacken the codebase Augie Fackler <augie@google.com> parents: 42935 diff changeset	2342
43077 687b865b95ad formatting: byteify all mercurial/ and hgext/ string literals Augie Fackler <augie@google.com> parents: 43076 diff changeset	2343 @parthandler(b'output')
21131 b7435117d951 bundle2: capture remote stdout while unbundling Pierre-Yves David <pierre-yves.david@fb.com> parents: 21130 diff changeset	2344 def handleoutput(op, inpart):
b7435117d951 bundle2: capture remote stdout while unbundling Pierre-Yves David <pierre-yves.david@fb.com> parents: 21130 diff changeset	2345 """forward output captured on the server to the client"""
b7435117d951 bundle2: capture remote stdout while unbundling Pierre-Yves David <pierre-yves.david@fb.com> parents: 21130 diff changeset	2346 for line in inpart.read().splitlines():
43077 687b865b95ad formatting: byteify all mercurial/ and hgext/ string literals Augie Fackler <augie@google.com> parents: 43076 diff changeset	2347 op.ui.status(_(b'remote: %s\n') % line)
687b865b95ad formatting: byteify all mercurial/ and hgext/ string literals Augie Fackler <augie@google.com> parents: 43076 diff changeset	2348
687b865b95ad formatting: byteify all mercurial/ and hgext/ string literals Augie Fackler <augie@google.com> parents: 43076 diff changeset	2349
687b865b95ad formatting: byteify all mercurial/ and hgext/ string literals Augie Fackler <augie@google.com> parents: 43076 diff changeset	2350 @parthandler(b'replycaps')
21130 1ff06386217f bundle2: introduce `replycaps` part for on-demand reply Pierre-Yves David <pierre-yves.david@fb.com> parents: 21129 diff changeset	2351 def handlereplycaps(op, inpart):
1ff06386217f bundle2: introduce `replycaps` part for on-demand reply Pierre-Yves David <pierre-yves.david@fb.com> parents: 21129 diff changeset	2352 """Notify that a reply bundle should be created
1ff06386217f bundle2: introduce `replycaps` part for on-demand reply Pierre-Yves David <pierre-yves.david@fb.com> parents: 21129 diff changeset	2353
21138 f469879d27ec bundle2: extract capabilities decoding Pierre-Yves David <pierre-yves.david@fb.com> parents: 21137 diff changeset	2354 The payload contains the capabilities information for the reply"""
f469879d27ec bundle2: extract capabilities decoding Pierre-Yves David <pierre-yves.david@fb.com> parents: 21137 diff changeset	2355 caps = decodecaps(inpart.read())
21130 1ff06386217f bundle2: introduce `replycaps` part for on-demand reply Pierre-Yves David <pierre-yves.david@fb.com> parents: 21129 diff changeset	2356 if op.reply is None:
21135 98fbf3adfd83 bundle2: add capabilities support in `replycaps` part Pierre-Yves David <pierre-yves.david@fb.com> parents: 21134 diff changeset	2357 op.reply = bundle20(op.ui, caps)
21131 b7435117d951 bundle2: capture remote stdout while unbundling Pierre-Yves David <pierre-yves.david@fb.com> parents: 21130 diff changeset	2358
43076 2372284d9457 formatting: blacken the codebase Augie Fackler <augie@google.com> parents: 42935 diff changeset	2359
26829 58f1645f72c3 bundle2: attribute remote failures to remote (issue4788) Gregory Szorc <gregory.szorc@gmail.com> parents: 26802 diff changeset	2360 class AbortFromPart(error.Abort):
58f1645f72c3 bundle2: attribute remote failures to remote (issue4788) Gregory Szorc <gregory.szorc@gmail.com> parents: 26802 diff changeset	2361 """Sub-class of Abort that denotes an error from a bundle2 part."""
58f1645f72c3 bundle2: attribute remote failures to remote (issue4788) Gregory Szorc <gregory.szorc@gmail.com> parents: 26802 diff changeset	2362
43076 2372284d9457 formatting: blacken the codebase Augie Fackler <augie@google.com> parents: 42935 diff changeset	2363
43077 687b865b95ad formatting: byteify all mercurial/ and hgext/ string literals Augie Fackler <augie@google.com> parents: 43076 diff changeset	2364 @parthandler(b'error:abort', (b'message', b'hint'))
24741 bb67e52362d6 bundle2: fix names for error part handler Pierre-Yves David <pierre-yves.david@fb.com> parents: 24733 diff changeset	2365 def handleerrorabort(op, inpart):
21177 952af771bc17 bundle2: gracefully handle abort during unbundle Pierre-Yves David <pierre-yves.david@fb.com> parents: 21176 diff changeset	2366 """Used to transmit abort error over the wire"""
43076 2372284d9457 formatting: blacken the codebase Augie Fackler <augie@google.com> parents: 42935 diff changeset	2367 raise AbortFromPart(
43077 687b865b95ad formatting: byteify all mercurial/ and hgext/ string literals Augie Fackler <augie@google.com> parents: 43076 diff changeset	2368 inpart.params[b'message'], hint=inpart.params.get(b'hint')
43076 2372284d9457 formatting: blacken the codebase Augie Fackler <augie@google.com> parents: 42935 diff changeset	2369 )
2372284d9457 formatting: blacken the codebase Augie Fackler <augie@google.com> parents: 42935 diff changeset	2370
2372284d9457 formatting: blacken the codebase Augie Fackler <augie@google.com> parents: 42935 diff changeset	2371
2372284d9457 formatting: blacken the codebase Augie Fackler <augie@google.com> parents: 42935 diff changeset	2372 @parthandler(
43077 687b865b95ad formatting: byteify all mercurial/ and hgext/ string literals Augie Fackler <augie@google.com> parents: 43076 diff changeset	2373 b'error:pushkey',
687b865b95ad formatting: byteify all mercurial/ and hgext/ string literals Augie Fackler <augie@google.com> parents: 43076 diff changeset	2374 (b'namespace', b'key', b'new', b'old', b'ret', b'in-reply-to'),
43076 2372284d9457 formatting: blacken the codebase Augie Fackler <augie@google.com> parents: 42935 diff changeset	2375 )
25493 d8e7b0781ad7 bundle2: convey PushkeyFailed error over the wire Pierre-Yves David <pierre-yves.david@fb.com> parents: 25492 diff changeset	2376 def handleerrorpushkey(op, inpart):
d8e7b0781ad7 bundle2: convey PushkeyFailed error over the wire Pierre-Yves David <pierre-yves.david@fb.com> parents: 25492 diff changeset	2377 """Used to transmit failure of a mandatory pushkey over the wire"""
d8e7b0781ad7 bundle2: convey PushkeyFailed error over the wire Pierre-Yves David <pierre-yves.david@fb.com> parents: 25492 diff changeset	2378 kwargs = {}
43077 687b865b95ad formatting: byteify all mercurial/ and hgext/ string literals Augie Fackler <augie@google.com> parents: 43076 diff changeset	2379 for name in (b'namespace', b'key', b'new', b'old', b'ret'):
25493 d8e7b0781ad7 bundle2: convey PushkeyFailed error over the wire Pierre-Yves David <pierre-yves.david@fb.com> parents: 25492 diff changeset	2380 value = inpart.params.get(name)
d8e7b0781ad7 bundle2: convey PushkeyFailed error over the wire Pierre-Yves David <pierre-yves.david@fb.com> parents: 25492 diff changeset	2381 if value is not None:
d8e7b0781ad7 bundle2: convey PushkeyFailed error over the wire Pierre-Yves David <pierre-yves.david@fb.com> parents: 25492 diff changeset	2382 kwargs[name] = value
43076 2372284d9457 formatting: blacken the codebase Augie Fackler <augie@google.com> parents: 42935 diff changeset	2383 raise error.PushkeyFailed(
43077 687b865b95ad formatting: byteify all mercurial/ and hgext/ string literals Augie Fackler <augie@google.com> parents: 43076 diff changeset	2384 inpart.params[b'in-reply-to'], **pycompat.strkwargs(kwargs)
43076 2372284d9457 formatting: blacken the codebase Augie Fackler <augie@google.com> parents: 42935 diff changeset	2385 )
2372284d9457 formatting: blacken the codebase Augie Fackler <augie@google.com> parents: 42935 diff changeset	2386
25493 d8e7b0781ad7 bundle2: convey PushkeyFailed error over the wire Pierre-Yves David <pierre-yves.david@fb.com> parents: 25492 diff changeset	2387
43077 687b865b95ad formatting: byteify all mercurial/ and hgext/ string literals Augie Fackler <augie@google.com> parents: 43076 diff changeset	2388 @parthandler(b'error:unsupportedcontent', (b'parttype', b'params'))
24741 bb67e52362d6 bundle2: fix names for error part handler Pierre-Yves David <pierre-yves.david@fb.com> parents: 24733 diff changeset	2389 def handleerrorunsupportedcontent(op, inpart):
21619 292331e906d7 bundle2: rename b2x:error:unknownpart to b2x:error:unsupportedcontent Pierre-Yves David <pierre-yves.david@fb.com> parents: 21618 diff changeset	2390 """Used to transmit unknown content error over the wire"""
21622 457492741007 bundle2: support transmission of params error over the wire Pierre-Yves David <pierre-yves.david@fb.com> parents: 21620 diff changeset	2391 kwargs = {}
43077 687b865b95ad formatting: byteify all mercurial/ and hgext/ string literals Augie Fackler <augie@google.com> parents: 43076 diff changeset	2392 parttype = inpart.params.get(b'parttype')
21627 3e8bcc90f07c bundle2: support None parttype in BundleValueError Pierre-Yves David <pierre-yves.david@fb.com> parents: 21626 diff changeset	2393 if parttype is not None:
43077 687b865b95ad formatting: byteify all mercurial/ and hgext/ string literals Augie Fackler <augie@google.com> parents: 43076 diff changeset	2394 kwargs[b'parttype'] = parttype
687b865b95ad formatting: byteify all mercurial/ and hgext/ string literals Augie Fackler <augie@google.com> parents: 43076 diff changeset	2395 params = inpart.params.get(b'params')
21622 457492741007 bundle2: support transmission of params error over the wire Pierre-Yves David <pierre-yves.david@fb.com> parents: 21620 diff changeset	2396 if params is not None:
43077 687b865b95ad formatting: byteify all mercurial/ and hgext/ string literals Augie Fackler <augie@google.com> parents: 43076 diff changeset	2397 kwargs[b'params'] = params.split(b'\0')
21622 457492741007 bundle2: support transmission of params error over the wire Pierre-Yves David <pierre-yves.david@fb.com> parents: 21620 diff changeset	2398
36457 247b473f408e bundle2: *strkwargs love on various kwargs constructions Augie Fackler <augie@google.com>* parents: 36306 diff changeset	2399 raise error.BundleUnknownFeatureError(**pycompat.strkwargs(kwargs))
21186 9f3652e851f8 bundle2: gracefully handle PushRaced error during unbundle Pierre-Yves David <pierre-yves.david@fb.com> parents: 21185 diff changeset	2400
43076 2372284d9457 formatting: blacken the codebase Augie Fackler <augie@google.com> parents: 42935 diff changeset	2401
43077 687b865b95ad formatting: byteify all mercurial/ and hgext/ string literals Augie Fackler <augie@google.com> parents: 43076 diff changeset	2402 @parthandler(b'error:pushraced', (b'message',))
24741 bb67e52362d6 bundle2: fix names for error part handler Pierre-Yves David <pierre-yves.david@fb.com> parents: 24733 diff changeset	2403 def handleerrorpushraced(op, inpart):
21186 9f3652e851f8 bundle2: gracefully handle PushRaced error during unbundle Pierre-Yves David <pierre-yves.david@fb.com> parents: 21185 diff changeset	2404 """Used to transmit push race error over the wire"""
43077 687b865b95ad formatting: byteify all mercurial/ and hgext/ string literals Augie Fackler <augie@google.com> parents: 43076 diff changeset	2405 raise error.ResponseError(_(b'push failed:'), inpart.params[b'message'])
687b865b95ad formatting: byteify all mercurial/ and hgext/ string literals Augie Fackler <augie@google.com> parents: 43076 diff changeset	2406
687b865b95ad formatting: byteify all mercurial/ and hgext/ string literals Augie Fackler <augie@google.com> parents: 43076 diff changeset	2407
687b865b95ad formatting: byteify all mercurial/ and hgext/ string literals Augie Fackler <augie@google.com> parents: 43076 diff changeset	2408 @parthandler(b'listkeys', (b'namespace',))
21655 35095f332846 bundle: introduce a listkey handler Pierre-Yves David <pierre-yves.david@fb.com> parents: 21644 diff changeset	2409 def handlelistkeys(op, inpart):
35095f332846 bundle: introduce a listkey handler Pierre-Yves David <pierre-yves.david@fb.com> parents: 21644 diff changeset	2410 """retrieve pushkey namespace content stored in a bundle2"""
43077 687b865b95ad formatting: byteify all mercurial/ and hgext/ string literals Augie Fackler <augie@google.com> parents: 43076 diff changeset	2411 namespace = inpart.params[b'namespace']
21655 35095f332846 bundle: introduce a listkey handler Pierre-Yves David <pierre-yves.david@fb.com> parents: 21644 diff changeset	2412 r = pushkey.decodekeys(inpart.read())
43077 687b865b95ad formatting: byteify all mercurial/ and hgext/ string literals Augie Fackler <augie@google.com> parents: 43076 diff changeset	2413 op.records.add(b'listkeys', (namespace, r))
687b865b95ad formatting: byteify all mercurial/ and hgext/ string literals Augie Fackler <augie@google.com> parents: 43076 diff changeset	2414
687b865b95ad formatting: byteify all mercurial/ and hgext/ string literals Augie Fackler <augie@google.com> parents: 43076 diff changeset	2415
687b865b95ad formatting: byteify all mercurial/ and hgext/ string literals Augie Fackler <augie@google.com> parents: 43076 diff changeset	2416 @parthandler(b'pushkey', (b'namespace', b'key', b'old', b'new'))
21660 e87d2a12d41b bundle2: add ``pushkey`` support Pierre-Yves David <pierre-yves.david@fb.com> parents: 21655 diff changeset	2417 def handlepushkey(op, inpart):
e87d2a12d41b bundle2: add ``pushkey`` support Pierre-Yves David <pierre-yves.david@fb.com> parents: 21655 diff changeset	2418 """process a pushkey request"""
e87d2a12d41b bundle2: add ``pushkey`` support Pierre-Yves David <pierre-yves.david@fb.com> parents: 21655 diff changeset	2419 dec = pushkey.decode
43077 687b865b95ad formatting: byteify all mercurial/ and hgext/ string literals Augie Fackler <augie@google.com> parents: 43076 diff changeset	2420 namespace = dec(inpart.params[b'namespace'])
687b865b95ad formatting: byteify all mercurial/ and hgext/ string literals Augie Fackler <augie@google.com> parents: 43076 diff changeset	2421 key = dec(inpart.params[b'key'])
687b865b95ad formatting: byteify all mercurial/ and hgext/ string literals Augie Fackler <augie@google.com> parents: 43076 diff changeset	2422 old = dec(inpart.params[b'old'])
687b865b95ad formatting: byteify all mercurial/ and hgext/ string literals Augie Fackler <augie@google.com> parents: 43076 diff changeset	2423 new = dec(inpart.params[b'new'])
26565 ee1bcb9aa6e4 bundle2: add op.gettransaction() to handlers that need the lock Durham Goode <durham@fb.com> parents: 26542 diff changeset	2424 # Grab the transaction to ensure that we have the lock before performing the
ee1bcb9aa6e4 bundle2: add op.gettransaction() to handlers that need the lock Durham Goode <durham@fb.com> parents: 26542 diff changeset	2425 # pushkey.
43077 687b865b95ad formatting: byteify all mercurial/ and hgext/ string literals Augie Fackler <augie@google.com> parents: 43076 diff changeset	2426 if op.ui.configbool(b'experimental', b'bundle2lazylocking'):
26565 ee1bcb9aa6e4 bundle2: add op.gettransaction() to handlers that need the lock Durham Goode <durham@fb.com> parents: 26542 diff changeset	2427 op.gettransaction()
21660 e87d2a12d41b bundle2: add ``pushkey`` support Pierre-Yves David <pierre-yves.david@fb.com> parents: 21655 diff changeset	2428 ret = op.repo.pushkey(namespace, key, old, new)
43077 687b865b95ad formatting: byteify all mercurial/ and hgext/ string literals Augie Fackler <augie@google.com> parents: 43076 diff changeset	2429 record = {b'namespace': namespace, b'key': key, b'old': old, b'new': new}
687b865b95ad formatting: byteify all mercurial/ and hgext/ string literals Augie Fackler <augie@google.com> parents: 43076 diff changeset	2430 op.records.add(b'pushkey', record)
21660 e87d2a12d41b bundle2: add ``pushkey`` support Pierre-Yves David <pierre-yves.david@fb.com> parents: 21655 diff changeset	2431 if op.reply is not None:
43077 687b865b95ad formatting: byteify all mercurial/ and hgext/ string literals Augie Fackler <augie@google.com> parents: 43076 diff changeset	2432 rpart = op.reply.newpart(b'reply:pushkey')
33701 24b7c75c6441 bundle2: use bytestr() instead of str() to convert part id to bytes Augie Fackler <augie@google.com> parents: 33700 diff changeset	2433 rpart.addparam(
43077 687b865b95ad formatting: byteify all mercurial/ and hgext/ string literals Augie Fackler <augie@google.com> parents: 43076 diff changeset	2434 b'in-reply-to', pycompat.bytestr(inpart.id), mandatory=False
43076 2372284d9457 formatting: blacken the codebase Augie Fackler <augie@google.com> parents: 42935 diff changeset	2435 )
43077 687b865b95ad formatting: byteify all mercurial/ and hgext/ string literals Augie Fackler <augie@google.com> parents: 43076 diff changeset	2436 rpart.addparam(b'return', b'%i' % ret, mandatory=False)
25481 6de96cb31176 bundle2: abort when a mandatory pushkey part fails Pierre-Yves David <pierre-yves.david@fb.com> parents: 25401 diff changeset	2437 if inpart.mandatory and not ret:
25484 a5192774e925 bundle2: introduce a PushkeyFail error to abort unbundle on pushkey error Pierre-Yves David <pierre-yves.david@fb.com> parents: 25481 diff changeset	2438 kwargs = {}
43077 687b865b95ad formatting: byteify all mercurial/ and hgext/ string literals Augie Fackler <augie@google.com> parents: 43076 diff changeset	2439 for key in (b'namespace', b'key', b'new', b'old', b'ret'):
25484 a5192774e925 bundle2: introduce a PushkeyFail error to abort unbundle on pushkey error Pierre-Yves David <pierre-yves.david@fb.com> parents: 25481 diff changeset	2440 if key in inpart.params:
a5192774e925 bundle2: introduce a PushkeyFail error to abort unbundle on pushkey error Pierre-Yves David <pierre-yves.david@fb.com> parents: 25481 diff changeset	2441 kwargs[key] = inpart.params[key]
43076 2372284d9457 formatting: blacken the codebase Augie Fackler <augie@google.com> parents: 42935 diff changeset	2442 raise error.PushkeyFailed(
43077 687b865b95ad formatting: byteify all mercurial/ and hgext/ string literals Augie Fackler <augie@google.com> parents: 43076 diff changeset	2443 partid=b'%d' % inpart.id, **pycompat.strkwargs(kwargs)
43076 2372284d9457 formatting: blacken the codebase Augie Fackler <augie@google.com> parents: 42935 diff changeset	2444 )
2372284d9457 formatting: blacken the codebase Augie Fackler <augie@google.com> parents: 42935 diff changeset	2445
21660 e87d2a12d41b bundle2: add ``pushkey`` support Pierre-Yves David <pierre-yves.david@fb.com> parents: 21655 diff changeset	2446
43077 687b865b95ad formatting: byteify all mercurial/ and hgext/ string literals Augie Fackler <augie@google.com> parents: 43076 diff changeset	2447 @parthandler(b'bookmarks')
35269 af5507203d01 bookmark: introduce a 'bookmarks' part Boris Feld <boris.feld@octobus.net> parents: 35268 diff changeset	2448 def handlebookmark(op, inpart):
af5507203d01 bookmark: introduce a 'bookmarks' part Boris Feld <boris.feld@octobus.net> parents: 35268 diff changeset	2449 """transmit bookmark information
af5507203d01 bookmark: introduce a 'bookmarks' part Boris Feld <boris.feld@octobus.net> parents: 35268 diff changeset	2450
35275 496154e41968 bundle2: support a 'records' mode for the 'bookmarks' part Boris Feld <boris.feld@octobus.net> parents: 35274 diff changeset	2451 The part contains binary encoded bookmark information.
496154e41968 bundle2: support a 'records' mode for the 'bookmarks' part Boris Feld <boris.feld@octobus.net> parents: 35274 diff changeset	2452
496154e41968 bundle2: support a 'records' mode for the 'bookmarks' part Boris Feld <boris.feld@octobus.net> parents: 35274 diff changeset	2453 The exact behavior of this part can be controlled by the 'bookmarks' mode
496154e41968 bundle2: support a 'records' mode for the 'bookmarks' part Boris Feld <boris.feld@octobus.net> parents: 35274 diff changeset	2454 on the bundle operation.
35269 af5507203d01 bookmark: introduce a 'bookmarks' part Boris Feld <boris.feld@octobus.net> parents: 35268 diff changeset	2455
35275 496154e41968 bundle2: support a 'records' mode for the 'bookmarks' part Boris Feld <boris.feld@octobus.net> parents: 35274 diff changeset	2456 When mode is 'apply' (the default) the bookmark information is applied as
496154e41968 bundle2: support a 'records' mode for the 'bookmarks' part Boris Feld <boris.feld@octobus.net> parents: 35274 diff changeset	2457 is to the unbundling repository. Make sure a 'check:bookmarks' part is
496154e41968 bundle2: support a 'records' mode for the 'bookmarks' part Boris Feld <boris.feld@octobus.net> parents: 35274 diff changeset	2458 issued earlier to check for push races in such update. This behavior is
496154e41968 bundle2: support a 'records' mode for the 'bookmarks' part Boris Feld <boris.feld@octobus.net> parents: 35274 diff changeset	2459 suitable for pushing.
496154e41968 bundle2: support a 'records' mode for the 'bookmarks' part Boris Feld <boris.feld@octobus.net> parents: 35274 diff changeset	2460
496154e41968 bundle2: support a 'records' mode for the 'bookmarks' part Boris Feld <boris.feld@octobus.net> parents: 35274 diff changeset	2461 When mode is 'records', the information is recorded into the 'bookmarks'
496154e41968 bundle2: support a 'records' mode for the 'bookmarks' part Boris Feld <boris.feld@octobus.net> parents: 35274 diff changeset	2462 records of the bundle operation. This behavior is suitable for pulling.
35269 af5507203d01 bookmark: introduce a 'bookmarks' part Boris Feld <boris.feld@octobus.net> parents: 35268 diff changeset	2463 """
46793 6266d19556ad node: introduce nodeconstants class Joerg Sonnenberger <joerg@bec.de> parents: 46726 diff changeset	2464 changes = bookmarks.binarydecode(op.repo, inpart)
35270 f392066d127c bookmark: add pushkey hook compatiblity to the bundle2 part Boris Feld <boris.feld@octobus.net> parents: 35269 diff changeset	2465
43077 687b865b95ad formatting: byteify all mercurial/ and hgext/ string literals Augie Fackler <augie@google.com> parents: 43076 diff changeset	2466 pushkeycompat = op.repo.ui.configbool(
687b865b95ad formatting: byteify all mercurial/ and hgext/ string literals Augie Fackler <augie@google.com> parents: 43076 diff changeset	2467 b'server', b'bookmarks-pushkey-compat'
687b865b95ad formatting: byteify all mercurial/ and hgext/ string literals Augie Fackler <augie@google.com> parents: 43076 diff changeset	2468 )
687b865b95ad formatting: byteify all mercurial/ and hgext/ string literals Augie Fackler <augie@google.com> parents: 43076 diff changeset	2469 bookmarksmode = op.modes.get(b'bookmarks', b'apply')
687b865b95ad formatting: byteify all mercurial/ and hgext/ string literals Augie Fackler <augie@google.com> parents: 43076 diff changeset	2470
687b865b95ad formatting: byteify all mercurial/ and hgext/ string literals Augie Fackler <augie@google.com> parents: 43076 diff changeset	2471 if bookmarksmode == b'apply':
35275 496154e41968 bundle2: support a 'records' mode for the 'bookmarks' part Boris Feld <boris.feld@octobus.net> parents: 35274 diff changeset	2472 tr = op.gettransaction()
496154e41968 bundle2: support a 'records' mode for the 'bookmarks' part Boris Feld <boris.feld@octobus.net> parents: 35274 diff changeset	2473 bookstore = op.repo._bookmarks
496154e41968 bundle2: support a 'records' mode for the 'bookmarks' part Boris Feld <boris.feld@octobus.net> parents: 35274 diff changeset	2474 if pushkeycompat:
496154e41968 bundle2: support a 'records' mode for the 'bookmarks' part Boris Feld <boris.feld@octobus.net> parents: 35274 diff changeset	2475 allhooks = []
496154e41968 bundle2: support a 'records' mode for the 'bookmarks' part Boris Feld <boris.feld@octobus.net> parents: 35274 diff changeset	2476 for book, node in changes:
496154e41968 bundle2: support a 'records' mode for the 'bookmarks' part Boris Feld <boris.feld@octobus.net> parents: 35274 diff changeset	2477 hookargs = tr.hookargs.copy()
43077 687b865b95ad formatting: byteify all mercurial/ and hgext/ string literals Augie Fackler <augie@google.com> parents: 43076 diff changeset	2478 hookargs[b'pushkeycompat'] = b'1'
687b865b95ad formatting: byteify all mercurial/ and hgext/ string literals Augie Fackler <augie@google.com> parents: 43076 diff changeset	2479 hookargs[b'namespace'] = b'bookmarks'
687b865b95ad formatting: byteify all mercurial/ and hgext/ string literals Augie Fackler <augie@google.com> parents: 43076 diff changeset	2480 hookargs[b'key'] = book
46114 59fa3890d40a node: import symbols explicitly Joerg Sonnenberger <joerg@bec.de> parents: 45957 diff changeset	2481 hookargs[b'old'] = hex(bookstore.get(book, b''))
59fa3890d40a node: import symbols explicitly Joerg Sonnenberger <joerg@bec.de> parents: 45957 diff changeset	2482 hookargs[b'new'] = hex(node if node is not None else b'')
35275 496154e41968 bundle2: support a 'records' mode for the 'bookmarks' part Boris Feld <boris.feld@octobus.net> parents: 35274 diff changeset	2483 allhooks.append(hookargs)
496154e41968 bundle2: support a 'records' mode for the 'bookmarks' part Boris Feld <boris.feld@octobus.net> parents: 35274 diff changeset	2484
496154e41968 bundle2: support a 'records' mode for the 'bookmarks' part Boris Feld <boris.feld@octobus.net> parents: 35274 diff changeset	2485 for hookargs in allhooks:
43076 2372284d9457 formatting: blacken the codebase Augie Fackler <augie@google.com> parents: 42935 diff changeset	2486 op.repo.hook(
43077 687b865b95ad formatting: byteify all mercurial/ and hgext/ string literals Augie Fackler <augie@google.com> parents: 43076 diff changeset	2487 b'prepushkey', throw=True, **pycompat.strkwargs(hookargs)
43076 2372284d9457 formatting: blacken the codebase Augie Fackler <augie@google.com> parents: 42935 diff changeset	2488 )
35275 496154e41968 bundle2: support a 'records' mode for the 'bookmarks' part Boris Feld <boris.feld@octobus.net> parents: 35274 diff changeset	2489
44406 8407031f195f bookmarks: prevent pushes of divergent bookmarks (foo@remote) Valentin Gatien-Baron <valentin.gatienbaron@gmail.com> parents: 44173 diff changeset	2490 for book, node in changes:
8407031f195f bookmarks: prevent pushes of divergent bookmarks (foo@remote) Valentin Gatien-Baron <valentin.gatienbaron@gmail.com> parents: 44173 diff changeset	2491 if bookmarks.isdivergent(book):
8407031f195f bookmarks: prevent pushes of divergent bookmarks (foo@remote) Valentin Gatien-Baron <valentin.gatienbaron@gmail.com> parents: 44173 diff changeset	2492 msg = _(b'cannot accept divergent bookmark %s!') % book
8407031f195f bookmarks: prevent pushes of divergent bookmarks (foo@remote) Valentin Gatien-Baron <valentin.gatienbaron@gmail.com> parents: 44173 diff changeset	2493 raise error.Abort(msg)
8407031f195f bookmarks: prevent pushes of divergent bookmarks (foo@remote) Valentin Gatien-Baron <valentin.gatienbaron@gmail.com> parents: 44173 diff changeset	2494
35275 496154e41968 bundle2: support a 'records' mode for the 'bookmarks' part Boris Feld <boris.feld@octobus.net> parents: 35274 diff changeset	2495 bookstore.applychanges(op.repo, op.gettransaction(), changes)
496154e41968 bundle2: support a 'records' mode for the 'bookmarks' part Boris Feld <boris.feld@octobus.net> parents: 35274 diff changeset	2496
496154e41968 bundle2: support a 'records' mode for the 'bookmarks' part Boris Feld <boris.feld@octobus.net> parents: 35274 diff changeset	2497 if pushkeycompat:
43076 2372284d9457 formatting: blacken the codebase Augie Fackler <augie@google.com> parents: 42935 diff changeset	2498
43798 888bd39ed555 lock: pass "success" boolean to _afterlock callbacks Kyle Lippincott <spectral@google.com> parents: 43554 diff changeset	2499 def runhook(unused_success):
35275 496154e41968 bundle2: support a 'records' mode for the 'bookmarks' part Boris Feld <boris.feld@octobus.net> parents: 35274 diff changeset	2500 for hookargs in allhooks:
43077 687b865b95ad formatting: byteify all mercurial/ and hgext/ string literals Augie Fackler <augie@google.com> parents: 43076 diff changeset	2501 op.repo.hook(b'pushkey', **pycompat.strkwargs(hookargs))
43076 2372284d9457 formatting: blacken the codebase Augie Fackler <augie@google.com> parents: 42935 diff changeset	2502
35275 496154e41968 bundle2: support a 'records' mode for the 'bookmarks' part Boris Feld <boris.feld@octobus.net> parents: 35274 diff changeset	2503 op.repo._afterlock(runhook)
496154e41968 bundle2: support a 'records' mode for the 'bookmarks' part Boris Feld <boris.feld@octobus.net> parents: 35274 diff changeset	2504
43077 687b865b95ad formatting: byteify all mercurial/ and hgext/ string literals Augie Fackler <augie@google.com> parents: 43076 diff changeset	2505 elif bookmarksmode == b'records':
35270 f392066d127c bookmark: add pushkey hook compatiblity to the bundle2 part Boris Feld <boris.feld@octobus.net> parents: 35269 diff changeset	2506 for book, node in changes:
43077 687b865b95ad formatting: byteify all mercurial/ and hgext/ string literals Augie Fackler <augie@google.com> parents: 43076 diff changeset	2507 record = {b'bookmark': book, b'node': node}
687b865b95ad formatting: byteify all mercurial/ and hgext/ string literals Augie Fackler <augie@google.com> parents: 43076 diff changeset	2508 op.records.add(b'bookmarks', record)
35275 496154e41968 bundle2: support a 'records' mode for the 'bookmarks' part Boris Feld <boris.feld@octobus.net> parents: 35274 diff changeset	2509 else:
43077 687b865b95ad formatting: byteify all mercurial/ and hgext/ string literals Augie Fackler <augie@google.com> parents: 43076 diff changeset	2510 raise error.ProgrammingError(
48469 fff5942d445f typo: s/unkown/unknown across the codebase Rapha?l Gom?s <rgomes@octobus.net> parents: 47237 diff changeset	2511 b'unknown bookmark mode: %s' % bookmarksmode
43077 687b865b95ad formatting: byteify all mercurial/ and hgext/ string literals Augie Fackler <augie@google.com> parents: 43076 diff changeset	2512 )
687b865b95ad formatting: byteify all mercurial/ and hgext/ string literals Augie Fackler <augie@google.com> parents: 43076 diff changeset	2513
687b865b95ad formatting: byteify all mercurial/ and hgext/ string literals Augie Fackler <augie@google.com> parents: 43076 diff changeset	2514
687b865b95ad formatting: byteify all mercurial/ and hgext/ string literals Augie Fackler <augie@google.com> parents: 43076 diff changeset	2515 @parthandler(b'phase-heads')
33043 e8c8d81eb864 bundle: add config option to include phases Martin von Zweigbergk <martinvonz@google.com> parents: 33042 diff changeset	2516 def handlephases(op, inpart):
e8c8d81eb864 bundle: add config option to include phases Martin von Zweigbergk <martinvonz@google.com> parents: 33042 diff changeset	2517 """apply phases from bundle part to repo"""
34327 12c42bcd4133 phases: move the binary decoding function in the phases module Boris Feld <boris.feld@octobus.net> parents: 34326 diff changeset	2518 headsbyphase = phases.binarydecode(inpart)
34328 4ef472b975ff bundle2: only grab a transaction when 'phase-heads' affect the repository Boris Feld <boris.feld@octobus.net> parents: 34327 diff changeset	2519 phases.updatephases(op.repo.unfiltered(), op.gettransaction, headsbyphase)
33043 e8c8d81eb864 bundle: add config option to include phases Martin von Zweigbergk <martinvonz@google.com> parents: 33042 diff changeset	2520
43076 2372284d9457 formatting: blacken the codebase Augie Fackler <augie@google.com> parents: 42935 diff changeset	2521
43077 687b865b95ad formatting: byteify all mercurial/ and hgext/ string literals Augie Fackler <augie@google.com> parents: 43076 diff changeset	2522 @parthandler(b'reply:pushkey', (b'return', b'in-reply-to'))
21660 e87d2a12d41b bundle2: add ``pushkey`` support Pierre-Yves David <pierre-yves.david@fb.com> parents: 21655 diff changeset	2523 def handlepushkeyreply(op, inpart):
e87d2a12d41b bundle2: add ``pushkey`` support Pierre-Yves David <pierre-yves.david@fb.com> parents: 21655 diff changeset	2524 """retrieve the result of a pushkey request"""
43077 687b865b95ad formatting: byteify all mercurial/ and hgext/ string literals Augie Fackler <augie@google.com> parents: 43076 diff changeset	2525 ret = int(inpart.params[b'return'])
687b865b95ad formatting: byteify all mercurial/ and hgext/ string literals Augie Fackler <augie@google.com> parents: 43076 diff changeset	2526 partid = int(inpart.params[b'in-reply-to'])
687b865b95ad formatting: byteify all mercurial/ and hgext/ string literals Augie Fackler <augie@google.com> parents: 43076 diff changeset	2527 op.records.add(b'pushkey', {b'return': ret}, partid)
687b865b95ad formatting: byteify all mercurial/ and hgext/ string literals Augie Fackler <augie@google.com> parents: 43076 diff changeset	2528
687b865b95ad formatting: byteify all mercurial/ and hgext/ string literals Augie Fackler <augie@google.com> parents: 43076 diff changeset	2529
687b865b95ad formatting: byteify all mercurial/ and hgext/ string literals Augie Fackler <augie@google.com> parents: 43076 diff changeset	2530 @parthandler(b'obsmarkers')
22336 60786c8a2f70 bundle2: add an obsmarkers part handler Pierre-Yves David <pierre-yves.david@fb.com> parents: 21900 diff changeset	2531 def handleobsmarker(op, inpart):
60786c8a2f70 bundle2: add an obsmarkers part handler Pierre-Yves David <pierre-yves.david@fb.com> parents: 21900 diff changeset	2532 """add a stream of obsmarkers to the repo"""
60786c8a2f70 bundle2: add an obsmarkers part handler Pierre-Yves David <pierre-yves.david@fb.com> parents: 21900 diff changeset	2533 tr = op.gettransaction()
24733 c00e4338fa4b obsolete: experimental flag to get debug about obsmarkers exchange Pierre-Yves David <pierre-yves.david@fb.com> parents: 24686 diff changeset	2534 markerdata = inpart.read()
43077 687b865b95ad formatting: byteify all mercurial/ and hgext/ string literals Augie Fackler <augie@google.com> parents: 43076 diff changeset	2535 if op.ui.config(b'experimental', b'obsmarkers-exchange-debug'):
43080 86e4daa2d54c cleanup: mark some ui.(status\|note\|warn\|write) calls as not needing i18n Augie Fackler <augie@google.com> parents: 43077 diff changeset	2536 op.ui.writenoi18n(
43077 687b865b95ad formatting: byteify all mercurial/ and hgext/ string literals Augie Fackler <augie@google.com> parents: 43076 diff changeset	2537 b'obsmarker-exchange: %i bytes received\n' % len(markerdata)
687b865b95ad formatting: byteify all mercurial/ and hgext/ string literals Augie Fackler <augie@google.com> parents: 43076 diff changeset	2538 )
26685 ea390d889d3a bundle2: gracefully skip 'obsmarkers' part if evolution is disabled Pierre-Yves David <pierre-yves.david@fb.com> parents: 26587 diff changeset	2539 # The mergemarkers call will crash if marker creation is not enabled.
ea390d889d3a bundle2: gracefully skip 'obsmarkers' part if evolution is disabled Pierre-Yves David <pierre-yves.david@fb.com> parents: 26587 diff changeset	2540 # we want to avoid this if the part is advisory.
ea390d889d3a bundle2: gracefully skip 'obsmarkers' part if evolution is disabled Pierre-Yves David <pierre-yves.david@fb.com> parents: 26587 diff changeset	2541 if not inpart.mandatory and op.repo.obsstore.readonly:
43077 687b865b95ad formatting: byteify all mercurial/ and hgext/ string literals Augie Fackler <augie@google.com> parents: 43076 diff changeset	2542 op.repo.ui.debug(
687b865b95ad formatting: byteify all mercurial/ and hgext/ string literals Augie Fackler <augie@google.com> parents: 43076 diff changeset	2543 b'ignoring obsolescence markers, feature not enabled\n'
687b865b95ad formatting: byteify all mercurial/ and hgext/ string literals Augie Fackler <augie@google.com> parents: 43076 diff changeset	2544 )
26685 ea390d889d3a bundle2: gracefully skip 'obsmarkers' part if evolution is disabled Pierre-Yves David <pierre-yves.david@fb.com> parents: 26587 diff changeset	2545 return
24733 c00e4338fa4b obsolete: experimental flag to get debug about obsmarkers exchange Pierre-Yves David <pierre-yves.david@fb.com> parents: 24686 diff changeset	2546 new = op.repo.obsstore.mergemarkers(tr, markerdata)
32124 99515353c72a obsolete: invalidate "volatile" set cache after merging marker Pierre-Yves David <pierre-yves.david@octobus.net> parents: 32024 diff changeset	2547 op.repo.invalidatevolatilesets()
43077 687b865b95ad formatting: byteify all mercurial/ and hgext/ string literals Augie Fackler <augie@google.com> parents: 43076 diff changeset	2548 op.records.add(b'obsmarkers', {b'new': new})
22340 394a17de6a2d obsmarker: produce a reply part for markers received through bundle2 Pierre-Yves David <pierre-yves.david@fb.com> parents: 22338 diff changeset	2549 if op.reply is not None:
43077 687b865b95ad formatting: byteify all mercurial/ and hgext/ string literals Augie Fackler <augie@google.com> parents: 43076 diff changeset	2550 rpart = op.reply.newpart(b'reply:obsmarkers')
33701 24b7c75c6441 bundle2: use bytestr() instead of str() to convert part id to bytes Augie Fackler <augie@google.com> parents: 33700 diff changeset	2551 rpart.addparam(
43077 687b865b95ad formatting: byteify all mercurial/ and hgext/ string literals Augie Fackler <augie@google.com> parents: 43076 diff changeset	2552 b'in-reply-to', pycompat.bytestr(inpart.id), mandatory=False
43076 2372284d9457 formatting: blacken the codebase Augie Fackler <augie@google.com> parents: 42935 diff changeset	2553 )
43077 687b865b95ad formatting: byteify all mercurial/ and hgext/ string literals Augie Fackler <augie@google.com> parents: 43076 diff changeset	2554 rpart.addparam(b'new', b'%i' % new, mandatory=False)
687b865b95ad formatting: byteify all mercurial/ and hgext/ string literals Augie Fackler <augie@google.com> parents: 43076 diff changeset	2555
687b865b95ad formatting: byteify all mercurial/ and hgext/ string literals Augie Fackler <augie@google.com> parents: 43076 diff changeset	2556
687b865b95ad formatting: byteify all mercurial/ and hgext/ string literals Augie Fackler <augie@google.com> parents: 43076 diff changeset	2557 @parthandler(b'reply:obsmarkers', (b'new', b'in-reply-to'))
25506 86472038dfd2 bundle2: rename duplicate handlepushkeyreply to handleobsmarkerreply Martin von Zweigbergk <martinvonz@google.com> parents: 25493 diff changeset	2558 def handleobsmarkerreply(op, inpart):
22340 394a17de6a2d obsmarker: produce a reply part for markers received through bundle2 Pierre-Yves David <pierre-yves.david@fb.com> parents: 22338 diff changeset	2559 """retrieve the result of a pushkey request"""
43077 687b865b95ad formatting: byteify all mercurial/ and hgext/ string literals Augie Fackler <augie@google.com> parents: 43076 diff changeset	2560 ret = int(inpart.params[b'new'])
687b865b95ad formatting: byteify all mercurial/ and hgext/ string literals Augie Fackler <augie@google.com> parents: 43076 diff changeset	2561 partid = int(inpart.params[b'in-reply-to'])
687b865b95ad formatting: byteify all mercurial/ and hgext/ string literals Augie Fackler <augie@google.com> parents: 43076 diff changeset	2562 op.records.add(b'obsmarkers', {b'new': ret}, partid)
687b865b95ad formatting: byteify all mercurial/ and hgext/ string literals Augie Fackler <augie@google.com> parents: 43076 diff changeset	2563
687b865b95ad formatting: byteify all mercurial/ and hgext/ string literals Augie Fackler <augie@google.com> parents: 43076 diff changeset	2564
687b865b95ad formatting: byteify all mercurial/ and hgext/ string literals Augie Fackler <augie@google.com> parents: 43076 diff changeset	2565 @parthandler(b'hgtagsfnodes')
25401 d29201352af7 bundle2: part handler for processing .hgtags fnodes mappings Gregory Szorc <gregory.szorc@gmail.com> parents: 25336 diff changeset	2566 def handlehgtagsfnodes(op, inpart):
d29201352af7 bundle2: part handler for processing .hgtags fnodes mappings Gregory Szorc <gregory.szorc@gmail.com> parents: 25336 diff changeset	2567 """Applies .hgtags fnodes cache entries to the local repo.
d29201352af7 bundle2: part handler for processing .hgtags fnodes mappings Gregory Szorc <gregory.szorc@gmail.com> parents: 25336 diff changeset	2568
d29201352af7 bundle2: part handler for processing .hgtags fnodes mappings Gregory Szorc <gregory.szorc@gmail.com> parents: 25336 diff changeset	2569 Payload is pairs of 20 byte changeset nodes and filenodes.
d29201352af7 bundle2: part handler for processing .hgtags fnodes mappings Gregory Szorc <gregory.szorc@gmail.com> parents: 25336 diff changeset	2570 """
26565 ee1bcb9aa6e4 bundle2: add op.gettransaction() to handlers that need the lock Durham Goode <durham@fb.com> parents: 26542 diff changeset	2571 # Grab the transaction so we ensure that we have the lock at this point.
43077 687b865b95ad formatting: byteify all mercurial/ and hgext/ string literals Augie Fackler <augie@google.com> parents: 43076 diff changeset	2572 if op.ui.configbool(b'experimental', b'bundle2lazylocking'):
26565 ee1bcb9aa6e4 bundle2: add op.gettransaction() to handlers that need the lock Durham Goode <durham@fb.com> parents: 26542 diff changeset	2573 op.gettransaction()
25401 d29201352af7 bundle2: part handler for processing .hgtags fnodes mappings Gregory Szorc <gregory.szorc@gmail.com> parents: 25336 diff changeset	2574 cache = tags.hgtagsfnodescache(op.repo.unfiltered())
d29201352af7 bundle2: part handler for processing .hgtags fnodes mappings Gregory Szorc <gregory.szorc@gmail.com> parents: 25336 diff changeset	2575
d29201352af7 bundle2: part handler for processing .hgtags fnodes mappings Gregory Szorc <gregory.szorc@gmail.com> parents: 25336 diff changeset	2576 count = 0
d29201352af7 bundle2: part handler for processing .hgtags fnodes mappings Gregory Szorc <gregory.szorc@gmail.com> parents: 25336 diff changeset	2577 while True:
d29201352af7 bundle2: part handler for processing .hgtags fnodes mappings Gregory Szorc <gregory.szorc@gmail.com> parents: 25336 diff changeset	2578 node = inpart.read(20)
d29201352af7 bundle2: part handler for processing .hgtags fnodes mappings Gregory Szorc <gregory.szorc@gmail.com> parents: 25336 diff changeset	2579 fnode = inpart.read(20)
d29201352af7 bundle2: part handler for processing .hgtags fnodes mappings Gregory Szorc <gregory.szorc@gmail.com> parents: 25336 diff changeset	2580 if len(node) < 20 or len(fnode) < 20:
43077 687b865b95ad formatting: byteify all mercurial/ and hgext/ string literals Augie Fackler <augie@google.com> parents: 43076 diff changeset	2581 op.ui.debug(b'ignoring incomplete received .hgtags fnodes data\n')
25401 d29201352af7 bundle2: part handler for processing .hgtags fnodes mappings Gregory Szorc <gregory.szorc@gmail.com> parents: 25336 diff changeset	2582 break
d29201352af7 bundle2: part handler for processing .hgtags fnodes mappings Gregory Szorc <gregory.szorc@gmail.com> parents: 25336 diff changeset	2583 cache.setfnode(node, fnode)
d29201352af7 bundle2: part handler for processing .hgtags fnodes mappings Gregory Szorc <gregory.szorc@gmail.com> parents: 25336 diff changeset	2584 count += 1
d29201352af7 bundle2: part handler for processing .hgtags fnodes mappings Gregory Szorc <gregory.szorc@gmail.com> parents: 25336 diff changeset	2585
d29201352af7 bundle2: part handler for processing .hgtags fnodes mappings Gregory Szorc <gregory.szorc@gmail.com> parents: 25336 diff changeset	2586 cache.write()
43077 687b865b95ad formatting: byteify all mercurial/ and hgext/ string literals Augie Fackler <augie@google.com> parents: 43076 diff changeset	2587 op.ui.debug(b'applied %i hgtags fnodes cache entries\n' % count)
687b865b95ad formatting: byteify all mercurial/ and hgext/ string literals Augie Fackler <augie@google.com> parents: 43076 diff changeset	2588
687b865b95ad formatting: byteify all mercurial/ and hgext/ string literals Augie Fackler <augie@google.com> parents: 43076 diff changeset	2589
687b865b95ad formatting: byteify all mercurial/ and hgext/ string literals Augie Fackler <augie@google.com> parents: 43076 diff changeset	2590 rbcstruct = struct.Struct(b'>III')
687b865b95ad formatting: byteify all mercurial/ and hgext/ string literals Augie Fackler <augie@google.com> parents: 43076 diff changeset	2591
687b865b95ad formatting: byteify all mercurial/ and hgext/ string literals Augie Fackler <augie@google.com> parents: 43076 diff changeset	2592
687b865b95ad formatting: byteify all mercurial/ and hgext/ string literals Augie Fackler <augie@google.com> parents: 43076 diff changeset	2593 @parthandler(b'cache:rev-branch-cache')
36969 9988fc10f49e revbranchcache: add a bundle2 handler for a rbc part Boris Feld <boris.feld@octobus.net> parents: 36458 diff changeset	2594 def handlerbc(op, inpart):
46444 3e91d9978bec branchmap: update rev-branch-cache incrementally Joerg Sonnenberger <joerg@bec.de> parents: 46166 diff changeset	2595 """Legacy part, ignored for compatibility with bundles from or
3e91d9978bec branchmap: update rev-branch-cache incrementally Joerg Sonnenberger <joerg@bec.de> parents: 46166 diff changeset	2596 for Mercurial before 5.7. Newer Mercurial computes the cache
3e91d9978bec branchmap: update rev-branch-cache incrementally Joerg Sonnenberger <joerg@bec.de> parents: 46166 diff changeset	2597 efficiently enough during unbundling that the additional transfer
3e91d9978bec branchmap: update rev-branch-cache incrementally Joerg Sonnenberger <joerg@bec.de> parents: 46166 diff changeset	2598 is unnecessary."""
36969 9988fc10f49e revbranchcache: add a bundle2 handler for a rbc part Boris Feld <boris.feld@octobus.net> parents: 36458 diff changeset	2599
43076 2372284d9457 formatting: blacken the codebase Augie Fackler <augie@google.com> parents: 42935 diff changeset	2600
43077 687b865b95ad formatting: byteify all mercurial/ and hgext/ string literals Augie Fackler <augie@google.com> parents: 43076 diff changeset	2601 @parthandler(b'pushvars')
33719 db3dc11356ed pushvars: move fb extension pushvars to core Pulkit Goyal <7895pulkit@gmail.com> parents: 33708 diff changeset	2602 def bundle2getvars(op, part):
db3dc11356ed pushvars: move fb extension pushvars to core Pulkit Goyal <7895pulkit@gmail.com> parents: 33708 diff changeset	2603 '''unbundle a bundle2 containing shellvars on the server'''
db3dc11356ed pushvars: move fb extension pushvars to core Pulkit Goyal <7895pulkit@gmail.com> parents: 33708 diff changeset	2604 # An option to disable unbundling on server-side for security reasons
43077 687b865b95ad formatting: byteify all mercurial/ and hgext/ string literals Augie Fackler <augie@google.com> parents: 43076 diff changeset	2605 if op.ui.configbool(b'push', b'pushvars.server'):
33719 db3dc11356ed pushvars: move fb extension pushvars to core Pulkit Goyal <7895pulkit@gmail.com> parents: 33708 diff changeset	2606 hookargs = {}
db3dc11356ed pushvars: move fb extension pushvars to core Pulkit Goyal <7895pulkit@gmail.com> parents: 33708 diff changeset	2607 for key, value in part.advisoryparams:
db3dc11356ed pushvars: move fb extension pushvars to core Pulkit Goyal <7895pulkit@gmail.com> parents: 33708 diff changeset	2608 key = key.upper()
db3dc11356ed pushvars: move fb extension pushvars to core Pulkit Goyal <7895pulkit@gmail.com> parents: 33708 diff changeset	2609 # We want pushed variables to have USERVAR_ prepended so we know
db3dc11356ed pushvars: move fb extension pushvars to core Pulkit Goyal <7895pulkit@gmail.com> parents: 33708 diff changeset	2610 # they came from the --pushvar flag.
43077 687b865b95ad formatting: byteify all mercurial/ and hgext/ string literals Augie Fackler <augie@google.com> parents: 43076 diff changeset	2611 key = b"USERVAR_" + key
33719 db3dc11356ed pushvars: move fb extension pushvars to core Pulkit Goyal <7895pulkit@gmail.com> parents: 33708 diff changeset	2612 hookargs[key] = value
db3dc11356ed pushvars: move fb extension pushvars to core Pulkit Goyal <7895pulkit@gmail.com> parents: 33708 diff changeset	2613 op.addhookargs(hookargs)
35758 b996ddf5963d bundle2: add a 'stream' part handler for stream cloning Boris Feld <boris.feld@octobus.net> parents: 35655 diff changeset	2614
43076 2372284d9457 formatting: blacken the codebase Augie Fackler <augie@google.com> parents: 42935 diff changeset	2615
43077 687b865b95ad formatting: byteify all mercurial/ and hgext/ string literals Augie Fackler <augie@google.com> parents: 43076 diff changeset	2616 @parthandler(b'stream2', (b'requirements', b'filecount', b'bytecount'))
35788 b116a66bcc44 bundle2: move version of stream clone into part name Gregory Szorc <gregory.szorc@gmail.com> parents: 35787 diff changeset	2617 def handlestreamv2bundle(op, part):
48821 d9ed7c5e915d streamclone: avoid some obscure error in a corner case Valentin Gatien-Baron <vgatien-baron@janestreet.com> parents: 48675 diff changeset	2618 requirements = urlreq.unquote(part.params[b'requirements'])
d9ed7c5e915d streamclone: avoid some obscure error in a corner case Valentin Gatien-Baron <vgatien-baron@janestreet.com> parents: 48675 diff changeset	2619 requirements = requirements.split(b',') if requirements else []
43077 687b865b95ad formatting: byteify all mercurial/ and hgext/ string literals Augie Fackler <augie@google.com> parents: 43076 diff changeset	2620 filecount = int(part.params[b'filecount'])
687b865b95ad formatting: byteify all mercurial/ and hgext/ string literals Augie Fackler <augie@google.com> parents: 43076 diff changeset	2621 bytecount = int(part.params[b'bytecount'])
35758 b996ddf5963d bundle2: add a 'stream' part handler for stream cloning Boris Feld <boris.feld@octobus.net> parents: 35655 diff changeset	2622
b996ddf5963d bundle2: add a 'stream' part handler for stream cloning Boris Feld <boris.feld@octobus.net> parents: 35655 diff changeset	2623 repo = op.repo
b996ddf5963d bundle2: add a 'stream' part handler for stream cloning Boris Feld <boris.feld@octobus.net> parents: 35655 diff changeset	2624 if len(repo):
43077 687b865b95ad formatting: byteify all mercurial/ and hgext/ string literals Augie Fackler <augie@google.com> parents: 43076 diff changeset	2625 msg = _(b'cannot apply stream clone to non empty repository')
35758 b996ddf5963d bundle2: add a 'stream' part handler for stream cloning Boris Feld <boris.feld@octobus.net> parents: 35655 diff changeset	2626 raise error.Abort(msg)
b996ddf5963d bundle2: add a 'stream' part handler for stream cloning Boris Feld <boris.feld@octobus.net> parents: 35655 diff changeset	2627
43077 687b865b95ad formatting: byteify all mercurial/ and hgext/ string literals Augie Fackler <augie@google.com> parents: 43076 diff changeset	2628 repo.ui.debug(b'applying stream bundle\n')
43076 2372284d9457 formatting: blacken the codebase Augie Fackler <augie@google.com> parents: 42935 diff changeset	2629 streamclone.applybundlev2(repo, part, filecount, bytecount, requirements)
2372284d9457 formatting: blacken the codebase Augie Fackler <augie@google.com> parents: 42935 diff changeset	2630
2372284d9457 formatting: blacken the codebase Augie Fackler <augie@google.com> parents: 42935 diff changeset	2631
50706 0452af304808 stream-clone: add a v3 version of the protocol Arseniy Alekseyev <aalekseyev@janestreet.com> parents: 50647 diff changeset	2632 @parthandler(b'stream3-exp', (b'requirements',))
50558 58adcabc295f stream-clone: introduce the notion of an experimental "v3" version Pierre-Yves David <pierre-yves.david@octobus.net> parents: 50553 diff changeset	2633 def handlestreamv3bundle(op, part):
50706 0452af304808 stream-clone: add a v3 version of the protocol Arseniy Alekseyev <aalekseyev@janestreet.com> parents: 50647 diff changeset	2634 requirements = urlreq.unquote(part.params[b'requirements'])
0452af304808 stream-clone: add a v3 version of the protocol Arseniy Alekseyev <aalekseyev@janestreet.com> parents: 50647 diff changeset	2635 requirements = requirements.split(b',') if requirements else []
0452af304808 stream-clone: add a v3 version of the protocol Arseniy Alekseyev <aalekseyev@janestreet.com> parents: 50647 diff changeset	2636
0452af304808 stream-clone: add a v3 version of the protocol Arseniy Alekseyev <aalekseyev@janestreet.com> parents: 50647 diff changeset	2637 repo = op.repo
0452af304808 stream-clone: add a v3 version of the protocol Arseniy Alekseyev <aalekseyev@janestreet.com> parents: 50647 diff changeset	2638 if len(repo):
0452af304808 stream-clone: add a v3 version of the protocol Arseniy Alekseyev <aalekseyev@janestreet.com> parents: 50647 diff changeset	2639 msg = _(b'cannot apply stream clone to non empty repository')
0452af304808 stream-clone: add a v3 version of the protocol Arseniy Alekseyev <aalekseyev@janestreet.com> parents: 50647 diff changeset	2640 raise error.Abort(msg)
0452af304808 stream-clone: add a v3 version of the protocol Arseniy Alekseyev <aalekseyev@janestreet.com> parents: 50647 diff changeset	2641
0452af304808 stream-clone: add a v3 version of the protocol Arseniy Alekseyev <aalekseyev@janestreet.com> parents: 50647 diff changeset	2642 repo.ui.debug(b'applying stream bundle\n')
0452af304808 stream-clone: add a v3 version of the protocol Arseniy Alekseyev <aalekseyev@janestreet.com> parents: 50647 diff changeset	2643 streamclone.applybundlev3(repo, part, requirements)
50558 58adcabc295f stream-clone: introduce the notion of an experimental "v3" version Pierre-Yves David <pierre-yves.david@octobus.net> parents: 50553 diff changeset	2644
58adcabc295f stream-clone: introduce the notion of an experimental "v3" version Pierre-Yves David <pierre-yves.david@octobus.net> parents: 50553 diff changeset	2645
43076 2372284d9457 formatting: blacken the codebase Augie Fackler <augie@google.com> parents: 42935 diff changeset	2646 def widen_bundle(
2372284d9457 formatting: blacken the codebase Augie Fackler <augie@google.com> parents: 42935 diff changeset	2647 bundler, repo, oldmatcher, newmatcher, common, known, cgversion, ellipses
2372284d9457 formatting: blacken the codebase Augie Fackler <augie@google.com> parents: 42935 diff changeset	2648 ):
40073 1ea80ac13f19 narrow: move the code to generate a widening bundle2 to core Pulkit Goyal <pulkit@yandex-team.ru> parents: 39855 diff changeset	2649 """generates bundle2 for widening a narrow clone
1ea80ac13f19 narrow: move the code to generate a widening bundle2 to core Pulkit Goyal <pulkit@yandex-team.ru> parents: 39855 diff changeset	2650
42434 a97b12f726e4 narrow: pass the bundle to bundle2.widen_bundle() instead of generating there Pulkit Goyal <7895pulkit@gmail.com> parents: 42143 diff changeset	2651 bundler is the bundle to which data should be added
40073 1ea80ac13f19 narrow: move the code to generate a widening bundle2 to core Pulkit Goyal <pulkit@yandex-team.ru> parents: 39855 diff changeset	2652 repo is the localrepository instance
40344 2c5835b4246b narrow: when widening, don't include manifests the client already has Martin von Zweigbergk <martinvonz@google.com> parents: 40338 diff changeset	2653 oldmatcher matches what the client already has
2c5835b4246b narrow: when widening, don't include manifests the client already has Martin von Zweigbergk <martinvonz@google.com> parents: 40338 diff changeset	2654 newmatcher matches what the client needs (including what it already has)
40073 1ea80ac13f19 narrow: move the code to generate a widening bundle2 to core Pulkit Goyal <pulkit@yandex-team.ru> parents: 39855 diff changeset	2655 common is set of common heads between server and client
1ea80ac13f19 narrow: move the code to generate a widening bundle2 to core Pulkit Goyal <pulkit@yandex-team.ru> parents: 39855 diff changeset	2656 known is a set of revs known on the client side (used in ellipses)
1ea80ac13f19 narrow: move the code to generate a widening bundle2 to core Pulkit Goyal <pulkit@yandex-team.ru> parents: 39855 diff changeset	2657 cgversion is the changegroup version to send
1ea80ac13f19 narrow: move the code to generate a widening bundle2 to core Pulkit Goyal <pulkit@yandex-team.ru> parents: 39855 diff changeset	2658 ellipses is boolean value telling whether to send ellipses data or not
1ea80ac13f19 narrow: move the code to generate a widening bundle2 to core Pulkit Goyal <pulkit@yandex-team.ru> parents: 39855 diff changeset	2659
1ea80ac13f19 narrow: move the code to generate a widening bundle2 to core Pulkit Goyal <pulkit@yandex-team.ru> parents: 39855 diff changeset	2660 returns bundle2 of the data required for extending
1ea80ac13f19 narrow: move the code to generate a widening bundle2 to core Pulkit Goyal <pulkit@yandex-team.ru> parents: 39855 diff changeset	2661 """
1ea80ac13f19 narrow: move the code to generate a widening bundle2 to core Pulkit Goyal <pulkit@yandex-team.ru> parents: 39855 diff changeset	2662 commonnodes = set()
1ea80ac13f19 narrow: move the code to generate a widening bundle2 to core Pulkit Goyal <pulkit@yandex-team.ru> parents: 39855 diff changeset	2663 cl = repo.changelog
43077 687b865b95ad formatting: byteify all mercurial/ and hgext/ string literals Augie Fackler <augie@google.com> parents: 43076 diff changeset	2664 for r in repo.revs(b"::%ln", common):
40073 1ea80ac13f19 narrow: move the code to generate a widening bundle2 to core Pulkit Goyal <pulkit@yandex-team.ru> parents: 39855 diff changeset	2665 commonnodes.add(cl.node(r))
1ea80ac13f19 narrow: move the code to generate a widening bundle2 to core Pulkit Goyal <pulkit@yandex-team.ru> parents: 39855 diff changeset	2666 if commonnodes:
43076 2372284d9457 formatting: blacken the codebase Augie Fackler <augie@google.com> parents: 42935 diff changeset	2667 packer = changegroup.getbundler(
2372284d9457 formatting: blacken the codebase Augie Fackler <augie@google.com> parents: 42935 diff changeset	2668 cgversion,
2372284d9457 formatting: blacken the codebase Augie Fackler <augie@google.com> parents: 42935 diff changeset	2669 repo,
2372284d9457 formatting: blacken the codebase Augie Fackler <augie@google.com> parents: 42935 diff changeset	2670 oldmatcher=oldmatcher,
2372284d9457 formatting: blacken the codebase Augie Fackler <augie@google.com> parents: 42935 diff changeset	2671 matcher=newmatcher,
2372284d9457 formatting: blacken the codebase Augie Fackler <augie@google.com> parents: 42935 diff changeset	2672 fullnodes=commonnodes,
2372284d9457 formatting: blacken the codebase Augie Fackler <augie@google.com> parents: 42935 diff changeset	2673 )
2372284d9457 formatting: blacken the codebase Augie Fackler <augie@google.com> parents: 42935 diff changeset	2674 cgdata = packer.generate(
47055 d55b71393907 node: replace nullid and friends with nodeconstants class Joerg Sonnenberger <joerg@bec.de> parents: 46907 diff changeset	2675 {repo.nullid},
43076 2372284d9457 formatting: blacken the codebase Augie Fackler <augie@google.com> parents: 42935 diff changeset	2676 list(commonnodes),
2372284d9457 formatting: blacken the codebase Augie Fackler <augie@google.com> parents: 42935 diff changeset	2677 False,
43077 687b865b95ad formatting: byteify all mercurial/ and hgext/ string literals Augie Fackler <augie@google.com> parents: 43076 diff changeset	2678 b'narrow_widen',
43076 2372284d9457 formatting: blacken the codebase Augie Fackler <augie@google.com> parents: 42935 diff changeset	2679 changelog=False,
2372284d9457 formatting: blacken the codebase Augie Fackler <augie@google.com> parents: 42935 diff changeset	2680 )
40073 1ea80ac13f19 narrow: move the code to generate a widening bundle2 to core Pulkit Goyal <pulkit@yandex-team.ru> parents: 39855 diff changeset	2681
43077 687b865b95ad formatting: byteify all mercurial/ and hgext/ string literals Augie Fackler <augie@google.com> parents: 43076 diff changeset	2682 part = bundler.newpart(b'changegroup', data=cgdata)
687b865b95ad formatting: byteify all mercurial/ and hgext/ string literals Augie Fackler <augie@google.com> parents: 43076 diff changeset	2683 part.addparam(b'version', cgversion)
45558 10284ce3d5ed scmutil: introduce function to check whether repo uses treemanifest or not Pulkit Goyal <7895pulkit@gmail.com> parents: 45392 diff changeset	2684 if scmutil.istreemanifest(repo):
43077 687b865b95ad formatting: byteify all mercurial/ and hgext/ string literals Augie Fackler <augie@google.com> parents: 43076 diff changeset	2685 part.addparam(b'treemanifest', b'1')
47237 19d4802cb304 sidedata: add a 'side-data' repository feature and use it Pierre-Yves David <pierre-yves.david@octobus.net> parents: 47142 diff changeset	2686 if repository.REPO_FEATURE_SIDE_DATA in repo.features:
19d4802cb304 sidedata: add a 'side-data' repository feature and use it Pierre-Yves David <pierre-yves.david@octobus.net> parents: 47142 diff changeset	2687 part.addparam(b'exp-sidedata', b'1')
19d4802cb304 sidedata: add a 'side-data' repository feature and use it Pierre-Yves David <pierre-yves.david@octobus.net> parents: 47142 diff changeset	2688 wanted = format_remote_wanted_sidedata(repo)
19d4802cb304 sidedata: add a 'side-data' repository feature and use it Pierre-Yves David <pierre-yves.david@octobus.net> parents: 47142 diff changeset	2689 part.addparam(b'exp-wanted-sidedata', wanted)
40073 1ea80ac13f19 narrow: move the code to generate a widening bundle2 to core Pulkit Goyal <pulkit@yandex-team.ru> parents: 39855 diff changeset	2690
1ea80ac13f19 narrow: move the code to generate a widening bundle2 to core Pulkit Goyal <pulkit@yandex-team.ru> parents: 39855 diff changeset	2691 return bundler

Mercurial > public > mercurial-scm > hg-stable

annotate mercurial/bundle2.py @ 52928:f863fc99bef9