mercurial-scm/hg: mercurial/bundle2.py annotate

annotate mercurial/bundle2.py @ 36963:9988fc10f49e

revbranchcache: add a bundle2 handler for a rbc part We add the necessary bit to process a part containing rev-branch-cache information. The local rev branch cache is then updated with the received information. Computing branch cache can become a significant part of the time spent pulling.

author	Boris Feld <boris.feld@octobus.net>
date	Wed, 21 Feb 2018 17:35:04 +0100
parents	0b697c650b04
children	79b73be4dda5

rev	line source
20801 9c5183cb9bca bundle2: very first version of a bundle2 bundler Pierre-Yves David <pierre-yves.david@fb.com> parents: diff changeset	1 # bundle2.py - generic container format to transmit arbitrary data.
9c5183cb9bca bundle2: very first version of a bundle2 bundler Pierre-Yves David <pierre-yves.david@fb.com> parents: diff changeset	2 #
9c5183cb9bca bundle2: very first version of a bundle2 bundler Pierre-Yves David <pierre-yves.david@fb.com> parents: diff changeset	3 # Copyright 2013 Facebook, Inc.
9c5183cb9bca bundle2: very first version of a bundle2 bundler Pierre-Yves David <pierre-yves.david@fb.com> parents: diff changeset	4 #
9c5183cb9bca bundle2: very first version of a bundle2 bundler Pierre-Yves David <pierre-yves.david@fb.com> parents: diff changeset	5 # This software may be used and distributed according to the terms of the
9c5183cb9bca bundle2: very first version of a bundle2 bundler Pierre-Yves David <pierre-yves.david@fb.com> parents: diff changeset	6 # GNU General Public License version 2 or any later version.
9c5183cb9bca bundle2: very first version of a bundle2 bundler Pierre-Yves David <pierre-yves.david@fb.com> parents: diff changeset	7 """Handling of the new bundle2 format
9c5183cb9bca bundle2: very first version of a bundle2 bundler Pierre-Yves David <pierre-yves.david@fb.com> parents: diff changeset	8
9c5183cb9bca bundle2: very first version of a bundle2 bundler Pierre-Yves David <pierre-yves.david@fb.com> parents: diff changeset	9 The goal of bundle2 is to act as an atomically packet to transmit a set of
9c5183cb9bca bundle2: very first version of a bundle2 bundler Pierre-Yves David <pierre-yves.david@fb.com> parents: diff changeset	10 payloads in an application agnostic way. It consist in a sequence of "parts"
9c5183cb9bca bundle2: very first version of a bundle2 bundler Pierre-Yves David <pierre-yves.david@fb.com> parents: diff changeset	11 that will be handed to and processed by the application layer.
9c5183cb9bca bundle2: very first version of a bundle2 bundler Pierre-Yves David <pierre-yves.david@fb.com> parents: diff changeset	12
9c5183cb9bca bundle2: very first version of a bundle2 bundler Pierre-Yves David <pierre-yves.david@fb.com> parents: diff changeset	13
9c5183cb9bca bundle2: very first version of a bundle2 bundler Pierre-Yves David <pierre-yves.david@fb.com> parents: diff changeset	14 General format architecture
9c5183cb9bca bundle2: very first version of a bundle2 bundler Pierre-Yves David <pierre-yves.david@fb.com> parents: diff changeset	15 ===========================
9c5183cb9bca bundle2: very first version of a bundle2 bundler Pierre-Yves David <pierre-yves.david@fb.com> parents: diff changeset	16
9c5183cb9bca bundle2: very first version of a bundle2 bundler Pierre-Yves David <pierre-yves.david@fb.com> parents: diff changeset	17 The format is architectured as follow
9c5183cb9bca bundle2: very first version of a bundle2 bundler Pierre-Yves David <pierre-yves.david@fb.com> parents: diff changeset	18
9c5183cb9bca bundle2: very first version of a bundle2 bundler Pierre-Yves David <pierre-yves.david@fb.com> parents: diff changeset	19 - magic string
9c5183cb9bca bundle2: very first version of a bundle2 bundler Pierre-Yves David <pierre-yves.david@fb.com> parents: diff changeset	20 - stream level parameters
9c5183cb9bca bundle2: very first version of a bundle2 bundler Pierre-Yves David <pierre-yves.david@fb.com> parents: diff changeset	21 - payload parts (any number)
9c5183cb9bca bundle2: very first version of a bundle2 bundler Pierre-Yves David <pierre-yves.david@fb.com> parents: diff changeset	22 - end of stream marker.
9c5183cb9bca bundle2: very first version of a bundle2 bundler Pierre-Yves David <pierre-yves.david@fb.com> parents: diff changeset	23
20856 8a6a86c9a5b5 bundle2: support bundling of empty part (with a type) Pierre-Yves David <pierre-yves.david@fb.com> parents: 20844 diff changeset	24 the Binary format
20801 9c5183cb9bca bundle2: very first version of a bundle2 bundler Pierre-Yves David <pierre-yves.david@fb.com> parents: diff changeset	25 ============================
9c5183cb9bca bundle2: very first version of a bundle2 bundler Pierre-Yves David <pierre-yves.david@fb.com> parents: diff changeset	26
21024 7731a2281cf0 spelling: fixes from spell checker Mads Kiilerich <madski@unity3d.com> parents: 21020 diff changeset	27 All numbers are unsigned and big-endian.
20801 9c5183cb9bca bundle2: very first version of a bundle2 bundler Pierre-Yves David <pierre-yves.david@fb.com> parents: diff changeset	28
9c5183cb9bca bundle2: very first version of a bundle2 bundler Pierre-Yves David <pierre-yves.david@fb.com> parents: diff changeset	29 stream level parameters
9c5183cb9bca bundle2: very first version of a bundle2 bundler Pierre-Yves David <pierre-yves.david@fb.com> parents: diff changeset	30 ------------------------
9c5183cb9bca bundle2: very first version of a bundle2 bundler Pierre-Yves David <pierre-yves.david@fb.com> parents: diff changeset	31
9c5183cb9bca bundle2: very first version of a bundle2 bundler Pierre-Yves David <pierre-yves.david@fb.com> parents: diff changeset	32 Binary format is as follow
9c5183cb9bca bundle2: very first version of a bundle2 bundler Pierre-Yves David <pierre-yves.david@fb.com> parents: diff changeset	33
23009 90f86ad3d4ff bundle2: change header size and make them signed (new format) Pierre-Yves David <pierre-yves.david@fb.com> parents: 23008 diff changeset	34 :params size: int32
20801 9c5183cb9bca bundle2: very first version of a bundle2 bundler Pierre-Yves David <pierre-yves.david@fb.com> parents: diff changeset	35
9c5183cb9bca bundle2: very first version of a bundle2 bundler Pierre-Yves David <pierre-yves.david@fb.com> parents: diff changeset	36 The total number of Bytes used by the parameters
9c5183cb9bca bundle2: very first version of a bundle2 bundler Pierre-Yves David <pierre-yves.david@fb.com> parents: diff changeset	37
9c5183cb9bca bundle2: very first version of a bundle2 bundler Pierre-Yves David <pierre-yves.david@fb.com> parents: diff changeset	38 :params value: arbitrary number of Bytes
9c5183cb9bca bundle2: very first version of a bundle2 bundler Pierre-Yves David <pierre-yves.david@fb.com> parents: diff changeset	39
9c5183cb9bca bundle2: very first version of a bundle2 bundler Pierre-Yves David <pierre-yves.david@fb.com> parents: diff changeset	40 A blob of `params size` containing the serialized version of all stream level
9c5183cb9bca bundle2: very first version of a bundle2 bundler Pierre-Yves David <pierre-yves.david@fb.com> parents: diff changeset	41 parameters.
9c5183cb9bca bundle2: very first version of a bundle2 bundler Pierre-Yves David <pierre-yves.david@fb.com> parents: diff changeset	42
21024 7731a2281cf0 spelling: fixes from spell checker Mads Kiilerich <madski@unity3d.com> parents: 21020 diff changeset	43 The blob contains a space separated list of parameters. Parameters with value
20811 9785c3f8f598 bundle2: urlquote stream parameter name and value Pierre-Yves David <pierre-yves.david@fb.com> parents: 20810 diff changeset	44 are stored in the form `<name>=<value>`. Both name and value are urlquoted.
20804 db9d3991d2c6 bundle2: support bundling simple parameter Pierre-Yves David <pierre-yves.david@fb.com> parents: 20803 diff changeset	45
20813 8c74b3ce5b70 bundle2: refuse empty parameter name Pierre-Yves David <pierre-yves.david@fb.com> parents: 20812 diff changeset	46 Empty name are obviously forbidden.
8c74b3ce5b70 bundle2: refuse empty parameter name Pierre-Yves David <pierre-yves.david@fb.com> parents: 20812 diff changeset	47
20844 2631204d7305 bundle2: implement the mandatory/advisory logic for parameter Pierre-Yves David <pierre-yves.david@fb.com> parents: 20843 diff changeset	48 Name MUST start with a letter. If this first letter is lower case, the
21024 7731a2281cf0 spelling: fixes from spell checker Mads Kiilerich <madski@unity3d.com> parents: 21020 diff changeset	49 parameter is advisory and can be safely ignored. However when the first
20844 2631204d7305 bundle2: implement the mandatory/advisory logic for parameter Pierre-Yves David <pierre-yves.david@fb.com> parents: 20843 diff changeset	50 letter is capital, the parameter is mandatory and the bundling process MUST
2631204d7305 bundle2: implement the mandatory/advisory logic for parameter Pierre-Yves David <pierre-yves.david@fb.com> parents: 20843 diff changeset	51 stop if he is not able to proceed it.
20814 8532f5e1b9df bundle2: force the first char of parameter to be an letter. Pierre-Yves David <pierre-yves.david@fb.com> parents: 20813 diff changeset	52
20808 4c9130c7a29f bundle2: clarify stream parameter design in the documentation Pierre-Yves David <pierre-yves.david@fb.com> parents: 20805 diff changeset	53 Stream parameters use a simple textual format for two main reasons:
20804 db9d3991d2c6 bundle2: support bundling simple parameter Pierre-Yves David <pierre-yves.david@fb.com> parents: 20803 diff changeset	54
21024 7731a2281cf0 spelling: fixes from spell checker Mads Kiilerich <madski@unity3d.com> parents: 21020 diff changeset	55 - Stream level parameters should remain simple and we want to discourage any
20808 4c9130c7a29f bundle2: clarify stream parameter design in the documentation Pierre-Yves David <pierre-yves.david@fb.com> parents: 20805 diff changeset	56 crazy usage.
21024 7731a2281cf0 spelling: fixes from spell checker Mads Kiilerich <madski@unity3d.com> parents: 21020 diff changeset	57 - Textual data allow easy human inspection of a bundle2 header in case of
20808 4c9130c7a29f bundle2: clarify stream parameter design in the documentation Pierre-Yves David <pierre-yves.david@fb.com> parents: 20805 diff changeset	58 troubles.
4c9130c7a29f bundle2: clarify stream parameter design in the documentation Pierre-Yves David <pierre-yves.david@fb.com> parents: 20805 diff changeset	59
4c9130c7a29f bundle2: clarify stream parameter design in the documentation Pierre-Yves David <pierre-yves.david@fb.com> parents: 20805 diff changeset	60 Any Applicative level options MUST go into a bundle2 part instead.
20801 9c5183cb9bca bundle2: very first version of a bundle2 bundler Pierre-Yves David <pierre-yves.david@fb.com> parents: diff changeset	61
9c5183cb9bca bundle2: very first version of a bundle2 bundler Pierre-Yves David <pierre-yves.david@fb.com> parents: diff changeset	62 Payload part
9c5183cb9bca bundle2: very first version of a bundle2 bundler Pierre-Yves David <pierre-yves.david@fb.com> parents: diff changeset	63 ------------------------
9c5183cb9bca bundle2: very first version of a bundle2 bundler Pierre-Yves David <pierre-yves.david@fb.com> parents: diff changeset	64
9c5183cb9bca bundle2: very first version of a bundle2 bundler Pierre-Yves David <pierre-yves.david@fb.com> parents: diff changeset	65 Binary format is as follow
9c5183cb9bca bundle2: very first version of a bundle2 bundler Pierre-Yves David <pierre-yves.david@fb.com> parents: diff changeset	66
23009 90f86ad3d4ff bundle2: change header size and make them signed (new format) Pierre-Yves David <pierre-yves.david@fb.com> parents: 23008 diff changeset	67 :header size: int32
20801 9c5183cb9bca bundle2: very first version of a bundle2 bundler Pierre-Yves David <pierre-yves.david@fb.com> parents: diff changeset	68
25507 5bee483705e8 bundle2: clarify in docstring that header size is for a single header Martin von Zweigbergk <martinvonz@google.com> parents: 25506 diff changeset	69 The total number of Bytes used by the part header. When the header is empty
20801 9c5183cb9bca bundle2: very first version of a bundle2 bundler Pierre-Yves David <pierre-yves.david@fb.com> parents: diff changeset	70 (size = 0) this is interpreted as the end of stream marker.
9c5183cb9bca bundle2: very first version of a bundle2 bundler Pierre-Yves David <pierre-yves.david@fb.com> parents: diff changeset	71
20856 8a6a86c9a5b5 bundle2: support bundling of empty part (with a type) Pierre-Yves David <pierre-yves.david@fb.com> parents: 20844 diff changeset	72 :header:
8a6a86c9a5b5 bundle2: support bundling of empty part (with a type) Pierre-Yves David <pierre-yves.david@fb.com> parents: 20844 diff changeset	73
8a6a86c9a5b5 bundle2: support bundling of empty part (with a type) Pierre-Yves David <pierre-yves.david@fb.com> parents: 20844 diff changeset	74 The header defines how to interpret the part. It contains two piece of
8a6a86c9a5b5 bundle2: support bundling of empty part (with a type) Pierre-Yves David <pierre-yves.david@fb.com> parents: 20844 diff changeset	75 data: the part type, and the part parameters.
8a6a86c9a5b5 bundle2: support bundling of empty part (with a type) Pierre-Yves David <pierre-yves.david@fb.com> parents: 20844 diff changeset	76
8a6a86c9a5b5 bundle2: support bundling of empty part (with a type) Pierre-Yves David <pierre-yves.david@fb.com> parents: 20844 diff changeset	77 The part type is used to route an application level handler, that can
8a6a86c9a5b5 bundle2: support bundling of empty part (with a type) Pierre-Yves David <pierre-yves.david@fb.com> parents: 20844 diff changeset	78 interpret payload.
8a6a86c9a5b5 bundle2: support bundling of empty part (with a type) Pierre-Yves David <pierre-yves.david@fb.com> parents: 20844 diff changeset	79
8a6a86c9a5b5 bundle2: support bundling of empty part (with a type) Pierre-Yves David <pierre-yves.david@fb.com> parents: 20844 diff changeset	80 Part parameters are passed to the application level handler. They are
8a6a86c9a5b5 bundle2: support bundling of empty part (with a type) Pierre-Yves David <pierre-yves.david@fb.com> parents: 20844 diff changeset	81 meant to convey information that will help the application level object to
8a6a86c9a5b5 bundle2: support bundling of empty part (with a type) Pierre-Yves David <pierre-yves.david@fb.com> parents: 20844 diff changeset	82 interpret the part payload.
8a6a86c9a5b5 bundle2: support bundling of empty part (with a type) Pierre-Yves David <pierre-yves.david@fb.com> parents: 20844 diff changeset	83
8a6a86c9a5b5 bundle2: support bundling of empty part (with a type) Pierre-Yves David <pierre-yves.david@fb.com> parents: 20844 diff changeset	84 The binary format of the header is has follow
8a6a86c9a5b5 bundle2: support bundling of empty part (with a type) Pierre-Yves David <pierre-yves.david@fb.com> parents: 20844 diff changeset	85
8a6a86c9a5b5 bundle2: support bundling of empty part (with a type) Pierre-Yves David <pierre-yves.david@fb.com> parents: 20844 diff changeset	86 :typesize: (one byte)
20877 9e9e3a4e9261 bundle2: part params Pierre-Yves David <pierre-yves.david@fb.com> parents: 20876 diff changeset	87
23916 a3f7c781786b bundle2: fix parttype enforcement Matt Mackall <mpm@selenic.com> parents: 23868 diff changeset	88 :parttype: alphanumerical part name (restricted to [a-zA-Z0-9_:-]*)
20877 9e9e3a4e9261 bundle2: part params Pierre-Yves David <pierre-yves.david@fb.com> parents: 20876 diff changeset	89
20995 e995d104c87f bundle2: add an integer id to part Pierre-Yves David <pierre-yves.david@fb.com> parents: 20952 diff changeset	90 :partid: A 32bits integer (unique in the bundle) that can be used to refer
e995d104c87f bundle2: add an integer id to part Pierre-Yves David <pierre-yves.david@fb.com> parents: 20952 diff changeset	91 to this part.
e995d104c87f bundle2: add an integer id to part Pierre-Yves David <pierre-yves.david@fb.com> parents: 20952 diff changeset	92
20877 9e9e3a4e9261 bundle2: part params Pierre-Yves David <pierre-yves.david@fb.com> parents: 20876 diff changeset	93 :parameters:
9e9e3a4e9261 bundle2: part params Pierre-Yves David <pierre-yves.david@fb.com> parents: 20876 diff changeset	94
21024 7731a2281cf0 spelling: fixes from spell checker Mads Kiilerich <madski@unity3d.com> parents: 21020 diff changeset	95 Part's parameter may have arbitrary content, the binary structure is::
20877 9e9e3a4e9261 bundle2: part params Pierre-Yves David <pierre-yves.david@fb.com> parents: 20876 diff changeset	96
9e9e3a4e9261 bundle2: part params Pierre-Yves David <pierre-yves.david@fb.com> parents: 20876 diff changeset	97 <mandatory-count><advisory-count><param-sizes><param-data>
9e9e3a4e9261 bundle2: part params Pierre-Yves David <pierre-yves.david@fb.com> parents: 20876 diff changeset	98
9e9e3a4e9261 bundle2: part params Pierre-Yves David <pierre-yves.david@fb.com> parents: 20876 diff changeset	99 :mandatory-count: 1 byte, number of mandatory parameters
9e9e3a4e9261 bundle2: part params Pierre-Yves David <pierre-yves.david@fb.com> parents: 20876 diff changeset	100
9e9e3a4e9261 bundle2: part params Pierre-Yves David <pierre-yves.david@fb.com> parents: 20876 diff changeset	101 :advisory-count: 1 byte, number of advisory parameters
9e9e3a4e9261 bundle2: part params Pierre-Yves David <pierre-yves.david@fb.com> parents: 20876 diff changeset	102
9e9e3a4e9261 bundle2: part params Pierre-Yves David <pierre-yves.david@fb.com> parents: 20876 diff changeset	103 :param-sizes:
9e9e3a4e9261 bundle2: part params Pierre-Yves David <pierre-yves.david@fb.com> parents: 20876 diff changeset	104
9e9e3a4e9261 bundle2: part params Pierre-Yves David <pierre-yves.david@fb.com> parents: 20876 diff changeset	105 N couple of bytes, where N is the total number of parameters. Each
9e9e3a4e9261 bundle2: part params Pierre-Yves David <pierre-yves.david@fb.com> parents: 20876 diff changeset	106 couple contains (<size-of-key>, <size-of-value) for one parameter.
9e9e3a4e9261 bundle2: part params Pierre-Yves David <pierre-yves.david@fb.com> parents: 20876 diff changeset	107
9e9e3a4e9261 bundle2: part params Pierre-Yves David <pierre-yves.david@fb.com> parents: 20876 diff changeset	108 :param-data:
9e9e3a4e9261 bundle2: part params Pierre-Yves David <pierre-yves.david@fb.com> parents: 20876 diff changeset	109
9e9e3a4e9261 bundle2: part params Pierre-Yves David <pierre-yves.david@fb.com> parents: 20876 diff changeset	110 A blob of bytes from which each parameter key and value can be
9e9e3a4e9261 bundle2: part params Pierre-Yves David <pierre-yves.david@fb.com> parents: 20876 diff changeset	111 retrieved using the list of size couples stored in the previous
9e9e3a4e9261 bundle2: part params Pierre-Yves David <pierre-yves.david@fb.com> parents: 20876 diff changeset	112 field.
9e9e3a4e9261 bundle2: part params Pierre-Yves David <pierre-yves.david@fb.com> parents: 20876 diff changeset	113
9e9e3a4e9261 bundle2: part params Pierre-Yves David <pierre-yves.david@fb.com> parents: 20876 diff changeset	114 Mandatory parameters comes first, then the advisory ones.
20856 8a6a86c9a5b5 bundle2: support bundling of empty part (with a type) Pierre-Yves David <pierre-yves.david@fb.com> parents: 20844 diff changeset	115
21607 054fa5176fa7 bundle2: forbid duplicate parameter keys Pierre-Yves David <pierre-yves.david@fb.com> parents: 21606 diff changeset	116 Each parameter's key MUST be unique within the part.
054fa5176fa7 bundle2: forbid duplicate parameter keys Pierre-Yves David <pierre-yves.david@fb.com> parents: 21606 diff changeset	117
20856 8a6a86c9a5b5 bundle2: support bundling of empty part (with a type) Pierre-Yves David <pierre-yves.david@fb.com> parents: 20844 diff changeset	118 :payload:
8a6a86c9a5b5 bundle2: support bundling of empty part (with a type) Pierre-Yves David <pierre-yves.david@fb.com> parents: 20844 diff changeset	119
20876 ddd56f3eb786 bundle2: support for bundling and unbundling payload Pierre-Yves David <pierre-yves.david@fb.com> parents: 20864 diff changeset	120 payload is a series of `<chunksize><chunkdata>`.
ddd56f3eb786 bundle2: support for bundling and unbundling payload Pierre-Yves David <pierre-yves.david@fb.com> parents: 20864 diff changeset	121
23009 90f86ad3d4ff bundle2: change header size and make them signed (new format) Pierre-Yves David <pierre-yves.david@fb.com> parents: 23008 diff changeset	122 `chunksize` is an int32, `chunkdata` are plain bytes (as much as
20876 ddd56f3eb786 bundle2: support for bundling and unbundling payload Pierre-Yves David <pierre-yves.david@fb.com> parents: 20864 diff changeset	123 `chunksize` says)` The payload part is concluded by a zero size chunk.
ddd56f3eb786 bundle2: support for bundling and unbundling payload Pierre-Yves David <pierre-yves.david@fb.com> parents: 20864 diff changeset	124
ddd56f3eb786 bundle2: support for bundling and unbundling payload Pierre-Yves David <pierre-yves.david@fb.com> parents: 20864 diff changeset	125 The current implementation always produces either zero or one chunk.
21024 7731a2281cf0 spelling: fixes from spell checker Mads Kiilerich <madski@unity3d.com> parents: 21020 diff changeset	126 This is an implementation limitation that will ultimately be lifted.
20891 1c6cd23fc221 bundle2: add some distinction between mandatory and advisory part Pierre-Yves David <pierre-yves.david@fb.com> parents: 20890 diff changeset	127
23009 90f86ad3d4ff bundle2: change header size and make them signed (new format) Pierre-Yves David <pierre-yves.david@fb.com> parents: 23008 diff changeset	128 `chunksize` can be negative to trigger special case processing. No such
90f86ad3d4ff bundle2: change header size and make them signed (new format) Pierre-Yves David <pierre-yves.david@fb.com> parents: 23008 diff changeset	129 processing is in place yet.
90f86ad3d4ff bundle2: change header size and make them signed (new format) Pierre-Yves David <pierre-yves.david@fb.com> parents: 23008 diff changeset	130
20891 1c6cd23fc221 bundle2: add some distinction between mandatory and advisory part Pierre-Yves David <pierre-yves.david@fb.com> parents: 20890 diff changeset	131 Bundle processing
1c6cd23fc221 bundle2: add some distinction between mandatory and advisory part Pierre-Yves David <pierre-yves.david@fb.com> parents: 20890 diff changeset	132 ============================
1c6cd23fc221 bundle2: add some distinction between mandatory and advisory part Pierre-Yves David <pierre-yves.david@fb.com> parents: 20890 diff changeset	133
1c6cd23fc221 bundle2: add some distinction between mandatory and advisory part Pierre-Yves David <pierre-yves.david@fb.com> parents: 20890 diff changeset	134 Each part is processed in order using a "part handler". Handler are registered
1c6cd23fc221 bundle2: add some distinction between mandatory and advisory part Pierre-Yves David <pierre-yves.david@fb.com> parents: 20890 diff changeset	135 for a certain part type.
1c6cd23fc221 bundle2: add some distinction between mandatory and advisory part Pierre-Yves David <pierre-yves.david@fb.com> parents: 20890 diff changeset	136
1c6cd23fc221 bundle2: add some distinction between mandatory and advisory part Pierre-Yves David <pierre-yves.david@fb.com> parents: 20890 diff changeset	137 The matching of a part to its handler is case insensitive. The case of the
1c6cd23fc221 bundle2: add some distinction between mandatory and advisory part Pierre-Yves David <pierre-yves.david@fb.com> parents: 20890 diff changeset	138 part type is used to know if a part is mandatory or advisory. If the Part type
1c6cd23fc221 bundle2: add some distinction between mandatory and advisory part Pierre-Yves David <pierre-yves.david@fb.com> parents: 20890 diff changeset	139 contains any uppercase char it is considered mandatory. When no handler is
1c6cd23fc221 bundle2: add some distinction between mandatory and advisory part Pierre-Yves David <pierre-yves.david@fb.com> parents: 20890 diff changeset	140 known for a Mandatory part, the process is aborted and an exception is raised.
20892 6fe95448596d bundle2: read the whole bundle from stream on abort Pierre-Yves David <pierre-yves.david@fb.com> parents: 20891 diff changeset	141 If the part is advisory and no handler is known, the part is ignored. When the
6fe95448596d bundle2: read the whole bundle from stream on abort Pierre-Yves David <pierre-yves.david@fb.com> parents: 20891 diff changeset	142 process is aborted, the full bundle is still read from the stream to keep the
6fe95448596d bundle2: read the whole bundle from stream on abort Pierre-Yves David <pierre-yves.david@fb.com> parents: 20891 diff changeset	143 channel usable. But none of the part read from an abort are processed. In the
6fe95448596d bundle2: read the whole bundle from stream on abort Pierre-Yves David <pierre-yves.david@fb.com> parents: 20891 diff changeset	144 future, dropping the stream may become an option for channel we do not care to
6fe95448596d bundle2: read the whole bundle from stream on abort Pierre-Yves David <pierre-yves.david@fb.com> parents: 20891 diff changeset	145 preserve.
20801 9c5183cb9bca bundle2: very first version of a bundle2 bundler Pierre-Yves David <pierre-yves.david@fb.com> parents: diff changeset	146 """
9c5183cb9bca bundle2: very first version of a bundle2 bundler Pierre-Yves David <pierre-yves.david@fb.com> parents: diff changeset	147
33672 da7c285ec6da bundle2: use modern Python division Augie Fackler <augie@google.com> parents: 33665 diff changeset	148 from __future__ import absolute_import, division
25919 8221fefaea08 bundle2: use absolute_import Gregory Szorc <gregory.szorc@gmail.com> parents: 25660 diff changeset	149
24026 3daef83a1873 bundle2.unpackermixin: control for underlying file descriptor Eric Sumner <ericsumner@fb.com> parents: 23916 diff changeset	150 import errno
35046 241d9caca11e bundle2: use os.SEEK_* constants Gregory Szorc <gregory.szorc@gmail.com> parents: 34820 diff changeset	151 import os
25919 8221fefaea08 bundle2: use absolute_import Gregory Szorc <gregory.szorc@gmail.com> parents: 25660 diff changeset	152 import re
20814 8532f5e1b9df bundle2: force the first char of parameter to be an letter. Pierre-Yves David <pierre-yves.david@fb.com> parents: 20813 diff changeset	153 import string
25919 8221fefaea08 bundle2: use absolute_import Gregory Szorc <gregory.szorc@gmail.com> parents: 25660 diff changeset	154 import struct
8221fefaea08 bundle2: use absolute_import Gregory Szorc <gregory.szorc@gmail.com> parents: 25660 diff changeset	155 import sys
20804 db9d3991d2c6 bundle2: support bundling simple parameter Pierre-Yves David <pierre-yves.david@fb.com> parents: 20803 diff changeset	156
25919 8221fefaea08 bundle2: use absolute_import Gregory Szorc <gregory.szorc@gmail.com> parents: 25660 diff changeset	157 from .i18n import _
8221fefaea08 bundle2: use absolute_import Gregory Szorc <gregory.szorc@gmail.com> parents: 25660 diff changeset	158 from . import (
35258 dbf868623daf bookmark: add a 'check:bookmarks' bundle2 part Boris Feld <boris.feld@octobus.net> parents: 35120 diff changeset	159 bookmarks,
25919 8221fefaea08 bundle2: use absolute_import Gregory Szorc <gregory.szorc@gmail.com> parents: 25660 diff changeset	160 changegroup,
36963 9988fc10f49e revbranchcache: add a bundle2 handler for a rbc part Boris Feld <boris.feld@octobus.net> parents: 36428 diff changeset	161 encoding,
25919 8221fefaea08 bundle2: use absolute_import Gregory Szorc <gregory.szorc@gmail.com> parents: 25660 diff changeset	162 error,
34820 a95067b1dca6 phase: introduce a new 'check:phases' part Boris Feld <boris.feld@octobus.net> parents: 34637 diff changeset	163 node as nodemod,
25919 8221fefaea08 bundle2: use absolute_import Gregory Szorc <gregory.szorc@gmail.com> parents: 25660 diff changeset	164 obsolete,
33031 e8c8d81eb864 bundle: add config option to include phases Martin von Zweigbergk <martinvonz@google.com> parents: 33030 diff changeset	165 phases,
25919 8221fefaea08 bundle2: use absolute_import Gregory Szorc <gregory.szorc@gmail.com> parents: 25660 diff changeset	166 pushkey,
30030 0f6d6fdd3c2a pycompat: provide 'ispy3' constant Yuya Nishihara <yuya@tcha.org> parents: 29937 diff changeset	167 pycompat,
35758 b996ddf5963d bundle2: add a 'stream' part handler for stream cloning Boris Feld <boris.feld@octobus.net> parents: 35655 diff changeset	168 streamclone,
25919 8221fefaea08 bundle2: use absolute_import Gregory Szorc <gregory.szorc@gmail.com> parents: 25660 diff changeset	169 tags,
8221fefaea08 bundle2: use absolute_import Gregory Szorc <gregory.szorc@gmail.com> parents: 25660 diff changeset	170 url,
8221fefaea08 bundle2: use absolute_import Gregory Szorc <gregory.szorc@gmail.com> parents: 25660 diff changeset	171 util,
8221fefaea08 bundle2: use absolute_import Gregory Szorc <gregory.szorc@gmail.com> parents: 25660 diff changeset	172 )
20802 520df53ad26a bundle2: a very first version of bundle2 unbundler Pierre-Yves David <pierre-yves.david@fb.com> parents: 20801 diff changeset	173
28883 032c4c2f802a pycompat: switch to util.urlreq/util.urlerr for py3 compat timeless <timeless@mozdev.org> parents: 28672 diff changeset	174 urlerr = util.urlerr
032c4c2f802a pycompat: switch to util.urlreq/util.urlerr for py3 compat timeless <timeless@mozdev.org> parents: 28672 diff changeset	175 urlreq = util.urlreq
032c4c2f802a pycompat: switch to util.urlreq/util.urlerr for py3 compat timeless <timeless@mozdev.org> parents: 28672 diff changeset	176
20804 db9d3991d2c6 bundle2: support bundling simple parameter Pierre-Yves David <pierre-yves.david@fb.com> parents: 20803 diff changeset	177 _pack = struct.pack
db9d3991d2c6 bundle2: support bundling simple parameter Pierre-Yves David <pierre-yves.david@fb.com> parents: 20803 diff changeset	178 _unpack = struct.unpack
db9d3991d2c6 bundle2: support bundling simple parameter Pierre-Yves David <pierre-yves.david@fb.com> parents: 20803 diff changeset	179
23009 90f86ad3d4ff bundle2: change header size and make them signed (new format) Pierre-Yves David <pierre-yves.david@fb.com> parents: 23008 diff changeset	180 _fstreamparamsize = '>i'
90f86ad3d4ff bundle2: change header size and make them signed (new format) Pierre-Yves David <pierre-yves.david@fb.com> parents: 23008 diff changeset	181 _fpartheadersize = '>i'
20856 8a6a86c9a5b5 bundle2: support bundling of empty part (with a type) Pierre-Yves David <pierre-yves.david@fb.com> parents: 20844 diff changeset	182 _fparttypesize = '>B'
20995 e995d104c87f bundle2: add an integer id to part Pierre-Yves David <pierre-yves.david@fb.com> parents: 20952 diff changeset	183 _fpartid = '>I'
23009 90f86ad3d4ff bundle2: change header size and make them signed (new format) Pierre-Yves David <pierre-yves.david@fb.com> parents: 23008 diff changeset	184 _fpayloadsize = '>i'
20877 9e9e3a4e9261 bundle2: part params Pierre-Yves David <pierre-yves.david@fb.com> parents: 20876 diff changeset	185 _fpartparamcount = '>BB'
9e9e3a4e9261 bundle2: part params Pierre-Yves David <pierre-yves.david@fb.com> parents: 20876 diff changeset	186
35793 4fb2bb61597c bundle2: increase payload part chunk size to 32kb Gregory Szorc <gregory.szorc@gmail.com> parents: 35792 diff changeset	187 preferedchunksize = 32768
21001 c93bb6a08fa1 bundle2: support chunk iterator as part data Pierre-Yves David <pierre-yves.david@fb.com> parents: 21000 diff changeset	188
23868 405eacbecc52 bundle2: enforce parttype as alphanumerical Pierre-Yves David <pierre-yves.david@fb.com> parents: 23590 diff changeset	189 _parttypeforbidden = re.compile('[^a-zA-Z0-9_:-]')
405eacbecc52 bundle2: enforce parttype as alphanumerical Pierre-Yves David <pierre-yves.david@fb.com> parents: 23590 diff changeset	190
25313 8f2c362bcb58 bundle2: introduce a specific function for bundling debug message Pierre-Yves David <pierre-yves.david@fb.com> parents: 25181 diff changeset	191 def outdebug(ui, message):
8f2c362bcb58 bundle2: introduce a specific function for bundling debug message Pierre-Yves David <pierre-yves.david@fb.com> parents: 25181 diff changeset	192 """debug regarding output stream (bundling)"""
33159 0224820688ac config: register the 'devel.bundle2.debug' config Pierre-Yves David <pierre-yves.david@octobus.net> parents: 33128 diff changeset	193 if ui.configbool('devel', 'bundle2.debug'):
25336 7c60a42265fb bundle2: hide bundle2 stream debug under a config flag Pierre-Yves David <pierre-yves.david@fb.com> parents: 25335 diff changeset	194 ui.debug('bundle2-output: %s\n' % message)
25313 8f2c362bcb58 bundle2: introduce a specific function for bundling debug message Pierre-Yves David <pierre-yves.david@fb.com> parents: 25181 diff changeset	195
25318 64dd8f25fcf4 bundle2: introduce a specific function for debug messages while unbundling Pierre-Yves David <pierre-yves.david@fb.com> parents: 25317 diff changeset	196 def indebug(ui, message):
64dd8f25fcf4 bundle2: introduce a specific function for debug messages while unbundling Pierre-Yves David <pierre-yves.david@fb.com> parents: 25317 diff changeset	197 """debug on input stream (unbundling)"""
33159 0224820688ac config: register the 'devel.bundle2.debug' config Pierre-Yves David <pierre-yves.david@octobus.net> parents: 33128 diff changeset	198 if ui.configbool('devel', 'bundle2.debug'):
25336 7c60a42265fb bundle2: hide bundle2 stream debug under a config flag Pierre-Yves David <pierre-yves.david@fb.com> parents: 25335 diff changeset	199 ui.debug('bundle2-input: %s\n' % message)
25318 64dd8f25fcf4 bundle2: introduce a specific function for debug messages while unbundling Pierre-Yves David <pierre-yves.david@fb.com> parents: 25317 diff changeset	200
23868 405eacbecc52 bundle2: enforce parttype as alphanumerical Pierre-Yves David <pierre-yves.david@fb.com> parents: 23590 diff changeset	201 def validateparttype(parttype):
405eacbecc52 bundle2: enforce parttype as alphanumerical Pierre-Yves David <pierre-yves.david@fb.com> parents: 23590 diff changeset	202 """raise ValueError if a parttype contains invalid character"""
23916 a3f7c781786b bundle2: fix parttype enforcement Matt Mackall <mpm@selenic.com> parents: 23868 diff changeset	203 if _parttypeforbidden.search(parttype):
23868 405eacbecc52 bundle2: enforce parttype as alphanumerical Pierre-Yves David <pierre-yves.david@fb.com> parents: 23590 diff changeset	204 raise ValueError(parttype)
405eacbecc52 bundle2: enforce parttype as alphanumerical Pierre-Yves David <pierre-yves.david@fb.com> parents: 23590 diff changeset	205
20877 9e9e3a4e9261 bundle2: part params Pierre-Yves David <pierre-yves.david@fb.com> parents: 20876 diff changeset	206 def _makefpartparamsizes(nbparams):
9e9e3a4e9261 bundle2: part params Pierre-Yves David <pierre-yves.david@fb.com> parents: 20876 diff changeset	207 """return a struct format to read part parameter sizes
9e9e3a4e9261 bundle2: part params Pierre-Yves David <pierre-yves.david@fb.com> parents: 20876 diff changeset	208
9e9e3a4e9261 bundle2: part params Pierre-Yves David <pierre-yves.david@fb.com> parents: 20876 diff changeset	209 The number parameters is variable so we need to build that format
9e9e3a4e9261 bundle2: part params Pierre-Yves David <pierre-yves.david@fb.com> parents: 20876 diff changeset	210 dynamically.
9e9e3a4e9261 bundle2: part params Pierre-Yves David <pierre-yves.david@fb.com> parents: 20876 diff changeset	211 """
9e9e3a4e9261 bundle2: part params Pierre-Yves David <pierre-yves.david@fb.com> parents: 20876 diff changeset	212 return '>'+('BB'*nbparams)
20804 db9d3991d2c6 bundle2: support bundling simple parameter Pierre-Yves David <pierre-yves.david@fb.com> parents: 20803 diff changeset	213
20890 ec7fc110faee bundle2: introduce a `parthandler` decorator Pierre-Yves David <pierre-yves.david@fb.com> parents: 20889 diff changeset	214 parthandlermapping = {}
20889 deed5edb72de bundle2: first version of a bundle processing Pierre-Yves David <pierre-yves.david@fb.com> parents: 20887 diff changeset	215
21623 5b26d82e4e2a bundle2: make it possible to declare params handled by a part handler Pierre-Yves David <pierre-yves.david@fb.com> parents: 21622 diff changeset	216 def parthandler(parttype, params=()):
20890 ec7fc110faee bundle2: introduce a `parthandler` decorator Pierre-Yves David <pierre-yves.david@fb.com> parents: 20889 diff changeset	217 """decorator that register a function as a bundle2 part handler
ec7fc110faee bundle2: introduce a `parthandler` decorator Pierre-Yves David <pierre-yves.david@fb.com> parents: 20889 diff changeset	218
ec7fc110faee bundle2: introduce a `parthandler` decorator Pierre-Yves David <pierre-yves.david@fb.com> parents: 20889 diff changeset	219 eg::
ec7fc110faee bundle2: introduce a `parthandler` decorator Pierre-Yves David <pierre-yves.david@fb.com> parents: 20889 diff changeset	220
21624 d61066d787c8 bundle2: declare supported parameters for all handlers Pierre-Yves David <pierre-yves.david@fb.com> parents: 21623 diff changeset	221 @parthandler('myparttype', ('mandatory', 'param', 'handled'))
20890 ec7fc110faee bundle2: introduce a `parthandler` decorator Pierre-Yves David <pierre-yves.david@fb.com> parents: 20889 diff changeset	222 def myparttypehandler(...):
ec7fc110faee bundle2: introduce a `parthandler` decorator Pierre-Yves David <pierre-yves.david@fb.com> parents: 20889 diff changeset	223 '''process a part of type "my part".'''
ec7fc110faee bundle2: introduce a `parthandler` decorator Pierre-Yves David <pierre-yves.david@fb.com> parents: 20889 diff changeset	224 ...
ec7fc110faee bundle2: introduce a `parthandler` decorator Pierre-Yves David <pierre-yves.david@fb.com> parents: 20889 diff changeset	225 """
23868 405eacbecc52 bundle2: enforce parttype as alphanumerical Pierre-Yves David <pierre-yves.david@fb.com> parents: 23590 diff changeset	226 validateparttype(parttype)
20890 ec7fc110faee bundle2: introduce a `parthandler` decorator Pierre-Yves David <pierre-yves.david@fb.com> parents: 20889 diff changeset	227 def _decorator(func):
20891 1c6cd23fc221 bundle2: add some distinction between mandatory and advisory part Pierre-Yves David <pierre-yves.david@fb.com> parents: 20890 diff changeset	228 lparttype = parttype.lower() # enforce lower case matching.
1c6cd23fc221 bundle2: add some distinction between mandatory and advisory part Pierre-Yves David <pierre-yves.david@fb.com> parents: 20890 diff changeset	229 assert lparttype not in parthandlermapping
1c6cd23fc221 bundle2: add some distinction between mandatory and advisory part Pierre-Yves David <pierre-yves.david@fb.com> parents: 20890 diff changeset	230 parthandlermapping[lparttype] = func
21623 5b26d82e4e2a bundle2: make it possible to declare params handled by a part handler Pierre-Yves David <pierre-yves.david@fb.com> parents: 21622 diff changeset	231 func.params = frozenset(params)
20890 ec7fc110faee bundle2: introduce a `parthandler` decorator Pierre-Yves David <pierre-yves.david@fb.com> parents: 20889 diff changeset	232 return func
ec7fc110faee bundle2: introduce a `parthandler` decorator Pierre-Yves David <pierre-yves.david@fb.com> parents: 20889 diff changeset	233 return _decorator
20889 deed5edb72de bundle2: first version of a bundle processing Pierre-Yves David <pierre-yves.david@fb.com> parents: 20887 diff changeset	234
20949 571f2903ff1e bundle2: record processing results in the bundleoperation object Pierre-Yves David <pierre-yves.david@fb.com> parents: 20948 diff changeset	235 class unbundlerecords(object):
571f2903ff1e bundle2: record processing results in the bundleoperation object Pierre-Yves David <pierre-yves.david@fb.com> parents: 20948 diff changeset	236 """keep record of what happens during and unbundle
571f2903ff1e bundle2: record processing results in the bundleoperation object Pierre-Yves David <pierre-yves.david@fb.com> parents: 20948 diff changeset	237
571f2903ff1e bundle2: record processing results in the bundleoperation object Pierre-Yves David <pierre-yves.david@fb.com> parents: 20948 diff changeset	238 New records are added using `records.add('cat', obj)`. Where 'cat' is a
21024 7731a2281cf0 spelling: fixes from spell checker Mads Kiilerich <madski@unity3d.com> parents: 21020 diff changeset	239 category of record and obj is an arbitrary object.
20949 571f2903ff1e bundle2: record processing results in the bundleoperation object Pierre-Yves David <pierre-yves.david@fb.com> parents: 20948 diff changeset	240
571f2903ff1e bundle2: record processing results in the bundleoperation object Pierre-Yves David <pierre-yves.david@fb.com> parents: 20948 diff changeset	241 `records['cat']` will return all entries of this category 'cat'.
571f2903ff1e bundle2: record processing results in the bundleoperation object Pierre-Yves David <pierre-yves.david@fb.com> parents: 20948 diff changeset	242
571f2903ff1e bundle2: record processing results in the bundleoperation object Pierre-Yves David <pierre-yves.david@fb.com> parents: 20948 diff changeset	243 Iterating on the object itself will yield `('category', obj)` tuples
571f2903ff1e bundle2: record processing results in the bundleoperation object Pierre-Yves David <pierre-yves.david@fb.com> parents: 20948 diff changeset	244 for all entries.
571f2903ff1e bundle2: record processing results in the bundleoperation object Pierre-Yves David <pierre-yves.david@fb.com> parents: 20948 diff changeset	245
571f2903ff1e bundle2: record processing results in the bundleoperation object Pierre-Yves David <pierre-yves.david@fb.com> parents: 20948 diff changeset	246 All iterations happens in chronological order.
571f2903ff1e bundle2: record processing results in the bundleoperation object Pierre-Yves David <pierre-yves.david@fb.com> parents: 20948 diff changeset	247 """
571f2903ff1e bundle2: record processing results in the bundleoperation object Pierre-Yves David <pierre-yves.david@fb.com> parents: 20948 diff changeset	248
571f2903ff1e bundle2: record processing results in the bundleoperation object Pierre-Yves David <pierre-yves.david@fb.com> parents: 20948 diff changeset	249 def __init__(self):
571f2903ff1e bundle2: record processing results in the bundleoperation object Pierre-Yves David <pierre-yves.david@fb.com> parents: 20948 diff changeset	250 self._categories = {}
571f2903ff1e bundle2: record processing results in the bundleoperation object Pierre-Yves David <pierre-yves.david@fb.com> parents: 20948 diff changeset	251 self._sequences = []
20996 ed3c5e18a047 bundle2: add reply awareness to unbundlerecords Pierre-Yves David <pierre-yves.david@fb.com> parents: 20995 diff changeset	252 self._replies = {}
20949 571f2903ff1e bundle2: record processing results in the bundleoperation object Pierre-Yves David <pierre-yves.david@fb.com> parents: 20948 diff changeset	253
20996 ed3c5e18a047 bundle2: add reply awareness to unbundlerecords Pierre-Yves David <pierre-yves.david@fb.com> parents: 20995 diff changeset	254 def add(self, category, entry, inreplyto=None):
20949 571f2903ff1e bundle2: record processing results in the bundleoperation object Pierre-Yves David <pierre-yves.david@fb.com> parents: 20948 diff changeset	255 """add a new record of a given category.
571f2903ff1e bundle2: record processing results in the bundleoperation object Pierre-Yves David <pierre-yves.david@fb.com> parents: 20948 diff changeset	256
571f2903ff1e bundle2: record processing results in the bundleoperation object Pierre-Yves David <pierre-yves.david@fb.com> parents: 20948 diff changeset	257 The entry can then be retrieved in the list returned by
571f2903ff1e bundle2: record processing results in the bundleoperation object Pierre-Yves David <pierre-yves.david@fb.com> parents: 20948 diff changeset	258 self['category']."""
571f2903ff1e bundle2: record processing results in the bundleoperation object Pierre-Yves David <pierre-yves.david@fb.com> parents: 20948 diff changeset	259 self._categories.setdefault(category, []).append(entry)
571f2903ff1e bundle2: record processing results in the bundleoperation object Pierre-Yves David <pierre-yves.david@fb.com> parents: 20948 diff changeset	260 self._sequences.append((category, entry))
20996 ed3c5e18a047 bundle2: add reply awareness to unbundlerecords Pierre-Yves David <pierre-yves.david@fb.com> parents: 20995 diff changeset	261 if inreplyto is not None:
ed3c5e18a047 bundle2: add reply awareness to unbundlerecords Pierre-Yves David <pierre-yves.david@fb.com> parents: 20995 diff changeset	262 self.getreplies(inreplyto).add(category, entry)
ed3c5e18a047 bundle2: add reply awareness to unbundlerecords Pierre-Yves David <pierre-yves.david@fb.com> parents: 20995 diff changeset	263
ed3c5e18a047 bundle2: add reply awareness to unbundlerecords Pierre-Yves David <pierre-yves.david@fb.com> parents: 20995 diff changeset	264 def getreplies(self, partid):
23139 e53f6b72a0e4 spelling: fixes from proofreading of spell checker issues Mads Kiilerich <madski@unity3d.com> parents: 23067 diff changeset	265 """get the records that are replies to a specific part"""
20996 ed3c5e18a047 bundle2: add reply awareness to unbundlerecords Pierre-Yves David <pierre-yves.david@fb.com> parents: 20995 diff changeset	266 return self._replies.setdefault(partid, unbundlerecords())
20949 571f2903ff1e bundle2: record processing results in the bundleoperation object Pierre-Yves David <pierre-yves.david@fb.com> parents: 20948 diff changeset	267
571f2903ff1e bundle2: record processing results in the bundleoperation object Pierre-Yves David <pierre-yves.david@fb.com> parents: 20948 diff changeset	268 def __getitem__(self, cat):
571f2903ff1e bundle2: record processing results in the bundleoperation object Pierre-Yves David <pierre-yves.david@fb.com> parents: 20948 diff changeset	269 return tuple(self._categories.get(cat, ()))
571f2903ff1e bundle2: record processing results in the bundleoperation object Pierre-Yves David <pierre-yves.david@fb.com> parents: 20948 diff changeset	270
571f2903ff1e bundle2: record processing results in the bundleoperation object Pierre-Yves David <pierre-yves.david@fb.com> parents: 20948 diff changeset	271 def __iter__(self):
571f2903ff1e bundle2: record processing results in the bundleoperation object Pierre-Yves David <pierre-yves.david@fb.com> parents: 20948 diff changeset	272 return iter(self._sequences)
571f2903ff1e bundle2: record processing results in the bundleoperation object Pierre-Yves David <pierre-yves.david@fb.com> parents: 20948 diff changeset	273
571f2903ff1e bundle2: record processing results in the bundleoperation object Pierre-Yves David <pierre-yves.david@fb.com> parents: 20948 diff changeset	274 def __len__(self):
571f2903ff1e bundle2: record processing results in the bundleoperation object Pierre-Yves David <pierre-yves.david@fb.com> parents: 20948 diff changeset	275 return len(self._sequences)
571f2903ff1e bundle2: record processing results in the bundleoperation object Pierre-Yves David <pierre-yves.david@fb.com> parents: 20948 diff changeset	276
571f2903ff1e bundle2: record processing results in the bundleoperation object Pierre-Yves David <pierre-yves.david@fb.com> parents: 20948 diff changeset	277 def __nonzero__(self):
571f2903ff1e bundle2: record processing results in the bundleoperation object Pierre-Yves David <pierre-yves.david@fb.com> parents: 20948 diff changeset	278 return bool(self._sequences)
571f2903ff1e bundle2: record processing results in the bundleoperation object Pierre-Yves David <pierre-yves.david@fb.com> parents: 20948 diff changeset	279
31476 413b44003462 py3: add __bool__ to every class defining __nonzero__ Gregory Szorc <gregory.szorc@gmail.com> parents: 30965 diff changeset	280 __bool__ = __nonzero__
413b44003462 py3: add __bool__ to every class defining __nonzero__ Gregory Szorc <gregory.szorc@gmail.com> parents: 30965 diff changeset	281
20948 329cd74b52bd bundle2: introduce a bundleoperation object Pierre-Yves David <pierre-yves.david@fb.com> parents: 20947 diff changeset	282 class bundleoperation(object):
329cd74b52bd bundle2: introduce a bundleoperation object Pierre-Yves David <pierre-yves.david@fb.com> parents: 20947 diff changeset	283 """an object that represents a single bundling process
329cd74b52bd bundle2: introduce a bundleoperation object Pierre-Yves David <pierre-yves.david@fb.com> parents: 20947 diff changeset	284
329cd74b52bd bundle2: introduce a bundleoperation object Pierre-Yves David <pierre-yves.david@fb.com> parents: 20947 diff changeset	285 Its purpose is to carry unbundle-related objects and states.
329cd74b52bd bundle2: introduce a bundleoperation object Pierre-Yves David <pierre-yves.david@fb.com> parents: 20947 diff changeset	286
329cd74b52bd bundle2: introduce a bundleoperation object Pierre-Yves David <pierre-yves.david@fb.com> parents: 20947 diff changeset	287 A new object should be created at the beginning of each bundle processing.
329cd74b52bd bundle2: introduce a bundleoperation object Pierre-Yves David <pierre-yves.david@fb.com> parents: 20947 diff changeset	288 The object is to be returned by the processing function.
329cd74b52bd bundle2: introduce a bundleoperation object Pierre-Yves David <pierre-yves.david@fb.com> parents: 20947 diff changeset	289
329cd74b52bd bundle2: introduce a bundleoperation object Pierre-Yves David <pierre-yves.david@fb.com> parents: 20947 diff changeset	290 The object has very little content now it will ultimately contain:
329cd74b52bd bundle2: introduce a bundleoperation object Pierre-Yves David <pierre-yves.david@fb.com> parents: 20947 diff changeset	291 * an access to the repo the bundle is applied to,
329cd74b52bd bundle2: introduce a bundleoperation object Pierre-Yves David <pierre-yves.david@fb.com> parents: 20947 diff changeset	292 * a ui object,
329cd74b52bd bundle2: introduce a bundleoperation object Pierre-Yves David <pierre-yves.david@fb.com> parents: 20947 diff changeset	293 * a way to retrieve a transaction to add changes to the repo,
329cd74b52bd bundle2: introduce a bundleoperation object Pierre-Yves David <pierre-yves.david@fb.com> parents: 20947 diff changeset	294 * a way to record the result of processing each part,
329cd74b52bd bundle2: introduce a bundleoperation object Pierre-Yves David <pierre-yves.david@fb.com> parents: 20947 diff changeset	295 * a way to construct a bundle response when applicable.
329cd74b52bd bundle2: introduce a bundleoperation object Pierre-Yves David <pierre-yves.david@fb.com> parents: 20947 diff changeset	296 """
329cd74b52bd bundle2: introduce a bundleoperation object Pierre-Yves David <pierre-yves.david@fb.com> parents: 20947 diff changeset	297
24878 e530cde6d115 bundle2: disable ouput capture unless we use http (issue4613 issue4615) Pierre-Yves David <pierre-yves.david@fb.com> parents: 24851 diff changeset	298 def __init__(self, repo, transactiongetter, captureoutput=True):
20948 329cd74b52bd bundle2: introduce a bundleoperation object Pierre-Yves David <pierre-yves.david@fb.com> parents: 20947 diff changeset	299 self.repo = repo
329cd74b52bd bundle2: introduce a bundleoperation object Pierre-Yves David <pierre-yves.david@fb.com> parents: 20947 diff changeset	300 self.ui = repo.ui
20949 571f2903ff1e bundle2: record processing results in the bundleoperation object Pierre-Yves David <pierre-yves.david@fb.com> parents: 20948 diff changeset	301 self.records = unbundlerecords()
20997 d7df4b7378ae bundle2: produce a bundle2 reply Pierre-Yves David <pierre-yves.david@fb.com> parents: 20996 diff changeset	302 self.reply = None
24878 e530cde6d115 bundle2: disable ouput capture unless we use http (issue4613 issue4615) Pierre-Yves David <pierre-yves.david@fb.com> parents: 24851 diff changeset	303 self.captureoutput = captureoutput
33664 f3407d56a6e8 bundle2: add the capability to store hookargs on bundle operation object Pulkit Goyal <7895pulkit@gmail.com> parents: 33541 diff changeset	304 self.hookargs = {}
33665 5fc4ddfbe626 bundle2: load hookargs from bundleoperation into transaction when started one Pulkit Goyal <7895pulkit@gmail.com> parents: 33664 diff changeset	305 self._gettransaction = transactiongetter
35265 1f30cbac34b6 bundle2: add a 'modes' dictionary to the bundle operation Boris Feld <boris.feld@octobus.net> parents: 35261 diff changeset	306 # carries value that can modify part behavior
1f30cbac34b6 bundle2: add a 'modes' dictionary to the bundle operation Boris Feld <boris.feld@octobus.net> parents: 35261 diff changeset	307 self.modes = {}
33665 5fc4ddfbe626 bundle2: load hookargs from bundleoperation into transaction when started one Pulkit Goyal <7895pulkit@gmail.com> parents: 33664 diff changeset	308
5fc4ddfbe626 bundle2: load hookargs from bundleoperation into transaction when started one Pulkit Goyal <7895pulkit@gmail.com> parents: 33664 diff changeset	309 def gettransaction(self):
5fc4ddfbe626 bundle2: load hookargs from bundleoperation into transaction when started one Pulkit Goyal <7895pulkit@gmail.com> parents: 33664 diff changeset	310 transaction = self._gettransaction()
5fc4ddfbe626 bundle2: load hookargs from bundleoperation into transaction when started one Pulkit Goyal <7895pulkit@gmail.com> parents: 33664 diff changeset	311
33810 b3a41f4d837b bundle2: relax the condition to update transaction.hookargs Yuya Nishihara <yuya@tcha.org> parents: 33809 diff changeset	312 if self.hookargs:
33665 5fc4ddfbe626 bundle2: load hookargs from bundleoperation into transaction when started one Pulkit Goyal <7895pulkit@gmail.com> parents: 33664 diff changeset	313 # the ones added to the transaction supercede those added
5fc4ddfbe626 bundle2: load hookargs from bundleoperation into transaction when started one Pulkit Goyal <7895pulkit@gmail.com> parents: 33664 diff changeset	314 # to the operation.
5fc4ddfbe626 bundle2: load hookargs from bundleoperation into transaction when started one Pulkit Goyal <7895pulkit@gmail.com> parents: 33664 diff changeset	315 self.hookargs.update(transaction.hookargs)
5fc4ddfbe626 bundle2: load hookargs from bundleoperation into transaction when started one Pulkit Goyal <7895pulkit@gmail.com> parents: 33664 diff changeset	316 transaction.hookargs = self.hookargs
5fc4ddfbe626 bundle2: load hookargs from bundleoperation into transaction when started one Pulkit Goyal <7895pulkit@gmail.com> parents: 33664 diff changeset	317
33810 b3a41f4d837b bundle2: relax the condition to update transaction.hookargs Yuya Nishihara <yuya@tcha.org> parents: 33809 diff changeset	318 # mark the hookargs as flushed. further attempts to add to
b3a41f4d837b bundle2: relax the condition to update transaction.hookargs Yuya Nishihara <yuya@tcha.org> parents: 33809 diff changeset	319 # hookargs will result in an abort.
b3a41f4d837b bundle2: relax the condition to update transaction.hookargs Yuya Nishihara <yuya@tcha.org> parents: 33809 diff changeset	320 self.hookargs = None
33665 5fc4ddfbe626 bundle2: load hookargs from bundleoperation into transaction when started one Pulkit Goyal <7895pulkit@gmail.com> parents: 33664 diff changeset	321
5fc4ddfbe626 bundle2: load hookargs from bundleoperation into transaction when started one Pulkit Goyal <7895pulkit@gmail.com> parents: 33664 diff changeset	322 return transaction
33664 f3407d56a6e8 bundle2: add the capability to store hookargs on bundle operation object Pulkit Goyal <7895pulkit@gmail.com> parents: 33541 diff changeset	323
f3407d56a6e8 bundle2: add the capability to store hookargs on bundle operation object Pulkit Goyal <7895pulkit@gmail.com> parents: 33541 diff changeset	324 def addhookargs(self, hookargs):
33665 5fc4ddfbe626 bundle2: load hookargs from bundleoperation into transaction when started one Pulkit Goyal <7895pulkit@gmail.com> parents: 33664 diff changeset	325 if self.hookargs is None:
33809 1bf5c5507614 bundle2: raise ProgrammingError for invalid call of addhookargs() Yuya Nishihara <yuya@tcha.org> parents: 33770 diff changeset	326 raise error.ProgrammingError('attempted to add hookargs to '
1bf5c5507614 bundle2: raise ProgrammingError for invalid call of addhookargs() Yuya Nishihara <yuya@tcha.org> parents: 33770 diff changeset	327 'operation after transaction started')
33664 f3407d56a6e8 bundle2: add the capability to store hookargs on bundle operation object Pulkit Goyal <7895pulkit@gmail.com> parents: 33541 diff changeset	328 self.hookargs.update(hookargs)
20948 329cd74b52bd bundle2: introduce a bundleoperation object Pierre-Yves David <pierre-yves.david@fb.com> parents: 20947 diff changeset	329
20952 b24ee5076b94 bundle2: make it possible have a global transaction for the unbundling Pierre-Yves David <pierre-yves.david@fb.com> parents: 20950 diff changeset	330 class TransactionUnavailable(RuntimeError):
b24ee5076b94 bundle2: make it possible have a global transaction for the unbundling Pierre-Yves David <pierre-yves.david@fb.com> parents: 20950 diff changeset	331 pass
b24ee5076b94 bundle2: make it possible have a global transaction for the unbundling Pierre-Yves David <pierre-yves.david@fb.com> parents: 20950 diff changeset	332
b24ee5076b94 bundle2: make it possible have a global transaction for the unbundling Pierre-Yves David <pierre-yves.david@fb.com> parents: 20950 diff changeset	333 def _notransaction():
b24ee5076b94 bundle2: make it possible have a global transaction for the unbundling Pierre-Yves David <pierre-yves.david@fb.com> parents: 20950 diff changeset	334 """default method to get a transaction while processing a bundle
b24ee5076b94 bundle2: make it possible have a global transaction for the unbundling Pierre-Yves David <pierre-yves.david@fb.com> parents: 20950 diff changeset	335
b24ee5076b94 bundle2: make it possible have a global transaction for the unbundling Pierre-Yves David <pierre-yves.david@fb.com> parents: 20950 diff changeset	336 Raise an exception to highlight the fact that no transaction was expected
b24ee5076b94 bundle2: make it possible have a global transaction for the unbundling Pierre-Yves David <pierre-yves.david@fb.com> parents: 20950 diff changeset	337 to be created"""
b24ee5076b94 bundle2: make it possible have a global transaction for the unbundling Pierre-Yves David <pierre-yves.david@fb.com> parents: 20950 diff changeset	338 raise TransactionUnavailable()
b24ee5076b94 bundle2: make it possible have a global transaction for the unbundling Pierre-Yves David <pierre-yves.david@fb.com> parents: 20950 diff changeset	339
33043 18c2489ac96d bundle: make applybundle() delegate v1 bundles to applybundle1() Martin von Zweigbergk <martinvonz@google.com> parents: 33040 diff changeset	340 def applybundle(repo, unbundler, tr, source=None, url=None, **kwargs):
26790 28a6c2d72097 bundle2: introduce an "applybundle" function Pierre-Yves David <pierre-yves.david@fb.com> parents: 26698 diff changeset	341 # transform me into unbundler.apply() as soon as the freeze is lifted
33043 18c2489ac96d bundle: make applybundle() delegate v1 bundles to applybundle1() Martin von Zweigbergk <martinvonz@google.com> parents: 33040 diff changeset	342 if isinstance(unbundler, unbundle20):
18c2489ac96d bundle: make applybundle() delegate v1 bundles to applybundle1() Martin von Zweigbergk <martinvonz@google.com> parents: 33040 diff changeset	343 tr.hookargs['bundle2'] = '1'
18c2489ac96d bundle: make applybundle() delegate v1 bundles to applybundle1() Martin von Zweigbergk <martinvonz@google.com> parents: 33040 diff changeset	344 if source is not None and 'source' not in tr.hookargs:
18c2489ac96d bundle: make applybundle() delegate v1 bundles to applybundle1() Martin von Zweigbergk <martinvonz@google.com> parents: 33040 diff changeset	345 tr.hookargs['source'] = source
18c2489ac96d bundle: make applybundle() delegate v1 bundles to applybundle1() Martin von Zweigbergk <martinvonz@google.com> parents: 33040 diff changeset	346 if url is not None and 'url' not in tr.hookargs:
18c2489ac96d bundle: make applybundle() delegate v1 bundles to applybundle1() Martin von Zweigbergk <martinvonz@google.com> parents: 33040 diff changeset	347 tr.hookargs['url'] = url
18c2489ac96d bundle: make applybundle() delegate v1 bundles to applybundle1() Martin von Zweigbergk <martinvonz@google.com> parents: 33040 diff changeset	348 return processbundle(repo, unbundler, lambda: tr)
18c2489ac96d bundle: make applybundle() delegate v1 bundles to applybundle1() Martin von Zweigbergk <martinvonz@google.com> parents: 33040 diff changeset	349 else:
33044 1d2b68952950 bundle: inline applybundle1() Martin von Zweigbergk <martinvonz@google.com> parents: 33043 diff changeset	350 # the transactiongetter won't be used, but we might as well set it
1d2b68952950 bundle: inline applybundle1() Martin von Zweigbergk <martinvonz@google.com> parents: 33043 diff changeset	351 op = bundleoperation(repo, lambda: tr)
1d2b68952950 bundle: inline applybundle1() Martin von Zweigbergk <martinvonz@google.com> parents: 33043 diff changeset	352 _processchangegroup(op, unbundler, tr, source, url, **kwargs)
1d2b68952950 bundle: inline applybundle1() Martin von Zweigbergk <martinvonz@google.com> parents: 33043 diff changeset	353 return op
26790 28a6c2d72097 bundle2: introduce an "applybundle" function Pierre-Yves David <pierre-yves.david@fb.com> parents: 26698 diff changeset	354
34150 e9e0e1143fc5 bundle2: move part iterator a separate class Durham Goode <durham@fb.com> parents: 34101 diff changeset	355 class partiterator(object):
34152 21c2df59a1da bundle2: move exception handling into part iterator Durham Goode <durham@fb.com> parents: 34151 diff changeset	356 def __init__(self, repo, op, unbundler):
34151 550343626bb2 bundle2: move part counter to partiterator Durham Goode <durham@fb.com> parents: 34150 diff changeset	357 self.repo = repo
34152 21c2df59a1da bundle2: move exception handling into part iterator Durham Goode <durham@fb.com> parents: 34151 diff changeset	358 self.op = op
34150 e9e0e1143fc5 bundle2: move part iterator a separate class Durham Goode <durham@fb.com> parents: 34101 diff changeset	359 self.unbundler = unbundler
34151 550343626bb2 bundle2: move part counter to partiterator Durham Goode <durham@fb.com> parents: 34150 diff changeset	360 self.iterator = None
550343626bb2 bundle2: move part counter to partiterator Durham Goode <durham@fb.com> parents: 34150 diff changeset	361 self.count = 0
34258 e71890f27767 bundle2: move processpart stream maintenance into part iterator Durham Goode <durham@fb.com> parents: 34249 diff changeset	362 self.current = None
34150 e9e0e1143fc5 bundle2: move part iterator a separate class Durham Goode <durham@fb.com> parents: 34101 diff changeset	363
e9e0e1143fc5 bundle2: move part iterator a separate class Durham Goode <durham@fb.com> parents: 34101 diff changeset	364 def __enter__(self):
34151 550343626bb2 bundle2: move part counter to partiterator Durham Goode <durham@fb.com> parents: 34150 diff changeset	365 def func():
550343626bb2 bundle2: move part counter to partiterator Durham Goode <durham@fb.com> parents: 34150 diff changeset	366 itr = enumerate(self.unbundler.iterparts())
550343626bb2 bundle2: move part counter to partiterator Durham Goode <durham@fb.com> parents: 34150 diff changeset	367 for count, p in itr:
550343626bb2 bundle2: move part counter to partiterator Durham Goode <durham@fb.com> parents: 34150 diff changeset	368 self.count = count
34258 e71890f27767 bundle2: move processpart stream maintenance into part iterator Durham Goode <durham@fb.com> parents: 34249 diff changeset	369 self.current = p
34151 550343626bb2 bundle2: move part counter to partiterator Durham Goode <durham@fb.com> parents: 34150 diff changeset	370 yield p
35114 db5038525718 bundle2: implement consume() API on unbundlepart Gregory Szorc <gregory.szorc@gmail.com> parents: 35113 diff changeset	371 p.consume()
34258 e71890f27767 bundle2: move processpart stream maintenance into part iterator Durham Goode <durham@fb.com> parents: 34249 diff changeset	372 self.current = None
34151 550343626bb2 bundle2: move part counter to partiterator Durham Goode <durham@fb.com> parents: 34150 diff changeset	373 self.iterator = func()
550343626bb2 bundle2: move part counter to partiterator Durham Goode <durham@fb.com> parents: 34150 diff changeset	374 return self.iterator
34150 e9e0e1143fc5 bundle2: move part iterator a separate class Durham Goode <durham@fb.com> parents: 34101 diff changeset	375
34152 21c2df59a1da bundle2: move exception handling into part iterator Durham Goode <durham@fb.com> parents: 34151 diff changeset	376 def __exit__(self, type, exc, tb):
34151 550343626bb2 bundle2: move part counter to partiterator Durham Goode <durham@fb.com> parents: 34150 diff changeset	377 if not self.iterator:
550343626bb2 bundle2: move part counter to partiterator Durham Goode <durham@fb.com> parents: 34150 diff changeset	378 return
550343626bb2 bundle2: move part counter to partiterator Durham Goode <durham@fb.com> parents: 34150 diff changeset	379
34637 5f79f5f8487a bundle2: immediate exit for ctrl+c (issue5692) Durham Goode <durham@fb.com> parents: 34323 diff changeset	380 # Only gracefully abort in a normal exception situation. User aborts
5f79f5f8487a bundle2: immediate exit for ctrl+c (issue5692) Durham Goode <durham@fb.com> parents: 34323 diff changeset	381 # like Ctrl+C throw a KeyboardInterrupt which is not a base Exception,
5f79f5f8487a bundle2: immediate exit for ctrl+c (issue5692) Durham Goode <durham@fb.com> parents: 34323 diff changeset	382 # and should not gracefully cleanup.
5f79f5f8487a bundle2: immediate exit for ctrl+c (issue5692) Durham Goode <durham@fb.com> parents: 34323 diff changeset	383 if isinstance(exc, Exception):
34152 21c2df59a1da bundle2: move exception handling into part iterator Durham Goode <durham@fb.com> parents: 34151 diff changeset	384 # Any exceptions seeking to the end of the bundle at this point are
21c2df59a1da bundle2: move exception handling into part iterator Durham Goode <durham@fb.com> parents: 34151 diff changeset	385 # almost certainly related to the underlying stream being bad.
21c2df59a1da bundle2: move exception handling into part iterator Durham Goode <durham@fb.com> parents: 34151 diff changeset	386 # And, chances are that the exception we're handling is related to
21c2df59a1da bundle2: move exception handling into part iterator Durham Goode <durham@fb.com> parents: 34151 diff changeset	387 # getting in that bad state. So, we swallow the seeking error and
21c2df59a1da bundle2: move exception handling into part iterator Durham Goode <durham@fb.com> parents: 34151 diff changeset	388 # re-raise the original error.
21c2df59a1da bundle2: move exception handling into part iterator Durham Goode <durham@fb.com> parents: 34151 diff changeset	389 seekerror = False
21c2df59a1da bundle2: move exception handling into part iterator Durham Goode <durham@fb.com> parents: 34151 diff changeset	390 try:
34637 5f79f5f8487a bundle2: immediate exit for ctrl+c (issue5692) Durham Goode <durham@fb.com> parents: 34323 diff changeset	391 if self.current:
5f79f5f8487a bundle2: immediate exit for ctrl+c (issue5692) Durham Goode <durham@fb.com> parents: 34323 diff changeset	392 # consume the part content to not corrupt the stream.
35114 db5038525718 bundle2: implement consume() API on unbundlepart Gregory Szorc <gregory.szorc@gmail.com> parents: 35113 diff changeset	393 self.current.consume()
34637 5f79f5f8487a bundle2: immediate exit for ctrl+c (issue5692) Durham Goode <durham@fb.com> parents: 34323 diff changeset	394
34152 21c2df59a1da bundle2: move exception handling into part iterator Durham Goode <durham@fb.com> parents: 34151 diff changeset	395 for part in self.iterator:
21c2df59a1da bundle2: move exception handling into part iterator Durham Goode <durham@fb.com> parents: 34151 diff changeset	396 # consume the bundle content
35114 db5038525718 bundle2: implement consume() API on unbundlepart Gregory Szorc <gregory.szorc@gmail.com> parents: 35113 diff changeset	397 part.consume()
34152 21c2df59a1da bundle2: move exception handling into part iterator Durham Goode <durham@fb.com> parents: 34151 diff changeset	398 except Exception:
21c2df59a1da bundle2: move exception handling into part iterator Durham Goode <durham@fb.com> parents: 34151 diff changeset	399 seekerror = True
21c2df59a1da bundle2: move exception handling into part iterator Durham Goode <durham@fb.com> parents: 34151 diff changeset	400
21c2df59a1da bundle2: move exception handling into part iterator Durham Goode <durham@fb.com> parents: 34151 diff changeset	401 # Small hack to let caller code distinguish exceptions from bundle2
21c2df59a1da bundle2: move exception handling into part iterator Durham Goode <durham@fb.com> parents: 34151 diff changeset	402 # processing from processing the old format. This is mostly needed
21c2df59a1da bundle2: move exception handling into part iterator Durham Goode <durham@fb.com> parents: 34151 diff changeset	403 # to handle different return codes to unbundle according to the type
21c2df59a1da bundle2: move exception handling into part iterator Durham Goode <durham@fb.com> parents: 34151 diff changeset	404 # of bundle. We should probably clean up or drop this return code
21c2df59a1da bundle2: move exception handling into part iterator Durham Goode <durham@fb.com> parents: 34151 diff changeset	405 # craziness in a future version.
21c2df59a1da bundle2: move exception handling into part iterator Durham Goode <durham@fb.com> parents: 34151 diff changeset	406 exc.duringunbundle2 = True
21c2df59a1da bundle2: move exception handling into part iterator Durham Goode <durham@fb.com> parents: 34151 diff changeset	407 salvaged = []
21c2df59a1da bundle2: move exception handling into part iterator Durham Goode <durham@fb.com> parents: 34151 diff changeset	408 replycaps = None
21c2df59a1da bundle2: move exception handling into part iterator Durham Goode <durham@fb.com> parents: 34151 diff changeset	409 if self.op.reply is not None:
21c2df59a1da bundle2: move exception handling into part iterator Durham Goode <durham@fb.com> parents: 34151 diff changeset	410 salvaged = self.op.reply.salvageoutput()
21c2df59a1da bundle2: move exception handling into part iterator Durham Goode <durham@fb.com> parents: 34151 diff changeset	411 replycaps = self.op.reply.capabilities
21c2df59a1da bundle2: move exception handling into part iterator Durham Goode <durham@fb.com> parents: 34151 diff changeset	412 exc._replycaps = replycaps
21c2df59a1da bundle2: move exception handling into part iterator Durham Goode <durham@fb.com> parents: 34151 diff changeset	413 exc._bundle2salvagedoutput = salvaged
21c2df59a1da bundle2: move exception handling into part iterator Durham Goode <durham@fb.com> parents: 34151 diff changeset	414
21c2df59a1da bundle2: move exception handling into part iterator Durham Goode <durham@fb.com> parents: 34151 diff changeset	415 # Re-raising from a variable loses the original stack. So only use
21c2df59a1da bundle2: move exception handling into part iterator Durham Goode <durham@fb.com> parents: 34151 diff changeset	416 # that form if we need to.
21c2df59a1da bundle2: move exception handling into part iterator Durham Goode <durham@fb.com> parents: 34151 diff changeset	417 if seekerror:
21c2df59a1da bundle2: move exception handling into part iterator Durham Goode <durham@fb.com> parents: 34151 diff changeset	418 raise exc
21c2df59a1da bundle2: move exception handling into part iterator Durham Goode <durham@fb.com> parents: 34151 diff changeset	419
34151 550343626bb2 bundle2: move part counter to partiterator Durham Goode <durham@fb.com> parents: 34150 diff changeset	420 self.repo.ui.debug('bundle2-input-bundle: %i parts total\n' %
550343626bb2 bundle2: move part counter to partiterator Durham Goode <durham@fb.com> parents: 34150 diff changeset	421 self.count)
34150 e9e0e1143fc5 bundle2: move part iterator a separate class Durham Goode <durham@fb.com> parents: 34101 diff changeset	422
24851 df0ce98c882f bundle2: also save output when error happens during part processing Pierre-Yves David <pierre-yves.david@fb.com> parents: 24849 diff changeset	423 def processbundle(repo, unbundler, transactiongetter=None, op=None):
20889 deed5edb72de bundle2: first version of a bundle processing Pierre-Yves David <pierre-yves.david@fb.com> parents: 20887 diff changeset	424 """This function process a bundle, apply effect to/from a repo
deed5edb72de bundle2: first version of a bundle processing Pierre-Yves David <pierre-yves.david@fb.com> parents: 20887 diff changeset	425
20947 c33d7bf53812 bundle2: feed a unbundle20 to the `processbundle` function Pierre-Yves David <pierre-yves.david@fb.com> parents: 20892 diff changeset	426 It iterates over each part then searches for and uses the proper handling
c33d7bf53812 bundle2: feed a unbundle20 to the `processbundle` function Pierre-Yves David <pierre-yves.david@fb.com> parents: 20892 diff changeset	427 code to process the part. Parts are processed in order.
20889 deed5edb72de bundle2: first version of a bundle processing Pierre-Yves David <pierre-yves.david@fb.com> parents: 20887 diff changeset	428
20891 1c6cd23fc221 bundle2: add some distinction between mandatory and advisory part Pierre-Yves David <pierre-yves.david@fb.com> parents: 20890 diff changeset	429 Unknown Mandatory part will abort the process.
24851 df0ce98c882f bundle2: also save output when error happens during part processing Pierre-Yves David <pierre-yves.david@fb.com> parents: 24849 diff changeset	430
df0ce98c882f bundle2: also save output when error happens during part processing Pierre-Yves David <pierre-yves.david@fb.com> parents: 24849 diff changeset	431 It is temporarily possible to provide a prebuilt bundleoperation to the
df0ce98c882f bundle2: also save output when error happens during part processing Pierre-Yves David <pierre-yves.david@fb.com> parents: 24849 diff changeset	432 function. This is used to ensure output is properly propagated in case of
df0ce98c882f bundle2: also save output when error happens during part processing Pierre-Yves David <pierre-yves.david@fb.com> parents: 24849 diff changeset	433 an error during the unbundling. This output capturing part will likely be
df0ce98c882f bundle2: also save output when error happens during part processing Pierre-Yves David <pierre-yves.david@fb.com> parents: 24849 diff changeset	434 reworked and this ability will probably go away in the process.
20889 deed5edb72de bundle2: first version of a bundle processing Pierre-Yves David <pierre-yves.david@fb.com> parents: 20887 diff changeset	435 """
24851 df0ce98c882f bundle2: also save output when error happens during part processing Pierre-Yves David <pierre-yves.david@fb.com> parents: 24849 diff changeset	436 if op is None:
df0ce98c882f bundle2: also save output when error happens during part processing Pierre-Yves David <pierre-yves.david@fb.com> parents: 24849 diff changeset	437 if transactiongetter is None:
df0ce98c882f bundle2: also save output when error happens during part processing Pierre-Yves David <pierre-yves.david@fb.com> parents: 24849 diff changeset	438 transactiongetter = _notransaction
df0ce98c882f bundle2: also save output when error happens during part processing Pierre-Yves David <pierre-yves.david@fb.com> parents: 24849 diff changeset	439 op = bundleoperation(repo, transactiongetter)
20889 deed5edb72de bundle2: first version of a bundle processing Pierre-Yves David <pierre-yves.david@fb.com> parents: 20887 diff changeset	440 # todo:
deed5edb72de bundle2: first version of a bundle processing Pierre-Yves David <pierre-yves.david@fb.com> parents: 20887 diff changeset	441 # - replace this is a init function soon.
deed5edb72de bundle2: first version of a bundle processing Pierre-Yves David <pierre-yves.david@fb.com> parents: 20887 diff changeset	442 # - exception catching
deed5edb72de bundle2: first version of a bundle processing Pierre-Yves David <pierre-yves.david@fb.com> parents: 20887 diff changeset	443 unbundler.params
25331 53ca47ef645d bundle2: add generic debug output regarding processed bundle Pierre-Yves David <pierre-yves.david@fb.com> parents: 25324 diff changeset	444 if repo.ui.debugflag:
53ca47ef645d bundle2: add generic debug output regarding processed bundle Pierre-Yves David <pierre-yves.david@fb.com> parents: 25324 diff changeset	445 msg = ['bundle2-input-bundle:']
53ca47ef645d bundle2: add generic debug output regarding processed bundle Pierre-Yves David <pierre-yves.david@fb.com> parents: 25324 diff changeset	446 if unbundler.params:
33128 126eae7dae74 bundle2: add debug info about the number of stream params Siddharth Agarwal <sid0@fb.com> parents: 33127 diff changeset	447 msg.append(' %i params' % len(unbundler.params))
33770 119e1c6be1ce bundle2: fix transaction availability detection Boris Feld <boris.feld@octobus.net> parents: 33711 diff changeset	448 if op._gettransaction is None or op._gettransaction is _notransaction:
25331 53ca47ef645d bundle2: add generic debug output regarding processed bundle Pierre-Yves David <pierre-yves.david@fb.com> parents: 25324 diff changeset	449 msg.append(' no-transaction')
53ca47ef645d bundle2: add generic debug output regarding processed bundle Pierre-Yves David <pierre-yves.david@fb.com> parents: 25324 diff changeset	450 else:
53ca47ef645d bundle2: add generic debug output regarding processed bundle Pierre-Yves David <pierre-yves.david@fb.com> parents: 25324 diff changeset	451 msg.append(' with-transaction')
53ca47ef645d bundle2: add generic debug output regarding processed bundle Pierre-Yves David <pierre-yves.david@fb.com> parents: 25324 diff changeset	452 msg.append('\n')
53ca47ef645d bundle2: add generic debug output regarding processed bundle Pierre-Yves David <pierre-yves.david@fb.com> parents: 25324 diff changeset	453 repo.ui.debug(''.join(msg))
34150 e9e0e1143fc5 bundle2: move part iterator a separate class Durham Goode <durham@fb.com> parents: 34101 diff changeset	454
34261 f010097c885c bundle2: move part processing to a separate function Durham Goode <durham@fb.com> parents: 34260 diff changeset	455 processparts(repo, op, unbundler)
f010097c885c bundle2: move part processing to a separate function Durham Goode <durham@fb.com> parents: 34260 diff changeset	456
f010097c885c bundle2: move part processing to a separate function Durham Goode <durham@fb.com> parents: 34260 diff changeset	457 return op
f010097c885c bundle2: move part processing to a separate function Durham Goode <durham@fb.com> parents: 34260 diff changeset	458
f010097c885c bundle2: move part processing to a separate function Durham Goode <durham@fb.com> parents: 34260 diff changeset	459 def processparts(repo, op, unbundler):
34152 21c2df59a1da bundle2: move exception handling into part iterator Durham Goode <durham@fb.com> parents: 34151 diff changeset	460 with partiterator(repo, op, unbundler) as parts:
21c2df59a1da bundle2: move exception handling into part iterator Durham Goode <durham@fb.com> parents: 34151 diff changeset	461 for part in parts:
21c2df59a1da bundle2: move exception handling into part iterator Durham Goode <durham@fb.com> parents: 34151 diff changeset	462 _processpart(op, part)
25332 0cdd1c811705 bundle2: add generic debug output at the end of bundle processing Pierre-Yves David <pierre-yves.david@fb.com> parents: 25331 diff changeset	463
33038 f0efd2bffe1e bundle: extract _processchangegroup() method Martin von Zweigbergk <martinvonz@google.com> parents: 33037 diff changeset	464 def _processchangegroup(op, cg, tr, source, url, **kwargs):
33461 bb72031f0ea8 changegroup: stop returning and recording added nodes in 'cg.apply' Boris Feld <boris.feld@octobus.net> parents: 33460 diff changeset	465 ret = cg.apply(op.repo, tr, source, url, **kwargs)
33038 f0efd2bffe1e bundle: extract _processchangegroup() method Martin von Zweigbergk <martinvonz@google.com> parents: 33037 diff changeset	466 op.records.add('changegroup', {
f0efd2bffe1e bundle: extract _processchangegroup() method Martin von Zweigbergk <martinvonz@google.com> parents: 33037 diff changeset	467 'return': ret,
f0efd2bffe1e bundle: extract _processchangegroup() method Martin von Zweigbergk <martinvonz@google.com> parents: 33037 diff changeset	468 })
f0efd2bffe1e bundle: extract _processchangegroup() method Martin von Zweigbergk <martinvonz@google.com> parents: 33037 diff changeset	469 return ret
f0efd2bffe1e bundle: extract _processchangegroup() method Martin von Zweigbergk <martinvonz@google.com> parents: 33037 diff changeset	470
34259 07e4170f02f3 bundle2: move handler validation out of processpart Durham Goode <durham@fb.com> parents: 34258 diff changeset	471 def _gethandler(op, part):
07e4170f02f3 bundle2: move handler validation out of processpart Durham Goode <durham@fb.com> parents: 34258 diff changeset	472 status = 'unknown' # used by debug output
07e4170f02f3 bundle2: move handler validation out of processpart Durham Goode <durham@fb.com> parents: 34258 diff changeset	473 try:
07e4170f02f3 bundle2: move handler validation out of processpart Durham Goode <durham@fb.com> parents: 34258 diff changeset	474 handler = parthandlermapping.get(part.type)
07e4170f02f3 bundle2: move handler validation out of processpart Durham Goode <durham@fb.com> parents: 34258 diff changeset	475 if handler is None:
07e4170f02f3 bundle2: move handler validation out of processpart Durham Goode <durham@fb.com> parents: 34258 diff changeset	476 status = 'unsupported-type'
07e4170f02f3 bundle2: move handler validation out of processpart Durham Goode <durham@fb.com> parents: 34258 diff changeset	477 raise error.BundleUnknownFeatureError(parttype=part.type)
34277 a48b3db0baba bundle2: stop using %r to quote part names Augie Fackler <augie@google.com> parents: 34261 diff changeset	478 indebug(op.ui, 'found a handler for part %s' % part.type)
34259 07e4170f02f3 bundle2: move handler validation out of processpart Durham Goode <durham@fb.com> parents: 34258 diff changeset	479 unknownparams = part.mandatorykeys - handler.params
07e4170f02f3 bundle2: move handler validation out of processpart Durham Goode <durham@fb.com> parents: 34258 diff changeset	480 if unknownparams:
07e4170f02f3 bundle2: move handler validation out of processpart Durham Goode <durham@fb.com> parents: 34258 diff changeset	481 unknownparams = list(unknownparams)
07e4170f02f3 bundle2: move handler validation out of processpart Durham Goode <durham@fb.com> parents: 34258 diff changeset	482 unknownparams.sort()
34277 a48b3db0baba bundle2: stop using %r to quote part names Augie Fackler <augie@google.com> parents: 34261 diff changeset	483 status = 'unsupported-params (%s)' % ', '.join(unknownparams)
34259 07e4170f02f3 bundle2: move handler validation out of processpart Durham Goode <durham@fb.com> parents: 34258 diff changeset	484 raise error.BundleUnknownFeatureError(parttype=part.type,
07e4170f02f3 bundle2: move handler validation out of processpart Durham Goode <durham@fb.com> parents: 34258 diff changeset	485 params=unknownparams)
07e4170f02f3 bundle2: move handler validation out of processpart Durham Goode <durham@fb.com> parents: 34258 diff changeset	486 status = 'supported'
07e4170f02f3 bundle2: move handler validation out of processpart Durham Goode <durham@fb.com> parents: 34258 diff changeset	487 except error.BundleUnknownFeatureError as exc:
07e4170f02f3 bundle2: move handler validation out of processpart Durham Goode <durham@fb.com> parents: 34258 diff changeset	488 if part.mandatory: # mandatory parts
07e4170f02f3 bundle2: move handler validation out of processpart Durham Goode <durham@fb.com> parents: 34258 diff changeset	489 raise
07e4170f02f3 bundle2: move handler validation out of processpart Durham Goode <durham@fb.com> parents: 34258 diff changeset	490 indebug(op.ui, 'ignoring unsupported advisory part %s' % exc)
07e4170f02f3 bundle2: move handler validation out of processpart Durham Goode <durham@fb.com> parents: 34258 diff changeset	491 return # skip to part processing
07e4170f02f3 bundle2: move handler validation out of processpart Durham Goode <durham@fb.com> parents: 34258 diff changeset	492 finally:
07e4170f02f3 bundle2: move handler validation out of processpart Durham Goode <durham@fb.com> parents: 34258 diff changeset	493 if op.ui.debugflag:
07e4170f02f3 bundle2: move handler validation out of processpart Durham Goode <durham@fb.com> parents: 34258 diff changeset	494 msg = ['bundle2-input-part: "%s"' % part.type]
07e4170f02f3 bundle2: move handler validation out of processpart Durham Goode <durham@fb.com> parents: 34258 diff changeset	495 if not part.mandatory:
07e4170f02f3 bundle2: move handler validation out of processpart Durham Goode <durham@fb.com> parents: 34258 diff changeset	496 msg.append(' (advisory)')
07e4170f02f3 bundle2: move handler validation out of processpart Durham Goode <durham@fb.com> parents: 34258 diff changeset	497 nbmp = len(part.mandatorykeys)
07e4170f02f3 bundle2: move handler validation out of processpart Durham Goode <durham@fb.com> parents: 34258 diff changeset	498 nbap = len(part.params) - nbmp
07e4170f02f3 bundle2: move handler validation out of processpart Durham Goode <durham@fb.com> parents: 34258 diff changeset	499 if nbmp or nbap:
07e4170f02f3 bundle2: move handler validation out of processpart Durham Goode <durham@fb.com> parents: 34258 diff changeset	500 msg.append(' (params:')
07e4170f02f3 bundle2: move handler validation out of processpart Durham Goode <durham@fb.com> parents: 34258 diff changeset	501 if nbmp:
07e4170f02f3 bundle2: move handler validation out of processpart Durham Goode <durham@fb.com> parents: 34258 diff changeset	502 msg.append(' %i mandatory' % nbmp)
07e4170f02f3 bundle2: move handler validation out of processpart Durham Goode <durham@fb.com> parents: 34258 diff changeset	503 if nbap:
07e4170f02f3 bundle2: move handler validation out of processpart Durham Goode <durham@fb.com> parents: 34258 diff changeset	504 msg.append(' %i advisory' % nbmp)
07e4170f02f3 bundle2: move handler validation out of processpart Durham Goode <durham@fb.com> parents: 34258 diff changeset	505 msg.append(')')
07e4170f02f3 bundle2: move handler validation out of processpart Durham Goode <durham@fb.com> parents: 34258 diff changeset	506 msg.append(' %s\n' % status)
07e4170f02f3 bundle2: move handler validation out of processpart Durham Goode <durham@fb.com> parents: 34258 diff changeset	507 op.ui.debug(''.join(msg))
07e4170f02f3 bundle2: move handler validation out of processpart Durham Goode <durham@fb.com> parents: 34258 diff changeset	508
07e4170f02f3 bundle2: move handler validation out of processpart Durham Goode <durham@fb.com> parents: 34258 diff changeset	509 return handler
07e4170f02f3 bundle2: move handler validation out of processpart Durham Goode <durham@fb.com> parents: 34258 diff changeset	510
23008 d3137827016a bundle2: extract processing of part into its own function Pierre-Yves David <pierre-yves.david@fb.com> parents: 23001 diff changeset	511 def _processpart(op, part):
d3137827016a bundle2: extract processing of part into its own function Pierre-Yves David <pierre-yves.david@fb.com> parents: 23001 diff changeset	512 """process a single part from a bundle
d3137827016a bundle2: extract processing of part into its own function Pierre-Yves David <pierre-yves.david@fb.com> parents: 23001 diff changeset	513
d3137827016a bundle2: extract processing of part into its own function Pierre-Yves David <pierre-yves.david@fb.com> parents: 23001 diff changeset	514 The part is guaranteed to have been fully consumed when the function exits
d3137827016a bundle2: extract processing of part into its own function Pierre-Yves David <pierre-yves.david@fb.com> parents: 23001 diff changeset	515 (even if an exception is raised)."""
34260 cc7b37c90616 bundle2: remove unnecessary try finally Durham Goode <durham@fb.com> parents: 34259 diff changeset	516 handler = _gethandler(op, part)
cc7b37c90616 bundle2: remove unnecessary try finally Durham Goode <durham@fb.com> parents: 34259 diff changeset	517 if handler is None:
cc7b37c90616 bundle2: remove unnecessary try finally Durham Goode <durham@fb.com> parents: 34259 diff changeset	518 return
23008 d3137827016a bundle2: extract processing of part into its own function Pierre-Yves David <pierre-yves.david@fb.com> parents: 23001 diff changeset	519
34260 cc7b37c90616 bundle2: remove unnecessary try finally Durham Goode <durham@fb.com> parents: 34259 diff changeset	520 # handler is called outside the above try block so that we don't
cc7b37c90616 bundle2: remove unnecessary try finally Durham Goode <durham@fb.com> parents: 34259 diff changeset	521 # risk catching KeyErrors from anything other than the
cc7b37c90616 bundle2: remove unnecessary try finally Durham Goode <durham@fb.com> parents: 34259 diff changeset	522 # parthandlermapping lookup (any KeyError raised by handler()
cc7b37c90616 bundle2: remove unnecessary try finally Durham Goode <durham@fb.com> parents: 34259 diff changeset	523 # itself represents a defect of a different variety).
cc7b37c90616 bundle2: remove unnecessary try finally Durham Goode <durham@fb.com> parents: 34259 diff changeset	524 output = None
cc7b37c90616 bundle2: remove unnecessary try finally Durham Goode <durham@fb.com> parents: 34259 diff changeset	525 if op.captureoutput and op.reply is not None:
cc7b37c90616 bundle2: remove unnecessary try finally Durham Goode <durham@fb.com> parents: 34259 diff changeset	526 op.ui.pushbuffer(error=True, subproc=True)
cc7b37c90616 bundle2: remove unnecessary try finally Durham Goode <durham@fb.com> parents: 34259 diff changeset	527 output = ''
cc7b37c90616 bundle2: remove unnecessary try finally Durham Goode <durham@fb.com> parents: 34259 diff changeset	528 try:
cc7b37c90616 bundle2: remove unnecessary try finally Durham Goode <durham@fb.com> parents: 34259 diff changeset	529 handler(op, part)
23008 d3137827016a bundle2: extract processing of part into its own function Pierre-Yves David <pierre-yves.david@fb.com> parents: 23001 diff changeset	530 finally:
34260 cc7b37c90616 bundle2: remove unnecessary try finally Durham Goode <durham@fb.com> parents: 34259 diff changeset	531 if output is not None:
cc7b37c90616 bundle2: remove unnecessary try finally Durham Goode <durham@fb.com> parents: 34259 diff changeset	532 output = op.ui.popbuffer()
cc7b37c90616 bundle2: remove unnecessary try finally Durham Goode <durham@fb.com> parents: 34259 diff changeset	533 if output:
cc7b37c90616 bundle2: remove unnecessary try finally Durham Goode <durham@fb.com> parents: 34259 diff changeset	534 outpart = op.reply.newpart('output', data=output,
cc7b37c90616 bundle2: remove unnecessary try finally Durham Goode <durham@fb.com> parents: 34259 diff changeset	535 mandatory=False)
cc7b37c90616 bundle2: remove unnecessary try finally Durham Goode <durham@fb.com> parents: 34259 diff changeset	536 outpart.addparam(
cc7b37c90616 bundle2: remove unnecessary try finally Durham Goode <durham@fb.com> parents: 34259 diff changeset	537 'in-reply-to', pycompat.bytestr(part.id), mandatory=False)
23008 d3137827016a bundle2: extract processing of part into its own function Pierre-Yves David <pierre-yves.david@fb.com> parents: 23001 diff changeset	538
21138 f469879d27ec bundle2: extract capabilities decoding Pierre-Yves David <pierre-yves.david@fb.com> parents: 21137 diff changeset	539 def decodecaps(blob):
23139 e53f6b72a0e4 spelling: fixes from proofreading of spell checker issues Mads Kiilerich <madski@unity3d.com> parents: 23067 diff changeset	540 """decode a bundle2 caps bytes blob into a dictionary
21138 f469879d27ec bundle2: extract capabilities decoding Pierre-Yves David <pierre-yves.david@fb.com> parents: 21137 diff changeset	541
f469879d27ec bundle2: extract capabilities decoding Pierre-Yves David <pierre-yves.david@fb.com> parents: 21137 diff changeset	542 The blob is a list of capabilities (one per line)
f469879d27ec bundle2: extract capabilities decoding Pierre-Yves David <pierre-yves.david@fb.com> parents: 21137 diff changeset	543 Capabilities may have values using a line of the form::
f469879d27ec bundle2: extract capabilities decoding Pierre-Yves David <pierre-yves.david@fb.com> parents: 21137 diff changeset	544
f469879d27ec bundle2: extract capabilities decoding Pierre-Yves David <pierre-yves.david@fb.com> parents: 21137 diff changeset	545 capability=value1,value2,value3
f469879d27ec bundle2: extract capabilities decoding Pierre-Yves David <pierre-yves.david@fb.com> parents: 21137 diff changeset	546
f469879d27ec bundle2: extract capabilities decoding Pierre-Yves David <pierre-yves.david@fb.com> parents: 21137 diff changeset	547 The values are always a list."""
f469879d27ec bundle2: extract capabilities decoding Pierre-Yves David <pierre-yves.david@fb.com> parents: 21137 diff changeset	548 caps = {}
f469879d27ec bundle2: extract capabilities decoding Pierre-Yves David <pierre-yves.david@fb.com> parents: 21137 diff changeset	549 for line in blob.splitlines():
f469879d27ec bundle2: extract capabilities decoding Pierre-Yves David <pierre-yves.david@fb.com> parents: 21137 diff changeset	550 if not line:
f469879d27ec bundle2: extract capabilities decoding Pierre-Yves David <pierre-yves.david@fb.com> parents: 21137 diff changeset	551 continue
f469879d27ec bundle2: extract capabilities decoding Pierre-Yves David <pierre-yves.david@fb.com> parents: 21137 diff changeset	552 if '=' not in line:
f469879d27ec bundle2: extract capabilities decoding Pierre-Yves David <pierre-yves.david@fb.com> parents: 21137 diff changeset	553 key, vals = line, ()
f469879d27ec bundle2: extract capabilities decoding Pierre-Yves David <pierre-yves.david@fb.com> parents: 21137 diff changeset	554 else:
f469879d27ec bundle2: extract capabilities decoding Pierre-Yves David <pierre-yves.david@fb.com> parents: 21137 diff changeset	555 key, vals = line.split('=', 1)
f469879d27ec bundle2: extract capabilities decoding Pierre-Yves David <pierre-yves.david@fb.com> parents: 21137 diff changeset	556 vals = vals.split(',')
28883 032c4c2f802a pycompat: switch to util.urlreq/util.urlerr for py3 compat timeless <timeless@mozdev.org> parents: 28672 diff changeset	557 key = urlreq.unquote(key)
032c4c2f802a pycompat: switch to util.urlreq/util.urlerr for py3 compat timeless <timeless@mozdev.org> parents: 28672 diff changeset	558 vals = [urlreq.unquote(v) for v in vals]
21138 f469879d27ec bundle2: extract capabilities decoding Pierre-Yves David <pierre-yves.david@fb.com> parents: 21137 diff changeset	559 caps[key] = vals
f469879d27ec bundle2: extract capabilities decoding Pierre-Yves David <pierre-yves.david@fb.com> parents: 21137 diff changeset	560 return caps
f469879d27ec bundle2: extract capabilities decoding Pierre-Yves David <pierre-yves.david@fb.com> parents: 21137 diff changeset	561
21139 2b8c82f7f11d bundle2: capabilities encoding Pierre-Yves David <pierre-yves.david@fb.com> parents: 21138 diff changeset	562 def encodecaps(caps):
2b8c82f7f11d bundle2: capabilities encoding Pierre-Yves David <pierre-yves.david@fb.com> parents: 21138 diff changeset	563 """encode a bundle2 caps dictionary into a bytes blob"""
2b8c82f7f11d bundle2: capabilities encoding Pierre-Yves David <pierre-yves.david@fb.com> parents: 21138 diff changeset	564 chunks = []
2b8c82f7f11d bundle2: capabilities encoding Pierre-Yves David <pierre-yves.david@fb.com> parents: 21138 diff changeset	565 for ca in sorted(caps):
2b8c82f7f11d bundle2: capabilities encoding Pierre-Yves David <pierre-yves.david@fb.com> parents: 21138 diff changeset	566 vals = caps[ca]
28883 032c4c2f802a pycompat: switch to util.urlreq/util.urlerr for py3 compat timeless <timeless@mozdev.org> parents: 28672 diff changeset	567 ca = urlreq.quote(ca)
032c4c2f802a pycompat: switch to util.urlreq/util.urlerr for py3 compat timeless <timeless@mozdev.org> parents: 28672 diff changeset	568 vals = [urlreq.quote(v) for v in vals]
21139 2b8c82f7f11d bundle2: capabilities encoding Pierre-Yves David <pierre-yves.david@fb.com> parents: 21138 diff changeset	569 if vals:
2b8c82f7f11d bundle2: capabilities encoding Pierre-Yves David <pierre-yves.david@fb.com> parents: 21138 diff changeset	570 ca = "%s=%s" % (ca, ','.join(vals))
2b8c82f7f11d bundle2: capabilities encoding Pierre-Yves David <pierre-yves.david@fb.com> parents: 21138 diff changeset	571 chunks.append(ca)
2b8c82f7f11d bundle2: capabilities encoding Pierre-Yves David <pierre-yves.david@fb.com> parents: 21138 diff changeset	572 return '\n'.join(chunks)
2b8c82f7f11d bundle2: capabilities encoding Pierre-Yves David <pierre-yves.david@fb.com> parents: 21138 diff changeset	573
28666 ae53ecc47414 bundle: move writebundle() from changegroup.py to bundle2.py (API) Martin von Zweigbergk <martinvonz@google.com> parents: 27953 diff changeset	574 bundletypes = {
30351 f81002f736d7 bundle2: use new compression engine API for compression Gregory Szorc <gregory.szorc@gmail.com> parents: 30177 diff changeset	575 "": ("", 'UN'), # only when using unbundle on ssh and old http servers
28666 ae53ecc47414 bundle: move writebundle() from changegroup.py to bundle2.py (API) Martin von Zweigbergk <martinvonz@google.com> parents: 27953 diff changeset	576 # since the unification ssh accepts a header but there
ae53ecc47414 bundle: move writebundle() from changegroup.py to bundle2.py (API) Martin von Zweigbergk <martinvonz@google.com> parents: 27953 diff changeset	577 # is no capability signaling it.
ae53ecc47414 bundle: move writebundle() from changegroup.py to bundle2.py (API) Martin von Zweigbergk <martinvonz@google.com> parents: 27953 diff changeset	578 "HG20": (), # special-cased below
30351 f81002f736d7 bundle2: use new compression engine API for compression Gregory Szorc <gregory.szorc@gmail.com> parents: 30177 diff changeset	579 "HG10UN": ("HG10UN", 'UN'),
28666 ae53ecc47414 bundle: move writebundle() from changegroup.py to bundle2.py (API) Martin von Zweigbergk <martinvonz@google.com> parents: 27953 diff changeset	580 "HG10BZ": ("HG10", 'BZ'),
ae53ecc47414 bundle: move writebundle() from changegroup.py to bundle2.py (API) Martin von Zweigbergk <martinvonz@google.com> parents: 27953 diff changeset	581 "HG10GZ": ("HG10GZ", 'GZ'),
ae53ecc47414 bundle: move writebundle() from changegroup.py to bundle2.py (API) Martin von Zweigbergk <martinvonz@google.com> parents: 27953 diff changeset	582 }
ae53ecc47414 bundle: move writebundle() from changegroup.py to bundle2.py (API) Martin von Zweigbergk <martinvonz@google.com> parents: 27953 diff changeset	583
ae53ecc47414 bundle: move writebundle() from changegroup.py to bundle2.py (API) Martin von Zweigbergk <martinvonz@google.com> parents: 27953 diff changeset	584 # hgweb uses this list to communicate its preferred type
ae53ecc47414 bundle: move writebundle() from changegroup.py to bundle2.py (API) Martin von Zweigbergk <martinvonz@google.com> parents: 27953 diff changeset	585 bundlepriority = ['HG10GZ', 'HG10BZ', 'HG10UN']
ae53ecc47414 bundle: move writebundle() from changegroup.py to bundle2.py (API) Martin von Zweigbergk <martinvonz@google.com> parents: 27953 diff changeset	586
20801 9c5183cb9bca bundle2: very first version of a bundle2 bundler Pierre-Yves David <pierre-yves.david@fb.com> parents: diff changeset	587 class bundle20(object):
9c5183cb9bca bundle2: very first version of a bundle2 bundler Pierre-Yves David <pierre-yves.david@fb.com> parents: diff changeset	588 """represent an outgoing bundle2 container
9c5183cb9bca bundle2: very first version of a bundle2 bundler Pierre-Yves David <pierre-yves.david@fb.com> parents: diff changeset	589
21599 57cd844d7a5b bundle2: have ``newpart`` automatically add the part to the bundle Pierre-Yves David <pierre-yves.david@fb.com> parents: 21598 diff changeset	590 Use the `addparam` method to add stream level parameter. and `newpart` to
20856 8a6a86c9a5b5 bundle2: support bundling of empty part (with a type) Pierre-Yves David <pierre-yves.david@fb.com> parents: 20844 diff changeset	591 populate it. Then call `getchunks` to retrieve all the binary chunks of
21024 7731a2281cf0 spelling: fixes from spell checker Mads Kiilerich <madski@unity3d.com> parents: 21020 diff changeset	592 data that compose the bundle2 container."""
20801 9c5183cb9bca bundle2: very first version of a bundle2 bundler Pierre-Yves David <pierre-yves.david@fb.com> parents: diff changeset	593
24686 e0e28e910fa3 bundle2: rename format, parts and config to final names Pierre-Yves David <pierre-yves.david@fb.com> parents: 24648 diff changeset	594 _magicstring = 'HG20'
24640 685639f9430d bundle20: move magic string into the class Pierre-Yves David <pierre-yves.david@fb.com> parents: 24071 diff changeset	595
21134 2f8c4fa237f5 bundle2: adds a capabilities attribute on bundler20 Pierre-Yves David <pierre-yves.david@fb.com> parents: 21133 diff changeset	596 def __init__(self, ui, capabilities=()):
20842 938718d72624 bundle2: print debug information during bundling Pierre-Yves David <pierre-yves.david@fb.com> parents: 20814 diff changeset	597 self.ui = ui
20801 9c5183cb9bca bundle2: very first version of a bundle2 bundler Pierre-Yves David <pierre-yves.david@fb.com> parents: diff changeset	598 self._params = []
9c5183cb9bca bundle2: very first version of a bundle2 bundler Pierre-Yves David <pierre-yves.david@fb.com> parents: diff changeset	599 self._parts = []
21136 b6fd496e5c72 bundle2: support for capabilities with values Pierre-Yves David <pierre-yves.david@fb.com> parents: 21135 diff changeset	600 self.capabilities = dict(capabilities)
30351 f81002f736d7 bundle2: use new compression engine API for compression Gregory Szorc <gregory.szorc@gmail.com> parents: 30177 diff changeset	601 self._compengine = util.compengines.forbundletype('UN')
30757 511a4bf52754 bundle2: allow compression options to be passed to compressor Gregory Szorc <gregory.szorc@gmail.com> parents: 30439 diff changeset	602 self._compopts = None
35787 a84dbc87dae9 exchange: send bundle2 stream clones uncompressed Gregory Szorc <gregory.szorc@gmail.com> parents: 35783 diff changeset	603 # If compression is being handled by a consumer of the raw
a84dbc87dae9 exchange: send bundle2 stream clones uncompressed Gregory Szorc <gregory.szorc@gmail.com> parents: 35783 diff changeset	604 # data (e.g. the wire protocol), unsetting this flag tells
a84dbc87dae9 exchange: send bundle2 stream clones uncompressed Gregory Szorc <gregory.szorc@gmail.com> parents: 35783 diff changeset	605 # consumers that the bundle is best left uncompressed.
a84dbc87dae9 exchange: send bundle2 stream clones uncompressed Gregory Szorc <gregory.szorc@gmail.com> parents: 35783 diff changeset	606 self.prefercompressed = True
26404 795f02a24b87 bundle2: allow compressed bundle Pierre-Yves David <pierre-yves.david@fb.com> parents: 26396 diff changeset	607
30757 511a4bf52754 bundle2: allow compression options to be passed to compressor Gregory Szorc <gregory.szorc@gmail.com> parents: 30439 diff changeset	608 def setcompression(self, alg, compopts=None):
26404 795f02a24b87 bundle2: allow compressed bundle Pierre-Yves David <pierre-yves.david@fb.com> parents: 26396 diff changeset	609 """setup core part compression to <alg>"""
30439 71b368e3b590 bundle2: equate 'UN' with no compression Gregory Szorc <gregory.szorc@gmail.com> parents: 30357 diff changeset	610 if alg in (None, 'UN'):
26404 795f02a24b87 bundle2: allow compressed bundle Pierre-Yves David <pierre-yves.david@fb.com> parents: 26396 diff changeset	611 return
30915 aa25989b0658 bundle2: fix assertion that 'compression' hasn't been set Siddharth Agarwal <sid0@fb.com> parents: 30757 diff changeset	612 assert not any(n.lower() == 'compression' for n, v in self._params)
26404 795f02a24b87 bundle2: allow compressed bundle Pierre-Yves David <pierre-yves.david@fb.com> parents: 26396 diff changeset	613 self.addparam('Compression', alg)
30351 f81002f736d7 bundle2: use new compression engine API for compression Gregory Szorc <gregory.szorc@gmail.com> parents: 30177 diff changeset	614 self._compengine = util.compengines.forbundletype(alg)
30757 511a4bf52754 bundle2: allow compression options to be passed to compressor Gregory Szorc <gregory.szorc@gmail.com> parents: 30439 diff changeset	615 self._compopts = compopts
20801 9c5183cb9bca bundle2: very first version of a bundle2 bundler Pierre-Yves David <pierre-yves.david@fb.com> parents: diff changeset	616
21900 b8bd97085ec9 bundle2: add a ``bundle20.nbparts`` property Pierre-Yves David <pierre-yves.david@fb.com> parents: 21660 diff changeset	617 @property
b8bd97085ec9 bundle2: add a ``bundle20.nbparts`` property Pierre-Yves David <pierre-yves.david@fb.com> parents: 21660 diff changeset	618 def nbparts(self):
b8bd97085ec9 bundle2: add a ``bundle20.nbparts`` property Pierre-Yves David <pierre-yves.david@fb.com> parents: 21660 diff changeset	619 """total number of parts added to the bundler"""
b8bd97085ec9 bundle2: add a ``bundle20.nbparts`` property Pierre-Yves David <pierre-yves.david@fb.com> parents: 21660 diff changeset	620 return len(self._parts)
b8bd97085ec9 bundle2: add a ``bundle20.nbparts`` property Pierre-Yves David <pierre-yves.david@fb.com> parents: 21660 diff changeset	621
21597 1daad9dcdba2 bundle2: small doc update on the bundler Pierre-Yves David <pierre-yves.david@fb.com> parents: 21186 diff changeset	622 # methods used to defines the bundle2 content
20804 db9d3991d2c6 bundle2: support bundling simple parameter Pierre-Yves David <pierre-yves.david@fb.com> parents: 20803 diff changeset	623 def addparam(self, name, value=None):
db9d3991d2c6 bundle2: support bundling simple parameter Pierre-Yves David <pierre-yves.david@fb.com> parents: 20803 diff changeset	624 """add a stream level parameter"""
20813 8c74b3ce5b70 bundle2: refuse empty parameter name Pierre-Yves David <pierre-yves.david@fb.com> parents: 20812 diff changeset	625 if not name:
34279 047f2af8ceb0 bundle2: make ValueError messages native strings Augie Fackler <augie@google.com> parents: 34278 diff changeset	626 raise ValueError(r'empty parameter name')
34280 2bdd55abd56c bundle2: portably grab first byte of part name for letter check Augie Fackler <augie@google.com> parents: 34279 diff changeset	627 if name[0:1] not in pycompat.bytestr(string.ascii_letters):
34279 047f2af8ceb0 bundle2: make ValueError messages native strings Augie Fackler <augie@google.com> parents: 34278 diff changeset	628 raise ValueError(r'non letter first character: %s' % name)
20804 db9d3991d2c6 bundle2: support bundling simple parameter Pierre-Yves David <pierre-yves.david@fb.com> parents: 20803 diff changeset	629 self._params.append((name, value))
db9d3991d2c6 bundle2: support bundling simple parameter Pierre-Yves David <pierre-yves.david@fb.com> parents: 20803 diff changeset	630
20856 8a6a86c9a5b5 bundle2: support bundling of empty part (with a type) Pierre-Yves David <pierre-yves.david@fb.com> parents: 20844 diff changeset	631 def addpart(self, part):
8a6a86c9a5b5 bundle2: support bundling of empty part (with a type) Pierre-Yves David <pierre-yves.david@fb.com> parents: 20844 diff changeset	632 """add a new part to the bundle2 container
8a6a86c9a5b5 bundle2: support bundling of empty part (with a type) Pierre-Yves David <pierre-yves.david@fb.com> parents: 20844 diff changeset	633
21024 7731a2281cf0 spelling: fixes from spell checker Mads Kiilerich <madski@unity3d.com> parents: 21020 diff changeset	634 Parts contains the actual applicative payload."""
20995 e995d104c87f bundle2: add an integer id to part Pierre-Yves David <pierre-yves.david@fb.com> parents: 20952 diff changeset	635 assert part.id is None
e995d104c87f bundle2: add an integer id to part Pierre-Yves David <pierre-yves.david@fb.com> parents: 20952 diff changeset	636 part.id = len(self._parts) # very cheap counter
20856 8a6a86c9a5b5 bundle2: support bundling of empty part (with a type) Pierre-Yves David <pierre-yves.david@fb.com> parents: 20844 diff changeset	637 self._parts.append(part)
8a6a86c9a5b5 bundle2: support bundling of empty part (with a type) Pierre-Yves David <pierre-yves.david@fb.com> parents: 20844 diff changeset	638
21598 1b0dbb91de5b bundle2: add a ``newpart`` method to ``bundle20`` Pierre-Yves David <pierre-yves.david@fb.com> parents: 21597 diff changeset	639 def newpart(self, typeid, args, *kwargs):
21602 cc33ae50bab3 bundle2: warn about error during initialization in ``newpart`` docstring Pierre-Yves David <pierre-yves.david@fb.com> parents: 21601 diff changeset	640 """create a new part and add it to the containers
cc33ae50bab3 bundle2: warn about error during initialization in ``newpart`` docstring Pierre-Yves David <pierre-yves.david@fb.com> parents: 21601 diff changeset	641
cc33ae50bab3 bundle2: warn about error during initialization in ``newpart`` docstring Pierre-Yves David <pierre-yves.david@fb.com> parents: 21601 diff changeset	642 As the part is directly added to the containers. For now, this means
cc33ae50bab3 bundle2: warn about error during initialization in ``newpart`` docstring Pierre-Yves David <pierre-yves.david@fb.com> parents: 21601 diff changeset	643 that any failure to properly initialize the part after calling
cc33ae50bab3 bundle2: warn about error during initialization in ``newpart`` docstring Pierre-Yves David <pierre-yves.david@fb.com> parents: 21601 diff changeset	644 ``newpart`` should result in a failure of the whole bundling process.
cc33ae50bab3 bundle2: warn about error during initialization in ``newpart`` docstring Pierre-Yves David <pierre-yves.david@fb.com> parents: 21601 diff changeset	645
cc33ae50bab3 bundle2: warn about error during initialization in ``newpart`` docstring Pierre-Yves David <pierre-yves.david@fb.com> parents: 21601 diff changeset	646 You can still fall back to manually create and add if you need better
cc33ae50bab3 bundle2: warn about error during initialization in ``newpart`` docstring Pierre-Yves David <pierre-yves.david@fb.com> parents: 21601 diff changeset	647 control."""
21598 1b0dbb91de5b bundle2: add a ``newpart`` method to ``bundle20`` Pierre-Yves David <pierre-yves.david@fb.com> parents: 21597 diff changeset	648 part = bundlepart(typeid, args, *kwargs)
21599 57cd844d7a5b bundle2: have ``newpart`` automatically add the part to the bundle Pierre-Yves David <pierre-yves.david@fb.com> parents: 21598 diff changeset	649 self.addpart(part)
21598 1b0dbb91de5b bundle2: add a ``newpart`` method to ``bundle20`` Pierre-Yves David <pierre-yves.david@fb.com> parents: 21597 diff changeset	650 return part
1b0dbb91de5b bundle2: add a ``newpart`` method to ``bundle20`` Pierre-Yves David <pierre-yves.david@fb.com> parents: 21597 diff changeset	651
21597 1daad9dcdba2 bundle2: small doc update on the bundler Pierre-Yves David <pierre-yves.david@fb.com> parents: 21186 diff changeset	652 # methods used to generate the bundle2 stream
20801 9c5183cb9bca bundle2: very first version of a bundle2 bundler Pierre-Yves David <pierre-yves.david@fb.com> parents: diff changeset	653 def getchunks(self):
25322 1f7c0403f8be bundle2: add generic debug output regarding generated bundle Pierre-Yves David <pierre-yves.david@fb.com> parents: 25321 diff changeset	654 if self.ui.debugflag:
1f7c0403f8be bundle2: add generic debug output regarding generated bundle Pierre-Yves David <pierre-yves.david@fb.com> parents: 25321 diff changeset	655 msg = ['bundle2-output-bundle: "%s",' % self._magicstring]
1f7c0403f8be bundle2: add generic debug output regarding generated bundle Pierre-Yves David <pierre-yves.david@fb.com> parents: 25321 diff changeset	656 if self._params:
1f7c0403f8be bundle2: add generic debug output regarding generated bundle Pierre-Yves David <pierre-yves.david@fb.com> parents: 25321 diff changeset	657 msg.append(' (%i params)' % len(self._params))
1f7c0403f8be bundle2: add generic debug output regarding generated bundle Pierre-Yves David <pierre-yves.david@fb.com> parents: 25321 diff changeset	658 msg.append(' %i parts total\n' % len(self._parts))
1f7c0403f8be bundle2: add generic debug output regarding generated bundle Pierre-Yves David <pierre-yves.david@fb.com> parents: 25321 diff changeset	659 self.ui.debug(''.join(msg))
25315 e28245b0da56 bundle2: handle new line in 'outdebug' function Pierre-Yves David <pierre-yves.david@fb.com> parents: 25314 diff changeset	660 outdebug(self.ui, 'start emission of %s stream' % self._magicstring)
24640 685639f9430d bundle20: move magic string into the class Pierre-Yves David <pierre-yves.david@fb.com> parents: 24071 diff changeset	661 yield self._magicstring
20804 db9d3991d2c6 bundle2: support bundling simple parameter Pierre-Yves David <pierre-yves.david@fb.com> parents: 20803 diff changeset	662 param = self._paramchunk()
25315 e28245b0da56 bundle2: handle new line in 'outdebug' function Pierre-Yves David <pierre-yves.david@fb.com> parents: 25314 diff changeset	663 outdebug(self.ui, 'bundle parameter: %s' % param)
20804 db9d3991d2c6 bundle2: support bundling simple parameter Pierre-Yves David <pierre-yves.david@fb.com> parents: 20803 diff changeset	664 yield _pack(_fstreamparamsize, len(param))
db9d3991d2c6 bundle2: support bundling simple parameter Pierre-Yves David <pierre-yves.david@fb.com> parents: 20803 diff changeset	665 if param:
db9d3991d2c6 bundle2: support bundling simple parameter Pierre-Yves David <pierre-yves.david@fb.com> parents: 20803 diff changeset	666 yield param
30757 511a4bf52754 bundle2: allow compression options to be passed to compressor Gregory Szorc <gregory.szorc@gmail.com> parents: 30439 diff changeset	667 for chunk in self._compengine.compressstream(self._getcorechunk(),
511a4bf52754 bundle2: allow compression options to be passed to compressor Gregory Szorc <gregory.szorc@gmail.com> parents: 30439 diff changeset	668 self._compopts):
30357 5925bda42dbd bundle2: use compressstream compression engine API Gregory Szorc <gregory.szorc@gmail.com> parents: 30353 diff changeset	669 yield chunk
20802 520df53ad26a bundle2: a very first version of bundle2 unbundler Pierre-Yves David <pierre-yves.david@fb.com> parents: 20801 diff changeset	670
20804 db9d3991d2c6 bundle2: support bundling simple parameter Pierre-Yves David <pierre-yves.david@fb.com> parents: 20803 diff changeset	671 def _paramchunk(self):
db9d3991d2c6 bundle2: support bundling simple parameter Pierre-Yves David <pierre-yves.david@fb.com> parents: 20803 diff changeset	672 """return a encoded version of all stream parameters"""
db9d3991d2c6 bundle2: support bundling simple parameter Pierre-Yves David <pierre-yves.david@fb.com> parents: 20803 diff changeset	673 blocks = []
20809 b93bb639451a bundle2: support for bundling parameter value Pierre-Yves David <pierre-yves.david@fb.com> parents: 20808 diff changeset	674 for par, value in self._params:
28883 032c4c2f802a pycompat: switch to util.urlreq/util.urlerr for py3 compat timeless <timeless@mozdev.org> parents: 28672 diff changeset	675 par = urlreq.quote(par)
20809 b93bb639451a bundle2: support for bundling parameter value Pierre-Yves David <pierre-yves.david@fb.com> parents: 20808 diff changeset	676 if value is not None:
28883 032c4c2f802a pycompat: switch to util.urlreq/util.urlerr for py3 compat timeless <timeless@mozdev.org> parents: 28672 diff changeset	677 value = urlreq.quote(value)
20809 b93bb639451a bundle2: support for bundling parameter value Pierre-Yves David <pierre-yves.david@fb.com> parents: 20808 diff changeset	678 par = '%s=%s' % (par, value)
b93bb639451a bundle2: support for bundling parameter value Pierre-Yves David <pierre-yves.david@fb.com> parents: 20808 diff changeset	679 blocks.append(par)
20804 db9d3991d2c6 bundle2: support bundling simple parameter Pierre-Yves David <pierre-yves.david@fb.com> parents: 20803 diff changeset	680 return ' '.join(blocks)
db9d3991d2c6 bundle2: support bundling simple parameter Pierre-Yves David <pierre-yves.david@fb.com> parents: 20803 diff changeset	681
26396 d90c30801cdf bundle20: extract core payload generation in its own function Pierre-Yves David <pierre-yves.david@fb.com> parents: 26395 diff changeset	682 def _getcorechunk(self):
d90c30801cdf bundle20: extract core payload generation in its own function Pierre-Yves David <pierre-yves.david@fb.com> parents: 26395 diff changeset	683 """yield chunk for the core part of the bundle
d90c30801cdf bundle20: extract core payload generation in its own function Pierre-Yves David <pierre-yves.david@fb.com> parents: 26395 diff changeset	684
d90c30801cdf bundle20: extract core payload generation in its own function Pierre-Yves David <pierre-yves.david@fb.com> parents: 26395 diff changeset	685 (all but headers and parameters)"""
d90c30801cdf bundle20: extract core payload generation in its own function Pierre-Yves David <pierre-yves.david@fb.com> parents: 26395 diff changeset	686 outdebug(self.ui, 'start of parts')
d90c30801cdf bundle20: extract core payload generation in its own function Pierre-Yves David <pierre-yves.david@fb.com> parents: 26395 diff changeset	687 for part in self._parts:
d90c30801cdf bundle20: extract core payload generation in its own function Pierre-Yves David <pierre-yves.david@fb.com> parents: 26395 diff changeset	688 outdebug(self.ui, 'bundle part: "%s"' % part.type)
d90c30801cdf bundle20: extract core payload generation in its own function Pierre-Yves David <pierre-yves.david@fb.com> parents: 26395 diff changeset	689 for chunk in part.getchunks(ui=self.ui):
d90c30801cdf bundle20: extract core payload generation in its own function Pierre-Yves David <pierre-yves.david@fb.com> parents: 26395 diff changeset	690 yield chunk
d90c30801cdf bundle20: extract core payload generation in its own function Pierre-Yves David <pierre-yves.david@fb.com> parents: 26395 diff changeset	691 outdebug(self.ui, 'end of bundle')
d90c30801cdf bundle20: extract core payload generation in its own function Pierre-Yves David <pierre-yves.david@fb.com> parents: 26395 diff changeset	692 yield _pack(_fpartheadersize, 0)
d90c30801cdf bundle20: extract core payload generation in its own function Pierre-Yves David <pierre-yves.david@fb.com> parents: 26395 diff changeset	693
d90c30801cdf bundle20: extract core payload generation in its own function Pierre-Yves David <pierre-yves.david@fb.com> parents: 26395 diff changeset	694
24794 21f2e8f48ae1 bundle2: add a 'salvageoutput' method on bundle20 Pierre-Yves David <pierre-yves.david@fb.com> parents: 24793 diff changeset	695 def salvageoutput(self):
21f2e8f48ae1 bundle2: add a 'salvageoutput' method on bundle20 Pierre-Yves David <pierre-yves.david@fb.com> parents: 24793 diff changeset	696 """return a list with a copy of all output parts in the bundle
21f2e8f48ae1 bundle2: add a 'salvageoutput' method on bundle20 Pierre-Yves David <pierre-yves.david@fb.com> parents: 24793 diff changeset	697
21f2e8f48ae1 bundle2: add a 'salvageoutput' method on bundle20 Pierre-Yves David <pierre-yves.david@fb.com> parents: 24793 diff changeset	698 This is meant to be used during error handling to make sure we preserve
21f2e8f48ae1 bundle2: add a 'salvageoutput' method on bundle20 Pierre-Yves David <pierre-yves.david@fb.com> parents: 24793 diff changeset	699 server output"""
21f2e8f48ae1 bundle2: add a 'salvageoutput' method on bundle20 Pierre-Yves David <pierre-yves.david@fb.com> parents: 24793 diff changeset	700 salvaged = []
21f2e8f48ae1 bundle2: add a 'salvageoutput' method on bundle20 Pierre-Yves David <pierre-yves.david@fb.com> parents: 24793 diff changeset	701 for part in self._parts:
21f2e8f48ae1 bundle2: add a 'salvageoutput' method on bundle20 Pierre-Yves David <pierre-yves.david@fb.com> parents: 24793 diff changeset	702 if part.type.startswith('output'):
21f2e8f48ae1 bundle2: add a 'salvageoutput' method on bundle20 Pierre-Yves David <pierre-yves.david@fb.com> parents: 24793 diff changeset	703 salvaged.append(part.copy())
21f2e8f48ae1 bundle2: add a 'salvageoutput' method on bundle20 Pierre-Yves David <pierre-yves.david@fb.com> parents: 24793 diff changeset	704 return salvaged
21f2e8f48ae1 bundle2: add a 'salvageoutput' method on bundle20 Pierre-Yves David <pierre-yves.david@fb.com> parents: 24793 diff changeset	705
21f2e8f48ae1 bundle2: add a 'salvageoutput' method on bundle20 Pierre-Yves David <pierre-yves.david@fb.com> parents: 24793 diff changeset	706
21013 a813caca89b3 bundle2: extract stream/unpack logic in an unpackermixin Pierre-Yves David <pierre-yves.david@fb.com> parents: 21005 diff changeset	707 class unpackermixin(object):
a813caca89b3 bundle2: extract stream/unpack logic in an unpackermixin Pierre-Yves David <pierre-yves.david@fb.com> parents: 21005 diff changeset	708 """A mixin to extract bytes and struct data from a stream"""
20802 520df53ad26a bundle2: a very first version of bundle2 unbundler Pierre-Yves David <pierre-yves.david@fb.com> parents: 20801 diff changeset	709
21013 a813caca89b3 bundle2: extract stream/unpack logic in an unpackermixin Pierre-Yves David <pierre-yves.david@fb.com> parents: 21005 diff changeset	710 def __init__(self, fp):
20802 520df53ad26a bundle2: a very first version of bundle2 unbundler Pierre-Yves David <pierre-yves.david@fb.com> parents: 20801 diff changeset	711 self._fp = fp
520df53ad26a bundle2: a very first version of bundle2 unbundler Pierre-Yves David <pierre-yves.david@fb.com> parents: 20801 diff changeset	712
520df53ad26a bundle2: a very first version of bundle2 unbundler Pierre-Yves David <pierre-yves.david@fb.com> parents: 20801 diff changeset	713 def _unpack(self, format):
31862 9bd9e9cb3f69 bundle2: clarify the docstring of unpackermixin methods Pierre-Yves David <pierre-yves.david@ens-lyon.org> parents: 31861 diff changeset	714 """unpack this struct format from the stream
9bd9e9cb3f69 bundle2: clarify the docstring of unpackermixin methods Pierre-Yves David <pierre-yves.david@ens-lyon.org> parents: 31861 diff changeset	715
9bd9e9cb3f69 bundle2: clarify the docstring of unpackermixin methods Pierre-Yves David <pierre-yves.david@ens-lyon.org> parents: 31861 diff changeset	716 This method is meant for internal usage by the bundle2 protocol only.
9bd9e9cb3f69 bundle2: clarify the docstring of unpackermixin methods Pierre-Yves David <pierre-yves.david@ens-lyon.org> parents: 31861 diff changeset	717 They directly manipulate the low level stream including bundle2 level
9bd9e9cb3f69 bundle2: clarify the docstring of unpackermixin methods Pierre-Yves David <pierre-yves.david@ens-lyon.org> parents: 31861 diff changeset	718 instruction.
9bd9e9cb3f69 bundle2: clarify the docstring of unpackermixin methods Pierre-Yves David <pierre-yves.david@ens-lyon.org> parents: 31861 diff changeset	719
9bd9e9cb3f69 bundle2: clarify the docstring of unpackermixin methods Pierre-Yves David <pierre-yves.david@ens-lyon.org> parents: 31861 diff changeset	720 Do not use it to implement higher-level logic or methods."""
20802 520df53ad26a bundle2: a very first version of bundle2 unbundler Pierre-Yves David <pierre-yves.david@fb.com> parents: 20801 diff changeset	721 data = self._readexact(struct.calcsize(format))
520df53ad26a bundle2: a very first version of bundle2 unbundler Pierre-Yves David <pierre-yves.david@fb.com> parents: 20801 diff changeset	722 return _unpack(format, data)
520df53ad26a bundle2: a very first version of bundle2 unbundler Pierre-Yves David <pierre-yves.david@fb.com> parents: 20801 diff changeset	723
520df53ad26a bundle2: a very first version of bundle2 unbundler Pierre-Yves David <pierre-yves.david@fb.com> parents: 20801 diff changeset	724 def _readexact(self, size):
31862 9bd9e9cb3f69 bundle2: clarify the docstring of unpackermixin methods Pierre-Yves David <pierre-yves.david@ens-lyon.org> parents: 31861 diff changeset	725 """read exactly <size> bytes from the stream
9bd9e9cb3f69 bundle2: clarify the docstring of unpackermixin methods Pierre-Yves David <pierre-yves.david@ens-lyon.org> parents: 31861 diff changeset	726
9bd9e9cb3f69 bundle2: clarify the docstring of unpackermixin methods Pierre-Yves David <pierre-yves.david@ens-lyon.org> parents: 31861 diff changeset	727 This method is meant for internal usage by the bundle2 protocol only.
9bd9e9cb3f69 bundle2: clarify the docstring of unpackermixin methods Pierre-Yves David <pierre-yves.david@ens-lyon.org> parents: 31861 diff changeset	728 They directly manipulate the low level stream including bundle2 level
9bd9e9cb3f69 bundle2: clarify the docstring of unpackermixin methods Pierre-Yves David <pierre-yves.david@ens-lyon.org> parents: 31861 diff changeset	729 instruction.
9bd9e9cb3f69 bundle2: clarify the docstring of unpackermixin methods Pierre-Yves David <pierre-yves.david@ens-lyon.org> parents: 31861 diff changeset	730
9bd9e9cb3f69 bundle2: clarify the docstring of unpackermixin methods Pierre-Yves David <pierre-yves.david@ens-lyon.org> parents: 31861 diff changeset	731 Do not use it to implement higher-level logic or methods."""
20802 520df53ad26a bundle2: a very first version of bundle2 unbundler Pierre-Yves David <pierre-yves.david@fb.com> parents: 20801 diff changeset	732 return changegroup.readexactly(self._fp, size)
520df53ad26a bundle2: a very first version of bundle2 unbundler Pierre-Yves David <pierre-yves.david@fb.com> parents: 20801 diff changeset	733
25640 39f0064a3079 bundle2.getunbundler: rename "header" to "magicstring" Pierre-Yves David <pierre-yves.david@fb.com> parents: 25518 diff changeset	734 def getunbundler(ui, fp, magicstring=None):
39f0064a3079 bundle2.getunbundler: rename "header" to "magicstring" Pierre-Yves David <pierre-yves.david@fb.com> parents: 25518 diff changeset	735 """return a valid unbundler object for a given magicstring"""
39f0064a3079 bundle2.getunbundler: rename "header" to "magicstring" Pierre-Yves David <pierre-yves.david@fb.com> parents: 25518 diff changeset	736 if magicstring is None:
39f0064a3079 bundle2.getunbundler: rename "header" to "magicstring" Pierre-Yves David <pierre-yves.david@fb.com> parents: 25518 diff changeset	737 magicstring = changegroup.readexactly(fp, 4)
39f0064a3079 bundle2.getunbundler: rename "header" to "magicstring" Pierre-Yves David <pierre-yves.david@fb.com> parents: 25518 diff changeset	738 magic, version = magicstring[0:2], magicstring[2:4]
24648 5cac3accdaa1 unbundle20: allow generic dispatch between unbundlers Pierre-Yves David <pierre-yves.david@fb.com> parents: 24642 diff changeset	739 if magic != 'HG':
33127 0aae80d14fed bundle2: add some debugging information to the not-a-bundle error Siddharth Agarwal <sid0@fb.com> parents: 33044 diff changeset	740 ui.debug(
0aae80d14fed bundle2: add some debugging information to the not-a-bundle error Siddharth Agarwal <sid0@fb.com> parents: 33044 diff changeset	741 "error: invalid magic: %r (version %r), should be 'HG'\n"
0aae80d14fed bundle2: add some debugging information to the not-a-bundle error Siddharth Agarwal <sid0@fb.com> parents: 33044 diff changeset	742 % (magic, version))
26587 56b2bcea2529 error: get Abort from 'error' instead of 'util' Pierre-Yves David <pierre-yves.david@fb.com> parents: 26565 diff changeset	743 raise error.Abort(_('not a Mercurial bundle'))
24648 5cac3accdaa1 unbundle20: allow generic dispatch between unbundlers Pierre-Yves David <pierre-yves.david@fb.com> parents: 24642 diff changeset	744 unbundlerclass = formatmap.get(version)
5cac3accdaa1 unbundle20: allow generic dispatch between unbundlers Pierre-Yves David <pierre-yves.david@fb.com> parents: 24642 diff changeset	745 if unbundlerclass is None:
26587 56b2bcea2529 error: get Abort from 'error' instead of 'util' Pierre-Yves David <pierre-yves.david@fb.com> parents: 26565 diff changeset	746 raise error.Abort(_('unknown bundle version %s') % version)
24648 5cac3accdaa1 unbundle20: allow generic dispatch between unbundlers Pierre-Yves David <pierre-yves.david@fb.com> parents: 24642 diff changeset	747 unbundler = unbundlerclass(ui, fp)
25640 39f0064a3079 bundle2.getunbundler: rename "header" to "magicstring" Pierre-Yves David <pierre-yves.david@fb.com> parents: 25518 diff changeset	748 indebug(ui, 'start processing of %s stream' % magicstring)
24642 54e5c239c2d9 unbundle20: move header parsing into the 'getunbundler' function Pierre-Yves David <pierre-yves.david@fb.com> parents: 24641 diff changeset	749 return unbundler
24641 60fecc5b14a4 unbundle20: retrieve unbundler instances through a factory function Pierre-Yves David <pierre-yves.david@fb.com> parents: 24640 diff changeset	750
21013 a813caca89b3 bundle2: extract stream/unpack logic in an unpackermixin Pierre-Yves David <pierre-yves.david@fb.com> parents: 21005 diff changeset	751 class unbundle20(unpackermixin):
a813caca89b3 bundle2: extract stream/unpack logic in an unpackermixin Pierre-Yves David <pierre-yves.david@fb.com> parents: 21005 diff changeset	752 """interpret a bundle2 stream
a813caca89b3 bundle2: extract stream/unpack logic in an unpackermixin Pierre-Yves David <pierre-yves.david@fb.com> parents: 21005 diff changeset	753
21129 07bcbf326c8d bundle2: use an official iterparts method to unbundle parts Pierre-Yves David <pierre-yves.david@fb.com> parents: 21066 diff changeset	754 This class is fed with a binary stream and yields parts through its
07bcbf326c8d bundle2: use an official iterparts method to unbundle parts Pierre-Yves David <pierre-yves.david@fb.com> parents: 21066 diff changeset	755 `iterparts` methods."""
21013 a813caca89b3 bundle2: extract stream/unpack logic in an unpackermixin Pierre-Yves David <pierre-yves.david@fb.com> parents: 21005 diff changeset	756
26542 b87e4638dabf bundle2: add a way to just forward the bundle2 stream to another user Pierre-Yves David <pierre-yves.david@fb.com> parents: 26541 diff changeset	757 _magicstring = 'HG20'
b87e4638dabf bundle2: add a way to just forward the bundle2 stream to another user Pierre-Yves David <pierre-yves.david@fb.com> parents: 26541 diff changeset	758
24642 54e5c239c2d9 unbundle20: move header parsing into the 'getunbundler' function Pierre-Yves David <pierre-yves.david@fb.com> parents: 24641 diff changeset	759 def __init__(self, ui, fp):
21066 5ecfe76d0d96 bundle2: make header reading optional Pierre-Yves David <pierre-yves.david@fb.com> parents: 21062 diff changeset	760 """If header is specified, we do not read it out of the stream."""
21013 a813caca89b3 bundle2: extract stream/unpack logic in an unpackermixin Pierre-Yves David <pierre-yves.david@fb.com> parents: 21005 diff changeset	761 self.ui = ui
30353 d045b4091197 bundle2: use compression engines API to obtain decompressor Gregory Szorc <gregory.szorc@gmail.com> parents: 30351 diff changeset	762 self._compengine = util.compengines.forbundletype('UN')
26802 42f705f2c02d bundle2: make unbundle.compressed return True when compressed Pierre-Yves David <pierre-yves.david@fb.com> parents: 26795 diff changeset	763 self._compressed = None
21013 a813caca89b3 bundle2: extract stream/unpack logic in an unpackermixin Pierre-Yves David <pierre-yves.david@fb.com> parents: 21005 diff changeset	764 super(unbundle20, self).__init__(fp)
a813caca89b3 bundle2: extract stream/unpack logic in an unpackermixin Pierre-Yves David <pierre-yves.david@fb.com> parents: 21005 diff changeset	765
20802 520df53ad26a bundle2: a very first version of bundle2 unbundler Pierre-Yves David <pierre-yves.david@fb.com> parents: 20801 diff changeset	766 @util.propertycache
520df53ad26a bundle2: a very first version of bundle2 unbundler Pierre-Yves David <pierre-yves.david@fb.com> parents: 20801 diff changeset	767 def params(self):
21024 7731a2281cf0 spelling: fixes from spell checker Mads Kiilerich <madski@unity3d.com> parents: 21020 diff changeset	768 """dictionary of stream level parameters"""
25320 697d8953b04d bundle2: handle new line in 'indebug' function Pierre-Yves David <pierre-yves.david@fb.com> parents: 25319 diff changeset	769 indebug(self.ui, 'reading bundle2 stream parameters')
20805 c5aaeca0cfbf bundle2: support for unbundling simple parameter Pierre-Yves David <pierre-yves.david@fb.com> parents: 20804 diff changeset	770 params = {}
c5aaeca0cfbf bundle2: support for unbundling simple parameter Pierre-Yves David <pierre-yves.david@fb.com> parents: 20804 diff changeset	771 paramssize = self._unpack(_fstreamparamsize)[0]
23011 006a81d07e57 bundle2: detect and disallow a negative chunk size Pierre-Yves David <pierre-yves.david@fb.com> parents: 23010 diff changeset	772 if paramssize < 0:
006a81d07e57 bundle2: detect and disallow a negative chunk size Pierre-Yves David <pierre-yves.david@fb.com> parents: 23010 diff changeset	773 raise error.BundleValueError('negative bundle param size: %i'
006a81d07e57 bundle2: detect and disallow a negative chunk size Pierre-Yves David <pierre-yves.david@fb.com> parents: 23010 diff changeset	774 % paramssize)
20805 c5aaeca0cfbf bundle2: support for unbundling simple parameter Pierre-Yves David <pierre-yves.david@fb.com> parents: 20804 diff changeset	775 if paramssize:
26541 d40029b4296e bundle2: split parameter retrieval and processing Pierre-Yves David <pierre-yves.david@fb.com> parents: 26404 diff changeset	776 params = self._readexact(paramssize)
d40029b4296e bundle2: split parameter retrieval and processing Pierre-Yves David <pierre-yves.david@fb.com> parents: 26404 diff changeset	777 params = self._processallparams(params)
20805 c5aaeca0cfbf bundle2: support for unbundling simple parameter Pierre-Yves David <pierre-yves.david@fb.com> parents: 20804 diff changeset	778 return params
20802 520df53ad26a bundle2: a very first version of bundle2 unbundler Pierre-Yves David <pierre-yves.david@fb.com> parents: 20801 diff changeset	779
26541 d40029b4296e bundle2: split parameter retrieval and processing Pierre-Yves David <pierre-yves.david@fb.com> parents: 26404 diff changeset	780 def _processallparams(self, paramsblock):
d40029b4296e bundle2: split parameter retrieval and processing Pierre-Yves David <pierre-yves.david@fb.com> parents: 26404 diff changeset	781 """"""
29591 6215b5537ba5 bundle2: use a sorted dict for holding parameters Gregory Szorc <gregory.szorc@gmail.com> parents: 29294 diff changeset	782 params = util.sortdict()
26541 d40029b4296e bundle2: split parameter retrieval and processing Pierre-Yves David <pierre-yves.david@fb.com> parents: 26404 diff changeset	783 for p in paramsblock.split(' '):
d40029b4296e bundle2: split parameter retrieval and processing Pierre-Yves David <pierre-yves.david@fb.com> parents: 26404 diff changeset	784 p = p.split('=', 1)
28883 032c4c2f802a pycompat: switch to util.urlreq/util.urlerr for py3 compat timeless <timeless@mozdev.org> parents: 28672 diff changeset	785 p = [urlreq.unquote(i) for i in p]
26541 d40029b4296e bundle2: split parameter retrieval and processing Pierre-Yves David <pierre-yves.david@fb.com> parents: 26404 diff changeset	786 if len(p) < 2:
d40029b4296e bundle2: split parameter retrieval and processing Pierre-Yves David <pierre-yves.david@fb.com> parents: 26404 diff changeset	787 p.append(None)
d40029b4296e bundle2: split parameter retrieval and processing Pierre-Yves David <pierre-yves.david@fb.com> parents: 26404 diff changeset	788 self._processparam(*p)
d40029b4296e bundle2: split parameter retrieval and processing Pierre-Yves David <pierre-yves.david@fb.com> parents: 26404 diff changeset	789 params[p[0]] = p[1]
d40029b4296e bundle2: split parameter retrieval and processing Pierre-Yves David <pierre-yves.david@fb.com> parents: 26404 diff changeset	790 return params
d40029b4296e bundle2: split parameter retrieval and processing Pierre-Yves David <pierre-yves.david@fb.com> parents: 26404 diff changeset	791
d40029b4296e bundle2: split parameter retrieval and processing Pierre-Yves David <pierre-yves.david@fb.com> parents: 26404 diff changeset	792
20844 2631204d7305 bundle2: implement the mandatory/advisory logic for parameter Pierre-Yves David <pierre-yves.david@fb.com> parents: 20843 diff changeset	793 def _processparam(self, name, value):
2631204d7305 bundle2: implement the mandatory/advisory logic for parameter Pierre-Yves David <pierre-yves.david@fb.com> parents: 20843 diff changeset	794 """process a parameter, applying its effect if needed
2631204d7305 bundle2: implement the mandatory/advisory logic for parameter Pierre-Yves David <pierre-yves.david@fb.com> parents: 20843 diff changeset	795
2631204d7305 bundle2: implement the mandatory/advisory logic for parameter Pierre-Yves David <pierre-yves.david@fb.com> parents: 20843 diff changeset	796 Parameter starting with a lower case letter are advisory and will be
2631204d7305 bundle2: implement the mandatory/advisory logic for parameter Pierre-Yves David <pierre-yves.david@fb.com> parents: 20843 diff changeset	797 ignored when unknown. Those starting with an upper case letter are
2631204d7305 bundle2: implement the mandatory/advisory logic for parameter Pierre-Yves David <pierre-yves.david@fb.com> parents: 20843 diff changeset	798 mandatory and will this function will raise a KeyError when unknown.
2631204d7305 bundle2: implement the mandatory/advisory logic for parameter Pierre-Yves David <pierre-yves.david@fb.com> parents: 20843 diff changeset	799
2631204d7305 bundle2: implement the mandatory/advisory logic for parameter Pierre-Yves David <pierre-yves.david@fb.com> parents: 20843 diff changeset	800 Note: no option are currently supported. Any input will be either
2631204d7305 bundle2: implement the mandatory/advisory logic for parameter Pierre-Yves David <pierre-yves.david@fb.com> parents: 20843 diff changeset	801 ignored or failing.
2631204d7305 bundle2: implement the mandatory/advisory logic for parameter Pierre-Yves David <pierre-yves.david@fb.com> parents: 20843 diff changeset	802 """
2631204d7305 bundle2: implement the mandatory/advisory logic for parameter Pierre-Yves David <pierre-yves.david@fb.com> parents: 20843 diff changeset	803 if not name:
34277 a48b3db0baba bundle2: stop using %r to quote part names Augie Fackler <augie@google.com> parents: 34261 diff changeset	804 raise ValueError(r'empty parameter name')
a48b3db0baba bundle2: stop using %r to quote part names Augie Fackler <augie@google.com> parents: 34261 diff changeset	805 if name[0:1] not in pycompat.bytestr(string.ascii_letters):
a48b3db0baba bundle2: stop using %r to quote part names Augie Fackler <augie@google.com> parents: 34261 diff changeset	806 raise ValueError(r'non letter first character: %s' % name)
26395 4e7b0bf9f0b1 unbundle20: allow registering handlers for stream level parameters Pierre-Yves David <pierre-yves.david@fb.com> parents: 26393 diff changeset	807 try:
4e7b0bf9f0b1 unbundle20: allow registering handlers for stream level parameters Pierre-Yves David <pierre-yves.david@fb.com> parents: 26393 diff changeset	808 handler = b2streamparamsmap[name.lower()]
4e7b0bf9f0b1 unbundle20: allow registering handlers for stream level parameters Pierre-Yves David <pierre-yves.david@fb.com> parents: 26393 diff changeset	809 except KeyError:
34277 a48b3db0baba bundle2: stop using %r to quote part names Augie Fackler <augie@google.com> parents: 34261 diff changeset	810 if name[0:1].islower():
a48b3db0baba bundle2: stop using %r to quote part names Augie Fackler <augie@google.com> parents: 34261 diff changeset	811 indebug(self.ui, "ignoring unknown parameter %s" % name)
26395 4e7b0bf9f0b1 unbundle20: allow registering handlers for stream level parameters Pierre-Yves David <pierre-yves.david@fb.com> parents: 26393 diff changeset	812 else:
4e7b0bf9f0b1 unbundle20: allow registering handlers for stream level parameters Pierre-Yves David <pierre-yves.david@fb.com> parents: 26393 diff changeset	813 raise error.BundleUnknownFeatureError(params=(name,))
20844 2631204d7305 bundle2: implement the mandatory/advisory logic for parameter Pierre-Yves David <pierre-yves.david@fb.com> parents: 20843 diff changeset	814 else:
26395 4e7b0bf9f0b1 unbundle20: allow registering handlers for stream level parameters Pierre-Yves David <pierre-yves.david@fb.com> parents: 26393 diff changeset	815 handler(self, name, value)
20844 2631204d7305 bundle2: implement the mandatory/advisory logic for parameter Pierre-Yves David <pierre-yves.david@fb.com> parents: 20843 diff changeset	816
26542 b87e4638dabf bundle2: add a way to just forward the bundle2 stream to another user Pierre-Yves David <pierre-yves.david@fb.com> parents: 26541 diff changeset	817 def _forwardchunks(self):
b87e4638dabf bundle2: add a way to just forward the bundle2 stream to another user Pierre-Yves David <pierre-yves.david@fb.com> parents: 26541 diff changeset	818 """utility to transfer a bundle2 as binary
b87e4638dabf bundle2: add a way to just forward the bundle2 stream to another user Pierre-Yves David <pierre-yves.david@fb.com> parents: 26541 diff changeset	819
b87e4638dabf bundle2: add a way to just forward the bundle2 stream to another user Pierre-Yves David <pierre-yves.david@fb.com> parents: 26541 diff changeset	820 This is made necessary by the fact the 'getbundle' command over 'ssh'
b87e4638dabf bundle2: add a way to just forward the bundle2 stream to another user Pierre-Yves David <pierre-yves.david@fb.com> parents: 26541 diff changeset	821 have no way to know then the reply end, relying on the bundle to be
b87e4638dabf bundle2: add a way to just forward the bundle2 stream to another user Pierre-Yves David <pierre-yves.david@fb.com> parents: 26541 diff changeset	822 interpreted to know its end. This is terrible and we are sorry, but we
b87e4638dabf bundle2: add a way to just forward the bundle2 stream to another user Pierre-Yves David <pierre-yves.david@fb.com> parents: 26541 diff changeset	823 needed to move forward to get general delta enabled.
b87e4638dabf bundle2: add a way to just forward the bundle2 stream to another user Pierre-Yves David <pierre-yves.david@fb.com> parents: 26541 diff changeset	824 """
b87e4638dabf bundle2: add a way to just forward the bundle2 stream to another user Pierre-Yves David <pierre-yves.david@fb.com> parents: 26541 diff changeset	825 yield self._magicstring
b87e4638dabf bundle2: add a way to just forward the bundle2 stream to another user Pierre-Yves David <pierre-yves.david@fb.com> parents: 26541 diff changeset	826 assert 'params' not in vars(self)
b87e4638dabf bundle2: add a way to just forward the bundle2 stream to another user Pierre-Yves David <pierre-yves.david@fb.com> parents: 26541 diff changeset	827 paramssize = self._unpack(_fstreamparamsize)[0]
b87e4638dabf bundle2: add a way to just forward the bundle2 stream to another user Pierre-Yves David <pierre-yves.david@fb.com> parents: 26541 diff changeset	828 if paramssize < 0:
b87e4638dabf bundle2: add a way to just forward the bundle2 stream to another user Pierre-Yves David <pierre-yves.david@fb.com> parents: 26541 diff changeset	829 raise error.BundleValueError('negative bundle param size: %i'
b87e4638dabf bundle2: add a way to just forward the bundle2 stream to another user Pierre-Yves David <pierre-yves.david@fb.com> parents: 26541 diff changeset	830 % paramssize)
b87e4638dabf bundle2: add a way to just forward the bundle2 stream to another user Pierre-Yves David <pierre-yves.david@fb.com> parents: 26541 diff changeset	831 yield _pack(_fstreamparamsize, paramssize)
b87e4638dabf bundle2: add a way to just forward the bundle2 stream to another user Pierre-Yves David <pierre-yves.david@fb.com> parents: 26541 diff changeset	832 if paramssize:
b87e4638dabf bundle2: add a way to just forward the bundle2 stream to another user Pierre-Yves David <pierre-yves.david@fb.com> parents: 26541 diff changeset	833 params = self._readexact(paramssize)
b87e4638dabf bundle2: add a way to just forward the bundle2 stream to another user Pierre-Yves David <pierre-yves.david@fb.com> parents: 26541 diff changeset	834 self._processallparams(params)
b87e4638dabf bundle2: add a way to just forward the bundle2 stream to another user Pierre-Yves David <pierre-yves.david@fb.com> parents: 26541 diff changeset	835 yield params
30353 d045b4091197 bundle2: use compression engines API to obtain decompressor Gregory Szorc <gregory.szorc@gmail.com> parents: 30351 diff changeset	836 assert self._compengine.bundletype == 'UN'
26542 b87e4638dabf bundle2: add a way to just forward the bundle2 stream to another user Pierre-Yves David <pierre-yves.david@fb.com> parents: 26541 diff changeset	837 # From there, payload might need to be decompressed
30353 d045b4091197 bundle2: use compression engines API to obtain decompressor Gregory Szorc <gregory.szorc@gmail.com> parents: 30351 diff changeset	838 self._fp = self._compengine.decompressorreader(self._fp)
26542 b87e4638dabf bundle2: add a way to just forward the bundle2 stream to another user Pierre-Yves David <pierre-yves.david@fb.com> parents: 26541 diff changeset	839 emptycount = 0
b87e4638dabf bundle2: add a way to just forward the bundle2 stream to another user Pierre-Yves David <pierre-yves.david@fb.com> parents: 26541 diff changeset	840 while emptycount < 2:
b87e4638dabf bundle2: add a way to just forward the bundle2 stream to another user Pierre-Yves David <pierre-yves.david@fb.com> parents: 26541 diff changeset	841 # so we can brainlessly loop
b87e4638dabf bundle2: add a way to just forward the bundle2 stream to another user Pierre-Yves David <pierre-yves.david@fb.com> parents: 26541 diff changeset	842 assert _fpartheadersize == _fpayloadsize
b87e4638dabf bundle2: add a way to just forward the bundle2 stream to another user Pierre-Yves David <pierre-yves.david@fb.com> parents: 26541 diff changeset	843 size = self._unpack(_fpartheadersize)[0]
b87e4638dabf bundle2: add a way to just forward the bundle2 stream to another user Pierre-Yves David <pierre-yves.david@fb.com> parents: 26541 diff changeset	844 yield _pack(_fpartheadersize, size)
b87e4638dabf bundle2: add a way to just forward the bundle2 stream to another user Pierre-Yves David <pierre-yves.david@fb.com> parents: 26541 diff changeset	845 if size:
b87e4638dabf bundle2: add a way to just forward the bundle2 stream to another user Pierre-Yves David <pierre-yves.david@fb.com> parents: 26541 diff changeset	846 emptycount = 0
b87e4638dabf bundle2: add a way to just forward the bundle2 stream to another user Pierre-Yves David <pierre-yves.david@fb.com> parents: 26541 diff changeset	847 else:
b87e4638dabf bundle2: add a way to just forward the bundle2 stream to another user Pierre-Yves David <pierre-yves.david@fb.com> parents: 26541 diff changeset	848 emptycount += 1
b87e4638dabf bundle2: add a way to just forward the bundle2 stream to another user Pierre-Yves David <pierre-yves.david@fb.com> parents: 26541 diff changeset	849 continue
b87e4638dabf bundle2: add a way to just forward the bundle2 stream to another user Pierre-Yves David <pierre-yves.david@fb.com> parents: 26541 diff changeset	850 if size == flaginterrupt:
b87e4638dabf bundle2: add a way to just forward the bundle2 stream to another user Pierre-Yves David <pierre-yves.david@fb.com> parents: 26541 diff changeset	851 continue
b87e4638dabf bundle2: add a way to just forward the bundle2 stream to another user Pierre-Yves David <pierre-yves.david@fb.com> parents: 26541 diff changeset	852 elif size < 0:
b87e4638dabf bundle2: add a way to just forward the bundle2 stream to another user Pierre-Yves David <pierre-yves.david@fb.com> parents: 26541 diff changeset	853 raise error.BundleValueError('negative chunk size: %i')
b87e4638dabf bundle2: add a way to just forward the bundle2 stream to another user Pierre-Yves David <pierre-yves.david@fb.com> parents: 26541 diff changeset	854 yield self._readexact(size)
b87e4638dabf bundle2: add a way to just forward the bundle2 stream to another user Pierre-Yves David <pierre-yves.david@fb.com> parents: 26541 diff changeset	855
b87e4638dabf bundle2: add a way to just forward the bundle2 stream to another user Pierre-Yves David <pierre-yves.david@fb.com> parents: 26541 diff changeset	856
35116 da91e7309daf bundle2: don't use seekable bundle2 parts by default (issue5691) Gregory Szorc <gregory.szorc@gmail.com> parents: 35115 diff changeset	857 def iterparts(self, seekable=False):
20802 520df53ad26a bundle2: a very first version of bundle2 unbundler Pierre-Yves David <pierre-yves.david@fb.com> parents: 20801 diff changeset	858 """yield all parts contained in the stream"""
35116 da91e7309daf bundle2: don't use seekable bundle2 parts by default (issue5691) Gregory Szorc <gregory.szorc@gmail.com> parents: 35115 diff changeset	859 cls = seekableunbundlepart if seekable else unbundlepart
20802 520df53ad26a bundle2: a very first version of bundle2 unbundler Pierre-Yves David <pierre-yves.david@fb.com> parents: 20801 diff changeset	860 # make sure param have been loaded
520df53ad26a bundle2: a very first version of bundle2 unbundler Pierre-Yves David <pierre-yves.david@fb.com> parents: 20801 diff changeset	861 self.params
26404 795f02a24b87 bundle2: allow compressed bundle Pierre-Yves David <pierre-yves.david@fb.com> parents: 26396 diff changeset	862 # From there, payload need to be decompressed
30353 d045b4091197 bundle2: use compression engines API to obtain decompressor Gregory Szorc <gregory.szorc@gmail.com> parents: 30351 diff changeset	863 self._fp = self._compengine.decompressorreader(self._fp)
25320 697d8953b04d bundle2: handle new line in 'indebug' function Pierre-Yves David <pierre-yves.david@fb.com> parents: 25319 diff changeset	864 indebug(self.ui, 'start extraction of bundle2 parts')
21014 a6246bba7b9e bundle2: add an unbundle part responsible from unbundling part Pierre-Yves David <pierre-yves.david@fb.com> parents: 21013 diff changeset	865 headerblock = self._readpartheader()
a6246bba7b9e bundle2: add an unbundle part responsible from unbundling part Pierre-Yves David <pierre-yves.david@fb.com> parents: 21013 diff changeset	866 while headerblock is not None:
35116 da91e7309daf bundle2: don't use seekable bundle2 parts by default (issue5691) Gregory Szorc <gregory.szorc@gmail.com> parents: 35115 diff changeset	867 part = cls(self.ui, headerblock, self._fp)
20802 520df53ad26a bundle2: a very first version of bundle2 unbundler Pierre-Yves David <pierre-yves.david@fb.com> parents: 20801 diff changeset	868 yield part
35114 db5038525718 bundle2: implement consume() API on unbundlepart Gregory Szorc <gregory.szorc@gmail.com> parents: 35113 diff changeset	869 # Ensure part is fully consumed so we can start reading the next
db5038525718 bundle2: implement consume() API on unbundlepart Gregory Szorc <gregory.szorc@gmail.com> parents: 35113 diff changeset	870 # part.
db5038525718 bundle2: implement consume() API on unbundlepart Gregory Szorc <gregory.szorc@gmail.com> parents: 35113 diff changeset	871 part.consume()
35115 2b72bc88043f bundle2: only seek to beginning of part in bundlerepo Gregory Szorc <gregory.szorc@gmail.com> parents: 35114 diff changeset	872
21014 a6246bba7b9e bundle2: add an unbundle part responsible from unbundling part Pierre-Yves David <pierre-yves.david@fb.com> parents: 21013 diff changeset	873 headerblock = self._readpartheader()
25320 697d8953b04d bundle2: handle new line in 'indebug' function Pierre-Yves David <pierre-yves.david@fb.com> parents: 25319 diff changeset	874 indebug(self.ui, 'end of bundle2 stream')
20802 520df53ad26a bundle2: a very first version of bundle2 unbundler Pierre-Yves David <pierre-yves.david@fb.com> parents: 20801 diff changeset	875
21014 a6246bba7b9e bundle2: add an unbundle part responsible from unbundling part Pierre-Yves David <pierre-yves.david@fb.com> parents: 21013 diff changeset	876 def _readpartheader(self):
a6246bba7b9e bundle2: add an unbundle part responsible from unbundling part Pierre-Yves David <pierre-yves.david@fb.com> parents: 21013 diff changeset	877 """reads a part header size and return the bytes blob
20864 9a75d2559cff bundle2: support unbundling empty part Pierre-Yves David <pierre-yves.david@fb.com> parents: 20856 diff changeset	878
21014 a6246bba7b9e bundle2: add an unbundle part responsible from unbundling part Pierre-Yves David <pierre-yves.david@fb.com> parents: 21013 diff changeset	879 returns None if empty"""
20864 9a75d2559cff bundle2: support unbundling empty part Pierre-Yves David <pierre-yves.david@fb.com> parents: 20856 diff changeset	880 headersize = self._unpack(_fpartheadersize)[0]
23011 006a81d07e57 bundle2: detect and disallow a negative chunk size Pierre-Yves David <pierre-yves.david@fb.com> parents: 23010 diff changeset	881 if headersize < 0:
006a81d07e57 bundle2: detect and disallow a negative chunk size Pierre-Yves David <pierre-yves.david@fb.com> parents: 23010 diff changeset	882 raise error.BundleValueError('negative part header size: %i'
006a81d07e57 bundle2: detect and disallow a negative chunk size Pierre-Yves David <pierre-yves.david@fb.com> parents: 23010 diff changeset	883 % headersize)
25320 697d8953b04d bundle2: handle new line in 'indebug' function Pierre-Yves David <pierre-yves.david@fb.com> parents: 25319 diff changeset	884 indebug(self.ui, 'part header size: %i' % headersize)
21014 a6246bba7b9e bundle2: add an unbundle part responsible from unbundling part Pierre-Yves David <pierre-yves.david@fb.com> parents: 21013 diff changeset	885 if headersize:
a6246bba7b9e bundle2: add an unbundle part responsible from unbundling part Pierre-Yves David <pierre-yves.david@fb.com> parents: 21013 diff changeset	886 return self._readexact(headersize)
a6246bba7b9e bundle2: add an unbundle part responsible from unbundling part Pierre-Yves David <pierre-yves.david@fb.com> parents: 21013 diff changeset	887 return None
20864 9a75d2559cff bundle2: support unbundling empty part Pierre-Yves David <pierre-yves.david@fb.com> parents: 20856 diff changeset	888
24071 184a2f6f40da bundle2.unbundle20: add compressed() method Eric Sumner <ericsumner@fb.com> parents: 24070 diff changeset	889 def compressed(self):
26802 42f705f2c02d bundle2: make unbundle.compressed return True when compressed Pierre-Yves David <pierre-yves.david@fb.com> parents: 26795 diff changeset	890 self.params # load params
42f705f2c02d bundle2: make unbundle.compressed return True when compressed Pierre-Yves David <pierre-yves.david@fb.com> parents: 26795 diff changeset	891 return self._compressed
20802 520df53ad26a bundle2: a very first version of bundle2 unbundler Pierre-Yves David <pierre-yves.david@fb.com> parents: 20801 diff changeset	892
31863 cd7aaf344d83 bundle2: move the 'close' method off the unpackermixin Pierre-Yves David <pierre-yves.david@ens-lyon.org> parents: 31862 diff changeset	893 def close(self):
cd7aaf344d83 bundle2: move the 'close' method off the unpackermixin Pierre-Yves David <pierre-yves.david@ens-lyon.org> parents: 31862 diff changeset	894 """close underlying file"""
cd7aaf344d83 bundle2: move the 'close' method off the unpackermixin Pierre-Yves David <pierre-yves.david@ens-lyon.org> parents: 31862 diff changeset	895 if util.safehasattr(self._fp, 'close'):
cd7aaf344d83 bundle2: move the 'close' method off the unpackermixin Pierre-Yves David <pierre-yves.david@ens-lyon.org> parents: 31862 diff changeset	896 return self._fp.close()
cd7aaf344d83 bundle2: move the 'close' method off the unpackermixin Pierre-Yves David <pierre-yves.david@ens-lyon.org> parents: 31862 diff changeset	897
24686 e0e28e910fa3 bundle2: rename format, parts and config to final names Pierre-Yves David <pierre-yves.david@fb.com> parents: 24648 diff changeset	898 formatmap = {'20': unbundle20}
24648 5cac3accdaa1 unbundle20: allow generic dispatch between unbundlers Pierre-Yves David <pierre-yves.david@fb.com> parents: 24642 diff changeset	899
26395 4e7b0bf9f0b1 unbundle20: allow registering handlers for stream level parameters Pierre-Yves David <pierre-yves.david@fb.com> parents: 26393 diff changeset	900 b2streamparamsmap = {}
4e7b0bf9f0b1 unbundle20: allow registering handlers for stream level parameters Pierre-Yves David <pierre-yves.david@fb.com> parents: 26393 diff changeset	901
4e7b0bf9f0b1 unbundle20: allow registering handlers for stream level parameters Pierre-Yves David <pierre-yves.david@fb.com> parents: 26393 diff changeset	902 def b2streamparamhandler(name):
4e7b0bf9f0b1 unbundle20: allow registering handlers for stream level parameters Pierre-Yves David <pierre-yves.david@fb.com> parents: 26393 diff changeset	903 """register a handler for a stream level parameter"""
4e7b0bf9f0b1 unbundle20: allow registering handlers for stream level parameters Pierre-Yves David <pierre-yves.david@fb.com> parents: 26393 diff changeset	904 def decorator(func):
4e7b0bf9f0b1 unbundle20: allow registering handlers for stream level parameters Pierre-Yves David <pierre-yves.david@fb.com> parents: 26393 diff changeset	905 assert name not in formatmap
4e7b0bf9f0b1 unbundle20: allow registering handlers for stream level parameters Pierre-Yves David <pierre-yves.david@fb.com> parents: 26393 diff changeset	906 b2streamparamsmap[name] = func
4e7b0bf9f0b1 unbundle20: allow registering handlers for stream level parameters Pierre-Yves David <pierre-yves.david@fb.com> parents: 26393 diff changeset	907 return func
4e7b0bf9f0b1 unbundle20: allow registering handlers for stream level parameters Pierre-Yves David <pierre-yves.david@fb.com> parents: 26393 diff changeset	908 return decorator
4e7b0bf9f0b1 unbundle20: allow registering handlers for stream level parameters Pierre-Yves David <pierre-yves.david@fb.com> parents: 26393 diff changeset	909
26404 795f02a24b87 bundle2: allow compressed bundle Pierre-Yves David <pierre-yves.david@fb.com> parents: 26396 diff changeset	910 @b2streamparamhandler('compression')
795f02a24b87 bundle2: allow compressed bundle Pierre-Yves David <pierre-yves.david@fb.com> parents: 26396 diff changeset	911 def processcompression(unbundler, param, value):
795f02a24b87 bundle2: allow compressed bundle Pierre-Yves David <pierre-yves.david@fb.com> parents: 26396 diff changeset	912 """read compression parameter and install payload decompression"""
30353 d045b4091197 bundle2: use compression engines API to obtain decompressor Gregory Szorc <gregory.szorc@gmail.com> parents: 30351 diff changeset	913 if value not in util.compengines.supportedbundletypes:
26404 795f02a24b87 bundle2: allow compressed bundle Pierre-Yves David <pierre-yves.david@fb.com> parents: 26396 diff changeset	914 raise error.BundleUnknownFeatureError(params=(param,),
795f02a24b87 bundle2: allow compressed bundle Pierre-Yves David <pierre-yves.david@fb.com> parents: 26396 diff changeset	915 values=(value,))
30353 d045b4091197 bundle2: use compression engines API to obtain decompressor Gregory Szorc <gregory.szorc@gmail.com> parents: 30351 diff changeset	916 unbundler._compengine = util.compengines.forbundletype(value)
26802 42f705f2c02d bundle2: make unbundle.compressed return True when compressed Pierre-Yves David <pierre-yves.david@fb.com> parents: 26795 diff changeset	917 if value is not None:
42f705f2c02d bundle2: make unbundle.compressed return True when compressed Pierre-Yves David <pierre-yves.david@fb.com> parents: 26795 diff changeset	918 unbundler._compressed = True
26404 795f02a24b87 bundle2: allow compressed bundle Pierre-Yves David <pierre-yves.david@fb.com> parents: 26396 diff changeset	919
21005 3d38ebb586fe bundle2: rename part to bundlepart Pierre-Yves David <pierre-yves.david@fb.com> parents: 21004 diff changeset	920 class bundlepart(object):
20856 8a6a86c9a5b5 bundle2: support bundling of empty part (with a type) Pierre-Yves David <pierre-yves.david@fb.com> parents: 20844 diff changeset	921 """A bundle2 part contains application level payload
8a6a86c9a5b5 bundle2: support bundling of empty part (with a type) Pierre-Yves David <pierre-yves.david@fb.com> parents: 20844 diff changeset	922
8a6a86c9a5b5 bundle2: support bundling of empty part (with a type) Pierre-Yves David <pierre-yves.david@fb.com> parents: 20844 diff changeset	923 The part `type` is used to route the part to the application level
8a6a86c9a5b5 bundle2: support bundling of empty part (with a type) Pierre-Yves David <pierre-yves.david@fb.com> parents: 20844 diff changeset	924 handler.
21604 c399bf961cb9 bundle2: the ability to set ``data`` attribute of the part is now official Pierre-Yves David <pierre-yves.david@fb.com> parents: 21603 diff changeset	925
c399bf961cb9 bundle2: the ability to set ``data`` attribute of the part is now official Pierre-Yves David <pierre-yves.david@fb.com> parents: 21603 diff changeset	926 The part payload is contained in ``part.data``. It could be raw bytes or a
21605 f9dabfceb259 bundle2: introduce a ``addparam`` method on part Pierre-Yves David <pierre-yves.david@fb.com> parents: 21604 diff changeset	927 generator of byte chunks.
f9dabfceb259 bundle2: introduce a ``addparam`` method on part Pierre-Yves David <pierre-yves.david@fb.com> parents: 21604 diff changeset	928
f9dabfceb259 bundle2: introduce a ``addparam`` method on part Pierre-Yves David <pierre-yves.david@fb.com> parents: 21604 diff changeset	929 You can add parameters to the part using the ``addparam`` method.
f9dabfceb259 bundle2: introduce a ``addparam`` method on part Pierre-Yves David <pierre-yves.david@fb.com> parents: 21604 diff changeset	930 Parameters can be either mandatory (default) or advisory. Remote side
f9dabfceb259 bundle2: introduce a ``addparam`` method on part Pierre-Yves David <pierre-yves.david@fb.com> parents: 21604 diff changeset	931 should be able to safely ignore the advisory ones.
f9dabfceb259 bundle2: introduce a ``addparam`` method on part Pierre-Yves David <pierre-yves.david@fb.com> parents: 21604 diff changeset	932
f9dabfceb259 bundle2: introduce a ``addparam`` method on part Pierre-Yves David <pierre-yves.david@fb.com> parents: 21604 diff changeset	933 Both data and parameters cannot be modified after the generation has begun.
20856 8a6a86c9a5b5 bundle2: support bundling of empty part (with a type) Pierre-Yves David <pierre-yves.david@fb.com> parents: 20844 diff changeset	934 """
8a6a86c9a5b5 bundle2: support bundling of empty part (with a type) Pierre-Yves David <pierre-yves.david@fb.com> parents: 20844 diff changeset	935
20877 9e9e3a4e9261 bundle2: part params Pierre-Yves David <pierre-yves.david@fb.com> parents: 20876 diff changeset	936 def __init__(self, parttype, mandatoryparams=(), advisoryparams=(),
23590 4440c7cc3728 bundle2.bundlepart: make mandatory part flag explicit in API Eric Sumner <ericsumner@fb.com> parents: 23586 diff changeset	937 data='', mandatory=True):
23868 405eacbecc52 bundle2: enforce parttype as alphanumerical Pierre-Yves David <pierre-yves.david@fb.com> parents: 23590 diff changeset	938 validateparttype(parttype)
20995 e995d104c87f bundle2: add an integer id to part Pierre-Yves David <pierre-yves.david@fb.com> parents: 20952 diff changeset	939 self.id = None
20856 8a6a86c9a5b5 bundle2: support bundling of empty part (with a type) Pierre-Yves David <pierre-yves.david@fb.com> parents: 20844 diff changeset	940 self.type = parttype
21604 c399bf961cb9 bundle2: the ability to set ``data`` attribute of the part is now official Pierre-Yves David <pierre-yves.david@fb.com> parents: 21603 diff changeset	941 self._data = data
21605 f9dabfceb259 bundle2: introduce a ``addparam`` method on part Pierre-Yves David <pierre-yves.david@fb.com> parents: 21604 diff changeset	942 self._mandatoryparams = list(mandatoryparams)
f9dabfceb259 bundle2: introduce a ``addparam`` method on part Pierre-Yves David <pierre-yves.david@fb.com> parents: 21604 diff changeset	943 self._advisoryparams = list(advisoryparams)
21607 054fa5176fa7 bundle2: forbid duplicate parameter keys Pierre-Yves David <pierre-yves.david@fb.com> parents: 21606 diff changeset	944 # checking for duplicated entries
054fa5176fa7 bundle2: forbid duplicate parameter keys Pierre-Yves David <pierre-yves.david@fb.com> parents: 21606 diff changeset	945 self._seenparams = set()
054fa5176fa7 bundle2: forbid duplicate parameter keys Pierre-Yves David <pierre-yves.david@fb.com> parents: 21606 diff changeset	946 for pname, __ in self._mandatoryparams + self._advisoryparams:
054fa5176fa7 bundle2: forbid duplicate parameter keys Pierre-Yves David <pierre-yves.david@fb.com> parents: 21606 diff changeset	947 if pname in self._seenparams:
31647 4dbef666c6c9 bundle2: use ProgrammingError Jun Wu <quark@fb.com> parents: 31476 diff changeset	948 raise error.ProgrammingError('duplicated params: %s' % pname)
21607 054fa5176fa7 bundle2: forbid duplicate parameter keys Pierre-Yves David <pierre-yves.david@fb.com> parents: 21606 diff changeset	949 self._seenparams.add(pname)
21601 7ff01befc7ec bundle2: track life cycle of parts Pierre-Yves David <pierre-yves.david@fb.com> parents: 21600 diff changeset	950 # status of the part's generation:
7ff01befc7ec bundle2: track life cycle of parts Pierre-Yves David <pierre-yves.david@fb.com> parents: 21600 diff changeset	951 # - None: not started,
7ff01befc7ec bundle2: track life cycle of parts Pierre-Yves David <pierre-yves.david@fb.com> parents: 21600 diff changeset	952 # - False: currently generated,
7ff01befc7ec bundle2: track life cycle of parts Pierre-Yves David <pierre-yves.david@fb.com> parents: 21600 diff changeset	953 # - True: generation done.
7ff01befc7ec bundle2: track life cycle of parts Pierre-Yves David <pierre-yves.david@fb.com> parents: 21600 diff changeset	954 self._generated = None
23590 4440c7cc3728 bundle2.bundlepart: make mandatory part flag explicit in API Eric Sumner <ericsumner@fb.com> parents: 23586 diff changeset	955 self.mandatory = mandatory
20856 8a6a86c9a5b5 bundle2: support bundling of empty part (with a type) Pierre-Yves David <pierre-yves.david@fb.com> parents: 20844 diff changeset	956
30872 1f51b4658f21 bundle2: implement a basic __repr__ for bundle2 part Pierre-Yves David <pierre-yves.david@ens-lyon.org> parents: 30871 diff changeset	957 def __repr__(self):
1f51b4658f21 bundle2: implement a basic __repr__ for bundle2 part Pierre-Yves David <pierre-yves.david@ens-lyon.org> parents: 30871 diff changeset	958 cls = "%s.%s" % (self.__class__.__module__, self.__class__.__name__)
1f51b4658f21 bundle2: implement a basic __repr__ for bundle2 part Pierre-Yves David <pierre-yves.david@ens-lyon.org> parents: 30871 diff changeset	959 return ('<%s object at %x; id: %s; type: %s; mandatory: %s>'
1f51b4658f21 bundle2: implement a basic __repr__ for bundle2 part Pierre-Yves David <pierre-yves.david@ens-lyon.org> parents: 30871 diff changeset	960 % (cls, id(self), self.id, self.type, self.mandatory))
1f51b4658f21 bundle2: implement a basic __repr__ for bundle2 part Pierre-Yves David <pierre-yves.david@ens-lyon.org> parents: 30871 diff changeset	961
24793 2ec894582ea2 bundle2: add a 'copy' method on parts Pierre-Yves David <pierre-yves.david@fb.com> parents: 24743 diff changeset	962 def copy(self):
2ec894582ea2 bundle2: add a 'copy' method on parts Pierre-Yves David <pierre-yves.david@fb.com> parents: 24743 diff changeset	963 """return a copy of the part
2ec894582ea2 bundle2: add a 'copy' method on parts Pierre-Yves David <pierre-yves.david@fb.com> parents: 24743 diff changeset	964
2ec894582ea2 bundle2: add a 'copy' method on parts Pierre-Yves David <pierre-yves.david@fb.com> parents: 24743 diff changeset	965 The new part have the very same content but no partid assigned yet.
2ec894582ea2 bundle2: add a 'copy' method on parts Pierre-Yves David <pierre-yves.david@fb.com> parents: 24743 diff changeset	966 Parts with generated data cannot be copied."""
2ec894582ea2 bundle2: add a 'copy' method on parts Pierre-Yves David <pierre-yves.david@fb.com> parents: 24743 diff changeset	967 assert not util.safehasattr(self.data, 'next')
2ec894582ea2 bundle2: add a 'copy' method on parts Pierre-Yves David <pierre-yves.david@fb.com> parents: 24743 diff changeset	968 return self.__class__(self.type, self._mandatoryparams,
2ec894582ea2 bundle2: add a 'copy' method on parts Pierre-Yves David <pierre-yves.david@fb.com> parents: 24743 diff changeset	969 self._advisoryparams, self._data, self.mandatory)
2ec894582ea2 bundle2: add a 'copy' method on parts Pierre-Yves David <pierre-yves.david@fb.com> parents: 24743 diff changeset	970
21604 c399bf961cb9 bundle2: the ability to set ``data`` attribute of the part is now official Pierre-Yves David <pierre-yves.david@fb.com> parents: 21603 diff changeset	971 # methods used to defines the part content
27879 52a4ad62b006 cleanup: use modern @property/@foo.setter property specification Augie Fackler <augie@google.com> parents: 27751 diff changeset	972 @property
52a4ad62b006 cleanup: use modern @property/@foo.setter property specification Augie Fackler <augie@google.com> parents: 27751 diff changeset	973 def data(self):
52a4ad62b006 cleanup: use modern @property/@foo.setter property specification Augie Fackler <augie@google.com> parents: 27751 diff changeset	974 return self._data
52a4ad62b006 cleanup: use modern @property/@foo.setter property specification Augie Fackler <augie@google.com> parents: 27751 diff changeset	975
52a4ad62b006 cleanup: use modern @property/@foo.setter property specification Augie Fackler <augie@google.com> parents: 27751 diff changeset	976 @data.setter
52a4ad62b006 cleanup: use modern @property/@foo.setter property specification Augie Fackler <augie@google.com> parents: 27751 diff changeset	977 def data(self, data):
21604 c399bf961cb9 bundle2: the ability to set ``data`` attribute of the part is now official Pierre-Yves David <pierre-yves.david@fb.com> parents: 21603 diff changeset	978 if self._generated is not None:
21618 7568f5c1c801 bundle2: move exception classes into the error module Pierre-Yves David <pierre-yves.david@fb.com> parents: 21617 diff changeset	979 raise error.ReadOnlyPartError('part is being generated')
21604 c399bf961cb9 bundle2: the ability to set ``data`` attribute of the part is now official Pierre-Yves David <pierre-yves.david@fb.com> parents: 21603 diff changeset	980 self._data = data
c399bf961cb9 bundle2: the ability to set ``data`` attribute of the part is now official Pierre-Yves David <pierre-yves.david@fb.com> parents: 21603 diff changeset	981
21605 f9dabfceb259 bundle2: introduce a ``addparam`` method on part Pierre-Yves David <pierre-yves.david@fb.com> parents: 21604 diff changeset	982 @property
f9dabfceb259 bundle2: introduce a ``addparam`` method on part Pierre-Yves David <pierre-yves.david@fb.com> parents: 21604 diff changeset	983 def mandatoryparams(self):
f9dabfceb259 bundle2: introduce a ``addparam`` method on part Pierre-Yves David <pierre-yves.david@fb.com> parents: 21604 diff changeset	984 # make it an immutable tuple to force people through ``addparam``
f9dabfceb259 bundle2: introduce a ``addparam`` method on part Pierre-Yves David <pierre-yves.david@fb.com> parents: 21604 diff changeset	985 return tuple(self._mandatoryparams)
f9dabfceb259 bundle2: introduce a ``addparam`` method on part Pierre-Yves David <pierre-yves.david@fb.com> parents: 21604 diff changeset	986
f9dabfceb259 bundle2: introduce a ``addparam`` method on part Pierre-Yves David <pierre-yves.david@fb.com> parents: 21604 diff changeset	987 @property
f9dabfceb259 bundle2: introduce a ``addparam`` method on part Pierre-Yves David <pierre-yves.david@fb.com> parents: 21604 diff changeset	988 def advisoryparams(self):
f9dabfceb259 bundle2: introduce a ``addparam`` method on part Pierre-Yves David <pierre-yves.david@fb.com> parents: 21604 diff changeset	989 # make it an immutable tuple to force people through ``addparam``
f9dabfceb259 bundle2: introduce a ``addparam`` method on part Pierre-Yves David <pierre-yves.david@fb.com> parents: 21604 diff changeset	990 return tuple(self._advisoryparams)
f9dabfceb259 bundle2: introduce a ``addparam`` method on part Pierre-Yves David <pierre-yves.david@fb.com> parents: 21604 diff changeset	991
f9dabfceb259 bundle2: introduce a ``addparam`` method on part Pierre-Yves David <pierre-yves.david@fb.com> parents: 21604 diff changeset	992 def addparam(self, name, value='', mandatory=True):
31861 6d055cd6669f bundle2: add documention to 'part.addparams' Pierre-Yves David <pierre-yves.david@ens-lyon.org> parents: 31647 diff changeset	993 """add a parameter to the part
6d055cd6669f bundle2: add documention to 'part.addparams' Pierre-Yves David <pierre-yves.david@ens-lyon.org> parents: 31647 diff changeset	994
6d055cd6669f bundle2: add documention to 'part.addparams' Pierre-Yves David <pierre-yves.david@ens-lyon.org> parents: 31647 diff changeset	995 If 'mandatory' is set to True, the remote handler must claim support
6d055cd6669f bundle2: add documention to 'part.addparams' Pierre-Yves David <pierre-yves.david@ens-lyon.org> parents: 31647 diff changeset	996 for this parameter or the unbundling will be aborted.
6d055cd6669f bundle2: add documention to 'part.addparams' Pierre-Yves David <pierre-yves.david@ens-lyon.org> parents: 31647 diff changeset	997
6d055cd6669f bundle2: add documention to 'part.addparams' Pierre-Yves David <pierre-yves.david@ens-lyon.org> parents: 31647 diff changeset	998 The 'name' and 'value' cannot exceed 255 bytes each.
6d055cd6669f bundle2: add documention to 'part.addparams' Pierre-Yves David <pierre-yves.david@ens-lyon.org> parents: 31647 diff changeset	999 """
21605 f9dabfceb259 bundle2: introduce a ``addparam`` method on part Pierre-Yves David <pierre-yves.david@fb.com> parents: 21604 diff changeset	1000 if self._generated is not None:
21618 7568f5c1c801 bundle2: move exception classes into the error module Pierre-Yves David <pierre-yves.david@fb.com> parents: 21617 diff changeset	1001 raise error.ReadOnlyPartError('part is being generated')
21607 054fa5176fa7 bundle2: forbid duplicate parameter keys Pierre-Yves David <pierre-yves.david@fb.com> parents: 21606 diff changeset	1002 if name in self._seenparams:
054fa5176fa7 bundle2: forbid duplicate parameter keys Pierre-Yves David <pierre-yves.david@fb.com> parents: 21606 diff changeset	1003 raise ValueError('duplicated params: %s' % name)
054fa5176fa7 bundle2: forbid duplicate parameter keys Pierre-Yves David <pierre-yves.david@fb.com> parents: 21606 diff changeset	1004 self._seenparams.add(name)
21605 f9dabfceb259 bundle2: introduce a ``addparam`` method on part Pierre-Yves David <pierre-yves.david@fb.com> parents: 21604 diff changeset	1005 params = self._advisoryparams
f9dabfceb259 bundle2: introduce a ``addparam`` method on part Pierre-Yves David <pierre-yves.david@fb.com> parents: 21604 diff changeset	1006 if mandatory:
f9dabfceb259 bundle2: introduce a ``addparam`` method on part Pierre-Yves David <pierre-yves.david@fb.com> parents: 21604 diff changeset	1007 params = self._mandatoryparams
f9dabfceb259 bundle2: introduce a ``addparam`` method on part Pierre-Yves David <pierre-yves.david@fb.com> parents: 21604 diff changeset	1008 params.append((name, value))
f9dabfceb259 bundle2: introduce a ``addparam`` method on part Pierre-Yves David <pierre-yves.david@fb.com> parents: 21604 diff changeset	1009
21601 7ff01befc7ec bundle2: track life cycle of parts Pierre-Yves David <pierre-yves.david@fb.com> parents: 21600 diff changeset	1010 # methods used to generates the bundle2 stream
25321 b44ee346211f bundle2: add debug output for part generation Pierre-Yves David <pierre-yves.david@fb.com> parents: 25320 diff changeset	1011 def getchunks(self, ui):
21601 7ff01befc7ec bundle2: track life cycle of parts Pierre-Yves David <pierre-yves.david@fb.com> parents: 21600 diff changeset	1012 if self._generated is not None:
31647 4dbef666c6c9 bundle2: use ProgrammingError Jun Wu <quark@fb.com> parents: 31476 diff changeset	1013 raise error.ProgrammingError('part can only be consumed once')
21601 7ff01befc7ec bundle2: track life cycle of parts Pierre-Yves David <pierre-yves.david@fb.com> parents: 21600 diff changeset	1014 self._generated = False
25323 21a25fb81d2c bundle2: add generic debug output regarding generated parts Pierre-Yves David <pierre-yves.david@fb.com> parents: 25322 diff changeset	1015
21a25fb81d2c bundle2: add generic debug output regarding generated parts Pierre-Yves David <pierre-yves.david@fb.com> parents: 25322 diff changeset	1016 if ui.debugflag:
21a25fb81d2c bundle2: add generic debug output regarding generated parts Pierre-Yves David <pierre-yves.david@fb.com> parents: 25322 diff changeset	1017 msg = ['bundle2-output-part: "%s"' % self.type]
21a25fb81d2c bundle2: add generic debug output regarding generated parts Pierre-Yves David <pierre-yves.david@fb.com> parents: 25322 diff changeset	1018 if not self.mandatory:
21a25fb81d2c bundle2: add generic debug output regarding generated parts Pierre-Yves David <pierre-yves.david@fb.com> parents: 25322 diff changeset	1019 msg.append(' (advisory)')
21a25fb81d2c bundle2: add generic debug output regarding generated parts Pierre-Yves David <pierre-yves.david@fb.com> parents: 25322 diff changeset	1020 nbmp = len(self.mandatoryparams)
21a25fb81d2c bundle2: add generic debug output regarding generated parts Pierre-Yves David <pierre-yves.david@fb.com> parents: 25322 diff changeset	1021 nbap = len(self.advisoryparams)
21a25fb81d2c bundle2: add generic debug output regarding generated parts Pierre-Yves David <pierre-yves.david@fb.com> parents: 25322 diff changeset	1022 if nbmp or nbap:
21a25fb81d2c bundle2: add generic debug output regarding generated parts Pierre-Yves David <pierre-yves.david@fb.com> parents: 25322 diff changeset	1023 msg.append(' (params:')
21a25fb81d2c bundle2: add generic debug output regarding generated parts Pierre-Yves David <pierre-yves.david@fb.com> parents: 25322 diff changeset	1024 if nbmp:
21a25fb81d2c bundle2: add generic debug output regarding generated parts Pierre-Yves David <pierre-yves.david@fb.com> parents: 25322 diff changeset	1025 msg.append(' %i mandatory' % nbmp)
21a25fb81d2c bundle2: add generic debug output regarding generated parts Pierre-Yves David <pierre-yves.david@fb.com> parents: 25322 diff changeset	1026 if nbap:
21a25fb81d2c bundle2: add generic debug output regarding generated parts Pierre-Yves David <pierre-yves.david@fb.com> parents: 25322 diff changeset	1027 msg.append(' %i advisory' % nbmp)
21a25fb81d2c bundle2: add generic debug output regarding generated parts Pierre-Yves David <pierre-yves.david@fb.com> parents: 25322 diff changeset	1028 msg.append(')')
21a25fb81d2c bundle2: add generic debug output regarding generated parts Pierre-Yves David <pierre-yves.david@fb.com> parents: 25322 diff changeset	1029 if not self.data:
21a25fb81d2c bundle2: add generic debug output regarding generated parts Pierre-Yves David <pierre-yves.david@fb.com> parents: 25322 diff changeset	1030 msg.append(' empty payload')
34278 ea7588c11c36 bundle2: update check for a generator to work on Python 3 Augie Fackler <augie@google.com> parents: 34277 diff changeset	1031 elif (util.safehasattr(self.data, 'next')
ea7588c11c36 bundle2: update check for a generator to work on Python 3 Augie Fackler <augie@google.com> parents: 34277 diff changeset	1032 or util.safehasattr(self.data, '__next__')):
25323 21a25fb81d2c bundle2: add generic debug output regarding generated parts Pierre-Yves David <pierre-yves.david@fb.com> parents: 25322 diff changeset	1033 msg.append(' streamed payload')
21a25fb81d2c bundle2: add generic debug output regarding generated parts Pierre-Yves David <pierre-yves.david@fb.com> parents: 25322 diff changeset	1034 else:
21a25fb81d2c bundle2: add generic debug output regarding generated parts Pierre-Yves David <pierre-yves.david@fb.com> parents: 25322 diff changeset	1035 msg.append(' %i bytes payload' % len(self.data))
21a25fb81d2c bundle2: add generic debug output regarding generated parts Pierre-Yves David <pierre-yves.david@fb.com> parents: 25322 diff changeset	1036 msg.append('\n')
21a25fb81d2c bundle2: add generic debug output regarding generated parts Pierre-Yves David <pierre-yves.david@fb.com> parents: 25322 diff changeset	1037 ui.debug(''.join(msg))
21a25fb81d2c bundle2: add generic debug output regarding generated parts Pierre-Yves David <pierre-yves.david@fb.com> parents: 25322 diff changeset	1038
20877 9e9e3a4e9261 bundle2: part params Pierre-Yves David <pierre-yves.david@fb.com> parents: 20876 diff changeset	1039 #### header
23590 4440c7cc3728 bundle2.bundlepart: make mandatory part flag explicit in API Eric Sumner <ericsumner@fb.com> parents: 23586 diff changeset	1040 if self.mandatory:
4440c7cc3728 bundle2.bundlepart: make mandatory part flag explicit in API Eric Sumner <ericsumner@fb.com> parents: 23586 diff changeset	1041 parttype = self.type.upper()
4440c7cc3728 bundle2.bundlepart: make mandatory part flag explicit in API Eric Sumner <ericsumner@fb.com> parents: 23586 diff changeset	1042 else:
4440c7cc3728 bundle2.bundlepart: make mandatory part flag explicit in API Eric Sumner <ericsumner@fb.com> parents: 23586 diff changeset	1043 parttype = self.type.lower()
33711 373ca5103ba8 bundle2: convert ints to strings using pycompat.bytestring() Augie Fackler <augie@google.com> parents: 33693 diff changeset	1044 outdebug(ui, 'part %s: "%s"' % (pycompat.bytestr(self.id), parttype))
20877 9e9e3a4e9261 bundle2: part params Pierre-Yves David <pierre-yves.david@fb.com> parents: 20876 diff changeset	1045 ## parttype
23590 4440c7cc3728 bundle2.bundlepart: make mandatory part flag explicit in API Eric Sumner <ericsumner@fb.com> parents: 23586 diff changeset	1046 header = [_pack(_fparttypesize, len(parttype)),
4440c7cc3728 bundle2.bundlepart: make mandatory part flag explicit in API Eric Sumner <ericsumner@fb.com> parents: 23586 diff changeset	1047 parttype, _pack(_fpartid, self.id),
20856 8a6a86c9a5b5 bundle2: support bundling of empty part (with a type) Pierre-Yves David <pierre-yves.david@fb.com> parents: 20844 diff changeset	1048 ]
20877 9e9e3a4e9261 bundle2: part params Pierre-Yves David <pierre-yves.david@fb.com> parents: 20876 diff changeset	1049 ## parameters
9e9e3a4e9261 bundle2: part params Pierre-Yves David <pierre-yves.david@fb.com> parents: 20876 diff changeset	1050 # count
9e9e3a4e9261 bundle2: part params Pierre-Yves David <pierre-yves.david@fb.com> parents: 20876 diff changeset	1051 manpar = self.mandatoryparams
9e9e3a4e9261 bundle2: part params Pierre-Yves David <pierre-yves.david@fb.com> parents: 20876 diff changeset	1052 advpar = self.advisoryparams
9e9e3a4e9261 bundle2: part params Pierre-Yves David <pierre-yves.david@fb.com> parents: 20876 diff changeset	1053 header.append(_pack(_fpartparamcount, len(manpar), len(advpar)))
9e9e3a4e9261 bundle2: part params Pierre-Yves David <pierre-yves.david@fb.com> parents: 20876 diff changeset	1054 # size
9e9e3a4e9261 bundle2: part params Pierre-Yves David <pierre-yves.david@fb.com> parents: 20876 diff changeset	1055 parsizes = []
9e9e3a4e9261 bundle2: part params Pierre-Yves David <pierre-yves.david@fb.com> parents: 20876 diff changeset	1056 for key, value in manpar:
9e9e3a4e9261 bundle2: part params Pierre-Yves David <pierre-yves.david@fb.com> parents: 20876 diff changeset	1057 parsizes.append(len(key))
9e9e3a4e9261 bundle2: part params Pierre-Yves David <pierre-yves.david@fb.com> parents: 20876 diff changeset	1058 parsizes.append(len(value))
9e9e3a4e9261 bundle2: part params Pierre-Yves David <pierre-yves.david@fb.com> parents: 20876 diff changeset	1059 for key, value in advpar:
9e9e3a4e9261 bundle2: part params Pierre-Yves David <pierre-yves.david@fb.com> parents: 20876 diff changeset	1060 parsizes.append(len(key))
9e9e3a4e9261 bundle2: part params Pierre-Yves David <pierre-yves.david@fb.com> parents: 20876 diff changeset	1061 parsizes.append(len(value))
33672 da7c285ec6da bundle2: use modern Python division Augie Fackler <augie@google.com> parents: 33665 diff changeset	1062 paramsizes = _pack(_makefpartparamsizes(len(parsizes) // 2), *parsizes)
20877 9e9e3a4e9261 bundle2: part params Pierre-Yves David <pierre-yves.david@fb.com> parents: 20876 diff changeset	1063 header.append(paramsizes)
9e9e3a4e9261 bundle2: part params Pierre-Yves David <pierre-yves.david@fb.com> parents: 20876 diff changeset	1064 # key, value
9e9e3a4e9261 bundle2: part params Pierre-Yves David <pierre-yves.david@fb.com> parents: 20876 diff changeset	1065 for key, value in manpar:
9e9e3a4e9261 bundle2: part params Pierre-Yves David <pierre-yves.david@fb.com> parents: 20876 diff changeset	1066 header.append(key)
9e9e3a4e9261 bundle2: part params Pierre-Yves David <pierre-yves.david@fb.com> parents: 20876 diff changeset	1067 header.append(value)
9e9e3a4e9261 bundle2: part params Pierre-Yves David <pierre-yves.david@fb.com> parents: 20876 diff changeset	1068 for key, value in advpar:
9e9e3a4e9261 bundle2: part params Pierre-Yves David <pierre-yves.david@fb.com> parents: 20876 diff changeset	1069 header.append(key)
9e9e3a4e9261 bundle2: part params Pierre-Yves David <pierre-yves.david@fb.com> parents: 20876 diff changeset	1070 header.append(value)
9e9e3a4e9261 bundle2: part params Pierre-Yves David <pierre-yves.david@fb.com> parents: 20876 diff changeset	1071 ## finalize header
34249 ab379eed2e31 bundle2: raise a more helpful error if building a bundle part header fails Augie Fackler <raf@durin42.com> parents: 34221 diff changeset	1072 try:
ab379eed2e31 bundle2: raise a more helpful error if building a bundle part header fails Augie Fackler <raf@durin42.com> parents: 34221 diff changeset	1073 headerchunk = ''.join(header)
ab379eed2e31 bundle2: raise a more helpful error if building a bundle part header fails Augie Fackler <raf@durin42.com> parents: 34221 diff changeset	1074 except TypeError:
ab379eed2e31 bundle2: raise a more helpful error if building a bundle part header fails Augie Fackler <raf@durin42.com> parents: 34221 diff changeset	1075 raise TypeError(r'Found a non-bytes trying to '
ab379eed2e31 bundle2: raise a more helpful error if building a bundle part header fails Augie Fackler <raf@durin42.com> parents: 34221 diff changeset	1076 r'build bundle part header: %r' % header)
25321 b44ee346211f bundle2: add debug output for part generation Pierre-Yves David <pierre-yves.david@fb.com> parents: 25320 diff changeset	1077 outdebug(ui, 'header chunk size: %i' % len(headerchunk))
20856 8a6a86c9a5b5 bundle2: support bundling of empty part (with a type) Pierre-Yves David <pierre-yves.david@fb.com> parents: 20844 diff changeset	1078 yield _pack(_fpartheadersize, len(headerchunk))
8a6a86c9a5b5 bundle2: support bundling of empty part (with a type) Pierre-Yves David <pierre-yves.david@fb.com> parents: 20844 diff changeset	1079 yield headerchunk
20877 9e9e3a4e9261 bundle2: part params Pierre-Yves David <pierre-yves.david@fb.com> parents: 20876 diff changeset	1080 ## payload
23067 420a051616ce bundle2: transmit exception during part generation Pierre-Yves David <pierre-yves.david@fb.com> parents: 23066 diff changeset	1081 try:
420a051616ce bundle2: transmit exception during part generation Pierre-Yves David <pierre-yves.david@fb.com> parents: 23066 diff changeset	1082 for chunk in self._payloadchunks():
25321 b44ee346211f bundle2: add debug output for part generation Pierre-Yves David <pierre-yves.david@fb.com> parents: 25320 diff changeset	1083 outdebug(ui, 'payload chunk size: %i' % len(chunk))
23067 420a051616ce bundle2: transmit exception during part generation Pierre-Yves David <pierre-yves.david@fb.com> parents: 23066 diff changeset	1084 yield _pack(_fpayloadsize, len(chunk))
420a051616ce bundle2: transmit exception during part generation Pierre-Yves David <pierre-yves.david@fb.com> parents: 23066 diff changeset	1085 yield chunk
26144 4bc3707f3e67 bundle2: don't try to recover from a GeneratorExit (issue4785) Augie Fackler <augie@google.com> parents: 25919 diff changeset	1086 except GeneratorExit:
4bc3707f3e67 bundle2: don't try to recover from a GeneratorExit (issue4785) Augie Fackler <augie@google.com> parents: 25919 diff changeset	1087 # GeneratorExit means that nobody is listening for our
4bc3707f3e67 bundle2: don't try to recover from a GeneratorExit (issue4785) Augie Fackler <augie@google.com> parents: 25919 diff changeset	1088 # results anyway, so just bail quickly rather than trying
4bc3707f3e67 bundle2: don't try to recover from a GeneratorExit (issue4785) Augie Fackler <augie@google.com> parents: 25919 diff changeset	1089 # to produce an error part.
4bc3707f3e67 bundle2: don't try to recover from a GeneratorExit (issue4785) Augie Fackler <augie@google.com> parents: 25919 diff changeset	1090 ui.debug('bundle2-generatorexit\n')
4bc3707f3e67 bundle2: don't try to recover from a GeneratorExit (issue4785) Augie Fackler <augie@google.com> parents: 25919 diff changeset	1091 raise
25660 328739ea70c3 global: mass rewrite to use modern exception syntax Gregory Szorc <gregory.szorc@gmail.com> parents: 25641 diff changeset	1092 except BaseException as exc:
33682 1d5e497c08b3 py3: convert arbitrary exception object to byte string more reliably Yuya Nishihara <yuya@tcha.org> parents: 33676 diff changeset	1093 bexc = util.forcebytestr(exc)
23067 420a051616ce bundle2: transmit exception during part generation Pierre-Yves David <pierre-yves.david@fb.com> parents: 23066 diff changeset	1094 # backup exception data for later
25324 deed9c6b12d2 bundle2: add generic debug output regarding generated interruption Pierre-Yves David <pierre-yves.david@fb.com> parents: 25323 diff changeset	1095 ui.debug('bundle2-input-stream-interrupt: encoding exception %s'
33676 9f862fe7a8bd bundle2: obtain repr() of exception in a python3-safe way Augie Fackler <augie@google.com> parents: 33675 diff changeset	1096 % bexc)
32186 76f9a0009b4b pycompat: extract helper to raise exception with traceback Yuya Nishihara <yuya@tcha.org> parents: 32024 diff changeset	1097 tb = sys.exc_info()[2]
33676 9f862fe7a8bd bundle2: obtain repr() of exception in a python3-safe way Augie Fackler <augie@google.com> parents: 33675 diff changeset	1098 msg = 'unexpected error: %s' % bexc
24686 e0e28e910fa3 bundle2: rename format, parts and config to final names Pierre-Yves David <pierre-yves.david@fb.com> parents: 24648 diff changeset	1099 interpart = bundlepart('error:abort', [('message', msg)],
23590 4440c7cc3728 bundle2.bundlepart: make mandatory part flag explicit in API Eric Sumner <ericsumner@fb.com> parents: 23586 diff changeset	1100 mandatory=False)
23067 420a051616ce bundle2: transmit exception during part generation Pierre-Yves David <pierre-yves.david@fb.com> parents: 23066 diff changeset	1101 interpart.id = 0
420a051616ce bundle2: transmit exception during part generation Pierre-Yves David <pierre-yves.david@fb.com> parents: 23066 diff changeset	1102 yield _pack(_fpayloadsize, -1)
25321 b44ee346211f bundle2: add debug output for part generation Pierre-Yves David <pierre-yves.david@fb.com> parents: 25320 diff changeset	1103 for chunk in interpart.getchunks(ui=ui):
23067 420a051616ce bundle2: transmit exception during part generation Pierre-Yves David <pierre-yves.david@fb.com> parents: 23066 diff changeset	1104 yield chunk
25321 b44ee346211f bundle2: add debug output for part generation Pierre-Yves David <pierre-yves.david@fb.com> parents: 25320 diff changeset	1105 outdebug(ui, 'closing payload chunk')
23067 420a051616ce bundle2: transmit exception during part generation Pierre-Yves David <pierre-yves.david@fb.com> parents: 23066 diff changeset	1106 # abort current part payload
420a051616ce bundle2: transmit exception during part generation Pierre-Yves David <pierre-yves.david@fb.com> parents: 23066 diff changeset	1107 yield _pack(_fpayloadsize, 0)
32186 76f9a0009b4b pycompat: extract helper to raise exception with traceback Yuya Nishihara <yuya@tcha.org> parents: 32024 diff changeset	1108 pycompat.raisewithtb(exc, tb)
21000 4cae06ae1562 bundle2: extract a _payloadchunks method for part Pierre-Yves David <pierre-yves.david@fb.com> parents: 20998 diff changeset	1109 # end of payload
25321 b44ee346211f bundle2: add debug output for part generation Pierre-Yves David <pierre-yves.david@fb.com> parents: 25320 diff changeset	1110 outdebug(ui, 'closing payload chunk')
21000 4cae06ae1562 bundle2: extract a _payloadchunks method for part Pierre-Yves David <pierre-yves.david@fb.com> parents: 20998 diff changeset	1111 yield _pack(_fpayloadsize, 0)
21601 7ff01befc7ec bundle2: track life cycle of parts Pierre-Yves David <pierre-yves.david@fb.com> parents: 21600 diff changeset	1112 self._generated = True
21000 4cae06ae1562 bundle2: extract a _payloadchunks method for part Pierre-Yves David <pierre-yves.david@fb.com> parents: 20998 diff changeset	1113
4cae06ae1562 bundle2: extract a _payloadchunks method for part Pierre-Yves David <pierre-yves.david@fb.com> parents: 20998 diff changeset	1114 def _payloadchunks(self):
4cae06ae1562 bundle2: extract a _payloadchunks method for part Pierre-Yves David <pierre-yves.david@fb.com> parents: 20998 diff changeset	1115 """yield chunks of a the part payload
4cae06ae1562 bundle2: extract a _payloadchunks method for part Pierre-Yves David <pierre-yves.david@fb.com> parents: 20998 diff changeset	1116
4cae06ae1562 bundle2: extract a _payloadchunks method for part Pierre-Yves David <pierre-yves.david@fb.com> parents: 20998 diff changeset	1117 Exists to handle the different methods to provide data to a part."""
20876 ddd56f3eb786 bundle2: support for bundling and unbundling payload Pierre-Yves David <pierre-yves.david@fb.com> parents: 20864 diff changeset	1118 # we only support fixed size data now.
ddd56f3eb786 bundle2: support for bundling and unbundling payload Pierre-Yves David <pierre-yves.david@fb.com> parents: 20864 diff changeset	1119 # This will be improved in the future.
33673 5ae35a1347fd bundle2: look for __next__ as well as next to identify iterators Augie Fackler <augie@google.com> parents: 33672 diff changeset	1120 if (util.safehasattr(self.data, 'next')
5ae35a1347fd bundle2: look for __next__ as well as next to identify iterators Augie Fackler <augie@google.com> parents: 33672 diff changeset	1121 or util.safehasattr(self.data, '__next__')):
21001 c93bb6a08fa1 bundle2: support chunk iterator as part data Pierre-Yves David <pierre-yves.david@fb.com> parents: 21000 diff changeset	1122 buff = util.chunkbuffer(self.data)
c93bb6a08fa1 bundle2: support chunk iterator as part data Pierre-Yves David <pierre-yves.david@fb.com> parents: 21000 diff changeset	1123 chunk = buff.read(preferedchunksize)
c93bb6a08fa1 bundle2: support chunk iterator as part data Pierre-Yves David <pierre-yves.david@fb.com> parents: 21000 diff changeset	1124 while chunk:
c93bb6a08fa1 bundle2: support chunk iterator as part data Pierre-Yves David <pierre-yves.david@fb.com> parents: 21000 diff changeset	1125 yield chunk
c93bb6a08fa1 bundle2: support chunk iterator as part data Pierre-Yves David <pierre-yves.david@fb.com> parents: 21000 diff changeset	1126 chunk = buff.read(preferedchunksize)
c93bb6a08fa1 bundle2: support chunk iterator as part data Pierre-Yves David <pierre-yves.david@fb.com> parents: 21000 diff changeset	1127 elif len(self.data):
20876 ddd56f3eb786 bundle2: support for bundling and unbundling payload Pierre-Yves David <pierre-yves.david@fb.com> parents: 20864 diff changeset	1128 yield self.data
20802 520df53ad26a bundle2: a very first version of bundle2 unbundler Pierre-Yves David <pierre-yves.david@fb.com> parents: 20801 diff changeset	1129
23066 ad144882318d bundle2: add a interrupt mechanism Pierre-Yves David <pierre-yves.david@fb.com> parents: 23029 diff changeset	1130
ad144882318d bundle2: add a interrupt mechanism Pierre-Yves David <pierre-yves.david@fb.com> parents: 23029 diff changeset	1131 flaginterrupt = -1
ad144882318d bundle2: add a interrupt mechanism Pierre-Yves David <pierre-yves.david@fb.com> parents: 23029 diff changeset	1132
ad144882318d bundle2: add a interrupt mechanism Pierre-Yves David <pierre-yves.david@fb.com> parents: 23029 diff changeset	1133 class interrupthandler(unpackermixin):
ad144882318d bundle2: add a interrupt mechanism Pierre-Yves David <pierre-yves.david@fb.com> parents: 23029 diff changeset	1134 """read one part and process it with restricted capability
ad144882318d bundle2: add a interrupt mechanism Pierre-Yves David <pierre-yves.david@fb.com> parents: 23029 diff changeset	1135
ad144882318d bundle2: add a interrupt mechanism Pierre-Yves David <pierre-yves.david@fb.com> parents: 23029 diff changeset	1136 This allows to transmit exception raised on the producer size during part
ad144882318d bundle2: add a interrupt mechanism Pierre-Yves David <pierre-yves.david@fb.com> parents: 23029 diff changeset	1137 iteration while the consumer is reading a part.
ad144882318d bundle2: add a interrupt mechanism Pierre-Yves David <pierre-yves.david@fb.com> parents: 23029 diff changeset	1138
ad144882318d bundle2: add a interrupt mechanism Pierre-Yves David <pierre-yves.david@fb.com> parents: 23029 diff changeset	1139 Part processed in this manner only have access to a ui object,"""
ad144882318d bundle2: add a interrupt mechanism Pierre-Yves David <pierre-yves.david@fb.com> parents: 23029 diff changeset	1140
ad144882318d bundle2: add a interrupt mechanism Pierre-Yves David <pierre-yves.david@fb.com> parents: 23029 diff changeset	1141 def __init__(self, ui, fp):
ad144882318d bundle2: add a interrupt mechanism Pierre-Yves David <pierre-yves.david@fb.com> parents: 23029 diff changeset	1142 super(interrupthandler, self).__init__(fp)
ad144882318d bundle2: add a interrupt mechanism Pierre-Yves David <pierre-yves.david@fb.com> parents: 23029 diff changeset	1143 self.ui = ui
ad144882318d bundle2: add a interrupt mechanism Pierre-Yves David <pierre-yves.david@fb.com> parents: 23029 diff changeset	1144
ad144882318d bundle2: add a interrupt mechanism Pierre-Yves David <pierre-yves.david@fb.com> parents: 23029 diff changeset	1145 def _readpartheader(self):
ad144882318d bundle2: add a interrupt mechanism Pierre-Yves David <pierre-yves.david@fb.com> parents: 23029 diff changeset	1146 """reads a part header size and return the bytes blob
ad144882318d bundle2: add a interrupt mechanism Pierre-Yves David <pierre-yves.david@fb.com> parents: 23029 diff changeset	1147
ad144882318d bundle2: add a interrupt mechanism Pierre-Yves David <pierre-yves.david@fb.com> parents: 23029 diff changeset	1148 returns None if empty"""
ad144882318d bundle2: add a interrupt mechanism Pierre-Yves David <pierre-yves.david@fb.com> parents: 23029 diff changeset	1149 headersize = self._unpack(_fpartheadersize)[0]
ad144882318d bundle2: add a interrupt mechanism Pierre-Yves David <pierre-yves.david@fb.com> parents: 23029 diff changeset	1150 if headersize < 0:
ad144882318d bundle2: add a interrupt mechanism Pierre-Yves David <pierre-yves.david@fb.com> parents: 23029 diff changeset	1151 raise error.BundleValueError('negative part header size: %i'
ad144882318d bundle2: add a interrupt mechanism Pierre-Yves David <pierre-yves.david@fb.com> parents: 23029 diff changeset	1152 % headersize)
25318 64dd8f25fcf4 bundle2: introduce a specific function for debug messages while unbundling Pierre-Yves David <pierre-yves.david@fb.com> parents: 25317 diff changeset	1153 indebug(self.ui, 'part header size: %i\n' % headersize)
23066 ad144882318d bundle2: add a interrupt mechanism Pierre-Yves David <pierre-yves.david@fb.com> parents: 23029 diff changeset	1154 if headersize:
ad144882318d bundle2: add a interrupt mechanism Pierre-Yves David <pierre-yves.david@fb.com> parents: 23029 diff changeset	1155 return self._readexact(headersize)
ad144882318d bundle2: add a interrupt mechanism Pierre-Yves David <pierre-yves.david@fb.com> parents: 23029 diff changeset	1156 return None
ad144882318d bundle2: add a interrupt mechanism Pierre-Yves David <pierre-yves.david@fb.com> parents: 23029 diff changeset	1157
ad144882318d bundle2: add a interrupt mechanism Pierre-Yves David <pierre-yves.david@fb.com> parents: 23029 diff changeset	1158 def __call__(self):
25335 8f7137a85a0e bundle2: add generic debug output regarding processed interruption Pierre-Yves David <pierre-yves.david@fb.com> parents: 25334 diff changeset	1159
8f7137a85a0e bundle2: add generic debug output regarding processed interruption Pierre-Yves David <pierre-yves.david@fb.com> parents: 25334 diff changeset	1160 self.ui.debug('bundle2-input-stream-interrupt:'
8f7137a85a0e bundle2: add generic debug output regarding processed interruption Pierre-Yves David <pierre-yves.david@fb.com> parents: 25334 diff changeset	1161 ' opening out of band context\n')
25320 697d8953b04d bundle2: handle new line in 'indebug' function Pierre-Yves David <pierre-yves.david@fb.com> parents: 25319 diff changeset	1162 indebug(self.ui, 'bundle2 stream interruption, looking for a part.')
23066 ad144882318d bundle2: add a interrupt mechanism Pierre-Yves David <pierre-yves.david@fb.com> parents: 23029 diff changeset	1163 headerblock = self._readpartheader()
ad144882318d bundle2: add a interrupt mechanism Pierre-Yves David <pierre-yves.david@fb.com> parents: 23029 diff changeset	1164 if headerblock is None:
25320 697d8953b04d bundle2: handle new line in 'indebug' function Pierre-Yves David <pierre-yves.david@fb.com> parents: 25319 diff changeset	1165 indebug(self.ui, 'no part found during interruption.')
23066 ad144882318d bundle2: add a interrupt mechanism Pierre-Yves David <pierre-yves.david@fb.com> parents: 23029 diff changeset	1166 return
35116 da91e7309daf bundle2: don't use seekable bundle2 parts by default (issue5691) Gregory Szorc <gregory.szorc@gmail.com> parents: 35115 diff changeset	1167 part = unbundlepart(self.ui, headerblock, self._fp)
23066 ad144882318d bundle2: add a interrupt mechanism Pierre-Yves David <pierre-yves.david@fb.com> parents: 23029 diff changeset	1168 op = interruptoperation(self.ui)
34258 e71890f27767 bundle2: move processpart stream maintenance into part iterator Durham Goode <durham@fb.com> parents: 34249 diff changeset	1169 hardabort = False
e71890f27767 bundle2: move processpart stream maintenance into part iterator Durham Goode <durham@fb.com> parents: 34249 diff changeset	1170 try:
e71890f27767 bundle2: move processpart stream maintenance into part iterator Durham Goode <durham@fb.com> parents: 34249 diff changeset	1171 _processpart(op, part)
e71890f27767 bundle2: move processpart stream maintenance into part iterator Durham Goode <durham@fb.com> parents: 34249 diff changeset	1172 except (SystemExit, KeyboardInterrupt):
e71890f27767 bundle2: move processpart stream maintenance into part iterator Durham Goode <durham@fb.com> parents: 34249 diff changeset	1173 hardabort = True
e71890f27767 bundle2: move processpart stream maintenance into part iterator Durham Goode <durham@fb.com> parents: 34249 diff changeset	1174 raise
e71890f27767 bundle2: move processpart stream maintenance into part iterator Durham Goode <durham@fb.com> parents: 34249 diff changeset	1175 finally:
e71890f27767 bundle2: move processpart stream maintenance into part iterator Durham Goode <durham@fb.com> parents: 34249 diff changeset	1176 if not hardabort:
35114 db5038525718 bundle2: implement consume() API on unbundlepart Gregory Szorc <gregory.szorc@gmail.com> parents: 35113 diff changeset	1177 part.consume()
25335 8f7137a85a0e bundle2: add generic debug output regarding processed interruption Pierre-Yves David <pierre-yves.david@fb.com> parents: 25334 diff changeset	1178 self.ui.debug('bundle2-input-stream-interrupt:'
8f7137a85a0e bundle2: add generic debug output regarding processed interruption Pierre-Yves David <pierre-yves.david@fb.com> parents: 25334 diff changeset	1179 ' closing out of band context\n')
23066 ad144882318d bundle2: add a interrupt mechanism Pierre-Yves David <pierre-yves.david@fb.com> parents: 23029 diff changeset	1180
ad144882318d bundle2: add a interrupt mechanism Pierre-Yves David <pierre-yves.david@fb.com> parents: 23029 diff changeset	1181 class interruptoperation(object):
ad144882318d bundle2: add a interrupt mechanism Pierre-Yves David <pierre-yves.david@fb.com> parents: 23029 diff changeset	1182 """A limited operation to be use by part handler during interruption
ad144882318d bundle2: add a interrupt mechanism Pierre-Yves David <pierre-yves.david@fb.com> parents: 23029 diff changeset	1183
ad144882318d bundle2: add a interrupt mechanism Pierre-Yves David <pierre-yves.david@fb.com> parents: 23029 diff changeset	1184 It only have access to an ui object.
ad144882318d bundle2: add a interrupt mechanism Pierre-Yves David <pierre-yves.david@fb.com> parents: 23029 diff changeset	1185 """
ad144882318d bundle2: add a interrupt mechanism Pierre-Yves David <pierre-yves.david@fb.com> parents: 23029 diff changeset	1186
ad144882318d bundle2: add a interrupt mechanism Pierre-Yves David <pierre-yves.david@fb.com> parents: 23029 diff changeset	1187 def __init__(self, ui):
ad144882318d bundle2: add a interrupt mechanism Pierre-Yves David <pierre-yves.david@fb.com> parents: 23029 diff changeset	1188 self.ui = ui
ad144882318d bundle2: add a interrupt mechanism Pierre-Yves David <pierre-yves.david@fb.com> parents: 23029 diff changeset	1189 self.reply = None
24878 e530cde6d115 bundle2: disable ouput capture unless we use http (issue4613 issue4615) Pierre-Yves David <pierre-yves.david@fb.com> parents: 24851 diff changeset	1190 self.captureoutput = False
23066 ad144882318d bundle2: add a interrupt mechanism Pierre-Yves David <pierre-yves.david@fb.com> parents: 23029 diff changeset	1191
ad144882318d bundle2: add a interrupt mechanism Pierre-Yves David <pierre-yves.david@fb.com> parents: 23029 diff changeset	1192 @property
ad144882318d bundle2: add a interrupt mechanism Pierre-Yves David <pierre-yves.david@fb.com> parents: 23029 diff changeset	1193 def repo(self):
31647 4dbef666c6c9 bundle2: use ProgrammingError Jun Wu <quark@fb.com> parents: 31476 diff changeset	1194 raise error.ProgrammingError('no repo access from stream interruption')
23066 ad144882318d bundle2: add a interrupt mechanism Pierre-Yves David <pierre-yves.david@fb.com> parents: 23029 diff changeset	1195
ad144882318d bundle2: add a interrupt mechanism Pierre-Yves David <pierre-yves.david@fb.com> parents: 23029 diff changeset	1196 def gettransaction(self):
ad144882318d bundle2: add a interrupt mechanism Pierre-Yves David <pierre-yves.david@fb.com> parents: 23029 diff changeset	1197 raise TransactionUnavailable('no repo access from stream interruption')
ad144882318d bundle2: add a interrupt mechanism Pierre-Yves David <pierre-yves.david@fb.com> parents: 23029 diff changeset	1198
35113 8aa43ff9c12c bundle2: implement generic part payload decoder Gregory Szorc <gregory.szorc@gmail.com> parents: 35112 diff changeset	1199 def decodepayloadchunks(ui, fh):
8aa43ff9c12c bundle2: implement generic part payload decoder Gregory Szorc <gregory.szorc@gmail.com> parents: 35112 diff changeset	1200 """Reads bundle2 part payload data into chunks.
8aa43ff9c12c bundle2: implement generic part payload decoder Gregory Szorc <gregory.szorc@gmail.com> parents: 35112 diff changeset	1201
8aa43ff9c12c bundle2: implement generic part payload decoder Gregory Szorc <gregory.szorc@gmail.com> parents: 35112 diff changeset	1202 Part payload data consists of framed chunks. This function takes
8aa43ff9c12c bundle2: implement generic part payload decoder Gregory Szorc <gregory.szorc@gmail.com> parents: 35112 diff changeset	1203 a file handle and emits those chunks.
8aa43ff9c12c bundle2: implement generic part payload decoder Gregory Szorc <gregory.szorc@gmail.com> parents: 35112 diff changeset	1204 """
35117 589eed45a452 bundle2: inline debug logging Gregory Szorc <gregory.szorc@gmail.com> parents: 35116 diff changeset	1205 dolog = ui.configbool('devel', 'bundle2.debug')
589eed45a452 bundle2: inline debug logging Gregory Szorc <gregory.szorc@gmail.com> parents: 35116 diff changeset	1206 debug = ui.debug
589eed45a452 bundle2: inline debug logging Gregory Szorc <gregory.szorc@gmail.com> parents: 35116 diff changeset	1207
35119 764e3ad1cf54 bundle2: inline struct operations Gregory Szorc <gregory.szorc@gmail.com> parents: 35118 diff changeset	1208 headerstruct = struct.Struct(_fpayloadsize)
764e3ad1cf54 bundle2: inline struct operations Gregory Szorc <gregory.szorc@gmail.com> parents: 35118 diff changeset	1209 headersize = headerstruct.size
764e3ad1cf54 bundle2: inline struct operations Gregory Szorc <gregory.szorc@gmail.com> parents: 35118 diff changeset	1210 unpack = headerstruct.unpack
764e3ad1cf54 bundle2: inline struct operations Gregory Szorc <gregory.szorc@gmail.com> parents: 35118 diff changeset	1211
35113 8aa43ff9c12c bundle2: implement generic part payload decoder Gregory Szorc <gregory.szorc@gmail.com> parents: 35112 diff changeset	1212 readexactly = changegroup.readexactly
35118 1fb0846ad792 bundle2: inline changegroup.readexactly() Gregory Szorc <gregory.szorc@gmail.com> parents: 35117 diff changeset	1213 read = fh.read
35113 8aa43ff9c12c bundle2: implement generic part payload decoder Gregory Szorc <gregory.szorc@gmail.com> parents: 35112 diff changeset	1214
35119 764e3ad1cf54 bundle2: inline struct operations Gregory Szorc <gregory.szorc@gmail.com> parents: 35118 diff changeset	1215 chunksize = unpack(readexactly(fh, headersize))[0]
35113 8aa43ff9c12c bundle2: implement generic part payload decoder Gregory Szorc <gregory.szorc@gmail.com> parents: 35112 diff changeset	1216 indebug(ui, 'payload chunk size: %i' % chunksize)
8aa43ff9c12c bundle2: implement generic part payload decoder Gregory Szorc <gregory.szorc@gmail.com> parents: 35112 diff changeset	1217
35118 1fb0846ad792 bundle2: inline changegroup.readexactly() Gregory Szorc <gregory.szorc@gmail.com> parents: 35117 diff changeset	1218 # changegroup.readexactly() is inlined below for performance.
35113 8aa43ff9c12c bundle2: implement generic part payload decoder Gregory Szorc <gregory.szorc@gmail.com> parents: 35112 diff changeset	1219 while chunksize:
8aa43ff9c12c bundle2: implement generic part payload decoder Gregory Szorc <gregory.szorc@gmail.com> parents: 35112 diff changeset	1220 if chunksize >= 0:
35118 1fb0846ad792 bundle2: inline changegroup.readexactly() Gregory Szorc <gregory.szorc@gmail.com> parents: 35117 diff changeset	1221 s = read(chunksize)
1fb0846ad792 bundle2: inline changegroup.readexactly() Gregory Szorc <gregory.szorc@gmail.com> parents: 35117 diff changeset	1222 if len(s) < chunksize:
1fb0846ad792 bundle2: inline changegroup.readexactly() Gregory Szorc <gregory.szorc@gmail.com> parents: 35117 diff changeset	1223 raise error.Abort(_('stream ended unexpectedly '
1fb0846ad792 bundle2: inline changegroup.readexactly() Gregory Szorc <gregory.szorc@gmail.com> parents: 35117 diff changeset	1224 ' (got %d bytes, expected %d)') %
1fb0846ad792 bundle2: inline changegroup.readexactly() Gregory Szorc <gregory.szorc@gmail.com> parents: 35117 diff changeset	1225 (len(s), chunksize))
1fb0846ad792 bundle2: inline changegroup.readexactly() Gregory Szorc <gregory.szorc@gmail.com> parents: 35117 diff changeset	1226
1fb0846ad792 bundle2: inline changegroup.readexactly() Gregory Szorc <gregory.szorc@gmail.com> parents: 35117 diff changeset	1227 yield s
35113 8aa43ff9c12c bundle2: implement generic part payload decoder Gregory Szorc <gregory.szorc@gmail.com> parents: 35112 diff changeset	1228 elif chunksize == flaginterrupt:
8aa43ff9c12c bundle2: implement generic part payload decoder Gregory Szorc <gregory.szorc@gmail.com> parents: 35112 diff changeset	1229 # Interrupt "signal" detected. The regular stream is interrupted
8aa43ff9c12c bundle2: implement generic part payload decoder Gregory Szorc <gregory.szorc@gmail.com> parents: 35112 diff changeset	1230 # and a bundle2 part follows. Consume it.
8aa43ff9c12c bundle2: implement generic part payload decoder Gregory Szorc <gregory.szorc@gmail.com> parents: 35112 diff changeset	1231 interrupthandler(ui, fh)()
8aa43ff9c12c bundle2: implement generic part payload decoder Gregory Szorc <gregory.szorc@gmail.com> parents: 35112 diff changeset	1232 else:
8aa43ff9c12c bundle2: implement generic part payload decoder Gregory Szorc <gregory.szorc@gmail.com> parents: 35112 diff changeset	1233 raise error.BundleValueError(
8aa43ff9c12c bundle2: implement generic part payload decoder Gregory Szorc <gregory.szorc@gmail.com> parents: 35112 diff changeset	1234 'negative payload chunk size: %s' % chunksize)
8aa43ff9c12c bundle2: implement generic part payload decoder Gregory Szorc <gregory.szorc@gmail.com> parents: 35112 diff changeset	1235
35118 1fb0846ad792 bundle2: inline changegroup.readexactly() Gregory Szorc <gregory.szorc@gmail.com> parents: 35117 diff changeset	1236 s = read(headersize)
1fb0846ad792 bundle2: inline changegroup.readexactly() Gregory Szorc <gregory.szorc@gmail.com> parents: 35117 diff changeset	1237 if len(s) < headersize:
1fb0846ad792 bundle2: inline changegroup.readexactly() Gregory Szorc <gregory.szorc@gmail.com> parents: 35117 diff changeset	1238 raise error.Abort(_('stream ended unexpectedly '
1fb0846ad792 bundle2: inline changegroup.readexactly() Gregory Szorc <gregory.szorc@gmail.com> parents: 35117 diff changeset	1239 ' (got %d bytes, expected %d)') %
1fb0846ad792 bundle2: inline changegroup.readexactly() Gregory Szorc <gregory.szorc@gmail.com> parents: 35117 diff changeset	1240 (len(s), chunksize))
1fb0846ad792 bundle2: inline changegroup.readexactly() Gregory Szorc <gregory.szorc@gmail.com> parents: 35117 diff changeset	1241
35119 764e3ad1cf54 bundle2: inline struct operations Gregory Szorc <gregory.szorc@gmail.com> parents: 35118 diff changeset	1242 chunksize = unpack(s)[0]
35117 589eed45a452 bundle2: inline debug logging Gregory Szorc <gregory.szorc@gmail.com> parents: 35116 diff changeset	1243
589eed45a452 bundle2: inline debug logging Gregory Szorc <gregory.szorc@gmail.com> parents: 35116 diff changeset	1244 # indebug() inlined for performance.
589eed45a452 bundle2: inline debug logging Gregory Szorc <gregory.szorc@gmail.com> parents: 35116 diff changeset	1245 if dolog:
589eed45a452 bundle2: inline debug logging Gregory Szorc <gregory.szorc@gmail.com> parents: 35116 diff changeset	1246 debug('bundle2-input: payload chunk size: %i\n' % chunksize)
35113 8aa43ff9c12c bundle2: implement generic part payload decoder Gregory Szorc <gregory.szorc@gmail.com> parents: 35112 diff changeset	1247
21014 a6246bba7b9e bundle2: add an unbundle part responsible from unbundling part Pierre-Yves David <pierre-yves.david@fb.com> parents: 21013 diff changeset	1248 class unbundlepart(unpackermixin):
a6246bba7b9e bundle2: add an unbundle part responsible from unbundling part Pierre-Yves David <pierre-yves.david@fb.com> parents: 21013 diff changeset	1249 """a bundle part read from a bundle"""
a6246bba7b9e bundle2: add an unbundle part responsible from unbundling part Pierre-Yves David <pierre-yves.david@fb.com> parents: 21013 diff changeset	1250
a6246bba7b9e bundle2: add an unbundle part responsible from unbundling part Pierre-Yves David <pierre-yves.david@fb.com> parents: 21013 diff changeset	1251 def __init__(self, ui, header, fp):
a6246bba7b9e bundle2: add an unbundle part responsible from unbundling part Pierre-Yves David <pierre-yves.david@fb.com> parents: 21013 diff changeset	1252 super(unbundlepart, self).__init__(fp)
31889 a02e773008f5 bundle2: move 'seek' and 'tell' methods off the unpackermixin class Pierre-Yves David <pierre-yves.david@ens-lyon.org> parents: 31863 diff changeset	1253 self._seekable = (util.safehasattr(fp, 'seek') and
a02e773008f5 bundle2: move 'seek' and 'tell' methods off the unpackermixin class Pierre-Yves David <pierre-yves.david@ens-lyon.org> parents: 31863 diff changeset	1254 util.safehasattr(fp, 'tell'))
21014 a6246bba7b9e bundle2: add an unbundle part responsible from unbundling part Pierre-Yves David <pierre-yves.david@fb.com> parents: 21013 diff changeset	1255 self.ui = ui
a6246bba7b9e bundle2: add an unbundle part responsible from unbundling part Pierre-Yves David <pierre-yves.david@fb.com> parents: 21013 diff changeset	1256 # unbundle state attr
a6246bba7b9e bundle2: add an unbundle part responsible from unbundling part Pierre-Yves David <pierre-yves.david@fb.com> parents: 21013 diff changeset	1257 self._headerdata = header
21015 14dd49260246 bundle2: move the fromheader closure into the class itself Pierre-Yves David <pierre-yves.david@fb.com> parents: 21014 diff changeset	1258 self._headeroffset = 0
21019 3dc09f831a2e bundle2: lazy unbundle of part payload Pierre-Yves David <pierre-yves.david@fb.com> parents: 21016 diff changeset	1259 self._initialized = False
3dc09f831a2e bundle2: lazy unbundle of part payload Pierre-Yves David <pierre-yves.david@fb.com> parents: 21016 diff changeset	1260 self.consumed = False
21014 a6246bba7b9e bundle2: add an unbundle part responsible from unbundling part Pierre-Yves David <pierre-yves.david@fb.com> parents: 21013 diff changeset	1261 # part data
a6246bba7b9e bundle2: add an unbundle part responsible from unbundling part Pierre-Yves David <pierre-yves.david@fb.com> parents: 21013 diff changeset	1262 self.id = None
a6246bba7b9e bundle2: add an unbundle part responsible from unbundling part Pierre-Yves David <pierre-yves.david@fb.com> parents: 21013 diff changeset	1263 self.type = None
a6246bba7b9e bundle2: add an unbundle part responsible from unbundling part Pierre-Yves David <pierre-yves.david@fb.com> parents: 21013 diff changeset	1264 self.mandatoryparams = None
a6246bba7b9e bundle2: add an unbundle part responsible from unbundling part Pierre-Yves David <pierre-yves.david@fb.com> parents: 21013 diff changeset	1265 self.advisoryparams = None
21610 d6056805f8f4 bundle2: introduce a ``params`` dictionary on unbundled parts Pierre-Yves David <pierre-yves.david@fb.com> parents: 21609 diff changeset	1266 self.params = None
21612 f221eb0531d9 bundle2: expose mandatory params in a mandatorykeys attribute Pierre-Yves David <pierre-yves.david@fb.com> parents: 21611 diff changeset	1267 self.mandatorykeys = ()
21019 3dc09f831a2e bundle2: lazy unbundle of part payload Pierre-Yves David <pierre-yves.david@fb.com> parents: 21016 diff changeset	1268 self._readheader()
23585 94b25d71dd0f bundle2.unbundlepart: decouple mandatory from parttype Eric Sumner <ericsumner@fb.com> parents: 23439 diff changeset	1269 self._mandatory = None
24036 c7601086338a bundle2.unbundlepart: tell() implementation Eric Sumner <ericsumner@fb.com> parents: 24035 diff changeset	1270 self._pos = 0
21014 a6246bba7b9e bundle2: add an unbundle part responsible from unbundling part Pierre-Yves David <pierre-yves.david@fb.com> parents: 21013 diff changeset	1271
21015 14dd49260246 bundle2: move the fromheader closure into the class itself Pierre-Yves David <pierre-yves.david@fb.com> parents: 21014 diff changeset	1272 def _fromheader(self, size):
14dd49260246 bundle2: move the fromheader closure into the class itself Pierre-Yves David <pierre-yves.david@fb.com> parents: 21014 diff changeset	1273 """return the next <size> byte from the header"""
14dd49260246 bundle2: move the fromheader closure into the class itself Pierre-Yves David <pierre-yves.david@fb.com> parents: 21014 diff changeset	1274 offset = self._headeroffset
14dd49260246 bundle2: move the fromheader closure into the class itself Pierre-Yves David <pierre-yves.david@fb.com> parents: 21014 diff changeset	1275 data = self._headerdata[offset:(offset + size)]
21019 3dc09f831a2e bundle2: lazy unbundle of part payload Pierre-Yves David <pierre-yves.david@fb.com> parents: 21016 diff changeset	1276 self._headeroffset = offset + size
21015 14dd49260246 bundle2: move the fromheader closure into the class itself Pierre-Yves David <pierre-yves.david@fb.com> parents: 21014 diff changeset	1277 return data
14dd49260246 bundle2: move the fromheader closure into the class itself Pierre-Yves David <pierre-yves.david@fb.com> parents: 21014 diff changeset	1278
21016 b477afb1c81e bundle2: move unpackheader closure into the class Pierre-Yves David <pierre-yves.david@fb.com> parents: 21015 diff changeset	1279 def _unpackheader(self, format):
b477afb1c81e bundle2: move unpackheader closure into the class Pierre-Yves David <pierre-yves.david@fb.com> parents: 21015 diff changeset	1280 """read given format from header
b477afb1c81e bundle2: move unpackheader closure into the class Pierre-Yves David <pierre-yves.david@fb.com> parents: 21015 diff changeset	1281
b477afb1c81e bundle2: move unpackheader closure into the class Pierre-Yves David <pierre-yves.david@fb.com> parents: 21015 diff changeset	1282 This automatically compute the size of the format to read."""
b477afb1c81e bundle2: move unpackheader closure into the class Pierre-Yves David <pierre-yves.david@fb.com> parents: 21015 diff changeset	1283 data = self._fromheader(struct.calcsize(format))
b477afb1c81e bundle2: move unpackheader closure into the class Pierre-Yves David <pierre-yves.david@fb.com> parents: 21015 diff changeset	1284 return _unpack(format, data)
b477afb1c81e bundle2: move unpackheader closure into the class Pierre-Yves David <pierre-yves.david@fb.com> parents: 21015 diff changeset	1285
21608 3cb96ca90c17 bundle2: introduce an ``_initparams`` method Pierre-Yves David <pierre-yves.david@fb.com> parents: 21607 diff changeset	1286 def _initparams(self, mandatoryparams, advisoryparams):
3cb96ca90c17 bundle2: introduce an ``_initparams`` method Pierre-Yves David <pierre-yves.david@fb.com> parents: 21607 diff changeset	1287 """internal function to setup all logic related parameters"""
21609 63cc2594ef8a bundle2: make sure unbundled part param are read-only Pierre-Yves David <pierre-yves.david@fb.com> parents: 21608 diff changeset	1288 # make it read only to prevent people touching it by mistake.
63cc2594ef8a bundle2: make sure unbundled part param are read-only Pierre-Yves David <pierre-yves.david@fb.com> parents: 21608 diff changeset	1289 self.mandatoryparams = tuple(mandatoryparams)
63cc2594ef8a bundle2: make sure unbundled part param are read-only Pierre-Yves David <pierre-yves.david@fb.com> parents: 21608 diff changeset	1290 self.advisoryparams = tuple(advisoryparams)
21610 d6056805f8f4 bundle2: introduce a ``params`` dictionary on unbundled parts Pierre-Yves David <pierre-yves.david@fb.com> parents: 21609 diff changeset	1291 # user friendly UI
29591 6215b5537ba5 bundle2: use a sorted dict for holding parameters Gregory Szorc <gregory.szorc@gmail.com> parents: 29294 diff changeset	1292 self.params = util.sortdict(self.mandatoryparams)
6215b5537ba5 bundle2: use a sorted dict for holding parameters Gregory Szorc <gregory.szorc@gmail.com> parents: 29294 diff changeset	1293 self.params.update(self.advisoryparams)
21612 f221eb0531d9 bundle2: expose mandatory params in a mandatorykeys attribute Pierre-Yves David <pierre-yves.david@fb.com> parents: 21611 diff changeset	1294 self.mandatorykeys = frozenset(p[0] for p in mandatoryparams)
21608 3cb96ca90c17 bundle2: introduce an ``_initparams`` method Pierre-Yves David <pierre-yves.david@fb.com> parents: 21607 diff changeset	1295
21019 3dc09f831a2e bundle2: lazy unbundle of part payload Pierre-Yves David <pierre-yves.david@fb.com> parents: 21016 diff changeset	1296 def _readheader(self):
21014 a6246bba7b9e bundle2: add an unbundle part responsible from unbundling part Pierre-Yves David <pierre-yves.david@fb.com> parents: 21013 diff changeset	1297 """read the header and setup the object"""
21016 b477afb1c81e bundle2: move unpackheader closure into the class Pierre-Yves David <pierre-yves.david@fb.com> parents: 21015 diff changeset	1298 typesize = self._unpackheader(_fparttypesize)[0]
21015 14dd49260246 bundle2: move the fromheader closure into the class itself Pierre-Yves David <pierre-yves.david@fb.com> parents: 21014 diff changeset	1299 self.type = self._fromheader(typesize)
25320 697d8953b04d bundle2: handle new line in 'indebug' function Pierre-Yves David <pierre-yves.david@fb.com> parents: 25319 diff changeset	1300 indebug(self.ui, 'part type: "%s"' % self.type)
21016 b477afb1c81e bundle2: move unpackheader closure into the class Pierre-Yves David <pierre-yves.david@fb.com> parents: 21015 diff changeset	1301 self.id = self._unpackheader(_fpartid)[0]
33711 373ca5103ba8 bundle2: convert ints to strings using pycompat.bytestring() Augie Fackler <augie@google.com> parents: 33693 diff changeset	1302 indebug(self.ui, 'part id: "%s"' % pycompat.bytestr(self.id))
23585 94b25d71dd0f bundle2.unbundlepart: decouple mandatory from parttype Eric Sumner <ericsumner@fb.com> parents: 23439 diff changeset	1303 # extract mandatory bit from type
94b25d71dd0f bundle2.unbundlepart: decouple mandatory from parttype Eric Sumner <ericsumner@fb.com> parents: 23439 diff changeset	1304 self.mandatory = (self.type != self.type.lower())
94b25d71dd0f bundle2.unbundlepart: decouple mandatory from parttype Eric Sumner <ericsumner@fb.com> parents: 23439 diff changeset	1305 self.type = self.type.lower()
21014 a6246bba7b9e bundle2: add an unbundle part responsible from unbundling part Pierre-Yves David <pierre-yves.david@fb.com> parents: 21013 diff changeset	1306 ## reading parameters
a6246bba7b9e bundle2: add an unbundle part responsible from unbundling part Pierre-Yves David <pierre-yves.david@fb.com> parents: 21013 diff changeset	1307 # param count
21016 b477afb1c81e bundle2: move unpackheader closure into the class Pierre-Yves David <pierre-yves.david@fb.com> parents: 21015 diff changeset	1308 mancount, advcount = self._unpackheader(_fpartparamcount)
25320 697d8953b04d bundle2: handle new line in 'indebug' function Pierre-Yves David <pierre-yves.david@fb.com> parents: 25319 diff changeset	1309 indebug(self.ui, 'part parameters: %i' % (mancount + advcount))
21014 a6246bba7b9e bundle2: add an unbundle part responsible from unbundling part Pierre-Yves David <pierre-yves.david@fb.com> parents: 21013 diff changeset	1310 # param size
21016 b477afb1c81e bundle2: move unpackheader closure into the class Pierre-Yves David <pierre-yves.david@fb.com> parents: 21015 diff changeset	1311 fparamsizes = _makefpartparamsizes(mancount + advcount)
b477afb1c81e bundle2: move unpackheader closure into the class Pierre-Yves David <pierre-yves.david@fb.com> parents: 21015 diff changeset	1312 paramsizes = self._unpackheader(fparamsizes)
21014 a6246bba7b9e bundle2: add an unbundle part responsible from unbundling part Pierre-Yves David <pierre-yves.david@fb.com> parents: 21013 diff changeset	1313 # make it a list of couple again
33674 9a323773216c bundle2: work around zip() being lazy in Python 3 Augie Fackler <augie@google.com> parents: 33673 diff changeset	1314 paramsizes = list(zip(paramsizes[::2], paramsizes[1::2]))
21014 a6246bba7b9e bundle2: add an unbundle part responsible from unbundling part Pierre-Yves David <pierre-yves.david@fb.com> parents: 21013 diff changeset	1315 # split mandatory from advisory
a6246bba7b9e bundle2: add an unbundle part responsible from unbundling part Pierre-Yves David <pierre-yves.david@fb.com> parents: 21013 diff changeset	1316 mansizes = paramsizes[:mancount]
a6246bba7b9e bundle2: add an unbundle part responsible from unbundling part Pierre-Yves David <pierre-yves.david@fb.com> parents: 21013 diff changeset	1317 advsizes = paramsizes[mancount:]
23139 e53f6b72a0e4 spelling: fixes from proofreading of spell checker issues Mads Kiilerich <madski@unity3d.com> parents: 23067 diff changeset	1318 # retrieve param value
21014 a6246bba7b9e bundle2: add an unbundle part responsible from unbundling part Pierre-Yves David <pierre-yves.david@fb.com> parents: 21013 diff changeset	1319 manparams = []
a6246bba7b9e bundle2: add an unbundle part responsible from unbundling part Pierre-Yves David <pierre-yves.david@fb.com> parents: 21013 diff changeset	1320 for key, value in mansizes:
21015 14dd49260246 bundle2: move the fromheader closure into the class itself Pierre-Yves David <pierre-yves.david@fb.com> parents: 21014 diff changeset	1321 manparams.append((self._fromheader(key), self._fromheader(value)))
21014 a6246bba7b9e bundle2: add an unbundle part responsible from unbundling part Pierre-Yves David <pierre-yves.david@fb.com> parents: 21013 diff changeset	1322 advparams = []
a6246bba7b9e bundle2: add an unbundle part responsible from unbundling part Pierre-Yves David <pierre-yves.david@fb.com> parents: 21013 diff changeset	1323 for key, value in advsizes:
21015 14dd49260246 bundle2: move the fromheader closure into the class itself Pierre-Yves David <pierre-yves.david@fb.com> parents: 21014 diff changeset	1324 advparams.append((self._fromheader(key), self._fromheader(value)))
21608 3cb96ca90c17 bundle2: introduce an ``_initparams`` method Pierre-Yves David <pierre-yves.david@fb.com> parents: 21607 diff changeset	1325 self._initparams(manparams, advparams)
21014 a6246bba7b9e bundle2: add an unbundle part responsible from unbundling part Pierre-Yves David <pierre-yves.david@fb.com> parents: 21013 diff changeset	1326 ## part payload
24034 9881a1437799 bundle2.unbundlepart: raise payloadchunks from a closure to a method Eric Sumner <ericsumner@fb.com> parents: 24026 diff changeset	1327 self._payloadstream = util.chunkbuffer(self._payloadchunks())
21019 3dc09f831a2e bundle2: lazy unbundle of part payload Pierre-Yves David <pierre-yves.david@fb.com> parents: 21016 diff changeset	1328 # we read the data, tell it
3dc09f831a2e bundle2: lazy unbundle of part payload Pierre-Yves David <pierre-yves.david@fb.com> parents: 21016 diff changeset	1329 self._initialized = True
3dc09f831a2e bundle2: lazy unbundle of part payload Pierre-Yves David <pierre-yves.david@fb.com> parents: 21016 diff changeset	1330
35113 8aa43ff9c12c bundle2: implement generic part payload decoder Gregory Szorc <gregory.szorc@gmail.com> parents: 35112 diff changeset	1331 def _payloadchunks(self):
8aa43ff9c12c bundle2: implement generic part payload decoder Gregory Szorc <gregory.szorc@gmail.com> parents: 35112 diff changeset	1332 """Generator of decoded chunks in the payload."""
8aa43ff9c12c bundle2: implement generic part payload decoder Gregory Szorc <gregory.szorc@gmail.com> parents: 35112 diff changeset	1333 return decodepayloadchunks(self.ui, self._fp)
8aa43ff9c12c bundle2: implement generic part payload decoder Gregory Szorc <gregory.szorc@gmail.com> parents: 35112 diff changeset	1334
35114 db5038525718 bundle2: implement consume() API on unbundlepart Gregory Szorc <gregory.szorc@gmail.com> parents: 35113 diff changeset	1335 def consume(self):
db5038525718 bundle2: implement consume() API on unbundlepart Gregory Szorc <gregory.szorc@gmail.com> parents: 35113 diff changeset	1336 """Read the part payload until completion.
db5038525718 bundle2: implement consume() API on unbundlepart Gregory Szorc <gregory.szorc@gmail.com> parents: 35113 diff changeset	1337
db5038525718 bundle2: implement consume() API on unbundlepart Gregory Szorc <gregory.szorc@gmail.com> parents: 35113 diff changeset	1338 By consuming the part data, the underlying stream read offset will
db5038525718 bundle2: implement consume() API on unbundlepart Gregory Szorc <gregory.szorc@gmail.com> parents: 35113 diff changeset	1339 be advanced to the next part (or end of stream).
db5038525718 bundle2: implement consume() API on unbundlepart Gregory Szorc <gregory.szorc@gmail.com> parents: 35113 diff changeset	1340 """
db5038525718 bundle2: implement consume() API on unbundlepart Gregory Szorc <gregory.szorc@gmail.com> parents: 35113 diff changeset	1341 if self.consumed:
db5038525718 bundle2: implement consume() API on unbundlepart Gregory Szorc <gregory.szorc@gmail.com> parents: 35113 diff changeset	1342 return
db5038525718 bundle2: implement consume() API on unbundlepart Gregory Szorc <gregory.szorc@gmail.com> parents: 35113 diff changeset	1343
db5038525718 bundle2: implement consume() API on unbundlepart Gregory Szorc <gregory.szorc@gmail.com> parents: 35113 diff changeset	1344 chunk = self.read(32768)
db5038525718 bundle2: implement consume() API on unbundlepart Gregory Szorc <gregory.szorc@gmail.com> parents: 35113 diff changeset	1345 while chunk:
db5038525718 bundle2: implement consume() API on unbundlepart Gregory Szorc <gregory.szorc@gmail.com> parents: 35113 diff changeset	1346 self._pos += len(chunk)
db5038525718 bundle2: implement consume() API on unbundlepart Gregory Szorc <gregory.szorc@gmail.com> parents: 35113 diff changeset	1347 chunk = self.read(32768)
db5038525718 bundle2: implement consume() API on unbundlepart Gregory Szorc <gregory.szorc@gmail.com> parents: 35113 diff changeset	1348
21019 3dc09f831a2e bundle2: lazy unbundle of part payload Pierre-Yves David <pierre-yves.david@fb.com> parents: 21016 diff changeset	1349 def read(self, size=None):
3dc09f831a2e bundle2: lazy unbundle of part payload Pierre-Yves David <pierre-yves.david@fb.com> parents: 21016 diff changeset	1350 """read payload data"""
3dc09f831a2e bundle2: lazy unbundle of part payload Pierre-Yves David <pierre-yves.david@fb.com> parents: 21016 diff changeset	1351 if not self._initialized:
3dc09f831a2e bundle2: lazy unbundle of part payload Pierre-Yves David <pierre-yves.david@fb.com> parents: 21016 diff changeset	1352 self._readheader()
3dc09f831a2e bundle2: lazy unbundle of part payload Pierre-Yves David <pierre-yves.david@fb.com> parents: 21016 diff changeset	1353 if size is None:
3dc09f831a2e bundle2: lazy unbundle of part payload Pierre-Yves David <pierre-yves.david@fb.com> parents: 21016 diff changeset	1354 data = self._payloadstream.read()
3dc09f831a2e bundle2: lazy unbundle of part payload Pierre-Yves David <pierre-yves.david@fb.com> parents: 21016 diff changeset	1355 else:
3dc09f831a2e bundle2: lazy unbundle of part payload Pierre-Yves David <pierre-yves.david@fb.com> parents: 21016 diff changeset	1356 data = self._payloadstream.read(size)
25334 76cba1ecf55f bundle2: add generic debug output regarding processed part payload Pierre-Yves David <pierre-yves.david@fb.com> parents: 25333 diff changeset	1357 self._pos += len(data)
21019 3dc09f831a2e bundle2: lazy unbundle of part payload Pierre-Yves David <pierre-yves.david@fb.com> parents: 21016 diff changeset	1358 if size is None or len(data) < size:
25334 76cba1ecf55f bundle2: add generic debug output regarding processed part payload Pierre-Yves David <pierre-yves.david@fb.com> parents: 25333 diff changeset	1359 if not self.consumed and self._pos:
76cba1ecf55f bundle2: add generic debug output regarding processed part payload Pierre-Yves David <pierre-yves.david@fb.com> parents: 25333 diff changeset	1360 self.ui.debug('bundle2-input-part: total payload size %i\n'
76cba1ecf55f bundle2: add generic debug output regarding processed part payload Pierre-Yves David <pierre-yves.david@fb.com> parents: 25333 diff changeset	1361 % self._pos)
21019 3dc09f831a2e bundle2: lazy unbundle of part payload Pierre-Yves David <pierre-yves.david@fb.com> parents: 21016 diff changeset	1362 self.consumed = True
3dc09f831a2e bundle2: lazy unbundle of part payload Pierre-Yves David <pierre-yves.david@fb.com> parents: 21016 diff changeset	1363 return data
3dc09f831a2e bundle2: lazy unbundle of part payload Pierre-Yves David <pierre-yves.david@fb.com> parents: 21016 diff changeset	1364
35112 073eec083e25 bundle2: extract logic for seeking bundle2 part into own class Gregory Szorc <gregory.szorc@gmail.com> parents: 35046 diff changeset	1365 class seekableunbundlepart(unbundlepart):
073eec083e25 bundle2: extract logic for seeking bundle2 part into own class Gregory Szorc <gregory.szorc@gmail.com> parents: 35046 diff changeset	1366 """A bundle2 part in a bundle that is seekable.
073eec083e25 bundle2: extract logic for seeking bundle2 part into own class Gregory Szorc <gregory.szorc@gmail.com> parents: 35046 diff changeset	1367
073eec083e25 bundle2: extract logic for seeking bundle2 part into own class Gregory Szorc <gregory.szorc@gmail.com> parents: 35046 diff changeset	1368 Regular ``unbundlepart`` instances can only be read once. This class
073eec083e25 bundle2: extract logic for seeking bundle2 part into own class Gregory Szorc <gregory.szorc@gmail.com> parents: 35046 diff changeset	1369 extends ``unbundlepart`` to enable bi-directional seeking within the
073eec083e25 bundle2: extract logic for seeking bundle2 part into own class Gregory Szorc <gregory.szorc@gmail.com> parents: 35046 diff changeset	1370 part.
073eec083e25 bundle2: extract logic for seeking bundle2 part into own class Gregory Szorc <gregory.szorc@gmail.com> parents: 35046 diff changeset	1371
073eec083e25 bundle2: extract logic for seeking bundle2 part into own class Gregory Szorc <gregory.szorc@gmail.com> parents: 35046 diff changeset	1372 Bundle2 part data consists of framed chunks. Offsets when seeking
073eec083e25 bundle2: extract logic for seeking bundle2 part into own class Gregory Szorc <gregory.szorc@gmail.com> parents: 35046 diff changeset	1373 refer to the decoded data, not the offsets in the underlying bundle2
073eec083e25 bundle2: extract logic for seeking bundle2 part into own class Gregory Szorc <gregory.szorc@gmail.com> parents: 35046 diff changeset	1374 stream.
073eec083e25 bundle2: extract logic for seeking bundle2 part into own class Gregory Szorc <gregory.szorc@gmail.com> parents: 35046 diff changeset	1375
073eec083e25 bundle2: extract logic for seeking bundle2 part into own class Gregory Szorc <gregory.szorc@gmail.com> parents: 35046 diff changeset	1376 To facilitate quickly seeking within the decoded data, instances of this
073eec083e25 bundle2: extract logic for seeking bundle2 part into own class Gregory Szorc <gregory.szorc@gmail.com> parents: 35046 diff changeset	1377 class maintain a mapping between offsets in the underlying stream and
073eec083e25 bundle2: extract logic for seeking bundle2 part into own class Gregory Szorc <gregory.szorc@gmail.com> parents: 35046 diff changeset	1378 the decoded payload. This mapping will consume memory in proportion
073eec083e25 bundle2: extract logic for seeking bundle2 part into own class Gregory Szorc <gregory.szorc@gmail.com> parents: 35046 diff changeset	1379 to the number of chunks within the payload (which almost certainly
073eec083e25 bundle2: extract logic for seeking bundle2 part into own class Gregory Szorc <gregory.szorc@gmail.com> parents: 35046 diff changeset	1380 increases in proportion with the size of the part).
073eec083e25 bundle2: extract logic for seeking bundle2 part into own class Gregory Szorc <gregory.szorc@gmail.com> parents: 35046 diff changeset	1381 """
073eec083e25 bundle2: extract logic for seeking bundle2 part into own class Gregory Szorc <gregory.szorc@gmail.com> parents: 35046 diff changeset	1382 def __init__(self, ui, header, fp):
073eec083e25 bundle2: extract logic for seeking bundle2 part into own class Gregory Szorc <gregory.szorc@gmail.com> parents: 35046 diff changeset	1383 # (payload, file) offsets for chunk starts.
073eec083e25 bundle2: extract logic for seeking bundle2 part into own class Gregory Szorc <gregory.szorc@gmail.com> parents: 35046 diff changeset	1384 self._chunkindex = []
073eec083e25 bundle2: extract logic for seeking bundle2 part into own class Gregory Szorc <gregory.szorc@gmail.com> parents: 35046 diff changeset	1385
073eec083e25 bundle2: extract logic for seeking bundle2 part into own class Gregory Szorc <gregory.szorc@gmail.com> parents: 35046 diff changeset	1386 super(seekableunbundlepart, self).__init__(ui, header, fp)
073eec083e25 bundle2: extract logic for seeking bundle2 part into own class Gregory Szorc <gregory.szorc@gmail.com> parents: 35046 diff changeset	1387
073eec083e25 bundle2: extract logic for seeking bundle2 part into own class Gregory Szorc <gregory.szorc@gmail.com> parents: 35046 diff changeset	1388 def _payloadchunks(self, chunknum=0):
073eec083e25 bundle2: extract logic for seeking bundle2 part into own class Gregory Szorc <gregory.szorc@gmail.com> parents: 35046 diff changeset	1389 '''seek to specified chunk and start yielding data'''
073eec083e25 bundle2: extract logic for seeking bundle2 part into own class Gregory Szorc <gregory.szorc@gmail.com> parents: 35046 diff changeset	1390 if len(self._chunkindex) == 0:
073eec083e25 bundle2: extract logic for seeking bundle2 part into own class Gregory Szorc <gregory.szorc@gmail.com> parents: 35046 diff changeset	1391 assert chunknum == 0, 'Must start with chunk 0'
073eec083e25 bundle2: extract logic for seeking bundle2 part into own class Gregory Szorc <gregory.szorc@gmail.com> parents: 35046 diff changeset	1392 self._chunkindex.append((0, self._tellfp()))
073eec083e25 bundle2: extract logic for seeking bundle2 part into own class Gregory Szorc <gregory.szorc@gmail.com> parents: 35046 diff changeset	1393 else:
073eec083e25 bundle2: extract logic for seeking bundle2 part into own class Gregory Szorc <gregory.szorc@gmail.com> parents: 35046 diff changeset	1394 assert chunknum < len(self._chunkindex), \
073eec083e25 bundle2: extract logic for seeking bundle2 part into own class Gregory Szorc <gregory.szorc@gmail.com> parents: 35046 diff changeset	1395 'Unknown chunk %d' % chunknum
073eec083e25 bundle2: extract logic for seeking bundle2 part into own class Gregory Szorc <gregory.szorc@gmail.com> parents: 35046 diff changeset	1396 self._seekfp(self._chunkindex[chunknum][1])
073eec083e25 bundle2: extract logic for seeking bundle2 part into own class Gregory Szorc <gregory.szorc@gmail.com> parents: 35046 diff changeset	1397
073eec083e25 bundle2: extract logic for seeking bundle2 part into own class Gregory Szorc <gregory.szorc@gmail.com> parents: 35046 diff changeset	1398 pos = self._chunkindex[chunknum][0]
35113 8aa43ff9c12c bundle2: implement generic part payload decoder Gregory Szorc <gregory.szorc@gmail.com> parents: 35112 diff changeset	1399
8aa43ff9c12c bundle2: implement generic part payload decoder Gregory Szorc <gregory.szorc@gmail.com> parents: 35112 diff changeset	1400 for chunk in decodepayloadchunks(self.ui, self._fp):
8aa43ff9c12c bundle2: implement generic part payload decoder Gregory Szorc <gregory.szorc@gmail.com> parents: 35112 diff changeset	1401 chunknum += 1
8aa43ff9c12c bundle2: implement generic part payload decoder Gregory Szorc <gregory.szorc@gmail.com> parents: 35112 diff changeset	1402 pos += len(chunk)
8aa43ff9c12c bundle2: implement generic part payload decoder Gregory Szorc <gregory.szorc@gmail.com> parents: 35112 diff changeset	1403 if chunknum == len(self._chunkindex):
8aa43ff9c12c bundle2: implement generic part payload decoder Gregory Szorc <gregory.szorc@gmail.com> parents: 35112 diff changeset	1404 self._chunkindex.append((pos, self._tellfp()))
8aa43ff9c12c bundle2: implement generic part payload decoder Gregory Szorc <gregory.szorc@gmail.com> parents: 35112 diff changeset	1405
8aa43ff9c12c bundle2: implement generic part payload decoder Gregory Szorc <gregory.szorc@gmail.com> parents: 35112 diff changeset	1406 yield chunk
35112 073eec083e25 bundle2: extract logic for seeking bundle2 part into own class Gregory Szorc <gregory.szorc@gmail.com> parents: 35046 diff changeset	1407
073eec083e25 bundle2: extract logic for seeking bundle2 part into own class Gregory Szorc <gregory.szorc@gmail.com> parents: 35046 diff changeset	1408 def _findchunk(self, pos):
073eec083e25 bundle2: extract logic for seeking bundle2 part into own class Gregory Szorc <gregory.szorc@gmail.com> parents: 35046 diff changeset	1409 '''for a given payload position, return a chunk number and offset'''
073eec083e25 bundle2: extract logic for seeking bundle2 part into own class Gregory Szorc <gregory.szorc@gmail.com> parents: 35046 diff changeset	1410 for chunk, (ppos, fpos) in enumerate(self._chunkindex):
073eec083e25 bundle2: extract logic for seeking bundle2 part into own class Gregory Szorc <gregory.szorc@gmail.com> parents: 35046 diff changeset	1411 if ppos == pos:
073eec083e25 bundle2: extract logic for seeking bundle2 part into own class Gregory Szorc <gregory.szorc@gmail.com> parents: 35046 diff changeset	1412 return chunk, 0
073eec083e25 bundle2: extract logic for seeking bundle2 part into own class Gregory Szorc <gregory.szorc@gmail.com> parents: 35046 diff changeset	1413 elif ppos > pos:
073eec083e25 bundle2: extract logic for seeking bundle2 part into own class Gregory Szorc <gregory.szorc@gmail.com> parents: 35046 diff changeset	1414 return chunk - 1, pos - self._chunkindex[chunk - 1][0]
073eec083e25 bundle2: extract logic for seeking bundle2 part into own class Gregory Szorc <gregory.szorc@gmail.com> parents: 35046 diff changeset	1415 raise ValueError('Unknown chunk')
073eec083e25 bundle2: extract logic for seeking bundle2 part into own class Gregory Szorc <gregory.szorc@gmail.com> parents: 35046 diff changeset	1416
24036 c7601086338a bundle2.unbundlepart: tell() implementation Eric Sumner <ericsumner@fb.com> parents: 24035 diff changeset	1417 def tell(self):
c7601086338a bundle2.unbundlepart: tell() implementation Eric Sumner <ericsumner@fb.com> parents: 24035 diff changeset	1418 return self._pos
c7601086338a bundle2.unbundlepart: tell() implementation Eric Sumner <ericsumner@fb.com> parents: 24035 diff changeset	1419
35046 241d9caca11e bundle2: use os.SEEK_* constants Gregory Szorc <gregory.szorc@gmail.com> parents: 34820 diff changeset	1420 def seek(self, offset, whence=os.SEEK_SET):
241d9caca11e bundle2: use os.SEEK_* constants Gregory Szorc <gregory.szorc@gmail.com> parents: 34820 diff changeset	1421 if whence == os.SEEK_SET:
24037 f0b498cfc5c8 bundle2.unbundlepart: implement seek() Eric Sumner <ericsumner@fb.com> parents: 24036 diff changeset	1422 newpos = offset
35046 241d9caca11e bundle2: use os.SEEK_* constants Gregory Szorc <gregory.szorc@gmail.com> parents: 34820 diff changeset	1423 elif whence == os.SEEK_CUR:
24037 f0b498cfc5c8 bundle2.unbundlepart: implement seek() Eric Sumner <ericsumner@fb.com> parents: 24036 diff changeset	1424 newpos = self._pos + offset
35046 241d9caca11e bundle2: use os.SEEK_* constants Gregory Szorc <gregory.szorc@gmail.com> parents: 34820 diff changeset	1425 elif whence == os.SEEK_END:
24037 f0b498cfc5c8 bundle2.unbundlepart: implement seek() Eric Sumner <ericsumner@fb.com> parents: 24036 diff changeset	1426 if not self.consumed:
35120 699b2a759319 bundle2: avoid unbound read when seeking Gregory Szorc <gregory.szorc@gmail.com> parents: 35119 diff changeset	1427 # Can't use self.consume() here because it advances self._pos.
699b2a759319 bundle2: avoid unbound read when seeking Gregory Szorc <gregory.szorc@gmail.com> parents: 35119 diff changeset	1428 chunk = self.read(32768)
699b2a759319 bundle2: avoid unbound read when seeking Gregory Szorc <gregory.szorc@gmail.com> parents: 35119 diff changeset	1429 while chunk:
699b2a759319 bundle2: avoid unbound read when seeking Gregory Szorc <gregory.szorc@gmail.com> parents: 35119 diff changeset	1430 chunk = self.read(32768)
24037 f0b498cfc5c8 bundle2.unbundlepart: implement seek() Eric Sumner <ericsumner@fb.com> parents: 24036 diff changeset	1431 newpos = self._chunkindex[-1][0] - offset
f0b498cfc5c8 bundle2.unbundlepart: implement seek() Eric Sumner <ericsumner@fb.com> parents: 24036 diff changeset	1432 else:
f0b498cfc5c8 bundle2.unbundlepart: implement seek() Eric Sumner <ericsumner@fb.com> parents: 24036 diff changeset	1433 raise ValueError('Unknown whence value: %r' % (whence,))
f0b498cfc5c8 bundle2.unbundlepart: implement seek() Eric Sumner <ericsumner@fb.com> parents: 24036 diff changeset	1434
f0b498cfc5c8 bundle2.unbundlepart: implement seek() Eric Sumner <ericsumner@fb.com> parents: 24036 diff changeset	1435 if newpos > self._chunkindex[-1][0] and not self.consumed:
35120 699b2a759319 bundle2: avoid unbound read when seeking Gregory Szorc <gregory.szorc@gmail.com> parents: 35119 diff changeset	1436 # Can't use self.consume() here because it advances self._pos.
699b2a759319 bundle2: avoid unbound read when seeking Gregory Szorc <gregory.szorc@gmail.com> parents: 35119 diff changeset	1437 chunk = self.read(32768)
699b2a759319 bundle2: avoid unbound read when seeking Gregory Szorc <gregory.szorc@gmail.com> parents: 35119 diff changeset	1438 while chunk:
699b2a759319 bundle2: avoid unbound read when seeking Gregory Szorc <gregory.szorc@gmail.com> parents: 35119 diff changeset	1439 chunk = self.read(32668)
699b2a759319 bundle2: avoid unbound read when seeking Gregory Szorc <gregory.szorc@gmail.com> parents: 35119 diff changeset	1440
24037 f0b498cfc5c8 bundle2.unbundlepart: implement seek() Eric Sumner <ericsumner@fb.com> parents: 24036 diff changeset	1441 if not 0 <= newpos <= self._chunkindex[-1][0]:
f0b498cfc5c8 bundle2.unbundlepart: implement seek() Eric Sumner <ericsumner@fb.com> parents: 24036 diff changeset	1442 raise ValueError('Offset out of range')
f0b498cfc5c8 bundle2.unbundlepart: implement seek() Eric Sumner <ericsumner@fb.com> parents: 24036 diff changeset	1443
f0b498cfc5c8 bundle2.unbundlepart: implement seek() Eric Sumner <ericsumner@fb.com> parents: 24036 diff changeset	1444 if self._pos != newpos:
f0b498cfc5c8 bundle2.unbundlepart: implement seek() Eric Sumner <ericsumner@fb.com> parents: 24036 diff changeset	1445 chunk, internaloffset = self._findchunk(newpos)
f0b498cfc5c8 bundle2.unbundlepart: implement seek() Eric Sumner <ericsumner@fb.com> parents: 24036 diff changeset	1446 self._payloadstream = util.chunkbuffer(self._payloadchunks(chunk))
f0b498cfc5c8 bundle2.unbundlepart: implement seek() Eric Sumner <ericsumner@fb.com> parents: 24036 diff changeset	1447 adjust = self.read(internaloffset)
f0b498cfc5c8 bundle2.unbundlepart: implement seek() Eric Sumner <ericsumner@fb.com> parents: 24036 diff changeset	1448 if len(adjust) != internaloffset:
26587 56b2bcea2529 error: get Abort from 'error' instead of 'util' Pierre-Yves David <pierre-yves.david@fb.com> parents: 26565 diff changeset	1449 raise error.Abort(_('Seek failed\n'))
24037 f0b498cfc5c8 bundle2.unbundlepart: implement seek() Eric Sumner <ericsumner@fb.com> parents: 24036 diff changeset	1450 self._pos = newpos
f0b498cfc5c8 bundle2.unbundlepart: implement seek() Eric Sumner <ericsumner@fb.com> parents: 24036 diff changeset	1451
31889 a02e773008f5 bundle2: move 'seek' and 'tell' methods off the unpackermixin class Pierre-Yves David <pierre-yves.david@ens-lyon.org> parents: 31863 diff changeset	1452 def _seekfp(self, offset, whence=0):
a02e773008f5 bundle2: move 'seek' and 'tell' methods off the unpackermixin class Pierre-Yves David <pierre-yves.david@ens-lyon.org> parents: 31863 diff changeset	1453 """move the underlying file pointer
a02e773008f5 bundle2: move 'seek' and 'tell' methods off the unpackermixin class Pierre-Yves David <pierre-yves.david@ens-lyon.org> parents: 31863 diff changeset	1454
a02e773008f5 bundle2: move 'seek' and 'tell' methods off the unpackermixin class Pierre-Yves David <pierre-yves.david@ens-lyon.org> parents: 31863 diff changeset	1455 This method is meant for internal usage by the bundle2 protocol only.
a02e773008f5 bundle2: move 'seek' and 'tell' methods off the unpackermixin class Pierre-Yves David <pierre-yves.david@ens-lyon.org> parents: 31863 diff changeset	1456 They directly manipulate the low level stream including bundle2 level
a02e773008f5 bundle2: move 'seek' and 'tell' methods off the unpackermixin class Pierre-Yves David <pierre-yves.david@ens-lyon.org> parents: 31863 diff changeset	1457 instruction.
a02e773008f5 bundle2: move 'seek' and 'tell' methods off the unpackermixin class Pierre-Yves David <pierre-yves.david@ens-lyon.org> parents: 31863 diff changeset	1458
a02e773008f5 bundle2: move 'seek' and 'tell' methods off the unpackermixin class Pierre-Yves David <pierre-yves.david@ens-lyon.org> parents: 31863 diff changeset	1459 Do not use it to implement higher-level logic or methods."""
a02e773008f5 bundle2: move 'seek' and 'tell' methods off the unpackermixin class Pierre-Yves David <pierre-yves.david@ens-lyon.org> parents: 31863 diff changeset	1460 if self._seekable:
a02e773008f5 bundle2: move 'seek' and 'tell' methods off the unpackermixin class Pierre-Yves David <pierre-yves.david@ens-lyon.org> parents: 31863 diff changeset	1461 return self._fp.seek(offset, whence)
a02e773008f5 bundle2: move 'seek' and 'tell' methods off the unpackermixin class Pierre-Yves David <pierre-yves.david@ens-lyon.org> parents: 31863 diff changeset	1462 else:
a02e773008f5 bundle2: move 'seek' and 'tell' methods off the unpackermixin class Pierre-Yves David <pierre-yves.david@ens-lyon.org> parents: 31863 diff changeset	1463 raise NotImplementedError(_('File pointer is not seekable'))
a02e773008f5 bundle2: move 'seek' and 'tell' methods off the unpackermixin class Pierre-Yves David <pierre-yves.david@ens-lyon.org> parents: 31863 diff changeset	1464
a02e773008f5 bundle2: move 'seek' and 'tell' methods off the unpackermixin class Pierre-Yves David <pierre-yves.david@ens-lyon.org> parents: 31863 diff changeset	1465 def _tellfp(self):
a02e773008f5 bundle2: move 'seek' and 'tell' methods off the unpackermixin class Pierre-Yves David <pierre-yves.david@ens-lyon.org> parents: 31863 diff changeset	1466 """return the file offset, or None if file is not seekable
a02e773008f5 bundle2: move 'seek' and 'tell' methods off the unpackermixin class Pierre-Yves David <pierre-yves.david@ens-lyon.org> parents: 31863 diff changeset	1467
a02e773008f5 bundle2: move 'seek' and 'tell' methods off the unpackermixin class Pierre-Yves David <pierre-yves.david@ens-lyon.org> parents: 31863 diff changeset	1468 This method is meant for internal usage by the bundle2 protocol only.
a02e773008f5 bundle2: move 'seek' and 'tell' methods off the unpackermixin class Pierre-Yves David <pierre-yves.david@ens-lyon.org> parents: 31863 diff changeset	1469 They directly manipulate the low level stream including bundle2 level
a02e773008f5 bundle2: move 'seek' and 'tell' methods off the unpackermixin class Pierre-Yves David <pierre-yves.david@ens-lyon.org> parents: 31863 diff changeset	1470 instruction.
a02e773008f5 bundle2: move 'seek' and 'tell' methods off the unpackermixin class Pierre-Yves David <pierre-yves.david@ens-lyon.org> parents: 31863 diff changeset	1471
a02e773008f5 bundle2: move 'seek' and 'tell' methods off the unpackermixin class Pierre-Yves David <pierre-yves.david@ens-lyon.org> parents: 31863 diff changeset	1472 Do not use it to implement higher-level logic or methods."""
a02e773008f5 bundle2: move 'seek' and 'tell' methods off the unpackermixin class Pierre-Yves David <pierre-yves.david@ens-lyon.org> parents: 31863 diff changeset	1473 if self._seekable:
a02e773008f5 bundle2: move 'seek' and 'tell' methods off the unpackermixin class Pierre-Yves David <pierre-yves.david@ens-lyon.org> parents: 31863 diff changeset	1474 try:
a02e773008f5 bundle2: move 'seek' and 'tell' methods off the unpackermixin class Pierre-Yves David <pierre-yves.david@ens-lyon.org> parents: 31863 diff changeset	1475 return self._fp.tell()
a02e773008f5 bundle2: move 'seek' and 'tell' methods off the unpackermixin class Pierre-Yves David <pierre-yves.david@ens-lyon.org> parents: 31863 diff changeset	1476 except IOError as e:
a02e773008f5 bundle2: move 'seek' and 'tell' methods off the unpackermixin class Pierre-Yves David <pierre-yves.david@ens-lyon.org> parents: 31863 diff changeset	1477 if e.errno == errno.ESPIPE:
a02e773008f5 bundle2: move 'seek' and 'tell' methods off the unpackermixin class Pierre-Yves David <pierre-yves.david@ens-lyon.org> parents: 31863 diff changeset	1478 self._seekable = False
a02e773008f5 bundle2: move 'seek' and 'tell' methods off the unpackermixin class Pierre-Yves David <pierre-yves.david@ens-lyon.org> parents: 31863 diff changeset	1479 else:
a02e773008f5 bundle2: move 'seek' and 'tell' methods off the unpackermixin class Pierre-Yves David <pierre-yves.david@ens-lyon.org> parents: 31863 diff changeset	1480 raise
a02e773008f5 bundle2: move 'seek' and 'tell' methods off the unpackermixin class Pierre-Yves David <pierre-yves.david@ens-lyon.org> parents: 31863 diff changeset	1481 return None
a02e773008f5 bundle2: move 'seek' and 'tell' methods off the unpackermixin class Pierre-Yves David <pierre-yves.david@ens-lyon.org> parents: 31863 diff changeset	1482
25317 5a5b7046d00e bundle2: add an informative comment to the capability dict Pierre-Yves David <pierre-yves.david@fb.com> parents: 25315 diff changeset	1483 # These are only the static capabilities.
5a5b7046d00e bundle2: add an informative comment to the capability dict Pierre-Yves David <pierre-yves.david@fb.com> parents: 25315 diff changeset	1484 # Check the 'getrepocaps' function for the rest.
24686 e0e28e910fa3 bundle2: rename format, parts and config to final names Pierre-Yves David <pierre-yves.david@fb.com> parents: 24648 diff changeset	1485 capabilities = {'HG20': (),
35259 ad5f2b923b0d push: include a 'check:bookmarks' part when possible Boris Feld <boris.feld@octobus.net> parents: 35258 diff changeset	1486 'bookmarks': (),
25493 d8e7b0781ad7 bundle2: convey PushkeyFailed error over the wire Pierre-Yves David <pierre-yves.david@fb.com> parents: 25492 diff changeset	1487 'error': ('abort', 'unsupportedcontent', 'pushraced',
d8e7b0781ad7 bundle2: convey PushkeyFailed error over the wire Pierre-Yves David <pierre-yves.david@fb.com> parents: 25492 diff changeset	1488 'pushkey'),
24686 e0e28e910fa3 bundle2: rename format, parts and config to final names Pierre-Yves David <pierre-yves.david@fb.com> parents: 24648 diff changeset	1489 'listkeys': (),
e0e28e910fa3 bundle2: rename format, parts and config to final names Pierre-Yves David <pierre-yves.david@fb.com> parents: 24648 diff changeset	1490 'pushkey': (),
23029 149fc8a44184 bundle2: client side support for a part to import external bundles Mike Hommey <mh@glandium.org> parents: 23011 diff changeset	1491 'digests': tuple(sorted(util.DIGESTS.keys())),
24686 e0e28e910fa3 bundle2: rename format, parts and config to final names Pierre-Yves David <pierre-yves.david@fb.com> parents: 24648 diff changeset	1492 'remote-changegroup': ('http', 'https'),
25401 d29201352af7 bundle2: part handler for processing .hgtags fnodes mappings Gregory Szorc <gregory.szorc@gmail.com> parents: 25336 diff changeset	1493 'hgtagsfnodes': (),
34322 10e162bb9bf5 pull: use 'phase-heads' to retrieve phase information Boris Feld <boris.feld@octobus.net> parents: 34321 diff changeset	1494 'phases': ('heads',),
35763 7eedbd5d4880 streamclone: add support for bundle2 based stream clone Boris Feld <boris.feld@octobus.net> parents: 35758 diff changeset	1495 'stream': ('v2',),
22341 2d16b39601b5 obsmarker: move bundle2caps from the localrepo class to the bundle2 module Pierre-Yves David <pierre-yves.david@fb.com> parents: 22340 diff changeset	1496 }
2d16b39601b5 obsmarker: move bundle2caps from the localrepo class to the bundle2 module Pierre-Yves David <pierre-yves.david@fb.com> parents: 22340 diff changeset	1497
35783 c97639ad6874 bundle2: specify what capabilities will be used for Gregory Szorc <gregory.szorc@gmail.com> parents: 35763 diff changeset	1498 def getrepocaps(repo, allowpushback=False, role=None):
22342 262c5cc126c1 bundle2: introduce a `getrepocaps` to retrieve the bundle2 caps of a repo Pierre-Yves David <pierre-yves.david@fb.com> parents: 22341 diff changeset	1499 """return the bundle2 capabilities for a given repo
262c5cc126c1 bundle2: introduce a `getrepocaps` to retrieve the bundle2 caps of a repo Pierre-Yves David <pierre-yves.david@fb.com> parents: 22341 diff changeset	1500
22343 76677a2c1cfd bundle2: advertise the obsmarker part in bundle2 capabilities Pierre-Yves David <pierre-yves.david@fb.com> parents: 22342 diff changeset	1501 Exists to allow extensions (like evolution) to mutate the capabilities.
35783 c97639ad6874 bundle2: specify what capabilities will be used for Gregory Szorc <gregory.szorc@gmail.com> parents: 35763 diff changeset	1502
c97639ad6874 bundle2: specify what capabilities will be used for Gregory Szorc <gregory.szorc@gmail.com> parents: 35763 diff changeset	1503 The returned value is used for servers advertising their capabilities as
c97639ad6874 bundle2: specify what capabilities will be used for Gregory Szorc <gregory.szorc@gmail.com> parents: 35763 diff changeset	1504 well as clients advertising their capabilities to servers as part of
c97639ad6874 bundle2: specify what capabilities will be used for Gregory Szorc <gregory.szorc@gmail.com> parents: 35763 diff changeset	1505 bundle2 requests. The ``role`` argument specifies which is which.
22342 262c5cc126c1 bundle2: introduce a `getrepocaps` to retrieve the bundle2 caps of a repo Pierre-Yves David <pierre-yves.david@fb.com> parents: 22341 diff changeset	1506 """
35783 c97639ad6874 bundle2: specify what capabilities will be used for Gregory Szorc <gregory.szorc@gmail.com> parents: 35763 diff changeset	1507 if role not in ('client', 'server'):
c97639ad6874 bundle2: specify what capabilities will be used for Gregory Szorc <gregory.szorc@gmail.com> parents: 35763 diff changeset	1508 raise error.ProgrammingError('role argument must be client or server')
c97639ad6874 bundle2: specify what capabilities will be used for Gregory Szorc <gregory.szorc@gmail.com> parents: 35763 diff changeset	1509
22343 76677a2c1cfd bundle2: advertise the obsmarker part in bundle2 capabilities Pierre-Yves David <pierre-yves.david@fb.com> parents: 22342 diff changeset	1510 caps = capabilities.copy()
27953 88609cfa3745 changegroup: fix pulling to treemanifest repo from flat repo (issue5066) Martin von Zweigbergk <martinvonz@google.com> parents: 27879 diff changeset	1511 caps['changegroup'] = tuple(sorted(
88609cfa3745 changegroup: fix pulling to treemanifest repo from flat repo (issue5066) Martin von Zweigbergk <martinvonz@google.com> parents: 27879 diff changeset	1512 changegroup.supportedincomingversions(repo)))
22953 b1d694d3975e obsolete: add exchange option Durham Goode <durham@fb.com> parents: 22661 diff changeset	1513 if obsolete.isenabled(repo, obsolete.exchangeopt):
22343 76677a2c1cfd bundle2: advertise the obsmarker part in bundle2 capabilities Pierre-Yves David <pierre-yves.david@fb.com> parents: 22342 diff changeset	1514 supportedformat = tuple('V%i' % v for v in obsolete.formats)
24686 e0e28e910fa3 bundle2: rename format, parts and config to final names Pierre-Yves David <pierre-yves.david@fb.com> parents: 24648 diff changeset	1515 caps['obsmarkers'] = supportedformat
23439 743736fc7c41 bundle2-push: provide transaction to reply unbundler Eric Sumner <ericsumner@fb.com> parents: 23438 diff changeset	1516 if allowpushback:
24686 e0e28e910fa3 bundle2: rename format, parts and config to final names Pierre-Yves David <pierre-yves.david@fb.com> parents: 24648 diff changeset	1517 caps['pushback'] = ()
33222 593ad8df9dd2 configitems: register the 'server.concurrent-push-mode' config Pierre-Yves David <pierre-yves.david@octobus.net> parents: 33159 diff changeset	1518 cpmode = repo.ui.config('server', 'concurrent-push-mode')
32892 a7851519ea02 check-concurrency: expose the feature as 'concurrent-push-mode' Pierre-Yves David <pierre-yves.david@octobus.net> parents: 32709 diff changeset	1519 if cpmode == 'check-related':
32709 16ada4cbb1a9 push: add a way to allow concurrent pushes on unrelated heads Pierre-Yves David <pierre-yves.david@octobus.net> parents: 32516 diff changeset	1520 caps['checkheads'] = ('related',)
34322 10e162bb9bf5 pull: use 'phase-heads' to retrieve phase information Boris Feld <boris.feld@octobus.net> parents: 34321 diff changeset	1521 if 'phases' in repo.ui.configlist('devel', 'legacy.exchange'):
10e162bb9bf5 pull: use 'phase-heads' to retrieve phase information Boris Feld <boris.feld@octobus.net> parents: 34321 diff changeset	1522 caps.pop('phases')
35790 c1b9eb15a51c bundle2: don't advertise stream bundle2 capability when feature disabled Gregory Szorc <gregory.szorc@gmail.com> parents: 35788 diff changeset	1523
c1b9eb15a51c bundle2: don't advertise stream bundle2 capability when feature disabled Gregory Szorc <gregory.szorc@gmail.com> parents: 35788 diff changeset	1524 # Don't advertise stream clone support in server mode if not configured.
c1b9eb15a51c bundle2: don't advertise stream bundle2 capability when feature disabled Gregory Szorc <gregory.szorc@gmail.com> parents: 35788 diff changeset	1525 if role == 'server':
c1b9eb15a51c bundle2: don't advertise stream bundle2 capability when feature disabled Gregory Szorc <gregory.szorc@gmail.com> parents: 35788 diff changeset	1526 streamsupported = repo.ui.configbool('server', 'uncompressed',
c1b9eb15a51c bundle2: don't advertise stream bundle2 capability when feature disabled Gregory Szorc <gregory.szorc@gmail.com> parents: 35788 diff changeset	1527 untrusted=True)
c1b9eb15a51c bundle2: don't advertise stream bundle2 capability when feature disabled Gregory Szorc <gregory.szorc@gmail.com> parents: 35788 diff changeset	1528 featuresupported = repo.ui.configbool('experimental', 'bundle2.stream')
c1b9eb15a51c bundle2: don't advertise stream bundle2 capability when feature disabled Gregory Szorc <gregory.szorc@gmail.com> parents: 35788 diff changeset	1529
c1b9eb15a51c bundle2: don't advertise stream bundle2 capability when feature disabled Gregory Szorc <gregory.szorc@gmail.com> parents: 35788 diff changeset	1530 if not streamsupported or not featuresupported:
c1b9eb15a51c bundle2: don't advertise stream bundle2 capability when feature disabled Gregory Szorc <gregory.szorc@gmail.com> parents: 35788 diff changeset	1531 caps.pop('stream')
35792 1d118f9f4f57 bundle2: always advertise client support for stream parts Gregory Szorc <gregory.szorc@gmail.com> parents: 35790 diff changeset	1532 # Else always advertise support on client, because payload support
1d118f9f4f57 bundle2: always advertise client support for stream parts Gregory Szorc <gregory.szorc@gmail.com> parents: 35790 diff changeset	1533 # should always be advertised.
35790 c1b9eb15a51c bundle2: don't advertise stream bundle2 capability when feature disabled Gregory Szorc <gregory.szorc@gmail.com> parents: 35788 diff changeset	1534
22343 76677a2c1cfd bundle2: advertise the obsmarker part in bundle2 capabilities Pierre-Yves David <pierre-yves.david@fb.com> parents: 22342 diff changeset	1535 return caps
22342 262c5cc126c1 bundle2: introduce a `getrepocaps` to retrieve the bundle2 caps of a repo Pierre-Yves David <pierre-yves.david@fb.com> parents: 22341 diff changeset	1536
21644 17755dd8c509 bundle2: introduce a bundle2caps function Pierre-Yves David <pierre-yves.david@fb.com> parents: 21628 diff changeset	1537 def bundle2caps(remote):
23139 e53f6b72a0e4 spelling: fixes from proofreading of spell checker issues Mads Kiilerich <madski@unity3d.com> parents: 23067 diff changeset	1538 """return the bundle capabilities of a peer as dict"""
24686 e0e28e910fa3 bundle2: rename format, parts and config to final names Pierre-Yves David <pierre-yves.david@fb.com> parents: 24648 diff changeset	1539 raw = remote.capable('bundle2')
21644 17755dd8c509 bundle2: introduce a bundle2caps function Pierre-Yves David <pierre-yves.david@fb.com> parents: 21628 diff changeset	1540 if not raw and raw != '':
17755dd8c509 bundle2: introduce a bundle2caps function Pierre-Yves David <pierre-yves.david@fb.com> parents: 21628 diff changeset	1541 return {}
28883 032c4c2f802a pycompat: switch to util.urlreq/util.urlerr for py3 compat timeless <timeless@mozdev.org> parents: 28672 diff changeset	1542 capsblob = urlreq.unquote(remote.capable('bundle2'))
21644 17755dd8c509 bundle2: introduce a bundle2caps function Pierre-Yves David <pierre-yves.david@fb.com> parents: 21628 diff changeset	1543 return decodecaps(capsblob)
21014 a6246bba7b9e bundle2: add an unbundle part responsible from unbundling part Pierre-Yves David <pierre-yves.david@fb.com> parents: 21013 diff changeset	1544
22344 9829b7948100 bundle2: add a `obsmarkersversion` function to extract supported version Pierre-Yves David <pierre-yves.david@fb.com> parents: 22343 diff changeset	1545 def obsmarkersversion(caps):
9829b7948100 bundle2: add a `obsmarkersversion` function to extract supported version Pierre-Yves David <pierre-yves.david@fb.com> parents: 22343 diff changeset	1546 """extract the list of supported obsmarkers versions from a bundle2caps dict
9829b7948100 bundle2: add a `obsmarkersversion` function to extract supported version Pierre-Yves David <pierre-yves.david@fb.com> parents: 22343 diff changeset	1547 """
24686 e0e28e910fa3 bundle2: rename format, parts and config to final names Pierre-Yves David <pierre-yves.david@fb.com> parents: 24648 diff changeset	1548 obscaps = caps.get('obsmarkers', ())
22344 9829b7948100 bundle2: add a `obsmarkersversion` function to extract supported version Pierre-Yves David <pierre-yves.david@fb.com> parents: 22343 diff changeset	1549 return [int(c[1:]) for c in obscaps if c.startswith('V')]
9829b7948100 bundle2: add a `obsmarkersversion` function to extract supported version Pierre-Yves David <pierre-yves.david@fb.com> parents: 22343 diff changeset	1550
32221 9dc36df78403 bundle: introduce an higher level function to write bundle on disk Pierre-Yves David <pierre-yves.david@octobus.net> parents: 32186 diff changeset	1551 def writenewbundle(ui, repo, source, filename, bundletype, outgoing, opts,
9dc36df78403 bundle: introduce an higher level function to write bundle on disk Pierre-Yves David <pierre-yves.david@octobus.net> parents: 32186 diff changeset	1552 vfs=None, compression=None, compopts=None):
9dc36df78403 bundle: introduce an higher level function to write bundle on disk Pierre-Yves David <pierre-yves.david@octobus.net> parents: 32186 diff changeset	1553 if bundletype.startswith('HG10'):
34101 5ede882c249c changegroup: replace getchangegroup with makechangegroup Durham Goode <durham@fb.com> parents: 33888 diff changeset	1554 cg = changegroup.makechangegroup(repo, outgoing, '01', source)
32221 9dc36df78403 bundle: introduce an higher level function to write bundle on disk Pierre-Yves David <pierre-yves.david@octobus.net> parents: 32186 diff changeset	1555 return writebundle(ui, cg, filename, bundletype, vfs=vfs,
9dc36df78403 bundle: introduce an higher level function to write bundle on disk Pierre-Yves David <pierre-yves.david@octobus.net> parents: 32186 diff changeset	1556 compression=compression, compopts=compopts)
9dc36df78403 bundle: introduce an higher level function to write bundle on disk Pierre-Yves David <pierre-yves.david@octobus.net> parents: 32186 diff changeset	1557 elif not bundletype.startswith('HG20'):
9dc36df78403 bundle: introduce an higher level function to write bundle on disk Pierre-Yves David <pierre-yves.david@octobus.net> parents: 32186 diff changeset	1558 raise error.ProgrammingError('unknown bundle type: %s' % bundletype)
9dc36df78403 bundle: introduce an higher level function to write bundle on disk Pierre-Yves David <pierre-yves.david@octobus.net> parents: 32186 diff changeset	1559
32516 37d70ba1d9d1 bundle: add an experimental knob to include obsmarkers in bundle Pierre-Yves David <pierre-yves.david@octobus.net> parents: 32515 diff changeset	1560 caps = {}
37d70ba1d9d1 bundle: add an experimental knob to include obsmarkers in bundle Pierre-Yves David <pierre-yves.david@octobus.net> parents: 32515 diff changeset	1561 if 'obsolescence' in opts:
37d70ba1d9d1 bundle: add an experimental knob to include obsmarkers in bundle Pierre-Yves David <pierre-yves.david@octobus.net> parents: 32515 diff changeset	1562 caps['obsmarkers'] = ('V1',)
37d70ba1d9d1 bundle: add an experimental knob to include obsmarkers in bundle Pierre-Yves David <pierre-yves.david@octobus.net> parents: 32515 diff changeset	1563 bundle = bundle20(ui, caps)
32221 9dc36df78403 bundle: introduce an higher level function to write bundle on disk Pierre-Yves David <pierre-yves.david@octobus.net> parents: 32186 diff changeset	1564 bundle.setcompression(compression, compopts)
9dc36df78403 bundle: introduce an higher level function to write bundle on disk Pierre-Yves David <pierre-yves.david@octobus.net> parents: 32186 diff changeset	1565 _addpartsfromopts(ui, repo, bundle, source, outgoing, opts)
9dc36df78403 bundle: introduce an higher level function to write bundle on disk Pierre-Yves David <pierre-yves.david@octobus.net> parents: 32186 diff changeset	1566 chunkiter = bundle.getchunks()
9dc36df78403 bundle: introduce an higher level function to write bundle on disk Pierre-Yves David <pierre-yves.david@octobus.net> parents: 32186 diff changeset	1567
9dc36df78403 bundle: introduce an higher level function to write bundle on disk Pierre-Yves David <pierre-yves.david@octobus.net> parents: 32186 diff changeset	1568 return changegroup.writechunks(ui, chunkiter, filename, vfs=vfs)
9dc36df78403 bundle: introduce an higher level function to write bundle on disk Pierre-Yves David <pierre-yves.david@octobus.net> parents: 32186 diff changeset	1569
9dc36df78403 bundle: introduce an higher level function to write bundle on disk Pierre-Yves David <pierre-yves.david@octobus.net> parents: 32186 diff changeset	1570 def _addpartsfromopts(ui, repo, bundler, source, outgoing, opts):
9dc36df78403 bundle: introduce an higher level function to write bundle on disk Pierre-Yves David <pierre-yves.david@octobus.net> parents: 32186 diff changeset	1571 # We should eventually reconcile this logic with the one behind
9dc36df78403 bundle: introduce an higher level function to write bundle on disk Pierre-Yves David <pierre-yves.david@octobus.net> parents: 32186 diff changeset	1572 # 'exchange.getbundle2partsgenerator'.
9dc36df78403 bundle: introduce an higher level function to write bundle on disk Pierre-Yves David <pierre-yves.david@octobus.net> parents: 32186 diff changeset	1573 #
9dc36df78403 bundle: introduce an higher level function to write bundle on disk Pierre-Yves David <pierre-yves.david@octobus.net> parents: 32186 diff changeset	1574 # The type of input from 'getbundle' and 'writenewbundle' are a bit
9dc36df78403 bundle: introduce an higher level function to write bundle on disk Pierre-Yves David <pierre-yves.david@octobus.net> parents: 32186 diff changeset	1575 # different right now. So we keep them separated for now for the sake of
9dc36df78403 bundle: introduce an higher level function to write bundle on disk Pierre-Yves David <pierre-yves.david@octobus.net> parents: 32186 diff changeset	1576 # simplicity.
9dc36df78403 bundle: introduce an higher level function to write bundle on disk Pierre-Yves David <pierre-yves.david@octobus.net> parents: 32186 diff changeset	1577
9dc36df78403 bundle: introduce an higher level function to write bundle on disk Pierre-Yves David <pierre-yves.david@octobus.net> parents: 32186 diff changeset	1578 # we always want a changegroup in such bundle
9dc36df78403 bundle: introduce an higher level function to write bundle on disk Pierre-Yves David <pierre-yves.david@octobus.net> parents: 32186 diff changeset	1579 cgversion = opts.get('cg.version')
9dc36df78403 bundle: introduce an higher level function to write bundle on disk Pierre-Yves David <pierre-yves.david@octobus.net> parents: 32186 diff changeset	1580 if cgversion is None:
9dc36df78403 bundle: introduce an higher level function to write bundle on disk Pierre-Yves David <pierre-yves.david@octobus.net> parents: 32186 diff changeset	1581 cgversion = changegroup.safeversion(repo)
34101 5ede882c249c changegroup: replace getchangegroup with makechangegroup Durham Goode <durham@fb.com> parents: 33888 diff changeset	1582 cg = changegroup.makechangegroup(repo, outgoing, cgversion, source)
32221 9dc36df78403 bundle: introduce an higher level function to write bundle on disk Pierre-Yves David <pierre-yves.david@octobus.net> parents: 32186 diff changeset	1583 part = bundler.newpart('changegroup', data=cg.getchunks())
9dc36df78403 bundle: introduce an higher level function to write bundle on disk Pierre-Yves David <pierre-yves.david@octobus.net> parents: 32186 diff changeset	1584 part.addparam('version', cg.version)
9dc36df78403 bundle: introduce an higher level function to write bundle on disk Pierre-Yves David <pierre-yves.david@octobus.net> parents: 32186 diff changeset	1585 if 'clcount' in cg.extras:
34221 8e0358024a36 bundles: turn nbchanges int into a bytestr using pycompat.bytestr Augie Fackler <raf@durin42.com> parents: 34152 diff changeset	1586 part.addparam('nbchanges', '%d' % cg.extras['clcount'],
32221 9dc36df78403 bundle: introduce an higher level function to write bundle on disk Pierre-Yves David <pierre-yves.david@octobus.net> parents: 32186 diff changeset	1587 mandatory=False)
33408 e3867c712d51 bundle2: automatically add 'targetphase' parameter in writenewbundle Boris Feld <boris.feld@octobus.net> parents: 33407 diff changeset	1588 if opts.get('phases') and repo.revs('%ln and secret()',
e3867c712d51 bundle2: automatically add 'targetphase' parameter in writenewbundle Boris Feld <boris.feld@octobus.net> parents: 33407 diff changeset	1589 outgoing.missingheads):
e3867c712d51 bundle2: automatically add 'targetphase' parameter in writenewbundle Boris Feld <boris.feld@octobus.net> parents: 33407 diff changeset	1590 part.addparam('targetphase', '%d' % phases.secret, mandatory=False)
32221 9dc36df78403 bundle: introduce an higher level function to write bundle on disk Pierre-Yves David <pierre-yves.david@octobus.net> parents: 32186 diff changeset	1591
32223 d7f93ebbbbdf bundle: add optional 'tagsfnodecache' data to on disk bundle (issue5543) Pierre-Yves David <pierre-yves.david@octobus.net> parents: 32222 diff changeset	1592 addparttagsfnodescache(repo, bundler, outgoing)
d7f93ebbbbdf bundle: add optional 'tagsfnodecache' data to on disk bundle (issue5543) Pierre-Yves David <pierre-yves.david@octobus.net> parents: 32222 diff changeset	1593
32516 37d70ba1d9d1 bundle: add an experimental knob to include obsmarkers in bundle Pierre-Yves David <pierre-yves.david@octobus.net> parents: 32515 diff changeset	1594 if opts.get('obsolescence', False):
37d70ba1d9d1 bundle: add an experimental knob to include obsmarkers in bundle Pierre-Yves David <pierre-yves.david@octobus.net> parents: 32515 diff changeset	1595 obsmarkers = repo.obsstore.relevantmarkers(outgoing.missing)
37d70ba1d9d1 bundle: add an experimental knob to include obsmarkers in bundle Pierre-Yves David <pierre-yves.david@octobus.net> parents: 32515 diff changeset	1596 buildobsmarkerspart(bundler, obsmarkers)
37d70ba1d9d1 bundle: add an experimental knob to include obsmarkers in bundle Pierre-Yves David <pierre-yves.david@octobus.net> parents: 32515 diff changeset	1597
33031 e8c8d81eb864 bundle: add config option to include phases Martin von Zweigbergk <martinvonz@google.com> parents: 33030 diff changeset	1598 if opts.get('phases', False):
e8c8d81eb864 bundle: add config option to include phases Martin von Zweigbergk <martinvonz@google.com> parents: 33030 diff changeset	1599 headsbyphase = phases.subsetphaseheads(repo, outgoing.missing)
34319 5779d096a696 phases: move binary encoding into a reusable function Boris Feld <boris.feld@octobus.net> parents: 34318 diff changeset	1600 phasedata = phases.binaryencode(headsbyphase)
5779d096a696 phases: move binary encoding into a reusable function Boris Feld <boris.feld@octobus.net> parents: 34318 diff changeset	1601 bundler.newpart('phase-heads', data=phasedata)
33031 e8c8d81eb864 bundle: add config option to include phases Martin von Zweigbergk <martinvonz@google.com> parents: 33030 diff changeset	1602
32222 6068712cbf03 bundle2: move tagsfnodecache generation in a generic function Pierre-Yves David <pierre-yves.david@octobus.net> parents: 32221 diff changeset	1603 def addparttagsfnodescache(repo, bundler, outgoing):
6068712cbf03 bundle2: move tagsfnodecache generation in a generic function Pierre-Yves David <pierre-yves.david@octobus.net> parents: 32221 diff changeset	1604 # we include the tags fnode cache for the bundle changeset
6068712cbf03 bundle2: move tagsfnodecache generation in a generic function Pierre-Yves David <pierre-yves.david@octobus.net> parents: 32221 diff changeset	1605 # (as an optional parts)
6068712cbf03 bundle2: move tagsfnodecache generation in a generic function Pierre-Yves David <pierre-yves.david@octobus.net> parents: 32221 diff changeset	1606 cache = tags.hgtagsfnodescache(repo.unfiltered())
6068712cbf03 bundle2: move tagsfnodecache generation in a generic function Pierre-Yves David <pierre-yves.david@octobus.net> parents: 32221 diff changeset	1607 chunks = []
6068712cbf03 bundle2: move tagsfnodecache generation in a generic function Pierre-Yves David <pierre-yves.david@octobus.net> parents: 32221 diff changeset	1608
6068712cbf03 bundle2: move tagsfnodecache generation in a generic function Pierre-Yves David <pierre-yves.david@octobus.net> parents: 32221 diff changeset	1609 # .hgtags fnodes are only relevant for head changesets. While we could
6068712cbf03 bundle2: move tagsfnodecache generation in a generic function Pierre-Yves David <pierre-yves.david@octobus.net> parents: 32221 diff changeset	1610 # transfer values for all known nodes, there will likely be little to
6068712cbf03 bundle2: move tagsfnodecache generation in a generic function Pierre-Yves David <pierre-yves.david@octobus.net> parents: 32221 diff changeset	1611 # no benefit.
6068712cbf03 bundle2: move tagsfnodecache generation in a generic function Pierre-Yves David <pierre-yves.david@octobus.net> parents: 32221 diff changeset	1612 #
6068712cbf03 bundle2: move tagsfnodecache generation in a generic function Pierre-Yves David <pierre-yves.david@octobus.net> parents: 32221 diff changeset	1613 # We don't bother using a generator to produce output data because
6068712cbf03 bundle2: move tagsfnodecache generation in a generic function Pierre-Yves David <pierre-yves.david@octobus.net> parents: 32221 diff changeset	1614 # a) we only have 40 bytes per head and even esoteric numbers of heads
6068712cbf03 bundle2: move tagsfnodecache generation in a generic function Pierre-Yves David <pierre-yves.david@octobus.net> parents: 32221 diff changeset	1615 # consume little memory (1M heads is 40MB) b) we don't want to send the
6068712cbf03 bundle2: move tagsfnodecache generation in a generic function Pierre-Yves David <pierre-yves.david@octobus.net> parents: 32221 diff changeset	1616 # part if we don't have entries and knowing if we have entries requires
6068712cbf03 bundle2: move tagsfnodecache generation in a generic function Pierre-Yves David <pierre-yves.david@octobus.net> parents: 32221 diff changeset	1617 # cache lookups.
6068712cbf03 bundle2: move tagsfnodecache generation in a generic function Pierre-Yves David <pierre-yves.david@octobus.net> parents: 32221 diff changeset	1618 for node in outgoing.missingheads:
6068712cbf03 bundle2: move tagsfnodecache generation in a generic function Pierre-Yves David <pierre-yves.david@octobus.net> parents: 32221 diff changeset	1619 # Don't compute missing, as this may slow down serving.
6068712cbf03 bundle2: move tagsfnodecache generation in a generic function Pierre-Yves David <pierre-yves.david@octobus.net> parents: 32221 diff changeset	1620 fnode = cache.getfnode(node, computemissing=False)
6068712cbf03 bundle2: move tagsfnodecache generation in a generic function Pierre-Yves David <pierre-yves.david@octobus.net> parents: 32221 diff changeset	1621 if fnode is not None:
6068712cbf03 bundle2: move tagsfnodecache generation in a generic function Pierre-Yves David <pierre-yves.david@octobus.net> parents: 32221 diff changeset	1622 chunks.extend([node, fnode])
6068712cbf03 bundle2: move tagsfnodecache generation in a generic function Pierre-Yves David <pierre-yves.david@octobus.net> parents: 32221 diff changeset	1623
6068712cbf03 bundle2: move tagsfnodecache generation in a generic function Pierre-Yves David <pierre-yves.david@octobus.net> parents: 32221 diff changeset	1624 if chunks:
6068712cbf03 bundle2: move tagsfnodecache generation in a generic function Pierre-Yves David <pierre-yves.david@octobus.net> parents: 32221 diff changeset	1625 bundler.newpart('hgtagsfnodes', data=''.join(chunks))
6068712cbf03 bundle2: move tagsfnodecache generation in a generic function Pierre-Yves David <pierre-yves.david@octobus.net> parents: 32221 diff changeset	1626
32515 e70d6dbde713 bundle2: move function building obsmarker-part in the bundle2 module Pierre-Yves David <pierre-yves.david@octobus.net> parents: 32387 diff changeset	1627 def buildobsmarkerspart(bundler, markers):
e70d6dbde713 bundle2: move function building obsmarker-part in the bundle2 module Pierre-Yves David <pierre-yves.david@octobus.net> parents: 32387 diff changeset	1628 """add an obsmarker part to the bundler with <markers>
e70d6dbde713 bundle2: move function building obsmarker-part in the bundle2 module Pierre-Yves David <pierre-yves.david@octobus.net> parents: 32387 diff changeset	1629
e70d6dbde713 bundle2: move function building obsmarker-part in the bundle2 module Pierre-Yves David <pierre-yves.david@octobus.net> parents: 32387 diff changeset	1630 No part is created if markers is empty.
e70d6dbde713 bundle2: move function building obsmarker-part in the bundle2 module Pierre-Yves David <pierre-yves.david@octobus.net> parents: 32387 diff changeset	1631 Raises ValueError if the bundler doesn't support any known obsmarker format.
e70d6dbde713 bundle2: move function building obsmarker-part in the bundle2 module Pierre-Yves David <pierre-yves.david@octobus.net> parents: 32387 diff changeset	1632 """
e70d6dbde713 bundle2: move function building obsmarker-part in the bundle2 module Pierre-Yves David <pierre-yves.david@octobus.net> parents: 32387 diff changeset	1633 if not markers:
e70d6dbde713 bundle2: move function building obsmarker-part in the bundle2 module Pierre-Yves David <pierre-yves.david@octobus.net> parents: 32387 diff changeset	1634 return None
e70d6dbde713 bundle2: move function building obsmarker-part in the bundle2 module Pierre-Yves David <pierre-yves.david@octobus.net> parents: 32387 diff changeset	1635
e70d6dbde713 bundle2: move function building obsmarker-part in the bundle2 module Pierre-Yves David <pierre-yves.david@octobus.net> parents: 32387 diff changeset	1636 remoteversions = obsmarkersversion(bundler.capabilities)
e70d6dbde713 bundle2: move function building obsmarker-part in the bundle2 module Pierre-Yves David <pierre-yves.david@octobus.net> parents: 32387 diff changeset	1637 version = obsolete.commonversion(remoteversions)
e70d6dbde713 bundle2: move function building obsmarker-part in the bundle2 module Pierre-Yves David <pierre-yves.david@octobus.net> parents: 32387 diff changeset	1638 if version is None:
e70d6dbde713 bundle2: move function building obsmarker-part in the bundle2 module Pierre-Yves David <pierre-yves.david@octobus.net> parents: 32387 diff changeset	1639 raise ValueError('bundler does not support common obsmarker format')
e70d6dbde713 bundle2: move function building obsmarker-part in the bundle2 module Pierre-Yves David <pierre-yves.david@octobus.net> parents: 32387 diff changeset	1640 stream = obsolete.encodemarkers(markers, True, version=version)
e70d6dbde713 bundle2: move function building obsmarker-part in the bundle2 module Pierre-Yves David <pierre-yves.david@octobus.net> parents: 32387 diff changeset	1641 return bundler.newpart('obsmarkers', data=stream)
e70d6dbde713 bundle2: move function building obsmarker-part in the bundle2 module Pierre-Yves David <pierre-yves.david@octobus.net> parents: 32387 diff changeset	1642
30757 511a4bf52754 bundle2: allow compression options to be passed to compressor Gregory Szorc <gregory.szorc@gmail.com> parents: 30439 diff changeset	1643 def writebundle(ui, cg, filename, bundletype, vfs=None, compression=None,
511a4bf52754 bundle2: allow compression options to be passed to compressor Gregory Szorc <gregory.szorc@gmail.com> parents: 30439 diff changeset	1644 compopts=None):
28666 ae53ecc47414 bundle: move writebundle() from changegroup.py to bundle2.py (API) Martin von Zweigbergk <martinvonz@google.com> parents: 27953 diff changeset	1645 """Write a bundle file and return its filename.
ae53ecc47414 bundle: move writebundle() from changegroup.py to bundle2.py (API) Martin von Zweigbergk <martinvonz@google.com> parents: 27953 diff changeset	1646
ae53ecc47414 bundle: move writebundle() from changegroup.py to bundle2.py (API) Martin von Zweigbergk <martinvonz@google.com> parents: 27953 diff changeset	1647 Existing files will not be overwritten.
ae53ecc47414 bundle: move writebundle() from changegroup.py to bundle2.py (API) Martin von Zweigbergk <martinvonz@google.com> parents: 27953 diff changeset	1648 If no filename is specified, a temporary file is created.
ae53ecc47414 bundle: move writebundle() from changegroup.py to bundle2.py (API) Martin von Zweigbergk <martinvonz@google.com> parents: 27953 diff changeset	1649 bz2 compression can be turned off.
ae53ecc47414 bundle: move writebundle() from changegroup.py to bundle2.py (API) Martin von Zweigbergk <martinvonz@google.com> parents: 27953 diff changeset	1650 The bundle file will be deleted in case of errors.
ae53ecc47414 bundle: move writebundle() from changegroup.py to bundle2.py (API) Martin von Zweigbergk <martinvonz@google.com> parents: 27953 diff changeset	1651 """
ae53ecc47414 bundle: move writebundle() from changegroup.py to bundle2.py (API) Martin von Zweigbergk <martinvonz@google.com> parents: 27953 diff changeset	1652
ae53ecc47414 bundle: move writebundle() from changegroup.py to bundle2.py (API) Martin von Zweigbergk <martinvonz@google.com> parents: 27953 diff changeset	1653 if bundletype == "HG20":
ae53ecc47414 bundle: move writebundle() from changegroup.py to bundle2.py (API) Martin von Zweigbergk <martinvonz@google.com> parents: 27953 diff changeset	1654 bundle = bundle20(ui)
30757 511a4bf52754 bundle2: allow compression options to be passed to compressor Gregory Szorc <gregory.szorc@gmail.com> parents: 30439 diff changeset	1655 bundle.setcompression(compression, compopts)
28666 ae53ecc47414 bundle: move writebundle() from changegroup.py to bundle2.py (API) Martin von Zweigbergk <martinvonz@google.com> parents: 27953 diff changeset	1656 part = bundle.newpart('changegroup', data=cg.getchunks())
ae53ecc47414 bundle: move writebundle() from changegroup.py to bundle2.py (API) Martin von Zweigbergk <martinvonz@google.com> parents: 27953 diff changeset	1657 part.addparam('version', cg.version)
29593 953839de96ab bundle2: store changeset count when creating file bundles Gregory Szorc <gregory.szorc@gmail.com> parents: 29591 diff changeset	1658 if 'clcount' in cg.extras:
34221 8e0358024a36 bundles: turn nbchanges int into a bytestr using pycompat.bytestr Augie Fackler <raf@durin42.com> parents: 34152 diff changeset	1659 part.addparam('nbchanges', '%d' % cg.extras['clcount'],
29593 953839de96ab bundle2: store changeset count when creating file bundles Gregory Szorc <gregory.szorc@gmail.com> parents: 29591 diff changeset	1660 mandatory=False)
28666 ae53ecc47414 bundle: move writebundle() from changegroup.py to bundle2.py (API) Martin von Zweigbergk <martinvonz@google.com> parents: 27953 diff changeset	1661 chunkiter = bundle.getchunks()
ae53ecc47414 bundle: move writebundle() from changegroup.py to bundle2.py (API) Martin von Zweigbergk <martinvonz@google.com> parents: 27953 diff changeset	1662 else:
ae53ecc47414 bundle: move writebundle() from changegroup.py to bundle2.py (API) Martin von Zweigbergk <martinvonz@google.com> parents: 27953 diff changeset	1663 # compression argument is only for the bundle2 case
ae53ecc47414 bundle: move writebundle() from changegroup.py to bundle2.py (API) Martin von Zweigbergk <martinvonz@google.com> parents: 27953 diff changeset	1664 assert compression is None
ae53ecc47414 bundle: move writebundle() from changegroup.py to bundle2.py (API) Martin von Zweigbergk <martinvonz@google.com> parents: 27953 diff changeset	1665 if cg.version != '01':
ae53ecc47414 bundle: move writebundle() from changegroup.py to bundle2.py (API) Martin von Zweigbergk <martinvonz@google.com> parents: 27953 diff changeset	1666 raise error.Abort(_('old bundle types only supports v1 '
ae53ecc47414 bundle: move writebundle() from changegroup.py to bundle2.py (API) Martin von Zweigbergk <martinvonz@google.com> parents: 27953 diff changeset	1667 'changegroups'))
ae53ecc47414 bundle: move writebundle() from changegroup.py to bundle2.py (API) Martin von Zweigbergk <martinvonz@google.com> parents: 27953 diff changeset	1668 header, comp = bundletypes[bundletype]
30351 f81002f736d7 bundle2: use new compression engine API for compression Gregory Szorc <gregory.szorc@gmail.com> parents: 30177 diff changeset	1669 if comp not in util.compengines.supportedbundletypes:
28666 ae53ecc47414 bundle: move writebundle() from changegroup.py to bundle2.py (API) Martin von Zweigbergk <martinvonz@google.com> parents: 27953 diff changeset	1670 raise error.Abort(_('unknown stream compression type: %s')
ae53ecc47414 bundle: move writebundle() from changegroup.py to bundle2.py (API) Martin von Zweigbergk <martinvonz@google.com> parents: 27953 diff changeset	1671 % comp)
30351 f81002f736d7 bundle2: use new compression engine API for compression Gregory Szorc <gregory.szorc@gmail.com> parents: 30177 diff changeset	1672 compengine = util.compengines.forbundletype(comp)
28666 ae53ecc47414 bundle: move writebundle() from changegroup.py to bundle2.py (API) Martin von Zweigbergk <martinvonz@google.com> parents: 27953 diff changeset	1673 def chunkiter():
ae53ecc47414 bundle: move writebundle() from changegroup.py to bundle2.py (API) Martin von Zweigbergk <martinvonz@google.com> parents: 27953 diff changeset	1674 yield header
30757 511a4bf52754 bundle2: allow compression options to be passed to compressor Gregory Szorc <gregory.szorc@gmail.com> parents: 30439 diff changeset	1675 for chunk in compengine.compressstream(cg.getchunks(), compopts):
30357 5925bda42dbd bundle2: use compressstream compression engine API Gregory Szorc <gregory.szorc@gmail.com> parents: 30353 diff changeset	1676 yield chunk
28666 ae53ecc47414 bundle: move writebundle() from changegroup.py to bundle2.py (API) Martin von Zweigbergk <martinvonz@google.com> parents: 27953 diff changeset	1677 chunkiter = chunkiter()
ae53ecc47414 bundle: move writebundle() from changegroup.py to bundle2.py (API) Martin von Zweigbergk <martinvonz@google.com> parents: 27953 diff changeset	1678
ae53ecc47414 bundle: move writebundle() from changegroup.py to bundle2.py (API) Martin von Zweigbergk <martinvonz@google.com> parents: 27953 diff changeset	1679 # parse the changegroup data, otherwise we will block
ae53ecc47414 bundle: move writebundle() from changegroup.py to bundle2.py (API) Martin von Zweigbergk <martinvonz@google.com> parents: 27953 diff changeset	1680 # in case of sshrepo because we don't know the end of the stream
ae53ecc47414 bundle: move writebundle() from changegroup.py to bundle2.py (API) Martin von Zweigbergk <martinvonz@google.com> parents: 27953 diff changeset	1681 return changegroup.writechunks(ui, chunkiter, filename, vfs=vfs)
ae53ecc47414 bundle: move writebundle() from changegroup.py to bundle2.py (API) Martin von Zweigbergk <martinvonz@google.com> parents: 27953 diff changeset	1682
33037 d765ad56081f bundle: make combinechangegroupresults() take a bundleoperation Martin von Zweigbergk <martinvonz@google.com> parents: 33036 diff changeset	1683 def combinechangegroupresults(op):
33036 52c7060b707a bundle: move combineresults() from changegroup to bundle2 Martin von Zweigbergk <martinvonz@google.com> parents: 33035 diff changeset	1684 """logic to combine 0 or more addchangegroup results into one"""
33037 d765ad56081f bundle: make combinechangegroupresults() take a bundleoperation Martin von Zweigbergk <martinvonz@google.com> parents: 33036 diff changeset	1685 results = [r.get('return', 0)
d765ad56081f bundle: make combinechangegroupresults() take a bundleoperation Martin von Zweigbergk <martinvonz@google.com> parents: 33036 diff changeset	1686 for r in op.records['changegroup']]
33036 52c7060b707a bundle: move combineresults() from changegroup to bundle2 Martin von Zweigbergk <martinvonz@google.com> parents: 33035 diff changeset	1687 changedheads = 0
52c7060b707a bundle: move combineresults() from changegroup to bundle2 Martin von Zweigbergk <martinvonz@google.com> parents: 33035 diff changeset	1688 result = 1
52c7060b707a bundle: move combineresults() from changegroup to bundle2 Martin von Zweigbergk <martinvonz@google.com> parents: 33035 diff changeset	1689 for ret in results:
52c7060b707a bundle: move combineresults() from changegroup to bundle2 Martin von Zweigbergk <martinvonz@google.com> parents: 33035 diff changeset	1690 # If any changegroup result is 0, return 0
52c7060b707a bundle: move combineresults() from changegroup to bundle2 Martin von Zweigbergk <martinvonz@google.com> parents: 33035 diff changeset	1691 if ret == 0:
52c7060b707a bundle: move combineresults() from changegroup to bundle2 Martin von Zweigbergk <martinvonz@google.com> parents: 33035 diff changeset	1692 result = 0
52c7060b707a bundle: move combineresults() from changegroup to bundle2 Martin von Zweigbergk <martinvonz@google.com> parents: 33035 diff changeset	1693 break
52c7060b707a bundle: move combineresults() from changegroup to bundle2 Martin von Zweigbergk <martinvonz@google.com> parents: 33035 diff changeset	1694 if ret < -1:
52c7060b707a bundle: move combineresults() from changegroup to bundle2 Martin von Zweigbergk <martinvonz@google.com> parents: 33035 diff changeset	1695 changedheads += ret + 1
52c7060b707a bundle: move combineresults() from changegroup to bundle2 Martin von Zweigbergk <martinvonz@google.com> parents: 33035 diff changeset	1696 elif ret > 1:
52c7060b707a bundle: move combineresults() from changegroup to bundle2 Martin von Zweigbergk <martinvonz@google.com> parents: 33035 diff changeset	1697 changedheads += ret - 1
52c7060b707a bundle: move combineresults() from changegroup to bundle2 Martin von Zweigbergk <martinvonz@google.com> parents: 33035 diff changeset	1698 if changedheads > 0:
52c7060b707a bundle: move combineresults() from changegroup to bundle2 Martin von Zweigbergk <martinvonz@google.com> parents: 33035 diff changeset	1699 result = 1 + changedheads
52c7060b707a bundle: move combineresults() from changegroup to bundle2 Martin von Zweigbergk <martinvonz@google.com> parents: 33035 diff changeset	1700 elif changedheads < 0:
52c7060b707a bundle: move combineresults() from changegroup to bundle2 Martin von Zweigbergk <martinvonz@google.com> parents: 33035 diff changeset	1701 result = -1 + changedheads
52c7060b707a bundle: move combineresults() from changegroup to bundle2 Martin von Zweigbergk <martinvonz@google.com> parents: 33035 diff changeset	1702 return result
52c7060b707a bundle: move combineresults() from changegroup to bundle2 Martin von Zweigbergk <martinvonz@google.com> parents: 33035 diff changeset	1703
33407 39d4e5a66f5a bundle2: support the 'targetphase' parameter for the changegroup part Boris Feld <boris.feld@octobus.net> parents: 33252 diff changeset	1704 @parthandler('changegroup', ('version', 'nbchanges', 'treemanifest',
39d4e5a66f5a bundle2: support the 'targetphase' parameter for the changegroup part Boris Feld <boris.feld@octobus.net> parents: 33252 diff changeset	1705 'targetphase'))
20998 93a3c5b58635 bundle2: use reply part to return result of addchangegroup Pierre-Yves David <pierre-yves.david@fb.com> parents: 20997 diff changeset	1706 def handlechangegroup(op, inpart):
20950 c7ceae0faf69 bundle2: first crude version of bundling changeset with bundle2 Pierre-Yves David <pierre-yves.david@fb.com> parents: 20949 diff changeset	1707 """apply a changegroup part on the repo
c7ceae0faf69 bundle2: first crude version of bundling changeset with bundle2 Pierre-Yves David <pierre-yves.david@fb.com> parents: 20949 diff changeset	1708
c7ceae0faf69 bundle2: first crude version of bundling changeset with bundle2 Pierre-Yves David <pierre-yves.david@fb.com> parents: 20949 diff changeset	1709 This is a very early implementation that will massive rework before being
c7ceae0faf69 bundle2: first crude version of bundling changeset with bundle2 Pierre-Yves David <pierre-yves.david@fb.com> parents: 20949 diff changeset	1710 inflicted to any end-user.
c7ceae0faf69 bundle2: first crude version of bundling changeset with bundle2 Pierre-Yves David <pierre-yves.david@fb.com> parents: 20949 diff changeset	1711 """
32930 af31d531dda0 changegroup: let callers pass in transaction to apply() (API) Martin von Zweigbergk <martinvonz@google.com> parents: 32892 diff changeset	1712 tr = op.gettransaction()
23170 02e8f9b60052 bundle2: support a "version" argument in `changegroup` part Pierre-Yves David <pierre-yves.david@fb.com> parents: 23169 diff changeset	1713 unpackerversion = inpart.params.get('version', '01')
02e8f9b60052 bundle2: support a "version" argument in `changegroup` part Pierre-Yves David <pierre-yves.david@fb.com> parents: 23169 diff changeset	1714 # We should raise an appropriate exception here
27751 a40e2f7fe49d changegroup: hide packermap behind methods Martin von Zweigbergk <martinvonz@google.com> parents: 27734 diff changeset	1715 cg = changegroup.getunbundler(unpackerversion, inpart, None)
23001 4df9b5e62f70 bundle2: add a comment about addchangegroup source and url Pierre-Yves David <pierre-yves.david@fb.com> parents: 22953 diff changeset	1716 # the source and url passed here are overwritten by the one contained in
4df9b5e62f70 bundle2: add a comment about addchangegroup source and url Pierre-Yves David <pierre-yves.david@fb.com> parents: 22953 diff changeset	1717 # the transaction.hookargs argument. So 'bundle2' is a placeholder
25518 ca656f3dffd7 bundle2: provide number of changesets information to 'addchangegroup' Pierre-Yves David <pierre-yves.david@fb.com> parents: 25507 diff changeset	1718 nbchangesets = None
ca656f3dffd7 bundle2: provide number of changesets information to 'addchangegroup' Pierre-Yves David <pierre-yves.david@fb.com> parents: 25507 diff changeset	1719 if 'nbchanges' in inpart.params:
ca656f3dffd7 bundle2: provide number of changesets information to 'addchangegroup' Pierre-Yves David <pierre-yves.david@fb.com> parents: 25507 diff changeset	1720 nbchangesets = int(inpart.params.get('nbchanges'))
27734 5c0fd878779c treemanifests: set bundle2 part parameter indicating treemanifest Martin von Zweigbergk <martinvonz@google.com> parents: 26829 diff changeset	1721 if ('treemanifest' in inpart.params and
5c0fd878779c treemanifests: set bundle2 part parameter indicating treemanifest Martin von Zweigbergk <martinvonz@google.com> parents: 26829 diff changeset	1722 'treemanifest' not in op.repo.requirements):
5c0fd878779c treemanifests: set bundle2 part parameter indicating treemanifest Martin von Zweigbergk <martinvonz@google.com> parents: 26829 diff changeset	1723 if len(op.repo.changelog) != 0:
5c0fd878779c treemanifests: set bundle2 part parameter indicating treemanifest Martin von Zweigbergk <martinvonz@google.com> parents: 26829 diff changeset	1724 raise error.Abort(_(
5c0fd878779c treemanifests: set bundle2 part parameter indicating treemanifest Martin von Zweigbergk <martinvonz@google.com> parents: 26829 diff changeset	1725 "bundle contains tree manifests, but local repo is "
5c0fd878779c treemanifests: set bundle2 part parameter indicating treemanifest Martin von Zweigbergk <martinvonz@google.com> parents: 26829 diff changeset	1726 "non-empty and does not use tree manifests"))
5c0fd878779c treemanifests: set bundle2 part parameter indicating treemanifest Martin von Zweigbergk <martinvonz@google.com> parents: 26829 diff changeset	1727 op.repo.requirements.add('treemanifest')
5c0fd878779c treemanifests: set bundle2 part parameter indicating treemanifest Martin von Zweigbergk <martinvonz@google.com> parents: 26829 diff changeset	1728 op.repo._applyopenerreqs()
5c0fd878779c treemanifests: set bundle2 part parameter indicating treemanifest Martin von Zweigbergk <martinvonz@google.com> parents: 26829 diff changeset	1729 op.repo._writerequirements()
33407 39d4e5a66f5a bundle2: support the 'targetphase' parameter for the changegroup part Boris Feld <boris.feld@octobus.net> parents: 33252 diff changeset	1730 extrakwargs = {}
39d4e5a66f5a bundle2: support the 'targetphase' parameter for the changegroup part Boris Feld <boris.feld@octobus.net> parents: 33252 diff changeset	1731 targetphase = inpart.params.get('targetphase')
39d4e5a66f5a bundle2: support the 'targetphase' parameter for the changegroup part Boris Feld <boris.feld@octobus.net> parents: 33252 diff changeset	1732 if targetphase is not None:
36116 b587a889b97e py3: use raw string for key in *kwargs Gregory Szorc <gregory.szorc@gmail.com>* parents: 35813 diff changeset	1733 extrakwargs[r'targetphase'] = int(targetphase)
33038 f0efd2bffe1e bundle: extract _processchangegroup() method Martin von Zweigbergk <martinvonz@google.com> parents: 33037 diff changeset	1734 ret = _processchangegroup(op, cg, tr, 'bundle2', 'bundle2',
33407 39d4e5a66f5a bundle2: support the 'targetphase' parameter for the changegroup part Boris Feld <boris.feld@octobus.net> parents: 33252 diff changeset	1735 expectedtotal=nbchangesets, **extrakwargs)
20998 93a3c5b58635 bundle2: use reply part to return result of addchangegroup Pierre-Yves David <pierre-yves.david@fb.com> parents: 20997 diff changeset	1736 if op.reply is not None:
23139 e53f6b72a0e4 spelling: fixes from proofreading of spell checker issues Mads Kiilerich <madski@unity3d.com> parents: 23067 diff changeset	1737 # This is definitely not the final form of this
20998 93a3c5b58635 bundle2: use reply part to return result of addchangegroup Pierre-Yves David <pierre-yves.david@fb.com> parents: 20997 diff changeset	1738 # return. But one need to start somewhere.
24686 e0e28e910fa3 bundle2: rename format, parts and config to final names Pierre-Yves David <pierre-yves.david@fb.com> parents: 24648 diff changeset	1739 part = op.reply.newpart('reply:changegroup', mandatory=False)
33675 24b7c75c6441 bundle2: use bytestr() instead of str() to convert part id to bytes Augie Fackler <augie@google.com> parents: 33674 diff changeset	1740 part.addparam(
24b7c75c6441 bundle2: use bytestr() instead of str() to convert part id to bytes Augie Fackler <augie@google.com> parents: 33674 diff changeset	1741 'in-reply-to', pycompat.bytestr(inpart.id), mandatory=False)
21606 e55888447958 bundle2: update part creators to ``addparam`` when relevant Pierre-Yves David <pierre-yves.david@fb.com> parents: 21605 diff changeset	1742 part.addparam('return', '%i' % ret, mandatory=False)
21019 3dc09f831a2e bundle2: lazy unbundle of part payload Pierre-Yves David <pierre-yves.david@fb.com> parents: 21016 diff changeset	1743 assert not inpart.read()
20950 c7ceae0faf69 bundle2: first crude version of bundling changeset with bundle2 Pierre-Yves David <pierre-yves.david@fb.com> parents: 20949 diff changeset	1744
23029 149fc8a44184 bundle2: client side support for a part to import external bundles Mike Hommey <mh@glandium.org> parents: 23011 diff changeset	1745 _remotechangegroupparams = tuple(['url', 'size', 'digests'] +
149fc8a44184 bundle2: client side support for a part to import external bundles Mike Hommey <mh@glandium.org> parents: 23011 diff changeset	1746 ['digest:%s' % k for k in util.DIGESTS.keys()])
24686 e0e28e910fa3 bundle2: rename format, parts and config to final names Pierre-Yves David <pierre-yves.david@fb.com> parents: 24648 diff changeset	1747 @parthandler('remote-changegroup', _remotechangegroupparams)
23029 149fc8a44184 bundle2: client side support for a part to import external bundles Mike Hommey <mh@glandium.org> parents: 23011 diff changeset	1748 def handleremotechangegroup(op, inpart):
149fc8a44184 bundle2: client side support for a part to import external bundles Mike Hommey <mh@glandium.org> parents: 23011 diff changeset	1749 """apply a bundle10 on the repo, given an url and validation information
149fc8a44184 bundle2: client side support for a part to import external bundles Mike Hommey <mh@glandium.org> parents: 23011 diff changeset	1750
149fc8a44184 bundle2: client side support for a part to import external bundles Mike Hommey <mh@glandium.org> parents: 23011 diff changeset	1751 All the information about the remote bundle to import are given as
149fc8a44184 bundle2: client side support for a part to import external bundles Mike Hommey <mh@glandium.org> parents: 23011 diff changeset	1752 parameters. The parameters include:
149fc8a44184 bundle2: client side support for a part to import external bundles Mike Hommey <mh@glandium.org> parents: 23011 diff changeset	1753 - url: the url to the bundle10.
149fc8a44184 bundle2: client side support for a part to import external bundles Mike Hommey <mh@glandium.org> parents: 23011 diff changeset	1754 - size: the bundle10 file size. It is used to validate what was
149fc8a44184 bundle2: client side support for a part to import external bundles Mike Hommey <mh@glandium.org> parents: 23011 diff changeset	1755 retrieved by the client matches the server knowledge about the bundle.
149fc8a44184 bundle2: client side support for a part to import external bundles Mike Hommey <mh@glandium.org> parents: 23011 diff changeset	1756 - digests: a space separated list of the digest types provided as
149fc8a44184 bundle2: client side support for a part to import external bundles Mike Hommey <mh@glandium.org> parents: 23011 diff changeset	1757 parameters.
149fc8a44184 bundle2: client side support for a part to import external bundles Mike Hommey <mh@glandium.org> parents: 23011 diff changeset	1758 - digest:<digest-type>: the hexadecimal representation of the digest with
149fc8a44184 bundle2: client side support for a part to import external bundles Mike Hommey <mh@glandium.org> parents: 23011 diff changeset	1759 that name. Like the size, it is used to validate what was retrieved by
149fc8a44184 bundle2: client side support for a part to import external bundles Mike Hommey <mh@glandium.org> parents: 23011 diff changeset	1760 the client matches what the server knows about the bundle.
149fc8a44184 bundle2: client side support for a part to import external bundles Mike Hommey <mh@glandium.org> parents: 23011 diff changeset	1761
149fc8a44184 bundle2: client side support for a part to import external bundles Mike Hommey <mh@glandium.org> parents: 23011 diff changeset	1762 When multiple digest types are given, all of them are checked.
149fc8a44184 bundle2: client side support for a part to import external bundles Mike Hommey <mh@glandium.org> parents: 23011 diff changeset	1763 """
149fc8a44184 bundle2: client side support for a part to import external bundles Mike Hommey <mh@glandium.org> parents: 23011 diff changeset	1764 try:
149fc8a44184 bundle2: client side support for a part to import external bundles Mike Hommey <mh@glandium.org> parents: 23011 diff changeset	1765 raw_url = inpart.params['url']
149fc8a44184 bundle2: client side support for a part to import external bundles Mike Hommey <mh@glandium.org> parents: 23011 diff changeset	1766 except KeyError:
26587 56b2bcea2529 error: get Abort from 'error' instead of 'util' Pierre-Yves David <pierre-yves.david@fb.com> parents: 26565 diff changeset	1767 raise error.Abort(_('remote-changegroup: missing "%s" param') % 'url')
23029 149fc8a44184 bundle2: client side support for a part to import external bundles Mike Hommey <mh@glandium.org> parents: 23011 diff changeset	1768 parsed_url = util.url(raw_url)
24686 e0e28e910fa3 bundle2: rename format, parts and config to final names Pierre-Yves David <pierre-yves.david@fb.com> parents: 24648 diff changeset	1769 if parsed_url.scheme not in capabilities['remote-changegroup']:
26587 56b2bcea2529 error: get Abort from 'error' instead of 'util' Pierre-Yves David <pierre-yves.david@fb.com> parents: 26565 diff changeset	1770 raise error.Abort(_('remote-changegroup does not support %s urls') %
23029 149fc8a44184 bundle2: client side support for a part to import external bundles Mike Hommey <mh@glandium.org> parents: 23011 diff changeset	1771 parsed_url.scheme)
149fc8a44184 bundle2: client side support for a part to import external bundles Mike Hommey <mh@glandium.org> parents: 23011 diff changeset	1772
149fc8a44184 bundle2: client side support for a part to import external bundles Mike Hommey <mh@glandium.org> parents: 23011 diff changeset	1773 try:
149fc8a44184 bundle2: client side support for a part to import external bundles Mike Hommey <mh@glandium.org> parents: 23011 diff changeset	1774 size = int(inpart.params['size'])
149fc8a44184 bundle2: client side support for a part to import external bundles Mike Hommey <mh@glandium.org> parents: 23011 diff changeset	1775 except ValueError:
26587 56b2bcea2529 error: get Abort from 'error' instead of 'util' Pierre-Yves David <pierre-yves.david@fb.com> parents: 26565 diff changeset	1776 raise error.Abort(_('remote-changegroup: invalid value for param "%s"')
23029 149fc8a44184 bundle2: client side support for a part to import external bundles Mike Hommey <mh@glandium.org> parents: 23011 diff changeset	1777 % 'size')
149fc8a44184 bundle2: client side support for a part to import external bundles Mike Hommey <mh@glandium.org> parents: 23011 diff changeset	1778 except KeyError:
26587 56b2bcea2529 error: get Abort from 'error' instead of 'util' Pierre-Yves David <pierre-yves.david@fb.com> parents: 26565 diff changeset	1779 raise error.Abort(_('remote-changegroup: missing "%s" param') % 'size')
23029 149fc8a44184 bundle2: client side support for a part to import external bundles Mike Hommey <mh@glandium.org> parents: 23011 diff changeset	1780
149fc8a44184 bundle2: client side support for a part to import external bundles Mike Hommey <mh@glandium.org> parents: 23011 diff changeset	1781 digests = {}
149fc8a44184 bundle2: client side support for a part to import external bundles Mike Hommey <mh@glandium.org> parents: 23011 diff changeset	1782 for typ in inpart.params.get('digests', '').split():
149fc8a44184 bundle2: client side support for a part to import external bundles Mike Hommey <mh@glandium.org> parents: 23011 diff changeset	1783 param = 'digest:%s' % typ
149fc8a44184 bundle2: client side support for a part to import external bundles Mike Hommey <mh@glandium.org> parents: 23011 diff changeset	1784 try:
149fc8a44184 bundle2: client side support for a part to import external bundles Mike Hommey <mh@glandium.org> parents: 23011 diff changeset	1785 value = inpart.params[param]
149fc8a44184 bundle2: client side support for a part to import external bundles Mike Hommey <mh@glandium.org> parents: 23011 diff changeset	1786 except KeyError:
26587 56b2bcea2529 error: get Abort from 'error' instead of 'util' Pierre-Yves David <pierre-yves.david@fb.com> parents: 26565 diff changeset	1787 raise error.Abort(_('remote-changegroup: missing "%s" param') %
23029 149fc8a44184 bundle2: client side support for a part to import external bundles Mike Hommey <mh@glandium.org> parents: 23011 diff changeset	1788 param)
149fc8a44184 bundle2: client side support for a part to import external bundles Mike Hommey <mh@glandium.org> parents: 23011 diff changeset	1789 digests[typ] = value
149fc8a44184 bundle2: client side support for a part to import external bundles Mike Hommey <mh@glandium.org> parents: 23011 diff changeset	1790
149fc8a44184 bundle2: client side support for a part to import external bundles Mike Hommey <mh@glandium.org> parents: 23011 diff changeset	1791 real_part = util.digestchecker(url.open(op.ui, raw_url), size, digests)
149fc8a44184 bundle2: client side support for a part to import external bundles Mike Hommey <mh@glandium.org> parents: 23011 diff changeset	1792
32930 af31d531dda0 changegroup: let callers pass in transaction to apply() (API) Martin von Zweigbergk <martinvonz@google.com> parents: 32892 diff changeset	1793 tr = op.gettransaction()
25919 8221fefaea08 bundle2: use absolute_import Gregory Szorc <gregory.szorc@gmail.com> parents: 25660 diff changeset	1794 from . import exchange
23029 149fc8a44184 bundle2: client side support for a part to import external bundles Mike Hommey <mh@glandium.org> parents: 23011 diff changeset	1795 cg = exchange.readbundle(op.repo.ui, real_part, raw_url)
149fc8a44184 bundle2: client side support for a part to import external bundles Mike Hommey <mh@glandium.org> parents: 23011 diff changeset	1796 if not isinstance(cg, changegroup.cg1unpacker):
26587 56b2bcea2529 error: get Abort from 'error' instead of 'util' Pierre-Yves David <pierre-yves.david@fb.com> parents: 26565 diff changeset	1797 raise error.Abort(_('%s: not a bundle version 1.0') %
23029 149fc8a44184 bundle2: client side support for a part to import external bundles Mike Hommey <mh@glandium.org> parents: 23011 diff changeset	1798 util.hidepassword(raw_url))
33038 f0efd2bffe1e bundle: extract _processchangegroup() method Martin von Zweigbergk <martinvonz@google.com> parents: 33037 diff changeset	1799 ret = _processchangegroup(op, cg, tr, 'bundle2', 'bundle2')
23029 149fc8a44184 bundle2: client side support for a part to import external bundles Mike Hommey <mh@glandium.org> parents: 23011 diff changeset	1800 if op.reply is not None:
23139 e53f6b72a0e4 spelling: fixes from proofreading of spell checker issues Mads Kiilerich <madski@unity3d.com> parents: 23067 diff changeset	1801 # This is definitely not the final form of this
23029 149fc8a44184 bundle2: client side support for a part to import external bundles Mike Hommey <mh@glandium.org> parents: 23011 diff changeset	1802 # return. But one need to start somewhere.
24686 e0e28e910fa3 bundle2: rename format, parts and config to final names Pierre-Yves David <pierre-yves.david@fb.com> parents: 24648 diff changeset	1803 part = op.reply.newpart('reply:changegroup')
33675 24b7c75c6441 bundle2: use bytestr() instead of str() to convert part id to bytes Augie Fackler <augie@google.com> parents: 33674 diff changeset	1804 part.addparam(
24b7c75c6441 bundle2: use bytestr() instead of str() to convert part id to bytes Augie Fackler <augie@google.com> parents: 33674 diff changeset	1805 'in-reply-to', pycompat.bytestr(inpart.id), mandatory=False)
23029 149fc8a44184 bundle2: client side support for a part to import external bundles Mike Hommey <mh@glandium.org> parents: 23011 diff changeset	1806 part.addparam('return', '%i' % ret, mandatory=False)
149fc8a44184 bundle2: client side support for a part to import external bundles Mike Hommey <mh@glandium.org> parents: 23011 diff changeset	1807 try:
149fc8a44184 bundle2: client side support for a part to import external bundles Mike Hommey <mh@glandium.org> parents: 23011 diff changeset	1808 real_part.validate()
26587 56b2bcea2529 error: get Abort from 'error' instead of 'util' Pierre-Yves David <pierre-yves.david@fb.com> parents: 26565 diff changeset	1809 except error.Abort as e:
56b2bcea2529 error: get Abort from 'error' instead of 'util' Pierre-Yves David <pierre-yves.david@fb.com> parents: 26565 diff changeset	1810 raise error.Abort(_('bundle at %s is corrupted:\n%s') %
23029 149fc8a44184 bundle2: client side support for a part to import external bundles Mike Hommey <mh@glandium.org> parents: 23011 diff changeset	1811 (util.hidepassword(raw_url), str(e)))
149fc8a44184 bundle2: client side support for a part to import external bundles Mike Hommey <mh@glandium.org> parents: 23011 diff changeset	1812 assert not inpart.read()
149fc8a44184 bundle2: client side support for a part to import external bundles Mike Hommey <mh@glandium.org> parents: 23011 diff changeset	1813
24686 e0e28e910fa3 bundle2: rename format, parts and config to final names Pierre-Yves David <pierre-yves.david@fb.com> parents: 24648 diff changeset	1814 @parthandler('reply:changegroup', ('return', 'in-reply-to'))
22548 8a1ae133770a bundle2: rename functions that have the same name Mike Hommey <mh@glandium.org> parents: 22390 diff changeset	1815 def handlereplychangegroup(op, inpart):
21611 71b7b3f79a3c bundle2: use the new ``part.params`` dictionary Pierre-Yves David <pierre-yves.david@fb.com> parents: 21610 diff changeset	1816 ret = int(inpart.params['return'])
71b7b3f79a3c bundle2: use the new ``part.params`` dictionary Pierre-Yves David <pierre-yves.david@fb.com> parents: 21610 diff changeset	1817 replyto = int(inpart.params['in-reply-to'])
71b7b3f79a3c bundle2: use the new ``part.params`` dictionary Pierre-Yves David <pierre-yves.david@fb.com> parents: 21610 diff changeset	1818 op.records.add('changegroup', {'return': ret}, replyto)
20950 c7ceae0faf69 bundle2: first crude version of bundling changeset with bundle2 Pierre-Yves David <pierre-yves.david@fb.com> parents: 20949 diff changeset	1819
35258 dbf868623daf bookmark: add a 'check:bookmarks' bundle2 part Boris Feld <boris.feld@octobus.net> parents: 35120 diff changeset	1820 @parthandler('check:bookmarks')
dbf868623daf bookmark: add a 'check:bookmarks' bundle2 part Boris Feld <boris.feld@octobus.net> parents: 35120 diff changeset	1821 def handlecheckbookmarks(op, inpart):
dbf868623daf bookmark: add a 'check:bookmarks' bundle2 part Boris Feld <boris.feld@octobus.net> parents: 35120 diff changeset	1822 """check location of bookmarks
dbf868623daf bookmark: add a 'check:bookmarks' bundle2 part Boris Feld <boris.feld@octobus.net> parents: 35120 diff changeset	1823
dbf868623daf bookmark: add a 'check:bookmarks' bundle2 part Boris Feld <boris.feld@octobus.net> parents: 35120 diff changeset	1824 This part is to be used to detect push race regarding bookmark, it
dbf868623daf bookmark: add a 'check:bookmarks' bundle2 part Boris Feld <boris.feld@octobus.net> parents: 35120 diff changeset	1825 contains binary encoded (bookmark, node) tuple. If the local state does
dbf868623daf bookmark: add a 'check:bookmarks' bundle2 part Boris Feld <boris.feld@octobus.net> parents: 35120 diff changeset	1826 not marks the one in the part, a PushRaced exception is raised
dbf868623daf bookmark: add a 'check:bookmarks' bundle2 part Boris Feld <boris.feld@octobus.net> parents: 35120 diff changeset	1827 """
dbf868623daf bookmark: add a 'check:bookmarks' bundle2 part Boris Feld <boris.feld@octobus.net> parents: 35120 diff changeset	1828 bookdata = bookmarks.binarydecode(inpart)
dbf868623daf bookmark: add a 'check:bookmarks' bundle2 part Boris Feld <boris.feld@octobus.net> parents: 35120 diff changeset	1829
dbf868623daf bookmark: add a 'check:bookmarks' bundle2 part Boris Feld <boris.feld@octobus.net> parents: 35120 diff changeset	1830 msgstandard = ('repository changed while pushing - please try again '
dbf868623daf bookmark: add a 'check:bookmarks' bundle2 part Boris Feld <boris.feld@octobus.net> parents: 35120 diff changeset	1831 '(bookmark "%s" move from %s to %s)')
dbf868623daf bookmark: add a 'check:bookmarks' bundle2 part Boris Feld <boris.feld@octobus.net> parents: 35120 diff changeset	1832 msgmissing = ('repository changed while pushing - please try again '
dbf868623daf bookmark: add a 'check:bookmarks' bundle2 part Boris Feld <boris.feld@octobus.net> parents: 35120 diff changeset	1833 '(bookmark "%s" is missing, expected %s)')
dbf868623daf bookmark: add a 'check:bookmarks' bundle2 part Boris Feld <boris.feld@octobus.net> parents: 35120 diff changeset	1834 msgexist = ('repository changed while pushing - please try again '
dbf868623daf bookmark: add a 'check:bookmarks' bundle2 part Boris Feld <boris.feld@octobus.net> parents: 35120 diff changeset	1835 '(bookmark "%s" set on %s, expected missing)')
dbf868623daf bookmark: add a 'check:bookmarks' bundle2 part Boris Feld <boris.feld@octobus.net> parents: 35120 diff changeset	1836 for book, node in bookdata:
dbf868623daf bookmark: add a 'check:bookmarks' bundle2 part Boris Feld <boris.feld@octobus.net> parents: 35120 diff changeset	1837 currentnode = op.repo._bookmarks.get(book)
dbf868623daf bookmark: add a 'check:bookmarks' bundle2 part Boris Feld <boris.feld@octobus.net> parents: 35120 diff changeset	1838 if currentnode != node:
dbf868623daf bookmark: add a 'check:bookmarks' bundle2 part Boris Feld <boris.feld@octobus.net> parents: 35120 diff changeset	1839 if node is None:
dbf868623daf bookmark: add a 'check:bookmarks' bundle2 part Boris Feld <boris.feld@octobus.net> parents: 35120 diff changeset	1840 finalmsg = msgexist % (book, nodemod.short(currentnode))
dbf868623daf bookmark: add a 'check:bookmarks' bundle2 part Boris Feld <boris.feld@octobus.net> parents: 35120 diff changeset	1841 elif currentnode is None:
dbf868623daf bookmark: add a 'check:bookmarks' bundle2 part Boris Feld <boris.feld@octobus.net> parents: 35120 diff changeset	1842 finalmsg = msgmissing % (book, nodemod.short(node))
dbf868623daf bookmark: add a 'check:bookmarks' bundle2 part Boris Feld <boris.feld@octobus.net> parents: 35120 diff changeset	1843 else:
dbf868623daf bookmark: add a 'check:bookmarks' bundle2 part Boris Feld <boris.feld@octobus.net> parents: 35120 diff changeset	1844 finalmsg = msgstandard % (book, nodemod.short(node),
dbf868623daf bookmark: add a 'check:bookmarks' bundle2 part Boris Feld <boris.feld@octobus.net> parents: 35120 diff changeset	1845 nodemod.short(currentnode))
dbf868623daf bookmark: add a 'check:bookmarks' bundle2 part Boris Feld <boris.feld@octobus.net> parents: 35120 diff changeset	1846 raise error.PushRaced(finalmsg)
dbf868623daf bookmark: add a 'check:bookmarks' bundle2 part Boris Feld <boris.feld@octobus.net> parents: 35120 diff changeset	1847
24686 e0e28e910fa3 bundle2: rename format, parts and config to final names Pierre-Yves David <pierre-yves.david@fb.com> parents: 24648 diff changeset	1848 @parthandler('check:heads')
22548 8a1ae133770a bundle2: rename functions that have the same name Mike Hommey <mh@glandium.org> parents: 22390 diff changeset	1849 def handlecheckheads(op, inpart):
21060 0bea9db7543b bundle2: add a "check:heads" handler Pierre-Yves David <pierre-yves.david@fb.com> parents: 21024 diff changeset	1850 """check that head of the repo did not change
0bea9db7543b bundle2: add a "check:heads" handler Pierre-Yves David <pierre-yves.david@fb.com> parents: 21024 diff changeset	1851
0bea9db7543b bundle2: add a "check:heads" handler Pierre-Yves David <pierre-yves.david@fb.com> parents: 21024 diff changeset	1852 This is used to detect a push race when using unbundle.
0bea9db7543b bundle2: add a "check:heads" handler Pierre-Yves David <pierre-yves.david@fb.com> parents: 21024 diff changeset	1853 This replaces the "heads" argument of unbundle."""
0bea9db7543b bundle2: add a "check:heads" handler Pierre-Yves David <pierre-yves.david@fb.com> parents: 21024 diff changeset	1854 h = inpart.read(20)
0bea9db7543b bundle2: add a "check:heads" handler Pierre-Yves David <pierre-yves.david@fb.com> parents: 21024 diff changeset	1855 heads = []
0bea9db7543b bundle2: add a "check:heads" handler Pierre-Yves David <pierre-yves.david@fb.com> parents: 21024 diff changeset	1856 while len(h) == 20:
0bea9db7543b bundle2: add a "check:heads" handler Pierre-Yves David <pierre-yves.david@fb.com> parents: 21024 diff changeset	1857 heads.append(h)
0bea9db7543b bundle2: add a "check:heads" handler Pierre-Yves David <pierre-yves.david@fb.com> parents: 21024 diff changeset	1858 h = inpart.read(20)
0bea9db7543b bundle2: add a "check:heads" handler Pierre-Yves David <pierre-yves.david@fb.com> parents: 21024 diff changeset	1859 assert not h
26565 ee1bcb9aa6e4 bundle2: add op.gettransaction() to handlers that need the lock Durham Goode <durham@fb.com> parents: 26542 diff changeset	1860 # Trigger a transaction so that we are guaranteed to have the lock now.
ee1bcb9aa6e4 bundle2: add op.gettransaction() to handlers that need the lock Durham Goode <durham@fb.com> parents: 26542 diff changeset	1861 if op.ui.configbool('experimental', 'bundle2lazylocking'):
ee1bcb9aa6e4 bundle2: add op.gettransaction() to handlers that need the lock Durham Goode <durham@fb.com> parents: 26542 diff changeset	1862 op.gettransaction()
29294 077d0535f51f bundle2: don't assume ordering of heads checked after push Mads Kiilerich <madski@unity3d.com> parents: 28883 diff changeset	1863 if sorted(heads) != sorted(op.repo.heads()):
21185 5b3717e1a3ea bundle2: add an error message to push race error Pierre-Yves David <pierre-yves.david@fb.com> parents: 21184 diff changeset	1864 raise error.PushRaced('repository changed while pushing - '
5b3717e1a3ea bundle2: add an error message to push race error Pierre-Yves David <pierre-yves.david@fb.com> parents: 21184 diff changeset	1865 'please try again')
21130 1ff06386217f bundle2: introduce `replycaps` part for on-demand reply Pierre-Yves David <pierre-yves.david@fb.com> parents: 21129 diff changeset	1866
32709 16ada4cbb1a9 push: add a way to allow concurrent pushes on unrelated heads Pierre-Yves David <pierre-yves.david@octobus.net> parents: 32516 diff changeset	1867 @parthandler('check:updated-heads')
16ada4cbb1a9 push: add a way to allow concurrent pushes on unrelated heads Pierre-Yves David <pierre-yves.david@octobus.net> parents: 32516 diff changeset	1868 def handlecheckupdatedheads(op, inpart):
16ada4cbb1a9 push: add a way to allow concurrent pushes on unrelated heads Pierre-Yves David <pierre-yves.david@octobus.net> parents: 32516 diff changeset	1869 """check for race on the heads touched by a push
16ada4cbb1a9 push: add a way to allow concurrent pushes on unrelated heads Pierre-Yves David <pierre-yves.david@octobus.net> parents: 32516 diff changeset	1870
16ada4cbb1a9 push: add a way to allow concurrent pushes on unrelated heads Pierre-Yves David <pierre-yves.david@octobus.net> parents: 32516 diff changeset	1871 This is similar to 'check:heads' but focus on the heads actually updated
16ada4cbb1a9 push: add a way to allow concurrent pushes on unrelated heads Pierre-Yves David <pierre-yves.david@octobus.net> parents: 32516 diff changeset	1872 during the push. If other activities happen on unrelated heads, it is
16ada4cbb1a9 push: add a way to allow concurrent pushes on unrelated heads Pierre-Yves David <pierre-yves.david@octobus.net> parents: 32516 diff changeset	1873 ignored.
16ada4cbb1a9 push: add a way to allow concurrent pushes on unrelated heads Pierre-Yves David <pierre-yves.david@octobus.net> parents: 32516 diff changeset	1874
16ada4cbb1a9 push: add a way to allow concurrent pushes on unrelated heads Pierre-Yves David <pierre-yves.david@octobus.net> parents: 32516 diff changeset	1875 This allow server with high traffic to avoid push contention as long as
16ada4cbb1a9 push: add a way to allow concurrent pushes on unrelated heads Pierre-Yves David <pierre-yves.david@octobus.net> parents: 32516 diff changeset	1876 unrelated parts of the graph are involved."""
16ada4cbb1a9 push: add a way to allow concurrent pushes on unrelated heads Pierre-Yves David <pierre-yves.david@octobus.net> parents: 32516 diff changeset	1877 h = inpart.read(20)
16ada4cbb1a9 push: add a way to allow concurrent pushes on unrelated heads Pierre-Yves David <pierre-yves.david@octobus.net> parents: 32516 diff changeset	1878 heads = []
16ada4cbb1a9 push: add a way to allow concurrent pushes on unrelated heads Pierre-Yves David <pierre-yves.david@octobus.net> parents: 32516 diff changeset	1879 while len(h) == 20:
16ada4cbb1a9 push: add a way to allow concurrent pushes on unrelated heads Pierre-Yves David <pierre-yves.david@octobus.net> parents: 32516 diff changeset	1880 heads.append(h)
16ada4cbb1a9 push: add a way to allow concurrent pushes on unrelated heads Pierre-Yves David <pierre-yves.david@octobus.net> parents: 32516 diff changeset	1881 h = inpart.read(20)
16ada4cbb1a9 push: add a way to allow concurrent pushes on unrelated heads Pierre-Yves David <pierre-yves.david@octobus.net> parents: 32516 diff changeset	1882 assert not h
16ada4cbb1a9 push: add a way to allow concurrent pushes on unrelated heads Pierre-Yves David <pierre-yves.david@octobus.net> parents: 32516 diff changeset	1883 # trigger a transaction so that we are guaranteed to have the lock now.
16ada4cbb1a9 push: add a way to allow concurrent pushes on unrelated heads Pierre-Yves David <pierre-yves.david@octobus.net> parents: 32516 diff changeset	1884 if op.ui.configbool('experimental', 'bundle2lazylocking'):
16ada4cbb1a9 push: add a way to allow concurrent pushes on unrelated heads Pierre-Yves David <pierre-yves.david@octobus.net> parents: 32516 diff changeset	1885 op.gettransaction()
16ada4cbb1a9 push: add a way to allow concurrent pushes on unrelated heads Pierre-Yves David <pierre-yves.david@octobus.net> parents: 32516 diff changeset	1886
16ada4cbb1a9 push: add a way to allow concurrent pushes on unrelated heads Pierre-Yves David <pierre-yves.david@octobus.net> parents: 32516 diff changeset	1887 currentheads = set()
16ada4cbb1a9 push: add a way to allow concurrent pushes on unrelated heads Pierre-Yves David <pierre-yves.david@octobus.net> parents: 32516 diff changeset	1888 for ls in op.repo.branchmap().itervalues():
16ada4cbb1a9 push: add a way to allow concurrent pushes on unrelated heads Pierre-Yves David <pierre-yves.david@octobus.net> parents: 32516 diff changeset	1889 currentheads.update(ls)
16ada4cbb1a9 push: add a way to allow concurrent pushes on unrelated heads Pierre-Yves David <pierre-yves.david@octobus.net> parents: 32516 diff changeset	1890
16ada4cbb1a9 push: add a way to allow concurrent pushes on unrelated heads Pierre-Yves David <pierre-yves.david@octobus.net> parents: 32516 diff changeset	1891 for h in heads:
16ada4cbb1a9 push: add a way to allow concurrent pushes on unrelated heads Pierre-Yves David <pierre-yves.david@octobus.net> parents: 32516 diff changeset	1892 if h not in currentheads:
16ada4cbb1a9 push: add a way to allow concurrent pushes on unrelated heads Pierre-Yves David <pierre-yves.david@octobus.net> parents: 32516 diff changeset	1893 raise error.PushRaced('repository changed while pushing - '
16ada4cbb1a9 push: add a way to allow concurrent pushes on unrelated heads Pierre-Yves David <pierre-yves.david@octobus.net> parents: 32516 diff changeset	1894 'please try again')
16ada4cbb1a9 push: add a way to allow concurrent pushes on unrelated heads Pierre-Yves David <pierre-yves.david@octobus.net> parents: 32516 diff changeset	1895
34820 a95067b1dca6 phase: introduce a new 'check:phases' part Boris Feld <boris.feld@octobus.net> parents: 34637 diff changeset	1896 @parthandler('check:phases')
a95067b1dca6 phase: introduce a new 'check:phases' part Boris Feld <boris.feld@octobus.net> parents: 34637 diff changeset	1897 def handlecheckphases(op, inpart):
a95067b1dca6 phase: introduce a new 'check:phases' part Boris Feld <boris.feld@octobus.net> parents: 34637 diff changeset	1898 """check that phase boundaries of the repository did not change
a95067b1dca6 phase: introduce a new 'check:phases' part Boris Feld <boris.feld@octobus.net> parents: 34637 diff changeset	1899
a95067b1dca6 phase: introduce a new 'check:phases' part Boris Feld <boris.feld@octobus.net> parents: 34637 diff changeset	1900 This is used to detect a push race.
a95067b1dca6 phase: introduce a new 'check:phases' part Boris Feld <boris.feld@octobus.net> parents: 34637 diff changeset	1901 """
a95067b1dca6 phase: introduce a new 'check:phases' part Boris Feld <boris.feld@octobus.net> parents: 34637 diff changeset	1902 phasetonodes = phases.binarydecode(inpart)
a95067b1dca6 phase: introduce a new 'check:phases' part Boris Feld <boris.feld@octobus.net> parents: 34637 diff changeset	1903 unfi = op.repo.unfiltered()
a95067b1dca6 phase: introduce a new 'check:phases' part Boris Feld <boris.feld@octobus.net> parents: 34637 diff changeset	1904 cl = unfi.changelog
a95067b1dca6 phase: introduce a new 'check:phases' part Boris Feld <boris.feld@octobus.net> parents: 34637 diff changeset	1905 phasecache = unfi._phasecache
a95067b1dca6 phase: introduce a new 'check:phases' part Boris Feld <boris.feld@octobus.net> parents: 34637 diff changeset	1906 msg = ('repository changed while pushing - please try again '
a95067b1dca6 phase: introduce a new 'check:phases' part Boris Feld <boris.feld@octobus.net> parents: 34637 diff changeset	1907 '(%s is %s expected %s)')
a95067b1dca6 phase: introduce a new 'check:phases' part Boris Feld <boris.feld@octobus.net> parents: 34637 diff changeset	1908 for expectedphase, nodes in enumerate(phasetonodes):
a95067b1dca6 phase: introduce a new 'check:phases' part Boris Feld <boris.feld@octobus.net> parents: 34637 diff changeset	1909 for n in nodes:
a95067b1dca6 phase: introduce a new 'check:phases' part Boris Feld <boris.feld@octobus.net> parents: 34637 diff changeset	1910 actualphase = phasecache.phase(unfi, cl.rev(n))
a95067b1dca6 phase: introduce a new 'check:phases' part Boris Feld <boris.feld@octobus.net> parents: 34637 diff changeset	1911 if actualphase != expectedphase:
a95067b1dca6 phase: introduce a new 'check:phases' part Boris Feld <boris.feld@octobus.net> parents: 34637 diff changeset	1912 finalmsg = msg % (nodemod.short(n),
a95067b1dca6 phase: introduce a new 'check:phases' part Boris Feld <boris.feld@octobus.net> parents: 34637 diff changeset	1913 phases.phasenames[actualphase],
a95067b1dca6 phase: introduce a new 'check:phases' part Boris Feld <boris.feld@octobus.net> parents: 34637 diff changeset	1914 phases.phasenames[expectedphase])
a95067b1dca6 phase: introduce a new 'check:phases' part Boris Feld <boris.feld@octobus.net> parents: 34637 diff changeset	1915 raise error.PushRaced(finalmsg)
a95067b1dca6 phase: introduce a new 'check:phases' part Boris Feld <boris.feld@octobus.net> parents: 34637 diff changeset	1916
24686 e0e28e910fa3 bundle2: rename format, parts and config to final names Pierre-Yves David <pierre-yves.david@fb.com> parents: 24648 diff changeset	1917 @parthandler('output')
21131 b7435117d951 bundle2: capture remote stdout while unbundling Pierre-Yves David <pierre-yves.david@fb.com> parents: 21130 diff changeset	1918 def handleoutput(op, inpart):
b7435117d951 bundle2: capture remote stdout while unbundling Pierre-Yves David <pierre-yves.david@fb.com> parents: 21130 diff changeset	1919 """forward output captured on the server to the client"""
b7435117d951 bundle2: capture remote stdout while unbundling Pierre-Yves David <pierre-yves.david@fb.com> parents: 21130 diff changeset	1920 for line in inpart.read().splitlines():
29851 ccd436f7db6d bundle2: localize handleoutput remote prompts Akihiko Odaki <akihiko.odaki.4i@stu.hosei.ac.jp> parents: 29847 diff changeset	1921 op.ui.status(_('remote: %s\n') % line)
21131 b7435117d951 bundle2: capture remote stdout while unbundling Pierre-Yves David <pierre-yves.david@fb.com> parents: 21130 diff changeset	1922
24686 e0e28e910fa3 bundle2: rename format, parts and config to final names Pierre-Yves David <pierre-yves.david@fb.com> parents: 24648 diff changeset	1923 @parthandler('replycaps')
21130 1ff06386217f bundle2: introduce `replycaps` part for on-demand reply Pierre-Yves David <pierre-yves.david@fb.com> parents: 21129 diff changeset	1924 def handlereplycaps(op, inpart):
1ff06386217f bundle2: introduce `replycaps` part for on-demand reply Pierre-Yves David <pierre-yves.david@fb.com> parents: 21129 diff changeset	1925 """Notify that a reply bundle should be created
1ff06386217f bundle2: introduce `replycaps` part for on-demand reply Pierre-Yves David <pierre-yves.david@fb.com> parents: 21129 diff changeset	1926
21138 f469879d27ec bundle2: extract capabilities decoding Pierre-Yves David <pierre-yves.david@fb.com> parents: 21137 diff changeset	1927 The payload contains the capabilities information for the reply"""
f469879d27ec bundle2: extract capabilities decoding Pierre-Yves David <pierre-yves.david@fb.com> parents: 21137 diff changeset	1928 caps = decodecaps(inpart.read())
21130 1ff06386217f bundle2: introduce `replycaps` part for on-demand reply Pierre-Yves David <pierre-yves.david@fb.com> parents: 21129 diff changeset	1929 if op.reply is None:
21135 98fbf3adfd83 bundle2: add capabilities support in `replycaps` part Pierre-Yves David <pierre-yves.david@fb.com> parents: 21134 diff changeset	1930 op.reply = bundle20(op.ui, caps)
21131 b7435117d951 bundle2: capture remote stdout while unbundling Pierre-Yves David <pierre-yves.david@fb.com> parents: 21130 diff changeset	1931
26829 58f1645f72c3 bundle2: attribute remote failures to remote (issue4788) Gregory Szorc <gregory.szorc@gmail.com> parents: 26802 diff changeset	1932 class AbortFromPart(error.Abort):
58f1645f72c3 bundle2: attribute remote failures to remote (issue4788) Gregory Szorc <gregory.szorc@gmail.com> parents: 26802 diff changeset	1933 """Sub-class of Abort that denotes an error from a bundle2 part."""
58f1645f72c3 bundle2: attribute remote failures to remote (issue4788) Gregory Szorc <gregory.szorc@gmail.com> parents: 26802 diff changeset	1934
24686 e0e28e910fa3 bundle2: rename format, parts and config to final names Pierre-Yves David <pierre-yves.david@fb.com> parents: 24648 diff changeset	1935 @parthandler('error:abort', ('message', 'hint'))
24741 bb67e52362d6 bundle2: fix names for error part handler Pierre-Yves David <pierre-yves.david@fb.com> parents: 24733 diff changeset	1936 def handleerrorabort(op, inpart):
21177 952af771bc17 bundle2: gracefully handle abort during unbundle Pierre-Yves David <pierre-yves.david@fb.com> parents: 21176 diff changeset	1937 """Used to transmit abort error over the wire"""
26829 58f1645f72c3 bundle2: attribute remote failures to remote (issue4788) Gregory Szorc <gregory.szorc@gmail.com> parents: 26802 diff changeset	1938 raise AbortFromPart(inpart.params['message'],
58f1645f72c3 bundle2: attribute remote failures to remote (issue4788) Gregory Szorc <gregory.szorc@gmail.com> parents: 26802 diff changeset	1939 hint=inpart.params.get('hint'))
21183 4345274adc4b bundle2: gracefully handle UnknownPartError during unbundle Pierre-Yves David <pierre-yves.david@fb.com> parents: 21179 diff changeset	1940
25493 d8e7b0781ad7 bundle2: convey PushkeyFailed error over the wire Pierre-Yves David <pierre-yves.david@fb.com> parents: 25492 diff changeset	1941 @parthandler('error:pushkey', ('namespace', 'key', 'new', 'old', 'ret',
d8e7b0781ad7 bundle2: convey PushkeyFailed error over the wire Pierre-Yves David <pierre-yves.david@fb.com> parents: 25492 diff changeset	1942 'in-reply-to'))
d8e7b0781ad7 bundle2: convey PushkeyFailed error over the wire Pierre-Yves David <pierre-yves.david@fb.com> parents: 25492 diff changeset	1943 def handleerrorpushkey(op, inpart):
d8e7b0781ad7 bundle2: convey PushkeyFailed error over the wire Pierre-Yves David <pierre-yves.david@fb.com> parents: 25492 diff changeset	1944 """Used to transmit failure of a mandatory pushkey over the wire"""
d8e7b0781ad7 bundle2: convey PushkeyFailed error over the wire Pierre-Yves David <pierre-yves.david@fb.com> parents: 25492 diff changeset	1945 kwargs = {}
d8e7b0781ad7 bundle2: convey PushkeyFailed error over the wire Pierre-Yves David <pierre-yves.david@fb.com> parents: 25492 diff changeset	1946 for name in ('namespace', 'key', 'new', 'old', 'ret'):
d8e7b0781ad7 bundle2: convey PushkeyFailed error over the wire Pierre-Yves David <pierre-yves.david@fb.com> parents: 25492 diff changeset	1947 value = inpart.params.get(name)
d8e7b0781ad7 bundle2: convey PushkeyFailed error over the wire Pierre-Yves David <pierre-yves.david@fb.com> parents: 25492 diff changeset	1948 if value is not None:
d8e7b0781ad7 bundle2: convey PushkeyFailed error over the wire Pierre-Yves David <pierre-yves.david@fb.com> parents: 25492 diff changeset	1949 kwargs[name] = value
36427 247b473f408e bundle2: *strkwargs love on various kwargs constructions Augie Fackler <augie@google.com>* parents: 36273 diff changeset	1950 raise error.PushkeyFailed(inpart.params['in-reply-to'],
247b473f408e bundle2: *strkwargs love on various kwargs constructions Augie Fackler <augie@google.com>* parents: 36273 diff changeset	1951 **pycompat.strkwargs(kwargs))
25493 d8e7b0781ad7 bundle2: convey PushkeyFailed error over the wire Pierre-Yves David <pierre-yves.david@fb.com> parents: 25492 diff changeset	1952
24686 e0e28e910fa3 bundle2: rename format, parts and config to final names Pierre-Yves David <pierre-yves.david@fb.com> parents: 24648 diff changeset	1953 @parthandler('error:unsupportedcontent', ('parttype', 'params'))
24741 bb67e52362d6 bundle2: fix names for error part handler Pierre-Yves David <pierre-yves.david@fb.com> parents: 24733 diff changeset	1954 def handleerrorunsupportedcontent(op, inpart):
21619 292331e906d7 bundle2: rename b2x:error:unknownpart to b2x:error:unsupportedcontent Pierre-Yves David <pierre-yves.david@fb.com> parents: 21618 diff changeset	1955 """Used to transmit unknown content error over the wire"""
21622 457492741007 bundle2: support transmission of params error over the wire Pierre-Yves David <pierre-yves.david@fb.com> parents: 21620 diff changeset	1956 kwargs = {}
21627 3e8bcc90f07c bundle2: support None parttype in BundleValueError Pierre-Yves David <pierre-yves.david@fb.com> parents: 21626 diff changeset	1957 parttype = inpart.params.get('parttype')
3e8bcc90f07c bundle2: support None parttype in BundleValueError Pierre-Yves David <pierre-yves.david@fb.com> parents: 21626 diff changeset	1958 if parttype is not None:
3e8bcc90f07c bundle2: support None parttype in BundleValueError Pierre-Yves David <pierre-yves.david@fb.com> parents: 21626 diff changeset	1959 kwargs['parttype'] = parttype
21622 457492741007 bundle2: support transmission of params error over the wire Pierre-Yves David <pierre-yves.david@fb.com> parents: 21620 diff changeset	1960 params = inpart.params.get('params')
457492741007 bundle2: support transmission of params error over the wire Pierre-Yves David <pierre-yves.david@fb.com> parents: 21620 diff changeset	1961 if params is not None:
457492741007 bundle2: support transmission of params error over the wire Pierre-Yves David <pierre-yves.david@fb.com> parents: 21620 diff changeset	1962 kwargs['params'] = params.split('\0')
457492741007 bundle2: support transmission of params error over the wire Pierre-Yves David <pierre-yves.david@fb.com> parents: 21620 diff changeset	1963
36427 247b473f408e bundle2: *strkwargs love on various kwargs constructions Augie Fackler <augie@google.com>* parents: 36273 diff changeset	1964 raise error.BundleUnknownFeatureError(**pycompat.strkwargs(kwargs))
21186 9f3652e851f8 bundle2: gracefully handle PushRaced error during unbundle Pierre-Yves David <pierre-yves.david@fb.com> parents: 21185 diff changeset	1965
24686 e0e28e910fa3 bundle2: rename format, parts and config to final names Pierre-Yves David <pierre-yves.david@fb.com> parents: 24648 diff changeset	1966 @parthandler('error:pushraced', ('message',))
24741 bb67e52362d6 bundle2: fix names for error part handler Pierre-Yves David <pierre-yves.david@fb.com> parents: 24733 diff changeset	1967 def handleerrorpushraced(op, inpart):
21186 9f3652e851f8 bundle2: gracefully handle PushRaced error during unbundle Pierre-Yves David <pierre-yves.david@fb.com> parents: 21185 diff changeset	1968 """Used to transmit push race error over the wire"""
21611 71b7b3f79a3c bundle2: use the new ``part.params`` dictionary Pierre-Yves David <pierre-yves.david@fb.com> parents: 21610 diff changeset	1969 raise error.ResponseError(_('push failed:'), inpart.params['message'])
21655 35095f332846 bundle: introduce a listkey handler Pierre-Yves David <pierre-yves.david@fb.com> parents: 21644 diff changeset	1970
24686 e0e28e910fa3 bundle2: rename format, parts and config to final names Pierre-Yves David <pierre-yves.david@fb.com> parents: 24648 diff changeset	1971 @parthandler('listkeys', ('namespace',))
21655 35095f332846 bundle: introduce a listkey handler Pierre-Yves David <pierre-yves.david@fb.com> parents: 21644 diff changeset	1972 def handlelistkeys(op, inpart):
35095f332846 bundle: introduce a listkey handler Pierre-Yves David <pierre-yves.david@fb.com> parents: 21644 diff changeset	1973 """retrieve pushkey namespace content stored in a bundle2"""
35095f332846 bundle: introduce a listkey handler Pierre-Yves David <pierre-yves.david@fb.com> parents: 21644 diff changeset	1974 namespace = inpart.params['namespace']
35095f332846 bundle: introduce a listkey handler Pierre-Yves David <pierre-yves.david@fb.com> parents: 21644 diff changeset	1975 r = pushkey.decodekeys(inpart.read())
35095f332846 bundle: introduce a listkey handler Pierre-Yves David <pierre-yves.david@fb.com> parents: 21644 diff changeset	1976 op.records.add('listkeys', (namespace, r))
21660 e87d2a12d41b bundle2: add ``pushkey`` support Pierre-Yves David <pierre-yves.david@fb.com> parents: 21655 diff changeset	1977
24686 e0e28e910fa3 bundle2: rename format, parts and config to final names Pierre-Yves David <pierre-yves.david@fb.com> parents: 24648 diff changeset	1978 @parthandler('pushkey', ('namespace', 'key', 'old', 'new'))
21660 e87d2a12d41b bundle2: add ``pushkey`` support Pierre-Yves David <pierre-yves.david@fb.com> parents: 21655 diff changeset	1979 def handlepushkey(op, inpart):
e87d2a12d41b bundle2: add ``pushkey`` support Pierre-Yves David <pierre-yves.david@fb.com> parents: 21655 diff changeset	1980 """process a pushkey request"""
e87d2a12d41b bundle2: add ``pushkey`` support Pierre-Yves David <pierre-yves.david@fb.com> parents: 21655 diff changeset	1981 dec = pushkey.decode
e87d2a12d41b bundle2: add ``pushkey`` support Pierre-Yves David <pierre-yves.david@fb.com> parents: 21655 diff changeset	1982 namespace = dec(inpart.params['namespace'])
e87d2a12d41b bundle2: add ``pushkey`` support Pierre-Yves David <pierre-yves.david@fb.com> parents: 21655 diff changeset	1983 key = dec(inpart.params['key'])
e87d2a12d41b bundle2: add ``pushkey`` support Pierre-Yves David <pierre-yves.david@fb.com> parents: 21655 diff changeset	1984 old = dec(inpart.params['old'])
e87d2a12d41b bundle2: add ``pushkey`` support Pierre-Yves David <pierre-yves.david@fb.com> parents: 21655 diff changeset	1985 new = dec(inpart.params['new'])
26565 ee1bcb9aa6e4 bundle2: add op.gettransaction() to handlers that need the lock Durham Goode <durham@fb.com> parents: 26542 diff changeset	1986 # Grab the transaction to ensure that we have the lock before performing the
ee1bcb9aa6e4 bundle2: add op.gettransaction() to handlers that need the lock Durham Goode <durham@fb.com> parents: 26542 diff changeset	1987 # pushkey.
ee1bcb9aa6e4 bundle2: add op.gettransaction() to handlers that need the lock Durham Goode <durham@fb.com> parents: 26542 diff changeset	1988 if op.ui.configbool('experimental', 'bundle2lazylocking'):
ee1bcb9aa6e4 bundle2: add op.gettransaction() to handlers that need the lock Durham Goode <durham@fb.com> parents: 26542 diff changeset	1989 op.gettransaction()
21660 e87d2a12d41b bundle2: add ``pushkey`` support Pierre-Yves David <pierre-yves.david@fb.com> parents: 21655 diff changeset	1990 ret = op.repo.pushkey(namespace, key, old, new)
e87d2a12d41b bundle2: add ``pushkey`` support Pierre-Yves David <pierre-yves.david@fb.com> parents: 21655 diff changeset	1991 record = {'namespace': namespace,
e87d2a12d41b bundle2: add ``pushkey`` support Pierre-Yves David <pierre-yves.david@fb.com> parents: 21655 diff changeset	1992 'key': key,
e87d2a12d41b bundle2: add ``pushkey`` support Pierre-Yves David <pierre-yves.david@fb.com> parents: 21655 diff changeset	1993 'old': old,
e87d2a12d41b bundle2: add ``pushkey`` support Pierre-Yves David <pierre-yves.david@fb.com> parents: 21655 diff changeset	1994 'new': new}
e87d2a12d41b bundle2: add ``pushkey`` support Pierre-Yves David <pierre-yves.david@fb.com> parents: 21655 diff changeset	1995 op.records.add('pushkey', record)
e87d2a12d41b bundle2: add ``pushkey`` support Pierre-Yves David <pierre-yves.david@fb.com> parents: 21655 diff changeset	1996 if op.reply is not None:
24686 e0e28e910fa3 bundle2: rename format, parts and config to final names Pierre-Yves David <pierre-yves.david@fb.com> parents: 24648 diff changeset	1997 rpart = op.reply.newpart('reply:pushkey')
33675 24b7c75c6441 bundle2: use bytestr() instead of str() to convert part id to bytes Augie Fackler <augie@google.com> parents: 33674 diff changeset	1998 rpart.addparam(
24b7c75c6441 bundle2: use bytestr() instead of str() to convert part id to bytes Augie Fackler <augie@google.com> parents: 33674 diff changeset	1999 'in-reply-to', pycompat.bytestr(inpart.id), mandatory=False)
21660 e87d2a12d41b bundle2: add ``pushkey`` support Pierre-Yves David <pierre-yves.david@fb.com> parents: 21655 diff changeset	2000 rpart.addparam('return', '%i' % ret, mandatory=False)
25481 6de96cb31176 bundle2: abort when a mandatory pushkey part fails Pierre-Yves David <pierre-yves.david@fb.com> parents: 25401 diff changeset	2001 if inpart.mandatory and not ret:
25484 a5192774e925 bundle2: introduce a PushkeyFail error to abort unbundle on pushkey error Pierre-Yves David <pierre-yves.david@fb.com> parents: 25481 diff changeset	2002 kwargs = {}
a5192774e925 bundle2: introduce a PushkeyFail error to abort unbundle on pushkey error Pierre-Yves David <pierre-yves.david@fb.com> parents: 25481 diff changeset	2003 for key in ('namespace', 'key', 'new', 'old', 'ret'):
a5192774e925 bundle2: introduce a PushkeyFail error to abort unbundle on pushkey error Pierre-Yves David <pierre-yves.david@fb.com> parents: 25481 diff changeset	2004 if key in inpart.params:
a5192774e925 bundle2: introduce a PushkeyFail error to abort unbundle on pushkey error Pierre-Yves David <pierre-yves.david@fb.com> parents: 25481 diff changeset	2005 kwargs[key] = inpart.params[key]
36428 0b697c650b04 bundle2: part id is an int, use %d to make it bytes Augie Fackler <augie@google.com> parents: 36427 diff changeset	2006 raise error.PushkeyFailed(partid='%d' % inpart.id,
36427 247b473f408e bundle2: *strkwargs love on various kwargs constructions Augie Fackler <augie@google.com>* parents: 36273 diff changeset	2007 **pycompat.strkwargs(kwargs))
21660 e87d2a12d41b bundle2: add ``pushkey`` support Pierre-Yves David <pierre-yves.david@fb.com> parents: 21655 diff changeset	2008
35260 af5507203d01 bookmark: introduce a 'bookmarks' part Boris Feld <boris.feld@octobus.net> parents: 35259 diff changeset	2009 @parthandler('bookmarks')
af5507203d01 bookmark: introduce a 'bookmarks' part Boris Feld <boris.feld@octobus.net> parents: 35259 diff changeset	2010 def handlebookmark(op, inpart):
af5507203d01 bookmark: introduce a 'bookmarks' part Boris Feld <boris.feld@octobus.net> parents: 35259 diff changeset	2011 """transmit bookmark information
af5507203d01 bookmark: introduce a 'bookmarks' part Boris Feld <boris.feld@octobus.net> parents: 35259 diff changeset	2012
35266 496154e41968 bundle2: support a 'records' mode for the 'bookmarks' part Boris Feld <boris.feld@octobus.net> parents: 35265 diff changeset	2013 The part contains binary encoded bookmark information.
496154e41968 bundle2: support a 'records' mode for the 'bookmarks' part Boris Feld <boris.feld@octobus.net> parents: 35265 diff changeset	2014
496154e41968 bundle2: support a 'records' mode for the 'bookmarks' part Boris Feld <boris.feld@octobus.net> parents: 35265 diff changeset	2015 The exact behavior of this part can be controlled by the 'bookmarks' mode
496154e41968 bundle2: support a 'records' mode for the 'bookmarks' part Boris Feld <boris.feld@octobus.net> parents: 35265 diff changeset	2016 on the bundle operation.
35260 af5507203d01 bookmark: introduce a 'bookmarks' part Boris Feld <boris.feld@octobus.net> parents: 35259 diff changeset	2017
35266 496154e41968 bundle2: support a 'records' mode for the 'bookmarks' part Boris Feld <boris.feld@octobus.net> parents: 35265 diff changeset	2018 When mode is 'apply' (the default) the bookmark information is applied as
496154e41968 bundle2: support a 'records' mode for the 'bookmarks' part Boris Feld <boris.feld@octobus.net> parents: 35265 diff changeset	2019 is to the unbundling repository. Make sure a 'check:bookmarks' part is
496154e41968 bundle2: support a 'records' mode for the 'bookmarks' part Boris Feld <boris.feld@octobus.net> parents: 35265 diff changeset	2020 issued earlier to check for push races in such update. This behavior is
496154e41968 bundle2: support a 'records' mode for the 'bookmarks' part Boris Feld <boris.feld@octobus.net> parents: 35265 diff changeset	2021 suitable for pushing.
496154e41968 bundle2: support a 'records' mode for the 'bookmarks' part Boris Feld <boris.feld@octobus.net> parents: 35265 diff changeset	2022
496154e41968 bundle2: support a 'records' mode for the 'bookmarks' part Boris Feld <boris.feld@octobus.net> parents: 35265 diff changeset	2023 When mode is 'records', the information is recorded into the 'bookmarks'
496154e41968 bundle2: support a 'records' mode for the 'bookmarks' part Boris Feld <boris.feld@octobus.net> parents: 35265 diff changeset	2024 records of the bundle operation. This behavior is suitable for pulling.
35260 af5507203d01 bookmark: introduce a 'bookmarks' part Boris Feld <boris.feld@octobus.net> parents: 35259 diff changeset	2025 """
af5507203d01 bookmark: introduce a 'bookmarks' part Boris Feld <boris.feld@octobus.net> parents: 35259 diff changeset	2026 changes = bookmarks.binarydecode(inpart)
35261 f392066d127c bookmark: add pushkey hook compatiblity to the bundle2 part Boris Feld <boris.feld@octobus.net> parents: 35260 diff changeset	2027
35266 496154e41968 bundle2: support a 'records' mode for the 'bookmarks' part Boris Feld <boris.feld@octobus.net> parents: 35265 diff changeset	2028 pushkeycompat = op.repo.ui.configbool('server', 'bookmarks-pushkey-compat')
496154e41968 bundle2: support a 'records' mode for the 'bookmarks' part Boris Feld <boris.feld@octobus.net> parents: 35265 diff changeset	2029 bookmarksmode = op.modes.get('bookmarks', 'apply')
35261 f392066d127c bookmark: add pushkey hook compatiblity to the bundle2 part Boris Feld <boris.feld@octobus.net> parents: 35260 diff changeset	2030
35266 496154e41968 bundle2: support a 'records' mode for the 'bookmarks' part Boris Feld <boris.feld@octobus.net> parents: 35265 diff changeset	2031 if bookmarksmode == 'apply':
496154e41968 bundle2: support a 'records' mode for the 'bookmarks' part Boris Feld <boris.feld@octobus.net> parents: 35265 diff changeset	2032 tr = op.gettransaction()
496154e41968 bundle2: support a 'records' mode for the 'bookmarks' part Boris Feld <boris.feld@octobus.net> parents: 35265 diff changeset	2033 bookstore = op.repo._bookmarks
496154e41968 bundle2: support a 'records' mode for the 'bookmarks' part Boris Feld <boris.feld@octobus.net> parents: 35265 diff changeset	2034 if pushkeycompat:
496154e41968 bundle2: support a 'records' mode for the 'bookmarks' part Boris Feld <boris.feld@octobus.net> parents: 35265 diff changeset	2035 allhooks = []
496154e41968 bundle2: support a 'records' mode for the 'bookmarks' part Boris Feld <boris.feld@octobus.net> parents: 35265 diff changeset	2036 for book, node in changes:
496154e41968 bundle2: support a 'records' mode for the 'bookmarks' part Boris Feld <boris.feld@octobus.net> parents: 35265 diff changeset	2037 hookargs = tr.hookargs.copy()
496154e41968 bundle2: support a 'records' mode for the 'bookmarks' part Boris Feld <boris.feld@octobus.net> parents: 35265 diff changeset	2038 hookargs['pushkeycompat'] = '1'
35811 e35320ce8043 bookmarks: fix pushkey compatibility mode (issue5777) Boris Feld <boris.feld@octobus.net> parents: 35804 diff changeset	2039 hookargs['namespace'] = 'bookmarks'
35266 496154e41968 bundle2: support a 'records' mode for the 'bookmarks' part Boris Feld <boris.feld@octobus.net> parents: 35265 diff changeset	2040 hookargs['key'] = book
496154e41968 bundle2: support a 'records' mode for the 'bookmarks' part Boris Feld <boris.feld@octobus.net> parents: 35265 diff changeset	2041 hookargs['old'] = nodemod.hex(bookstore.get(book, ''))
496154e41968 bundle2: support a 'records' mode for the 'bookmarks' part Boris Feld <boris.feld@octobus.net> parents: 35265 diff changeset	2042 hookargs['new'] = nodemod.hex(node if node is not None else '')
496154e41968 bundle2: support a 'records' mode for the 'bookmarks' part Boris Feld <boris.feld@octobus.net> parents: 35265 diff changeset	2043 allhooks.append(hookargs)
496154e41968 bundle2: support a 'records' mode for the 'bookmarks' part Boris Feld <boris.feld@octobus.net> parents: 35265 diff changeset	2044
496154e41968 bundle2: support a 'records' mode for the 'bookmarks' part Boris Feld <boris.feld@octobus.net> parents: 35265 diff changeset	2045 for hookargs in allhooks:
36273 1ec6fd26f841 bundle2: use pycompat.strkwargs as needed Augie Fackler <augie@google.com> parents: 36116 diff changeset	2046 op.repo.hook('prepushkey', throw=True,
1ec6fd26f841 bundle2: use pycompat.strkwargs as needed Augie Fackler <augie@google.com> parents: 36116 diff changeset	2047 **pycompat.strkwargs(hookargs))
35266 496154e41968 bundle2: support a 'records' mode for the 'bookmarks' part Boris Feld <boris.feld@octobus.net> parents: 35265 diff changeset	2048
496154e41968 bundle2: support a 'records' mode for the 'bookmarks' part Boris Feld <boris.feld@octobus.net> parents: 35265 diff changeset	2049 bookstore.applychanges(op.repo, op.gettransaction(), changes)
496154e41968 bundle2: support a 'records' mode for the 'bookmarks' part Boris Feld <boris.feld@octobus.net> parents: 35265 diff changeset	2050
496154e41968 bundle2: support a 'records' mode for the 'bookmarks' part Boris Feld <boris.feld@octobus.net> parents: 35265 diff changeset	2051 if pushkeycompat:
496154e41968 bundle2: support a 'records' mode for the 'bookmarks' part Boris Feld <boris.feld@octobus.net> parents: 35265 diff changeset	2052 def runhook():
496154e41968 bundle2: support a 'records' mode for the 'bookmarks' part Boris Feld <boris.feld@octobus.net> parents: 35265 diff changeset	2053 for hookargs in allhooks:
36273 1ec6fd26f841 bundle2: use pycompat.strkwargs as needed Augie Fackler <augie@google.com> parents: 36116 diff changeset	2054 op.repo.hook('pushkey', **pycompat.strkwargs(hookargs))
35266 496154e41968 bundle2: support a 'records' mode for the 'bookmarks' part Boris Feld <boris.feld@octobus.net> parents: 35265 diff changeset	2055 op.repo._afterlock(runhook)
496154e41968 bundle2: support a 'records' mode for the 'bookmarks' part Boris Feld <boris.feld@octobus.net> parents: 35265 diff changeset	2056
496154e41968 bundle2: support a 'records' mode for the 'bookmarks' part Boris Feld <boris.feld@octobus.net> parents: 35265 diff changeset	2057 elif bookmarksmode == 'records':
35261 f392066d127c bookmark: add pushkey hook compatiblity to the bundle2 part Boris Feld <boris.feld@octobus.net> parents: 35260 diff changeset	2058 for book, node in changes:
35266 496154e41968 bundle2: support a 'records' mode for the 'bookmarks' part Boris Feld <boris.feld@octobus.net> parents: 35265 diff changeset	2059 record = {'bookmark': book, 'node': node}
496154e41968 bundle2: support a 'records' mode for the 'bookmarks' part Boris Feld <boris.feld@octobus.net> parents: 35265 diff changeset	2060 op.records.add('bookmarks', record)
496154e41968 bundle2: support a 'records' mode for the 'bookmarks' part Boris Feld <boris.feld@octobus.net> parents: 35265 diff changeset	2061 else:
496154e41968 bundle2: support a 'records' mode for the 'bookmarks' part Boris Feld <boris.feld@octobus.net> parents: 35265 diff changeset	2062 raise error.ProgrammingError('unkown bookmark mode: %s' % bookmarksmode)
35260 af5507203d01 bookmark: introduce a 'bookmarks' part Boris Feld <boris.feld@octobus.net> parents: 35259 diff changeset	2063
33031 e8c8d81eb864 bundle: add config option to include phases Martin von Zweigbergk <martinvonz@google.com> parents: 33030 diff changeset	2064 @parthandler('phase-heads')
e8c8d81eb864 bundle: add config option to include phases Martin von Zweigbergk <martinvonz@google.com> parents: 33030 diff changeset	2065 def handlephases(op, inpart):
e8c8d81eb864 bundle: add config option to include phases Martin von Zweigbergk <martinvonz@google.com> parents: 33030 diff changeset	2066 """apply phases from bundle part to repo"""
34320 12c42bcd4133 phases: move the binary decoding function in the phases module Boris Feld <boris.feld@octobus.net> parents: 34319 diff changeset	2067 headsbyphase = phases.binarydecode(inpart)
34321 4ef472b975ff bundle2: only grab a transaction when 'phase-heads' affect the repository Boris Feld <boris.feld@octobus.net> parents: 34320 diff changeset	2068 phases.updatephases(op.repo.unfiltered(), op.gettransaction, headsbyphase)
33031 e8c8d81eb864 bundle: add config option to include phases Martin von Zweigbergk <martinvonz@google.com> parents: 33030 diff changeset	2069
24686 e0e28e910fa3 bundle2: rename format, parts and config to final names Pierre-Yves David <pierre-yves.david@fb.com> parents: 24648 diff changeset	2070 @parthandler('reply:pushkey', ('return', 'in-reply-to'))
21660 e87d2a12d41b bundle2: add ``pushkey`` support Pierre-Yves David <pierre-yves.david@fb.com> parents: 21655 diff changeset	2071 def handlepushkeyreply(op, inpart):
e87d2a12d41b bundle2: add ``pushkey`` support Pierre-Yves David <pierre-yves.david@fb.com> parents: 21655 diff changeset	2072 """retrieve the result of a pushkey request"""
e87d2a12d41b bundle2: add ``pushkey`` support Pierre-Yves David <pierre-yves.david@fb.com> parents: 21655 diff changeset	2073 ret = int(inpart.params['return'])
e87d2a12d41b bundle2: add ``pushkey`` support Pierre-Yves David <pierre-yves.david@fb.com> parents: 21655 diff changeset	2074 partid = int(inpart.params['in-reply-to'])
e87d2a12d41b bundle2: add ``pushkey`` support Pierre-Yves David <pierre-yves.david@fb.com> parents: 21655 diff changeset	2075 op.records.add('pushkey', {'return': ret}, partid)
22336 60786c8a2f70 bundle2: add an obsmarkers part handler Pierre-Yves David <pierre-yves.david@fb.com> parents: 21900 diff changeset	2076
24686 e0e28e910fa3 bundle2: rename format, parts and config to final names Pierre-Yves David <pierre-yves.david@fb.com> parents: 24648 diff changeset	2077 @parthandler('obsmarkers')
22336 60786c8a2f70 bundle2: add an obsmarkers part handler Pierre-Yves David <pierre-yves.david@fb.com> parents: 21900 diff changeset	2078 def handleobsmarker(op, inpart):
60786c8a2f70 bundle2: add an obsmarkers part handler Pierre-Yves David <pierre-yves.david@fb.com> parents: 21900 diff changeset	2079 """add a stream of obsmarkers to the repo"""
60786c8a2f70 bundle2: add an obsmarkers part handler Pierre-Yves David <pierre-yves.david@fb.com> parents: 21900 diff changeset	2080 tr = op.gettransaction()
24733 c00e4338fa4b obsolete: experimental flag to get debug about obsmarkers exchange Pierre-Yves David <pierre-yves.david@fb.com> parents: 24686 diff changeset	2081 markerdata = inpart.read()
33499 0407a51b9d8c codemod: register core configitems using a script Jun Wu <quark@fb.com> parents: 33461 diff changeset	2082 if op.ui.config('experimental', 'obsmarkers-exchange-debug'):
24733 c00e4338fa4b obsolete: experimental flag to get debug about obsmarkers exchange Pierre-Yves David <pierre-yves.david@fb.com> parents: 24686 diff changeset	2083 op.ui.write(('obsmarker-exchange: %i bytes received\n')
c00e4338fa4b obsolete: experimental flag to get debug about obsmarkers exchange Pierre-Yves David <pierre-yves.david@fb.com> parents: 24686 diff changeset	2084 % len(markerdata))
26685 ea390d889d3a bundle2: gracefully skip 'obsmarkers' part if evolution is disabled Pierre-Yves David <pierre-yves.david@fb.com> parents: 26587 diff changeset	2085 # The mergemarkers call will crash if marker creation is not enabled.
ea390d889d3a bundle2: gracefully skip 'obsmarkers' part if evolution is disabled Pierre-Yves David <pierre-yves.david@fb.com> parents: 26587 diff changeset	2086 # we want to avoid this if the part is advisory.
ea390d889d3a bundle2: gracefully skip 'obsmarkers' part if evolution is disabled Pierre-Yves David <pierre-yves.david@fb.com> parents: 26587 diff changeset	2087 if not inpart.mandatory and op.repo.obsstore.readonly:
35466 7906354cbc68 debug: add newlines at the end of three locations that appear to need it Kyle Lippincott <spectral@google.com> parents: 35266 diff changeset	2088 op.repo.ui.debug('ignoring obsolescence markers, feature not enabled\n')
26685 ea390d889d3a bundle2: gracefully skip 'obsmarkers' part if evolution is disabled Pierre-Yves David <pierre-yves.david@fb.com> parents: 26587 diff changeset	2089 return
24733 c00e4338fa4b obsolete: experimental flag to get debug about obsmarkers exchange Pierre-Yves David <pierre-yves.david@fb.com> parents: 24686 diff changeset	2090 new = op.repo.obsstore.mergemarkers(tr, markerdata)
32326 99515353c72a obsolete: invalidate "volatile" set cache after merging marker Pierre-Yves David <pierre-yves.david@octobus.net> parents: 32024 diff changeset	2091 op.repo.invalidatevolatilesets()
22337 c380fe290290 obsmarker: write a message with the number of markers added through bundle2 Pierre-Yves David <pierre-yves.david@fb.com> parents: 22336 diff changeset	2092 if new:
c380fe290290 obsmarker: write a message with the number of markers added through bundle2 Pierre-Yves David <pierre-yves.david@fb.com> parents: 22336 diff changeset	2093 op.repo.ui.status(_('%i new obsolescence markers\n') % new)
22338 8fcd56095d3b obssmarker: add a bundle2 record with the number of markers added Pierre-Yves David <pierre-yves.david@fb.com> parents: 22337 diff changeset	2094 op.records.add('obsmarkers', {'new': new})
22340 394a17de6a2d obsmarker: produce a reply part for markers received through bundle2 Pierre-Yves David <pierre-yves.david@fb.com> parents: 22338 diff changeset	2095 if op.reply is not None:
24686 e0e28e910fa3 bundle2: rename format, parts and config to final names Pierre-Yves David <pierre-yves.david@fb.com> parents: 24648 diff changeset	2096 rpart = op.reply.newpart('reply:obsmarkers')
33675 24b7c75c6441 bundle2: use bytestr() instead of str() to convert part id to bytes Augie Fackler <augie@google.com> parents: 33674 diff changeset	2097 rpart.addparam(
24b7c75c6441 bundle2: use bytestr() instead of str() to convert part id to bytes Augie Fackler <augie@google.com> parents: 33674 diff changeset	2098 'in-reply-to', pycompat.bytestr(inpart.id), mandatory=False)
22340 394a17de6a2d obsmarker: produce a reply part for markers received through bundle2 Pierre-Yves David <pierre-yves.david@fb.com> parents: 22338 diff changeset	2099 rpart.addparam('new', '%i' % new, mandatory=False)
394a17de6a2d obsmarker: produce a reply part for markers received through bundle2 Pierre-Yves David <pierre-yves.david@fb.com> parents: 22338 diff changeset	2100
394a17de6a2d obsmarker: produce a reply part for markers received through bundle2 Pierre-Yves David <pierre-yves.david@fb.com> parents: 22338 diff changeset	2101
24686 e0e28e910fa3 bundle2: rename format, parts and config to final names Pierre-Yves David <pierre-yves.david@fb.com> parents: 24648 diff changeset	2102 @parthandler('reply:obsmarkers', ('new', 'in-reply-to'))
25506 86472038dfd2 bundle2: rename duplicate handlepushkeyreply to handleobsmarkerreply Martin von Zweigbergk <martinvonz@google.com> parents: 25493 diff changeset	2103 def handleobsmarkerreply(op, inpart):
22340 394a17de6a2d obsmarker: produce a reply part for markers received through bundle2 Pierre-Yves David <pierre-yves.david@fb.com> parents: 22338 diff changeset	2104 """retrieve the result of a pushkey request"""
394a17de6a2d obsmarker: produce a reply part for markers received through bundle2 Pierre-Yves David <pierre-yves.david@fb.com> parents: 22338 diff changeset	2105 ret = int(inpart.params['new'])
394a17de6a2d obsmarker: produce a reply part for markers received through bundle2 Pierre-Yves David <pierre-yves.david@fb.com> parents: 22338 diff changeset	2106 partid = int(inpart.params['in-reply-to'])
394a17de6a2d obsmarker: produce a reply part for markers received through bundle2 Pierre-Yves David <pierre-yves.david@fb.com> parents: 22338 diff changeset	2107 op.records.add('obsmarkers', {'new': ret}, partid)
25401 d29201352af7 bundle2: part handler for processing .hgtags fnodes mappings Gregory Szorc <gregory.szorc@gmail.com> parents: 25336 diff changeset	2108
d29201352af7 bundle2: part handler for processing .hgtags fnodes mappings Gregory Szorc <gregory.szorc@gmail.com> parents: 25336 diff changeset	2109 @parthandler('hgtagsfnodes')
d29201352af7 bundle2: part handler for processing .hgtags fnodes mappings Gregory Szorc <gregory.szorc@gmail.com> parents: 25336 diff changeset	2110 def handlehgtagsfnodes(op, inpart):
d29201352af7 bundle2: part handler for processing .hgtags fnodes mappings Gregory Szorc <gregory.szorc@gmail.com> parents: 25336 diff changeset	2111 """Applies .hgtags fnodes cache entries to the local repo.
d29201352af7 bundle2: part handler for processing .hgtags fnodes mappings Gregory Szorc <gregory.szorc@gmail.com> parents: 25336 diff changeset	2112
d29201352af7 bundle2: part handler for processing .hgtags fnodes mappings Gregory Szorc <gregory.szorc@gmail.com> parents: 25336 diff changeset	2113 Payload is pairs of 20 byte changeset nodes and filenodes.
d29201352af7 bundle2: part handler for processing .hgtags fnodes mappings Gregory Szorc <gregory.szorc@gmail.com> parents: 25336 diff changeset	2114 """
26565 ee1bcb9aa6e4 bundle2: add op.gettransaction() to handlers that need the lock Durham Goode <durham@fb.com> parents: 26542 diff changeset	2115 # Grab the transaction so we ensure that we have the lock at this point.
ee1bcb9aa6e4 bundle2: add op.gettransaction() to handlers that need the lock Durham Goode <durham@fb.com> parents: 26542 diff changeset	2116 if op.ui.configbool('experimental', 'bundle2lazylocking'):
ee1bcb9aa6e4 bundle2: add op.gettransaction() to handlers that need the lock Durham Goode <durham@fb.com> parents: 26542 diff changeset	2117 op.gettransaction()
25401 d29201352af7 bundle2: part handler for processing .hgtags fnodes mappings Gregory Szorc <gregory.szorc@gmail.com> parents: 25336 diff changeset	2118 cache = tags.hgtagsfnodescache(op.repo.unfiltered())
d29201352af7 bundle2: part handler for processing .hgtags fnodes mappings Gregory Szorc <gregory.szorc@gmail.com> parents: 25336 diff changeset	2119
d29201352af7 bundle2: part handler for processing .hgtags fnodes mappings Gregory Szorc <gregory.szorc@gmail.com> parents: 25336 diff changeset	2120 count = 0
d29201352af7 bundle2: part handler for processing .hgtags fnodes mappings Gregory Szorc <gregory.szorc@gmail.com> parents: 25336 diff changeset	2121 while True:
d29201352af7 bundle2: part handler for processing .hgtags fnodes mappings Gregory Szorc <gregory.szorc@gmail.com> parents: 25336 diff changeset	2122 node = inpart.read(20)
d29201352af7 bundle2: part handler for processing .hgtags fnodes mappings Gregory Szorc <gregory.szorc@gmail.com> parents: 25336 diff changeset	2123 fnode = inpart.read(20)
d29201352af7 bundle2: part handler for processing .hgtags fnodes mappings Gregory Szorc <gregory.szorc@gmail.com> parents: 25336 diff changeset	2124 if len(node) < 20 or len(fnode) < 20:
25641 c0bdfe87b245 bundle2: reword debug message for invalid .hgtags data Gregory Szorc <gregory.szorc@gmail.com> parents: 25640 diff changeset	2125 op.ui.debug('ignoring incomplete received .hgtags fnodes data\n')
25401 d29201352af7 bundle2: part handler for processing .hgtags fnodes mappings Gregory Szorc <gregory.szorc@gmail.com> parents: 25336 diff changeset	2126 break
d29201352af7 bundle2: part handler for processing .hgtags fnodes mappings Gregory Szorc <gregory.szorc@gmail.com> parents: 25336 diff changeset	2127 cache.setfnode(node, fnode)
d29201352af7 bundle2: part handler for processing .hgtags fnodes mappings Gregory Szorc <gregory.szorc@gmail.com> parents: 25336 diff changeset	2128 count += 1
d29201352af7 bundle2: part handler for processing .hgtags fnodes mappings Gregory Szorc <gregory.szorc@gmail.com> parents: 25336 diff changeset	2129
d29201352af7 bundle2: part handler for processing .hgtags fnodes mappings Gregory Szorc <gregory.szorc@gmail.com> parents: 25336 diff changeset	2130 cache.write()
d29201352af7 bundle2: part handler for processing .hgtags fnodes mappings Gregory Szorc <gregory.szorc@gmail.com> parents: 25336 diff changeset	2131 op.ui.debug('applied %i hgtags fnodes cache entries\n' % count)
33693 db3dc11356ed pushvars: move fb extension pushvars to core Pulkit Goyal <7895pulkit@gmail.com> parents: 33682 diff changeset	2132
36963 9988fc10f49e revbranchcache: add a bundle2 handler for a rbc part Boris Feld <boris.feld@octobus.net> parents: 36428 diff changeset	2133 rbcstruct = struct.Struct('>III')
9988fc10f49e revbranchcache: add a bundle2 handler for a rbc part Boris Feld <boris.feld@octobus.net> parents: 36428 diff changeset	2134
9988fc10f49e revbranchcache: add a bundle2 handler for a rbc part Boris Feld <boris.feld@octobus.net> parents: 36428 diff changeset	2135 @parthandler('cache:rev-branch-cache')
9988fc10f49e revbranchcache: add a bundle2 handler for a rbc part Boris Feld <boris.feld@octobus.net> parents: 36428 diff changeset	2136 def handlerbc(op, inpart):
9988fc10f49e revbranchcache: add a bundle2 handler for a rbc part Boris Feld <boris.feld@octobus.net> parents: 36428 diff changeset	2137 """receive a rev-branch-cache payload and update the local cache
9988fc10f49e revbranchcache: add a bundle2 handler for a rbc part Boris Feld <boris.feld@octobus.net> parents: 36428 diff changeset	2138
9988fc10f49e revbranchcache: add a bundle2 handler for a rbc part Boris Feld <boris.feld@octobus.net> parents: 36428 diff changeset	2139 The payload is a series of data related to each branch
9988fc10f49e revbranchcache: add a bundle2 handler for a rbc part Boris Feld <boris.feld@octobus.net> parents: 36428 diff changeset	2140
9988fc10f49e revbranchcache: add a bundle2 handler for a rbc part Boris Feld <boris.feld@octobus.net> parents: 36428 diff changeset	2141 1) branch name length
9988fc10f49e revbranchcache: add a bundle2 handler for a rbc part Boris Feld <boris.feld@octobus.net> parents: 36428 diff changeset	2142 2) number of open heads
9988fc10f49e revbranchcache: add a bundle2 handler for a rbc part Boris Feld <boris.feld@octobus.net> parents: 36428 diff changeset	2143 3) number of closed heads
9988fc10f49e revbranchcache: add a bundle2 handler for a rbc part Boris Feld <boris.feld@octobus.net> parents: 36428 diff changeset	2144 4) open heads nodes
9988fc10f49e revbranchcache: add a bundle2 handler for a rbc part Boris Feld <boris.feld@octobus.net> parents: 36428 diff changeset	2145 5) closed heads nodes
9988fc10f49e revbranchcache: add a bundle2 handler for a rbc part Boris Feld <boris.feld@octobus.net> parents: 36428 diff changeset	2146 """
9988fc10f49e revbranchcache: add a bundle2 handler for a rbc part Boris Feld <boris.feld@octobus.net> parents: 36428 diff changeset	2147 total = 0
9988fc10f49e revbranchcache: add a bundle2 handler for a rbc part Boris Feld <boris.feld@octobus.net> parents: 36428 diff changeset	2148 rawheader = inpart.read(rbcstruct.size)
9988fc10f49e revbranchcache: add a bundle2 handler for a rbc part Boris Feld <boris.feld@octobus.net> parents: 36428 diff changeset	2149 cache = op.repo.revbranchcache()
9988fc10f49e revbranchcache: add a bundle2 handler for a rbc part Boris Feld <boris.feld@octobus.net> parents: 36428 diff changeset	2150 cl = op.repo.unfiltered().changelog
9988fc10f49e revbranchcache: add a bundle2 handler for a rbc part Boris Feld <boris.feld@octobus.net> parents: 36428 diff changeset	2151 while rawheader:
9988fc10f49e revbranchcache: add a bundle2 handler for a rbc part Boris Feld <boris.feld@octobus.net> parents: 36428 diff changeset	2152 header = rbcstruct.unpack(rawheader)
9988fc10f49e revbranchcache: add a bundle2 handler for a rbc part Boris Feld <boris.feld@octobus.net> parents: 36428 diff changeset	2153 total += header[1] + header[2]
9988fc10f49e revbranchcache: add a bundle2 handler for a rbc part Boris Feld <boris.feld@octobus.net> parents: 36428 diff changeset	2154 utf8branch = inpart.read(header[0])
9988fc10f49e revbranchcache: add a bundle2 handler for a rbc part Boris Feld <boris.feld@octobus.net> parents: 36428 diff changeset	2155 branch = encoding.tolocal(utf8branch)
9988fc10f49e revbranchcache: add a bundle2 handler for a rbc part Boris Feld <boris.feld@octobus.net> parents: 36428 diff changeset	2156 for x in xrange(header[1]):
9988fc10f49e revbranchcache: add a bundle2 handler for a rbc part Boris Feld <boris.feld@octobus.net> parents: 36428 diff changeset	2157 node = inpart.read(20)
9988fc10f49e revbranchcache: add a bundle2 handler for a rbc part Boris Feld <boris.feld@octobus.net> parents: 36428 diff changeset	2158 rev = cl.rev(node)
9988fc10f49e revbranchcache: add a bundle2 handler for a rbc part Boris Feld <boris.feld@octobus.net> parents: 36428 diff changeset	2159 cache.setdata(branch, rev, node, False)
9988fc10f49e revbranchcache: add a bundle2 handler for a rbc part Boris Feld <boris.feld@octobus.net> parents: 36428 diff changeset	2160 for x in xrange(header[2]):
9988fc10f49e revbranchcache: add a bundle2 handler for a rbc part Boris Feld <boris.feld@octobus.net> parents: 36428 diff changeset	2161 node = inpart.read(20)
9988fc10f49e revbranchcache: add a bundle2 handler for a rbc part Boris Feld <boris.feld@octobus.net> parents: 36428 diff changeset	2162 rev = cl.rev(node)
9988fc10f49e revbranchcache: add a bundle2 handler for a rbc part Boris Feld <boris.feld@octobus.net> parents: 36428 diff changeset	2163 cache.setdata(branch, rev, node, True)
9988fc10f49e revbranchcache: add a bundle2 handler for a rbc part Boris Feld <boris.feld@octobus.net> parents: 36428 diff changeset	2164 rawheader = inpart.read(rbcstruct.size)
9988fc10f49e revbranchcache: add a bundle2 handler for a rbc part Boris Feld <boris.feld@octobus.net> parents: 36428 diff changeset	2165 cache.write()
9988fc10f49e revbranchcache: add a bundle2 handler for a rbc part Boris Feld <boris.feld@octobus.net> parents: 36428 diff changeset	2166
33693 db3dc11356ed pushvars: move fb extension pushvars to core Pulkit Goyal <7895pulkit@gmail.com> parents: 33682 diff changeset	2167 @parthandler('pushvars')
db3dc11356ed pushvars: move fb extension pushvars to core Pulkit Goyal <7895pulkit@gmail.com> parents: 33682 diff changeset	2168 def bundle2getvars(op, part):
db3dc11356ed pushvars: move fb extension pushvars to core Pulkit Goyal <7895pulkit@gmail.com> parents: 33682 diff changeset	2169 '''unbundle a bundle2 containing shellvars on the server'''
db3dc11356ed pushvars: move fb extension pushvars to core Pulkit Goyal <7895pulkit@gmail.com> parents: 33682 diff changeset	2170 # An option to disable unbundling on server-side for security reasons
33835 057d31ceace3 pushvars: add a coreconfigitem for push.pushvars.server Pulkit Goyal <7895pulkit@gmail.com> parents: 33810 diff changeset	2171 if op.ui.configbool('push', 'pushvars.server'):
33693 db3dc11356ed pushvars: move fb extension pushvars to core Pulkit Goyal <7895pulkit@gmail.com> parents: 33682 diff changeset	2172 hookargs = {}
db3dc11356ed pushvars: move fb extension pushvars to core Pulkit Goyal <7895pulkit@gmail.com> parents: 33682 diff changeset	2173 for key, value in part.advisoryparams:
db3dc11356ed pushvars: move fb extension pushvars to core Pulkit Goyal <7895pulkit@gmail.com> parents: 33682 diff changeset	2174 key = key.upper()
db3dc11356ed pushvars: move fb extension pushvars to core Pulkit Goyal <7895pulkit@gmail.com> parents: 33682 diff changeset	2175 # We want pushed variables to have USERVAR_ prepended so we know
db3dc11356ed pushvars: move fb extension pushvars to core Pulkit Goyal <7895pulkit@gmail.com> parents: 33682 diff changeset	2176 # they came from the --pushvar flag.
db3dc11356ed pushvars: move fb extension pushvars to core Pulkit Goyal <7895pulkit@gmail.com> parents: 33682 diff changeset	2177 key = "USERVAR_" + key
db3dc11356ed pushvars: move fb extension pushvars to core Pulkit Goyal <7895pulkit@gmail.com> parents: 33682 diff changeset	2178 hookargs[key] = value
db3dc11356ed pushvars: move fb extension pushvars to core Pulkit Goyal <7895pulkit@gmail.com> parents: 33682 diff changeset	2179 op.addhookargs(hookargs)
35758 b996ddf5963d bundle2: add a 'stream' part handler for stream cloning Boris Feld <boris.feld@octobus.net> parents: 35655 diff changeset	2180
35788 b116a66bcc44 bundle2: move version of stream clone into part name Gregory Szorc <gregory.szorc@gmail.com> parents: 35787 diff changeset	2181 @parthandler('stream2', ('requirements', 'filecount', 'bytecount'))
b116a66bcc44 bundle2: move version of stream clone into part name Gregory Szorc <gregory.szorc@gmail.com> parents: 35787 diff changeset	2182 def handlestreamv2bundle(op, part):
35758 b996ddf5963d bundle2: add a 'stream' part handler for stream cloning Boris Feld <boris.feld@octobus.net> parents: 35655 diff changeset	2183
35813 768326377e4d bundle2: fix the formatting of the stream part requirements Boris Feld <boris.feld@octobus.net> parents: 35811 diff changeset	2184 requirements = urlreq.unquote(part.params['requirements']).split(',')
35758 b996ddf5963d bundle2: add a 'stream' part handler for stream cloning Boris Feld <boris.feld@octobus.net> parents: 35655 diff changeset	2185 filecount = int(part.params['filecount'])
b996ddf5963d bundle2: add a 'stream' part handler for stream cloning Boris Feld <boris.feld@octobus.net> parents: 35655 diff changeset	2186 bytecount = int(part.params['bytecount'])
b996ddf5963d bundle2: add a 'stream' part handler for stream cloning Boris Feld <boris.feld@octobus.net> parents: 35655 diff changeset	2187
b996ddf5963d bundle2: add a 'stream' part handler for stream cloning Boris Feld <boris.feld@octobus.net> parents: 35655 diff changeset	2188 repo = op.repo
b996ddf5963d bundle2: add a 'stream' part handler for stream cloning Boris Feld <boris.feld@octobus.net> parents: 35655 diff changeset	2189 if len(repo):
b996ddf5963d bundle2: add a 'stream' part handler for stream cloning Boris Feld <boris.feld@octobus.net> parents: 35655 diff changeset	2190 msg = _('cannot apply stream clone to non empty repository')
b996ddf5963d bundle2: add a 'stream' part handler for stream cloning Boris Feld <boris.feld@octobus.net> parents: 35655 diff changeset	2191 raise error.Abort(msg)
b996ddf5963d bundle2: add a 'stream' part handler for stream cloning Boris Feld <boris.feld@octobus.net> parents: 35655 diff changeset	2192
b996ddf5963d bundle2: add a 'stream' part handler for stream cloning Boris Feld <boris.feld@octobus.net> parents: 35655 diff changeset	2193 repo.ui.debug('applying stream bundle\n')
b996ddf5963d bundle2: add a 'stream' part handler for stream cloning Boris Feld <boris.feld@octobus.net> parents: 35655 diff changeset	2194 streamclone.applybundlev2(repo, part, filecount, bytecount,
b996ddf5963d bundle2: add a 'stream' part handler for stream cloning Boris Feld <boris.feld@octobus.net> parents: 35655 diff changeset	2195 requirements)

Mercurial > public > mercurial-scm > hg

annotate mercurial/bundle2.py @ 36963:9988fc10f49e