mercurial-scm/hg-stable: mercurial/bundle2.py annotate

annotate mercurial/bundle2.py @ 24794:21f2e8f48ae1

bundle2: add a 'salvageoutput' method on bundle20 This method returns a copy of all 'output' parts added to the bundler. This is the second step in our quest for preserving the server output on error (issue4594). We want to be able to copy the output parts from the aborted reply into the exception bundle. The function will be used in a later patch.

author	Pierre-Yves David <pierre-yves.david@fb.com>
date	Thu, 16 Apr 2015 03:16:04 -0400
parents	2ec894582ea2
children	f9aa4cb8f2dd

rev	line source
20801 9c5183cb9bca bundle2: very first version of a bundle2 bundler Pierre-Yves David <pierre-yves.david@fb.com> parents: diff changeset	1 # bundle2.py - generic container format to transmit arbitrary data.
9c5183cb9bca bundle2: very first version of a bundle2 bundler Pierre-Yves David <pierre-yves.david@fb.com> parents: diff changeset	2 #
9c5183cb9bca bundle2: very first version of a bundle2 bundler Pierre-Yves David <pierre-yves.david@fb.com> parents: diff changeset	3 # Copyright 2013 Facebook, Inc.
9c5183cb9bca bundle2: very first version of a bundle2 bundler Pierre-Yves David <pierre-yves.david@fb.com> parents: diff changeset	4 #
9c5183cb9bca bundle2: very first version of a bundle2 bundler Pierre-Yves David <pierre-yves.david@fb.com> parents: diff changeset	5 # This software may be used and distributed according to the terms of the
9c5183cb9bca bundle2: very first version of a bundle2 bundler Pierre-Yves David <pierre-yves.david@fb.com> parents: diff changeset	6 # GNU General Public License version 2 or any later version.
9c5183cb9bca bundle2: very first version of a bundle2 bundler Pierre-Yves David <pierre-yves.david@fb.com> parents: diff changeset	7 """Handling of the new bundle2 format
9c5183cb9bca bundle2: very first version of a bundle2 bundler Pierre-Yves David <pierre-yves.david@fb.com> parents: diff changeset	8
9c5183cb9bca bundle2: very first version of a bundle2 bundler Pierre-Yves David <pierre-yves.david@fb.com> parents: diff changeset	9 The goal of bundle2 is to act as an atomically packet to transmit a set of
9c5183cb9bca bundle2: very first version of a bundle2 bundler Pierre-Yves David <pierre-yves.david@fb.com> parents: diff changeset	10 payloads in an application agnostic way. It consist in a sequence of "parts"
9c5183cb9bca bundle2: very first version of a bundle2 bundler Pierre-Yves David <pierre-yves.david@fb.com> parents: diff changeset	11 that will be handed to and processed by the application layer.
9c5183cb9bca bundle2: very first version of a bundle2 bundler Pierre-Yves David <pierre-yves.david@fb.com> parents: diff changeset	12
9c5183cb9bca bundle2: very first version of a bundle2 bundler Pierre-Yves David <pierre-yves.david@fb.com> parents: diff changeset	13
9c5183cb9bca bundle2: very first version of a bundle2 bundler Pierre-Yves David <pierre-yves.david@fb.com> parents: diff changeset	14 General format architecture
9c5183cb9bca bundle2: very first version of a bundle2 bundler Pierre-Yves David <pierre-yves.david@fb.com> parents: diff changeset	15 ===========================
9c5183cb9bca bundle2: very first version of a bundle2 bundler Pierre-Yves David <pierre-yves.david@fb.com> parents: diff changeset	16
9c5183cb9bca bundle2: very first version of a bundle2 bundler Pierre-Yves David <pierre-yves.david@fb.com> parents: diff changeset	17 The format is architectured as follow
9c5183cb9bca bundle2: very first version of a bundle2 bundler Pierre-Yves David <pierre-yves.david@fb.com> parents: diff changeset	18
9c5183cb9bca bundle2: very first version of a bundle2 bundler Pierre-Yves David <pierre-yves.david@fb.com> parents: diff changeset	19 - magic string
9c5183cb9bca bundle2: very first version of a bundle2 bundler Pierre-Yves David <pierre-yves.david@fb.com> parents: diff changeset	20 - stream level parameters
9c5183cb9bca bundle2: very first version of a bundle2 bundler Pierre-Yves David <pierre-yves.david@fb.com> parents: diff changeset	21 - payload parts (any number)
9c5183cb9bca bundle2: very first version of a bundle2 bundler Pierre-Yves David <pierre-yves.david@fb.com> parents: diff changeset	22 - end of stream marker.
9c5183cb9bca bundle2: very first version of a bundle2 bundler Pierre-Yves David <pierre-yves.david@fb.com> parents: diff changeset	23
20856 8a6a86c9a5b5 bundle2: support bundling of empty part (with a type) Pierre-Yves David <pierre-yves.david@fb.com> parents: 20844 diff changeset	24 the Binary format
20801 9c5183cb9bca bundle2: very first version of a bundle2 bundler Pierre-Yves David <pierre-yves.david@fb.com> parents: diff changeset	25 ============================
9c5183cb9bca bundle2: very first version of a bundle2 bundler Pierre-Yves David <pierre-yves.david@fb.com> parents: diff changeset	26
21024 7731a2281cf0 spelling: fixes from spell checker Mads Kiilerich <madski@unity3d.com> parents: 21020 diff changeset	27 All numbers are unsigned and big-endian.
20801 9c5183cb9bca bundle2: very first version of a bundle2 bundler Pierre-Yves David <pierre-yves.david@fb.com> parents: diff changeset	28
9c5183cb9bca bundle2: very first version of a bundle2 bundler Pierre-Yves David <pierre-yves.david@fb.com> parents: diff changeset	29 stream level parameters
9c5183cb9bca bundle2: very first version of a bundle2 bundler Pierre-Yves David <pierre-yves.david@fb.com> parents: diff changeset	30 ------------------------
9c5183cb9bca bundle2: very first version of a bundle2 bundler Pierre-Yves David <pierre-yves.david@fb.com> parents: diff changeset	31
9c5183cb9bca bundle2: very first version of a bundle2 bundler Pierre-Yves David <pierre-yves.david@fb.com> parents: diff changeset	32 Binary format is as follow
9c5183cb9bca bundle2: very first version of a bundle2 bundler Pierre-Yves David <pierre-yves.david@fb.com> parents: diff changeset	33
23009 90f86ad3d4ff bundle2: change header size and make them signed (new format) Pierre-Yves David <pierre-yves.david@fb.com> parents: 23008 diff changeset	34 :params size: int32
20801 9c5183cb9bca bundle2: very first version of a bundle2 bundler Pierre-Yves David <pierre-yves.david@fb.com> parents: diff changeset	35
9c5183cb9bca bundle2: very first version of a bundle2 bundler Pierre-Yves David <pierre-yves.david@fb.com> parents: diff changeset	36 The total number of Bytes used by the parameters
9c5183cb9bca bundle2: very first version of a bundle2 bundler Pierre-Yves David <pierre-yves.david@fb.com> parents: diff changeset	37
9c5183cb9bca bundle2: very first version of a bundle2 bundler Pierre-Yves David <pierre-yves.david@fb.com> parents: diff changeset	38 :params value: arbitrary number of Bytes
9c5183cb9bca bundle2: very first version of a bundle2 bundler Pierre-Yves David <pierre-yves.david@fb.com> parents: diff changeset	39
9c5183cb9bca bundle2: very first version of a bundle2 bundler Pierre-Yves David <pierre-yves.david@fb.com> parents: diff changeset	40 A blob of `params size` containing the serialized version of all stream level
9c5183cb9bca bundle2: very first version of a bundle2 bundler Pierre-Yves David <pierre-yves.david@fb.com> parents: diff changeset	41 parameters.
9c5183cb9bca bundle2: very first version of a bundle2 bundler Pierre-Yves David <pierre-yves.david@fb.com> parents: diff changeset	42
21024 7731a2281cf0 spelling: fixes from spell checker Mads Kiilerich <madski@unity3d.com> parents: 21020 diff changeset	43 The blob contains a space separated list of parameters. Parameters with value
20811 9785c3f8f598 bundle2: urlquote stream parameter name and value Pierre-Yves David <pierre-yves.david@fb.com> parents: 20810 diff changeset	44 are stored in the form `<name>=<value>`. Both name and value are urlquoted.
20804 db9d3991d2c6 bundle2: support bundling simple parameter Pierre-Yves David <pierre-yves.david@fb.com> parents: 20803 diff changeset	45
20813 8c74b3ce5b70 bundle2: refuse empty parameter name Pierre-Yves David <pierre-yves.david@fb.com> parents: 20812 diff changeset	46 Empty name are obviously forbidden.
8c74b3ce5b70 bundle2: refuse empty parameter name Pierre-Yves David <pierre-yves.david@fb.com> parents: 20812 diff changeset	47
20844 2631204d7305 bundle2: implement the mandatory/advisory logic for parameter Pierre-Yves David <pierre-yves.david@fb.com> parents: 20843 diff changeset	48 Name MUST start with a letter. If this first letter is lower case, the
21024 7731a2281cf0 spelling: fixes from spell checker Mads Kiilerich <madski@unity3d.com> parents: 21020 diff changeset	49 parameter is advisory and can be safely ignored. However when the first
20844 2631204d7305 bundle2: implement the mandatory/advisory logic for parameter Pierre-Yves David <pierre-yves.david@fb.com> parents: 20843 diff changeset	50 letter is capital, the parameter is mandatory and the bundling process MUST
2631204d7305 bundle2: implement the mandatory/advisory logic for parameter Pierre-Yves David <pierre-yves.david@fb.com> parents: 20843 diff changeset	51 stop if he is not able to proceed it.
20814 8532f5e1b9df bundle2: force the first char of parameter to be an letter. Pierre-Yves David <pierre-yves.david@fb.com> parents: 20813 diff changeset	52
20808 4c9130c7a29f bundle2: clarify stream parameter design in the documentation Pierre-Yves David <pierre-yves.david@fb.com> parents: 20805 diff changeset	53 Stream parameters use a simple textual format for two main reasons:
20804 db9d3991d2c6 bundle2: support bundling simple parameter Pierre-Yves David <pierre-yves.david@fb.com> parents: 20803 diff changeset	54
21024 7731a2281cf0 spelling: fixes from spell checker Mads Kiilerich <madski@unity3d.com> parents: 21020 diff changeset	55 - Stream level parameters should remain simple and we want to discourage any
20808 4c9130c7a29f bundle2: clarify stream parameter design in the documentation Pierre-Yves David <pierre-yves.david@fb.com> parents: 20805 diff changeset	56 crazy usage.
21024 7731a2281cf0 spelling: fixes from spell checker Mads Kiilerich <madski@unity3d.com> parents: 21020 diff changeset	57 - Textual data allow easy human inspection of a bundle2 header in case of
20808 4c9130c7a29f bundle2: clarify stream parameter design in the documentation Pierre-Yves David <pierre-yves.david@fb.com> parents: 20805 diff changeset	58 troubles.
4c9130c7a29f bundle2: clarify stream parameter design in the documentation Pierre-Yves David <pierre-yves.david@fb.com> parents: 20805 diff changeset	59
4c9130c7a29f bundle2: clarify stream parameter design in the documentation Pierre-Yves David <pierre-yves.david@fb.com> parents: 20805 diff changeset	60 Any Applicative level options MUST go into a bundle2 part instead.
20801 9c5183cb9bca bundle2: very first version of a bundle2 bundler Pierre-Yves David <pierre-yves.david@fb.com> parents: diff changeset	61
9c5183cb9bca bundle2: very first version of a bundle2 bundler Pierre-Yves David <pierre-yves.david@fb.com> parents: diff changeset	62 Payload part
9c5183cb9bca bundle2: very first version of a bundle2 bundler Pierre-Yves David <pierre-yves.david@fb.com> parents: diff changeset	63 ------------------------
9c5183cb9bca bundle2: very first version of a bundle2 bundler Pierre-Yves David <pierre-yves.david@fb.com> parents: diff changeset	64
9c5183cb9bca bundle2: very first version of a bundle2 bundler Pierre-Yves David <pierre-yves.david@fb.com> parents: diff changeset	65 Binary format is as follow
9c5183cb9bca bundle2: very first version of a bundle2 bundler Pierre-Yves David <pierre-yves.david@fb.com> parents: diff changeset	66
23009 90f86ad3d4ff bundle2: change header size and make them signed (new format) Pierre-Yves David <pierre-yves.david@fb.com> parents: 23008 diff changeset	67 :header size: int32
20801 9c5183cb9bca bundle2: very first version of a bundle2 bundler Pierre-Yves David <pierre-yves.david@fb.com> parents: diff changeset	68
9c5183cb9bca bundle2: very first version of a bundle2 bundler Pierre-Yves David <pierre-yves.david@fb.com> parents: diff changeset	69 The total number of Bytes used by the part headers. When the header is empty
9c5183cb9bca bundle2: very first version of a bundle2 bundler Pierre-Yves David <pierre-yves.david@fb.com> parents: diff changeset	70 (size = 0) this is interpreted as the end of stream marker.
9c5183cb9bca bundle2: very first version of a bundle2 bundler Pierre-Yves David <pierre-yves.david@fb.com> parents: diff changeset	71
20856 8a6a86c9a5b5 bundle2: support bundling of empty part (with a type) Pierre-Yves David <pierre-yves.david@fb.com> parents: 20844 diff changeset	72 :header:
8a6a86c9a5b5 bundle2: support bundling of empty part (with a type) Pierre-Yves David <pierre-yves.david@fb.com> parents: 20844 diff changeset	73
8a6a86c9a5b5 bundle2: support bundling of empty part (with a type) Pierre-Yves David <pierre-yves.david@fb.com> parents: 20844 diff changeset	74 The header defines how to interpret the part. It contains two piece of
8a6a86c9a5b5 bundle2: support bundling of empty part (with a type) Pierre-Yves David <pierre-yves.david@fb.com> parents: 20844 diff changeset	75 data: the part type, and the part parameters.
8a6a86c9a5b5 bundle2: support bundling of empty part (with a type) Pierre-Yves David <pierre-yves.david@fb.com> parents: 20844 diff changeset	76
8a6a86c9a5b5 bundle2: support bundling of empty part (with a type) Pierre-Yves David <pierre-yves.david@fb.com> parents: 20844 diff changeset	77 The part type is used to route an application level handler, that can
8a6a86c9a5b5 bundle2: support bundling of empty part (with a type) Pierre-Yves David <pierre-yves.david@fb.com> parents: 20844 diff changeset	78 interpret payload.
8a6a86c9a5b5 bundle2: support bundling of empty part (with a type) Pierre-Yves David <pierre-yves.david@fb.com> parents: 20844 diff changeset	79
8a6a86c9a5b5 bundle2: support bundling of empty part (with a type) Pierre-Yves David <pierre-yves.david@fb.com> parents: 20844 diff changeset	80 Part parameters are passed to the application level handler. They are
8a6a86c9a5b5 bundle2: support bundling of empty part (with a type) Pierre-Yves David <pierre-yves.david@fb.com> parents: 20844 diff changeset	81 meant to convey information that will help the application level object to
8a6a86c9a5b5 bundle2: support bundling of empty part (with a type) Pierre-Yves David <pierre-yves.david@fb.com> parents: 20844 diff changeset	82 interpret the part payload.
8a6a86c9a5b5 bundle2: support bundling of empty part (with a type) Pierre-Yves David <pierre-yves.david@fb.com> parents: 20844 diff changeset	83
8a6a86c9a5b5 bundle2: support bundling of empty part (with a type) Pierre-Yves David <pierre-yves.david@fb.com> parents: 20844 diff changeset	84 The binary format of the header is has follow
8a6a86c9a5b5 bundle2: support bundling of empty part (with a type) Pierre-Yves David <pierre-yves.david@fb.com> parents: 20844 diff changeset	85
8a6a86c9a5b5 bundle2: support bundling of empty part (with a type) Pierre-Yves David <pierre-yves.david@fb.com> parents: 20844 diff changeset	86 :typesize: (one byte)
20877 9e9e3a4e9261 bundle2: part params Pierre-Yves David <pierre-yves.david@fb.com> parents: 20876 diff changeset	87
23916 a3f7c781786b bundle2: fix parttype enforcement Matt Mackall <mpm@selenic.com> parents: 23868 diff changeset	88 :parttype: alphanumerical part name (restricted to [a-zA-Z0-9_:-]*)
20877 9e9e3a4e9261 bundle2: part params Pierre-Yves David <pierre-yves.david@fb.com> parents: 20876 diff changeset	89
20995 e995d104c87f bundle2: add an integer id to part Pierre-Yves David <pierre-yves.david@fb.com> parents: 20952 diff changeset	90 :partid: A 32bits integer (unique in the bundle) that can be used to refer
e995d104c87f bundle2: add an integer id to part Pierre-Yves David <pierre-yves.david@fb.com> parents: 20952 diff changeset	91 to this part.
e995d104c87f bundle2: add an integer id to part Pierre-Yves David <pierre-yves.david@fb.com> parents: 20952 diff changeset	92
20877 9e9e3a4e9261 bundle2: part params Pierre-Yves David <pierre-yves.david@fb.com> parents: 20876 diff changeset	93 :parameters:
9e9e3a4e9261 bundle2: part params Pierre-Yves David <pierre-yves.david@fb.com> parents: 20876 diff changeset	94
21024 7731a2281cf0 spelling: fixes from spell checker Mads Kiilerich <madski@unity3d.com> parents: 21020 diff changeset	95 Part's parameter may have arbitrary content, the binary structure is::
20877 9e9e3a4e9261 bundle2: part params Pierre-Yves David <pierre-yves.david@fb.com> parents: 20876 diff changeset	96
9e9e3a4e9261 bundle2: part params Pierre-Yves David <pierre-yves.david@fb.com> parents: 20876 diff changeset	97 <mandatory-count><advisory-count><param-sizes><param-data>
9e9e3a4e9261 bundle2: part params Pierre-Yves David <pierre-yves.david@fb.com> parents: 20876 diff changeset	98
9e9e3a4e9261 bundle2: part params Pierre-Yves David <pierre-yves.david@fb.com> parents: 20876 diff changeset	99 :mandatory-count: 1 byte, number of mandatory parameters
9e9e3a4e9261 bundle2: part params Pierre-Yves David <pierre-yves.david@fb.com> parents: 20876 diff changeset	100
9e9e3a4e9261 bundle2: part params Pierre-Yves David <pierre-yves.david@fb.com> parents: 20876 diff changeset	101 :advisory-count: 1 byte, number of advisory parameters
9e9e3a4e9261 bundle2: part params Pierre-Yves David <pierre-yves.david@fb.com> parents: 20876 diff changeset	102
9e9e3a4e9261 bundle2: part params Pierre-Yves David <pierre-yves.david@fb.com> parents: 20876 diff changeset	103 :param-sizes:
9e9e3a4e9261 bundle2: part params Pierre-Yves David <pierre-yves.david@fb.com> parents: 20876 diff changeset	104
9e9e3a4e9261 bundle2: part params Pierre-Yves David <pierre-yves.david@fb.com> parents: 20876 diff changeset	105 N couple of bytes, where N is the total number of parameters. Each
9e9e3a4e9261 bundle2: part params Pierre-Yves David <pierre-yves.david@fb.com> parents: 20876 diff changeset	106 couple contains (<size-of-key>, <size-of-value) for one parameter.
9e9e3a4e9261 bundle2: part params Pierre-Yves David <pierre-yves.david@fb.com> parents: 20876 diff changeset	107
9e9e3a4e9261 bundle2: part params Pierre-Yves David <pierre-yves.david@fb.com> parents: 20876 diff changeset	108 :param-data:
9e9e3a4e9261 bundle2: part params Pierre-Yves David <pierre-yves.david@fb.com> parents: 20876 diff changeset	109
9e9e3a4e9261 bundle2: part params Pierre-Yves David <pierre-yves.david@fb.com> parents: 20876 diff changeset	110 A blob of bytes from which each parameter key and value can be
9e9e3a4e9261 bundle2: part params Pierre-Yves David <pierre-yves.david@fb.com> parents: 20876 diff changeset	111 retrieved using the list of size couples stored in the previous
9e9e3a4e9261 bundle2: part params Pierre-Yves David <pierre-yves.david@fb.com> parents: 20876 diff changeset	112 field.
9e9e3a4e9261 bundle2: part params Pierre-Yves David <pierre-yves.david@fb.com> parents: 20876 diff changeset	113
9e9e3a4e9261 bundle2: part params Pierre-Yves David <pierre-yves.david@fb.com> parents: 20876 diff changeset	114 Mandatory parameters comes first, then the advisory ones.
20856 8a6a86c9a5b5 bundle2: support bundling of empty part (with a type) Pierre-Yves David <pierre-yves.david@fb.com> parents: 20844 diff changeset	115
21607 054fa5176fa7 bundle2: forbid duplicate parameter keys Pierre-Yves David <pierre-yves.david@fb.com> parents: 21606 diff changeset	116 Each parameter's key MUST be unique within the part.
054fa5176fa7 bundle2: forbid duplicate parameter keys Pierre-Yves David <pierre-yves.david@fb.com> parents: 21606 diff changeset	117
20856 8a6a86c9a5b5 bundle2: support bundling of empty part (with a type) Pierre-Yves David <pierre-yves.david@fb.com> parents: 20844 diff changeset	118 :payload:
8a6a86c9a5b5 bundle2: support bundling of empty part (with a type) Pierre-Yves David <pierre-yves.david@fb.com> parents: 20844 diff changeset	119
20876 ddd56f3eb786 bundle2: support for bundling and unbundling payload Pierre-Yves David <pierre-yves.david@fb.com> parents: 20864 diff changeset	120 payload is a series of `<chunksize><chunkdata>`.
ddd56f3eb786 bundle2: support for bundling and unbundling payload Pierre-Yves David <pierre-yves.david@fb.com> parents: 20864 diff changeset	121
23009 90f86ad3d4ff bundle2: change header size and make them signed (new format) Pierre-Yves David <pierre-yves.david@fb.com> parents: 23008 diff changeset	122 `chunksize` is an int32, `chunkdata` are plain bytes (as much as
20876 ddd56f3eb786 bundle2: support for bundling and unbundling payload Pierre-Yves David <pierre-yves.david@fb.com> parents: 20864 diff changeset	123 `chunksize` says)` The payload part is concluded by a zero size chunk.
ddd56f3eb786 bundle2: support for bundling and unbundling payload Pierre-Yves David <pierre-yves.david@fb.com> parents: 20864 diff changeset	124
ddd56f3eb786 bundle2: support for bundling and unbundling payload Pierre-Yves David <pierre-yves.david@fb.com> parents: 20864 diff changeset	125 The current implementation always produces either zero or one chunk.
21024 7731a2281cf0 spelling: fixes from spell checker Mads Kiilerich <madski@unity3d.com> parents: 21020 diff changeset	126 This is an implementation limitation that will ultimately be lifted.
20891 1c6cd23fc221 bundle2: add some distinction between mandatory and advisory part Pierre-Yves David <pierre-yves.david@fb.com> parents: 20890 diff changeset	127
23009 90f86ad3d4ff bundle2: change header size and make them signed (new format) Pierre-Yves David <pierre-yves.david@fb.com> parents: 23008 diff changeset	128 `chunksize` can be negative to trigger special case processing. No such
90f86ad3d4ff bundle2: change header size and make them signed (new format) Pierre-Yves David <pierre-yves.david@fb.com> parents: 23008 diff changeset	129 processing is in place yet.
90f86ad3d4ff bundle2: change header size and make them signed (new format) Pierre-Yves David <pierre-yves.david@fb.com> parents: 23008 diff changeset	130
20891 1c6cd23fc221 bundle2: add some distinction between mandatory and advisory part Pierre-Yves David <pierre-yves.david@fb.com> parents: 20890 diff changeset	131 Bundle processing
1c6cd23fc221 bundle2: add some distinction between mandatory and advisory part Pierre-Yves David <pierre-yves.david@fb.com> parents: 20890 diff changeset	132 ============================
1c6cd23fc221 bundle2: add some distinction between mandatory and advisory part Pierre-Yves David <pierre-yves.david@fb.com> parents: 20890 diff changeset	133
1c6cd23fc221 bundle2: add some distinction between mandatory and advisory part Pierre-Yves David <pierre-yves.david@fb.com> parents: 20890 diff changeset	134 Each part is processed in order using a "part handler". Handler are registered
1c6cd23fc221 bundle2: add some distinction between mandatory and advisory part Pierre-Yves David <pierre-yves.david@fb.com> parents: 20890 diff changeset	135 for a certain part type.
1c6cd23fc221 bundle2: add some distinction between mandatory and advisory part Pierre-Yves David <pierre-yves.david@fb.com> parents: 20890 diff changeset	136
1c6cd23fc221 bundle2: add some distinction between mandatory and advisory part Pierre-Yves David <pierre-yves.david@fb.com> parents: 20890 diff changeset	137 The matching of a part to its handler is case insensitive. The case of the
1c6cd23fc221 bundle2: add some distinction between mandatory and advisory part Pierre-Yves David <pierre-yves.david@fb.com> parents: 20890 diff changeset	138 part type is used to know if a part is mandatory or advisory. If the Part type
1c6cd23fc221 bundle2: add some distinction between mandatory and advisory part Pierre-Yves David <pierre-yves.david@fb.com> parents: 20890 diff changeset	139 contains any uppercase char it is considered mandatory. When no handler is
1c6cd23fc221 bundle2: add some distinction between mandatory and advisory part Pierre-Yves David <pierre-yves.david@fb.com> parents: 20890 diff changeset	140 known for a Mandatory part, the process is aborted and an exception is raised.
20892 6fe95448596d bundle2: read the whole bundle from stream on abort Pierre-Yves David <pierre-yves.david@fb.com> parents: 20891 diff changeset	141 If the part is advisory and no handler is known, the part is ignored. When the
6fe95448596d bundle2: read the whole bundle from stream on abort Pierre-Yves David <pierre-yves.david@fb.com> parents: 20891 diff changeset	142 process is aborted, the full bundle is still read from the stream to keep the
6fe95448596d bundle2: read the whole bundle from stream on abort Pierre-Yves David <pierre-yves.david@fb.com> parents: 20891 diff changeset	143 channel usable. But none of the part read from an abort are processed. In the
6fe95448596d bundle2: read the whole bundle from stream on abort Pierre-Yves David <pierre-yves.david@fb.com> parents: 20891 diff changeset	144 future, dropping the stream may become an option for channel we do not care to
6fe95448596d bundle2: read the whole bundle from stream on abort Pierre-Yves David <pierre-yves.david@fb.com> parents: 20891 diff changeset	145 preserve.
20801 9c5183cb9bca bundle2: very first version of a bundle2 bundler Pierre-Yves David <pierre-yves.david@fb.com> parents: diff changeset	146 """
9c5183cb9bca bundle2: very first version of a bundle2 bundler Pierre-Yves David <pierre-yves.david@fb.com> parents: diff changeset	147
24026 3daef83a1873 bundle2.unpackermixin: control for underlying file descriptor Eric Sumner <ericsumner@fb.com> parents: 23916 diff changeset	148 import errno
23067 420a051616ce bundle2: transmit exception during part generation Pierre-Yves David <pierre-yves.david@fb.com> parents: 23066 diff changeset	149 import sys
20802 520df53ad26a bundle2: a very first version of bundle2 unbundler Pierre-Yves David <pierre-yves.david@fb.com> parents: 20801 diff changeset	150 import util
20804 db9d3991d2c6 bundle2: support bundling simple parameter Pierre-Yves David <pierre-yves.david@fb.com> parents: 20803 diff changeset	151 import struct
20811 9785c3f8f598 bundle2: urlquote stream parameter name and value Pierre-Yves David <pierre-yves.david@fb.com> parents: 20810 diff changeset	152 import urllib
20814 8532f5e1b9df bundle2: force the first char of parameter to be an letter. Pierre-Yves David <pierre-yves.david@fb.com> parents: 20813 diff changeset	153 import string
22344 9829b7948100 bundle2: add a `obsmarkersversion` function to extract supported version Pierre-Yves David <pierre-yves.david@fb.com> parents: 22343 diff changeset	154 import obsolete
21655 35095f332846 bundle: introduce a listkey handler Pierre-Yves David <pierre-yves.david@fb.com> parents: 21644 diff changeset	155 import pushkey
23029 149fc8a44184 bundle2: client side support for a part to import external bundles Mike Hommey <mh@glandium.org> parents: 23011 diff changeset	156 import url
23868 405eacbecc52 bundle2: enforce parttype as alphanumerical Pierre-Yves David <pierre-yves.david@fb.com> parents: 23590 diff changeset	157 import re
20804 db9d3991d2c6 bundle2: support bundling simple parameter Pierre-Yves David <pierre-yves.david@fb.com> parents: 20803 diff changeset	158
21184 28d76afa1568 bundle2: fix raising errors during heads checking Pierre-Yves David <pierre-yves.david@fb.com> parents: 21183 diff changeset	159 import changegroup, error
20803 88db3e615319 bundle2: make sure the unbundler refuse non bundle2 stream Pierre-Yves David <pierre-yves.david@fb.com> parents: 20802 diff changeset	160 from i18n import _
20802 520df53ad26a bundle2: a very first version of bundle2 unbundler Pierre-Yves David <pierre-yves.david@fb.com> parents: 20801 diff changeset	161
20804 db9d3991d2c6 bundle2: support bundling simple parameter Pierre-Yves David <pierre-yves.david@fb.com> parents: 20803 diff changeset	162 _pack = struct.pack
db9d3991d2c6 bundle2: support bundling simple parameter Pierre-Yves David <pierre-yves.david@fb.com> parents: 20803 diff changeset	163 _unpack = struct.unpack
db9d3991d2c6 bundle2: support bundling simple parameter Pierre-Yves David <pierre-yves.david@fb.com> parents: 20803 diff changeset	164
23009 90f86ad3d4ff bundle2: change header size and make them signed (new format) Pierre-Yves David <pierre-yves.david@fb.com> parents: 23008 diff changeset	165 _fstreamparamsize = '>i'
90f86ad3d4ff bundle2: change header size and make them signed (new format) Pierre-Yves David <pierre-yves.david@fb.com> parents: 23008 diff changeset	166 _fpartheadersize = '>i'
20856 8a6a86c9a5b5 bundle2: support bundling of empty part (with a type) Pierre-Yves David <pierre-yves.david@fb.com> parents: 20844 diff changeset	167 _fparttypesize = '>B'
20995 e995d104c87f bundle2: add an integer id to part Pierre-Yves David <pierre-yves.david@fb.com> parents: 20952 diff changeset	168 _fpartid = '>I'
23009 90f86ad3d4ff bundle2: change header size and make them signed (new format) Pierre-Yves David <pierre-yves.david@fb.com> parents: 23008 diff changeset	169 _fpayloadsize = '>i'
20877 9e9e3a4e9261 bundle2: part params Pierre-Yves David <pierre-yves.david@fb.com> parents: 20876 diff changeset	170 _fpartparamcount = '>BB'
9e9e3a4e9261 bundle2: part params Pierre-Yves David <pierre-yves.david@fb.com> parents: 20876 diff changeset	171
21001 c93bb6a08fa1 bundle2: support chunk iterator as part data Pierre-Yves David <pierre-yves.david@fb.com> parents: 21000 diff changeset	172 preferedchunksize = 4096
c93bb6a08fa1 bundle2: support chunk iterator as part data Pierre-Yves David <pierre-yves.david@fb.com> parents: 21000 diff changeset	173
23868 405eacbecc52 bundle2: enforce parttype as alphanumerical Pierre-Yves David <pierre-yves.david@fb.com> parents: 23590 diff changeset	174 _parttypeforbidden = re.compile('[^a-zA-Z0-9_:-]')
405eacbecc52 bundle2: enforce parttype as alphanumerical Pierre-Yves David <pierre-yves.david@fb.com> parents: 23590 diff changeset	175
405eacbecc52 bundle2: enforce parttype as alphanumerical Pierre-Yves David <pierre-yves.david@fb.com> parents: 23590 diff changeset	176 def validateparttype(parttype):
405eacbecc52 bundle2: enforce parttype as alphanumerical Pierre-Yves David <pierre-yves.david@fb.com> parents: 23590 diff changeset	177 """raise ValueError if a parttype contains invalid character"""
23916 a3f7c781786b bundle2: fix parttype enforcement Matt Mackall <mpm@selenic.com> parents: 23868 diff changeset	178 if _parttypeforbidden.search(parttype):
23868 405eacbecc52 bundle2: enforce parttype as alphanumerical Pierre-Yves David <pierre-yves.david@fb.com> parents: 23590 diff changeset	179 raise ValueError(parttype)
405eacbecc52 bundle2: enforce parttype as alphanumerical Pierre-Yves David <pierre-yves.david@fb.com> parents: 23590 diff changeset	180
20877 9e9e3a4e9261 bundle2: part params Pierre-Yves David <pierre-yves.david@fb.com> parents: 20876 diff changeset	181 def _makefpartparamsizes(nbparams):
9e9e3a4e9261 bundle2: part params Pierre-Yves David <pierre-yves.david@fb.com> parents: 20876 diff changeset	182 """return a struct format to read part parameter sizes
9e9e3a4e9261 bundle2: part params Pierre-Yves David <pierre-yves.david@fb.com> parents: 20876 diff changeset	183
9e9e3a4e9261 bundle2: part params Pierre-Yves David <pierre-yves.david@fb.com> parents: 20876 diff changeset	184 The number parameters is variable so we need to build that format
9e9e3a4e9261 bundle2: part params Pierre-Yves David <pierre-yves.david@fb.com> parents: 20876 diff changeset	185 dynamically.
9e9e3a4e9261 bundle2: part params Pierre-Yves David <pierre-yves.david@fb.com> parents: 20876 diff changeset	186 """
9e9e3a4e9261 bundle2: part params Pierre-Yves David <pierre-yves.david@fb.com> parents: 20876 diff changeset	187 return '>'+('BB'*nbparams)
20804 db9d3991d2c6 bundle2: support bundling simple parameter Pierre-Yves David <pierre-yves.david@fb.com> parents: 20803 diff changeset	188
20890 ec7fc110faee bundle2: introduce a `parthandler` decorator Pierre-Yves David <pierre-yves.david@fb.com> parents: 20889 diff changeset	189 parthandlermapping = {}
20889 deed5edb72de bundle2: first version of a bundle processing Pierre-Yves David <pierre-yves.david@fb.com> parents: 20887 diff changeset	190
21623 5b26d82e4e2a bundle2: make it possible to declare params handled by a part handler Pierre-Yves David <pierre-yves.david@fb.com> parents: 21622 diff changeset	191 def parthandler(parttype, params=()):
20890 ec7fc110faee bundle2: introduce a `parthandler` decorator Pierre-Yves David <pierre-yves.david@fb.com> parents: 20889 diff changeset	192 """decorator that register a function as a bundle2 part handler
ec7fc110faee bundle2: introduce a `parthandler` decorator Pierre-Yves David <pierre-yves.david@fb.com> parents: 20889 diff changeset	193
ec7fc110faee bundle2: introduce a `parthandler` decorator Pierre-Yves David <pierre-yves.david@fb.com> parents: 20889 diff changeset	194 eg::
ec7fc110faee bundle2: introduce a `parthandler` decorator Pierre-Yves David <pierre-yves.david@fb.com> parents: 20889 diff changeset	195
21624 d61066d787c8 bundle2: declare supported parameters for all handlers Pierre-Yves David <pierre-yves.david@fb.com> parents: 21623 diff changeset	196 @parthandler('myparttype', ('mandatory', 'param', 'handled'))
20890 ec7fc110faee bundle2: introduce a `parthandler` decorator Pierre-Yves David <pierre-yves.david@fb.com> parents: 20889 diff changeset	197 def myparttypehandler(...):
ec7fc110faee bundle2: introduce a `parthandler` decorator Pierre-Yves David <pierre-yves.david@fb.com> parents: 20889 diff changeset	198 '''process a part of type "my part".'''
ec7fc110faee bundle2: introduce a `parthandler` decorator Pierre-Yves David <pierre-yves.david@fb.com> parents: 20889 diff changeset	199 ...
ec7fc110faee bundle2: introduce a `parthandler` decorator Pierre-Yves David <pierre-yves.david@fb.com> parents: 20889 diff changeset	200 """
23868 405eacbecc52 bundle2: enforce parttype as alphanumerical Pierre-Yves David <pierre-yves.david@fb.com> parents: 23590 diff changeset	201 validateparttype(parttype)
20890 ec7fc110faee bundle2: introduce a `parthandler` decorator Pierre-Yves David <pierre-yves.david@fb.com> parents: 20889 diff changeset	202 def _decorator(func):
20891 1c6cd23fc221 bundle2: add some distinction between mandatory and advisory part Pierre-Yves David <pierre-yves.david@fb.com> parents: 20890 diff changeset	203 lparttype = parttype.lower() # enforce lower case matching.
1c6cd23fc221 bundle2: add some distinction between mandatory and advisory part Pierre-Yves David <pierre-yves.david@fb.com> parents: 20890 diff changeset	204 assert lparttype not in parthandlermapping
1c6cd23fc221 bundle2: add some distinction between mandatory and advisory part Pierre-Yves David <pierre-yves.david@fb.com> parents: 20890 diff changeset	205 parthandlermapping[lparttype] = func
21623 5b26d82e4e2a bundle2: make it possible to declare params handled by a part handler Pierre-Yves David <pierre-yves.david@fb.com> parents: 21622 diff changeset	206 func.params = frozenset(params)
20890 ec7fc110faee bundle2: introduce a `parthandler` decorator Pierre-Yves David <pierre-yves.david@fb.com> parents: 20889 diff changeset	207 return func
ec7fc110faee bundle2: introduce a `parthandler` decorator Pierre-Yves David <pierre-yves.david@fb.com> parents: 20889 diff changeset	208 return _decorator
20889 deed5edb72de bundle2: first version of a bundle processing Pierre-Yves David <pierre-yves.david@fb.com> parents: 20887 diff changeset	209
20949 571f2903ff1e bundle2: record processing results in the bundleoperation object Pierre-Yves David <pierre-yves.david@fb.com> parents: 20948 diff changeset	210 class unbundlerecords(object):
571f2903ff1e bundle2: record processing results in the bundleoperation object Pierre-Yves David <pierre-yves.david@fb.com> parents: 20948 diff changeset	211 """keep record of what happens during and unbundle
571f2903ff1e bundle2: record processing results in the bundleoperation object Pierre-Yves David <pierre-yves.david@fb.com> parents: 20948 diff changeset	212
571f2903ff1e bundle2: record processing results in the bundleoperation object Pierre-Yves David <pierre-yves.david@fb.com> parents: 20948 diff changeset	213 New records are added using `records.add('cat', obj)`. Where 'cat' is a
21024 7731a2281cf0 spelling: fixes from spell checker Mads Kiilerich <madski@unity3d.com> parents: 21020 diff changeset	214 category of record and obj is an arbitrary object.
20949 571f2903ff1e bundle2: record processing results in the bundleoperation object Pierre-Yves David <pierre-yves.david@fb.com> parents: 20948 diff changeset	215
571f2903ff1e bundle2: record processing results in the bundleoperation object Pierre-Yves David <pierre-yves.david@fb.com> parents: 20948 diff changeset	216 `records['cat']` will return all entries of this category 'cat'.
571f2903ff1e bundle2: record processing results in the bundleoperation object Pierre-Yves David <pierre-yves.david@fb.com> parents: 20948 diff changeset	217
571f2903ff1e bundle2: record processing results in the bundleoperation object Pierre-Yves David <pierre-yves.david@fb.com> parents: 20948 diff changeset	218 Iterating on the object itself will yield `('category', obj)` tuples
571f2903ff1e bundle2: record processing results in the bundleoperation object Pierre-Yves David <pierre-yves.david@fb.com> parents: 20948 diff changeset	219 for all entries.
571f2903ff1e bundle2: record processing results in the bundleoperation object Pierre-Yves David <pierre-yves.david@fb.com> parents: 20948 diff changeset	220
571f2903ff1e bundle2: record processing results in the bundleoperation object Pierre-Yves David <pierre-yves.david@fb.com> parents: 20948 diff changeset	221 All iterations happens in chronological order.
571f2903ff1e bundle2: record processing results in the bundleoperation object Pierre-Yves David <pierre-yves.david@fb.com> parents: 20948 diff changeset	222 """
571f2903ff1e bundle2: record processing results in the bundleoperation object Pierre-Yves David <pierre-yves.david@fb.com> parents: 20948 diff changeset	223
571f2903ff1e bundle2: record processing results in the bundleoperation object Pierre-Yves David <pierre-yves.david@fb.com> parents: 20948 diff changeset	224 def __init__(self):
571f2903ff1e bundle2: record processing results in the bundleoperation object Pierre-Yves David <pierre-yves.david@fb.com> parents: 20948 diff changeset	225 self._categories = {}
571f2903ff1e bundle2: record processing results in the bundleoperation object Pierre-Yves David <pierre-yves.david@fb.com> parents: 20948 diff changeset	226 self._sequences = []
20996 ed3c5e18a047 bundle2: add reply awareness to unbundlerecords Pierre-Yves David <pierre-yves.david@fb.com> parents: 20995 diff changeset	227 self._replies = {}
20949 571f2903ff1e bundle2: record processing results in the bundleoperation object Pierre-Yves David <pierre-yves.david@fb.com> parents: 20948 diff changeset	228
20996 ed3c5e18a047 bundle2: add reply awareness to unbundlerecords Pierre-Yves David <pierre-yves.david@fb.com> parents: 20995 diff changeset	229 def add(self, category, entry, inreplyto=None):
20949 571f2903ff1e bundle2: record processing results in the bundleoperation object Pierre-Yves David <pierre-yves.david@fb.com> parents: 20948 diff changeset	230 """add a new record of a given category.
571f2903ff1e bundle2: record processing results in the bundleoperation object Pierre-Yves David <pierre-yves.david@fb.com> parents: 20948 diff changeset	231
571f2903ff1e bundle2: record processing results in the bundleoperation object Pierre-Yves David <pierre-yves.david@fb.com> parents: 20948 diff changeset	232 The entry can then be retrieved in the list returned by
571f2903ff1e bundle2: record processing results in the bundleoperation object Pierre-Yves David <pierre-yves.david@fb.com> parents: 20948 diff changeset	233 self['category']."""
571f2903ff1e bundle2: record processing results in the bundleoperation object Pierre-Yves David <pierre-yves.david@fb.com> parents: 20948 diff changeset	234 self._categories.setdefault(category, []).append(entry)
571f2903ff1e bundle2: record processing results in the bundleoperation object Pierre-Yves David <pierre-yves.david@fb.com> parents: 20948 diff changeset	235 self._sequences.append((category, entry))
20996 ed3c5e18a047 bundle2: add reply awareness to unbundlerecords Pierre-Yves David <pierre-yves.david@fb.com> parents: 20995 diff changeset	236 if inreplyto is not None:
ed3c5e18a047 bundle2: add reply awareness to unbundlerecords Pierre-Yves David <pierre-yves.david@fb.com> parents: 20995 diff changeset	237 self.getreplies(inreplyto).add(category, entry)
ed3c5e18a047 bundle2: add reply awareness to unbundlerecords Pierre-Yves David <pierre-yves.david@fb.com> parents: 20995 diff changeset	238
ed3c5e18a047 bundle2: add reply awareness to unbundlerecords Pierre-Yves David <pierre-yves.david@fb.com> parents: 20995 diff changeset	239 def getreplies(self, partid):
23139 e53f6b72a0e4 spelling: fixes from proofreading of spell checker issues Mads Kiilerich <madski@unity3d.com> parents: 23067 diff changeset	240 """get the records that are replies to a specific part"""
20996 ed3c5e18a047 bundle2: add reply awareness to unbundlerecords Pierre-Yves David <pierre-yves.david@fb.com> parents: 20995 diff changeset	241 return self._replies.setdefault(partid, unbundlerecords())
20949 571f2903ff1e bundle2: record processing results in the bundleoperation object Pierre-Yves David <pierre-yves.david@fb.com> parents: 20948 diff changeset	242
571f2903ff1e bundle2: record processing results in the bundleoperation object Pierre-Yves David <pierre-yves.david@fb.com> parents: 20948 diff changeset	243 def __getitem__(self, cat):
571f2903ff1e bundle2: record processing results in the bundleoperation object Pierre-Yves David <pierre-yves.david@fb.com> parents: 20948 diff changeset	244 return tuple(self._categories.get(cat, ()))
571f2903ff1e bundle2: record processing results in the bundleoperation object Pierre-Yves David <pierre-yves.david@fb.com> parents: 20948 diff changeset	245
571f2903ff1e bundle2: record processing results in the bundleoperation object Pierre-Yves David <pierre-yves.david@fb.com> parents: 20948 diff changeset	246 def __iter__(self):
571f2903ff1e bundle2: record processing results in the bundleoperation object Pierre-Yves David <pierre-yves.david@fb.com> parents: 20948 diff changeset	247 return iter(self._sequences)
571f2903ff1e bundle2: record processing results in the bundleoperation object Pierre-Yves David <pierre-yves.david@fb.com> parents: 20948 diff changeset	248
571f2903ff1e bundle2: record processing results in the bundleoperation object Pierre-Yves David <pierre-yves.david@fb.com> parents: 20948 diff changeset	249 def __len__(self):
571f2903ff1e bundle2: record processing results in the bundleoperation object Pierre-Yves David <pierre-yves.david@fb.com> parents: 20948 diff changeset	250 return len(self._sequences)
571f2903ff1e bundle2: record processing results in the bundleoperation object Pierre-Yves David <pierre-yves.david@fb.com> parents: 20948 diff changeset	251
571f2903ff1e bundle2: record processing results in the bundleoperation object Pierre-Yves David <pierre-yves.david@fb.com> parents: 20948 diff changeset	252 def __nonzero__(self):
571f2903ff1e bundle2: record processing results in the bundleoperation object Pierre-Yves David <pierre-yves.david@fb.com> parents: 20948 diff changeset	253 return bool(self._sequences)
571f2903ff1e bundle2: record processing results in the bundleoperation object Pierre-Yves David <pierre-yves.david@fb.com> parents: 20948 diff changeset	254
20948 329cd74b52bd bundle2: introduce a bundleoperation object Pierre-Yves David <pierre-yves.david@fb.com> parents: 20947 diff changeset	255 class bundleoperation(object):
329cd74b52bd bundle2: introduce a bundleoperation object Pierre-Yves David <pierre-yves.david@fb.com> parents: 20947 diff changeset	256 """an object that represents a single bundling process
329cd74b52bd bundle2: introduce a bundleoperation object Pierre-Yves David <pierre-yves.david@fb.com> parents: 20947 diff changeset	257
329cd74b52bd bundle2: introduce a bundleoperation object Pierre-Yves David <pierre-yves.david@fb.com> parents: 20947 diff changeset	258 Its purpose is to carry unbundle-related objects and states.
329cd74b52bd bundle2: introduce a bundleoperation object Pierre-Yves David <pierre-yves.david@fb.com> parents: 20947 diff changeset	259
329cd74b52bd bundle2: introduce a bundleoperation object Pierre-Yves David <pierre-yves.david@fb.com> parents: 20947 diff changeset	260 A new object should be created at the beginning of each bundle processing.
329cd74b52bd bundle2: introduce a bundleoperation object Pierre-Yves David <pierre-yves.david@fb.com> parents: 20947 diff changeset	261 The object is to be returned by the processing function.
329cd74b52bd bundle2: introduce a bundleoperation object Pierre-Yves David <pierre-yves.david@fb.com> parents: 20947 diff changeset	262
329cd74b52bd bundle2: introduce a bundleoperation object Pierre-Yves David <pierre-yves.david@fb.com> parents: 20947 diff changeset	263 The object has very little content now it will ultimately contain:
329cd74b52bd bundle2: introduce a bundleoperation object Pierre-Yves David <pierre-yves.david@fb.com> parents: 20947 diff changeset	264 * an access to the repo the bundle is applied to,
329cd74b52bd bundle2: introduce a bundleoperation object Pierre-Yves David <pierre-yves.david@fb.com> parents: 20947 diff changeset	265 * a ui object,
329cd74b52bd bundle2: introduce a bundleoperation object Pierre-Yves David <pierre-yves.david@fb.com> parents: 20947 diff changeset	266 * a way to retrieve a transaction to add changes to the repo,
329cd74b52bd bundle2: introduce a bundleoperation object Pierre-Yves David <pierre-yves.david@fb.com> parents: 20947 diff changeset	267 * a way to record the result of processing each part,
329cd74b52bd bundle2: introduce a bundleoperation object Pierre-Yves David <pierre-yves.david@fb.com> parents: 20947 diff changeset	268 * a way to construct a bundle response when applicable.
329cd74b52bd bundle2: introduce a bundleoperation object Pierre-Yves David <pierre-yves.david@fb.com> parents: 20947 diff changeset	269 """
329cd74b52bd bundle2: introduce a bundleoperation object Pierre-Yves David <pierre-yves.david@fb.com> parents: 20947 diff changeset	270
20952 b24ee5076b94 bundle2: make it possible have a global transaction for the unbundling Pierre-Yves David <pierre-yves.david@fb.com> parents: 20950 diff changeset	271 def __init__(self, repo, transactiongetter):
20948 329cd74b52bd bundle2: introduce a bundleoperation object Pierre-Yves David <pierre-yves.david@fb.com> parents: 20947 diff changeset	272 self.repo = repo
329cd74b52bd bundle2: introduce a bundleoperation object Pierre-Yves David <pierre-yves.david@fb.com> parents: 20947 diff changeset	273 self.ui = repo.ui
20949 571f2903ff1e bundle2: record processing results in the bundleoperation object Pierre-Yves David <pierre-yves.david@fb.com> parents: 20948 diff changeset	274 self.records = unbundlerecords()
20952 b24ee5076b94 bundle2: make it possible have a global transaction for the unbundling Pierre-Yves David <pierre-yves.david@fb.com> parents: 20950 diff changeset	275 self.gettransaction = transactiongetter
20997 d7df4b7378ae bundle2: produce a bundle2 reply Pierre-Yves David <pierre-yves.david@fb.com> parents: 20996 diff changeset	276 self.reply = None
20948 329cd74b52bd bundle2: introduce a bundleoperation object Pierre-Yves David <pierre-yves.david@fb.com> parents: 20947 diff changeset	277
20952 b24ee5076b94 bundle2: make it possible have a global transaction for the unbundling Pierre-Yves David <pierre-yves.david@fb.com> parents: 20950 diff changeset	278 class TransactionUnavailable(RuntimeError):
b24ee5076b94 bundle2: make it possible have a global transaction for the unbundling Pierre-Yves David <pierre-yves.david@fb.com> parents: 20950 diff changeset	279 pass
b24ee5076b94 bundle2: make it possible have a global transaction for the unbundling Pierre-Yves David <pierre-yves.david@fb.com> parents: 20950 diff changeset	280
b24ee5076b94 bundle2: make it possible have a global transaction for the unbundling Pierre-Yves David <pierre-yves.david@fb.com> parents: 20950 diff changeset	281 def _notransaction():
b24ee5076b94 bundle2: make it possible have a global transaction for the unbundling Pierre-Yves David <pierre-yves.david@fb.com> parents: 20950 diff changeset	282 """default method to get a transaction while processing a bundle
b24ee5076b94 bundle2: make it possible have a global transaction for the unbundling Pierre-Yves David <pierre-yves.david@fb.com> parents: 20950 diff changeset	283
b24ee5076b94 bundle2: make it possible have a global transaction for the unbundling Pierre-Yves David <pierre-yves.david@fb.com> parents: 20950 diff changeset	284 Raise an exception to highlight the fact that no transaction was expected
b24ee5076b94 bundle2: make it possible have a global transaction for the unbundling Pierre-Yves David <pierre-yves.david@fb.com> parents: 20950 diff changeset	285 to be created"""
b24ee5076b94 bundle2: make it possible have a global transaction for the unbundling Pierre-Yves David <pierre-yves.david@fb.com> parents: 20950 diff changeset	286 raise TransactionUnavailable()
b24ee5076b94 bundle2: make it possible have a global transaction for the unbundling Pierre-Yves David <pierre-yves.david@fb.com> parents: 20950 diff changeset	287
23438 6e0ecb9a2e19 bundle2.processbundle: let callers request default behavior Eric Sumner <ericsumner@fb.com> parents: 23207 diff changeset	288 def processbundle(repo, unbundler, transactiongetter=None):
20889 deed5edb72de bundle2: first version of a bundle processing Pierre-Yves David <pierre-yves.david@fb.com> parents: 20887 diff changeset	289 """This function process a bundle, apply effect to/from a repo
deed5edb72de bundle2: first version of a bundle processing Pierre-Yves David <pierre-yves.david@fb.com> parents: 20887 diff changeset	290
20947 c33d7bf53812 bundle2: feed a unbundle20 to the `processbundle` function Pierre-Yves David <pierre-yves.david@fb.com> parents: 20892 diff changeset	291 It iterates over each part then searches for and uses the proper handling
c33d7bf53812 bundle2: feed a unbundle20 to the `processbundle` function Pierre-Yves David <pierre-yves.david@fb.com> parents: 20892 diff changeset	292 code to process the part. Parts are processed in order.
20889 deed5edb72de bundle2: first version of a bundle processing Pierre-Yves David <pierre-yves.david@fb.com> parents: 20887 diff changeset	293
deed5edb72de bundle2: first version of a bundle processing Pierre-Yves David <pierre-yves.david@fb.com> parents: 20887 diff changeset	294 This is very early version of this function that will be strongly reworked
deed5edb72de bundle2: first version of a bundle processing Pierre-Yves David <pierre-yves.david@fb.com> parents: 20887 diff changeset	295 before final usage.
deed5edb72de bundle2: first version of a bundle processing Pierre-Yves David <pierre-yves.david@fb.com> parents: 20887 diff changeset	296
20891 1c6cd23fc221 bundle2: add some distinction between mandatory and advisory part Pierre-Yves David <pierre-yves.david@fb.com> parents: 20890 diff changeset	297 Unknown Mandatory part will abort the process.
20889 deed5edb72de bundle2: first version of a bundle processing Pierre-Yves David <pierre-yves.david@fb.com> parents: 20887 diff changeset	298 """
23438 6e0ecb9a2e19 bundle2.processbundle: let callers request default behavior Eric Sumner <ericsumner@fb.com> parents: 23207 diff changeset	299 if transactiongetter is None:
6e0ecb9a2e19 bundle2.processbundle: let callers request default behavior Eric Sumner <ericsumner@fb.com> parents: 23207 diff changeset	300 transactiongetter = _notransaction
20952 b24ee5076b94 bundle2: make it possible have a global transaction for the unbundling Pierre-Yves David <pierre-yves.david@fb.com> parents: 20950 diff changeset	301 op = bundleoperation(repo, transactiongetter)
20889 deed5edb72de bundle2: first version of a bundle processing Pierre-Yves David <pierre-yves.david@fb.com> parents: 20887 diff changeset	302 # todo:
deed5edb72de bundle2: first version of a bundle processing Pierre-Yves David <pierre-yves.david@fb.com> parents: 20887 diff changeset	303 # - replace this is a init function soon.
deed5edb72de bundle2: first version of a bundle processing Pierre-Yves David <pierre-yves.david@fb.com> parents: 20887 diff changeset	304 # - exception catching
deed5edb72de bundle2: first version of a bundle processing Pierre-Yves David <pierre-yves.david@fb.com> parents: 20887 diff changeset	305 unbundler.params
21129 07bcbf326c8d bundle2: use an official iterparts method to unbundle parts Pierre-Yves David <pierre-yves.david@fb.com> parents: 21066 diff changeset	306 iterparts = unbundler.iterparts()
21019 3dc09f831a2e bundle2: lazy unbundle of part payload Pierre-Yves David <pierre-yves.david@fb.com> parents: 21016 diff changeset	307 part = None
20892 6fe95448596d bundle2: read the whole bundle from stream on abort Pierre-Yves David <pierre-yves.david@fb.com> parents: 20891 diff changeset	308 try:
6fe95448596d bundle2: read the whole bundle from stream on abort Pierre-Yves David <pierre-yves.david@fb.com> parents: 20891 diff changeset	309 for part in iterparts:
23008 d3137827016a bundle2: extract processing of part into its own function Pierre-Yves David <pierre-yves.david@fb.com> parents: 23001 diff changeset	310 _processpart(op, part)
21176 70fcb0a71445 bundle2: decorate exception raised during bundle processing Pierre-Yves David <pierre-yves.david@fb.com> parents: 21146 diff changeset	311 except Exception, exc:
20892 6fe95448596d bundle2: read the whole bundle from stream on abort Pierre-Yves David <pierre-yves.david@fb.com> parents: 20891 diff changeset	312 for part in iterparts:
21019 3dc09f831a2e bundle2: lazy unbundle of part payload Pierre-Yves David <pierre-yves.david@fb.com> parents: 21016 diff changeset	313 # consume the bundle content
24047 731fa8e3e580 bundle2: now that we have a seek implementation, use it Eric Sumner <ericsumner@fb.com> parents: 24037 diff changeset	314 part.seek(0, 2)
21176 70fcb0a71445 bundle2: decorate exception raised during bundle processing Pierre-Yves David <pierre-yves.david@fb.com> parents: 21146 diff changeset	315 # Small hack to let caller code distinguish exceptions from bundle2
23139 e53f6b72a0e4 spelling: fixes from proofreading of spell checker issues Mads Kiilerich <madski@unity3d.com> parents: 23067 diff changeset	316 # processing from processing the old format. This is mostly
21176 70fcb0a71445 bundle2: decorate exception raised during bundle processing Pierre-Yves David <pierre-yves.david@fb.com> parents: 21146 diff changeset	317 # needed to handle different return codes to unbundle according to the
70fcb0a71445 bundle2: decorate exception raised during bundle processing Pierre-Yves David <pierre-yves.david@fb.com> parents: 21146 diff changeset	318 # type of bundle. We should probably clean up or drop this return code
70fcb0a71445 bundle2: decorate exception raised during bundle processing Pierre-Yves David <pierre-yves.david@fb.com> parents: 21146 diff changeset	319 # craziness in a future version.
70fcb0a71445 bundle2: decorate exception raised during bundle processing Pierre-Yves David <pierre-yves.david@fb.com> parents: 21146 diff changeset	320 exc.duringunbundle2 = True
20892 6fe95448596d bundle2: read the whole bundle from stream on abort Pierre-Yves David <pierre-yves.david@fb.com> parents: 20891 diff changeset	321 raise
20949 571f2903ff1e bundle2: record processing results in the bundleoperation object Pierre-Yves David <pierre-yves.david@fb.com> parents: 20948 diff changeset	322 return op
20889 deed5edb72de bundle2: first version of a bundle processing Pierre-Yves David <pierre-yves.david@fb.com> parents: 20887 diff changeset	323
23008 d3137827016a bundle2: extract processing of part into its own function Pierre-Yves David <pierre-yves.david@fb.com> parents: 23001 diff changeset	324 def _processpart(op, part):
d3137827016a bundle2: extract processing of part into its own function Pierre-Yves David <pierre-yves.david@fb.com> parents: 23001 diff changeset	325 """process a single part from a bundle
d3137827016a bundle2: extract processing of part into its own function Pierre-Yves David <pierre-yves.david@fb.com> parents: 23001 diff changeset	326
d3137827016a bundle2: extract processing of part into its own function Pierre-Yves David <pierre-yves.david@fb.com> parents: 23001 diff changeset	327 The part is guaranteed to have been fully consumed when the function exits
d3137827016a bundle2: extract processing of part into its own function Pierre-Yves David <pierre-yves.david@fb.com> parents: 23001 diff changeset	328 (even if an exception is raised)."""
d3137827016a bundle2: extract processing of part into its own function Pierre-Yves David <pierre-yves.david@fb.com> parents: 23001 diff changeset	329 try:
d3137827016a bundle2: extract processing of part into its own function Pierre-Yves David <pierre-yves.david@fb.com> parents: 23001 diff changeset	330 try:
23586 112f9c73a0e5 bundle2._processpart: forcing lower-case compare is no longer necessary Eric Sumner <ericsumner@fb.com> parents: 23585 diff changeset	331 handler = parthandlermapping.get(part.type)
23008 d3137827016a bundle2: extract processing of part into its own function Pierre-Yves David <pierre-yves.david@fb.com> parents: 23001 diff changeset	332 if handler is None:
23586 112f9c73a0e5 bundle2._processpart: forcing lower-case compare is no longer necessary Eric Sumner <ericsumner@fb.com> parents: 23585 diff changeset	333 raise error.UnsupportedPartError(parttype=part.type)
112f9c73a0e5 bundle2._processpart: forcing lower-case compare is no longer necessary Eric Sumner <ericsumner@fb.com> parents: 23585 diff changeset	334 op.ui.debug('found a handler for part %r\n' % part.type)
23008 d3137827016a bundle2: extract processing of part into its own function Pierre-Yves David <pierre-yves.david@fb.com> parents: 23001 diff changeset	335 unknownparams = part.mandatorykeys - handler.params
d3137827016a bundle2: extract processing of part into its own function Pierre-Yves David <pierre-yves.david@fb.com> parents: 23001 diff changeset	336 if unknownparams:
d3137827016a bundle2: extract processing of part into its own function Pierre-Yves David <pierre-yves.david@fb.com> parents: 23001 diff changeset	337 unknownparams = list(unknownparams)
d3137827016a bundle2: extract processing of part into its own function Pierre-Yves David <pierre-yves.david@fb.com> parents: 23001 diff changeset	338 unknownparams.sort()
23586 112f9c73a0e5 bundle2._processpart: forcing lower-case compare is no longer necessary Eric Sumner <ericsumner@fb.com> parents: 23585 diff changeset	339 raise error.UnsupportedPartError(parttype=part.type,
23008 d3137827016a bundle2: extract processing of part into its own function Pierre-Yves David <pierre-yves.david@fb.com> parents: 23001 diff changeset	340 params=unknownparams)
23010 73f394f4affc bundle2: add an UnsupportedPartError Pierre-Yves David <pierre-yves.david@fb.com> parents: 23009 diff changeset	341 except error.UnsupportedPartError, exc:
23585 94b25d71dd0f bundle2.unbundlepart: decouple mandatory from parttype Eric Sumner <ericsumner@fb.com> parents: 23439 diff changeset	342 if part.mandatory: # mandatory parts
23008 d3137827016a bundle2: extract processing of part into its own function Pierre-Yves David <pierre-yves.david@fb.com> parents: 23001 diff changeset	343 raise
d3137827016a bundle2: extract processing of part into its own function Pierre-Yves David <pierre-yves.david@fb.com> parents: 23001 diff changeset	344 op.ui.debug('ignoring unsupported advisory part %s\n' % exc)
d3137827016a bundle2: extract processing of part into its own function Pierre-Yves David <pierre-yves.david@fb.com> parents: 23001 diff changeset	345 return # skip to part processing
d3137827016a bundle2: extract processing of part into its own function Pierre-Yves David <pierre-yves.david@fb.com> parents: 23001 diff changeset	346
d3137827016a bundle2: extract processing of part into its own function Pierre-Yves David <pierre-yves.david@fb.com> parents: 23001 diff changeset	347 # handler is called outside the above try block so that we don't
d3137827016a bundle2: extract processing of part into its own function Pierre-Yves David <pierre-yves.david@fb.com> parents: 23001 diff changeset	348 # risk catching KeyErrors from anything other than the
d3137827016a bundle2: extract processing of part into its own function Pierre-Yves David <pierre-yves.david@fb.com> parents: 23001 diff changeset	349 # parthandlermapping lookup (any KeyError raised by handler()
d3137827016a bundle2: extract processing of part into its own function Pierre-Yves David <pierre-yves.david@fb.com> parents: 23001 diff changeset	350 # itself represents a defect of a different variety).
d3137827016a bundle2: extract processing of part into its own function Pierre-Yves David <pierre-yves.david@fb.com> parents: 23001 diff changeset	351 output = None
d3137827016a bundle2: extract processing of part into its own function Pierre-Yves David <pierre-yves.david@fb.com> parents: 23001 diff changeset	352 if op.reply is not None:
d3137827016a bundle2: extract processing of part into its own function Pierre-Yves David <pierre-yves.david@fb.com> parents: 23001 diff changeset	353 op.ui.pushbuffer(error=True)
d3137827016a bundle2: extract processing of part into its own function Pierre-Yves David <pierre-yves.david@fb.com> parents: 23001 diff changeset	354 output = ''
d3137827016a bundle2: extract processing of part into its own function Pierre-Yves David <pierre-yves.david@fb.com> parents: 23001 diff changeset	355 try:
d3137827016a bundle2: extract processing of part into its own function Pierre-Yves David <pierre-yves.david@fb.com> parents: 23001 diff changeset	356 handler(op, part)
d3137827016a bundle2: extract processing of part into its own function Pierre-Yves David <pierre-yves.david@fb.com> parents: 23001 diff changeset	357 finally:
d3137827016a bundle2: extract processing of part into its own function Pierre-Yves David <pierre-yves.david@fb.com> parents: 23001 diff changeset	358 if output is not None:
d3137827016a bundle2: extract processing of part into its own function Pierre-Yves David <pierre-yves.david@fb.com> parents: 23001 diff changeset	359 output = op.ui.popbuffer()
24743 a2ef1dc3b59b bundle2: flush output in a part in all cases Pierre-Yves David <pierre-yves.david@fb.com> parents: 24741 diff changeset	360 if output:
a2ef1dc3b59b bundle2: flush output in a part in all cases Pierre-Yves David <pierre-yves.david@fb.com> parents: 24741 diff changeset	361 outpart = op.reply.newpart('output', data=output,
a2ef1dc3b59b bundle2: flush output in a part in all cases Pierre-Yves David <pierre-yves.david@fb.com> parents: 24741 diff changeset	362 mandatory=False)
a2ef1dc3b59b bundle2: flush output in a part in all cases Pierre-Yves David <pierre-yves.david@fb.com> parents: 24741 diff changeset	363 outpart.addparam('in-reply-to', str(part.id), mandatory=False)
23008 d3137827016a bundle2: extract processing of part into its own function Pierre-Yves David <pierre-yves.david@fb.com> parents: 23001 diff changeset	364 finally:
d3137827016a bundle2: extract processing of part into its own function Pierre-Yves David <pierre-yves.david@fb.com> parents: 23001 diff changeset	365 # consume the part content to not corrupt the stream.
24047 731fa8e3e580 bundle2: now that we have a seek implementation, use it Eric Sumner <ericsumner@fb.com> parents: 24037 diff changeset	366 part.seek(0, 2)
23008 d3137827016a bundle2: extract processing of part into its own function Pierre-Yves David <pierre-yves.david@fb.com> parents: 23001 diff changeset	367
d3137827016a bundle2: extract processing of part into its own function Pierre-Yves David <pierre-yves.david@fb.com> parents: 23001 diff changeset	368
21138 f469879d27ec bundle2: extract capabilities decoding Pierre-Yves David <pierre-yves.david@fb.com> parents: 21137 diff changeset	369 def decodecaps(blob):
23139 e53f6b72a0e4 spelling: fixes from proofreading of spell checker issues Mads Kiilerich <madski@unity3d.com> parents: 23067 diff changeset	370 """decode a bundle2 caps bytes blob into a dictionary
21138 f469879d27ec bundle2: extract capabilities decoding Pierre-Yves David <pierre-yves.david@fb.com> parents: 21137 diff changeset	371
f469879d27ec bundle2: extract capabilities decoding Pierre-Yves David <pierre-yves.david@fb.com> parents: 21137 diff changeset	372 The blob is a list of capabilities (one per line)
f469879d27ec bundle2: extract capabilities decoding Pierre-Yves David <pierre-yves.david@fb.com> parents: 21137 diff changeset	373 Capabilities may have values using a line of the form::
f469879d27ec bundle2: extract capabilities decoding Pierre-Yves David <pierre-yves.david@fb.com> parents: 21137 diff changeset	374
f469879d27ec bundle2: extract capabilities decoding Pierre-Yves David <pierre-yves.david@fb.com> parents: 21137 diff changeset	375 capability=value1,value2,value3
f469879d27ec bundle2: extract capabilities decoding Pierre-Yves David <pierre-yves.david@fb.com> parents: 21137 diff changeset	376
f469879d27ec bundle2: extract capabilities decoding Pierre-Yves David <pierre-yves.david@fb.com> parents: 21137 diff changeset	377 The values are always a list."""
f469879d27ec bundle2: extract capabilities decoding Pierre-Yves David <pierre-yves.david@fb.com> parents: 21137 diff changeset	378 caps = {}
f469879d27ec bundle2: extract capabilities decoding Pierre-Yves David <pierre-yves.david@fb.com> parents: 21137 diff changeset	379 for line in blob.splitlines():
f469879d27ec bundle2: extract capabilities decoding Pierre-Yves David <pierre-yves.david@fb.com> parents: 21137 diff changeset	380 if not line:
f469879d27ec bundle2: extract capabilities decoding Pierre-Yves David <pierre-yves.david@fb.com> parents: 21137 diff changeset	381 continue
f469879d27ec bundle2: extract capabilities decoding Pierre-Yves David <pierre-yves.david@fb.com> parents: 21137 diff changeset	382 if '=' not in line:
f469879d27ec bundle2: extract capabilities decoding Pierre-Yves David <pierre-yves.david@fb.com> parents: 21137 diff changeset	383 key, vals = line, ()
f469879d27ec bundle2: extract capabilities decoding Pierre-Yves David <pierre-yves.david@fb.com> parents: 21137 diff changeset	384 else:
f469879d27ec bundle2: extract capabilities decoding Pierre-Yves David <pierre-yves.david@fb.com> parents: 21137 diff changeset	385 key, vals = line.split('=', 1)
f469879d27ec bundle2: extract capabilities decoding Pierre-Yves David <pierre-yves.david@fb.com> parents: 21137 diff changeset	386 vals = vals.split(',')
f469879d27ec bundle2: extract capabilities decoding Pierre-Yves David <pierre-yves.david@fb.com> parents: 21137 diff changeset	387 key = urllib.unquote(key)
f469879d27ec bundle2: extract capabilities decoding Pierre-Yves David <pierre-yves.david@fb.com> parents: 21137 diff changeset	388 vals = [urllib.unquote(v) for v in vals]
f469879d27ec bundle2: extract capabilities decoding Pierre-Yves David <pierre-yves.david@fb.com> parents: 21137 diff changeset	389 caps[key] = vals
f469879d27ec bundle2: extract capabilities decoding Pierre-Yves David <pierre-yves.david@fb.com> parents: 21137 diff changeset	390 return caps
f469879d27ec bundle2: extract capabilities decoding Pierre-Yves David <pierre-yves.david@fb.com> parents: 21137 diff changeset	391
21139 2b8c82f7f11d bundle2: capabilities encoding Pierre-Yves David <pierre-yves.david@fb.com> parents: 21138 diff changeset	392 def encodecaps(caps):
2b8c82f7f11d bundle2: capabilities encoding Pierre-Yves David <pierre-yves.david@fb.com> parents: 21138 diff changeset	393 """encode a bundle2 caps dictionary into a bytes blob"""
2b8c82f7f11d bundle2: capabilities encoding Pierre-Yves David <pierre-yves.david@fb.com> parents: 21138 diff changeset	394 chunks = []
2b8c82f7f11d bundle2: capabilities encoding Pierre-Yves David <pierre-yves.david@fb.com> parents: 21138 diff changeset	395 for ca in sorted(caps):
2b8c82f7f11d bundle2: capabilities encoding Pierre-Yves David <pierre-yves.david@fb.com> parents: 21138 diff changeset	396 vals = caps[ca]
2b8c82f7f11d bundle2: capabilities encoding Pierre-Yves David <pierre-yves.david@fb.com> parents: 21138 diff changeset	397 ca = urllib.quote(ca)
2b8c82f7f11d bundle2: capabilities encoding Pierre-Yves David <pierre-yves.david@fb.com> parents: 21138 diff changeset	398 vals = [urllib.quote(v) for v in vals]
2b8c82f7f11d bundle2: capabilities encoding Pierre-Yves David <pierre-yves.david@fb.com> parents: 21138 diff changeset	399 if vals:
2b8c82f7f11d bundle2: capabilities encoding Pierre-Yves David <pierre-yves.david@fb.com> parents: 21138 diff changeset	400 ca = "%s=%s" % (ca, ','.join(vals))
2b8c82f7f11d bundle2: capabilities encoding Pierre-Yves David <pierre-yves.david@fb.com> parents: 21138 diff changeset	401 chunks.append(ca)
2b8c82f7f11d bundle2: capabilities encoding Pierre-Yves David <pierre-yves.david@fb.com> parents: 21138 diff changeset	402 return '\n'.join(chunks)
2b8c82f7f11d bundle2: capabilities encoding Pierre-Yves David <pierre-yves.david@fb.com> parents: 21138 diff changeset	403
20801 9c5183cb9bca bundle2: very first version of a bundle2 bundler Pierre-Yves David <pierre-yves.david@fb.com> parents: diff changeset	404 class bundle20(object):
9c5183cb9bca bundle2: very first version of a bundle2 bundler Pierre-Yves David <pierre-yves.david@fb.com> parents: diff changeset	405 """represent an outgoing bundle2 container
9c5183cb9bca bundle2: very first version of a bundle2 bundler Pierre-Yves David <pierre-yves.david@fb.com> parents: diff changeset	406
21599 57cd844d7a5b bundle2: have ``newpart`` automatically add the part to the bundle Pierre-Yves David <pierre-yves.david@fb.com> parents: 21598 diff changeset	407 Use the `addparam` method to add stream level parameter. and `newpart` to
20856 8a6a86c9a5b5 bundle2: support bundling of empty part (with a type) Pierre-Yves David <pierre-yves.david@fb.com> parents: 20844 diff changeset	408 populate it. Then call `getchunks` to retrieve all the binary chunks of
21024 7731a2281cf0 spelling: fixes from spell checker Mads Kiilerich <madski@unity3d.com> parents: 21020 diff changeset	409 data that compose the bundle2 container."""
20801 9c5183cb9bca bundle2: very first version of a bundle2 bundler Pierre-Yves David <pierre-yves.david@fb.com> parents: diff changeset	410
24686 e0e28e910fa3 bundle2: rename format, parts and config to final names Pierre-Yves David <pierre-yves.david@fb.com> parents: 24648 diff changeset	411 _magicstring = 'HG20'
24640 685639f9430d bundle20: move magic string into the class Pierre-Yves David <pierre-yves.david@fb.com> parents: 24071 diff changeset	412
21134 2f8c4fa237f5 bundle2: adds a capabilities attribute on bundler20 Pierre-Yves David <pierre-yves.david@fb.com> parents: 21133 diff changeset	413 def __init__(self, ui, capabilities=()):
20842 938718d72624 bundle2: print debug information during bundling Pierre-Yves David <pierre-yves.david@fb.com> parents: 20814 diff changeset	414 self.ui = ui
20801 9c5183cb9bca bundle2: very first version of a bundle2 bundler Pierre-Yves David <pierre-yves.david@fb.com> parents: diff changeset	415 self._params = []
9c5183cb9bca bundle2: very first version of a bundle2 bundler Pierre-Yves David <pierre-yves.david@fb.com> parents: diff changeset	416 self._parts = []
21136 b6fd496e5c72 bundle2: support for capabilities with values Pierre-Yves David <pierre-yves.david@fb.com> parents: 21135 diff changeset	417 self.capabilities = dict(capabilities)
20801 9c5183cb9bca bundle2: very first version of a bundle2 bundler Pierre-Yves David <pierre-yves.david@fb.com> parents: diff changeset	418
21900 b8bd97085ec9 bundle2: add a ``bundle20.nbparts`` property Pierre-Yves David <pierre-yves.david@fb.com> parents: 21660 diff changeset	419 @property
b8bd97085ec9 bundle2: add a ``bundle20.nbparts`` property Pierre-Yves David <pierre-yves.david@fb.com> parents: 21660 diff changeset	420 def nbparts(self):
b8bd97085ec9 bundle2: add a ``bundle20.nbparts`` property Pierre-Yves David <pierre-yves.david@fb.com> parents: 21660 diff changeset	421 """total number of parts added to the bundler"""
b8bd97085ec9 bundle2: add a ``bundle20.nbparts`` property Pierre-Yves David <pierre-yves.david@fb.com> parents: 21660 diff changeset	422 return len(self._parts)
b8bd97085ec9 bundle2: add a ``bundle20.nbparts`` property Pierre-Yves David <pierre-yves.david@fb.com> parents: 21660 diff changeset	423
21597 1daad9dcdba2 bundle2: small doc update on the bundler Pierre-Yves David <pierre-yves.david@fb.com> parents: 21186 diff changeset	424 # methods used to defines the bundle2 content
20804 db9d3991d2c6 bundle2: support bundling simple parameter Pierre-Yves David <pierre-yves.david@fb.com> parents: 20803 diff changeset	425 def addparam(self, name, value=None):
db9d3991d2c6 bundle2: support bundling simple parameter Pierre-Yves David <pierre-yves.david@fb.com> parents: 20803 diff changeset	426 """add a stream level parameter"""
20813 8c74b3ce5b70 bundle2: refuse empty parameter name Pierre-Yves David <pierre-yves.david@fb.com> parents: 20812 diff changeset	427 if not name:
8c74b3ce5b70 bundle2: refuse empty parameter name Pierre-Yves David <pierre-yves.david@fb.com> parents: 20812 diff changeset	428 raise ValueError('empty parameter name')
20814 8532f5e1b9df bundle2: force the first char of parameter to be an letter. Pierre-Yves David <pierre-yves.david@fb.com> parents: 20813 diff changeset	429 if name[0] not in string.letters:
8532f5e1b9df bundle2: force the first char of parameter to be an letter. Pierre-Yves David <pierre-yves.david@fb.com> parents: 20813 diff changeset	430 raise ValueError('non letter first character: %r' % name)
20804 db9d3991d2c6 bundle2: support bundling simple parameter Pierre-Yves David <pierre-yves.david@fb.com> parents: 20803 diff changeset	431 self._params.append((name, value))
db9d3991d2c6 bundle2: support bundling simple parameter Pierre-Yves David <pierre-yves.david@fb.com> parents: 20803 diff changeset	432
20856 8a6a86c9a5b5 bundle2: support bundling of empty part (with a type) Pierre-Yves David <pierre-yves.david@fb.com> parents: 20844 diff changeset	433 def addpart(self, part):
8a6a86c9a5b5 bundle2: support bundling of empty part (with a type) Pierre-Yves David <pierre-yves.david@fb.com> parents: 20844 diff changeset	434 """add a new part to the bundle2 container
8a6a86c9a5b5 bundle2: support bundling of empty part (with a type) Pierre-Yves David <pierre-yves.david@fb.com> parents: 20844 diff changeset	435
21024 7731a2281cf0 spelling: fixes from spell checker Mads Kiilerich <madski@unity3d.com> parents: 21020 diff changeset	436 Parts contains the actual applicative payload."""
20995 e995d104c87f bundle2: add an integer id to part Pierre-Yves David <pierre-yves.david@fb.com> parents: 20952 diff changeset	437 assert part.id is None
e995d104c87f bundle2: add an integer id to part Pierre-Yves David <pierre-yves.david@fb.com> parents: 20952 diff changeset	438 part.id = len(self._parts) # very cheap counter
20856 8a6a86c9a5b5 bundle2: support bundling of empty part (with a type) Pierre-Yves David <pierre-yves.david@fb.com> parents: 20844 diff changeset	439 self._parts.append(part)
8a6a86c9a5b5 bundle2: support bundling of empty part (with a type) Pierre-Yves David <pierre-yves.david@fb.com> parents: 20844 diff changeset	440
21598 1b0dbb91de5b bundle2: add a ``newpart`` method to ``bundle20`` Pierre-Yves David <pierre-yves.david@fb.com> parents: 21597 diff changeset	441 def newpart(self, typeid, args, *kwargs):
21602 cc33ae50bab3 bundle2: warn about error during initialization in ``newpart`` docstring Pierre-Yves David <pierre-yves.david@fb.com> parents: 21601 diff changeset	442 """create a new part and add it to the containers
cc33ae50bab3 bundle2: warn about error during initialization in ``newpart`` docstring Pierre-Yves David <pierre-yves.david@fb.com> parents: 21601 diff changeset	443
cc33ae50bab3 bundle2: warn about error during initialization in ``newpart`` docstring Pierre-Yves David <pierre-yves.david@fb.com> parents: 21601 diff changeset	444 As the part is directly added to the containers. For now, this means
cc33ae50bab3 bundle2: warn about error during initialization in ``newpart`` docstring Pierre-Yves David <pierre-yves.david@fb.com> parents: 21601 diff changeset	445 that any failure to properly initialize the part after calling
cc33ae50bab3 bundle2: warn about error during initialization in ``newpart`` docstring Pierre-Yves David <pierre-yves.david@fb.com> parents: 21601 diff changeset	446 ``newpart`` should result in a failure of the whole bundling process.
cc33ae50bab3 bundle2: warn about error during initialization in ``newpart`` docstring Pierre-Yves David <pierre-yves.david@fb.com> parents: 21601 diff changeset	447
cc33ae50bab3 bundle2: warn about error during initialization in ``newpart`` docstring Pierre-Yves David <pierre-yves.david@fb.com> parents: 21601 diff changeset	448 You can still fall back to manually create and add if you need better
cc33ae50bab3 bundle2: warn about error during initialization in ``newpart`` docstring Pierre-Yves David <pierre-yves.david@fb.com> parents: 21601 diff changeset	449 control."""
21598 1b0dbb91de5b bundle2: add a ``newpart`` method to ``bundle20`` Pierre-Yves David <pierre-yves.david@fb.com> parents: 21597 diff changeset	450 part = bundlepart(typeid, args, *kwargs)
21599 57cd844d7a5b bundle2: have ``newpart`` automatically add the part to the bundle Pierre-Yves David <pierre-yves.david@fb.com> parents: 21598 diff changeset	451 self.addpart(part)
21598 1b0dbb91de5b bundle2: add a ``newpart`` method to ``bundle20`` Pierre-Yves David <pierre-yves.david@fb.com> parents: 21597 diff changeset	452 return part
1b0dbb91de5b bundle2: add a ``newpart`` method to ``bundle20`` Pierre-Yves David <pierre-yves.david@fb.com> parents: 21597 diff changeset	453
21597 1daad9dcdba2 bundle2: small doc update on the bundler Pierre-Yves David <pierre-yves.david@fb.com> parents: 21186 diff changeset	454 # methods used to generate the bundle2 stream
20801 9c5183cb9bca bundle2: very first version of a bundle2 bundler Pierre-Yves David <pierre-yves.david@fb.com> parents: diff changeset	455 def getchunks(self):
24640 685639f9430d bundle20: move magic string into the class Pierre-Yves David <pierre-yves.david@fb.com> parents: 24071 diff changeset	456 self.ui.debug('start emission of %s stream\n' % self._magicstring)
685639f9430d bundle20: move magic string into the class Pierre-Yves David <pierre-yves.david@fb.com> parents: 24071 diff changeset	457 yield self._magicstring
20804 db9d3991d2c6 bundle2: support bundling simple parameter Pierre-Yves David <pierre-yves.david@fb.com> parents: 20803 diff changeset	458 param = self._paramchunk()
20842 938718d72624 bundle2: print debug information during bundling Pierre-Yves David <pierre-yves.david@fb.com> parents: 20814 diff changeset	459 self.ui.debug('bundle parameter: %s\n' % param)
20804 db9d3991d2c6 bundle2: support bundling simple parameter Pierre-Yves David <pierre-yves.david@fb.com> parents: 20803 diff changeset	460 yield _pack(_fstreamparamsize, len(param))
db9d3991d2c6 bundle2: support bundling simple parameter Pierre-Yves David <pierre-yves.david@fb.com> parents: 20803 diff changeset	461 if param:
db9d3991d2c6 bundle2: support bundling simple parameter Pierre-Yves David <pierre-yves.david@fb.com> parents: 20803 diff changeset	462 yield param
db9d3991d2c6 bundle2: support bundling simple parameter Pierre-Yves David <pierre-yves.david@fb.com> parents: 20803 diff changeset	463
20856 8a6a86c9a5b5 bundle2: support bundling of empty part (with a type) Pierre-Yves David <pierre-yves.david@fb.com> parents: 20844 diff changeset	464 self.ui.debug('start of parts\n')
8a6a86c9a5b5 bundle2: support bundling of empty part (with a type) Pierre-Yves David <pierre-yves.david@fb.com> parents: 20844 diff changeset	465 for part in self._parts:
8a6a86c9a5b5 bundle2: support bundling of empty part (with a type) Pierre-Yves David <pierre-yves.david@fb.com> parents: 20844 diff changeset	466 self.ui.debug('bundle part: "%s"\n' % part.type)
8a6a86c9a5b5 bundle2: support bundling of empty part (with a type) Pierre-Yves David <pierre-yves.david@fb.com> parents: 20844 diff changeset	467 for chunk in part.getchunks():
8a6a86c9a5b5 bundle2: support bundling of empty part (with a type) Pierre-Yves David <pierre-yves.david@fb.com> parents: 20844 diff changeset	468 yield chunk
20842 938718d72624 bundle2: print debug information during bundling Pierre-Yves David <pierre-yves.david@fb.com> parents: 20814 diff changeset	469 self.ui.debug('end of bundle\n')
22661 9ea2913e7c41 bundle2: remove an explicit packing Pierre-Yves David <pierre-yves.david@fb.com> parents: 22548 diff changeset	470 yield _pack(_fpartheadersize, 0)
20802 520df53ad26a bundle2: a very first version of bundle2 unbundler Pierre-Yves David <pierre-yves.david@fb.com> parents: 20801 diff changeset	471
20804 db9d3991d2c6 bundle2: support bundling simple parameter Pierre-Yves David <pierre-yves.david@fb.com> parents: 20803 diff changeset	472 def _paramchunk(self):
db9d3991d2c6 bundle2: support bundling simple parameter Pierre-Yves David <pierre-yves.david@fb.com> parents: 20803 diff changeset	473 """return a encoded version of all stream parameters"""
db9d3991d2c6 bundle2: support bundling simple parameter Pierre-Yves David <pierre-yves.david@fb.com> parents: 20803 diff changeset	474 blocks = []
20809 b93bb639451a bundle2: support for bundling parameter value Pierre-Yves David <pierre-yves.david@fb.com> parents: 20808 diff changeset	475 for par, value in self._params:
20811 9785c3f8f598 bundle2: urlquote stream parameter name and value Pierre-Yves David <pierre-yves.david@fb.com> parents: 20810 diff changeset	476 par = urllib.quote(par)
20809 b93bb639451a bundle2: support for bundling parameter value Pierre-Yves David <pierre-yves.david@fb.com> parents: 20808 diff changeset	477 if value is not None:
20811 9785c3f8f598 bundle2: urlquote stream parameter name and value Pierre-Yves David <pierre-yves.david@fb.com> parents: 20810 diff changeset	478 value = urllib.quote(value)
20809 b93bb639451a bundle2: support for bundling parameter value Pierre-Yves David <pierre-yves.david@fb.com> parents: 20808 diff changeset	479 par = '%s=%s' % (par, value)
b93bb639451a bundle2: support for bundling parameter value Pierre-Yves David <pierre-yves.david@fb.com> parents: 20808 diff changeset	480 blocks.append(par)
20804 db9d3991d2c6 bundle2: support bundling simple parameter Pierre-Yves David <pierre-yves.david@fb.com> parents: 20803 diff changeset	481 return ' '.join(blocks)
db9d3991d2c6 bundle2: support bundling simple parameter Pierre-Yves David <pierre-yves.david@fb.com> parents: 20803 diff changeset	482
24794 21f2e8f48ae1 bundle2: add a 'salvageoutput' method on bundle20 Pierre-Yves David <pierre-yves.david@fb.com> parents: 24793 diff changeset	483 def salvageoutput(self):
21f2e8f48ae1 bundle2: add a 'salvageoutput' method on bundle20 Pierre-Yves David <pierre-yves.david@fb.com> parents: 24793 diff changeset	484 """return a list with a copy of all output parts in the bundle
21f2e8f48ae1 bundle2: add a 'salvageoutput' method on bundle20 Pierre-Yves David <pierre-yves.david@fb.com> parents: 24793 diff changeset	485
21f2e8f48ae1 bundle2: add a 'salvageoutput' method on bundle20 Pierre-Yves David <pierre-yves.david@fb.com> parents: 24793 diff changeset	486 This is meant to be used during error handling to make sure we preserve
21f2e8f48ae1 bundle2: add a 'salvageoutput' method on bundle20 Pierre-Yves David <pierre-yves.david@fb.com> parents: 24793 diff changeset	487 server output"""
21f2e8f48ae1 bundle2: add a 'salvageoutput' method on bundle20 Pierre-Yves David <pierre-yves.david@fb.com> parents: 24793 diff changeset	488 salvaged = []
21f2e8f48ae1 bundle2: add a 'salvageoutput' method on bundle20 Pierre-Yves David <pierre-yves.david@fb.com> parents: 24793 diff changeset	489 for part in self._parts:
21f2e8f48ae1 bundle2: add a 'salvageoutput' method on bundle20 Pierre-Yves David <pierre-yves.david@fb.com> parents: 24793 diff changeset	490 if part.type.startswith('output'):
21f2e8f48ae1 bundle2: add a 'salvageoutput' method on bundle20 Pierre-Yves David <pierre-yves.david@fb.com> parents: 24793 diff changeset	491 salvaged.append(part.copy())
21f2e8f48ae1 bundle2: add a 'salvageoutput' method on bundle20 Pierre-Yves David <pierre-yves.david@fb.com> parents: 24793 diff changeset	492 return salvaged
21f2e8f48ae1 bundle2: add a 'salvageoutput' method on bundle20 Pierre-Yves David <pierre-yves.david@fb.com> parents: 24793 diff changeset	493
21f2e8f48ae1 bundle2: add a 'salvageoutput' method on bundle20 Pierre-Yves David <pierre-yves.david@fb.com> parents: 24793 diff changeset	494
21013 a813caca89b3 bundle2: extract stream/unpack logic in an unpackermixin Pierre-Yves David <pierre-yves.david@fb.com> parents: 21005 diff changeset	495 class unpackermixin(object):
a813caca89b3 bundle2: extract stream/unpack logic in an unpackermixin Pierre-Yves David <pierre-yves.david@fb.com> parents: 21005 diff changeset	496 """A mixin to extract bytes and struct data from a stream"""
20802 520df53ad26a bundle2: a very first version of bundle2 unbundler Pierre-Yves David <pierre-yves.david@fb.com> parents: 20801 diff changeset	497
21013 a813caca89b3 bundle2: extract stream/unpack logic in an unpackermixin Pierre-Yves David <pierre-yves.david@fb.com> parents: 21005 diff changeset	498 def __init__(self, fp):
20802 520df53ad26a bundle2: a very first version of bundle2 unbundler Pierre-Yves David <pierre-yves.david@fb.com> parents: 20801 diff changeset	499 self._fp = fp
24026 3daef83a1873 bundle2.unpackermixin: control for underlying file descriptor Eric Sumner <ericsumner@fb.com> parents: 23916 diff changeset	500 self._seekable = (util.safehasattr(fp, 'seek') and
3daef83a1873 bundle2.unpackermixin: control for underlying file descriptor Eric Sumner <ericsumner@fb.com> parents: 23916 diff changeset	501 util.safehasattr(fp, 'tell'))
20802 520df53ad26a bundle2: a very first version of bundle2 unbundler Pierre-Yves David <pierre-yves.david@fb.com> parents: 20801 diff changeset	502
520df53ad26a bundle2: a very first version of bundle2 unbundler Pierre-Yves David <pierre-yves.david@fb.com> parents: 20801 diff changeset	503 def _unpack(self, format):
520df53ad26a bundle2: a very first version of bundle2 unbundler Pierre-Yves David <pierre-yves.david@fb.com> parents: 20801 diff changeset	504 """unpack this struct format from the stream"""
520df53ad26a bundle2: a very first version of bundle2 unbundler Pierre-Yves David <pierre-yves.david@fb.com> parents: 20801 diff changeset	505 data = self._readexact(struct.calcsize(format))
520df53ad26a bundle2: a very first version of bundle2 unbundler Pierre-Yves David <pierre-yves.david@fb.com> parents: 20801 diff changeset	506 return _unpack(format, data)
520df53ad26a bundle2: a very first version of bundle2 unbundler Pierre-Yves David <pierre-yves.david@fb.com> parents: 20801 diff changeset	507
520df53ad26a bundle2: a very first version of bundle2 unbundler Pierre-Yves David <pierre-yves.david@fb.com> parents: 20801 diff changeset	508 def _readexact(self, size):
520df53ad26a bundle2: a very first version of bundle2 unbundler Pierre-Yves David <pierre-yves.david@fb.com> parents: 20801 diff changeset	509 """read exactly <size> bytes from the stream"""
520df53ad26a bundle2: a very first version of bundle2 unbundler Pierre-Yves David <pierre-yves.david@fb.com> parents: 20801 diff changeset	510 return changegroup.readexactly(self._fp, size)
520df53ad26a bundle2: a very first version of bundle2 unbundler Pierre-Yves David <pierre-yves.david@fb.com> parents: 20801 diff changeset	511
24070 de32e9881698 bundle2.unpackermixin: default value for seek() whence parameter Eric Sumner <ericsumner@fb.com> parents: 24048 diff changeset	512 def seek(self, offset, whence=0):
24026 3daef83a1873 bundle2.unpackermixin: control for underlying file descriptor Eric Sumner <ericsumner@fb.com> parents: 23916 diff changeset	513 """move the underlying file pointer"""
3daef83a1873 bundle2.unpackermixin: control for underlying file descriptor Eric Sumner <ericsumner@fb.com> parents: 23916 diff changeset	514 if self._seekable:
3daef83a1873 bundle2.unpackermixin: control for underlying file descriptor Eric Sumner <ericsumner@fb.com> parents: 23916 diff changeset	515 return self._fp.seek(offset, whence)
3daef83a1873 bundle2.unpackermixin: control for underlying file descriptor Eric Sumner <ericsumner@fb.com> parents: 23916 diff changeset	516 else:
3daef83a1873 bundle2.unpackermixin: control for underlying file descriptor Eric Sumner <ericsumner@fb.com> parents: 23916 diff changeset	517 raise NotImplementedError(_('File pointer is not seekable'))
3daef83a1873 bundle2.unpackermixin: control for underlying file descriptor Eric Sumner <ericsumner@fb.com> parents: 23916 diff changeset	518
3daef83a1873 bundle2.unpackermixin: control for underlying file descriptor Eric Sumner <ericsumner@fb.com> parents: 23916 diff changeset	519 def tell(self):
3daef83a1873 bundle2.unpackermixin: control for underlying file descriptor Eric Sumner <ericsumner@fb.com> parents: 23916 diff changeset	520 """return the file offset, or None if file is not seekable"""
3daef83a1873 bundle2.unpackermixin: control for underlying file descriptor Eric Sumner <ericsumner@fb.com> parents: 23916 diff changeset	521 if self._seekable:
3daef83a1873 bundle2.unpackermixin: control for underlying file descriptor Eric Sumner <ericsumner@fb.com> parents: 23916 diff changeset	522 try:
3daef83a1873 bundle2.unpackermixin: control for underlying file descriptor Eric Sumner <ericsumner@fb.com> parents: 23916 diff changeset	523 return self._fp.tell()
3daef83a1873 bundle2.unpackermixin: control for underlying file descriptor Eric Sumner <ericsumner@fb.com> parents: 23916 diff changeset	524 except IOError, e:
3daef83a1873 bundle2.unpackermixin: control for underlying file descriptor Eric Sumner <ericsumner@fb.com> parents: 23916 diff changeset	525 if e.errno == errno.ESPIPE:
3daef83a1873 bundle2.unpackermixin: control for underlying file descriptor Eric Sumner <ericsumner@fb.com> parents: 23916 diff changeset	526 self._seekable = False
3daef83a1873 bundle2.unpackermixin: control for underlying file descriptor Eric Sumner <ericsumner@fb.com> parents: 23916 diff changeset	527 else:
3daef83a1873 bundle2.unpackermixin: control for underlying file descriptor Eric Sumner <ericsumner@fb.com> parents: 23916 diff changeset	528 raise
3daef83a1873 bundle2.unpackermixin: control for underlying file descriptor Eric Sumner <ericsumner@fb.com> parents: 23916 diff changeset	529 return None
3daef83a1873 bundle2.unpackermixin: control for underlying file descriptor Eric Sumner <ericsumner@fb.com> parents: 23916 diff changeset	530
3daef83a1873 bundle2.unpackermixin: control for underlying file descriptor Eric Sumner <ericsumner@fb.com> parents: 23916 diff changeset	531 def close(self):
3daef83a1873 bundle2.unpackermixin: control for underlying file descriptor Eric Sumner <ericsumner@fb.com> parents: 23916 diff changeset	532 """close underlying file"""
3daef83a1873 bundle2.unpackermixin: control for underlying file descriptor Eric Sumner <ericsumner@fb.com> parents: 23916 diff changeset	533 if util.safehasattr(self._fp, 'close'):
3daef83a1873 bundle2.unpackermixin: control for underlying file descriptor Eric Sumner <ericsumner@fb.com> parents: 23916 diff changeset	534 return self._fp.close()
21013 a813caca89b3 bundle2: extract stream/unpack logic in an unpackermixin Pierre-Yves David <pierre-yves.david@fb.com> parents: 21005 diff changeset	535
24641 60fecc5b14a4 unbundle20: retrieve unbundler instances through a factory function Pierre-Yves David <pierre-yves.david@fb.com> parents: 24640 diff changeset	536 def getunbundler(ui, fp, header=None):
60fecc5b14a4 unbundle20: retrieve unbundler instances through a factory function Pierre-Yves David <pierre-yves.david@fb.com> parents: 24640 diff changeset	537 """return a valid unbundler object for a given header"""
24642 54e5c239c2d9 unbundle20: move header parsing into the 'getunbundler' function Pierre-Yves David <pierre-yves.david@fb.com> parents: 24641 diff changeset	538 if header is None:
54e5c239c2d9 unbundle20: move header parsing into the 'getunbundler' function Pierre-Yves David <pierre-yves.david@fb.com> parents: 24641 diff changeset	539 header = changegroup.readexactly(fp, 4)
24648 5cac3accdaa1 unbundle20: allow generic dispatch between unbundlers Pierre-Yves David <pierre-yves.david@fb.com> parents: 24642 diff changeset	540 magic, version = header[0:2], header[2:4]
5cac3accdaa1 unbundle20: allow generic dispatch between unbundlers Pierre-Yves David <pierre-yves.david@fb.com> parents: 24642 diff changeset	541 if magic != 'HG':
5cac3accdaa1 unbundle20: allow generic dispatch between unbundlers Pierre-Yves David <pierre-yves.david@fb.com> parents: 24642 diff changeset	542 raise util.Abort(_('not a Mercurial bundle'))
5cac3accdaa1 unbundle20: allow generic dispatch between unbundlers Pierre-Yves David <pierre-yves.david@fb.com> parents: 24642 diff changeset	543 unbundlerclass = formatmap.get(version)
5cac3accdaa1 unbundle20: allow generic dispatch between unbundlers Pierre-Yves David <pierre-yves.david@fb.com> parents: 24642 diff changeset	544 if unbundlerclass is None:
5cac3accdaa1 unbundle20: allow generic dispatch between unbundlers Pierre-Yves David <pierre-yves.david@fb.com> parents: 24642 diff changeset	545 raise util.Abort(_('unknown bundle version %s') % version)
5cac3accdaa1 unbundle20: allow generic dispatch between unbundlers Pierre-Yves David <pierre-yves.david@fb.com> parents: 24642 diff changeset	546 unbundler = unbundlerclass(ui, fp)
24642 54e5c239c2d9 unbundle20: move header parsing into the 'getunbundler' function Pierre-Yves David <pierre-yves.david@fb.com> parents: 24641 diff changeset	547 ui.debug('start processing of %s stream\n' % header)
54e5c239c2d9 unbundle20: move header parsing into the 'getunbundler' function Pierre-Yves David <pierre-yves.david@fb.com> parents: 24641 diff changeset	548 return unbundler
24641 60fecc5b14a4 unbundle20: retrieve unbundler instances through a factory function Pierre-Yves David <pierre-yves.david@fb.com> parents: 24640 diff changeset	549
21013 a813caca89b3 bundle2: extract stream/unpack logic in an unpackermixin Pierre-Yves David <pierre-yves.david@fb.com> parents: 21005 diff changeset	550 class unbundle20(unpackermixin):
a813caca89b3 bundle2: extract stream/unpack logic in an unpackermixin Pierre-Yves David <pierre-yves.david@fb.com> parents: 21005 diff changeset	551 """interpret a bundle2 stream
a813caca89b3 bundle2: extract stream/unpack logic in an unpackermixin Pierre-Yves David <pierre-yves.david@fb.com> parents: 21005 diff changeset	552
21129 07bcbf326c8d bundle2: use an official iterparts method to unbundle parts Pierre-Yves David <pierre-yves.david@fb.com> parents: 21066 diff changeset	553 This class is fed with a binary stream and yields parts through its
07bcbf326c8d bundle2: use an official iterparts method to unbundle parts Pierre-Yves David <pierre-yves.david@fb.com> parents: 21066 diff changeset	554 `iterparts` methods."""
21013 a813caca89b3 bundle2: extract stream/unpack logic in an unpackermixin Pierre-Yves David <pierre-yves.david@fb.com> parents: 21005 diff changeset	555
24642 54e5c239c2d9 unbundle20: move header parsing into the 'getunbundler' function Pierre-Yves David <pierre-yves.david@fb.com> parents: 24641 diff changeset	556 def __init__(self, ui, fp):
21066 5ecfe76d0d96 bundle2: make header reading optional Pierre-Yves David <pierre-yves.david@fb.com> parents: 21062 diff changeset	557 """If header is specified, we do not read it out of the stream."""
21013 a813caca89b3 bundle2: extract stream/unpack logic in an unpackermixin Pierre-Yves David <pierre-yves.david@fb.com> parents: 21005 diff changeset	558 self.ui = ui
a813caca89b3 bundle2: extract stream/unpack logic in an unpackermixin Pierre-Yves David <pierre-yves.david@fb.com> parents: 21005 diff changeset	559 super(unbundle20, self).__init__(fp)
a813caca89b3 bundle2: extract stream/unpack logic in an unpackermixin Pierre-Yves David <pierre-yves.david@fb.com> parents: 21005 diff changeset	560
20802 520df53ad26a bundle2: a very first version of bundle2 unbundler Pierre-Yves David <pierre-yves.david@fb.com> parents: 20801 diff changeset	561 @util.propertycache
520df53ad26a bundle2: a very first version of bundle2 unbundler Pierre-Yves David <pierre-yves.david@fb.com> parents: 20801 diff changeset	562 def params(self):
21024 7731a2281cf0 spelling: fixes from spell checker Mads Kiilerich <madski@unity3d.com> parents: 21020 diff changeset	563 """dictionary of stream level parameters"""
20843 0641b41b0b49 bundle2: print debug information during unbundling Pierre-Yves David <pierre-yves.david@fb.com> parents: 20842 diff changeset	564 self.ui.debug('reading bundle2 stream parameters\n')
20805 c5aaeca0cfbf bundle2: support for unbundling simple parameter Pierre-Yves David <pierre-yves.david@fb.com> parents: 20804 diff changeset	565 params = {}
c5aaeca0cfbf bundle2: support for unbundling simple parameter Pierre-Yves David <pierre-yves.david@fb.com> parents: 20804 diff changeset	566 paramssize = self._unpack(_fstreamparamsize)[0]
23011 006a81d07e57 bundle2: detect and disallow a negative chunk size Pierre-Yves David <pierre-yves.david@fb.com> parents: 23010 diff changeset	567 if paramssize < 0:
006a81d07e57 bundle2: detect and disallow a negative chunk size Pierre-Yves David <pierre-yves.david@fb.com> parents: 23010 diff changeset	568 raise error.BundleValueError('negative bundle param size: %i'
006a81d07e57 bundle2: detect and disallow a negative chunk size Pierre-Yves David <pierre-yves.david@fb.com> parents: 23010 diff changeset	569 % paramssize)
20805 c5aaeca0cfbf bundle2: support for unbundling simple parameter Pierre-Yves David <pierre-yves.david@fb.com> parents: 20804 diff changeset	570 if paramssize:
c5aaeca0cfbf bundle2: support for unbundling simple parameter Pierre-Yves David <pierre-yves.david@fb.com> parents: 20804 diff changeset	571 for p in self._readexact(paramssize).split(' '):
20810 47293877b54c bundle2: support for unbundling parameter value Pierre-Yves David <pierre-yves.david@fb.com> parents: 20809 diff changeset	572 p = p.split('=', 1)
20812 e2f908773754 bundle2: urlunquote stream parameter name and value during unbundling Pierre-Yves David <pierre-yves.david@fb.com> parents: 20811 diff changeset	573 p = [urllib.unquote(i) for i in p]
20810 47293877b54c bundle2: support for unbundling parameter value Pierre-Yves David <pierre-yves.david@fb.com> parents: 20809 diff changeset	574 if len(p) < 2:
47293877b54c bundle2: support for unbundling parameter value Pierre-Yves David <pierre-yves.david@fb.com> parents: 20809 diff changeset	575 p.append(None)
20844 2631204d7305 bundle2: implement the mandatory/advisory logic for parameter Pierre-Yves David <pierre-yves.david@fb.com> parents: 20843 diff changeset	576 self._processparam(*p)
20810 47293877b54c bundle2: support for unbundling parameter value Pierre-Yves David <pierre-yves.david@fb.com> parents: 20809 diff changeset	577 params[p[0]] = p[1]
20805 c5aaeca0cfbf bundle2: support for unbundling simple parameter Pierre-Yves David <pierre-yves.david@fb.com> parents: 20804 diff changeset	578 return params
20802 520df53ad26a bundle2: a very first version of bundle2 unbundler Pierre-Yves David <pierre-yves.david@fb.com> parents: 20801 diff changeset	579
20844 2631204d7305 bundle2: implement the mandatory/advisory logic for parameter Pierre-Yves David <pierre-yves.david@fb.com> parents: 20843 diff changeset	580 def _processparam(self, name, value):
2631204d7305 bundle2: implement the mandatory/advisory logic for parameter Pierre-Yves David <pierre-yves.david@fb.com> parents: 20843 diff changeset	581 """process a parameter, applying its effect if needed
2631204d7305 bundle2: implement the mandatory/advisory logic for parameter Pierre-Yves David <pierre-yves.david@fb.com> parents: 20843 diff changeset	582
2631204d7305 bundle2: implement the mandatory/advisory logic for parameter Pierre-Yves David <pierre-yves.david@fb.com> parents: 20843 diff changeset	583 Parameter starting with a lower case letter are advisory and will be
2631204d7305 bundle2: implement the mandatory/advisory logic for parameter Pierre-Yves David <pierre-yves.david@fb.com> parents: 20843 diff changeset	584 ignored when unknown. Those starting with an upper case letter are
2631204d7305 bundle2: implement the mandatory/advisory logic for parameter Pierre-Yves David <pierre-yves.david@fb.com> parents: 20843 diff changeset	585 mandatory and will this function will raise a KeyError when unknown.
2631204d7305 bundle2: implement the mandatory/advisory logic for parameter Pierre-Yves David <pierre-yves.david@fb.com> parents: 20843 diff changeset	586
2631204d7305 bundle2: implement the mandatory/advisory logic for parameter Pierre-Yves David <pierre-yves.david@fb.com> parents: 20843 diff changeset	587 Note: no option are currently supported. Any input will be either
2631204d7305 bundle2: implement the mandatory/advisory logic for parameter Pierre-Yves David <pierre-yves.david@fb.com> parents: 20843 diff changeset	588 ignored or failing.
2631204d7305 bundle2: implement the mandatory/advisory logic for parameter Pierre-Yves David <pierre-yves.david@fb.com> parents: 20843 diff changeset	589 """
2631204d7305 bundle2: implement the mandatory/advisory logic for parameter Pierre-Yves David <pierre-yves.david@fb.com> parents: 20843 diff changeset	590 if not name:
2631204d7305 bundle2: implement the mandatory/advisory logic for parameter Pierre-Yves David <pierre-yves.david@fb.com> parents: 20843 diff changeset	591 raise ValueError('empty parameter name')
2631204d7305 bundle2: implement the mandatory/advisory logic for parameter Pierre-Yves David <pierre-yves.david@fb.com> parents: 20843 diff changeset	592 if name[0] not in string.letters:
2631204d7305 bundle2: implement the mandatory/advisory logic for parameter Pierre-Yves David <pierre-yves.david@fb.com> parents: 20843 diff changeset	593 raise ValueError('non letter first character: %r' % name)
2631204d7305 bundle2: implement the mandatory/advisory logic for parameter Pierre-Yves David <pierre-yves.david@fb.com> parents: 20843 diff changeset	594 # Some logic will be later added here to try to process the option for
2631204d7305 bundle2: implement the mandatory/advisory logic for parameter Pierre-Yves David <pierre-yves.david@fb.com> parents: 20843 diff changeset	595 # a dict of known parameter.
2631204d7305 bundle2: implement the mandatory/advisory logic for parameter Pierre-Yves David <pierre-yves.david@fb.com> parents: 20843 diff changeset	596 if name[0].islower():
2631204d7305 bundle2: implement the mandatory/advisory logic for parameter Pierre-Yves David <pierre-yves.david@fb.com> parents: 20843 diff changeset	597 self.ui.debug("ignoring unknown parameter %r\n" % name)
2631204d7305 bundle2: implement the mandatory/advisory logic for parameter Pierre-Yves David <pierre-yves.david@fb.com> parents: 20843 diff changeset	598 else:
23010 73f394f4affc bundle2: add an UnsupportedPartError Pierre-Yves David <pierre-yves.david@fb.com> parents: 23009 diff changeset	599 raise error.UnsupportedPartError(params=(name,))
20844 2631204d7305 bundle2: implement the mandatory/advisory logic for parameter Pierre-Yves David <pierre-yves.david@fb.com> parents: 20843 diff changeset	600
2631204d7305 bundle2: implement the mandatory/advisory logic for parameter Pierre-Yves David <pierre-yves.david@fb.com> parents: 20843 diff changeset	601
21129 07bcbf326c8d bundle2: use an official iterparts method to unbundle parts Pierre-Yves David <pierre-yves.david@fb.com> parents: 21066 diff changeset	602 def iterparts(self):
20802 520df53ad26a bundle2: a very first version of bundle2 unbundler Pierre-Yves David <pierre-yves.david@fb.com> parents: 20801 diff changeset	603 """yield all parts contained in the stream"""
520df53ad26a bundle2: a very first version of bundle2 unbundler Pierre-Yves David <pierre-yves.david@fb.com> parents: 20801 diff changeset	604 # make sure param have been loaded
520df53ad26a bundle2: a very first version of bundle2 unbundler Pierre-Yves David <pierre-yves.david@fb.com> parents: 20801 diff changeset	605 self.params
20843 0641b41b0b49 bundle2: print debug information during unbundling Pierre-Yves David <pierre-yves.david@fb.com> parents: 20842 diff changeset	606 self.ui.debug('start extraction of bundle2 parts\n')
21014 a6246bba7b9e bundle2: add an unbundle part responsible from unbundling part Pierre-Yves David <pierre-yves.david@fb.com> parents: 21013 diff changeset	607 headerblock = self._readpartheader()
a6246bba7b9e bundle2: add an unbundle part responsible from unbundling part Pierre-Yves David <pierre-yves.david@fb.com> parents: 21013 diff changeset	608 while headerblock is not None:
a6246bba7b9e bundle2: add an unbundle part responsible from unbundling part Pierre-Yves David <pierre-yves.david@fb.com> parents: 21013 diff changeset	609 part = unbundlepart(self.ui, headerblock, self._fp)
20802 520df53ad26a bundle2: a very first version of bundle2 unbundler Pierre-Yves David <pierre-yves.david@fb.com> parents: 20801 diff changeset	610 yield part
24048 13d88b7eb3a0 bundle2: seek in part iterator Eric Sumner <ericsumner@fb.com> parents: 24047 diff changeset	611 part.seek(0, 2)
21014 a6246bba7b9e bundle2: add an unbundle part responsible from unbundling part Pierre-Yves David <pierre-yves.david@fb.com> parents: 21013 diff changeset	612 headerblock = self._readpartheader()
20843 0641b41b0b49 bundle2: print debug information during unbundling Pierre-Yves David <pierre-yves.david@fb.com> parents: 20842 diff changeset	613 self.ui.debug('end of bundle2 stream\n')
20802 520df53ad26a bundle2: a very first version of bundle2 unbundler Pierre-Yves David <pierre-yves.david@fb.com> parents: 20801 diff changeset	614
21014 a6246bba7b9e bundle2: add an unbundle part responsible from unbundling part Pierre-Yves David <pierre-yves.david@fb.com> parents: 21013 diff changeset	615 def _readpartheader(self):
a6246bba7b9e bundle2: add an unbundle part responsible from unbundling part Pierre-Yves David <pierre-yves.david@fb.com> parents: 21013 diff changeset	616 """reads a part header size and return the bytes blob
20864 9a75d2559cff bundle2: support unbundling empty part Pierre-Yves David <pierre-yves.david@fb.com> parents: 20856 diff changeset	617
21014 a6246bba7b9e bundle2: add an unbundle part responsible from unbundling part Pierre-Yves David <pierre-yves.david@fb.com> parents: 21013 diff changeset	618 returns None if empty"""
20864 9a75d2559cff bundle2: support unbundling empty part Pierre-Yves David <pierre-yves.david@fb.com> parents: 20856 diff changeset	619 headersize = self._unpack(_fpartheadersize)[0]
23011 006a81d07e57 bundle2: detect and disallow a negative chunk size Pierre-Yves David <pierre-yves.david@fb.com> parents: 23010 diff changeset	620 if headersize < 0:
006a81d07e57 bundle2: detect and disallow a negative chunk size Pierre-Yves David <pierre-yves.david@fb.com> parents: 23010 diff changeset	621 raise error.BundleValueError('negative part header size: %i'
006a81d07e57 bundle2: detect and disallow a negative chunk size Pierre-Yves David <pierre-yves.david@fb.com> parents: 23010 diff changeset	622 % headersize)
20864 9a75d2559cff bundle2: support unbundling empty part Pierre-Yves David <pierre-yves.david@fb.com> parents: 20856 diff changeset	623 self.ui.debug('part header size: %i\n' % headersize)
21014 a6246bba7b9e bundle2: add an unbundle part responsible from unbundling part Pierre-Yves David <pierre-yves.david@fb.com> parents: 21013 diff changeset	624 if headersize:
a6246bba7b9e bundle2: add an unbundle part responsible from unbundling part Pierre-Yves David <pierre-yves.david@fb.com> parents: 21013 diff changeset	625 return self._readexact(headersize)
a6246bba7b9e bundle2: add an unbundle part responsible from unbundling part Pierre-Yves David <pierre-yves.david@fb.com> parents: 21013 diff changeset	626 return None
20864 9a75d2559cff bundle2: support unbundling empty part Pierre-Yves David <pierre-yves.david@fb.com> parents: 20856 diff changeset	627
24071 184a2f6f40da bundle2.unbundle20: add compressed() method Eric Sumner <ericsumner@fb.com> parents: 24070 diff changeset	628 def compressed(self):
184a2f6f40da bundle2.unbundle20: add compressed() method Eric Sumner <ericsumner@fb.com> parents: 24070 diff changeset	629 return False
20802 520df53ad26a bundle2: a very first version of bundle2 unbundler Pierre-Yves David <pierre-yves.david@fb.com> parents: 20801 diff changeset	630
24686 e0e28e910fa3 bundle2: rename format, parts and config to final names Pierre-Yves David <pierre-yves.david@fb.com> parents: 24648 diff changeset	631 formatmap = {'20': unbundle20}
24648 5cac3accdaa1 unbundle20: allow generic dispatch between unbundlers Pierre-Yves David <pierre-yves.david@fb.com> parents: 24642 diff changeset	632
21005 3d38ebb586fe bundle2: rename part to bundlepart Pierre-Yves David <pierre-yves.david@fb.com> parents: 21004 diff changeset	633 class bundlepart(object):
20856 8a6a86c9a5b5 bundle2: support bundling of empty part (with a type) Pierre-Yves David <pierre-yves.david@fb.com> parents: 20844 diff changeset	634 """A bundle2 part contains application level payload
8a6a86c9a5b5 bundle2: support bundling of empty part (with a type) Pierre-Yves David <pierre-yves.david@fb.com> parents: 20844 diff changeset	635
8a6a86c9a5b5 bundle2: support bundling of empty part (with a type) Pierre-Yves David <pierre-yves.david@fb.com> parents: 20844 diff changeset	636 The part `type` is used to route the part to the application level
8a6a86c9a5b5 bundle2: support bundling of empty part (with a type) Pierre-Yves David <pierre-yves.david@fb.com> parents: 20844 diff changeset	637 handler.
21604 c399bf961cb9 bundle2: the ability to set ``data`` attribute of the part is now official Pierre-Yves David <pierre-yves.david@fb.com> parents: 21603 diff changeset	638
c399bf961cb9 bundle2: the ability to set ``data`` attribute of the part is now official Pierre-Yves David <pierre-yves.david@fb.com> parents: 21603 diff changeset	639 The part payload is contained in ``part.data``. It could be raw bytes or a
21605 f9dabfceb259 bundle2: introduce a ``addparam`` method on part Pierre-Yves David <pierre-yves.david@fb.com> parents: 21604 diff changeset	640 generator of byte chunks.
f9dabfceb259 bundle2: introduce a ``addparam`` method on part Pierre-Yves David <pierre-yves.david@fb.com> parents: 21604 diff changeset	641
f9dabfceb259 bundle2: introduce a ``addparam`` method on part Pierre-Yves David <pierre-yves.david@fb.com> parents: 21604 diff changeset	642 You can add parameters to the part using the ``addparam`` method.
f9dabfceb259 bundle2: introduce a ``addparam`` method on part Pierre-Yves David <pierre-yves.david@fb.com> parents: 21604 diff changeset	643 Parameters can be either mandatory (default) or advisory. Remote side
f9dabfceb259 bundle2: introduce a ``addparam`` method on part Pierre-Yves David <pierre-yves.david@fb.com> parents: 21604 diff changeset	644 should be able to safely ignore the advisory ones.
f9dabfceb259 bundle2: introduce a ``addparam`` method on part Pierre-Yves David <pierre-yves.david@fb.com> parents: 21604 diff changeset	645
f9dabfceb259 bundle2: introduce a ``addparam`` method on part Pierre-Yves David <pierre-yves.david@fb.com> parents: 21604 diff changeset	646 Both data and parameters cannot be modified after the generation has begun.
20856 8a6a86c9a5b5 bundle2: support bundling of empty part (with a type) Pierre-Yves David <pierre-yves.david@fb.com> parents: 20844 diff changeset	647 """
8a6a86c9a5b5 bundle2: support bundling of empty part (with a type) Pierre-Yves David <pierre-yves.david@fb.com> parents: 20844 diff changeset	648
20877 9e9e3a4e9261 bundle2: part params Pierre-Yves David <pierre-yves.david@fb.com> parents: 20876 diff changeset	649 def __init__(self, parttype, mandatoryparams=(), advisoryparams=(),
23590 4440c7cc3728 bundle2.bundlepart: make mandatory part flag explicit in API Eric Sumner <ericsumner@fb.com> parents: 23586 diff changeset	650 data='', mandatory=True):
23868 405eacbecc52 bundle2: enforce parttype as alphanumerical Pierre-Yves David <pierre-yves.david@fb.com> parents: 23590 diff changeset	651 validateparttype(parttype)
20995 e995d104c87f bundle2: add an integer id to part Pierre-Yves David <pierre-yves.david@fb.com> parents: 20952 diff changeset	652 self.id = None
20856 8a6a86c9a5b5 bundle2: support bundling of empty part (with a type) Pierre-Yves David <pierre-yves.david@fb.com> parents: 20844 diff changeset	653 self.type = parttype
21604 c399bf961cb9 bundle2: the ability to set ``data`` attribute of the part is now official Pierre-Yves David <pierre-yves.david@fb.com> parents: 21603 diff changeset	654 self._data = data
21605 f9dabfceb259 bundle2: introduce a ``addparam`` method on part Pierre-Yves David <pierre-yves.david@fb.com> parents: 21604 diff changeset	655 self._mandatoryparams = list(mandatoryparams)
f9dabfceb259 bundle2: introduce a ``addparam`` method on part Pierre-Yves David <pierre-yves.david@fb.com> parents: 21604 diff changeset	656 self._advisoryparams = list(advisoryparams)
21607 054fa5176fa7 bundle2: forbid duplicate parameter keys Pierre-Yves David <pierre-yves.david@fb.com> parents: 21606 diff changeset	657 # checking for duplicated entries
054fa5176fa7 bundle2: forbid duplicate parameter keys Pierre-Yves David <pierre-yves.david@fb.com> parents: 21606 diff changeset	658 self._seenparams = set()
054fa5176fa7 bundle2: forbid duplicate parameter keys Pierre-Yves David <pierre-yves.david@fb.com> parents: 21606 diff changeset	659 for pname, __ in self._mandatoryparams + self._advisoryparams:
054fa5176fa7 bundle2: forbid duplicate parameter keys Pierre-Yves David <pierre-yves.david@fb.com> parents: 21606 diff changeset	660 if pname in self._seenparams:
054fa5176fa7 bundle2: forbid duplicate parameter keys Pierre-Yves David <pierre-yves.david@fb.com> parents: 21606 diff changeset	661 raise RuntimeError('duplicated params: %s' % pname)
054fa5176fa7 bundle2: forbid duplicate parameter keys Pierre-Yves David <pierre-yves.david@fb.com> parents: 21606 diff changeset	662 self._seenparams.add(pname)
21601 7ff01befc7ec bundle2: track life cycle of parts Pierre-Yves David <pierre-yves.david@fb.com> parents: 21600 diff changeset	663 # status of the part's generation:
7ff01befc7ec bundle2: track life cycle of parts Pierre-Yves David <pierre-yves.david@fb.com> parents: 21600 diff changeset	664 # - None: not started,
7ff01befc7ec bundle2: track life cycle of parts Pierre-Yves David <pierre-yves.david@fb.com> parents: 21600 diff changeset	665 # - False: currently generated,
7ff01befc7ec bundle2: track life cycle of parts Pierre-Yves David <pierre-yves.david@fb.com> parents: 21600 diff changeset	666 # - True: generation done.
7ff01befc7ec bundle2: track life cycle of parts Pierre-Yves David <pierre-yves.david@fb.com> parents: 21600 diff changeset	667 self._generated = None
23590 4440c7cc3728 bundle2.bundlepart: make mandatory part flag explicit in API Eric Sumner <ericsumner@fb.com> parents: 23586 diff changeset	668 self.mandatory = mandatory
20856 8a6a86c9a5b5 bundle2: support bundling of empty part (with a type) Pierre-Yves David <pierre-yves.david@fb.com> parents: 20844 diff changeset	669
24793 2ec894582ea2 bundle2: add a 'copy' method on parts Pierre-Yves David <pierre-yves.david@fb.com> parents: 24743 diff changeset	670 def copy(self):
2ec894582ea2 bundle2: add a 'copy' method on parts Pierre-Yves David <pierre-yves.david@fb.com> parents: 24743 diff changeset	671 """return a copy of the part
2ec894582ea2 bundle2: add a 'copy' method on parts Pierre-Yves David <pierre-yves.david@fb.com> parents: 24743 diff changeset	672
2ec894582ea2 bundle2: add a 'copy' method on parts Pierre-Yves David <pierre-yves.david@fb.com> parents: 24743 diff changeset	673 The new part have the very same content but no partid assigned yet.
2ec894582ea2 bundle2: add a 'copy' method on parts Pierre-Yves David <pierre-yves.david@fb.com> parents: 24743 diff changeset	674 Parts with generated data cannot be copied."""
2ec894582ea2 bundle2: add a 'copy' method on parts Pierre-Yves David <pierre-yves.david@fb.com> parents: 24743 diff changeset	675 assert not util.safehasattr(self.data, 'next')
2ec894582ea2 bundle2: add a 'copy' method on parts Pierre-Yves David <pierre-yves.david@fb.com> parents: 24743 diff changeset	676 return self.__class__(self.type, self._mandatoryparams,
2ec894582ea2 bundle2: add a 'copy' method on parts Pierre-Yves David <pierre-yves.david@fb.com> parents: 24743 diff changeset	677 self._advisoryparams, self._data, self.mandatory)
2ec894582ea2 bundle2: add a 'copy' method on parts Pierre-Yves David <pierre-yves.david@fb.com> parents: 24743 diff changeset	678
21604 c399bf961cb9 bundle2: the ability to set ``data`` attribute of the part is now official Pierre-Yves David <pierre-yves.david@fb.com> parents: 21603 diff changeset	679 # methods used to defines the part content
c399bf961cb9 bundle2: the ability to set ``data`` attribute of the part is now official Pierre-Yves David <pierre-yves.david@fb.com> parents: 21603 diff changeset	680 def __setdata(self, data):
c399bf961cb9 bundle2: the ability to set ``data`` attribute of the part is now official Pierre-Yves David <pierre-yves.david@fb.com> parents: 21603 diff changeset	681 if self._generated is not None:
21618 7568f5c1c801 bundle2: move exception classes into the error module Pierre-Yves David <pierre-yves.david@fb.com> parents: 21617 diff changeset	682 raise error.ReadOnlyPartError('part is being generated')
21604 c399bf961cb9 bundle2: the ability to set ``data`` attribute of the part is now official Pierre-Yves David <pierre-yves.david@fb.com> parents: 21603 diff changeset	683 self._data = data
c399bf961cb9 bundle2: the ability to set ``data`` attribute of the part is now official Pierre-Yves David <pierre-yves.david@fb.com> parents: 21603 diff changeset	684 def __getdata(self):
c399bf961cb9 bundle2: the ability to set ``data`` attribute of the part is now official Pierre-Yves David <pierre-yves.david@fb.com> parents: 21603 diff changeset	685 return self._data
c399bf961cb9 bundle2: the ability to set ``data`` attribute of the part is now official Pierre-Yves David <pierre-yves.david@fb.com> parents: 21603 diff changeset	686 data = property(__getdata, __setdata)
c399bf961cb9 bundle2: the ability to set ``data`` attribute of the part is now official Pierre-Yves David <pierre-yves.david@fb.com> parents: 21603 diff changeset	687
21605 f9dabfceb259 bundle2: introduce a ``addparam`` method on part Pierre-Yves David <pierre-yves.david@fb.com> parents: 21604 diff changeset	688 @property
f9dabfceb259 bundle2: introduce a ``addparam`` method on part Pierre-Yves David <pierre-yves.david@fb.com> parents: 21604 diff changeset	689 def mandatoryparams(self):
f9dabfceb259 bundle2: introduce a ``addparam`` method on part Pierre-Yves David <pierre-yves.david@fb.com> parents: 21604 diff changeset	690 # make it an immutable tuple to force people through ``addparam``
f9dabfceb259 bundle2: introduce a ``addparam`` method on part Pierre-Yves David <pierre-yves.david@fb.com> parents: 21604 diff changeset	691 return tuple(self._mandatoryparams)
f9dabfceb259 bundle2: introduce a ``addparam`` method on part Pierre-Yves David <pierre-yves.david@fb.com> parents: 21604 diff changeset	692
f9dabfceb259 bundle2: introduce a ``addparam`` method on part Pierre-Yves David <pierre-yves.david@fb.com> parents: 21604 diff changeset	693 @property
f9dabfceb259 bundle2: introduce a ``addparam`` method on part Pierre-Yves David <pierre-yves.david@fb.com> parents: 21604 diff changeset	694 def advisoryparams(self):
f9dabfceb259 bundle2: introduce a ``addparam`` method on part Pierre-Yves David <pierre-yves.david@fb.com> parents: 21604 diff changeset	695 # make it an immutable tuple to force people through ``addparam``
f9dabfceb259 bundle2: introduce a ``addparam`` method on part Pierre-Yves David <pierre-yves.david@fb.com> parents: 21604 diff changeset	696 return tuple(self._advisoryparams)
f9dabfceb259 bundle2: introduce a ``addparam`` method on part Pierre-Yves David <pierre-yves.david@fb.com> parents: 21604 diff changeset	697
f9dabfceb259 bundle2: introduce a ``addparam`` method on part Pierre-Yves David <pierre-yves.david@fb.com> parents: 21604 diff changeset	698 def addparam(self, name, value='', mandatory=True):
f9dabfceb259 bundle2: introduce a ``addparam`` method on part Pierre-Yves David <pierre-yves.david@fb.com> parents: 21604 diff changeset	699 if self._generated is not None:
21618 7568f5c1c801 bundle2: move exception classes into the error module Pierre-Yves David <pierre-yves.david@fb.com> parents: 21617 diff changeset	700 raise error.ReadOnlyPartError('part is being generated')
21607 054fa5176fa7 bundle2: forbid duplicate parameter keys Pierre-Yves David <pierre-yves.david@fb.com> parents: 21606 diff changeset	701 if name in self._seenparams:
054fa5176fa7 bundle2: forbid duplicate parameter keys Pierre-Yves David <pierre-yves.david@fb.com> parents: 21606 diff changeset	702 raise ValueError('duplicated params: %s' % name)
054fa5176fa7 bundle2: forbid duplicate parameter keys Pierre-Yves David <pierre-yves.david@fb.com> parents: 21606 diff changeset	703 self._seenparams.add(name)
21605 f9dabfceb259 bundle2: introduce a ``addparam`` method on part Pierre-Yves David <pierre-yves.david@fb.com> parents: 21604 diff changeset	704 params = self._advisoryparams
f9dabfceb259 bundle2: introduce a ``addparam`` method on part Pierre-Yves David <pierre-yves.david@fb.com> parents: 21604 diff changeset	705 if mandatory:
f9dabfceb259 bundle2: introduce a ``addparam`` method on part Pierre-Yves David <pierre-yves.david@fb.com> parents: 21604 diff changeset	706 params = self._mandatoryparams
f9dabfceb259 bundle2: introduce a ``addparam`` method on part Pierre-Yves David <pierre-yves.david@fb.com> parents: 21604 diff changeset	707 params.append((name, value))
f9dabfceb259 bundle2: introduce a ``addparam`` method on part Pierre-Yves David <pierre-yves.david@fb.com> parents: 21604 diff changeset	708
21601 7ff01befc7ec bundle2: track life cycle of parts Pierre-Yves David <pierre-yves.david@fb.com> parents: 21600 diff changeset	709 # methods used to generates the bundle2 stream
20856 8a6a86c9a5b5 bundle2: support bundling of empty part (with a type) Pierre-Yves David <pierre-yves.david@fb.com> parents: 20844 diff changeset	710 def getchunks(self):
21601 7ff01befc7ec bundle2: track life cycle of parts Pierre-Yves David <pierre-yves.david@fb.com> parents: 21600 diff changeset	711 if self._generated is not None:
7ff01befc7ec bundle2: track life cycle of parts Pierre-Yves David <pierre-yves.david@fb.com> parents: 21600 diff changeset	712 raise RuntimeError('part can only be consumed once')
7ff01befc7ec bundle2: track life cycle of parts Pierre-Yves David <pierre-yves.david@fb.com> parents: 21600 diff changeset	713 self._generated = False
20877 9e9e3a4e9261 bundle2: part params Pierre-Yves David <pierre-yves.david@fb.com> parents: 20876 diff changeset	714 #### header
23590 4440c7cc3728 bundle2.bundlepart: make mandatory part flag explicit in API Eric Sumner <ericsumner@fb.com> parents: 23586 diff changeset	715 if self.mandatory:
4440c7cc3728 bundle2.bundlepart: make mandatory part flag explicit in API Eric Sumner <ericsumner@fb.com> parents: 23586 diff changeset	716 parttype = self.type.upper()
4440c7cc3728 bundle2.bundlepart: make mandatory part flag explicit in API Eric Sumner <ericsumner@fb.com> parents: 23586 diff changeset	717 else:
4440c7cc3728 bundle2.bundlepart: make mandatory part flag explicit in API Eric Sumner <ericsumner@fb.com> parents: 23586 diff changeset	718 parttype = self.type.lower()
20877 9e9e3a4e9261 bundle2: part params Pierre-Yves David <pierre-yves.david@fb.com> parents: 20876 diff changeset	719 ## parttype
23590 4440c7cc3728 bundle2.bundlepart: make mandatory part flag explicit in API Eric Sumner <ericsumner@fb.com> parents: 23586 diff changeset	720 header = [_pack(_fparttypesize, len(parttype)),
4440c7cc3728 bundle2.bundlepart: make mandatory part flag explicit in API Eric Sumner <ericsumner@fb.com> parents: 23586 diff changeset	721 parttype, _pack(_fpartid, self.id),
20856 8a6a86c9a5b5 bundle2: support bundling of empty part (with a type) Pierre-Yves David <pierre-yves.david@fb.com> parents: 20844 diff changeset	722 ]
20877 9e9e3a4e9261 bundle2: part params Pierre-Yves David <pierre-yves.david@fb.com> parents: 20876 diff changeset	723 ## parameters
9e9e3a4e9261 bundle2: part params Pierre-Yves David <pierre-yves.david@fb.com> parents: 20876 diff changeset	724 # count
9e9e3a4e9261 bundle2: part params Pierre-Yves David <pierre-yves.david@fb.com> parents: 20876 diff changeset	725 manpar = self.mandatoryparams
9e9e3a4e9261 bundle2: part params Pierre-Yves David <pierre-yves.david@fb.com> parents: 20876 diff changeset	726 advpar = self.advisoryparams
9e9e3a4e9261 bundle2: part params Pierre-Yves David <pierre-yves.david@fb.com> parents: 20876 diff changeset	727 header.append(_pack(_fpartparamcount, len(manpar), len(advpar)))
9e9e3a4e9261 bundle2: part params Pierre-Yves David <pierre-yves.david@fb.com> parents: 20876 diff changeset	728 # size
9e9e3a4e9261 bundle2: part params Pierre-Yves David <pierre-yves.david@fb.com> parents: 20876 diff changeset	729 parsizes = []
9e9e3a4e9261 bundle2: part params Pierre-Yves David <pierre-yves.david@fb.com> parents: 20876 diff changeset	730 for key, value in manpar:
9e9e3a4e9261 bundle2: part params Pierre-Yves David <pierre-yves.david@fb.com> parents: 20876 diff changeset	731 parsizes.append(len(key))
9e9e3a4e9261 bundle2: part params Pierre-Yves David <pierre-yves.david@fb.com> parents: 20876 diff changeset	732 parsizes.append(len(value))
9e9e3a4e9261 bundle2: part params Pierre-Yves David <pierre-yves.david@fb.com> parents: 20876 diff changeset	733 for key, value in advpar:
9e9e3a4e9261 bundle2: part params Pierre-Yves David <pierre-yves.david@fb.com> parents: 20876 diff changeset	734 parsizes.append(len(key))
9e9e3a4e9261 bundle2: part params Pierre-Yves David <pierre-yves.david@fb.com> parents: 20876 diff changeset	735 parsizes.append(len(value))
9e9e3a4e9261 bundle2: part params Pierre-Yves David <pierre-yves.david@fb.com> parents: 20876 diff changeset	736 paramsizes = _pack(_makefpartparamsizes(len(parsizes) / 2), *parsizes)
9e9e3a4e9261 bundle2: part params Pierre-Yves David <pierre-yves.david@fb.com> parents: 20876 diff changeset	737 header.append(paramsizes)
9e9e3a4e9261 bundle2: part params Pierre-Yves David <pierre-yves.david@fb.com> parents: 20876 diff changeset	738 # key, value
9e9e3a4e9261 bundle2: part params Pierre-Yves David <pierre-yves.david@fb.com> parents: 20876 diff changeset	739 for key, value in manpar:
9e9e3a4e9261 bundle2: part params Pierre-Yves David <pierre-yves.david@fb.com> parents: 20876 diff changeset	740 header.append(key)
9e9e3a4e9261 bundle2: part params Pierre-Yves David <pierre-yves.david@fb.com> parents: 20876 diff changeset	741 header.append(value)
9e9e3a4e9261 bundle2: part params Pierre-Yves David <pierre-yves.david@fb.com> parents: 20876 diff changeset	742 for key, value in advpar:
9e9e3a4e9261 bundle2: part params Pierre-Yves David <pierre-yves.david@fb.com> parents: 20876 diff changeset	743 header.append(key)
9e9e3a4e9261 bundle2: part params Pierre-Yves David <pierre-yves.david@fb.com> parents: 20876 diff changeset	744 header.append(value)
9e9e3a4e9261 bundle2: part params Pierre-Yves David <pierre-yves.david@fb.com> parents: 20876 diff changeset	745 ## finalize header
20856 8a6a86c9a5b5 bundle2: support bundling of empty part (with a type) Pierre-Yves David <pierre-yves.david@fb.com> parents: 20844 diff changeset	746 headerchunk = ''.join(header)
8a6a86c9a5b5 bundle2: support bundling of empty part (with a type) Pierre-Yves David <pierre-yves.david@fb.com> parents: 20844 diff changeset	747 yield _pack(_fpartheadersize, len(headerchunk))
8a6a86c9a5b5 bundle2: support bundling of empty part (with a type) Pierre-Yves David <pierre-yves.david@fb.com> parents: 20844 diff changeset	748 yield headerchunk
20877 9e9e3a4e9261 bundle2: part params Pierre-Yves David <pierre-yves.david@fb.com> parents: 20876 diff changeset	749 ## payload
23067 420a051616ce bundle2: transmit exception during part generation Pierre-Yves David <pierre-yves.david@fb.com> parents: 23066 diff changeset	750 try:
420a051616ce bundle2: transmit exception during part generation Pierre-Yves David <pierre-yves.david@fb.com> parents: 23066 diff changeset	751 for chunk in self._payloadchunks():
420a051616ce bundle2: transmit exception during part generation Pierre-Yves David <pierre-yves.david@fb.com> parents: 23066 diff changeset	752 yield _pack(_fpayloadsize, len(chunk))
420a051616ce bundle2: transmit exception during part generation Pierre-Yves David <pierre-yves.david@fb.com> parents: 23066 diff changeset	753 yield chunk
420a051616ce bundle2: transmit exception during part generation Pierre-Yves David <pierre-yves.david@fb.com> parents: 23066 diff changeset	754 except Exception, exc:
420a051616ce bundle2: transmit exception during part generation Pierre-Yves David <pierre-yves.david@fb.com> parents: 23066 diff changeset	755 # backup exception data for later
420a051616ce bundle2: transmit exception during part generation Pierre-Yves David <pierre-yves.david@fb.com> parents: 23066 diff changeset	756 exc_info = sys.exc_info()
420a051616ce bundle2: transmit exception during part generation Pierre-Yves David <pierre-yves.david@fb.com> parents: 23066 diff changeset	757 msg = 'unexpected error: %s' % exc
24686 e0e28e910fa3 bundle2: rename format, parts and config to final names Pierre-Yves David <pierre-yves.david@fb.com> parents: 24648 diff changeset	758 interpart = bundlepart('error:abort', [('message', msg)],
23590 4440c7cc3728 bundle2.bundlepart: make mandatory part flag explicit in API Eric Sumner <ericsumner@fb.com> parents: 23586 diff changeset	759 mandatory=False)
23067 420a051616ce bundle2: transmit exception during part generation Pierre-Yves David <pierre-yves.david@fb.com> parents: 23066 diff changeset	760 interpart.id = 0
420a051616ce bundle2: transmit exception during part generation Pierre-Yves David <pierre-yves.david@fb.com> parents: 23066 diff changeset	761 yield _pack(_fpayloadsize, -1)
420a051616ce bundle2: transmit exception during part generation Pierre-Yves David <pierre-yves.david@fb.com> parents: 23066 diff changeset	762 for chunk in interpart.getchunks():
420a051616ce bundle2: transmit exception during part generation Pierre-Yves David <pierre-yves.david@fb.com> parents: 23066 diff changeset	763 yield chunk
420a051616ce bundle2: transmit exception during part generation Pierre-Yves David <pierre-yves.david@fb.com> parents: 23066 diff changeset	764 # abort current part payload
420a051616ce bundle2: transmit exception during part generation Pierre-Yves David <pierre-yves.david@fb.com> parents: 23066 diff changeset	765 yield _pack(_fpayloadsize, 0)
420a051616ce bundle2: transmit exception during part generation Pierre-Yves David <pierre-yves.david@fb.com> parents: 23066 diff changeset	766 raise exc_info[0], exc_info[1], exc_info[2]
21000 4cae06ae1562 bundle2: extract a _payloadchunks method for part Pierre-Yves David <pierre-yves.david@fb.com> parents: 20998 diff changeset	767 # end of payload
4cae06ae1562 bundle2: extract a _payloadchunks method for part Pierre-Yves David <pierre-yves.david@fb.com> parents: 20998 diff changeset	768 yield _pack(_fpayloadsize, 0)
21601 7ff01befc7ec bundle2: track life cycle of parts Pierre-Yves David <pierre-yves.david@fb.com> parents: 21600 diff changeset	769 self._generated = True
21000 4cae06ae1562 bundle2: extract a _payloadchunks method for part Pierre-Yves David <pierre-yves.david@fb.com> parents: 20998 diff changeset	770
4cae06ae1562 bundle2: extract a _payloadchunks method for part Pierre-Yves David <pierre-yves.david@fb.com> parents: 20998 diff changeset	771 def _payloadchunks(self):
4cae06ae1562 bundle2: extract a _payloadchunks method for part Pierre-Yves David <pierre-yves.david@fb.com> parents: 20998 diff changeset	772 """yield chunks of a the part payload
4cae06ae1562 bundle2: extract a _payloadchunks method for part Pierre-Yves David <pierre-yves.david@fb.com> parents: 20998 diff changeset	773
4cae06ae1562 bundle2: extract a _payloadchunks method for part Pierre-Yves David <pierre-yves.david@fb.com> parents: 20998 diff changeset	774 Exists to handle the different methods to provide data to a part."""
20876 ddd56f3eb786 bundle2: support for bundling and unbundling payload Pierre-Yves David <pierre-yves.david@fb.com> parents: 20864 diff changeset	775 # we only support fixed size data now.
ddd56f3eb786 bundle2: support for bundling and unbundling payload Pierre-Yves David <pierre-yves.david@fb.com> parents: 20864 diff changeset	776 # This will be improved in the future.
21001 c93bb6a08fa1 bundle2: support chunk iterator as part data Pierre-Yves David <pierre-yves.david@fb.com> parents: 21000 diff changeset	777 if util.safehasattr(self.data, 'next'):
c93bb6a08fa1 bundle2: support chunk iterator as part data Pierre-Yves David <pierre-yves.david@fb.com> parents: 21000 diff changeset	778 buff = util.chunkbuffer(self.data)
c93bb6a08fa1 bundle2: support chunk iterator as part data Pierre-Yves David <pierre-yves.david@fb.com> parents: 21000 diff changeset	779 chunk = buff.read(preferedchunksize)
c93bb6a08fa1 bundle2: support chunk iterator as part data Pierre-Yves David <pierre-yves.david@fb.com> parents: 21000 diff changeset	780 while chunk:
c93bb6a08fa1 bundle2: support chunk iterator as part data Pierre-Yves David <pierre-yves.david@fb.com> parents: 21000 diff changeset	781 yield chunk
c93bb6a08fa1 bundle2: support chunk iterator as part data Pierre-Yves David <pierre-yves.david@fb.com> parents: 21000 diff changeset	782 chunk = buff.read(preferedchunksize)
c93bb6a08fa1 bundle2: support chunk iterator as part data Pierre-Yves David <pierre-yves.david@fb.com> parents: 21000 diff changeset	783 elif len(self.data):
20876 ddd56f3eb786 bundle2: support for bundling and unbundling payload Pierre-Yves David <pierre-yves.david@fb.com> parents: 20864 diff changeset	784 yield self.data
20802 520df53ad26a bundle2: a very first version of bundle2 unbundler Pierre-Yves David <pierre-yves.david@fb.com> parents: 20801 diff changeset	785
23066 ad144882318d bundle2: add a interrupt mechanism Pierre-Yves David <pierre-yves.david@fb.com> parents: 23029 diff changeset	786
ad144882318d bundle2: add a interrupt mechanism Pierre-Yves David <pierre-yves.david@fb.com> parents: 23029 diff changeset	787 flaginterrupt = -1
ad144882318d bundle2: add a interrupt mechanism Pierre-Yves David <pierre-yves.david@fb.com> parents: 23029 diff changeset	788
ad144882318d bundle2: add a interrupt mechanism Pierre-Yves David <pierre-yves.david@fb.com> parents: 23029 diff changeset	789 class interrupthandler(unpackermixin):
ad144882318d bundle2: add a interrupt mechanism Pierre-Yves David <pierre-yves.david@fb.com> parents: 23029 diff changeset	790 """read one part and process it with restricted capability
ad144882318d bundle2: add a interrupt mechanism Pierre-Yves David <pierre-yves.david@fb.com> parents: 23029 diff changeset	791
ad144882318d bundle2: add a interrupt mechanism Pierre-Yves David <pierre-yves.david@fb.com> parents: 23029 diff changeset	792 This allows to transmit exception raised on the producer size during part
ad144882318d bundle2: add a interrupt mechanism Pierre-Yves David <pierre-yves.david@fb.com> parents: 23029 diff changeset	793 iteration while the consumer is reading a part.
ad144882318d bundle2: add a interrupt mechanism Pierre-Yves David <pierre-yves.david@fb.com> parents: 23029 diff changeset	794
ad144882318d bundle2: add a interrupt mechanism Pierre-Yves David <pierre-yves.david@fb.com> parents: 23029 diff changeset	795 Part processed in this manner only have access to a ui object,"""
ad144882318d bundle2: add a interrupt mechanism Pierre-Yves David <pierre-yves.david@fb.com> parents: 23029 diff changeset	796
ad144882318d bundle2: add a interrupt mechanism Pierre-Yves David <pierre-yves.david@fb.com> parents: 23029 diff changeset	797 def __init__(self, ui, fp):
ad144882318d bundle2: add a interrupt mechanism Pierre-Yves David <pierre-yves.david@fb.com> parents: 23029 diff changeset	798 super(interrupthandler, self).__init__(fp)
ad144882318d bundle2: add a interrupt mechanism Pierre-Yves David <pierre-yves.david@fb.com> parents: 23029 diff changeset	799 self.ui = ui
ad144882318d bundle2: add a interrupt mechanism Pierre-Yves David <pierre-yves.david@fb.com> parents: 23029 diff changeset	800
ad144882318d bundle2: add a interrupt mechanism Pierre-Yves David <pierre-yves.david@fb.com> parents: 23029 diff changeset	801 def _readpartheader(self):
ad144882318d bundle2: add a interrupt mechanism Pierre-Yves David <pierre-yves.david@fb.com> parents: 23029 diff changeset	802 """reads a part header size and return the bytes blob
ad144882318d bundle2: add a interrupt mechanism Pierre-Yves David <pierre-yves.david@fb.com> parents: 23029 diff changeset	803
ad144882318d bundle2: add a interrupt mechanism Pierre-Yves David <pierre-yves.david@fb.com> parents: 23029 diff changeset	804 returns None if empty"""
ad144882318d bundle2: add a interrupt mechanism Pierre-Yves David <pierre-yves.david@fb.com> parents: 23029 diff changeset	805 headersize = self._unpack(_fpartheadersize)[0]
ad144882318d bundle2: add a interrupt mechanism Pierre-Yves David <pierre-yves.david@fb.com> parents: 23029 diff changeset	806 if headersize < 0:
ad144882318d bundle2: add a interrupt mechanism Pierre-Yves David <pierre-yves.david@fb.com> parents: 23029 diff changeset	807 raise error.BundleValueError('negative part header size: %i'
ad144882318d bundle2: add a interrupt mechanism Pierre-Yves David <pierre-yves.david@fb.com> parents: 23029 diff changeset	808 % headersize)
ad144882318d bundle2: add a interrupt mechanism Pierre-Yves David <pierre-yves.david@fb.com> parents: 23029 diff changeset	809 self.ui.debug('part header size: %i\n' % headersize)
ad144882318d bundle2: add a interrupt mechanism Pierre-Yves David <pierre-yves.david@fb.com> parents: 23029 diff changeset	810 if headersize:
ad144882318d bundle2: add a interrupt mechanism Pierre-Yves David <pierre-yves.david@fb.com> parents: 23029 diff changeset	811 return self._readexact(headersize)
ad144882318d bundle2: add a interrupt mechanism Pierre-Yves David <pierre-yves.david@fb.com> parents: 23029 diff changeset	812 return None
ad144882318d bundle2: add a interrupt mechanism Pierre-Yves David <pierre-yves.david@fb.com> parents: 23029 diff changeset	813
ad144882318d bundle2: add a interrupt mechanism Pierre-Yves David <pierre-yves.david@fb.com> parents: 23029 diff changeset	814 def __call__(self):
ad144882318d bundle2: add a interrupt mechanism Pierre-Yves David <pierre-yves.david@fb.com> parents: 23029 diff changeset	815 self.ui.debug('bundle2 stream interruption, looking for a part.\n')
ad144882318d bundle2: add a interrupt mechanism Pierre-Yves David <pierre-yves.david@fb.com> parents: 23029 diff changeset	816 headerblock = self._readpartheader()
ad144882318d bundle2: add a interrupt mechanism Pierre-Yves David <pierre-yves.david@fb.com> parents: 23029 diff changeset	817 if headerblock is None:
23139 e53f6b72a0e4 spelling: fixes from proofreading of spell checker issues Mads Kiilerich <madski@unity3d.com> parents: 23067 diff changeset	818 self.ui.debug('no part found during interruption.\n')
23066 ad144882318d bundle2: add a interrupt mechanism Pierre-Yves David <pierre-yves.david@fb.com> parents: 23029 diff changeset	819 return
ad144882318d bundle2: add a interrupt mechanism Pierre-Yves David <pierre-yves.david@fb.com> parents: 23029 diff changeset	820 part = unbundlepart(self.ui, headerblock, self._fp)
ad144882318d bundle2: add a interrupt mechanism Pierre-Yves David <pierre-yves.david@fb.com> parents: 23029 diff changeset	821 op = interruptoperation(self.ui)
ad144882318d bundle2: add a interrupt mechanism Pierre-Yves David <pierre-yves.david@fb.com> parents: 23029 diff changeset	822 _processpart(op, part)
ad144882318d bundle2: add a interrupt mechanism Pierre-Yves David <pierre-yves.david@fb.com> parents: 23029 diff changeset	823
ad144882318d bundle2: add a interrupt mechanism Pierre-Yves David <pierre-yves.david@fb.com> parents: 23029 diff changeset	824 class interruptoperation(object):
ad144882318d bundle2: add a interrupt mechanism Pierre-Yves David <pierre-yves.david@fb.com> parents: 23029 diff changeset	825 """A limited operation to be use by part handler during interruption
ad144882318d bundle2: add a interrupt mechanism Pierre-Yves David <pierre-yves.david@fb.com> parents: 23029 diff changeset	826
ad144882318d bundle2: add a interrupt mechanism Pierre-Yves David <pierre-yves.david@fb.com> parents: 23029 diff changeset	827 It only have access to an ui object.
ad144882318d bundle2: add a interrupt mechanism Pierre-Yves David <pierre-yves.david@fb.com> parents: 23029 diff changeset	828 """
ad144882318d bundle2: add a interrupt mechanism Pierre-Yves David <pierre-yves.david@fb.com> parents: 23029 diff changeset	829
ad144882318d bundle2: add a interrupt mechanism Pierre-Yves David <pierre-yves.david@fb.com> parents: 23029 diff changeset	830 def __init__(self, ui):
ad144882318d bundle2: add a interrupt mechanism Pierre-Yves David <pierre-yves.david@fb.com> parents: 23029 diff changeset	831 self.ui = ui
ad144882318d bundle2: add a interrupt mechanism Pierre-Yves David <pierre-yves.david@fb.com> parents: 23029 diff changeset	832 self.reply = None
ad144882318d bundle2: add a interrupt mechanism Pierre-Yves David <pierre-yves.david@fb.com> parents: 23029 diff changeset	833
ad144882318d bundle2: add a interrupt mechanism Pierre-Yves David <pierre-yves.david@fb.com> parents: 23029 diff changeset	834 @property
ad144882318d bundle2: add a interrupt mechanism Pierre-Yves David <pierre-yves.david@fb.com> parents: 23029 diff changeset	835 def repo(self):
ad144882318d bundle2: add a interrupt mechanism Pierre-Yves David <pierre-yves.david@fb.com> parents: 23029 diff changeset	836 raise RuntimeError('no repo access from stream interruption')
ad144882318d bundle2: add a interrupt mechanism Pierre-Yves David <pierre-yves.david@fb.com> parents: 23029 diff changeset	837
ad144882318d bundle2: add a interrupt mechanism Pierre-Yves David <pierre-yves.david@fb.com> parents: 23029 diff changeset	838 def gettransaction(self):
ad144882318d bundle2: add a interrupt mechanism Pierre-Yves David <pierre-yves.david@fb.com> parents: 23029 diff changeset	839 raise TransactionUnavailable('no repo access from stream interruption')
ad144882318d bundle2: add a interrupt mechanism Pierre-Yves David <pierre-yves.david@fb.com> parents: 23029 diff changeset	840
21014 a6246bba7b9e bundle2: add an unbundle part responsible from unbundling part Pierre-Yves David <pierre-yves.david@fb.com> parents: 21013 diff changeset	841 class unbundlepart(unpackermixin):
a6246bba7b9e bundle2: add an unbundle part responsible from unbundling part Pierre-Yves David <pierre-yves.david@fb.com> parents: 21013 diff changeset	842 """a bundle part read from a bundle"""
a6246bba7b9e bundle2: add an unbundle part responsible from unbundling part Pierre-Yves David <pierre-yves.david@fb.com> parents: 21013 diff changeset	843
a6246bba7b9e bundle2: add an unbundle part responsible from unbundling part Pierre-Yves David <pierre-yves.david@fb.com> parents: 21013 diff changeset	844 def __init__(self, ui, header, fp):
a6246bba7b9e bundle2: add an unbundle part responsible from unbundling part Pierre-Yves David <pierre-yves.david@fb.com> parents: 21013 diff changeset	845 super(unbundlepart, self).__init__(fp)
a6246bba7b9e bundle2: add an unbundle part responsible from unbundling part Pierre-Yves David <pierre-yves.david@fb.com> parents: 21013 diff changeset	846 self.ui = ui
a6246bba7b9e bundle2: add an unbundle part responsible from unbundling part Pierre-Yves David <pierre-yves.david@fb.com> parents: 21013 diff changeset	847 # unbundle state attr
a6246bba7b9e bundle2: add an unbundle part responsible from unbundling part Pierre-Yves David <pierre-yves.david@fb.com> parents: 21013 diff changeset	848 self._headerdata = header
21015 14dd49260246 bundle2: move the fromheader closure into the class itself Pierre-Yves David <pierre-yves.david@fb.com> parents: 21014 diff changeset	849 self._headeroffset = 0
21019 3dc09f831a2e bundle2: lazy unbundle of part payload Pierre-Yves David <pierre-yves.david@fb.com> parents: 21016 diff changeset	850 self._initialized = False
3dc09f831a2e bundle2: lazy unbundle of part payload Pierre-Yves David <pierre-yves.david@fb.com> parents: 21016 diff changeset	851 self.consumed = False
21014 a6246bba7b9e bundle2: add an unbundle part responsible from unbundling part Pierre-Yves David <pierre-yves.david@fb.com> parents: 21013 diff changeset	852 # part data
a6246bba7b9e bundle2: add an unbundle part responsible from unbundling part Pierre-Yves David <pierre-yves.david@fb.com> parents: 21013 diff changeset	853 self.id = None
a6246bba7b9e bundle2: add an unbundle part responsible from unbundling part Pierre-Yves David <pierre-yves.david@fb.com> parents: 21013 diff changeset	854 self.type = None
a6246bba7b9e bundle2: add an unbundle part responsible from unbundling part Pierre-Yves David <pierre-yves.david@fb.com> parents: 21013 diff changeset	855 self.mandatoryparams = None
a6246bba7b9e bundle2: add an unbundle part responsible from unbundling part Pierre-Yves David <pierre-yves.david@fb.com> parents: 21013 diff changeset	856 self.advisoryparams = None
21610 d6056805f8f4 bundle2: introduce a ``params`` dictionary on unbundled parts Pierre-Yves David <pierre-yves.david@fb.com> parents: 21609 diff changeset	857 self.params = None
21612 f221eb0531d9 bundle2: expose mandatory params in a mandatorykeys attribute Pierre-Yves David <pierre-yves.david@fb.com> parents: 21611 diff changeset	858 self.mandatorykeys = ()
21019 3dc09f831a2e bundle2: lazy unbundle of part payload Pierre-Yves David <pierre-yves.david@fb.com> parents: 21016 diff changeset	859 self._payloadstream = None
3dc09f831a2e bundle2: lazy unbundle of part payload Pierre-Yves David <pierre-yves.david@fb.com> parents: 21016 diff changeset	860 self._readheader()
23585 94b25d71dd0f bundle2.unbundlepart: decouple mandatory from parttype Eric Sumner <ericsumner@fb.com> parents: 23439 diff changeset	861 self._mandatory = None
24035 7eb26415bef6 bundle2.unbundlepart: keep an index of chunks and their locations Eric Sumner <ericsumner@fb.com> parents: 24034 diff changeset	862 self._chunkindex = [] #(payload, file) position tuples for chunk starts
24036 c7601086338a bundle2.unbundlepart: tell() implementation Eric Sumner <ericsumner@fb.com> parents: 24035 diff changeset	863 self._pos = 0
21014 a6246bba7b9e bundle2: add an unbundle part responsible from unbundling part Pierre-Yves David <pierre-yves.david@fb.com> parents: 21013 diff changeset	864
21015 14dd49260246 bundle2: move the fromheader closure into the class itself Pierre-Yves David <pierre-yves.david@fb.com> parents: 21014 diff changeset	865 def _fromheader(self, size):
14dd49260246 bundle2: move the fromheader closure into the class itself Pierre-Yves David <pierre-yves.david@fb.com> parents: 21014 diff changeset	866 """return the next <size> byte from the header"""
14dd49260246 bundle2: move the fromheader closure into the class itself Pierre-Yves David <pierre-yves.david@fb.com> parents: 21014 diff changeset	867 offset = self._headeroffset
14dd49260246 bundle2: move the fromheader closure into the class itself Pierre-Yves David <pierre-yves.david@fb.com> parents: 21014 diff changeset	868 data = self._headerdata[offset:(offset + size)]
21019 3dc09f831a2e bundle2: lazy unbundle of part payload Pierre-Yves David <pierre-yves.david@fb.com> parents: 21016 diff changeset	869 self._headeroffset = offset + size
21015 14dd49260246 bundle2: move the fromheader closure into the class itself Pierre-Yves David <pierre-yves.david@fb.com> parents: 21014 diff changeset	870 return data
14dd49260246 bundle2: move the fromheader closure into the class itself Pierre-Yves David <pierre-yves.david@fb.com> parents: 21014 diff changeset	871
21016 b477afb1c81e bundle2: move unpackheader closure into the class Pierre-Yves David <pierre-yves.david@fb.com> parents: 21015 diff changeset	872 def _unpackheader(self, format):
b477afb1c81e bundle2: move unpackheader closure into the class Pierre-Yves David <pierre-yves.david@fb.com> parents: 21015 diff changeset	873 """read given format from header
b477afb1c81e bundle2: move unpackheader closure into the class Pierre-Yves David <pierre-yves.david@fb.com> parents: 21015 diff changeset	874
b477afb1c81e bundle2: move unpackheader closure into the class Pierre-Yves David <pierre-yves.david@fb.com> parents: 21015 diff changeset	875 This automatically compute the size of the format to read."""
b477afb1c81e bundle2: move unpackheader closure into the class Pierre-Yves David <pierre-yves.david@fb.com> parents: 21015 diff changeset	876 data = self._fromheader(struct.calcsize(format))
b477afb1c81e bundle2: move unpackheader closure into the class Pierre-Yves David <pierre-yves.david@fb.com> parents: 21015 diff changeset	877 return _unpack(format, data)
b477afb1c81e bundle2: move unpackheader closure into the class Pierre-Yves David <pierre-yves.david@fb.com> parents: 21015 diff changeset	878
21608 3cb96ca90c17 bundle2: introduce an ``_initparams`` method Pierre-Yves David <pierre-yves.david@fb.com> parents: 21607 diff changeset	879 def _initparams(self, mandatoryparams, advisoryparams):
3cb96ca90c17 bundle2: introduce an ``_initparams`` method Pierre-Yves David <pierre-yves.david@fb.com> parents: 21607 diff changeset	880 """internal function to setup all logic related parameters"""
21609 63cc2594ef8a bundle2: make sure unbundled part param are read-only Pierre-Yves David <pierre-yves.david@fb.com> parents: 21608 diff changeset	881 # make it read only to prevent people touching it by mistake.
63cc2594ef8a bundle2: make sure unbundled part param are read-only Pierre-Yves David <pierre-yves.david@fb.com> parents: 21608 diff changeset	882 self.mandatoryparams = tuple(mandatoryparams)
63cc2594ef8a bundle2: make sure unbundled part param are read-only Pierre-Yves David <pierre-yves.david@fb.com> parents: 21608 diff changeset	883 self.advisoryparams = tuple(advisoryparams)
21610 d6056805f8f4 bundle2: introduce a ``params`` dictionary on unbundled parts Pierre-Yves David <pierre-yves.david@fb.com> parents: 21609 diff changeset	884 # user friendly UI
d6056805f8f4 bundle2: introduce a ``params`` dictionary on unbundled parts Pierre-Yves David <pierre-yves.david@fb.com> parents: 21609 diff changeset	885 self.params = dict(self.mandatoryparams)
d6056805f8f4 bundle2: introduce a ``params`` dictionary on unbundled parts Pierre-Yves David <pierre-yves.david@fb.com> parents: 21609 diff changeset	886 self.params.update(dict(self.advisoryparams))
21612 f221eb0531d9 bundle2: expose mandatory params in a mandatorykeys attribute Pierre-Yves David <pierre-yves.david@fb.com> parents: 21611 diff changeset	887 self.mandatorykeys = frozenset(p[0] for p in mandatoryparams)
21608 3cb96ca90c17 bundle2: introduce an ``_initparams`` method Pierre-Yves David <pierre-yves.david@fb.com> parents: 21607 diff changeset	888
24035 7eb26415bef6 bundle2.unbundlepart: keep an index of chunks and their locations Eric Sumner <ericsumner@fb.com> parents: 24034 diff changeset	889 def _payloadchunks(self, chunknum=0):
7eb26415bef6 bundle2.unbundlepart: keep an index of chunks and their locations Eric Sumner <ericsumner@fb.com> parents: 24034 diff changeset	890 '''seek to specified chunk and start yielding data'''
7eb26415bef6 bundle2.unbundlepart: keep an index of chunks and their locations Eric Sumner <ericsumner@fb.com> parents: 24034 diff changeset	891 if len(self._chunkindex) == 0:
7eb26415bef6 bundle2.unbundlepart: keep an index of chunks and their locations Eric Sumner <ericsumner@fb.com> parents: 24034 diff changeset	892 assert chunknum == 0, 'Must start with chunk 0'
7eb26415bef6 bundle2.unbundlepart: keep an index of chunks and their locations Eric Sumner <ericsumner@fb.com> parents: 24034 diff changeset	893 self._chunkindex.append((0, super(unbundlepart, self).tell()))
7eb26415bef6 bundle2.unbundlepart: keep an index of chunks and their locations Eric Sumner <ericsumner@fb.com> parents: 24034 diff changeset	894 else:
7eb26415bef6 bundle2.unbundlepart: keep an index of chunks and their locations Eric Sumner <ericsumner@fb.com> parents: 24034 diff changeset	895 assert chunknum < len(self._chunkindex), \
7eb26415bef6 bundle2.unbundlepart: keep an index of chunks and their locations Eric Sumner <ericsumner@fb.com> parents: 24034 diff changeset	896 'Unknown chunk %d' % chunknum
7eb26415bef6 bundle2.unbundlepart: keep an index of chunks and their locations Eric Sumner <ericsumner@fb.com> parents: 24034 diff changeset	897 super(unbundlepart, self).seek(self._chunkindex[chunknum][1])
7eb26415bef6 bundle2.unbundlepart: keep an index of chunks and their locations Eric Sumner <ericsumner@fb.com> parents: 24034 diff changeset	898
7eb26415bef6 bundle2.unbundlepart: keep an index of chunks and their locations Eric Sumner <ericsumner@fb.com> parents: 24034 diff changeset	899 pos = self._chunkindex[chunknum][0]
24034 9881a1437799 bundle2.unbundlepart: raise payloadchunks from a closure to a method Eric Sumner <ericsumner@fb.com> parents: 24026 diff changeset	900 payloadsize = self._unpack(_fpayloadsize)[0]
9881a1437799 bundle2.unbundlepart: raise payloadchunks from a closure to a method Eric Sumner <ericsumner@fb.com> parents: 24026 diff changeset	901 self.ui.debug('payload chunk size: %i\n' % payloadsize)
9881a1437799 bundle2.unbundlepart: raise payloadchunks from a closure to a method Eric Sumner <ericsumner@fb.com> parents: 24026 diff changeset	902 while payloadsize:
9881a1437799 bundle2.unbundlepart: raise payloadchunks from a closure to a method Eric Sumner <ericsumner@fb.com> parents: 24026 diff changeset	903 if payloadsize == flaginterrupt:
9881a1437799 bundle2.unbundlepart: raise payloadchunks from a closure to a method Eric Sumner <ericsumner@fb.com> parents: 24026 diff changeset	904 # interruption detection, the handler will now read a
9881a1437799 bundle2.unbundlepart: raise payloadchunks from a closure to a method Eric Sumner <ericsumner@fb.com> parents: 24026 diff changeset	905 # single part and process it.
9881a1437799 bundle2.unbundlepart: raise payloadchunks from a closure to a method Eric Sumner <ericsumner@fb.com> parents: 24026 diff changeset	906 interrupthandler(self.ui, self._fp)()
9881a1437799 bundle2.unbundlepart: raise payloadchunks from a closure to a method Eric Sumner <ericsumner@fb.com> parents: 24026 diff changeset	907 elif payloadsize < 0:
9881a1437799 bundle2.unbundlepart: raise payloadchunks from a closure to a method Eric Sumner <ericsumner@fb.com> parents: 24026 diff changeset	908 msg = 'negative payload chunk size: %i' % payloadsize
9881a1437799 bundle2.unbundlepart: raise payloadchunks from a closure to a method Eric Sumner <ericsumner@fb.com> parents: 24026 diff changeset	909 raise error.BundleValueError(msg)
9881a1437799 bundle2.unbundlepart: raise payloadchunks from a closure to a method Eric Sumner <ericsumner@fb.com> parents: 24026 diff changeset	910 else:
24035 7eb26415bef6 bundle2.unbundlepart: keep an index of chunks and their locations Eric Sumner <ericsumner@fb.com> parents: 24034 diff changeset	911 result = self._readexact(payloadsize)
7eb26415bef6 bundle2.unbundlepart: keep an index of chunks and their locations Eric Sumner <ericsumner@fb.com> parents: 24034 diff changeset	912 chunknum += 1
7eb26415bef6 bundle2.unbundlepart: keep an index of chunks and their locations Eric Sumner <ericsumner@fb.com> parents: 24034 diff changeset	913 pos += payloadsize
7eb26415bef6 bundle2.unbundlepart: keep an index of chunks and their locations Eric Sumner <ericsumner@fb.com> parents: 24034 diff changeset	914 if chunknum == len(self._chunkindex):
7eb26415bef6 bundle2.unbundlepart: keep an index of chunks and their locations Eric Sumner <ericsumner@fb.com> parents: 24034 diff changeset	915 self._chunkindex.append((pos,
7eb26415bef6 bundle2.unbundlepart: keep an index of chunks and their locations Eric Sumner <ericsumner@fb.com> parents: 24034 diff changeset	916 super(unbundlepart, self).tell()))
7eb26415bef6 bundle2.unbundlepart: keep an index of chunks and their locations Eric Sumner <ericsumner@fb.com> parents: 24034 diff changeset	917 yield result
24034 9881a1437799 bundle2.unbundlepart: raise payloadchunks from a closure to a method Eric Sumner <ericsumner@fb.com> parents: 24026 diff changeset	918 payloadsize = self._unpack(_fpayloadsize)[0]
9881a1437799 bundle2.unbundlepart: raise payloadchunks from a closure to a method Eric Sumner <ericsumner@fb.com> parents: 24026 diff changeset	919 self.ui.debug('payload chunk size: %i\n' % payloadsize)
9881a1437799 bundle2.unbundlepart: raise payloadchunks from a closure to a method Eric Sumner <ericsumner@fb.com> parents: 24026 diff changeset	920
24037 f0b498cfc5c8 bundle2.unbundlepart: implement seek() Eric Sumner <ericsumner@fb.com> parents: 24036 diff changeset	921 def _findchunk(self, pos):
f0b498cfc5c8 bundle2.unbundlepart: implement seek() Eric Sumner <ericsumner@fb.com> parents: 24036 diff changeset	922 '''for a given payload position, return a chunk number and offset'''
f0b498cfc5c8 bundle2.unbundlepart: implement seek() Eric Sumner <ericsumner@fb.com> parents: 24036 diff changeset	923 for chunk, (ppos, fpos) in enumerate(self._chunkindex):
f0b498cfc5c8 bundle2.unbundlepart: implement seek() Eric Sumner <ericsumner@fb.com> parents: 24036 diff changeset	924 if ppos == pos:
f0b498cfc5c8 bundle2.unbundlepart: implement seek() Eric Sumner <ericsumner@fb.com> parents: 24036 diff changeset	925 return chunk, 0
f0b498cfc5c8 bundle2.unbundlepart: implement seek() Eric Sumner <ericsumner@fb.com> parents: 24036 diff changeset	926 elif ppos > pos:
f0b498cfc5c8 bundle2.unbundlepart: implement seek() Eric Sumner <ericsumner@fb.com> parents: 24036 diff changeset	927 return chunk - 1, pos - self._chunkindex[chunk - 1][0]
f0b498cfc5c8 bundle2.unbundlepart: implement seek() Eric Sumner <ericsumner@fb.com> parents: 24036 diff changeset	928 raise ValueError('Unknown chunk')
f0b498cfc5c8 bundle2.unbundlepart: implement seek() Eric Sumner <ericsumner@fb.com> parents: 24036 diff changeset	929
21019 3dc09f831a2e bundle2: lazy unbundle of part payload Pierre-Yves David <pierre-yves.david@fb.com> parents: 21016 diff changeset	930 def _readheader(self):
21014 a6246bba7b9e bundle2: add an unbundle part responsible from unbundling part Pierre-Yves David <pierre-yves.david@fb.com> parents: 21013 diff changeset	931 """read the header and setup the object"""
21016 b477afb1c81e bundle2: move unpackheader closure into the class Pierre-Yves David <pierre-yves.david@fb.com> parents: 21015 diff changeset	932 typesize = self._unpackheader(_fparttypesize)[0]
21015 14dd49260246 bundle2: move the fromheader closure into the class itself Pierre-Yves David <pierre-yves.david@fb.com> parents: 21014 diff changeset	933 self.type = self._fromheader(typesize)
21014 a6246bba7b9e bundle2: add an unbundle part responsible from unbundling part Pierre-Yves David <pierre-yves.david@fb.com> parents: 21013 diff changeset	934 self.ui.debug('part type: "%s"\n' % self.type)
21016 b477afb1c81e bundle2: move unpackheader closure into the class Pierre-Yves David <pierre-yves.david@fb.com> parents: 21015 diff changeset	935 self.id = self._unpackheader(_fpartid)[0]
21014 a6246bba7b9e bundle2: add an unbundle part responsible from unbundling part Pierre-Yves David <pierre-yves.david@fb.com> parents: 21013 diff changeset	936 self.ui.debug('part id: "%s"\n' % self.id)
23585 94b25d71dd0f bundle2.unbundlepart: decouple mandatory from parttype Eric Sumner <ericsumner@fb.com> parents: 23439 diff changeset	937 # extract mandatory bit from type
94b25d71dd0f bundle2.unbundlepart: decouple mandatory from parttype Eric Sumner <ericsumner@fb.com> parents: 23439 diff changeset	938 self.mandatory = (self.type != self.type.lower())
94b25d71dd0f bundle2.unbundlepart: decouple mandatory from parttype Eric Sumner <ericsumner@fb.com> parents: 23439 diff changeset	939 self.type = self.type.lower()
21014 a6246bba7b9e bundle2: add an unbundle part responsible from unbundling part Pierre-Yves David <pierre-yves.david@fb.com> parents: 21013 diff changeset	940 ## reading parameters
a6246bba7b9e bundle2: add an unbundle part responsible from unbundling part Pierre-Yves David <pierre-yves.david@fb.com> parents: 21013 diff changeset	941 # param count
21016 b477afb1c81e bundle2: move unpackheader closure into the class Pierre-Yves David <pierre-yves.david@fb.com> parents: 21015 diff changeset	942 mancount, advcount = self._unpackheader(_fpartparamcount)
21014 a6246bba7b9e bundle2: add an unbundle part responsible from unbundling part Pierre-Yves David <pierre-yves.david@fb.com> parents: 21013 diff changeset	943 self.ui.debug('part parameters: %i\n' % (mancount + advcount))
a6246bba7b9e bundle2: add an unbundle part responsible from unbundling part Pierre-Yves David <pierre-yves.david@fb.com> parents: 21013 diff changeset	944 # param size
21016 b477afb1c81e bundle2: move unpackheader closure into the class Pierre-Yves David <pierre-yves.david@fb.com> parents: 21015 diff changeset	945 fparamsizes = _makefpartparamsizes(mancount + advcount)
b477afb1c81e bundle2: move unpackheader closure into the class Pierre-Yves David <pierre-yves.david@fb.com> parents: 21015 diff changeset	946 paramsizes = self._unpackheader(fparamsizes)
21014 a6246bba7b9e bundle2: add an unbundle part responsible from unbundling part Pierre-Yves David <pierre-yves.david@fb.com> parents: 21013 diff changeset	947 # make it a list of couple again
a6246bba7b9e bundle2: add an unbundle part responsible from unbundling part Pierre-Yves David <pierre-yves.david@fb.com> parents: 21013 diff changeset	948 paramsizes = zip(paramsizes[::2], paramsizes[1::2])
a6246bba7b9e bundle2: add an unbundle part responsible from unbundling part Pierre-Yves David <pierre-yves.david@fb.com> parents: 21013 diff changeset	949 # split mandatory from advisory
a6246bba7b9e bundle2: add an unbundle part responsible from unbundling part Pierre-Yves David <pierre-yves.david@fb.com> parents: 21013 diff changeset	950 mansizes = paramsizes[:mancount]
a6246bba7b9e bundle2: add an unbundle part responsible from unbundling part Pierre-Yves David <pierre-yves.david@fb.com> parents: 21013 diff changeset	951 advsizes = paramsizes[mancount:]
23139 e53f6b72a0e4 spelling: fixes from proofreading of spell checker issues Mads Kiilerich <madski@unity3d.com> parents: 23067 diff changeset	952 # retrieve param value
21014 a6246bba7b9e bundle2: add an unbundle part responsible from unbundling part Pierre-Yves David <pierre-yves.david@fb.com> parents: 21013 diff changeset	953 manparams = []
a6246bba7b9e bundle2: add an unbundle part responsible from unbundling part Pierre-Yves David <pierre-yves.david@fb.com> parents: 21013 diff changeset	954 for key, value in mansizes:
21015 14dd49260246 bundle2: move the fromheader closure into the class itself Pierre-Yves David <pierre-yves.david@fb.com> parents: 21014 diff changeset	955 manparams.append((self._fromheader(key), self._fromheader(value)))
21014 a6246bba7b9e bundle2: add an unbundle part responsible from unbundling part Pierre-Yves David <pierre-yves.david@fb.com> parents: 21013 diff changeset	956 advparams = []
a6246bba7b9e bundle2: add an unbundle part responsible from unbundling part Pierre-Yves David <pierre-yves.david@fb.com> parents: 21013 diff changeset	957 for key, value in advsizes:
21015 14dd49260246 bundle2: move the fromheader closure into the class itself Pierre-Yves David <pierre-yves.david@fb.com> parents: 21014 diff changeset	958 advparams.append((self._fromheader(key), self._fromheader(value)))
21608 3cb96ca90c17 bundle2: introduce an ``_initparams`` method Pierre-Yves David <pierre-yves.david@fb.com> parents: 21607 diff changeset	959 self._initparams(manparams, advparams)
21014 a6246bba7b9e bundle2: add an unbundle part responsible from unbundling part Pierre-Yves David <pierre-yves.david@fb.com> parents: 21013 diff changeset	960 ## part payload
24034 9881a1437799 bundle2.unbundlepart: raise payloadchunks from a closure to a method Eric Sumner <ericsumner@fb.com> parents: 24026 diff changeset	961 self._payloadstream = util.chunkbuffer(self._payloadchunks())
21019 3dc09f831a2e bundle2: lazy unbundle of part payload Pierre-Yves David <pierre-yves.david@fb.com> parents: 21016 diff changeset	962 # we read the data, tell it
3dc09f831a2e bundle2: lazy unbundle of part payload Pierre-Yves David <pierre-yves.david@fb.com> parents: 21016 diff changeset	963 self._initialized = True
3dc09f831a2e bundle2: lazy unbundle of part payload Pierre-Yves David <pierre-yves.david@fb.com> parents: 21016 diff changeset	964
3dc09f831a2e bundle2: lazy unbundle of part payload Pierre-Yves David <pierre-yves.david@fb.com> parents: 21016 diff changeset	965 def read(self, size=None):
3dc09f831a2e bundle2: lazy unbundle of part payload Pierre-Yves David <pierre-yves.david@fb.com> parents: 21016 diff changeset	966 """read payload data"""
3dc09f831a2e bundle2: lazy unbundle of part payload Pierre-Yves David <pierre-yves.david@fb.com> parents: 21016 diff changeset	967 if not self._initialized:
3dc09f831a2e bundle2: lazy unbundle of part payload Pierre-Yves David <pierre-yves.david@fb.com> parents: 21016 diff changeset	968 self._readheader()
3dc09f831a2e bundle2: lazy unbundle of part payload Pierre-Yves David <pierre-yves.david@fb.com> parents: 21016 diff changeset	969 if size is None:
3dc09f831a2e bundle2: lazy unbundle of part payload Pierre-Yves David <pierre-yves.david@fb.com> parents: 21016 diff changeset	970 data = self._payloadstream.read()
3dc09f831a2e bundle2: lazy unbundle of part payload Pierre-Yves David <pierre-yves.david@fb.com> parents: 21016 diff changeset	971 else:
3dc09f831a2e bundle2: lazy unbundle of part payload Pierre-Yves David <pierre-yves.david@fb.com> parents: 21016 diff changeset	972 data = self._payloadstream.read(size)
3dc09f831a2e bundle2: lazy unbundle of part payload Pierre-Yves David <pierre-yves.david@fb.com> parents: 21016 diff changeset	973 if size is None or len(data) < size:
3dc09f831a2e bundle2: lazy unbundle of part payload Pierre-Yves David <pierre-yves.david@fb.com> parents: 21016 diff changeset	974 self.consumed = True
24036 c7601086338a bundle2.unbundlepart: tell() implementation Eric Sumner <ericsumner@fb.com> parents: 24035 diff changeset	975 self._pos += len(data)
21019 3dc09f831a2e bundle2: lazy unbundle of part payload Pierre-Yves David <pierre-yves.david@fb.com> parents: 21016 diff changeset	976 return data
3dc09f831a2e bundle2: lazy unbundle of part payload Pierre-Yves David <pierre-yves.david@fb.com> parents: 21016 diff changeset	977
24036 c7601086338a bundle2.unbundlepart: tell() implementation Eric Sumner <ericsumner@fb.com> parents: 24035 diff changeset	978 def tell(self):
c7601086338a bundle2.unbundlepart: tell() implementation Eric Sumner <ericsumner@fb.com> parents: 24035 diff changeset	979 return self._pos
c7601086338a bundle2.unbundlepart: tell() implementation Eric Sumner <ericsumner@fb.com> parents: 24035 diff changeset	980
24037 f0b498cfc5c8 bundle2.unbundlepart: implement seek() Eric Sumner <ericsumner@fb.com> parents: 24036 diff changeset	981 def seek(self, offset, whence=0):
f0b498cfc5c8 bundle2.unbundlepart: implement seek() Eric Sumner <ericsumner@fb.com> parents: 24036 diff changeset	982 if whence == 0:
f0b498cfc5c8 bundle2.unbundlepart: implement seek() Eric Sumner <ericsumner@fb.com> parents: 24036 diff changeset	983 newpos = offset
f0b498cfc5c8 bundle2.unbundlepart: implement seek() Eric Sumner <ericsumner@fb.com> parents: 24036 diff changeset	984 elif whence == 1:
f0b498cfc5c8 bundle2.unbundlepart: implement seek() Eric Sumner <ericsumner@fb.com> parents: 24036 diff changeset	985 newpos = self._pos + offset
f0b498cfc5c8 bundle2.unbundlepart: implement seek() Eric Sumner <ericsumner@fb.com> parents: 24036 diff changeset	986 elif whence == 2:
f0b498cfc5c8 bundle2.unbundlepart: implement seek() Eric Sumner <ericsumner@fb.com> parents: 24036 diff changeset	987 if not self.consumed:
f0b498cfc5c8 bundle2.unbundlepart: implement seek() Eric Sumner <ericsumner@fb.com> parents: 24036 diff changeset	988 self.read()
f0b498cfc5c8 bundle2.unbundlepart: implement seek() Eric Sumner <ericsumner@fb.com> parents: 24036 diff changeset	989 newpos = self._chunkindex[-1][0] - offset
f0b498cfc5c8 bundle2.unbundlepart: implement seek() Eric Sumner <ericsumner@fb.com> parents: 24036 diff changeset	990 else:
f0b498cfc5c8 bundle2.unbundlepart: implement seek() Eric Sumner <ericsumner@fb.com> parents: 24036 diff changeset	991 raise ValueError('Unknown whence value: %r' % (whence,))
f0b498cfc5c8 bundle2.unbundlepart: implement seek() Eric Sumner <ericsumner@fb.com> parents: 24036 diff changeset	992
f0b498cfc5c8 bundle2.unbundlepart: implement seek() Eric Sumner <ericsumner@fb.com> parents: 24036 diff changeset	993 if newpos > self._chunkindex[-1][0] and not self.consumed:
f0b498cfc5c8 bundle2.unbundlepart: implement seek() Eric Sumner <ericsumner@fb.com> parents: 24036 diff changeset	994 self.read()
f0b498cfc5c8 bundle2.unbundlepart: implement seek() Eric Sumner <ericsumner@fb.com> parents: 24036 diff changeset	995 if not 0 <= newpos <= self._chunkindex[-1][0]:
f0b498cfc5c8 bundle2.unbundlepart: implement seek() Eric Sumner <ericsumner@fb.com> parents: 24036 diff changeset	996 raise ValueError('Offset out of range')
f0b498cfc5c8 bundle2.unbundlepart: implement seek() Eric Sumner <ericsumner@fb.com> parents: 24036 diff changeset	997
f0b498cfc5c8 bundle2.unbundlepart: implement seek() Eric Sumner <ericsumner@fb.com> parents: 24036 diff changeset	998 if self._pos != newpos:
f0b498cfc5c8 bundle2.unbundlepart: implement seek() Eric Sumner <ericsumner@fb.com> parents: 24036 diff changeset	999 chunk, internaloffset = self._findchunk(newpos)
f0b498cfc5c8 bundle2.unbundlepart: implement seek() Eric Sumner <ericsumner@fb.com> parents: 24036 diff changeset	1000 self._payloadstream = util.chunkbuffer(self._payloadchunks(chunk))
f0b498cfc5c8 bundle2.unbundlepart: implement seek() Eric Sumner <ericsumner@fb.com> parents: 24036 diff changeset	1001 adjust = self.read(internaloffset)
f0b498cfc5c8 bundle2.unbundlepart: implement seek() Eric Sumner <ericsumner@fb.com> parents: 24036 diff changeset	1002 if len(adjust) != internaloffset:
f0b498cfc5c8 bundle2.unbundlepart: implement seek() Eric Sumner <ericsumner@fb.com> parents: 24036 diff changeset	1003 raise util.Abort(_('Seek failed\n'))
f0b498cfc5c8 bundle2.unbundlepart: implement seek() Eric Sumner <ericsumner@fb.com> parents: 24036 diff changeset	1004 self._pos = newpos
f0b498cfc5c8 bundle2.unbundlepart: implement seek() Eric Sumner <ericsumner@fb.com> parents: 24036 diff changeset	1005
24686 e0e28e910fa3 bundle2: rename format, parts and config to final names Pierre-Yves David <pierre-yves.david@fb.com> parents: 24648 diff changeset	1006 capabilities = {'HG20': (),
e0e28e910fa3 bundle2: rename format, parts and config to final names Pierre-Yves David <pierre-yves.david@fb.com> parents: 24648 diff changeset	1007 'listkeys': (),
e0e28e910fa3 bundle2: rename format, parts and config to final names Pierre-Yves David <pierre-yves.david@fb.com> parents: 24648 diff changeset	1008 'pushkey': (),
23029 149fc8a44184 bundle2: client side support for a part to import external bundles Mike Hommey <mh@glandium.org> parents: 23011 diff changeset	1009 'digests': tuple(sorted(util.DIGESTS.keys())),
24686 e0e28e910fa3 bundle2: rename format, parts and config to final names Pierre-Yves David <pierre-yves.david@fb.com> parents: 24648 diff changeset	1010 'remote-changegroup': ('http', 'https'),
22341 2d16b39601b5 obsmarker: move bundle2caps from the localrepo class to the bundle2 module Pierre-Yves David <pierre-yves.david@fb.com> parents: 22340 diff changeset	1011 }
2d16b39601b5 obsmarker: move bundle2caps from the localrepo class to the bundle2 module Pierre-Yves David <pierre-yves.david@fb.com> parents: 22340 diff changeset	1012
23439 743736fc7c41 bundle2-push: provide transaction to reply unbundler Eric Sumner <ericsumner@fb.com> parents: 23438 diff changeset	1013 def getrepocaps(repo, allowpushback=False):
22342 262c5cc126c1 bundle2: introduce a `getrepocaps` to retrieve the bundle2 caps of a repo Pierre-Yves David <pierre-yves.david@fb.com> parents: 22341 diff changeset	1014 """return the bundle2 capabilities for a given repo
262c5cc126c1 bundle2: introduce a `getrepocaps` to retrieve the bundle2 caps of a repo Pierre-Yves David <pierre-yves.david@fb.com> parents: 22341 diff changeset	1015
22343 76677a2c1cfd bundle2: advertise the obsmarker part in bundle2 capabilities Pierre-Yves David <pierre-yves.david@fb.com> parents: 22342 diff changeset	1016 Exists to allow extensions (like evolution) to mutate the capabilities.
22342 262c5cc126c1 bundle2: introduce a `getrepocaps` to retrieve the bundle2 caps of a repo Pierre-Yves David <pierre-yves.david@fb.com> parents: 22341 diff changeset	1017 """
22343 76677a2c1cfd bundle2: advertise the obsmarker part in bundle2 capabilities Pierre-Yves David <pierre-yves.david@fb.com> parents: 22342 diff changeset	1018 caps = capabilities.copy()
24686 e0e28e910fa3 bundle2: rename format, parts and config to final names Pierre-Yves David <pierre-yves.david@fb.com> parents: 24648 diff changeset	1019 caps['changegroup'] = tuple(sorted(changegroup.packermap.keys()))
22953 b1d694d3975e obsolete: add exchange option Durham Goode <durham@fb.com> parents: 22661 diff changeset	1020 if obsolete.isenabled(repo, obsolete.exchangeopt):
22343 76677a2c1cfd bundle2: advertise the obsmarker part in bundle2 capabilities Pierre-Yves David <pierre-yves.david@fb.com> parents: 22342 diff changeset	1021 supportedformat = tuple('V%i' % v for v in obsolete.formats)
24686 e0e28e910fa3 bundle2: rename format, parts and config to final names Pierre-Yves David <pierre-yves.david@fb.com> parents: 24648 diff changeset	1022 caps['obsmarkers'] = supportedformat
23439 743736fc7c41 bundle2-push: provide transaction to reply unbundler Eric Sumner <ericsumner@fb.com> parents: 23438 diff changeset	1023 if allowpushback:
24686 e0e28e910fa3 bundle2: rename format, parts and config to final names Pierre-Yves David <pierre-yves.david@fb.com> parents: 24648 diff changeset	1024 caps['pushback'] = ()
22343 76677a2c1cfd bundle2: advertise the obsmarker part in bundle2 capabilities Pierre-Yves David <pierre-yves.david@fb.com> parents: 22342 diff changeset	1025 return caps
22342 262c5cc126c1 bundle2: introduce a `getrepocaps` to retrieve the bundle2 caps of a repo Pierre-Yves David <pierre-yves.david@fb.com> parents: 22341 diff changeset	1026
21644 17755dd8c509 bundle2: introduce a bundle2caps function Pierre-Yves David <pierre-yves.david@fb.com> parents: 21628 diff changeset	1027 def bundle2caps(remote):
23139 e53f6b72a0e4 spelling: fixes from proofreading of spell checker issues Mads Kiilerich <madski@unity3d.com> parents: 23067 diff changeset	1028 """return the bundle capabilities of a peer as dict"""
24686 e0e28e910fa3 bundle2: rename format, parts and config to final names Pierre-Yves David <pierre-yves.david@fb.com> parents: 24648 diff changeset	1029 raw = remote.capable('bundle2')
21644 17755dd8c509 bundle2: introduce a bundle2caps function Pierre-Yves David <pierre-yves.david@fb.com> parents: 21628 diff changeset	1030 if not raw and raw != '':
17755dd8c509 bundle2: introduce a bundle2caps function Pierre-Yves David <pierre-yves.david@fb.com> parents: 21628 diff changeset	1031 return {}
24686 e0e28e910fa3 bundle2: rename format, parts and config to final names Pierre-Yves David <pierre-yves.david@fb.com> parents: 24648 diff changeset	1032 capsblob = urllib.unquote(remote.capable('bundle2'))
21644 17755dd8c509 bundle2: introduce a bundle2caps function Pierre-Yves David <pierre-yves.david@fb.com> parents: 21628 diff changeset	1033 return decodecaps(capsblob)
21014 a6246bba7b9e bundle2: add an unbundle part responsible from unbundling part Pierre-Yves David <pierre-yves.david@fb.com> parents: 21013 diff changeset	1034
22344 9829b7948100 bundle2: add a `obsmarkersversion` function to extract supported version Pierre-Yves David <pierre-yves.david@fb.com> parents: 22343 diff changeset	1035 def obsmarkersversion(caps):
9829b7948100 bundle2: add a `obsmarkersversion` function to extract supported version Pierre-Yves David <pierre-yves.david@fb.com> parents: 22343 diff changeset	1036 """extract the list of supported obsmarkers versions from a bundle2caps dict
9829b7948100 bundle2: add a `obsmarkersversion` function to extract supported version Pierre-Yves David <pierre-yves.david@fb.com> parents: 22343 diff changeset	1037 """
24686 e0e28e910fa3 bundle2: rename format, parts and config to final names Pierre-Yves David <pierre-yves.david@fb.com> parents: 24648 diff changeset	1038 obscaps = caps.get('obsmarkers', ())
22344 9829b7948100 bundle2: add a `obsmarkersversion` function to extract supported version Pierre-Yves David <pierre-yves.david@fb.com> parents: 22343 diff changeset	1039 return [int(c[1:]) for c in obscaps if c.startswith('V')]
9829b7948100 bundle2: add a `obsmarkersversion` function to extract supported version Pierre-Yves David <pierre-yves.david@fb.com> parents: 22343 diff changeset	1040
24686 e0e28e910fa3 bundle2: rename format, parts and config to final names Pierre-Yves David <pierre-yves.david@fb.com> parents: 24648 diff changeset	1041 @parthandler('changegroup', ('version',))
20998 93a3c5b58635 bundle2: use reply part to return result of addchangegroup Pierre-Yves David <pierre-yves.david@fb.com> parents: 20997 diff changeset	1042 def handlechangegroup(op, inpart):
20950 c7ceae0faf69 bundle2: first crude version of bundling changeset with bundle2 Pierre-Yves David <pierre-yves.david@fb.com> parents: 20949 diff changeset	1043 """apply a changegroup part on the repo
c7ceae0faf69 bundle2: first crude version of bundling changeset with bundle2 Pierre-Yves David <pierre-yves.david@fb.com> parents: 20949 diff changeset	1044
c7ceae0faf69 bundle2: first crude version of bundling changeset with bundle2 Pierre-Yves David <pierre-yves.david@fb.com> parents: 20949 diff changeset	1045 This is a very early implementation that will massive rework before being
c7ceae0faf69 bundle2: first crude version of bundling changeset with bundle2 Pierre-Yves David <pierre-yves.david@fb.com> parents: 20949 diff changeset	1046 inflicted to any end-user.
c7ceae0faf69 bundle2: first crude version of bundling changeset with bundle2 Pierre-Yves David <pierre-yves.david@fb.com> parents: 20949 diff changeset	1047 """
20952 b24ee5076b94 bundle2: make it possible have a global transaction for the unbundling Pierre-Yves David <pierre-yves.david@fb.com> parents: 20950 diff changeset	1048 # Make sure we trigger a transaction creation
b24ee5076b94 bundle2: make it possible have a global transaction for the unbundling Pierre-Yves David <pierre-yves.david@fb.com> parents: 20950 diff changeset	1049 #
b24ee5076b94 bundle2: make it possible have a global transaction for the unbundling Pierre-Yves David <pierre-yves.david@fb.com> parents: 20950 diff changeset	1050 # The addchangegroup function will get a transaction object by itself, but
b24ee5076b94 bundle2: make it possible have a global transaction for the unbundling Pierre-Yves David <pierre-yves.david@fb.com> parents: 20950 diff changeset	1051 # we need to make sure we trigger the creation of a transaction object used
b24ee5076b94 bundle2: make it possible have a global transaction for the unbundling Pierre-Yves David <pierre-yves.david@fb.com> parents: 20950 diff changeset	1052 # for the whole processing scope.
b24ee5076b94 bundle2: make it possible have a global transaction for the unbundling Pierre-Yves David <pierre-yves.david@fb.com> parents: 20950 diff changeset	1053 op.gettransaction()
23170 02e8f9b60052 bundle2: support a "version" argument in `changegroup` part Pierre-Yves David <pierre-yves.david@fb.com> parents: 23169 diff changeset	1054 unpackerversion = inpart.params.get('version', '01')
02e8f9b60052 bundle2: support a "version" argument in `changegroup` part Pierre-Yves David <pierre-yves.david@fb.com> parents: 23169 diff changeset	1055 # We should raise an appropriate exception here
02e8f9b60052 bundle2: support a "version" argument in `changegroup` part Pierre-Yves David <pierre-yves.david@fb.com> parents: 23169 diff changeset	1056 unpacker = changegroup.packermap[unpackerversion][1]
02e8f9b60052 bundle2: support a "version" argument in `changegroup` part Pierre-Yves David <pierre-yves.david@fb.com> parents: 23169 diff changeset	1057 cg = unpacker(inpart, 'UN')
23001 4df9b5e62f70 bundle2: add a comment about addchangegroup source and url Pierre-Yves David <pierre-yves.david@fb.com> parents: 22953 diff changeset	1058 # the source and url passed here are overwritten by the one contained in
4df9b5e62f70 bundle2: add a comment about addchangegroup source and url Pierre-Yves David <pierre-yves.david@fb.com> parents: 22953 diff changeset	1059 # the transaction.hookargs argument. So 'bundle2' is a placeholder
20950 c7ceae0faf69 bundle2: first crude version of bundling changeset with bundle2 Pierre-Yves David <pierre-yves.david@fb.com> parents: 20949 diff changeset	1060 ret = changegroup.addchangegroup(op.repo, cg, 'bundle2', 'bundle2')
c7ceae0faf69 bundle2: first crude version of bundling changeset with bundle2 Pierre-Yves David <pierre-yves.david@fb.com> parents: 20949 diff changeset	1061 op.records.add('changegroup', {'return': ret})
20998 93a3c5b58635 bundle2: use reply part to return result of addchangegroup Pierre-Yves David <pierre-yves.david@fb.com> parents: 20997 diff changeset	1062 if op.reply is not None:
23139 e53f6b72a0e4 spelling: fixes from proofreading of spell checker issues Mads Kiilerich <madski@unity3d.com> parents: 23067 diff changeset	1063 # This is definitely not the final form of this
20998 93a3c5b58635 bundle2: use reply part to return result of addchangegroup Pierre-Yves David <pierre-yves.david@fb.com> parents: 20997 diff changeset	1064 # return. But one need to start somewhere.
24686 e0e28e910fa3 bundle2: rename format, parts and config to final names Pierre-Yves David <pierre-yves.david@fb.com> parents: 24648 diff changeset	1065 part = op.reply.newpart('reply:changegroup', mandatory=False)
21606 e55888447958 bundle2: update part creators to ``addparam`` when relevant Pierre-Yves David <pierre-yves.david@fb.com> parents: 21605 diff changeset	1066 part.addparam('in-reply-to', str(inpart.id), mandatory=False)
e55888447958 bundle2: update part creators to ``addparam`` when relevant Pierre-Yves David <pierre-yves.david@fb.com> parents: 21605 diff changeset	1067 part.addparam('return', '%i' % ret, mandatory=False)
21019 3dc09f831a2e bundle2: lazy unbundle of part payload Pierre-Yves David <pierre-yves.david@fb.com> parents: 21016 diff changeset	1068 assert not inpart.read()
20950 c7ceae0faf69 bundle2: first crude version of bundling changeset with bundle2 Pierre-Yves David <pierre-yves.david@fb.com> parents: 20949 diff changeset	1069
23029 149fc8a44184 bundle2: client side support for a part to import external bundles Mike Hommey <mh@glandium.org> parents: 23011 diff changeset	1070 _remotechangegroupparams = tuple(['url', 'size', 'digests'] +
149fc8a44184 bundle2: client side support for a part to import external bundles Mike Hommey <mh@glandium.org> parents: 23011 diff changeset	1071 ['digest:%s' % k for k in util.DIGESTS.keys()])
24686 e0e28e910fa3 bundle2: rename format, parts and config to final names Pierre-Yves David <pierre-yves.david@fb.com> parents: 24648 diff changeset	1072 @parthandler('remote-changegroup', _remotechangegroupparams)
23029 149fc8a44184 bundle2: client side support for a part to import external bundles Mike Hommey <mh@glandium.org> parents: 23011 diff changeset	1073 def handleremotechangegroup(op, inpart):
149fc8a44184 bundle2: client side support for a part to import external bundles Mike Hommey <mh@glandium.org> parents: 23011 diff changeset	1074 """apply a bundle10 on the repo, given an url and validation information
149fc8a44184 bundle2: client side support for a part to import external bundles Mike Hommey <mh@glandium.org> parents: 23011 diff changeset	1075
149fc8a44184 bundle2: client side support for a part to import external bundles Mike Hommey <mh@glandium.org> parents: 23011 diff changeset	1076 All the information about the remote bundle to import are given as
149fc8a44184 bundle2: client side support for a part to import external bundles Mike Hommey <mh@glandium.org> parents: 23011 diff changeset	1077 parameters. The parameters include:
149fc8a44184 bundle2: client side support for a part to import external bundles Mike Hommey <mh@glandium.org> parents: 23011 diff changeset	1078 - url: the url to the bundle10.
149fc8a44184 bundle2: client side support for a part to import external bundles Mike Hommey <mh@glandium.org> parents: 23011 diff changeset	1079 - size: the bundle10 file size. It is used to validate what was
149fc8a44184 bundle2: client side support for a part to import external bundles Mike Hommey <mh@glandium.org> parents: 23011 diff changeset	1080 retrieved by the client matches the server knowledge about the bundle.
149fc8a44184 bundle2: client side support for a part to import external bundles Mike Hommey <mh@glandium.org> parents: 23011 diff changeset	1081 - digests: a space separated list of the digest types provided as
149fc8a44184 bundle2: client side support for a part to import external bundles Mike Hommey <mh@glandium.org> parents: 23011 diff changeset	1082 parameters.
149fc8a44184 bundle2: client side support for a part to import external bundles Mike Hommey <mh@glandium.org> parents: 23011 diff changeset	1083 - digest:<digest-type>: the hexadecimal representation of the digest with
149fc8a44184 bundle2: client side support for a part to import external bundles Mike Hommey <mh@glandium.org> parents: 23011 diff changeset	1084 that name. Like the size, it is used to validate what was retrieved by
149fc8a44184 bundle2: client side support for a part to import external bundles Mike Hommey <mh@glandium.org> parents: 23011 diff changeset	1085 the client matches what the server knows about the bundle.
149fc8a44184 bundle2: client side support for a part to import external bundles Mike Hommey <mh@glandium.org> parents: 23011 diff changeset	1086
149fc8a44184 bundle2: client side support for a part to import external bundles Mike Hommey <mh@glandium.org> parents: 23011 diff changeset	1087 When multiple digest types are given, all of them are checked.
149fc8a44184 bundle2: client side support for a part to import external bundles Mike Hommey <mh@glandium.org> parents: 23011 diff changeset	1088 """
149fc8a44184 bundle2: client side support for a part to import external bundles Mike Hommey <mh@glandium.org> parents: 23011 diff changeset	1089 try:
149fc8a44184 bundle2: client side support for a part to import external bundles Mike Hommey <mh@glandium.org> parents: 23011 diff changeset	1090 raw_url = inpart.params['url']
149fc8a44184 bundle2: client side support for a part to import external bundles Mike Hommey <mh@glandium.org> parents: 23011 diff changeset	1091 except KeyError:
149fc8a44184 bundle2: client side support for a part to import external bundles Mike Hommey <mh@glandium.org> parents: 23011 diff changeset	1092 raise util.Abort(_('remote-changegroup: missing "%s" param') % 'url')
149fc8a44184 bundle2: client side support for a part to import external bundles Mike Hommey <mh@glandium.org> parents: 23011 diff changeset	1093 parsed_url = util.url(raw_url)
24686 e0e28e910fa3 bundle2: rename format, parts and config to final names Pierre-Yves David <pierre-yves.david@fb.com> parents: 24648 diff changeset	1094 if parsed_url.scheme not in capabilities['remote-changegroup']:
23029 149fc8a44184 bundle2: client side support for a part to import external bundles Mike Hommey <mh@glandium.org> parents: 23011 diff changeset	1095 raise util.Abort(_('remote-changegroup does not support %s urls') %
149fc8a44184 bundle2: client side support for a part to import external bundles Mike Hommey <mh@glandium.org> parents: 23011 diff changeset	1096 parsed_url.scheme)
149fc8a44184 bundle2: client side support for a part to import external bundles Mike Hommey <mh@glandium.org> parents: 23011 diff changeset	1097
149fc8a44184 bundle2: client side support for a part to import external bundles Mike Hommey <mh@glandium.org> parents: 23011 diff changeset	1098 try:
149fc8a44184 bundle2: client side support for a part to import external bundles Mike Hommey <mh@glandium.org> parents: 23011 diff changeset	1099 size = int(inpart.params['size'])
149fc8a44184 bundle2: client side support for a part to import external bundles Mike Hommey <mh@glandium.org> parents: 23011 diff changeset	1100 except ValueError:
149fc8a44184 bundle2: client side support for a part to import external bundles Mike Hommey <mh@glandium.org> parents: 23011 diff changeset	1101 raise util.Abort(_('remote-changegroup: invalid value for param "%s"')
149fc8a44184 bundle2: client side support for a part to import external bundles Mike Hommey <mh@glandium.org> parents: 23011 diff changeset	1102 % 'size')
149fc8a44184 bundle2: client side support for a part to import external bundles Mike Hommey <mh@glandium.org> parents: 23011 diff changeset	1103 except KeyError:
149fc8a44184 bundle2: client side support for a part to import external bundles Mike Hommey <mh@glandium.org> parents: 23011 diff changeset	1104 raise util.Abort(_('remote-changegroup: missing "%s" param') % 'size')
149fc8a44184 bundle2: client side support for a part to import external bundles Mike Hommey <mh@glandium.org> parents: 23011 diff changeset	1105
149fc8a44184 bundle2: client side support for a part to import external bundles Mike Hommey <mh@glandium.org> parents: 23011 diff changeset	1106 digests = {}
149fc8a44184 bundle2: client side support for a part to import external bundles Mike Hommey <mh@glandium.org> parents: 23011 diff changeset	1107 for typ in inpart.params.get('digests', '').split():
149fc8a44184 bundle2: client side support for a part to import external bundles Mike Hommey <mh@glandium.org> parents: 23011 diff changeset	1108 param = 'digest:%s' % typ
149fc8a44184 bundle2: client side support for a part to import external bundles Mike Hommey <mh@glandium.org> parents: 23011 diff changeset	1109 try:
149fc8a44184 bundle2: client side support for a part to import external bundles Mike Hommey <mh@glandium.org> parents: 23011 diff changeset	1110 value = inpart.params[param]
149fc8a44184 bundle2: client side support for a part to import external bundles Mike Hommey <mh@glandium.org> parents: 23011 diff changeset	1111 except KeyError:
149fc8a44184 bundle2: client side support for a part to import external bundles Mike Hommey <mh@glandium.org> parents: 23011 diff changeset	1112 raise util.Abort(_('remote-changegroup: missing "%s" param') %
149fc8a44184 bundle2: client side support for a part to import external bundles Mike Hommey <mh@glandium.org> parents: 23011 diff changeset	1113 param)
149fc8a44184 bundle2: client side support for a part to import external bundles Mike Hommey <mh@glandium.org> parents: 23011 diff changeset	1114 digests[typ] = value
149fc8a44184 bundle2: client side support for a part to import external bundles Mike Hommey <mh@glandium.org> parents: 23011 diff changeset	1115
149fc8a44184 bundle2: client side support for a part to import external bundles Mike Hommey <mh@glandium.org> parents: 23011 diff changeset	1116 real_part = util.digestchecker(url.open(op.ui, raw_url), size, digests)
149fc8a44184 bundle2: client side support for a part to import external bundles Mike Hommey <mh@glandium.org> parents: 23011 diff changeset	1117
149fc8a44184 bundle2: client side support for a part to import external bundles Mike Hommey <mh@glandium.org> parents: 23011 diff changeset	1118 # Make sure we trigger a transaction creation
149fc8a44184 bundle2: client side support for a part to import external bundles Mike Hommey <mh@glandium.org> parents: 23011 diff changeset	1119 #
149fc8a44184 bundle2: client side support for a part to import external bundles Mike Hommey <mh@glandium.org> parents: 23011 diff changeset	1120 # The addchangegroup function will get a transaction object by itself, but
149fc8a44184 bundle2: client side support for a part to import external bundles Mike Hommey <mh@glandium.org> parents: 23011 diff changeset	1121 # we need to make sure we trigger the creation of a transaction object used
149fc8a44184 bundle2: client side support for a part to import external bundles Mike Hommey <mh@glandium.org> parents: 23011 diff changeset	1122 # for the whole processing scope.
149fc8a44184 bundle2: client side support for a part to import external bundles Mike Hommey <mh@glandium.org> parents: 23011 diff changeset	1123 op.gettransaction()
149fc8a44184 bundle2: client side support for a part to import external bundles Mike Hommey <mh@glandium.org> parents: 23011 diff changeset	1124 import exchange
149fc8a44184 bundle2: client side support for a part to import external bundles Mike Hommey <mh@glandium.org> parents: 23011 diff changeset	1125 cg = exchange.readbundle(op.repo.ui, real_part, raw_url)
149fc8a44184 bundle2: client side support for a part to import external bundles Mike Hommey <mh@glandium.org> parents: 23011 diff changeset	1126 if not isinstance(cg, changegroup.cg1unpacker):
149fc8a44184 bundle2: client side support for a part to import external bundles Mike Hommey <mh@glandium.org> parents: 23011 diff changeset	1127 raise util.Abort(_('%s: not a bundle version 1.0') %
149fc8a44184 bundle2: client side support for a part to import external bundles Mike Hommey <mh@glandium.org> parents: 23011 diff changeset	1128 util.hidepassword(raw_url))
149fc8a44184 bundle2: client side support for a part to import external bundles Mike Hommey <mh@glandium.org> parents: 23011 diff changeset	1129 ret = changegroup.addchangegroup(op.repo, cg, 'bundle2', 'bundle2')
149fc8a44184 bundle2: client side support for a part to import external bundles Mike Hommey <mh@glandium.org> parents: 23011 diff changeset	1130 op.records.add('changegroup', {'return': ret})
149fc8a44184 bundle2: client side support for a part to import external bundles Mike Hommey <mh@glandium.org> parents: 23011 diff changeset	1131 if op.reply is not None:
23139 e53f6b72a0e4 spelling: fixes from proofreading of spell checker issues Mads Kiilerich <madski@unity3d.com> parents: 23067 diff changeset	1132 # This is definitely not the final form of this
23029 149fc8a44184 bundle2: client side support for a part to import external bundles Mike Hommey <mh@glandium.org> parents: 23011 diff changeset	1133 # return. But one need to start somewhere.
24686 e0e28e910fa3 bundle2: rename format, parts and config to final names Pierre-Yves David <pierre-yves.david@fb.com> parents: 24648 diff changeset	1134 part = op.reply.newpart('reply:changegroup')
23029 149fc8a44184 bundle2: client side support for a part to import external bundles Mike Hommey <mh@glandium.org> parents: 23011 diff changeset	1135 part.addparam('in-reply-to', str(inpart.id), mandatory=False)
149fc8a44184 bundle2: client side support for a part to import external bundles Mike Hommey <mh@glandium.org> parents: 23011 diff changeset	1136 part.addparam('return', '%i' % ret, mandatory=False)
149fc8a44184 bundle2: client side support for a part to import external bundles Mike Hommey <mh@glandium.org> parents: 23011 diff changeset	1137 try:
149fc8a44184 bundle2: client side support for a part to import external bundles Mike Hommey <mh@glandium.org> parents: 23011 diff changeset	1138 real_part.validate()
149fc8a44184 bundle2: client side support for a part to import external bundles Mike Hommey <mh@glandium.org> parents: 23011 diff changeset	1139 except util.Abort, e:
149fc8a44184 bundle2: client side support for a part to import external bundles Mike Hommey <mh@glandium.org> parents: 23011 diff changeset	1140 raise util.Abort(_('bundle at %s is corrupted:\n%s') %
149fc8a44184 bundle2: client side support for a part to import external bundles Mike Hommey <mh@glandium.org> parents: 23011 diff changeset	1141 (util.hidepassword(raw_url), str(e)))
149fc8a44184 bundle2: client side support for a part to import external bundles Mike Hommey <mh@glandium.org> parents: 23011 diff changeset	1142 assert not inpart.read()
149fc8a44184 bundle2: client side support for a part to import external bundles Mike Hommey <mh@glandium.org> parents: 23011 diff changeset	1143
24686 e0e28e910fa3 bundle2: rename format, parts and config to final names Pierre-Yves David <pierre-yves.david@fb.com> parents: 24648 diff changeset	1144 @parthandler('reply:changegroup', ('return', 'in-reply-to'))
22548 8a1ae133770a bundle2: rename functions that have the same name Mike Hommey <mh@glandium.org> parents: 22390 diff changeset	1145 def handlereplychangegroup(op, inpart):
21611 71b7b3f79a3c bundle2: use the new ``part.params`` dictionary Pierre-Yves David <pierre-yves.david@fb.com> parents: 21610 diff changeset	1146 ret = int(inpart.params['return'])
71b7b3f79a3c bundle2: use the new ``part.params`` dictionary Pierre-Yves David <pierre-yves.david@fb.com> parents: 21610 diff changeset	1147 replyto = int(inpart.params['in-reply-to'])
71b7b3f79a3c bundle2: use the new ``part.params`` dictionary Pierre-Yves David <pierre-yves.david@fb.com> parents: 21610 diff changeset	1148 op.records.add('changegroup', {'return': ret}, replyto)
20950 c7ceae0faf69 bundle2: first crude version of bundling changeset with bundle2 Pierre-Yves David <pierre-yves.david@fb.com> parents: 20949 diff changeset	1149
24686 e0e28e910fa3 bundle2: rename format, parts and config to final names Pierre-Yves David <pierre-yves.david@fb.com> parents: 24648 diff changeset	1150 @parthandler('check:heads')
22548 8a1ae133770a bundle2: rename functions that have the same name Mike Hommey <mh@glandium.org> parents: 22390 diff changeset	1151 def handlecheckheads(op, inpart):
21060 0bea9db7543b bundle2: add a "check:heads" handler Pierre-Yves David <pierre-yves.david@fb.com> parents: 21024 diff changeset	1152 """check that head of the repo did not change
0bea9db7543b bundle2: add a "check:heads" handler Pierre-Yves David <pierre-yves.david@fb.com> parents: 21024 diff changeset	1153
0bea9db7543b bundle2: add a "check:heads" handler Pierre-Yves David <pierre-yves.david@fb.com> parents: 21024 diff changeset	1154 This is used to detect a push race when using unbundle.
0bea9db7543b bundle2: add a "check:heads" handler Pierre-Yves David <pierre-yves.david@fb.com> parents: 21024 diff changeset	1155 This replaces the "heads" argument of unbundle."""
0bea9db7543b bundle2: add a "check:heads" handler Pierre-Yves David <pierre-yves.david@fb.com> parents: 21024 diff changeset	1156 h = inpart.read(20)
0bea9db7543b bundle2: add a "check:heads" handler Pierre-Yves David <pierre-yves.david@fb.com> parents: 21024 diff changeset	1157 heads = []
0bea9db7543b bundle2: add a "check:heads" handler Pierre-Yves David <pierre-yves.david@fb.com> parents: 21024 diff changeset	1158 while len(h) == 20:
0bea9db7543b bundle2: add a "check:heads" handler Pierre-Yves David <pierre-yves.david@fb.com> parents: 21024 diff changeset	1159 heads.append(h)
0bea9db7543b bundle2: add a "check:heads" handler Pierre-Yves David <pierre-yves.david@fb.com> parents: 21024 diff changeset	1160 h = inpart.read(20)
0bea9db7543b bundle2: add a "check:heads" handler Pierre-Yves David <pierre-yves.david@fb.com> parents: 21024 diff changeset	1161 assert not h
0bea9db7543b bundle2: add a "check:heads" handler Pierre-Yves David <pierre-yves.david@fb.com> parents: 21024 diff changeset	1162 if heads != op.repo.heads():
21185 5b3717e1a3ea bundle2: add an error message to push race error Pierre-Yves David <pierre-yves.david@fb.com> parents: 21184 diff changeset	1163 raise error.PushRaced('repository changed while pushing - '
5b3717e1a3ea bundle2: add an error message to push race error Pierre-Yves David <pierre-yves.david@fb.com> parents: 21184 diff changeset	1164 'please try again')
21130 1ff06386217f bundle2: introduce `replycaps` part for on-demand reply Pierre-Yves David <pierre-yves.david@fb.com> parents: 21129 diff changeset	1165
24686 e0e28e910fa3 bundle2: rename format, parts and config to final names Pierre-Yves David <pierre-yves.david@fb.com> parents: 24648 diff changeset	1166 @parthandler('output')
21131 b7435117d951 bundle2: capture remote stdout while unbundling Pierre-Yves David <pierre-yves.david@fb.com> parents: 21130 diff changeset	1167 def handleoutput(op, inpart):
b7435117d951 bundle2: capture remote stdout while unbundling Pierre-Yves David <pierre-yves.david@fb.com> parents: 21130 diff changeset	1168 """forward output captured on the server to the client"""
b7435117d951 bundle2: capture remote stdout while unbundling Pierre-Yves David <pierre-yves.david@fb.com> parents: 21130 diff changeset	1169 for line in inpart.read().splitlines():
b7435117d951 bundle2: capture remote stdout while unbundling Pierre-Yves David <pierre-yves.david@fb.com> parents: 21130 diff changeset	1170 op.ui.write(('remote: %s\n' % line))
b7435117d951 bundle2: capture remote stdout while unbundling Pierre-Yves David <pierre-yves.david@fb.com> parents: 21130 diff changeset	1171
24686 e0e28e910fa3 bundle2: rename format, parts and config to final names Pierre-Yves David <pierre-yves.david@fb.com> parents: 24648 diff changeset	1172 @parthandler('replycaps')
21130 1ff06386217f bundle2: introduce `replycaps` part for on-demand reply Pierre-Yves David <pierre-yves.david@fb.com> parents: 21129 diff changeset	1173 def handlereplycaps(op, inpart):
1ff06386217f bundle2: introduce `replycaps` part for on-demand reply Pierre-Yves David <pierre-yves.david@fb.com> parents: 21129 diff changeset	1174 """Notify that a reply bundle should be created
1ff06386217f bundle2: introduce `replycaps` part for on-demand reply Pierre-Yves David <pierre-yves.david@fb.com> parents: 21129 diff changeset	1175
21138 f469879d27ec bundle2: extract capabilities decoding Pierre-Yves David <pierre-yves.david@fb.com> parents: 21137 diff changeset	1176 The payload contains the capabilities information for the reply"""
f469879d27ec bundle2: extract capabilities decoding Pierre-Yves David <pierre-yves.david@fb.com> parents: 21137 diff changeset	1177 caps = decodecaps(inpart.read())
21130 1ff06386217f bundle2: introduce `replycaps` part for on-demand reply Pierre-Yves David <pierre-yves.david@fb.com> parents: 21129 diff changeset	1178 if op.reply is None:
21135 98fbf3adfd83 bundle2: add capabilities support in `replycaps` part Pierre-Yves David <pierre-yves.david@fb.com> parents: 21134 diff changeset	1179 op.reply = bundle20(op.ui, caps)
21131 b7435117d951 bundle2: capture remote stdout while unbundling Pierre-Yves David <pierre-yves.david@fb.com> parents: 21130 diff changeset	1180
24686 e0e28e910fa3 bundle2: rename format, parts and config to final names Pierre-Yves David <pierre-yves.david@fb.com> parents: 24648 diff changeset	1181 @parthandler('error:abort', ('message', 'hint'))
24741 bb67e52362d6 bundle2: fix names for error part handler Pierre-Yves David <pierre-yves.david@fb.com> parents: 24733 diff changeset	1182 def handleerrorabort(op, inpart):
21177 952af771bc17 bundle2: gracefully handle abort during unbundle Pierre-Yves David <pierre-yves.david@fb.com> parents: 21176 diff changeset	1183 """Used to transmit abort error over the wire"""
21611 71b7b3f79a3c bundle2: use the new ``part.params`` dictionary Pierre-Yves David <pierre-yves.david@fb.com> parents: 21610 diff changeset	1184 raise util.Abort(inpart.params['message'], hint=inpart.params.get('hint'))
21183 4345274adc4b bundle2: gracefully handle UnknownPartError during unbundle Pierre-Yves David <pierre-yves.david@fb.com> parents: 21179 diff changeset	1185
24686 e0e28e910fa3 bundle2: rename format, parts and config to final names Pierre-Yves David <pierre-yves.david@fb.com> parents: 24648 diff changeset	1186 @parthandler('error:unsupportedcontent', ('parttype', 'params'))
24741 bb67e52362d6 bundle2: fix names for error part handler Pierre-Yves David <pierre-yves.david@fb.com> parents: 24733 diff changeset	1187 def handleerrorunsupportedcontent(op, inpart):
21619 292331e906d7 bundle2: rename b2x:error:unknownpart to b2x:error:unsupportedcontent Pierre-Yves David <pierre-yves.david@fb.com> parents: 21618 diff changeset	1188 """Used to transmit unknown content error over the wire"""
21622 457492741007 bundle2: support transmission of params error over the wire Pierre-Yves David <pierre-yves.david@fb.com> parents: 21620 diff changeset	1189 kwargs = {}
21627 3e8bcc90f07c bundle2: support None parttype in BundleValueError Pierre-Yves David <pierre-yves.david@fb.com> parents: 21626 diff changeset	1190 parttype = inpart.params.get('parttype')
3e8bcc90f07c bundle2: support None parttype in BundleValueError Pierre-Yves David <pierre-yves.david@fb.com> parents: 21626 diff changeset	1191 if parttype is not None:
3e8bcc90f07c bundle2: support None parttype in BundleValueError Pierre-Yves David <pierre-yves.david@fb.com> parents: 21626 diff changeset	1192 kwargs['parttype'] = parttype
21622 457492741007 bundle2: support transmission of params error over the wire Pierre-Yves David <pierre-yves.david@fb.com> parents: 21620 diff changeset	1193 params = inpart.params.get('params')
457492741007 bundle2: support transmission of params error over the wire Pierre-Yves David <pierre-yves.david@fb.com> parents: 21620 diff changeset	1194 if params is not None:
457492741007 bundle2: support transmission of params error over the wire Pierre-Yves David <pierre-yves.david@fb.com> parents: 21620 diff changeset	1195 kwargs['params'] = params.split('\0')
457492741007 bundle2: support transmission of params error over the wire Pierre-Yves David <pierre-yves.david@fb.com> parents: 21620 diff changeset	1196
23010 73f394f4affc bundle2: add an UnsupportedPartError Pierre-Yves David <pierre-yves.david@fb.com> parents: 23009 diff changeset	1197 raise error.UnsupportedPartError(**kwargs)
21186 9f3652e851f8 bundle2: gracefully handle PushRaced error during unbundle Pierre-Yves David <pierre-yves.david@fb.com> parents: 21185 diff changeset	1198
24686 e0e28e910fa3 bundle2: rename format, parts and config to final names Pierre-Yves David <pierre-yves.david@fb.com> parents: 24648 diff changeset	1199 @parthandler('error:pushraced', ('message',))
24741 bb67e52362d6 bundle2: fix names for error part handler Pierre-Yves David <pierre-yves.david@fb.com> parents: 24733 diff changeset	1200 def handleerrorpushraced(op, inpart):
21186 9f3652e851f8 bundle2: gracefully handle PushRaced error during unbundle Pierre-Yves David <pierre-yves.david@fb.com> parents: 21185 diff changeset	1201 """Used to transmit push race error over the wire"""
21611 71b7b3f79a3c bundle2: use the new ``part.params`` dictionary Pierre-Yves David <pierre-yves.david@fb.com> parents: 21610 diff changeset	1202 raise error.ResponseError(_('push failed:'), inpart.params['message'])
21655 35095f332846 bundle: introduce a listkey handler Pierre-Yves David <pierre-yves.david@fb.com> parents: 21644 diff changeset	1203
24686 e0e28e910fa3 bundle2: rename format, parts and config to final names Pierre-Yves David <pierre-yves.david@fb.com> parents: 24648 diff changeset	1204 @parthandler('listkeys', ('namespace',))
21655 35095f332846 bundle: introduce a listkey handler Pierre-Yves David <pierre-yves.david@fb.com> parents: 21644 diff changeset	1205 def handlelistkeys(op, inpart):
35095f332846 bundle: introduce a listkey handler Pierre-Yves David <pierre-yves.david@fb.com> parents: 21644 diff changeset	1206 """retrieve pushkey namespace content stored in a bundle2"""
35095f332846 bundle: introduce a listkey handler Pierre-Yves David <pierre-yves.david@fb.com> parents: 21644 diff changeset	1207 namespace = inpart.params['namespace']
35095f332846 bundle: introduce a listkey handler Pierre-Yves David <pierre-yves.david@fb.com> parents: 21644 diff changeset	1208 r = pushkey.decodekeys(inpart.read())
35095f332846 bundle: introduce a listkey handler Pierre-Yves David <pierre-yves.david@fb.com> parents: 21644 diff changeset	1209 op.records.add('listkeys', (namespace, r))
21660 e87d2a12d41b bundle2: add ``pushkey`` support Pierre-Yves David <pierre-yves.david@fb.com> parents: 21655 diff changeset	1210
24686 e0e28e910fa3 bundle2: rename format, parts and config to final names Pierre-Yves David <pierre-yves.david@fb.com> parents: 24648 diff changeset	1211 @parthandler('pushkey', ('namespace', 'key', 'old', 'new'))
21660 e87d2a12d41b bundle2: add ``pushkey`` support Pierre-Yves David <pierre-yves.david@fb.com> parents: 21655 diff changeset	1212 def handlepushkey(op, inpart):
e87d2a12d41b bundle2: add ``pushkey`` support Pierre-Yves David <pierre-yves.david@fb.com> parents: 21655 diff changeset	1213 """process a pushkey request"""
e87d2a12d41b bundle2: add ``pushkey`` support Pierre-Yves David <pierre-yves.david@fb.com> parents: 21655 diff changeset	1214 dec = pushkey.decode
e87d2a12d41b bundle2: add ``pushkey`` support Pierre-Yves David <pierre-yves.david@fb.com> parents: 21655 diff changeset	1215 namespace = dec(inpart.params['namespace'])
e87d2a12d41b bundle2: add ``pushkey`` support Pierre-Yves David <pierre-yves.david@fb.com> parents: 21655 diff changeset	1216 key = dec(inpart.params['key'])
e87d2a12d41b bundle2: add ``pushkey`` support Pierre-Yves David <pierre-yves.david@fb.com> parents: 21655 diff changeset	1217 old = dec(inpart.params['old'])
e87d2a12d41b bundle2: add ``pushkey`` support Pierre-Yves David <pierre-yves.david@fb.com> parents: 21655 diff changeset	1218 new = dec(inpart.params['new'])
e87d2a12d41b bundle2: add ``pushkey`` support Pierre-Yves David <pierre-yves.david@fb.com> parents: 21655 diff changeset	1219 ret = op.repo.pushkey(namespace, key, old, new)
e87d2a12d41b bundle2: add ``pushkey`` support Pierre-Yves David <pierre-yves.david@fb.com> parents: 21655 diff changeset	1220 record = {'namespace': namespace,
e87d2a12d41b bundle2: add ``pushkey`` support Pierre-Yves David <pierre-yves.david@fb.com> parents: 21655 diff changeset	1221 'key': key,
e87d2a12d41b bundle2: add ``pushkey`` support Pierre-Yves David <pierre-yves.david@fb.com> parents: 21655 diff changeset	1222 'old': old,
e87d2a12d41b bundle2: add ``pushkey`` support Pierre-Yves David <pierre-yves.david@fb.com> parents: 21655 diff changeset	1223 'new': new}
e87d2a12d41b bundle2: add ``pushkey`` support Pierre-Yves David <pierre-yves.david@fb.com> parents: 21655 diff changeset	1224 op.records.add('pushkey', record)
e87d2a12d41b bundle2: add ``pushkey`` support Pierre-Yves David <pierre-yves.david@fb.com> parents: 21655 diff changeset	1225 if op.reply is not None:
24686 e0e28e910fa3 bundle2: rename format, parts and config to final names Pierre-Yves David <pierre-yves.david@fb.com> parents: 24648 diff changeset	1226 rpart = op.reply.newpart('reply:pushkey')
21660 e87d2a12d41b bundle2: add ``pushkey`` support Pierre-Yves David <pierre-yves.david@fb.com> parents: 21655 diff changeset	1227 rpart.addparam('in-reply-to', str(inpart.id), mandatory=False)
e87d2a12d41b bundle2: add ``pushkey`` support Pierre-Yves David <pierre-yves.david@fb.com> parents: 21655 diff changeset	1228 rpart.addparam('return', '%i' % ret, mandatory=False)
e87d2a12d41b bundle2: add ``pushkey`` support Pierre-Yves David <pierre-yves.david@fb.com> parents: 21655 diff changeset	1229
24686 e0e28e910fa3 bundle2: rename format, parts and config to final names Pierre-Yves David <pierre-yves.david@fb.com> parents: 24648 diff changeset	1230 @parthandler('reply:pushkey', ('return', 'in-reply-to'))
21660 e87d2a12d41b bundle2: add ``pushkey`` support Pierre-Yves David <pierre-yves.david@fb.com> parents: 21655 diff changeset	1231 def handlepushkeyreply(op, inpart):
e87d2a12d41b bundle2: add ``pushkey`` support Pierre-Yves David <pierre-yves.david@fb.com> parents: 21655 diff changeset	1232 """retrieve the result of a pushkey request"""
e87d2a12d41b bundle2: add ``pushkey`` support Pierre-Yves David <pierre-yves.david@fb.com> parents: 21655 diff changeset	1233 ret = int(inpart.params['return'])
e87d2a12d41b bundle2: add ``pushkey`` support Pierre-Yves David <pierre-yves.david@fb.com> parents: 21655 diff changeset	1234 partid = int(inpart.params['in-reply-to'])
e87d2a12d41b bundle2: add ``pushkey`` support Pierre-Yves David <pierre-yves.david@fb.com> parents: 21655 diff changeset	1235 op.records.add('pushkey', {'return': ret}, partid)
22336 60786c8a2f70 bundle2: add an obsmarkers part handler Pierre-Yves David <pierre-yves.david@fb.com> parents: 21900 diff changeset	1236
24686 e0e28e910fa3 bundle2: rename format, parts and config to final names Pierre-Yves David <pierre-yves.david@fb.com> parents: 24648 diff changeset	1237 @parthandler('obsmarkers')
22336 60786c8a2f70 bundle2: add an obsmarkers part handler Pierre-Yves David <pierre-yves.david@fb.com> parents: 21900 diff changeset	1238 def handleobsmarker(op, inpart):
60786c8a2f70 bundle2: add an obsmarkers part handler Pierre-Yves David <pierre-yves.david@fb.com> parents: 21900 diff changeset	1239 """add a stream of obsmarkers to the repo"""
60786c8a2f70 bundle2: add an obsmarkers part handler Pierre-Yves David <pierre-yves.david@fb.com> parents: 21900 diff changeset	1240 tr = op.gettransaction()
24733 c00e4338fa4b obsolete: experimental flag to get debug about obsmarkers exchange Pierre-Yves David <pierre-yves.david@fb.com> parents: 24686 diff changeset	1241 markerdata = inpart.read()
c00e4338fa4b obsolete: experimental flag to get debug about obsmarkers exchange Pierre-Yves David <pierre-yves.david@fb.com> parents: 24686 diff changeset	1242 if op.ui.config('experimental', 'obsmarkers-exchange-debug', False):
c00e4338fa4b obsolete: experimental flag to get debug about obsmarkers exchange Pierre-Yves David <pierre-yves.david@fb.com> parents: 24686 diff changeset	1243 op.ui.write(('obsmarker-exchange: %i bytes received\n')
c00e4338fa4b obsolete: experimental flag to get debug about obsmarkers exchange Pierre-Yves David <pierre-yves.david@fb.com> parents: 24686 diff changeset	1244 % len(markerdata))
c00e4338fa4b obsolete: experimental flag to get debug about obsmarkers exchange Pierre-Yves David <pierre-yves.david@fb.com> parents: 24686 diff changeset	1245 new = op.repo.obsstore.mergemarkers(tr, markerdata)
22337 c380fe290290 obsmarker: write a message with the number of markers added through bundle2 Pierre-Yves David <pierre-yves.david@fb.com> parents: 22336 diff changeset	1246 if new:
c380fe290290 obsmarker: write a message with the number of markers added through bundle2 Pierre-Yves David <pierre-yves.david@fb.com> parents: 22336 diff changeset	1247 op.repo.ui.status(_('%i new obsolescence markers\n') % new)
22338 8fcd56095d3b obssmarker: add a bundle2 record with the number of markers added Pierre-Yves David <pierre-yves.david@fb.com> parents: 22337 diff changeset	1248 op.records.add('obsmarkers', {'new': new})
22340 394a17de6a2d obsmarker: produce a reply part for markers received through bundle2 Pierre-Yves David <pierre-yves.david@fb.com> parents: 22338 diff changeset	1249 if op.reply is not None:
24686 e0e28e910fa3 bundle2: rename format, parts and config to final names Pierre-Yves David <pierre-yves.david@fb.com> parents: 24648 diff changeset	1250 rpart = op.reply.newpart('reply:obsmarkers')
22340 394a17de6a2d obsmarker: produce a reply part for markers received through bundle2 Pierre-Yves David <pierre-yves.david@fb.com> parents: 22338 diff changeset	1251 rpart.addparam('in-reply-to', str(inpart.id), mandatory=False)
394a17de6a2d obsmarker: produce a reply part for markers received through bundle2 Pierre-Yves David <pierre-yves.david@fb.com> parents: 22338 diff changeset	1252 rpart.addparam('new', '%i' % new, mandatory=False)
394a17de6a2d obsmarker: produce a reply part for markers received through bundle2 Pierre-Yves David <pierre-yves.david@fb.com> parents: 22338 diff changeset	1253
394a17de6a2d obsmarker: produce a reply part for markers received through bundle2 Pierre-Yves David <pierre-yves.david@fb.com> parents: 22338 diff changeset	1254
24686 e0e28e910fa3 bundle2: rename format, parts and config to final names Pierre-Yves David <pierre-yves.david@fb.com> parents: 24648 diff changeset	1255 @parthandler('reply:obsmarkers', ('new', 'in-reply-to'))
22340 394a17de6a2d obsmarker: produce a reply part for markers received through bundle2 Pierre-Yves David <pierre-yves.david@fb.com> parents: 22338 diff changeset	1256 def handlepushkeyreply(op, inpart):
394a17de6a2d obsmarker: produce a reply part for markers received through bundle2 Pierre-Yves David <pierre-yves.david@fb.com> parents: 22338 diff changeset	1257 """retrieve the result of a pushkey request"""
394a17de6a2d obsmarker: produce a reply part for markers received through bundle2 Pierre-Yves David <pierre-yves.david@fb.com> parents: 22338 diff changeset	1258 ret = int(inpart.params['new'])
394a17de6a2d obsmarker: produce a reply part for markers received through bundle2 Pierre-Yves David <pierre-yves.david@fb.com> parents: 22338 diff changeset	1259 partid = int(inpart.params['in-reply-to'])
394a17de6a2d obsmarker: produce a reply part for markers received through bundle2 Pierre-Yves David <pierre-yves.david@fb.com> parents: 22338 diff changeset	1260 op.records.add('obsmarkers', {'new': ret}, partid)

Mercurial > public > mercurial-scm > hg-stable

annotate mercurial/bundle2.py @ 24794:21f2e8f48ae1