mercurial-scm/hg-stable: mercurial/bundle2.py annotate

annotate mercurial/bundle2.py @ 21005:3d38ebb586fe

bundle2: rename part to bundlepart We are going to introduce an `unbundlepart` dedicated to reading bundle. So we need to rename the one used to create bundle. Even if dedicated to creation, this is still used for unbundling until we get the new class.

author	Pierre-Yves David <pierre-yves.david@fb.com>
date	Fri, 11 Apr 2014 07:36:14 -0700
parents	27ab4b8d2503
children	a813caca89b3

rev	line source
20801 9c5183cb9bca bundle2: very first version of a bundle2 bundler Pierre-Yves David <pierre-yves.david@fb.com> parents: diff changeset	1 # bundle2.py - generic container format to transmit arbitrary data.
9c5183cb9bca bundle2: very first version of a bundle2 bundler Pierre-Yves David <pierre-yves.david@fb.com> parents: diff changeset	2 #
9c5183cb9bca bundle2: very first version of a bundle2 bundler Pierre-Yves David <pierre-yves.david@fb.com> parents: diff changeset	3 # Copyright 2013 Facebook, Inc.
9c5183cb9bca bundle2: very first version of a bundle2 bundler Pierre-Yves David <pierre-yves.david@fb.com> parents: diff changeset	4 #
9c5183cb9bca bundle2: very first version of a bundle2 bundler Pierre-Yves David <pierre-yves.david@fb.com> parents: diff changeset	5 # This software may be used and distributed according to the terms of the
9c5183cb9bca bundle2: very first version of a bundle2 bundler Pierre-Yves David <pierre-yves.david@fb.com> parents: diff changeset	6 # GNU General Public License version 2 or any later version.
9c5183cb9bca bundle2: very first version of a bundle2 bundler Pierre-Yves David <pierre-yves.david@fb.com> parents: diff changeset	7 """Handling of the new bundle2 format
9c5183cb9bca bundle2: very first version of a bundle2 bundler Pierre-Yves David <pierre-yves.david@fb.com> parents: diff changeset	8
9c5183cb9bca bundle2: very first version of a bundle2 bundler Pierre-Yves David <pierre-yves.david@fb.com> parents: diff changeset	9 The goal of bundle2 is to act as an atomically packet to transmit a set of
9c5183cb9bca bundle2: very first version of a bundle2 bundler Pierre-Yves David <pierre-yves.david@fb.com> parents: diff changeset	10 payloads in an application agnostic way. It consist in a sequence of "parts"
9c5183cb9bca bundle2: very first version of a bundle2 bundler Pierre-Yves David <pierre-yves.david@fb.com> parents: diff changeset	11 that will be handed to and processed by the application layer.
9c5183cb9bca bundle2: very first version of a bundle2 bundler Pierre-Yves David <pierre-yves.david@fb.com> parents: diff changeset	12
9c5183cb9bca bundle2: very first version of a bundle2 bundler Pierre-Yves David <pierre-yves.david@fb.com> parents: diff changeset	13
9c5183cb9bca bundle2: very first version of a bundle2 bundler Pierre-Yves David <pierre-yves.david@fb.com> parents: diff changeset	14 General format architecture
9c5183cb9bca bundle2: very first version of a bundle2 bundler Pierre-Yves David <pierre-yves.david@fb.com> parents: diff changeset	15 ===========================
9c5183cb9bca bundle2: very first version of a bundle2 bundler Pierre-Yves David <pierre-yves.david@fb.com> parents: diff changeset	16
9c5183cb9bca bundle2: very first version of a bundle2 bundler Pierre-Yves David <pierre-yves.david@fb.com> parents: diff changeset	17 The format is architectured as follow
9c5183cb9bca bundle2: very first version of a bundle2 bundler Pierre-Yves David <pierre-yves.david@fb.com> parents: diff changeset	18
9c5183cb9bca bundle2: very first version of a bundle2 bundler Pierre-Yves David <pierre-yves.david@fb.com> parents: diff changeset	19 - magic string
9c5183cb9bca bundle2: very first version of a bundle2 bundler Pierre-Yves David <pierre-yves.david@fb.com> parents: diff changeset	20 - stream level parameters
9c5183cb9bca bundle2: very first version of a bundle2 bundler Pierre-Yves David <pierre-yves.david@fb.com> parents: diff changeset	21 - payload parts (any number)
9c5183cb9bca bundle2: very first version of a bundle2 bundler Pierre-Yves David <pierre-yves.david@fb.com> parents: diff changeset	22 - end of stream marker.
9c5183cb9bca bundle2: very first version of a bundle2 bundler Pierre-Yves David <pierre-yves.david@fb.com> parents: diff changeset	23
20856 8a6a86c9a5b5 bundle2: support bundling of empty part (with a type) Pierre-Yves David <pierre-yves.david@fb.com> parents: 20844 diff changeset	24 the Binary format
20801 9c5183cb9bca bundle2: very first version of a bundle2 bundler Pierre-Yves David <pierre-yves.david@fb.com> parents: diff changeset	25 ============================
9c5183cb9bca bundle2: very first version of a bundle2 bundler Pierre-Yves David <pierre-yves.david@fb.com> parents: diff changeset	26
9c5183cb9bca bundle2: very first version of a bundle2 bundler Pierre-Yves David <pierre-yves.david@fb.com> parents: diff changeset	27 All numbers are unsigned and big endian.
9c5183cb9bca bundle2: very first version of a bundle2 bundler Pierre-Yves David <pierre-yves.david@fb.com> parents: diff changeset	28
9c5183cb9bca bundle2: very first version of a bundle2 bundler Pierre-Yves David <pierre-yves.david@fb.com> parents: diff changeset	29 stream level parameters
9c5183cb9bca bundle2: very first version of a bundle2 bundler Pierre-Yves David <pierre-yves.david@fb.com> parents: diff changeset	30 ------------------------
9c5183cb9bca bundle2: very first version of a bundle2 bundler Pierre-Yves David <pierre-yves.david@fb.com> parents: diff changeset	31
9c5183cb9bca bundle2: very first version of a bundle2 bundler Pierre-Yves David <pierre-yves.david@fb.com> parents: diff changeset	32 Binary format is as follow
9c5183cb9bca bundle2: very first version of a bundle2 bundler Pierre-Yves David <pierre-yves.david@fb.com> parents: diff changeset	33
9c5183cb9bca bundle2: very first version of a bundle2 bundler Pierre-Yves David <pierre-yves.david@fb.com> parents: diff changeset	34 :params size: (16 bits integer)
9c5183cb9bca bundle2: very first version of a bundle2 bundler Pierre-Yves David <pierre-yves.david@fb.com> parents: diff changeset	35
9c5183cb9bca bundle2: very first version of a bundle2 bundler Pierre-Yves David <pierre-yves.david@fb.com> parents: diff changeset	36 The total number of Bytes used by the parameters
9c5183cb9bca bundle2: very first version of a bundle2 bundler Pierre-Yves David <pierre-yves.david@fb.com> parents: diff changeset	37
9c5183cb9bca bundle2: very first version of a bundle2 bundler Pierre-Yves David <pierre-yves.david@fb.com> parents: diff changeset	38 :params value: arbitrary number of Bytes
9c5183cb9bca bundle2: very first version of a bundle2 bundler Pierre-Yves David <pierre-yves.david@fb.com> parents: diff changeset	39
9c5183cb9bca bundle2: very first version of a bundle2 bundler Pierre-Yves David <pierre-yves.david@fb.com> parents: diff changeset	40 A blob of `params size` containing the serialized version of all stream level
9c5183cb9bca bundle2: very first version of a bundle2 bundler Pierre-Yves David <pierre-yves.david@fb.com> parents: diff changeset	41 parameters.
9c5183cb9bca bundle2: very first version of a bundle2 bundler Pierre-Yves David <pierre-yves.david@fb.com> parents: diff changeset	42
20809 b93bb639451a bundle2: support for bundling parameter value Pierre-Yves David <pierre-yves.david@fb.com> parents: 20808 diff changeset	43 The blob contains a space separated list of parameters. parameter with value
20811 9785c3f8f598 bundle2: urlquote stream parameter name and value Pierre-Yves David <pierre-yves.david@fb.com> parents: 20810 diff changeset	44 are stored in the form `<name>=<value>`. Both name and value are urlquoted.
20804 db9d3991d2c6 bundle2: support bundling simple parameter Pierre-Yves David <pierre-yves.david@fb.com> parents: 20803 diff changeset	45
20813 8c74b3ce5b70 bundle2: refuse empty parameter name Pierre-Yves David <pierre-yves.david@fb.com> parents: 20812 diff changeset	46 Empty name are obviously forbidden.
8c74b3ce5b70 bundle2: refuse empty parameter name Pierre-Yves David <pierre-yves.david@fb.com> parents: 20812 diff changeset	47
20844 2631204d7305 bundle2: implement the mandatory/advisory logic for parameter Pierre-Yves David <pierre-yves.david@fb.com> parents: 20843 diff changeset	48 Name MUST start with a letter. If this first letter is lower case, the
2631204d7305 bundle2: implement the mandatory/advisory logic for parameter Pierre-Yves David <pierre-yves.david@fb.com> parents: 20843 diff changeset	49 parameter is advisory and can be safefly ignored. However when the first
2631204d7305 bundle2: implement the mandatory/advisory logic for parameter Pierre-Yves David <pierre-yves.david@fb.com> parents: 20843 diff changeset	50 letter is capital, the parameter is mandatory and the bundling process MUST
2631204d7305 bundle2: implement the mandatory/advisory logic for parameter Pierre-Yves David <pierre-yves.david@fb.com> parents: 20843 diff changeset	51 stop if he is not able to proceed it.
20814 8532f5e1b9df bundle2: force the first char of parameter to be an letter. Pierre-Yves David <pierre-yves.david@fb.com> parents: 20813 diff changeset	52
20808 4c9130c7a29f bundle2: clarify stream parameter design in the documentation Pierre-Yves David <pierre-yves.david@fb.com> parents: 20805 diff changeset	53 Stream parameters use a simple textual format for two main reasons:
20804 db9d3991d2c6 bundle2: support bundling simple parameter Pierre-Yves David <pierre-yves.david@fb.com> parents: 20803 diff changeset	54
20808 4c9130c7a29f bundle2: clarify stream parameter design in the documentation Pierre-Yves David <pierre-yves.david@fb.com> parents: 20805 diff changeset	55 - Stream level parameters should remains simple and we want to discourage any
4c9130c7a29f bundle2: clarify stream parameter design in the documentation Pierre-Yves David <pierre-yves.david@fb.com> parents: 20805 diff changeset	56 crazy usage.
4c9130c7a29f bundle2: clarify stream parameter design in the documentation Pierre-Yves David <pierre-yves.david@fb.com> parents: 20805 diff changeset	57 - Textual data allow easy human inspection of a the bundle2 header in case of
4c9130c7a29f bundle2: clarify stream parameter design in the documentation Pierre-Yves David <pierre-yves.david@fb.com> parents: 20805 diff changeset	58 troubles.
4c9130c7a29f bundle2: clarify stream parameter design in the documentation Pierre-Yves David <pierre-yves.david@fb.com> parents: 20805 diff changeset	59
4c9130c7a29f bundle2: clarify stream parameter design in the documentation Pierre-Yves David <pierre-yves.david@fb.com> parents: 20805 diff changeset	60 Any Applicative level options MUST go into a bundle2 part instead.
20801 9c5183cb9bca bundle2: very first version of a bundle2 bundler Pierre-Yves David <pierre-yves.david@fb.com> parents: diff changeset	61
9c5183cb9bca bundle2: very first version of a bundle2 bundler Pierre-Yves David <pierre-yves.david@fb.com> parents: diff changeset	62 Payload part
9c5183cb9bca bundle2: very first version of a bundle2 bundler Pierre-Yves David <pierre-yves.david@fb.com> parents: diff changeset	63 ------------------------
9c5183cb9bca bundle2: very first version of a bundle2 bundler Pierre-Yves David <pierre-yves.david@fb.com> parents: diff changeset	64
9c5183cb9bca bundle2: very first version of a bundle2 bundler Pierre-Yves David <pierre-yves.david@fb.com> parents: diff changeset	65 Binary format is as follow
9c5183cb9bca bundle2: very first version of a bundle2 bundler Pierre-Yves David <pierre-yves.david@fb.com> parents: diff changeset	66
9c5183cb9bca bundle2: very first version of a bundle2 bundler Pierre-Yves David <pierre-yves.david@fb.com> parents: diff changeset	67 :header size: (16 bits inter)
9c5183cb9bca bundle2: very first version of a bundle2 bundler Pierre-Yves David <pierre-yves.david@fb.com> parents: diff changeset	68
9c5183cb9bca bundle2: very first version of a bundle2 bundler Pierre-Yves David <pierre-yves.david@fb.com> parents: diff changeset	69 The total number of Bytes used by the part headers. When the header is empty
9c5183cb9bca bundle2: very first version of a bundle2 bundler Pierre-Yves David <pierre-yves.david@fb.com> parents: diff changeset	70 (size = 0) this is interpreted as the end of stream marker.
9c5183cb9bca bundle2: very first version of a bundle2 bundler Pierre-Yves David <pierre-yves.david@fb.com> parents: diff changeset	71
20856 8a6a86c9a5b5 bundle2: support bundling of empty part (with a type) Pierre-Yves David <pierre-yves.david@fb.com> parents: 20844 diff changeset	72 :header:
8a6a86c9a5b5 bundle2: support bundling of empty part (with a type) Pierre-Yves David <pierre-yves.david@fb.com> parents: 20844 diff changeset	73
8a6a86c9a5b5 bundle2: support bundling of empty part (with a type) Pierre-Yves David <pierre-yves.david@fb.com> parents: 20844 diff changeset	74 The header defines how to interpret the part. It contains two piece of
8a6a86c9a5b5 bundle2: support bundling of empty part (with a type) Pierre-Yves David <pierre-yves.david@fb.com> parents: 20844 diff changeset	75 data: the part type, and the part parameters.
8a6a86c9a5b5 bundle2: support bundling of empty part (with a type) Pierre-Yves David <pierre-yves.david@fb.com> parents: 20844 diff changeset	76
8a6a86c9a5b5 bundle2: support bundling of empty part (with a type) Pierre-Yves David <pierre-yves.david@fb.com> parents: 20844 diff changeset	77 The part type is used to route an application level handler, that can
8a6a86c9a5b5 bundle2: support bundling of empty part (with a type) Pierre-Yves David <pierre-yves.david@fb.com> parents: 20844 diff changeset	78 interpret payload.
8a6a86c9a5b5 bundle2: support bundling of empty part (with a type) Pierre-Yves David <pierre-yves.david@fb.com> parents: 20844 diff changeset	79
8a6a86c9a5b5 bundle2: support bundling of empty part (with a type) Pierre-Yves David <pierre-yves.david@fb.com> parents: 20844 diff changeset	80 Part parameters are passed to the application level handler. They are
8a6a86c9a5b5 bundle2: support bundling of empty part (with a type) Pierre-Yves David <pierre-yves.david@fb.com> parents: 20844 diff changeset	81 meant to convey information that will help the application level object to
8a6a86c9a5b5 bundle2: support bundling of empty part (with a type) Pierre-Yves David <pierre-yves.david@fb.com> parents: 20844 diff changeset	82 interpret the part payload.
8a6a86c9a5b5 bundle2: support bundling of empty part (with a type) Pierre-Yves David <pierre-yves.david@fb.com> parents: 20844 diff changeset	83
8a6a86c9a5b5 bundle2: support bundling of empty part (with a type) Pierre-Yves David <pierre-yves.david@fb.com> parents: 20844 diff changeset	84 The binary format of the header is has follow
8a6a86c9a5b5 bundle2: support bundling of empty part (with a type) Pierre-Yves David <pierre-yves.david@fb.com> parents: 20844 diff changeset	85
8a6a86c9a5b5 bundle2: support bundling of empty part (with a type) Pierre-Yves David <pierre-yves.david@fb.com> parents: 20844 diff changeset	86 :typesize: (one byte)
20877 9e9e3a4e9261 bundle2: part params Pierre-Yves David <pierre-yves.david@fb.com> parents: 20876 diff changeset	87
20856 8a6a86c9a5b5 bundle2: support bundling of empty part (with a type) Pierre-Yves David <pierre-yves.david@fb.com> parents: 20844 diff changeset	88 :typename: alphanumerical part name
20877 9e9e3a4e9261 bundle2: part params Pierre-Yves David <pierre-yves.david@fb.com> parents: 20876 diff changeset	89
20995 e995d104c87f bundle2: add an integer id to part Pierre-Yves David <pierre-yves.david@fb.com> parents: 20952 diff changeset	90 :partid: A 32bits integer (unique in the bundle) that can be used to refer
e995d104c87f bundle2: add an integer id to part Pierre-Yves David <pierre-yves.david@fb.com> parents: 20952 diff changeset	91 to this part.
e995d104c87f bundle2: add an integer id to part Pierre-Yves David <pierre-yves.david@fb.com> parents: 20952 diff changeset	92
20877 9e9e3a4e9261 bundle2: part params Pierre-Yves David <pierre-yves.david@fb.com> parents: 20876 diff changeset	93 :parameters:
9e9e3a4e9261 bundle2: part params Pierre-Yves David <pierre-yves.david@fb.com> parents: 20876 diff changeset	94
9e9e3a4e9261 bundle2: part params Pierre-Yves David <pierre-yves.david@fb.com> parents: 20876 diff changeset	95 Part's parameter may have arbitraty content, the binary structure is::
9e9e3a4e9261 bundle2: part params Pierre-Yves David <pierre-yves.david@fb.com> parents: 20876 diff changeset	96
9e9e3a4e9261 bundle2: part params Pierre-Yves David <pierre-yves.david@fb.com> parents: 20876 diff changeset	97 <mandatory-count><advisory-count><param-sizes><param-data>
9e9e3a4e9261 bundle2: part params Pierre-Yves David <pierre-yves.david@fb.com> parents: 20876 diff changeset	98
9e9e3a4e9261 bundle2: part params Pierre-Yves David <pierre-yves.david@fb.com> parents: 20876 diff changeset	99 :mandatory-count: 1 byte, number of mandatory parameters
9e9e3a4e9261 bundle2: part params Pierre-Yves David <pierre-yves.david@fb.com> parents: 20876 diff changeset	100
9e9e3a4e9261 bundle2: part params Pierre-Yves David <pierre-yves.david@fb.com> parents: 20876 diff changeset	101 :advisory-count: 1 byte, number of advisory parameters
9e9e3a4e9261 bundle2: part params Pierre-Yves David <pierre-yves.david@fb.com> parents: 20876 diff changeset	102
9e9e3a4e9261 bundle2: part params Pierre-Yves David <pierre-yves.david@fb.com> parents: 20876 diff changeset	103 :param-sizes:
9e9e3a4e9261 bundle2: part params Pierre-Yves David <pierre-yves.david@fb.com> parents: 20876 diff changeset	104
9e9e3a4e9261 bundle2: part params Pierre-Yves David <pierre-yves.david@fb.com> parents: 20876 diff changeset	105 N couple of bytes, where N is the total number of parameters. Each
9e9e3a4e9261 bundle2: part params Pierre-Yves David <pierre-yves.david@fb.com> parents: 20876 diff changeset	106 couple contains (<size-of-key>, <size-of-value) for one parameter.
9e9e3a4e9261 bundle2: part params Pierre-Yves David <pierre-yves.david@fb.com> parents: 20876 diff changeset	107
9e9e3a4e9261 bundle2: part params Pierre-Yves David <pierre-yves.david@fb.com> parents: 20876 diff changeset	108 :param-data:
9e9e3a4e9261 bundle2: part params Pierre-Yves David <pierre-yves.david@fb.com> parents: 20876 diff changeset	109
9e9e3a4e9261 bundle2: part params Pierre-Yves David <pierre-yves.david@fb.com> parents: 20876 diff changeset	110 A blob of bytes from which each parameter key and value can be
9e9e3a4e9261 bundle2: part params Pierre-Yves David <pierre-yves.david@fb.com> parents: 20876 diff changeset	111 retrieved using the list of size couples stored in the previous
9e9e3a4e9261 bundle2: part params Pierre-Yves David <pierre-yves.david@fb.com> parents: 20876 diff changeset	112 field.
9e9e3a4e9261 bundle2: part params Pierre-Yves David <pierre-yves.david@fb.com> parents: 20876 diff changeset	113
9e9e3a4e9261 bundle2: part params Pierre-Yves David <pierre-yves.david@fb.com> parents: 20876 diff changeset	114 Mandatory parameters comes first, then the advisory ones.
20856 8a6a86c9a5b5 bundle2: support bundling of empty part (with a type) Pierre-Yves David <pierre-yves.david@fb.com> parents: 20844 diff changeset	115
8a6a86c9a5b5 bundle2: support bundling of empty part (with a type) Pierre-Yves David <pierre-yves.david@fb.com> parents: 20844 diff changeset	116 :payload:
8a6a86c9a5b5 bundle2: support bundling of empty part (with a type) Pierre-Yves David <pierre-yves.david@fb.com> parents: 20844 diff changeset	117
20876 ddd56f3eb786 bundle2: support for bundling and unbundling payload Pierre-Yves David <pierre-yves.david@fb.com> parents: 20864 diff changeset	118 payload is a series of `<chunksize><chunkdata>`.
ddd56f3eb786 bundle2: support for bundling and unbundling payload Pierre-Yves David <pierre-yves.david@fb.com> parents: 20864 diff changeset	119
ddd56f3eb786 bundle2: support for bundling and unbundling payload Pierre-Yves David <pierre-yves.david@fb.com> parents: 20864 diff changeset	120 `chunksize` is a 32 bits integer, `chunkdata` are plain bytes (as much as
ddd56f3eb786 bundle2: support for bundling and unbundling payload Pierre-Yves David <pierre-yves.david@fb.com> parents: 20864 diff changeset	121 `chunksize` says)` The payload part is concluded by a zero size chunk.
ddd56f3eb786 bundle2: support for bundling and unbundling payload Pierre-Yves David <pierre-yves.david@fb.com> parents: 20864 diff changeset	122
ddd56f3eb786 bundle2: support for bundling and unbundling payload Pierre-Yves David <pierre-yves.david@fb.com> parents: 20864 diff changeset	123 The current implementation always produces either zero or one chunk.
ddd56f3eb786 bundle2: support for bundling and unbundling payload Pierre-Yves David <pierre-yves.david@fb.com> parents: 20864 diff changeset	124 This is an implementation limitation that will ultimatly be lifted.
20891 1c6cd23fc221 bundle2: add some distinction between mandatory and advisory part Pierre-Yves David <pierre-yves.david@fb.com> parents: 20890 diff changeset	125
1c6cd23fc221 bundle2: add some distinction between mandatory and advisory part Pierre-Yves David <pierre-yves.david@fb.com> parents: 20890 diff changeset	126 Bundle processing
1c6cd23fc221 bundle2: add some distinction between mandatory and advisory part Pierre-Yves David <pierre-yves.david@fb.com> parents: 20890 diff changeset	127 ============================
1c6cd23fc221 bundle2: add some distinction between mandatory and advisory part Pierre-Yves David <pierre-yves.david@fb.com> parents: 20890 diff changeset	128
1c6cd23fc221 bundle2: add some distinction between mandatory and advisory part Pierre-Yves David <pierre-yves.david@fb.com> parents: 20890 diff changeset	129 Each part is processed in order using a "part handler". Handler are registered
1c6cd23fc221 bundle2: add some distinction between mandatory and advisory part Pierre-Yves David <pierre-yves.david@fb.com> parents: 20890 diff changeset	130 for a certain part type.
1c6cd23fc221 bundle2: add some distinction between mandatory and advisory part Pierre-Yves David <pierre-yves.david@fb.com> parents: 20890 diff changeset	131
1c6cd23fc221 bundle2: add some distinction between mandatory and advisory part Pierre-Yves David <pierre-yves.david@fb.com> parents: 20890 diff changeset	132 The matching of a part to its handler is case insensitive. The case of the
1c6cd23fc221 bundle2: add some distinction between mandatory and advisory part Pierre-Yves David <pierre-yves.david@fb.com> parents: 20890 diff changeset	133 part type is used to know if a part is mandatory or advisory. If the Part type
1c6cd23fc221 bundle2: add some distinction between mandatory and advisory part Pierre-Yves David <pierre-yves.david@fb.com> parents: 20890 diff changeset	134 contains any uppercase char it is considered mandatory. When no handler is
1c6cd23fc221 bundle2: add some distinction between mandatory and advisory part Pierre-Yves David <pierre-yves.david@fb.com> parents: 20890 diff changeset	135 known for a Mandatory part, the process is aborted and an exception is raised.
20892 6fe95448596d bundle2: read the whole bundle from stream on abort Pierre-Yves David <pierre-yves.david@fb.com> parents: 20891 diff changeset	136 If the part is advisory and no handler is known, the part is ignored. When the
6fe95448596d bundle2: read the whole bundle from stream on abort Pierre-Yves David <pierre-yves.david@fb.com> parents: 20891 diff changeset	137 process is aborted, the full bundle is still read from the stream to keep the
6fe95448596d bundle2: read the whole bundle from stream on abort Pierre-Yves David <pierre-yves.david@fb.com> parents: 20891 diff changeset	138 channel usable. But none of the part read from an abort are processed. In the
6fe95448596d bundle2: read the whole bundle from stream on abort Pierre-Yves David <pierre-yves.david@fb.com> parents: 20891 diff changeset	139 future, dropping the stream may become an option for channel we do not care to
6fe95448596d bundle2: read the whole bundle from stream on abort Pierre-Yves David <pierre-yves.david@fb.com> parents: 20891 diff changeset	140 preserve.
20801 9c5183cb9bca bundle2: very first version of a bundle2 bundler Pierre-Yves David <pierre-yves.david@fb.com> parents: diff changeset	141 """
9c5183cb9bca bundle2: very first version of a bundle2 bundler Pierre-Yves David <pierre-yves.david@fb.com> parents: diff changeset	142
20802 520df53ad26a bundle2: a very first version of bundle2 unbundler Pierre-Yves David <pierre-yves.david@fb.com> parents: 20801 diff changeset	143 import util
20804 db9d3991d2c6 bundle2: support bundling simple parameter Pierre-Yves David <pierre-yves.david@fb.com> parents: 20803 diff changeset	144 import struct
20811 9785c3f8f598 bundle2: urlquote stream parameter name and value Pierre-Yves David <pierre-yves.david@fb.com> parents: 20810 diff changeset	145 import urllib
20814 8532f5e1b9df bundle2: force the first char of parameter to be an letter. Pierre-Yves David <pierre-yves.david@fb.com> parents: 20813 diff changeset	146 import string
20950 c7ceae0faf69 bundle2: first crude version of bundling changeset with bundle2 Pierre-Yves David <pierre-yves.david@fb.com> parents: 20949 diff changeset	147 import StringIO
20804 db9d3991d2c6 bundle2: support bundling simple parameter Pierre-Yves David <pierre-yves.david@fb.com> parents: 20803 diff changeset	148
20802 520df53ad26a bundle2: a very first version of bundle2 unbundler Pierre-Yves David <pierre-yves.david@fb.com> parents: 20801 diff changeset	149 import changegroup
20803 88db3e615319 bundle2: make sure the unbundler refuse non bundle2 stream Pierre-Yves David <pierre-yves.david@fb.com> parents: 20802 diff changeset	150 from i18n import _
20802 520df53ad26a bundle2: a very first version of bundle2 unbundler Pierre-Yves David <pierre-yves.david@fb.com> parents: 20801 diff changeset	151
20804 db9d3991d2c6 bundle2: support bundling simple parameter Pierre-Yves David <pierre-yves.david@fb.com> parents: 20803 diff changeset	152 _pack = struct.pack
db9d3991d2c6 bundle2: support bundling simple parameter Pierre-Yves David <pierre-yves.david@fb.com> parents: 20803 diff changeset	153 _unpack = struct.unpack
db9d3991d2c6 bundle2: support bundling simple parameter Pierre-Yves David <pierre-yves.david@fb.com> parents: 20803 diff changeset	154
20801 9c5183cb9bca bundle2: very first version of a bundle2 bundler Pierre-Yves David <pierre-yves.david@fb.com> parents: diff changeset	155 _magicstring = 'HG20'
9c5183cb9bca bundle2: very first version of a bundle2 bundler Pierre-Yves David <pierre-yves.david@fb.com> parents: diff changeset	156
20804 db9d3991d2c6 bundle2: support bundling simple parameter Pierre-Yves David <pierre-yves.david@fb.com> parents: 20803 diff changeset	157 _fstreamparamsize = '>H'
20856 8a6a86c9a5b5 bundle2: support bundling of empty part (with a type) Pierre-Yves David <pierre-yves.david@fb.com> parents: 20844 diff changeset	158 _fpartheadersize = '>H'
8a6a86c9a5b5 bundle2: support bundling of empty part (with a type) Pierre-Yves David <pierre-yves.david@fb.com> parents: 20844 diff changeset	159 _fparttypesize = '>B'
20995 e995d104c87f bundle2: add an integer id to part Pierre-Yves David <pierre-yves.david@fb.com> parents: 20952 diff changeset	160 _fpartid = '>I'
20876 ddd56f3eb786 bundle2: support for bundling and unbundling payload Pierre-Yves David <pierre-yves.david@fb.com> parents: 20864 diff changeset	161 _fpayloadsize = '>I'
20877 9e9e3a4e9261 bundle2: part params Pierre-Yves David <pierre-yves.david@fb.com> parents: 20876 diff changeset	162 _fpartparamcount = '>BB'
9e9e3a4e9261 bundle2: part params Pierre-Yves David <pierre-yves.david@fb.com> parents: 20876 diff changeset	163
21001 c93bb6a08fa1 bundle2: support chunk iterator as part data Pierre-Yves David <pierre-yves.david@fb.com> parents: 21000 diff changeset	164 preferedchunksize = 4096
c93bb6a08fa1 bundle2: support chunk iterator as part data Pierre-Yves David <pierre-yves.david@fb.com> parents: 21000 diff changeset	165
20877 9e9e3a4e9261 bundle2: part params Pierre-Yves David <pierre-yves.david@fb.com> parents: 20876 diff changeset	166 def _makefpartparamsizes(nbparams):
9e9e3a4e9261 bundle2: part params Pierre-Yves David <pierre-yves.david@fb.com> parents: 20876 diff changeset	167 """return a struct format to read part parameter sizes
9e9e3a4e9261 bundle2: part params Pierre-Yves David <pierre-yves.david@fb.com> parents: 20876 diff changeset	168
9e9e3a4e9261 bundle2: part params Pierre-Yves David <pierre-yves.david@fb.com> parents: 20876 diff changeset	169 The number parameters is variable so we need to build that format
9e9e3a4e9261 bundle2: part params Pierre-Yves David <pierre-yves.david@fb.com> parents: 20876 diff changeset	170 dynamically.
9e9e3a4e9261 bundle2: part params Pierre-Yves David <pierre-yves.david@fb.com> parents: 20876 diff changeset	171 """
9e9e3a4e9261 bundle2: part params Pierre-Yves David <pierre-yves.david@fb.com> parents: 20876 diff changeset	172 return '>'+('BB'*nbparams)
20804 db9d3991d2c6 bundle2: support bundling simple parameter Pierre-Yves David <pierre-yves.david@fb.com> parents: 20803 diff changeset	173
20890 ec7fc110faee bundle2: introduce a `parthandler` decorator Pierre-Yves David <pierre-yves.david@fb.com> parents: 20889 diff changeset	174 parthandlermapping = {}
20889 deed5edb72de bundle2: first version of a bundle processing Pierre-Yves David <pierre-yves.david@fb.com> parents: 20887 diff changeset	175
20890 ec7fc110faee bundle2: introduce a `parthandler` decorator Pierre-Yves David <pierre-yves.david@fb.com> parents: 20889 diff changeset	176 def parthandler(parttype):
ec7fc110faee bundle2: introduce a `parthandler` decorator Pierre-Yves David <pierre-yves.david@fb.com> parents: 20889 diff changeset	177 """decorator that register a function as a bundle2 part handler
ec7fc110faee bundle2: introduce a `parthandler` decorator Pierre-Yves David <pierre-yves.david@fb.com> parents: 20889 diff changeset	178
ec7fc110faee bundle2: introduce a `parthandler` decorator Pierre-Yves David <pierre-yves.david@fb.com> parents: 20889 diff changeset	179 eg::
ec7fc110faee bundle2: introduce a `parthandler` decorator Pierre-Yves David <pierre-yves.david@fb.com> parents: 20889 diff changeset	180
ec7fc110faee bundle2: introduce a `parthandler` decorator Pierre-Yves David <pierre-yves.david@fb.com> parents: 20889 diff changeset	181 @parthandler('myparttype')
ec7fc110faee bundle2: introduce a `parthandler` decorator Pierre-Yves David <pierre-yves.david@fb.com> parents: 20889 diff changeset	182 def myparttypehandler(...):
ec7fc110faee bundle2: introduce a `parthandler` decorator Pierre-Yves David <pierre-yves.david@fb.com> parents: 20889 diff changeset	183 '''process a part of type "my part".'''
ec7fc110faee bundle2: introduce a `parthandler` decorator Pierre-Yves David <pierre-yves.david@fb.com> parents: 20889 diff changeset	184 ...
ec7fc110faee bundle2: introduce a `parthandler` decorator Pierre-Yves David <pierre-yves.david@fb.com> parents: 20889 diff changeset	185 """
ec7fc110faee bundle2: introduce a `parthandler` decorator Pierre-Yves David <pierre-yves.david@fb.com> parents: 20889 diff changeset	186 def _decorator(func):
20891 1c6cd23fc221 bundle2: add some distinction between mandatory and advisory part Pierre-Yves David <pierre-yves.david@fb.com> parents: 20890 diff changeset	187 lparttype = parttype.lower() # enforce lower case matching.
1c6cd23fc221 bundle2: add some distinction between mandatory and advisory part Pierre-Yves David <pierre-yves.david@fb.com> parents: 20890 diff changeset	188 assert lparttype not in parthandlermapping
1c6cd23fc221 bundle2: add some distinction between mandatory and advisory part Pierre-Yves David <pierre-yves.david@fb.com> parents: 20890 diff changeset	189 parthandlermapping[lparttype] = func
20890 ec7fc110faee bundle2: introduce a `parthandler` decorator Pierre-Yves David <pierre-yves.david@fb.com> parents: 20889 diff changeset	190 return func
ec7fc110faee bundle2: introduce a `parthandler` decorator Pierre-Yves David <pierre-yves.david@fb.com> parents: 20889 diff changeset	191 return _decorator
20889 deed5edb72de bundle2: first version of a bundle processing Pierre-Yves David <pierre-yves.david@fb.com> parents: 20887 diff changeset	192
20949 571f2903ff1e bundle2: record processing results in the bundleoperation object Pierre-Yves David <pierre-yves.david@fb.com> parents: 20948 diff changeset	193 class unbundlerecords(object):
571f2903ff1e bundle2: record processing results in the bundleoperation object Pierre-Yves David <pierre-yves.david@fb.com> parents: 20948 diff changeset	194 """keep record of what happens during and unbundle
571f2903ff1e bundle2: record processing results in the bundleoperation object Pierre-Yves David <pierre-yves.david@fb.com> parents: 20948 diff changeset	195
571f2903ff1e bundle2: record processing results in the bundleoperation object Pierre-Yves David <pierre-yves.david@fb.com> parents: 20948 diff changeset	196 New records are added using `records.add('cat', obj)`. Where 'cat' is a
571f2903ff1e bundle2: record processing results in the bundleoperation object Pierre-Yves David <pierre-yves.david@fb.com> parents: 20948 diff changeset	197 category of record and obj is an arbitraty object.
571f2903ff1e bundle2: record processing results in the bundleoperation object Pierre-Yves David <pierre-yves.david@fb.com> parents: 20948 diff changeset	198
571f2903ff1e bundle2: record processing results in the bundleoperation object Pierre-Yves David <pierre-yves.david@fb.com> parents: 20948 diff changeset	199 `records['cat']` will return all entries of this category 'cat'.
571f2903ff1e bundle2: record processing results in the bundleoperation object Pierre-Yves David <pierre-yves.david@fb.com> parents: 20948 diff changeset	200
571f2903ff1e bundle2: record processing results in the bundleoperation object Pierre-Yves David <pierre-yves.david@fb.com> parents: 20948 diff changeset	201 Iterating on the object itself will yield `('category', obj)` tuples
571f2903ff1e bundle2: record processing results in the bundleoperation object Pierre-Yves David <pierre-yves.david@fb.com> parents: 20948 diff changeset	202 for all entries.
571f2903ff1e bundle2: record processing results in the bundleoperation object Pierre-Yves David <pierre-yves.david@fb.com> parents: 20948 diff changeset	203
571f2903ff1e bundle2: record processing results in the bundleoperation object Pierre-Yves David <pierre-yves.david@fb.com> parents: 20948 diff changeset	204 All iterations happens in chronological order.
571f2903ff1e bundle2: record processing results in the bundleoperation object Pierre-Yves David <pierre-yves.david@fb.com> parents: 20948 diff changeset	205 """
571f2903ff1e bundle2: record processing results in the bundleoperation object Pierre-Yves David <pierre-yves.david@fb.com> parents: 20948 diff changeset	206
571f2903ff1e bundle2: record processing results in the bundleoperation object Pierre-Yves David <pierre-yves.david@fb.com> parents: 20948 diff changeset	207 def __init__(self):
571f2903ff1e bundle2: record processing results in the bundleoperation object Pierre-Yves David <pierre-yves.david@fb.com> parents: 20948 diff changeset	208 self._categories = {}
571f2903ff1e bundle2: record processing results in the bundleoperation object Pierre-Yves David <pierre-yves.david@fb.com> parents: 20948 diff changeset	209 self._sequences = []
20996 ed3c5e18a047 bundle2: add reply awareness to unbundlerecords Pierre-Yves David <pierre-yves.david@fb.com> parents: 20995 diff changeset	210 self._replies = {}
20949 571f2903ff1e bundle2: record processing results in the bundleoperation object Pierre-Yves David <pierre-yves.david@fb.com> parents: 20948 diff changeset	211
20996 ed3c5e18a047 bundle2: add reply awareness to unbundlerecords Pierre-Yves David <pierre-yves.david@fb.com> parents: 20995 diff changeset	212 def add(self, category, entry, inreplyto=None):
20949 571f2903ff1e bundle2: record processing results in the bundleoperation object Pierre-Yves David <pierre-yves.david@fb.com> parents: 20948 diff changeset	213 """add a new record of a given category.
571f2903ff1e bundle2: record processing results in the bundleoperation object Pierre-Yves David <pierre-yves.david@fb.com> parents: 20948 diff changeset	214
571f2903ff1e bundle2: record processing results in the bundleoperation object Pierre-Yves David <pierre-yves.david@fb.com> parents: 20948 diff changeset	215 The entry can then be retrieved in the list returned by
571f2903ff1e bundle2: record processing results in the bundleoperation object Pierre-Yves David <pierre-yves.david@fb.com> parents: 20948 diff changeset	216 self['category']."""
571f2903ff1e bundle2: record processing results in the bundleoperation object Pierre-Yves David <pierre-yves.david@fb.com> parents: 20948 diff changeset	217 self._categories.setdefault(category, []).append(entry)
571f2903ff1e bundle2: record processing results in the bundleoperation object Pierre-Yves David <pierre-yves.david@fb.com> parents: 20948 diff changeset	218 self._sequences.append((category, entry))
20996 ed3c5e18a047 bundle2: add reply awareness to unbundlerecords Pierre-Yves David <pierre-yves.david@fb.com> parents: 20995 diff changeset	219 if inreplyto is not None:
ed3c5e18a047 bundle2: add reply awareness to unbundlerecords Pierre-Yves David <pierre-yves.david@fb.com> parents: 20995 diff changeset	220 self.getreplies(inreplyto).add(category, entry)
ed3c5e18a047 bundle2: add reply awareness to unbundlerecords Pierre-Yves David <pierre-yves.david@fb.com> parents: 20995 diff changeset	221
ed3c5e18a047 bundle2: add reply awareness to unbundlerecords Pierre-Yves David <pierre-yves.david@fb.com> parents: 20995 diff changeset	222 def getreplies(self, partid):
ed3c5e18a047 bundle2: add reply awareness to unbundlerecords Pierre-Yves David <pierre-yves.david@fb.com> parents: 20995 diff changeset	223 """get the subrecords that replies to a specific part"""
ed3c5e18a047 bundle2: add reply awareness to unbundlerecords Pierre-Yves David <pierre-yves.david@fb.com> parents: 20995 diff changeset	224 return self._replies.setdefault(partid, unbundlerecords())
20949 571f2903ff1e bundle2: record processing results in the bundleoperation object Pierre-Yves David <pierre-yves.david@fb.com> parents: 20948 diff changeset	225
571f2903ff1e bundle2: record processing results in the bundleoperation object Pierre-Yves David <pierre-yves.david@fb.com> parents: 20948 diff changeset	226 def __getitem__(self, cat):
571f2903ff1e bundle2: record processing results in the bundleoperation object Pierre-Yves David <pierre-yves.david@fb.com> parents: 20948 diff changeset	227 return tuple(self._categories.get(cat, ()))
571f2903ff1e bundle2: record processing results in the bundleoperation object Pierre-Yves David <pierre-yves.david@fb.com> parents: 20948 diff changeset	228
571f2903ff1e bundle2: record processing results in the bundleoperation object Pierre-Yves David <pierre-yves.david@fb.com> parents: 20948 diff changeset	229 def __iter__(self):
571f2903ff1e bundle2: record processing results in the bundleoperation object Pierre-Yves David <pierre-yves.david@fb.com> parents: 20948 diff changeset	230 return iter(self._sequences)
571f2903ff1e bundle2: record processing results in the bundleoperation object Pierre-Yves David <pierre-yves.david@fb.com> parents: 20948 diff changeset	231
571f2903ff1e bundle2: record processing results in the bundleoperation object Pierre-Yves David <pierre-yves.david@fb.com> parents: 20948 diff changeset	232 def __len__(self):
571f2903ff1e bundle2: record processing results in the bundleoperation object Pierre-Yves David <pierre-yves.david@fb.com> parents: 20948 diff changeset	233 return len(self._sequences)
571f2903ff1e bundle2: record processing results in the bundleoperation object Pierre-Yves David <pierre-yves.david@fb.com> parents: 20948 diff changeset	234
571f2903ff1e bundle2: record processing results in the bundleoperation object Pierre-Yves David <pierre-yves.david@fb.com> parents: 20948 diff changeset	235 def __nonzero__(self):
571f2903ff1e bundle2: record processing results in the bundleoperation object Pierre-Yves David <pierre-yves.david@fb.com> parents: 20948 diff changeset	236 return bool(self._sequences)
571f2903ff1e bundle2: record processing results in the bundleoperation object Pierre-Yves David <pierre-yves.david@fb.com> parents: 20948 diff changeset	237
20948 329cd74b52bd bundle2: introduce a bundleoperation object Pierre-Yves David <pierre-yves.david@fb.com> parents: 20947 diff changeset	238 class bundleoperation(object):
329cd74b52bd bundle2: introduce a bundleoperation object Pierre-Yves David <pierre-yves.david@fb.com> parents: 20947 diff changeset	239 """an object that represents a single bundling process
329cd74b52bd bundle2: introduce a bundleoperation object Pierre-Yves David <pierre-yves.david@fb.com> parents: 20947 diff changeset	240
329cd74b52bd bundle2: introduce a bundleoperation object Pierre-Yves David <pierre-yves.david@fb.com> parents: 20947 diff changeset	241 Its purpose is to carry unbundle-related objects and states.
329cd74b52bd bundle2: introduce a bundleoperation object Pierre-Yves David <pierre-yves.david@fb.com> parents: 20947 diff changeset	242
329cd74b52bd bundle2: introduce a bundleoperation object Pierre-Yves David <pierre-yves.david@fb.com> parents: 20947 diff changeset	243 A new object should be created at the beginning of each bundle processing.
329cd74b52bd bundle2: introduce a bundleoperation object Pierre-Yves David <pierre-yves.david@fb.com> parents: 20947 diff changeset	244 The object is to be returned by the processing function.
329cd74b52bd bundle2: introduce a bundleoperation object Pierre-Yves David <pierre-yves.david@fb.com> parents: 20947 diff changeset	245
329cd74b52bd bundle2: introduce a bundleoperation object Pierre-Yves David <pierre-yves.david@fb.com> parents: 20947 diff changeset	246 The object has very little content now it will ultimately contain:
329cd74b52bd bundle2: introduce a bundleoperation object Pierre-Yves David <pierre-yves.david@fb.com> parents: 20947 diff changeset	247 * an access to the repo the bundle is applied to,
329cd74b52bd bundle2: introduce a bundleoperation object Pierre-Yves David <pierre-yves.david@fb.com> parents: 20947 diff changeset	248 * a ui object,
329cd74b52bd bundle2: introduce a bundleoperation object Pierre-Yves David <pierre-yves.david@fb.com> parents: 20947 diff changeset	249 * a way to retrieve a transaction to add changes to the repo,
329cd74b52bd bundle2: introduce a bundleoperation object Pierre-Yves David <pierre-yves.david@fb.com> parents: 20947 diff changeset	250 * a way to record the result of processing each part,
329cd74b52bd bundle2: introduce a bundleoperation object Pierre-Yves David <pierre-yves.david@fb.com> parents: 20947 diff changeset	251 * a way to construct a bundle response when applicable.
329cd74b52bd bundle2: introduce a bundleoperation object Pierre-Yves David <pierre-yves.david@fb.com> parents: 20947 diff changeset	252 """
329cd74b52bd bundle2: introduce a bundleoperation object Pierre-Yves David <pierre-yves.david@fb.com> parents: 20947 diff changeset	253
20952 b24ee5076b94 bundle2: make it possible have a global transaction for the unbundling Pierre-Yves David <pierre-yves.david@fb.com> parents: 20950 diff changeset	254 def __init__(self, repo, transactiongetter):
20948 329cd74b52bd bundle2: introduce a bundleoperation object Pierre-Yves David <pierre-yves.david@fb.com> parents: 20947 diff changeset	255 self.repo = repo
329cd74b52bd bundle2: introduce a bundleoperation object Pierre-Yves David <pierre-yves.david@fb.com> parents: 20947 diff changeset	256 self.ui = repo.ui
20949 571f2903ff1e bundle2: record processing results in the bundleoperation object Pierre-Yves David <pierre-yves.david@fb.com> parents: 20948 diff changeset	257 self.records = unbundlerecords()
20952 b24ee5076b94 bundle2: make it possible have a global transaction for the unbundling Pierre-Yves David <pierre-yves.david@fb.com> parents: 20950 diff changeset	258 self.gettransaction = transactiongetter
20997 d7df4b7378ae bundle2: produce a bundle2 reply Pierre-Yves David <pierre-yves.david@fb.com> parents: 20996 diff changeset	259 self.reply = None
20948 329cd74b52bd bundle2: introduce a bundleoperation object Pierre-Yves David <pierre-yves.david@fb.com> parents: 20947 diff changeset	260
20952 b24ee5076b94 bundle2: make it possible have a global transaction for the unbundling Pierre-Yves David <pierre-yves.david@fb.com> parents: 20950 diff changeset	261 class TransactionUnavailable(RuntimeError):
b24ee5076b94 bundle2: make it possible have a global transaction for the unbundling Pierre-Yves David <pierre-yves.david@fb.com> parents: 20950 diff changeset	262 pass
b24ee5076b94 bundle2: make it possible have a global transaction for the unbundling Pierre-Yves David <pierre-yves.david@fb.com> parents: 20950 diff changeset	263
b24ee5076b94 bundle2: make it possible have a global transaction for the unbundling Pierre-Yves David <pierre-yves.david@fb.com> parents: 20950 diff changeset	264 def _notransaction():
b24ee5076b94 bundle2: make it possible have a global transaction for the unbundling Pierre-Yves David <pierre-yves.david@fb.com> parents: 20950 diff changeset	265 """default method to get a transaction while processing a bundle
b24ee5076b94 bundle2: make it possible have a global transaction for the unbundling Pierre-Yves David <pierre-yves.david@fb.com> parents: 20950 diff changeset	266
b24ee5076b94 bundle2: make it possible have a global transaction for the unbundling Pierre-Yves David <pierre-yves.david@fb.com> parents: 20950 diff changeset	267 Raise an exception to highlight the fact that no transaction was expected
b24ee5076b94 bundle2: make it possible have a global transaction for the unbundling Pierre-Yves David <pierre-yves.david@fb.com> parents: 20950 diff changeset	268 to be created"""
b24ee5076b94 bundle2: make it possible have a global transaction for the unbundling Pierre-Yves David <pierre-yves.david@fb.com> parents: 20950 diff changeset	269 raise TransactionUnavailable()
b24ee5076b94 bundle2: make it possible have a global transaction for the unbundling Pierre-Yves David <pierre-yves.david@fb.com> parents: 20950 diff changeset	270
b24ee5076b94 bundle2: make it possible have a global transaction for the unbundling Pierre-Yves David <pierre-yves.david@fb.com> parents: 20950 diff changeset	271 def processbundle(repo, unbundler, transactiongetter=_notransaction):
20889 deed5edb72de bundle2: first version of a bundle processing Pierre-Yves David <pierre-yves.david@fb.com> parents: 20887 diff changeset	272 """This function process a bundle, apply effect to/from a repo
deed5edb72de bundle2: first version of a bundle processing Pierre-Yves David <pierre-yves.david@fb.com> parents: 20887 diff changeset	273
20947 c33d7bf53812 bundle2: feed a unbundle20 to the `processbundle` function Pierre-Yves David <pierre-yves.david@fb.com> parents: 20892 diff changeset	274 It iterates over each part then searches for and uses the proper handling
c33d7bf53812 bundle2: feed a unbundle20 to the `processbundle` function Pierre-Yves David <pierre-yves.david@fb.com> parents: 20892 diff changeset	275 code to process the part. Parts are processed in order.
20889 deed5edb72de bundle2: first version of a bundle processing Pierre-Yves David <pierre-yves.david@fb.com> parents: 20887 diff changeset	276
deed5edb72de bundle2: first version of a bundle processing Pierre-Yves David <pierre-yves.david@fb.com> parents: 20887 diff changeset	277 This is very early version of this function that will be strongly reworked
deed5edb72de bundle2: first version of a bundle processing Pierre-Yves David <pierre-yves.david@fb.com> parents: 20887 diff changeset	278 before final usage.
deed5edb72de bundle2: first version of a bundle processing Pierre-Yves David <pierre-yves.david@fb.com> parents: 20887 diff changeset	279
20891 1c6cd23fc221 bundle2: add some distinction between mandatory and advisory part Pierre-Yves David <pierre-yves.david@fb.com> parents: 20890 diff changeset	280 Unknown Mandatory part will abort the process.
20889 deed5edb72de bundle2: first version of a bundle processing Pierre-Yves David <pierre-yves.david@fb.com> parents: 20887 diff changeset	281 """
20952 b24ee5076b94 bundle2: make it possible have a global transaction for the unbundling Pierre-Yves David <pierre-yves.david@fb.com> parents: 20950 diff changeset	282 op = bundleoperation(repo, transactiongetter)
20889 deed5edb72de bundle2: first version of a bundle processing Pierre-Yves David <pierre-yves.david@fb.com> parents: 20887 diff changeset	283 # todo:
20997 d7df4b7378ae bundle2: produce a bundle2 reply Pierre-Yves David <pierre-yves.david@fb.com> parents: 20996 diff changeset	284 # - only create reply bundle if requested.
d7df4b7378ae bundle2: produce a bundle2 reply Pierre-Yves David <pierre-yves.david@fb.com> parents: 20996 diff changeset	285 op.reply = bundle20(op.ui)
d7df4b7378ae bundle2: produce a bundle2 reply Pierre-Yves David <pierre-yves.david@fb.com> parents: 20996 diff changeset	286 # todo:
20889 deed5edb72de bundle2: first version of a bundle processing Pierre-Yves David <pierre-yves.david@fb.com> parents: 20887 diff changeset	287 # - replace this is a init function soon.
deed5edb72de bundle2: first version of a bundle processing Pierre-Yves David <pierre-yves.david@fb.com> parents: 20887 diff changeset	288 # - exception catching
deed5edb72de bundle2: first version of a bundle processing Pierre-Yves David <pierre-yves.david@fb.com> parents: 20887 diff changeset	289 unbundler.params
20892 6fe95448596d bundle2: read the whole bundle from stream on abort Pierre-Yves David <pierre-yves.david@fb.com> parents: 20891 diff changeset	290 iterparts = iter(unbundler)
6fe95448596d bundle2: read the whole bundle from stream on abort Pierre-Yves David <pierre-yves.david@fb.com> parents: 20891 diff changeset	291 try:
6fe95448596d bundle2: read the whole bundle from stream on abort Pierre-Yves David <pierre-yves.david@fb.com> parents: 20891 diff changeset	292 for part in iterparts:
6fe95448596d bundle2: read the whole bundle from stream on abort Pierre-Yves David <pierre-yves.david@fb.com> parents: 20891 diff changeset	293 parttype = part.type
6fe95448596d bundle2: read the whole bundle from stream on abort Pierre-Yves David <pierre-yves.david@fb.com> parents: 20891 diff changeset	294 # part key are matched lower case
6fe95448596d bundle2: read the whole bundle from stream on abort Pierre-Yves David <pierre-yves.david@fb.com> parents: 20891 diff changeset	295 key = parttype.lower()
6fe95448596d bundle2: read the whole bundle from stream on abort Pierre-Yves David <pierre-yves.david@fb.com> parents: 20891 diff changeset	296 try:
6fe95448596d bundle2: read the whole bundle from stream on abort Pierre-Yves David <pierre-yves.david@fb.com> parents: 20891 diff changeset	297 handler = parthandlermapping[key]
20948 329cd74b52bd bundle2: introduce a bundleoperation object Pierre-Yves David <pierre-yves.david@fb.com> parents: 20947 diff changeset	298 op.ui.debug('found a handler for part %r\n' % parttype)
20892 6fe95448596d bundle2: read the whole bundle from stream on abort Pierre-Yves David <pierre-yves.david@fb.com> parents: 20891 diff changeset	299 except KeyError:
6fe95448596d bundle2: read the whole bundle from stream on abort Pierre-Yves David <pierre-yves.david@fb.com> parents: 20891 diff changeset	300 if key != parttype: # mandatory parts
6fe95448596d bundle2: read the whole bundle from stream on abort Pierre-Yves David <pierre-yves.david@fb.com> parents: 20891 diff changeset	301 # todo:
6fe95448596d bundle2: read the whole bundle from stream on abort Pierre-Yves David <pierre-yves.david@fb.com> parents: 20891 diff changeset	302 # - use a more precise exception
6fe95448596d bundle2: read the whole bundle from stream on abort Pierre-Yves David <pierre-yves.david@fb.com> parents: 20891 diff changeset	303 raise
20948 329cd74b52bd bundle2: introduce a bundleoperation object Pierre-Yves David <pierre-yves.david@fb.com> parents: 20947 diff changeset	304 op.ui.debug('ignoring unknown advisory part %r\n' % key)
20891 1c6cd23fc221 bundle2: add some distinction between mandatory and advisory part Pierre-Yves David <pierre-yves.david@fb.com> parents: 20890 diff changeset	305 # todo:
20892 6fe95448596d bundle2: read the whole bundle from stream on abort Pierre-Yves David <pierre-yves.david@fb.com> parents: 20891 diff changeset	306 # - consume the part once we use streaming
6fe95448596d bundle2: read the whole bundle from stream on abort Pierre-Yves David <pierre-yves.david@fb.com> parents: 20891 diff changeset	307 continue
21004 27ab4b8d2503 bundle2: comment to clarify why the handler call is where it is Pierre-Yves David <pierre-yves.david@fb.com> parents: 21001 diff changeset	308
27ab4b8d2503 bundle2: comment to clarify why the handler call is where it is Pierre-Yves David <pierre-yves.david@fb.com> parents: 21001 diff changeset	309 # handler is called outside the above try block so that we don't
27ab4b8d2503 bundle2: comment to clarify why the handler call is where it is Pierre-Yves David <pierre-yves.david@fb.com> parents: 21001 diff changeset	310 # risk catching KeyErrors from anything other than the
27ab4b8d2503 bundle2: comment to clarify why the handler call is where it is Pierre-Yves David <pierre-yves.david@fb.com> parents: 21001 diff changeset	311 # parthandlermapping lookup (any KeyError raised by handler()
27ab4b8d2503 bundle2: comment to clarify why the handler call is where it is Pierre-Yves David <pierre-yves.david@fb.com> parents: 21001 diff changeset	312 # itself represents a defect of a different variety).
20948 329cd74b52bd bundle2: introduce a bundleoperation object Pierre-Yves David <pierre-yves.david@fb.com> parents: 20947 diff changeset	313 handler(op, part)
20892 6fe95448596d bundle2: read the whole bundle from stream on abort Pierre-Yves David <pierre-yves.david@fb.com> parents: 20891 diff changeset	314 except Exception:
6fe95448596d bundle2: read the whole bundle from stream on abort Pierre-Yves David <pierre-yves.david@fb.com> parents: 20891 diff changeset	315 for part in iterparts:
6fe95448596d bundle2: read the whole bundle from stream on abort Pierre-Yves David <pierre-yves.david@fb.com> parents: 20891 diff changeset	316 pass # consume the bundle content
6fe95448596d bundle2: read the whole bundle from stream on abort Pierre-Yves David <pierre-yves.david@fb.com> parents: 20891 diff changeset	317 raise
20949 571f2903ff1e bundle2: record processing results in the bundleoperation object Pierre-Yves David <pierre-yves.david@fb.com> parents: 20948 diff changeset	318 return op
20889 deed5edb72de bundle2: first version of a bundle processing Pierre-Yves David <pierre-yves.david@fb.com> parents: 20887 diff changeset	319
20801 9c5183cb9bca bundle2: very first version of a bundle2 bundler Pierre-Yves David <pierre-yves.david@fb.com> parents: diff changeset	320 class bundle20(object):
9c5183cb9bca bundle2: very first version of a bundle2 bundler Pierre-Yves David <pierre-yves.david@fb.com> parents: diff changeset	321 """represent an outgoing bundle2 container
9c5183cb9bca bundle2: very first version of a bundle2 bundler Pierre-Yves David <pierre-yves.david@fb.com> parents: diff changeset	322
20856 8a6a86c9a5b5 bundle2: support bundling of empty part (with a type) Pierre-Yves David <pierre-yves.david@fb.com> parents: 20844 diff changeset	323 Use the `addparam` method to add stream level parameter. and `addpart` to
8a6a86c9a5b5 bundle2: support bundling of empty part (with a type) Pierre-Yves David <pierre-yves.david@fb.com> parents: 20844 diff changeset	324 populate it. Then call `getchunks` to retrieve all the binary chunks of
8a6a86c9a5b5 bundle2: support bundling of empty part (with a type) Pierre-Yves David <pierre-yves.david@fb.com> parents: 20844 diff changeset	325 datathat compose the bundle2 container."""
20801 9c5183cb9bca bundle2: very first version of a bundle2 bundler Pierre-Yves David <pierre-yves.david@fb.com> parents: diff changeset	326
20842 938718d72624 bundle2: print debug information during bundling Pierre-Yves David <pierre-yves.david@fb.com> parents: 20814 diff changeset	327 def __init__(self, ui):
938718d72624 bundle2: print debug information during bundling Pierre-Yves David <pierre-yves.david@fb.com> parents: 20814 diff changeset	328 self.ui = ui
20801 9c5183cb9bca bundle2: very first version of a bundle2 bundler Pierre-Yves David <pierre-yves.david@fb.com> parents: diff changeset	329 self._params = []
9c5183cb9bca bundle2: very first version of a bundle2 bundler Pierre-Yves David <pierre-yves.david@fb.com> parents: diff changeset	330 self._parts = []
9c5183cb9bca bundle2: very first version of a bundle2 bundler Pierre-Yves David <pierre-yves.david@fb.com> parents: diff changeset	331
20804 db9d3991d2c6 bundle2: support bundling simple parameter Pierre-Yves David <pierre-yves.david@fb.com> parents: 20803 diff changeset	332 def addparam(self, name, value=None):
db9d3991d2c6 bundle2: support bundling simple parameter Pierre-Yves David <pierre-yves.david@fb.com> parents: 20803 diff changeset	333 """add a stream level parameter"""
20813 8c74b3ce5b70 bundle2: refuse empty parameter name Pierre-Yves David <pierre-yves.david@fb.com> parents: 20812 diff changeset	334 if not name:
8c74b3ce5b70 bundle2: refuse empty parameter name Pierre-Yves David <pierre-yves.david@fb.com> parents: 20812 diff changeset	335 raise ValueError('empty parameter name')
20814 8532f5e1b9df bundle2: force the first char of parameter to be an letter. Pierre-Yves David <pierre-yves.david@fb.com> parents: 20813 diff changeset	336 if name[0] not in string.letters:
8532f5e1b9df bundle2: force the first char of parameter to be an letter. Pierre-Yves David <pierre-yves.david@fb.com> parents: 20813 diff changeset	337 raise ValueError('non letter first character: %r' % name)
20804 db9d3991d2c6 bundle2: support bundling simple parameter Pierre-Yves David <pierre-yves.david@fb.com> parents: 20803 diff changeset	338 self._params.append((name, value))
db9d3991d2c6 bundle2: support bundling simple parameter Pierre-Yves David <pierre-yves.david@fb.com> parents: 20803 diff changeset	339
20856 8a6a86c9a5b5 bundle2: support bundling of empty part (with a type) Pierre-Yves David <pierre-yves.david@fb.com> parents: 20844 diff changeset	340 def addpart(self, part):
8a6a86c9a5b5 bundle2: support bundling of empty part (with a type) Pierre-Yves David <pierre-yves.david@fb.com> parents: 20844 diff changeset	341 """add a new part to the bundle2 container
8a6a86c9a5b5 bundle2: support bundling of empty part (with a type) Pierre-Yves David <pierre-yves.david@fb.com> parents: 20844 diff changeset	342
8a6a86c9a5b5 bundle2: support bundling of empty part (with a type) Pierre-Yves David <pierre-yves.david@fb.com> parents: 20844 diff changeset	343 Parts contains the actuall applicative payload."""
20995 e995d104c87f bundle2: add an integer id to part Pierre-Yves David <pierre-yves.david@fb.com> parents: 20952 diff changeset	344 assert part.id is None
e995d104c87f bundle2: add an integer id to part Pierre-Yves David <pierre-yves.david@fb.com> parents: 20952 diff changeset	345 part.id = len(self._parts) # very cheap counter
20856 8a6a86c9a5b5 bundle2: support bundling of empty part (with a type) Pierre-Yves David <pierre-yves.david@fb.com> parents: 20844 diff changeset	346 self._parts.append(part)
8a6a86c9a5b5 bundle2: support bundling of empty part (with a type) Pierre-Yves David <pierre-yves.david@fb.com> parents: 20844 diff changeset	347
20801 9c5183cb9bca bundle2: very first version of a bundle2 bundler Pierre-Yves David <pierre-yves.david@fb.com> parents: diff changeset	348 def getchunks(self):
20842 938718d72624 bundle2: print debug information during bundling Pierre-Yves David <pierre-yves.david@fb.com> parents: 20814 diff changeset	349 self.ui.debug('start emission of %s stream\n' % _magicstring)
20801 9c5183cb9bca bundle2: very first version of a bundle2 bundler Pierre-Yves David <pierre-yves.david@fb.com> parents: diff changeset	350 yield _magicstring
20804 db9d3991d2c6 bundle2: support bundling simple parameter Pierre-Yves David <pierre-yves.david@fb.com> parents: 20803 diff changeset	351 param = self._paramchunk()
20842 938718d72624 bundle2: print debug information during bundling Pierre-Yves David <pierre-yves.david@fb.com> parents: 20814 diff changeset	352 self.ui.debug('bundle parameter: %s\n' % param)
20804 db9d3991d2c6 bundle2: support bundling simple parameter Pierre-Yves David <pierre-yves.david@fb.com> parents: 20803 diff changeset	353 yield _pack(_fstreamparamsize, len(param))
db9d3991d2c6 bundle2: support bundling simple parameter Pierre-Yves David <pierre-yves.david@fb.com> parents: 20803 diff changeset	354 if param:
db9d3991d2c6 bundle2: support bundling simple parameter Pierre-Yves David <pierre-yves.david@fb.com> parents: 20803 diff changeset	355 yield param
db9d3991d2c6 bundle2: support bundling simple parameter Pierre-Yves David <pierre-yves.david@fb.com> parents: 20803 diff changeset	356
20856 8a6a86c9a5b5 bundle2: support bundling of empty part (with a type) Pierre-Yves David <pierre-yves.david@fb.com> parents: 20844 diff changeset	357 self.ui.debug('start of parts\n')
8a6a86c9a5b5 bundle2: support bundling of empty part (with a type) Pierre-Yves David <pierre-yves.david@fb.com> parents: 20844 diff changeset	358 for part in self._parts:
8a6a86c9a5b5 bundle2: support bundling of empty part (with a type) Pierre-Yves David <pierre-yves.david@fb.com> parents: 20844 diff changeset	359 self.ui.debug('bundle part: "%s"\n' % part.type)
8a6a86c9a5b5 bundle2: support bundling of empty part (with a type) Pierre-Yves David <pierre-yves.david@fb.com> parents: 20844 diff changeset	360 for chunk in part.getchunks():
8a6a86c9a5b5 bundle2: support bundling of empty part (with a type) Pierre-Yves David <pierre-yves.david@fb.com> parents: 20844 diff changeset	361 yield chunk
20842 938718d72624 bundle2: print debug information during bundling Pierre-Yves David <pierre-yves.david@fb.com> parents: 20814 diff changeset	362 self.ui.debug('end of bundle\n')
20801 9c5183cb9bca bundle2: very first version of a bundle2 bundler Pierre-Yves David <pierre-yves.david@fb.com> parents: diff changeset	363 yield '\0\0'
20802 520df53ad26a bundle2: a very first version of bundle2 unbundler Pierre-Yves David <pierre-yves.david@fb.com> parents: 20801 diff changeset	364
20804 db9d3991d2c6 bundle2: support bundling simple parameter Pierre-Yves David <pierre-yves.david@fb.com> parents: 20803 diff changeset	365 def _paramchunk(self):
db9d3991d2c6 bundle2: support bundling simple parameter Pierre-Yves David <pierre-yves.david@fb.com> parents: 20803 diff changeset	366 """return a encoded version of all stream parameters"""
db9d3991d2c6 bundle2: support bundling simple parameter Pierre-Yves David <pierre-yves.david@fb.com> parents: 20803 diff changeset	367 blocks = []
20809 b93bb639451a bundle2: support for bundling parameter value Pierre-Yves David <pierre-yves.david@fb.com> parents: 20808 diff changeset	368 for par, value in self._params:
20811 9785c3f8f598 bundle2: urlquote stream parameter name and value Pierre-Yves David <pierre-yves.david@fb.com> parents: 20810 diff changeset	369 par = urllib.quote(par)
20809 b93bb639451a bundle2: support for bundling parameter value Pierre-Yves David <pierre-yves.david@fb.com> parents: 20808 diff changeset	370 if value is not None:
20811 9785c3f8f598 bundle2: urlquote stream parameter name and value Pierre-Yves David <pierre-yves.david@fb.com> parents: 20810 diff changeset	371 value = urllib.quote(value)
20809 b93bb639451a bundle2: support for bundling parameter value Pierre-Yves David <pierre-yves.david@fb.com> parents: 20808 diff changeset	372 par = '%s=%s' % (par, value)
b93bb639451a bundle2: support for bundling parameter value Pierre-Yves David <pierre-yves.david@fb.com> parents: 20808 diff changeset	373 blocks.append(par)
20804 db9d3991d2c6 bundle2: support bundling simple parameter Pierre-Yves David <pierre-yves.david@fb.com> parents: 20803 diff changeset	374 return ' '.join(blocks)
db9d3991d2c6 bundle2: support bundling simple parameter Pierre-Yves David <pierre-yves.david@fb.com> parents: 20803 diff changeset	375
20802 520df53ad26a bundle2: a very first version of bundle2 unbundler Pierre-Yves David <pierre-yves.david@fb.com> parents: 20801 diff changeset	376 class unbundle20(object):
520df53ad26a bundle2: a very first version of bundle2 unbundler Pierre-Yves David <pierre-yves.david@fb.com> parents: 20801 diff changeset	377 """interpret a bundle2 stream
520df53ad26a bundle2: a very first version of bundle2 unbundler Pierre-Yves David <pierre-yves.david@fb.com> parents: 20801 diff changeset	378
520df53ad26a bundle2: a very first version of bundle2 unbundler Pierre-Yves David <pierre-yves.david@fb.com> parents: 20801 diff changeset	379 (this will eventually yield parts)"""
520df53ad26a bundle2: a very first version of bundle2 unbundler Pierre-Yves David <pierre-yves.david@fb.com> parents: 20801 diff changeset	380
20843 0641b41b0b49 bundle2: print debug information during unbundling Pierre-Yves David <pierre-yves.david@fb.com> parents: 20842 diff changeset	381 def __init__(self, ui, fp):
0641b41b0b49 bundle2: print debug information during unbundling Pierre-Yves David <pierre-yves.david@fb.com> parents: 20842 diff changeset	382 self.ui = ui
20802 520df53ad26a bundle2: a very first version of bundle2 unbundler Pierre-Yves David <pierre-yves.david@fb.com> parents: 20801 diff changeset	383 self._fp = fp
20803 88db3e615319 bundle2: make sure the unbundler refuse non bundle2 stream Pierre-Yves David <pierre-yves.david@fb.com> parents: 20802 diff changeset	384 header = self._readexact(4)
88db3e615319 bundle2: make sure the unbundler refuse non bundle2 stream Pierre-Yves David <pierre-yves.david@fb.com> parents: 20802 diff changeset	385 magic, version = header[0:2], header[2:4]
88db3e615319 bundle2: make sure the unbundler refuse non bundle2 stream Pierre-Yves David <pierre-yves.david@fb.com> parents: 20802 diff changeset	386 if magic != 'HG':
88db3e615319 bundle2: make sure the unbundler refuse non bundle2 stream Pierre-Yves David <pierre-yves.david@fb.com> parents: 20802 diff changeset	387 raise util.Abort(_('not a Mercurial bundle'))
88db3e615319 bundle2: make sure the unbundler refuse non bundle2 stream Pierre-Yves David <pierre-yves.david@fb.com> parents: 20802 diff changeset	388 if version != '20':
88db3e615319 bundle2: make sure the unbundler refuse non bundle2 stream Pierre-Yves David <pierre-yves.david@fb.com> parents: 20802 diff changeset	389 raise util.Abort(_('unknown bundle version %s') % version)
20843 0641b41b0b49 bundle2: print debug information during unbundling Pierre-Yves David <pierre-yves.david@fb.com> parents: 20842 diff changeset	390 self.ui.debug('start processing of %s stream\n' % header)
20802 520df53ad26a bundle2: a very first version of bundle2 unbundler Pierre-Yves David <pierre-yves.david@fb.com> parents: 20801 diff changeset	391
520df53ad26a bundle2: a very first version of bundle2 unbundler Pierre-Yves David <pierre-yves.david@fb.com> parents: 20801 diff changeset	392 def _unpack(self, format):
520df53ad26a bundle2: a very first version of bundle2 unbundler Pierre-Yves David <pierre-yves.david@fb.com> parents: 20801 diff changeset	393 """unpack this struct format from the stream"""
520df53ad26a bundle2: a very first version of bundle2 unbundler Pierre-Yves David <pierre-yves.david@fb.com> parents: 20801 diff changeset	394 data = self._readexact(struct.calcsize(format))
520df53ad26a bundle2: a very first version of bundle2 unbundler Pierre-Yves David <pierre-yves.david@fb.com> parents: 20801 diff changeset	395 return _unpack(format, data)
520df53ad26a bundle2: a very first version of bundle2 unbundler Pierre-Yves David <pierre-yves.david@fb.com> parents: 20801 diff changeset	396
520df53ad26a bundle2: a very first version of bundle2 unbundler Pierre-Yves David <pierre-yves.david@fb.com> parents: 20801 diff changeset	397 def _readexact(self, size):
520df53ad26a bundle2: a very first version of bundle2 unbundler Pierre-Yves David <pierre-yves.david@fb.com> parents: 20801 diff changeset	398 """read exactly <size> bytes from the stream"""
520df53ad26a bundle2: a very first version of bundle2 unbundler Pierre-Yves David <pierre-yves.david@fb.com> parents: 20801 diff changeset	399 return changegroup.readexactly(self._fp, size)
520df53ad26a bundle2: a very first version of bundle2 unbundler Pierre-Yves David <pierre-yves.david@fb.com> parents: 20801 diff changeset	400
520df53ad26a bundle2: a very first version of bundle2 unbundler Pierre-Yves David <pierre-yves.david@fb.com> parents: 20801 diff changeset	401 @util.propertycache
520df53ad26a bundle2: a very first version of bundle2 unbundler Pierre-Yves David <pierre-yves.david@fb.com> parents: 20801 diff changeset	402 def params(self):
520df53ad26a bundle2: a very first version of bundle2 unbundler Pierre-Yves David <pierre-yves.david@fb.com> parents: 20801 diff changeset	403 """dictionnary of stream level parameters"""
20843 0641b41b0b49 bundle2: print debug information during unbundling Pierre-Yves David <pierre-yves.david@fb.com> parents: 20842 diff changeset	404 self.ui.debug('reading bundle2 stream parameters\n')
20805 c5aaeca0cfbf bundle2: support for unbundling simple parameter Pierre-Yves David <pierre-yves.david@fb.com> parents: 20804 diff changeset	405 params = {}
c5aaeca0cfbf bundle2: support for unbundling simple parameter Pierre-Yves David <pierre-yves.david@fb.com> parents: 20804 diff changeset	406 paramssize = self._unpack(_fstreamparamsize)[0]
c5aaeca0cfbf bundle2: support for unbundling simple parameter Pierre-Yves David <pierre-yves.david@fb.com> parents: 20804 diff changeset	407 if paramssize:
c5aaeca0cfbf bundle2: support for unbundling simple parameter Pierre-Yves David <pierre-yves.david@fb.com> parents: 20804 diff changeset	408 for p in self._readexact(paramssize).split(' '):
20810 47293877b54c bundle2: support for unbundling parameter value Pierre-Yves David <pierre-yves.david@fb.com> parents: 20809 diff changeset	409 p = p.split('=', 1)
20812 e2f908773754 bundle2: urlunquote stream parameter name and value during unbundling Pierre-Yves David <pierre-yves.david@fb.com> parents: 20811 diff changeset	410 p = [urllib.unquote(i) for i in p]
20810 47293877b54c bundle2: support for unbundling parameter value Pierre-Yves David <pierre-yves.david@fb.com> parents: 20809 diff changeset	411 if len(p) < 2:
47293877b54c bundle2: support for unbundling parameter value Pierre-Yves David <pierre-yves.david@fb.com> parents: 20809 diff changeset	412 p.append(None)
20844 2631204d7305 bundle2: implement the mandatory/advisory logic for parameter Pierre-Yves David <pierre-yves.david@fb.com> parents: 20843 diff changeset	413 self._processparam(*p)
20810 47293877b54c bundle2: support for unbundling parameter value Pierre-Yves David <pierre-yves.david@fb.com> parents: 20809 diff changeset	414 params[p[0]] = p[1]
20805 c5aaeca0cfbf bundle2: support for unbundling simple parameter Pierre-Yves David <pierre-yves.david@fb.com> parents: 20804 diff changeset	415 return params
20802 520df53ad26a bundle2: a very first version of bundle2 unbundler Pierre-Yves David <pierre-yves.david@fb.com> parents: 20801 diff changeset	416
20844 2631204d7305 bundle2: implement the mandatory/advisory logic for parameter Pierre-Yves David <pierre-yves.david@fb.com> parents: 20843 diff changeset	417 def _processparam(self, name, value):
2631204d7305 bundle2: implement the mandatory/advisory logic for parameter Pierre-Yves David <pierre-yves.david@fb.com> parents: 20843 diff changeset	418 """process a parameter, applying its effect if needed
2631204d7305 bundle2: implement the mandatory/advisory logic for parameter Pierre-Yves David <pierre-yves.david@fb.com> parents: 20843 diff changeset	419
2631204d7305 bundle2: implement the mandatory/advisory logic for parameter Pierre-Yves David <pierre-yves.david@fb.com> parents: 20843 diff changeset	420 Parameter starting with a lower case letter are advisory and will be
2631204d7305 bundle2: implement the mandatory/advisory logic for parameter Pierre-Yves David <pierre-yves.david@fb.com> parents: 20843 diff changeset	421 ignored when unknown. Those starting with an upper case letter are
2631204d7305 bundle2: implement the mandatory/advisory logic for parameter Pierre-Yves David <pierre-yves.david@fb.com> parents: 20843 diff changeset	422 mandatory and will this function will raise a KeyError when unknown.
2631204d7305 bundle2: implement the mandatory/advisory logic for parameter Pierre-Yves David <pierre-yves.david@fb.com> parents: 20843 diff changeset	423
2631204d7305 bundle2: implement the mandatory/advisory logic for parameter Pierre-Yves David <pierre-yves.david@fb.com> parents: 20843 diff changeset	424 Note: no option are currently supported. Any input will be either
2631204d7305 bundle2: implement the mandatory/advisory logic for parameter Pierre-Yves David <pierre-yves.david@fb.com> parents: 20843 diff changeset	425 ignored or failing.
2631204d7305 bundle2: implement the mandatory/advisory logic for parameter Pierre-Yves David <pierre-yves.david@fb.com> parents: 20843 diff changeset	426 """
2631204d7305 bundle2: implement the mandatory/advisory logic for parameter Pierre-Yves David <pierre-yves.david@fb.com> parents: 20843 diff changeset	427 if not name:
2631204d7305 bundle2: implement the mandatory/advisory logic for parameter Pierre-Yves David <pierre-yves.david@fb.com> parents: 20843 diff changeset	428 raise ValueError('empty parameter name')
2631204d7305 bundle2: implement the mandatory/advisory logic for parameter Pierre-Yves David <pierre-yves.david@fb.com> parents: 20843 diff changeset	429 if name[0] not in string.letters:
2631204d7305 bundle2: implement the mandatory/advisory logic for parameter Pierre-Yves David <pierre-yves.david@fb.com> parents: 20843 diff changeset	430 raise ValueError('non letter first character: %r' % name)
2631204d7305 bundle2: implement the mandatory/advisory logic for parameter Pierre-Yves David <pierre-yves.david@fb.com> parents: 20843 diff changeset	431 # Some logic will be later added here to try to process the option for
2631204d7305 bundle2: implement the mandatory/advisory logic for parameter Pierre-Yves David <pierre-yves.david@fb.com> parents: 20843 diff changeset	432 # a dict of known parameter.
2631204d7305 bundle2: implement the mandatory/advisory logic for parameter Pierre-Yves David <pierre-yves.david@fb.com> parents: 20843 diff changeset	433 if name[0].islower():
2631204d7305 bundle2: implement the mandatory/advisory logic for parameter Pierre-Yves David <pierre-yves.david@fb.com> parents: 20843 diff changeset	434 self.ui.debug("ignoring unknown parameter %r\n" % name)
2631204d7305 bundle2: implement the mandatory/advisory logic for parameter Pierre-Yves David <pierre-yves.david@fb.com> parents: 20843 diff changeset	435 else:
2631204d7305 bundle2: implement the mandatory/advisory logic for parameter Pierre-Yves David <pierre-yves.david@fb.com> parents: 20843 diff changeset	436 raise KeyError(name)
2631204d7305 bundle2: implement the mandatory/advisory logic for parameter Pierre-Yves David <pierre-yves.david@fb.com> parents: 20843 diff changeset	437
2631204d7305 bundle2: implement the mandatory/advisory logic for parameter Pierre-Yves David <pierre-yves.david@fb.com> parents: 20843 diff changeset	438
20802 520df53ad26a bundle2: a very first version of bundle2 unbundler Pierre-Yves David <pierre-yves.david@fb.com> parents: 20801 diff changeset	439 def __iter__(self):
520df53ad26a bundle2: a very first version of bundle2 unbundler Pierre-Yves David <pierre-yves.david@fb.com> parents: 20801 diff changeset	440 """yield all parts contained in the stream"""
520df53ad26a bundle2: a very first version of bundle2 unbundler Pierre-Yves David <pierre-yves.david@fb.com> parents: 20801 diff changeset	441 # make sure param have been loaded
520df53ad26a bundle2: a very first version of bundle2 unbundler Pierre-Yves David <pierre-yves.david@fb.com> parents: 20801 diff changeset	442 self.params
20843 0641b41b0b49 bundle2: print debug information during unbundling Pierre-Yves David <pierre-yves.david@fb.com> parents: 20842 diff changeset	443 self.ui.debug('start extraction of bundle2 parts\n')
20802 520df53ad26a bundle2: a very first version of bundle2 unbundler Pierre-Yves David <pierre-yves.david@fb.com> parents: 20801 diff changeset	444 part = self._readpart()
520df53ad26a bundle2: a very first version of bundle2 unbundler Pierre-Yves David <pierre-yves.david@fb.com> parents: 20801 diff changeset	445 while part is not None:
520df53ad26a bundle2: a very first version of bundle2 unbundler Pierre-Yves David <pierre-yves.david@fb.com> parents: 20801 diff changeset	446 yield part
520df53ad26a bundle2: a very first version of bundle2 unbundler Pierre-Yves David <pierre-yves.david@fb.com> parents: 20801 diff changeset	447 part = self._readpart()
20843 0641b41b0b49 bundle2: print debug information during unbundling Pierre-Yves David <pierre-yves.david@fb.com> parents: 20842 diff changeset	448 self.ui.debug('end of bundle2 stream\n')
20802 520df53ad26a bundle2: a very first version of bundle2 unbundler Pierre-Yves David <pierre-yves.david@fb.com> parents: 20801 diff changeset	449
520df53ad26a bundle2: a very first version of bundle2 unbundler Pierre-Yves David <pierre-yves.david@fb.com> parents: 20801 diff changeset	450 def _readpart(self):
520df53ad26a bundle2: a very first version of bundle2 unbundler Pierre-Yves David <pierre-yves.david@fb.com> parents: 20801 diff changeset	451 """return None when an end of stream markers is reach"""
20864 9a75d2559cff bundle2: support unbundling empty part Pierre-Yves David <pierre-yves.david@fb.com> parents: 20856 diff changeset	452
9a75d2559cff bundle2: support unbundling empty part Pierre-Yves David <pierre-yves.david@fb.com> parents: 20856 diff changeset	453 headersize = self._unpack(_fpartheadersize)[0]
9a75d2559cff bundle2: support unbundling empty part Pierre-Yves David <pierre-yves.david@fb.com> parents: 20856 diff changeset	454 self.ui.debug('part header size: %i\n' % headersize)
9a75d2559cff bundle2: support unbundling empty part Pierre-Yves David <pierre-yves.david@fb.com> parents: 20856 diff changeset	455 if not headersize:
9a75d2559cff bundle2: support unbundling empty part Pierre-Yves David <pierre-yves.david@fb.com> parents: 20856 diff changeset	456 return None
9a75d2559cff bundle2: support unbundling empty part Pierre-Yves David <pierre-yves.david@fb.com> parents: 20856 diff changeset	457 headerblock = self._readexact(headersize)
9a75d2559cff bundle2: support unbundling empty part Pierre-Yves David <pierre-yves.david@fb.com> parents: 20856 diff changeset	458 # some utility to help reading from the header block
9a75d2559cff bundle2: support unbundling empty part Pierre-Yves David <pierre-yves.david@fb.com> parents: 20856 diff changeset	459 self._offset = 0 # layer violation to have something easy to understand
9a75d2559cff bundle2: support unbundling empty part Pierre-Yves David <pierre-yves.david@fb.com> parents: 20856 diff changeset	460 def fromheader(size):
9a75d2559cff bundle2: support unbundling empty part Pierre-Yves David <pierre-yves.david@fb.com> parents: 20856 diff changeset	461 """return the next <size> byte from the header"""
9a75d2559cff bundle2: support unbundling empty part Pierre-Yves David <pierre-yves.david@fb.com> parents: 20856 diff changeset	462 offset = self._offset
9a75d2559cff bundle2: support unbundling empty part Pierre-Yves David <pierre-yves.david@fb.com> parents: 20856 diff changeset	463 data = headerblock[offset:(offset + size)]
9a75d2559cff bundle2: support unbundling empty part Pierre-Yves David <pierre-yves.david@fb.com> parents: 20856 diff changeset	464 self._offset = offset + size
9a75d2559cff bundle2: support unbundling empty part Pierre-Yves David <pierre-yves.david@fb.com> parents: 20856 diff changeset	465 return data
20887 662b79be093c bundle2: safely read unpack data from part header Pierre-Yves David <pierre-yves.david@fb.com> parents: 20877 diff changeset	466 def unpackheader(format):
662b79be093c bundle2: safely read unpack data from part header Pierre-Yves David <pierre-yves.david@fb.com> parents: 20877 diff changeset	467 """read given format from header
662b79be093c bundle2: safely read unpack data from part header Pierre-Yves David <pierre-yves.david@fb.com> parents: 20877 diff changeset	468
662b79be093c bundle2: safely read unpack data from part header Pierre-Yves David <pierre-yves.david@fb.com> parents: 20877 diff changeset	469 This automatically compute the size of the format to read."""
662b79be093c bundle2: safely read unpack data from part header Pierre-Yves David <pierre-yves.david@fb.com> parents: 20877 diff changeset	470 data = fromheader(struct.calcsize(format))
662b79be093c bundle2: safely read unpack data from part header Pierre-Yves David <pierre-yves.david@fb.com> parents: 20877 diff changeset	471 return _unpack(format, data)
662b79be093c bundle2: safely read unpack data from part header Pierre-Yves David <pierre-yves.david@fb.com> parents: 20877 diff changeset	472
662b79be093c bundle2: safely read unpack data from part header Pierre-Yves David <pierre-yves.david@fb.com> parents: 20877 diff changeset	473 typesize = unpackheader(_fparttypesize)[0]
20864 9a75d2559cff bundle2: support unbundling empty part Pierre-Yves David <pierre-yves.david@fb.com> parents: 20856 diff changeset	474 parttype = fromheader(typesize)
9a75d2559cff bundle2: support unbundling empty part Pierre-Yves David <pierre-yves.david@fb.com> parents: 20856 diff changeset	475 self.ui.debug('part type: "%s"\n' % parttype)
20995 e995d104c87f bundle2: add an integer id to part Pierre-Yves David <pierre-yves.david@fb.com> parents: 20952 diff changeset	476 partid = unpackheader(_fpartid)[0]
e995d104c87f bundle2: add an integer id to part Pierre-Yves David <pierre-yves.david@fb.com> parents: 20952 diff changeset	477 self.ui.debug('part id: "%s"\n' % partid)
20877 9e9e3a4e9261 bundle2: part params Pierre-Yves David <pierre-yves.david@fb.com> parents: 20876 diff changeset	478 ## reading parameters
9e9e3a4e9261 bundle2: part params Pierre-Yves David <pierre-yves.david@fb.com> parents: 20876 diff changeset	479 # param count
20887 662b79be093c bundle2: safely read unpack data from part header Pierre-Yves David <pierre-yves.david@fb.com> parents: 20877 diff changeset	480 mancount, advcount = unpackheader(_fpartparamcount)
20877 9e9e3a4e9261 bundle2: part params Pierre-Yves David <pierre-yves.david@fb.com> parents: 20876 diff changeset	481 self.ui.debug('part parameters: %i\n' % (mancount + advcount))
9e9e3a4e9261 bundle2: part params Pierre-Yves David <pierre-yves.david@fb.com> parents: 20876 diff changeset	482 # param size
20887 662b79be093c bundle2: safely read unpack data from part header Pierre-Yves David <pierre-yves.david@fb.com> parents: 20877 diff changeset	483 paramsizes = unpackheader(_makefpartparamsizes(mancount + advcount))
20877 9e9e3a4e9261 bundle2: part params Pierre-Yves David <pierre-yves.david@fb.com> parents: 20876 diff changeset	484 # make it a list of couple again
9e9e3a4e9261 bundle2: part params Pierre-Yves David <pierre-yves.david@fb.com> parents: 20876 diff changeset	485 paramsizes = zip(paramsizes[::2], paramsizes[1::2])
9e9e3a4e9261 bundle2: part params Pierre-Yves David <pierre-yves.david@fb.com> parents: 20876 diff changeset	486 # split mandatory from advisory
9e9e3a4e9261 bundle2: part params Pierre-Yves David <pierre-yves.david@fb.com> parents: 20876 diff changeset	487 mansizes = paramsizes[:mancount]
9e9e3a4e9261 bundle2: part params Pierre-Yves David <pierre-yves.david@fb.com> parents: 20876 diff changeset	488 advsizes = paramsizes[mancount:]
9e9e3a4e9261 bundle2: part params Pierre-Yves David <pierre-yves.david@fb.com> parents: 20876 diff changeset	489 # retrive param value
9e9e3a4e9261 bundle2: part params Pierre-Yves David <pierre-yves.david@fb.com> parents: 20876 diff changeset	490 manparams = []
9e9e3a4e9261 bundle2: part params Pierre-Yves David <pierre-yves.david@fb.com> parents: 20876 diff changeset	491 for key, value in mansizes:
9e9e3a4e9261 bundle2: part params Pierre-Yves David <pierre-yves.david@fb.com> parents: 20876 diff changeset	492 manparams.append((fromheader(key), fromheader(value)))
9e9e3a4e9261 bundle2: part params Pierre-Yves David <pierre-yves.david@fb.com> parents: 20876 diff changeset	493 advparams = []
9e9e3a4e9261 bundle2: part params Pierre-Yves David <pierre-yves.david@fb.com> parents: 20876 diff changeset	494 for key, value in advsizes:
9e9e3a4e9261 bundle2: part params Pierre-Yves David <pierre-yves.david@fb.com> parents: 20876 diff changeset	495 advparams.append((fromheader(key), fromheader(value)))
20864 9a75d2559cff bundle2: support unbundling empty part Pierre-Yves David <pierre-yves.david@fb.com> parents: 20856 diff changeset	496 del self._offset # clean up layer, nobody saw anything.
20877 9e9e3a4e9261 bundle2: part params Pierre-Yves David <pierre-yves.david@fb.com> parents: 20876 diff changeset	497 ## part payload
20876 ddd56f3eb786 bundle2: support for bundling and unbundling payload Pierre-Yves David <pierre-yves.david@fb.com> parents: 20864 diff changeset	498 payload = []
ddd56f3eb786 bundle2: support for bundling and unbundling payload Pierre-Yves David <pierre-yves.david@fb.com> parents: 20864 diff changeset	499 payloadsize = self._unpack(_fpayloadsize)[0]
ddd56f3eb786 bundle2: support for bundling and unbundling payload Pierre-Yves David <pierre-yves.david@fb.com> parents: 20864 diff changeset	500 self.ui.debug('payload chunk size: %i\n' % payloadsize)
ddd56f3eb786 bundle2: support for bundling and unbundling payload Pierre-Yves David <pierre-yves.david@fb.com> parents: 20864 diff changeset	501 while payloadsize:
ddd56f3eb786 bundle2: support for bundling and unbundling payload Pierre-Yves David <pierre-yves.david@fb.com> parents: 20864 diff changeset	502 payload.append(self._readexact(payloadsize))
ddd56f3eb786 bundle2: support for bundling and unbundling payload Pierre-Yves David <pierre-yves.david@fb.com> parents: 20864 diff changeset	503 payloadsize = self._unpack(_fpayloadsize)[0]
ddd56f3eb786 bundle2: support for bundling and unbundling payload Pierre-Yves David <pierre-yves.david@fb.com> parents: 20864 diff changeset	504 self.ui.debug('payload chunk size: %i\n' % payloadsize)
ddd56f3eb786 bundle2: support for bundling and unbundling payload Pierre-Yves David <pierre-yves.david@fb.com> parents: 20864 diff changeset	505 payload = ''.join(payload)
21005 3d38ebb586fe bundle2: rename part to bundlepart Pierre-Yves David <pierre-yves.david@fb.com> parents: 21004 diff changeset	506 current = bundlepart(parttype, manparams, advparams, data=payload)
20995 e995d104c87f bundle2: add an integer id to part Pierre-Yves David <pierre-yves.david@fb.com> parents: 20952 diff changeset	507 current.id = partid
20864 9a75d2559cff bundle2: support unbundling empty part Pierre-Yves David <pierre-yves.david@fb.com> parents: 20856 diff changeset	508 return current
9a75d2559cff bundle2: support unbundling empty part Pierre-Yves David <pierre-yves.david@fb.com> parents: 20856 diff changeset	509
20802 520df53ad26a bundle2: a very first version of bundle2 unbundler Pierre-Yves David <pierre-yves.david@fb.com> parents: 20801 diff changeset	510
21005 3d38ebb586fe bundle2: rename part to bundlepart Pierre-Yves David <pierre-yves.david@fb.com> parents: 21004 diff changeset	511 class bundlepart(object):
20856 8a6a86c9a5b5 bundle2: support bundling of empty part (with a type) Pierre-Yves David <pierre-yves.david@fb.com> parents: 20844 diff changeset	512 """A bundle2 part contains application level payload
8a6a86c9a5b5 bundle2: support bundling of empty part (with a type) Pierre-Yves David <pierre-yves.david@fb.com> parents: 20844 diff changeset	513
8a6a86c9a5b5 bundle2: support bundling of empty part (with a type) Pierre-Yves David <pierre-yves.david@fb.com> parents: 20844 diff changeset	514 The part `type` is used to route the part to the application level
8a6a86c9a5b5 bundle2: support bundling of empty part (with a type) Pierre-Yves David <pierre-yves.david@fb.com> parents: 20844 diff changeset	515 handler.
8a6a86c9a5b5 bundle2: support bundling of empty part (with a type) Pierre-Yves David <pierre-yves.david@fb.com> parents: 20844 diff changeset	516 """
8a6a86c9a5b5 bundle2: support bundling of empty part (with a type) Pierre-Yves David <pierre-yves.david@fb.com> parents: 20844 diff changeset	517
20877 9e9e3a4e9261 bundle2: part params Pierre-Yves David <pierre-yves.david@fb.com> parents: 20876 diff changeset	518 def __init__(self, parttype, mandatoryparams=(), advisoryparams=(),
9e9e3a4e9261 bundle2: part params Pierre-Yves David <pierre-yves.david@fb.com> parents: 20876 diff changeset	519 data=''):
20995 e995d104c87f bundle2: add an integer id to part Pierre-Yves David <pierre-yves.david@fb.com> parents: 20952 diff changeset	520 self.id = None
20856 8a6a86c9a5b5 bundle2: support bundling of empty part (with a type) Pierre-Yves David <pierre-yves.david@fb.com> parents: 20844 diff changeset	521 self.type = parttype
20864 9a75d2559cff bundle2: support unbundling empty part Pierre-Yves David <pierre-yves.david@fb.com> parents: 20856 diff changeset	522 self.data = data
20877 9e9e3a4e9261 bundle2: part params Pierre-Yves David <pierre-yves.david@fb.com> parents: 20876 diff changeset	523 self.mandatoryparams = mandatoryparams
9e9e3a4e9261 bundle2: part params Pierre-Yves David <pierre-yves.david@fb.com> parents: 20876 diff changeset	524 self.advisoryparams = advisoryparams
20856 8a6a86c9a5b5 bundle2: support bundling of empty part (with a type) Pierre-Yves David <pierre-yves.david@fb.com> parents: 20844 diff changeset	525
8a6a86c9a5b5 bundle2: support bundling of empty part (with a type) Pierre-Yves David <pierre-yves.david@fb.com> parents: 20844 diff changeset	526 def getchunks(self):
20877 9e9e3a4e9261 bundle2: part params Pierre-Yves David <pierre-yves.david@fb.com> parents: 20876 diff changeset	527 #### header
9e9e3a4e9261 bundle2: part params Pierre-Yves David <pierre-yves.david@fb.com> parents: 20876 diff changeset	528 ## parttype
20856 8a6a86c9a5b5 bundle2: support bundling of empty part (with a type) Pierre-Yves David <pierre-yves.david@fb.com> parents: 20844 diff changeset	529 header = [_pack(_fparttypesize, len(self.type)),
20995 e995d104c87f bundle2: add an integer id to part Pierre-Yves David <pierre-yves.david@fb.com> parents: 20952 diff changeset	530 self.type, _pack(_fpartid, self.id),
20856 8a6a86c9a5b5 bundle2: support bundling of empty part (with a type) Pierre-Yves David <pierre-yves.david@fb.com> parents: 20844 diff changeset	531 ]
20877 9e9e3a4e9261 bundle2: part params Pierre-Yves David <pierre-yves.david@fb.com> parents: 20876 diff changeset	532 ## parameters
9e9e3a4e9261 bundle2: part params Pierre-Yves David <pierre-yves.david@fb.com> parents: 20876 diff changeset	533 # count
9e9e3a4e9261 bundle2: part params Pierre-Yves David <pierre-yves.david@fb.com> parents: 20876 diff changeset	534 manpar = self.mandatoryparams
9e9e3a4e9261 bundle2: part params Pierre-Yves David <pierre-yves.david@fb.com> parents: 20876 diff changeset	535 advpar = self.advisoryparams
9e9e3a4e9261 bundle2: part params Pierre-Yves David <pierre-yves.david@fb.com> parents: 20876 diff changeset	536 header.append(_pack(_fpartparamcount, len(manpar), len(advpar)))
9e9e3a4e9261 bundle2: part params Pierre-Yves David <pierre-yves.david@fb.com> parents: 20876 diff changeset	537 # size
9e9e3a4e9261 bundle2: part params Pierre-Yves David <pierre-yves.david@fb.com> parents: 20876 diff changeset	538 parsizes = []
9e9e3a4e9261 bundle2: part params Pierre-Yves David <pierre-yves.david@fb.com> parents: 20876 diff changeset	539 for key, value in manpar:
9e9e3a4e9261 bundle2: part params Pierre-Yves David <pierre-yves.david@fb.com> parents: 20876 diff changeset	540 parsizes.append(len(key))
9e9e3a4e9261 bundle2: part params Pierre-Yves David <pierre-yves.david@fb.com> parents: 20876 diff changeset	541 parsizes.append(len(value))
9e9e3a4e9261 bundle2: part params Pierre-Yves David <pierre-yves.david@fb.com> parents: 20876 diff changeset	542 for key, value in advpar:
9e9e3a4e9261 bundle2: part params Pierre-Yves David <pierre-yves.david@fb.com> parents: 20876 diff changeset	543 parsizes.append(len(key))
9e9e3a4e9261 bundle2: part params Pierre-Yves David <pierre-yves.david@fb.com> parents: 20876 diff changeset	544 parsizes.append(len(value))
9e9e3a4e9261 bundle2: part params Pierre-Yves David <pierre-yves.david@fb.com> parents: 20876 diff changeset	545 paramsizes = _pack(_makefpartparamsizes(len(parsizes) / 2), *parsizes)
9e9e3a4e9261 bundle2: part params Pierre-Yves David <pierre-yves.david@fb.com> parents: 20876 diff changeset	546 header.append(paramsizes)
9e9e3a4e9261 bundle2: part params Pierre-Yves David <pierre-yves.david@fb.com> parents: 20876 diff changeset	547 # key, value
9e9e3a4e9261 bundle2: part params Pierre-Yves David <pierre-yves.david@fb.com> parents: 20876 diff changeset	548 for key, value in manpar:
9e9e3a4e9261 bundle2: part params Pierre-Yves David <pierre-yves.david@fb.com> parents: 20876 diff changeset	549 header.append(key)
9e9e3a4e9261 bundle2: part params Pierre-Yves David <pierre-yves.david@fb.com> parents: 20876 diff changeset	550 header.append(value)
9e9e3a4e9261 bundle2: part params Pierre-Yves David <pierre-yves.david@fb.com> parents: 20876 diff changeset	551 for key, value in advpar:
9e9e3a4e9261 bundle2: part params Pierre-Yves David <pierre-yves.david@fb.com> parents: 20876 diff changeset	552 header.append(key)
9e9e3a4e9261 bundle2: part params Pierre-Yves David <pierre-yves.david@fb.com> parents: 20876 diff changeset	553 header.append(value)
9e9e3a4e9261 bundle2: part params Pierre-Yves David <pierre-yves.david@fb.com> parents: 20876 diff changeset	554 ## finalize header
20856 8a6a86c9a5b5 bundle2: support bundling of empty part (with a type) Pierre-Yves David <pierre-yves.david@fb.com> parents: 20844 diff changeset	555 headerchunk = ''.join(header)
8a6a86c9a5b5 bundle2: support bundling of empty part (with a type) Pierre-Yves David <pierre-yves.david@fb.com> parents: 20844 diff changeset	556 yield _pack(_fpartheadersize, len(headerchunk))
8a6a86c9a5b5 bundle2: support bundling of empty part (with a type) Pierre-Yves David <pierre-yves.david@fb.com> parents: 20844 diff changeset	557 yield headerchunk
20877 9e9e3a4e9261 bundle2: part params Pierre-Yves David <pierre-yves.david@fb.com> parents: 20876 diff changeset	558 ## payload
21000 4cae06ae1562 bundle2: extract a _payloadchunks method for part Pierre-Yves David <pierre-yves.david@fb.com> parents: 20998 diff changeset	559 for chunk in self._payloadchunks():
4cae06ae1562 bundle2: extract a _payloadchunks method for part Pierre-Yves David <pierre-yves.david@fb.com> parents: 20998 diff changeset	560 yield _pack(_fpayloadsize, len(chunk))
4cae06ae1562 bundle2: extract a _payloadchunks method for part Pierre-Yves David <pierre-yves.david@fb.com> parents: 20998 diff changeset	561 yield chunk
4cae06ae1562 bundle2: extract a _payloadchunks method for part Pierre-Yves David <pierre-yves.david@fb.com> parents: 20998 diff changeset	562 # end of payload
4cae06ae1562 bundle2: extract a _payloadchunks method for part Pierre-Yves David <pierre-yves.david@fb.com> parents: 20998 diff changeset	563 yield _pack(_fpayloadsize, 0)
4cae06ae1562 bundle2: extract a _payloadchunks method for part Pierre-Yves David <pierre-yves.david@fb.com> parents: 20998 diff changeset	564
4cae06ae1562 bundle2: extract a _payloadchunks method for part Pierre-Yves David <pierre-yves.david@fb.com> parents: 20998 diff changeset	565 def _payloadchunks(self):
4cae06ae1562 bundle2: extract a _payloadchunks method for part Pierre-Yves David <pierre-yves.david@fb.com> parents: 20998 diff changeset	566 """yield chunks of a the part payload
4cae06ae1562 bundle2: extract a _payloadchunks method for part Pierre-Yves David <pierre-yves.david@fb.com> parents: 20998 diff changeset	567
4cae06ae1562 bundle2: extract a _payloadchunks method for part Pierre-Yves David <pierre-yves.david@fb.com> parents: 20998 diff changeset	568 Exists to handle the different methods to provide data to a part."""
20876 ddd56f3eb786 bundle2: support for bundling and unbundling payload Pierre-Yves David <pierre-yves.david@fb.com> parents: 20864 diff changeset	569 # we only support fixed size data now.
ddd56f3eb786 bundle2: support for bundling and unbundling payload Pierre-Yves David <pierre-yves.david@fb.com> parents: 20864 diff changeset	570 # This will be improved in the future.
21001 c93bb6a08fa1 bundle2: support chunk iterator as part data Pierre-Yves David <pierre-yves.david@fb.com> parents: 21000 diff changeset	571 if util.safehasattr(self.data, 'next'):
c93bb6a08fa1 bundle2: support chunk iterator as part data Pierre-Yves David <pierre-yves.david@fb.com> parents: 21000 diff changeset	572 buff = util.chunkbuffer(self.data)
c93bb6a08fa1 bundle2: support chunk iterator as part data Pierre-Yves David <pierre-yves.david@fb.com> parents: 21000 diff changeset	573 chunk = buff.read(preferedchunksize)
c93bb6a08fa1 bundle2: support chunk iterator as part data Pierre-Yves David <pierre-yves.david@fb.com> parents: 21000 diff changeset	574 while chunk:
c93bb6a08fa1 bundle2: support chunk iterator as part data Pierre-Yves David <pierre-yves.david@fb.com> parents: 21000 diff changeset	575 yield chunk
c93bb6a08fa1 bundle2: support chunk iterator as part data Pierre-Yves David <pierre-yves.david@fb.com> parents: 21000 diff changeset	576 chunk = buff.read(preferedchunksize)
c93bb6a08fa1 bundle2: support chunk iterator as part data Pierre-Yves David <pierre-yves.david@fb.com> parents: 21000 diff changeset	577 elif len(self.data):
20876 ddd56f3eb786 bundle2: support for bundling and unbundling payload Pierre-Yves David <pierre-yves.david@fb.com> parents: 20864 diff changeset	578 yield self.data
20802 520df53ad26a bundle2: a very first version of bundle2 unbundler Pierre-Yves David <pierre-yves.david@fb.com> parents: 20801 diff changeset	579
20950 c7ceae0faf69 bundle2: first crude version of bundling changeset with bundle2 Pierre-Yves David <pierre-yves.david@fb.com> parents: 20949 diff changeset	580 @parthandler('changegroup')
20998 93a3c5b58635 bundle2: use reply part to return result of addchangegroup Pierre-Yves David <pierre-yves.david@fb.com> parents: 20997 diff changeset	581 def handlechangegroup(op, inpart):
20950 c7ceae0faf69 bundle2: first crude version of bundling changeset with bundle2 Pierre-Yves David <pierre-yves.david@fb.com> parents: 20949 diff changeset	582 """apply a changegroup part on the repo
c7ceae0faf69 bundle2: first crude version of bundling changeset with bundle2 Pierre-Yves David <pierre-yves.david@fb.com> parents: 20949 diff changeset	583
c7ceae0faf69 bundle2: first crude version of bundling changeset with bundle2 Pierre-Yves David <pierre-yves.david@fb.com> parents: 20949 diff changeset	584 This is a very early implementation that will massive rework before being
c7ceae0faf69 bundle2: first crude version of bundling changeset with bundle2 Pierre-Yves David <pierre-yves.david@fb.com> parents: 20949 diff changeset	585 inflicted to any end-user.
c7ceae0faf69 bundle2: first crude version of bundling changeset with bundle2 Pierre-Yves David <pierre-yves.david@fb.com> parents: 20949 diff changeset	586 """
20952 b24ee5076b94 bundle2: make it possible have a global transaction for the unbundling Pierre-Yves David <pierre-yves.david@fb.com> parents: 20950 diff changeset	587 # Make sure we trigger a transaction creation
b24ee5076b94 bundle2: make it possible have a global transaction for the unbundling Pierre-Yves David <pierre-yves.david@fb.com> parents: 20950 diff changeset	588 #
b24ee5076b94 bundle2: make it possible have a global transaction for the unbundling Pierre-Yves David <pierre-yves.david@fb.com> parents: 20950 diff changeset	589 # The addchangegroup function will get a transaction object by itself, but
b24ee5076b94 bundle2: make it possible have a global transaction for the unbundling Pierre-Yves David <pierre-yves.david@fb.com> parents: 20950 diff changeset	590 # we need to make sure we trigger the creation of a transaction object used
b24ee5076b94 bundle2: make it possible have a global transaction for the unbundling Pierre-Yves David <pierre-yves.david@fb.com> parents: 20950 diff changeset	591 # for the whole processing scope.
b24ee5076b94 bundle2: make it possible have a global transaction for the unbundling Pierre-Yves David <pierre-yves.david@fb.com> parents: 20950 diff changeset	592 op.gettransaction()
20998 93a3c5b58635 bundle2: use reply part to return result of addchangegroup Pierre-Yves David <pierre-yves.david@fb.com> parents: 20997 diff changeset	593 data = StringIO.StringIO(inpart.data)
20950 c7ceae0faf69 bundle2: first crude version of bundling changeset with bundle2 Pierre-Yves David <pierre-yves.david@fb.com> parents: 20949 diff changeset	594 data.seek(0)
c7ceae0faf69 bundle2: first crude version of bundling changeset with bundle2 Pierre-Yves David <pierre-yves.david@fb.com> parents: 20949 diff changeset	595 cg = changegroup.readbundle(data, 'bundle2part')
c7ceae0faf69 bundle2: first crude version of bundling changeset with bundle2 Pierre-Yves David <pierre-yves.david@fb.com> parents: 20949 diff changeset	596 ret = changegroup.addchangegroup(op.repo, cg, 'bundle2', 'bundle2')
c7ceae0faf69 bundle2: first crude version of bundling changeset with bundle2 Pierre-Yves David <pierre-yves.david@fb.com> parents: 20949 diff changeset	597 op.records.add('changegroup', {'return': ret})
20998 93a3c5b58635 bundle2: use reply part to return result of addchangegroup Pierre-Yves David <pierre-yves.david@fb.com> parents: 20997 diff changeset	598 if op.reply is not None:
93a3c5b58635 bundle2: use reply part to return result of addchangegroup Pierre-Yves David <pierre-yves.david@fb.com> parents: 20997 diff changeset	599 # This is definitly not the final form of this
93a3c5b58635 bundle2: use reply part to return result of addchangegroup Pierre-Yves David <pierre-yves.david@fb.com> parents: 20997 diff changeset	600 # return. But one need to start somewhere.
21005 3d38ebb586fe bundle2: rename part to bundlepart Pierre-Yves David <pierre-yves.david@fb.com> parents: 21004 diff changeset	601 op.reply.addpart(bundlepart('reply:changegroup', (),
3d38ebb586fe bundle2: rename part to bundlepart Pierre-Yves David <pierre-yves.david@fb.com> parents: 21004 diff changeset	602 [('in-reply-to', str(inpart.id)),
3d38ebb586fe bundle2: rename part to bundlepart Pierre-Yves David <pierre-yves.david@fb.com> parents: 21004 diff changeset	603 ('return', '%i' % ret)]))
20950 c7ceae0faf69 bundle2: first crude version of bundling changeset with bundle2 Pierre-Yves David <pierre-yves.david@fb.com> parents: 20949 diff changeset	604
20998 93a3c5b58635 bundle2: use reply part to return result of addchangegroup Pierre-Yves David <pierre-yves.david@fb.com> parents: 20997 diff changeset	605 @parthandler('reply:changegroup')
93a3c5b58635 bundle2: use reply part to return result of addchangegroup Pierre-Yves David <pierre-yves.david@fb.com> parents: 20997 diff changeset	606 def handlechangegroup(op, inpart):
93a3c5b58635 bundle2: use reply part to return result of addchangegroup Pierre-Yves David <pierre-yves.david@fb.com> parents: 20997 diff changeset	607 p = dict(inpart.advisoryparams)
93a3c5b58635 bundle2: use reply part to return result of addchangegroup Pierre-Yves David <pierre-yves.david@fb.com> parents: 20997 diff changeset	608 ret = int(p['return'])
93a3c5b58635 bundle2: use reply part to return result of addchangegroup Pierre-Yves David <pierre-yves.david@fb.com> parents: 20997 diff changeset	609 op.records.add('changegroup', {'return': ret}, int(p['in-reply-to']))
20950 c7ceae0faf69 bundle2: first crude version of bundling changeset with bundle2 Pierre-Yves David <pierre-yves.david@fb.com> parents: 20949 diff changeset	610

Mercurial > public > mercurial-scm > hg-stable

annotate mercurial/bundle2.py @ 21005:3d38ebb586fe