mercurial/bundle2.py
author Pierre-Yves David <pierre-yves.david@fb.com>
Tue, 18 Mar 2014 14:00:50 -0700
changeset 20801 9c5183cb9bca
child 20802 520df53ad26a
permissions -rw-r--r--
bundle2: very first version of a bundle2 bundler This changeset is the very first of a long series. It create a new bundle2 module and add a simple class that generate and empty bundle2 container. The module is documented with the current state of the implementation. For information about the final goal you may want to consult the mercurial wiki page: http://mercurial.selenic.com/wiki/BundleFormat2 The documentation of the module will be updated with later patches adding more and more feature to the format. This patches also introduce a test case. This test case build and use its own small extension that use the new bundle2 module. Since the new format is unable to do anything right now, we could not use real mercurial code to test it. Moreover, some advanced feature of the bundle2 spec will not be used by core mercurial at all. So we need to have them tested.
Ignore whitespace changes - Everywhere: Within whitespace: At end of lines:
20801
9c5183cb9bca bundle2: very first version of a bundle2 bundler
Pierre-Yves David <pierre-yves.david@fb.com>
parents:
diff changeset
     1
# bundle2.py - generic container format to transmit arbitrary data.
9c5183cb9bca bundle2: very first version of a bundle2 bundler
Pierre-Yves David <pierre-yves.david@fb.com>
parents:
diff changeset
     2
#
9c5183cb9bca bundle2: very first version of a bundle2 bundler
Pierre-Yves David <pierre-yves.david@fb.com>
parents:
diff changeset
     3
# Copyright 2013 Facebook, Inc.
9c5183cb9bca bundle2: very first version of a bundle2 bundler
Pierre-Yves David <pierre-yves.david@fb.com>
parents:
diff changeset
     4
#
9c5183cb9bca bundle2: very first version of a bundle2 bundler
Pierre-Yves David <pierre-yves.david@fb.com>
parents:
diff changeset
     5
# This software may be used and distributed according to the terms of the
9c5183cb9bca bundle2: very first version of a bundle2 bundler
Pierre-Yves David <pierre-yves.david@fb.com>
parents:
diff changeset
     6
# GNU General Public License version 2 or any later version.
9c5183cb9bca bundle2: very first version of a bundle2 bundler
Pierre-Yves David <pierre-yves.david@fb.com>
parents:
diff changeset
     7
"""Handling of the new bundle2 format
9c5183cb9bca bundle2: very first version of a bundle2 bundler
Pierre-Yves David <pierre-yves.david@fb.com>
parents:
diff changeset
     8
9c5183cb9bca bundle2: very first version of a bundle2 bundler
Pierre-Yves David <pierre-yves.david@fb.com>
parents:
diff changeset
     9
The goal of bundle2 is to act as an atomically packet to transmit a set of
9c5183cb9bca bundle2: very first version of a bundle2 bundler
Pierre-Yves David <pierre-yves.david@fb.com>
parents:
diff changeset
    10
payloads in an application agnostic way. It consist in a sequence of "parts"
9c5183cb9bca bundle2: very first version of a bundle2 bundler
Pierre-Yves David <pierre-yves.david@fb.com>
parents:
diff changeset
    11
that will be handed to and processed by the application layer.
9c5183cb9bca bundle2: very first version of a bundle2 bundler
Pierre-Yves David <pierre-yves.david@fb.com>
parents:
diff changeset
    12
9c5183cb9bca bundle2: very first version of a bundle2 bundler
Pierre-Yves David <pierre-yves.david@fb.com>
parents:
diff changeset
    13
9c5183cb9bca bundle2: very first version of a bundle2 bundler
Pierre-Yves David <pierre-yves.david@fb.com>
parents:
diff changeset
    14
General format architecture
9c5183cb9bca bundle2: very first version of a bundle2 bundler
Pierre-Yves David <pierre-yves.david@fb.com>
parents:
diff changeset
    15
===========================
9c5183cb9bca bundle2: very first version of a bundle2 bundler
Pierre-Yves David <pierre-yves.david@fb.com>
parents:
diff changeset
    16
9c5183cb9bca bundle2: very first version of a bundle2 bundler
Pierre-Yves David <pierre-yves.david@fb.com>
parents:
diff changeset
    17
The format is architectured as follow
9c5183cb9bca bundle2: very first version of a bundle2 bundler
Pierre-Yves David <pierre-yves.david@fb.com>
parents:
diff changeset
    18
9c5183cb9bca bundle2: very first version of a bundle2 bundler
Pierre-Yves David <pierre-yves.david@fb.com>
parents:
diff changeset
    19
 - magic string
9c5183cb9bca bundle2: very first version of a bundle2 bundler
Pierre-Yves David <pierre-yves.david@fb.com>
parents:
diff changeset
    20
 - stream level parameters
9c5183cb9bca bundle2: very first version of a bundle2 bundler
Pierre-Yves David <pierre-yves.david@fb.com>
parents:
diff changeset
    21
 - payload parts (any number)
9c5183cb9bca bundle2: very first version of a bundle2 bundler
Pierre-Yves David <pierre-yves.david@fb.com>
parents:
diff changeset
    22
 - end of stream marker.
9c5183cb9bca bundle2: very first version of a bundle2 bundler
Pierre-Yves David <pierre-yves.david@fb.com>
parents:
diff changeset
    23
9c5183cb9bca bundle2: very first version of a bundle2 bundler
Pierre-Yves David <pierre-yves.david@fb.com>
parents:
diff changeset
    24
The current implementation is limited to empty bundle.
9c5183cb9bca bundle2: very first version of a bundle2 bundler
Pierre-Yves David <pierre-yves.david@fb.com>
parents:
diff changeset
    25
9c5183cb9bca bundle2: very first version of a bundle2 bundler
Pierre-Yves David <pierre-yves.david@fb.com>
parents:
diff changeset
    26
Details on the Binary format
9c5183cb9bca bundle2: very first version of a bundle2 bundler
Pierre-Yves David <pierre-yves.david@fb.com>
parents:
diff changeset
    27
============================
9c5183cb9bca bundle2: very first version of a bundle2 bundler
Pierre-Yves David <pierre-yves.david@fb.com>
parents:
diff changeset
    28
9c5183cb9bca bundle2: very first version of a bundle2 bundler
Pierre-Yves David <pierre-yves.david@fb.com>
parents:
diff changeset
    29
All numbers are unsigned and big endian.
9c5183cb9bca bundle2: very first version of a bundle2 bundler
Pierre-Yves David <pierre-yves.david@fb.com>
parents:
diff changeset
    30
9c5183cb9bca bundle2: very first version of a bundle2 bundler
Pierre-Yves David <pierre-yves.david@fb.com>
parents:
diff changeset
    31
stream level parameters
9c5183cb9bca bundle2: very first version of a bundle2 bundler
Pierre-Yves David <pierre-yves.david@fb.com>
parents:
diff changeset
    32
------------------------
9c5183cb9bca bundle2: very first version of a bundle2 bundler
Pierre-Yves David <pierre-yves.david@fb.com>
parents:
diff changeset
    33
9c5183cb9bca bundle2: very first version of a bundle2 bundler
Pierre-Yves David <pierre-yves.david@fb.com>
parents:
diff changeset
    34
Binary format is as follow
9c5183cb9bca bundle2: very first version of a bundle2 bundler
Pierre-Yves David <pierre-yves.david@fb.com>
parents:
diff changeset
    35
9c5183cb9bca bundle2: very first version of a bundle2 bundler
Pierre-Yves David <pierre-yves.david@fb.com>
parents:
diff changeset
    36
:params size: (16 bits integer)
9c5183cb9bca bundle2: very first version of a bundle2 bundler
Pierre-Yves David <pierre-yves.david@fb.com>
parents:
diff changeset
    37
9c5183cb9bca bundle2: very first version of a bundle2 bundler
Pierre-Yves David <pierre-yves.david@fb.com>
parents:
diff changeset
    38
  The total number of Bytes used by the parameters
9c5183cb9bca bundle2: very first version of a bundle2 bundler
Pierre-Yves David <pierre-yves.david@fb.com>
parents:
diff changeset
    39
9c5183cb9bca bundle2: very first version of a bundle2 bundler
Pierre-Yves David <pierre-yves.david@fb.com>
parents:
diff changeset
    40
  Currently force to 0.
9c5183cb9bca bundle2: very first version of a bundle2 bundler
Pierre-Yves David <pierre-yves.david@fb.com>
parents:
diff changeset
    41
9c5183cb9bca bundle2: very first version of a bundle2 bundler
Pierre-Yves David <pierre-yves.david@fb.com>
parents:
diff changeset
    42
:params value: arbitrary number of Bytes
9c5183cb9bca bundle2: very first version of a bundle2 bundler
Pierre-Yves David <pierre-yves.david@fb.com>
parents:
diff changeset
    43
9c5183cb9bca bundle2: very first version of a bundle2 bundler
Pierre-Yves David <pierre-yves.david@fb.com>
parents:
diff changeset
    44
  A blob of `params size` containing the serialized version of all stream level
9c5183cb9bca bundle2: very first version of a bundle2 bundler
Pierre-Yves David <pierre-yves.david@fb.com>
parents:
diff changeset
    45
  parameters.
9c5183cb9bca bundle2: very first version of a bundle2 bundler
Pierre-Yves David <pierre-yves.david@fb.com>
parents:
diff changeset
    46
9c5183cb9bca bundle2: very first version of a bundle2 bundler
Pierre-Yves David <pierre-yves.david@fb.com>
parents:
diff changeset
    47
  Currently always empty.
9c5183cb9bca bundle2: very first version of a bundle2 bundler
Pierre-Yves David <pierre-yves.david@fb.com>
parents:
diff changeset
    48
9c5183cb9bca bundle2: very first version of a bundle2 bundler
Pierre-Yves David <pierre-yves.david@fb.com>
parents:
diff changeset
    49
9c5183cb9bca bundle2: very first version of a bundle2 bundler
Pierre-Yves David <pierre-yves.david@fb.com>
parents:
diff changeset
    50
Payload part
9c5183cb9bca bundle2: very first version of a bundle2 bundler
Pierre-Yves David <pierre-yves.david@fb.com>
parents:
diff changeset
    51
------------------------
9c5183cb9bca bundle2: very first version of a bundle2 bundler
Pierre-Yves David <pierre-yves.david@fb.com>
parents:
diff changeset
    52
9c5183cb9bca bundle2: very first version of a bundle2 bundler
Pierre-Yves David <pierre-yves.david@fb.com>
parents:
diff changeset
    53
Binary format is as follow
9c5183cb9bca bundle2: very first version of a bundle2 bundler
Pierre-Yves David <pierre-yves.david@fb.com>
parents:
diff changeset
    54
9c5183cb9bca bundle2: very first version of a bundle2 bundler
Pierre-Yves David <pierre-yves.david@fb.com>
parents:
diff changeset
    55
:header size: (16 bits inter)
9c5183cb9bca bundle2: very first version of a bundle2 bundler
Pierre-Yves David <pierre-yves.david@fb.com>
parents:
diff changeset
    56
9c5183cb9bca bundle2: very first version of a bundle2 bundler
Pierre-Yves David <pierre-yves.david@fb.com>
parents:
diff changeset
    57
  The total number of Bytes used by the part headers. When the header is empty
9c5183cb9bca bundle2: very first version of a bundle2 bundler
Pierre-Yves David <pierre-yves.david@fb.com>
parents:
diff changeset
    58
  (size = 0) this is interpreted as the end of stream marker.
9c5183cb9bca bundle2: very first version of a bundle2 bundler
Pierre-Yves David <pierre-yves.david@fb.com>
parents:
diff changeset
    59
9c5183cb9bca bundle2: very first version of a bundle2 bundler
Pierre-Yves David <pierre-yves.david@fb.com>
parents:
diff changeset
    60
  Currently forced to 0 in the current state of the implementation
9c5183cb9bca bundle2: very first version of a bundle2 bundler
Pierre-Yves David <pierre-yves.david@fb.com>
parents:
diff changeset
    61
"""
9c5183cb9bca bundle2: very first version of a bundle2 bundler
Pierre-Yves David <pierre-yves.david@fb.com>
parents:
diff changeset
    62
9c5183cb9bca bundle2: very first version of a bundle2 bundler
Pierre-Yves David <pierre-yves.david@fb.com>
parents:
diff changeset
    63
_magicstring = 'HG20'
9c5183cb9bca bundle2: very first version of a bundle2 bundler
Pierre-Yves David <pierre-yves.david@fb.com>
parents:
diff changeset
    64
9c5183cb9bca bundle2: very first version of a bundle2 bundler
Pierre-Yves David <pierre-yves.david@fb.com>
parents:
diff changeset
    65
class bundle20(object):
9c5183cb9bca bundle2: very first version of a bundle2 bundler
Pierre-Yves David <pierre-yves.david@fb.com>
parents:
diff changeset
    66
    """represent an outgoing bundle2 container
9c5183cb9bca bundle2: very first version of a bundle2 bundler
Pierre-Yves David <pierre-yves.david@fb.com>
parents:
diff changeset
    67
9c5183cb9bca bundle2: very first version of a bundle2 bundler
Pierre-Yves David <pierre-yves.david@fb.com>
parents:
diff changeset
    68
    People will eventually be able to add param and parts to this object and
9c5183cb9bca bundle2: very first version of a bundle2 bundler
Pierre-Yves David <pierre-yves.david@fb.com>
parents:
diff changeset
    69
    generated a stream from it."""
9c5183cb9bca bundle2: very first version of a bundle2 bundler
Pierre-Yves David <pierre-yves.david@fb.com>
parents:
diff changeset
    70
9c5183cb9bca bundle2: very first version of a bundle2 bundler
Pierre-Yves David <pierre-yves.david@fb.com>
parents:
diff changeset
    71
    def __init__(self):
9c5183cb9bca bundle2: very first version of a bundle2 bundler
Pierre-Yves David <pierre-yves.david@fb.com>
parents:
diff changeset
    72
        self._params = []
9c5183cb9bca bundle2: very first version of a bundle2 bundler
Pierre-Yves David <pierre-yves.david@fb.com>
parents:
diff changeset
    73
        self._parts = []
9c5183cb9bca bundle2: very first version of a bundle2 bundler
Pierre-Yves David <pierre-yves.david@fb.com>
parents:
diff changeset
    74
9c5183cb9bca bundle2: very first version of a bundle2 bundler
Pierre-Yves David <pierre-yves.david@fb.com>
parents:
diff changeset
    75
    def getchunks(self):
9c5183cb9bca bundle2: very first version of a bundle2 bundler
Pierre-Yves David <pierre-yves.david@fb.com>
parents:
diff changeset
    76
        yield _magicstring
9c5183cb9bca bundle2: very first version of a bundle2 bundler
Pierre-Yves David <pierre-yves.david@fb.com>
parents:
diff changeset
    77
        # no support for any param yet
9c5183cb9bca bundle2: very first version of a bundle2 bundler
Pierre-Yves David <pierre-yves.david@fb.com>
parents:
diff changeset
    78
        # to be obviously fixed soon.
9c5183cb9bca bundle2: very first version of a bundle2 bundler
Pierre-Yves David <pierre-yves.david@fb.com>
parents:
diff changeset
    79
        assert not self._params
9c5183cb9bca bundle2: very first version of a bundle2 bundler
Pierre-Yves David <pierre-yves.david@fb.com>
parents:
diff changeset
    80
        yield '\0\0'
9c5183cb9bca bundle2: very first version of a bundle2 bundler
Pierre-Yves David <pierre-yves.david@fb.com>
parents:
diff changeset
    81
        # no support for parts
9c5183cb9bca bundle2: very first version of a bundle2 bundler
Pierre-Yves David <pierre-yves.david@fb.com>
parents:
diff changeset
    82
        # to be obviously fixed soon.
9c5183cb9bca bundle2: very first version of a bundle2 bundler
Pierre-Yves David <pierre-yves.david@fb.com>
parents:
diff changeset
    83
        assert not self._parts
9c5183cb9bca bundle2: very first version of a bundle2 bundler
Pierre-Yves David <pierre-yves.david@fb.com>
parents:
diff changeset
    84
        yield '\0\0'