mercurial-scm/hg: mercurial/bundlecaches.py annotate

annotate mercurial/bundlecaches.py @ 53028:89ab2459f62a stable 6.9.3

stream-bundle: properly process requirements When the manifest bundle constain some known requirement that does not affect the stream clone, we used to crash. However since we know them and know they don't affect the stream clone, we can ignore them. Mozilla generated such buggy manifest bundle for a time which allowed us to catch this error. The issue was not caught until 961900fbd67c (released in 6.9.2) as the requirements information were ignored for stream-v2 until then. We fix the issue, refactor the code for robustness and adds more tests to better catch this kind of issue in the future.

author	Pierre-Yves David <pierre-yves.david@octobus.net>
date	Wed, 05 Mar 2025 11:41:45 +0100
parents	961900fbd67c
children	e705fec4a03f

rev	line source
45784 74271829ddc0 clonebundles: move a bundle of clone bundle related code to a new module Pierre-Yves David <pierre-yves.david@octobus.net> parents: diff changeset	1 # bundlecaches.py - utility to deal with pre-computed bundle for servers
74271829ddc0 clonebundles: move a bundle of clone bundle related code to a new module Pierre-Yves David <pierre-yves.david@octobus.net> parents: diff changeset	2 #
74271829ddc0 clonebundles: move a bundle of clone bundle related code to a new module Pierre-Yves David <pierre-yves.david@octobus.net> parents: diff changeset	3 # This software may be used and distributed according to the terms of the
74271829ddc0 clonebundles: move a bundle of clone bundle related code to a new module Pierre-Yves David <pierre-yves.david@octobus.net> parents: diff changeset	4 # GNU General Public License version 2 or any later version.
74271829ddc0 clonebundles: move a bundle of clone bundle related code to a new module Pierre-Yves David <pierre-yves.david@octobus.net> parents: diff changeset	5
51860 1c5810ce737e typing: add `from __future__ import annotations` to remaining source files Matt Harbison <matt_harbison@yahoo.com> parents: 51725 diff changeset	6 from __future__ import annotations
1c5810ce737e typing: add `from __future__ import annotations` to remaining source files Matt Harbison <matt_harbison@yahoo.com> parents: 51725 diff changeset	7
49325 bf66f7a1e3f8 bundlespec: merge the contentopts and params dictionnary Pierre-Yves David <pierre-yves.david@octobus.net> parents: 49322 diff changeset	8 import collections
51978 aa7f4a45d8fa clonebundles: allow manifest to specify sha256 digest of bundles Joerg Sonnenberger <joerg@bec.de> parents: 51860 diff changeset	9 import re
51725 278af66e6595 typing: induce pytype to use the standard `attr` instead of the vendored copy Matt Harbison <matt_harbison@yahoo.com> parents: 51668 diff changeset	10 import typing
49325 bf66f7a1e3f8 bundlespec: merge the contentopts and params dictionnary Pierre-Yves David <pierre-yves.david@octobus.net> parents: 49322 diff changeset	11
49648 9be765b82a90 typing: minor tweaks to allow updating to pytype 2022.11.18 Matt Harbison <matt_harbison@yahoo.com> parents: 49332 diff changeset	12 from typing import (
51571 9b51a4fb0f96 bundlespec: type the _bundlespeccontentopts dictionary Pierre-Yves David <pierre-yves.david@octobus.net> parents: 51570 diff changeset	13 Dict,
9b51a4fb0f96 bundlespec: type the _bundlespeccontentopts dictionary Pierre-Yves David <pierre-yves.david@octobus.net> parents: 51570 diff changeset	14 Union,
49648 9be765b82a90 typing: minor tweaks to allow updating to pytype 2022.11.18 Matt Harbison <matt_harbison@yahoo.com> parents: 49332 diff changeset	15 cast,
9be765b82a90 typing: minor tweaks to allow updating to pytype 2022.11.18 Matt Harbison <matt_harbison@yahoo.com> parents: 49332 diff changeset	16 )
9be765b82a90 typing: minor tweaks to allow updating to pytype 2022.11.18 Matt Harbison <matt_harbison@yahoo.com> parents: 49332 diff changeset	17
45784 74271829ddc0 clonebundles: move a bundle of clone bundle related code to a new module Pierre-Yves David <pierre-yves.david@octobus.net> parents: diff changeset	18 from .i18n import _
74271829ddc0 clonebundles: move a bundle of clone bundle related code to a new module Pierre-Yves David <pierre-yves.david@octobus.net> parents: diff changeset	19
74271829ddc0 clonebundles: move a bundle of clone bundle related code to a new module Pierre-Yves David <pierre-yves.david@octobus.net> parents: diff changeset	20 from .thirdparty import attr
74271829ddc0 clonebundles: move a bundle of clone bundle related code to a new module Pierre-Yves David <pierre-yves.david@octobus.net> parents: diff changeset	21
51725 278af66e6595 typing: induce pytype to use the standard `attr` instead of the vendored copy Matt Harbison <matt_harbison@yahoo.com> parents: 51668 diff changeset	22 # Force pytype to use the non-vendored package
278af66e6595 typing: induce pytype to use the standard `attr` instead of the vendored copy Matt Harbison <matt_harbison@yahoo.com> parents: 51668 diff changeset	23 if typing.TYPE_CHECKING:
278af66e6595 typing: induce pytype to use the standard `attr` instead of the vendored copy Matt Harbison <matt_harbison@yahoo.com> parents: 51668 diff changeset	24 # noinspection PyPackageRequirements
278af66e6595 typing: induce pytype to use the standard `attr` instead of the vendored copy Matt Harbison <matt_harbison@yahoo.com> parents: 51668 diff changeset	25 import attr
278af66e6595 typing: induce pytype to use the standard `attr` instead of the vendored copy Matt Harbison <matt_harbison@yahoo.com> parents: 51668 diff changeset	26
45784 74271829ddc0 clonebundles: move a bundle of clone bundle related code to a new module Pierre-Yves David <pierre-yves.david@octobus.net> parents: diff changeset	27 from . import (
74271829ddc0 clonebundles: move a bundle of clone bundle related code to a new module Pierre-Yves David <pierre-yves.david@octobus.net> parents: diff changeset	28 error,
46627 f4c325bf80fc requirements: also add a generaldelta constant Rapha?l Gom?s <rgomes@octobus.net> parents: 45785 diff changeset	29 requirements as requirementsmod,
45784 74271829ddc0 clonebundles: move a bundle of clone bundle related code to a new module Pierre-Yves David <pierre-yves.david@octobus.net> parents: diff changeset	30 sslutil,
51978 aa7f4a45d8fa clonebundles: allow manifest to specify sha256 digest of bundles Joerg Sonnenberger <joerg@bec.de> parents: 51860 diff changeset	31 url as urlmod,
45784 74271829ddc0 clonebundles: move a bundle of clone bundle related code to a new module Pierre-Yves David <pierre-yves.david@octobus.net> parents: diff changeset	32 util,
74271829ddc0 clonebundles: move a bundle of clone bundle related code to a new module Pierre-Yves David <pierre-yves.david@octobus.net> parents: diff changeset	33 )
74271829ddc0 clonebundles: move a bundle of clone bundle related code to a new module Pierre-Yves David <pierre-yves.david@octobus.net> parents: diff changeset	34 from .utils import stringutil
74271829ddc0 clonebundles: move a bundle of clone bundle related code to a new module Pierre-Yves David <pierre-yves.david@octobus.net> parents: diff changeset	35
74271829ddc0 clonebundles: move a bundle of clone bundle related code to a new module Pierre-Yves David <pierre-yves.david@octobus.net> parents: diff changeset	36 urlreq = util.urlreq
74271829ddc0 clonebundles: move a bundle of clone bundle related code to a new module Pierre-Yves David <pierre-yves.david@octobus.net> parents: diff changeset	37
50683 a41eeb877d07 branching: merge with stable Pierre-Yves David <pierre-yves.david@octobus.net> parents: 50539 diff changeset	38 BUNDLE_CACHE_DIR = b'bundle-cache'
45785 80f32ec8653a clonebundle: move the manifest filename to a constant Pierre-Yves David <pierre-yves.david@octobus.net> parents: 45784 diff changeset	39 CB_MANIFEST_FILE = b'clonebundles.manifest'
50683 a41eeb877d07 branching: merge with stable Pierre-Yves David <pierre-yves.david@octobus.net> parents: 50539 diff changeset	40 CLONEBUNDLESCHEME = b"peer-bundle-cache://"
a41eeb877d07 branching: merge with stable Pierre-Yves David <pierre-yves.david@octobus.net> parents: 50539 diff changeset	41
a41eeb877d07 branching: merge with stable Pierre-Yves David <pierre-yves.david@octobus.net> parents: 50539 diff changeset	42
51668 138ab7c6a6ff typing: add some trivial type hints to `mercurial/bundlecaches.py` Matt Harbison <matt_harbison@yahoo.com> parents: 51597 diff changeset	43 def get_manifest(repo) -> bytes:
50683 a41eeb877d07 branching: merge with stable Pierre-Yves David <pierre-yves.david@octobus.net> parents: 50539 diff changeset	44 """get the bundle manifest to be served to a client from a server"""
a41eeb877d07 branching: merge with stable Pierre-Yves David <pierre-yves.david@octobus.net> parents: 50539 diff changeset	45 raw_text = repo.vfs.tryread(CB_MANIFEST_FILE)
a41eeb877d07 branching: merge with stable Pierre-Yves David <pierre-yves.david@octobus.net> parents: 50539 diff changeset	46 entries = [e.split(b' ', 1) for e in raw_text.splitlines()]
a41eeb877d07 branching: merge with stable Pierre-Yves David <pierre-yves.david@octobus.net> parents: 50539 diff changeset	47
a41eeb877d07 branching: merge with stable Pierre-Yves David <pierre-yves.david@octobus.net> parents: 50539 diff changeset	48 new_lines = []
a41eeb877d07 branching: merge with stable Pierre-Yves David <pierre-yves.david@octobus.net> parents: 50539 diff changeset	49 for e in entries:
a41eeb877d07 branching: merge with stable Pierre-Yves David <pierre-yves.david@octobus.net> parents: 50539 diff changeset	50 url = alter_bundle_url(repo, e[0])
a41eeb877d07 branching: merge with stable Pierre-Yves David <pierre-yves.david@octobus.net> parents: 50539 diff changeset	51 if len(e) == 1:
a41eeb877d07 branching: merge with stable Pierre-Yves David <pierre-yves.david@octobus.net> parents: 50539 diff changeset	52 line = url + b'\n'
a41eeb877d07 branching: merge with stable Pierre-Yves David <pierre-yves.david@octobus.net> parents: 50539 diff changeset	53 else:
a41eeb877d07 branching: merge with stable Pierre-Yves David <pierre-yves.david@octobus.net> parents: 50539 diff changeset	54 line = b"%s %s\n" % (url, e[1])
a41eeb877d07 branching: merge with stable Pierre-Yves David <pierre-yves.david@octobus.net> parents: 50539 diff changeset	55 new_lines.append(line)
a41eeb877d07 branching: merge with stable Pierre-Yves David <pierre-yves.david@octobus.net> parents: 50539 diff changeset	56 return b''.join(new_lines)
a41eeb877d07 branching: merge with stable Pierre-Yves David <pierre-yves.david@octobus.net> parents: 50539 diff changeset	57
a41eeb877d07 branching: merge with stable Pierre-Yves David <pierre-yves.david@octobus.net> parents: 50539 diff changeset	58
51668 138ab7c6a6ff typing: add some trivial type hints to `mercurial/bundlecaches.py` Matt Harbison <matt_harbison@yahoo.com> parents: 51597 diff changeset	59 def alter_bundle_url(repo, url: bytes) -> bytes:
50683 a41eeb877d07 branching: merge with stable Pierre-Yves David <pierre-yves.david@octobus.net> parents: 50539 diff changeset	60 """a function that exist to help extension and hosting to alter the url
a41eeb877d07 branching: merge with stable Pierre-Yves David <pierre-yves.david@octobus.net> parents: 50539 diff changeset	61
a41eeb877d07 branching: merge with stable Pierre-Yves David <pierre-yves.david@octobus.net> parents: 50539 diff changeset	62 This will typically be used to inject authentication information in the url
a41eeb877d07 branching: merge with stable Pierre-Yves David <pierre-yves.david@octobus.net> parents: 50539 diff changeset	63 of cached bundles."""
a41eeb877d07 branching: merge with stable Pierre-Yves David <pierre-yves.david@octobus.net> parents: 50539 diff changeset	64 return url
a41eeb877d07 branching: merge with stable Pierre-Yves David <pierre-yves.david@octobus.net> parents: 50539 diff changeset	65
a41eeb877d07 branching: merge with stable Pierre-Yves David <pierre-yves.david@octobus.net> parents: 50539 diff changeset	66
50539 7b723217d368 clonebundles: filter out invalid schemes instead of failing on them Mathias De Mare <mathias.de_mare@nokia.com> parents: 49648 diff changeset	67 SUPPORTED_CLONEBUNDLE_SCHEMES = [
7b723217d368 clonebundles: filter out invalid schemes instead of failing on them Mathias De Mare <mathias.de_mare@nokia.com> parents: 49648 diff changeset	68 b"http://",
7b723217d368 clonebundles: filter out invalid schemes instead of failing on them Mathias De Mare <mathias.de_mare@nokia.com> parents: 49648 diff changeset	69 b"https://",
7b723217d368 clonebundles: filter out invalid schemes instead of failing on them Mathias De Mare <mathias.de_mare@nokia.com> parents: 49648 diff changeset	70 b"largefile://",
50683 a41eeb877d07 branching: merge with stable Pierre-Yves David <pierre-yves.david@octobus.net> parents: 50539 diff changeset	71 CLONEBUNDLESCHEME,
50539 7b723217d368 clonebundles: filter out invalid schemes instead of failing on them Mathias De Mare <mathias.de_mare@nokia.com> parents: 49648 diff changeset	72 ]
45785 80f32ec8653a clonebundle: move the manifest filename to a constant Pierre-Yves David <pierre-yves.david@octobus.net> parents: 45784 diff changeset	73
45784 74271829ddc0 clonebundles: move a bundle of clone bundle related code to a new module Pierre-Yves David <pierre-yves.david@octobus.net> parents: diff changeset	74
74271829ddc0 clonebundles: move a bundle of clone bundle related code to a new module Pierre-Yves David <pierre-yves.david@octobus.net> parents: diff changeset	75 @attr.s
48946 642e31cb55f0 py3: use class X: instead of class X(object): Gregory Szorc <gregory.szorc@gmail.com> parents: 48601 diff changeset	76 class bundlespec:
45784 74271829ddc0 clonebundles: move a bundle of clone bundle related code to a new module Pierre-Yves David <pierre-yves.david@octobus.net> parents: diff changeset	77 compression = attr.ib()
74271829ddc0 clonebundles: move a bundle of clone bundle related code to a new module Pierre-Yves David <pierre-yves.david@octobus.net> parents: diff changeset	78 wirecompression = attr.ib()
74271829ddc0 clonebundles: move a bundle of clone bundle related code to a new module Pierre-Yves David <pierre-yves.david@octobus.net> parents: diff changeset	79 version = attr.ib()
74271829ddc0 clonebundles: move a bundle of clone bundle related code to a new module Pierre-Yves David <pierre-yves.david@octobus.net> parents: diff changeset	80 wireversion = attr.ib()
49325 bf66f7a1e3f8 bundlespec: merge the contentopts and params dictionnary Pierre-Yves David <pierre-yves.david@octobus.net> parents: 49322 diff changeset	81 # parameters explicitly overwritten by the config or the specification
bf66f7a1e3f8 bundlespec: merge the contentopts and params dictionnary Pierre-Yves David <pierre-yves.david@octobus.net> parents: 49322 diff changeset	82 _explicit_params = attr.ib()
bf66f7a1e3f8 bundlespec: merge the contentopts and params dictionnary Pierre-Yves David <pierre-yves.david@octobus.net> parents: 49322 diff changeset	83 # default parameter for the version
bf66f7a1e3f8 bundlespec: merge the contentopts and params dictionnary Pierre-Yves David <pierre-yves.david@octobus.net> parents: 49322 diff changeset	84 #
bf66f7a1e3f8 bundlespec: merge the contentopts and params dictionnary Pierre-Yves David <pierre-yves.david@octobus.net> parents: 49322 diff changeset	85 # Keeping it separated is useful to check what was actually overwritten.
bf66f7a1e3f8 bundlespec: merge the contentopts and params dictionnary Pierre-Yves David <pierre-yves.david@octobus.net> parents: 49322 diff changeset	86 _default_opts = attr.ib()
bf66f7a1e3f8 bundlespec: merge the contentopts and params dictionnary Pierre-Yves David <pierre-yves.david@octobus.net> parents: 49322 diff changeset	87
bf66f7a1e3f8 bundlespec: merge the contentopts and params dictionnary Pierre-Yves David <pierre-yves.david@octobus.net> parents: 49322 diff changeset	88 @property
bf66f7a1e3f8 bundlespec: merge the contentopts and params dictionnary Pierre-Yves David <pierre-yves.david@octobus.net> parents: 49322 diff changeset	89 def params(self):
bf66f7a1e3f8 bundlespec: merge the contentopts and params dictionnary Pierre-Yves David <pierre-yves.david@octobus.net> parents: 49322 diff changeset	90 return collections.ChainMap(self._explicit_params, self._default_opts)
bf66f7a1e3f8 bundlespec: merge the contentopts and params dictionnary Pierre-Yves David <pierre-yves.david@octobus.net> parents: 49322 diff changeset	91
bf66f7a1e3f8 bundlespec: merge the contentopts and params dictionnary Pierre-Yves David <pierre-yves.david@octobus.net> parents: 49322 diff changeset	92 @property
bf66f7a1e3f8 bundlespec: merge the contentopts and params dictionnary Pierre-Yves David <pierre-yves.david@octobus.net> parents: 49322 diff changeset	93 def contentopts(self):
bf66f7a1e3f8 bundlespec: merge the contentopts and params dictionnary Pierre-Yves David <pierre-yves.david@octobus.net> parents: 49322 diff changeset	94 # kept for Backward Compatibility concerns.
bf66f7a1e3f8 bundlespec: merge the contentopts and params dictionnary Pierre-Yves David <pierre-yves.david@octobus.net> parents: 49322 diff changeset	95 return self.params
bf66f7a1e3f8 bundlespec: merge the contentopts and params dictionnary Pierre-Yves David <pierre-yves.david@octobus.net> parents: 49322 diff changeset	96
49326 3840d16595cf bundlespec: add a `overwrite` parameter to set_param Pierre-Yves David <pierre-yves.david@octobus.net> parents: 49325 diff changeset	97 def set_param(self, key, value, overwrite=True):
3840d16595cf bundlespec: add a `overwrite` parameter to set_param Pierre-Yves David <pierre-yves.david@octobus.net> parents: 49325 diff changeset	98 """Set a bundle parameter value.
3840d16595cf bundlespec: add a `overwrite` parameter to set_param Pierre-Yves David <pierre-yves.david@octobus.net> parents: 49325 diff changeset	99
3840d16595cf bundlespec: add a `overwrite` parameter to set_param Pierre-Yves David <pierre-yves.david@octobus.net> parents: 49325 diff changeset	100 Will only overwrite if overwrite is true"""
3840d16595cf bundlespec: add a `overwrite` parameter to set_param Pierre-Yves David <pierre-yves.david@octobus.net> parents: 49325 diff changeset	101 if overwrite or key not in self._explicit_params:
3840d16595cf bundlespec: add a `overwrite` parameter to set_param Pierre-Yves David <pierre-yves.david@octobus.net> parents: 49325 diff changeset	102 self._explicit_params[key] = value
45784 74271829ddc0 clonebundles: move a bundle of clone bundle related code to a new module Pierre-Yves David <pierre-yves.david@octobus.net> parents: diff changeset	103
50709 0913a49e020c bundle-spec: add an official method to format a spec into a string Pierre-Yves David <pierre-yves.david@octobus.net> parents: 50683 diff changeset	104 def as_spec(self):
0913a49e020c bundle-spec: add an official method to format a spec into a string Pierre-Yves David <pierre-yves.david@octobus.net> parents: 50683 diff changeset	105 parts = [b"%s-%s" % (self.compression, self.version)]
0913a49e020c bundle-spec: add an official method to format a spec into a string Pierre-Yves David <pierre-yves.david@octobus.net> parents: 50683 diff changeset	106 for param in sorted(self._explicit_params.items()):
0913a49e020c bundle-spec: add an official method to format a spec into a string Pierre-Yves David <pierre-yves.david@octobus.net> parents: 50683 diff changeset	107 parts.append(b'%s=%s' % param)
0913a49e020c bundle-spec: add an official method to format a spec into a string Pierre-Yves David <pierre-yves.david@octobus.net> parents: 50683 diff changeset	108 return b';'.join(parts)
0913a49e020c bundle-spec: add an official method to format a spec into a string Pierre-Yves David <pierre-yves.david@octobus.net> parents: 50683 diff changeset	109
45784 74271829ddc0 clonebundles: move a bundle of clone bundle related code to a new module Pierre-Yves David <pierre-yves.david@octobus.net> parents: diff changeset	110
74271829ddc0 clonebundles: move a bundle of clone bundle related code to a new module Pierre-Yves David <pierre-yves.david@octobus.net> parents: diff changeset	111 # Maps bundle version human names to changegroup versions.
74271829ddc0 clonebundles: move a bundle of clone bundle related code to a new module Pierre-Yves David <pierre-yves.david@octobus.net> parents: diff changeset	112 _bundlespeccgversions = {
74271829ddc0 clonebundles: move a bundle of clone bundle related code to a new module Pierre-Yves David <pierre-yves.david@octobus.net> parents: diff changeset	113 b'v1': b'01',
74271829ddc0 clonebundles: move a bundle of clone bundle related code to a new module Pierre-Yves David <pierre-yves.david@octobus.net> parents: diff changeset	114 b'v2': b'02',
50683 a41eeb877d07 branching: merge with stable Pierre-Yves David <pierre-yves.david@octobus.net> parents: 50539 diff changeset	115 b'v3': b'03',
45784 74271829ddc0 clonebundles: move a bundle of clone bundle related code to a new module Pierre-Yves David <pierre-yves.david@octobus.net> parents: diff changeset	116 b'packed1': b's1',
74271829ddc0 clonebundles: move a bundle of clone bundle related code to a new module Pierre-Yves David <pierre-yves.david@octobus.net> parents: diff changeset	117 b'bundle2': b'02', # legacy
74271829ddc0 clonebundles: move a bundle of clone bundle related code to a new module Pierre-Yves David <pierre-yves.david@octobus.net> parents: diff changeset	118 }
74271829ddc0 clonebundles: move a bundle of clone bundle related code to a new module Pierre-Yves David <pierre-yves.david@octobus.net> parents: diff changeset	119
74271829ddc0 clonebundles: move a bundle of clone bundle related code to a new module Pierre-Yves David <pierre-yves.david@octobus.net> parents: diff changeset	120 # Maps bundle version with content opts to choose which part to bundle
51571 9b51a4fb0f96 bundlespec: type the _bundlespeccontentopts dictionary Pierre-Yves David <pierre-yves.david@octobus.net> parents: 51570 diff changeset	121 _bundlespeccontentopts: Dict[bytes, Dict[bytes, Union[bool, bytes]]] = {
45784 74271829ddc0 clonebundles: move a bundle of clone bundle related code to a new module Pierre-Yves David <pierre-yves.david@octobus.net> parents: diff changeset	122 b'v1': {
74271829ddc0 clonebundles: move a bundle of clone bundle related code to a new module Pierre-Yves David <pierre-yves.david@octobus.net> parents: diff changeset	123 b'changegroup': True,
74271829ddc0 clonebundles: move a bundle of clone bundle related code to a new module Pierre-Yves David <pierre-yves.david@octobus.net> parents: diff changeset	124 b'cg.version': b'01',
74271829ddc0 clonebundles: move a bundle of clone bundle related code to a new module Pierre-Yves David <pierre-yves.david@octobus.net> parents: diff changeset	125 b'obsolescence': False,
74271829ddc0 clonebundles: move a bundle of clone bundle related code to a new module Pierre-Yves David <pierre-yves.david@octobus.net> parents: diff changeset	126 b'phases': False,
74271829ddc0 clonebundles: move a bundle of clone bundle related code to a new module Pierre-Yves David <pierre-yves.david@octobus.net> parents: diff changeset	127 b'tagsfnodescache': False,
74271829ddc0 clonebundles: move a bundle of clone bundle related code to a new module Pierre-Yves David <pierre-yves.david@octobus.net> parents: diff changeset	128 b'revbranchcache': False,
74271829ddc0 clonebundles: move a bundle of clone bundle related code to a new module Pierre-Yves David <pierre-yves.david@octobus.net> parents: diff changeset	129 },
74271829ddc0 clonebundles: move a bundle of clone bundle related code to a new module Pierre-Yves David <pierre-yves.david@octobus.net> parents: diff changeset	130 b'v2': {
74271829ddc0 clonebundles: move a bundle of clone bundle related code to a new module Pierre-Yves David <pierre-yves.david@octobus.net> parents: diff changeset	131 b'changegroup': True,
74271829ddc0 clonebundles: move a bundle of clone bundle related code to a new module Pierre-Yves David <pierre-yves.david@octobus.net> parents: diff changeset	132 b'cg.version': b'02',
74271829ddc0 clonebundles: move a bundle of clone bundle related code to a new module Pierre-Yves David <pierre-yves.david@octobus.net> parents: diff changeset	133 b'obsolescence': False,
74271829ddc0 clonebundles: move a bundle of clone bundle related code to a new module Pierre-Yves David <pierre-yves.david@octobus.net> parents: diff changeset	134 b'phases': False,
74271829ddc0 clonebundles: move a bundle of clone bundle related code to a new module Pierre-Yves David <pierre-yves.david@octobus.net> parents: diff changeset	135 b'tagsfnodescache': True,
74271829ddc0 clonebundles: move a bundle of clone bundle related code to a new module Pierre-Yves David <pierre-yves.david@octobus.net> parents: diff changeset	136 b'revbranchcache': True,
74271829ddc0 clonebundles: move a bundle of clone bundle related code to a new module Pierre-Yves David <pierre-yves.david@octobus.net> parents: diff changeset	137 },
50683 a41eeb877d07 branching: merge with stable Pierre-Yves David <pierre-yves.david@octobus.net> parents: 50539 diff changeset	138 b'v3': {
a41eeb877d07 branching: merge with stable Pierre-Yves David <pierre-yves.david@octobus.net> parents: 50539 diff changeset	139 b'changegroup': True,
a41eeb877d07 branching: merge with stable Pierre-Yves David <pierre-yves.david@octobus.net> parents: 50539 diff changeset	140 b'cg.version': b'03',
a41eeb877d07 branching: merge with stable Pierre-Yves David <pierre-yves.david@octobus.net> parents: 50539 diff changeset	141 b'obsolescence': False,
a41eeb877d07 branching: merge with stable Pierre-Yves David <pierre-yves.david@octobus.net> parents: 50539 diff changeset	142 b'phases': True,
a41eeb877d07 branching: merge with stable Pierre-Yves David <pierre-yves.david@octobus.net> parents: 50539 diff changeset	143 b'tagsfnodescache': True,
a41eeb877d07 branching: merge with stable Pierre-Yves David <pierre-yves.david@octobus.net> parents: 50539 diff changeset	144 b'revbranchcache': True,
a41eeb877d07 branching: merge with stable Pierre-Yves David <pierre-yves.david@octobus.net> parents: 50539 diff changeset	145 },
49322 d587f09cad98 bundlespec: make the `stream` case less special Pierre-Yves David <pierre-yves.david@octobus.net> parents: 49321 diff changeset	146 b'streamv2': {
d587f09cad98 bundlespec: make the `stream` case less special Pierre-Yves David <pierre-yves.david@octobus.net> parents: 49321 diff changeset	147 b'changegroup': False,
d587f09cad98 bundlespec: make the `stream` case less special Pierre-Yves David <pierre-yves.david@octobus.net> parents: 49321 diff changeset	148 b'cg.version': b'02',
d587f09cad98 bundlespec: make the `stream` case less special Pierre-Yves David <pierre-yves.david@octobus.net> parents: 49321 diff changeset	149 b'obsolescence': False,
d587f09cad98 bundlespec: make the `stream` case less special Pierre-Yves David <pierre-yves.david@octobus.net> parents: 49321 diff changeset	150 b'phases': False,
51570 4a9d0898bf16 bundlespec: fix the "streamv2" and "streamv3-exp" variant Pierre-Yves David <pierre-yves.david@octobus.net> parents: 51555 diff changeset	151 b"stream": b"v2",
49322 d587f09cad98 bundlespec: make the `stream` case less special Pierre-Yves David <pierre-yves.david@octobus.net> parents: 49321 diff changeset	152 b'tagsfnodescache': False,
d587f09cad98 bundlespec: make the `stream` case less special Pierre-Yves David <pierre-yves.david@octobus.net> parents: 49321 diff changeset	153 b'revbranchcache': False,
d587f09cad98 bundlespec: make the `stream` case less special Pierre-Yves David <pierre-yves.david@octobus.net> parents: 49321 diff changeset	154 },
50683 a41eeb877d07 branching: merge with stable Pierre-Yves David <pierre-yves.david@octobus.net> parents: 50539 diff changeset	155 b'streamv3-exp': {
a41eeb877d07 branching: merge with stable Pierre-Yves David <pierre-yves.david@octobus.net> parents: 50539 diff changeset	156 b'changegroup': False,
a41eeb877d07 branching: merge with stable Pierre-Yves David <pierre-yves.david@octobus.net> parents: 50539 diff changeset	157 b'cg.version': b'03',
a41eeb877d07 branching: merge with stable Pierre-Yves David <pierre-yves.david@octobus.net> parents: 50539 diff changeset	158 b'obsolescence': False,
a41eeb877d07 branching: merge with stable Pierre-Yves David <pierre-yves.david@octobus.net> parents: 50539 diff changeset	159 b'phases': False,
51570 4a9d0898bf16 bundlespec: fix the "streamv2" and "streamv3-exp" variant Pierre-Yves David <pierre-yves.david@octobus.net> parents: 51555 diff changeset	160 b"stream": b"v3-exp",
50683 a41eeb877d07 branching: merge with stable Pierre-Yves David <pierre-yves.david@octobus.net> parents: 50539 diff changeset	161 b'tagsfnodescache': False,
a41eeb877d07 branching: merge with stable Pierre-Yves David <pierre-yves.david@octobus.net> parents: 50539 diff changeset	162 b'revbranchcache': False,
a41eeb877d07 branching: merge with stable Pierre-Yves David <pierre-yves.david@octobus.net> parents: 50539 diff changeset	163 },
49321 c12c843f0db7 bundlespec: phase out the `_bundlespeccgversions` mapping Pierre-Yves David <pierre-yves.david@octobus.net> parents: 48946 diff changeset	164 b'packed1': {
c12c843f0db7 bundlespec: phase out the `_bundlespeccgversions` mapping Pierre-Yves David <pierre-yves.david@octobus.net> parents: 48946 diff changeset	165 b'cg.version': b's1',
c12c843f0db7 bundlespec: phase out the `_bundlespeccgversions` mapping Pierre-Yves David <pierre-yves.david@octobus.net> parents: 48946 diff changeset	166 },
c12c843f0db7 bundlespec: phase out the `_bundlespeccgversions` mapping Pierre-Yves David <pierre-yves.david@octobus.net> parents: 48946 diff changeset	167 b'bundle2': { # legacy
c12c843f0db7 bundlespec: phase out the `_bundlespeccgversions` mapping Pierre-Yves David <pierre-yves.david@octobus.net> parents: 48946 diff changeset	168 b'cg.version': b'02',
c12c843f0db7 bundlespec: phase out the `_bundlespeccgversions` mapping Pierre-Yves David <pierre-yves.david@octobus.net> parents: 48946 diff changeset	169 },
45784 74271829ddc0 clonebundles: move a bundle of clone bundle related code to a new module Pierre-Yves David <pierre-yves.david@octobus.net> parents: diff changeset	170 }
74271829ddc0 clonebundles: move a bundle of clone bundle related code to a new module Pierre-Yves David <pierre-yves.david@octobus.net> parents: diff changeset	171 _bundlespeccontentopts[b'bundle2'] = _bundlespeccontentopts[b'v2']
74271829ddc0 clonebundles: move a bundle of clone bundle related code to a new module Pierre-Yves David <pierre-yves.david@octobus.net> parents: diff changeset	172
74271829ddc0 clonebundles: move a bundle of clone bundle related code to a new module Pierre-Yves David <pierre-yves.david@octobus.net> parents: diff changeset	173 # Compression engines allowed in version 1. THIS SHOULD NEVER CHANGE.
74271829ddc0 clonebundles: move a bundle of clone bundle related code to a new module Pierre-Yves David <pierre-yves.david@octobus.net> parents: diff changeset	174 _bundlespecv1compengines = {b'gzip', b'bzip2', b'none'}
74271829ddc0 clonebundles: move a bundle of clone bundle related code to a new module Pierre-Yves David <pierre-yves.david@octobus.net> parents: diff changeset	175
74271829ddc0 clonebundles: move a bundle of clone bundle related code to a new module Pierre-Yves David <pierre-yves.david@octobus.net> parents: diff changeset	176
49331 1b04d5213d0f bundlespec: add processing of some parameter value Pierre-Yves David <pierre-yves.david@octobus.net> parents: 49330 diff changeset	177 def param_bool(key, value):
1b04d5213d0f bundlespec: add processing of some parameter value Pierre-Yves David <pierre-yves.david@octobus.net> parents: 49330 diff changeset	178 """make a boolean out of a parameter value"""
1b04d5213d0f bundlespec: add processing of some parameter value Pierre-Yves David <pierre-yves.david@octobus.net> parents: 49330 diff changeset	179 b = stringutil.parsebool(value)
1b04d5213d0f bundlespec: add processing of some parameter value Pierre-Yves David <pierre-yves.david@octobus.net> parents: 49330 diff changeset	180 if b is None:
1b04d5213d0f bundlespec: add processing of some parameter value Pierre-Yves David <pierre-yves.david@octobus.net> parents: 49330 diff changeset	181 msg = _(b"parameter %s should be a boolean ('%s')")
1b04d5213d0f bundlespec: add processing of some parameter value Pierre-Yves David <pierre-yves.david@octobus.net> parents: 49330 diff changeset	182 msg %= (key, value)
1b04d5213d0f bundlespec: add processing of some parameter value Pierre-Yves David <pierre-yves.david@octobus.net> parents: 49330 diff changeset	183 raise error.InvalidBundleSpecification(msg)
1b04d5213d0f bundlespec: add processing of some parameter value Pierre-Yves David <pierre-yves.david@octobus.net> parents: 49330 diff changeset	184 return b
1b04d5213d0f bundlespec: add processing of some parameter value Pierre-Yves David <pierre-yves.david@octobus.net> parents: 49330 diff changeset	185
1b04d5213d0f bundlespec: add processing of some parameter value Pierre-Yves David <pierre-yves.david@octobus.net> parents: 49330 diff changeset	186
1b04d5213d0f bundlespec: add processing of some parameter value Pierre-Yves David <pierre-yves.david@octobus.net> parents: 49330 diff changeset	187 # mapping of known parameter name need their value processed
1b04d5213d0f bundlespec: add processing of some parameter value Pierre-Yves David <pierre-yves.david@octobus.net> parents: 49330 diff changeset	188 bundle_spec_param_processing = {
1b04d5213d0f bundlespec: add processing of some parameter value Pierre-Yves David <pierre-yves.david@octobus.net> parents: 49330 diff changeset	189 b"obsolescence": param_bool,
1b04d5213d0f bundlespec: add processing of some parameter value Pierre-Yves David <pierre-yves.david@octobus.net> parents: 49330 diff changeset	190 b"obsolescence-mandatory": param_bool,
1b04d5213d0f bundlespec: add processing of some parameter value Pierre-Yves David <pierre-yves.david@octobus.net> parents: 49330 diff changeset	191 b"phases": param_bool,
51559 eac84af0c8cc bundle-spec: properly parse boolean configuration as boolean Pierre-Yves David <pierre-yves.david@octobus.net> parents: 50709 diff changeset	192 b"changegroup": param_bool,
eac84af0c8cc bundle-spec: properly parse boolean configuration as boolean Pierre-Yves David <pierre-yves.david@octobus.net> parents: 50709 diff changeset	193 b"tagsfnodescache": param_bool,
eac84af0c8cc bundle-spec: properly parse boolean configuration as boolean Pierre-Yves David <pierre-yves.david@octobus.net> parents: 50709 diff changeset	194 b"revbranchcache": param_bool,
49331 1b04d5213d0f bundlespec: add processing of some parameter value Pierre-Yves David <pierre-yves.david@octobus.net> parents: 49330 diff changeset	195 }
1b04d5213d0f bundlespec: add processing of some parameter value Pierre-Yves David <pierre-yves.david@octobus.net> parents: 49330 diff changeset	196
1b04d5213d0f bundlespec: add processing of some parameter value Pierre-Yves David <pierre-yves.david@octobus.net> parents: 49330 diff changeset	197
49330 5d17dd74177d bundlespec: extract the parseparams closure Pierre-Yves David <pierre-yves.david@octobus.net> parents: 49326 diff changeset	198 def _parseparams(s):
5d17dd74177d bundlespec: extract the parseparams closure Pierre-Yves David <pierre-yves.david@octobus.net> parents: 49326 diff changeset	199 """parse bundlespec parameter section
5d17dd74177d bundlespec: extract the parseparams closure Pierre-Yves David <pierre-yves.david@octobus.net> parents: 49326 diff changeset	200
5d17dd74177d bundlespec: extract the parseparams closure Pierre-Yves David <pierre-yves.david@octobus.net> parents: 49326 diff changeset	201 input: "comp-version;params" string
5d17dd74177d bundlespec: extract the parseparams closure Pierre-Yves David <pierre-yves.david@octobus.net> parents: 49326 diff changeset	202
5d17dd74177d bundlespec: extract the parseparams closure Pierre-Yves David <pierre-yves.david@octobus.net> parents: 49326 diff changeset	203 return: (spec; {param_key: param_value})
5d17dd74177d bundlespec: extract the parseparams closure Pierre-Yves David <pierre-yves.david@octobus.net> parents: 49326 diff changeset	204 """
5d17dd74177d bundlespec: extract the parseparams closure Pierre-Yves David <pierre-yves.david@octobus.net> parents: 49326 diff changeset	205 if b';' not in s:
5d17dd74177d bundlespec: extract the parseparams closure Pierre-Yves David <pierre-yves.david@octobus.net> parents: 49326 diff changeset	206 return s, {}
5d17dd74177d bundlespec: extract the parseparams closure Pierre-Yves David <pierre-yves.david@octobus.net> parents: 49326 diff changeset	207
5d17dd74177d bundlespec: extract the parseparams closure Pierre-Yves David <pierre-yves.david@octobus.net> parents: 49326 diff changeset	208 params = {}
5d17dd74177d bundlespec: extract the parseparams closure Pierre-Yves David <pierre-yves.david@octobus.net> parents: 49326 diff changeset	209 version, paramstr = s.split(b';', 1)
5d17dd74177d bundlespec: extract the parseparams closure Pierre-Yves David <pierre-yves.david@octobus.net> parents: 49326 diff changeset	210
5d17dd74177d bundlespec: extract the parseparams closure Pierre-Yves David <pierre-yves.david@octobus.net> parents: 49326 diff changeset	211 err = _(b'invalid bundle specification: missing "=" in parameter: %s')
5d17dd74177d bundlespec: extract the parseparams closure Pierre-Yves David <pierre-yves.david@octobus.net> parents: 49326 diff changeset	212 for p in paramstr.split(b';'):
5d17dd74177d bundlespec: extract the parseparams closure Pierre-Yves David <pierre-yves.david@octobus.net> parents: 49326 diff changeset	213 if b'=' not in p:
5d17dd74177d bundlespec: extract the parseparams closure Pierre-Yves David <pierre-yves.david@octobus.net> parents: 49326 diff changeset	214 msg = err % p
5d17dd74177d bundlespec: extract the parseparams closure Pierre-Yves David <pierre-yves.david@octobus.net> parents: 49326 diff changeset	215 raise error.InvalidBundleSpecification(msg)
5d17dd74177d bundlespec: extract the parseparams closure Pierre-Yves David <pierre-yves.david@octobus.net> parents: 49326 diff changeset	216
5d17dd74177d bundlespec: extract the parseparams closure Pierre-Yves David <pierre-yves.david@octobus.net> parents: 49326 diff changeset	217 key, value = p.split(b'=', 1)
5d17dd74177d bundlespec: extract the parseparams closure Pierre-Yves David <pierre-yves.david@octobus.net> parents: 49326 diff changeset	218 key = urlreq.unquote(key)
5d17dd74177d bundlespec: extract the parseparams closure Pierre-Yves David <pierre-yves.david@octobus.net> parents: 49326 diff changeset	219 value = urlreq.unquote(value)
49331 1b04d5213d0f bundlespec: add processing of some parameter value Pierre-Yves David <pierre-yves.david@octobus.net> parents: 49330 diff changeset	220 process = bundle_spec_param_processing.get(key)
1b04d5213d0f bundlespec: add processing of some parameter value Pierre-Yves David <pierre-yves.david@octobus.net> parents: 49330 diff changeset	221 if process is not None:
1b04d5213d0f bundlespec: add processing of some parameter value Pierre-Yves David <pierre-yves.david@octobus.net> parents: 49330 diff changeset	222 value = process(key, value)
49330 5d17dd74177d bundlespec: extract the parseparams closure Pierre-Yves David <pierre-yves.david@octobus.net> parents: 49326 diff changeset	223 params[key] = value
5d17dd74177d bundlespec: extract the parseparams closure Pierre-Yves David <pierre-yves.david@octobus.net> parents: 49326 diff changeset	224
5d17dd74177d bundlespec: extract the parseparams closure Pierre-Yves David <pierre-yves.david@octobus.net> parents: 49326 diff changeset	225 return version, params
5d17dd74177d bundlespec: extract the parseparams closure Pierre-Yves David <pierre-yves.david@octobus.net> parents: 49326 diff changeset	226
5d17dd74177d bundlespec: extract the parseparams closure Pierre-Yves David <pierre-yves.david@octobus.net> parents: 49326 diff changeset	227
45784 74271829ddc0 clonebundles: move a bundle of clone bundle related code to a new module Pierre-Yves David <pierre-yves.david@octobus.net> parents: diff changeset	228 def parsebundlespec(repo, spec, strict=True):
74271829ddc0 clonebundles: move a bundle of clone bundle related code to a new module Pierre-Yves David <pierre-yves.david@octobus.net> parents: diff changeset	229 """Parse a bundle string specification into parts.
74271829ddc0 clonebundles: move a bundle of clone bundle related code to a new module Pierre-Yves David <pierre-yves.david@octobus.net> parents: diff changeset	230
74271829ddc0 clonebundles: move a bundle of clone bundle related code to a new module Pierre-Yves David <pierre-yves.david@octobus.net> parents: diff changeset	231 Bundle specifications denote a well-defined bundle/exchange format.
74271829ddc0 clonebundles: move a bundle of clone bundle related code to a new module Pierre-Yves David <pierre-yves.david@octobus.net> parents: diff changeset	232 The content of a given specification should not change over time in
74271829ddc0 clonebundles: move a bundle of clone bundle related code to a new module Pierre-Yves David <pierre-yves.david@octobus.net> parents: diff changeset	233 order to ensure that bundles produced by a newer version of Mercurial are
74271829ddc0 clonebundles: move a bundle of clone bundle related code to a new module Pierre-Yves David <pierre-yves.david@octobus.net> parents: diff changeset	234 readable from an older version.
74271829ddc0 clonebundles: move a bundle of clone bundle related code to a new module Pierre-Yves David <pierre-yves.david@octobus.net> parents: diff changeset	235
74271829ddc0 clonebundles: move a bundle of clone bundle related code to a new module Pierre-Yves David <pierre-yves.david@octobus.net> parents: diff changeset	236 The string currently has the form:
74271829ddc0 clonebundles: move a bundle of clone bundle related code to a new module Pierre-Yves David <pierre-yves.david@octobus.net> parents: diff changeset	237
74271829ddc0 clonebundles: move a bundle of clone bundle related code to a new module Pierre-Yves David <pierre-yves.david@octobus.net> parents: diff changeset	238 <compression>-<type>[;<parameter0>[;<parameter1>]]
74271829ddc0 clonebundles: move a bundle of clone bundle related code to a new module Pierre-Yves David <pierre-yves.david@octobus.net> parents: diff changeset	239
74271829ddc0 clonebundles: move a bundle of clone bundle related code to a new module Pierre-Yves David <pierre-yves.david@octobus.net> parents: diff changeset	240 Where <compression> is one of the supported compression formats
74271829ddc0 clonebundles: move a bundle of clone bundle related code to a new module Pierre-Yves David <pierre-yves.david@octobus.net> parents: diff changeset	241 and <type> is (currently) a version string. A ";" can follow the type and
74271829ddc0 clonebundles: move a bundle of clone bundle related code to a new module Pierre-Yves David <pierre-yves.david@octobus.net> parents: diff changeset	242 all text afterwards is interpreted as URI encoded, ";" delimited key=value
74271829ddc0 clonebundles: move a bundle of clone bundle related code to a new module Pierre-Yves David <pierre-yves.david@octobus.net> parents: diff changeset	243 pairs.
74271829ddc0 clonebundles: move a bundle of clone bundle related code to a new module Pierre-Yves David <pierre-yves.david@octobus.net> parents: diff changeset	244
74271829ddc0 clonebundles: move a bundle of clone bundle related code to a new module Pierre-Yves David <pierre-yves.david@octobus.net> parents: diff changeset	245 If ``strict`` is True (the default) <compression> is required. Otherwise,
74271829ddc0 clonebundles: move a bundle of clone bundle related code to a new module Pierre-Yves David <pierre-yves.david@octobus.net> parents: diff changeset	246 it is optional.
74271829ddc0 clonebundles: move a bundle of clone bundle related code to a new module Pierre-Yves David <pierre-yves.david@octobus.net> parents: diff changeset	247
74271829ddc0 clonebundles: move a bundle of clone bundle related code to a new module Pierre-Yves David <pierre-yves.david@octobus.net> parents: diff changeset	248 Returns a bundlespec object of (compression, version, parameters).
74271829ddc0 clonebundles: move a bundle of clone bundle related code to a new module Pierre-Yves David <pierre-yves.david@octobus.net> parents: diff changeset	249 Compression will be ``None`` if not in strict mode and a compression isn't
74271829ddc0 clonebundles: move a bundle of clone bundle related code to a new module Pierre-Yves David <pierre-yves.david@octobus.net> parents: diff changeset	250 defined.
74271829ddc0 clonebundles: move a bundle of clone bundle related code to a new module Pierre-Yves David <pierre-yves.david@octobus.net> parents: diff changeset	251
74271829ddc0 clonebundles: move a bundle of clone bundle related code to a new module Pierre-Yves David <pierre-yves.david@octobus.net> parents: diff changeset	252 An ``InvalidBundleSpecification`` is raised when the specification is
74271829ddc0 clonebundles: move a bundle of clone bundle related code to a new module Pierre-Yves David <pierre-yves.david@octobus.net> parents: diff changeset	253 not syntactically well formed.
74271829ddc0 clonebundles: move a bundle of clone bundle related code to a new module Pierre-Yves David <pierre-yves.david@octobus.net> parents: diff changeset	254
74271829ddc0 clonebundles: move a bundle of clone bundle related code to a new module Pierre-Yves David <pierre-yves.david@octobus.net> parents: diff changeset	255 An ``UnsupportedBundleSpecification`` is raised when the compression or
74271829ddc0 clonebundles: move a bundle of clone bundle related code to a new module Pierre-Yves David <pierre-yves.david@octobus.net> parents: diff changeset	256 bundle type/version is not recognized.
74271829ddc0 clonebundles: move a bundle of clone bundle related code to a new module Pierre-Yves David <pierre-yves.david@octobus.net> parents: diff changeset	257
74271829ddc0 clonebundles: move a bundle of clone bundle related code to a new module Pierre-Yves David <pierre-yves.david@octobus.net> parents: diff changeset	258 Note: this function will likely eventually return a more complex data
74271829ddc0 clonebundles: move a bundle of clone bundle related code to a new module Pierre-Yves David <pierre-yves.david@octobus.net> parents: diff changeset	259 structure, including bundle2 part information.
74271829ddc0 clonebundles: move a bundle of clone bundle related code to a new module Pierre-Yves David <pierre-yves.david@octobus.net> parents: diff changeset	260 """
74271829ddc0 clonebundles: move a bundle of clone bundle related code to a new module Pierre-Yves David <pierre-yves.david@octobus.net> parents: diff changeset	261 if strict and b'-' not in spec:
74271829ddc0 clonebundles: move a bundle of clone bundle related code to a new module Pierre-Yves David <pierre-yves.david@octobus.net> parents: diff changeset	262 raise error.InvalidBundleSpecification(
74271829ddc0 clonebundles: move a bundle of clone bundle related code to a new module Pierre-Yves David <pierre-yves.david@octobus.net> parents: diff changeset	263 _(
74271829ddc0 clonebundles: move a bundle of clone bundle related code to a new module Pierre-Yves David <pierre-yves.david@octobus.net> parents: diff changeset	264 b'invalid bundle specification; '
74271829ddc0 clonebundles: move a bundle of clone bundle related code to a new module Pierre-Yves David <pierre-yves.david@octobus.net> parents: diff changeset	265 b'must be prefixed with compression: %s'
74271829ddc0 clonebundles: move a bundle of clone bundle related code to a new module Pierre-Yves David <pierre-yves.david@octobus.net> parents: diff changeset	266 )
74271829ddc0 clonebundles: move a bundle of clone bundle related code to a new module Pierre-Yves David <pierre-yves.david@octobus.net> parents: diff changeset	267 % spec
74271829ddc0 clonebundles: move a bundle of clone bundle related code to a new module Pierre-Yves David <pierre-yves.david@octobus.net> parents: diff changeset	268 )
74271829ddc0 clonebundles: move a bundle of clone bundle related code to a new module Pierre-Yves David <pierre-yves.david@octobus.net> parents: diff changeset	269
49332 d89bfc075289 bundlespec: do not check for `-` in the params portion of the bundlespec Pierre-Yves David <pierre-yves.david@octobus.net> parents: 49331 diff changeset	270 pre_args = spec.split(b';', 1)[0]
d89bfc075289 bundlespec: do not check for `-` in the params portion of the bundlespec Pierre-Yves David <pierre-yves.david@octobus.net> parents: 49331 diff changeset	271 if b'-' in pre_args:
45784 74271829ddc0 clonebundles: move a bundle of clone bundle related code to a new module Pierre-Yves David <pierre-yves.david@octobus.net> parents: diff changeset	272 compression, version = spec.split(b'-', 1)
74271829ddc0 clonebundles: move a bundle of clone bundle related code to a new module Pierre-Yves David <pierre-yves.david@octobus.net> parents: diff changeset	273
74271829ddc0 clonebundles: move a bundle of clone bundle related code to a new module Pierre-Yves David <pierre-yves.david@octobus.net> parents: diff changeset	274 if compression not in util.compengines.supportedbundlenames:
74271829ddc0 clonebundles: move a bundle of clone bundle related code to a new module Pierre-Yves David <pierre-yves.david@octobus.net> parents: diff changeset	275 raise error.UnsupportedBundleSpecification(
74271829ddc0 clonebundles: move a bundle of clone bundle related code to a new module Pierre-Yves David <pierre-yves.david@octobus.net> parents: diff changeset	276 _(b'%s compression is not supported') % compression
74271829ddc0 clonebundles: move a bundle of clone bundle related code to a new module Pierre-Yves David <pierre-yves.david@octobus.net> parents: diff changeset	277 )
74271829ddc0 clonebundles: move a bundle of clone bundle related code to a new module Pierre-Yves David <pierre-yves.david@octobus.net> parents: diff changeset	278
49330 5d17dd74177d bundlespec: extract the parseparams closure Pierre-Yves David <pierre-yves.david@octobus.net> parents: 49326 diff changeset	279 version, params = _parseparams(version)
45784 74271829ddc0 clonebundles: move a bundle of clone bundle related code to a new module Pierre-Yves David <pierre-yves.david@octobus.net> parents: diff changeset	280
49321 c12c843f0db7 bundlespec: phase out the `_bundlespeccgversions` mapping Pierre-Yves David <pierre-yves.david@octobus.net> parents: 48946 diff changeset	281 if version not in _bundlespeccontentopts:
45784 74271829ddc0 clonebundles: move a bundle of clone bundle related code to a new module Pierre-Yves David <pierre-yves.david@octobus.net> parents: diff changeset	282 raise error.UnsupportedBundleSpecification(
74271829ddc0 clonebundles: move a bundle of clone bundle related code to a new module Pierre-Yves David <pierre-yves.david@octobus.net> parents: diff changeset	283 _(b'%s is not a recognized bundle version') % version
74271829ddc0 clonebundles: move a bundle of clone bundle related code to a new module Pierre-Yves David <pierre-yves.david@octobus.net> parents: diff changeset	284 )
74271829ddc0 clonebundles: move a bundle of clone bundle related code to a new module Pierre-Yves David <pierre-yves.david@octobus.net> parents: diff changeset	285 else:
74271829ddc0 clonebundles: move a bundle of clone bundle related code to a new module Pierre-Yves David <pierre-yves.david@octobus.net> parents: diff changeset	286 # Value could be just the compression or just the version, in which
74271829ddc0 clonebundles: move a bundle of clone bundle related code to a new module Pierre-Yves David <pierre-yves.david@octobus.net> parents: diff changeset	287 # case some defaults are assumed (but only when not in strict mode).
74271829ddc0 clonebundles: move a bundle of clone bundle related code to a new module Pierre-Yves David <pierre-yves.david@octobus.net> parents: diff changeset	288 assert not strict
74271829ddc0 clonebundles: move a bundle of clone bundle related code to a new module Pierre-Yves David <pierre-yves.david@octobus.net> parents: diff changeset	289
49330 5d17dd74177d bundlespec: extract the parseparams closure Pierre-Yves David <pierre-yves.david@octobus.net> parents: 49326 diff changeset	290 spec, params = _parseparams(spec)
45784 74271829ddc0 clonebundles: move a bundle of clone bundle related code to a new module Pierre-Yves David <pierre-yves.david@octobus.net> parents: diff changeset	291
74271829ddc0 clonebundles: move a bundle of clone bundle related code to a new module Pierre-Yves David <pierre-yves.david@octobus.net> parents: diff changeset	292 if spec in util.compengines.supportedbundlenames:
74271829ddc0 clonebundles: move a bundle of clone bundle related code to a new module Pierre-Yves David <pierre-yves.david@octobus.net> parents: diff changeset	293 compression = spec
74271829ddc0 clonebundles: move a bundle of clone bundle related code to a new module Pierre-Yves David <pierre-yves.david@octobus.net> parents: diff changeset	294 version = b'v1'
74271829ddc0 clonebundles: move a bundle of clone bundle related code to a new module Pierre-Yves David <pierre-yves.david@octobus.net> parents: diff changeset	295 # Generaldelta repos require v2.
46627 f4c325bf80fc requirements: also add a generaldelta constant Rapha?l Gom?s <rgomes@octobus.net> parents: 45785 diff changeset	296 if requirementsmod.GENERALDELTA_REQUIREMENT in repo.requirements:
45784 74271829ddc0 clonebundles: move a bundle of clone bundle related code to a new module Pierre-Yves David <pierre-yves.david@octobus.net> parents: diff changeset	297 version = b'v2'
47342 dc3800569157 revlogv2: make sure bundling pick a compatible bundle format Pierre-Yves David <pierre-yves.david@octobus.net> parents: 46627 diff changeset	298 elif requirementsmod.REVLOGV2_REQUIREMENT in repo.requirements:
dc3800569157 revlogv2: make sure bundling pick a compatible bundle format Pierre-Yves David <pierre-yves.david@octobus.net> parents: 46627 diff changeset	299 version = b'v2'
45784 74271829ddc0 clonebundles: move a bundle of clone bundle related code to a new module Pierre-Yves David <pierre-yves.david@octobus.net> parents: diff changeset	300 # Modern compression engines require v2.
74271829ddc0 clonebundles: move a bundle of clone bundle related code to a new module Pierre-Yves David <pierre-yves.david@octobus.net> parents: diff changeset	301 if compression not in _bundlespecv1compengines:
74271829ddc0 clonebundles: move a bundle of clone bundle related code to a new module Pierre-Yves David <pierre-yves.david@octobus.net> parents: diff changeset	302 version = b'v2'
49321 c12c843f0db7 bundlespec: phase out the `_bundlespeccgversions` mapping Pierre-Yves David <pierre-yves.david@octobus.net> parents: 48946 diff changeset	303 elif spec in _bundlespeccontentopts:
45784 74271829ddc0 clonebundles: move a bundle of clone bundle related code to a new module Pierre-Yves David <pierre-yves.david@octobus.net> parents: diff changeset	304 if spec == b'packed1':
74271829ddc0 clonebundles: move a bundle of clone bundle related code to a new module Pierre-Yves David <pierre-yves.david@octobus.net> parents: diff changeset	305 compression = b'none'
74271829ddc0 clonebundles: move a bundle of clone bundle related code to a new module Pierre-Yves David <pierre-yves.david@octobus.net> parents: diff changeset	306 else:
74271829ddc0 clonebundles: move a bundle of clone bundle related code to a new module Pierre-Yves David <pierre-yves.david@octobus.net> parents: diff changeset	307 compression = b'bzip2'
74271829ddc0 clonebundles: move a bundle of clone bundle related code to a new module Pierre-Yves David <pierre-yves.david@octobus.net> parents: diff changeset	308 version = spec
74271829ddc0 clonebundles: move a bundle of clone bundle related code to a new module Pierre-Yves David <pierre-yves.david@octobus.net> parents: diff changeset	309 else:
74271829ddc0 clonebundles: move a bundle of clone bundle related code to a new module Pierre-Yves David <pierre-yves.david@octobus.net> parents: diff changeset	310 raise error.UnsupportedBundleSpecification(
74271829ddc0 clonebundles: move a bundle of clone bundle related code to a new module Pierre-Yves David <pierre-yves.david@octobus.net> parents: diff changeset	311 _(b'%s is not a recognized bundle specification') % spec
74271829ddc0 clonebundles: move a bundle of clone bundle related code to a new module Pierre-Yves David <pierre-yves.david@octobus.net> parents: diff changeset	312 )
74271829ddc0 clonebundles: move a bundle of clone bundle related code to a new module Pierre-Yves David <pierre-yves.david@octobus.net> parents: diff changeset	313
74271829ddc0 clonebundles: move a bundle of clone bundle related code to a new module Pierre-Yves David <pierre-yves.david@octobus.net> parents: diff changeset	314 # Bundle version 1 only supports a known set of compression engines.
74271829ddc0 clonebundles: move a bundle of clone bundle related code to a new module Pierre-Yves David <pierre-yves.david@octobus.net> parents: diff changeset	315 if version == b'v1' and compression not in _bundlespecv1compengines:
74271829ddc0 clonebundles: move a bundle of clone bundle related code to a new module Pierre-Yves David <pierre-yves.david@octobus.net> parents: diff changeset	316 raise error.UnsupportedBundleSpecification(
74271829ddc0 clonebundles: move a bundle of clone bundle related code to a new module Pierre-Yves David <pierre-yves.david@octobus.net> parents: diff changeset	317 _(b'compression engine %s is not supported on v1 bundles')
74271829ddc0 clonebundles: move a bundle of clone bundle related code to a new module Pierre-Yves David <pierre-yves.david@octobus.net> parents: diff changeset	318 % compression
74271829ddc0 clonebundles: move a bundle of clone bundle related code to a new module Pierre-Yves David <pierre-yves.david@octobus.net> parents: diff changeset	319 )
74271829ddc0 clonebundles: move a bundle of clone bundle related code to a new module Pierre-Yves David <pierre-yves.david@octobus.net> parents: diff changeset	320
52957 961900fbd67c bundles: filter out unsupported requirements for non-packed1 format Julien Cristau <jcristau@mozilla.com> parents: 51978 diff changeset	321 # The specification for stream bundles can optionally declare the data formats
45784 74271829ddc0 clonebundles: move a bundle of clone bundle related code to a new module Pierre-Yves David <pierre-yves.david@octobus.net> parents: diff changeset	322 # required to apply it. If we see this metadata, compare against what the
74271829ddc0 clonebundles: move a bundle of clone bundle related code to a new module Pierre-Yves David <pierre-yves.david@octobus.net> parents: diff changeset	323 # repo supports and error if the bundle isn't compatible.
52957 961900fbd67c bundles: filter out unsupported requirements for non-packed1 format Julien Cristau <jcristau@mozilla.com> parents: 51978 diff changeset	324 if b'requirements' in params:
49648 9be765b82a90 typing: minor tweaks to allow updating to pytype 2022.11.18 Matt Harbison <matt_harbison@yahoo.com> parents: 49332 diff changeset	325 requirements = set(cast(bytes, params[b'requirements']).split(b','))
53028 89ab2459f62a stream-bundle: properly process requirements Pierre-Yves David <pierre-yves.david@octobus.net> parents: 52957 diff changeset	326 relevant_reqs = (
89ab2459f62a stream-bundle: properly process requirements Pierre-Yves David <pierre-yves.david@octobus.net> parents: 52957 diff changeset	327 requirements - requirementsmod.STREAM_IGNORABLE_REQUIREMENTS
89ab2459f62a stream-bundle: properly process requirements Pierre-Yves David <pierre-yves.david@octobus.net> parents: 52957 diff changeset	328 )
89ab2459f62a stream-bundle: properly process requirements Pierre-Yves David <pierre-yves.david@octobus.net> parents: 52957 diff changeset	329 # avoid cycle (not great for pytype)
89ab2459f62a stream-bundle: properly process requirements Pierre-Yves David <pierre-yves.david@octobus.net> parents: 52957 diff changeset	330 from . import localrepo
89ab2459f62a stream-bundle: properly process requirements Pierre-Yves David <pierre-yves.david@octobus.net> parents: 52957 diff changeset	331
89ab2459f62a stream-bundle: properly process requirements Pierre-Yves David <pierre-yves.david@octobus.net> parents: 52957 diff changeset	332 supported_req = localrepo.gathersupportedrequirements(repo.ui)
89ab2459f62a stream-bundle: properly process requirements Pierre-Yves David <pierre-yves.david@octobus.net> parents: 52957 diff changeset	333 missing_reqs = relevant_reqs - supported_req
89ab2459f62a stream-bundle: properly process requirements Pierre-Yves David <pierre-yves.david@octobus.net> parents: 52957 diff changeset	334 if missing_reqs:
45784 74271829ddc0 clonebundles: move a bundle of clone bundle related code to a new module Pierre-Yves David <pierre-yves.david@octobus.net> parents: diff changeset	335 raise error.UnsupportedBundleSpecification(
74271829ddc0 clonebundles: move a bundle of clone bundle related code to a new module Pierre-Yves David <pierre-yves.david@octobus.net> parents: diff changeset	336 _(b'missing support for repository features: %s')
53028 89ab2459f62a stream-bundle: properly process requirements Pierre-Yves David <pierre-yves.david@octobus.net> parents: 52957 diff changeset	337 % b', '.join(sorted(missing_reqs))
45784 74271829ddc0 clonebundles: move a bundle of clone bundle related code to a new module Pierre-Yves David <pierre-yves.david@octobus.net> parents: diff changeset	338 )
74271829ddc0 clonebundles: move a bundle of clone bundle related code to a new module Pierre-Yves David <pierre-yves.david@octobus.net> parents: diff changeset	339
74271829ddc0 clonebundles: move a bundle of clone bundle related code to a new module Pierre-Yves David <pierre-yves.david@octobus.net> parents: diff changeset	340 # Compute contentopts based on the version
50683 a41eeb877d07 branching: merge with stable Pierre-Yves David <pierre-yves.david@octobus.net> parents: 50539 diff changeset	341 if b"stream" in params:
a41eeb877d07 branching: merge with stable Pierre-Yves David <pierre-yves.david@octobus.net> parents: 50539 diff changeset	342 # This case is fishy as this mostly derails the version selection
49322 d587f09cad98 bundlespec: make the `stream` case less special Pierre-Yves David <pierre-yves.david@octobus.net> parents: 49321 diff changeset	343 # mechanism. `stream` bundles are quite specific and used differently
d587f09cad98 bundlespec: make the `stream` case less special Pierre-Yves David <pierre-yves.david@octobus.net> parents: 49321 diff changeset	344 # as "normal" bundles.
d587f09cad98 bundlespec: make the `stream` case less special Pierre-Yves David <pierre-yves.david@octobus.net> parents: 49321 diff changeset	345 #
d587f09cad98 bundlespec: make the `stream` case less special Pierre-Yves David <pierre-yves.david@octobus.net> parents: 49321 diff changeset	346 # (we should probably define a cleaner way to do this and raise a
50683 a41eeb877d07 branching: merge with stable Pierre-Yves David <pierre-yves.david@octobus.net> parents: 50539 diff changeset	347 # warning when the old way is encountered)
a41eeb877d07 branching: merge with stable Pierre-Yves David <pierre-yves.david@octobus.net> parents: 50539 diff changeset	348 if params[b"stream"] == b"v2":
a41eeb877d07 branching: merge with stable Pierre-Yves David <pierre-yves.david@octobus.net> parents: 50539 diff changeset	349 version = b"streamv2"
a41eeb877d07 branching: merge with stable Pierre-Yves David <pierre-yves.david@octobus.net> parents: 50539 diff changeset	350 if params[b"stream"] == b"v3-exp":
a41eeb877d07 branching: merge with stable Pierre-Yves David <pierre-yves.david@octobus.net> parents: 50539 diff changeset	351 version = b"streamv3-exp"
45784 74271829ddc0 clonebundles: move a bundle of clone bundle related code to a new module Pierre-Yves David <pierre-yves.david@octobus.net> parents: diff changeset	352 contentopts = _bundlespeccontentopts.get(version, {}).copy()
50683 a41eeb877d07 branching: merge with stable Pierre-Yves David <pierre-yves.david@octobus.net> parents: 50539 diff changeset	353 if version == b"streamv2" or version == b"streamv3-exp":
49322 d587f09cad98 bundlespec: make the `stream` case less special Pierre-Yves David <pierre-yves.david@octobus.net> parents: 49321 diff changeset	354 # streamv2 have been reported as "v2" for a while.
d587f09cad98 bundlespec: make the `stream` case less special Pierre-Yves David <pierre-yves.david@octobus.net> parents: 49321 diff changeset	355 version = b"v2"
45784 74271829ddc0 clonebundles: move a bundle of clone bundle related code to a new module Pierre-Yves David <pierre-yves.david@octobus.net> parents: diff changeset	356
74271829ddc0 clonebundles: move a bundle of clone bundle related code to a new module Pierre-Yves David <pierre-yves.david@octobus.net> parents: diff changeset	357 engine = util.compengines.forbundlename(compression)
74271829ddc0 clonebundles: move a bundle of clone bundle related code to a new module Pierre-Yves David <pierre-yves.david@octobus.net> parents: diff changeset	358 compression, wirecompression = engine.bundletype()
49321 c12c843f0db7 bundlespec: phase out the `_bundlespeccgversions` mapping Pierre-Yves David <pierre-yves.david@octobus.net> parents: 48946 diff changeset	359 wireversion = _bundlespeccontentopts[version][b'cg.version']
45784 74271829ddc0 clonebundles: move a bundle of clone bundle related code to a new module Pierre-Yves David <pierre-yves.david@octobus.net> parents: diff changeset	360
74271829ddc0 clonebundles: move a bundle of clone bundle related code to a new module Pierre-Yves David <pierre-yves.david@octobus.net> parents: diff changeset	361 return bundlespec(
74271829ddc0 clonebundles: move a bundle of clone bundle related code to a new module Pierre-Yves David <pierre-yves.david@octobus.net> parents: diff changeset	362 compression, wirecompression, version, wireversion, params, contentopts
74271829ddc0 clonebundles: move a bundle of clone bundle related code to a new module Pierre-Yves David <pierre-yves.david@octobus.net> parents: diff changeset	363 )
74271829ddc0 clonebundles: move a bundle of clone bundle related code to a new module Pierre-Yves David <pierre-yves.david@octobus.net> parents: diff changeset	364
74271829ddc0 clonebundles: move a bundle of clone bundle related code to a new module Pierre-Yves David <pierre-yves.david@octobus.net> parents: diff changeset	365
74271829ddc0 clonebundles: move a bundle of clone bundle related code to a new module Pierre-Yves David <pierre-yves.david@octobus.net> parents: diff changeset	366 def parseclonebundlesmanifest(repo, s):
74271829ddc0 clonebundles: move a bundle of clone bundle related code to a new module Pierre-Yves David <pierre-yves.david@octobus.net> parents: diff changeset	367 """Parses the raw text of a clone bundles manifest.
74271829ddc0 clonebundles: move a bundle of clone bundle related code to a new module Pierre-Yves David <pierre-yves.david@octobus.net> parents: diff changeset	368
74271829ddc0 clonebundles: move a bundle of clone bundle related code to a new module Pierre-Yves David <pierre-yves.david@octobus.net> parents: diff changeset	369 Returns a list of dicts. The dicts have a ``URL`` key corresponding
74271829ddc0 clonebundles: move a bundle of clone bundle related code to a new module Pierre-Yves David <pierre-yves.david@octobus.net> parents: diff changeset	370 to the URL and other keys are the attributes for the entry.
74271829ddc0 clonebundles: move a bundle of clone bundle related code to a new module Pierre-Yves David <pierre-yves.david@octobus.net> parents: diff changeset	371 """
74271829ddc0 clonebundles: move a bundle of clone bundle related code to a new module Pierre-Yves David <pierre-yves.david@octobus.net> parents: diff changeset	372 m = []
74271829ddc0 clonebundles: move a bundle of clone bundle related code to a new module Pierre-Yves David <pierre-yves.david@octobus.net> parents: diff changeset	373 for line in s.splitlines():
74271829ddc0 clonebundles: move a bundle of clone bundle related code to a new module Pierre-Yves David <pierre-yves.david@octobus.net> parents: diff changeset	374 fields = line.split()
74271829ddc0 clonebundles: move a bundle of clone bundle related code to a new module Pierre-Yves David <pierre-yves.david@octobus.net> parents: diff changeset	375 if not fields:
74271829ddc0 clonebundles: move a bundle of clone bundle related code to a new module Pierre-Yves David <pierre-yves.david@octobus.net> parents: diff changeset	376 continue
74271829ddc0 clonebundles: move a bundle of clone bundle related code to a new module Pierre-Yves David <pierre-yves.david@octobus.net> parents: diff changeset	377 attrs = {b'URL': fields[0]}
74271829ddc0 clonebundles: move a bundle of clone bundle related code to a new module Pierre-Yves David <pierre-yves.david@octobus.net> parents: diff changeset	378 for rawattr in fields[1:]:
74271829ddc0 clonebundles: move a bundle of clone bundle related code to a new module Pierre-Yves David <pierre-yves.david@octobus.net> parents: diff changeset	379 key, value = rawattr.split(b'=', 1)
74271829ddc0 clonebundles: move a bundle of clone bundle related code to a new module Pierre-Yves David <pierre-yves.david@octobus.net> parents: diff changeset	380 key = util.urlreq.unquote(key)
74271829ddc0 clonebundles: move a bundle of clone bundle related code to a new module Pierre-Yves David <pierre-yves.david@octobus.net> parents: diff changeset	381 value = util.urlreq.unquote(value)
74271829ddc0 clonebundles: move a bundle of clone bundle related code to a new module Pierre-Yves David <pierre-yves.david@octobus.net> parents: diff changeset	382 attrs[key] = value
74271829ddc0 clonebundles: move a bundle of clone bundle related code to a new module Pierre-Yves David <pierre-yves.david@octobus.net> parents: diff changeset	383
74271829ddc0 clonebundles: move a bundle of clone bundle related code to a new module Pierre-Yves David <pierre-yves.david@octobus.net> parents: diff changeset	384 # Parse BUNDLESPEC into components. This makes client-side
74271829ddc0 clonebundles: move a bundle of clone bundle related code to a new module Pierre-Yves David <pierre-yves.david@octobus.net> parents: diff changeset	385 # preferences easier to specify since you can prefer a single
74271829ddc0 clonebundles: move a bundle of clone bundle related code to a new module Pierre-Yves David <pierre-yves.david@octobus.net> parents: diff changeset	386 # component of the BUNDLESPEC.
74271829ddc0 clonebundles: move a bundle of clone bundle related code to a new module Pierre-Yves David <pierre-yves.david@octobus.net> parents: diff changeset	387 if key == b'BUNDLESPEC':
74271829ddc0 clonebundles: move a bundle of clone bundle related code to a new module Pierre-Yves David <pierre-yves.david@octobus.net> parents: diff changeset	388 try:
74271829ddc0 clonebundles: move a bundle of clone bundle related code to a new module Pierre-Yves David <pierre-yves.david@octobus.net> parents: diff changeset	389 bundlespec = parsebundlespec(repo, value)
74271829ddc0 clonebundles: move a bundle of clone bundle related code to a new module Pierre-Yves David <pierre-yves.david@octobus.net> parents: diff changeset	390 attrs[b'COMPRESSION'] = bundlespec.compression
74271829ddc0 clonebundles: move a bundle of clone bundle related code to a new module Pierre-Yves David <pierre-yves.david@octobus.net> parents: diff changeset	391 attrs[b'VERSION'] = bundlespec.version
74271829ddc0 clonebundles: move a bundle of clone bundle related code to a new module Pierre-Yves David <pierre-yves.david@octobus.net> parents: diff changeset	392 except error.InvalidBundleSpecification:
74271829ddc0 clonebundles: move a bundle of clone bundle related code to a new module Pierre-Yves David <pierre-yves.david@octobus.net> parents: diff changeset	393 pass
74271829ddc0 clonebundles: move a bundle of clone bundle related code to a new module Pierre-Yves David <pierre-yves.david@octobus.net> parents: diff changeset	394 except error.UnsupportedBundleSpecification:
74271829ddc0 clonebundles: move a bundle of clone bundle related code to a new module Pierre-Yves David <pierre-yves.david@octobus.net> parents: diff changeset	395 pass
74271829ddc0 clonebundles: move a bundle of clone bundle related code to a new module Pierre-Yves David <pierre-yves.david@octobus.net> parents: diff changeset	396
74271829ddc0 clonebundles: move a bundle of clone bundle related code to a new module Pierre-Yves David <pierre-yves.david@octobus.net> parents: diff changeset	397 m.append(attrs)
74271829ddc0 clonebundles: move a bundle of clone bundle related code to a new module Pierre-Yves David <pierre-yves.david@octobus.net> parents: diff changeset	398
74271829ddc0 clonebundles: move a bundle of clone bundle related code to a new module Pierre-Yves David <pierre-yves.david@octobus.net> parents: diff changeset	399 return m
74271829ddc0 clonebundles: move a bundle of clone bundle related code to a new module Pierre-Yves David <pierre-yves.david@octobus.net> parents: diff changeset	400
74271829ddc0 clonebundles: move a bundle of clone bundle related code to a new module Pierre-Yves David <pierre-yves.david@octobus.net> parents: diff changeset	401
74271829ddc0 clonebundles: move a bundle of clone bundle related code to a new module Pierre-Yves David <pierre-yves.david@octobus.net> parents: diff changeset	402 def isstreamclonespec(bundlespec):
74271829ddc0 clonebundles: move a bundle of clone bundle related code to a new module Pierre-Yves David <pierre-yves.david@octobus.net> parents: diff changeset	403 # Stream clone v1
74271829ddc0 clonebundles: move a bundle of clone bundle related code to a new module Pierre-Yves David <pierre-yves.david@octobus.net> parents: diff changeset	404 if bundlespec.wirecompression == b'UN' and bundlespec.wireversion == b's1':
74271829ddc0 clonebundles: move a bundle of clone bundle related code to a new module Pierre-Yves David <pierre-yves.david@octobus.net> parents: diff changeset	405 return True
74271829ddc0 clonebundles: move a bundle of clone bundle related code to a new module Pierre-Yves David <pierre-yves.david@octobus.net> parents: diff changeset	406
74271829ddc0 clonebundles: move a bundle of clone bundle related code to a new module Pierre-Yves David <pierre-yves.david@octobus.net> parents: diff changeset	407 # Stream clone v2
74271829ddc0 clonebundles: move a bundle of clone bundle related code to a new module Pierre-Yves David <pierre-yves.david@octobus.net> parents: diff changeset	408 if (
74271829ddc0 clonebundles: move a bundle of clone bundle related code to a new module Pierre-Yves David <pierre-yves.david@octobus.net> parents: diff changeset	409 bundlespec.wirecompression == b'UN'
74271829ddc0 clonebundles: move a bundle of clone bundle related code to a new module Pierre-Yves David <pierre-yves.david@octobus.net> parents: diff changeset	410 and bundlespec.wireversion == b'02'
51555 c4aab3661f25 bundlespec: rationalize the way we specify stream bundle version Pierre-Yves David <pierre-yves.david@octobus.net> parents: 50709 diff changeset	411 and bundlespec.contentopts.get(b'stream', None) in (b"v2", b"v3-exp")
45784 74271829ddc0 clonebundles: move a bundle of clone bundle related code to a new module Pierre-Yves David <pierre-yves.david@octobus.net> parents: diff changeset	412 ):
74271829ddc0 clonebundles: move a bundle of clone bundle related code to a new module Pierre-Yves David <pierre-yves.david@octobus.net> parents: diff changeset	413 return True
74271829ddc0 clonebundles: move a bundle of clone bundle related code to a new module Pierre-Yves David <pierre-yves.david@octobus.net> parents: diff changeset	414
74271829ddc0 clonebundles: move a bundle of clone bundle related code to a new module Pierre-Yves David <pierre-yves.david@octobus.net> parents: diff changeset	415 return False
74271829ddc0 clonebundles: move a bundle of clone bundle related code to a new module Pierre-Yves David <pierre-yves.david@octobus.net> parents: diff changeset	416
74271829ddc0 clonebundles: move a bundle of clone bundle related code to a new module Pierre-Yves David <pierre-yves.david@octobus.net> parents: diff changeset	417
51978 aa7f4a45d8fa clonebundles: allow manifest to specify sha256 digest of bundles Joerg Sonnenberger <joerg@bec.de> parents: 51860 diff changeset	418 digest_regex = re.compile(b'^[a-z0-9]+:[0-9a-f]+(,[a-z0-9]+:[0-9a-f]+)*$')
aa7f4a45d8fa clonebundles: allow manifest to specify sha256 digest of bundles Joerg Sonnenberger <joerg@bec.de> parents: 51860 diff changeset	419
aa7f4a45d8fa clonebundles: allow manifest to specify sha256 digest of bundles Joerg Sonnenberger <joerg@bec.de> parents: 51860 diff changeset	420
50539 7b723217d368 clonebundles: filter out invalid schemes instead of failing on them Mathias De Mare <mathias.de_mare@nokia.com> parents: 49648 diff changeset	421 def filterclonebundleentries(
7b723217d368 clonebundles: filter out invalid schemes instead of failing on them Mathias De Mare <mathias.de_mare@nokia.com> parents: 49648 diff changeset	422 repo, entries, streamclonerequested=False, pullbundles=False
7b723217d368 clonebundles: filter out invalid schemes instead of failing on them Mathias De Mare <mathias.de_mare@nokia.com> parents: 49648 diff changeset	423 ):
45784 74271829ddc0 clonebundles: move a bundle of clone bundle related code to a new module Pierre-Yves David <pierre-yves.david@octobus.net> parents: diff changeset	424 """Remove incompatible clone bundle manifest entries.
74271829ddc0 clonebundles: move a bundle of clone bundle related code to a new module Pierre-Yves David <pierre-yves.david@octobus.net> parents: diff changeset	425
74271829ddc0 clonebundles: move a bundle of clone bundle related code to a new module Pierre-Yves David <pierre-yves.david@octobus.net> parents: diff changeset	426 Accepts a list of entries parsed with ``parseclonebundlesmanifest``
74271829ddc0 clonebundles: move a bundle of clone bundle related code to a new module Pierre-Yves David <pierre-yves.david@octobus.net> parents: diff changeset	427 and returns a new list consisting of only the entries that this client
74271829ddc0 clonebundles: move a bundle of clone bundle related code to a new module Pierre-Yves David <pierre-yves.david@octobus.net> parents: diff changeset	428 should be able to apply.
74271829ddc0 clonebundles: move a bundle of clone bundle related code to a new module Pierre-Yves David <pierre-yves.david@octobus.net> parents: diff changeset	429
74271829ddc0 clonebundles: move a bundle of clone bundle related code to a new module Pierre-Yves David <pierre-yves.david@octobus.net> parents: diff changeset	430 There is no guarantee we'll be able to apply all returned entries because
74271829ddc0 clonebundles: move a bundle of clone bundle related code to a new module Pierre-Yves David <pierre-yves.david@octobus.net> parents: diff changeset	431 the metadata we use to filter on may be missing or wrong.
74271829ddc0 clonebundles: move a bundle of clone bundle related code to a new module Pierre-Yves David <pierre-yves.david@octobus.net> parents: diff changeset	432 """
74271829ddc0 clonebundles: move a bundle of clone bundle related code to a new module Pierre-Yves David <pierre-yves.david@octobus.net> parents: diff changeset	433 newentries = []
74271829ddc0 clonebundles: move a bundle of clone bundle related code to a new module Pierre-Yves David <pierre-yves.david@octobus.net> parents: diff changeset	434 for entry in entries:
50539 7b723217d368 clonebundles: filter out invalid schemes instead of failing on them Mathias De Mare <mathias.de_mare@nokia.com> parents: 49648 diff changeset	435 url = entry.get(b'URL')
7b723217d368 clonebundles: filter out invalid schemes instead of failing on them Mathias De Mare <mathias.de_mare@nokia.com> parents: 49648 diff changeset	436 if not pullbundles and not any(
7b723217d368 clonebundles: filter out invalid schemes instead of failing on them Mathias De Mare <mathias.de_mare@nokia.com> parents: 49648 diff changeset	437 [url.startswith(scheme) for scheme in SUPPORTED_CLONEBUNDLE_SCHEMES]
7b723217d368 clonebundles: filter out invalid schemes instead of failing on them Mathias De Mare <mathias.de_mare@nokia.com> parents: 49648 diff changeset	438 ):
7b723217d368 clonebundles: filter out invalid schemes instead of failing on them Mathias De Mare <mathias.de_mare@nokia.com> parents: 49648 diff changeset	439 repo.ui.debug(
7b723217d368 clonebundles: filter out invalid schemes instead of failing on them Mathias De Mare <mathias.de_mare@nokia.com> parents: 49648 diff changeset	440 b'filtering %s because not a supported clonebundle scheme\n'
7b723217d368 clonebundles: filter out invalid schemes instead of failing on them Mathias De Mare <mathias.de_mare@nokia.com> parents: 49648 diff changeset	441 % url
7b723217d368 clonebundles: filter out invalid schemes instead of failing on them Mathias De Mare <mathias.de_mare@nokia.com> parents: 49648 diff changeset	442 )
7b723217d368 clonebundles: filter out invalid schemes instead of failing on them Mathias De Mare <mathias.de_mare@nokia.com> parents: 49648 diff changeset	443 continue
7b723217d368 clonebundles: filter out invalid schemes instead of failing on them Mathias De Mare <mathias.de_mare@nokia.com> parents: 49648 diff changeset	444
45784 74271829ddc0 clonebundles: move a bundle of clone bundle related code to a new module Pierre-Yves David <pierre-yves.david@octobus.net> parents: diff changeset	445 spec = entry.get(b'BUNDLESPEC')
74271829ddc0 clonebundles: move a bundle of clone bundle related code to a new module Pierre-Yves David <pierre-yves.david@octobus.net> parents: diff changeset	446 if spec:
74271829ddc0 clonebundles: move a bundle of clone bundle related code to a new module Pierre-Yves David <pierre-yves.david@octobus.net> parents: diff changeset	447 try:
74271829ddc0 clonebundles: move a bundle of clone bundle related code to a new module Pierre-Yves David <pierre-yves.david@octobus.net> parents: diff changeset	448 bundlespec = parsebundlespec(repo, spec, strict=True)
74271829ddc0 clonebundles: move a bundle of clone bundle related code to a new module Pierre-Yves David <pierre-yves.david@octobus.net> parents: diff changeset	449
74271829ddc0 clonebundles: move a bundle of clone bundle related code to a new module Pierre-Yves David <pierre-yves.david@octobus.net> parents: diff changeset	450 # If a stream clone was requested, filter out non-streamclone
74271829ddc0 clonebundles: move a bundle of clone bundle related code to a new module Pierre-Yves David <pierre-yves.david@octobus.net> parents: diff changeset	451 # entries.
74271829ddc0 clonebundles: move a bundle of clone bundle related code to a new module Pierre-Yves David <pierre-yves.david@octobus.net> parents: diff changeset	452 if streamclonerequested and not isstreamclonespec(bundlespec):
74271829ddc0 clonebundles: move a bundle of clone bundle related code to a new module Pierre-Yves David <pierre-yves.david@octobus.net> parents: diff changeset	453 repo.ui.debug(
50539 7b723217d368 clonebundles: filter out invalid schemes instead of failing on them Mathias De Mare <mathias.de_mare@nokia.com> parents: 49648 diff changeset	454 b'filtering %s because not a stream clone\n' % url
45784 74271829ddc0 clonebundles: move a bundle of clone bundle related code to a new module Pierre-Yves David <pierre-yves.david@octobus.net> parents: diff changeset	455 )
74271829ddc0 clonebundles: move a bundle of clone bundle related code to a new module Pierre-Yves David <pierre-yves.david@octobus.net> parents: diff changeset	456 continue
74271829ddc0 clonebundles: move a bundle of clone bundle related code to a new module Pierre-Yves David <pierre-yves.david@octobus.net> parents: diff changeset	457
74271829ddc0 clonebundles: move a bundle of clone bundle related code to a new module Pierre-Yves David <pierre-yves.david@octobus.net> parents: diff changeset	458 except error.InvalidBundleSpecification as e:
74271829ddc0 clonebundles: move a bundle of clone bundle related code to a new module Pierre-Yves David <pierre-yves.david@octobus.net> parents: diff changeset	459 repo.ui.debug(stringutil.forcebytestr(e) + b'\n')
74271829ddc0 clonebundles: move a bundle of clone bundle related code to a new module Pierre-Yves David <pierre-yves.david@octobus.net> parents: diff changeset	460 continue
74271829ddc0 clonebundles: move a bundle of clone bundle related code to a new module Pierre-Yves David <pierre-yves.david@octobus.net> parents: diff changeset	461 except error.UnsupportedBundleSpecification as e:
74271829ddc0 clonebundles: move a bundle of clone bundle related code to a new module Pierre-Yves David <pierre-yves.david@octobus.net> parents: diff changeset	462 repo.ui.debug(
74271829ddc0 clonebundles: move a bundle of clone bundle related code to a new module Pierre-Yves David <pierre-yves.david@octobus.net> parents: diff changeset	463 b'filtering %s because unsupported bundle '
50539 7b723217d368 clonebundles: filter out invalid schemes instead of failing on them Mathias De Mare <mathias.de_mare@nokia.com> parents: 49648 diff changeset	464 b'spec: %s\n' % (url, stringutil.forcebytestr(e))
45784 74271829ddc0 clonebundles: move a bundle of clone bundle related code to a new module Pierre-Yves David <pierre-yves.david@octobus.net> parents: diff changeset	465 )
74271829ddc0 clonebundles: move a bundle of clone bundle related code to a new module Pierre-Yves David <pierre-yves.david@octobus.net> parents: diff changeset	466 continue
74271829ddc0 clonebundles: move a bundle of clone bundle related code to a new module Pierre-Yves David <pierre-yves.david@octobus.net> parents: diff changeset	467 # If we don't have a spec and requested a stream clone, we don't know
74271829ddc0 clonebundles: move a bundle of clone bundle related code to a new module Pierre-Yves David <pierre-yves.david@octobus.net> parents: diff changeset	468 # what the entry is so don't attempt to apply it.
74271829ddc0 clonebundles: move a bundle of clone bundle related code to a new module Pierre-Yves David <pierre-yves.david@octobus.net> parents: diff changeset	469 elif streamclonerequested:
74271829ddc0 clonebundles: move a bundle of clone bundle related code to a new module Pierre-Yves David <pierre-yves.david@octobus.net> parents: diff changeset	470 repo.ui.debug(
74271829ddc0 clonebundles: move a bundle of clone bundle related code to a new module Pierre-Yves David <pierre-yves.david@octobus.net> parents: diff changeset	471 b'filtering %s because cannot determine if a stream '
50539 7b723217d368 clonebundles: filter out invalid schemes instead of failing on them Mathias De Mare <mathias.de_mare@nokia.com> parents: 49648 diff changeset	472 b'clone bundle\n' % url
45784 74271829ddc0 clonebundles: move a bundle of clone bundle related code to a new module Pierre-Yves David <pierre-yves.david@octobus.net> parents: diff changeset	473 )
74271829ddc0 clonebundles: move a bundle of clone bundle related code to a new module Pierre-Yves David <pierre-yves.david@octobus.net> parents: diff changeset	474 continue
74271829ddc0 clonebundles: move a bundle of clone bundle related code to a new module Pierre-Yves David <pierre-yves.david@octobus.net> parents: diff changeset	475
74271829ddc0 clonebundles: move a bundle of clone bundle related code to a new module Pierre-Yves David <pierre-yves.david@octobus.net> parents: diff changeset	476 if b'REQUIRESNI' in entry and not sslutil.hassni:
50539 7b723217d368 clonebundles: filter out invalid schemes instead of failing on them Mathias De Mare <mathias.de_mare@nokia.com> parents: 49648 diff changeset	477 repo.ui.debug(b'filtering %s because SNI not supported\n' % url)
45784 74271829ddc0 clonebundles: move a bundle of clone bundle related code to a new module Pierre-Yves David <pierre-yves.david@octobus.net> parents: diff changeset	478 continue
74271829ddc0 clonebundles: move a bundle of clone bundle related code to a new module Pierre-Yves David <pierre-yves.david@octobus.net> parents: diff changeset	479
74271829ddc0 clonebundles: move a bundle of clone bundle related code to a new module Pierre-Yves David <pierre-yves.david@octobus.net> parents: diff changeset	480 if b'REQUIREDRAM' in entry:
74271829ddc0 clonebundles: move a bundle of clone bundle related code to a new module Pierre-Yves David <pierre-yves.david@octobus.net> parents: diff changeset	481 try:
74271829ddc0 clonebundles: move a bundle of clone bundle related code to a new module Pierre-Yves David <pierre-yves.david@octobus.net> parents: diff changeset	482 requiredram = util.sizetoint(entry[b'REQUIREDRAM'])
74271829ddc0 clonebundles: move a bundle of clone bundle related code to a new module Pierre-Yves David <pierre-yves.david@octobus.net> parents: diff changeset	483 except error.ParseError:
74271829ddc0 clonebundles: move a bundle of clone bundle related code to a new module Pierre-Yves David <pierre-yves.david@octobus.net> parents: diff changeset	484 repo.ui.debug(
50539 7b723217d368 clonebundles: filter out invalid schemes instead of failing on them Mathias De Mare <mathias.de_mare@nokia.com> parents: 49648 diff changeset	485 b'filtering %s due to a bad REQUIREDRAM attribute\n' % url
45784 74271829ddc0 clonebundles: move a bundle of clone bundle related code to a new module Pierre-Yves David <pierre-yves.david@octobus.net> parents: diff changeset	486 )
74271829ddc0 clonebundles: move a bundle of clone bundle related code to a new module Pierre-Yves David <pierre-yves.david@octobus.net> parents: diff changeset	487 continue
74271829ddc0 clonebundles: move a bundle of clone bundle related code to a new module Pierre-Yves David <pierre-yves.david@octobus.net> parents: diff changeset	488 actualram = repo.ui.estimatememory()
74271829ddc0 clonebundles: move a bundle of clone bundle related code to a new module Pierre-Yves David <pierre-yves.david@octobus.net> parents: diff changeset	489 if actualram is not None and actualram * 0.66 < requiredram:
74271829ddc0 clonebundles: move a bundle of clone bundle related code to a new module Pierre-Yves David <pierre-yves.david@octobus.net> parents: diff changeset	490 repo.ui.debug(
74271829ddc0 clonebundles: move a bundle of clone bundle related code to a new module Pierre-Yves David <pierre-yves.david@octobus.net> parents: diff changeset	491 b'filtering %s as it needs more than 2/3 of system memory\n'
50539 7b723217d368 clonebundles: filter out invalid schemes instead of failing on them Mathias De Mare <mathias.de_mare@nokia.com> parents: 49648 diff changeset	492 % url
45784 74271829ddc0 clonebundles: move a bundle of clone bundle related code to a new module Pierre-Yves David <pierre-yves.david@octobus.net> parents: diff changeset	493 )
74271829ddc0 clonebundles: move a bundle of clone bundle related code to a new module Pierre-Yves David <pierre-yves.david@octobus.net> parents: diff changeset	494 continue
74271829ddc0 clonebundles: move a bundle of clone bundle related code to a new module Pierre-Yves David <pierre-yves.david@octobus.net> parents: diff changeset	495
51978 aa7f4a45d8fa clonebundles: allow manifest to specify sha256 digest of bundles Joerg Sonnenberger <joerg@bec.de> parents: 51860 diff changeset	496 if b'DIGEST' in entry:
aa7f4a45d8fa clonebundles: allow manifest to specify sha256 digest of bundles Joerg Sonnenberger <joerg@bec.de> parents: 51860 diff changeset	497 if not digest_regex.match(entry[b'DIGEST']):
aa7f4a45d8fa clonebundles: allow manifest to specify sha256 digest of bundles Joerg Sonnenberger <joerg@bec.de> parents: 51860 diff changeset	498 repo.ui.debug(
aa7f4a45d8fa clonebundles: allow manifest to specify sha256 digest of bundles Joerg Sonnenberger <joerg@bec.de> parents: 51860 diff changeset	499 b'filtering %s due to a bad DIGEST attribute\n' % url
aa7f4a45d8fa clonebundles: allow manifest to specify sha256 digest of bundles Joerg Sonnenberger <joerg@bec.de> parents: 51860 diff changeset	500 )
aa7f4a45d8fa clonebundles: allow manifest to specify sha256 digest of bundles Joerg Sonnenberger <joerg@bec.de> parents: 51860 diff changeset	501 continue
aa7f4a45d8fa clonebundles: allow manifest to specify sha256 digest of bundles Joerg Sonnenberger <joerg@bec.de> parents: 51860 diff changeset	502 supported = 0
aa7f4a45d8fa clonebundles: allow manifest to specify sha256 digest of bundles Joerg Sonnenberger <joerg@bec.de> parents: 51860 diff changeset	503 seen = {}
aa7f4a45d8fa clonebundles: allow manifest to specify sha256 digest of bundles Joerg Sonnenberger <joerg@bec.de> parents: 51860 diff changeset	504 for digest_entry in entry[b'DIGEST'].split(b','):
aa7f4a45d8fa clonebundles: allow manifest to specify sha256 digest of bundles Joerg Sonnenberger <joerg@bec.de> parents: 51860 diff changeset	505 algo, digest = digest_entry.split(b':')
aa7f4a45d8fa clonebundles: allow manifest to specify sha256 digest of bundles Joerg Sonnenberger <joerg@bec.de> parents: 51860 diff changeset	506 if algo not in seen:
aa7f4a45d8fa clonebundles: allow manifest to specify sha256 digest of bundles Joerg Sonnenberger <joerg@bec.de> parents: 51860 diff changeset	507 seen[algo] = digest
aa7f4a45d8fa clonebundles: allow manifest to specify sha256 digest of bundles Joerg Sonnenberger <joerg@bec.de> parents: 51860 diff changeset	508 elif seen[algo] != digest:
aa7f4a45d8fa clonebundles: allow manifest to specify sha256 digest of bundles Joerg Sonnenberger <joerg@bec.de> parents: 51860 diff changeset	509 repo.ui.debug(
aa7f4a45d8fa clonebundles: allow manifest to specify sha256 digest of bundles Joerg Sonnenberger <joerg@bec.de> parents: 51860 diff changeset	510 b'filtering %s due to conflicting %s digests\n'
aa7f4a45d8fa clonebundles: allow manifest to specify sha256 digest of bundles Joerg Sonnenberger <joerg@bec.de> parents: 51860 diff changeset	511 % (url, algo)
aa7f4a45d8fa clonebundles: allow manifest to specify sha256 digest of bundles Joerg Sonnenberger <joerg@bec.de> parents: 51860 diff changeset	512 )
aa7f4a45d8fa clonebundles: allow manifest to specify sha256 digest of bundles Joerg Sonnenberger <joerg@bec.de> parents: 51860 diff changeset	513 supported = 0
aa7f4a45d8fa clonebundles: allow manifest to specify sha256 digest of bundles Joerg Sonnenberger <joerg@bec.de> parents: 51860 diff changeset	514 break
aa7f4a45d8fa clonebundles: allow manifest to specify sha256 digest of bundles Joerg Sonnenberger <joerg@bec.de> parents: 51860 diff changeset	515 digester = urlmod.digesthandler.digest_algorithms.get(algo)
aa7f4a45d8fa clonebundles: allow manifest to specify sha256 digest of bundles Joerg Sonnenberger <joerg@bec.de> parents: 51860 diff changeset	516 if digester is None:
aa7f4a45d8fa clonebundles: allow manifest to specify sha256 digest of bundles Joerg Sonnenberger <joerg@bec.de> parents: 51860 diff changeset	517 continue
aa7f4a45d8fa clonebundles: allow manifest to specify sha256 digest of bundles Joerg Sonnenberger <joerg@bec.de> parents: 51860 diff changeset	518 if len(digest) != digester().digest_size * 2:
aa7f4a45d8fa clonebundles: allow manifest to specify sha256 digest of bundles Joerg Sonnenberger <joerg@bec.de> parents: 51860 diff changeset	519 repo.ui.debug(
aa7f4a45d8fa clonebundles: allow manifest to specify sha256 digest of bundles Joerg Sonnenberger <joerg@bec.de> parents: 51860 diff changeset	520 b'filtering %s due to a bad %s digest\n' % (url, algo)
aa7f4a45d8fa clonebundles: allow manifest to specify sha256 digest of bundles Joerg Sonnenberger <joerg@bec.de> parents: 51860 diff changeset	521 )
aa7f4a45d8fa clonebundles: allow manifest to specify sha256 digest of bundles Joerg Sonnenberger <joerg@bec.de> parents: 51860 diff changeset	522 supported = 0
aa7f4a45d8fa clonebundles: allow manifest to specify sha256 digest of bundles Joerg Sonnenberger <joerg@bec.de> parents: 51860 diff changeset	523 break
aa7f4a45d8fa clonebundles: allow manifest to specify sha256 digest of bundles Joerg Sonnenberger <joerg@bec.de> parents: 51860 diff changeset	524 supported += 1
aa7f4a45d8fa clonebundles: allow manifest to specify sha256 digest of bundles Joerg Sonnenberger <joerg@bec.de> parents: 51860 diff changeset	525 else:
aa7f4a45d8fa clonebundles: allow manifest to specify sha256 digest of bundles Joerg Sonnenberger <joerg@bec.de> parents: 51860 diff changeset	526 if supported == 0:
aa7f4a45d8fa clonebundles: allow manifest to specify sha256 digest of bundles Joerg Sonnenberger <joerg@bec.de> parents: 51860 diff changeset	527 repo.ui.debug(
aa7f4a45d8fa clonebundles: allow manifest to specify sha256 digest of bundles Joerg Sonnenberger <joerg@bec.de> parents: 51860 diff changeset	528 b'filtering %s due to lack of supported digest\n' % url
aa7f4a45d8fa clonebundles: allow manifest to specify sha256 digest of bundles Joerg Sonnenberger <joerg@bec.de> parents: 51860 diff changeset	529 )
aa7f4a45d8fa clonebundles: allow manifest to specify sha256 digest of bundles Joerg Sonnenberger <joerg@bec.de> parents: 51860 diff changeset	530 if supported == 0:
aa7f4a45d8fa clonebundles: allow manifest to specify sha256 digest of bundles Joerg Sonnenberger <joerg@bec.de> parents: 51860 diff changeset	531 continue
aa7f4a45d8fa clonebundles: allow manifest to specify sha256 digest of bundles Joerg Sonnenberger <joerg@bec.de> parents: 51860 diff changeset	532
45784 74271829ddc0 clonebundles: move a bundle of clone bundle related code to a new module Pierre-Yves David <pierre-yves.david@octobus.net> parents: diff changeset	533 newentries.append(entry)
74271829ddc0 clonebundles: move a bundle of clone bundle related code to a new module Pierre-Yves David <pierre-yves.david@octobus.net> parents: diff changeset	534
74271829ddc0 clonebundles: move a bundle of clone bundle related code to a new module Pierre-Yves David <pierre-yves.david@octobus.net> parents: diff changeset	535 return newentries
74271829ddc0 clonebundles: move a bundle of clone bundle related code to a new module Pierre-Yves David <pierre-yves.david@octobus.net> parents: diff changeset	536
74271829ddc0 clonebundles: move a bundle of clone bundle related code to a new module Pierre-Yves David <pierre-yves.david@octobus.net> parents: diff changeset	537
48946 642e31cb55f0 py3: use class X: instead of class X(object): Gregory Szorc <gregory.szorc@gmail.com> parents: 48601 diff changeset	538 class clonebundleentry:
45784 74271829ddc0 clonebundles: move a bundle of clone bundle related code to a new module Pierre-Yves David <pierre-yves.david@octobus.net> parents: diff changeset	539 """Represents an item in a clone bundles manifest.
74271829ddc0 clonebundles: move a bundle of clone bundle related code to a new module Pierre-Yves David <pierre-yves.david@octobus.net> parents: diff changeset	540
74271829ddc0 clonebundles: move a bundle of clone bundle related code to a new module Pierre-Yves David <pierre-yves.david@octobus.net> parents: diff changeset	541 This rich class is needed to support sorting since sorted() in Python 3
74271829ddc0 clonebundles: move a bundle of clone bundle related code to a new module Pierre-Yves David <pierre-yves.david@octobus.net> parents: diff changeset	542 doesn't support ``cmp`` and our comparison is complex enough that ``key=``
74271829ddc0 clonebundles: move a bundle of clone bundle related code to a new module Pierre-Yves David <pierre-yves.david@octobus.net> parents: diff changeset	543 won't work.
74271829ddc0 clonebundles: move a bundle of clone bundle related code to a new module Pierre-Yves David <pierre-yves.david@octobus.net> parents: diff changeset	544 """
74271829ddc0 clonebundles: move a bundle of clone bundle related code to a new module Pierre-Yves David <pierre-yves.david@octobus.net> parents: diff changeset	545
74271829ddc0 clonebundles: move a bundle of clone bundle related code to a new module Pierre-Yves David <pierre-yves.david@octobus.net> parents: diff changeset	546 def __init__(self, value, prefers):
74271829ddc0 clonebundles: move a bundle of clone bundle related code to a new module Pierre-Yves David <pierre-yves.david@octobus.net> parents: diff changeset	547 self.value = value
74271829ddc0 clonebundles: move a bundle of clone bundle related code to a new module Pierre-Yves David <pierre-yves.david@octobus.net> parents: diff changeset	548 self.prefers = prefers
74271829ddc0 clonebundles: move a bundle of clone bundle related code to a new module Pierre-Yves David <pierre-yves.david@octobus.net> parents: diff changeset	549
74271829ddc0 clonebundles: move a bundle of clone bundle related code to a new module Pierre-Yves David <pierre-yves.david@octobus.net> parents: diff changeset	550 def _cmp(self, other):
74271829ddc0 clonebundles: move a bundle of clone bundle related code to a new module Pierre-Yves David <pierre-yves.david@octobus.net> parents: diff changeset	551 for prefkey, prefvalue in self.prefers:
74271829ddc0 clonebundles: move a bundle of clone bundle related code to a new module Pierre-Yves David <pierre-yves.david@octobus.net> parents: diff changeset	552 avalue = self.value.get(prefkey)
74271829ddc0 clonebundles: move a bundle of clone bundle related code to a new module Pierre-Yves David <pierre-yves.david@octobus.net> parents: diff changeset	553 bvalue = other.value.get(prefkey)
74271829ddc0 clonebundles: move a bundle of clone bundle related code to a new module Pierre-Yves David <pierre-yves.david@octobus.net> parents: diff changeset	554
74271829ddc0 clonebundles: move a bundle of clone bundle related code to a new module Pierre-Yves David <pierre-yves.david@octobus.net> parents: diff changeset	555 # Special case for b missing attribute and a matches exactly.
74271829ddc0 clonebundles: move a bundle of clone bundle related code to a new module Pierre-Yves David <pierre-yves.david@octobus.net> parents: diff changeset	556 if avalue is not None and bvalue is None and avalue == prefvalue:
74271829ddc0 clonebundles: move a bundle of clone bundle related code to a new module Pierre-Yves David <pierre-yves.david@octobus.net> parents: diff changeset	557 return -1
74271829ddc0 clonebundles: move a bundle of clone bundle related code to a new module Pierre-Yves David <pierre-yves.david@octobus.net> parents: diff changeset	558
74271829ddc0 clonebundles: move a bundle of clone bundle related code to a new module Pierre-Yves David <pierre-yves.david@octobus.net> parents: diff changeset	559 # Special case for a missing attribute and b matches exactly.
74271829ddc0 clonebundles: move a bundle of clone bundle related code to a new module Pierre-Yves David <pierre-yves.david@octobus.net> parents: diff changeset	560 if bvalue is not None and avalue is None and bvalue == prefvalue:
74271829ddc0 clonebundles: move a bundle of clone bundle related code to a new module Pierre-Yves David <pierre-yves.david@octobus.net> parents: diff changeset	561 return 1
74271829ddc0 clonebundles: move a bundle of clone bundle related code to a new module Pierre-Yves David <pierre-yves.david@octobus.net> parents: diff changeset	562
74271829ddc0 clonebundles: move a bundle of clone bundle related code to a new module Pierre-Yves David <pierre-yves.david@octobus.net> parents: diff changeset	563 # We can't compare unless attribute present on both.
74271829ddc0 clonebundles: move a bundle of clone bundle related code to a new module Pierre-Yves David <pierre-yves.david@octobus.net> parents: diff changeset	564 if avalue is None or bvalue is None:
74271829ddc0 clonebundles: move a bundle of clone bundle related code to a new module Pierre-Yves David <pierre-yves.david@octobus.net> parents: diff changeset	565 continue
74271829ddc0 clonebundles: move a bundle of clone bundle related code to a new module Pierre-Yves David <pierre-yves.david@octobus.net> parents: diff changeset	566
74271829ddc0 clonebundles: move a bundle of clone bundle related code to a new module Pierre-Yves David <pierre-yves.david@octobus.net> parents: diff changeset	567 # Same values should fall back to next attribute.
74271829ddc0 clonebundles: move a bundle of clone bundle related code to a new module Pierre-Yves David <pierre-yves.david@octobus.net> parents: diff changeset	568 if avalue == bvalue:
74271829ddc0 clonebundles: move a bundle of clone bundle related code to a new module Pierre-Yves David <pierre-yves.david@octobus.net> parents: diff changeset	569 continue
74271829ddc0 clonebundles: move a bundle of clone bundle related code to a new module Pierre-Yves David <pierre-yves.david@octobus.net> parents: diff changeset	570
74271829ddc0 clonebundles: move a bundle of clone bundle related code to a new module Pierre-Yves David <pierre-yves.david@octobus.net> parents: diff changeset	571 # Exact matches come first.
74271829ddc0 clonebundles: move a bundle of clone bundle related code to a new module Pierre-Yves David <pierre-yves.david@octobus.net> parents: diff changeset	572 if avalue == prefvalue:
74271829ddc0 clonebundles: move a bundle of clone bundle related code to a new module Pierre-Yves David <pierre-yves.david@octobus.net> parents: diff changeset	573 return -1
74271829ddc0 clonebundles: move a bundle of clone bundle related code to a new module Pierre-Yves David <pierre-yves.david@octobus.net> parents: diff changeset	574 if bvalue == prefvalue:
74271829ddc0 clonebundles: move a bundle of clone bundle related code to a new module Pierre-Yves David <pierre-yves.david@octobus.net> parents: diff changeset	575 return 1
74271829ddc0 clonebundles: move a bundle of clone bundle related code to a new module Pierre-Yves David <pierre-yves.david@octobus.net> parents: diff changeset	576
74271829ddc0 clonebundles: move a bundle of clone bundle related code to a new module Pierre-Yves David <pierre-yves.david@octobus.net> parents: diff changeset	577 # Fall back to next attribute.
74271829ddc0 clonebundles: move a bundle of clone bundle related code to a new module Pierre-Yves David <pierre-yves.david@octobus.net> parents: diff changeset	578 continue
74271829ddc0 clonebundles: move a bundle of clone bundle related code to a new module Pierre-Yves David <pierre-yves.david@octobus.net> parents: diff changeset	579
74271829ddc0 clonebundles: move a bundle of clone bundle related code to a new module Pierre-Yves David <pierre-yves.david@octobus.net> parents: diff changeset	580 # If we got here we couldn't sort by attributes and prefers. Fall
74271829ddc0 clonebundles: move a bundle of clone bundle related code to a new module Pierre-Yves David <pierre-yves.david@octobus.net> parents: diff changeset	581 # back to index order.
74271829ddc0 clonebundles: move a bundle of clone bundle related code to a new module Pierre-Yves David <pierre-yves.david@octobus.net> parents: diff changeset	582 return 0
74271829ddc0 clonebundles: move a bundle of clone bundle related code to a new module Pierre-Yves David <pierre-yves.david@octobus.net> parents: diff changeset	583
74271829ddc0 clonebundles: move a bundle of clone bundle related code to a new module Pierre-Yves David <pierre-yves.david@octobus.net> parents: diff changeset	584 def __lt__(self, other):
74271829ddc0 clonebundles: move a bundle of clone bundle related code to a new module Pierre-Yves David <pierre-yves.david@octobus.net> parents: diff changeset	585 return self._cmp(other) < 0
74271829ddc0 clonebundles: move a bundle of clone bundle related code to a new module Pierre-Yves David <pierre-yves.david@octobus.net> parents: diff changeset	586
74271829ddc0 clonebundles: move a bundle of clone bundle related code to a new module Pierre-Yves David <pierre-yves.david@octobus.net> parents: diff changeset	587 def __gt__(self, other):
74271829ddc0 clonebundles: move a bundle of clone bundle related code to a new module Pierre-Yves David <pierre-yves.david@octobus.net> parents: diff changeset	588 return self._cmp(other) > 0
74271829ddc0 clonebundles: move a bundle of clone bundle related code to a new module Pierre-Yves David <pierre-yves.david@octobus.net> parents: diff changeset	589
74271829ddc0 clonebundles: move a bundle of clone bundle related code to a new module Pierre-Yves David <pierre-yves.david@octobus.net> parents: diff changeset	590 def __eq__(self, other):
74271829ddc0 clonebundles: move a bundle of clone bundle related code to a new module Pierre-Yves David <pierre-yves.david@octobus.net> parents: diff changeset	591 return self._cmp(other) == 0
74271829ddc0 clonebundles: move a bundle of clone bundle related code to a new module Pierre-Yves David <pierre-yves.david@octobus.net> parents: diff changeset	592
74271829ddc0 clonebundles: move a bundle of clone bundle related code to a new module Pierre-Yves David <pierre-yves.david@octobus.net> parents: diff changeset	593 def __le__(self, other):
74271829ddc0 clonebundles: move a bundle of clone bundle related code to a new module Pierre-Yves David <pierre-yves.david@octobus.net> parents: diff changeset	594 return self._cmp(other) <= 0
74271829ddc0 clonebundles: move a bundle of clone bundle related code to a new module Pierre-Yves David <pierre-yves.david@octobus.net> parents: diff changeset	595
74271829ddc0 clonebundles: move a bundle of clone bundle related code to a new module Pierre-Yves David <pierre-yves.david@octobus.net> parents: diff changeset	596 def __ge__(self, other):
74271829ddc0 clonebundles: move a bundle of clone bundle related code to a new module Pierre-Yves David <pierre-yves.david@octobus.net> parents: diff changeset	597 return self._cmp(other) >= 0
74271829ddc0 clonebundles: move a bundle of clone bundle related code to a new module Pierre-Yves David <pierre-yves.david@octobus.net> parents: diff changeset	598
74271829ddc0 clonebundles: move a bundle of clone bundle related code to a new module Pierre-Yves David <pierre-yves.david@octobus.net> parents: diff changeset	599 def __ne__(self, other):
74271829ddc0 clonebundles: move a bundle of clone bundle related code to a new module Pierre-Yves David <pierre-yves.david@octobus.net> parents: diff changeset	600 return self._cmp(other) != 0
74271829ddc0 clonebundles: move a bundle of clone bundle related code to a new module Pierre-Yves David <pierre-yves.david@octobus.net> parents: diff changeset	601
74271829ddc0 clonebundles: move a bundle of clone bundle related code to a new module Pierre-Yves David <pierre-yves.david@octobus.net> parents: diff changeset	602
74271829ddc0 clonebundles: move a bundle of clone bundle related code to a new module Pierre-Yves David <pierre-yves.david@octobus.net> parents: diff changeset	603 def sortclonebundleentries(ui, entries):
74271829ddc0 clonebundles: move a bundle of clone bundle related code to a new module Pierre-Yves David <pierre-yves.david@octobus.net> parents: diff changeset	604 prefers = ui.configlist(b'ui', b'clonebundleprefers')
74271829ddc0 clonebundles: move a bundle of clone bundle related code to a new module Pierre-Yves David <pierre-yves.david@octobus.net> parents: diff changeset	605 if not prefers:
74271829ddc0 clonebundles: move a bundle of clone bundle related code to a new module Pierre-Yves David <pierre-yves.david@octobus.net> parents: diff changeset	606 return list(entries)
74271829ddc0 clonebundles: move a bundle of clone bundle related code to a new module Pierre-Yves David <pierre-yves.david@octobus.net> parents: diff changeset	607
74271829ddc0 clonebundles: move a bundle of clone bundle related code to a new module Pierre-Yves David <pierre-yves.david@octobus.net> parents: diff changeset	608 def _split(p):
74271829ddc0 clonebundles: move a bundle of clone bundle related code to a new module Pierre-Yves David <pierre-yves.david@octobus.net> parents: diff changeset	609 if b'=' not in p:
74271829ddc0 clonebundles: move a bundle of clone bundle related code to a new module Pierre-Yves David <pierre-yves.david@octobus.net> parents: diff changeset	610 hint = _(b"each comma separated item should be key=value pairs")
74271829ddc0 clonebundles: move a bundle of clone bundle related code to a new module Pierre-Yves David <pierre-yves.david@octobus.net> parents: diff changeset	611 raise error.Abort(
74271829ddc0 clonebundles: move a bundle of clone bundle related code to a new module Pierre-Yves David <pierre-yves.david@octobus.net> parents: diff changeset	612 _(b"invalid ui.clonebundleprefers item: %s") % p, hint=hint
74271829ddc0 clonebundles: move a bundle of clone bundle related code to a new module Pierre-Yves David <pierre-yves.david@octobus.net> parents: diff changeset	613 )
74271829ddc0 clonebundles: move a bundle of clone bundle related code to a new module Pierre-Yves David <pierre-yves.david@octobus.net> parents: diff changeset	614 return p.split(b'=', 1)
74271829ddc0 clonebundles: move a bundle of clone bundle related code to a new module Pierre-Yves David <pierre-yves.david@octobus.net> parents: diff changeset	615
74271829ddc0 clonebundles: move a bundle of clone bundle related code to a new module Pierre-Yves David <pierre-yves.david@octobus.net> parents: diff changeset	616 prefers = [_split(p) for p in prefers]
74271829ddc0 clonebundles: move a bundle of clone bundle related code to a new module Pierre-Yves David <pierre-yves.david@octobus.net> parents: diff changeset	617
74271829ddc0 clonebundles: move a bundle of clone bundle related code to a new module Pierre-Yves David <pierre-yves.david@octobus.net> parents: diff changeset	618 items = sorted(clonebundleentry(v, prefers) for v in entries)
74271829ddc0 clonebundles: move a bundle of clone bundle related code to a new module Pierre-Yves David <pierre-yves.david@octobus.net> parents: diff changeset	619 return [i.value for i in items]

Mercurial > public > mercurial-scm > hg

annotate mercurial/bundlecaches.py @ 53028:89ab2459f62a stable 6.9.3