mercurial-scm/hg-stable: mercurial/bundlecaches.py annotate

annotate mercurial/bundlecaches.py @ 52971:469b9a628b51

dirstatemap: update, document and type the identity tracking This new form should hopefully be clearer and less error prone.

author	Pierre-Yves David <pierre-yves.david@octobus.net>
date	Tue, 18 Feb 2025 22:24:08 +0100
parents	42f78c859dd1
children	e705fec4a03f

rev	line source
45807 74271829ddc0 clonebundles: move a bundle of clone bundle related code to a new module Pierre-Yves David <pierre-yves.david@octobus.net> parents: diff changeset	1 # bundlecaches.py - utility to deal with pre-computed bundle for servers
74271829ddc0 clonebundles: move a bundle of clone bundle related code to a new module Pierre-Yves David <pierre-yves.david@octobus.net> parents: diff changeset	2 #
74271829ddc0 clonebundles: move a bundle of clone bundle related code to a new module Pierre-Yves David <pierre-yves.david@octobus.net> parents: diff changeset	3 # This software may be used and distributed according to the terms of the
74271829ddc0 clonebundles: move a bundle of clone bundle related code to a new module Pierre-Yves David <pierre-yves.david@octobus.net> parents: diff changeset	4 # GNU General Public License version 2 or any later version.
74271829ddc0 clonebundles: move a bundle of clone bundle related code to a new module Pierre-Yves David <pierre-yves.david@octobus.net> parents: diff changeset	5
51902 1c5810ce737e typing: add `from __future__ import annotations` to remaining source files Matt Harbison <matt_harbison@yahoo.com> parents: 51788 diff changeset	6 from __future__ import annotations
1c5810ce737e typing: add `from __future__ import annotations` to remaining source files Matt Harbison <matt_harbison@yahoo.com> parents: 51788 diff changeset	7
49330 bf66f7a1e3f8 bundlespec: merge the contentopts and params dictionnary Pierre-Yves David <pierre-yves.david@octobus.net> parents: 49327 diff changeset	8 import collections
52012 aa7f4a45d8fa clonebundles: allow manifest to specify sha256 digest of bundles Joerg Sonnenberger <joerg@bec.de> parents: 51902 diff changeset	9 import re
51788 278af66e6595 typing: induce pytype to use the standard `attr` instead of the vendored copy Matt Harbison <matt_harbison@yahoo.com> parents: 51732 diff changeset	10 import typing
49330 bf66f7a1e3f8 bundlespec: merge the contentopts and params dictionnary Pierre-Yves David <pierre-yves.david@octobus.net> parents: 49327 diff changeset	11
49764 9be765b82a90 typing: minor tweaks to allow updating to pytype 2022.11.18 Matt Harbison <matt_harbison@yahoo.com> parents: 49337 diff changeset	12 from typing import (
51599 9b51a4fb0f96 bundlespec: type the _bundlespeccontentopts dictionary Pierre-Yves David <pierre-yves.david@octobus.net> parents: 51598 diff changeset	13 Dict,
9b51a4fb0f96 bundlespec: type the _bundlespeccontentopts dictionary Pierre-Yves David <pierre-yves.david@octobus.net> parents: 51598 diff changeset	14 Union,
49764 9be765b82a90 typing: minor tweaks to allow updating to pytype 2022.11.18 Matt Harbison <matt_harbison@yahoo.com> parents: 49337 diff changeset	15 cast,
9be765b82a90 typing: minor tweaks to allow updating to pytype 2022.11.18 Matt Harbison <matt_harbison@yahoo.com> parents: 49337 diff changeset	16 )
9be765b82a90 typing: minor tweaks to allow updating to pytype 2022.11.18 Matt Harbison <matt_harbison@yahoo.com> parents: 49337 diff changeset	17
45807 74271829ddc0 clonebundles: move a bundle of clone bundle related code to a new module Pierre-Yves David <pierre-yves.david@octobus.net> parents: diff changeset	18 from .i18n import _
74271829ddc0 clonebundles: move a bundle of clone bundle related code to a new module Pierre-Yves David <pierre-yves.david@octobus.net> parents: diff changeset	19
74271829ddc0 clonebundles: move a bundle of clone bundle related code to a new module Pierre-Yves David <pierre-yves.david@octobus.net> parents: diff changeset	20 from .thirdparty import attr
74271829ddc0 clonebundles: move a bundle of clone bundle related code to a new module Pierre-Yves David <pierre-yves.david@octobus.net> parents: diff changeset	21
51788 278af66e6595 typing: induce pytype to use the standard `attr` instead of the vendored copy Matt Harbison <matt_harbison@yahoo.com> parents: 51732 diff changeset	22 # Force pytype to use the non-vendored package
278af66e6595 typing: induce pytype to use the standard `attr` instead of the vendored copy Matt Harbison <matt_harbison@yahoo.com> parents: 51732 diff changeset	23 if typing.TYPE_CHECKING:
278af66e6595 typing: induce pytype to use the standard `attr` instead of the vendored copy Matt Harbison <matt_harbison@yahoo.com> parents: 51732 diff changeset	24 # noinspection PyPackageRequirements
278af66e6595 typing: induce pytype to use the standard `attr` instead of the vendored copy Matt Harbison <matt_harbison@yahoo.com> parents: 51732 diff changeset	25 import attr
278af66e6595 typing: induce pytype to use the standard `attr` instead of the vendored copy Matt Harbison <matt_harbison@yahoo.com> parents: 51732 diff changeset	26
45807 74271829ddc0 clonebundles: move a bundle of clone bundle related code to a new module Pierre-Yves David <pierre-yves.david@octobus.net> parents: diff changeset	27 from . import (
74271829ddc0 clonebundles: move a bundle of clone bundle related code to a new module Pierre-Yves David <pierre-yves.david@octobus.net> parents: diff changeset	28 error,
46666 f4c325bf80fc requirements: also add a generaldelta constant Rapha?l Gom?s <rgomes@octobus.net> parents: 45808 diff changeset	29 requirements as requirementsmod,
45807 74271829ddc0 clonebundles: move a bundle of clone bundle related code to a new module Pierre-Yves David <pierre-yves.david@octobus.net> parents: diff changeset	30 sslutil,
52012 aa7f4a45d8fa clonebundles: allow manifest to specify sha256 digest of bundles Joerg Sonnenberger <joerg@bec.de> parents: 51902 diff changeset	31 url as urlmod,
45807 74271829ddc0 clonebundles: move a bundle of clone bundle related code to a new module Pierre-Yves David <pierre-yves.david@octobus.net> parents: diff changeset	32 util,
74271829ddc0 clonebundles: move a bundle of clone bundle related code to a new module Pierre-Yves David <pierre-yves.david@octobus.net> parents: diff changeset	33 )
74271829ddc0 clonebundles: move a bundle of clone bundle related code to a new module Pierre-Yves David <pierre-yves.david@octobus.net> parents: diff changeset	34 from .utils import stringutil
74271829ddc0 clonebundles: move a bundle of clone bundle related code to a new module Pierre-Yves David <pierre-yves.david@octobus.net> parents: diff changeset	35
74271829ddc0 clonebundles: move a bundle of clone bundle related code to a new module Pierre-Yves David <pierre-yves.david@octobus.net> parents: diff changeset	36 urlreq = util.urlreq
74271829ddc0 clonebundles: move a bundle of clone bundle related code to a new module Pierre-Yves David <pierre-yves.david@octobus.net> parents: diff changeset	37
45808 80f32ec8653a clonebundle: move the manifest filename to a constant Pierre-Yves David <pierre-yves.david@octobus.net> parents: 45807 diff changeset	38 CB_MANIFEST_FILE = b'clonebundles.manifest'
50694 a41eeb877d07 branching: merge with stable Pierre-Yves David <pierre-yves.david@octobus.net> parents: 50371 diff changeset	39 CLONEBUNDLESCHEME = b"peer-bundle-cache://"
a41eeb877d07 branching: merge with stable Pierre-Yves David <pierre-yves.david@octobus.net> parents: 50371 diff changeset	40
a41eeb877d07 branching: merge with stable Pierre-Yves David <pierre-yves.david@octobus.net> parents: 50371 diff changeset	41
51732 138ab7c6a6ff typing: add some trivial type hints to `mercurial/bundlecaches.py` Matt Harbison <matt_harbison@yahoo.com> parents: 51625 diff changeset	42 def get_manifest(repo) -> bytes:
50694 a41eeb877d07 branching: merge with stable Pierre-Yves David <pierre-yves.david@octobus.net> parents: 50371 diff changeset	43 """get the bundle manifest to be served to a client from a server"""
a41eeb877d07 branching: merge with stable Pierre-Yves David <pierre-yves.david@octobus.net> parents: 50371 diff changeset	44 raw_text = repo.vfs.tryread(CB_MANIFEST_FILE)
a41eeb877d07 branching: merge with stable Pierre-Yves David <pierre-yves.david@octobus.net> parents: 50371 diff changeset	45 entries = [e.split(b' ', 1) for e in raw_text.splitlines()]
a41eeb877d07 branching: merge with stable Pierre-Yves David <pierre-yves.david@octobus.net> parents: 50371 diff changeset	46
a41eeb877d07 branching: merge with stable Pierre-Yves David <pierre-yves.david@octobus.net> parents: 50371 diff changeset	47 new_lines = []
a41eeb877d07 branching: merge with stable Pierre-Yves David <pierre-yves.david@octobus.net> parents: 50371 diff changeset	48 for e in entries:
a41eeb877d07 branching: merge with stable Pierre-Yves David <pierre-yves.david@octobus.net> parents: 50371 diff changeset	49 url = alter_bundle_url(repo, e[0])
a41eeb877d07 branching: merge with stable Pierre-Yves David <pierre-yves.david@octobus.net> parents: 50371 diff changeset	50 if len(e) == 1:
a41eeb877d07 branching: merge with stable Pierre-Yves David <pierre-yves.david@octobus.net> parents: 50371 diff changeset	51 line = url + b'\n'
a41eeb877d07 branching: merge with stable Pierre-Yves David <pierre-yves.david@octobus.net> parents: 50371 diff changeset	52 else:
a41eeb877d07 branching: merge with stable Pierre-Yves David <pierre-yves.david@octobus.net> parents: 50371 diff changeset	53 line = b"%s %s\n" % (url, e[1])
a41eeb877d07 branching: merge with stable Pierre-Yves David <pierre-yves.david@octobus.net> parents: 50371 diff changeset	54 new_lines.append(line)
a41eeb877d07 branching: merge with stable Pierre-Yves David <pierre-yves.david@octobus.net> parents: 50371 diff changeset	55 return b''.join(new_lines)
a41eeb877d07 branching: merge with stable Pierre-Yves David <pierre-yves.david@octobus.net> parents: 50371 diff changeset	56
a41eeb877d07 branching: merge with stable Pierre-Yves David <pierre-yves.david@octobus.net> parents: 50371 diff changeset	57
51732 138ab7c6a6ff typing: add some trivial type hints to `mercurial/bundlecaches.py` Matt Harbison <matt_harbison@yahoo.com> parents: 51625 diff changeset	58 def alter_bundle_url(repo, url: bytes) -> bytes:
50694 a41eeb877d07 branching: merge with stable Pierre-Yves David <pierre-yves.david@octobus.net> parents: 50371 diff changeset	59 """a function that exist to help extension and hosting to alter the url
a41eeb877d07 branching: merge with stable Pierre-Yves David <pierre-yves.david@octobus.net> parents: 50371 diff changeset	60
a41eeb877d07 branching: merge with stable Pierre-Yves David <pierre-yves.david@octobus.net> parents: 50371 diff changeset	61 This will typically be used to inject authentication information in the url
a41eeb877d07 branching: merge with stable Pierre-Yves David <pierre-yves.david@octobus.net> parents: 50371 diff changeset	62 of cached bundles."""
a41eeb877d07 branching: merge with stable Pierre-Yves David <pierre-yves.david@octobus.net> parents: 50371 diff changeset	63 return url
a41eeb877d07 branching: merge with stable Pierre-Yves David <pierre-yves.david@octobus.net> parents: 50371 diff changeset	64
a41eeb877d07 branching: merge with stable Pierre-Yves David <pierre-yves.david@octobus.net> parents: 50371 diff changeset	65
50371 7b723217d368 clonebundles: filter out invalid schemes instead of failing on them Mathias De Mare <mathias.de_mare@nokia.com> parents: 49764 diff changeset	66 SUPPORTED_CLONEBUNDLE_SCHEMES = [
7b723217d368 clonebundles: filter out invalid schemes instead of failing on them Mathias De Mare <mathias.de_mare@nokia.com> parents: 49764 diff changeset	67 b"http://",
7b723217d368 clonebundles: filter out invalid schemes instead of failing on them Mathias De Mare <mathias.de_mare@nokia.com> parents: 49764 diff changeset	68 b"https://",
7b723217d368 clonebundles: filter out invalid schemes instead of failing on them Mathias De Mare <mathias.de_mare@nokia.com> parents: 49764 diff changeset	69 b"largefile://",
50694 a41eeb877d07 branching: merge with stable Pierre-Yves David <pierre-yves.david@octobus.net> parents: 50371 diff changeset	70 CLONEBUNDLESCHEME,
50371 7b723217d368 clonebundles: filter out invalid schemes instead of failing on them Mathias De Mare <mathias.de_mare@nokia.com> parents: 49764 diff changeset	71 ]
45808 80f32ec8653a clonebundle: move the manifest filename to a constant Pierre-Yves David <pierre-yves.david@octobus.net> parents: 45807 diff changeset	72
45807 74271829ddc0 clonebundles: move a bundle of clone bundle related code to a new module Pierre-Yves David <pierre-yves.david@octobus.net> parents: diff changeset	73
74271829ddc0 clonebundles: move a bundle of clone bundle related code to a new module Pierre-Yves David <pierre-yves.david@octobus.net> parents: diff changeset	74 @attr.s
49037 642e31cb55f0 py3: use class X: instead of class X(object): Gregory Szorc <gregory.szorc@gmail.com> parents: 48623 diff changeset	75 class bundlespec:
45807 74271829ddc0 clonebundles: move a bundle of clone bundle related code to a new module Pierre-Yves David <pierre-yves.david@octobus.net> parents: diff changeset	76 compression = attr.ib()
74271829ddc0 clonebundles: move a bundle of clone bundle related code to a new module Pierre-Yves David <pierre-yves.david@octobus.net> parents: diff changeset	77 wirecompression = attr.ib()
74271829ddc0 clonebundles: move a bundle of clone bundle related code to a new module Pierre-Yves David <pierre-yves.david@octobus.net> parents: diff changeset	78 version = attr.ib()
74271829ddc0 clonebundles: move a bundle of clone bundle related code to a new module Pierre-Yves David <pierre-yves.david@octobus.net> parents: diff changeset	79 wireversion = attr.ib()
49330 bf66f7a1e3f8 bundlespec: merge the contentopts and params dictionnary Pierre-Yves David <pierre-yves.david@octobus.net> parents: 49327 diff changeset	80 # parameters explicitly overwritten by the config or the specification
bf66f7a1e3f8 bundlespec: merge the contentopts and params dictionnary Pierre-Yves David <pierre-yves.david@octobus.net> parents: 49327 diff changeset	81 _explicit_params = attr.ib()
bf66f7a1e3f8 bundlespec: merge the contentopts and params dictionnary Pierre-Yves David <pierre-yves.david@octobus.net> parents: 49327 diff changeset	82 # default parameter for the version
bf66f7a1e3f8 bundlespec: merge the contentopts and params dictionnary Pierre-Yves David <pierre-yves.david@octobus.net> parents: 49327 diff changeset	83 #
bf66f7a1e3f8 bundlespec: merge the contentopts and params dictionnary Pierre-Yves David <pierre-yves.david@octobus.net> parents: 49327 diff changeset	84 # Keeping it separated is useful to check what was actually overwritten.
bf66f7a1e3f8 bundlespec: merge the contentopts and params dictionnary Pierre-Yves David <pierre-yves.david@octobus.net> parents: 49327 diff changeset	85 _default_opts = attr.ib()
bf66f7a1e3f8 bundlespec: merge the contentopts and params dictionnary Pierre-Yves David <pierre-yves.david@octobus.net> parents: 49327 diff changeset	86
bf66f7a1e3f8 bundlespec: merge the contentopts and params dictionnary Pierre-Yves David <pierre-yves.david@octobus.net> parents: 49327 diff changeset	87 @property
bf66f7a1e3f8 bundlespec: merge the contentopts and params dictionnary Pierre-Yves David <pierre-yves.david@octobus.net> parents: 49327 diff changeset	88 def params(self):
bf66f7a1e3f8 bundlespec: merge the contentopts and params dictionnary Pierre-Yves David <pierre-yves.david@octobus.net> parents: 49327 diff changeset	89 return collections.ChainMap(self._explicit_params, self._default_opts)
bf66f7a1e3f8 bundlespec: merge the contentopts and params dictionnary Pierre-Yves David <pierre-yves.david@octobus.net> parents: 49327 diff changeset	90
bf66f7a1e3f8 bundlespec: merge the contentopts and params dictionnary Pierre-Yves David <pierre-yves.david@octobus.net> parents: 49327 diff changeset	91 @property
bf66f7a1e3f8 bundlespec: merge the contentopts and params dictionnary Pierre-Yves David <pierre-yves.david@octobus.net> parents: 49327 diff changeset	92 def contentopts(self):
bf66f7a1e3f8 bundlespec: merge the contentopts and params dictionnary Pierre-Yves David <pierre-yves.david@octobus.net> parents: 49327 diff changeset	93 # kept for Backward Compatibility concerns.
bf66f7a1e3f8 bundlespec: merge the contentopts and params dictionnary Pierre-Yves David <pierre-yves.david@octobus.net> parents: 49327 diff changeset	94 return self.params
bf66f7a1e3f8 bundlespec: merge the contentopts and params dictionnary Pierre-Yves David <pierre-yves.david@octobus.net> parents: 49327 diff changeset	95
49331 3840d16595cf bundlespec: add a `overwrite` parameter to set_param Pierre-Yves David <pierre-yves.david@octobus.net> parents: 49330 diff changeset	96 def set_param(self, key, value, overwrite=True):
3840d16595cf bundlespec: add a `overwrite` parameter to set_param Pierre-Yves David <pierre-yves.david@octobus.net> parents: 49330 diff changeset	97 """Set a bundle parameter value.
3840d16595cf bundlespec: add a `overwrite` parameter to set_param Pierre-Yves David <pierre-yves.david@octobus.net> parents: 49330 diff changeset	98
3840d16595cf bundlespec: add a `overwrite` parameter to set_param Pierre-Yves David <pierre-yves.david@octobus.net> parents: 49330 diff changeset	99 Will only overwrite if overwrite is true"""
3840d16595cf bundlespec: add a `overwrite` parameter to set_param Pierre-Yves David <pierre-yves.david@octobus.net> parents: 49330 diff changeset	100 if overwrite or key not in self._explicit_params:
3840d16595cf bundlespec: add a `overwrite` parameter to set_param Pierre-Yves David <pierre-yves.david@octobus.net> parents: 49330 diff changeset	101 self._explicit_params[key] = value
45807 74271829ddc0 clonebundles: move a bundle of clone bundle related code to a new module Pierre-Yves David <pierre-yves.david@octobus.net> parents: diff changeset	102
50712 0913a49e020c bundle-spec: add an official method to format a spec into a string Pierre-Yves David <pierre-yves.david@octobus.net> parents: 50694 diff changeset	103 def as_spec(self):
0913a49e020c bundle-spec: add an official method to format a spec into a string Pierre-Yves David <pierre-yves.david@octobus.net> parents: 50694 diff changeset	104 parts = [b"%s-%s" % (self.compression, self.version)]
0913a49e020c bundle-spec: add an official method to format a spec into a string Pierre-Yves David <pierre-yves.david@octobus.net> parents: 50694 diff changeset	105 for param in sorted(self._explicit_params.items()):
0913a49e020c bundle-spec: add an official method to format a spec into a string Pierre-Yves David <pierre-yves.david@octobus.net> parents: 50694 diff changeset	106 parts.append(b'%s=%s' % param)
0913a49e020c bundle-spec: add an official method to format a spec into a string Pierre-Yves David <pierre-yves.david@octobus.net> parents: 50694 diff changeset	107 return b';'.join(parts)
0913a49e020c bundle-spec: add an official method to format a spec into a string Pierre-Yves David <pierre-yves.david@octobus.net> parents: 50694 diff changeset	108
45807 74271829ddc0 clonebundles: move a bundle of clone bundle related code to a new module Pierre-Yves David <pierre-yves.david@octobus.net> parents: diff changeset	109
74271829ddc0 clonebundles: move a bundle of clone bundle related code to a new module Pierre-Yves David <pierre-yves.david@octobus.net> parents: diff changeset	110 # Maps bundle version human names to changegroup versions.
74271829ddc0 clonebundles: move a bundle of clone bundle related code to a new module Pierre-Yves David <pierre-yves.david@octobus.net> parents: diff changeset	111 _bundlespeccgversions = {
74271829ddc0 clonebundles: move a bundle of clone bundle related code to a new module Pierre-Yves David <pierre-yves.david@octobus.net> parents: diff changeset	112 b'v1': b'01',
74271829ddc0 clonebundles: move a bundle of clone bundle related code to a new module Pierre-Yves David <pierre-yves.david@octobus.net> parents: diff changeset	113 b'v2': b'02',
50694 a41eeb877d07 branching: merge with stable Pierre-Yves David <pierre-yves.david@octobus.net> parents: 50371 diff changeset	114 b'v3': b'03',
45807 74271829ddc0 clonebundles: move a bundle of clone bundle related code to a new module Pierre-Yves David <pierre-yves.david@octobus.net> parents: diff changeset	115 b'packed1': b's1',
74271829ddc0 clonebundles: move a bundle of clone bundle related code to a new module Pierre-Yves David <pierre-yves.david@octobus.net> parents: diff changeset	116 b'bundle2': b'02', # legacy
74271829ddc0 clonebundles: move a bundle of clone bundle related code to a new module Pierre-Yves David <pierre-yves.david@octobus.net> parents: diff changeset	117 }
74271829ddc0 clonebundles: move a bundle of clone bundle related code to a new module Pierre-Yves David <pierre-yves.david@octobus.net> parents: diff changeset	118
74271829ddc0 clonebundles: move a bundle of clone bundle related code to a new module Pierre-Yves David <pierre-yves.david@octobus.net> parents: diff changeset	119 # Maps bundle version with content opts to choose which part to bundle
51599 9b51a4fb0f96 bundlespec: type the _bundlespeccontentopts dictionary Pierre-Yves David <pierre-yves.david@octobus.net> parents: 51598 diff changeset	120 _bundlespeccontentopts: Dict[bytes, Dict[bytes, Union[bool, bytes]]] = {
45807 74271829ddc0 clonebundles: move a bundle of clone bundle related code to a new module Pierre-Yves David <pierre-yves.david@octobus.net> parents: diff changeset	121 b'v1': {
74271829ddc0 clonebundles: move a bundle of clone bundle related code to a new module Pierre-Yves David <pierre-yves.david@octobus.net> parents: diff changeset	122 b'changegroup': True,
74271829ddc0 clonebundles: move a bundle of clone bundle related code to a new module Pierre-Yves David <pierre-yves.david@octobus.net> parents: diff changeset	123 b'cg.version': b'01',
74271829ddc0 clonebundles: move a bundle of clone bundle related code to a new module Pierre-Yves David <pierre-yves.david@octobus.net> parents: diff changeset	124 b'obsolescence': False,
74271829ddc0 clonebundles: move a bundle of clone bundle related code to a new module Pierre-Yves David <pierre-yves.david@octobus.net> parents: diff changeset	125 b'phases': False,
74271829ddc0 clonebundles: move a bundle of clone bundle related code to a new module Pierre-Yves David <pierre-yves.david@octobus.net> parents: diff changeset	126 b'tagsfnodescache': False,
74271829ddc0 clonebundles: move a bundle of clone bundle related code to a new module Pierre-Yves David <pierre-yves.david@octobus.net> parents: diff changeset	127 b'revbranchcache': False,
74271829ddc0 clonebundles: move a bundle of clone bundle related code to a new module Pierre-Yves David <pierre-yves.david@octobus.net> parents: diff changeset	128 },
74271829ddc0 clonebundles: move a bundle of clone bundle related code to a new module Pierre-Yves David <pierre-yves.david@octobus.net> parents: diff changeset	129 b'v2': {
74271829ddc0 clonebundles: move a bundle of clone bundle related code to a new module Pierre-Yves David <pierre-yves.david@octobus.net> parents: diff changeset	130 b'changegroup': True,
74271829ddc0 clonebundles: move a bundle of clone bundle related code to a new module Pierre-Yves David <pierre-yves.david@octobus.net> parents: diff changeset	131 b'cg.version': b'02',
74271829ddc0 clonebundles: move a bundle of clone bundle related code to a new module Pierre-Yves David <pierre-yves.david@octobus.net> parents: diff changeset	132 b'obsolescence': False,
74271829ddc0 clonebundles: move a bundle of clone bundle related code to a new module Pierre-Yves David <pierre-yves.david@octobus.net> parents: diff changeset	133 b'phases': False,
74271829ddc0 clonebundles: move a bundle of clone bundle related code to a new module Pierre-Yves David <pierre-yves.david@octobus.net> parents: diff changeset	134 b'tagsfnodescache': True,
74271829ddc0 clonebundles: move a bundle of clone bundle related code to a new module Pierre-Yves David <pierre-yves.david@octobus.net> parents: diff changeset	135 b'revbranchcache': True,
74271829ddc0 clonebundles: move a bundle of clone bundle related code to a new module Pierre-Yves David <pierre-yves.david@octobus.net> parents: diff changeset	136 },
50694 a41eeb877d07 branching: merge with stable Pierre-Yves David <pierre-yves.david@octobus.net> parents: 50371 diff changeset	137 b'v3': {
a41eeb877d07 branching: merge with stable Pierre-Yves David <pierre-yves.david@octobus.net> parents: 50371 diff changeset	138 b'changegroup': True,
a41eeb877d07 branching: merge with stable Pierre-Yves David <pierre-yves.david@octobus.net> parents: 50371 diff changeset	139 b'cg.version': b'03',
a41eeb877d07 branching: merge with stable Pierre-Yves David <pierre-yves.david@octobus.net> parents: 50371 diff changeset	140 b'obsolescence': False,
a41eeb877d07 branching: merge with stable Pierre-Yves David <pierre-yves.david@octobus.net> parents: 50371 diff changeset	141 b'phases': True,
a41eeb877d07 branching: merge with stable Pierre-Yves David <pierre-yves.david@octobus.net> parents: 50371 diff changeset	142 b'tagsfnodescache': True,
a41eeb877d07 branching: merge with stable Pierre-Yves David <pierre-yves.david@octobus.net> parents: 50371 diff changeset	143 b'revbranchcache': True,
a41eeb877d07 branching: merge with stable Pierre-Yves David <pierre-yves.david@octobus.net> parents: 50371 diff changeset	144 },
49327 d587f09cad98 bundlespec: make the `stream` case less special Pierre-Yves David <pierre-yves.david@octobus.net> parents: 49326 diff changeset	145 b'streamv2': {
d587f09cad98 bundlespec: make the `stream` case less special Pierre-Yves David <pierre-yves.david@octobus.net> parents: 49326 diff changeset	146 b'changegroup': False,
d587f09cad98 bundlespec: make the `stream` case less special Pierre-Yves David <pierre-yves.david@octobus.net> parents: 49326 diff changeset	147 b'cg.version': b'02',
d587f09cad98 bundlespec: make the `stream` case less special Pierre-Yves David <pierre-yves.david@octobus.net> parents: 49326 diff changeset	148 b'obsolescence': False,
d587f09cad98 bundlespec: make the `stream` case less special Pierre-Yves David <pierre-yves.david@octobus.net> parents: 49326 diff changeset	149 b'phases': False,
51598 4a9d0898bf16 bundlespec: fix the "streamv2" and "streamv3-exp" variant Pierre-Yves David <pierre-yves.david@octobus.net> parents: 51596 diff changeset	150 b"stream": b"v2",
49327 d587f09cad98 bundlespec: make the `stream` case less special Pierre-Yves David <pierre-yves.david@octobus.net> parents: 49326 diff changeset	151 b'tagsfnodescache': False,
d587f09cad98 bundlespec: make the `stream` case less special Pierre-Yves David <pierre-yves.david@octobus.net> parents: 49326 diff changeset	152 b'revbranchcache': False,
d587f09cad98 bundlespec: make the `stream` case less special Pierre-Yves David <pierre-yves.david@octobus.net> parents: 49326 diff changeset	153 },
50694 a41eeb877d07 branching: merge with stable Pierre-Yves David <pierre-yves.david@octobus.net> parents: 50371 diff changeset	154 b'streamv3-exp': {
a41eeb877d07 branching: merge with stable Pierre-Yves David <pierre-yves.david@octobus.net> parents: 50371 diff changeset	155 b'changegroup': False,
a41eeb877d07 branching: merge with stable Pierre-Yves David <pierre-yves.david@octobus.net> parents: 50371 diff changeset	156 b'cg.version': b'03',
a41eeb877d07 branching: merge with stable Pierre-Yves David <pierre-yves.david@octobus.net> parents: 50371 diff changeset	157 b'obsolescence': False,
a41eeb877d07 branching: merge with stable Pierre-Yves David <pierre-yves.david@octobus.net> parents: 50371 diff changeset	158 b'phases': False,
51598 4a9d0898bf16 bundlespec: fix the "streamv2" and "streamv3-exp" variant Pierre-Yves David <pierre-yves.david@octobus.net> parents: 51596 diff changeset	159 b"stream": b"v3-exp",
50694 a41eeb877d07 branching: merge with stable Pierre-Yves David <pierre-yves.david@octobus.net> parents: 50371 diff changeset	160 b'tagsfnodescache': False,
a41eeb877d07 branching: merge with stable Pierre-Yves David <pierre-yves.david@octobus.net> parents: 50371 diff changeset	161 b'revbranchcache': False,
a41eeb877d07 branching: merge with stable Pierre-Yves David <pierre-yves.david@octobus.net> parents: 50371 diff changeset	162 },
49326 c12c843f0db7 bundlespec: phase out the `_bundlespeccgversions` mapping Pierre-Yves David <pierre-yves.david@octobus.net> parents: 49037 diff changeset	163 b'packed1': {
c12c843f0db7 bundlespec: phase out the `_bundlespeccgversions` mapping Pierre-Yves David <pierre-yves.david@octobus.net> parents: 49037 diff changeset	164 b'cg.version': b's1',
c12c843f0db7 bundlespec: phase out the `_bundlespeccgversions` mapping Pierre-Yves David <pierre-yves.david@octobus.net> parents: 49037 diff changeset	165 },
c12c843f0db7 bundlespec: phase out the `_bundlespeccgversions` mapping Pierre-Yves David <pierre-yves.david@octobus.net> parents: 49037 diff changeset	166 b'bundle2': { # legacy
c12c843f0db7 bundlespec: phase out the `_bundlespeccgversions` mapping Pierre-Yves David <pierre-yves.david@octobus.net> parents: 49037 diff changeset	167 b'cg.version': b'02',
c12c843f0db7 bundlespec: phase out the `_bundlespeccgversions` mapping Pierre-Yves David <pierre-yves.david@octobus.net> parents: 49037 diff changeset	168 },
45807 74271829ddc0 clonebundles: move a bundle of clone bundle related code to a new module Pierre-Yves David <pierre-yves.david@octobus.net> parents: diff changeset	169 }
74271829ddc0 clonebundles: move a bundle of clone bundle related code to a new module Pierre-Yves David <pierre-yves.david@octobus.net> parents: diff changeset	170 _bundlespeccontentopts[b'bundle2'] = _bundlespeccontentopts[b'v2']
74271829ddc0 clonebundles: move a bundle of clone bundle related code to a new module Pierre-Yves David <pierre-yves.david@octobus.net> parents: diff changeset	171
74271829ddc0 clonebundles: move a bundle of clone bundle related code to a new module Pierre-Yves David <pierre-yves.david@octobus.net> parents: diff changeset	172 # Compression engines allowed in version 1. THIS SHOULD NEVER CHANGE.
74271829ddc0 clonebundles: move a bundle of clone bundle related code to a new module Pierre-Yves David <pierre-yves.david@octobus.net> parents: diff changeset	173 _bundlespecv1compengines = {b'gzip', b'bzip2', b'none'}
74271829ddc0 clonebundles: move a bundle of clone bundle related code to a new module Pierre-Yves David <pierre-yves.david@octobus.net> parents: diff changeset	174
74271829ddc0 clonebundles: move a bundle of clone bundle related code to a new module Pierre-Yves David <pierre-yves.david@octobus.net> parents: diff changeset	175
49336 1b04d5213d0f bundlespec: add processing of some parameter value Pierre-Yves David <pierre-yves.david@octobus.net> parents: 49335 diff changeset	176 def param_bool(key, value):
1b04d5213d0f bundlespec: add processing of some parameter value Pierre-Yves David <pierre-yves.david@octobus.net> parents: 49335 diff changeset	177 """make a boolean out of a parameter value"""
1b04d5213d0f bundlespec: add processing of some parameter value Pierre-Yves David <pierre-yves.david@octobus.net> parents: 49335 diff changeset	178 b = stringutil.parsebool(value)
1b04d5213d0f bundlespec: add processing of some parameter value Pierre-Yves David <pierre-yves.david@octobus.net> parents: 49335 diff changeset	179 if b is None:
1b04d5213d0f bundlespec: add processing of some parameter value Pierre-Yves David <pierre-yves.david@octobus.net> parents: 49335 diff changeset	180 msg = _(b"parameter %s should be a boolean ('%s')")
1b04d5213d0f bundlespec: add processing of some parameter value Pierre-Yves David <pierre-yves.david@octobus.net> parents: 49335 diff changeset	181 msg %= (key, value)
1b04d5213d0f bundlespec: add processing of some parameter value Pierre-Yves David <pierre-yves.david@octobus.net> parents: 49335 diff changeset	182 raise error.InvalidBundleSpecification(msg)
1b04d5213d0f bundlespec: add processing of some parameter value Pierre-Yves David <pierre-yves.david@octobus.net> parents: 49335 diff changeset	183 return b
1b04d5213d0f bundlespec: add processing of some parameter value Pierre-Yves David <pierre-yves.david@octobus.net> parents: 49335 diff changeset	184
1b04d5213d0f bundlespec: add processing of some parameter value Pierre-Yves David <pierre-yves.david@octobus.net> parents: 49335 diff changeset	185
1b04d5213d0f bundlespec: add processing of some parameter value Pierre-Yves David <pierre-yves.david@octobus.net> parents: 49335 diff changeset	186 # mapping of known parameter name need their value processed
1b04d5213d0f bundlespec: add processing of some parameter value Pierre-Yves David <pierre-yves.david@octobus.net> parents: 49335 diff changeset	187 bundle_spec_param_processing = {
1b04d5213d0f bundlespec: add processing of some parameter value Pierre-Yves David <pierre-yves.david@octobus.net> parents: 49335 diff changeset	188 b"obsolescence": param_bool,
1b04d5213d0f bundlespec: add processing of some parameter value Pierre-Yves David <pierre-yves.david@octobus.net> parents: 49335 diff changeset	189 b"obsolescence-mandatory": param_bool,
1b04d5213d0f bundlespec: add processing of some parameter value Pierre-Yves David <pierre-yves.david@octobus.net> parents: 49335 diff changeset	190 b"phases": param_bool,
51473 eac84af0c8cc bundle-spec: properly parse boolean configuration as boolean Pierre-Yves David <pierre-yves.david@octobus.net> parents: 50712 diff changeset	191 b"changegroup": param_bool,
eac84af0c8cc bundle-spec: properly parse boolean configuration as boolean Pierre-Yves David <pierre-yves.david@octobus.net> parents: 50712 diff changeset	192 b"tagsfnodescache": param_bool,
eac84af0c8cc bundle-spec: properly parse boolean configuration as boolean Pierre-Yves David <pierre-yves.david@octobus.net> parents: 50712 diff changeset	193 b"revbranchcache": param_bool,
49336 1b04d5213d0f bundlespec: add processing of some parameter value Pierre-Yves David <pierre-yves.david@octobus.net> parents: 49335 diff changeset	194 }
1b04d5213d0f bundlespec: add processing of some parameter value Pierre-Yves David <pierre-yves.david@octobus.net> parents: 49335 diff changeset	195
1b04d5213d0f bundlespec: add processing of some parameter value Pierre-Yves David <pierre-yves.david@octobus.net> parents: 49335 diff changeset	196
49335 5d17dd74177d bundlespec: extract the parseparams closure Pierre-Yves David <pierre-yves.david@octobus.net> parents: 49331 diff changeset	197 def _parseparams(s):
5d17dd74177d bundlespec: extract the parseparams closure Pierre-Yves David <pierre-yves.david@octobus.net> parents: 49331 diff changeset	198 """parse bundlespec parameter section
5d17dd74177d bundlespec: extract the parseparams closure Pierre-Yves David <pierre-yves.david@octobus.net> parents: 49331 diff changeset	199
5d17dd74177d bundlespec: extract the parseparams closure Pierre-Yves David <pierre-yves.david@octobus.net> parents: 49331 diff changeset	200 input: "comp-version;params" string
5d17dd74177d bundlespec: extract the parseparams closure Pierre-Yves David <pierre-yves.david@octobus.net> parents: 49331 diff changeset	201
5d17dd74177d bundlespec: extract the parseparams closure Pierre-Yves David <pierre-yves.david@octobus.net> parents: 49331 diff changeset	202 return: (spec; {param_key: param_value})
5d17dd74177d bundlespec: extract the parseparams closure Pierre-Yves David <pierre-yves.david@octobus.net> parents: 49331 diff changeset	203 """
5d17dd74177d bundlespec: extract the parseparams closure Pierre-Yves David <pierre-yves.david@octobus.net> parents: 49331 diff changeset	204 if b';' not in s:
5d17dd74177d bundlespec: extract the parseparams closure Pierre-Yves David <pierre-yves.david@octobus.net> parents: 49331 diff changeset	205 return s, {}
5d17dd74177d bundlespec: extract the parseparams closure Pierre-Yves David <pierre-yves.david@octobus.net> parents: 49331 diff changeset	206
5d17dd74177d bundlespec: extract the parseparams closure Pierre-Yves David <pierre-yves.david@octobus.net> parents: 49331 diff changeset	207 params = {}
5d17dd74177d bundlespec: extract the parseparams closure Pierre-Yves David <pierre-yves.david@octobus.net> parents: 49331 diff changeset	208 version, paramstr = s.split(b';', 1)
5d17dd74177d bundlespec: extract the parseparams closure Pierre-Yves David <pierre-yves.david@octobus.net> parents: 49331 diff changeset	209
5d17dd74177d bundlespec: extract the parseparams closure Pierre-Yves David <pierre-yves.david@octobus.net> parents: 49331 diff changeset	210 err = _(b'invalid bundle specification: missing "=" in parameter: %s')
5d17dd74177d bundlespec: extract the parseparams closure Pierre-Yves David <pierre-yves.david@octobus.net> parents: 49331 diff changeset	211 for p in paramstr.split(b';'):
5d17dd74177d bundlespec: extract the parseparams closure Pierre-Yves David <pierre-yves.david@octobus.net> parents: 49331 diff changeset	212 if b'=' not in p:
5d17dd74177d bundlespec: extract the parseparams closure Pierre-Yves David <pierre-yves.david@octobus.net> parents: 49331 diff changeset	213 msg = err % p
5d17dd74177d bundlespec: extract the parseparams closure Pierre-Yves David <pierre-yves.david@octobus.net> parents: 49331 diff changeset	214 raise error.InvalidBundleSpecification(msg)
5d17dd74177d bundlespec: extract the parseparams closure Pierre-Yves David <pierre-yves.david@octobus.net> parents: 49331 diff changeset	215
5d17dd74177d bundlespec: extract the parseparams closure Pierre-Yves David <pierre-yves.david@octobus.net> parents: 49331 diff changeset	216 key, value = p.split(b'=', 1)
5d17dd74177d bundlespec: extract the parseparams closure Pierre-Yves David <pierre-yves.david@octobus.net> parents: 49331 diff changeset	217 key = urlreq.unquote(key)
5d17dd74177d bundlespec: extract the parseparams closure Pierre-Yves David <pierre-yves.david@octobus.net> parents: 49331 diff changeset	218 value = urlreq.unquote(value)
49336 1b04d5213d0f bundlespec: add processing of some parameter value Pierre-Yves David <pierre-yves.david@octobus.net> parents: 49335 diff changeset	219 process = bundle_spec_param_processing.get(key)
1b04d5213d0f bundlespec: add processing of some parameter value Pierre-Yves David <pierre-yves.david@octobus.net> parents: 49335 diff changeset	220 if process is not None:
1b04d5213d0f bundlespec: add processing of some parameter value Pierre-Yves David <pierre-yves.david@octobus.net> parents: 49335 diff changeset	221 value = process(key, value)
49335 5d17dd74177d bundlespec: extract the parseparams closure Pierre-Yves David <pierre-yves.david@octobus.net> parents: 49331 diff changeset	222 params[key] = value
5d17dd74177d bundlespec: extract the parseparams closure Pierre-Yves David <pierre-yves.david@octobus.net> parents: 49331 diff changeset	223
5d17dd74177d bundlespec: extract the parseparams closure Pierre-Yves David <pierre-yves.david@octobus.net> parents: 49331 diff changeset	224 return version, params
5d17dd74177d bundlespec: extract the parseparams closure Pierre-Yves David <pierre-yves.david@octobus.net> parents: 49331 diff changeset	225
5d17dd74177d bundlespec: extract the parseparams closure Pierre-Yves David <pierre-yves.david@octobus.net> parents: 49331 diff changeset	226
45807 74271829ddc0 clonebundles: move a bundle of clone bundle related code to a new module Pierre-Yves David <pierre-yves.david@octobus.net> parents: diff changeset	227 def parsebundlespec(repo, spec, strict=True):
74271829ddc0 clonebundles: move a bundle of clone bundle related code to a new module Pierre-Yves David <pierre-yves.david@octobus.net> parents: diff changeset	228 """Parse a bundle string specification into parts.
74271829ddc0 clonebundles: move a bundle of clone bundle related code to a new module Pierre-Yves David <pierre-yves.david@octobus.net> parents: diff changeset	229
74271829ddc0 clonebundles: move a bundle of clone bundle related code to a new module Pierre-Yves David <pierre-yves.david@octobus.net> parents: diff changeset	230 Bundle specifications denote a well-defined bundle/exchange format.
74271829ddc0 clonebundles: move a bundle of clone bundle related code to a new module Pierre-Yves David <pierre-yves.david@octobus.net> parents: diff changeset	231 The content of a given specification should not change over time in
74271829ddc0 clonebundles: move a bundle of clone bundle related code to a new module Pierre-Yves David <pierre-yves.david@octobus.net> parents: diff changeset	232 order to ensure that bundles produced by a newer version of Mercurial are
74271829ddc0 clonebundles: move a bundle of clone bundle related code to a new module Pierre-Yves David <pierre-yves.david@octobus.net> parents: diff changeset	233 readable from an older version.
74271829ddc0 clonebundles: move a bundle of clone bundle related code to a new module Pierre-Yves David <pierre-yves.david@octobus.net> parents: diff changeset	234
74271829ddc0 clonebundles: move a bundle of clone bundle related code to a new module Pierre-Yves David <pierre-yves.david@octobus.net> parents: diff changeset	235 The string currently has the form:
74271829ddc0 clonebundles: move a bundle of clone bundle related code to a new module Pierre-Yves David <pierre-yves.david@octobus.net> parents: diff changeset	236
74271829ddc0 clonebundles: move a bundle of clone bundle related code to a new module Pierre-Yves David <pierre-yves.david@octobus.net> parents: diff changeset	237 <compression>-<type>[;<parameter0>[;<parameter1>]]
74271829ddc0 clonebundles: move a bundle of clone bundle related code to a new module Pierre-Yves David <pierre-yves.david@octobus.net> parents: diff changeset	238
74271829ddc0 clonebundles: move a bundle of clone bundle related code to a new module Pierre-Yves David <pierre-yves.david@octobus.net> parents: diff changeset	239 Where <compression> is one of the supported compression formats
74271829ddc0 clonebundles: move a bundle of clone bundle related code to a new module Pierre-Yves David <pierre-yves.david@octobus.net> parents: diff changeset	240 and <type> is (currently) a version string. A ";" can follow the type and
74271829ddc0 clonebundles: move a bundle of clone bundle related code to a new module Pierre-Yves David <pierre-yves.david@octobus.net> parents: diff changeset	241 all text afterwards is interpreted as URI encoded, ";" delimited key=value
74271829ddc0 clonebundles: move a bundle of clone bundle related code to a new module Pierre-Yves David <pierre-yves.david@octobus.net> parents: diff changeset	242 pairs.
74271829ddc0 clonebundles: move a bundle of clone bundle related code to a new module Pierre-Yves David <pierre-yves.david@octobus.net> parents: diff changeset	243
74271829ddc0 clonebundles: move a bundle of clone bundle related code to a new module Pierre-Yves David <pierre-yves.david@octobus.net> parents: diff changeset	244 If ``strict`` is True (the default) <compression> is required. Otherwise,
74271829ddc0 clonebundles: move a bundle of clone bundle related code to a new module Pierre-Yves David <pierre-yves.david@octobus.net> parents: diff changeset	245 it is optional.
74271829ddc0 clonebundles: move a bundle of clone bundle related code to a new module Pierre-Yves David <pierre-yves.david@octobus.net> parents: diff changeset	246
74271829ddc0 clonebundles: move a bundle of clone bundle related code to a new module Pierre-Yves David <pierre-yves.david@octobus.net> parents: diff changeset	247 Returns a bundlespec object of (compression, version, parameters).
74271829ddc0 clonebundles: move a bundle of clone bundle related code to a new module Pierre-Yves David <pierre-yves.david@octobus.net> parents: diff changeset	248 Compression will be ``None`` if not in strict mode and a compression isn't
74271829ddc0 clonebundles: move a bundle of clone bundle related code to a new module Pierre-Yves David <pierre-yves.david@octobus.net> parents: diff changeset	249 defined.
74271829ddc0 clonebundles: move a bundle of clone bundle related code to a new module Pierre-Yves David <pierre-yves.david@octobus.net> parents: diff changeset	250
74271829ddc0 clonebundles: move a bundle of clone bundle related code to a new module Pierre-Yves David <pierre-yves.david@octobus.net> parents: diff changeset	251 An ``InvalidBundleSpecification`` is raised when the specification is
74271829ddc0 clonebundles: move a bundle of clone bundle related code to a new module Pierre-Yves David <pierre-yves.david@octobus.net> parents: diff changeset	252 not syntactically well formed.
74271829ddc0 clonebundles: move a bundle of clone bundle related code to a new module Pierre-Yves David <pierre-yves.david@octobus.net> parents: diff changeset	253
74271829ddc0 clonebundles: move a bundle of clone bundle related code to a new module Pierre-Yves David <pierre-yves.david@octobus.net> parents: diff changeset	254 An ``UnsupportedBundleSpecification`` is raised when the compression or
74271829ddc0 clonebundles: move a bundle of clone bundle related code to a new module Pierre-Yves David <pierre-yves.david@octobus.net> parents: diff changeset	255 bundle type/version is not recognized.
74271829ddc0 clonebundles: move a bundle of clone bundle related code to a new module Pierre-Yves David <pierre-yves.david@octobus.net> parents: diff changeset	256
74271829ddc0 clonebundles: move a bundle of clone bundle related code to a new module Pierre-Yves David <pierre-yves.david@octobus.net> parents: diff changeset	257 Note: this function will likely eventually return a more complex data
74271829ddc0 clonebundles: move a bundle of clone bundle related code to a new module Pierre-Yves David <pierre-yves.david@octobus.net> parents: diff changeset	258 structure, including bundle2 part information.
74271829ddc0 clonebundles: move a bundle of clone bundle related code to a new module Pierre-Yves David <pierre-yves.david@octobus.net> parents: diff changeset	259 """
74271829ddc0 clonebundles: move a bundle of clone bundle related code to a new module Pierre-Yves David <pierre-yves.david@octobus.net> parents: diff changeset	260 if strict and b'-' not in spec:
74271829ddc0 clonebundles: move a bundle of clone bundle related code to a new module Pierre-Yves David <pierre-yves.david@octobus.net> parents: diff changeset	261 raise error.InvalidBundleSpecification(
74271829ddc0 clonebundles: move a bundle of clone bundle related code to a new module Pierre-Yves David <pierre-yves.david@octobus.net> parents: diff changeset	262 _(
74271829ddc0 clonebundles: move a bundle of clone bundle related code to a new module Pierre-Yves David <pierre-yves.david@octobus.net> parents: diff changeset	263 b'invalid bundle specification; '
74271829ddc0 clonebundles: move a bundle of clone bundle related code to a new module Pierre-Yves David <pierre-yves.david@octobus.net> parents: diff changeset	264 b'must be prefixed with compression: %s'
74271829ddc0 clonebundles: move a bundle of clone bundle related code to a new module Pierre-Yves David <pierre-yves.david@octobus.net> parents: diff changeset	265 )
74271829ddc0 clonebundles: move a bundle of clone bundle related code to a new module Pierre-Yves David <pierre-yves.david@octobus.net> parents: diff changeset	266 % spec
74271829ddc0 clonebundles: move a bundle of clone bundle related code to a new module Pierre-Yves David <pierre-yves.david@octobus.net> parents: diff changeset	267 )
74271829ddc0 clonebundles: move a bundle of clone bundle related code to a new module Pierre-Yves David <pierre-yves.david@octobus.net> parents: diff changeset	268
49337 d89bfc075289 bundlespec: do not check for `-` in the params portion of the bundlespec Pierre-Yves David <pierre-yves.david@octobus.net> parents: 49336 diff changeset	269 pre_args = spec.split(b';', 1)[0]
d89bfc075289 bundlespec: do not check for `-` in the params portion of the bundlespec Pierre-Yves David <pierre-yves.david@octobus.net> parents: 49336 diff changeset	270 if b'-' in pre_args:
45807 74271829ddc0 clonebundles: move a bundle of clone bundle related code to a new module Pierre-Yves David <pierre-yves.david@octobus.net> parents: diff changeset	271 compression, version = spec.split(b'-', 1)
74271829ddc0 clonebundles: move a bundle of clone bundle related code to a new module Pierre-Yves David <pierre-yves.david@octobus.net> parents: diff changeset	272
74271829ddc0 clonebundles: move a bundle of clone bundle related code to a new module Pierre-Yves David <pierre-yves.david@octobus.net> parents: diff changeset	273 if compression not in util.compengines.supportedbundlenames:
74271829ddc0 clonebundles: move a bundle of clone bundle related code to a new module Pierre-Yves David <pierre-yves.david@octobus.net> parents: diff changeset	274 raise error.UnsupportedBundleSpecification(
74271829ddc0 clonebundles: move a bundle of clone bundle related code to a new module Pierre-Yves David <pierre-yves.david@octobus.net> parents: diff changeset	275 _(b'%s compression is not supported') % compression
74271829ddc0 clonebundles: move a bundle of clone bundle related code to a new module Pierre-Yves David <pierre-yves.david@octobus.net> parents: diff changeset	276 )
74271829ddc0 clonebundles: move a bundle of clone bundle related code to a new module Pierre-Yves David <pierre-yves.david@octobus.net> parents: diff changeset	277
49335 5d17dd74177d bundlespec: extract the parseparams closure Pierre-Yves David <pierre-yves.david@octobus.net> parents: 49331 diff changeset	278 version, params = _parseparams(version)
45807 74271829ddc0 clonebundles: move a bundle of clone bundle related code to a new module Pierre-Yves David <pierre-yves.david@octobus.net> parents: diff changeset	279
49326 c12c843f0db7 bundlespec: phase out the `_bundlespeccgversions` mapping Pierre-Yves David <pierre-yves.david@octobus.net> parents: 49037 diff changeset	280 if version not in _bundlespeccontentopts:
45807 74271829ddc0 clonebundles: move a bundle of clone bundle related code to a new module Pierre-Yves David <pierre-yves.david@octobus.net> parents: diff changeset	281 raise error.UnsupportedBundleSpecification(
74271829ddc0 clonebundles: move a bundle of clone bundle related code to a new module Pierre-Yves David <pierre-yves.david@octobus.net> parents: diff changeset	282 _(b'%s is not a recognized bundle version') % version
74271829ddc0 clonebundles: move a bundle of clone bundle related code to a new module Pierre-Yves David <pierre-yves.david@octobus.net> parents: diff changeset	283 )
74271829ddc0 clonebundles: move a bundle of clone bundle related code to a new module Pierre-Yves David <pierre-yves.david@octobus.net> parents: diff changeset	284 else:
74271829ddc0 clonebundles: move a bundle of clone bundle related code to a new module Pierre-Yves David <pierre-yves.david@octobus.net> parents: diff changeset	285 # Value could be just the compression or just the version, in which
74271829ddc0 clonebundles: move a bundle of clone bundle related code to a new module Pierre-Yves David <pierre-yves.david@octobus.net> parents: diff changeset	286 # case some defaults are assumed (but only when not in strict mode).
74271829ddc0 clonebundles: move a bundle of clone bundle related code to a new module Pierre-Yves David <pierre-yves.david@octobus.net> parents: diff changeset	287 assert not strict
74271829ddc0 clonebundles: move a bundle of clone bundle related code to a new module Pierre-Yves David <pierre-yves.david@octobus.net> parents: diff changeset	288
49335 5d17dd74177d bundlespec: extract the parseparams closure Pierre-Yves David <pierre-yves.david@octobus.net> parents: 49331 diff changeset	289 spec, params = _parseparams(spec)
45807 74271829ddc0 clonebundles: move a bundle of clone bundle related code to a new module Pierre-Yves David <pierre-yves.david@octobus.net> parents: diff changeset	290
74271829ddc0 clonebundles: move a bundle of clone bundle related code to a new module Pierre-Yves David <pierre-yves.david@octobus.net> parents: diff changeset	291 if spec in util.compengines.supportedbundlenames:
74271829ddc0 clonebundles: move a bundle of clone bundle related code to a new module Pierre-Yves David <pierre-yves.david@octobus.net> parents: diff changeset	292 compression = spec
74271829ddc0 clonebundles: move a bundle of clone bundle related code to a new module Pierre-Yves David <pierre-yves.david@octobus.net> parents: diff changeset	293 version = b'v1'
74271829ddc0 clonebundles: move a bundle of clone bundle related code to a new module Pierre-Yves David <pierre-yves.david@octobus.net> parents: diff changeset	294 # Generaldelta repos require v2.
46666 f4c325bf80fc requirements: also add a generaldelta constant Rapha?l Gom?s <rgomes@octobus.net> parents: 45808 diff changeset	295 if requirementsmod.GENERALDELTA_REQUIREMENT in repo.requirements:
45807 74271829ddc0 clonebundles: move a bundle of clone bundle related code to a new module Pierre-Yves David <pierre-yves.david@octobus.net> parents: diff changeset	296 version = b'v2'
47350 dc3800569157 revlogv2: make sure bundling pick a compatible bundle format Pierre-Yves David <pierre-yves.david@octobus.net> parents: 46666 diff changeset	297 elif requirementsmod.REVLOGV2_REQUIREMENT in repo.requirements:
dc3800569157 revlogv2: make sure bundling pick a compatible bundle format Pierre-Yves David <pierre-yves.david@octobus.net> parents: 46666 diff changeset	298 version = b'v2'
45807 74271829ddc0 clonebundles: move a bundle of clone bundle related code to a new module Pierre-Yves David <pierre-yves.david@octobus.net> parents: diff changeset	299 # Modern compression engines require v2.
74271829ddc0 clonebundles: move a bundle of clone bundle related code to a new module Pierre-Yves David <pierre-yves.david@octobus.net> parents: diff changeset	300 if compression not in _bundlespecv1compengines:
74271829ddc0 clonebundles: move a bundle of clone bundle related code to a new module Pierre-Yves David <pierre-yves.david@octobus.net> parents: diff changeset	301 version = b'v2'
49326 c12c843f0db7 bundlespec: phase out the `_bundlespeccgversions` mapping Pierre-Yves David <pierre-yves.david@octobus.net> parents: 49037 diff changeset	302 elif spec in _bundlespeccontentopts:
45807 74271829ddc0 clonebundles: move a bundle of clone bundle related code to a new module Pierre-Yves David <pierre-yves.david@octobus.net> parents: diff changeset	303 if spec == b'packed1':
74271829ddc0 clonebundles: move a bundle of clone bundle related code to a new module Pierre-Yves David <pierre-yves.david@octobus.net> parents: diff changeset	304 compression = b'none'
74271829ddc0 clonebundles: move a bundle of clone bundle related code to a new module Pierre-Yves David <pierre-yves.david@octobus.net> parents: diff changeset	305 else:
74271829ddc0 clonebundles: move a bundle of clone bundle related code to a new module Pierre-Yves David <pierre-yves.david@octobus.net> parents: diff changeset	306 compression = b'bzip2'
74271829ddc0 clonebundles: move a bundle of clone bundle related code to a new module Pierre-Yves David <pierre-yves.david@octobus.net> parents: diff changeset	307 version = spec
74271829ddc0 clonebundles: move a bundle of clone bundle related code to a new module Pierre-Yves David <pierre-yves.david@octobus.net> parents: diff changeset	308 else:
74271829ddc0 clonebundles: move a bundle of clone bundle related code to a new module Pierre-Yves David <pierre-yves.david@octobus.net> parents: diff changeset	309 raise error.UnsupportedBundleSpecification(
74271829ddc0 clonebundles: move a bundle of clone bundle related code to a new module Pierre-Yves David <pierre-yves.david@octobus.net> parents: diff changeset	310 _(b'%s is not a recognized bundle specification') % spec
74271829ddc0 clonebundles: move a bundle of clone bundle related code to a new module Pierre-Yves David <pierre-yves.david@octobus.net> parents: diff changeset	311 )
74271829ddc0 clonebundles: move a bundle of clone bundle related code to a new module Pierre-Yves David <pierre-yves.david@octobus.net> parents: diff changeset	312
74271829ddc0 clonebundles: move a bundle of clone bundle related code to a new module Pierre-Yves David <pierre-yves.david@octobus.net> parents: diff changeset	313 # Bundle version 1 only supports a known set of compression engines.
74271829ddc0 clonebundles: move a bundle of clone bundle related code to a new module Pierre-Yves David <pierre-yves.david@octobus.net> parents: diff changeset	314 if version == b'v1' and compression not in _bundlespecv1compengines:
74271829ddc0 clonebundles: move a bundle of clone bundle related code to a new module Pierre-Yves David <pierre-yves.david@octobus.net> parents: diff changeset	315 raise error.UnsupportedBundleSpecification(
74271829ddc0 clonebundles: move a bundle of clone bundle related code to a new module Pierre-Yves David <pierre-yves.david@octobus.net> parents: diff changeset	316 _(b'compression engine %s is not supported on v1 bundles')
74271829ddc0 clonebundles: move a bundle of clone bundle related code to a new module Pierre-Yves David <pierre-yves.david@octobus.net> parents: diff changeset	317 % compression
74271829ddc0 clonebundles: move a bundle of clone bundle related code to a new module Pierre-Yves David <pierre-yves.david@octobus.net> parents: diff changeset	318 )
74271829ddc0 clonebundles: move a bundle of clone bundle related code to a new module Pierre-Yves David <pierre-yves.david@octobus.net> parents: diff changeset	319
52271 961900fbd67c bundles: filter out unsupported requirements for non-packed1 format Julien Cristau <jcristau@mozilla.com> parents: 52012 diff changeset	320 # The specification for stream bundles can optionally declare the data formats
45807 74271829ddc0 clonebundles: move a bundle of clone bundle related code to a new module Pierre-Yves David <pierre-yves.david@octobus.net> parents: diff changeset	321 # required to apply it. If we see this metadata, compare against what the
74271829ddc0 clonebundles: move a bundle of clone bundle related code to a new module Pierre-Yves David <pierre-yves.david@octobus.net> parents: diff changeset	322 # repo supports and error if the bundle isn't compatible.
52271 961900fbd67c bundles: filter out unsupported requirements for non-packed1 format Julien Cristau <jcristau@mozilla.com> parents: 52012 diff changeset	323 if b'requirements' in params:
49764 9be765b82a90 typing: minor tweaks to allow updating to pytype 2022.11.18 Matt Harbison <matt_harbison@yahoo.com> parents: 49337 diff changeset	324 requirements = set(cast(bytes, params[b'requirements']).split(b','))
48623 baddab229b86 stream-clone: add a explicit set list requirements relevant to stream clone Pierre-Yves David <pierre-yves.david@octobus.net> parents: 47350 diff changeset	325 missingreqs = requirements - requirementsmod.STREAM_FIXED_REQUIREMENTS
45807 74271829ddc0 clonebundles: move a bundle of clone bundle related code to a new module Pierre-Yves David <pierre-yves.david@octobus.net> parents: diff changeset	326 if missingreqs:
74271829ddc0 clonebundles: move a bundle of clone bundle related code to a new module Pierre-Yves David <pierre-yves.david@octobus.net> parents: diff changeset	327 raise error.UnsupportedBundleSpecification(
74271829ddc0 clonebundles: move a bundle of clone bundle related code to a new module Pierre-Yves David <pierre-yves.david@octobus.net> parents: diff changeset	328 _(b'missing support for repository features: %s')
74271829ddc0 clonebundles: move a bundle of clone bundle related code to a new module Pierre-Yves David <pierre-yves.david@octobus.net> parents: diff changeset	329 % b', '.join(sorted(missingreqs))
74271829ddc0 clonebundles: move a bundle of clone bundle related code to a new module Pierre-Yves David <pierre-yves.david@octobus.net> parents: diff changeset	330 )
74271829ddc0 clonebundles: move a bundle of clone bundle related code to a new module Pierre-Yves David <pierre-yves.david@octobus.net> parents: diff changeset	331
74271829ddc0 clonebundles: move a bundle of clone bundle related code to a new module Pierre-Yves David <pierre-yves.david@octobus.net> parents: diff changeset	332 # Compute contentopts based on the version
50694 a41eeb877d07 branching: merge with stable Pierre-Yves David <pierre-yves.david@octobus.net> parents: 50371 diff changeset	333 if b"stream" in params:
a41eeb877d07 branching: merge with stable Pierre-Yves David <pierre-yves.david@octobus.net> parents: 50371 diff changeset	334 # This case is fishy as this mostly derails the version selection
49327 d587f09cad98 bundlespec: make the `stream` case less special Pierre-Yves David <pierre-yves.david@octobus.net> parents: 49326 diff changeset	335 # mechanism. `stream` bundles are quite specific and used differently
d587f09cad98 bundlespec: make the `stream` case less special Pierre-Yves David <pierre-yves.david@octobus.net> parents: 49326 diff changeset	336 # as "normal" bundles.
d587f09cad98 bundlespec: make the `stream` case less special Pierre-Yves David <pierre-yves.david@octobus.net> parents: 49326 diff changeset	337 #
d587f09cad98 bundlespec: make the `stream` case less special Pierre-Yves David <pierre-yves.david@octobus.net> parents: 49326 diff changeset	338 # (we should probably define a cleaner way to do this and raise a
50694 a41eeb877d07 branching: merge with stable Pierre-Yves David <pierre-yves.david@octobus.net> parents: 50371 diff changeset	339 # warning when the old way is encountered)
a41eeb877d07 branching: merge with stable Pierre-Yves David <pierre-yves.david@octobus.net> parents: 50371 diff changeset	340 if params[b"stream"] == b"v2":
a41eeb877d07 branching: merge with stable Pierre-Yves David <pierre-yves.david@octobus.net> parents: 50371 diff changeset	341 version = b"streamv2"
a41eeb877d07 branching: merge with stable Pierre-Yves David <pierre-yves.david@octobus.net> parents: 50371 diff changeset	342 if params[b"stream"] == b"v3-exp":
a41eeb877d07 branching: merge with stable Pierre-Yves David <pierre-yves.david@octobus.net> parents: 50371 diff changeset	343 version = b"streamv3-exp"
45807 74271829ddc0 clonebundles: move a bundle of clone bundle related code to a new module Pierre-Yves David <pierre-yves.david@octobus.net> parents: diff changeset	344 contentopts = _bundlespeccontentopts.get(version, {}).copy()
50694 a41eeb877d07 branching: merge with stable Pierre-Yves David <pierre-yves.david@octobus.net> parents: 50371 diff changeset	345 if version == b"streamv2" or version == b"streamv3-exp":
49327 d587f09cad98 bundlespec: make the `stream` case less special Pierre-Yves David <pierre-yves.david@octobus.net> parents: 49326 diff changeset	346 # streamv2 have been reported as "v2" for a while.
d587f09cad98 bundlespec: make the `stream` case less special Pierre-Yves David <pierre-yves.david@octobus.net> parents: 49326 diff changeset	347 version = b"v2"
45807 74271829ddc0 clonebundles: move a bundle of clone bundle related code to a new module Pierre-Yves David <pierre-yves.david@octobus.net> parents: diff changeset	348
74271829ddc0 clonebundles: move a bundle of clone bundle related code to a new module Pierre-Yves David <pierre-yves.david@octobus.net> parents: diff changeset	349 engine = util.compengines.forbundlename(compression)
74271829ddc0 clonebundles: move a bundle of clone bundle related code to a new module Pierre-Yves David <pierre-yves.david@octobus.net> parents: diff changeset	350 compression, wirecompression = engine.bundletype()
49326 c12c843f0db7 bundlespec: phase out the `_bundlespeccgversions` mapping Pierre-Yves David <pierre-yves.david@octobus.net> parents: 49037 diff changeset	351 wireversion = _bundlespeccontentopts[version][b'cg.version']
45807 74271829ddc0 clonebundles: move a bundle of clone bundle related code to a new module Pierre-Yves David <pierre-yves.david@octobus.net> parents: diff changeset	352
74271829ddc0 clonebundles: move a bundle of clone bundle related code to a new module Pierre-Yves David <pierre-yves.david@octobus.net> parents: diff changeset	353 return bundlespec(
74271829ddc0 clonebundles: move a bundle of clone bundle related code to a new module Pierre-Yves David <pierre-yves.david@octobus.net> parents: diff changeset	354 compression, wirecompression, version, wireversion, params, contentopts
74271829ddc0 clonebundles: move a bundle of clone bundle related code to a new module Pierre-Yves David <pierre-yves.david@octobus.net> parents: diff changeset	355 )
74271829ddc0 clonebundles: move a bundle of clone bundle related code to a new module Pierre-Yves David <pierre-yves.david@octobus.net> parents: diff changeset	356
74271829ddc0 clonebundles: move a bundle of clone bundle related code to a new module Pierre-Yves David <pierre-yves.david@octobus.net> parents: diff changeset	357
74271829ddc0 clonebundles: move a bundle of clone bundle related code to a new module Pierre-Yves David <pierre-yves.david@octobus.net> parents: diff changeset	358 def parseclonebundlesmanifest(repo, s):
74271829ddc0 clonebundles: move a bundle of clone bundle related code to a new module Pierre-Yves David <pierre-yves.david@octobus.net> parents: diff changeset	359 """Parses the raw text of a clone bundles manifest.
74271829ddc0 clonebundles: move a bundle of clone bundle related code to a new module Pierre-Yves David <pierre-yves.david@octobus.net> parents: diff changeset	360
74271829ddc0 clonebundles: move a bundle of clone bundle related code to a new module Pierre-Yves David <pierre-yves.david@octobus.net> parents: diff changeset	361 Returns a list of dicts. The dicts have a ``URL`` key corresponding
74271829ddc0 clonebundles: move a bundle of clone bundle related code to a new module Pierre-Yves David <pierre-yves.david@octobus.net> parents: diff changeset	362 to the URL and other keys are the attributes for the entry.
74271829ddc0 clonebundles: move a bundle of clone bundle related code to a new module Pierre-Yves David <pierre-yves.david@octobus.net> parents: diff changeset	363 """
74271829ddc0 clonebundles: move a bundle of clone bundle related code to a new module Pierre-Yves David <pierre-yves.david@octobus.net> parents: diff changeset	364 m = []
74271829ddc0 clonebundles: move a bundle of clone bundle related code to a new module Pierre-Yves David <pierre-yves.david@octobus.net> parents: diff changeset	365 for line in s.splitlines():
74271829ddc0 clonebundles: move a bundle of clone bundle related code to a new module Pierre-Yves David <pierre-yves.david@octobus.net> parents: diff changeset	366 fields = line.split()
74271829ddc0 clonebundles: move a bundle of clone bundle related code to a new module Pierre-Yves David <pierre-yves.david@octobus.net> parents: diff changeset	367 if not fields:
74271829ddc0 clonebundles: move a bundle of clone bundle related code to a new module Pierre-Yves David <pierre-yves.david@octobus.net> parents: diff changeset	368 continue
74271829ddc0 clonebundles: move a bundle of clone bundle related code to a new module Pierre-Yves David <pierre-yves.david@octobus.net> parents: diff changeset	369 attrs = {b'URL': fields[0]}
74271829ddc0 clonebundles: move a bundle of clone bundle related code to a new module Pierre-Yves David <pierre-yves.david@octobus.net> parents: diff changeset	370 for rawattr in fields[1:]:
74271829ddc0 clonebundles: move a bundle of clone bundle related code to a new module Pierre-Yves David <pierre-yves.david@octobus.net> parents: diff changeset	371 key, value = rawattr.split(b'=', 1)
74271829ddc0 clonebundles: move a bundle of clone bundle related code to a new module Pierre-Yves David <pierre-yves.david@octobus.net> parents: diff changeset	372 key = util.urlreq.unquote(key)
74271829ddc0 clonebundles: move a bundle of clone bundle related code to a new module Pierre-Yves David <pierre-yves.david@octobus.net> parents: diff changeset	373 value = util.urlreq.unquote(value)
74271829ddc0 clonebundles: move a bundle of clone bundle related code to a new module Pierre-Yves David <pierre-yves.david@octobus.net> parents: diff changeset	374 attrs[key] = value
74271829ddc0 clonebundles: move a bundle of clone bundle related code to a new module Pierre-Yves David <pierre-yves.david@octobus.net> parents: diff changeset	375
74271829ddc0 clonebundles: move a bundle of clone bundle related code to a new module Pierre-Yves David <pierre-yves.david@octobus.net> parents: diff changeset	376 # Parse BUNDLESPEC into components. This makes client-side
74271829ddc0 clonebundles: move a bundle of clone bundle related code to a new module Pierre-Yves David <pierre-yves.david@octobus.net> parents: diff changeset	377 # preferences easier to specify since you can prefer a single
74271829ddc0 clonebundles: move a bundle of clone bundle related code to a new module Pierre-Yves David <pierre-yves.david@octobus.net> parents: diff changeset	378 # component of the BUNDLESPEC.
74271829ddc0 clonebundles: move a bundle of clone bundle related code to a new module Pierre-Yves David <pierre-yves.david@octobus.net> parents: diff changeset	379 if key == b'BUNDLESPEC':
74271829ddc0 clonebundles: move a bundle of clone bundle related code to a new module Pierre-Yves David <pierre-yves.david@octobus.net> parents: diff changeset	380 try:
74271829ddc0 clonebundles: move a bundle of clone bundle related code to a new module Pierre-Yves David <pierre-yves.david@octobus.net> parents: diff changeset	381 bundlespec = parsebundlespec(repo, value)
74271829ddc0 clonebundles: move a bundle of clone bundle related code to a new module Pierre-Yves David <pierre-yves.david@octobus.net> parents: diff changeset	382 attrs[b'COMPRESSION'] = bundlespec.compression
74271829ddc0 clonebundles: move a bundle of clone bundle related code to a new module Pierre-Yves David <pierre-yves.david@octobus.net> parents: diff changeset	383 attrs[b'VERSION'] = bundlespec.version
74271829ddc0 clonebundles: move a bundle of clone bundle related code to a new module Pierre-Yves David <pierre-yves.david@octobus.net> parents: diff changeset	384 except error.InvalidBundleSpecification:
74271829ddc0 clonebundles: move a bundle of clone bundle related code to a new module Pierre-Yves David <pierre-yves.david@octobus.net> parents: diff changeset	385 pass
74271829ddc0 clonebundles: move a bundle of clone bundle related code to a new module Pierre-Yves David <pierre-yves.david@octobus.net> parents: diff changeset	386 except error.UnsupportedBundleSpecification:
74271829ddc0 clonebundles: move a bundle of clone bundle related code to a new module Pierre-Yves David <pierre-yves.david@octobus.net> parents: diff changeset	387 pass
74271829ddc0 clonebundles: move a bundle of clone bundle related code to a new module Pierre-Yves David <pierre-yves.david@octobus.net> parents: diff changeset	388
74271829ddc0 clonebundles: move a bundle of clone bundle related code to a new module Pierre-Yves David <pierre-yves.david@octobus.net> parents: diff changeset	389 m.append(attrs)
74271829ddc0 clonebundles: move a bundle of clone bundle related code to a new module Pierre-Yves David <pierre-yves.david@octobus.net> parents: diff changeset	390
74271829ddc0 clonebundles: move a bundle of clone bundle related code to a new module Pierre-Yves David <pierre-yves.david@octobus.net> parents: diff changeset	391 return m
74271829ddc0 clonebundles: move a bundle of clone bundle related code to a new module Pierre-Yves David <pierre-yves.david@octobus.net> parents: diff changeset	392
74271829ddc0 clonebundles: move a bundle of clone bundle related code to a new module Pierre-Yves David <pierre-yves.david@octobus.net> parents: diff changeset	393
74271829ddc0 clonebundles: move a bundle of clone bundle related code to a new module Pierre-Yves David <pierre-yves.david@octobus.net> parents: diff changeset	394 def isstreamclonespec(bundlespec):
74271829ddc0 clonebundles: move a bundle of clone bundle related code to a new module Pierre-Yves David <pierre-yves.david@octobus.net> parents: diff changeset	395 # Stream clone v1
74271829ddc0 clonebundles: move a bundle of clone bundle related code to a new module Pierre-Yves David <pierre-yves.david@octobus.net> parents: diff changeset	396 if bundlespec.wirecompression == b'UN' and bundlespec.wireversion == b's1':
74271829ddc0 clonebundles: move a bundle of clone bundle related code to a new module Pierre-Yves David <pierre-yves.david@octobus.net> parents: diff changeset	397 return True
74271829ddc0 clonebundles: move a bundle of clone bundle related code to a new module Pierre-Yves David <pierre-yves.david@octobus.net> parents: diff changeset	398
74271829ddc0 clonebundles: move a bundle of clone bundle related code to a new module Pierre-Yves David <pierre-yves.david@octobus.net> parents: diff changeset	399 # Stream clone v2
74271829ddc0 clonebundles: move a bundle of clone bundle related code to a new module Pierre-Yves David <pierre-yves.david@octobus.net> parents: diff changeset	400 if (
74271829ddc0 clonebundles: move a bundle of clone bundle related code to a new module Pierre-Yves David <pierre-yves.david@octobus.net> parents: diff changeset	401 bundlespec.wirecompression == b'UN'
74271829ddc0 clonebundles: move a bundle of clone bundle related code to a new module Pierre-Yves David <pierre-yves.david@octobus.net> parents: diff changeset	402 and bundlespec.wireversion == b'02'
51596 c4aab3661f25 bundlespec: rationalize the way we specify stream bundle version Pierre-Yves David <pierre-yves.david@octobus.net> parents: 50712 diff changeset	403 and bundlespec.contentopts.get(b'stream', None) in (b"v2", b"v3-exp")
45807 74271829ddc0 clonebundles: move a bundle of clone bundle related code to a new module Pierre-Yves David <pierre-yves.david@octobus.net> parents: diff changeset	404 ):
74271829ddc0 clonebundles: move a bundle of clone bundle related code to a new module Pierre-Yves David <pierre-yves.david@octobus.net> parents: diff changeset	405 return True
74271829ddc0 clonebundles: move a bundle of clone bundle related code to a new module Pierre-Yves David <pierre-yves.david@octobus.net> parents: diff changeset	406
74271829ddc0 clonebundles: move a bundle of clone bundle related code to a new module Pierre-Yves David <pierre-yves.david@octobus.net> parents: diff changeset	407 return False
74271829ddc0 clonebundles: move a bundle of clone bundle related code to a new module Pierre-Yves David <pierre-yves.david@octobus.net> parents: diff changeset	408
74271829ddc0 clonebundles: move a bundle of clone bundle related code to a new module Pierre-Yves David <pierre-yves.david@octobus.net> parents: diff changeset	409
52012 aa7f4a45d8fa clonebundles: allow manifest to specify sha256 digest of bundles Joerg Sonnenberger <joerg@bec.de> parents: 51902 diff changeset	410 digest_regex = re.compile(b'^[a-z0-9]+:[0-9a-f]+(,[a-z0-9]+:[0-9a-f]+)*$')
aa7f4a45d8fa clonebundles: allow manifest to specify sha256 digest of bundles Joerg Sonnenberger <joerg@bec.de> parents: 51902 diff changeset	411
aa7f4a45d8fa clonebundles: allow manifest to specify sha256 digest of bundles Joerg Sonnenberger <joerg@bec.de> parents: 51902 diff changeset	412
50371 7b723217d368 clonebundles: filter out invalid schemes instead of failing on them Mathias De Mare <mathias.de_mare@nokia.com> parents: 49764 diff changeset	413 def filterclonebundleentries(
7b723217d368 clonebundles: filter out invalid schemes instead of failing on them Mathias De Mare <mathias.de_mare@nokia.com> parents: 49764 diff changeset	414 repo, entries, streamclonerequested=False, pullbundles=False
7b723217d368 clonebundles: filter out invalid schemes instead of failing on them Mathias De Mare <mathias.de_mare@nokia.com> parents: 49764 diff changeset	415 ):
45807 74271829ddc0 clonebundles: move a bundle of clone bundle related code to a new module Pierre-Yves David <pierre-yves.david@octobus.net> parents: diff changeset	416 """Remove incompatible clone bundle manifest entries.
74271829ddc0 clonebundles: move a bundle of clone bundle related code to a new module Pierre-Yves David <pierre-yves.david@octobus.net> parents: diff changeset	417
74271829ddc0 clonebundles: move a bundle of clone bundle related code to a new module Pierre-Yves David <pierre-yves.david@octobus.net> parents: diff changeset	418 Accepts a list of entries parsed with ``parseclonebundlesmanifest``
74271829ddc0 clonebundles: move a bundle of clone bundle related code to a new module Pierre-Yves David <pierre-yves.david@octobus.net> parents: diff changeset	419 and returns a new list consisting of only the entries that this client
74271829ddc0 clonebundles: move a bundle of clone bundle related code to a new module Pierre-Yves David <pierre-yves.david@octobus.net> parents: diff changeset	420 should be able to apply.
74271829ddc0 clonebundles: move a bundle of clone bundle related code to a new module Pierre-Yves David <pierre-yves.david@octobus.net> parents: diff changeset	421
74271829ddc0 clonebundles: move a bundle of clone bundle related code to a new module Pierre-Yves David <pierre-yves.david@octobus.net> parents: diff changeset	422 There is no guarantee we'll be able to apply all returned entries because
74271829ddc0 clonebundles: move a bundle of clone bundle related code to a new module Pierre-Yves David <pierre-yves.david@octobus.net> parents: diff changeset	423 the metadata we use to filter on may be missing or wrong.
74271829ddc0 clonebundles: move a bundle of clone bundle related code to a new module Pierre-Yves David <pierre-yves.david@octobus.net> parents: diff changeset	424 """
74271829ddc0 clonebundles: move a bundle of clone bundle related code to a new module Pierre-Yves David <pierre-yves.david@octobus.net> parents: diff changeset	425 newentries = []
74271829ddc0 clonebundles: move a bundle of clone bundle related code to a new module Pierre-Yves David <pierre-yves.david@octobus.net> parents: diff changeset	426 for entry in entries:
50371 7b723217d368 clonebundles: filter out invalid schemes instead of failing on them Mathias De Mare <mathias.de_mare@nokia.com> parents: 49764 diff changeset	427 url = entry.get(b'URL')
7b723217d368 clonebundles: filter out invalid schemes instead of failing on them Mathias De Mare <mathias.de_mare@nokia.com> parents: 49764 diff changeset	428 if not pullbundles and not any(
7b723217d368 clonebundles: filter out invalid schemes instead of failing on them Mathias De Mare <mathias.de_mare@nokia.com> parents: 49764 diff changeset	429 [url.startswith(scheme) for scheme in SUPPORTED_CLONEBUNDLE_SCHEMES]
7b723217d368 clonebundles: filter out invalid schemes instead of failing on them Mathias De Mare <mathias.de_mare@nokia.com> parents: 49764 diff changeset	430 ):
7b723217d368 clonebundles: filter out invalid schemes instead of failing on them Mathias De Mare <mathias.de_mare@nokia.com> parents: 49764 diff changeset	431 repo.ui.debug(
7b723217d368 clonebundles: filter out invalid schemes instead of failing on them Mathias De Mare <mathias.de_mare@nokia.com> parents: 49764 diff changeset	432 b'filtering %s because not a supported clonebundle scheme\n'
7b723217d368 clonebundles: filter out invalid schemes instead of failing on them Mathias De Mare <mathias.de_mare@nokia.com> parents: 49764 diff changeset	433 % url
7b723217d368 clonebundles: filter out invalid schemes instead of failing on them Mathias De Mare <mathias.de_mare@nokia.com> parents: 49764 diff changeset	434 )
7b723217d368 clonebundles: filter out invalid schemes instead of failing on them Mathias De Mare <mathias.de_mare@nokia.com> parents: 49764 diff changeset	435 continue
7b723217d368 clonebundles: filter out invalid schemes instead of failing on them Mathias De Mare <mathias.de_mare@nokia.com> parents: 49764 diff changeset	436
45807 74271829ddc0 clonebundles: move a bundle of clone bundle related code to a new module Pierre-Yves David <pierre-yves.david@octobus.net> parents: diff changeset	437 spec = entry.get(b'BUNDLESPEC')
74271829ddc0 clonebundles: move a bundle of clone bundle related code to a new module Pierre-Yves David <pierre-yves.david@octobus.net> parents: diff changeset	438 if spec:
74271829ddc0 clonebundles: move a bundle of clone bundle related code to a new module Pierre-Yves David <pierre-yves.david@octobus.net> parents: diff changeset	439 try:
74271829ddc0 clonebundles: move a bundle of clone bundle related code to a new module Pierre-Yves David <pierre-yves.david@octobus.net> parents: diff changeset	440 bundlespec = parsebundlespec(repo, spec, strict=True)
74271829ddc0 clonebundles: move a bundle of clone bundle related code to a new module Pierre-Yves David <pierre-yves.david@octobus.net> parents: diff changeset	441
74271829ddc0 clonebundles: move a bundle of clone bundle related code to a new module Pierre-Yves David <pierre-yves.david@octobus.net> parents: diff changeset	442 # If a stream clone was requested, filter out non-streamclone
74271829ddc0 clonebundles: move a bundle of clone bundle related code to a new module Pierre-Yves David <pierre-yves.david@octobus.net> parents: diff changeset	443 # entries.
74271829ddc0 clonebundles: move a bundle of clone bundle related code to a new module Pierre-Yves David <pierre-yves.david@octobus.net> parents: diff changeset	444 if streamclonerequested and not isstreamclonespec(bundlespec):
74271829ddc0 clonebundles: move a bundle of clone bundle related code to a new module Pierre-Yves David <pierre-yves.david@octobus.net> parents: diff changeset	445 repo.ui.debug(
50371 7b723217d368 clonebundles: filter out invalid schemes instead of failing on them Mathias De Mare <mathias.de_mare@nokia.com> parents: 49764 diff changeset	446 b'filtering %s because not a stream clone\n' % url
45807 74271829ddc0 clonebundles: move a bundle of clone bundle related code to a new module Pierre-Yves David <pierre-yves.david@octobus.net> parents: diff changeset	447 )
74271829ddc0 clonebundles: move a bundle of clone bundle related code to a new module Pierre-Yves David <pierre-yves.david@octobus.net> parents: diff changeset	448 continue
74271829ddc0 clonebundles: move a bundle of clone bundle related code to a new module Pierre-Yves David <pierre-yves.david@octobus.net> parents: diff changeset	449
74271829ddc0 clonebundles: move a bundle of clone bundle related code to a new module Pierre-Yves David <pierre-yves.david@octobus.net> parents: diff changeset	450 except error.InvalidBundleSpecification as e:
74271829ddc0 clonebundles: move a bundle of clone bundle related code to a new module Pierre-Yves David <pierre-yves.david@octobus.net> parents: diff changeset	451 repo.ui.debug(stringutil.forcebytestr(e) + b'\n')
74271829ddc0 clonebundles: move a bundle of clone bundle related code to a new module Pierre-Yves David <pierre-yves.david@octobus.net> parents: diff changeset	452 continue
74271829ddc0 clonebundles: move a bundle of clone bundle related code to a new module Pierre-Yves David <pierre-yves.david@octobus.net> parents: diff changeset	453 except error.UnsupportedBundleSpecification as e:
74271829ddc0 clonebundles: move a bundle of clone bundle related code to a new module Pierre-Yves David <pierre-yves.david@octobus.net> parents: diff changeset	454 repo.ui.debug(
74271829ddc0 clonebundles: move a bundle of clone bundle related code to a new module Pierre-Yves David <pierre-yves.david@octobus.net> parents: diff changeset	455 b'filtering %s because unsupported bundle '
50371 7b723217d368 clonebundles: filter out invalid schemes instead of failing on them Mathias De Mare <mathias.de_mare@nokia.com> parents: 49764 diff changeset	456 b'spec: %s\n' % (url, stringutil.forcebytestr(e))
45807 74271829ddc0 clonebundles: move a bundle of clone bundle related code to a new module Pierre-Yves David <pierre-yves.david@octobus.net> parents: diff changeset	457 )
74271829ddc0 clonebundles: move a bundle of clone bundle related code to a new module Pierre-Yves David <pierre-yves.david@octobus.net> parents: diff changeset	458 continue
74271829ddc0 clonebundles: move a bundle of clone bundle related code to a new module Pierre-Yves David <pierre-yves.david@octobus.net> parents: diff changeset	459 # If we don't have a spec and requested a stream clone, we don't know
74271829ddc0 clonebundles: move a bundle of clone bundle related code to a new module Pierre-Yves David <pierre-yves.david@octobus.net> parents: diff changeset	460 # what the entry is so don't attempt to apply it.
74271829ddc0 clonebundles: move a bundle of clone bundle related code to a new module Pierre-Yves David <pierre-yves.david@octobus.net> parents: diff changeset	461 elif streamclonerequested:
74271829ddc0 clonebundles: move a bundle of clone bundle related code to a new module Pierre-Yves David <pierre-yves.david@octobus.net> parents: diff changeset	462 repo.ui.debug(
74271829ddc0 clonebundles: move a bundle of clone bundle related code to a new module Pierre-Yves David <pierre-yves.david@octobus.net> parents: diff changeset	463 b'filtering %s because cannot determine if a stream '
50371 7b723217d368 clonebundles: filter out invalid schemes instead of failing on them Mathias De Mare <mathias.de_mare@nokia.com> parents: 49764 diff changeset	464 b'clone bundle\n' % url
45807 74271829ddc0 clonebundles: move a bundle of clone bundle related code to a new module Pierre-Yves David <pierre-yves.david@octobus.net> parents: diff changeset	465 )
74271829ddc0 clonebundles: move a bundle of clone bundle related code to a new module Pierre-Yves David <pierre-yves.david@octobus.net> parents: diff changeset	466 continue
74271829ddc0 clonebundles: move a bundle of clone bundle related code to a new module Pierre-Yves David <pierre-yves.david@octobus.net> parents: diff changeset	467
74271829ddc0 clonebundles: move a bundle of clone bundle related code to a new module Pierre-Yves David <pierre-yves.david@octobus.net> parents: diff changeset	468 if b'REQUIRESNI' in entry and not sslutil.hassni:
50371 7b723217d368 clonebundles: filter out invalid schemes instead of failing on them Mathias De Mare <mathias.de_mare@nokia.com> parents: 49764 diff changeset	469 repo.ui.debug(b'filtering %s because SNI not supported\n' % url)
45807 74271829ddc0 clonebundles: move a bundle of clone bundle related code to a new module Pierre-Yves David <pierre-yves.david@octobus.net> parents: diff changeset	470 continue
74271829ddc0 clonebundles: move a bundle of clone bundle related code to a new module Pierre-Yves David <pierre-yves.david@octobus.net> parents: diff changeset	471
74271829ddc0 clonebundles: move a bundle of clone bundle related code to a new module Pierre-Yves David <pierre-yves.david@octobus.net> parents: diff changeset	472 if b'REQUIREDRAM' in entry:
74271829ddc0 clonebundles: move a bundle of clone bundle related code to a new module Pierre-Yves David <pierre-yves.david@octobus.net> parents: diff changeset	473 try:
74271829ddc0 clonebundles: move a bundle of clone bundle related code to a new module Pierre-Yves David <pierre-yves.david@octobus.net> parents: diff changeset	474 requiredram = util.sizetoint(entry[b'REQUIREDRAM'])
74271829ddc0 clonebundles: move a bundle of clone bundle related code to a new module Pierre-Yves David <pierre-yves.david@octobus.net> parents: diff changeset	475 except error.ParseError:
74271829ddc0 clonebundles: move a bundle of clone bundle related code to a new module Pierre-Yves David <pierre-yves.david@octobus.net> parents: diff changeset	476 repo.ui.debug(
50371 7b723217d368 clonebundles: filter out invalid schemes instead of failing on them Mathias De Mare <mathias.de_mare@nokia.com> parents: 49764 diff changeset	477 b'filtering %s due to a bad REQUIREDRAM attribute\n' % url
45807 74271829ddc0 clonebundles: move a bundle of clone bundle related code to a new module Pierre-Yves David <pierre-yves.david@octobus.net> parents: diff changeset	478 )
74271829ddc0 clonebundles: move a bundle of clone bundle related code to a new module Pierre-Yves David <pierre-yves.david@octobus.net> parents: diff changeset	479 continue
74271829ddc0 clonebundles: move a bundle of clone bundle related code to a new module Pierre-Yves David <pierre-yves.david@octobus.net> parents: diff changeset	480 actualram = repo.ui.estimatememory()
74271829ddc0 clonebundles: move a bundle of clone bundle related code to a new module Pierre-Yves David <pierre-yves.david@octobus.net> parents: diff changeset	481 if actualram is not None and actualram * 0.66 < requiredram:
74271829ddc0 clonebundles: move a bundle of clone bundle related code to a new module Pierre-Yves David <pierre-yves.david@octobus.net> parents: diff changeset	482 repo.ui.debug(
74271829ddc0 clonebundles: move a bundle of clone bundle related code to a new module Pierre-Yves David <pierre-yves.david@octobus.net> parents: diff changeset	483 b'filtering %s as it needs more than 2/3 of system memory\n'
50371 7b723217d368 clonebundles: filter out invalid schemes instead of failing on them Mathias De Mare <mathias.de_mare@nokia.com> parents: 49764 diff changeset	484 % url
45807 74271829ddc0 clonebundles: move a bundle of clone bundle related code to a new module Pierre-Yves David <pierre-yves.david@octobus.net> parents: diff changeset	485 )
74271829ddc0 clonebundles: move a bundle of clone bundle related code to a new module Pierre-Yves David <pierre-yves.david@octobus.net> parents: diff changeset	486 continue
74271829ddc0 clonebundles: move a bundle of clone bundle related code to a new module Pierre-Yves David <pierre-yves.david@octobus.net> parents: diff changeset	487
52012 aa7f4a45d8fa clonebundles: allow manifest to specify sha256 digest of bundles Joerg Sonnenberger <joerg@bec.de> parents: 51902 diff changeset	488 if b'DIGEST' in entry:
aa7f4a45d8fa clonebundles: allow manifest to specify sha256 digest of bundles Joerg Sonnenberger <joerg@bec.de> parents: 51902 diff changeset	489 if not digest_regex.match(entry[b'DIGEST']):
aa7f4a45d8fa clonebundles: allow manifest to specify sha256 digest of bundles Joerg Sonnenberger <joerg@bec.de> parents: 51902 diff changeset	490 repo.ui.debug(
aa7f4a45d8fa clonebundles: allow manifest to specify sha256 digest of bundles Joerg Sonnenberger <joerg@bec.de> parents: 51902 diff changeset	491 b'filtering %s due to a bad DIGEST attribute\n' % url
aa7f4a45d8fa clonebundles: allow manifest to specify sha256 digest of bundles Joerg Sonnenberger <joerg@bec.de> parents: 51902 diff changeset	492 )
aa7f4a45d8fa clonebundles: allow manifest to specify sha256 digest of bundles Joerg Sonnenberger <joerg@bec.de> parents: 51902 diff changeset	493 continue
aa7f4a45d8fa clonebundles: allow manifest to specify sha256 digest of bundles Joerg Sonnenberger <joerg@bec.de> parents: 51902 diff changeset	494 supported = 0
aa7f4a45d8fa clonebundles: allow manifest to specify sha256 digest of bundles Joerg Sonnenberger <joerg@bec.de> parents: 51902 diff changeset	495 seen = {}
aa7f4a45d8fa clonebundles: allow manifest to specify sha256 digest of bundles Joerg Sonnenberger <joerg@bec.de> parents: 51902 diff changeset	496 for digest_entry in entry[b'DIGEST'].split(b','):
aa7f4a45d8fa clonebundles: allow manifest to specify sha256 digest of bundles Joerg Sonnenberger <joerg@bec.de> parents: 51902 diff changeset	497 algo, digest = digest_entry.split(b':')
aa7f4a45d8fa clonebundles: allow manifest to specify sha256 digest of bundles Joerg Sonnenberger <joerg@bec.de> parents: 51902 diff changeset	498 if algo not in seen:
aa7f4a45d8fa clonebundles: allow manifest to specify sha256 digest of bundles Joerg Sonnenberger <joerg@bec.de> parents: 51902 diff changeset	499 seen[algo] = digest
aa7f4a45d8fa clonebundles: allow manifest to specify sha256 digest of bundles Joerg Sonnenberger <joerg@bec.de> parents: 51902 diff changeset	500 elif seen[algo] != digest:
aa7f4a45d8fa clonebundles: allow manifest to specify sha256 digest of bundles Joerg Sonnenberger <joerg@bec.de> parents: 51902 diff changeset	501 repo.ui.debug(
aa7f4a45d8fa clonebundles: allow manifest to specify sha256 digest of bundles Joerg Sonnenberger <joerg@bec.de> parents: 51902 diff changeset	502 b'filtering %s due to conflicting %s digests\n'
aa7f4a45d8fa clonebundles: allow manifest to specify sha256 digest of bundles Joerg Sonnenberger <joerg@bec.de> parents: 51902 diff changeset	503 % (url, algo)
aa7f4a45d8fa clonebundles: allow manifest to specify sha256 digest of bundles Joerg Sonnenberger <joerg@bec.de> parents: 51902 diff changeset	504 )
aa7f4a45d8fa clonebundles: allow manifest to specify sha256 digest of bundles Joerg Sonnenberger <joerg@bec.de> parents: 51902 diff changeset	505 supported = 0
aa7f4a45d8fa clonebundles: allow manifest to specify sha256 digest of bundles Joerg Sonnenberger <joerg@bec.de> parents: 51902 diff changeset	506 break
aa7f4a45d8fa clonebundles: allow manifest to specify sha256 digest of bundles Joerg Sonnenberger <joerg@bec.de> parents: 51902 diff changeset	507 digester = urlmod.digesthandler.digest_algorithms.get(algo)
aa7f4a45d8fa clonebundles: allow manifest to specify sha256 digest of bundles Joerg Sonnenberger <joerg@bec.de> parents: 51902 diff changeset	508 if digester is None:
aa7f4a45d8fa clonebundles: allow manifest to specify sha256 digest of bundles Joerg Sonnenberger <joerg@bec.de> parents: 51902 diff changeset	509 continue
aa7f4a45d8fa clonebundles: allow manifest to specify sha256 digest of bundles Joerg Sonnenberger <joerg@bec.de> parents: 51902 diff changeset	510 if len(digest) != digester().digest_size * 2:
aa7f4a45d8fa clonebundles: allow manifest to specify sha256 digest of bundles Joerg Sonnenberger <joerg@bec.de> parents: 51902 diff changeset	511 repo.ui.debug(
aa7f4a45d8fa clonebundles: allow manifest to specify sha256 digest of bundles Joerg Sonnenberger <joerg@bec.de> parents: 51902 diff changeset	512 b'filtering %s due to a bad %s digest\n' % (url, algo)
aa7f4a45d8fa clonebundles: allow manifest to specify sha256 digest of bundles Joerg Sonnenberger <joerg@bec.de> parents: 51902 diff changeset	513 )
aa7f4a45d8fa clonebundles: allow manifest to specify sha256 digest of bundles Joerg Sonnenberger <joerg@bec.de> parents: 51902 diff changeset	514 supported = 0
aa7f4a45d8fa clonebundles: allow manifest to specify sha256 digest of bundles Joerg Sonnenberger <joerg@bec.de> parents: 51902 diff changeset	515 break
aa7f4a45d8fa clonebundles: allow manifest to specify sha256 digest of bundles Joerg Sonnenberger <joerg@bec.de> parents: 51902 diff changeset	516 supported += 1
aa7f4a45d8fa clonebundles: allow manifest to specify sha256 digest of bundles Joerg Sonnenberger <joerg@bec.de> parents: 51902 diff changeset	517 else:
aa7f4a45d8fa clonebundles: allow manifest to specify sha256 digest of bundles Joerg Sonnenberger <joerg@bec.de> parents: 51902 diff changeset	518 if supported == 0:
aa7f4a45d8fa clonebundles: allow manifest to specify sha256 digest of bundles Joerg Sonnenberger <joerg@bec.de> parents: 51902 diff changeset	519 repo.ui.debug(
aa7f4a45d8fa clonebundles: allow manifest to specify sha256 digest of bundles Joerg Sonnenberger <joerg@bec.de> parents: 51902 diff changeset	520 b'filtering %s due to lack of supported digest\n' % url
aa7f4a45d8fa clonebundles: allow manifest to specify sha256 digest of bundles Joerg Sonnenberger <joerg@bec.de> parents: 51902 diff changeset	521 )
aa7f4a45d8fa clonebundles: allow manifest to specify sha256 digest of bundles Joerg Sonnenberger <joerg@bec.de> parents: 51902 diff changeset	522 if supported == 0:
aa7f4a45d8fa clonebundles: allow manifest to specify sha256 digest of bundles Joerg Sonnenberger <joerg@bec.de> parents: 51902 diff changeset	523 continue
aa7f4a45d8fa clonebundles: allow manifest to specify sha256 digest of bundles Joerg Sonnenberger <joerg@bec.de> parents: 51902 diff changeset	524
45807 74271829ddc0 clonebundles: move a bundle of clone bundle related code to a new module Pierre-Yves David <pierre-yves.david@octobus.net> parents: diff changeset	525 newentries.append(entry)
74271829ddc0 clonebundles: move a bundle of clone bundle related code to a new module Pierre-Yves David <pierre-yves.david@octobus.net> parents: diff changeset	526
74271829ddc0 clonebundles: move a bundle of clone bundle related code to a new module Pierre-Yves David <pierre-yves.david@octobus.net> parents: diff changeset	527 return newentries
74271829ddc0 clonebundles: move a bundle of clone bundle related code to a new module Pierre-Yves David <pierre-yves.david@octobus.net> parents: diff changeset	528
74271829ddc0 clonebundles: move a bundle of clone bundle related code to a new module Pierre-Yves David <pierre-yves.david@octobus.net> parents: diff changeset	529
49037 642e31cb55f0 py3: use class X: instead of class X(object): Gregory Szorc <gregory.szorc@gmail.com> parents: 48623 diff changeset	530 class clonebundleentry:
45807 74271829ddc0 clonebundles: move a bundle of clone bundle related code to a new module Pierre-Yves David <pierre-yves.david@octobus.net> parents: diff changeset	531 """Represents an item in a clone bundles manifest.
74271829ddc0 clonebundles: move a bundle of clone bundle related code to a new module Pierre-Yves David <pierre-yves.david@octobus.net> parents: diff changeset	532
74271829ddc0 clonebundles: move a bundle of clone bundle related code to a new module Pierre-Yves David <pierre-yves.david@octobus.net> parents: diff changeset	533 This rich class is needed to support sorting since sorted() in Python 3
74271829ddc0 clonebundles: move a bundle of clone bundle related code to a new module Pierre-Yves David <pierre-yves.david@octobus.net> parents: diff changeset	534 doesn't support ``cmp`` and our comparison is complex enough that ``key=``
74271829ddc0 clonebundles: move a bundle of clone bundle related code to a new module Pierre-Yves David <pierre-yves.david@octobus.net> parents: diff changeset	535 won't work.
74271829ddc0 clonebundles: move a bundle of clone bundle related code to a new module Pierre-Yves David <pierre-yves.david@octobus.net> parents: diff changeset	536 """
74271829ddc0 clonebundles: move a bundle of clone bundle related code to a new module Pierre-Yves David <pierre-yves.david@octobus.net> parents: diff changeset	537
74271829ddc0 clonebundles: move a bundle of clone bundle related code to a new module Pierre-Yves David <pierre-yves.david@octobus.net> parents: diff changeset	538 def __init__(self, value, prefers):
74271829ddc0 clonebundles: move a bundle of clone bundle related code to a new module Pierre-Yves David <pierre-yves.david@octobus.net> parents: diff changeset	539 self.value = value
74271829ddc0 clonebundles: move a bundle of clone bundle related code to a new module Pierre-Yves David <pierre-yves.david@octobus.net> parents: diff changeset	540 self.prefers = prefers
74271829ddc0 clonebundles: move a bundle of clone bundle related code to a new module Pierre-Yves David <pierre-yves.david@octobus.net> parents: diff changeset	541
74271829ddc0 clonebundles: move a bundle of clone bundle related code to a new module Pierre-Yves David <pierre-yves.david@octobus.net> parents: diff changeset	542 def _cmp(self, other):
74271829ddc0 clonebundles: move a bundle of clone bundle related code to a new module Pierre-Yves David <pierre-yves.david@octobus.net> parents: diff changeset	543 for prefkey, prefvalue in self.prefers:
74271829ddc0 clonebundles: move a bundle of clone bundle related code to a new module Pierre-Yves David <pierre-yves.david@octobus.net> parents: diff changeset	544 avalue = self.value.get(prefkey)
74271829ddc0 clonebundles: move a bundle of clone bundle related code to a new module Pierre-Yves David <pierre-yves.david@octobus.net> parents: diff changeset	545 bvalue = other.value.get(prefkey)
74271829ddc0 clonebundles: move a bundle of clone bundle related code to a new module Pierre-Yves David <pierre-yves.david@octobus.net> parents: diff changeset	546
74271829ddc0 clonebundles: move a bundle of clone bundle related code to a new module Pierre-Yves David <pierre-yves.david@octobus.net> parents: diff changeset	547 # Special case for b missing attribute and a matches exactly.
74271829ddc0 clonebundles: move a bundle of clone bundle related code to a new module Pierre-Yves David <pierre-yves.david@octobus.net> parents: diff changeset	548 if avalue is not None and bvalue is None and avalue == prefvalue:
74271829ddc0 clonebundles: move a bundle of clone bundle related code to a new module Pierre-Yves David <pierre-yves.david@octobus.net> parents: diff changeset	549 return -1
74271829ddc0 clonebundles: move a bundle of clone bundle related code to a new module Pierre-Yves David <pierre-yves.david@octobus.net> parents: diff changeset	550
74271829ddc0 clonebundles: move a bundle of clone bundle related code to a new module Pierre-Yves David <pierre-yves.david@octobus.net> parents: diff changeset	551 # Special case for a missing attribute and b matches exactly.
74271829ddc0 clonebundles: move a bundle of clone bundle related code to a new module Pierre-Yves David <pierre-yves.david@octobus.net> parents: diff changeset	552 if bvalue is not None and avalue is None and bvalue == prefvalue:
74271829ddc0 clonebundles: move a bundle of clone bundle related code to a new module Pierre-Yves David <pierre-yves.david@octobus.net> parents: diff changeset	553 return 1
74271829ddc0 clonebundles: move a bundle of clone bundle related code to a new module Pierre-Yves David <pierre-yves.david@octobus.net> parents: diff changeset	554
74271829ddc0 clonebundles: move a bundle of clone bundle related code to a new module Pierre-Yves David <pierre-yves.david@octobus.net> parents: diff changeset	555 # We can't compare unless attribute present on both.
74271829ddc0 clonebundles: move a bundle of clone bundle related code to a new module Pierre-Yves David <pierre-yves.david@octobus.net> parents: diff changeset	556 if avalue is None or bvalue is None:
74271829ddc0 clonebundles: move a bundle of clone bundle related code to a new module Pierre-Yves David <pierre-yves.david@octobus.net> parents: diff changeset	557 continue
74271829ddc0 clonebundles: move a bundle of clone bundle related code to a new module Pierre-Yves David <pierre-yves.david@octobus.net> parents: diff changeset	558
74271829ddc0 clonebundles: move a bundle of clone bundle related code to a new module Pierre-Yves David <pierre-yves.david@octobus.net> parents: diff changeset	559 # Same values should fall back to next attribute.
74271829ddc0 clonebundles: move a bundle of clone bundle related code to a new module Pierre-Yves David <pierre-yves.david@octobus.net> parents: diff changeset	560 if avalue == bvalue:
74271829ddc0 clonebundles: move a bundle of clone bundle related code to a new module Pierre-Yves David <pierre-yves.david@octobus.net> parents: diff changeset	561 continue
74271829ddc0 clonebundles: move a bundle of clone bundle related code to a new module Pierre-Yves David <pierre-yves.david@octobus.net> parents: diff changeset	562
74271829ddc0 clonebundles: move a bundle of clone bundle related code to a new module Pierre-Yves David <pierre-yves.david@octobus.net> parents: diff changeset	563 # Exact matches come first.
74271829ddc0 clonebundles: move a bundle of clone bundle related code to a new module Pierre-Yves David <pierre-yves.david@octobus.net> parents: diff changeset	564 if avalue == prefvalue:
74271829ddc0 clonebundles: move a bundle of clone bundle related code to a new module Pierre-Yves David <pierre-yves.david@octobus.net> parents: diff changeset	565 return -1
74271829ddc0 clonebundles: move a bundle of clone bundle related code to a new module Pierre-Yves David <pierre-yves.david@octobus.net> parents: diff changeset	566 if bvalue == prefvalue:
74271829ddc0 clonebundles: move a bundle of clone bundle related code to a new module Pierre-Yves David <pierre-yves.david@octobus.net> parents: diff changeset	567 return 1
74271829ddc0 clonebundles: move a bundle of clone bundle related code to a new module Pierre-Yves David <pierre-yves.david@octobus.net> parents: diff changeset	568
74271829ddc0 clonebundles: move a bundle of clone bundle related code to a new module Pierre-Yves David <pierre-yves.david@octobus.net> parents: diff changeset	569 # Fall back to next attribute.
74271829ddc0 clonebundles: move a bundle of clone bundle related code to a new module Pierre-Yves David <pierre-yves.david@octobus.net> parents: diff changeset	570 continue
74271829ddc0 clonebundles: move a bundle of clone bundle related code to a new module Pierre-Yves David <pierre-yves.david@octobus.net> parents: diff changeset	571
74271829ddc0 clonebundles: move a bundle of clone bundle related code to a new module Pierre-Yves David <pierre-yves.david@octobus.net> parents: diff changeset	572 # If we got here we couldn't sort by attributes and prefers. Fall
74271829ddc0 clonebundles: move a bundle of clone bundle related code to a new module Pierre-Yves David <pierre-yves.david@octobus.net> parents: diff changeset	573 # back to index order.
74271829ddc0 clonebundles: move a bundle of clone bundle related code to a new module Pierre-Yves David <pierre-yves.david@octobus.net> parents: diff changeset	574 return 0
74271829ddc0 clonebundles: move a bundle of clone bundle related code to a new module Pierre-Yves David <pierre-yves.david@octobus.net> parents: diff changeset	575
74271829ddc0 clonebundles: move a bundle of clone bundle related code to a new module Pierre-Yves David <pierre-yves.david@octobus.net> parents: diff changeset	576 def __lt__(self, other):
74271829ddc0 clonebundles: move a bundle of clone bundle related code to a new module Pierre-Yves David <pierre-yves.david@octobus.net> parents: diff changeset	577 return self._cmp(other) < 0
74271829ddc0 clonebundles: move a bundle of clone bundle related code to a new module Pierre-Yves David <pierre-yves.david@octobus.net> parents: diff changeset	578
74271829ddc0 clonebundles: move a bundle of clone bundle related code to a new module Pierre-Yves David <pierre-yves.david@octobus.net> parents: diff changeset	579 def __gt__(self, other):
74271829ddc0 clonebundles: move a bundle of clone bundle related code to a new module Pierre-Yves David <pierre-yves.david@octobus.net> parents: diff changeset	580 return self._cmp(other) > 0
74271829ddc0 clonebundles: move a bundle of clone bundle related code to a new module Pierre-Yves David <pierre-yves.david@octobus.net> parents: diff changeset	581
74271829ddc0 clonebundles: move a bundle of clone bundle related code to a new module Pierre-Yves David <pierre-yves.david@octobus.net> parents: diff changeset	582 def __eq__(self, other):
74271829ddc0 clonebundles: move a bundle of clone bundle related code to a new module Pierre-Yves David <pierre-yves.david@octobus.net> parents: diff changeset	583 return self._cmp(other) == 0
74271829ddc0 clonebundles: move a bundle of clone bundle related code to a new module Pierre-Yves David <pierre-yves.david@octobus.net> parents: diff changeset	584
74271829ddc0 clonebundles: move a bundle of clone bundle related code to a new module Pierre-Yves David <pierre-yves.david@octobus.net> parents: diff changeset	585 def __le__(self, other):
74271829ddc0 clonebundles: move a bundle of clone bundle related code to a new module Pierre-Yves David <pierre-yves.david@octobus.net> parents: diff changeset	586 return self._cmp(other) <= 0
74271829ddc0 clonebundles: move a bundle of clone bundle related code to a new module Pierre-Yves David <pierre-yves.david@octobus.net> parents: diff changeset	587
74271829ddc0 clonebundles: move a bundle of clone bundle related code to a new module Pierre-Yves David <pierre-yves.david@octobus.net> parents: diff changeset	588 def __ge__(self, other):
74271829ddc0 clonebundles: move a bundle of clone bundle related code to a new module Pierre-Yves David <pierre-yves.david@octobus.net> parents: diff changeset	589 return self._cmp(other) >= 0
74271829ddc0 clonebundles: move a bundle of clone bundle related code to a new module Pierre-Yves David <pierre-yves.david@octobus.net> parents: diff changeset	590
74271829ddc0 clonebundles: move a bundle of clone bundle related code to a new module Pierre-Yves David <pierre-yves.david@octobus.net> parents: diff changeset	591 def __ne__(self, other):
74271829ddc0 clonebundles: move a bundle of clone bundle related code to a new module Pierre-Yves David <pierre-yves.david@octobus.net> parents: diff changeset	592 return self._cmp(other) != 0
74271829ddc0 clonebundles: move a bundle of clone bundle related code to a new module Pierre-Yves David <pierre-yves.david@octobus.net> parents: diff changeset	593
74271829ddc0 clonebundles: move a bundle of clone bundle related code to a new module Pierre-Yves David <pierre-yves.david@octobus.net> parents: diff changeset	594
74271829ddc0 clonebundles: move a bundle of clone bundle related code to a new module Pierre-Yves David <pierre-yves.david@octobus.net> parents: diff changeset	595 def sortclonebundleentries(ui, entries):
74271829ddc0 clonebundles: move a bundle of clone bundle related code to a new module Pierre-Yves David <pierre-yves.david@octobus.net> parents: diff changeset	596 prefers = ui.configlist(b'ui', b'clonebundleprefers')
74271829ddc0 clonebundles: move a bundle of clone bundle related code to a new module Pierre-Yves David <pierre-yves.david@octobus.net> parents: diff changeset	597 if not prefers:
74271829ddc0 clonebundles: move a bundle of clone bundle related code to a new module Pierre-Yves David <pierre-yves.david@octobus.net> parents: diff changeset	598 return list(entries)
74271829ddc0 clonebundles: move a bundle of clone bundle related code to a new module Pierre-Yves David <pierre-yves.david@octobus.net> parents: diff changeset	599
74271829ddc0 clonebundles: move a bundle of clone bundle related code to a new module Pierre-Yves David <pierre-yves.david@octobus.net> parents: diff changeset	600 def _split(p):
74271829ddc0 clonebundles: move a bundle of clone bundle related code to a new module Pierre-Yves David <pierre-yves.david@octobus.net> parents: diff changeset	601 if b'=' not in p:
74271829ddc0 clonebundles: move a bundle of clone bundle related code to a new module Pierre-Yves David <pierre-yves.david@octobus.net> parents: diff changeset	602 hint = _(b"each comma separated item should be key=value pairs")
74271829ddc0 clonebundles: move a bundle of clone bundle related code to a new module Pierre-Yves David <pierre-yves.david@octobus.net> parents: diff changeset	603 raise error.Abort(
74271829ddc0 clonebundles: move a bundle of clone bundle related code to a new module Pierre-Yves David <pierre-yves.david@octobus.net> parents: diff changeset	604 _(b"invalid ui.clonebundleprefers item: %s") % p, hint=hint
74271829ddc0 clonebundles: move a bundle of clone bundle related code to a new module Pierre-Yves David <pierre-yves.david@octobus.net> parents: diff changeset	605 )
74271829ddc0 clonebundles: move a bundle of clone bundle related code to a new module Pierre-Yves David <pierre-yves.david@octobus.net> parents: diff changeset	606 return p.split(b'=', 1)
74271829ddc0 clonebundles: move a bundle of clone bundle related code to a new module Pierre-Yves David <pierre-yves.david@octobus.net> parents: diff changeset	607
74271829ddc0 clonebundles: move a bundle of clone bundle related code to a new module Pierre-Yves David <pierre-yves.david@octobus.net> parents: diff changeset	608 prefers = [_split(p) for p in prefers]
74271829ddc0 clonebundles: move a bundle of clone bundle related code to a new module Pierre-Yves David <pierre-yves.david@octobus.net> parents: diff changeset	609
74271829ddc0 clonebundles: move a bundle of clone bundle related code to a new module Pierre-Yves David <pierre-yves.david@octobus.net> parents: diff changeset	610 items = sorted(clonebundleentry(v, prefers) for v in entries)
74271829ddc0 clonebundles: move a bundle of clone bundle related code to a new module Pierre-Yves David <pierre-yves.david@octobus.net> parents: diff changeset	611 return [i.value for i in items]

Mercurial > public > mercurial-scm > hg-stable

annotate mercurial/bundlecaches.py @ 52971:469b9a628b51