mercurial-scm/hg: mercurial/bundlecaches.py annotate

annotate mercurial/bundlecaches.py @ 50539:7b723217d368 stable

clonebundles: filter out invalid schemes instead of failing on them Previously, an invalid clonebundle scheme would result in a failed clone. By specifying a list of schemes we support, we can make sure adding a new scheme (like the one for inline clonebundles) does not result in clones failing for older clients.

author	Mathias De Mare <mathias.de_mare@nokia.com>
date	Thu, 20 Apr 2023 11:23:45 +0200
parents	9be765b82a90
children	a41eeb877d07

rev	line source
45784 74271829ddc0 clonebundles: move a bundle of clone bundle related code to a new module Pierre-Yves David <pierre-yves.david@octobus.net> parents: diff changeset	1 # bundlecaches.py - utility to deal with pre-computed bundle for servers
74271829ddc0 clonebundles: move a bundle of clone bundle related code to a new module Pierre-Yves David <pierre-yves.david@octobus.net> parents: diff changeset	2 #
74271829ddc0 clonebundles: move a bundle of clone bundle related code to a new module Pierre-Yves David <pierre-yves.david@octobus.net> parents: diff changeset	3 # This software may be used and distributed according to the terms of the
74271829ddc0 clonebundles: move a bundle of clone bundle related code to a new module Pierre-Yves David <pierre-yves.david@octobus.net> parents: diff changeset	4 # GNU General Public License version 2 or any later version.
74271829ddc0 clonebundles: move a bundle of clone bundle related code to a new module Pierre-Yves David <pierre-yves.david@octobus.net> parents: diff changeset	5
49325 bf66f7a1e3f8 bundlespec: merge the contentopts and params dictionnary Pierre-Yves David <pierre-yves.david@octobus.net> parents: 49322 diff changeset	6 import collections
bf66f7a1e3f8 bundlespec: merge the contentopts and params dictionnary Pierre-Yves David <pierre-yves.david@octobus.net> parents: 49322 diff changeset	7
49648 9be765b82a90 typing: minor tweaks to allow updating to pytype 2022.11.18 Matt Harbison <matt_harbison@yahoo.com> parents: 49332 diff changeset	8 from typing import (
9be765b82a90 typing: minor tweaks to allow updating to pytype 2022.11.18 Matt Harbison <matt_harbison@yahoo.com> parents: 49332 diff changeset	9 cast,
9be765b82a90 typing: minor tweaks to allow updating to pytype 2022.11.18 Matt Harbison <matt_harbison@yahoo.com> parents: 49332 diff changeset	10 )
9be765b82a90 typing: minor tweaks to allow updating to pytype 2022.11.18 Matt Harbison <matt_harbison@yahoo.com> parents: 49332 diff changeset	11
45784 74271829ddc0 clonebundles: move a bundle of clone bundle related code to a new module Pierre-Yves David <pierre-yves.david@octobus.net> parents: diff changeset	12 from .i18n import _
74271829ddc0 clonebundles: move a bundle of clone bundle related code to a new module Pierre-Yves David <pierre-yves.david@octobus.net> parents: diff changeset	13
74271829ddc0 clonebundles: move a bundle of clone bundle related code to a new module Pierre-Yves David <pierre-yves.david@octobus.net> parents: diff changeset	14 from .thirdparty import attr
74271829ddc0 clonebundles: move a bundle of clone bundle related code to a new module Pierre-Yves David <pierre-yves.david@octobus.net> parents: diff changeset	15
74271829ddc0 clonebundles: move a bundle of clone bundle related code to a new module Pierre-Yves David <pierre-yves.david@octobus.net> parents: diff changeset	16 from . import (
74271829ddc0 clonebundles: move a bundle of clone bundle related code to a new module Pierre-Yves David <pierre-yves.david@octobus.net> parents: diff changeset	17 error,
46627 f4c325bf80fc requirements: also add a generaldelta constant Rapha?l Gom?s <rgomes@octobus.net> parents: 45785 diff changeset	18 requirements as requirementsmod,
45784 74271829ddc0 clonebundles: move a bundle of clone bundle related code to a new module Pierre-Yves David <pierre-yves.david@octobus.net> parents: diff changeset	19 sslutil,
74271829ddc0 clonebundles: move a bundle of clone bundle related code to a new module Pierre-Yves David <pierre-yves.david@octobus.net> parents: diff changeset	20 util,
74271829ddc0 clonebundles: move a bundle of clone bundle related code to a new module Pierre-Yves David <pierre-yves.david@octobus.net> parents: diff changeset	21 )
74271829ddc0 clonebundles: move a bundle of clone bundle related code to a new module Pierre-Yves David <pierre-yves.david@octobus.net> parents: diff changeset	22 from .utils import stringutil
74271829ddc0 clonebundles: move a bundle of clone bundle related code to a new module Pierre-Yves David <pierre-yves.david@octobus.net> parents: diff changeset	23
74271829ddc0 clonebundles: move a bundle of clone bundle related code to a new module Pierre-Yves David <pierre-yves.david@octobus.net> parents: diff changeset	24 urlreq = util.urlreq
74271829ddc0 clonebundles: move a bundle of clone bundle related code to a new module Pierre-Yves David <pierre-yves.david@octobus.net> parents: diff changeset	25
45785 80f32ec8653a clonebundle: move the manifest filename to a constant Pierre-Yves David <pierre-yves.david@octobus.net> parents: 45784 diff changeset	26 CB_MANIFEST_FILE = b'clonebundles.manifest'
50539 7b723217d368 clonebundles: filter out invalid schemes instead of failing on them Mathias De Mare <mathias.de_mare@nokia.com> parents: 49648 diff changeset	27 SUPPORTED_CLONEBUNDLE_SCHEMES = [
7b723217d368 clonebundles: filter out invalid schemes instead of failing on them Mathias De Mare <mathias.de_mare@nokia.com> parents: 49648 diff changeset	28 b"http://",
7b723217d368 clonebundles: filter out invalid schemes instead of failing on them Mathias De Mare <mathias.de_mare@nokia.com> parents: 49648 diff changeset	29 b"https://",
7b723217d368 clonebundles: filter out invalid schemes instead of failing on them Mathias De Mare <mathias.de_mare@nokia.com> parents: 49648 diff changeset	30 b"largefile://",
7b723217d368 clonebundles: filter out invalid schemes instead of failing on them Mathias De Mare <mathias.de_mare@nokia.com> parents: 49648 diff changeset	31 ]
45785 80f32ec8653a clonebundle: move the manifest filename to a constant Pierre-Yves David <pierre-yves.david@octobus.net> parents: 45784 diff changeset	32
45784 74271829ddc0 clonebundles: move a bundle of clone bundle related code to a new module Pierre-Yves David <pierre-yves.david@octobus.net> parents: diff changeset	33
74271829ddc0 clonebundles: move a bundle of clone bundle related code to a new module Pierre-Yves David <pierre-yves.david@octobus.net> parents: diff changeset	34 @attr.s
48946 642e31cb55f0 py3: use class X: instead of class X(object): Gregory Szorc <gregory.szorc@gmail.com> parents: 48601 diff changeset	35 class bundlespec:
45784 74271829ddc0 clonebundles: move a bundle of clone bundle related code to a new module Pierre-Yves David <pierre-yves.david@octobus.net> parents: diff changeset	36 compression = attr.ib()
74271829ddc0 clonebundles: move a bundle of clone bundle related code to a new module Pierre-Yves David <pierre-yves.david@octobus.net> parents: diff changeset	37 wirecompression = attr.ib()
74271829ddc0 clonebundles: move a bundle of clone bundle related code to a new module Pierre-Yves David <pierre-yves.david@octobus.net> parents: diff changeset	38 version = attr.ib()
74271829ddc0 clonebundles: move a bundle of clone bundle related code to a new module Pierre-Yves David <pierre-yves.david@octobus.net> parents: diff changeset	39 wireversion = attr.ib()
49325 bf66f7a1e3f8 bundlespec: merge the contentopts and params dictionnary Pierre-Yves David <pierre-yves.david@octobus.net> parents: 49322 diff changeset	40 # parameters explicitly overwritten by the config or the specification
bf66f7a1e3f8 bundlespec: merge the contentopts and params dictionnary Pierre-Yves David <pierre-yves.david@octobus.net> parents: 49322 diff changeset	41 _explicit_params = attr.ib()
bf66f7a1e3f8 bundlespec: merge the contentopts and params dictionnary Pierre-Yves David <pierre-yves.david@octobus.net> parents: 49322 diff changeset	42 # default parameter for the version
bf66f7a1e3f8 bundlespec: merge the contentopts and params dictionnary Pierre-Yves David <pierre-yves.david@octobus.net> parents: 49322 diff changeset	43 #
bf66f7a1e3f8 bundlespec: merge the contentopts and params dictionnary Pierre-Yves David <pierre-yves.david@octobus.net> parents: 49322 diff changeset	44 # Keeping it separated is useful to check what was actually overwritten.
bf66f7a1e3f8 bundlespec: merge the contentopts and params dictionnary Pierre-Yves David <pierre-yves.david@octobus.net> parents: 49322 diff changeset	45 _default_opts = attr.ib()
bf66f7a1e3f8 bundlespec: merge the contentopts and params dictionnary Pierre-Yves David <pierre-yves.david@octobus.net> parents: 49322 diff changeset	46
bf66f7a1e3f8 bundlespec: merge the contentopts and params dictionnary Pierre-Yves David <pierre-yves.david@octobus.net> parents: 49322 diff changeset	47 @property
bf66f7a1e3f8 bundlespec: merge the contentopts and params dictionnary Pierre-Yves David <pierre-yves.david@octobus.net> parents: 49322 diff changeset	48 def params(self):
bf66f7a1e3f8 bundlespec: merge the contentopts and params dictionnary Pierre-Yves David <pierre-yves.david@octobus.net> parents: 49322 diff changeset	49 return collections.ChainMap(self._explicit_params, self._default_opts)
bf66f7a1e3f8 bundlespec: merge the contentopts and params dictionnary Pierre-Yves David <pierre-yves.david@octobus.net> parents: 49322 diff changeset	50
bf66f7a1e3f8 bundlespec: merge the contentopts and params dictionnary Pierre-Yves David <pierre-yves.david@octobus.net> parents: 49322 diff changeset	51 @property
bf66f7a1e3f8 bundlespec: merge the contentopts and params dictionnary Pierre-Yves David <pierre-yves.david@octobus.net> parents: 49322 diff changeset	52 def contentopts(self):
bf66f7a1e3f8 bundlespec: merge the contentopts and params dictionnary Pierre-Yves David <pierre-yves.david@octobus.net> parents: 49322 diff changeset	53 # kept for Backward Compatibility concerns.
bf66f7a1e3f8 bundlespec: merge the contentopts and params dictionnary Pierre-Yves David <pierre-yves.david@octobus.net> parents: 49322 diff changeset	54 return self.params
bf66f7a1e3f8 bundlespec: merge the contentopts and params dictionnary Pierre-Yves David <pierre-yves.david@octobus.net> parents: 49322 diff changeset	55
49326 3840d16595cf bundlespec: add a `overwrite` parameter to set_param Pierre-Yves David <pierre-yves.david@octobus.net> parents: 49325 diff changeset	56 def set_param(self, key, value, overwrite=True):
3840d16595cf bundlespec: add a `overwrite` parameter to set_param Pierre-Yves David <pierre-yves.david@octobus.net> parents: 49325 diff changeset	57 """Set a bundle parameter value.
3840d16595cf bundlespec: add a `overwrite` parameter to set_param Pierre-Yves David <pierre-yves.david@octobus.net> parents: 49325 diff changeset	58
3840d16595cf bundlespec: add a `overwrite` parameter to set_param Pierre-Yves David <pierre-yves.david@octobus.net> parents: 49325 diff changeset	59 Will only overwrite if overwrite is true"""
3840d16595cf bundlespec: add a `overwrite` parameter to set_param Pierre-Yves David <pierre-yves.david@octobus.net> parents: 49325 diff changeset	60 if overwrite or key not in self._explicit_params:
3840d16595cf bundlespec: add a `overwrite` parameter to set_param Pierre-Yves David <pierre-yves.david@octobus.net> parents: 49325 diff changeset	61 self._explicit_params[key] = value
45784 74271829ddc0 clonebundles: move a bundle of clone bundle related code to a new module Pierre-Yves David <pierre-yves.david@octobus.net> parents: diff changeset	62
74271829ddc0 clonebundles: move a bundle of clone bundle related code to a new module Pierre-Yves David <pierre-yves.david@octobus.net> parents: diff changeset	63
74271829ddc0 clonebundles: move a bundle of clone bundle related code to a new module Pierre-Yves David <pierre-yves.david@octobus.net> parents: diff changeset	64 # Maps bundle version human names to changegroup versions.
74271829ddc0 clonebundles: move a bundle of clone bundle related code to a new module Pierre-Yves David <pierre-yves.david@octobus.net> parents: diff changeset	65 _bundlespeccgversions = {
74271829ddc0 clonebundles: move a bundle of clone bundle related code to a new module Pierre-Yves David <pierre-yves.david@octobus.net> parents: diff changeset	66 b'v1': b'01',
74271829ddc0 clonebundles: move a bundle of clone bundle related code to a new module Pierre-Yves David <pierre-yves.david@octobus.net> parents: diff changeset	67 b'v2': b'02',
74271829ddc0 clonebundles: move a bundle of clone bundle related code to a new module Pierre-Yves David <pierre-yves.david@octobus.net> parents: diff changeset	68 b'packed1': b's1',
74271829ddc0 clonebundles: move a bundle of clone bundle related code to a new module Pierre-Yves David <pierre-yves.david@octobus.net> parents: diff changeset	69 b'bundle2': b'02', # legacy
74271829ddc0 clonebundles: move a bundle of clone bundle related code to a new module Pierre-Yves David <pierre-yves.david@octobus.net> parents: diff changeset	70 }
74271829ddc0 clonebundles: move a bundle of clone bundle related code to a new module Pierre-Yves David <pierre-yves.david@octobus.net> parents: diff changeset	71
74271829ddc0 clonebundles: move a bundle of clone bundle related code to a new module Pierre-Yves David <pierre-yves.david@octobus.net> parents: diff changeset	72 # Maps bundle version with content opts to choose which part to bundle
74271829ddc0 clonebundles: move a bundle of clone bundle related code to a new module Pierre-Yves David <pierre-yves.david@octobus.net> parents: diff changeset	73 _bundlespeccontentopts = {
74271829ddc0 clonebundles: move a bundle of clone bundle related code to a new module Pierre-Yves David <pierre-yves.david@octobus.net> parents: diff changeset	74 b'v1': {
74271829ddc0 clonebundles: move a bundle of clone bundle related code to a new module Pierre-Yves David <pierre-yves.david@octobus.net> parents: diff changeset	75 b'changegroup': True,
74271829ddc0 clonebundles: move a bundle of clone bundle related code to a new module Pierre-Yves David <pierre-yves.david@octobus.net> parents: diff changeset	76 b'cg.version': b'01',
74271829ddc0 clonebundles: move a bundle of clone bundle related code to a new module Pierre-Yves David <pierre-yves.david@octobus.net> parents: diff changeset	77 b'obsolescence': False,
74271829ddc0 clonebundles: move a bundle of clone bundle related code to a new module Pierre-Yves David <pierre-yves.david@octobus.net> parents: diff changeset	78 b'phases': False,
74271829ddc0 clonebundles: move a bundle of clone bundle related code to a new module Pierre-Yves David <pierre-yves.david@octobus.net> parents: diff changeset	79 b'tagsfnodescache': False,
74271829ddc0 clonebundles: move a bundle of clone bundle related code to a new module Pierre-Yves David <pierre-yves.david@octobus.net> parents: diff changeset	80 b'revbranchcache': False,
74271829ddc0 clonebundles: move a bundle of clone bundle related code to a new module Pierre-Yves David <pierre-yves.david@octobus.net> parents: diff changeset	81 },
74271829ddc0 clonebundles: move a bundle of clone bundle related code to a new module Pierre-Yves David <pierre-yves.david@octobus.net> parents: diff changeset	82 b'v2': {
74271829ddc0 clonebundles: move a bundle of clone bundle related code to a new module Pierre-Yves David <pierre-yves.david@octobus.net> parents: diff changeset	83 b'changegroup': True,
74271829ddc0 clonebundles: move a bundle of clone bundle related code to a new module Pierre-Yves David <pierre-yves.david@octobus.net> parents: diff changeset	84 b'cg.version': b'02',
74271829ddc0 clonebundles: move a bundle of clone bundle related code to a new module Pierre-Yves David <pierre-yves.david@octobus.net> parents: diff changeset	85 b'obsolescence': False,
74271829ddc0 clonebundles: move a bundle of clone bundle related code to a new module Pierre-Yves David <pierre-yves.david@octobus.net> parents: diff changeset	86 b'phases': False,
74271829ddc0 clonebundles: move a bundle of clone bundle related code to a new module Pierre-Yves David <pierre-yves.david@octobus.net> parents: diff changeset	87 b'tagsfnodescache': True,
74271829ddc0 clonebundles: move a bundle of clone bundle related code to a new module Pierre-Yves David <pierre-yves.david@octobus.net> parents: diff changeset	88 b'revbranchcache': True,
74271829ddc0 clonebundles: move a bundle of clone bundle related code to a new module Pierre-Yves David <pierre-yves.david@octobus.net> parents: diff changeset	89 },
49322 d587f09cad98 bundlespec: make the `stream` case less special Pierre-Yves David <pierre-yves.david@octobus.net> parents: 49321 diff changeset	90 b'streamv2': {
d587f09cad98 bundlespec: make the `stream` case less special Pierre-Yves David <pierre-yves.david@octobus.net> parents: 49321 diff changeset	91 b'changegroup': False,
d587f09cad98 bundlespec: make the `stream` case less special Pierre-Yves David <pierre-yves.david@octobus.net> parents: 49321 diff changeset	92 b'cg.version': b'02',
d587f09cad98 bundlespec: make the `stream` case less special Pierre-Yves David <pierre-yves.david@octobus.net> parents: 49321 diff changeset	93 b'obsolescence': False,
d587f09cad98 bundlespec: make the `stream` case less special Pierre-Yves David <pierre-yves.david@octobus.net> parents: 49321 diff changeset	94 b'phases': False,
d587f09cad98 bundlespec: make the `stream` case less special Pierre-Yves David <pierre-yves.david@octobus.net> parents: 49321 diff changeset	95 b"streamv2": True,
d587f09cad98 bundlespec: make the `stream` case less special Pierre-Yves David <pierre-yves.david@octobus.net> parents: 49321 diff changeset	96 b'tagsfnodescache': False,
d587f09cad98 bundlespec: make the `stream` case less special Pierre-Yves David <pierre-yves.david@octobus.net> parents: 49321 diff changeset	97 b'revbranchcache': False,
d587f09cad98 bundlespec: make the `stream` case less special Pierre-Yves David <pierre-yves.david@octobus.net> parents: 49321 diff changeset	98 },
49321 c12c843f0db7 bundlespec: phase out the `_bundlespeccgversions` mapping Pierre-Yves David <pierre-yves.david@octobus.net> parents: 48946 diff changeset	99 b'packed1': {
c12c843f0db7 bundlespec: phase out the `_bundlespeccgversions` mapping Pierre-Yves David <pierre-yves.david@octobus.net> parents: 48946 diff changeset	100 b'cg.version': b's1',
c12c843f0db7 bundlespec: phase out the `_bundlespeccgversions` mapping Pierre-Yves David <pierre-yves.david@octobus.net> parents: 48946 diff changeset	101 },
c12c843f0db7 bundlespec: phase out the `_bundlespeccgversions` mapping Pierre-Yves David <pierre-yves.david@octobus.net> parents: 48946 diff changeset	102 b'bundle2': { # legacy
c12c843f0db7 bundlespec: phase out the `_bundlespeccgversions` mapping Pierre-Yves David <pierre-yves.david@octobus.net> parents: 48946 diff changeset	103 b'cg.version': b'02',
c12c843f0db7 bundlespec: phase out the `_bundlespeccgversions` mapping Pierre-Yves David <pierre-yves.david@octobus.net> parents: 48946 diff changeset	104 },
45784 74271829ddc0 clonebundles: move a bundle of clone bundle related code to a new module Pierre-Yves David <pierre-yves.david@octobus.net> parents: diff changeset	105 }
74271829ddc0 clonebundles: move a bundle of clone bundle related code to a new module Pierre-Yves David <pierre-yves.david@octobus.net> parents: diff changeset	106 _bundlespeccontentopts[b'bundle2'] = _bundlespeccontentopts[b'v2']
74271829ddc0 clonebundles: move a bundle of clone bundle related code to a new module Pierre-Yves David <pierre-yves.david@octobus.net> parents: diff changeset	107
49322 d587f09cad98 bundlespec: make the `stream` case less special Pierre-Yves David <pierre-yves.david@octobus.net> parents: 49321 diff changeset	108 _bundlespecvariants = {b"streamv2": {}}
45784 74271829ddc0 clonebundles: move a bundle of clone bundle related code to a new module Pierre-Yves David <pierre-yves.david@octobus.net> parents: diff changeset	109
74271829ddc0 clonebundles: move a bundle of clone bundle related code to a new module Pierre-Yves David <pierre-yves.david@octobus.net> parents: diff changeset	110 # Compression engines allowed in version 1. THIS SHOULD NEVER CHANGE.
74271829ddc0 clonebundles: move a bundle of clone bundle related code to a new module Pierre-Yves David <pierre-yves.david@octobus.net> parents: diff changeset	111 _bundlespecv1compengines = {b'gzip', b'bzip2', b'none'}
74271829ddc0 clonebundles: move a bundle of clone bundle related code to a new module Pierre-Yves David <pierre-yves.david@octobus.net> parents: diff changeset	112
74271829ddc0 clonebundles: move a bundle of clone bundle related code to a new module Pierre-Yves David <pierre-yves.david@octobus.net> parents: diff changeset	113
49331 1b04d5213d0f bundlespec: add processing of some parameter value Pierre-Yves David <pierre-yves.david@octobus.net> parents: 49330 diff changeset	114 def param_bool(key, value):
1b04d5213d0f bundlespec: add processing of some parameter value Pierre-Yves David <pierre-yves.david@octobus.net> parents: 49330 diff changeset	115 """make a boolean out of a parameter value"""
1b04d5213d0f bundlespec: add processing of some parameter value Pierre-Yves David <pierre-yves.david@octobus.net> parents: 49330 diff changeset	116 b = stringutil.parsebool(value)
1b04d5213d0f bundlespec: add processing of some parameter value Pierre-Yves David <pierre-yves.david@octobus.net> parents: 49330 diff changeset	117 if b is None:
1b04d5213d0f bundlespec: add processing of some parameter value Pierre-Yves David <pierre-yves.david@octobus.net> parents: 49330 diff changeset	118 msg = _(b"parameter %s should be a boolean ('%s')")
1b04d5213d0f bundlespec: add processing of some parameter value Pierre-Yves David <pierre-yves.david@octobus.net> parents: 49330 diff changeset	119 msg %= (key, value)
1b04d5213d0f bundlespec: add processing of some parameter value Pierre-Yves David <pierre-yves.david@octobus.net> parents: 49330 diff changeset	120 raise error.InvalidBundleSpecification(msg)
1b04d5213d0f bundlespec: add processing of some parameter value Pierre-Yves David <pierre-yves.david@octobus.net> parents: 49330 diff changeset	121 return b
1b04d5213d0f bundlespec: add processing of some parameter value Pierre-Yves David <pierre-yves.david@octobus.net> parents: 49330 diff changeset	122
1b04d5213d0f bundlespec: add processing of some parameter value Pierre-Yves David <pierre-yves.david@octobus.net> parents: 49330 diff changeset	123
1b04d5213d0f bundlespec: add processing of some parameter value Pierre-Yves David <pierre-yves.david@octobus.net> parents: 49330 diff changeset	124 # mapping of known parameter name need their value processed
1b04d5213d0f bundlespec: add processing of some parameter value Pierre-Yves David <pierre-yves.david@octobus.net> parents: 49330 diff changeset	125 bundle_spec_param_processing = {
1b04d5213d0f bundlespec: add processing of some parameter value Pierre-Yves David <pierre-yves.david@octobus.net> parents: 49330 diff changeset	126 b"obsolescence": param_bool,
1b04d5213d0f bundlespec: add processing of some parameter value Pierre-Yves David <pierre-yves.david@octobus.net> parents: 49330 diff changeset	127 b"obsolescence-mandatory": param_bool,
1b04d5213d0f bundlespec: add processing of some parameter value Pierre-Yves David <pierre-yves.david@octobus.net> parents: 49330 diff changeset	128 b"phases": param_bool,
1b04d5213d0f bundlespec: add processing of some parameter value Pierre-Yves David <pierre-yves.david@octobus.net> parents: 49330 diff changeset	129 }
1b04d5213d0f bundlespec: add processing of some parameter value Pierre-Yves David <pierre-yves.david@octobus.net> parents: 49330 diff changeset	130
1b04d5213d0f bundlespec: add processing of some parameter value Pierre-Yves David <pierre-yves.david@octobus.net> parents: 49330 diff changeset	131
49330 5d17dd74177d bundlespec: extract the parseparams closure Pierre-Yves David <pierre-yves.david@octobus.net> parents: 49326 diff changeset	132 def _parseparams(s):
5d17dd74177d bundlespec: extract the parseparams closure Pierre-Yves David <pierre-yves.david@octobus.net> parents: 49326 diff changeset	133 """parse bundlespec parameter section
5d17dd74177d bundlespec: extract the parseparams closure Pierre-Yves David <pierre-yves.david@octobus.net> parents: 49326 diff changeset	134
5d17dd74177d bundlespec: extract the parseparams closure Pierre-Yves David <pierre-yves.david@octobus.net> parents: 49326 diff changeset	135 input: "comp-version;params" string
5d17dd74177d bundlespec: extract the parseparams closure Pierre-Yves David <pierre-yves.david@octobus.net> parents: 49326 diff changeset	136
5d17dd74177d bundlespec: extract the parseparams closure Pierre-Yves David <pierre-yves.david@octobus.net> parents: 49326 diff changeset	137 return: (spec; {param_key: param_value})
5d17dd74177d bundlespec: extract the parseparams closure Pierre-Yves David <pierre-yves.david@octobus.net> parents: 49326 diff changeset	138 """
5d17dd74177d bundlespec: extract the parseparams closure Pierre-Yves David <pierre-yves.david@octobus.net> parents: 49326 diff changeset	139 if b';' not in s:
5d17dd74177d bundlespec: extract the parseparams closure Pierre-Yves David <pierre-yves.david@octobus.net> parents: 49326 diff changeset	140 return s, {}
5d17dd74177d bundlespec: extract the parseparams closure Pierre-Yves David <pierre-yves.david@octobus.net> parents: 49326 diff changeset	141
5d17dd74177d bundlespec: extract the parseparams closure Pierre-Yves David <pierre-yves.david@octobus.net> parents: 49326 diff changeset	142 params = {}
5d17dd74177d bundlespec: extract the parseparams closure Pierre-Yves David <pierre-yves.david@octobus.net> parents: 49326 diff changeset	143 version, paramstr = s.split(b';', 1)
5d17dd74177d bundlespec: extract the parseparams closure Pierre-Yves David <pierre-yves.david@octobus.net> parents: 49326 diff changeset	144
5d17dd74177d bundlespec: extract the parseparams closure Pierre-Yves David <pierre-yves.david@octobus.net> parents: 49326 diff changeset	145 err = _(b'invalid bundle specification: missing "=" in parameter: %s')
5d17dd74177d bundlespec: extract the parseparams closure Pierre-Yves David <pierre-yves.david@octobus.net> parents: 49326 diff changeset	146 for p in paramstr.split(b';'):
5d17dd74177d bundlespec: extract the parseparams closure Pierre-Yves David <pierre-yves.david@octobus.net> parents: 49326 diff changeset	147 if b'=' not in p:
5d17dd74177d bundlespec: extract the parseparams closure Pierre-Yves David <pierre-yves.david@octobus.net> parents: 49326 diff changeset	148 msg = err % p
5d17dd74177d bundlespec: extract the parseparams closure Pierre-Yves David <pierre-yves.david@octobus.net> parents: 49326 diff changeset	149 raise error.InvalidBundleSpecification(msg)
5d17dd74177d bundlespec: extract the parseparams closure Pierre-Yves David <pierre-yves.david@octobus.net> parents: 49326 diff changeset	150
5d17dd74177d bundlespec: extract the parseparams closure Pierre-Yves David <pierre-yves.david@octobus.net> parents: 49326 diff changeset	151 key, value = p.split(b'=', 1)
5d17dd74177d bundlespec: extract the parseparams closure Pierre-Yves David <pierre-yves.david@octobus.net> parents: 49326 diff changeset	152 key = urlreq.unquote(key)
5d17dd74177d bundlespec: extract the parseparams closure Pierre-Yves David <pierre-yves.david@octobus.net> parents: 49326 diff changeset	153 value = urlreq.unquote(value)
49331 1b04d5213d0f bundlespec: add processing of some parameter value Pierre-Yves David <pierre-yves.david@octobus.net> parents: 49330 diff changeset	154 process = bundle_spec_param_processing.get(key)
1b04d5213d0f bundlespec: add processing of some parameter value Pierre-Yves David <pierre-yves.david@octobus.net> parents: 49330 diff changeset	155 if process is not None:
1b04d5213d0f bundlespec: add processing of some parameter value Pierre-Yves David <pierre-yves.david@octobus.net> parents: 49330 diff changeset	156 value = process(key, value)
49330 5d17dd74177d bundlespec: extract the parseparams closure Pierre-Yves David <pierre-yves.david@octobus.net> parents: 49326 diff changeset	157 params[key] = value
5d17dd74177d bundlespec: extract the parseparams closure Pierre-Yves David <pierre-yves.david@octobus.net> parents: 49326 diff changeset	158
5d17dd74177d bundlespec: extract the parseparams closure Pierre-Yves David <pierre-yves.david@octobus.net> parents: 49326 diff changeset	159 return version, params
5d17dd74177d bundlespec: extract the parseparams closure Pierre-Yves David <pierre-yves.david@octobus.net> parents: 49326 diff changeset	160
5d17dd74177d bundlespec: extract the parseparams closure Pierre-Yves David <pierre-yves.david@octobus.net> parents: 49326 diff changeset	161
45784 74271829ddc0 clonebundles: move a bundle of clone bundle related code to a new module Pierre-Yves David <pierre-yves.david@octobus.net> parents: diff changeset	162 def parsebundlespec(repo, spec, strict=True):
74271829ddc0 clonebundles: move a bundle of clone bundle related code to a new module Pierre-Yves David <pierre-yves.david@octobus.net> parents: diff changeset	163 """Parse a bundle string specification into parts.
74271829ddc0 clonebundles: move a bundle of clone bundle related code to a new module Pierre-Yves David <pierre-yves.david@octobus.net> parents: diff changeset	164
74271829ddc0 clonebundles: move a bundle of clone bundle related code to a new module Pierre-Yves David <pierre-yves.david@octobus.net> parents: diff changeset	165 Bundle specifications denote a well-defined bundle/exchange format.
74271829ddc0 clonebundles: move a bundle of clone bundle related code to a new module Pierre-Yves David <pierre-yves.david@octobus.net> parents: diff changeset	166 The content of a given specification should not change over time in
74271829ddc0 clonebundles: move a bundle of clone bundle related code to a new module Pierre-Yves David <pierre-yves.david@octobus.net> parents: diff changeset	167 order to ensure that bundles produced by a newer version of Mercurial are
74271829ddc0 clonebundles: move a bundle of clone bundle related code to a new module Pierre-Yves David <pierre-yves.david@octobus.net> parents: diff changeset	168 readable from an older version.
74271829ddc0 clonebundles: move a bundle of clone bundle related code to a new module Pierre-Yves David <pierre-yves.david@octobus.net> parents: diff changeset	169
74271829ddc0 clonebundles: move a bundle of clone bundle related code to a new module Pierre-Yves David <pierre-yves.david@octobus.net> parents: diff changeset	170 The string currently has the form:
74271829ddc0 clonebundles: move a bundle of clone bundle related code to a new module Pierre-Yves David <pierre-yves.david@octobus.net> parents: diff changeset	171
74271829ddc0 clonebundles: move a bundle of clone bundle related code to a new module Pierre-Yves David <pierre-yves.david@octobus.net> parents: diff changeset	172 <compression>-<type>[;<parameter0>[;<parameter1>]]
74271829ddc0 clonebundles: move a bundle of clone bundle related code to a new module Pierre-Yves David <pierre-yves.david@octobus.net> parents: diff changeset	173
74271829ddc0 clonebundles: move a bundle of clone bundle related code to a new module Pierre-Yves David <pierre-yves.david@octobus.net> parents: diff changeset	174 Where <compression> is one of the supported compression formats
74271829ddc0 clonebundles: move a bundle of clone bundle related code to a new module Pierre-Yves David <pierre-yves.david@octobus.net> parents: diff changeset	175 and <type> is (currently) a version string. A ";" can follow the type and
74271829ddc0 clonebundles: move a bundle of clone bundle related code to a new module Pierre-Yves David <pierre-yves.david@octobus.net> parents: diff changeset	176 all text afterwards is interpreted as URI encoded, ";" delimited key=value
74271829ddc0 clonebundles: move a bundle of clone bundle related code to a new module Pierre-Yves David <pierre-yves.david@octobus.net> parents: diff changeset	177 pairs.
74271829ddc0 clonebundles: move a bundle of clone bundle related code to a new module Pierre-Yves David <pierre-yves.david@octobus.net> parents: diff changeset	178
74271829ddc0 clonebundles: move a bundle of clone bundle related code to a new module Pierre-Yves David <pierre-yves.david@octobus.net> parents: diff changeset	179 If ``strict`` is True (the default) <compression> is required. Otherwise,
74271829ddc0 clonebundles: move a bundle of clone bundle related code to a new module Pierre-Yves David <pierre-yves.david@octobus.net> parents: diff changeset	180 it is optional.
74271829ddc0 clonebundles: move a bundle of clone bundle related code to a new module Pierre-Yves David <pierre-yves.david@octobus.net> parents: diff changeset	181
74271829ddc0 clonebundles: move a bundle of clone bundle related code to a new module Pierre-Yves David <pierre-yves.david@octobus.net> parents: diff changeset	182 Returns a bundlespec object of (compression, version, parameters).
74271829ddc0 clonebundles: move a bundle of clone bundle related code to a new module Pierre-Yves David <pierre-yves.david@octobus.net> parents: diff changeset	183 Compression will be ``None`` if not in strict mode and a compression isn't
74271829ddc0 clonebundles: move a bundle of clone bundle related code to a new module Pierre-Yves David <pierre-yves.david@octobus.net> parents: diff changeset	184 defined.
74271829ddc0 clonebundles: move a bundle of clone bundle related code to a new module Pierre-Yves David <pierre-yves.david@octobus.net> parents: diff changeset	185
74271829ddc0 clonebundles: move a bundle of clone bundle related code to a new module Pierre-Yves David <pierre-yves.david@octobus.net> parents: diff changeset	186 An ``InvalidBundleSpecification`` is raised when the specification is
74271829ddc0 clonebundles: move a bundle of clone bundle related code to a new module Pierre-Yves David <pierre-yves.david@octobus.net> parents: diff changeset	187 not syntactically well formed.
74271829ddc0 clonebundles: move a bundle of clone bundle related code to a new module Pierre-Yves David <pierre-yves.david@octobus.net> parents: diff changeset	188
74271829ddc0 clonebundles: move a bundle of clone bundle related code to a new module Pierre-Yves David <pierre-yves.david@octobus.net> parents: diff changeset	189 An ``UnsupportedBundleSpecification`` is raised when the compression or
74271829ddc0 clonebundles: move a bundle of clone bundle related code to a new module Pierre-Yves David <pierre-yves.david@octobus.net> parents: diff changeset	190 bundle type/version is not recognized.
74271829ddc0 clonebundles: move a bundle of clone bundle related code to a new module Pierre-Yves David <pierre-yves.david@octobus.net> parents: diff changeset	191
74271829ddc0 clonebundles: move a bundle of clone bundle related code to a new module Pierre-Yves David <pierre-yves.david@octobus.net> parents: diff changeset	192 Note: this function will likely eventually return a more complex data
74271829ddc0 clonebundles: move a bundle of clone bundle related code to a new module Pierre-Yves David <pierre-yves.david@octobus.net> parents: diff changeset	193 structure, including bundle2 part information.
74271829ddc0 clonebundles: move a bundle of clone bundle related code to a new module Pierre-Yves David <pierre-yves.david@octobus.net> parents: diff changeset	194 """
74271829ddc0 clonebundles: move a bundle of clone bundle related code to a new module Pierre-Yves David <pierre-yves.david@octobus.net> parents: diff changeset	195 if strict and b'-' not in spec:
74271829ddc0 clonebundles: move a bundle of clone bundle related code to a new module Pierre-Yves David <pierre-yves.david@octobus.net> parents: diff changeset	196 raise error.InvalidBundleSpecification(
74271829ddc0 clonebundles: move a bundle of clone bundle related code to a new module Pierre-Yves David <pierre-yves.david@octobus.net> parents: diff changeset	197 _(
74271829ddc0 clonebundles: move a bundle of clone bundle related code to a new module Pierre-Yves David <pierre-yves.david@octobus.net> parents: diff changeset	198 b'invalid bundle specification; '
74271829ddc0 clonebundles: move a bundle of clone bundle related code to a new module Pierre-Yves David <pierre-yves.david@octobus.net> parents: diff changeset	199 b'must be prefixed with compression: %s'
74271829ddc0 clonebundles: move a bundle of clone bundle related code to a new module Pierre-Yves David <pierre-yves.david@octobus.net> parents: diff changeset	200 )
74271829ddc0 clonebundles: move a bundle of clone bundle related code to a new module Pierre-Yves David <pierre-yves.david@octobus.net> parents: diff changeset	201 % spec
74271829ddc0 clonebundles: move a bundle of clone bundle related code to a new module Pierre-Yves David <pierre-yves.david@octobus.net> parents: diff changeset	202 )
74271829ddc0 clonebundles: move a bundle of clone bundle related code to a new module Pierre-Yves David <pierre-yves.david@octobus.net> parents: diff changeset	203
49332 d89bfc075289 bundlespec: do not check for `-` in the params portion of the bundlespec Pierre-Yves David <pierre-yves.david@octobus.net> parents: 49331 diff changeset	204 pre_args = spec.split(b';', 1)[0]
d89bfc075289 bundlespec: do not check for `-` in the params portion of the bundlespec Pierre-Yves David <pierre-yves.david@octobus.net> parents: 49331 diff changeset	205 if b'-' in pre_args:
45784 74271829ddc0 clonebundles: move a bundle of clone bundle related code to a new module Pierre-Yves David <pierre-yves.david@octobus.net> parents: diff changeset	206 compression, version = spec.split(b'-', 1)
74271829ddc0 clonebundles: move a bundle of clone bundle related code to a new module Pierre-Yves David <pierre-yves.david@octobus.net> parents: diff changeset	207
74271829ddc0 clonebundles: move a bundle of clone bundle related code to a new module Pierre-Yves David <pierre-yves.david@octobus.net> parents: diff changeset	208 if compression not in util.compengines.supportedbundlenames:
74271829ddc0 clonebundles: move a bundle of clone bundle related code to a new module Pierre-Yves David <pierre-yves.david@octobus.net> parents: diff changeset	209 raise error.UnsupportedBundleSpecification(
74271829ddc0 clonebundles: move a bundle of clone bundle related code to a new module Pierre-Yves David <pierre-yves.david@octobus.net> parents: diff changeset	210 _(b'%s compression is not supported') % compression
74271829ddc0 clonebundles: move a bundle of clone bundle related code to a new module Pierre-Yves David <pierre-yves.david@octobus.net> parents: diff changeset	211 )
74271829ddc0 clonebundles: move a bundle of clone bundle related code to a new module Pierre-Yves David <pierre-yves.david@octobus.net> parents: diff changeset	212
49330 5d17dd74177d bundlespec: extract the parseparams closure Pierre-Yves David <pierre-yves.david@octobus.net> parents: 49326 diff changeset	213 version, params = _parseparams(version)
45784 74271829ddc0 clonebundles: move a bundle of clone bundle related code to a new module Pierre-Yves David <pierre-yves.david@octobus.net> parents: diff changeset	214
49321 c12c843f0db7 bundlespec: phase out the `_bundlespeccgversions` mapping Pierre-Yves David <pierre-yves.david@octobus.net> parents: 48946 diff changeset	215 if version not in _bundlespeccontentopts:
45784 74271829ddc0 clonebundles: move a bundle of clone bundle related code to a new module Pierre-Yves David <pierre-yves.david@octobus.net> parents: diff changeset	216 raise error.UnsupportedBundleSpecification(
74271829ddc0 clonebundles: move a bundle of clone bundle related code to a new module Pierre-Yves David <pierre-yves.david@octobus.net> parents: diff changeset	217 _(b'%s is not a recognized bundle version') % version
74271829ddc0 clonebundles: move a bundle of clone bundle related code to a new module Pierre-Yves David <pierre-yves.david@octobus.net> parents: diff changeset	218 )
74271829ddc0 clonebundles: move a bundle of clone bundle related code to a new module Pierre-Yves David <pierre-yves.david@octobus.net> parents: diff changeset	219 else:
74271829ddc0 clonebundles: move a bundle of clone bundle related code to a new module Pierre-Yves David <pierre-yves.david@octobus.net> parents: diff changeset	220 # Value could be just the compression or just the version, in which
74271829ddc0 clonebundles: move a bundle of clone bundle related code to a new module Pierre-Yves David <pierre-yves.david@octobus.net> parents: diff changeset	221 # case some defaults are assumed (but only when not in strict mode).
74271829ddc0 clonebundles: move a bundle of clone bundle related code to a new module Pierre-Yves David <pierre-yves.david@octobus.net> parents: diff changeset	222 assert not strict
74271829ddc0 clonebundles: move a bundle of clone bundle related code to a new module Pierre-Yves David <pierre-yves.david@octobus.net> parents: diff changeset	223
49330 5d17dd74177d bundlespec: extract the parseparams closure Pierre-Yves David <pierre-yves.david@octobus.net> parents: 49326 diff changeset	224 spec, params = _parseparams(spec)
45784 74271829ddc0 clonebundles: move a bundle of clone bundle related code to a new module Pierre-Yves David <pierre-yves.david@octobus.net> parents: diff changeset	225
74271829ddc0 clonebundles: move a bundle of clone bundle related code to a new module Pierre-Yves David <pierre-yves.david@octobus.net> parents: diff changeset	226 if spec in util.compengines.supportedbundlenames:
74271829ddc0 clonebundles: move a bundle of clone bundle related code to a new module Pierre-Yves David <pierre-yves.david@octobus.net> parents: diff changeset	227 compression = spec
74271829ddc0 clonebundles: move a bundle of clone bundle related code to a new module Pierre-Yves David <pierre-yves.david@octobus.net> parents: diff changeset	228 version = b'v1'
74271829ddc0 clonebundles: move a bundle of clone bundle related code to a new module Pierre-Yves David <pierre-yves.david@octobus.net> parents: diff changeset	229 # Generaldelta repos require v2.
46627 f4c325bf80fc requirements: also add a generaldelta constant Rapha?l Gom?s <rgomes@octobus.net> parents: 45785 diff changeset	230 if requirementsmod.GENERALDELTA_REQUIREMENT in repo.requirements:
45784 74271829ddc0 clonebundles: move a bundle of clone bundle related code to a new module Pierre-Yves David <pierre-yves.david@octobus.net> parents: diff changeset	231 version = b'v2'
47342 dc3800569157 revlogv2: make sure bundling pick a compatible bundle format Pierre-Yves David <pierre-yves.david@octobus.net> parents: 46627 diff changeset	232 elif requirementsmod.REVLOGV2_REQUIREMENT in repo.requirements:
dc3800569157 revlogv2: make sure bundling pick a compatible bundle format Pierre-Yves David <pierre-yves.david@octobus.net> parents: 46627 diff changeset	233 version = b'v2'
45784 74271829ddc0 clonebundles: move a bundle of clone bundle related code to a new module Pierre-Yves David <pierre-yves.david@octobus.net> parents: diff changeset	234 # Modern compression engines require v2.
74271829ddc0 clonebundles: move a bundle of clone bundle related code to a new module Pierre-Yves David <pierre-yves.david@octobus.net> parents: diff changeset	235 if compression not in _bundlespecv1compengines:
74271829ddc0 clonebundles: move a bundle of clone bundle related code to a new module Pierre-Yves David <pierre-yves.david@octobus.net> parents: diff changeset	236 version = b'v2'
49321 c12c843f0db7 bundlespec: phase out the `_bundlespeccgversions` mapping Pierre-Yves David <pierre-yves.david@octobus.net> parents: 48946 diff changeset	237 elif spec in _bundlespeccontentopts:
45784 74271829ddc0 clonebundles: move a bundle of clone bundle related code to a new module Pierre-Yves David <pierre-yves.david@octobus.net> parents: diff changeset	238 if spec == b'packed1':
74271829ddc0 clonebundles: move a bundle of clone bundle related code to a new module Pierre-Yves David <pierre-yves.david@octobus.net> parents: diff changeset	239 compression = b'none'
74271829ddc0 clonebundles: move a bundle of clone bundle related code to a new module Pierre-Yves David <pierre-yves.david@octobus.net> parents: diff changeset	240 else:
74271829ddc0 clonebundles: move a bundle of clone bundle related code to a new module Pierre-Yves David <pierre-yves.david@octobus.net> parents: diff changeset	241 compression = b'bzip2'
74271829ddc0 clonebundles: move a bundle of clone bundle related code to a new module Pierre-Yves David <pierre-yves.david@octobus.net> parents: diff changeset	242 version = spec
74271829ddc0 clonebundles: move a bundle of clone bundle related code to a new module Pierre-Yves David <pierre-yves.david@octobus.net> parents: diff changeset	243 else:
74271829ddc0 clonebundles: move a bundle of clone bundle related code to a new module Pierre-Yves David <pierre-yves.david@octobus.net> parents: diff changeset	244 raise error.UnsupportedBundleSpecification(
74271829ddc0 clonebundles: move a bundle of clone bundle related code to a new module Pierre-Yves David <pierre-yves.david@octobus.net> parents: diff changeset	245 _(b'%s is not a recognized bundle specification') % spec
74271829ddc0 clonebundles: move a bundle of clone bundle related code to a new module Pierre-Yves David <pierre-yves.david@octobus.net> parents: diff changeset	246 )
74271829ddc0 clonebundles: move a bundle of clone bundle related code to a new module Pierre-Yves David <pierre-yves.david@octobus.net> parents: diff changeset	247
74271829ddc0 clonebundles: move a bundle of clone bundle related code to a new module Pierre-Yves David <pierre-yves.david@octobus.net> parents: diff changeset	248 # Bundle version 1 only supports a known set of compression engines.
74271829ddc0 clonebundles: move a bundle of clone bundle related code to a new module Pierre-Yves David <pierre-yves.david@octobus.net> parents: diff changeset	249 if version == b'v1' and compression not in _bundlespecv1compengines:
74271829ddc0 clonebundles: move a bundle of clone bundle related code to a new module Pierre-Yves David <pierre-yves.david@octobus.net> parents: diff changeset	250 raise error.UnsupportedBundleSpecification(
74271829ddc0 clonebundles: move a bundle of clone bundle related code to a new module Pierre-Yves David <pierre-yves.david@octobus.net> parents: diff changeset	251 _(b'compression engine %s is not supported on v1 bundles')
74271829ddc0 clonebundles: move a bundle of clone bundle related code to a new module Pierre-Yves David <pierre-yves.david@octobus.net> parents: diff changeset	252 % compression
74271829ddc0 clonebundles: move a bundle of clone bundle related code to a new module Pierre-Yves David <pierre-yves.david@octobus.net> parents: diff changeset	253 )
74271829ddc0 clonebundles: move a bundle of clone bundle related code to a new module Pierre-Yves David <pierre-yves.david@octobus.net> parents: diff changeset	254
74271829ddc0 clonebundles: move a bundle of clone bundle related code to a new module Pierre-Yves David <pierre-yves.david@octobus.net> parents: diff changeset	255 # The specification for packed1 can optionally declare the data formats
74271829ddc0 clonebundles: move a bundle of clone bundle related code to a new module Pierre-Yves David <pierre-yves.david@octobus.net> parents: diff changeset	256 # required to apply it. If we see this metadata, compare against what the
74271829ddc0 clonebundles: move a bundle of clone bundle related code to a new module Pierre-Yves David <pierre-yves.david@octobus.net> parents: diff changeset	257 # repo supports and error if the bundle isn't compatible.
74271829ddc0 clonebundles: move a bundle of clone bundle related code to a new module Pierre-Yves David <pierre-yves.david@octobus.net> parents: diff changeset	258 if version == b'packed1' and b'requirements' in params:
49648 9be765b82a90 typing: minor tweaks to allow updating to pytype 2022.11.18 Matt Harbison <matt_harbison@yahoo.com> parents: 49332 diff changeset	259 requirements = set(cast(bytes, params[b'requirements']).split(b','))
48601 baddab229b86 stream-clone: add a explicit set list requirements relevant to stream clone Pierre-Yves David <pierre-yves.david@octobus.net> parents: 47342 diff changeset	260 missingreqs = requirements - requirementsmod.STREAM_FIXED_REQUIREMENTS
45784 74271829ddc0 clonebundles: move a bundle of clone bundle related code to a new module Pierre-Yves David <pierre-yves.david@octobus.net> parents: diff changeset	261 if missingreqs:
74271829ddc0 clonebundles: move a bundle of clone bundle related code to a new module Pierre-Yves David <pierre-yves.david@octobus.net> parents: diff changeset	262 raise error.UnsupportedBundleSpecification(
74271829ddc0 clonebundles: move a bundle of clone bundle related code to a new module Pierre-Yves David <pierre-yves.david@octobus.net> parents: diff changeset	263 _(b'missing support for repository features: %s')
74271829ddc0 clonebundles: move a bundle of clone bundle related code to a new module Pierre-Yves David <pierre-yves.david@octobus.net> parents: diff changeset	264 % b', '.join(sorted(missingreqs))
74271829ddc0 clonebundles: move a bundle of clone bundle related code to a new module Pierre-Yves David <pierre-yves.david@octobus.net> parents: diff changeset	265 )
74271829ddc0 clonebundles: move a bundle of clone bundle related code to a new module Pierre-Yves David <pierre-yves.david@octobus.net> parents: diff changeset	266
74271829ddc0 clonebundles: move a bundle of clone bundle related code to a new module Pierre-Yves David <pierre-yves.david@octobus.net> parents: diff changeset	267 # Compute contentopts based on the version
49322 d587f09cad98 bundlespec: make the `stream` case less special Pierre-Yves David <pierre-yves.david@octobus.net> parents: 49321 diff changeset	268 if b"stream" in params and params[b"stream"] == b"v2":
d587f09cad98 bundlespec: make the `stream` case less special Pierre-Yves David <pierre-yves.david@octobus.net> parents: 49321 diff changeset	269 # That case is fishy as this mostly derails the version selection
d587f09cad98 bundlespec: make the `stream` case less special Pierre-Yves David <pierre-yves.david@octobus.net> parents: 49321 diff changeset	270 # mechanism. `stream` bundles are quite specific and used differently
d587f09cad98 bundlespec: make the `stream` case less special Pierre-Yves David <pierre-yves.david@octobus.net> parents: 49321 diff changeset	271 # as "normal" bundles.
d587f09cad98 bundlespec: make the `stream` case less special Pierre-Yves David <pierre-yves.david@octobus.net> parents: 49321 diff changeset	272 #
d587f09cad98 bundlespec: make the `stream` case less special Pierre-Yves David <pierre-yves.david@octobus.net> parents: 49321 diff changeset	273 # So we are pinning this to "v2", as this will likely be
d587f09cad98 bundlespec: make the `stream` case less special Pierre-Yves David <pierre-yves.david@octobus.net> parents: 49321 diff changeset	274 # compatible forever. (see the next conditional).
d587f09cad98 bundlespec: make the `stream` case less special Pierre-Yves David <pierre-yves.david@octobus.net> parents: 49321 diff changeset	275 #
d587f09cad98 bundlespec: make the `stream` case less special Pierre-Yves David <pierre-yves.david@octobus.net> parents: 49321 diff changeset	276 # (we should probably define a cleaner way to do this and raise a
d587f09cad98 bundlespec: make the `stream` case less special Pierre-Yves David <pierre-yves.david@octobus.net> parents: 49321 diff changeset	277 # warning when the old way is encounter)
d587f09cad98 bundlespec: make the `stream` case less special Pierre-Yves David <pierre-yves.david@octobus.net> parents: 49321 diff changeset	278 version = b"streamv2"
45784 74271829ddc0 clonebundles: move a bundle of clone bundle related code to a new module Pierre-Yves David <pierre-yves.david@octobus.net> parents: diff changeset	279 contentopts = _bundlespeccontentopts.get(version, {}).copy()
49322 d587f09cad98 bundlespec: make the `stream` case less special Pierre-Yves David <pierre-yves.david@octobus.net> parents: 49321 diff changeset	280 if version == b"streamv2":
d587f09cad98 bundlespec: make the `stream` case less special Pierre-Yves David <pierre-yves.david@octobus.net> parents: 49321 diff changeset	281 # streamv2 have been reported as "v2" for a while.
d587f09cad98 bundlespec: make the `stream` case less special Pierre-Yves David <pierre-yves.david@octobus.net> parents: 49321 diff changeset	282 version = b"v2"
45784 74271829ddc0 clonebundles: move a bundle of clone bundle related code to a new module Pierre-Yves David <pierre-yves.david@octobus.net> parents: diff changeset	283
74271829ddc0 clonebundles: move a bundle of clone bundle related code to a new module Pierre-Yves David <pierre-yves.david@octobus.net> parents: diff changeset	284 engine = util.compengines.forbundlename(compression)
74271829ddc0 clonebundles: move a bundle of clone bundle related code to a new module Pierre-Yves David <pierre-yves.david@octobus.net> parents: diff changeset	285 compression, wirecompression = engine.bundletype()
49321 c12c843f0db7 bundlespec: phase out the `_bundlespeccgversions` mapping Pierre-Yves David <pierre-yves.david@octobus.net> parents: 48946 diff changeset	286 wireversion = _bundlespeccontentopts[version][b'cg.version']
45784 74271829ddc0 clonebundles: move a bundle of clone bundle related code to a new module Pierre-Yves David <pierre-yves.david@octobus.net> parents: diff changeset	287
74271829ddc0 clonebundles: move a bundle of clone bundle related code to a new module Pierre-Yves David <pierre-yves.david@octobus.net> parents: diff changeset	288 return bundlespec(
74271829ddc0 clonebundles: move a bundle of clone bundle related code to a new module Pierre-Yves David <pierre-yves.david@octobus.net> parents: diff changeset	289 compression, wirecompression, version, wireversion, params, contentopts
74271829ddc0 clonebundles: move a bundle of clone bundle related code to a new module Pierre-Yves David <pierre-yves.david@octobus.net> parents: diff changeset	290 )
74271829ddc0 clonebundles: move a bundle of clone bundle related code to a new module Pierre-Yves David <pierre-yves.david@octobus.net> parents: diff changeset	291
74271829ddc0 clonebundles: move a bundle of clone bundle related code to a new module Pierre-Yves David <pierre-yves.david@octobus.net> parents: diff changeset	292
74271829ddc0 clonebundles: move a bundle of clone bundle related code to a new module Pierre-Yves David <pierre-yves.david@octobus.net> parents: diff changeset	293 def parseclonebundlesmanifest(repo, s):
74271829ddc0 clonebundles: move a bundle of clone bundle related code to a new module Pierre-Yves David <pierre-yves.david@octobus.net> parents: diff changeset	294 """Parses the raw text of a clone bundles manifest.
74271829ddc0 clonebundles: move a bundle of clone bundle related code to a new module Pierre-Yves David <pierre-yves.david@octobus.net> parents: diff changeset	295
74271829ddc0 clonebundles: move a bundle of clone bundle related code to a new module Pierre-Yves David <pierre-yves.david@octobus.net> parents: diff changeset	296 Returns a list of dicts. The dicts have a ``URL`` key corresponding
74271829ddc0 clonebundles: move a bundle of clone bundle related code to a new module Pierre-Yves David <pierre-yves.david@octobus.net> parents: diff changeset	297 to the URL and other keys are the attributes for the entry.
74271829ddc0 clonebundles: move a bundle of clone bundle related code to a new module Pierre-Yves David <pierre-yves.david@octobus.net> parents: diff changeset	298 """
74271829ddc0 clonebundles: move a bundle of clone bundle related code to a new module Pierre-Yves David <pierre-yves.david@octobus.net> parents: diff changeset	299 m = []
74271829ddc0 clonebundles: move a bundle of clone bundle related code to a new module Pierre-Yves David <pierre-yves.david@octobus.net> parents: diff changeset	300 for line in s.splitlines():
74271829ddc0 clonebundles: move a bundle of clone bundle related code to a new module Pierre-Yves David <pierre-yves.david@octobus.net> parents: diff changeset	301 fields = line.split()
74271829ddc0 clonebundles: move a bundle of clone bundle related code to a new module Pierre-Yves David <pierre-yves.david@octobus.net> parents: diff changeset	302 if not fields:
74271829ddc0 clonebundles: move a bundle of clone bundle related code to a new module Pierre-Yves David <pierre-yves.david@octobus.net> parents: diff changeset	303 continue
74271829ddc0 clonebundles: move a bundle of clone bundle related code to a new module Pierre-Yves David <pierre-yves.david@octobus.net> parents: diff changeset	304 attrs = {b'URL': fields[0]}
74271829ddc0 clonebundles: move a bundle of clone bundle related code to a new module Pierre-Yves David <pierre-yves.david@octobus.net> parents: diff changeset	305 for rawattr in fields[1:]:
74271829ddc0 clonebundles: move a bundle of clone bundle related code to a new module Pierre-Yves David <pierre-yves.david@octobus.net> parents: diff changeset	306 key, value = rawattr.split(b'=', 1)
74271829ddc0 clonebundles: move a bundle of clone bundle related code to a new module Pierre-Yves David <pierre-yves.david@octobus.net> parents: diff changeset	307 key = util.urlreq.unquote(key)
74271829ddc0 clonebundles: move a bundle of clone bundle related code to a new module Pierre-Yves David <pierre-yves.david@octobus.net> parents: diff changeset	308 value = util.urlreq.unquote(value)
74271829ddc0 clonebundles: move a bundle of clone bundle related code to a new module Pierre-Yves David <pierre-yves.david@octobus.net> parents: diff changeset	309 attrs[key] = value
74271829ddc0 clonebundles: move a bundle of clone bundle related code to a new module Pierre-Yves David <pierre-yves.david@octobus.net> parents: diff changeset	310
74271829ddc0 clonebundles: move a bundle of clone bundle related code to a new module Pierre-Yves David <pierre-yves.david@octobus.net> parents: diff changeset	311 # Parse BUNDLESPEC into components. This makes client-side
74271829ddc0 clonebundles: move a bundle of clone bundle related code to a new module Pierre-Yves David <pierre-yves.david@octobus.net> parents: diff changeset	312 # preferences easier to specify since you can prefer a single
74271829ddc0 clonebundles: move a bundle of clone bundle related code to a new module Pierre-Yves David <pierre-yves.david@octobus.net> parents: diff changeset	313 # component of the BUNDLESPEC.
74271829ddc0 clonebundles: move a bundle of clone bundle related code to a new module Pierre-Yves David <pierre-yves.david@octobus.net> parents: diff changeset	314 if key == b'BUNDLESPEC':
74271829ddc0 clonebundles: move a bundle of clone bundle related code to a new module Pierre-Yves David <pierre-yves.david@octobus.net> parents: diff changeset	315 try:
74271829ddc0 clonebundles: move a bundle of clone bundle related code to a new module Pierre-Yves David <pierre-yves.david@octobus.net> parents: diff changeset	316 bundlespec = parsebundlespec(repo, value)
74271829ddc0 clonebundles: move a bundle of clone bundle related code to a new module Pierre-Yves David <pierre-yves.david@octobus.net> parents: diff changeset	317 attrs[b'COMPRESSION'] = bundlespec.compression
74271829ddc0 clonebundles: move a bundle of clone bundle related code to a new module Pierre-Yves David <pierre-yves.david@octobus.net> parents: diff changeset	318 attrs[b'VERSION'] = bundlespec.version
74271829ddc0 clonebundles: move a bundle of clone bundle related code to a new module Pierre-Yves David <pierre-yves.david@octobus.net> parents: diff changeset	319 except error.InvalidBundleSpecification:
74271829ddc0 clonebundles: move a bundle of clone bundle related code to a new module Pierre-Yves David <pierre-yves.david@octobus.net> parents: diff changeset	320 pass
74271829ddc0 clonebundles: move a bundle of clone bundle related code to a new module Pierre-Yves David <pierre-yves.david@octobus.net> parents: diff changeset	321 except error.UnsupportedBundleSpecification:
74271829ddc0 clonebundles: move a bundle of clone bundle related code to a new module Pierre-Yves David <pierre-yves.david@octobus.net> parents: diff changeset	322 pass
74271829ddc0 clonebundles: move a bundle of clone bundle related code to a new module Pierre-Yves David <pierre-yves.david@octobus.net> parents: diff changeset	323
74271829ddc0 clonebundles: move a bundle of clone bundle related code to a new module Pierre-Yves David <pierre-yves.david@octobus.net> parents: diff changeset	324 m.append(attrs)
74271829ddc0 clonebundles: move a bundle of clone bundle related code to a new module Pierre-Yves David <pierre-yves.david@octobus.net> parents: diff changeset	325
74271829ddc0 clonebundles: move a bundle of clone bundle related code to a new module Pierre-Yves David <pierre-yves.david@octobus.net> parents: diff changeset	326 return m
74271829ddc0 clonebundles: move a bundle of clone bundle related code to a new module Pierre-Yves David <pierre-yves.david@octobus.net> parents: diff changeset	327
74271829ddc0 clonebundles: move a bundle of clone bundle related code to a new module Pierre-Yves David <pierre-yves.david@octobus.net> parents: diff changeset	328
74271829ddc0 clonebundles: move a bundle of clone bundle related code to a new module Pierre-Yves David <pierre-yves.david@octobus.net> parents: diff changeset	329 def isstreamclonespec(bundlespec):
74271829ddc0 clonebundles: move a bundle of clone bundle related code to a new module Pierre-Yves David <pierre-yves.david@octobus.net> parents: diff changeset	330 # Stream clone v1
74271829ddc0 clonebundles: move a bundle of clone bundle related code to a new module Pierre-Yves David <pierre-yves.david@octobus.net> parents: diff changeset	331 if bundlespec.wirecompression == b'UN' and bundlespec.wireversion == b's1':
74271829ddc0 clonebundles: move a bundle of clone bundle related code to a new module Pierre-Yves David <pierre-yves.david@octobus.net> parents: diff changeset	332 return True
74271829ddc0 clonebundles: move a bundle of clone bundle related code to a new module Pierre-Yves David <pierre-yves.david@octobus.net> parents: diff changeset	333
74271829ddc0 clonebundles: move a bundle of clone bundle related code to a new module Pierre-Yves David <pierre-yves.david@octobus.net> parents: diff changeset	334 # Stream clone v2
74271829ddc0 clonebundles: move a bundle of clone bundle related code to a new module Pierre-Yves David <pierre-yves.david@octobus.net> parents: diff changeset	335 if (
74271829ddc0 clonebundles: move a bundle of clone bundle related code to a new module Pierre-Yves David <pierre-yves.david@octobus.net> parents: diff changeset	336 bundlespec.wirecompression == b'UN'
74271829ddc0 clonebundles: move a bundle of clone bundle related code to a new module Pierre-Yves David <pierre-yves.david@octobus.net> parents: diff changeset	337 and bundlespec.wireversion == b'02'
74271829ddc0 clonebundles: move a bundle of clone bundle related code to a new module Pierre-Yves David <pierre-yves.david@octobus.net> parents: diff changeset	338 and bundlespec.contentopts.get(b'streamv2')
74271829ddc0 clonebundles: move a bundle of clone bundle related code to a new module Pierre-Yves David <pierre-yves.david@octobus.net> parents: diff changeset	339 ):
74271829ddc0 clonebundles: move a bundle of clone bundle related code to a new module Pierre-Yves David <pierre-yves.david@octobus.net> parents: diff changeset	340 return True
74271829ddc0 clonebundles: move a bundle of clone bundle related code to a new module Pierre-Yves David <pierre-yves.david@octobus.net> parents: diff changeset	341
74271829ddc0 clonebundles: move a bundle of clone bundle related code to a new module Pierre-Yves David <pierre-yves.david@octobus.net> parents: diff changeset	342 return False
74271829ddc0 clonebundles: move a bundle of clone bundle related code to a new module Pierre-Yves David <pierre-yves.david@octobus.net> parents: diff changeset	343
74271829ddc0 clonebundles: move a bundle of clone bundle related code to a new module Pierre-Yves David <pierre-yves.david@octobus.net> parents: diff changeset	344
50539 7b723217d368 clonebundles: filter out invalid schemes instead of failing on them Mathias De Mare <mathias.de_mare@nokia.com> parents: 49648 diff changeset	345 def filterclonebundleentries(
7b723217d368 clonebundles: filter out invalid schemes instead of failing on them Mathias De Mare <mathias.de_mare@nokia.com> parents: 49648 diff changeset	346 repo, entries, streamclonerequested=False, pullbundles=False
7b723217d368 clonebundles: filter out invalid schemes instead of failing on them Mathias De Mare <mathias.de_mare@nokia.com> parents: 49648 diff changeset	347 ):
45784 74271829ddc0 clonebundles: move a bundle of clone bundle related code to a new module Pierre-Yves David <pierre-yves.david@octobus.net> parents: diff changeset	348 """Remove incompatible clone bundle manifest entries.
74271829ddc0 clonebundles: move a bundle of clone bundle related code to a new module Pierre-Yves David <pierre-yves.david@octobus.net> parents: diff changeset	349
74271829ddc0 clonebundles: move a bundle of clone bundle related code to a new module Pierre-Yves David <pierre-yves.david@octobus.net> parents: diff changeset	350 Accepts a list of entries parsed with ``parseclonebundlesmanifest``
74271829ddc0 clonebundles: move a bundle of clone bundle related code to a new module Pierre-Yves David <pierre-yves.david@octobus.net> parents: diff changeset	351 and returns a new list consisting of only the entries that this client
74271829ddc0 clonebundles: move a bundle of clone bundle related code to a new module Pierre-Yves David <pierre-yves.david@octobus.net> parents: diff changeset	352 should be able to apply.
74271829ddc0 clonebundles: move a bundle of clone bundle related code to a new module Pierre-Yves David <pierre-yves.david@octobus.net> parents: diff changeset	353
74271829ddc0 clonebundles: move a bundle of clone bundle related code to a new module Pierre-Yves David <pierre-yves.david@octobus.net> parents: diff changeset	354 There is no guarantee we'll be able to apply all returned entries because
74271829ddc0 clonebundles: move a bundle of clone bundle related code to a new module Pierre-Yves David <pierre-yves.david@octobus.net> parents: diff changeset	355 the metadata we use to filter on may be missing or wrong.
74271829ddc0 clonebundles: move a bundle of clone bundle related code to a new module Pierre-Yves David <pierre-yves.david@octobus.net> parents: diff changeset	356 """
74271829ddc0 clonebundles: move a bundle of clone bundle related code to a new module Pierre-Yves David <pierre-yves.david@octobus.net> parents: diff changeset	357 newentries = []
74271829ddc0 clonebundles: move a bundle of clone bundle related code to a new module Pierre-Yves David <pierre-yves.david@octobus.net> parents: diff changeset	358 for entry in entries:
50539 7b723217d368 clonebundles: filter out invalid schemes instead of failing on them Mathias De Mare <mathias.de_mare@nokia.com> parents: 49648 diff changeset	359 url = entry.get(b'URL')
7b723217d368 clonebundles: filter out invalid schemes instead of failing on them Mathias De Mare <mathias.de_mare@nokia.com> parents: 49648 diff changeset	360 if not pullbundles and not any(
7b723217d368 clonebundles: filter out invalid schemes instead of failing on them Mathias De Mare <mathias.de_mare@nokia.com> parents: 49648 diff changeset	361 [url.startswith(scheme) for scheme in SUPPORTED_CLONEBUNDLE_SCHEMES]
7b723217d368 clonebundles: filter out invalid schemes instead of failing on them Mathias De Mare <mathias.de_mare@nokia.com> parents: 49648 diff changeset	362 ):
7b723217d368 clonebundles: filter out invalid schemes instead of failing on them Mathias De Mare <mathias.de_mare@nokia.com> parents: 49648 diff changeset	363 repo.ui.debug(
7b723217d368 clonebundles: filter out invalid schemes instead of failing on them Mathias De Mare <mathias.de_mare@nokia.com> parents: 49648 diff changeset	364 b'filtering %s because not a supported clonebundle scheme\n'
7b723217d368 clonebundles: filter out invalid schemes instead of failing on them Mathias De Mare <mathias.de_mare@nokia.com> parents: 49648 diff changeset	365 % url
7b723217d368 clonebundles: filter out invalid schemes instead of failing on them Mathias De Mare <mathias.de_mare@nokia.com> parents: 49648 diff changeset	366 )
7b723217d368 clonebundles: filter out invalid schemes instead of failing on them Mathias De Mare <mathias.de_mare@nokia.com> parents: 49648 diff changeset	367 continue
7b723217d368 clonebundles: filter out invalid schemes instead of failing on them Mathias De Mare <mathias.de_mare@nokia.com> parents: 49648 diff changeset	368
45784 74271829ddc0 clonebundles: move a bundle of clone bundle related code to a new module Pierre-Yves David <pierre-yves.david@octobus.net> parents: diff changeset	369 spec = entry.get(b'BUNDLESPEC')
74271829ddc0 clonebundles: move a bundle of clone bundle related code to a new module Pierre-Yves David <pierre-yves.david@octobus.net> parents: diff changeset	370 if spec:
74271829ddc0 clonebundles: move a bundle of clone bundle related code to a new module Pierre-Yves David <pierre-yves.david@octobus.net> parents: diff changeset	371 try:
74271829ddc0 clonebundles: move a bundle of clone bundle related code to a new module Pierre-Yves David <pierre-yves.david@octobus.net> parents: diff changeset	372 bundlespec = parsebundlespec(repo, spec, strict=True)
74271829ddc0 clonebundles: move a bundle of clone bundle related code to a new module Pierre-Yves David <pierre-yves.david@octobus.net> parents: diff changeset	373
74271829ddc0 clonebundles: move a bundle of clone bundle related code to a new module Pierre-Yves David <pierre-yves.david@octobus.net> parents: diff changeset	374 # If a stream clone was requested, filter out non-streamclone
74271829ddc0 clonebundles: move a bundle of clone bundle related code to a new module Pierre-Yves David <pierre-yves.david@octobus.net> parents: diff changeset	375 # entries.
74271829ddc0 clonebundles: move a bundle of clone bundle related code to a new module Pierre-Yves David <pierre-yves.david@octobus.net> parents: diff changeset	376 if streamclonerequested and not isstreamclonespec(bundlespec):
74271829ddc0 clonebundles: move a bundle of clone bundle related code to a new module Pierre-Yves David <pierre-yves.david@octobus.net> parents: diff changeset	377 repo.ui.debug(
50539 7b723217d368 clonebundles: filter out invalid schemes instead of failing on them Mathias De Mare <mathias.de_mare@nokia.com> parents: 49648 diff changeset	378 b'filtering %s because not a stream clone\n' % url
45784 74271829ddc0 clonebundles: move a bundle of clone bundle related code to a new module Pierre-Yves David <pierre-yves.david@octobus.net> parents: diff changeset	379 )
74271829ddc0 clonebundles: move a bundle of clone bundle related code to a new module Pierre-Yves David <pierre-yves.david@octobus.net> parents: diff changeset	380 continue
74271829ddc0 clonebundles: move a bundle of clone bundle related code to a new module Pierre-Yves David <pierre-yves.david@octobus.net> parents: diff changeset	381
74271829ddc0 clonebundles: move a bundle of clone bundle related code to a new module Pierre-Yves David <pierre-yves.david@octobus.net> parents: diff changeset	382 except error.InvalidBundleSpecification as e:
74271829ddc0 clonebundles: move a bundle of clone bundle related code to a new module Pierre-Yves David <pierre-yves.david@octobus.net> parents: diff changeset	383 repo.ui.debug(stringutil.forcebytestr(e) + b'\n')
74271829ddc0 clonebundles: move a bundle of clone bundle related code to a new module Pierre-Yves David <pierre-yves.david@octobus.net> parents: diff changeset	384 continue
74271829ddc0 clonebundles: move a bundle of clone bundle related code to a new module Pierre-Yves David <pierre-yves.david@octobus.net> parents: diff changeset	385 except error.UnsupportedBundleSpecification as e:
74271829ddc0 clonebundles: move a bundle of clone bundle related code to a new module Pierre-Yves David <pierre-yves.david@octobus.net> parents: diff changeset	386 repo.ui.debug(
74271829ddc0 clonebundles: move a bundle of clone bundle related code to a new module Pierre-Yves David <pierre-yves.david@octobus.net> parents: diff changeset	387 b'filtering %s because unsupported bundle '
50539 7b723217d368 clonebundles: filter out invalid schemes instead of failing on them Mathias De Mare <mathias.de_mare@nokia.com> parents: 49648 diff changeset	388 b'spec: %s\n' % (url, stringutil.forcebytestr(e))
45784 74271829ddc0 clonebundles: move a bundle of clone bundle related code to a new module Pierre-Yves David <pierre-yves.david@octobus.net> parents: diff changeset	389 )
74271829ddc0 clonebundles: move a bundle of clone bundle related code to a new module Pierre-Yves David <pierre-yves.david@octobus.net> parents: diff changeset	390 continue
74271829ddc0 clonebundles: move a bundle of clone bundle related code to a new module Pierre-Yves David <pierre-yves.david@octobus.net> parents: diff changeset	391 # If we don't have a spec and requested a stream clone, we don't know
74271829ddc0 clonebundles: move a bundle of clone bundle related code to a new module Pierre-Yves David <pierre-yves.david@octobus.net> parents: diff changeset	392 # what the entry is so don't attempt to apply it.
74271829ddc0 clonebundles: move a bundle of clone bundle related code to a new module Pierre-Yves David <pierre-yves.david@octobus.net> parents: diff changeset	393 elif streamclonerequested:
74271829ddc0 clonebundles: move a bundle of clone bundle related code to a new module Pierre-Yves David <pierre-yves.david@octobus.net> parents: diff changeset	394 repo.ui.debug(
74271829ddc0 clonebundles: move a bundle of clone bundle related code to a new module Pierre-Yves David <pierre-yves.david@octobus.net> parents: diff changeset	395 b'filtering %s because cannot determine if a stream '
50539 7b723217d368 clonebundles: filter out invalid schemes instead of failing on them Mathias De Mare <mathias.de_mare@nokia.com> parents: 49648 diff changeset	396 b'clone bundle\n' % url
45784 74271829ddc0 clonebundles: move a bundle of clone bundle related code to a new module Pierre-Yves David <pierre-yves.david@octobus.net> parents: diff changeset	397 )
74271829ddc0 clonebundles: move a bundle of clone bundle related code to a new module Pierre-Yves David <pierre-yves.david@octobus.net> parents: diff changeset	398 continue
74271829ddc0 clonebundles: move a bundle of clone bundle related code to a new module Pierre-Yves David <pierre-yves.david@octobus.net> parents: diff changeset	399
74271829ddc0 clonebundles: move a bundle of clone bundle related code to a new module Pierre-Yves David <pierre-yves.david@octobus.net> parents: diff changeset	400 if b'REQUIRESNI' in entry and not sslutil.hassni:
50539 7b723217d368 clonebundles: filter out invalid schemes instead of failing on them Mathias De Mare <mathias.de_mare@nokia.com> parents: 49648 diff changeset	401 repo.ui.debug(b'filtering %s because SNI not supported\n' % url)
45784 74271829ddc0 clonebundles: move a bundle of clone bundle related code to a new module Pierre-Yves David <pierre-yves.david@octobus.net> parents: diff changeset	402 continue
74271829ddc0 clonebundles: move a bundle of clone bundle related code to a new module Pierre-Yves David <pierre-yves.david@octobus.net> parents: diff changeset	403
74271829ddc0 clonebundles: move a bundle of clone bundle related code to a new module Pierre-Yves David <pierre-yves.david@octobus.net> parents: diff changeset	404 if b'REQUIREDRAM' in entry:
74271829ddc0 clonebundles: move a bundle of clone bundle related code to a new module Pierre-Yves David <pierre-yves.david@octobus.net> parents: diff changeset	405 try:
74271829ddc0 clonebundles: move a bundle of clone bundle related code to a new module Pierre-Yves David <pierre-yves.david@octobus.net> parents: diff changeset	406 requiredram = util.sizetoint(entry[b'REQUIREDRAM'])
74271829ddc0 clonebundles: move a bundle of clone bundle related code to a new module Pierre-Yves David <pierre-yves.david@octobus.net> parents: diff changeset	407 except error.ParseError:
74271829ddc0 clonebundles: move a bundle of clone bundle related code to a new module Pierre-Yves David <pierre-yves.david@octobus.net> parents: diff changeset	408 repo.ui.debug(
50539 7b723217d368 clonebundles: filter out invalid schemes instead of failing on them Mathias De Mare <mathias.de_mare@nokia.com> parents: 49648 diff changeset	409 b'filtering %s due to a bad REQUIREDRAM attribute\n' % url
45784 74271829ddc0 clonebundles: move a bundle of clone bundle related code to a new module Pierre-Yves David <pierre-yves.david@octobus.net> parents: diff changeset	410 )
74271829ddc0 clonebundles: move a bundle of clone bundle related code to a new module Pierre-Yves David <pierre-yves.david@octobus.net> parents: diff changeset	411 continue
74271829ddc0 clonebundles: move a bundle of clone bundle related code to a new module Pierre-Yves David <pierre-yves.david@octobus.net> parents: diff changeset	412 actualram = repo.ui.estimatememory()
74271829ddc0 clonebundles: move a bundle of clone bundle related code to a new module Pierre-Yves David <pierre-yves.david@octobus.net> parents: diff changeset	413 if actualram is not None and actualram * 0.66 < requiredram:
74271829ddc0 clonebundles: move a bundle of clone bundle related code to a new module Pierre-Yves David <pierre-yves.david@octobus.net> parents: diff changeset	414 repo.ui.debug(
74271829ddc0 clonebundles: move a bundle of clone bundle related code to a new module Pierre-Yves David <pierre-yves.david@octobus.net> parents: diff changeset	415 b'filtering %s as it needs more than 2/3 of system memory\n'
50539 7b723217d368 clonebundles: filter out invalid schemes instead of failing on them Mathias De Mare <mathias.de_mare@nokia.com> parents: 49648 diff changeset	416 % url
45784 74271829ddc0 clonebundles: move a bundle of clone bundle related code to a new module Pierre-Yves David <pierre-yves.david@octobus.net> parents: diff changeset	417 )
74271829ddc0 clonebundles: move a bundle of clone bundle related code to a new module Pierre-Yves David <pierre-yves.david@octobus.net> parents: diff changeset	418 continue
74271829ddc0 clonebundles: move a bundle of clone bundle related code to a new module Pierre-Yves David <pierre-yves.david@octobus.net> parents: diff changeset	419
74271829ddc0 clonebundles: move a bundle of clone bundle related code to a new module Pierre-Yves David <pierre-yves.david@octobus.net> parents: diff changeset	420 newentries.append(entry)
74271829ddc0 clonebundles: move a bundle of clone bundle related code to a new module Pierre-Yves David <pierre-yves.david@octobus.net> parents: diff changeset	421
74271829ddc0 clonebundles: move a bundle of clone bundle related code to a new module Pierre-Yves David <pierre-yves.david@octobus.net> parents: diff changeset	422 return newentries
74271829ddc0 clonebundles: move a bundle of clone bundle related code to a new module Pierre-Yves David <pierre-yves.david@octobus.net> parents: diff changeset	423
74271829ddc0 clonebundles: move a bundle of clone bundle related code to a new module Pierre-Yves David <pierre-yves.david@octobus.net> parents: diff changeset	424
48946 642e31cb55f0 py3: use class X: instead of class X(object): Gregory Szorc <gregory.szorc@gmail.com> parents: 48601 diff changeset	425 class clonebundleentry:
45784 74271829ddc0 clonebundles: move a bundle of clone bundle related code to a new module Pierre-Yves David <pierre-yves.david@octobus.net> parents: diff changeset	426 """Represents an item in a clone bundles manifest.
74271829ddc0 clonebundles: move a bundle of clone bundle related code to a new module Pierre-Yves David <pierre-yves.david@octobus.net> parents: diff changeset	427
74271829ddc0 clonebundles: move a bundle of clone bundle related code to a new module Pierre-Yves David <pierre-yves.david@octobus.net> parents: diff changeset	428 This rich class is needed to support sorting since sorted() in Python 3
74271829ddc0 clonebundles: move a bundle of clone bundle related code to a new module Pierre-Yves David <pierre-yves.david@octobus.net> parents: diff changeset	429 doesn't support ``cmp`` and our comparison is complex enough that ``key=``
74271829ddc0 clonebundles: move a bundle of clone bundle related code to a new module Pierre-Yves David <pierre-yves.david@octobus.net> parents: diff changeset	430 won't work.
74271829ddc0 clonebundles: move a bundle of clone bundle related code to a new module Pierre-Yves David <pierre-yves.david@octobus.net> parents: diff changeset	431 """
74271829ddc0 clonebundles: move a bundle of clone bundle related code to a new module Pierre-Yves David <pierre-yves.david@octobus.net> parents: diff changeset	432
74271829ddc0 clonebundles: move a bundle of clone bundle related code to a new module Pierre-Yves David <pierre-yves.david@octobus.net> parents: diff changeset	433 def __init__(self, value, prefers):
74271829ddc0 clonebundles: move a bundle of clone bundle related code to a new module Pierre-Yves David <pierre-yves.david@octobus.net> parents: diff changeset	434 self.value = value
74271829ddc0 clonebundles: move a bundle of clone bundle related code to a new module Pierre-Yves David <pierre-yves.david@octobus.net> parents: diff changeset	435 self.prefers = prefers
74271829ddc0 clonebundles: move a bundle of clone bundle related code to a new module Pierre-Yves David <pierre-yves.david@octobus.net> parents: diff changeset	436
74271829ddc0 clonebundles: move a bundle of clone bundle related code to a new module Pierre-Yves David <pierre-yves.david@octobus.net> parents: diff changeset	437 def _cmp(self, other):
74271829ddc0 clonebundles: move a bundle of clone bundle related code to a new module Pierre-Yves David <pierre-yves.david@octobus.net> parents: diff changeset	438 for prefkey, prefvalue in self.prefers:
74271829ddc0 clonebundles: move a bundle of clone bundle related code to a new module Pierre-Yves David <pierre-yves.david@octobus.net> parents: diff changeset	439 avalue = self.value.get(prefkey)
74271829ddc0 clonebundles: move a bundle of clone bundle related code to a new module Pierre-Yves David <pierre-yves.david@octobus.net> parents: diff changeset	440 bvalue = other.value.get(prefkey)
74271829ddc0 clonebundles: move a bundle of clone bundle related code to a new module Pierre-Yves David <pierre-yves.david@octobus.net> parents: diff changeset	441
74271829ddc0 clonebundles: move a bundle of clone bundle related code to a new module Pierre-Yves David <pierre-yves.david@octobus.net> parents: diff changeset	442 # Special case for b missing attribute and a matches exactly.
74271829ddc0 clonebundles: move a bundle of clone bundle related code to a new module Pierre-Yves David <pierre-yves.david@octobus.net> parents: diff changeset	443 if avalue is not None and bvalue is None and avalue == prefvalue:
74271829ddc0 clonebundles: move a bundle of clone bundle related code to a new module Pierre-Yves David <pierre-yves.david@octobus.net> parents: diff changeset	444 return -1
74271829ddc0 clonebundles: move a bundle of clone bundle related code to a new module Pierre-Yves David <pierre-yves.david@octobus.net> parents: diff changeset	445
74271829ddc0 clonebundles: move a bundle of clone bundle related code to a new module Pierre-Yves David <pierre-yves.david@octobus.net> parents: diff changeset	446 # Special case for a missing attribute and b matches exactly.
74271829ddc0 clonebundles: move a bundle of clone bundle related code to a new module Pierre-Yves David <pierre-yves.david@octobus.net> parents: diff changeset	447 if bvalue is not None and avalue is None and bvalue == prefvalue:
74271829ddc0 clonebundles: move a bundle of clone bundle related code to a new module Pierre-Yves David <pierre-yves.david@octobus.net> parents: diff changeset	448 return 1
74271829ddc0 clonebundles: move a bundle of clone bundle related code to a new module Pierre-Yves David <pierre-yves.david@octobus.net> parents: diff changeset	449
74271829ddc0 clonebundles: move a bundle of clone bundle related code to a new module Pierre-Yves David <pierre-yves.david@octobus.net> parents: diff changeset	450 # We can't compare unless attribute present on both.
74271829ddc0 clonebundles: move a bundle of clone bundle related code to a new module Pierre-Yves David <pierre-yves.david@octobus.net> parents: diff changeset	451 if avalue is None or bvalue is None:
74271829ddc0 clonebundles: move a bundle of clone bundle related code to a new module Pierre-Yves David <pierre-yves.david@octobus.net> parents: diff changeset	452 continue
74271829ddc0 clonebundles: move a bundle of clone bundle related code to a new module Pierre-Yves David <pierre-yves.david@octobus.net> parents: diff changeset	453
74271829ddc0 clonebundles: move a bundle of clone bundle related code to a new module Pierre-Yves David <pierre-yves.david@octobus.net> parents: diff changeset	454 # Same values should fall back to next attribute.
74271829ddc0 clonebundles: move a bundle of clone bundle related code to a new module Pierre-Yves David <pierre-yves.david@octobus.net> parents: diff changeset	455 if avalue == bvalue:
74271829ddc0 clonebundles: move a bundle of clone bundle related code to a new module Pierre-Yves David <pierre-yves.david@octobus.net> parents: diff changeset	456 continue
74271829ddc0 clonebundles: move a bundle of clone bundle related code to a new module Pierre-Yves David <pierre-yves.david@octobus.net> parents: diff changeset	457
74271829ddc0 clonebundles: move a bundle of clone bundle related code to a new module Pierre-Yves David <pierre-yves.david@octobus.net> parents: diff changeset	458 # Exact matches come first.
74271829ddc0 clonebundles: move a bundle of clone bundle related code to a new module Pierre-Yves David <pierre-yves.david@octobus.net> parents: diff changeset	459 if avalue == prefvalue:
74271829ddc0 clonebundles: move a bundle of clone bundle related code to a new module Pierre-Yves David <pierre-yves.david@octobus.net> parents: diff changeset	460 return -1
74271829ddc0 clonebundles: move a bundle of clone bundle related code to a new module Pierre-Yves David <pierre-yves.david@octobus.net> parents: diff changeset	461 if bvalue == prefvalue:
74271829ddc0 clonebundles: move a bundle of clone bundle related code to a new module Pierre-Yves David <pierre-yves.david@octobus.net> parents: diff changeset	462 return 1
74271829ddc0 clonebundles: move a bundle of clone bundle related code to a new module Pierre-Yves David <pierre-yves.david@octobus.net> parents: diff changeset	463
74271829ddc0 clonebundles: move a bundle of clone bundle related code to a new module Pierre-Yves David <pierre-yves.david@octobus.net> parents: diff changeset	464 # Fall back to next attribute.
74271829ddc0 clonebundles: move a bundle of clone bundle related code to a new module Pierre-Yves David <pierre-yves.david@octobus.net> parents: diff changeset	465 continue
74271829ddc0 clonebundles: move a bundle of clone bundle related code to a new module Pierre-Yves David <pierre-yves.david@octobus.net> parents: diff changeset	466
74271829ddc0 clonebundles: move a bundle of clone bundle related code to a new module Pierre-Yves David <pierre-yves.david@octobus.net> parents: diff changeset	467 # If we got here we couldn't sort by attributes and prefers. Fall
74271829ddc0 clonebundles: move a bundle of clone bundle related code to a new module Pierre-Yves David <pierre-yves.david@octobus.net> parents: diff changeset	468 # back to index order.
74271829ddc0 clonebundles: move a bundle of clone bundle related code to a new module Pierre-Yves David <pierre-yves.david@octobus.net> parents: diff changeset	469 return 0
74271829ddc0 clonebundles: move a bundle of clone bundle related code to a new module Pierre-Yves David <pierre-yves.david@octobus.net> parents: diff changeset	470
74271829ddc0 clonebundles: move a bundle of clone bundle related code to a new module Pierre-Yves David <pierre-yves.david@octobus.net> parents: diff changeset	471 def __lt__(self, other):
74271829ddc0 clonebundles: move a bundle of clone bundle related code to a new module Pierre-Yves David <pierre-yves.david@octobus.net> parents: diff changeset	472 return self._cmp(other) < 0
74271829ddc0 clonebundles: move a bundle of clone bundle related code to a new module Pierre-Yves David <pierre-yves.david@octobus.net> parents: diff changeset	473
74271829ddc0 clonebundles: move a bundle of clone bundle related code to a new module Pierre-Yves David <pierre-yves.david@octobus.net> parents: diff changeset	474 def __gt__(self, other):
74271829ddc0 clonebundles: move a bundle of clone bundle related code to a new module Pierre-Yves David <pierre-yves.david@octobus.net> parents: diff changeset	475 return self._cmp(other) > 0
74271829ddc0 clonebundles: move a bundle of clone bundle related code to a new module Pierre-Yves David <pierre-yves.david@octobus.net> parents: diff changeset	476
74271829ddc0 clonebundles: move a bundle of clone bundle related code to a new module Pierre-Yves David <pierre-yves.david@octobus.net> parents: diff changeset	477 def __eq__(self, other):
74271829ddc0 clonebundles: move a bundle of clone bundle related code to a new module Pierre-Yves David <pierre-yves.david@octobus.net> parents: diff changeset	478 return self._cmp(other) == 0
74271829ddc0 clonebundles: move a bundle of clone bundle related code to a new module Pierre-Yves David <pierre-yves.david@octobus.net> parents: diff changeset	479
74271829ddc0 clonebundles: move a bundle of clone bundle related code to a new module Pierre-Yves David <pierre-yves.david@octobus.net> parents: diff changeset	480 def __le__(self, other):
74271829ddc0 clonebundles: move a bundle of clone bundle related code to a new module Pierre-Yves David <pierre-yves.david@octobus.net> parents: diff changeset	481 return self._cmp(other) <= 0
74271829ddc0 clonebundles: move a bundle of clone bundle related code to a new module Pierre-Yves David <pierre-yves.david@octobus.net> parents: diff changeset	482
74271829ddc0 clonebundles: move a bundle of clone bundle related code to a new module Pierre-Yves David <pierre-yves.david@octobus.net> parents: diff changeset	483 def __ge__(self, other):
74271829ddc0 clonebundles: move a bundle of clone bundle related code to a new module Pierre-Yves David <pierre-yves.david@octobus.net> parents: diff changeset	484 return self._cmp(other) >= 0
74271829ddc0 clonebundles: move a bundle of clone bundle related code to a new module Pierre-Yves David <pierre-yves.david@octobus.net> parents: diff changeset	485
74271829ddc0 clonebundles: move a bundle of clone bundle related code to a new module Pierre-Yves David <pierre-yves.david@octobus.net> parents: diff changeset	486 def __ne__(self, other):
74271829ddc0 clonebundles: move a bundle of clone bundle related code to a new module Pierre-Yves David <pierre-yves.david@octobus.net> parents: diff changeset	487 return self._cmp(other) != 0
74271829ddc0 clonebundles: move a bundle of clone bundle related code to a new module Pierre-Yves David <pierre-yves.david@octobus.net> parents: diff changeset	488
74271829ddc0 clonebundles: move a bundle of clone bundle related code to a new module Pierre-Yves David <pierre-yves.david@octobus.net> parents: diff changeset	489
74271829ddc0 clonebundles: move a bundle of clone bundle related code to a new module Pierre-Yves David <pierre-yves.david@octobus.net> parents: diff changeset	490 def sortclonebundleentries(ui, entries):
74271829ddc0 clonebundles: move a bundle of clone bundle related code to a new module Pierre-Yves David <pierre-yves.david@octobus.net> parents: diff changeset	491 prefers = ui.configlist(b'ui', b'clonebundleprefers')
74271829ddc0 clonebundles: move a bundle of clone bundle related code to a new module Pierre-Yves David <pierre-yves.david@octobus.net> parents: diff changeset	492 if not prefers:
74271829ddc0 clonebundles: move a bundle of clone bundle related code to a new module Pierre-Yves David <pierre-yves.david@octobus.net> parents: diff changeset	493 return list(entries)
74271829ddc0 clonebundles: move a bundle of clone bundle related code to a new module Pierre-Yves David <pierre-yves.david@octobus.net> parents: diff changeset	494
74271829ddc0 clonebundles: move a bundle of clone bundle related code to a new module Pierre-Yves David <pierre-yves.david@octobus.net> parents: diff changeset	495 def _split(p):
74271829ddc0 clonebundles: move a bundle of clone bundle related code to a new module Pierre-Yves David <pierre-yves.david@octobus.net> parents: diff changeset	496 if b'=' not in p:
74271829ddc0 clonebundles: move a bundle of clone bundle related code to a new module Pierre-Yves David <pierre-yves.david@octobus.net> parents: diff changeset	497 hint = _(b"each comma separated item should be key=value pairs")
74271829ddc0 clonebundles: move a bundle of clone bundle related code to a new module Pierre-Yves David <pierre-yves.david@octobus.net> parents: diff changeset	498 raise error.Abort(
74271829ddc0 clonebundles: move a bundle of clone bundle related code to a new module Pierre-Yves David <pierre-yves.david@octobus.net> parents: diff changeset	499 _(b"invalid ui.clonebundleprefers item: %s") % p, hint=hint
74271829ddc0 clonebundles: move a bundle of clone bundle related code to a new module Pierre-Yves David <pierre-yves.david@octobus.net> parents: diff changeset	500 )
74271829ddc0 clonebundles: move a bundle of clone bundle related code to a new module Pierre-Yves David <pierre-yves.david@octobus.net> parents: diff changeset	501 return p.split(b'=', 1)
74271829ddc0 clonebundles: move a bundle of clone bundle related code to a new module Pierre-Yves David <pierre-yves.david@octobus.net> parents: diff changeset	502
74271829ddc0 clonebundles: move a bundle of clone bundle related code to a new module Pierre-Yves David <pierre-yves.david@octobus.net> parents: diff changeset	503 prefers = [_split(p) for p in prefers]
74271829ddc0 clonebundles: move a bundle of clone bundle related code to a new module Pierre-Yves David <pierre-yves.david@octobus.net> parents: diff changeset	504
74271829ddc0 clonebundles: move a bundle of clone bundle related code to a new module Pierre-Yves David <pierre-yves.david@octobus.net> parents: diff changeset	505 items = sorted(clonebundleentry(v, prefers) for v in entries)
74271829ddc0 clonebundles: move a bundle of clone bundle related code to a new module Pierre-Yves David <pierre-yves.david@octobus.net> parents: diff changeset	506 return [i.value for i in items]

Mercurial > public > mercurial-scm > hg

annotate mercurial/bundlecaches.py @ 50539:7b723217d368 stable