mercurial-scm/hg: mercurial/utils/storageutil.py annotate

annotate mercurial/utils/storageutil.py @ 52049:af54626bf358

dirstate-map: add a missing debug wait point when accessing the v2 docket fc8e37c380d3 added synchronization points to the dirstate to allow for race condition testing without actually requiring a time-based race condition to happen. This changes adds the `pre-read-file` wait point before we read the docket, since callers might ask for the parents before anything else is read, leading to the first read being done before the wait point. This removes some differences in test output which were presumed to be speed related, but weren't.

author	Rapha?l Gom?s <rgomes@octobus.net>
date	Mon, 14 Oct 2024 14:14:21 +0200
parents	f4733654f144
children	f066fc0bdc7a

rev	line source
39877 f8eb71f9e3bd storageutil: new module for storage primitives (API) Gregory Szorc <gregory.szorc@gmail.com> parents: diff changeset	1 # storageutil.py - Storage functionality agnostic of backend implementation.
f8eb71f9e3bd storageutil: new module for storage primitives (API) Gregory Szorc <gregory.szorc@gmail.com> parents: diff changeset	2 #
f8eb71f9e3bd storageutil: new module for storage primitives (API) Gregory Szorc <gregory.szorc@gmail.com> parents: diff changeset	3 # Copyright 2018 Gregory Szorc <gregory.szorc@gmail.com>
f8eb71f9e3bd storageutil: new module for storage primitives (API) Gregory Szorc <gregory.szorc@gmail.com> parents: diff changeset	4 #
f8eb71f9e3bd storageutil: new module for storage primitives (API) Gregory Szorc <gregory.szorc@gmail.com> parents: diff changeset	5 # This software may be used and distributed according to the terms of the
f8eb71f9e3bd storageutil: new module for storage primitives (API) Gregory Szorc <gregory.szorc@gmail.com> parents: diff changeset	6 # GNU General Public License version 2 or any later version.
f8eb71f9e3bd storageutil: new module for storage primitives (API) Gregory Szorc <gregory.szorc@gmail.com> parents: diff changeset	7
51859 f4733654f144 typing: add `from __future__ import annotations` to most files Matt Harbison <matt_harbison@yahoo.com> parents: 49967 diff changeset	8 from __future__ import annotations
39877 f8eb71f9e3bd storageutil: new module for storage primitives (API) Gregory Szorc <gregory.szorc@gmail.com> parents: diff changeset	9
39878 3e896b51aa5d storageutil: move metadata parsing and packing from revlog (API) Gregory Szorc <gregory.szorc@gmail.com> parents: 39877 diff changeset	10 import re
40325 b0fbd1792e2d storageutil: extract most of peek_censored from revlog Gregory Szorc <gregory.szorc@gmail.com> parents: 40321 diff changeset	11 import struct
39877 f8eb71f9e3bd storageutil: new module for storage primitives (API) Gregory Szorc <gregory.szorc@gmail.com> parents: diff changeset	12
40002 0e8836be9541 storageutil: implement file identifier resolution method (BC) Gregory Szorc <gregory.szorc@gmail.com> parents: 39881 diff changeset	13 from ..i18n import _
39877 f8eb71f9e3bd storageutil: new module for storage primitives (API) Gregory Szorc <gregory.szorc@gmail.com> parents: diff changeset	14 from ..node import (
40002 0e8836be9541 storageutil: implement file identifier resolution method (BC) Gregory Szorc <gregory.szorc@gmail.com> parents: 39881 diff changeset	15 bin,
40004 fa3dc85a747e storageutil: extract functionality for resolving strip revisions Gregory Szorc <gregory.szorc@gmail.com> parents: 40003 diff changeset	16 nullrev,
47012 d55b71393907 node: replace nullid and friends with nodeconstants class Joerg Sonnenberger <joerg@bec.de> parents: 46715 diff changeset	17 sha1nodeconstants,
39877 f8eb71f9e3bd storageutil: new module for storage primitives (API) Gregory Szorc <gregory.szorc@gmail.com> parents: diff changeset	18 )
39881 d63153611ed5 storageutil: extract revision number iteration Gregory Szorc <gregory.szorc@gmail.com> parents: 39880 diff changeset	19 from .. import (
40010 f5d819d84461 storageutil: pass nodes into emitrevisions() Gregory Szorc <gregory.szorc@gmail.com> parents: 40009 diff changeset	20 dagop,
40002 0e8836be9541 storageutil: implement file identifier resolution method (BC) Gregory Szorc <gregory.szorc@gmail.com> parents: 39881 diff changeset	21 error,
40009 631c6f5058b9 storageutil: make all callables optional Gregory Szorc <gregory.szorc@gmail.com> parents: 40008 diff changeset	22 mdiff,
39881 d63153611ed5 storageutil: extract revision number iteration Gregory Szorc <gregory.szorc@gmail.com> parents: 39880 diff changeset	23 )
42813 268662aac075 interfaces: create a new folder for interfaces and move repository.py in it Pulkit Goyal <pulkit@yandex-team.ru> parents: 42781 diff changeset	24 from ..interfaces import repository
46715 45f0d5297698 changegroupv4: add sidedata helpers Rapha?l Gom?s <rgomes@octobus.net> parents: 46712 diff changeset	25 from ..revlogutils import sidedata as sidedatamod
44060 a61287a95dc3 core: migrate uses of hashlib.sha1 to hashutil.sha1 Augie Fackler <augie@google.com> parents: 43077 diff changeset	26 from ..utils import hashutil
39877 f8eb71f9e3bd storageutil: new module for storage primitives (API) Gregory Szorc <gregory.szorc@gmail.com> parents: diff changeset	27
47012 d55b71393907 node: replace nullid and friends with nodeconstants class Joerg Sonnenberger <joerg@bec.de> parents: 46715 diff changeset	28 _nullhash = hashutil.sha1(sha1nodeconstants.nullid)
39877 f8eb71f9e3bd storageutil: new module for storage primitives (API) Gregory Szorc <gregory.szorc@gmail.com> parents: diff changeset	29
47077 119790e1c67c cg4: introduce protocol flag to signify the presence of sidedata Rapha?l Gom?s <rgomes@octobus.net> parents: 47012 diff changeset	30 # revision data contains extra metadata not part of the official digest
119790e1c67c cg4: introduce protocol flag to signify the presence of sidedata Rapha?l Gom?s <rgomes@octobus.net> parents: 47012 diff changeset	31 # Only used in changegroup >= v4.
119790e1c67c cg4: introduce protocol flag to signify the presence of sidedata Rapha?l Gom?s <rgomes@octobus.net> parents: 47012 diff changeset	32 CG_FLAG_SIDEDATA = 1
119790e1c67c cg4: introduce protocol flag to signify the presence of sidedata Rapha?l Gom?s <rgomes@octobus.net> parents: 47012 diff changeset	33
43076 2372284d9457 formatting: blacken the codebase Augie Fackler <augie@google.com> parents: 42813 diff changeset	34
39877 f8eb71f9e3bd storageutil: new module for storage primitives (API) Gregory Szorc <gregory.szorc@gmail.com> parents: diff changeset	35 def hashrevisionsha1(text, p1, p2):
f8eb71f9e3bd storageutil: new module for storage primitives (API) Gregory Szorc <gregory.szorc@gmail.com> parents: diff changeset	36 """Compute the SHA-1 for revision data and its parents.
f8eb71f9e3bd storageutil: new module for storage primitives (API) Gregory Szorc <gregory.szorc@gmail.com> parents: diff changeset	37
f8eb71f9e3bd storageutil: new module for storage primitives (API) Gregory Szorc <gregory.szorc@gmail.com> parents: diff changeset	38 This hash combines both the current file contents and its history
f8eb71f9e3bd storageutil: new module for storage primitives (API) Gregory Szorc <gregory.szorc@gmail.com> parents: diff changeset	39 in a manner that makes it easy to distinguish nodes with the same
f8eb71f9e3bd storageutil: new module for storage primitives (API) Gregory Szorc <gregory.szorc@gmail.com> parents: diff changeset	40 content in the revision graph.
f8eb71f9e3bd storageutil: new module for storage primitives (API) Gregory Szorc <gregory.szorc@gmail.com> parents: diff changeset	41 """
f8eb71f9e3bd storageutil: new module for storage primitives (API) Gregory Szorc <gregory.szorc@gmail.com> parents: diff changeset	42 # As of now, if one of the parent node is null, p2 is null
47012 d55b71393907 node: replace nullid and friends with nodeconstants class Joerg Sonnenberger <joerg@bec.de> parents: 46715 diff changeset	43 if p2 == sha1nodeconstants.nullid:
39877 f8eb71f9e3bd storageutil: new module for storage primitives (API) Gregory Szorc <gregory.szorc@gmail.com> parents: diff changeset	44 # deep copy of a hash is faster than creating one
f8eb71f9e3bd storageutil: new module for storage primitives (API) Gregory Szorc <gregory.szorc@gmail.com> parents: diff changeset	45 s = _nullhash.copy()
f8eb71f9e3bd storageutil: new module for storage primitives (API) Gregory Szorc <gregory.szorc@gmail.com> parents: diff changeset	46 s.update(p1)
f8eb71f9e3bd storageutil: new module for storage primitives (API) Gregory Szorc <gregory.szorc@gmail.com> parents: diff changeset	47 else:
f8eb71f9e3bd storageutil: new module for storage primitives (API) Gregory Szorc <gregory.szorc@gmail.com> parents: diff changeset	48 # none of the parent nodes are nullid
f8eb71f9e3bd storageutil: new module for storage primitives (API) Gregory Szorc <gregory.szorc@gmail.com> parents: diff changeset	49 if p1 < p2:
f8eb71f9e3bd storageutil: new module for storage primitives (API) Gregory Szorc <gregory.szorc@gmail.com> parents: diff changeset	50 a = p1
f8eb71f9e3bd storageutil: new module for storage primitives (API) Gregory Szorc <gregory.szorc@gmail.com> parents: diff changeset	51 b = p2
f8eb71f9e3bd storageutil: new module for storage primitives (API) Gregory Szorc <gregory.szorc@gmail.com> parents: diff changeset	52 else:
f8eb71f9e3bd storageutil: new module for storage primitives (API) Gregory Szorc <gregory.szorc@gmail.com> parents: diff changeset	53 a = p2
f8eb71f9e3bd storageutil: new module for storage primitives (API) Gregory Szorc <gregory.szorc@gmail.com> parents: diff changeset	54 b = p1
44060 a61287a95dc3 core: migrate uses of hashlib.sha1 to hashutil.sha1 Augie Fackler <augie@google.com> parents: 43077 diff changeset	55 s = hashutil.sha1(a)
39877 f8eb71f9e3bd storageutil: new module for storage primitives (API) Gregory Szorc <gregory.szorc@gmail.com> parents: diff changeset	56 s.update(b)
f8eb71f9e3bd storageutil: new module for storage primitives (API) Gregory Szorc <gregory.szorc@gmail.com> parents: diff changeset	57 s.update(text)
f8eb71f9e3bd storageutil: new module for storage primitives (API) Gregory Szorc <gregory.szorc@gmail.com> parents: diff changeset	58 return s.digest()
39878 3e896b51aa5d storageutil: move metadata parsing and packing from revlog (API) Gregory Szorc <gregory.szorc@gmail.com> parents: 39877 diff changeset	59
43076 2372284d9457 formatting: blacken the codebase Augie Fackler <augie@google.com> parents: 42813 diff changeset	60
39878 3e896b51aa5d storageutil: move metadata parsing and packing from revlog (API) Gregory Szorc <gregory.szorc@gmail.com> parents: 39877 diff changeset	61 METADATA_RE = re.compile(b'\x01\n')
3e896b51aa5d storageutil: move metadata parsing and packing from revlog (API) Gregory Szorc <gregory.szorc@gmail.com> parents: 39877 diff changeset	62
43076 2372284d9457 formatting: blacken the codebase Augie Fackler <augie@google.com> parents: 42813 diff changeset	63
39878 3e896b51aa5d storageutil: move metadata parsing and packing from revlog (API) Gregory Szorc <gregory.szorc@gmail.com> parents: 39877 diff changeset	64 def parsemeta(text):
3e896b51aa5d storageutil: move metadata parsing and packing from revlog (API) Gregory Szorc <gregory.szorc@gmail.com> parents: 39877 diff changeset	65 """Parse metadata header from revision data.
3e896b51aa5d storageutil: move metadata parsing and packing from revlog (API) Gregory Szorc <gregory.szorc@gmail.com> parents: 39877 diff changeset	66
3e896b51aa5d storageutil: move metadata parsing and packing from revlog (API) Gregory Szorc <gregory.szorc@gmail.com> parents: 39877 diff changeset	67 Returns a 2-tuple of (metadata, offset), where both can be None if there
3e896b51aa5d storageutil: move metadata parsing and packing from revlog (API) Gregory Szorc <gregory.szorc@gmail.com> parents: 39877 diff changeset	68 is no metadata.
3e896b51aa5d storageutil: move metadata parsing and packing from revlog (API) Gregory Szorc <gregory.szorc@gmail.com> parents: 39877 diff changeset	69 """
3e896b51aa5d storageutil: move metadata parsing and packing from revlog (API) Gregory Szorc <gregory.szorc@gmail.com> parents: 39877 diff changeset	70 # text can be buffer, so we can't use .startswith or .index
3e896b51aa5d storageutil: move metadata parsing and packing from revlog (API) Gregory Szorc <gregory.szorc@gmail.com> parents: 39877 diff changeset	71 if text[:2] != b'\x01\n':
3e896b51aa5d storageutil: move metadata parsing and packing from revlog (API) Gregory Szorc <gregory.szorc@gmail.com> parents: 39877 diff changeset	72 return None, None
3e896b51aa5d storageutil: move metadata parsing and packing from revlog (API) Gregory Szorc <gregory.szorc@gmail.com> parents: 39877 diff changeset	73 s = METADATA_RE.search(text, 2).start()
3e896b51aa5d storageutil: move metadata parsing and packing from revlog (API) Gregory Szorc <gregory.szorc@gmail.com> parents: 39877 diff changeset	74 mtext = text[2:s]
3e896b51aa5d storageutil: move metadata parsing and packing from revlog (API) Gregory Szorc <gregory.szorc@gmail.com> parents: 39877 diff changeset	75 meta = {}
3e896b51aa5d storageutil: move metadata parsing and packing from revlog (API) Gregory Szorc <gregory.szorc@gmail.com> parents: 39877 diff changeset	76 for l in mtext.splitlines():
3e896b51aa5d storageutil: move metadata parsing and packing from revlog (API) Gregory Szorc <gregory.szorc@gmail.com> parents: 39877 diff changeset	77 k, v = l.split(b': ', 1)
3e896b51aa5d storageutil: move metadata parsing and packing from revlog (API) Gregory Szorc <gregory.szorc@gmail.com> parents: 39877 diff changeset	78 meta[k] = v
3e896b51aa5d storageutil: move metadata parsing and packing from revlog (API) Gregory Szorc <gregory.szorc@gmail.com> parents: 39877 diff changeset	79 return meta, s + 2
3e896b51aa5d storageutil: move metadata parsing and packing from revlog (API) Gregory Szorc <gregory.szorc@gmail.com> parents: 39877 diff changeset	80
43076 2372284d9457 formatting: blacken the codebase Augie Fackler <augie@google.com> parents: 42813 diff changeset	81
39878 3e896b51aa5d storageutil: move metadata parsing and packing from revlog (API) Gregory Szorc <gregory.szorc@gmail.com> parents: 39877 diff changeset	82 def packmeta(meta, text):
3e896b51aa5d storageutil: move metadata parsing and packing from revlog (API) Gregory Szorc <gregory.szorc@gmail.com> parents: 39877 diff changeset	83 """Add metadata to fulltext to produce revision text."""
3e896b51aa5d storageutil: move metadata parsing and packing from revlog (API) Gregory Szorc <gregory.szorc@gmail.com> parents: 39877 diff changeset	84 keys = sorted(meta)
3e896b51aa5d storageutil: move metadata parsing and packing from revlog (API) Gregory Szorc <gregory.szorc@gmail.com> parents: 39877 diff changeset	85 metatext = b''.join(b'%s: %s\n' % (k, meta[k]) for k in keys)
3e896b51aa5d storageutil: move metadata parsing and packing from revlog (API) Gregory Szorc <gregory.szorc@gmail.com> parents: 39877 diff changeset	86 return b'\x01\n%s\x01\n%s' % (metatext, text)
39879 d269ddbf54f0 storageutil: move _censoredtext() from revlog Gregory Szorc <gregory.szorc@gmail.com> parents: 39878 diff changeset	87
43076 2372284d9457 formatting: blacken the codebase Augie Fackler <augie@google.com> parents: 42813 diff changeset	88
39879 d269ddbf54f0 storageutil: move _censoredtext() from revlog Gregory Szorc <gregory.szorc@gmail.com> parents: 39878 diff changeset	89 def iscensoredtext(text):
d269ddbf54f0 storageutil: move _censoredtext() from revlog Gregory Szorc <gregory.szorc@gmail.com> parents: 39878 diff changeset	90 meta = parsemeta(text)[0]
d269ddbf54f0 storageutil: move _censoredtext() from revlog Gregory Szorc <gregory.szorc@gmail.com> parents: 39878 diff changeset	91 return meta and b'censored' in meta
39880 1b65fb4d43d6 storageutil: new function for extracting metadata-less content from text Gregory Szorc <gregory.szorc@gmail.com> parents: 39879 diff changeset	92
43076 2372284d9457 formatting: blacken the codebase Augie Fackler <augie@google.com> parents: 42813 diff changeset	93
39880 1b65fb4d43d6 storageutil: new function for extracting metadata-less content from text Gregory Szorc <gregory.szorc@gmail.com> parents: 39879 diff changeset	94 def filtermetadata(text):
1b65fb4d43d6 storageutil: new function for extracting metadata-less content from text Gregory Szorc <gregory.szorc@gmail.com> parents: 39879 diff changeset	95 """Extract just the revision data from source text.
1b65fb4d43d6 storageutil: new function for extracting metadata-less content from text Gregory Szorc <gregory.szorc@gmail.com> parents: 39879 diff changeset	96
1b65fb4d43d6 storageutil: new function for extracting metadata-less content from text Gregory Szorc <gregory.szorc@gmail.com> parents: 39879 diff changeset	97 Returns ``text`` unless it has a metadata header, in which case we return
1b65fb4d43d6 storageutil: new function for extracting metadata-less content from text Gregory Szorc <gregory.szorc@gmail.com> parents: 39879 diff changeset	98 a new buffer without hte metadata.
1b65fb4d43d6 storageutil: new function for extracting metadata-less content from text Gregory Szorc <gregory.szorc@gmail.com> parents: 39879 diff changeset	99 """
1b65fb4d43d6 storageutil: new function for extracting metadata-less content from text Gregory Szorc <gregory.szorc@gmail.com> parents: 39879 diff changeset	100 if not text.startswith(b'\x01\n'):
1b65fb4d43d6 storageutil: new function for extracting metadata-less content from text Gregory Szorc <gregory.szorc@gmail.com> parents: 39879 diff changeset	101 return text
1b65fb4d43d6 storageutil: new function for extracting metadata-less content from text Gregory Szorc <gregory.szorc@gmail.com> parents: 39879 diff changeset	102
1b65fb4d43d6 storageutil: new function for extracting metadata-less content from text Gregory Szorc <gregory.szorc@gmail.com> parents: 39879 diff changeset	103 offset = text.index(b'\x01\n', 2)
43076 2372284d9457 formatting: blacken the codebase Augie Fackler <augie@google.com> parents: 42813 diff changeset	104 return text[offset + 2 :]
2372284d9457 formatting: blacken the codebase Augie Fackler <augie@google.com> parents: 42813 diff changeset	105
39881 d63153611ed5 storageutil: extract revision number iteration Gregory Szorc <gregory.szorc@gmail.com> parents: 39880 diff changeset	106
40005 1d97a332c6d9 storageutil: extract copy metadata retrieval out of filelog Gregory Szorc <gregory.szorc@gmail.com> parents: 40004 diff changeset	107 def filerevisioncopied(store, node):
1d97a332c6d9 storageutil: extract copy metadata retrieval out of filelog Gregory Szorc <gregory.szorc@gmail.com> parents: 40004 diff changeset	108 """Resolve file revision copy metadata.
1d97a332c6d9 storageutil: extract copy metadata retrieval out of filelog Gregory Szorc <gregory.szorc@gmail.com> parents: 40004 diff changeset	109
1d97a332c6d9 storageutil: extract copy metadata retrieval out of filelog Gregory Szorc <gregory.szorc@gmail.com> parents: 40004 diff changeset	110 Returns ``False`` if the file has no copy metadata. Otherwise a
1d97a332c6d9 storageutil: extract copy metadata retrieval out of filelog Gregory Szorc <gregory.szorc@gmail.com> parents: 40004 diff changeset	111 2-tuple of the source filename and node.
1d97a332c6d9 storageutil: extract copy metadata retrieval out of filelog Gregory Szorc <gregory.szorc@gmail.com> parents: 40004 diff changeset	112 """
47012 d55b71393907 node: replace nullid and friends with nodeconstants class Joerg Sonnenberger <joerg@bec.de> parents: 46715 diff changeset	113 if store.parents(node)[0] != sha1nodeconstants.nullid:
48545 5026a0d37526 filelog: add a comment explaining a fast path in filerevisioncopied() Simon Sapin <simon.sapin@octobus.net> parents: 47341 diff changeset	114 # When creating a copy or move we set filelog parents to null,
5026a0d37526 filelog: add a comment explaining a fast path in filerevisioncopied() Simon Sapin <simon.sapin@octobus.net> parents: 47341 diff changeset	115 # because contents are probably unrelated and making a delta
5026a0d37526 filelog: add a comment explaining a fast path in filerevisioncopied() Simon Sapin <simon.sapin@octobus.net> parents: 47341 diff changeset	116 # would not be useful.
5026a0d37526 filelog: add a comment explaining a fast path in filerevisioncopied() Simon Sapin <simon.sapin@octobus.net> parents: 47341 diff changeset	117 # Conversely, if filelog p1 is non-null we know
5026a0d37526 filelog: add a comment explaining a fast path in filerevisioncopied() Simon Sapin <simon.sapin@octobus.net> parents: 47341 diff changeset	118 # there is no copy metadata.
5026a0d37526 filelog: add a comment explaining a fast path in filerevisioncopied() Simon Sapin <simon.sapin@octobus.net> parents: 47341 diff changeset	119 # In the presence of merges, this reasoning becomes invalid
5026a0d37526 filelog: add a comment explaining a fast path in filerevisioncopied() Simon Sapin <simon.sapin@octobus.net> parents: 47341 diff changeset	120 # if we reorder parents. See tests/test-issue6528.t.
40005 1d97a332c6d9 storageutil: extract copy metadata retrieval out of filelog Gregory Szorc <gregory.szorc@gmail.com> parents: 40004 diff changeset	121 return False
1d97a332c6d9 storageutil: extract copy metadata retrieval out of filelog Gregory Szorc <gregory.szorc@gmail.com> parents: 40004 diff changeset	122
1d97a332c6d9 storageutil: extract copy metadata retrieval out of filelog Gregory Szorc <gregory.szorc@gmail.com> parents: 40004 diff changeset	123 meta = parsemeta(store.revision(node))[0]
1d97a332c6d9 storageutil: extract copy metadata retrieval out of filelog Gregory Szorc <gregory.szorc@gmail.com> parents: 40004 diff changeset	124
1d97a332c6d9 storageutil: extract copy metadata retrieval out of filelog Gregory Szorc <gregory.szorc@gmail.com> parents: 40004 diff changeset	125 # copy and copyrev occur in pairs. In rare cases due to old bugs,
1d97a332c6d9 storageutil: extract copy metadata retrieval out of filelog Gregory Szorc <gregory.szorc@gmail.com> parents: 40004 diff changeset	126 # one can occur without the other. So ensure both are present to flag
1d97a332c6d9 storageutil: extract copy metadata retrieval out of filelog Gregory Szorc <gregory.szorc@gmail.com> parents: 40004 diff changeset	127 # as a copy.
1d97a332c6d9 storageutil: extract copy metadata retrieval out of filelog Gregory Szorc <gregory.szorc@gmail.com> parents: 40004 diff changeset	128 if meta and b'copy' in meta and b'copyrev' in meta:
1d97a332c6d9 storageutil: extract copy metadata retrieval out of filelog Gregory Szorc <gregory.szorc@gmail.com> parents: 40004 diff changeset	129 return meta[b'copy'], bin(meta[b'copyrev'])
1d97a332c6d9 storageutil: extract copy metadata retrieval out of filelog Gregory Szorc <gregory.szorc@gmail.com> parents: 40004 diff changeset	130
1d97a332c6d9 storageutil: extract copy metadata retrieval out of filelog Gregory Szorc <gregory.szorc@gmail.com> parents: 40004 diff changeset	131 return False
1d97a332c6d9 storageutil: extract copy metadata retrieval out of filelog Gregory Szorc <gregory.szorc@gmail.com> parents: 40004 diff changeset	132
43076 2372284d9457 formatting: blacken the codebase Augie Fackler <augie@google.com> parents: 42813 diff changeset	133
40007 1470183068b8 storageutil: invert logic of file data comparison Gregory Szorc <gregory.szorc@gmail.com> parents: 40006 diff changeset	134 def filedataequivalent(store, node, filedata):
1470183068b8 storageutil: invert logic of file data comparison Gregory Szorc <gregory.szorc@gmail.com> parents: 40006 diff changeset	135 """Determines whether file data is equivalent to a stored node.
1470183068b8 storageutil: invert logic of file data comparison Gregory Szorc <gregory.szorc@gmail.com> parents: 40006 diff changeset	136
1470183068b8 storageutil: invert logic of file data comparison Gregory Szorc <gregory.szorc@gmail.com> parents: 40006 diff changeset	137 Returns True if the passed file data would hash to the same value
1470183068b8 storageutil: invert logic of file data comparison Gregory Szorc <gregory.szorc@gmail.com> parents: 40006 diff changeset	138 as a stored revision and False otherwise.
1470183068b8 storageutil: invert logic of file data comparison Gregory Szorc <gregory.szorc@gmail.com> parents: 40006 diff changeset	139
1470183068b8 storageutil: invert logic of file data comparison Gregory Szorc <gregory.szorc@gmail.com> parents: 40006 diff changeset	140 When a stored revision is censored, filedata must be empty to have
1470183068b8 storageutil: invert logic of file data comparison Gregory Szorc <gregory.szorc@gmail.com> parents: 40006 diff changeset	141 equivalence.
1470183068b8 storageutil: invert logic of file data comparison Gregory Szorc <gregory.szorc@gmail.com> parents: 40006 diff changeset	142
1470183068b8 storageutil: invert logic of file data comparison Gregory Szorc <gregory.szorc@gmail.com> parents: 40006 diff changeset	143 When a stored revision has copy metadata, it is ignored as part
1470183068b8 storageutil: invert logic of file data comparison Gregory Szorc <gregory.szorc@gmail.com> parents: 40006 diff changeset	144 of the compare.
1470183068b8 storageutil: invert logic of file data comparison Gregory Szorc <gregory.szorc@gmail.com> parents: 40006 diff changeset	145 """
40006 422beffd71ba storageutil: extract filelog.cmp() to a standalone function Gregory Szorc <gregory.szorc@gmail.com> parents: 40005 diff changeset	146
422beffd71ba storageutil: extract filelog.cmp() to a standalone function Gregory Szorc <gregory.szorc@gmail.com> parents: 40005 diff changeset	147 if filedata.startswith(b'\x01\n'):
422beffd71ba storageutil: extract filelog.cmp() to a standalone function Gregory Szorc <gregory.szorc@gmail.com> parents: 40005 diff changeset	148 revisiontext = b'\x01\n\x01\n' + filedata
422beffd71ba storageutil: extract filelog.cmp() to a standalone function Gregory Szorc <gregory.szorc@gmail.com> parents: 40005 diff changeset	149 else:
422beffd71ba storageutil: extract filelog.cmp() to a standalone function Gregory Szorc <gregory.szorc@gmail.com> parents: 40005 diff changeset	150 revisiontext = filedata
422beffd71ba storageutil: extract filelog.cmp() to a standalone function Gregory Szorc <gregory.szorc@gmail.com> parents: 40005 diff changeset	151
422beffd71ba storageutil: extract filelog.cmp() to a standalone function Gregory Szorc <gregory.szorc@gmail.com> parents: 40005 diff changeset	152 p1, p2 = store.parents(node)
422beffd71ba storageutil: extract filelog.cmp() to a standalone function Gregory Szorc <gregory.szorc@gmail.com> parents: 40005 diff changeset	153
422beffd71ba storageutil: extract filelog.cmp() to a standalone function Gregory Szorc <gregory.szorc@gmail.com> parents: 40005 diff changeset	154 computednode = hashrevisionsha1(revisiontext, p1, p2)
422beffd71ba storageutil: extract filelog.cmp() to a standalone function Gregory Szorc <gregory.szorc@gmail.com> parents: 40005 diff changeset	155
422beffd71ba storageutil: extract filelog.cmp() to a standalone function Gregory Szorc <gregory.szorc@gmail.com> parents: 40005 diff changeset	156 if computednode == node:
40007 1470183068b8 storageutil: invert logic of file data comparison Gregory Szorc <gregory.szorc@gmail.com> parents: 40006 diff changeset	157 return True
40006 422beffd71ba storageutil: extract filelog.cmp() to a standalone function Gregory Szorc <gregory.szorc@gmail.com> parents: 40005 diff changeset	158
422beffd71ba storageutil: extract filelog.cmp() to a standalone function Gregory Szorc <gregory.szorc@gmail.com> parents: 40005 diff changeset	159 # Censored files compare against the empty file.
422beffd71ba storageutil: extract filelog.cmp() to a standalone function Gregory Szorc <gregory.szorc@gmail.com> parents: 40005 diff changeset	160 if store.iscensored(store.rev(node)):
40007 1470183068b8 storageutil: invert logic of file data comparison Gregory Szorc <gregory.szorc@gmail.com> parents: 40006 diff changeset	161 return filedata == b''
40006 422beffd71ba storageutil: extract filelog.cmp() to a standalone function Gregory Szorc <gregory.szorc@gmail.com> parents: 40005 diff changeset	162
422beffd71ba storageutil: extract filelog.cmp() to a standalone function Gregory Szorc <gregory.szorc@gmail.com> parents: 40005 diff changeset	163 # Renaming a file produces a different hash, even if the data
422beffd71ba storageutil: extract filelog.cmp() to a standalone function Gregory Szorc <gregory.szorc@gmail.com> parents: 40005 diff changeset	164 # remains unchanged. Check if that's the case.
422beffd71ba storageutil: extract filelog.cmp() to a standalone function Gregory Szorc <gregory.szorc@gmail.com> parents: 40005 diff changeset	165 if store.renamed(node):
40007 1470183068b8 storageutil: invert logic of file data comparison Gregory Szorc <gregory.szorc@gmail.com> parents: 40006 diff changeset	166 return store.read(node) == filedata
40006 422beffd71ba storageutil: extract filelog.cmp() to a standalone function Gregory Szorc <gregory.szorc@gmail.com> parents: 40005 diff changeset	167
40007 1470183068b8 storageutil: invert logic of file data comparison Gregory Szorc <gregory.szorc@gmail.com> parents: 40006 diff changeset	168 return False
40006 422beffd71ba storageutil: extract filelog.cmp() to a standalone function Gregory Szorc <gregory.szorc@gmail.com> parents: 40005 diff changeset	169
43076 2372284d9457 formatting: blacken the codebase Augie Fackler <augie@google.com> parents: 42813 diff changeset	170
39881 d63153611ed5 storageutil: extract revision number iteration Gregory Szorc <gregory.szorc@gmail.com> parents: 39880 diff changeset	171 def iterrevs(storelen, start=0, stop=None):
d63153611ed5 storageutil: extract revision number iteration Gregory Szorc <gregory.szorc@gmail.com> parents: 39880 diff changeset	172 """Iterate over revision numbers in a store."""
d63153611ed5 storageutil: extract revision number iteration Gregory Szorc <gregory.szorc@gmail.com> parents: 39880 diff changeset	173 step = 1
d63153611ed5 storageutil: extract revision number iteration Gregory Szorc <gregory.szorc@gmail.com> parents: 39880 diff changeset	174
d63153611ed5 storageutil: extract revision number iteration Gregory Szorc <gregory.szorc@gmail.com> parents: 39880 diff changeset	175 if stop is not None:
d63153611ed5 storageutil: extract revision number iteration Gregory Szorc <gregory.szorc@gmail.com> parents: 39880 diff changeset	176 if start > stop:
d63153611ed5 storageutil: extract revision number iteration Gregory Szorc <gregory.szorc@gmail.com> parents: 39880 diff changeset	177 step = -1
d63153611ed5 storageutil: extract revision number iteration Gregory Szorc <gregory.szorc@gmail.com> parents: 39880 diff changeset	178 stop += step
d63153611ed5 storageutil: extract revision number iteration Gregory Szorc <gregory.szorc@gmail.com> parents: 39880 diff changeset	179 if stop > storelen:
d63153611ed5 storageutil: extract revision number iteration Gregory Szorc <gregory.szorc@gmail.com> parents: 39880 diff changeset	180 stop = storelen
d63153611ed5 storageutil: extract revision number iteration Gregory Szorc <gregory.szorc@gmail.com> parents: 39880 diff changeset	181 else:
d63153611ed5 storageutil: extract revision number iteration Gregory Szorc <gregory.szorc@gmail.com> parents: 39880 diff changeset	182 stop = storelen
d63153611ed5 storageutil: extract revision number iteration Gregory Szorc <gregory.szorc@gmail.com> parents: 39880 diff changeset	183
49284 d44e3c45f0e4 py3: replace `pycompat.xrange` by `range` Manuel Jacob <me@manueljacob.de> parents: 48875 diff changeset	184 return range(start, stop, step)
40002 0e8836be9541 storageutil: implement file identifier resolution method (BC) Gregory Szorc <gregory.szorc@gmail.com> parents: 39881 diff changeset	185
43076 2372284d9457 formatting: blacken the codebase Augie Fackler <augie@google.com> parents: 42813 diff changeset	186
40002 0e8836be9541 storageutil: implement file identifier resolution method (BC) Gregory Szorc <gregory.szorc@gmail.com> parents: 39881 diff changeset	187 def fileidlookup(store, fileid, identifier):
0e8836be9541 storageutil: implement file identifier resolution method (BC) Gregory Szorc <gregory.szorc@gmail.com> parents: 39881 diff changeset	188 """Resolve the file node for a value.
0e8836be9541 storageutil: implement file identifier resolution method (BC) Gregory Szorc <gregory.szorc@gmail.com> parents: 39881 diff changeset	189
0e8836be9541 storageutil: implement file identifier resolution method (BC) Gregory Szorc <gregory.szorc@gmail.com> parents: 39881 diff changeset	190 ``store`` is an object implementing the ``ifileindex`` interface.
0e8836be9541 storageutil: implement file identifier resolution method (BC) Gregory Szorc <gregory.szorc@gmail.com> parents: 39881 diff changeset	191
0e8836be9541 storageutil: implement file identifier resolution method (BC) Gregory Szorc <gregory.szorc@gmail.com> parents: 39881 diff changeset	192 ``fileid`` can be:
0e8836be9541 storageutil: implement file identifier resolution method (BC) Gregory Szorc <gregory.szorc@gmail.com> parents: 39881 diff changeset	193
49967 5698c5eee12b storageutil: match node length with repository Joerg Sonnenberger <joerg@bec.de> parents: 49845 diff changeset	194 * A binary node of appropiate size (e.g. 20/32 Bytes).
40002 0e8836be9541 storageutil: implement file identifier resolution method (BC) Gregory Szorc <gregory.szorc@gmail.com> parents: 39881 diff changeset	195 * An integer revision number
49967 5698c5eee12b storageutil: match node length with repository Joerg Sonnenberger <joerg@bec.de> parents: 49845 diff changeset	196 * A hex node of appropiate size (e.g. 40/64 Bytes).
40002 0e8836be9541 storageutil: implement file identifier resolution method (BC) Gregory Szorc <gregory.szorc@gmail.com> parents: 39881 diff changeset	197 * A bytes that can be parsed as an integer representing a revision number.
0e8836be9541 storageutil: implement file identifier resolution method (BC) Gregory Szorc <gregory.szorc@gmail.com> parents: 39881 diff changeset	198
0e8836be9541 storageutil: implement file identifier resolution method (BC) Gregory Szorc <gregory.szorc@gmail.com> parents: 39881 diff changeset	199 ``identifier`` is used to populate ``error.LookupError`` with an identifier
0e8836be9541 storageutil: implement file identifier resolution method (BC) Gregory Szorc <gregory.szorc@gmail.com> parents: 39881 diff changeset	200 for the store.
0e8836be9541 storageutil: implement file identifier resolution method (BC) Gregory Szorc <gregory.szorc@gmail.com> parents: 39881 diff changeset	201
0e8836be9541 storageutil: implement file identifier resolution method (BC) Gregory Szorc <gregory.szorc@gmail.com> parents: 39881 diff changeset	202 Raises ``error.LookupError`` on failure.
0e8836be9541 storageutil: implement file identifier resolution method (BC) Gregory Szorc <gregory.szorc@gmail.com> parents: 39881 diff changeset	203 """
0e8836be9541 storageutil: implement file identifier resolution method (BC) Gregory Szorc <gregory.szorc@gmail.com> parents: 39881 diff changeset	204 if isinstance(fileid, int):
40003 ad8389ecd3f5 storageutil: consistently raise LookupError (API) Gregory Szorc <gregory.szorc@gmail.com> parents: 40002 diff changeset	205 try:
ad8389ecd3f5 storageutil: consistently raise LookupError (API) Gregory Szorc <gregory.szorc@gmail.com> parents: 40002 diff changeset	206 return store.node(fileid)
ad8389ecd3f5 storageutil: consistently raise LookupError (API) Gregory Szorc <gregory.szorc@gmail.com> parents: 40002 diff changeset	207 except IndexError:
43076 2372284d9457 formatting: blacken the codebase Augie Fackler <augie@google.com> parents: 42813 diff changeset	208 raise error.LookupError(
43077 687b865b95ad formatting: byteify all mercurial/ and hgext/ string literals Augie Fackler <augie@google.com> parents: 43076 diff changeset	209 b'%d' % fileid, identifier, _(b'no match found')
43076 2372284d9457 formatting: blacken the codebase Augie Fackler <augie@google.com> parents: 42813 diff changeset	210 )
40002 0e8836be9541 storageutil: implement file identifier resolution method (BC) Gregory Szorc <gregory.szorc@gmail.com> parents: 39881 diff changeset	211
49967 5698c5eee12b storageutil: match node length with repository Joerg Sonnenberger <joerg@bec.de> parents: 49845 diff changeset	212 if len(fileid) == len(store.nullid):
40002 0e8836be9541 storageutil: implement file identifier resolution method (BC) Gregory Szorc <gregory.szorc@gmail.com> parents: 39881 diff changeset	213 try:
0e8836be9541 storageutil: implement file identifier resolution method (BC) Gregory Szorc <gregory.szorc@gmail.com> parents: 39881 diff changeset	214 store.rev(fileid)
0e8836be9541 storageutil: implement file identifier resolution method (BC) Gregory Szorc <gregory.szorc@gmail.com> parents: 39881 diff changeset	215 return fileid
0e8836be9541 storageutil: implement file identifier resolution method (BC) Gregory Szorc <gregory.szorc@gmail.com> parents: 39881 diff changeset	216 except error.LookupError:
0e8836be9541 storageutil: implement file identifier resolution method (BC) Gregory Szorc <gregory.szorc@gmail.com> parents: 39881 diff changeset	217 pass
0e8836be9541 storageutil: implement file identifier resolution method (BC) Gregory Szorc <gregory.szorc@gmail.com> parents: 39881 diff changeset	218
49967 5698c5eee12b storageutil: match node length with repository Joerg Sonnenberger <joerg@bec.de> parents: 49845 diff changeset	219 if len(fileid) == 2 * len(store.nullid):
40002 0e8836be9541 storageutil: implement file identifier resolution method (BC) Gregory Szorc <gregory.szorc@gmail.com> parents: 39881 diff changeset	220 try:
0e8836be9541 storageutil: implement file identifier resolution method (BC) Gregory Szorc <gregory.szorc@gmail.com> parents: 39881 diff changeset	221 rawnode = bin(fileid)
0e8836be9541 storageutil: implement file identifier resolution method (BC) Gregory Szorc <gregory.szorc@gmail.com> parents: 39881 diff changeset	222 store.rev(rawnode)
0e8836be9541 storageutil: implement file identifier resolution method (BC) Gregory Szorc <gregory.szorc@gmail.com> parents: 39881 diff changeset	223 return rawnode
0e8836be9541 storageutil: implement file identifier resolution method (BC) Gregory Szorc <gregory.szorc@gmail.com> parents: 39881 diff changeset	224 except TypeError:
0e8836be9541 storageutil: implement file identifier resolution method (BC) Gregory Szorc <gregory.szorc@gmail.com> parents: 39881 diff changeset	225 pass
0e8836be9541 storageutil: implement file identifier resolution method (BC) Gregory Szorc <gregory.szorc@gmail.com> parents: 39881 diff changeset	226
0e8836be9541 storageutil: implement file identifier resolution method (BC) Gregory Szorc <gregory.szorc@gmail.com> parents: 39881 diff changeset	227 try:
0e8836be9541 storageutil: implement file identifier resolution method (BC) Gregory Szorc <gregory.szorc@gmail.com> parents: 39881 diff changeset	228 rev = int(fileid)
0e8836be9541 storageutil: implement file identifier resolution method (BC) Gregory Szorc <gregory.szorc@gmail.com> parents: 39881 diff changeset	229
0e8836be9541 storageutil: implement file identifier resolution method (BC) Gregory Szorc <gregory.szorc@gmail.com> parents: 39881 diff changeset	230 if b'%d' % rev != fileid:
0e8836be9541 storageutil: implement file identifier resolution method (BC) Gregory Szorc <gregory.szorc@gmail.com> parents: 39881 diff changeset	231 raise ValueError
0e8836be9541 storageutil: implement file identifier resolution method (BC) Gregory Szorc <gregory.szorc@gmail.com> parents: 39881 diff changeset	232
0e8836be9541 storageutil: implement file identifier resolution method (BC) Gregory Szorc <gregory.szorc@gmail.com> parents: 39881 diff changeset	233 try:
0e8836be9541 storageutil: implement file identifier resolution method (BC) Gregory Szorc <gregory.szorc@gmail.com> parents: 39881 diff changeset	234 return store.node(rev)
0e8836be9541 storageutil: implement file identifier resolution method (BC) Gregory Szorc <gregory.szorc@gmail.com> parents: 39881 diff changeset	235 except (IndexError, TypeError):
0e8836be9541 storageutil: implement file identifier resolution method (BC) Gregory Szorc <gregory.szorc@gmail.com> parents: 39881 diff changeset	236 pass
0e8836be9541 storageutil: implement file identifier resolution method (BC) Gregory Szorc <gregory.szorc@gmail.com> parents: 39881 diff changeset	237 except (ValueError, OverflowError):
0e8836be9541 storageutil: implement file identifier resolution method (BC) Gregory Szorc <gregory.szorc@gmail.com> parents: 39881 diff changeset	238 pass
0e8836be9541 storageutil: implement file identifier resolution method (BC) Gregory Szorc <gregory.szorc@gmail.com> parents: 39881 diff changeset	239
43077 687b865b95ad formatting: byteify all mercurial/ and hgext/ string literals Augie Fackler <augie@google.com> parents: 43076 diff changeset	240 raise error.LookupError(fileid, identifier, _(b'no match found'))
40004 fa3dc85a747e storageutil: extract functionality for resolving strip revisions Gregory Szorc <gregory.szorc@gmail.com> parents: 40003 diff changeset	241
43076 2372284d9457 formatting: blacken the codebase Augie Fackler <augie@google.com> parents: 42813 diff changeset	242
40004 fa3dc85a747e storageutil: extract functionality for resolving strip revisions Gregory Szorc <gregory.szorc@gmail.com> parents: 40003 diff changeset	243 def resolvestripinfo(minlinkrev, tiprev, headrevs, linkrevfn, parentrevsfn):
fa3dc85a747e storageutil: extract functionality for resolving strip revisions Gregory Szorc <gregory.szorc@gmail.com> parents: 40003 diff changeset	244 """Resolve information needed to strip revisions.
fa3dc85a747e storageutil: extract functionality for resolving strip revisions Gregory Szorc <gregory.szorc@gmail.com> parents: 40003 diff changeset	245
fa3dc85a747e storageutil: extract functionality for resolving strip revisions Gregory Szorc <gregory.szorc@gmail.com> parents: 40003 diff changeset	246 Finds the minimum revision number that must be stripped in order to
fa3dc85a747e storageutil: extract functionality for resolving strip revisions Gregory Szorc <gregory.szorc@gmail.com> parents: 40003 diff changeset	247 strip ``minlinkrev``.
fa3dc85a747e storageutil: extract functionality for resolving strip revisions Gregory Szorc <gregory.szorc@gmail.com> parents: 40003 diff changeset	248
fa3dc85a747e storageutil: extract functionality for resolving strip revisions Gregory Szorc <gregory.szorc@gmail.com> parents: 40003 diff changeset	249 Returns a 2-tuple of the minimum revision number to do that and a set
fa3dc85a747e storageutil: extract functionality for resolving strip revisions Gregory Szorc <gregory.szorc@gmail.com> parents: 40003 diff changeset	250 of all revision numbers that have linkrevs that would be broken
fa3dc85a747e storageutil: extract functionality for resolving strip revisions Gregory Szorc <gregory.szorc@gmail.com> parents: 40003 diff changeset	251 by that strip.
fa3dc85a747e storageutil: extract functionality for resolving strip revisions Gregory Szorc <gregory.szorc@gmail.com> parents: 40003 diff changeset	252
fa3dc85a747e storageutil: extract functionality for resolving strip revisions Gregory Szorc <gregory.szorc@gmail.com> parents: 40003 diff changeset	253 ``tiprev`` is the current tip-most revision. It is ``len(store) - 1``.
fa3dc85a747e storageutil: extract functionality for resolving strip revisions Gregory Szorc <gregory.szorc@gmail.com> parents: 40003 diff changeset	254 ``headrevs`` is an iterable of head revisions.
fa3dc85a747e storageutil: extract functionality for resolving strip revisions Gregory Szorc <gregory.szorc@gmail.com> parents: 40003 diff changeset	255 ``linkrevfn`` is a callable that receives a revision and returns a linked
fa3dc85a747e storageutil: extract functionality for resolving strip revisions Gregory Szorc <gregory.szorc@gmail.com> parents: 40003 diff changeset	256 revision.
fa3dc85a747e storageutil: extract functionality for resolving strip revisions Gregory Szorc <gregory.szorc@gmail.com> parents: 40003 diff changeset	257 ``parentrevsfn`` is a callable that receives a revision number and returns
fa3dc85a747e storageutil: extract functionality for resolving strip revisions Gregory Szorc <gregory.szorc@gmail.com> parents: 40003 diff changeset	258 an iterable of its parent revision numbers.
fa3dc85a747e storageutil: extract functionality for resolving strip revisions Gregory Szorc <gregory.szorc@gmail.com> parents: 40003 diff changeset	259 """
fa3dc85a747e storageutil: extract functionality for resolving strip revisions Gregory Szorc <gregory.szorc@gmail.com> parents: 40003 diff changeset	260 brokenrevs = set()
fa3dc85a747e storageutil: extract functionality for resolving strip revisions Gregory Szorc <gregory.szorc@gmail.com> parents: 40003 diff changeset	261 strippoint = tiprev + 1
fa3dc85a747e storageutil: extract functionality for resolving strip revisions Gregory Szorc <gregory.szorc@gmail.com> parents: 40003 diff changeset	262
fa3dc85a747e storageutil: extract functionality for resolving strip revisions Gregory Szorc <gregory.szorc@gmail.com> parents: 40003 diff changeset	263 heads = {}
fa3dc85a747e storageutil: extract functionality for resolving strip revisions Gregory Szorc <gregory.szorc@gmail.com> parents: 40003 diff changeset	264 futurelargelinkrevs = set()
fa3dc85a747e storageutil: extract functionality for resolving strip revisions Gregory Szorc <gregory.szorc@gmail.com> parents: 40003 diff changeset	265 for head in headrevs:
fa3dc85a747e storageutil: extract functionality for resolving strip revisions Gregory Szorc <gregory.szorc@gmail.com> parents: 40003 diff changeset	266 headlinkrev = linkrevfn(head)
fa3dc85a747e storageutil: extract functionality for resolving strip revisions Gregory Szorc <gregory.szorc@gmail.com> parents: 40003 diff changeset	267 heads[head] = headlinkrev
fa3dc85a747e storageutil: extract functionality for resolving strip revisions Gregory Szorc <gregory.szorc@gmail.com> parents: 40003 diff changeset	268 if headlinkrev >= minlinkrev:
fa3dc85a747e storageutil: extract functionality for resolving strip revisions Gregory Szorc <gregory.szorc@gmail.com> parents: 40003 diff changeset	269 futurelargelinkrevs.add(headlinkrev)
fa3dc85a747e storageutil: extract functionality for resolving strip revisions Gregory Szorc <gregory.szorc@gmail.com> parents: 40003 diff changeset	270
fa3dc85a747e storageutil: extract functionality for resolving strip revisions Gregory Szorc <gregory.szorc@gmail.com> parents: 40003 diff changeset	271 # This algorithm involves walking down the rev graph, starting at the
fa3dc85a747e storageutil: extract functionality for resolving strip revisions Gregory Szorc <gregory.szorc@gmail.com> parents: 40003 diff changeset	272 # heads. Since the revs are topologically sorted according to linkrev,
fa3dc85a747e storageutil: extract functionality for resolving strip revisions Gregory Szorc <gregory.szorc@gmail.com> parents: 40003 diff changeset	273 # once all head linkrevs are below the minlink, we know there are
fa3dc85a747e storageutil: extract functionality for resolving strip revisions Gregory Szorc <gregory.szorc@gmail.com> parents: 40003 diff changeset	274 # no more revs that could have a linkrev greater than minlink.
fa3dc85a747e storageutil: extract functionality for resolving strip revisions Gregory Szorc <gregory.szorc@gmail.com> parents: 40003 diff changeset	275 # So we can stop walking.
fa3dc85a747e storageutil: extract functionality for resolving strip revisions Gregory Szorc <gregory.szorc@gmail.com> parents: 40003 diff changeset	276 while futurelargelinkrevs:
fa3dc85a747e storageutil: extract functionality for resolving strip revisions Gregory Szorc <gregory.szorc@gmail.com> parents: 40003 diff changeset	277 strippoint -= 1
fa3dc85a747e storageutil: extract functionality for resolving strip revisions Gregory Szorc <gregory.szorc@gmail.com> parents: 40003 diff changeset	278 linkrev = heads.pop(strippoint)
fa3dc85a747e storageutil: extract functionality for resolving strip revisions Gregory Szorc <gregory.szorc@gmail.com> parents: 40003 diff changeset	279
fa3dc85a747e storageutil: extract functionality for resolving strip revisions Gregory Szorc <gregory.szorc@gmail.com> parents: 40003 diff changeset	280 if linkrev < minlinkrev:
fa3dc85a747e storageutil: extract functionality for resolving strip revisions Gregory Szorc <gregory.szorc@gmail.com> parents: 40003 diff changeset	281 brokenrevs.add(strippoint)
fa3dc85a747e storageutil: extract functionality for resolving strip revisions Gregory Szorc <gregory.szorc@gmail.com> parents: 40003 diff changeset	282 else:
fa3dc85a747e storageutil: extract functionality for resolving strip revisions Gregory Szorc <gregory.szorc@gmail.com> parents: 40003 diff changeset	283 futurelargelinkrevs.remove(linkrev)
fa3dc85a747e storageutil: extract functionality for resolving strip revisions Gregory Szorc <gregory.szorc@gmail.com> parents: 40003 diff changeset	284
fa3dc85a747e storageutil: extract functionality for resolving strip revisions Gregory Szorc <gregory.szorc@gmail.com> parents: 40003 diff changeset	285 for p in parentrevsfn(strippoint):
fa3dc85a747e storageutil: extract functionality for resolving strip revisions Gregory Szorc <gregory.szorc@gmail.com> parents: 40003 diff changeset	286 if p != nullrev:
fa3dc85a747e storageutil: extract functionality for resolving strip revisions Gregory Szorc <gregory.szorc@gmail.com> parents: 40003 diff changeset	287 plinkrev = linkrevfn(p)
fa3dc85a747e storageutil: extract functionality for resolving strip revisions Gregory Szorc <gregory.szorc@gmail.com> parents: 40003 diff changeset	288 heads[p] = plinkrev
fa3dc85a747e storageutil: extract functionality for resolving strip revisions Gregory Szorc <gregory.szorc@gmail.com> parents: 40003 diff changeset	289 if plinkrev >= minlinkrev:
fa3dc85a747e storageutil: extract functionality for resolving strip revisions Gregory Szorc <gregory.szorc@gmail.com> parents: 40003 diff changeset	290 futurelargelinkrevs.add(plinkrev)
fa3dc85a747e storageutil: extract functionality for resolving strip revisions Gregory Szorc <gregory.szorc@gmail.com> parents: 40003 diff changeset	291
fa3dc85a747e storageutil: extract functionality for resolving strip revisions Gregory Szorc <gregory.szorc@gmail.com> parents: 40003 diff changeset	292 return strippoint, brokenrevs
40008 842ffcf1d42f storageutil: extract most of emitrevisions() to standalone function Gregory Szorc <gregory.szorc@gmail.com> parents: 40007 diff changeset	293
43076 2372284d9457 formatting: blacken the codebase Augie Fackler <augie@google.com> parents: 42813 diff changeset	294
2372284d9457 formatting: blacken the codebase Augie Fackler <augie@google.com> parents: 42813 diff changeset	295 def emitrevisions(
2372284d9457 formatting: blacken the codebase Augie Fackler <augie@google.com> parents: 42813 diff changeset	296 store,
2372284d9457 formatting: blacken the codebase Augie Fackler <augie@google.com> parents: 42813 diff changeset	297 nodes,
2372284d9457 formatting: blacken the codebase Augie Fackler <augie@google.com> parents: 42813 diff changeset	298 nodesorder,
2372284d9457 formatting: blacken the codebase Augie Fackler <augie@google.com> parents: 42813 diff changeset	299 resultcls,
2372284d9457 formatting: blacken the codebase Augie Fackler <augie@google.com> parents: 42813 diff changeset	300 deltaparentfn=None,
2372284d9457 formatting: blacken the codebase Augie Fackler <augie@google.com> parents: 42813 diff changeset	301 candeltafn=None,
2372284d9457 formatting: blacken the codebase Augie Fackler <augie@google.com> parents: 42813 diff changeset	302 rawsizefn=None,
2372284d9457 formatting: blacken the codebase Augie Fackler <augie@google.com> parents: 42813 diff changeset	303 revdifffn=None,
2372284d9457 formatting: blacken the codebase Augie Fackler <augie@google.com> parents: 42813 diff changeset	304 flagsfn=None,
2372284d9457 formatting: blacken the codebase Augie Fackler <augie@google.com> parents: 42813 diff changeset	305 deltamode=repository.CG_DELTAMODE_STD,
2372284d9457 formatting: blacken the codebase Augie Fackler <augie@google.com> parents: 42813 diff changeset	306 revisiondata=False,
2372284d9457 formatting: blacken the codebase Augie Fackler <augie@google.com> parents: 42813 diff changeset	307 assumehaveparentrevisions=False,
46715 45f0d5297698 changegroupv4: add sidedata helpers Rapha?l Gom?s <rgomes@octobus.net> parents: 46712 diff changeset	308 sidedata_helpers=None,
49609 9cac281eb9c0 debug: add an option to display statistic about a bundling operation Pierre-Yves David <pierre-yves.david@octobus.net> parents: 49284 diff changeset	309 debug_info=None,
43076 2372284d9457 formatting: blacken the codebase Augie Fackler <augie@google.com> parents: 42813 diff changeset	310 ):
40008 842ffcf1d42f storageutil: extract most of emitrevisions() to standalone function Gregory Szorc <gregory.szorc@gmail.com> parents: 40007 diff changeset	311 """Generic implementation of ifiledata.emitrevisions().
842ffcf1d42f storageutil: extract most of emitrevisions() to standalone function Gregory Szorc <gregory.szorc@gmail.com> parents: 40007 diff changeset	312
842ffcf1d42f storageutil: extract most of emitrevisions() to standalone function Gregory Szorc <gregory.szorc@gmail.com> parents: 40007 diff changeset	313 Emitting revision data is subtly complex. This function attempts to
842ffcf1d42f storageutil: extract most of emitrevisions() to standalone function Gregory Szorc <gregory.szorc@gmail.com> parents: 40007 diff changeset	314 encapsulate all the logic for doing so in a backend-agnostic way.
842ffcf1d42f storageutil: extract most of emitrevisions() to standalone function Gregory Szorc <gregory.szorc@gmail.com> parents: 40007 diff changeset	315
842ffcf1d42f storageutil: extract most of emitrevisions() to standalone function Gregory Szorc <gregory.szorc@gmail.com> parents: 40007 diff changeset	316 ``store``
842ffcf1d42f storageutil: extract most of emitrevisions() to standalone function Gregory Szorc <gregory.szorc@gmail.com> parents: 40007 diff changeset	317 Object conforming to ``ifilestorage`` interface.
842ffcf1d42f storageutil: extract most of emitrevisions() to standalone function Gregory Szorc <gregory.szorc@gmail.com> parents: 40007 diff changeset	318
40010 f5d819d84461 storageutil: pass nodes into emitrevisions() Gregory Szorc <gregory.szorc@gmail.com> parents: 40009 diff changeset	319 ``nodes``
f5d819d84461 storageutil: pass nodes into emitrevisions() Gregory Szorc <gregory.szorc@gmail.com> parents: 40009 diff changeset	320 List of revision nodes whose data to emit.
40008 842ffcf1d42f storageutil: extract most of emitrevisions() to standalone function Gregory Szorc <gregory.szorc@gmail.com> parents: 40007 diff changeset	321
842ffcf1d42f storageutil: extract most of emitrevisions() to standalone function Gregory Szorc <gregory.szorc@gmail.com> parents: 40007 diff changeset	322 ``resultcls``
842ffcf1d42f storageutil: extract most of emitrevisions() to standalone function Gregory Szorc <gregory.szorc@gmail.com> parents: 40007 diff changeset	323 A type implementing the ``irevisiondelta`` interface that will be
842ffcf1d42f storageutil: extract most of emitrevisions() to standalone function Gregory Szorc <gregory.szorc@gmail.com> parents: 40007 diff changeset	324 constructed and returned.
842ffcf1d42f storageutil: extract most of emitrevisions() to standalone function Gregory Szorc <gregory.szorc@gmail.com> parents: 40007 diff changeset	325
40009 631c6f5058b9 storageutil: make all callables optional Gregory Szorc <gregory.szorc@gmail.com> parents: 40008 diff changeset	326 ``deltaparentfn`` (optional)
40008 842ffcf1d42f storageutil: extract most of emitrevisions() to standalone function Gregory Szorc <gregory.szorc@gmail.com> parents: 40007 diff changeset	327 Callable receiving a revision number and returning the revision number
842ffcf1d42f storageutil: extract most of emitrevisions() to standalone function Gregory Szorc <gregory.szorc@gmail.com> parents: 40007 diff changeset	328 of a revision that the internal delta is stored against. This delta
842ffcf1d42f storageutil: extract most of emitrevisions() to standalone function Gregory Szorc <gregory.szorc@gmail.com> parents: 40007 diff changeset	329 will be preferred over computing a new arbitrary delta.
842ffcf1d42f storageutil: extract most of emitrevisions() to standalone function Gregory Szorc <gregory.szorc@gmail.com> parents: 40007 diff changeset	330
40009 631c6f5058b9 storageutil: make all callables optional Gregory Szorc <gregory.szorc@gmail.com> parents: 40008 diff changeset	331 If not defined, a delta will always be computed from raw revision
631c6f5058b9 storageutil: make all callables optional Gregory Szorc <gregory.szorc@gmail.com> parents: 40008 diff changeset	332 data.
631c6f5058b9 storageutil: make all callables optional Gregory Szorc <gregory.szorc@gmail.com> parents: 40008 diff changeset	333
631c6f5058b9 storageutil: make all callables optional Gregory Szorc <gregory.szorc@gmail.com> parents: 40008 diff changeset	334 ``candeltafn`` (optional)
40008 842ffcf1d42f storageutil: extract most of emitrevisions() to standalone function Gregory Szorc <gregory.szorc@gmail.com> parents: 40007 diff changeset	335 Callable receiving a pair of revision numbers that returns a bool
842ffcf1d42f storageutil: extract most of emitrevisions() to standalone function Gregory Szorc <gregory.szorc@gmail.com> parents: 40007 diff changeset	336 indicating whether a delta between them can be produced.
842ffcf1d42f storageutil: extract most of emitrevisions() to standalone function Gregory Szorc <gregory.szorc@gmail.com> parents: 40007 diff changeset	337
40009 631c6f5058b9 storageutil: make all callables optional Gregory Szorc <gregory.szorc@gmail.com> parents: 40008 diff changeset	338 If not defined, it is assumed that any two revisions can delta with
631c6f5058b9 storageutil: make all callables optional Gregory Szorc <gregory.szorc@gmail.com> parents: 40008 diff changeset	339 each other.
631c6f5058b9 storageutil: make all callables optional Gregory Szorc <gregory.szorc@gmail.com> parents: 40008 diff changeset	340
631c6f5058b9 storageutil: make all callables optional Gregory Szorc <gregory.szorc@gmail.com> parents: 40008 diff changeset	341 ``rawsizefn`` (optional)
40008 842ffcf1d42f storageutil: extract most of emitrevisions() to standalone function Gregory Szorc <gregory.szorc@gmail.com> parents: 40007 diff changeset	342 Callable receiving a revision number and returning the length of the
42781 aeb2be20b33b rawdata: update callers in storageutils Pierre-Yves David <pierre-yves.david@octobus.net> parents: 40460 diff changeset	343 ``store.rawdata(rev)``.
40008 842ffcf1d42f storageutil: extract most of emitrevisions() to standalone function Gregory Szorc <gregory.szorc@gmail.com> parents: 40007 diff changeset	344
42781 aeb2be20b33b rawdata: update callers in storageutils Pierre-Yves David <pierre-yves.david@octobus.net> parents: 40460 diff changeset	345 If not defined, ``len(store.rawdata(rev))`` will be called.
40009 631c6f5058b9 storageutil: make all callables optional Gregory Szorc <gregory.szorc@gmail.com> parents: 40008 diff changeset	346
631c6f5058b9 storageutil: make all callables optional Gregory Szorc <gregory.szorc@gmail.com> parents: 40008 diff changeset	347 ``revdifffn`` (optional)
40008 842ffcf1d42f storageutil: extract most of emitrevisions() to standalone function Gregory Szorc <gregory.szorc@gmail.com> parents: 40007 diff changeset	348 Callable receiving a pair of revision numbers that returns a delta
842ffcf1d42f storageutil: extract most of emitrevisions() to standalone function Gregory Szorc <gregory.szorc@gmail.com> parents: 40007 diff changeset	349 between them.
842ffcf1d42f storageutil: extract most of emitrevisions() to standalone function Gregory Szorc <gregory.szorc@gmail.com> parents: 40007 diff changeset	350
40009 631c6f5058b9 storageutil: make all callables optional Gregory Szorc <gregory.szorc@gmail.com> parents: 40008 diff changeset	351 If not defined, a delta will be computed by invoking mdiff code
631c6f5058b9 storageutil: make all callables optional Gregory Szorc <gregory.szorc@gmail.com> parents: 40008 diff changeset	352 on ``store.revision()`` results.
631c6f5058b9 storageutil: make all callables optional Gregory Szorc <gregory.szorc@gmail.com> parents: 40008 diff changeset	353
631c6f5058b9 storageutil: make all callables optional Gregory Szorc <gregory.szorc@gmail.com> parents: 40008 diff changeset	354 Defining this function allows a precomputed or stored delta to be
631c6f5058b9 storageutil: make all callables optional Gregory Szorc <gregory.szorc@gmail.com> parents: 40008 diff changeset	355 used without having to compute on.
631c6f5058b9 storageutil: make all callables optional Gregory Szorc <gregory.szorc@gmail.com> parents: 40008 diff changeset	356
631c6f5058b9 storageutil: make all callables optional Gregory Szorc <gregory.szorc@gmail.com> parents: 40008 diff changeset	357 ``flagsfn`` (optional)
40008 842ffcf1d42f storageutil: extract most of emitrevisions() to standalone function Gregory Szorc <gregory.szorc@gmail.com> parents: 40007 diff changeset	358 Callable receiving a revision number and returns the integer flags
40009 631c6f5058b9 storageutil: make all callables optional Gregory Szorc <gregory.szorc@gmail.com> parents: 40008 diff changeset	359 value for it. If not defined, flags value will be 0.
40008 842ffcf1d42f storageutil: extract most of emitrevisions() to standalone function Gregory Szorc <gregory.szorc@gmail.com> parents: 40007 diff changeset	360
40427 59a870a4ad6e changegroup: refactor emitrevision to use a `deltamode` argument Boris Feld <boris.feld@octobus.net> parents: 40325 diff changeset	361 ``deltamode``
59a870a4ad6e changegroup: refactor emitrevision to use a `deltamode` argument Boris Feld <boris.feld@octobus.net> parents: 40325 diff changeset	362 constaint on delta to be sent:
59a870a4ad6e changegroup: refactor emitrevision to use a `deltamode` argument Boris Feld <boris.feld@octobus.net> parents: 40325 diff changeset	363 * CG_DELTAMODE_STD - normal mode, try to reuse storage deltas,
59a870a4ad6e changegroup: refactor emitrevision to use a `deltamode` argument Boris Feld <boris.feld@octobus.net> parents: 40325 diff changeset	364 * CG_DELTAMODE_PREV - only delta against "prev",
59a870a4ad6e changegroup: refactor emitrevision to use a `deltamode` argument Boris Feld <boris.feld@octobus.net> parents: 40325 diff changeset	365 * CG_DELTAMODE_FULL - only issue full snapshot.
59a870a4ad6e changegroup: refactor emitrevision to use a `deltamode` argument Boris Feld <boris.feld@octobus.net> parents: 40325 diff changeset	366
40008 842ffcf1d42f storageutil: extract most of emitrevisions() to standalone function Gregory Szorc <gregory.szorc@gmail.com> parents: 40007 diff changeset	367 Whether to send fulltext revisions instead of deltas, if allowed.
842ffcf1d42f storageutil: extract most of emitrevisions() to standalone function Gregory Szorc <gregory.szorc@gmail.com> parents: 40007 diff changeset	368
40010 f5d819d84461 storageutil: pass nodes into emitrevisions() Gregory Szorc <gregory.szorc@gmail.com> parents: 40009 diff changeset	369 ``nodesorder``
40008 842ffcf1d42f storageutil: extract most of emitrevisions() to standalone function Gregory Szorc <gregory.szorc@gmail.com> parents: 40007 diff changeset	370 ``revisiondata``
842ffcf1d42f storageutil: extract most of emitrevisions() to standalone function Gregory Szorc <gregory.szorc@gmail.com> parents: 40007 diff changeset	371 ``assumehaveparentrevisions``
46715 45f0d5297698 changegroupv4: add sidedata helpers Rapha?l Gom?s <rgomes@octobus.net> parents: 46712 diff changeset	372 ``sidedata_helpers`` (optional)
45f0d5297698 changegroupv4: add sidedata helpers Rapha?l Gom?s <rgomes@octobus.net> parents: 46712 diff changeset	373 If not None, means that sidedata should be included.
47086 8bd769b5c941 sidedata: move documentation about sidedata helpers to sidedata module Rapha?l Gom?s <rgomes@octobus.net> parents: 47085 diff changeset	374 See `revlogutil.sidedata.get_sidedata_helpers`.
49609 9cac281eb9c0 debug: add an option to display statistic about a bundling operation Pierre-Yves David <pierre-yves.david@octobus.net> parents: 49284 diff changeset	375
9cac281eb9c0 debug: add an option to display statistic about a bundling operation Pierre-Yves David <pierre-yves.david@octobus.net> parents: 49284 diff changeset	376 ``debug_info`
9cac281eb9c0 debug: add an option to display statistic about a bundling operation Pierre-Yves David <pierre-yves.david@octobus.net> parents: 49284 diff changeset	377 An optionnal dictionnary to gather information about the bundling
9cac281eb9c0 debug: add an option to display statistic about a bundling operation Pierre-Yves David <pierre-yves.david@octobus.net> parents: 49284 diff changeset	378 process (if present, see config: debug.bundling.stats.
40008 842ffcf1d42f storageutil: extract most of emitrevisions() to standalone function Gregory Szorc <gregory.szorc@gmail.com> parents: 40007 diff changeset	379 """
842ffcf1d42f storageutil: extract most of emitrevisions() to standalone function Gregory Szorc <gregory.szorc@gmail.com> parents: 40007 diff changeset	380
842ffcf1d42f storageutil: extract most of emitrevisions() to standalone function Gregory Szorc <gregory.szorc@gmail.com> parents: 40007 diff changeset	381 fnode = store.node
40010 f5d819d84461 storageutil: pass nodes into emitrevisions() Gregory Szorc <gregory.szorc@gmail.com> parents: 40009 diff changeset	382 frev = store.rev
49790 f463eb675e85 emitrevision: consider ancestors revision to emit as available base Pierre-Yves David <pierre-yves.david@octobus.net> parents: 49788 diff changeset	383 parents = store.parentrevs
40010 f5d819d84461 storageutil: pass nodes into emitrevisions() Gregory Szorc <gregory.szorc@gmail.com> parents: 40009 diff changeset	384
43077 687b865b95ad formatting: byteify all mercurial/ and hgext/ string literals Augie Fackler <augie@google.com> parents: 43076 diff changeset	385 if nodesorder == b'nodes':
40010 f5d819d84461 storageutil: pass nodes into emitrevisions() Gregory Szorc <gregory.szorc@gmail.com> parents: 40009 diff changeset	386 revs = [frev(n) for n in nodes]
43077 687b865b95ad formatting: byteify all mercurial/ and hgext/ string literals Augie Fackler <augie@google.com> parents: 43076 diff changeset	387 elif nodesorder == b'linear':
44452 9d2b2df2c2ba cleanup: run pyupgrade on our source tree to clean up varying things Augie Fackler <augie@google.com> parents: 44060 diff changeset	388 revs = {frev(n) for n in nodes}
40010 f5d819d84461 storageutil: pass nodes into emitrevisions() Gregory Szorc <gregory.szorc@gmail.com> parents: 40009 diff changeset	389 revs = dagop.linearize(revs, store.parentrevs)
43076 2372284d9457 formatting: blacken the codebase Augie Fackler <augie@google.com> parents: 42813 diff changeset	390 else: # storage and default
40445 634b45317459 changegroup: restore default node ordering (issue6001) Boris Feld <boris.feld@octobus.net> parents: 40325 diff changeset	391 revs = sorted(frev(n) for n in nodes)
40008 842ffcf1d42f storageutil: extract most of emitrevisions() to standalone function Gregory Szorc <gregory.szorc@gmail.com> parents: 40007 diff changeset	392
842ffcf1d42f storageutil: extract most of emitrevisions() to standalone function Gregory Szorc <gregory.szorc@gmail.com> parents: 40007 diff changeset	393 prevrev = None
842ffcf1d42f storageutil: extract most of emitrevisions() to standalone function Gregory Szorc <gregory.szorc@gmail.com> parents: 40007 diff changeset	394
40427 59a870a4ad6e changegroup: refactor emitrevision to use a `deltamode` argument Boris Feld <boris.feld@octobus.net> parents: 40325 diff changeset	395 if deltamode == repository.CG_DELTAMODE_PREV or assumehaveparentrevisions:
49790 f463eb675e85 emitrevision: consider ancestors revision to emit as available base Pierre-Yves David <pierre-yves.david@octobus.net> parents: 49788 diff changeset	396 prevrev = parents(revs[0])[0]
40008 842ffcf1d42f storageutil: extract most of emitrevisions() to standalone function Gregory Szorc <gregory.szorc@gmail.com> parents: 40007 diff changeset	397
49790 f463eb675e85 emitrevision: consider ancestors revision to emit as available base Pierre-Yves David <pierre-yves.david@octobus.net> parents: 49788 diff changeset	398 # Sets of revs available to delta against.
f463eb675e85 emitrevision: consider ancestors revision to emit as available base Pierre-Yves David <pierre-yves.david@octobus.net> parents: 49788 diff changeset	399 emitted = set()
40008 842ffcf1d42f storageutil: extract most of emitrevisions() to standalone function Gregory Szorc <gregory.szorc@gmail.com> parents: 40007 diff changeset	400 available = set()
49790 f463eb675e85 emitrevision: consider ancestors revision to emit as available base Pierre-Yves David <pierre-yves.david@octobus.net> parents: 49788 diff changeset	401 if assumehaveparentrevisions:
f463eb675e85 emitrevision: consider ancestors revision to emit as available base Pierre-Yves David <pierre-yves.david@octobus.net> parents: 49788 diff changeset	402 common_heads = set(p for r in revs for p in parents(r))
f463eb675e85 emitrevision: consider ancestors revision to emit as available base Pierre-Yves David <pierre-yves.david@octobus.net> parents: 49788 diff changeset	403 common_heads.difference_update(revs)
f463eb675e85 emitrevision: consider ancestors revision to emit as available base Pierre-Yves David <pierre-yves.david@octobus.net> parents: 49788 diff changeset	404 available = store.ancestors(common_heads, inclusive=True)
40008 842ffcf1d42f storageutil: extract most of emitrevisions() to standalone function Gregory Szorc <gregory.szorc@gmail.com> parents: 40007 diff changeset	405
49785 fa955e3f6aee emitrevision: add a small closure to check if a base is usable Pierre-Yves David <pierre-yves.david@octobus.net> parents: 49284 diff changeset	406 def is_usable_base(rev):
49786 0bda07f34c01 emitrevision: also check the parents in the availability closure Pierre-Yves David <pierre-yves.david@octobus.net> parents: 49785 diff changeset	407 """Is a delta against this revision usable over the wire"""
0bda07f34c01 emitrevision: also check the parents in the availability closure Pierre-Yves David <pierre-yves.david@octobus.net> parents: 49785 diff changeset	408 if rev == nullrev:
0bda07f34c01 emitrevision: also check the parents in the availability closure Pierre-Yves David <pierre-yves.david@octobus.net> parents: 49785 diff changeset	409 return False
49790 f463eb675e85 emitrevision: consider ancestors revision to emit as available base Pierre-Yves David <pierre-yves.david@octobus.net> parents: 49788 diff changeset	410 return rev in emitted or rev in available
49785 fa955e3f6aee emitrevision: add a small closure to check if a base is usable Pierre-Yves David <pierre-yves.david@octobus.net> parents: 49284 diff changeset	411
40008 842ffcf1d42f storageutil: extract most of emitrevisions() to standalone function Gregory Szorc <gregory.szorc@gmail.com> parents: 40007 diff changeset	412 for rev in revs:
842ffcf1d42f storageutil: extract most of emitrevisions() to standalone function Gregory Szorc <gregory.szorc@gmail.com> parents: 40007 diff changeset	413 if rev == nullrev:
842ffcf1d42f storageutil: extract most of emitrevisions() to standalone function Gregory Szorc <gregory.szorc@gmail.com> parents: 40007 diff changeset	414 continue
842ffcf1d42f storageutil: extract most of emitrevisions() to standalone function Gregory Szorc <gregory.szorc@gmail.com> parents: 40007 diff changeset	415
49609 9cac281eb9c0 debug: add an option to display statistic about a bundling operation Pierre-Yves David <pierre-yves.david@octobus.net> parents: 49284 diff changeset	416 debug_delta_source = None
9cac281eb9c0 debug: add an option to display statistic about a bundling operation Pierre-Yves David <pierre-yves.david@octobus.net> parents: 49284 diff changeset	417 if debug_info is not None:
9cac281eb9c0 debug: add an option to display statistic about a bundling operation Pierre-Yves David <pierre-yves.david@octobus.net> parents: 49284 diff changeset	418 debug_info['revision-total'] += 1
9cac281eb9c0 debug: add an option to display statistic about a bundling operation Pierre-Yves David <pierre-yves.david@octobus.net> parents: 49284 diff changeset	419
40008 842ffcf1d42f storageutil: extract most of emitrevisions() to standalone function Gregory Szorc <gregory.szorc@gmail.com> parents: 40007 diff changeset	420 node = fnode(rev)
49790 f463eb675e85 emitrevision: consider ancestors revision to emit as available base Pierre-Yves David <pierre-yves.david@octobus.net> parents: 49788 diff changeset	421 p1rev, p2rev = parents(rev)
40008 842ffcf1d42f storageutil: extract most of emitrevisions() to standalone function Gregory Szorc <gregory.szorc@gmail.com> parents: 40007 diff changeset	422
49609 9cac281eb9c0 debug: add an option to display statistic about a bundling operation Pierre-Yves David <pierre-yves.david@octobus.net> parents: 49284 diff changeset	423 if debug_info is not None:
9cac281eb9c0 debug: add an option to display statistic about a bundling operation Pierre-Yves David <pierre-yves.david@octobus.net> parents: 49284 diff changeset	424 if p1rev != p2rev and p1rev != nullrev and p2rev != nullrev:
9cac281eb9c0 debug: add an option to display statistic about a bundling operation Pierre-Yves David <pierre-yves.david@octobus.net> parents: 49284 diff changeset	425 debug_info['merge-total'] += 1
9cac281eb9c0 debug: add an option to display statistic about a bundling operation Pierre-Yves David <pierre-yves.david@octobus.net> parents: 49284 diff changeset	426
40009 631c6f5058b9 storageutil: make all callables optional Gregory Szorc <gregory.szorc@gmail.com> parents: 40008 diff changeset	427 if deltaparentfn:
631c6f5058b9 storageutil: make all callables optional Gregory Szorc <gregory.szorc@gmail.com> parents: 40008 diff changeset	428 deltaparentrev = deltaparentfn(rev)
49609 9cac281eb9c0 debug: add an option to display statistic about a bundling operation Pierre-Yves David <pierre-yves.david@octobus.net> parents: 49284 diff changeset	429 if debug_info is not None:
9cac281eb9c0 debug: add an option to display statistic about a bundling operation Pierre-Yves David <pierre-yves.david@octobus.net> parents: 49284 diff changeset	430 if deltaparentrev == nullrev:
9cac281eb9c0 debug: add an option to display statistic about a bundling operation Pierre-Yves David <pierre-yves.david@octobus.net> parents: 49284 diff changeset	431 debug_info['available-full'] += 1
9cac281eb9c0 debug: add an option to display statistic about a bundling operation Pierre-Yves David <pierre-yves.david@octobus.net> parents: 49284 diff changeset	432 else:
9cac281eb9c0 debug: add an option to display statistic about a bundling operation Pierre-Yves David <pierre-yves.david@octobus.net> parents: 49284 diff changeset	433 debug_info['available-delta'] += 1
9cac281eb9c0 debug: add an option to display statistic about a bundling operation Pierre-Yves David <pierre-yves.david@octobus.net> parents: 49284 diff changeset	434
40009 631c6f5058b9 storageutil: make all callables optional Gregory Szorc <gregory.szorc@gmail.com> parents: 40008 diff changeset	435 else:
631c6f5058b9 storageutil: make all callables optional Gregory Szorc <gregory.szorc@gmail.com> parents: 40008 diff changeset	436 deltaparentrev = nullrev
631c6f5058b9 storageutil: make all callables optional Gregory Szorc <gregory.szorc@gmail.com> parents: 40008 diff changeset	437
40008 842ffcf1d42f storageutil: extract most of emitrevisions() to standalone function Gregory Szorc <gregory.szorc@gmail.com> parents: 40007 diff changeset	438 # Forced delta against previous mode.
40427 59a870a4ad6e changegroup: refactor emitrevision to use a `deltamode` argument Boris Feld <boris.feld@octobus.net> parents: 40325 diff changeset	439 if deltamode == repository.CG_DELTAMODE_PREV:
49609 9cac281eb9c0 debug: add an option to display statistic about a bundling operation Pierre-Yves David <pierre-yves.david@octobus.net> parents: 49284 diff changeset	440 if debug_info is not None:
9cac281eb9c0 debug: add an option to display statistic about a bundling operation Pierre-Yves David <pierre-yves.david@octobus.net> parents: 49284 diff changeset	441 debug_delta_source = "prev"
40008 842ffcf1d42f storageutil: extract most of emitrevisions() to standalone function Gregory Szorc <gregory.szorc@gmail.com> parents: 40007 diff changeset	442 baserev = prevrev
842ffcf1d42f storageutil: extract most of emitrevisions() to standalone function Gregory Szorc <gregory.szorc@gmail.com> parents: 40007 diff changeset	443
842ffcf1d42f storageutil: extract most of emitrevisions() to standalone function Gregory Szorc <gregory.szorc@gmail.com> parents: 40007 diff changeset	444 # We're instructed to send fulltext. Honor that.
40427 59a870a4ad6e changegroup: refactor emitrevision to use a `deltamode` argument Boris Feld <boris.feld@octobus.net> parents: 40325 diff changeset	445 elif deltamode == repository.CG_DELTAMODE_FULL:
49609 9cac281eb9c0 debug: add an option to display statistic about a bundling operation Pierre-Yves David <pierre-yves.david@octobus.net> parents: 49284 diff changeset	446 if debug_info is not None:
9cac281eb9c0 debug: add an option to display statistic about a bundling operation Pierre-Yves David <pierre-yves.david@octobus.net> parents: 49284 diff changeset	447 debug_delta_source = "full"
40008 842ffcf1d42f storageutil: extract most of emitrevisions() to standalone function Gregory Szorc <gregory.szorc@gmail.com> parents: 40007 diff changeset	448 baserev = nullrev
40432 968dd7e02ac5 changegroup: allow to force delta to be against p1 Boris Feld <boris.feld@octobus.net> parents: 40427 diff changeset	449 # We're instructed to use p1. Honor that
968dd7e02ac5 changegroup: allow to force delta to be against p1 Boris Feld <boris.feld@octobus.net> parents: 40427 diff changeset	450 elif deltamode == repository.CG_DELTAMODE_P1:
49609 9cac281eb9c0 debug: add an option to display statistic about a bundling operation Pierre-Yves David <pierre-yves.david@octobus.net> parents: 49284 diff changeset	451 if debug_info is not None:
9cac281eb9c0 debug: add an option to display statistic about a bundling operation Pierre-Yves David <pierre-yves.david@octobus.net> parents: 49284 diff changeset	452 debug_delta_source = "p1"
40432 968dd7e02ac5 changegroup: allow to force delta to be against p1 Boris Feld <boris.feld@octobus.net> parents: 40427 diff changeset	453 baserev = p1rev
40008 842ffcf1d42f storageutil: extract most of emitrevisions() to standalone function Gregory Szorc <gregory.szorc@gmail.com> parents: 40007 diff changeset	454
842ffcf1d42f storageutil: extract most of emitrevisions() to standalone function Gregory Szorc <gregory.szorc@gmail.com> parents: 40007 diff changeset	455 # There is a delta in storage. We try to use that because it
842ffcf1d42f storageutil: extract most of emitrevisions() to standalone function Gregory Szorc <gregory.szorc@gmail.com> parents: 40007 diff changeset	456 # amounts to effectively copying data from storage and is
842ffcf1d42f storageutil: extract most of emitrevisions() to standalone function Gregory Szorc <gregory.szorc@gmail.com> parents: 40007 diff changeset	457 # therefore the fastest.
49787 bb2c663c840f emitrevision: simplify the fallback to computed delta Pierre-Yves David <pierre-yves.david@octobus.net> parents: 49786 diff changeset	458 elif is_usable_base(deltaparentrev):
49669 f064b03d061a emitrevision: simplify the fallback to computed delta Pierre-Yves David <pierre-yves.david@octobus.net> parents: 49668 diff changeset	459 if debug_info is not None:
f064b03d061a emitrevision: simplify the fallback to computed delta Pierre-Yves David <pierre-yves.david@octobus.net> parents: 49668 diff changeset	460 debug_delta_source = "storage"
f064b03d061a emitrevision: simplify the fallback to computed delta Pierre-Yves David <pierre-yves.david@octobus.net> parents: 49668 diff changeset	461 baserev = deltaparentrev
49777 e1953a34c110 bundle: emit full snapshot as is, without doing a redelta Pierre-Yves David <pierre-yves.david@octobus.net> parents: 49672 diff changeset	462 elif deltaparentrev == nullrev:
e1953a34c110 bundle: emit full snapshot as is, without doing a redelta Pierre-Yves David <pierre-yves.david@octobus.net> parents: 49672 diff changeset	463 if debug_info is not None:
e1953a34c110 bundle: emit full snapshot as is, without doing a redelta Pierre-Yves David <pierre-yves.david@octobus.net> parents: 49672 diff changeset	464 debug_delta_source = "storage"
49787 bb2c663c840f emitrevision: simplify the fallback to computed delta Pierre-Yves David <pierre-yves.david@octobus.net> parents: 49786 diff changeset	465 baserev = deltaparentrev
bb2c663c840f emitrevision: simplify the fallback to computed delta Pierre-Yves David <pierre-yves.david@octobus.net> parents: 49786 diff changeset	466 else:
49670 2fd8750f3722 emitrevision: if we need to compute a delta on the fly, try p1 or p2 first Pierre-Yves David <pierre-yves.david@octobus.net> parents: 49669 diff changeset	467 if deltaparentrev != nullrev and debug_info is not None:
2fd8750f3722 emitrevision: if we need to compute a delta on the fly, try p1 or p2 first Pierre-Yves David <pierre-yves.david@octobus.net> parents: 49669 diff changeset	468 debug_info['denied-base-not-available'] += 1
49787 bb2c663c840f emitrevision: simplify the fallback to computed delta Pierre-Yves David <pierre-yves.david@octobus.net> parents: 49786 diff changeset	469 # No guarantee the receiver has the delta parent, or Storage has a
bb2c663c840f emitrevision: simplify the fallback to computed delta Pierre-Yves David <pierre-yves.david@octobus.net> parents: 49786 diff changeset	470 # fulltext revision.
bb2c663c840f emitrevision: simplify the fallback to computed delta Pierre-Yves David <pierre-yves.david@octobus.net> parents: 49786 diff changeset	471 #
49788 31b4675ca998 emitrevision: if we need to compute a delta on the fly, try p1 or p2 first Pierre-Yves David <pierre-yves.david@octobus.net> parents: 49787 diff changeset	472 # We compute a delta on the fly to send over the wire.
31b4675ca998 emitrevision: if we need to compute a delta on the fly, try p1 or p2 first Pierre-Yves David <pierre-yves.david@octobus.net> parents: 49787 diff changeset	473 #
31b4675ca998 emitrevision: if we need to compute a delta on the fly, try p1 or p2 first Pierre-Yves David <pierre-yves.david@octobus.net> parents: 49787 diff changeset	474 # We start with a try against p1, which in the common case should
31b4675ca998 emitrevision: if we need to compute a delta on the fly, try p1 or p2 first Pierre-Yves David <pierre-yves.david@octobus.net> parents: 49787 diff changeset	475 # be close to this revision content.
31b4675ca998 emitrevision: if we need to compute a delta on the fly, try p1 or p2 first Pierre-Yves David <pierre-yves.david@octobus.net> parents: 49787 diff changeset	476 #
31b4675ca998 emitrevision: if we need to compute a delta on the fly, try p1 or p2 first Pierre-Yves David <pierre-yves.david@octobus.net> parents: 49787 diff changeset	477 # note: we could optimize between p1 and p2 in merges cases.
49670 2fd8750f3722 emitrevision: if we need to compute a delta on the fly, try p1 or p2 first Pierre-Yves David <pierre-yves.david@octobus.net> parents: 49669 diff changeset	478 elif is_usable_base(p1rev):
2fd8750f3722 emitrevision: if we need to compute a delta on the fly, try p1 or p2 first Pierre-Yves David <pierre-yves.david@octobus.net> parents: 49669 diff changeset	479 if debug_info is not None:
2fd8750f3722 emitrevision: if we need to compute a delta on the fly, try p1 or p2 first Pierre-Yves David <pierre-yves.david@octobus.net> parents: 49669 diff changeset	480 debug_delta_source = "p1"
49788 31b4675ca998 emitrevision: if we need to compute a delta on the fly, try p1 or p2 first Pierre-Yves David <pierre-yves.david@octobus.net> parents: 49787 diff changeset	481 baserev = p1rev
31b4675ca998 emitrevision: if we need to compute a delta on the fly, try p1 or p2 first Pierre-Yves David <pierre-yves.david@octobus.net> parents: 49787 diff changeset	482 # if p1 was not an option, try p2
31b4675ca998 emitrevision: if we need to compute a delta on the fly, try p1 or p2 first Pierre-Yves David <pierre-yves.david@octobus.net> parents: 49787 diff changeset	483 elif is_usable_base(p2rev):
49670 2fd8750f3722 emitrevision: if we need to compute a delta on the fly, try p1 or p2 first Pierre-Yves David <pierre-yves.david@octobus.net> parents: 49669 diff changeset	484 if debug_info is not None:
2fd8750f3722 emitrevision: if we need to compute a delta on the fly, try p1 or p2 first Pierre-Yves David <pierre-yves.david@octobus.net> parents: 49669 diff changeset	485 debug_delta_source = "p2"
49788 31b4675ca998 emitrevision: if we need to compute a delta on the fly, try p1 or p2 first Pierre-Yves David <pierre-yves.david@octobus.net> parents: 49787 diff changeset	486 baserev = p2rev
31b4675ca998 emitrevision: if we need to compute a delta on the fly, try p1 or p2 first Pierre-Yves David <pierre-yves.david@octobus.net> parents: 49787 diff changeset	487 # Send delta against prev in despair
31b4675ca998 emitrevision: if we need to compute a delta on the fly, try p1 or p2 first Pierre-Yves David <pierre-yves.david@octobus.net> parents: 49787 diff changeset	488 #
31b4675ca998 emitrevision: if we need to compute a delta on the fly, try p1 or p2 first Pierre-Yves David <pierre-yves.david@octobus.net> parents: 49787 diff changeset	489 # using the closest available ancestors first might be better?
31b4675ca998 emitrevision: if we need to compute a delta on the fly, try p1 or p2 first Pierre-Yves David <pierre-yves.david@octobus.net> parents: 49787 diff changeset	490 elif prevrev is not None:
49609 9cac281eb9c0 debug: add an option to display statistic about a bundling operation Pierre-Yves David <pierre-yves.david@octobus.net> parents: 49284 diff changeset	491 if debug_info is not None:
9cac281eb9c0 debug: add an option to display statistic about a bundling operation Pierre-Yves David <pierre-yves.david@octobus.net> parents: 49284 diff changeset	492 debug_delta_source = "prev"
40008 842ffcf1d42f storageutil: extract most of emitrevisions() to standalone function Gregory Szorc <gregory.szorc@gmail.com> parents: 40007 diff changeset	493 baserev = prevrev
842ffcf1d42f storageutil: extract most of emitrevisions() to standalone function Gregory Szorc <gregory.szorc@gmail.com> parents: 40007 diff changeset	494 else:
49609 9cac281eb9c0 debug: add an option to display statistic about a bundling operation Pierre-Yves David <pierre-yves.david@octobus.net> parents: 49284 diff changeset	495 if debug_info is not None:
9cac281eb9c0 debug: add an option to display statistic about a bundling operation Pierre-Yves David <pierre-yves.david@octobus.net> parents: 49284 diff changeset	496 debug_delta_source = "full"
40008 842ffcf1d42f storageutil: extract most of emitrevisions() to standalone function Gregory Szorc <gregory.szorc@gmail.com> parents: 40007 diff changeset	497 baserev = nullrev
842ffcf1d42f storageutil: extract most of emitrevisions() to standalone function Gregory Szorc <gregory.szorc@gmail.com> parents: 40007 diff changeset	498
842ffcf1d42f storageutil: extract most of emitrevisions() to standalone function Gregory Szorc <gregory.szorc@gmail.com> parents: 40007 diff changeset	499 # But we can't actually use our chosen delta base for whatever
842ffcf1d42f storageutil: extract most of emitrevisions() to standalone function Gregory Szorc <gregory.szorc@gmail.com> parents: 40007 diff changeset	500 # reason. Reset to fulltext.
49609 9cac281eb9c0 debug: add an option to display statistic about a bundling operation Pierre-Yves David <pierre-yves.david@octobus.net> parents: 49284 diff changeset	501 if (
9cac281eb9c0 debug: add an option to display statistic about a bundling operation Pierre-Yves David <pierre-yves.david@octobus.net> parents: 49284 diff changeset	502 baserev != nullrev
9cac281eb9c0 debug: add an option to display statistic about a bundling operation Pierre-Yves David <pierre-yves.david@octobus.net> parents: 49284 diff changeset	503 and candeltafn is not None
9cac281eb9c0 debug: add an option to display statistic about a bundling operation Pierre-Yves David <pierre-yves.david@octobus.net> parents: 49284 diff changeset	504 and not candeltafn(baserev, rev)
9cac281eb9c0 debug: add an option to display statistic about a bundling operation Pierre-Yves David <pierre-yves.david@octobus.net> parents: 49284 diff changeset	505 ):
9cac281eb9c0 debug: add an option to display statistic about a bundling operation Pierre-Yves David <pierre-yves.david@octobus.net> parents: 49284 diff changeset	506 if debug_info is not None:
9cac281eb9c0 debug: add an option to display statistic about a bundling operation Pierre-Yves David <pierre-yves.david@octobus.net> parents: 49284 diff changeset	507 debug_delta_source = "full"
9cac281eb9c0 debug: add an option to display statistic about a bundling operation Pierre-Yves David <pierre-yves.david@octobus.net> parents: 49284 diff changeset	508 debug_info['denied-delta-candeltafn'] += 1
40008 842ffcf1d42f storageutil: extract most of emitrevisions() to standalone function Gregory Szorc <gregory.szorc@gmail.com> parents: 40007 diff changeset	509 baserev = nullrev
842ffcf1d42f storageutil: extract most of emitrevisions() to standalone function Gregory Szorc <gregory.szorc@gmail.com> parents: 40007 diff changeset	510
842ffcf1d42f storageutil: extract most of emitrevisions() to standalone function Gregory Szorc <gregory.szorc@gmail.com> parents: 40007 diff changeset	511 revision = None
842ffcf1d42f storageutil: extract most of emitrevisions() to standalone function Gregory Szorc <gregory.szorc@gmail.com> parents: 40007 diff changeset	512 delta = None
842ffcf1d42f storageutil: extract most of emitrevisions() to standalone function Gregory Szorc <gregory.szorc@gmail.com> parents: 40007 diff changeset	513 baserevisionsize = None
842ffcf1d42f storageutil: extract most of emitrevisions() to standalone function Gregory Szorc <gregory.szorc@gmail.com> parents: 40007 diff changeset	514
842ffcf1d42f storageutil: extract most of emitrevisions() to standalone function Gregory Szorc <gregory.szorc@gmail.com> parents: 40007 diff changeset	515 if revisiondata:
842ffcf1d42f storageutil: extract most of emitrevisions() to standalone function Gregory Szorc <gregory.szorc@gmail.com> parents: 40007 diff changeset	516 if store.iscensored(baserev) or store.iscensored(rev):
842ffcf1d42f storageutil: extract most of emitrevisions() to standalone function Gregory Szorc <gregory.szorc@gmail.com> parents: 40007 diff changeset	517 try:
42781 aeb2be20b33b rawdata: update callers in storageutils Pierre-Yves David <pierre-yves.david@octobus.net> parents: 40460 diff changeset	518 revision = store.rawdata(node)
40008 842ffcf1d42f storageutil: extract most of emitrevisions() to standalone function Gregory Szorc <gregory.szorc@gmail.com> parents: 40007 diff changeset	519 except error.CensoredNodeError as e:
49609 9cac281eb9c0 debug: add an option to display statistic about a bundling operation Pierre-Yves David <pierre-yves.david@octobus.net> parents: 49284 diff changeset	520 if debug_info is not None:
9cac281eb9c0 debug: add an option to display statistic about a bundling operation Pierre-Yves David <pierre-yves.david@octobus.net> parents: 49284 diff changeset	521 debug_delta_source = "full"
9cac281eb9c0 debug: add an option to display statistic about a bundling operation Pierre-Yves David <pierre-yves.david@octobus.net> parents: 49284 diff changeset	522 debug_info['denied-delta-not-available'] += 1
40008 842ffcf1d42f storageutil: extract most of emitrevisions() to standalone function Gregory Szorc <gregory.szorc@gmail.com> parents: 40007 diff changeset	523 revision = e.tombstone
842ffcf1d42f storageutil: extract most of emitrevisions() to standalone function Gregory Szorc <gregory.szorc@gmail.com> parents: 40007 diff changeset	524
842ffcf1d42f storageutil: extract most of emitrevisions() to standalone function Gregory Szorc <gregory.szorc@gmail.com> parents: 40007 diff changeset	525 if baserev != nullrev:
40009 631c6f5058b9 storageutil: make all callables optional Gregory Szorc <gregory.szorc@gmail.com> parents: 40008 diff changeset	526 if rawsizefn:
631c6f5058b9 storageutil: make all callables optional Gregory Szorc <gregory.szorc@gmail.com> parents: 40008 diff changeset	527 baserevisionsize = rawsizefn(baserev)
631c6f5058b9 storageutil: make all callables optional Gregory Szorc <gregory.szorc@gmail.com> parents: 40008 diff changeset	528 else:
42781 aeb2be20b33b rawdata: update callers in storageutils Pierre-Yves David <pierre-yves.david@octobus.net> parents: 40460 diff changeset	529 baserevisionsize = len(store.rawdata(baserev))
40008 842ffcf1d42f storageutil: extract most of emitrevisions() to standalone function Gregory Szorc <gregory.szorc@gmail.com> parents: 40007 diff changeset	530
43076 2372284d9457 formatting: blacken the codebase Augie Fackler <augie@google.com> parents: 42813 diff changeset	531 elif (
2372284d9457 formatting: blacken the codebase Augie Fackler <augie@google.com> parents: 42813 diff changeset	532 baserev == nullrev and deltamode != repository.CG_DELTAMODE_PREV
2372284d9457 formatting: blacken the codebase Augie Fackler <augie@google.com> parents: 42813 diff changeset	533 ):
49609 9cac281eb9c0 debug: add an option to display statistic about a bundling operation Pierre-Yves David <pierre-yves.david@octobus.net> parents: 49284 diff changeset	534 if debug_info is not None:
9cac281eb9c0 debug: add an option to display statistic about a bundling operation Pierre-Yves David <pierre-yves.david@octobus.net> parents: 49284 diff changeset	535 debug_info['computed-delta'] += 1 # close enough
9cac281eb9c0 debug: add an option to display statistic about a bundling operation Pierre-Yves David <pierre-yves.david@octobus.net> parents: 49284 diff changeset	536 debug_info['delta-full'] += 1
42781 aeb2be20b33b rawdata: update callers in storageutils Pierre-Yves David <pierre-yves.david@octobus.net> parents: 40460 diff changeset	537 revision = store.rawdata(node)
49790 f463eb675e85 emitrevision: consider ancestors revision to emit as available base Pierre-Yves David <pierre-yves.david@octobus.net> parents: 49788 diff changeset	538 emitted.add(rev)
40008 842ffcf1d42f storageutil: extract most of emitrevisions() to standalone function Gregory Szorc <gregory.szorc@gmail.com> parents: 40007 diff changeset	539 else:
40009 631c6f5058b9 storageutil: make all callables optional Gregory Szorc <gregory.szorc@gmail.com> parents: 40008 diff changeset	540 if revdifffn:
49609 9cac281eb9c0 debug: add an option to display statistic about a bundling operation Pierre-Yves David <pierre-yves.david@octobus.net> parents: 49284 diff changeset	541 if debug_info is not None:
9cac281eb9c0 debug: add an option to display statistic about a bundling operation Pierre-Yves David <pierre-yves.david@octobus.net> parents: 49284 diff changeset	542 if debug_delta_source == "full":
9cac281eb9c0 debug: add an option to display statistic about a bundling operation Pierre-Yves David <pierre-yves.david@octobus.net> parents: 49284 diff changeset	543 debug_info['computed-delta'] += 1
9cac281eb9c0 debug: add an option to display statistic about a bundling operation Pierre-Yves David <pierre-yves.david@octobus.net> parents: 49284 diff changeset	544 debug_info['delta-full'] += 1
9cac281eb9c0 debug: add an option to display statistic about a bundling operation Pierre-Yves David <pierre-yves.david@octobus.net> parents: 49284 diff changeset	545 elif debug_delta_source == "prev":
9cac281eb9c0 debug: add an option to display statistic about a bundling operation Pierre-Yves David <pierre-yves.david@octobus.net> parents: 49284 diff changeset	546 debug_info['computed-delta'] += 1
9cac281eb9c0 debug: add an option to display statistic about a bundling operation Pierre-Yves David <pierre-yves.david@octobus.net> parents: 49284 diff changeset	547 debug_info['delta-against-prev'] += 1
9cac281eb9c0 debug: add an option to display statistic about a bundling operation Pierre-Yves David <pierre-yves.david@octobus.net> parents: 49284 diff changeset	548 elif debug_delta_source == "p1":
9cac281eb9c0 debug: add an option to display statistic about a bundling operation Pierre-Yves David <pierre-yves.david@octobus.net> parents: 49284 diff changeset	549 debug_info['computed-delta'] += 1
9cac281eb9c0 debug: add an option to display statistic about a bundling operation Pierre-Yves David <pierre-yves.david@octobus.net> parents: 49284 diff changeset	550 debug_info['delta-against-p1'] += 1
9cac281eb9c0 debug: add an option to display statistic about a bundling operation Pierre-Yves David <pierre-yves.david@octobus.net> parents: 49284 diff changeset	551 elif debug_delta_source == "storage":
9cac281eb9c0 debug: add an option to display statistic about a bundling operation Pierre-Yves David <pierre-yves.david@octobus.net> parents: 49284 diff changeset	552 debug_info['reused-storage-delta'] += 1
9cac281eb9c0 debug: add an option to display statistic about a bundling operation Pierre-Yves David <pierre-yves.david@octobus.net> parents: 49284 diff changeset	553 else:
9cac281eb9c0 debug: add an option to display statistic about a bundling operation Pierre-Yves David <pierre-yves.david@octobus.net> parents: 49284 diff changeset	554 assert False, 'unreachable'
9cac281eb9c0 debug: add an option to display statistic about a bundling operation Pierre-Yves David <pierre-yves.david@octobus.net> parents: 49284 diff changeset	555
40009 631c6f5058b9 storageutil: make all callables optional Gregory Szorc <gregory.szorc@gmail.com> parents: 40008 diff changeset	556 delta = revdifffn(baserev, rev)
631c6f5058b9 storageutil: make all callables optional Gregory Szorc <gregory.szorc@gmail.com> parents: 40008 diff changeset	557 else:
49609 9cac281eb9c0 debug: add an option to display statistic about a bundling operation Pierre-Yves David <pierre-yves.david@octobus.net> parents: 49284 diff changeset	558 if debug_info is not None:
9cac281eb9c0 debug: add an option to display statistic about a bundling operation Pierre-Yves David <pierre-yves.david@octobus.net> parents: 49284 diff changeset	559 if debug_delta_source == "full":
9cac281eb9c0 debug: add an option to display statistic about a bundling operation Pierre-Yves David <pierre-yves.david@octobus.net> parents: 49284 diff changeset	560 debug_info['computed-delta'] += 1
9cac281eb9c0 debug: add an option to display statistic about a bundling operation Pierre-Yves David <pierre-yves.david@octobus.net> parents: 49284 diff changeset	561 debug_info['delta-full'] += 1
9cac281eb9c0 debug: add an option to display statistic about a bundling operation Pierre-Yves David <pierre-yves.david@octobus.net> parents: 49284 diff changeset	562 elif debug_delta_source == "prev":
9cac281eb9c0 debug: add an option to display statistic about a bundling operation Pierre-Yves David <pierre-yves.david@octobus.net> parents: 49284 diff changeset	563 debug_info['computed-delta'] += 1
9cac281eb9c0 debug: add an option to display statistic about a bundling operation Pierre-Yves David <pierre-yves.david@octobus.net> parents: 49284 diff changeset	564 debug_info['delta-against-prev'] += 1
9cac281eb9c0 debug: add an option to display statistic about a bundling operation Pierre-Yves David <pierre-yves.david@octobus.net> parents: 49284 diff changeset	565 elif debug_delta_source == "p1":
9cac281eb9c0 debug: add an option to display statistic about a bundling operation Pierre-Yves David <pierre-yves.david@octobus.net> parents: 49284 diff changeset	566 debug_info['computed-delta'] += 1
9cac281eb9c0 debug: add an option to display statistic about a bundling operation Pierre-Yves David <pierre-yves.david@octobus.net> parents: 49284 diff changeset	567 debug_info['delta-against-p1'] += 1
9cac281eb9c0 debug: add an option to display statistic about a bundling operation Pierre-Yves David <pierre-yves.david@octobus.net> parents: 49284 diff changeset	568 elif debug_delta_source == "storage":
9cac281eb9c0 debug: add an option to display statistic about a bundling operation Pierre-Yves David <pierre-yves.david@octobus.net> parents: 49284 diff changeset	569 # seem quite unlikelry to happens
9cac281eb9c0 debug: add an option to display statistic about a bundling operation Pierre-Yves David <pierre-yves.david@octobus.net> parents: 49284 diff changeset	570 debug_info['computed-delta'] += 1
9cac281eb9c0 debug: add an option to display statistic about a bundling operation Pierre-Yves David <pierre-yves.david@octobus.net> parents: 49284 diff changeset	571 debug_info['reused-storage-delta'] += 1
9cac281eb9c0 debug: add an option to display statistic about a bundling operation Pierre-Yves David <pierre-yves.david@octobus.net> parents: 49284 diff changeset	572 else:
9cac281eb9c0 debug: add an option to display statistic about a bundling operation Pierre-Yves David <pierre-yves.david@octobus.net> parents: 49284 diff changeset	573 assert False, 'unreachable'
43076 2372284d9457 formatting: blacken the codebase Augie Fackler <augie@google.com> parents: 42813 diff changeset	574 delta = mdiff.textdiff(
2372284d9457 formatting: blacken the codebase Augie Fackler <augie@google.com> parents: 42813 diff changeset	575 store.rawdata(baserev), store.rawdata(rev)
2372284d9457 formatting: blacken the codebase Augie Fackler <augie@google.com> parents: 42813 diff changeset	576 )
40009 631c6f5058b9 storageutil: make all callables optional Gregory Szorc <gregory.szorc@gmail.com> parents: 40008 diff changeset	577
49790 f463eb675e85 emitrevision: consider ancestors revision to emit as available base Pierre-Yves David <pierre-yves.david@octobus.net> parents: 49788 diff changeset	578 emitted.add(rev)
40008 842ffcf1d42f storageutil: extract most of emitrevisions() to standalone function Gregory Szorc <gregory.szorc@gmail.com> parents: 40007 diff changeset	579
47077 119790e1c67c cg4: introduce protocol flag to signify the presence of sidedata Rapha?l Gom?s <rgomes@octobus.net> parents: 47012 diff changeset	580 serialized_sidedata = None
47078 223b47235d1c sidedata: enable sidedata computers to optionally rewrite flags Rapha?l Gom?s <rgomes@octobus.net> parents: 47077 diff changeset	581 sidedata_flags = (0, 0)
46715 45f0d5297698 changegroupv4: add sidedata helpers Rapha?l Gom?s <rgomes@octobus.net> parents: 46712 diff changeset	582 if sidedata_helpers:
47341 24ea3ef35238 censor: do not process sidedata of censored revision while bundling Pierre-Yves David <pierre-yves.david@octobus.net> parents: 47086 diff changeset	583 try:
24ea3ef35238 censor: do not process sidedata of censored revision while bundling Pierre-Yves David <pierre-yves.david@octobus.net> parents: 47086 diff changeset	584 old_sidedata = store.sidedata(rev)
24ea3ef35238 censor: do not process sidedata of censored revision while bundling Pierre-Yves David <pierre-yves.david@octobus.net> parents: 47086 diff changeset	585 except error.CensoredNodeError:
24ea3ef35238 censor: do not process sidedata of censored revision while bundling Pierre-Yves David <pierre-yves.david@octobus.net> parents: 47086 diff changeset	586 # skip any potential sidedata of the censored revision
24ea3ef35238 censor: do not process sidedata of censored revision while bundling Pierre-Yves David <pierre-yves.david@octobus.net> parents: 47086 diff changeset	587 sidedata = {}
24ea3ef35238 censor: do not process sidedata of censored revision while bundling Pierre-Yves David <pierre-yves.david@octobus.net> parents: 47086 diff changeset	588 else:
24ea3ef35238 censor: do not process sidedata of censored revision while bundling Pierre-Yves David <pierre-yves.david@octobus.net> parents: 47086 diff changeset	589 sidedata, sidedata_flags = sidedatamod.run_sidedata_helpers(
24ea3ef35238 censor: do not process sidedata of censored revision while bundling Pierre-Yves David <pierre-yves.david@octobus.net> parents: 47086 diff changeset	590 store=store,
24ea3ef35238 censor: do not process sidedata of censored revision while bundling Pierre-Yves David <pierre-yves.david@octobus.net> parents: 47086 diff changeset	591 sidedata_helpers=sidedata_helpers,
24ea3ef35238 censor: do not process sidedata of censored revision while bundling Pierre-Yves David <pierre-yves.david@octobus.net> parents: 47086 diff changeset	592 sidedata=old_sidedata,
24ea3ef35238 censor: do not process sidedata of censored revision while bundling Pierre-Yves David <pierre-yves.david@octobus.net> parents: 47086 diff changeset	593 rev=rev,
24ea3ef35238 censor: do not process sidedata of censored revision while bundling Pierre-Yves David <pierre-yves.david@octobus.net> parents: 47086 diff changeset	594 )
47077 119790e1c67c cg4: introduce protocol flag to signify the presence of sidedata Rapha?l Gom?s <rgomes@octobus.net> parents: 47012 diff changeset	595 if sidedata:
119790e1c67c cg4: introduce protocol flag to signify the presence of sidedata Rapha?l Gom?s <rgomes@octobus.net> parents: 47012 diff changeset	596 serialized_sidedata = sidedatamod.serialize_sidedata(sidedata)
119790e1c67c cg4: introduce protocol flag to signify the presence of sidedata Rapha?l Gom?s <rgomes@octobus.net> parents: 47012 diff changeset	597
119790e1c67c cg4: introduce protocol flag to signify the presence of sidedata Rapha?l Gom?s <rgomes@octobus.net> parents: 47012 diff changeset	598 flags = flagsfn(rev) if flagsfn else 0
119790e1c67c cg4: introduce protocol flag to signify the presence of sidedata Rapha?l Gom?s <rgomes@octobus.net> parents: 47012 diff changeset	599 protocol_flags = 0
119790e1c67c cg4: introduce protocol flag to signify the presence of sidedata Rapha?l Gom?s <rgomes@octobus.net> parents: 47012 diff changeset	600 if serialized_sidedata:
119790e1c67c cg4: introduce protocol flag to signify the presence of sidedata Rapha?l Gom?s <rgomes@octobus.net> parents: 47012 diff changeset	601 # Advertise that sidedata exists to the other side
119790e1c67c cg4: introduce protocol flag to signify the presence of sidedata Rapha?l Gom?s <rgomes@octobus.net> parents: 47012 diff changeset	602 protocol_flags \|= CG_FLAG_SIDEDATA
47078 223b47235d1c sidedata: enable sidedata computers to optionally rewrite flags Rapha?l Gom?s <rgomes@octobus.net> parents: 47077 diff changeset	603 # Computers and removers can return flags to add and/or remove
223b47235d1c sidedata: enable sidedata computers to optionally rewrite flags Rapha?l Gom?s <rgomes@octobus.net> parents: 47077 diff changeset	604 flags = flags \| sidedata_flags[0] & ~sidedata_flags[1]
46715 45f0d5297698 changegroupv4: add sidedata helpers Rapha?l Gom?s <rgomes@octobus.net> parents: 46712 diff changeset	605
40008 842ffcf1d42f storageutil: extract most of emitrevisions() to standalone function Gregory Szorc <gregory.szorc@gmail.com> parents: 40007 diff changeset	606 yield resultcls(
842ffcf1d42f storageutil: extract most of emitrevisions() to standalone function Gregory Szorc <gregory.szorc@gmail.com> parents: 40007 diff changeset	607 node=node,
842ffcf1d42f storageutil: extract most of emitrevisions() to standalone function Gregory Szorc <gregory.szorc@gmail.com> parents: 40007 diff changeset	608 p1node=fnode(p1rev),
842ffcf1d42f storageutil: extract most of emitrevisions() to standalone function Gregory Szorc <gregory.szorc@gmail.com> parents: 40007 diff changeset	609 p2node=fnode(p2rev),
842ffcf1d42f storageutil: extract most of emitrevisions() to standalone function Gregory Szorc <gregory.szorc@gmail.com> parents: 40007 diff changeset	610 basenode=fnode(baserev),
47077 119790e1c67c cg4: introduce protocol flag to signify the presence of sidedata Rapha?l Gom?s <rgomes@octobus.net> parents: 47012 diff changeset	611 flags=flags,
40008 842ffcf1d42f storageutil: extract most of emitrevisions() to standalone function Gregory Szorc <gregory.szorc@gmail.com> parents: 40007 diff changeset	612 baserevisionsize=baserevisionsize,
842ffcf1d42f storageutil: extract most of emitrevisions() to standalone function Gregory Szorc <gregory.szorc@gmail.com> parents: 40007 diff changeset	613 revision=revision,
43076 2372284d9457 formatting: blacken the codebase Augie Fackler <augie@google.com> parents: 42813 diff changeset	614 delta=delta,
47077 119790e1c67c cg4: introduce protocol flag to signify the presence of sidedata Rapha?l Gom?s <rgomes@octobus.net> parents: 47012 diff changeset	615 sidedata=serialized_sidedata,
119790e1c67c cg4: introduce protocol flag to signify the presence of sidedata Rapha?l Gom?s <rgomes@octobus.net> parents: 47012 diff changeset	616 protocol_flags=protocol_flags,
43076 2372284d9457 formatting: blacken the codebase Augie Fackler <augie@google.com> parents: 42813 diff changeset	617 )
40008 842ffcf1d42f storageutil: extract most of emitrevisions() to standalone function Gregory Szorc <gregory.szorc@gmail.com> parents: 40007 diff changeset	618
842ffcf1d42f storageutil: extract most of emitrevisions() to standalone function Gregory Szorc <gregory.szorc@gmail.com> parents: 40007 diff changeset	619 prevrev = rev
40325 b0fbd1792e2d storageutil: extract most of peek_censored from revlog Gregory Szorc <gregory.szorc@gmail.com> parents: 40321 diff changeset	620
43076 2372284d9457 formatting: blacken the codebase Augie Fackler <augie@google.com> parents: 42813 diff changeset	621
40325 b0fbd1792e2d storageutil: extract most of peek_censored from revlog Gregory Szorc <gregory.szorc@gmail.com> parents: 40321 diff changeset	622 def deltaiscensored(delta, baserev, baselenfn):
b0fbd1792e2d storageutil: extract most of peek_censored from revlog Gregory Szorc <gregory.szorc@gmail.com> parents: 40321 diff changeset	623 """Determine if a delta represents censored revision data.
b0fbd1792e2d storageutil: extract most of peek_censored from revlog Gregory Szorc <gregory.szorc@gmail.com> parents: 40321 diff changeset	624
b0fbd1792e2d storageutil: extract most of peek_censored from revlog Gregory Szorc <gregory.szorc@gmail.com> parents: 40321 diff changeset	625 ``baserev`` is the base revision this delta is encoded against.
b0fbd1792e2d storageutil: extract most of peek_censored from revlog Gregory Szorc <gregory.szorc@gmail.com> parents: 40321 diff changeset	626 ``baselenfn`` is a callable receiving a revision number that resolves the
b0fbd1792e2d storageutil: extract most of peek_censored from revlog Gregory Szorc <gregory.szorc@gmail.com> parents: 40321 diff changeset	627 length of the revision fulltext.
b0fbd1792e2d storageutil: extract most of peek_censored from revlog Gregory Szorc <gregory.szorc@gmail.com> parents: 40321 diff changeset	628
b0fbd1792e2d storageutil: extract most of peek_censored from revlog Gregory Szorc <gregory.szorc@gmail.com> parents: 40321 diff changeset	629 Returns a bool indicating if the result of the delta represents a censored
b0fbd1792e2d storageutil: extract most of peek_censored from revlog Gregory Szorc <gregory.szorc@gmail.com> parents: 40321 diff changeset	630 revision.
b0fbd1792e2d storageutil: extract most of peek_censored from revlog Gregory Szorc <gregory.szorc@gmail.com> parents: 40321 diff changeset	631 """
b0fbd1792e2d storageutil: extract most of peek_censored from revlog Gregory Szorc <gregory.szorc@gmail.com> parents: 40321 diff changeset	632 # Fragile heuristic: unless new file meta keys are added alphabetically
b0fbd1792e2d storageutil: extract most of peek_censored from revlog Gregory Szorc <gregory.szorc@gmail.com> parents: 40321 diff changeset	633 # preceding "censored", all censored revisions are prefixed by
b0fbd1792e2d storageutil: extract most of peek_censored from revlog Gregory Szorc <gregory.szorc@gmail.com> parents: 40321 diff changeset	634 # "\1\ncensored:". A delta producing such a censored revision must be a
b0fbd1792e2d storageutil: extract most of peek_censored from revlog Gregory Szorc <gregory.szorc@gmail.com> parents: 40321 diff changeset	635 # full-replacement delta, so we inspect the first and only patch in the
b0fbd1792e2d storageutil: extract most of peek_censored from revlog Gregory Szorc <gregory.szorc@gmail.com> parents: 40321 diff changeset	636 # delta for this prefix.
43077 687b865b95ad formatting: byteify all mercurial/ and hgext/ string literals Augie Fackler <augie@google.com> parents: 43076 diff changeset	637 hlen = struct.calcsize(b">lll")
40325 b0fbd1792e2d storageutil: extract most of peek_censored from revlog Gregory Szorc <gregory.szorc@gmail.com> parents: 40321 diff changeset	638 if len(delta) <= hlen:
b0fbd1792e2d storageutil: extract most of peek_censored from revlog Gregory Szorc <gregory.szorc@gmail.com> parents: 40321 diff changeset	639 return False
b0fbd1792e2d storageutil: extract most of peek_censored from revlog Gregory Szorc <gregory.szorc@gmail.com> parents: 40321 diff changeset	640
b0fbd1792e2d storageutil: extract most of peek_censored from revlog Gregory Szorc <gregory.szorc@gmail.com> parents: 40321 diff changeset	641 oldlen = baselenfn(baserev)
b0fbd1792e2d storageutil: extract most of peek_censored from revlog Gregory Szorc <gregory.szorc@gmail.com> parents: 40321 diff changeset	642 newlen = len(delta) - hlen
b0fbd1792e2d storageutil: extract most of peek_censored from revlog Gregory Szorc <gregory.szorc@gmail.com> parents: 40321 diff changeset	643 if delta[:hlen] != mdiff.replacediffheader(oldlen, newlen):
b0fbd1792e2d storageutil: extract most of peek_censored from revlog Gregory Szorc <gregory.szorc@gmail.com> parents: 40321 diff changeset	644 return False
b0fbd1792e2d storageutil: extract most of peek_censored from revlog Gregory Szorc <gregory.szorc@gmail.com> parents: 40321 diff changeset	645
43077 687b865b95ad formatting: byteify all mercurial/ and hgext/ string literals Augie Fackler <augie@google.com> parents: 43076 diff changeset	646 add = b"\1\ncensored:"
40325 b0fbd1792e2d storageutil: extract most of peek_censored from revlog Gregory Szorc <gregory.szorc@gmail.com> parents: 40321 diff changeset	647 addlen = len(add)
43076 2372284d9457 formatting: blacken the codebase Augie Fackler <augie@google.com> parents: 42813 diff changeset	648 return newlen >= addlen and delta[hlen : hlen + addlen] == add

Mercurial > public > mercurial-scm > hg

annotate mercurial/utils/storageutil.py @ 52049:af54626bf358