mercurial-scm/hg: mercurial/revlogutils/deltas.py annotate

annotate mercurial/revlogutils/deltas.py @ 51352:fac6038b11f5

delta-find: move the base of the delta search in its own function That logic is complicated enough that is is worth puting in its own function. Another method will be introduced in the next changeset to deal with the actual refining.

author	Pierre-Yves David <pierre-yves.david@octobus.net>
date	Thu, 23 Nov 2023 22:29:02 +0100
parents	94fe4474b053
children	701caeabbee7

rev	line source
39330 655b5b465953 revlog: split functionality related to deltas computation in a new module Boris Feld <boris.feld@octobus.net> parents: 39329 diff changeset	1 # revlogdeltas.py - Logic around delta computation for revlog
8226 8b2cd04a6e97 put license and copyright info into comment blocks Martin Geisler <mg@lazybytes.net> parents: 8225 diff changeset	2 #
46819 d4ba4d51f85f contributor: change mentions of mpm to olivia Rapha?l Gom?s <rgomes@octobus.net> parents: 43089 diff changeset	3 # Copyright 2005-2007 Olivia Mackall <olivia@selenic.com>
39330 655b5b465953 revlog: split functionality related to deltas computation in a new module Boris Feld <boris.feld@octobus.net> parents: 39329 diff changeset	4 # Copyright 2018 Octobus <contact@octobus.net>
8226 8b2cd04a6e97 put license and copyright info into comment blocks Martin Geisler <mg@lazybytes.net> parents: 8225 diff changeset	5 #
8b2cd04a6e97 put license and copyright info into comment blocks Martin Geisler <mg@lazybytes.net> parents: 8225 diff changeset	6 # This software may be used and distributed according to the terms of the
10263 25e572394f5c Update license to GPLv2+ Matt Mackall <mpm@selenic.com> parents: 10047 diff changeset	7 # GNU General Public License version 2 or any later version.
39330 655b5b465953 revlog: split functionality related to deltas computation in a new module Boris Feld <boris.feld@octobus.net> parents: 39329 diff changeset	8 """Helper class to compute deltas stored inside revlogs"""
8227 0a9542703300 turn some comments back into module docstrings Martin Geisler <mg@lazybytes.net> parents: 8226 diff changeset	9
27361 29f50344fa83 revlog: use absolute_import Gregory Szorc <gregory.szorc@gmail.com> parents: 27251 diff changeset	10
51343 2a333d791ecf delta-find: introduce a base class for _DeltaSearch Pierre-Yves David <pierre-yves.david@octobus.net> parents: 51342 diff changeset	11 import abc
39493 3ca144f1c8dd snapshot: search for unrelated but reusable full-snapshot Boris Feld <boris.feld@octobus.net> parents: 39492 diff changeset	12 import collections
27361 29f50344fa83 revlog: use absolute_import Gregory Szorc <gregory.szorc@gmail.com> parents: 27251 diff changeset	13 import struct
29f50344fa83 revlog: use absolute_import Gregory Szorc <gregory.szorc@gmail.com> parents: 27251 diff changeset	14
29f50344fa83 revlog: use absolute_import Gregory Szorc <gregory.szorc@gmail.com> parents: 27251 diff changeset	15 # import stuff from node for others to import from revlog
43076 2372284d9457 formatting: blacken the codebase Augie Fackler <augie@google.com> parents: 42992 diff changeset	16 from ..node import nullrev
39330 655b5b465953 revlog: split functionality related to deltas computation in a new module Boris Feld <boris.feld@octobus.net> parents: 39329 diff changeset	17 from ..i18n import _
655b5b465953 revlog: split functionality related to deltas computation in a new module Boris Feld <boris.feld@octobus.net> parents: 39329 diff changeset	18
655b5b465953 revlog: split functionality related to deltas computation in a new module Boris Feld <boris.feld@octobus.net> parents: 39329 diff changeset	19 from .constants import (
47452 c6844912c327 revlog: factor the logic to determine the delta compression out Pierre-Yves David <pierre-yves.david@octobus.net> parents: 47401 diff changeset	20 COMP_MODE_DEFAULT,
c6844912c327 revlog: factor the logic to determine the delta compression out Pierre-Yves David <pierre-yves.david@octobus.net> parents: 47401 diff changeset	21 COMP_MODE_INLINE,
c6844912c327 revlog: factor the logic to determine the delta compression out Pierre-Yves David <pierre-yves.david@octobus.net> parents: 47401 diff changeset	22 COMP_MODE_PLAIN,
49768 bcae90c53def delta-find: add a delta-reuse policy that blindly accepts incoming deltas Pierre-Yves David <pierre-yves.david@octobus.net> parents: 49681 diff changeset	23 DELTA_BASE_REUSE_FORCE,
49677 05db41701ece find-delta: pass the cache-delta usage policy alongside the cache-delta Pierre-Yves David <pierre-yves.david@octobus.net> parents: 49675 diff changeset	24 DELTA_BASE_REUSE_NO,
49226 e6b7c6fbeb48 deltas: add code to display information about the result of `finddeltainfo` Pierre-Yves David <pierre-yves.david@octobus.net> parents: 48946 diff changeset	25 KIND_CHANGELOG,
e6b7c6fbeb48 deltas: add code to display information about the result of `finddeltainfo` Pierre-Yves David <pierre-yves.david@octobus.net> parents: 48946 diff changeset	26 KIND_FILELOG,
e6b7c6fbeb48 deltas: add code to display information about the result of `finddeltainfo` Pierre-Yves David <pierre-yves.david@octobus.net> parents: 48946 diff changeset	27 KIND_MANIFESTLOG,
39329 729082bb9938 revlog: split constants into a new `revlogutils.constants` module Boris Feld <boris.feld@octobus.net> parents: 39232 diff changeset	28 REVIDX_ISCENSORED,
729082bb9938 revlog: split constants into a new `revlogutils.constants` module Boris Feld <boris.feld@octobus.net> parents: 39232 diff changeset	29 REVIDX_RAWTEXT_CHANGING_FLAGS,
729082bb9938 revlog: split constants into a new `revlogutils.constants` module Boris Feld <boris.feld@octobus.net> parents: 39232 diff changeset	30 )
39330 655b5b465953 revlog: split functionality related to deltas computation in a new module Boris Feld <boris.feld@octobus.net> parents: 39329 diff changeset	31
43076 2372284d9457 formatting: blacken the codebase Augie Fackler <augie@google.com> parents: 42992 diff changeset	32 from ..thirdparty import attr
39330 655b5b465953 revlog: split functionality related to deltas computation in a new module Boris Feld <boris.feld@octobus.net> parents: 39329 diff changeset	33
655b5b465953 revlog: split functionality related to deltas computation in a new module Boris Feld <boris.feld@octobus.net> parents: 39329 diff changeset	34 from .. import (
27361 29f50344fa83 revlog: use absolute_import Gregory Szorc <gregory.szorc@gmail.com> parents: 27251 diff changeset	35 error,
29f50344fa83 revlog: use absolute_import Gregory Szorc <gregory.szorc@gmail.com> parents: 27251 diff changeset	36 mdiff,
41108 38e88450138c delta: have a native implementation of _findsnapshot Boris Feld <boris.feld@octobus.net> parents: 41079 diff changeset	37 util,
27361 29f50344fa83 revlog: use absolute_import Gregory Szorc <gregory.szorc@gmail.com> parents: 27251 diff changeset	38 )
10913 f2ecc5733c89 revlog: factor out _maxinline global. Greg Ward <greg-hg@gerg.ca> parents: 10404 diff changeset	39
43076 2372284d9457 formatting: blacken the codebase Augie Fackler <augie@google.com> parents: 42992 diff changeset	40 from . import flagutil
42992 dff95420480f flagprocessors: make `processflagsraw` a module level function Pierre-Yves David <pierre-yves.david@octobus.net> parents: 42878 diff changeset	41
39330 655b5b465953 revlog: split functionality related to deltas computation in a new module Boris Feld <boris.feld@octobus.net> parents: 39329 diff changeset	42 # maximum <delta-chain-data>/<revision-text-length> ratio
655b5b465953 revlog: split functionality related to deltas computation in a new module Boris Feld <boris.feld@octobus.net> parents: 39329 diff changeset	43 LIMIT_DELTA2TEXT = 2
1091 d62130f99a73 Move hash function back to revlog from node mpm@selenic.com parents: 1089 diff changeset	44
43076 2372284d9457 formatting: blacken the codebase Augie Fackler <augie@google.com> parents: 42992 diff changeset	45
48946 642e31cb55f0 py3: use class X: instead of class X(object): Gregory Szorc <gregory.szorc@gmail.com> parents: 48875 diff changeset	46 class _testrevlog:
38637 e33f784f2a44 revlog: introduce a tiny mock of a revlog class Boris Feld <boris.feld@octobus.net> parents: 38636 diff changeset	47 """minimalist fake revlog to use in doctests"""
e33f784f2a44 revlog: introduce a tiny mock of a revlog class Boris Feld <boris.feld@octobus.net> parents: 38636 diff changeset	48
40641 85b14f0dc334 doctest: add a `issnapshot` method to _testrevlog Boris Feld <boris.feld@octobus.net> parents: 40608 diff changeset	49 def __init__(self, data, density=0.5, mingap=0, snapshot=()):
38637 e33f784f2a44 revlog: introduce a tiny mock of a revlog class Boris Feld <boris.feld@octobus.net> parents: 38636 diff changeset	50 """data is an list of revision payload boundaries"""
51042 127656e0b97b revlog: use the new Config classes in _testrevlog Pierre-Yves David <pierre-yves.david@octobus.net> parents: 51017 diff changeset	51 from .. import revlog
127656e0b97b revlog: use the new Config classes in _testrevlog Pierre-Yves David <pierre-yves.david@octobus.net> parents: 51017 diff changeset	52
38637 e33f784f2a44 revlog: introduce a tiny mock of a revlog class Boris Feld <boris.feld@octobus.net> parents: 38636 diff changeset	53 self._data = data
51042 127656e0b97b revlog: use the new Config classes in _testrevlog Pierre-Yves David <pierre-yves.david@octobus.net> parents: 51017 diff changeset	54 self.data_config = revlog.DataConfig()
127656e0b97b revlog: use the new Config classes in _testrevlog Pierre-Yves David <pierre-yves.david@octobus.net> parents: 51017 diff changeset	55 self.data_config.sr_density_threshold = density
127656e0b97b revlog: use the new Config classes in _testrevlog Pierre-Yves David <pierre-yves.david@octobus.net> parents: 51017 diff changeset	56 self.data_config.sr_min_gap_size = mingap
127656e0b97b revlog: use the new Config classes in _testrevlog Pierre-Yves David <pierre-yves.david@octobus.net> parents: 51017 diff changeset	57 self.delta_config = revlog.DeltaConfig()
127656e0b97b revlog: use the new Config classes in _testrevlog Pierre-Yves David <pierre-yves.david@octobus.net> parents: 51017 diff changeset	58 self.feature_config = revlog.FeatureConfig()
40641 85b14f0dc334 doctest: add a `issnapshot` method to _testrevlog Boris Feld <boris.feld@octobus.net> parents: 40608 diff changeset	59 self._snapshot = set(snapshot)
40709 39d29542fe40 sparse-revlog: put the native implementation of slicechunktodensity to use Boris Feld <boris.feld@octobus.net> parents: 40657 diff changeset	60 self.index = None
38637 e33f784f2a44 revlog: introduce a tiny mock of a revlog class Boris Feld <boris.feld@octobus.net> parents: 38636 diff changeset	61
e33f784f2a44 revlog: introduce a tiny mock of a revlog class Boris Feld <boris.feld@octobus.net> parents: 38636 diff changeset	62 def start(self, rev):
41079 88d813cd9acd revlog: fix pure python slicing test when chain contains nullrev Boris Feld <boris.feld@octobus.net> parents: 41033 diff changeset	63 if rev == nullrev:
88d813cd9acd revlog: fix pure python slicing test when chain contains nullrev Boris Feld <boris.feld@octobus.net> parents: 41033 diff changeset	64 return 0
38637 e33f784f2a44 revlog: introduce a tiny mock of a revlog class Boris Feld <boris.feld@octobus.net> parents: 38636 diff changeset	65 if rev == 0:
e33f784f2a44 revlog: introduce a tiny mock of a revlog class Boris Feld <boris.feld@octobus.net> parents: 38636 diff changeset	66 return 0
e33f784f2a44 revlog: introduce a tiny mock of a revlog class Boris Feld <boris.feld@octobus.net> parents: 38636 diff changeset	67 return self._data[rev - 1]
e33f784f2a44 revlog: introduce a tiny mock of a revlog class Boris Feld <boris.feld@octobus.net> parents: 38636 diff changeset	68
e33f784f2a44 revlog: introduce a tiny mock of a revlog class Boris Feld <boris.feld@octobus.net> parents: 38636 diff changeset	69 def end(self, rev):
41079 88d813cd9acd revlog: fix pure python slicing test when chain contains nullrev Boris Feld <boris.feld@octobus.net> parents: 41033 diff changeset	70 if rev == nullrev:
88d813cd9acd revlog: fix pure python slicing test when chain contains nullrev Boris Feld <boris.feld@octobus.net> parents: 41033 diff changeset	71 return 0
38637 e33f784f2a44 revlog: introduce a tiny mock of a revlog class Boris Feld <boris.feld@octobus.net> parents: 38636 diff changeset	72 return self._data[rev]
e33f784f2a44 revlog: introduce a tiny mock of a revlog class Boris Feld <boris.feld@octobus.net> parents: 38636 diff changeset	73
e33f784f2a44 revlog: introduce a tiny mock of a revlog class Boris Feld <boris.feld@octobus.net> parents: 38636 diff changeset	74 def length(self, rev):
e33f784f2a44 revlog: introduce a tiny mock of a revlog class Boris Feld <boris.feld@octobus.net> parents: 38636 diff changeset	75 return self.end(rev) - self.start(rev)
e33f784f2a44 revlog: introduce a tiny mock of a revlog class Boris Feld <boris.feld@octobus.net> parents: 38636 diff changeset	76
38718 f8762ea73e0d sparse-revlog: implement algorithm to write sparse delta chains (issue5480) Paul Morelle <paul.morelle@octobus.net> parents: 38717 diff changeset	77 def __len__(self):
f8762ea73e0d sparse-revlog: implement algorithm to write sparse delta chains (issue5480) Paul Morelle <paul.morelle@octobus.net> parents: 38717 diff changeset	78 return len(self._data)
f8762ea73e0d sparse-revlog: implement algorithm to write sparse delta chains (issue5480) Paul Morelle <paul.morelle@octobus.net> parents: 38717 diff changeset	79
40641 85b14f0dc334 doctest: add a `issnapshot` method to _testrevlog Boris Feld <boris.feld@octobus.net> parents: 40608 diff changeset	80 def issnapshot(self, rev):
41079 88d813cd9acd revlog: fix pure python slicing test when chain contains nullrev Boris Feld <boris.feld@octobus.net> parents: 41033 diff changeset	81 if rev == nullrev:
88d813cd9acd revlog: fix pure python slicing test when chain contains nullrev Boris Feld <boris.feld@octobus.net> parents: 41033 diff changeset	82 return True
40641 85b14f0dc334 doctest: add a `issnapshot` method to _testrevlog Boris Feld <boris.feld@octobus.net> parents: 40608 diff changeset	83 return rev in self._snapshot
85b14f0dc334 doctest: add a `issnapshot` method to _testrevlog Boris Feld <boris.feld@octobus.net> parents: 40608 diff changeset	84
43076 2372284d9457 formatting: blacken the codebase Augie Fackler <augie@google.com> parents: 42992 diff changeset	85
40604 3ac23dad6364 sparse-revlog: drop unused deltainfo parameter from _slicechunktodensity Boris Feld <boris.feld@octobus.net> parents: 40603 diff changeset	86 def slicechunk(revlog, revs, targetsize=None):
34824 e2ad93bcc084 revlog: introduce an experimental flag to slice chunks reads when too sparse Paul Morelle <paul.morelle@octobus.net> parents: 34823 diff changeset	87 """slice revs to reduce the amount of unrelated data to be read from disk.
e2ad93bcc084 revlog: introduce an experimental flag to slice chunks reads when too sparse Paul Morelle <paul.morelle@octobus.net> parents: 34823 diff changeset	88
e2ad93bcc084 revlog: introduce an experimental flag to slice chunks reads when too sparse Paul Morelle <paul.morelle@octobus.net> parents: 34823 diff changeset	89 ``revs`` is sliced into groups that should be read in one time.
e2ad93bcc084 revlog: introduce an experimental flag to slice chunks reads when too sparse Paul Morelle <paul.morelle@octobus.net> parents: 34823 diff changeset	90 Assume that revs are sorted.
38640 f62b8fb0a484 revlog: document and test _slicechunk Boris Feld <boris.feld@octobus.net> parents: 38639 diff changeset	91
f62b8fb0a484 revlog: document and test _slicechunk Boris Feld <boris.feld@octobus.net> parents: 38639 diff changeset	92 The initial chunk is sliced until the overall density (payload/chunks-span
51059 8c614fa16330 revlog: remove legacy usage of `_srdensitythreshold` Pierre-Yves David <pierre-yves.david@octobus.net> parents: 51057 diff changeset	93 ratio) is above `revlog.data_config.sr_density_threshold`. No gap smaller
51060 533d6943f6a3 revlog: remove legacy usage of `_srmingapsize` Pierre-Yves David <pierre-yves.david@octobus.net> parents: 51059 diff changeset	94 than `revlog.data_config.sr_min_gap_size` is skipped.
38640 f62b8fb0a484 revlog: document and test _slicechunk Boris Feld <boris.feld@octobus.net> parents: 38639 diff changeset	95
38643 967fee55e8d9 revlog: postprocess chunk to slice them down to a certain size Boris Feld <boris.feld@octobus.net> parents: 38642 diff changeset	96 If `targetsize` is set, no chunk larger than `targetsize` will be yield.
967fee55e8d9 revlog: postprocess chunk to slice them down to a certain size Boris Feld <boris.feld@octobus.net> parents: 38642 diff changeset	97 For consistency with other slicing choice, this limit won't go lower than
51060 533d6943f6a3 revlog: remove legacy usage of `_srmingapsize` Pierre-Yves David <pierre-yves.david@octobus.net> parents: 51059 diff changeset	98 `revlog.data_config.sr_min_gap_size`.
38643 967fee55e8d9 revlog: postprocess chunk to slice them down to a certain size Boris Feld <boris.feld@octobus.net> parents: 38642 diff changeset	99
967fee55e8d9 revlog: postprocess chunk to slice them down to a certain size Boris Feld <boris.feld@octobus.net> parents: 38642 diff changeset	100 If individual revisions chunk are larger than this limit, they will still
967fee55e8d9 revlog: postprocess chunk to slice them down to a certain size Boris Feld <boris.feld@octobus.net> parents: 38642 diff changeset	101 be raised individually.
967fee55e8d9 revlog: postprocess chunk to slice them down to a certain size Boris Feld <boris.feld@octobus.net> parents: 38642 diff changeset	102
40642 9c3c697267db sparse-revlog: rework the way we enforce chunk size limit Boris Feld <boris.feld@octobus.net> parents: 40641 diff changeset	103 >>> data = [
38640 f62b8fb0a484 revlog: document and test _slicechunk Boris Feld <boris.feld@octobus.net> parents: 38639 diff changeset	104 ... 5, #00 (5)
f62b8fb0a484 revlog: document and test _slicechunk Boris Feld <boris.feld@octobus.net> parents: 38639 diff changeset	105 ... 10, #01 (5)
f62b8fb0a484 revlog: document and test _slicechunk Boris Feld <boris.feld@octobus.net> parents: 38639 diff changeset	106 ... 12, #02 (2)
f62b8fb0a484 revlog: document and test _slicechunk Boris Feld <boris.feld@octobus.net> parents: 38639 diff changeset	107 ... 12, #03 (empty)
f62b8fb0a484 revlog: document and test _slicechunk Boris Feld <boris.feld@octobus.net> parents: 38639 diff changeset	108 ... 27, #04 (15)
f62b8fb0a484 revlog: document and test _slicechunk Boris Feld <boris.feld@octobus.net> parents: 38639 diff changeset	109 ... 31, #05 (4)
f62b8fb0a484 revlog: document and test _slicechunk Boris Feld <boris.feld@octobus.net> parents: 38639 diff changeset	110 ... 31, #06 (empty)
f62b8fb0a484 revlog: document and test _slicechunk Boris Feld <boris.feld@octobus.net> parents: 38639 diff changeset	111 ... 42, #07 (11)
f62b8fb0a484 revlog: document and test _slicechunk Boris Feld <boris.feld@octobus.net> parents: 38639 diff changeset	112 ... 47, #08 (5)
f62b8fb0a484 revlog: document and test _slicechunk Boris Feld <boris.feld@octobus.net> parents: 38639 diff changeset	113 ... 47, #09 (empty)
f62b8fb0a484 revlog: document and test _slicechunk Boris Feld <boris.feld@octobus.net> parents: 38639 diff changeset	114 ... 48, #10 (1)
f62b8fb0a484 revlog: document and test _slicechunk Boris Feld <boris.feld@octobus.net> parents: 38639 diff changeset	115 ... 51, #11 (3)
f62b8fb0a484 revlog: document and test _slicechunk Boris Feld <boris.feld@octobus.net> parents: 38639 diff changeset	116 ... 74, #12 (23)
f62b8fb0a484 revlog: document and test _slicechunk Boris Feld <boris.feld@octobus.net> parents: 38639 diff changeset	117 ... 85, #13 (11)
f62b8fb0a484 revlog: document and test _slicechunk Boris Feld <boris.feld@octobus.net> parents: 38639 diff changeset	118 ... 86, #14 (1)
f62b8fb0a484 revlog: document and test _slicechunk Boris Feld <boris.feld@octobus.net> parents: 38639 diff changeset	119 ... 91, #15 (5)
40642 9c3c697267db sparse-revlog: rework the way we enforce chunk size limit Boris Feld <boris.feld@octobus.net> parents: 40641 diff changeset	120 ... ]
9c3c697267db sparse-revlog: rework the way we enforce chunk size limit Boris Feld <boris.feld@octobus.net> parents: 40641 diff changeset	121 >>> revlog = _testrevlog(data, snapshot=range(16))
38640 f62b8fb0a484 revlog: document and test _slicechunk Boris Feld <boris.feld@octobus.net> parents: 38639 diff changeset	122
39330 655b5b465953 revlog: split functionality related to deltas computation in a new module Boris Feld <boris.feld@octobus.net> parents: 39329 diff changeset	123 >>> list(slicechunk(revlog, list(range(16))))
38640 f62b8fb0a484 revlog: document and test _slicechunk Boris Feld <boris.feld@octobus.net> parents: 38639 diff changeset	124 [[0, 1, 2, 3, 4, 5, 6, 7, 8, 9, 10, 11, 12, 13, 14, 15]]
39330 655b5b465953 revlog: split functionality related to deltas computation in a new module Boris Feld <boris.feld@octobus.net> parents: 39329 diff changeset	125 >>> list(slicechunk(revlog, [0, 15]))
38640 f62b8fb0a484 revlog: document and test _slicechunk Boris Feld <boris.feld@octobus.net> parents: 38639 diff changeset	126 [[0], [15]]
39330 655b5b465953 revlog: split functionality related to deltas computation in a new module Boris Feld <boris.feld@octobus.net> parents: 39329 diff changeset	127 >>> list(slicechunk(revlog, [0, 11, 15]))
38640 f62b8fb0a484 revlog: document and test _slicechunk Boris Feld <boris.feld@octobus.net> parents: 38639 diff changeset	128 [[0], [11], [15]]
39330 655b5b465953 revlog: split functionality related to deltas computation in a new module Boris Feld <boris.feld@octobus.net> parents: 39329 diff changeset	129 >>> list(slicechunk(revlog, [0, 11, 13, 15]))
38640 f62b8fb0a484 revlog: document and test _slicechunk Boris Feld <boris.feld@octobus.net> parents: 38639 diff changeset	130 [[0], [11, 13, 15]]
39330 655b5b465953 revlog: split functionality related to deltas computation in a new module Boris Feld <boris.feld@octobus.net> parents: 39329 diff changeset	131 >>> list(slicechunk(revlog, [1, 2, 3, 5, 8, 10, 11, 14]))
38640 f62b8fb0a484 revlog: document and test _slicechunk Boris Feld <boris.feld@octobus.net> parents: 38639 diff changeset	132 [[1, 2], [5, 8, 10, 11], [14]]
38643 967fee55e8d9 revlog: postprocess chunk to slice them down to a certain size Boris Feld <boris.feld@octobus.net> parents: 38642 diff changeset	133
967fee55e8d9 revlog: postprocess chunk to slice them down to a certain size Boris Feld <boris.feld@octobus.net> parents: 38642 diff changeset	134 Slicing with a maximum chunk size
39330 655b5b465953 revlog: split functionality related to deltas computation in a new module Boris Feld <boris.feld@octobus.net> parents: 39329 diff changeset	135 >>> list(slicechunk(revlog, [0, 11, 13, 15], targetsize=15))
38643 967fee55e8d9 revlog: postprocess chunk to slice them down to a certain size Boris Feld <boris.feld@octobus.net> parents: 38642 diff changeset	136 [[0], [11], [13], [15]]
39330 655b5b465953 revlog: split functionality related to deltas computation in a new module Boris Feld <boris.feld@octobus.net> parents: 39329 diff changeset	137 >>> list(slicechunk(revlog, [0, 11, 13, 15], targetsize=20))
38643 967fee55e8d9 revlog: postprocess chunk to slice them down to a certain size Boris Feld <boris.feld@octobus.net> parents: 38642 diff changeset	138 [[0], [11], [13, 15]]
41079 88d813cd9acd revlog: fix pure python slicing test when chain contains nullrev Boris Feld <boris.feld@octobus.net> parents: 41033 diff changeset	139
88d813cd9acd revlog: fix pure python slicing test when chain contains nullrev Boris Feld <boris.feld@octobus.net> parents: 41033 diff changeset	140 Slicing involving nullrev
88d813cd9acd revlog: fix pure python slicing test when chain contains nullrev Boris Feld <boris.feld@octobus.net> parents: 41033 diff changeset	141 >>> list(slicechunk(revlog, [-1, 0, 11, 13, 15], targetsize=20))
88d813cd9acd revlog: fix pure python slicing test when chain contains nullrev Boris Feld <boris.feld@octobus.net> parents: 41033 diff changeset	142 [[-1, 0], [11], [13, 15]]
88d813cd9acd revlog: fix pure python slicing test when chain contains nullrev Boris Feld <boris.feld@octobus.net> parents: 41033 diff changeset	143 >>> list(slicechunk(revlog, [-1, 13, 15], targetsize=5))
88d813cd9acd revlog: fix pure python slicing test when chain contains nullrev Boris Feld <boris.feld@octobus.net> parents: 41033 diff changeset	144 [[-1], [13], [15]]
34824 e2ad93bcc084 revlog: introduce an experimental flag to slice chunks reads when too sparse Paul Morelle <paul.morelle@octobus.net> parents: 34823 diff changeset	145 """
38643 967fee55e8d9 revlog: postprocess chunk to slice them down to a certain size Boris Feld <boris.feld@octobus.net> parents: 38642 diff changeset	146 if targetsize is not None:
51060 533d6943f6a3 revlog: remove legacy usage of `_srmingapsize` Pierre-Yves David <pierre-yves.david@octobus.net> parents: 51059 diff changeset	147 targetsize = max(targetsize, revlog.data_config.sr_min_gap_size)
38718 f8762ea73e0d sparse-revlog: implement algorithm to write sparse delta chains (issue5480) Paul Morelle <paul.morelle@octobus.net> parents: 38717 diff changeset	148 # targetsize should not be specified when evaluating delta candidates:
f8762ea73e0d sparse-revlog: implement algorithm to write sparse delta chains (issue5480) Paul Morelle <paul.morelle@octobus.net> parents: 38717 diff changeset	149 # * targetsize is used to ensure we stay within specification when reading,
40709 39d29542fe40 sparse-revlog: put the native implementation of slicechunktodensity to use Boris Feld <boris.feld@octobus.net> parents: 40657 diff changeset	150 densityslicing = getattr(revlog.index, 'slicechunktodensity', None)
39d29542fe40 sparse-revlog: put the native implementation of slicechunktodensity to use Boris Feld <boris.feld@octobus.net> parents: 40657 diff changeset	151 if densityslicing is None:
39d29542fe40 sparse-revlog: put the native implementation of slicechunktodensity to use Boris Feld <boris.feld@octobus.net> parents: 40657 diff changeset	152 densityslicing = lambda x, y, z: _slicechunktodensity(revlog, x, y, z)
43076 2372284d9457 formatting: blacken the codebase Augie Fackler <augie@google.com> parents: 42992 diff changeset	153 for chunk in densityslicing(
51060 533d6943f6a3 revlog: remove legacy usage of `_srmingapsize` Pierre-Yves David <pierre-yves.david@octobus.net> parents: 51059 diff changeset	154 revs,
533d6943f6a3 revlog: remove legacy usage of `_srmingapsize` Pierre-Yves David <pierre-yves.david@octobus.net> parents: 51059 diff changeset	155 revlog.data_config.sr_density_threshold,
533d6943f6a3 revlog: remove legacy usage of `_srmingapsize` Pierre-Yves David <pierre-yves.david@octobus.net> parents: 51059 diff changeset	156 revlog.data_config.sr_min_gap_size,
43076 2372284d9457 formatting: blacken the codebase Augie Fackler <augie@google.com> parents: 42992 diff changeset	157 ):
38643 967fee55e8d9 revlog: postprocess chunk to slice them down to a certain size Boris Feld <boris.feld@octobus.net> parents: 38642 diff changeset	158 for subchunk in _slicechunktosize(revlog, chunk, targetsize):
967fee55e8d9 revlog: postprocess chunk to slice them down to a certain size Boris Feld <boris.feld@octobus.net> parents: 38642 diff changeset	159 yield subchunk
38641 feba6be0941b revlog: extract density based slicing into its own function Boris Feld <boris.feld@octobus.net> parents: 38640 diff changeset	160
43076 2372284d9457 formatting: blacken the codebase Augie Fackler <augie@google.com> parents: 42992 diff changeset	161
38718 f8762ea73e0d sparse-revlog: implement algorithm to write sparse delta chains (issue5480) Paul Morelle <paul.morelle@octobus.net> parents: 38717 diff changeset	162 def _slicechunktosize(revlog, revs, targetsize=None):
38642 e59e27e52297 revlog: add function to slice chunk down to a given size Boris Feld <boris.feld@octobus.net> parents: 38641 diff changeset	163 """slice revs to match the target size
e59e27e52297 revlog: add function to slice chunk down to a given size Boris Feld <boris.feld@octobus.net> parents: 38641 diff changeset	164
e59e27e52297 revlog: add function to slice chunk down to a given size Boris Feld <boris.feld@octobus.net> parents: 38641 diff changeset	165 This is intended to be used on chunk that density slicing selected by that
e59e27e52297 revlog: add function to slice chunk down to a given size Boris Feld <boris.feld@octobus.net> parents: 38641 diff changeset	166 are still too large compared to the read garantee of revlog. This might
e59e27e52297 revlog: add function to slice chunk down to a given size Boris Feld <boris.feld@octobus.net> parents: 38641 diff changeset	167 happens when "minimal gap size" interrupted the slicing or when chain are
e59e27e52297 revlog: add function to slice chunk down to a given size Boris Feld <boris.feld@octobus.net> parents: 38641 diff changeset	168 built in a way that create large blocks next to each other.
e59e27e52297 revlog: add function to slice chunk down to a given size Boris Feld <boris.feld@octobus.net> parents: 38641 diff changeset	169
40642 9c3c697267db sparse-revlog: rework the way we enforce chunk size limit Boris Feld <boris.feld@octobus.net> parents: 40641 diff changeset	170 >>> data = [
38642 e59e27e52297 revlog: add function to slice chunk down to a given size Boris Feld <boris.feld@octobus.net> parents: 38641 diff changeset	171 ... 3, #0 (3)
e59e27e52297 revlog: add function to slice chunk down to a given size Boris Feld <boris.feld@octobus.net> parents: 38641 diff changeset	172 ... 5, #1 (2)
e59e27e52297 revlog: add function to slice chunk down to a given size Boris Feld <boris.feld@octobus.net> parents: 38641 diff changeset	173 ... 6, #2 (1)
e59e27e52297 revlog: add function to slice chunk down to a given size Boris Feld <boris.feld@octobus.net> parents: 38641 diff changeset	174 ... 8, #3 (2)
e59e27e52297 revlog: add function to slice chunk down to a given size Boris Feld <boris.feld@octobus.net> parents: 38641 diff changeset	175 ... 8, #4 (empty)
e59e27e52297 revlog: add function to slice chunk down to a given size Boris Feld <boris.feld@octobus.net> parents: 38641 diff changeset	176 ... 11, #5 (3)
e59e27e52297 revlog: add function to slice chunk down to a given size Boris Feld <boris.feld@octobus.net> parents: 38641 diff changeset	177 ... 12, #6 (1)
e59e27e52297 revlog: add function to slice chunk down to a given size Boris Feld <boris.feld@octobus.net> parents: 38641 diff changeset	178 ... 13, #7 (1)
e59e27e52297 revlog: add function to slice chunk down to a given size Boris Feld <boris.feld@octobus.net> parents: 38641 diff changeset	179 ... 14, #8 (1)
40642 9c3c697267db sparse-revlog: rework the way we enforce chunk size limit Boris Feld <boris.feld@octobus.net> parents: 40641 diff changeset	180 ... ]
9c3c697267db sparse-revlog: rework the way we enforce chunk size limit Boris Feld <boris.feld@octobus.net> parents: 40641 diff changeset	181
9c3c697267db sparse-revlog: rework the way we enforce chunk size limit Boris Feld <boris.feld@octobus.net> parents: 40641 diff changeset	182 == All snapshots cases ==
9c3c697267db sparse-revlog: rework the way we enforce chunk size limit Boris Feld <boris.feld@octobus.net> parents: 40641 diff changeset	183 >>> revlog = _testrevlog(data, snapshot=range(9))
38642 e59e27e52297 revlog: add function to slice chunk down to a given size Boris Feld <boris.feld@octobus.net> parents: 38641 diff changeset	184
e59e27e52297 revlog: add function to slice chunk down to a given size Boris Feld <boris.feld@octobus.net> parents: 38641 diff changeset	185 Cases where chunk is already small enough
e59e27e52297 revlog: add function to slice chunk down to a given size Boris Feld <boris.feld@octobus.net> parents: 38641 diff changeset	186 >>> list(_slicechunktosize(revlog, [0], 3))
e59e27e52297 revlog: add function to slice chunk down to a given size Boris Feld <boris.feld@octobus.net> parents: 38641 diff changeset	187 [[0]]
e59e27e52297 revlog: add function to slice chunk down to a given size Boris Feld <boris.feld@octobus.net> parents: 38641 diff changeset	188 >>> list(_slicechunktosize(revlog, [6, 7], 3))
e59e27e52297 revlog: add function to slice chunk down to a given size Boris Feld <boris.feld@octobus.net> parents: 38641 diff changeset	189 [[6, 7]]
e59e27e52297 revlog: add function to slice chunk down to a given size Boris Feld <boris.feld@octobus.net> parents: 38641 diff changeset	190 >>> list(_slicechunktosize(revlog, [0], None))
e59e27e52297 revlog: add function to slice chunk down to a given size Boris Feld <boris.feld@octobus.net> parents: 38641 diff changeset	191 [[0]]
e59e27e52297 revlog: add function to slice chunk down to a given size Boris Feld <boris.feld@octobus.net> parents: 38641 diff changeset	192 >>> list(_slicechunktosize(revlog, [6, 7], None))
e59e27e52297 revlog: add function to slice chunk down to a given size Boris Feld <boris.feld@octobus.net> parents: 38641 diff changeset	193 [[6, 7]]
e59e27e52297 revlog: add function to slice chunk down to a given size Boris Feld <boris.feld@octobus.net> parents: 38641 diff changeset	194
e59e27e52297 revlog: add function to slice chunk down to a given size Boris Feld <boris.feld@octobus.net> parents: 38641 diff changeset	195 cases where we need actual slicing
e59e27e52297 revlog: add function to slice chunk down to a given size Boris Feld <boris.feld@octobus.net> parents: 38641 diff changeset	196 >>> list(_slicechunktosize(revlog, [0, 1], 3))
e59e27e52297 revlog: add function to slice chunk down to a given size Boris Feld <boris.feld@octobus.net> parents: 38641 diff changeset	197 [[0], [1]]
e59e27e52297 revlog: add function to slice chunk down to a given size Boris Feld <boris.feld@octobus.net> parents: 38641 diff changeset	198 >>> list(_slicechunktosize(revlog, [1, 3], 3))
e59e27e52297 revlog: add function to slice chunk down to a given size Boris Feld <boris.feld@octobus.net> parents: 38641 diff changeset	199 [[1], [3]]
e59e27e52297 revlog: add function to slice chunk down to a given size Boris Feld <boris.feld@octobus.net> parents: 38641 diff changeset	200 >>> list(_slicechunktosize(revlog, [1, 2, 3], 3))
e59e27e52297 revlog: add function to slice chunk down to a given size Boris Feld <boris.feld@octobus.net> parents: 38641 diff changeset	201 [[1, 2], [3]]
e59e27e52297 revlog: add function to slice chunk down to a given size Boris Feld <boris.feld@octobus.net> parents: 38641 diff changeset	202 >>> list(_slicechunktosize(revlog, [3, 5], 3))
e59e27e52297 revlog: add function to slice chunk down to a given size Boris Feld <boris.feld@octobus.net> parents: 38641 diff changeset	203 [[3], [5]]
e59e27e52297 revlog: add function to slice chunk down to a given size Boris Feld <boris.feld@octobus.net> parents: 38641 diff changeset	204 >>> list(_slicechunktosize(revlog, [3, 4, 5], 3))
e59e27e52297 revlog: add function to slice chunk down to a given size Boris Feld <boris.feld@octobus.net> parents: 38641 diff changeset	205 [[3], [5]]
e59e27e52297 revlog: add function to slice chunk down to a given size Boris Feld <boris.feld@octobus.net> parents: 38641 diff changeset	206 >>> list(_slicechunktosize(revlog, [5, 6, 7, 8], 3))
e59e27e52297 revlog: add function to slice chunk down to a given size Boris Feld <boris.feld@octobus.net> parents: 38641 diff changeset	207 [[5], [6, 7, 8]]
e59e27e52297 revlog: add function to slice chunk down to a given size Boris Feld <boris.feld@octobus.net> parents: 38641 diff changeset	208 >>> list(_slicechunktosize(revlog, [0, 1, 2, 3, 4, 5, 6, 7, 8], 3))
e59e27e52297 revlog: add function to slice chunk down to a given size Boris Feld <boris.feld@octobus.net> parents: 38641 diff changeset	209 [[0], [1, 2], [3], [5], [6, 7, 8]]
e59e27e52297 revlog: add function to slice chunk down to a given size Boris Feld <boris.feld@octobus.net> parents: 38641 diff changeset	210
e59e27e52297 revlog: add function to slice chunk down to a given size Boris Feld <boris.feld@octobus.net> parents: 38641 diff changeset	211 Case with too large individual chunk (must return valid chunk)
e59e27e52297 revlog: add function to slice chunk down to a given size Boris Feld <boris.feld@octobus.net> parents: 38641 diff changeset	212 >>> list(_slicechunktosize(revlog, [0, 1], 2))
e59e27e52297 revlog: add function to slice chunk down to a given size Boris Feld <boris.feld@octobus.net> parents: 38641 diff changeset	213 [[0], [1]]
e59e27e52297 revlog: add function to slice chunk down to a given size Boris Feld <boris.feld@octobus.net> parents: 38641 diff changeset	214 >>> list(_slicechunktosize(revlog, [1, 3], 1))
e59e27e52297 revlog: add function to slice chunk down to a given size Boris Feld <boris.feld@octobus.net> parents: 38641 diff changeset	215 [[1], [3]]
e59e27e52297 revlog: add function to slice chunk down to a given size Boris Feld <boris.feld@octobus.net> parents: 38641 diff changeset	216 >>> list(_slicechunktosize(revlog, [3, 4, 5], 2))
e59e27e52297 revlog: add function to slice chunk down to a given size Boris Feld <boris.feld@octobus.net> parents: 38641 diff changeset	217 [[3], [5]]
40642 9c3c697267db sparse-revlog: rework the way we enforce chunk size limit Boris Feld <boris.feld@octobus.net> parents: 40641 diff changeset	218
9c3c697267db sparse-revlog: rework the way we enforce chunk size limit Boris Feld <boris.feld@octobus.net> parents: 40641 diff changeset	219 == No Snapshot cases ==
9c3c697267db sparse-revlog: rework the way we enforce chunk size limit Boris Feld <boris.feld@octobus.net> parents: 40641 diff changeset	220 >>> revlog = _testrevlog(data)
9c3c697267db sparse-revlog: rework the way we enforce chunk size limit Boris Feld <boris.feld@octobus.net> parents: 40641 diff changeset	221
9c3c697267db sparse-revlog: rework the way we enforce chunk size limit Boris Feld <boris.feld@octobus.net> parents: 40641 diff changeset	222 Cases where chunk is already small enough
9c3c697267db sparse-revlog: rework the way we enforce chunk size limit Boris Feld <boris.feld@octobus.net> parents: 40641 diff changeset	223 >>> list(_slicechunktosize(revlog, [0], 3))
9c3c697267db sparse-revlog: rework the way we enforce chunk size limit Boris Feld <boris.feld@octobus.net> parents: 40641 diff changeset	224 [[0]]
9c3c697267db sparse-revlog: rework the way we enforce chunk size limit Boris Feld <boris.feld@octobus.net> parents: 40641 diff changeset	225 >>> list(_slicechunktosize(revlog, [6, 7], 3))
9c3c697267db sparse-revlog: rework the way we enforce chunk size limit Boris Feld <boris.feld@octobus.net> parents: 40641 diff changeset	226 [[6, 7]]
9c3c697267db sparse-revlog: rework the way we enforce chunk size limit Boris Feld <boris.feld@octobus.net> parents: 40641 diff changeset	227 >>> list(_slicechunktosize(revlog, [0], None))
9c3c697267db sparse-revlog: rework the way we enforce chunk size limit Boris Feld <boris.feld@octobus.net> parents: 40641 diff changeset	228 [[0]]
9c3c697267db sparse-revlog: rework the way we enforce chunk size limit Boris Feld <boris.feld@octobus.net> parents: 40641 diff changeset	229 >>> list(_slicechunktosize(revlog, [6, 7], None))
9c3c697267db sparse-revlog: rework the way we enforce chunk size limit Boris Feld <boris.feld@octobus.net> parents: 40641 diff changeset	230 [[6, 7]]
9c3c697267db sparse-revlog: rework the way we enforce chunk size limit Boris Feld <boris.feld@octobus.net> parents: 40641 diff changeset	231
9c3c697267db sparse-revlog: rework the way we enforce chunk size limit Boris Feld <boris.feld@octobus.net> parents: 40641 diff changeset	232 cases where we need actual slicing
9c3c697267db sparse-revlog: rework the way we enforce chunk size limit Boris Feld <boris.feld@octobus.net> parents: 40641 diff changeset	233 >>> list(_slicechunktosize(revlog, [0, 1], 3))
9c3c697267db sparse-revlog: rework the way we enforce chunk size limit Boris Feld <boris.feld@octobus.net> parents: 40641 diff changeset	234 [[0], [1]]
9c3c697267db sparse-revlog: rework the way we enforce chunk size limit Boris Feld <boris.feld@octobus.net> parents: 40641 diff changeset	235 >>> list(_slicechunktosize(revlog, [1, 3], 3))
9c3c697267db sparse-revlog: rework the way we enforce chunk size limit Boris Feld <boris.feld@octobus.net> parents: 40641 diff changeset	236 [[1], [3]]
9c3c697267db sparse-revlog: rework the way we enforce chunk size limit Boris Feld <boris.feld@octobus.net> parents: 40641 diff changeset	237 >>> list(_slicechunktosize(revlog, [1, 2, 3], 3))
9c3c697267db sparse-revlog: rework the way we enforce chunk size limit Boris Feld <boris.feld@octobus.net> parents: 40641 diff changeset	238 [[1], [2, 3]]
9c3c697267db sparse-revlog: rework the way we enforce chunk size limit Boris Feld <boris.feld@octobus.net> parents: 40641 diff changeset	239 >>> list(_slicechunktosize(revlog, [3, 5], 3))
9c3c697267db sparse-revlog: rework the way we enforce chunk size limit Boris Feld <boris.feld@octobus.net> parents: 40641 diff changeset	240 [[3], [5]]
9c3c697267db sparse-revlog: rework the way we enforce chunk size limit Boris Feld <boris.feld@octobus.net> parents: 40641 diff changeset	241 >>> list(_slicechunktosize(revlog, [3, 4, 5], 3))
9c3c697267db sparse-revlog: rework the way we enforce chunk size limit Boris Feld <boris.feld@octobus.net> parents: 40641 diff changeset	242 [[3], [4, 5]]
9c3c697267db sparse-revlog: rework the way we enforce chunk size limit Boris Feld <boris.feld@octobus.net> parents: 40641 diff changeset	243 >>> list(_slicechunktosize(revlog, [5, 6, 7, 8], 3))
9c3c697267db sparse-revlog: rework the way we enforce chunk size limit Boris Feld <boris.feld@octobus.net> parents: 40641 diff changeset	244 [[5], [6, 7, 8]]
9c3c697267db sparse-revlog: rework the way we enforce chunk size limit Boris Feld <boris.feld@octobus.net> parents: 40641 diff changeset	245 >>> list(_slicechunktosize(revlog, [0, 1, 2, 3, 4, 5, 6, 7, 8], 3))
9c3c697267db sparse-revlog: rework the way we enforce chunk size limit Boris Feld <boris.feld@octobus.net> parents: 40641 diff changeset	246 [[0], [1, 2], [3], [5], [6, 7, 8]]
9c3c697267db sparse-revlog: rework the way we enforce chunk size limit Boris Feld <boris.feld@octobus.net> parents: 40641 diff changeset	247
9c3c697267db sparse-revlog: rework the way we enforce chunk size limit Boris Feld <boris.feld@octobus.net> parents: 40641 diff changeset	248 Case with too large individual chunk (must return valid chunk)
9c3c697267db sparse-revlog: rework the way we enforce chunk size limit Boris Feld <boris.feld@octobus.net> parents: 40641 diff changeset	249 >>> list(_slicechunktosize(revlog, [0, 1], 2))
9c3c697267db sparse-revlog: rework the way we enforce chunk size limit Boris Feld <boris.feld@octobus.net> parents: 40641 diff changeset	250 [[0], [1]]
9c3c697267db sparse-revlog: rework the way we enforce chunk size limit Boris Feld <boris.feld@octobus.net> parents: 40641 diff changeset	251 >>> list(_slicechunktosize(revlog, [1, 3], 1))
9c3c697267db sparse-revlog: rework the way we enforce chunk size limit Boris Feld <boris.feld@octobus.net> parents: 40641 diff changeset	252 [[1], [3]]
9c3c697267db sparse-revlog: rework the way we enforce chunk size limit Boris Feld <boris.feld@octobus.net> parents: 40641 diff changeset	253 >>> list(_slicechunktosize(revlog, [3, 4, 5], 2))
9c3c697267db sparse-revlog: rework the way we enforce chunk size limit Boris Feld <boris.feld@octobus.net> parents: 40641 diff changeset	254 [[3], [5]]
9c3c697267db sparse-revlog: rework the way we enforce chunk size limit Boris Feld <boris.feld@octobus.net> parents: 40641 diff changeset	255
9c3c697267db sparse-revlog: rework the way we enforce chunk size limit Boris Feld <boris.feld@octobus.net> parents: 40641 diff changeset	256 == mixed case ==
9c3c697267db sparse-revlog: rework the way we enforce chunk size limit Boris Feld <boris.feld@octobus.net> parents: 40641 diff changeset	257 >>> revlog = _testrevlog(data, snapshot=[0, 1, 2])
9c3c697267db sparse-revlog: rework the way we enforce chunk size limit Boris Feld <boris.feld@octobus.net> parents: 40641 diff changeset	258 >>> list(_slicechunktosize(revlog, list(range(9)), 5))
9c3c697267db sparse-revlog: rework the way we enforce chunk size limit Boris Feld <boris.feld@octobus.net> parents: 40641 diff changeset	259 [[0, 1], [2], [3, 4, 5], [6, 7, 8]]
38642 e59e27e52297 revlog: add function to slice chunk down to a given size Boris Feld <boris.feld@octobus.net> parents: 38641 diff changeset	260 """
e59e27e52297 revlog: add function to slice chunk down to a given size Boris Feld <boris.feld@octobus.net> parents: 38641 diff changeset	261 assert targetsize is None or 0 <= targetsize
40642 9c3c697267db sparse-revlog: rework the way we enforce chunk size limit Boris Feld <boris.feld@octobus.net> parents: 40641 diff changeset	262 startdata = revlog.start(revs[0])
9c3c697267db sparse-revlog: rework the way we enforce chunk size limit Boris Feld <boris.feld@octobus.net> parents: 40641 diff changeset	263 enddata = revlog.end(revs[-1])
9c3c697267db sparse-revlog: rework the way we enforce chunk size limit Boris Feld <boris.feld@octobus.net> parents: 40641 diff changeset	264 fullspan = enddata - startdata
9c3c697267db sparse-revlog: rework the way we enforce chunk size limit Boris Feld <boris.feld@octobus.net> parents: 40641 diff changeset	265 if targetsize is None or fullspan <= targetsize:
38642 e59e27e52297 revlog: add function to slice chunk down to a given size Boris Feld <boris.feld@octobus.net> parents: 38641 diff changeset	266 yield revs
e59e27e52297 revlog: add function to slice chunk down to a given size Boris Feld <boris.feld@octobus.net> parents: 38641 diff changeset	267 return
e59e27e52297 revlog: add function to slice chunk down to a given size Boris Feld <boris.feld@octobus.net> parents: 38641 diff changeset	268
e59e27e52297 revlog: add function to slice chunk down to a given size Boris Feld <boris.feld@octobus.net> parents: 38641 diff changeset	269 startrevidx = 0
40657 2eb48aa0acce sparse-revlog: align endrevidx usages in the _slicechunktosize Boris Feld <boris.feld@octobus.net> parents: 40654 diff changeset	270 endrevidx = 1
38642 e59e27e52297 revlog: add function to slice chunk down to a given size Boris Feld <boris.feld@octobus.net> parents: 38641 diff changeset	271 iterrevs = enumerate(revs)
43076 2372284d9457 formatting: blacken the codebase Augie Fackler <augie@google.com> parents: 42992 diff changeset	272 next(iterrevs) # skip first rev.
40642 9c3c697267db sparse-revlog: rework the way we enforce chunk size limit Boris Feld <boris.feld@octobus.net> parents: 40641 diff changeset	273 # first step: get snapshots out of the way
38642 e59e27e52297 revlog: add function to slice chunk down to a given size Boris Feld <boris.feld@octobus.net> parents: 38641 diff changeset	274 for idx, r in iterrevs:
e59e27e52297 revlog: add function to slice chunk down to a given size Boris Feld <boris.feld@octobus.net> parents: 38641 diff changeset	275 span = revlog.end(r) - startdata
40642 9c3c697267db sparse-revlog: rework the way we enforce chunk size limit Boris Feld <boris.feld@octobus.net> parents: 40641 diff changeset	276 snapshot = revlog.issnapshot(r)
9c3c697267db sparse-revlog: rework the way we enforce chunk size limit Boris Feld <boris.feld@octobus.net> parents: 40641 diff changeset	277 if span <= targetsize and snapshot:
40657 2eb48aa0acce sparse-revlog: align endrevidx usages in the _slicechunktosize Boris Feld <boris.feld@octobus.net> parents: 40654 diff changeset	278 endrevidx = idx + 1
38642 e59e27e52297 revlog: add function to slice chunk down to a given size Boris Feld <boris.feld@octobus.net> parents: 38641 diff changeset	279 else:
40657 2eb48aa0acce sparse-revlog: align endrevidx usages in the _slicechunktosize Boris Feld <boris.feld@octobus.net> parents: 40654 diff changeset	280 chunk = _trimchunk(revlog, revs, startrevidx, endrevidx)
38642 e59e27e52297 revlog: add function to slice chunk down to a given size Boris Feld <boris.feld@octobus.net> parents: 38641 diff changeset	281 if chunk:
e59e27e52297 revlog: add function to slice chunk down to a given size Boris Feld <boris.feld@octobus.net> parents: 38641 diff changeset	282 yield chunk
e59e27e52297 revlog: add function to slice chunk down to a given size Boris Feld <boris.feld@octobus.net> parents: 38641 diff changeset	283 startrevidx = idx
e59e27e52297 revlog: add function to slice chunk down to a given size Boris Feld <boris.feld@octobus.net> parents: 38641 diff changeset	284 startdata = revlog.start(r)
40657 2eb48aa0acce sparse-revlog: align endrevidx usages in the _slicechunktosize Boris Feld <boris.feld@octobus.net> parents: 40654 diff changeset	285 endrevidx = idx + 1
40642 9c3c697267db sparse-revlog: rework the way we enforce chunk size limit Boris Feld <boris.feld@octobus.net> parents: 40641 diff changeset	286 if not snapshot:
9c3c697267db sparse-revlog: rework the way we enforce chunk size limit Boris Feld <boris.feld@octobus.net> parents: 40641 diff changeset	287 break
9c3c697267db sparse-revlog: rework the way we enforce chunk size limit Boris Feld <boris.feld@octobus.net> parents: 40641 diff changeset	288
9c3c697267db sparse-revlog: rework the way we enforce chunk size limit Boris Feld <boris.feld@octobus.net> parents: 40641 diff changeset	289 # for the others, we use binary slicing to quickly converge toward valid
9c3c697267db sparse-revlog: rework the way we enforce chunk size limit Boris Feld <boris.feld@octobus.net> parents: 40641 diff changeset	290 # chunks (otherwise, we might end up looking for start/end of many
9c3c697267db sparse-revlog: rework the way we enforce chunk size limit Boris Feld <boris.feld@octobus.net> parents: 40641 diff changeset	291 # revisions). This logic is not looking for the perfect slicing point, it
9c3c697267db sparse-revlog: rework the way we enforce chunk size limit Boris Feld <boris.feld@octobus.net> parents: 40641 diff changeset	292 # focuses on quickly converging toward valid chunks.
9c3c697267db sparse-revlog: rework the way we enforce chunk size limit Boris Feld <boris.feld@octobus.net> parents: 40641 diff changeset	293 nbitem = len(revs)
9c3c697267db sparse-revlog: rework the way we enforce chunk size limit Boris Feld <boris.feld@octobus.net> parents: 40641 diff changeset	294 while (enddata - startdata) > targetsize:
9c3c697267db sparse-revlog: rework the way we enforce chunk size limit Boris Feld <boris.feld@octobus.net> parents: 40641 diff changeset	295 endrevidx = nbitem
9c3c697267db sparse-revlog: rework the way we enforce chunk size limit Boris Feld <boris.feld@octobus.net> parents: 40641 diff changeset	296 if nbitem - startrevidx <= 1:
43076 2372284d9457 formatting: blacken the codebase Augie Fackler <augie@google.com> parents: 42992 diff changeset	297 break # protect against individual chunk larger than limit
40642 9c3c697267db sparse-revlog: rework the way we enforce chunk size limit Boris Feld <boris.feld@octobus.net> parents: 40641 diff changeset	298 localenddata = revlog.end(revs[endrevidx - 1])
9c3c697267db sparse-revlog: rework the way we enforce chunk size limit Boris Feld <boris.feld@octobus.net> parents: 40641 diff changeset	299 span = localenddata - startdata
40654 fd1d41ccbe38 sparse-revlog: use `span` variable as intended Boris Feld <boris.feld@octobus.net> parents: 40642 diff changeset	300 while span > targetsize:
40642 9c3c697267db sparse-revlog: rework the way we enforce chunk size limit Boris Feld <boris.feld@octobus.net> parents: 40641 diff changeset	301 if endrevidx - startrevidx <= 1:
43076 2372284d9457 formatting: blacken the codebase Augie Fackler <augie@google.com> parents: 42992 diff changeset	302 break # protect against individual chunk larger than limit
40642 9c3c697267db sparse-revlog: rework the way we enforce chunk size limit Boris Feld <boris.feld@octobus.net> parents: 40641 diff changeset	303 endrevidx -= (endrevidx - startrevidx) // 2
9c3c697267db sparse-revlog: rework the way we enforce chunk size limit Boris Feld <boris.feld@octobus.net> parents: 40641 diff changeset	304 localenddata = revlog.end(revs[endrevidx - 1])
9c3c697267db sparse-revlog: rework the way we enforce chunk size limit Boris Feld <boris.feld@octobus.net> parents: 40641 diff changeset	305 span = localenddata - startdata
9c3c697267db sparse-revlog: rework the way we enforce chunk size limit Boris Feld <boris.feld@octobus.net> parents: 40641 diff changeset	306 chunk = _trimchunk(revlog, revs, startrevidx, endrevidx)
9c3c697267db sparse-revlog: rework the way we enforce chunk size limit Boris Feld <boris.feld@octobus.net> parents: 40641 diff changeset	307 if chunk:
9c3c697267db sparse-revlog: rework the way we enforce chunk size limit Boris Feld <boris.feld@octobus.net> parents: 40641 diff changeset	308 yield chunk
9c3c697267db sparse-revlog: rework the way we enforce chunk size limit Boris Feld <boris.feld@octobus.net> parents: 40641 diff changeset	309 startrevidx = endrevidx
9c3c697267db sparse-revlog: rework the way we enforce chunk size limit Boris Feld <boris.feld@octobus.net> parents: 40641 diff changeset	310 startdata = revlog.start(revs[startrevidx])
9c3c697267db sparse-revlog: rework the way we enforce chunk size limit Boris Feld <boris.feld@octobus.net> parents: 40641 diff changeset	311
9c3c697267db sparse-revlog: rework the way we enforce chunk size limit Boris Feld <boris.feld@octobus.net> parents: 40641 diff changeset	312 chunk = _trimchunk(revlog, revs, startrevidx)
9c3c697267db sparse-revlog: rework the way we enforce chunk size limit Boris Feld <boris.feld@octobus.net> parents: 40641 diff changeset	313 if chunk:
9c3c697267db sparse-revlog: rework the way we enforce chunk size limit Boris Feld <boris.feld@octobus.net> parents: 40641 diff changeset	314 yield chunk
38642 e59e27e52297 revlog: add function to slice chunk down to a given size Boris Feld <boris.feld@octobus.net> parents: 38641 diff changeset	315
43076 2372284d9457 formatting: blacken the codebase Augie Fackler <augie@google.com> parents: 42992 diff changeset	316
2372284d9457 formatting: blacken the codebase Augie Fackler <augie@google.com> parents: 42992 diff changeset	317 def _slicechunktodensity(revlog, revs, targetdensity=0.5, mingapsize=0):
38641 feba6be0941b revlog: extract density based slicing into its own function Boris Feld <boris.feld@octobus.net> parents: 38640 diff changeset	318 """slice revs to reduce the amount of unrelated data to be read from disk.
feba6be0941b revlog: extract density based slicing into its own function Boris Feld <boris.feld@octobus.net> parents: 38640 diff changeset	319
feba6be0941b revlog: extract density based slicing into its own function Boris Feld <boris.feld@octobus.net> parents: 38640 diff changeset	320 ``revs`` is sliced into groups that should be read in one time.
feba6be0941b revlog: extract density based slicing into its own function Boris Feld <boris.feld@octobus.net> parents: 38640 diff changeset	321 Assume that revs are sorted.
feba6be0941b revlog: extract density based slicing into its own function Boris Feld <boris.feld@octobus.net> parents: 38640 diff changeset	322
feba6be0941b revlog: extract density based slicing into its own function Boris Feld <boris.feld@octobus.net> parents: 38640 diff changeset	323 The initial chunk is sliced until the overall density (payload/chunks-span
feba6be0941b revlog: extract density based slicing into its own function Boris Feld <boris.feld@octobus.net> parents: 38640 diff changeset	324 ratio) is above `targetdensity`. No gap smaller than `mingapsize` is
feba6be0941b revlog: extract density based slicing into its own function Boris Feld <boris.feld@octobus.net> parents: 38640 diff changeset	325 skipped.
feba6be0941b revlog: extract density based slicing into its own function Boris Feld <boris.feld@octobus.net> parents: 38640 diff changeset	326
feba6be0941b revlog: extract density based slicing into its own function Boris Feld <boris.feld@octobus.net> parents: 38640 diff changeset	327 >>> revlog = _testrevlog([
feba6be0941b revlog: extract density based slicing into its own function Boris Feld <boris.feld@octobus.net> parents: 38640 diff changeset	328 ... 5, #00 (5)
feba6be0941b revlog: extract density based slicing into its own function Boris Feld <boris.feld@octobus.net> parents: 38640 diff changeset	329 ... 10, #01 (5)
feba6be0941b revlog: extract density based slicing into its own function Boris Feld <boris.feld@octobus.net> parents: 38640 diff changeset	330 ... 12, #02 (2)
feba6be0941b revlog: extract density based slicing into its own function Boris Feld <boris.feld@octobus.net> parents: 38640 diff changeset	331 ... 12, #03 (empty)
feba6be0941b revlog: extract density based slicing into its own function Boris Feld <boris.feld@octobus.net> parents: 38640 diff changeset	332 ... 27, #04 (15)
feba6be0941b revlog: extract density based slicing into its own function Boris Feld <boris.feld@octobus.net> parents: 38640 diff changeset	333 ... 31, #05 (4)
feba6be0941b revlog: extract density based slicing into its own function Boris Feld <boris.feld@octobus.net> parents: 38640 diff changeset	334 ... 31, #06 (empty)
feba6be0941b revlog: extract density based slicing into its own function Boris Feld <boris.feld@octobus.net> parents: 38640 diff changeset	335 ... 42, #07 (11)
feba6be0941b revlog: extract density based slicing into its own function Boris Feld <boris.feld@octobus.net> parents: 38640 diff changeset	336 ... 47, #08 (5)
feba6be0941b revlog: extract density based slicing into its own function Boris Feld <boris.feld@octobus.net> parents: 38640 diff changeset	337 ... 47, #09 (empty)
feba6be0941b revlog: extract density based slicing into its own function Boris Feld <boris.feld@octobus.net> parents: 38640 diff changeset	338 ... 48, #10 (1)
feba6be0941b revlog: extract density based slicing into its own function Boris Feld <boris.feld@octobus.net> parents: 38640 diff changeset	339 ... 51, #11 (3)
feba6be0941b revlog: extract density based slicing into its own function Boris Feld <boris.feld@octobus.net> parents: 38640 diff changeset	340 ... 74, #12 (23)
feba6be0941b revlog: extract density based slicing into its own function Boris Feld <boris.feld@octobus.net> parents: 38640 diff changeset	341 ... 85, #13 (11)
feba6be0941b revlog: extract density based slicing into its own function Boris Feld <boris.feld@octobus.net> parents: 38640 diff changeset	342 ... 86, #14 (1)
feba6be0941b revlog: extract density based slicing into its own function Boris Feld <boris.feld@octobus.net> parents: 38640 diff changeset	343 ... 91, #15 (5)
feba6be0941b revlog: extract density based slicing into its own function Boris Feld <boris.feld@octobus.net> parents: 38640 diff changeset	344 ... ])
feba6be0941b revlog: extract density based slicing into its own function Boris Feld <boris.feld@octobus.net> parents: 38640 diff changeset	345
38655 cd1c484e31e8 revlog: adjust doctest examples to be portable to Python 3 Augie Fackler <augie@google.com> parents: 38644 diff changeset	346 >>> list(_slicechunktodensity(revlog, list(range(16))))
38641 feba6be0941b revlog: extract density based slicing into its own function Boris Feld <boris.feld@octobus.net> parents: 38640 diff changeset	347 [[0, 1, 2, 3, 4, 5, 6, 7, 8, 9, 10, 11, 12, 13, 14, 15]]
feba6be0941b revlog: extract density based slicing into its own function Boris Feld <boris.feld@octobus.net> parents: 38640 diff changeset	348 >>> list(_slicechunktodensity(revlog, [0, 15]))
feba6be0941b revlog: extract density based slicing into its own function Boris Feld <boris.feld@octobus.net> parents: 38640 diff changeset	349 [[0], [15]]
feba6be0941b revlog: extract density based slicing into its own function Boris Feld <boris.feld@octobus.net> parents: 38640 diff changeset	350 >>> list(_slicechunktodensity(revlog, [0, 11, 15]))
feba6be0941b revlog: extract density based slicing into its own function Boris Feld <boris.feld@octobus.net> parents: 38640 diff changeset	351 [[0], [11], [15]]
feba6be0941b revlog: extract density based slicing into its own function Boris Feld <boris.feld@octobus.net> parents: 38640 diff changeset	352 >>> list(_slicechunktodensity(revlog, [0, 11, 13, 15]))
feba6be0941b revlog: extract density based slicing into its own function Boris Feld <boris.feld@octobus.net> parents: 38640 diff changeset	353 [[0], [11, 13, 15]]
feba6be0941b revlog: extract density based slicing into its own function Boris Feld <boris.feld@octobus.net> parents: 38640 diff changeset	354 >>> list(_slicechunktodensity(revlog, [1, 2, 3, 5, 8, 10, 11, 14]))
feba6be0941b revlog: extract density based slicing into its own function Boris Feld <boris.feld@octobus.net> parents: 38640 diff changeset	355 [[1, 2], [5, 8, 10, 11], [14]]
feba6be0941b revlog: extract density based slicing into its own function Boris Feld <boris.feld@octobus.net> parents: 38640 diff changeset	356 >>> list(_slicechunktodensity(revlog, [1, 2, 3, 5, 8, 10, 11, 14],
feba6be0941b revlog: extract density based slicing into its own function Boris Feld <boris.feld@octobus.net> parents: 38640 diff changeset	357 ... mingapsize=20))
feba6be0941b revlog: extract density based slicing into its own function Boris Feld <boris.feld@octobus.net> parents: 38640 diff changeset	358 [[1, 2, 3, 5, 8, 10, 11], [14]]
feba6be0941b revlog: extract density based slicing into its own function Boris Feld <boris.feld@octobus.net> parents: 38640 diff changeset	359 >>> list(_slicechunktodensity(revlog, [1, 2, 3, 5, 8, 10, 11, 14],
feba6be0941b revlog: extract density based slicing into its own function Boris Feld <boris.feld@octobus.net> parents: 38640 diff changeset	360 ... targetdensity=0.95))
feba6be0941b revlog: extract density based slicing into its own function Boris Feld <boris.feld@octobus.net> parents: 38640 diff changeset	361 [[1, 2], [5], [8, 10, 11], [14]]
feba6be0941b revlog: extract density based slicing into its own function Boris Feld <boris.feld@octobus.net> parents: 38640 diff changeset	362 >>> list(_slicechunktodensity(revlog, [1, 2, 3, 5, 8, 10, 11, 14],
feba6be0941b revlog: extract density based slicing into its own function Boris Feld <boris.feld@octobus.net> parents: 38640 diff changeset	363 ... targetdensity=0.95, mingapsize=12))
feba6be0941b revlog: extract density based slicing into its own function Boris Feld <boris.feld@octobus.net> parents: 38640 diff changeset	364 [[1, 2], [5, 8, 10, 11], [14]]
feba6be0941b revlog: extract density based slicing into its own function Boris Feld <boris.feld@octobus.net> parents: 38640 diff changeset	365 """
34824 e2ad93bcc084 revlog: introduce an experimental flag to slice chunks reads when too sparse Paul Morelle <paul.morelle@octobus.net> parents: 34823 diff changeset	366 start = revlog.start
e2ad93bcc084 revlog: introduce an experimental flag to slice chunks reads when too sparse Paul Morelle <paul.morelle@octobus.net> parents: 34823 diff changeset	367 length = revlog.length
e2ad93bcc084 revlog: introduce an experimental flag to slice chunks reads when too sparse Paul Morelle <paul.morelle@octobus.net> parents: 34823 diff changeset	368
34880 9e18ab7f7240 sparse-read: move from a recursive-based approach to a heap-based one Boris Feld <boris.feld@octobus.net> parents: 34825 diff changeset	369 if len(revs) <= 1:
9e18ab7f7240 sparse-read: move from a recursive-based approach to a heap-based one Boris Feld <boris.feld@octobus.net> parents: 34825 diff changeset	370 yield revs
9e18ab7f7240 sparse-read: move from a recursive-based approach to a heap-based one Boris Feld <boris.feld@octobus.net> parents: 34825 diff changeset	371 return
34824 e2ad93bcc084 revlog: introduce an experimental flag to slice chunks reads when too sparse Paul Morelle <paul.morelle@octobus.net> parents: 34823 diff changeset	372
40604 3ac23dad6364 sparse-revlog: drop unused deltainfo parameter from _slicechunktodensity Boris Feld <boris.feld@octobus.net> parents: 40603 diff changeset	373 deltachainspan = segmentspan(revlog, revs)
34880 9e18ab7f7240 sparse-read: move from a recursive-based approach to a heap-based one Boris Feld <boris.feld@octobus.net> parents: 34825 diff changeset	374
38641 feba6be0941b revlog: extract density based slicing into its own function Boris Feld <boris.feld@octobus.net> parents: 38640 diff changeset	375 if deltachainspan < mingapsize:
38635 d083ae26c325 revlog: early return in _slicechunk when span is already small enough Boris Feld <boris.feld@octobus.net> parents: 38634 diff changeset	376 yield revs
d083ae26c325 revlog: early return in _slicechunk when span is already small enough Boris Feld <boris.feld@octobus.net> parents: 38634 diff changeset	377 return
d083ae26c325 revlog: early return in _slicechunk when span is already small enough Boris Feld <boris.feld@octobus.net> parents: 38634 diff changeset	378
38718 f8762ea73e0d sparse-revlog: implement algorithm to write sparse delta chains (issue5480) Paul Morelle <paul.morelle@octobus.net> parents: 38717 diff changeset	379 readdata = deltachainspan
40606 bfbfd15d65bd sparse-revlog: fast-path before computing payload size Boris Feld <boris.feld@octobus.net> parents: 40605 diff changeset	380 chainpayload = sum(length(r) for r in revs)
34824 e2ad93bcc084 revlog: introduce an experimental flag to slice chunks reads when too sparse Paul Morelle <paul.morelle@octobus.net> parents: 34823 diff changeset	381
34880 9e18ab7f7240 sparse-read: move from a recursive-based approach to a heap-based one Boris Feld <boris.feld@octobus.net> parents: 34825 diff changeset	382 if deltachainspan:
9e18ab7f7240 sparse-read: move from a recursive-based approach to a heap-based one Boris Feld <boris.feld@octobus.net> parents: 34825 diff changeset	383 density = chainpayload / float(deltachainspan)
9e18ab7f7240 sparse-read: move from a recursive-based approach to a heap-based one Boris Feld <boris.feld@octobus.net> parents: 34825 diff changeset	384 else:
9e18ab7f7240 sparse-read: move from a recursive-based approach to a heap-based one Boris Feld <boris.feld@octobus.net> parents: 34825 diff changeset	385 density = 1.0
34824 e2ad93bcc084 revlog: introduce an experimental flag to slice chunks reads when too sparse Paul Morelle <paul.morelle@octobus.net> parents: 34823 diff changeset	386
38641 feba6be0941b revlog: extract density based slicing into its own function Boris Feld <boris.feld@octobus.net> parents: 38640 diff changeset	387 if density >= targetdensity:
38634 f0ea8b847831 revlog: early return in _slicechunk when density is already good Paul Morelle <paul.morelle@octobus.net> parents: 38632 diff changeset	388 yield revs
f0ea8b847831 revlog: early return in _slicechunk when density is already good Paul Morelle <paul.morelle@octobus.net> parents: 38632 diff changeset	389 return
f0ea8b847831 revlog: early return in _slicechunk when density is already good Paul Morelle <paul.morelle@octobus.net> parents: 38632 diff changeset	390
34880 9e18ab7f7240 sparse-read: move from a recursive-based approach to a heap-based one Boris Feld <boris.feld@octobus.net> parents: 34825 diff changeset	391 # Store the gaps in a heap to have them sorted by decreasing size
40607 54de23400b2a sparse-revlog: stop using a heap to track gaps Boris Feld <boris.feld@octobus.net> parents: 40606 diff changeset	392 gaps = []
34880 9e18ab7f7240 sparse-read: move from a recursive-based approach to a heap-based one Boris Feld <boris.feld@octobus.net> parents: 34825 diff changeset	393 prevend = None
9e18ab7f7240 sparse-read: move from a recursive-based approach to a heap-based one Boris Feld <boris.feld@octobus.net> parents: 34825 diff changeset	394 for i, rev in enumerate(revs):
40604 3ac23dad6364 sparse-revlog: drop unused deltainfo parameter from _slicechunktodensity Boris Feld <boris.feld@octobus.net> parents: 40603 diff changeset	395 revstart = start(rev)
3ac23dad6364 sparse-revlog: drop unused deltainfo parameter from _slicechunktodensity Boris Feld <boris.feld@octobus.net> parents: 40603 diff changeset	396 revlen = length(rev)
34824 e2ad93bcc084 revlog: introduce an experimental flag to slice chunks reads when too sparse Paul Morelle <paul.morelle@octobus.net> parents: 34823 diff changeset	397
34898 1bde8e8e5de0 sparse-read: ignore trailing empty revs in each read chunk Paul Morelle <paul.morelle@octobus.net> parents: 34881 diff changeset	398 # Skip empty revisions to form larger holes
1bde8e8e5de0 sparse-read: ignore trailing empty revs in each read chunk Paul Morelle <paul.morelle@octobus.net> parents: 34881 diff changeset	399 if revlen == 0:
1bde8e8e5de0 sparse-read: ignore trailing empty revs in each read chunk Paul Morelle <paul.morelle@octobus.net> parents: 34881 diff changeset	400 continue
1bde8e8e5de0 sparse-read: ignore trailing empty revs in each read chunk Paul Morelle <paul.morelle@octobus.net> parents: 34881 diff changeset	401
34880 9e18ab7f7240 sparse-read: move from a recursive-based approach to a heap-based one Boris Feld <boris.feld@octobus.net> parents: 34825 diff changeset	402 if prevend is not None:
9e18ab7f7240 sparse-read: move from a recursive-based approach to a heap-based one Boris Feld <boris.feld@octobus.net> parents: 34825 diff changeset	403 gapsize = revstart - prevend
34881 8c9b08a0c48c sparse-read: skip gaps too small to be worth splitting Paul Morelle <paul.morelle@octobus.net> parents: 34880 diff changeset	404 # only consider holes that are large enough
38641 feba6be0941b revlog: extract density based slicing into its own function Boris Feld <boris.feld@octobus.net> parents: 38640 diff changeset	405 if gapsize > mingapsize:
40607 54de23400b2a sparse-revlog: stop using a heap to track gaps Boris Feld <boris.feld@octobus.net> parents: 40606 diff changeset	406 gaps.append((gapsize, i))
34880 9e18ab7f7240 sparse-read: move from a recursive-based approach to a heap-based one Boris Feld <boris.feld@octobus.net> parents: 34825 diff changeset	407
9e18ab7f7240 sparse-read: move from a recursive-based approach to a heap-based one Boris Feld <boris.feld@octobus.net> parents: 34825 diff changeset	408 prevend = revstart + revlen
40607 54de23400b2a sparse-revlog: stop using a heap to track gaps Boris Feld <boris.feld@octobus.net> parents: 40606 diff changeset	409 # sort the gaps to pop them from largest to small
54de23400b2a sparse-revlog: stop using a heap to track gaps Boris Feld <boris.feld@octobus.net> parents: 40606 diff changeset	410 gaps.sort()
34880 9e18ab7f7240 sparse-read: move from a recursive-based approach to a heap-based one Boris Feld <boris.feld@octobus.net> parents: 34825 diff changeset	411
9e18ab7f7240 sparse-read: move from a recursive-based approach to a heap-based one Boris Feld <boris.feld@octobus.net> parents: 34825 diff changeset	412 # Collect the indices of the largest holes until the density is acceptable
40608 526ee887c4d5 sparse-revlog: stop using a heap to track selected gap Boris Feld <boris.feld@octobus.net> parents: 40607 diff changeset	413 selected = []
40607 54de23400b2a sparse-revlog: stop using a heap to track gaps Boris Feld <boris.feld@octobus.net> parents: 40606 diff changeset	414 while gaps and density < targetdensity:
54de23400b2a sparse-revlog: stop using a heap to track gaps Boris Feld <boris.feld@octobus.net> parents: 40606 diff changeset	415 gapsize, gapidx = gaps.pop()
34880 9e18ab7f7240 sparse-read: move from a recursive-based approach to a heap-based one Boris Feld <boris.feld@octobus.net> parents: 34825 diff changeset	416
40608 526ee887c4d5 sparse-revlog: stop using a heap to track selected gap Boris Feld <boris.feld@octobus.net> parents: 40607 diff changeset	417 selected.append(gapidx)
34824 e2ad93bcc084 revlog: introduce an experimental flag to slice chunks reads when too sparse Paul Morelle <paul.morelle@octobus.net> parents: 34823 diff changeset	418
34880 9e18ab7f7240 sparse-read: move from a recursive-based approach to a heap-based one Boris Feld <boris.feld@octobus.net> parents: 34825 diff changeset	419 # the gap sizes are stored as negatives to be sorted decreasingly
9e18ab7f7240 sparse-read: move from a recursive-based approach to a heap-based one Boris Feld <boris.feld@octobus.net> parents: 34825 diff changeset	420 # by the heap
40607 54de23400b2a sparse-revlog: stop using a heap to track gaps Boris Feld <boris.feld@octobus.net> parents: 40606 diff changeset	421 readdata -= gapsize
34880 9e18ab7f7240 sparse-read: move from a recursive-based approach to a heap-based one Boris Feld <boris.feld@octobus.net> parents: 34825 diff changeset	422 if readdata > 0:
9e18ab7f7240 sparse-read: move from a recursive-based approach to a heap-based one Boris Feld <boris.feld@octobus.net> parents: 34825 diff changeset	423 density = chainpayload / float(readdata)
9e18ab7f7240 sparse-read: move from a recursive-based approach to a heap-based one Boris Feld <boris.feld@octobus.net> parents: 34825 diff changeset	424 else:
9e18ab7f7240 sparse-read: move from a recursive-based approach to a heap-based one Boris Feld <boris.feld@octobus.net> parents: 34825 diff changeset	425 density = 1.0
40608 526ee887c4d5 sparse-revlog: stop using a heap to track selected gap Boris Feld <boris.feld@octobus.net> parents: 40607 diff changeset	426 selected.sort()
34824 e2ad93bcc084 revlog: introduce an experimental flag to slice chunks reads when too sparse Paul Morelle <paul.morelle@octobus.net> parents: 34823 diff changeset	427
34880 9e18ab7f7240 sparse-read: move from a recursive-based approach to a heap-based one Boris Feld <boris.feld@octobus.net> parents: 34825 diff changeset	428 # Cut the revs at collected indices
9e18ab7f7240 sparse-read: move from a recursive-based approach to a heap-based one Boris Feld <boris.feld@octobus.net> parents: 34825 diff changeset	429 previdx = 0
40608 526ee887c4d5 sparse-revlog: stop using a heap to track selected gap Boris Feld <boris.feld@octobus.net> parents: 40607 diff changeset	430 for idx in selected:
34898 1bde8e8e5de0 sparse-read: ignore trailing empty revs in each read chunk Paul Morelle <paul.morelle@octobus.net> parents: 34881 diff changeset	431
1bde8e8e5de0 sparse-read: ignore trailing empty revs in each read chunk Paul Morelle <paul.morelle@octobus.net> parents: 34881 diff changeset	432 chunk = _trimchunk(revlog, revs, previdx, idx)
1bde8e8e5de0 sparse-read: ignore trailing empty revs in each read chunk Paul Morelle <paul.morelle@octobus.net> parents: 34881 diff changeset	433 if chunk:
1bde8e8e5de0 sparse-read: ignore trailing empty revs in each read chunk Paul Morelle <paul.morelle@octobus.net> parents: 34881 diff changeset	434 yield chunk
1bde8e8e5de0 sparse-read: ignore trailing empty revs in each read chunk Paul Morelle <paul.morelle@octobus.net> parents: 34881 diff changeset	435
34880 9e18ab7f7240 sparse-read: move from a recursive-based approach to a heap-based one Boris Feld <boris.feld@octobus.net> parents: 34825 diff changeset	436 previdx = idx
34898 1bde8e8e5de0 sparse-read: ignore trailing empty revs in each read chunk Paul Morelle <paul.morelle@octobus.net> parents: 34881 diff changeset	437
1bde8e8e5de0 sparse-read: ignore trailing empty revs in each read chunk Paul Morelle <paul.morelle@octobus.net> parents: 34881 diff changeset	438 chunk = _trimchunk(revlog, revs, previdx)
1bde8e8e5de0 sparse-read: ignore trailing empty revs in each read chunk Paul Morelle <paul.morelle@octobus.net> parents: 34881 diff changeset	439 if chunk:
1bde8e8e5de0 sparse-read: ignore trailing empty revs in each read chunk Paul Morelle <paul.morelle@octobus.net> parents: 34881 diff changeset	440 yield chunk
34824 e2ad93bcc084 revlog: introduce an experimental flag to slice chunks reads when too sparse Paul Morelle <paul.morelle@octobus.net> parents: 34823 diff changeset	441
43076 2372284d9457 formatting: blacken the codebase Augie Fackler <augie@google.com> parents: 42992 diff changeset	442
39330 655b5b465953 revlog: split functionality related to deltas computation in a new module Boris Feld <boris.feld@octobus.net> parents: 39329 diff changeset	443 def _trimchunk(revlog, revs, startidx, endidx=None):
655b5b465953 revlog: split functionality related to deltas computation in a new module Boris Feld <boris.feld@octobus.net> parents: 39329 diff changeset	444 """returns revs[startidx:endidx] without empty trailing revs
655b5b465953 revlog: split functionality related to deltas computation in a new module Boris Feld <boris.feld@octobus.net> parents: 39329 diff changeset	445
655b5b465953 revlog: split functionality related to deltas computation in a new module Boris Feld <boris.feld@octobus.net> parents: 39329 diff changeset	446 Doctest Setup
655b5b465953 revlog: split functionality related to deltas computation in a new module Boris Feld <boris.feld@octobus.net> parents: 39329 diff changeset	447 >>> revlog = _testrevlog([
655b5b465953 revlog: split functionality related to deltas computation in a new module Boris Feld <boris.feld@octobus.net> parents: 39329 diff changeset	448 ... 5, #0
655b5b465953 revlog: split functionality related to deltas computation in a new module Boris Feld <boris.feld@octobus.net> parents: 39329 diff changeset	449 ... 10, #1
655b5b465953 revlog: split functionality related to deltas computation in a new module Boris Feld <boris.feld@octobus.net> parents: 39329 diff changeset	450 ... 12, #2
655b5b465953 revlog: split functionality related to deltas computation in a new module Boris Feld <boris.feld@octobus.net> parents: 39329 diff changeset	451 ... 12, #3 (empty)
655b5b465953 revlog: split functionality related to deltas computation in a new module Boris Feld <boris.feld@octobus.net> parents: 39329 diff changeset	452 ... 17, #4
655b5b465953 revlog: split functionality related to deltas computation in a new module Boris Feld <boris.feld@octobus.net> parents: 39329 diff changeset	453 ... 21, #5
655b5b465953 revlog: split functionality related to deltas computation in a new module Boris Feld <boris.feld@octobus.net> parents: 39329 diff changeset	454 ... 21, #6 (empty)
655b5b465953 revlog: split functionality related to deltas computation in a new module Boris Feld <boris.feld@octobus.net> parents: 39329 diff changeset	455 ... ])
655b5b465953 revlog: split functionality related to deltas computation in a new module Boris Feld <boris.feld@octobus.net> parents: 39329 diff changeset	456
655b5b465953 revlog: split functionality related to deltas computation in a new module Boris Feld <boris.feld@octobus.net> parents: 39329 diff changeset	457 Contiguous cases:
655b5b465953 revlog: split functionality related to deltas computation in a new module Boris Feld <boris.feld@octobus.net> parents: 39329 diff changeset	458 >>> _trimchunk(revlog, [0, 1, 2, 3, 4, 5, 6], 0)
655b5b465953 revlog: split functionality related to deltas computation in a new module Boris Feld <boris.feld@octobus.net> parents: 39329 diff changeset	459 [0, 1, 2, 3, 4, 5]
655b5b465953 revlog: split functionality related to deltas computation in a new module Boris Feld <boris.feld@octobus.net> parents: 39329 diff changeset	460 >>> _trimchunk(revlog, [0, 1, 2, 3, 4, 5, 6], 0, 5)
655b5b465953 revlog: split functionality related to deltas computation in a new module Boris Feld <boris.feld@octobus.net> parents: 39329 diff changeset	461 [0, 1, 2, 3, 4]
655b5b465953 revlog: split functionality related to deltas computation in a new module Boris Feld <boris.feld@octobus.net> parents: 39329 diff changeset	462 >>> _trimchunk(revlog, [0, 1, 2, 3, 4, 5, 6], 0, 4)
655b5b465953 revlog: split functionality related to deltas computation in a new module Boris Feld <boris.feld@octobus.net> parents: 39329 diff changeset	463 [0, 1, 2]
655b5b465953 revlog: split functionality related to deltas computation in a new module Boris Feld <boris.feld@octobus.net> parents: 39329 diff changeset	464 >>> _trimchunk(revlog, [0, 1, 2, 3, 4, 5, 6], 2, 4)
655b5b465953 revlog: split functionality related to deltas computation in a new module Boris Feld <boris.feld@octobus.net> parents: 39329 diff changeset	465 [2]
655b5b465953 revlog: split functionality related to deltas computation in a new module Boris Feld <boris.feld@octobus.net> parents: 39329 diff changeset	466 >>> _trimchunk(revlog, [0, 1, 2, 3, 4, 5, 6], 3)
655b5b465953 revlog: split functionality related to deltas computation in a new module Boris Feld <boris.feld@octobus.net> parents: 39329 diff changeset	467 [3, 4, 5]
655b5b465953 revlog: split functionality related to deltas computation in a new module Boris Feld <boris.feld@octobus.net> parents: 39329 diff changeset	468 >>> _trimchunk(revlog, [0, 1, 2, 3, 4, 5, 6], 3, 5)
655b5b465953 revlog: split functionality related to deltas computation in a new module Boris Feld <boris.feld@octobus.net> parents: 39329 diff changeset	469 [3, 4]
655b5b465953 revlog: split functionality related to deltas computation in a new module Boris Feld <boris.feld@octobus.net> parents: 39329 diff changeset	470
655b5b465953 revlog: split functionality related to deltas computation in a new module Boris Feld <boris.feld@octobus.net> parents: 39329 diff changeset	471 Discontiguous cases:
655b5b465953 revlog: split functionality related to deltas computation in a new module Boris Feld <boris.feld@octobus.net> parents: 39329 diff changeset	472 >>> _trimchunk(revlog, [1, 3, 5, 6], 0)
655b5b465953 revlog: split functionality related to deltas computation in a new module Boris Feld <boris.feld@octobus.net> parents: 39329 diff changeset	473 [1, 3, 5]
655b5b465953 revlog: split functionality related to deltas computation in a new module Boris Feld <boris.feld@octobus.net> parents: 39329 diff changeset	474 >>> _trimchunk(revlog, [1, 3, 5, 6], 0, 2)
655b5b465953 revlog: split functionality related to deltas computation in a new module Boris Feld <boris.feld@octobus.net> parents: 39329 diff changeset	475 [1]
655b5b465953 revlog: split functionality related to deltas computation in a new module Boris Feld <boris.feld@octobus.net> parents: 39329 diff changeset	476 >>> _trimchunk(revlog, [1, 3, 5, 6], 1, 3)
655b5b465953 revlog: split functionality related to deltas computation in a new module Boris Feld <boris.feld@octobus.net> parents: 39329 diff changeset	477 [3, 5]
655b5b465953 revlog: split functionality related to deltas computation in a new module Boris Feld <boris.feld@octobus.net> parents: 39329 diff changeset	478 >>> _trimchunk(revlog, [1, 3, 5, 6], 1)
655b5b465953 revlog: split functionality related to deltas computation in a new module Boris Feld <boris.feld@octobus.net> parents: 39329 diff changeset	479 [3, 5]
655b5b465953 revlog: split functionality related to deltas computation in a new module Boris Feld <boris.feld@octobus.net> parents: 39329 diff changeset	480 """
655b5b465953 revlog: split functionality related to deltas computation in a new module Boris Feld <boris.feld@octobus.net> parents: 39329 diff changeset	481 length = revlog.length
655b5b465953 revlog: split functionality related to deltas computation in a new module Boris Feld <boris.feld@octobus.net> parents: 39329 diff changeset	482
655b5b465953 revlog: split functionality related to deltas computation in a new module Boris Feld <boris.feld@octobus.net> parents: 39329 diff changeset	483 if endidx is None:
655b5b465953 revlog: split functionality related to deltas computation in a new module Boris Feld <boris.feld@octobus.net> parents: 39329 diff changeset	484 endidx = len(revs)
655b5b465953 revlog: split functionality related to deltas computation in a new module Boris Feld <boris.feld@octobus.net> parents: 39329 diff changeset	485
655b5b465953 revlog: split functionality related to deltas computation in a new module Boris Feld <boris.feld@octobus.net> parents: 39329 diff changeset	486 # If we have a non-emtpy delta candidate, there are nothing to trim
655b5b465953 revlog: split functionality related to deltas computation in a new module Boris Feld <boris.feld@octobus.net> parents: 39329 diff changeset	487 if revs[endidx - 1] < len(revlog):
655b5b465953 revlog: split functionality related to deltas computation in a new module Boris Feld <boris.feld@octobus.net> parents: 39329 diff changeset	488 # Trim empty revs at the end, except the very first revision of a chain
43076 2372284d9457 formatting: blacken the codebase Augie Fackler <augie@google.com> parents: 42992 diff changeset	489 while (
2372284d9457 formatting: blacken the codebase Augie Fackler <augie@google.com> parents: 42992 diff changeset	490 endidx > 1 and endidx > startidx and length(revs[endidx - 1]) == 0
2372284d9457 formatting: blacken the codebase Augie Fackler <augie@google.com> parents: 42992 diff changeset	491 ):
39330 655b5b465953 revlog: split functionality related to deltas computation in a new module Boris Feld <boris.feld@octobus.net> parents: 39329 diff changeset	492 endidx -= 1
655b5b465953 revlog: split functionality related to deltas computation in a new module Boris Feld <boris.feld@octobus.net> parents: 39329 diff changeset	493
655b5b465953 revlog: split functionality related to deltas computation in a new module Boris Feld <boris.feld@octobus.net> parents: 39329 diff changeset	494 return revs[startidx:endidx]
655b5b465953 revlog: split functionality related to deltas computation in a new module Boris Feld <boris.feld@octobus.net> parents: 39329 diff changeset	495
43076 2372284d9457 formatting: blacken the codebase Augie Fackler <augie@google.com> parents: 42992 diff changeset	496
40605 a32ccd32982b sparse-revlog: drop unused deltainfo parameter from segmentspan Boris Feld <boris.feld@octobus.net> parents: 40604 diff changeset	497 def segmentspan(revlog, revs):
39330 655b5b465953 revlog: split functionality related to deltas computation in a new module Boris Feld <boris.feld@octobus.net> parents: 39329 diff changeset	498 """Get the byte span of a segment of revisions
655b5b465953 revlog: split functionality related to deltas computation in a new module Boris Feld <boris.feld@octobus.net> parents: 39329 diff changeset	499
655b5b465953 revlog: split functionality related to deltas computation in a new module Boris Feld <boris.feld@octobus.net> parents: 39329 diff changeset	500 revs is a sorted array of revision numbers
655b5b465953 revlog: split functionality related to deltas computation in a new module Boris Feld <boris.feld@octobus.net> parents: 39329 diff changeset	501
655b5b465953 revlog: split functionality related to deltas computation in a new module Boris Feld <boris.feld@octobus.net> parents: 39329 diff changeset	502 >>> revlog = _testrevlog([
655b5b465953 revlog: split functionality related to deltas computation in a new module Boris Feld <boris.feld@octobus.net> parents: 39329 diff changeset	503 ... 5, #0
655b5b465953 revlog: split functionality related to deltas computation in a new module Boris Feld <boris.feld@octobus.net> parents: 39329 diff changeset	504 ... 10, #1
655b5b465953 revlog: split functionality related to deltas computation in a new module Boris Feld <boris.feld@octobus.net> parents: 39329 diff changeset	505 ... 12, #2
655b5b465953 revlog: split functionality related to deltas computation in a new module Boris Feld <boris.feld@octobus.net> parents: 39329 diff changeset	506 ... 12, #3 (empty)
655b5b465953 revlog: split functionality related to deltas computation in a new module Boris Feld <boris.feld@octobus.net> parents: 39329 diff changeset	507 ... 17, #4
655b5b465953 revlog: split functionality related to deltas computation in a new module Boris Feld <boris.feld@octobus.net> parents: 39329 diff changeset	508 ... ])
655b5b465953 revlog: split functionality related to deltas computation in a new module Boris Feld <boris.feld@octobus.net> parents: 39329 diff changeset	509
655b5b465953 revlog: split functionality related to deltas computation in a new module Boris Feld <boris.feld@octobus.net> parents: 39329 diff changeset	510 >>> segmentspan(revlog, [0, 1, 2, 3, 4])
655b5b465953 revlog: split functionality related to deltas computation in a new module Boris Feld <boris.feld@octobus.net> parents: 39329 diff changeset	511 17
655b5b465953 revlog: split functionality related to deltas computation in a new module Boris Feld <boris.feld@octobus.net> parents: 39329 diff changeset	512 >>> segmentspan(revlog, [0, 4])
655b5b465953 revlog: split functionality related to deltas computation in a new module Boris Feld <boris.feld@octobus.net> parents: 39329 diff changeset	513 17
655b5b465953 revlog: split functionality related to deltas computation in a new module Boris Feld <boris.feld@octobus.net> parents: 39329 diff changeset	514 >>> segmentspan(revlog, [3, 4])
655b5b465953 revlog: split functionality related to deltas computation in a new module Boris Feld <boris.feld@octobus.net> parents: 39329 diff changeset	515 5
655b5b465953 revlog: split functionality related to deltas computation in a new module Boris Feld <boris.feld@octobus.net> parents: 39329 diff changeset	516 >>> segmentspan(revlog, [1, 2, 3,])
655b5b465953 revlog: split functionality related to deltas computation in a new module Boris Feld <boris.feld@octobus.net> parents: 39329 diff changeset	517 7
655b5b465953 revlog: split functionality related to deltas computation in a new module Boris Feld <boris.feld@octobus.net> parents: 39329 diff changeset	518 >>> segmentspan(revlog, [1, 3])
655b5b465953 revlog: split functionality related to deltas computation in a new module Boris Feld <boris.feld@octobus.net> parents: 39329 diff changeset	519 7
655b5b465953 revlog: split functionality related to deltas computation in a new module Boris Feld <boris.feld@octobus.net> parents: 39329 diff changeset	520 """
655b5b465953 revlog: split functionality related to deltas computation in a new module Boris Feld <boris.feld@octobus.net> parents: 39329 diff changeset	521 if not revs:
655b5b465953 revlog: split functionality related to deltas computation in a new module Boris Feld <boris.feld@octobus.net> parents: 39329 diff changeset	522 return 0
40605 a32ccd32982b sparse-revlog: drop unused deltainfo parameter from segmentspan Boris Feld <boris.feld@octobus.net> parents: 40604 diff changeset	523 end = revlog.end(revs[-1])
39330 655b5b465953 revlog: split functionality related to deltas computation in a new module Boris Feld <boris.feld@octobus.net> parents: 39329 diff changeset	524 return end - revlog.start(revs[0])
655b5b465953 revlog: split functionality related to deltas computation in a new module Boris Feld <boris.feld@octobus.net> parents: 39329 diff changeset	525
43076 2372284d9457 formatting: blacken the codebase Augie Fackler <augie@google.com> parents: 42992 diff changeset	526
51017 509f0f7fc89e delta-computer: stop explicitly taking file handle Pierre-Yves David <pierre-yves.david@octobus.net> parents: 50926 diff changeset	527 def _textfromdelta(revlog, baserev, delta, p1, p2, flags, expectednode):
39331 fd0150a3c2fe revlogdeltas: extra fulltext building in its own function Boris Feld <boris.feld@octobus.net> parents: 39330 diff changeset	528 """build full text from a (base, delta) pair and other metadata"""
fd0150a3c2fe revlogdeltas: extra fulltext building in its own function Boris Feld <boris.feld@octobus.net> parents: 39330 diff changeset	529 # special case deltas which replace entire base; no need to decode
fd0150a3c2fe revlogdeltas: extra fulltext building in its own function Boris Feld <boris.feld@octobus.net> parents: 39330 diff changeset	530 # base revision. this neatly avoids censored bases, which throw when
fd0150a3c2fe revlogdeltas: extra fulltext building in its own function Boris Feld <boris.feld@octobus.net> parents: 39330 diff changeset	531 # they're decoded.
43077 687b865b95ad formatting: byteify all mercurial/ and hgext/ string literals Augie Fackler <augie@google.com> parents: 43076 diff changeset	532 hlen = struct.calcsize(b">lll")
43076 2372284d9457 formatting: blacken the codebase Augie Fackler <augie@google.com> parents: 42992 diff changeset	533 if delta[:hlen] == mdiff.replacediffheader(
2372284d9457 formatting: blacken the codebase Augie Fackler <augie@google.com> parents: 42992 diff changeset	534 revlog.rawsize(baserev), len(delta) - hlen
2372284d9457 formatting: blacken the codebase Augie Fackler <augie@google.com> parents: 42992 diff changeset	535 ):
39331 fd0150a3c2fe revlogdeltas: extra fulltext building in its own function Boris Feld <boris.feld@octobus.net> parents: 39330 diff changeset	536 fulltext = delta[hlen:]
fd0150a3c2fe revlogdeltas: extra fulltext building in its own function Boris Feld <boris.feld@octobus.net> parents: 39330 diff changeset	537 else:
fd0150a3c2fe revlogdeltas: extra fulltext building in its own function Boris Feld <boris.feld@octobus.net> parents: 39330 diff changeset	538 # deltabase is rawtext before changed by flag processors, which is
fd0150a3c2fe revlogdeltas: extra fulltext building in its own function Boris Feld <boris.feld@octobus.net> parents: 39330 diff changeset	539 # equivalent to non-raw text
51017 509f0f7fc89e delta-computer: stop explicitly taking file handle Pierre-Yves David <pierre-yves.david@octobus.net> parents: 50926 diff changeset	540 basetext = revlog.revision(baserev)
39331 fd0150a3c2fe revlogdeltas: extra fulltext building in its own function Boris Feld <boris.feld@octobus.net> parents: 39330 diff changeset	541 fulltext = mdiff.patch(basetext, delta)
fd0150a3c2fe revlogdeltas: extra fulltext building in its own function Boris Feld <boris.feld@octobus.net> parents: 39330 diff changeset	542
fd0150a3c2fe revlogdeltas: extra fulltext building in its own function Boris Feld <boris.feld@octobus.net> parents: 39330 diff changeset	543 try:
42992 dff95420480f flagprocessors: make `processflagsraw` a module level function Pierre-Yves David <pierre-yves.david@octobus.net> parents: 42878 diff changeset	544 validatehash = flagutil.processflagsraw(revlog, fulltext, flags)
39331 fd0150a3c2fe revlogdeltas: extra fulltext building in its own function Boris Feld <boris.feld@octobus.net> parents: 39330 diff changeset	545 if validatehash:
fd0150a3c2fe revlogdeltas: extra fulltext building in its own function Boris Feld <boris.feld@octobus.net> parents: 39330 diff changeset	546 revlog.checkhash(fulltext, expectednode, p1=p1, p2=p2)
fd0150a3c2fe revlogdeltas: extra fulltext building in its own function Boris Feld <boris.feld@octobus.net> parents: 39330 diff changeset	547 if flags & REVIDX_ISCENSORED:
43076 2372284d9457 formatting: blacken the codebase Augie Fackler <augie@google.com> parents: 42992 diff changeset	548 raise error.StorageError(
43077 687b865b95ad formatting: byteify all mercurial/ and hgext/ string literals Augie Fackler <augie@google.com> parents: 43076 diff changeset	549 _(b'node %s is not censored') % expectednode
43076 2372284d9457 formatting: blacken the codebase Augie Fackler <augie@google.com> parents: 42992 diff changeset	550 )
39774 4a2466b2a434 revlog: drop some more error aliases (API) Gregory Szorc <gregory.szorc@gmail.com> parents: 39773 diff changeset	551 except error.CensoredNodeError:
39331 fd0150a3c2fe revlogdeltas: extra fulltext building in its own function Boris Feld <boris.feld@octobus.net> parents: 39330 diff changeset	552 # must pass the censored index flag to add censored revisions
fd0150a3c2fe revlogdeltas: extra fulltext building in its own function Boris Feld <boris.feld@octobus.net> parents: 39330 diff changeset	553 if not flags & REVIDX_ISCENSORED:
fd0150a3c2fe revlogdeltas: extra fulltext building in its own function Boris Feld <boris.feld@octobus.net> parents: 39330 diff changeset	554 raise
fd0150a3c2fe revlogdeltas: extra fulltext building in its own function Boris Feld <boris.feld@octobus.net> parents: 39330 diff changeset	555 return fulltext
fd0150a3c2fe revlogdeltas: extra fulltext building in its own function Boris Feld <boris.feld@octobus.net> parents: 39330 diff changeset	556
43076 2372284d9457 formatting: blacken the codebase Augie Fackler <augie@google.com> parents: 42992 diff changeset	557
35638 edc9330acac1 revlog: introduce 'deltainfo' to distinguish from 'delta' Paul Morelle <paul.morelle@octobus.net> parents: 35637 diff changeset	558 @attr.s(slots=True, frozen=True)
48946 642e31cb55f0 py3: use class X: instead of class X(object): Gregory Szorc <gregory.szorc@gmail.com> parents: 48875 diff changeset	559 class _deltainfo:
35638 edc9330acac1 revlog: introduce 'deltainfo' to distinguish from 'delta' Paul Morelle <paul.morelle@octobus.net> parents: 35637 diff changeset	560 distance = attr.ib()
edc9330acac1 revlog: introduce 'deltainfo' to distinguish from 'delta' Paul Morelle <paul.morelle@octobus.net> parents: 35637 diff changeset	561 deltalen = attr.ib()
edc9330acac1 revlog: introduce 'deltainfo' to distinguish from 'delta' Paul Morelle <paul.morelle@octobus.net> parents: 35637 diff changeset	562 data = attr.ib()
edc9330acac1 revlog: introduce 'deltainfo' to distinguish from 'delta' Paul Morelle <paul.morelle@octobus.net> parents: 35637 diff changeset	563 base = attr.ib()
edc9330acac1 revlog: introduce 'deltainfo' to distinguish from 'delta' Paul Morelle <paul.morelle@octobus.net> parents: 35637 diff changeset	564 chainbase = attr.ib()
edc9330acac1 revlog: introduce 'deltainfo' to distinguish from 'delta' Paul Morelle <paul.morelle@octobus.net> parents: 35637 diff changeset	565 chainlen = attr.ib()
edc9330acac1 revlog: introduce 'deltainfo' to distinguish from 'delta' Paul Morelle <paul.morelle@octobus.net> parents: 35637 diff changeset	566 compresseddeltalen = attr.ib()
39154 e0da43e2f71f revlog: compute snapshot depth on delta info Boris Feld <boris.feld@octobus.net> parents: 39152 diff changeset	567 snapshotdepth = attr.ib()
35638 edc9330acac1 revlog: introduce 'deltainfo' to distinguish from 'delta' Paul Morelle <paul.morelle@octobus.net> parents: 35637 diff changeset	568
43076 2372284d9457 formatting: blacken the codebase Augie Fackler <augie@google.com> parents: 42992 diff changeset	569
47253 b876f0bf7366 revlog: introduce a plain compression mode Pierre-Yves David <pierre-yves.david@octobus.net> parents: 46819 diff changeset	570 def drop_u_compression(delta):
b876f0bf7366 revlog: introduce a plain compression mode Pierre-Yves David <pierre-yves.david@octobus.net> parents: 46819 diff changeset	571 """turn into a "u" (no-compression) into no-compression without header
b876f0bf7366 revlog: introduce a plain compression mode Pierre-Yves David <pierre-yves.david@octobus.net> parents: 46819 diff changeset	572
b876f0bf7366 revlog: introduce a plain compression mode Pierre-Yves David <pierre-yves.david@octobus.net> parents: 46819 diff changeset	573 This is useful for revlog format that has better compression method.
b876f0bf7366 revlog: introduce a plain compression mode Pierre-Yves David <pierre-yves.david@octobus.net> parents: 46819 diff changeset	574 """
b876f0bf7366 revlog: introduce a plain compression mode Pierre-Yves David <pierre-yves.david@octobus.net> parents: 46819 diff changeset	575 assert delta.data[0] == b'u', delta.data[0]
b876f0bf7366 revlog: introduce a plain compression mode Pierre-Yves David <pierre-yves.david@octobus.net> parents: 46819 diff changeset	576 return _deltainfo(
b876f0bf7366 revlog: introduce a plain compression mode Pierre-Yves David <pierre-yves.david@octobus.net> parents: 46819 diff changeset	577 delta.distance,
b876f0bf7366 revlog: introduce a plain compression mode Pierre-Yves David <pierre-yves.david@octobus.net> parents: 46819 diff changeset	578 delta.deltalen - 1,
b876f0bf7366 revlog: introduce a plain compression mode Pierre-Yves David <pierre-yves.david@octobus.net> parents: 46819 diff changeset	579 (b'', delta.data[1]),
b876f0bf7366 revlog: introduce a plain compression mode Pierre-Yves David <pierre-yves.david@octobus.net> parents: 46819 diff changeset	580 delta.base,
b876f0bf7366 revlog: introduce a plain compression mode Pierre-Yves David <pierre-yves.david@octobus.net> parents: 46819 diff changeset	581 delta.chainbase,
b876f0bf7366 revlog: introduce a plain compression mode Pierre-Yves David <pierre-yves.david@octobus.net> parents: 46819 diff changeset	582 delta.chainlen,
b876f0bf7366 revlog: introduce a plain compression mode Pierre-Yves David <pierre-yves.david@octobus.net> parents: 46819 diff changeset	583 delta.compresseddeltalen,
b876f0bf7366 revlog: introduce a plain compression mode Pierre-Yves David <pierre-yves.david@octobus.net> parents: 46819 diff changeset	584 delta.snapshotdepth,
b876f0bf7366 revlog: introduce a plain compression mode Pierre-Yves David <pierre-yves.david@octobus.net> parents: 46819 diff changeset	585 )
b876f0bf7366 revlog: introduce a plain compression mode Pierre-Yves David <pierre-yves.david@octobus.net> parents: 46819 diff changeset	586
b876f0bf7366 revlog: introduce a plain compression mode Pierre-Yves David <pierre-yves.david@octobus.net> parents: 46819 diff changeset	587
40978 42f59d3f714d delta: exclude base candidate much smaller than the target Boris Feld <boris.feld@octobus.net> parents: 40957 diff changeset	588 # If a revision's full text is that much bigger than a base candidate full
42f59d3f714d delta: exclude base candidate much smaller than the target Boris Feld <boris.feld@octobus.net> parents: 40957 diff changeset	589 # text's, it is very unlikely that it will produce a valid delta. We no longer
42f59d3f714d delta: exclude base candidate much smaller than the target Boris Feld <boris.feld@octobus.net> parents: 40957 diff changeset	590 # consider these candidates.
41033 b373477948df revlog: limit base to rev size ratio to 500 instead of 50 Boris Feld <boris.feld@octobus.net> parents: 40979 diff changeset	591 LIMIT_BASE2TEXT = 500
40978 42f59d3f714d delta: exclude base candidate much smaller than the target Boris Feld <boris.feld@octobus.net> parents: 40957 diff changeset	592
51346 cc806f20d756 delta-find: explicitly track stage of the search Pierre-Yves David <pierre-yves.david@octobus.net> parents: 51345 diff changeset	593 ### stage of the search, used for debug and to select and to adjust some logic.
cc806f20d756 delta-find: explicitly track stage of the search Pierre-Yves David <pierre-yves.david@octobus.net> parents: 51345 diff changeset	594 # initial stage, next step is unknown
cc806f20d756 delta-find: explicitly track stage of the search Pierre-Yves David <pierre-yves.david@octobus.net> parents: 51345 diff changeset	595 _STAGE_UNSPECIFIED = "unspecified"
cc806f20d756 delta-find: explicitly track stage of the search Pierre-Yves David <pierre-yves.david@octobus.net> parents: 51345 diff changeset	596 # trying the cached delta
cc806f20d756 delta-find: explicitly track stage of the search Pierre-Yves David <pierre-yves.david@octobus.net> parents: 51345 diff changeset	597 _STAGE_CACHED = "cached"
cc806f20d756 delta-find: explicitly track stage of the search Pierre-Yves David <pierre-yves.david@octobus.net> parents: 51345 diff changeset	598 # trying delta based on parents
cc806f20d756 delta-find: explicitly track stage of the search Pierre-Yves David <pierre-yves.david@octobus.net> parents: 51345 diff changeset	599 _STAGE_PARENTS = "parents"
cc806f20d756 delta-find: explicitly track stage of the search Pierre-Yves David <pierre-yves.david@octobus.net> parents: 51345 diff changeset	600 # trying to build a valid snapshot of any level
cc806f20d756 delta-find: explicitly track stage of the search Pierre-Yves David <pierre-yves.david@octobus.net> parents: 51345 diff changeset	601 _STAGE_SNAPSHOT = "snapshot"
cc806f20d756 delta-find: explicitly track stage of the search Pierre-Yves David <pierre-yves.david@octobus.net> parents: 51345 diff changeset	602 # trying to build a delta based of the previous revision
cc806f20d756 delta-find: explicitly track stage of the search Pierre-Yves David <pierre-yves.david@octobus.net> parents: 51345 diff changeset	603 _STAGE_PREV = "prev"
cc806f20d756 delta-find: explicitly track stage of the search Pierre-Yves David <pierre-yves.david@octobus.net> parents: 51345 diff changeset	604 # trying to build a full snapshot
cc806f20d756 delta-find: explicitly track stage of the search Pierre-Yves David <pierre-yves.david@octobus.net> parents: 51345 diff changeset	605 _STAGE_FULL = "full"
cc806f20d756 delta-find: explicitly track stage of the search Pierre-Yves David <pierre-yves.david@octobus.net> parents: 51345 diff changeset	606
43076 2372284d9457 formatting: blacken the codebase Augie Fackler <augie@google.com> parents: 42992 diff changeset	607
51343 2a333d791ecf delta-find: introduce a base class for _DeltaSearch Pierre-Yves David <pierre-yves.david@octobus.net> parents: 51342 diff changeset	608 class _BaseDeltaSearch(abc.ABC):
51317 c82e03b102a6 delta-find: introduce a _DeltaSearch object Pierre-Yves David <pierre-yves.david@octobus.net> parents: 51316 diff changeset	609 """perform the search of a good delta for a single revlog revision
50354 ca1522fe4ec8 delta-find: assume the target-rev if not specified Pierre-Yves David <pierre-yves.david@octobus.net> parents: 49776 diff changeset	610
51317 c82e03b102a6 delta-find: introduce a _DeltaSearch object Pierre-Yves David <pierre-yves.david@octobus.net> parents: 51316 diff changeset	611 note: some of the deltacomputer.finddeltainfo logic should probably move
c82e03b102a6 delta-find: introduce a _DeltaSearch object Pierre-Yves David <pierre-yves.david@octobus.net> parents: 51316 diff changeset	612 here.
c82e03b102a6 delta-find: introduce a _DeltaSearch object Pierre-Yves David <pierre-yves.david@octobus.net> parents: 51316 diff changeset	613 """
50355 0232571255d3 delta-find: never do anything fancy when general delta is off Pierre-Yves David <pierre-yves.david@octobus.net> parents: 50354 diff changeset	614
51317 c82e03b102a6 delta-find: introduce a _DeltaSearch object Pierre-Yves David <pierre-yves.david@octobus.net> parents: 51316 diff changeset	615 def __init__(
c82e03b102a6 delta-find: introduce a _DeltaSearch object Pierre-Yves David <pierre-yves.david@octobus.net> parents: 51316 diff changeset	616 self,
49610 35d4c2124073 debug: add an option to display statistic about a unbundling operation Pierre-Yves David <pierre-yves.david@octobus.net> parents: 49608 diff changeset	617 revlog,
51327 49401b7dec0c delta-find: feed revinfo to _DeltaSearch Pierre-Yves David <pierre-yves.david@octobus.net> parents: 51325 diff changeset	618 revinfo,
49610 35d4c2124073 debug: add an option to display statistic about a unbundling operation Pierre-Yves David <pierre-yves.david@octobus.net> parents: 49608 diff changeset	619 p1,
35d4c2124073 debug: add an option to display statistic about a unbundling operation Pierre-Yves David <pierre-yves.david@octobus.net> parents: 49608 diff changeset	620 p2,
35d4c2124073 debug: add an option to display statistic about a unbundling operation Pierre-Yves David <pierre-yves.david@octobus.net> parents: 49608 diff changeset	621 cachedelta,
51317 c82e03b102a6 delta-find: introduce a _DeltaSearch object Pierre-Yves David <pierre-yves.david@octobus.net> parents: 51316 diff changeset	622 excluded_bases=None,
c82e03b102a6 delta-find: introduce a _DeltaSearch object Pierre-Yves David <pierre-yves.david@octobus.net> parents: 51316 diff changeset	623 target_rev=None,
c82e03b102a6 delta-find: introduce a _DeltaSearch object Pierre-Yves David <pierre-yves.david@octobus.net> parents: 51316 diff changeset	624 snapshot_cache=None,
c82e03b102a6 delta-find: introduce a _DeltaSearch object Pierre-Yves David <pierre-yves.david@octobus.net> parents: 51316 diff changeset	625 ):
51322 c9ab452e5611 delta-find: check DELTA_BASE_REUSE_FORCE in the _DeltaSearch.__init__ Pierre-Yves David <pierre-yves.david@octobus.net> parents: 51321 diff changeset	626 # the DELTA_BASE_REUSE_FORCE case should have been taken care of sooner
c9ab452e5611 delta-find: check DELTA_BASE_REUSE_FORCE in the _DeltaSearch.__init__ Pierre-Yves David <pierre-yves.david@octobus.net> parents: 51321 diff changeset	627 # so we should never end up asking such question. Adding the assert as
c9ab452e5611 delta-find: check DELTA_BASE_REUSE_FORCE in the _DeltaSearch.__init__ Pierre-Yves David <pierre-yves.david@octobus.net> parents: 51321 diff changeset	628 # a safe-guard to detect anything that would be fishy in this regard.
c9ab452e5611 delta-find: check DELTA_BASE_REUSE_FORCE in the _DeltaSearch.__init__ Pierre-Yves David <pierre-yves.david@octobus.net> parents: 51321 diff changeset	629 assert (
c9ab452e5611 delta-find: check DELTA_BASE_REUSE_FORCE in the _DeltaSearch.__init__ Pierre-Yves David <pierre-yves.david@octobus.net> parents: 51321 diff changeset	630 cachedelta is None
c9ab452e5611 delta-find: check DELTA_BASE_REUSE_FORCE in the _DeltaSearch.__init__ Pierre-Yves David <pierre-yves.david@octobus.net> parents: 51321 diff changeset	631 or cachedelta[2] != DELTA_BASE_REUSE_FORCE
c9ab452e5611 delta-find: check DELTA_BASE_REUSE_FORCE in the _DeltaSearch.__init__ Pierre-Yves David <pierre-yves.david@octobus.net> parents: 51321 diff changeset	632 or not revlog.delta_config.general_delta
c9ab452e5611 delta-find: check DELTA_BASE_REUSE_FORCE in the _DeltaSearch.__init__ Pierre-Yves David <pierre-yves.david@octobus.net> parents: 51321 diff changeset	633 )
51317 c82e03b102a6 delta-find: introduce a _DeltaSearch object Pierre-Yves David <pierre-yves.david@octobus.net> parents: 51316 diff changeset	634 self.revlog = revlog
51327 49401b7dec0c delta-find: feed revinfo to _DeltaSearch Pierre-Yves David <pierre-yves.david@octobus.net> parents: 51325 diff changeset	635 self.revinfo = revinfo
49401b7dec0c delta-find: feed revinfo to _DeltaSearch Pierre-Yves David <pierre-yves.david@octobus.net> parents: 51325 diff changeset	636 self.textlen = revinfo.textlen
51317 c82e03b102a6 delta-find: introduce a _DeltaSearch object Pierre-Yves David <pierre-yves.david@octobus.net> parents: 51316 diff changeset	637 self.p1 = p1
c82e03b102a6 delta-find: introduce a _DeltaSearch object Pierre-Yves David <pierre-yves.david@octobus.net> parents: 51316 diff changeset	638 self.p2 = p2
c82e03b102a6 delta-find: introduce a _DeltaSearch object Pierre-Yves David <pierre-yves.david@octobus.net> parents: 51316 diff changeset	639 self.cachedelta = cachedelta
c82e03b102a6 delta-find: introduce a _DeltaSearch object Pierre-Yves David <pierre-yves.david@octobus.net> parents: 51316 diff changeset	640 self.excluded_bases = excluded_bases
51321 047358417d01 delta-find: move target_rev in the _DeltaSearch.__init__ Pierre-Yves David <pierre-yves.david@octobus.net> parents: 51320 diff changeset	641 if target_rev is None:
047358417d01 delta-find: move target_rev in the _DeltaSearch.__init__ Pierre-Yves David <pierre-yves.david@octobus.net> parents: 51320 diff changeset	642 self.target_rev = len(self.revlog)
51317 c82e03b102a6 delta-find: introduce a _DeltaSearch object Pierre-Yves David <pierre-yves.david@octobus.net> parents: 51316 diff changeset	643 self.target_rev = target_rev
51320 87fce139e927 delta-find: move snapshot_cache in the _DeltaSearch.__init__ Pierre-Yves David <pierre-yves.david@octobus.net> parents: 51319 diff changeset	644 if snapshot_cache is None:
87fce139e927 delta-find: move snapshot_cache in the _DeltaSearch.__init__ Pierre-Yves David <pierre-yves.david@octobus.net> parents: 51319 diff changeset	645 # map: base-rev: [snapshot-revs]
87fce139e927 delta-find: move snapshot_cache in the _DeltaSearch.__init__ Pierre-Yves David <pierre-yves.david@octobus.net> parents: 51319 diff changeset	646 snapshot_cache = SnapshotCache()
51317 c82e03b102a6 delta-find: introduce a _DeltaSearch object Pierre-Yves David <pierre-yves.david@octobus.net> parents: 51316 diff changeset	647 self.snapshot_cache = snapshot_cache
c82e03b102a6 delta-find: introduce a _DeltaSearch object Pierre-Yves David <pierre-yves.david@octobus.net> parents: 51316 diff changeset	648
51323 01db705bbe32 delta-find: move tested in the _DeltaSearch.__init__ Pierre-Yves David <pierre-yves.david@octobus.net> parents: 51322 diff changeset	649 self.tested = {nullrev}
01db705bbe32 delta-find: move tested in the _DeltaSearch.__init__ Pierre-Yves David <pierre-yves.david@octobus.net> parents: 51322 diff changeset	650
51346 cc806f20d756 delta-find: explicitly track stage of the search Pierre-Yves David <pierre-yves.david@octobus.net> parents: 51345 diff changeset	651 self.current_stage = _STAGE_UNSPECIFIED
51343 2a333d791ecf delta-find: introduce a base class for _DeltaSearch Pierre-Yves David <pierre-yves.david@octobus.net> parents: 51342 diff changeset	652 self.current_group = None
2a333d791ecf delta-find: introduce a base class for _DeltaSearch Pierre-Yves David <pierre-yves.david@octobus.net> parents: 51342 diff changeset	653 self._init_group()
51331 d0d869fccd20 delta-find: move away from the generator API for _DeltaSearch Pierre-Yves David <pierre-yves.david@octobus.net> parents: 51330 diff changeset	654
51341 f3f35b37f4b2 delta-find: move good delta code earlier in the class Pierre-Yves David <pierre-yves.david@octobus.net> parents: 51340 diff changeset	655 def is_good_delta_info(self, deltainfo):
f3f35b37f4b2 delta-find: move good delta code earlier in the class Pierre-Yves David <pierre-yves.david@octobus.net> parents: 51340 diff changeset	656 """Returns True if the given delta is good.
f3f35b37f4b2 delta-find: move good delta code earlier in the class Pierre-Yves David <pierre-yves.david@octobus.net> parents: 51340 diff changeset	657
f3f35b37f4b2 delta-find: move good delta code earlier in the class Pierre-Yves David <pierre-yves.david@octobus.net> parents: 51340 diff changeset	658 Good means that it is within the disk span, disk size, and chain length
f3f35b37f4b2 delta-find: move good delta code earlier in the class Pierre-Yves David <pierre-yves.david@octobus.net> parents: 51340 diff changeset	659 bounds that we know to be performant.
f3f35b37f4b2 delta-find: move good delta code earlier in the class Pierre-Yves David <pierre-yves.david@octobus.net> parents: 51340 diff changeset	660 """
f3f35b37f4b2 delta-find: move good delta code earlier in the class Pierre-Yves David <pierre-yves.david@octobus.net> parents: 51340 diff changeset	661 if not self._is_good_delta_info_universal(deltainfo):
f3f35b37f4b2 delta-find: move good delta code earlier in the class Pierre-Yves David <pierre-yves.david@octobus.net> parents: 51340 diff changeset	662 return False
f3f35b37f4b2 delta-find: move good delta code earlier in the class Pierre-Yves David <pierre-yves.david@octobus.net> parents: 51340 diff changeset	663 if not self._is_good_delta_info_chain_quality(deltainfo):
f3f35b37f4b2 delta-find: move good delta code earlier in the class Pierre-Yves David <pierre-yves.david@octobus.net> parents: 51340 diff changeset	664 return False
f3f35b37f4b2 delta-find: move good delta code earlier in the class Pierre-Yves David <pierre-yves.david@octobus.net> parents: 51340 diff changeset	665 if not self._is_good_delta_info_snapshot_constraints(deltainfo):
f3f35b37f4b2 delta-find: move good delta code earlier in the class Pierre-Yves David <pierre-yves.david@octobus.net> parents: 51340 diff changeset	666 return False
f3f35b37f4b2 delta-find: move good delta code earlier in the class Pierre-Yves David <pierre-yves.david@octobus.net> parents: 51340 diff changeset	667 return True
f3f35b37f4b2 delta-find: move good delta code earlier in the class Pierre-Yves David <pierre-yves.david@octobus.net> parents: 51340 diff changeset	668
f3f35b37f4b2 delta-find: move good delta code earlier in the class Pierre-Yves David <pierre-yves.david@octobus.net> parents: 51340 diff changeset	669 def _is_good_delta_info_universal(self, deltainfo):
f3f35b37f4b2 delta-find: move good delta code earlier in the class Pierre-Yves David <pierre-yves.david@octobus.net> parents: 51340 diff changeset	670 """Returns True if the given delta is good.
f3f35b37f4b2 delta-find: move good delta code earlier in the class Pierre-Yves David <pierre-yves.david@octobus.net> parents: 51340 diff changeset	671
f3f35b37f4b2 delta-find: move good delta code earlier in the class Pierre-Yves David <pierre-yves.david@octobus.net> parents: 51340 diff changeset	672 This performs generic checks needed by all format variants.
f3f35b37f4b2 delta-find: move good delta code earlier in the class Pierre-Yves David <pierre-yves.david@octobus.net> parents: 51340 diff changeset	673
f3f35b37f4b2 delta-find: move good delta code earlier in the class Pierre-Yves David <pierre-yves.david@octobus.net> parents: 51340 diff changeset	674 This is used by is_good_delta_info.
f3f35b37f4b2 delta-find: move good delta code earlier in the class Pierre-Yves David <pierre-yves.david@octobus.net> parents: 51340 diff changeset	675 """
f3f35b37f4b2 delta-find: move good delta code earlier in the class Pierre-Yves David <pierre-yves.david@octobus.net> parents: 51340 diff changeset	676
f3f35b37f4b2 delta-find: move good delta code earlier in the class Pierre-Yves David <pierre-yves.david@octobus.net> parents: 51340 diff changeset	677 if deltainfo is None:
f3f35b37f4b2 delta-find: move good delta code earlier in the class Pierre-Yves David <pierre-yves.david@octobus.net> parents: 51340 diff changeset	678 return False
f3f35b37f4b2 delta-find: move good delta code earlier in the class Pierre-Yves David <pierre-yves.david@octobus.net> parents: 51340 diff changeset	679
f3f35b37f4b2 delta-find: move good delta code earlier in the class Pierre-Yves David <pierre-yves.david@octobus.net> parents: 51340 diff changeset	680 # the DELTA_BASE_REUSE_FORCE case should have been taken care of sooner
f3f35b37f4b2 delta-find: move good delta code earlier in the class Pierre-Yves David <pierre-yves.david@octobus.net> parents: 51340 diff changeset	681 # so we should never end up asking such question. Adding the assert as
f3f35b37f4b2 delta-find: move good delta code earlier in the class Pierre-Yves David <pierre-yves.david@octobus.net> parents: 51340 diff changeset	682 # a safe-guard to detect anything that would be fishy in this regard.
f3f35b37f4b2 delta-find: move good delta code earlier in the class Pierre-Yves David <pierre-yves.david@octobus.net> parents: 51340 diff changeset	683 assert (
f3f35b37f4b2 delta-find: move good delta code earlier in the class Pierre-Yves David <pierre-yves.david@octobus.net> parents: 51340 diff changeset	684 self.revinfo.cachedelta is None
f3f35b37f4b2 delta-find: move good delta code earlier in the class Pierre-Yves David <pierre-yves.david@octobus.net> parents: 51340 diff changeset	685 or self.revinfo.cachedelta[2] != DELTA_BASE_REUSE_FORCE
f3f35b37f4b2 delta-find: move good delta code earlier in the class Pierre-Yves David <pierre-yves.david@octobus.net> parents: 51340 diff changeset	686 or not self.revlog.delta_config.general_delta
f3f35b37f4b2 delta-find: move good delta code earlier in the class Pierre-Yves David <pierre-yves.david@octobus.net> parents: 51340 diff changeset	687 )
f3f35b37f4b2 delta-find: move good delta code earlier in the class Pierre-Yves David <pierre-yves.david@octobus.net> parents: 51340 diff changeset	688
f3f35b37f4b2 delta-find: move good delta code earlier in the class Pierre-Yves David <pierre-yves.david@octobus.net> parents: 51340 diff changeset	689 # Bad delta from new delta size:
f3f35b37f4b2 delta-find: move good delta code earlier in the class Pierre-Yves David <pierre-yves.david@octobus.net> parents: 51340 diff changeset	690 #
f3f35b37f4b2 delta-find: move good delta code earlier in the class Pierre-Yves David <pierre-yves.david@octobus.net> parents: 51340 diff changeset	691 # If the delta size is larger than the target text, storing the delta
f3f35b37f4b2 delta-find: move good delta code earlier in the class Pierre-Yves David <pierre-yves.david@octobus.net> parents: 51340 diff changeset	692 # will be inefficient.
f3f35b37f4b2 delta-find: move good delta code earlier in the class Pierre-Yves David <pierre-yves.david@octobus.net> parents: 51340 diff changeset	693 if self.revinfo.textlen < deltainfo.deltalen:
f3f35b37f4b2 delta-find: move good delta code earlier in the class Pierre-Yves David <pierre-yves.david@octobus.net> parents: 51340 diff changeset	694 return False
f3f35b37f4b2 delta-find: move good delta code earlier in the class Pierre-Yves David <pierre-yves.david@octobus.net> parents: 51340 diff changeset	695
f3f35b37f4b2 delta-find: move good delta code earlier in the class Pierre-Yves David <pierre-yves.david@octobus.net> parents: 51340 diff changeset	696 return True
f3f35b37f4b2 delta-find: move good delta code earlier in the class Pierre-Yves David <pierre-yves.david@octobus.net> parents: 51340 diff changeset	697
f3f35b37f4b2 delta-find: move good delta code earlier in the class Pierre-Yves David <pierre-yves.david@octobus.net> parents: 51340 diff changeset	698 def _is_good_delta_info_chain_quality(self, deltainfo):
f3f35b37f4b2 delta-find: move good delta code earlier in the class Pierre-Yves David <pierre-yves.david@octobus.net> parents: 51340 diff changeset	699 """Returns True if the chain associated with the delta is good.
f3f35b37f4b2 delta-find: move good delta code earlier in the class Pierre-Yves David <pierre-yves.david@octobus.net> parents: 51340 diff changeset	700
f3f35b37f4b2 delta-find: move good delta code earlier in the class Pierre-Yves David <pierre-yves.david@octobus.net> parents: 51340 diff changeset	701 This performs checks for format that use delta chains.
f3f35b37f4b2 delta-find: move good delta code earlier in the class Pierre-Yves David <pierre-yves.david@octobus.net> parents: 51340 diff changeset	702
f3f35b37f4b2 delta-find: move good delta code earlier in the class Pierre-Yves David <pierre-yves.david@octobus.net> parents: 51340 diff changeset	703 This is used by is_good_delta_info.
f3f35b37f4b2 delta-find: move good delta code earlier in the class Pierre-Yves David <pierre-yves.david@octobus.net> parents: 51340 diff changeset	704 """
f3f35b37f4b2 delta-find: move good delta code earlier in the class Pierre-Yves David <pierre-yves.david@octobus.net> parents: 51340 diff changeset	705 # - 'deltainfo.distance' is the distance from the base revision --
f3f35b37f4b2 delta-find: move good delta code earlier in the class Pierre-Yves David <pierre-yves.david@octobus.net> parents: 51340 diff changeset	706 # bounding it limits the amount of I/O we need to do.
f3f35b37f4b2 delta-find: move good delta code earlier in the class Pierre-Yves David <pierre-yves.david@octobus.net> parents: 51340 diff changeset	707
f3f35b37f4b2 delta-find: move good delta code earlier in the class Pierre-Yves David <pierre-yves.david@octobus.net> parents: 51340 diff changeset	708 defaultmax = self.revinfo.textlen * 4
f3f35b37f4b2 delta-find: move good delta code earlier in the class Pierre-Yves David <pierre-yves.david@octobus.net> parents: 51340 diff changeset	709 maxdist = self.revlog.delta_config.max_deltachain_span
f3f35b37f4b2 delta-find: move good delta code earlier in the class Pierre-Yves David <pierre-yves.david@octobus.net> parents: 51340 diff changeset	710 if not maxdist:
f3f35b37f4b2 delta-find: move good delta code earlier in the class Pierre-Yves David <pierre-yves.david@octobus.net> parents: 51340 diff changeset	711 maxdist = deltainfo.distance # ensure the conditional pass
f3f35b37f4b2 delta-find: move good delta code earlier in the class Pierre-Yves David <pierre-yves.david@octobus.net> parents: 51340 diff changeset	712 maxdist = max(maxdist, defaultmax)
f3f35b37f4b2 delta-find: move good delta code earlier in the class Pierre-Yves David <pierre-yves.david@octobus.net> parents: 51340 diff changeset	713
f3f35b37f4b2 delta-find: move good delta code earlier in the class Pierre-Yves David <pierre-yves.david@octobus.net> parents: 51340 diff changeset	714 # Bad delta from read span:
f3f35b37f4b2 delta-find: move good delta code earlier in the class Pierre-Yves David <pierre-yves.david@octobus.net> parents: 51340 diff changeset	715 #
f3f35b37f4b2 delta-find: move good delta code earlier in the class Pierre-Yves David <pierre-yves.david@octobus.net> parents: 51340 diff changeset	716 # If the span of data read is larger than the maximum allowed.
f3f35b37f4b2 delta-find: move good delta code earlier in the class Pierre-Yves David <pierre-yves.david@octobus.net> parents: 51340 diff changeset	717 #
f3f35b37f4b2 delta-find: move good delta code earlier in the class Pierre-Yves David <pierre-yves.david@octobus.net> parents: 51340 diff changeset	718 # In the sparse-revlog case, we rely on the associated "sparse
f3f35b37f4b2 delta-find: move good delta code earlier in the class Pierre-Yves David <pierre-yves.david@octobus.net> parents: 51340 diff changeset	719 # reading" to avoid issue related to the span of data. In theory, it
f3f35b37f4b2 delta-find: move good delta code earlier in the class Pierre-Yves David <pierre-yves.david@octobus.net> parents: 51340 diff changeset	720 # would be possible to build pathological revlog where delta pattern
f3f35b37f4b2 delta-find: move good delta code earlier in the class Pierre-Yves David <pierre-yves.david@octobus.net> parents: 51340 diff changeset	721 # would lead to too many reads. However, they do not happen in
f3f35b37f4b2 delta-find: move good delta code earlier in the class Pierre-Yves David <pierre-yves.david@octobus.net> parents: 51340 diff changeset	722 # practice at all. So we skip the span check entirely.
f3f35b37f4b2 delta-find: move good delta code earlier in the class Pierre-Yves David <pierre-yves.david@octobus.net> parents: 51340 diff changeset	723 if (
f3f35b37f4b2 delta-find: move good delta code earlier in the class Pierre-Yves David <pierre-yves.david@octobus.net> parents: 51340 diff changeset	724 not self.revlog.delta_config.sparse_revlog
f3f35b37f4b2 delta-find: move good delta code earlier in the class Pierre-Yves David <pierre-yves.david@octobus.net> parents: 51340 diff changeset	725 and maxdist < deltainfo.distance
f3f35b37f4b2 delta-find: move good delta code earlier in the class Pierre-Yves David <pierre-yves.david@octobus.net> parents: 51340 diff changeset	726 ):
f3f35b37f4b2 delta-find: move good delta code earlier in the class Pierre-Yves David <pierre-yves.david@octobus.net> parents: 51340 diff changeset	727 return False
f3f35b37f4b2 delta-find: move good delta code earlier in the class Pierre-Yves David <pierre-yves.david@octobus.net> parents: 51340 diff changeset	728
f3f35b37f4b2 delta-find: move good delta code earlier in the class Pierre-Yves David <pierre-yves.david@octobus.net> parents: 51340 diff changeset	729 # Bad delta from cumulated payload size:
f3f35b37f4b2 delta-find: move good delta code earlier in the class Pierre-Yves David <pierre-yves.david@octobus.net> parents: 51340 diff changeset	730 #
f3f35b37f4b2 delta-find: move good delta code earlier in the class Pierre-Yves David <pierre-yves.david@octobus.net> parents: 51340 diff changeset	731 # - 'deltainfo.compresseddeltalen' is the sum of the total size of
f3f35b37f4b2 delta-find: move good delta code earlier in the class Pierre-Yves David <pierre-yves.david@octobus.net> parents: 51340 diff changeset	732 # deltas we need to apply -- bounding it limits the amount of CPU
f3f35b37f4b2 delta-find: move good delta code earlier in the class Pierre-Yves David <pierre-yves.david@octobus.net> parents: 51340 diff changeset	733 # we consume.
f3f35b37f4b2 delta-find: move good delta code earlier in the class Pierre-Yves David <pierre-yves.david@octobus.net> parents: 51340 diff changeset	734 max_chain_data = self.revinfo.textlen * LIMIT_DELTA2TEXT
f3f35b37f4b2 delta-find: move good delta code earlier in the class Pierre-Yves David <pierre-yves.david@octobus.net> parents: 51340 diff changeset	735 # If the sum of delta get larger than K * target text length.
f3f35b37f4b2 delta-find: move good delta code earlier in the class Pierre-Yves David <pierre-yves.david@octobus.net> parents: 51340 diff changeset	736 if max_chain_data < deltainfo.compresseddeltalen:
f3f35b37f4b2 delta-find: move good delta code earlier in the class Pierre-Yves David <pierre-yves.david@octobus.net> parents: 51340 diff changeset	737 return False
f3f35b37f4b2 delta-find: move good delta code earlier in the class Pierre-Yves David <pierre-yves.david@octobus.net> parents: 51340 diff changeset	738
f3f35b37f4b2 delta-find: move good delta code earlier in the class Pierre-Yves David <pierre-yves.david@octobus.net> parents: 51340 diff changeset	739 # Bad delta from chain length:
f3f35b37f4b2 delta-find: move good delta code earlier in the class Pierre-Yves David <pierre-yves.david@octobus.net> parents: 51340 diff changeset	740 #
f3f35b37f4b2 delta-find: move good delta code earlier in the class Pierre-Yves David <pierre-yves.david@octobus.net> parents: 51340 diff changeset	741 # If the number of delta in the chain gets too high.
f3f35b37f4b2 delta-find: move good delta code earlier in the class Pierre-Yves David <pierre-yves.david@octobus.net> parents: 51340 diff changeset	742 if (
f3f35b37f4b2 delta-find: move good delta code earlier in the class Pierre-Yves David <pierre-yves.david@octobus.net> parents: 51340 diff changeset	743 self.revlog.delta_config.max_chain_len
f3f35b37f4b2 delta-find: move good delta code earlier in the class Pierre-Yves David <pierre-yves.david@octobus.net> parents: 51340 diff changeset	744 and self.revlog.delta_config.max_chain_len < deltainfo.chainlen
f3f35b37f4b2 delta-find: move good delta code earlier in the class Pierre-Yves David <pierre-yves.david@octobus.net> parents: 51340 diff changeset	745 ):
f3f35b37f4b2 delta-find: move good delta code earlier in the class Pierre-Yves David <pierre-yves.david@octobus.net> parents: 51340 diff changeset	746 return False
f3f35b37f4b2 delta-find: move good delta code earlier in the class Pierre-Yves David <pierre-yves.david@octobus.net> parents: 51340 diff changeset	747 return True
f3f35b37f4b2 delta-find: move good delta code earlier in the class Pierre-Yves David <pierre-yves.david@octobus.net> parents: 51340 diff changeset	748
f3f35b37f4b2 delta-find: move good delta code earlier in the class Pierre-Yves David <pierre-yves.david@octobus.net> parents: 51340 diff changeset	749 def _is_good_delta_info_snapshot_constraints(self, deltainfo):
f3f35b37f4b2 delta-find: move good delta code earlier in the class Pierre-Yves David <pierre-yves.david@octobus.net> parents: 51340 diff changeset	750 """Returns True if the chain associated with snapshots
f3f35b37f4b2 delta-find: move good delta code earlier in the class Pierre-Yves David <pierre-yves.david@octobus.net> parents: 51340 diff changeset	751
f3f35b37f4b2 delta-find: move good delta code earlier in the class Pierre-Yves David <pierre-yves.david@octobus.net> parents: 51340 diff changeset	752 This performs checks for format that use sparse-revlog and intermediate
f3f35b37f4b2 delta-find: move good delta code earlier in the class Pierre-Yves David <pierre-yves.david@octobus.net> parents: 51340 diff changeset	753 snapshots.
f3f35b37f4b2 delta-find: move good delta code earlier in the class Pierre-Yves David <pierre-yves.david@octobus.net> parents: 51340 diff changeset	754
f3f35b37f4b2 delta-find: move good delta code earlier in the class Pierre-Yves David <pierre-yves.david@octobus.net> parents: 51340 diff changeset	755 This is used by is_good_delta_info.
f3f35b37f4b2 delta-find: move good delta code earlier in the class Pierre-Yves David <pierre-yves.david@octobus.net> parents: 51340 diff changeset	756 """
51342 866ab9f447d4 delta-find: simplify the delta checking function for snapshot Pierre-Yves David <pierre-yves.david@octobus.net> parents: 51341 diff changeset	757 # if not a snapshot, this method has no filtering to do
866ab9f447d4 delta-find: simplify the delta checking function for snapshot Pierre-Yves David <pierre-yves.david@octobus.net> parents: 51341 diff changeset	758 if deltainfo.snapshotdepth is None:
866ab9f447d4 delta-find: simplify the delta checking function for snapshot Pierre-Yves David <pierre-yves.david@octobus.net> parents: 51341 diff changeset	759 return True
51341 f3f35b37f4b2 delta-find: move good delta code earlier in the class Pierre-Yves David <pierre-yves.david@octobus.net> parents: 51340 diff changeset	760 # bad delta from intermediate snapshot size limit
f3f35b37f4b2 delta-find: move good delta code earlier in the class Pierre-Yves David <pierre-yves.david@octobus.net> parents: 51340 diff changeset	761 #
f3f35b37f4b2 delta-find: move good delta code earlier in the class Pierre-Yves David <pierre-yves.david@octobus.net> parents: 51340 diff changeset	762 # If an intermediate snapshot size is higher than the limit. The
f3f35b37f4b2 delta-find: move good delta code earlier in the class Pierre-Yves David <pierre-yves.david@octobus.net> parents: 51340 diff changeset	763 # limit exist to prevent endless chain of intermediate delta to be
f3f35b37f4b2 delta-find: move good delta code earlier in the class Pierre-Yves David <pierre-yves.david@octobus.net> parents: 51340 diff changeset	764 # created.
f3f35b37f4b2 delta-find: move good delta code earlier in the class Pierre-Yves David <pierre-yves.david@octobus.net> parents: 51340 diff changeset	765 if (
51342 866ab9f447d4 delta-find: simplify the delta checking function for snapshot Pierre-Yves David <pierre-yves.david@octobus.net> parents: 51341 diff changeset	766 self.revinfo.textlen >> deltainfo.snapshotdepth
866ab9f447d4 delta-find: simplify the delta checking function for snapshot Pierre-Yves David <pierre-yves.david@octobus.net> parents: 51341 diff changeset	767 ) < deltainfo.deltalen:
51341 f3f35b37f4b2 delta-find: move good delta code earlier in the class Pierre-Yves David <pierre-yves.david@octobus.net> parents: 51340 diff changeset	768 return False
f3f35b37f4b2 delta-find: move good delta code earlier in the class Pierre-Yves David <pierre-yves.david@octobus.net> parents: 51340 diff changeset	769
f3f35b37f4b2 delta-find: move good delta code earlier in the class Pierre-Yves David <pierre-yves.david@octobus.net> parents: 51340 diff changeset	770 # bad delta if new intermediate snapshot is larger than the previous
f3f35b37f4b2 delta-find: move good delta code earlier in the class Pierre-Yves David <pierre-yves.david@octobus.net> parents: 51340 diff changeset	771 # snapshot
51342 866ab9f447d4 delta-find: simplify the delta checking function for snapshot Pierre-Yves David <pierre-yves.david@octobus.net> parents: 51341 diff changeset	772 if self.revlog.length(deltainfo.base) < deltainfo.deltalen:
51341 f3f35b37f4b2 delta-find: move good delta code earlier in the class Pierre-Yves David <pierre-yves.david@octobus.net> parents: 51340 diff changeset	773 return False
f3f35b37f4b2 delta-find: move good delta code earlier in the class Pierre-Yves David <pierre-yves.david@octobus.net> parents: 51340 diff changeset	774
f3f35b37f4b2 delta-find: move good delta code earlier in the class Pierre-Yves David <pierre-yves.david@octobus.net> parents: 51340 diff changeset	775 return True
f3f35b37f4b2 delta-find: move good delta code earlier in the class Pierre-Yves David <pierre-yves.david@octobus.net> parents: 51340 diff changeset	776
51331 d0d869fccd20 delta-find: move away from the generator API for _DeltaSearch Pierre-Yves David <pierre-yves.david@octobus.net> parents: 51330 diff changeset	777 @property
d0d869fccd20 delta-find: move away from the generator API for _DeltaSearch Pierre-Yves David <pierre-yves.david@octobus.net> parents: 51330 diff changeset	778 def done(self):
d0d869fccd20 delta-find: move away from the generator API for _DeltaSearch Pierre-Yves David <pierre-yves.david@octobus.net> parents: 51330 diff changeset	779 """True when all possible candidate have been tested"""
d0d869fccd20 delta-find: move away from the generator API for _DeltaSearch Pierre-Yves David <pierre-yves.david@octobus.net> parents: 51330 diff changeset	780 return self.current_group is None
d0d869fccd20 delta-find: move away from the generator API for _DeltaSearch Pierre-Yves David <pierre-yves.david@octobus.net> parents: 51330 diff changeset	781
51343 2a333d791ecf delta-find: introduce a base class for _DeltaSearch Pierre-Yves David <pierre-yves.david@octobus.net> parents: 51342 diff changeset	782 @abc.abstractmethod
51331 d0d869fccd20 delta-find: move away from the generator API for _DeltaSearch Pierre-Yves David <pierre-yves.david@octobus.net> parents: 51330 diff changeset	783 def next_group(self, good_delta=None):
d0d869fccd20 delta-find: move away from the generator API for _DeltaSearch Pierre-Yves David <pierre-yves.david@octobus.net> parents: 51330 diff changeset	784 """move to the next group to test
d0d869fccd20 delta-find: move away from the generator API for _DeltaSearch Pierre-Yves David <pierre-yves.david@octobus.net> parents: 51330 diff changeset	785
d0d869fccd20 delta-find: move away from the generator API for _DeltaSearch Pierre-Yves David <pierre-yves.david@octobus.net> parents: 51330 diff changeset	786 The group of revision to test will be available in
d0d869fccd20 delta-find: move away from the generator API for _DeltaSearch Pierre-Yves David <pierre-yves.david@octobus.net> parents: 51330 diff changeset	787 `self.current_group`. If the previous group had any good delta, the
d0d869fccd20 delta-find: move away from the generator API for _DeltaSearch Pierre-Yves David <pierre-yves.david@octobus.net> parents: 51330 diff changeset	788 best one can be passed as the `good_delta` parameter to help selecting
d0d869fccd20 delta-find: move away from the generator API for _DeltaSearch Pierre-Yves David <pierre-yves.david@octobus.net> parents: 51330 diff changeset	789 the next group.
d0d869fccd20 delta-find: move away from the generator API for _DeltaSearch Pierre-Yves David <pierre-yves.david@octobus.net> parents: 51330 diff changeset	790
d0d869fccd20 delta-find: move away from the generator API for _DeltaSearch Pierre-Yves David <pierre-yves.david@octobus.net> parents: 51330 diff changeset	791 If not revision remains to be, `self.done` will be True and
d0d869fccd20 delta-find: move away from the generator API for _DeltaSearch Pierre-Yves David <pierre-yves.david@octobus.net> parents: 51330 diff changeset	792 `self.current_group` will be None.
d0d869fccd20 delta-find: move away from the generator API for _DeltaSearch Pierre-Yves David <pierre-yves.david@octobus.net> parents: 51330 diff changeset	793 """
51343 2a333d791ecf delta-find: introduce a base class for _DeltaSearch Pierre-Yves David <pierre-yves.david@octobus.net> parents: 51342 diff changeset	794 pass
2a333d791ecf delta-find: introduce a base class for _DeltaSearch Pierre-Yves David <pierre-yves.david@octobus.net> parents: 51342 diff changeset	795
2a333d791ecf delta-find: introduce a base class for _DeltaSearch Pierre-Yves David <pierre-yves.david@octobus.net> parents: 51342 diff changeset	796 @abc.abstractmethod
2a333d791ecf delta-find: introduce a base class for _DeltaSearch Pierre-Yves David <pierre-yves.david@octobus.net> parents: 51342 diff changeset	797 def _init_group(self):
2a333d791ecf delta-find: introduce a base class for _DeltaSearch Pierre-Yves David <pierre-yves.david@octobus.net> parents: 51342 diff changeset	798 pass
2a333d791ecf delta-find: introduce a base class for _DeltaSearch Pierre-Yves David <pierre-yves.david@octobus.net> parents: 51342 diff changeset	799
2a333d791ecf delta-find: introduce a base class for _DeltaSearch Pierre-Yves David <pierre-yves.david@octobus.net> parents: 51342 diff changeset	800
51344 555826073625 delta-find: introduce and use specialized _DeltaSearch class Pierre-Yves David <pierre-yves.david@octobus.net> parents: 51343 diff changeset	801 class _NoDeltaSearch(_BaseDeltaSearch):
555826073625 delta-find: introduce and use specialized _DeltaSearch class Pierre-Yves David <pierre-yves.david@octobus.net> parents: 51343 diff changeset	802 """Search for no delta.
555826073625 delta-find: introduce and use specialized _DeltaSearch class Pierre-Yves David <pierre-yves.david@octobus.net> parents: 51343 diff changeset	803
555826073625 delta-find: introduce and use specialized _DeltaSearch class Pierre-Yves David <pierre-yves.david@octobus.net> parents: 51343 diff changeset	804 This search variant is to be used in case where we should not store delta.
555826073625 delta-find: introduce and use specialized _DeltaSearch class Pierre-Yves David <pierre-yves.david@octobus.net> parents: 51343 diff changeset	805 """
555826073625 delta-find: introduce and use specialized _DeltaSearch class Pierre-Yves David <pierre-yves.david@octobus.net> parents: 51343 diff changeset	806
555826073625 delta-find: introduce and use specialized _DeltaSearch class Pierre-Yves David <pierre-yves.david@octobus.net> parents: 51343 diff changeset	807 def _init_group(self):
51346 cc806f20d756 delta-find: explicitly track stage of the search Pierre-Yves David <pierre-yves.david@octobus.net> parents: 51345 diff changeset	808 self.current_stage = _STAGE_FULL
51344 555826073625 delta-find: introduce and use specialized _DeltaSearch class Pierre-Yves David <pierre-yves.david@octobus.net> parents: 51343 diff changeset	809
555826073625 delta-find: introduce and use specialized _DeltaSearch class Pierre-Yves David <pierre-yves.david@octobus.net> parents: 51343 diff changeset	810 def next_group(self, good_delta=None):
555826073625 delta-find: introduce and use specialized _DeltaSearch class Pierre-Yves David <pierre-yves.david@octobus.net> parents: 51343 diff changeset	811 pass
555826073625 delta-find: introduce and use specialized _DeltaSearch class Pierre-Yves David <pierre-yves.david@octobus.net> parents: 51343 diff changeset	812
555826073625 delta-find: introduce and use specialized _DeltaSearch class Pierre-Yves David <pierre-yves.david@octobus.net> parents: 51343 diff changeset	813
555826073625 delta-find: introduce and use specialized _DeltaSearch class Pierre-Yves David <pierre-yves.david@octobus.net> parents: 51343 diff changeset	814 class _PrevDeltaSearch(_BaseDeltaSearch):
555826073625 delta-find: introduce and use specialized _DeltaSearch class Pierre-Yves David <pierre-yves.david@octobus.net> parents: 51343 diff changeset	815 """Search for delta against the previous revision only
555826073625 delta-find: introduce and use specialized _DeltaSearch class Pierre-Yves David <pierre-yves.david@octobus.net> parents: 51343 diff changeset	816
555826073625 delta-find: introduce and use specialized _DeltaSearch class Pierre-Yves David <pierre-yves.david@octobus.net> parents: 51343 diff changeset	817 This search variant is to be used when the format does not allow for delta
555826073625 delta-find: introduce and use specialized _DeltaSearch class Pierre-Yves David <pierre-yves.david@octobus.net> parents: 51343 diff changeset	818 against arbitrary bases.
555826073625 delta-find: introduce and use specialized _DeltaSearch class Pierre-Yves David <pierre-yves.david@octobus.net> parents: 51343 diff changeset	819 """
555826073625 delta-find: introduce and use specialized _DeltaSearch class Pierre-Yves David <pierre-yves.david@octobus.net> parents: 51343 diff changeset	820
555826073625 delta-find: introduce and use specialized _DeltaSearch class Pierre-Yves David <pierre-yves.david@octobus.net> parents: 51343 diff changeset	821 def _init_group(self):
51346 cc806f20d756 delta-find: explicitly track stage of the search Pierre-Yves David <pierre-yves.david@octobus.net> parents: 51345 diff changeset	822 self.current_stage = _STAGE_PREV
51344 555826073625 delta-find: introduce and use specialized _DeltaSearch class Pierre-Yves David <pierre-yves.david@octobus.net> parents: 51343 diff changeset	823 self.current_group = [self.target_rev - 1]
51348 bbbfa00d1e67 delta-find: remove the "candidate groups" layer Pierre-Yves David <pierre-yves.david@octobus.net> parents: 51347 diff changeset	824 self.tested.update(self.current_group)
51344 555826073625 delta-find: introduce and use specialized _DeltaSearch class Pierre-Yves David <pierre-yves.david@octobus.net> parents: 51343 diff changeset	825
555826073625 delta-find: introduce and use specialized _DeltaSearch class Pierre-Yves David <pierre-yves.david@octobus.net> parents: 51343 diff changeset	826 def next_group(self, good_delta=None):
51346 cc806f20d756 delta-find: explicitly track stage of the search Pierre-Yves David <pierre-yves.david@octobus.net> parents: 51345 diff changeset	827 self.current_stage = _STAGE_FULL
51344 555826073625 delta-find: introduce and use specialized _DeltaSearch class Pierre-Yves David <pierre-yves.david@octobus.net> parents: 51343 diff changeset	828 self.current_group = None
555826073625 delta-find: introduce and use specialized _DeltaSearch class Pierre-Yves David <pierre-yves.david@octobus.net> parents: 51343 diff changeset	829
555826073625 delta-find: introduce and use specialized _DeltaSearch class Pierre-Yves David <pierre-yves.david@octobus.net> parents: 51343 diff changeset	830
51343 2a333d791ecf delta-find: introduce a base class for _DeltaSearch Pierre-Yves David <pierre-yves.david@octobus.net> parents: 51342 diff changeset	831 class _DeltaSearch(_BaseDeltaSearch):
51344 555826073625 delta-find: introduce and use specialized _DeltaSearch class Pierre-Yves David <pierre-yves.david@octobus.net> parents: 51343 diff changeset	832 """Generic delta search variants
555826073625 delta-find: introduce and use specialized _DeltaSearch class Pierre-Yves David <pierre-yves.david@octobus.net> parents: 51343 diff changeset	833
555826073625 delta-find: introduce and use specialized _DeltaSearch class Pierre-Yves David <pierre-yves.david@octobus.net> parents: 51343 diff changeset	834 (expect this to be split further)
555826073625 delta-find: introduce and use specialized _DeltaSearch class Pierre-Yves David <pierre-yves.david@octobus.net> parents: 51343 diff changeset	835 """
555826073625 delta-find: introduce and use specialized _DeltaSearch class Pierre-Yves David <pierre-yves.david@octobus.net> parents: 51343 diff changeset	836
51343 2a333d791ecf delta-find: introduce a base class for _DeltaSearch Pierre-Yves David <pierre-yves.david@octobus.net> parents: 51342 diff changeset	837 def _init_group(self):
51344 555826073625 delta-find: introduce and use specialized _DeltaSearch class Pierre-Yves David <pierre-yves.david@octobus.net> parents: 51343 diff changeset	838 # Why search for delta base if we cannot use a delta base ?
555826073625 delta-find: introduce and use specialized _DeltaSearch class Pierre-Yves David <pierre-yves.david@octobus.net> parents: 51343 diff changeset	839 # also see issue6056
555826073625 delta-find: introduce and use specialized _DeltaSearch class Pierre-Yves David <pierre-yves.david@octobus.net> parents: 51343 diff changeset	840 assert self.revlog.delta_config.general_delta
51348 bbbfa00d1e67 delta-find: remove the "candidate groups" layer Pierre-Yves David <pierre-yves.david@octobus.net> parents: 51347 diff changeset	841 self._candidates_iterator = self._refined_groups()
51343 2a333d791ecf delta-find: introduce a base class for _DeltaSearch Pierre-Yves David <pierre-yves.david@octobus.net> parents: 51342 diff changeset	842 self._last_good = None
51349 7083b33a2699 delta-find: explicitly deal with usage of the cached revision Pierre-Yves David <pierre-yves.david@octobus.net> parents: 51348 diff changeset	843 if (
7083b33a2699 delta-find: explicitly deal with usage of the cached revision Pierre-Yves David <pierre-yves.david@octobus.net> parents: 51348 diff changeset	844 self.cachedelta is not None
7083b33a2699 delta-find: explicitly deal with usage of the cached revision Pierre-Yves David <pierre-yves.david@octobus.net> parents: 51348 diff changeset	845 and self.cachedelta[2] > DELTA_BASE_REUSE_NO
7083b33a2699 delta-find: explicitly deal with usage of the cached revision Pierre-Yves David <pierre-yves.david@octobus.net> parents: 51348 diff changeset	846 and self._pre_filter_rev(self.cachedelta[0])
7083b33a2699 delta-find: explicitly deal with usage of the cached revision Pierre-Yves David <pierre-yves.david@octobus.net> parents: 51348 diff changeset	847 ):
7083b33a2699 delta-find: explicitly deal with usage of the cached revision Pierre-Yves David <pierre-yves.david@octobus.net> parents: 51348 diff changeset	848 # First we try to reuse a the delta contained in the bundle. (or from
7083b33a2699 delta-find: explicitly deal with usage of the cached revision Pierre-Yves David <pierre-yves.david@octobus.net> parents: 51348 diff changeset	849 # the source revlog)
7083b33a2699 delta-find: explicitly deal with usage of the cached revision Pierre-Yves David <pierre-yves.david@octobus.net> parents: 51348 diff changeset	850 #
7083b33a2699 delta-find: explicitly deal with usage of the cached revision Pierre-Yves David <pierre-yves.david@octobus.net> parents: 51348 diff changeset	851 # This logic only applies to general delta repositories and can be
7083b33a2699 delta-find: explicitly deal with usage of the cached revision Pierre-Yves David <pierre-yves.david@octobus.net> parents: 51348 diff changeset	852 # disabled through configuration. Disabling reuse source delta is
7083b33a2699 delta-find: explicitly deal with usage of the cached revision Pierre-Yves David <pierre-yves.david@octobus.net> parents: 51348 diff changeset	853 # useful when we want to make sure we recomputed "optimal" deltas.
7083b33a2699 delta-find: explicitly deal with usage of the cached revision Pierre-Yves David <pierre-yves.david@octobus.net> parents: 51348 diff changeset	854 self.current_stage = _STAGE_CACHED
7083b33a2699 delta-find: explicitly deal with usage of the cached revision Pierre-Yves David <pierre-yves.david@octobus.net> parents: 51348 diff changeset	855 self._internal_group = (self.cachedelta[0],)
7083b33a2699 delta-find: explicitly deal with usage of the cached revision Pierre-Yves David <pierre-yves.david@octobus.net> parents: 51348 diff changeset	856 self._internal_idx = 0
7083b33a2699 delta-find: explicitly deal with usage of the cached revision Pierre-Yves David <pierre-yves.david@octobus.net> parents: 51348 diff changeset	857 self.current_group = self._internal_group
7083b33a2699 delta-find: explicitly deal with usage of the cached revision Pierre-Yves David <pierre-yves.david@octobus.net> parents: 51348 diff changeset	858 self.tested.update(self.current_group)
7083b33a2699 delta-find: explicitly deal with usage of the cached revision Pierre-Yves David <pierre-yves.david@octobus.net> parents: 51348 diff changeset	859 else:
7083b33a2699 delta-find: explicitly deal with usage of the cached revision Pierre-Yves David <pierre-yves.david@octobus.net> parents: 51348 diff changeset	860 self._next_internal_group()
51348 bbbfa00d1e67 delta-find: remove the "candidate groups" layer Pierre-Yves David <pierre-yves.david@octobus.net> parents: 51347 diff changeset	861
bbbfa00d1e67 delta-find: remove the "candidate groups" layer Pierre-Yves David <pierre-yves.david@octobus.net> parents: 51347 diff changeset	862 def _next_internal_group(self):
bbbfa00d1e67 delta-find: remove the "candidate groups" layer Pierre-Yves David <pierre-yves.david@octobus.net> parents: 51347 diff changeset	863 # self._internal_group can be larger than self.current_group
bbbfa00d1e67 delta-find: remove the "candidate groups" layer Pierre-Yves David <pierre-yves.david@octobus.net> parents: 51347 diff changeset	864 self._internal_idx = 0
bbbfa00d1e67 delta-find: remove the "candidate groups" layer Pierre-Yves David <pierre-yves.david@octobus.net> parents: 51347 diff changeset	865 group = self._candidates_iterator.send(self._last_good)
bbbfa00d1e67 delta-find: remove the "candidate groups" layer Pierre-Yves David <pierre-yves.david@octobus.net> parents: 51347 diff changeset	866 if group is not None:
bbbfa00d1e67 delta-find: remove the "candidate groups" layer Pierre-Yves David <pierre-yves.david@octobus.net> parents: 51347 diff changeset	867 group = self._pre_filter_candidate_revs(group)
bbbfa00d1e67 delta-find: remove the "candidate groups" layer Pierre-Yves David <pierre-yves.david@octobus.net> parents: 51347 diff changeset	868 self._internal_group = group
bbbfa00d1e67 delta-find: remove the "candidate groups" layer Pierre-Yves David <pierre-yves.david@octobus.net> parents: 51347 diff changeset	869 if self._internal_group is None:
bbbfa00d1e67 delta-find: remove the "candidate groups" layer Pierre-Yves David <pierre-yves.david@octobus.net> parents: 51347 diff changeset	870 self.current_group = None
bbbfa00d1e67 delta-find: remove the "candidate groups" layer Pierre-Yves David <pierre-yves.david@octobus.net> parents: 51347 diff changeset	871 elif len(self._internal_group) == 0:
bbbfa00d1e67 delta-find: remove the "candidate groups" layer Pierre-Yves David <pierre-yves.david@octobus.net> parents: 51347 diff changeset	872 self.next_group()
bbbfa00d1e67 delta-find: remove the "candidate groups" layer Pierre-Yves David <pierre-yves.david@octobus.net> parents: 51347 diff changeset	873 else:
bbbfa00d1e67 delta-find: remove the "candidate groups" layer Pierre-Yves David <pierre-yves.david@octobus.net> parents: 51347 diff changeset	874 chunk_size = self.revlog.delta_config.candidate_group_chunk_size
bbbfa00d1e67 delta-find: remove the "candidate groups" layer Pierre-Yves David <pierre-yves.david@octobus.net> parents: 51347 diff changeset	875 if chunk_size > 0:
bbbfa00d1e67 delta-find: remove the "candidate groups" layer Pierre-Yves David <pierre-yves.david@octobus.net> parents: 51347 diff changeset	876 self.current_group = self._internal_group[:chunk_size]
bbbfa00d1e67 delta-find: remove the "candidate groups" layer Pierre-Yves David <pierre-yves.david@octobus.net> parents: 51347 diff changeset	877 self._internal_idx += chunk_size
bbbfa00d1e67 delta-find: remove the "candidate groups" layer Pierre-Yves David <pierre-yves.david@octobus.net> parents: 51347 diff changeset	878 else:
bbbfa00d1e67 delta-find: remove the "candidate groups" layer Pierre-Yves David <pierre-yves.david@octobus.net> parents: 51347 diff changeset	879 self.current_group = self._internal_group
bbbfa00d1e67 delta-find: remove the "candidate groups" layer Pierre-Yves David <pierre-yves.david@octobus.net> parents: 51347 diff changeset	880 self._internal_idx += len(self.current_group)
bbbfa00d1e67 delta-find: remove the "candidate groups" layer Pierre-Yves David <pierre-yves.david@octobus.net> parents: 51347 diff changeset	881
bbbfa00d1e67 delta-find: remove the "candidate groups" layer Pierre-Yves David <pierre-yves.david@octobus.net> parents: 51347 diff changeset	882 self.tested.update(self.current_group)
51343 2a333d791ecf delta-find: introduce a base class for _DeltaSearch Pierre-Yves David <pierre-yves.david@octobus.net> parents: 51342 diff changeset	883
2a333d791ecf delta-find: introduce a base class for _DeltaSearch Pierre-Yves David <pierre-yves.david@octobus.net> parents: 51342 diff changeset	884 def next_group(self, good_delta=None):
51348 bbbfa00d1e67 delta-find: remove the "candidate groups" layer Pierre-Yves David <pierre-yves.david@octobus.net> parents: 51347 diff changeset	885 old_good = self._last_good
51331 d0d869fccd20 delta-find: move away from the generator API for _DeltaSearch Pierre-Yves David <pierre-yves.david@octobus.net> parents: 51330 diff changeset	886 if good_delta is not None:
d0d869fccd20 delta-find: move away from the generator API for _DeltaSearch Pierre-Yves David <pierre-yves.david@octobus.net> parents: 51330 diff changeset	887 self._last_good = good_delta.base
51349 7083b33a2699 delta-find: explicitly deal with usage of the cached revision Pierre-Yves David <pierre-yves.david@octobus.net> parents: 51348 diff changeset	888 if self.current_stage == _STAGE_CACHED and good_delta is not None:
7083b33a2699 delta-find: explicitly deal with usage of the cached revision Pierre-Yves David <pierre-yves.david@octobus.net> parents: 51348 diff changeset	889 # the cache is good, let us use the cache as requested
7083b33a2699 delta-find: explicitly deal with usage of the cached revision Pierre-Yves David <pierre-yves.david@octobus.net> parents: 51348 diff changeset	890 self._candidates_iterator = None
7083b33a2699 delta-find: explicitly deal with usage of the cached revision Pierre-Yves David <pierre-yves.david@octobus.net> parents: 51348 diff changeset	891 self._internal_group = None
7083b33a2699 delta-find: explicitly deal with usage of the cached revision Pierre-Yves David <pierre-yves.david@octobus.net> parents: 51348 diff changeset	892 self._internal_idx = None
7083b33a2699 delta-find: explicitly deal with usage of the cached revision Pierre-Yves David <pierre-yves.david@octobus.net> parents: 51348 diff changeset	893 self.current_group = None
7083b33a2699 delta-find: explicitly deal with usage of the cached revision Pierre-Yves David <pierre-yves.david@octobus.net> parents: 51348 diff changeset	894 return
7083b33a2699 delta-find: explicitly deal with usage of the cached revision Pierre-Yves David <pierre-yves.david@octobus.net> parents: 51348 diff changeset	895
51348 bbbfa00d1e67 delta-find: remove the "candidate groups" layer Pierre-Yves David <pierre-yves.david@octobus.net> parents: 51347 diff changeset	896 if (self._internal_idx < len(self._internal_group)) and (
bbbfa00d1e67 delta-find: remove the "candidate groups" layer Pierre-Yves David <pierre-yves.david@octobus.net> parents: 51347 diff changeset	897 old_good != good_delta
bbbfa00d1e67 delta-find: remove the "candidate groups" layer Pierre-Yves David <pierre-yves.david@octobus.net> parents: 51347 diff changeset	898 ):
bbbfa00d1e67 delta-find: remove the "candidate groups" layer Pierre-Yves David <pierre-yves.david@octobus.net> parents: 51347 diff changeset	899 # When the size of the candidate group is big, it can result in
bbbfa00d1e67 delta-find: remove the "candidate groups" layer Pierre-Yves David <pierre-yves.david@octobus.net> parents: 51347 diff changeset	900 # a quite significant performance impact. To reduce this, we
bbbfa00d1e67 delta-find: remove the "candidate groups" layer Pierre-Yves David <pierre-yves.david@octobus.net> parents: 51347 diff changeset	901 # can send them in smaller batches until the new batch does not
bbbfa00d1e67 delta-find: remove the "candidate groups" layer Pierre-Yves David <pierre-yves.david@octobus.net> parents: 51347 diff changeset	902 # provide any improvements.
bbbfa00d1e67 delta-find: remove the "candidate groups" layer Pierre-Yves David <pierre-yves.david@octobus.net> parents: 51347 diff changeset	903 #
bbbfa00d1e67 delta-find: remove the "candidate groups" layer Pierre-Yves David <pierre-yves.david@octobus.net> parents: 51347 diff changeset	904 # This might reduce the overall efficiency of the compression
bbbfa00d1e67 delta-find: remove the "candidate groups" layer Pierre-Yves David <pierre-yves.david@octobus.net> parents: 51347 diff changeset	905 # in some corner cases, but that should also prevent very
bbbfa00d1e67 delta-find: remove the "candidate groups" layer Pierre-Yves David <pierre-yves.david@octobus.net> parents: 51347 diff changeset	906 # pathological cases from being an issue. (eg. 20 000
bbbfa00d1e67 delta-find: remove the "candidate groups" layer Pierre-Yves David <pierre-yves.david@octobus.net> parents: 51347 diff changeset	907 # candidates).
bbbfa00d1e67 delta-find: remove the "candidate groups" layer Pierre-Yves David <pierre-yves.david@octobus.net> parents: 51347 diff changeset	908 #
bbbfa00d1e67 delta-find: remove the "candidate groups" layer Pierre-Yves David <pierre-yves.david@octobus.net> parents: 51347 diff changeset	909 # XXX note that the ordering of the group becomes important as
bbbfa00d1e67 delta-find: remove the "candidate groups" layer Pierre-Yves David <pierre-yves.david@octobus.net> parents: 51347 diff changeset	910 # it now impacts the final result. The current order is
bbbfa00d1e67 delta-find: remove the "candidate groups" layer Pierre-Yves David <pierre-yves.david@octobus.net> parents: 51347 diff changeset	911 # unprocessed and can be improved.
bbbfa00d1e67 delta-find: remove the "candidate groups" layer Pierre-Yves David <pierre-yves.david@octobus.net> parents: 51347 diff changeset	912 next_idx = self._internal_idx + self._group_chunk_size
bbbfa00d1e67 delta-find: remove the "candidate groups" layer Pierre-Yves David <pierre-yves.david@octobus.net> parents: 51347 diff changeset	913 self.current_group = self._internal_group[
bbbfa00d1e67 delta-find: remove the "candidate groups" layer Pierre-Yves David <pierre-yves.david@octobus.net> parents: 51347 diff changeset	914 self._internal_idx : next_idx
bbbfa00d1e67 delta-find: remove the "candidate groups" layer Pierre-Yves David <pierre-yves.david@octobus.net> parents: 51347 diff changeset	915 ]
bbbfa00d1e67 delta-find: remove the "candidate groups" layer Pierre-Yves David <pierre-yves.david@octobus.net> parents: 51347 diff changeset	916 self.tested.update(self.current_group)
bbbfa00d1e67 delta-find: remove the "candidate groups" layer Pierre-Yves David <pierre-yves.david@octobus.net> parents: 51347 diff changeset	917 self._internal_idx = next_idx
bbbfa00d1e67 delta-find: remove the "candidate groups" layer Pierre-Yves David <pierre-yves.david@octobus.net> parents: 51347 diff changeset	918 else:
bbbfa00d1e67 delta-find: remove the "candidate groups" layer Pierre-Yves David <pierre-yves.david@octobus.net> parents: 51347 diff changeset	919 self._next_internal_group()
39336 1c6ff52fe9cf revlogdeltas: split candidate groups selection from the filtering logic Boris Feld <boris.feld@octobus.net> parents: 39335 diff changeset	920
51332 02cc19f4f091 delta-find: move pre-filtering of candidates in its own function Pierre-Yves David <pierre-yves.david@octobus.net> parents: 51331 diff changeset	921 def _pre_filter_candidate_revs(self, temptative):
02cc19f4f091 delta-find: move pre-filtering of candidates in its own function Pierre-Yves David <pierre-yves.david@octobus.net> parents: 51331 diff changeset	922 """filter possible candidate before computing a delta
02cc19f4f091 delta-find: move pre-filtering of candidates in its own function Pierre-Yves David <pierre-yves.david@octobus.net> parents: 51331 diff changeset	923
02cc19f4f091 delta-find: move pre-filtering of candidates in its own function Pierre-Yves David <pierre-yves.david@octobus.net> parents: 51331 diff changeset	924 This function use various criteria to pre-filter candidate delta base
02cc19f4f091 delta-find: move pre-filtering of candidates in its own function Pierre-Yves David <pierre-yves.david@octobus.net> parents: 51331 diff changeset	925 before we compute a delta and evaluate its quality.
02cc19f4f091 delta-find: move pre-filtering of candidates in its own function Pierre-Yves David <pierre-yves.david@octobus.net> parents: 51331 diff changeset	926
02cc19f4f091 delta-find: move pre-filtering of candidates in its own function Pierre-Yves David <pierre-yves.david@octobus.net> parents: 51331 diff changeset	927 Such pre-filter limit the number of computed delta, an expensive operation.
02cc19f4f091 delta-find: move pre-filtering of candidates in its own function Pierre-Yves David <pierre-yves.david@octobus.net> parents: 51331 diff changeset	928
02cc19f4f091 delta-find: move pre-filtering of candidates in its own function Pierre-Yves David <pierre-yves.david@octobus.net> parents: 51331 diff changeset	929 return the updated list of revision to test
02cc19f4f091 delta-find: move pre-filtering of candidates in its own function Pierre-Yves David <pierre-yves.david@octobus.net> parents: 51331 diff changeset	930 """
02cc19f4f091 delta-find: move pre-filtering of candidates in its own function Pierre-Yves David <pierre-yves.david@octobus.net> parents: 51331 diff changeset	931 deltalength = self.revlog.length
02cc19f4f091 delta-find: move pre-filtering of candidates in its own function Pierre-Yves David <pierre-yves.david@octobus.net> parents: 51331 diff changeset	932 deltaparent = self.revlog.deltaparent
02cc19f4f091 delta-find: move pre-filtering of candidates in its own function Pierre-Yves David <pierre-yves.david@octobus.net> parents: 51331 diff changeset	933
02cc19f4f091 delta-find: move pre-filtering of candidates in its own function Pierre-Yves David <pierre-yves.david@octobus.net> parents: 51331 diff changeset	934 tested = self.tested
02cc19f4f091 delta-find: move pre-filtering of candidates in its own function Pierre-Yves David <pierre-yves.david@octobus.net> parents: 51331 diff changeset	935 group = []
02cc19f4f091 delta-find: move pre-filtering of candidates in its own function Pierre-Yves David <pierre-yves.david@octobus.net> parents: 51331 diff changeset	936 for rev in temptative:
02cc19f4f091 delta-find: move pre-filtering of candidates in its own function Pierre-Yves David <pierre-yves.david@octobus.net> parents: 51331 diff changeset	937 # skip over empty delta (no need to include them in a chain)
02cc19f4f091 delta-find: move pre-filtering of candidates in its own function Pierre-Yves David <pierre-yves.david@octobus.net> parents: 51331 diff changeset	938 while not (rev == nullrev or rev in tested or deltalength(rev)):
02cc19f4f091 delta-find: move pre-filtering of candidates in its own function Pierre-Yves David <pierre-yves.david@octobus.net> parents: 51331 diff changeset	939 tested.add(rev)
02cc19f4f091 delta-find: move pre-filtering of candidates in its own function Pierre-Yves David <pierre-yves.david@octobus.net> parents: 51331 diff changeset	940 rev = deltaparent(rev)
51333 898c212e1b2f delta-find: move pre-filtering of individual revision in its own function Pierre-Yves David <pierre-yves.david@octobus.net> parents: 51332 diff changeset	941 if self._pre_filter_rev(rev):
898c212e1b2f delta-find: move pre-filtering of individual revision in its own function Pierre-Yves David <pierre-yves.david@octobus.net> parents: 51332 diff changeset	942 group.append(rev)
898c212e1b2f delta-find: move pre-filtering of individual revision in its own function Pierre-Yves David <pierre-yves.david@octobus.net> parents: 51332 diff changeset	943 else:
898c212e1b2f delta-find: move pre-filtering of individual revision in its own function Pierre-Yves David <pierre-yves.david@octobus.net> parents: 51332 diff changeset	944 self.tested.add(rev)
898c212e1b2f delta-find: move pre-filtering of individual revision in its own function Pierre-Yves David <pierre-yves.david@octobus.net> parents: 51332 diff changeset	945 return group
898c212e1b2f delta-find: move pre-filtering of individual revision in its own function Pierre-Yves David <pierre-yves.david@octobus.net> parents: 51332 diff changeset	946
51335 2407af4f2eea delta-find: split the generic part of `_pre_filter_rev` in a method Pierre-Yves David <pierre-yves.david@octobus.net> parents: 51334 diff changeset	947 def _pre_filter_rev_universal(self, rev):
2407af4f2eea delta-find: split the generic part of `_pre_filter_rev` in a method Pierre-Yves David <pierre-yves.david@octobus.net> parents: 51334 diff changeset	948 """pre filtering that is need in all cases.
2407af4f2eea delta-find: split the generic part of `_pre_filter_rev` in a method Pierre-Yves David <pierre-yves.david@octobus.net> parents: 51334 diff changeset	949
2407af4f2eea delta-find: split the generic part of `_pre_filter_rev` in a method Pierre-Yves David <pierre-yves.david@octobus.net> parents: 51334 diff changeset	950 return True if it seems okay to test a rev, False otherwise.
2407af4f2eea delta-find: split the generic part of `_pre_filter_rev` in a method Pierre-Yves David <pierre-yves.david@octobus.net> parents: 51334 diff changeset	951
2407af4f2eea delta-find: split the generic part of `_pre_filter_rev` in a method Pierre-Yves David <pierre-yves.david@octobus.net> parents: 51334 diff changeset	952 used by _pre_filter_rev.
2407af4f2eea delta-find: split the generic part of `_pre_filter_rev` in a method Pierre-Yves David <pierre-yves.david@octobus.net> parents: 51334 diff changeset	953 """
51334 ac8b798e602b delta-find: drop the temporary indent Pierre-Yves David <pierre-yves.david@octobus.net> parents: 51333 diff changeset	954 # no need to try a delta against nullrev, this will be done as
ac8b798e602b delta-find: drop the temporary indent Pierre-Yves David <pierre-yves.david@octobus.net> parents: 51333 diff changeset	955 # a last resort.
ac8b798e602b delta-find: drop the temporary indent Pierre-Yves David <pierre-yves.david@octobus.net> parents: 51333 diff changeset	956 if rev == nullrev:
ac8b798e602b delta-find: drop the temporary indent Pierre-Yves David <pierre-yves.david@octobus.net> parents: 51333 diff changeset	957 return False
ac8b798e602b delta-find: drop the temporary indent Pierre-Yves David <pierre-yves.david@octobus.net> parents: 51333 diff changeset	958 # filter out revision we tested already
ac8b798e602b delta-find: drop the temporary indent Pierre-Yves David <pierre-yves.david@octobus.net> parents: 51333 diff changeset	959 if rev in self.tested:
ac8b798e602b delta-find: drop the temporary indent Pierre-Yves David <pierre-yves.david@octobus.net> parents: 51333 diff changeset	960 return False
51332 02cc19f4f091 delta-find: move pre-filtering of candidates in its own function Pierre-Yves David <pierre-yves.david@octobus.net> parents: 51331 diff changeset	961
51334 ac8b798e602b delta-find: drop the temporary indent Pierre-Yves David <pierre-yves.david@octobus.net> parents: 51333 diff changeset	962 # an higher authority deamed the base unworthy (e.g. censored)
ac8b798e602b delta-find: drop the temporary indent Pierre-Yves David <pierre-yves.david@octobus.net> parents: 51333 diff changeset	963 if self.excluded_bases is not None and rev in self.excluded_bases:
ac8b798e602b delta-find: drop the temporary indent Pierre-Yves David <pierre-yves.david@octobus.net> parents: 51333 diff changeset	964 return False
ac8b798e602b delta-find: drop the temporary indent Pierre-Yves David <pierre-yves.david@octobus.net> parents: 51333 diff changeset	965 # We are in some recomputation cases and that rev is too high
ac8b798e602b delta-find: drop the temporary indent Pierre-Yves David <pierre-yves.david@octobus.net> parents: 51333 diff changeset	966 # in the revlog
ac8b798e602b delta-find: drop the temporary indent Pierre-Yves David <pierre-yves.david@octobus.net> parents: 51333 diff changeset	967 if self.target_rev is not None and rev >= self.target_rev:
ac8b798e602b delta-find: drop the temporary indent Pierre-Yves David <pierre-yves.david@octobus.net> parents: 51333 diff changeset	968 return False
51335 2407af4f2eea delta-find: split the generic part of `_pre_filter_rev` in a method Pierre-Yves David <pierre-yves.david@octobus.net> parents: 51334 diff changeset	969 # no delta for rawtext-changing revs (see "candelta" for why)
2407af4f2eea delta-find: split the generic part of `_pre_filter_rev` in a method Pierre-Yves David <pierre-yves.david@octobus.net> parents: 51334 diff changeset	970 if self.revlog.flags(rev) & REVIDX_RAWTEXT_CHANGING_FLAGS:
2407af4f2eea delta-find: split the generic part of `_pre_filter_rev` in a method Pierre-Yves David <pierre-yves.david@octobus.net> parents: 51334 diff changeset	971 return False
2407af4f2eea delta-find: split the generic part of `_pre_filter_rev` in a method Pierre-Yves David <pierre-yves.david@octobus.net> parents: 51334 diff changeset	972 return True
2407af4f2eea delta-find: split the generic part of `_pre_filter_rev` in a method Pierre-Yves David <pierre-yves.david@octobus.net> parents: 51334 diff changeset	973
51336 9a1239c362ae delta-find: split the "sparse" part of `_pre_filter_rev` in a method Pierre-Yves David <pierre-yves.david@octobus.net> parents: 51335 diff changeset	974 def _pre_filter_rev_sparse(self, rev):
9a1239c362ae delta-find: split the "sparse" part of `_pre_filter_rev` in a method Pierre-Yves David <pierre-yves.david@octobus.net> parents: 51335 diff changeset	975 """pre filtering that is needed in sparse revlog cases
51333 898c212e1b2f delta-find: move pre-filtering of individual revision in its own function Pierre-Yves David <pierre-yves.david@octobus.net> parents: 51332 diff changeset	976
51336 9a1239c362ae delta-find: split the "sparse" part of `_pre_filter_rev` in a method Pierre-Yves David <pierre-yves.david@octobus.net> parents: 51335 diff changeset	977 return True if it seems okay to test a rev, False otherwise.
51332 02cc19f4f091 delta-find: move pre-filtering of candidates in its own function Pierre-Yves David <pierre-yves.david@octobus.net> parents: 51331 diff changeset	978
51336 9a1239c362ae delta-find: split the "sparse" part of `_pre_filter_rev` in a method Pierre-Yves David <pierre-yves.david@octobus.net> parents: 51335 diff changeset	979 used by _pre_filter_rev.
9a1239c362ae delta-find: split the "sparse" part of `_pre_filter_rev` in a method Pierre-Yves David <pierre-yves.david@octobus.net> parents: 51335 diff changeset	980 """
9a1239c362ae delta-find: split the "sparse" part of `_pre_filter_rev` in a method Pierre-Yves David <pierre-yves.david@octobus.net> parents: 51335 diff changeset	981 assert self.revlog.delta_config.sparse_revlog
51334 ac8b798e602b delta-find: drop the temporary indent Pierre-Yves David <pierre-yves.david@octobus.net> parents: 51333 diff changeset	982 # if the revision we test again is too small, the resulting delta
ac8b798e602b delta-find: drop the temporary indent Pierre-Yves David <pierre-yves.david@octobus.net> parents: 51333 diff changeset	983 # will be large anyway as that amount of data to be added is big
51336 9a1239c362ae delta-find: split the "sparse" part of `_pre_filter_rev` in a method Pierre-Yves David <pierre-yves.david@octobus.net> parents: 51335 diff changeset	984 if self.revlog.rawsize(rev) < (self.textlen // LIMIT_BASE2TEXT):
51334 ac8b798e602b delta-find: drop the temporary indent Pierre-Yves David <pierre-yves.david@octobus.net> parents: 51333 diff changeset	985 return False
51332 02cc19f4f091 delta-find: move pre-filtering of candidates in its own function Pierre-Yves David <pierre-yves.david@octobus.net> parents: 51331 diff changeset	986
51336 9a1239c362ae delta-find: split the "sparse" part of `_pre_filter_rev` in a method Pierre-Yves David <pierre-yves.david@octobus.net> parents: 51335 diff changeset	987 if self.revlog.delta_config.upper_bound_comp is not None:
51334 ac8b798e602b delta-find: drop the temporary indent Pierre-Yves David <pierre-yves.david@octobus.net> parents: 51333 diff changeset	988 maxcomp = self.revlog.delta_config.upper_bound_comp
ac8b798e602b delta-find: drop the temporary indent Pierre-Yves David <pierre-yves.david@octobus.net> parents: 51333 diff changeset	989 basenotsnap = (self.p1, self.p2, nullrev)
ac8b798e602b delta-find: drop the temporary indent Pierre-Yves David <pierre-yves.david@octobus.net> parents: 51333 diff changeset	990 if rev not in basenotsnap and self.revlog.issnapshot(rev):
ac8b798e602b delta-find: drop the temporary indent Pierre-Yves David <pierre-yves.david@octobus.net> parents: 51333 diff changeset	991 snapshotdepth = self.revlog.snapshotdepth(rev)
ac8b798e602b delta-find: drop the temporary indent Pierre-Yves David <pierre-yves.david@octobus.net> parents: 51333 diff changeset	992 # If text is significantly larger than the base, we can
ac8b798e602b delta-find: drop the temporary indent Pierre-Yves David <pierre-yves.david@octobus.net> parents: 51333 diff changeset	993 # expect the resulting delta to be proportional to the size
ac8b798e602b delta-find: drop the temporary indent Pierre-Yves David <pierre-yves.david@octobus.net> parents: 51333 diff changeset	994 # difference
ac8b798e602b delta-find: drop the temporary indent Pierre-Yves David <pierre-yves.david@octobus.net> parents: 51333 diff changeset	995 revsize = self.revlog.rawsize(rev)
ac8b798e602b delta-find: drop the temporary indent Pierre-Yves David <pierre-yves.david@octobus.net> parents: 51333 diff changeset	996 rawsizedistance = max(self.textlen - revsize, 0)
ac8b798e602b delta-find: drop the temporary indent Pierre-Yves David <pierre-yves.david@octobus.net> parents: 51333 diff changeset	997 # use an estimate of the compression upper bound.
ac8b798e602b delta-find: drop the temporary indent Pierre-Yves David <pierre-yves.david@octobus.net> parents: 51333 diff changeset	998 lowestrealisticdeltalen = rawsizedistance // maxcomp
51332 02cc19f4f091 delta-find: move pre-filtering of candidates in its own function Pierre-Yves David <pierre-yves.david@octobus.net> parents: 51331 diff changeset	999
51334 ac8b798e602b delta-find: drop the temporary indent Pierre-Yves David <pierre-yves.david@octobus.net> parents: 51333 diff changeset	1000 # check the absolute constraint on the delta size
ac8b798e602b delta-find: drop the temporary indent Pierre-Yves David <pierre-yves.david@octobus.net> parents: 51333 diff changeset	1001 snapshotlimit = self.textlen >> snapshotdepth
ac8b798e602b delta-find: drop the temporary indent Pierre-Yves David <pierre-yves.david@octobus.net> parents: 51333 diff changeset	1002 if snapshotlimit < lowestrealisticdeltalen:
ac8b798e602b delta-find: drop the temporary indent Pierre-Yves David <pierre-yves.david@octobus.net> parents: 51333 diff changeset	1003 # delta lower bound is larger than accepted upper
ac8b798e602b delta-find: drop the temporary indent Pierre-Yves David <pierre-yves.david@octobus.net> parents: 51333 diff changeset	1004 # bound
ac8b798e602b delta-find: drop the temporary indent Pierre-Yves David <pierre-yves.david@octobus.net> parents: 51333 diff changeset	1005 return False
51332 02cc19f4f091 delta-find: move pre-filtering of candidates in its own function Pierre-Yves David <pierre-yves.david@octobus.net> parents: 51331 diff changeset	1006
51334 ac8b798e602b delta-find: drop the temporary indent Pierre-Yves David <pierre-yves.david@octobus.net> parents: 51333 diff changeset	1007 # check the relative constraint on the delta size
ac8b798e602b delta-find: drop the temporary indent Pierre-Yves David <pierre-yves.david@octobus.net> parents: 51333 diff changeset	1008 revlength = self.revlog.length(rev)
ac8b798e602b delta-find: drop the temporary indent Pierre-Yves David <pierre-yves.david@octobus.net> parents: 51333 diff changeset	1009 if revlength < lowestrealisticdeltalen:
ac8b798e602b delta-find: drop the temporary indent Pierre-Yves David <pierre-yves.david@octobus.net> parents: 51333 diff changeset	1010 # delta probable lower bound is larger than target
ac8b798e602b delta-find: drop the temporary indent Pierre-Yves David <pierre-yves.david@octobus.net> parents: 51333 diff changeset	1011 # base
ac8b798e602b delta-find: drop the temporary indent Pierre-Yves David <pierre-yves.david@octobus.net> parents: 51333 diff changeset	1012 return False
51333 898c212e1b2f delta-find: move pre-filtering of individual revision in its own function Pierre-Yves David <pierre-yves.david@octobus.net> parents: 51332 diff changeset	1013 return True
51332 02cc19f4f091 delta-find: move pre-filtering of candidates in its own function Pierre-Yves David <pierre-yves.david@octobus.net> parents: 51331 diff changeset	1014
51337 1ea56b10dd4a delta-find: split the delta-chain part of `_pre_filter_rev` in a method Pierre-Yves David <pierre-yves.david@octobus.net> parents: 51336 diff changeset	1015 def _pre_filter_rev_delta_chain(self, rev):
1ea56b10dd4a delta-find: split the delta-chain part of `_pre_filter_rev` in a method Pierre-Yves David <pierre-yves.david@octobus.net> parents: 51336 diff changeset	1016 """pre filtering that is needed in sparse revlog cases
51336 9a1239c362ae delta-find: split the "sparse" part of `_pre_filter_rev` in a method Pierre-Yves David <pierre-yves.david@octobus.net> parents: 51335 diff changeset	1017
51337 1ea56b10dd4a delta-find: split the delta-chain part of `_pre_filter_rev` in a method Pierre-Yves David <pierre-yves.david@octobus.net> parents: 51336 diff changeset	1018 return True if it seems okay to test a rev, False otherwise.
1ea56b10dd4a delta-find: split the delta-chain part of `_pre_filter_rev` in a method Pierre-Yves David <pierre-yves.david@octobus.net> parents: 51336 diff changeset	1019
1ea56b10dd4a delta-find: split the delta-chain part of `_pre_filter_rev` in a method Pierre-Yves David <pierre-yves.david@octobus.net> parents: 51336 diff changeset	1020 used by _pre_filter_rev.
1ea56b10dd4a delta-find: split the delta-chain part of `_pre_filter_rev` in a method Pierre-Yves David <pierre-yves.david@octobus.net> parents: 51336 diff changeset	1021 """
51336 9a1239c362ae delta-find: split the "sparse" part of `_pre_filter_rev` in a method Pierre-Yves David <pierre-yves.david@octobus.net> parents: 51335 diff changeset	1022 deltas_limit = self.revinfo.textlen * LIMIT_DELTA2TEXT
9a1239c362ae delta-find: split the "sparse" part of `_pre_filter_rev` in a method Pierre-Yves David <pierre-yves.david@octobus.net> parents: 51335 diff changeset	1023 # filter out delta base that will never produce good delta
9a1239c362ae delta-find: split the "sparse" part of `_pre_filter_rev` in a method Pierre-Yves David <pierre-yves.david@octobus.net> parents: 51335 diff changeset	1024 #
9a1239c362ae delta-find: split the "sparse" part of `_pre_filter_rev` in a method Pierre-Yves David <pierre-yves.david@octobus.net> parents: 51335 diff changeset	1025 # if the delta of that base is already bigger than the limit
9a1239c362ae delta-find: split the "sparse" part of `_pre_filter_rev` in a method Pierre-Yves David <pierre-yves.david@octobus.net> parents: 51335 diff changeset	1026 # for the delta chain size, doing a delta is hopeless.
9a1239c362ae delta-find: split the "sparse" part of `_pre_filter_rev` in a method Pierre-Yves David <pierre-yves.david@octobus.net> parents: 51335 diff changeset	1027 if deltas_limit < self.revlog.length(rev):
9a1239c362ae delta-find: split the "sparse" part of `_pre_filter_rev` in a method Pierre-Yves David <pierre-yves.david@octobus.net> parents: 51335 diff changeset	1028 return False
9a1239c362ae delta-find: split the "sparse" part of `_pre_filter_rev` in a method Pierre-Yves David <pierre-yves.david@octobus.net> parents: 51335 diff changeset	1029
9a1239c362ae delta-find: split the "sparse" part of `_pre_filter_rev` in a method Pierre-Yves David <pierre-yves.david@octobus.net> parents: 51335 diff changeset	1030 # If we reach here, we are about to build and test a delta.
9a1239c362ae delta-find: split the "sparse" part of `_pre_filter_rev` in a method Pierre-Yves David <pierre-yves.david@octobus.net> parents: 51335 diff changeset	1031 # The delta building process will compute the chaininfo in all
9a1239c362ae delta-find: split the "sparse" part of `_pre_filter_rev` in a method Pierre-Yves David <pierre-yves.david@octobus.net> parents: 51335 diff changeset	1032 # case, since that computation is cached, it is fine to access
9a1239c362ae delta-find: split the "sparse" part of `_pre_filter_rev` in a method Pierre-Yves David <pierre-yves.david@octobus.net> parents: 51335 diff changeset	1033 # it here too.
9a1239c362ae delta-find: split the "sparse" part of `_pre_filter_rev` in a method Pierre-Yves David <pierre-yves.david@octobus.net> parents: 51335 diff changeset	1034 chainlen, chainsize = self.revlog._chaininfo(rev)
9a1239c362ae delta-find: split the "sparse" part of `_pre_filter_rev` in a method Pierre-Yves David <pierre-yves.david@octobus.net> parents: 51335 diff changeset	1035 # if chain will be too long, skip base
9a1239c362ae delta-find: split the "sparse" part of `_pre_filter_rev` in a method Pierre-Yves David <pierre-yves.david@octobus.net> parents: 51335 diff changeset	1036 if (
9a1239c362ae delta-find: split the "sparse" part of `_pre_filter_rev` in a method Pierre-Yves David <pierre-yves.david@octobus.net> parents: 51335 diff changeset	1037 self.revlog.delta_config.max_chain_len
9a1239c362ae delta-find: split the "sparse" part of `_pre_filter_rev` in a method Pierre-Yves David <pierre-yves.david@octobus.net> parents: 51335 diff changeset	1038 and chainlen >= self.revlog.delta_config.max_chain_len
9a1239c362ae delta-find: split the "sparse" part of `_pre_filter_rev` in a method Pierre-Yves David <pierre-yves.david@octobus.net> parents: 51335 diff changeset	1039 ):
9a1239c362ae delta-find: split the "sparse" part of `_pre_filter_rev` in a method Pierre-Yves David <pierre-yves.david@octobus.net> parents: 51335 diff changeset	1040 return False
9a1239c362ae delta-find: split the "sparse" part of `_pre_filter_rev` in a method Pierre-Yves David <pierre-yves.david@octobus.net> parents: 51335 diff changeset	1041 # if chain already have too much data, skip base
9a1239c362ae delta-find: split the "sparse" part of `_pre_filter_rev` in a method Pierre-Yves David <pierre-yves.david@octobus.net> parents: 51335 diff changeset	1042 if deltas_limit < chainsize:
9a1239c362ae delta-find: split the "sparse" part of `_pre_filter_rev` in a method Pierre-Yves David <pierre-yves.david@octobus.net> parents: 51335 diff changeset	1043 return False
51337 1ea56b10dd4a delta-find: split the delta-chain part of `_pre_filter_rev` in a method Pierre-Yves David <pierre-yves.david@octobus.net> parents: 51336 diff changeset	1044 return True
51336 9a1239c362ae delta-find: split the "sparse" part of `_pre_filter_rev` in a method Pierre-Yves David <pierre-yves.david@octobus.net> parents: 51335 diff changeset	1045
51337 1ea56b10dd4a delta-find: split the delta-chain part of `_pre_filter_rev` in a method Pierre-Yves David <pierre-yves.david@octobus.net> parents: 51336 diff changeset	1046 def _pre_filter_rev(self, rev):
1ea56b10dd4a delta-find: split the delta-chain part of `_pre_filter_rev` in a method Pierre-Yves David <pierre-yves.david@octobus.net> parents: 51336 diff changeset	1047 """return True if it seems okay to test a rev, False otherwise"""
1ea56b10dd4a delta-find: split the delta-chain part of `_pre_filter_rev` in a method Pierre-Yves David <pierre-yves.david@octobus.net> parents: 51336 diff changeset	1048 if not self._pre_filter_rev_universal(rev):
1ea56b10dd4a delta-find: split the delta-chain part of `_pre_filter_rev` in a method Pierre-Yves David <pierre-yves.david@octobus.net> parents: 51336 diff changeset	1049 return False
1ea56b10dd4a delta-find: split the delta-chain part of `_pre_filter_rev` in a method Pierre-Yves David <pierre-yves.david@octobus.net> parents: 51336 diff changeset	1050 if not self._pre_filter_rev_delta_chain(rev):
1ea56b10dd4a delta-find: split the delta-chain part of `_pre_filter_rev` in a method Pierre-Yves David <pierre-yves.david@octobus.net> parents: 51336 diff changeset	1051 return False
51336 9a1239c362ae delta-find: split the "sparse" part of `_pre_filter_rev` in a method Pierre-Yves David <pierre-yves.david@octobus.net> parents: 51335 diff changeset	1052 if self.revlog.delta_config.sparse_revlog:
9a1239c362ae delta-find: split the "sparse" part of `_pre_filter_rev` in a method Pierre-Yves David <pierre-yves.david@octobus.net> parents: 51335 diff changeset	1053 if not self._pre_filter_rev_sparse(rev):
9a1239c362ae delta-find: split the "sparse" part of `_pre_filter_rev` in a method Pierre-Yves David <pierre-yves.david@octobus.net> parents: 51335 diff changeset	1054 return False
9a1239c362ae delta-find: split the "sparse" part of `_pre_filter_rev` in a method Pierre-Yves David <pierre-yves.david@octobus.net> parents: 51335 diff changeset	1055
9a1239c362ae delta-find: split the "sparse" part of `_pre_filter_rev` in a method Pierre-Yves David <pierre-yves.david@octobus.net> parents: 51335 diff changeset	1056 return True
9a1239c362ae delta-find: split the "sparse" part of `_pre_filter_rev` in a method Pierre-Yves David <pierre-yves.david@octobus.net> parents: 51335 diff changeset	1057
51350 5cc04a6da19d delta-find: move the emotion of parents in a dedicated method Pierre-Yves David <pierre-yves.david@octobus.net> parents: 51349 diff changeset	1058 def _iter_parents(self):
5cc04a6da19d delta-find: move the emotion of parents in a dedicated method Pierre-Yves David <pierre-yves.david@octobus.net> parents: 51349 diff changeset	1059 # exclude already lazy tested base if any
5cc04a6da19d delta-find: move the emotion of parents in a dedicated method Pierre-Yves David <pierre-yves.david@octobus.net> parents: 51349 diff changeset	1060 parents = [p for p in (self.p1, self.p2) if p != nullrev]
5cc04a6da19d delta-find: move the emotion of parents in a dedicated method Pierre-Yves David <pierre-yves.david@octobus.net> parents: 51349 diff changeset	1061
5cc04a6da19d delta-find: move the emotion of parents in a dedicated method Pierre-Yves David <pierre-yves.david@octobus.net> parents: 51349 diff changeset	1062 self.current_stage = _STAGE_PARENTS
5cc04a6da19d delta-find: move the emotion of parents in a dedicated method Pierre-Yves David <pierre-yves.david@octobus.net> parents: 51349 diff changeset	1063 if (
5cc04a6da19d delta-find: move the emotion of parents in a dedicated method Pierre-Yves David <pierre-yves.david@octobus.net> parents: 51349 diff changeset	1064 not self.revlog.delta_config.delta_both_parents
5cc04a6da19d delta-find: move the emotion of parents in a dedicated method Pierre-Yves David <pierre-yves.david@octobus.net> parents: 51349 diff changeset	1065 and len(parents) == 2
5cc04a6da19d delta-find: move the emotion of parents in a dedicated method Pierre-Yves David <pierre-yves.david@octobus.net> parents: 51349 diff changeset	1066 ):
5cc04a6da19d delta-find: move the emotion of parents in a dedicated method Pierre-Yves David <pierre-yves.david@octobus.net> parents: 51349 diff changeset	1067 parents.sort()
5cc04a6da19d delta-find: move the emotion of parents in a dedicated method Pierre-Yves David <pierre-yves.david@octobus.net> parents: 51349 diff changeset	1068 # To minimize the chance of having to build a fulltext,
5cc04a6da19d delta-find: move the emotion of parents in a dedicated method Pierre-Yves David <pierre-yves.david@octobus.net> parents: 51349 diff changeset	1069 # pick first whichever parent is closest to us (max rev)
5cc04a6da19d delta-find: move the emotion of parents in a dedicated method Pierre-Yves David <pierre-yves.david@octobus.net> parents: 51349 diff changeset	1070 yield (parents[1],)
5cc04a6da19d delta-find: move the emotion of parents in a dedicated method Pierre-Yves David <pierre-yves.david@octobus.net> parents: 51349 diff changeset	1071 # then the other one (min rev) if the first did not fit
5cc04a6da19d delta-find: move the emotion of parents in a dedicated method Pierre-Yves David <pierre-yves.david@octobus.net> parents: 51349 diff changeset	1072 yield (parents[0],)
5cc04a6da19d delta-find: move the emotion of parents in a dedicated method Pierre-Yves David <pierre-yves.david@octobus.net> parents: 51349 diff changeset	1073 elif len(parents) > 0:
5cc04a6da19d delta-find: move the emotion of parents in a dedicated method Pierre-Yves David <pierre-yves.david@octobus.net> parents: 51349 diff changeset	1074 # Test all parents (1 or 2), and keep the best candidate
5cc04a6da19d delta-find: move the emotion of parents in a dedicated method Pierre-Yves David <pierre-yves.david@octobus.net> parents: 51349 diff changeset	1075 yield parents
5cc04a6da19d delta-find: move the emotion of parents in a dedicated method Pierre-Yves David <pierre-yves.david@octobus.net> parents: 51349 diff changeset	1076
51351 94fe4474b053 delta-find: move the emotion of prev in a dedicated method Pierre-Yves David <pierre-yves.david@octobus.net> parents: 51350 diff changeset	1077 def _iter_prev(self):
94fe4474b053 delta-find: move the emotion of prev in a dedicated method Pierre-Yves David <pierre-yves.david@octobus.net> parents: 51350 diff changeset	1078 # other approach failed try against prev to hopefully save us a
94fe4474b053 delta-find: move the emotion of prev in a dedicated method Pierre-Yves David <pierre-yves.david@octobus.net> parents: 51350 diff changeset	1079 # fulltext.
94fe4474b053 delta-find: move the emotion of prev in a dedicated method Pierre-Yves David <pierre-yves.david@octobus.net> parents: 51350 diff changeset	1080 self.current_stage = _STAGE_PREV
94fe4474b053 delta-find: move the emotion of prev in a dedicated method Pierre-Yves David <pierre-yves.david@octobus.net> parents: 51350 diff changeset	1081 yield (self.target_rev - 1,)
94fe4474b053 delta-find: move the emotion of prev in a dedicated method Pierre-Yves David <pierre-yves.david@octobus.net> parents: 51350 diff changeset	1082
51352 fac6038b11f5 delta-find: move the base of the delta search in its own function Pierre-Yves David <pierre-yves.david@octobus.net> parents: 51351 diff changeset	1083 def _iter_snapshots_base(self):
fac6038b11f5 delta-find: move the base of the delta search in its own function Pierre-Yves David <pierre-yves.david@octobus.net> parents: 51351 diff changeset	1084 assert self.revlog.delta_config.sparse_revlog
fac6038b11f5 delta-find: move the base of the delta search in its own function Pierre-Yves David <pierre-yves.david@octobus.net> parents: 51351 diff changeset	1085 prev = self.target_rev - 1
fac6038b11f5 delta-find: move the base of the delta search in its own function Pierre-Yves David <pierre-yves.david@octobus.net> parents: 51351 diff changeset	1086 deltachain = lambda rev: self.revlog._deltachain(rev)[0]
fac6038b11f5 delta-find: move the base of the delta search in its own function Pierre-Yves David <pierre-yves.david@octobus.net> parents: 51351 diff changeset	1087
fac6038b11f5 delta-find: move the base of the delta search in its own function Pierre-Yves David <pierre-yves.david@octobus.net> parents: 51351 diff changeset	1088 parents = [p for p in (self.p1, self.p2) if p != nullrev]
fac6038b11f5 delta-find: move the base of the delta search in its own function Pierre-Yves David <pierre-yves.david@octobus.net> parents: 51351 diff changeset	1089 if not parents:
fac6038b11f5 delta-find: move the base of the delta search in its own function Pierre-Yves David <pierre-yves.david@octobus.net> parents: 51351 diff changeset	1090 return
fac6038b11f5 delta-find: move the base of the delta search in its own function Pierre-Yves David <pierre-yves.david@octobus.net> parents: 51351 diff changeset	1091 self.current_stage = _STAGE_SNAPSHOT
fac6038b11f5 delta-find: move the base of the delta search in its own function Pierre-Yves David <pierre-yves.david@octobus.net> parents: 51351 diff changeset	1092 # See if we can use an existing snapshot in the parent chains to
fac6038b11f5 delta-find: move the base of the delta search in its own function Pierre-Yves David <pierre-yves.david@octobus.net> parents: 51351 diff changeset	1093 # use as a base for a new intermediate-snapshot
fac6038b11f5 delta-find: move the base of the delta search in its own function Pierre-Yves David <pierre-yves.david@octobus.net> parents: 51351 diff changeset	1094 #
fac6038b11f5 delta-find: move the base of the delta search in its own function Pierre-Yves David <pierre-yves.david@octobus.net> parents: 51351 diff changeset	1095 # search for snapshot in parents delta chain map: snapshot-level:
fac6038b11f5 delta-find: move the base of the delta search in its own function Pierre-Yves David <pierre-yves.david@octobus.net> parents: 51351 diff changeset	1096 # snapshot-rev
fac6038b11f5 delta-find: move the base of the delta search in its own function Pierre-Yves David <pierre-yves.david@octobus.net> parents: 51351 diff changeset	1097 parents_snaps = collections.defaultdict(set)
fac6038b11f5 delta-find: move the base of the delta search in its own function Pierre-Yves David <pierre-yves.david@octobus.net> parents: 51351 diff changeset	1098 candidate_chains = [deltachain(p) for p in parents]
fac6038b11f5 delta-find: move the base of the delta search in its own function Pierre-Yves David <pierre-yves.david@octobus.net> parents: 51351 diff changeset	1099 for chain in candidate_chains:
fac6038b11f5 delta-find: move the base of the delta search in its own function Pierre-Yves David <pierre-yves.david@octobus.net> parents: 51351 diff changeset	1100 for idx, s in enumerate(chain):
fac6038b11f5 delta-find: move the base of the delta search in its own function Pierre-Yves David <pierre-yves.david@octobus.net> parents: 51351 diff changeset	1101 if not self.revlog.issnapshot(s):
fac6038b11f5 delta-find: move the base of the delta search in its own function Pierre-Yves David <pierre-yves.david@octobus.net> parents: 51351 diff changeset	1102 break
fac6038b11f5 delta-find: move the base of the delta search in its own function Pierre-Yves David <pierre-yves.david@octobus.net> parents: 51351 diff changeset	1103 parents_snaps[idx].add(s)
fac6038b11f5 delta-find: move the base of the delta search in its own function Pierre-Yves David <pierre-yves.david@octobus.net> parents: 51351 diff changeset	1104 snapfloor = min(parents_snaps[0]) + 1
fac6038b11f5 delta-find: move the base of the delta search in its own function Pierre-Yves David <pierre-yves.david@octobus.net> parents: 51351 diff changeset	1105 self.snapshot_cache.update(self.revlog, snapfloor)
fac6038b11f5 delta-find: move the base of the delta search in its own function Pierre-Yves David <pierre-yves.david@octobus.net> parents: 51351 diff changeset	1106 # search for the highest "unrelated" revision
fac6038b11f5 delta-find: move the base of the delta search in its own function Pierre-Yves David <pierre-yves.david@octobus.net> parents: 51351 diff changeset	1107 #
fac6038b11f5 delta-find: move the base of the delta search in its own function Pierre-Yves David <pierre-yves.david@octobus.net> parents: 51351 diff changeset	1108 # Adding snapshots used by "unrelated" revision increase the odd we
fac6038b11f5 delta-find: move the base of the delta search in its own function Pierre-Yves David <pierre-yves.david@octobus.net> parents: 51351 diff changeset	1109 # reuse an independant, yet better snapshot chain.
fac6038b11f5 delta-find: move the base of the delta search in its own function Pierre-Yves David <pierre-yves.david@octobus.net> parents: 51351 diff changeset	1110 #
fac6038b11f5 delta-find: move the base of the delta search in its own function Pierre-Yves David <pierre-yves.david@octobus.net> parents: 51351 diff changeset	1111 # XXX instead of building a set of revisions, we could lazily
fac6038b11f5 delta-find: move the base of the delta search in its own function Pierre-Yves David <pierre-yves.david@octobus.net> parents: 51351 diff changeset	1112 # enumerate over the chains. That would be more efficient, however
fac6038b11f5 delta-find: move the base of the delta search in its own function Pierre-Yves David <pierre-yves.david@octobus.net> parents: 51351 diff changeset	1113 # we stick to simple code for now.
fac6038b11f5 delta-find: move the base of the delta search in its own function Pierre-Yves David <pierre-yves.david@octobus.net> parents: 51351 diff changeset	1114 all_revs = set()
fac6038b11f5 delta-find: move the base of the delta search in its own function Pierre-Yves David <pierre-yves.david@octobus.net> parents: 51351 diff changeset	1115 for chain in candidate_chains:
fac6038b11f5 delta-find: move the base of the delta search in its own function Pierre-Yves David <pierre-yves.david@octobus.net> parents: 51351 diff changeset	1116 all_revs.update(chain)
fac6038b11f5 delta-find: move the base of the delta search in its own function Pierre-Yves David <pierre-yves.david@octobus.net> parents: 51351 diff changeset	1117 other = None
fac6038b11f5 delta-find: move the base of the delta search in its own function Pierre-Yves David <pierre-yves.david@octobus.net> parents: 51351 diff changeset	1118 for r in self.revlog.revs(prev, snapfloor):
fac6038b11f5 delta-find: move the base of the delta search in its own function Pierre-Yves David <pierre-yves.david@octobus.net> parents: 51351 diff changeset	1119 if r not in all_revs:
fac6038b11f5 delta-find: move the base of the delta search in its own function Pierre-Yves David <pierre-yves.david@octobus.net> parents: 51351 diff changeset	1120 other = r
fac6038b11f5 delta-find: move the base of the delta search in its own function Pierre-Yves David <pierre-yves.david@octobus.net> parents: 51351 diff changeset	1121 break
fac6038b11f5 delta-find: move the base of the delta search in its own function Pierre-Yves David <pierre-yves.david@octobus.net> parents: 51351 diff changeset	1122 if other is not None:
fac6038b11f5 delta-find: move the base of the delta search in its own function Pierre-Yves David <pierre-yves.david@octobus.net> parents: 51351 diff changeset	1123 # To avoid unfair competition, we won't use unrelated
fac6038b11f5 delta-find: move the base of the delta search in its own function Pierre-Yves David <pierre-yves.david@octobus.net> parents: 51351 diff changeset	1124 # intermediate snapshot that are deeper than the ones from the
fac6038b11f5 delta-find: move the base of the delta search in its own function Pierre-Yves David <pierre-yves.david@octobus.net> parents: 51351 diff changeset	1125 # parent delta chain.
fac6038b11f5 delta-find: move the base of the delta search in its own function Pierre-Yves David <pierre-yves.david@octobus.net> parents: 51351 diff changeset	1126 max_depth = max(parents_snaps.keys())
fac6038b11f5 delta-find: move the base of the delta search in its own function Pierre-Yves David <pierre-yves.david@octobus.net> parents: 51351 diff changeset	1127 chain = deltachain(other)
fac6038b11f5 delta-find: move the base of the delta search in its own function Pierre-Yves David <pierre-yves.david@octobus.net> parents: 51351 diff changeset	1128 for depth, s in enumerate(chain):
fac6038b11f5 delta-find: move the base of the delta search in its own function Pierre-Yves David <pierre-yves.david@octobus.net> parents: 51351 diff changeset	1129 if s < snapfloor:
fac6038b11f5 delta-find: move the base of the delta search in its own function Pierre-Yves David <pierre-yves.david@octobus.net> parents: 51351 diff changeset	1130 continue
fac6038b11f5 delta-find: move the base of the delta search in its own function Pierre-Yves David <pierre-yves.david@octobus.net> parents: 51351 diff changeset	1131 if max_depth < depth:
fac6038b11f5 delta-find: move the base of the delta search in its own function Pierre-Yves David <pierre-yves.david@octobus.net> parents: 51351 diff changeset	1132 break
fac6038b11f5 delta-find: move the base of the delta search in its own function Pierre-Yves David <pierre-yves.david@octobus.net> parents: 51351 diff changeset	1133 if not self.revlog.issnapshot(s):
fac6038b11f5 delta-find: move the base of the delta search in its own function Pierre-Yves David <pierre-yves.david@octobus.net> parents: 51351 diff changeset	1134 break
fac6038b11f5 delta-find: move the base of the delta search in its own function Pierre-Yves David <pierre-yves.david@octobus.net> parents: 51351 diff changeset	1135 parents_snaps[depth].add(s)
fac6038b11f5 delta-find: move the base of the delta search in its own function Pierre-Yves David <pierre-yves.david@octobus.net> parents: 51351 diff changeset	1136 # Test them as possible intermediate snapshot base We test them
fac6038b11f5 delta-find: move the base of the delta search in its own function Pierre-Yves David <pierre-yves.david@octobus.net> parents: 51351 diff changeset	1137 # from highest to lowest level. High level one are more likely to
fac6038b11f5 delta-find: move the base of the delta search in its own function Pierre-Yves David <pierre-yves.david@octobus.net> parents: 51351 diff changeset	1138 # result in small delta
fac6038b11f5 delta-find: move the base of the delta search in its own function Pierre-Yves David <pierre-yves.david@octobus.net> parents: 51351 diff changeset	1139 floor = None
fac6038b11f5 delta-find: move the base of the delta search in its own function Pierre-Yves David <pierre-yves.david@octobus.net> parents: 51351 diff changeset	1140 for idx, snaps in sorted(parents_snaps.items(), reverse=True):
fac6038b11f5 delta-find: move the base of the delta search in its own function Pierre-Yves David <pierre-yves.david@octobus.net> parents: 51351 diff changeset	1141 siblings = set()
fac6038b11f5 delta-find: move the base of the delta search in its own function Pierre-Yves David <pierre-yves.david@octobus.net> parents: 51351 diff changeset	1142 for s in snaps:
fac6038b11f5 delta-find: move the base of the delta search in its own function Pierre-Yves David <pierre-yves.david@octobus.net> parents: 51351 diff changeset	1143 siblings.update(self.snapshot_cache.snapshots[s])
fac6038b11f5 delta-find: move the base of the delta search in its own function Pierre-Yves David <pierre-yves.david@octobus.net> parents: 51351 diff changeset	1144 # Before considering making a new intermediate snapshot, we
fac6038b11f5 delta-find: move the base of the delta search in its own function Pierre-Yves David <pierre-yves.david@octobus.net> parents: 51351 diff changeset	1145 # check if an existing snapshot, children of base we consider,
fac6038b11f5 delta-find: move the base of the delta search in its own function Pierre-Yves David <pierre-yves.david@octobus.net> parents: 51351 diff changeset	1146 # would be suitable.
fac6038b11f5 delta-find: move the base of the delta search in its own function Pierre-Yves David <pierre-yves.david@octobus.net> parents: 51351 diff changeset	1147 #
fac6038b11f5 delta-find: move the base of the delta search in its own function Pierre-Yves David <pierre-yves.david@octobus.net> parents: 51351 diff changeset	1148 # It give a change to reuse a delta chain "unrelated" to the
fac6038b11f5 delta-find: move the base of the delta search in its own function Pierre-Yves David <pierre-yves.david@octobus.net> parents: 51351 diff changeset	1149 # current revision instead of starting our own. Without such
fac6038b11f5 delta-find: move the base of the delta search in its own function Pierre-Yves David <pierre-yves.david@octobus.net> parents: 51351 diff changeset	1150 # re-use, topological branches would keep reopening new chains.
fac6038b11f5 delta-find: move the base of the delta search in its own function Pierre-Yves David <pierre-yves.david@octobus.net> parents: 51351 diff changeset	1151 # Creating more and more snapshot as the repository grow.
fac6038b11f5 delta-find: move the base of the delta search in its own function Pierre-Yves David <pierre-yves.david@octobus.net> parents: 51351 diff changeset	1152
fac6038b11f5 delta-find: move the base of the delta search in its own function Pierre-Yves David <pierre-yves.david@octobus.net> parents: 51351 diff changeset	1153 if floor is not None:
fac6038b11f5 delta-find: move the base of the delta search in its own function Pierre-Yves David <pierre-yves.david@octobus.net> parents: 51351 diff changeset	1154 # We only do this for siblings created after the one in our
fac6038b11f5 delta-find: move the base of the delta search in its own function Pierre-Yves David <pierre-yves.david@octobus.net> parents: 51351 diff changeset	1155 # parent's delta chain. Those created before has less
fac6038b11f5 delta-find: move the base of the delta search in its own function Pierre-Yves David <pierre-yves.david@octobus.net> parents: 51351 diff changeset	1156 # chances to be valid base since our ancestors had to
fac6038b11f5 delta-find: move the base of the delta search in its own function Pierre-Yves David <pierre-yves.david@octobus.net> parents: 51351 diff changeset	1157 # create a new snapshot.
fac6038b11f5 delta-find: move the base of the delta search in its own function Pierre-Yves David <pierre-yves.david@octobus.net> parents: 51351 diff changeset	1158 siblings = [r for r in siblings if floor < r]
fac6038b11f5 delta-find: move the base of the delta search in its own function Pierre-Yves David <pierre-yves.david@octobus.net> parents: 51351 diff changeset	1159 yield tuple(sorted(siblings))
fac6038b11f5 delta-find: move the base of the delta search in its own function Pierre-Yves David <pierre-yves.david@octobus.net> parents: 51351 diff changeset	1160 # then test the base from our parent's delta chain.
fac6038b11f5 delta-find: move the base of the delta search in its own function Pierre-Yves David <pierre-yves.david@octobus.net> parents: 51351 diff changeset	1161 yield tuple(sorted(snaps))
fac6038b11f5 delta-find: move the base of the delta search in its own function Pierre-Yves David <pierre-yves.david@octobus.net> parents: 51351 diff changeset	1162 floor = min(snaps)
fac6038b11f5 delta-find: move the base of the delta search in its own function Pierre-Yves David <pierre-yves.david@octobus.net> parents: 51351 diff changeset	1163 # No suitable base found in the parent chain, search if any full
fac6038b11f5 delta-find: move the base of the delta search in its own function Pierre-Yves David <pierre-yves.david@octobus.net> parents: 51351 diff changeset	1164 # snapshots emitted since parent's base would be a suitable base
fac6038b11f5 delta-find: move the base of the delta search in its own function Pierre-Yves David <pierre-yves.david@octobus.net> parents: 51351 diff changeset	1165 # for an intermediate snapshot.
fac6038b11f5 delta-find: move the base of the delta search in its own function Pierre-Yves David <pierre-yves.david@octobus.net> parents: 51351 diff changeset	1166 #
fac6038b11f5 delta-find: move the base of the delta search in its own function Pierre-Yves David <pierre-yves.david@octobus.net> parents: 51351 diff changeset	1167 # It give a chance to reuse a delta chain unrelated to the current
fac6038b11f5 delta-find: move the base of the delta search in its own function Pierre-Yves David <pierre-yves.david@octobus.net> parents: 51351 diff changeset	1168 # revisions instead of starting our own. Without such re-use,
fac6038b11f5 delta-find: move the base of the delta search in its own function Pierre-Yves David <pierre-yves.david@octobus.net> parents: 51351 diff changeset	1169 # topological branches would keep reopening new full chains.
fac6038b11f5 delta-find: move the base of the delta search in its own function Pierre-Yves David <pierre-yves.david@octobus.net> parents: 51351 diff changeset	1170 # Creating more and more snapshot as the repository grow.
fac6038b11f5 delta-find: move the base of the delta search in its own function Pierre-Yves David <pierre-yves.david@octobus.net> parents: 51351 diff changeset	1171 full = [
fac6038b11f5 delta-find: move the base of the delta search in its own function Pierre-Yves David <pierre-yves.david@octobus.net> parents: 51351 diff changeset	1172 r for r in self.snapshot_cache.snapshots[nullrev] if snapfloor <= r
fac6038b11f5 delta-find: move the base of the delta search in its own function Pierre-Yves David <pierre-yves.david@octobus.net> parents: 51351 diff changeset	1173 ]
fac6038b11f5 delta-find: move the base of the delta search in its own function Pierre-Yves David <pierre-yves.david@octobus.net> parents: 51351 diff changeset	1174 yield tuple(sorted(full))
fac6038b11f5 delta-find: move the base of the delta search in its own function Pierre-Yves David <pierre-yves.david@octobus.net> parents: 51351 diff changeset	1175
51318 c83074405276 delta-find: move `_refinedgroups` on the `_DeltaSearch` object Pierre-Yves David <pierre-yves.david@octobus.net> parents: 51317 diff changeset	1176 def _refined_groups(self):
c83074405276 delta-find: move `_refinedgroups` on the `_DeltaSearch` object Pierre-Yves David <pierre-yves.david@octobus.net> parents: 51317 diff changeset	1177 good = None
51319 a227e061bb4b delta-find: move `_rawgroups` on the `_DeltaSearch` object Pierre-Yves David <pierre-yves.david@octobus.net> parents: 51318 diff changeset	1178 groups = self._raw_groups()
51318 c83074405276 delta-find: move `_refinedgroups` on the `_DeltaSearch` object Pierre-Yves David <pierre-yves.david@octobus.net> parents: 51317 diff changeset	1179 for candidates in groups:
c83074405276 delta-find: move `_refinedgroups` on the `_DeltaSearch` object Pierre-Yves David <pierre-yves.david@octobus.net> parents: 51317 diff changeset	1180 good = yield candidates
c83074405276 delta-find: move `_refinedgroups` on the `_DeltaSearch` object Pierre-Yves David <pierre-yves.david@octobus.net> parents: 51317 diff changeset	1181 if good is not None:
c83074405276 delta-find: move `_refinedgroups` on the `_DeltaSearch` object Pierre-Yves David <pierre-yves.david@octobus.net> parents: 51317 diff changeset	1182 break
43076 2372284d9457 formatting: blacken the codebase Augie Fackler <augie@google.com> parents: 42992 diff changeset	1183
51318 c83074405276 delta-find: move `_refinedgroups` on the `_DeltaSearch` object Pierre-Yves David <pierre-yves.david@octobus.net> parents: 51317 diff changeset	1184 # If sparse revlog is enabled, we can try to refine the available
c83074405276 delta-find: move `_refinedgroups` on the `_DeltaSearch` object Pierre-Yves David <pierre-yves.david@octobus.net> parents: 51317 diff changeset	1185 # deltas
c83074405276 delta-find: move `_refinedgroups` on the `_DeltaSearch` object Pierre-Yves David <pierre-yves.david@octobus.net> parents: 51317 diff changeset	1186 if not self.revlog.delta_config.sparse_revlog:
51346 cc806f20d756 delta-find: explicitly track stage of the search Pierre-Yves David <pierre-yves.david@octobus.net> parents: 51345 diff changeset	1187 self.current_stage = _STAGE_FULL
39501 993d7e2c8b79 snapshot: make sure we'll never refine delta base from a reused source Boris Feld <boris.feld@octobus.net> parents: 39500 diff changeset	1188 yield None
993d7e2c8b79 snapshot: make sure we'll never refine delta base from a reused source Boris Feld <boris.feld@octobus.net> parents: 39500 diff changeset	1189 return
40428 bafa1c4bb7a8 sparse-revlog: only refine delta candidates in the sparse case (issue6006) Boris Feld <boris.feld@octobus.net> parents: 39777 diff changeset	1190
51318 c83074405276 delta-find: move `_refinedgroups` on the `_DeltaSearch` object Pierre-Yves David <pierre-yves.david@octobus.net> parents: 51317 diff changeset	1191 # if we have a refinable value, try to refine it
c83074405276 delta-find: move `_refinedgroups` on the `_DeltaSearch` object Pierre-Yves David <pierre-yves.david@octobus.net> parents: 51317 diff changeset	1192 if (
c83074405276 delta-find: move `_refinedgroups` on the `_DeltaSearch` object Pierre-Yves David <pierre-yves.david@octobus.net> parents: 51317 diff changeset	1193 good is not None
c83074405276 delta-find: move `_refinedgroups` on the `_DeltaSearch` object Pierre-Yves David <pierre-yves.david@octobus.net> parents: 51317 diff changeset	1194 and good not in (self.p1, self.p2)
c83074405276 delta-find: move `_refinedgroups` on the `_DeltaSearch` object Pierre-Yves David <pierre-yves.david@octobus.net> parents: 51317 diff changeset	1195 and self.revlog.issnapshot(good)
c83074405276 delta-find: move `_refinedgroups` on the `_DeltaSearch` object Pierre-Yves David <pierre-yves.david@octobus.net> parents: 51317 diff changeset	1196 ):
51347 670e68729aa7 delta-find: stop using heuristic to determine if we are creating a snapshot Pierre-Yves David <pierre-yves.david@octobus.net> parents: 51346 diff changeset	1197 assert self.current_stage == _STAGE_SNAPSHOT
51318 c83074405276 delta-find: move `_refinedgroups` on the `_DeltaSearch` object Pierre-Yves David <pierre-yves.david@octobus.net> parents: 51317 diff changeset	1198 # refine snapshot down
c83074405276 delta-find: move `_refinedgroups` on the `_DeltaSearch` object Pierre-Yves David <pierre-yves.david@octobus.net> parents: 51317 diff changeset	1199 previous = None
c83074405276 delta-find: move `_refinedgroups` on the `_DeltaSearch` object Pierre-Yves David <pierre-yves.david@octobus.net> parents: 51317 diff changeset	1200 while previous != good:
c83074405276 delta-find: move `_refinedgroups` on the `_DeltaSearch` object Pierre-Yves David <pierre-yves.david@octobus.net> parents: 51317 diff changeset	1201 previous = good
c83074405276 delta-find: move `_refinedgroups` on the `_DeltaSearch` object Pierre-Yves David <pierre-yves.david@octobus.net> parents: 51317 diff changeset	1202 base = self.revlog.deltaparent(good)
c83074405276 delta-find: move `_refinedgroups` on the `_DeltaSearch` object Pierre-Yves David <pierre-yves.david@octobus.net> parents: 51317 diff changeset	1203 if base == nullrev:
c83074405276 delta-find: move `_refinedgroups` on the `_DeltaSearch` object Pierre-Yves David <pierre-yves.david@octobus.net> parents: 51317 diff changeset	1204 break
c83074405276 delta-find: move `_refinedgroups` on the `_DeltaSearch` object Pierre-Yves David <pierre-yves.david@octobus.net> parents: 51317 diff changeset	1205 good = yield (base,)
c83074405276 delta-find: move `_refinedgroups` on the `_DeltaSearch` object Pierre-Yves David <pierre-yves.david@octobus.net> parents: 51317 diff changeset	1206 # refine snapshot up
c83074405276 delta-find: move `_refinedgroups` on the `_DeltaSearch` object Pierre-Yves David <pierre-yves.david@octobus.net> parents: 51317 diff changeset	1207 if not self.snapshot_cache.snapshots:
c83074405276 delta-find: move `_refinedgroups` on the `_DeltaSearch` object Pierre-Yves David <pierre-yves.david@octobus.net> parents: 51317 diff changeset	1208 self.snapshot_cache.update(self.revlog, good + 1)
c83074405276 delta-find: move `_refinedgroups` on the `_DeltaSearch` object Pierre-Yves David <pierre-yves.david@octobus.net> parents: 51317 diff changeset	1209 previous = None
c83074405276 delta-find: move `_refinedgroups` on the `_DeltaSearch` object Pierre-Yves David <pierre-yves.david@octobus.net> parents: 51317 diff changeset	1210 while good != previous:
c83074405276 delta-find: move `_refinedgroups` on the `_DeltaSearch` object Pierre-Yves David <pierre-yves.david@octobus.net> parents: 51317 diff changeset	1211 previous = good
c83074405276 delta-find: move `_refinedgroups` on the `_DeltaSearch` object Pierre-Yves David <pierre-yves.david@octobus.net> parents: 51317 diff changeset	1212 children = tuple(
c83074405276 delta-find: move `_refinedgroups` on the `_DeltaSearch` object Pierre-Yves David <pierre-yves.david@octobus.net> parents: 51317 diff changeset	1213 sorted(c for c in self.snapshot_cache.snapshots[good])
c83074405276 delta-find: move `_refinedgroups` on the `_DeltaSearch` object Pierre-Yves David <pierre-yves.david@octobus.net> parents: 51317 diff changeset	1214 )
c83074405276 delta-find: move `_refinedgroups` on the `_DeltaSearch` object Pierre-Yves David <pierre-yves.david@octobus.net> parents: 51317 diff changeset	1215 good = yield children
39503 5aef5afa8654 snapshot: refine candidate snapshot base upward Boris Feld <boris.feld@octobus.net> parents: 39502 diff changeset	1216
51346 cc806f20d756 delta-find: explicitly track stage of the search Pierre-Yves David <pierre-yves.david@octobus.net> parents: 51345 diff changeset	1217 self.current_stage = _STAGE_FULL
51318 c83074405276 delta-find: move `_refinedgroups` on the `_DeltaSearch` object Pierre-Yves David <pierre-yves.david@octobus.net> parents: 51317 diff changeset	1218 yield None
39496 2f9f7889549b snapshot: introduce an intermediate `_refinedgroups` generator Boris Feld <boris.feld@octobus.net> parents: 39495 diff changeset	1219
51319 a227e061bb4b delta-find: move `_rawgroups` on the `_DeltaSearch` object Pierre-Yves David <pierre-yves.david@octobus.net> parents: 51318 diff changeset	1220 def _raw_groups(self):
a227e061bb4b delta-find: move `_rawgroups` on the `_DeltaSearch` object Pierre-Yves David <pierre-yves.david@octobus.net> parents: 51318 diff changeset	1221 """Provides group of revision to be tested as delta base
43076 2372284d9457 formatting: blacken the codebase Augie Fackler <augie@google.com> parents: 42992 diff changeset	1222
51319 a227e061bb4b delta-find: move `_rawgroups` on the `_DeltaSearch` object Pierre-Yves David <pierre-yves.david@octobus.net> parents: 51318 diff changeset	1223 This lower level function focus on emitting delta theorically
a227e061bb4b delta-find: move `_rawgroups` on the `_DeltaSearch` object Pierre-Yves David <pierre-yves.david@octobus.net> parents: 51318 diff changeset	1224 interresting without looking it any practical details.
39336 1c6ff52fe9cf revlogdeltas: split candidate groups selection from the filtering logic Boris Feld <boris.feld@octobus.net> parents: 39335 diff changeset	1225
51319 a227e061bb4b delta-find: move `_rawgroups` on the `_DeltaSearch` object Pierre-Yves David <pierre-yves.david@octobus.net> parents: 51318 diff changeset	1226 The group order aims at providing fast or small candidates first.
a227e061bb4b delta-find: move `_rawgroups` on the `_DeltaSearch` object Pierre-Yves David <pierre-yves.david@octobus.net> parents: 51318 diff changeset	1227 """
51350 5cc04a6da19d delta-find: move the emotion of parents in a dedicated method Pierre-Yves David <pierre-yves.david@octobus.net> parents: 51349 diff changeset	1228 yield from self._iter_parents()
51352 fac6038b11f5 delta-find: move the base of the delta search in its own function Pierre-Yves David <pierre-yves.david@octobus.net> parents: 51351 diff changeset	1229 if self.revlog.delta_config.sparse_revlog:
fac6038b11f5 delta-find: move the base of the delta search in its own function Pierre-Yves David <pierre-yves.david@octobus.net> parents: 51351 diff changeset	1230 yield from self._iter_snapshots_base()
fac6038b11f5 delta-find: move the base of the delta search in its own function Pierre-Yves David <pierre-yves.david@octobus.net> parents: 51351 diff changeset	1231 else:
51351 94fe4474b053 delta-find: move the emotion of prev in a dedicated method Pierre-Yves David <pierre-yves.david@octobus.net> parents: 51350 diff changeset	1232 yield from self._iter_prev()
39334 507f5b1dd7c8 revlogdeltas: extract _getcandidaterevs in a function Boris Feld <boris.feld@octobus.net> parents: 39333 diff changeset	1233
43076 2372284d9457 formatting: blacken the codebase Augie Fackler <augie@google.com> parents: 42992 diff changeset	1234
49678 efbbc2f9121e delta-find: use a smarter object for snapshot caching Pierre-Yves David <pierre-yves.david@octobus.net> parents: 49677 diff changeset	1235 class SnapshotCache:
efbbc2f9121e delta-find: use a smarter object for snapshot caching Pierre-Yves David <pierre-yves.david@octobus.net> parents: 49677 diff changeset	1236 __slots__ = ('snapshots', '_start_rev', '_end_rev')
efbbc2f9121e delta-find: use a smarter object for snapshot caching Pierre-Yves David <pierre-yves.david@octobus.net> parents: 49677 diff changeset	1237
efbbc2f9121e delta-find: use a smarter object for snapshot caching Pierre-Yves David <pierre-yves.david@octobus.net> parents: 49677 diff changeset	1238 def __init__(self):
49679 b670eb3dd6c9 delta-find: use sets instead of list in the snapshot cache Pierre-Yves David <pierre-yves.david@octobus.net> parents: 49678 diff changeset	1239 self.snapshots = collections.defaultdict(set)
49678 efbbc2f9121e delta-find: use a smarter object for snapshot caching Pierre-Yves David <pierre-yves.david@octobus.net> parents: 49677 diff changeset	1240 self._start_rev = None
efbbc2f9121e delta-find: use a smarter object for snapshot caching Pierre-Yves David <pierre-yves.david@octobus.net> parents: 49677 diff changeset	1241 self._end_rev = None
efbbc2f9121e delta-find: use a smarter object for snapshot caching Pierre-Yves David <pierre-yves.david@octobus.net> parents: 49677 diff changeset	1242
efbbc2f9121e delta-find: use a smarter object for snapshot caching Pierre-Yves David <pierre-yves.david@octobus.net> parents: 49677 diff changeset	1243 def update(self, revlog, start_rev=0):
efbbc2f9121e delta-find: use a smarter object for snapshot caching Pierre-Yves David <pierre-yves.david@octobus.net> parents: 49677 diff changeset	1244 """find snapshots from start_rev to tip"""
efbbc2f9121e delta-find: use a smarter object for snapshot caching Pierre-Yves David <pierre-yves.david@octobus.net> parents: 49677 diff changeset	1245 nb_revs = len(revlog)
efbbc2f9121e delta-find: use a smarter object for snapshot caching Pierre-Yves David <pierre-yves.david@octobus.net> parents: 49677 diff changeset	1246 end_rev = nb_revs - 1
efbbc2f9121e delta-find: use a smarter object for snapshot caching Pierre-Yves David <pierre-yves.david@octobus.net> parents: 49677 diff changeset	1247 if start_rev > end_rev:
efbbc2f9121e delta-find: use a smarter object for snapshot caching Pierre-Yves David <pierre-yves.david@octobus.net> parents: 49677 diff changeset	1248 return # range is empty
efbbc2f9121e delta-find: use a smarter object for snapshot caching Pierre-Yves David <pierre-yves.david@octobus.net> parents: 49677 diff changeset	1249
efbbc2f9121e delta-find: use a smarter object for snapshot caching Pierre-Yves David <pierre-yves.david@octobus.net> parents: 49677 diff changeset	1250 if self._start_rev is None:
efbbc2f9121e delta-find: use a smarter object for snapshot caching Pierre-Yves David <pierre-yves.david@octobus.net> parents: 49677 diff changeset	1251 assert self._end_rev is None
efbbc2f9121e delta-find: use a smarter object for snapshot caching Pierre-Yves David <pierre-yves.david@octobus.net> parents: 49677 diff changeset	1252 self._update(revlog, start_rev, end_rev)
efbbc2f9121e delta-find: use a smarter object for snapshot caching Pierre-Yves David <pierre-yves.david@octobus.net> parents: 49677 diff changeset	1253 elif not (self._start_rev <= start_rev and end_rev <= self._end_rev):
efbbc2f9121e delta-find: use a smarter object for snapshot caching Pierre-Yves David <pierre-yves.david@octobus.net> parents: 49677 diff changeset	1254 if start_rev < self._start_rev:
efbbc2f9121e delta-find: use a smarter object for snapshot caching Pierre-Yves David <pierre-yves.david@octobus.net> parents: 49677 diff changeset	1255 self._update(revlog, start_rev, self._start_rev - 1)
efbbc2f9121e delta-find: use a smarter object for snapshot caching Pierre-Yves David <pierre-yves.david@octobus.net> parents: 49677 diff changeset	1256 if self._end_rev < end_rev:
efbbc2f9121e delta-find: use a smarter object for snapshot caching Pierre-Yves David <pierre-yves.david@octobus.net> parents: 49677 diff changeset	1257 self._update(revlog, self._end_rev + 1, end_rev)
efbbc2f9121e delta-find: use a smarter object for snapshot caching Pierre-Yves David <pierre-yves.david@octobus.net> parents: 49677 diff changeset	1258
efbbc2f9121e delta-find: use a smarter object for snapshot caching Pierre-Yves David <pierre-yves.david@octobus.net> parents: 49677 diff changeset	1259 if self._start_rev is None:
efbbc2f9121e delta-find: use a smarter object for snapshot caching Pierre-Yves David <pierre-yves.david@octobus.net> parents: 49677 diff changeset	1260 assert self._end_rev is None
efbbc2f9121e delta-find: use a smarter object for snapshot caching Pierre-Yves David <pierre-yves.david@octobus.net> parents: 49677 diff changeset	1261 self._end_rev = end_rev
efbbc2f9121e delta-find: use a smarter object for snapshot caching Pierre-Yves David <pierre-yves.david@octobus.net> parents: 49677 diff changeset	1262 self._start_rev = start_rev
efbbc2f9121e delta-find: use a smarter object for snapshot caching Pierre-Yves David <pierre-yves.david@octobus.net> parents: 49677 diff changeset	1263 else:
efbbc2f9121e delta-find: use a smarter object for snapshot caching Pierre-Yves David <pierre-yves.david@octobus.net> parents: 49677 diff changeset	1264 self._start_rev = min(self._start_rev, start_rev)
efbbc2f9121e delta-find: use a smarter object for snapshot caching Pierre-Yves David <pierre-yves.david@octobus.net> parents: 49677 diff changeset	1265 self._end_rev = max(self._end_rev, end_rev)
efbbc2f9121e delta-find: use a smarter object for snapshot caching Pierre-Yves David <pierre-yves.david@octobus.net> parents: 49677 diff changeset	1266 assert self._start_rev <= self._end_rev, (
efbbc2f9121e delta-find: use a smarter object for snapshot caching Pierre-Yves David <pierre-yves.david@octobus.net> parents: 49677 diff changeset	1267 self._start_rev,
efbbc2f9121e delta-find: use a smarter object for snapshot caching Pierre-Yves David <pierre-yves.david@octobus.net> parents: 49677 diff changeset	1268 self._end_rev,
efbbc2f9121e delta-find: use a smarter object for snapshot caching Pierre-Yves David <pierre-yves.david@octobus.net> parents: 49677 diff changeset	1269 )
efbbc2f9121e delta-find: use a smarter object for snapshot caching Pierre-Yves David <pierre-yves.david@octobus.net> parents: 49677 diff changeset	1270
efbbc2f9121e delta-find: use a smarter object for snapshot caching Pierre-Yves David <pierre-yves.david@octobus.net> parents: 49677 diff changeset	1271 def _update(self, revlog, start_rev, end_rev):
efbbc2f9121e delta-find: use a smarter object for snapshot caching Pierre-Yves David <pierre-yves.david@octobus.net> parents: 49677 diff changeset	1272 """internal method that actually do update content"""
efbbc2f9121e delta-find: use a smarter object for snapshot caching Pierre-Yves David <pierre-yves.david@octobus.net> parents: 49677 diff changeset	1273 assert self._start_rev is None or (
efbbc2f9121e delta-find: use a smarter object for snapshot caching Pierre-Yves David <pierre-yves.david@octobus.net> parents: 49677 diff changeset	1274 start_rev < self._start_rev or start_rev > self._end_rev
efbbc2f9121e delta-find: use a smarter object for snapshot caching Pierre-Yves David <pierre-yves.david@octobus.net> parents: 49677 diff changeset	1275 ), (self._start_rev, self._end_rev, start_rev, end_rev)
efbbc2f9121e delta-find: use a smarter object for snapshot caching Pierre-Yves David <pierre-yves.david@octobus.net> parents: 49677 diff changeset	1276 assert self._start_rev is None or (
efbbc2f9121e delta-find: use a smarter object for snapshot caching Pierre-Yves David <pierre-yves.david@octobus.net> parents: 49677 diff changeset	1277 end_rev < self._start_rev or end_rev > self._end_rev
efbbc2f9121e delta-find: use a smarter object for snapshot caching Pierre-Yves David <pierre-yves.david@octobus.net> parents: 49677 diff changeset	1278 ), (self._start_rev, self._end_rev, start_rev, end_rev)
efbbc2f9121e delta-find: use a smarter object for snapshot caching Pierre-Yves David <pierre-yves.david@octobus.net> parents: 49677 diff changeset	1279 cache = self.snapshots
50925 d718eddf01d9 safehasattr: drop usage in favor of hasattr Pierre-Yves David <pierre-yves.david@octobus.net> parents: 50695 diff changeset	1280 if hasattr(revlog.index, 'findsnapshots'):
49678 efbbc2f9121e delta-find: use a smarter object for snapshot caching Pierre-Yves David <pierre-yves.david@octobus.net> parents: 49677 diff changeset	1281 revlog.index.findsnapshots(cache, start_rev, end_rev)
efbbc2f9121e delta-find: use a smarter object for snapshot caching Pierre-Yves David <pierre-yves.david@octobus.net> parents: 49677 diff changeset	1282 else:
efbbc2f9121e delta-find: use a smarter object for snapshot caching Pierre-Yves David <pierre-yves.david@octobus.net> parents: 49677 diff changeset	1283 deltaparent = revlog.deltaparent
efbbc2f9121e delta-find: use a smarter object for snapshot caching Pierre-Yves David <pierre-yves.david@octobus.net> parents: 49677 diff changeset	1284 issnapshot = revlog.issnapshot
efbbc2f9121e delta-find: use a smarter object for snapshot caching Pierre-Yves David <pierre-yves.david@octobus.net> parents: 49677 diff changeset	1285 for rev in revlog.revs(start_rev, end_rev):
efbbc2f9121e delta-find: use a smarter object for snapshot caching Pierre-Yves David <pierre-yves.david@octobus.net> parents: 49677 diff changeset	1286 if issnapshot(rev):
49679 b670eb3dd6c9 delta-find: use sets instead of list in the snapshot cache Pierre-Yves David <pierre-yves.david@octobus.net> parents: 49678 diff changeset	1287 cache[deltaparent(rev)].add(rev)
49678 efbbc2f9121e delta-find: use a smarter object for snapshot caching Pierre-Yves David <pierre-yves.david@octobus.net> parents: 49677 diff changeset	1288
efbbc2f9121e delta-find: use a smarter object for snapshot caching Pierre-Yves David <pierre-yves.david@octobus.net> parents: 49677 diff changeset	1289
48946 642e31cb55f0 py3: use class X: instead of class X(object): Gregory Szorc <gregory.szorc@gmail.com> parents: 48875 diff changeset	1290 class deltacomputer:
51316 da7ecb4deaec delta-find: add a small docstring to deltacomputer Pierre-Yves David <pierre-yves.david@octobus.net> parents: 51088 diff changeset	1291 """object capable of computing delta and finding delta for multiple revision
da7ecb4deaec delta-find: add a small docstring to deltacomputer Pierre-Yves David <pierre-yves.david@octobus.net> parents: 51088 diff changeset	1292
da7ecb4deaec delta-find: add a small docstring to deltacomputer Pierre-Yves David <pierre-yves.david@octobus.net> parents: 51088 diff changeset	1293 This object is meant to compute and find multiple delta applied to the same
da7ecb4deaec delta-find: add a small docstring to deltacomputer Pierre-Yves David <pierre-yves.david@octobus.net> parents: 51088 diff changeset	1294 revlog.
da7ecb4deaec delta-find: add a small docstring to deltacomputer Pierre-Yves David <pierre-yves.david@octobus.net> parents: 51088 diff changeset	1295 """
da7ecb4deaec delta-find: add a small docstring to deltacomputer Pierre-Yves David <pierre-yves.david@octobus.net> parents: 51088 diff changeset	1296
49610 35d4c2124073 debug: add an option to display statistic about a unbundling operation Pierre-Yves David <pierre-yves.david@octobus.net> parents: 49608 diff changeset	1297 def __init__(
35d4c2124073 debug: add an option to display statistic about a unbundling operation Pierre-Yves David <pierre-yves.david@octobus.net> parents: 49608 diff changeset	1298 self,
35d4c2124073 debug: add an option to display statistic about a unbundling operation Pierre-Yves David <pierre-yves.david@octobus.net> parents: 49608 diff changeset	1299 revlog,
35d4c2124073 debug: add an option to display statistic about a unbundling operation Pierre-Yves David <pierre-yves.david@octobus.net> parents: 49608 diff changeset	1300 write_debug=None,
35d4c2124073 debug: add an option to display statistic about a unbundling operation Pierre-Yves David <pierre-yves.david@octobus.net> parents: 49608 diff changeset	1301 debug_search=False,
35d4c2124073 debug: add an option to display statistic about a unbundling operation Pierre-Yves David <pierre-yves.david@octobus.net> parents: 49608 diff changeset	1302 debug_info=None,
35d4c2124073 debug: add an option to display statistic about a unbundling operation Pierre-Yves David <pierre-yves.david@octobus.net> parents: 49608 diff changeset	1303 ):
35738 f90f6fd130c1 revlog: group delta computation methods under _deltacomputer object Paul Morelle <paul.morelle@octobus.net> parents: 35737 diff changeset	1304 self.revlog = revlog
49226 e6b7c6fbeb48 deltas: add code to display information about the result of `finddeltainfo` Pierre-Yves David <pierre-yves.david@octobus.net> parents: 48946 diff changeset	1305 self._write_debug = write_debug
50646 c84cc0ac77e4 delta-fine: use the `_debug_search` attribute directly Pierre-Yves David <pierre-yves.david@octobus.net> parents: 50645 diff changeset	1306 if write_debug is None:
c84cc0ac77e4 delta-fine: use the `_debug_search` attribute directly Pierre-Yves David <pierre-yves.david@octobus.net> parents: 50645 diff changeset	1307 self._debug_search = False
c84cc0ac77e4 delta-fine: use the `_debug_search` attribute directly Pierre-Yves David <pierre-yves.david@octobus.net> parents: 50645 diff changeset	1308 else:
c84cc0ac77e4 delta-fine: use the `_debug_search` attribute directly Pierre-Yves David <pierre-yves.david@octobus.net> parents: 50645 diff changeset	1309 self._debug_search = debug_search
49610 35d4c2124073 debug: add an option to display statistic about a unbundling operation Pierre-Yves David <pierre-yves.david@octobus.net> parents: 49608 diff changeset	1310 self._debug_info = debug_info
49681 c261a628e525 delta-find: use a single snapshot cache when applying a group to an object Pierre-Yves David <pierre-yves.david@octobus.net> parents: 49680 diff changeset	1311 self._snapshot_cache = SnapshotCache()
35738 f90f6fd130c1 revlog: group delta computation methods under _deltacomputer object Paul Morelle <paul.morelle@octobus.net> parents: 35737 diff changeset	1312
50645 5d210ff4b657 delta-find: move the `gather_debug` logic in a property Pierre-Yves David <pierre-yves.david@octobus.net> parents: 50357 diff changeset	1313 @property
5d210ff4b657 delta-find: move the `gather_debug` logic in a property Pierre-Yves David <pierre-yves.david@octobus.net> parents: 50357 diff changeset	1314 def _gather_debug(self):
5d210ff4b657 delta-find: move the `gather_debug` logic in a property Pierre-Yves David <pierre-yves.david@octobus.net> parents: 50357 diff changeset	1315 return self._write_debug is not None or self._debug_info is not None
5d210ff4b657 delta-find: move the `gather_debug` logic in a property Pierre-Yves David <pierre-yves.david@octobus.net> parents: 50357 diff changeset	1316
51017 509f0f7fc89e delta-computer: stop explicitly taking file handle Pierre-Yves David <pierre-yves.david@octobus.net> parents: 50926 diff changeset	1317 def buildtext(self, revinfo):
35738 f90f6fd130c1 revlog: group delta computation methods under _deltacomputer object Paul Morelle <paul.morelle@octobus.net> parents: 35737 diff changeset	1318 """Builds a fulltext version of a revision
f90f6fd130c1 revlog: group delta computation methods under _deltacomputer object Paul Morelle <paul.morelle@octobus.net> parents: 35737 diff changeset	1319
47399 34cc102c73f5 revlog: move `revisioninfo` in `revlogutils` Pierre-Yves David <pierre-yves.david@octobus.net> parents: 47253 diff changeset	1320 revinfo: revisioninfo instance that contains all needed info
35738 f90f6fd130c1 revlog: group delta computation methods under _deltacomputer object Paul Morelle <paul.morelle@octobus.net> parents: 35737 diff changeset	1321 """
f90f6fd130c1 revlog: group delta computation methods under _deltacomputer object Paul Morelle <paul.morelle@octobus.net> parents: 35737 diff changeset	1322 btext = revinfo.btext
f90f6fd130c1 revlog: group delta computation methods under _deltacomputer object Paul Morelle <paul.morelle@octobus.net> parents: 35737 diff changeset	1323 if btext[0] is not None:
f90f6fd130c1 revlog: group delta computation methods under _deltacomputer object Paul Morelle <paul.morelle@octobus.net> parents: 35737 diff changeset	1324 return btext[0]
f90f6fd130c1 revlog: group delta computation methods under _deltacomputer object Paul Morelle <paul.morelle@octobus.net> parents: 35737 diff changeset	1325
f90f6fd130c1 revlog: group delta computation methods under _deltacomputer object Paul Morelle <paul.morelle@octobus.net> parents: 35737 diff changeset	1326 revlog = self.revlog
f90f6fd130c1 revlog: group delta computation methods under _deltacomputer object Paul Morelle <paul.morelle@octobus.net> parents: 35737 diff changeset	1327 cachedelta = revinfo.cachedelta
f90f6fd130c1 revlog: group delta computation methods under _deltacomputer object Paul Morelle <paul.morelle@octobus.net> parents: 35737 diff changeset	1328 baserev = cachedelta[0]
f90f6fd130c1 revlog: group delta computation methods under _deltacomputer object Paul Morelle <paul.morelle@octobus.net> parents: 35737 diff changeset	1329 delta = cachedelta[1]
f90f6fd130c1 revlog: group delta computation methods under _deltacomputer object Paul Morelle <paul.morelle@octobus.net> parents: 35737 diff changeset	1330
43076 2372284d9457 formatting: blacken the codebase Augie Fackler <augie@google.com> parents: 42992 diff changeset	1331 fulltext = btext[0] = _textfromdelta(
2372284d9457 formatting: blacken the codebase Augie Fackler <augie@google.com> parents: 42992 diff changeset	1332 revlog,
2372284d9457 formatting: blacken the codebase Augie Fackler <augie@google.com> parents: 42992 diff changeset	1333 baserev,
2372284d9457 formatting: blacken the codebase Augie Fackler <augie@google.com> parents: 42992 diff changeset	1334 delta,
2372284d9457 formatting: blacken the codebase Augie Fackler <augie@google.com> parents: 42992 diff changeset	1335 revinfo.p1,
2372284d9457 formatting: blacken the codebase Augie Fackler <augie@google.com> parents: 42992 diff changeset	1336 revinfo.p2,
2372284d9457 formatting: blacken the codebase Augie Fackler <augie@google.com> parents: 42992 diff changeset	1337 revinfo.flags,
2372284d9457 formatting: blacken the codebase Augie Fackler <augie@google.com> parents: 42992 diff changeset	1338 revinfo.node,
2372284d9457 formatting: blacken the codebase Augie Fackler <augie@google.com> parents: 42992 diff changeset	1339 )
39331 fd0150a3c2fe revlogdeltas: extra fulltext building in its own function Boris Feld <boris.feld@octobus.net> parents: 39330 diff changeset	1340 return fulltext
35738 f90f6fd130c1 revlog: group delta computation methods under _deltacomputer object Paul Morelle <paul.morelle@octobus.net> parents: 35737 diff changeset	1341
51017 509f0f7fc89e delta-computer: stop explicitly taking file handle Pierre-Yves David <pierre-yves.david@octobus.net> parents: 50926 diff changeset	1342 def _builddeltadiff(self, base, revinfo):
35738 f90f6fd130c1 revlog: group delta computation methods under _deltacomputer object Paul Morelle <paul.morelle@octobus.net> parents: 35737 diff changeset	1343 revlog = self.revlog
51017 509f0f7fc89e delta-computer: stop explicitly taking file handle Pierre-Yves David <pierre-yves.david@octobus.net> parents: 50926 diff changeset	1344 t = self.buildtext(revinfo)
35738 f90f6fd130c1 revlog: group delta computation methods under _deltacomputer object Paul Morelle <paul.morelle@octobus.net> parents: 35737 diff changeset	1345 if revlog.iscensored(base):
f90f6fd130c1 revlog: group delta computation methods under _deltacomputer object Paul Morelle <paul.morelle@octobus.net> parents: 35737 diff changeset	1346 # deltas based on a censored revision must replace the
f90f6fd130c1 revlog: group delta computation methods under _deltacomputer object Paul Morelle <paul.morelle@octobus.net> parents: 35737 diff changeset	1347 # full content in one patch, so delta works everywhere
f90f6fd130c1 revlog: group delta computation methods under _deltacomputer object Paul Morelle <paul.morelle@octobus.net> parents: 35737 diff changeset	1348 header = mdiff.replacediffheader(revlog.rawsize(base), len(t))
f90f6fd130c1 revlog: group delta computation methods under _deltacomputer object Paul Morelle <paul.morelle@octobus.net> parents: 35737 diff changeset	1349 delta = header + t
f90f6fd130c1 revlog: group delta computation methods under _deltacomputer object Paul Morelle <paul.morelle@octobus.net> parents: 35737 diff changeset	1350 else:
51017 509f0f7fc89e delta-computer: stop explicitly taking file handle Pierre-Yves David <pierre-yves.david@octobus.net> parents: 50926 diff changeset	1351 ptext = revlog.rawdata(base)
35738 f90f6fd130c1 revlog: group delta computation methods under _deltacomputer object Paul Morelle <paul.morelle@octobus.net> parents: 35737 diff changeset	1352 delta = mdiff.textdiff(ptext, t)
f90f6fd130c1 revlog: group delta computation methods under _deltacomputer object Paul Morelle <paul.morelle@octobus.net> parents: 35737 diff changeset	1353
f90f6fd130c1 revlog: group delta computation methods under _deltacomputer object Paul Morelle <paul.morelle@octobus.net> parents: 35737 diff changeset	1354 return delta
f90f6fd130c1 revlog: group delta computation methods under _deltacomputer object Paul Morelle <paul.morelle@octobus.net> parents: 35737 diff changeset	1355
51347 670e68729aa7 delta-find: stop using heuristic to determine if we are creating a snapshot Pierre-Yves David <pierre-yves.david@octobus.net> parents: 51346 diff changeset	1356 def _builddeltainfo(
670e68729aa7 delta-find: stop using heuristic to determine if we are creating a snapshot Pierre-Yves David <pierre-yves.david@octobus.net> parents: 51346 diff changeset	1357 self, revinfo, base, target_rev=None, as_snapshot=False
670e68729aa7 delta-find: stop using heuristic to determine if we are creating a snapshot Pierre-Yves David <pierre-yves.david@octobus.net> parents: 51346 diff changeset	1358 ):
35738 f90f6fd130c1 revlog: group delta computation methods under _deltacomputer object Paul Morelle <paul.morelle@octobus.net> parents: 35737 diff changeset	1359 # can we use the cached delta?
42465 6e9ba867a946 delta: move some delta chain related computation earlier in deltainfo Valentin Gatien-Baron <vgatien-baron@janestreet.com> parents: 42464 diff changeset	1360 revlog = self.revlog
6e9ba867a946 delta: move some delta chain related computation earlier in deltainfo Valentin Gatien-Baron <vgatien-baron@janestreet.com> parents: 42464 diff changeset	1361 chainbase = revlog.chainbase(base)
51043 f636103c4d67 revlog: remove legacy usage of `_generaldelta` Pierre-Yves David <pierre-yves.david@octobus.net> parents: 51042 diff changeset	1362 if revlog.delta_config.general_delta:
42465 6e9ba867a946 delta: move some delta chain related computation earlier in deltainfo Valentin Gatien-Baron <vgatien-baron@janestreet.com> parents: 42464 diff changeset	1363 deltabase = base
6e9ba867a946 delta: move some delta chain related computation earlier in deltainfo Valentin Gatien-Baron <vgatien-baron@janestreet.com> parents: 42464 diff changeset	1364 else:
50356 2a6949ab9d23 delta-find: add a simple safeguard to prevent bad non-general-delta Pierre-Yves David <pierre-yves.david@octobus.net> parents: 50355 diff changeset	1365 if target_rev is not None and base != target_rev - 1:
2a6949ab9d23 delta-find: add a simple safeguard to prevent bad non-general-delta Pierre-Yves David <pierre-yves.david@octobus.net> parents: 50355 diff changeset	1366 msg = (
2a6949ab9d23 delta-find: add a simple safeguard to prevent bad non-general-delta Pierre-Yves David <pierre-yves.david@octobus.net> parents: 50355 diff changeset	1367 b'general delta cannot use delta for something else '
2a6949ab9d23 delta-find: add a simple safeguard to prevent bad non-general-delta Pierre-Yves David <pierre-yves.david@octobus.net> parents: 50355 diff changeset	1368 b'than `prev`: %d<-%d'
2a6949ab9d23 delta-find: add a simple safeguard to prevent bad non-general-delta Pierre-Yves David <pierre-yves.david@octobus.net> parents: 50355 diff changeset	1369 )
2a6949ab9d23 delta-find: add a simple safeguard to prevent bad non-general-delta Pierre-Yves David <pierre-yves.david@octobus.net> parents: 50355 diff changeset	1370 msg %= (base, target_rev)
2a6949ab9d23 delta-find: add a simple safeguard to prevent bad non-general-delta Pierre-Yves David <pierre-yves.david@octobus.net> parents: 50355 diff changeset	1371 raise error.ProgrammingError(msg)
42465 6e9ba867a946 delta: move some delta chain related computation earlier in deltainfo Valentin Gatien-Baron <vgatien-baron@janestreet.com> parents: 42464 diff changeset	1372 deltabase = chainbase
6e9ba867a946 delta: move some delta chain related computation earlier in deltainfo Valentin Gatien-Baron <vgatien-baron@janestreet.com> parents: 42464 diff changeset	1373 snapshotdepth = None
51057 f71f07a679b4 revlog: remove legacy usage of `_sparserevlog` Pierre-Yves David <pierre-yves.david@octobus.net> parents: 51055 diff changeset	1374 if revlog.delta_config.sparse_revlog and deltabase == nullrev:
42465 6e9ba867a946 delta: move some delta chain related computation earlier in deltainfo Valentin Gatien-Baron <vgatien-baron@janestreet.com> parents: 42464 diff changeset	1375 snapshotdepth = 0
51347 670e68729aa7 delta-find: stop using heuristic to determine if we are creating a snapshot Pierre-Yves David <pierre-yves.david@octobus.net> parents: 51346 diff changeset	1376 elif revlog.delta_config.sparse_revlog and as_snapshot:
670e68729aa7 delta-find: stop using heuristic to determine if we are creating a snapshot Pierre-Yves David <pierre-yves.david@octobus.net> parents: 51346 diff changeset	1377 assert revlog.issnapshot(deltabase)
42465 6e9ba867a946 delta: move some delta chain related computation earlier in deltainfo Valentin Gatien-Baron <vgatien-baron@janestreet.com> parents: 42464 diff changeset	1378 # A delta chain should always be one full snapshot,
6e9ba867a946 delta: move some delta chain related computation earlier in deltainfo Valentin Gatien-Baron <vgatien-baron@janestreet.com> parents: 42464 diff changeset	1379 # zero or more semi-snapshots, and zero or more deltas
6e9ba867a946 delta: move some delta chain related computation earlier in deltainfo Valentin Gatien-Baron <vgatien-baron@janestreet.com> parents: 42464 diff changeset	1380 p1, p2 = revlog.rev(revinfo.p1), revlog.rev(revinfo.p2)
6e9ba867a946 delta: move some delta chain related computation earlier in deltainfo Valentin Gatien-Baron <vgatien-baron@janestreet.com> parents: 42464 diff changeset	1381 if deltabase not in (p1, p2) and revlog.issnapshot(deltabase):
6e9ba867a946 delta: move some delta chain related computation earlier in deltainfo Valentin Gatien-Baron <vgatien-baron@janestreet.com> parents: 42464 diff changeset	1382 snapshotdepth = len(revlog._deltachain(deltabase)[0])
39595 a911932d5003 revlog: reuse cached delta for identical base revision (issue5975) Boris Feld <boris.feld@octobus.net> parents: 39594 diff changeset	1383 delta = None
a911932d5003 revlog: reuse cached delta for identical base revision (issue5975) Boris Feld <boris.feld@octobus.net> parents: 39594 diff changeset	1384 if revinfo.cachedelta:
49675 0fca63953810 find-delta: minor preparatory change Pierre-Yves David <pierre-yves.david@octobus.net> parents: 49674 diff changeset	1385 cachebase = revinfo.cachedelta[0]
43076 2372284d9457 formatting: blacken the codebase Augie Fackler <augie@google.com> parents: 42992 diff changeset	1386 # check if the diff still apply
39595 a911932d5003 revlog: reuse cached delta for identical base revision (issue5975) Boris Feld <boris.feld@octobus.net> parents: 39594 diff changeset	1387 currentbase = cachebase
43076 2372284d9457 formatting: blacken the codebase Augie Fackler <augie@google.com> parents: 42992 diff changeset	1388 while (
2372284d9457 formatting: blacken the codebase Augie Fackler <augie@google.com> parents: 42992 diff changeset	1389 currentbase != nullrev
2372284d9457 formatting: blacken the codebase Augie Fackler <augie@google.com> parents: 42992 diff changeset	1390 and currentbase != base
2372284d9457 formatting: blacken the codebase Augie Fackler <augie@google.com> parents: 42992 diff changeset	1391 and self.revlog.length(currentbase) == 0
2372284d9457 formatting: blacken the codebase Augie Fackler <augie@google.com> parents: 42992 diff changeset	1392 ):
39595 a911932d5003 revlog: reuse cached delta for identical base revision (issue5975) Boris Feld <boris.feld@octobus.net> parents: 39594 diff changeset	1393 currentbase = self.revlog.deltaparent(currentbase)
51064 26dcdbe15024 revlog: remove legacy usage of `_lazydeltabase` Pierre-Yves David <pierre-yves.david@octobus.net> parents: 51060 diff changeset	1394 if self.revlog.delta_config.lazy_delta and currentbase == base:
39595 a911932d5003 revlog: reuse cached delta for identical base revision (issue5975) Boris Feld <boris.feld@octobus.net> parents: 39594 diff changeset	1395 delta = revinfo.cachedelta[1]
a911932d5003 revlog: reuse cached delta for identical base revision (issue5975) Boris Feld <boris.feld@octobus.net> parents: 39594 diff changeset	1396 if delta is None:
51017 509f0f7fc89e delta-computer: stop explicitly taking file handle Pierre-Yves David <pierre-yves.david@octobus.net> parents: 50926 diff changeset	1397 delta = self._builddeltadiff(base, revinfo)
50646 c84cc0ac77e4 delta-fine: use the `_debug_search` attribute directly Pierre-Yves David <pierre-yves.david@octobus.net> parents: 50645 diff changeset	1398 if self._debug_search:
49228 b909dd35d9ab deltas: add a debug-delta-find command to analyse delta search Pierre-Yves David <pierre-yves.david@octobus.net> parents: 49226 diff changeset	1399 msg = b"DBG-DELTAS-SEARCH: uncompressed-delta-size=%d\n"
b909dd35d9ab deltas: add a debug-delta-find command to analyse delta search Pierre-Yves David <pierre-yves.david@octobus.net> parents: 49226 diff changeset	1400 msg %= len(delta)
b909dd35d9ab deltas: add a debug-delta-find command to analyse delta search Pierre-Yves David <pierre-yves.david@octobus.net> parents: 49226 diff changeset	1401 self._write_debug(msg)
42467 c1c1872d25d1 deltas: skip if projected compressed size does not match text size constraint Valentin Gatien-Baron <vgatien-baron@janestreet.com> parents: 42466 diff changeset	1402 # snapshotdept need to be neither None nor 0 level snapshot
51080 5645524c6b62 revlog: also migrates `revlog.upperboundcomp` to ConfigClass Pierre-Yves David <pierre-yves.david@octobus.net> parents: 51064 diff changeset	1403 if revlog.delta_config.upper_bound_comp is not None and snapshotdepth:
5645524c6b62 revlog: also migrates `revlog.upperboundcomp` to ConfigClass Pierre-Yves David <pierre-yves.david@octobus.net> parents: 51064 diff changeset	1404 lowestrealisticdeltalen = (
5645524c6b62 revlog: also migrates `revlog.upperboundcomp` to ConfigClass Pierre-Yves David <pierre-yves.david@octobus.net> parents: 51064 diff changeset	1405 len(delta) // revlog.delta_config.upper_bound_comp
5645524c6b62 revlog: also migrates `revlog.upperboundcomp` to ConfigClass Pierre-Yves David <pierre-yves.david@octobus.net> parents: 51064 diff changeset	1406 )
42467 c1c1872d25d1 deltas: skip if projected compressed size does not match text size constraint Valentin Gatien-Baron <vgatien-baron@janestreet.com> parents: 42466 diff changeset	1407 snapshotlimit = revinfo.textlen >> snapshotdepth
50646 c84cc0ac77e4 delta-fine: use the `_debug_search` attribute directly Pierre-Yves David <pierre-yves.david@octobus.net> parents: 50645 diff changeset	1408 if self._debug_search:
49228 b909dd35d9ab deltas: add a debug-delta-find command to analyse delta search Pierre-Yves David <pierre-yves.david@octobus.net> parents: 49226 diff changeset	1409 msg = b"DBG-DELTAS-SEARCH: projected-lower-size=%d\n"
b909dd35d9ab deltas: add a debug-delta-find command to analyse delta search Pierre-Yves David <pierre-yves.david@octobus.net> parents: 49226 diff changeset	1410 msg %= lowestrealisticdeltalen
b909dd35d9ab deltas: add a debug-delta-find command to analyse delta search Pierre-Yves David <pierre-yves.david@octobus.net> parents: 49226 diff changeset	1411 self._write_debug(msg)
42467 c1c1872d25d1 deltas: skip if projected compressed size does not match text size constraint Valentin Gatien-Baron <vgatien-baron@janestreet.com> parents: 42466 diff changeset	1412 if snapshotlimit < lowestrealisticdeltalen:
50646 c84cc0ac77e4 delta-fine: use the `_debug_search` attribute directly Pierre-Yves David <pierre-yves.david@octobus.net> parents: 50645 diff changeset	1413 if self._debug_search:
49228 b909dd35d9ab deltas: add a debug-delta-find command to analyse delta search Pierre-Yves David <pierre-yves.david@octobus.net> parents: 49226 diff changeset	1414 msg = b"DBG-DELTAS-SEARCH: DISCARDED (snapshot limit)\n"
b909dd35d9ab deltas: add a debug-delta-find command to analyse delta search Pierre-Yves David <pierre-yves.david@octobus.net> parents: 49226 diff changeset	1415 self._write_debug(msg)
42467 c1c1872d25d1 deltas: skip if projected compressed size does not match text size constraint Valentin Gatien-Baron <vgatien-baron@janestreet.com> parents: 42466 diff changeset	1416 return None
42468 9b5fbe5ead89 deltas: skip if projected compressed size is bigger than previous snapshot Valentin Gatien-Baron <vgatien-baron@janestreet.com> parents: 42467 diff changeset	1417 if revlog.length(base) < lowestrealisticdeltalen:
50646 c84cc0ac77e4 delta-fine: use the `_debug_search` attribute directly Pierre-Yves David <pierre-yves.david@octobus.net> parents: 50645 diff changeset	1418 if self._debug_search:
49228 b909dd35d9ab deltas: add a debug-delta-find command to analyse delta search Pierre-Yves David <pierre-yves.david@octobus.net> parents: 49226 diff changeset	1419 msg = b"DBG-DELTAS-SEARCH: DISCARDED (prev size)\n"
b909dd35d9ab deltas: add a debug-delta-find command to analyse delta search Pierre-Yves David <pierre-yves.david@octobus.net> parents: 49226 diff changeset	1420 self._write_debug(msg)
42468 9b5fbe5ead89 deltas: skip if projected compressed size is bigger than previous snapshot Valentin Gatien-Baron <vgatien-baron@janestreet.com> parents: 42467 diff changeset	1421 return None
51088 a82704902db8 revlog: move the compression/decompression logic on the inner object Pierre-Yves David <pierre-yves.david@octobus.net> parents: 51080 diff changeset	1422 header, data = revlog._inner.compress(delta)
35738 f90f6fd130c1 revlog: group delta computation methods under _deltacomputer object Paul Morelle <paul.morelle@octobus.net> parents: 35737 diff changeset	1423 deltalen = len(header) + len(data)
f90f6fd130c1 revlog: group delta computation methods under _deltacomputer object Paul Morelle <paul.morelle@octobus.net> parents: 35737 diff changeset	1424 offset = revlog.end(len(revlog) - 1)
f90f6fd130c1 revlog: group delta computation methods under _deltacomputer object Paul Morelle <paul.morelle@octobus.net> parents: 35737 diff changeset	1425 dist = deltalen + offset - revlog.start(chainbase)
f90f6fd130c1 revlog: group delta computation methods under _deltacomputer object Paul Morelle <paul.morelle@octobus.net> parents: 35737 diff changeset	1426 chainlen, compresseddeltalen = revlog._chaininfo(base)
f90f6fd130c1 revlog: group delta computation methods under _deltacomputer object Paul Morelle <paul.morelle@octobus.net> parents: 35737 diff changeset	1427 chainlen += 1
f90f6fd130c1 revlog: group delta computation methods under _deltacomputer object Paul Morelle <paul.morelle@octobus.net> parents: 35737 diff changeset	1428 compresseddeltalen += deltalen
39154 e0da43e2f71f revlog: compute snapshot depth on delta info Boris Feld <boris.feld@octobus.net> parents: 39152 diff changeset	1429
43076 2372284d9457 formatting: blacken the codebase Augie Fackler <augie@google.com> parents: 42992 diff changeset	1430 return _deltainfo(
2372284d9457 formatting: blacken the codebase Augie Fackler <augie@google.com> parents: 42992 diff changeset	1431 dist,
2372284d9457 formatting: blacken the codebase Augie Fackler <augie@google.com> parents: 42992 diff changeset	1432 deltalen,
2372284d9457 formatting: blacken the codebase Augie Fackler <augie@google.com> parents: 42992 diff changeset	1433 (header, data),
2372284d9457 formatting: blacken the codebase Augie Fackler <augie@google.com> parents: 42992 diff changeset	1434 deltabase,
2372284d9457 formatting: blacken the codebase Augie Fackler <augie@google.com> parents: 42992 diff changeset	1435 chainbase,
2372284d9457 formatting: blacken the codebase Augie Fackler <augie@google.com> parents: 42992 diff changeset	1436 chainlen,
2372284d9457 formatting: blacken the codebase Augie Fackler <augie@google.com> parents: 42992 diff changeset	1437 compresseddeltalen,
2372284d9457 formatting: blacken the codebase Augie Fackler <augie@google.com> parents: 42992 diff changeset	1438 snapshotdepth,
2372284d9457 formatting: blacken the codebase Augie Fackler <augie@google.com> parents: 42992 diff changeset	1439 )
35738 f90f6fd130c1 revlog: group delta computation methods under _deltacomputer object Paul Morelle <paul.morelle@octobus.net> parents: 35737 diff changeset	1440
51017 509f0f7fc89e delta-computer: stop explicitly taking file handle Pierre-Yves David <pierre-yves.david@octobus.net> parents: 50926 diff changeset	1441 def _fullsnapshotinfo(self, revinfo, curr):
509f0f7fc89e delta-computer: stop explicitly taking file handle Pierre-Yves David <pierre-yves.david@octobus.net> parents: 50926 diff changeset	1442 rawtext = self.buildtext(revinfo)
51088 a82704902db8 revlog: move the compression/decompression logic on the inner object Pierre-Yves David <pierre-yves.david@octobus.net> parents: 51080 diff changeset	1443 data = self.revlog._inner.compress(rawtext)
39333 5d343a24bff5 revlogdeltas: always return a delta info object in finddeltainfo Boris Feld <boris.feld@octobus.net> parents: 39332 diff changeset	1444 compresseddeltalen = deltalen = dist = len(data[1]) + len(data[0])
5d343a24bff5 revlogdeltas: always return a delta info object in finddeltainfo Boris Feld <boris.feld@octobus.net> parents: 39332 diff changeset	1445 deltabase = chainbase = curr
5d343a24bff5 revlogdeltas: always return a delta info object in finddeltainfo Boris Feld <boris.feld@octobus.net> parents: 39332 diff changeset	1446 snapshotdepth = 0
5d343a24bff5 revlogdeltas: always return a delta info object in finddeltainfo Boris Feld <boris.feld@octobus.net> parents: 39332 diff changeset	1447 chainlen = 1
5d343a24bff5 revlogdeltas: always return a delta info object in finddeltainfo Boris Feld <boris.feld@octobus.net> parents: 39332 diff changeset	1448
43076 2372284d9457 formatting: blacken the codebase Augie Fackler <augie@google.com> parents: 42992 diff changeset	1449 return _deltainfo(
2372284d9457 formatting: blacken the codebase Augie Fackler <augie@google.com> parents: 42992 diff changeset	1450 dist,
2372284d9457 formatting: blacken the codebase Augie Fackler <augie@google.com> parents: 42992 diff changeset	1451 deltalen,
2372284d9457 formatting: blacken the codebase Augie Fackler <augie@google.com> parents: 42992 diff changeset	1452 data,
2372284d9457 formatting: blacken the codebase Augie Fackler <augie@google.com> parents: 42992 diff changeset	1453 deltabase,
2372284d9457 formatting: blacken the codebase Augie Fackler <augie@google.com> parents: 42992 diff changeset	1454 chainbase,
2372284d9457 formatting: blacken the codebase Augie Fackler <augie@google.com> parents: 42992 diff changeset	1455 chainlen,
2372284d9457 formatting: blacken the codebase Augie Fackler <augie@google.com> parents: 42992 diff changeset	1456 compresseddeltalen,
2372284d9457 formatting: blacken the codebase Augie Fackler <augie@google.com> parents: 42992 diff changeset	1457 snapshotdepth,
2372284d9457 formatting: blacken the codebase Augie Fackler <augie@google.com> parents: 42992 diff changeset	1458 )
39333 5d343a24bff5 revlogdeltas: always return a delta info object in finddeltainfo Boris Feld <boris.feld@octobus.net> parents: 39332 diff changeset	1459
51017 509f0f7fc89e delta-computer: stop explicitly taking file handle Pierre-Yves David <pierre-yves.david@octobus.net> parents: 50926 diff changeset	1460 def finddeltainfo(self, revinfo, excluded_bases=None, target_rev=None):
35738 f90f6fd130c1 revlog: group delta computation methods under _deltacomputer object Paul Morelle <paul.morelle@octobus.net> parents: 35737 diff changeset	1461 """Find an acceptable delta against a candidate revision
f90f6fd130c1 revlog: group delta computation methods under _deltacomputer object Paul Morelle <paul.morelle@octobus.net> parents: 35737 diff changeset	1462
f90f6fd130c1 revlog: group delta computation methods under _deltacomputer object Paul Morelle <paul.morelle@octobus.net> parents: 35737 diff changeset	1463 revinfo: information about the revision (instance of _revisioninfo)
f90f6fd130c1 revlog: group delta computation methods under _deltacomputer object Paul Morelle <paul.morelle@octobus.net> parents: 35737 diff changeset	1464
f90f6fd130c1 revlog: group delta computation methods under _deltacomputer object Paul Morelle <paul.morelle@octobus.net> parents: 35737 diff changeset	1465 Returns the first acceptable candidate revision, as ordered by
39334 507f5b1dd7c8 revlogdeltas: extract _getcandidaterevs in a function Boris Feld <boris.feld@octobus.net> parents: 39333 diff changeset	1466 _candidategroups
39333 5d343a24bff5 revlogdeltas: always return a delta info object in finddeltainfo Boris Feld <boris.feld@octobus.net> parents: 39332 diff changeset	1467
5d343a24bff5 revlogdeltas: always return a delta info object in finddeltainfo Boris Feld <boris.feld@octobus.net> parents: 39332 diff changeset	1468 If no suitable deltabase is found, we return delta info for a full
5d343a24bff5 revlogdeltas: always return a delta info object in finddeltainfo Boris Feld <boris.feld@octobus.net> parents: 39332 diff changeset	1469 snapshot.
47401 1efe3cdef53a revlog: add a ways to blacklist some revision when searching for a delta Pierre-Yves David <pierre-yves.david@octobus.net> parents: 47399 diff changeset	1470
1efe3cdef53a revlog: add a ways to blacklist some revision when searching for a delta Pierre-Yves David <pierre-yves.david@octobus.net> parents: 47399 diff changeset	1471 `excluded_bases` is an optional set of revision that cannot be used as
1efe3cdef53a revlog: add a ways to blacklist some revision when searching for a delta Pierre-Yves David <pierre-yves.david@octobus.net> parents: 47399 diff changeset	1472 a delta base. Use this to recompute delta suitable in censor or strip
1efe3cdef53a revlog: add a ways to blacklist some revision when searching for a delta Pierre-Yves David <pierre-yves.david@octobus.net> parents: 47399 diff changeset	1473 context.
35738 f90f6fd130c1 revlog: group delta computation methods under _deltacomputer object Paul Morelle <paul.morelle@octobus.net> parents: 35737 diff changeset	1474 """
47456 93f4e183b3f5 deltas: at a `target_rev` parameter to finddeltainfo Pierre-Yves David <pierre-yves.david@octobus.net> parents: 47452 diff changeset	1475 if target_rev is None:
47457 f8330a3fc39f censor: implement censoring for revlogv2 Pierre-Yves David <pierre-yves.david@octobus.net> parents: 47456 diff changeset	1476 target_rev = len(self.revlog)
47456 93f4e183b3f5 deltas: at a `target_rev` parameter to finddeltainfo Pierre-Yves David <pierre-yves.david@octobus.net> parents: 47452 diff changeset	1477
50650 f930b1b1d671 delta-find: initialize the debug information much sooner (when possible) Pierre-Yves David <pierre-yves.david@octobus.net> parents: 50649 diff changeset	1478 gather_debug = self._gather_debug
f930b1b1d671 delta-find: initialize the debug information much sooner (when possible) Pierre-Yves David <pierre-yves.david@octobus.net> parents: 50649 diff changeset	1479 cachedelta = revinfo.cachedelta
f930b1b1d671 delta-find: initialize the debug information much sooner (when possible) Pierre-Yves David <pierre-yves.david@octobus.net> parents: 50649 diff changeset	1480 revlog = self.revlog
50654 e77ca247b85b delta-find: fix pulled-delta-reuse-policy=forced behavior Pierre-Yves David <pierre-yves.david@octobus.net> parents: 50652 diff changeset	1481 p1r = p2r = None
50650 f930b1b1d671 delta-find: initialize the debug information much sooner (when possible) Pierre-Yves David <pierre-yves.david@octobus.net> parents: 50649 diff changeset	1482
50654 e77ca247b85b delta-find: fix pulled-delta-reuse-policy=forced behavior Pierre-Yves David <pierre-yves.david@octobus.net> parents: 50652 diff changeset	1483 if excluded_bases is None:
e77ca247b85b delta-find: fix pulled-delta-reuse-policy=forced behavior Pierre-Yves David <pierre-yves.david@octobus.net> parents: 50652 diff changeset	1484 excluded_bases = set()
50650 f930b1b1d671 delta-find: initialize the debug information much sooner (when possible) Pierre-Yves David <pierre-yves.david@octobus.net> parents: 50649 diff changeset	1485
f930b1b1d671 delta-find: initialize the debug information much sooner (when possible) Pierre-Yves David <pierre-yves.david@octobus.net> parents: 50649 diff changeset	1486 if gather_debug:
f930b1b1d671 delta-find: initialize the debug information much sooner (when possible) Pierre-Yves David <pierre-yves.david@octobus.net> parents: 50649 diff changeset	1487 start = util.timer()
f930b1b1d671 delta-find: initialize the debug information much sooner (when possible) Pierre-Yves David <pierre-yves.david@octobus.net> parents: 50649 diff changeset	1488 dbg = self._one_dbg_data()
f930b1b1d671 delta-find: initialize the debug information much sooner (when possible) Pierre-Yves David <pierre-yves.david@octobus.net> parents: 50649 diff changeset	1489 dbg['revision'] = target_rev
f930b1b1d671 delta-find: initialize the debug information much sooner (when possible) Pierre-Yves David <pierre-yves.david@octobus.net> parents: 50649 diff changeset	1490 p1r = revlog.rev(revinfo.p1)
f930b1b1d671 delta-find: initialize the debug information much sooner (when possible) Pierre-Yves David <pierre-yves.david@octobus.net> parents: 50649 diff changeset	1491 p2r = revlog.rev(revinfo.p2)
f930b1b1d671 delta-find: initialize the debug information much sooner (when possible) Pierre-Yves David <pierre-yves.david@octobus.net> parents: 50649 diff changeset	1492 if p1r != nullrev:
f930b1b1d671 delta-find: initialize the debug information much sooner (when possible) Pierre-Yves David <pierre-yves.david@octobus.net> parents: 50649 diff changeset	1493 p1_chain_len = revlog._chaininfo(p1r)[0]
f930b1b1d671 delta-find: initialize the debug information much sooner (when possible) Pierre-Yves David <pierre-yves.david@octobus.net> parents: 50649 diff changeset	1494 else:
f930b1b1d671 delta-find: initialize the debug information much sooner (when possible) Pierre-Yves David <pierre-yves.david@octobus.net> parents: 50649 diff changeset	1495 p1_chain_len = -1
f930b1b1d671 delta-find: initialize the debug information much sooner (when possible) Pierre-Yves David <pierre-yves.david@octobus.net> parents: 50649 diff changeset	1496 if p2r != nullrev:
f930b1b1d671 delta-find: initialize the debug information much sooner (when possible) Pierre-Yves David <pierre-yves.david@octobus.net> parents: 50649 diff changeset	1497 p2_chain_len = revlog._chaininfo(p2r)[0]
f930b1b1d671 delta-find: initialize the debug information much sooner (when possible) Pierre-Yves David <pierre-yves.david@octobus.net> parents: 50649 diff changeset	1498 else:
f930b1b1d671 delta-find: initialize the debug information much sooner (when possible) Pierre-Yves David <pierre-yves.david@octobus.net> parents: 50649 diff changeset	1499 p2_chain_len = -1
f930b1b1d671 delta-find: initialize the debug information much sooner (when possible) Pierre-Yves David <pierre-yves.david@octobus.net> parents: 50649 diff changeset	1500 dbg['p1-chain-len'] = p1_chain_len
f930b1b1d671 delta-find: initialize the debug information much sooner (when possible) Pierre-Yves David <pierre-yves.david@octobus.net> parents: 50649 diff changeset	1501 dbg['p2-chain-len'] = p2_chain_len
f930b1b1d671 delta-find: initialize the debug information much sooner (when possible) Pierre-Yves David <pierre-yves.david@octobus.net> parents: 50649 diff changeset	1502
50651 bfb6404089a5 delta-find: gather the condition to blindly use a full snapshot together Pierre-Yves David <pierre-yves.david@octobus.net> parents: 50650 diff changeset	1503 # 1) if the revision is empty, no amount of delta can beat it
bfb6404089a5 delta-find: gather the condition to blindly use a full snapshot together Pierre-Yves David <pierre-yves.david@octobus.net> parents: 50650 diff changeset	1504 #
bfb6404089a5 delta-find: gather the condition to blindly use a full snapshot together Pierre-Yves David <pierre-yves.david@octobus.net> parents: 50650 diff changeset	1505 # 2) no delta for flag processor revision (see "candelta" for why)
bfb6404089a5 delta-find: gather the condition to blindly use a full snapshot together Pierre-Yves David <pierre-yves.david@octobus.net> parents: 50650 diff changeset	1506 # not calling candelta since only one revision needs test, also to
bfb6404089a5 delta-find: gather the condition to blindly use a full snapshot together Pierre-Yves David <pierre-yves.david@octobus.net> parents: 50650 diff changeset	1507 # avoid overhead fetching flags again.
bfb6404089a5 delta-find: gather the condition to blindly use a full snapshot together Pierre-Yves David <pierre-yves.david@octobus.net> parents: 50650 diff changeset	1508 if not revinfo.textlen or revinfo.flags & REVIDX_RAWTEXT_CHANGING_FLAGS:
51017 509f0f7fc89e delta-computer: stop explicitly taking file handle Pierre-Yves David <pierre-yves.david@octobus.net> parents: 50926 diff changeset	1509 deltainfo = self._fullsnapshotinfo(revinfo, target_rev)
50652 485c9410b75a deltafind: issue debug information when we fast-path rivial case too Pierre-Yves David <pierre-yves.david@octobus.net> parents: 50651 diff changeset	1510 if gather_debug:
485c9410b75a deltafind: issue debug information when we fast-path rivial case too Pierre-Yves David <pierre-yves.david@octobus.net> parents: 50651 diff changeset	1511 end = util.timer()
485c9410b75a deltafind: issue debug information when we fast-path rivial case too Pierre-Yves David <pierre-yves.david@octobus.net> parents: 50651 diff changeset	1512 dbg['duration'] = end - start
485c9410b75a deltafind: issue debug information when we fast-path rivial case too Pierre-Yves David <pierre-yves.david@octobus.net> parents: 50651 diff changeset	1513 dbg[
485c9410b75a deltafind: issue debug information when we fast-path rivial case too Pierre-Yves David <pierre-yves.david@octobus.net> parents: 50651 diff changeset	1514 'delta-base'
485c9410b75a deltafind: issue debug information when we fast-path rivial case too Pierre-Yves David <pierre-yves.david@octobus.net> parents: 50651 diff changeset	1515 ] = deltainfo.base # pytype: disable=attribute-error
485c9410b75a deltafind: issue debug information when we fast-path rivial case too Pierre-Yves David <pierre-yves.david@octobus.net> parents: 50651 diff changeset	1516 dbg['search_round_count'] = 0
50654 e77ca247b85b delta-find: fix pulled-delta-reuse-policy=forced behavior Pierre-Yves David <pierre-yves.david@octobus.net> parents: 50652 diff changeset	1517 dbg['using-cached-base'] = False
50652 485c9410b75a deltafind: issue debug information when we fast-path rivial case too Pierre-Yves David <pierre-yves.david@octobus.net> parents: 50651 diff changeset	1518 dbg['delta_try_count'] = 0
485c9410b75a deltafind: issue debug information when we fast-path rivial case too Pierre-Yves David <pierre-yves.david@octobus.net> parents: 50651 diff changeset	1519 dbg['type'] = b"full"
485c9410b75a deltafind: issue debug information when we fast-path rivial case too Pierre-Yves David <pierre-yves.david@octobus.net> parents: 50651 diff changeset	1520 dbg['snapshot-depth'] = 0
485c9410b75a deltafind: issue debug information when we fast-path rivial case too Pierre-Yves David <pierre-yves.david@octobus.net> parents: 50651 diff changeset	1521 self._dbg_process_data(dbg)
485c9410b75a deltafind: issue debug information when we fast-path rivial case too Pierre-Yves David <pierre-yves.david@octobus.net> parents: 50651 diff changeset	1522 return deltainfo
39085 dbb3e9e44fce revlog: do not search for delta for empty content Boris Feld <boris.feld@octobus.net> parents: 39084 diff changeset	1523
50654 e77ca247b85b delta-find: fix pulled-delta-reuse-policy=forced behavior Pierre-Yves David <pierre-yves.david@octobus.net> parents: 50652 diff changeset	1524 deltainfo = None
e77ca247b85b delta-find: fix pulled-delta-reuse-policy=forced behavior Pierre-Yves David <pierre-yves.david@octobus.net> parents: 50652 diff changeset	1525
e77ca247b85b delta-find: fix pulled-delta-reuse-policy=forced behavior Pierre-Yves David <pierre-yves.david@octobus.net> parents: 50652 diff changeset	1526 # If this source delta are to be forcibly reuse, let us comply early.
e77ca247b85b delta-find: fix pulled-delta-reuse-policy=forced behavior Pierre-Yves David <pierre-yves.david@octobus.net> parents: 50652 diff changeset	1527 if (
51043 f636103c4d67 revlog: remove legacy usage of `_generaldelta` Pierre-Yves David <pierre-yves.david@octobus.net> parents: 51042 diff changeset	1528 revlog.delta_config.general_delta
50654 e77ca247b85b delta-find: fix pulled-delta-reuse-policy=forced behavior Pierre-Yves David <pierre-yves.david@octobus.net> parents: 50652 diff changeset	1529 and revinfo.cachedelta is not None
e77ca247b85b delta-find: fix pulled-delta-reuse-policy=forced behavior Pierre-Yves David <pierre-yves.david@octobus.net> parents: 50652 diff changeset	1530 and revinfo.cachedelta[2] == DELTA_BASE_REUSE_FORCE
e77ca247b85b delta-find: fix pulled-delta-reuse-policy=forced behavior Pierre-Yves David <pierre-yves.david@octobus.net> parents: 50652 diff changeset	1531 ):
e77ca247b85b delta-find: fix pulled-delta-reuse-policy=forced behavior Pierre-Yves David <pierre-yves.david@octobus.net> parents: 50652 diff changeset	1532 base = revinfo.cachedelta[0]
e77ca247b85b delta-find: fix pulled-delta-reuse-policy=forced behavior Pierre-Yves David <pierre-yves.david@octobus.net> parents: 50652 diff changeset	1533 if base == nullrev:
e77ca247b85b delta-find: fix pulled-delta-reuse-policy=forced behavior Pierre-Yves David <pierre-yves.david@octobus.net> parents: 50652 diff changeset	1534 dbg_type = b"full"
51017 509f0f7fc89e delta-computer: stop explicitly taking file handle Pierre-Yves David <pierre-yves.david@octobus.net> parents: 50926 diff changeset	1535 deltainfo = self._fullsnapshotinfo(revinfo, target_rev)
50654 e77ca247b85b delta-find: fix pulled-delta-reuse-policy=forced behavior Pierre-Yves David <pierre-yves.david@octobus.net> parents: 50652 diff changeset	1536 if gather_debug:
e77ca247b85b delta-find: fix pulled-delta-reuse-policy=forced behavior Pierre-Yves David <pierre-yves.david@octobus.net> parents: 50652 diff changeset	1537 snapshotdepth = 0
e77ca247b85b delta-find: fix pulled-delta-reuse-policy=forced behavior Pierre-Yves David <pierre-yves.david@octobus.net> parents: 50652 diff changeset	1538 elif base not in excluded_bases:
e77ca247b85b delta-find: fix pulled-delta-reuse-policy=forced behavior Pierre-Yves David <pierre-yves.david@octobus.net> parents: 50652 diff changeset	1539 delta = revinfo.cachedelta[1]
e77ca247b85b delta-find: fix pulled-delta-reuse-policy=forced behavior Pierre-Yves David <pierre-yves.david@octobus.net> parents: 50652 diff changeset	1540 header, data = revlog.compress(delta)
e77ca247b85b delta-find: fix pulled-delta-reuse-policy=forced behavior Pierre-Yves David <pierre-yves.david@octobus.net> parents: 50652 diff changeset	1541 deltalen = len(header) + len(data)
e77ca247b85b delta-find: fix pulled-delta-reuse-policy=forced behavior Pierre-Yves David <pierre-yves.david@octobus.net> parents: 50652 diff changeset	1542 if gather_debug:
e77ca247b85b delta-find: fix pulled-delta-reuse-policy=forced behavior Pierre-Yves David <pierre-yves.david@octobus.net> parents: 50652 diff changeset	1543 offset = revlog.end(len(revlog) - 1)
e77ca247b85b delta-find: fix pulled-delta-reuse-policy=forced behavior Pierre-Yves David <pierre-yves.david@octobus.net> parents: 50652 diff changeset	1544 chainbase = revlog.chainbase(base)
e77ca247b85b delta-find: fix pulled-delta-reuse-policy=forced behavior Pierre-Yves David <pierre-yves.david@octobus.net> parents: 50652 diff changeset	1545 distance = deltalen + offset - revlog.start(chainbase)
e77ca247b85b delta-find: fix pulled-delta-reuse-policy=forced behavior Pierre-Yves David <pierre-yves.david@octobus.net> parents: 50652 diff changeset	1546 chainlen, compresseddeltalen = revlog._chaininfo(base)
e77ca247b85b delta-find: fix pulled-delta-reuse-policy=forced behavior Pierre-Yves David <pierre-yves.david@octobus.net> parents: 50652 diff changeset	1547 chainlen += 1
e77ca247b85b delta-find: fix pulled-delta-reuse-policy=forced behavior Pierre-Yves David <pierre-yves.david@octobus.net> parents: 50652 diff changeset	1548 compresseddeltalen += deltalen
e77ca247b85b delta-find: fix pulled-delta-reuse-policy=forced behavior Pierre-Yves David <pierre-yves.david@octobus.net> parents: 50652 diff changeset	1549 if base == p1r or base == p2r:
e77ca247b85b delta-find: fix pulled-delta-reuse-policy=forced behavior Pierre-Yves David <pierre-yves.david@octobus.net> parents: 50652 diff changeset	1550 dbg_type = b"delta"
e77ca247b85b delta-find: fix pulled-delta-reuse-policy=forced behavior Pierre-Yves David <pierre-yves.david@octobus.net> parents: 50652 diff changeset	1551 snapshotdepth = None
e77ca247b85b delta-find: fix pulled-delta-reuse-policy=forced behavior Pierre-Yves David <pierre-yves.david@octobus.net> parents: 50652 diff changeset	1552 elif not revlog.issnapshot(base):
e77ca247b85b delta-find: fix pulled-delta-reuse-policy=forced behavior Pierre-Yves David <pierre-yves.david@octobus.net> parents: 50652 diff changeset	1553 snapshotdepth = None
e77ca247b85b delta-find: fix pulled-delta-reuse-policy=forced behavior Pierre-Yves David <pierre-yves.david@octobus.net> parents: 50652 diff changeset	1554 else:
e77ca247b85b delta-find: fix pulled-delta-reuse-policy=forced behavior Pierre-Yves David <pierre-yves.david@octobus.net> parents: 50652 diff changeset	1555 dbg_type = b"snapshot"
e77ca247b85b delta-find: fix pulled-delta-reuse-policy=forced behavior Pierre-Yves David <pierre-yves.david@octobus.net> parents: 50652 diff changeset	1556 snapshotdepth = revlog.snapshotdepth(base) + 1
e77ca247b85b delta-find: fix pulled-delta-reuse-policy=forced behavior Pierre-Yves David <pierre-yves.david@octobus.net> parents: 50652 diff changeset	1557 else:
e77ca247b85b delta-find: fix pulled-delta-reuse-policy=forced behavior Pierre-Yves David <pierre-yves.david@octobus.net> parents: 50652 diff changeset	1558 distance = None
e77ca247b85b delta-find: fix pulled-delta-reuse-policy=forced behavior Pierre-Yves David <pierre-yves.david@octobus.net> parents: 50652 diff changeset	1559 chainbase = None
e77ca247b85b delta-find: fix pulled-delta-reuse-policy=forced behavior Pierre-Yves David <pierre-yves.david@octobus.net> parents: 50652 diff changeset	1560 chainlen = None
e77ca247b85b delta-find: fix pulled-delta-reuse-policy=forced behavior Pierre-Yves David <pierre-yves.david@octobus.net> parents: 50652 diff changeset	1561 compresseddeltalen = None
e77ca247b85b delta-find: fix pulled-delta-reuse-policy=forced behavior Pierre-Yves David <pierre-yves.david@octobus.net> parents: 50652 diff changeset	1562 snapshotdepth = None
e77ca247b85b delta-find: fix pulled-delta-reuse-policy=forced behavior Pierre-Yves David <pierre-yves.david@octobus.net> parents: 50652 diff changeset	1563 deltainfo = _deltainfo(
e77ca247b85b delta-find: fix pulled-delta-reuse-policy=forced behavior Pierre-Yves David <pierre-yves.david@octobus.net> parents: 50652 diff changeset	1564 distance=distance,
e77ca247b85b delta-find: fix pulled-delta-reuse-policy=forced behavior Pierre-Yves David <pierre-yves.david@octobus.net> parents: 50652 diff changeset	1565 deltalen=deltalen,
e77ca247b85b delta-find: fix pulled-delta-reuse-policy=forced behavior Pierre-Yves David <pierre-yves.david@octobus.net> parents: 50652 diff changeset	1566 data=(header, data),
e77ca247b85b delta-find: fix pulled-delta-reuse-policy=forced behavior Pierre-Yves David <pierre-yves.david@octobus.net> parents: 50652 diff changeset	1567 base=base,
e77ca247b85b delta-find: fix pulled-delta-reuse-policy=forced behavior Pierre-Yves David <pierre-yves.david@octobus.net> parents: 50652 diff changeset	1568 chainbase=chainbase,
e77ca247b85b delta-find: fix pulled-delta-reuse-policy=forced behavior Pierre-Yves David <pierre-yves.david@octobus.net> parents: 50652 diff changeset	1569 chainlen=chainlen,
e77ca247b85b delta-find: fix pulled-delta-reuse-policy=forced behavior Pierre-Yves David <pierre-yves.david@octobus.net> parents: 50652 diff changeset	1570 compresseddeltalen=compresseddeltalen,
e77ca247b85b delta-find: fix pulled-delta-reuse-policy=forced behavior Pierre-Yves David <pierre-yves.david@octobus.net> parents: 50652 diff changeset	1571 snapshotdepth=snapshotdepth,
e77ca247b85b delta-find: fix pulled-delta-reuse-policy=forced behavior Pierre-Yves David <pierre-yves.david@octobus.net> parents: 50652 diff changeset	1572 )
e77ca247b85b delta-find: fix pulled-delta-reuse-policy=forced behavior Pierre-Yves David <pierre-yves.david@octobus.net> parents: 50652 diff changeset	1573
e77ca247b85b delta-find: fix pulled-delta-reuse-policy=forced behavior Pierre-Yves David <pierre-yves.david@octobus.net> parents: 50652 diff changeset	1574 if deltainfo is not None:
e77ca247b85b delta-find: fix pulled-delta-reuse-policy=forced behavior Pierre-Yves David <pierre-yves.david@octobus.net> parents: 50652 diff changeset	1575 if gather_debug:
e77ca247b85b delta-find: fix pulled-delta-reuse-policy=forced behavior Pierre-Yves David <pierre-yves.david@octobus.net> parents: 50652 diff changeset	1576 end = util.timer()
e77ca247b85b delta-find: fix pulled-delta-reuse-policy=forced behavior Pierre-Yves David <pierre-yves.david@octobus.net> parents: 50652 diff changeset	1577 dbg['duration'] = end - start
e77ca247b85b delta-find: fix pulled-delta-reuse-policy=forced behavior Pierre-Yves David <pierre-yves.david@octobus.net> parents: 50652 diff changeset	1578 dbg[
e77ca247b85b delta-find: fix pulled-delta-reuse-policy=forced behavior Pierre-Yves David <pierre-yves.david@octobus.net> parents: 50652 diff changeset	1579 'delta-base'
e77ca247b85b delta-find: fix pulled-delta-reuse-policy=forced behavior Pierre-Yves David <pierre-yves.david@octobus.net> parents: 50652 diff changeset	1580 ] = deltainfo.base # pytype: disable=attribute-error
e77ca247b85b delta-find: fix pulled-delta-reuse-policy=forced behavior Pierre-Yves David <pierre-yves.david@octobus.net> parents: 50652 diff changeset	1581 dbg['search_round_count'] = 0
e77ca247b85b delta-find: fix pulled-delta-reuse-policy=forced behavior Pierre-Yves David <pierre-yves.david@octobus.net> parents: 50652 diff changeset	1582 dbg['using-cached-base'] = True
e77ca247b85b delta-find: fix pulled-delta-reuse-policy=forced behavior Pierre-Yves David <pierre-yves.david@octobus.net> parents: 50652 diff changeset	1583 dbg['delta_try_count'] = 0
e77ca247b85b delta-find: fix pulled-delta-reuse-policy=forced behavior Pierre-Yves David <pierre-yves.david@octobus.net> parents: 50652 diff changeset	1584 dbg['type'] = b"full"
e77ca247b85b delta-find: fix pulled-delta-reuse-policy=forced behavior Pierre-Yves David <pierre-yves.david@octobus.net> parents: 50652 diff changeset	1585 if snapshotdepth is None:
51330 0d92d62ecde0 delta-find: use "-1" as depth snapshot-dept for non snapshot in debug Pierre-Yves David <pierre-yves.david@octobus.net> parents: 51329 diff changeset	1586 dbg['snapshot-depth'] = -1
50654 e77ca247b85b delta-find: fix pulled-delta-reuse-policy=forced behavior Pierre-Yves David <pierre-yves.david@octobus.net> parents: 50652 diff changeset	1587 else:
e77ca247b85b delta-find: fix pulled-delta-reuse-policy=forced behavior Pierre-Yves David <pierre-yves.david@octobus.net> parents: 50652 diff changeset	1588 dbg['snapshot-depth'] = snapshotdepth
e77ca247b85b delta-find: fix pulled-delta-reuse-policy=forced behavior Pierre-Yves David <pierre-yves.david@octobus.net> parents: 50652 diff changeset	1589 self._dbg_process_data(dbg)
e77ca247b85b delta-find: fix pulled-delta-reuse-policy=forced behavior Pierre-Yves David <pierre-yves.david@octobus.net> parents: 50652 diff changeset	1590 return deltainfo
47401 1efe3cdef53a revlog: add a ways to blacklist some revision when searching for a delta Pierre-Yves David <pierre-yves.david@octobus.net> parents: 47399 diff changeset	1591
49226 e6b7c6fbeb48 deltas: add code to display information about the result of `finddeltainfo` Pierre-Yves David <pierre-yves.david@octobus.net> parents: 48946 diff changeset	1592 # count the number of different delta we tried (for debug purpose)
e6b7c6fbeb48 deltas: add code to display information about the result of `finddeltainfo` Pierre-Yves David <pierre-yves.david@octobus.net> parents: 48946 diff changeset	1593 dbg_try_count = 0
e6b7c6fbeb48 deltas: add code to display information about the result of `finddeltainfo` Pierre-Yves David <pierre-yves.david@octobus.net> parents: 48946 diff changeset	1594 # count the number of "search round" we did. (for debug purpose)
e6b7c6fbeb48 deltas: add code to display information about the result of `finddeltainfo` Pierre-Yves David <pierre-yves.david@octobus.net> parents: 48946 diff changeset	1595 dbg_try_rounds = 0
e6b7c6fbeb48 deltas: add code to display information about the result of `finddeltainfo` Pierre-Yves David <pierre-yves.david@octobus.net> parents: 48946 diff changeset	1596 dbg_type = b'unknown'
e6b7c6fbeb48 deltas: add code to display information about the result of `finddeltainfo` Pierre-Yves David <pierre-yves.david@octobus.net> parents: 48946 diff changeset	1597
50650 f930b1b1d671 delta-find: initialize the debug information much sooner (when possible) Pierre-Yves David <pierre-yves.david@octobus.net> parents: 50649 diff changeset	1598 if p1r is None:
f930b1b1d671 delta-find: initialize the debug information much sooner (when possible) Pierre-Yves David <pierre-yves.david@octobus.net> parents: 50649 diff changeset	1599 p1r = revlog.rev(revinfo.p1)
f930b1b1d671 delta-find: initialize the debug information much sooner (when possible) Pierre-Yves David <pierre-yves.david@octobus.net> parents: 50649 diff changeset	1600 p2r = revlog.rev(revinfo.p2)
49226 e6b7c6fbeb48 deltas: add code to display information about the result of `finddeltainfo` Pierre-Yves David <pierre-yves.david@octobus.net> parents: 48946 diff changeset	1601
50646 c84cc0ac77e4 delta-fine: use the `_debug_search` attribute directly Pierre-Yves David <pierre-yves.david@octobus.net> parents: 50645 diff changeset	1602 if self._debug_search:
49228 b909dd35d9ab deltas: add a debug-delta-find command to analyse delta search Pierre-Yves David <pierre-yves.david@octobus.net> parents: 49226 diff changeset	1603 msg = b"DBG-DELTAS-SEARCH: SEARCH rev=%d\n"
b909dd35d9ab deltas: add a debug-delta-find command to analyse delta search Pierre-Yves David <pierre-yves.david@octobus.net> parents: 49226 diff changeset	1604 msg %= target_rev
b909dd35d9ab deltas: add a debug-delta-find command to analyse delta search Pierre-Yves David <pierre-yves.david@octobus.net> parents: 49226 diff changeset	1605 self._write_debug(msg)
49226 e6b7c6fbeb48 deltas: add code to display information about the result of `finddeltainfo` Pierre-Yves David <pierre-yves.david@octobus.net> parents: 48946 diff changeset	1606
51344 555826073625 delta-find: introduce and use specialized _DeltaSearch class Pierre-Yves David <pierre-yves.david@octobus.net> parents: 51343 diff changeset	1607 # should we try to build a delta?
555826073625 delta-find: introduce and use specialized _DeltaSearch class Pierre-Yves David <pierre-yves.david@octobus.net> parents: 51343 diff changeset	1608 if not (len(self.revlog) and self.revlog._storedeltachains):
555826073625 delta-find: introduce and use specialized _DeltaSearch class Pierre-Yves David <pierre-yves.david@octobus.net> parents: 51343 diff changeset	1609 search_cls = _NoDeltaSearch
555826073625 delta-find: introduce and use specialized _DeltaSearch class Pierre-Yves David <pierre-yves.david@octobus.net> parents: 51343 diff changeset	1610 elif not self.revlog.delta_config.general_delta:
555826073625 delta-find: introduce and use specialized _DeltaSearch class Pierre-Yves David <pierre-yves.david@octobus.net> parents: 51343 diff changeset	1611 # before general delta, there is only one possible delta base
555826073625 delta-find: introduce and use specialized _DeltaSearch class Pierre-Yves David <pierre-yves.david@octobus.net> parents: 51343 diff changeset	1612 search_cls = _PrevDeltaSearch
555826073625 delta-find: introduce and use specialized _DeltaSearch class Pierre-Yves David <pierre-yves.david@octobus.net> parents: 51343 diff changeset	1613 else:
555826073625 delta-find: introduce and use specialized _DeltaSearch class Pierre-Yves David <pierre-yves.david@octobus.net> parents: 51343 diff changeset	1614 search_cls = _DeltaSearch
555826073625 delta-find: introduce and use specialized _DeltaSearch class Pierre-Yves David <pierre-yves.david@octobus.net> parents: 51343 diff changeset	1615
555826073625 delta-find: introduce and use specialized _DeltaSearch class Pierre-Yves David <pierre-yves.david@octobus.net> parents: 51343 diff changeset	1616 search = search_cls(
49611 2afee2176775 delta-find: expand a function definition and call before extendin it Pierre-Yves David <pierre-yves.david@octobus.net> parents: 49610 diff changeset	1617 self.revlog,
51327 49401b7dec0c delta-find: feed revinfo to _DeltaSearch Pierre-Yves David <pierre-yves.david@octobus.net> parents: 51325 diff changeset	1618 revinfo,
49611 2afee2176775 delta-find: expand a function definition and call before extendin it Pierre-Yves David <pierre-yves.david@octobus.net> parents: 49610 diff changeset	1619 p1r,
2afee2176775 delta-find: expand a function definition and call before extendin it Pierre-Yves David <pierre-yves.david@octobus.net> parents: 49610 diff changeset	1620 p2r,
2afee2176775 delta-find: expand a function definition and call before extendin it Pierre-Yves David <pierre-yves.david@octobus.net> parents: 49610 diff changeset	1621 cachedelta,
49612 e706bb41fdb3 delta-find: move pre-filtering with other pre-filtering logic Pierre-Yves David <pierre-yves.david@octobus.net> parents: 49611 diff changeset	1622 excluded_bases,
e706bb41fdb3 delta-find: move pre-filtering with other pre-filtering logic Pierre-Yves David <pierre-yves.david@octobus.net> parents: 49611 diff changeset	1623 target_rev,
49681 c261a628e525 delta-find: use a single snapshot cache when applying a group to an object Pierre-Yves David <pierre-yves.david@octobus.net> parents: 49680 diff changeset	1624 snapshot_cache=self._snapshot_cache,
43076 2372284d9457 formatting: blacken the codebase Augie Fackler <augie@google.com> parents: 42992 diff changeset	1625 )
51317 c82e03b102a6 delta-find: introduce a _DeltaSearch object Pierre-Yves David <pierre-yves.david@octobus.net> parents: 51316 diff changeset	1626
51331 d0d869fccd20 delta-find: move away from the generator API for _DeltaSearch Pierre-Yves David <pierre-yves.david@octobus.net> parents: 51330 diff changeset	1627 while not search.done:
d0d869fccd20 delta-find: move away from the generator API for _DeltaSearch Pierre-Yves David <pierre-yves.david@octobus.net> parents: 51330 diff changeset	1628 current_group = search.current_group
d0d869fccd20 delta-find: move away from the generator API for _DeltaSearch Pierre-Yves David <pierre-yves.david@octobus.net> parents: 51330 diff changeset	1629 # current_group can be `None`, but not is search.done is False
d0d869fccd20 delta-find: move away from the generator API for _DeltaSearch Pierre-Yves David <pierre-yves.david@octobus.net> parents: 51330 diff changeset	1630 # We add this assert to help pytype
d0d869fccd20 delta-find: move away from the generator API for _DeltaSearch Pierre-Yves David <pierre-yves.david@octobus.net> parents: 51330 diff changeset	1631 assert current_group is not None
d0d869fccd20 delta-find: move away from the generator API for _DeltaSearch Pierre-Yves David <pierre-yves.david@octobus.net> parents: 51330 diff changeset	1632 candidaterevs = current_group
49226 e6b7c6fbeb48 deltas: add code to display information about the result of `finddeltainfo` Pierre-Yves David <pierre-yves.david@octobus.net> parents: 48946 diff changeset	1633 dbg_try_rounds += 1
50646 c84cc0ac77e4 delta-fine: use the `_debug_search` attribute directly Pierre-Yves David <pierre-yves.david@octobus.net> parents: 50645 diff changeset	1634 if self._debug_search:
49228 b909dd35d9ab deltas: add a debug-delta-find command to analyse delta search Pierre-Yves David <pierre-yves.david@octobus.net> parents: 49226 diff changeset	1635 prev = None
b909dd35d9ab deltas: add a debug-delta-find command to analyse delta search Pierre-Yves David <pierre-yves.david@octobus.net> parents: 49226 diff changeset	1636 if deltainfo is not None:
b909dd35d9ab deltas: add a debug-delta-find command to analyse delta search Pierre-Yves David <pierre-yves.david@octobus.net> parents: 49226 diff changeset	1637 prev = deltainfo.base
b909dd35d9ab deltas: add a debug-delta-find command to analyse delta search Pierre-Yves David <pierre-yves.david@octobus.net> parents: 49226 diff changeset	1638
49608 78ba41878f2e delta-find: add debug information about reuse of cached data Pierre-Yves David <pierre-yves.david@octobus.net> parents: 49228 diff changeset	1639 if (
78ba41878f2e delta-find: add debug information about reuse of cached data Pierre-Yves David <pierre-yves.david@octobus.net> parents: 49228 diff changeset	1640 cachedelta is not None
78ba41878f2e delta-find: add debug information about reuse of cached data Pierre-Yves David <pierre-yves.david@octobus.net> parents: 49228 diff changeset	1641 and len(candidaterevs) == 1
78ba41878f2e delta-find: add debug information about reuse of cached data Pierre-Yves David <pierre-yves.david@octobus.net> parents: 49228 diff changeset	1642 and cachedelta[0] in candidaterevs
78ba41878f2e delta-find: add debug information about reuse of cached data Pierre-Yves David <pierre-yves.david@octobus.net> parents: 49228 diff changeset	1643 ):
78ba41878f2e delta-find: add debug information about reuse of cached data Pierre-Yves David <pierre-yves.david@octobus.net> parents: 49228 diff changeset	1644 round_type = b"cached-delta"
50649 ebb292ffdf4d delta-find: fix `parents` round detection Pierre-Yves David <pierre-yves.david@octobus.net> parents: 50648 diff changeset	1645 elif p1r in candidaterevs or p2r in candidaterevs:
49228 b909dd35d9ab deltas: add a debug-delta-find command to analyse delta search Pierre-Yves David <pierre-yves.david@octobus.net> parents: 49226 diff changeset	1646 round_type = b"parents"
b909dd35d9ab deltas: add a debug-delta-find command to analyse delta search Pierre-Yves David <pierre-yves.david@octobus.net> parents: 49226 diff changeset	1647 elif prev is not None and all(c < prev for c in candidaterevs):
b909dd35d9ab deltas: add a debug-delta-find command to analyse delta search Pierre-Yves David <pierre-yves.david@octobus.net> parents: 49226 diff changeset	1648 round_type = b"refine-down"
b909dd35d9ab deltas: add a debug-delta-find command to analyse delta search Pierre-Yves David <pierre-yves.david@octobus.net> parents: 49226 diff changeset	1649 elif prev is not None and all(c > prev for c in candidaterevs):
b909dd35d9ab deltas: add a debug-delta-find command to analyse delta search Pierre-Yves David <pierre-yves.david@octobus.net> parents: 49226 diff changeset	1650 round_type = b"refine-up"
b909dd35d9ab deltas: add a debug-delta-find command to analyse delta search Pierre-Yves David <pierre-yves.david@octobus.net> parents: 49226 diff changeset	1651 else:
b909dd35d9ab deltas: add a debug-delta-find command to analyse delta search Pierre-Yves David <pierre-yves.david@octobus.net> parents: 49226 diff changeset	1652 round_type = b"search-down"
b909dd35d9ab deltas: add a debug-delta-find command to analyse delta search Pierre-Yves David <pierre-yves.david@octobus.net> parents: 49226 diff changeset	1653 msg = b"DBG-DELTAS-SEARCH: ROUND #%d - %d candidates - %s\n"
b909dd35d9ab deltas: add a debug-delta-find command to analyse delta search Pierre-Yves David <pierre-yves.david@octobus.net> parents: 49226 diff changeset	1654 msg %= (dbg_try_rounds, len(candidaterevs), round_type)
b909dd35d9ab deltas: add a debug-delta-find command to analyse delta search Pierre-Yves David <pierre-yves.david@octobus.net> parents: 49226 diff changeset	1655 self._write_debug(msg)
35738 f90f6fd130c1 revlog: group delta computation methods under _deltacomputer object Paul Morelle <paul.morelle@octobus.net> parents: 35737 diff changeset	1656 nominateddeltas = []
39498 04b75f3a3f2a snapshot: add refining logic at the findeltainfo level Boris Feld <boris.feld@octobus.net> parents: 39497 diff changeset	1657 if deltainfo is not None:
50646 c84cc0ac77e4 delta-fine: use the `_debug_search` attribute directly Pierre-Yves David <pierre-yves.david@octobus.net> parents: 50645 diff changeset	1658 if self._debug_search:
49228 b909dd35d9ab deltas: add a debug-delta-find command to analyse delta search Pierre-Yves David <pierre-yves.david@octobus.net> parents: 49226 diff changeset	1659 msg = (
b909dd35d9ab deltas: add a debug-delta-find command to analyse delta search Pierre-Yves David <pierre-yves.david@octobus.net> parents: 49226 diff changeset	1660 b"DBG-DELTAS-SEARCH: CONTENDER: rev=%d - length=%d\n"
b909dd35d9ab deltas: add a debug-delta-find command to analyse delta search Pierre-Yves David <pierre-yves.david@octobus.net> parents: 49226 diff changeset	1661 )
b909dd35d9ab deltas: add a debug-delta-find command to analyse delta search Pierre-Yves David <pierre-yves.david@octobus.net> parents: 49226 diff changeset	1662 msg %= (deltainfo.base, deltainfo.deltalen)
b909dd35d9ab deltas: add a debug-delta-find command to analyse delta search Pierre-Yves David <pierre-yves.david@octobus.net> parents: 49226 diff changeset	1663 self._write_debug(msg)
39498 04b75f3a3f2a snapshot: add refining logic at the findeltainfo level Boris Feld <boris.feld@octobus.net> parents: 39497 diff changeset	1664 # if we already found a good delta,
04b75f3a3f2a snapshot: add refining logic at the findeltainfo level Boris Feld <boris.feld@octobus.net> parents: 39497 diff changeset	1665 # challenge it against refined candidates
04b75f3a3f2a snapshot: add refining logic at the findeltainfo level Boris Feld <boris.feld@octobus.net> parents: 39497 diff changeset	1666 nominateddeltas.append(deltainfo)
35738 f90f6fd130c1 revlog: group delta computation methods under _deltacomputer object Paul Morelle <paul.morelle@octobus.net> parents: 35737 diff changeset	1667 for candidaterev in candidaterevs:
50646 c84cc0ac77e4 delta-fine: use the `_debug_search` attribute directly Pierre-Yves David <pierre-yves.david@octobus.net> parents: 50645 diff changeset	1668 if self._debug_search:
49228 b909dd35d9ab deltas: add a debug-delta-find command to analyse delta search Pierre-Yves David <pierre-yves.david@octobus.net> parents: 49226 diff changeset	1669 msg = b"DBG-DELTAS-SEARCH: CANDIDATE: rev=%d\n"
b909dd35d9ab deltas: add a debug-delta-find command to analyse delta search Pierre-Yves David <pierre-yves.david@octobus.net> parents: 49226 diff changeset	1670 msg %= candidaterev
b909dd35d9ab deltas: add a debug-delta-find command to analyse delta search Pierre-Yves David <pierre-yves.david@octobus.net> parents: 49226 diff changeset	1671 self._write_debug(msg)
b909dd35d9ab deltas: add a debug-delta-find command to analyse delta search Pierre-Yves David <pierre-yves.david@octobus.net> parents: 49226 diff changeset	1672 candidate_type = None
50649 ebb292ffdf4d delta-find: fix `parents` round detection Pierre-Yves David <pierre-yves.david@octobus.net> parents: 50648 diff changeset	1673 if candidaterev == p1r:
49228 b909dd35d9ab deltas: add a debug-delta-find command to analyse delta search Pierre-Yves David <pierre-yves.david@octobus.net> parents: 49226 diff changeset	1674 candidate_type = b"p1"
50649 ebb292ffdf4d delta-find: fix `parents` round detection Pierre-Yves David <pierre-yves.david@octobus.net> parents: 50648 diff changeset	1675 elif candidaterev == p2r:
49228 b909dd35d9ab deltas: add a debug-delta-find command to analyse delta search Pierre-Yves David <pierre-yves.david@octobus.net> parents: 49226 diff changeset	1676 candidate_type = b"p2"
b909dd35d9ab deltas: add a debug-delta-find command to analyse delta search Pierre-Yves David <pierre-yves.david@octobus.net> parents: 49226 diff changeset	1677 elif self.revlog.issnapshot(candidaterev):
b909dd35d9ab deltas: add a debug-delta-find command to analyse delta search Pierre-Yves David <pierre-yves.david@octobus.net> parents: 49226 diff changeset	1678 candidate_type = b"snapshot-%d"
b909dd35d9ab deltas: add a debug-delta-find command to analyse delta search Pierre-Yves David <pierre-yves.david@octobus.net> parents: 49226 diff changeset	1679 candidate_type %= self.revlog.snapshotdepth(
b909dd35d9ab deltas: add a debug-delta-find command to analyse delta search Pierre-Yves David <pierre-yves.david@octobus.net> parents: 49226 diff changeset	1680 candidaterev
b909dd35d9ab deltas: add a debug-delta-find command to analyse delta search Pierre-Yves David <pierre-yves.david@octobus.net> parents: 49226 diff changeset	1681 )
b909dd35d9ab deltas: add a debug-delta-find command to analyse delta search Pierre-Yves David <pierre-yves.david@octobus.net> parents: 49226 diff changeset	1682
b909dd35d9ab deltas: add a debug-delta-find command to analyse delta search Pierre-Yves David <pierre-yves.david@octobus.net> parents: 49226 diff changeset	1683 if candidate_type is not None:
b909dd35d9ab deltas: add a debug-delta-find command to analyse delta search Pierre-Yves David <pierre-yves.david@octobus.net> parents: 49226 diff changeset	1684 msg = b"DBG-DELTAS-SEARCH: type=%s\n"
b909dd35d9ab deltas: add a debug-delta-find command to analyse delta search Pierre-Yves David <pierre-yves.david@octobus.net> parents: 49226 diff changeset	1685 msg %= candidate_type
b909dd35d9ab deltas: add a debug-delta-find command to analyse delta search Pierre-Yves David <pierre-yves.david@octobus.net> parents: 49226 diff changeset	1686 self._write_debug(msg)
b909dd35d9ab deltas: add a debug-delta-find command to analyse delta search Pierre-Yves David <pierre-yves.david@octobus.net> parents: 49226 diff changeset	1687 msg = b"DBG-DELTAS-SEARCH: size=%d\n"
b909dd35d9ab deltas: add a debug-delta-find command to analyse delta search Pierre-Yves David <pierre-yves.david@octobus.net> parents: 49226 diff changeset	1688 msg %= self.revlog.length(candidaterev)
b909dd35d9ab deltas: add a debug-delta-find command to analyse delta search Pierre-Yves David <pierre-yves.david@octobus.net> parents: 49226 diff changeset	1689 self._write_debug(msg)
b909dd35d9ab deltas: add a debug-delta-find command to analyse delta search Pierre-Yves David <pierre-yves.david@octobus.net> parents: 49226 diff changeset	1690 msg = b"DBG-DELTAS-SEARCH: base=%d\n"
b909dd35d9ab deltas: add a debug-delta-find command to analyse delta search Pierre-Yves David <pierre-yves.david@octobus.net> parents: 49226 diff changeset	1691 msg %= self.revlog.deltaparent(candidaterev)
b909dd35d9ab deltas: add a debug-delta-find command to analyse delta search Pierre-Yves David <pierre-yves.david@octobus.net> parents: 49226 diff changeset	1692 self._write_debug(msg)
49612 e706bb41fdb3 delta-find: move pre-filtering with other pre-filtering logic Pierre-Yves David <pierre-yves.david@octobus.net> parents: 49611 diff changeset	1693
49226 e6b7c6fbeb48 deltas: add code to display information about the result of `finddeltainfo` Pierre-Yves David <pierre-yves.david@octobus.net> parents: 48946 diff changeset	1694 dbg_try_count += 1
49228 b909dd35d9ab deltas: add a debug-delta-find command to analyse delta search Pierre-Yves David <pierre-yves.david@octobus.net> parents: 49226 diff changeset	1695
50646 c84cc0ac77e4 delta-fine: use the `_debug_search` attribute directly Pierre-Yves David <pierre-yves.david@octobus.net> parents: 50645 diff changeset	1696 if self._debug_search:
49228 b909dd35d9ab deltas: add a debug-delta-find command to analyse delta search Pierre-Yves David <pierre-yves.david@octobus.net> parents: 49226 diff changeset	1697 delta_start = util.timer()
50356 2a6949ab9d23 delta-find: add a simple safeguard to prevent bad non-general-delta Pierre-Yves David <pierre-yves.david@octobus.net> parents: 50355 diff changeset	1698 candidatedelta = self._builddeltainfo(
2a6949ab9d23 delta-find: add a simple safeguard to prevent bad non-general-delta Pierre-Yves David <pierre-yves.david@octobus.net> parents: 50355 diff changeset	1699 revinfo,
2a6949ab9d23 delta-find: add a simple safeguard to prevent bad non-general-delta Pierre-Yves David <pierre-yves.david@octobus.net> parents: 50355 diff changeset	1700 candidaterev,
2a6949ab9d23 delta-find: add a simple safeguard to prevent bad non-general-delta Pierre-Yves David <pierre-yves.david@octobus.net> parents: 50355 diff changeset	1701 target_rev=target_rev,
51347 670e68729aa7 delta-find: stop using heuristic to determine if we are creating a snapshot Pierre-Yves David <pierre-yves.david@octobus.net> parents: 51346 diff changeset	1702 as_snapshot=search.current_stage == _STAGE_SNAPSHOT,
50356 2a6949ab9d23 delta-find: add a simple safeguard to prevent bad non-general-delta Pierre-Yves David <pierre-yves.david@octobus.net> parents: 50355 diff changeset	1703 )
50646 c84cc0ac77e4 delta-fine: use the `_debug_search` attribute directly Pierre-Yves David <pierre-yves.david@octobus.net> parents: 50645 diff changeset	1704 if self._debug_search:
49228 b909dd35d9ab deltas: add a debug-delta-find command to analyse delta search Pierre-Yves David <pierre-yves.david@octobus.net> parents: 49226 diff changeset	1705 delta_end = util.timer()
b909dd35d9ab deltas: add a debug-delta-find command to analyse delta search Pierre-Yves David <pierre-yves.david@octobus.net> parents: 49226 diff changeset	1706 msg = b"DBG-DELTAS-SEARCH: delta-search-time=%f\n"
b909dd35d9ab deltas: add a debug-delta-find command to analyse delta search Pierre-Yves David <pierre-yves.david@octobus.net> parents: 49226 diff changeset	1707 msg %= delta_end - delta_start
b909dd35d9ab deltas: add a debug-delta-find command to analyse delta search Pierre-Yves David <pierre-yves.david@octobus.net> parents: 49226 diff changeset	1708 self._write_debug(msg)
42466 465f2d0df9ae deltas: accept and skip None return for delta info Valentin Gatien-Baron <vgatien-baron@janestreet.com> parents: 42465 diff changeset	1709 if candidatedelta is not None:
51328 7455cae67260 delta-find: move is_good_delta_info on the _DeltaSearch class Pierre-Yves David <pierre-yves.david@octobus.net> parents: 51327 diff changeset	1710 if search.is_good_delta_info(candidatedelta):
50646 c84cc0ac77e4 delta-fine: use the `_debug_search` attribute directly Pierre-Yves David <pierre-yves.david@octobus.net> parents: 50645 diff changeset	1711 if self._debug_search:
49228 b909dd35d9ab deltas: add a debug-delta-find command to analyse delta search Pierre-Yves David <pierre-yves.david@octobus.net> parents: 49226 diff changeset	1712 msg = b"DBG-DELTAS-SEARCH: DELTA: length=%d (GOOD)\n"
b909dd35d9ab deltas: add a debug-delta-find command to analyse delta search Pierre-Yves David <pierre-yves.david@octobus.net> parents: 49226 diff changeset	1713 msg %= candidatedelta.deltalen
b909dd35d9ab deltas: add a debug-delta-find command to analyse delta search Pierre-Yves David <pierre-yves.david@octobus.net> parents: 49226 diff changeset	1714 self._write_debug(msg)
42466 465f2d0df9ae deltas: accept and skip None return for delta info Valentin Gatien-Baron <vgatien-baron@janestreet.com> parents: 42465 diff changeset	1715 nominateddeltas.append(candidatedelta)
50646 c84cc0ac77e4 delta-fine: use the `_debug_search` attribute directly Pierre-Yves David <pierre-yves.david@octobus.net> parents: 50645 diff changeset	1716 elif self._debug_search:
49228 b909dd35d9ab deltas: add a debug-delta-find command to analyse delta search Pierre-Yves David <pierre-yves.david@octobus.net> parents: 49226 diff changeset	1717 msg = b"DBG-DELTAS-SEARCH: DELTA: length=%d (BAD)\n"
b909dd35d9ab deltas: add a debug-delta-find command to analyse delta search Pierre-Yves David <pierre-yves.david@octobus.net> parents: 49226 diff changeset	1718 msg %= candidatedelta.deltalen
b909dd35d9ab deltas: add a debug-delta-find command to analyse delta search Pierre-Yves David <pierre-yves.david@octobus.net> parents: 49226 diff changeset	1719 self._write_debug(msg)
50646 c84cc0ac77e4 delta-fine: use the `_debug_search` attribute directly Pierre-Yves David <pierre-yves.david@octobus.net> parents: 50645 diff changeset	1720 elif self._debug_search:
49228 b909dd35d9ab deltas: add a debug-delta-find command to analyse delta search Pierre-Yves David <pierre-yves.david@octobus.net> parents: 49226 diff changeset	1721 msg = b"DBG-DELTAS-SEARCH: NO-DELTA\n"
b909dd35d9ab deltas: add a debug-delta-find command to analyse delta search Pierre-Yves David <pierre-yves.david@octobus.net> parents: 49226 diff changeset	1722 self._write_debug(msg)
35738 f90f6fd130c1 revlog: group delta computation methods under _deltacomputer object Paul Morelle <paul.morelle@octobus.net> parents: 35737 diff changeset	1723 if nominateddeltas:
f90f6fd130c1 revlog: group delta computation methods under _deltacomputer object Paul Morelle <paul.morelle@octobus.net> parents: 35737 diff changeset	1724 deltainfo = min(nominateddeltas, key=lambda x: x.deltalen)
51331 d0d869fccd20 delta-find: move away from the generator API for _DeltaSearch Pierre-Yves David <pierre-yves.david@octobus.net> parents: 51330 diff changeset	1725 search.next_group(deltainfo)
35738 f90f6fd130c1 revlog: group delta computation methods under _deltacomputer object Paul Morelle <paul.morelle@octobus.net> parents: 35737 diff changeset	1726
39333 5d343a24bff5 revlogdeltas: always return a delta info object in finddeltainfo Boris Feld <boris.feld@octobus.net> parents: 39332 diff changeset	1727 if deltainfo is None:
49226 e6b7c6fbeb48 deltas: add code to display information about the result of `finddeltainfo` Pierre-Yves David <pierre-yves.david@octobus.net> parents: 48946 diff changeset	1728 dbg_type = b"full"
51017 509f0f7fc89e delta-computer: stop explicitly taking file handle Pierre-Yves David <pierre-yves.david@octobus.net> parents: 50926 diff changeset	1729 deltainfo = self._fullsnapshotinfo(revinfo, target_rev)
49228 b909dd35d9ab deltas: add a debug-delta-find command to analyse delta search Pierre-Yves David <pierre-yves.david@octobus.net> parents: 49226 diff changeset	1730 elif deltainfo.snapshotdepth: # pytype: disable=attribute-error
49226 e6b7c6fbeb48 deltas: add code to display information about the result of `finddeltainfo` Pierre-Yves David <pierre-yves.david@octobus.net> parents: 48946 diff changeset	1731 dbg_type = b"snapshot"
e6b7c6fbeb48 deltas: add code to display information about the result of `finddeltainfo` Pierre-Yves David <pierre-yves.david@octobus.net> parents: 48946 diff changeset	1732 else:
e6b7c6fbeb48 deltas: add code to display information about the result of `finddeltainfo` Pierre-Yves David <pierre-yves.david@octobus.net> parents: 48946 diff changeset	1733 dbg_type = b"delta"
e6b7c6fbeb48 deltas: add code to display information about the result of `finddeltainfo` Pierre-Yves David <pierre-yves.david@octobus.net> parents: 48946 diff changeset	1734
49610 35d4c2124073 debug: add an option to display statistic about a unbundling operation Pierre-Yves David <pierre-yves.david@octobus.net> parents: 49608 diff changeset	1735 if gather_debug:
49226 e6b7c6fbeb48 deltas: add code to display information about the result of `finddeltainfo` Pierre-Yves David <pierre-yves.david@octobus.net> parents: 48946 diff changeset	1736 end = util.timer()
49775 d57b966cdeb1 delta-find: properly report full snapshot used from cache as such Pierre-Yves David <pierre-yves.david@octobus.net> parents: 49768 diff changeset	1737 if dbg_type == b'full':
d57b966cdeb1 delta-find: properly report full snapshot used from cache as such Pierre-Yves David <pierre-yves.david@octobus.net> parents: 49768 diff changeset	1738 used_cached = (
d57b966cdeb1 delta-find: properly report full snapshot used from cache as such Pierre-Yves David <pierre-yves.david@octobus.net> parents: 49768 diff changeset	1739 cachedelta is not None
d57b966cdeb1 delta-find: properly report full snapshot used from cache as such Pierre-Yves David <pierre-yves.david@octobus.net> parents: 49768 diff changeset	1740 and dbg_try_rounds == 0
d57b966cdeb1 delta-find: properly report full snapshot used from cache as such Pierre-Yves David <pierre-yves.david@octobus.net> parents: 49768 diff changeset	1741 and dbg_try_count == 0
d57b966cdeb1 delta-find: properly report full snapshot used from cache as such Pierre-Yves David <pierre-yves.david@octobus.net> parents: 49768 diff changeset	1742 and cachedelta[0] == nullrev
d57b966cdeb1 delta-find: properly report full snapshot used from cache as such Pierre-Yves David <pierre-yves.david@octobus.net> parents: 49768 diff changeset	1743 )
d57b966cdeb1 delta-find: properly report full snapshot used from cache as such Pierre-Yves David <pierre-yves.david@octobus.net> parents: 49768 diff changeset	1744 else:
d57b966cdeb1 delta-find: properly report full snapshot used from cache as such Pierre-Yves David <pierre-yves.david@octobus.net> parents: 49768 diff changeset	1745 used_cached = (
d57b966cdeb1 delta-find: properly report full snapshot used from cache as such Pierre-Yves David <pierre-yves.david@octobus.net> parents: 49768 diff changeset	1746 cachedelta is not None
d57b966cdeb1 delta-find: properly report full snapshot used from cache as such Pierre-Yves David <pierre-yves.david@octobus.net> parents: 49768 diff changeset	1747 and dbg_try_rounds == 1
d57b966cdeb1 delta-find: properly report full snapshot used from cache as such Pierre-Yves David <pierre-yves.david@octobus.net> parents: 49768 diff changeset	1748 and dbg_try_count == 1
d57b966cdeb1 delta-find: properly report full snapshot used from cache as such Pierre-Yves David <pierre-yves.david@octobus.net> parents: 49768 diff changeset	1749 and deltainfo.base == cachedelta[0]
d57b966cdeb1 delta-find: properly report full snapshot used from cache as such Pierre-Yves David <pierre-yves.david@octobus.net> parents: 49768 diff changeset	1750 )
50648 d1dc4a03125e delta-find: intrduce a `_one_dbg_data` method Pierre-Yves David <pierre-yves.david@octobus.net> parents: 50647 diff changeset	1751 dbg['duration'] = end - start
d1dc4a03125e delta-find: intrduce a `_one_dbg_data` method Pierre-Yves David <pierre-yves.david@octobus.net> parents: 50647 diff changeset	1752 dbg[
d1dc4a03125e delta-find: intrduce a `_one_dbg_data` method Pierre-Yves David <pierre-yves.david@octobus.net> parents: 50647 diff changeset	1753 'delta-base'
d1dc4a03125e delta-find: intrduce a `_one_dbg_data` method Pierre-Yves David <pierre-yves.david@octobus.net> parents: 50647 diff changeset	1754 ] = deltainfo.base # pytype: disable=attribute-error
d1dc4a03125e delta-find: intrduce a `_one_dbg_data` method Pierre-Yves David <pierre-yves.david@octobus.net> parents: 50647 diff changeset	1755 dbg['search_round_count'] = dbg_try_rounds
d1dc4a03125e delta-find: intrduce a `_one_dbg_data` method Pierre-Yves David <pierre-yves.david@octobus.net> parents: 50647 diff changeset	1756 dbg['using-cached-base'] = used_cached
d1dc4a03125e delta-find: intrduce a `_one_dbg_data` method Pierre-Yves David <pierre-yves.david@octobus.net> parents: 50647 diff changeset	1757 dbg['delta_try_count'] = dbg_try_count
d1dc4a03125e delta-find: intrduce a `_one_dbg_data` method Pierre-Yves David <pierre-yves.david@octobus.net> parents: 50647 diff changeset	1758 dbg['type'] = dbg_type
49228 b909dd35d9ab deltas: add a debug-delta-find command to analyse delta search Pierre-Yves David <pierre-yves.david@octobus.net> parents: 49226 diff changeset	1759 if (
b909dd35d9ab deltas: add a debug-delta-find command to analyse delta search Pierre-Yves David <pierre-yves.david@octobus.net> parents: 49226 diff changeset	1760 deltainfo.snapshotdepth # pytype: disable=attribute-error
b909dd35d9ab deltas: add a debug-delta-find command to analyse delta search Pierre-Yves David <pierre-yves.david@octobus.net> parents: 49226 diff changeset	1761 is not None
b909dd35d9ab deltas: add a debug-delta-find command to analyse delta search Pierre-Yves David <pierre-yves.david@octobus.net> parents: 49226 diff changeset	1762 ):
b909dd35d9ab deltas: add a debug-delta-find command to analyse delta search Pierre-Yves David <pierre-yves.david@octobus.net> parents: 49226 diff changeset	1763 dbg[
b909dd35d9ab deltas: add a debug-delta-find command to analyse delta search Pierre-Yves David <pierre-yves.david@octobus.net> parents: 49226 diff changeset	1764 'snapshot-depth'
b909dd35d9ab deltas: add a debug-delta-find command to analyse delta search Pierre-Yves David <pierre-yves.david@octobus.net> parents: 49226 diff changeset	1765 ] = deltainfo.snapshotdepth # pytype: disable=attribute-error
49226 e6b7c6fbeb48 deltas: add code to display information about the result of `finddeltainfo` Pierre-Yves David <pierre-yves.david@octobus.net> parents: 48946 diff changeset	1766 else:
51330 0d92d62ecde0 delta-find: use "-1" as depth snapshot-dept for non snapshot in debug Pierre-Yves David <pierre-yves.david@octobus.net> parents: 51329 diff changeset	1767 dbg['snapshot-depth'] = -1
50647 4d84b6d52e93 delta-find: move final debug processing in a `_dbg_process_data` method Pierre-Yves David <pierre-yves.david@octobus.net> parents: 50646 diff changeset	1768 self._dbg_process_data(dbg)
4d84b6d52e93 delta-find: move final debug processing in a `_dbg_process_data` method Pierre-Yves David <pierre-yves.david@octobus.net> parents: 50646 diff changeset	1769 return deltainfo
49226 e6b7c6fbeb48 deltas: add code to display information about the result of `finddeltainfo` Pierre-Yves David <pierre-yves.david@octobus.net> parents: 48946 diff changeset	1770
50648 d1dc4a03125e delta-find: intrduce a `_one_dbg_data` method Pierre-Yves David <pierre-yves.david@octobus.net> parents: 50647 diff changeset	1771 def _one_dbg_data(self):
51325 46415df4eb1c delta-find: move filing of some debug data in `_one_dbg_data` Pierre-Yves David <pierre-yves.david@octobus.net> parents: 51324 diff changeset	1772 dbg = {
50648 d1dc4a03125e delta-find: intrduce a `_one_dbg_data` method Pierre-Yves David <pierre-yves.david@octobus.net> parents: 50647 diff changeset	1773 'duration': None,
d1dc4a03125e delta-find: intrduce a `_one_dbg_data` method Pierre-Yves David <pierre-yves.david@octobus.net> parents: 50647 diff changeset	1774 'revision': None,
d1dc4a03125e delta-find: intrduce a `_one_dbg_data` method Pierre-Yves David <pierre-yves.david@octobus.net> parents: 50647 diff changeset	1775 'delta-base': None,
d1dc4a03125e delta-find: intrduce a `_one_dbg_data` method Pierre-Yves David <pierre-yves.david@octobus.net> parents: 50647 diff changeset	1776 'search_round_count': None,
d1dc4a03125e delta-find: intrduce a `_one_dbg_data` method Pierre-Yves David <pierre-yves.david@octobus.net> parents: 50647 diff changeset	1777 'using-cached-base': None,
d1dc4a03125e delta-find: intrduce a `_one_dbg_data` method Pierre-Yves David <pierre-yves.david@octobus.net> parents: 50647 diff changeset	1778 'delta_try_count': None,
d1dc4a03125e delta-find: intrduce a `_one_dbg_data` method Pierre-Yves David <pierre-yves.david@octobus.net> parents: 50647 diff changeset	1779 'type': None,
d1dc4a03125e delta-find: intrduce a `_one_dbg_data` method Pierre-Yves David <pierre-yves.david@octobus.net> parents: 50647 diff changeset	1780 'p1-chain-len': None,
d1dc4a03125e delta-find: intrduce a `_one_dbg_data` method Pierre-Yves David <pierre-yves.david@octobus.net> parents: 50647 diff changeset	1781 'p2-chain-len': None,
d1dc4a03125e delta-find: intrduce a `_one_dbg_data` method Pierre-Yves David <pierre-yves.david@octobus.net> parents: 50647 diff changeset	1782 'snapshot-depth': None,
d1dc4a03125e delta-find: intrduce a `_one_dbg_data` method Pierre-Yves David <pierre-yves.david@octobus.net> parents: 50647 diff changeset	1783 'target-revlog': None,
d1dc4a03125e delta-find: intrduce a `_one_dbg_data` method Pierre-Yves David <pierre-yves.david@octobus.net> parents: 50647 diff changeset	1784 }
51325 46415df4eb1c delta-find: move filing of some debug data in `_one_dbg_data` Pierre-Yves David <pierre-yves.david@octobus.net> parents: 51324 diff changeset	1785 target_revlog = b"UNKNOWN"
46415df4eb1c delta-find: move filing of some debug data in `_one_dbg_data` Pierre-Yves David <pierre-yves.david@octobus.net> parents: 51324 diff changeset	1786 target_type = self.revlog.target[0]
46415df4eb1c delta-find: move filing of some debug data in `_one_dbg_data` Pierre-Yves David <pierre-yves.david@octobus.net> parents: 51324 diff changeset	1787 target_key = self.revlog.target[1]
46415df4eb1c delta-find: move filing of some debug data in `_one_dbg_data` Pierre-Yves David <pierre-yves.david@octobus.net> parents: 51324 diff changeset	1788 if target_type == KIND_CHANGELOG:
46415df4eb1c delta-find: move filing of some debug data in `_one_dbg_data` Pierre-Yves David <pierre-yves.david@octobus.net> parents: 51324 diff changeset	1789 target_revlog = b'CHANGELOG:'
46415df4eb1c delta-find: move filing of some debug data in `_one_dbg_data` Pierre-Yves David <pierre-yves.david@octobus.net> parents: 51324 diff changeset	1790 elif target_type == KIND_MANIFESTLOG:
46415df4eb1c delta-find: move filing of some debug data in `_one_dbg_data` Pierre-Yves David <pierre-yves.david@octobus.net> parents: 51324 diff changeset	1791 target_revlog = b'MANIFESTLOG:'
46415df4eb1c delta-find: move filing of some debug data in `_one_dbg_data` Pierre-Yves David <pierre-yves.david@octobus.net> parents: 51324 diff changeset	1792 if target_key:
46415df4eb1c delta-find: move filing of some debug data in `_one_dbg_data` Pierre-Yves David <pierre-yves.david@octobus.net> parents: 51324 diff changeset	1793 target_revlog += b'%s:' % target_key
46415df4eb1c delta-find: move filing of some debug data in `_one_dbg_data` Pierre-Yves David <pierre-yves.david@octobus.net> parents: 51324 diff changeset	1794 elif target_type == KIND_FILELOG:
46415df4eb1c delta-find: move filing of some debug data in `_one_dbg_data` Pierre-Yves David <pierre-yves.david@octobus.net> parents: 51324 diff changeset	1795 target_revlog = b'FILELOG:'
46415df4eb1c delta-find: move filing of some debug data in `_one_dbg_data` Pierre-Yves David <pierre-yves.david@octobus.net> parents: 51324 diff changeset	1796 if target_key:
46415df4eb1c delta-find: move filing of some debug data in `_one_dbg_data` Pierre-Yves David <pierre-yves.david@octobus.net> parents: 51324 diff changeset	1797 target_revlog += b'%s:' % target_key
46415df4eb1c delta-find: move filing of some debug data in `_one_dbg_data` Pierre-Yves David <pierre-yves.david@octobus.net> parents: 51324 diff changeset	1798 dbg['target-revlog'] = target_revlog
46415df4eb1c delta-find: move filing of some debug data in `_one_dbg_data` Pierre-Yves David <pierre-yves.david@octobus.net> parents: 51324 diff changeset	1799 return dbg
50648 d1dc4a03125e delta-find: intrduce a `_one_dbg_data` method Pierre-Yves David <pierre-yves.david@octobus.net> parents: 50647 diff changeset	1800
50647 4d84b6d52e93 delta-find: move final debug processing in a `_dbg_process_data` method Pierre-Yves David <pierre-yves.david@octobus.net> parents: 50646 diff changeset	1801 def _dbg_process_data(self, dbg):
4d84b6d52e93 delta-find: move final debug processing in a `_dbg_process_data` method Pierre-Yves David <pierre-yves.david@octobus.net> parents: 50646 diff changeset	1802 if self._debug_info is not None:
4d84b6d52e93 delta-find: move final debug processing in a `_dbg_process_data` method Pierre-Yves David <pierre-yves.david@octobus.net> parents: 50646 diff changeset	1803 self._debug_info.append(dbg)
49610 35d4c2124073 debug: add an option to display statistic about a unbundling operation Pierre-Yves David <pierre-yves.david@octobus.net> parents: 49608 diff changeset	1804
50647 4d84b6d52e93 delta-find: move final debug processing in a `_dbg_process_data` method Pierre-Yves David <pierre-yves.david@octobus.net> parents: 50646 diff changeset	1805 if self._write_debug is not None:
4d84b6d52e93 delta-find: move final debug processing in a `_dbg_process_data` method Pierre-Yves David <pierre-yves.david@octobus.net> parents: 50646 diff changeset	1806 msg = (
4d84b6d52e93 delta-find: move final debug processing in a `_dbg_process_data` method Pierre-Yves David <pierre-yves.david@octobus.net> parents: 50646 diff changeset	1807 b"DBG-DELTAS:"
4d84b6d52e93 delta-find: move final debug processing in a `_dbg_process_data` method Pierre-Yves David <pierre-yves.david@octobus.net> parents: 50646 diff changeset	1808 b" %-12s"
4d84b6d52e93 delta-find: move final debug processing in a `_dbg_process_data` method Pierre-Yves David <pierre-yves.david@octobus.net> parents: 50646 diff changeset	1809 b" rev=%d:"
4d84b6d52e93 delta-find: move final debug processing in a `_dbg_process_data` method Pierre-Yves David <pierre-yves.david@octobus.net> parents: 50646 diff changeset	1810 b" delta-base=%d"
4d84b6d52e93 delta-find: move final debug processing in a `_dbg_process_data` method Pierre-Yves David <pierre-yves.david@octobus.net> parents: 50646 diff changeset	1811 b" is-cached=%d"
4d84b6d52e93 delta-find: move final debug processing in a `_dbg_process_data` method Pierre-Yves David <pierre-yves.david@octobus.net> parents: 50646 diff changeset	1812 b" - search-rounds=%d"
4d84b6d52e93 delta-find: move final debug processing in a `_dbg_process_data` method Pierre-Yves David <pierre-yves.david@octobus.net> parents: 50646 diff changeset	1813 b" try-count=%d"
4d84b6d52e93 delta-find: move final debug processing in a `_dbg_process_data` method Pierre-Yves David <pierre-yves.david@octobus.net> parents: 50646 diff changeset	1814 b" - delta-type=%-6s"
4d84b6d52e93 delta-find: move final debug processing in a `_dbg_process_data` method Pierre-Yves David <pierre-yves.david@octobus.net> parents: 50646 diff changeset	1815 b" snap-depth=%d"
4d84b6d52e93 delta-find: move final debug processing in a `_dbg_process_data` method Pierre-Yves David <pierre-yves.david@octobus.net> parents: 50646 diff changeset	1816 b" - p1-chain-length=%d"
4d84b6d52e93 delta-find: move final debug processing in a `_dbg_process_data` method Pierre-Yves David <pierre-yves.david@octobus.net> parents: 50646 diff changeset	1817 b" p2-chain-length=%d"
4d84b6d52e93 delta-find: move final debug processing in a `_dbg_process_data` method Pierre-Yves David <pierre-yves.david@octobus.net> parents: 50646 diff changeset	1818 b" - duration=%f"
4d84b6d52e93 delta-find: move final debug processing in a `_dbg_process_data` method Pierre-Yves David <pierre-yves.david@octobus.net> parents: 50646 diff changeset	1819 b"\n"
4d84b6d52e93 delta-find: move final debug processing in a `_dbg_process_data` method Pierre-Yves David <pierre-yves.david@octobus.net> parents: 50646 diff changeset	1820 )
4d84b6d52e93 delta-find: move final debug processing in a `_dbg_process_data` method Pierre-Yves David <pierre-yves.david@octobus.net> parents: 50646 diff changeset	1821 msg %= (
4d84b6d52e93 delta-find: move final debug processing in a `_dbg_process_data` method Pierre-Yves David <pierre-yves.david@octobus.net> parents: 50646 diff changeset	1822 dbg["target-revlog"],
4d84b6d52e93 delta-find: move final debug processing in a `_dbg_process_data` method Pierre-Yves David <pierre-yves.david@octobus.net> parents: 50646 diff changeset	1823 dbg["revision"],
4d84b6d52e93 delta-find: move final debug processing in a `_dbg_process_data` method Pierre-Yves David <pierre-yves.david@octobus.net> parents: 50646 diff changeset	1824 dbg["delta-base"],
4d84b6d52e93 delta-find: move final debug processing in a `_dbg_process_data` method Pierre-Yves David <pierre-yves.david@octobus.net> parents: 50646 diff changeset	1825 dbg["using-cached-base"],
4d84b6d52e93 delta-find: move final debug processing in a `_dbg_process_data` method Pierre-Yves David <pierre-yves.david@octobus.net> parents: 50646 diff changeset	1826 dbg["search_round_count"],
4d84b6d52e93 delta-find: move final debug processing in a `_dbg_process_data` method Pierre-Yves David <pierre-yves.david@octobus.net> parents: 50646 diff changeset	1827 dbg["delta_try_count"],
4d84b6d52e93 delta-find: move final debug processing in a `_dbg_process_data` method Pierre-Yves David <pierre-yves.david@octobus.net> parents: 50646 diff changeset	1828 dbg["type"],
4d84b6d52e93 delta-find: move final debug processing in a `_dbg_process_data` method Pierre-Yves David <pierre-yves.david@octobus.net> parents: 50646 diff changeset	1829 dbg["snapshot-depth"],
4d84b6d52e93 delta-find: move final debug processing in a `_dbg_process_data` method Pierre-Yves David <pierre-yves.david@octobus.net> parents: 50646 diff changeset	1830 dbg["p1-chain-len"],
4d84b6d52e93 delta-find: move final debug processing in a `_dbg_process_data` method Pierre-Yves David <pierre-yves.david@octobus.net> parents: 50646 diff changeset	1831 dbg["p2-chain-len"],
4d84b6d52e93 delta-find: move final debug processing in a `_dbg_process_data` method Pierre-Yves David <pierre-yves.david@octobus.net> parents: 50646 diff changeset	1832 dbg["duration"],
4d84b6d52e93 delta-find: move final debug processing in a `_dbg_process_data` method Pierre-Yves David <pierre-yves.david@octobus.net> parents: 50646 diff changeset	1833 )
4d84b6d52e93 delta-find: move final debug processing in a `_dbg_process_data` method Pierre-Yves David <pierre-yves.david@octobus.net> parents: 50646 diff changeset	1834 self._write_debug(msg)
47452 c6844912c327 revlog: factor the logic to determine the delta compression out Pierre-Yves David <pierre-yves.david@octobus.net> parents: 47401 diff changeset	1835
c6844912c327 revlog: factor the logic to determine the delta compression out Pierre-Yves David <pierre-yves.david@octobus.net> parents: 47401 diff changeset	1836
c6844912c327 revlog: factor the logic to determine the delta compression out Pierre-Yves David <pierre-yves.david@octobus.net> parents: 47401 diff changeset	1837 def delta_compression(default_compression_header, deltainfo):
c6844912c327 revlog: factor the logic to determine the delta compression out Pierre-Yves David <pierre-yves.david@octobus.net> parents: 47401 diff changeset	1838 """return (COMPRESSION_MODE, deltainfo)
c6844912c327 revlog: factor the logic to determine the delta compression out Pierre-Yves David <pierre-yves.david@octobus.net> parents: 47401 diff changeset	1839
c6844912c327 revlog: factor the logic to determine the delta compression out Pierre-Yves David <pierre-yves.david@octobus.net> parents: 47401 diff changeset	1840 used by revlog v2+ format to dispatch between PLAIN and DEFAULT
c6844912c327 revlog: factor the logic to determine the delta compression out Pierre-Yves David <pierre-yves.david@octobus.net> parents: 47401 diff changeset	1841 compression.
c6844912c327 revlog: factor the logic to determine the delta compression out Pierre-Yves David <pierre-yves.david@octobus.net> parents: 47401 diff changeset	1842 """
c6844912c327 revlog: factor the logic to determine the delta compression out Pierre-Yves David <pierre-yves.david@octobus.net> parents: 47401 diff changeset	1843 h, d = deltainfo.data
c6844912c327 revlog: factor the logic to determine the delta compression out Pierre-Yves David <pierre-yves.david@octobus.net> parents: 47401 diff changeset	1844 compression_mode = COMP_MODE_INLINE
c6844912c327 revlog: factor the logic to determine the delta compression out Pierre-Yves David <pierre-yves.david@octobus.net> parents: 47401 diff changeset	1845 if not h and not d:
c6844912c327 revlog: factor the logic to determine the delta compression out Pierre-Yves David <pierre-yves.david@octobus.net> parents: 47401 diff changeset	1846 # not data to store at all... declare them uncompressed
c6844912c327 revlog: factor the logic to determine the delta compression out Pierre-Yves David <pierre-yves.david@octobus.net> parents: 47401 diff changeset	1847 compression_mode = COMP_MODE_PLAIN
c6844912c327 revlog: factor the logic to determine the delta compression out Pierre-Yves David <pierre-yves.david@octobus.net> parents: 47401 diff changeset	1848 elif not h:
c6844912c327 revlog: factor the logic to determine the delta compression out Pierre-Yves David <pierre-yves.david@octobus.net> parents: 47401 diff changeset	1849 t = d[0:1]
c6844912c327 revlog: factor the logic to determine the delta compression out Pierre-Yves David <pierre-yves.david@octobus.net> parents: 47401 diff changeset	1850 if t == b'\0':
c6844912c327 revlog: factor the logic to determine the delta compression out Pierre-Yves David <pierre-yves.david@octobus.net> parents: 47401 diff changeset	1851 compression_mode = COMP_MODE_PLAIN
c6844912c327 revlog: factor the logic to determine the delta compression out Pierre-Yves David <pierre-yves.david@octobus.net> parents: 47401 diff changeset	1852 elif t == default_compression_header:
c6844912c327 revlog: factor the logic to determine the delta compression out Pierre-Yves David <pierre-yves.david@octobus.net> parents: 47401 diff changeset	1853 compression_mode = COMP_MODE_DEFAULT
c6844912c327 revlog: factor the logic to determine the delta compression out Pierre-Yves David <pierre-yves.david@octobus.net> parents: 47401 diff changeset	1854 elif h == b'u':
c6844912c327 revlog: factor the logic to determine the delta compression out Pierre-Yves David <pierre-yves.david@octobus.net> parents: 47401 diff changeset	1855 # we have a more efficient way to declare uncompressed
c6844912c327 revlog: factor the logic to determine the delta compression out Pierre-Yves David <pierre-yves.david@octobus.net> parents: 47401 diff changeset	1856 h = b''
c6844912c327 revlog: factor the logic to determine the delta compression out Pierre-Yves David <pierre-yves.david@octobus.net> parents: 47401 diff changeset	1857 compression_mode = COMP_MODE_PLAIN
c6844912c327 revlog: factor the logic to determine the delta compression out Pierre-Yves David <pierre-yves.david@octobus.net> parents: 47401 diff changeset	1858 deltainfo = drop_u_compression(deltainfo)
c6844912c327 revlog: factor the logic to determine the delta compression out Pierre-Yves David <pierre-yves.david@octobus.net> parents: 47401 diff changeset	1859 return compression_mode, deltainfo

Mercurial > public > mercurial-scm > hg

annotate mercurial/revlogutils/deltas.py @ 51352:fac6038b11f5