mercurial-scm/hg-stable: mercurial/revlogutils/deltas.py annotate

annotate mercurial/revlogutils/deltas.py @ 42483:c1c1872d25d1

deltas: skip if projected compressed size does not match text size constraint If we have a delta, we check constraints against a lower bound estimate of the resulting compressed delta. We then checks this projected size against the ?? size constraints. This allows to exclude potential base candidates before doing any expensive computation. This only apply to the intermediate-snapshot case since this constraint only apply to them. For some pathological cases of a private repository this step provide a further performance boost (timing from `hg perfrevlogwrite`): before: 3.145906 seconds after: 3.010646 seconds

author	Valentin Gatien-Baron <vgatien-baron@janestreet.com>
date	Mon, 21 Jan 2019 22:46:18 +0100
parents	465f2d0df9ae
children	9b5fbe5ead89

rev	line source
39357 655b5b465953 revlog: split functionality related to deltas computation in a new module Boris Feld <boris.feld@octobus.net> parents: 39356 diff changeset	1 # revlogdeltas.py - Logic around delta computation for revlog
8226 8b2cd04a6e97 put license and copyright info into comment blocks Martin Geisler <mg@lazybytes.net> parents: 8225 diff changeset	2 #
8b2cd04a6e97 put license and copyright info into comment blocks Martin Geisler <mg@lazybytes.net> parents: 8225 diff changeset	3 # Copyright 2005-2007 Matt Mackall <mpm@selenic.com>
39357 655b5b465953 revlog: split functionality related to deltas computation in a new module Boris Feld <boris.feld@octobus.net> parents: 39356 diff changeset	4 # Copyright 2018 Octobus <contact@octobus.net>
8226 8b2cd04a6e97 put license and copyright info into comment blocks Martin Geisler <mg@lazybytes.net> parents: 8225 diff changeset	5 #
8b2cd04a6e97 put license and copyright info into comment blocks Martin Geisler <mg@lazybytes.net> parents: 8225 diff changeset	6 # This software may be used and distributed according to the terms of the
10263 25e572394f5c Update license to GPLv2+ Matt Mackall <mpm@selenic.com> parents: 10047 diff changeset	7 # GNU General Public License version 2 or any later version.
39357 655b5b465953 revlog: split functionality related to deltas computation in a new module Boris Feld <boris.feld@octobus.net> parents: 39356 diff changeset	8 """Helper class to compute deltas stored inside revlogs"""
8227 0a9542703300 turn some comments back into module docstrings Martin Geisler <mg@lazybytes.net> parents: 8226 diff changeset	9
27361 29f50344fa83 revlog: use absolute_import Gregory Szorc <gregory.szorc@gmail.com> parents: 27251 diff changeset	10 from __future__ import absolute_import
29f50344fa83 revlog: use absolute_import Gregory Szorc <gregory.szorc@gmail.com> parents: 27251 diff changeset	11
39510 3ca144f1c8dd snapshot: search for unrelated but reusable full-snapshot Boris Feld <boris.feld@octobus.net> parents: 39509 diff changeset	12 import collections
27361 29f50344fa83 revlog: use absolute_import Gregory Szorc <gregory.szorc@gmail.com> parents: 27251 diff changeset	13 import struct
29f50344fa83 revlog: use absolute_import Gregory Szorc <gregory.szorc@gmail.com> parents: 27251 diff changeset	14
29f50344fa83 revlog: use absolute_import Gregory Szorc <gregory.szorc@gmail.com> parents: 27251 diff changeset	15 # import stuff from node for others to import from revlog
39357 655b5b465953 revlog: split functionality related to deltas computation in a new module Boris Feld <boris.feld@octobus.net> parents: 39356 diff changeset	16 from ..node import (
27361 29f50344fa83 revlog: use absolute_import Gregory Szorc <gregory.szorc@gmail.com> parents: 27251 diff changeset	17 nullrev,
29f50344fa83 revlog: use absolute_import Gregory Szorc <gregory.szorc@gmail.com> parents: 27251 diff changeset	18 )
39357 655b5b465953 revlog: split functionality related to deltas computation in a new module Boris Feld <boris.feld@octobus.net> parents: 39356 diff changeset	19 from ..i18n import _
655b5b465953 revlog: split functionality related to deltas computation in a new module Boris Feld <boris.feld@octobus.net> parents: 39356 diff changeset	20
655b5b465953 revlog: split functionality related to deltas computation in a new module Boris Feld <boris.feld@octobus.net> parents: 39356 diff changeset	21 from .constants import (
39356 729082bb9938 revlog: split constants into a new `revlogutils.constants` module Boris Feld <boris.feld@octobus.net> parents: 39260 diff changeset	22 REVIDX_ISCENSORED,
729082bb9938 revlog: split constants into a new `revlogutils.constants` module Boris Feld <boris.feld@octobus.net> parents: 39260 diff changeset	23 REVIDX_RAWTEXT_CHANGING_FLAGS,
729082bb9938 revlog: split constants into a new `revlogutils.constants` module Boris Feld <boris.feld@octobus.net> parents: 39260 diff changeset	24 )
39357 655b5b465953 revlog: split functionality related to deltas computation in a new module Boris Feld <boris.feld@octobus.net> parents: 39356 diff changeset	25
655b5b465953 revlog: split functionality related to deltas computation in a new module Boris Feld <boris.feld@octobus.net> parents: 39356 diff changeset	26 from ..thirdparty import (
35638 edc9330acac1 revlog: introduce 'deltainfo' to distinguish from 'delta' Paul Morelle <paul.morelle@octobus.net> parents: 35637 diff changeset	27 attr,
edc9330acac1 revlog: introduce 'deltainfo' to distinguish from 'delta' Paul Morelle <paul.morelle@octobus.net> parents: 35637 diff changeset	28 )
39357 655b5b465953 revlog: split functionality related to deltas computation in a new module Boris Feld <boris.feld@octobus.net> parents: 39356 diff changeset	29
655b5b465953 revlog: split functionality related to deltas computation in a new module Boris Feld <boris.feld@octobus.net> parents: 39356 diff changeset	30 from .. import (
27361 29f50344fa83 revlog: use absolute_import Gregory Szorc <gregory.szorc@gmail.com> parents: 27251 diff changeset	31 error,
29f50344fa83 revlog: use absolute_import Gregory Szorc <gregory.szorc@gmail.com> parents: 27251 diff changeset	32 mdiff,
41111 38e88450138c delta: have a native implementation of _findsnapshot Boris Feld <boris.feld@octobus.net> parents: 41082 diff changeset	33 util,
27361 29f50344fa83 revlog: use absolute_import Gregory Szorc <gregory.szorc@gmail.com> parents: 27251 diff changeset	34 )
10913 f2ecc5733c89 revlog: factor out _maxinline global. Greg Ward <greg-hg@gerg.ca> parents: 10404 diff changeset	35
39357 655b5b465953 revlog: split functionality related to deltas computation in a new module Boris Feld <boris.feld@octobus.net> parents: 39356 diff changeset	36 # maximum <delta-chain-data>/<revision-text-length> ratio
655b5b465953 revlog: split functionality related to deltas computation in a new module Boris Feld <boris.feld@octobus.net> parents: 39356 diff changeset	37 LIMIT_DELTA2TEXT = 2
1091 d62130f99a73 Move hash function back to revlog from node mpm@selenic.com parents: 1089 diff changeset	38
38637 e33f784f2a44 revlog: introduce a tiny mock of a revlog class Boris Feld <boris.feld@octobus.net> parents: 38636 diff changeset	39 class _testrevlog(object):
e33f784f2a44 revlog: introduce a tiny mock of a revlog class Boris Feld <boris.feld@octobus.net> parents: 38636 diff changeset	40 """minimalist fake revlog to use in doctests"""
e33f784f2a44 revlog: introduce a tiny mock of a revlog class Boris Feld <boris.feld@octobus.net> parents: 38636 diff changeset	41
40684 85b14f0dc334 doctest: add a `issnapshot` method to _testrevlog Boris Feld <boris.feld@octobus.net> parents: 40655 diff changeset	42 def __init__(self, data, density=0.5, mingap=0, snapshot=()):
38637 e33f784f2a44 revlog: introduce a tiny mock of a revlog class Boris Feld <boris.feld@octobus.net> parents: 38636 diff changeset	43 """data is an list of revision payload boundaries"""
e33f784f2a44 revlog: introduce a tiny mock of a revlog class Boris Feld <boris.feld@octobus.net> parents: 38636 diff changeset	44 self._data = data
e33f784f2a44 revlog: introduce a tiny mock of a revlog class Boris Feld <boris.feld@octobus.net> parents: 38636 diff changeset	45 self._srdensitythreshold = density
e33f784f2a44 revlog: introduce a tiny mock of a revlog class Boris Feld <boris.feld@octobus.net> parents: 38636 diff changeset	46 self._srmingapsize = mingap
40684 85b14f0dc334 doctest: add a `issnapshot` method to _testrevlog Boris Feld <boris.feld@octobus.net> parents: 40655 diff changeset	47 self._snapshot = set(snapshot)
40748 39d29542fe40 sparse-revlog: put the native implementation of slicechunktodensity to use Boris Feld <boris.feld@octobus.net> parents: 40698 diff changeset	48 self.index = None
38637 e33f784f2a44 revlog: introduce a tiny mock of a revlog class Boris Feld <boris.feld@octobus.net> parents: 38636 diff changeset	49
e33f784f2a44 revlog: introduce a tiny mock of a revlog class Boris Feld <boris.feld@octobus.net> parents: 38636 diff changeset	50 def start(self, rev):
41082 88d813cd9acd revlog: fix pure python slicing test when chain contains nullrev Boris Feld <boris.feld@octobus.net> parents: 41036 diff changeset	51 if rev == nullrev:
88d813cd9acd revlog: fix pure python slicing test when chain contains nullrev Boris Feld <boris.feld@octobus.net> parents: 41036 diff changeset	52 return 0
38637 e33f784f2a44 revlog: introduce a tiny mock of a revlog class Boris Feld <boris.feld@octobus.net> parents: 38636 diff changeset	53 if rev == 0:
e33f784f2a44 revlog: introduce a tiny mock of a revlog class Boris Feld <boris.feld@octobus.net> parents: 38636 diff changeset	54 return 0
e33f784f2a44 revlog: introduce a tiny mock of a revlog class Boris Feld <boris.feld@octobus.net> parents: 38636 diff changeset	55 return self._data[rev - 1]
e33f784f2a44 revlog: introduce a tiny mock of a revlog class Boris Feld <boris.feld@octobus.net> parents: 38636 diff changeset	56
e33f784f2a44 revlog: introduce a tiny mock of a revlog class Boris Feld <boris.feld@octobus.net> parents: 38636 diff changeset	57 def end(self, rev):
41082 88d813cd9acd revlog: fix pure python slicing test when chain contains nullrev Boris Feld <boris.feld@octobus.net> parents: 41036 diff changeset	58 if rev == nullrev:
88d813cd9acd revlog: fix pure python slicing test when chain contains nullrev Boris Feld <boris.feld@octobus.net> parents: 41036 diff changeset	59 return 0
38637 e33f784f2a44 revlog: introduce a tiny mock of a revlog class Boris Feld <boris.feld@octobus.net> parents: 38636 diff changeset	60 return self._data[rev]
e33f784f2a44 revlog: introduce a tiny mock of a revlog class Boris Feld <boris.feld@octobus.net> parents: 38636 diff changeset	61
e33f784f2a44 revlog: introduce a tiny mock of a revlog class Boris Feld <boris.feld@octobus.net> parents: 38636 diff changeset	62 def length(self, rev):
e33f784f2a44 revlog: introduce a tiny mock of a revlog class Boris Feld <boris.feld@octobus.net> parents: 38636 diff changeset	63 return self.end(rev) - self.start(rev)
e33f784f2a44 revlog: introduce a tiny mock of a revlog class Boris Feld <boris.feld@octobus.net> parents: 38636 diff changeset	64
38718 f8762ea73e0d sparse-revlog: implement algorithm to write sparse delta chains (issue5480) Paul Morelle <paul.morelle@octobus.net> parents: 38717 diff changeset	65 def __len__(self):
f8762ea73e0d sparse-revlog: implement algorithm to write sparse delta chains (issue5480) Paul Morelle <paul.morelle@octobus.net> parents: 38717 diff changeset	66 return len(self._data)
f8762ea73e0d sparse-revlog: implement algorithm to write sparse delta chains (issue5480) Paul Morelle <paul.morelle@octobus.net> parents: 38717 diff changeset	67
40684 85b14f0dc334 doctest: add a `issnapshot` method to _testrevlog Boris Feld <boris.feld@octobus.net> parents: 40655 diff changeset	68 def issnapshot(self, rev):
41082 88d813cd9acd revlog: fix pure python slicing test when chain contains nullrev Boris Feld <boris.feld@octobus.net> parents: 41036 diff changeset	69 if rev == nullrev:
88d813cd9acd revlog: fix pure python slicing test when chain contains nullrev Boris Feld <boris.feld@octobus.net> parents: 41036 diff changeset	70 return True
40684 85b14f0dc334 doctest: add a `issnapshot` method to _testrevlog Boris Feld <boris.feld@octobus.net> parents: 40655 diff changeset	71 return rev in self._snapshot
85b14f0dc334 doctest: add a `issnapshot` method to _testrevlog Boris Feld <boris.feld@octobus.net> parents: 40655 diff changeset	72
40651 3ac23dad6364 sparse-revlog: drop unused deltainfo parameter from _slicechunktodensity Boris Feld <boris.feld@octobus.net> parents: 40650 diff changeset	73 def slicechunk(revlog, revs, targetsize=None):
34824 e2ad93bcc084 revlog: introduce an experimental flag to slice chunks reads when too sparse Paul Morelle <paul.morelle@octobus.net> parents: 34823 diff changeset	74 """slice revs to reduce the amount of unrelated data to be read from disk.
e2ad93bcc084 revlog: introduce an experimental flag to slice chunks reads when too sparse Paul Morelle <paul.morelle@octobus.net> parents: 34823 diff changeset	75
e2ad93bcc084 revlog: introduce an experimental flag to slice chunks reads when too sparse Paul Morelle <paul.morelle@octobus.net> parents: 34823 diff changeset	76 ``revs`` is sliced into groups that should be read in one time.
e2ad93bcc084 revlog: introduce an experimental flag to slice chunks reads when too sparse Paul Morelle <paul.morelle@octobus.net> parents: 34823 diff changeset	77 Assume that revs are sorted.
38640 f62b8fb0a484 revlog: document and test _slicechunk Boris Feld <boris.feld@octobus.net> parents: 38639 diff changeset	78
f62b8fb0a484 revlog: document and test _slicechunk Boris Feld <boris.feld@octobus.net> parents: 38639 diff changeset	79 The initial chunk is sliced until the overall density (payload/chunks-span
f62b8fb0a484 revlog: document and test _slicechunk Boris Feld <boris.feld@octobus.net> parents: 38639 diff changeset	80 ratio) is above `revlog._srdensitythreshold`. No gap smaller than
f62b8fb0a484 revlog: document and test _slicechunk Boris Feld <boris.feld@octobus.net> parents: 38639 diff changeset	81 `revlog._srmingapsize` is skipped.
f62b8fb0a484 revlog: document and test _slicechunk Boris Feld <boris.feld@octobus.net> parents: 38639 diff changeset	82
38643 967fee55e8d9 revlog: postprocess chunk to slice them down to a certain size Boris Feld <boris.feld@octobus.net> parents: 38642 diff changeset	83 If `targetsize` is set, no chunk larger than `targetsize` will be yield.
967fee55e8d9 revlog: postprocess chunk to slice them down to a certain size Boris Feld <boris.feld@octobus.net> parents: 38642 diff changeset	84 For consistency with other slicing choice, this limit won't go lower than
967fee55e8d9 revlog: postprocess chunk to slice them down to a certain size Boris Feld <boris.feld@octobus.net> parents: 38642 diff changeset	85 `revlog._srmingapsize`.
967fee55e8d9 revlog: postprocess chunk to slice them down to a certain size Boris Feld <boris.feld@octobus.net> parents: 38642 diff changeset	86
967fee55e8d9 revlog: postprocess chunk to slice them down to a certain size Boris Feld <boris.feld@octobus.net> parents: 38642 diff changeset	87 If individual revisions chunk are larger than this limit, they will still
967fee55e8d9 revlog: postprocess chunk to slice them down to a certain size Boris Feld <boris.feld@octobus.net> parents: 38642 diff changeset	88 be raised individually.
967fee55e8d9 revlog: postprocess chunk to slice them down to a certain size Boris Feld <boris.feld@octobus.net> parents: 38642 diff changeset	89
40685 9c3c697267db sparse-revlog: rework the way we enforce chunk size limit Boris Feld <boris.feld@octobus.net> parents: 40684 diff changeset	90 >>> data = [
38640 f62b8fb0a484 revlog: document and test _slicechunk Boris Feld <boris.feld@octobus.net> parents: 38639 diff changeset	91 ... 5, #00 (5)
f62b8fb0a484 revlog: document and test _slicechunk Boris Feld <boris.feld@octobus.net> parents: 38639 diff changeset	92 ... 10, #01 (5)
f62b8fb0a484 revlog: document and test _slicechunk Boris Feld <boris.feld@octobus.net> parents: 38639 diff changeset	93 ... 12, #02 (2)
f62b8fb0a484 revlog: document and test _slicechunk Boris Feld <boris.feld@octobus.net> parents: 38639 diff changeset	94 ... 12, #03 (empty)
f62b8fb0a484 revlog: document and test _slicechunk Boris Feld <boris.feld@octobus.net> parents: 38639 diff changeset	95 ... 27, #04 (15)
f62b8fb0a484 revlog: document and test _slicechunk Boris Feld <boris.feld@octobus.net> parents: 38639 diff changeset	96 ... 31, #05 (4)
f62b8fb0a484 revlog: document and test _slicechunk Boris Feld <boris.feld@octobus.net> parents: 38639 diff changeset	97 ... 31, #06 (empty)
f62b8fb0a484 revlog: document and test _slicechunk Boris Feld <boris.feld@octobus.net> parents: 38639 diff changeset	98 ... 42, #07 (11)
f62b8fb0a484 revlog: document and test _slicechunk Boris Feld <boris.feld@octobus.net> parents: 38639 diff changeset	99 ... 47, #08 (5)
f62b8fb0a484 revlog: document and test _slicechunk Boris Feld <boris.feld@octobus.net> parents: 38639 diff changeset	100 ... 47, #09 (empty)
f62b8fb0a484 revlog: document and test _slicechunk Boris Feld <boris.feld@octobus.net> parents: 38639 diff changeset	101 ... 48, #10 (1)
f62b8fb0a484 revlog: document and test _slicechunk Boris Feld <boris.feld@octobus.net> parents: 38639 diff changeset	102 ... 51, #11 (3)
f62b8fb0a484 revlog: document and test _slicechunk Boris Feld <boris.feld@octobus.net> parents: 38639 diff changeset	103 ... 74, #12 (23)
f62b8fb0a484 revlog: document and test _slicechunk Boris Feld <boris.feld@octobus.net> parents: 38639 diff changeset	104 ... 85, #13 (11)
f62b8fb0a484 revlog: document and test _slicechunk Boris Feld <boris.feld@octobus.net> parents: 38639 diff changeset	105 ... 86, #14 (1)
f62b8fb0a484 revlog: document and test _slicechunk Boris Feld <boris.feld@octobus.net> parents: 38639 diff changeset	106 ... 91, #15 (5)
40685 9c3c697267db sparse-revlog: rework the way we enforce chunk size limit Boris Feld <boris.feld@octobus.net> parents: 40684 diff changeset	107 ... ]
9c3c697267db sparse-revlog: rework the way we enforce chunk size limit Boris Feld <boris.feld@octobus.net> parents: 40684 diff changeset	108 >>> revlog = _testrevlog(data, snapshot=range(16))
38640 f62b8fb0a484 revlog: document and test _slicechunk Boris Feld <boris.feld@octobus.net> parents: 38639 diff changeset	109
39357 655b5b465953 revlog: split functionality related to deltas computation in a new module Boris Feld <boris.feld@octobus.net> parents: 39356 diff changeset	110 >>> list(slicechunk(revlog, list(range(16))))
38640 f62b8fb0a484 revlog: document and test _slicechunk Boris Feld <boris.feld@octobus.net> parents: 38639 diff changeset	111 [[0, 1, 2, 3, 4, 5, 6, 7, 8, 9, 10, 11, 12, 13, 14, 15]]
39357 655b5b465953 revlog: split functionality related to deltas computation in a new module Boris Feld <boris.feld@octobus.net> parents: 39356 diff changeset	112 >>> list(slicechunk(revlog, [0, 15]))
38640 f62b8fb0a484 revlog: document and test _slicechunk Boris Feld <boris.feld@octobus.net> parents: 38639 diff changeset	113 [[0], [15]]
39357 655b5b465953 revlog: split functionality related to deltas computation in a new module Boris Feld <boris.feld@octobus.net> parents: 39356 diff changeset	114 >>> list(slicechunk(revlog, [0, 11, 15]))
38640 f62b8fb0a484 revlog: document and test _slicechunk Boris Feld <boris.feld@octobus.net> parents: 38639 diff changeset	115 [[0], [11], [15]]
39357 655b5b465953 revlog: split functionality related to deltas computation in a new module Boris Feld <boris.feld@octobus.net> parents: 39356 diff changeset	116 >>> list(slicechunk(revlog, [0, 11, 13, 15]))
38640 f62b8fb0a484 revlog: document and test _slicechunk Boris Feld <boris.feld@octobus.net> parents: 38639 diff changeset	117 [[0], [11, 13, 15]]
39357 655b5b465953 revlog: split functionality related to deltas computation in a new module Boris Feld <boris.feld@octobus.net> parents: 39356 diff changeset	118 >>> list(slicechunk(revlog, [1, 2, 3, 5, 8, 10, 11, 14]))
38640 f62b8fb0a484 revlog: document and test _slicechunk Boris Feld <boris.feld@octobus.net> parents: 38639 diff changeset	119 [[1, 2], [5, 8, 10, 11], [14]]
38643 967fee55e8d9 revlog: postprocess chunk to slice them down to a certain size Boris Feld <boris.feld@octobus.net> parents: 38642 diff changeset	120
967fee55e8d9 revlog: postprocess chunk to slice them down to a certain size Boris Feld <boris.feld@octobus.net> parents: 38642 diff changeset	121 Slicing with a maximum chunk size
39357 655b5b465953 revlog: split functionality related to deltas computation in a new module Boris Feld <boris.feld@octobus.net> parents: 39356 diff changeset	122 >>> list(slicechunk(revlog, [0, 11, 13, 15], targetsize=15))
38643 967fee55e8d9 revlog: postprocess chunk to slice them down to a certain size Boris Feld <boris.feld@octobus.net> parents: 38642 diff changeset	123 [[0], [11], [13], [15]]
39357 655b5b465953 revlog: split functionality related to deltas computation in a new module Boris Feld <boris.feld@octobus.net> parents: 39356 diff changeset	124 >>> list(slicechunk(revlog, [0, 11, 13, 15], targetsize=20))
38643 967fee55e8d9 revlog: postprocess chunk to slice them down to a certain size Boris Feld <boris.feld@octobus.net> parents: 38642 diff changeset	125 [[0], [11], [13, 15]]
41082 88d813cd9acd revlog: fix pure python slicing test when chain contains nullrev Boris Feld <boris.feld@octobus.net> parents: 41036 diff changeset	126
88d813cd9acd revlog: fix pure python slicing test when chain contains nullrev Boris Feld <boris.feld@octobus.net> parents: 41036 diff changeset	127 Slicing involving nullrev
88d813cd9acd revlog: fix pure python slicing test when chain contains nullrev Boris Feld <boris.feld@octobus.net> parents: 41036 diff changeset	128 >>> list(slicechunk(revlog, [-1, 0, 11, 13, 15], targetsize=20))
88d813cd9acd revlog: fix pure python slicing test when chain contains nullrev Boris Feld <boris.feld@octobus.net> parents: 41036 diff changeset	129 [[-1, 0], [11], [13, 15]]
88d813cd9acd revlog: fix pure python slicing test when chain contains nullrev Boris Feld <boris.feld@octobus.net> parents: 41036 diff changeset	130 >>> list(slicechunk(revlog, [-1, 13, 15], targetsize=5))
88d813cd9acd revlog: fix pure python slicing test when chain contains nullrev Boris Feld <boris.feld@octobus.net> parents: 41036 diff changeset	131 [[-1], [13], [15]]
34824 e2ad93bcc084 revlog: introduce an experimental flag to slice chunks reads when too sparse Paul Morelle <paul.morelle@octobus.net> parents: 34823 diff changeset	132 """
38643 967fee55e8d9 revlog: postprocess chunk to slice them down to a certain size Boris Feld <boris.feld@octobus.net> parents: 38642 diff changeset	133 if targetsize is not None:
967fee55e8d9 revlog: postprocess chunk to slice them down to a certain size Boris Feld <boris.feld@octobus.net> parents: 38642 diff changeset	134 targetsize = max(targetsize, revlog._srmingapsize)
38718 f8762ea73e0d sparse-revlog: implement algorithm to write sparse delta chains (issue5480) Paul Morelle <paul.morelle@octobus.net> parents: 38717 diff changeset	135 # targetsize should not be specified when evaluating delta candidates:
f8762ea73e0d sparse-revlog: implement algorithm to write sparse delta chains (issue5480) Paul Morelle <paul.morelle@octobus.net> parents: 38717 diff changeset	136 # * targetsize is used to ensure we stay within specification when reading,
40748 39d29542fe40 sparse-revlog: put the native implementation of slicechunktodensity to use Boris Feld <boris.feld@octobus.net> parents: 40698 diff changeset	137 densityslicing = getattr(revlog.index, 'slicechunktodensity', None)
39d29542fe40 sparse-revlog: put the native implementation of slicechunktodensity to use Boris Feld <boris.feld@octobus.net> parents: 40698 diff changeset	138 if densityslicing is None:
39d29542fe40 sparse-revlog: put the native implementation of slicechunktodensity to use Boris Feld <boris.feld@octobus.net> parents: 40698 diff changeset	139 densityslicing = lambda x, y, z: _slicechunktodensity(revlog, x, y, z)
39d29542fe40 sparse-revlog: put the native implementation of slicechunktodensity to use Boris Feld <boris.feld@octobus.net> parents: 40698 diff changeset	140 for chunk in densityslicing(revs,
39d29542fe40 sparse-revlog: put the native implementation of slicechunktodensity to use Boris Feld <boris.feld@octobus.net> parents: 40698 diff changeset	141 revlog._srdensitythreshold,
39d29542fe40 sparse-revlog: put the native implementation of slicechunktodensity to use Boris Feld <boris.feld@octobus.net> parents: 40698 diff changeset	142 revlog._srmingapsize):
38643 967fee55e8d9 revlog: postprocess chunk to slice them down to a certain size Boris Feld <boris.feld@octobus.net> parents: 38642 diff changeset	143 for subchunk in _slicechunktosize(revlog, chunk, targetsize):
967fee55e8d9 revlog: postprocess chunk to slice them down to a certain size Boris Feld <boris.feld@octobus.net> parents: 38642 diff changeset	144 yield subchunk
38641 feba6be0941b revlog: extract density based slicing into its own function Boris Feld <boris.feld@octobus.net> parents: 38640 diff changeset	145
38718 f8762ea73e0d sparse-revlog: implement algorithm to write sparse delta chains (issue5480) Paul Morelle <paul.morelle@octobus.net> parents: 38717 diff changeset	146 def _slicechunktosize(revlog, revs, targetsize=None):
38642 e59e27e52297 revlog: add function to slice chunk down to a given size Boris Feld <boris.feld@octobus.net> parents: 38641 diff changeset	147 """slice revs to match the target size
e59e27e52297 revlog: add function to slice chunk down to a given size Boris Feld <boris.feld@octobus.net> parents: 38641 diff changeset	148
e59e27e52297 revlog: add function to slice chunk down to a given size Boris Feld <boris.feld@octobus.net> parents: 38641 diff changeset	149 This is intended to be used on chunk that density slicing selected by that
e59e27e52297 revlog: add function to slice chunk down to a given size Boris Feld <boris.feld@octobus.net> parents: 38641 diff changeset	150 are still too large compared to the read garantee of revlog. This might
e59e27e52297 revlog: add function to slice chunk down to a given size Boris Feld <boris.feld@octobus.net> parents: 38641 diff changeset	151 happens when "minimal gap size" interrupted the slicing or when chain are
e59e27e52297 revlog: add function to slice chunk down to a given size Boris Feld <boris.feld@octobus.net> parents: 38641 diff changeset	152 built in a way that create large blocks next to each other.
e59e27e52297 revlog: add function to slice chunk down to a given size Boris Feld <boris.feld@octobus.net> parents: 38641 diff changeset	153
40685 9c3c697267db sparse-revlog: rework the way we enforce chunk size limit Boris Feld <boris.feld@octobus.net> parents: 40684 diff changeset	154 >>> data = [
38642 e59e27e52297 revlog: add function to slice chunk down to a given size Boris Feld <boris.feld@octobus.net> parents: 38641 diff changeset	155 ... 3, #0 (3)
e59e27e52297 revlog: add function to slice chunk down to a given size Boris Feld <boris.feld@octobus.net> parents: 38641 diff changeset	156 ... 5, #1 (2)
e59e27e52297 revlog: add function to slice chunk down to a given size Boris Feld <boris.feld@octobus.net> parents: 38641 diff changeset	157 ... 6, #2 (1)
e59e27e52297 revlog: add function to slice chunk down to a given size Boris Feld <boris.feld@octobus.net> parents: 38641 diff changeset	158 ... 8, #3 (2)
e59e27e52297 revlog: add function to slice chunk down to a given size Boris Feld <boris.feld@octobus.net> parents: 38641 diff changeset	159 ... 8, #4 (empty)
e59e27e52297 revlog: add function to slice chunk down to a given size Boris Feld <boris.feld@octobus.net> parents: 38641 diff changeset	160 ... 11, #5 (3)
e59e27e52297 revlog: add function to slice chunk down to a given size Boris Feld <boris.feld@octobus.net> parents: 38641 diff changeset	161 ... 12, #6 (1)
e59e27e52297 revlog: add function to slice chunk down to a given size Boris Feld <boris.feld@octobus.net> parents: 38641 diff changeset	162 ... 13, #7 (1)
e59e27e52297 revlog: add function to slice chunk down to a given size Boris Feld <boris.feld@octobus.net> parents: 38641 diff changeset	163 ... 14, #8 (1)
40685 9c3c697267db sparse-revlog: rework the way we enforce chunk size limit Boris Feld <boris.feld@octobus.net> parents: 40684 diff changeset	164 ... ]
9c3c697267db sparse-revlog: rework the way we enforce chunk size limit Boris Feld <boris.feld@octobus.net> parents: 40684 diff changeset	165
9c3c697267db sparse-revlog: rework the way we enforce chunk size limit Boris Feld <boris.feld@octobus.net> parents: 40684 diff changeset	166 == All snapshots cases ==
9c3c697267db sparse-revlog: rework the way we enforce chunk size limit Boris Feld <boris.feld@octobus.net> parents: 40684 diff changeset	167 >>> revlog = _testrevlog(data, snapshot=range(9))
38642 e59e27e52297 revlog: add function to slice chunk down to a given size Boris Feld <boris.feld@octobus.net> parents: 38641 diff changeset	168
e59e27e52297 revlog: add function to slice chunk down to a given size Boris Feld <boris.feld@octobus.net> parents: 38641 diff changeset	169 Cases where chunk is already small enough
e59e27e52297 revlog: add function to slice chunk down to a given size Boris Feld <boris.feld@octobus.net> parents: 38641 diff changeset	170 >>> list(_slicechunktosize(revlog, [0], 3))
e59e27e52297 revlog: add function to slice chunk down to a given size Boris Feld <boris.feld@octobus.net> parents: 38641 diff changeset	171 [[0]]
e59e27e52297 revlog: add function to slice chunk down to a given size Boris Feld <boris.feld@octobus.net> parents: 38641 diff changeset	172 >>> list(_slicechunktosize(revlog, [6, 7], 3))
e59e27e52297 revlog: add function to slice chunk down to a given size Boris Feld <boris.feld@octobus.net> parents: 38641 diff changeset	173 [[6, 7]]
e59e27e52297 revlog: add function to slice chunk down to a given size Boris Feld <boris.feld@octobus.net> parents: 38641 diff changeset	174 >>> list(_slicechunktosize(revlog, [0], None))
e59e27e52297 revlog: add function to slice chunk down to a given size Boris Feld <boris.feld@octobus.net> parents: 38641 diff changeset	175 [[0]]
e59e27e52297 revlog: add function to slice chunk down to a given size Boris Feld <boris.feld@octobus.net> parents: 38641 diff changeset	176 >>> list(_slicechunktosize(revlog, [6, 7], None))
e59e27e52297 revlog: add function to slice chunk down to a given size Boris Feld <boris.feld@octobus.net> parents: 38641 diff changeset	177 [[6, 7]]
e59e27e52297 revlog: add function to slice chunk down to a given size Boris Feld <boris.feld@octobus.net> parents: 38641 diff changeset	178
e59e27e52297 revlog: add function to slice chunk down to a given size Boris Feld <boris.feld@octobus.net> parents: 38641 diff changeset	179 cases where we need actual slicing
e59e27e52297 revlog: add function to slice chunk down to a given size Boris Feld <boris.feld@octobus.net> parents: 38641 diff changeset	180 >>> list(_slicechunktosize(revlog, [0, 1], 3))
e59e27e52297 revlog: add function to slice chunk down to a given size Boris Feld <boris.feld@octobus.net> parents: 38641 diff changeset	181 [[0], [1]]
e59e27e52297 revlog: add function to slice chunk down to a given size Boris Feld <boris.feld@octobus.net> parents: 38641 diff changeset	182 >>> list(_slicechunktosize(revlog, [1, 3], 3))
e59e27e52297 revlog: add function to slice chunk down to a given size Boris Feld <boris.feld@octobus.net> parents: 38641 diff changeset	183 [[1], [3]]
e59e27e52297 revlog: add function to slice chunk down to a given size Boris Feld <boris.feld@octobus.net> parents: 38641 diff changeset	184 >>> list(_slicechunktosize(revlog, [1, 2, 3], 3))
e59e27e52297 revlog: add function to slice chunk down to a given size Boris Feld <boris.feld@octobus.net> parents: 38641 diff changeset	185 [[1, 2], [3]]
e59e27e52297 revlog: add function to slice chunk down to a given size Boris Feld <boris.feld@octobus.net> parents: 38641 diff changeset	186 >>> list(_slicechunktosize(revlog, [3, 5], 3))
e59e27e52297 revlog: add function to slice chunk down to a given size Boris Feld <boris.feld@octobus.net> parents: 38641 diff changeset	187 [[3], [5]]
e59e27e52297 revlog: add function to slice chunk down to a given size Boris Feld <boris.feld@octobus.net> parents: 38641 diff changeset	188 >>> list(_slicechunktosize(revlog, [3, 4, 5], 3))
e59e27e52297 revlog: add function to slice chunk down to a given size Boris Feld <boris.feld@octobus.net> parents: 38641 diff changeset	189 [[3], [5]]
e59e27e52297 revlog: add function to slice chunk down to a given size Boris Feld <boris.feld@octobus.net> parents: 38641 diff changeset	190 >>> list(_slicechunktosize(revlog, [5, 6, 7, 8], 3))
e59e27e52297 revlog: add function to slice chunk down to a given size Boris Feld <boris.feld@octobus.net> parents: 38641 diff changeset	191 [[5], [6, 7, 8]]
e59e27e52297 revlog: add function to slice chunk down to a given size Boris Feld <boris.feld@octobus.net> parents: 38641 diff changeset	192 >>> list(_slicechunktosize(revlog, [0, 1, 2, 3, 4, 5, 6, 7, 8], 3))
e59e27e52297 revlog: add function to slice chunk down to a given size Boris Feld <boris.feld@octobus.net> parents: 38641 diff changeset	193 [[0], [1, 2], [3], [5], [6, 7, 8]]
e59e27e52297 revlog: add function to slice chunk down to a given size Boris Feld <boris.feld@octobus.net> parents: 38641 diff changeset	194
e59e27e52297 revlog: add function to slice chunk down to a given size Boris Feld <boris.feld@octobus.net> parents: 38641 diff changeset	195 Case with too large individual chunk (must return valid chunk)
e59e27e52297 revlog: add function to slice chunk down to a given size Boris Feld <boris.feld@octobus.net> parents: 38641 diff changeset	196 >>> list(_slicechunktosize(revlog, [0, 1], 2))
e59e27e52297 revlog: add function to slice chunk down to a given size Boris Feld <boris.feld@octobus.net> parents: 38641 diff changeset	197 [[0], [1]]
e59e27e52297 revlog: add function to slice chunk down to a given size Boris Feld <boris.feld@octobus.net> parents: 38641 diff changeset	198 >>> list(_slicechunktosize(revlog, [1, 3], 1))
e59e27e52297 revlog: add function to slice chunk down to a given size Boris Feld <boris.feld@octobus.net> parents: 38641 diff changeset	199 [[1], [3]]
e59e27e52297 revlog: add function to slice chunk down to a given size Boris Feld <boris.feld@octobus.net> parents: 38641 diff changeset	200 >>> list(_slicechunktosize(revlog, [3, 4, 5], 2))
e59e27e52297 revlog: add function to slice chunk down to a given size Boris Feld <boris.feld@octobus.net> parents: 38641 diff changeset	201 [[3], [5]]
40685 9c3c697267db sparse-revlog: rework the way we enforce chunk size limit Boris Feld <boris.feld@octobus.net> parents: 40684 diff changeset	202
9c3c697267db sparse-revlog: rework the way we enforce chunk size limit Boris Feld <boris.feld@octobus.net> parents: 40684 diff changeset	203 == No Snapshot cases ==
9c3c697267db sparse-revlog: rework the way we enforce chunk size limit Boris Feld <boris.feld@octobus.net> parents: 40684 diff changeset	204 >>> revlog = _testrevlog(data)
9c3c697267db sparse-revlog: rework the way we enforce chunk size limit Boris Feld <boris.feld@octobus.net> parents: 40684 diff changeset	205
9c3c697267db sparse-revlog: rework the way we enforce chunk size limit Boris Feld <boris.feld@octobus.net> parents: 40684 diff changeset	206 Cases where chunk is already small enough
9c3c697267db sparse-revlog: rework the way we enforce chunk size limit Boris Feld <boris.feld@octobus.net> parents: 40684 diff changeset	207 >>> list(_slicechunktosize(revlog, [0], 3))
9c3c697267db sparse-revlog: rework the way we enforce chunk size limit Boris Feld <boris.feld@octobus.net> parents: 40684 diff changeset	208 [[0]]
9c3c697267db sparse-revlog: rework the way we enforce chunk size limit Boris Feld <boris.feld@octobus.net> parents: 40684 diff changeset	209 >>> list(_slicechunktosize(revlog, [6, 7], 3))
9c3c697267db sparse-revlog: rework the way we enforce chunk size limit Boris Feld <boris.feld@octobus.net> parents: 40684 diff changeset	210 [[6, 7]]
9c3c697267db sparse-revlog: rework the way we enforce chunk size limit Boris Feld <boris.feld@octobus.net> parents: 40684 diff changeset	211 >>> list(_slicechunktosize(revlog, [0], None))
9c3c697267db sparse-revlog: rework the way we enforce chunk size limit Boris Feld <boris.feld@octobus.net> parents: 40684 diff changeset	212 [[0]]
9c3c697267db sparse-revlog: rework the way we enforce chunk size limit Boris Feld <boris.feld@octobus.net> parents: 40684 diff changeset	213 >>> list(_slicechunktosize(revlog, [6, 7], None))
9c3c697267db sparse-revlog: rework the way we enforce chunk size limit Boris Feld <boris.feld@octobus.net> parents: 40684 diff changeset	214 [[6, 7]]
9c3c697267db sparse-revlog: rework the way we enforce chunk size limit Boris Feld <boris.feld@octobus.net> parents: 40684 diff changeset	215
9c3c697267db sparse-revlog: rework the way we enforce chunk size limit Boris Feld <boris.feld@octobus.net> parents: 40684 diff changeset	216 cases where we need actual slicing
9c3c697267db sparse-revlog: rework the way we enforce chunk size limit Boris Feld <boris.feld@octobus.net> parents: 40684 diff changeset	217 >>> list(_slicechunktosize(revlog, [0, 1], 3))
9c3c697267db sparse-revlog: rework the way we enforce chunk size limit Boris Feld <boris.feld@octobus.net> parents: 40684 diff changeset	218 [[0], [1]]
9c3c697267db sparse-revlog: rework the way we enforce chunk size limit Boris Feld <boris.feld@octobus.net> parents: 40684 diff changeset	219 >>> list(_slicechunktosize(revlog, [1, 3], 3))
9c3c697267db sparse-revlog: rework the way we enforce chunk size limit Boris Feld <boris.feld@octobus.net> parents: 40684 diff changeset	220 [[1], [3]]
9c3c697267db sparse-revlog: rework the way we enforce chunk size limit Boris Feld <boris.feld@octobus.net> parents: 40684 diff changeset	221 >>> list(_slicechunktosize(revlog, [1, 2, 3], 3))
9c3c697267db sparse-revlog: rework the way we enforce chunk size limit Boris Feld <boris.feld@octobus.net> parents: 40684 diff changeset	222 [[1], [2, 3]]
9c3c697267db sparse-revlog: rework the way we enforce chunk size limit Boris Feld <boris.feld@octobus.net> parents: 40684 diff changeset	223 >>> list(_slicechunktosize(revlog, [3, 5], 3))
9c3c697267db sparse-revlog: rework the way we enforce chunk size limit Boris Feld <boris.feld@octobus.net> parents: 40684 diff changeset	224 [[3], [5]]
9c3c697267db sparse-revlog: rework the way we enforce chunk size limit Boris Feld <boris.feld@octobus.net> parents: 40684 diff changeset	225 >>> list(_slicechunktosize(revlog, [3, 4, 5], 3))
9c3c697267db sparse-revlog: rework the way we enforce chunk size limit Boris Feld <boris.feld@octobus.net> parents: 40684 diff changeset	226 [[3], [4, 5]]
9c3c697267db sparse-revlog: rework the way we enforce chunk size limit Boris Feld <boris.feld@octobus.net> parents: 40684 diff changeset	227 >>> list(_slicechunktosize(revlog, [5, 6, 7, 8], 3))
9c3c697267db sparse-revlog: rework the way we enforce chunk size limit Boris Feld <boris.feld@octobus.net> parents: 40684 diff changeset	228 [[5], [6, 7, 8]]
9c3c697267db sparse-revlog: rework the way we enforce chunk size limit Boris Feld <boris.feld@octobus.net> parents: 40684 diff changeset	229 >>> list(_slicechunktosize(revlog, [0, 1, 2, 3, 4, 5, 6, 7, 8], 3))
9c3c697267db sparse-revlog: rework the way we enforce chunk size limit Boris Feld <boris.feld@octobus.net> parents: 40684 diff changeset	230 [[0], [1, 2], [3], [5], [6, 7, 8]]
9c3c697267db sparse-revlog: rework the way we enforce chunk size limit Boris Feld <boris.feld@octobus.net> parents: 40684 diff changeset	231
9c3c697267db sparse-revlog: rework the way we enforce chunk size limit Boris Feld <boris.feld@octobus.net> parents: 40684 diff changeset	232 Case with too large individual chunk (must return valid chunk)
9c3c697267db sparse-revlog: rework the way we enforce chunk size limit Boris Feld <boris.feld@octobus.net> parents: 40684 diff changeset	233 >>> list(_slicechunktosize(revlog, [0, 1], 2))
9c3c697267db sparse-revlog: rework the way we enforce chunk size limit Boris Feld <boris.feld@octobus.net> parents: 40684 diff changeset	234 [[0], [1]]
9c3c697267db sparse-revlog: rework the way we enforce chunk size limit Boris Feld <boris.feld@octobus.net> parents: 40684 diff changeset	235 >>> list(_slicechunktosize(revlog, [1, 3], 1))
9c3c697267db sparse-revlog: rework the way we enforce chunk size limit Boris Feld <boris.feld@octobus.net> parents: 40684 diff changeset	236 [[1], [3]]
9c3c697267db sparse-revlog: rework the way we enforce chunk size limit Boris Feld <boris.feld@octobus.net> parents: 40684 diff changeset	237 >>> list(_slicechunktosize(revlog, [3, 4, 5], 2))
9c3c697267db sparse-revlog: rework the way we enforce chunk size limit Boris Feld <boris.feld@octobus.net> parents: 40684 diff changeset	238 [[3], [5]]
9c3c697267db sparse-revlog: rework the way we enforce chunk size limit Boris Feld <boris.feld@octobus.net> parents: 40684 diff changeset	239
9c3c697267db sparse-revlog: rework the way we enforce chunk size limit Boris Feld <boris.feld@octobus.net> parents: 40684 diff changeset	240 == mixed case ==
9c3c697267db sparse-revlog: rework the way we enforce chunk size limit Boris Feld <boris.feld@octobus.net> parents: 40684 diff changeset	241 >>> revlog = _testrevlog(data, snapshot=[0, 1, 2])
9c3c697267db sparse-revlog: rework the way we enforce chunk size limit Boris Feld <boris.feld@octobus.net> parents: 40684 diff changeset	242 >>> list(_slicechunktosize(revlog, list(range(9)), 5))
9c3c697267db sparse-revlog: rework the way we enforce chunk size limit Boris Feld <boris.feld@octobus.net> parents: 40684 diff changeset	243 [[0, 1], [2], [3, 4, 5], [6, 7, 8]]
38642 e59e27e52297 revlog: add function to slice chunk down to a given size Boris Feld <boris.feld@octobus.net> parents: 38641 diff changeset	244 """
e59e27e52297 revlog: add function to slice chunk down to a given size Boris Feld <boris.feld@octobus.net> parents: 38641 diff changeset	245 assert targetsize is None or 0 <= targetsize
40685 9c3c697267db sparse-revlog: rework the way we enforce chunk size limit Boris Feld <boris.feld@octobus.net> parents: 40684 diff changeset	246 startdata = revlog.start(revs[0])
9c3c697267db sparse-revlog: rework the way we enforce chunk size limit Boris Feld <boris.feld@octobus.net> parents: 40684 diff changeset	247 enddata = revlog.end(revs[-1])
9c3c697267db sparse-revlog: rework the way we enforce chunk size limit Boris Feld <boris.feld@octobus.net> parents: 40684 diff changeset	248 fullspan = enddata - startdata
9c3c697267db sparse-revlog: rework the way we enforce chunk size limit Boris Feld <boris.feld@octobus.net> parents: 40684 diff changeset	249 if targetsize is None or fullspan <= targetsize:
38642 e59e27e52297 revlog: add function to slice chunk down to a given size Boris Feld <boris.feld@octobus.net> parents: 38641 diff changeset	250 yield revs
e59e27e52297 revlog: add function to slice chunk down to a given size Boris Feld <boris.feld@octobus.net> parents: 38641 diff changeset	251 return
e59e27e52297 revlog: add function to slice chunk down to a given size Boris Feld <boris.feld@octobus.net> parents: 38641 diff changeset	252
e59e27e52297 revlog: add function to slice chunk down to a given size Boris Feld <boris.feld@octobus.net> parents: 38641 diff changeset	253 startrevidx = 0
40698 2eb48aa0acce sparse-revlog: align endrevidx usages in the _slicechunktosize Boris Feld <boris.feld@octobus.net> parents: 40697 diff changeset	254 endrevidx = 1
38642 e59e27e52297 revlog: add function to slice chunk down to a given size Boris Feld <boris.feld@octobus.net> parents: 38641 diff changeset	255 iterrevs = enumerate(revs)
e59e27e52297 revlog: add function to slice chunk down to a given size Boris Feld <boris.feld@octobus.net> parents: 38641 diff changeset	256 next(iterrevs) # skip first rev.
40685 9c3c697267db sparse-revlog: rework the way we enforce chunk size limit Boris Feld <boris.feld@octobus.net> parents: 40684 diff changeset	257 # first step: get snapshots out of the way
38642 e59e27e52297 revlog: add function to slice chunk down to a given size Boris Feld <boris.feld@octobus.net> parents: 38641 diff changeset	258 for idx, r in iterrevs:
e59e27e52297 revlog: add function to slice chunk down to a given size Boris Feld <boris.feld@octobus.net> parents: 38641 diff changeset	259 span = revlog.end(r) - startdata
40685 9c3c697267db sparse-revlog: rework the way we enforce chunk size limit Boris Feld <boris.feld@octobus.net> parents: 40684 diff changeset	260 snapshot = revlog.issnapshot(r)
9c3c697267db sparse-revlog: rework the way we enforce chunk size limit Boris Feld <boris.feld@octobus.net> parents: 40684 diff changeset	261 if span <= targetsize and snapshot:
40698 2eb48aa0acce sparse-revlog: align endrevidx usages in the _slicechunktosize Boris Feld <boris.feld@octobus.net> parents: 40697 diff changeset	262 endrevidx = idx + 1
38642 e59e27e52297 revlog: add function to slice chunk down to a given size Boris Feld <boris.feld@octobus.net> parents: 38641 diff changeset	263 else:
40698 2eb48aa0acce sparse-revlog: align endrevidx usages in the _slicechunktosize Boris Feld <boris.feld@octobus.net> parents: 40697 diff changeset	264 chunk = _trimchunk(revlog, revs, startrevidx, endrevidx)
38642 e59e27e52297 revlog: add function to slice chunk down to a given size Boris Feld <boris.feld@octobus.net> parents: 38641 diff changeset	265 if chunk:
e59e27e52297 revlog: add function to slice chunk down to a given size Boris Feld <boris.feld@octobus.net> parents: 38641 diff changeset	266 yield chunk
e59e27e52297 revlog: add function to slice chunk down to a given size Boris Feld <boris.feld@octobus.net> parents: 38641 diff changeset	267 startrevidx = idx
e59e27e52297 revlog: add function to slice chunk down to a given size Boris Feld <boris.feld@octobus.net> parents: 38641 diff changeset	268 startdata = revlog.start(r)
40698 2eb48aa0acce sparse-revlog: align endrevidx usages in the _slicechunktosize Boris Feld <boris.feld@octobus.net> parents: 40697 diff changeset	269 endrevidx = idx + 1
40685 9c3c697267db sparse-revlog: rework the way we enforce chunk size limit Boris Feld <boris.feld@octobus.net> parents: 40684 diff changeset	270 if not snapshot:
9c3c697267db sparse-revlog: rework the way we enforce chunk size limit Boris Feld <boris.feld@octobus.net> parents: 40684 diff changeset	271 break
9c3c697267db sparse-revlog: rework the way we enforce chunk size limit Boris Feld <boris.feld@octobus.net> parents: 40684 diff changeset	272
9c3c697267db sparse-revlog: rework the way we enforce chunk size limit Boris Feld <boris.feld@octobus.net> parents: 40684 diff changeset	273 # for the others, we use binary slicing to quickly converge toward valid
9c3c697267db sparse-revlog: rework the way we enforce chunk size limit Boris Feld <boris.feld@octobus.net> parents: 40684 diff changeset	274 # chunks (otherwise, we might end up looking for start/end of many
9c3c697267db sparse-revlog: rework the way we enforce chunk size limit Boris Feld <boris.feld@octobus.net> parents: 40684 diff changeset	275 # revisions). This logic is not looking for the perfect slicing point, it
9c3c697267db sparse-revlog: rework the way we enforce chunk size limit Boris Feld <boris.feld@octobus.net> parents: 40684 diff changeset	276 # focuses on quickly converging toward valid chunks.
9c3c697267db sparse-revlog: rework the way we enforce chunk size limit Boris Feld <boris.feld@octobus.net> parents: 40684 diff changeset	277 nbitem = len(revs)
9c3c697267db sparse-revlog: rework the way we enforce chunk size limit Boris Feld <boris.feld@octobus.net> parents: 40684 diff changeset	278 while (enddata - startdata) > targetsize:
9c3c697267db sparse-revlog: rework the way we enforce chunk size limit Boris Feld <boris.feld@octobus.net> parents: 40684 diff changeset	279 endrevidx = nbitem
9c3c697267db sparse-revlog: rework the way we enforce chunk size limit Boris Feld <boris.feld@octobus.net> parents: 40684 diff changeset	280 if nbitem - startrevidx <= 1:
9c3c697267db sparse-revlog: rework the way we enforce chunk size limit Boris Feld <boris.feld@octobus.net> parents: 40684 diff changeset	281 break # protect against individual chunk larger than limit
9c3c697267db sparse-revlog: rework the way we enforce chunk size limit Boris Feld <boris.feld@octobus.net> parents: 40684 diff changeset	282 localenddata = revlog.end(revs[endrevidx - 1])
9c3c697267db sparse-revlog: rework the way we enforce chunk size limit Boris Feld <boris.feld@octobus.net> parents: 40684 diff changeset	283 span = localenddata - startdata
40697 fd1d41ccbe38 sparse-revlog: use `span` variable as intended Boris Feld <boris.feld@octobus.net> parents: 40685 diff changeset	284 while span > targetsize:
40685 9c3c697267db sparse-revlog: rework the way we enforce chunk size limit Boris Feld <boris.feld@octobus.net> parents: 40684 diff changeset	285 if endrevidx - startrevidx <= 1:
9c3c697267db sparse-revlog: rework the way we enforce chunk size limit Boris Feld <boris.feld@octobus.net> parents: 40684 diff changeset	286 break # protect against individual chunk larger than limit
9c3c697267db sparse-revlog: rework the way we enforce chunk size limit Boris Feld <boris.feld@octobus.net> parents: 40684 diff changeset	287 endrevidx -= (endrevidx - startrevidx) // 2
9c3c697267db sparse-revlog: rework the way we enforce chunk size limit Boris Feld <boris.feld@octobus.net> parents: 40684 diff changeset	288 localenddata = revlog.end(revs[endrevidx - 1])
9c3c697267db sparse-revlog: rework the way we enforce chunk size limit Boris Feld <boris.feld@octobus.net> parents: 40684 diff changeset	289 span = localenddata - startdata
9c3c697267db sparse-revlog: rework the way we enforce chunk size limit Boris Feld <boris.feld@octobus.net> parents: 40684 diff changeset	290 chunk = _trimchunk(revlog, revs, startrevidx, endrevidx)
9c3c697267db sparse-revlog: rework the way we enforce chunk size limit Boris Feld <boris.feld@octobus.net> parents: 40684 diff changeset	291 if chunk:
9c3c697267db sparse-revlog: rework the way we enforce chunk size limit Boris Feld <boris.feld@octobus.net> parents: 40684 diff changeset	292 yield chunk
9c3c697267db sparse-revlog: rework the way we enforce chunk size limit Boris Feld <boris.feld@octobus.net> parents: 40684 diff changeset	293 startrevidx = endrevidx
9c3c697267db sparse-revlog: rework the way we enforce chunk size limit Boris Feld <boris.feld@octobus.net> parents: 40684 diff changeset	294 startdata = revlog.start(revs[startrevidx])
9c3c697267db sparse-revlog: rework the way we enforce chunk size limit Boris Feld <boris.feld@octobus.net> parents: 40684 diff changeset	295
9c3c697267db sparse-revlog: rework the way we enforce chunk size limit Boris Feld <boris.feld@octobus.net> parents: 40684 diff changeset	296 chunk = _trimchunk(revlog, revs, startrevidx)
9c3c697267db sparse-revlog: rework the way we enforce chunk size limit Boris Feld <boris.feld@octobus.net> parents: 40684 diff changeset	297 if chunk:
9c3c697267db sparse-revlog: rework the way we enforce chunk size limit Boris Feld <boris.feld@octobus.net> parents: 40684 diff changeset	298 yield chunk
38642 e59e27e52297 revlog: add function to slice chunk down to a given size Boris Feld <boris.feld@octobus.net> parents: 38641 diff changeset	299
40651 3ac23dad6364 sparse-revlog: drop unused deltainfo parameter from _slicechunktodensity Boris Feld <boris.feld@octobus.net> parents: 40650 diff changeset	300 def _slicechunktodensity(revlog, revs, targetdensity=0.5,
38718 f8762ea73e0d sparse-revlog: implement algorithm to write sparse delta chains (issue5480) Paul Morelle <paul.morelle@octobus.net> parents: 38717 diff changeset	301 mingapsize=0):
38641 feba6be0941b revlog: extract density based slicing into its own function Boris Feld <boris.feld@octobus.net> parents: 38640 diff changeset	302 """slice revs to reduce the amount of unrelated data to be read from disk.
feba6be0941b revlog: extract density based slicing into its own function Boris Feld <boris.feld@octobus.net> parents: 38640 diff changeset	303
feba6be0941b revlog: extract density based slicing into its own function Boris Feld <boris.feld@octobus.net> parents: 38640 diff changeset	304 ``revs`` is sliced into groups that should be read in one time.
feba6be0941b revlog: extract density based slicing into its own function Boris Feld <boris.feld@octobus.net> parents: 38640 diff changeset	305 Assume that revs are sorted.
feba6be0941b revlog: extract density based slicing into its own function Boris Feld <boris.feld@octobus.net> parents: 38640 diff changeset	306
feba6be0941b revlog: extract density based slicing into its own function Boris Feld <boris.feld@octobus.net> parents: 38640 diff changeset	307 The initial chunk is sliced until the overall density (payload/chunks-span
feba6be0941b revlog: extract density based slicing into its own function Boris Feld <boris.feld@octobus.net> parents: 38640 diff changeset	308 ratio) is above `targetdensity`. No gap smaller than `mingapsize` is
feba6be0941b revlog: extract density based slicing into its own function Boris Feld <boris.feld@octobus.net> parents: 38640 diff changeset	309 skipped.
feba6be0941b revlog: extract density based slicing into its own function Boris Feld <boris.feld@octobus.net> parents: 38640 diff changeset	310
feba6be0941b revlog: extract density based slicing into its own function Boris Feld <boris.feld@octobus.net> parents: 38640 diff changeset	311 >>> revlog = _testrevlog([
feba6be0941b revlog: extract density based slicing into its own function Boris Feld <boris.feld@octobus.net> parents: 38640 diff changeset	312 ... 5, #00 (5)
feba6be0941b revlog: extract density based slicing into its own function Boris Feld <boris.feld@octobus.net> parents: 38640 diff changeset	313 ... 10, #01 (5)
feba6be0941b revlog: extract density based slicing into its own function Boris Feld <boris.feld@octobus.net> parents: 38640 diff changeset	314 ... 12, #02 (2)
feba6be0941b revlog: extract density based slicing into its own function Boris Feld <boris.feld@octobus.net> parents: 38640 diff changeset	315 ... 12, #03 (empty)
feba6be0941b revlog: extract density based slicing into its own function Boris Feld <boris.feld@octobus.net> parents: 38640 diff changeset	316 ... 27, #04 (15)
feba6be0941b revlog: extract density based slicing into its own function Boris Feld <boris.feld@octobus.net> parents: 38640 diff changeset	317 ... 31, #05 (4)
feba6be0941b revlog: extract density based slicing into its own function Boris Feld <boris.feld@octobus.net> parents: 38640 diff changeset	318 ... 31, #06 (empty)
feba6be0941b revlog: extract density based slicing into its own function Boris Feld <boris.feld@octobus.net> parents: 38640 diff changeset	319 ... 42, #07 (11)
feba6be0941b revlog: extract density based slicing into its own function Boris Feld <boris.feld@octobus.net> parents: 38640 diff changeset	320 ... 47, #08 (5)
feba6be0941b revlog: extract density based slicing into its own function Boris Feld <boris.feld@octobus.net> parents: 38640 diff changeset	321 ... 47, #09 (empty)
feba6be0941b revlog: extract density based slicing into its own function Boris Feld <boris.feld@octobus.net> parents: 38640 diff changeset	322 ... 48, #10 (1)
feba6be0941b revlog: extract density based slicing into its own function Boris Feld <boris.feld@octobus.net> parents: 38640 diff changeset	323 ... 51, #11 (3)
feba6be0941b revlog: extract density based slicing into its own function Boris Feld <boris.feld@octobus.net> parents: 38640 diff changeset	324 ... 74, #12 (23)
feba6be0941b revlog: extract density based slicing into its own function Boris Feld <boris.feld@octobus.net> parents: 38640 diff changeset	325 ... 85, #13 (11)
feba6be0941b revlog: extract density based slicing into its own function Boris Feld <boris.feld@octobus.net> parents: 38640 diff changeset	326 ... 86, #14 (1)
feba6be0941b revlog: extract density based slicing into its own function Boris Feld <boris.feld@octobus.net> parents: 38640 diff changeset	327 ... 91, #15 (5)
feba6be0941b revlog: extract density based slicing into its own function Boris Feld <boris.feld@octobus.net> parents: 38640 diff changeset	328 ... ])
feba6be0941b revlog: extract density based slicing into its own function Boris Feld <boris.feld@octobus.net> parents: 38640 diff changeset	329
38655 cd1c484e31e8 revlog: adjust doctest examples to be portable to Python 3 Augie Fackler <augie@google.com> parents: 38644 diff changeset	330 >>> list(_slicechunktodensity(revlog, list(range(16))))
38641 feba6be0941b revlog: extract density based slicing into its own function Boris Feld <boris.feld@octobus.net> parents: 38640 diff changeset	331 [[0, 1, 2, 3, 4, 5, 6, 7, 8, 9, 10, 11, 12, 13, 14, 15]]
feba6be0941b revlog: extract density based slicing into its own function Boris Feld <boris.feld@octobus.net> parents: 38640 diff changeset	332 >>> list(_slicechunktodensity(revlog, [0, 15]))
feba6be0941b revlog: extract density based slicing into its own function Boris Feld <boris.feld@octobus.net> parents: 38640 diff changeset	333 [[0], [15]]
feba6be0941b revlog: extract density based slicing into its own function Boris Feld <boris.feld@octobus.net> parents: 38640 diff changeset	334 >>> list(_slicechunktodensity(revlog, [0, 11, 15]))
feba6be0941b revlog: extract density based slicing into its own function Boris Feld <boris.feld@octobus.net> parents: 38640 diff changeset	335 [[0], [11], [15]]
feba6be0941b revlog: extract density based slicing into its own function Boris Feld <boris.feld@octobus.net> parents: 38640 diff changeset	336 >>> list(_slicechunktodensity(revlog, [0, 11, 13, 15]))
feba6be0941b revlog: extract density based slicing into its own function Boris Feld <boris.feld@octobus.net> parents: 38640 diff changeset	337 [[0], [11, 13, 15]]
feba6be0941b revlog: extract density based slicing into its own function Boris Feld <boris.feld@octobus.net> parents: 38640 diff changeset	338 >>> list(_slicechunktodensity(revlog, [1, 2, 3, 5, 8, 10, 11, 14]))
feba6be0941b revlog: extract density based slicing into its own function Boris Feld <boris.feld@octobus.net> parents: 38640 diff changeset	339 [[1, 2], [5, 8, 10, 11], [14]]
feba6be0941b revlog: extract density based slicing into its own function Boris Feld <boris.feld@octobus.net> parents: 38640 diff changeset	340 >>> list(_slicechunktodensity(revlog, [1, 2, 3, 5, 8, 10, 11, 14],
feba6be0941b revlog: extract density based slicing into its own function Boris Feld <boris.feld@octobus.net> parents: 38640 diff changeset	341 ... mingapsize=20))
feba6be0941b revlog: extract density based slicing into its own function Boris Feld <boris.feld@octobus.net> parents: 38640 diff changeset	342 [[1, 2, 3, 5, 8, 10, 11], [14]]
feba6be0941b revlog: extract density based slicing into its own function Boris Feld <boris.feld@octobus.net> parents: 38640 diff changeset	343 >>> list(_slicechunktodensity(revlog, [1, 2, 3, 5, 8, 10, 11, 14],
feba6be0941b revlog: extract density based slicing into its own function Boris Feld <boris.feld@octobus.net> parents: 38640 diff changeset	344 ... targetdensity=0.95))
feba6be0941b revlog: extract density based slicing into its own function Boris Feld <boris.feld@octobus.net> parents: 38640 diff changeset	345 [[1, 2], [5], [8, 10, 11], [14]]
feba6be0941b revlog: extract density based slicing into its own function Boris Feld <boris.feld@octobus.net> parents: 38640 diff changeset	346 >>> list(_slicechunktodensity(revlog, [1, 2, 3, 5, 8, 10, 11, 14],
feba6be0941b revlog: extract density based slicing into its own function Boris Feld <boris.feld@octobus.net> parents: 38640 diff changeset	347 ... targetdensity=0.95, mingapsize=12))
feba6be0941b revlog: extract density based slicing into its own function Boris Feld <boris.feld@octobus.net> parents: 38640 diff changeset	348 [[1, 2], [5, 8, 10, 11], [14]]
feba6be0941b revlog: extract density based slicing into its own function Boris Feld <boris.feld@octobus.net> parents: 38640 diff changeset	349 """
34824 e2ad93bcc084 revlog: introduce an experimental flag to slice chunks reads when too sparse Paul Morelle <paul.morelle@octobus.net> parents: 34823 diff changeset	350 start = revlog.start
e2ad93bcc084 revlog: introduce an experimental flag to slice chunks reads when too sparse Paul Morelle <paul.morelle@octobus.net> parents: 34823 diff changeset	351 length = revlog.length
e2ad93bcc084 revlog: introduce an experimental flag to slice chunks reads when too sparse Paul Morelle <paul.morelle@octobus.net> parents: 34823 diff changeset	352
34880 9e18ab7f7240 sparse-read: move from a recursive-based approach to a heap-based one Boris Feld <boris.feld@octobus.net> parents: 34825 diff changeset	353 if len(revs) <= 1:
9e18ab7f7240 sparse-read: move from a recursive-based approach to a heap-based one Boris Feld <boris.feld@octobus.net> parents: 34825 diff changeset	354 yield revs
9e18ab7f7240 sparse-read: move from a recursive-based approach to a heap-based one Boris Feld <boris.feld@octobus.net> parents: 34825 diff changeset	355 return
34824 e2ad93bcc084 revlog: introduce an experimental flag to slice chunks reads when too sparse Paul Morelle <paul.morelle@octobus.net> parents: 34823 diff changeset	356
40651 3ac23dad6364 sparse-revlog: drop unused deltainfo parameter from _slicechunktodensity Boris Feld <boris.feld@octobus.net> parents: 40650 diff changeset	357 deltachainspan = segmentspan(revlog, revs)
34880 9e18ab7f7240 sparse-read: move from a recursive-based approach to a heap-based one Boris Feld <boris.feld@octobus.net> parents: 34825 diff changeset	358
38641 feba6be0941b revlog: extract density based slicing into its own function Boris Feld <boris.feld@octobus.net> parents: 38640 diff changeset	359 if deltachainspan < mingapsize:
38635 d083ae26c325 revlog: early return in _slicechunk when span is already small enough Boris Feld <boris.feld@octobus.net> parents: 38634 diff changeset	360 yield revs
d083ae26c325 revlog: early return in _slicechunk when span is already small enough Boris Feld <boris.feld@octobus.net> parents: 38634 diff changeset	361 return
d083ae26c325 revlog: early return in _slicechunk when span is already small enough Boris Feld <boris.feld@octobus.net> parents: 38634 diff changeset	362
38718 f8762ea73e0d sparse-revlog: implement algorithm to write sparse delta chains (issue5480) Paul Morelle <paul.morelle@octobus.net> parents: 38717 diff changeset	363 readdata = deltachainspan
40653 bfbfd15d65bd sparse-revlog: fast-path before computing payload size Boris Feld <boris.feld@octobus.net> parents: 40652 diff changeset	364 chainpayload = sum(length(r) for r in revs)
34824 e2ad93bcc084 revlog: introduce an experimental flag to slice chunks reads when too sparse Paul Morelle <paul.morelle@octobus.net> parents: 34823 diff changeset	365
34880 9e18ab7f7240 sparse-read: move from a recursive-based approach to a heap-based one Boris Feld <boris.feld@octobus.net> parents: 34825 diff changeset	366 if deltachainspan:
9e18ab7f7240 sparse-read: move from a recursive-based approach to a heap-based one Boris Feld <boris.feld@octobus.net> parents: 34825 diff changeset	367 density = chainpayload / float(deltachainspan)
9e18ab7f7240 sparse-read: move from a recursive-based approach to a heap-based one Boris Feld <boris.feld@octobus.net> parents: 34825 diff changeset	368 else:
9e18ab7f7240 sparse-read: move from a recursive-based approach to a heap-based one Boris Feld <boris.feld@octobus.net> parents: 34825 diff changeset	369 density = 1.0
34824 e2ad93bcc084 revlog: introduce an experimental flag to slice chunks reads when too sparse Paul Morelle <paul.morelle@octobus.net> parents: 34823 diff changeset	370
38641 feba6be0941b revlog: extract density based slicing into its own function Boris Feld <boris.feld@octobus.net> parents: 38640 diff changeset	371 if density >= targetdensity:
38634 f0ea8b847831 revlog: early return in _slicechunk when density is already good Paul Morelle <paul.morelle@octobus.net> parents: 38632 diff changeset	372 yield revs
f0ea8b847831 revlog: early return in _slicechunk when density is already good Paul Morelle <paul.morelle@octobus.net> parents: 38632 diff changeset	373 return
f0ea8b847831 revlog: early return in _slicechunk when density is already good Paul Morelle <paul.morelle@octobus.net> parents: 38632 diff changeset	374
34880 9e18ab7f7240 sparse-read: move from a recursive-based approach to a heap-based one Boris Feld <boris.feld@octobus.net> parents: 34825 diff changeset	375 # Store the gaps in a heap to have them sorted by decreasing size
40654 54de23400b2a sparse-revlog: stop using a heap to track gaps Boris Feld <boris.feld@octobus.net> parents: 40653 diff changeset	376 gaps = []
34880 9e18ab7f7240 sparse-read: move from a recursive-based approach to a heap-based one Boris Feld <boris.feld@octobus.net> parents: 34825 diff changeset	377 prevend = None
9e18ab7f7240 sparse-read: move from a recursive-based approach to a heap-based one Boris Feld <boris.feld@octobus.net> parents: 34825 diff changeset	378 for i, rev in enumerate(revs):
40651 3ac23dad6364 sparse-revlog: drop unused deltainfo parameter from _slicechunktodensity Boris Feld <boris.feld@octobus.net> parents: 40650 diff changeset	379 revstart = start(rev)
3ac23dad6364 sparse-revlog: drop unused deltainfo parameter from _slicechunktodensity Boris Feld <boris.feld@octobus.net> parents: 40650 diff changeset	380 revlen = length(rev)
34824 e2ad93bcc084 revlog: introduce an experimental flag to slice chunks reads when too sparse Paul Morelle <paul.morelle@octobus.net> parents: 34823 diff changeset	381
34898 1bde8e8e5de0 sparse-read: ignore trailing empty revs in each read chunk Paul Morelle <paul.morelle@octobus.net> parents: 34881 diff changeset	382 # Skip empty revisions to form larger holes
1bde8e8e5de0 sparse-read: ignore trailing empty revs in each read chunk Paul Morelle <paul.morelle@octobus.net> parents: 34881 diff changeset	383 if revlen == 0:
1bde8e8e5de0 sparse-read: ignore trailing empty revs in each read chunk Paul Morelle <paul.morelle@octobus.net> parents: 34881 diff changeset	384 continue
1bde8e8e5de0 sparse-read: ignore trailing empty revs in each read chunk Paul Morelle <paul.morelle@octobus.net> parents: 34881 diff changeset	385
34880 9e18ab7f7240 sparse-read: move from a recursive-based approach to a heap-based one Boris Feld <boris.feld@octobus.net> parents: 34825 diff changeset	386 if prevend is not None:
9e18ab7f7240 sparse-read: move from a recursive-based approach to a heap-based one Boris Feld <boris.feld@octobus.net> parents: 34825 diff changeset	387 gapsize = revstart - prevend
34881 8c9b08a0c48c sparse-read: skip gaps too small to be worth splitting Paul Morelle <paul.morelle@octobus.net> parents: 34880 diff changeset	388 # only consider holes that are large enough
38641 feba6be0941b revlog: extract density based slicing into its own function Boris Feld <boris.feld@octobus.net> parents: 38640 diff changeset	389 if gapsize > mingapsize:
40654 54de23400b2a sparse-revlog: stop using a heap to track gaps Boris Feld <boris.feld@octobus.net> parents: 40653 diff changeset	390 gaps.append((gapsize, i))
34880 9e18ab7f7240 sparse-read: move from a recursive-based approach to a heap-based one Boris Feld <boris.feld@octobus.net> parents: 34825 diff changeset	391
9e18ab7f7240 sparse-read: move from a recursive-based approach to a heap-based one Boris Feld <boris.feld@octobus.net> parents: 34825 diff changeset	392 prevend = revstart + revlen
40654 54de23400b2a sparse-revlog: stop using a heap to track gaps Boris Feld <boris.feld@octobus.net> parents: 40653 diff changeset	393 # sort the gaps to pop them from largest to small
54de23400b2a sparse-revlog: stop using a heap to track gaps Boris Feld <boris.feld@octobus.net> parents: 40653 diff changeset	394 gaps.sort()
34880 9e18ab7f7240 sparse-read: move from a recursive-based approach to a heap-based one Boris Feld <boris.feld@octobus.net> parents: 34825 diff changeset	395
9e18ab7f7240 sparse-read: move from a recursive-based approach to a heap-based one Boris Feld <boris.feld@octobus.net> parents: 34825 diff changeset	396 # Collect the indices of the largest holes until the density is acceptable
40655 526ee887c4d5 sparse-revlog: stop using a heap to track selected gap Boris Feld <boris.feld@octobus.net> parents: 40654 diff changeset	397 selected = []
40654 54de23400b2a sparse-revlog: stop using a heap to track gaps Boris Feld <boris.feld@octobus.net> parents: 40653 diff changeset	398 while gaps and density < targetdensity:
54de23400b2a sparse-revlog: stop using a heap to track gaps Boris Feld <boris.feld@octobus.net> parents: 40653 diff changeset	399 gapsize, gapidx = gaps.pop()
34880 9e18ab7f7240 sparse-read: move from a recursive-based approach to a heap-based one Boris Feld <boris.feld@octobus.net> parents: 34825 diff changeset	400
40655 526ee887c4d5 sparse-revlog: stop using a heap to track selected gap Boris Feld <boris.feld@octobus.net> parents: 40654 diff changeset	401 selected.append(gapidx)
34824 e2ad93bcc084 revlog: introduce an experimental flag to slice chunks reads when too sparse Paul Morelle <paul.morelle@octobus.net> parents: 34823 diff changeset	402
34880 9e18ab7f7240 sparse-read: move from a recursive-based approach to a heap-based one Boris Feld <boris.feld@octobus.net> parents: 34825 diff changeset	403 # the gap sizes are stored as negatives to be sorted decreasingly
9e18ab7f7240 sparse-read: move from a recursive-based approach to a heap-based one Boris Feld <boris.feld@octobus.net> parents: 34825 diff changeset	404 # by the heap
40654 54de23400b2a sparse-revlog: stop using a heap to track gaps Boris Feld <boris.feld@octobus.net> parents: 40653 diff changeset	405 readdata -= gapsize
34880 9e18ab7f7240 sparse-read: move from a recursive-based approach to a heap-based one Boris Feld <boris.feld@octobus.net> parents: 34825 diff changeset	406 if readdata > 0:
9e18ab7f7240 sparse-read: move from a recursive-based approach to a heap-based one Boris Feld <boris.feld@octobus.net> parents: 34825 diff changeset	407 density = chainpayload / float(readdata)
9e18ab7f7240 sparse-read: move from a recursive-based approach to a heap-based one Boris Feld <boris.feld@octobus.net> parents: 34825 diff changeset	408 else:
9e18ab7f7240 sparse-read: move from a recursive-based approach to a heap-based one Boris Feld <boris.feld@octobus.net> parents: 34825 diff changeset	409 density = 1.0
40655 526ee887c4d5 sparse-revlog: stop using a heap to track selected gap Boris Feld <boris.feld@octobus.net> parents: 40654 diff changeset	410 selected.sort()
34824 e2ad93bcc084 revlog: introduce an experimental flag to slice chunks reads when too sparse Paul Morelle <paul.morelle@octobus.net> parents: 34823 diff changeset	411
34880 9e18ab7f7240 sparse-read: move from a recursive-based approach to a heap-based one Boris Feld <boris.feld@octobus.net> parents: 34825 diff changeset	412 # Cut the revs at collected indices
9e18ab7f7240 sparse-read: move from a recursive-based approach to a heap-based one Boris Feld <boris.feld@octobus.net> parents: 34825 diff changeset	413 previdx = 0
40655 526ee887c4d5 sparse-revlog: stop using a heap to track selected gap Boris Feld <boris.feld@octobus.net> parents: 40654 diff changeset	414 for idx in selected:
34898 1bde8e8e5de0 sparse-read: ignore trailing empty revs in each read chunk Paul Morelle <paul.morelle@octobus.net> parents: 34881 diff changeset	415
1bde8e8e5de0 sparse-read: ignore trailing empty revs in each read chunk Paul Morelle <paul.morelle@octobus.net> parents: 34881 diff changeset	416 chunk = _trimchunk(revlog, revs, previdx, idx)
1bde8e8e5de0 sparse-read: ignore trailing empty revs in each read chunk Paul Morelle <paul.morelle@octobus.net> parents: 34881 diff changeset	417 if chunk:
1bde8e8e5de0 sparse-read: ignore trailing empty revs in each read chunk Paul Morelle <paul.morelle@octobus.net> parents: 34881 diff changeset	418 yield chunk
1bde8e8e5de0 sparse-read: ignore trailing empty revs in each read chunk Paul Morelle <paul.morelle@octobus.net> parents: 34881 diff changeset	419
34880 9e18ab7f7240 sparse-read: move from a recursive-based approach to a heap-based one Boris Feld <boris.feld@octobus.net> parents: 34825 diff changeset	420 previdx = idx
34898 1bde8e8e5de0 sparse-read: ignore trailing empty revs in each read chunk Paul Morelle <paul.morelle@octobus.net> parents: 34881 diff changeset	421
1bde8e8e5de0 sparse-read: ignore trailing empty revs in each read chunk Paul Morelle <paul.morelle@octobus.net> parents: 34881 diff changeset	422 chunk = _trimchunk(revlog, revs, previdx)
1bde8e8e5de0 sparse-read: ignore trailing empty revs in each read chunk Paul Morelle <paul.morelle@octobus.net> parents: 34881 diff changeset	423 if chunk:
1bde8e8e5de0 sparse-read: ignore trailing empty revs in each read chunk Paul Morelle <paul.morelle@octobus.net> parents: 34881 diff changeset	424 yield chunk
34824 e2ad93bcc084 revlog: introduce an experimental flag to slice chunks reads when too sparse Paul Morelle <paul.morelle@octobus.net> parents: 34823 diff changeset	425
39357 655b5b465953 revlog: split functionality related to deltas computation in a new module Boris Feld <boris.feld@octobus.net> parents: 39356 diff changeset	426 def _trimchunk(revlog, revs, startidx, endidx=None):
655b5b465953 revlog: split functionality related to deltas computation in a new module Boris Feld <boris.feld@octobus.net> parents: 39356 diff changeset	427 """returns revs[startidx:endidx] without empty trailing revs
655b5b465953 revlog: split functionality related to deltas computation in a new module Boris Feld <boris.feld@octobus.net> parents: 39356 diff changeset	428
655b5b465953 revlog: split functionality related to deltas computation in a new module Boris Feld <boris.feld@octobus.net> parents: 39356 diff changeset	429 Doctest Setup
655b5b465953 revlog: split functionality related to deltas computation in a new module Boris Feld <boris.feld@octobus.net> parents: 39356 diff changeset	430 >>> revlog = _testrevlog([
655b5b465953 revlog: split functionality related to deltas computation in a new module Boris Feld <boris.feld@octobus.net> parents: 39356 diff changeset	431 ... 5, #0
655b5b465953 revlog: split functionality related to deltas computation in a new module Boris Feld <boris.feld@octobus.net> parents: 39356 diff changeset	432 ... 10, #1
655b5b465953 revlog: split functionality related to deltas computation in a new module Boris Feld <boris.feld@octobus.net> parents: 39356 diff changeset	433 ... 12, #2
655b5b465953 revlog: split functionality related to deltas computation in a new module Boris Feld <boris.feld@octobus.net> parents: 39356 diff changeset	434 ... 12, #3 (empty)
655b5b465953 revlog: split functionality related to deltas computation in a new module Boris Feld <boris.feld@octobus.net> parents: 39356 diff changeset	435 ... 17, #4
655b5b465953 revlog: split functionality related to deltas computation in a new module Boris Feld <boris.feld@octobus.net> parents: 39356 diff changeset	436 ... 21, #5
655b5b465953 revlog: split functionality related to deltas computation in a new module Boris Feld <boris.feld@octobus.net> parents: 39356 diff changeset	437 ... 21, #6 (empty)
655b5b465953 revlog: split functionality related to deltas computation in a new module Boris Feld <boris.feld@octobus.net> parents: 39356 diff changeset	438 ... ])
655b5b465953 revlog: split functionality related to deltas computation in a new module Boris Feld <boris.feld@octobus.net> parents: 39356 diff changeset	439
655b5b465953 revlog: split functionality related to deltas computation in a new module Boris Feld <boris.feld@octobus.net> parents: 39356 diff changeset	440 Contiguous cases:
655b5b465953 revlog: split functionality related to deltas computation in a new module Boris Feld <boris.feld@octobus.net> parents: 39356 diff changeset	441 >>> _trimchunk(revlog, [0, 1, 2, 3, 4, 5, 6], 0)
655b5b465953 revlog: split functionality related to deltas computation in a new module Boris Feld <boris.feld@octobus.net> parents: 39356 diff changeset	442 [0, 1, 2, 3, 4, 5]
655b5b465953 revlog: split functionality related to deltas computation in a new module Boris Feld <boris.feld@octobus.net> parents: 39356 diff changeset	443 >>> _trimchunk(revlog, [0, 1, 2, 3, 4, 5, 6], 0, 5)
655b5b465953 revlog: split functionality related to deltas computation in a new module Boris Feld <boris.feld@octobus.net> parents: 39356 diff changeset	444 [0, 1, 2, 3, 4]
655b5b465953 revlog: split functionality related to deltas computation in a new module Boris Feld <boris.feld@octobus.net> parents: 39356 diff changeset	445 >>> _trimchunk(revlog, [0, 1, 2, 3, 4, 5, 6], 0, 4)
655b5b465953 revlog: split functionality related to deltas computation in a new module Boris Feld <boris.feld@octobus.net> parents: 39356 diff changeset	446 [0, 1, 2]
655b5b465953 revlog: split functionality related to deltas computation in a new module Boris Feld <boris.feld@octobus.net> parents: 39356 diff changeset	447 >>> _trimchunk(revlog, [0, 1, 2, 3, 4, 5, 6], 2, 4)
655b5b465953 revlog: split functionality related to deltas computation in a new module Boris Feld <boris.feld@octobus.net> parents: 39356 diff changeset	448 [2]
655b5b465953 revlog: split functionality related to deltas computation in a new module Boris Feld <boris.feld@octobus.net> parents: 39356 diff changeset	449 >>> _trimchunk(revlog, [0, 1, 2, 3, 4, 5, 6], 3)
655b5b465953 revlog: split functionality related to deltas computation in a new module Boris Feld <boris.feld@octobus.net> parents: 39356 diff changeset	450 [3, 4, 5]
655b5b465953 revlog: split functionality related to deltas computation in a new module Boris Feld <boris.feld@octobus.net> parents: 39356 diff changeset	451 >>> _trimchunk(revlog, [0, 1, 2, 3, 4, 5, 6], 3, 5)
655b5b465953 revlog: split functionality related to deltas computation in a new module Boris Feld <boris.feld@octobus.net> parents: 39356 diff changeset	452 [3, 4]
655b5b465953 revlog: split functionality related to deltas computation in a new module Boris Feld <boris.feld@octobus.net> parents: 39356 diff changeset	453
655b5b465953 revlog: split functionality related to deltas computation in a new module Boris Feld <boris.feld@octobus.net> parents: 39356 diff changeset	454 Discontiguous cases:
655b5b465953 revlog: split functionality related to deltas computation in a new module Boris Feld <boris.feld@octobus.net> parents: 39356 diff changeset	455 >>> _trimchunk(revlog, [1, 3, 5, 6], 0)
655b5b465953 revlog: split functionality related to deltas computation in a new module Boris Feld <boris.feld@octobus.net> parents: 39356 diff changeset	456 [1, 3, 5]
655b5b465953 revlog: split functionality related to deltas computation in a new module Boris Feld <boris.feld@octobus.net> parents: 39356 diff changeset	457 >>> _trimchunk(revlog, [1, 3, 5, 6], 0, 2)
655b5b465953 revlog: split functionality related to deltas computation in a new module Boris Feld <boris.feld@octobus.net> parents: 39356 diff changeset	458 [1]
655b5b465953 revlog: split functionality related to deltas computation in a new module Boris Feld <boris.feld@octobus.net> parents: 39356 diff changeset	459 >>> _trimchunk(revlog, [1, 3, 5, 6], 1, 3)
655b5b465953 revlog: split functionality related to deltas computation in a new module Boris Feld <boris.feld@octobus.net> parents: 39356 diff changeset	460 [3, 5]
655b5b465953 revlog: split functionality related to deltas computation in a new module Boris Feld <boris.feld@octobus.net> parents: 39356 diff changeset	461 >>> _trimchunk(revlog, [1, 3, 5, 6], 1)
655b5b465953 revlog: split functionality related to deltas computation in a new module Boris Feld <boris.feld@octobus.net> parents: 39356 diff changeset	462 [3, 5]
655b5b465953 revlog: split functionality related to deltas computation in a new module Boris Feld <boris.feld@octobus.net> parents: 39356 diff changeset	463 """
655b5b465953 revlog: split functionality related to deltas computation in a new module Boris Feld <boris.feld@octobus.net> parents: 39356 diff changeset	464 length = revlog.length
655b5b465953 revlog: split functionality related to deltas computation in a new module Boris Feld <boris.feld@octobus.net> parents: 39356 diff changeset	465
655b5b465953 revlog: split functionality related to deltas computation in a new module Boris Feld <boris.feld@octobus.net> parents: 39356 diff changeset	466 if endidx is None:
655b5b465953 revlog: split functionality related to deltas computation in a new module Boris Feld <boris.feld@octobus.net> parents: 39356 diff changeset	467 endidx = len(revs)
655b5b465953 revlog: split functionality related to deltas computation in a new module Boris Feld <boris.feld@octobus.net> parents: 39356 diff changeset	468
655b5b465953 revlog: split functionality related to deltas computation in a new module Boris Feld <boris.feld@octobus.net> parents: 39356 diff changeset	469 # If we have a non-emtpy delta candidate, there are nothing to trim
655b5b465953 revlog: split functionality related to deltas computation in a new module Boris Feld <boris.feld@octobus.net> parents: 39356 diff changeset	470 if revs[endidx - 1] < len(revlog):
655b5b465953 revlog: split functionality related to deltas computation in a new module Boris Feld <boris.feld@octobus.net> parents: 39356 diff changeset	471 # Trim empty revs at the end, except the very first revision of a chain
655b5b465953 revlog: split functionality related to deltas computation in a new module Boris Feld <boris.feld@octobus.net> parents: 39356 diff changeset	472 while (endidx > 1
655b5b465953 revlog: split functionality related to deltas computation in a new module Boris Feld <boris.feld@octobus.net> parents: 39356 diff changeset	473 and endidx > startidx
655b5b465953 revlog: split functionality related to deltas computation in a new module Boris Feld <boris.feld@octobus.net> parents: 39356 diff changeset	474 and length(revs[endidx - 1]) == 0):
655b5b465953 revlog: split functionality related to deltas computation in a new module Boris Feld <boris.feld@octobus.net> parents: 39356 diff changeset	475 endidx -= 1
655b5b465953 revlog: split functionality related to deltas computation in a new module Boris Feld <boris.feld@octobus.net> parents: 39356 diff changeset	476
655b5b465953 revlog: split functionality related to deltas computation in a new module Boris Feld <boris.feld@octobus.net> parents: 39356 diff changeset	477 return revs[startidx:endidx]
655b5b465953 revlog: split functionality related to deltas computation in a new module Boris Feld <boris.feld@octobus.net> parents: 39356 diff changeset	478
40652 a32ccd32982b sparse-revlog: drop unused deltainfo parameter from segmentspan Boris Feld <boris.feld@octobus.net> parents: 40651 diff changeset	479 def segmentspan(revlog, revs):
39357 655b5b465953 revlog: split functionality related to deltas computation in a new module Boris Feld <boris.feld@octobus.net> parents: 39356 diff changeset	480 """Get the byte span of a segment of revisions
655b5b465953 revlog: split functionality related to deltas computation in a new module Boris Feld <boris.feld@octobus.net> parents: 39356 diff changeset	481
655b5b465953 revlog: split functionality related to deltas computation in a new module Boris Feld <boris.feld@octobus.net> parents: 39356 diff changeset	482 revs is a sorted array of revision numbers
655b5b465953 revlog: split functionality related to deltas computation in a new module Boris Feld <boris.feld@octobus.net> parents: 39356 diff changeset	483
655b5b465953 revlog: split functionality related to deltas computation in a new module Boris Feld <boris.feld@octobus.net> parents: 39356 diff changeset	484 >>> revlog = _testrevlog([
655b5b465953 revlog: split functionality related to deltas computation in a new module Boris Feld <boris.feld@octobus.net> parents: 39356 diff changeset	485 ... 5, #0
655b5b465953 revlog: split functionality related to deltas computation in a new module Boris Feld <boris.feld@octobus.net> parents: 39356 diff changeset	486 ... 10, #1
655b5b465953 revlog: split functionality related to deltas computation in a new module Boris Feld <boris.feld@octobus.net> parents: 39356 diff changeset	487 ... 12, #2
655b5b465953 revlog: split functionality related to deltas computation in a new module Boris Feld <boris.feld@octobus.net> parents: 39356 diff changeset	488 ... 12, #3 (empty)
655b5b465953 revlog: split functionality related to deltas computation in a new module Boris Feld <boris.feld@octobus.net> parents: 39356 diff changeset	489 ... 17, #4
655b5b465953 revlog: split functionality related to deltas computation in a new module Boris Feld <boris.feld@octobus.net> parents: 39356 diff changeset	490 ... ])
655b5b465953 revlog: split functionality related to deltas computation in a new module Boris Feld <boris.feld@octobus.net> parents: 39356 diff changeset	491
655b5b465953 revlog: split functionality related to deltas computation in a new module Boris Feld <boris.feld@octobus.net> parents: 39356 diff changeset	492 >>> segmentspan(revlog, [0, 1, 2, 3, 4])
655b5b465953 revlog: split functionality related to deltas computation in a new module Boris Feld <boris.feld@octobus.net> parents: 39356 diff changeset	493 17
655b5b465953 revlog: split functionality related to deltas computation in a new module Boris Feld <boris.feld@octobus.net> parents: 39356 diff changeset	494 >>> segmentspan(revlog, [0, 4])
655b5b465953 revlog: split functionality related to deltas computation in a new module Boris Feld <boris.feld@octobus.net> parents: 39356 diff changeset	495 17
655b5b465953 revlog: split functionality related to deltas computation in a new module Boris Feld <boris.feld@octobus.net> parents: 39356 diff changeset	496 >>> segmentspan(revlog, [3, 4])
655b5b465953 revlog: split functionality related to deltas computation in a new module Boris Feld <boris.feld@octobus.net> parents: 39356 diff changeset	497 5
655b5b465953 revlog: split functionality related to deltas computation in a new module Boris Feld <boris.feld@octobus.net> parents: 39356 diff changeset	498 >>> segmentspan(revlog, [1, 2, 3,])
655b5b465953 revlog: split functionality related to deltas computation in a new module Boris Feld <boris.feld@octobus.net> parents: 39356 diff changeset	499 7
655b5b465953 revlog: split functionality related to deltas computation in a new module Boris Feld <boris.feld@octobus.net> parents: 39356 diff changeset	500 >>> segmentspan(revlog, [1, 3])
655b5b465953 revlog: split functionality related to deltas computation in a new module Boris Feld <boris.feld@octobus.net> parents: 39356 diff changeset	501 7
655b5b465953 revlog: split functionality related to deltas computation in a new module Boris Feld <boris.feld@octobus.net> parents: 39356 diff changeset	502 """
655b5b465953 revlog: split functionality related to deltas computation in a new module Boris Feld <boris.feld@octobus.net> parents: 39356 diff changeset	503 if not revs:
655b5b465953 revlog: split functionality related to deltas computation in a new module Boris Feld <boris.feld@octobus.net> parents: 39356 diff changeset	504 return 0
40652 a32ccd32982b sparse-revlog: drop unused deltainfo parameter from segmentspan Boris Feld <boris.feld@octobus.net> parents: 40651 diff changeset	505 end = revlog.end(revs[-1])
39357 655b5b465953 revlog: split functionality related to deltas computation in a new module Boris Feld <boris.feld@octobus.net> parents: 39356 diff changeset	506 return end - revlog.start(revs[0])
655b5b465953 revlog: split functionality related to deltas computation in a new module Boris Feld <boris.feld@octobus.net> parents: 39356 diff changeset	507
39358 fd0150a3c2fe revlogdeltas: extra fulltext building in its own function Boris Feld <boris.feld@octobus.net> parents: 39357 diff changeset	508 def _textfromdelta(fh, revlog, baserev, delta, p1, p2, flags, expectednode):
fd0150a3c2fe revlogdeltas: extra fulltext building in its own function Boris Feld <boris.feld@octobus.net> parents: 39357 diff changeset	509 """build full text from a (base, delta) pair and other metadata"""
fd0150a3c2fe revlogdeltas: extra fulltext building in its own function Boris Feld <boris.feld@octobus.net> parents: 39357 diff changeset	510 # special case deltas which replace entire base; no need to decode
fd0150a3c2fe revlogdeltas: extra fulltext building in its own function Boris Feld <boris.feld@octobus.net> parents: 39357 diff changeset	511 # base revision. this neatly avoids censored bases, which throw when
fd0150a3c2fe revlogdeltas: extra fulltext building in its own function Boris Feld <boris.feld@octobus.net> parents: 39357 diff changeset	512 # they're decoded.
fd0150a3c2fe revlogdeltas: extra fulltext building in its own function Boris Feld <boris.feld@octobus.net> parents: 39357 diff changeset	513 hlen = struct.calcsize(">lll")
fd0150a3c2fe revlogdeltas: extra fulltext building in its own function Boris Feld <boris.feld@octobus.net> parents: 39357 diff changeset	514 if delta[:hlen] == mdiff.replacediffheader(revlog.rawsize(baserev),
fd0150a3c2fe revlogdeltas: extra fulltext building in its own function Boris Feld <boris.feld@octobus.net> parents: 39357 diff changeset	515 len(delta) - hlen):
fd0150a3c2fe revlogdeltas: extra fulltext building in its own function Boris Feld <boris.feld@octobus.net> parents: 39357 diff changeset	516 fulltext = delta[hlen:]
fd0150a3c2fe revlogdeltas: extra fulltext building in its own function Boris Feld <boris.feld@octobus.net> parents: 39357 diff changeset	517 else:
fd0150a3c2fe revlogdeltas: extra fulltext building in its own function Boris Feld <boris.feld@octobus.net> parents: 39357 diff changeset	518 # deltabase is rawtext before changed by flag processors, which is
fd0150a3c2fe revlogdeltas: extra fulltext building in its own function Boris Feld <boris.feld@octobus.net> parents: 39357 diff changeset	519 # equivalent to non-raw text
fd0150a3c2fe revlogdeltas: extra fulltext building in its own function Boris Feld <boris.feld@octobus.net> parents: 39357 diff changeset	520 basetext = revlog.revision(baserev, _df=fh, raw=False)
fd0150a3c2fe revlogdeltas: extra fulltext building in its own function Boris Feld <boris.feld@octobus.net> parents: 39357 diff changeset	521 fulltext = mdiff.patch(basetext, delta)
fd0150a3c2fe revlogdeltas: extra fulltext building in its own function Boris Feld <boris.feld@octobus.net> parents: 39357 diff changeset	522
fd0150a3c2fe revlogdeltas: extra fulltext building in its own function Boris Feld <boris.feld@octobus.net> parents: 39357 diff changeset	523 try:
fd0150a3c2fe revlogdeltas: extra fulltext building in its own function Boris Feld <boris.feld@octobus.net> parents: 39357 diff changeset	524 res = revlog._processflags(fulltext, flags, 'read', raw=True)
fd0150a3c2fe revlogdeltas: extra fulltext building in its own function Boris Feld <boris.feld@octobus.net> parents: 39357 diff changeset	525 fulltext, validatehash = res
fd0150a3c2fe revlogdeltas: extra fulltext building in its own function Boris Feld <boris.feld@octobus.net> parents: 39357 diff changeset	526 if validatehash:
fd0150a3c2fe revlogdeltas: extra fulltext building in its own function Boris Feld <boris.feld@octobus.net> parents: 39357 diff changeset	527 revlog.checkhash(fulltext, expectednode, p1=p1, p2=p2)
fd0150a3c2fe revlogdeltas: extra fulltext building in its own function Boris Feld <boris.feld@octobus.net> parents: 39357 diff changeset	528 if flags & REVIDX_ISCENSORED:
39793 b63dee7bd0d9 global: replace most uses of RevlogError with StorageError (API) Gregory Szorc <gregory.szorc@gmail.com> parents: 39790 diff changeset	529 raise error.StorageError(_('node %s is not censored') %
b63dee7bd0d9 global: replace most uses of RevlogError with StorageError (API) Gregory Szorc <gregory.szorc@gmail.com> parents: 39790 diff changeset	530 expectednode)
39790 4a2466b2a434 revlog: drop some more error aliases (API) Gregory Szorc <gregory.szorc@gmail.com> parents: 39789 diff changeset	531 except error.CensoredNodeError:
39358 fd0150a3c2fe revlogdeltas: extra fulltext building in its own function Boris Feld <boris.feld@octobus.net> parents: 39357 diff changeset	532 # must pass the censored index flag to add censored revisions
fd0150a3c2fe revlogdeltas: extra fulltext building in its own function Boris Feld <boris.feld@octobus.net> parents: 39357 diff changeset	533 if not flags & REVIDX_ISCENSORED:
fd0150a3c2fe revlogdeltas: extra fulltext building in its own function Boris Feld <boris.feld@octobus.net> parents: 39357 diff changeset	534 raise
fd0150a3c2fe revlogdeltas: extra fulltext building in its own function Boris Feld <boris.feld@octobus.net> parents: 39357 diff changeset	535 return fulltext
fd0150a3c2fe revlogdeltas: extra fulltext building in its own function Boris Feld <boris.feld@octobus.net> parents: 39357 diff changeset	536
35638 edc9330acac1 revlog: introduce 'deltainfo' to distinguish from 'delta' Paul Morelle <paul.morelle@octobus.net> parents: 35637 diff changeset	537 @attr.s(slots=True, frozen=True)
edc9330acac1 revlog: introduce 'deltainfo' to distinguish from 'delta' Paul Morelle <paul.morelle@octobus.net> parents: 35637 diff changeset	538 class _deltainfo(object):
edc9330acac1 revlog: introduce 'deltainfo' to distinguish from 'delta' Paul Morelle <paul.morelle@octobus.net> parents: 35637 diff changeset	539 distance = attr.ib()
edc9330acac1 revlog: introduce 'deltainfo' to distinguish from 'delta' Paul Morelle <paul.morelle@octobus.net> parents: 35637 diff changeset	540 deltalen = attr.ib()
edc9330acac1 revlog: introduce 'deltainfo' to distinguish from 'delta' Paul Morelle <paul.morelle@octobus.net> parents: 35637 diff changeset	541 data = attr.ib()
edc9330acac1 revlog: introduce 'deltainfo' to distinguish from 'delta' Paul Morelle <paul.morelle@octobus.net> parents: 35637 diff changeset	542 base = attr.ib()
edc9330acac1 revlog: introduce 'deltainfo' to distinguish from 'delta' Paul Morelle <paul.morelle@octobus.net> parents: 35637 diff changeset	543 chainbase = attr.ib()
edc9330acac1 revlog: introduce 'deltainfo' to distinguish from 'delta' Paul Morelle <paul.morelle@octobus.net> parents: 35637 diff changeset	544 chainlen = attr.ib()
edc9330acac1 revlog: introduce 'deltainfo' to distinguish from 'delta' Paul Morelle <paul.morelle@octobus.net> parents: 35637 diff changeset	545 compresseddeltalen = attr.ib()
39187 e0da43e2f71f revlog: compute snapshot depth on delta info Boris Feld <boris.feld@octobus.net> parents: 39185 diff changeset	546 snapshotdepth = attr.ib()
35638 edc9330acac1 revlog: introduce 'deltainfo' to distinguish from 'delta' Paul Morelle <paul.morelle@octobus.net> parents: 35637 diff changeset	547
39357 655b5b465953 revlog: split functionality related to deltas computation in a new module Boris Feld <boris.feld@octobus.net> parents: 39356 diff changeset	548 def isgooddeltainfo(revlog, deltainfo, revinfo):
655b5b465953 revlog: split functionality related to deltas computation in a new module Boris Feld <boris.feld@octobus.net> parents: 39356 diff changeset	549 """Returns True if the given delta is good. Good means that it is within
655b5b465953 revlog: split functionality related to deltas computation in a new module Boris Feld <boris.feld@octobus.net> parents: 39356 diff changeset	550 the disk span, disk size, and chain length bounds that we know to be
655b5b465953 revlog: split functionality related to deltas computation in a new module Boris Feld <boris.feld@octobus.net> parents: 39356 diff changeset	551 performant."""
655b5b465953 revlog: split functionality related to deltas computation in a new module Boris Feld <boris.feld@octobus.net> parents: 39356 diff changeset	552 if deltainfo is None:
655b5b465953 revlog: split functionality related to deltas computation in a new module Boris Feld <boris.feld@octobus.net> parents: 39356 diff changeset	553 return False
655b5b465953 revlog: split functionality related to deltas computation in a new module Boris Feld <boris.feld@octobus.net> parents: 39356 diff changeset	554
655b5b465953 revlog: split functionality related to deltas computation in a new module Boris Feld <boris.feld@octobus.net> parents: 39356 diff changeset	555 # - 'deltainfo.distance' is the distance from the base revision --
655b5b465953 revlog: split functionality related to deltas computation in a new module Boris Feld <boris.feld@octobus.net> parents: 39356 diff changeset	556 # bounding it limits the amount of I/O we need to do.
655b5b465953 revlog: split functionality related to deltas computation in a new module Boris Feld <boris.feld@octobus.net> parents: 39356 diff changeset	557 # - 'deltainfo.compresseddeltalen' is the sum of the total size of
655b5b465953 revlog: split functionality related to deltas computation in a new module Boris Feld <boris.feld@octobus.net> parents: 39356 diff changeset	558 # deltas we need to apply -- bounding it limits the amount of CPU
655b5b465953 revlog: split functionality related to deltas computation in a new module Boris Feld <boris.feld@octobus.net> parents: 39356 diff changeset	559 # we consume.
655b5b465953 revlog: split functionality related to deltas computation in a new module Boris Feld <boris.feld@octobus.net> parents: 39356 diff changeset	560
655b5b465953 revlog: split functionality related to deltas computation in a new module Boris Feld <boris.feld@octobus.net> parents: 39356 diff changeset	561 textlen = revinfo.textlen
655b5b465953 revlog: split functionality related to deltas computation in a new module Boris Feld <boris.feld@octobus.net> parents: 39356 diff changeset	562 defaultmax = textlen * 4
655b5b465953 revlog: split functionality related to deltas computation in a new module Boris Feld <boris.feld@octobus.net> parents: 39356 diff changeset	563 maxdist = revlog._maxdeltachainspan
655b5b465953 revlog: split functionality related to deltas computation in a new module Boris Feld <boris.feld@octobus.net> parents: 39356 diff changeset	564 if not maxdist:
40650 2f7e531ef3e7 sparse-revlog: skip the span check in the sparse-revlog case Boris Feld <boris.feld@octobus.net> parents: 40425 diff changeset	565 maxdist = deltainfo.distance # ensure the conditional pass
39357 655b5b465953 revlog: split functionality related to deltas computation in a new module Boris Feld <boris.feld@octobus.net> parents: 39356 diff changeset	566 maxdist = max(maxdist, defaultmax)
655b5b465953 revlog: split functionality related to deltas computation in a new module Boris Feld <boris.feld@octobus.net> parents: 39356 diff changeset	567
655b5b465953 revlog: split functionality related to deltas computation in a new module Boris Feld <boris.feld@octobus.net> parents: 39356 diff changeset	568 # Bad delta from read span:
655b5b465953 revlog: split functionality related to deltas computation in a new module Boris Feld <boris.feld@octobus.net> parents: 39356 diff changeset	569 #
655b5b465953 revlog: split functionality related to deltas computation in a new module Boris Feld <boris.feld@octobus.net> parents: 39356 diff changeset	570 # If the span of data read is larger than the maximum allowed.
40650 2f7e531ef3e7 sparse-revlog: skip the span check in the sparse-revlog case Boris Feld <boris.feld@octobus.net> parents: 40425 diff changeset	571 #
2f7e531ef3e7 sparse-revlog: skip the span check in the sparse-revlog case Boris Feld <boris.feld@octobus.net> parents: 40425 diff changeset	572 # In the sparse-revlog case, we rely on the associated "sparse reading"
2f7e531ef3e7 sparse-revlog: skip the span check in the sparse-revlog case Boris Feld <boris.feld@octobus.net> parents: 40425 diff changeset	573 # to avoid issue related to the span of data. In theory, it would be
2f7e531ef3e7 sparse-revlog: skip the span check in the sparse-revlog case Boris Feld <boris.feld@octobus.net> parents: 40425 diff changeset	574 # possible to build pathological revlog where delta pattern would lead
2f7e531ef3e7 sparse-revlog: skip the span check in the sparse-revlog case Boris Feld <boris.feld@octobus.net> parents: 40425 diff changeset	575 # to too many reads. However, they do not happen in practice at all. So
2f7e531ef3e7 sparse-revlog: skip the span check in the sparse-revlog case Boris Feld <boris.feld@octobus.net> parents: 40425 diff changeset	576 # we skip the span check entirely.
2f7e531ef3e7 sparse-revlog: skip the span check in the sparse-revlog case Boris Feld <boris.feld@octobus.net> parents: 40425 diff changeset	577 if not revlog._sparserevlog and maxdist < deltainfo.distance:
39357 655b5b465953 revlog: split functionality related to deltas computation in a new module Boris Feld <boris.feld@octobus.net> parents: 39356 diff changeset	578 return False
655b5b465953 revlog: split functionality related to deltas computation in a new module Boris Feld <boris.feld@octobus.net> parents: 39356 diff changeset	579
655b5b465953 revlog: split functionality related to deltas computation in a new module Boris Feld <boris.feld@octobus.net> parents: 39356 diff changeset	580 # Bad delta from new delta size:
655b5b465953 revlog: split functionality related to deltas computation in a new module Boris Feld <boris.feld@octobus.net> parents: 39356 diff changeset	581 #
655b5b465953 revlog: split functionality related to deltas computation in a new module Boris Feld <boris.feld@octobus.net> parents: 39356 diff changeset	582 # If the delta size is larger than the target text, storing the
655b5b465953 revlog: split functionality related to deltas computation in a new module Boris Feld <boris.feld@octobus.net> parents: 39356 diff changeset	583 # delta will be inefficient.
655b5b465953 revlog: split functionality related to deltas computation in a new module Boris Feld <boris.feld@octobus.net> parents: 39356 diff changeset	584 if textlen < deltainfo.deltalen:
655b5b465953 revlog: split functionality related to deltas computation in a new module Boris Feld <boris.feld@octobus.net> parents: 39356 diff changeset	585 return False
655b5b465953 revlog: split functionality related to deltas computation in a new module Boris Feld <boris.feld@octobus.net> parents: 39356 diff changeset	586
655b5b465953 revlog: split functionality related to deltas computation in a new module Boris Feld <boris.feld@octobus.net> parents: 39356 diff changeset	587 # Bad delta from cumulated payload size:
655b5b465953 revlog: split functionality related to deltas computation in a new module Boris Feld <boris.feld@octobus.net> parents: 39356 diff changeset	588 #
655b5b465953 revlog: split functionality related to deltas computation in a new module Boris Feld <boris.feld@octobus.net> parents: 39356 diff changeset	589 # If the sum of delta get larger than K * target text length.
655b5b465953 revlog: split functionality related to deltas computation in a new module Boris Feld <boris.feld@octobus.net> parents: 39356 diff changeset	590 if textlen * LIMIT_DELTA2TEXT < deltainfo.compresseddeltalen:
655b5b465953 revlog: split functionality related to deltas computation in a new module Boris Feld <boris.feld@octobus.net> parents: 39356 diff changeset	591 return False
655b5b465953 revlog: split functionality related to deltas computation in a new module Boris Feld <boris.feld@octobus.net> parents: 39356 diff changeset	592
655b5b465953 revlog: split functionality related to deltas computation in a new module Boris Feld <boris.feld@octobus.net> parents: 39356 diff changeset	593 # Bad delta from chain length:
655b5b465953 revlog: split functionality related to deltas computation in a new module Boris Feld <boris.feld@octobus.net> parents: 39356 diff changeset	594 #
655b5b465953 revlog: split functionality related to deltas computation in a new module Boris Feld <boris.feld@octobus.net> parents: 39356 diff changeset	595 # If the number of delta in the chain gets too high.
655b5b465953 revlog: split functionality related to deltas computation in a new module Boris Feld <boris.feld@octobus.net> parents: 39356 diff changeset	596 if (revlog._maxchainlen
655b5b465953 revlog: split functionality related to deltas computation in a new module Boris Feld <boris.feld@octobus.net> parents: 39356 diff changeset	597 and revlog._maxchainlen < deltainfo.chainlen):
655b5b465953 revlog: split functionality related to deltas computation in a new module Boris Feld <boris.feld@octobus.net> parents: 39356 diff changeset	598 return False
655b5b465953 revlog: split functionality related to deltas computation in a new module Boris Feld <boris.feld@octobus.net> parents: 39356 diff changeset	599
655b5b465953 revlog: split functionality related to deltas computation in a new module Boris Feld <boris.feld@octobus.net> parents: 39356 diff changeset	600 # bad delta from intermediate snapshot size limit
655b5b465953 revlog: split functionality related to deltas computation in a new module Boris Feld <boris.feld@octobus.net> parents: 39356 diff changeset	601 #
655b5b465953 revlog: split functionality related to deltas computation in a new module Boris Feld <boris.feld@octobus.net> parents: 39356 diff changeset	602 # If an intermediate snapshot size is higher than the limit. The
655b5b465953 revlog: split functionality related to deltas computation in a new module Boris Feld <boris.feld@octobus.net> parents: 39356 diff changeset	603 # limit exist to prevent endless chain of intermediate delta to be
655b5b465953 revlog: split functionality related to deltas computation in a new module Boris Feld <boris.feld@octobus.net> parents: 39356 diff changeset	604 # created.
655b5b465953 revlog: split functionality related to deltas computation in a new module Boris Feld <boris.feld@octobus.net> parents: 39356 diff changeset	605 if (deltainfo.snapshotdepth is not None and
655b5b465953 revlog: split functionality related to deltas computation in a new module Boris Feld <boris.feld@octobus.net> parents: 39356 diff changeset	606 (textlen >> deltainfo.snapshotdepth) < deltainfo.deltalen):
655b5b465953 revlog: split functionality related to deltas computation in a new module Boris Feld <boris.feld@octobus.net> parents: 39356 diff changeset	607 return False
655b5b465953 revlog: split functionality related to deltas computation in a new module Boris Feld <boris.feld@octobus.net> parents: 39356 diff changeset	608
655b5b465953 revlog: split functionality related to deltas computation in a new module Boris Feld <boris.feld@octobus.net> parents: 39356 diff changeset	609 # bad delta if new intermediate snapshot is larger than the previous
655b5b465953 revlog: split functionality related to deltas computation in a new module Boris Feld <boris.feld@octobus.net> parents: 39356 diff changeset	610 # snapshot
655b5b465953 revlog: split functionality related to deltas computation in a new module Boris Feld <boris.feld@octobus.net> parents: 39356 diff changeset	611 if (deltainfo.snapshotdepth
655b5b465953 revlog: split functionality related to deltas computation in a new module Boris Feld <boris.feld@octobus.net> parents: 39356 diff changeset	612 and revlog.length(deltainfo.base) < deltainfo.deltalen):
655b5b465953 revlog: split functionality related to deltas computation in a new module Boris Feld <boris.feld@octobus.net> parents: 39356 diff changeset	613 return False
655b5b465953 revlog: split functionality related to deltas computation in a new module Boris Feld <boris.feld@octobus.net> parents: 39356 diff changeset	614
655b5b465953 revlog: split functionality related to deltas computation in a new module Boris Feld <boris.feld@octobus.net> parents: 39356 diff changeset	615 return True
655b5b465953 revlog: split functionality related to deltas computation in a new module Boris Feld <boris.feld@octobus.net> parents: 39356 diff changeset	616
40991 42f59d3f714d delta: exclude base candidate much smaller than the target Boris Feld <boris.feld@octobus.net> parents: 40970 diff changeset	617 # If a revision's full text is that much bigger than a base candidate full
42f59d3f714d delta: exclude base candidate much smaller than the target Boris Feld <boris.feld@octobus.net> parents: 40970 diff changeset	618 # text's, it is very unlikely that it will produce a valid delta. We no longer
42f59d3f714d delta: exclude base candidate much smaller than the target Boris Feld <boris.feld@octobus.net> parents: 40970 diff changeset	619 # consider these candidates.
41036 b373477948df revlog: limit base to rev size ratio to 500 instead of 50 Boris Feld <boris.feld@octobus.net> parents: 40992 diff changeset	620 LIMIT_BASE2TEXT = 500
40991 42f59d3f714d delta: exclude base candidate much smaller than the target Boris Feld <boris.feld@octobus.net> parents: 40970 diff changeset	621
39364 37957e07138c revlogdeltas: move finddeltainfo filtering inside _candidategroups Boris Feld <boris.feld@octobus.net> parents: 39363 diff changeset	622 def _candidategroups(revlog, textlen, p1, p2, cachedelta):
39363 1c6ff52fe9cf revlogdeltas: split candidate groups selection from the filtering logic Boris Feld <boris.feld@octobus.net> parents: 39362 diff changeset	623 """Provides group of revision to be tested as delta base
1c6ff52fe9cf revlogdeltas: split candidate groups selection from the filtering logic Boris Feld <boris.feld@octobus.net> parents: 39362 diff changeset	624
1c6ff52fe9cf revlogdeltas: split candidate groups selection from the filtering logic Boris Feld <boris.feld@octobus.net> parents: 39362 diff changeset	625 This top level function focus on emitting groups with unique and worthwhile
1c6ff52fe9cf revlogdeltas: split candidate groups selection from the filtering logic Boris Feld <boris.feld@octobus.net> parents: 39362 diff changeset	626 content. See _raw_candidate_groups for details about the group order.
39361 507f5b1dd7c8 revlogdeltas: extract _getcandidaterevs in a function Boris Feld <boris.feld@octobus.net> parents: 39360 diff changeset	627 """
39363 1c6ff52fe9cf revlogdeltas: split candidate groups selection from the filtering logic Boris Feld <boris.feld@octobus.net> parents: 39362 diff changeset	628 # should we try to build a delta?
1c6ff52fe9cf revlogdeltas: split candidate groups selection from the filtering logic Boris Feld <boris.feld@octobus.net> parents: 39362 diff changeset	629 if not (len(revlog) and revlog._storedeltachains):
39514 5b308a4e6d03 snapshot: use None as a stop value when looking for a good delta Boris Feld <boris.feld@octobus.net> parents: 39513 diff changeset	630 yield None
39363 1c6ff52fe9cf revlogdeltas: split candidate groups selection from the filtering logic Boris Feld <boris.feld@octobus.net> parents: 39362 diff changeset	631 return
1c6ff52fe9cf revlogdeltas: split candidate groups selection from the filtering logic Boris Feld <boris.feld@octobus.net> parents: 39362 diff changeset	632
39364 37957e07138c revlogdeltas: move finddeltainfo filtering inside _candidategroups Boris Feld <boris.feld@octobus.net> parents: 39363 diff changeset	633 deltalength = revlog.length
37957e07138c revlogdeltas: move finddeltainfo filtering inside _candidategroups Boris Feld <boris.feld@octobus.net> parents: 39363 diff changeset	634 deltaparent = revlog.deltaparent
40991 42f59d3f714d delta: exclude base candidate much smaller than the target Boris Feld <boris.feld@octobus.net> parents: 40970 diff changeset	635 sparse = revlog._sparserevlog
39515 04b75f3a3f2a snapshot: add refining logic at the findeltainfo level Boris Feld <boris.feld@octobus.net> parents: 39514 diff changeset	636 good = None
39364 37957e07138c revlogdeltas: move finddeltainfo filtering inside _candidategroups Boris Feld <boris.feld@octobus.net> parents: 39363 diff changeset	637
37957e07138c revlogdeltas: move finddeltainfo filtering inside _candidategroups Boris Feld <boris.feld@octobus.net> parents: 39363 diff changeset	638 deltas_limit = textlen * LIMIT_DELTA2TEXT
37957e07138c revlogdeltas: move finddeltainfo filtering inside _candidategroups Boris Feld <boris.feld@octobus.net> parents: 39363 diff changeset	639
42057 566daffc607d cleanup: use set literals where possible Martin von Zweigbergk <martinvonz@google.com> parents: 41824 diff changeset	640 tested = {nullrev}
39516 51cec7fb672e snapshot: also use None as a stop value for `_refinegroup` Boris Feld <boris.feld@octobus.net> parents: 39515 diff changeset	641 candidates = _refinedgroups(revlog, p1, p2, cachedelta)
51cec7fb672e snapshot: also use None as a stop value for `_refinegroup` Boris Feld <boris.feld@octobus.net> parents: 39515 diff changeset	642 while True:
39517 cc85ebb68ff9 snapshot: turn _refinedgroups into a coroutine Boris Feld <boris.feld@octobus.net> parents: 39516 diff changeset	643 temptative = candidates.send(good)
39516 51cec7fb672e snapshot: also use None as a stop value for `_refinegroup` Boris Feld <boris.feld@octobus.net> parents: 39515 diff changeset	644 if temptative is None:
51cec7fb672e snapshot: also use None as a stop value for `_refinegroup` Boris Feld <boris.feld@octobus.net> parents: 39515 diff changeset	645 break
39364 37957e07138c revlogdeltas: move finddeltainfo filtering inside _candidategroups Boris Feld <boris.feld@octobus.net> parents: 39363 diff changeset	646 group = []
37957e07138c revlogdeltas: move finddeltainfo filtering inside _candidategroups Boris Feld <boris.feld@octobus.net> parents: 39363 diff changeset	647 for rev in temptative:
37957e07138c revlogdeltas: move finddeltainfo filtering inside _candidategroups Boris Feld <boris.feld@octobus.net> parents: 39363 diff changeset	648 # skip over empty delta (no need to include them in a chain)
40425 324ba8b14d78 delta: skip "empty delta" optimisation for non-general case (issue6006) Boris Feld <boris.feld@octobus.net> parents: 40415 diff changeset	649 while (revlog._generaldelta
324ba8b14d78 delta: skip "empty delta" optimisation for non-general case (issue6006) Boris Feld <boris.feld@octobus.net> parents: 40415 diff changeset	650 and not (rev == nullrev
324ba8b14d78 delta: skip "empty delta" optimisation for non-general case (issue6006) Boris Feld <boris.feld@octobus.net> parents: 40415 diff changeset	651 or rev in tested
324ba8b14d78 delta: skip "empty delta" optimisation for non-general case (issue6006) Boris Feld <boris.feld@octobus.net> parents: 40415 diff changeset	652 or deltalength(rev))):
39610 bdb41eaa8b59 snapshot: fix line order when skipping over empty deltas Boris Feld <boris.feld@octobus.net> parents: 39522 diff changeset	653 tested.add(rev)
39364 37957e07138c revlogdeltas: move finddeltainfo filtering inside _candidategroups Boris Feld <boris.feld@octobus.net> parents: 39363 diff changeset	654 rev = deltaparent(rev)
40970 f960c51eebf3 delta: filter nullrev out first Boris Feld <boris.feld@octobus.net> parents: 40748 diff changeset	655 # no need to try a delta against nullrev, this will be done as a
f960c51eebf3 delta: filter nullrev out first Boris Feld <boris.feld@octobus.net> parents: 40748 diff changeset	656 # last resort.
f960c51eebf3 delta: filter nullrev out first Boris Feld <boris.feld@octobus.net> parents: 40748 diff changeset	657 if rev == nullrev:
f960c51eebf3 delta: filter nullrev out first Boris Feld <boris.feld@octobus.net> parents: 40748 diff changeset	658 continue
39364 37957e07138c revlogdeltas: move finddeltainfo filtering inside _candidategroups Boris Feld <boris.feld@octobus.net> parents: 39363 diff changeset	659 # filter out revision we tested already
37957e07138c revlogdeltas: move finddeltainfo filtering inside _candidategroups Boris Feld <boris.feld@octobus.net> parents: 39363 diff changeset	660 if rev in tested:
37957e07138c revlogdeltas: move finddeltainfo filtering inside _candidategroups Boris Feld <boris.feld@octobus.net> parents: 39363 diff changeset	661 continue
37957e07138c revlogdeltas: move finddeltainfo filtering inside _candidategroups Boris Feld <boris.feld@octobus.net> parents: 39363 diff changeset	662 tested.add(rev)
37957e07138c revlogdeltas: move finddeltainfo filtering inside _candidategroups Boris Feld <boris.feld@octobus.net> parents: 39363 diff changeset	663 # filter out delta base that will never produce good delta
37957e07138c revlogdeltas: move finddeltainfo filtering inside _candidategroups Boris Feld <boris.feld@octobus.net> parents: 39363 diff changeset	664 if deltas_limit < revlog.length(rev):
37957e07138c revlogdeltas: move finddeltainfo filtering inside _candidategroups Boris Feld <boris.feld@octobus.net> parents: 39363 diff changeset	665 continue
40991 42f59d3f714d delta: exclude base candidate much smaller than the target Boris Feld <boris.feld@octobus.net> parents: 40970 diff changeset	666 if sparse and revlog.rawsize(rev) < (textlen // LIMIT_BASE2TEXT):
42f59d3f714d delta: exclude base candidate much smaller than the target Boris Feld <boris.feld@octobus.net> parents: 40970 diff changeset	667 continue
39364 37957e07138c revlogdeltas: move finddeltainfo filtering inside _candidategroups Boris Feld <boris.feld@octobus.net> parents: 39363 diff changeset	668 # no delta for rawtext-changing revs (see "candelta" for why)
37957e07138c revlogdeltas: move finddeltainfo filtering inside _candidategroups Boris Feld <boris.feld@octobus.net> parents: 39363 diff changeset	669 if revlog.flags(rev) & REVIDX_RAWTEXT_CHANGING_FLAGS:
37957e07138c revlogdeltas: move finddeltainfo filtering inside _candidategroups Boris Feld <boris.feld@octobus.net> parents: 39363 diff changeset	670 continue
40992 ba09db267cb6 delta: ignore base whose chains already don't match expectations Boris Feld <boris.feld@octobus.net> parents: 40991 diff changeset	671 # If we reach here, we are about to build and test a delta.
ba09db267cb6 delta: ignore base whose chains already don't match expectations Boris Feld <boris.feld@octobus.net> parents: 40991 diff changeset	672 # The delta building process will compute the chaininfo in all
ba09db267cb6 delta: ignore base whose chains already don't match expectations Boris Feld <boris.feld@octobus.net> parents: 40991 diff changeset	673 # case, since that computation is cached, it is fine to access it
ba09db267cb6 delta: ignore base whose chains already don't match expectations Boris Feld <boris.feld@octobus.net> parents: 40991 diff changeset	674 # here too.
ba09db267cb6 delta: ignore base whose chains already don't match expectations Boris Feld <boris.feld@octobus.net> parents: 40991 diff changeset	675 chainlen, chainsize = revlog._chaininfo(rev)
ba09db267cb6 delta: ignore base whose chains already don't match expectations Boris Feld <boris.feld@octobus.net> parents: 40991 diff changeset	676 # if chain will be too long, skip base
ba09db267cb6 delta: ignore base whose chains already don't match expectations Boris Feld <boris.feld@octobus.net> parents: 40991 diff changeset	677 if revlog._maxchainlen and chainlen >= revlog._maxchainlen:
ba09db267cb6 delta: ignore base whose chains already don't match expectations Boris Feld <boris.feld@octobus.net> parents: 40991 diff changeset	678 continue
ba09db267cb6 delta: ignore base whose chains already don't match expectations Boris Feld <boris.feld@octobus.net> parents: 40991 diff changeset	679 # if chain already have too much data, skip base
ba09db267cb6 delta: ignore base whose chains already don't match expectations Boris Feld <boris.feld@octobus.net> parents: 40991 diff changeset	680 if deltas_limit < chainsize:
ba09db267cb6 delta: ignore base whose chains already don't match expectations Boris Feld <boris.feld@octobus.net> parents: 40991 diff changeset	681 continue
42479 a0b26fc8fbba deltas: skip if projected delta size does not match text size constraint Valentin Gatien-Baron <vgatien-baron@janestreet.com>, Pierre-Yves David <pierre-yves.david@octobus.net> parents: 42057 diff changeset	682 if sparse and revlog.upperboundcomp is not None:
a0b26fc8fbba deltas: skip if projected delta size does not match text size constraint Valentin Gatien-Baron <vgatien-baron@janestreet.com>, Pierre-Yves David <pierre-yves.david@octobus.net> parents: 42057 diff changeset	683 maxcomp = revlog.upperboundcomp
a0b26fc8fbba deltas: skip if projected delta size does not match text size constraint Valentin Gatien-Baron <vgatien-baron@janestreet.com>, Pierre-Yves David <pierre-yves.david@octobus.net> parents: 42057 diff changeset	684 basenotsnap = (p1, p2, nullrev)
a0b26fc8fbba deltas: skip if projected delta size does not match text size constraint Valentin Gatien-Baron <vgatien-baron@janestreet.com>, Pierre-Yves David <pierre-yves.david@octobus.net> parents: 42057 diff changeset	685 if rev not in basenotsnap and revlog.issnapshot(rev):
a0b26fc8fbba deltas: skip if projected delta size does not match text size constraint Valentin Gatien-Baron <vgatien-baron@janestreet.com>, Pierre-Yves David <pierre-yves.david@octobus.net> parents: 42057 diff changeset	686 snapshotdepth = revlog.snapshotdepth(rev)
a0b26fc8fbba deltas: skip if projected delta size does not match text size constraint Valentin Gatien-Baron <vgatien-baron@janestreet.com>, Pierre-Yves David <pierre-yves.david@octobus.net> parents: 42057 diff changeset	687 # If text is significantly larger than the base, we can
a0b26fc8fbba deltas: skip if projected delta size does not match text size constraint Valentin Gatien-Baron <vgatien-baron@janestreet.com>, Pierre-Yves David <pierre-yves.david@octobus.net> parents: 42057 diff changeset	688 # expect the resulting delta to be proportional to the size
a0b26fc8fbba deltas: skip if projected delta size does not match text size constraint Valentin Gatien-Baron <vgatien-baron@janestreet.com>, Pierre-Yves David <pierre-yves.david@octobus.net> parents: 42057 diff changeset	689 # difference
a0b26fc8fbba deltas: skip if projected delta size does not match text size constraint Valentin Gatien-Baron <vgatien-baron@janestreet.com>, Pierre-Yves David <pierre-yves.david@octobus.net> parents: 42057 diff changeset	690 revsize = revlog.rawsize(rev)
a0b26fc8fbba deltas: skip if projected delta size does not match text size constraint Valentin Gatien-Baron <vgatien-baron@janestreet.com>, Pierre-Yves David <pierre-yves.david@octobus.net> parents: 42057 diff changeset	691 rawsizedistance = max(textlen - revsize, 0)
a0b26fc8fbba deltas: skip if projected delta size does not match text size constraint Valentin Gatien-Baron <vgatien-baron@janestreet.com>, Pierre-Yves David <pierre-yves.david@octobus.net> parents: 42057 diff changeset	692 # use an estimate of the compression upper bound.
a0b26fc8fbba deltas: skip if projected delta size does not match text size constraint Valentin Gatien-Baron <vgatien-baron@janestreet.com>, Pierre-Yves David <pierre-yves.david@octobus.net> parents: 42057 diff changeset	693 lowestrealisticdeltalen = rawsizedistance // maxcomp
a0b26fc8fbba deltas: skip if projected delta size does not match text size constraint Valentin Gatien-Baron <vgatien-baron@janestreet.com>, Pierre-Yves David <pierre-yves.david@octobus.net> parents: 42057 diff changeset	694
a0b26fc8fbba deltas: skip if projected delta size does not match text size constraint Valentin Gatien-Baron <vgatien-baron@janestreet.com>, Pierre-Yves David <pierre-yves.david@octobus.net> parents: 42057 diff changeset	695 # check the absolute constraint on the delta size
a0b26fc8fbba deltas: skip if projected delta size does not match text size constraint Valentin Gatien-Baron <vgatien-baron@janestreet.com>, Pierre-Yves David <pierre-yves.david@octobus.net> parents: 42057 diff changeset	696 snapshotlimit = textlen >> snapshotdepth
a0b26fc8fbba deltas: skip if projected delta size does not match text size constraint Valentin Gatien-Baron <vgatien-baron@janestreet.com>, Pierre-Yves David <pierre-yves.david@octobus.net> parents: 42057 diff changeset	697 if snapshotlimit < lowestrealisticdeltalen:
a0b26fc8fbba deltas: skip if projected delta size does not match text size constraint Valentin Gatien-Baron <vgatien-baron@janestreet.com>, Pierre-Yves David <pierre-yves.david@octobus.net> parents: 42057 diff changeset	698 # delta lower bound is larger than accepted upper bound
a0b26fc8fbba deltas: skip if projected delta size does not match text size constraint Valentin Gatien-Baron <vgatien-baron@janestreet.com>, Pierre-Yves David <pierre-yves.david@octobus.net> parents: 42057 diff changeset	699 continue
a0b26fc8fbba deltas: skip if projected delta size does not match text size constraint Valentin Gatien-Baron <vgatien-baron@janestreet.com>, Pierre-Yves David <pierre-yves.david@octobus.net> parents: 42057 diff changeset	700
42480 66c27df1be84 deltas: skip if projected delta size is bigger than previous snapshot Valentin Gatien-Baron <vgatien-baron@janestreet.com> parents: 42479 diff changeset	701 # check the relative constraint on the delta size
66c27df1be84 deltas: skip if projected delta size is bigger than previous snapshot Valentin Gatien-Baron <vgatien-baron@janestreet.com> parents: 42479 diff changeset	702 revlength = revlog.length(rev)
66c27df1be84 deltas: skip if projected delta size is bigger than previous snapshot Valentin Gatien-Baron <vgatien-baron@janestreet.com> parents: 42479 diff changeset	703 if revlength < lowestrealisticdeltalen:
66c27df1be84 deltas: skip if projected delta size is bigger than previous snapshot Valentin Gatien-Baron <vgatien-baron@janestreet.com> parents: 42479 diff changeset	704 # delta probable lower bound is larger than target base
66c27df1be84 deltas: skip if projected delta size is bigger than previous snapshot Valentin Gatien-Baron <vgatien-baron@janestreet.com> parents: 42479 diff changeset	705 continue
66c27df1be84 deltas: skip if projected delta size is bigger than previous snapshot Valentin Gatien-Baron <vgatien-baron@janestreet.com> parents: 42479 diff changeset	706
39364 37957e07138c revlogdeltas: move finddeltainfo filtering inside _candidategroups Boris Feld <boris.feld@octobus.net> parents: 39363 diff changeset	707 group.append(rev)
39363 1c6ff52fe9cf revlogdeltas: split candidate groups selection from the filtering logic Boris Feld <boris.feld@octobus.net> parents: 39362 diff changeset	708 if group:
39510 3ca144f1c8dd snapshot: search for unrelated but reusable full-snapshot Boris Feld <boris.feld@octobus.net> parents: 39509 diff changeset	709 # XXX: in the sparse revlog case, group can become large,
3ca144f1c8dd snapshot: search for unrelated but reusable full-snapshot Boris Feld <boris.feld@octobus.net> parents: 39509 diff changeset	710 # impacting performances. Some bounding or slicing mecanism
3ca144f1c8dd snapshot: search for unrelated but reusable full-snapshot Boris Feld <boris.feld@octobus.net> parents: 39509 diff changeset	711 # would help to reduce this impact.
39515 04b75f3a3f2a snapshot: add refining logic at the findeltainfo level Boris Feld <boris.feld@octobus.net> parents: 39514 diff changeset	712 good = yield tuple(group)
39514 5b308a4e6d03 snapshot: use None as a stop value when looking for a good delta Boris Feld <boris.feld@octobus.net> parents: 39513 diff changeset	713 yield None
39363 1c6ff52fe9cf revlogdeltas: split candidate groups selection from the filtering logic Boris Feld <boris.feld@octobus.net> parents: 39362 diff changeset	714
39510 3ca144f1c8dd snapshot: search for unrelated but reusable full-snapshot Boris Feld <boris.feld@octobus.net> parents: 39509 diff changeset	715 def _findsnapshots(revlog, cache, start_rev):
3ca144f1c8dd snapshot: search for unrelated but reusable full-snapshot Boris Feld <boris.feld@octobus.net> parents: 39509 diff changeset	716 """find snapshot from start_rev to tip"""
41111 38e88450138c delta: have a native implementation of _findsnapshot Boris Feld <boris.feld@octobus.net> parents: 41082 diff changeset	717 if util.safehasattr(revlog.index, 'findsnapshots'):
38e88450138c delta: have a native implementation of _findsnapshot Boris Feld <boris.feld@octobus.net> parents: 41082 diff changeset	718 revlog.index.findsnapshots(cache, start_rev)
38e88450138c delta: have a native implementation of _findsnapshot Boris Feld <boris.feld@octobus.net> parents: 41082 diff changeset	719 else:
38e88450138c delta: have a native implementation of _findsnapshot Boris Feld <boris.feld@octobus.net> parents: 41082 diff changeset	720 deltaparent = revlog.deltaparent
38e88450138c delta: have a native implementation of _findsnapshot Boris Feld <boris.feld@octobus.net> parents: 41082 diff changeset	721 issnapshot = revlog.issnapshot
38e88450138c delta: have a native implementation of _findsnapshot Boris Feld <boris.feld@octobus.net> parents: 41082 diff changeset	722 for rev in revlog.revs(start_rev):
38e88450138c delta: have a native implementation of _findsnapshot Boris Feld <boris.feld@octobus.net> parents: 41082 diff changeset	723 if issnapshot(rev):
38e88450138c delta: have a native implementation of _findsnapshot Boris Feld <boris.feld@octobus.net> parents: 41082 diff changeset	724 cache[deltaparent(rev)].append(rev)
39510 3ca144f1c8dd snapshot: search for unrelated but reusable full-snapshot Boris Feld <boris.feld@octobus.net> parents: 39509 diff changeset	725
39513 2f9f7889549b snapshot: introduce an intermediate `_refinedgroups` generator Boris Feld <boris.feld@octobus.net> parents: 39512 diff changeset	726 def _refinedgroups(revlog, p1, p2, cachedelta):
2f9f7889549b snapshot: introduce an intermediate `_refinedgroups` generator Boris Feld <boris.feld@octobus.net> parents: 39512 diff changeset	727 good = None
39518 993d7e2c8b79 snapshot: make sure we'll never refine delta base from a reused source Boris Feld <boris.feld@octobus.net> parents: 39517 diff changeset	728 # First we try to reuse a the delta contained in the bundle.
993d7e2c8b79 snapshot: make sure we'll never refine delta base from a reused source Boris Feld <boris.feld@octobus.net> parents: 39517 diff changeset	729 # (or from the source revlog)
993d7e2c8b79 snapshot: make sure we'll never refine delta base from a reused source Boris Feld <boris.feld@octobus.net> parents: 39517 diff changeset	730 #
993d7e2c8b79 snapshot: make sure we'll never refine delta base from a reused source Boris Feld <boris.feld@octobus.net> parents: 39517 diff changeset	731 # This logic only applies to general delta repositories and can be disabled
993d7e2c8b79 snapshot: make sure we'll never refine delta base from a reused source Boris Feld <boris.feld@octobus.net> parents: 39517 diff changeset	732 # through configuration. Disabling reuse source delta is useful when
993d7e2c8b79 snapshot: make sure we'll never refine delta base from a reused source Boris Feld <boris.feld@octobus.net> parents: 39517 diff changeset	733 # we want to make sure we recomputed "optimal" deltas.
993d7e2c8b79 snapshot: make sure we'll never refine delta base from a reused source Boris Feld <boris.feld@octobus.net> parents: 39517 diff changeset	734 if cachedelta and revlog._generaldelta and revlog._lazydeltabase:
993d7e2c8b79 snapshot: make sure we'll never refine delta base from a reused source Boris Feld <boris.feld@octobus.net> parents: 39517 diff changeset	735 # Assume what we received from the server is a good choice
993d7e2c8b79 snapshot: make sure we'll never refine delta base from a reused source Boris Feld <boris.feld@octobus.net> parents: 39517 diff changeset	736 # build delta will reuse the cache
993d7e2c8b79 snapshot: make sure we'll never refine delta base from a reused source Boris Feld <boris.feld@octobus.net> parents: 39517 diff changeset	737 good = yield (cachedelta[0],)
993d7e2c8b79 snapshot: make sure we'll never refine delta base from a reused source Boris Feld <boris.feld@octobus.net> parents: 39517 diff changeset	738 if good is not None:
993d7e2c8b79 snapshot: make sure we'll never refine delta base from a reused source Boris Feld <boris.feld@octobus.net> parents: 39517 diff changeset	739 yield None
993d7e2c8b79 snapshot: make sure we'll never refine delta base from a reused source Boris Feld <boris.feld@octobus.net> parents: 39517 diff changeset	740 return
41112 3e1960e23e6b delta: reuse _findsnapshot call from previous stage Boris Feld <boris.feld@octobus.net> parents: 41111 diff changeset	741 snapshots = collections.defaultdict(list)
3e1960e23e6b delta: reuse _findsnapshot call from previous stage Boris Feld <boris.feld@octobus.net> parents: 41111 diff changeset	742 for candidates in _rawgroups(revlog, p1, p2, cachedelta, snapshots):
39513 2f9f7889549b snapshot: introduce an intermediate `_refinedgroups` generator Boris Feld <boris.feld@octobus.net> parents: 39512 diff changeset	743 good = yield candidates
2f9f7889549b snapshot: introduce an intermediate `_refinedgroups` generator Boris Feld <boris.feld@octobus.net> parents: 39512 diff changeset	744 if good is not None:
2f9f7889549b snapshot: introduce an intermediate `_refinedgroups` generator Boris Feld <boris.feld@octobus.net> parents: 39512 diff changeset	745 break
39519 e4d4361d0bcd snapshot: try to refine new snapshot base down the chain Boris Feld <boris.feld@octobus.net> parents: 39518 diff changeset	746
40415 bafa1c4bb7a8 sparse-revlog: only refine delta candidates in the sparse case (issue6006) Boris Feld <boris.feld@octobus.net> parents: 39793 diff changeset	747 # If sparse revlog is enabled, we can try to refine the available deltas
bafa1c4bb7a8 sparse-revlog: only refine delta candidates in the sparse case (issue6006) Boris Feld <boris.feld@octobus.net> parents: 39793 diff changeset	748 if not revlog._sparserevlog:
bafa1c4bb7a8 sparse-revlog: only refine delta candidates in the sparse case (issue6006) Boris Feld <boris.feld@octobus.net> parents: 39793 diff changeset	749 yield None
bafa1c4bb7a8 sparse-revlog: only refine delta candidates in the sparse case (issue6006) Boris Feld <boris.feld@octobus.net> parents: 39793 diff changeset	750 return
bafa1c4bb7a8 sparse-revlog: only refine delta candidates in the sparse case (issue6006) Boris Feld <boris.feld@octobus.net> parents: 39793 diff changeset	751
39519 e4d4361d0bcd snapshot: try to refine new snapshot base down the chain Boris Feld <boris.feld@octobus.net> parents: 39518 diff changeset	752 # if we have a refinable value, try to refine it
e4d4361d0bcd snapshot: try to refine new snapshot base down the chain Boris Feld <boris.feld@octobus.net> parents: 39518 diff changeset	753 if good is not None and good not in (p1, p2) and revlog.issnapshot(good):
e4d4361d0bcd snapshot: try to refine new snapshot base down the chain Boris Feld <boris.feld@octobus.net> parents: 39518 diff changeset	754 # refine snapshot down
e4d4361d0bcd snapshot: try to refine new snapshot base down the chain Boris Feld <boris.feld@octobus.net> parents: 39518 diff changeset	755 previous = None
e4d4361d0bcd snapshot: try to refine new snapshot base down the chain Boris Feld <boris.feld@octobus.net> parents: 39518 diff changeset	756 while previous != good:
e4d4361d0bcd snapshot: try to refine new snapshot base down the chain Boris Feld <boris.feld@octobus.net> parents: 39518 diff changeset	757 previous = good
e4d4361d0bcd snapshot: try to refine new snapshot base down the chain Boris Feld <boris.feld@octobus.net> parents: 39518 diff changeset	758 base = revlog.deltaparent(good)
e4d4361d0bcd snapshot: try to refine new snapshot base down the chain Boris Feld <boris.feld@octobus.net> parents: 39518 diff changeset	759 if base == nullrev:
e4d4361d0bcd snapshot: try to refine new snapshot base down the chain Boris Feld <boris.feld@octobus.net> parents: 39518 diff changeset	760 break
e4d4361d0bcd snapshot: try to refine new snapshot base down the chain Boris Feld <boris.feld@octobus.net> parents: 39518 diff changeset	761 good = yield (base,)
39520 5aef5afa8654 snapshot: refine candidate snapshot base upward Boris Feld <boris.feld@octobus.net> parents: 39519 diff changeset	762 # refine snapshot up
41112 3e1960e23e6b delta: reuse _findsnapshot call from previous stage Boris Feld <boris.feld@octobus.net> parents: 41111 diff changeset	763 if not snapshots:
3e1960e23e6b delta: reuse _findsnapshot call from previous stage Boris Feld <boris.feld@octobus.net> parents: 41111 diff changeset	764 _findsnapshots(revlog, snapshots, good + 1)
39520 5aef5afa8654 snapshot: refine candidate snapshot base upward Boris Feld <boris.feld@octobus.net> parents: 39519 diff changeset	765 previous = None
5aef5afa8654 snapshot: refine candidate snapshot base upward Boris Feld <boris.feld@octobus.net> parents: 39519 diff changeset	766 while good != previous:
5aef5afa8654 snapshot: refine candidate snapshot base upward Boris Feld <boris.feld@octobus.net> parents: 39519 diff changeset	767 previous = good
5aef5afa8654 snapshot: refine candidate snapshot base upward Boris Feld <boris.feld@octobus.net> parents: 39519 diff changeset	768 children = tuple(sorted(c for c in snapshots[good]))
5aef5afa8654 snapshot: refine candidate snapshot base upward Boris Feld <boris.feld@octobus.net> parents: 39519 diff changeset	769 good = yield children
5aef5afa8654 snapshot: refine candidate snapshot base upward Boris Feld <boris.feld@octobus.net> parents: 39519 diff changeset	770
39516 51cec7fb672e snapshot: also use None as a stop value for `_refinegroup` Boris Feld <boris.feld@octobus.net> parents: 39515 diff changeset	771 # we have found nothing
51cec7fb672e snapshot: also use None as a stop value for `_refinegroup` Boris Feld <boris.feld@octobus.net> parents: 39515 diff changeset	772 yield None
39513 2f9f7889549b snapshot: introduce an intermediate `_refinedgroups` generator Boris Feld <boris.feld@octobus.net> parents: 39512 diff changeset	773
41112 3e1960e23e6b delta: reuse _findsnapshot call from previous stage Boris Feld <boris.feld@octobus.net> parents: 41111 diff changeset	774 def _rawgroups(revlog, p1, p2, cachedelta, snapshots=None):
39363 1c6ff52fe9cf revlogdeltas: split candidate groups selection from the filtering logic Boris Feld <boris.feld@octobus.net> parents: 39362 diff changeset	775 """Provides group of revision to be tested as delta base
1c6ff52fe9cf revlogdeltas: split candidate groups selection from the filtering logic Boris Feld <boris.feld@octobus.net> parents: 39362 diff changeset	776
1c6ff52fe9cf revlogdeltas: split candidate groups selection from the filtering logic Boris Feld <boris.feld@octobus.net> parents: 39362 diff changeset	777 This lower level function focus on emitting delta theorically interresting
1c6ff52fe9cf revlogdeltas: split candidate groups selection from the filtering logic Boris Feld <boris.feld@octobus.net> parents: 39362 diff changeset	778 without looking it any practical details.
1c6ff52fe9cf revlogdeltas: split candidate groups selection from the filtering logic Boris Feld <boris.feld@octobus.net> parents: 39362 diff changeset	779
1c6ff52fe9cf revlogdeltas: split candidate groups selection from the filtering logic Boris Feld <boris.feld@octobus.net> parents: 39362 diff changeset	780 The group order aims at providing fast or small candidates first.
39361 507f5b1dd7c8 revlogdeltas: extract _getcandidaterevs in a function Boris Feld <boris.feld@octobus.net> parents: 39360 diff changeset	781 """
507f5b1dd7c8 revlogdeltas: extract _getcandidaterevs in a function Boris Feld <boris.feld@octobus.net> parents: 39360 diff changeset	782 gdelta = revlog._generaldelta
41312 189e06b2d719 revlog: make sure we never use sparserevlog without general delta (issue6056) Boris Feld <boris.feld@octobus.net> parents: 41112 diff changeset	783 # gate sparse behind general-delta because of issue6056
189e06b2d719 revlog: make sure we never use sparserevlog without general delta (issue6056) Boris Feld <boris.feld@octobus.net> parents: 41112 diff changeset	784 sparse = gdelta and revlog._sparserevlog
39361 507f5b1dd7c8 revlogdeltas: extract _getcandidaterevs in a function Boris Feld <boris.feld@octobus.net> parents: 39360 diff changeset	785 curr = len(revlog)
507f5b1dd7c8 revlogdeltas: extract _getcandidaterevs in a function Boris Feld <boris.feld@octobus.net> parents: 39360 diff changeset	786 prev = curr - 1
39509 a33f394b2bfd snapshot: try intermediate snapshot against parents' base Boris Feld <boris.feld@octobus.net> parents: 39505 diff changeset	787 deltachain = lambda rev: revlog._deltachain(rev)[0]
39361 507f5b1dd7c8 revlogdeltas: extract _getcandidaterevs in a function Boris Feld <boris.feld@octobus.net> parents: 39360 diff changeset	788
39363 1c6ff52fe9cf revlogdeltas: split candidate groups selection from the filtering logic Boris Feld <boris.feld@octobus.net> parents: 39362 diff changeset	789 if gdelta:
1c6ff52fe9cf revlogdeltas: split candidate groups selection from the filtering logic Boris Feld <boris.feld@octobus.net> parents: 39362 diff changeset	790 # exclude already lazy tested base if any
1c6ff52fe9cf revlogdeltas: split candidate groups selection from the filtering logic Boris Feld <boris.feld@octobus.net> parents: 39362 diff changeset	791 parents = [p for p in (p1, p2) if p != nullrev]
39361 507f5b1dd7c8 revlogdeltas: extract _getcandidaterevs in a function Boris Feld <boris.feld@octobus.net> parents: 39360 diff changeset	792
39363 1c6ff52fe9cf revlogdeltas: split candidate groups selection from the filtering logic Boris Feld <boris.feld@octobus.net> parents: 39362 diff changeset	793 if not revlog._deltabothparents and len(parents) == 2:
1c6ff52fe9cf revlogdeltas: split candidate groups selection from the filtering logic Boris Feld <boris.feld@octobus.net> parents: 39362 diff changeset	794 parents.sort()
1c6ff52fe9cf revlogdeltas: split candidate groups selection from the filtering logic Boris Feld <boris.feld@octobus.net> parents: 39362 diff changeset	795 # To minimize the chance of having to build a fulltext,
1c6ff52fe9cf revlogdeltas: split candidate groups selection from the filtering logic Boris Feld <boris.feld@octobus.net> parents: 39362 diff changeset	796 # pick first whichever parent is closest to us (max rev)
1c6ff52fe9cf revlogdeltas: split candidate groups selection from the filtering logic Boris Feld <boris.feld@octobus.net> parents: 39362 diff changeset	797 yield (parents[1],)
1c6ff52fe9cf revlogdeltas: split candidate groups selection from the filtering logic Boris Feld <boris.feld@octobus.net> parents: 39362 diff changeset	798 # then the other one (min rev) if the first did not fit
1c6ff52fe9cf revlogdeltas: split candidate groups selection from the filtering logic Boris Feld <boris.feld@octobus.net> parents: 39362 diff changeset	799 yield (parents[0],)
1c6ff52fe9cf revlogdeltas: split candidate groups selection from the filtering logic Boris Feld <boris.feld@octobus.net> parents: 39362 diff changeset	800 elif len(parents) > 0:
1c6ff52fe9cf revlogdeltas: split candidate groups selection from the filtering logic Boris Feld <boris.feld@octobus.net> parents: 39362 diff changeset	801 # Test all parents (1 or 2), and keep the best candidate
1c6ff52fe9cf revlogdeltas: split candidate groups selection from the filtering logic Boris Feld <boris.feld@octobus.net> parents: 39362 diff changeset	802 yield parents
39361 507f5b1dd7c8 revlogdeltas: extract _getcandidaterevs in a function Boris Feld <boris.feld@octobus.net> parents: 39360 diff changeset	803
39509 a33f394b2bfd snapshot: try intermediate snapshot against parents' base Boris Feld <boris.feld@octobus.net> parents: 39505 diff changeset	804 if sparse and parents:
41112 3e1960e23e6b delta: reuse _findsnapshot call from previous stage Boris Feld <boris.feld@octobus.net> parents: 41111 diff changeset	805 if snapshots is None:
3e1960e23e6b delta: reuse _findsnapshot call from previous stage Boris Feld <boris.feld@octobus.net> parents: 41111 diff changeset	806 # map: base-rev: snapshot-rev
3e1960e23e6b delta: reuse _findsnapshot call from previous stage Boris Feld <boris.feld@octobus.net> parents: 41111 diff changeset	807 snapshots = collections.defaultdict(list)
39509 a33f394b2bfd snapshot: try intermediate snapshot against parents' base Boris Feld <boris.feld@octobus.net> parents: 39505 diff changeset	808 # See if we can use an existing snapshot in the parent chains to use as
a33f394b2bfd snapshot: try intermediate snapshot against parents' base Boris Feld <boris.feld@octobus.net> parents: 39505 diff changeset	809 # a base for a new intermediate-snapshot
39511 e72130f58f5d snapshot: consider all snapshots in the parents' chains Boris Feld <boris.feld@octobus.net> parents: 39510 diff changeset	810 #
e72130f58f5d snapshot: consider all snapshots in the parents' chains Boris Feld <boris.feld@octobus.net> parents: 39510 diff changeset	811 # search for snapshot in parents delta chain
e72130f58f5d snapshot: consider all snapshots in the parents' chains Boris Feld <boris.feld@octobus.net> parents: 39510 diff changeset	812 # map: snapshot-level: snapshot-rev
e72130f58f5d snapshot: consider all snapshots in the parents' chains Boris Feld <boris.feld@octobus.net> parents: 39510 diff changeset	813 parents_snaps = collections.defaultdict(set)
39521 05a165dc4f55 snapshot: extract parent chain computation Boris Feld <boris.feld@octobus.net> parents: 39520 diff changeset	814 candidate_chains = [deltachain(p) for p in parents]
05a165dc4f55 snapshot: extract parent chain computation Boris Feld <boris.feld@octobus.net> parents: 39520 diff changeset	815 for chain in candidate_chains:
05a165dc4f55 snapshot: extract parent chain computation Boris Feld <boris.feld@octobus.net> parents: 39520 diff changeset	816 for idx, s in enumerate(chain):
39511 e72130f58f5d snapshot: consider all snapshots in the parents' chains Boris Feld <boris.feld@octobus.net> parents: 39510 diff changeset	817 if not revlog.issnapshot(s):
e72130f58f5d snapshot: consider all snapshots in the parents' chains Boris Feld <boris.feld@octobus.net> parents: 39510 diff changeset	818 break
e72130f58f5d snapshot: consider all snapshots in the parents' chains Boris Feld <boris.feld@octobus.net> parents: 39510 diff changeset	819 parents_snaps[idx].add(s)
39512 6a53842727c1 snapshot: consider unrelated snapshots at a similar level first Boris Feld <boris.feld@octobus.net> parents: 39511 diff changeset	820 snapfloor = min(parents_snaps[0]) + 1
6a53842727c1 snapshot: consider unrelated snapshots at a similar level first Boris Feld <boris.feld@octobus.net> parents: 39511 diff changeset	821 _findsnapshots(revlog, snapshots, snapfloor)
39522 c6b8eab5db19 snapshot: also consider the snapshot chain of one unrelated revision Boris Feld <boris.feld@octobus.net> parents: 39521 diff changeset	822 # search for the highest "unrelated" revision
c6b8eab5db19 snapshot: also consider the snapshot chain of one unrelated revision Boris Feld <boris.feld@octobus.net> parents: 39521 diff changeset	823 #
c6b8eab5db19 snapshot: also consider the snapshot chain of one unrelated revision Boris Feld <boris.feld@octobus.net> parents: 39521 diff changeset	824 # Adding snapshots used by "unrelated" revision increase the odd we
c6b8eab5db19 snapshot: also consider the snapshot chain of one unrelated revision Boris Feld <boris.feld@octobus.net> parents: 39521 diff changeset	825 # reuse an independant, yet better snapshot chain.
c6b8eab5db19 snapshot: also consider the snapshot chain of one unrelated revision Boris Feld <boris.feld@octobus.net> parents: 39521 diff changeset	826 #
c6b8eab5db19 snapshot: also consider the snapshot chain of one unrelated revision Boris Feld <boris.feld@octobus.net> parents: 39521 diff changeset	827 # XXX instead of building a set of revisions, we could lazily enumerate
c6b8eab5db19 snapshot: also consider the snapshot chain of one unrelated revision Boris Feld <boris.feld@octobus.net> parents: 39521 diff changeset	828 # over the chains. That would be more efficient, however we stick to
c6b8eab5db19 snapshot: also consider the snapshot chain of one unrelated revision Boris Feld <boris.feld@octobus.net> parents: 39521 diff changeset	829 # simple code for now.
c6b8eab5db19 snapshot: also consider the snapshot chain of one unrelated revision Boris Feld <boris.feld@octobus.net> parents: 39521 diff changeset	830 all_revs = set()
c6b8eab5db19 snapshot: also consider the snapshot chain of one unrelated revision Boris Feld <boris.feld@octobus.net> parents: 39521 diff changeset	831 for chain in candidate_chains:
c6b8eab5db19 snapshot: also consider the snapshot chain of one unrelated revision Boris Feld <boris.feld@octobus.net> parents: 39521 diff changeset	832 all_revs.update(chain)
c6b8eab5db19 snapshot: also consider the snapshot chain of one unrelated revision Boris Feld <boris.feld@octobus.net> parents: 39521 diff changeset	833 other = None
c6b8eab5db19 snapshot: also consider the snapshot chain of one unrelated revision Boris Feld <boris.feld@octobus.net> parents: 39521 diff changeset	834 for r in revlog.revs(prev, snapfloor):
c6b8eab5db19 snapshot: also consider the snapshot chain of one unrelated revision Boris Feld <boris.feld@octobus.net> parents: 39521 diff changeset	835 if r not in all_revs:
c6b8eab5db19 snapshot: also consider the snapshot chain of one unrelated revision Boris Feld <boris.feld@octobus.net> parents: 39521 diff changeset	836 other = r
c6b8eab5db19 snapshot: also consider the snapshot chain of one unrelated revision Boris Feld <boris.feld@octobus.net> parents: 39521 diff changeset	837 break
c6b8eab5db19 snapshot: also consider the snapshot chain of one unrelated revision Boris Feld <boris.feld@octobus.net> parents: 39521 diff changeset	838 if other is not None:
c6b8eab5db19 snapshot: also consider the snapshot chain of one unrelated revision Boris Feld <boris.feld@octobus.net> parents: 39521 diff changeset	839 # To avoid unfair competition, we won't use unrelated intermediate
c6b8eab5db19 snapshot: also consider the snapshot chain of one unrelated revision Boris Feld <boris.feld@octobus.net> parents: 39521 diff changeset	840 # snapshot that are deeper than the ones from the parent delta
c6b8eab5db19 snapshot: also consider the snapshot chain of one unrelated revision Boris Feld <boris.feld@octobus.net> parents: 39521 diff changeset	841 # chain.
c6b8eab5db19 snapshot: also consider the snapshot chain of one unrelated revision Boris Feld <boris.feld@octobus.net> parents: 39521 diff changeset	842 max_depth = max(parents_snaps.keys())
c6b8eab5db19 snapshot: also consider the snapshot chain of one unrelated revision Boris Feld <boris.feld@octobus.net> parents: 39521 diff changeset	843 chain = deltachain(other)
c6b8eab5db19 snapshot: also consider the snapshot chain of one unrelated revision Boris Feld <boris.feld@octobus.net> parents: 39521 diff changeset	844 for idx, s in enumerate(chain):
c6b8eab5db19 snapshot: also consider the snapshot chain of one unrelated revision Boris Feld <boris.feld@octobus.net> parents: 39521 diff changeset	845 if s < snapfloor:
c6b8eab5db19 snapshot: also consider the snapshot chain of one unrelated revision Boris Feld <boris.feld@octobus.net> parents: 39521 diff changeset	846 continue
c6b8eab5db19 snapshot: also consider the snapshot chain of one unrelated revision Boris Feld <boris.feld@octobus.net> parents: 39521 diff changeset	847 if max_depth < idx:
c6b8eab5db19 snapshot: also consider the snapshot chain of one unrelated revision Boris Feld <boris.feld@octobus.net> parents: 39521 diff changeset	848 break
c6b8eab5db19 snapshot: also consider the snapshot chain of one unrelated revision Boris Feld <boris.feld@octobus.net> parents: 39521 diff changeset	849 if not revlog.issnapshot(s):
c6b8eab5db19 snapshot: also consider the snapshot chain of one unrelated revision Boris Feld <boris.feld@octobus.net> parents: 39521 diff changeset	850 break
c6b8eab5db19 snapshot: also consider the snapshot chain of one unrelated revision Boris Feld <boris.feld@octobus.net> parents: 39521 diff changeset	851 parents_snaps[idx].add(s)
39511 e72130f58f5d snapshot: consider all snapshots in the parents' chains Boris Feld <boris.feld@octobus.net> parents: 39510 diff changeset	852 # Test them as possible intermediate snapshot base
e72130f58f5d snapshot: consider all snapshots in the parents' chains Boris Feld <boris.feld@octobus.net> parents: 39510 diff changeset	853 # We test them from highest to lowest level. High level one are more
e72130f58f5d snapshot: consider all snapshots in the parents' chains Boris Feld <boris.feld@octobus.net> parents: 39510 diff changeset	854 # likely to result in small delta
39512 6a53842727c1 snapshot: consider unrelated snapshots at a similar level first Boris Feld <boris.feld@octobus.net> parents: 39511 diff changeset	855 floor = None
39511 e72130f58f5d snapshot: consider all snapshots in the parents' chains Boris Feld <boris.feld@octobus.net> parents: 39510 diff changeset	856 for idx, snaps in sorted(parents_snaps.items(), reverse=True):
39512 6a53842727c1 snapshot: consider unrelated snapshots at a similar level first Boris Feld <boris.feld@octobus.net> parents: 39511 diff changeset	857 siblings = set()
6a53842727c1 snapshot: consider unrelated snapshots at a similar level first Boris Feld <boris.feld@octobus.net> parents: 39511 diff changeset	858 for s in snaps:
6a53842727c1 snapshot: consider unrelated snapshots at a similar level first Boris Feld <boris.feld@octobus.net> parents: 39511 diff changeset	859 siblings.update(snapshots[s])
6a53842727c1 snapshot: consider unrelated snapshots at a similar level first Boris Feld <boris.feld@octobus.net> parents: 39511 diff changeset	860 # Before considering making a new intermediate snapshot, we check
6a53842727c1 snapshot: consider unrelated snapshots at a similar level first Boris Feld <boris.feld@octobus.net> parents: 39511 diff changeset	861 # if an existing snapshot, children of base we consider, would be
6a53842727c1 snapshot: consider unrelated snapshots at a similar level first Boris Feld <boris.feld@octobus.net> parents: 39511 diff changeset	862 # suitable.
6a53842727c1 snapshot: consider unrelated snapshots at a similar level first Boris Feld <boris.feld@octobus.net> parents: 39511 diff changeset	863 #
6a53842727c1 snapshot: consider unrelated snapshots at a similar level first Boris Feld <boris.feld@octobus.net> parents: 39511 diff changeset	864 # It give a change to reuse a delta chain "unrelated" to the
6a53842727c1 snapshot: consider unrelated snapshots at a similar level first Boris Feld <boris.feld@octobus.net> parents: 39511 diff changeset	865 # current revision instead of starting our own. Without such
6a53842727c1 snapshot: consider unrelated snapshots at a similar level first Boris Feld <boris.feld@octobus.net> parents: 39511 diff changeset	866 # re-use, topological branches would keep reopening new chains.
6a53842727c1 snapshot: consider unrelated snapshots at a similar level first Boris Feld <boris.feld@octobus.net> parents: 39511 diff changeset	867 # Creating more and more snapshot as the repository grow.
6a53842727c1 snapshot: consider unrelated snapshots at a similar level first Boris Feld <boris.feld@octobus.net> parents: 39511 diff changeset	868
6a53842727c1 snapshot: consider unrelated snapshots at a similar level first Boris Feld <boris.feld@octobus.net> parents: 39511 diff changeset	869 if floor is not None:
6a53842727c1 snapshot: consider unrelated snapshots at a similar level first Boris Feld <boris.feld@octobus.net> parents: 39511 diff changeset	870 # We only do this for siblings created after the one in our
6a53842727c1 snapshot: consider unrelated snapshots at a similar level first Boris Feld <boris.feld@octobus.net> parents: 39511 diff changeset	871 # parent's delta chain. Those created before has less chances
6a53842727c1 snapshot: consider unrelated snapshots at a similar level first Boris Feld <boris.feld@octobus.net> parents: 39511 diff changeset	872 # to be valid base since our ancestors had to create a new
6a53842727c1 snapshot: consider unrelated snapshots at a similar level first Boris Feld <boris.feld@octobus.net> parents: 39511 diff changeset	873 # snapshot.
6a53842727c1 snapshot: consider unrelated snapshots at a similar level first Boris Feld <boris.feld@octobus.net> parents: 39511 diff changeset	874 siblings = [r for r in siblings if floor < r]
6a53842727c1 snapshot: consider unrelated snapshots at a similar level first Boris Feld <boris.feld@octobus.net> parents: 39511 diff changeset	875 yield tuple(sorted(siblings))
6a53842727c1 snapshot: consider unrelated snapshots at a similar level first Boris Feld <boris.feld@octobus.net> parents: 39511 diff changeset	876 # then test the base from our parent's delta chain.
39511 e72130f58f5d snapshot: consider all snapshots in the parents' chains Boris Feld <boris.feld@octobus.net> parents: 39510 diff changeset	877 yield tuple(sorted(snaps))
39512 6a53842727c1 snapshot: consider unrelated snapshots at a similar level first Boris Feld <boris.feld@octobus.net> parents: 39511 diff changeset	878 floor = min(snaps)
39510 3ca144f1c8dd snapshot: search for unrelated but reusable full-snapshot Boris Feld <boris.feld@octobus.net> parents: 39509 diff changeset	879 # No suitable base found in the parent chain, search if any full
3ca144f1c8dd snapshot: search for unrelated but reusable full-snapshot Boris Feld <boris.feld@octobus.net> parents: 39509 diff changeset	880 # snapshots emitted since parent's base would be a suitable base for an
3ca144f1c8dd snapshot: search for unrelated but reusable full-snapshot Boris Feld <boris.feld@octobus.net> parents: 39509 diff changeset	881 # intermediate snapshot.
3ca144f1c8dd snapshot: search for unrelated but reusable full-snapshot Boris Feld <boris.feld@octobus.net> parents: 39509 diff changeset	882 #
3ca144f1c8dd snapshot: search for unrelated but reusable full-snapshot Boris Feld <boris.feld@octobus.net> parents: 39509 diff changeset	883 # It give a chance to reuse a delta chain unrelated to the current
3ca144f1c8dd snapshot: search for unrelated but reusable full-snapshot Boris Feld <boris.feld@octobus.net> parents: 39509 diff changeset	884 # revisions instead of starting our own. Without such re-use,
3ca144f1c8dd snapshot: search for unrelated but reusable full-snapshot Boris Feld <boris.feld@octobus.net> parents: 39509 diff changeset	885 # topological branches would keep reopening new full chains. Creating
3ca144f1c8dd snapshot: search for unrelated but reusable full-snapshot Boris Feld <boris.feld@octobus.net> parents: 39509 diff changeset	886 # more and more snapshot as the repository grow.
3ca144f1c8dd snapshot: search for unrelated but reusable full-snapshot Boris Feld <boris.feld@octobus.net> parents: 39509 diff changeset	887 yield tuple(snapshots[nullrev])
39509 a33f394b2bfd snapshot: try intermediate snapshot against parents' base Boris Feld <boris.feld@octobus.net> parents: 39505 diff changeset	888
39522 c6b8eab5db19 snapshot: also consider the snapshot chain of one unrelated revision Boris Feld <boris.feld@octobus.net> parents: 39521 diff changeset	889 if not sparse:
c6b8eab5db19 snapshot: also consider the snapshot chain of one unrelated revision Boris Feld <boris.feld@octobus.net> parents: 39521 diff changeset	890 # other approach failed try against prev to hopefully save us a
c6b8eab5db19 snapshot: also consider the snapshot chain of one unrelated revision Boris Feld <boris.feld@octobus.net> parents: 39521 diff changeset	891 # fulltext.
c6b8eab5db19 snapshot: also consider the snapshot chain of one unrelated revision Boris Feld <boris.feld@octobus.net> parents: 39521 diff changeset	892 yield (prev,)
39361 507f5b1dd7c8 revlogdeltas: extract _getcandidaterevs in a function Boris Feld <boris.feld@octobus.net> parents: 39360 diff changeset	893
39357 655b5b465953 revlog: split functionality related to deltas computation in a new module Boris Feld <boris.feld@octobus.net> parents: 39356 diff changeset	894 class deltacomputer(object):
35738 f90f6fd130c1 revlog: group delta computation methods under _deltacomputer object Paul Morelle <paul.morelle@octobus.net> parents: 35737 diff changeset	895 def __init__(self, revlog):
f90f6fd130c1 revlog: group delta computation methods under _deltacomputer object Paul Morelle <paul.morelle@octobus.net> parents: 35737 diff changeset	896 self.revlog = revlog
f90f6fd130c1 revlog: group delta computation methods under _deltacomputer object Paul Morelle <paul.morelle@octobus.net> parents: 35737 diff changeset	897
f90f6fd130c1 revlog: group delta computation methods under _deltacomputer object Paul Morelle <paul.morelle@octobus.net> parents: 35737 diff changeset	898 def buildtext(self, revinfo, fh):
f90f6fd130c1 revlog: group delta computation methods under _deltacomputer object Paul Morelle <paul.morelle@octobus.net> parents: 35737 diff changeset	899 """Builds a fulltext version of a revision
f90f6fd130c1 revlog: group delta computation methods under _deltacomputer object Paul Morelle <paul.morelle@octobus.net> parents: 35737 diff changeset	900
f90f6fd130c1 revlog: group delta computation methods under _deltacomputer object Paul Morelle <paul.morelle@octobus.net> parents: 35737 diff changeset	901 revinfo: _revisioninfo instance that contains all needed info
f90f6fd130c1 revlog: group delta computation methods under _deltacomputer object Paul Morelle <paul.morelle@octobus.net> parents: 35737 diff changeset	902 fh: file handle to either the .i or the .d revlog file,
f90f6fd130c1 revlog: group delta computation methods under _deltacomputer object Paul Morelle <paul.morelle@octobus.net> parents: 35737 diff changeset	903 depending on whether it is inlined or not
f90f6fd130c1 revlog: group delta computation methods under _deltacomputer object Paul Morelle <paul.morelle@octobus.net> parents: 35737 diff changeset	904 """
f90f6fd130c1 revlog: group delta computation methods under _deltacomputer object Paul Morelle <paul.morelle@octobus.net> parents: 35737 diff changeset	905 btext = revinfo.btext
f90f6fd130c1 revlog: group delta computation methods under _deltacomputer object Paul Morelle <paul.morelle@octobus.net> parents: 35737 diff changeset	906 if btext[0] is not None:
f90f6fd130c1 revlog: group delta computation methods under _deltacomputer object Paul Morelle <paul.morelle@octobus.net> parents: 35737 diff changeset	907 return btext[0]
f90f6fd130c1 revlog: group delta computation methods under _deltacomputer object Paul Morelle <paul.morelle@octobus.net> parents: 35737 diff changeset	908
f90f6fd130c1 revlog: group delta computation methods under _deltacomputer object Paul Morelle <paul.morelle@octobus.net> parents: 35737 diff changeset	909 revlog = self.revlog
f90f6fd130c1 revlog: group delta computation methods under _deltacomputer object Paul Morelle <paul.morelle@octobus.net> parents: 35737 diff changeset	910 cachedelta = revinfo.cachedelta
f90f6fd130c1 revlog: group delta computation methods under _deltacomputer object Paul Morelle <paul.morelle@octobus.net> parents: 35737 diff changeset	911 baserev = cachedelta[0]
f90f6fd130c1 revlog: group delta computation methods under _deltacomputer object Paul Morelle <paul.morelle@octobus.net> parents: 35737 diff changeset	912 delta = cachedelta[1]
f90f6fd130c1 revlog: group delta computation methods under _deltacomputer object Paul Morelle <paul.morelle@octobus.net> parents: 35737 diff changeset	913
39358 fd0150a3c2fe revlogdeltas: extra fulltext building in its own function Boris Feld <boris.feld@octobus.net> parents: 39357 diff changeset	914 fulltext = btext[0] = _textfromdelta(fh, revlog, baserev, delta,
fd0150a3c2fe revlogdeltas: extra fulltext building in its own function Boris Feld <boris.feld@octobus.net> parents: 39357 diff changeset	915 revinfo.p1, revinfo.p2,
fd0150a3c2fe revlogdeltas: extra fulltext building in its own function Boris Feld <boris.feld@octobus.net> parents: 39357 diff changeset	916 revinfo.flags, revinfo.node)
fd0150a3c2fe revlogdeltas: extra fulltext building in its own function Boris Feld <boris.feld@octobus.net> parents: 39357 diff changeset	917 return fulltext
35738 f90f6fd130c1 revlog: group delta computation methods under _deltacomputer object Paul Morelle <paul.morelle@octobus.net> parents: 35737 diff changeset	918
f90f6fd130c1 revlog: group delta computation methods under _deltacomputer object Paul Morelle <paul.morelle@octobus.net> parents: 35737 diff changeset	919 def _builddeltadiff(self, base, revinfo, fh):
f90f6fd130c1 revlog: group delta computation methods under _deltacomputer object Paul Morelle <paul.morelle@octobus.net> parents: 35737 diff changeset	920 revlog = self.revlog
f90f6fd130c1 revlog: group delta computation methods under _deltacomputer object Paul Morelle <paul.morelle@octobus.net> parents: 35737 diff changeset	921 t = self.buildtext(revinfo, fh)
f90f6fd130c1 revlog: group delta computation methods under _deltacomputer object Paul Morelle <paul.morelle@octobus.net> parents: 35737 diff changeset	922 if revlog.iscensored(base):
f90f6fd130c1 revlog: group delta computation methods under _deltacomputer object Paul Morelle <paul.morelle@octobus.net> parents: 35737 diff changeset	923 # deltas based on a censored revision must replace the
f90f6fd130c1 revlog: group delta computation methods under _deltacomputer object Paul Morelle <paul.morelle@octobus.net> parents: 35737 diff changeset	924 # full content in one patch, so delta works everywhere
f90f6fd130c1 revlog: group delta computation methods under _deltacomputer object Paul Morelle <paul.morelle@octobus.net> parents: 35737 diff changeset	925 header = mdiff.replacediffheader(revlog.rawsize(base), len(t))
f90f6fd130c1 revlog: group delta computation methods under _deltacomputer object Paul Morelle <paul.morelle@octobus.net> parents: 35737 diff changeset	926 delta = header + t
f90f6fd130c1 revlog: group delta computation methods under _deltacomputer object Paul Morelle <paul.morelle@octobus.net> parents: 35737 diff changeset	927 else:
f90f6fd130c1 revlog: group delta computation methods under _deltacomputer object Paul Morelle <paul.morelle@octobus.net> parents: 35737 diff changeset	928 ptext = revlog.revision(base, _df=fh, raw=True)
f90f6fd130c1 revlog: group delta computation methods under _deltacomputer object Paul Morelle <paul.morelle@octobus.net> parents: 35737 diff changeset	929 delta = mdiff.textdiff(ptext, t)
f90f6fd130c1 revlog: group delta computation methods under _deltacomputer object Paul Morelle <paul.morelle@octobus.net> parents: 35737 diff changeset	930
f90f6fd130c1 revlog: group delta computation methods under _deltacomputer object Paul Morelle <paul.morelle@octobus.net> parents: 35737 diff changeset	931 return delta
f90f6fd130c1 revlog: group delta computation methods under _deltacomputer object Paul Morelle <paul.morelle@octobus.net> parents: 35737 diff changeset	932
f90f6fd130c1 revlog: group delta computation methods under _deltacomputer object Paul Morelle <paul.morelle@octobus.net> parents: 35737 diff changeset	933 def _builddeltainfo(self, revinfo, base, fh):
f90f6fd130c1 revlog: group delta computation methods under _deltacomputer object Paul Morelle <paul.morelle@octobus.net> parents: 35737 diff changeset	934 # can we use the cached delta?
42481 6e9ba867a946 delta: move some delta chain related computation earlier in deltainfo Valentin Gatien-Baron <vgatien-baron@janestreet.com> parents: 42480 diff changeset	935 revlog = self.revlog
6e9ba867a946 delta: move some delta chain related computation earlier in deltainfo Valentin Gatien-Baron <vgatien-baron@janestreet.com> parents: 42480 diff changeset	936 chainbase = revlog.chainbase(base)
6e9ba867a946 delta: move some delta chain related computation earlier in deltainfo Valentin Gatien-Baron <vgatien-baron@janestreet.com> parents: 42480 diff changeset	937 if revlog._generaldelta:
6e9ba867a946 delta: move some delta chain related computation earlier in deltainfo Valentin Gatien-Baron <vgatien-baron@janestreet.com> parents: 42480 diff changeset	938 deltabase = base
6e9ba867a946 delta: move some delta chain related computation earlier in deltainfo Valentin Gatien-Baron <vgatien-baron@janestreet.com> parents: 42480 diff changeset	939 else:
6e9ba867a946 delta: move some delta chain related computation earlier in deltainfo Valentin Gatien-Baron <vgatien-baron@janestreet.com> parents: 42480 diff changeset	940 deltabase = chainbase
6e9ba867a946 delta: move some delta chain related computation earlier in deltainfo Valentin Gatien-Baron <vgatien-baron@janestreet.com> parents: 42480 diff changeset	941 snapshotdepth = None
6e9ba867a946 delta: move some delta chain related computation earlier in deltainfo Valentin Gatien-Baron <vgatien-baron@janestreet.com> parents: 42480 diff changeset	942 if revlog._sparserevlog and deltabase == nullrev:
6e9ba867a946 delta: move some delta chain related computation earlier in deltainfo Valentin Gatien-Baron <vgatien-baron@janestreet.com> parents: 42480 diff changeset	943 snapshotdepth = 0
6e9ba867a946 delta: move some delta chain related computation earlier in deltainfo Valentin Gatien-Baron <vgatien-baron@janestreet.com> parents: 42480 diff changeset	944 elif revlog._sparserevlog and revlog.issnapshot(deltabase):
6e9ba867a946 delta: move some delta chain related computation earlier in deltainfo Valentin Gatien-Baron <vgatien-baron@janestreet.com> parents: 42480 diff changeset	945 # A delta chain should always be one full snapshot,
6e9ba867a946 delta: move some delta chain related computation earlier in deltainfo Valentin Gatien-Baron <vgatien-baron@janestreet.com> parents: 42480 diff changeset	946 # zero or more semi-snapshots, and zero or more deltas
6e9ba867a946 delta: move some delta chain related computation earlier in deltainfo Valentin Gatien-Baron <vgatien-baron@janestreet.com> parents: 42480 diff changeset	947 p1, p2 = revlog.rev(revinfo.p1), revlog.rev(revinfo.p2)
6e9ba867a946 delta: move some delta chain related computation earlier in deltainfo Valentin Gatien-Baron <vgatien-baron@janestreet.com> parents: 42480 diff changeset	948 if deltabase not in (p1, p2) and revlog.issnapshot(deltabase):
6e9ba867a946 delta: move some delta chain related computation earlier in deltainfo Valentin Gatien-Baron <vgatien-baron@janestreet.com> parents: 42480 diff changeset	949 snapshotdepth = len(revlog._deltachain(deltabase)[0])
39611 a911932d5003 revlog: reuse cached delta for identical base revision (issue5975) Boris Feld <boris.feld@octobus.net> parents: 39610 diff changeset	950 delta = None
a911932d5003 revlog: reuse cached delta for identical base revision (issue5975) Boris Feld <boris.feld@octobus.net> parents: 39610 diff changeset	951 if revinfo.cachedelta:
a911932d5003 revlog: reuse cached delta for identical base revision (issue5975) Boris Feld <boris.feld@octobus.net> parents: 39610 diff changeset	952 cachebase, cachediff = revinfo.cachedelta
a911932d5003 revlog: reuse cached delta for identical base revision (issue5975) Boris Feld <boris.feld@octobus.net> parents: 39610 diff changeset	953 #check if the diff still apply
a911932d5003 revlog: reuse cached delta for identical base revision (issue5975) Boris Feld <boris.feld@octobus.net> parents: 39610 diff changeset	954 currentbase = cachebase
a911932d5003 revlog: reuse cached delta for identical base revision (issue5975) Boris Feld <boris.feld@octobus.net> parents: 39610 diff changeset	955 while (currentbase != nullrev
a911932d5003 revlog: reuse cached delta for identical base revision (issue5975) Boris Feld <boris.feld@octobus.net> parents: 39610 diff changeset	956 and currentbase != base
a911932d5003 revlog: reuse cached delta for identical base revision (issue5975) Boris Feld <boris.feld@octobus.net> parents: 39610 diff changeset	957 and self.revlog.length(currentbase) == 0):
a911932d5003 revlog: reuse cached delta for identical base revision (issue5975) Boris Feld <boris.feld@octobus.net> parents: 39610 diff changeset	958 currentbase = self.revlog.deltaparent(currentbase)
41824 688fc33e105d storage: introduce a `revlog.reuse-external-delta` config Pierre-Yves David <pierre-yves.david@octobus.net> parents: 41312 diff changeset	959 if self.revlog._lazydelta and currentbase == base:
39611 a911932d5003 revlog: reuse cached delta for identical base revision (issue5975) Boris Feld <boris.feld@octobus.net> parents: 39610 diff changeset	960 delta = revinfo.cachedelta[1]
a911932d5003 revlog: reuse cached delta for identical base revision (issue5975) Boris Feld <boris.feld@octobus.net> parents: 39610 diff changeset	961 if delta is None:
35738 f90f6fd130c1 revlog: group delta computation methods under _deltacomputer object Paul Morelle <paul.morelle@octobus.net> parents: 35737 diff changeset	962 delta = self._builddeltadiff(base, revinfo, fh)
42483 c1c1872d25d1 deltas: skip if projected compressed size does not match text size constraint Valentin Gatien-Baron <vgatien-baron@janestreet.com> parents: 42482 diff changeset	963 # snapshotdept need to be neither None nor 0 level snapshot
c1c1872d25d1 deltas: skip if projected compressed size does not match text size constraint Valentin Gatien-Baron <vgatien-baron@janestreet.com> parents: 42482 diff changeset	964 if revlog.upperboundcomp is not None and snapshotdepth:
c1c1872d25d1 deltas: skip if projected compressed size does not match text size constraint Valentin Gatien-Baron <vgatien-baron@janestreet.com> parents: 42482 diff changeset	965 lowestrealisticdeltalen = len(delta) // revlog.upperboundcomp
c1c1872d25d1 deltas: skip if projected compressed size does not match text size constraint Valentin Gatien-Baron <vgatien-baron@janestreet.com> parents: 42482 diff changeset	966 snapshotlimit = revinfo.textlen >> snapshotdepth
c1c1872d25d1 deltas: skip if projected compressed size does not match text size constraint Valentin Gatien-Baron <vgatien-baron@janestreet.com> parents: 42482 diff changeset	967 if snapshotlimit < lowestrealisticdeltalen:
c1c1872d25d1 deltas: skip if projected compressed size does not match text size constraint Valentin Gatien-Baron <vgatien-baron@janestreet.com> parents: 42482 diff changeset	968 return None
35738 f90f6fd130c1 revlog: group delta computation methods under _deltacomputer object Paul Morelle <paul.morelle@octobus.net> parents: 35737 diff changeset	969 header, data = revlog.compress(delta)
f90f6fd130c1 revlog: group delta computation methods under _deltacomputer object Paul Morelle <paul.morelle@octobus.net> parents: 35737 diff changeset	970 deltalen = len(header) + len(data)
f90f6fd130c1 revlog: group delta computation methods under _deltacomputer object Paul Morelle <paul.morelle@octobus.net> parents: 35737 diff changeset	971 offset = revlog.end(len(revlog) - 1)
f90f6fd130c1 revlog: group delta computation methods under _deltacomputer object Paul Morelle <paul.morelle@octobus.net> parents: 35737 diff changeset	972 dist = deltalen + offset - revlog.start(chainbase)
f90f6fd130c1 revlog: group delta computation methods under _deltacomputer object Paul Morelle <paul.morelle@octobus.net> parents: 35737 diff changeset	973 chainlen, compresseddeltalen = revlog._chaininfo(base)
f90f6fd130c1 revlog: group delta computation methods under _deltacomputer object Paul Morelle <paul.morelle@octobus.net> parents: 35737 diff changeset	974 chainlen += 1
f90f6fd130c1 revlog: group delta computation methods under _deltacomputer object Paul Morelle <paul.morelle@octobus.net> parents: 35737 diff changeset	975 compresseddeltalen += deltalen
39187 e0da43e2f71f revlog: compute snapshot depth on delta info Boris Feld <boris.feld@octobus.net> parents: 39185 diff changeset	976
35738 f90f6fd130c1 revlog: group delta computation methods under _deltacomputer object Paul Morelle <paul.morelle@octobus.net> parents: 35737 diff changeset	977 return _deltainfo(dist, deltalen, (header, data), deltabase,
39187 e0da43e2f71f revlog: compute snapshot depth on delta info Boris Feld <boris.feld@octobus.net> parents: 39185 diff changeset	978 chainbase, chainlen, compresseddeltalen,
e0da43e2f71f revlog: compute snapshot depth on delta info Boris Feld <boris.feld@octobus.net> parents: 39185 diff changeset	979 snapshotdepth)
35738 f90f6fd130c1 revlog: group delta computation methods under _deltacomputer object Paul Morelle <paul.morelle@octobus.net> parents: 35737 diff changeset	980
39360 5d343a24bff5 revlogdeltas: always return a delta info object in finddeltainfo Boris Feld <boris.feld@octobus.net> parents: 39359 diff changeset	981 def _fullsnapshotinfo(self, fh, revinfo):
5d343a24bff5 revlogdeltas: always return a delta info object in finddeltainfo Boris Feld <boris.feld@octobus.net> parents: 39359 diff changeset	982 curr = len(self.revlog)
5d343a24bff5 revlogdeltas: always return a delta info object in finddeltainfo Boris Feld <boris.feld@octobus.net> parents: 39359 diff changeset	983 rawtext = self.buildtext(revinfo, fh)
5d343a24bff5 revlogdeltas: always return a delta info object in finddeltainfo Boris Feld <boris.feld@octobus.net> parents: 39359 diff changeset	984 data = self.revlog.compress(rawtext)
5d343a24bff5 revlogdeltas: always return a delta info object in finddeltainfo Boris Feld <boris.feld@octobus.net> parents: 39359 diff changeset	985 compresseddeltalen = deltalen = dist = len(data[1]) + len(data[0])
5d343a24bff5 revlogdeltas: always return a delta info object in finddeltainfo Boris Feld <boris.feld@octobus.net> parents: 39359 diff changeset	986 deltabase = chainbase = curr
5d343a24bff5 revlogdeltas: always return a delta info object in finddeltainfo Boris Feld <boris.feld@octobus.net> parents: 39359 diff changeset	987 snapshotdepth = 0
5d343a24bff5 revlogdeltas: always return a delta info object in finddeltainfo Boris Feld <boris.feld@octobus.net> parents: 39359 diff changeset	988 chainlen = 1
5d343a24bff5 revlogdeltas: always return a delta info object in finddeltainfo Boris Feld <boris.feld@octobus.net> parents: 39359 diff changeset	989
5d343a24bff5 revlogdeltas: always return a delta info object in finddeltainfo Boris Feld <boris.feld@octobus.net> parents: 39359 diff changeset	990 return _deltainfo(dist, deltalen, data, deltabase,
5d343a24bff5 revlogdeltas: always return a delta info object in finddeltainfo Boris Feld <boris.feld@octobus.net> parents: 39359 diff changeset	991 chainbase, chainlen, compresseddeltalen,
5d343a24bff5 revlogdeltas: always return a delta info object in finddeltainfo Boris Feld <boris.feld@octobus.net> parents: 39359 diff changeset	992 snapshotdepth)
5d343a24bff5 revlogdeltas: always return a delta info object in finddeltainfo Boris Feld <boris.feld@octobus.net> parents: 39359 diff changeset	993
35738 f90f6fd130c1 revlog: group delta computation methods under _deltacomputer object Paul Morelle <paul.morelle@octobus.net> parents: 35737 diff changeset	994 def finddeltainfo(self, revinfo, fh):
f90f6fd130c1 revlog: group delta computation methods under _deltacomputer object Paul Morelle <paul.morelle@octobus.net> parents: 35737 diff changeset	995 """Find an acceptable delta against a candidate revision
f90f6fd130c1 revlog: group delta computation methods under _deltacomputer object Paul Morelle <paul.morelle@octobus.net> parents: 35737 diff changeset	996
f90f6fd130c1 revlog: group delta computation methods under _deltacomputer object Paul Morelle <paul.morelle@octobus.net> parents: 35737 diff changeset	997 revinfo: information about the revision (instance of _revisioninfo)
f90f6fd130c1 revlog: group delta computation methods under _deltacomputer object Paul Morelle <paul.morelle@octobus.net> parents: 35737 diff changeset	998 fh: file handle to either the .i or the .d revlog file,
f90f6fd130c1 revlog: group delta computation methods under _deltacomputer object Paul Morelle <paul.morelle@octobus.net> parents: 35737 diff changeset	999 depending on whether it is inlined or not
f90f6fd130c1 revlog: group delta computation methods under _deltacomputer object Paul Morelle <paul.morelle@octobus.net> parents: 35737 diff changeset	1000
f90f6fd130c1 revlog: group delta computation methods under _deltacomputer object Paul Morelle <paul.morelle@octobus.net> parents: 35737 diff changeset	1001 Returns the first acceptable candidate revision, as ordered by
39361 507f5b1dd7c8 revlogdeltas: extract _getcandidaterevs in a function Boris Feld <boris.feld@octobus.net> parents: 39360 diff changeset	1002 _candidategroups
39360 5d343a24bff5 revlogdeltas: always return a delta info object in finddeltainfo Boris Feld <boris.feld@octobus.net> parents: 39359 diff changeset	1003
5d343a24bff5 revlogdeltas: always return a delta info object in finddeltainfo Boris Feld <boris.feld@octobus.net> parents: 39359 diff changeset	1004 If no suitable deltabase is found, we return delta info for a full
5d343a24bff5 revlogdeltas: always return a delta info object in finddeltainfo Boris Feld <boris.feld@octobus.net> parents: 39359 diff changeset	1005 snapshot.
35738 f90f6fd130c1 revlog: group delta computation methods under _deltacomputer object Paul Morelle <paul.morelle@octobus.net> parents: 35737 diff changeset	1006 """
39122 dbb3e9e44fce revlog: do not search for delta for empty content Boris Feld <boris.feld@octobus.net> parents: 39121 diff changeset	1007 if not revinfo.textlen:
39360 5d343a24bff5 revlogdeltas: always return a delta info object in finddeltainfo Boris Feld <boris.feld@octobus.net> parents: 39359 diff changeset	1008 return self._fullsnapshotinfo(fh, revinfo)
39122 dbb3e9e44fce revlog: do not search for delta for empty content Boris Feld <boris.feld@octobus.net> parents: 39121 diff changeset	1009
39359 6f4b8f607a31 revlogdeltas: move special cases around raw revisions in finddeltainfo Boris Feld <boris.feld@octobus.net> parents: 39358 diff changeset	1010 # no delta for flag processor revision (see "candelta" for why)
6f4b8f607a31 revlogdeltas: move special cases around raw revisions in finddeltainfo Boris Feld <boris.feld@octobus.net> parents: 39358 diff changeset	1011 # not calling candelta since only one revision needs test, also to
6f4b8f607a31 revlogdeltas: move special cases around raw revisions in finddeltainfo Boris Feld <boris.feld@octobus.net> parents: 39358 diff changeset	1012 # avoid overhead fetching flags again.
6f4b8f607a31 revlogdeltas: move special cases around raw revisions in finddeltainfo Boris Feld <boris.feld@octobus.net> parents: 39358 diff changeset	1013 if revinfo.flags & REVIDX_RAWTEXT_CHANGING_FLAGS:
39360 5d343a24bff5 revlogdeltas: always return a delta info object in finddeltainfo Boris Feld <boris.feld@octobus.net> parents: 39359 diff changeset	1014 return self._fullsnapshotinfo(fh, revinfo)
39359 6f4b8f607a31 revlogdeltas: move special cases around raw revisions in finddeltainfo Boris Feld <boris.feld@octobus.net> parents: 39358 diff changeset	1015
35738 f90f6fd130c1 revlog: group delta computation methods under _deltacomputer object Paul Morelle <paul.morelle@octobus.net> parents: 35737 diff changeset	1016 cachedelta = revinfo.cachedelta
f90f6fd130c1 revlog: group delta computation methods under _deltacomputer object Paul Morelle <paul.morelle@octobus.net> parents: 35737 diff changeset	1017 p1 = revinfo.p1
f90f6fd130c1 revlog: group delta computation methods under _deltacomputer object Paul Morelle <paul.morelle@octobus.net> parents: 35737 diff changeset	1018 p2 = revinfo.p2
f90f6fd130c1 revlog: group delta computation methods under _deltacomputer object Paul Morelle <paul.morelle@octobus.net> parents: 35737 diff changeset	1019 revlog = self.revlog
f90f6fd130c1 revlog: group delta computation methods under _deltacomputer object Paul Morelle <paul.morelle@octobus.net> parents: 35737 diff changeset	1020
f90f6fd130c1 revlog: group delta computation methods under _deltacomputer object Paul Morelle <paul.morelle@octobus.net> parents: 35737 diff changeset	1021 deltainfo = None
39362 1441eb38849f revlogdeltas: pass revision number to _candidatesgroups Boris Feld <boris.feld@octobus.net> parents: 39361 diff changeset	1022 p1r, p2r = revlog.rev(p1), revlog.rev(p2)
39364 37957e07138c revlogdeltas: move finddeltainfo filtering inside _candidategroups Boris Feld <boris.feld@octobus.net> parents: 39363 diff changeset	1023 groups = _candidategroups(self.revlog, revinfo.textlen,
37957e07138c revlogdeltas: move finddeltainfo filtering inside _candidategroups Boris Feld <boris.feld@octobus.net> parents: 39363 diff changeset	1024 p1r, p2r, cachedelta)
39514 5b308a4e6d03 snapshot: use None as a stop value when looking for a good delta Boris Feld <boris.feld@octobus.net> parents: 39513 diff changeset	1025 candidaterevs = next(groups)
5b308a4e6d03 snapshot: use None as a stop value when looking for a good delta Boris Feld <boris.feld@octobus.net> parents: 39513 diff changeset	1026 while candidaterevs is not None:
35738 f90f6fd130c1 revlog: group delta computation methods under _deltacomputer object Paul Morelle <paul.morelle@octobus.net> parents: 35737 diff changeset	1027 nominateddeltas = []
39515 04b75f3a3f2a snapshot: add refining logic at the findeltainfo level Boris Feld <boris.feld@octobus.net> parents: 39514 diff changeset	1028 if deltainfo is not None:
04b75f3a3f2a snapshot: add refining logic at the findeltainfo level Boris Feld <boris.feld@octobus.net> parents: 39514 diff changeset	1029 # if we already found a good delta,
04b75f3a3f2a snapshot: add refining logic at the findeltainfo level Boris Feld <boris.feld@octobus.net> parents: 39514 diff changeset	1030 # challenge it against refined candidates
04b75f3a3f2a snapshot: add refining logic at the findeltainfo level Boris Feld <boris.feld@octobus.net> parents: 39514 diff changeset	1031 nominateddeltas.append(deltainfo)
35738 f90f6fd130c1 revlog: group delta computation methods under _deltacomputer object Paul Morelle <paul.morelle@octobus.net> parents: 35737 diff changeset	1032 for candidaterev in candidaterevs:
f90f6fd130c1 revlog: group delta computation methods under _deltacomputer object Paul Morelle <paul.morelle@octobus.net> parents: 35737 diff changeset	1033 candidatedelta = self._builddeltainfo(revinfo, candidaterev, fh)
42482 465f2d0df9ae deltas: accept and skip None return for delta info Valentin Gatien-Baron <vgatien-baron@janestreet.com> parents: 42481 diff changeset	1034 if candidatedelta is not None:
465f2d0df9ae deltas: accept and skip None return for delta info Valentin Gatien-Baron <vgatien-baron@janestreet.com> parents: 42481 diff changeset	1035 if isgooddeltainfo(self.revlog, candidatedelta, revinfo):
465f2d0df9ae deltas: accept and skip None return for delta info Valentin Gatien-Baron <vgatien-baron@janestreet.com> parents: 42481 diff changeset	1036 nominateddeltas.append(candidatedelta)
35738 f90f6fd130c1 revlog: group delta computation methods under _deltacomputer object Paul Morelle <paul.morelle@octobus.net> parents: 35737 diff changeset	1037 if nominateddeltas:
f90f6fd130c1 revlog: group delta computation methods under _deltacomputer object Paul Morelle <paul.morelle@octobus.net> parents: 35737 diff changeset	1038 deltainfo = min(nominateddeltas, key=lambda x: x.deltalen)
39515 04b75f3a3f2a snapshot: add refining logic at the findeltainfo level Boris Feld <boris.feld@octobus.net> parents: 39514 diff changeset	1039 if deltainfo is not None:
04b75f3a3f2a snapshot: add refining logic at the findeltainfo level Boris Feld <boris.feld@octobus.net> parents: 39514 diff changeset	1040 candidaterevs = groups.send(deltainfo.base)
04b75f3a3f2a snapshot: add refining logic at the findeltainfo level Boris Feld <boris.feld@octobus.net> parents: 39514 diff changeset	1041 else:
04b75f3a3f2a snapshot: add refining logic at the findeltainfo level Boris Feld <boris.feld@octobus.net> parents: 39514 diff changeset	1042 candidaterevs = next(groups)
35738 f90f6fd130c1 revlog: group delta computation methods under _deltacomputer object Paul Morelle <paul.morelle@octobus.net> parents: 35737 diff changeset	1043
39360 5d343a24bff5 revlogdeltas: always return a delta info object in finddeltainfo Boris Feld <boris.feld@octobus.net> parents: 39359 diff changeset	1044 if deltainfo is None:
5d343a24bff5 revlogdeltas: always return a delta info object in finddeltainfo Boris Feld <boris.feld@octobus.net> parents: 39359 diff changeset	1045 deltainfo = self._fullsnapshotinfo(fh, revinfo)
35738 f90f6fd130c1 revlog: group delta computation methods under _deltacomputer object Paul Morelle <paul.morelle@octobus.net> parents: 35737 diff changeset	1046 return deltainfo

Mercurial > public > mercurial-scm > hg-stable

annotate mercurial/revlogutils/deltas.py @ 42483:c1c1872d25d1