mercurial-scm/hg-stable: mercurial/branchmap.py annotate

annotate mercurial/branchmap.py @ 42001:624d6683c705

branchmap: remove the dict interface from the branchcache class (API) The current branchmap computation involves reading the whole branchmap from disk, validating all the nodes even if they are not required. This leads to a lot of time on repos which have large branchmap or a lot of branches. On large repos, this can validate around 1000's of nodes. On some operations, like finding whether a branch exists or not, we don't need to validate all the nodes. Or updating heads for a single branch. Before this patch, branchcache class was having dict interface and it was hard to keep track of reads. This patch removes the dict interface. Upcoming patches will implement lazy loading and validation of data and implement better API's. Differential Revision: https://phab.mercurial-scm.org/D6151

author	Pulkit Goyal <pulkit@yandex-team.ru>
date	Mon, 18 Mar 2019 18:59:38 +0300
parents	38de3300414f
children	662ffdde5adf

rev	line source
18116 bcee63733aad branchmap: create a mercurial.branchmap module Pierre-Yves David <pierre-yves.david@ens-lyon.org> parents: diff changeset	1 # branchmap.py - logic to computes, maintain and stores branchmap for local repo
bcee63733aad branchmap: create a mercurial.branchmap module Pierre-Yves David <pierre-yves.david@ens-lyon.org> parents: diff changeset	2 #
bcee63733aad branchmap: create a mercurial.branchmap module Pierre-Yves David <pierre-yves.david@ens-lyon.org> parents: diff changeset	3 # Copyright 2005-2007 Matt Mackall <mpm@selenic.com>
bcee63733aad branchmap: create a mercurial.branchmap module Pierre-Yves David <pierre-yves.david@ens-lyon.org> parents: diff changeset	4 #
bcee63733aad branchmap: create a mercurial.branchmap module Pierre-Yves David <pierre-yves.david@ens-lyon.org> parents: diff changeset	5 # This software may be used and distributed according to the terms of the
bcee63733aad branchmap: create a mercurial.branchmap module Pierre-Yves David <pierre-yves.david@ens-lyon.org> parents: diff changeset	6 # GNU General Public License version 2 or any later version.
18117 526e7ec5c96e branchmap: extract write logic from localrepo Pierre-Yves David <pierre-yves.david@ens-lyon.org> parents: 18116 diff changeset	7
25918 47f36e050c2e branchmap: use absolute_import Gregory Szorc <gregory.szorc@gmail.com> parents: 25660 diff changeset	8 from __future__ import absolute_import
47f36e050c2e branchmap: use absolute_import Gregory Szorc <gregory.szorc@gmail.com> parents: 25660 diff changeset	9
47f36e050c2e branchmap: use absolute_import Gregory Szorc <gregory.szorc@gmail.com> parents: 25660 diff changeset	10 import struct
47f36e050c2e branchmap: use absolute_import Gregory Szorc <gregory.szorc@gmail.com> parents: 25660 diff changeset	11
47f36e050c2e branchmap: use absolute_import Gregory Szorc <gregory.szorc@gmail.com> parents: 25660 diff changeset	12 from .node import (
47f36e050c2e branchmap: use absolute_import Gregory Szorc <gregory.szorc@gmail.com> parents: 25660 diff changeset	13 bin,
47f36e050c2e branchmap: use absolute_import Gregory Szorc <gregory.szorc@gmail.com> parents: 25660 diff changeset	14 hex,
47f36e050c2e branchmap: use absolute_import Gregory Szorc <gregory.szorc@gmail.com> parents: 25660 diff changeset	15 nullid,
47f36e050c2e branchmap: use absolute_import Gregory Szorc <gregory.szorc@gmail.com> parents: 25660 diff changeset	16 nullrev,
47f36e050c2e branchmap: use absolute_import Gregory Szorc <gregory.szorc@gmail.com> parents: 25660 diff changeset	17 )
47f36e050c2e branchmap: use absolute_import Gregory Szorc <gregory.szorc@gmail.com> parents: 25660 diff changeset	18 from . import (
47f36e050c2e branchmap: use absolute_import Gregory Szorc <gregory.szorc@gmail.com> parents: 25660 diff changeset	19 encoding,
26587 56b2bcea2529 error: get Abort from 'error' instead of 'util' Pierre-Yves David <pierre-yves.david@fb.com> parents: 26460 diff changeset	20 error,
35871 a011e4140435 branchmap: make error messages consistent between Python 2 and 3 Augie Fackler <augie@google.com> parents: 35499 diff changeset	21 pycompat,
25918 47f36e050c2e branchmap: use absolute_import Gregory Szorc <gregory.szorc@gmail.com> parents: 25660 diff changeset	22 scmutil,
30995 22fbca1d11ed mercurial: switch to util.timer for all interval timings Simon Farnsworth <simonfar@fb.com> parents: 29758 diff changeset	23 util,
25918 47f36e050c2e branchmap: use absolute_import Gregory Szorc <gregory.szorc@gmail.com> parents: 25660 diff changeset	24 )
37087 f0b6fbea00cf stringutil: bulk-replace call sites to point to new module Yuya Nishihara <yuya@tcha.org> parents: 36968 diff changeset	25 from .utils import (
f0b6fbea00cf stringutil: bulk-replace call sites to point to new module Yuya Nishihara <yuya@tcha.org> parents: 36968 diff changeset	26 stringutil,
f0b6fbea00cf stringutil: bulk-replace call sites to point to new module Yuya Nishihara <yuya@tcha.org> parents: 36968 diff changeset	27 )
25918 47f36e050c2e branchmap: use absolute_import Gregory Szorc <gregory.szorc@gmail.com> parents: 25660 diff changeset	28
47f36e050c2e branchmap: use absolute_import Gregory Szorc <gregory.szorc@gmail.com> parents: 25660 diff changeset	29 calcsize = struct.calcsize
31379 906be86990c4 rbc: use struct unpack_from and pack_into instead of unpack and pack Mads Kiilerich <madski@unity3d.com> parents: 31369 diff changeset	30 pack_into = struct.pack_into
906be86990c4 rbc: use struct unpack_from and pack_into instead of unpack and pack Mads Kiilerich <madski@unity3d.com> parents: 31369 diff changeset	31 unpack_from = struct.unpack_from
18117 526e7ec5c96e branchmap: extract write logic from localrepo Pierre-Yves David <pierre-yves.david@ens-lyon.org> parents: 18116 diff changeset	32
18118 e70ff1e599f4 branchmap: extract read logic from repo Pierre-Yves David <pierre-yves.david@logilab.fr> parents: 18117 diff changeset	33
20032 175c6fd8cacc subsettable: move from repoview to branchmap, the only place it's used Augie Fackler <raf@durin42.com> parents: 19839 diff changeset	34 ### Nearest subset relation
175c6fd8cacc subsettable: move from repoview to branchmap, the only place it's used Augie Fackler <raf@durin42.com> parents: 19839 diff changeset	35 # Nearest subset of filter X is a filter Y so that:
175c6fd8cacc subsettable: move from repoview to branchmap, the only place it's used Augie Fackler <raf@durin42.com> parents: 19839 diff changeset	36 # * Y is included in X,
175c6fd8cacc subsettable: move from repoview to branchmap, the only place it's used Augie Fackler <raf@durin42.com> parents: 19839 diff changeset	37 # * X - Y is as small as possible.
175c6fd8cacc subsettable: move from repoview to branchmap, the only place it's used Augie Fackler <raf@durin42.com> parents: 19839 diff changeset	38 # This create and ordering used for branchmap purpose.
175c6fd8cacc subsettable: move from repoview to branchmap, the only place it's used Augie Fackler <raf@durin42.com> parents: 19839 diff changeset	39 # the ordering may be partial
175c6fd8cacc subsettable: move from repoview to branchmap, the only place it's used Augie Fackler <raf@durin42.com> parents: 19839 diff changeset	40 subsettable = {None: 'visible',
35499 07fdac1d5c66 repoview: add a new filtername for accessing hidden commits Pulkit Goyal <7895pulkit@gmail.com> parents: 34371 diff changeset	41 'visible-hidden': 'visible',
20032 175c6fd8cacc subsettable: move from repoview to branchmap, the only place it's used Augie Fackler <raf@durin42.com> parents: 19839 diff changeset	42 'visible': 'served',
175c6fd8cacc subsettable: move from repoview to branchmap, the only place it's used Augie Fackler <raf@durin42.com> parents: 19839 diff changeset	43 'served': 'immutable',
175c6fd8cacc subsettable: move from repoview to branchmap, the only place it's used Augie Fackler <raf@durin42.com> parents: 19839 diff changeset	44 'immutable': 'base'}
175c6fd8cacc subsettable: move from repoview to branchmap, the only place it's used Augie Fackler <raf@durin42.com> parents: 19839 diff changeset	45
41626 328ca3b9e545 branchmap: encapsulate cache updating in the map itself Martijn Pieters <mj@octobus.net> parents: 41581 diff changeset	46
328ca3b9e545 branchmap: encapsulate cache updating in the map itself Martijn Pieters <mj@octobus.net> parents: 41581 diff changeset	47 class BranchMapCache(object):
41729 a87ca1d7e61d branchmap: improve doc about BranchMapCache class Pulkit Goyal <pulkit@yandex-team.ru> parents: 41689 diff changeset	48 """mapping of filtered views of repo with their branchcache"""
41626 328ca3b9e545 branchmap: encapsulate cache updating in the map itself Martijn Pieters <mj@octobus.net> parents: 41581 diff changeset	49 def __init__(self):
328ca3b9e545 branchmap: encapsulate cache updating in the map itself Martijn Pieters <mj@octobus.net> parents: 41581 diff changeset	50 self._per_filter = {}
41581 c795c462b1d6 branchmap: add some clarifications and clean up flow Martijn Pieters <mj@octobus.net> parents: 41580 diff changeset	51
41626 328ca3b9e545 branchmap: encapsulate cache updating in the map itself Martijn Pieters <mj@octobus.net> parents: 41581 diff changeset	52 def __getitem__(self, repo):
328ca3b9e545 branchmap: encapsulate cache updating in the map itself Martijn Pieters <mj@octobus.net> parents: 41581 diff changeset	53 self.updatecache(repo)
328ca3b9e545 branchmap: encapsulate cache updating in the map itself Martijn Pieters <mj@octobus.net> parents: 41581 diff changeset	54 return self._per_filter[repo.filtername]
328ca3b9e545 branchmap: encapsulate cache updating in the map itself Martijn Pieters <mj@octobus.net> parents: 41581 diff changeset	55
328ca3b9e545 branchmap: encapsulate cache updating in the map itself Martijn Pieters <mj@octobus.net> parents: 41581 diff changeset	56 def updatecache(self, repo):
328ca3b9e545 branchmap: encapsulate cache updating in the map itself Martijn Pieters <mj@octobus.net> parents: 41581 diff changeset	57 """Update the cache for the given filtered view on a repository"""
328ca3b9e545 branchmap: encapsulate cache updating in the map itself Martijn Pieters <mj@octobus.net> parents: 41581 diff changeset	58 # This can trigger updates for the caches for subsets of the filtered
328ca3b9e545 branchmap: encapsulate cache updating in the map itself Martijn Pieters <mj@octobus.net> parents: 41581 diff changeset	59 # view, e.g. when there is no cache for this filtered view or the cache
328ca3b9e545 branchmap: encapsulate cache updating in the map itself Martijn Pieters <mj@octobus.net> parents: 41581 diff changeset	60 # is stale.
18121 f8a13f061a8a branchmap: extract updatebranchcache from repo Pierre-Yves David <pierre-yves.david@logilab.fr> parents: 18120 diff changeset	61
41626 328ca3b9e545 branchmap: encapsulate cache updating in the map itself Martijn Pieters <mj@octobus.net> parents: 41581 diff changeset	62 cl = repo.changelog
328ca3b9e545 branchmap: encapsulate cache updating in the map itself Martijn Pieters <mj@octobus.net> parents: 41581 diff changeset	63 filtername = repo.filtername
328ca3b9e545 branchmap: encapsulate cache updating in the map itself Martijn Pieters <mj@octobus.net> parents: 41581 diff changeset	64 bcache = self._per_filter.get(filtername)
328ca3b9e545 branchmap: encapsulate cache updating in the map itself Martijn Pieters <mj@octobus.net> parents: 41581 diff changeset	65 if bcache is None or not bcache.validfor(repo):
328ca3b9e545 branchmap: encapsulate cache updating in the map itself Martijn Pieters <mj@octobus.net> parents: 41581 diff changeset	66 # cache object missing or cache object stale? Read from disk
328ca3b9e545 branchmap: encapsulate cache updating in the map itself Martijn Pieters <mj@octobus.net> parents: 41581 diff changeset	67 bcache = branchcache.fromfile(repo)
41581 c795c462b1d6 branchmap: add some clarifications and clean up flow Martijn Pieters <mj@octobus.net> parents: 41580 diff changeset	68
41626 328ca3b9e545 branchmap: encapsulate cache updating in the map itself Martijn Pieters <mj@octobus.net> parents: 41581 diff changeset	69 revs = []
328ca3b9e545 branchmap: encapsulate cache updating in the map itself Martijn Pieters <mj@octobus.net> parents: 41581 diff changeset	70 if bcache is None:
328ca3b9e545 branchmap: encapsulate cache updating in the map itself Martijn Pieters <mj@octobus.net> parents: 41581 diff changeset	71 # no (fresh) cache available anymore, perhaps we can re-use
328ca3b9e545 branchmap: encapsulate cache updating in the map itself Martijn Pieters <mj@octobus.net> parents: 41581 diff changeset	72 # the cache for a subset, then extend that to add info on missing
328ca3b9e545 branchmap: encapsulate cache updating in the map itself Martijn Pieters <mj@octobus.net> parents: 41581 diff changeset	73 # revisions.
328ca3b9e545 branchmap: encapsulate cache updating in the map itself Martijn Pieters <mj@octobus.net> parents: 41581 diff changeset	74 subsetname = subsettable.get(filtername)
328ca3b9e545 branchmap: encapsulate cache updating in the map itself Martijn Pieters <mj@octobus.net> parents: 41581 diff changeset	75 if subsetname is not None:
328ca3b9e545 branchmap: encapsulate cache updating in the map itself Martijn Pieters <mj@octobus.net> parents: 41581 diff changeset	76 subset = repo.filtered(subsetname)
328ca3b9e545 branchmap: encapsulate cache updating in the map itself Martijn Pieters <mj@octobus.net> parents: 41581 diff changeset	77 bcache = self[subset].copy()
328ca3b9e545 branchmap: encapsulate cache updating in the map itself Martijn Pieters <mj@octobus.net> parents: 41581 diff changeset	78 extrarevs = subset.changelog.filteredrevs - cl.filteredrevs
328ca3b9e545 branchmap: encapsulate cache updating in the map itself Martijn Pieters <mj@octobus.net> parents: 41581 diff changeset	79 revs.extend(r for r in extrarevs if r <= bcache.tiprev)
328ca3b9e545 branchmap: encapsulate cache updating in the map itself Martijn Pieters <mj@octobus.net> parents: 41581 diff changeset	80 else:
328ca3b9e545 branchmap: encapsulate cache updating in the map itself Martijn Pieters <mj@octobus.net> parents: 41581 diff changeset	81 # nothing to fall back on, start empty.
328ca3b9e545 branchmap: encapsulate cache updating in the map itself Martijn Pieters <mj@octobus.net> parents: 41581 diff changeset	82 bcache = branchcache()
24373 59cc09240afb revbranchcache: move out of branchmap onto localrepo Durham Goode <durham@fb.com> parents: 24163 diff changeset	83
41626 328ca3b9e545 branchmap: encapsulate cache updating in the map itself Martijn Pieters <mj@octobus.net> parents: 41581 diff changeset	84 revs.extend(cl.revs(start=bcache.tiprev + 1))
328ca3b9e545 branchmap: encapsulate cache updating in the map itself Martijn Pieters <mj@octobus.net> parents: 41581 diff changeset	85 if revs:
328ca3b9e545 branchmap: encapsulate cache updating in the map itself Martijn Pieters <mj@octobus.net> parents: 41581 diff changeset	86 bcache.update(repo, revs)
18124 79db6d40bced branchmap: store branchcache in a dedicated object Pierre-Yves David <pierre-yves.david@logilab.fr> parents: 18121 diff changeset	87
41626 328ca3b9e545 branchmap: encapsulate cache updating in the map itself Martijn Pieters <mj@octobus.net> parents: 41581 diff changeset	88 assert bcache.validfor(repo), filtername
328ca3b9e545 branchmap: encapsulate cache updating in the map itself Martijn Pieters <mj@octobus.net> parents: 41581 diff changeset	89 self._per_filter[repo.filtername] = bcache
328ca3b9e545 branchmap: encapsulate cache updating in the map itself Martijn Pieters <mj@octobus.net> parents: 41581 diff changeset	90
328ca3b9e545 branchmap: encapsulate cache updating in the map itself Martijn Pieters <mj@octobus.net> parents: 41581 diff changeset	91 def replace(self, repo, remotebranchmap):
328ca3b9e545 branchmap: encapsulate cache updating in the map itself Martijn Pieters <mj@octobus.net> parents: 41581 diff changeset	92 """Replace the branchmap cache for a repo with a branch mapping.
328ca3b9e545 branchmap: encapsulate cache updating in the map itself Martijn Pieters <mj@octobus.net> parents: 41581 diff changeset	93
328ca3b9e545 branchmap: encapsulate cache updating in the map itself Martijn Pieters <mj@octobus.net> parents: 41581 diff changeset	94 This is likely only called during clone with a branch map from a
328ca3b9e545 branchmap: encapsulate cache updating in the map itself Martijn Pieters <mj@octobus.net> parents: 41581 diff changeset	95 remote.
26460 79ef867538ea branchmap: move branch cache code out of streamclone.py Gregory Szorc <gregory.szorc@gmail.com> parents: 25918 diff changeset	96
41626 328ca3b9e545 branchmap: encapsulate cache updating in the map itself Martijn Pieters <mj@octobus.net> parents: 41581 diff changeset	97 """
328ca3b9e545 branchmap: encapsulate cache updating in the map itself Martijn Pieters <mj@octobus.net> parents: 41581 diff changeset	98 cl = repo.changelog
328ca3b9e545 branchmap: encapsulate cache updating in the map itself Martijn Pieters <mj@octobus.net> parents: 41581 diff changeset	99 clrev = cl.rev
328ca3b9e545 branchmap: encapsulate cache updating in the map itself Martijn Pieters <mj@octobus.net> parents: 41581 diff changeset	100 clbranchinfo = cl.branchinfo
328ca3b9e545 branchmap: encapsulate cache updating in the map itself Martijn Pieters <mj@octobus.net> parents: 41581 diff changeset	101 rbheads = []
328ca3b9e545 branchmap: encapsulate cache updating in the map itself Martijn Pieters <mj@octobus.net> parents: 41581 diff changeset	102 closed = []
328ca3b9e545 branchmap: encapsulate cache updating in the map itself Martijn Pieters <mj@octobus.net> parents: 41581 diff changeset	103 for bheads in remotebranchmap.itervalues():
328ca3b9e545 branchmap: encapsulate cache updating in the map itself Martijn Pieters <mj@octobus.net> parents: 41581 diff changeset	104 rbheads += bheads
328ca3b9e545 branchmap: encapsulate cache updating in the map itself Martijn Pieters <mj@octobus.net> parents: 41581 diff changeset	105 for h in bheads:
328ca3b9e545 branchmap: encapsulate cache updating in the map itself Martijn Pieters <mj@octobus.net> parents: 41581 diff changeset	106 r = clrev(h)
328ca3b9e545 branchmap: encapsulate cache updating in the map itself Martijn Pieters <mj@octobus.net> parents: 41581 diff changeset	107 b, c = clbranchinfo(r)
328ca3b9e545 branchmap: encapsulate cache updating in the map itself Martijn Pieters <mj@octobus.net> parents: 41581 diff changeset	108 if c:
328ca3b9e545 branchmap: encapsulate cache updating in the map itself Martijn Pieters <mj@octobus.net> parents: 41581 diff changeset	109 closed.append(h)
26460 79ef867538ea branchmap: move branch cache code out of streamclone.py Gregory Szorc <gregory.szorc@gmail.com> parents: 25918 diff changeset	110
41626 328ca3b9e545 branchmap: encapsulate cache updating in the map itself Martijn Pieters <mj@octobus.net> parents: 41581 diff changeset	111 if rbheads:
328ca3b9e545 branchmap: encapsulate cache updating in the map itself Martijn Pieters <mj@octobus.net> parents: 41581 diff changeset	112 rtiprev = max((int(clrev(node)) for node in rbheads))
328ca3b9e545 branchmap: encapsulate cache updating in the map itself Martijn Pieters <mj@octobus.net> parents: 41581 diff changeset	113 cache = branchcache(
328ca3b9e545 branchmap: encapsulate cache updating in the map itself Martijn Pieters <mj@octobus.net> parents: 41581 diff changeset	114 remotebranchmap, repo[rtiprev].node(), rtiprev,
328ca3b9e545 branchmap: encapsulate cache updating in the map itself Martijn Pieters <mj@octobus.net> parents: 41581 diff changeset	115 closednodes=closed)
26460 79ef867538ea branchmap: move branch cache code out of streamclone.py Gregory Szorc <gregory.szorc@gmail.com> parents: 25918 diff changeset	116
41626 328ca3b9e545 branchmap: encapsulate cache updating in the map itself Martijn Pieters <mj@octobus.net> parents: 41581 diff changeset	117 # Try to stick it as low as possible
328ca3b9e545 branchmap: encapsulate cache updating in the map itself Martijn Pieters <mj@octobus.net> parents: 41581 diff changeset	118 # filter above served are unlikely to be fetch from a clone
328ca3b9e545 branchmap: encapsulate cache updating in the map itself Martijn Pieters <mj@octobus.net> parents: 41581 diff changeset	119 for candidate in ('base', 'immutable', 'served'):
328ca3b9e545 branchmap: encapsulate cache updating in the map itself Martijn Pieters <mj@octobus.net> parents: 41581 diff changeset	120 rview = repo.filtered(candidate)
328ca3b9e545 branchmap: encapsulate cache updating in the map itself Martijn Pieters <mj@octobus.net> parents: 41581 diff changeset	121 if cache.validfor(rview):
328ca3b9e545 branchmap: encapsulate cache updating in the map itself Martijn Pieters <mj@octobus.net> parents: 41581 diff changeset	122 self._per_filter[candidate] = cache
328ca3b9e545 branchmap: encapsulate cache updating in the map itself Martijn Pieters <mj@octobus.net> parents: 41581 diff changeset	123 cache.write(rview)
328ca3b9e545 branchmap: encapsulate cache updating in the map itself Martijn Pieters <mj@octobus.net> parents: 41581 diff changeset	124 return
328ca3b9e545 branchmap: encapsulate cache updating in the map itself Martijn Pieters <mj@octobus.net> parents: 41581 diff changeset	125
328ca3b9e545 branchmap: encapsulate cache updating in the map itself Martijn Pieters <mj@octobus.net> parents: 41581 diff changeset	126 def clear(self):
328ca3b9e545 branchmap: encapsulate cache updating in the map itself Martijn Pieters <mj@octobus.net> parents: 41581 diff changeset	127 self._per_filter.clear()
328ca3b9e545 branchmap: encapsulate cache updating in the map itself Martijn Pieters <mj@octobus.net> parents: 41581 diff changeset	128
26460 79ef867538ea branchmap: move branch cache code out of streamclone.py Gregory Szorc <gregory.szorc@gmail.com> parents: 25918 diff changeset	129
42001 624d6683c705 branchmap: remove the dict interface from the branchcache class (API) Pulkit Goyal <pulkit@yandex-team.ru> parents: 41844 diff changeset	130 class branchcache(object):
20181 b9515fb9e72a branchmap: add documentation on the branchcache on-disk format Brodie Rao <brodie@sf.io> parents: 20032 diff changeset	131 """A dict like object that hold branches heads cache.
b9515fb9e72a branchmap: add documentation on the branchcache on-disk format Brodie Rao <brodie@sf.io> parents: 20032 diff changeset	132
b9515fb9e72a branchmap: add documentation on the branchcache on-disk format Brodie Rao <brodie@sf.io> parents: 20032 diff changeset	133 This cache is used to avoid costly computations to determine all the
b9515fb9e72a branchmap: add documentation on the branchcache on-disk format Brodie Rao <brodie@sf.io> parents: 20032 diff changeset	134 branch heads of a repo.
b9515fb9e72a branchmap: add documentation on the branchcache on-disk format Brodie Rao <brodie@sf.io> parents: 20032 diff changeset	135
b9515fb9e72a branchmap: add documentation on the branchcache on-disk format Brodie Rao <brodie@sf.io> parents: 20032 diff changeset	136 The cache is serialized on disk in the following format:
b9515fb9e72a branchmap: add documentation on the branchcache on-disk format Brodie Rao <brodie@sf.io> parents: 20032 diff changeset	137
b9515fb9e72a branchmap: add documentation on the branchcache on-disk format Brodie Rao <brodie@sf.io> parents: 20032 diff changeset	138 <tip hex node> <tip rev number> [optional filtered repo hex hash]
20185 7d4219512823 branchmap: cache open/closed branch head information Brodie Rao <brodie@sf.io> parents: 20181 diff changeset	139 <branch head hex node> <open/closed state> <branch name>
7d4219512823 branchmap: cache open/closed branch head information Brodie Rao <brodie@sf.io> parents: 20181 diff changeset	140 <branch head hex node> <open/closed state> <branch name>
20181 b9515fb9e72a branchmap: add documentation on the branchcache on-disk format Brodie Rao <brodie@sf.io> parents: 20032 diff changeset	141 ...
b9515fb9e72a branchmap: add documentation on the branchcache on-disk format Brodie Rao <brodie@sf.io> parents: 20032 diff changeset	142
b9515fb9e72a branchmap: add documentation on the branchcache on-disk format Brodie Rao <brodie@sf.io> parents: 20032 diff changeset	143 The first line is used to check if the cache is still valid. If the
b9515fb9e72a branchmap: add documentation on the branchcache on-disk format Brodie Rao <brodie@sf.io> parents: 20032 diff changeset	144 branch cache is for a filtered repo view, an optional third hash is
b9515fb9e72a branchmap: add documentation on the branchcache on-disk format Brodie Rao <brodie@sf.io> parents: 20032 diff changeset	145 included that hashes the hashes of all filtered revisions.
20185 7d4219512823 branchmap: cache open/closed branch head information Brodie Rao <brodie@sf.io> parents: 20181 diff changeset	146
7d4219512823 branchmap: cache open/closed branch head information Brodie Rao <brodie@sf.io> parents: 20181 diff changeset	147 The open/closed state is represented by a single letter 'o' or 'c'.
7d4219512823 branchmap: cache open/closed branch head information Brodie Rao <brodie@sf.io> parents: 20181 diff changeset	148 This field can be used to avoid changelog reads when determining if a
7d4219512823 branchmap: cache open/closed branch head information Brodie Rao <brodie@sf.io> parents: 20181 diff changeset	149 branch head closes a branch or not.
20181 b9515fb9e72a branchmap: add documentation on the branchcache on-disk format Brodie Rao <brodie@sf.io> parents: 20032 diff changeset	150 """
41688 bfc49f1df615 branchmap: move __init__ up in branchcache class Pulkit Goyal <pulkit@yandex-team.ru> parents: 41626 diff changeset	151
bfc49f1df615 branchmap: move __init__ up in branchcache class Pulkit Goyal <pulkit@yandex-team.ru> parents: 41626 diff changeset	152 def __init__(self, entries=(), tipnode=nullid, tiprev=nullrev,
bfc49f1df615 branchmap: move __init__ up in branchcache class Pulkit Goyal <pulkit@yandex-team.ru> parents: 41626 diff changeset	153 filteredhash=None, closednodes=None):
bfc49f1df615 branchmap: move __init__ up in branchcache class Pulkit Goyal <pulkit@yandex-team.ru> parents: 41626 diff changeset	154 self.tipnode = tipnode
bfc49f1df615 branchmap: move __init__ up in branchcache class Pulkit Goyal <pulkit@yandex-team.ru> parents: 41626 diff changeset	155 self.tiprev = tiprev
bfc49f1df615 branchmap: move __init__ up in branchcache class Pulkit Goyal <pulkit@yandex-team.ru> parents: 41626 diff changeset	156 self.filteredhash = filteredhash
bfc49f1df615 branchmap: move __init__ up in branchcache class Pulkit Goyal <pulkit@yandex-team.ru> parents: 41626 diff changeset	157 # closednodes is a set of nodes that close their branch. If the branch
bfc49f1df615 branchmap: move __init__ up in branchcache class Pulkit Goyal <pulkit@yandex-team.ru> parents: 41626 diff changeset	158 # cache has been updated, it may contain nodes that are no longer
bfc49f1df615 branchmap: move __init__ up in branchcache class Pulkit Goyal <pulkit@yandex-team.ru> parents: 41626 diff changeset	159 # heads.
bfc49f1df615 branchmap: move __init__ up in branchcache class Pulkit Goyal <pulkit@yandex-team.ru> parents: 41626 diff changeset	160 if closednodes is None:
bfc49f1df615 branchmap: move __init__ up in branchcache class Pulkit Goyal <pulkit@yandex-team.ru> parents: 41626 diff changeset	161 self._closednodes = set()
bfc49f1df615 branchmap: move __init__ up in branchcache class Pulkit Goyal <pulkit@yandex-team.ru> parents: 41626 diff changeset	162 else:
bfc49f1df615 branchmap: move __init__ up in branchcache class Pulkit Goyal <pulkit@yandex-team.ru> parents: 41626 diff changeset	163 self._closednodes = closednodes
42001 624d6683c705 branchmap: remove the dict interface from the branchcache class (API) Pulkit Goyal <pulkit@yandex-team.ru> parents: 41844 diff changeset	164 self.entries = dict(entries)
624d6683c705 branchmap: remove the dict interface from the branchcache class (API) Pulkit Goyal <pulkit@yandex-team.ru> parents: 41844 diff changeset	165
624d6683c705 branchmap: remove the dict interface from the branchcache class (API) Pulkit Goyal <pulkit@yandex-team.ru> parents: 41844 diff changeset	166 def __iter__(self):
624d6683c705 branchmap: remove the dict interface from the branchcache class (API) Pulkit Goyal <pulkit@yandex-team.ru> parents: 41844 diff changeset	167 return iter(self.entries)
624d6683c705 branchmap: remove the dict interface from the branchcache class (API) Pulkit Goyal <pulkit@yandex-team.ru> parents: 41844 diff changeset	168
624d6683c705 branchmap: remove the dict interface from the branchcache class (API) Pulkit Goyal <pulkit@yandex-team.ru> parents: 41844 diff changeset	169 def __setitem__(self, key, value):
624d6683c705 branchmap: remove the dict interface from the branchcache class (API) Pulkit Goyal <pulkit@yandex-team.ru> parents: 41844 diff changeset	170 self.entries[key] = value
624d6683c705 branchmap: remove the dict interface from the branchcache class (API) Pulkit Goyal <pulkit@yandex-team.ru> parents: 41844 diff changeset	171
624d6683c705 branchmap: remove the dict interface from the branchcache class (API) Pulkit Goyal <pulkit@yandex-team.ru> parents: 41844 diff changeset	172 def __getitem__(self, key):
624d6683c705 branchmap: remove the dict interface from the branchcache class (API) Pulkit Goyal <pulkit@yandex-team.ru> parents: 41844 diff changeset	173 return self.entries[key]
624d6683c705 branchmap: remove the dict interface from the branchcache class (API) Pulkit Goyal <pulkit@yandex-team.ru> parents: 41844 diff changeset	174
624d6683c705 branchmap: remove the dict interface from the branchcache class (API) Pulkit Goyal <pulkit@yandex-team.ru> parents: 41844 diff changeset	175 def setdefault(self, *args):
624d6683c705 branchmap: remove the dict interface from the branchcache class (API) Pulkit Goyal <pulkit@yandex-team.ru> parents: 41844 diff changeset	176 return self.entries.setdefault(*args)
624d6683c705 branchmap: remove the dict interface from the branchcache class (API) Pulkit Goyal <pulkit@yandex-team.ru> parents: 41844 diff changeset	177
624d6683c705 branchmap: remove the dict interface from the branchcache class (API) Pulkit Goyal <pulkit@yandex-team.ru> parents: 41844 diff changeset	178 def iteritems(self):
624d6683c705 branchmap: remove the dict interface from the branchcache class (API) Pulkit Goyal <pulkit@yandex-team.ru> parents: 41844 diff changeset	179 return self.entries.iteritems()
624d6683c705 branchmap: remove the dict interface from the branchcache class (API) Pulkit Goyal <pulkit@yandex-team.ru> parents: 41844 diff changeset	180
624d6683c705 branchmap: remove the dict interface from the branchcache class (API) Pulkit Goyal <pulkit@yandex-team.ru> parents: 41844 diff changeset	181 def itervalues(self):
624d6683c705 branchmap: remove the dict interface from the branchcache class (API) Pulkit Goyal <pulkit@yandex-team.ru> parents: 41844 diff changeset	182 return self.entries.itervalues()
41688 bfc49f1df615 branchmap: move __init__ up in branchcache class Pulkit Goyal <pulkit@yandex-team.ru> parents: 41626 diff changeset	183
41579 bf7fb97aecf1 branchmap: make branchcache responsible for reading Martijn Pieters <mj@octobus.net> parents: 41418 diff changeset	184 @classmethod
bf7fb97aecf1 branchmap: make branchcache responsible for reading Martijn Pieters <mj@octobus.net> parents: 41418 diff changeset	185 def fromfile(cls, repo):
bf7fb97aecf1 branchmap: make branchcache responsible for reading Martijn Pieters <mj@octobus.net> parents: 41418 diff changeset	186 f = None
bf7fb97aecf1 branchmap: make branchcache responsible for reading Martijn Pieters <mj@octobus.net> parents: 41418 diff changeset	187 try:
bf7fb97aecf1 branchmap: make branchcache responsible for reading Martijn Pieters <mj@octobus.net> parents: 41418 diff changeset	188 f = repo.cachevfs(cls._filename(repo))
bf7fb97aecf1 branchmap: make branchcache responsible for reading Martijn Pieters <mj@octobus.net> parents: 41418 diff changeset	189 lineiter = iter(f)
bf7fb97aecf1 branchmap: make branchcache responsible for reading Martijn Pieters <mj@octobus.net> parents: 41418 diff changeset	190 cachekey = next(lineiter).rstrip('\n').split(" ", 2)
bf7fb97aecf1 branchmap: make branchcache responsible for reading Martijn Pieters <mj@octobus.net> parents: 41418 diff changeset	191 last, lrev = cachekey[:2]
bf7fb97aecf1 branchmap: make branchcache responsible for reading Martijn Pieters <mj@octobus.net> parents: 41418 diff changeset	192 last, lrev = bin(last), int(lrev)
bf7fb97aecf1 branchmap: make branchcache responsible for reading Martijn Pieters <mj@octobus.net> parents: 41418 diff changeset	193 filteredhash = None
bf7fb97aecf1 branchmap: make branchcache responsible for reading Martijn Pieters <mj@octobus.net> parents: 41418 diff changeset	194 if len(cachekey) > 2:
bf7fb97aecf1 branchmap: make branchcache responsible for reading Martijn Pieters <mj@octobus.net> parents: 41418 diff changeset	195 filteredhash = bin(cachekey[2])
bf7fb97aecf1 branchmap: make branchcache responsible for reading Martijn Pieters <mj@octobus.net> parents: 41418 diff changeset	196 bcache = cls(tipnode=last, tiprev=lrev, filteredhash=filteredhash)
bf7fb97aecf1 branchmap: make branchcache responsible for reading Martijn Pieters <mj@octobus.net> parents: 41418 diff changeset	197 if not bcache.validfor(repo):
bf7fb97aecf1 branchmap: make branchcache responsible for reading Martijn Pieters <mj@octobus.net> parents: 41418 diff changeset	198 # invalidate the cache
bf7fb97aecf1 branchmap: make branchcache responsible for reading Martijn Pieters <mj@octobus.net> parents: 41418 diff changeset	199 raise ValueError(r'tip differs')
41813 8ad46ac6728e branchmap: prevent reading the file twice through different iterators Pulkit Goyal <pulkit@yandex-team.ru> parents: 41802 diff changeset	200 bcache.load(repo, lineiter)
41579 bf7fb97aecf1 branchmap: make branchcache responsible for reading Martijn Pieters <mj@octobus.net> parents: 41418 diff changeset	201 except (IOError, OSError):
bf7fb97aecf1 branchmap: make branchcache responsible for reading Martijn Pieters <mj@octobus.net> parents: 41418 diff changeset	202 return None
bf7fb97aecf1 branchmap: make branchcache responsible for reading Martijn Pieters <mj@octobus.net> parents: 41418 diff changeset	203
bf7fb97aecf1 branchmap: make branchcache responsible for reading Martijn Pieters <mj@octobus.net> parents: 41418 diff changeset	204 except Exception as inst:
bf7fb97aecf1 branchmap: make branchcache responsible for reading Martijn Pieters <mj@octobus.net> parents: 41418 diff changeset	205 if repo.ui.debugflag:
bf7fb97aecf1 branchmap: make branchcache responsible for reading Martijn Pieters <mj@octobus.net> parents: 41418 diff changeset	206 msg = 'invalid branchheads cache'
bf7fb97aecf1 branchmap: make branchcache responsible for reading Martijn Pieters <mj@octobus.net> parents: 41418 diff changeset	207 if repo.filtername is not None:
bf7fb97aecf1 branchmap: make branchcache responsible for reading Martijn Pieters <mj@octobus.net> parents: 41418 diff changeset	208 msg += ' (%s)' % repo.filtername
bf7fb97aecf1 branchmap: make branchcache responsible for reading Martijn Pieters <mj@octobus.net> parents: 41418 diff changeset	209 msg += ': %s\n'
bf7fb97aecf1 branchmap: make branchcache responsible for reading Martijn Pieters <mj@octobus.net> parents: 41418 diff changeset	210 repo.ui.debug(msg % pycompat.bytestr(inst))
bf7fb97aecf1 branchmap: make branchcache responsible for reading Martijn Pieters <mj@octobus.net> parents: 41418 diff changeset	211 bcache = None
bf7fb97aecf1 branchmap: make branchcache responsible for reading Martijn Pieters <mj@octobus.net> parents: 41418 diff changeset	212
bf7fb97aecf1 branchmap: make branchcache responsible for reading Martijn Pieters <mj@octobus.net> parents: 41418 diff changeset	213 finally:
bf7fb97aecf1 branchmap: make branchcache responsible for reading Martijn Pieters <mj@octobus.net> parents: 41418 diff changeset	214 if f:
bf7fb97aecf1 branchmap: make branchcache responsible for reading Martijn Pieters <mj@octobus.net> parents: 41418 diff changeset	215 f.close()
bf7fb97aecf1 branchmap: make branchcache responsible for reading Martijn Pieters <mj@octobus.net> parents: 41418 diff changeset	216
bf7fb97aecf1 branchmap: make branchcache responsible for reading Martijn Pieters <mj@octobus.net> parents: 41418 diff changeset	217 return bcache
bf7fb97aecf1 branchmap: make branchcache responsible for reading Martijn Pieters <mj@octobus.net> parents: 41418 diff changeset	218
41813 8ad46ac6728e branchmap: prevent reading the file twice through different iterators Pulkit Goyal <pulkit@yandex-team.ru> parents: 41802 diff changeset	219 def load(self, repo, lineiter):
8ad46ac6728e branchmap: prevent reading the file twice through different iterators Pulkit Goyal <pulkit@yandex-team.ru> parents: 41802 diff changeset	220 """ fully loads the branchcache by reading from the file using the line
8ad46ac6728e branchmap: prevent reading the file twice through different iterators Pulkit Goyal <pulkit@yandex-team.ru> parents: 41802 diff changeset	221 iterator passed"""
41802 68bbcc70e274 branchcache: move loading of branch names and nodes into it's own function Pulkit Goyal <pulkit@yandex-team.ru> parents: 41768 diff changeset	222 cl = repo.changelog
68bbcc70e274 branchcache: move loading of branch names and nodes into it's own function Pulkit Goyal <pulkit@yandex-team.ru> parents: 41768 diff changeset	223 for line in lineiter:
68bbcc70e274 branchcache: move loading of branch names and nodes into it's own function Pulkit Goyal <pulkit@yandex-team.ru> parents: 41768 diff changeset	224 line = line.rstrip('\n')
68bbcc70e274 branchcache: move loading of branch names and nodes into it's own function Pulkit Goyal <pulkit@yandex-team.ru> parents: 41768 diff changeset	225 if not line:
68bbcc70e274 branchcache: move loading of branch names and nodes into it's own function Pulkit Goyal <pulkit@yandex-team.ru> parents: 41768 diff changeset	226 continue
68bbcc70e274 branchcache: move loading of branch names and nodes into it's own function Pulkit Goyal <pulkit@yandex-team.ru> parents: 41768 diff changeset	227 node, state, label = line.split(" ", 2)
68bbcc70e274 branchcache: move loading of branch names and nodes into it's own function Pulkit Goyal <pulkit@yandex-team.ru> parents: 41768 diff changeset	228 if state not in 'oc':
68bbcc70e274 branchcache: move loading of branch names and nodes into it's own function Pulkit Goyal <pulkit@yandex-team.ru> parents: 41768 diff changeset	229 raise ValueError(r'invalid branch state')
68bbcc70e274 branchcache: move loading of branch names and nodes into it's own function Pulkit Goyal <pulkit@yandex-team.ru> parents: 41768 diff changeset	230 label = encoding.tolocal(label.strip())
68bbcc70e274 branchcache: move loading of branch names and nodes into it's own function Pulkit Goyal <pulkit@yandex-team.ru> parents: 41768 diff changeset	231 node = bin(node)
68bbcc70e274 branchcache: move loading of branch names and nodes into it's own function Pulkit Goyal <pulkit@yandex-team.ru> parents: 41768 diff changeset	232 if not cl.hasnode(node):
68bbcc70e274 branchcache: move loading of branch names and nodes into it's own function Pulkit Goyal <pulkit@yandex-team.ru> parents: 41768 diff changeset	233 raise ValueError(
68bbcc70e274 branchcache: move loading of branch names and nodes into it's own function Pulkit Goyal <pulkit@yandex-team.ru> parents: 41768 diff changeset	234 r'node %s does not exist' % pycompat.sysstr(hex(node)))
68bbcc70e274 branchcache: move loading of branch names and nodes into it's own function Pulkit Goyal <pulkit@yandex-team.ru> parents: 41768 diff changeset	235 self.setdefault(label, []).append(node)
68bbcc70e274 branchcache: move loading of branch names and nodes into it's own function Pulkit Goyal <pulkit@yandex-team.ru> parents: 41768 diff changeset	236 if state == 'c':
68bbcc70e274 branchcache: move loading of branch names and nodes into it's own function Pulkit Goyal <pulkit@yandex-team.ru> parents: 41768 diff changeset	237 self._closednodes.add(node)
68bbcc70e274 branchcache: move loading of branch names and nodes into it's own function Pulkit Goyal <pulkit@yandex-team.ru> parents: 41768 diff changeset	238
41579 bf7fb97aecf1 branchmap: make branchcache responsible for reading Martijn Pieters <mj@octobus.net> parents: 41418 diff changeset	239 @staticmethod
bf7fb97aecf1 branchmap: make branchcache responsible for reading Martijn Pieters <mj@octobus.net> parents: 41418 diff changeset	240 def _filename(repo):
bf7fb97aecf1 branchmap: make branchcache responsible for reading Martijn Pieters <mj@octobus.net> parents: 41418 diff changeset	241 """name of a branchcache file for a given repo or repoview"""
bf7fb97aecf1 branchmap: make branchcache responsible for reading Martijn Pieters <mj@octobus.net> parents: 41418 diff changeset	242 filename = "branch2"
bf7fb97aecf1 branchmap: make branchcache responsible for reading Martijn Pieters <mj@octobus.net> parents: 41418 diff changeset	243 if repo.filtername:
bf7fb97aecf1 branchmap: make branchcache responsible for reading Martijn Pieters <mj@octobus.net> parents: 41418 diff changeset	244 filename = '%s-%s' % (filename, repo.filtername)
bf7fb97aecf1 branchmap: make branchcache responsible for reading Martijn Pieters <mj@octobus.net> parents: 41418 diff changeset	245 return filename
18124 79db6d40bced branchmap: store branchcache in a dedicated object Pierre-Yves David <pierre-yves.david@logilab.fr> parents: 18121 diff changeset	246
18132 db25bf1dc828 branchmap: move validity logic in the object itself Pierre-Yves David <pierre-yves.david@ens-lyon.org> parents: 18131 diff changeset	247 def validfor(self, repo):
18644 3e92772d5383 spelling: fix some minor issues found by spell checker Mads Kiilerich <mads@kiilerich.com> parents: 18451 diff changeset	248 """Is the cache content valid regarding a repo
18132 db25bf1dc828 branchmap: move validity logic in the object itself Pierre-Yves David <pierre-yves.david@ens-lyon.org> parents: 18131 diff changeset	249
18644 3e92772d5383 spelling: fix some minor issues found by spell checker Mads Kiilerich <mads@kiilerich.com> parents: 18451 diff changeset	250 - False when cached tipnode is unknown or if we detect a strip.
18132 db25bf1dc828 branchmap: move validity logic in the object itself Pierre-Yves David <pierre-yves.david@ens-lyon.org> parents: 18131 diff changeset	251 - True when cache is up to date or a subset of current repo."""
db25bf1dc828 branchmap: move validity logic in the object itself Pierre-Yves David <pierre-yves.david@ens-lyon.org> parents: 18131 diff changeset	252 try:
18168 c351759ab0a0 branchmap: takes filtered revision in account for cache calculation Pierre-Yves David <pierre-yves.david@ens-lyon.org> parents: 18167 diff changeset	253 return ((self.tipnode == repo.changelog.node(self.tiprev))
41768 aaad36b88298 cleanup: use () to wrap long lines instead of \ Augie Fackler <augie@google.com> parents: 41729 diff changeset	254 and (self.filteredhash ==
24723 467a33142425 repoview: move function for computing filtered hash Gregory Szorc <gregory.szorc@gmail.com> parents: 24378 diff changeset	255 scmutil.filteredhash(repo, self.tiprev)))
18132 db25bf1dc828 branchmap: move validity logic in the object itself Pierre-Yves David <pierre-yves.david@ens-lyon.org> parents: 18131 diff changeset	256 except IndexError:
db25bf1dc828 branchmap: move validity logic in the object itself Pierre-Yves David <pierre-yves.david@ens-lyon.org> parents: 18131 diff changeset	257 return False
db25bf1dc828 branchmap: move validity logic in the object itself Pierre-Yves David <pierre-yves.david@ens-lyon.org> parents: 18131 diff changeset	258
20186 f5b461a4bc55 branchmap: introduce branchtip() method Brodie Rao <brodie@sf.io> parents: 20185 diff changeset	259 def _branchtip(self, heads):
20245 4edd179fefb8 help: branch names primarily denote the tipmost unclosed branch head Mads Kiilerich <madski@unity3d.com> parents: 20190 diff changeset	260 '''Return tuple with last open head in heads and false,
4edd179fefb8 help: branch names primarily denote the tipmost unclosed branch head Mads Kiilerich <madski@unity3d.com> parents: 20190 diff changeset	261 otherwise return last closed head and true.'''
20186 f5b461a4bc55 branchmap: introduce branchtip() method Brodie Rao <brodie@sf.io> parents: 20185 diff changeset	262 tip = heads[-1]
f5b461a4bc55 branchmap: introduce branchtip() method Brodie Rao <brodie@sf.io> parents: 20185 diff changeset	263 closed = True
f5b461a4bc55 branchmap: introduce branchtip() method Brodie Rao <brodie@sf.io> parents: 20185 diff changeset	264 for h in reversed(heads):
f5b461a4bc55 branchmap: introduce branchtip() method Brodie Rao <brodie@sf.io> parents: 20185 diff changeset	265 if h not in self._closednodes:
f5b461a4bc55 branchmap: introduce branchtip() method Brodie Rao <brodie@sf.io> parents: 20185 diff changeset	266 tip = h
f5b461a4bc55 branchmap: introduce branchtip() method Brodie Rao <brodie@sf.io> parents: 20185 diff changeset	267 closed = False
f5b461a4bc55 branchmap: introduce branchtip() method Brodie Rao <brodie@sf.io> parents: 20185 diff changeset	268 break
f5b461a4bc55 branchmap: introduce branchtip() method Brodie Rao <brodie@sf.io> parents: 20185 diff changeset	269 return tip, closed
f5b461a4bc55 branchmap: introduce branchtip() method Brodie Rao <brodie@sf.io> parents: 20185 diff changeset	270
f5b461a4bc55 branchmap: introduce branchtip() method Brodie Rao <brodie@sf.io> parents: 20185 diff changeset	271 def branchtip(self, branch):
20245 4edd179fefb8 help: branch names primarily denote the tipmost unclosed branch head Mads Kiilerich <madski@unity3d.com> parents: 20190 diff changeset	272 '''Return the tipmost open head on branch head, otherwise return the
4edd179fefb8 help: branch names primarily denote the tipmost unclosed branch head Mads Kiilerich <madski@unity3d.com> parents: 20190 diff changeset	273 tipmost closed head on branch.
4edd179fefb8 help: branch names primarily denote the tipmost unclosed branch head Mads Kiilerich <madski@unity3d.com> parents: 20190 diff changeset	274 Raise KeyError for unknown branch.'''
20186 f5b461a4bc55 branchmap: introduce branchtip() method Brodie Rao <brodie@sf.io> parents: 20185 diff changeset	275 return self._branchtip(self[branch])[0]
f5b461a4bc55 branchmap: introduce branchtip() method Brodie Rao <brodie@sf.io> parents: 20185 diff changeset	276
34091 abf91c4f9608 branches: correctly show inactive multiheaded branches the31k <the31k@thethirty.one> parents: 33661 diff changeset	277 def iteropen(self, nodes):
abf91c4f9608 branches: correctly show inactive multiheaded branches the31k <the31k@thethirty.one> parents: 33661 diff changeset	278 return (n for n in nodes if n not in self._closednodes)
abf91c4f9608 branches: correctly show inactive multiheaded branches the31k <the31k@thethirty.one> parents: 33661 diff changeset	279
20188 3a3727829607 branchmap: introduce branchheads() method Brodie Rao <brodie@sf.io> parents: 20186 diff changeset	280 def branchheads(self, branch, closed=False):
3a3727829607 branchmap: introduce branchheads() method Brodie Rao <brodie@sf.io> parents: 20186 diff changeset	281 heads = self[branch]
3a3727829607 branchmap: introduce branchheads() method Brodie Rao <brodie@sf.io> parents: 20186 diff changeset	282 if not closed:
34091 abf91c4f9608 branches: correctly show inactive multiheaded branches the31k <the31k@thethirty.one> parents: 33661 diff changeset	283 heads = list(self.iteropen(heads))
20188 3a3727829607 branchmap: introduce branchheads() method Brodie Rao <brodie@sf.io> parents: 20186 diff changeset	284 return heads
3a3727829607 branchmap: introduce branchheads() method Brodie Rao <brodie@sf.io> parents: 20186 diff changeset	285
20190 d5d25e541637 branchmap: introduce iterbranches() method Brodie Rao <brodie@sf.io> parents: 20188 diff changeset	286 def iterbranches(self):
d5d25e541637 branchmap: introduce iterbranches() method Brodie Rao <brodie@sf.io> parents: 20188 diff changeset	287 for bn, heads in self.iteritems():
d5d25e541637 branchmap: introduce iterbranches() method Brodie Rao <brodie@sf.io> parents: 20188 diff changeset	288 yield (bn, heads) + self._branchtip(heads)
d5d25e541637 branchmap: introduce iterbranches() method Brodie Rao <brodie@sf.io> parents: 20188 diff changeset	289
18232 dd0b636b0b65 branchmap: add a copy method Pierre-Yves David <pierre-yves.david@ens-lyon.org> parents: 18218 diff changeset	290 def copy(self):
dd0b636b0b65 branchmap: add a copy method Pierre-Yves David <pierre-yves.david@ens-lyon.org> parents: 18218 diff changeset	291 """return an deep copy of the branchcache object"""
42001 624d6683c705 branchmap: remove the dict interface from the branchcache class (API) Pulkit Goyal <pulkit@yandex-team.ru> parents: 41844 diff changeset	292 return branchcache(
624d6683c705 branchmap: remove the dict interface from the branchcache class (API) Pulkit Goyal <pulkit@yandex-team.ru> parents: 41844 diff changeset	293 self.entries, self.tipnode, self.tiprev, self.filteredhash,
41580 eb7ce452e0fb branchmap: updating triggers a write Martijn Pieters <mj@octobus.net> parents: 41579 diff changeset	294 self._closednodes)
18132 db25bf1dc828 branchmap: move validity logic in the object itself Pierre-Yves David <pierre-yves.david@ens-lyon.org> parents: 18131 diff changeset	295
18128 f0d56efaa35a branchmap: make write a method on the branchmap object Pierre-Yves David <pierre-yves.david@logilab.fr> parents: 18127 diff changeset	296 def write(self, repo):
f0d56efaa35a branchmap: make write a method on the branchmap object Pierre-Yves David <pierre-yves.david@logilab.fr> parents: 18127 diff changeset	297 try:
41579 bf7fb97aecf1 branchmap: make branchcache responsible for reading Martijn Pieters <mj@octobus.net> parents: 41418 diff changeset	298 f = repo.cachevfs(self._filename(repo), "w", atomictemp=True)
31357 fc06c5260639 branchmap: stringify int in a portable way Augie Fackler <augie@google.com> parents: 31356 diff changeset	299 cachekey = [hex(self.tipnode), '%d' % self.tiprev]
18184 8d48af68e2ae branchmap: read and write key part related to filtered revision Pierre-Yves David <pierre-yves.david@logilab.fr> parents: 18168 diff changeset	300 if self.filteredhash is not None:
8d48af68e2ae branchmap: read and write key part related to filtered revision Pierre-Yves David <pierre-yves.david@logilab.fr> parents: 18168 diff changeset	301 cachekey.append(hex(self.filteredhash))
8d48af68e2ae branchmap: read and write key part related to filtered revision Pierre-Yves David <pierre-yves.david@logilab.fr> parents: 18168 diff changeset	302 f.write(" ".join(cachekey) + '\n')
21031 05cfcecb3aef branchmap: log events related to branch cache Gregory Szorc <gregory.szorc@gmail.com> parents: 20264 diff changeset	303 nodecount = 0
18357 a4ab37ca887b localrepo: store branchheads sorted Mads Kiilerich <mads@kiilerich.com> parents: 18307 diff changeset	304 for label, nodes in sorted(self.iteritems()):
41689 9d0d8793e847 branchmap: decode a label only once Pulkit Goyal <pulkit@yandex-team.ru> parents: 41688 diff changeset	305 label = encoding.fromlocal(label)
18128 f0d56efaa35a branchmap: make write a method on the branchmap object Pierre-Yves David <pierre-yves.david@logilab.fr> parents: 18127 diff changeset	306 for node in nodes:
21031 05cfcecb3aef branchmap: log events related to branch cache Gregory Szorc <gregory.szorc@gmail.com> parents: 20264 diff changeset	307 nodecount += 1
20185 7d4219512823 branchmap: cache open/closed branch head information Brodie Rao <brodie@sf.io> parents: 20181 diff changeset	308 if node in self._closednodes:
7d4219512823 branchmap: cache open/closed branch head information Brodie Rao <brodie@sf.io> parents: 20181 diff changeset	309 state = 'c'
7d4219512823 branchmap: cache open/closed branch head information Brodie Rao <brodie@sf.io> parents: 20181 diff changeset	310 else:
7d4219512823 branchmap: cache open/closed branch head information Brodie Rao <brodie@sf.io> parents: 20181 diff changeset	311 state = 'o'
41689 9d0d8793e847 branchmap: decode a label only once Pulkit Goyal <pulkit@yandex-team.ru> parents: 41688 diff changeset	312 f.write("%s %s %s\n" % (hex(node), state, label))
18128 f0d56efaa35a branchmap: make write a method on the branchmap object Pierre-Yves David <pierre-yves.david@logilab.fr> parents: 18127 diff changeset	313 f.close()
21031 05cfcecb3aef branchmap: log events related to branch cache Gregory Szorc <gregory.szorc@gmail.com> parents: 20264 diff changeset	314 repo.ui.log('branchcache',
05cfcecb3aef branchmap: log events related to branch cache Gregory Szorc <gregory.szorc@gmail.com> parents: 20264 diff changeset	315 'wrote %s branch cache with %d labels and %d nodes\n',
42001 624d6683c705 branchmap: remove the dict interface from the branchcache class (API) Pulkit Goyal <pulkit@yandex-team.ru> parents: 41844 diff changeset	316 repo.filtername, len(self.entries), nodecount)
26587 56b2bcea2529 error: get Abort from 'error' instead of 'util' Pierre-Yves David <pierre-yves.david@fb.com> parents: 26460 diff changeset	317 except (IOError, OSError, error.Abort) as inst:
34371 d0db41af73c0 branchmap: remove superfluous pass statements Augie Fackler <augie@google.com> parents: 34091 diff changeset	318 # Abort may be raised by read only opener, so log and continue
36426 743b293c3ca0 py3: use util.forcebytestr to convert error messages to bytes Pulkit Goyal <7895pulkit@gmail.com> parents: 36239 diff changeset	319 repo.ui.debug("couldn't write branch cache: %s\n" %
37087 f0b6fbea00cf stringutil: bulk-replace call sites to point to new module Yuya Nishihara <yuya@tcha.org> parents: 36968 diff changeset	320 stringutil.forcebytestr(inst))
18131 f0eeb9b3444a branchmap: make update a method Pierre-Yves David <pierre-yves.david@ens-lyon.org> parents: 18130 diff changeset	321
18305 2502a15e033d branchmap: pass revision insteads of changectx to the update function Pierre-Yves David <pierre-yves.david@ens-lyon.org> parents: 18234 diff changeset	322 def update(self, repo, revgen):
18131 f0eeb9b3444a branchmap: make update a method Pierre-Yves David <pierre-yves.david@ens-lyon.org> parents: 18130 diff changeset	323 """Given a branchhead cache, self, that may have extra nodes or be
20263 ea4996754d91 branchmap: simplify update code Pierre-Yves David <pierre-yves.david@ens-lyon.org> parents: 20262 diff changeset	324 missing heads, and a generator of nodes that are strictly a superset of
18131 f0eeb9b3444a branchmap: make update a method Pierre-Yves David <pierre-yves.david@ens-lyon.org> parents: 18130 diff changeset	325 heads missing, this function updates self to be correct.
f0eeb9b3444a branchmap: make update a method Pierre-Yves David <pierre-yves.david@ens-lyon.org> parents: 18130 diff changeset	326 """
30995 22fbca1d11ed mercurial: switch to util.timer for all interval timings Simon Farnsworth <simonfar@fb.com> parents: 29758 diff changeset	327 starttime = util.timer()
18131 f0eeb9b3444a branchmap: make update a method Pierre-Yves David <pierre-yves.david@ens-lyon.org> parents: 18130 diff changeset	328 cl = repo.changelog
f0eeb9b3444a branchmap: make update a method Pierre-Yves David <pierre-yves.david@ens-lyon.org> parents: 18130 diff changeset	329 # collect new branch entries
f0eeb9b3444a branchmap: make update a method Pierre-Yves David <pierre-yves.david@ens-lyon.org> parents: 18130 diff changeset	330 newbranches = {}
24373 59cc09240afb revbranchcache: move out of branchmap onto localrepo Durham Goode <durham@fb.com> parents: 24163 diff changeset	331 getbranchinfo = repo.revbranchcache().branchinfo
18307 0eed2546118a branchmap: Save changectx creation during update Pierre-Yves David <pierre-yves.david@logilab.fr> parents: 18305 diff changeset	332 for r in revgen:
40494 5e5c8f2a1eb5 branchmap: do not specify changelog as an argument Yuya Nishihara <yuya@tcha.org> parents: 40375 diff changeset	333 branch, closesbranch = getbranchinfo(r)
20262 cf450ee3f8f7 branchmap: stop useless rev -> node -> rev round trip Pierre-Yves David <pierre-yves.david@ens-lyon.org> parents: 20261 diff changeset	334 newbranches.setdefault(branch, []).append(r)
20185 7d4219512823 branchmap: cache open/closed branch head information Brodie Rao <brodie@sf.io> parents: 20181 diff changeset	335 if closesbranch:
20262 cf450ee3f8f7 branchmap: stop useless rev -> node -> rev round trip Pierre-Yves David <pierre-yves.david@ens-lyon.org> parents: 20261 diff changeset	336 self._closednodes.add(cl.node(r))
22357 9c3c3dc14a65 branchmap: pre-filter topological heads before ancestors based filtering Pierre-Yves David <pierre-yves.david@fb.com> parents: 22356 diff changeset	337
9c3c3dc14a65 branchmap: pre-filter topological heads before ancestors based filtering Pierre-Yves David <pierre-yves.david@fb.com> parents: 22356 diff changeset	338 # fetch current topological heads to speed up filtering
9c3c3dc14a65 branchmap: pre-filter topological heads before ancestors based filtering Pierre-Yves David <pierre-yves.david@fb.com> parents: 22356 diff changeset	339 topoheads = set(cl.headrevs())
9c3c3dc14a65 branchmap: pre-filter topological heads before ancestors based filtering Pierre-Yves David <pierre-yves.david@fb.com> parents: 22356 diff changeset	340
18131 f0eeb9b3444a branchmap: make update a method Pierre-Yves David <pierre-yves.david@ens-lyon.org> parents: 18130 diff changeset	341 # if older branchheads are reachable from new ones, they aren't
f0eeb9b3444a branchmap: make update a method Pierre-Yves David <pierre-yves.david@ens-lyon.org> parents: 18130 diff changeset	342 # really branchheads. Note checking parents is insufficient:
f0eeb9b3444a branchmap: make update a method Pierre-Yves David <pierre-yves.david@ens-lyon.org> parents: 18130 diff changeset	343 # 1 (branch a) -> 2 (branch b) -> 3 (branch a)
20262 cf450ee3f8f7 branchmap: stop useless rev -> node -> rev round trip Pierre-Yves David <pierre-yves.david@ens-lyon.org> parents: 20261 diff changeset	344 for branch, newheadrevs in newbranches.iteritems():
18131 f0eeb9b3444a branchmap: make update a method Pierre-Yves David <pierre-yves.david@ens-lyon.org> parents: 18130 diff changeset	345 bheads = self.setdefault(branch, [])
20264 d9e1c167943b branchmap: use set for update code Pierre-Yves David <pierre-yves.david@ens-lyon.org> parents: 20263 diff changeset	346 bheadset = set(cl.rev(node) for node in bheads)
18131 f0eeb9b3444a branchmap: make update a method Pierre-Yves David <pierre-yves.david@ens-lyon.org> parents: 18130 diff changeset	347
20263 ea4996754d91 branchmap: simplify update code Pierre-Yves David <pierre-yves.david@ens-lyon.org> parents: 20262 diff changeset	348 # This have been tested True on all internal usage of this function.
ea4996754d91 branchmap: simplify update code Pierre-Yves David <pierre-yves.david@ens-lyon.org> parents: 20262 diff changeset	349 # run it again in case of doubt
ea4996754d91 branchmap: simplify update code Pierre-Yves David <pierre-yves.david@ens-lyon.org> parents: 20262 diff changeset	350 # assert not (set(bheadrevs) & set(newheadrevs))
20264 d9e1c167943b branchmap: use set for update code Pierre-Yves David <pierre-yves.david@ens-lyon.org> parents: 20263 diff changeset	351 bheadset.update(newheadrevs)
18131 f0eeb9b3444a branchmap: make update a method Pierre-Yves David <pierre-yves.david@ens-lyon.org> parents: 18130 diff changeset	352
22356 3c8fb24334e9 branchmap: issue a single call to `ancestors` for all heads Pierre-Yves David <pierre-yves.david@fb.com> parents: 22059 diff changeset	353 # This prunes out two kinds of heads - heads that are superseded by
3c8fb24334e9 branchmap: issue a single call to `ancestors` for all heads Pierre-Yves David <pierre-yves.david@fb.com> parents: 22059 diff changeset	354 # a head in newheadrevs, and newheadrevs that are not heads because
3c8fb24334e9 branchmap: issue a single call to `ancestors` for all heads Pierre-Yves David <pierre-yves.david@fb.com> parents: 22059 diff changeset	355 # an existing head is their descendant.
22357 9c3c3dc14a65 branchmap: pre-filter topological heads before ancestors based filtering Pierre-Yves David <pierre-yves.david@fb.com> parents: 22356 diff changeset	356 uncertain = bheadset - topoheads
9c3c3dc14a65 branchmap: pre-filter topological heads before ancestors based filtering Pierre-Yves David <pierre-yves.david@fb.com> parents: 22356 diff changeset	357 if uncertain:
9c3c3dc14a65 branchmap: pre-filter topological heads before ancestors based filtering Pierre-Yves David <pierre-yves.david@fb.com> parents: 22356 diff changeset	358 floorrev = min(uncertain)
9c3c3dc14a65 branchmap: pre-filter topological heads before ancestors based filtering Pierre-Yves David <pierre-yves.david@fb.com> parents: 22356 diff changeset	359 ancestors = set(cl.ancestors(newheadrevs, floorrev))
9c3c3dc14a65 branchmap: pre-filter topological heads before ancestors based filtering Pierre-Yves David <pierre-yves.david@fb.com> parents: 22356 diff changeset	360 bheadset -= ancestors
20264 d9e1c167943b branchmap: use set for update code Pierre-Yves David <pierre-yves.david@ens-lyon.org> parents: 20263 diff changeset	361 bheadrevs = sorted(bheadset)
18131 f0eeb9b3444a branchmap: make update a method Pierre-Yves David <pierre-yves.david@ens-lyon.org> parents: 18130 diff changeset	362 self[branch] = [cl.node(rev) for rev in bheadrevs]
20263 ea4996754d91 branchmap: simplify update code Pierre-Yves David <pierre-yves.david@ens-lyon.org> parents: 20262 diff changeset	363 tiprev = bheadrevs[-1]
18131 f0eeb9b3444a branchmap: make update a method Pierre-Yves David <pierre-yves.david@ens-lyon.org> parents: 18130 diff changeset	364 if tiprev > self.tiprev:
f0eeb9b3444a branchmap: make update a method Pierre-Yves David <pierre-yves.david@ens-lyon.org> parents: 18130 diff changeset	365 self.tipnode = cl.node(tiprev)
f0eeb9b3444a branchmap: make update a method Pierre-Yves David <pierre-yves.david@ens-lyon.org> parents: 18130 diff changeset	366 self.tiprev = tiprev
f0eeb9b3444a branchmap: make update a method Pierre-Yves David <pierre-yves.david@ens-lyon.org> parents: 18130 diff changeset	367
19838 23386881abeb branchmap: remove the droppednodes logic Pierre-Yves David <pierre-yves.david@ens-lyon.org> parents: 19837 diff changeset	368 if not self.validfor(repo):
18131 f0eeb9b3444a branchmap: make update a method Pierre-Yves David <pierre-yves.david@ens-lyon.org> parents: 18130 diff changeset	369 # cache key are not valid anymore
f0eeb9b3444a branchmap: make update a method Pierre-Yves David <pierre-yves.david@ens-lyon.org> parents: 18130 diff changeset	370 self.tipnode = nullid
f0eeb9b3444a branchmap: make update a method Pierre-Yves David <pierre-yves.david@ens-lyon.org> parents: 18130 diff changeset	371 self.tiprev = nullrev
42001 624d6683c705 branchmap: remove the dict interface from the branchcache class (API) Pulkit Goyal <pulkit@yandex-team.ru> parents: 41844 diff changeset	372 for heads in self.itervalues():
18131 f0eeb9b3444a branchmap: make update a method Pierre-Yves David <pierre-yves.david@ens-lyon.org> parents: 18130 diff changeset	373 tiprev = max(cl.rev(node) for node in heads)
f0eeb9b3444a branchmap: make update a method Pierre-Yves David <pierre-yves.david@ens-lyon.org> parents: 18130 diff changeset	374 if tiprev > self.tiprev:
f0eeb9b3444a branchmap: make update a method Pierre-Yves David <pierre-yves.david@ens-lyon.org> parents: 18130 diff changeset	375 self.tipnode = cl.node(tiprev)
f0eeb9b3444a branchmap: make update a method Pierre-Yves David <pierre-yves.david@ens-lyon.org> parents: 18130 diff changeset	376 self.tiprev = tiprev
24723 467a33142425 repoview: move function for computing filtered hash Gregory Szorc <gregory.szorc@gmail.com> parents: 24378 diff changeset	377 self.filteredhash = scmutil.filteredhash(repo, self.tiprev)
21031 05cfcecb3aef branchmap: log events related to branch cache Gregory Szorc <gregory.szorc@gmail.com> parents: 20264 diff changeset	378
30995 22fbca1d11ed mercurial: switch to util.timer for all interval timings Simon Farnsworth <simonfar@fb.com> parents: 29758 diff changeset	379 duration = util.timer() - starttime
21031 05cfcecb3aef branchmap: log events related to branch cache Gregory Szorc <gregory.szorc@gmail.com> parents: 20264 diff changeset	380 repo.ui.log('branchcache', 'updated %s branch cache in %.4f seconds\n',
41844 38de3300414f py3: convert filtername to str if it's None Pulkit Goyal <pulkit@yandex-team.ru> parents: 41813 diff changeset	381 repo.filtername or b'None', duration)
23785 cb99bacb9b4e branchcache: introduce revbranchcache for caching of revision branch names Mads Kiilerich <madski@unity3d.com> parents: 22357 diff changeset	382
41580 eb7ce452e0fb branchmap: updating triggers a write Martijn Pieters <mj@octobus.net> parents: 41579 diff changeset	383 self.write(repo)
eb7ce452e0fb branchmap: updating triggers a write Martijn Pieters <mj@octobus.net> parents: 41579 diff changeset	384
eb7ce452e0fb branchmap: updating triggers a write Martijn Pieters <mj@octobus.net> parents: 41579 diff changeset	385
eb7ce452e0fb branchmap: updating triggers a write Martijn Pieters <mj@octobus.net> parents: 41579 diff changeset	386 class remotebranchcache(branchcache):
eb7ce452e0fb branchmap: updating triggers a write Martijn Pieters <mj@octobus.net> parents: 41579 diff changeset	387 """Branchmap info for a remote connection, should not write locally"""
eb7ce452e0fb branchmap: updating triggers a write Martijn Pieters <mj@octobus.net> parents: 41579 diff changeset	388 def write(self, repo):
eb7ce452e0fb branchmap: updating triggers a write Martijn Pieters <mj@octobus.net> parents: 41579 diff changeset	389 pass
eb7ce452e0fb branchmap: updating triggers a write Martijn Pieters <mj@octobus.net> parents: 41579 diff changeset	390
eb7ce452e0fb branchmap: updating triggers a write Martijn Pieters <mj@octobus.net> parents: 41579 diff changeset	391
23785 cb99bacb9b4e branchcache: introduce revbranchcache for caching of revision branch names Mads Kiilerich <madski@unity3d.com> parents: 22357 diff changeset	392 # Revision branch info cache
cb99bacb9b4e branchcache: introduce revbranchcache for caching of revision branch names Mads Kiilerich <madski@unity3d.com> parents: 22357 diff changeset	393
cb99bacb9b4e branchcache: introduce revbranchcache for caching of revision branch names Mads Kiilerich <madski@unity3d.com> parents: 22357 diff changeset	394 _rbcversion = '-v1'
33535 755e6532e81d cachevfs: migration the revbranchcache to 'cachevfs' Boris Feld <boris.feld@octobus.net> parents: 33534 diff changeset	395 _rbcnames = 'rbc-names' + _rbcversion
755e6532e81d cachevfs: migration the revbranchcache to 'cachevfs' Boris Feld <boris.feld@octobus.net> parents: 33534 diff changeset	396 _rbcrevs = 'rbc-revs' + _rbcversion
23785 cb99bacb9b4e branchcache: introduce revbranchcache for caching of revision branch names Mads Kiilerich <madski@unity3d.com> parents: 22357 diff changeset	397 # [4 byte hash prefix][4 byte branch name number with sign bit indicating open]
cb99bacb9b4e branchcache: introduce revbranchcache for caching of revision branch names Mads Kiilerich <madski@unity3d.com> parents: 22357 diff changeset	398 _rbcrecfmt = '>4sI'
cb99bacb9b4e branchcache: introduce revbranchcache for caching of revision branch names Mads Kiilerich <madski@unity3d.com> parents: 22357 diff changeset	399 _rbcrecsize = calcsize(_rbcrecfmt)
cb99bacb9b4e branchcache: introduce revbranchcache for caching of revision branch names Mads Kiilerich <madski@unity3d.com> parents: 22357 diff changeset	400 _rbcnodelen = 4
cb99bacb9b4e branchcache: introduce revbranchcache for caching of revision branch names Mads Kiilerich <madski@unity3d.com> parents: 22357 diff changeset	401 _rbcbranchidxmask = 0x7fffffff
cb99bacb9b4e branchcache: introduce revbranchcache for caching of revision branch names Mads Kiilerich <madski@unity3d.com> parents: 22357 diff changeset	402 _rbccloseflag = 0x80000000
cb99bacb9b4e branchcache: introduce revbranchcache for caching of revision branch names Mads Kiilerich <madski@unity3d.com> parents: 22357 diff changeset	403
cb99bacb9b4e branchcache: introduce revbranchcache for caching of revision branch names Mads Kiilerich <madski@unity3d.com> parents: 22357 diff changeset	404 class revbranchcache(object):
cb99bacb9b4e branchcache: introduce revbranchcache for caching of revision branch names Mads Kiilerich <madski@unity3d.com> parents: 22357 diff changeset	405 """Persistent cache, mapping from revision number to branch name and close.
cb99bacb9b4e branchcache: introduce revbranchcache for caching of revision branch names Mads Kiilerich <madski@unity3d.com> parents: 22357 diff changeset	406 This is a low level cache, independent of filtering.
cb99bacb9b4e branchcache: introduce revbranchcache for caching of revision branch names Mads Kiilerich <madski@unity3d.com> parents: 22357 diff changeset	407
cb99bacb9b4e branchcache: introduce revbranchcache for caching of revision branch names Mads Kiilerich <madski@unity3d.com> parents: 22357 diff changeset	408 Branch names are stored in rbc-names in internal encoding separated by 0.
cb99bacb9b4e branchcache: introduce revbranchcache for caching of revision branch names Mads Kiilerich <madski@unity3d.com> parents: 22357 diff changeset	409 rbc-names is append-only, and each branch name is only stored once and will
cb99bacb9b4e branchcache: introduce revbranchcache for caching of revision branch names Mads Kiilerich <madski@unity3d.com> parents: 22357 diff changeset	410 thus have a unique index.
cb99bacb9b4e branchcache: introduce revbranchcache for caching of revision branch names Mads Kiilerich <madski@unity3d.com> parents: 22357 diff changeset	411
cb99bacb9b4e branchcache: introduce revbranchcache for caching of revision branch names Mads Kiilerich <madski@unity3d.com> parents: 22357 diff changeset	412 The branch info for each revision is stored in rbc-revs as constant size
cb99bacb9b4e branchcache: introduce revbranchcache for caching of revision branch names Mads Kiilerich <madski@unity3d.com> parents: 22357 diff changeset	413 records. The whole file is read into memory, but it is only 'parsed' on
cb99bacb9b4e branchcache: introduce revbranchcache for caching of revision branch names Mads Kiilerich <madski@unity3d.com> parents: 22357 diff changeset	414 demand. The file is usually append-only but will be truncated if repo
cb99bacb9b4e branchcache: introduce revbranchcache for caching of revision branch names Mads Kiilerich <madski@unity3d.com> parents: 22357 diff changeset	415 modification is detected.
cb99bacb9b4e branchcache: introduce revbranchcache for caching of revision branch names Mads Kiilerich <madski@unity3d.com> parents: 22357 diff changeset	416 The record for each revision contains the first 4 bytes of the
cb99bacb9b4e branchcache: introduce revbranchcache for caching of revision branch names Mads Kiilerich <madski@unity3d.com> parents: 22357 diff changeset	417 corresponding node hash, and the record is only used if it still matches.
cb99bacb9b4e branchcache: introduce revbranchcache for caching of revision branch names Mads Kiilerich <madski@unity3d.com> parents: 22357 diff changeset	418 Even a completely trashed rbc-revs fill thus still give the right result
cb99bacb9b4e branchcache: introduce revbranchcache for caching of revision branch names Mads Kiilerich <madski@unity3d.com> parents: 22357 diff changeset	419 while converging towards full recovery ... assuming no incorrectly matching
cb99bacb9b4e branchcache: introduce revbranchcache for caching of revision branch names Mads Kiilerich <madski@unity3d.com> parents: 22357 diff changeset	420 node hashes.
cb99bacb9b4e branchcache: introduce revbranchcache for caching of revision branch names Mads Kiilerich <madski@unity3d.com> parents: 22357 diff changeset	421 The record also contains 4 bytes where 31 bits contains the index of the
cb99bacb9b4e branchcache: introduce revbranchcache for caching of revision branch names Mads Kiilerich <madski@unity3d.com> parents: 22357 diff changeset	422 branch and the last bit indicate that it is a branch close commit.
cb99bacb9b4e branchcache: introduce revbranchcache for caching of revision branch names Mads Kiilerich <madski@unity3d.com> parents: 22357 diff changeset	423 The usage pattern for rbc-revs is thus somewhat similar to 00changelog.i
cb99bacb9b4e branchcache: introduce revbranchcache for caching of revision branch names Mads Kiilerich <madski@unity3d.com> parents: 22357 diff changeset	424 and will grow with it but be 1/8th of its size.
cb99bacb9b4e branchcache: introduce revbranchcache for caching of revision branch names Mads Kiilerich <madski@unity3d.com> parents: 22357 diff changeset	425 """
cb99bacb9b4e branchcache: introduce revbranchcache for caching of revision branch names Mads Kiilerich <madski@unity3d.com> parents: 22357 diff changeset	426
24159 5b4ed033390b revisionbranchcache: fall back to slow path if starting readonly (issue4531) Mads Kiilerich <madski@unity3d.com> parents: 23877 diff changeset	427 def __init__(self, repo, readonly=True):
23785 cb99bacb9b4e branchcache: introduce revbranchcache for caching of revision branch names Mads Kiilerich <madski@unity3d.com> parents: 22357 diff changeset	428 assert repo.filtername is None
24374 77fd1fb538cd revbranchcache: store repo on the object Durham Goode <durham@fb.com> parents: 24373 diff changeset	429 self._repo = repo
23785 cb99bacb9b4e branchcache: introduce revbranchcache for caching of revision branch names Mads Kiilerich <madski@unity3d.com> parents: 22357 diff changeset	430 self._names = [] # branch names in local encoding with static index
31355 2a18e9e6ca43 py3: use bytearray() instead of array('c', ...) constructions Augie Fackler <augie@google.com> parents: 30995 diff changeset	431 self._rbcrevs = bytearray()
29615 a2a380e2750f rbc: fix superfluous rebuilding from scratch - don't abuse self._rbcnamescount Mads Kiilerich <madski@unity3d.com> parents: 29604 diff changeset	432 self._rbcsnameslen = 0 # length of names read at _rbcsnameslen
23785 cb99bacb9b4e branchcache: introduce revbranchcache for caching of revision branch names Mads Kiilerich <madski@unity3d.com> parents: 22357 diff changeset	433 try:
33535 755e6532e81d cachevfs: migration the revbranchcache to 'cachevfs' Boris Feld <boris.feld@octobus.net> parents: 33534 diff changeset	434 bndata = repo.cachevfs.read(_rbcnames)
23785 cb99bacb9b4e branchcache: introduce revbranchcache for caching of revision branch names Mads Kiilerich <madski@unity3d.com> parents: 22357 diff changeset	435 self._rbcsnameslen = len(bndata) # for verification before writing
31380 7dd2f51f38ac rbc: empty (and invalid) rbc-names file should give an empty name list Mads Kiilerich <mads@kiilerich.com> parents: 31379 diff changeset	436 if bndata:
7dd2f51f38ac rbc: empty (and invalid) rbc-names file should give an empty name list Mads Kiilerich <mads@kiilerich.com> parents: 31379 diff changeset	437 self._names = [encoding.tolocal(bn)
7dd2f51f38ac rbc: empty (and invalid) rbc-names file should give an empty name list Mads Kiilerich <mads@kiilerich.com> parents: 31379 diff changeset	438 for bn in bndata.split('\0')]
29423 d2c6f3a948fa branchmap: remove unused exception variable Gregory Szorc <gregory.szorc@gmail.com> parents: 28558 diff changeset	439 except (IOError, OSError):
24159 5b4ed033390b revisionbranchcache: fall back to slow path if starting readonly (issue4531) Mads Kiilerich <madski@unity3d.com> parents: 23877 diff changeset	440 if readonly:
5b4ed033390b revisionbranchcache: fall back to slow path if starting readonly (issue4531) Mads Kiilerich <madski@unity3d.com> parents: 23877 diff changeset	441 # don't try to use cache - fall back to the slow path
5b4ed033390b revisionbranchcache: fall back to slow path if starting readonly (issue4531) Mads Kiilerich <madski@unity3d.com> parents: 23877 diff changeset	442 self.branchinfo = self._branchinfo
5b4ed033390b revisionbranchcache: fall back to slow path if starting readonly (issue4531) Mads Kiilerich <madski@unity3d.com> parents: 23877 diff changeset	443
23785 cb99bacb9b4e branchcache: introduce revbranchcache for caching of revision branch names Mads Kiilerich <madski@unity3d.com> parents: 22357 diff changeset	444 if self._names:
cb99bacb9b4e branchcache: introduce revbranchcache for caching of revision branch names Mads Kiilerich <madski@unity3d.com> parents: 22357 diff changeset	445 try:
33535 755e6532e81d cachevfs: migration the revbranchcache to 'cachevfs' Boris Feld <boris.feld@octobus.net> parents: 33534 diff changeset	446 data = repo.cachevfs.read(_rbcrevs)
31355 2a18e9e6ca43 py3: use bytearray() instead of array('c', ...) constructions Augie Fackler <augie@google.com> parents: 30995 diff changeset	447 self._rbcrevs[:] = data
25660 328739ea70c3 global: mass rewrite to use modern exception syntax Gregory Szorc <gregory.szorc@gmail.com> parents: 25266 diff changeset	448 except (IOError, OSError) as inst:
23785 cb99bacb9b4e branchcache: introduce revbranchcache for caching of revision branch names Mads Kiilerich <madski@unity3d.com> parents: 22357 diff changeset	449 repo.ui.debug("couldn't read revision branch cache: %s\n" %
37087 f0b6fbea00cf stringutil: bulk-replace call sites to point to new module Yuya Nishihara <yuya@tcha.org> parents: 36968 diff changeset	450 stringutil.forcebytestr(inst))
23785 cb99bacb9b4e branchcache: introduce revbranchcache for caching of revision branch names Mads Kiilerich <madski@unity3d.com> parents: 22357 diff changeset	451 # remember number of good records on disk
cb99bacb9b4e branchcache: introduce revbranchcache for caching of revision branch names Mads Kiilerich <madski@unity3d.com> parents: 22357 diff changeset	452 self._rbcrevslen = min(len(self._rbcrevs) // _rbcrecsize,
cb99bacb9b4e branchcache: introduce revbranchcache for caching of revision branch names Mads Kiilerich <madski@unity3d.com> parents: 22357 diff changeset	453 len(repo.changelog))
cb99bacb9b4e branchcache: introduce revbranchcache for caching of revision branch names Mads Kiilerich <madski@unity3d.com> parents: 22357 diff changeset	454 if self._rbcrevslen == 0:
cb99bacb9b4e branchcache: introduce revbranchcache for caching of revision branch names Mads Kiilerich <madski@unity3d.com> parents: 22357 diff changeset	455 self._names = []
29615 a2a380e2750f rbc: fix superfluous rebuilding from scratch - don't abuse self._rbcnamescount Mads Kiilerich <madski@unity3d.com> parents: 29604 diff changeset	456 self._rbcnamescount = len(self._names) # number of names read at
a2a380e2750f rbc: fix superfluous rebuilding from scratch - don't abuse self._rbcnamescount Mads Kiilerich <madski@unity3d.com> parents: 29604 diff changeset	457 # _rbcsnameslen
23785 cb99bacb9b4e branchcache: introduce revbranchcache for caching of revision branch names Mads Kiilerich <madski@unity3d.com> parents: 22357 diff changeset	458
28558 bcd106d456c4 cache: rebuild branch cache from scratch when inconsistencies are detected Mads Kiilerich <madski@unity3d.com> parents: 28557 diff changeset	459 def _clear(self):
bcd106d456c4 cache: rebuild branch cache from scratch when inconsistencies are detected Mads Kiilerich <madski@unity3d.com> parents: 28557 diff changeset	460 self._rbcsnameslen = 0
bcd106d456c4 cache: rebuild branch cache from scratch when inconsistencies are detected Mads Kiilerich <madski@unity3d.com> parents: 28557 diff changeset	461 del self._names[:]
bcd106d456c4 cache: rebuild branch cache from scratch when inconsistencies are detected Mads Kiilerich <madski@unity3d.com> parents: 28557 diff changeset	462 self._rbcnamescount = 0
bcd106d456c4 cache: rebuild branch cache from scratch when inconsistencies are detected Mads Kiilerich <madski@unity3d.com> parents: 28557 diff changeset	463 self._rbcrevslen = len(self._repo.changelog)
31355 2a18e9e6ca43 py3: use bytearray() instead of array('c', ...) constructions Augie Fackler <augie@google.com> parents: 30995 diff changeset	464 self._rbcrevs = bytearray(self._rbcrevslen * _rbcrecsize)
40749 50a64c321c1e branchmap: build the revbranchcache._namesreverse() only when required Pulkit Goyal <pulkit@yandex-team.ru> parents: 40494 diff changeset	465 util.clearcachedproperty(self, '_namesreverse')
50a64c321c1e branchmap: build the revbranchcache._namesreverse() only when required Pulkit Goyal <pulkit@yandex-team.ru> parents: 40494 diff changeset	466
50a64c321c1e branchmap: build the revbranchcache._namesreverse() only when required Pulkit Goyal <pulkit@yandex-team.ru> parents: 40494 diff changeset	467 @util.propertycache
50a64c321c1e branchmap: build the revbranchcache._namesreverse() only when required Pulkit Goyal <pulkit@yandex-team.ru> parents: 40494 diff changeset	468 def _namesreverse(self):
50a64c321c1e branchmap: build the revbranchcache._namesreverse() only when required Pulkit Goyal <pulkit@yandex-team.ru> parents: 40494 diff changeset	469 return dict((b, r) for r, b in enumerate(self._names))
28558 bcd106d456c4 cache: rebuild branch cache from scratch when inconsistencies are detected Mads Kiilerich <madski@unity3d.com> parents: 28557 diff changeset	470
40494 5e5c8f2a1eb5 branchmap: do not specify changelog as an argument Yuya Nishihara <yuya@tcha.org> parents: 40375 diff changeset	471 def branchinfo(self, rev):
23785 cb99bacb9b4e branchcache: introduce revbranchcache for caching of revision branch names Mads Kiilerich <madski@unity3d.com> parents: 22357 diff changeset	472 """Return branch name and close flag for rev, using and updating
cb99bacb9b4e branchcache: introduce revbranchcache for caching of revision branch names Mads Kiilerich <madski@unity3d.com> parents: 22357 diff changeset	473 persistent cache."""
40494 5e5c8f2a1eb5 branchmap: do not specify changelog as an argument Yuya Nishihara <yuya@tcha.org> parents: 40375 diff changeset	474 changelog = self._repo.changelog
23785 cb99bacb9b4e branchcache: introduce revbranchcache for caching of revision branch names Mads Kiilerich <madski@unity3d.com> parents: 22357 diff changeset	475 rbcrevidx = rev * _rbcrecsize
cb99bacb9b4e branchcache: introduce revbranchcache for caching of revision branch names Mads Kiilerich <madski@unity3d.com> parents: 22357 diff changeset	476
25266 38117278f295 revbranchcache: return uncached branchinfo for nullrev (issue4683) Yuya Nishihara <yuya@tcha.org> parents: 24728 diff changeset	477 # avoid negative index, changelog.read(nullrev) is fast without cache
38117278f295 revbranchcache: return uncached branchinfo for nullrev (issue4683) Yuya Nishihara <yuya@tcha.org> parents: 24728 diff changeset	478 if rev == nullrev:
38117278f295 revbranchcache: return uncached branchinfo for nullrev (issue4683) Yuya Nishihara <yuya@tcha.org> parents: 24728 diff changeset	479 return changelog.branchinfo(rev)
38117278f295 revbranchcache: return uncached branchinfo for nullrev (issue4683) Yuya Nishihara <yuya@tcha.org> parents: 24728 diff changeset	480
29604 db0095c83344 rbc: fix invalid rbc-revs entries caused by missing cache growth Mads Kiilerich <madski@unity3d.com> parents: 29423 diff changeset	481 # if requested rev isn't allocated, grow and cache the rev info
23785 cb99bacb9b4e branchcache: introduce revbranchcache for caching of revision branch names Mads Kiilerich <madski@unity3d.com> parents: 22357 diff changeset	482 if len(self._rbcrevs) < rbcrevidx + _rbcrecsize:
40494 5e5c8f2a1eb5 branchmap: do not specify changelog as an argument Yuya Nishihara <yuya@tcha.org> parents: 40375 diff changeset	483 return self._branchinfo(rev)
23785 cb99bacb9b4e branchcache: introduce revbranchcache for caching of revision branch names Mads Kiilerich <madski@unity3d.com> parents: 22357 diff changeset	484
cb99bacb9b4e branchcache: introduce revbranchcache for caching of revision branch names Mads Kiilerich <madski@unity3d.com> parents: 22357 diff changeset	485 # fast path: extract data from cache, use it if node is matching
cb99bacb9b4e branchcache: introduce revbranchcache for caching of revision branch names Mads Kiilerich <madski@unity3d.com> parents: 22357 diff changeset	486 reponode = changelog.node(rev)[:_rbcnodelen]
33661 1814ca418b30 branchmap: revert c34532365b38 for Python 2.7 compatibility Mike Hommey <mh@glandium.org> parents: 33535 diff changeset	487 cachenode, branchidx = unpack_from(
1814ca418b30 branchmap: revert c34532365b38 for Python 2.7 compatibility Mike Hommey <mh@glandium.org> parents: 33535 diff changeset	488 _rbcrecfmt, util.buffer(self._rbcrevs), rbcrevidx)
23785 cb99bacb9b4e branchcache: introduce revbranchcache for caching of revision branch names Mads Kiilerich <madski@unity3d.com> parents: 22357 diff changeset	489 close = bool(branchidx & _rbccloseflag)
cb99bacb9b4e branchcache: introduce revbranchcache for caching of revision branch names Mads Kiilerich <madski@unity3d.com> parents: 22357 diff changeset	490 if close:
cb99bacb9b4e branchcache: introduce revbranchcache for caching of revision branch names Mads Kiilerich <madski@unity3d.com> parents: 22357 diff changeset	491 branchidx &= _rbcbranchidxmask
24376 203a078da052 revbranchcache: populate cache incrementally Durham Goode <durham@fb.com> parents: 24375 diff changeset	492 if cachenode == '\0\0\0\0':
203a078da052 revbranchcache: populate cache incrementally Durham Goode <durham@fb.com> parents: 24375 diff changeset	493 pass
203a078da052 revbranchcache: populate cache incrementally Durham Goode <durham@fb.com> parents: 24375 diff changeset	494 elif cachenode == reponode:
29615 a2a380e2750f rbc: fix superfluous rebuilding from scratch - don't abuse self._rbcnamescount Mads Kiilerich <madski@unity3d.com> parents: 29604 diff changeset	495 try:
28558 bcd106d456c4 cache: rebuild branch cache from scratch when inconsistencies are detected Mads Kiilerich <madski@unity3d.com> parents: 28557 diff changeset	496 return self._names[branchidx], close
29615 a2a380e2750f rbc: fix superfluous rebuilding from scratch - don't abuse self._rbcnamescount Mads Kiilerich <madski@unity3d.com> parents: 29604 diff changeset	497 except IndexError:
a2a380e2750f rbc: fix superfluous rebuilding from scratch - don't abuse self._rbcnamescount Mads Kiilerich <madski@unity3d.com> parents: 29604 diff changeset	498 # recover from invalid reference to unknown branch
a2a380e2750f rbc: fix superfluous rebuilding from scratch - don't abuse self._rbcnamescount Mads Kiilerich <madski@unity3d.com> parents: 29604 diff changeset	499 self._repo.ui.debug("referenced branch names not found"
a2a380e2750f rbc: fix superfluous rebuilding from scratch - don't abuse self._rbcnamescount Mads Kiilerich <madski@unity3d.com> parents: 29604 diff changeset	500 " - rebuilding revision branch cache from scratch\n")
a2a380e2750f rbc: fix superfluous rebuilding from scratch - don't abuse self._rbcnamescount Mads Kiilerich <madski@unity3d.com> parents: 29604 diff changeset	501 self._clear()
24376 203a078da052 revbranchcache: populate cache incrementally Durham Goode <durham@fb.com> parents: 24375 diff changeset	502 else:
203a078da052 revbranchcache: populate cache incrementally Durham Goode <durham@fb.com> parents: 24375 diff changeset	503 # rev/node map has changed, invalidate the cache from here up
29615 a2a380e2750f rbc: fix superfluous rebuilding from scratch - don't abuse self._rbcnamescount Mads Kiilerich <madski@unity3d.com> parents: 29604 diff changeset	504 self._repo.ui.debug("history modification detected - truncating "
31504 a369482e9649 branchmap: be more careful about using %d on ints Augie Fackler <augie@google.com> parents: 31463 diff changeset	505 "revision branch cache to revision %d\n" % rev)
24376 203a078da052 revbranchcache: populate cache incrementally Durham Goode <durham@fb.com> parents: 24375 diff changeset	506 truncate = rbcrevidx + _rbcrecsize
203a078da052 revbranchcache: populate cache incrementally Durham Goode <durham@fb.com> parents: 24375 diff changeset	507 del self._rbcrevs[truncate:]
203a078da052 revbranchcache: populate cache incrementally Durham Goode <durham@fb.com> parents: 24375 diff changeset	508 self._rbcrevslen = min(self._rbcrevslen, truncate)
203a078da052 revbranchcache: populate cache incrementally Durham Goode <durham@fb.com> parents: 24375 diff changeset	509
23785 cb99bacb9b4e branchcache: introduce revbranchcache for caching of revision branch names Mads Kiilerich <madski@unity3d.com> parents: 22357 diff changeset	510 # fall back to slow path and make sure it will be written to disk
40494 5e5c8f2a1eb5 branchmap: do not specify changelog as an argument Yuya Nishihara <yuya@tcha.org> parents: 40375 diff changeset	511 return self._branchinfo(rev)
23785 cb99bacb9b4e branchcache: introduce revbranchcache for caching of revision branch names Mads Kiilerich <madski@unity3d.com> parents: 22357 diff changeset	512
40494 5e5c8f2a1eb5 branchmap: do not specify changelog as an argument Yuya Nishihara <yuya@tcha.org> parents: 40375 diff changeset	513 def _branchinfo(self, rev):
23785 cb99bacb9b4e branchcache: introduce revbranchcache for caching of revision branch names Mads Kiilerich <madski@unity3d.com> parents: 22357 diff changeset	514 """Retrieve branch info from changelog and update _rbcrevs"""
40494 5e5c8f2a1eb5 branchmap: do not specify changelog as an argument Yuya Nishihara <yuya@tcha.org> parents: 40375 diff changeset	515 changelog = self._repo.changelog
23785 cb99bacb9b4e branchcache: introduce revbranchcache for caching of revision branch names Mads Kiilerich <madski@unity3d.com> parents: 22357 diff changeset	516 b, close = changelog.branchinfo(rev)
cb99bacb9b4e branchcache: introduce revbranchcache for caching of revision branch names Mads Kiilerich <madski@unity3d.com> parents: 22357 diff changeset	517 if b in self._namesreverse:
cb99bacb9b4e branchcache: introduce revbranchcache for caching of revision branch names Mads Kiilerich <madski@unity3d.com> parents: 22357 diff changeset	518 branchidx = self._namesreverse[b]
cb99bacb9b4e branchcache: introduce revbranchcache for caching of revision branch names Mads Kiilerich <madski@unity3d.com> parents: 22357 diff changeset	519 else:
cb99bacb9b4e branchcache: introduce revbranchcache for caching of revision branch names Mads Kiilerich <madski@unity3d.com> parents: 22357 diff changeset	520 branchidx = len(self._names)
cb99bacb9b4e branchcache: introduce revbranchcache for caching of revision branch names Mads Kiilerich <madski@unity3d.com> parents: 22357 diff changeset	521 self._names.append(b)
cb99bacb9b4e branchcache: introduce revbranchcache for caching of revision branch names Mads Kiilerich <madski@unity3d.com> parents: 22357 diff changeset	522 self._namesreverse[b] = branchidx
cb99bacb9b4e branchcache: introduce revbranchcache for caching of revision branch names Mads Kiilerich <madski@unity3d.com> parents: 22357 diff changeset	523 reponode = changelog.node(rev)
cb99bacb9b4e branchcache: introduce revbranchcache for caching of revision branch names Mads Kiilerich <madski@unity3d.com> parents: 22357 diff changeset	524 if close:
cb99bacb9b4e branchcache: introduce revbranchcache for caching of revision branch names Mads Kiilerich <madski@unity3d.com> parents: 22357 diff changeset	525 branchidx \|= _rbccloseflag
40494 5e5c8f2a1eb5 branchmap: do not specify changelog as an argument Yuya Nishihara <yuya@tcha.org> parents: 40375 diff changeset	526 self._setcachedata(rev, reponode, branchidx)
24375 fe255b2525d5 revbranchcache: move entry writing to a separate function Durham Goode <durham@fb.com> parents: 24374 diff changeset	527 return b, close
fe255b2525d5 revbranchcache: move entry writing to a separate function Durham Goode <durham@fb.com> parents: 24374 diff changeset	528
36968 95f4f1bfb650 revbranchcache: add a public function to update the data Boris Feld <boris.feld@octobus.net> parents: 36504 diff changeset	529 def setdata(self, branch, rev, node, close):
95f4f1bfb650 revbranchcache: add a public function to update the data Boris Feld <boris.feld@octobus.net> parents: 36504 diff changeset	530 """add new data information to the cache"""
95f4f1bfb650 revbranchcache: add a public function to update the data Boris Feld <boris.feld@octobus.net> parents: 36504 diff changeset	531 if branch in self._namesreverse:
95f4f1bfb650 revbranchcache: add a public function to update the data Boris Feld <boris.feld@octobus.net> parents: 36504 diff changeset	532 branchidx = self._namesreverse[branch]
95f4f1bfb650 revbranchcache: add a public function to update the data Boris Feld <boris.feld@octobus.net> parents: 36504 diff changeset	533 else:
95f4f1bfb650 revbranchcache: add a public function to update the data Boris Feld <boris.feld@octobus.net> parents: 36504 diff changeset	534 branchidx = len(self._names)
95f4f1bfb650 revbranchcache: add a public function to update the data Boris Feld <boris.feld@octobus.net> parents: 36504 diff changeset	535 self._names.append(branch)
95f4f1bfb650 revbranchcache: add a public function to update the data Boris Feld <boris.feld@octobus.net> parents: 36504 diff changeset	536 self._namesreverse[branch] = branchidx
95f4f1bfb650 revbranchcache: add a public function to update the data Boris Feld <boris.feld@octobus.net> parents: 36504 diff changeset	537 if close:
95f4f1bfb650 revbranchcache: add a public function to update the data Boris Feld <boris.feld@octobus.net> parents: 36504 diff changeset	538 branchidx \|= _rbccloseflag
95f4f1bfb650 revbranchcache: add a public function to update the data Boris Feld <boris.feld@octobus.net> parents: 36504 diff changeset	539 self._setcachedata(rev, node, branchidx)
95f4f1bfb650 revbranchcache: add a public function to update the data Boris Feld <boris.feld@octobus.net> parents: 36504 diff changeset	540 # If no cache data were readable (non exists, bad permission, etc)
95f4f1bfb650 revbranchcache: add a public function to update the data Boris Feld <boris.feld@octobus.net> parents: 36504 diff changeset	541 # the cache was bypassing itself by setting:
95f4f1bfb650 revbranchcache: add a public function to update the data Boris Feld <boris.feld@octobus.net> parents: 36504 diff changeset	542 #
95f4f1bfb650 revbranchcache: add a public function to update the data Boris Feld <boris.feld@octobus.net> parents: 36504 diff changeset	543 # self.branchinfo = self._branchinfo
95f4f1bfb650 revbranchcache: add a public function to update the data Boris Feld <boris.feld@octobus.net> parents: 36504 diff changeset	544 #
95f4f1bfb650 revbranchcache: add a public function to update the data Boris Feld <boris.feld@octobus.net> parents: 36504 diff changeset	545 # Since we now have data in the cache, we need to drop this bypassing.
40232 d99a588d8515 py3: pass in system string to vars(branchmap).__contains__() Yuya Nishihara <yuya@tcha.org> parents: 39175 diff changeset	546 if r'branchinfo' in vars(self):
36968 95f4f1bfb650 revbranchcache: add a public function to update the data Boris Feld <boris.feld@octobus.net> parents: 36504 diff changeset	547 del self.branchinfo
95f4f1bfb650 revbranchcache: add a public function to update the data Boris Feld <boris.feld@octobus.net> parents: 36504 diff changeset	548
40494 5e5c8f2a1eb5 branchmap: do not specify changelog as an argument Yuya Nishihara <yuya@tcha.org> parents: 40375 diff changeset	549 def _setcachedata(self, rev, node, branchidx):
24375 fe255b2525d5 revbranchcache: move entry writing to a separate function Durham Goode <durham@fb.com> parents: 24374 diff changeset	550 """Writes the node's branch data to the in-memory cache data."""
31463 a5bad127128d branchmap: handle nullrev in setcachedata Durham Goode <durham@fb.com> parents: 31390 diff changeset	551 if rev == nullrev:
a5bad127128d branchmap: handle nullrev in setcachedata Durham Goode <durham@fb.com> parents: 31390 diff changeset	552 return
23785 cb99bacb9b4e branchcache: introduce revbranchcache for caching of revision branch names Mads Kiilerich <madski@unity3d.com> parents: 22357 diff changeset	553 rbcrevidx = rev * _rbcrecsize
29604 db0095c83344 rbc: fix invalid rbc-revs entries caused by missing cache growth Mads Kiilerich <madski@unity3d.com> parents: 29423 diff changeset	554 if len(self._rbcrevs) < rbcrevidx + _rbcrecsize:
db0095c83344 rbc: fix invalid rbc-revs entries caused by missing cache growth Mads Kiilerich <madski@unity3d.com> parents: 29423 diff changeset	555 self._rbcrevs.extend('\0' *
40494 5e5c8f2a1eb5 branchmap: do not specify changelog as an argument Yuya Nishihara <yuya@tcha.org> parents: 40375 diff changeset	556 (len(self._repo.changelog) * _rbcrecsize -
29604 db0095c83344 rbc: fix invalid rbc-revs entries caused by missing cache growth Mads Kiilerich <madski@unity3d.com> parents: 29423 diff changeset	557 len(self._rbcrevs)))
31379 906be86990c4 rbc: use struct unpack_from and pack_into instead of unpack and pack Mads Kiilerich <madski@unity3d.com> parents: 31369 diff changeset	558 pack_into(_rbcrecfmt, self._rbcrevs, rbcrevidx, node, branchidx)
24376 203a078da052 revbranchcache: populate cache incrementally Durham Goode <durham@fb.com> parents: 24375 diff changeset	559 self._rbcrevslen = min(self._rbcrevslen, rev)
23785 cb99bacb9b4e branchcache: introduce revbranchcache for caching of revision branch names Mads Kiilerich <madski@unity3d.com> parents: 22357 diff changeset	560
24377 656f93ce66d5 revbranchcache: move cache writing to the transaction finalizer Durham Goode <durham@fb.com> parents: 24376 diff changeset	561 tr = self._repo.currenttransaction()
656f93ce66d5 revbranchcache: move cache writing to the transaction finalizer Durham Goode <durham@fb.com> parents: 24376 diff changeset	562 if tr:
656f93ce66d5 revbranchcache: move cache writing to the transaction finalizer Durham Goode <durham@fb.com> parents: 24376 diff changeset	563 tr.addfinalize('write-revbranchcache', self.write)
656f93ce66d5 revbranchcache: move cache writing to the transaction finalizer Durham Goode <durham@fb.com> parents: 24376 diff changeset	564
656f93ce66d5 revbranchcache: move cache writing to the transaction finalizer Durham Goode <durham@fb.com> parents: 24376 diff changeset	565 def write(self, tr=None):
23785 cb99bacb9b4e branchcache: introduce revbranchcache for caching of revision branch names Mads Kiilerich <madski@unity3d.com> parents: 22357 diff changeset	566 """Save branch cache if it is dirty."""
24374 77fd1fb538cd revbranchcache: store repo on the object Durham Goode <durham@fb.com> parents: 24373 diff changeset	567 repo = self._repo
29756 0d588332ad2c branchmap: acquires lock before writting the rev branch cache Pierre-Yves David <pierre-yves.david@ens-lyon.org> parents: 29755 diff changeset	568 wlock = None
29757 3b184adfb5be branchmap: simplify error handlind when writing rev branch cache Pierre-Yves David <pierre-yves.david@ens-lyon.org> parents: 29756 diff changeset	569 step = ''
29756 0d588332ad2c branchmap: acquires lock before writting the rev branch cache Pierre-Yves David <pierre-yves.david@ens-lyon.org> parents: 29755 diff changeset	570 try:
29755 9f3c49ee4486 branchmap: preparatory indent of indent the branch rev writing code Pierre-Yves David <pierre-yves.david@ens-lyon.org> parents: 29615 diff changeset	571 if self._rbcnamescount < len(self._names):
29757 3b184adfb5be branchmap: simplify error handlind when writing rev branch cache Pierre-Yves David <pierre-yves.david@ens-lyon.org> parents: 29756 diff changeset	572 step = ' names'
29756 0d588332ad2c branchmap: acquires lock before writting the rev branch cache Pierre-Yves David <pierre-yves.david@ens-lyon.org> parents: 29755 diff changeset	573 wlock = repo.wlock(wait=False)
29758 3dbc95f3eb31 branchmap: remove extra indent Pierre-Yves David <pierre-yves.david@ens-lyon.org> parents: 29757 diff changeset	574 if self._rbcnamescount != 0:
33535 755e6532e81d cachevfs: migration the revbranchcache to 'cachevfs' Boris Feld <boris.feld@octobus.net> parents: 33534 diff changeset	575 f = repo.cachevfs.open(_rbcnames, 'ab')
29758 3dbc95f3eb31 branchmap: remove extra indent Pierre-Yves David <pierre-yves.david@ens-lyon.org> parents: 29757 diff changeset	576 if f.tell() == self._rbcsnameslen:
3dbc95f3eb31 branchmap: remove extra indent Pierre-Yves David <pierre-yves.david@ens-lyon.org> parents: 29757 diff changeset	577 f.write('\0')
3dbc95f3eb31 branchmap: remove extra indent Pierre-Yves David <pierre-yves.david@ens-lyon.org> parents: 29757 diff changeset	578 else:
3dbc95f3eb31 branchmap: remove extra indent Pierre-Yves David <pierre-yves.david@ens-lyon.org> parents: 29757 diff changeset	579 f.close()
3dbc95f3eb31 branchmap: remove extra indent Pierre-Yves David <pierre-yves.david@ens-lyon.org> parents: 29757 diff changeset	580 repo.ui.debug("%s changed - rewriting it\n" % _rbcnames)
3dbc95f3eb31 branchmap: remove extra indent Pierre-Yves David <pierre-yves.david@ens-lyon.org> parents: 29757 diff changeset	581 self._rbcnamescount = 0
3dbc95f3eb31 branchmap: remove extra indent Pierre-Yves David <pierre-yves.david@ens-lyon.org> parents: 29757 diff changeset	582 self._rbcrevslen = 0
3dbc95f3eb31 branchmap: remove extra indent Pierre-Yves David <pierre-yves.david@ens-lyon.org> parents: 29757 diff changeset	583 if self._rbcnamescount == 0:
3dbc95f3eb31 branchmap: remove extra indent Pierre-Yves David <pierre-yves.david@ens-lyon.org> parents: 29757 diff changeset	584 # before rewriting names, make sure references are removed
33535 755e6532e81d cachevfs: migration the revbranchcache to 'cachevfs' Boris Feld <boris.feld@octobus.net> parents: 33534 diff changeset	585 repo.cachevfs.unlinkpath(_rbcrevs, ignoremissing=True)
755e6532e81d cachevfs: migration the revbranchcache to 'cachevfs' Boris Feld <boris.feld@octobus.net> parents: 33534 diff changeset	586 f = repo.cachevfs.open(_rbcnames, 'wb')
29758 3dbc95f3eb31 branchmap: remove extra indent Pierre-Yves David <pierre-yves.david@ens-lyon.org> parents: 29757 diff changeset	587 f.write('\0'.join(encoding.fromlocal(b)
3dbc95f3eb31 branchmap: remove extra indent Pierre-Yves David <pierre-yves.david@ens-lyon.org> parents: 29757 diff changeset	588 for b in self._names[self._rbcnamescount:]))
3dbc95f3eb31 branchmap: remove extra indent Pierre-Yves David <pierre-yves.david@ens-lyon.org> parents: 29757 diff changeset	589 self._rbcsnameslen = f.tell()
3dbc95f3eb31 branchmap: remove extra indent Pierre-Yves David <pierre-yves.david@ens-lyon.org> parents: 29757 diff changeset	590 f.close()
29755 9f3c49ee4486 branchmap: preparatory indent of indent the branch rev writing code Pierre-Yves David <pierre-yves.david@ens-lyon.org> parents: 29615 diff changeset	591 self._rbcnamescount = len(self._names)
23785 cb99bacb9b4e branchcache: introduce revbranchcache for caching of revision branch names Mads Kiilerich <madski@unity3d.com> parents: 22357 diff changeset	592
29755 9f3c49ee4486 branchmap: preparatory indent of indent the branch rev writing code Pierre-Yves David <pierre-yves.david@ens-lyon.org> parents: 29615 diff changeset	593 start = self._rbcrevslen * _rbcrecsize
9f3c49ee4486 branchmap: preparatory indent of indent the branch rev writing code Pierre-Yves David <pierre-yves.david@ens-lyon.org> parents: 29615 diff changeset	594 if start != len(self._rbcrevs):
29757 3b184adfb5be branchmap: simplify error handlind when writing rev branch cache Pierre-Yves David <pierre-yves.david@ens-lyon.org> parents: 29756 diff changeset	595 step = ''
29756 0d588332ad2c branchmap: acquires lock before writting the rev branch cache Pierre-Yves David <pierre-yves.david@ens-lyon.org> parents: 29755 diff changeset	596 if wlock is None:
0d588332ad2c branchmap: acquires lock before writting the rev branch cache Pierre-Yves David <pierre-yves.david@ens-lyon.org> parents: 29755 diff changeset	597 wlock = repo.wlock(wait=False)
29755 9f3c49ee4486 branchmap: preparatory indent of indent the branch rev writing code Pierre-Yves David <pierre-yves.david@ens-lyon.org> parents: 29615 diff changeset	598 revs = min(len(repo.changelog),
9f3c49ee4486 branchmap: preparatory indent of indent the branch rev writing code Pierre-Yves David <pierre-yves.david@ens-lyon.org> parents: 29615 diff changeset	599 len(self._rbcrevs) // _rbcrecsize)
33535 755e6532e81d cachevfs: migration the revbranchcache to 'cachevfs' Boris Feld <boris.feld@octobus.net> parents: 33534 diff changeset	600 f = repo.cachevfs.open(_rbcrevs, 'ab')
29758 3dbc95f3eb31 branchmap: remove extra indent Pierre-Yves David <pierre-yves.david@ens-lyon.org> parents: 29757 diff changeset	601 if f.tell() != start:
33535 755e6532e81d cachevfs: migration the revbranchcache to 'cachevfs' Boris Feld <boris.feld@octobus.net> parents: 33534 diff changeset	602 repo.ui.debug("truncating cache/%s to %d\n"
755e6532e81d cachevfs: migration the revbranchcache to 'cachevfs' Boris Feld <boris.feld@octobus.net> parents: 33534 diff changeset	603 % (_rbcrevs, start))
29758 3dbc95f3eb31 branchmap: remove extra indent Pierre-Yves David <pierre-yves.david@ens-lyon.org> parents: 29757 diff changeset	604 f.seek(start)
28557 4b83507bfc91 cache: safer handling of failing seek when writing revision branch cache Mads Kiilerich <madski@unity3d.com> parents: 28556 diff changeset	605 if f.tell() != start:
29758 3dbc95f3eb31 branchmap: remove extra indent Pierre-Yves David <pierre-yves.david@ens-lyon.org> parents: 29757 diff changeset	606 start = 0
28557 4b83507bfc91 cache: safer handling of failing seek when writing revision branch cache Mads Kiilerich <madski@unity3d.com> parents: 28556 diff changeset	607 f.seek(start)
29758 3dbc95f3eb31 branchmap: remove extra indent Pierre-Yves David <pierre-yves.david@ens-lyon.org> parents: 29757 diff changeset	608 f.truncate()
3dbc95f3eb31 branchmap: remove extra indent Pierre-Yves David <pierre-yves.david@ens-lyon.org> parents: 29757 diff changeset	609 end = revs * _rbcrecsize
3dbc95f3eb31 branchmap: remove extra indent Pierre-Yves David <pierre-yves.david@ens-lyon.org> parents: 29757 diff changeset	610 f.write(self._rbcrevs[start:end])
3dbc95f3eb31 branchmap: remove extra indent Pierre-Yves David <pierre-yves.david@ens-lyon.org> parents: 29757 diff changeset	611 f.close()
29755 9f3c49ee4486 branchmap: preparatory indent of indent the branch rev writing code Pierre-Yves David <pierre-yves.david@ens-lyon.org> parents: 29615 diff changeset	612 self._rbcrevslen = revs
29757 3b184adfb5be branchmap: simplify error handlind when writing rev branch cache Pierre-Yves David <pierre-yves.david@ens-lyon.org> parents: 29756 diff changeset	613 except (IOError, OSError, error.Abort, error.LockError) as inst:
3b184adfb5be branchmap: simplify error handlind when writing rev branch cache Pierre-Yves David <pierre-yves.david@ens-lyon.org> parents: 29756 diff changeset	614 repo.ui.debug("couldn't write revision branch cache%s: %s\n"
37087 f0b6fbea00cf stringutil: bulk-replace call sites to point to new module Yuya Nishihara <yuya@tcha.org> parents: 36968 diff changeset	615 % (step, stringutil.forcebytestr(inst)))
29756 0d588332ad2c branchmap: acquires lock before writting the rev branch cache Pierre-Yves David <pierre-yves.david@ens-lyon.org> parents: 29755 diff changeset	616 finally:
0d588332ad2c branchmap: acquires lock before writting the rev branch cache Pierre-Yves David <pierre-yves.david@ens-lyon.org> parents: 29755 diff changeset	617 if wlock is not None:
0d588332ad2c branchmap: acquires lock before writting the rev branch cache Pierre-Yves David <pierre-yves.david@ens-lyon.org> parents: 29755 diff changeset	618 wlock.release()

Mercurial > public > mercurial-scm > hg-stable

annotate mercurial/branchmap.py @ 42001:624d6683c705