Mercurial > public > mercurial-scm > hg-stable

annotate mercurial/branchmap.py @ 42006:111de135fc76

Find changesets by keywords (author, files, the commit message), revision number or hash, or revset expression.
branchcache: add attributes to track which nodes are verified Half of the cost of loading branchcache comes from verifiying all the nodes it has. We don't need to verify all the nodes in all the cases. Sometimes we need to verify only a set of nodes for a set of branches. We can ignore nodes of other branches as we are not going to read them. This patch introduces two attributes to branchcache class. _verifiedbranches is a set which will tell the branches for which it's head nodes are verified. _closedverified is a boolean which will tell whether all closednodes are verified or not. Another idea which I had was to keep a set of nodes which are verified, but it will be more ugly and difficult to track things on node level. Differential Revision: https://phab.mercurial-scm.org/D6156
author Pulkit Goyal <pulkit@yandex-team.ru>
date Tue, 19 Mar 2019 16:20:02 +0300
parents b137a6793c51
children b5511845f9d5
Ignore whitespace changes - Everywhere: Within whitespace: At end of lines:
rev   line source
18116
bcee63733aad branchmap: create a mercurial.branchmap module
Pierre-Yves David <pierre-yves.david@ens-lyon.org>
parents:
diff changeset
1 # branchmap.py - logic to computes, maintain and stores branchmap for local repo
bcee63733aad branchmap: create a mercurial.branchmap module
Pierre-Yves David <pierre-yves.david@ens-lyon.org>
parents:
diff changeset
2 #
bcee63733aad branchmap: create a mercurial.branchmap module
Pierre-Yves David <pierre-yves.david@ens-lyon.org>
parents:
diff changeset
3 # Copyright 2005-2007 Matt Mackall <mpm@selenic.com>
bcee63733aad branchmap: create a mercurial.branchmap module
Pierre-Yves David <pierre-yves.david@ens-lyon.org>
parents:
diff changeset
4 #
bcee63733aad branchmap: create a mercurial.branchmap module
Pierre-Yves David <pierre-yves.david@ens-lyon.org>
parents:
diff changeset
5 # This software may be used and distributed according to the terms of the
bcee63733aad branchmap: create a mercurial.branchmap module
Pierre-Yves David <pierre-yves.david@ens-lyon.org>
parents:
diff changeset
6 # GNU General Public License version 2 or any later version.
18117
526e7ec5c96e branchmap: extract write logic from localrepo
Pierre-Yves David <pierre-yves.david@ens-lyon.org>
parents: 18116
diff changeset
7
25918
47f36e050c2e branchmap: use absolute_import
Gregory Szorc <gregory.szorc@gmail.com>
parents: 25660
diff changeset
8 from __future__ import absolute_import
47f36e050c2e branchmap: use absolute_import
Gregory Szorc <gregory.szorc@gmail.com>
parents: 25660
diff changeset
9
47f36e050c2e branchmap: use absolute_import
Gregory Szorc <gregory.szorc@gmail.com>
parents: 25660
diff changeset
10 import struct
47f36e050c2e branchmap: use absolute_import
Gregory Szorc <gregory.szorc@gmail.com>
parents: 25660
diff changeset
11
47f36e050c2e branchmap: use absolute_import
Gregory Szorc <gregory.szorc@gmail.com>
parents: 25660
diff changeset
12 from .node import (
47f36e050c2e branchmap: use absolute_import
Gregory Szorc <gregory.szorc@gmail.com>
parents: 25660
diff changeset
13 bin,
47f36e050c2e branchmap: use absolute_import
Gregory Szorc <gregory.szorc@gmail.com>
parents: 25660
diff changeset
14 hex,
47f36e050c2e branchmap: use absolute_import
Gregory Szorc <gregory.szorc@gmail.com>
parents: 25660
diff changeset
15 nullid,
47f36e050c2e branchmap: use absolute_import
Gregory Szorc <gregory.szorc@gmail.com>
parents: 25660
diff changeset
16 nullrev,
47f36e050c2e branchmap: use absolute_import
Gregory Szorc <gregory.szorc@gmail.com>
parents: 25660
diff changeset
17 )
47f36e050c2e branchmap: use absolute_import
Gregory Szorc <gregory.szorc@gmail.com>
parents: 25660
diff changeset
18 from . import (
47f36e050c2e branchmap: use absolute_import
Gregory Szorc <gregory.szorc@gmail.com>
parents: 25660
diff changeset
19 encoding,
26587
56b2bcea2529 error: get Abort from 'error' instead of 'util'
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 26460
diff changeset
20 error,
35871
a011e4140435 branchmap: make error messages consistent between Python 2 and 3
Augie Fackler <augie@google.com>
parents: 35499
diff changeset
21 pycompat,
25918
47f36e050c2e branchmap: use absolute_import
Gregory Szorc <gregory.szorc@gmail.com>
parents: 25660
diff changeset
22 scmutil,
30995
22fbca1d11ed mercurial: switch to util.timer for all interval timings
Simon Farnsworth <simonfar@fb.com>
parents: 29758
diff changeset
23 util,
25918
47f36e050c2e branchmap: use absolute_import
Gregory Szorc <gregory.szorc@gmail.com>
parents: 25660
diff changeset
24 )
37087
f0b6fbea00cf stringutil: bulk-replace call sites to point to new module
Yuya Nishihara <yuya@tcha.org>
parents: 36968
diff changeset
25 from .utils import (
f0b6fbea00cf stringutil: bulk-replace call sites to point to new module
Yuya Nishihara <yuya@tcha.org>
parents: 36968
diff changeset
26 stringutil,
f0b6fbea00cf stringutil: bulk-replace call sites to point to new module
Yuya Nishihara <yuya@tcha.org>
parents: 36968
diff changeset
27 )
25918
47f36e050c2e branchmap: use absolute_import
Gregory Szorc <gregory.szorc@gmail.com>
parents: 25660
diff changeset
28
47f36e050c2e branchmap: use absolute_import
Gregory Szorc <gregory.szorc@gmail.com>
parents: 25660
diff changeset
29 calcsize = struct.calcsize
31379
906be86990c4 rbc: use struct unpack_from and pack_into instead of unpack and pack
Mads Kiilerich <madski@unity3d.com>
parents: 31369
diff changeset
30 pack_into = struct.pack_into
906be86990c4 rbc: use struct unpack_from and pack_into instead of unpack and pack
Mads Kiilerich <madski@unity3d.com>
parents: 31369
diff changeset
31 unpack_from = struct.unpack_from
18117
526e7ec5c96e branchmap: extract write logic from localrepo
Pierre-Yves David <pierre-yves.david@ens-lyon.org>
parents: 18116
diff changeset
32
18118
e70ff1e599f4 branchmap: extract read logic from repo
Pierre-Yves David <pierre-yves.david@logilab.fr>
parents: 18117
diff changeset
33
20032
175c6fd8cacc subsettable: move from repoview to branchmap, the only place it's used
Augie Fackler <raf@durin42.com>
parents: 19839
diff changeset
34 ### Nearest subset relation
175c6fd8cacc subsettable: move from repoview to branchmap, the only place it's used
Augie Fackler <raf@durin42.com>
parents: 19839
diff changeset
35 # Nearest subset of filter X is a filter Y so that:
175c6fd8cacc subsettable: move from repoview to branchmap, the only place it's used
Augie Fackler <raf@durin42.com>
parents: 19839
diff changeset
36 # * Y is included in X,
175c6fd8cacc subsettable: move from repoview to branchmap, the only place it's used
Augie Fackler <raf@durin42.com>
parents: 19839
diff changeset
37 # * X - Y is as small as possible.
175c6fd8cacc subsettable: move from repoview to branchmap, the only place it's used
Augie Fackler <raf@durin42.com>
parents: 19839
diff changeset
38 # This create and ordering used for branchmap purpose.
175c6fd8cacc subsettable: move from repoview to branchmap, the only place it's used
Augie Fackler <raf@durin42.com>
parents: 19839
diff changeset
39 # the ordering may be partial
175c6fd8cacc subsettable: move from repoview to branchmap, the only place it's used
Augie Fackler <raf@durin42.com>
parents: 19839
diff changeset
40 subsettable = {None: 'visible',
35499
07fdac1d5c66 repoview: add a new filtername for accessing hidden commits
Pulkit Goyal <7895pulkit@gmail.com>
parents: 34371
diff changeset
41 'visible-hidden': 'visible',
20032
175c6fd8cacc subsettable: move from repoview to branchmap, the only place it's used
Augie Fackler <raf@durin42.com>
parents: 19839
diff changeset
42 'visible': 'served',
175c6fd8cacc subsettable: move from repoview to branchmap, the only place it's used
Augie Fackler <raf@durin42.com>
parents: 19839
diff changeset
43 'served': 'immutable',
175c6fd8cacc subsettable: move from repoview to branchmap, the only place it's used
Augie Fackler <raf@durin42.com>
parents: 19839
diff changeset
44 'immutable': 'base'}
175c6fd8cacc subsettable: move from repoview to branchmap, the only place it's used
Augie Fackler <raf@durin42.com>
parents: 19839
diff changeset
45
41626
328ca3b9e545 branchmap: encapsulate cache updating in the map itself
Martijn Pieters <mj@octobus.net>
parents: 41581
diff changeset
46
328ca3b9e545 branchmap: encapsulate cache updating in the map itself
Martijn Pieters <mj@octobus.net>
parents: 41581
diff changeset
47 class BranchMapCache(object):
41729
a87ca1d7e61d branchmap: improve doc about BranchMapCache class
Pulkit Goyal <pulkit@yandex-team.ru>
parents: 41689
diff changeset
48 """mapping of filtered views of repo with their branchcache"""
41626
328ca3b9e545 branchmap: encapsulate cache updating in the map itself
Martijn Pieters <mj@octobus.net>
parents: 41581
diff changeset
49 def __init__(self):
328ca3b9e545 branchmap: encapsulate cache updating in the map itself
Martijn Pieters <mj@octobus.net>
parents: 41581
diff changeset
50 self._per_filter = {}
41581
c795c462b1d6 branchmap: add some clarifications and clean up flow
Martijn Pieters <mj@octobus.net>
parents: 41580
diff changeset
51
41626
328ca3b9e545 branchmap: encapsulate cache updating in the map itself
Martijn Pieters <mj@octobus.net>
parents: 41581
diff changeset
52 def __getitem__(self, repo):
328ca3b9e545 branchmap: encapsulate cache updating in the map itself
Martijn Pieters <mj@octobus.net>
parents: 41581
diff changeset
53 self.updatecache(repo)
328ca3b9e545 branchmap: encapsulate cache updating in the map itself
Martijn Pieters <mj@octobus.net>
parents: 41581
diff changeset
54 return self._per_filter[repo.filtername]
328ca3b9e545 branchmap: encapsulate cache updating in the map itself
Martijn Pieters <mj@octobus.net>
parents: 41581
diff changeset
55
328ca3b9e545 branchmap: encapsulate cache updating in the map itself
Martijn Pieters <mj@octobus.net>
parents: 41581
diff changeset
56 def updatecache(self, repo):
328ca3b9e545 branchmap: encapsulate cache updating in the map itself
Martijn Pieters <mj@octobus.net>
parents: 41581
diff changeset
57 """Update the cache for the given filtered view on a repository"""
328ca3b9e545 branchmap: encapsulate cache updating in the map itself
Martijn Pieters <mj@octobus.net>
parents: 41581
diff changeset
58 # This can trigger updates for the caches for subsets of the filtered
328ca3b9e545 branchmap: encapsulate cache updating in the map itself
Martijn Pieters <mj@octobus.net>
parents: 41581
diff changeset
59 # view, e.g. when there is no cache for this filtered view or the cache
328ca3b9e545 branchmap: encapsulate cache updating in the map itself
Martijn Pieters <mj@octobus.net>
parents: 41581
diff changeset
60 # is stale.
18121
f8a13f061a8a branchmap: extract updatebranchcache from repo
Pierre-Yves David <pierre-yves.david@logilab.fr>
parents: 18120
diff changeset
61
41626
328ca3b9e545 branchmap: encapsulate cache updating in the map itself
Martijn Pieters <mj@octobus.net>
parents: 41581
diff changeset
62 cl = repo.changelog
328ca3b9e545 branchmap: encapsulate cache updating in the map itself
Martijn Pieters <mj@octobus.net>
parents: 41581
diff changeset
63 filtername = repo.filtername
328ca3b9e545 branchmap: encapsulate cache updating in the map itself
Martijn Pieters <mj@octobus.net>
parents: 41581
diff changeset
64 bcache = self._per_filter.get(filtername)
328ca3b9e545 branchmap: encapsulate cache updating in the map itself
Martijn Pieters <mj@octobus.net>
parents: 41581
diff changeset
65 if bcache is None or not bcache.validfor(repo):
328ca3b9e545 branchmap: encapsulate cache updating in the map itself
Martijn Pieters <mj@octobus.net>
parents: 41581
diff changeset
66 # cache object missing or cache object stale? Read from disk
328ca3b9e545 branchmap: encapsulate cache updating in the map itself
Martijn Pieters <mj@octobus.net>
parents: 41581
diff changeset
67 bcache = branchcache.fromfile(repo)
41581
c795c462b1d6 branchmap: add some clarifications and clean up flow
Martijn Pieters <mj@octobus.net>
parents: 41580
diff changeset
68
41626
328ca3b9e545 branchmap: encapsulate cache updating in the map itself
Martijn Pieters <mj@octobus.net>
parents: 41581
diff changeset
69 revs = []
328ca3b9e545 branchmap: encapsulate cache updating in the map itself
Martijn Pieters <mj@octobus.net>
parents: 41581
diff changeset
70 if bcache is None:
328ca3b9e545 branchmap: encapsulate cache updating in the map itself
Martijn Pieters <mj@octobus.net>
parents: 41581
diff changeset
71 # no (fresh) cache available anymore, perhaps we can re-use
328ca3b9e545 branchmap: encapsulate cache updating in the map itself
Martijn Pieters <mj@octobus.net>
parents: 41581
diff changeset
72 # the cache for a subset, then extend that to add info on missing
328ca3b9e545 branchmap: encapsulate cache updating in the map itself
Martijn Pieters <mj@octobus.net>
parents: 41581
diff changeset
73 # revisions.
328ca3b9e545 branchmap: encapsulate cache updating in the map itself
Martijn Pieters <mj@octobus.net>
parents: 41581
diff changeset
74 subsetname = subsettable.get(filtername)
328ca3b9e545 branchmap: encapsulate cache updating in the map itself
Martijn Pieters <mj@octobus.net>
parents: 41581
diff changeset
75 if subsetname is not None:
328ca3b9e545 branchmap: encapsulate cache updating in the map itself
Martijn Pieters <mj@octobus.net>
parents: 41581
diff changeset
76 subset = repo.filtered(subsetname)
328ca3b9e545 branchmap: encapsulate cache updating in the map itself
Martijn Pieters <mj@octobus.net>
parents: 41581
diff changeset
77 bcache = self[subset].copy()
328ca3b9e545 branchmap: encapsulate cache updating in the map itself
Martijn Pieters <mj@octobus.net>
parents: 41581
diff changeset
78 extrarevs = subset.changelog.filteredrevs - cl.filteredrevs
328ca3b9e545 branchmap: encapsulate cache updating in the map itself
Martijn Pieters <mj@octobus.net>
parents: 41581
diff changeset
79 revs.extend(r for r in extrarevs if r <= bcache.tiprev)
328ca3b9e545 branchmap: encapsulate cache updating in the map itself
Martijn Pieters <mj@octobus.net>
parents: 41581
diff changeset
80 else:
328ca3b9e545 branchmap: encapsulate cache updating in the map itself
Martijn Pieters <mj@octobus.net>
parents: 41581
diff changeset
81 # nothing to fall back on, start empty.
328ca3b9e545 branchmap: encapsulate cache updating in the map itself
Martijn Pieters <mj@octobus.net>
parents: 41581
diff changeset
82 bcache = branchcache()
24373
59cc09240afb revbranchcache: move out of branchmap onto localrepo
Durham Goode <durham@fb.com>
parents: 24163
diff changeset
83
41626
328ca3b9e545 branchmap: encapsulate cache updating in the map itself
Martijn Pieters <mj@octobus.net>
parents: 41581
diff changeset
84 revs.extend(cl.revs(start=bcache.tiprev + 1))
328ca3b9e545 branchmap: encapsulate cache updating in the map itself
Martijn Pieters <mj@octobus.net>
parents: 41581
diff changeset
85 if revs:
328ca3b9e545 branchmap: encapsulate cache updating in the map itself
Martijn Pieters <mj@octobus.net>
parents: 41581
diff changeset
86 bcache.update(repo, revs)
18124
79db6d40bced branchmap: store branchcache in a dedicated object
Pierre-Yves David <pierre-yves.david@logilab.fr>
parents: 18121
diff changeset
87
41626
328ca3b9e545 branchmap: encapsulate cache updating in the map itself
Martijn Pieters <mj@octobus.net>
parents: 41581
diff changeset
88 assert bcache.validfor(repo), filtername
328ca3b9e545 branchmap: encapsulate cache updating in the map itself
Martijn Pieters <mj@octobus.net>
parents: 41581
diff changeset
89 self._per_filter[repo.filtername] = bcache
328ca3b9e545 branchmap: encapsulate cache updating in the map itself
Martijn Pieters <mj@octobus.net>
parents: 41581
diff changeset
90
328ca3b9e545 branchmap: encapsulate cache updating in the map itself
Martijn Pieters <mj@octobus.net>
parents: 41581
diff changeset
91 def replace(self, repo, remotebranchmap):
328ca3b9e545 branchmap: encapsulate cache updating in the map itself
Martijn Pieters <mj@octobus.net>
parents: 41581
diff changeset
92 """Replace the branchmap cache for a repo with a branch mapping.
328ca3b9e545 branchmap: encapsulate cache updating in the map itself
Martijn Pieters <mj@octobus.net>
parents: 41581
diff changeset
93
328ca3b9e545 branchmap: encapsulate cache updating in the map itself
Martijn Pieters <mj@octobus.net>
parents: 41581
diff changeset
94 This is likely only called during clone with a branch map from a
328ca3b9e545 branchmap: encapsulate cache updating in the map itself
Martijn Pieters <mj@octobus.net>
parents: 41581
diff changeset
95 remote.
26460
79ef867538ea branchmap: move branch cache code out of streamclone.py
Gregory Szorc <gregory.szorc@gmail.com>
parents: 25918
diff changeset
96
41626
328ca3b9e545 branchmap: encapsulate cache updating in the map itself
Martijn Pieters <mj@octobus.net>
parents: 41581
diff changeset
97 """
328ca3b9e545 branchmap: encapsulate cache updating in the map itself
Martijn Pieters <mj@octobus.net>
parents: 41581
diff changeset
98 cl = repo.changelog
328ca3b9e545 branchmap: encapsulate cache updating in the map itself
Martijn Pieters <mj@octobus.net>
parents: 41581
diff changeset
99 clrev = cl.rev
328ca3b9e545 branchmap: encapsulate cache updating in the map itself
Martijn Pieters <mj@octobus.net>
parents: 41581
diff changeset
100 clbranchinfo = cl.branchinfo
328ca3b9e545 branchmap: encapsulate cache updating in the map itself
Martijn Pieters <mj@octobus.net>
parents: 41581
diff changeset
101 rbheads = []
328ca3b9e545 branchmap: encapsulate cache updating in the map itself
Martijn Pieters <mj@octobus.net>
parents: 41581
diff changeset
102 closed = []
328ca3b9e545 branchmap: encapsulate cache updating in the map itself
Martijn Pieters <mj@octobus.net>
parents: 41581
diff changeset
103 for bheads in remotebranchmap.itervalues():
328ca3b9e545 branchmap: encapsulate cache updating in the map itself
Martijn Pieters <mj@octobus.net>
parents: 41581
diff changeset
104 rbheads += bheads
328ca3b9e545 branchmap: encapsulate cache updating in the map itself
Martijn Pieters <mj@octobus.net>
parents: 41581
diff changeset
105 for h in bheads:
328ca3b9e545 branchmap: encapsulate cache updating in the map itself
Martijn Pieters <mj@octobus.net>
parents: 41581
diff changeset
106 r = clrev(h)
328ca3b9e545 branchmap: encapsulate cache updating in the map itself
Martijn Pieters <mj@octobus.net>
parents: 41581
diff changeset
107 b, c = clbranchinfo(r)
328ca3b9e545 branchmap: encapsulate cache updating in the map itself
Martijn Pieters <mj@octobus.net>
parents: 41581
diff changeset
108 if c:
328ca3b9e545 branchmap: encapsulate cache updating in the map itself
Martijn Pieters <mj@octobus.net>
parents: 41581
diff changeset
109 closed.append(h)
26460
79ef867538ea branchmap: move branch cache code out of streamclone.py
Gregory Szorc <gregory.szorc@gmail.com>
parents: 25918
diff changeset
110
41626
328ca3b9e545 branchmap: encapsulate cache updating in the map itself
Martijn Pieters <mj@octobus.net>
parents: 41581
diff changeset
111 if rbheads:
328ca3b9e545 branchmap: encapsulate cache updating in the map itself
Martijn Pieters <mj@octobus.net>
parents: 41581
diff changeset
112 rtiprev = max((int(clrev(node)) for node in rbheads))
328ca3b9e545 branchmap: encapsulate cache updating in the map itself
Martijn Pieters <mj@octobus.net>
parents: 41581
diff changeset
113 cache = branchcache(
328ca3b9e545 branchmap: encapsulate cache updating in the map itself
Martijn Pieters <mj@octobus.net>
parents: 41581
diff changeset
114 remotebranchmap, repo[rtiprev].node(), rtiprev,
328ca3b9e545 branchmap: encapsulate cache updating in the map itself
Martijn Pieters <mj@octobus.net>
parents: 41581
diff changeset
115 closednodes=closed)
26460
79ef867538ea branchmap: move branch cache code out of streamclone.py
Gregory Szorc <gregory.szorc@gmail.com>
parents: 25918
diff changeset
116
41626
328ca3b9e545 branchmap: encapsulate cache updating in the map itself
Martijn Pieters <mj@octobus.net>
parents: 41581
diff changeset
117 # Try to stick it as low as possible
328ca3b9e545 branchmap: encapsulate cache updating in the map itself
Martijn Pieters <mj@octobus.net>
parents: 41581
diff changeset
118 # filter above served are unlikely to be fetch from a clone
328ca3b9e545 branchmap: encapsulate cache updating in the map itself
Martijn Pieters <mj@octobus.net>
parents: 41581
diff changeset
119 for candidate in ('base', 'immutable', 'served'):
328ca3b9e545 branchmap: encapsulate cache updating in the map itself
Martijn Pieters <mj@octobus.net>
parents: 41581
diff changeset
120 rview = repo.filtered(candidate)
328ca3b9e545 branchmap: encapsulate cache updating in the map itself
Martijn Pieters <mj@octobus.net>
parents: 41581
diff changeset
121 if cache.validfor(rview):
328ca3b9e545 branchmap: encapsulate cache updating in the map itself
Martijn Pieters <mj@octobus.net>
parents: 41581
diff changeset
122 self._per_filter[candidate] = cache
328ca3b9e545 branchmap: encapsulate cache updating in the map itself
Martijn Pieters <mj@octobus.net>
parents: 41581
diff changeset
123 cache.write(rview)
328ca3b9e545 branchmap: encapsulate cache updating in the map itself
Martijn Pieters <mj@octobus.net>
parents: 41581
diff changeset
124 return
328ca3b9e545 branchmap: encapsulate cache updating in the map itself
Martijn Pieters <mj@octobus.net>
parents: 41581
diff changeset
125
328ca3b9e545 branchmap: encapsulate cache updating in the map itself
Martijn Pieters <mj@octobus.net>
parents: 41581
diff changeset
126 def clear(self):
328ca3b9e545 branchmap: encapsulate cache updating in the map itself
Martijn Pieters <mj@octobus.net>
parents: 41581
diff changeset
127 self._per_filter.clear()
328ca3b9e545 branchmap: encapsulate cache updating in the map itself
Martijn Pieters <mj@octobus.net>
parents: 41581
diff changeset
128
26460
79ef867538ea branchmap: move branch cache code out of streamclone.py
Gregory Szorc <gregory.szorc@gmail.com>
parents: 25918
diff changeset
129
42001
624d6683c705 branchmap: remove the dict interface from the branchcache class (API)
Pulkit Goyal <pulkit@yandex-team.ru>
parents: 41844
diff changeset
130 class branchcache(object):
20181
b9515fb9e72a branchmap: add documentation on the branchcache on-disk format
Brodie Rao <brodie@sf.io>
parents: 20032
diff changeset
131 """A dict like object that hold branches heads cache.
b9515fb9e72a branchmap: add documentation on the branchcache on-disk format
Brodie Rao <brodie@sf.io>
parents: 20032
diff changeset
132
b9515fb9e72a branchmap: add documentation on the branchcache on-disk format
Brodie Rao <brodie@sf.io>
parents: 20032
diff changeset
133 This cache is used to avoid costly computations to determine all the
b9515fb9e72a branchmap: add documentation on the branchcache on-disk format
Brodie Rao <brodie@sf.io>
parents: 20032
diff changeset
134 branch heads of a repo.
b9515fb9e72a branchmap: add documentation on the branchcache on-disk format
Brodie Rao <brodie@sf.io>
parents: 20032
diff changeset
135
b9515fb9e72a branchmap: add documentation on the branchcache on-disk format
Brodie Rao <brodie@sf.io>
parents: 20032
diff changeset
136 The cache is serialized on disk in the following format:
b9515fb9e72a branchmap: add documentation on the branchcache on-disk format
Brodie Rao <brodie@sf.io>
parents: 20032
diff changeset
137
b9515fb9e72a branchmap: add documentation on the branchcache on-disk format
Brodie Rao <brodie@sf.io>
parents: 20032
diff changeset
138 <tip hex node> <tip rev number> [optional filtered repo hex hash]
20185
7d4219512823 branchmap: cache open/closed branch head information
Brodie Rao <brodie@sf.io>
parents: 20181
diff changeset
139 <branch head hex node> <open/closed state> <branch name>
7d4219512823 branchmap: cache open/closed branch head information
Brodie Rao <brodie@sf.io>
parents: 20181
diff changeset
140 <branch head hex node> <open/closed state> <branch name>
20181
b9515fb9e72a branchmap: add documentation on the branchcache on-disk format
Brodie Rao <brodie@sf.io>
parents: 20032
diff changeset
141 ...
b9515fb9e72a branchmap: add documentation on the branchcache on-disk format
Brodie Rao <brodie@sf.io>
parents: 20032
diff changeset
142
b9515fb9e72a branchmap: add documentation on the branchcache on-disk format
Brodie Rao <brodie@sf.io>
parents: 20032
diff changeset
143 The first line is used to check if the cache is still valid. If the
b9515fb9e72a branchmap: add documentation on the branchcache on-disk format
Brodie Rao <brodie@sf.io>
parents: 20032
diff changeset
144 branch cache is for a filtered repo view, an optional third hash is
b9515fb9e72a branchmap: add documentation on the branchcache on-disk format
Brodie Rao <brodie@sf.io>
parents: 20032
diff changeset
145 included that hashes the hashes of all filtered revisions.
20185
7d4219512823 branchmap: cache open/closed branch head information
Brodie Rao <brodie@sf.io>
parents: 20181
diff changeset
146
7d4219512823 branchmap: cache open/closed branch head information
Brodie Rao <brodie@sf.io>
parents: 20181
diff changeset
147 The open/closed state is represented by a single letter 'o' or 'c'.
7d4219512823 branchmap: cache open/closed branch head information
Brodie Rao <brodie@sf.io>
parents: 20181
diff changeset
148 This field can be used to avoid changelog reads when determining if a
7d4219512823 branchmap: cache open/closed branch head information
Brodie Rao <brodie@sf.io>
parents: 20181
diff changeset
149 branch head closes a branch or not.
20181
b9515fb9e72a branchmap: add documentation on the branchcache on-disk format
Brodie Rao <brodie@sf.io>
parents: 20032
diff changeset
150 """
41688
bfc49f1df615 branchmap: move __init__ up in branchcache class
Pulkit Goyal <pulkit@yandex-team.ru>
parents: 41626
diff changeset
151
bfc49f1df615 branchmap: move __init__ up in branchcache class
Pulkit Goyal <pulkit@yandex-team.ru>
parents: 41626
diff changeset
152 def __init__(self, entries=(), tipnode=nullid, tiprev=nullrev,
bfc49f1df615 branchmap: move __init__ up in branchcache class
Pulkit Goyal <pulkit@yandex-team.ru>
parents: 41626
diff changeset
153 filteredhash=None, closednodes=None):
bfc49f1df615 branchmap: move __init__ up in branchcache class
Pulkit Goyal <pulkit@yandex-team.ru>
parents: 41626
diff changeset
154 self.tipnode = tipnode
bfc49f1df615 branchmap: move __init__ up in branchcache class
Pulkit Goyal <pulkit@yandex-team.ru>
parents: 41626
diff changeset
155 self.tiprev = tiprev
bfc49f1df615 branchmap: move __init__ up in branchcache class
Pulkit Goyal <pulkit@yandex-team.ru>
parents: 41626
diff changeset
156 self.filteredhash = filteredhash
bfc49f1df615 branchmap: move __init__ up in branchcache class
Pulkit Goyal <pulkit@yandex-team.ru>
parents: 41626
diff changeset
157 # closednodes is a set of nodes that close their branch. If the branch
bfc49f1df615 branchmap: move __init__ up in branchcache class
Pulkit Goyal <pulkit@yandex-team.ru>
parents: 41626
diff changeset
158 # cache has been updated, it may contain nodes that are no longer
bfc49f1df615 branchmap: move __init__ up in branchcache class
Pulkit Goyal <pulkit@yandex-team.ru>
parents: 41626
diff changeset
159 # heads.
bfc49f1df615 branchmap: move __init__ up in branchcache class
Pulkit Goyal <pulkit@yandex-team.ru>
parents: 41626
diff changeset
160 if closednodes is None:
bfc49f1df615 branchmap: move __init__ up in branchcache class
Pulkit Goyal <pulkit@yandex-team.ru>
parents: 41626
diff changeset
161 self._closednodes = set()
bfc49f1df615 branchmap: move __init__ up in branchcache class
Pulkit Goyal <pulkit@yandex-team.ru>
parents: 41626
diff changeset
162 else:
bfc49f1df615 branchmap: move __init__ up in branchcache class
Pulkit Goyal <pulkit@yandex-team.ru>
parents: 41626
diff changeset
163 self._closednodes = closednodes
42005
b137a6793c51 branchcache: make entries a private attribute
Pulkit Goyal <pulkit@yandex-team.ru>
parents: 42004
diff changeset
164 self._entries = dict(entries)
42006
111de135fc76 branchcache: add attributes to track which nodes are verified
Pulkit Goyal <pulkit@yandex-team.ru>
parents: 42005
diff changeset
165 # whether closed nodes are verified or not
111de135fc76 branchcache: add attributes to track which nodes are verified
Pulkit Goyal <pulkit@yandex-team.ru>
parents: 42005
diff changeset
166 self._closedverified = False
111de135fc76 branchcache: add attributes to track which nodes are verified
Pulkit Goyal <pulkit@yandex-team.ru>
parents: 42005
diff changeset
167 # branches for which nodes are verified
111de135fc76 branchcache: add attributes to track which nodes are verified
Pulkit Goyal <pulkit@yandex-team.ru>
parents: 42005
diff changeset
168 self._verifiedbranches = set()
42001
624d6683c705 branchmap: remove the dict interface from the branchcache class (API)
Pulkit Goyal <pulkit@yandex-team.ru>
parents: 41844
diff changeset
169
624d6683c705 branchmap: remove the dict interface from the branchcache class (API)
Pulkit Goyal <pulkit@yandex-team.ru>
parents: 41844
diff changeset
170 def __iter__(self):
42005
b137a6793c51 branchcache: make entries a private attribute
Pulkit Goyal <pulkit@yandex-team.ru>
parents: 42004
diff changeset
171 return iter(self._entries)
42001
624d6683c705 branchmap: remove the dict interface from the branchcache class (API)
Pulkit Goyal <pulkit@yandex-team.ru>
parents: 41844
diff changeset
172
624d6683c705 branchmap: remove the dict interface from the branchcache class (API)
Pulkit Goyal <pulkit@yandex-team.ru>
parents: 41844
diff changeset
173 def __setitem__(self, key, value):
42005
b137a6793c51 branchcache: make entries a private attribute
Pulkit Goyal <pulkit@yandex-team.ru>
parents: 42004
diff changeset
174 self._entries[key] = value
42001
624d6683c705 branchmap: remove the dict interface from the branchcache class (API)
Pulkit Goyal <pulkit@yandex-team.ru>
parents: 41844
diff changeset
175
624d6683c705 branchmap: remove the dict interface from the branchcache class (API)
Pulkit Goyal <pulkit@yandex-team.ru>
parents: 41844
diff changeset
176 def __getitem__(self, key):
42005
b137a6793c51 branchcache: make entries a private attribute
Pulkit Goyal <pulkit@yandex-team.ru>
parents: 42004
diff changeset
177 return self._entries[key]
42001
624d6683c705 branchmap: remove the dict interface from the branchcache class (API)
Pulkit Goyal <pulkit@yandex-team.ru>
parents: 41844
diff changeset
178
624d6683c705 branchmap: remove the dict interface from the branchcache class (API)
Pulkit Goyal <pulkit@yandex-team.ru>
parents: 41844
diff changeset
179 def iteritems(self):
42005
b137a6793c51 branchcache: make entries a private attribute
Pulkit Goyal <pulkit@yandex-team.ru>
parents: 42004
diff changeset
180 return self._entries.iteritems()
42001
624d6683c705 branchmap: remove the dict interface from the branchcache class (API)
Pulkit Goyal <pulkit@yandex-team.ru>
parents: 41844
diff changeset
181
42004
0bd730fbcc2b branchcache: introduce hasbranch()
Pulkit Goyal <pulkit@yandex-team.ru>
parents: 42003
diff changeset
182 def hasbranch(self, label):
0bd730fbcc2b branchcache: introduce hasbranch()
Pulkit Goyal <pulkit@yandex-team.ru>
parents: 42003
diff changeset
183 """ checks whether a branch of this name exists or not """
42005
b137a6793c51 branchcache: make entries a private attribute
Pulkit Goyal <pulkit@yandex-team.ru>
parents: 42004
diff changeset
184 return label in self._entries
42004
0bd730fbcc2b branchcache: introduce hasbranch()
Pulkit Goyal <pulkit@yandex-team.ru>
parents: 42003
diff changeset
185
41579
bf7fb97aecf1 branchmap: make branchcache responsible for reading
Martijn Pieters <mj@octobus.net>
parents: 41418
diff changeset
186 @classmethod
bf7fb97aecf1 branchmap: make branchcache responsible for reading
Martijn Pieters <mj@octobus.net>
parents: 41418
diff changeset
187 def fromfile(cls, repo):
bf7fb97aecf1 branchmap: make branchcache responsible for reading
Martijn Pieters <mj@octobus.net>
parents: 41418
diff changeset
188 f = None
bf7fb97aecf1 branchmap: make branchcache responsible for reading
Martijn Pieters <mj@octobus.net>
parents: 41418
diff changeset
189 try:
bf7fb97aecf1 branchmap: make branchcache responsible for reading
Martijn Pieters <mj@octobus.net>
parents: 41418
diff changeset
190 f = repo.cachevfs(cls._filename(repo))
bf7fb97aecf1 branchmap: make branchcache responsible for reading
Martijn Pieters <mj@octobus.net>
parents: 41418
diff changeset
191 lineiter = iter(f)
bf7fb97aecf1 branchmap: make branchcache responsible for reading
Martijn Pieters <mj@octobus.net>
parents: 41418
diff changeset
192 cachekey = next(lineiter).rstrip('\n').split(" ", 2)
bf7fb97aecf1 branchmap: make branchcache responsible for reading
Martijn Pieters <mj@octobus.net>
parents: 41418
diff changeset
193 last, lrev = cachekey[:2]
bf7fb97aecf1 branchmap: make branchcache responsible for reading
Martijn Pieters <mj@octobus.net>
parents: 41418
diff changeset
194 last, lrev = bin(last), int(lrev)
bf7fb97aecf1 branchmap: make branchcache responsible for reading
Martijn Pieters <mj@octobus.net>
parents: 41418
diff changeset
195 filteredhash = None
bf7fb97aecf1 branchmap: make branchcache responsible for reading
Martijn Pieters <mj@octobus.net>
parents: 41418
diff changeset
196 if len(cachekey) > 2:
bf7fb97aecf1 branchmap: make branchcache responsible for reading
Martijn Pieters <mj@octobus.net>
parents: 41418
diff changeset
197 filteredhash = bin(cachekey[2])
bf7fb97aecf1 branchmap: make branchcache responsible for reading
Martijn Pieters <mj@octobus.net>
parents: 41418
diff changeset
198 bcache = cls(tipnode=last, tiprev=lrev, filteredhash=filteredhash)
bf7fb97aecf1 branchmap: make branchcache responsible for reading
Martijn Pieters <mj@octobus.net>
parents: 41418
diff changeset
199 if not bcache.validfor(repo):
bf7fb97aecf1 branchmap: make branchcache responsible for reading
Martijn Pieters <mj@octobus.net>
parents: 41418
diff changeset
200 # invalidate the cache
bf7fb97aecf1 branchmap: make branchcache responsible for reading
Martijn Pieters <mj@octobus.net>
parents: 41418
diff changeset
201 raise ValueError(r'tip differs')
41813
8ad46ac6728e branchmap: prevent reading the file twice through different iterators
Pulkit Goyal <pulkit@yandex-team.ru>
parents: 41802
diff changeset
202 bcache.load(repo, lineiter)
41579
bf7fb97aecf1 branchmap: make branchcache responsible for reading
Martijn Pieters <mj@octobus.net>
parents: 41418
diff changeset
203 except (IOError, OSError):
bf7fb97aecf1 branchmap: make branchcache responsible for reading
Martijn Pieters <mj@octobus.net>
parents: 41418
diff changeset
204 return None
bf7fb97aecf1 branchmap: make branchcache responsible for reading
Martijn Pieters <mj@octobus.net>
parents: 41418
diff changeset
205
bf7fb97aecf1 branchmap: make branchcache responsible for reading
Martijn Pieters <mj@octobus.net>
parents: 41418
diff changeset
206 except Exception as inst:
bf7fb97aecf1 branchmap: make branchcache responsible for reading
Martijn Pieters <mj@octobus.net>
parents: 41418
diff changeset
207 if repo.ui.debugflag:
bf7fb97aecf1 branchmap: make branchcache responsible for reading
Martijn Pieters <mj@octobus.net>
parents: 41418
diff changeset
208 msg = 'invalid branchheads cache'
bf7fb97aecf1 branchmap: make branchcache responsible for reading
Martijn Pieters <mj@octobus.net>
parents: 41418
diff changeset
209 if repo.filtername is not None:
bf7fb97aecf1 branchmap: make branchcache responsible for reading
Martijn Pieters <mj@octobus.net>
parents: 41418
diff changeset
210 msg += ' (%s)' % repo.filtername
bf7fb97aecf1 branchmap: make branchcache responsible for reading
Martijn Pieters <mj@octobus.net>
parents: 41418
diff changeset
211 msg += ': %s\n'
bf7fb97aecf1 branchmap: make branchcache responsible for reading
Martijn Pieters <mj@octobus.net>
parents: 41418
diff changeset
212 repo.ui.debug(msg % pycompat.bytestr(inst))
bf7fb97aecf1 branchmap: make branchcache responsible for reading
Martijn Pieters <mj@octobus.net>
parents: 41418
diff changeset
213 bcache = None
bf7fb97aecf1 branchmap: make branchcache responsible for reading
Martijn Pieters <mj@octobus.net>
parents: 41418
diff changeset
214
bf7fb97aecf1 branchmap: make branchcache responsible for reading
Martijn Pieters <mj@octobus.net>
parents: 41418
diff changeset
215 finally:
bf7fb97aecf1 branchmap: make branchcache responsible for reading
Martijn Pieters <mj@octobus.net>
parents: 41418
diff changeset
216 if f:
bf7fb97aecf1 branchmap: make branchcache responsible for reading
Martijn Pieters <mj@octobus.net>
parents: 41418
diff changeset
217 f.close()
bf7fb97aecf1 branchmap: make branchcache responsible for reading
Martijn Pieters <mj@octobus.net>
parents: 41418
diff changeset
218
bf7fb97aecf1 branchmap: make branchcache responsible for reading
Martijn Pieters <mj@octobus.net>
parents: 41418
diff changeset
219 return bcache
bf7fb97aecf1 branchmap: make branchcache responsible for reading
Martijn Pieters <mj@octobus.net>
parents: 41418
diff changeset
220
41813
8ad46ac6728e branchmap: prevent reading the file twice through different iterators
Pulkit Goyal <pulkit@yandex-team.ru>
parents: 41802
diff changeset
221 def load(self, repo, lineiter):
8ad46ac6728e branchmap: prevent reading the file twice through different iterators
Pulkit Goyal <pulkit@yandex-team.ru>
parents: 41802
diff changeset
222 """ fully loads the branchcache by reading from the file using the line
8ad46ac6728e branchmap: prevent reading the file twice through different iterators
Pulkit Goyal <pulkit@yandex-team.ru>
parents: 41802
diff changeset
223 iterator passed"""
41802
68bbcc70e274 branchcache: move loading of branch names and nodes into it's own function
Pulkit Goyal <pulkit@yandex-team.ru>
parents: 41768
diff changeset
224 cl = repo.changelog
68bbcc70e274 branchcache: move loading of branch names and nodes into it's own function
Pulkit Goyal <pulkit@yandex-team.ru>
parents: 41768
diff changeset
225 for line in lineiter:
68bbcc70e274 branchcache: move loading of branch names and nodes into it's own function
Pulkit Goyal <pulkit@yandex-team.ru>
parents: 41768
diff changeset
226 line = line.rstrip('\n')
68bbcc70e274 branchcache: move loading of branch names and nodes into it's own function
Pulkit Goyal <pulkit@yandex-team.ru>
parents: 41768
diff changeset
227 if not line:
68bbcc70e274 branchcache: move loading of branch names and nodes into it's own function
Pulkit Goyal <pulkit@yandex-team.ru>
parents: 41768
diff changeset
228 continue
68bbcc70e274 branchcache: move loading of branch names and nodes into it's own function
Pulkit Goyal <pulkit@yandex-team.ru>
parents: 41768
diff changeset
229 node, state, label = line.split(" ", 2)
68bbcc70e274 branchcache: move loading of branch names and nodes into it's own function
Pulkit Goyal <pulkit@yandex-team.ru>
parents: 41768
diff changeset
230 if state not in 'oc':
68bbcc70e274 branchcache: move loading of branch names and nodes into it's own function
Pulkit Goyal <pulkit@yandex-team.ru>
parents: 41768
diff changeset
231 raise ValueError(r'invalid branch state')
68bbcc70e274 branchcache: move loading of branch names and nodes into it's own function
Pulkit Goyal <pulkit@yandex-team.ru>
parents: 41768
diff changeset
232 label = encoding.tolocal(label.strip())
68bbcc70e274 branchcache: move loading of branch names and nodes into it's own function
Pulkit Goyal <pulkit@yandex-team.ru>
parents: 41768
diff changeset
233 node = bin(node)
68bbcc70e274 branchcache: move loading of branch names and nodes into it's own function
Pulkit Goyal <pulkit@yandex-team.ru>
parents: 41768
diff changeset
234 if not cl.hasnode(node):
68bbcc70e274 branchcache: move loading of branch names and nodes into it's own function
Pulkit Goyal <pulkit@yandex-team.ru>
parents: 41768
diff changeset
235 raise ValueError(
68bbcc70e274 branchcache: move loading of branch names and nodes into it's own function
Pulkit Goyal <pulkit@yandex-team.ru>
parents: 41768
diff changeset
236 r'node %s does not exist' % pycompat.sysstr(hex(node)))
42005
b137a6793c51 branchcache: make entries a private attribute
Pulkit Goyal <pulkit@yandex-team.ru>
parents: 42004
diff changeset
237 self._entries.setdefault(label, []).append(node)
42006
111de135fc76 branchcache: add attributes to track which nodes are verified
Pulkit Goyal <pulkit@yandex-team.ru>
parents: 42005
diff changeset
238 self._verifiedbranches.add(label)
41802
68bbcc70e274 branchcache: move loading of branch names and nodes into it's own function
Pulkit Goyal <pulkit@yandex-team.ru>
parents: 41768
diff changeset
239 if state == 'c':
68bbcc70e274 branchcache: move loading of branch names and nodes into it's own function
Pulkit Goyal <pulkit@yandex-team.ru>
parents: 41768
diff changeset
240 self._closednodes.add(node)
42006
111de135fc76 branchcache: add attributes to track which nodes are verified
Pulkit Goyal <pulkit@yandex-team.ru>
parents: 42005
diff changeset
241 self._closedverified = True
41802
68bbcc70e274 branchcache: move loading of branch names and nodes into it's own function
Pulkit Goyal <pulkit@yandex-team.ru>
parents: 41768
diff changeset
242
41579
bf7fb97aecf1 branchmap: make branchcache responsible for reading
Martijn Pieters <mj@octobus.net>
parents: 41418
diff changeset
243 @staticmethod
bf7fb97aecf1 branchmap: make branchcache responsible for reading
Martijn Pieters <mj@octobus.net>
parents: 41418
diff changeset
244 def _filename(repo):
bf7fb97aecf1 branchmap: make branchcache responsible for reading
Martijn Pieters <mj@octobus.net>
parents: 41418
diff changeset
245 """name of a branchcache file for a given repo or repoview"""
bf7fb97aecf1 branchmap: make branchcache responsible for reading
Martijn Pieters <mj@octobus.net>
parents: 41418
diff changeset
246 filename = "branch2"
bf7fb97aecf1 branchmap: make branchcache responsible for reading
Martijn Pieters <mj@octobus.net>
parents: 41418
diff changeset
247 if repo.filtername:
bf7fb97aecf1 branchmap: make branchcache responsible for reading
Martijn Pieters <mj@octobus.net>
parents: 41418
diff changeset
248 filename = '%s-%s' % (filename, repo.filtername)
bf7fb97aecf1 branchmap: make branchcache responsible for reading
Martijn Pieters <mj@octobus.net>
parents: 41418
diff changeset
249 return filename
18124
79db6d40bced branchmap: store branchcache in a dedicated object
Pierre-Yves David <pierre-yves.david@logilab.fr>
parents: 18121
diff changeset
250
18132
db25bf1dc828 branchmap: move validity logic in the object itself
Pierre-Yves David <pierre-yves.david@ens-lyon.org>
parents: 18131
diff changeset
251 def validfor(self, repo):
18644
3e92772d5383 spelling: fix some minor issues found by spell checker
Mads Kiilerich <mads@kiilerich.com>
parents: 18451
diff changeset
252 """Is the cache content valid regarding a repo
18132
db25bf1dc828 branchmap: move validity logic in the object itself
Pierre-Yves David <pierre-yves.david@ens-lyon.org>
parents: 18131
diff changeset
253
18644
3e92772d5383 spelling: fix some minor issues found by spell checker
Mads Kiilerich <mads@kiilerich.com>
parents: 18451
diff changeset
254 - False when cached tipnode is unknown or if we detect a strip.
18132
db25bf1dc828 branchmap: move validity logic in the object itself
Pierre-Yves David <pierre-yves.david@ens-lyon.org>
parents: 18131
diff changeset
255 - True when cache is up to date or a subset of current repo."""
db25bf1dc828 branchmap: move validity logic in the object itself
Pierre-Yves David <pierre-yves.david@ens-lyon.org>
parents: 18131
diff changeset
256 try:
18168
c351759ab0a0 branchmap: takes filtered revision in account for cache calculation
Pierre-Yves David <pierre-yves.david@ens-lyon.org>
parents: 18167
diff changeset
257 return ((self.tipnode == repo.changelog.node(self.tiprev))
41768
aaad36b88298 cleanup: use () to wrap long lines instead of \
Augie Fackler <augie@google.com>
parents: 41729
diff changeset
258 and (self.filteredhash ==
24723
467a33142425 repoview: move function for computing filtered hash
Gregory Szorc <gregory.szorc@gmail.com>
parents: 24378
diff changeset
259 scmutil.filteredhash(repo, self.tiprev)))
18132
db25bf1dc828 branchmap: move validity logic in the object itself
Pierre-Yves David <pierre-yves.david@ens-lyon.org>
parents: 18131
diff changeset
260 except IndexError:
db25bf1dc828 branchmap: move validity logic in the object itself
Pierre-Yves David <pierre-yves.david@ens-lyon.org>
parents: 18131
diff changeset
261 return False
db25bf1dc828 branchmap: move validity logic in the object itself
Pierre-Yves David <pierre-yves.david@ens-lyon.org>
parents: 18131
diff changeset
262
20186
f5b461a4bc55 branchmap: introduce branchtip() method
Brodie Rao <brodie@sf.io>
parents: 20185
diff changeset
263 def _branchtip(self, heads):
20245
4edd179fefb8 help: branch names primarily denote the tipmost unclosed branch head
Mads Kiilerich <madski@unity3d.com>
parents: 20190
diff changeset
264 '''Return tuple with last open head in heads and false,
4edd179fefb8 help: branch names primarily denote the tipmost unclosed branch head
Mads Kiilerich <madski@unity3d.com>
parents: 20190
diff changeset
265 otherwise return last closed head and true.'''
20186
f5b461a4bc55 branchmap: introduce branchtip() method
Brodie Rao <brodie@sf.io>
parents: 20185
diff changeset
266 tip = heads[-1]
f5b461a4bc55 branchmap: introduce branchtip() method
Brodie Rao <brodie@sf.io>
parents: 20185
diff changeset
267 closed = True
f5b461a4bc55 branchmap: introduce branchtip() method
Brodie Rao <brodie@sf.io>
parents: 20185
diff changeset
268 for h in reversed(heads):
f5b461a4bc55 branchmap: introduce branchtip() method
Brodie Rao <brodie@sf.io>
parents: 20185
diff changeset
269 if h not in self._closednodes:
f5b461a4bc55 branchmap: introduce branchtip() method
Brodie Rao <brodie@sf.io>
parents: 20185
diff changeset
270 tip = h
f5b461a4bc55 branchmap: introduce branchtip() method
Brodie Rao <brodie@sf.io>
parents: 20185
diff changeset
271 closed = False
f5b461a4bc55 branchmap: introduce branchtip() method
Brodie Rao <brodie@sf.io>
parents: 20185
diff changeset
272 break
f5b461a4bc55 branchmap: introduce branchtip() method
Brodie Rao <brodie@sf.io>
parents: 20185
diff changeset
273 return tip, closed
f5b461a4bc55 branchmap: introduce branchtip() method
Brodie Rao <brodie@sf.io>
parents: 20185
diff changeset
274
f5b461a4bc55 branchmap: introduce branchtip() method
Brodie Rao <brodie@sf.io>
parents: 20185
diff changeset
275 def branchtip(self, branch):
20245
4edd179fefb8 help: branch names primarily denote the tipmost unclosed branch head
Mads Kiilerich <madski@unity3d.com>
parents: 20190
diff changeset
276 '''Return the tipmost open head on branch head, otherwise return the
4edd179fefb8 help: branch names primarily denote the tipmost unclosed branch head
Mads Kiilerich <madski@unity3d.com>
parents: 20190
diff changeset
277 tipmost closed head on branch.
4edd179fefb8 help: branch names primarily denote the tipmost unclosed branch head
Mads Kiilerich <madski@unity3d.com>
parents: 20190
diff changeset
278 Raise KeyError for unknown branch.'''
20186
f5b461a4bc55 branchmap: introduce branchtip() method
Brodie Rao <brodie@sf.io>
parents: 20185
diff changeset
279 return self._branchtip(self[branch])[0]
f5b461a4bc55 branchmap: introduce branchtip() method
Brodie Rao <brodie@sf.io>
parents: 20185
diff changeset
280
34091
abf91c4f9608 branches: correctly show inactive multiheaded branches
the31k <the31k@thethirty.one>
parents: 33661
diff changeset
281 def iteropen(self, nodes):
abf91c4f9608 branches: correctly show inactive multiheaded branches
the31k <the31k@thethirty.one>
parents: 33661
diff changeset
282 return (n for n in nodes if n not in self._closednodes)
abf91c4f9608 branches: correctly show inactive multiheaded branches
the31k <the31k@thethirty.one>
parents: 33661
diff changeset
283
20188
3a3727829607 branchmap: introduce branchheads() method
Brodie Rao <brodie@sf.io>
parents: 20186
diff changeset
284 def branchheads(self, branch, closed=False):
3a3727829607 branchmap: introduce branchheads() method
Brodie Rao <brodie@sf.io>
parents: 20186
diff changeset
285 heads = self[branch]
3a3727829607 branchmap: introduce branchheads() method
Brodie Rao <brodie@sf.io>
parents: 20186
diff changeset
286 if not closed:
34091
abf91c4f9608 branches: correctly show inactive multiheaded branches
the31k <the31k@thethirty.one>
parents: 33661
diff changeset
287 heads = list(self.iteropen(heads))
20188
3a3727829607 branchmap: introduce branchheads() method
Brodie Rao <brodie@sf.io>
parents: 20186
diff changeset
288 return heads
3a3727829607 branchmap: introduce branchheads() method
Brodie Rao <brodie@sf.io>
parents: 20186
diff changeset
289
20190
d5d25e541637 branchmap: introduce iterbranches() method
Brodie Rao <brodie@sf.io>
parents: 20188
diff changeset
290 def iterbranches(self):
d5d25e541637 branchmap: introduce iterbranches() method
Brodie Rao <brodie@sf.io>
parents: 20188
diff changeset
291 for bn, heads in self.iteritems():
d5d25e541637 branchmap: introduce iterbranches() method
Brodie Rao <brodie@sf.io>
parents: 20188
diff changeset
292 yield (bn, heads) + self._branchtip(heads)
d5d25e541637 branchmap: introduce iterbranches() method
Brodie Rao <brodie@sf.io>
parents: 20188
diff changeset
293
42002
662ffdde5adf branchcache: rename itervalues() to iterheads()
Pulkit Goyal <pulkit@yandex-team.ru>
parents: 42001
diff changeset
294 def iterheads(self):
662ffdde5adf branchcache: rename itervalues() to iterheads()
Pulkit Goyal <pulkit@yandex-team.ru>
parents: 42001
diff changeset
295 """ returns all the heads """
42005
b137a6793c51 branchcache: make entries a private attribute
Pulkit Goyal <pulkit@yandex-team.ru>
parents: 42004
diff changeset
296 return self._entries.itervalues()
42002
662ffdde5adf branchcache: rename itervalues() to iterheads()
Pulkit Goyal <pulkit@yandex-team.ru>
parents: 42001
diff changeset
297
18232
dd0b636b0b65 branchmap: add a copy method
Pierre-Yves David <pierre-yves.david@ens-lyon.org>
parents: 18218
diff changeset
298 def copy(self):
dd0b636b0b65 branchmap: add a copy method
Pierre-Yves David <pierre-yves.david@ens-lyon.org>
parents: 18218
diff changeset
299 """return an deep copy of the branchcache object"""
42001
624d6683c705 branchmap: remove the dict interface from the branchcache class (API)
Pulkit Goyal <pulkit@yandex-team.ru>
parents: 41844
diff changeset
300 return branchcache(
42005
b137a6793c51 branchcache: make entries a private attribute
Pulkit Goyal <pulkit@yandex-team.ru>
parents: 42004
diff changeset
301 self._entries, self.tipnode, self.tiprev, self.filteredhash,
41580
eb7ce452e0fb branchmap: updating triggers a write
Martijn Pieters <mj@octobus.net>
parents: 41579
diff changeset
302 self._closednodes)
18132
db25bf1dc828 branchmap: move validity logic in the object itself
Pierre-Yves David <pierre-yves.david@ens-lyon.org>
parents: 18131
diff changeset
303
18128
f0d56efaa35a branchmap: make write a method on the branchmap object
Pierre-Yves David <pierre-yves.david@logilab.fr>
parents: 18127
diff changeset
304 def write(self, repo):
f0d56efaa35a branchmap: make write a method on the branchmap object
Pierre-Yves David <pierre-yves.david@logilab.fr>
parents: 18127
diff changeset
305 try:
41579
bf7fb97aecf1 branchmap: make branchcache responsible for reading
Martijn Pieters <mj@octobus.net>
parents: 41418
diff changeset
306 f = repo.cachevfs(self._filename(repo), "w", atomictemp=True)
31357
fc06c5260639 branchmap: stringify int in a portable way
Augie Fackler <augie@google.com>
parents: 31356
diff changeset
307 cachekey = [hex(self.tipnode), '%d' % self.tiprev]
18184
8d48af68e2ae branchmap: read and write key part related to filtered revision
Pierre-Yves David <pierre-yves.david@logilab.fr>
parents: 18168
diff changeset
308 if self.filteredhash is not None:
8d48af68e2ae branchmap: read and write key part related to filtered revision
Pierre-Yves David <pierre-yves.david@logilab.fr>
parents: 18168
diff changeset
309 cachekey.append(hex(self.filteredhash))
8d48af68e2ae branchmap: read and write key part related to filtered revision
Pierre-Yves David <pierre-yves.david@logilab.fr>
parents: 18168
diff changeset
310 f.write(" ".join(cachekey) + '\n')
21031
05cfcecb3aef branchmap: log events related to branch cache
Gregory Szorc <gregory.szorc@gmail.com>
parents: 20264
diff changeset
311 nodecount = 0
18357
a4ab37ca887b localrepo: store branchheads sorted
Mads Kiilerich <mads@kiilerich.com>
parents: 18307
diff changeset
312 for label, nodes in sorted(self.iteritems()):
41689
9d0d8793e847 branchmap: decode a label only once
Pulkit Goyal <pulkit@yandex-team.ru>
parents: 41688
diff changeset
313 label = encoding.fromlocal(label)
18128
f0d56efaa35a branchmap: make write a method on the branchmap object
Pierre-Yves David <pierre-yves.david@logilab.fr>
parents: 18127
diff changeset
314 for node in nodes:
21031
05cfcecb3aef branchmap: log events related to branch cache
Gregory Szorc <gregory.szorc@gmail.com>
parents: 20264
diff changeset
315 nodecount += 1
20185
7d4219512823 branchmap: cache open/closed branch head information
Brodie Rao <brodie@sf.io>
parents: 20181
diff changeset
316 if node in self._closednodes:
7d4219512823 branchmap: cache open/closed branch head information
Brodie Rao <brodie@sf.io>
parents: 20181
diff changeset
317 state = 'c'
7d4219512823 branchmap: cache open/closed branch head information
Brodie Rao <brodie@sf.io>
parents: 20181
diff changeset
318 else:
7d4219512823 branchmap: cache open/closed branch head information
Brodie Rao <brodie@sf.io>
parents: 20181
diff changeset
319 state = 'o'
41689
9d0d8793e847 branchmap: decode a label only once
Pulkit Goyal <pulkit@yandex-team.ru>
parents: 41688
diff changeset
320 f.write("%s %s %s\n" % (hex(node), state, label))
18128
f0d56efaa35a branchmap: make write a method on the branchmap object
Pierre-Yves David <pierre-yves.david@logilab.fr>
parents: 18127
diff changeset
321 f.close()
21031
05cfcecb3aef branchmap: log events related to branch cache
Gregory Szorc <gregory.szorc@gmail.com>
parents: 20264
diff changeset
322 repo.ui.log('branchcache',
05cfcecb3aef branchmap: log events related to branch cache
Gregory Szorc <gregory.szorc@gmail.com>
parents: 20264
diff changeset
323 'wrote %s branch cache with %d labels and %d nodes\n',
42005
b137a6793c51 branchcache: make entries a private attribute
Pulkit Goyal <pulkit@yandex-team.ru>
parents: 42004
diff changeset
324 repo.filtername, len(self._entries), nodecount)
26587
56b2bcea2529 error: get Abort from 'error' instead of 'util'
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 26460
diff changeset
325 except (IOError, OSError, error.Abort) as inst:
34371
d0db41af73c0 branchmap: remove superfluous pass statements
Augie Fackler <augie@google.com>
parents: 34091
diff changeset
326 # Abort may be raised by read only opener, so log and continue
36426
743b293c3ca0 py3: use util.forcebytestr to convert error messages to bytes
Pulkit Goyal <7895pulkit@gmail.com>
parents: 36239
diff changeset
327 repo.ui.debug("couldn't write branch cache: %s\n" %
37087
f0b6fbea00cf stringutil: bulk-replace call sites to point to new module
Yuya Nishihara <yuya@tcha.org>
parents: 36968
diff changeset
328 stringutil.forcebytestr(inst))
18131
f0eeb9b3444a branchmap: make update a method
Pierre-Yves David <pierre-yves.david@ens-lyon.org>
parents: 18130
diff changeset
329
18305
2502a15e033d branchmap: pass revision insteads of changectx to the update function
Pierre-Yves David <pierre-yves.david@ens-lyon.org>
parents: 18234
diff changeset
330 def update(self, repo, revgen):
18131
f0eeb9b3444a branchmap: make update a method
Pierre-Yves David <pierre-yves.david@ens-lyon.org>
parents: 18130
diff changeset
331 """Given a branchhead cache, self, that may have extra nodes or be
20263
ea4996754d91 branchmap: simplify update code
Pierre-Yves David <pierre-yves.david@ens-lyon.org>
parents: 20262
diff changeset
332 missing heads, and a generator of nodes that are strictly a superset of
18131
f0eeb9b3444a branchmap: make update a method
Pierre-Yves David <pierre-yves.david@ens-lyon.org>
parents: 18130
diff changeset
333 heads missing, this function updates self to be correct.
f0eeb9b3444a branchmap: make update a method
Pierre-Yves David <pierre-yves.david@ens-lyon.org>
parents: 18130
diff changeset
334 """
30995
22fbca1d11ed mercurial: switch to util.timer for all interval timings
Simon Farnsworth <simonfar@fb.com>
parents: 29758
diff changeset
335 starttime = util.timer()
18131
f0eeb9b3444a branchmap: make update a method
Pierre-Yves David <pierre-yves.david@ens-lyon.org>
parents: 18130
diff changeset
336 cl = repo.changelog
f0eeb9b3444a branchmap: make update a method
Pierre-Yves David <pierre-yves.david@ens-lyon.org>
parents: 18130
diff changeset
337 # collect new branch entries
f0eeb9b3444a branchmap: make update a method
Pierre-Yves David <pierre-yves.david@ens-lyon.org>
parents: 18130
diff changeset
338 newbranches = {}
24373
59cc09240afb revbranchcache: move out of branchmap onto localrepo
Durham Goode <durham@fb.com>
parents: 24163
diff changeset
339 getbranchinfo = repo.revbranchcache().branchinfo
18307
0eed2546118a branchmap: Save changectx creation during update
Pierre-Yves David <pierre-yves.david@logilab.fr>
parents: 18305
diff changeset
340 for r in revgen:
40494
5e5c8f2a1eb5 branchmap: do not specify changelog as an argument
Yuya Nishihara <yuya@tcha.org>
parents: 40375
diff changeset
341 branch, closesbranch = getbranchinfo(r)
20262
cf450ee3f8f7 branchmap: stop useless rev -> node -> rev round trip
Pierre-Yves David <pierre-yves.david@ens-lyon.org>
parents: 20261
diff changeset
342 newbranches.setdefault(branch, []).append(r)
20185
7d4219512823 branchmap: cache open/closed branch head information
Brodie Rao <brodie@sf.io>
parents: 20181
diff changeset
343 if closesbranch:
20262
cf450ee3f8f7 branchmap: stop useless rev -> node -> rev round trip
Pierre-Yves David <pierre-yves.david@ens-lyon.org>
parents: 20261
diff changeset
344 self._closednodes.add(cl.node(r))
22357
9c3c3dc14a65 branchmap: pre-filter topological heads before ancestors based filtering
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 22356
diff changeset
345
9c3c3dc14a65 branchmap: pre-filter topological heads before ancestors based filtering
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 22356
diff changeset
346 # fetch current topological heads to speed up filtering
9c3c3dc14a65 branchmap: pre-filter topological heads before ancestors based filtering
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 22356
diff changeset
347 topoheads = set(cl.headrevs())
9c3c3dc14a65 branchmap: pre-filter topological heads before ancestors based filtering
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 22356
diff changeset
348
18131
f0eeb9b3444a branchmap: make update a method
Pierre-Yves David <pierre-yves.david@ens-lyon.org>
parents: 18130
diff changeset
349 # if older branchheads are reachable from new ones, they aren't
f0eeb9b3444a branchmap: make update a method
Pierre-Yves David <pierre-yves.david@ens-lyon.org>
parents: 18130
diff changeset
350 # really branchheads. Note checking parents is insufficient:
f0eeb9b3444a branchmap: make update a method
Pierre-Yves David <pierre-yves.david@ens-lyon.org>
parents: 18130
diff changeset
351 # 1 (branch a) -> 2 (branch b) -> 3 (branch a)
20262
cf450ee3f8f7 branchmap: stop useless rev -> node -> rev round trip
Pierre-Yves David <pierre-yves.david@ens-lyon.org>
parents: 20261
diff changeset
352 for branch, newheadrevs in newbranches.iteritems():
42005
b137a6793c51 branchcache: make entries a private attribute
Pulkit Goyal <pulkit@yandex-team.ru>
parents: 42004
diff changeset
353 bheads = self._entries.setdefault(branch, [])
20264
d9e1c167943b branchmap: use set for update code
Pierre-Yves David <pierre-yves.david@ens-lyon.org>
parents: 20263
diff changeset
354 bheadset = set(cl.rev(node) for node in bheads)
18131
f0eeb9b3444a branchmap: make update a method
Pierre-Yves David <pierre-yves.david@ens-lyon.org>
parents: 18130
diff changeset
355
20263
ea4996754d91 branchmap: simplify update code
Pierre-Yves David <pierre-yves.david@ens-lyon.org>
parents: 20262
diff changeset
356 # This have been tested True on all internal usage of this function.
ea4996754d91 branchmap: simplify update code
Pierre-Yves David <pierre-yves.david@ens-lyon.org>
parents: 20262
diff changeset
357 # run it again in case of doubt
ea4996754d91 branchmap: simplify update code
Pierre-Yves David <pierre-yves.david@ens-lyon.org>
parents: 20262
diff changeset
358 # assert not (set(bheadrevs) & set(newheadrevs))
20264
d9e1c167943b branchmap: use set for update code
Pierre-Yves David <pierre-yves.david@ens-lyon.org>
parents: 20263
diff changeset
359 bheadset.update(newheadrevs)
18131
f0eeb9b3444a branchmap: make update a method
Pierre-Yves David <pierre-yves.david@ens-lyon.org>
parents: 18130
diff changeset
360
22356
3c8fb24334e9 branchmap: issue a single call to `ancestors` for all heads
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 22059
diff changeset
361 # This prunes out two kinds of heads - heads that are superseded by
3c8fb24334e9 branchmap: issue a single call to `ancestors` for all heads
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 22059
diff changeset
362 # a head in newheadrevs, and newheadrevs that are not heads because
3c8fb24334e9 branchmap: issue a single call to `ancestors` for all heads
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 22059
diff changeset
363 # an existing head is their descendant.
22357
9c3c3dc14a65 branchmap: pre-filter topological heads before ancestors based filtering
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 22356
diff changeset
364 uncertain = bheadset - topoheads
9c3c3dc14a65 branchmap: pre-filter topological heads before ancestors based filtering
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 22356
diff changeset
365 if uncertain:
9c3c3dc14a65 branchmap: pre-filter topological heads before ancestors based filtering
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 22356
diff changeset
366 floorrev = min(uncertain)
9c3c3dc14a65 branchmap: pre-filter topological heads before ancestors based filtering
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 22356
diff changeset
367 ancestors = set(cl.ancestors(newheadrevs, floorrev))
9c3c3dc14a65 branchmap: pre-filter topological heads before ancestors based filtering
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 22356
diff changeset
368 bheadset -= ancestors
20264
d9e1c167943b branchmap: use set for update code
Pierre-Yves David <pierre-yves.david@ens-lyon.org>
parents: 20263
diff changeset
369 bheadrevs = sorted(bheadset)
18131
f0eeb9b3444a branchmap: make update a method
Pierre-Yves David <pierre-yves.david@ens-lyon.org>
parents: 18130
diff changeset
370 self[branch] = [cl.node(rev) for rev in bheadrevs]
20263
ea4996754d91 branchmap: simplify update code
Pierre-Yves David <pierre-yves.david@ens-lyon.org>
parents: 20262
diff changeset
371 tiprev = bheadrevs[-1]
18131
f0eeb9b3444a branchmap: make update a method
Pierre-Yves David <pierre-yves.david@ens-lyon.org>
parents: 18130
diff changeset
372 if tiprev > self.tiprev:
f0eeb9b3444a branchmap: make update a method
Pierre-Yves David <pierre-yves.david@ens-lyon.org>
parents: 18130
diff changeset
373 self.tipnode = cl.node(tiprev)
f0eeb9b3444a branchmap: make update a method
Pierre-Yves David <pierre-yves.david@ens-lyon.org>
parents: 18130
diff changeset
374 self.tiprev = tiprev
f0eeb9b3444a branchmap: make update a method
Pierre-Yves David <pierre-yves.david@ens-lyon.org>
parents: 18130
diff changeset
375
19838
23386881abeb branchmap: remove the droppednodes logic
Pierre-Yves David <pierre-yves.david@ens-lyon.org>
parents: 19837
diff changeset
376 if not self.validfor(repo):
18131
f0eeb9b3444a branchmap: make update a method
Pierre-Yves David <pierre-yves.david@ens-lyon.org>
parents: 18130
diff changeset
377 # cache key are not valid anymore
f0eeb9b3444a branchmap: make update a method
Pierre-Yves David <pierre-yves.david@ens-lyon.org>
parents: 18130
diff changeset
378 self.tipnode = nullid
f0eeb9b3444a branchmap: make update a method
Pierre-Yves David <pierre-yves.david@ens-lyon.org>
parents: 18130
diff changeset
379 self.tiprev = nullrev
42002
662ffdde5adf branchcache: rename itervalues() to iterheads()
Pulkit Goyal <pulkit@yandex-team.ru>
parents: 42001
diff changeset
380 for heads in self.iterheads():
18131
f0eeb9b3444a branchmap: make update a method
Pierre-Yves David <pierre-yves.david@ens-lyon.org>
parents: 18130
diff changeset
381 tiprev = max(cl.rev(node) for node in heads)
f0eeb9b3444a branchmap: make update a method
Pierre-Yves David <pierre-yves.david@ens-lyon.org>
parents: 18130
diff changeset
382 if tiprev > self.tiprev:
f0eeb9b3444a branchmap: make update a method
Pierre-Yves David <pierre-yves.david@ens-lyon.org>
parents: 18130
diff changeset
383 self.tipnode = cl.node(tiprev)
f0eeb9b3444a branchmap: make update a method
Pierre-Yves David <pierre-yves.david@ens-lyon.org>
parents: 18130
diff changeset
384 self.tiprev = tiprev
24723
467a33142425 repoview: move function for computing filtered hash
Gregory Szorc <gregory.szorc@gmail.com>
parents: 24378
diff changeset
385 self.filteredhash = scmutil.filteredhash(repo, self.tiprev)
21031
05cfcecb3aef branchmap: log events related to branch cache
Gregory Szorc <gregory.szorc@gmail.com>
parents: 20264
diff changeset
386
30995
22fbca1d11ed mercurial: switch to util.timer for all interval timings
Simon Farnsworth <simonfar@fb.com>
parents: 29758
diff changeset
387 duration = util.timer() - starttime
21031
05cfcecb3aef branchmap: log events related to branch cache
Gregory Szorc <gregory.szorc@gmail.com>
parents: 20264
diff changeset
388 repo.ui.log('branchcache', 'updated %s branch cache in %.4f seconds\n',
41844
38de3300414f py3: convert filtername to str if it's None
Pulkit Goyal <pulkit@yandex-team.ru>
parents: 41813
diff changeset
389 repo.filtername or b'None', duration)
23785
cb99bacb9b4e branchcache: introduce revbranchcache for caching of revision branch names
Mads Kiilerich <madski@unity3d.com>
parents: 22357
diff changeset
390
41580
eb7ce452e0fb branchmap: updating triggers a write
Martijn Pieters <mj@octobus.net>
parents: 41579
diff changeset
391 self.write(repo)
eb7ce452e0fb branchmap: updating triggers a write
Martijn Pieters <mj@octobus.net>
parents: 41579
diff changeset
392
eb7ce452e0fb branchmap: updating triggers a write
Martijn Pieters <mj@octobus.net>
parents: 41579
diff changeset
393
eb7ce452e0fb branchmap: updating triggers a write
Martijn Pieters <mj@octobus.net>
parents: 41579
diff changeset
394 class remotebranchcache(branchcache):
eb7ce452e0fb branchmap: updating triggers a write
Martijn Pieters <mj@octobus.net>
parents: 41579
diff changeset
395 """Branchmap info for a remote connection, should not write locally"""
eb7ce452e0fb branchmap: updating triggers a write
Martijn Pieters <mj@octobus.net>
parents: 41579
diff changeset
396 def write(self, repo):
eb7ce452e0fb branchmap: updating triggers a write
Martijn Pieters <mj@octobus.net>
parents: 41579
diff changeset
397 pass
eb7ce452e0fb branchmap: updating triggers a write
Martijn Pieters <mj@octobus.net>
parents: 41579
diff changeset
398
eb7ce452e0fb branchmap: updating triggers a write
Martijn Pieters <mj@octobus.net>
parents: 41579
diff changeset
399
23785
cb99bacb9b4e branchcache: introduce revbranchcache for caching of revision branch names
Mads Kiilerich <madski@unity3d.com>
parents: 22357
diff changeset
400 # Revision branch info cache
cb99bacb9b4e branchcache: introduce revbranchcache for caching of revision branch names
Mads Kiilerich <madski@unity3d.com>
parents: 22357
diff changeset
401
cb99bacb9b4e branchcache: introduce revbranchcache for caching of revision branch names
Mads Kiilerich <madski@unity3d.com>
parents: 22357
diff changeset
402 _rbcversion = '-v1'
33535
755e6532e81d cachevfs: migration the revbranchcache to 'cachevfs'
Boris Feld <boris.feld@octobus.net>
parents: 33534
diff changeset
403 _rbcnames = 'rbc-names' + _rbcversion
755e6532e81d cachevfs: migration the revbranchcache to 'cachevfs'
Boris Feld <boris.feld@octobus.net>
parents: 33534
diff changeset
404 _rbcrevs = 'rbc-revs' + _rbcversion
23785
cb99bacb9b4e branchcache: introduce revbranchcache for caching of revision branch names
Mads Kiilerich <madski@unity3d.com>
parents: 22357
diff changeset
405 # [4 byte hash prefix][4 byte branch name number with sign bit indicating open]
cb99bacb9b4e branchcache: introduce revbranchcache for caching of revision branch names
Mads Kiilerich <madski@unity3d.com>
parents: 22357
diff changeset
406 _rbcrecfmt = '>4sI'
cb99bacb9b4e branchcache: introduce revbranchcache for caching of revision branch names
Mads Kiilerich <madski@unity3d.com>
parents: 22357
diff changeset
407 _rbcrecsize = calcsize(_rbcrecfmt)
cb99bacb9b4e branchcache: introduce revbranchcache for caching of revision branch names
Mads Kiilerich <madski@unity3d.com>
parents: 22357
diff changeset
408 _rbcnodelen = 4
cb99bacb9b4e branchcache: introduce revbranchcache for caching of revision branch names
Mads Kiilerich <madski@unity3d.com>
parents: 22357
diff changeset
409 _rbcbranchidxmask = 0x7fffffff
cb99bacb9b4e branchcache: introduce revbranchcache for caching of revision branch names
Mads Kiilerich <madski@unity3d.com>
parents: 22357
diff changeset
410 _rbccloseflag = 0x80000000
cb99bacb9b4e branchcache: introduce revbranchcache for caching of revision branch names
Mads Kiilerich <madski@unity3d.com>
parents: 22357
diff changeset
411
cb99bacb9b4e branchcache: introduce revbranchcache for caching of revision branch names
Mads Kiilerich <madski@unity3d.com>
parents: 22357
diff changeset
412 class revbranchcache(object):
cb99bacb9b4e branchcache: introduce revbranchcache for caching of revision branch names
Mads Kiilerich <madski@unity3d.com>
parents: 22357
diff changeset
413 """Persistent cache, mapping from revision number to branch name and close.
cb99bacb9b4e branchcache: introduce revbranchcache for caching of revision branch names
Mads Kiilerich <madski@unity3d.com>
parents: 22357
diff changeset
414 This is a low level cache, independent of filtering.
cb99bacb9b4e branchcache: introduce revbranchcache for caching of revision branch names
Mads Kiilerich <madski@unity3d.com>
parents: 22357
diff changeset
415
cb99bacb9b4e branchcache: introduce revbranchcache for caching of revision branch names
Mads Kiilerich <madski@unity3d.com>
parents: 22357
diff changeset
416 Branch names are stored in rbc-names in internal encoding separated by 0.
cb99bacb9b4e branchcache: introduce revbranchcache for caching of revision branch names
Mads Kiilerich <madski@unity3d.com>
parents: 22357
diff changeset
417 rbc-names is append-only, and each branch name is only stored once and will
cb99bacb9b4e branchcache: introduce revbranchcache for caching of revision branch names
Mads Kiilerich <madski@unity3d.com>
parents: 22357
diff changeset
418 thus have a unique index.
cb99bacb9b4e branchcache: introduce revbranchcache for caching of revision branch names
Mads Kiilerich <madski@unity3d.com>
parents: 22357
diff changeset
419
cb99bacb9b4e branchcache: introduce revbranchcache for caching of revision branch names
Mads Kiilerich <madski@unity3d.com>
parents: 22357
diff changeset
420 The branch info for each revision is stored in rbc-revs as constant size
cb99bacb9b4e branchcache: introduce revbranchcache for caching of revision branch names
Mads Kiilerich <madski@unity3d.com>
parents: 22357
diff changeset
421 records. The whole file is read into memory, but it is only 'parsed' on
cb99bacb9b4e branchcache: introduce revbranchcache for caching of revision branch names
Mads Kiilerich <madski@unity3d.com>
parents: 22357
diff changeset
422 demand. The file is usually append-only but will be truncated if repo
cb99bacb9b4e branchcache: introduce revbranchcache for caching of revision branch names
Mads Kiilerich <madski@unity3d.com>
parents: 22357
diff changeset
423 modification is detected.
cb99bacb9b4e branchcache: introduce revbranchcache for caching of revision branch names
Mads Kiilerich <madski@unity3d.com>
parents: 22357
diff changeset
424 The record for each revision contains the first 4 bytes of the
cb99bacb9b4e branchcache: introduce revbranchcache for caching of revision branch names
Mads Kiilerich <madski@unity3d.com>
parents: 22357
diff changeset
425 corresponding node hash, and the record is only used if it still matches.
cb99bacb9b4e branchcache: introduce revbranchcache for caching of revision branch names
Mads Kiilerich <madski@unity3d.com>
parents: 22357
diff changeset
426 Even a completely trashed rbc-revs fill thus still give the right result
cb99bacb9b4e branchcache: introduce revbranchcache for caching of revision branch names
Mads Kiilerich <madski@unity3d.com>
parents: 22357
diff changeset
427 while converging towards full recovery ... assuming no incorrectly matching
cb99bacb9b4e branchcache: introduce revbranchcache for caching of revision branch names
Mads Kiilerich <madski@unity3d.com>
parents: 22357
diff changeset
428 node hashes.
cb99bacb9b4e branchcache: introduce revbranchcache for caching of revision branch names
Mads Kiilerich <madski@unity3d.com>
parents: 22357
diff changeset
429 The record also contains 4 bytes where 31 bits contains the index of the
cb99bacb9b4e branchcache: introduce revbranchcache for caching of revision branch names
Mads Kiilerich <madski@unity3d.com>
parents: 22357
diff changeset
430 branch and the last bit indicate that it is a branch close commit.
cb99bacb9b4e branchcache: introduce revbranchcache for caching of revision branch names
Mads Kiilerich <madski@unity3d.com>
parents: 22357
diff changeset
431 The usage pattern for rbc-revs is thus somewhat similar to 00changelog.i
cb99bacb9b4e branchcache: introduce revbranchcache for caching of revision branch names
Mads Kiilerich <madski@unity3d.com>
parents: 22357
diff changeset
432 and will grow with it but be 1/8th of its size.
cb99bacb9b4e branchcache: introduce revbranchcache for caching of revision branch names
Mads Kiilerich <madski@unity3d.com>
parents: 22357
diff changeset
433 """
cb99bacb9b4e branchcache: introduce revbranchcache for caching of revision branch names
Mads Kiilerich <madski@unity3d.com>
parents: 22357
diff changeset
434
24159
5b4ed033390b revisionbranchcache: fall back to slow path if starting readonly (issue4531)
Mads Kiilerich <madski@unity3d.com>
parents: 23877
diff changeset
435 def __init__(self, repo, readonly=True):
23785
cb99bacb9b4e branchcache: introduce revbranchcache for caching of revision branch names
Mads Kiilerich <madski@unity3d.com>
parents: 22357
diff changeset
436 assert repo.filtername is None
24374
77fd1fb538cd revbranchcache: store repo on the object
Durham Goode <durham@fb.com>
parents: 24373
diff changeset
437 self._repo = repo
23785
cb99bacb9b4e branchcache: introduce revbranchcache for caching of revision branch names
Mads Kiilerich <madski@unity3d.com>
parents: 22357
diff changeset
438 self._names = [] # branch names in local encoding with static index
31355
2a18e9e6ca43 py3: use bytearray() instead of array('c', ...) constructions
Augie Fackler <augie@google.com>
parents: 30995
diff changeset
439 self._rbcrevs = bytearray()
29615
a2a380e2750f rbc: fix superfluous rebuilding from scratch - don't abuse self._rbcnamescount
Mads Kiilerich <madski@unity3d.com>
parents: 29604
diff changeset
440 self._rbcsnameslen = 0 # length of names read at _rbcsnameslen
23785
cb99bacb9b4e branchcache: introduce revbranchcache for caching of revision branch names
Mads Kiilerich <madski@unity3d.com>
parents: 22357
diff changeset
441 try:
33535
755e6532e81d cachevfs: migration the revbranchcache to 'cachevfs'
Boris Feld <boris.feld@octobus.net>
parents: 33534
diff changeset
442 bndata = repo.cachevfs.read(_rbcnames)
23785
cb99bacb9b4e branchcache: introduce revbranchcache for caching of revision branch names
Mads Kiilerich <madski@unity3d.com>
parents: 22357
diff changeset
443 self._rbcsnameslen = len(bndata) # for verification before writing
31380
7dd2f51f38ac rbc: empty (and invalid) rbc-names file should give an empty name list
Mads Kiilerich <mads@kiilerich.com>
parents: 31379
diff changeset
444 if bndata:
7dd2f51f38ac rbc: empty (and invalid) rbc-names file should give an empty name list
Mads Kiilerich <mads@kiilerich.com>
parents: 31379
diff changeset
445 self._names = [encoding.tolocal(bn)
7dd2f51f38ac rbc: empty (and invalid) rbc-names file should give an empty name list
Mads Kiilerich <mads@kiilerich.com>
parents: 31379
diff changeset
446 for bn in bndata.split('\0')]
29423
d2c6f3a948fa branchmap: remove unused exception variable
Gregory Szorc <gregory.szorc@gmail.com>
parents: 28558
diff changeset
447 except (IOError, OSError):
24159
5b4ed033390b revisionbranchcache: fall back to slow path if starting readonly (issue4531)
Mads Kiilerich <madski@unity3d.com>
parents: 23877
diff changeset
448 if readonly:
5b4ed033390b revisionbranchcache: fall back to slow path if starting readonly (issue4531)
Mads Kiilerich <madski@unity3d.com>
parents: 23877
diff changeset
449 # don't try to use cache - fall back to the slow path
5b4ed033390b revisionbranchcache: fall back to slow path if starting readonly (issue4531)
Mads Kiilerich <madski@unity3d.com>
parents: 23877
diff changeset
450 self.branchinfo = self._branchinfo
5b4ed033390b revisionbranchcache: fall back to slow path if starting readonly (issue4531)
Mads Kiilerich <madski@unity3d.com>
parents: 23877
diff changeset
451
23785
cb99bacb9b4e branchcache: introduce revbranchcache for caching of revision branch names
Mads Kiilerich <madski@unity3d.com>
parents: 22357
diff changeset
452 if self._names:
cb99bacb9b4e branchcache: introduce revbranchcache for caching of revision branch names
Mads Kiilerich <madski@unity3d.com>
parents: 22357
diff changeset
453 try:
33535
755e6532e81d cachevfs: migration the revbranchcache to 'cachevfs'
Boris Feld <boris.feld@octobus.net>
parents: 33534
diff changeset
454 data = repo.cachevfs.read(_rbcrevs)
31355
2a18e9e6ca43 py3: use bytearray() instead of array('c', ...) constructions
Augie Fackler <augie@google.com>
parents: 30995
diff changeset
455 self._rbcrevs[:] = data
25660
328739ea70c3 global: mass rewrite to use modern exception syntax
Gregory Szorc <gregory.szorc@gmail.com>
parents: 25266
diff changeset
456 except (IOError, OSError) as inst:
23785
cb99bacb9b4e branchcache: introduce revbranchcache for caching of revision branch names
Mads Kiilerich <madski@unity3d.com>
parents: 22357
diff changeset
457 repo.ui.debug("couldn't read revision branch cache: %s\n" %
37087
f0b6fbea00cf stringutil: bulk-replace call sites to point to new module
Yuya Nishihara <yuya@tcha.org>
parents: 36968
diff changeset
458 stringutil.forcebytestr(inst))
23785
cb99bacb9b4e branchcache: introduce revbranchcache for caching of revision branch names
Mads Kiilerich <madski@unity3d.com>
parents: 22357
diff changeset
459 # remember number of good records on disk
cb99bacb9b4e branchcache: introduce revbranchcache for caching of revision branch names
Mads Kiilerich <madski@unity3d.com>
parents: 22357
diff changeset
460 self._rbcrevslen = min(len(self._rbcrevs) // _rbcrecsize,
cb99bacb9b4e branchcache: introduce revbranchcache for caching of revision branch names
Mads Kiilerich <madski@unity3d.com>
parents: 22357
diff changeset
461 len(repo.changelog))
cb99bacb9b4e branchcache: introduce revbranchcache for caching of revision branch names
Mads Kiilerich <madski@unity3d.com>
parents: 22357
diff changeset
462 if self._rbcrevslen == 0:
cb99bacb9b4e branchcache: introduce revbranchcache for caching of revision branch names
Mads Kiilerich <madski@unity3d.com>
parents: 22357
diff changeset
463 self._names = []
29615
a2a380e2750f rbc: fix superfluous rebuilding from scratch - don't abuse self._rbcnamescount
Mads Kiilerich <madski@unity3d.com>
parents: 29604
diff changeset
464 self._rbcnamescount = len(self._names) # number of names read at
a2a380e2750f rbc: fix superfluous rebuilding from scratch - don't abuse self._rbcnamescount
Mads Kiilerich <madski@unity3d.com>
parents: 29604
diff changeset
465 # _rbcsnameslen
23785
cb99bacb9b4e branchcache: introduce revbranchcache for caching of revision branch names
Mads Kiilerich <madski@unity3d.com>
parents: 22357
diff changeset
466
28558
bcd106d456c4 cache: rebuild branch cache from scratch when inconsistencies are detected
Mads Kiilerich <madski@unity3d.com>
parents: 28557
diff changeset
467 def _clear(self):
bcd106d456c4 cache: rebuild branch cache from scratch when inconsistencies are detected
Mads Kiilerich <madski@unity3d.com>
parents: 28557
diff changeset
468 self._rbcsnameslen = 0
bcd106d456c4 cache: rebuild branch cache from scratch when inconsistencies are detected
Mads Kiilerich <madski@unity3d.com>
parents: 28557
diff changeset
469 del self._names[:]
bcd106d456c4 cache: rebuild branch cache from scratch when inconsistencies are detected
Mads Kiilerich <madski@unity3d.com>
parents: 28557
diff changeset
470 self._rbcnamescount = 0
bcd106d456c4 cache: rebuild branch cache from scratch when inconsistencies are detected
Mads Kiilerich <madski@unity3d.com>
parents: 28557
diff changeset
471 self._rbcrevslen = len(self._repo.changelog)
31355
2a18e9e6ca43 py3: use bytearray() instead of array('c', ...) constructions
Augie Fackler <augie@google.com>
parents: 30995
diff changeset
472 self._rbcrevs = bytearray(self._rbcrevslen * _rbcrecsize)
40749
50a64c321c1e branchmap: build the revbranchcache._namesreverse() only when required
Pulkit Goyal <pulkit@yandex-team.ru>
parents: 40494
diff changeset
473 util.clearcachedproperty(self, '_namesreverse')
50a64c321c1e branchmap: build the revbranchcache._namesreverse() only when required
Pulkit Goyal <pulkit@yandex-team.ru>
parents: 40494
diff changeset
474
50a64c321c1e branchmap: build the revbranchcache._namesreverse() only when required
Pulkit Goyal <pulkit@yandex-team.ru>
parents: 40494
diff changeset
475 @util.propertycache
50a64c321c1e branchmap: build the revbranchcache._namesreverse() only when required
Pulkit Goyal <pulkit@yandex-team.ru>
parents: 40494
diff changeset
476 def _namesreverse(self):
50a64c321c1e branchmap: build the revbranchcache._namesreverse() only when required
Pulkit Goyal <pulkit@yandex-team.ru>
parents: 40494
diff changeset
477 return dict((b, r) for r, b in enumerate(self._names))
28558
bcd106d456c4 cache: rebuild branch cache from scratch when inconsistencies are detected
Mads Kiilerich <madski@unity3d.com>
parents: 28557
diff changeset
478
40494
5e5c8f2a1eb5 branchmap: do not specify changelog as an argument
Yuya Nishihara <yuya@tcha.org>
parents: 40375
diff changeset
479 def branchinfo(self, rev):
23785
cb99bacb9b4e branchcache: introduce revbranchcache for caching of revision branch names
Mads Kiilerich <madski@unity3d.com>
parents: 22357
diff changeset
480 """Return branch name and close flag for rev, using and updating
cb99bacb9b4e branchcache: introduce revbranchcache for caching of revision branch names
Mads Kiilerich <madski@unity3d.com>
parents: 22357
diff changeset
481 persistent cache."""
40494
5e5c8f2a1eb5 branchmap: do not specify changelog as an argument
Yuya Nishihara <yuya@tcha.org>
parents: 40375
diff changeset
482 changelog = self._repo.changelog
23785
cb99bacb9b4e branchcache: introduce revbranchcache for caching of revision branch names
Mads Kiilerich <madski@unity3d.com>
parents: 22357
diff changeset
483 rbcrevidx = rev * _rbcrecsize
cb99bacb9b4e branchcache: introduce revbranchcache for caching of revision branch names
Mads Kiilerich <madski@unity3d.com>
parents: 22357
diff changeset
484
25266
38117278f295 revbranchcache: return uncached branchinfo for nullrev (issue4683)
Yuya Nishihara <yuya@tcha.org>
parents: 24728
diff changeset
485 # avoid negative index, changelog.read(nullrev) is fast without cache
38117278f295 revbranchcache: return uncached branchinfo for nullrev (issue4683)
Yuya Nishihara <yuya@tcha.org>
parents: 24728
diff changeset
486 if rev == nullrev:
38117278f295 revbranchcache: return uncached branchinfo for nullrev (issue4683)
Yuya Nishihara <yuya@tcha.org>
parents: 24728
diff changeset
487 return changelog.branchinfo(rev)
38117278f295 revbranchcache: return uncached branchinfo for nullrev (issue4683)
Yuya Nishihara <yuya@tcha.org>
parents: 24728
diff changeset
488
29604
db0095c83344 rbc: fix invalid rbc-revs entries caused by missing cache growth
Mads Kiilerich <madski@unity3d.com>
parents: 29423
diff changeset
489 # if requested rev isn't allocated, grow and cache the rev info
23785
cb99bacb9b4e branchcache: introduce revbranchcache for caching of revision branch names
Mads Kiilerich <madski@unity3d.com>
parents: 22357
diff changeset
490 if len(self._rbcrevs) < rbcrevidx + _rbcrecsize:
40494
5e5c8f2a1eb5 branchmap: do not specify changelog as an argument
Yuya Nishihara <yuya@tcha.org>
parents: 40375
diff changeset
491 return self._branchinfo(rev)
23785
cb99bacb9b4e branchcache: introduce revbranchcache for caching of revision branch names
Mads Kiilerich <madski@unity3d.com>
parents: 22357
diff changeset
492
cb99bacb9b4e branchcache: introduce revbranchcache for caching of revision branch names
Mads Kiilerich <madski@unity3d.com>
parents: 22357
diff changeset
493 # fast path: extract data from cache, use it if node is matching
cb99bacb9b4e branchcache: introduce revbranchcache for caching of revision branch names
Mads Kiilerich <madski@unity3d.com>
parents: 22357
diff changeset
494 reponode = changelog.node(rev)[:_rbcnodelen]
33661
1814ca418b30 branchmap: revert c34532365b38 for Python 2.7 compatibility
Mike Hommey <mh@glandium.org>
parents: 33535
diff changeset
495 cachenode, branchidx = unpack_from(
1814ca418b30 branchmap: revert c34532365b38 for Python 2.7 compatibility
Mike Hommey <mh@glandium.org>
parents: 33535
diff changeset
496 _rbcrecfmt, util.buffer(self._rbcrevs), rbcrevidx)
23785
cb99bacb9b4e branchcache: introduce revbranchcache for caching of revision branch names
Mads Kiilerich <madski@unity3d.com>
parents: 22357
diff changeset
497 close = bool(branchidx & _rbccloseflag)
cb99bacb9b4e branchcache: introduce revbranchcache for caching of revision branch names
Mads Kiilerich <madski@unity3d.com>
parents: 22357
diff changeset
498 if close:
cb99bacb9b4e branchcache: introduce revbranchcache for caching of revision branch names
Mads Kiilerich <madski@unity3d.com>
parents: 22357
diff changeset
499 branchidx &= _rbcbranchidxmask
24376
203a078da052 revbranchcache: populate cache incrementally
Durham Goode <durham@fb.com>
parents: 24375
diff changeset
500 if cachenode == '\0\0\0\0':
203a078da052 revbranchcache: populate cache incrementally
Durham Goode <durham@fb.com>
parents: 24375
diff changeset
501 pass
203a078da052 revbranchcache: populate cache incrementally
Durham Goode <durham@fb.com>
parents: 24375
diff changeset
502 elif cachenode == reponode:
29615
a2a380e2750f rbc: fix superfluous rebuilding from scratch - don't abuse self._rbcnamescount
Mads Kiilerich <madski@unity3d.com>
parents: 29604
diff changeset
503 try:
28558
bcd106d456c4 cache: rebuild branch cache from scratch when inconsistencies are detected
Mads Kiilerich <madski@unity3d.com>
parents: 28557
diff changeset
504 return self._names[branchidx], close
29615
a2a380e2750f rbc: fix superfluous rebuilding from scratch - don't abuse self._rbcnamescount
Mads Kiilerich <madski@unity3d.com>
parents: 29604
diff changeset
505 except IndexError:
a2a380e2750f rbc: fix superfluous rebuilding from scratch - don't abuse self._rbcnamescount
Mads Kiilerich <madski@unity3d.com>
parents: 29604
diff changeset
506 # recover from invalid reference to unknown branch
a2a380e2750f rbc: fix superfluous rebuilding from scratch - don't abuse self._rbcnamescount
Mads Kiilerich <madski@unity3d.com>
parents: 29604
diff changeset
507 self._repo.ui.debug("referenced branch names not found"
a2a380e2750f rbc: fix superfluous rebuilding from scratch - don't abuse self._rbcnamescount
Mads Kiilerich <madski@unity3d.com>
parents: 29604
diff changeset
508 " - rebuilding revision branch cache from scratch\n")
a2a380e2750f rbc: fix superfluous rebuilding from scratch - don't abuse self._rbcnamescount
Mads Kiilerich <madski@unity3d.com>
parents: 29604
diff changeset
509 self._clear()
24376
203a078da052 revbranchcache: populate cache incrementally
Durham Goode <durham@fb.com>
parents: 24375
diff changeset
510 else:
203a078da052 revbranchcache: populate cache incrementally
Durham Goode <durham@fb.com>
parents: 24375
diff changeset
511 # rev/node map has changed, invalidate the cache from here up
29615
a2a380e2750f rbc: fix superfluous rebuilding from scratch - don't abuse self._rbcnamescount
Mads Kiilerich <madski@unity3d.com>
parents: 29604
diff changeset
512 self._repo.ui.debug("history modification detected - truncating "
31504
a369482e9649 branchmap: be more careful about using %d on ints
Augie Fackler <augie@google.com>
parents: 31463
diff changeset
513 "revision branch cache to revision %d\n" % rev)
24376
203a078da052 revbranchcache: populate cache incrementally
Durham Goode <durham@fb.com>
parents: 24375
diff changeset
514 truncate = rbcrevidx + _rbcrecsize
203a078da052 revbranchcache: populate cache incrementally
Durham Goode <durham@fb.com>
parents: 24375
diff changeset
515 del self._rbcrevs[truncate:]
203a078da052 revbranchcache: populate cache incrementally
Durham Goode <durham@fb.com>
parents: 24375
diff changeset
516 self._rbcrevslen = min(self._rbcrevslen, truncate)
203a078da052 revbranchcache: populate cache incrementally
Durham Goode <durham@fb.com>
parents: 24375
diff changeset
517
23785
cb99bacb9b4e branchcache: introduce revbranchcache for caching of revision branch names
Mads Kiilerich <madski@unity3d.com>
parents: 22357
diff changeset
518 # fall back to slow path and make sure it will be written to disk
40494
5e5c8f2a1eb5 branchmap: do not specify changelog as an argument
Yuya Nishihara <yuya@tcha.org>
parents: 40375
diff changeset
519 return self._branchinfo(rev)
23785
cb99bacb9b4e branchcache: introduce revbranchcache for caching of revision branch names
Mads Kiilerich <madski@unity3d.com>
parents: 22357
diff changeset
520
40494
5e5c8f2a1eb5 branchmap: do not specify changelog as an argument
Yuya Nishihara <yuya@tcha.org>
parents: 40375
diff changeset
521 def _branchinfo(self, rev):
23785
cb99bacb9b4e branchcache: introduce revbranchcache for caching of revision branch names
Mads Kiilerich <madski@unity3d.com>
parents: 22357
diff changeset
522 """Retrieve branch info from changelog and update _rbcrevs"""
40494
5e5c8f2a1eb5 branchmap: do not specify changelog as an argument
Yuya Nishihara <yuya@tcha.org>
parents: 40375
diff changeset
523 changelog = self._repo.changelog
23785
cb99bacb9b4e branchcache: introduce revbranchcache for caching of revision branch names
Mads Kiilerich <madski@unity3d.com>
parents: 22357
diff changeset
524 b, close = changelog.branchinfo(rev)
cb99bacb9b4e branchcache: introduce revbranchcache for caching of revision branch names
Mads Kiilerich <madski@unity3d.com>
parents: 22357
diff changeset
525 if b in self._namesreverse:
cb99bacb9b4e branchcache: introduce revbranchcache for caching of revision branch names
Mads Kiilerich <madski@unity3d.com>
parents: 22357
diff changeset
526 branchidx = self._namesreverse[b]
cb99bacb9b4e branchcache: introduce revbranchcache for caching of revision branch names
Mads Kiilerich <madski@unity3d.com>
parents: 22357
diff changeset
527 else:
cb99bacb9b4e branchcache: introduce revbranchcache for caching of revision branch names
Mads Kiilerich <madski@unity3d.com>
parents: 22357
diff changeset
528 branchidx = len(self._names)
cb99bacb9b4e branchcache: introduce revbranchcache for caching of revision branch names
Mads Kiilerich <madski@unity3d.com>
parents: 22357
diff changeset
529 self._names.append(b)
cb99bacb9b4e branchcache: introduce revbranchcache for caching of revision branch names
Mads Kiilerich <madski@unity3d.com>
parents: 22357
diff changeset
530 self._namesreverse[b] = branchidx
cb99bacb9b4e branchcache: introduce revbranchcache for caching of revision branch names
Mads Kiilerich <madski@unity3d.com>
parents: 22357
diff changeset
531 reponode = changelog.node(rev)
cb99bacb9b4e branchcache: introduce revbranchcache for caching of revision branch names
Mads Kiilerich <madski@unity3d.com>
parents: 22357
diff changeset
532 if close:
cb99bacb9b4e branchcache: introduce revbranchcache for caching of revision branch names
Mads Kiilerich <madski@unity3d.com>
parents: 22357
diff changeset
533 branchidx |= _rbccloseflag
40494
5e5c8f2a1eb5 branchmap: do not specify changelog as an argument
Yuya Nishihara <yuya@tcha.org>
parents: 40375
diff changeset
534 self._setcachedata(rev, reponode, branchidx)
24375
fe255b2525d5 revbranchcache: move entry writing to a separate function
Durham Goode <durham@fb.com>
parents: 24374
diff changeset
535 return b, close
fe255b2525d5 revbranchcache: move entry writing to a separate function
Durham Goode <durham@fb.com>
parents: 24374
diff changeset
536
36968
95f4f1bfb650 revbranchcache: add a public function to update the data
Boris Feld <boris.feld@octobus.net>
parents: 36504
diff changeset
537 def setdata(self, branch, rev, node, close):
95f4f1bfb650 revbranchcache: add a public function to update the data
Boris Feld <boris.feld@octobus.net>
parents: 36504
diff changeset
538 """add new data information to the cache"""
95f4f1bfb650 revbranchcache: add a public function to update the data
Boris Feld <boris.feld@octobus.net>
parents: 36504
diff changeset
539 if branch in self._namesreverse:
95f4f1bfb650 revbranchcache: add a public function to update the data
Boris Feld <boris.feld@octobus.net>
parents: 36504
diff changeset
540 branchidx = self._namesreverse[branch]
95f4f1bfb650 revbranchcache: add a public function to update the data
Boris Feld <boris.feld@octobus.net>
parents: 36504
diff changeset
541 else:
95f4f1bfb650 revbranchcache: add a public function to update the data
Boris Feld <boris.feld@octobus.net>
parents: 36504
diff changeset
542 branchidx = len(self._names)
95f4f1bfb650 revbranchcache: add a public function to update the data
Boris Feld <boris.feld@octobus.net>
parents: 36504
diff changeset
543 self._names.append(branch)
95f4f1bfb650 revbranchcache: add a public function to update the data
Boris Feld <boris.feld@octobus.net>
parents: 36504
diff changeset
544 self._namesreverse[branch] = branchidx
95f4f1bfb650 revbranchcache: add a public function to update the data
Boris Feld <boris.feld@octobus.net>
parents: 36504
diff changeset
545 if close:
95f4f1bfb650 revbranchcache: add a public function to update the data
Boris Feld <boris.feld@octobus.net>
parents: 36504
diff changeset
546 branchidx |= _rbccloseflag
95f4f1bfb650 revbranchcache: add a public function to update the data
Boris Feld <boris.feld@octobus.net>
parents: 36504
diff changeset
547 self._setcachedata(rev, node, branchidx)
95f4f1bfb650 revbranchcache: add a public function to update the data
Boris Feld <boris.feld@octobus.net>
parents: 36504
diff changeset
548 # If no cache data were readable (non exists, bad permission, etc)
95f4f1bfb650 revbranchcache: add a public function to update the data
Boris Feld <boris.feld@octobus.net>
parents: 36504
diff changeset
549 # the cache was bypassing itself by setting:
95f4f1bfb650 revbranchcache: add a public function to update the data
Boris Feld <boris.feld@octobus.net>
parents: 36504
diff changeset
550 #
95f4f1bfb650 revbranchcache: add a public function to update the data
Boris Feld <boris.feld@octobus.net>
parents: 36504
diff changeset
551 # self.branchinfo = self._branchinfo
95f4f1bfb650 revbranchcache: add a public function to update the data
Boris Feld <boris.feld@octobus.net>
parents: 36504
diff changeset
552 #
95f4f1bfb650 revbranchcache: add a public function to update the data
Boris Feld <boris.feld@octobus.net>
parents: 36504
diff changeset
553 # Since we now have data in the cache, we need to drop this bypassing.
40232
d99a588d8515 py3: pass in system string to vars(branchmap).__contains__()
Yuya Nishihara <yuya@tcha.org>
parents: 39175
diff changeset
554 if r'branchinfo' in vars(self):
36968
95f4f1bfb650 revbranchcache: add a public function to update the data
Boris Feld <boris.feld@octobus.net>
parents: 36504
diff changeset
555 del self.branchinfo
95f4f1bfb650 revbranchcache: add a public function to update the data
Boris Feld <boris.feld@octobus.net>
parents: 36504
diff changeset
556
40494
5e5c8f2a1eb5 branchmap: do not specify changelog as an argument
Yuya Nishihara <yuya@tcha.org>
parents: 40375
diff changeset
557 def _setcachedata(self, rev, node, branchidx):
24375
fe255b2525d5 revbranchcache: move entry writing to a separate function
Durham Goode <durham@fb.com>
parents: 24374
diff changeset
558 """Writes the node's branch data to the in-memory cache data."""
31463
a5bad127128d branchmap: handle nullrev in setcachedata
Durham Goode <durham@fb.com>
parents: 31390
diff changeset
559 if rev == nullrev:
a5bad127128d branchmap: handle nullrev in setcachedata
Durham Goode <durham@fb.com>
parents: 31390
diff changeset
560 return
23785
cb99bacb9b4e branchcache: introduce revbranchcache for caching of revision branch names
Mads Kiilerich <madski@unity3d.com>
parents: 22357
diff changeset
561 rbcrevidx = rev * _rbcrecsize
29604
db0095c83344 rbc: fix invalid rbc-revs entries caused by missing cache growth
Mads Kiilerich <madski@unity3d.com>
parents: 29423
diff changeset
562 if len(self._rbcrevs) < rbcrevidx + _rbcrecsize:
db0095c83344 rbc: fix invalid rbc-revs entries caused by missing cache growth
Mads Kiilerich <madski@unity3d.com>
parents: 29423
diff changeset
563 self._rbcrevs.extend('\0' *
40494
5e5c8f2a1eb5 branchmap: do not specify changelog as an argument
Yuya Nishihara <yuya@tcha.org>
parents: 40375
diff changeset
564 (len(self._repo.changelog) * _rbcrecsize -
29604
db0095c83344 rbc: fix invalid rbc-revs entries caused by missing cache growth
Mads Kiilerich <madski@unity3d.com>
parents: 29423
diff changeset
565 len(self._rbcrevs)))
31379
906be86990c4 rbc: use struct unpack_from and pack_into instead of unpack and pack
Mads Kiilerich <madski@unity3d.com>
parents: 31369
diff changeset
566 pack_into(_rbcrecfmt, self._rbcrevs, rbcrevidx, node, branchidx)
24376
203a078da052 revbranchcache: populate cache incrementally
Durham Goode <durham@fb.com>
parents: 24375
diff changeset
567 self._rbcrevslen = min(self._rbcrevslen, rev)
23785
cb99bacb9b4e branchcache: introduce revbranchcache for caching of revision branch names
Mads Kiilerich <madski@unity3d.com>
parents: 22357
diff changeset
568
24377
656f93ce66d5 revbranchcache: move cache writing to the transaction finalizer
Durham Goode <durham@fb.com>
parents: 24376
diff changeset
569 tr = self._repo.currenttransaction()
656f93ce66d5 revbranchcache: move cache writing to the transaction finalizer
Durham Goode <durham@fb.com>
parents: 24376
diff changeset
570 if tr:
656f93ce66d5 revbranchcache: move cache writing to the transaction finalizer
Durham Goode <durham@fb.com>
parents: 24376
diff changeset
571 tr.addfinalize('write-revbranchcache', self.write)
656f93ce66d5 revbranchcache: move cache writing to the transaction finalizer
Durham Goode <durham@fb.com>
parents: 24376
diff changeset
572
656f93ce66d5 revbranchcache: move cache writing to the transaction finalizer
Durham Goode <durham@fb.com>
parents: 24376
diff changeset
573 def write(self, tr=None):
23785
cb99bacb9b4e branchcache: introduce revbranchcache for caching of revision branch names
Mads Kiilerich <madski@unity3d.com>
parents: 22357
diff changeset
574 """Save branch cache if it is dirty."""
24374
77fd1fb538cd revbranchcache: store repo on the object
Durham Goode <durham@fb.com>
parents: 24373
diff changeset
575 repo = self._repo
29756
0d588332ad2c branchmap: acquires lock before writting the rev branch cache
Pierre-Yves David <pierre-yves.david@ens-lyon.org>
parents: 29755
diff changeset
576 wlock = None
29757
3b184adfb5be branchmap: simplify error handlind when writing rev branch cache
Pierre-Yves David <pierre-yves.david@ens-lyon.org>
parents: 29756
diff changeset
577 step = ''
29756
0d588332ad2c branchmap: acquires lock before writting the rev branch cache
Pierre-Yves David <pierre-yves.david@ens-lyon.org>
parents: 29755
diff changeset
578 try:
29755
9f3c49ee4486 branchmap: preparatory indent of indent the branch rev writing code
Pierre-Yves David <pierre-yves.david@ens-lyon.org>
parents: 29615
diff changeset
579 if self._rbcnamescount < len(self._names):
29757
3b184adfb5be branchmap: simplify error handlind when writing rev branch cache
Pierre-Yves David <pierre-yves.david@ens-lyon.org>
parents: 29756
diff changeset
580 step = ' names'
29756
0d588332ad2c branchmap: acquires lock before writting the rev branch cache
Pierre-Yves David <pierre-yves.david@ens-lyon.org>
parents: 29755
diff changeset
581 wlock = repo.wlock(wait=False)
29758
3dbc95f3eb31 branchmap: remove extra indent
Pierre-Yves David <pierre-yves.david@ens-lyon.org>
parents: 29757
diff changeset
582 if self._rbcnamescount != 0:
33535
755e6532e81d cachevfs: migration the revbranchcache to 'cachevfs'
Boris Feld <boris.feld@octobus.net>
parents: 33534
diff changeset
583 f = repo.cachevfs.open(_rbcnames, 'ab')
29758
3dbc95f3eb31 branchmap: remove extra indent
Pierre-Yves David <pierre-yves.david@ens-lyon.org>
parents: 29757
diff changeset
584 if f.tell() == self._rbcsnameslen:
3dbc95f3eb31 branchmap: remove extra indent
Pierre-Yves David <pierre-yves.david@ens-lyon.org>
parents: 29757
diff changeset
585 f.write('\0')
3dbc95f3eb31 branchmap: remove extra indent
Pierre-Yves David <pierre-yves.david@ens-lyon.org>
parents: 29757
diff changeset
586 else:
3dbc95f3eb31 branchmap: remove extra indent
Pierre-Yves David <pierre-yves.david@ens-lyon.org>
parents: 29757
diff changeset
587 f.close()
3dbc95f3eb31 branchmap: remove extra indent
Pierre-Yves David <pierre-yves.david@ens-lyon.org>
parents: 29757
diff changeset
588 repo.ui.debug("%s changed - rewriting it\n" % _rbcnames)
3dbc95f3eb31 branchmap: remove extra indent
Pierre-Yves David <pierre-yves.david@ens-lyon.org>
parents: 29757
diff changeset
589 self._rbcnamescount = 0
3dbc95f3eb31 branchmap: remove extra indent
Pierre-Yves David <pierre-yves.david@ens-lyon.org>
parents: 29757
diff changeset
590 self._rbcrevslen = 0
3dbc95f3eb31 branchmap: remove extra indent
Pierre-Yves David <pierre-yves.david@ens-lyon.org>
parents: 29757
diff changeset
591 if self._rbcnamescount == 0:
3dbc95f3eb31 branchmap: remove extra indent
Pierre-Yves David <pierre-yves.david@ens-lyon.org>
parents: 29757
diff changeset
592 # before rewriting names, make sure references are removed
33535
755e6532e81d cachevfs: migration the revbranchcache to 'cachevfs'
Boris Feld <boris.feld@octobus.net>
parents: 33534
diff changeset
593 repo.cachevfs.unlinkpath(_rbcrevs, ignoremissing=True)
755e6532e81d cachevfs: migration the revbranchcache to 'cachevfs'
Boris Feld <boris.feld@octobus.net>
parents: 33534
diff changeset
594 f = repo.cachevfs.open(_rbcnames, 'wb')
29758
3dbc95f3eb31 branchmap: remove extra indent
Pierre-Yves David <pierre-yves.david@ens-lyon.org>
parents: 29757
diff changeset
595 f.write('\0'.join(encoding.fromlocal(b)
3dbc95f3eb31 branchmap: remove extra indent
Pierre-Yves David <pierre-yves.david@ens-lyon.org>
parents: 29757
diff changeset
596 for b in self._names[self._rbcnamescount:]))
3dbc95f3eb31 branchmap: remove extra indent
Pierre-Yves David <pierre-yves.david@ens-lyon.org>
parents: 29757
diff changeset
597 self._rbcsnameslen = f.tell()
3dbc95f3eb31 branchmap: remove extra indent
Pierre-Yves David <pierre-yves.david@ens-lyon.org>
parents: 29757
diff changeset
598 f.close()
29755
9f3c49ee4486 branchmap: preparatory indent of indent the branch rev writing code
Pierre-Yves David <pierre-yves.david@ens-lyon.org>
parents: 29615
diff changeset
599 self._rbcnamescount = len(self._names)
23785
cb99bacb9b4e branchcache: introduce revbranchcache for caching of revision branch names
Mads Kiilerich <madski@unity3d.com>
parents: 22357
diff changeset
600
29755
9f3c49ee4486 branchmap: preparatory indent of indent the branch rev writing code
Pierre-Yves David <pierre-yves.david@ens-lyon.org>
parents: 29615
diff changeset
601 start = self._rbcrevslen * _rbcrecsize
9f3c49ee4486 branchmap: preparatory indent of indent the branch rev writing code
Pierre-Yves David <pierre-yves.david@ens-lyon.org>
parents: 29615
diff changeset
602 if start != len(self._rbcrevs):
29757
3b184adfb5be branchmap: simplify error handlind when writing rev branch cache
Pierre-Yves David <pierre-yves.david@ens-lyon.org>
parents: 29756
diff changeset
603 step = ''
29756
0d588332ad2c branchmap: acquires lock before writting the rev branch cache
Pierre-Yves David <pierre-yves.david@ens-lyon.org>
parents: 29755
diff changeset
604 if wlock is None:
0d588332ad2c branchmap: acquires lock before writting the rev branch cache
Pierre-Yves David <pierre-yves.david@ens-lyon.org>
parents: 29755
diff changeset
605 wlock = repo.wlock(wait=False)
29755
9f3c49ee4486 branchmap: preparatory indent of indent the branch rev writing code
Pierre-Yves David <pierre-yves.david@ens-lyon.org>
parents: 29615
diff changeset
606 revs = min(len(repo.changelog),
9f3c49ee4486 branchmap: preparatory indent of indent the branch rev writing code
Pierre-Yves David <pierre-yves.david@ens-lyon.org>
parents: 29615
diff changeset
607 len(self._rbcrevs) // _rbcrecsize)
33535
755e6532e81d cachevfs: migration the revbranchcache to 'cachevfs'
Boris Feld <boris.feld@octobus.net>
parents: 33534
diff changeset
608 f = repo.cachevfs.open(_rbcrevs, 'ab')
29758
3dbc95f3eb31 branchmap: remove extra indent
Pierre-Yves David <pierre-yves.david@ens-lyon.org>
parents: 29757
diff changeset
609 if f.tell() != start:
33535
755e6532e81d cachevfs: migration the revbranchcache to 'cachevfs'
Boris Feld <boris.feld@octobus.net>
parents: 33534
diff changeset
610 repo.ui.debug("truncating cache/%s to %d\n"
755e6532e81d cachevfs: migration the revbranchcache to 'cachevfs'
Boris Feld <boris.feld@octobus.net>
parents: 33534
diff changeset
611 % (_rbcrevs, start))
29758
3dbc95f3eb31 branchmap: remove extra indent
Pierre-Yves David <pierre-yves.david@ens-lyon.org>
parents: 29757
diff changeset
612 f.seek(start)
28557
4b83507bfc91 cache: safer handling of failing seek when writing revision branch cache
Mads Kiilerich <madski@unity3d.com>
parents: 28556
diff changeset
613 if f.tell() != start:
29758
3dbc95f3eb31 branchmap: remove extra indent
Pierre-Yves David <pierre-yves.david@ens-lyon.org>
parents: 29757
diff changeset
614 start = 0
28557
4b83507bfc91 cache: safer handling of failing seek when writing revision branch cache
Mads Kiilerich <madski@unity3d.com>
parents: 28556
diff changeset
615 f.seek(start)
29758
3dbc95f3eb31 branchmap: remove extra indent
Pierre-Yves David <pierre-yves.david@ens-lyon.org>
parents: 29757
diff changeset
616 f.truncate()
3dbc95f3eb31 branchmap: remove extra indent
Pierre-Yves David <pierre-yves.david@ens-lyon.org>
parents: 29757
diff changeset
617 end = revs * _rbcrecsize
3dbc95f3eb31 branchmap: remove extra indent
Pierre-Yves David <pierre-yves.david@ens-lyon.org>
parents: 29757
diff changeset
618 f.write(self._rbcrevs[start:end])
3dbc95f3eb31 branchmap: remove extra indent
Pierre-Yves David <pierre-yves.david@ens-lyon.org>
parents: 29757
diff changeset
619 f.close()
29755
9f3c49ee4486 branchmap: preparatory indent of indent the branch rev writing code
Pierre-Yves David <pierre-yves.david@ens-lyon.org>
parents: 29615
diff changeset
620 self._rbcrevslen = revs
29757
3b184adfb5be branchmap: simplify error handlind when writing rev branch cache
Pierre-Yves David <pierre-yves.david@ens-lyon.org>
parents: 29756
diff changeset
621 except (IOError, OSError, error.Abort, error.LockError) as inst:
3b184adfb5be branchmap: simplify error handlind when writing rev branch cache
Pierre-Yves David <pierre-yves.david@ens-lyon.org>
parents: 29756
diff changeset
622 repo.ui.debug("couldn't write revision branch cache%s: %s\n"
37087
f0b6fbea00cf stringutil: bulk-replace call sites to point to new module
Yuya Nishihara <yuya@tcha.org>
parents: 36968
diff changeset
623 % (step, stringutil.forcebytestr(inst)))
29756
0d588332ad2c branchmap: acquires lock before writting the rev branch cache
Pierre-Yves David <pierre-yves.david@ens-lyon.org>
parents: 29755
diff changeset
624 finally:
0d588332ad2c branchmap: acquires lock before writting the rev branch cache
Pierre-Yves David <pierre-yves.david@ens-lyon.org>
parents: 29755
diff changeset
625 if wlock is not None:
0d588332ad2c branchmap: acquires lock before writting the rev branch cache
Pierre-Yves David <pierre-yves.david@ens-lyon.org>
parents: 29755
diff changeset
626 wlock.release()