Mercurial > public > mercurial-scm > hg-stable

annotate mercurial/branching/rev_cache.py @ 51944:bd7359c18d69

Find changesets by keywords (author, files, the commit message), revision number or hash, or revset expression.
rev-branch-cache: fallback on "v1" data if no v2 is found This will help smooth the transition to the v2 format for existing large repository.
author Pierre-Yves David <pierre-yves.david@octobus.net>
date Tue, 24 Sep 2024 15:44:10 +0200
parents 0f26ee69cf36
children 16efed18ae4e
Ignore whitespace changes - Everywhere: Within whitespace: At end of lines:
rev   line source
51938
f0e07efc199f rev-branch-cache: move the code in a dedicated module
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 51901
diff changeset
1 # rev_cache.py - caching branch information per revision
18116
bcee63733aad branchmap: create a mercurial.branchmap module
Pierre-Yves David <pierre-yves.david@ens-lyon.org>
parents:
diff changeset
2 #
bcee63733aad branchmap: create a mercurial.branchmap module
Pierre-Yves David <pierre-yves.david@ens-lyon.org>
parents:
diff changeset
3 # This software may be used and distributed according to the terms of the
bcee63733aad branchmap: create a mercurial.branchmap module
Pierre-Yves David <pierre-yves.david@ens-lyon.org>
parents:
diff changeset
4 # GNU General Public License version 2 or any later version.
51901
f4733654f144 typing: add `from __future__ import annotations` to most files
Matt Harbison <matt_harbison@yahoo.com>
parents: 51651
diff changeset
5 from __future__ import annotations
25918
47f36e050c2e branchmap: use absolute_import
Gregory Szorc <gregory.szorc@gmail.com>
parents: 25660
diff changeset
6
51942
c564be351754 rev-branch-cache: stop truncating cache file
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 51941
diff changeset
7 import os
25918
47f36e050c2e branchmap: use absolute_import
Gregory Szorc <gregory.szorc@gmail.com>
parents: 25660
diff changeset
8 import struct
47f36e050c2e branchmap: use absolute_import
Gregory Szorc <gregory.szorc@gmail.com>
parents: 25660
diff changeset
9
51938
f0e07efc199f rev-branch-cache: move the code in a dedicated module
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 51901
diff changeset
10 from ..node import (
25918
47f36e050c2e branchmap: use absolute_import
Gregory Szorc <gregory.szorc@gmail.com>
parents: 25660
diff changeset
11 nullrev,
47f36e050c2e branchmap: use absolute_import
Gregory Szorc <gregory.szorc@gmail.com>
parents: 25660
diff changeset
12 )
51302
9d3721552b6c pytype: import typing directly
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 49308
diff changeset
13
51938
f0e07efc199f rev-branch-cache: move the code in a dedicated module
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 51901
diff changeset
14 from .. import (
25918
47f36e050c2e branchmap: use absolute_import
Gregory Szorc <gregory.szorc@gmail.com>
parents: 25660
diff changeset
15 encoding,
26587
56b2bcea2529 error: get Abort from 'error' instead of 'util'
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 26460
diff changeset
16 error,
30995
22fbca1d11ed mercurial: switch to util.timer for all interval timings
Simon Farnsworth <simonfar@fb.com>
parents: 29758
diff changeset
17 util,
25918
47f36e050c2e branchmap: use absolute_import
Gregory Szorc <gregory.szorc@gmail.com>
parents: 25660
diff changeset
18 )
51302
9d3721552b6c pytype: import typing directly
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 49308
diff changeset
19
51938
f0e07efc199f rev-branch-cache: move the code in a dedicated module
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 51901
diff changeset
20 from ..utils import (
37087
f0b6fbea00cf stringutil: bulk-replace call sites to point to new module
Yuya Nishihara <yuya@tcha.org>
parents: 36968
diff changeset
21 stringutil,
f0b6fbea00cf stringutil: bulk-replace call sites to point to new module
Yuya Nishihara <yuya@tcha.org>
parents: 36968
diff changeset
22 )
25918
47f36e050c2e branchmap: use absolute_import
Gregory Szorc <gregory.szorc@gmail.com>
parents: 25660
diff changeset
23
47f36e050c2e branchmap: use absolute_import
Gregory Szorc <gregory.szorc@gmail.com>
parents: 25660
diff changeset
24 calcsize = struct.calcsize
31379
906be86990c4 rbc: use struct unpack_from and pack_into instead of unpack and pack
Mads Kiilerich <madski@unity3d.com>
parents: 31369
diff changeset
25 pack_into = struct.pack_into
906be86990c4 rbc: use struct unpack_from and pack_into instead of unpack and pack
Mads Kiilerich <madski@unity3d.com>
parents: 31369
diff changeset
26 unpack_from = struct.unpack_from
18117
526e7ec5c96e branchmap: extract write logic from localrepo
Pierre-Yves David <pierre-yves.david@ens-lyon.org>
parents: 18116
diff changeset
27
18118
e70ff1e599f4 branchmap: extract read logic from repo
Pierre-Yves David <pierre-yves.david@logilab.fr>
parents: 18117
diff changeset
28
23785
cb99bacb9b4e branchcache: introduce revbranchcache for caching of revision branch names
Mads Kiilerich <madski@unity3d.com>
parents: 22357
diff changeset
29 # Revision branch info cache
cb99bacb9b4e branchcache: introduce revbranchcache for caching of revision branch names
Mads Kiilerich <madski@unity3d.com>
parents: 22357
diff changeset
30
51943
0f26ee69cf36 rev-branch-cache: increment the version to "v2"
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 51942
diff changeset
31 # The "V2" version use the same format as the "V1" but garantee it won't be
0f26ee69cf36 rev-branch-cache: increment the version to "v2"
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 51942
diff changeset
32 # truncated, preventing SIGBUS when it is mmap-ed
0f26ee69cf36 rev-branch-cache: increment the version to "v2"
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 51942
diff changeset
33 _rbcversion = b'-v2'
43077
687b865b95ad formatting: byteify all mercurial/ and hgext/ string literals
Augie Fackler <augie@google.com>
parents: 43076
diff changeset
34 _rbcnames = b'rbc-names' + _rbcversion
687b865b95ad formatting: byteify all mercurial/ and hgext/ string literals
Augie Fackler <augie@google.com>
parents: 43076
diff changeset
35 _rbcrevs = b'rbc-revs' + _rbcversion
51944
bd7359c18d69 rev-branch-cache: fallback on "v1" data if no v2 is found
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 51943
diff changeset
36 _rbc_legacy_version = b'-v1'
bd7359c18d69 rev-branch-cache: fallback on "v1" data if no v2 is found
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 51943
diff changeset
37 _rbc_legacy_names = b'rbc-names' + _rbc_legacy_version
bd7359c18d69 rev-branch-cache: fallback on "v1" data if no v2 is found
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 51943
diff changeset
38 _rbc_legacy_revs = b'rbc-revs' + _rbc_legacy_version
23785
cb99bacb9b4e branchcache: introduce revbranchcache for caching of revision branch names
Mads Kiilerich <madski@unity3d.com>
parents: 22357
diff changeset
39 # [4 byte hash prefix][4 byte branch name number with sign bit indicating open]
43077
687b865b95ad formatting: byteify all mercurial/ and hgext/ string literals
Augie Fackler <augie@google.com>
parents: 43076
diff changeset
40 _rbcrecfmt = b'>4sI'
23785
cb99bacb9b4e branchcache: introduce revbranchcache for caching of revision branch names
Mads Kiilerich <madski@unity3d.com>
parents: 22357
diff changeset
41 _rbcrecsize = calcsize(_rbcrecfmt)
46434
1726a53a8494 reverse-branch-cache: switch to doubling allocating scheme
Joerg Sonnenberger <joerg@bec.de>
parents: 46254
diff changeset
42 _rbcmininc = 64 * _rbcrecsize
23785
cb99bacb9b4e branchcache: introduce revbranchcache for caching of revision branch names
Mads Kiilerich <madski@unity3d.com>
parents: 22357
diff changeset
43 _rbcnodelen = 4
43076
2372284d9457 formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents: 42603
diff changeset
44 _rbcbranchidxmask = 0x7FFFFFFF
23785
cb99bacb9b4e branchcache: introduce revbranchcache for caching of revision branch names
Mads Kiilerich <madski@unity3d.com>
parents: 22357
diff changeset
45 _rbccloseflag = 0x80000000
cb99bacb9b4e branchcache: introduce revbranchcache for caching of revision branch names
Mads Kiilerich <madski@unity3d.com>
parents: 22357
diff changeset
46
43076
2372284d9457 formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents: 42603
diff changeset
47
51942
c564be351754 rev-branch-cache: stop truncating cache file
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 51941
diff changeset
48 # with atomic replacement.
c564be351754 rev-branch-cache: stop truncating cache file
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 51941
diff changeset
49 REWRITE_RATIO = 0.2
c564be351754 rev-branch-cache: stop truncating cache file
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 51941
diff changeset
50
c564be351754 rev-branch-cache: stop truncating cache file
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 51941
diff changeset
51
51377
02e7d79edf62 branchmap: use mmap for faster revbranchcache loading
Arseniy Alekseyev <aalekseyev@janestreet.com>
parents: 51307
diff changeset
52 class rbcrevs:
02e7d79edf62 branchmap: use mmap for faster revbranchcache loading
Arseniy Alekseyev <aalekseyev@janestreet.com>
parents: 51307
diff changeset
53 """a byte string consisting of an immutable prefix followed by a mutable suffix"""
02e7d79edf62 branchmap: use mmap for faster revbranchcache loading
Arseniy Alekseyev <aalekseyev@janestreet.com>
parents: 51307
diff changeset
54
02e7d79edf62 branchmap: use mmap for faster revbranchcache loading
Arseniy Alekseyev <aalekseyev@janestreet.com>
parents: 51307
diff changeset
55 def __init__(self, revs):
02e7d79edf62 branchmap: use mmap for faster revbranchcache loading
Arseniy Alekseyev <aalekseyev@janestreet.com>
parents: 51307
diff changeset
56 self._prefix = revs
02e7d79edf62 branchmap: use mmap for faster revbranchcache loading
Arseniy Alekseyev <aalekseyev@janestreet.com>
parents: 51307
diff changeset
57 self._rest = bytearray()
02e7d79edf62 branchmap: use mmap for faster revbranchcache loading
Arseniy Alekseyev <aalekseyev@janestreet.com>
parents: 51307
diff changeset
58
02e7d79edf62 branchmap: use mmap for faster revbranchcache loading
Arseniy Alekseyev <aalekseyev@janestreet.com>
parents: 51307
diff changeset
59 def __len__(self):
02e7d79edf62 branchmap: use mmap for faster revbranchcache loading
Arseniy Alekseyev <aalekseyev@janestreet.com>
parents: 51307
diff changeset
60 return len(self._prefix) + len(self._rest)
02e7d79edf62 branchmap: use mmap for faster revbranchcache loading
Arseniy Alekseyev <aalekseyev@janestreet.com>
parents: 51307
diff changeset
61
02e7d79edf62 branchmap: use mmap for faster revbranchcache loading
Arseniy Alekseyev <aalekseyev@janestreet.com>
parents: 51307
diff changeset
62 def unpack_record(self, rbcrevidx):
02e7d79edf62 branchmap: use mmap for faster revbranchcache loading
Arseniy Alekseyev <aalekseyev@janestreet.com>
parents: 51307
diff changeset
63 if rbcrevidx < len(self._prefix):
02e7d79edf62 branchmap: use mmap for faster revbranchcache loading
Arseniy Alekseyev <aalekseyev@janestreet.com>
parents: 51307
diff changeset
64 return unpack_from(_rbcrecfmt, util.buffer(self._prefix), rbcrevidx)
02e7d79edf62 branchmap: use mmap for faster revbranchcache loading
Arseniy Alekseyev <aalekseyev@janestreet.com>
parents: 51307
diff changeset
65 else:
02e7d79edf62 branchmap: use mmap for faster revbranchcache loading
Arseniy Alekseyev <aalekseyev@janestreet.com>
parents: 51307
diff changeset
66 return unpack_from(
02e7d79edf62 branchmap: use mmap for faster revbranchcache loading
Arseniy Alekseyev <aalekseyev@janestreet.com>
parents: 51307
diff changeset
67 _rbcrecfmt,
02e7d79edf62 branchmap: use mmap for faster revbranchcache loading
Arseniy Alekseyev <aalekseyev@janestreet.com>
parents: 51307
diff changeset
68 util.buffer(self._rest),
02e7d79edf62 branchmap: use mmap for faster revbranchcache loading
Arseniy Alekseyev <aalekseyev@janestreet.com>
parents: 51307
diff changeset
69 rbcrevidx - len(self._prefix),
02e7d79edf62 branchmap: use mmap for faster revbranchcache loading
Arseniy Alekseyev <aalekseyev@janestreet.com>
parents: 51307
diff changeset
70 )
02e7d79edf62 branchmap: use mmap for faster revbranchcache loading
Arseniy Alekseyev <aalekseyev@janestreet.com>
parents: 51307
diff changeset
71
02e7d79edf62 branchmap: use mmap for faster revbranchcache loading
Arseniy Alekseyev <aalekseyev@janestreet.com>
parents: 51307
diff changeset
72 def make_mutable(self):
02e7d79edf62 branchmap: use mmap for faster revbranchcache loading
Arseniy Alekseyev <aalekseyev@janestreet.com>
parents: 51307
diff changeset
73 if len(self._prefix) > 0:
02e7d79edf62 branchmap: use mmap for faster revbranchcache loading
Arseniy Alekseyev <aalekseyev@janestreet.com>
parents: 51307
diff changeset
74 entirety = bytearray()
02e7d79edf62 branchmap: use mmap for faster revbranchcache loading
Arseniy Alekseyev <aalekseyev@janestreet.com>
parents: 51307
diff changeset
75 entirety[:] = self._prefix
02e7d79edf62 branchmap: use mmap for faster revbranchcache loading
Arseniy Alekseyev <aalekseyev@janestreet.com>
parents: 51307
diff changeset
76 entirety.extend(self._rest)
02e7d79edf62 branchmap: use mmap for faster revbranchcache loading
Arseniy Alekseyev <aalekseyev@janestreet.com>
parents: 51307
diff changeset
77 self._rest = entirety
02e7d79edf62 branchmap: use mmap for faster revbranchcache loading
Arseniy Alekseyev <aalekseyev@janestreet.com>
parents: 51307
diff changeset
78 self._prefix = bytearray()
02e7d79edf62 branchmap: use mmap for faster revbranchcache loading
Arseniy Alekseyev <aalekseyev@janestreet.com>
parents: 51307
diff changeset
79
02e7d79edf62 branchmap: use mmap for faster revbranchcache loading
Arseniy Alekseyev <aalekseyev@janestreet.com>
parents: 51307
diff changeset
80 def truncate(self, pos):
02e7d79edf62 branchmap: use mmap for faster revbranchcache loading
Arseniy Alekseyev <aalekseyev@janestreet.com>
parents: 51307
diff changeset
81 self.make_mutable()
02e7d79edf62 branchmap: use mmap for faster revbranchcache loading
Arseniy Alekseyev <aalekseyev@janestreet.com>
parents: 51307
diff changeset
82 del self._rest[pos:]
02e7d79edf62 branchmap: use mmap for faster revbranchcache loading
Arseniy Alekseyev <aalekseyev@janestreet.com>
parents: 51307
diff changeset
83
02e7d79edf62 branchmap: use mmap for faster revbranchcache loading
Arseniy Alekseyev <aalekseyev@janestreet.com>
parents: 51307
diff changeset
84 def pack_into(self, rbcrevidx, node, branchidx):
02e7d79edf62 branchmap: use mmap for faster revbranchcache loading
Arseniy Alekseyev <aalekseyev@janestreet.com>
parents: 51307
diff changeset
85 if rbcrevidx < len(self._prefix):
02e7d79edf62 branchmap: use mmap for faster revbranchcache loading
Arseniy Alekseyev <aalekseyev@janestreet.com>
parents: 51307
diff changeset
86 self.make_mutable()
02e7d79edf62 branchmap: use mmap for faster revbranchcache loading
Arseniy Alekseyev <aalekseyev@janestreet.com>
parents: 51307
diff changeset
87 buf = self._rest
02e7d79edf62 branchmap: use mmap for faster revbranchcache loading
Arseniy Alekseyev <aalekseyev@janestreet.com>
parents: 51307
diff changeset
88 start_offset = rbcrevidx - len(self._prefix)
02e7d79edf62 branchmap: use mmap for faster revbranchcache loading
Arseniy Alekseyev <aalekseyev@janestreet.com>
parents: 51307
diff changeset
89 end_offset = start_offset + _rbcrecsize
02e7d79edf62 branchmap: use mmap for faster revbranchcache loading
Arseniy Alekseyev <aalekseyev@janestreet.com>
parents: 51307
diff changeset
90
02e7d79edf62 branchmap: use mmap for faster revbranchcache loading
Arseniy Alekseyev <aalekseyev@janestreet.com>
parents: 51307
diff changeset
91 if len(self._rest) < end_offset:
02e7d79edf62 branchmap: use mmap for faster revbranchcache loading
Arseniy Alekseyev <aalekseyev@janestreet.com>
parents: 51307
diff changeset
92 # bytearray doesn't allocate extra space at least in Python 3.7.
02e7d79edf62 branchmap: use mmap for faster revbranchcache loading
Arseniy Alekseyev <aalekseyev@janestreet.com>
parents: 51307
diff changeset
93 # When multiple changesets are added in a row, precise resize would
02e7d79edf62 branchmap: use mmap for faster revbranchcache loading
Arseniy Alekseyev <aalekseyev@janestreet.com>
parents: 51307
diff changeset
94 # result in quadratic complexity. Overallocate to compensate by
02e7d79edf62 branchmap: use mmap for faster revbranchcache loading
Arseniy Alekseyev <aalekseyev@janestreet.com>
parents: 51307
diff changeset
95 # using the classic doubling technique for dynamic arrays instead.
02e7d79edf62 branchmap: use mmap for faster revbranchcache loading
Arseniy Alekseyev <aalekseyev@janestreet.com>
parents: 51307
diff changeset
96 # If there was a gap in the map before, less space will be reserved.
02e7d79edf62 branchmap: use mmap for faster revbranchcache loading
Arseniy Alekseyev <aalekseyev@janestreet.com>
parents: 51307
diff changeset
97 self._rest.extend(b'\0' * end_offset)
02e7d79edf62 branchmap: use mmap for faster revbranchcache loading
Arseniy Alekseyev <aalekseyev@janestreet.com>
parents: 51307
diff changeset
98 return pack_into(
02e7d79edf62 branchmap: use mmap for faster revbranchcache loading
Arseniy Alekseyev <aalekseyev@janestreet.com>
parents: 51307
diff changeset
99 _rbcrecfmt,
02e7d79edf62 branchmap: use mmap for faster revbranchcache loading
Arseniy Alekseyev <aalekseyev@janestreet.com>
parents: 51307
diff changeset
100 buf,
02e7d79edf62 branchmap: use mmap for faster revbranchcache loading
Arseniy Alekseyev <aalekseyev@janestreet.com>
parents: 51307
diff changeset
101 start_offset,
02e7d79edf62 branchmap: use mmap for faster revbranchcache loading
Arseniy Alekseyev <aalekseyev@janestreet.com>
parents: 51307
diff changeset
102 node,
02e7d79edf62 branchmap: use mmap for faster revbranchcache loading
Arseniy Alekseyev <aalekseyev@janestreet.com>
parents: 51307
diff changeset
103 branchidx,
02e7d79edf62 branchmap: use mmap for faster revbranchcache loading
Arseniy Alekseyev <aalekseyev@janestreet.com>
parents: 51307
diff changeset
104 )
02e7d79edf62 branchmap: use mmap for faster revbranchcache loading
Arseniy Alekseyev <aalekseyev@janestreet.com>
parents: 51307
diff changeset
105
02e7d79edf62 branchmap: use mmap for faster revbranchcache loading
Arseniy Alekseyev <aalekseyev@janestreet.com>
parents: 51307
diff changeset
106 def extend(self, extension):
02e7d79edf62 branchmap: use mmap for faster revbranchcache loading
Arseniy Alekseyev <aalekseyev@janestreet.com>
parents: 51307
diff changeset
107 return self._rest.extend(extension)
02e7d79edf62 branchmap: use mmap for faster revbranchcache loading
Arseniy Alekseyev <aalekseyev@janestreet.com>
parents: 51307
diff changeset
108
02e7d79edf62 branchmap: use mmap for faster revbranchcache loading
Arseniy Alekseyev <aalekseyev@janestreet.com>
parents: 51307
diff changeset
109 def slice(self, begin, end):
02e7d79edf62 branchmap: use mmap for faster revbranchcache loading
Arseniy Alekseyev <aalekseyev@janestreet.com>
parents: 51307
diff changeset
110 if begin < len(self._prefix):
02e7d79edf62 branchmap: use mmap for faster revbranchcache loading
Arseniy Alekseyev <aalekseyev@janestreet.com>
parents: 51307
diff changeset
111 acc = bytearray()
02e7d79edf62 branchmap: use mmap for faster revbranchcache loading
Arseniy Alekseyev <aalekseyev@janestreet.com>
parents: 51307
diff changeset
112 acc[:] = self._prefix[begin:end]
02e7d79edf62 branchmap: use mmap for faster revbranchcache loading
Arseniy Alekseyev <aalekseyev@janestreet.com>
parents: 51307
diff changeset
113 acc.extend(
02e7d79edf62 branchmap: use mmap for faster revbranchcache loading
Arseniy Alekseyev <aalekseyev@janestreet.com>
parents: 51307
diff changeset
114 self._rest[begin - len(self._prefix) : end - len(self._prefix)]
02e7d79edf62 branchmap: use mmap for faster revbranchcache loading
Arseniy Alekseyev <aalekseyev@janestreet.com>
parents: 51307
diff changeset
115 )
02e7d79edf62 branchmap: use mmap for faster revbranchcache loading
Arseniy Alekseyev <aalekseyev@janestreet.com>
parents: 51307
diff changeset
116 return acc
02e7d79edf62 branchmap: use mmap for faster revbranchcache loading
Arseniy Alekseyev <aalekseyev@janestreet.com>
parents: 51307
diff changeset
117 return self._rest[begin - len(self._prefix) : end - len(self._prefix)]
02e7d79edf62 branchmap: use mmap for faster revbranchcache loading
Arseniy Alekseyev <aalekseyev@janestreet.com>
parents: 51307
diff changeset
118
02e7d79edf62 branchmap: use mmap for faster revbranchcache loading
Arseniy Alekseyev <aalekseyev@janestreet.com>
parents: 51307
diff changeset
119
49037
642e31cb55f0 py3: use class X: instead of class X(object):
Gregory Szorc <gregory.szorc@gmail.com>
parents: 49026
diff changeset
120 class revbranchcache:
23785
cb99bacb9b4e branchcache: introduce revbranchcache for caching of revision branch names
Mads Kiilerich <madski@unity3d.com>
parents: 22357
diff changeset
121 """Persistent cache, mapping from revision number to branch name and close.
cb99bacb9b4e branchcache: introduce revbranchcache for caching of revision branch names
Mads Kiilerich <madski@unity3d.com>
parents: 22357
diff changeset
122 This is a low level cache, independent of filtering.
cb99bacb9b4e branchcache: introduce revbranchcache for caching of revision branch names
Mads Kiilerich <madski@unity3d.com>
parents: 22357
diff changeset
123
cb99bacb9b4e branchcache: introduce revbranchcache for caching of revision branch names
Mads Kiilerich <madski@unity3d.com>
parents: 22357
diff changeset
124 Branch names are stored in rbc-names in internal encoding separated by 0.
cb99bacb9b4e branchcache: introduce revbranchcache for caching of revision branch names
Mads Kiilerich <madski@unity3d.com>
parents: 22357
diff changeset
125 rbc-names is append-only, and each branch name is only stored once and will
cb99bacb9b4e branchcache: introduce revbranchcache for caching of revision branch names
Mads Kiilerich <madski@unity3d.com>
parents: 22357
diff changeset
126 thus have a unique index.
cb99bacb9b4e branchcache: introduce revbranchcache for caching of revision branch names
Mads Kiilerich <madski@unity3d.com>
parents: 22357
diff changeset
127
cb99bacb9b4e branchcache: introduce revbranchcache for caching of revision branch names
Mads Kiilerich <madski@unity3d.com>
parents: 22357
diff changeset
128 The branch info for each revision is stored in rbc-revs as constant size
cb99bacb9b4e branchcache: introduce revbranchcache for caching of revision branch names
Mads Kiilerich <madski@unity3d.com>
parents: 22357
diff changeset
129 records. The whole file is read into memory, but it is only 'parsed' on
cb99bacb9b4e branchcache: introduce revbranchcache for caching of revision branch names
Mads Kiilerich <madski@unity3d.com>
parents: 22357
diff changeset
130 demand. The file is usually append-only but will be truncated if repo
cb99bacb9b4e branchcache: introduce revbranchcache for caching of revision branch names
Mads Kiilerich <madski@unity3d.com>
parents: 22357
diff changeset
131 modification is detected.
cb99bacb9b4e branchcache: introduce revbranchcache for caching of revision branch names
Mads Kiilerich <madski@unity3d.com>
parents: 22357
diff changeset
132 The record for each revision contains the first 4 bytes of the
cb99bacb9b4e branchcache: introduce revbranchcache for caching of revision branch names
Mads Kiilerich <madski@unity3d.com>
parents: 22357
diff changeset
133 corresponding node hash, and the record is only used if it still matches.
cb99bacb9b4e branchcache: introduce revbranchcache for caching of revision branch names
Mads Kiilerich <madski@unity3d.com>
parents: 22357
diff changeset
134 Even a completely trashed rbc-revs fill thus still give the right result
cb99bacb9b4e branchcache: introduce revbranchcache for caching of revision branch names
Mads Kiilerich <madski@unity3d.com>
parents: 22357
diff changeset
135 while converging towards full recovery ... assuming no incorrectly matching
cb99bacb9b4e branchcache: introduce revbranchcache for caching of revision branch names
Mads Kiilerich <madski@unity3d.com>
parents: 22357
diff changeset
136 node hashes.
cb99bacb9b4e branchcache: introduce revbranchcache for caching of revision branch names
Mads Kiilerich <madski@unity3d.com>
parents: 22357
diff changeset
137 The record also contains 4 bytes where 31 bits contains the index of the
cb99bacb9b4e branchcache: introduce revbranchcache for caching of revision branch names
Mads Kiilerich <madski@unity3d.com>
parents: 22357
diff changeset
138 branch and the last bit indicate that it is a branch close commit.
cb99bacb9b4e branchcache: introduce revbranchcache for caching of revision branch names
Mads Kiilerich <madski@unity3d.com>
parents: 22357
diff changeset
139 The usage pattern for rbc-revs is thus somewhat similar to 00changelog.i
cb99bacb9b4e branchcache: introduce revbranchcache for caching of revision branch names
Mads Kiilerich <madski@unity3d.com>
parents: 22357
diff changeset
140 and will grow with it but be 1/8th of its size.
cb99bacb9b4e branchcache: introduce revbranchcache for caching of revision branch names
Mads Kiilerich <madski@unity3d.com>
parents: 22357
diff changeset
141 """
cb99bacb9b4e branchcache: introduce revbranchcache for caching of revision branch names
Mads Kiilerich <madski@unity3d.com>
parents: 22357
diff changeset
142
24159
5b4ed033390b revisionbranchcache: fall back to slow path if starting readonly (issue4531)
Mads Kiilerich <madski@unity3d.com>
parents: 23877
diff changeset
143 def __init__(self, repo, readonly=True):
23785
cb99bacb9b4e branchcache: introduce revbranchcache for caching of revision branch names
Mads Kiilerich <madski@unity3d.com>
parents: 22357
diff changeset
144 assert repo.filtername is None
24374
77fd1fb538cd revbranchcache: store repo on the object
Durham Goode <durham@fb.com>
parents: 24373
diff changeset
145 self._repo = repo
43076
2372284d9457 formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents: 42603
diff changeset
146 self._names = [] # branch names in local encoding with static index
51377
02e7d79edf62 branchmap: use mmap for faster revbranchcache loading
Arseniy Alekseyev <aalekseyev@janestreet.com>
parents: 51307
diff changeset
147 self._rbcrevs = rbcrevs(bytearray())
43076
2372284d9457 formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents: 42603
diff changeset
148 self._rbcsnameslen = 0 # length of names read at _rbcsnameslen
51944
bd7359c18d69 rev-branch-cache: fallback on "v1" data if no v2 is found
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 51943
diff changeset
149 v1_fallback = False
23785
cb99bacb9b4e branchcache: introduce revbranchcache for caching of revision branch names
Mads Kiilerich <madski@unity3d.com>
parents: 22357
diff changeset
150 try:
51944
bd7359c18d69 rev-branch-cache: fallback on "v1" data if no v2 is found
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 51943
diff changeset
151 try:
bd7359c18d69 rev-branch-cache: fallback on "v1" data if no v2 is found
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 51943
diff changeset
152 bndata = repo.cachevfs.read(_rbcnames)
bd7359c18d69 rev-branch-cache: fallback on "v1" data if no v2 is found
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 51943
diff changeset
153 except (IOError, OSError):
bd7359c18d69 rev-branch-cache: fallback on "v1" data if no v2 is found
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 51943
diff changeset
154 # If we don't have "v2" data, we might have "v1" data worth
bd7359c18d69 rev-branch-cache: fallback on "v1" data if no v2 is found
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 51943
diff changeset
155 # using.
bd7359c18d69 rev-branch-cache: fallback on "v1" data if no v2 is found
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 51943
diff changeset
156 #
bd7359c18d69 rev-branch-cache: fallback on "v1" data if no v2 is found
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 51943
diff changeset
157 # consider stop doing this many version after hg-6.9 release
bd7359c18d69 rev-branch-cache: fallback on "v1" data if no v2 is found
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 51943
diff changeset
158 bndata = repo.cachevfs.read(_rbc_legacy_names)
bd7359c18d69 rev-branch-cache: fallback on "v1" data if no v2 is found
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 51943
diff changeset
159 v1_fallback = True
43076
2372284d9457 formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents: 42603
diff changeset
160 self._rbcsnameslen = len(bndata) # for verification before writing
31380
7dd2f51f38ac rbc: empty (and invalid) rbc-names file should give an empty name list
Mads Kiilerich <mads@kiilerich.com>
parents: 31379
diff changeset
161 if bndata:
43076
2372284d9457 formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents: 42603
diff changeset
162 self._names = [
43077
687b865b95ad formatting: byteify all mercurial/ and hgext/ string literals
Augie Fackler <augie@google.com>
parents: 43076
diff changeset
163 encoding.tolocal(bn) for bn in bndata.split(b'\0')
43076
2372284d9457 formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents: 42603
diff changeset
164 ]
29423
d2c6f3a948fa branchmap: remove unused exception variable
Gregory Szorc <gregory.szorc@gmail.com>
parents: 28558
diff changeset
165 except (IOError, OSError):
24159
5b4ed033390b revisionbranchcache: fall back to slow path if starting readonly (issue4531)
Mads Kiilerich <madski@unity3d.com>
parents: 23877
diff changeset
166 if readonly:
5b4ed033390b revisionbranchcache: fall back to slow path if starting readonly (issue4531)
Mads Kiilerich <madski@unity3d.com>
parents: 23877
diff changeset
167 # don't try to use cache - fall back to the slow path
5b4ed033390b revisionbranchcache: fall back to slow path if starting readonly (issue4531)
Mads Kiilerich <madski@unity3d.com>
parents: 23877
diff changeset
168 self.branchinfo = self._branchinfo
5b4ed033390b revisionbranchcache: fall back to slow path if starting readonly (issue4531)
Mads Kiilerich <madski@unity3d.com>
parents: 23877
diff changeset
169
23785
cb99bacb9b4e branchcache: introduce revbranchcache for caching of revision branch names
Mads Kiilerich <madski@unity3d.com>
parents: 22357
diff changeset
170 if self._names:
cb99bacb9b4e branchcache: introduce revbranchcache for caching of revision branch names
Mads Kiilerich <madski@unity3d.com>
parents: 22357
diff changeset
171 try:
51651
c0e30a019ce1 mmap: only use mmap to read rev-branch-cache data if it is safe
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 51644
diff changeset
172 usemmap = repo.ui.configbool(b'storage', b'revbranchcache.mmap')
51944
bd7359c18d69 rev-branch-cache: fallback on "v1" data if no v2 is found
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 51943
diff changeset
173 if not v1_fallback:
bd7359c18d69 rev-branch-cache: fallback on "v1" data if no v2 is found
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 51943
diff changeset
174 with repo.cachevfs(_rbcrevs) as fp:
bd7359c18d69 rev-branch-cache: fallback on "v1" data if no v2 is found
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 51943
diff changeset
175 if usemmap and repo.cachevfs.is_mmap_safe(_rbcrevs):
bd7359c18d69 rev-branch-cache: fallback on "v1" data if no v2 is found
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 51943
diff changeset
176 data = util.buffer(util.mmapread(fp))
bd7359c18d69 rev-branch-cache: fallback on "v1" data if no v2 is found
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 51943
diff changeset
177 else:
bd7359c18d69 rev-branch-cache: fallback on "v1" data if no v2 is found
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 51943
diff changeset
178 data = fp.read()
bd7359c18d69 rev-branch-cache: fallback on "v1" data if no v2 is found
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 51943
diff changeset
179 else:
bd7359c18d69 rev-branch-cache: fallback on "v1" data if no v2 is found
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 51943
diff changeset
180 # If we don't have "v2" data, we might have "v1" data worth
bd7359c18d69 rev-branch-cache: fallback on "v1" data if no v2 is found
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 51943
diff changeset
181 # using.
bd7359c18d69 rev-branch-cache: fallback on "v1" data if no v2 is found
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 51943
diff changeset
182 #
bd7359c18d69 rev-branch-cache: fallback on "v1" data if no v2 is found
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 51943
diff changeset
183 # Consider stop doing this many version after hg-6.9
bd7359c18d69 rev-branch-cache: fallback on "v1" data if no v2 is found
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 51943
diff changeset
184 # release.
bd7359c18d69 rev-branch-cache: fallback on "v1" data if no v2 is found
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 51943
diff changeset
185 with repo.cachevfs(_rbc_legacy_revs) as fp:
51651
c0e30a019ce1 mmap: only use mmap to read rev-branch-cache data if it is safe
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 51644
diff changeset
186 data = fp.read()
51377
02e7d79edf62 branchmap: use mmap for faster revbranchcache loading
Arseniy Alekseyev <aalekseyev@janestreet.com>
parents: 51307
diff changeset
187 self._rbcrevs = rbcrevs(data)
25660
328739ea70c3 global: mass rewrite to use modern exception syntax
Gregory Szorc <gregory.szorc@gmail.com>
parents: 25266
diff changeset
188 except (IOError, OSError) as inst:
43076
2372284d9457 formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents: 42603
diff changeset
189 repo.ui.debug(
43077
687b865b95ad formatting: byteify all mercurial/ and hgext/ string literals
Augie Fackler <augie@google.com>
parents: 43076
diff changeset
190 b"couldn't read revision branch cache: %s\n"
43076
2372284d9457 formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents: 42603
diff changeset
191 % stringutil.forcebytestr(inst)
2372284d9457 formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents: 42603
diff changeset
192 )
23785
cb99bacb9b4e branchcache: introduce revbranchcache for caching of revision branch names
Mads Kiilerich <madski@unity3d.com>
parents: 22357
diff changeset
193 # remember number of good records on disk
43076
2372284d9457 formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents: 42603
diff changeset
194 self._rbcrevslen = min(
2372284d9457 formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents: 42603
diff changeset
195 len(self._rbcrevs) // _rbcrecsize, len(repo.changelog)
2372284d9457 formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents: 42603
diff changeset
196 )
23785
cb99bacb9b4e branchcache: introduce revbranchcache for caching of revision branch names
Mads Kiilerich <madski@unity3d.com>
parents: 22357
diff changeset
197 if self._rbcrevslen == 0:
cb99bacb9b4e branchcache: introduce revbranchcache for caching of revision branch names
Mads Kiilerich <madski@unity3d.com>
parents: 22357
diff changeset
198 self._names = []
43076
2372284d9457 formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents: 42603
diff changeset
199 self._rbcnamescount = len(self._names) # number of names read at
2372284d9457 formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents: 42603
diff changeset
200 # _rbcsnameslen
51940
9f7cf869e9f4 rev-branch-cache: add a way to force rewrite of the cache
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 51939
diff changeset
201 self._force_overwrite = False
23785
cb99bacb9b4e branchcache: introduce revbranchcache for caching of revision branch names
Mads Kiilerich <madski@unity3d.com>
parents: 22357
diff changeset
202
28558
bcd106d456c4 cache: rebuild branch cache from scratch when inconsistencies are detected
Mads Kiilerich <madski@unity3d.com>
parents: 28557
diff changeset
203 def _clear(self):
bcd106d456c4 cache: rebuild branch cache from scratch when inconsistencies are detected
Mads Kiilerich <madski@unity3d.com>
parents: 28557
diff changeset
204 self._rbcsnameslen = 0
bcd106d456c4 cache: rebuild branch cache from scratch when inconsistencies are detected
Mads Kiilerich <madski@unity3d.com>
parents: 28557
diff changeset
205 del self._names[:]
bcd106d456c4 cache: rebuild branch cache from scratch when inconsistencies are detected
Mads Kiilerich <madski@unity3d.com>
parents: 28557
diff changeset
206 self._rbcnamescount = 0
bcd106d456c4 cache: rebuild branch cache from scratch when inconsistencies are detected
Mads Kiilerich <madski@unity3d.com>
parents: 28557
diff changeset
207 self._rbcrevslen = len(self._repo.changelog)
51377
02e7d79edf62 branchmap: use mmap for faster revbranchcache loading
Arseniy Alekseyev <aalekseyev@janestreet.com>
parents: 51307
diff changeset
208 self._rbcrevs = rbcrevs(bytearray(self._rbcrevslen * _rbcrecsize))
43077
687b865b95ad formatting: byteify all mercurial/ and hgext/ string literals
Augie Fackler <augie@google.com>
parents: 43076
diff changeset
209 util.clearcachedproperty(self, b'_namesreverse')
51940
9f7cf869e9f4 rev-branch-cache: add a way to force rewrite of the cache
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 51939
diff changeset
210 self._force_overwrite = True
9f7cf869e9f4 rev-branch-cache: add a way to force rewrite of the cache
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 51939
diff changeset
211
9f7cf869e9f4 rev-branch-cache: add a way to force rewrite of the cache
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 51939
diff changeset
212 def invalidate(self, rev=0):
9f7cf869e9f4 rev-branch-cache: add a way to force rewrite of the cache
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 51939
diff changeset
213 self._rbcrevslen = rev
9f7cf869e9f4 rev-branch-cache: add a way to force rewrite of the cache
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 51939
diff changeset
214 self._rbcrevs.truncate(rev)
9f7cf869e9f4 rev-branch-cache: add a way to force rewrite of the cache
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 51939
diff changeset
215 self._force_overwrite = True
40749
50a64c321c1e branchmap: build the revbranchcache._namesreverse() only when required
Pulkit Goyal <pulkit@yandex-team.ru>
parents: 40494
diff changeset
216
50a64c321c1e branchmap: build the revbranchcache._namesreverse() only when required
Pulkit Goyal <pulkit@yandex-team.ru>
parents: 40494
diff changeset
217 @util.propertycache
50a64c321c1e branchmap: build the revbranchcache._namesreverse() only when required
Pulkit Goyal <pulkit@yandex-team.ru>
parents: 40494
diff changeset
218 def _namesreverse(self):
44470
9d2b2df2c2ba cleanup: run pyupgrade on our source tree to clean up varying things
Augie Fackler <augie@google.com>
parents: 44349
diff changeset
219 return {b: r for r, b in enumerate(self._names)}
28558
bcd106d456c4 cache: rebuild branch cache from scratch when inconsistencies are detected
Mads Kiilerich <madski@unity3d.com>
parents: 28557
diff changeset
220
40494
5e5c8f2a1eb5 branchmap: do not specify changelog as an argument
Yuya Nishihara <yuya@tcha.org>
parents: 40375
diff changeset
221 def branchinfo(self, rev):
23785
cb99bacb9b4e branchcache: introduce revbranchcache for caching of revision branch names
Mads Kiilerich <madski@unity3d.com>
parents: 22357
diff changeset
222 """Return branch name and close flag for rev, using and updating
cb99bacb9b4e branchcache: introduce revbranchcache for caching of revision branch names
Mads Kiilerich <madski@unity3d.com>
parents: 22357
diff changeset
223 persistent cache."""
40494
5e5c8f2a1eb5 branchmap: do not specify changelog as an argument
Yuya Nishihara <yuya@tcha.org>
parents: 40375
diff changeset
224 changelog = self._repo.changelog
23785
cb99bacb9b4e branchcache: introduce revbranchcache for caching of revision branch names
Mads Kiilerich <madski@unity3d.com>
parents: 22357
diff changeset
225 rbcrevidx = rev * _rbcrecsize
cb99bacb9b4e branchcache: introduce revbranchcache for caching of revision branch names
Mads Kiilerich <madski@unity3d.com>
parents: 22357
diff changeset
226
25266
38117278f295 revbranchcache: return uncached branchinfo for nullrev (issue4683)
Yuya Nishihara <yuya@tcha.org>
parents: 24728
diff changeset
227 # avoid negative index, changelog.read(nullrev) is fast without cache
38117278f295 revbranchcache: return uncached branchinfo for nullrev (issue4683)
Yuya Nishihara <yuya@tcha.org>
parents: 24728
diff changeset
228 if rev == nullrev:
38117278f295 revbranchcache: return uncached branchinfo for nullrev (issue4683)
Yuya Nishihara <yuya@tcha.org>
parents: 24728
diff changeset
229 return changelog.branchinfo(rev)
38117278f295 revbranchcache: return uncached branchinfo for nullrev (issue4683)
Yuya Nishihara <yuya@tcha.org>
parents: 24728
diff changeset
230
29604
db0095c83344 rbc: fix invalid rbc-revs entries caused by missing cache growth
Mads Kiilerich <madski@unity3d.com>
parents: 29423
diff changeset
231 # if requested rev isn't allocated, grow and cache the rev info
23785
cb99bacb9b4e branchcache: introduce revbranchcache for caching of revision branch names
Mads Kiilerich <madski@unity3d.com>
parents: 22357
diff changeset
232 if len(self._rbcrevs) < rbcrevidx + _rbcrecsize:
40494
5e5c8f2a1eb5 branchmap: do not specify changelog as an argument
Yuya Nishihara <yuya@tcha.org>
parents: 40375
diff changeset
233 return self._branchinfo(rev)
23785
cb99bacb9b4e branchcache: introduce revbranchcache for caching of revision branch names
Mads Kiilerich <madski@unity3d.com>
parents: 22357
diff changeset
234
cb99bacb9b4e branchcache: introduce revbranchcache for caching of revision branch names
Mads Kiilerich <madski@unity3d.com>
parents: 22357
diff changeset
235 # fast path: extract data from cache, use it if node is matching
cb99bacb9b4e branchcache: introduce revbranchcache for caching of revision branch names
Mads Kiilerich <madski@unity3d.com>
parents: 22357
diff changeset
236 reponode = changelog.node(rev)[:_rbcnodelen]
51377
02e7d79edf62 branchmap: use mmap for faster revbranchcache loading
Arseniy Alekseyev <aalekseyev@janestreet.com>
parents: 51307
diff changeset
237 cachenode, branchidx = self._rbcrevs.unpack_record(rbcrevidx)
23785
cb99bacb9b4e branchcache: introduce revbranchcache for caching of revision branch names
Mads Kiilerich <madski@unity3d.com>
parents: 22357
diff changeset
238 close = bool(branchidx & _rbccloseflag)
cb99bacb9b4e branchcache: introduce revbranchcache for caching of revision branch names
Mads Kiilerich <madski@unity3d.com>
parents: 22357
diff changeset
239 if close:
cb99bacb9b4e branchcache: introduce revbranchcache for caching of revision branch names
Mads Kiilerich <madski@unity3d.com>
parents: 22357
diff changeset
240 branchidx &= _rbcbranchidxmask
43077
687b865b95ad formatting: byteify all mercurial/ and hgext/ string literals
Augie Fackler <augie@google.com>
parents: 43076
diff changeset
241 if cachenode == b'\0\0\0\0':
24376
203a078da052 revbranchcache: populate cache incrementally
Durham Goode <durham@fb.com>
parents: 24375
diff changeset
242 pass
203a078da052 revbranchcache: populate cache incrementally
Durham Goode <durham@fb.com>
parents: 24375
diff changeset
243 elif cachenode == reponode:
29615
a2a380e2750f rbc: fix superfluous rebuilding from scratch - don't abuse self._rbcnamescount
Mads Kiilerich <madski@unity3d.com>
parents: 29604
diff changeset
244 try:
28558
bcd106d456c4 cache: rebuild branch cache from scratch when inconsistencies are detected
Mads Kiilerich <madski@unity3d.com>
parents: 28557
diff changeset
245 return self._names[branchidx], close
29615
a2a380e2750f rbc: fix superfluous rebuilding from scratch - don't abuse self._rbcnamescount
Mads Kiilerich <madski@unity3d.com>
parents: 29604
diff changeset
246 except IndexError:
a2a380e2750f rbc: fix superfluous rebuilding from scratch - don't abuse self._rbcnamescount
Mads Kiilerich <madski@unity3d.com>
parents: 29604
diff changeset
247 # recover from invalid reference to unknown branch
43076
2372284d9457 formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents: 42603
diff changeset
248 self._repo.ui.debug(
43077
687b865b95ad formatting: byteify all mercurial/ and hgext/ string literals
Augie Fackler <augie@google.com>
parents: 43076
diff changeset
249 b"referenced branch names not found"
687b865b95ad formatting: byteify all mercurial/ and hgext/ string literals
Augie Fackler <augie@google.com>
parents: 43076
diff changeset
250 b" - rebuilding revision branch cache from scratch\n"
43076
2372284d9457 formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents: 42603
diff changeset
251 )
29615
a2a380e2750f rbc: fix superfluous rebuilding from scratch - don't abuse self._rbcnamescount
Mads Kiilerich <madski@unity3d.com>
parents: 29604
diff changeset
252 self._clear()
24376
203a078da052 revbranchcache: populate cache incrementally
Durham Goode <durham@fb.com>
parents: 24375
diff changeset
253 else:
203a078da052 revbranchcache: populate cache incrementally
Durham Goode <durham@fb.com>
parents: 24375
diff changeset
254 # rev/node map has changed, invalidate the cache from here up
43076
2372284d9457 formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents: 42603
diff changeset
255 self._repo.ui.debug(
43077
687b865b95ad formatting: byteify all mercurial/ and hgext/ string literals
Augie Fackler <augie@google.com>
parents: 43076
diff changeset
256 b"history modification detected - truncating "
687b865b95ad formatting: byteify all mercurial/ and hgext/ string literals
Augie Fackler <augie@google.com>
parents: 43076
diff changeset
257 b"revision branch cache to revision %d\n" % rev
43076
2372284d9457 formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents: 42603
diff changeset
258 )
24376
203a078da052 revbranchcache: populate cache incrementally
Durham Goode <durham@fb.com>
parents: 24375
diff changeset
259 truncate = rbcrevidx + _rbcrecsize
51377
02e7d79edf62 branchmap: use mmap for faster revbranchcache loading
Arseniy Alekseyev <aalekseyev@janestreet.com>
parents: 51307
diff changeset
260 self._rbcrevs.truncate(truncate)
24376
203a078da052 revbranchcache: populate cache incrementally
Durham Goode <durham@fb.com>
parents: 24375
diff changeset
261 self._rbcrevslen = min(self._rbcrevslen, truncate)
203a078da052 revbranchcache: populate cache incrementally
Durham Goode <durham@fb.com>
parents: 24375
diff changeset
262
23785
cb99bacb9b4e branchcache: introduce revbranchcache for caching of revision branch names
Mads Kiilerich <madski@unity3d.com>
parents: 22357
diff changeset
263 # fall back to slow path and make sure it will be written to disk
40494
5e5c8f2a1eb5 branchmap: do not specify changelog as an argument
Yuya Nishihara <yuya@tcha.org>
parents: 40375
diff changeset
264 return self._branchinfo(rev)
23785
cb99bacb9b4e branchcache: introduce revbranchcache for caching of revision branch names
Mads Kiilerich <madski@unity3d.com>
parents: 22357
diff changeset
265
40494
5e5c8f2a1eb5 branchmap: do not specify changelog as an argument
Yuya Nishihara <yuya@tcha.org>
parents: 40375
diff changeset
266 def _branchinfo(self, rev):
23785
cb99bacb9b4e branchcache: introduce revbranchcache for caching of revision branch names
Mads Kiilerich <madski@unity3d.com>
parents: 22357
diff changeset
267 """Retrieve branch info from changelog and update _rbcrevs"""
40494
5e5c8f2a1eb5 branchmap: do not specify changelog as an argument
Yuya Nishihara <yuya@tcha.org>
parents: 40375
diff changeset
268 changelog = self._repo.changelog
23785
cb99bacb9b4e branchcache: introduce revbranchcache for caching of revision branch names
Mads Kiilerich <madski@unity3d.com>
parents: 22357
diff changeset
269 b, close = changelog.branchinfo(rev)
cb99bacb9b4e branchcache: introduce revbranchcache for caching of revision branch names
Mads Kiilerich <madski@unity3d.com>
parents: 22357
diff changeset
270 if b in self._namesreverse:
cb99bacb9b4e branchcache: introduce revbranchcache for caching of revision branch names
Mads Kiilerich <madski@unity3d.com>
parents: 22357
diff changeset
271 branchidx = self._namesreverse[b]
cb99bacb9b4e branchcache: introduce revbranchcache for caching of revision branch names
Mads Kiilerich <madski@unity3d.com>
parents: 22357
diff changeset
272 else:
cb99bacb9b4e branchcache: introduce revbranchcache for caching of revision branch names
Mads Kiilerich <madski@unity3d.com>
parents: 22357
diff changeset
273 branchidx = len(self._names)
cb99bacb9b4e branchcache: introduce revbranchcache for caching of revision branch names
Mads Kiilerich <madski@unity3d.com>
parents: 22357
diff changeset
274 self._names.append(b)
cb99bacb9b4e branchcache: introduce revbranchcache for caching of revision branch names
Mads Kiilerich <madski@unity3d.com>
parents: 22357
diff changeset
275 self._namesreverse[b] = branchidx
cb99bacb9b4e branchcache: introduce revbranchcache for caching of revision branch names
Mads Kiilerich <madski@unity3d.com>
parents: 22357
diff changeset
276 reponode = changelog.node(rev)
cb99bacb9b4e branchcache: introduce revbranchcache for caching of revision branch names
Mads Kiilerich <madski@unity3d.com>
parents: 22357
diff changeset
277 if close:
cb99bacb9b4e branchcache: introduce revbranchcache for caching of revision branch names
Mads Kiilerich <madski@unity3d.com>
parents: 22357
diff changeset
278 branchidx |= _rbccloseflag
40494
5e5c8f2a1eb5 branchmap: do not specify changelog as an argument
Yuya Nishihara <yuya@tcha.org>
parents: 40375
diff changeset
279 self._setcachedata(rev, reponode, branchidx)
24375
fe255b2525d5 revbranchcache: move entry writing to a separate function
Durham Goode <durham@fb.com>
parents: 24374
diff changeset
280 return b, close
fe255b2525d5 revbranchcache: move entry writing to a separate function
Durham Goode <durham@fb.com>
parents: 24374
diff changeset
281
46444
3e91d9978bec branchmap: update rev-branch-cache incrementally
Joerg Sonnenberger <joerg@bec.de>
parents: 46434
diff changeset
282 def setdata(self, rev, changelogrevision):
36968
95f4f1bfb650 revbranchcache: add a public function to update the data
Boris Feld <boris.feld@octobus.net>
parents: 36504
diff changeset
283 """add new data information to the cache"""
46444
3e91d9978bec branchmap: update rev-branch-cache incrementally
Joerg Sonnenberger <joerg@bec.de>
parents: 46434
diff changeset
284 branch, close = changelogrevision.branchinfo
3e91d9978bec branchmap: update rev-branch-cache incrementally
Joerg Sonnenberger <joerg@bec.de>
parents: 46434
diff changeset
285
36968
95f4f1bfb650 revbranchcache: add a public function to update the data
Boris Feld <boris.feld@octobus.net>
parents: 36504
diff changeset
286 if branch in self._namesreverse:
95f4f1bfb650 revbranchcache: add a public function to update the data
Boris Feld <boris.feld@octobus.net>
parents: 36504
diff changeset
287 branchidx = self._namesreverse[branch]
95f4f1bfb650 revbranchcache: add a public function to update the data
Boris Feld <boris.feld@octobus.net>
parents: 36504
diff changeset
288 else:
95f4f1bfb650 revbranchcache: add a public function to update the data
Boris Feld <boris.feld@octobus.net>
parents: 36504
diff changeset
289 branchidx = len(self._names)
95f4f1bfb650 revbranchcache: add a public function to update the data
Boris Feld <boris.feld@octobus.net>
parents: 36504
diff changeset
290 self._names.append(branch)
95f4f1bfb650 revbranchcache: add a public function to update the data
Boris Feld <boris.feld@octobus.net>
parents: 36504
diff changeset
291 self._namesreverse[branch] = branchidx
95f4f1bfb650 revbranchcache: add a public function to update the data
Boris Feld <boris.feld@octobus.net>
parents: 36504
diff changeset
292 if close:
95f4f1bfb650 revbranchcache: add a public function to update the data
Boris Feld <boris.feld@octobus.net>
parents: 36504
diff changeset
293 branchidx |= _rbccloseflag
46444
3e91d9978bec branchmap: update rev-branch-cache incrementally
Joerg Sonnenberger <joerg@bec.de>
parents: 46434
diff changeset
294 self._setcachedata(rev, self._repo.changelog.node(rev), branchidx)
36968
95f4f1bfb650 revbranchcache: add a public function to update the data
Boris Feld <boris.feld@octobus.net>
parents: 36504
diff changeset
295 # If no cache data were readable (non exists, bad permission, etc)
95f4f1bfb650 revbranchcache: add a public function to update the data
Boris Feld <boris.feld@octobus.net>
parents: 36504
diff changeset
296 # the cache was bypassing itself by setting:
95f4f1bfb650 revbranchcache: add a public function to update the data
Boris Feld <boris.feld@octobus.net>
parents: 36504
diff changeset
297 #
95f4f1bfb650 revbranchcache: add a public function to update the data
Boris Feld <boris.feld@octobus.net>
parents: 36504
diff changeset
298 # self.branchinfo = self._branchinfo
95f4f1bfb650 revbranchcache: add a public function to update the data
Boris Feld <boris.feld@octobus.net>
parents: 36504
diff changeset
299 #
95f4f1bfb650 revbranchcache: add a public function to update the data
Boris Feld <boris.feld@octobus.net>
parents: 36504
diff changeset
300 # Since we now have data in the cache, we need to drop this bypassing.
43554
9f70512ae2cf cleanup: remove pointless r-prefixes on single-quoted strings
Augie Fackler <augie@google.com>
parents: 43547
diff changeset
301 if 'branchinfo' in vars(self):
36968
95f4f1bfb650 revbranchcache: add a public function to update the data
Boris Feld <boris.feld@octobus.net>
parents: 36504
diff changeset
302 del self.branchinfo
95f4f1bfb650 revbranchcache: add a public function to update the data
Boris Feld <boris.feld@octobus.net>
parents: 36504
diff changeset
303
40494
5e5c8f2a1eb5 branchmap: do not specify changelog as an argument
Yuya Nishihara <yuya@tcha.org>
parents: 40375
diff changeset
304 def _setcachedata(self, rev, node, branchidx):
24375
fe255b2525d5 revbranchcache: move entry writing to a separate function
Durham Goode <durham@fb.com>
parents: 24374
diff changeset
305 """Writes the node's branch data to the in-memory cache data."""
31463
a5bad127128d branchmap: handle nullrev in setcachedata
Durham Goode <durham@fb.com>
parents: 31390
diff changeset
306 if rev == nullrev:
a5bad127128d branchmap: handle nullrev in setcachedata
Durham Goode <durham@fb.com>
parents: 31390
diff changeset
307 return
23785
cb99bacb9b4e branchcache: introduce revbranchcache for caching of revision branch names
Mads Kiilerich <madski@unity3d.com>
parents: 22357
diff changeset
308 rbcrevidx = rev * _rbcrecsize
51377
02e7d79edf62 branchmap: use mmap for faster revbranchcache loading
Arseniy Alekseyev <aalekseyev@janestreet.com>
parents: 51307
diff changeset
309 self._rbcrevs.pack_into(rbcrevidx, node, branchidx)
24376
203a078da052 revbranchcache: populate cache incrementally
Durham Goode <durham@fb.com>
parents: 24375
diff changeset
310 self._rbcrevslen = min(self._rbcrevslen, rev)
23785
cb99bacb9b4e branchcache: introduce revbranchcache for caching of revision branch names
Mads Kiilerich <madski@unity3d.com>
parents: 22357
diff changeset
311
24377
656f93ce66d5 revbranchcache: move cache writing to the transaction finalizer
Durham Goode <durham@fb.com>
parents: 24376
diff changeset
312 tr = self._repo.currenttransaction()
656f93ce66d5 revbranchcache: move cache writing to the transaction finalizer
Durham Goode <durham@fb.com>
parents: 24376
diff changeset
313 if tr:
43077
687b865b95ad formatting: byteify all mercurial/ and hgext/ string literals
Augie Fackler <augie@google.com>
parents: 43076
diff changeset
314 tr.addfinalize(b'write-revbranchcache', self.write)
24377
656f93ce66d5 revbranchcache: move cache writing to the transaction finalizer
Durham Goode <durham@fb.com>
parents: 24376
diff changeset
315
656f93ce66d5 revbranchcache: move cache writing to the transaction finalizer
Durham Goode <durham@fb.com>
parents: 24376
diff changeset
316 def write(self, tr=None):
23785
cb99bacb9b4e branchcache: introduce revbranchcache for caching of revision branch names
Mads Kiilerich <madski@unity3d.com>
parents: 22357
diff changeset
317 """Save branch cache if it is dirty."""
24374
77fd1fb538cd revbranchcache: store repo on the object
Durham Goode <durham@fb.com>
parents: 24373
diff changeset
318 repo = self._repo
29756
0d588332ad2c branchmap: acquires lock before writting the rev branch cache
Pierre-Yves David <pierre-yves.david@ens-lyon.org>
parents: 29755
diff changeset
319 wlock = None
43077
687b865b95ad formatting: byteify all mercurial/ and hgext/ string literals
Augie Fackler <augie@google.com>
parents: 43076
diff changeset
320 step = b''
29756
0d588332ad2c branchmap: acquires lock before writting the rev branch cache
Pierre-Yves David <pierre-yves.david@ens-lyon.org>
parents: 29755
diff changeset
321 try:
42222
09fd338522fa revbranchcache: factor logic to write names and revs in separate functions
Pulkit Goyal <pulkit@yandex-team.ru>
parents: 42138
diff changeset
322 # write the new names
29755
9f3c49ee4486 branchmap: preparatory indent of indent the branch rev writing code
Pierre-Yves David <pierre-yves.david@ens-lyon.org>
parents: 29615
diff changeset
323 if self._rbcnamescount < len(self._names):
29756
0d588332ad2c branchmap: acquires lock before writting the rev branch cache
Pierre-Yves David <pierre-yves.david@ens-lyon.org>
parents: 29755
diff changeset
324 wlock = repo.wlock(wait=False)
43077
687b865b95ad formatting: byteify all mercurial/ and hgext/ string literals
Augie Fackler <augie@google.com>
parents: 43076
diff changeset
325 step = b' names'
42222
09fd338522fa revbranchcache: factor logic to write names and revs in separate functions
Pulkit Goyal <pulkit@yandex-team.ru>
parents: 42138
diff changeset
326 self._writenames(repo)
23785
cb99bacb9b4e branchcache: introduce revbranchcache for caching of revision branch names
Mads Kiilerich <madski@unity3d.com>
parents: 22357
diff changeset
327
42222
09fd338522fa revbranchcache: factor logic to write names and revs in separate functions
Pulkit Goyal <pulkit@yandex-team.ru>
parents: 42138
diff changeset
328 # write the new revs
29755
9f3c49ee4486 branchmap: preparatory indent of indent the branch rev writing code
Pierre-Yves David <pierre-yves.david@ens-lyon.org>
parents: 29615
diff changeset
329 start = self._rbcrevslen * _rbcrecsize
51940
9f7cf869e9f4 rev-branch-cache: add a way to force rewrite of the cache
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 51939
diff changeset
330 if self._force_overwrite or start != len(self._rbcrevs):
43077
687b865b95ad formatting: byteify all mercurial/ and hgext/ string literals
Augie Fackler <augie@google.com>
parents: 43076
diff changeset
331 step = b''
29756
0d588332ad2c branchmap: acquires lock before writting the rev branch cache
Pierre-Yves David <pierre-yves.david@ens-lyon.org>
parents: 29755
diff changeset
332 if wlock is None:
0d588332ad2c branchmap: acquires lock before writting the rev branch cache
Pierre-Yves David <pierre-yves.david@ens-lyon.org>
parents: 29755
diff changeset
333 wlock = repo.wlock(wait=False)
42222
09fd338522fa revbranchcache: factor logic to write names and revs in separate functions
Pulkit Goyal <pulkit@yandex-team.ru>
parents: 42138
diff changeset
334 self._writerevs(repo, start)
09fd338522fa revbranchcache: factor logic to write names and revs in separate functions
Pulkit Goyal <pulkit@yandex-team.ru>
parents: 42138
diff changeset
335
29757
3b184adfb5be branchmap: simplify error handlind when writing rev branch cache
Pierre-Yves David <pierre-yves.david@ens-lyon.org>
parents: 29756
diff changeset
336 except (IOError, OSError, error.Abort, error.LockError) as inst:
43076
2372284d9457 formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents: 42603
diff changeset
337 repo.ui.debug(
43077
687b865b95ad formatting: byteify all mercurial/ and hgext/ string literals
Augie Fackler <augie@google.com>
parents: 43076
diff changeset
338 b"couldn't write revision branch cache%s: %s\n"
43076
2372284d9457 formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents: 42603
diff changeset
339 % (step, stringutil.forcebytestr(inst))
2372284d9457 formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents: 42603
diff changeset
340 )
29756
0d588332ad2c branchmap: acquires lock before writting the rev branch cache
Pierre-Yves David <pierre-yves.david@ens-lyon.org>
parents: 29755
diff changeset
341 finally:
0d588332ad2c branchmap: acquires lock before writting the rev branch cache
Pierre-Yves David <pierre-yves.david@ens-lyon.org>
parents: 29755
diff changeset
342 if wlock is not None:
0d588332ad2c branchmap: acquires lock before writting the rev branch cache
Pierre-Yves David <pierre-yves.david@ens-lyon.org>
parents: 29755
diff changeset
343 wlock.release()
42222
09fd338522fa revbranchcache: factor logic to write names and revs in separate functions
Pulkit Goyal <pulkit@yandex-team.ru>
parents: 42138
diff changeset
344
09fd338522fa revbranchcache: factor logic to write names and revs in separate functions
Pulkit Goyal <pulkit@yandex-team.ru>
parents: 42138
diff changeset
345 def _writenames(self, repo):
47031
f38bf44e077f black: make codebase compatible with black v21.4b2 and v20.8b1
Kyle Lippincott <spectral@google.com>
parents: 46819
diff changeset
346 """write the new branch names to revbranchcache"""
51941
7032da075572 rev-branch-cache: make sure we close the name file we open
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 51940
diff changeset
347 f = None
7032da075572 rev-branch-cache: make sure we close the name file we open
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 51940
diff changeset
348 try:
7032da075572 rev-branch-cache: make sure we close the name file we open
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 51940
diff changeset
349 if self._rbcnamescount != 0:
7032da075572 rev-branch-cache: make sure we close the name file we open
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 51940
diff changeset
350 f = repo.cachevfs.open(_rbcnames, b'ab')
7032da075572 rev-branch-cache: make sure we close the name file we open
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 51940
diff changeset
351 if f.tell() == self._rbcsnameslen:
7032da075572 rev-branch-cache: make sure we close the name file we open
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 51940
diff changeset
352 f.write(b'\0')
7032da075572 rev-branch-cache: make sure we close the name file we open
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 51940
diff changeset
353 else:
7032da075572 rev-branch-cache: make sure we close the name file we open
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 51940
diff changeset
354 f.close()
7032da075572 rev-branch-cache: make sure we close the name file we open
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 51940
diff changeset
355 f = None
7032da075572 rev-branch-cache: make sure we close the name file we open
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 51940
diff changeset
356 repo.ui.debug(b"%s changed - rewriting it\n" % _rbcnames)
7032da075572 rev-branch-cache: make sure we close the name file we open
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 51940
diff changeset
357 self._rbcnamescount = 0
7032da075572 rev-branch-cache: make sure we close the name file we open
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 51940
diff changeset
358 self._rbcrevslen = 0
7032da075572 rev-branch-cache: make sure we close the name file we open
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 51940
diff changeset
359 if self._rbcnamescount == 0:
7032da075572 rev-branch-cache: make sure we close the name file we open
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 51940
diff changeset
360 # before rewriting names, make sure references are removed
7032da075572 rev-branch-cache: make sure we close the name file we open
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 51940
diff changeset
361 repo.cachevfs.unlinkpath(_rbcrevs, ignoremissing=True)
7032da075572 rev-branch-cache: make sure we close the name file we open
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 51940
diff changeset
362 f = repo.cachevfs.open(_rbcnames, b'wb')
7032da075572 rev-branch-cache: make sure we close the name file we open
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 51940
diff changeset
363 names = self._names[self._rbcnamescount :]
7032da075572 rev-branch-cache: make sure we close the name file we open
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 51940
diff changeset
364 from_local = encoding.fromlocal
7032da075572 rev-branch-cache: make sure we close the name file we open
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 51940
diff changeset
365 data = b'\0'.join(from_local(b) for b in names)
7032da075572 rev-branch-cache: make sure we close the name file we open
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 51940
diff changeset
366 f.write(data)
7032da075572 rev-branch-cache: make sure we close the name file we open
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 51940
diff changeset
367 self._rbcsnameslen = f.tell()
7032da075572 rev-branch-cache: make sure we close the name file we open
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 51940
diff changeset
368 finally:
7032da075572 rev-branch-cache: make sure we close the name file we open
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 51940
diff changeset
369 if f is not None:
42222
09fd338522fa revbranchcache: factor logic to write names and revs in separate functions
Pulkit Goyal <pulkit@yandex-team.ru>
parents: 42138
diff changeset
370 f.close()
09fd338522fa revbranchcache: factor logic to write names and revs in separate functions
Pulkit Goyal <pulkit@yandex-team.ru>
parents: 42138
diff changeset
371 self._rbcnamescount = len(self._names)
09fd338522fa revbranchcache: factor logic to write names and revs in separate functions
Pulkit Goyal <pulkit@yandex-team.ru>
parents: 42138
diff changeset
372
09fd338522fa revbranchcache: factor logic to write names and revs in separate functions
Pulkit Goyal <pulkit@yandex-team.ru>
parents: 42138
diff changeset
373 def _writerevs(self, repo, start):
47031
f38bf44e077f black: make codebase compatible with black v21.4b2 and v20.8b1
Kyle Lippincott <spectral@google.com>
parents: 46819
diff changeset
374 """write the new revs to revbranchcache"""
42223
ececa45c80d8 revbranchcache: use context manager in _writerevs() to write to file
Pulkit Goyal <pulkit@yandex-team.ru>
parents: 42222
diff changeset
375 revs = min(len(repo.changelog), len(self._rbcrevs) // _rbcrecsize)
51942
c564be351754 rev-branch-cache: stop truncating cache file
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 51941
diff changeset
376
c564be351754 rev-branch-cache: stop truncating cache file
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 51941
diff changeset
377 end = revs * _rbcrecsize
51940
9f7cf869e9f4 rev-branch-cache: add a way to force rewrite of the cache
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 51939
diff changeset
378 if self._force_overwrite:
9f7cf869e9f4 rev-branch-cache: add a way to force rewrite of the cache
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 51939
diff changeset
379 start = 0
51942
c564be351754 rev-branch-cache: stop truncating cache file
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 51941
diff changeset
380
c564be351754 rev-branch-cache: stop truncating cache file
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 51941
diff changeset
381 with repo.cachevfs.open(_rbcrevs, b'a+b') as f:
c564be351754 rev-branch-cache: stop truncating cache file
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 51941
diff changeset
382 pass # this make sure the file exist…
c564be351754 rev-branch-cache: stop truncating cache file
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 51941
diff changeset
383 with repo.cachevfs.open(_rbcrevs, b'r+b') as f:
c564be351754 rev-branch-cache: stop truncating cache file
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 51941
diff changeset
384 f.seek(0, os.SEEK_END)
51939
1eb2317c1762 rev-branch-cache: issue more truthful "truncating" message
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 51938
diff changeset
385 current_size = f.tell()
1eb2317c1762 rev-branch-cache: issue more truthful "truncating" message
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 51938
diff changeset
386 if current_size < start:
1eb2317c1762 rev-branch-cache: issue more truthful "truncating" message
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 51938
diff changeset
387 start = 0
1eb2317c1762 rev-branch-cache: issue more truthful "truncating" message
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 51938
diff changeset
388 if current_size != start:
51942
c564be351754 rev-branch-cache: stop truncating cache file
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 51941
diff changeset
389 threshold = current_size * REWRITE_RATIO
c564be351754 rev-branch-cache: stop truncating cache file
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 51941
diff changeset
390 if (max(end, current_size) - start) < threshold:
c564be351754 rev-branch-cache: stop truncating cache file
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 51941
diff changeset
391 # end affected, let overwrite the bad value
c564be351754 rev-branch-cache: stop truncating cache file
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 51941
diff changeset
392 dbg = b"overwriting %d bytes from %d in cache/%s"
c564be351754 rev-branch-cache: stop truncating cache file
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 51941
diff changeset
393 dbg %= (current_size - start, start, _rbcrevs)
c564be351754 rev-branch-cache: stop truncating cache file
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 51941
diff changeset
394 if end < current_size:
c564be351754 rev-branch-cache: stop truncating cache file
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 51941
diff changeset
395 extra = b" leaving (%d trailing bytes)"
c564be351754 rev-branch-cache: stop truncating cache file
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 51941
diff changeset
396 extra %= current_size - end
c564be351754 rev-branch-cache: stop truncating cache file
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 51941
diff changeset
397 dbg += extra
c564be351754 rev-branch-cache: stop truncating cache file
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 51941
diff changeset
398 dbg += b'\n'
c564be351754 rev-branch-cache: stop truncating cache file
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 51941
diff changeset
399 repo.ui.debug(dbg)
c564be351754 rev-branch-cache: stop truncating cache file
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 51941
diff changeset
400 else:
c564be351754 rev-branch-cache: stop truncating cache file
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 51941
diff changeset
401 start = 0
c564be351754 rev-branch-cache: stop truncating cache file
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 51941
diff changeset
402 dbg = b"resetting content of cache/%s\n" % _rbcrevs
c564be351754 rev-branch-cache: stop truncating cache file
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 51941
diff changeset
403 repo.ui.debug(dbg)
c564be351754 rev-branch-cache: stop truncating cache file
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 51941
diff changeset
404 if start > 0:
42222
09fd338522fa revbranchcache: factor logic to write names and revs in separate functions
Pulkit Goyal <pulkit@yandex-team.ru>
parents: 42138
diff changeset
405 f.seek(start)
51942
c564be351754 rev-branch-cache: stop truncating cache file
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 51941
diff changeset
406 f.write(self._rbcrevs.slice(start, end))
c564be351754 rev-branch-cache: stop truncating cache file
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 51941
diff changeset
407 else:
c564be351754 rev-branch-cache: stop truncating cache file
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 51941
diff changeset
408 f.close()
c564be351754 rev-branch-cache: stop truncating cache file
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 51941
diff changeset
409 with repo.cachevfs.open(
c564be351754 rev-branch-cache: stop truncating cache file
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 51941
diff changeset
410 _rbcrevs,
c564be351754 rev-branch-cache: stop truncating cache file
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 51941
diff changeset
411 b'wb',
c564be351754 rev-branch-cache: stop truncating cache file
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 51941
diff changeset
412 atomictemp=True,
c564be351754 rev-branch-cache: stop truncating cache file
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 51941
diff changeset
413 ) as rev_file:
c564be351754 rev-branch-cache: stop truncating cache file
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 51941
diff changeset
414 rev_file.write(self._rbcrevs.slice(start, end))
42222
09fd338522fa revbranchcache: factor logic to write names and revs in separate functions
Pulkit Goyal <pulkit@yandex-team.ru>
parents: 42138
diff changeset
415 self._rbcrevslen = revs
51940
9f7cf869e9f4 rev-branch-cache: add a way to force rewrite of the cache
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 51939
diff changeset
416 self._force_overwrite = False