mercurial-scm/hg-stable: mercurial/branching/rev

annotate mercurial/branching/rev_cache.py @ 53040:cdd7bf612c7b stable tip

bundle-spec: properly format boolean parameter (issue6960) This was breaking automatic clone bundle generation. This changeset fixes it and add a test to catch it in the future.

author	Pierre-Yves David <pierre-yves.david@octobus.net>
date	Tue, 11 Mar 2025 02:29:42 +0100
parents	24ee91ba9aa8
children

rev	line source
51938 f0e07efc199f rev-branch-cache: move the code in a dedicated module Pierre-Yves David <pierre-yves.david@octobus.net> parents: 51901 diff changeset	1 # rev_cache.py - caching branch information per revision
18116 bcee63733aad branchmap: create a mercurial.branchmap module Pierre-Yves David <pierre-yves.david@ens-lyon.org> parents: diff changeset	2 #
bcee63733aad branchmap: create a mercurial.branchmap module Pierre-Yves David <pierre-yves.david@ens-lyon.org> parents: diff changeset	3 # This software may be used and distributed according to the terms of the
bcee63733aad branchmap: create a mercurial.branchmap module Pierre-Yves David <pierre-yves.david@ens-lyon.org> parents: diff changeset	4 # GNU General Public License version 2 or any later version.
51901 f4733654f144 typing: add `from __future__ import annotations` to most files Matt Harbison <matt_harbison@yahoo.com> parents: 51651 diff changeset	5 from __future__ import annotations
25918 47f36e050c2e branchmap: use absolute_import Gregory Szorc <gregory.szorc@gmail.com> parents: 25660 diff changeset	6
51942 c564be351754 rev-branch-cache: stop truncating cache file Pierre-Yves David <pierre-yves.david@octobus.net> parents: 51941 diff changeset	7 import os
25918 47f36e050c2e branchmap: use absolute_import Gregory Szorc <gregory.szorc@gmail.com> parents: 25660 diff changeset	8 import struct
47f36e050c2e branchmap: use absolute_import Gregory Szorc <gregory.szorc@gmail.com> parents: 25660 diff changeset	9
51938 f0e07efc199f rev-branch-cache: move the code in a dedicated module Pierre-Yves David <pierre-yves.david@octobus.net> parents: 51901 diff changeset	10 from ..node import (
25918 47f36e050c2e branchmap: use absolute_import Gregory Szorc <gregory.szorc@gmail.com> parents: 25660 diff changeset	11 nullrev,
47f36e050c2e branchmap: use absolute_import Gregory Szorc <gregory.szorc@gmail.com> parents: 25660 diff changeset	12 )
51302 9d3721552b6c pytype: import typing directly Pierre-Yves David <pierre-yves.david@octobus.net> parents: 49308 diff changeset	13
51938 f0e07efc199f rev-branch-cache: move the code in a dedicated module Pierre-Yves David <pierre-yves.david@octobus.net> parents: 51901 diff changeset	14 from .. import (
25918 47f36e050c2e branchmap: use absolute_import Gregory Szorc <gregory.szorc@gmail.com> parents: 25660 diff changeset	15 encoding,
26587 56b2bcea2529 error: get Abort from 'error' instead of 'util' Pierre-Yves David <pierre-yves.david@fb.com> parents: 26460 diff changeset	16 error,
52122 db1980a361cb rev-branch-cache: disable mmapping by default on Windows Matt Harbison <matt_harbison@yahoo.com> parents: 52005 diff changeset	17 pycompat,
30995 22fbca1d11ed mercurial: switch to util.timer for all interval timings Simon Farnsworth <simonfar@fb.com> parents: 29758 diff changeset	18 util,
25918 47f36e050c2e branchmap: use absolute_import Gregory Szorc <gregory.szorc@gmail.com> parents: 25660 diff changeset	19 )
51302 9d3721552b6c pytype: import typing directly Pierre-Yves David <pierre-yves.david@octobus.net> parents: 49308 diff changeset	20
51938 f0e07efc199f rev-branch-cache: move the code in a dedicated module Pierre-Yves David <pierre-yves.david@octobus.net> parents: 51901 diff changeset	21 from ..utils import (
37087 f0b6fbea00cf stringutil: bulk-replace call sites to point to new module Yuya Nishihara <yuya@tcha.org> parents: 36968 diff changeset	22 stringutil,
f0b6fbea00cf stringutil: bulk-replace call sites to point to new module Yuya Nishihara <yuya@tcha.org> parents: 36968 diff changeset	23 )
25918 47f36e050c2e branchmap: use absolute_import Gregory Szorc <gregory.szorc@gmail.com> parents: 25660 diff changeset	24
47f36e050c2e branchmap: use absolute_import Gregory Szorc <gregory.szorc@gmail.com> parents: 25660 diff changeset	25 calcsize = struct.calcsize
31379 906be86990c4 rbc: use struct unpack_from and pack_into instead of unpack and pack Mads Kiilerich <madski@unity3d.com> parents: 31369 diff changeset	26 pack_into = struct.pack_into
906be86990c4 rbc: use struct unpack_from and pack_into instead of unpack and pack Mads Kiilerich <madski@unity3d.com> parents: 31369 diff changeset	27 unpack_from = struct.unpack_from
18117 526e7ec5c96e branchmap: extract write logic from localrepo Pierre-Yves David <pierre-yves.david@ens-lyon.org> parents: 18116 diff changeset	28
18118 e70ff1e599f4 branchmap: extract read logic from repo Pierre-Yves David <pierre-yves.david@logilab.fr> parents: 18117 diff changeset	29
23785 cb99bacb9b4e branchcache: introduce revbranchcache for caching of revision branch names Mads Kiilerich <madski@unity3d.com> parents: 22357 diff changeset	30 # Revision branch info cache
cb99bacb9b4e branchcache: introduce revbranchcache for caching of revision branch names Mads Kiilerich <madski@unity3d.com> parents: 22357 diff changeset	31
51943 0f26ee69cf36 rev-branch-cache: increment the version to "v2" Pierre-Yves David <pierre-yves.david@octobus.net> parents: 51942 diff changeset	32 # The "V2" version use the same format as the "V1" but garantee it won't be
0f26ee69cf36 rev-branch-cache: increment the version to "v2" Pierre-Yves David <pierre-yves.david@octobus.net> parents: 51942 diff changeset	33 # truncated, preventing SIGBUS when it is mmap-ed
0f26ee69cf36 rev-branch-cache: increment the version to "v2" Pierre-Yves David <pierre-yves.david@octobus.net> parents: 51942 diff changeset	34 _rbcversion = b'-v2'
43077 687b865b95ad formatting: byteify all mercurial/ and hgext/ string literals Augie Fackler <augie@google.com> parents: 43076 diff changeset	35 _rbcnames = b'rbc-names' + _rbcversion
687b865b95ad formatting: byteify all mercurial/ and hgext/ string literals Augie Fackler <augie@google.com> parents: 43076 diff changeset	36 _rbcrevs = b'rbc-revs' + _rbcversion
51944 bd7359c18d69 rev-branch-cache: fallback on "v1" data if no v2 is found Pierre-Yves David <pierre-yves.david@octobus.net> parents: 51943 diff changeset	37 _rbc_legacy_version = b'-v1'
bd7359c18d69 rev-branch-cache: fallback on "v1" data if no v2 is found Pierre-Yves David <pierre-yves.david@octobus.net> parents: 51943 diff changeset	38 _rbc_legacy_names = b'rbc-names' + _rbc_legacy_version
bd7359c18d69 rev-branch-cache: fallback on "v1" data if no v2 is found Pierre-Yves David <pierre-yves.david@octobus.net> parents: 51943 diff changeset	39 _rbc_legacy_revs = b'rbc-revs' + _rbc_legacy_version
23785 cb99bacb9b4e branchcache: introduce revbranchcache for caching of revision branch names Mads Kiilerich <madski@unity3d.com> parents: 22357 diff changeset	40 # [4 byte hash prefix][4 byte branch name number with sign bit indicating open]
43077 687b865b95ad formatting: byteify all mercurial/ and hgext/ string literals Augie Fackler <augie@google.com> parents: 43076 diff changeset	41 _rbcrecfmt = b'>4sI'
23785 cb99bacb9b4e branchcache: introduce revbranchcache for caching of revision branch names Mads Kiilerich <madski@unity3d.com> parents: 22357 diff changeset	42 _rbcrecsize = calcsize(_rbcrecfmt)
46434 1726a53a8494 reverse-branch-cache: switch to doubling allocating scheme Joerg Sonnenberger <joerg@bec.de> parents: 46254 diff changeset	43 _rbcmininc = 64 * _rbcrecsize
23785 cb99bacb9b4e branchcache: introduce revbranchcache for caching of revision branch names Mads Kiilerich <madski@unity3d.com> parents: 22357 diff changeset	44 _rbcnodelen = 4
43076 2372284d9457 formatting: blacken the codebase Augie Fackler <augie@google.com> parents: 42603 diff changeset	45 _rbcbranchidxmask = 0x7FFFFFFF
23785 cb99bacb9b4e branchcache: introduce revbranchcache for caching of revision branch names Mads Kiilerich <madski@unity3d.com> parents: 22357 diff changeset	46 _rbccloseflag = 0x80000000
cb99bacb9b4e branchcache: introduce revbranchcache for caching of revision branch names Mads Kiilerich <madski@unity3d.com> parents: 22357 diff changeset	47
43076 2372284d9457 formatting: blacken the codebase Augie Fackler <augie@google.com> parents: 42603 diff changeset	48
51942 c564be351754 rev-branch-cache: stop truncating cache file Pierre-Yves David <pierre-yves.david@octobus.net> parents: 51941 diff changeset	49 # with atomic replacement.
c564be351754 rev-branch-cache: stop truncating cache file Pierre-Yves David <pierre-yves.david@octobus.net> parents: 51941 diff changeset	50 REWRITE_RATIO = 0.2
c564be351754 rev-branch-cache: stop truncating cache file Pierre-Yves David <pierre-yves.david@octobus.net> parents: 51941 diff changeset	51
c564be351754 rev-branch-cache: stop truncating cache file Pierre-Yves David <pierre-yves.david@octobus.net> parents: 51941 diff changeset	52
51377 02e7d79edf62 branchmap: use mmap for faster revbranchcache loading Arseniy Alekseyev <aalekseyev@janestreet.com> parents: 51307 diff changeset	53 class rbcrevs:
02e7d79edf62 branchmap: use mmap for faster revbranchcache loading Arseniy Alekseyev <aalekseyev@janestreet.com> parents: 51307 diff changeset	54 """a byte string consisting of an immutable prefix followed by a mutable suffix"""
02e7d79edf62 branchmap: use mmap for faster revbranchcache loading Arseniy Alekseyev <aalekseyev@janestreet.com> parents: 51307 diff changeset	55
02e7d79edf62 branchmap: use mmap for faster revbranchcache loading Arseniy Alekseyev <aalekseyev@janestreet.com> parents: 51307 diff changeset	56 def __init__(self, revs):
02e7d79edf62 branchmap: use mmap for faster revbranchcache loading Arseniy Alekseyev <aalekseyev@janestreet.com> parents: 51307 diff changeset	57 self._prefix = revs
02e7d79edf62 branchmap: use mmap for faster revbranchcache loading Arseniy Alekseyev <aalekseyev@janestreet.com> parents: 51307 diff changeset	58 self._rest = bytearray()
02e7d79edf62 branchmap: use mmap for faster revbranchcache loading Arseniy Alekseyev <aalekseyev@janestreet.com> parents: 51307 diff changeset	59
52005 76416b6e9d9b rev-branch-cache: properly ignores unaligned trailing data Pierre-Yves David <pierre-yves.david@octobus.net> parents: 52004 diff changeset	60 @property
76416b6e9d9b rev-branch-cache: properly ignores unaligned trailing data Pierre-Yves David <pierre-yves.david@octobus.net> parents: 52004 diff changeset	61 def len_prefix(self):
76416b6e9d9b rev-branch-cache: properly ignores unaligned trailing data Pierre-Yves David <pierre-yves.david@octobus.net> parents: 52004 diff changeset	62 size = len(self._prefix)
76416b6e9d9b rev-branch-cache: properly ignores unaligned trailing data Pierre-Yves David <pierre-yves.david@octobus.net> parents: 52004 diff changeset	63 return size - (size % _rbcrecsize)
76416b6e9d9b rev-branch-cache: properly ignores unaligned trailing data Pierre-Yves David <pierre-yves.david@octobus.net> parents: 52004 diff changeset	64
51377 02e7d79edf62 branchmap: use mmap for faster revbranchcache loading Arseniy Alekseyev <aalekseyev@janestreet.com> parents: 51307 diff changeset	65 def __len__(self):
52005 76416b6e9d9b rev-branch-cache: properly ignores unaligned trailing data Pierre-Yves David <pierre-yves.david@octobus.net> parents: 52004 diff changeset	66 return self.len_prefix + len(self._rest)
51377 02e7d79edf62 branchmap: use mmap for faster revbranchcache loading Arseniy Alekseyev <aalekseyev@janestreet.com> parents: 51307 diff changeset	67
02e7d79edf62 branchmap: use mmap for faster revbranchcache loading Arseniy Alekseyev <aalekseyev@janestreet.com> parents: 51307 diff changeset	68 def unpack_record(self, rbcrevidx):
52005 76416b6e9d9b rev-branch-cache: properly ignores unaligned trailing data Pierre-Yves David <pierre-yves.david@octobus.net> parents: 52004 diff changeset	69 if rbcrevidx < self.len_prefix:
51377 02e7d79edf62 branchmap: use mmap for faster revbranchcache loading Arseniy Alekseyev <aalekseyev@janestreet.com> parents: 51307 diff changeset	70 return unpack_from(_rbcrecfmt, util.buffer(self._prefix), rbcrevidx)
02e7d79edf62 branchmap: use mmap for faster revbranchcache loading Arseniy Alekseyev <aalekseyev@janestreet.com> parents: 51307 diff changeset	71 else:
02e7d79edf62 branchmap: use mmap for faster revbranchcache loading Arseniy Alekseyev <aalekseyev@janestreet.com> parents: 51307 diff changeset	72 return unpack_from(
02e7d79edf62 branchmap: use mmap for faster revbranchcache loading Arseniy Alekseyev <aalekseyev@janestreet.com> parents: 51307 diff changeset	73 _rbcrecfmt,
02e7d79edf62 branchmap: use mmap for faster revbranchcache loading Arseniy Alekseyev <aalekseyev@janestreet.com> parents: 51307 diff changeset	74 util.buffer(self._rest),
52005 76416b6e9d9b rev-branch-cache: properly ignores unaligned trailing data Pierre-Yves David <pierre-yves.david@octobus.net> parents: 52004 diff changeset	75 rbcrevidx - self.len_prefix,
51377 02e7d79edf62 branchmap: use mmap for faster revbranchcache loading Arseniy Alekseyev <aalekseyev@janestreet.com> parents: 51307 diff changeset	76 )
02e7d79edf62 branchmap: use mmap for faster revbranchcache loading Arseniy Alekseyev <aalekseyev@janestreet.com> parents: 51307 diff changeset	77
02e7d79edf62 branchmap: use mmap for faster revbranchcache loading Arseniy Alekseyev <aalekseyev@janestreet.com> parents: 51307 diff changeset	78 def make_mutable(self):
52005 76416b6e9d9b rev-branch-cache: properly ignores unaligned trailing data Pierre-Yves David <pierre-yves.david@octobus.net> parents: 52004 diff changeset	79 if self.len_prefix > 0:
51377 02e7d79edf62 branchmap: use mmap for faster revbranchcache loading Arseniy Alekseyev <aalekseyev@janestreet.com> parents: 51307 diff changeset	80 entirety = bytearray()
52005 76416b6e9d9b rev-branch-cache: properly ignores unaligned trailing data Pierre-Yves David <pierre-yves.david@octobus.net> parents: 52004 diff changeset	81 entirety[:] = self._prefix[: self.len_prefix]
51377 02e7d79edf62 branchmap: use mmap for faster revbranchcache loading Arseniy Alekseyev <aalekseyev@janestreet.com> parents: 51307 diff changeset	82 entirety.extend(self._rest)
02e7d79edf62 branchmap: use mmap for faster revbranchcache loading Arseniy Alekseyev <aalekseyev@janestreet.com> parents: 51307 diff changeset	83 self._rest = entirety
02e7d79edf62 branchmap: use mmap for faster revbranchcache loading Arseniy Alekseyev <aalekseyev@janestreet.com> parents: 51307 diff changeset	84 self._prefix = bytearray()
02e7d79edf62 branchmap: use mmap for faster revbranchcache loading Arseniy Alekseyev <aalekseyev@janestreet.com> parents: 51307 diff changeset	85
02e7d79edf62 branchmap: use mmap for faster revbranchcache loading Arseniy Alekseyev <aalekseyev@janestreet.com> parents: 51307 diff changeset	86 def truncate(self, pos):
02e7d79edf62 branchmap: use mmap for faster revbranchcache loading Arseniy Alekseyev <aalekseyev@janestreet.com> parents: 51307 diff changeset	87 self.make_mutable()
02e7d79edf62 branchmap: use mmap for faster revbranchcache loading Arseniy Alekseyev <aalekseyev@janestreet.com> parents: 51307 diff changeset	88 del self._rest[pos:]
02e7d79edf62 branchmap: use mmap for faster revbranchcache loading Arseniy Alekseyev <aalekseyev@janestreet.com> parents: 51307 diff changeset	89
02e7d79edf62 branchmap: use mmap for faster revbranchcache loading Arseniy Alekseyev <aalekseyev@janestreet.com> parents: 51307 diff changeset	90 def pack_into(self, rbcrevidx, node, branchidx):
52005 76416b6e9d9b rev-branch-cache: properly ignores unaligned trailing data Pierre-Yves David <pierre-yves.david@octobus.net> parents: 52004 diff changeset	91 if rbcrevidx < self.len_prefix:
51377 02e7d79edf62 branchmap: use mmap for faster revbranchcache loading Arseniy Alekseyev <aalekseyev@janestreet.com> parents: 51307 diff changeset	92 self.make_mutable()
02e7d79edf62 branchmap: use mmap for faster revbranchcache loading Arseniy Alekseyev <aalekseyev@janestreet.com> parents: 51307 diff changeset	93 buf = self._rest
52005 76416b6e9d9b rev-branch-cache: properly ignores unaligned trailing data Pierre-Yves David <pierre-yves.david@octobus.net> parents: 52004 diff changeset	94 start_offset = rbcrevidx - self.len_prefix
51377 02e7d79edf62 branchmap: use mmap for faster revbranchcache loading Arseniy Alekseyev <aalekseyev@janestreet.com> parents: 51307 diff changeset	95 end_offset = start_offset + _rbcrecsize
02e7d79edf62 branchmap: use mmap for faster revbranchcache loading Arseniy Alekseyev <aalekseyev@janestreet.com> parents: 51307 diff changeset	96
02e7d79edf62 branchmap: use mmap for faster revbranchcache loading Arseniy Alekseyev <aalekseyev@janestreet.com> parents: 51307 diff changeset	97 if len(self._rest) < end_offset:
02e7d79edf62 branchmap: use mmap for faster revbranchcache loading Arseniy Alekseyev <aalekseyev@janestreet.com> parents: 51307 diff changeset	98 # bytearray doesn't allocate extra space at least in Python 3.7.
02e7d79edf62 branchmap: use mmap for faster revbranchcache loading Arseniy Alekseyev <aalekseyev@janestreet.com> parents: 51307 diff changeset	99 # When multiple changesets are added in a row, precise resize would
02e7d79edf62 branchmap: use mmap for faster revbranchcache loading Arseniy Alekseyev <aalekseyev@janestreet.com> parents: 51307 diff changeset	100 # result in quadratic complexity. Overallocate to compensate by
02e7d79edf62 branchmap: use mmap for faster revbranchcache loading Arseniy Alekseyev <aalekseyev@janestreet.com> parents: 51307 diff changeset	101 # using the classic doubling technique for dynamic arrays instead.
02e7d79edf62 branchmap: use mmap for faster revbranchcache loading Arseniy Alekseyev <aalekseyev@janestreet.com> parents: 51307 diff changeset	102 # If there was a gap in the map before, less space will be reserved.
02e7d79edf62 branchmap: use mmap for faster revbranchcache loading Arseniy Alekseyev <aalekseyev@janestreet.com> parents: 51307 diff changeset	103 self._rest.extend(b'\0' * end_offset)
02e7d79edf62 branchmap: use mmap for faster revbranchcache loading Arseniy Alekseyev <aalekseyev@janestreet.com> parents: 51307 diff changeset	104 return pack_into(
02e7d79edf62 branchmap: use mmap for faster revbranchcache loading Arseniy Alekseyev <aalekseyev@janestreet.com> parents: 51307 diff changeset	105 _rbcrecfmt,
02e7d79edf62 branchmap: use mmap for faster revbranchcache loading Arseniy Alekseyev <aalekseyev@janestreet.com> parents: 51307 diff changeset	106 buf,
02e7d79edf62 branchmap: use mmap for faster revbranchcache loading Arseniy Alekseyev <aalekseyev@janestreet.com> parents: 51307 diff changeset	107 start_offset,
02e7d79edf62 branchmap: use mmap for faster revbranchcache loading Arseniy Alekseyev <aalekseyev@janestreet.com> parents: 51307 diff changeset	108 node,
02e7d79edf62 branchmap: use mmap for faster revbranchcache loading Arseniy Alekseyev <aalekseyev@janestreet.com> parents: 51307 diff changeset	109 branchidx,
02e7d79edf62 branchmap: use mmap for faster revbranchcache loading Arseniy Alekseyev <aalekseyev@janestreet.com> parents: 51307 diff changeset	110 )
02e7d79edf62 branchmap: use mmap for faster revbranchcache loading Arseniy Alekseyev <aalekseyev@janestreet.com> parents: 51307 diff changeset	111
02e7d79edf62 branchmap: use mmap for faster revbranchcache loading Arseniy Alekseyev <aalekseyev@janestreet.com> parents: 51307 diff changeset	112 def extend(self, extension):
02e7d79edf62 branchmap: use mmap for faster revbranchcache loading Arseniy Alekseyev <aalekseyev@janestreet.com> parents: 51307 diff changeset	113 return self._rest.extend(extension)
02e7d79edf62 branchmap: use mmap for faster revbranchcache loading Arseniy Alekseyev <aalekseyev@janestreet.com> parents: 51307 diff changeset	114
02e7d79edf62 branchmap: use mmap for faster revbranchcache loading Arseniy Alekseyev <aalekseyev@janestreet.com> parents: 51307 diff changeset	115 def slice(self, begin, end):
52005 76416b6e9d9b rev-branch-cache: properly ignores unaligned trailing data Pierre-Yves David <pierre-yves.david@octobus.net> parents: 52004 diff changeset	116 if begin < self.len_prefix:
51377 02e7d79edf62 branchmap: use mmap for faster revbranchcache loading Arseniy Alekseyev <aalekseyev@janestreet.com> parents: 51307 diff changeset	117 acc = bytearray()
52005 76416b6e9d9b rev-branch-cache: properly ignores unaligned trailing data Pierre-Yves David <pierre-yves.david@octobus.net> parents: 52004 diff changeset	118 acc[:] = self._prefix[begin : min(end, self.len_prefix)]
51377 02e7d79edf62 branchmap: use mmap for faster revbranchcache loading Arseniy Alekseyev <aalekseyev@janestreet.com> parents: 51307 diff changeset	119 acc.extend(
52005 76416b6e9d9b rev-branch-cache: properly ignores unaligned trailing data Pierre-Yves David <pierre-yves.david@octobus.net> parents: 52004 diff changeset	120 self._rest[begin - self.len_prefix : end - self.len_prefix]
51377 02e7d79edf62 branchmap: use mmap for faster revbranchcache loading Arseniy Alekseyev <aalekseyev@janestreet.com> parents: 51307 diff changeset	121 )
02e7d79edf62 branchmap: use mmap for faster revbranchcache loading Arseniy Alekseyev <aalekseyev@janestreet.com> parents: 51307 diff changeset	122 return acc
52005 76416b6e9d9b rev-branch-cache: properly ignores unaligned trailing data Pierre-Yves David <pierre-yves.david@octobus.net> parents: 52004 diff changeset	123 return self._rest[begin - self.len_prefix : end - self.len_prefix]
51377 02e7d79edf62 branchmap: use mmap for faster revbranchcache loading Arseniy Alekseyev <aalekseyev@janestreet.com> parents: 51307 diff changeset	124
02e7d79edf62 branchmap: use mmap for faster revbranchcache loading Arseniy Alekseyev <aalekseyev@janestreet.com> parents: 51307 diff changeset	125
49037 642e31cb55f0 py3: use class X: instead of class X(object): Gregory Szorc <gregory.szorc@gmail.com> parents: 49026 diff changeset	126 class revbranchcache:
23785 cb99bacb9b4e branchcache: introduce revbranchcache for caching of revision branch names Mads Kiilerich <madski@unity3d.com> parents: 22357 diff changeset	127 """Persistent cache, mapping from revision number to branch name and close.
cb99bacb9b4e branchcache: introduce revbranchcache for caching of revision branch names Mads Kiilerich <madski@unity3d.com> parents: 22357 diff changeset	128 This is a low level cache, independent of filtering.
cb99bacb9b4e branchcache: introduce revbranchcache for caching of revision branch names Mads Kiilerich <madski@unity3d.com> parents: 22357 diff changeset	129
cb99bacb9b4e branchcache: introduce revbranchcache for caching of revision branch names Mads Kiilerich <madski@unity3d.com> parents: 22357 diff changeset	130 Branch names are stored in rbc-names in internal encoding separated by 0.
cb99bacb9b4e branchcache: introduce revbranchcache for caching of revision branch names Mads Kiilerich <madski@unity3d.com> parents: 22357 diff changeset	131 rbc-names is append-only, and each branch name is only stored once and will
cb99bacb9b4e branchcache: introduce revbranchcache for caching of revision branch names Mads Kiilerich <madski@unity3d.com> parents: 22357 diff changeset	132 thus have a unique index.
cb99bacb9b4e branchcache: introduce revbranchcache for caching of revision branch names Mads Kiilerich <madski@unity3d.com> parents: 22357 diff changeset	133
cb99bacb9b4e branchcache: introduce revbranchcache for caching of revision branch names Mads Kiilerich <madski@unity3d.com> parents: 22357 diff changeset	134 The branch info for each revision is stored in rbc-revs as constant size
cb99bacb9b4e branchcache: introduce revbranchcache for caching of revision branch names Mads Kiilerich <madski@unity3d.com> parents: 22357 diff changeset	135 records. The whole file is read into memory, but it is only 'parsed' on
cb99bacb9b4e branchcache: introduce revbranchcache for caching of revision branch names Mads Kiilerich <madski@unity3d.com> parents: 22357 diff changeset	136 demand. The file is usually append-only but will be truncated if repo
cb99bacb9b4e branchcache: introduce revbranchcache for caching of revision branch names Mads Kiilerich <madski@unity3d.com> parents: 22357 diff changeset	137 modification is detected.
cb99bacb9b4e branchcache: introduce revbranchcache for caching of revision branch names Mads Kiilerich <madski@unity3d.com> parents: 22357 diff changeset	138 The record for each revision contains the first 4 bytes of the
cb99bacb9b4e branchcache: introduce revbranchcache for caching of revision branch names Mads Kiilerich <madski@unity3d.com> parents: 22357 diff changeset	139 corresponding node hash, and the record is only used if it still matches.
cb99bacb9b4e branchcache: introduce revbranchcache for caching of revision branch names Mads Kiilerich <madski@unity3d.com> parents: 22357 diff changeset	140 Even a completely trashed rbc-revs fill thus still give the right result
cb99bacb9b4e branchcache: introduce revbranchcache for caching of revision branch names Mads Kiilerich <madski@unity3d.com> parents: 22357 diff changeset	141 while converging towards full recovery ... assuming no incorrectly matching
cb99bacb9b4e branchcache: introduce revbranchcache for caching of revision branch names Mads Kiilerich <madski@unity3d.com> parents: 22357 diff changeset	142 node hashes.
cb99bacb9b4e branchcache: introduce revbranchcache for caching of revision branch names Mads Kiilerich <madski@unity3d.com> parents: 22357 diff changeset	143 The record also contains 4 bytes where 31 bits contains the index of the
cb99bacb9b4e branchcache: introduce revbranchcache for caching of revision branch names Mads Kiilerich <madski@unity3d.com> parents: 22357 diff changeset	144 branch and the last bit indicate that it is a branch close commit.
cb99bacb9b4e branchcache: introduce revbranchcache for caching of revision branch names Mads Kiilerich <madski@unity3d.com> parents: 22357 diff changeset	145 The usage pattern for rbc-revs is thus somewhat similar to 00changelog.i
cb99bacb9b4e branchcache: introduce revbranchcache for caching of revision branch names Mads Kiilerich <madski@unity3d.com> parents: 22357 diff changeset	146 and will grow with it but be 1/8th of its size.
cb99bacb9b4e branchcache: introduce revbranchcache for caching of revision branch names Mads Kiilerich <madski@unity3d.com> parents: 22357 diff changeset	147 """
cb99bacb9b4e branchcache: introduce revbranchcache for caching of revision branch names Mads Kiilerich <madski@unity3d.com> parents: 22357 diff changeset	148
24159 5b4ed033390b revisionbranchcache: fall back to slow path if starting readonly (issue4531) Mads Kiilerich <madski@unity3d.com> parents: 23877 diff changeset	149 def __init__(self, repo, readonly=True):
23785 cb99bacb9b4e branchcache: introduce revbranchcache for caching of revision branch names Mads Kiilerich <madski@unity3d.com> parents: 22357 diff changeset	150 assert repo.filtername is None
24374 77fd1fb538cd revbranchcache: store repo on the object Durham Goode <durham@fb.com> parents: 24373 diff changeset	151 self._repo = repo
43076 2372284d9457 formatting: blacken the codebase Augie Fackler <augie@google.com> parents: 42603 diff changeset	152 self._names = [] # branch names in local encoding with static index
51377 02e7d79edf62 branchmap: use mmap for faster revbranchcache loading Arseniy Alekseyev <aalekseyev@janestreet.com> parents: 51307 diff changeset	153 self._rbcrevs = rbcrevs(bytearray())
43076 2372284d9457 formatting: blacken the codebase Augie Fackler <augie@google.com> parents: 42603 diff changeset	154 self._rbcsnameslen = 0 # length of names read at _rbcsnameslen
51945 16efed18ae4e rev-branch-cache: schedule a write of the "v2" format if we read from "v1" Pierre-Yves David <pierre-yves.david@octobus.net> parents: 51944 diff changeset	155 self._force_overwrite = False
51944 bd7359c18d69 rev-branch-cache: fallback on "v1" data if no v2 is found Pierre-Yves David <pierre-yves.david@octobus.net> parents: 51943 diff changeset	156 v1_fallback = False
23785 cb99bacb9b4e branchcache: introduce revbranchcache for caching of revision branch names Mads Kiilerich <madski@unity3d.com> parents: 22357 diff changeset	157 try:
51944 bd7359c18d69 rev-branch-cache: fallback on "v1" data if no v2 is found Pierre-Yves David <pierre-yves.david@octobus.net> parents: 51943 diff changeset	158 try:
bd7359c18d69 rev-branch-cache: fallback on "v1" data if no v2 is found Pierre-Yves David <pierre-yves.david@octobus.net> parents: 51943 diff changeset	159 bndata = repo.cachevfs.read(_rbcnames)
52665 24ee91ba9aa8 pyupgrade: drop usage of py3 aliases for `OSError` Matt Harbison <matt_harbison@yahoo.com> parents: 52122 diff changeset	160 except OSError:
51944 bd7359c18d69 rev-branch-cache: fallback on "v1" data if no v2 is found Pierre-Yves David <pierre-yves.david@octobus.net> parents: 51943 diff changeset	161 # If we don't have "v2" data, we might have "v1" data worth
bd7359c18d69 rev-branch-cache: fallback on "v1" data if no v2 is found Pierre-Yves David <pierre-yves.david@octobus.net> parents: 51943 diff changeset	162 # using.
bd7359c18d69 rev-branch-cache: fallback on "v1" data if no v2 is found Pierre-Yves David <pierre-yves.david@octobus.net> parents: 51943 diff changeset	163 #
bd7359c18d69 rev-branch-cache: fallback on "v1" data if no v2 is found Pierre-Yves David <pierre-yves.david@octobus.net> parents: 51943 diff changeset	164 # consider stop doing this many version after hg-6.9 release
bd7359c18d69 rev-branch-cache: fallback on "v1" data if no v2 is found Pierre-Yves David <pierre-yves.david@octobus.net> parents: 51943 diff changeset	165 bndata = repo.cachevfs.read(_rbc_legacy_names)
bd7359c18d69 rev-branch-cache: fallback on "v1" data if no v2 is found Pierre-Yves David <pierre-yves.david@octobus.net> parents: 51943 diff changeset	166 v1_fallback = True
51945 16efed18ae4e rev-branch-cache: schedule a write of the "v2" format if we read from "v1" Pierre-Yves David <pierre-yves.david@octobus.net> parents: 51944 diff changeset	167 self._force_overwrite = True
43076 2372284d9457 formatting: blacken the codebase Augie Fackler <augie@google.com> parents: 42603 diff changeset	168 self._rbcsnameslen = len(bndata) # for verification before writing
31380 7dd2f51f38ac rbc: empty (and invalid) rbc-names file should give an empty name list Mads Kiilerich <mads@kiilerich.com> parents: 31379 diff changeset	169 if bndata:
43076 2372284d9457 formatting: blacken the codebase Augie Fackler <augie@google.com> parents: 42603 diff changeset	170 self._names = [
43077 687b865b95ad formatting: byteify all mercurial/ and hgext/ string literals Augie Fackler <augie@google.com> parents: 43076 diff changeset	171 encoding.tolocal(bn) for bn in bndata.split(b'\0')
43076 2372284d9457 formatting: blacken the codebase Augie Fackler <augie@google.com> parents: 42603 diff changeset	172 ]
52665 24ee91ba9aa8 pyupgrade: drop usage of py3 aliases for `OSError` Matt Harbison <matt_harbison@yahoo.com> parents: 52122 diff changeset	173 except OSError:
24159 5b4ed033390b revisionbranchcache: fall back to slow path if starting readonly (issue4531) Mads Kiilerich <madski@unity3d.com> parents: 23877 diff changeset	174 if readonly:
5b4ed033390b revisionbranchcache: fall back to slow path if starting readonly (issue4531) Mads Kiilerich <madski@unity3d.com> parents: 23877 diff changeset	175 # don't try to use cache - fall back to the slow path
5b4ed033390b revisionbranchcache: fall back to slow path if starting readonly (issue4531) Mads Kiilerich <madski@unity3d.com> parents: 23877 diff changeset	176 self.branchinfo = self._branchinfo
5b4ed033390b revisionbranchcache: fall back to slow path if starting readonly (issue4531) Mads Kiilerich <madski@unity3d.com> parents: 23877 diff changeset	177
23785 cb99bacb9b4e branchcache: introduce revbranchcache for caching of revision branch names Mads Kiilerich <madski@unity3d.com> parents: 22357 diff changeset	178 if self._names:
cb99bacb9b4e branchcache: introduce revbranchcache for caching of revision branch names Mads Kiilerich <madski@unity3d.com> parents: 22357 diff changeset	179 try:
52122 db1980a361cb rev-branch-cache: disable mmapping by default on Windows Matt Harbison <matt_harbison@yahoo.com> parents: 52005 diff changeset	180 # In order to rename the atomictempfile in _writerevs(), the
db1980a361cb rev-branch-cache: disable mmapping by default on Windows Matt Harbison <matt_harbison@yahoo.com> parents: 52005 diff changeset	181 # existing file needs to be removed. The Windows code
db1980a361cb rev-branch-cache: disable mmapping by default on Windows Matt Harbison <matt_harbison@yahoo.com> parents: 52005 diff changeset	182 # (successfully) renames it to a temp file first, before moving
db1980a361cb rev-branch-cache: disable mmapping by default on Windows Matt Harbison <matt_harbison@yahoo.com> parents: 52005 diff changeset	183 # the temp file into its place. But the removal of the original
db1980a361cb rev-branch-cache: disable mmapping by default on Windows Matt Harbison <matt_harbison@yahoo.com> parents: 52005 diff changeset	184 # file then fails, because it's still mapped. The mmap object
db1980a361cb rev-branch-cache: disable mmapping by default on Windows Matt Harbison <matt_harbison@yahoo.com> parents: 52005 diff changeset	185 # needs to be closed in order to remove the file, but in order
db1980a361cb rev-branch-cache: disable mmapping by default on Windows Matt Harbison <matt_harbison@yahoo.com> parents: 52005 diff changeset	186 # to do that, the memoryview returned by util.buffer needs to be
db1980a361cb rev-branch-cache: disable mmapping by default on Windows Matt Harbison <matt_harbison@yahoo.com> parents: 52005 diff changeset	187 # released.
db1980a361cb rev-branch-cache: disable mmapping by default on Windows Matt Harbison <matt_harbison@yahoo.com> parents: 52005 diff changeset	188 usemmap = repo.ui.configbool(
db1980a361cb rev-branch-cache: disable mmapping by default on Windows Matt Harbison <matt_harbison@yahoo.com> parents: 52005 diff changeset	189 b'storage',
db1980a361cb rev-branch-cache: disable mmapping by default on Windows Matt Harbison <matt_harbison@yahoo.com> parents: 52005 diff changeset	190 b'revbranchcache.mmap',
db1980a361cb rev-branch-cache: disable mmapping by default on Windows Matt Harbison <matt_harbison@yahoo.com> parents: 52005 diff changeset	191 default=not pycompat.iswindows,
db1980a361cb rev-branch-cache: disable mmapping by default on Windows Matt Harbison <matt_harbison@yahoo.com> parents: 52005 diff changeset	192 )
51944 bd7359c18d69 rev-branch-cache: fallback on "v1" data if no v2 is found Pierre-Yves David <pierre-yves.david@octobus.net> parents: 51943 diff changeset	193 if not v1_fallback:
bd7359c18d69 rev-branch-cache: fallback on "v1" data if no v2 is found Pierre-Yves David <pierre-yves.david@octobus.net> parents: 51943 diff changeset	194 with repo.cachevfs(_rbcrevs) as fp:
bd7359c18d69 rev-branch-cache: fallback on "v1" data if no v2 is found Pierre-Yves David <pierre-yves.david@octobus.net> parents: 51943 diff changeset	195 if usemmap and repo.cachevfs.is_mmap_safe(_rbcrevs):
bd7359c18d69 rev-branch-cache: fallback on "v1" data if no v2 is found Pierre-Yves David <pierre-yves.david@octobus.net> parents: 51943 diff changeset	196 data = util.buffer(util.mmapread(fp))
bd7359c18d69 rev-branch-cache: fallback on "v1" data if no v2 is found Pierre-Yves David <pierre-yves.david@octobus.net> parents: 51943 diff changeset	197 else:
bd7359c18d69 rev-branch-cache: fallback on "v1" data if no v2 is found Pierre-Yves David <pierre-yves.david@octobus.net> parents: 51943 diff changeset	198 data = fp.read()
bd7359c18d69 rev-branch-cache: fallback on "v1" data if no v2 is found Pierre-Yves David <pierre-yves.david@octobus.net> parents: 51943 diff changeset	199 else:
bd7359c18d69 rev-branch-cache: fallback on "v1" data if no v2 is found Pierre-Yves David <pierre-yves.david@octobus.net> parents: 51943 diff changeset	200 # If we don't have "v2" data, we might have "v1" data worth
bd7359c18d69 rev-branch-cache: fallback on "v1" data if no v2 is found Pierre-Yves David <pierre-yves.david@octobus.net> parents: 51943 diff changeset	201 # using.
bd7359c18d69 rev-branch-cache: fallback on "v1" data if no v2 is found Pierre-Yves David <pierre-yves.david@octobus.net> parents: 51943 diff changeset	202 #
bd7359c18d69 rev-branch-cache: fallback on "v1" data if no v2 is found Pierre-Yves David <pierre-yves.david@octobus.net> parents: 51943 diff changeset	203 # Consider stop doing this many version after hg-6.9
bd7359c18d69 rev-branch-cache: fallback on "v1" data if no v2 is found Pierre-Yves David <pierre-yves.david@octobus.net> parents: 51943 diff changeset	204 # release.
bd7359c18d69 rev-branch-cache: fallback on "v1" data if no v2 is found Pierre-Yves David <pierre-yves.david@octobus.net> parents: 51943 diff changeset	205 with repo.cachevfs(_rbc_legacy_revs) as fp:
51651 c0e30a019ce1 mmap: only use mmap to read rev-branch-cache data if it is safe Pierre-Yves David <pierre-yves.david@octobus.net> parents: 51644 diff changeset	206 data = fp.read()
51377 02e7d79edf62 branchmap: use mmap for faster revbranchcache loading Arseniy Alekseyev <aalekseyev@janestreet.com> parents: 51307 diff changeset	207 self._rbcrevs = rbcrevs(data)
52665 24ee91ba9aa8 pyupgrade: drop usage of py3 aliases for `OSError` Matt Harbison <matt_harbison@yahoo.com> parents: 52122 diff changeset	208 except OSError as inst:
43076 2372284d9457 formatting: blacken the codebase Augie Fackler <augie@google.com> parents: 42603 diff changeset	209 repo.ui.debug(
43077 687b865b95ad formatting: byteify all mercurial/ and hgext/ string literals Augie Fackler <augie@google.com> parents: 43076 diff changeset	210 b"couldn't read revision branch cache: %s\n"
43076 2372284d9457 formatting: blacken the codebase Augie Fackler <augie@google.com> parents: 42603 diff changeset	211 % stringutil.forcebytestr(inst)
2372284d9457 formatting: blacken the codebase Augie Fackler <augie@google.com> parents: 42603 diff changeset	212 )
23785 cb99bacb9b4e branchcache: introduce revbranchcache for caching of revision branch names Mads Kiilerich <madski@unity3d.com> parents: 22357 diff changeset	213 # remember number of good records on disk
43076 2372284d9457 formatting: blacken the codebase Augie Fackler <augie@google.com> parents: 42603 diff changeset	214 self._rbcrevslen = min(
2372284d9457 formatting: blacken the codebase Augie Fackler <augie@google.com> parents: 42603 diff changeset	215 len(self._rbcrevs) // _rbcrecsize, len(repo.changelog)
2372284d9457 formatting: blacken the codebase Augie Fackler <augie@google.com> parents: 42603 diff changeset	216 )
23785 cb99bacb9b4e branchcache: introduce revbranchcache for caching of revision branch names Mads Kiilerich <madski@unity3d.com> parents: 22357 diff changeset	217 if self._rbcrevslen == 0:
cb99bacb9b4e branchcache: introduce revbranchcache for caching of revision branch names Mads Kiilerich <madski@unity3d.com> parents: 22357 diff changeset	218 self._names = []
43076 2372284d9457 formatting: blacken the codebase Augie Fackler <augie@google.com> parents: 42603 diff changeset	219 self._rbcnamescount = len(self._names) # number of names read at
2372284d9457 formatting: blacken the codebase Augie Fackler <augie@google.com> parents: 42603 diff changeset	220 # _rbcsnameslen
23785 cb99bacb9b4e branchcache: introduce revbranchcache for caching of revision branch names Mads Kiilerich <madski@unity3d.com> parents: 22357 diff changeset	221
28558 bcd106d456c4 cache: rebuild branch cache from scratch when inconsistencies are detected Mads Kiilerich <madski@unity3d.com> parents: 28557 diff changeset	222 def _clear(self):
bcd106d456c4 cache: rebuild branch cache from scratch when inconsistencies are detected Mads Kiilerich <madski@unity3d.com> parents: 28557 diff changeset	223 self._rbcsnameslen = 0
bcd106d456c4 cache: rebuild branch cache from scratch when inconsistencies are detected Mads Kiilerich <madski@unity3d.com> parents: 28557 diff changeset	224 del self._names[:]
bcd106d456c4 cache: rebuild branch cache from scratch when inconsistencies are detected Mads Kiilerich <madski@unity3d.com> parents: 28557 diff changeset	225 self._rbcnamescount = 0
bcd106d456c4 cache: rebuild branch cache from scratch when inconsistencies are detected Mads Kiilerich <madski@unity3d.com> parents: 28557 diff changeset	226 self._rbcrevslen = len(self._repo.changelog)
51377 02e7d79edf62 branchmap: use mmap for faster revbranchcache loading Arseniy Alekseyev <aalekseyev@janestreet.com> parents: 51307 diff changeset	227 self._rbcrevs = rbcrevs(bytearray(self._rbcrevslen * _rbcrecsize))
43077 687b865b95ad formatting: byteify all mercurial/ and hgext/ string literals Augie Fackler <augie@google.com> parents: 43076 diff changeset	228 util.clearcachedproperty(self, b'_namesreverse')
51940 9f7cf869e9f4 rev-branch-cache: add a way to force rewrite of the cache Pierre-Yves David <pierre-yves.david@octobus.net> parents: 51939 diff changeset	229 self._force_overwrite = True
9f7cf869e9f4 rev-branch-cache: add a way to force rewrite of the cache Pierre-Yves David <pierre-yves.david@octobus.net> parents: 51939 diff changeset	230
9f7cf869e9f4 rev-branch-cache: add a way to force rewrite of the cache Pierre-Yves David <pierre-yves.david@octobus.net> parents: 51939 diff changeset	231 def invalidate(self, rev=0):
9f7cf869e9f4 rev-branch-cache: add a way to force rewrite of the cache Pierre-Yves David <pierre-yves.david@octobus.net> parents: 51939 diff changeset	232 self._rbcrevslen = rev
9f7cf869e9f4 rev-branch-cache: add a way to force rewrite of the cache Pierre-Yves David <pierre-yves.david@octobus.net> parents: 51939 diff changeset	233 self._rbcrevs.truncate(rev)
9f7cf869e9f4 rev-branch-cache: add a way to force rewrite of the cache Pierre-Yves David <pierre-yves.david@octobus.net> parents: 51939 diff changeset	234 self._force_overwrite = True
40749 50a64c321c1e branchmap: build the revbranchcache._namesreverse() only when required Pulkit Goyal <pulkit@yandex-team.ru> parents: 40494 diff changeset	235
50a64c321c1e branchmap: build the revbranchcache._namesreverse() only when required Pulkit Goyal <pulkit@yandex-team.ru> parents: 40494 diff changeset	236 @util.propertycache
50a64c321c1e branchmap: build the revbranchcache._namesreverse() only when required Pulkit Goyal <pulkit@yandex-team.ru> parents: 40494 diff changeset	237 def _namesreverse(self):
44470 9d2b2df2c2ba cleanup: run pyupgrade on our source tree to clean up varying things Augie Fackler <augie@google.com> parents: 44349 diff changeset	238 return {b: r for r, b in enumerate(self._names)}
28558 bcd106d456c4 cache: rebuild branch cache from scratch when inconsistencies are detected Mads Kiilerich <madski@unity3d.com> parents: 28557 diff changeset	239
40494 5e5c8f2a1eb5 branchmap: do not specify changelog as an argument Yuya Nishihara <yuya@tcha.org> parents: 40375 diff changeset	240 def branchinfo(self, rev):
23785 cb99bacb9b4e branchcache: introduce revbranchcache for caching of revision branch names Mads Kiilerich <madski@unity3d.com> parents: 22357 diff changeset	241 """Return branch name and close flag for rev, using and updating
cb99bacb9b4e branchcache: introduce revbranchcache for caching of revision branch names Mads Kiilerich <madski@unity3d.com> parents: 22357 diff changeset	242 persistent cache."""
40494 5e5c8f2a1eb5 branchmap: do not specify changelog as an argument Yuya Nishihara <yuya@tcha.org> parents: 40375 diff changeset	243 changelog = self._repo.changelog
23785 cb99bacb9b4e branchcache: introduce revbranchcache for caching of revision branch names Mads Kiilerich <madski@unity3d.com> parents: 22357 diff changeset	244 rbcrevidx = rev * _rbcrecsize
cb99bacb9b4e branchcache: introduce revbranchcache for caching of revision branch names Mads Kiilerich <madski@unity3d.com> parents: 22357 diff changeset	245
25266 38117278f295 revbranchcache: return uncached branchinfo for nullrev (issue4683) Yuya Nishihara <yuya@tcha.org> parents: 24728 diff changeset	246 # avoid negative index, changelog.read(nullrev) is fast without cache
38117278f295 revbranchcache: return uncached branchinfo for nullrev (issue4683) Yuya Nishihara <yuya@tcha.org> parents: 24728 diff changeset	247 if rev == nullrev:
38117278f295 revbranchcache: return uncached branchinfo for nullrev (issue4683) Yuya Nishihara <yuya@tcha.org> parents: 24728 diff changeset	248 return changelog.branchinfo(rev)
38117278f295 revbranchcache: return uncached branchinfo for nullrev (issue4683) Yuya Nishihara <yuya@tcha.org> parents: 24728 diff changeset	249
29604 db0095c83344 rbc: fix invalid rbc-revs entries caused by missing cache growth Mads Kiilerich <madski@unity3d.com> parents: 29423 diff changeset	250 # if requested rev isn't allocated, grow and cache the rev info
23785 cb99bacb9b4e branchcache: introduce revbranchcache for caching of revision branch names Mads Kiilerich <madski@unity3d.com> parents: 22357 diff changeset	251 if len(self._rbcrevs) < rbcrevidx + _rbcrecsize:
40494 5e5c8f2a1eb5 branchmap: do not specify changelog as an argument Yuya Nishihara <yuya@tcha.org> parents: 40375 diff changeset	252 return self._branchinfo(rev)
23785 cb99bacb9b4e branchcache: introduce revbranchcache for caching of revision branch names Mads Kiilerich <madski@unity3d.com> parents: 22357 diff changeset	253
cb99bacb9b4e branchcache: introduce revbranchcache for caching of revision branch names Mads Kiilerich <madski@unity3d.com> parents: 22357 diff changeset	254 # fast path: extract data from cache, use it if node is matching
cb99bacb9b4e branchcache: introduce revbranchcache for caching of revision branch names Mads Kiilerich <madski@unity3d.com> parents: 22357 diff changeset	255 reponode = changelog.node(rev)[:_rbcnodelen]
51377 02e7d79edf62 branchmap: use mmap for faster revbranchcache loading Arseniy Alekseyev <aalekseyev@janestreet.com> parents: 51307 diff changeset	256 cachenode, branchidx = self._rbcrevs.unpack_record(rbcrevidx)
23785 cb99bacb9b4e branchcache: introduce revbranchcache for caching of revision branch names Mads Kiilerich <madski@unity3d.com> parents: 22357 diff changeset	257 close = bool(branchidx & _rbccloseflag)
cb99bacb9b4e branchcache: introduce revbranchcache for caching of revision branch names Mads Kiilerich <madski@unity3d.com> parents: 22357 diff changeset	258 if close:
cb99bacb9b4e branchcache: introduce revbranchcache for caching of revision branch names Mads Kiilerich <madski@unity3d.com> parents: 22357 diff changeset	259 branchidx &= _rbcbranchidxmask
43077 687b865b95ad formatting: byteify all mercurial/ and hgext/ string literals Augie Fackler <augie@google.com> parents: 43076 diff changeset	260 if cachenode == b'\0\0\0\0':
24376 203a078da052 revbranchcache: populate cache incrementally Durham Goode <durham@fb.com> parents: 24375 diff changeset	261 pass
203a078da052 revbranchcache: populate cache incrementally Durham Goode <durham@fb.com> parents: 24375 diff changeset	262 elif cachenode == reponode:
29615 a2a380e2750f rbc: fix superfluous rebuilding from scratch - don't abuse self._rbcnamescount Mads Kiilerich <madski@unity3d.com> parents: 29604 diff changeset	263 try:
28558 bcd106d456c4 cache: rebuild branch cache from scratch when inconsistencies are detected Mads Kiilerich <madski@unity3d.com> parents: 28557 diff changeset	264 return self._names[branchidx], close
29615 a2a380e2750f rbc: fix superfluous rebuilding from scratch - don't abuse self._rbcnamescount Mads Kiilerich <madski@unity3d.com> parents: 29604 diff changeset	265 except IndexError:
a2a380e2750f rbc: fix superfluous rebuilding from scratch - don't abuse self._rbcnamescount Mads Kiilerich <madski@unity3d.com> parents: 29604 diff changeset	266 # recover from invalid reference to unknown branch
43076 2372284d9457 formatting: blacken the codebase Augie Fackler <augie@google.com> parents: 42603 diff changeset	267 self._repo.ui.debug(
43077 687b865b95ad formatting: byteify all mercurial/ and hgext/ string literals Augie Fackler <augie@google.com> parents: 43076 diff changeset	268 b"referenced branch names not found"
687b865b95ad formatting: byteify all mercurial/ and hgext/ string literals Augie Fackler <augie@google.com> parents: 43076 diff changeset	269 b" - rebuilding revision branch cache from scratch\n"
43076 2372284d9457 formatting: blacken the codebase Augie Fackler <augie@google.com> parents: 42603 diff changeset	270 )
29615 a2a380e2750f rbc: fix superfluous rebuilding from scratch - don't abuse self._rbcnamescount Mads Kiilerich <madski@unity3d.com> parents: 29604 diff changeset	271 self._clear()
24376 203a078da052 revbranchcache: populate cache incrementally Durham Goode <durham@fb.com> parents: 24375 diff changeset	272 else:
203a078da052 revbranchcache: populate cache incrementally Durham Goode <durham@fb.com> parents: 24375 diff changeset	273 # rev/node map has changed, invalidate the cache from here up
43076 2372284d9457 formatting: blacken the codebase Augie Fackler <augie@google.com> parents: 42603 diff changeset	274 self._repo.ui.debug(
43077 687b865b95ad formatting: byteify all mercurial/ and hgext/ string literals Augie Fackler <augie@google.com> parents: 43076 diff changeset	275 b"history modification detected - truncating "
687b865b95ad formatting: byteify all mercurial/ and hgext/ string literals Augie Fackler <augie@google.com> parents: 43076 diff changeset	276 b"revision branch cache to revision %d\n" % rev
43076 2372284d9457 formatting: blacken the codebase Augie Fackler <augie@google.com> parents: 42603 diff changeset	277 )
24376 203a078da052 revbranchcache: populate cache incrementally Durham Goode <durham@fb.com> parents: 24375 diff changeset	278 truncate = rbcrevidx + _rbcrecsize
51377 02e7d79edf62 branchmap: use mmap for faster revbranchcache loading Arseniy Alekseyev <aalekseyev@janestreet.com> parents: 51307 diff changeset	279 self._rbcrevs.truncate(truncate)
24376 203a078da052 revbranchcache: populate cache incrementally Durham Goode <durham@fb.com> parents: 24375 diff changeset	280 self._rbcrevslen = min(self._rbcrevslen, truncate)
203a078da052 revbranchcache: populate cache incrementally Durham Goode <durham@fb.com> parents: 24375 diff changeset	281
23785 cb99bacb9b4e branchcache: introduce revbranchcache for caching of revision branch names Mads Kiilerich <madski@unity3d.com> parents: 22357 diff changeset	282 # fall back to slow path and make sure it will be written to disk
40494 5e5c8f2a1eb5 branchmap: do not specify changelog as an argument Yuya Nishihara <yuya@tcha.org> parents: 40375 diff changeset	283 return self._branchinfo(rev)
23785 cb99bacb9b4e branchcache: introduce revbranchcache for caching of revision branch names Mads Kiilerich <madski@unity3d.com> parents: 22357 diff changeset	284
40494 5e5c8f2a1eb5 branchmap: do not specify changelog as an argument Yuya Nishihara <yuya@tcha.org> parents: 40375 diff changeset	285 def _branchinfo(self, rev):
23785 cb99bacb9b4e branchcache: introduce revbranchcache for caching of revision branch names Mads Kiilerich <madski@unity3d.com> parents: 22357 diff changeset	286 """Retrieve branch info from changelog and update _rbcrevs"""
40494 5e5c8f2a1eb5 branchmap: do not specify changelog as an argument Yuya Nishihara <yuya@tcha.org> parents: 40375 diff changeset	287 changelog = self._repo.changelog
23785 cb99bacb9b4e branchcache: introduce revbranchcache for caching of revision branch names Mads Kiilerich <madski@unity3d.com> parents: 22357 diff changeset	288 b, close = changelog.branchinfo(rev)
cb99bacb9b4e branchcache: introduce revbranchcache for caching of revision branch names Mads Kiilerich <madski@unity3d.com> parents: 22357 diff changeset	289 if b in self._namesreverse:
cb99bacb9b4e branchcache: introduce revbranchcache for caching of revision branch names Mads Kiilerich <madski@unity3d.com> parents: 22357 diff changeset	290 branchidx = self._namesreverse[b]
cb99bacb9b4e branchcache: introduce revbranchcache for caching of revision branch names Mads Kiilerich <madski@unity3d.com> parents: 22357 diff changeset	291 else:
cb99bacb9b4e branchcache: introduce revbranchcache for caching of revision branch names Mads Kiilerich <madski@unity3d.com> parents: 22357 diff changeset	292 branchidx = len(self._names)
cb99bacb9b4e branchcache: introduce revbranchcache for caching of revision branch names Mads Kiilerich <madski@unity3d.com> parents: 22357 diff changeset	293 self._names.append(b)
cb99bacb9b4e branchcache: introduce revbranchcache for caching of revision branch names Mads Kiilerich <madski@unity3d.com> parents: 22357 diff changeset	294 self._namesreverse[b] = branchidx
cb99bacb9b4e branchcache: introduce revbranchcache for caching of revision branch names Mads Kiilerich <madski@unity3d.com> parents: 22357 diff changeset	295 reponode = changelog.node(rev)
cb99bacb9b4e branchcache: introduce revbranchcache for caching of revision branch names Mads Kiilerich <madski@unity3d.com> parents: 22357 diff changeset	296 if close:
cb99bacb9b4e branchcache: introduce revbranchcache for caching of revision branch names Mads Kiilerich <madski@unity3d.com> parents: 22357 diff changeset	297 branchidx \|= _rbccloseflag
40494 5e5c8f2a1eb5 branchmap: do not specify changelog as an argument Yuya Nishihara <yuya@tcha.org> parents: 40375 diff changeset	298 self._setcachedata(rev, reponode, branchidx)
24375 fe255b2525d5 revbranchcache: move entry writing to a separate function Durham Goode <durham@fb.com> parents: 24374 diff changeset	299 return b, close
fe255b2525d5 revbranchcache: move entry writing to a separate function Durham Goode <durham@fb.com> parents: 24374 diff changeset	300
46444 3e91d9978bec branchmap: update rev-branch-cache incrementally Joerg Sonnenberger <joerg@bec.de> parents: 46434 diff changeset	301 def setdata(self, rev, changelogrevision):
36968 95f4f1bfb650 revbranchcache: add a public function to update the data Boris Feld <boris.feld@octobus.net> parents: 36504 diff changeset	302 """add new data information to the cache"""
46444 3e91d9978bec branchmap: update rev-branch-cache incrementally Joerg Sonnenberger <joerg@bec.de> parents: 46434 diff changeset	303 branch, close = changelogrevision.branchinfo
3e91d9978bec branchmap: update rev-branch-cache incrementally Joerg Sonnenberger <joerg@bec.de> parents: 46434 diff changeset	304
36968 95f4f1bfb650 revbranchcache: add a public function to update the data Boris Feld <boris.feld@octobus.net> parents: 36504 diff changeset	305 if branch in self._namesreverse:
95f4f1bfb650 revbranchcache: add a public function to update the data Boris Feld <boris.feld@octobus.net> parents: 36504 diff changeset	306 branchidx = self._namesreverse[branch]
95f4f1bfb650 revbranchcache: add a public function to update the data Boris Feld <boris.feld@octobus.net> parents: 36504 diff changeset	307 else:
95f4f1bfb650 revbranchcache: add a public function to update the data Boris Feld <boris.feld@octobus.net> parents: 36504 diff changeset	308 branchidx = len(self._names)
95f4f1bfb650 revbranchcache: add a public function to update the data Boris Feld <boris.feld@octobus.net> parents: 36504 diff changeset	309 self._names.append(branch)
95f4f1bfb650 revbranchcache: add a public function to update the data Boris Feld <boris.feld@octobus.net> parents: 36504 diff changeset	310 self._namesreverse[branch] = branchidx
95f4f1bfb650 revbranchcache: add a public function to update the data Boris Feld <boris.feld@octobus.net> parents: 36504 diff changeset	311 if close:
95f4f1bfb650 revbranchcache: add a public function to update the data Boris Feld <boris.feld@octobus.net> parents: 36504 diff changeset	312 branchidx \|= _rbccloseflag
46444 3e91d9978bec branchmap: update rev-branch-cache incrementally Joerg Sonnenberger <joerg@bec.de> parents: 46434 diff changeset	313 self._setcachedata(rev, self._repo.changelog.node(rev), branchidx)
36968 95f4f1bfb650 revbranchcache: add a public function to update the data Boris Feld <boris.feld@octobus.net> parents: 36504 diff changeset	314 # If no cache data were readable (non exists, bad permission, etc)
95f4f1bfb650 revbranchcache: add a public function to update the data Boris Feld <boris.feld@octobus.net> parents: 36504 diff changeset	315 # the cache was bypassing itself by setting:
95f4f1bfb650 revbranchcache: add a public function to update the data Boris Feld <boris.feld@octobus.net> parents: 36504 diff changeset	316 #
95f4f1bfb650 revbranchcache: add a public function to update the data Boris Feld <boris.feld@octobus.net> parents: 36504 diff changeset	317 # self.branchinfo = self._branchinfo
95f4f1bfb650 revbranchcache: add a public function to update the data Boris Feld <boris.feld@octobus.net> parents: 36504 diff changeset	318 #
95f4f1bfb650 revbranchcache: add a public function to update the data Boris Feld <boris.feld@octobus.net> parents: 36504 diff changeset	319 # Since we now have data in the cache, we need to drop this bypassing.
43554 9f70512ae2cf cleanup: remove pointless r-prefixes on single-quoted strings Augie Fackler <augie@google.com> parents: 43547 diff changeset	320 if 'branchinfo' in vars(self):
36968 95f4f1bfb650 revbranchcache: add a public function to update the data Boris Feld <boris.feld@octobus.net> parents: 36504 diff changeset	321 del self.branchinfo
95f4f1bfb650 revbranchcache: add a public function to update the data Boris Feld <boris.feld@octobus.net> parents: 36504 diff changeset	322
40494 5e5c8f2a1eb5 branchmap: do not specify changelog as an argument Yuya Nishihara <yuya@tcha.org> parents: 40375 diff changeset	323 def _setcachedata(self, rev, node, branchidx):
24375 fe255b2525d5 revbranchcache: move entry writing to a separate function Durham Goode <durham@fb.com> parents: 24374 diff changeset	324 """Writes the node's branch data to the in-memory cache data."""
31463 a5bad127128d branchmap: handle nullrev in setcachedata Durham Goode <durham@fb.com> parents: 31390 diff changeset	325 if rev == nullrev:
a5bad127128d branchmap: handle nullrev in setcachedata Durham Goode <durham@fb.com> parents: 31390 diff changeset	326 return
23785 cb99bacb9b4e branchcache: introduce revbranchcache for caching of revision branch names Mads Kiilerich <madski@unity3d.com> parents: 22357 diff changeset	327 rbcrevidx = rev * _rbcrecsize
51377 02e7d79edf62 branchmap: use mmap for faster revbranchcache loading Arseniy Alekseyev <aalekseyev@janestreet.com> parents: 51307 diff changeset	328 self._rbcrevs.pack_into(rbcrevidx, node, branchidx)
24376 203a078da052 revbranchcache: populate cache incrementally Durham Goode <durham@fb.com> parents: 24375 diff changeset	329 self._rbcrevslen = min(self._rbcrevslen, rev)
23785 cb99bacb9b4e branchcache: introduce revbranchcache for caching of revision branch names Mads Kiilerich <madski@unity3d.com> parents: 22357 diff changeset	330
24377 656f93ce66d5 revbranchcache: move cache writing to the transaction finalizer Durham Goode <durham@fb.com> parents: 24376 diff changeset	331 tr = self._repo.currenttransaction()
656f93ce66d5 revbranchcache: move cache writing to the transaction finalizer Durham Goode <durham@fb.com> parents: 24376 diff changeset	332 if tr:
43077 687b865b95ad formatting: byteify all mercurial/ and hgext/ string literals Augie Fackler <augie@google.com> parents: 43076 diff changeset	333 tr.addfinalize(b'write-revbranchcache', self.write)
24377 656f93ce66d5 revbranchcache: move cache writing to the transaction finalizer Durham Goode <durham@fb.com> parents: 24376 diff changeset	334
656f93ce66d5 revbranchcache: move cache writing to the transaction finalizer Durham Goode <durham@fb.com> parents: 24376 diff changeset	335 def write(self, tr=None):
23785 cb99bacb9b4e branchcache: introduce revbranchcache for caching of revision branch names Mads Kiilerich <madski@unity3d.com> parents: 22357 diff changeset	336 """Save branch cache if it is dirty."""
24374 77fd1fb538cd revbranchcache: store repo on the object Durham Goode <durham@fb.com> parents: 24373 diff changeset	337 repo = self._repo
29756 0d588332ad2c branchmap: acquires lock before writting the rev branch cache Pierre-Yves David <pierre-yves.david@ens-lyon.org> parents: 29755 diff changeset	338 wlock = None
43077 687b865b95ad formatting: byteify all mercurial/ and hgext/ string literals Augie Fackler <augie@google.com> parents: 43076 diff changeset	339 step = b''
29756 0d588332ad2c branchmap: acquires lock before writting the rev branch cache Pierre-Yves David <pierre-yves.david@ens-lyon.org> parents: 29755 diff changeset	340 try:
42222 09fd338522fa revbranchcache: factor logic to write names and revs in separate functions Pulkit Goyal <pulkit@yandex-team.ru> parents: 42138 diff changeset	341 # write the new names
51945 16efed18ae4e rev-branch-cache: schedule a write of the "v2" format if we read from "v1" Pierre-Yves David <pierre-yves.david@octobus.net> parents: 51944 diff changeset	342 if self._force_overwrite or self._rbcnamescount < len(self._names):
29756 0d588332ad2c branchmap: acquires lock before writting the rev branch cache Pierre-Yves David <pierre-yves.david@ens-lyon.org> parents: 29755 diff changeset	343 wlock = repo.wlock(wait=False)
43077 687b865b95ad formatting: byteify all mercurial/ and hgext/ string literals Augie Fackler <augie@google.com> parents: 43076 diff changeset	344 step = b' names'
42222 09fd338522fa revbranchcache: factor logic to write names and revs in separate functions Pulkit Goyal <pulkit@yandex-team.ru> parents: 42138 diff changeset	345 self._writenames(repo)
23785 cb99bacb9b4e branchcache: introduce revbranchcache for caching of revision branch names Mads Kiilerich <madski@unity3d.com> parents: 22357 diff changeset	346
42222 09fd338522fa revbranchcache: factor logic to write names and revs in separate functions Pulkit Goyal <pulkit@yandex-team.ru> parents: 42138 diff changeset	347 # write the new revs
29755 9f3c49ee4486 branchmap: preparatory indent of indent the branch rev writing code Pierre-Yves David <pierre-yves.david@ens-lyon.org> parents: 29615 diff changeset	348 start = self._rbcrevslen * _rbcrecsize
51940 9f7cf869e9f4 rev-branch-cache: add a way to force rewrite of the cache Pierre-Yves David <pierre-yves.david@octobus.net> parents: 51939 diff changeset	349 if self._force_overwrite or start != len(self._rbcrevs):
43077 687b865b95ad formatting: byteify all mercurial/ and hgext/ string literals Augie Fackler <augie@google.com> parents: 43076 diff changeset	350 step = b''
29756 0d588332ad2c branchmap: acquires lock before writting the rev branch cache Pierre-Yves David <pierre-yves.david@ens-lyon.org> parents: 29755 diff changeset	351 if wlock is None:
0d588332ad2c branchmap: acquires lock before writting the rev branch cache Pierre-Yves David <pierre-yves.david@ens-lyon.org> parents: 29755 diff changeset	352 wlock = repo.wlock(wait=False)
42222 09fd338522fa revbranchcache: factor logic to write names and revs in separate functions Pulkit Goyal <pulkit@yandex-team.ru> parents: 42138 diff changeset	353 self._writerevs(repo, start)
09fd338522fa revbranchcache: factor logic to write names and revs in separate functions Pulkit Goyal <pulkit@yandex-team.ru> parents: 42138 diff changeset	354
52665 24ee91ba9aa8 pyupgrade: drop usage of py3 aliases for `OSError` Matt Harbison <matt_harbison@yahoo.com> parents: 52122 diff changeset	355 except (OSError, error.Abort, error.LockError) as inst:
43076 2372284d9457 formatting: blacken the codebase Augie Fackler <augie@google.com> parents: 42603 diff changeset	356 repo.ui.debug(
43077 687b865b95ad formatting: byteify all mercurial/ and hgext/ string literals Augie Fackler <augie@google.com> parents: 43076 diff changeset	357 b"couldn't write revision branch cache%s: %s\n"
43076 2372284d9457 formatting: blacken the codebase Augie Fackler <augie@google.com> parents: 42603 diff changeset	358 % (step, stringutil.forcebytestr(inst))
2372284d9457 formatting: blacken the codebase Augie Fackler <augie@google.com> parents: 42603 diff changeset	359 )
29756 0d588332ad2c branchmap: acquires lock before writting the rev branch cache Pierre-Yves David <pierre-yves.david@ens-lyon.org> parents: 29755 diff changeset	360 finally:
0d588332ad2c branchmap: acquires lock before writting the rev branch cache Pierre-Yves David <pierre-yves.david@ens-lyon.org> parents: 29755 diff changeset	361 if wlock is not None:
0d588332ad2c branchmap: acquires lock before writting the rev branch cache Pierre-Yves David <pierre-yves.david@ens-lyon.org> parents: 29755 diff changeset	362 wlock.release()
42222 09fd338522fa revbranchcache: factor logic to write names and revs in separate functions Pulkit Goyal <pulkit@yandex-team.ru> parents: 42138 diff changeset	363
09fd338522fa revbranchcache: factor logic to write names and revs in separate functions Pulkit Goyal <pulkit@yandex-team.ru> parents: 42138 diff changeset	364 def _writenames(self, repo):
47031 f38bf44e077f black: make codebase compatible with black v21.4b2 and v20.8b1 Kyle Lippincott <spectral@google.com> parents: 46819 diff changeset	365 """write the new branch names to revbranchcache"""
51941 7032da075572 rev-branch-cache: make sure we close the name file we open Pierre-Yves David <pierre-yves.david@octobus.net> parents: 51940 diff changeset	366 f = None
51945 16efed18ae4e rev-branch-cache: schedule a write of the "v2" format if we read from "v1" Pierre-Yves David <pierre-yves.david@octobus.net> parents: 51944 diff changeset	367 if self._force_overwrite:
16efed18ae4e rev-branch-cache: schedule a write of the "v2" format if we read from "v1" Pierre-Yves David <pierre-yves.david@octobus.net> parents: 51944 diff changeset	368 self._rbcsnameslen = 0
16efed18ae4e rev-branch-cache: schedule a write of the "v2" format if we read from "v1" Pierre-Yves David <pierre-yves.david@octobus.net> parents: 51944 diff changeset	369 self._rbcnamescount = 0
51941 7032da075572 rev-branch-cache: make sure we close the name file we open Pierre-Yves David <pierre-yves.david@octobus.net> parents: 51940 diff changeset	370 try:
51945 16efed18ae4e rev-branch-cache: schedule a write of the "v2" format if we read from "v1" Pierre-Yves David <pierre-yves.david@octobus.net> parents: 51944 diff changeset	371 if self._force_overwrite or self._rbcnamescount != 0:
51941 7032da075572 rev-branch-cache: make sure we close the name file we open Pierre-Yves David <pierre-yves.david@octobus.net> parents: 51940 diff changeset	372 f = repo.cachevfs.open(_rbcnames, b'ab')
51945 16efed18ae4e rev-branch-cache: schedule a write of the "v2" format if we read from "v1" Pierre-Yves David <pierre-yves.david@octobus.net> parents: 51944 diff changeset	373 current_size = f.tell()
16efed18ae4e rev-branch-cache: schedule a write of the "v2" format if we read from "v1" Pierre-Yves David <pierre-yves.david@octobus.net> parents: 51944 diff changeset	374 if current_size == self._rbcsnameslen:
51941 7032da075572 rev-branch-cache: make sure we close the name file we open Pierre-Yves David <pierre-yves.david@octobus.net> parents: 51940 diff changeset	375 f.write(b'\0')
7032da075572 rev-branch-cache: make sure we close the name file we open Pierre-Yves David <pierre-yves.david@octobus.net> parents: 51940 diff changeset	376 else:
7032da075572 rev-branch-cache: make sure we close the name file we open Pierre-Yves David <pierre-yves.david@octobus.net> parents: 51940 diff changeset	377 f.close()
51945 16efed18ae4e rev-branch-cache: schedule a write of the "v2" format if we read from "v1" Pierre-Yves David <pierre-yves.david@octobus.net> parents: 51944 diff changeset	378 if self._force_overwrite:
16efed18ae4e rev-branch-cache: schedule a write of the "v2" format if we read from "v1" Pierre-Yves David <pierre-yves.david@octobus.net> parents: 51944 diff changeset	379 dbg = b"resetting content of %s\n"
16efed18ae4e rev-branch-cache: schedule a write of the "v2" format if we read from "v1" Pierre-Yves David <pierre-yves.david@octobus.net> parents: 51944 diff changeset	380 elif current_size > 0:
16efed18ae4e rev-branch-cache: schedule a write of the "v2" format if we read from "v1" Pierre-Yves David <pierre-yves.david@octobus.net> parents: 51944 diff changeset	381 dbg = b"%s changed - rewriting it\n"
16efed18ae4e rev-branch-cache: schedule a write of the "v2" format if we read from "v1" Pierre-Yves David <pierre-yves.david@octobus.net> parents: 51944 diff changeset	382 else:
16efed18ae4e rev-branch-cache: schedule a write of the "v2" format if we read from "v1" Pierre-Yves David <pierre-yves.david@octobus.net> parents: 51944 diff changeset	383 dbg = b"%s is missing - rewriting it\n"
16efed18ae4e rev-branch-cache: schedule a write of the "v2" format if we read from "v1" Pierre-Yves David <pierre-yves.david@octobus.net> parents: 51944 diff changeset	384 repo.ui.debug(dbg % _rbcnames)
51941 7032da075572 rev-branch-cache: make sure we close the name file we open Pierre-Yves David <pierre-yves.david@octobus.net> parents: 51940 diff changeset	385 self._rbcnamescount = 0
7032da075572 rev-branch-cache: make sure we close the name file we open Pierre-Yves David <pierre-yves.david@octobus.net> parents: 51940 diff changeset	386 self._rbcrevslen = 0
7032da075572 rev-branch-cache: make sure we close the name file we open Pierre-Yves David <pierre-yves.david@octobus.net> parents: 51940 diff changeset	387 if self._rbcnamescount == 0:
7032da075572 rev-branch-cache: make sure we close the name file we open Pierre-Yves David <pierre-yves.david@octobus.net> parents: 51940 diff changeset	388 # before rewriting names, make sure references are removed
7032da075572 rev-branch-cache: make sure we close the name file we open Pierre-Yves David <pierre-yves.david@octobus.net> parents: 51940 diff changeset	389 repo.cachevfs.unlinkpath(_rbcrevs, ignoremissing=True)
7032da075572 rev-branch-cache: make sure we close the name file we open Pierre-Yves David <pierre-yves.david@octobus.net> parents: 51940 diff changeset	390 f = repo.cachevfs.open(_rbcnames, b'wb')
7032da075572 rev-branch-cache: make sure we close the name file we open Pierre-Yves David <pierre-yves.david@octobus.net> parents: 51940 diff changeset	391 names = self._names[self._rbcnamescount :]
7032da075572 rev-branch-cache: make sure we close the name file we open Pierre-Yves David <pierre-yves.david@octobus.net> parents: 51940 diff changeset	392 from_local = encoding.fromlocal
7032da075572 rev-branch-cache: make sure we close the name file we open Pierre-Yves David <pierre-yves.david@octobus.net> parents: 51940 diff changeset	393 data = b'\0'.join(from_local(b) for b in names)
7032da075572 rev-branch-cache: make sure we close the name file we open Pierre-Yves David <pierre-yves.david@octobus.net> parents: 51940 diff changeset	394 f.write(data)
7032da075572 rev-branch-cache: make sure we close the name file we open Pierre-Yves David <pierre-yves.david@octobus.net> parents: 51940 diff changeset	395 self._rbcsnameslen = f.tell()
7032da075572 rev-branch-cache: make sure we close the name file we open Pierre-Yves David <pierre-yves.david@octobus.net> parents: 51940 diff changeset	396 finally:
7032da075572 rev-branch-cache: make sure we close the name file we open Pierre-Yves David <pierre-yves.david@octobus.net> parents: 51940 diff changeset	397 if f is not None:
42222 09fd338522fa revbranchcache: factor logic to write names and revs in separate functions Pulkit Goyal <pulkit@yandex-team.ru> parents: 42138 diff changeset	398 f.close()
09fd338522fa revbranchcache: factor logic to write names and revs in separate functions Pulkit Goyal <pulkit@yandex-team.ru> parents: 42138 diff changeset	399 self._rbcnamescount = len(self._names)
09fd338522fa revbranchcache: factor logic to write names and revs in separate functions Pulkit Goyal <pulkit@yandex-team.ru> parents: 42138 diff changeset	400
09fd338522fa revbranchcache: factor logic to write names and revs in separate functions Pulkit Goyal <pulkit@yandex-team.ru> parents: 42138 diff changeset	401 def _writerevs(self, repo, start):
47031 f38bf44e077f black: make codebase compatible with black v21.4b2 and v20.8b1 Kyle Lippincott <spectral@google.com> parents: 46819 diff changeset	402 """write the new revs to revbranchcache"""
42223 ececa45c80d8 revbranchcache: use context manager in _writerevs() to write to file Pulkit Goyal <pulkit@yandex-team.ru> parents: 42222 diff changeset	403 revs = min(len(repo.changelog), len(self._rbcrevs) // _rbcrecsize)
51942 c564be351754 rev-branch-cache: stop truncating cache file Pierre-Yves David <pierre-yves.david@octobus.net> parents: 51941 diff changeset	404
c564be351754 rev-branch-cache: stop truncating cache file Pierre-Yves David <pierre-yves.david@octobus.net> parents: 51941 diff changeset	405 end = revs * _rbcrecsize
51940 9f7cf869e9f4 rev-branch-cache: add a way to force rewrite of the cache Pierre-Yves David <pierre-yves.david@octobus.net> parents: 51939 diff changeset	406 if self._force_overwrite:
9f7cf869e9f4 rev-branch-cache: add a way to force rewrite of the cache Pierre-Yves David <pierre-yves.david@octobus.net> parents: 51939 diff changeset	407 start = 0
51942 c564be351754 rev-branch-cache: stop truncating cache file Pierre-Yves David <pierre-yves.david@octobus.net> parents: 51941 diff changeset	408
52005 76416b6e9d9b rev-branch-cache: properly ignores unaligned trailing data Pierre-Yves David <pierre-yves.david@octobus.net> parents: 52004 diff changeset	409 # align start on entry boundary
76416b6e9d9b rev-branch-cache: properly ignores unaligned trailing data Pierre-Yves David <pierre-yves.david@octobus.net> parents: 52004 diff changeset	410 start = _rbcrecsize * (start // _rbcrecsize)
76416b6e9d9b rev-branch-cache: properly ignores unaligned trailing data Pierre-Yves David <pierre-yves.david@octobus.net> parents: 52004 diff changeset	411
51942 c564be351754 rev-branch-cache: stop truncating cache file Pierre-Yves David <pierre-yves.david@octobus.net> parents: 51941 diff changeset	412 with repo.cachevfs.open(_rbcrevs, b'a+b') as f:
c564be351754 rev-branch-cache: stop truncating cache file Pierre-Yves David <pierre-yves.david@octobus.net> parents: 51941 diff changeset	413 pass # this make sure the file exist…
c564be351754 rev-branch-cache: stop truncating cache file Pierre-Yves David <pierre-yves.david@octobus.net> parents: 51941 diff changeset	414 with repo.cachevfs.open(_rbcrevs, b'r+b') as f:
c564be351754 rev-branch-cache: stop truncating cache file Pierre-Yves David <pierre-yves.david@octobus.net> parents: 51941 diff changeset	415 f.seek(0, os.SEEK_END)
51939 1eb2317c1762 rev-branch-cache: issue more truthful "truncating" message Pierre-Yves David <pierre-yves.david@octobus.net> parents: 51938 diff changeset	416 current_size = f.tell()
1eb2317c1762 rev-branch-cache: issue more truthful "truncating" message Pierre-Yves David <pierre-yves.david@octobus.net> parents: 51938 diff changeset	417 if current_size < start:
1eb2317c1762 rev-branch-cache: issue more truthful "truncating" message Pierre-Yves David <pierre-yves.david@octobus.net> parents: 51938 diff changeset	418 start = 0
1eb2317c1762 rev-branch-cache: issue more truthful "truncating" message Pierre-Yves David <pierre-yves.david@octobus.net> parents: 51938 diff changeset	419 if current_size != start:
51942 c564be351754 rev-branch-cache: stop truncating cache file Pierre-Yves David <pierre-yves.david@octobus.net> parents: 51941 diff changeset	420 threshold = current_size * REWRITE_RATIO
52004 4c885d5ff132 rev-branch-cache: stop pretending we will overwrite data when we don't Pierre-Yves David <pierre-yves.david@octobus.net> parents: 52003 diff changeset	421 overwritten = min(end, current_size) - start
4c885d5ff132 rev-branch-cache: stop pretending we will overwrite data when we don't Pierre-Yves David <pierre-yves.david@octobus.net> parents: 52003 diff changeset	422 if (max(end, current_size) - start) >= threshold:
4c885d5ff132 rev-branch-cache: stop pretending we will overwrite data when we don't Pierre-Yves David <pierre-yves.david@octobus.net> parents: 52003 diff changeset	423 start = 0
4c885d5ff132 rev-branch-cache: stop pretending we will overwrite data when we don't Pierre-Yves David <pierre-yves.david@octobus.net> parents: 52003 diff changeset	424 dbg = b"resetting content of cache/%s\n" % _rbcrevs
4c885d5ff132 rev-branch-cache: stop pretending we will overwrite data when we don't Pierre-Yves David <pierre-yves.david@octobus.net> parents: 52003 diff changeset	425 repo.ui.debug(dbg)
4c885d5ff132 rev-branch-cache: stop pretending we will overwrite data when we don't Pierre-Yves David <pierre-yves.david@octobus.net> parents: 52003 diff changeset	426 elif overwritten > 0:
4c885d5ff132 rev-branch-cache: stop pretending we will overwrite data when we don't Pierre-Yves David <pierre-yves.david@octobus.net> parents: 52003 diff changeset	427 # end affected, let us overwrite the bad value
51942 c564be351754 rev-branch-cache: stop truncating cache file Pierre-Yves David <pierre-yves.david@octobus.net> parents: 51941 diff changeset	428 dbg = b"overwriting %d bytes from %d in cache/%s"
52004 4c885d5ff132 rev-branch-cache: stop pretending we will overwrite data when we don't Pierre-Yves David <pierre-yves.david@octobus.net> parents: 52003 diff changeset	429 dbg %= (current_size - start, start, _rbcrevs)
51942 c564be351754 rev-branch-cache: stop truncating cache file Pierre-Yves David <pierre-yves.david@octobus.net> parents: 51941 diff changeset	430 if end < current_size:
c564be351754 rev-branch-cache: stop truncating cache file Pierre-Yves David <pierre-yves.david@octobus.net> parents: 51941 diff changeset	431 extra = b" leaving (%d trailing bytes)"
c564be351754 rev-branch-cache: stop truncating cache file Pierre-Yves David <pierre-yves.david@octobus.net> parents: 51941 diff changeset	432 extra %= current_size - end
c564be351754 rev-branch-cache: stop truncating cache file Pierre-Yves David <pierre-yves.david@octobus.net> parents: 51941 diff changeset	433 dbg += extra
c564be351754 rev-branch-cache: stop truncating cache file Pierre-Yves David <pierre-yves.david@octobus.net> parents: 51941 diff changeset	434 dbg += b'\n'
c564be351754 rev-branch-cache: stop truncating cache file Pierre-Yves David <pierre-yves.david@octobus.net> parents: 51941 diff changeset	435 repo.ui.debug(dbg)
c564be351754 rev-branch-cache: stop truncating cache file Pierre-Yves David <pierre-yves.david@octobus.net> parents: 51941 diff changeset	436 else:
52004 4c885d5ff132 rev-branch-cache: stop pretending we will overwrite data when we don't Pierre-Yves David <pierre-yves.david@octobus.net> parents: 52003 diff changeset	437 # extra untouched data at the end, lets warn about them
4c885d5ff132 rev-branch-cache: stop pretending we will overwrite data when we don't Pierre-Yves David <pierre-yves.david@octobus.net> parents: 52003 diff changeset	438 assert start == end # since don't write anything
4c885d5ff132 rev-branch-cache: stop pretending we will overwrite data when we don't Pierre-Yves David <pierre-yves.david@octobus.net> parents: 52003 diff changeset	439 dbg = b"cache/%s contains %d unknown trailing bytes\n"
4c885d5ff132 rev-branch-cache: stop pretending we will overwrite data when we don't Pierre-Yves David <pierre-yves.david@octobus.net> parents: 52003 diff changeset	440 dbg %= (_rbcrevs, current_size - start)
51942 c564be351754 rev-branch-cache: stop truncating cache file Pierre-Yves David <pierre-yves.david@octobus.net> parents: 51941 diff changeset	441 repo.ui.debug(dbg)
52004 4c885d5ff132 rev-branch-cache: stop pretending we will overwrite data when we don't Pierre-Yves David <pierre-yves.david@octobus.net> parents: 52003 diff changeset	442
51942 c564be351754 rev-branch-cache: stop truncating cache file Pierre-Yves David <pierre-yves.david@octobus.net> parents: 51941 diff changeset	443 if start > 0:
42222 09fd338522fa revbranchcache: factor logic to write names and revs in separate functions Pulkit Goyal <pulkit@yandex-team.ru> parents: 42138 diff changeset	444 f.seek(start)
51942 c564be351754 rev-branch-cache: stop truncating cache file Pierre-Yves David <pierre-yves.david@octobus.net> parents: 51941 diff changeset	445 f.write(self._rbcrevs.slice(start, end))
c564be351754 rev-branch-cache: stop truncating cache file Pierre-Yves David <pierre-yves.david@octobus.net> parents: 51941 diff changeset	446 else:
c564be351754 rev-branch-cache: stop truncating cache file Pierre-Yves David <pierre-yves.david@octobus.net> parents: 51941 diff changeset	447 f.close()
c564be351754 rev-branch-cache: stop truncating cache file Pierre-Yves David <pierre-yves.david@octobus.net> parents: 51941 diff changeset	448 with repo.cachevfs.open(
c564be351754 rev-branch-cache: stop truncating cache file Pierre-Yves David <pierre-yves.david@octobus.net> parents: 51941 diff changeset	449 _rbcrevs,
c564be351754 rev-branch-cache: stop truncating cache file Pierre-Yves David <pierre-yves.david@octobus.net> parents: 51941 diff changeset	450 b'wb',
c564be351754 rev-branch-cache: stop truncating cache file Pierre-Yves David <pierre-yves.david@octobus.net> parents: 51941 diff changeset	451 atomictemp=True,
c564be351754 rev-branch-cache: stop truncating cache file Pierre-Yves David <pierre-yves.david@octobus.net> parents: 51941 diff changeset	452 ) as rev_file:
c564be351754 rev-branch-cache: stop truncating cache file Pierre-Yves David <pierre-yves.david@octobus.net> parents: 51941 diff changeset	453 rev_file.write(self._rbcrevs.slice(start, end))
42222 09fd338522fa revbranchcache: factor logic to write names and revs in separate functions Pulkit Goyal <pulkit@yandex-team.ru> parents: 42138 diff changeset	454 self._rbcrevslen = revs
51940 9f7cf869e9f4 rev-branch-cache: add a way to force rewrite of the cache Pierre-Yves David <pierre-yves.david@octobus.net> parents: 51939 diff changeset	455 self._force_overwrite = False

Mercurial > public > mercurial-scm > hg-stable

annotate mercurial/branching/rev_cache.py @ 53040:cdd7bf612c7b stable tip