mercurial-scm/hg-stable: mercurial/branching/rev

annotate mercurial/branching/rev_cache.py @ 52005:76416b6e9d9b

rev-branch-cache: properly ignores unaligned trailing data Previously, trailing data could lead to crash and would be written back to disk, disaligning all new data? This is no longer the cases. This was detected while playing with branchmap-v3 that access the rev-branch-cache much more aggressively.

author	Pierre-Yves David <pierre-yves.david@octobus.net>
date	Fri, 27 Sep 2024 15:19:10 +0200
parents	4c885d5ff132
children	db1980a361cb

rev	line source
51938 f0e07efc199f rev-branch-cache: move the code in a dedicated module Pierre-Yves David <pierre-yves.david@octobus.net> parents: 51901 diff changeset	1 # rev_cache.py - caching branch information per revision
18116 bcee63733aad branchmap: create a mercurial.branchmap module Pierre-Yves David <pierre-yves.david@ens-lyon.org> parents: diff changeset	2 #
bcee63733aad branchmap: create a mercurial.branchmap module Pierre-Yves David <pierre-yves.david@ens-lyon.org> parents: diff changeset	3 # This software may be used and distributed according to the terms of the
bcee63733aad branchmap: create a mercurial.branchmap module Pierre-Yves David <pierre-yves.david@ens-lyon.org> parents: diff changeset	4 # GNU General Public License version 2 or any later version.
51901 f4733654f144 typing: add `from __future__ import annotations` to most files Matt Harbison <matt_harbison@yahoo.com> parents: 51651 diff changeset	5 from __future__ import annotations
25918 47f36e050c2e branchmap: use absolute_import Gregory Szorc <gregory.szorc@gmail.com> parents: 25660 diff changeset	6
51942 c564be351754 rev-branch-cache: stop truncating cache file Pierre-Yves David <pierre-yves.david@octobus.net> parents: 51941 diff changeset	7 import os
25918 47f36e050c2e branchmap: use absolute_import Gregory Szorc <gregory.szorc@gmail.com> parents: 25660 diff changeset	8 import struct
47f36e050c2e branchmap: use absolute_import Gregory Szorc <gregory.szorc@gmail.com> parents: 25660 diff changeset	9
51938 f0e07efc199f rev-branch-cache: move the code in a dedicated module Pierre-Yves David <pierre-yves.david@octobus.net> parents: 51901 diff changeset	10 from ..node import (
25918 47f36e050c2e branchmap: use absolute_import Gregory Szorc <gregory.szorc@gmail.com> parents: 25660 diff changeset	11 nullrev,
47f36e050c2e branchmap: use absolute_import Gregory Szorc <gregory.szorc@gmail.com> parents: 25660 diff changeset	12 )
51302 9d3721552b6c pytype: import typing directly Pierre-Yves David <pierre-yves.david@octobus.net> parents: 49308 diff changeset	13
51938 f0e07efc199f rev-branch-cache: move the code in a dedicated module Pierre-Yves David <pierre-yves.david@octobus.net> parents: 51901 diff changeset	14 from .. import (
25918 47f36e050c2e branchmap: use absolute_import Gregory Szorc <gregory.szorc@gmail.com> parents: 25660 diff changeset	15 encoding,
26587 56b2bcea2529 error: get Abort from 'error' instead of 'util' Pierre-Yves David <pierre-yves.david@fb.com> parents: 26460 diff changeset	16 error,
30995 22fbca1d11ed mercurial: switch to util.timer for all interval timings Simon Farnsworth <simonfar@fb.com> parents: 29758 diff changeset	17 util,
25918 47f36e050c2e branchmap: use absolute_import Gregory Szorc <gregory.szorc@gmail.com> parents: 25660 diff changeset	18 )
51302 9d3721552b6c pytype: import typing directly Pierre-Yves David <pierre-yves.david@octobus.net> parents: 49308 diff changeset	19
51938 f0e07efc199f rev-branch-cache: move the code in a dedicated module Pierre-Yves David <pierre-yves.david@octobus.net> parents: 51901 diff changeset	20 from ..utils import (
37087 f0b6fbea00cf stringutil: bulk-replace call sites to point to new module Yuya Nishihara <yuya@tcha.org> parents: 36968 diff changeset	21 stringutil,
f0b6fbea00cf stringutil: bulk-replace call sites to point to new module Yuya Nishihara <yuya@tcha.org> parents: 36968 diff changeset	22 )
25918 47f36e050c2e branchmap: use absolute_import Gregory Szorc <gregory.szorc@gmail.com> parents: 25660 diff changeset	23
47f36e050c2e branchmap: use absolute_import Gregory Szorc <gregory.szorc@gmail.com> parents: 25660 diff changeset	24 calcsize = struct.calcsize
31379 906be86990c4 rbc: use struct unpack_from and pack_into instead of unpack and pack Mads Kiilerich <madski@unity3d.com> parents: 31369 diff changeset	25 pack_into = struct.pack_into
906be86990c4 rbc: use struct unpack_from and pack_into instead of unpack and pack Mads Kiilerich <madski@unity3d.com> parents: 31369 diff changeset	26 unpack_from = struct.unpack_from
18117 526e7ec5c96e branchmap: extract write logic from localrepo Pierre-Yves David <pierre-yves.david@ens-lyon.org> parents: 18116 diff changeset	27
18118 e70ff1e599f4 branchmap: extract read logic from repo Pierre-Yves David <pierre-yves.david@logilab.fr> parents: 18117 diff changeset	28
23785 cb99bacb9b4e branchcache: introduce revbranchcache for caching of revision branch names Mads Kiilerich <madski@unity3d.com> parents: 22357 diff changeset	29 # Revision branch info cache
cb99bacb9b4e branchcache: introduce revbranchcache for caching of revision branch names Mads Kiilerich <madski@unity3d.com> parents: 22357 diff changeset	30
51943 0f26ee69cf36 rev-branch-cache: increment the version to "v2" Pierre-Yves David <pierre-yves.david@octobus.net> parents: 51942 diff changeset	31 # The "V2" version use the same format as the "V1" but garantee it won't be
0f26ee69cf36 rev-branch-cache: increment the version to "v2" Pierre-Yves David <pierre-yves.david@octobus.net> parents: 51942 diff changeset	32 # truncated, preventing SIGBUS when it is mmap-ed
0f26ee69cf36 rev-branch-cache: increment the version to "v2" Pierre-Yves David <pierre-yves.david@octobus.net> parents: 51942 diff changeset	33 _rbcversion = b'-v2'
43077 687b865b95ad formatting: byteify all mercurial/ and hgext/ string literals Augie Fackler <augie@google.com> parents: 43076 diff changeset	34 _rbcnames = b'rbc-names' + _rbcversion
687b865b95ad formatting: byteify all mercurial/ and hgext/ string literals Augie Fackler <augie@google.com> parents: 43076 diff changeset	35 _rbcrevs = b'rbc-revs' + _rbcversion
51944 bd7359c18d69 rev-branch-cache: fallback on "v1" data if no v2 is found Pierre-Yves David <pierre-yves.david@octobus.net> parents: 51943 diff changeset	36 _rbc_legacy_version = b'-v1'
bd7359c18d69 rev-branch-cache: fallback on "v1" data if no v2 is found Pierre-Yves David <pierre-yves.david@octobus.net> parents: 51943 diff changeset	37 _rbc_legacy_names = b'rbc-names' + _rbc_legacy_version
bd7359c18d69 rev-branch-cache: fallback on "v1" data if no v2 is found Pierre-Yves David <pierre-yves.david@octobus.net> parents: 51943 diff changeset	38 _rbc_legacy_revs = b'rbc-revs' + _rbc_legacy_version
23785 cb99bacb9b4e branchcache: introduce revbranchcache for caching of revision branch names Mads Kiilerich <madski@unity3d.com> parents: 22357 diff changeset	39 # [4 byte hash prefix][4 byte branch name number with sign bit indicating open]
43077 687b865b95ad formatting: byteify all mercurial/ and hgext/ string literals Augie Fackler <augie@google.com> parents: 43076 diff changeset	40 _rbcrecfmt = b'>4sI'
23785 cb99bacb9b4e branchcache: introduce revbranchcache for caching of revision branch names Mads Kiilerich <madski@unity3d.com> parents: 22357 diff changeset	41 _rbcrecsize = calcsize(_rbcrecfmt)
46434 1726a53a8494 reverse-branch-cache: switch to doubling allocating scheme Joerg Sonnenberger <joerg@bec.de> parents: 46254 diff changeset	42 _rbcmininc = 64 * _rbcrecsize
23785 cb99bacb9b4e branchcache: introduce revbranchcache for caching of revision branch names Mads Kiilerich <madski@unity3d.com> parents: 22357 diff changeset	43 _rbcnodelen = 4
43076 2372284d9457 formatting: blacken the codebase Augie Fackler <augie@google.com> parents: 42603 diff changeset	44 _rbcbranchidxmask = 0x7FFFFFFF
23785 cb99bacb9b4e branchcache: introduce revbranchcache for caching of revision branch names Mads Kiilerich <madski@unity3d.com> parents: 22357 diff changeset	45 _rbccloseflag = 0x80000000
cb99bacb9b4e branchcache: introduce revbranchcache for caching of revision branch names Mads Kiilerich <madski@unity3d.com> parents: 22357 diff changeset	46
43076 2372284d9457 formatting: blacken the codebase Augie Fackler <augie@google.com> parents: 42603 diff changeset	47
51942 c564be351754 rev-branch-cache: stop truncating cache file Pierre-Yves David <pierre-yves.david@octobus.net> parents: 51941 diff changeset	48 # with atomic replacement.
c564be351754 rev-branch-cache: stop truncating cache file Pierre-Yves David <pierre-yves.david@octobus.net> parents: 51941 diff changeset	49 REWRITE_RATIO = 0.2
c564be351754 rev-branch-cache: stop truncating cache file Pierre-Yves David <pierre-yves.david@octobus.net> parents: 51941 diff changeset	50
c564be351754 rev-branch-cache: stop truncating cache file Pierre-Yves David <pierre-yves.david@octobus.net> parents: 51941 diff changeset	51
51377 02e7d79edf62 branchmap: use mmap for faster revbranchcache loading Arseniy Alekseyev <aalekseyev@janestreet.com> parents: 51307 diff changeset	52 class rbcrevs:
02e7d79edf62 branchmap: use mmap for faster revbranchcache loading Arseniy Alekseyev <aalekseyev@janestreet.com> parents: 51307 diff changeset	53 """a byte string consisting of an immutable prefix followed by a mutable suffix"""
02e7d79edf62 branchmap: use mmap for faster revbranchcache loading Arseniy Alekseyev <aalekseyev@janestreet.com> parents: 51307 diff changeset	54
02e7d79edf62 branchmap: use mmap for faster revbranchcache loading Arseniy Alekseyev <aalekseyev@janestreet.com> parents: 51307 diff changeset	55 def __init__(self, revs):
02e7d79edf62 branchmap: use mmap for faster revbranchcache loading Arseniy Alekseyev <aalekseyev@janestreet.com> parents: 51307 diff changeset	56 self._prefix = revs
02e7d79edf62 branchmap: use mmap for faster revbranchcache loading Arseniy Alekseyev <aalekseyev@janestreet.com> parents: 51307 diff changeset	57 self._rest = bytearray()
02e7d79edf62 branchmap: use mmap for faster revbranchcache loading Arseniy Alekseyev <aalekseyev@janestreet.com> parents: 51307 diff changeset	58
52005 76416b6e9d9b rev-branch-cache: properly ignores unaligned trailing data Pierre-Yves David <pierre-yves.david@octobus.net> parents: 52004 diff changeset	59 @property
76416b6e9d9b rev-branch-cache: properly ignores unaligned trailing data Pierre-Yves David <pierre-yves.david@octobus.net> parents: 52004 diff changeset	60 def len_prefix(self):
76416b6e9d9b rev-branch-cache: properly ignores unaligned trailing data Pierre-Yves David <pierre-yves.david@octobus.net> parents: 52004 diff changeset	61 size = len(self._prefix)
76416b6e9d9b rev-branch-cache: properly ignores unaligned trailing data Pierre-Yves David <pierre-yves.david@octobus.net> parents: 52004 diff changeset	62 return size - (size % _rbcrecsize)
76416b6e9d9b rev-branch-cache: properly ignores unaligned trailing data Pierre-Yves David <pierre-yves.david@octobus.net> parents: 52004 diff changeset	63
51377 02e7d79edf62 branchmap: use mmap for faster revbranchcache loading Arseniy Alekseyev <aalekseyev@janestreet.com> parents: 51307 diff changeset	64 def __len__(self):
52005 76416b6e9d9b rev-branch-cache: properly ignores unaligned trailing data Pierre-Yves David <pierre-yves.david@octobus.net> parents: 52004 diff changeset	65 return self.len_prefix + len(self._rest)
51377 02e7d79edf62 branchmap: use mmap for faster revbranchcache loading Arseniy Alekseyev <aalekseyev@janestreet.com> parents: 51307 diff changeset	66
02e7d79edf62 branchmap: use mmap for faster revbranchcache loading Arseniy Alekseyev <aalekseyev@janestreet.com> parents: 51307 diff changeset	67 def unpack_record(self, rbcrevidx):
52005 76416b6e9d9b rev-branch-cache: properly ignores unaligned trailing data Pierre-Yves David <pierre-yves.david@octobus.net> parents: 52004 diff changeset	68 if rbcrevidx < self.len_prefix:
51377 02e7d79edf62 branchmap: use mmap for faster revbranchcache loading Arseniy Alekseyev <aalekseyev@janestreet.com> parents: 51307 diff changeset	69 return unpack_from(_rbcrecfmt, util.buffer(self._prefix), rbcrevidx)
02e7d79edf62 branchmap: use mmap for faster revbranchcache loading Arseniy Alekseyev <aalekseyev@janestreet.com> parents: 51307 diff changeset	70 else:
02e7d79edf62 branchmap: use mmap for faster revbranchcache loading Arseniy Alekseyev <aalekseyev@janestreet.com> parents: 51307 diff changeset	71 return unpack_from(
02e7d79edf62 branchmap: use mmap for faster revbranchcache loading Arseniy Alekseyev <aalekseyev@janestreet.com> parents: 51307 diff changeset	72 _rbcrecfmt,
02e7d79edf62 branchmap: use mmap for faster revbranchcache loading Arseniy Alekseyev <aalekseyev@janestreet.com> parents: 51307 diff changeset	73 util.buffer(self._rest),
52005 76416b6e9d9b rev-branch-cache: properly ignores unaligned trailing data Pierre-Yves David <pierre-yves.david@octobus.net> parents: 52004 diff changeset	74 rbcrevidx - self.len_prefix,
51377 02e7d79edf62 branchmap: use mmap for faster revbranchcache loading Arseniy Alekseyev <aalekseyev@janestreet.com> parents: 51307 diff changeset	75 )
02e7d79edf62 branchmap: use mmap for faster revbranchcache loading Arseniy Alekseyev <aalekseyev@janestreet.com> parents: 51307 diff changeset	76
02e7d79edf62 branchmap: use mmap for faster revbranchcache loading Arseniy Alekseyev <aalekseyev@janestreet.com> parents: 51307 diff changeset	77 def make_mutable(self):
52005 76416b6e9d9b rev-branch-cache: properly ignores unaligned trailing data Pierre-Yves David <pierre-yves.david@octobus.net> parents: 52004 diff changeset	78 if self.len_prefix > 0:
51377 02e7d79edf62 branchmap: use mmap for faster revbranchcache loading Arseniy Alekseyev <aalekseyev@janestreet.com> parents: 51307 diff changeset	79 entirety = bytearray()
52005 76416b6e9d9b rev-branch-cache: properly ignores unaligned trailing data Pierre-Yves David <pierre-yves.david@octobus.net> parents: 52004 diff changeset	80 entirety[:] = self._prefix[: self.len_prefix]
51377 02e7d79edf62 branchmap: use mmap for faster revbranchcache loading Arseniy Alekseyev <aalekseyev@janestreet.com> parents: 51307 diff changeset	81 entirety.extend(self._rest)
02e7d79edf62 branchmap: use mmap for faster revbranchcache loading Arseniy Alekseyev <aalekseyev@janestreet.com> parents: 51307 diff changeset	82 self._rest = entirety
02e7d79edf62 branchmap: use mmap for faster revbranchcache loading Arseniy Alekseyev <aalekseyev@janestreet.com> parents: 51307 diff changeset	83 self._prefix = bytearray()
02e7d79edf62 branchmap: use mmap for faster revbranchcache loading Arseniy Alekseyev <aalekseyev@janestreet.com> parents: 51307 diff changeset	84
02e7d79edf62 branchmap: use mmap for faster revbranchcache loading Arseniy Alekseyev <aalekseyev@janestreet.com> parents: 51307 diff changeset	85 def truncate(self, pos):
02e7d79edf62 branchmap: use mmap for faster revbranchcache loading Arseniy Alekseyev <aalekseyev@janestreet.com> parents: 51307 diff changeset	86 self.make_mutable()
02e7d79edf62 branchmap: use mmap for faster revbranchcache loading Arseniy Alekseyev <aalekseyev@janestreet.com> parents: 51307 diff changeset	87 del self._rest[pos:]
02e7d79edf62 branchmap: use mmap for faster revbranchcache loading Arseniy Alekseyev <aalekseyev@janestreet.com> parents: 51307 diff changeset	88
02e7d79edf62 branchmap: use mmap for faster revbranchcache loading Arseniy Alekseyev <aalekseyev@janestreet.com> parents: 51307 diff changeset	89 def pack_into(self, rbcrevidx, node, branchidx):
52005 76416b6e9d9b rev-branch-cache: properly ignores unaligned trailing data Pierre-Yves David <pierre-yves.david@octobus.net> parents: 52004 diff changeset	90 if rbcrevidx < self.len_prefix:
51377 02e7d79edf62 branchmap: use mmap for faster revbranchcache loading Arseniy Alekseyev <aalekseyev@janestreet.com> parents: 51307 diff changeset	91 self.make_mutable()
02e7d79edf62 branchmap: use mmap for faster revbranchcache loading Arseniy Alekseyev <aalekseyev@janestreet.com> parents: 51307 diff changeset	92 buf = self._rest
52005 76416b6e9d9b rev-branch-cache: properly ignores unaligned trailing data Pierre-Yves David <pierre-yves.david@octobus.net> parents: 52004 diff changeset	93 start_offset = rbcrevidx - self.len_prefix
51377 02e7d79edf62 branchmap: use mmap for faster revbranchcache loading Arseniy Alekseyev <aalekseyev@janestreet.com> parents: 51307 diff changeset	94 end_offset = start_offset + _rbcrecsize
02e7d79edf62 branchmap: use mmap for faster revbranchcache loading Arseniy Alekseyev <aalekseyev@janestreet.com> parents: 51307 diff changeset	95
02e7d79edf62 branchmap: use mmap for faster revbranchcache loading Arseniy Alekseyev <aalekseyev@janestreet.com> parents: 51307 diff changeset	96 if len(self._rest) < end_offset:
02e7d79edf62 branchmap: use mmap for faster revbranchcache loading Arseniy Alekseyev <aalekseyev@janestreet.com> parents: 51307 diff changeset	97 # bytearray doesn't allocate extra space at least in Python 3.7.
02e7d79edf62 branchmap: use mmap for faster revbranchcache loading Arseniy Alekseyev <aalekseyev@janestreet.com> parents: 51307 diff changeset	98 # When multiple changesets are added in a row, precise resize would
02e7d79edf62 branchmap: use mmap for faster revbranchcache loading Arseniy Alekseyev <aalekseyev@janestreet.com> parents: 51307 diff changeset	99 # result in quadratic complexity. Overallocate to compensate by
02e7d79edf62 branchmap: use mmap for faster revbranchcache loading Arseniy Alekseyev <aalekseyev@janestreet.com> parents: 51307 diff changeset	100 # using the classic doubling technique for dynamic arrays instead.
02e7d79edf62 branchmap: use mmap for faster revbranchcache loading Arseniy Alekseyev <aalekseyev@janestreet.com> parents: 51307 diff changeset	101 # If there was a gap in the map before, less space will be reserved.
02e7d79edf62 branchmap: use mmap for faster revbranchcache loading Arseniy Alekseyev <aalekseyev@janestreet.com> parents: 51307 diff changeset	102 self._rest.extend(b'\0' * end_offset)
02e7d79edf62 branchmap: use mmap for faster revbranchcache loading Arseniy Alekseyev <aalekseyev@janestreet.com> parents: 51307 diff changeset	103 return pack_into(
02e7d79edf62 branchmap: use mmap for faster revbranchcache loading Arseniy Alekseyev <aalekseyev@janestreet.com> parents: 51307 diff changeset	104 _rbcrecfmt,
02e7d79edf62 branchmap: use mmap for faster revbranchcache loading Arseniy Alekseyev <aalekseyev@janestreet.com> parents: 51307 diff changeset	105 buf,
02e7d79edf62 branchmap: use mmap for faster revbranchcache loading Arseniy Alekseyev <aalekseyev@janestreet.com> parents: 51307 diff changeset	106 start_offset,
02e7d79edf62 branchmap: use mmap for faster revbranchcache loading Arseniy Alekseyev <aalekseyev@janestreet.com> parents: 51307 diff changeset	107 node,
02e7d79edf62 branchmap: use mmap for faster revbranchcache loading Arseniy Alekseyev <aalekseyev@janestreet.com> parents: 51307 diff changeset	108 branchidx,
02e7d79edf62 branchmap: use mmap for faster revbranchcache loading Arseniy Alekseyev <aalekseyev@janestreet.com> parents: 51307 diff changeset	109 )
02e7d79edf62 branchmap: use mmap for faster revbranchcache loading Arseniy Alekseyev <aalekseyev@janestreet.com> parents: 51307 diff changeset	110
02e7d79edf62 branchmap: use mmap for faster revbranchcache loading Arseniy Alekseyev <aalekseyev@janestreet.com> parents: 51307 diff changeset	111 def extend(self, extension):
02e7d79edf62 branchmap: use mmap for faster revbranchcache loading Arseniy Alekseyev <aalekseyev@janestreet.com> parents: 51307 diff changeset	112 return self._rest.extend(extension)
02e7d79edf62 branchmap: use mmap for faster revbranchcache loading Arseniy Alekseyev <aalekseyev@janestreet.com> parents: 51307 diff changeset	113
02e7d79edf62 branchmap: use mmap for faster revbranchcache loading Arseniy Alekseyev <aalekseyev@janestreet.com> parents: 51307 diff changeset	114 def slice(self, begin, end):
52005 76416b6e9d9b rev-branch-cache: properly ignores unaligned trailing data Pierre-Yves David <pierre-yves.david@octobus.net> parents: 52004 diff changeset	115 if begin < self.len_prefix:
51377 02e7d79edf62 branchmap: use mmap for faster revbranchcache loading Arseniy Alekseyev <aalekseyev@janestreet.com> parents: 51307 diff changeset	116 acc = bytearray()
52005 76416b6e9d9b rev-branch-cache: properly ignores unaligned trailing data Pierre-Yves David <pierre-yves.david@octobus.net> parents: 52004 diff changeset	117 acc[:] = self._prefix[begin : min(end, self.len_prefix)]
51377 02e7d79edf62 branchmap: use mmap for faster revbranchcache loading Arseniy Alekseyev <aalekseyev@janestreet.com> parents: 51307 diff changeset	118 acc.extend(
52005 76416b6e9d9b rev-branch-cache: properly ignores unaligned trailing data Pierre-Yves David <pierre-yves.david@octobus.net> parents: 52004 diff changeset	119 self._rest[begin - self.len_prefix : end - self.len_prefix]
51377 02e7d79edf62 branchmap: use mmap for faster revbranchcache loading Arseniy Alekseyev <aalekseyev@janestreet.com> parents: 51307 diff changeset	120 )
02e7d79edf62 branchmap: use mmap for faster revbranchcache loading Arseniy Alekseyev <aalekseyev@janestreet.com> parents: 51307 diff changeset	121 return acc
52005 76416b6e9d9b rev-branch-cache: properly ignores unaligned trailing data Pierre-Yves David <pierre-yves.david@octobus.net> parents: 52004 diff changeset	122 return self._rest[begin - self.len_prefix : end - self.len_prefix]
51377 02e7d79edf62 branchmap: use mmap for faster revbranchcache loading Arseniy Alekseyev <aalekseyev@janestreet.com> parents: 51307 diff changeset	123
02e7d79edf62 branchmap: use mmap for faster revbranchcache loading Arseniy Alekseyev <aalekseyev@janestreet.com> parents: 51307 diff changeset	124
49037 642e31cb55f0 py3: use class X: instead of class X(object): Gregory Szorc <gregory.szorc@gmail.com> parents: 49026 diff changeset	125 class revbranchcache:
23785 cb99bacb9b4e branchcache: introduce revbranchcache for caching of revision branch names Mads Kiilerich <madski@unity3d.com> parents: 22357 diff changeset	126 """Persistent cache, mapping from revision number to branch name and close.
cb99bacb9b4e branchcache: introduce revbranchcache for caching of revision branch names Mads Kiilerich <madski@unity3d.com> parents: 22357 diff changeset	127 This is a low level cache, independent of filtering.
cb99bacb9b4e branchcache: introduce revbranchcache for caching of revision branch names Mads Kiilerich <madski@unity3d.com> parents: 22357 diff changeset	128
cb99bacb9b4e branchcache: introduce revbranchcache for caching of revision branch names Mads Kiilerich <madski@unity3d.com> parents: 22357 diff changeset	129 Branch names are stored in rbc-names in internal encoding separated by 0.
cb99bacb9b4e branchcache: introduce revbranchcache for caching of revision branch names Mads Kiilerich <madski@unity3d.com> parents: 22357 diff changeset	130 rbc-names is append-only, and each branch name is only stored once and will
cb99bacb9b4e branchcache: introduce revbranchcache for caching of revision branch names Mads Kiilerich <madski@unity3d.com> parents: 22357 diff changeset	131 thus have a unique index.
cb99bacb9b4e branchcache: introduce revbranchcache for caching of revision branch names Mads Kiilerich <madski@unity3d.com> parents: 22357 diff changeset	132
cb99bacb9b4e branchcache: introduce revbranchcache for caching of revision branch names Mads Kiilerich <madski@unity3d.com> parents: 22357 diff changeset	133 The branch info for each revision is stored in rbc-revs as constant size
cb99bacb9b4e branchcache: introduce revbranchcache for caching of revision branch names Mads Kiilerich <madski@unity3d.com> parents: 22357 diff changeset	134 records. The whole file is read into memory, but it is only 'parsed' on
cb99bacb9b4e branchcache: introduce revbranchcache for caching of revision branch names Mads Kiilerich <madski@unity3d.com> parents: 22357 diff changeset	135 demand. The file is usually append-only but will be truncated if repo
cb99bacb9b4e branchcache: introduce revbranchcache for caching of revision branch names Mads Kiilerich <madski@unity3d.com> parents: 22357 diff changeset	136 modification is detected.
cb99bacb9b4e branchcache: introduce revbranchcache for caching of revision branch names Mads Kiilerich <madski@unity3d.com> parents: 22357 diff changeset	137 The record for each revision contains the first 4 bytes of the
cb99bacb9b4e branchcache: introduce revbranchcache for caching of revision branch names Mads Kiilerich <madski@unity3d.com> parents: 22357 diff changeset	138 corresponding node hash, and the record is only used if it still matches.
cb99bacb9b4e branchcache: introduce revbranchcache for caching of revision branch names Mads Kiilerich <madski@unity3d.com> parents: 22357 diff changeset	139 Even a completely trashed rbc-revs fill thus still give the right result
cb99bacb9b4e branchcache: introduce revbranchcache for caching of revision branch names Mads Kiilerich <madski@unity3d.com> parents: 22357 diff changeset	140 while converging towards full recovery ... assuming no incorrectly matching
cb99bacb9b4e branchcache: introduce revbranchcache for caching of revision branch names Mads Kiilerich <madski@unity3d.com> parents: 22357 diff changeset	141 node hashes.
cb99bacb9b4e branchcache: introduce revbranchcache for caching of revision branch names Mads Kiilerich <madski@unity3d.com> parents: 22357 diff changeset	142 The record also contains 4 bytes where 31 bits contains the index of the
cb99bacb9b4e branchcache: introduce revbranchcache for caching of revision branch names Mads Kiilerich <madski@unity3d.com> parents: 22357 diff changeset	143 branch and the last bit indicate that it is a branch close commit.
cb99bacb9b4e branchcache: introduce revbranchcache for caching of revision branch names Mads Kiilerich <madski@unity3d.com> parents: 22357 diff changeset	144 The usage pattern for rbc-revs is thus somewhat similar to 00changelog.i
cb99bacb9b4e branchcache: introduce revbranchcache for caching of revision branch names Mads Kiilerich <madski@unity3d.com> parents: 22357 diff changeset	145 and will grow with it but be 1/8th of its size.
cb99bacb9b4e branchcache: introduce revbranchcache for caching of revision branch names Mads Kiilerich <madski@unity3d.com> parents: 22357 diff changeset	146 """
cb99bacb9b4e branchcache: introduce revbranchcache for caching of revision branch names Mads Kiilerich <madski@unity3d.com> parents: 22357 diff changeset	147
24159 5b4ed033390b revisionbranchcache: fall back to slow path if starting readonly (issue4531) Mads Kiilerich <madski@unity3d.com> parents: 23877 diff changeset	148 def __init__(self, repo, readonly=True):
23785 cb99bacb9b4e branchcache: introduce revbranchcache for caching of revision branch names Mads Kiilerich <madski@unity3d.com> parents: 22357 diff changeset	149 assert repo.filtername is None
24374 77fd1fb538cd revbranchcache: store repo on the object Durham Goode <durham@fb.com> parents: 24373 diff changeset	150 self._repo = repo
43076 2372284d9457 formatting: blacken the codebase Augie Fackler <augie@google.com> parents: 42603 diff changeset	151 self._names = [] # branch names in local encoding with static index
51377 02e7d79edf62 branchmap: use mmap for faster revbranchcache loading Arseniy Alekseyev <aalekseyev@janestreet.com> parents: 51307 diff changeset	152 self._rbcrevs = rbcrevs(bytearray())
43076 2372284d9457 formatting: blacken the codebase Augie Fackler <augie@google.com> parents: 42603 diff changeset	153 self._rbcsnameslen = 0 # length of names read at _rbcsnameslen
51945 16efed18ae4e rev-branch-cache: schedule a write of the "v2" format if we read from "v1" Pierre-Yves David <pierre-yves.david@octobus.net> parents: 51944 diff changeset	154 self._force_overwrite = False
51944 bd7359c18d69 rev-branch-cache: fallback on "v1" data if no v2 is found Pierre-Yves David <pierre-yves.david@octobus.net> parents: 51943 diff changeset	155 v1_fallback = False
23785 cb99bacb9b4e branchcache: introduce revbranchcache for caching of revision branch names Mads Kiilerich <madski@unity3d.com> parents: 22357 diff changeset	156 try:
51944 bd7359c18d69 rev-branch-cache: fallback on "v1" data if no v2 is found Pierre-Yves David <pierre-yves.david@octobus.net> parents: 51943 diff changeset	157 try:
bd7359c18d69 rev-branch-cache: fallback on "v1" data if no v2 is found Pierre-Yves David <pierre-yves.david@octobus.net> parents: 51943 diff changeset	158 bndata = repo.cachevfs.read(_rbcnames)
bd7359c18d69 rev-branch-cache: fallback on "v1" data if no v2 is found Pierre-Yves David <pierre-yves.david@octobus.net> parents: 51943 diff changeset	159 except (IOError, OSError):
bd7359c18d69 rev-branch-cache: fallback on "v1" data if no v2 is found Pierre-Yves David <pierre-yves.david@octobus.net> parents: 51943 diff changeset	160 # If we don't have "v2" data, we might have "v1" data worth
bd7359c18d69 rev-branch-cache: fallback on "v1" data if no v2 is found Pierre-Yves David <pierre-yves.david@octobus.net> parents: 51943 diff changeset	161 # using.
bd7359c18d69 rev-branch-cache: fallback on "v1" data if no v2 is found Pierre-Yves David <pierre-yves.david@octobus.net> parents: 51943 diff changeset	162 #
bd7359c18d69 rev-branch-cache: fallback on "v1" data if no v2 is found Pierre-Yves David <pierre-yves.david@octobus.net> parents: 51943 diff changeset	163 # consider stop doing this many version after hg-6.9 release
bd7359c18d69 rev-branch-cache: fallback on "v1" data if no v2 is found Pierre-Yves David <pierre-yves.david@octobus.net> parents: 51943 diff changeset	164 bndata = repo.cachevfs.read(_rbc_legacy_names)
bd7359c18d69 rev-branch-cache: fallback on "v1" data if no v2 is found Pierre-Yves David <pierre-yves.david@octobus.net> parents: 51943 diff changeset	165 v1_fallback = True
51945 16efed18ae4e rev-branch-cache: schedule a write of the "v2" format if we read from "v1" Pierre-Yves David <pierre-yves.david@octobus.net> parents: 51944 diff changeset	166 self._force_overwrite = True
43076 2372284d9457 formatting: blacken the codebase Augie Fackler <augie@google.com> parents: 42603 diff changeset	167 self._rbcsnameslen = len(bndata) # for verification before writing
31380 7dd2f51f38ac rbc: empty (and invalid) rbc-names file should give an empty name list Mads Kiilerich <mads@kiilerich.com> parents: 31379 diff changeset	168 if bndata:
43076 2372284d9457 formatting: blacken the codebase Augie Fackler <augie@google.com> parents: 42603 diff changeset	169 self._names = [
43077 687b865b95ad formatting: byteify all mercurial/ and hgext/ string literals Augie Fackler <augie@google.com> parents: 43076 diff changeset	170 encoding.tolocal(bn) for bn in bndata.split(b'\0')
43076 2372284d9457 formatting: blacken the codebase Augie Fackler <augie@google.com> parents: 42603 diff changeset	171 ]
29423 d2c6f3a948fa branchmap: remove unused exception variable Gregory Szorc <gregory.szorc@gmail.com> parents: 28558 diff changeset	172 except (IOError, OSError):
24159 5b4ed033390b revisionbranchcache: fall back to slow path if starting readonly (issue4531) Mads Kiilerich <madski@unity3d.com> parents: 23877 diff changeset	173 if readonly:
5b4ed033390b revisionbranchcache: fall back to slow path if starting readonly (issue4531) Mads Kiilerich <madski@unity3d.com> parents: 23877 diff changeset	174 # don't try to use cache - fall back to the slow path
5b4ed033390b revisionbranchcache: fall back to slow path if starting readonly (issue4531) Mads Kiilerich <madski@unity3d.com> parents: 23877 diff changeset	175 self.branchinfo = self._branchinfo
5b4ed033390b revisionbranchcache: fall back to slow path if starting readonly (issue4531) Mads Kiilerich <madski@unity3d.com> parents: 23877 diff changeset	176
23785 cb99bacb9b4e branchcache: introduce revbranchcache for caching of revision branch names Mads Kiilerich <madski@unity3d.com> parents: 22357 diff changeset	177 if self._names:
cb99bacb9b4e branchcache: introduce revbranchcache for caching of revision branch names Mads Kiilerich <madski@unity3d.com> parents: 22357 diff changeset	178 try:
51651 c0e30a019ce1 mmap: only use mmap to read rev-branch-cache data if it is safe Pierre-Yves David <pierre-yves.david@octobus.net> parents: 51644 diff changeset	179 usemmap = repo.ui.configbool(b'storage', b'revbranchcache.mmap')
51944 bd7359c18d69 rev-branch-cache: fallback on "v1" data if no v2 is found Pierre-Yves David <pierre-yves.david@octobus.net> parents: 51943 diff changeset	180 if not v1_fallback:
bd7359c18d69 rev-branch-cache: fallback on "v1" data if no v2 is found Pierre-Yves David <pierre-yves.david@octobus.net> parents: 51943 diff changeset	181 with repo.cachevfs(_rbcrevs) as fp:
bd7359c18d69 rev-branch-cache: fallback on "v1" data if no v2 is found Pierre-Yves David <pierre-yves.david@octobus.net> parents: 51943 diff changeset	182 if usemmap and repo.cachevfs.is_mmap_safe(_rbcrevs):
bd7359c18d69 rev-branch-cache: fallback on "v1" data if no v2 is found Pierre-Yves David <pierre-yves.david@octobus.net> parents: 51943 diff changeset	183 data = util.buffer(util.mmapread(fp))
bd7359c18d69 rev-branch-cache: fallback on "v1" data if no v2 is found Pierre-Yves David <pierre-yves.david@octobus.net> parents: 51943 diff changeset	184 else:
bd7359c18d69 rev-branch-cache: fallback on "v1" data if no v2 is found Pierre-Yves David <pierre-yves.david@octobus.net> parents: 51943 diff changeset	185 data = fp.read()
bd7359c18d69 rev-branch-cache: fallback on "v1" data if no v2 is found Pierre-Yves David <pierre-yves.david@octobus.net> parents: 51943 diff changeset	186 else:
bd7359c18d69 rev-branch-cache: fallback on "v1" data if no v2 is found Pierre-Yves David <pierre-yves.david@octobus.net> parents: 51943 diff changeset	187 # If we don't have "v2" data, we might have "v1" data worth
bd7359c18d69 rev-branch-cache: fallback on "v1" data if no v2 is found Pierre-Yves David <pierre-yves.david@octobus.net> parents: 51943 diff changeset	188 # using.
bd7359c18d69 rev-branch-cache: fallback on "v1" data if no v2 is found Pierre-Yves David <pierre-yves.david@octobus.net> parents: 51943 diff changeset	189 #
bd7359c18d69 rev-branch-cache: fallback on "v1" data if no v2 is found Pierre-Yves David <pierre-yves.david@octobus.net> parents: 51943 diff changeset	190 # Consider stop doing this many version after hg-6.9
bd7359c18d69 rev-branch-cache: fallback on "v1" data if no v2 is found Pierre-Yves David <pierre-yves.david@octobus.net> parents: 51943 diff changeset	191 # release.
bd7359c18d69 rev-branch-cache: fallback on "v1" data if no v2 is found Pierre-Yves David <pierre-yves.david@octobus.net> parents: 51943 diff changeset	192 with repo.cachevfs(_rbc_legacy_revs) as fp:
51651 c0e30a019ce1 mmap: only use mmap to read rev-branch-cache data if it is safe Pierre-Yves David <pierre-yves.david@octobus.net> parents: 51644 diff changeset	193 data = fp.read()
51377 02e7d79edf62 branchmap: use mmap for faster revbranchcache loading Arseniy Alekseyev <aalekseyev@janestreet.com> parents: 51307 diff changeset	194 self._rbcrevs = rbcrevs(data)
25660 328739ea70c3 global: mass rewrite to use modern exception syntax Gregory Szorc <gregory.szorc@gmail.com> parents: 25266 diff changeset	195 except (IOError, OSError) as inst:
43076 2372284d9457 formatting: blacken the codebase Augie Fackler <augie@google.com> parents: 42603 diff changeset	196 repo.ui.debug(
43077 687b865b95ad formatting: byteify all mercurial/ and hgext/ string literals Augie Fackler <augie@google.com> parents: 43076 diff changeset	197 b"couldn't read revision branch cache: %s\n"
43076 2372284d9457 formatting: blacken the codebase Augie Fackler <augie@google.com> parents: 42603 diff changeset	198 % stringutil.forcebytestr(inst)
2372284d9457 formatting: blacken the codebase Augie Fackler <augie@google.com> parents: 42603 diff changeset	199 )
23785 cb99bacb9b4e branchcache: introduce revbranchcache for caching of revision branch names Mads Kiilerich <madski@unity3d.com> parents: 22357 diff changeset	200 # remember number of good records on disk
43076 2372284d9457 formatting: blacken the codebase Augie Fackler <augie@google.com> parents: 42603 diff changeset	201 self._rbcrevslen = min(
2372284d9457 formatting: blacken the codebase Augie Fackler <augie@google.com> parents: 42603 diff changeset	202 len(self._rbcrevs) // _rbcrecsize, len(repo.changelog)
2372284d9457 formatting: blacken the codebase Augie Fackler <augie@google.com> parents: 42603 diff changeset	203 )
23785 cb99bacb9b4e branchcache: introduce revbranchcache for caching of revision branch names Mads Kiilerich <madski@unity3d.com> parents: 22357 diff changeset	204 if self._rbcrevslen == 0:
cb99bacb9b4e branchcache: introduce revbranchcache for caching of revision branch names Mads Kiilerich <madski@unity3d.com> parents: 22357 diff changeset	205 self._names = []
43076 2372284d9457 formatting: blacken the codebase Augie Fackler <augie@google.com> parents: 42603 diff changeset	206 self._rbcnamescount = len(self._names) # number of names read at
2372284d9457 formatting: blacken the codebase Augie Fackler <augie@google.com> parents: 42603 diff changeset	207 # _rbcsnameslen
23785 cb99bacb9b4e branchcache: introduce revbranchcache for caching of revision branch names Mads Kiilerich <madski@unity3d.com> parents: 22357 diff changeset	208
28558 bcd106d456c4 cache: rebuild branch cache from scratch when inconsistencies are detected Mads Kiilerich <madski@unity3d.com> parents: 28557 diff changeset	209 def _clear(self):
bcd106d456c4 cache: rebuild branch cache from scratch when inconsistencies are detected Mads Kiilerich <madski@unity3d.com> parents: 28557 diff changeset	210 self._rbcsnameslen = 0
bcd106d456c4 cache: rebuild branch cache from scratch when inconsistencies are detected Mads Kiilerich <madski@unity3d.com> parents: 28557 diff changeset	211 del self._names[:]
bcd106d456c4 cache: rebuild branch cache from scratch when inconsistencies are detected Mads Kiilerich <madski@unity3d.com> parents: 28557 diff changeset	212 self._rbcnamescount = 0
bcd106d456c4 cache: rebuild branch cache from scratch when inconsistencies are detected Mads Kiilerich <madski@unity3d.com> parents: 28557 diff changeset	213 self._rbcrevslen = len(self._repo.changelog)
51377 02e7d79edf62 branchmap: use mmap for faster revbranchcache loading Arseniy Alekseyev <aalekseyev@janestreet.com> parents: 51307 diff changeset	214 self._rbcrevs = rbcrevs(bytearray(self._rbcrevslen * _rbcrecsize))
43077 687b865b95ad formatting: byteify all mercurial/ and hgext/ string literals Augie Fackler <augie@google.com> parents: 43076 diff changeset	215 util.clearcachedproperty(self, b'_namesreverse')
51940 9f7cf869e9f4 rev-branch-cache: add a way to force rewrite of the cache Pierre-Yves David <pierre-yves.david@octobus.net> parents: 51939 diff changeset	216 self._force_overwrite = True
9f7cf869e9f4 rev-branch-cache: add a way to force rewrite of the cache Pierre-Yves David <pierre-yves.david@octobus.net> parents: 51939 diff changeset	217
9f7cf869e9f4 rev-branch-cache: add a way to force rewrite of the cache Pierre-Yves David <pierre-yves.david@octobus.net> parents: 51939 diff changeset	218 def invalidate(self, rev=0):
9f7cf869e9f4 rev-branch-cache: add a way to force rewrite of the cache Pierre-Yves David <pierre-yves.david@octobus.net> parents: 51939 diff changeset	219 self._rbcrevslen = rev
9f7cf869e9f4 rev-branch-cache: add a way to force rewrite of the cache Pierre-Yves David <pierre-yves.david@octobus.net> parents: 51939 diff changeset	220 self._rbcrevs.truncate(rev)
9f7cf869e9f4 rev-branch-cache: add a way to force rewrite of the cache Pierre-Yves David <pierre-yves.david@octobus.net> parents: 51939 diff changeset	221 self._force_overwrite = True
40749 50a64c321c1e branchmap: build the revbranchcache._namesreverse() only when required Pulkit Goyal <pulkit@yandex-team.ru> parents: 40494 diff changeset	222
50a64c321c1e branchmap: build the revbranchcache._namesreverse() only when required Pulkit Goyal <pulkit@yandex-team.ru> parents: 40494 diff changeset	223 @util.propertycache
50a64c321c1e branchmap: build the revbranchcache._namesreverse() only when required Pulkit Goyal <pulkit@yandex-team.ru> parents: 40494 diff changeset	224 def _namesreverse(self):
44470 9d2b2df2c2ba cleanup: run pyupgrade on our source tree to clean up varying things Augie Fackler <augie@google.com> parents: 44349 diff changeset	225 return {b: r for r, b in enumerate(self._names)}
28558 bcd106d456c4 cache: rebuild branch cache from scratch when inconsistencies are detected Mads Kiilerich <madski@unity3d.com> parents: 28557 diff changeset	226
40494 5e5c8f2a1eb5 branchmap: do not specify changelog as an argument Yuya Nishihara <yuya@tcha.org> parents: 40375 diff changeset	227 def branchinfo(self, rev):
23785 cb99bacb9b4e branchcache: introduce revbranchcache for caching of revision branch names Mads Kiilerich <madski@unity3d.com> parents: 22357 diff changeset	228 """Return branch name and close flag for rev, using and updating
cb99bacb9b4e branchcache: introduce revbranchcache for caching of revision branch names Mads Kiilerich <madski@unity3d.com> parents: 22357 diff changeset	229 persistent cache."""
40494 5e5c8f2a1eb5 branchmap: do not specify changelog as an argument Yuya Nishihara <yuya@tcha.org> parents: 40375 diff changeset	230 changelog = self._repo.changelog
23785 cb99bacb9b4e branchcache: introduce revbranchcache for caching of revision branch names Mads Kiilerich <madski@unity3d.com> parents: 22357 diff changeset	231 rbcrevidx = rev * _rbcrecsize
cb99bacb9b4e branchcache: introduce revbranchcache for caching of revision branch names Mads Kiilerich <madski@unity3d.com> parents: 22357 diff changeset	232
25266 38117278f295 revbranchcache: return uncached branchinfo for nullrev (issue4683) Yuya Nishihara <yuya@tcha.org> parents: 24728 diff changeset	233 # avoid negative index, changelog.read(nullrev) is fast without cache
38117278f295 revbranchcache: return uncached branchinfo for nullrev (issue4683) Yuya Nishihara <yuya@tcha.org> parents: 24728 diff changeset	234 if rev == nullrev:
38117278f295 revbranchcache: return uncached branchinfo for nullrev (issue4683) Yuya Nishihara <yuya@tcha.org> parents: 24728 diff changeset	235 return changelog.branchinfo(rev)
38117278f295 revbranchcache: return uncached branchinfo for nullrev (issue4683) Yuya Nishihara <yuya@tcha.org> parents: 24728 diff changeset	236
29604 db0095c83344 rbc: fix invalid rbc-revs entries caused by missing cache growth Mads Kiilerich <madski@unity3d.com> parents: 29423 diff changeset	237 # if requested rev isn't allocated, grow and cache the rev info
23785 cb99bacb9b4e branchcache: introduce revbranchcache for caching of revision branch names Mads Kiilerich <madski@unity3d.com> parents: 22357 diff changeset	238 if len(self._rbcrevs) < rbcrevidx + _rbcrecsize:
40494 5e5c8f2a1eb5 branchmap: do not specify changelog as an argument Yuya Nishihara <yuya@tcha.org> parents: 40375 diff changeset	239 return self._branchinfo(rev)
23785 cb99bacb9b4e branchcache: introduce revbranchcache for caching of revision branch names Mads Kiilerich <madski@unity3d.com> parents: 22357 diff changeset	240
cb99bacb9b4e branchcache: introduce revbranchcache for caching of revision branch names Mads Kiilerich <madski@unity3d.com> parents: 22357 diff changeset	241 # fast path: extract data from cache, use it if node is matching
cb99bacb9b4e branchcache: introduce revbranchcache for caching of revision branch names Mads Kiilerich <madski@unity3d.com> parents: 22357 diff changeset	242 reponode = changelog.node(rev)[:_rbcnodelen]
51377 02e7d79edf62 branchmap: use mmap for faster revbranchcache loading Arseniy Alekseyev <aalekseyev@janestreet.com> parents: 51307 diff changeset	243 cachenode, branchidx = self._rbcrevs.unpack_record(rbcrevidx)
23785 cb99bacb9b4e branchcache: introduce revbranchcache for caching of revision branch names Mads Kiilerich <madski@unity3d.com> parents: 22357 diff changeset	244 close = bool(branchidx & _rbccloseflag)
cb99bacb9b4e branchcache: introduce revbranchcache for caching of revision branch names Mads Kiilerich <madski@unity3d.com> parents: 22357 diff changeset	245 if close:
cb99bacb9b4e branchcache: introduce revbranchcache for caching of revision branch names Mads Kiilerich <madski@unity3d.com> parents: 22357 diff changeset	246 branchidx &= _rbcbranchidxmask
43077 687b865b95ad formatting: byteify all mercurial/ and hgext/ string literals Augie Fackler <augie@google.com> parents: 43076 diff changeset	247 if cachenode == b'\0\0\0\0':
24376 203a078da052 revbranchcache: populate cache incrementally Durham Goode <durham@fb.com> parents: 24375 diff changeset	248 pass
203a078da052 revbranchcache: populate cache incrementally Durham Goode <durham@fb.com> parents: 24375 diff changeset	249 elif cachenode == reponode:
29615 a2a380e2750f rbc: fix superfluous rebuilding from scratch - don't abuse self._rbcnamescount Mads Kiilerich <madski@unity3d.com> parents: 29604 diff changeset	250 try:
28558 bcd106d456c4 cache: rebuild branch cache from scratch when inconsistencies are detected Mads Kiilerich <madski@unity3d.com> parents: 28557 diff changeset	251 return self._names[branchidx], close
29615 a2a380e2750f rbc: fix superfluous rebuilding from scratch - don't abuse self._rbcnamescount Mads Kiilerich <madski@unity3d.com> parents: 29604 diff changeset	252 except IndexError:
a2a380e2750f rbc: fix superfluous rebuilding from scratch - don't abuse self._rbcnamescount Mads Kiilerich <madski@unity3d.com> parents: 29604 diff changeset	253 # recover from invalid reference to unknown branch
43076 2372284d9457 formatting: blacken the codebase Augie Fackler <augie@google.com> parents: 42603 diff changeset	254 self._repo.ui.debug(
43077 687b865b95ad formatting: byteify all mercurial/ and hgext/ string literals Augie Fackler <augie@google.com> parents: 43076 diff changeset	255 b"referenced branch names not found"
687b865b95ad formatting: byteify all mercurial/ and hgext/ string literals Augie Fackler <augie@google.com> parents: 43076 diff changeset	256 b" - rebuilding revision branch cache from scratch\n"
43076 2372284d9457 formatting: blacken the codebase Augie Fackler <augie@google.com> parents: 42603 diff changeset	257 )
29615 a2a380e2750f rbc: fix superfluous rebuilding from scratch - don't abuse self._rbcnamescount Mads Kiilerich <madski@unity3d.com> parents: 29604 diff changeset	258 self._clear()
24376 203a078da052 revbranchcache: populate cache incrementally Durham Goode <durham@fb.com> parents: 24375 diff changeset	259 else:
203a078da052 revbranchcache: populate cache incrementally Durham Goode <durham@fb.com> parents: 24375 diff changeset	260 # rev/node map has changed, invalidate the cache from here up
43076 2372284d9457 formatting: blacken the codebase Augie Fackler <augie@google.com> parents: 42603 diff changeset	261 self._repo.ui.debug(
43077 687b865b95ad formatting: byteify all mercurial/ and hgext/ string literals Augie Fackler <augie@google.com> parents: 43076 diff changeset	262 b"history modification detected - truncating "
687b865b95ad formatting: byteify all mercurial/ and hgext/ string literals Augie Fackler <augie@google.com> parents: 43076 diff changeset	263 b"revision branch cache to revision %d\n" % rev
43076 2372284d9457 formatting: blacken the codebase Augie Fackler <augie@google.com> parents: 42603 diff changeset	264 )
24376 203a078da052 revbranchcache: populate cache incrementally Durham Goode <durham@fb.com> parents: 24375 diff changeset	265 truncate = rbcrevidx + _rbcrecsize
51377 02e7d79edf62 branchmap: use mmap for faster revbranchcache loading Arseniy Alekseyev <aalekseyev@janestreet.com> parents: 51307 diff changeset	266 self._rbcrevs.truncate(truncate)
24376 203a078da052 revbranchcache: populate cache incrementally Durham Goode <durham@fb.com> parents: 24375 diff changeset	267 self._rbcrevslen = min(self._rbcrevslen, truncate)
203a078da052 revbranchcache: populate cache incrementally Durham Goode <durham@fb.com> parents: 24375 diff changeset	268
23785 cb99bacb9b4e branchcache: introduce revbranchcache for caching of revision branch names Mads Kiilerich <madski@unity3d.com> parents: 22357 diff changeset	269 # fall back to slow path and make sure it will be written to disk
40494 5e5c8f2a1eb5 branchmap: do not specify changelog as an argument Yuya Nishihara <yuya@tcha.org> parents: 40375 diff changeset	270 return self._branchinfo(rev)
23785 cb99bacb9b4e branchcache: introduce revbranchcache for caching of revision branch names Mads Kiilerich <madski@unity3d.com> parents: 22357 diff changeset	271
40494 5e5c8f2a1eb5 branchmap: do not specify changelog as an argument Yuya Nishihara <yuya@tcha.org> parents: 40375 diff changeset	272 def _branchinfo(self, rev):
23785 cb99bacb9b4e branchcache: introduce revbranchcache for caching of revision branch names Mads Kiilerich <madski@unity3d.com> parents: 22357 diff changeset	273 """Retrieve branch info from changelog and update _rbcrevs"""
40494 5e5c8f2a1eb5 branchmap: do not specify changelog as an argument Yuya Nishihara <yuya@tcha.org> parents: 40375 diff changeset	274 changelog = self._repo.changelog
23785 cb99bacb9b4e branchcache: introduce revbranchcache for caching of revision branch names Mads Kiilerich <madski@unity3d.com> parents: 22357 diff changeset	275 b, close = changelog.branchinfo(rev)
cb99bacb9b4e branchcache: introduce revbranchcache for caching of revision branch names Mads Kiilerich <madski@unity3d.com> parents: 22357 diff changeset	276 if b in self._namesreverse:
cb99bacb9b4e branchcache: introduce revbranchcache for caching of revision branch names Mads Kiilerich <madski@unity3d.com> parents: 22357 diff changeset	277 branchidx = self._namesreverse[b]
cb99bacb9b4e branchcache: introduce revbranchcache for caching of revision branch names Mads Kiilerich <madski@unity3d.com> parents: 22357 diff changeset	278 else:
cb99bacb9b4e branchcache: introduce revbranchcache for caching of revision branch names Mads Kiilerich <madski@unity3d.com> parents: 22357 diff changeset	279 branchidx = len(self._names)
cb99bacb9b4e branchcache: introduce revbranchcache for caching of revision branch names Mads Kiilerich <madski@unity3d.com> parents: 22357 diff changeset	280 self._names.append(b)
cb99bacb9b4e branchcache: introduce revbranchcache for caching of revision branch names Mads Kiilerich <madski@unity3d.com> parents: 22357 diff changeset	281 self._namesreverse[b] = branchidx
cb99bacb9b4e branchcache: introduce revbranchcache for caching of revision branch names Mads Kiilerich <madski@unity3d.com> parents: 22357 diff changeset	282 reponode = changelog.node(rev)
cb99bacb9b4e branchcache: introduce revbranchcache for caching of revision branch names Mads Kiilerich <madski@unity3d.com> parents: 22357 diff changeset	283 if close:
cb99bacb9b4e branchcache: introduce revbranchcache for caching of revision branch names Mads Kiilerich <madski@unity3d.com> parents: 22357 diff changeset	284 branchidx \|= _rbccloseflag
40494 5e5c8f2a1eb5 branchmap: do not specify changelog as an argument Yuya Nishihara <yuya@tcha.org> parents: 40375 diff changeset	285 self._setcachedata(rev, reponode, branchidx)
24375 fe255b2525d5 revbranchcache: move entry writing to a separate function Durham Goode <durham@fb.com> parents: 24374 diff changeset	286 return b, close
fe255b2525d5 revbranchcache: move entry writing to a separate function Durham Goode <durham@fb.com> parents: 24374 diff changeset	287
46444 3e91d9978bec branchmap: update rev-branch-cache incrementally Joerg Sonnenberger <joerg@bec.de> parents: 46434 diff changeset	288 def setdata(self, rev, changelogrevision):
36968 95f4f1bfb650 revbranchcache: add a public function to update the data Boris Feld <boris.feld@octobus.net> parents: 36504 diff changeset	289 """add new data information to the cache"""
46444 3e91d9978bec branchmap: update rev-branch-cache incrementally Joerg Sonnenberger <joerg@bec.de> parents: 46434 diff changeset	290 branch, close = changelogrevision.branchinfo
3e91d9978bec branchmap: update rev-branch-cache incrementally Joerg Sonnenberger <joerg@bec.de> parents: 46434 diff changeset	291
36968 95f4f1bfb650 revbranchcache: add a public function to update the data Boris Feld <boris.feld@octobus.net> parents: 36504 diff changeset	292 if branch in self._namesreverse:
95f4f1bfb650 revbranchcache: add a public function to update the data Boris Feld <boris.feld@octobus.net> parents: 36504 diff changeset	293 branchidx = self._namesreverse[branch]
95f4f1bfb650 revbranchcache: add a public function to update the data Boris Feld <boris.feld@octobus.net> parents: 36504 diff changeset	294 else:
95f4f1bfb650 revbranchcache: add a public function to update the data Boris Feld <boris.feld@octobus.net> parents: 36504 diff changeset	295 branchidx = len(self._names)
95f4f1bfb650 revbranchcache: add a public function to update the data Boris Feld <boris.feld@octobus.net> parents: 36504 diff changeset	296 self._names.append(branch)
95f4f1bfb650 revbranchcache: add a public function to update the data Boris Feld <boris.feld@octobus.net> parents: 36504 diff changeset	297 self._namesreverse[branch] = branchidx
95f4f1bfb650 revbranchcache: add a public function to update the data Boris Feld <boris.feld@octobus.net> parents: 36504 diff changeset	298 if close:
95f4f1bfb650 revbranchcache: add a public function to update the data Boris Feld <boris.feld@octobus.net> parents: 36504 diff changeset	299 branchidx \|= _rbccloseflag
46444 3e91d9978bec branchmap: update rev-branch-cache incrementally Joerg Sonnenberger <joerg@bec.de> parents: 46434 diff changeset	300 self._setcachedata(rev, self._repo.changelog.node(rev), branchidx)
36968 95f4f1bfb650 revbranchcache: add a public function to update the data Boris Feld <boris.feld@octobus.net> parents: 36504 diff changeset	301 # If no cache data were readable (non exists, bad permission, etc)
95f4f1bfb650 revbranchcache: add a public function to update the data Boris Feld <boris.feld@octobus.net> parents: 36504 diff changeset	302 # the cache was bypassing itself by setting:
95f4f1bfb650 revbranchcache: add a public function to update the data Boris Feld <boris.feld@octobus.net> parents: 36504 diff changeset	303 #
95f4f1bfb650 revbranchcache: add a public function to update the data Boris Feld <boris.feld@octobus.net> parents: 36504 diff changeset	304 # self.branchinfo = self._branchinfo
95f4f1bfb650 revbranchcache: add a public function to update the data Boris Feld <boris.feld@octobus.net> parents: 36504 diff changeset	305 #
95f4f1bfb650 revbranchcache: add a public function to update the data Boris Feld <boris.feld@octobus.net> parents: 36504 diff changeset	306 # Since we now have data in the cache, we need to drop this bypassing.
43554 9f70512ae2cf cleanup: remove pointless r-prefixes on single-quoted strings Augie Fackler <augie@google.com> parents: 43547 diff changeset	307 if 'branchinfo' in vars(self):
36968 95f4f1bfb650 revbranchcache: add a public function to update the data Boris Feld <boris.feld@octobus.net> parents: 36504 diff changeset	308 del self.branchinfo
95f4f1bfb650 revbranchcache: add a public function to update the data Boris Feld <boris.feld@octobus.net> parents: 36504 diff changeset	309
40494 5e5c8f2a1eb5 branchmap: do not specify changelog as an argument Yuya Nishihara <yuya@tcha.org> parents: 40375 diff changeset	310 def _setcachedata(self, rev, node, branchidx):
24375 fe255b2525d5 revbranchcache: move entry writing to a separate function Durham Goode <durham@fb.com> parents: 24374 diff changeset	311 """Writes the node's branch data to the in-memory cache data."""
31463 a5bad127128d branchmap: handle nullrev in setcachedata Durham Goode <durham@fb.com> parents: 31390 diff changeset	312 if rev == nullrev:
a5bad127128d branchmap: handle nullrev in setcachedata Durham Goode <durham@fb.com> parents: 31390 diff changeset	313 return
23785 cb99bacb9b4e branchcache: introduce revbranchcache for caching of revision branch names Mads Kiilerich <madski@unity3d.com> parents: 22357 diff changeset	314 rbcrevidx = rev * _rbcrecsize
51377 02e7d79edf62 branchmap: use mmap for faster revbranchcache loading Arseniy Alekseyev <aalekseyev@janestreet.com> parents: 51307 diff changeset	315 self._rbcrevs.pack_into(rbcrevidx, node, branchidx)
24376 203a078da052 revbranchcache: populate cache incrementally Durham Goode <durham@fb.com> parents: 24375 diff changeset	316 self._rbcrevslen = min(self._rbcrevslen, rev)
23785 cb99bacb9b4e branchcache: introduce revbranchcache for caching of revision branch names Mads Kiilerich <madski@unity3d.com> parents: 22357 diff changeset	317
24377 656f93ce66d5 revbranchcache: move cache writing to the transaction finalizer Durham Goode <durham@fb.com> parents: 24376 diff changeset	318 tr = self._repo.currenttransaction()
656f93ce66d5 revbranchcache: move cache writing to the transaction finalizer Durham Goode <durham@fb.com> parents: 24376 diff changeset	319 if tr:
43077 687b865b95ad formatting: byteify all mercurial/ and hgext/ string literals Augie Fackler <augie@google.com> parents: 43076 diff changeset	320 tr.addfinalize(b'write-revbranchcache', self.write)
24377 656f93ce66d5 revbranchcache: move cache writing to the transaction finalizer Durham Goode <durham@fb.com> parents: 24376 diff changeset	321
656f93ce66d5 revbranchcache: move cache writing to the transaction finalizer Durham Goode <durham@fb.com> parents: 24376 diff changeset	322 def write(self, tr=None):
23785 cb99bacb9b4e branchcache: introduce revbranchcache for caching of revision branch names Mads Kiilerich <madski@unity3d.com> parents: 22357 diff changeset	323 """Save branch cache if it is dirty."""
24374 77fd1fb538cd revbranchcache: store repo on the object Durham Goode <durham@fb.com> parents: 24373 diff changeset	324 repo = self._repo
29756 0d588332ad2c branchmap: acquires lock before writting the rev branch cache Pierre-Yves David <pierre-yves.david@ens-lyon.org> parents: 29755 diff changeset	325 wlock = None
43077 687b865b95ad formatting: byteify all mercurial/ and hgext/ string literals Augie Fackler <augie@google.com> parents: 43076 diff changeset	326 step = b''
29756 0d588332ad2c branchmap: acquires lock before writting the rev branch cache Pierre-Yves David <pierre-yves.david@ens-lyon.org> parents: 29755 diff changeset	327 try:
42222 09fd338522fa revbranchcache: factor logic to write names and revs in separate functions Pulkit Goyal <pulkit@yandex-team.ru> parents: 42138 diff changeset	328 # write the new names
51945 16efed18ae4e rev-branch-cache: schedule a write of the "v2" format if we read from "v1" Pierre-Yves David <pierre-yves.david@octobus.net> parents: 51944 diff changeset	329 if self._force_overwrite or self._rbcnamescount < len(self._names):
29756 0d588332ad2c branchmap: acquires lock before writting the rev branch cache Pierre-Yves David <pierre-yves.david@ens-lyon.org> parents: 29755 diff changeset	330 wlock = repo.wlock(wait=False)
43077 687b865b95ad formatting: byteify all mercurial/ and hgext/ string literals Augie Fackler <augie@google.com> parents: 43076 diff changeset	331 step = b' names'
42222 09fd338522fa revbranchcache: factor logic to write names and revs in separate functions Pulkit Goyal <pulkit@yandex-team.ru> parents: 42138 diff changeset	332 self._writenames(repo)
23785 cb99bacb9b4e branchcache: introduce revbranchcache for caching of revision branch names Mads Kiilerich <madski@unity3d.com> parents: 22357 diff changeset	333
42222 09fd338522fa revbranchcache: factor logic to write names and revs in separate functions Pulkit Goyal <pulkit@yandex-team.ru> parents: 42138 diff changeset	334 # write the new revs
29755 9f3c49ee4486 branchmap: preparatory indent of indent the branch rev writing code Pierre-Yves David <pierre-yves.david@ens-lyon.org> parents: 29615 diff changeset	335 start = self._rbcrevslen * _rbcrecsize
51940 9f7cf869e9f4 rev-branch-cache: add a way to force rewrite of the cache Pierre-Yves David <pierre-yves.david@octobus.net> parents: 51939 diff changeset	336 if self._force_overwrite or start != len(self._rbcrevs):
43077 687b865b95ad formatting: byteify all mercurial/ and hgext/ string literals Augie Fackler <augie@google.com> parents: 43076 diff changeset	337 step = b''
29756 0d588332ad2c branchmap: acquires lock before writting the rev branch cache Pierre-Yves David <pierre-yves.david@ens-lyon.org> parents: 29755 diff changeset	338 if wlock is None:
0d588332ad2c branchmap: acquires lock before writting the rev branch cache Pierre-Yves David <pierre-yves.david@ens-lyon.org> parents: 29755 diff changeset	339 wlock = repo.wlock(wait=False)
42222 09fd338522fa revbranchcache: factor logic to write names and revs in separate functions Pulkit Goyal <pulkit@yandex-team.ru> parents: 42138 diff changeset	340 self._writerevs(repo, start)
09fd338522fa revbranchcache: factor logic to write names and revs in separate functions Pulkit Goyal <pulkit@yandex-team.ru> parents: 42138 diff changeset	341
29757 3b184adfb5be branchmap: simplify error handlind when writing rev branch cache Pierre-Yves David <pierre-yves.david@ens-lyon.org> parents: 29756 diff changeset	342 except (IOError, OSError, error.Abort, error.LockError) as inst:
43076 2372284d9457 formatting: blacken the codebase Augie Fackler <augie@google.com> parents: 42603 diff changeset	343 repo.ui.debug(
43077 687b865b95ad formatting: byteify all mercurial/ and hgext/ string literals Augie Fackler <augie@google.com> parents: 43076 diff changeset	344 b"couldn't write revision branch cache%s: %s\n"
43076 2372284d9457 formatting: blacken the codebase Augie Fackler <augie@google.com> parents: 42603 diff changeset	345 % (step, stringutil.forcebytestr(inst))
2372284d9457 formatting: blacken the codebase Augie Fackler <augie@google.com> parents: 42603 diff changeset	346 )
29756 0d588332ad2c branchmap: acquires lock before writting the rev branch cache Pierre-Yves David <pierre-yves.david@ens-lyon.org> parents: 29755 diff changeset	347 finally:
0d588332ad2c branchmap: acquires lock before writting the rev branch cache Pierre-Yves David <pierre-yves.david@ens-lyon.org> parents: 29755 diff changeset	348 if wlock is not None:
0d588332ad2c branchmap: acquires lock before writting the rev branch cache Pierre-Yves David <pierre-yves.david@ens-lyon.org> parents: 29755 diff changeset	349 wlock.release()
42222 09fd338522fa revbranchcache: factor logic to write names and revs in separate functions Pulkit Goyal <pulkit@yandex-team.ru> parents: 42138 diff changeset	350
09fd338522fa revbranchcache: factor logic to write names and revs in separate functions Pulkit Goyal <pulkit@yandex-team.ru> parents: 42138 diff changeset	351 def _writenames(self, repo):
47031 f38bf44e077f black: make codebase compatible with black v21.4b2 and v20.8b1 Kyle Lippincott <spectral@google.com> parents: 46819 diff changeset	352 """write the new branch names to revbranchcache"""
51941 7032da075572 rev-branch-cache: make sure we close the name file we open Pierre-Yves David <pierre-yves.david@octobus.net> parents: 51940 diff changeset	353 f = None
51945 16efed18ae4e rev-branch-cache: schedule a write of the "v2" format if we read from "v1" Pierre-Yves David <pierre-yves.david@octobus.net> parents: 51944 diff changeset	354 if self._force_overwrite:
16efed18ae4e rev-branch-cache: schedule a write of the "v2" format if we read from "v1" Pierre-Yves David <pierre-yves.david@octobus.net> parents: 51944 diff changeset	355 self._rbcsnameslen = 0
16efed18ae4e rev-branch-cache: schedule a write of the "v2" format if we read from "v1" Pierre-Yves David <pierre-yves.david@octobus.net> parents: 51944 diff changeset	356 self._rbcnamescount = 0
51941 7032da075572 rev-branch-cache: make sure we close the name file we open Pierre-Yves David <pierre-yves.david@octobus.net> parents: 51940 diff changeset	357 try:
51945 16efed18ae4e rev-branch-cache: schedule a write of the "v2" format if we read from "v1" Pierre-Yves David <pierre-yves.david@octobus.net> parents: 51944 diff changeset	358 if self._force_overwrite or self._rbcnamescount != 0:
51941 7032da075572 rev-branch-cache: make sure we close the name file we open Pierre-Yves David <pierre-yves.david@octobus.net> parents: 51940 diff changeset	359 f = repo.cachevfs.open(_rbcnames, b'ab')
51945 16efed18ae4e rev-branch-cache: schedule a write of the "v2" format if we read from "v1" Pierre-Yves David <pierre-yves.david@octobus.net> parents: 51944 diff changeset	360 current_size = f.tell()
16efed18ae4e rev-branch-cache: schedule a write of the "v2" format if we read from "v1" Pierre-Yves David <pierre-yves.david@octobus.net> parents: 51944 diff changeset	361 if current_size == self._rbcsnameslen:
51941 7032da075572 rev-branch-cache: make sure we close the name file we open Pierre-Yves David <pierre-yves.david@octobus.net> parents: 51940 diff changeset	362 f.write(b'\0')
7032da075572 rev-branch-cache: make sure we close the name file we open Pierre-Yves David <pierre-yves.david@octobus.net> parents: 51940 diff changeset	363 else:
7032da075572 rev-branch-cache: make sure we close the name file we open Pierre-Yves David <pierre-yves.david@octobus.net> parents: 51940 diff changeset	364 f.close()
51945 16efed18ae4e rev-branch-cache: schedule a write of the "v2" format if we read from "v1" Pierre-Yves David <pierre-yves.david@octobus.net> parents: 51944 diff changeset	365 if self._force_overwrite:
16efed18ae4e rev-branch-cache: schedule a write of the "v2" format if we read from "v1" Pierre-Yves David <pierre-yves.david@octobus.net> parents: 51944 diff changeset	366 dbg = b"resetting content of %s\n"
16efed18ae4e rev-branch-cache: schedule a write of the "v2" format if we read from "v1" Pierre-Yves David <pierre-yves.david@octobus.net> parents: 51944 diff changeset	367 elif current_size > 0:
16efed18ae4e rev-branch-cache: schedule a write of the "v2" format if we read from "v1" Pierre-Yves David <pierre-yves.david@octobus.net> parents: 51944 diff changeset	368 dbg = b"%s changed - rewriting it\n"
16efed18ae4e rev-branch-cache: schedule a write of the "v2" format if we read from "v1" Pierre-Yves David <pierre-yves.david@octobus.net> parents: 51944 diff changeset	369 else:
16efed18ae4e rev-branch-cache: schedule a write of the "v2" format if we read from "v1" Pierre-Yves David <pierre-yves.david@octobus.net> parents: 51944 diff changeset	370 dbg = b"%s is missing - rewriting it\n"
16efed18ae4e rev-branch-cache: schedule a write of the "v2" format if we read from "v1" Pierre-Yves David <pierre-yves.david@octobus.net> parents: 51944 diff changeset	371 repo.ui.debug(dbg % _rbcnames)
51941 7032da075572 rev-branch-cache: make sure we close the name file we open Pierre-Yves David <pierre-yves.david@octobus.net> parents: 51940 diff changeset	372 self._rbcnamescount = 0
7032da075572 rev-branch-cache: make sure we close the name file we open Pierre-Yves David <pierre-yves.david@octobus.net> parents: 51940 diff changeset	373 self._rbcrevslen = 0
7032da075572 rev-branch-cache: make sure we close the name file we open Pierre-Yves David <pierre-yves.david@octobus.net> parents: 51940 diff changeset	374 if self._rbcnamescount == 0:
7032da075572 rev-branch-cache: make sure we close the name file we open Pierre-Yves David <pierre-yves.david@octobus.net> parents: 51940 diff changeset	375 # before rewriting names, make sure references are removed
7032da075572 rev-branch-cache: make sure we close the name file we open Pierre-Yves David <pierre-yves.david@octobus.net> parents: 51940 diff changeset	376 repo.cachevfs.unlinkpath(_rbcrevs, ignoremissing=True)
7032da075572 rev-branch-cache: make sure we close the name file we open Pierre-Yves David <pierre-yves.david@octobus.net> parents: 51940 diff changeset	377 f = repo.cachevfs.open(_rbcnames, b'wb')
7032da075572 rev-branch-cache: make sure we close the name file we open Pierre-Yves David <pierre-yves.david@octobus.net> parents: 51940 diff changeset	378 names = self._names[self._rbcnamescount :]
7032da075572 rev-branch-cache: make sure we close the name file we open Pierre-Yves David <pierre-yves.david@octobus.net> parents: 51940 diff changeset	379 from_local = encoding.fromlocal
7032da075572 rev-branch-cache: make sure we close the name file we open Pierre-Yves David <pierre-yves.david@octobus.net> parents: 51940 diff changeset	380 data = b'\0'.join(from_local(b) for b in names)
7032da075572 rev-branch-cache: make sure we close the name file we open Pierre-Yves David <pierre-yves.david@octobus.net> parents: 51940 diff changeset	381 f.write(data)
7032da075572 rev-branch-cache: make sure we close the name file we open Pierre-Yves David <pierre-yves.david@octobus.net> parents: 51940 diff changeset	382 self._rbcsnameslen = f.tell()
7032da075572 rev-branch-cache: make sure we close the name file we open Pierre-Yves David <pierre-yves.david@octobus.net> parents: 51940 diff changeset	383 finally:
7032da075572 rev-branch-cache: make sure we close the name file we open Pierre-Yves David <pierre-yves.david@octobus.net> parents: 51940 diff changeset	384 if f is not None:
42222 09fd338522fa revbranchcache: factor logic to write names and revs in separate functions Pulkit Goyal <pulkit@yandex-team.ru> parents: 42138 diff changeset	385 f.close()
09fd338522fa revbranchcache: factor logic to write names and revs in separate functions Pulkit Goyal <pulkit@yandex-team.ru> parents: 42138 diff changeset	386 self._rbcnamescount = len(self._names)
09fd338522fa revbranchcache: factor logic to write names and revs in separate functions Pulkit Goyal <pulkit@yandex-team.ru> parents: 42138 diff changeset	387
09fd338522fa revbranchcache: factor logic to write names and revs in separate functions Pulkit Goyal <pulkit@yandex-team.ru> parents: 42138 diff changeset	388 def _writerevs(self, repo, start):
47031 f38bf44e077f black: make codebase compatible with black v21.4b2 and v20.8b1 Kyle Lippincott <spectral@google.com> parents: 46819 diff changeset	389 """write the new revs to revbranchcache"""
42223 ececa45c80d8 revbranchcache: use context manager in _writerevs() to write to file Pulkit Goyal <pulkit@yandex-team.ru> parents: 42222 diff changeset	390 revs = min(len(repo.changelog), len(self._rbcrevs) // _rbcrecsize)
51942 c564be351754 rev-branch-cache: stop truncating cache file Pierre-Yves David <pierre-yves.david@octobus.net> parents: 51941 diff changeset	391
c564be351754 rev-branch-cache: stop truncating cache file Pierre-Yves David <pierre-yves.david@octobus.net> parents: 51941 diff changeset	392 end = revs * _rbcrecsize
51940 9f7cf869e9f4 rev-branch-cache: add a way to force rewrite of the cache Pierre-Yves David <pierre-yves.david@octobus.net> parents: 51939 diff changeset	393 if self._force_overwrite:
9f7cf869e9f4 rev-branch-cache: add a way to force rewrite of the cache Pierre-Yves David <pierre-yves.david@octobus.net> parents: 51939 diff changeset	394 start = 0
51942 c564be351754 rev-branch-cache: stop truncating cache file Pierre-Yves David <pierre-yves.david@octobus.net> parents: 51941 diff changeset	395
52005 76416b6e9d9b rev-branch-cache: properly ignores unaligned trailing data Pierre-Yves David <pierre-yves.david@octobus.net> parents: 52004 diff changeset	396 # align start on entry boundary
76416b6e9d9b rev-branch-cache: properly ignores unaligned trailing data Pierre-Yves David <pierre-yves.david@octobus.net> parents: 52004 diff changeset	397 start = _rbcrecsize * (start // _rbcrecsize)
76416b6e9d9b rev-branch-cache: properly ignores unaligned trailing data Pierre-Yves David <pierre-yves.david@octobus.net> parents: 52004 diff changeset	398
51942 c564be351754 rev-branch-cache: stop truncating cache file Pierre-Yves David <pierre-yves.david@octobus.net> parents: 51941 diff changeset	399 with repo.cachevfs.open(_rbcrevs, b'a+b') as f:
c564be351754 rev-branch-cache: stop truncating cache file Pierre-Yves David <pierre-yves.david@octobus.net> parents: 51941 diff changeset	400 pass # this make sure the file exist…
c564be351754 rev-branch-cache: stop truncating cache file Pierre-Yves David <pierre-yves.david@octobus.net> parents: 51941 diff changeset	401 with repo.cachevfs.open(_rbcrevs, b'r+b') as f:
c564be351754 rev-branch-cache: stop truncating cache file Pierre-Yves David <pierre-yves.david@octobus.net> parents: 51941 diff changeset	402 f.seek(0, os.SEEK_END)
51939 1eb2317c1762 rev-branch-cache: issue more truthful "truncating" message Pierre-Yves David <pierre-yves.david@octobus.net> parents: 51938 diff changeset	403 current_size = f.tell()
1eb2317c1762 rev-branch-cache: issue more truthful "truncating" message Pierre-Yves David <pierre-yves.david@octobus.net> parents: 51938 diff changeset	404 if current_size < start:
1eb2317c1762 rev-branch-cache: issue more truthful "truncating" message Pierre-Yves David <pierre-yves.david@octobus.net> parents: 51938 diff changeset	405 start = 0
1eb2317c1762 rev-branch-cache: issue more truthful "truncating" message Pierre-Yves David <pierre-yves.david@octobus.net> parents: 51938 diff changeset	406 if current_size != start:
51942 c564be351754 rev-branch-cache: stop truncating cache file Pierre-Yves David <pierre-yves.david@octobus.net> parents: 51941 diff changeset	407 threshold = current_size * REWRITE_RATIO
52004 4c885d5ff132 rev-branch-cache: stop pretending we will overwrite data when we don't Pierre-Yves David <pierre-yves.david@octobus.net> parents: 52003 diff changeset	408 overwritten = min(end, current_size) - start
4c885d5ff132 rev-branch-cache: stop pretending we will overwrite data when we don't Pierre-Yves David <pierre-yves.david@octobus.net> parents: 52003 diff changeset	409 if (max(end, current_size) - start) >= threshold:
4c885d5ff132 rev-branch-cache: stop pretending we will overwrite data when we don't Pierre-Yves David <pierre-yves.david@octobus.net> parents: 52003 diff changeset	410 start = 0
4c885d5ff132 rev-branch-cache: stop pretending we will overwrite data when we don't Pierre-Yves David <pierre-yves.david@octobus.net> parents: 52003 diff changeset	411 dbg = b"resetting content of cache/%s\n" % _rbcrevs
4c885d5ff132 rev-branch-cache: stop pretending we will overwrite data when we don't Pierre-Yves David <pierre-yves.david@octobus.net> parents: 52003 diff changeset	412 repo.ui.debug(dbg)
4c885d5ff132 rev-branch-cache: stop pretending we will overwrite data when we don't Pierre-Yves David <pierre-yves.david@octobus.net> parents: 52003 diff changeset	413 elif overwritten > 0:
4c885d5ff132 rev-branch-cache: stop pretending we will overwrite data when we don't Pierre-Yves David <pierre-yves.david@octobus.net> parents: 52003 diff changeset	414 # end affected, let us overwrite the bad value
51942 c564be351754 rev-branch-cache: stop truncating cache file Pierre-Yves David <pierre-yves.david@octobus.net> parents: 51941 diff changeset	415 dbg = b"overwriting %d bytes from %d in cache/%s"
52004 4c885d5ff132 rev-branch-cache: stop pretending we will overwrite data when we don't Pierre-Yves David <pierre-yves.david@octobus.net> parents: 52003 diff changeset	416 dbg %= (current_size - start, start, _rbcrevs)
51942 c564be351754 rev-branch-cache: stop truncating cache file Pierre-Yves David <pierre-yves.david@octobus.net> parents: 51941 diff changeset	417 if end < current_size:
c564be351754 rev-branch-cache: stop truncating cache file Pierre-Yves David <pierre-yves.david@octobus.net> parents: 51941 diff changeset	418 extra = b" leaving (%d trailing bytes)"
c564be351754 rev-branch-cache: stop truncating cache file Pierre-Yves David <pierre-yves.david@octobus.net> parents: 51941 diff changeset	419 extra %= current_size - end
c564be351754 rev-branch-cache: stop truncating cache file Pierre-Yves David <pierre-yves.david@octobus.net> parents: 51941 diff changeset	420 dbg += extra
c564be351754 rev-branch-cache: stop truncating cache file Pierre-Yves David <pierre-yves.david@octobus.net> parents: 51941 diff changeset	421 dbg += b'\n'
c564be351754 rev-branch-cache: stop truncating cache file Pierre-Yves David <pierre-yves.david@octobus.net> parents: 51941 diff changeset	422 repo.ui.debug(dbg)
c564be351754 rev-branch-cache: stop truncating cache file Pierre-Yves David <pierre-yves.david@octobus.net> parents: 51941 diff changeset	423 else:
52004 4c885d5ff132 rev-branch-cache: stop pretending we will overwrite data when we don't Pierre-Yves David <pierre-yves.david@octobus.net> parents: 52003 diff changeset	424 # extra untouched data at the end, lets warn about them
4c885d5ff132 rev-branch-cache: stop pretending we will overwrite data when we don't Pierre-Yves David <pierre-yves.david@octobus.net> parents: 52003 diff changeset	425 assert start == end # since don't write anything
4c885d5ff132 rev-branch-cache: stop pretending we will overwrite data when we don't Pierre-Yves David <pierre-yves.david@octobus.net> parents: 52003 diff changeset	426 dbg = b"cache/%s contains %d unknown trailing bytes\n"
4c885d5ff132 rev-branch-cache: stop pretending we will overwrite data when we don't Pierre-Yves David <pierre-yves.david@octobus.net> parents: 52003 diff changeset	427 dbg %= (_rbcrevs, current_size - start)
51942 c564be351754 rev-branch-cache: stop truncating cache file Pierre-Yves David <pierre-yves.david@octobus.net> parents: 51941 diff changeset	428 repo.ui.debug(dbg)
52004 4c885d5ff132 rev-branch-cache: stop pretending we will overwrite data when we don't Pierre-Yves David <pierre-yves.david@octobus.net> parents: 52003 diff changeset	429
51942 c564be351754 rev-branch-cache: stop truncating cache file Pierre-Yves David <pierre-yves.david@octobus.net> parents: 51941 diff changeset	430 if start > 0:
42222 09fd338522fa revbranchcache: factor logic to write names and revs in separate functions Pulkit Goyal <pulkit@yandex-team.ru> parents: 42138 diff changeset	431 f.seek(start)
51942 c564be351754 rev-branch-cache: stop truncating cache file Pierre-Yves David <pierre-yves.david@octobus.net> parents: 51941 diff changeset	432 f.write(self._rbcrevs.slice(start, end))
c564be351754 rev-branch-cache: stop truncating cache file Pierre-Yves David <pierre-yves.david@octobus.net> parents: 51941 diff changeset	433 else:
c564be351754 rev-branch-cache: stop truncating cache file Pierre-Yves David <pierre-yves.david@octobus.net> parents: 51941 diff changeset	434 f.close()
c564be351754 rev-branch-cache: stop truncating cache file Pierre-Yves David <pierre-yves.david@octobus.net> parents: 51941 diff changeset	435 with repo.cachevfs.open(
c564be351754 rev-branch-cache: stop truncating cache file Pierre-Yves David <pierre-yves.david@octobus.net> parents: 51941 diff changeset	436 _rbcrevs,
c564be351754 rev-branch-cache: stop truncating cache file Pierre-Yves David <pierre-yves.david@octobus.net> parents: 51941 diff changeset	437 b'wb',
c564be351754 rev-branch-cache: stop truncating cache file Pierre-Yves David <pierre-yves.david@octobus.net> parents: 51941 diff changeset	438 atomictemp=True,
c564be351754 rev-branch-cache: stop truncating cache file Pierre-Yves David <pierre-yves.david@octobus.net> parents: 51941 diff changeset	439 ) as rev_file:
c564be351754 rev-branch-cache: stop truncating cache file Pierre-Yves David <pierre-yves.david@octobus.net> parents: 51941 diff changeset	440 rev_file.write(self._rbcrevs.slice(start, end))
42222 09fd338522fa revbranchcache: factor logic to write names and revs in separate functions Pulkit Goyal <pulkit@yandex-team.ru> parents: 42138 diff changeset	441 self._rbcrevslen = revs
51940 9f7cf869e9f4 rev-branch-cache: add a way to force rewrite of the cache Pierre-Yves David <pierre-yves.david@octobus.net> parents: 51939 diff changeset	442 self._force_overwrite = False

Mercurial > public > mercurial-scm > hg-stable

annotate mercurial/branching/rev_cache.py @ 52005:76416b6e9d9b