mercurial-scm/hg: mercurial/branching/rev

annotate mercurial/branching/rev_cache.py @ 52113:db1980a361cb

rev-branch-cache: disable mmapping by default on Windows See the inline comment for why. The commands work, other than leaving extra files laying around. Perhaps there's some way to get this to work like on posix with some `CreateFile` magic (though it already uses `FILE_SHARE_DELETE`, so I'm not sure offhand what else we can do). However big picture- it seems wrong that the old file is left mmapped, a new one moved into place, and the mapping left over the old file instead of retargeted to the new file. That's got to be a bug on posix too, in a long running process like chg, right? If the memory is read again for some reason, it will be stale data.

author	Matt Harbison <matt_harbison@yahoo.com>
date	Fri, 18 Oct 2024 13:21:23 -0400
parents	76416b6e9d9b
children	24ee91ba9aa8

rev	line source
51897 f0e07efc199f rev-branch-cache: move the code in a dedicated module Pierre-Yves David <pierre-yves.david@octobus.net> parents: 51859 diff changeset	1 # rev_cache.py - caching branch information per revision
18116 bcee63733aad branchmap: create a mercurial.branchmap module Pierre-Yves David <pierre-yves.david@ens-lyon.org> parents: diff changeset	2 #
bcee63733aad branchmap: create a mercurial.branchmap module Pierre-Yves David <pierre-yves.david@ens-lyon.org> parents: diff changeset	3 # This software may be used and distributed according to the terms of the
bcee63733aad branchmap: create a mercurial.branchmap module Pierre-Yves David <pierre-yves.david@ens-lyon.org> parents: diff changeset	4 # GNU General Public License version 2 or any later version.
51859 f4733654f144 typing: add `from __future__ import annotations` to most files Matt Harbison <matt_harbison@yahoo.com> parents: 51651 diff changeset	5 from __future__ import annotations
25918 47f36e050c2e branchmap: use absolute_import Gregory Szorc <gregory.szorc@gmail.com> parents: 25660 diff changeset	6
51901 c564be351754 rev-branch-cache: stop truncating cache file Pierre-Yves David <pierre-yves.david@octobus.net> parents: 51900 diff changeset	7 import os
25918 47f36e050c2e branchmap: use absolute_import Gregory Szorc <gregory.szorc@gmail.com> parents: 25660 diff changeset	8 import struct
47f36e050c2e branchmap: use absolute_import Gregory Szorc <gregory.szorc@gmail.com> parents: 25660 diff changeset	9
51897 f0e07efc199f rev-branch-cache: move the code in a dedicated module Pierre-Yves David <pierre-yves.david@octobus.net> parents: 51859 diff changeset	10 from ..node import (
25918 47f36e050c2e branchmap: use absolute_import Gregory Szorc <gregory.szorc@gmail.com> parents: 25660 diff changeset	11 nullrev,
47f36e050c2e branchmap: use absolute_import Gregory Szorc <gregory.szorc@gmail.com> parents: 25660 diff changeset	12 )
51282 9d3721552b6c pytype: import typing directly Pierre-Yves David <pierre-yves.david@octobus.net> parents: 49300 diff changeset	13
51897 f0e07efc199f rev-branch-cache: move the code in a dedicated module Pierre-Yves David <pierre-yves.david@octobus.net> parents: 51859 diff changeset	14 from .. import (
25918 47f36e050c2e branchmap: use absolute_import Gregory Szorc <gregory.szorc@gmail.com> parents: 25660 diff changeset	15 encoding,
26587 56b2bcea2529 error: get Abort from 'error' instead of 'util' Pierre-Yves David <pierre-yves.david@fb.com> parents: 26460 diff changeset	16 error,
52113 db1980a361cb rev-branch-cache: disable mmapping by default on Windows Matt Harbison <matt_harbison@yahoo.com> parents: 51971 diff changeset	17 pycompat,
30975 22fbca1d11ed mercurial: switch to util.timer for all interval timings Simon Farnsworth <simonfar@fb.com> parents: 29746 diff changeset	18 util,
25918 47f36e050c2e branchmap: use absolute_import Gregory Szorc <gregory.szorc@gmail.com> parents: 25660 diff changeset	19 )
51282 9d3721552b6c pytype: import typing directly Pierre-Yves David <pierre-yves.david@octobus.net> parents: 49300 diff changeset	20
51897 f0e07efc199f rev-branch-cache: move the code in a dedicated module Pierre-Yves David <pierre-yves.david@octobus.net> parents: 51859 diff changeset	21 from ..utils import (
37084 f0b6fbea00cf stringutil: bulk-replace call sites to point to new module Yuya Nishihara <yuya@tcha.org> parents: 36962 diff changeset	22 stringutil,
f0b6fbea00cf stringutil: bulk-replace call sites to point to new module Yuya Nishihara <yuya@tcha.org> parents: 36962 diff changeset	23 )
25918 47f36e050c2e branchmap: use absolute_import Gregory Szorc <gregory.szorc@gmail.com> parents: 25660 diff changeset	24
47f36e050c2e branchmap: use absolute_import Gregory Szorc <gregory.szorc@gmail.com> parents: 25660 diff changeset	25 calcsize = struct.calcsize
31370 906be86990c4 rbc: use struct unpack_from and pack_into instead of unpack and pack Mads Kiilerich <madski@unity3d.com> parents: 31360 diff changeset	26 pack_into = struct.pack_into
906be86990c4 rbc: use struct unpack_from and pack_into instead of unpack and pack Mads Kiilerich <madski@unity3d.com> parents: 31360 diff changeset	27 unpack_from = struct.unpack_from
18117 526e7ec5c96e branchmap: extract write logic from localrepo Pierre-Yves David <pierre-yves.david@ens-lyon.org> parents: 18116 diff changeset	28
18118 e70ff1e599f4 branchmap: extract read logic from repo Pierre-Yves David <pierre-yves.david@logilab.fr> parents: 18117 diff changeset	29
23785 cb99bacb9b4e branchcache: introduce revbranchcache for caching of revision branch names Mads Kiilerich <madski@unity3d.com> parents: 22357 diff changeset	30 # Revision branch info cache
cb99bacb9b4e branchcache: introduce revbranchcache for caching of revision branch names Mads Kiilerich <madski@unity3d.com> parents: 22357 diff changeset	31
51902 0f26ee69cf36 rev-branch-cache: increment the version to "v2" Pierre-Yves David <pierre-yves.david@octobus.net> parents: 51901 diff changeset	32 # The "V2" version use the same format as the "V1" but garantee it won't be
0f26ee69cf36 rev-branch-cache: increment the version to "v2" Pierre-Yves David <pierre-yves.david@octobus.net> parents: 51901 diff changeset	33 # truncated, preventing SIGBUS when it is mmap-ed
0f26ee69cf36 rev-branch-cache: increment the version to "v2" Pierre-Yves David <pierre-yves.david@octobus.net> parents: 51901 diff changeset	34 _rbcversion = b'-v2'
43077 687b865b95ad formatting: byteify all mercurial/ and hgext/ string literals Augie Fackler <augie@google.com> parents: 43076 diff changeset	35 _rbcnames = b'rbc-names' + _rbcversion
687b865b95ad formatting: byteify all mercurial/ and hgext/ string literals Augie Fackler <augie@google.com> parents: 43076 diff changeset	36 _rbcrevs = b'rbc-revs' + _rbcversion
51903 bd7359c18d69 rev-branch-cache: fallback on "v1" data if no v2 is found Pierre-Yves David <pierre-yves.david@octobus.net> parents: 51902 diff changeset	37 _rbc_legacy_version = b'-v1'
bd7359c18d69 rev-branch-cache: fallback on "v1" data if no v2 is found Pierre-Yves David <pierre-yves.david@octobus.net> parents: 51902 diff changeset	38 _rbc_legacy_names = b'rbc-names' + _rbc_legacy_version
bd7359c18d69 rev-branch-cache: fallback on "v1" data if no v2 is found Pierre-Yves David <pierre-yves.david@octobus.net> parents: 51902 diff changeset	39 _rbc_legacy_revs = b'rbc-revs' + _rbc_legacy_version
23785 cb99bacb9b4e branchcache: introduce revbranchcache for caching of revision branch names Mads Kiilerich <madski@unity3d.com> parents: 22357 diff changeset	40 # [4 byte hash prefix][4 byte branch name number with sign bit indicating open]
43077 687b865b95ad formatting: byteify all mercurial/ and hgext/ string literals Augie Fackler <augie@google.com> parents: 43076 diff changeset	41 _rbcrecfmt = b'>4sI'
23785 cb99bacb9b4e branchcache: introduce revbranchcache for caching of revision branch names Mads Kiilerich <madski@unity3d.com> parents: 22357 diff changeset	42 _rbcrecsize = calcsize(_rbcrecfmt)
46360 1726a53a8494 reverse-branch-cache: switch to doubling allocating scheme Joerg Sonnenberger <joerg@bec.de> parents: 46254 diff changeset	43 _rbcmininc = 64 * _rbcrecsize
23785 cb99bacb9b4e branchcache: introduce revbranchcache for caching of revision branch names Mads Kiilerich <madski@unity3d.com> parents: 22357 diff changeset	44 _rbcnodelen = 4
43076 2372284d9457 formatting: blacken the codebase Augie Fackler <augie@google.com> parents: 42603 diff changeset	45 _rbcbranchidxmask = 0x7FFFFFFF
23785 cb99bacb9b4e branchcache: introduce revbranchcache for caching of revision branch names Mads Kiilerich <madski@unity3d.com> parents: 22357 diff changeset	46 _rbccloseflag = 0x80000000
cb99bacb9b4e branchcache: introduce revbranchcache for caching of revision branch names Mads Kiilerich <madski@unity3d.com> parents: 22357 diff changeset	47
43076 2372284d9457 formatting: blacken the codebase Augie Fackler <augie@google.com> parents: 42603 diff changeset	48
51901 c564be351754 rev-branch-cache: stop truncating cache file Pierre-Yves David <pierre-yves.david@octobus.net> parents: 51900 diff changeset	49 # with atomic replacement.
c564be351754 rev-branch-cache: stop truncating cache file Pierre-Yves David <pierre-yves.david@octobus.net> parents: 51900 diff changeset	50 REWRITE_RATIO = 0.2
c564be351754 rev-branch-cache: stop truncating cache file Pierre-Yves David <pierre-yves.david@octobus.net> parents: 51900 diff changeset	51
c564be351754 rev-branch-cache: stop truncating cache file Pierre-Yves David <pierre-yves.david@octobus.net> parents: 51900 diff changeset	52
51372 02e7d79edf62 branchmap: use mmap for faster revbranchcache loading Arseniy Alekseyev <aalekseyev@janestreet.com> parents: 51287 diff changeset	53 class rbcrevs:
02e7d79edf62 branchmap: use mmap for faster revbranchcache loading Arseniy Alekseyev <aalekseyev@janestreet.com> parents: 51287 diff changeset	54 """a byte string consisting of an immutable prefix followed by a mutable suffix"""
02e7d79edf62 branchmap: use mmap for faster revbranchcache loading Arseniy Alekseyev <aalekseyev@janestreet.com> parents: 51287 diff changeset	55
02e7d79edf62 branchmap: use mmap for faster revbranchcache loading Arseniy Alekseyev <aalekseyev@janestreet.com> parents: 51287 diff changeset	56 def __init__(self, revs):
02e7d79edf62 branchmap: use mmap for faster revbranchcache loading Arseniy Alekseyev <aalekseyev@janestreet.com> parents: 51287 diff changeset	57 self._prefix = revs
02e7d79edf62 branchmap: use mmap for faster revbranchcache loading Arseniy Alekseyev <aalekseyev@janestreet.com> parents: 51287 diff changeset	58 self._rest = bytearray()
02e7d79edf62 branchmap: use mmap for faster revbranchcache loading Arseniy Alekseyev <aalekseyev@janestreet.com> parents: 51287 diff changeset	59
51971 76416b6e9d9b rev-branch-cache: properly ignores unaligned trailing data Pierre-Yves David <pierre-yves.david@octobus.net> parents: 51970 diff changeset	60 @property
76416b6e9d9b rev-branch-cache: properly ignores unaligned trailing data Pierre-Yves David <pierre-yves.david@octobus.net> parents: 51970 diff changeset	61 def len_prefix(self):
76416b6e9d9b rev-branch-cache: properly ignores unaligned trailing data Pierre-Yves David <pierre-yves.david@octobus.net> parents: 51970 diff changeset	62 size = len(self._prefix)
76416b6e9d9b rev-branch-cache: properly ignores unaligned trailing data Pierre-Yves David <pierre-yves.david@octobus.net> parents: 51970 diff changeset	63 return size - (size % _rbcrecsize)
76416b6e9d9b rev-branch-cache: properly ignores unaligned trailing data Pierre-Yves David <pierre-yves.david@octobus.net> parents: 51970 diff changeset	64
51372 02e7d79edf62 branchmap: use mmap for faster revbranchcache loading Arseniy Alekseyev <aalekseyev@janestreet.com> parents: 51287 diff changeset	65 def __len__(self):
51971 76416b6e9d9b rev-branch-cache: properly ignores unaligned trailing data Pierre-Yves David <pierre-yves.david@octobus.net> parents: 51970 diff changeset	66 return self.len_prefix + len(self._rest)
51372 02e7d79edf62 branchmap: use mmap for faster revbranchcache loading Arseniy Alekseyev <aalekseyev@janestreet.com> parents: 51287 diff changeset	67
02e7d79edf62 branchmap: use mmap for faster revbranchcache loading Arseniy Alekseyev <aalekseyev@janestreet.com> parents: 51287 diff changeset	68 def unpack_record(self, rbcrevidx):
51971 76416b6e9d9b rev-branch-cache: properly ignores unaligned trailing data Pierre-Yves David <pierre-yves.david@octobus.net> parents: 51970 diff changeset	69 if rbcrevidx < self.len_prefix:
51372 02e7d79edf62 branchmap: use mmap for faster revbranchcache loading Arseniy Alekseyev <aalekseyev@janestreet.com> parents: 51287 diff changeset	70 return unpack_from(_rbcrecfmt, util.buffer(self._prefix), rbcrevidx)
02e7d79edf62 branchmap: use mmap for faster revbranchcache loading Arseniy Alekseyev <aalekseyev@janestreet.com> parents: 51287 diff changeset	71 else:
02e7d79edf62 branchmap: use mmap for faster revbranchcache loading Arseniy Alekseyev <aalekseyev@janestreet.com> parents: 51287 diff changeset	72 return unpack_from(
02e7d79edf62 branchmap: use mmap for faster revbranchcache loading Arseniy Alekseyev <aalekseyev@janestreet.com> parents: 51287 diff changeset	73 _rbcrecfmt,
02e7d79edf62 branchmap: use mmap for faster revbranchcache loading Arseniy Alekseyev <aalekseyev@janestreet.com> parents: 51287 diff changeset	74 util.buffer(self._rest),
51971 76416b6e9d9b rev-branch-cache: properly ignores unaligned trailing data Pierre-Yves David <pierre-yves.david@octobus.net> parents: 51970 diff changeset	75 rbcrevidx - self.len_prefix,
51372 02e7d79edf62 branchmap: use mmap for faster revbranchcache loading Arseniy Alekseyev <aalekseyev@janestreet.com> parents: 51287 diff changeset	76 )
02e7d79edf62 branchmap: use mmap for faster revbranchcache loading Arseniy Alekseyev <aalekseyev@janestreet.com> parents: 51287 diff changeset	77
02e7d79edf62 branchmap: use mmap for faster revbranchcache loading Arseniy Alekseyev <aalekseyev@janestreet.com> parents: 51287 diff changeset	78 def make_mutable(self):
51971 76416b6e9d9b rev-branch-cache: properly ignores unaligned trailing data Pierre-Yves David <pierre-yves.david@octobus.net> parents: 51970 diff changeset	79 if self.len_prefix > 0:
51372 02e7d79edf62 branchmap: use mmap for faster revbranchcache loading Arseniy Alekseyev <aalekseyev@janestreet.com> parents: 51287 diff changeset	80 entirety = bytearray()
51971 76416b6e9d9b rev-branch-cache: properly ignores unaligned trailing data Pierre-Yves David <pierre-yves.david@octobus.net> parents: 51970 diff changeset	81 entirety[:] = self._prefix[: self.len_prefix]
51372 02e7d79edf62 branchmap: use mmap for faster revbranchcache loading Arseniy Alekseyev <aalekseyev@janestreet.com> parents: 51287 diff changeset	82 entirety.extend(self._rest)
02e7d79edf62 branchmap: use mmap for faster revbranchcache loading Arseniy Alekseyev <aalekseyev@janestreet.com> parents: 51287 diff changeset	83 self._rest = entirety
02e7d79edf62 branchmap: use mmap for faster revbranchcache loading Arseniy Alekseyev <aalekseyev@janestreet.com> parents: 51287 diff changeset	84 self._prefix = bytearray()
02e7d79edf62 branchmap: use mmap for faster revbranchcache loading Arseniy Alekseyev <aalekseyev@janestreet.com> parents: 51287 diff changeset	85
02e7d79edf62 branchmap: use mmap for faster revbranchcache loading Arseniy Alekseyev <aalekseyev@janestreet.com> parents: 51287 diff changeset	86 def truncate(self, pos):
02e7d79edf62 branchmap: use mmap for faster revbranchcache loading Arseniy Alekseyev <aalekseyev@janestreet.com> parents: 51287 diff changeset	87 self.make_mutable()
02e7d79edf62 branchmap: use mmap for faster revbranchcache loading Arseniy Alekseyev <aalekseyev@janestreet.com> parents: 51287 diff changeset	88 del self._rest[pos:]
02e7d79edf62 branchmap: use mmap for faster revbranchcache loading Arseniy Alekseyev <aalekseyev@janestreet.com> parents: 51287 diff changeset	89
02e7d79edf62 branchmap: use mmap for faster revbranchcache loading Arseniy Alekseyev <aalekseyev@janestreet.com> parents: 51287 diff changeset	90 def pack_into(self, rbcrevidx, node, branchidx):
51971 76416b6e9d9b rev-branch-cache: properly ignores unaligned trailing data Pierre-Yves David <pierre-yves.david@octobus.net> parents: 51970 diff changeset	91 if rbcrevidx < self.len_prefix:
51372 02e7d79edf62 branchmap: use mmap for faster revbranchcache loading Arseniy Alekseyev <aalekseyev@janestreet.com> parents: 51287 diff changeset	92 self.make_mutable()
02e7d79edf62 branchmap: use mmap for faster revbranchcache loading Arseniy Alekseyev <aalekseyev@janestreet.com> parents: 51287 diff changeset	93 buf = self._rest
51971 76416b6e9d9b rev-branch-cache: properly ignores unaligned trailing data Pierre-Yves David <pierre-yves.david@octobus.net> parents: 51970 diff changeset	94 start_offset = rbcrevidx - self.len_prefix
51372 02e7d79edf62 branchmap: use mmap for faster revbranchcache loading Arseniy Alekseyev <aalekseyev@janestreet.com> parents: 51287 diff changeset	95 end_offset = start_offset + _rbcrecsize
02e7d79edf62 branchmap: use mmap for faster revbranchcache loading Arseniy Alekseyev <aalekseyev@janestreet.com> parents: 51287 diff changeset	96
02e7d79edf62 branchmap: use mmap for faster revbranchcache loading Arseniy Alekseyev <aalekseyev@janestreet.com> parents: 51287 diff changeset	97 if len(self._rest) < end_offset:
02e7d79edf62 branchmap: use mmap for faster revbranchcache loading Arseniy Alekseyev <aalekseyev@janestreet.com> parents: 51287 diff changeset	98 # bytearray doesn't allocate extra space at least in Python 3.7.
02e7d79edf62 branchmap: use mmap for faster revbranchcache loading Arseniy Alekseyev <aalekseyev@janestreet.com> parents: 51287 diff changeset	99 # When multiple changesets are added in a row, precise resize would
02e7d79edf62 branchmap: use mmap for faster revbranchcache loading Arseniy Alekseyev <aalekseyev@janestreet.com> parents: 51287 diff changeset	100 # result in quadratic complexity. Overallocate to compensate by
02e7d79edf62 branchmap: use mmap for faster revbranchcache loading Arseniy Alekseyev <aalekseyev@janestreet.com> parents: 51287 diff changeset	101 # using the classic doubling technique for dynamic arrays instead.
02e7d79edf62 branchmap: use mmap for faster revbranchcache loading Arseniy Alekseyev <aalekseyev@janestreet.com> parents: 51287 diff changeset	102 # If there was a gap in the map before, less space will be reserved.
02e7d79edf62 branchmap: use mmap for faster revbranchcache loading Arseniy Alekseyev <aalekseyev@janestreet.com> parents: 51287 diff changeset	103 self._rest.extend(b'\0' * end_offset)
02e7d79edf62 branchmap: use mmap for faster revbranchcache loading Arseniy Alekseyev <aalekseyev@janestreet.com> parents: 51287 diff changeset	104 return pack_into(
02e7d79edf62 branchmap: use mmap for faster revbranchcache loading Arseniy Alekseyev <aalekseyev@janestreet.com> parents: 51287 diff changeset	105 _rbcrecfmt,
02e7d79edf62 branchmap: use mmap for faster revbranchcache loading Arseniy Alekseyev <aalekseyev@janestreet.com> parents: 51287 diff changeset	106 buf,
02e7d79edf62 branchmap: use mmap for faster revbranchcache loading Arseniy Alekseyev <aalekseyev@janestreet.com> parents: 51287 diff changeset	107 start_offset,
02e7d79edf62 branchmap: use mmap for faster revbranchcache loading Arseniy Alekseyev <aalekseyev@janestreet.com> parents: 51287 diff changeset	108 node,
02e7d79edf62 branchmap: use mmap for faster revbranchcache loading Arseniy Alekseyev <aalekseyev@janestreet.com> parents: 51287 diff changeset	109 branchidx,
02e7d79edf62 branchmap: use mmap for faster revbranchcache loading Arseniy Alekseyev <aalekseyev@janestreet.com> parents: 51287 diff changeset	110 )
02e7d79edf62 branchmap: use mmap for faster revbranchcache loading Arseniy Alekseyev <aalekseyev@janestreet.com> parents: 51287 diff changeset	111
02e7d79edf62 branchmap: use mmap for faster revbranchcache loading Arseniy Alekseyev <aalekseyev@janestreet.com> parents: 51287 diff changeset	112 def extend(self, extension):
02e7d79edf62 branchmap: use mmap for faster revbranchcache loading Arseniy Alekseyev <aalekseyev@janestreet.com> parents: 51287 diff changeset	113 return self._rest.extend(extension)
02e7d79edf62 branchmap: use mmap for faster revbranchcache loading Arseniy Alekseyev <aalekseyev@janestreet.com> parents: 51287 diff changeset	114
02e7d79edf62 branchmap: use mmap for faster revbranchcache loading Arseniy Alekseyev <aalekseyev@janestreet.com> parents: 51287 diff changeset	115 def slice(self, begin, end):
51971 76416b6e9d9b rev-branch-cache: properly ignores unaligned trailing data Pierre-Yves David <pierre-yves.david@octobus.net> parents: 51970 diff changeset	116 if begin < self.len_prefix:
51372 02e7d79edf62 branchmap: use mmap for faster revbranchcache loading Arseniy Alekseyev <aalekseyev@janestreet.com> parents: 51287 diff changeset	117 acc = bytearray()
51971 76416b6e9d9b rev-branch-cache: properly ignores unaligned trailing data Pierre-Yves David <pierre-yves.david@octobus.net> parents: 51970 diff changeset	118 acc[:] = self._prefix[begin : min(end, self.len_prefix)]
51372 02e7d79edf62 branchmap: use mmap for faster revbranchcache loading Arseniy Alekseyev <aalekseyev@janestreet.com> parents: 51287 diff changeset	119 acc.extend(
51971 76416b6e9d9b rev-branch-cache: properly ignores unaligned trailing data Pierre-Yves David <pierre-yves.david@octobus.net> parents: 51970 diff changeset	120 self._rest[begin - self.len_prefix : end - self.len_prefix]
51372 02e7d79edf62 branchmap: use mmap for faster revbranchcache loading Arseniy Alekseyev <aalekseyev@janestreet.com> parents: 51287 diff changeset	121 )
02e7d79edf62 branchmap: use mmap for faster revbranchcache loading Arseniy Alekseyev <aalekseyev@janestreet.com> parents: 51287 diff changeset	122 return acc
51971 76416b6e9d9b rev-branch-cache: properly ignores unaligned trailing data Pierre-Yves David <pierre-yves.david@octobus.net> parents: 51970 diff changeset	123 return self._rest[begin - self.len_prefix : end - self.len_prefix]
51372 02e7d79edf62 branchmap: use mmap for faster revbranchcache loading Arseniy Alekseyev <aalekseyev@janestreet.com> parents: 51287 diff changeset	124
02e7d79edf62 branchmap: use mmap for faster revbranchcache loading Arseniy Alekseyev <aalekseyev@janestreet.com> parents: 51287 diff changeset	125
48946 642e31cb55f0 py3: use class X: instead of class X(object): Gregory Szorc <gregory.szorc@gmail.com> parents: 48935 diff changeset	126 class revbranchcache:
23785 cb99bacb9b4e branchcache: introduce revbranchcache for caching of revision branch names Mads Kiilerich <madski@unity3d.com> parents: 22357 diff changeset	127 """Persistent cache, mapping from revision number to branch name and close.
cb99bacb9b4e branchcache: introduce revbranchcache for caching of revision branch names Mads Kiilerich <madski@unity3d.com> parents: 22357 diff changeset	128 This is a low level cache, independent of filtering.
cb99bacb9b4e branchcache: introduce revbranchcache for caching of revision branch names Mads Kiilerich <madski@unity3d.com> parents: 22357 diff changeset	129
cb99bacb9b4e branchcache: introduce revbranchcache for caching of revision branch names Mads Kiilerich <madski@unity3d.com> parents: 22357 diff changeset	130 Branch names are stored in rbc-names in internal encoding separated by 0.
cb99bacb9b4e branchcache: introduce revbranchcache for caching of revision branch names Mads Kiilerich <madski@unity3d.com> parents: 22357 diff changeset	131 rbc-names is append-only, and each branch name is only stored once and will
cb99bacb9b4e branchcache: introduce revbranchcache for caching of revision branch names Mads Kiilerich <madski@unity3d.com> parents: 22357 diff changeset	132 thus have a unique index.
cb99bacb9b4e branchcache: introduce revbranchcache for caching of revision branch names Mads Kiilerich <madski@unity3d.com> parents: 22357 diff changeset	133
cb99bacb9b4e branchcache: introduce revbranchcache for caching of revision branch names Mads Kiilerich <madski@unity3d.com> parents: 22357 diff changeset	134 The branch info for each revision is stored in rbc-revs as constant size
cb99bacb9b4e branchcache: introduce revbranchcache for caching of revision branch names Mads Kiilerich <madski@unity3d.com> parents: 22357 diff changeset	135 records. The whole file is read into memory, but it is only 'parsed' on
cb99bacb9b4e branchcache: introduce revbranchcache for caching of revision branch names Mads Kiilerich <madski@unity3d.com> parents: 22357 diff changeset	136 demand. The file is usually append-only but will be truncated if repo
cb99bacb9b4e branchcache: introduce revbranchcache for caching of revision branch names Mads Kiilerich <madski@unity3d.com> parents: 22357 diff changeset	137 modification is detected.
cb99bacb9b4e branchcache: introduce revbranchcache for caching of revision branch names Mads Kiilerich <madski@unity3d.com> parents: 22357 diff changeset	138 The record for each revision contains the first 4 bytes of the
cb99bacb9b4e branchcache: introduce revbranchcache for caching of revision branch names Mads Kiilerich <madski@unity3d.com> parents: 22357 diff changeset	139 corresponding node hash, and the record is only used if it still matches.
cb99bacb9b4e branchcache: introduce revbranchcache for caching of revision branch names Mads Kiilerich <madski@unity3d.com> parents: 22357 diff changeset	140 Even a completely trashed rbc-revs fill thus still give the right result
cb99bacb9b4e branchcache: introduce revbranchcache for caching of revision branch names Mads Kiilerich <madski@unity3d.com> parents: 22357 diff changeset	141 while converging towards full recovery ... assuming no incorrectly matching
cb99bacb9b4e branchcache: introduce revbranchcache for caching of revision branch names Mads Kiilerich <madski@unity3d.com> parents: 22357 diff changeset	142 node hashes.
cb99bacb9b4e branchcache: introduce revbranchcache for caching of revision branch names Mads Kiilerich <madski@unity3d.com> parents: 22357 diff changeset	143 The record also contains 4 bytes where 31 bits contains the index of the
cb99bacb9b4e branchcache: introduce revbranchcache for caching of revision branch names Mads Kiilerich <madski@unity3d.com> parents: 22357 diff changeset	144 branch and the last bit indicate that it is a branch close commit.
cb99bacb9b4e branchcache: introduce revbranchcache for caching of revision branch names Mads Kiilerich <madski@unity3d.com> parents: 22357 diff changeset	145 The usage pattern for rbc-revs is thus somewhat similar to 00changelog.i
cb99bacb9b4e branchcache: introduce revbranchcache for caching of revision branch names Mads Kiilerich <madski@unity3d.com> parents: 22357 diff changeset	146 and will grow with it but be 1/8th of its size.
cb99bacb9b4e branchcache: introduce revbranchcache for caching of revision branch names Mads Kiilerich <madski@unity3d.com> parents: 22357 diff changeset	147 """
cb99bacb9b4e branchcache: introduce revbranchcache for caching of revision branch names Mads Kiilerich <madski@unity3d.com> parents: 22357 diff changeset	148
24159 5b4ed033390b revisionbranchcache: fall back to slow path if starting readonly (issue4531) Mads Kiilerich <madski@unity3d.com> parents: 23877 diff changeset	149 def __init__(self, repo, readonly=True):
23785 cb99bacb9b4e branchcache: introduce revbranchcache for caching of revision branch names Mads Kiilerich <madski@unity3d.com> parents: 22357 diff changeset	150 assert repo.filtername is None
24374 77fd1fb538cd revbranchcache: store repo on the object Durham Goode <durham@fb.com> parents: 24373 diff changeset	151 self._repo = repo
43076 2372284d9457 formatting: blacken the codebase Augie Fackler <augie@google.com> parents: 42603 diff changeset	152 self._names = [] # branch names in local encoding with static index
51372 02e7d79edf62 branchmap: use mmap for faster revbranchcache loading Arseniy Alekseyev <aalekseyev@janestreet.com> parents: 51287 diff changeset	153 self._rbcrevs = rbcrevs(bytearray())
43076 2372284d9457 formatting: blacken the codebase Augie Fackler <augie@google.com> parents: 42603 diff changeset	154 self._rbcsnameslen = 0 # length of names read at _rbcsnameslen
51904 16efed18ae4e rev-branch-cache: schedule a write of the "v2" format if we read from "v1" Pierre-Yves David <pierre-yves.david@octobus.net> parents: 51903 diff changeset	155 self._force_overwrite = False
51903 bd7359c18d69 rev-branch-cache: fallback on "v1" data if no v2 is found Pierre-Yves David <pierre-yves.david@octobus.net> parents: 51902 diff changeset	156 v1_fallback = False
23785 cb99bacb9b4e branchcache: introduce revbranchcache for caching of revision branch names Mads Kiilerich <madski@unity3d.com> parents: 22357 diff changeset	157 try:
51903 bd7359c18d69 rev-branch-cache: fallback on "v1" data if no v2 is found Pierre-Yves David <pierre-yves.david@octobus.net> parents: 51902 diff changeset	158 try:
bd7359c18d69 rev-branch-cache: fallback on "v1" data if no v2 is found Pierre-Yves David <pierre-yves.david@octobus.net> parents: 51902 diff changeset	159 bndata = repo.cachevfs.read(_rbcnames)
bd7359c18d69 rev-branch-cache: fallback on "v1" data if no v2 is found Pierre-Yves David <pierre-yves.david@octobus.net> parents: 51902 diff changeset	160 except (IOError, OSError):
bd7359c18d69 rev-branch-cache: fallback on "v1" data if no v2 is found Pierre-Yves David <pierre-yves.david@octobus.net> parents: 51902 diff changeset	161 # If we don't have "v2" data, we might have "v1" data worth
bd7359c18d69 rev-branch-cache: fallback on "v1" data if no v2 is found Pierre-Yves David <pierre-yves.david@octobus.net> parents: 51902 diff changeset	162 # using.
bd7359c18d69 rev-branch-cache: fallback on "v1" data if no v2 is found Pierre-Yves David <pierre-yves.david@octobus.net> parents: 51902 diff changeset	163 #
bd7359c18d69 rev-branch-cache: fallback on "v1" data if no v2 is found Pierre-Yves David <pierre-yves.david@octobus.net> parents: 51902 diff changeset	164 # consider stop doing this many version after hg-6.9 release
bd7359c18d69 rev-branch-cache: fallback on "v1" data if no v2 is found Pierre-Yves David <pierre-yves.david@octobus.net> parents: 51902 diff changeset	165 bndata = repo.cachevfs.read(_rbc_legacy_names)
bd7359c18d69 rev-branch-cache: fallback on "v1" data if no v2 is found Pierre-Yves David <pierre-yves.david@octobus.net> parents: 51902 diff changeset	166 v1_fallback = True
51904 16efed18ae4e rev-branch-cache: schedule a write of the "v2" format if we read from "v1" Pierre-Yves David <pierre-yves.david@octobus.net> parents: 51903 diff changeset	167 self._force_overwrite = True
43076 2372284d9457 formatting: blacken the codebase Augie Fackler <augie@google.com> parents: 42603 diff changeset	168 self._rbcsnameslen = len(bndata) # for verification before writing
31371 7dd2f51f38ac rbc: empty (and invalid) rbc-names file should give an empty name list Mads Kiilerich <mads@kiilerich.com> parents: 31370 diff changeset	169 if bndata:
43076 2372284d9457 formatting: blacken the codebase Augie Fackler <augie@google.com> parents: 42603 diff changeset	170 self._names = [
43077 687b865b95ad formatting: byteify all mercurial/ and hgext/ string literals Augie Fackler <augie@google.com> parents: 43076 diff changeset	171 encoding.tolocal(bn) for bn in bndata.split(b'\0')
43076 2372284d9457 formatting: blacken the codebase Augie Fackler <augie@google.com> parents: 42603 diff changeset	172 ]
29423 d2c6f3a948fa branchmap: remove unused exception variable Gregory Szorc <gregory.szorc@gmail.com> parents: 28558 diff changeset	173 except (IOError, OSError):
24159 5b4ed033390b revisionbranchcache: fall back to slow path if starting readonly (issue4531) Mads Kiilerich <madski@unity3d.com> parents: 23877 diff changeset	174 if readonly:
5b4ed033390b revisionbranchcache: fall back to slow path if starting readonly (issue4531) Mads Kiilerich <madski@unity3d.com> parents: 23877 diff changeset	175 # don't try to use cache - fall back to the slow path
5b4ed033390b revisionbranchcache: fall back to slow path if starting readonly (issue4531) Mads Kiilerich <madski@unity3d.com> parents: 23877 diff changeset	176 self.branchinfo = self._branchinfo
5b4ed033390b revisionbranchcache: fall back to slow path if starting readonly (issue4531) Mads Kiilerich <madski@unity3d.com> parents: 23877 diff changeset	177
23785 cb99bacb9b4e branchcache: introduce revbranchcache for caching of revision branch names Mads Kiilerich <madski@unity3d.com> parents: 22357 diff changeset	178 if self._names:
cb99bacb9b4e branchcache: introduce revbranchcache for caching of revision branch names Mads Kiilerich <madski@unity3d.com> parents: 22357 diff changeset	179 try:
52113 db1980a361cb rev-branch-cache: disable mmapping by default on Windows Matt Harbison <matt_harbison@yahoo.com> parents: 51971 diff changeset	180 # In order to rename the atomictempfile in _writerevs(), the
db1980a361cb rev-branch-cache: disable mmapping by default on Windows Matt Harbison <matt_harbison@yahoo.com> parents: 51971 diff changeset	181 # existing file needs to be removed. The Windows code
db1980a361cb rev-branch-cache: disable mmapping by default on Windows Matt Harbison <matt_harbison@yahoo.com> parents: 51971 diff changeset	182 # (successfully) renames it to a temp file first, before moving
db1980a361cb rev-branch-cache: disable mmapping by default on Windows Matt Harbison <matt_harbison@yahoo.com> parents: 51971 diff changeset	183 # the temp file into its place. But the removal of the original
db1980a361cb rev-branch-cache: disable mmapping by default on Windows Matt Harbison <matt_harbison@yahoo.com> parents: 51971 diff changeset	184 # file then fails, because it's still mapped. The mmap object
db1980a361cb rev-branch-cache: disable mmapping by default on Windows Matt Harbison <matt_harbison@yahoo.com> parents: 51971 diff changeset	185 # needs to be closed in order to remove the file, but in order
db1980a361cb rev-branch-cache: disable mmapping by default on Windows Matt Harbison <matt_harbison@yahoo.com> parents: 51971 diff changeset	186 # to do that, the memoryview returned by util.buffer needs to be
db1980a361cb rev-branch-cache: disable mmapping by default on Windows Matt Harbison <matt_harbison@yahoo.com> parents: 51971 diff changeset	187 # released.
db1980a361cb rev-branch-cache: disable mmapping by default on Windows Matt Harbison <matt_harbison@yahoo.com> parents: 51971 diff changeset	188 usemmap = repo.ui.configbool(
db1980a361cb rev-branch-cache: disable mmapping by default on Windows Matt Harbison <matt_harbison@yahoo.com> parents: 51971 diff changeset	189 b'storage',
db1980a361cb rev-branch-cache: disable mmapping by default on Windows Matt Harbison <matt_harbison@yahoo.com> parents: 51971 diff changeset	190 b'revbranchcache.mmap',
db1980a361cb rev-branch-cache: disable mmapping by default on Windows Matt Harbison <matt_harbison@yahoo.com> parents: 51971 diff changeset	191 default=not pycompat.iswindows,
db1980a361cb rev-branch-cache: disable mmapping by default on Windows Matt Harbison <matt_harbison@yahoo.com> parents: 51971 diff changeset	192 )
51903 bd7359c18d69 rev-branch-cache: fallback on "v1" data if no v2 is found Pierre-Yves David <pierre-yves.david@octobus.net> parents: 51902 diff changeset	193 if not v1_fallback:
bd7359c18d69 rev-branch-cache: fallback on "v1" data if no v2 is found Pierre-Yves David <pierre-yves.david@octobus.net> parents: 51902 diff changeset	194 with repo.cachevfs(_rbcrevs) as fp:
bd7359c18d69 rev-branch-cache: fallback on "v1" data if no v2 is found Pierre-Yves David <pierre-yves.david@octobus.net> parents: 51902 diff changeset	195 if usemmap and repo.cachevfs.is_mmap_safe(_rbcrevs):
bd7359c18d69 rev-branch-cache: fallback on "v1" data if no v2 is found Pierre-Yves David <pierre-yves.david@octobus.net> parents: 51902 diff changeset	196 data = util.buffer(util.mmapread(fp))
bd7359c18d69 rev-branch-cache: fallback on "v1" data if no v2 is found Pierre-Yves David <pierre-yves.david@octobus.net> parents: 51902 diff changeset	197 else:
bd7359c18d69 rev-branch-cache: fallback on "v1" data if no v2 is found Pierre-Yves David <pierre-yves.david@octobus.net> parents: 51902 diff changeset	198 data = fp.read()
bd7359c18d69 rev-branch-cache: fallback on "v1" data if no v2 is found Pierre-Yves David <pierre-yves.david@octobus.net> parents: 51902 diff changeset	199 else:
bd7359c18d69 rev-branch-cache: fallback on "v1" data if no v2 is found Pierre-Yves David <pierre-yves.david@octobus.net> parents: 51902 diff changeset	200 # If we don't have "v2" data, we might have "v1" data worth
bd7359c18d69 rev-branch-cache: fallback on "v1" data if no v2 is found Pierre-Yves David <pierre-yves.david@octobus.net> parents: 51902 diff changeset	201 # using.
bd7359c18d69 rev-branch-cache: fallback on "v1" data if no v2 is found Pierre-Yves David <pierre-yves.david@octobus.net> parents: 51902 diff changeset	202 #
bd7359c18d69 rev-branch-cache: fallback on "v1" data if no v2 is found Pierre-Yves David <pierre-yves.david@octobus.net> parents: 51902 diff changeset	203 # Consider stop doing this many version after hg-6.9
bd7359c18d69 rev-branch-cache: fallback on "v1" data if no v2 is found Pierre-Yves David <pierre-yves.david@octobus.net> parents: 51902 diff changeset	204 # release.
bd7359c18d69 rev-branch-cache: fallback on "v1" data if no v2 is found Pierre-Yves David <pierre-yves.david@octobus.net> parents: 51902 diff changeset	205 with repo.cachevfs(_rbc_legacy_revs) as fp:
51651 c0e30a019ce1 mmap: only use mmap to read rev-branch-cache data if it is safe Pierre-Yves David <pierre-yves.david@octobus.net> parents: 51643 diff changeset	206 data = fp.read()
51372 02e7d79edf62 branchmap: use mmap for faster revbranchcache loading Arseniy Alekseyev <aalekseyev@janestreet.com> parents: 51287 diff changeset	207 self._rbcrevs = rbcrevs(data)
25660 328739ea70c3 global: mass rewrite to use modern exception syntax Gregory Szorc <gregory.szorc@gmail.com> parents: 25266 diff changeset	208 except (IOError, OSError) as inst:
43076 2372284d9457 formatting: blacken the codebase Augie Fackler <augie@google.com> parents: 42603 diff changeset	209 repo.ui.debug(
43077 687b865b95ad formatting: byteify all mercurial/ and hgext/ string literals Augie Fackler <augie@google.com> parents: 43076 diff changeset	210 b"couldn't read revision branch cache: %s\n"
43076 2372284d9457 formatting: blacken the codebase Augie Fackler <augie@google.com> parents: 42603 diff changeset	211 % stringutil.forcebytestr(inst)
2372284d9457 formatting: blacken the codebase Augie Fackler <augie@google.com> parents: 42603 diff changeset	212 )
23785 cb99bacb9b4e branchcache: introduce revbranchcache for caching of revision branch names Mads Kiilerich <madski@unity3d.com> parents: 22357 diff changeset	213 # remember number of good records on disk
43076 2372284d9457 formatting: blacken the codebase Augie Fackler <augie@google.com> parents: 42603 diff changeset	214 self._rbcrevslen = min(
2372284d9457 formatting: blacken the codebase Augie Fackler <augie@google.com> parents: 42603 diff changeset	215 len(self._rbcrevs) // _rbcrecsize, len(repo.changelog)
2372284d9457 formatting: blacken the codebase Augie Fackler <augie@google.com> parents: 42603 diff changeset	216 )
23785 cb99bacb9b4e branchcache: introduce revbranchcache for caching of revision branch names Mads Kiilerich <madski@unity3d.com> parents: 22357 diff changeset	217 if self._rbcrevslen == 0:
cb99bacb9b4e branchcache: introduce revbranchcache for caching of revision branch names Mads Kiilerich <madski@unity3d.com> parents: 22357 diff changeset	218 self._names = []
43076 2372284d9457 formatting: blacken the codebase Augie Fackler <augie@google.com> parents: 42603 diff changeset	219 self._rbcnamescount = len(self._names) # number of names read at
2372284d9457 formatting: blacken the codebase Augie Fackler <augie@google.com> parents: 42603 diff changeset	220 # _rbcsnameslen
23785 cb99bacb9b4e branchcache: introduce revbranchcache for caching of revision branch names Mads Kiilerich <madski@unity3d.com> parents: 22357 diff changeset	221
28558 bcd106d456c4 cache: rebuild branch cache from scratch when inconsistencies are detected Mads Kiilerich <madski@unity3d.com> parents: 28557 diff changeset	222 def _clear(self):
bcd106d456c4 cache: rebuild branch cache from scratch when inconsistencies are detected Mads Kiilerich <madski@unity3d.com> parents: 28557 diff changeset	223 self._rbcsnameslen = 0
bcd106d456c4 cache: rebuild branch cache from scratch when inconsistencies are detected Mads Kiilerich <madski@unity3d.com> parents: 28557 diff changeset	224 del self._names[:]
bcd106d456c4 cache: rebuild branch cache from scratch when inconsistencies are detected Mads Kiilerich <madski@unity3d.com> parents: 28557 diff changeset	225 self._rbcnamescount = 0
bcd106d456c4 cache: rebuild branch cache from scratch when inconsistencies are detected Mads Kiilerich <madski@unity3d.com> parents: 28557 diff changeset	226 self._rbcrevslen = len(self._repo.changelog)
51372 02e7d79edf62 branchmap: use mmap for faster revbranchcache loading Arseniy Alekseyev <aalekseyev@janestreet.com> parents: 51287 diff changeset	227 self._rbcrevs = rbcrevs(bytearray(self._rbcrevslen * _rbcrecsize))
43077 687b865b95ad formatting: byteify all mercurial/ and hgext/ string literals Augie Fackler <augie@google.com> parents: 43076 diff changeset	228 util.clearcachedproperty(self, b'_namesreverse')
51899 9f7cf869e9f4 rev-branch-cache: add a way to force rewrite of the cache Pierre-Yves David <pierre-yves.david@octobus.net> parents: 51898 diff changeset	229 self._force_overwrite = True
9f7cf869e9f4 rev-branch-cache: add a way to force rewrite of the cache Pierre-Yves David <pierre-yves.david@octobus.net> parents: 51898 diff changeset	230
9f7cf869e9f4 rev-branch-cache: add a way to force rewrite of the cache Pierre-Yves David <pierre-yves.david@octobus.net> parents: 51898 diff changeset	231 def invalidate(self, rev=0):
9f7cf869e9f4 rev-branch-cache: add a way to force rewrite of the cache Pierre-Yves David <pierre-yves.david@octobus.net> parents: 51898 diff changeset	232 self._rbcrevslen = rev
9f7cf869e9f4 rev-branch-cache: add a way to force rewrite of the cache Pierre-Yves David <pierre-yves.david@octobus.net> parents: 51898 diff changeset	233 self._rbcrevs.truncate(rev)
9f7cf869e9f4 rev-branch-cache: add a way to force rewrite of the cache Pierre-Yves David <pierre-yves.david@octobus.net> parents: 51898 diff changeset	234 self._force_overwrite = True
40710 50a64c321c1e branchmap: build the revbranchcache._namesreverse() only when required Pulkit Goyal <pulkit@yandex-team.ru> parents: 40425 diff changeset	235
50a64c321c1e branchmap: build the revbranchcache._namesreverse() only when required Pulkit Goyal <pulkit@yandex-team.ru> parents: 40425 diff changeset	236 @util.propertycache
50a64c321c1e branchmap: build the revbranchcache._namesreverse() only when required Pulkit Goyal <pulkit@yandex-team.ru> parents: 40425 diff changeset	237 def _namesreverse(self):
44452 9d2b2df2c2ba cleanup: run pyupgrade on our source tree to clean up varying things Augie Fackler <augie@google.com> parents: 44306 diff changeset	238 return {b: r for r, b in enumerate(self._names)}
28558 bcd106d456c4 cache: rebuild branch cache from scratch when inconsistencies are detected Mads Kiilerich <madski@unity3d.com> parents: 28557 diff changeset	239
40425 5e5c8f2a1eb5 branchmap: do not specify changelog as an argument Yuya Nishihara <yuya@tcha.org> parents: 40375 diff changeset	240 def branchinfo(self, rev):
23785 cb99bacb9b4e branchcache: introduce revbranchcache for caching of revision branch names Mads Kiilerich <madski@unity3d.com> parents: 22357 diff changeset	241 """Return branch name and close flag for rev, using and updating
cb99bacb9b4e branchcache: introduce revbranchcache for caching of revision branch names Mads Kiilerich <madski@unity3d.com> parents: 22357 diff changeset	242 persistent cache."""
40425 5e5c8f2a1eb5 branchmap: do not specify changelog as an argument Yuya Nishihara <yuya@tcha.org> parents: 40375 diff changeset	243 changelog = self._repo.changelog
23785 cb99bacb9b4e branchcache: introduce revbranchcache for caching of revision branch names Mads Kiilerich <madski@unity3d.com> parents: 22357 diff changeset	244 rbcrevidx = rev * _rbcrecsize
cb99bacb9b4e branchcache: introduce revbranchcache for caching of revision branch names Mads Kiilerich <madski@unity3d.com> parents: 22357 diff changeset	245
25266 38117278f295 revbranchcache: return uncached branchinfo for nullrev (issue4683) Yuya Nishihara <yuya@tcha.org> parents: 24728 diff changeset	246 # avoid negative index, changelog.read(nullrev) is fast without cache
38117278f295 revbranchcache: return uncached branchinfo for nullrev (issue4683) Yuya Nishihara <yuya@tcha.org> parents: 24728 diff changeset	247 if rev == nullrev:
38117278f295 revbranchcache: return uncached branchinfo for nullrev (issue4683) Yuya Nishihara <yuya@tcha.org> parents: 24728 diff changeset	248 return changelog.branchinfo(rev)
38117278f295 revbranchcache: return uncached branchinfo for nullrev (issue4683) Yuya Nishihara <yuya@tcha.org> parents: 24728 diff changeset	249
29604 db0095c83344 rbc: fix invalid rbc-revs entries caused by missing cache growth Mads Kiilerich <madski@unity3d.com> parents: 29423 diff changeset	250 # if requested rev isn't allocated, grow and cache the rev info
23785 cb99bacb9b4e branchcache: introduce revbranchcache for caching of revision branch names Mads Kiilerich <madski@unity3d.com> parents: 22357 diff changeset	251 if len(self._rbcrevs) < rbcrevidx + _rbcrecsize:
40425 5e5c8f2a1eb5 branchmap: do not specify changelog as an argument Yuya Nishihara <yuya@tcha.org> parents: 40375 diff changeset	252 return self._branchinfo(rev)
23785 cb99bacb9b4e branchcache: introduce revbranchcache for caching of revision branch names Mads Kiilerich <madski@unity3d.com> parents: 22357 diff changeset	253
cb99bacb9b4e branchcache: introduce revbranchcache for caching of revision branch names Mads Kiilerich <madski@unity3d.com> parents: 22357 diff changeset	254 # fast path: extract data from cache, use it if node is matching
cb99bacb9b4e branchcache: introduce revbranchcache for caching of revision branch names Mads Kiilerich <madski@unity3d.com> parents: 22357 diff changeset	255 reponode = changelog.node(rev)[:_rbcnodelen]
51372 02e7d79edf62 branchmap: use mmap for faster revbranchcache loading Arseniy Alekseyev <aalekseyev@janestreet.com> parents: 51287 diff changeset	256 cachenode, branchidx = self._rbcrevs.unpack_record(rbcrevidx)
23785 cb99bacb9b4e branchcache: introduce revbranchcache for caching of revision branch names Mads Kiilerich <madski@unity3d.com> parents: 22357 diff changeset	257 close = bool(branchidx & _rbccloseflag)
cb99bacb9b4e branchcache: introduce revbranchcache for caching of revision branch names Mads Kiilerich <madski@unity3d.com> parents: 22357 diff changeset	258 if close:
cb99bacb9b4e branchcache: introduce revbranchcache for caching of revision branch names Mads Kiilerich <madski@unity3d.com> parents: 22357 diff changeset	259 branchidx &= _rbcbranchidxmask
43077 687b865b95ad formatting: byteify all mercurial/ and hgext/ string literals Augie Fackler <augie@google.com> parents: 43076 diff changeset	260 if cachenode == b'\0\0\0\0':
24376 203a078da052 revbranchcache: populate cache incrementally Durham Goode <durham@fb.com> parents: 24375 diff changeset	261 pass
203a078da052 revbranchcache: populate cache incrementally Durham Goode <durham@fb.com> parents: 24375 diff changeset	262 elif cachenode == reponode:
29615 a2a380e2750f rbc: fix superfluous rebuilding from scratch - don't abuse self._rbcnamescount Mads Kiilerich <madski@unity3d.com> parents: 29604 diff changeset	263 try:
28558 bcd106d456c4 cache: rebuild branch cache from scratch when inconsistencies are detected Mads Kiilerich <madski@unity3d.com> parents: 28557 diff changeset	264 return self._names[branchidx], close
29615 a2a380e2750f rbc: fix superfluous rebuilding from scratch - don't abuse self._rbcnamescount Mads Kiilerich <madski@unity3d.com> parents: 29604 diff changeset	265 except IndexError:
a2a380e2750f rbc: fix superfluous rebuilding from scratch - don't abuse self._rbcnamescount Mads Kiilerich <madski@unity3d.com> parents: 29604 diff changeset	266 # recover from invalid reference to unknown branch
43076 2372284d9457 formatting: blacken the codebase Augie Fackler <augie@google.com> parents: 42603 diff changeset	267 self._repo.ui.debug(
43077 687b865b95ad formatting: byteify all mercurial/ and hgext/ string literals Augie Fackler <augie@google.com> parents: 43076 diff changeset	268 b"referenced branch names not found"
687b865b95ad formatting: byteify all mercurial/ and hgext/ string literals Augie Fackler <augie@google.com> parents: 43076 diff changeset	269 b" - rebuilding revision branch cache from scratch\n"
43076 2372284d9457 formatting: blacken the codebase Augie Fackler <augie@google.com> parents: 42603 diff changeset	270 )
29615 a2a380e2750f rbc: fix superfluous rebuilding from scratch - don't abuse self._rbcnamescount Mads Kiilerich <madski@unity3d.com> parents: 29604 diff changeset	271 self._clear()
24376 203a078da052 revbranchcache: populate cache incrementally Durham Goode <durham@fb.com> parents: 24375 diff changeset	272 else:
203a078da052 revbranchcache: populate cache incrementally Durham Goode <durham@fb.com> parents: 24375 diff changeset	273 # rev/node map has changed, invalidate the cache from here up
43076 2372284d9457 formatting: blacken the codebase Augie Fackler <augie@google.com> parents: 42603 diff changeset	274 self._repo.ui.debug(
43077 687b865b95ad formatting: byteify all mercurial/ and hgext/ string literals Augie Fackler <augie@google.com> parents: 43076 diff changeset	275 b"history modification detected - truncating "
687b865b95ad formatting: byteify all mercurial/ and hgext/ string literals Augie Fackler <augie@google.com> parents: 43076 diff changeset	276 b"revision branch cache to revision %d\n" % rev
43076 2372284d9457 formatting: blacken the codebase Augie Fackler <augie@google.com> parents: 42603 diff changeset	277 )
24376 203a078da052 revbranchcache: populate cache incrementally Durham Goode <durham@fb.com> parents: 24375 diff changeset	278 truncate = rbcrevidx + _rbcrecsize
51372 02e7d79edf62 branchmap: use mmap for faster revbranchcache loading Arseniy Alekseyev <aalekseyev@janestreet.com> parents: 51287 diff changeset	279 self._rbcrevs.truncate(truncate)
24376 203a078da052 revbranchcache: populate cache incrementally Durham Goode <durham@fb.com> parents: 24375 diff changeset	280 self._rbcrevslen = min(self._rbcrevslen, truncate)
203a078da052 revbranchcache: populate cache incrementally Durham Goode <durham@fb.com> parents: 24375 diff changeset	281
23785 cb99bacb9b4e branchcache: introduce revbranchcache for caching of revision branch names Mads Kiilerich <madski@unity3d.com> parents: 22357 diff changeset	282 # fall back to slow path and make sure it will be written to disk
40425 5e5c8f2a1eb5 branchmap: do not specify changelog as an argument Yuya Nishihara <yuya@tcha.org> parents: 40375 diff changeset	283 return self._branchinfo(rev)
23785 cb99bacb9b4e branchcache: introduce revbranchcache for caching of revision branch names Mads Kiilerich <madski@unity3d.com> parents: 22357 diff changeset	284
40425 5e5c8f2a1eb5 branchmap: do not specify changelog as an argument Yuya Nishihara <yuya@tcha.org> parents: 40375 diff changeset	285 def _branchinfo(self, rev):
23785 cb99bacb9b4e branchcache: introduce revbranchcache for caching of revision branch names Mads Kiilerich <madski@unity3d.com> parents: 22357 diff changeset	286 """Retrieve branch info from changelog and update _rbcrevs"""
40425 5e5c8f2a1eb5 branchmap: do not specify changelog as an argument Yuya Nishihara <yuya@tcha.org> parents: 40375 diff changeset	287 changelog = self._repo.changelog
23785 cb99bacb9b4e branchcache: introduce revbranchcache for caching of revision branch names Mads Kiilerich <madski@unity3d.com> parents: 22357 diff changeset	288 b, close = changelog.branchinfo(rev)
cb99bacb9b4e branchcache: introduce revbranchcache for caching of revision branch names Mads Kiilerich <madski@unity3d.com> parents: 22357 diff changeset	289 if b in self._namesreverse:
cb99bacb9b4e branchcache: introduce revbranchcache for caching of revision branch names Mads Kiilerich <madski@unity3d.com> parents: 22357 diff changeset	290 branchidx = self._namesreverse[b]
cb99bacb9b4e branchcache: introduce revbranchcache for caching of revision branch names Mads Kiilerich <madski@unity3d.com> parents: 22357 diff changeset	291 else:
cb99bacb9b4e branchcache: introduce revbranchcache for caching of revision branch names Mads Kiilerich <madski@unity3d.com> parents: 22357 diff changeset	292 branchidx = len(self._names)
cb99bacb9b4e branchcache: introduce revbranchcache for caching of revision branch names Mads Kiilerich <madski@unity3d.com> parents: 22357 diff changeset	293 self._names.append(b)
cb99bacb9b4e branchcache: introduce revbranchcache for caching of revision branch names Mads Kiilerich <madski@unity3d.com> parents: 22357 diff changeset	294 self._namesreverse[b] = branchidx
cb99bacb9b4e branchcache: introduce revbranchcache for caching of revision branch names Mads Kiilerich <madski@unity3d.com> parents: 22357 diff changeset	295 reponode = changelog.node(rev)
cb99bacb9b4e branchcache: introduce revbranchcache for caching of revision branch names Mads Kiilerich <madski@unity3d.com> parents: 22357 diff changeset	296 if close:
cb99bacb9b4e branchcache: introduce revbranchcache for caching of revision branch names Mads Kiilerich <madski@unity3d.com> parents: 22357 diff changeset	297 branchidx \|= _rbccloseflag
40425 5e5c8f2a1eb5 branchmap: do not specify changelog as an argument Yuya Nishihara <yuya@tcha.org> parents: 40375 diff changeset	298 self._setcachedata(rev, reponode, branchidx)
24375 fe255b2525d5 revbranchcache: move entry writing to a separate function Durham Goode <durham@fb.com> parents: 24374 diff changeset	299 return b, close
fe255b2525d5 revbranchcache: move entry writing to a separate function Durham Goode <durham@fb.com> parents: 24374 diff changeset	300
46372 3e91d9978bec branchmap: update rev-branch-cache incrementally Joerg Sonnenberger <joerg@bec.de> parents: 46360 diff changeset	301 def setdata(self, rev, changelogrevision):
36962 95f4f1bfb650 revbranchcache: add a public function to update the data Boris Feld <boris.feld@octobus.net> parents: 36474 diff changeset	302 """add new data information to the cache"""
46372 3e91d9978bec branchmap: update rev-branch-cache incrementally Joerg Sonnenberger <joerg@bec.de> parents: 46360 diff changeset	303 branch, close = changelogrevision.branchinfo
3e91d9978bec branchmap: update rev-branch-cache incrementally Joerg Sonnenberger <joerg@bec.de> parents: 46360 diff changeset	304
36962 95f4f1bfb650 revbranchcache: add a public function to update the data Boris Feld <boris.feld@octobus.net> parents: 36474 diff changeset	305 if branch in self._namesreverse:
95f4f1bfb650 revbranchcache: add a public function to update the data Boris Feld <boris.feld@octobus.net> parents: 36474 diff changeset	306 branchidx = self._namesreverse[branch]
95f4f1bfb650 revbranchcache: add a public function to update the data Boris Feld <boris.feld@octobus.net> parents: 36474 diff changeset	307 else:
95f4f1bfb650 revbranchcache: add a public function to update the data Boris Feld <boris.feld@octobus.net> parents: 36474 diff changeset	308 branchidx = len(self._names)
95f4f1bfb650 revbranchcache: add a public function to update the data Boris Feld <boris.feld@octobus.net> parents: 36474 diff changeset	309 self._names.append(branch)
95f4f1bfb650 revbranchcache: add a public function to update the data Boris Feld <boris.feld@octobus.net> parents: 36474 diff changeset	310 self._namesreverse[branch] = branchidx
95f4f1bfb650 revbranchcache: add a public function to update the data Boris Feld <boris.feld@octobus.net> parents: 36474 diff changeset	311 if close:
95f4f1bfb650 revbranchcache: add a public function to update the data Boris Feld <boris.feld@octobus.net> parents: 36474 diff changeset	312 branchidx \|= _rbccloseflag
46372 3e91d9978bec branchmap: update rev-branch-cache incrementally Joerg Sonnenberger <joerg@bec.de> parents: 46360 diff changeset	313 self._setcachedata(rev, self._repo.changelog.node(rev), branchidx)
36962 95f4f1bfb650 revbranchcache: add a public function to update the data Boris Feld <boris.feld@octobus.net> parents: 36474 diff changeset	314 # If no cache data were readable (non exists, bad permission, etc)
95f4f1bfb650 revbranchcache: add a public function to update the data Boris Feld <boris.feld@octobus.net> parents: 36474 diff changeset	315 # the cache was bypassing itself by setting:
95f4f1bfb650 revbranchcache: add a public function to update the data Boris Feld <boris.feld@octobus.net> parents: 36474 diff changeset	316 #
95f4f1bfb650 revbranchcache: add a public function to update the data Boris Feld <boris.feld@octobus.net> parents: 36474 diff changeset	317 # self.branchinfo = self._branchinfo
95f4f1bfb650 revbranchcache: add a public function to update the data Boris Feld <boris.feld@octobus.net> parents: 36474 diff changeset	318 #
95f4f1bfb650 revbranchcache: add a public function to update the data Boris Feld <boris.feld@octobus.net> parents: 36474 diff changeset	319 # Since we now have data in the cache, we need to drop this bypassing.
43506 9f70512ae2cf cleanup: remove pointless r-prefixes on single-quoted strings Augie Fackler <augie@google.com> parents: 43499 diff changeset	320 if 'branchinfo' in vars(self):
36962 95f4f1bfb650 revbranchcache: add a public function to update the data Boris Feld <boris.feld@octobus.net> parents: 36474 diff changeset	321 del self.branchinfo
95f4f1bfb650 revbranchcache: add a public function to update the data Boris Feld <boris.feld@octobus.net> parents: 36474 diff changeset	322
40425 5e5c8f2a1eb5 branchmap: do not specify changelog as an argument Yuya Nishihara <yuya@tcha.org> parents: 40375 diff changeset	323 def _setcachedata(self, rev, node, branchidx):
24375 fe255b2525d5 revbranchcache: move entry writing to a separate function Durham Goode <durham@fb.com> parents: 24374 diff changeset	324 """Writes the node's branch data to the in-memory cache data."""
31454 a5bad127128d branchmap: handle nullrev in setcachedata Durham Goode <durham@fb.com> parents: 31381 diff changeset	325 if rev == nullrev:
a5bad127128d branchmap: handle nullrev in setcachedata Durham Goode <durham@fb.com> parents: 31381 diff changeset	326 return
23785 cb99bacb9b4e branchcache: introduce revbranchcache for caching of revision branch names Mads Kiilerich <madski@unity3d.com> parents: 22357 diff changeset	327 rbcrevidx = rev * _rbcrecsize
51372 02e7d79edf62 branchmap: use mmap for faster revbranchcache loading Arseniy Alekseyev <aalekseyev@janestreet.com> parents: 51287 diff changeset	328 self._rbcrevs.pack_into(rbcrevidx, node, branchidx)
24376 203a078da052 revbranchcache: populate cache incrementally Durham Goode <durham@fb.com> parents: 24375 diff changeset	329 self._rbcrevslen = min(self._rbcrevslen, rev)
23785 cb99bacb9b4e branchcache: introduce revbranchcache for caching of revision branch names Mads Kiilerich <madski@unity3d.com> parents: 22357 diff changeset	330
24377 656f93ce66d5 revbranchcache: move cache writing to the transaction finalizer Durham Goode <durham@fb.com> parents: 24376 diff changeset	331 tr = self._repo.currenttransaction()
656f93ce66d5 revbranchcache: move cache writing to the transaction finalizer Durham Goode <durham@fb.com> parents: 24376 diff changeset	332 if tr:
43077 687b865b95ad formatting: byteify all mercurial/ and hgext/ string literals Augie Fackler <augie@google.com> parents: 43076 diff changeset	333 tr.addfinalize(b'write-revbranchcache', self.write)
24377 656f93ce66d5 revbranchcache: move cache writing to the transaction finalizer Durham Goode <durham@fb.com> parents: 24376 diff changeset	334
656f93ce66d5 revbranchcache: move cache writing to the transaction finalizer Durham Goode <durham@fb.com> parents: 24376 diff changeset	335 def write(self, tr=None):
23785 cb99bacb9b4e branchcache: introduce revbranchcache for caching of revision branch names Mads Kiilerich <madski@unity3d.com> parents: 22357 diff changeset	336 """Save branch cache if it is dirty."""
24374 77fd1fb538cd revbranchcache: store repo on the object Durham Goode <durham@fb.com> parents: 24373 diff changeset	337 repo = self._repo
29744 0d588332ad2c branchmap: acquires lock before writting the rev branch cache Pierre-Yves David <pierre-yves.david@ens-lyon.org> parents: 29743 diff changeset	338 wlock = None
43077 687b865b95ad formatting: byteify all mercurial/ and hgext/ string literals Augie Fackler <augie@google.com> parents: 43076 diff changeset	339 step = b''
29744 0d588332ad2c branchmap: acquires lock before writting the rev branch cache Pierre-Yves David <pierre-yves.david@ens-lyon.org> parents: 29743 diff changeset	340 try:
42184 09fd338522fa revbranchcache: factor logic to write names and revs in separate functions Pulkit Goyal <pulkit@yandex-team.ru> parents: 42138 diff changeset	341 # write the new names
51904 16efed18ae4e rev-branch-cache: schedule a write of the "v2" format if we read from "v1" Pierre-Yves David <pierre-yves.david@octobus.net> parents: 51903 diff changeset	342 if self._force_overwrite or self._rbcnamescount < len(self._names):
29744 0d588332ad2c branchmap: acquires lock before writting the rev branch cache Pierre-Yves David <pierre-yves.david@ens-lyon.org> parents: 29743 diff changeset	343 wlock = repo.wlock(wait=False)
43077 687b865b95ad formatting: byteify all mercurial/ and hgext/ string literals Augie Fackler <augie@google.com> parents: 43076 diff changeset	344 step = b' names'
42184 09fd338522fa revbranchcache: factor logic to write names and revs in separate functions Pulkit Goyal <pulkit@yandex-team.ru> parents: 42138 diff changeset	345 self._writenames(repo)
23785 cb99bacb9b4e branchcache: introduce revbranchcache for caching of revision branch names Mads Kiilerich <madski@unity3d.com> parents: 22357 diff changeset	346
42184 09fd338522fa revbranchcache: factor logic to write names and revs in separate functions Pulkit Goyal <pulkit@yandex-team.ru> parents: 42138 diff changeset	347 # write the new revs
29743 9f3c49ee4486 branchmap: preparatory indent of indent the branch rev writing code Pierre-Yves David <pierre-yves.david@ens-lyon.org> parents: 29615 diff changeset	348 start = self._rbcrevslen * _rbcrecsize
51899 9f7cf869e9f4 rev-branch-cache: add a way to force rewrite of the cache Pierre-Yves David <pierre-yves.david@octobus.net> parents: 51898 diff changeset	349 if self._force_overwrite or start != len(self._rbcrevs):
43077 687b865b95ad formatting: byteify all mercurial/ and hgext/ string literals Augie Fackler <augie@google.com> parents: 43076 diff changeset	350 step = b''
29744 0d588332ad2c branchmap: acquires lock before writting the rev branch cache Pierre-Yves David <pierre-yves.david@ens-lyon.org> parents: 29743 diff changeset	351 if wlock is None:
0d588332ad2c branchmap: acquires lock before writting the rev branch cache Pierre-Yves David <pierre-yves.david@ens-lyon.org> parents: 29743 diff changeset	352 wlock = repo.wlock(wait=False)
42184 09fd338522fa revbranchcache: factor logic to write names and revs in separate functions Pulkit Goyal <pulkit@yandex-team.ru> parents: 42138 diff changeset	353 self._writerevs(repo, start)
09fd338522fa revbranchcache: factor logic to write names and revs in separate functions Pulkit Goyal <pulkit@yandex-team.ru> parents: 42138 diff changeset	354
29745 3b184adfb5be branchmap: simplify error handlind when writing rev branch cache Pierre-Yves David <pierre-yves.david@ens-lyon.org> parents: 29744 diff changeset	355 except (IOError, OSError, error.Abort, error.LockError) as inst:
43076 2372284d9457 formatting: blacken the codebase Augie Fackler <augie@google.com> parents: 42603 diff changeset	356 repo.ui.debug(
43077 687b865b95ad formatting: byteify all mercurial/ and hgext/ string literals Augie Fackler <augie@google.com> parents: 43076 diff changeset	357 b"couldn't write revision branch cache%s: %s\n"
43076 2372284d9457 formatting: blacken the codebase Augie Fackler <augie@google.com> parents: 42603 diff changeset	358 % (step, stringutil.forcebytestr(inst))
2372284d9457 formatting: blacken the codebase Augie Fackler <augie@google.com> parents: 42603 diff changeset	359 )
29744 0d588332ad2c branchmap: acquires lock before writting the rev branch cache Pierre-Yves David <pierre-yves.david@ens-lyon.org> parents: 29743 diff changeset	360 finally:
0d588332ad2c branchmap: acquires lock before writting the rev branch cache Pierre-Yves David <pierre-yves.david@ens-lyon.org> parents: 29743 diff changeset	361 if wlock is not None:
0d588332ad2c branchmap: acquires lock before writting the rev branch cache Pierre-Yves David <pierre-yves.david@ens-lyon.org> parents: 29743 diff changeset	362 wlock.release()
42184 09fd338522fa revbranchcache: factor logic to write names and revs in separate functions Pulkit Goyal <pulkit@yandex-team.ru> parents: 42138 diff changeset	363
09fd338522fa revbranchcache: factor logic to write names and revs in separate functions Pulkit Goyal <pulkit@yandex-team.ru> parents: 42138 diff changeset	364 def _writenames(self, repo):
47062 f38bf44e077f black: make codebase compatible with black v21.4b2 and v20.8b1 Kyle Lippincott <spectral@google.com> parents: 46819 diff changeset	365 """write the new branch names to revbranchcache"""
51900 7032da075572 rev-branch-cache: make sure we close the name file we open Pierre-Yves David <pierre-yves.david@octobus.net> parents: 51899 diff changeset	366 f = None
51904 16efed18ae4e rev-branch-cache: schedule a write of the "v2" format if we read from "v1" Pierre-Yves David <pierre-yves.david@octobus.net> parents: 51903 diff changeset	367 if self._force_overwrite:
16efed18ae4e rev-branch-cache: schedule a write of the "v2" format if we read from "v1" Pierre-Yves David <pierre-yves.david@octobus.net> parents: 51903 diff changeset	368 self._rbcsnameslen = 0
16efed18ae4e rev-branch-cache: schedule a write of the "v2" format if we read from "v1" Pierre-Yves David <pierre-yves.david@octobus.net> parents: 51903 diff changeset	369 self._rbcnamescount = 0
51900 7032da075572 rev-branch-cache: make sure we close the name file we open Pierre-Yves David <pierre-yves.david@octobus.net> parents: 51899 diff changeset	370 try:
51904 16efed18ae4e rev-branch-cache: schedule a write of the "v2" format if we read from "v1" Pierre-Yves David <pierre-yves.david@octobus.net> parents: 51903 diff changeset	371 if self._force_overwrite or self._rbcnamescount != 0:
51900 7032da075572 rev-branch-cache: make sure we close the name file we open Pierre-Yves David <pierre-yves.david@octobus.net> parents: 51899 diff changeset	372 f = repo.cachevfs.open(_rbcnames, b'ab')
51904 16efed18ae4e rev-branch-cache: schedule a write of the "v2" format if we read from "v1" Pierre-Yves David <pierre-yves.david@octobus.net> parents: 51903 diff changeset	373 current_size = f.tell()
16efed18ae4e rev-branch-cache: schedule a write of the "v2" format if we read from "v1" Pierre-Yves David <pierre-yves.david@octobus.net> parents: 51903 diff changeset	374 if current_size == self._rbcsnameslen:
51900 7032da075572 rev-branch-cache: make sure we close the name file we open Pierre-Yves David <pierre-yves.david@octobus.net> parents: 51899 diff changeset	375 f.write(b'\0')
7032da075572 rev-branch-cache: make sure we close the name file we open Pierre-Yves David <pierre-yves.david@octobus.net> parents: 51899 diff changeset	376 else:
7032da075572 rev-branch-cache: make sure we close the name file we open Pierre-Yves David <pierre-yves.david@octobus.net> parents: 51899 diff changeset	377 f.close()
51904 16efed18ae4e rev-branch-cache: schedule a write of the "v2" format if we read from "v1" Pierre-Yves David <pierre-yves.david@octobus.net> parents: 51903 diff changeset	378 if self._force_overwrite:
16efed18ae4e rev-branch-cache: schedule a write of the "v2" format if we read from "v1" Pierre-Yves David <pierre-yves.david@octobus.net> parents: 51903 diff changeset	379 dbg = b"resetting content of %s\n"
16efed18ae4e rev-branch-cache: schedule a write of the "v2" format if we read from "v1" Pierre-Yves David <pierre-yves.david@octobus.net> parents: 51903 diff changeset	380 elif current_size > 0:
16efed18ae4e rev-branch-cache: schedule a write of the "v2" format if we read from "v1" Pierre-Yves David <pierre-yves.david@octobus.net> parents: 51903 diff changeset	381 dbg = b"%s changed - rewriting it\n"
16efed18ae4e rev-branch-cache: schedule a write of the "v2" format if we read from "v1" Pierre-Yves David <pierre-yves.david@octobus.net> parents: 51903 diff changeset	382 else:
16efed18ae4e rev-branch-cache: schedule a write of the "v2" format if we read from "v1" Pierre-Yves David <pierre-yves.david@octobus.net> parents: 51903 diff changeset	383 dbg = b"%s is missing - rewriting it\n"
16efed18ae4e rev-branch-cache: schedule a write of the "v2" format if we read from "v1" Pierre-Yves David <pierre-yves.david@octobus.net> parents: 51903 diff changeset	384 repo.ui.debug(dbg % _rbcnames)
51900 7032da075572 rev-branch-cache: make sure we close the name file we open Pierre-Yves David <pierre-yves.david@octobus.net> parents: 51899 diff changeset	385 self._rbcnamescount = 0
7032da075572 rev-branch-cache: make sure we close the name file we open Pierre-Yves David <pierre-yves.david@octobus.net> parents: 51899 diff changeset	386 self._rbcrevslen = 0
7032da075572 rev-branch-cache: make sure we close the name file we open Pierre-Yves David <pierre-yves.david@octobus.net> parents: 51899 diff changeset	387 if self._rbcnamescount == 0:
7032da075572 rev-branch-cache: make sure we close the name file we open Pierre-Yves David <pierre-yves.david@octobus.net> parents: 51899 diff changeset	388 # before rewriting names, make sure references are removed
7032da075572 rev-branch-cache: make sure we close the name file we open Pierre-Yves David <pierre-yves.david@octobus.net> parents: 51899 diff changeset	389 repo.cachevfs.unlinkpath(_rbcrevs, ignoremissing=True)
7032da075572 rev-branch-cache: make sure we close the name file we open Pierre-Yves David <pierre-yves.david@octobus.net> parents: 51899 diff changeset	390 f = repo.cachevfs.open(_rbcnames, b'wb')
7032da075572 rev-branch-cache: make sure we close the name file we open Pierre-Yves David <pierre-yves.david@octobus.net> parents: 51899 diff changeset	391 names = self._names[self._rbcnamescount :]
7032da075572 rev-branch-cache: make sure we close the name file we open Pierre-Yves David <pierre-yves.david@octobus.net> parents: 51899 diff changeset	392 from_local = encoding.fromlocal
7032da075572 rev-branch-cache: make sure we close the name file we open Pierre-Yves David <pierre-yves.david@octobus.net> parents: 51899 diff changeset	393 data = b'\0'.join(from_local(b) for b in names)
7032da075572 rev-branch-cache: make sure we close the name file we open Pierre-Yves David <pierre-yves.david@octobus.net> parents: 51899 diff changeset	394 f.write(data)
7032da075572 rev-branch-cache: make sure we close the name file we open Pierre-Yves David <pierre-yves.david@octobus.net> parents: 51899 diff changeset	395 self._rbcsnameslen = f.tell()
7032da075572 rev-branch-cache: make sure we close the name file we open Pierre-Yves David <pierre-yves.david@octobus.net> parents: 51899 diff changeset	396 finally:
7032da075572 rev-branch-cache: make sure we close the name file we open Pierre-Yves David <pierre-yves.david@octobus.net> parents: 51899 diff changeset	397 if f is not None:
42184 09fd338522fa revbranchcache: factor logic to write names and revs in separate functions Pulkit Goyal <pulkit@yandex-team.ru> parents: 42138 diff changeset	398 f.close()
09fd338522fa revbranchcache: factor logic to write names and revs in separate functions Pulkit Goyal <pulkit@yandex-team.ru> parents: 42138 diff changeset	399 self._rbcnamescount = len(self._names)
09fd338522fa revbranchcache: factor logic to write names and revs in separate functions Pulkit Goyal <pulkit@yandex-team.ru> parents: 42138 diff changeset	400
09fd338522fa revbranchcache: factor logic to write names and revs in separate functions Pulkit Goyal <pulkit@yandex-team.ru> parents: 42138 diff changeset	401 def _writerevs(self, repo, start):
47062 f38bf44e077f black: make codebase compatible with black v21.4b2 and v20.8b1 Kyle Lippincott <spectral@google.com> parents: 46819 diff changeset	402 """write the new revs to revbranchcache"""
42185 ececa45c80d8 revbranchcache: use context manager in _writerevs() to write to file Pulkit Goyal <pulkit@yandex-team.ru> parents: 42184 diff changeset	403 revs = min(len(repo.changelog), len(self._rbcrevs) // _rbcrecsize)
51901 c564be351754 rev-branch-cache: stop truncating cache file Pierre-Yves David <pierre-yves.david@octobus.net> parents: 51900 diff changeset	404
c564be351754 rev-branch-cache: stop truncating cache file Pierre-Yves David <pierre-yves.david@octobus.net> parents: 51900 diff changeset	405 end = revs * _rbcrecsize
51899 9f7cf869e9f4 rev-branch-cache: add a way to force rewrite of the cache Pierre-Yves David <pierre-yves.david@octobus.net> parents: 51898 diff changeset	406 if self._force_overwrite:
9f7cf869e9f4 rev-branch-cache: add a way to force rewrite of the cache Pierre-Yves David <pierre-yves.david@octobus.net> parents: 51898 diff changeset	407 start = 0
51901 c564be351754 rev-branch-cache: stop truncating cache file Pierre-Yves David <pierre-yves.david@octobus.net> parents: 51900 diff changeset	408
51971 76416b6e9d9b rev-branch-cache: properly ignores unaligned trailing data Pierre-Yves David <pierre-yves.david@octobus.net> parents: 51970 diff changeset	409 # align start on entry boundary
76416b6e9d9b rev-branch-cache: properly ignores unaligned trailing data Pierre-Yves David <pierre-yves.david@octobus.net> parents: 51970 diff changeset	410 start = _rbcrecsize * (start // _rbcrecsize)
76416b6e9d9b rev-branch-cache: properly ignores unaligned trailing data Pierre-Yves David <pierre-yves.david@octobus.net> parents: 51970 diff changeset	411
51901 c564be351754 rev-branch-cache: stop truncating cache file Pierre-Yves David <pierre-yves.david@octobus.net> parents: 51900 diff changeset	412 with repo.cachevfs.open(_rbcrevs, b'a+b') as f:
c564be351754 rev-branch-cache: stop truncating cache file Pierre-Yves David <pierre-yves.david@octobus.net> parents: 51900 diff changeset	413 pass # this make sure the file exist…
c564be351754 rev-branch-cache: stop truncating cache file Pierre-Yves David <pierre-yves.david@octobus.net> parents: 51900 diff changeset	414 with repo.cachevfs.open(_rbcrevs, b'r+b') as f:
c564be351754 rev-branch-cache: stop truncating cache file Pierre-Yves David <pierre-yves.david@octobus.net> parents: 51900 diff changeset	415 f.seek(0, os.SEEK_END)
51898 1eb2317c1762 rev-branch-cache: issue more truthful "truncating" message Pierre-Yves David <pierre-yves.david@octobus.net> parents: 51897 diff changeset	416 current_size = f.tell()
1eb2317c1762 rev-branch-cache: issue more truthful "truncating" message Pierre-Yves David <pierre-yves.david@octobus.net> parents: 51897 diff changeset	417 if current_size < start:
1eb2317c1762 rev-branch-cache: issue more truthful "truncating" message Pierre-Yves David <pierre-yves.david@octobus.net> parents: 51897 diff changeset	418 start = 0
1eb2317c1762 rev-branch-cache: issue more truthful "truncating" message Pierre-Yves David <pierre-yves.david@octobus.net> parents: 51897 diff changeset	419 if current_size != start:
51901 c564be351754 rev-branch-cache: stop truncating cache file Pierre-Yves David <pierre-yves.david@octobus.net> parents: 51900 diff changeset	420 threshold = current_size * REWRITE_RATIO
51970 4c885d5ff132 rev-branch-cache: stop pretending we will overwrite data when we don't Pierre-Yves David <pierre-yves.david@octobus.net> parents: 51969 diff changeset	421 overwritten = min(end, current_size) - start
4c885d5ff132 rev-branch-cache: stop pretending we will overwrite data when we don't Pierre-Yves David <pierre-yves.david@octobus.net> parents: 51969 diff changeset	422 if (max(end, current_size) - start) >= threshold:
4c885d5ff132 rev-branch-cache: stop pretending we will overwrite data when we don't Pierre-Yves David <pierre-yves.david@octobus.net> parents: 51969 diff changeset	423 start = 0
4c885d5ff132 rev-branch-cache: stop pretending we will overwrite data when we don't Pierre-Yves David <pierre-yves.david@octobus.net> parents: 51969 diff changeset	424 dbg = b"resetting content of cache/%s\n" % _rbcrevs
4c885d5ff132 rev-branch-cache: stop pretending we will overwrite data when we don't Pierre-Yves David <pierre-yves.david@octobus.net> parents: 51969 diff changeset	425 repo.ui.debug(dbg)
4c885d5ff132 rev-branch-cache: stop pretending we will overwrite data when we don't Pierre-Yves David <pierre-yves.david@octobus.net> parents: 51969 diff changeset	426 elif overwritten > 0:
4c885d5ff132 rev-branch-cache: stop pretending we will overwrite data when we don't Pierre-Yves David <pierre-yves.david@octobus.net> parents: 51969 diff changeset	427 # end affected, let us overwrite the bad value
51901 c564be351754 rev-branch-cache: stop truncating cache file Pierre-Yves David <pierre-yves.david@octobus.net> parents: 51900 diff changeset	428 dbg = b"overwriting %d bytes from %d in cache/%s"
51970 4c885d5ff132 rev-branch-cache: stop pretending we will overwrite data when we don't Pierre-Yves David <pierre-yves.david@octobus.net> parents: 51969 diff changeset	429 dbg %= (current_size - start, start, _rbcrevs)
51901 c564be351754 rev-branch-cache: stop truncating cache file Pierre-Yves David <pierre-yves.david@octobus.net> parents: 51900 diff changeset	430 if end < current_size:
c564be351754 rev-branch-cache: stop truncating cache file Pierre-Yves David <pierre-yves.david@octobus.net> parents: 51900 diff changeset	431 extra = b" leaving (%d trailing bytes)"
c564be351754 rev-branch-cache: stop truncating cache file Pierre-Yves David <pierre-yves.david@octobus.net> parents: 51900 diff changeset	432 extra %= current_size - end
c564be351754 rev-branch-cache: stop truncating cache file Pierre-Yves David <pierre-yves.david@octobus.net> parents: 51900 diff changeset	433 dbg += extra
c564be351754 rev-branch-cache: stop truncating cache file Pierre-Yves David <pierre-yves.david@octobus.net> parents: 51900 diff changeset	434 dbg += b'\n'
c564be351754 rev-branch-cache: stop truncating cache file Pierre-Yves David <pierre-yves.david@octobus.net> parents: 51900 diff changeset	435 repo.ui.debug(dbg)
c564be351754 rev-branch-cache: stop truncating cache file Pierre-Yves David <pierre-yves.david@octobus.net> parents: 51900 diff changeset	436 else:
51970 4c885d5ff132 rev-branch-cache: stop pretending we will overwrite data when we don't Pierre-Yves David <pierre-yves.david@octobus.net> parents: 51969 diff changeset	437 # extra untouched data at the end, lets warn about them
4c885d5ff132 rev-branch-cache: stop pretending we will overwrite data when we don't Pierre-Yves David <pierre-yves.david@octobus.net> parents: 51969 diff changeset	438 assert start == end # since don't write anything
4c885d5ff132 rev-branch-cache: stop pretending we will overwrite data when we don't Pierre-Yves David <pierre-yves.david@octobus.net> parents: 51969 diff changeset	439 dbg = b"cache/%s contains %d unknown trailing bytes\n"
4c885d5ff132 rev-branch-cache: stop pretending we will overwrite data when we don't Pierre-Yves David <pierre-yves.david@octobus.net> parents: 51969 diff changeset	440 dbg %= (_rbcrevs, current_size - start)
51901 c564be351754 rev-branch-cache: stop truncating cache file Pierre-Yves David <pierre-yves.david@octobus.net> parents: 51900 diff changeset	441 repo.ui.debug(dbg)
51970 4c885d5ff132 rev-branch-cache: stop pretending we will overwrite data when we don't Pierre-Yves David <pierre-yves.david@octobus.net> parents: 51969 diff changeset	442
51901 c564be351754 rev-branch-cache: stop truncating cache file Pierre-Yves David <pierre-yves.david@octobus.net> parents: 51900 diff changeset	443 if start > 0:
42184 09fd338522fa revbranchcache: factor logic to write names and revs in separate functions Pulkit Goyal <pulkit@yandex-team.ru> parents: 42138 diff changeset	444 f.seek(start)
51901 c564be351754 rev-branch-cache: stop truncating cache file Pierre-Yves David <pierre-yves.david@octobus.net> parents: 51900 diff changeset	445 f.write(self._rbcrevs.slice(start, end))
c564be351754 rev-branch-cache: stop truncating cache file Pierre-Yves David <pierre-yves.david@octobus.net> parents: 51900 diff changeset	446 else:
c564be351754 rev-branch-cache: stop truncating cache file Pierre-Yves David <pierre-yves.david@octobus.net> parents: 51900 diff changeset	447 f.close()
c564be351754 rev-branch-cache: stop truncating cache file Pierre-Yves David <pierre-yves.david@octobus.net> parents: 51900 diff changeset	448 with repo.cachevfs.open(
c564be351754 rev-branch-cache: stop truncating cache file Pierre-Yves David <pierre-yves.david@octobus.net> parents: 51900 diff changeset	449 _rbcrevs,
c564be351754 rev-branch-cache: stop truncating cache file Pierre-Yves David <pierre-yves.david@octobus.net> parents: 51900 diff changeset	450 b'wb',
c564be351754 rev-branch-cache: stop truncating cache file Pierre-Yves David <pierre-yves.david@octobus.net> parents: 51900 diff changeset	451 atomictemp=True,
c564be351754 rev-branch-cache: stop truncating cache file Pierre-Yves David <pierre-yves.david@octobus.net> parents: 51900 diff changeset	452 ) as rev_file:
c564be351754 rev-branch-cache: stop truncating cache file Pierre-Yves David <pierre-yves.david@octobus.net> parents: 51900 diff changeset	453 rev_file.write(self._rbcrevs.slice(start, end))
42184 09fd338522fa revbranchcache: factor logic to write names and revs in separate functions Pulkit Goyal <pulkit@yandex-team.ru> parents: 42138 diff changeset	454 self._rbcrevslen = revs
51899 9f7cf869e9f4 rev-branch-cache: add a way to force rewrite of the cache Pierre-Yves David <pierre-yves.david@octobus.net> parents: 51898 diff changeset	455 self._force_overwrite = False

Mercurial > public > mercurial-scm > hg

annotate mercurial/branching/rev_cache.py @ 52113:db1980a361cb