mercurial-scm/hg-stable: mercurial/branching/rev

annotate mercurial/branching/rev_cache.py @ 51945:16efed18ae4e

rev-branch-cache: schedule a write of the "v2" format if we read from "v1" The new file can be memorymapped, while the old one cannot. So there is value in having the v2 format around as soon a possible.

author	Pierre-Yves David <pierre-yves.david@octobus.net>
date	Wed, 25 Sep 2024 12:49:32 +0200
parents	bd7359c18d69
children	bb281ecf4036

rev	line source
51938 f0e07efc199f rev-branch-cache: move the code in a dedicated module Pierre-Yves David <pierre-yves.david@octobus.net> parents: 51901 diff changeset	1 # rev_cache.py - caching branch information per revision
18116 bcee63733aad branchmap: create a mercurial.branchmap module Pierre-Yves David <pierre-yves.david@ens-lyon.org> parents: diff changeset	2 #
bcee63733aad branchmap: create a mercurial.branchmap module Pierre-Yves David <pierre-yves.david@ens-lyon.org> parents: diff changeset	3 # This software may be used and distributed according to the terms of the
bcee63733aad branchmap: create a mercurial.branchmap module Pierre-Yves David <pierre-yves.david@ens-lyon.org> parents: diff changeset	4 # GNU General Public License version 2 or any later version.
51901 f4733654f144 typing: add `from __future__ import annotations` to most files Matt Harbison <matt_harbison@yahoo.com> parents: 51651 diff changeset	5 from __future__ import annotations
25918 47f36e050c2e branchmap: use absolute_import Gregory Szorc <gregory.szorc@gmail.com> parents: 25660 diff changeset	6
51942 c564be351754 rev-branch-cache: stop truncating cache file Pierre-Yves David <pierre-yves.david@octobus.net> parents: 51941 diff changeset	7 import os
25918 47f36e050c2e branchmap: use absolute_import Gregory Szorc <gregory.szorc@gmail.com> parents: 25660 diff changeset	8 import struct
47f36e050c2e branchmap: use absolute_import Gregory Szorc <gregory.szorc@gmail.com> parents: 25660 diff changeset	9
51938 f0e07efc199f rev-branch-cache: move the code in a dedicated module Pierre-Yves David <pierre-yves.david@octobus.net> parents: 51901 diff changeset	10 from ..node import (
25918 47f36e050c2e branchmap: use absolute_import Gregory Szorc <gregory.szorc@gmail.com> parents: 25660 diff changeset	11 nullrev,
47f36e050c2e branchmap: use absolute_import Gregory Szorc <gregory.szorc@gmail.com> parents: 25660 diff changeset	12 )
51302 9d3721552b6c pytype: import typing directly Pierre-Yves David <pierre-yves.david@octobus.net> parents: 49308 diff changeset	13
51938 f0e07efc199f rev-branch-cache: move the code in a dedicated module Pierre-Yves David <pierre-yves.david@octobus.net> parents: 51901 diff changeset	14 from .. import (
25918 47f36e050c2e branchmap: use absolute_import Gregory Szorc <gregory.szorc@gmail.com> parents: 25660 diff changeset	15 encoding,
26587 56b2bcea2529 error: get Abort from 'error' instead of 'util' Pierre-Yves David <pierre-yves.david@fb.com> parents: 26460 diff changeset	16 error,
30995 22fbca1d11ed mercurial: switch to util.timer for all interval timings Simon Farnsworth <simonfar@fb.com> parents: 29758 diff changeset	17 util,
25918 47f36e050c2e branchmap: use absolute_import Gregory Szorc <gregory.szorc@gmail.com> parents: 25660 diff changeset	18 )
51302 9d3721552b6c pytype: import typing directly Pierre-Yves David <pierre-yves.david@octobus.net> parents: 49308 diff changeset	19
51938 f0e07efc199f rev-branch-cache: move the code in a dedicated module Pierre-Yves David <pierre-yves.david@octobus.net> parents: 51901 diff changeset	20 from ..utils import (
37087 f0b6fbea00cf stringutil: bulk-replace call sites to point to new module Yuya Nishihara <yuya@tcha.org> parents: 36968 diff changeset	21 stringutil,
f0b6fbea00cf stringutil: bulk-replace call sites to point to new module Yuya Nishihara <yuya@tcha.org> parents: 36968 diff changeset	22 )
25918 47f36e050c2e branchmap: use absolute_import Gregory Szorc <gregory.szorc@gmail.com> parents: 25660 diff changeset	23
47f36e050c2e branchmap: use absolute_import Gregory Szorc <gregory.szorc@gmail.com> parents: 25660 diff changeset	24 calcsize = struct.calcsize
31379 906be86990c4 rbc: use struct unpack_from and pack_into instead of unpack and pack Mads Kiilerich <madski@unity3d.com> parents: 31369 diff changeset	25 pack_into = struct.pack_into
906be86990c4 rbc: use struct unpack_from and pack_into instead of unpack and pack Mads Kiilerich <madski@unity3d.com> parents: 31369 diff changeset	26 unpack_from = struct.unpack_from
18117 526e7ec5c96e branchmap: extract write logic from localrepo Pierre-Yves David <pierre-yves.david@ens-lyon.org> parents: 18116 diff changeset	27
18118 e70ff1e599f4 branchmap: extract read logic from repo Pierre-Yves David <pierre-yves.david@logilab.fr> parents: 18117 diff changeset	28
23785 cb99bacb9b4e branchcache: introduce revbranchcache for caching of revision branch names Mads Kiilerich <madski@unity3d.com> parents: 22357 diff changeset	29 # Revision branch info cache
cb99bacb9b4e branchcache: introduce revbranchcache for caching of revision branch names Mads Kiilerich <madski@unity3d.com> parents: 22357 diff changeset	30
51943 0f26ee69cf36 rev-branch-cache: increment the version to "v2" Pierre-Yves David <pierre-yves.david@octobus.net> parents: 51942 diff changeset	31 # The "V2" version use the same format as the "V1" but garantee it won't be
0f26ee69cf36 rev-branch-cache: increment the version to "v2" Pierre-Yves David <pierre-yves.david@octobus.net> parents: 51942 diff changeset	32 # truncated, preventing SIGBUS when it is mmap-ed
0f26ee69cf36 rev-branch-cache: increment the version to "v2" Pierre-Yves David <pierre-yves.david@octobus.net> parents: 51942 diff changeset	33 _rbcversion = b'-v2'
43077 687b865b95ad formatting: byteify all mercurial/ and hgext/ string literals Augie Fackler <augie@google.com> parents: 43076 diff changeset	34 _rbcnames = b'rbc-names' + _rbcversion
687b865b95ad formatting: byteify all mercurial/ and hgext/ string literals Augie Fackler <augie@google.com> parents: 43076 diff changeset	35 _rbcrevs = b'rbc-revs' + _rbcversion
51944 bd7359c18d69 rev-branch-cache: fallback on "v1" data if no v2 is found Pierre-Yves David <pierre-yves.david@octobus.net> parents: 51943 diff changeset	36 _rbc_legacy_version = b'-v1'
bd7359c18d69 rev-branch-cache: fallback on "v1" data if no v2 is found Pierre-Yves David <pierre-yves.david@octobus.net> parents: 51943 diff changeset	37 _rbc_legacy_names = b'rbc-names' + _rbc_legacy_version
bd7359c18d69 rev-branch-cache: fallback on "v1" data if no v2 is found Pierre-Yves David <pierre-yves.david@octobus.net> parents: 51943 diff changeset	38 _rbc_legacy_revs = b'rbc-revs' + _rbc_legacy_version
23785 cb99bacb9b4e branchcache: introduce revbranchcache for caching of revision branch names Mads Kiilerich <madski@unity3d.com> parents: 22357 diff changeset	39 # [4 byte hash prefix][4 byte branch name number with sign bit indicating open]
43077 687b865b95ad formatting: byteify all mercurial/ and hgext/ string literals Augie Fackler <augie@google.com> parents: 43076 diff changeset	40 _rbcrecfmt = b'>4sI'
23785 cb99bacb9b4e branchcache: introduce revbranchcache for caching of revision branch names Mads Kiilerich <madski@unity3d.com> parents: 22357 diff changeset	41 _rbcrecsize = calcsize(_rbcrecfmt)
46434 1726a53a8494 reverse-branch-cache: switch to doubling allocating scheme Joerg Sonnenberger <joerg@bec.de> parents: 46254 diff changeset	42 _rbcmininc = 64 * _rbcrecsize
23785 cb99bacb9b4e branchcache: introduce revbranchcache for caching of revision branch names Mads Kiilerich <madski@unity3d.com> parents: 22357 diff changeset	43 _rbcnodelen = 4
43076 2372284d9457 formatting: blacken the codebase Augie Fackler <augie@google.com> parents: 42603 diff changeset	44 _rbcbranchidxmask = 0x7FFFFFFF
23785 cb99bacb9b4e branchcache: introduce revbranchcache for caching of revision branch names Mads Kiilerich <madski@unity3d.com> parents: 22357 diff changeset	45 _rbccloseflag = 0x80000000
cb99bacb9b4e branchcache: introduce revbranchcache for caching of revision branch names Mads Kiilerich <madski@unity3d.com> parents: 22357 diff changeset	46
43076 2372284d9457 formatting: blacken the codebase Augie Fackler <augie@google.com> parents: 42603 diff changeset	47
51942 c564be351754 rev-branch-cache: stop truncating cache file Pierre-Yves David <pierre-yves.david@octobus.net> parents: 51941 diff changeset	48 # with atomic replacement.
c564be351754 rev-branch-cache: stop truncating cache file Pierre-Yves David <pierre-yves.david@octobus.net> parents: 51941 diff changeset	49 REWRITE_RATIO = 0.2
c564be351754 rev-branch-cache: stop truncating cache file Pierre-Yves David <pierre-yves.david@octobus.net> parents: 51941 diff changeset	50
c564be351754 rev-branch-cache: stop truncating cache file Pierre-Yves David <pierre-yves.david@octobus.net> parents: 51941 diff changeset	51
51377 02e7d79edf62 branchmap: use mmap for faster revbranchcache loading Arseniy Alekseyev <aalekseyev@janestreet.com> parents: 51307 diff changeset	52 class rbcrevs:
02e7d79edf62 branchmap: use mmap for faster revbranchcache loading Arseniy Alekseyev <aalekseyev@janestreet.com> parents: 51307 diff changeset	53 """a byte string consisting of an immutable prefix followed by a mutable suffix"""
02e7d79edf62 branchmap: use mmap for faster revbranchcache loading Arseniy Alekseyev <aalekseyev@janestreet.com> parents: 51307 diff changeset	54
02e7d79edf62 branchmap: use mmap for faster revbranchcache loading Arseniy Alekseyev <aalekseyev@janestreet.com> parents: 51307 diff changeset	55 def __init__(self, revs):
02e7d79edf62 branchmap: use mmap for faster revbranchcache loading Arseniy Alekseyev <aalekseyev@janestreet.com> parents: 51307 diff changeset	56 self._prefix = revs
02e7d79edf62 branchmap: use mmap for faster revbranchcache loading Arseniy Alekseyev <aalekseyev@janestreet.com> parents: 51307 diff changeset	57 self._rest = bytearray()
02e7d79edf62 branchmap: use mmap for faster revbranchcache loading Arseniy Alekseyev <aalekseyev@janestreet.com> parents: 51307 diff changeset	58
02e7d79edf62 branchmap: use mmap for faster revbranchcache loading Arseniy Alekseyev <aalekseyev@janestreet.com> parents: 51307 diff changeset	59 def __len__(self):
02e7d79edf62 branchmap: use mmap for faster revbranchcache loading Arseniy Alekseyev <aalekseyev@janestreet.com> parents: 51307 diff changeset	60 return len(self._prefix) + len(self._rest)
02e7d79edf62 branchmap: use mmap for faster revbranchcache loading Arseniy Alekseyev <aalekseyev@janestreet.com> parents: 51307 diff changeset	61
02e7d79edf62 branchmap: use mmap for faster revbranchcache loading Arseniy Alekseyev <aalekseyev@janestreet.com> parents: 51307 diff changeset	62 def unpack_record(self, rbcrevidx):
02e7d79edf62 branchmap: use mmap for faster revbranchcache loading Arseniy Alekseyev <aalekseyev@janestreet.com> parents: 51307 diff changeset	63 if rbcrevidx < len(self._prefix):
02e7d79edf62 branchmap: use mmap for faster revbranchcache loading Arseniy Alekseyev <aalekseyev@janestreet.com> parents: 51307 diff changeset	64 return unpack_from(_rbcrecfmt, util.buffer(self._prefix), rbcrevidx)
02e7d79edf62 branchmap: use mmap for faster revbranchcache loading Arseniy Alekseyev <aalekseyev@janestreet.com> parents: 51307 diff changeset	65 else:
02e7d79edf62 branchmap: use mmap for faster revbranchcache loading Arseniy Alekseyev <aalekseyev@janestreet.com> parents: 51307 diff changeset	66 return unpack_from(
02e7d79edf62 branchmap: use mmap for faster revbranchcache loading Arseniy Alekseyev <aalekseyev@janestreet.com> parents: 51307 diff changeset	67 _rbcrecfmt,
02e7d79edf62 branchmap: use mmap for faster revbranchcache loading Arseniy Alekseyev <aalekseyev@janestreet.com> parents: 51307 diff changeset	68 util.buffer(self._rest),
02e7d79edf62 branchmap: use mmap for faster revbranchcache loading Arseniy Alekseyev <aalekseyev@janestreet.com> parents: 51307 diff changeset	69 rbcrevidx - len(self._prefix),
02e7d79edf62 branchmap: use mmap for faster revbranchcache loading Arseniy Alekseyev <aalekseyev@janestreet.com> parents: 51307 diff changeset	70 )
02e7d79edf62 branchmap: use mmap for faster revbranchcache loading Arseniy Alekseyev <aalekseyev@janestreet.com> parents: 51307 diff changeset	71
02e7d79edf62 branchmap: use mmap for faster revbranchcache loading Arseniy Alekseyev <aalekseyev@janestreet.com> parents: 51307 diff changeset	72 def make_mutable(self):
02e7d79edf62 branchmap: use mmap for faster revbranchcache loading Arseniy Alekseyev <aalekseyev@janestreet.com> parents: 51307 diff changeset	73 if len(self._prefix) > 0:
02e7d79edf62 branchmap: use mmap for faster revbranchcache loading Arseniy Alekseyev <aalekseyev@janestreet.com> parents: 51307 diff changeset	74 entirety = bytearray()
02e7d79edf62 branchmap: use mmap for faster revbranchcache loading Arseniy Alekseyev <aalekseyev@janestreet.com> parents: 51307 diff changeset	75 entirety[:] = self._prefix
02e7d79edf62 branchmap: use mmap for faster revbranchcache loading Arseniy Alekseyev <aalekseyev@janestreet.com> parents: 51307 diff changeset	76 entirety.extend(self._rest)
02e7d79edf62 branchmap: use mmap for faster revbranchcache loading Arseniy Alekseyev <aalekseyev@janestreet.com> parents: 51307 diff changeset	77 self._rest = entirety
02e7d79edf62 branchmap: use mmap for faster revbranchcache loading Arseniy Alekseyev <aalekseyev@janestreet.com> parents: 51307 diff changeset	78 self._prefix = bytearray()
02e7d79edf62 branchmap: use mmap for faster revbranchcache loading Arseniy Alekseyev <aalekseyev@janestreet.com> parents: 51307 diff changeset	79
02e7d79edf62 branchmap: use mmap for faster revbranchcache loading Arseniy Alekseyev <aalekseyev@janestreet.com> parents: 51307 diff changeset	80 def truncate(self, pos):
02e7d79edf62 branchmap: use mmap for faster revbranchcache loading Arseniy Alekseyev <aalekseyev@janestreet.com> parents: 51307 diff changeset	81 self.make_mutable()
02e7d79edf62 branchmap: use mmap for faster revbranchcache loading Arseniy Alekseyev <aalekseyev@janestreet.com> parents: 51307 diff changeset	82 del self._rest[pos:]
02e7d79edf62 branchmap: use mmap for faster revbranchcache loading Arseniy Alekseyev <aalekseyev@janestreet.com> parents: 51307 diff changeset	83
02e7d79edf62 branchmap: use mmap for faster revbranchcache loading Arseniy Alekseyev <aalekseyev@janestreet.com> parents: 51307 diff changeset	84 def pack_into(self, rbcrevidx, node, branchidx):
02e7d79edf62 branchmap: use mmap for faster revbranchcache loading Arseniy Alekseyev <aalekseyev@janestreet.com> parents: 51307 diff changeset	85 if rbcrevidx < len(self._prefix):
02e7d79edf62 branchmap: use mmap for faster revbranchcache loading Arseniy Alekseyev <aalekseyev@janestreet.com> parents: 51307 diff changeset	86 self.make_mutable()
02e7d79edf62 branchmap: use mmap for faster revbranchcache loading Arseniy Alekseyev <aalekseyev@janestreet.com> parents: 51307 diff changeset	87 buf = self._rest
02e7d79edf62 branchmap: use mmap for faster revbranchcache loading Arseniy Alekseyev <aalekseyev@janestreet.com> parents: 51307 diff changeset	88 start_offset = rbcrevidx - len(self._prefix)
02e7d79edf62 branchmap: use mmap for faster revbranchcache loading Arseniy Alekseyev <aalekseyev@janestreet.com> parents: 51307 diff changeset	89 end_offset = start_offset + _rbcrecsize
02e7d79edf62 branchmap: use mmap for faster revbranchcache loading Arseniy Alekseyev <aalekseyev@janestreet.com> parents: 51307 diff changeset	90
02e7d79edf62 branchmap: use mmap for faster revbranchcache loading Arseniy Alekseyev <aalekseyev@janestreet.com> parents: 51307 diff changeset	91 if len(self._rest) < end_offset:
02e7d79edf62 branchmap: use mmap for faster revbranchcache loading Arseniy Alekseyev <aalekseyev@janestreet.com> parents: 51307 diff changeset	92 # bytearray doesn't allocate extra space at least in Python 3.7.
02e7d79edf62 branchmap: use mmap for faster revbranchcache loading Arseniy Alekseyev <aalekseyev@janestreet.com> parents: 51307 diff changeset	93 # When multiple changesets are added in a row, precise resize would
02e7d79edf62 branchmap: use mmap for faster revbranchcache loading Arseniy Alekseyev <aalekseyev@janestreet.com> parents: 51307 diff changeset	94 # result in quadratic complexity. Overallocate to compensate by
02e7d79edf62 branchmap: use mmap for faster revbranchcache loading Arseniy Alekseyev <aalekseyev@janestreet.com> parents: 51307 diff changeset	95 # using the classic doubling technique for dynamic arrays instead.
02e7d79edf62 branchmap: use mmap for faster revbranchcache loading Arseniy Alekseyev <aalekseyev@janestreet.com> parents: 51307 diff changeset	96 # If there was a gap in the map before, less space will be reserved.
02e7d79edf62 branchmap: use mmap for faster revbranchcache loading Arseniy Alekseyev <aalekseyev@janestreet.com> parents: 51307 diff changeset	97 self._rest.extend(b'\0' * end_offset)
02e7d79edf62 branchmap: use mmap for faster revbranchcache loading Arseniy Alekseyev <aalekseyev@janestreet.com> parents: 51307 diff changeset	98 return pack_into(
02e7d79edf62 branchmap: use mmap for faster revbranchcache loading Arseniy Alekseyev <aalekseyev@janestreet.com> parents: 51307 diff changeset	99 _rbcrecfmt,
02e7d79edf62 branchmap: use mmap for faster revbranchcache loading Arseniy Alekseyev <aalekseyev@janestreet.com> parents: 51307 diff changeset	100 buf,
02e7d79edf62 branchmap: use mmap for faster revbranchcache loading Arseniy Alekseyev <aalekseyev@janestreet.com> parents: 51307 diff changeset	101 start_offset,
02e7d79edf62 branchmap: use mmap for faster revbranchcache loading Arseniy Alekseyev <aalekseyev@janestreet.com> parents: 51307 diff changeset	102 node,
02e7d79edf62 branchmap: use mmap for faster revbranchcache loading Arseniy Alekseyev <aalekseyev@janestreet.com> parents: 51307 diff changeset	103 branchidx,
02e7d79edf62 branchmap: use mmap for faster revbranchcache loading Arseniy Alekseyev <aalekseyev@janestreet.com> parents: 51307 diff changeset	104 )
02e7d79edf62 branchmap: use mmap for faster revbranchcache loading Arseniy Alekseyev <aalekseyev@janestreet.com> parents: 51307 diff changeset	105
02e7d79edf62 branchmap: use mmap for faster revbranchcache loading Arseniy Alekseyev <aalekseyev@janestreet.com> parents: 51307 diff changeset	106 def extend(self, extension):
02e7d79edf62 branchmap: use mmap for faster revbranchcache loading Arseniy Alekseyev <aalekseyev@janestreet.com> parents: 51307 diff changeset	107 return self._rest.extend(extension)
02e7d79edf62 branchmap: use mmap for faster revbranchcache loading Arseniy Alekseyev <aalekseyev@janestreet.com> parents: 51307 diff changeset	108
02e7d79edf62 branchmap: use mmap for faster revbranchcache loading Arseniy Alekseyev <aalekseyev@janestreet.com> parents: 51307 diff changeset	109 def slice(self, begin, end):
02e7d79edf62 branchmap: use mmap for faster revbranchcache loading Arseniy Alekseyev <aalekseyev@janestreet.com> parents: 51307 diff changeset	110 if begin < len(self._prefix):
02e7d79edf62 branchmap: use mmap for faster revbranchcache loading Arseniy Alekseyev <aalekseyev@janestreet.com> parents: 51307 diff changeset	111 acc = bytearray()
02e7d79edf62 branchmap: use mmap for faster revbranchcache loading Arseniy Alekseyev <aalekseyev@janestreet.com> parents: 51307 diff changeset	112 acc[:] = self._prefix[begin:end]
02e7d79edf62 branchmap: use mmap for faster revbranchcache loading Arseniy Alekseyev <aalekseyev@janestreet.com> parents: 51307 diff changeset	113 acc.extend(
02e7d79edf62 branchmap: use mmap for faster revbranchcache loading Arseniy Alekseyev <aalekseyev@janestreet.com> parents: 51307 diff changeset	114 self._rest[begin - len(self._prefix) : end - len(self._prefix)]
02e7d79edf62 branchmap: use mmap for faster revbranchcache loading Arseniy Alekseyev <aalekseyev@janestreet.com> parents: 51307 diff changeset	115 )
02e7d79edf62 branchmap: use mmap for faster revbranchcache loading Arseniy Alekseyev <aalekseyev@janestreet.com> parents: 51307 diff changeset	116 return acc
02e7d79edf62 branchmap: use mmap for faster revbranchcache loading Arseniy Alekseyev <aalekseyev@janestreet.com> parents: 51307 diff changeset	117 return self._rest[begin - len(self._prefix) : end - len(self._prefix)]
02e7d79edf62 branchmap: use mmap for faster revbranchcache loading Arseniy Alekseyev <aalekseyev@janestreet.com> parents: 51307 diff changeset	118
02e7d79edf62 branchmap: use mmap for faster revbranchcache loading Arseniy Alekseyev <aalekseyev@janestreet.com> parents: 51307 diff changeset	119
49037 642e31cb55f0 py3: use class X: instead of class X(object): Gregory Szorc <gregory.szorc@gmail.com> parents: 49026 diff changeset	120 class revbranchcache:
23785 cb99bacb9b4e branchcache: introduce revbranchcache for caching of revision branch names Mads Kiilerich <madski@unity3d.com> parents: 22357 diff changeset	121 """Persistent cache, mapping from revision number to branch name and close.
cb99bacb9b4e branchcache: introduce revbranchcache for caching of revision branch names Mads Kiilerich <madski@unity3d.com> parents: 22357 diff changeset	122 This is a low level cache, independent of filtering.
cb99bacb9b4e branchcache: introduce revbranchcache for caching of revision branch names Mads Kiilerich <madski@unity3d.com> parents: 22357 diff changeset	123
cb99bacb9b4e branchcache: introduce revbranchcache for caching of revision branch names Mads Kiilerich <madski@unity3d.com> parents: 22357 diff changeset	124 Branch names are stored in rbc-names in internal encoding separated by 0.
cb99bacb9b4e branchcache: introduce revbranchcache for caching of revision branch names Mads Kiilerich <madski@unity3d.com> parents: 22357 diff changeset	125 rbc-names is append-only, and each branch name is only stored once and will
cb99bacb9b4e branchcache: introduce revbranchcache for caching of revision branch names Mads Kiilerich <madski@unity3d.com> parents: 22357 diff changeset	126 thus have a unique index.
cb99bacb9b4e branchcache: introduce revbranchcache for caching of revision branch names Mads Kiilerich <madski@unity3d.com> parents: 22357 diff changeset	127
cb99bacb9b4e branchcache: introduce revbranchcache for caching of revision branch names Mads Kiilerich <madski@unity3d.com> parents: 22357 diff changeset	128 The branch info for each revision is stored in rbc-revs as constant size
cb99bacb9b4e branchcache: introduce revbranchcache for caching of revision branch names Mads Kiilerich <madski@unity3d.com> parents: 22357 diff changeset	129 records. The whole file is read into memory, but it is only 'parsed' on
cb99bacb9b4e branchcache: introduce revbranchcache for caching of revision branch names Mads Kiilerich <madski@unity3d.com> parents: 22357 diff changeset	130 demand. The file is usually append-only but will be truncated if repo
cb99bacb9b4e branchcache: introduce revbranchcache for caching of revision branch names Mads Kiilerich <madski@unity3d.com> parents: 22357 diff changeset	131 modification is detected.
cb99bacb9b4e branchcache: introduce revbranchcache for caching of revision branch names Mads Kiilerich <madski@unity3d.com> parents: 22357 diff changeset	132 The record for each revision contains the first 4 bytes of the
cb99bacb9b4e branchcache: introduce revbranchcache for caching of revision branch names Mads Kiilerich <madski@unity3d.com> parents: 22357 diff changeset	133 corresponding node hash, and the record is only used if it still matches.
cb99bacb9b4e branchcache: introduce revbranchcache for caching of revision branch names Mads Kiilerich <madski@unity3d.com> parents: 22357 diff changeset	134 Even a completely trashed rbc-revs fill thus still give the right result
cb99bacb9b4e branchcache: introduce revbranchcache for caching of revision branch names Mads Kiilerich <madski@unity3d.com> parents: 22357 diff changeset	135 while converging towards full recovery ... assuming no incorrectly matching
cb99bacb9b4e branchcache: introduce revbranchcache for caching of revision branch names Mads Kiilerich <madski@unity3d.com> parents: 22357 diff changeset	136 node hashes.
cb99bacb9b4e branchcache: introduce revbranchcache for caching of revision branch names Mads Kiilerich <madski@unity3d.com> parents: 22357 diff changeset	137 The record also contains 4 bytes where 31 bits contains the index of the
cb99bacb9b4e branchcache: introduce revbranchcache for caching of revision branch names Mads Kiilerich <madski@unity3d.com> parents: 22357 diff changeset	138 branch and the last bit indicate that it is a branch close commit.
cb99bacb9b4e branchcache: introduce revbranchcache for caching of revision branch names Mads Kiilerich <madski@unity3d.com> parents: 22357 diff changeset	139 The usage pattern for rbc-revs is thus somewhat similar to 00changelog.i
cb99bacb9b4e branchcache: introduce revbranchcache for caching of revision branch names Mads Kiilerich <madski@unity3d.com> parents: 22357 diff changeset	140 and will grow with it but be 1/8th of its size.
cb99bacb9b4e branchcache: introduce revbranchcache for caching of revision branch names Mads Kiilerich <madski@unity3d.com> parents: 22357 diff changeset	141 """
cb99bacb9b4e branchcache: introduce revbranchcache for caching of revision branch names Mads Kiilerich <madski@unity3d.com> parents: 22357 diff changeset	142
24159 5b4ed033390b revisionbranchcache: fall back to slow path if starting readonly (issue4531) Mads Kiilerich <madski@unity3d.com> parents: 23877 diff changeset	143 def __init__(self, repo, readonly=True):
23785 cb99bacb9b4e branchcache: introduce revbranchcache for caching of revision branch names Mads Kiilerich <madski@unity3d.com> parents: 22357 diff changeset	144 assert repo.filtername is None
24374 77fd1fb538cd revbranchcache: store repo on the object Durham Goode <durham@fb.com> parents: 24373 diff changeset	145 self._repo = repo
43076 2372284d9457 formatting: blacken the codebase Augie Fackler <augie@google.com> parents: 42603 diff changeset	146 self._names = [] # branch names in local encoding with static index
51377 02e7d79edf62 branchmap: use mmap for faster revbranchcache loading Arseniy Alekseyev <aalekseyev@janestreet.com> parents: 51307 diff changeset	147 self._rbcrevs = rbcrevs(bytearray())
43076 2372284d9457 formatting: blacken the codebase Augie Fackler <augie@google.com> parents: 42603 diff changeset	148 self._rbcsnameslen = 0 # length of names read at _rbcsnameslen
51945 16efed18ae4e rev-branch-cache: schedule a write of the "v2" format if we read from "v1" Pierre-Yves David <pierre-yves.david@octobus.net> parents: 51944 diff changeset	149 self._force_overwrite = False
51944 bd7359c18d69 rev-branch-cache: fallback on "v1" data if no v2 is found Pierre-Yves David <pierre-yves.david@octobus.net> parents: 51943 diff changeset	150 v1_fallback = False
23785 cb99bacb9b4e branchcache: introduce revbranchcache for caching of revision branch names Mads Kiilerich <madski@unity3d.com> parents: 22357 diff changeset	151 try:
51944 bd7359c18d69 rev-branch-cache: fallback on "v1" data if no v2 is found Pierre-Yves David <pierre-yves.david@octobus.net> parents: 51943 diff changeset	152 try:
bd7359c18d69 rev-branch-cache: fallback on "v1" data if no v2 is found Pierre-Yves David <pierre-yves.david@octobus.net> parents: 51943 diff changeset	153 bndata = repo.cachevfs.read(_rbcnames)
bd7359c18d69 rev-branch-cache: fallback on "v1" data if no v2 is found Pierre-Yves David <pierre-yves.david@octobus.net> parents: 51943 diff changeset	154 except (IOError, OSError):
bd7359c18d69 rev-branch-cache: fallback on "v1" data if no v2 is found Pierre-Yves David <pierre-yves.david@octobus.net> parents: 51943 diff changeset	155 # If we don't have "v2" data, we might have "v1" data worth
bd7359c18d69 rev-branch-cache: fallback on "v1" data if no v2 is found Pierre-Yves David <pierre-yves.david@octobus.net> parents: 51943 diff changeset	156 # using.
bd7359c18d69 rev-branch-cache: fallback on "v1" data if no v2 is found Pierre-Yves David <pierre-yves.david@octobus.net> parents: 51943 diff changeset	157 #
bd7359c18d69 rev-branch-cache: fallback on "v1" data if no v2 is found Pierre-Yves David <pierre-yves.david@octobus.net> parents: 51943 diff changeset	158 # consider stop doing this many version after hg-6.9 release
bd7359c18d69 rev-branch-cache: fallback on "v1" data if no v2 is found Pierre-Yves David <pierre-yves.david@octobus.net> parents: 51943 diff changeset	159 bndata = repo.cachevfs.read(_rbc_legacy_names)
bd7359c18d69 rev-branch-cache: fallback on "v1" data if no v2 is found Pierre-Yves David <pierre-yves.david@octobus.net> parents: 51943 diff changeset	160 v1_fallback = True
51945 16efed18ae4e rev-branch-cache: schedule a write of the "v2" format if we read from "v1" Pierre-Yves David <pierre-yves.david@octobus.net> parents: 51944 diff changeset	161 self._force_overwrite = True
43076 2372284d9457 formatting: blacken the codebase Augie Fackler <augie@google.com> parents: 42603 diff changeset	162 self._rbcsnameslen = len(bndata) # for verification before writing
31380 7dd2f51f38ac rbc: empty (and invalid) rbc-names file should give an empty name list Mads Kiilerich <mads@kiilerich.com> parents: 31379 diff changeset	163 if bndata:
43076 2372284d9457 formatting: blacken the codebase Augie Fackler <augie@google.com> parents: 42603 diff changeset	164 self._names = [
43077 687b865b95ad formatting: byteify all mercurial/ and hgext/ string literals Augie Fackler <augie@google.com> parents: 43076 diff changeset	165 encoding.tolocal(bn) for bn in bndata.split(b'\0')
43076 2372284d9457 formatting: blacken the codebase Augie Fackler <augie@google.com> parents: 42603 diff changeset	166 ]
29423 d2c6f3a948fa branchmap: remove unused exception variable Gregory Szorc <gregory.szorc@gmail.com> parents: 28558 diff changeset	167 except (IOError, OSError):
24159 5b4ed033390b revisionbranchcache: fall back to slow path if starting readonly (issue4531) Mads Kiilerich <madski@unity3d.com> parents: 23877 diff changeset	168 if readonly:
5b4ed033390b revisionbranchcache: fall back to slow path if starting readonly (issue4531) Mads Kiilerich <madski@unity3d.com> parents: 23877 diff changeset	169 # don't try to use cache - fall back to the slow path
5b4ed033390b revisionbranchcache: fall back to slow path if starting readonly (issue4531) Mads Kiilerich <madski@unity3d.com> parents: 23877 diff changeset	170 self.branchinfo = self._branchinfo
5b4ed033390b revisionbranchcache: fall back to slow path if starting readonly (issue4531) Mads Kiilerich <madski@unity3d.com> parents: 23877 diff changeset	171
23785 cb99bacb9b4e branchcache: introduce revbranchcache for caching of revision branch names Mads Kiilerich <madski@unity3d.com> parents: 22357 diff changeset	172 if self._names:
cb99bacb9b4e branchcache: introduce revbranchcache for caching of revision branch names Mads Kiilerich <madski@unity3d.com> parents: 22357 diff changeset	173 try:
51651 c0e30a019ce1 mmap: only use mmap to read rev-branch-cache data if it is safe Pierre-Yves David <pierre-yves.david@octobus.net> parents: 51644 diff changeset	174 usemmap = repo.ui.configbool(b'storage', b'revbranchcache.mmap')
51944 bd7359c18d69 rev-branch-cache: fallback on "v1" data if no v2 is found Pierre-Yves David <pierre-yves.david@octobus.net> parents: 51943 diff changeset	175 if not v1_fallback:
bd7359c18d69 rev-branch-cache: fallback on "v1" data if no v2 is found Pierre-Yves David <pierre-yves.david@octobus.net> parents: 51943 diff changeset	176 with repo.cachevfs(_rbcrevs) as fp:
bd7359c18d69 rev-branch-cache: fallback on "v1" data if no v2 is found Pierre-Yves David <pierre-yves.david@octobus.net> parents: 51943 diff changeset	177 if usemmap and repo.cachevfs.is_mmap_safe(_rbcrevs):
bd7359c18d69 rev-branch-cache: fallback on "v1" data if no v2 is found Pierre-Yves David <pierre-yves.david@octobus.net> parents: 51943 diff changeset	178 data = util.buffer(util.mmapread(fp))
bd7359c18d69 rev-branch-cache: fallback on "v1" data if no v2 is found Pierre-Yves David <pierre-yves.david@octobus.net> parents: 51943 diff changeset	179 else:
bd7359c18d69 rev-branch-cache: fallback on "v1" data if no v2 is found Pierre-Yves David <pierre-yves.david@octobus.net> parents: 51943 diff changeset	180 data = fp.read()
bd7359c18d69 rev-branch-cache: fallback on "v1" data if no v2 is found Pierre-Yves David <pierre-yves.david@octobus.net> parents: 51943 diff changeset	181 else:
bd7359c18d69 rev-branch-cache: fallback on "v1" data if no v2 is found Pierre-Yves David <pierre-yves.david@octobus.net> parents: 51943 diff changeset	182 # If we don't have "v2" data, we might have "v1" data worth
bd7359c18d69 rev-branch-cache: fallback on "v1" data if no v2 is found Pierre-Yves David <pierre-yves.david@octobus.net> parents: 51943 diff changeset	183 # using.
bd7359c18d69 rev-branch-cache: fallback on "v1" data if no v2 is found Pierre-Yves David <pierre-yves.david@octobus.net> parents: 51943 diff changeset	184 #
bd7359c18d69 rev-branch-cache: fallback on "v1" data if no v2 is found Pierre-Yves David <pierre-yves.david@octobus.net> parents: 51943 diff changeset	185 # Consider stop doing this many version after hg-6.9
bd7359c18d69 rev-branch-cache: fallback on "v1" data if no v2 is found Pierre-Yves David <pierre-yves.david@octobus.net> parents: 51943 diff changeset	186 # release.
bd7359c18d69 rev-branch-cache: fallback on "v1" data if no v2 is found Pierre-Yves David <pierre-yves.david@octobus.net> parents: 51943 diff changeset	187 with repo.cachevfs(_rbc_legacy_revs) as fp:
51651 c0e30a019ce1 mmap: only use mmap to read rev-branch-cache data if it is safe Pierre-Yves David <pierre-yves.david@octobus.net> parents: 51644 diff changeset	188 data = fp.read()
51377 02e7d79edf62 branchmap: use mmap for faster revbranchcache loading Arseniy Alekseyev <aalekseyev@janestreet.com> parents: 51307 diff changeset	189 self._rbcrevs = rbcrevs(data)
25660 328739ea70c3 global: mass rewrite to use modern exception syntax Gregory Szorc <gregory.szorc@gmail.com> parents: 25266 diff changeset	190 except (IOError, OSError) as inst:
43076 2372284d9457 formatting: blacken the codebase Augie Fackler <augie@google.com> parents: 42603 diff changeset	191 repo.ui.debug(
43077 687b865b95ad formatting: byteify all mercurial/ and hgext/ string literals Augie Fackler <augie@google.com> parents: 43076 diff changeset	192 b"couldn't read revision branch cache: %s\n"
43076 2372284d9457 formatting: blacken the codebase Augie Fackler <augie@google.com> parents: 42603 diff changeset	193 % stringutil.forcebytestr(inst)
2372284d9457 formatting: blacken the codebase Augie Fackler <augie@google.com> parents: 42603 diff changeset	194 )
23785 cb99bacb9b4e branchcache: introduce revbranchcache for caching of revision branch names Mads Kiilerich <madski@unity3d.com> parents: 22357 diff changeset	195 # remember number of good records on disk
43076 2372284d9457 formatting: blacken the codebase Augie Fackler <augie@google.com> parents: 42603 diff changeset	196 self._rbcrevslen = min(
2372284d9457 formatting: blacken the codebase Augie Fackler <augie@google.com> parents: 42603 diff changeset	197 len(self._rbcrevs) // _rbcrecsize, len(repo.changelog)
2372284d9457 formatting: blacken the codebase Augie Fackler <augie@google.com> parents: 42603 diff changeset	198 )
23785 cb99bacb9b4e branchcache: introduce revbranchcache for caching of revision branch names Mads Kiilerich <madski@unity3d.com> parents: 22357 diff changeset	199 if self._rbcrevslen == 0:
cb99bacb9b4e branchcache: introduce revbranchcache for caching of revision branch names Mads Kiilerich <madski@unity3d.com> parents: 22357 diff changeset	200 self._names = []
43076 2372284d9457 formatting: blacken the codebase Augie Fackler <augie@google.com> parents: 42603 diff changeset	201 self._rbcnamescount = len(self._names) # number of names read at
2372284d9457 formatting: blacken the codebase Augie Fackler <augie@google.com> parents: 42603 diff changeset	202 # _rbcsnameslen
23785 cb99bacb9b4e branchcache: introduce revbranchcache for caching of revision branch names Mads Kiilerich <madski@unity3d.com> parents: 22357 diff changeset	203
28558 bcd106d456c4 cache: rebuild branch cache from scratch when inconsistencies are detected Mads Kiilerich <madski@unity3d.com> parents: 28557 diff changeset	204 def _clear(self):
bcd106d456c4 cache: rebuild branch cache from scratch when inconsistencies are detected Mads Kiilerich <madski@unity3d.com> parents: 28557 diff changeset	205 self._rbcsnameslen = 0
bcd106d456c4 cache: rebuild branch cache from scratch when inconsistencies are detected Mads Kiilerich <madski@unity3d.com> parents: 28557 diff changeset	206 del self._names[:]
bcd106d456c4 cache: rebuild branch cache from scratch when inconsistencies are detected Mads Kiilerich <madski@unity3d.com> parents: 28557 diff changeset	207 self._rbcnamescount = 0
bcd106d456c4 cache: rebuild branch cache from scratch when inconsistencies are detected Mads Kiilerich <madski@unity3d.com> parents: 28557 diff changeset	208 self._rbcrevslen = len(self._repo.changelog)
51377 02e7d79edf62 branchmap: use mmap for faster revbranchcache loading Arseniy Alekseyev <aalekseyev@janestreet.com> parents: 51307 diff changeset	209 self._rbcrevs = rbcrevs(bytearray(self._rbcrevslen * _rbcrecsize))
43077 687b865b95ad formatting: byteify all mercurial/ and hgext/ string literals Augie Fackler <augie@google.com> parents: 43076 diff changeset	210 util.clearcachedproperty(self, b'_namesreverse')
51940 9f7cf869e9f4 rev-branch-cache: add a way to force rewrite of the cache Pierre-Yves David <pierre-yves.david@octobus.net> parents: 51939 diff changeset	211 self._force_overwrite = True
9f7cf869e9f4 rev-branch-cache: add a way to force rewrite of the cache Pierre-Yves David <pierre-yves.david@octobus.net> parents: 51939 diff changeset	212
9f7cf869e9f4 rev-branch-cache: add a way to force rewrite of the cache Pierre-Yves David <pierre-yves.david@octobus.net> parents: 51939 diff changeset	213 def invalidate(self, rev=0):
9f7cf869e9f4 rev-branch-cache: add a way to force rewrite of the cache Pierre-Yves David <pierre-yves.david@octobus.net> parents: 51939 diff changeset	214 self._rbcrevslen = rev
9f7cf869e9f4 rev-branch-cache: add a way to force rewrite of the cache Pierre-Yves David <pierre-yves.david@octobus.net> parents: 51939 diff changeset	215 self._rbcrevs.truncate(rev)
9f7cf869e9f4 rev-branch-cache: add a way to force rewrite of the cache Pierre-Yves David <pierre-yves.david@octobus.net> parents: 51939 diff changeset	216 self._force_overwrite = True
40749 50a64c321c1e branchmap: build the revbranchcache._namesreverse() only when required Pulkit Goyal <pulkit@yandex-team.ru> parents: 40494 diff changeset	217
50a64c321c1e branchmap: build the revbranchcache._namesreverse() only when required Pulkit Goyal <pulkit@yandex-team.ru> parents: 40494 diff changeset	218 @util.propertycache
50a64c321c1e branchmap: build the revbranchcache._namesreverse() only when required Pulkit Goyal <pulkit@yandex-team.ru> parents: 40494 diff changeset	219 def _namesreverse(self):
44470 9d2b2df2c2ba cleanup: run pyupgrade on our source tree to clean up varying things Augie Fackler <augie@google.com> parents: 44349 diff changeset	220 return {b: r for r, b in enumerate(self._names)}
28558 bcd106d456c4 cache: rebuild branch cache from scratch when inconsistencies are detected Mads Kiilerich <madski@unity3d.com> parents: 28557 diff changeset	221
40494 5e5c8f2a1eb5 branchmap: do not specify changelog as an argument Yuya Nishihara <yuya@tcha.org> parents: 40375 diff changeset	222 def branchinfo(self, rev):
23785 cb99bacb9b4e branchcache: introduce revbranchcache for caching of revision branch names Mads Kiilerich <madski@unity3d.com> parents: 22357 diff changeset	223 """Return branch name and close flag for rev, using and updating
cb99bacb9b4e branchcache: introduce revbranchcache for caching of revision branch names Mads Kiilerich <madski@unity3d.com> parents: 22357 diff changeset	224 persistent cache."""
40494 5e5c8f2a1eb5 branchmap: do not specify changelog as an argument Yuya Nishihara <yuya@tcha.org> parents: 40375 diff changeset	225 changelog = self._repo.changelog
23785 cb99bacb9b4e branchcache: introduce revbranchcache for caching of revision branch names Mads Kiilerich <madski@unity3d.com> parents: 22357 diff changeset	226 rbcrevidx = rev * _rbcrecsize
cb99bacb9b4e branchcache: introduce revbranchcache for caching of revision branch names Mads Kiilerich <madski@unity3d.com> parents: 22357 diff changeset	227
25266 38117278f295 revbranchcache: return uncached branchinfo for nullrev (issue4683) Yuya Nishihara <yuya@tcha.org> parents: 24728 diff changeset	228 # avoid negative index, changelog.read(nullrev) is fast without cache
38117278f295 revbranchcache: return uncached branchinfo for nullrev (issue4683) Yuya Nishihara <yuya@tcha.org> parents: 24728 diff changeset	229 if rev == nullrev:
38117278f295 revbranchcache: return uncached branchinfo for nullrev (issue4683) Yuya Nishihara <yuya@tcha.org> parents: 24728 diff changeset	230 return changelog.branchinfo(rev)
38117278f295 revbranchcache: return uncached branchinfo for nullrev (issue4683) Yuya Nishihara <yuya@tcha.org> parents: 24728 diff changeset	231
29604 db0095c83344 rbc: fix invalid rbc-revs entries caused by missing cache growth Mads Kiilerich <madski@unity3d.com> parents: 29423 diff changeset	232 # if requested rev isn't allocated, grow and cache the rev info
23785 cb99bacb9b4e branchcache: introduce revbranchcache for caching of revision branch names Mads Kiilerich <madski@unity3d.com> parents: 22357 diff changeset	233 if len(self._rbcrevs) < rbcrevidx + _rbcrecsize:
40494 5e5c8f2a1eb5 branchmap: do not specify changelog as an argument Yuya Nishihara <yuya@tcha.org> parents: 40375 diff changeset	234 return self._branchinfo(rev)
23785 cb99bacb9b4e branchcache: introduce revbranchcache for caching of revision branch names Mads Kiilerich <madski@unity3d.com> parents: 22357 diff changeset	235
cb99bacb9b4e branchcache: introduce revbranchcache for caching of revision branch names Mads Kiilerich <madski@unity3d.com> parents: 22357 diff changeset	236 # fast path: extract data from cache, use it if node is matching
cb99bacb9b4e branchcache: introduce revbranchcache for caching of revision branch names Mads Kiilerich <madski@unity3d.com> parents: 22357 diff changeset	237 reponode = changelog.node(rev)[:_rbcnodelen]
51377 02e7d79edf62 branchmap: use mmap for faster revbranchcache loading Arseniy Alekseyev <aalekseyev@janestreet.com> parents: 51307 diff changeset	238 cachenode, branchidx = self._rbcrevs.unpack_record(rbcrevidx)
23785 cb99bacb9b4e branchcache: introduce revbranchcache for caching of revision branch names Mads Kiilerich <madski@unity3d.com> parents: 22357 diff changeset	239 close = bool(branchidx & _rbccloseflag)
cb99bacb9b4e branchcache: introduce revbranchcache for caching of revision branch names Mads Kiilerich <madski@unity3d.com> parents: 22357 diff changeset	240 if close:
cb99bacb9b4e branchcache: introduce revbranchcache for caching of revision branch names Mads Kiilerich <madski@unity3d.com> parents: 22357 diff changeset	241 branchidx &= _rbcbranchidxmask
43077 687b865b95ad formatting: byteify all mercurial/ and hgext/ string literals Augie Fackler <augie@google.com> parents: 43076 diff changeset	242 if cachenode == b'\0\0\0\0':
24376 203a078da052 revbranchcache: populate cache incrementally Durham Goode <durham@fb.com> parents: 24375 diff changeset	243 pass
203a078da052 revbranchcache: populate cache incrementally Durham Goode <durham@fb.com> parents: 24375 diff changeset	244 elif cachenode == reponode:
29615 a2a380e2750f rbc: fix superfluous rebuilding from scratch - don't abuse self._rbcnamescount Mads Kiilerich <madski@unity3d.com> parents: 29604 diff changeset	245 try:
28558 bcd106d456c4 cache: rebuild branch cache from scratch when inconsistencies are detected Mads Kiilerich <madski@unity3d.com> parents: 28557 diff changeset	246 return self._names[branchidx], close
29615 a2a380e2750f rbc: fix superfluous rebuilding from scratch - don't abuse self._rbcnamescount Mads Kiilerich <madski@unity3d.com> parents: 29604 diff changeset	247 except IndexError:
a2a380e2750f rbc: fix superfluous rebuilding from scratch - don't abuse self._rbcnamescount Mads Kiilerich <madski@unity3d.com> parents: 29604 diff changeset	248 # recover from invalid reference to unknown branch
43076 2372284d9457 formatting: blacken the codebase Augie Fackler <augie@google.com> parents: 42603 diff changeset	249 self._repo.ui.debug(
43077 687b865b95ad formatting: byteify all mercurial/ and hgext/ string literals Augie Fackler <augie@google.com> parents: 43076 diff changeset	250 b"referenced branch names not found"
687b865b95ad formatting: byteify all mercurial/ and hgext/ string literals Augie Fackler <augie@google.com> parents: 43076 diff changeset	251 b" - rebuilding revision branch cache from scratch\n"
43076 2372284d9457 formatting: blacken the codebase Augie Fackler <augie@google.com> parents: 42603 diff changeset	252 )
29615 a2a380e2750f rbc: fix superfluous rebuilding from scratch - don't abuse self._rbcnamescount Mads Kiilerich <madski@unity3d.com> parents: 29604 diff changeset	253 self._clear()
24376 203a078da052 revbranchcache: populate cache incrementally Durham Goode <durham@fb.com> parents: 24375 diff changeset	254 else:
203a078da052 revbranchcache: populate cache incrementally Durham Goode <durham@fb.com> parents: 24375 diff changeset	255 # rev/node map has changed, invalidate the cache from here up
43076 2372284d9457 formatting: blacken the codebase Augie Fackler <augie@google.com> parents: 42603 diff changeset	256 self._repo.ui.debug(
43077 687b865b95ad formatting: byteify all mercurial/ and hgext/ string literals Augie Fackler <augie@google.com> parents: 43076 diff changeset	257 b"history modification detected - truncating "
687b865b95ad formatting: byteify all mercurial/ and hgext/ string literals Augie Fackler <augie@google.com> parents: 43076 diff changeset	258 b"revision branch cache to revision %d\n" % rev
43076 2372284d9457 formatting: blacken the codebase Augie Fackler <augie@google.com> parents: 42603 diff changeset	259 )
24376 203a078da052 revbranchcache: populate cache incrementally Durham Goode <durham@fb.com> parents: 24375 diff changeset	260 truncate = rbcrevidx + _rbcrecsize
51377 02e7d79edf62 branchmap: use mmap for faster revbranchcache loading Arseniy Alekseyev <aalekseyev@janestreet.com> parents: 51307 diff changeset	261 self._rbcrevs.truncate(truncate)
24376 203a078da052 revbranchcache: populate cache incrementally Durham Goode <durham@fb.com> parents: 24375 diff changeset	262 self._rbcrevslen = min(self._rbcrevslen, truncate)
203a078da052 revbranchcache: populate cache incrementally Durham Goode <durham@fb.com> parents: 24375 diff changeset	263
23785 cb99bacb9b4e branchcache: introduce revbranchcache for caching of revision branch names Mads Kiilerich <madski@unity3d.com> parents: 22357 diff changeset	264 # fall back to slow path and make sure it will be written to disk
40494 5e5c8f2a1eb5 branchmap: do not specify changelog as an argument Yuya Nishihara <yuya@tcha.org> parents: 40375 diff changeset	265 return self._branchinfo(rev)
23785 cb99bacb9b4e branchcache: introduce revbranchcache for caching of revision branch names Mads Kiilerich <madski@unity3d.com> parents: 22357 diff changeset	266
40494 5e5c8f2a1eb5 branchmap: do not specify changelog as an argument Yuya Nishihara <yuya@tcha.org> parents: 40375 diff changeset	267 def _branchinfo(self, rev):
23785 cb99bacb9b4e branchcache: introduce revbranchcache for caching of revision branch names Mads Kiilerich <madski@unity3d.com> parents: 22357 diff changeset	268 """Retrieve branch info from changelog and update _rbcrevs"""
40494 5e5c8f2a1eb5 branchmap: do not specify changelog as an argument Yuya Nishihara <yuya@tcha.org> parents: 40375 diff changeset	269 changelog = self._repo.changelog
23785 cb99bacb9b4e branchcache: introduce revbranchcache for caching of revision branch names Mads Kiilerich <madski@unity3d.com> parents: 22357 diff changeset	270 b, close = changelog.branchinfo(rev)
cb99bacb9b4e branchcache: introduce revbranchcache for caching of revision branch names Mads Kiilerich <madski@unity3d.com> parents: 22357 diff changeset	271 if b in self._namesreverse:
cb99bacb9b4e branchcache: introduce revbranchcache for caching of revision branch names Mads Kiilerich <madski@unity3d.com> parents: 22357 diff changeset	272 branchidx = self._namesreverse[b]
cb99bacb9b4e branchcache: introduce revbranchcache for caching of revision branch names Mads Kiilerich <madski@unity3d.com> parents: 22357 diff changeset	273 else:
cb99bacb9b4e branchcache: introduce revbranchcache for caching of revision branch names Mads Kiilerich <madski@unity3d.com> parents: 22357 diff changeset	274 branchidx = len(self._names)
cb99bacb9b4e branchcache: introduce revbranchcache for caching of revision branch names Mads Kiilerich <madski@unity3d.com> parents: 22357 diff changeset	275 self._names.append(b)
cb99bacb9b4e branchcache: introduce revbranchcache for caching of revision branch names Mads Kiilerich <madski@unity3d.com> parents: 22357 diff changeset	276 self._namesreverse[b] = branchidx
cb99bacb9b4e branchcache: introduce revbranchcache for caching of revision branch names Mads Kiilerich <madski@unity3d.com> parents: 22357 diff changeset	277 reponode = changelog.node(rev)
cb99bacb9b4e branchcache: introduce revbranchcache for caching of revision branch names Mads Kiilerich <madski@unity3d.com> parents: 22357 diff changeset	278 if close:
cb99bacb9b4e branchcache: introduce revbranchcache for caching of revision branch names Mads Kiilerich <madski@unity3d.com> parents: 22357 diff changeset	279 branchidx \|= _rbccloseflag
40494 5e5c8f2a1eb5 branchmap: do not specify changelog as an argument Yuya Nishihara <yuya@tcha.org> parents: 40375 diff changeset	280 self._setcachedata(rev, reponode, branchidx)
24375 fe255b2525d5 revbranchcache: move entry writing to a separate function Durham Goode <durham@fb.com> parents: 24374 diff changeset	281 return b, close
fe255b2525d5 revbranchcache: move entry writing to a separate function Durham Goode <durham@fb.com> parents: 24374 diff changeset	282
46444 3e91d9978bec branchmap: update rev-branch-cache incrementally Joerg Sonnenberger <joerg@bec.de> parents: 46434 diff changeset	283 def setdata(self, rev, changelogrevision):
36968 95f4f1bfb650 revbranchcache: add a public function to update the data Boris Feld <boris.feld@octobus.net> parents: 36504 diff changeset	284 """add new data information to the cache"""
46444 3e91d9978bec branchmap: update rev-branch-cache incrementally Joerg Sonnenberger <joerg@bec.de> parents: 46434 diff changeset	285 branch, close = changelogrevision.branchinfo
3e91d9978bec branchmap: update rev-branch-cache incrementally Joerg Sonnenberger <joerg@bec.de> parents: 46434 diff changeset	286
36968 95f4f1bfb650 revbranchcache: add a public function to update the data Boris Feld <boris.feld@octobus.net> parents: 36504 diff changeset	287 if branch in self._namesreverse:
95f4f1bfb650 revbranchcache: add a public function to update the data Boris Feld <boris.feld@octobus.net> parents: 36504 diff changeset	288 branchidx = self._namesreverse[branch]
95f4f1bfb650 revbranchcache: add a public function to update the data Boris Feld <boris.feld@octobus.net> parents: 36504 diff changeset	289 else:
95f4f1bfb650 revbranchcache: add a public function to update the data Boris Feld <boris.feld@octobus.net> parents: 36504 diff changeset	290 branchidx = len(self._names)
95f4f1bfb650 revbranchcache: add a public function to update the data Boris Feld <boris.feld@octobus.net> parents: 36504 diff changeset	291 self._names.append(branch)
95f4f1bfb650 revbranchcache: add a public function to update the data Boris Feld <boris.feld@octobus.net> parents: 36504 diff changeset	292 self._namesreverse[branch] = branchidx
95f4f1bfb650 revbranchcache: add a public function to update the data Boris Feld <boris.feld@octobus.net> parents: 36504 diff changeset	293 if close:
95f4f1bfb650 revbranchcache: add a public function to update the data Boris Feld <boris.feld@octobus.net> parents: 36504 diff changeset	294 branchidx \|= _rbccloseflag
46444 3e91d9978bec branchmap: update rev-branch-cache incrementally Joerg Sonnenberger <joerg@bec.de> parents: 46434 diff changeset	295 self._setcachedata(rev, self._repo.changelog.node(rev), branchidx)
36968 95f4f1bfb650 revbranchcache: add a public function to update the data Boris Feld <boris.feld@octobus.net> parents: 36504 diff changeset	296 # If no cache data were readable (non exists, bad permission, etc)
95f4f1bfb650 revbranchcache: add a public function to update the data Boris Feld <boris.feld@octobus.net> parents: 36504 diff changeset	297 # the cache was bypassing itself by setting:
95f4f1bfb650 revbranchcache: add a public function to update the data Boris Feld <boris.feld@octobus.net> parents: 36504 diff changeset	298 #
95f4f1bfb650 revbranchcache: add a public function to update the data Boris Feld <boris.feld@octobus.net> parents: 36504 diff changeset	299 # self.branchinfo = self._branchinfo
95f4f1bfb650 revbranchcache: add a public function to update the data Boris Feld <boris.feld@octobus.net> parents: 36504 diff changeset	300 #
95f4f1bfb650 revbranchcache: add a public function to update the data Boris Feld <boris.feld@octobus.net> parents: 36504 diff changeset	301 # Since we now have data in the cache, we need to drop this bypassing.
43554 9f70512ae2cf cleanup: remove pointless r-prefixes on single-quoted strings Augie Fackler <augie@google.com> parents: 43547 diff changeset	302 if 'branchinfo' in vars(self):
36968 95f4f1bfb650 revbranchcache: add a public function to update the data Boris Feld <boris.feld@octobus.net> parents: 36504 diff changeset	303 del self.branchinfo
95f4f1bfb650 revbranchcache: add a public function to update the data Boris Feld <boris.feld@octobus.net> parents: 36504 diff changeset	304
40494 5e5c8f2a1eb5 branchmap: do not specify changelog as an argument Yuya Nishihara <yuya@tcha.org> parents: 40375 diff changeset	305 def _setcachedata(self, rev, node, branchidx):
24375 fe255b2525d5 revbranchcache: move entry writing to a separate function Durham Goode <durham@fb.com> parents: 24374 diff changeset	306 """Writes the node's branch data to the in-memory cache data."""
31463 a5bad127128d branchmap: handle nullrev in setcachedata Durham Goode <durham@fb.com> parents: 31390 diff changeset	307 if rev == nullrev:
a5bad127128d branchmap: handle nullrev in setcachedata Durham Goode <durham@fb.com> parents: 31390 diff changeset	308 return
23785 cb99bacb9b4e branchcache: introduce revbranchcache for caching of revision branch names Mads Kiilerich <madski@unity3d.com> parents: 22357 diff changeset	309 rbcrevidx = rev * _rbcrecsize
51377 02e7d79edf62 branchmap: use mmap for faster revbranchcache loading Arseniy Alekseyev <aalekseyev@janestreet.com> parents: 51307 diff changeset	310 self._rbcrevs.pack_into(rbcrevidx, node, branchidx)
24376 203a078da052 revbranchcache: populate cache incrementally Durham Goode <durham@fb.com> parents: 24375 diff changeset	311 self._rbcrevslen = min(self._rbcrevslen, rev)
23785 cb99bacb9b4e branchcache: introduce revbranchcache for caching of revision branch names Mads Kiilerich <madski@unity3d.com> parents: 22357 diff changeset	312
24377 656f93ce66d5 revbranchcache: move cache writing to the transaction finalizer Durham Goode <durham@fb.com> parents: 24376 diff changeset	313 tr = self._repo.currenttransaction()
656f93ce66d5 revbranchcache: move cache writing to the transaction finalizer Durham Goode <durham@fb.com> parents: 24376 diff changeset	314 if tr:
43077 687b865b95ad formatting: byteify all mercurial/ and hgext/ string literals Augie Fackler <augie@google.com> parents: 43076 diff changeset	315 tr.addfinalize(b'write-revbranchcache', self.write)
24377 656f93ce66d5 revbranchcache: move cache writing to the transaction finalizer Durham Goode <durham@fb.com> parents: 24376 diff changeset	316
656f93ce66d5 revbranchcache: move cache writing to the transaction finalizer Durham Goode <durham@fb.com> parents: 24376 diff changeset	317 def write(self, tr=None):
23785 cb99bacb9b4e branchcache: introduce revbranchcache for caching of revision branch names Mads Kiilerich <madski@unity3d.com> parents: 22357 diff changeset	318 """Save branch cache if it is dirty."""
24374 77fd1fb538cd revbranchcache: store repo on the object Durham Goode <durham@fb.com> parents: 24373 diff changeset	319 repo = self._repo
29756 0d588332ad2c branchmap: acquires lock before writting the rev branch cache Pierre-Yves David <pierre-yves.david@ens-lyon.org> parents: 29755 diff changeset	320 wlock = None
43077 687b865b95ad formatting: byteify all mercurial/ and hgext/ string literals Augie Fackler <augie@google.com> parents: 43076 diff changeset	321 step = b''
29756 0d588332ad2c branchmap: acquires lock before writting the rev branch cache Pierre-Yves David <pierre-yves.david@ens-lyon.org> parents: 29755 diff changeset	322 try:
42222 09fd338522fa revbranchcache: factor logic to write names and revs in separate functions Pulkit Goyal <pulkit@yandex-team.ru> parents: 42138 diff changeset	323 # write the new names
51945 16efed18ae4e rev-branch-cache: schedule a write of the "v2" format if we read from "v1" Pierre-Yves David <pierre-yves.david@octobus.net> parents: 51944 diff changeset	324 if self._force_overwrite or self._rbcnamescount < len(self._names):
29756 0d588332ad2c branchmap: acquires lock before writting the rev branch cache Pierre-Yves David <pierre-yves.david@ens-lyon.org> parents: 29755 diff changeset	325 wlock = repo.wlock(wait=False)
43077 687b865b95ad formatting: byteify all mercurial/ and hgext/ string literals Augie Fackler <augie@google.com> parents: 43076 diff changeset	326 step = b' names'
42222 09fd338522fa revbranchcache: factor logic to write names and revs in separate functions Pulkit Goyal <pulkit@yandex-team.ru> parents: 42138 diff changeset	327 self._writenames(repo)
23785 cb99bacb9b4e branchcache: introduce revbranchcache for caching of revision branch names Mads Kiilerich <madski@unity3d.com> parents: 22357 diff changeset	328
42222 09fd338522fa revbranchcache: factor logic to write names and revs in separate functions Pulkit Goyal <pulkit@yandex-team.ru> parents: 42138 diff changeset	329 # write the new revs
29755 9f3c49ee4486 branchmap: preparatory indent of indent the branch rev writing code Pierre-Yves David <pierre-yves.david@ens-lyon.org> parents: 29615 diff changeset	330 start = self._rbcrevslen * _rbcrecsize
51940 9f7cf869e9f4 rev-branch-cache: add a way to force rewrite of the cache Pierre-Yves David <pierre-yves.david@octobus.net> parents: 51939 diff changeset	331 if self._force_overwrite or start != len(self._rbcrevs):
43077 687b865b95ad formatting: byteify all mercurial/ and hgext/ string literals Augie Fackler <augie@google.com> parents: 43076 diff changeset	332 step = b''
29756 0d588332ad2c branchmap: acquires lock before writting the rev branch cache Pierre-Yves David <pierre-yves.david@ens-lyon.org> parents: 29755 diff changeset	333 if wlock is None:
0d588332ad2c branchmap: acquires lock before writting the rev branch cache Pierre-Yves David <pierre-yves.david@ens-lyon.org> parents: 29755 diff changeset	334 wlock = repo.wlock(wait=False)
42222 09fd338522fa revbranchcache: factor logic to write names and revs in separate functions Pulkit Goyal <pulkit@yandex-team.ru> parents: 42138 diff changeset	335 self._writerevs(repo, start)
09fd338522fa revbranchcache: factor logic to write names and revs in separate functions Pulkit Goyal <pulkit@yandex-team.ru> parents: 42138 diff changeset	336
29757 3b184adfb5be branchmap: simplify error handlind when writing rev branch cache Pierre-Yves David <pierre-yves.david@ens-lyon.org> parents: 29756 diff changeset	337 except (IOError, OSError, error.Abort, error.LockError) as inst:
43076 2372284d9457 formatting: blacken the codebase Augie Fackler <augie@google.com> parents: 42603 diff changeset	338 repo.ui.debug(
43077 687b865b95ad formatting: byteify all mercurial/ and hgext/ string literals Augie Fackler <augie@google.com> parents: 43076 diff changeset	339 b"couldn't write revision branch cache%s: %s\n"
43076 2372284d9457 formatting: blacken the codebase Augie Fackler <augie@google.com> parents: 42603 diff changeset	340 % (step, stringutil.forcebytestr(inst))
2372284d9457 formatting: blacken the codebase Augie Fackler <augie@google.com> parents: 42603 diff changeset	341 )
29756 0d588332ad2c branchmap: acquires lock before writting the rev branch cache Pierre-Yves David <pierre-yves.david@ens-lyon.org> parents: 29755 diff changeset	342 finally:
0d588332ad2c branchmap: acquires lock before writting the rev branch cache Pierre-Yves David <pierre-yves.david@ens-lyon.org> parents: 29755 diff changeset	343 if wlock is not None:
0d588332ad2c branchmap: acquires lock before writting the rev branch cache Pierre-Yves David <pierre-yves.david@ens-lyon.org> parents: 29755 diff changeset	344 wlock.release()
42222 09fd338522fa revbranchcache: factor logic to write names and revs in separate functions Pulkit Goyal <pulkit@yandex-team.ru> parents: 42138 diff changeset	345
09fd338522fa revbranchcache: factor logic to write names and revs in separate functions Pulkit Goyal <pulkit@yandex-team.ru> parents: 42138 diff changeset	346 def _writenames(self, repo):
47031 f38bf44e077f black: make codebase compatible with black v21.4b2 and v20.8b1 Kyle Lippincott <spectral@google.com> parents: 46819 diff changeset	347 """write the new branch names to revbranchcache"""
51941 7032da075572 rev-branch-cache: make sure we close the name file we open Pierre-Yves David <pierre-yves.david@octobus.net> parents: 51940 diff changeset	348 f = None
51945 16efed18ae4e rev-branch-cache: schedule a write of the "v2" format if we read from "v1" Pierre-Yves David <pierre-yves.david@octobus.net> parents: 51944 diff changeset	349 if self._force_overwrite:
16efed18ae4e rev-branch-cache: schedule a write of the "v2" format if we read from "v1" Pierre-Yves David <pierre-yves.david@octobus.net> parents: 51944 diff changeset	350 self._rbcsnameslen = 0
16efed18ae4e rev-branch-cache: schedule a write of the "v2" format if we read from "v1" Pierre-Yves David <pierre-yves.david@octobus.net> parents: 51944 diff changeset	351 self._rbcnamescount = 0
51941 7032da075572 rev-branch-cache: make sure we close the name file we open Pierre-Yves David <pierre-yves.david@octobus.net> parents: 51940 diff changeset	352 try:
51945 16efed18ae4e rev-branch-cache: schedule a write of the "v2" format if we read from "v1" Pierre-Yves David <pierre-yves.david@octobus.net> parents: 51944 diff changeset	353 if self._force_overwrite or self._rbcnamescount != 0:
51941 7032da075572 rev-branch-cache: make sure we close the name file we open Pierre-Yves David <pierre-yves.david@octobus.net> parents: 51940 diff changeset	354 f = repo.cachevfs.open(_rbcnames, b'ab')
51945 16efed18ae4e rev-branch-cache: schedule a write of the "v2" format if we read from "v1" Pierre-Yves David <pierre-yves.david@octobus.net> parents: 51944 diff changeset	355 current_size = f.tell()
16efed18ae4e rev-branch-cache: schedule a write of the "v2" format if we read from "v1" Pierre-Yves David <pierre-yves.david@octobus.net> parents: 51944 diff changeset	356 if current_size == self._rbcsnameslen:
51941 7032da075572 rev-branch-cache: make sure we close the name file we open Pierre-Yves David <pierre-yves.david@octobus.net> parents: 51940 diff changeset	357 f.write(b'\0')
7032da075572 rev-branch-cache: make sure we close the name file we open Pierre-Yves David <pierre-yves.david@octobus.net> parents: 51940 diff changeset	358 else:
7032da075572 rev-branch-cache: make sure we close the name file we open Pierre-Yves David <pierre-yves.david@octobus.net> parents: 51940 diff changeset	359 f.close()
51945 16efed18ae4e rev-branch-cache: schedule a write of the "v2" format if we read from "v1" Pierre-Yves David <pierre-yves.david@octobus.net> parents: 51944 diff changeset	360 if self._force_overwrite:
16efed18ae4e rev-branch-cache: schedule a write of the "v2" format if we read from "v1" Pierre-Yves David <pierre-yves.david@octobus.net> parents: 51944 diff changeset	361 dbg = b"resetting content of %s\n"
16efed18ae4e rev-branch-cache: schedule a write of the "v2" format if we read from "v1" Pierre-Yves David <pierre-yves.david@octobus.net> parents: 51944 diff changeset	362 elif current_size > 0:
16efed18ae4e rev-branch-cache: schedule a write of the "v2" format if we read from "v1" Pierre-Yves David <pierre-yves.david@octobus.net> parents: 51944 diff changeset	363 dbg = b"%s changed - rewriting it\n"
16efed18ae4e rev-branch-cache: schedule a write of the "v2" format if we read from "v1" Pierre-Yves David <pierre-yves.david@octobus.net> parents: 51944 diff changeset	364 else:
16efed18ae4e rev-branch-cache: schedule a write of the "v2" format if we read from "v1" Pierre-Yves David <pierre-yves.david@octobus.net> parents: 51944 diff changeset	365 dbg = b"%s is missing - rewriting it\n"
16efed18ae4e rev-branch-cache: schedule a write of the "v2" format if we read from "v1" Pierre-Yves David <pierre-yves.david@octobus.net> parents: 51944 diff changeset	366 repo.ui.debug(dbg % _rbcnames)
51941 7032da075572 rev-branch-cache: make sure we close the name file we open Pierre-Yves David <pierre-yves.david@octobus.net> parents: 51940 diff changeset	367 self._rbcnamescount = 0
7032da075572 rev-branch-cache: make sure we close the name file we open Pierre-Yves David <pierre-yves.david@octobus.net> parents: 51940 diff changeset	368 self._rbcrevslen = 0
7032da075572 rev-branch-cache: make sure we close the name file we open Pierre-Yves David <pierre-yves.david@octobus.net> parents: 51940 diff changeset	369 if self._rbcnamescount == 0:
7032da075572 rev-branch-cache: make sure we close the name file we open Pierre-Yves David <pierre-yves.david@octobus.net> parents: 51940 diff changeset	370 # before rewriting names, make sure references are removed
7032da075572 rev-branch-cache: make sure we close the name file we open Pierre-Yves David <pierre-yves.david@octobus.net> parents: 51940 diff changeset	371 repo.cachevfs.unlinkpath(_rbcrevs, ignoremissing=True)
7032da075572 rev-branch-cache: make sure we close the name file we open Pierre-Yves David <pierre-yves.david@octobus.net> parents: 51940 diff changeset	372 f = repo.cachevfs.open(_rbcnames, b'wb')
7032da075572 rev-branch-cache: make sure we close the name file we open Pierre-Yves David <pierre-yves.david@octobus.net> parents: 51940 diff changeset	373 names = self._names[self._rbcnamescount :]
7032da075572 rev-branch-cache: make sure we close the name file we open Pierre-Yves David <pierre-yves.david@octobus.net> parents: 51940 diff changeset	374 from_local = encoding.fromlocal
7032da075572 rev-branch-cache: make sure we close the name file we open Pierre-Yves David <pierre-yves.david@octobus.net> parents: 51940 diff changeset	375 data = b'\0'.join(from_local(b) for b in names)
7032da075572 rev-branch-cache: make sure we close the name file we open Pierre-Yves David <pierre-yves.david@octobus.net> parents: 51940 diff changeset	376 f.write(data)
7032da075572 rev-branch-cache: make sure we close the name file we open Pierre-Yves David <pierre-yves.david@octobus.net> parents: 51940 diff changeset	377 self._rbcsnameslen = f.tell()
7032da075572 rev-branch-cache: make sure we close the name file we open Pierre-Yves David <pierre-yves.david@octobus.net> parents: 51940 diff changeset	378 finally:
7032da075572 rev-branch-cache: make sure we close the name file we open Pierre-Yves David <pierre-yves.david@octobus.net> parents: 51940 diff changeset	379 if f is not None:
42222 09fd338522fa revbranchcache: factor logic to write names and revs in separate functions Pulkit Goyal <pulkit@yandex-team.ru> parents: 42138 diff changeset	380 f.close()
09fd338522fa revbranchcache: factor logic to write names and revs in separate functions Pulkit Goyal <pulkit@yandex-team.ru> parents: 42138 diff changeset	381 self._rbcnamescount = len(self._names)
09fd338522fa revbranchcache: factor logic to write names and revs in separate functions Pulkit Goyal <pulkit@yandex-team.ru> parents: 42138 diff changeset	382
09fd338522fa revbranchcache: factor logic to write names and revs in separate functions Pulkit Goyal <pulkit@yandex-team.ru> parents: 42138 diff changeset	383 def _writerevs(self, repo, start):
47031 f38bf44e077f black: make codebase compatible with black v21.4b2 and v20.8b1 Kyle Lippincott <spectral@google.com> parents: 46819 diff changeset	384 """write the new revs to revbranchcache"""
42223 ececa45c80d8 revbranchcache: use context manager in _writerevs() to write to file Pulkit Goyal <pulkit@yandex-team.ru> parents: 42222 diff changeset	385 revs = min(len(repo.changelog), len(self._rbcrevs) // _rbcrecsize)
51942 c564be351754 rev-branch-cache: stop truncating cache file Pierre-Yves David <pierre-yves.david@octobus.net> parents: 51941 diff changeset	386
c564be351754 rev-branch-cache: stop truncating cache file Pierre-Yves David <pierre-yves.david@octobus.net> parents: 51941 diff changeset	387 end = revs * _rbcrecsize
51940 9f7cf869e9f4 rev-branch-cache: add a way to force rewrite of the cache Pierre-Yves David <pierre-yves.david@octobus.net> parents: 51939 diff changeset	388 if self._force_overwrite:
9f7cf869e9f4 rev-branch-cache: add a way to force rewrite of the cache Pierre-Yves David <pierre-yves.david@octobus.net> parents: 51939 diff changeset	389 start = 0
51942 c564be351754 rev-branch-cache: stop truncating cache file Pierre-Yves David <pierre-yves.david@octobus.net> parents: 51941 diff changeset	390
c564be351754 rev-branch-cache: stop truncating cache file Pierre-Yves David <pierre-yves.david@octobus.net> parents: 51941 diff changeset	391 with repo.cachevfs.open(_rbcrevs, b'a+b') as f:
c564be351754 rev-branch-cache: stop truncating cache file Pierre-Yves David <pierre-yves.david@octobus.net> parents: 51941 diff changeset	392 pass # this make sure the file exist…
c564be351754 rev-branch-cache: stop truncating cache file Pierre-Yves David <pierre-yves.david@octobus.net> parents: 51941 diff changeset	393 with repo.cachevfs.open(_rbcrevs, b'r+b') as f:
c564be351754 rev-branch-cache: stop truncating cache file Pierre-Yves David <pierre-yves.david@octobus.net> parents: 51941 diff changeset	394 f.seek(0, os.SEEK_END)
51939 1eb2317c1762 rev-branch-cache: issue more truthful "truncating" message Pierre-Yves David <pierre-yves.david@octobus.net> parents: 51938 diff changeset	395 current_size = f.tell()
1eb2317c1762 rev-branch-cache: issue more truthful "truncating" message Pierre-Yves David <pierre-yves.david@octobus.net> parents: 51938 diff changeset	396 if current_size < start:
1eb2317c1762 rev-branch-cache: issue more truthful "truncating" message Pierre-Yves David <pierre-yves.david@octobus.net> parents: 51938 diff changeset	397 start = 0
1eb2317c1762 rev-branch-cache: issue more truthful "truncating" message Pierre-Yves David <pierre-yves.david@octobus.net> parents: 51938 diff changeset	398 if current_size != start:
51942 c564be351754 rev-branch-cache: stop truncating cache file Pierre-Yves David <pierre-yves.david@octobus.net> parents: 51941 diff changeset	399 threshold = current_size * REWRITE_RATIO
c564be351754 rev-branch-cache: stop truncating cache file Pierre-Yves David <pierre-yves.david@octobus.net> parents: 51941 diff changeset	400 if (max(end, current_size) - start) < threshold:
c564be351754 rev-branch-cache: stop truncating cache file Pierre-Yves David <pierre-yves.david@octobus.net> parents: 51941 diff changeset	401 # end affected, let overwrite the bad value
c564be351754 rev-branch-cache: stop truncating cache file Pierre-Yves David <pierre-yves.david@octobus.net> parents: 51941 diff changeset	402 dbg = b"overwriting %d bytes from %d in cache/%s"
c564be351754 rev-branch-cache: stop truncating cache file Pierre-Yves David <pierre-yves.david@octobus.net> parents: 51941 diff changeset	403 dbg %= (current_size - start, start, _rbcrevs)
c564be351754 rev-branch-cache: stop truncating cache file Pierre-Yves David <pierre-yves.david@octobus.net> parents: 51941 diff changeset	404 if end < current_size:
c564be351754 rev-branch-cache: stop truncating cache file Pierre-Yves David <pierre-yves.david@octobus.net> parents: 51941 diff changeset	405 extra = b" leaving (%d trailing bytes)"
c564be351754 rev-branch-cache: stop truncating cache file Pierre-Yves David <pierre-yves.david@octobus.net> parents: 51941 diff changeset	406 extra %= current_size - end
c564be351754 rev-branch-cache: stop truncating cache file Pierre-Yves David <pierre-yves.david@octobus.net> parents: 51941 diff changeset	407 dbg += extra
c564be351754 rev-branch-cache: stop truncating cache file Pierre-Yves David <pierre-yves.david@octobus.net> parents: 51941 diff changeset	408 dbg += b'\n'
c564be351754 rev-branch-cache: stop truncating cache file Pierre-Yves David <pierre-yves.david@octobus.net> parents: 51941 diff changeset	409 repo.ui.debug(dbg)
c564be351754 rev-branch-cache: stop truncating cache file Pierre-Yves David <pierre-yves.david@octobus.net> parents: 51941 diff changeset	410 else:
c564be351754 rev-branch-cache: stop truncating cache file Pierre-Yves David <pierre-yves.david@octobus.net> parents: 51941 diff changeset	411 start = 0
c564be351754 rev-branch-cache: stop truncating cache file Pierre-Yves David <pierre-yves.david@octobus.net> parents: 51941 diff changeset	412 dbg = b"resetting content of cache/%s\n" % _rbcrevs
c564be351754 rev-branch-cache: stop truncating cache file Pierre-Yves David <pierre-yves.david@octobus.net> parents: 51941 diff changeset	413 repo.ui.debug(dbg)
c564be351754 rev-branch-cache: stop truncating cache file Pierre-Yves David <pierre-yves.david@octobus.net> parents: 51941 diff changeset	414 if start > 0:
42222 09fd338522fa revbranchcache: factor logic to write names and revs in separate functions Pulkit Goyal <pulkit@yandex-team.ru> parents: 42138 diff changeset	415 f.seek(start)
51942 c564be351754 rev-branch-cache: stop truncating cache file Pierre-Yves David <pierre-yves.david@octobus.net> parents: 51941 diff changeset	416 f.write(self._rbcrevs.slice(start, end))
c564be351754 rev-branch-cache: stop truncating cache file Pierre-Yves David <pierre-yves.david@octobus.net> parents: 51941 diff changeset	417 else:
c564be351754 rev-branch-cache: stop truncating cache file Pierre-Yves David <pierre-yves.david@octobus.net> parents: 51941 diff changeset	418 f.close()
c564be351754 rev-branch-cache: stop truncating cache file Pierre-Yves David <pierre-yves.david@octobus.net> parents: 51941 diff changeset	419 with repo.cachevfs.open(
c564be351754 rev-branch-cache: stop truncating cache file Pierre-Yves David <pierre-yves.david@octobus.net> parents: 51941 diff changeset	420 _rbcrevs,
c564be351754 rev-branch-cache: stop truncating cache file Pierre-Yves David <pierre-yves.david@octobus.net> parents: 51941 diff changeset	421 b'wb',
c564be351754 rev-branch-cache: stop truncating cache file Pierre-Yves David <pierre-yves.david@octobus.net> parents: 51941 diff changeset	422 atomictemp=True,
c564be351754 rev-branch-cache: stop truncating cache file Pierre-Yves David <pierre-yves.david@octobus.net> parents: 51941 diff changeset	423 ) as rev_file:
c564be351754 rev-branch-cache: stop truncating cache file Pierre-Yves David <pierre-yves.david@octobus.net> parents: 51941 diff changeset	424 rev_file.write(self._rbcrevs.slice(start, end))
42222 09fd338522fa revbranchcache: factor logic to write names and revs in separate functions Pulkit Goyal <pulkit@yandex-team.ru> parents: 42138 diff changeset	425 self._rbcrevslen = revs
51940 9f7cf869e9f4 rev-branch-cache: add a way to force rewrite of the cache Pierre-Yves David <pierre-yves.david@octobus.net> parents: 51939 diff changeset	426 self._force_overwrite = False

Mercurial > public > mercurial-scm > hg-stable

annotate mercurial/branching/rev_cache.py @ 51945:16efed18ae4e