mercurial-scm/hg-stable: mercurial/branching/rev

annotate mercurial/branching/rev_cache.py @ 51939:1eb2317c1762

rev-branch-cache: issue more truthful "truncating" message First, don't pretend it truncate to 40 when it actually truncate to 0. Second, don't pretend to truncate to 0 when the file is already empty/missing.

author	Pierre-Yves David <pierre-yves.david@octobus.net>
date	Tue, 24 Sep 2024 00:01:30 +0200
parents	f0e07efc199f
children	9f7cf869e9f4

rev	line source
51938 f0e07efc199f rev-branch-cache: move the code in a dedicated module Pierre-Yves David <pierre-yves.david@octobus.net> parents: 51901 diff changeset	1 # rev_cache.py - caching branch information per revision
18116 bcee63733aad branchmap: create a mercurial.branchmap module Pierre-Yves David <pierre-yves.david@ens-lyon.org> parents: diff changeset	2 #
bcee63733aad branchmap: create a mercurial.branchmap module Pierre-Yves David <pierre-yves.david@ens-lyon.org> parents: diff changeset	3 # This software may be used and distributed according to the terms of the
bcee63733aad branchmap: create a mercurial.branchmap module Pierre-Yves David <pierre-yves.david@ens-lyon.org> parents: diff changeset	4 # GNU General Public License version 2 or any later version.
51901 f4733654f144 typing: add `from __future__ import annotations` to most files Matt Harbison <matt_harbison@yahoo.com> parents: 51651 diff changeset	5 from __future__ import annotations
25918 47f36e050c2e branchmap: use absolute_import Gregory Szorc <gregory.szorc@gmail.com> parents: 25660 diff changeset	6
47f36e050c2e branchmap: use absolute_import Gregory Szorc <gregory.szorc@gmail.com> parents: 25660 diff changeset	7 import struct
47f36e050c2e branchmap: use absolute_import Gregory Szorc <gregory.szorc@gmail.com> parents: 25660 diff changeset	8
51938 f0e07efc199f rev-branch-cache: move the code in a dedicated module Pierre-Yves David <pierre-yves.david@octobus.net> parents: 51901 diff changeset	9 from ..node import (
25918 47f36e050c2e branchmap: use absolute_import Gregory Szorc <gregory.szorc@gmail.com> parents: 25660 diff changeset	10 nullrev,
47f36e050c2e branchmap: use absolute_import Gregory Szorc <gregory.szorc@gmail.com> parents: 25660 diff changeset	11 )
51302 9d3721552b6c pytype: import typing directly Pierre-Yves David <pierre-yves.david@octobus.net> parents: 49308 diff changeset	12
51938 f0e07efc199f rev-branch-cache: move the code in a dedicated module Pierre-Yves David <pierre-yves.david@octobus.net> parents: 51901 diff changeset	13 from .. import (
25918 47f36e050c2e branchmap: use absolute_import Gregory Szorc <gregory.szorc@gmail.com> parents: 25660 diff changeset	14 encoding,
26587 56b2bcea2529 error: get Abort from 'error' instead of 'util' Pierre-Yves David <pierre-yves.david@fb.com> parents: 26460 diff changeset	15 error,
30995 22fbca1d11ed mercurial: switch to util.timer for all interval timings Simon Farnsworth <simonfar@fb.com> parents: 29758 diff changeset	16 util,
25918 47f36e050c2e branchmap: use absolute_import Gregory Szorc <gregory.szorc@gmail.com> parents: 25660 diff changeset	17 )
51302 9d3721552b6c pytype: import typing directly Pierre-Yves David <pierre-yves.david@octobus.net> parents: 49308 diff changeset	18
51938 f0e07efc199f rev-branch-cache: move the code in a dedicated module Pierre-Yves David <pierre-yves.david@octobus.net> parents: 51901 diff changeset	19 from ..utils import (
37087 f0b6fbea00cf stringutil: bulk-replace call sites to point to new module Yuya Nishihara <yuya@tcha.org> parents: 36968 diff changeset	20 stringutil,
f0b6fbea00cf stringutil: bulk-replace call sites to point to new module Yuya Nishihara <yuya@tcha.org> parents: 36968 diff changeset	21 )
25918 47f36e050c2e branchmap: use absolute_import Gregory Szorc <gregory.szorc@gmail.com> parents: 25660 diff changeset	22
47f36e050c2e branchmap: use absolute_import Gregory Szorc <gregory.szorc@gmail.com> parents: 25660 diff changeset	23 calcsize = struct.calcsize
31379 906be86990c4 rbc: use struct unpack_from and pack_into instead of unpack and pack Mads Kiilerich <madski@unity3d.com> parents: 31369 diff changeset	24 pack_into = struct.pack_into
906be86990c4 rbc: use struct unpack_from and pack_into instead of unpack and pack Mads Kiilerich <madski@unity3d.com> parents: 31369 diff changeset	25 unpack_from = struct.unpack_from
18117 526e7ec5c96e branchmap: extract write logic from localrepo Pierre-Yves David <pierre-yves.david@ens-lyon.org> parents: 18116 diff changeset	26
18118 e70ff1e599f4 branchmap: extract read logic from repo Pierre-Yves David <pierre-yves.david@logilab.fr> parents: 18117 diff changeset	27
23785 cb99bacb9b4e branchcache: introduce revbranchcache for caching of revision branch names Mads Kiilerich <madski@unity3d.com> parents: 22357 diff changeset	28 # Revision branch info cache
cb99bacb9b4e branchcache: introduce revbranchcache for caching of revision branch names Mads Kiilerich <madski@unity3d.com> parents: 22357 diff changeset	29
43077 687b865b95ad formatting: byteify all mercurial/ and hgext/ string literals Augie Fackler <augie@google.com> parents: 43076 diff changeset	30 _rbcversion = b'-v1'
687b865b95ad formatting: byteify all mercurial/ and hgext/ string literals Augie Fackler <augie@google.com> parents: 43076 diff changeset	31 _rbcnames = b'rbc-names' + _rbcversion
687b865b95ad formatting: byteify all mercurial/ and hgext/ string literals Augie Fackler <augie@google.com> parents: 43076 diff changeset	32 _rbcrevs = b'rbc-revs' + _rbcversion
23785 cb99bacb9b4e branchcache: introduce revbranchcache for caching of revision branch names Mads Kiilerich <madski@unity3d.com> parents: 22357 diff changeset	33 # [4 byte hash prefix][4 byte branch name number with sign bit indicating open]
43077 687b865b95ad formatting: byteify all mercurial/ and hgext/ string literals Augie Fackler <augie@google.com> parents: 43076 diff changeset	34 _rbcrecfmt = b'>4sI'
23785 cb99bacb9b4e branchcache: introduce revbranchcache for caching of revision branch names Mads Kiilerich <madski@unity3d.com> parents: 22357 diff changeset	35 _rbcrecsize = calcsize(_rbcrecfmt)
46434 1726a53a8494 reverse-branch-cache: switch to doubling allocating scheme Joerg Sonnenberger <joerg@bec.de> parents: 46254 diff changeset	36 _rbcmininc = 64 * _rbcrecsize
23785 cb99bacb9b4e branchcache: introduce revbranchcache for caching of revision branch names Mads Kiilerich <madski@unity3d.com> parents: 22357 diff changeset	37 _rbcnodelen = 4
43076 2372284d9457 formatting: blacken the codebase Augie Fackler <augie@google.com> parents: 42603 diff changeset	38 _rbcbranchidxmask = 0x7FFFFFFF
23785 cb99bacb9b4e branchcache: introduce revbranchcache for caching of revision branch names Mads Kiilerich <madski@unity3d.com> parents: 22357 diff changeset	39 _rbccloseflag = 0x80000000
cb99bacb9b4e branchcache: introduce revbranchcache for caching of revision branch names Mads Kiilerich <madski@unity3d.com> parents: 22357 diff changeset	40
43076 2372284d9457 formatting: blacken the codebase Augie Fackler <augie@google.com> parents: 42603 diff changeset	41
51377 02e7d79edf62 branchmap: use mmap for faster revbranchcache loading Arseniy Alekseyev <aalekseyev@janestreet.com> parents: 51307 diff changeset	42 class rbcrevs:
02e7d79edf62 branchmap: use mmap for faster revbranchcache loading Arseniy Alekseyev <aalekseyev@janestreet.com> parents: 51307 diff changeset	43 """a byte string consisting of an immutable prefix followed by a mutable suffix"""
02e7d79edf62 branchmap: use mmap for faster revbranchcache loading Arseniy Alekseyev <aalekseyev@janestreet.com> parents: 51307 diff changeset	44
02e7d79edf62 branchmap: use mmap for faster revbranchcache loading Arseniy Alekseyev <aalekseyev@janestreet.com> parents: 51307 diff changeset	45 def __init__(self, revs):
02e7d79edf62 branchmap: use mmap for faster revbranchcache loading Arseniy Alekseyev <aalekseyev@janestreet.com> parents: 51307 diff changeset	46 self._prefix = revs
02e7d79edf62 branchmap: use mmap for faster revbranchcache loading Arseniy Alekseyev <aalekseyev@janestreet.com> parents: 51307 diff changeset	47 self._rest = bytearray()
02e7d79edf62 branchmap: use mmap for faster revbranchcache loading Arseniy Alekseyev <aalekseyev@janestreet.com> parents: 51307 diff changeset	48
02e7d79edf62 branchmap: use mmap for faster revbranchcache loading Arseniy Alekseyev <aalekseyev@janestreet.com> parents: 51307 diff changeset	49 def __len__(self):
02e7d79edf62 branchmap: use mmap for faster revbranchcache loading Arseniy Alekseyev <aalekseyev@janestreet.com> parents: 51307 diff changeset	50 return len(self._prefix) + len(self._rest)
02e7d79edf62 branchmap: use mmap for faster revbranchcache loading Arseniy Alekseyev <aalekseyev@janestreet.com> parents: 51307 diff changeset	51
02e7d79edf62 branchmap: use mmap for faster revbranchcache loading Arseniy Alekseyev <aalekseyev@janestreet.com> parents: 51307 diff changeset	52 def unpack_record(self, rbcrevidx):
02e7d79edf62 branchmap: use mmap for faster revbranchcache loading Arseniy Alekseyev <aalekseyev@janestreet.com> parents: 51307 diff changeset	53 if rbcrevidx < len(self._prefix):
02e7d79edf62 branchmap: use mmap for faster revbranchcache loading Arseniy Alekseyev <aalekseyev@janestreet.com> parents: 51307 diff changeset	54 return unpack_from(_rbcrecfmt, util.buffer(self._prefix), rbcrevidx)
02e7d79edf62 branchmap: use mmap for faster revbranchcache loading Arseniy Alekseyev <aalekseyev@janestreet.com> parents: 51307 diff changeset	55 else:
02e7d79edf62 branchmap: use mmap for faster revbranchcache loading Arseniy Alekseyev <aalekseyev@janestreet.com> parents: 51307 diff changeset	56 return unpack_from(
02e7d79edf62 branchmap: use mmap for faster revbranchcache loading Arseniy Alekseyev <aalekseyev@janestreet.com> parents: 51307 diff changeset	57 _rbcrecfmt,
02e7d79edf62 branchmap: use mmap for faster revbranchcache loading Arseniy Alekseyev <aalekseyev@janestreet.com> parents: 51307 diff changeset	58 util.buffer(self._rest),
02e7d79edf62 branchmap: use mmap for faster revbranchcache loading Arseniy Alekseyev <aalekseyev@janestreet.com> parents: 51307 diff changeset	59 rbcrevidx - len(self._prefix),
02e7d79edf62 branchmap: use mmap for faster revbranchcache loading Arseniy Alekseyev <aalekseyev@janestreet.com> parents: 51307 diff changeset	60 )
02e7d79edf62 branchmap: use mmap for faster revbranchcache loading Arseniy Alekseyev <aalekseyev@janestreet.com> parents: 51307 diff changeset	61
02e7d79edf62 branchmap: use mmap for faster revbranchcache loading Arseniy Alekseyev <aalekseyev@janestreet.com> parents: 51307 diff changeset	62 def make_mutable(self):
02e7d79edf62 branchmap: use mmap for faster revbranchcache loading Arseniy Alekseyev <aalekseyev@janestreet.com> parents: 51307 diff changeset	63 if len(self._prefix) > 0:
02e7d79edf62 branchmap: use mmap for faster revbranchcache loading Arseniy Alekseyev <aalekseyev@janestreet.com> parents: 51307 diff changeset	64 entirety = bytearray()
02e7d79edf62 branchmap: use mmap for faster revbranchcache loading Arseniy Alekseyev <aalekseyev@janestreet.com> parents: 51307 diff changeset	65 entirety[:] = self._prefix
02e7d79edf62 branchmap: use mmap for faster revbranchcache loading Arseniy Alekseyev <aalekseyev@janestreet.com> parents: 51307 diff changeset	66 entirety.extend(self._rest)
02e7d79edf62 branchmap: use mmap for faster revbranchcache loading Arseniy Alekseyev <aalekseyev@janestreet.com> parents: 51307 diff changeset	67 self._rest = entirety
02e7d79edf62 branchmap: use mmap for faster revbranchcache loading Arseniy Alekseyev <aalekseyev@janestreet.com> parents: 51307 diff changeset	68 self._prefix = bytearray()
02e7d79edf62 branchmap: use mmap for faster revbranchcache loading Arseniy Alekseyev <aalekseyev@janestreet.com> parents: 51307 diff changeset	69
02e7d79edf62 branchmap: use mmap for faster revbranchcache loading Arseniy Alekseyev <aalekseyev@janestreet.com> parents: 51307 diff changeset	70 def truncate(self, pos):
02e7d79edf62 branchmap: use mmap for faster revbranchcache loading Arseniy Alekseyev <aalekseyev@janestreet.com> parents: 51307 diff changeset	71 self.make_mutable()
02e7d79edf62 branchmap: use mmap for faster revbranchcache loading Arseniy Alekseyev <aalekseyev@janestreet.com> parents: 51307 diff changeset	72 del self._rest[pos:]
02e7d79edf62 branchmap: use mmap for faster revbranchcache loading Arseniy Alekseyev <aalekseyev@janestreet.com> parents: 51307 diff changeset	73
02e7d79edf62 branchmap: use mmap for faster revbranchcache loading Arseniy Alekseyev <aalekseyev@janestreet.com> parents: 51307 diff changeset	74 def pack_into(self, rbcrevidx, node, branchidx):
02e7d79edf62 branchmap: use mmap for faster revbranchcache loading Arseniy Alekseyev <aalekseyev@janestreet.com> parents: 51307 diff changeset	75 if rbcrevidx < len(self._prefix):
02e7d79edf62 branchmap: use mmap for faster revbranchcache loading Arseniy Alekseyev <aalekseyev@janestreet.com> parents: 51307 diff changeset	76 self.make_mutable()
02e7d79edf62 branchmap: use mmap for faster revbranchcache loading Arseniy Alekseyev <aalekseyev@janestreet.com> parents: 51307 diff changeset	77 buf = self._rest
02e7d79edf62 branchmap: use mmap for faster revbranchcache loading Arseniy Alekseyev <aalekseyev@janestreet.com> parents: 51307 diff changeset	78 start_offset = rbcrevidx - len(self._prefix)
02e7d79edf62 branchmap: use mmap for faster revbranchcache loading Arseniy Alekseyev <aalekseyev@janestreet.com> parents: 51307 diff changeset	79 end_offset = start_offset + _rbcrecsize
02e7d79edf62 branchmap: use mmap for faster revbranchcache loading Arseniy Alekseyev <aalekseyev@janestreet.com> parents: 51307 diff changeset	80
02e7d79edf62 branchmap: use mmap for faster revbranchcache loading Arseniy Alekseyev <aalekseyev@janestreet.com> parents: 51307 diff changeset	81 if len(self._rest) < end_offset:
02e7d79edf62 branchmap: use mmap for faster revbranchcache loading Arseniy Alekseyev <aalekseyev@janestreet.com> parents: 51307 diff changeset	82 # bytearray doesn't allocate extra space at least in Python 3.7.
02e7d79edf62 branchmap: use mmap for faster revbranchcache loading Arseniy Alekseyev <aalekseyev@janestreet.com> parents: 51307 diff changeset	83 # When multiple changesets are added in a row, precise resize would
02e7d79edf62 branchmap: use mmap for faster revbranchcache loading Arseniy Alekseyev <aalekseyev@janestreet.com> parents: 51307 diff changeset	84 # result in quadratic complexity. Overallocate to compensate by
02e7d79edf62 branchmap: use mmap for faster revbranchcache loading Arseniy Alekseyev <aalekseyev@janestreet.com> parents: 51307 diff changeset	85 # using the classic doubling technique for dynamic arrays instead.
02e7d79edf62 branchmap: use mmap for faster revbranchcache loading Arseniy Alekseyev <aalekseyev@janestreet.com> parents: 51307 diff changeset	86 # If there was a gap in the map before, less space will be reserved.
02e7d79edf62 branchmap: use mmap for faster revbranchcache loading Arseniy Alekseyev <aalekseyev@janestreet.com> parents: 51307 diff changeset	87 self._rest.extend(b'\0' * end_offset)
02e7d79edf62 branchmap: use mmap for faster revbranchcache loading Arseniy Alekseyev <aalekseyev@janestreet.com> parents: 51307 diff changeset	88 return pack_into(
02e7d79edf62 branchmap: use mmap for faster revbranchcache loading Arseniy Alekseyev <aalekseyev@janestreet.com> parents: 51307 diff changeset	89 _rbcrecfmt,
02e7d79edf62 branchmap: use mmap for faster revbranchcache loading Arseniy Alekseyev <aalekseyev@janestreet.com> parents: 51307 diff changeset	90 buf,
02e7d79edf62 branchmap: use mmap for faster revbranchcache loading Arseniy Alekseyev <aalekseyev@janestreet.com> parents: 51307 diff changeset	91 start_offset,
02e7d79edf62 branchmap: use mmap for faster revbranchcache loading Arseniy Alekseyev <aalekseyev@janestreet.com> parents: 51307 diff changeset	92 node,
02e7d79edf62 branchmap: use mmap for faster revbranchcache loading Arseniy Alekseyev <aalekseyev@janestreet.com> parents: 51307 diff changeset	93 branchidx,
02e7d79edf62 branchmap: use mmap for faster revbranchcache loading Arseniy Alekseyev <aalekseyev@janestreet.com> parents: 51307 diff changeset	94 )
02e7d79edf62 branchmap: use mmap for faster revbranchcache loading Arseniy Alekseyev <aalekseyev@janestreet.com> parents: 51307 diff changeset	95
02e7d79edf62 branchmap: use mmap for faster revbranchcache loading Arseniy Alekseyev <aalekseyev@janestreet.com> parents: 51307 diff changeset	96 def extend(self, extension):
02e7d79edf62 branchmap: use mmap for faster revbranchcache loading Arseniy Alekseyev <aalekseyev@janestreet.com> parents: 51307 diff changeset	97 return self._rest.extend(extension)
02e7d79edf62 branchmap: use mmap for faster revbranchcache loading Arseniy Alekseyev <aalekseyev@janestreet.com> parents: 51307 diff changeset	98
02e7d79edf62 branchmap: use mmap for faster revbranchcache loading Arseniy Alekseyev <aalekseyev@janestreet.com> parents: 51307 diff changeset	99 def slice(self, begin, end):
02e7d79edf62 branchmap: use mmap for faster revbranchcache loading Arseniy Alekseyev <aalekseyev@janestreet.com> parents: 51307 diff changeset	100 if begin < len(self._prefix):
02e7d79edf62 branchmap: use mmap for faster revbranchcache loading Arseniy Alekseyev <aalekseyev@janestreet.com> parents: 51307 diff changeset	101 acc = bytearray()
02e7d79edf62 branchmap: use mmap for faster revbranchcache loading Arseniy Alekseyev <aalekseyev@janestreet.com> parents: 51307 diff changeset	102 acc[:] = self._prefix[begin:end]
02e7d79edf62 branchmap: use mmap for faster revbranchcache loading Arseniy Alekseyev <aalekseyev@janestreet.com> parents: 51307 diff changeset	103 acc.extend(
02e7d79edf62 branchmap: use mmap for faster revbranchcache loading Arseniy Alekseyev <aalekseyev@janestreet.com> parents: 51307 diff changeset	104 self._rest[begin - len(self._prefix) : end - len(self._prefix)]
02e7d79edf62 branchmap: use mmap for faster revbranchcache loading Arseniy Alekseyev <aalekseyev@janestreet.com> parents: 51307 diff changeset	105 )
02e7d79edf62 branchmap: use mmap for faster revbranchcache loading Arseniy Alekseyev <aalekseyev@janestreet.com> parents: 51307 diff changeset	106 return acc
02e7d79edf62 branchmap: use mmap for faster revbranchcache loading Arseniy Alekseyev <aalekseyev@janestreet.com> parents: 51307 diff changeset	107 return self._rest[begin - len(self._prefix) : end - len(self._prefix)]
02e7d79edf62 branchmap: use mmap for faster revbranchcache loading Arseniy Alekseyev <aalekseyev@janestreet.com> parents: 51307 diff changeset	108
02e7d79edf62 branchmap: use mmap for faster revbranchcache loading Arseniy Alekseyev <aalekseyev@janestreet.com> parents: 51307 diff changeset	109
49037 642e31cb55f0 py3: use class X: instead of class X(object): Gregory Szorc <gregory.szorc@gmail.com> parents: 49026 diff changeset	110 class revbranchcache:
23785 cb99bacb9b4e branchcache: introduce revbranchcache for caching of revision branch names Mads Kiilerich <madski@unity3d.com> parents: 22357 diff changeset	111 """Persistent cache, mapping from revision number to branch name and close.
cb99bacb9b4e branchcache: introduce revbranchcache for caching of revision branch names Mads Kiilerich <madski@unity3d.com> parents: 22357 diff changeset	112 This is a low level cache, independent of filtering.
cb99bacb9b4e branchcache: introduce revbranchcache for caching of revision branch names Mads Kiilerich <madski@unity3d.com> parents: 22357 diff changeset	113
cb99bacb9b4e branchcache: introduce revbranchcache for caching of revision branch names Mads Kiilerich <madski@unity3d.com> parents: 22357 diff changeset	114 Branch names are stored in rbc-names in internal encoding separated by 0.
cb99bacb9b4e branchcache: introduce revbranchcache for caching of revision branch names Mads Kiilerich <madski@unity3d.com> parents: 22357 diff changeset	115 rbc-names is append-only, and each branch name is only stored once and will
cb99bacb9b4e branchcache: introduce revbranchcache for caching of revision branch names Mads Kiilerich <madski@unity3d.com> parents: 22357 diff changeset	116 thus have a unique index.
cb99bacb9b4e branchcache: introduce revbranchcache for caching of revision branch names Mads Kiilerich <madski@unity3d.com> parents: 22357 diff changeset	117
cb99bacb9b4e branchcache: introduce revbranchcache for caching of revision branch names Mads Kiilerich <madski@unity3d.com> parents: 22357 diff changeset	118 The branch info for each revision is stored in rbc-revs as constant size
cb99bacb9b4e branchcache: introduce revbranchcache for caching of revision branch names Mads Kiilerich <madski@unity3d.com> parents: 22357 diff changeset	119 records. The whole file is read into memory, but it is only 'parsed' on
cb99bacb9b4e branchcache: introduce revbranchcache for caching of revision branch names Mads Kiilerich <madski@unity3d.com> parents: 22357 diff changeset	120 demand. The file is usually append-only but will be truncated if repo
cb99bacb9b4e branchcache: introduce revbranchcache for caching of revision branch names Mads Kiilerich <madski@unity3d.com> parents: 22357 diff changeset	121 modification is detected.
cb99bacb9b4e branchcache: introduce revbranchcache for caching of revision branch names Mads Kiilerich <madski@unity3d.com> parents: 22357 diff changeset	122 The record for each revision contains the first 4 bytes of the
cb99bacb9b4e branchcache: introduce revbranchcache for caching of revision branch names Mads Kiilerich <madski@unity3d.com> parents: 22357 diff changeset	123 corresponding node hash, and the record is only used if it still matches.
cb99bacb9b4e branchcache: introduce revbranchcache for caching of revision branch names Mads Kiilerich <madski@unity3d.com> parents: 22357 diff changeset	124 Even a completely trashed rbc-revs fill thus still give the right result
cb99bacb9b4e branchcache: introduce revbranchcache for caching of revision branch names Mads Kiilerich <madski@unity3d.com> parents: 22357 diff changeset	125 while converging towards full recovery ... assuming no incorrectly matching
cb99bacb9b4e branchcache: introduce revbranchcache for caching of revision branch names Mads Kiilerich <madski@unity3d.com> parents: 22357 diff changeset	126 node hashes.
cb99bacb9b4e branchcache: introduce revbranchcache for caching of revision branch names Mads Kiilerich <madski@unity3d.com> parents: 22357 diff changeset	127 The record also contains 4 bytes where 31 bits contains the index of the
cb99bacb9b4e branchcache: introduce revbranchcache for caching of revision branch names Mads Kiilerich <madski@unity3d.com> parents: 22357 diff changeset	128 branch and the last bit indicate that it is a branch close commit.
cb99bacb9b4e branchcache: introduce revbranchcache for caching of revision branch names Mads Kiilerich <madski@unity3d.com> parents: 22357 diff changeset	129 The usage pattern for rbc-revs is thus somewhat similar to 00changelog.i
cb99bacb9b4e branchcache: introduce revbranchcache for caching of revision branch names Mads Kiilerich <madski@unity3d.com> parents: 22357 diff changeset	130 and will grow with it but be 1/8th of its size.
cb99bacb9b4e branchcache: introduce revbranchcache for caching of revision branch names Mads Kiilerich <madski@unity3d.com> parents: 22357 diff changeset	131 """
cb99bacb9b4e branchcache: introduce revbranchcache for caching of revision branch names Mads Kiilerich <madski@unity3d.com> parents: 22357 diff changeset	132
24159 5b4ed033390b revisionbranchcache: fall back to slow path if starting readonly (issue4531) Mads Kiilerich <madski@unity3d.com> parents: 23877 diff changeset	133 def __init__(self, repo, readonly=True):
23785 cb99bacb9b4e branchcache: introduce revbranchcache for caching of revision branch names Mads Kiilerich <madski@unity3d.com> parents: 22357 diff changeset	134 assert repo.filtername is None
24374 77fd1fb538cd revbranchcache: store repo on the object Durham Goode <durham@fb.com> parents: 24373 diff changeset	135 self._repo = repo
43076 2372284d9457 formatting: blacken the codebase Augie Fackler <augie@google.com> parents: 42603 diff changeset	136 self._names = [] # branch names in local encoding with static index
51377 02e7d79edf62 branchmap: use mmap for faster revbranchcache loading Arseniy Alekseyev <aalekseyev@janestreet.com> parents: 51307 diff changeset	137 self._rbcrevs = rbcrevs(bytearray())
43076 2372284d9457 formatting: blacken the codebase Augie Fackler <augie@google.com> parents: 42603 diff changeset	138 self._rbcsnameslen = 0 # length of names read at _rbcsnameslen
23785 cb99bacb9b4e branchcache: introduce revbranchcache for caching of revision branch names Mads Kiilerich <madski@unity3d.com> parents: 22357 diff changeset	139 try:
33535 755e6532e81d cachevfs: migration the revbranchcache to 'cachevfs' Boris Feld <boris.feld@octobus.net> parents: 33534 diff changeset	140 bndata = repo.cachevfs.read(_rbcnames)
43076 2372284d9457 formatting: blacken the codebase Augie Fackler <augie@google.com> parents: 42603 diff changeset	141 self._rbcsnameslen = len(bndata) # for verification before writing
31380 7dd2f51f38ac rbc: empty (and invalid) rbc-names file should give an empty name list Mads Kiilerich <mads@kiilerich.com> parents: 31379 diff changeset	142 if bndata:
43076 2372284d9457 formatting: blacken the codebase Augie Fackler <augie@google.com> parents: 42603 diff changeset	143 self._names = [
43077 687b865b95ad formatting: byteify all mercurial/ and hgext/ string literals Augie Fackler <augie@google.com> parents: 43076 diff changeset	144 encoding.tolocal(bn) for bn in bndata.split(b'\0')
43076 2372284d9457 formatting: blacken the codebase Augie Fackler <augie@google.com> parents: 42603 diff changeset	145 ]
29423 d2c6f3a948fa branchmap: remove unused exception variable Gregory Szorc <gregory.szorc@gmail.com> parents: 28558 diff changeset	146 except (IOError, OSError):
24159 5b4ed033390b revisionbranchcache: fall back to slow path if starting readonly (issue4531) Mads Kiilerich <madski@unity3d.com> parents: 23877 diff changeset	147 if readonly:
5b4ed033390b revisionbranchcache: fall back to slow path if starting readonly (issue4531) Mads Kiilerich <madski@unity3d.com> parents: 23877 diff changeset	148 # don't try to use cache - fall back to the slow path
5b4ed033390b revisionbranchcache: fall back to slow path if starting readonly (issue4531) Mads Kiilerich <madski@unity3d.com> parents: 23877 diff changeset	149 self.branchinfo = self._branchinfo
5b4ed033390b revisionbranchcache: fall back to slow path if starting readonly (issue4531) Mads Kiilerich <madski@unity3d.com> parents: 23877 diff changeset	150
23785 cb99bacb9b4e branchcache: introduce revbranchcache for caching of revision branch names Mads Kiilerich <madski@unity3d.com> parents: 22357 diff changeset	151 if self._names:
cb99bacb9b4e branchcache: introduce revbranchcache for caching of revision branch names Mads Kiilerich <madski@unity3d.com> parents: 22357 diff changeset	152 try:
51651 c0e30a019ce1 mmap: only use mmap to read rev-branch-cache data if it is safe Pierre-Yves David <pierre-yves.david@octobus.net> parents: 51644 diff changeset	153 usemmap = repo.ui.configbool(b'storage', b'revbranchcache.mmap')
c0e30a019ce1 mmap: only use mmap to read rev-branch-cache data if it is safe Pierre-Yves David <pierre-yves.david@octobus.net> parents: 51644 diff changeset	154 with repo.cachevfs(_rbcrevs) as fp:
c0e30a019ce1 mmap: only use mmap to read rev-branch-cache data if it is safe Pierre-Yves David <pierre-yves.david@octobus.net> parents: 51644 diff changeset	155 if usemmap and repo.cachevfs.is_mmap_safe(_rbcrevs):
51377 02e7d79edf62 branchmap: use mmap for faster revbranchcache loading Arseniy Alekseyev <aalekseyev@janestreet.com> parents: 51307 diff changeset	156 data = util.buffer(util.mmapread(fp))
51651 c0e30a019ce1 mmap: only use mmap to read rev-branch-cache data if it is safe Pierre-Yves David <pierre-yves.david@octobus.net> parents: 51644 diff changeset	157 else:
c0e30a019ce1 mmap: only use mmap to read rev-branch-cache data if it is safe Pierre-Yves David <pierre-yves.david@octobus.net> parents: 51644 diff changeset	158 data = fp.read()
51377 02e7d79edf62 branchmap: use mmap for faster revbranchcache loading Arseniy Alekseyev <aalekseyev@janestreet.com> parents: 51307 diff changeset	159 self._rbcrevs = rbcrevs(data)
25660 328739ea70c3 global: mass rewrite to use modern exception syntax Gregory Szorc <gregory.szorc@gmail.com> parents: 25266 diff changeset	160 except (IOError, OSError) as inst:
43076 2372284d9457 formatting: blacken the codebase Augie Fackler <augie@google.com> parents: 42603 diff changeset	161 repo.ui.debug(
43077 687b865b95ad formatting: byteify all mercurial/ and hgext/ string literals Augie Fackler <augie@google.com> parents: 43076 diff changeset	162 b"couldn't read revision branch cache: %s\n"
43076 2372284d9457 formatting: blacken the codebase Augie Fackler <augie@google.com> parents: 42603 diff changeset	163 % stringutil.forcebytestr(inst)
2372284d9457 formatting: blacken the codebase Augie Fackler <augie@google.com> parents: 42603 diff changeset	164 )
23785 cb99bacb9b4e branchcache: introduce revbranchcache for caching of revision branch names Mads Kiilerich <madski@unity3d.com> parents: 22357 diff changeset	165 # remember number of good records on disk
43076 2372284d9457 formatting: blacken the codebase Augie Fackler <augie@google.com> parents: 42603 diff changeset	166 self._rbcrevslen = min(
2372284d9457 formatting: blacken the codebase Augie Fackler <augie@google.com> parents: 42603 diff changeset	167 len(self._rbcrevs) // _rbcrecsize, len(repo.changelog)
2372284d9457 formatting: blacken the codebase Augie Fackler <augie@google.com> parents: 42603 diff changeset	168 )
23785 cb99bacb9b4e branchcache: introduce revbranchcache for caching of revision branch names Mads Kiilerich <madski@unity3d.com> parents: 22357 diff changeset	169 if self._rbcrevslen == 0:
cb99bacb9b4e branchcache: introduce revbranchcache for caching of revision branch names Mads Kiilerich <madski@unity3d.com> parents: 22357 diff changeset	170 self._names = []
43076 2372284d9457 formatting: blacken the codebase Augie Fackler <augie@google.com> parents: 42603 diff changeset	171 self._rbcnamescount = len(self._names) # number of names read at
2372284d9457 formatting: blacken the codebase Augie Fackler <augie@google.com> parents: 42603 diff changeset	172 # _rbcsnameslen
23785 cb99bacb9b4e branchcache: introduce revbranchcache for caching of revision branch names Mads Kiilerich <madski@unity3d.com> parents: 22357 diff changeset	173
28558 bcd106d456c4 cache: rebuild branch cache from scratch when inconsistencies are detected Mads Kiilerich <madski@unity3d.com> parents: 28557 diff changeset	174 def _clear(self):
bcd106d456c4 cache: rebuild branch cache from scratch when inconsistencies are detected Mads Kiilerich <madski@unity3d.com> parents: 28557 diff changeset	175 self._rbcsnameslen = 0
bcd106d456c4 cache: rebuild branch cache from scratch when inconsistencies are detected Mads Kiilerich <madski@unity3d.com> parents: 28557 diff changeset	176 del self._names[:]
bcd106d456c4 cache: rebuild branch cache from scratch when inconsistencies are detected Mads Kiilerich <madski@unity3d.com> parents: 28557 diff changeset	177 self._rbcnamescount = 0
bcd106d456c4 cache: rebuild branch cache from scratch when inconsistencies are detected Mads Kiilerich <madski@unity3d.com> parents: 28557 diff changeset	178 self._rbcrevslen = len(self._repo.changelog)
51377 02e7d79edf62 branchmap: use mmap for faster revbranchcache loading Arseniy Alekseyev <aalekseyev@janestreet.com> parents: 51307 diff changeset	179 self._rbcrevs = rbcrevs(bytearray(self._rbcrevslen * _rbcrecsize))
43077 687b865b95ad formatting: byteify all mercurial/ and hgext/ string literals Augie Fackler <augie@google.com> parents: 43076 diff changeset	180 util.clearcachedproperty(self, b'_namesreverse')
40749 50a64c321c1e branchmap: build the revbranchcache._namesreverse() only when required Pulkit Goyal <pulkit@yandex-team.ru> parents: 40494 diff changeset	181
50a64c321c1e branchmap: build the revbranchcache._namesreverse() only when required Pulkit Goyal <pulkit@yandex-team.ru> parents: 40494 diff changeset	182 @util.propertycache
50a64c321c1e branchmap: build the revbranchcache._namesreverse() only when required Pulkit Goyal <pulkit@yandex-team.ru> parents: 40494 diff changeset	183 def _namesreverse(self):
44470 9d2b2df2c2ba cleanup: run pyupgrade on our source tree to clean up varying things Augie Fackler <augie@google.com> parents: 44349 diff changeset	184 return {b: r for r, b in enumerate(self._names)}
28558 bcd106d456c4 cache: rebuild branch cache from scratch when inconsistencies are detected Mads Kiilerich <madski@unity3d.com> parents: 28557 diff changeset	185
40494 5e5c8f2a1eb5 branchmap: do not specify changelog as an argument Yuya Nishihara <yuya@tcha.org> parents: 40375 diff changeset	186 def branchinfo(self, rev):
23785 cb99bacb9b4e branchcache: introduce revbranchcache for caching of revision branch names Mads Kiilerich <madski@unity3d.com> parents: 22357 diff changeset	187 """Return branch name and close flag for rev, using and updating
cb99bacb9b4e branchcache: introduce revbranchcache for caching of revision branch names Mads Kiilerich <madski@unity3d.com> parents: 22357 diff changeset	188 persistent cache."""
40494 5e5c8f2a1eb5 branchmap: do not specify changelog as an argument Yuya Nishihara <yuya@tcha.org> parents: 40375 diff changeset	189 changelog = self._repo.changelog
23785 cb99bacb9b4e branchcache: introduce revbranchcache for caching of revision branch names Mads Kiilerich <madski@unity3d.com> parents: 22357 diff changeset	190 rbcrevidx = rev * _rbcrecsize
cb99bacb9b4e branchcache: introduce revbranchcache for caching of revision branch names Mads Kiilerich <madski@unity3d.com> parents: 22357 diff changeset	191
25266 38117278f295 revbranchcache: return uncached branchinfo for nullrev (issue4683) Yuya Nishihara <yuya@tcha.org> parents: 24728 diff changeset	192 # avoid negative index, changelog.read(nullrev) is fast without cache
38117278f295 revbranchcache: return uncached branchinfo for nullrev (issue4683) Yuya Nishihara <yuya@tcha.org> parents: 24728 diff changeset	193 if rev == nullrev:
38117278f295 revbranchcache: return uncached branchinfo for nullrev (issue4683) Yuya Nishihara <yuya@tcha.org> parents: 24728 diff changeset	194 return changelog.branchinfo(rev)
38117278f295 revbranchcache: return uncached branchinfo for nullrev (issue4683) Yuya Nishihara <yuya@tcha.org> parents: 24728 diff changeset	195
29604 db0095c83344 rbc: fix invalid rbc-revs entries caused by missing cache growth Mads Kiilerich <madski@unity3d.com> parents: 29423 diff changeset	196 # if requested rev isn't allocated, grow and cache the rev info
23785 cb99bacb9b4e branchcache: introduce revbranchcache for caching of revision branch names Mads Kiilerich <madski@unity3d.com> parents: 22357 diff changeset	197 if len(self._rbcrevs) < rbcrevidx + _rbcrecsize:
40494 5e5c8f2a1eb5 branchmap: do not specify changelog as an argument Yuya Nishihara <yuya@tcha.org> parents: 40375 diff changeset	198 return self._branchinfo(rev)
23785 cb99bacb9b4e branchcache: introduce revbranchcache for caching of revision branch names Mads Kiilerich <madski@unity3d.com> parents: 22357 diff changeset	199
cb99bacb9b4e branchcache: introduce revbranchcache for caching of revision branch names Mads Kiilerich <madski@unity3d.com> parents: 22357 diff changeset	200 # fast path: extract data from cache, use it if node is matching
cb99bacb9b4e branchcache: introduce revbranchcache for caching of revision branch names Mads Kiilerich <madski@unity3d.com> parents: 22357 diff changeset	201 reponode = changelog.node(rev)[:_rbcnodelen]
51377 02e7d79edf62 branchmap: use mmap for faster revbranchcache loading Arseniy Alekseyev <aalekseyev@janestreet.com> parents: 51307 diff changeset	202 cachenode, branchidx = self._rbcrevs.unpack_record(rbcrevidx)
23785 cb99bacb9b4e branchcache: introduce revbranchcache for caching of revision branch names Mads Kiilerich <madski@unity3d.com> parents: 22357 diff changeset	203 close = bool(branchidx & _rbccloseflag)
cb99bacb9b4e branchcache: introduce revbranchcache for caching of revision branch names Mads Kiilerich <madski@unity3d.com> parents: 22357 diff changeset	204 if close:
cb99bacb9b4e branchcache: introduce revbranchcache for caching of revision branch names Mads Kiilerich <madski@unity3d.com> parents: 22357 diff changeset	205 branchidx &= _rbcbranchidxmask
43077 687b865b95ad formatting: byteify all mercurial/ and hgext/ string literals Augie Fackler <augie@google.com> parents: 43076 diff changeset	206 if cachenode == b'\0\0\0\0':
24376 203a078da052 revbranchcache: populate cache incrementally Durham Goode <durham@fb.com> parents: 24375 diff changeset	207 pass
203a078da052 revbranchcache: populate cache incrementally Durham Goode <durham@fb.com> parents: 24375 diff changeset	208 elif cachenode == reponode:
29615 a2a380e2750f rbc: fix superfluous rebuilding from scratch - don't abuse self._rbcnamescount Mads Kiilerich <madski@unity3d.com> parents: 29604 diff changeset	209 try:
28558 bcd106d456c4 cache: rebuild branch cache from scratch when inconsistencies are detected Mads Kiilerich <madski@unity3d.com> parents: 28557 diff changeset	210 return self._names[branchidx], close
29615 a2a380e2750f rbc: fix superfluous rebuilding from scratch - don't abuse self._rbcnamescount Mads Kiilerich <madski@unity3d.com> parents: 29604 diff changeset	211 except IndexError:
a2a380e2750f rbc: fix superfluous rebuilding from scratch - don't abuse self._rbcnamescount Mads Kiilerich <madski@unity3d.com> parents: 29604 diff changeset	212 # recover from invalid reference to unknown branch
43076 2372284d9457 formatting: blacken the codebase Augie Fackler <augie@google.com> parents: 42603 diff changeset	213 self._repo.ui.debug(
43077 687b865b95ad formatting: byteify all mercurial/ and hgext/ string literals Augie Fackler <augie@google.com> parents: 43076 diff changeset	214 b"referenced branch names not found"
687b865b95ad formatting: byteify all mercurial/ and hgext/ string literals Augie Fackler <augie@google.com> parents: 43076 diff changeset	215 b" - rebuilding revision branch cache from scratch\n"
43076 2372284d9457 formatting: blacken the codebase Augie Fackler <augie@google.com> parents: 42603 diff changeset	216 )
29615 a2a380e2750f rbc: fix superfluous rebuilding from scratch - don't abuse self._rbcnamescount Mads Kiilerich <madski@unity3d.com> parents: 29604 diff changeset	217 self._clear()
24376 203a078da052 revbranchcache: populate cache incrementally Durham Goode <durham@fb.com> parents: 24375 diff changeset	218 else:
203a078da052 revbranchcache: populate cache incrementally Durham Goode <durham@fb.com> parents: 24375 diff changeset	219 # rev/node map has changed, invalidate the cache from here up
43076 2372284d9457 formatting: blacken the codebase Augie Fackler <augie@google.com> parents: 42603 diff changeset	220 self._repo.ui.debug(
43077 687b865b95ad formatting: byteify all mercurial/ and hgext/ string literals Augie Fackler <augie@google.com> parents: 43076 diff changeset	221 b"history modification detected - truncating "
687b865b95ad formatting: byteify all mercurial/ and hgext/ string literals Augie Fackler <augie@google.com> parents: 43076 diff changeset	222 b"revision branch cache to revision %d\n" % rev
43076 2372284d9457 formatting: blacken the codebase Augie Fackler <augie@google.com> parents: 42603 diff changeset	223 )
24376 203a078da052 revbranchcache: populate cache incrementally Durham Goode <durham@fb.com> parents: 24375 diff changeset	224 truncate = rbcrevidx + _rbcrecsize
51377 02e7d79edf62 branchmap: use mmap for faster revbranchcache loading Arseniy Alekseyev <aalekseyev@janestreet.com> parents: 51307 diff changeset	225 self._rbcrevs.truncate(truncate)
24376 203a078da052 revbranchcache: populate cache incrementally Durham Goode <durham@fb.com> parents: 24375 diff changeset	226 self._rbcrevslen = min(self._rbcrevslen, truncate)
203a078da052 revbranchcache: populate cache incrementally Durham Goode <durham@fb.com> parents: 24375 diff changeset	227
23785 cb99bacb9b4e branchcache: introduce revbranchcache for caching of revision branch names Mads Kiilerich <madski@unity3d.com> parents: 22357 diff changeset	228 # fall back to slow path and make sure it will be written to disk
40494 5e5c8f2a1eb5 branchmap: do not specify changelog as an argument Yuya Nishihara <yuya@tcha.org> parents: 40375 diff changeset	229 return self._branchinfo(rev)
23785 cb99bacb9b4e branchcache: introduce revbranchcache for caching of revision branch names Mads Kiilerich <madski@unity3d.com> parents: 22357 diff changeset	230
40494 5e5c8f2a1eb5 branchmap: do not specify changelog as an argument Yuya Nishihara <yuya@tcha.org> parents: 40375 diff changeset	231 def _branchinfo(self, rev):
23785 cb99bacb9b4e branchcache: introduce revbranchcache for caching of revision branch names Mads Kiilerich <madski@unity3d.com> parents: 22357 diff changeset	232 """Retrieve branch info from changelog and update _rbcrevs"""
40494 5e5c8f2a1eb5 branchmap: do not specify changelog as an argument Yuya Nishihara <yuya@tcha.org> parents: 40375 diff changeset	233 changelog = self._repo.changelog
23785 cb99bacb9b4e branchcache: introduce revbranchcache for caching of revision branch names Mads Kiilerich <madski@unity3d.com> parents: 22357 diff changeset	234 b, close = changelog.branchinfo(rev)
cb99bacb9b4e branchcache: introduce revbranchcache for caching of revision branch names Mads Kiilerich <madski@unity3d.com> parents: 22357 diff changeset	235 if b in self._namesreverse:
cb99bacb9b4e branchcache: introduce revbranchcache for caching of revision branch names Mads Kiilerich <madski@unity3d.com> parents: 22357 diff changeset	236 branchidx = self._namesreverse[b]
cb99bacb9b4e branchcache: introduce revbranchcache for caching of revision branch names Mads Kiilerich <madski@unity3d.com> parents: 22357 diff changeset	237 else:
cb99bacb9b4e branchcache: introduce revbranchcache for caching of revision branch names Mads Kiilerich <madski@unity3d.com> parents: 22357 diff changeset	238 branchidx = len(self._names)
cb99bacb9b4e branchcache: introduce revbranchcache for caching of revision branch names Mads Kiilerich <madski@unity3d.com> parents: 22357 diff changeset	239 self._names.append(b)
cb99bacb9b4e branchcache: introduce revbranchcache for caching of revision branch names Mads Kiilerich <madski@unity3d.com> parents: 22357 diff changeset	240 self._namesreverse[b] = branchidx
cb99bacb9b4e branchcache: introduce revbranchcache for caching of revision branch names Mads Kiilerich <madski@unity3d.com> parents: 22357 diff changeset	241 reponode = changelog.node(rev)
cb99bacb9b4e branchcache: introduce revbranchcache for caching of revision branch names Mads Kiilerich <madski@unity3d.com> parents: 22357 diff changeset	242 if close:
cb99bacb9b4e branchcache: introduce revbranchcache for caching of revision branch names Mads Kiilerich <madski@unity3d.com> parents: 22357 diff changeset	243 branchidx \|= _rbccloseflag
40494 5e5c8f2a1eb5 branchmap: do not specify changelog as an argument Yuya Nishihara <yuya@tcha.org> parents: 40375 diff changeset	244 self._setcachedata(rev, reponode, branchidx)
24375 fe255b2525d5 revbranchcache: move entry writing to a separate function Durham Goode <durham@fb.com> parents: 24374 diff changeset	245 return b, close
fe255b2525d5 revbranchcache: move entry writing to a separate function Durham Goode <durham@fb.com> parents: 24374 diff changeset	246
46444 3e91d9978bec branchmap: update rev-branch-cache incrementally Joerg Sonnenberger <joerg@bec.de> parents: 46434 diff changeset	247 def setdata(self, rev, changelogrevision):
36968 95f4f1bfb650 revbranchcache: add a public function to update the data Boris Feld <boris.feld@octobus.net> parents: 36504 diff changeset	248 """add new data information to the cache"""
46444 3e91d9978bec branchmap: update rev-branch-cache incrementally Joerg Sonnenberger <joerg@bec.de> parents: 46434 diff changeset	249 branch, close = changelogrevision.branchinfo
3e91d9978bec branchmap: update rev-branch-cache incrementally Joerg Sonnenberger <joerg@bec.de> parents: 46434 diff changeset	250
36968 95f4f1bfb650 revbranchcache: add a public function to update the data Boris Feld <boris.feld@octobus.net> parents: 36504 diff changeset	251 if branch in self._namesreverse:
95f4f1bfb650 revbranchcache: add a public function to update the data Boris Feld <boris.feld@octobus.net> parents: 36504 diff changeset	252 branchidx = self._namesreverse[branch]
95f4f1bfb650 revbranchcache: add a public function to update the data Boris Feld <boris.feld@octobus.net> parents: 36504 diff changeset	253 else:
95f4f1bfb650 revbranchcache: add a public function to update the data Boris Feld <boris.feld@octobus.net> parents: 36504 diff changeset	254 branchidx = len(self._names)
95f4f1bfb650 revbranchcache: add a public function to update the data Boris Feld <boris.feld@octobus.net> parents: 36504 diff changeset	255 self._names.append(branch)
95f4f1bfb650 revbranchcache: add a public function to update the data Boris Feld <boris.feld@octobus.net> parents: 36504 diff changeset	256 self._namesreverse[branch] = branchidx
95f4f1bfb650 revbranchcache: add a public function to update the data Boris Feld <boris.feld@octobus.net> parents: 36504 diff changeset	257 if close:
95f4f1bfb650 revbranchcache: add a public function to update the data Boris Feld <boris.feld@octobus.net> parents: 36504 diff changeset	258 branchidx \|= _rbccloseflag
46444 3e91d9978bec branchmap: update rev-branch-cache incrementally Joerg Sonnenberger <joerg@bec.de> parents: 46434 diff changeset	259 self._setcachedata(rev, self._repo.changelog.node(rev), branchidx)
36968 95f4f1bfb650 revbranchcache: add a public function to update the data Boris Feld <boris.feld@octobus.net> parents: 36504 diff changeset	260 # If no cache data were readable (non exists, bad permission, etc)
95f4f1bfb650 revbranchcache: add a public function to update the data Boris Feld <boris.feld@octobus.net> parents: 36504 diff changeset	261 # the cache was bypassing itself by setting:
95f4f1bfb650 revbranchcache: add a public function to update the data Boris Feld <boris.feld@octobus.net> parents: 36504 diff changeset	262 #
95f4f1bfb650 revbranchcache: add a public function to update the data Boris Feld <boris.feld@octobus.net> parents: 36504 diff changeset	263 # self.branchinfo = self._branchinfo
95f4f1bfb650 revbranchcache: add a public function to update the data Boris Feld <boris.feld@octobus.net> parents: 36504 diff changeset	264 #
95f4f1bfb650 revbranchcache: add a public function to update the data Boris Feld <boris.feld@octobus.net> parents: 36504 diff changeset	265 # Since we now have data in the cache, we need to drop this bypassing.
43554 9f70512ae2cf cleanup: remove pointless r-prefixes on single-quoted strings Augie Fackler <augie@google.com> parents: 43547 diff changeset	266 if 'branchinfo' in vars(self):
36968 95f4f1bfb650 revbranchcache: add a public function to update the data Boris Feld <boris.feld@octobus.net> parents: 36504 diff changeset	267 del self.branchinfo
95f4f1bfb650 revbranchcache: add a public function to update the data Boris Feld <boris.feld@octobus.net> parents: 36504 diff changeset	268
40494 5e5c8f2a1eb5 branchmap: do not specify changelog as an argument Yuya Nishihara <yuya@tcha.org> parents: 40375 diff changeset	269 def _setcachedata(self, rev, node, branchidx):
24375 fe255b2525d5 revbranchcache: move entry writing to a separate function Durham Goode <durham@fb.com> parents: 24374 diff changeset	270 """Writes the node's branch data to the in-memory cache data."""
31463 a5bad127128d branchmap: handle nullrev in setcachedata Durham Goode <durham@fb.com> parents: 31390 diff changeset	271 if rev == nullrev:
a5bad127128d branchmap: handle nullrev in setcachedata Durham Goode <durham@fb.com> parents: 31390 diff changeset	272 return
23785 cb99bacb9b4e branchcache: introduce revbranchcache for caching of revision branch names Mads Kiilerich <madski@unity3d.com> parents: 22357 diff changeset	273 rbcrevidx = rev * _rbcrecsize
51377 02e7d79edf62 branchmap: use mmap for faster revbranchcache loading Arseniy Alekseyev <aalekseyev@janestreet.com> parents: 51307 diff changeset	274 self._rbcrevs.pack_into(rbcrevidx, node, branchidx)
24376 203a078da052 revbranchcache: populate cache incrementally Durham Goode <durham@fb.com> parents: 24375 diff changeset	275 self._rbcrevslen = min(self._rbcrevslen, rev)
23785 cb99bacb9b4e branchcache: introduce revbranchcache for caching of revision branch names Mads Kiilerich <madski@unity3d.com> parents: 22357 diff changeset	276
24377 656f93ce66d5 revbranchcache: move cache writing to the transaction finalizer Durham Goode <durham@fb.com> parents: 24376 diff changeset	277 tr = self._repo.currenttransaction()
656f93ce66d5 revbranchcache: move cache writing to the transaction finalizer Durham Goode <durham@fb.com> parents: 24376 diff changeset	278 if tr:
43077 687b865b95ad formatting: byteify all mercurial/ and hgext/ string literals Augie Fackler <augie@google.com> parents: 43076 diff changeset	279 tr.addfinalize(b'write-revbranchcache', self.write)
24377 656f93ce66d5 revbranchcache: move cache writing to the transaction finalizer Durham Goode <durham@fb.com> parents: 24376 diff changeset	280
656f93ce66d5 revbranchcache: move cache writing to the transaction finalizer Durham Goode <durham@fb.com> parents: 24376 diff changeset	281 def write(self, tr=None):
23785 cb99bacb9b4e branchcache: introduce revbranchcache for caching of revision branch names Mads Kiilerich <madski@unity3d.com> parents: 22357 diff changeset	282 """Save branch cache if it is dirty."""
24374 77fd1fb538cd revbranchcache: store repo on the object Durham Goode <durham@fb.com> parents: 24373 diff changeset	283 repo = self._repo
29756 0d588332ad2c branchmap: acquires lock before writting the rev branch cache Pierre-Yves David <pierre-yves.david@ens-lyon.org> parents: 29755 diff changeset	284 wlock = None
43077 687b865b95ad formatting: byteify all mercurial/ and hgext/ string literals Augie Fackler <augie@google.com> parents: 43076 diff changeset	285 step = b''
29756 0d588332ad2c branchmap: acquires lock before writting the rev branch cache Pierre-Yves David <pierre-yves.david@ens-lyon.org> parents: 29755 diff changeset	286 try:
42222 09fd338522fa revbranchcache: factor logic to write names and revs in separate functions Pulkit Goyal <pulkit@yandex-team.ru> parents: 42138 diff changeset	287 # write the new names
29755 9f3c49ee4486 branchmap: preparatory indent of indent the branch rev writing code Pierre-Yves David <pierre-yves.david@ens-lyon.org> parents: 29615 diff changeset	288 if self._rbcnamescount < len(self._names):
29756 0d588332ad2c branchmap: acquires lock before writting the rev branch cache Pierre-Yves David <pierre-yves.david@ens-lyon.org> parents: 29755 diff changeset	289 wlock = repo.wlock(wait=False)
43077 687b865b95ad formatting: byteify all mercurial/ and hgext/ string literals Augie Fackler <augie@google.com> parents: 43076 diff changeset	290 step = b' names'
42222 09fd338522fa revbranchcache: factor logic to write names and revs in separate functions Pulkit Goyal <pulkit@yandex-team.ru> parents: 42138 diff changeset	291 self._writenames(repo)
23785 cb99bacb9b4e branchcache: introduce revbranchcache for caching of revision branch names Mads Kiilerich <madski@unity3d.com> parents: 22357 diff changeset	292
42222 09fd338522fa revbranchcache: factor logic to write names and revs in separate functions Pulkit Goyal <pulkit@yandex-team.ru> parents: 42138 diff changeset	293 # write the new revs
29755 9f3c49ee4486 branchmap: preparatory indent of indent the branch rev writing code Pierre-Yves David <pierre-yves.david@ens-lyon.org> parents: 29615 diff changeset	294 start = self._rbcrevslen * _rbcrecsize
9f3c49ee4486 branchmap: preparatory indent of indent the branch rev writing code Pierre-Yves David <pierre-yves.david@ens-lyon.org> parents: 29615 diff changeset	295 if start != len(self._rbcrevs):
43077 687b865b95ad formatting: byteify all mercurial/ and hgext/ string literals Augie Fackler <augie@google.com> parents: 43076 diff changeset	296 step = b''
29756 0d588332ad2c branchmap: acquires lock before writting the rev branch cache Pierre-Yves David <pierre-yves.david@ens-lyon.org> parents: 29755 diff changeset	297 if wlock is None:
0d588332ad2c branchmap: acquires lock before writting the rev branch cache Pierre-Yves David <pierre-yves.david@ens-lyon.org> parents: 29755 diff changeset	298 wlock = repo.wlock(wait=False)
42222 09fd338522fa revbranchcache: factor logic to write names and revs in separate functions Pulkit Goyal <pulkit@yandex-team.ru> parents: 42138 diff changeset	299 self._writerevs(repo, start)
09fd338522fa revbranchcache: factor logic to write names and revs in separate functions Pulkit Goyal <pulkit@yandex-team.ru> parents: 42138 diff changeset	300
29757 3b184adfb5be branchmap: simplify error handlind when writing rev branch cache Pierre-Yves David <pierre-yves.david@ens-lyon.org> parents: 29756 diff changeset	301 except (IOError, OSError, error.Abort, error.LockError) as inst:
43076 2372284d9457 formatting: blacken the codebase Augie Fackler <augie@google.com> parents: 42603 diff changeset	302 repo.ui.debug(
43077 687b865b95ad formatting: byteify all mercurial/ and hgext/ string literals Augie Fackler <augie@google.com> parents: 43076 diff changeset	303 b"couldn't write revision branch cache%s: %s\n"
43076 2372284d9457 formatting: blacken the codebase Augie Fackler <augie@google.com> parents: 42603 diff changeset	304 % (step, stringutil.forcebytestr(inst))
2372284d9457 formatting: blacken the codebase Augie Fackler <augie@google.com> parents: 42603 diff changeset	305 )
29756 0d588332ad2c branchmap: acquires lock before writting the rev branch cache Pierre-Yves David <pierre-yves.david@ens-lyon.org> parents: 29755 diff changeset	306 finally:
0d588332ad2c branchmap: acquires lock before writting the rev branch cache Pierre-Yves David <pierre-yves.david@ens-lyon.org> parents: 29755 diff changeset	307 if wlock is not None:
0d588332ad2c branchmap: acquires lock before writting the rev branch cache Pierre-Yves David <pierre-yves.david@ens-lyon.org> parents: 29755 diff changeset	308 wlock.release()
42222 09fd338522fa revbranchcache: factor logic to write names and revs in separate functions Pulkit Goyal <pulkit@yandex-team.ru> parents: 42138 diff changeset	309
09fd338522fa revbranchcache: factor logic to write names and revs in separate functions Pulkit Goyal <pulkit@yandex-team.ru> parents: 42138 diff changeset	310 def _writenames(self, repo):
47031 f38bf44e077f black: make codebase compatible with black v21.4b2 and v20.8b1 Kyle Lippincott <spectral@google.com> parents: 46819 diff changeset	311 """write the new branch names to revbranchcache"""
42222 09fd338522fa revbranchcache: factor logic to write names and revs in separate functions Pulkit Goyal <pulkit@yandex-team.ru> parents: 42138 diff changeset	312 if self._rbcnamescount != 0:
43077 687b865b95ad formatting: byteify all mercurial/ and hgext/ string literals Augie Fackler <augie@google.com> parents: 43076 diff changeset	313 f = repo.cachevfs.open(_rbcnames, b'ab')
42222 09fd338522fa revbranchcache: factor logic to write names and revs in separate functions Pulkit Goyal <pulkit@yandex-team.ru> parents: 42138 diff changeset	314 if f.tell() == self._rbcsnameslen:
43077 687b865b95ad formatting: byteify all mercurial/ and hgext/ string literals Augie Fackler <augie@google.com> parents: 43076 diff changeset	315 f.write(b'\0')
42222 09fd338522fa revbranchcache: factor logic to write names and revs in separate functions Pulkit Goyal <pulkit@yandex-team.ru> parents: 42138 diff changeset	316 else:
09fd338522fa revbranchcache: factor logic to write names and revs in separate functions Pulkit Goyal <pulkit@yandex-team.ru> parents: 42138 diff changeset	317 f.close()
43077 687b865b95ad formatting: byteify all mercurial/ and hgext/ string literals Augie Fackler <augie@google.com> parents: 43076 diff changeset	318 repo.ui.debug(b"%s changed - rewriting it\n" % _rbcnames)
42222 09fd338522fa revbranchcache: factor logic to write names and revs in separate functions Pulkit Goyal <pulkit@yandex-team.ru> parents: 42138 diff changeset	319 self._rbcnamescount = 0
09fd338522fa revbranchcache: factor logic to write names and revs in separate functions Pulkit Goyal <pulkit@yandex-team.ru> parents: 42138 diff changeset	320 self._rbcrevslen = 0
09fd338522fa revbranchcache: factor logic to write names and revs in separate functions Pulkit Goyal <pulkit@yandex-team.ru> parents: 42138 diff changeset	321 if self._rbcnamescount == 0:
09fd338522fa revbranchcache: factor logic to write names and revs in separate functions Pulkit Goyal <pulkit@yandex-team.ru> parents: 42138 diff changeset	322 # before rewriting names, make sure references are removed
09fd338522fa revbranchcache: factor logic to write names and revs in separate functions Pulkit Goyal <pulkit@yandex-team.ru> parents: 42138 diff changeset	323 repo.cachevfs.unlinkpath(_rbcrevs, ignoremissing=True)
43077 687b865b95ad formatting: byteify all mercurial/ and hgext/ string literals Augie Fackler <augie@google.com> parents: 43076 diff changeset	324 f = repo.cachevfs.open(_rbcnames, b'wb')
43076 2372284d9457 formatting: blacken the codebase Augie Fackler <augie@google.com> parents: 42603 diff changeset	325 f.write(
43077 687b865b95ad formatting: byteify all mercurial/ and hgext/ string literals Augie Fackler <augie@google.com> parents: 43076 diff changeset	326 b'\0'.join(
43076 2372284d9457 formatting: blacken the codebase Augie Fackler <augie@google.com> parents: 42603 diff changeset	327 encoding.fromlocal(b)
2372284d9457 formatting: blacken the codebase Augie Fackler <augie@google.com> parents: 42603 diff changeset	328 for b in self._names[self._rbcnamescount :]
2372284d9457 formatting: blacken the codebase Augie Fackler <augie@google.com> parents: 42603 diff changeset	329 )
2372284d9457 formatting: blacken the codebase Augie Fackler <augie@google.com> parents: 42603 diff changeset	330 )
42222 09fd338522fa revbranchcache: factor logic to write names and revs in separate functions Pulkit Goyal <pulkit@yandex-team.ru> parents: 42138 diff changeset	331 self._rbcsnameslen = f.tell()
09fd338522fa revbranchcache: factor logic to write names and revs in separate functions Pulkit Goyal <pulkit@yandex-team.ru> parents: 42138 diff changeset	332 f.close()
09fd338522fa revbranchcache: factor logic to write names and revs in separate functions Pulkit Goyal <pulkit@yandex-team.ru> parents: 42138 diff changeset	333 self._rbcnamescount = len(self._names)
09fd338522fa revbranchcache: factor logic to write names and revs in separate functions Pulkit Goyal <pulkit@yandex-team.ru> parents: 42138 diff changeset	334
09fd338522fa revbranchcache: factor logic to write names and revs in separate functions Pulkit Goyal <pulkit@yandex-team.ru> parents: 42138 diff changeset	335 def _writerevs(self, repo, start):
47031 f38bf44e077f black: make codebase compatible with black v21.4b2 and v20.8b1 Kyle Lippincott <spectral@google.com> parents: 46819 diff changeset	336 """write the new revs to revbranchcache"""
42223 ececa45c80d8 revbranchcache: use context manager in _writerevs() to write to file Pulkit Goyal <pulkit@yandex-team.ru> parents: 42222 diff changeset	337 revs = min(len(repo.changelog), len(self._rbcrevs) // _rbcrecsize)
43077 687b865b95ad formatting: byteify all mercurial/ and hgext/ string literals Augie Fackler <augie@google.com> parents: 43076 diff changeset	338 with repo.cachevfs.open(_rbcrevs, b'ab') as f:
51939 1eb2317c1762 rev-branch-cache: issue more truthful "truncating" message Pierre-Yves David <pierre-yves.david@octobus.net> parents: 51938 diff changeset	339 current_size = f.tell()
1eb2317c1762 rev-branch-cache: issue more truthful "truncating" message Pierre-Yves David <pierre-yves.david@octobus.net> parents: 51938 diff changeset	340 if current_size < start:
1eb2317c1762 rev-branch-cache: issue more truthful "truncating" message Pierre-Yves David <pierre-yves.david@octobus.net> parents: 51938 diff changeset	341 start = 0
1eb2317c1762 rev-branch-cache: issue more truthful "truncating" message Pierre-Yves David <pierre-yves.david@octobus.net> parents: 51938 diff changeset	342 if current_size != start:
1eb2317c1762 rev-branch-cache: issue more truthful "truncating" message Pierre-Yves David <pierre-yves.david@octobus.net> parents: 51938 diff changeset	343 msg = b"truncating cache/%s to %d\n"
1eb2317c1762 rev-branch-cache: issue more truthful "truncating" message Pierre-Yves David <pierre-yves.david@octobus.net> parents: 51938 diff changeset	344 msg %= (_rbcrevs, start)
1eb2317c1762 rev-branch-cache: issue more truthful "truncating" message Pierre-Yves David <pierre-yves.david@octobus.net> parents: 51938 diff changeset	345 repo.ui.debug(msg)
42222 09fd338522fa revbranchcache: factor logic to write names and revs in separate functions Pulkit Goyal <pulkit@yandex-team.ru> parents: 42138 diff changeset	346 f.seek(start)
42223 ececa45c80d8 revbranchcache: use context manager in _writerevs() to write to file Pulkit Goyal <pulkit@yandex-team.ru> parents: 42222 diff changeset	347 f.truncate()
ececa45c80d8 revbranchcache: use context manager in _writerevs() to write to file Pulkit Goyal <pulkit@yandex-team.ru> parents: 42222 diff changeset	348 end = revs * _rbcrecsize
51377 02e7d79edf62 branchmap: use mmap for faster revbranchcache loading Arseniy Alekseyev <aalekseyev@janestreet.com> parents: 51307 diff changeset	349 f.write(self._rbcrevs.slice(start, end))
42222 09fd338522fa revbranchcache: factor logic to write names and revs in separate functions Pulkit Goyal <pulkit@yandex-team.ru> parents: 42138 diff changeset	350 self._rbcrevslen = revs

Mercurial > public > mercurial-scm > hg-stable

annotate mercurial/branching/rev_cache.py @ 51939:1eb2317c1762