annotate mercurial/revlogutils/docket.py @ 48178:f12a19d03d2c

fix: reduce number of tool executions By grouping together (path, ctx) pairs according to the inputs they would provide to fixer tools, we can deduplicate executions of fixer tools to significantly reduce the amount of time spent running slow tools. This change does not handle clean files in the working copy, which could still be deduplicated against the files in the checked out commit. It's a little harder to do that because the filerev is not available in the workingfilectx (and it doesn't exist for added files). Anecdotally, this change makes some real uses cases at Google 10x faster. I think we were originally hesitant to do this because the benefits weren't obvious, and implementing it efficiently is kind of tricky. If we simply memoized the formatter execution function, we would be keeping tons of file content in memory. Also included is a regression test for a corner case that I broke with my first attempt at optimizing this code. Differential Revision: https://phab.mercurial-scm.org/D11280
author Danny Hooper <hooper@google.com>
date Thu, 02 Sep 2021 14:08:45 -0700
parents f93a867a6486
children 6000f5b25c9b
Ignore whitespace changes - Everywhere: Within whitespace: At end of lines:
rev   line source
47234
616b8f412676 revlogv2: introduce a very basic docket file
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
diff changeset
1 # docket - code related to revlog "docket"
616b8f412676 revlogv2: introduce a very basic docket file
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
diff changeset
2 #
616b8f412676 revlogv2: introduce a very basic docket file
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
diff changeset
3 # Copyright 2021 Pierre-Yves David <pierre-yves.david@octobus.net>
616b8f412676 revlogv2: introduce a very basic docket file
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
diff changeset
4 #
616b8f412676 revlogv2: introduce a very basic docket file
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
diff changeset
5 # This software may be used and distributed according to the terms of the
616b8f412676 revlogv2: introduce a very basic docket file
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
diff changeset
6 # GNU General Public License version 2 or any later version.
616b8f412676 revlogv2: introduce a very basic docket file
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
diff changeset
7
616b8f412676 revlogv2: introduce a very basic docket file
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
diff changeset
8 ### Revlog docket file
616b8f412676 revlogv2: introduce a very basic docket file
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
diff changeset
9 #
616b8f412676 revlogv2: introduce a very basic docket file
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
diff changeset
10 # The revlog is stored on disk using multiple files:
616b8f412676 revlogv2: introduce a very basic docket file
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
diff changeset
11 #
616b8f412676 revlogv2: introduce a very basic docket file
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
diff changeset
12 # * a small docket file, containing metadata and a pointer,
616b8f412676 revlogv2: introduce a very basic docket file
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
diff changeset
13 #
616b8f412676 revlogv2: introduce a very basic docket file
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
diff changeset
14 # * an index file, containing fixed width information about revisions,
616b8f412676 revlogv2: introduce a very basic docket file
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
diff changeset
15 #
616b8f412676 revlogv2: introduce a very basic docket file
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
diff changeset
16 # * a data file, containing variable width data for these revisions,
616b8f412676 revlogv2: introduce a very basic docket file
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
diff changeset
17
616b8f412676 revlogv2: introduce a very basic docket file
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
diff changeset
18 from __future__ import absolute_import
616b8f412676 revlogv2: introduce a very basic docket file
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
diff changeset
19
47312
7ea39d633cf3 docket: move the uid logic in the `revlogutils.docket` module
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 47266
diff changeset
20 import errno
7ea39d633cf3 docket: move the uid logic in the `revlogutils.docket` module
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 47266
diff changeset
21 import os
7ea39d633cf3 docket: move the uid logic in the `revlogutils.docket` module
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 47266
diff changeset
22 import random
47234
616b8f412676 revlogv2: introduce a very basic docket file
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
diff changeset
23 import struct
616b8f412676 revlogv2: introduce a very basic docket file
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
diff changeset
24
47241
2219853a1503 revlogv2: track pending write in the docket and expose it to hooks
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 47238
diff changeset
25 from .. import (
47312
7ea39d633cf3 docket: move the uid logic in the `revlogutils.docket` module
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 47266
diff changeset
26 encoding,
47241
2219853a1503 revlogv2: track pending write in the docket and expose it to hooks
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 47238
diff changeset
27 error,
47312
7ea39d633cf3 docket: move the uid logic in the `revlogutils.docket` module
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 47266
diff changeset
28 node,
7ea39d633cf3 docket: move the uid logic in the `revlogutils.docket` module
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 47266
diff changeset
29 pycompat,
47255
ff9fd7107d11 revlog: implement a "default compression" mode
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 47242
diff changeset
30 util,
47241
2219853a1503 revlogv2: track pending write in the docket and expose it to hooks
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 47238
diff changeset
31 )
2219853a1503 revlogv2: track pending write in the docket and expose it to hooks
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 47238
diff changeset
32
47234
616b8f412676 revlogv2: introduce a very basic docket file
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
diff changeset
33 from . import (
616b8f412676 revlogv2: introduce a very basic docket file
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
diff changeset
34 constants,
616b8f412676 revlogv2: introduce a very basic docket file
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
diff changeset
35 )
616b8f412676 revlogv2: introduce a very basic docket file
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
diff changeset
36
47312
7ea39d633cf3 docket: move the uid logic in the `revlogutils.docket` module
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 47266
diff changeset
37
7ea39d633cf3 docket: move the uid logic in the `revlogutils.docket` module
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 47266
diff changeset
38 def make_uid(id_size=8):
7ea39d633cf3 docket: move the uid logic in the `revlogutils.docket` module
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 47266
diff changeset
39 """return a new unique identifier.
7ea39d633cf3 docket: move the uid logic in the `revlogutils.docket` module
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 47266
diff changeset
40
7ea39d633cf3 docket: move the uid logic in the `revlogutils.docket` module
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 47266
diff changeset
41 The identifier is random and composed of ascii characters."""
7ea39d633cf3 docket: move the uid logic in the `revlogutils.docket` module
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 47266
diff changeset
42 # size we "hex" the result we need half the number of bits to have a final
7ea39d633cf3 docket: move the uid logic in the `revlogutils.docket` module
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 47266
diff changeset
43 # uuid of size ID_SIZE
7ea39d633cf3 docket: move the uid logic in the `revlogutils.docket` module
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 47266
diff changeset
44 return node.hex(os.urandom(id_size // 2))
7ea39d633cf3 docket: move the uid logic in the `revlogutils.docket` module
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 47266
diff changeset
45
7ea39d633cf3 docket: move the uid logic in the `revlogutils.docket` module
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 47266
diff changeset
46
7ea39d633cf3 docket: move the uid logic in the `revlogutils.docket` module
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 47266
diff changeset
47 # some special test logic to avoid anoying random output in the test
7ea39d633cf3 docket: move the uid logic in the `revlogutils.docket` module
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 47266
diff changeset
48 stable_docket_file = encoding.environ.get(b'HGTEST_UUIDFILE')
7ea39d633cf3 docket: move the uid logic in the `revlogutils.docket` module
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 47266
diff changeset
49
7ea39d633cf3 docket: move the uid logic in the `revlogutils.docket` module
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 47266
diff changeset
50 if stable_docket_file:
7ea39d633cf3 docket: move the uid logic in the `revlogutils.docket` module
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 47266
diff changeset
51
7ea39d633cf3 docket: move the uid logic in the `revlogutils.docket` module
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 47266
diff changeset
52 def make_uid(id_size=8):
7ea39d633cf3 docket: move the uid logic in the `revlogutils.docket` module
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 47266
diff changeset
53 try:
7ea39d633cf3 docket: move the uid logic in the `revlogutils.docket` module
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 47266
diff changeset
54 with open(stable_docket_file, mode='rb') as f:
7ea39d633cf3 docket: move the uid logic in the `revlogutils.docket` module
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 47266
diff changeset
55 seed = f.read().strip()
7ea39d633cf3 docket: move the uid logic in the `revlogutils.docket` module
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 47266
diff changeset
56 except IOError as inst:
7ea39d633cf3 docket: move the uid logic in the `revlogutils.docket` module
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 47266
diff changeset
57 if inst.errno != errno.ENOENT:
7ea39d633cf3 docket: move the uid logic in the `revlogutils.docket` module
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 47266
diff changeset
58 raise
7ea39d633cf3 docket: move the uid logic in the `revlogutils.docket` module
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 47266
diff changeset
59 seed = b'04' # chosen by a fair dice roll. garanteed to be random
7ea39d633cf3 docket: move the uid logic in the `revlogutils.docket` module
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 47266
diff changeset
60 if pycompat.ispy3:
7ea39d633cf3 docket: move the uid logic in the `revlogutils.docket` module
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 47266
diff changeset
61 iter_seed = iter(seed)
7ea39d633cf3 docket: move the uid logic in the `revlogutils.docket` module
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 47266
diff changeset
62 else:
47423
be903d043099 typing: suppress a false error in mercurial/revlogutils/docket.py on py2
Matt Harbison <matt_harbison@yahoo.com>
parents: 47389
diff changeset
63 # pytype: disable=wrong-arg-types
47312
7ea39d633cf3 docket: move the uid logic in the `revlogutils.docket` module
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 47266
diff changeset
64 iter_seed = (ord(c) for c in seed)
47423
be903d043099 typing: suppress a false error in mercurial/revlogutils/docket.py on py2
Matt Harbison <matt_harbison@yahoo.com>
parents: 47389
diff changeset
65 # pytype: enable=wrong-arg-types
47312
7ea39d633cf3 docket: move the uid logic in the `revlogutils.docket` module
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 47266
diff changeset
66 # some basic circular sum hashing on 64 bits
7ea39d633cf3 docket: move the uid logic in the `revlogutils.docket` module
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 47266
diff changeset
67 int_seed = 0
7ea39d633cf3 docket: move the uid logic in the `revlogutils.docket` module
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 47266
diff changeset
68 low_mask = int('1' * 35, 2)
7ea39d633cf3 docket: move the uid logic in the `revlogutils.docket` module
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 47266
diff changeset
69 for i in iter_seed:
7ea39d633cf3 docket: move the uid logic in the `revlogutils.docket` module
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 47266
diff changeset
70 high_part = int_seed >> 35
7ea39d633cf3 docket: move the uid logic in the `revlogutils.docket` module
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 47266
diff changeset
71 low_part = (int_seed & low_mask) << 28
7ea39d633cf3 docket: move the uid logic in the `revlogutils.docket` module
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 47266
diff changeset
72 int_seed = high_part + low_part + i
7ea39d633cf3 docket: move the uid logic in the `revlogutils.docket` module
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 47266
diff changeset
73 r = random.Random()
7ea39d633cf3 docket: move the uid logic in the `revlogutils.docket` module
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 47266
diff changeset
74 if pycompat.ispy3:
7ea39d633cf3 docket: move the uid logic in the `revlogutils.docket` module
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 47266
diff changeset
75 r.seed(int_seed, version=1)
7ea39d633cf3 docket: move the uid logic in the `revlogutils.docket` module
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 47266
diff changeset
76 else:
7ea39d633cf3 docket: move the uid logic in the `revlogutils.docket` module
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 47266
diff changeset
77 r.seed(int_seed)
7ea39d633cf3 docket: move the uid logic in the `revlogutils.docket` module
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 47266
diff changeset
78 # once we drop python 3.8 support we can simply use r.randbytes
7ea39d633cf3 docket: move the uid logic in the `revlogutils.docket` module
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 47266
diff changeset
79 raw = r.getrandbits(id_size * 4)
7ea39d633cf3 docket: move the uid logic in the `revlogutils.docket` module
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 47266
diff changeset
80 assert id_size == 8
7ea39d633cf3 docket: move the uid logic in the `revlogutils.docket` module
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 47266
diff changeset
81 p = struct.pack('>L', raw)
7ea39d633cf3 docket: move the uid logic in the `revlogutils.docket` module
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 47266
diff changeset
82 new = node.hex(p)
7ea39d633cf3 docket: move the uid logic in the `revlogutils.docket` module
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 47266
diff changeset
83 with open(stable_docket_file, 'wb') as f:
7ea39d633cf3 docket: move the uid logic in the `revlogutils.docket` module
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 47266
diff changeset
84 f.write(new)
7ea39d633cf3 docket: move the uid logic in the `revlogutils.docket` module
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 47266
diff changeset
85 return new
7ea39d633cf3 docket: move the uid logic in the `revlogutils.docket` module
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 47266
diff changeset
86
7ea39d633cf3 docket: move the uid logic in the `revlogutils.docket` module
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 47266
diff changeset
87
47234
616b8f412676 revlogv2: introduce a very basic docket file
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
diff changeset
88 # Docket format
616b8f412676 revlogv2: introduce a very basic docket file
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
diff changeset
89 #
616b8f412676 revlogv2: introduce a very basic docket file
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
diff changeset
90 # * 4 bytes: revlog version
616b8f412676 revlogv2: introduce a very basic docket file
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
diff changeset
91 # | This is mandatory as docket must be compatible with the previous
616b8f412676 revlogv2: introduce a very basic docket file
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
diff changeset
92 # | revlog index header.
47323
f612db768c7a revlogv2: use a unique filename for index
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 47312
diff changeset
93 # * 1 bytes: size of index uuid
47453
c252631500e4 revlog: add a way to keep track of older uids in the docket
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 47423
diff changeset
94 # * 1 bytes: number of outdated index uuid
47324
0a3fa41fa719 revlogv2: use a unique filename for data
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 47323
diff changeset
95 # * 1 bytes: size of data uuid
47453
c252631500e4 revlog: add a way to keep track of older uids in the docket
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 47423
diff changeset
96 # * 1 bytes: number of outdated data uuid
47389
e6292eb33384 revlog: store sidedata in their own file
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 47388
diff changeset
97 # * 1 bytes: size of sizedata uuid
47453
c252631500e4 revlog: add a way to keep track of older uids in the docket
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 47423
diff changeset
98 # * 1 bytes: number of outdated data uuid
47242
4abd474a10af revlogv2: also keep track for the size of the "data" file
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 47241
diff changeset
99 # * 8 bytes: size of index-data
4abd474a10af revlogv2: also keep track for the size of the "data" file
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 47241
diff changeset
100 # * 8 bytes: pending size of index-data
4abd474a10af revlogv2: also keep track for the size of the "data" file
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 47241
diff changeset
101 # * 8 bytes: size of data
47389
e6292eb33384 revlog: store sidedata in their own file
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 47388
diff changeset
102 # * 8 bytes: size of sidedata
47242
4abd474a10af revlogv2: also keep track for the size of the "data" file
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 47241
diff changeset
103 # * 8 bytes: pending size of data
47389
e6292eb33384 revlog: store sidedata in their own file
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 47388
diff changeset
104 # * 8 bytes: pending size of sidedata
47255
ff9fd7107d11 revlog: implement a "default compression" mode
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 47242
diff changeset
105 # * 1 bytes: default compression header
47453
c252631500e4 revlog: add a way to keep track of older uids in the docket
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 47423
diff changeset
106 S_HEADER = struct.Struct(constants.INDEX_HEADER_FMT + b'BBBBBBLLLLLLc')
c252631500e4 revlog: add a way to keep track of older uids in the docket
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 47423
diff changeset
107 # * 1 bytes: size of index uuid
c252631500e4 revlog: add a way to keep track of older uids in the docket
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 47423
diff changeset
108 # * 8 bytes: size of file
c252631500e4 revlog: add a way to keep track of older uids in the docket
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 47423
diff changeset
109 S_OLD_UID = struct.Struct('>BL')
47234
616b8f412676 revlogv2: introduce a very basic docket file
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
diff changeset
110
616b8f412676 revlogv2: introduce a very basic docket file
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
diff changeset
111
616b8f412676 revlogv2: introduce a very basic docket file
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
diff changeset
112 class RevlogDocket(object):
616b8f412676 revlogv2: introduce a very basic docket file
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
diff changeset
113 """metadata associated with revlog"""
616b8f412676 revlogv2: introduce a very basic docket file
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
diff changeset
114
47241
2219853a1503 revlogv2: track pending write in the docket and expose it to hooks
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 47238
diff changeset
115 def __init__(
2219853a1503 revlogv2: track pending write in the docket and expose it to hooks
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 47238
diff changeset
116 self,
2219853a1503 revlogv2: track pending write in the docket and expose it to hooks
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 47238
diff changeset
117 revlog,
2219853a1503 revlogv2: track pending write in the docket and expose it to hooks
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 47238
diff changeset
118 use_pending=False,
2219853a1503 revlogv2: track pending write in the docket and expose it to hooks
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 47238
diff changeset
119 version_header=None,
47323
f612db768c7a revlogv2: use a unique filename for index
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 47312
diff changeset
120 index_uuid=None,
47453
c252631500e4 revlog: add a way to keep track of older uids in the docket
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 47423
diff changeset
121 older_index_uuids=(),
47324
0a3fa41fa719 revlogv2: use a unique filename for data
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 47323
diff changeset
122 data_uuid=None,
47453
c252631500e4 revlog: add a way to keep track of older uids in the docket
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 47423
diff changeset
123 older_data_uuids=(),
47389
e6292eb33384 revlog: store sidedata in their own file
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 47388
diff changeset
124 sidedata_uuid=None,
47453
c252631500e4 revlog: add a way to keep track of older uids in the docket
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 47423
diff changeset
125 older_sidedata_uuids=(),
47241
2219853a1503 revlogv2: track pending write in the docket and expose it to hooks
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 47238
diff changeset
126 index_end=0,
2219853a1503 revlogv2: track pending write in the docket and expose it to hooks
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 47238
diff changeset
127 pending_index_end=0,
47242
4abd474a10af revlogv2: also keep track for the size of the "data" file
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 47241
diff changeset
128 data_end=0,
4abd474a10af revlogv2: also keep track for the size of the "data" file
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 47241
diff changeset
129 pending_data_end=0,
47389
e6292eb33384 revlog: store sidedata in their own file
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 47388
diff changeset
130 sidedata_end=0,
e6292eb33384 revlog: store sidedata in their own file
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 47388
diff changeset
131 pending_sidedata_end=0,
47255
ff9fd7107d11 revlog: implement a "default compression" mode
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 47242
diff changeset
132 default_compression_header=None,
47241
2219853a1503 revlogv2: track pending write in the docket and expose it to hooks
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 47238
diff changeset
133 ):
47234
616b8f412676 revlogv2: introduce a very basic docket file
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
diff changeset
134 self._version_header = version_header
47241
2219853a1503 revlogv2: track pending write in the docket and expose it to hooks
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 47238
diff changeset
135 self._read_only = bool(use_pending)
47234
616b8f412676 revlogv2: introduce a very basic docket file
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
diff changeset
136 self._dirty = False
616b8f412676 revlogv2: introduce a very basic docket file
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
diff changeset
137 self._radix = revlog.radix
616b8f412676 revlogv2: introduce a very basic docket file
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
diff changeset
138 self._path = revlog._docket_file
616b8f412676 revlogv2: introduce a very basic docket file
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
diff changeset
139 self._opener = revlog.opener
47323
f612db768c7a revlogv2: use a unique filename for index
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 47312
diff changeset
140 self._index_uuid = index_uuid
47453
c252631500e4 revlog: add a way to keep track of older uids in the docket
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 47423
diff changeset
141 self._older_index_uuids = older_index_uuids
47324
0a3fa41fa719 revlogv2: use a unique filename for data
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 47323
diff changeset
142 self._data_uuid = data_uuid
47453
c252631500e4 revlog: add a way to keep track of older uids in the docket
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 47423
diff changeset
143 self._older_data_uuids = older_data_uuids
47389
e6292eb33384 revlog: store sidedata in their own file
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 47388
diff changeset
144 self._sidedata_uuid = sidedata_uuid
47453
c252631500e4 revlog: add a way to keep track of older uids in the docket
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 47423
diff changeset
145 self._older_sidedata_uuids = older_sidedata_uuids
c252631500e4 revlog: add a way to keep track of older uids in the docket
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 47423
diff changeset
146 assert not set(older_index_uuids) & set(older_data_uuids)
c252631500e4 revlog: add a way to keep track of older uids in the docket
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 47423
diff changeset
147 assert not set(older_data_uuids) & set(older_sidedata_uuids)
c252631500e4 revlog: add a way to keep track of older uids in the docket
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 47423
diff changeset
148 assert not set(older_index_uuids) & set(older_sidedata_uuids)
47242
4abd474a10af revlogv2: also keep track for the size of the "data" file
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 47241
diff changeset
149 # thes asserts should be True as long as we have a single index filename
47241
2219853a1503 revlogv2: track pending write in the docket and expose it to hooks
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 47238
diff changeset
150 assert index_end <= pending_index_end
47242
4abd474a10af revlogv2: also keep track for the size of the "data" file
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 47241
diff changeset
151 assert data_end <= pending_data_end
47389
e6292eb33384 revlog: store sidedata in their own file
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 47388
diff changeset
152 assert sidedata_end <= pending_sidedata_end
47241
2219853a1503 revlogv2: track pending write in the docket and expose it to hooks
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 47238
diff changeset
153 self._initial_index_end = index_end
2219853a1503 revlogv2: track pending write in the docket and expose it to hooks
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 47238
diff changeset
154 self._pending_index_end = pending_index_end
47242
4abd474a10af revlogv2: also keep track for the size of the "data" file
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 47241
diff changeset
155 self._initial_data_end = data_end
4abd474a10af revlogv2: also keep track for the size of the "data" file
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 47241
diff changeset
156 self._pending_data_end = pending_data_end
47389
e6292eb33384 revlog: store sidedata in their own file
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 47388
diff changeset
157 self._initial_sidedata_end = sidedata_end
e6292eb33384 revlog: store sidedata in their own file
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 47388
diff changeset
158 self._pending_sidedata_end = pending_sidedata_end
47241
2219853a1503 revlogv2: track pending write in the docket and expose it to hooks
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 47238
diff changeset
159 if use_pending:
2219853a1503 revlogv2: track pending write in the docket and expose it to hooks
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 47238
diff changeset
160 self._index_end = self._pending_index_end
47242
4abd474a10af revlogv2: also keep track for the size of the "data" file
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 47241
diff changeset
161 self._data_end = self._pending_data_end
47389
e6292eb33384 revlog: store sidedata in their own file
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 47388
diff changeset
162 self._sidedata_end = self._pending_sidedata_end
47241
2219853a1503 revlogv2: track pending write in the docket and expose it to hooks
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 47238
diff changeset
163 else:
2219853a1503 revlogv2: track pending write in the docket and expose it to hooks
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 47238
diff changeset
164 self._index_end = self._initial_index_end
47242
4abd474a10af revlogv2: also keep track for the size of the "data" file
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 47241
diff changeset
165 self._data_end = self._initial_data_end
47389
e6292eb33384 revlog: store sidedata in their own file
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 47388
diff changeset
166 self._sidedata_end = self._initial_sidedata_end
47255
ff9fd7107d11 revlog: implement a "default compression" mode
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 47242
diff changeset
167 self.default_compression_header = default_compression_header
47234
616b8f412676 revlogv2: introduce a very basic docket file
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
diff changeset
168
616b8f412676 revlogv2: introduce a very basic docket file
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
diff changeset
169 def index_filepath(self):
616b8f412676 revlogv2: introduce a very basic docket file
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
diff changeset
170 """file path to the current index file associated to this docket"""
616b8f412676 revlogv2: introduce a very basic docket file
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
diff changeset
171 # very simplistic version at first
47323
f612db768c7a revlogv2: use a unique filename for index
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 47312
diff changeset
172 if self._index_uuid is None:
f612db768c7a revlogv2: use a unique filename for index
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 47312
diff changeset
173 self._index_uuid = make_uid()
f612db768c7a revlogv2: use a unique filename for index
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 47312
diff changeset
174 return b"%s-%s.idx" % (self._radix, self._index_uuid)
47234
616b8f412676 revlogv2: introduce a very basic docket file
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
diff changeset
175
47454
865c260d7163 revlog: add docket method to request new content files
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 47453
diff changeset
176 def new_index_file(self):
865c260d7163 revlog: add docket method to request new content files
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 47453
diff changeset
177 """switch index file to a new UID
865c260d7163 revlog: add docket method to request new content files
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 47453
diff changeset
178
865c260d7163 revlog: add docket method to request new content files
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 47453
diff changeset
179 The previous index UID is moved to the "older" list."""
865c260d7163 revlog: add docket method to request new content files
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 47453
diff changeset
180 old = (self._index_uuid, self._index_end)
865c260d7163 revlog: add docket method to request new content files
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 47453
diff changeset
181 self._older_index_uuids.insert(0, old)
865c260d7163 revlog: add docket method to request new content files
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 47453
diff changeset
182 self._index_uuid = make_uid()
865c260d7163 revlog: add docket method to request new content files
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 47453
diff changeset
183 return self.index_filepath()
865c260d7163 revlog: add docket method to request new content files
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 47453
diff changeset
184
47455
f93a867a6486 revlog: list older-but-still-around file in `files`
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 47454
diff changeset
185 def old_index_filepaths(self, include_empty=True):
f93a867a6486 revlog: list older-but-still-around file in `files`
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 47454
diff changeset
186 """yield file path to older index files associated to this docket"""
f93a867a6486 revlog: list older-but-still-around file in `files`
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 47454
diff changeset
187 # very simplistic version at first
f93a867a6486 revlog: list older-but-still-around file in `files`
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 47454
diff changeset
188 for uuid, size in self._older_index_uuids:
f93a867a6486 revlog: list older-but-still-around file in `files`
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 47454
diff changeset
189 if include_empty or size > 0:
f93a867a6486 revlog: list older-but-still-around file in `files`
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 47454
diff changeset
190 yield b"%s-%s.idx" % (self._radix, uuid)
f93a867a6486 revlog: list older-but-still-around file in `files`
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 47454
diff changeset
191
47324
0a3fa41fa719 revlogv2: use a unique filename for data
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 47323
diff changeset
192 def data_filepath(self):
47388
bcf92bdc2bca revlog: fix docket.date_filepath docstring
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 47371
diff changeset
193 """file path to the current data file associated to this docket"""
47324
0a3fa41fa719 revlogv2: use a unique filename for data
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 47323
diff changeset
194 # very simplistic version at first
0a3fa41fa719 revlogv2: use a unique filename for data
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 47323
diff changeset
195 if self._data_uuid is None:
0a3fa41fa719 revlogv2: use a unique filename for data
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 47323
diff changeset
196 self._data_uuid = make_uid()
0a3fa41fa719 revlogv2: use a unique filename for data
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 47323
diff changeset
197 return b"%s-%s.dat" % (self._radix, self._data_uuid)
0a3fa41fa719 revlogv2: use a unique filename for data
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 47323
diff changeset
198
47454
865c260d7163 revlog: add docket method to request new content files
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 47453
diff changeset
199 def new_data_file(self):
865c260d7163 revlog: add docket method to request new content files
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 47453
diff changeset
200 """switch data file to a new UID
865c260d7163 revlog: add docket method to request new content files
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 47453
diff changeset
201
865c260d7163 revlog: add docket method to request new content files
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 47453
diff changeset
202 The previous data UID is moved to the "older" list."""
865c260d7163 revlog: add docket method to request new content files
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 47453
diff changeset
203 old = (self._data_uuid, self._data_end)
865c260d7163 revlog: add docket method to request new content files
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 47453
diff changeset
204 self._older_data_uuids.insert(0, old)
865c260d7163 revlog: add docket method to request new content files
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 47453
diff changeset
205 self._data_uuid = make_uid()
865c260d7163 revlog: add docket method to request new content files
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 47453
diff changeset
206 return self.data_filepath()
865c260d7163 revlog: add docket method to request new content files
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 47453
diff changeset
207
47455
f93a867a6486 revlog: list older-but-still-around file in `files`
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 47454
diff changeset
208 def old_data_filepaths(self, include_empty=True):
f93a867a6486 revlog: list older-but-still-around file in `files`
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 47454
diff changeset
209 """yield file path to older data files associated to this docket"""
f93a867a6486 revlog: list older-but-still-around file in `files`
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 47454
diff changeset
210 # very simplistic version at first
f93a867a6486 revlog: list older-but-still-around file in `files`
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 47454
diff changeset
211 for uuid, size in self._older_data_uuids:
f93a867a6486 revlog: list older-but-still-around file in `files`
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 47454
diff changeset
212 if include_empty or size > 0:
f93a867a6486 revlog: list older-but-still-around file in `files`
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 47454
diff changeset
213 yield b"%s-%s.dat" % (self._radix, uuid)
f93a867a6486 revlog: list older-but-still-around file in `files`
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 47454
diff changeset
214
47389
e6292eb33384 revlog: store sidedata in their own file
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 47388
diff changeset
215 def sidedata_filepath(self):
e6292eb33384 revlog: store sidedata in their own file
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 47388
diff changeset
216 """file path to the current sidedata file associated to this docket"""
e6292eb33384 revlog: store sidedata in their own file
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 47388
diff changeset
217 # very simplistic version at first
e6292eb33384 revlog: store sidedata in their own file
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 47388
diff changeset
218 if self._sidedata_uuid is None:
e6292eb33384 revlog: store sidedata in their own file
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 47388
diff changeset
219 self._sidedata_uuid = make_uid()
e6292eb33384 revlog: store sidedata in their own file
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 47388
diff changeset
220 return b"%s-%s.sda" % (self._radix, self._sidedata_uuid)
e6292eb33384 revlog: store sidedata in their own file
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 47388
diff changeset
221
47454
865c260d7163 revlog: add docket method to request new content files
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 47453
diff changeset
222 def new_sidedata_file(self):
865c260d7163 revlog: add docket method to request new content files
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 47453
diff changeset
223 """switch sidedata file to a new UID
865c260d7163 revlog: add docket method to request new content files
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 47453
diff changeset
224
865c260d7163 revlog: add docket method to request new content files
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 47453
diff changeset
225 The previous sidedata UID is moved to the "older" list."""
865c260d7163 revlog: add docket method to request new content files
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 47453
diff changeset
226 old = (self._sidedata_uuid, self._sidedata_end)
865c260d7163 revlog: add docket method to request new content files
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 47453
diff changeset
227 self._older_sidedata_uuids.insert(0, old)
865c260d7163 revlog: add docket method to request new content files
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 47453
diff changeset
228 self._sidedata_uuid = make_uid()
865c260d7163 revlog: add docket method to request new content files
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 47453
diff changeset
229 return self.sidedata_filepath()
865c260d7163 revlog: add docket method to request new content files
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 47453
diff changeset
230
47455
f93a867a6486 revlog: list older-but-still-around file in `files`
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 47454
diff changeset
231 def old_sidedata_filepaths(self, include_empty=True):
f93a867a6486 revlog: list older-but-still-around file in `files`
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 47454
diff changeset
232 """yield file path to older sidedata files associated to this docket"""
f93a867a6486 revlog: list older-but-still-around file in `files`
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 47454
diff changeset
233 # very simplistic version at first
f93a867a6486 revlog: list older-but-still-around file in `files`
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 47454
diff changeset
234 for uuid, size in self._older_sidedata_uuids:
f93a867a6486 revlog: list older-but-still-around file in `files`
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 47454
diff changeset
235 if include_empty or size > 0:
f93a867a6486 revlog: list older-but-still-around file in `files`
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 47454
diff changeset
236 yield b"%s-%s.sda" % (self._radix, uuid)
f93a867a6486 revlog: list older-but-still-around file in `files`
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 47454
diff changeset
237
47238
6597255a4f94 revlogv2: track current index size in the docket
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 47234
diff changeset
238 @property
6597255a4f94 revlogv2: track current index size in the docket
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 47234
diff changeset
239 def index_end(self):
6597255a4f94 revlogv2: track current index size in the docket
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 47234
diff changeset
240 return self._index_end
6597255a4f94 revlogv2: track current index size in the docket
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 47234
diff changeset
241
6597255a4f94 revlogv2: track current index size in the docket
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 47234
diff changeset
242 @index_end.setter
6597255a4f94 revlogv2: track current index size in the docket
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 47234
diff changeset
243 def index_end(self, new_size):
6597255a4f94 revlogv2: track current index size in the docket
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 47234
diff changeset
244 if new_size != self._index_end:
6597255a4f94 revlogv2: track current index size in the docket
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 47234
diff changeset
245 self._index_end = new_size
6597255a4f94 revlogv2: track current index size in the docket
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 47234
diff changeset
246 self._dirty = True
6597255a4f94 revlogv2: track current index size in the docket
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 47234
diff changeset
247
47242
4abd474a10af revlogv2: also keep track for the size of the "data" file
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 47241
diff changeset
248 @property
4abd474a10af revlogv2: also keep track for the size of the "data" file
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 47241
diff changeset
249 def data_end(self):
4abd474a10af revlogv2: also keep track for the size of the "data" file
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 47241
diff changeset
250 return self._data_end
4abd474a10af revlogv2: also keep track for the size of the "data" file
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 47241
diff changeset
251
4abd474a10af revlogv2: also keep track for the size of the "data" file
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 47241
diff changeset
252 @data_end.setter
4abd474a10af revlogv2: also keep track for the size of the "data" file
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 47241
diff changeset
253 def data_end(self, new_size):
4abd474a10af revlogv2: also keep track for the size of the "data" file
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 47241
diff changeset
254 if new_size != self._data_end:
4abd474a10af revlogv2: also keep track for the size of the "data" file
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 47241
diff changeset
255 self._data_end = new_size
4abd474a10af revlogv2: also keep track for the size of the "data" file
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 47241
diff changeset
256 self._dirty = True
4abd474a10af revlogv2: also keep track for the size of the "data" file
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 47241
diff changeset
257
47389
e6292eb33384 revlog: store sidedata in their own file
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 47388
diff changeset
258 @property
e6292eb33384 revlog: store sidedata in their own file
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 47388
diff changeset
259 def sidedata_end(self):
e6292eb33384 revlog: store sidedata in their own file
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 47388
diff changeset
260 return self._sidedata_end
e6292eb33384 revlog: store sidedata in their own file
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 47388
diff changeset
261
e6292eb33384 revlog: store sidedata in their own file
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 47388
diff changeset
262 @sidedata_end.setter
e6292eb33384 revlog: store sidedata in their own file
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 47388
diff changeset
263 def sidedata_end(self, new_size):
e6292eb33384 revlog: store sidedata in their own file
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 47388
diff changeset
264 if new_size != self._sidedata_end:
e6292eb33384 revlog: store sidedata in their own file
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 47388
diff changeset
265 self._sidedata_end = new_size
e6292eb33384 revlog: store sidedata in their own file
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 47388
diff changeset
266 self._dirty = True
e6292eb33384 revlog: store sidedata in their own file
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 47388
diff changeset
267
47241
2219853a1503 revlogv2: track pending write in the docket and expose it to hooks
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 47238
diff changeset
268 def write(self, transaction, pending=False, stripping=False):
47234
616b8f412676 revlogv2: introduce a very basic docket file
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
diff changeset
269 """write the modification of disk if any
616b8f412676 revlogv2: introduce a very basic docket file
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
diff changeset
270
616b8f412676 revlogv2: introduce a very basic docket file
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
diff changeset
271 This make the new content visible to all process"""
47241
2219853a1503 revlogv2: track pending write in the docket and expose it to hooks
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 47238
diff changeset
272 if not self._dirty:
2219853a1503 revlogv2: track pending write in the docket and expose it to hooks
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 47238
diff changeset
273 return False
2219853a1503 revlogv2: track pending write in the docket and expose it to hooks
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 47238
diff changeset
274 else:
2219853a1503 revlogv2: track pending write in the docket and expose it to hooks
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 47238
diff changeset
275 if self._read_only:
2219853a1503 revlogv2: track pending write in the docket and expose it to hooks
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 47238
diff changeset
276 msg = b'writing read-only docket: %s'
2219853a1503 revlogv2: track pending write in the docket and expose it to hooks
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 47238
diff changeset
277 msg %= self._path
2219853a1503 revlogv2: track pending write in the docket and expose it to hooks
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 47238
diff changeset
278 raise error.ProgrammingError(msg)
47238
6597255a4f94 revlogv2: track current index size in the docket
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 47234
diff changeset
279 if not stripping:
6597255a4f94 revlogv2: track current index size in the docket
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 47234
diff changeset
280 # XXX we could, leverage the docket while stripping. However it
6597255a4f94 revlogv2: track current index size in the docket
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 47234
diff changeset
281 # is not powerfull enough at the time of this comment
6597255a4f94 revlogv2: track current index size in the docket
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 47234
diff changeset
282 transaction.addbackup(self._path, location=b'store')
47234
616b8f412676 revlogv2: introduce a very basic docket file
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
diff changeset
283 with self._opener(self._path, mode=b'w', atomictemp=True) as f:
47241
2219853a1503 revlogv2: track pending write in the docket and expose it to hooks
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 47238
diff changeset
284 f.write(self._serialize(pending=pending))
2219853a1503 revlogv2: track pending write in the docket and expose it to hooks
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 47238
diff changeset
285 # if pending we still need to the write final data eventually
2219853a1503 revlogv2: track pending write in the docket and expose it to hooks
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 47238
diff changeset
286 self._dirty = pending
2219853a1503 revlogv2: track pending write in the docket and expose it to hooks
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 47238
diff changeset
287 return True
47234
616b8f412676 revlogv2: introduce a very basic docket file
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
diff changeset
288
47241
2219853a1503 revlogv2: track pending write in the docket and expose it to hooks
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 47238
diff changeset
289 def _serialize(self, pending=False):
2219853a1503 revlogv2: track pending write in the docket and expose it to hooks
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 47238
diff changeset
290 if pending:
2219853a1503 revlogv2: track pending write in the docket and expose it to hooks
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 47238
diff changeset
291 official_index_end = self._initial_index_end
47242
4abd474a10af revlogv2: also keep track for the size of the "data" file
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 47241
diff changeset
292 official_data_end = self._initial_data_end
47389
e6292eb33384 revlog: store sidedata in their own file
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 47388
diff changeset
293 official_sidedata_end = self._initial_sidedata_end
47241
2219853a1503 revlogv2: track pending write in the docket and expose it to hooks
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 47238
diff changeset
294 else:
2219853a1503 revlogv2: track pending write in the docket and expose it to hooks
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 47238
diff changeset
295 official_index_end = self._index_end
47242
4abd474a10af revlogv2: also keep track for the size of the "data" file
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 47241
diff changeset
296 official_data_end = self._data_end
47389
e6292eb33384 revlog: store sidedata in their own file
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 47388
diff changeset
297 official_sidedata_end = self._sidedata_end
47241
2219853a1503 revlogv2: track pending write in the docket and expose it to hooks
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 47238
diff changeset
298
2219853a1503 revlogv2: track pending write in the docket and expose it to hooks
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 47238
diff changeset
299 # this assert should be True as long as we have a single index filename
47242
4abd474a10af revlogv2: also keep track for the size of the "data" file
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 47241
diff changeset
300 assert official_data_end <= self._data_end
47389
e6292eb33384 revlog: store sidedata in their own file
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 47388
diff changeset
301 assert official_sidedata_end <= self._sidedata_end
47238
6597255a4f94 revlogv2: track current index size in the docket
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 47234
diff changeset
302 data = (
6597255a4f94 revlogv2: track current index size in the docket
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 47234
diff changeset
303 self._version_header,
47323
f612db768c7a revlogv2: use a unique filename for index
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 47312
diff changeset
304 len(self._index_uuid),
47453
c252631500e4 revlog: add a way to keep track of older uids in the docket
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 47423
diff changeset
305 len(self._older_index_uuids),
47324
0a3fa41fa719 revlogv2: use a unique filename for data
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 47323
diff changeset
306 len(self._data_uuid),
47453
c252631500e4 revlog: add a way to keep track of older uids in the docket
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 47423
diff changeset
307 len(self._older_data_uuids),
47389
e6292eb33384 revlog: store sidedata in their own file
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 47388
diff changeset
308 len(self._sidedata_uuid),
47453
c252631500e4 revlog: add a way to keep track of older uids in the docket
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 47423
diff changeset
309 len(self._older_sidedata_uuids),
47241
2219853a1503 revlogv2: track pending write in the docket and expose it to hooks
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 47238
diff changeset
310 official_index_end,
47238
6597255a4f94 revlogv2: track current index size in the docket
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 47234
diff changeset
311 self._index_end,
47242
4abd474a10af revlogv2: also keep track for the size of the "data" file
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 47241
diff changeset
312 official_data_end,
4abd474a10af revlogv2: also keep track for the size of the "data" file
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 47241
diff changeset
313 self._data_end,
47389
e6292eb33384 revlog: store sidedata in their own file
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 47388
diff changeset
314 official_sidedata_end,
e6292eb33384 revlog: store sidedata in their own file
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 47388
diff changeset
315 self._sidedata_end,
47255
ff9fd7107d11 revlog: implement a "default compression" mode
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 47242
diff changeset
316 self.default_compression_header,
47238
6597255a4f94 revlogv2: track current index size in the docket
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 47234
diff changeset
317 )
47323
f612db768c7a revlogv2: use a unique filename for index
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 47312
diff changeset
318 s = []
f612db768c7a revlogv2: use a unique filename for index
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 47312
diff changeset
319 s.append(S_HEADER.pack(*data))
47453
c252631500e4 revlog: add a way to keep track of older uids in the docket
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 47423
diff changeset
320
47323
f612db768c7a revlogv2: use a unique filename for index
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 47312
diff changeset
321 s.append(self._index_uuid)
47453
c252631500e4 revlog: add a way to keep track of older uids in the docket
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 47423
diff changeset
322 for u, size in self._older_index_uuids:
c252631500e4 revlog: add a way to keep track of older uids in the docket
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 47423
diff changeset
323 s.append(S_OLD_UID.pack(len(u), size))
c252631500e4 revlog: add a way to keep track of older uids in the docket
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 47423
diff changeset
324 for u, size in self._older_index_uuids:
c252631500e4 revlog: add a way to keep track of older uids in the docket
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 47423
diff changeset
325 s.append(u)
c252631500e4 revlog: add a way to keep track of older uids in the docket
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 47423
diff changeset
326
47324
0a3fa41fa719 revlogv2: use a unique filename for data
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 47323
diff changeset
327 s.append(self._data_uuid)
47453
c252631500e4 revlog: add a way to keep track of older uids in the docket
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 47423
diff changeset
328 for u, size in self._older_data_uuids:
c252631500e4 revlog: add a way to keep track of older uids in the docket
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 47423
diff changeset
329 s.append(S_OLD_UID.pack(len(u), size))
c252631500e4 revlog: add a way to keep track of older uids in the docket
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 47423
diff changeset
330 for u, size in self._older_data_uuids:
c252631500e4 revlog: add a way to keep track of older uids in the docket
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 47423
diff changeset
331 s.append(u)
c252631500e4 revlog: add a way to keep track of older uids in the docket
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 47423
diff changeset
332
47389
e6292eb33384 revlog: store sidedata in their own file
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 47388
diff changeset
333 s.append(self._sidedata_uuid)
47453
c252631500e4 revlog: add a way to keep track of older uids in the docket
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 47423
diff changeset
334 for u, size in self._older_sidedata_uuids:
c252631500e4 revlog: add a way to keep track of older uids in the docket
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 47423
diff changeset
335 s.append(S_OLD_UID.pack(len(u), size))
c252631500e4 revlog: add a way to keep track of older uids in the docket
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 47423
diff changeset
336 for u, size in self._older_sidedata_uuids:
c252631500e4 revlog: add a way to keep track of older uids in the docket
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 47423
diff changeset
337 s.append(u)
47323
f612db768c7a revlogv2: use a unique filename for index
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 47312
diff changeset
338 return b''.join(s)
47234
616b8f412676 revlogv2: introduce a very basic docket file
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
diff changeset
339
616b8f412676 revlogv2: introduce a very basic docket file
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
diff changeset
340
616b8f412676 revlogv2: introduce a very basic docket file
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
diff changeset
341 def default_docket(revlog, version_header):
616b8f412676 revlogv2: introduce a very basic docket file
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
diff changeset
342 """given a revlog version a new docket object for the given revlog"""
47266
921648d31553 changelogv2: use a dedicated version number
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 47255
diff changeset
343 rl_version = version_header & 0xFFFF
921648d31553 changelogv2: use a dedicated version number
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 47255
diff changeset
344 if rl_version not in (constants.REVLOGV2, constants.CHANGELOGV2):
47234
616b8f412676 revlogv2: introduce a very basic docket file
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
diff changeset
345 return None
47255
ff9fd7107d11 revlog: implement a "default compression" mode
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 47242
diff changeset
346 comp = util.compengines[revlog._compengine].revlogheader()
ff9fd7107d11 revlog: implement a "default compression" mode
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 47242
diff changeset
347 docket = RevlogDocket(
ff9fd7107d11 revlog: implement a "default compression" mode
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 47242
diff changeset
348 revlog,
ff9fd7107d11 revlog: implement a "default compression" mode
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 47242
diff changeset
349 version_header=version_header,
ff9fd7107d11 revlog: implement a "default compression" mode
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 47242
diff changeset
350 default_compression_header=comp,
ff9fd7107d11 revlog: implement a "default compression" mode
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 47242
diff changeset
351 )
47234
616b8f412676 revlogv2: introduce a very basic docket file
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
diff changeset
352 docket._dirty = True
616b8f412676 revlogv2: introduce a very basic docket file
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
diff changeset
353 return docket
616b8f412676 revlogv2: introduce a very basic docket file
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
diff changeset
354
616b8f412676 revlogv2: introduce a very basic docket file
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
diff changeset
355
47453
c252631500e4 revlog: add a way to keep track of older uids in the docket
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 47423
diff changeset
356 def _parse_old_uids(get_data, count):
c252631500e4 revlog: add a way to keep track of older uids in the docket
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 47423
diff changeset
357 all_sizes = []
c252631500e4 revlog: add a way to keep track of older uids in the docket
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 47423
diff changeset
358 all_uids = []
c252631500e4 revlog: add a way to keep track of older uids in the docket
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 47423
diff changeset
359 for i in range(0, count):
c252631500e4 revlog: add a way to keep track of older uids in the docket
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 47423
diff changeset
360 raw = get_data(S_OLD_UID.size)
c252631500e4 revlog: add a way to keep track of older uids in the docket
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 47423
diff changeset
361 all_sizes.append(S_OLD_UID.unpack(raw))
c252631500e4 revlog: add a way to keep track of older uids in the docket
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 47423
diff changeset
362
c252631500e4 revlog: add a way to keep track of older uids in the docket
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 47423
diff changeset
363 for uid_size, file_size in all_sizes:
c252631500e4 revlog: add a way to keep track of older uids in the docket
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 47423
diff changeset
364 uid = get_data(uid_size)
c252631500e4 revlog: add a way to keep track of older uids in the docket
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 47423
diff changeset
365 all_uids.append((uid, file_size))
c252631500e4 revlog: add a way to keep track of older uids in the docket
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 47423
diff changeset
366 return all_uids
c252631500e4 revlog: add a way to keep track of older uids in the docket
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 47423
diff changeset
367
c252631500e4 revlog: add a way to keep track of older uids in the docket
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 47423
diff changeset
368
47241
2219853a1503 revlogv2: track pending write in the docket and expose it to hooks
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 47238
diff changeset
369 def parse_docket(revlog, data, use_pending=False):
47234
616b8f412676 revlogv2: introduce a very basic docket file
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
diff changeset
370 """given some docket data return a docket object for the given revlog"""
616b8f412676 revlogv2: introduce a very basic docket file
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
diff changeset
371 header = S_HEADER.unpack(data[: S_HEADER.size])
47326
53ab13d6a5db revlogv2: add a `get_data` helper to grab the next piece of docket
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 47325
diff changeset
372
53ab13d6a5db revlogv2: add a `get_data` helper to grab the next piece of docket
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 47325
diff changeset
373 # this is a mutable closure capture used in `get_data`
53ab13d6a5db revlogv2: add a `get_data` helper to grab the next piece of docket
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 47325
diff changeset
374 offset = [S_HEADER.size]
53ab13d6a5db revlogv2: add a `get_data` helper to grab the next piece of docket
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 47325
diff changeset
375
53ab13d6a5db revlogv2: add a `get_data` helper to grab the next piece of docket
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 47325
diff changeset
376 def get_data(size):
53ab13d6a5db revlogv2: add a `get_data` helper to grab the next piece of docket
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 47325
diff changeset
377 """utility closure to access the `size` next bytes"""
53ab13d6a5db revlogv2: add a `get_data` helper to grab the next piece of docket
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 47325
diff changeset
378 if offset[0] + size > len(data):
53ab13d6a5db revlogv2: add a `get_data` helper to grab the next piece of docket
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 47325
diff changeset
379 # XXX better class
53ab13d6a5db revlogv2: add a `get_data` helper to grab the next piece of docket
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 47325
diff changeset
380 msg = b"docket is too short, expected %d got %d"
53ab13d6a5db revlogv2: add a `get_data` helper to grab the next piece of docket
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 47325
diff changeset
381 msg %= (offset[0] + size, len(data))
53ab13d6a5db revlogv2: add a `get_data` helper to grab the next piece of docket
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 47325
diff changeset
382 raise error.Abort(msg)
53ab13d6a5db revlogv2: add a `get_data` helper to grab the next piece of docket
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 47325
diff changeset
383 raw = data[offset[0] : offset[0] + size]
53ab13d6a5db revlogv2: add a `get_data` helper to grab the next piece of docket
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 47325
diff changeset
384 offset[0] += size
53ab13d6a5db revlogv2: add a `get_data` helper to grab the next piece of docket
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 47325
diff changeset
385 return raw
47325
f286d715f9ab revlogv2: simplify and clarify the processing of each entry
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 47324
diff changeset
386
f286d715f9ab revlogv2: simplify and clarify the processing of each entry
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 47324
diff changeset
387 iheader = iter(header)
f286d715f9ab revlogv2: simplify and clarify the processing of each entry
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 47324
diff changeset
388
f286d715f9ab revlogv2: simplify and clarify the processing of each entry
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 47324
diff changeset
389 version_header = next(iheader)
f286d715f9ab revlogv2: simplify and clarify the processing of each entry
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 47324
diff changeset
390
f286d715f9ab revlogv2: simplify and clarify the processing of each entry
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 47324
diff changeset
391 index_uuid_size = next(iheader)
47326
53ab13d6a5db revlogv2: add a `get_data` helper to grab the next piece of docket
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 47325
diff changeset
392 index_uuid = get_data(index_uuid_size)
47325
f286d715f9ab revlogv2: simplify and clarify the processing of each entry
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 47324
diff changeset
393
47453
c252631500e4 revlog: add a way to keep track of older uids in the docket
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 47423
diff changeset
394 older_index_uuid_count = next(iheader)
c252631500e4 revlog: add a way to keep track of older uids in the docket
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 47423
diff changeset
395 older_index_uuids = _parse_old_uids(get_data, older_index_uuid_count)
c252631500e4 revlog: add a way to keep track of older uids in the docket
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 47423
diff changeset
396
47325
f286d715f9ab revlogv2: simplify and clarify the processing of each entry
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 47324
diff changeset
397 data_uuid_size = next(iheader)
47326
53ab13d6a5db revlogv2: add a `get_data` helper to grab the next piece of docket
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 47325
diff changeset
398 data_uuid = get_data(data_uuid_size)
47325
f286d715f9ab revlogv2: simplify and clarify the processing of each entry
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 47324
diff changeset
399
47453
c252631500e4 revlog: add a way to keep track of older uids in the docket
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 47423
diff changeset
400 older_data_uuid_count = next(iheader)
c252631500e4 revlog: add a way to keep track of older uids in the docket
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 47423
diff changeset
401 older_data_uuids = _parse_old_uids(get_data, older_data_uuid_count)
c252631500e4 revlog: add a way to keep track of older uids in the docket
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 47423
diff changeset
402
47389
e6292eb33384 revlog: store sidedata in their own file
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 47388
diff changeset
403 sidedata_uuid_size = next(iheader)
e6292eb33384 revlog: store sidedata in their own file
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 47388
diff changeset
404 sidedata_uuid = get_data(sidedata_uuid_size)
e6292eb33384 revlog: store sidedata in their own file
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 47388
diff changeset
405
47453
c252631500e4 revlog: add a way to keep track of older uids in the docket
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 47423
diff changeset
406 older_sidedata_uuid_count = next(iheader)
c252631500e4 revlog: add a way to keep track of older uids in the docket
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 47423
diff changeset
407 older_sidedata_uuids = _parse_old_uids(get_data, older_sidedata_uuid_count)
c252631500e4 revlog: add a way to keep track of older uids in the docket
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 47423
diff changeset
408
47325
f286d715f9ab revlogv2: simplify and clarify the processing of each entry
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 47324
diff changeset
409 index_size = next(iheader)
f286d715f9ab revlogv2: simplify and clarify the processing of each entry
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 47324
diff changeset
410
f286d715f9ab revlogv2: simplify and clarify the processing of each entry
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 47324
diff changeset
411 pending_index_size = next(iheader)
f286d715f9ab revlogv2: simplify and clarify the processing of each entry
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 47324
diff changeset
412
f286d715f9ab revlogv2: simplify and clarify the processing of each entry
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 47324
diff changeset
413 data_size = next(iheader)
f286d715f9ab revlogv2: simplify and clarify the processing of each entry
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 47324
diff changeset
414
f286d715f9ab revlogv2: simplify and clarify the processing of each entry
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 47324
diff changeset
415 pending_data_size = next(iheader)
f286d715f9ab revlogv2: simplify and clarify the processing of each entry
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 47324
diff changeset
416
47389
e6292eb33384 revlog: store sidedata in their own file
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 47388
diff changeset
417 sidedata_size = next(iheader)
e6292eb33384 revlog: store sidedata in their own file
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 47388
diff changeset
418
e6292eb33384 revlog: store sidedata in their own file
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 47388
diff changeset
419 pending_sidedata_size = next(iheader)
e6292eb33384 revlog: store sidedata in their own file
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 47388
diff changeset
420
47325
f286d715f9ab revlogv2: simplify and clarify the processing of each entry
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 47324
diff changeset
421 default_compression_header = next(iheader)
f286d715f9ab revlogv2: simplify and clarify the processing of each entry
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 47324
diff changeset
422
47234
616b8f412676 revlogv2: introduce a very basic docket file
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
diff changeset
423 docket = RevlogDocket(
616b8f412676 revlogv2: introduce a very basic docket file
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
diff changeset
424 revlog,
47241
2219853a1503 revlogv2: track pending write in the docket and expose it to hooks
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 47238
diff changeset
425 use_pending=use_pending,
47234
616b8f412676 revlogv2: introduce a very basic docket file
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
diff changeset
426 version_header=version_header,
47323
f612db768c7a revlogv2: use a unique filename for index
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 47312
diff changeset
427 index_uuid=index_uuid,
47453
c252631500e4 revlog: add a way to keep track of older uids in the docket
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 47423
diff changeset
428 older_index_uuids=older_index_uuids,
47324
0a3fa41fa719 revlogv2: use a unique filename for data
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 47323
diff changeset
429 data_uuid=data_uuid,
47453
c252631500e4 revlog: add a way to keep track of older uids in the docket
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 47423
diff changeset
430 older_data_uuids=older_data_uuids,
47389
e6292eb33384 revlog: store sidedata in their own file
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 47388
diff changeset
431 sidedata_uuid=sidedata_uuid,
47453
c252631500e4 revlog: add a way to keep track of older uids in the docket
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 47423
diff changeset
432 older_sidedata_uuids=older_sidedata_uuids,
47238
6597255a4f94 revlogv2: track current index size in the docket
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 47234
diff changeset
433 index_end=index_size,
47241
2219853a1503 revlogv2: track pending write in the docket and expose it to hooks
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 47238
diff changeset
434 pending_index_end=pending_index_size,
47242
4abd474a10af revlogv2: also keep track for the size of the "data" file
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 47241
diff changeset
435 data_end=data_size,
4abd474a10af revlogv2: also keep track for the size of the "data" file
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 47241
diff changeset
436 pending_data_end=pending_data_size,
47389
e6292eb33384 revlog: store sidedata in their own file
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 47388
diff changeset
437 sidedata_end=sidedata_size,
e6292eb33384 revlog: store sidedata in their own file
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 47388
diff changeset
438 pending_sidedata_end=pending_sidedata_size,
47255
ff9fd7107d11 revlog: implement a "default compression" mode
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 47242
diff changeset
439 default_compression_header=default_compression_header,
47234
616b8f412676 revlogv2: introduce a very basic docket file
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
diff changeset
440 )
616b8f412676 revlogv2: introduce a very basic docket file
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
diff changeset
441 return docket