annotate mercurial/revlogutils/docket.py @ 51528:f85f23f1479b

branchcache: skip entries that are topological heads in the on disk file In the majority of cases, topological heads are also branch heads. We have efficient way to get the topological heads and efficient way to retrieve their branch information. So there is little value in putting them in the branch cache file explicitly. On the contrary, writing them explicitly tend to create very large cache file that are inefficient to read and update. So the branch cache v3 format is no longer including them. This changeset focus on the format aspect and have no focus on the performance aspect. We will cover that later.
author Pierre-Yves David <pierre-yves.david@octobus.net>
date Thu, 07 Mar 2024 10:55:22 +0100
parents f0756d0636d1
children f4733654f144
Ignore whitespace changes - Everywhere: Within whitespace: At end of lines:
rev   line source
47234
616b8f412676 revlogv2: introduce a very basic docket file
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
diff changeset
1 # docket - code related to revlog "docket"
616b8f412676 revlogv2: introduce a very basic docket file
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
diff changeset
2 #
616b8f412676 revlogv2: introduce a very basic docket file
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
diff changeset
3 # Copyright 2021 Pierre-Yves David <pierre-yves.david@octobus.net>
616b8f412676 revlogv2: introduce a very basic docket file
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
diff changeset
4 #
616b8f412676 revlogv2: introduce a very basic docket file
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
diff changeset
5 # This software may be used and distributed according to the terms of the
616b8f412676 revlogv2: introduce a very basic docket file
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
diff changeset
6 # GNU General Public License version 2 or any later version.
616b8f412676 revlogv2: introduce a very basic docket file
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
diff changeset
7
616b8f412676 revlogv2: introduce a very basic docket file
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
diff changeset
8 ### Revlog docket file
616b8f412676 revlogv2: introduce a very basic docket file
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
diff changeset
9 #
616b8f412676 revlogv2: introduce a very basic docket file
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
diff changeset
10 # The revlog is stored on disk using multiple files:
616b8f412676 revlogv2: introduce a very basic docket file
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
diff changeset
11 #
616b8f412676 revlogv2: introduce a very basic docket file
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
diff changeset
12 # * a small docket file, containing metadata and a pointer,
616b8f412676 revlogv2: introduce a very basic docket file
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
diff changeset
13 #
616b8f412676 revlogv2: introduce a very basic docket file
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
diff changeset
14 # * an index file, containing fixed width information about revisions,
616b8f412676 revlogv2: introduce a very basic docket file
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
diff changeset
15 #
616b8f412676 revlogv2: introduce a very basic docket file
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
diff changeset
16 # * a data file, containing variable width data for these revisions,
616b8f412676 revlogv2: introduce a very basic docket file
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
diff changeset
17
616b8f412676 revlogv2: introduce a very basic docket file
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
diff changeset
18
47312
7ea39d633cf3 docket: move the uid logic in the `revlogutils.docket` module
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 47266
diff changeset
19 import os
7ea39d633cf3 docket: move the uid logic in the `revlogutils.docket` module
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 47266
diff changeset
20 import random
47234
616b8f412676 revlogv2: introduce a very basic docket file
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
diff changeset
21 import struct
616b8f412676 revlogv2: introduce a very basic docket file
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
diff changeset
22
47241
2219853a1503 revlogv2: track pending write in the docket and expose it to hooks
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 47238
diff changeset
23 from .. import (
47312
7ea39d633cf3 docket: move the uid logic in the `revlogutils.docket` module
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 47266
diff changeset
24 encoding,
47241
2219853a1503 revlogv2: track pending write in the docket and expose it to hooks
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 47238
diff changeset
25 error,
47312
7ea39d633cf3 docket: move the uid logic in the `revlogutils.docket` module
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 47266
diff changeset
26 node,
47255
ff9fd7107d11 revlog: implement a "default compression" mode
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 47242
diff changeset
27 util,
47241
2219853a1503 revlogv2: track pending write in the docket and expose it to hooks
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 47238
diff changeset
28 )
2219853a1503 revlogv2: track pending write in the docket and expose it to hooks
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 47238
diff changeset
29
47234
616b8f412676 revlogv2: introduce a very basic docket file
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
diff changeset
30 from . import (
616b8f412676 revlogv2: introduce a very basic docket file
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
diff changeset
31 constants,
616b8f412676 revlogv2: introduce a very basic docket file
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
diff changeset
32 )
616b8f412676 revlogv2: introduce a very basic docket file
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
diff changeset
33
47312
7ea39d633cf3 docket: move the uid logic in the `revlogutils.docket` module
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 47266
diff changeset
34
7ea39d633cf3 docket: move the uid logic in the `revlogutils.docket` module
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 47266
diff changeset
35 def make_uid(id_size=8):
7ea39d633cf3 docket: move the uid logic in the `revlogutils.docket` module
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 47266
diff changeset
36 """return a new unique identifier.
7ea39d633cf3 docket: move the uid logic in the `revlogutils.docket` module
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 47266
diff changeset
37
7ea39d633cf3 docket: move the uid logic in the `revlogutils.docket` module
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 47266
diff changeset
38 The identifier is random and composed of ascii characters."""
7ea39d633cf3 docket: move the uid logic in the `revlogutils.docket` module
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 47266
diff changeset
39 # size we "hex" the result we need half the number of bits to have a final
7ea39d633cf3 docket: move the uid logic in the `revlogutils.docket` module
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 47266
diff changeset
40 # uuid of size ID_SIZE
7ea39d633cf3 docket: move the uid logic in the `revlogutils.docket` module
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 47266
diff changeset
41 return node.hex(os.urandom(id_size // 2))
7ea39d633cf3 docket: move the uid logic in the `revlogutils.docket` module
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 47266
diff changeset
42
7ea39d633cf3 docket: move the uid logic in the `revlogutils.docket` module
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 47266
diff changeset
43
7ea39d633cf3 docket: move the uid logic in the `revlogutils.docket` module
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 47266
diff changeset
44 # some special test logic to avoid anoying random output in the test
7ea39d633cf3 docket: move the uid logic in the `revlogutils.docket` module
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 47266
diff changeset
45 stable_docket_file = encoding.environ.get(b'HGTEST_UUIDFILE')
7ea39d633cf3 docket: move the uid logic in the `revlogutils.docket` module
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 47266
diff changeset
46
7ea39d633cf3 docket: move the uid logic in the `revlogutils.docket` module
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 47266
diff changeset
47 if stable_docket_file:
7ea39d633cf3 docket: move the uid logic in the `revlogutils.docket` module
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 47266
diff changeset
48
7ea39d633cf3 docket: move the uid logic in the `revlogutils.docket` module
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 47266
diff changeset
49 def make_uid(id_size=8):
7ea39d633cf3 docket: move the uid logic in the `revlogutils.docket` module
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 47266
diff changeset
50 try:
7ea39d633cf3 docket: move the uid logic in the `revlogutils.docket` module
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 47266
diff changeset
51 with open(stable_docket_file, mode='rb') as f:
7ea39d633cf3 docket: move the uid logic in the `revlogutils.docket` module
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 47266
diff changeset
52 seed = f.read().strip()
49306
2e726c934fcd py3: catch FileNotFoundError instead of checking errno == ENOENT
Manuel Jacob <me@manueljacob.de>
parents: 48946
diff changeset
53 except FileNotFoundError:
47312
7ea39d633cf3 docket: move the uid logic in the `revlogutils.docket` module
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 47266
diff changeset
54 seed = b'04' # chosen by a fair dice roll. garanteed to be random
48908
ed2af4560f6e revlogutils: remove Python 2 variant for iter_seed
Gregory Szorc <gregory.szorc@gmail.com>
parents: 48875
diff changeset
55 iter_seed = iter(seed)
47312
7ea39d633cf3 docket: move the uid logic in the `revlogutils.docket` module
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 47266
diff changeset
56 # some basic circular sum hashing on 64 bits
7ea39d633cf3 docket: move the uid logic in the `revlogutils.docket` module
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 47266
diff changeset
57 int_seed = 0
7ea39d633cf3 docket: move the uid logic in the `revlogutils.docket` module
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 47266
diff changeset
58 low_mask = int('1' * 35, 2)
7ea39d633cf3 docket: move the uid logic in the `revlogutils.docket` module
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 47266
diff changeset
59 for i in iter_seed:
7ea39d633cf3 docket: move the uid logic in the `revlogutils.docket` module
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 47266
diff changeset
60 high_part = int_seed >> 35
7ea39d633cf3 docket: move the uid logic in the `revlogutils.docket` module
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 47266
diff changeset
61 low_part = (int_seed & low_mask) << 28
7ea39d633cf3 docket: move the uid logic in the `revlogutils.docket` module
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 47266
diff changeset
62 int_seed = high_part + low_part + i
7ea39d633cf3 docket: move the uid logic in the `revlogutils.docket` module
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 47266
diff changeset
63 r = random.Random()
48909
0aae0e2e90fc revlogutils: unconditionally pass version to random seed
Gregory Szorc <gregory.szorc@gmail.com>
parents: 48908
diff changeset
64 r.seed(int_seed, version=1)
47312
7ea39d633cf3 docket: move the uid logic in the `revlogutils.docket` module
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 47266
diff changeset
65 # once we drop python 3.8 support we can simply use r.randbytes
7ea39d633cf3 docket: move the uid logic in the `revlogutils.docket` module
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 47266
diff changeset
66 raw = r.getrandbits(id_size * 4)
7ea39d633cf3 docket: move the uid logic in the `revlogutils.docket` module
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 47266
diff changeset
67 assert id_size == 8
7ea39d633cf3 docket: move the uid logic in the `revlogutils.docket` module
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 47266
diff changeset
68 p = struct.pack('>L', raw)
7ea39d633cf3 docket: move the uid logic in the `revlogutils.docket` module
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 47266
diff changeset
69 new = node.hex(p)
7ea39d633cf3 docket: move the uid logic in the `revlogutils.docket` module
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 47266
diff changeset
70 with open(stable_docket_file, 'wb') as f:
7ea39d633cf3 docket: move the uid logic in the `revlogutils.docket` module
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 47266
diff changeset
71 f.write(new)
7ea39d633cf3 docket: move the uid logic in the `revlogutils.docket` module
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 47266
diff changeset
72 return new
7ea39d633cf3 docket: move the uid logic in the `revlogutils.docket` module
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 47266
diff changeset
73
7ea39d633cf3 docket: move the uid logic in the `revlogutils.docket` module
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 47266
diff changeset
74
47234
616b8f412676 revlogv2: introduce a very basic docket file
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
diff changeset
75 # Docket format
616b8f412676 revlogv2: introduce a very basic docket file
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
diff changeset
76 #
616b8f412676 revlogv2: introduce a very basic docket file
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
diff changeset
77 # * 4 bytes: revlog version
616b8f412676 revlogv2: introduce a very basic docket file
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
diff changeset
78 # | This is mandatory as docket must be compatible with the previous
616b8f412676 revlogv2: introduce a very basic docket file
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
diff changeset
79 # | revlog index header.
47323
f612db768c7a revlogv2: use a unique filename for index
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 47312
diff changeset
80 # * 1 bytes: size of index uuid
47453
c252631500e4 revlog: add a way to keep track of older uids in the docket
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 47423
diff changeset
81 # * 1 bytes: number of outdated index uuid
47324
0a3fa41fa719 revlogv2: use a unique filename for data
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 47323
diff changeset
82 # * 1 bytes: size of data uuid
47453
c252631500e4 revlog: add a way to keep track of older uids in the docket
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 47423
diff changeset
83 # * 1 bytes: number of outdated data uuid
47389
e6292eb33384 revlog: store sidedata in their own file
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 47388
diff changeset
84 # * 1 bytes: size of sizedata uuid
47453
c252631500e4 revlog: add a way to keep track of older uids in the docket
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 47423
diff changeset
85 # * 1 bytes: number of outdated data uuid
47242
4abd474a10af revlogv2: also keep track for the size of the "data" file
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 47241
diff changeset
86 # * 8 bytes: size of index-data
4abd474a10af revlogv2: also keep track for the size of the "data" file
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 47241
diff changeset
87 # * 8 bytes: pending size of index-data
4abd474a10af revlogv2: also keep track for the size of the "data" file
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 47241
diff changeset
88 # * 8 bytes: size of data
47389
e6292eb33384 revlog: store sidedata in their own file
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 47388
diff changeset
89 # * 8 bytes: size of sidedata
47242
4abd474a10af revlogv2: also keep track for the size of the "data" file
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 47241
diff changeset
90 # * 8 bytes: pending size of data
47389
e6292eb33384 revlog: store sidedata in their own file
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 47388
diff changeset
91 # * 8 bytes: pending size of sidedata
47255
ff9fd7107d11 revlog: implement a "default compression" mode
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 47242
diff changeset
92 # * 1 bytes: default compression header
49714
f04d459909c6 changelog-v2: fix the docket `struct`
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 49306
diff changeset
93 S_HEADER = struct.Struct(constants.INDEX_HEADER_FMT + b'BBBBBBQQQQQQc')
47453
c252631500e4 revlog: add a way to keep track of older uids in the docket
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 47423
diff changeset
94 # * 1 bytes: size of index uuid
c252631500e4 revlog: add a way to keep track of older uids in the docket
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 47423
diff changeset
95 # * 8 bytes: size of file
c252631500e4 revlog: add a way to keep track of older uids in the docket
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 47423
diff changeset
96 S_OLD_UID = struct.Struct('>BL')
47234
616b8f412676 revlogv2: introduce a very basic docket file
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
diff changeset
97
616b8f412676 revlogv2: introduce a very basic docket file
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
diff changeset
98
48946
642e31cb55f0 py3: use class X: instead of class X(object):
Gregory Szorc <gregory.szorc@gmail.com>
parents: 48909
diff changeset
99 class RevlogDocket:
47234
616b8f412676 revlogv2: introduce a very basic docket file
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
diff changeset
100 """metadata associated with revlog"""
616b8f412676 revlogv2: introduce a very basic docket file
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
diff changeset
101
47241
2219853a1503 revlogv2: track pending write in the docket and expose it to hooks
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 47238
diff changeset
102 def __init__(
2219853a1503 revlogv2: track pending write in the docket and expose it to hooks
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 47238
diff changeset
103 self,
2219853a1503 revlogv2: track pending write in the docket and expose it to hooks
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 47238
diff changeset
104 revlog,
2219853a1503 revlogv2: track pending write in the docket and expose it to hooks
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 47238
diff changeset
105 use_pending=False,
2219853a1503 revlogv2: track pending write in the docket and expose it to hooks
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 47238
diff changeset
106 version_header=None,
47323
f612db768c7a revlogv2: use a unique filename for index
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 47312
diff changeset
107 index_uuid=None,
47453
c252631500e4 revlog: add a way to keep track of older uids in the docket
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 47423
diff changeset
108 older_index_uuids=(),
47324
0a3fa41fa719 revlogv2: use a unique filename for data
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 47323
diff changeset
109 data_uuid=None,
47453
c252631500e4 revlog: add a way to keep track of older uids in the docket
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 47423
diff changeset
110 older_data_uuids=(),
47389
e6292eb33384 revlog: store sidedata in their own file
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 47388
diff changeset
111 sidedata_uuid=None,
47453
c252631500e4 revlog: add a way to keep track of older uids in the docket
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 47423
diff changeset
112 older_sidedata_uuids=(),
47241
2219853a1503 revlogv2: track pending write in the docket and expose it to hooks
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 47238
diff changeset
113 index_end=0,
2219853a1503 revlogv2: track pending write in the docket and expose it to hooks
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 47238
diff changeset
114 pending_index_end=0,
47242
4abd474a10af revlogv2: also keep track for the size of the "data" file
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 47241
diff changeset
115 data_end=0,
4abd474a10af revlogv2: also keep track for the size of the "data" file
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 47241
diff changeset
116 pending_data_end=0,
47389
e6292eb33384 revlog: store sidedata in their own file
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 47388
diff changeset
117 sidedata_end=0,
e6292eb33384 revlog: store sidedata in their own file
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 47388
diff changeset
118 pending_sidedata_end=0,
47255
ff9fd7107d11 revlog: implement a "default compression" mode
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 47242
diff changeset
119 default_compression_header=None,
47241
2219853a1503 revlogv2: track pending write in the docket and expose it to hooks
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 47238
diff changeset
120 ):
47234
616b8f412676 revlogv2: introduce a very basic docket file
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
diff changeset
121 self._version_header = version_header
47241
2219853a1503 revlogv2: track pending write in the docket and expose it to hooks
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 47238
diff changeset
122 self._read_only = bool(use_pending)
47234
616b8f412676 revlogv2: introduce a very basic docket file
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
diff changeset
123 self._dirty = False
616b8f412676 revlogv2: introduce a very basic docket file
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
diff changeset
124 self._radix = revlog.radix
616b8f412676 revlogv2: introduce a very basic docket file
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
diff changeset
125 self._path = revlog._docket_file
616b8f412676 revlogv2: introduce a very basic docket file
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
diff changeset
126 self._opener = revlog.opener
47323
f612db768c7a revlogv2: use a unique filename for index
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 47312
diff changeset
127 self._index_uuid = index_uuid
47453
c252631500e4 revlog: add a way to keep track of older uids in the docket
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 47423
diff changeset
128 self._older_index_uuids = older_index_uuids
47324
0a3fa41fa719 revlogv2: use a unique filename for data
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 47323
diff changeset
129 self._data_uuid = data_uuid
47453
c252631500e4 revlog: add a way to keep track of older uids in the docket
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 47423
diff changeset
130 self._older_data_uuids = older_data_uuids
47389
e6292eb33384 revlog: store sidedata in their own file
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 47388
diff changeset
131 self._sidedata_uuid = sidedata_uuid
47453
c252631500e4 revlog: add a way to keep track of older uids in the docket
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 47423
diff changeset
132 self._older_sidedata_uuids = older_sidedata_uuids
c252631500e4 revlog: add a way to keep track of older uids in the docket
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 47423
diff changeset
133 assert not set(older_index_uuids) & set(older_data_uuids)
c252631500e4 revlog: add a way to keep track of older uids in the docket
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 47423
diff changeset
134 assert not set(older_data_uuids) & set(older_sidedata_uuids)
c252631500e4 revlog: add a way to keep track of older uids in the docket
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 47423
diff changeset
135 assert not set(older_index_uuids) & set(older_sidedata_uuids)
47242
4abd474a10af revlogv2: also keep track for the size of the "data" file
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 47241
diff changeset
136 # thes asserts should be True as long as we have a single index filename
47241
2219853a1503 revlogv2: track pending write in the docket and expose it to hooks
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 47238
diff changeset
137 assert index_end <= pending_index_end
47242
4abd474a10af revlogv2: also keep track for the size of the "data" file
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 47241
diff changeset
138 assert data_end <= pending_data_end
47389
e6292eb33384 revlog: store sidedata in their own file
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 47388
diff changeset
139 assert sidedata_end <= pending_sidedata_end
47241
2219853a1503 revlogv2: track pending write in the docket and expose it to hooks
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 47238
diff changeset
140 self._initial_index_end = index_end
2219853a1503 revlogv2: track pending write in the docket and expose it to hooks
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 47238
diff changeset
141 self._pending_index_end = pending_index_end
47242
4abd474a10af revlogv2: also keep track for the size of the "data" file
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 47241
diff changeset
142 self._initial_data_end = data_end
4abd474a10af revlogv2: also keep track for the size of the "data" file
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 47241
diff changeset
143 self._pending_data_end = pending_data_end
47389
e6292eb33384 revlog: store sidedata in their own file
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 47388
diff changeset
144 self._initial_sidedata_end = sidedata_end
e6292eb33384 revlog: store sidedata in their own file
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 47388
diff changeset
145 self._pending_sidedata_end = pending_sidedata_end
47241
2219853a1503 revlogv2: track pending write in the docket and expose it to hooks
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 47238
diff changeset
146 if use_pending:
2219853a1503 revlogv2: track pending write in the docket and expose it to hooks
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 47238
diff changeset
147 self._index_end = self._pending_index_end
47242
4abd474a10af revlogv2: also keep track for the size of the "data" file
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 47241
diff changeset
148 self._data_end = self._pending_data_end
47389
e6292eb33384 revlog: store sidedata in their own file
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 47388
diff changeset
149 self._sidedata_end = self._pending_sidedata_end
47241
2219853a1503 revlogv2: track pending write in the docket and expose it to hooks
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 47238
diff changeset
150 else:
2219853a1503 revlogv2: track pending write in the docket and expose it to hooks
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 47238
diff changeset
151 self._index_end = self._initial_index_end
47242
4abd474a10af revlogv2: also keep track for the size of the "data" file
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 47241
diff changeset
152 self._data_end = self._initial_data_end
47389
e6292eb33384 revlog: store sidedata in their own file
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 47388
diff changeset
153 self._sidedata_end = self._initial_sidedata_end
47255
ff9fd7107d11 revlog: implement a "default compression" mode
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 47242
diff changeset
154 self.default_compression_header = default_compression_header
47234
616b8f412676 revlogv2: introduce a very basic docket file
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
diff changeset
155
616b8f412676 revlogv2: introduce a very basic docket file
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
diff changeset
156 def index_filepath(self):
616b8f412676 revlogv2: introduce a very basic docket file
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
diff changeset
157 """file path to the current index file associated to this docket"""
616b8f412676 revlogv2: introduce a very basic docket file
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
diff changeset
158 # very simplistic version at first
47323
f612db768c7a revlogv2: use a unique filename for index
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 47312
diff changeset
159 if self._index_uuid is None:
f612db768c7a revlogv2: use a unique filename for index
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 47312
diff changeset
160 self._index_uuid = make_uid()
f612db768c7a revlogv2: use a unique filename for index
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 47312
diff changeset
161 return b"%s-%s.idx" % (self._radix, self._index_uuid)
47234
616b8f412676 revlogv2: introduce a very basic docket file
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
diff changeset
162
47454
865c260d7163 revlog: add docket method to request new content files
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 47453
diff changeset
163 def new_index_file(self):
865c260d7163 revlog: add docket method to request new content files
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 47453
diff changeset
164 """switch index file to a new UID
865c260d7163 revlog: add docket method to request new content files
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 47453
diff changeset
165
865c260d7163 revlog: add docket method to request new content files
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 47453
diff changeset
166 The previous index UID is moved to the "older" list."""
865c260d7163 revlog: add docket method to request new content files
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 47453
diff changeset
167 old = (self._index_uuid, self._index_end)
865c260d7163 revlog: add docket method to request new content files
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 47453
diff changeset
168 self._older_index_uuids.insert(0, old)
865c260d7163 revlog: add docket method to request new content files
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 47453
diff changeset
169 self._index_uuid = make_uid()
865c260d7163 revlog: add docket method to request new content files
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 47453
diff changeset
170 return self.index_filepath()
865c260d7163 revlog: add docket method to request new content files
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 47453
diff changeset
171
47455
f93a867a6486 revlog: list older-but-still-around file in `files`
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 47454
diff changeset
172 def old_index_filepaths(self, include_empty=True):
f93a867a6486 revlog: list older-but-still-around file in `files`
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 47454
diff changeset
173 """yield file path to older index files associated to this docket"""
f93a867a6486 revlog: list older-but-still-around file in `files`
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 47454
diff changeset
174 # very simplistic version at first
f93a867a6486 revlog: list older-but-still-around file in `files`
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 47454
diff changeset
175 for uuid, size in self._older_index_uuids:
f93a867a6486 revlog: list older-but-still-around file in `files`
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 47454
diff changeset
176 if include_empty or size > 0:
f93a867a6486 revlog: list older-but-still-around file in `files`
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 47454
diff changeset
177 yield b"%s-%s.idx" % (self._radix, uuid)
f93a867a6486 revlog: list older-but-still-around file in `files`
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 47454
diff changeset
178
47324
0a3fa41fa719 revlogv2: use a unique filename for data
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 47323
diff changeset
179 def data_filepath(self):
47388
bcf92bdc2bca revlog: fix docket.date_filepath docstring
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 47371
diff changeset
180 """file path to the current data file associated to this docket"""
47324
0a3fa41fa719 revlogv2: use a unique filename for data
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 47323
diff changeset
181 # very simplistic version at first
0a3fa41fa719 revlogv2: use a unique filename for data
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 47323
diff changeset
182 if self._data_uuid is None:
0a3fa41fa719 revlogv2: use a unique filename for data
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 47323
diff changeset
183 self._data_uuid = make_uid()
0a3fa41fa719 revlogv2: use a unique filename for data
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 47323
diff changeset
184 return b"%s-%s.dat" % (self._radix, self._data_uuid)
0a3fa41fa719 revlogv2: use a unique filename for data
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 47323
diff changeset
185
47454
865c260d7163 revlog: add docket method to request new content files
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 47453
diff changeset
186 def new_data_file(self):
865c260d7163 revlog: add docket method to request new content files
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 47453
diff changeset
187 """switch data file to a new UID
865c260d7163 revlog: add docket method to request new content files
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 47453
diff changeset
188
865c260d7163 revlog: add docket method to request new content files
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 47453
diff changeset
189 The previous data UID is moved to the "older" list."""
865c260d7163 revlog: add docket method to request new content files
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 47453
diff changeset
190 old = (self._data_uuid, self._data_end)
865c260d7163 revlog: add docket method to request new content files
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 47453
diff changeset
191 self._older_data_uuids.insert(0, old)
865c260d7163 revlog: add docket method to request new content files
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 47453
diff changeset
192 self._data_uuid = make_uid()
865c260d7163 revlog: add docket method to request new content files
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 47453
diff changeset
193 return self.data_filepath()
865c260d7163 revlog: add docket method to request new content files
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 47453
diff changeset
194
47455
f93a867a6486 revlog: list older-but-still-around file in `files`
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 47454
diff changeset
195 def old_data_filepaths(self, include_empty=True):
f93a867a6486 revlog: list older-but-still-around file in `files`
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 47454
diff changeset
196 """yield file path to older data files associated to this docket"""
f93a867a6486 revlog: list older-but-still-around file in `files`
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 47454
diff changeset
197 # very simplistic version at first
f93a867a6486 revlog: list older-but-still-around file in `files`
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 47454
diff changeset
198 for uuid, size in self._older_data_uuids:
f93a867a6486 revlog: list older-but-still-around file in `files`
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 47454
diff changeset
199 if include_empty or size > 0:
f93a867a6486 revlog: list older-but-still-around file in `files`
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 47454
diff changeset
200 yield b"%s-%s.dat" % (self._radix, uuid)
f93a867a6486 revlog: list older-but-still-around file in `files`
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 47454
diff changeset
201
47389
e6292eb33384 revlog: store sidedata in their own file
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 47388
diff changeset
202 def sidedata_filepath(self):
e6292eb33384 revlog: store sidedata in their own file
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 47388
diff changeset
203 """file path to the current sidedata file associated to this docket"""
e6292eb33384 revlog: store sidedata in their own file
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 47388
diff changeset
204 # very simplistic version at first
e6292eb33384 revlog: store sidedata in their own file
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 47388
diff changeset
205 if self._sidedata_uuid is None:
e6292eb33384 revlog: store sidedata in their own file
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 47388
diff changeset
206 self._sidedata_uuid = make_uid()
e6292eb33384 revlog: store sidedata in their own file
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 47388
diff changeset
207 return b"%s-%s.sda" % (self._radix, self._sidedata_uuid)
e6292eb33384 revlog: store sidedata in their own file
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 47388
diff changeset
208
47454
865c260d7163 revlog: add docket method to request new content files
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 47453
diff changeset
209 def new_sidedata_file(self):
865c260d7163 revlog: add docket method to request new content files
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 47453
diff changeset
210 """switch sidedata file to a new UID
865c260d7163 revlog: add docket method to request new content files
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 47453
diff changeset
211
865c260d7163 revlog: add docket method to request new content files
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 47453
diff changeset
212 The previous sidedata UID is moved to the "older" list."""
865c260d7163 revlog: add docket method to request new content files
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 47453
diff changeset
213 old = (self._sidedata_uuid, self._sidedata_end)
865c260d7163 revlog: add docket method to request new content files
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 47453
diff changeset
214 self._older_sidedata_uuids.insert(0, old)
865c260d7163 revlog: add docket method to request new content files
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 47453
diff changeset
215 self._sidedata_uuid = make_uid()
865c260d7163 revlog: add docket method to request new content files
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 47453
diff changeset
216 return self.sidedata_filepath()
865c260d7163 revlog: add docket method to request new content files
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 47453
diff changeset
217
47455
f93a867a6486 revlog: list older-but-still-around file in `files`
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 47454
diff changeset
218 def old_sidedata_filepaths(self, include_empty=True):
f93a867a6486 revlog: list older-but-still-around file in `files`
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 47454
diff changeset
219 """yield file path to older sidedata files associated to this docket"""
f93a867a6486 revlog: list older-but-still-around file in `files`
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 47454
diff changeset
220 # very simplistic version at first
f93a867a6486 revlog: list older-but-still-around file in `files`
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 47454
diff changeset
221 for uuid, size in self._older_sidedata_uuids:
f93a867a6486 revlog: list older-but-still-around file in `files`
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 47454
diff changeset
222 if include_empty or size > 0:
f93a867a6486 revlog: list older-but-still-around file in `files`
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 47454
diff changeset
223 yield b"%s-%s.sda" % (self._radix, uuid)
f93a867a6486 revlog: list older-but-still-around file in `files`
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 47454
diff changeset
224
47238
6597255a4f94 revlogv2: track current index size in the docket
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 47234
diff changeset
225 @property
6597255a4f94 revlogv2: track current index size in the docket
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 47234
diff changeset
226 def index_end(self):
6597255a4f94 revlogv2: track current index size in the docket
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 47234
diff changeset
227 return self._index_end
6597255a4f94 revlogv2: track current index size in the docket
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 47234
diff changeset
228
6597255a4f94 revlogv2: track current index size in the docket
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 47234
diff changeset
229 @index_end.setter
6597255a4f94 revlogv2: track current index size in the docket
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 47234
diff changeset
230 def index_end(self, new_size):
6597255a4f94 revlogv2: track current index size in the docket
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 47234
diff changeset
231 if new_size != self._index_end:
6597255a4f94 revlogv2: track current index size in the docket
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 47234
diff changeset
232 self._index_end = new_size
6597255a4f94 revlogv2: track current index size in the docket
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 47234
diff changeset
233 self._dirty = True
6597255a4f94 revlogv2: track current index size in the docket
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 47234
diff changeset
234
47242
4abd474a10af revlogv2: also keep track for the size of the "data" file
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 47241
diff changeset
235 @property
4abd474a10af revlogv2: also keep track for the size of the "data" file
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 47241
diff changeset
236 def data_end(self):
4abd474a10af revlogv2: also keep track for the size of the "data" file
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 47241
diff changeset
237 return self._data_end
4abd474a10af revlogv2: also keep track for the size of the "data" file
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 47241
diff changeset
238
4abd474a10af revlogv2: also keep track for the size of the "data" file
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 47241
diff changeset
239 @data_end.setter
4abd474a10af revlogv2: also keep track for the size of the "data" file
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 47241
diff changeset
240 def data_end(self, new_size):
4abd474a10af revlogv2: also keep track for the size of the "data" file
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 47241
diff changeset
241 if new_size != self._data_end:
4abd474a10af revlogv2: also keep track for the size of the "data" file
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 47241
diff changeset
242 self._data_end = new_size
4abd474a10af revlogv2: also keep track for the size of the "data" file
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 47241
diff changeset
243 self._dirty = True
4abd474a10af revlogv2: also keep track for the size of the "data" file
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 47241
diff changeset
244
47389
e6292eb33384 revlog: store sidedata in their own file
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 47388
diff changeset
245 @property
e6292eb33384 revlog: store sidedata in their own file
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 47388
diff changeset
246 def sidedata_end(self):
e6292eb33384 revlog: store sidedata in their own file
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 47388
diff changeset
247 return self._sidedata_end
e6292eb33384 revlog: store sidedata in their own file
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 47388
diff changeset
248
e6292eb33384 revlog: store sidedata in their own file
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 47388
diff changeset
249 @sidedata_end.setter
e6292eb33384 revlog: store sidedata in their own file
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 47388
diff changeset
250 def sidedata_end(self, new_size):
e6292eb33384 revlog: store sidedata in their own file
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 47388
diff changeset
251 if new_size != self._sidedata_end:
e6292eb33384 revlog: store sidedata in their own file
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 47388
diff changeset
252 self._sidedata_end = new_size
e6292eb33384 revlog: store sidedata in their own file
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 47388
diff changeset
253 self._dirty = True
e6292eb33384 revlog: store sidedata in their own file
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 47388
diff changeset
254
47241
2219853a1503 revlogv2: track pending write in the docket and expose it to hooks
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 47238
diff changeset
255 def write(self, transaction, pending=False, stripping=False):
47234
616b8f412676 revlogv2: introduce a very basic docket file
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
diff changeset
256 """write the modification of disk if any
616b8f412676 revlogv2: introduce a very basic docket file
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
diff changeset
257
616b8f412676 revlogv2: introduce a very basic docket file
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
diff changeset
258 This make the new content visible to all process"""
47241
2219853a1503 revlogv2: track pending write in the docket and expose it to hooks
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 47238
diff changeset
259 if not self._dirty:
2219853a1503 revlogv2: track pending write in the docket and expose it to hooks
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 47238
diff changeset
260 return False
2219853a1503 revlogv2: track pending write in the docket and expose it to hooks
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 47238
diff changeset
261 else:
2219853a1503 revlogv2: track pending write in the docket and expose it to hooks
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 47238
diff changeset
262 if self._read_only:
2219853a1503 revlogv2: track pending write in the docket and expose it to hooks
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 47238
diff changeset
263 msg = b'writing read-only docket: %s'
2219853a1503 revlogv2: track pending write in the docket and expose it to hooks
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 47238
diff changeset
264 msg %= self._path
2219853a1503 revlogv2: track pending write in the docket and expose it to hooks
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 47238
diff changeset
265 raise error.ProgrammingError(msg)
47238
6597255a4f94 revlogv2: track current index size in the docket
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 47234
diff changeset
266 if not stripping:
6597255a4f94 revlogv2: track current index size in the docket
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 47234
diff changeset
267 # XXX we could, leverage the docket while stripping. However it
6597255a4f94 revlogv2: track current index size in the docket
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 47234
diff changeset
268 # is not powerfull enough at the time of this comment
6597255a4f94 revlogv2: track current index size in the docket
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 47234
diff changeset
269 transaction.addbackup(self._path, location=b'store')
47234
616b8f412676 revlogv2: introduce a very basic docket file
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
diff changeset
270 with self._opener(self._path, mode=b'w', atomictemp=True) as f:
47241
2219853a1503 revlogv2: track pending write in the docket and expose it to hooks
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 47238
diff changeset
271 f.write(self._serialize(pending=pending))
2219853a1503 revlogv2: track pending write in the docket and expose it to hooks
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 47238
diff changeset
272 # if pending we still need to the write final data eventually
2219853a1503 revlogv2: track pending write in the docket and expose it to hooks
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 47238
diff changeset
273 self._dirty = pending
2219853a1503 revlogv2: track pending write in the docket and expose it to hooks
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 47238
diff changeset
274 return True
47234
616b8f412676 revlogv2: introduce a very basic docket file
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
diff changeset
275
47241
2219853a1503 revlogv2: track pending write in the docket and expose it to hooks
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 47238
diff changeset
276 def _serialize(self, pending=False):
2219853a1503 revlogv2: track pending write in the docket and expose it to hooks
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 47238
diff changeset
277 if pending:
2219853a1503 revlogv2: track pending write in the docket and expose it to hooks
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 47238
diff changeset
278 official_index_end = self._initial_index_end
47242
4abd474a10af revlogv2: also keep track for the size of the "data" file
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 47241
diff changeset
279 official_data_end = self._initial_data_end
47389
e6292eb33384 revlog: store sidedata in their own file
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 47388
diff changeset
280 official_sidedata_end = self._initial_sidedata_end
47241
2219853a1503 revlogv2: track pending write in the docket and expose it to hooks
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 47238
diff changeset
281 else:
2219853a1503 revlogv2: track pending write in the docket and expose it to hooks
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 47238
diff changeset
282 official_index_end = self._index_end
47242
4abd474a10af revlogv2: also keep track for the size of the "data" file
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 47241
diff changeset
283 official_data_end = self._data_end
47389
e6292eb33384 revlog: store sidedata in their own file
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 47388
diff changeset
284 official_sidedata_end = self._sidedata_end
47241
2219853a1503 revlogv2: track pending write in the docket and expose it to hooks
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 47238
diff changeset
285
2219853a1503 revlogv2: track pending write in the docket and expose it to hooks
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 47238
diff changeset
286 # this assert should be True as long as we have a single index filename
47242
4abd474a10af revlogv2: also keep track for the size of the "data" file
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 47241
diff changeset
287 assert official_data_end <= self._data_end
47389
e6292eb33384 revlog: store sidedata in their own file
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 47388
diff changeset
288 assert official_sidedata_end <= self._sidedata_end
47238
6597255a4f94 revlogv2: track current index size in the docket
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 47234
diff changeset
289 data = (
6597255a4f94 revlogv2: track current index size in the docket
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 47234
diff changeset
290 self._version_header,
47323
f612db768c7a revlogv2: use a unique filename for index
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 47312
diff changeset
291 len(self._index_uuid),
47453
c252631500e4 revlog: add a way to keep track of older uids in the docket
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 47423
diff changeset
292 len(self._older_index_uuids),
47324
0a3fa41fa719 revlogv2: use a unique filename for data
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 47323
diff changeset
293 len(self._data_uuid),
47453
c252631500e4 revlog: add a way to keep track of older uids in the docket
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 47423
diff changeset
294 len(self._older_data_uuids),
47389
e6292eb33384 revlog: store sidedata in their own file
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 47388
diff changeset
295 len(self._sidedata_uuid),
47453
c252631500e4 revlog: add a way to keep track of older uids in the docket
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 47423
diff changeset
296 len(self._older_sidedata_uuids),
47241
2219853a1503 revlogv2: track pending write in the docket and expose it to hooks
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 47238
diff changeset
297 official_index_end,
47238
6597255a4f94 revlogv2: track current index size in the docket
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 47234
diff changeset
298 self._index_end,
47242
4abd474a10af revlogv2: also keep track for the size of the "data" file
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 47241
diff changeset
299 official_data_end,
4abd474a10af revlogv2: also keep track for the size of the "data" file
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 47241
diff changeset
300 self._data_end,
47389
e6292eb33384 revlog: store sidedata in their own file
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 47388
diff changeset
301 official_sidedata_end,
e6292eb33384 revlog: store sidedata in their own file
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 47388
diff changeset
302 self._sidedata_end,
47255
ff9fd7107d11 revlog: implement a "default compression" mode
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 47242
diff changeset
303 self.default_compression_header,
47238
6597255a4f94 revlogv2: track current index size in the docket
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 47234
diff changeset
304 )
47323
f612db768c7a revlogv2: use a unique filename for index
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 47312
diff changeset
305 s = []
f612db768c7a revlogv2: use a unique filename for index
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 47312
diff changeset
306 s.append(S_HEADER.pack(*data))
47453
c252631500e4 revlog: add a way to keep track of older uids in the docket
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 47423
diff changeset
307
47323
f612db768c7a revlogv2: use a unique filename for index
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 47312
diff changeset
308 s.append(self._index_uuid)
47453
c252631500e4 revlog: add a way to keep track of older uids in the docket
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 47423
diff changeset
309 for u, size in self._older_index_uuids:
c252631500e4 revlog: add a way to keep track of older uids in the docket
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 47423
diff changeset
310 s.append(S_OLD_UID.pack(len(u), size))
c252631500e4 revlog: add a way to keep track of older uids in the docket
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 47423
diff changeset
311 for u, size in self._older_index_uuids:
c252631500e4 revlog: add a way to keep track of older uids in the docket
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 47423
diff changeset
312 s.append(u)
c252631500e4 revlog: add a way to keep track of older uids in the docket
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 47423
diff changeset
313
47324
0a3fa41fa719 revlogv2: use a unique filename for data
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 47323
diff changeset
314 s.append(self._data_uuid)
47453
c252631500e4 revlog: add a way to keep track of older uids in the docket
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 47423
diff changeset
315 for u, size in self._older_data_uuids:
c252631500e4 revlog: add a way to keep track of older uids in the docket
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 47423
diff changeset
316 s.append(S_OLD_UID.pack(len(u), size))
c252631500e4 revlog: add a way to keep track of older uids in the docket
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 47423
diff changeset
317 for u, size in self._older_data_uuids:
c252631500e4 revlog: add a way to keep track of older uids in the docket
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 47423
diff changeset
318 s.append(u)
c252631500e4 revlog: add a way to keep track of older uids in the docket
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 47423
diff changeset
319
47389
e6292eb33384 revlog: store sidedata in their own file
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 47388
diff changeset
320 s.append(self._sidedata_uuid)
47453
c252631500e4 revlog: add a way to keep track of older uids in the docket
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 47423
diff changeset
321 for u, size in self._older_sidedata_uuids:
c252631500e4 revlog: add a way to keep track of older uids in the docket
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 47423
diff changeset
322 s.append(S_OLD_UID.pack(len(u), size))
c252631500e4 revlog: add a way to keep track of older uids in the docket
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 47423
diff changeset
323 for u, size in self._older_sidedata_uuids:
c252631500e4 revlog: add a way to keep track of older uids in the docket
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 47423
diff changeset
324 s.append(u)
47323
f612db768c7a revlogv2: use a unique filename for index
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 47312
diff changeset
325 return b''.join(s)
47234
616b8f412676 revlogv2: introduce a very basic docket file
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
diff changeset
326
616b8f412676 revlogv2: introduce a very basic docket file
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
diff changeset
327
616b8f412676 revlogv2: introduce a very basic docket file
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
diff changeset
328 def default_docket(revlog, version_header):
616b8f412676 revlogv2: introduce a very basic docket file
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
diff changeset
329 """given a revlog version a new docket object for the given revlog"""
47266
921648d31553 changelogv2: use a dedicated version number
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 47255
diff changeset
330 rl_version = version_header & 0xFFFF
921648d31553 changelogv2: use a dedicated version number
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 47255
diff changeset
331 if rl_version not in (constants.REVLOGV2, constants.CHANGELOGV2):
47234
616b8f412676 revlogv2: introduce a very basic docket file
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
diff changeset
332 return None
51053
f0756d0636d1 revlog: remove legacy usage of `_compengine`
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 49714
diff changeset
333 comp = util.compengines[
f0756d0636d1 revlog: remove legacy usage of `_compengine`
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 49714
diff changeset
334 revlog.feature_config.compression_engine
f0756d0636d1 revlog: remove legacy usage of `_compengine`
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 49714
diff changeset
335 ].revlogheader()
47255
ff9fd7107d11 revlog: implement a "default compression" mode
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 47242
diff changeset
336 docket = RevlogDocket(
ff9fd7107d11 revlog: implement a "default compression" mode
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 47242
diff changeset
337 revlog,
ff9fd7107d11 revlog: implement a "default compression" mode
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 47242
diff changeset
338 version_header=version_header,
ff9fd7107d11 revlog: implement a "default compression" mode
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 47242
diff changeset
339 default_compression_header=comp,
ff9fd7107d11 revlog: implement a "default compression" mode
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 47242
diff changeset
340 )
47234
616b8f412676 revlogv2: introduce a very basic docket file
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
diff changeset
341 docket._dirty = True
616b8f412676 revlogv2: introduce a very basic docket file
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
diff changeset
342 return docket
616b8f412676 revlogv2: introduce a very basic docket file
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
diff changeset
343
616b8f412676 revlogv2: introduce a very basic docket file
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
diff changeset
344
47453
c252631500e4 revlog: add a way to keep track of older uids in the docket
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 47423
diff changeset
345 def _parse_old_uids(get_data, count):
c252631500e4 revlog: add a way to keep track of older uids in the docket
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 47423
diff changeset
346 all_sizes = []
c252631500e4 revlog: add a way to keep track of older uids in the docket
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 47423
diff changeset
347 all_uids = []
c252631500e4 revlog: add a way to keep track of older uids in the docket
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 47423
diff changeset
348 for i in range(0, count):
c252631500e4 revlog: add a way to keep track of older uids in the docket
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 47423
diff changeset
349 raw = get_data(S_OLD_UID.size)
c252631500e4 revlog: add a way to keep track of older uids in the docket
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 47423
diff changeset
350 all_sizes.append(S_OLD_UID.unpack(raw))
c252631500e4 revlog: add a way to keep track of older uids in the docket
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 47423
diff changeset
351
c252631500e4 revlog: add a way to keep track of older uids in the docket
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 47423
diff changeset
352 for uid_size, file_size in all_sizes:
c252631500e4 revlog: add a way to keep track of older uids in the docket
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 47423
diff changeset
353 uid = get_data(uid_size)
c252631500e4 revlog: add a way to keep track of older uids in the docket
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 47423
diff changeset
354 all_uids.append((uid, file_size))
c252631500e4 revlog: add a way to keep track of older uids in the docket
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 47423
diff changeset
355 return all_uids
c252631500e4 revlog: add a way to keep track of older uids in the docket
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 47423
diff changeset
356
c252631500e4 revlog: add a way to keep track of older uids in the docket
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 47423
diff changeset
357
47241
2219853a1503 revlogv2: track pending write in the docket and expose it to hooks
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 47238
diff changeset
358 def parse_docket(revlog, data, use_pending=False):
47234
616b8f412676 revlogv2: introduce a very basic docket file
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
diff changeset
359 """given some docket data return a docket object for the given revlog"""
616b8f412676 revlogv2: introduce a very basic docket file
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
diff changeset
360 header = S_HEADER.unpack(data[: S_HEADER.size])
47326
53ab13d6a5db revlogv2: add a `get_data` helper to grab the next piece of docket
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 47325
diff changeset
361
53ab13d6a5db revlogv2: add a `get_data` helper to grab the next piece of docket
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 47325
diff changeset
362 # this is a mutable closure capture used in `get_data`
53ab13d6a5db revlogv2: add a `get_data` helper to grab the next piece of docket
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 47325
diff changeset
363 offset = [S_HEADER.size]
53ab13d6a5db revlogv2: add a `get_data` helper to grab the next piece of docket
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 47325
diff changeset
364
53ab13d6a5db revlogv2: add a `get_data` helper to grab the next piece of docket
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 47325
diff changeset
365 def get_data(size):
53ab13d6a5db revlogv2: add a `get_data` helper to grab the next piece of docket
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 47325
diff changeset
366 """utility closure to access the `size` next bytes"""
53ab13d6a5db revlogv2: add a `get_data` helper to grab the next piece of docket
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 47325
diff changeset
367 if offset[0] + size > len(data):
53ab13d6a5db revlogv2: add a `get_data` helper to grab the next piece of docket
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 47325
diff changeset
368 # XXX better class
53ab13d6a5db revlogv2: add a `get_data` helper to grab the next piece of docket
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 47325
diff changeset
369 msg = b"docket is too short, expected %d got %d"
53ab13d6a5db revlogv2: add a `get_data` helper to grab the next piece of docket
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 47325
diff changeset
370 msg %= (offset[0] + size, len(data))
53ab13d6a5db revlogv2: add a `get_data` helper to grab the next piece of docket
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 47325
diff changeset
371 raise error.Abort(msg)
53ab13d6a5db revlogv2: add a `get_data` helper to grab the next piece of docket
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 47325
diff changeset
372 raw = data[offset[0] : offset[0] + size]
53ab13d6a5db revlogv2: add a `get_data` helper to grab the next piece of docket
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 47325
diff changeset
373 offset[0] += size
53ab13d6a5db revlogv2: add a `get_data` helper to grab the next piece of docket
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 47325
diff changeset
374 return raw
47325
f286d715f9ab revlogv2: simplify and clarify the processing of each entry
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 47324
diff changeset
375
f286d715f9ab revlogv2: simplify and clarify the processing of each entry
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 47324
diff changeset
376 iheader = iter(header)
f286d715f9ab revlogv2: simplify and clarify the processing of each entry
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 47324
diff changeset
377
f286d715f9ab revlogv2: simplify and clarify the processing of each entry
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 47324
diff changeset
378 version_header = next(iheader)
f286d715f9ab revlogv2: simplify and clarify the processing of each entry
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 47324
diff changeset
379
f286d715f9ab revlogv2: simplify and clarify the processing of each entry
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 47324
diff changeset
380 index_uuid_size = next(iheader)
47326
53ab13d6a5db revlogv2: add a `get_data` helper to grab the next piece of docket
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 47325
diff changeset
381 index_uuid = get_data(index_uuid_size)
47325
f286d715f9ab revlogv2: simplify and clarify the processing of each entry
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 47324
diff changeset
382
47453
c252631500e4 revlog: add a way to keep track of older uids in the docket
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 47423
diff changeset
383 older_index_uuid_count = next(iheader)
c252631500e4 revlog: add a way to keep track of older uids in the docket
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 47423
diff changeset
384 older_index_uuids = _parse_old_uids(get_data, older_index_uuid_count)
c252631500e4 revlog: add a way to keep track of older uids in the docket
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 47423
diff changeset
385
47325
f286d715f9ab revlogv2: simplify and clarify the processing of each entry
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 47324
diff changeset
386 data_uuid_size = next(iheader)
47326
53ab13d6a5db revlogv2: add a `get_data` helper to grab the next piece of docket
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 47325
diff changeset
387 data_uuid = get_data(data_uuid_size)
47325
f286d715f9ab revlogv2: simplify and clarify the processing of each entry
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 47324
diff changeset
388
47453
c252631500e4 revlog: add a way to keep track of older uids in the docket
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 47423
diff changeset
389 older_data_uuid_count = next(iheader)
c252631500e4 revlog: add a way to keep track of older uids in the docket
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 47423
diff changeset
390 older_data_uuids = _parse_old_uids(get_data, older_data_uuid_count)
c252631500e4 revlog: add a way to keep track of older uids in the docket
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 47423
diff changeset
391
47389
e6292eb33384 revlog: store sidedata in their own file
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 47388
diff changeset
392 sidedata_uuid_size = next(iheader)
e6292eb33384 revlog: store sidedata in their own file
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 47388
diff changeset
393 sidedata_uuid = get_data(sidedata_uuid_size)
e6292eb33384 revlog: store sidedata in their own file
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 47388
diff changeset
394
47453
c252631500e4 revlog: add a way to keep track of older uids in the docket
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 47423
diff changeset
395 older_sidedata_uuid_count = next(iheader)
c252631500e4 revlog: add a way to keep track of older uids in the docket
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 47423
diff changeset
396 older_sidedata_uuids = _parse_old_uids(get_data, older_sidedata_uuid_count)
c252631500e4 revlog: add a way to keep track of older uids in the docket
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 47423
diff changeset
397
47325
f286d715f9ab revlogv2: simplify and clarify the processing of each entry
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 47324
diff changeset
398 index_size = next(iheader)
f286d715f9ab revlogv2: simplify and clarify the processing of each entry
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 47324
diff changeset
399
f286d715f9ab revlogv2: simplify and clarify the processing of each entry
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 47324
diff changeset
400 pending_index_size = next(iheader)
f286d715f9ab revlogv2: simplify and clarify the processing of each entry
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 47324
diff changeset
401
f286d715f9ab revlogv2: simplify and clarify the processing of each entry
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 47324
diff changeset
402 data_size = next(iheader)
f286d715f9ab revlogv2: simplify and clarify the processing of each entry
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 47324
diff changeset
403
f286d715f9ab revlogv2: simplify and clarify the processing of each entry
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 47324
diff changeset
404 pending_data_size = next(iheader)
f286d715f9ab revlogv2: simplify and clarify the processing of each entry
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 47324
diff changeset
405
47389
e6292eb33384 revlog: store sidedata in their own file
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 47388
diff changeset
406 sidedata_size = next(iheader)
e6292eb33384 revlog: store sidedata in their own file
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 47388
diff changeset
407
e6292eb33384 revlog: store sidedata in their own file
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 47388
diff changeset
408 pending_sidedata_size = next(iheader)
e6292eb33384 revlog: store sidedata in their own file
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 47388
diff changeset
409
47325
f286d715f9ab revlogv2: simplify and clarify the processing of each entry
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 47324
diff changeset
410 default_compression_header = next(iheader)
f286d715f9ab revlogv2: simplify and clarify the processing of each entry
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 47324
diff changeset
411
47234
616b8f412676 revlogv2: introduce a very basic docket file
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
diff changeset
412 docket = RevlogDocket(
616b8f412676 revlogv2: introduce a very basic docket file
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
diff changeset
413 revlog,
47241
2219853a1503 revlogv2: track pending write in the docket and expose it to hooks
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 47238
diff changeset
414 use_pending=use_pending,
47234
616b8f412676 revlogv2: introduce a very basic docket file
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
diff changeset
415 version_header=version_header,
47323
f612db768c7a revlogv2: use a unique filename for index
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 47312
diff changeset
416 index_uuid=index_uuid,
47453
c252631500e4 revlog: add a way to keep track of older uids in the docket
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 47423
diff changeset
417 older_index_uuids=older_index_uuids,
47324
0a3fa41fa719 revlogv2: use a unique filename for data
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 47323
diff changeset
418 data_uuid=data_uuid,
47453
c252631500e4 revlog: add a way to keep track of older uids in the docket
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 47423
diff changeset
419 older_data_uuids=older_data_uuids,
47389
e6292eb33384 revlog: store sidedata in their own file
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 47388
diff changeset
420 sidedata_uuid=sidedata_uuid,
47453
c252631500e4 revlog: add a way to keep track of older uids in the docket
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 47423
diff changeset
421 older_sidedata_uuids=older_sidedata_uuids,
47238
6597255a4f94 revlogv2: track current index size in the docket
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 47234
diff changeset
422 index_end=index_size,
47241
2219853a1503 revlogv2: track pending write in the docket and expose it to hooks
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 47238
diff changeset
423 pending_index_end=pending_index_size,
47242
4abd474a10af revlogv2: also keep track for the size of the "data" file
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 47241
diff changeset
424 data_end=data_size,
4abd474a10af revlogv2: also keep track for the size of the "data" file
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 47241
diff changeset
425 pending_data_end=pending_data_size,
47389
e6292eb33384 revlog: store sidedata in their own file
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 47388
diff changeset
426 sidedata_end=sidedata_size,
e6292eb33384 revlog: store sidedata in their own file
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 47388
diff changeset
427 pending_sidedata_end=pending_sidedata_size,
47255
ff9fd7107d11 revlog: implement a "default compression" mode
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 47242
diff changeset
428 default_compression_header=default_compression_header,
47234
616b8f412676 revlogv2: introduce a very basic docket file
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
diff changeset
429 )
616b8f412676 revlogv2: introduce a very basic docket file
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
diff changeset
430 return docket