Mercurial > public > mercurial-scm > hg
annotate mercurial/revlogutils/docket.py @ 48178:f12a19d03d2c
fix: reduce number of tool executions
By grouping together (path, ctx) pairs according to the inputs they would
provide to fixer tools, we can deduplicate executions of fixer tools to
significantly reduce the amount of time spent running slow tools.
This change does not handle clean files in the working copy, which could still
be deduplicated against the files in the checked out commit. It's a little
harder to do that because the filerev is not available in the workingfilectx
(and it doesn't exist for added files).
Anecdotally, this change makes some real uses cases at Google 10x faster. I
think we were originally hesitant to do this because the benefits weren't
obvious, and implementing it efficiently is kind of tricky. If we simply
memoized the formatter execution function, we would be keeping tons of file
content in memory.
Also included is a regression test for a corner case that I broke with my first
attempt at optimizing this code.
Differential Revision: https://phab.mercurial-scm.org/D11280
author | Danny Hooper <hooper@google.com> |
---|---|
date | Thu, 02 Sep 2021 14:08:45 -0700 |
parents | f93a867a6486 |
children | 6000f5b25c9b |
rev | line source |
---|---|
47234
616b8f412676
revlogv2: introduce a very basic docket file
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
diff
changeset
|
1 # docket - code related to revlog "docket" |
616b8f412676
revlogv2: introduce a very basic docket file
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
diff
changeset
|
2 # |
616b8f412676
revlogv2: introduce a very basic docket file
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
diff
changeset
|
3 # Copyright 2021 Pierre-Yves David <pierre-yves.david@octobus.net> |
616b8f412676
revlogv2: introduce a very basic docket file
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
diff
changeset
|
4 # |
616b8f412676
revlogv2: introduce a very basic docket file
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
diff
changeset
|
5 # This software may be used and distributed according to the terms of the |
616b8f412676
revlogv2: introduce a very basic docket file
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
diff
changeset
|
6 # GNU General Public License version 2 or any later version. |
616b8f412676
revlogv2: introduce a very basic docket file
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
diff
changeset
|
7 |
616b8f412676
revlogv2: introduce a very basic docket file
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
diff
changeset
|
8 ### Revlog docket file |
616b8f412676
revlogv2: introduce a very basic docket file
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
diff
changeset
|
9 # |
616b8f412676
revlogv2: introduce a very basic docket file
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
diff
changeset
|
10 # The revlog is stored on disk using multiple files: |
616b8f412676
revlogv2: introduce a very basic docket file
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
diff
changeset
|
11 # |
616b8f412676
revlogv2: introduce a very basic docket file
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
diff
changeset
|
12 # * a small docket file, containing metadata and a pointer, |
616b8f412676
revlogv2: introduce a very basic docket file
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
diff
changeset
|
13 # |
616b8f412676
revlogv2: introduce a very basic docket file
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
diff
changeset
|
14 # * an index file, containing fixed width information about revisions, |
616b8f412676
revlogv2: introduce a very basic docket file
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
diff
changeset
|
15 # |
616b8f412676
revlogv2: introduce a very basic docket file
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
diff
changeset
|
16 # * a data file, containing variable width data for these revisions, |
616b8f412676
revlogv2: introduce a very basic docket file
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
diff
changeset
|
17 |
616b8f412676
revlogv2: introduce a very basic docket file
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
diff
changeset
|
18 from __future__ import absolute_import |
616b8f412676
revlogv2: introduce a very basic docket file
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
diff
changeset
|
19 |
47312
7ea39d633cf3
docket: move the uid logic in the `revlogutils.docket` module
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
47266
diff
changeset
|
20 import errno |
7ea39d633cf3
docket: move the uid logic in the `revlogutils.docket` module
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
47266
diff
changeset
|
21 import os |
7ea39d633cf3
docket: move the uid logic in the `revlogutils.docket` module
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
47266
diff
changeset
|
22 import random |
47234
616b8f412676
revlogv2: introduce a very basic docket file
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
diff
changeset
|
23 import struct |
616b8f412676
revlogv2: introduce a very basic docket file
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
diff
changeset
|
24 |
47241
2219853a1503
revlogv2: track pending write in the docket and expose it to hooks
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
47238
diff
changeset
|
25 from .. import ( |
47312
7ea39d633cf3
docket: move the uid logic in the `revlogutils.docket` module
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
47266
diff
changeset
|
26 encoding, |
47241
2219853a1503
revlogv2: track pending write in the docket and expose it to hooks
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
47238
diff
changeset
|
27 error, |
47312
7ea39d633cf3
docket: move the uid logic in the `revlogutils.docket` module
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
47266
diff
changeset
|
28 node, |
7ea39d633cf3
docket: move the uid logic in the `revlogutils.docket` module
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
47266
diff
changeset
|
29 pycompat, |
47255
ff9fd7107d11
revlog: implement a "default compression" mode
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
47242
diff
changeset
|
30 util, |
47241
2219853a1503
revlogv2: track pending write in the docket and expose it to hooks
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
47238
diff
changeset
|
31 ) |
2219853a1503
revlogv2: track pending write in the docket and expose it to hooks
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
47238
diff
changeset
|
32 |
47234
616b8f412676
revlogv2: introduce a very basic docket file
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
diff
changeset
|
33 from . import ( |
616b8f412676
revlogv2: introduce a very basic docket file
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
diff
changeset
|
34 constants, |
616b8f412676
revlogv2: introduce a very basic docket file
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
diff
changeset
|
35 ) |
616b8f412676
revlogv2: introduce a very basic docket file
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
diff
changeset
|
36 |
47312
7ea39d633cf3
docket: move the uid logic in the `revlogutils.docket` module
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
47266
diff
changeset
|
37 |
7ea39d633cf3
docket: move the uid logic in the `revlogutils.docket` module
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
47266
diff
changeset
|
38 def make_uid(id_size=8): |
7ea39d633cf3
docket: move the uid logic in the `revlogutils.docket` module
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
47266
diff
changeset
|
39 """return a new unique identifier. |
7ea39d633cf3
docket: move the uid logic in the `revlogutils.docket` module
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
47266
diff
changeset
|
40 |
7ea39d633cf3
docket: move the uid logic in the `revlogutils.docket` module
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
47266
diff
changeset
|
41 The identifier is random and composed of ascii characters.""" |
7ea39d633cf3
docket: move the uid logic in the `revlogutils.docket` module
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
47266
diff
changeset
|
42 # size we "hex" the result we need half the number of bits to have a final |
7ea39d633cf3
docket: move the uid logic in the `revlogutils.docket` module
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
47266
diff
changeset
|
43 # uuid of size ID_SIZE |
7ea39d633cf3
docket: move the uid logic in the `revlogutils.docket` module
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
47266
diff
changeset
|
44 return node.hex(os.urandom(id_size // 2)) |
7ea39d633cf3
docket: move the uid logic in the `revlogutils.docket` module
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
47266
diff
changeset
|
45 |
7ea39d633cf3
docket: move the uid logic in the `revlogutils.docket` module
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
47266
diff
changeset
|
46 |
7ea39d633cf3
docket: move the uid logic in the `revlogutils.docket` module
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
47266
diff
changeset
|
47 # some special test logic to avoid anoying random output in the test |
7ea39d633cf3
docket: move the uid logic in the `revlogutils.docket` module
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
47266
diff
changeset
|
48 stable_docket_file = encoding.environ.get(b'HGTEST_UUIDFILE') |
7ea39d633cf3
docket: move the uid logic in the `revlogutils.docket` module
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
47266
diff
changeset
|
49 |
7ea39d633cf3
docket: move the uid logic in the `revlogutils.docket` module
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
47266
diff
changeset
|
50 if stable_docket_file: |
7ea39d633cf3
docket: move the uid logic in the `revlogutils.docket` module
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
47266
diff
changeset
|
51 |
7ea39d633cf3
docket: move the uid logic in the `revlogutils.docket` module
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
47266
diff
changeset
|
52 def make_uid(id_size=8): |
7ea39d633cf3
docket: move the uid logic in the `revlogutils.docket` module
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
47266
diff
changeset
|
53 try: |
7ea39d633cf3
docket: move the uid logic in the `revlogutils.docket` module
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
47266
diff
changeset
|
54 with open(stable_docket_file, mode='rb') as f: |
7ea39d633cf3
docket: move the uid logic in the `revlogutils.docket` module
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
47266
diff
changeset
|
55 seed = f.read().strip() |
7ea39d633cf3
docket: move the uid logic in the `revlogutils.docket` module
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
47266
diff
changeset
|
56 except IOError as inst: |
7ea39d633cf3
docket: move the uid logic in the `revlogutils.docket` module
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
47266
diff
changeset
|
57 if inst.errno != errno.ENOENT: |
7ea39d633cf3
docket: move the uid logic in the `revlogutils.docket` module
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
47266
diff
changeset
|
58 raise |
7ea39d633cf3
docket: move the uid logic in the `revlogutils.docket` module
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
47266
diff
changeset
|
59 seed = b'04' # chosen by a fair dice roll. garanteed to be random |
7ea39d633cf3
docket: move the uid logic in the `revlogutils.docket` module
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
47266
diff
changeset
|
60 if pycompat.ispy3: |
7ea39d633cf3
docket: move the uid logic in the `revlogutils.docket` module
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
47266
diff
changeset
|
61 iter_seed = iter(seed) |
7ea39d633cf3
docket: move the uid logic in the `revlogutils.docket` module
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
47266
diff
changeset
|
62 else: |
47423
be903d043099
typing: suppress a false error in mercurial/revlogutils/docket.py on py2
Matt Harbison <matt_harbison@yahoo.com>
parents:
47389
diff
changeset
|
63 # pytype: disable=wrong-arg-types |
47312
7ea39d633cf3
docket: move the uid logic in the `revlogutils.docket` module
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
47266
diff
changeset
|
64 iter_seed = (ord(c) for c in seed) |
47423
be903d043099
typing: suppress a false error in mercurial/revlogutils/docket.py on py2
Matt Harbison <matt_harbison@yahoo.com>
parents:
47389
diff
changeset
|
65 # pytype: enable=wrong-arg-types |
47312
7ea39d633cf3
docket: move the uid logic in the `revlogutils.docket` module
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
47266
diff
changeset
|
66 # some basic circular sum hashing on 64 bits |
7ea39d633cf3
docket: move the uid logic in the `revlogutils.docket` module
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
47266
diff
changeset
|
67 int_seed = 0 |
7ea39d633cf3
docket: move the uid logic in the `revlogutils.docket` module
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
47266
diff
changeset
|
68 low_mask = int('1' * 35, 2) |
7ea39d633cf3
docket: move the uid logic in the `revlogutils.docket` module
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
47266
diff
changeset
|
69 for i in iter_seed: |
7ea39d633cf3
docket: move the uid logic in the `revlogutils.docket` module
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
47266
diff
changeset
|
70 high_part = int_seed >> 35 |
7ea39d633cf3
docket: move the uid logic in the `revlogutils.docket` module
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
47266
diff
changeset
|
71 low_part = (int_seed & low_mask) << 28 |
7ea39d633cf3
docket: move the uid logic in the `revlogutils.docket` module
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
47266
diff
changeset
|
72 int_seed = high_part + low_part + i |
7ea39d633cf3
docket: move the uid logic in the `revlogutils.docket` module
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
47266
diff
changeset
|
73 r = random.Random() |
7ea39d633cf3
docket: move the uid logic in the `revlogutils.docket` module
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
47266
diff
changeset
|
74 if pycompat.ispy3: |
7ea39d633cf3
docket: move the uid logic in the `revlogutils.docket` module
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
47266
diff
changeset
|
75 r.seed(int_seed, version=1) |
7ea39d633cf3
docket: move the uid logic in the `revlogutils.docket` module
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
47266
diff
changeset
|
76 else: |
7ea39d633cf3
docket: move the uid logic in the `revlogutils.docket` module
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
47266
diff
changeset
|
77 r.seed(int_seed) |
7ea39d633cf3
docket: move the uid logic in the `revlogutils.docket` module
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
47266
diff
changeset
|
78 # once we drop python 3.8 support we can simply use r.randbytes |
7ea39d633cf3
docket: move the uid logic in the `revlogutils.docket` module
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
47266
diff
changeset
|
79 raw = r.getrandbits(id_size * 4) |
7ea39d633cf3
docket: move the uid logic in the `revlogutils.docket` module
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
47266
diff
changeset
|
80 assert id_size == 8 |
7ea39d633cf3
docket: move the uid logic in the `revlogutils.docket` module
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
47266
diff
changeset
|
81 p = struct.pack('>L', raw) |
7ea39d633cf3
docket: move the uid logic in the `revlogutils.docket` module
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
47266
diff
changeset
|
82 new = node.hex(p) |
7ea39d633cf3
docket: move the uid logic in the `revlogutils.docket` module
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
47266
diff
changeset
|
83 with open(stable_docket_file, 'wb') as f: |
7ea39d633cf3
docket: move the uid logic in the `revlogutils.docket` module
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
47266
diff
changeset
|
84 f.write(new) |
7ea39d633cf3
docket: move the uid logic in the `revlogutils.docket` module
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
47266
diff
changeset
|
85 return new |
7ea39d633cf3
docket: move the uid logic in the `revlogutils.docket` module
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
47266
diff
changeset
|
86 |
7ea39d633cf3
docket: move the uid logic in the `revlogutils.docket` module
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
47266
diff
changeset
|
87 |
47234
616b8f412676
revlogv2: introduce a very basic docket file
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
diff
changeset
|
88 # Docket format |
616b8f412676
revlogv2: introduce a very basic docket file
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
diff
changeset
|
89 # |
616b8f412676
revlogv2: introduce a very basic docket file
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
diff
changeset
|
90 # * 4 bytes: revlog version |
616b8f412676
revlogv2: introduce a very basic docket file
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
diff
changeset
|
91 # | This is mandatory as docket must be compatible with the previous |
616b8f412676
revlogv2: introduce a very basic docket file
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
diff
changeset
|
92 # | revlog index header. |
47323
f612db768c7a
revlogv2: use a unique filename for index
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
47312
diff
changeset
|
93 # * 1 bytes: size of index uuid |
47453
c252631500e4
revlog: add a way to keep track of older uids in the docket
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
47423
diff
changeset
|
94 # * 1 bytes: number of outdated index uuid |
47324
0a3fa41fa719
revlogv2: use a unique filename for data
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
47323
diff
changeset
|
95 # * 1 bytes: size of data uuid |
47453
c252631500e4
revlog: add a way to keep track of older uids in the docket
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
47423
diff
changeset
|
96 # * 1 bytes: number of outdated data uuid |
47389
e6292eb33384
revlog: store sidedata in their own file
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
47388
diff
changeset
|
97 # * 1 bytes: size of sizedata uuid |
47453
c252631500e4
revlog: add a way to keep track of older uids in the docket
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
47423
diff
changeset
|
98 # * 1 bytes: number of outdated data uuid |
47242
4abd474a10af
revlogv2: also keep track for the size of the "data" file
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
47241
diff
changeset
|
99 # * 8 bytes: size of index-data |
4abd474a10af
revlogv2: also keep track for the size of the "data" file
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
47241
diff
changeset
|
100 # * 8 bytes: pending size of index-data |
4abd474a10af
revlogv2: also keep track for the size of the "data" file
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
47241
diff
changeset
|
101 # * 8 bytes: size of data |
47389
e6292eb33384
revlog: store sidedata in their own file
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
47388
diff
changeset
|
102 # * 8 bytes: size of sidedata |
47242
4abd474a10af
revlogv2: also keep track for the size of the "data" file
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
47241
diff
changeset
|
103 # * 8 bytes: pending size of data |
47389
e6292eb33384
revlog: store sidedata in their own file
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
47388
diff
changeset
|
104 # * 8 bytes: pending size of sidedata |
47255
ff9fd7107d11
revlog: implement a "default compression" mode
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
47242
diff
changeset
|
105 # * 1 bytes: default compression header |
47453
c252631500e4
revlog: add a way to keep track of older uids in the docket
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
47423
diff
changeset
|
106 S_HEADER = struct.Struct(constants.INDEX_HEADER_FMT + b'BBBBBBLLLLLLc') |
c252631500e4
revlog: add a way to keep track of older uids in the docket
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
47423
diff
changeset
|
107 # * 1 bytes: size of index uuid |
c252631500e4
revlog: add a way to keep track of older uids in the docket
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
47423
diff
changeset
|
108 # * 8 bytes: size of file |
c252631500e4
revlog: add a way to keep track of older uids in the docket
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
47423
diff
changeset
|
109 S_OLD_UID = struct.Struct('>BL') |
47234
616b8f412676
revlogv2: introduce a very basic docket file
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
diff
changeset
|
110 |
616b8f412676
revlogv2: introduce a very basic docket file
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
diff
changeset
|
111 |
616b8f412676
revlogv2: introduce a very basic docket file
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
diff
changeset
|
112 class RevlogDocket(object): |
616b8f412676
revlogv2: introduce a very basic docket file
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
diff
changeset
|
113 """metadata associated with revlog""" |
616b8f412676
revlogv2: introduce a very basic docket file
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
diff
changeset
|
114 |
47241
2219853a1503
revlogv2: track pending write in the docket and expose it to hooks
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
47238
diff
changeset
|
115 def __init__( |
2219853a1503
revlogv2: track pending write in the docket and expose it to hooks
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
47238
diff
changeset
|
116 self, |
2219853a1503
revlogv2: track pending write in the docket and expose it to hooks
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
47238
diff
changeset
|
117 revlog, |
2219853a1503
revlogv2: track pending write in the docket and expose it to hooks
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
47238
diff
changeset
|
118 use_pending=False, |
2219853a1503
revlogv2: track pending write in the docket and expose it to hooks
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
47238
diff
changeset
|
119 version_header=None, |
47323
f612db768c7a
revlogv2: use a unique filename for index
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
47312
diff
changeset
|
120 index_uuid=None, |
47453
c252631500e4
revlog: add a way to keep track of older uids in the docket
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
47423
diff
changeset
|
121 older_index_uuids=(), |
47324
0a3fa41fa719
revlogv2: use a unique filename for data
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
47323
diff
changeset
|
122 data_uuid=None, |
47453
c252631500e4
revlog: add a way to keep track of older uids in the docket
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
47423
diff
changeset
|
123 older_data_uuids=(), |
47389
e6292eb33384
revlog: store sidedata in their own file
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
47388
diff
changeset
|
124 sidedata_uuid=None, |
47453
c252631500e4
revlog: add a way to keep track of older uids in the docket
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
47423
diff
changeset
|
125 older_sidedata_uuids=(), |
47241
2219853a1503
revlogv2: track pending write in the docket and expose it to hooks
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
47238
diff
changeset
|
126 index_end=0, |
2219853a1503
revlogv2: track pending write in the docket and expose it to hooks
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
47238
diff
changeset
|
127 pending_index_end=0, |
47242
4abd474a10af
revlogv2: also keep track for the size of the "data" file
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
47241
diff
changeset
|
128 data_end=0, |
4abd474a10af
revlogv2: also keep track for the size of the "data" file
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
47241
diff
changeset
|
129 pending_data_end=0, |
47389
e6292eb33384
revlog: store sidedata in their own file
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
47388
diff
changeset
|
130 sidedata_end=0, |
e6292eb33384
revlog: store sidedata in their own file
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
47388
diff
changeset
|
131 pending_sidedata_end=0, |
47255
ff9fd7107d11
revlog: implement a "default compression" mode
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
47242
diff
changeset
|
132 default_compression_header=None, |
47241
2219853a1503
revlogv2: track pending write in the docket and expose it to hooks
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
47238
diff
changeset
|
133 ): |
47234
616b8f412676
revlogv2: introduce a very basic docket file
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
diff
changeset
|
134 self._version_header = version_header |
47241
2219853a1503
revlogv2: track pending write in the docket and expose it to hooks
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
47238
diff
changeset
|
135 self._read_only = bool(use_pending) |
47234
616b8f412676
revlogv2: introduce a very basic docket file
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
diff
changeset
|
136 self._dirty = False |
616b8f412676
revlogv2: introduce a very basic docket file
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
diff
changeset
|
137 self._radix = revlog.radix |
616b8f412676
revlogv2: introduce a very basic docket file
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
diff
changeset
|
138 self._path = revlog._docket_file |
616b8f412676
revlogv2: introduce a very basic docket file
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
diff
changeset
|
139 self._opener = revlog.opener |
47323
f612db768c7a
revlogv2: use a unique filename for index
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
47312
diff
changeset
|
140 self._index_uuid = index_uuid |
47453
c252631500e4
revlog: add a way to keep track of older uids in the docket
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
47423
diff
changeset
|
141 self._older_index_uuids = older_index_uuids |
47324
0a3fa41fa719
revlogv2: use a unique filename for data
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
47323
diff
changeset
|
142 self._data_uuid = data_uuid |
47453
c252631500e4
revlog: add a way to keep track of older uids in the docket
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
47423
diff
changeset
|
143 self._older_data_uuids = older_data_uuids |
47389
e6292eb33384
revlog: store sidedata in their own file
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
47388
diff
changeset
|
144 self._sidedata_uuid = sidedata_uuid |
47453
c252631500e4
revlog: add a way to keep track of older uids in the docket
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
47423
diff
changeset
|
145 self._older_sidedata_uuids = older_sidedata_uuids |
c252631500e4
revlog: add a way to keep track of older uids in the docket
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
47423
diff
changeset
|
146 assert not set(older_index_uuids) & set(older_data_uuids) |
c252631500e4
revlog: add a way to keep track of older uids in the docket
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
47423
diff
changeset
|
147 assert not set(older_data_uuids) & set(older_sidedata_uuids) |
c252631500e4
revlog: add a way to keep track of older uids in the docket
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
47423
diff
changeset
|
148 assert not set(older_index_uuids) & set(older_sidedata_uuids) |
47242
4abd474a10af
revlogv2: also keep track for the size of the "data" file
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
47241
diff
changeset
|
149 # thes asserts should be True as long as we have a single index filename |
47241
2219853a1503
revlogv2: track pending write in the docket and expose it to hooks
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
47238
diff
changeset
|
150 assert index_end <= pending_index_end |
47242
4abd474a10af
revlogv2: also keep track for the size of the "data" file
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
47241
diff
changeset
|
151 assert data_end <= pending_data_end |
47389
e6292eb33384
revlog: store sidedata in their own file
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
47388
diff
changeset
|
152 assert sidedata_end <= pending_sidedata_end |
47241
2219853a1503
revlogv2: track pending write in the docket and expose it to hooks
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
47238
diff
changeset
|
153 self._initial_index_end = index_end |
2219853a1503
revlogv2: track pending write in the docket and expose it to hooks
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
47238
diff
changeset
|
154 self._pending_index_end = pending_index_end |
47242
4abd474a10af
revlogv2: also keep track for the size of the "data" file
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
47241
diff
changeset
|
155 self._initial_data_end = data_end |
4abd474a10af
revlogv2: also keep track for the size of the "data" file
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
47241
diff
changeset
|
156 self._pending_data_end = pending_data_end |
47389
e6292eb33384
revlog: store sidedata in their own file
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
47388
diff
changeset
|
157 self._initial_sidedata_end = sidedata_end |
e6292eb33384
revlog: store sidedata in their own file
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
47388
diff
changeset
|
158 self._pending_sidedata_end = pending_sidedata_end |
47241
2219853a1503
revlogv2: track pending write in the docket and expose it to hooks
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
47238
diff
changeset
|
159 if use_pending: |
2219853a1503
revlogv2: track pending write in the docket and expose it to hooks
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
47238
diff
changeset
|
160 self._index_end = self._pending_index_end |
47242
4abd474a10af
revlogv2: also keep track for the size of the "data" file
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
47241
diff
changeset
|
161 self._data_end = self._pending_data_end |
47389
e6292eb33384
revlog: store sidedata in their own file
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
47388
diff
changeset
|
162 self._sidedata_end = self._pending_sidedata_end |
47241
2219853a1503
revlogv2: track pending write in the docket and expose it to hooks
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
47238
diff
changeset
|
163 else: |
2219853a1503
revlogv2: track pending write in the docket and expose it to hooks
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
47238
diff
changeset
|
164 self._index_end = self._initial_index_end |
47242
4abd474a10af
revlogv2: also keep track for the size of the "data" file
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
47241
diff
changeset
|
165 self._data_end = self._initial_data_end |
47389
e6292eb33384
revlog: store sidedata in their own file
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
47388
diff
changeset
|
166 self._sidedata_end = self._initial_sidedata_end |
47255
ff9fd7107d11
revlog: implement a "default compression" mode
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
47242
diff
changeset
|
167 self.default_compression_header = default_compression_header |
47234
616b8f412676
revlogv2: introduce a very basic docket file
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
diff
changeset
|
168 |
616b8f412676
revlogv2: introduce a very basic docket file
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
diff
changeset
|
169 def index_filepath(self): |
616b8f412676
revlogv2: introduce a very basic docket file
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
diff
changeset
|
170 """file path to the current index file associated to this docket""" |
616b8f412676
revlogv2: introduce a very basic docket file
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
diff
changeset
|
171 # very simplistic version at first |
47323
f612db768c7a
revlogv2: use a unique filename for index
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
47312
diff
changeset
|
172 if self._index_uuid is None: |
f612db768c7a
revlogv2: use a unique filename for index
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
47312
diff
changeset
|
173 self._index_uuid = make_uid() |
f612db768c7a
revlogv2: use a unique filename for index
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
47312
diff
changeset
|
174 return b"%s-%s.idx" % (self._radix, self._index_uuid) |
47234
616b8f412676
revlogv2: introduce a very basic docket file
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
diff
changeset
|
175 |
47454
865c260d7163
revlog: add docket method to request new content files
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
47453
diff
changeset
|
176 def new_index_file(self): |
865c260d7163
revlog: add docket method to request new content files
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
47453
diff
changeset
|
177 """switch index file to a new UID |
865c260d7163
revlog: add docket method to request new content files
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
47453
diff
changeset
|
178 |
865c260d7163
revlog: add docket method to request new content files
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
47453
diff
changeset
|
179 The previous index UID is moved to the "older" list.""" |
865c260d7163
revlog: add docket method to request new content files
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
47453
diff
changeset
|
180 old = (self._index_uuid, self._index_end) |
865c260d7163
revlog: add docket method to request new content files
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
47453
diff
changeset
|
181 self._older_index_uuids.insert(0, old) |
865c260d7163
revlog: add docket method to request new content files
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
47453
diff
changeset
|
182 self._index_uuid = make_uid() |
865c260d7163
revlog: add docket method to request new content files
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
47453
diff
changeset
|
183 return self.index_filepath() |
865c260d7163
revlog: add docket method to request new content files
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
47453
diff
changeset
|
184 |
47455
f93a867a6486
revlog: list older-but-still-around file in `files`
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
47454
diff
changeset
|
185 def old_index_filepaths(self, include_empty=True): |
f93a867a6486
revlog: list older-but-still-around file in `files`
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
47454
diff
changeset
|
186 """yield file path to older index files associated to this docket""" |
f93a867a6486
revlog: list older-but-still-around file in `files`
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
47454
diff
changeset
|
187 # very simplistic version at first |
f93a867a6486
revlog: list older-but-still-around file in `files`
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
47454
diff
changeset
|
188 for uuid, size in self._older_index_uuids: |
f93a867a6486
revlog: list older-but-still-around file in `files`
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
47454
diff
changeset
|
189 if include_empty or size > 0: |
f93a867a6486
revlog: list older-but-still-around file in `files`
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
47454
diff
changeset
|
190 yield b"%s-%s.idx" % (self._radix, uuid) |
f93a867a6486
revlog: list older-but-still-around file in `files`
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
47454
diff
changeset
|
191 |
47324
0a3fa41fa719
revlogv2: use a unique filename for data
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
47323
diff
changeset
|
192 def data_filepath(self): |
47388
bcf92bdc2bca
revlog: fix docket.date_filepath docstring
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
47371
diff
changeset
|
193 """file path to the current data file associated to this docket""" |
47324
0a3fa41fa719
revlogv2: use a unique filename for data
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
47323
diff
changeset
|
194 # very simplistic version at first |
0a3fa41fa719
revlogv2: use a unique filename for data
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
47323
diff
changeset
|
195 if self._data_uuid is None: |
0a3fa41fa719
revlogv2: use a unique filename for data
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
47323
diff
changeset
|
196 self._data_uuid = make_uid() |
0a3fa41fa719
revlogv2: use a unique filename for data
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
47323
diff
changeset
|
197 return b"%s-%s.dat" % (self._radix, self._data_uuid) |
0a3fa41fa719
revlogv2: use a unique filename for data
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
47323
diff
changeset
|
198 |
47454
865c260d7163
revlog: add docket method to request new content files
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
47453
diff
changeset
|
199 def new_data_file(self): |
865c260d7163
revlog: add docket method to request new content files
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
47453
diff
changeset
|
200 """switch data file to a new UID |
865c260d7163
revlog: add docket method to request new content files
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
47453
diff
changeset
|
201 |
865c260d7163
revlog: add docket method to request new content files
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
47453
diff
changeset
|
202 The previous data UID is moved to the "older" list.""" |
865c260d7163
revlog: add docket method to request new content files
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
47453
diff
changeset
|
203 old = (self._data_uuid, self._data_end) |
865c260d7163
revlog: add docket method to request new content files
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
47453
diff
changeset
|
204 self._older_data_uuids.insert(0, old) |
865c260d7163
revlog: add docket method to request new content files
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
47453
diff
changeset
|
205 self._data_uuid = make_uid() |
865c260d7163
revlog: add docket method to request new content files
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
47453
diff
changeset
|
206 return self.data_filepath() |
865c260d7163
revlog: add docket method to request new content files
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
47453
diff
changeset
|
207 |
47455
f93a867a6486
revlog: list older-but-still-around file in `files`
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
47454
diff
changeset
|
208 def old_data_filepaths(self, include_empty=True): |
f93a867a6486
revlog: list older-but-still-around file in `files`
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
47454
diff
changeset
|
209 """yield file path to older data files associated to this docket""" |
f93a867a6486
revlog: list older-but-still-around file in `files`
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
47454
diff
changeset
|
210 # very simplistic version at first |
f93a867a6486
revlog: list older-but-still-around file in `files`
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
47454
diff
changeset
|
211 for uuid, size in self._older_data_uuids: |
f93a867a6486
revlog: list older-but-still-around file in `files`
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
47454
diff
changeset
|
212 if include_empty or size > 0: |
f93a867a6486
revlog: list older-but-still-around file in `files`
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
47454
diff
changeset
|
213 yield b"%s-%s.dat" % (self._radix, uuid) |
f93a867a6486
revlog: list older-but-still-around file in `files`
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
47454
diff
changeset
|
214 |
47389
e6292eb33384
revlog: store sidedata in their own file
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
47388
diff
changeset
|
215 def sidedata_filepath(self): |
e6292eb33384
revlog: store sidedata in their own file
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
47388
diff
changeset
|
216 """file path to the current sidedata file associated to this docket""" |
e6292eb33384
revlog: store sidedata in their own file
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
47388
diff
changeset
|
217 # very simplistic version at first |
e6292eb33384
revlog: store sidedata in their own file
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
47388
diff
changeset
|
218 if self._sidedata_uuid is None: |
e6292eb33384
revlog: store sidedata in their own file
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
47388
diff
changeset
|
219 self._sidedata_uuid = make_uid() |
e6292eb33384
revlog: store sidedata in their own file
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
47388
diff
changeset
|
220 return b"%s-%s.sda" % (self._radix, self._sidedata_uuid) |
e6292eb33384
revlog: store sidedata in their own file
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
47388
diff
changeset
|
221 |
47454
865c260d7163
revlog: add docket method to request new content files
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
47453
diff
changeset
|
222 def new_sidedata_file(self): |
865c260d7163
revlog: add docket method to request new content files
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
47453
diff
changeset
|
223 """switch sidedata file to a new UID |
865c260d7163
revlog: add docket method to request new content files
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
47453
diff
changeset
|
224 |
865c260d7163
revlog: add docket method to request new content files
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
47453
diff
changeset
|
225 The previous sidedata UID is moved to the "older" list.""" |
865c260d7163
revlog: add docket method to request new content files
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
47453
diff
changeset
|
226 old = (self._sidedata_uuid, self._sidedata_end) |
865c260d7163
revlog: add docket method to request new content files
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
47453
diff
changeset
|
227 self._older_sidedata_uuids.insert(0, old) |
865c260d7163
revlog: add docket method to request new content files
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
47453
diff
changeset
|
228 self._sidedata_uuid = make_uid() |
865c260d7163
revlog: add docket method to request new content files
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
47453
diff
changeset
|
229 return self.sidedata_filepath() |
865c260d7163
revlog: add docket method to request new content files
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
47453
diff
changeset
|
230 |
47455
f93a867a6486
revlog: list older-but-still-around file in `files`
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
47454
diff
changeset
|
231 def old_sidedata_filepaths(self, include_empty=True): |
f93a867a6486
revlog: list older-but-still-around file in `files`
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
47454
diff
changeset
|
232 """yield file path to older sidedata files associated to this docket""" |
f93a867a6486
revlog: list older-but-still-around file in `files`
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
47454
diff
changeset
|
233 # very simplistic version at first |
f93a867a6486
revlog: list older-but-still-around file in `files`
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
47454
diff
changeset
|
234 for uuid, size in self._older_sidedata_uuids: |
f93a867a6486
revlog: list older-but-still-around file in `files`
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
47454
diff
changeset
|
235 if include_empty or size > 0: |
f93a867a6486
revlog: list older-but-still-around file in `files`
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
47454
diff
changeset
|
236 yield b"%s-%s.sda" % (self._radix, uuid) |
f93a867a6486
revlog: list older-but-still-around file in `files`
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
47454
diff
changeset
|
237 |
47238
6597255a4f94
revlogv2: track current index size in the docket
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
47234
diff
changeset
|
238 @property |
6597255a4f94
revlogv2: track current index size in the docket
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
47234
diff
changeset
|
239 def index_end(self): |
6597255a4f94
revlogv2: track current index size in the docket
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
47234
diff
changeset
|
240 return self._index_end |
6597255a4f94
revlogv2: track current index size in the docket
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
47234
diff
changeset
|
241 |
6597255a4f94
revlogv2: track current index size in the docket
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
47234
diff
changeset
|
242 @index_end.setter |
6597255a4f94
revlogv2: track current index size in the docket
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
47234
diff
changeset
|
243 def index_end(self, new_size): |
6597255a4f94
revlogv2: track current index size in the docket
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
47234
diff
changeset
|
244 if new_size != self._index_end: |
6597255a4f94
revlogv2: track current index size in the docket
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
47234
diff
changeset
|
245 self._index_end = new_size |
6597255a4f94
revlogv2: track current index size in the docket
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
47234
diff
changeset
|
246 self._dirty = True |
6597255a4f94
revlogv2: track current index size in the docket
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
47234
diff
changeset
|
247 |
47242
4abd474a10af
revlogv2: also keep track for the size of the "data" file
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
47241
diff
changeset
|
248 @property |
4abd474a10af
revlogv2: also keep track for the size of the "data" file
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
47241
diff
changeset
|
249 def data_end(self): |
4abd474a10af
revlogv2: also keep track for the size of the "data" file
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
47241
diff
changeset
|
250 return self._data_end |
4abd474a10af
revlogv2: also keep track for the size of the "data" file
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
47241
diff
changeset
|
251 |
4abd474a10af
revlogv2: also keep track for the size of the "data" file
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
47241
diff
changeset
|
252 @data_end.setter |
4abd474a10af
revlogv2: also keep track for the size of the "data" file
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
47241
diff
changeset
|
253 def data_end(self, new_size): |
4abd474a10af
revlogv2: also keep track for the size of the "data" file
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
47241
diff
changeset
|
254 if new_size != self._data_end: |
4abd474a10af
revlogv2: also keep track for the size of the "data" file
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
47241
diff
changeset
|
255 self._data_end = new_size |
4abd474a10af
revlogv2: also keep track for the size of the "data" file
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
47241
diff
changeset
|
256 self._dirty = True |
4abd474a10af
revlogv2: also keep track for the size of the "data" file
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
47241
diff
changeset
|
257 |
47389
e6292eb33384
revlog: store sidedata in their own file
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
47388
diff
changeset
|
258 @property |
e6292eb33384
revlog: store sidedata in their own file
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
47388
diff
changeset
|
259 def sidedata_end(self): |
e6292eb33384
revlog: store sidedata in their own file
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
47388
diff
changeset
|
260 return self._sidedata_end |
e6292eb33384
revlog: store sidedata in their own file
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
47388
diff
changeset
|
261 |
e6292eb33384
revlog: store sidedata in their own file
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
47388
diff
changeset
|
262 @sidedata_end.setter |
e6292eb33384
revlog: store sidedata in their own file
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
47388
diff
changeset
|
263 def sidedata_end(self, new_size): |
e6292eb33384
revlog: store sidedata in their own file
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
47388
diff
changeset
|
264 if new_size != self._sidedata_end: |
e6292eb33384
revlog: store sidedata in their own file
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
47388
diff
changeset
|
265 self._sidedata_end = new_size |
e6292eb33384
revlog: store sidedata in their own file
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
47388
diff
changeset
|
266 self._dirty = True |
e6292eb33384
revlog: store sidedata in their own file
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
47388
diff
changeset
|
267 |
47241
2219853a1503
revlogv2: track pending write in the docket and expose it to hooks
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
47238
diff
changeset
|
268 def write(self, transaction, pending=False, stripping=False): |
47234
616b8f412676
revlogv2: introduce a very basic docket file
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
diff
changeset
|
269 """write the modification of disk if any |
616b8f412676
revlogv2: introduce a very basic docket file
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
diff
changeset
|
270 |
616b8f412676
revlogv2: introduce a very basic docket file
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
diff
changeset
|
271 This make the new content visible to all process""" |
47241
2219853a1503
revlogv2: track pending write in the docket and expose it to hooks
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
47238
diff
changeset
|
272 if not self._dirty: |
2219853a1503
revlogv2: track pending write in the docket and expose it to hooks
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
47238
diff
changeset
|
273 return False |
2219853a1503
revlogv2: track pending write in the docket and expose it to hooks
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
47238
diff
changeset
|
274 else: |
2219853a1503
revlogv2: track pending write in the docket and expose it to hooks
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
47238
diff
changeset
|
275 if self._read_only: |
2219853a1503
revlogv2: track pending write in the docket and expose it to hooks
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
47238
diff
changeset
|
276 msg = b'writing read-only docket: %s' |
2219853a1503
revlogv2: track pending write in the docket and expose it to hooks
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
47238
diff
changeset
|
277 msg %= self._path |
2219853a1503
revlogv2: track pending write in the docket and expose it to hooks
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
47238
diff
changeset
|
278 raise error.ProgrammingError(msg) |
47238
6597255a4f94
revlogv2: track current index size in the docket
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
47234
diff
changeset
|
279 if not stripping: |
6597255a4f94
revlogv2: track current index size in the docket
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
47234
diff
changeset
|
280 # XXX we could, leverage the docket while stripping. However it |
6597255a4f94
revlogv2: track current index size in the docket
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
47234
diff
changeset
|
281 # is not powerfull enough at the time of this comment |
6597255a4f94
revlogv2: track current index size in the docket
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
47234
diff
changeset
|
282 transaction.addbackup(self._path, location=b'store') |
47234
616b8f412676
revlogv2: introduce a very basic docket file
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
diff
changeset
|
283 with self._opener(self._path, mode=b'w', atomictemp=True) as f: |
47241
2219853a1503
revlogv2: track pending write in the docket and expose it to hooks
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
47238
diff
changeset
|
284 f.write(self._serialize(pending=pending)) |
2219853a1503
revlogv2: track pending write in the docket and expose it to hooks
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
47238
diff
changeset
|
285 # if pending we still need to the write final data eventually |
2219853a1503
revlogv2: track pending write in the docket and expose it to hooks
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
47238
diff
changeset
|
286 self._dirty = pending |
2219853a1503
revlogv2: track pending write in the docket and expose it to hooks
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
47238
diff
changeset
|
287 return True |
47234
616b8f412676
revlogv2: introduce a very basic docket file
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
diff
changeset
|
288 |
47241
2219853a1503
revlogv2: track pending write in the docket and expose it to hooks
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
47238
diff
changeset
|
289 def _serialize(self, pending=False): |
2219853a1503
revlogv2: track pending write in the docket and expose it to hooks
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
47238
diff
changeset
|
290 if pending: |
2219853a1503
revlogv2: track pending write in the docket and expose it to hooks
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
47238
diff
changeset
|
291 official_index_end = self._initial_index_end |
47242
4abd474a10af
revlogv2: also keep track for the size of the "data" file
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
47241
diff
changeset
|
292 official_data_end = self._initial_data_end |
47389
e6292eb33384
revlog: store sidedata in their own file
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
47388
diff
changeset
|
293 official_sidedata_end = self._initial_sidedata_end |
47241
2219853a1503
revlogv2: track pending write in the docket and expose it to hooks
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
47238
diff
changeset
|
294 else: |
2219853a1503
revlogv2: track pending write in the docket and expose it to hooks
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
47238
diff
changeset
|
295 official_index_end = self._index_end |
47242
4abd474a10af
revlogv2: also keep track for the size of the "data" file
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
47241
diff
changeset
|
296 official_data_end = self._data_end |
47389
e6292eb33384
revlog: store sidedata in their own file
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
47388
diff
changeset
|
297 official_sidedata_end = self._sidedata_end |
47241
2219853a1503
revlogv2: track pending write in the docket and expose it to hooks
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
47238
diff
changeset
|
298 |
2219853a1503
revlogv2: track pending write in the docket and expose it to hooks
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
47238
diff
changeset
|
299 # this assert should be True as long as we have a single index filename |
47242
4abd474a10af
revlogv2: also keep track for the size of the "data" file
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
47241
diff
changeset
|
300 assert official_data_end <= self._data_end |
47389
e6292eb33384
revlog: store sidedata in their own file
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
47388
diff
changeset
|
301 assert official_sidedata_end <= self._sidedata_end |
47238
6597255a4f94
revlogv2: track current index size in the docket
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
47234
diff
changeset
|
302 data = ( |
6597255a4f94
revlogv2: track current index size in the docket
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
47234
diff
changeset
|
303 self._version_header, |
47323
f612db768c7a
revlogv2: use a unique filename for index
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
47312
diff
changeset
|
304 len(self._index_uuid), |
47453
c252631500e4
revlog: add a way to keep track of older uids in the docket
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
47423
diff
changeset
|
305 len(self._older_index_uuids), |
47324
0a3fa41fa719
revlogv2: use a unique filename for data
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
47323
diff
changeset
|
306 len(self._data_uuid), |
47453
c252631500e4
revlog: add a way to keep track of older uids in the docket
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
47423
diff
changeset
|
307 len(self._older_data_uuids), |
47389
e6292eb33384
revlog: store sidedata in their own file
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
47388
diff
changeset
|
308 len(self._sidedata_uuid), |
47453
c252631500e4
revlog: add a way to keep track of older uids in the docket
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
47423
diff
changeset
|
309 len(self._older_sidedata_uuids), |
47241
2219853a1503
revlogv2: track pending write in the docket and expose it to hooks
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
47238
diff
changeset
|
310 official_index_end, |
47238
6597255a4f94
revlogv2: track current index size in the docket
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
47234
diff
changeset
|
311 self._index_end, |
47242
4abd474a10af
revlogv2: also keep track for the size of the "data" file
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
47241
diff
changeset
|
312 official_data_end, |
4abd474a10af
revlogv2: also keep track for the size of the "data" file
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
47241
diff
changeset
|
313 self._data_end, |
47389
e6292eb33384
revlog: store sidedata in their own file
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
47388
diff
changeset
|
314 official_sidedata_end, |
e6292eb33384
revlog: store sidedata in their own file
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
47388
diff
changeset
|
315 self._sidedata_end, |
47255
ff9fd7107d11
revlog: implement a "default compression" mode
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
47242
diff
changeset
|
316 self.default_compression_header, |
47238
6597255a4f94
revlogv2: track current index size in the docket
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
47234
diff
changeset
|
317 ) |
47323
f612db768c7a
revlogv2: use a unique filename for index
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
47312
diff
changeset
|
318 s = [] |
f612db768c7a
revlogv2: use a unique filename for index
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
47312
diff
changeset
|
319 s.append(S_HEADER.pack(*data)) |
47453
c252631500e4
revlog: add a way to keep track of older uids in the docket
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
47423
diff
changeset
|
320 |
47323
f612db768c7a
revlogv2: use a unique filename for index
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
47312
diff
changeset
|
321 s.append(self._index_uuid) |
47453
c252631500e4
revlog: add a way to keep track of older uids in the docket
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
47423
diff
changeset
|
322 for u, size in self._older_index_uuids: |
c252631500e4
revlog: add a way to keep track of older uids in the docket
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
47423
diff
changeset
|
323 s.append(S_OLD_UID.pack(len(u), size)) |
c252631500e4
revlog: add a way to keep track of older uids in the docket
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
47423
diff
changeset
|
324 for u, size in self._older_index_uuids: |
c252631500e4
revlog: add a way to keep track of older uids in the docket
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
47423
diff
changeset
|
325 s.append(u) |
c252631500e4
revlog: add a way to keep track of older uids in the docket
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
47423
diff
changeset
|
326 |
47324
0a3fa41fa719
revlogv2: use a unique filename for data
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
47323
diff
changeset
|
327 s.append(self._data_uuid) |
47453
c252631500e4
revlog: add a way to keep track of older uids in the docket
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
47423
diff
changeset
|
328 for u, size in self._older_data_uuids: |
c252631500e4
revlog: add a way to keep track of older uids in the docket
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
47423
diff
changeset
|
329 s.append(S_OLD_UID.pack(len(u), size)) |
c252631500e4
revlog: add a way to keep track of older uids in the docket
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
47423
diff
changeset
|
330 for u, size in self._older_data_uuids: |
c252631500e4
revlog: add a way to keep track of older uids in the docket
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
47423
diff
changeset
|
331 s.append(u) |
c252631500e4
revlog: add a way to keep track of older uids in the docket
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
47423
diff
changeset
|
332 |
47389
e6292eb33384
revlog: store sidedata in their own file
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
47388
diff
changeset
|
333 s.append(self._sidedata_uuid) |
47453
c252631500e4
revlog: add a way to keep track of older uids in the docket
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
47423
diff
changeset
|
334 for u, size in self._older_sidedata_uuids: |
c252631500e4
revlog: add a way to keep track of older uids in the docket
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
47423
diff
changeset
|
335 s.append(S_OLD_UID.pack(len(u), size)) |
c252631500e4
revlog: add a way to keep track of older uids in the docket
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
47423
diff
changeset
|
336 for u, size in self._older_sidedata_uuids: |
c252631500e4
revlog: add a way to keep track of older uids in the docket
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
47423
diff
changeset
|
337 s.append(u) |
47323
f612db768c7a
revlogv2: use a unique filename for index
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
47312
diff
changeset
|
338 return b''.join(s) |
47234
616b8f412676
revlogv2: introduce a very basic docket file
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
diff
changeset
|
339 |
616b8f412676
revlogv2: introduce a very basic docket file
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
diff
changeset
|
340 |
616b8f412676
revlogv2: introduce a very basic docket file
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
diff
changeset
|
341 def default_docket(revlog, version_header): |
616b8f412676
revlogv2: introduce a very basic docket file
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
diff
changeset
|
342 """given a revlog version a new docket object for the given revlog""" |
47266
921648d31553
changelogv2: use a dedicated version number
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
47255
diff
changeset
|
343 rl_version = version_header & 0xFFFF |
921648d31553
changelogv2: use a dedicated version number
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
47255
diff
changeset
|
344 if rl_version not in (constants.REVLOGV2, constants.CHANGELOGV2): |
47234
616b8f412676
revlogv2: introduce a very basic docket file
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
diff
changeset
|
345 return None |
47255
ff9fd7107d11
revlog: implement a "default compression" mode
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
47242
diff
changeset
|
346 comp = util.compengines[revlog._compengine].revlogheader() |
ff9fd7107d11
revlog: implement a "default compression" mode
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
47242
diff
changeset
|
347 docket = RevlogDocket( |
ff9fd7107d11
revlog: implement a "default compression" mode
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
47242
diff
changeset
|
348 revlog, |
ff9fd7107d11
revlog: implement a "default compression" mode
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
47242
diff
changeset
|
349 version_header=version_header, |
ff9fd7107d11
revlog: implement a "default compression" mode
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
47242
diff
changeset
|
350 default_compression_header=comp, |
ff9fd7107d11
revlog: implement a "default compression" mode
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
47242
diff
changeset
|
351 ) |
47234
616b8f412676
revlogv2: introduce a very basic docket file
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
diff
changeset
|
352 docket._dirty = True |
616b8f412676
revlogv2: introduce a very basic docket file
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
diff
changeset
|
353 return docket |
616b8f412676
revlogv2: introduce a very basic docket file
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
diff
changeset
|
354 |
616b8f412676
revlogv2: introduce a very basic docket file
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
diff
changeset
|
355 |
47453
c252631500e4
revlog: add a way to keep track of older uids in the docket
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
47423
diff
changeset
|
356 def _parse_old_uids(get_data, count): |
c252631500e4
revlog: add a way to keep track of older uids in the docket
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
47423
diff
changeset
|
357 all_sizes = [] |
c252631500e4
revlog: add a way to keep track of older uids in the docket
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
47423
diff
changeset
|
358 all_uids = [] |
c252631500e4
revlog: add a way to keep track of older uids in the docket
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
47423
diff
changeset
|
359 for i in range(0, count): |
c252631500e4
revlog: add a way to keep track of older uids in the docket
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
47423
diff
changeset
|
360 raw = get_data(S_OLD_UID.size) |
c252631500e4
revlog: add a way to keep track of older uids in the docket
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
47423
diff
changeset
|
361 all_sizes.append(S_OLD_UID.unpack(raw)) |
c252631500e4
revlog: add a way to keep track of older uids in the docket
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
47423
diff
changeset
|
362 |
c252631500e4
revlog: add a way to keep track of older uids in the docket
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
47423
diff
changeset
|
363 for uid_size, file_size in all_sizes: |
c252631500e4
revlog: add a way to keep track of older uids in the docket
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
47423
diff
changeset
|
364 uid = get_data(uid_size) |
c252631500e4
revlog: add a way to keep track of older uids in the docket
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
47423
diff
changeset
|
365 all_uids.append((uid, file_size)) |
c252631500e4
revlog: add a way to keep track of older uids in the docket
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
47423
diff
changeset
|
366 return all_uids |
c252631500e4
revlog: add a way to keep track of older uids in the docket
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
47423
diff
changeset
|
367 |
c252631500e4
revlog: add a way to keep track of older uids in the docket
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
47423
diff
changeset
|
368 |
47241
2219853a1503
revlogv2: track pending write in the docket and expose it to hooks
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
47238
diff
changeset
|
369 def parse_docket(revlog, data, use_pending=False): |
47234
616b8f412676
revlogv2: introduce a very basic docket file
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
diff
changeset
|
370 """given some docket data return a docket object for the given revlog""" |
616b8f412676
revlogv2: introduce a very basic docket file
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
diff
changeset
|
371 header = S_HEADER.unpack(data[: S_HEADER.size]) |
47326
53ab13d6a5db
revlogv2: add a `get_data` helper to grab the next piece of docket
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
47325
diff
changeset
|
372 |
53ab13d6a5db
revlogv2: add a `get_data` helper to grab the next piece of docket
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
47325
diff
changeset
|
373 # this is a mutable closure capture used in `get_data` |
53ab13d6a5db
revlogv2: add a `get_data` helper to grab the next piece of docket
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
47325
diff
changeset
|
374 offset = [S_HEADER.size] |
53ab13d6a5db
revlogv2: add a `get_data` helper to grab the next piece of docket
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
47325
diff
changeset
|
375 |
53ab13d6a5db
revlogv2: add a `get_data` helper to grab the next piece of docket
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
47325
diff
changeset
|
376 def get_data(size): |
53ab13d6a5db
revlogv2: add a `get_data` helper to grab the next piece of docket
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
47325
diff
changeset
|
377 """utility closure to access the `size` next bytes""" |
53ab13d6a5db
revlogv2: add a `get_data` helper to grab the next piece of docket
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
47325
diff
changeset
|
378 if offset[0] + size > len(data): |
53ab13d6a5db
revlogv2: add a `get_data` helper to grab the next piece of docket
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
47325
diff
changeset
|
379 # XXX better class |
53ab13d6a5db
revlogv2: add a `get_data` helper to grab the next piece of docket
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
47325
diff
changeset
|
380 msg = b"docket is too short, expected %d got %d" |
53ab13d6a5db
revlogv2: add a `get_data` helper to grab the next piece of docket
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
47325
diff
changeset
|
381 msg %= (offset[0] + size, len(data)) |
53ab13d6a5db
revlogv2: add a `get_data` helper to grab the next piece of docket
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
47325
diff
changeset
|
382 raise error.Abort(msg) |
53ab13d6a5db
revlogv2: add a `get_data` helper to grab the next piece of docket
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
47325
diff
changeset
|
383 raw = data[offset[0] : offset[0] + size] |
53ab13d6a5db
revlogv2: add a `get_data` helper to grab the next piece of docket
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
47325
diff
changeset
|
384 offset[0] += size |
53ab13d6a5db
revlogv2: add a `get_data` helper to grab the next piece of docket
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
47325
diff
changeset
|
385 return raw |
47325
f286d715f9ab
revlogv2: simplify and clarify the processing of each entry
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
47324
diff
changeset
|
386 |
f286d715f9ab
revlogv2: simplify and clarify the processing of each entry
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
47324
diff
changeset
|
387 iheader = iter(header) |
f286d715f9ab
revlogv2: simplify and clarify the processing of each entry
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
47324
diff
changeset
|
388 |
f286d715f9ab
revlogv2: simplify and clarify the processing of each entry
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
47324
diff
changeset
|
389 version_header = next(iheader) |
f286d715f9ab
revlogv2: simplify and clarify the processing of each entry
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
47324
diff
changeset
|
390 |
f286d715f9ab
revlogv2: simplify and clarify the processing of each entry
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
47324
diff
changeset
|
391 index_uuid_size = next(iheader) |
47326
53ab13d6a5db
revlogv2: add a `get_data` helper to grab the next piece of docket
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
47325
diff
changeset
|
392 index_uuid = get_data(index_uuid_size) |
47325
f286d715f9ab
revlogv2: simplify and clarify the processing of each entry
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
47324
diff
changeset
|
393 |
47453
c252631500e4
revlog: add a way to keep track of older uids in the docket
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
47423
diff
changeset
|
394 older_index_uuid_count = next(iheader) |
c252631500e4
revlog: add a way to keep track of older uids in the docket
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
47423
diff
changeset
|
395 older_index_uuids = _parse_old_uids(get_data, older_index_uuid_count) |
c252631500e4
revlog: add a way to keep track of older uids in the docket
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
47423
diff
changeset
|
396 |
47325
f286d715f9ab
revlogv2: simplify and clarify the processing of each entry
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
47324
diff
changeset
|
397 data_uuid_size = next(iheader) |
47326
53ab13d6a5db
revlogv2: add a `get_data` helper to grab the next piece of docket
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
47325
diff
changeset
|
398 data_uuid = get_data(data_uuid_size) |
47325
f286d715f9ab
revlogv2: simplify and clarify the processing of each entry
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
47324
diff
changeset
|
399 |
47453
c252631500e4
revlog: add a way to keep track of older uids in the docket
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
47423
diff
changeset
|
400 older_data_uuid_count = next(iheader) |
c252631500e4
revlog: add a way to keep track of older uids in the docket
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
47423
diff
changeset
|
401 older_data_uuids = _parse_old_uids(get_data, older_data_uuid_count) |
c252631500e4
revlog: add a way to keep track of older uids in the docket
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
47423
diff
changeset
|
402 |
47389
e6292eb33384
revlog: store sidedata in their own file
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
47388
diff
changeset
|
403 sidedata_uuid_size = next(iheader) |
e6292eb33384
revlog: store sidedata in their own file
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
47388
diff
changeset
|
404 sidedata_uuid = get_data(sidedata_uuid_size) |
e6292eb33384
revlog: store sidedata in their own file
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
47388
diff
changeset
|
405 |
47453
c252631500e4
revlog: add a way to keep track of older uids in the docket
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
47423
diff
changeset
|
406 older_sidedata_uuid_count = next(iheader) |
c252631500e4
revlog: add a way to keep track of older uids in the docket
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
47423
diff
changeset
|
407 older_sidedata_uuids = _parse_old_uids(get_data, older_sidedata_uuid_count) |
c252631500e4
revlog: add a way to keep track of older uids in the docket
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
47423
diff
changeset
|
408 |
47325
f286d715f9ab
revlogv2: simplify and clarify the processing of each entry
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
47324
diff
changeset
|
409 index_size = next(iheader) |
f286d715f9ab
revlogv2: simplify and clarify the processing of each entry
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
47324
diff
changeset
|
410 |
f286d715f9ab
revlogv2: simplify and clarify the processing of each entry
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
47324
diff
changeset
|
411 pending_index_size = next(iheader) |
f286d715f9ab
revlogv2: simplify and clarify the processing of each entry
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
47324
diff
changeset
|
412 |
f286d715f9ab
revlogv2: simplify and clarify the processing of each entry
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
47324
diff
changeset
|
413 data_size = next(iheader) |
f286d715f9ab
revlogv2: simplify and clarify the processing of each entry
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
47324
diff
changeset
|
414 |
f286d715f9ab
revlogv2: simplify and clarify the processing of each entry
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
47324
diff
changeset
|
415 pending_data_size = next(iheader) |
f286d715f9ab
revlogv2: simplify and clarify the processing of each entry
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
47324
diff
changeset
|
416 |
47389
e6292eb33384
revlog: store sidedata in their own file
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
47388
diff
changeset
|
417 sidedata_size = next(iheader) |
e6292eb33384
revlog: store sidedata in their own file
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
47388
diff
changeset
|
418 |
e6292eb33384
revlog: store sidedata in their own file
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
47388
diff
changeset
|
419 pending_sidedata_size = next(iheader) |
e6292eb33384
revlog: store sidedata in their own file
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
47388
diff
changeset
|
420 |
47325
f286d715f9ab
revlogv2: simplify and clarify the processing of each entry
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
47324
diff
changeset
|
421 default_compression_header = next(iheader) |
f286d715f9ab
revlogv2: simplify and clarify the processing of each entry
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
47324
diff
changeset
|
422 |
47234
616b8f412676
revlogv2: introduce a very basic docket file
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
diff
changeset
|
423 docket = RevlogDocket( |
616b8f412676
revlogv2: introduce a very basic docket file
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
diff
changeset
|
424 revlog, |
47241
2219853a1503
revlogv2: track pending write in the docket and expose it to hooks
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
47238
diff
changeset
|
425 use_pending=use_pending, |
47234
616b8f412676
revlogv2: introduce a very basic docket file
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
diff
changeset
|
426 version_header=version_header, |
47323
f612db768c7a
revlogv2: use a unique filename for index
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
47312
diff
changeset
|
427 index_uuid=index_uuid, |
47453
c252631500e4
revlog: add a way to keep track of older uids in the docket
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
47423
diff
changeset
|
428 older_index_uuids=older_index_uuids, |
47324
0a3fa41fa719
revlogv2: use a unique filename for data
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
47323
diff
changeset
|
429 data_uuid=data_uuid, |
47453
c252631500e4
revlog: add a way to keep track of older uids in the docket
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
47423
diff
changeset
|
430 older_data_uuids=older_data_uuids, |
47389
e6292eb33384
revlog: store sidedata in their own file
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
47388
diff
changeset
|
431 sidedata_uuid=sidedata_uuid, |
47453
c252631500e4
revlog: add a way to keep track of older uids in the docket
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
47423
diff
changeset
|
432 older_sidedata_uuids=older_sidedata_uuids, |
47238
6597255a4f94
revlogv2: track current index size in the docket
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
47234
diff
changeset
|
433 index_end=index_size, |
47241
2219853a1503
revlogv2: track pending write in the docket and expose it to hooks
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
47238
diff
changeset
|
434 pending_index_end=pending_index_size, |
47242
4abd474a10af
revlogv2: also keep track for the size of the "data" file
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
47241
diff
changeset
|
435 data_end=data_size, |
4abd474a10af
revlogv2: also keep track for the size of the "data" file
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
47241
diff
changeset
|
436 pending_data_end=pending_data_size, |
47389
e6292eb33384
revlog: store sidedata in their own file
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
47388
diff
changeset
|
437 sidedata_end=sidedata_size, |
e6292eb33384
revlog: store sidedata in their own file
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
47388
diff
changeset
|
438 pending_sidedata_end=pending_sidedata_size, |
47255
ff9fd7107d11
revlog: implement a "default compression" mode
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
47242
diff
changeset
|
439 default_compression_header=default_compression_header, |
47234
616b8f412676
revlogv2: introduce a very basic docket file
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
diff
changeset
|
440 ) |
616b8f412676
revlogv2: introduce a very basic docket file
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
diff
changeset
|
441 return docket |