Mercurial > public > mercurial-scm > hg-stable

annotate mercurial/revlogutils/docket.py @ 47249:6597255a4f94

Find changesets by keywords (author, files, the commit message), revision number or hash, or revset expression.
revlogv2: track current index size in the docket This help use to fix transaction safety on repos. See next changesets for details. Differential Revision: https://phab.mercurial-scm.org/D10628
author Pierre-Yves David <pierre-yves.david@octobus.net>
date Mon, 03 May 2021 12:34:52 +0200
parents 616b8f412676
children 2219853a1503
Ignore whitespace changes - Everywhere: Within whitespace: At end of lines:
rev   line source
47245
616b8f412676 revlogv2: introduce a very basic docket file
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
diff changeset
1 # docket - code related to revlog "docket"
616b8f412676 revlogv2: introduce a very basic docket file
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
diff changeset
2 #
616b8f412676 revlogv2: introduce a very basic docket file
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
diff changeset
3 # Copyright 2021 Pierre-Yves David <pierre-yves.david@octobus.net>
616b8f412676 revlogv2: introduce a very basic docket file
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
diff changeset
4 #
616b8f412676 revlogv2: introduce a very basic docket file
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
diff changeset
5 # This software may be used and distributed according to the terms of the
616b8f412676 revlogv2: introduce a very basic docket file
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
diff changeset
6 # GNU General Public License version 2 or any later version.
616b8f412676 revlogv2: introduce a very basic docket file
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
diff changeset
7
616b8f412676 revlogv2: introduce a very basic docket file
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
diff changeset
8 ### Revlog docket file
616b8f412676 revlogv2: introduce a very basic docket file
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
diff changeset
9 #
616b8f412676 revlogv2: introduce a very basic docket file
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
diff changeset
10 # The revlog is stored on disk using multiple files:
616b8f412676 revlogv2: introduce a very basic docket file
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
diff changeset
11 #
616b8f412676 revlogv2: introduce a very basic docket file
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
diff changeset
12 # * a small docket file, containing metadata and a pointer,
616b8f412676 revlogv2: introduce a very basic docket file
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
diff changeset
13 #
616b8f412676 revlogv2: introduce a very basic docket file
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
diff changeset
14 # * an index file, containing fixed width information about revisions,
616b8f412676 revlogv2: introduce a very basic docket file
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
diff changeset
15 #
616b8f412676 revlogv2: introduce a very basic docket file
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
diff changeset
16 # * a data file, containing variable width data for these revisions,
616b8f412676 revlogv2: introduce a very basic docket file
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
diff changeset
17
616b8f412676 revlogv2: introduce a very basic docket file
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
diff changeset
18 from __future__ import absolute_import
616b8f412676 revlogv2: introduce a very basic docket file
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
diff changeset
19
616b8f412676 revlogv2: introduce a very basic docket file
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
diff changeset
20 import struct
616b8f412676 revlogv2: introduce a very basic docket file
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
diff changeset
21
616b8f412676 revlogv2: introduce a very basic docket file
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
diff changeset
22 from . import (
616b8f412676 revlogv2: introduce a very basic docket file
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
diff changeset
23 constants,
616b8f412676 revlogv2: introduce a very basic docket file
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
diff changeset
24 )
616b8f412676 revlogv2: introduce a very basic docket file
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
diff changeset
25
616b8f412676 revlogv2: introduce a very basic docket file
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
diff changeset
26 # Docket format
616b8f412676 revlogv2: introduce a very basic docket file
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
diff changeset
27 #
616b8f412676 revlogv2: introduce a very basic docket file
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
diff changeset
28 # * 4 bytes: revlog version
616b8f412676 revlogv2: introduce a very basic docket file
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
diff changeset
29 # | This is mandatory as docket must be compatible with the previous
616b8f412676 revlogv2: introduce a very basic docket file
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
diff changeset
30 # | revlog index header.
47249
6597255a4f94 revlogv2: track current index size in the docket
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 47245
diff changeset
31 # * 8 bytes: size of index data
6597255a4f94 revlogv2: track current index size in the docket
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 47245
diff changeset
32 S_HEADER = struct.Struct(constants.INDEX_HEADER.format + 'L')
47245
616b8f412676 revlogv2: introduce a very basic docket file
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
diff changeset
33
616b8f412676 revlogv2: introduce a very basic docket file
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
diff changeset
34
616b8f412676 revlogv2: introduce a very basic docket file
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
diff changeset
35 class RevlogDocket(object):
616b8f412676 revlogv2: introduce a very basic docket file
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
diff changeset
36 """metadata associated with revlog"""
616b8f412676 revlogv2: introduce a very basic docket file
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
diff changeset
37
47249
6597255a4f94 revlogv2: track current index size in the docket
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 47245
diff changeset
38 def __init__(self, revlog, version_header=None, index_end=0):
47245
616b8f412676 revlogv2: introduce a very basic docket file
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
diff changeset
39 self._version_header = version_header
616b8f412676 revlogv2: introduce a very basic docket file
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
diff changeset
40 self._dirty = False
616b8f412676 revlogv2: introduce a very basic docket file
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
diff changeset
41 self._radix = revlog.radix
616b8f412676 revlogv2: introduce a very basic docket file
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
diff changeset
42 self._path = revlog._docket_file
616b8f412676 revlogv2: introduce a very basic docket file
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
diff changeset
43 self._opener = revlog.opener
47249
6597255a4f94 revlogv2: track current index size in the docket
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 47245
diff changeset
44 self._index_end = index_end
47245
616b8f412676 revlogv2: introduce a very basic docket file
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
diff changeset
45
616b8f412676 revlogv2: introduce a very basic docket file
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
diff changeset
46 def index_filepath(self):
616b8f412676 revlogv2: introduce a very basic docket file
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
diff changeset
47 """file path to the current index file associated to this docket"""
616b8f412676 revlogv2: introduce a very basic docket file
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
diff changeset
48 # very simplistic version at first
616b8f412676 revlogv2: introduce a very basic docket file
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
diff changeset
49 return b"%s.idx" % self._radix
616b8f412676 revlogv2: introduce a very basic docket file
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
diff changeset
50
47249
6597255a4f94 revlogv2: track current index size in the docket
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 47245
diff changeset
51 @property
6597255a4f94 revlogv2: track current index size in the docket
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 47245
diff changeset
52 def index_end(self):
6597255a4f94 revlogv2: track current index size in the docket
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 47245
diff changeset
53 return self._index_end
6597255a4f94 revlogv2: track current index size in the docket
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 47245
diff changeset
54
6597255a4f94 revlogv2: track current index size in the docket
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 47245
diff changeset
55 @index_end.setter
6597255a4f94 revlogv2: track current index size in the docket
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 47245
diff changeset
56 def index_end(self, new_size):
6597255a4f94 revlogv2: track current index size in the docket
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 47245
diff changeset
57 if new_size != self._index_end:
6597255a4f94 revlogv2: track current index size in the docket
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 47245
diff changeset
58 self._index_end = new_size
6597255a4f94 revlogv2: track current index size in the docket
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 47245
diff changeset
59 self._dirty = True
6597255a4f94 revlogv2: track current index size in the docket
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 47245
diff changeset
60
6597255a4f94 revlogv2: track current index size in the docket
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 47245
diff changeset
61 def write(self, transaction, stripping=False):
47245
616b8f412676 revlogv2: introduce a very basic docket file
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
diff changeset
62 """write the modification of disk if any
616b8f412676 revlogv2: introduce a very basic docket file
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
diff changeset
63
616b8f412676 revlogv2: introduce a very basic docket file
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
diff changeset
64 This make the new content visible to all process"""
616b8f412676 revlogv2: introduce a very basic docket file
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
diff changeset
65 if self._dirty:
47249
6597255a4f94 revlogv2: track current index size in the docket
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 47245
diff changeset
66 if not stripping:
6597255a4f94 revlogv2: track current index size in the docket
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 47245
diff changeset
67 # XXX we could, leverage the docket while stripping. However it
6597255a4f94 revlogv2: track current index size in the docket
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 47245
diff changeset
68 # is not powerfull enough at the time of this comment
6597255a4f94 revlogv2: track current index size in the docket
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 47245
diff changeset
69 transaction.addbackup(self._path, location=b'store')
47245
616b8f412676 revlogv2: introduce a very basic docket file
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
diff changeset
70 with self._opener(self._path, mode=b'w', atomictemp=True) as f:
616b8f412676 revlogv2: introduce a very basic docket file
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
diff changeset
71 f.write(self._serialize())
616b8f412676 revlogv2: introduce a very basic docket file
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
diff changeset
72 self._dirty = False
616b8f412676 revlogv2: introduce a very basic docket file
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
diff changeset
73
616b8f412676 revlogv2: introduce a very basic docket file
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
diff changeset
74 def _serialize(self):
47249
6597255a4f94 revlogv2: track current index size in the docket
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 47245
diff changeset
75 data = (
6597255a4f94 revlogv2: track current index size in the docket
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 47245
diff changeset
76 self._version_header,
6597255a4f94 revlogv2: track current index size in the docket
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 47245
diff changeset
77 self._index_end,
6597255a4f94 revlogv2: track current index size in the docket
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 47245
diff changeset
78 )
6597255a4f94 revlogv2: track current index size in the docket
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 47245
diff changeset
79 return S_HEADER.pack(*data)
47245
616b8f412676 revlogv2: introduce a very basic docket file
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
diff changeset
80
616b8f412676 revlogv2: introduce a very basic docket file
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
diff changeset
81
616b8f412676 revlogv2: introduce a very basic docket file
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
diff changeset
82 def default_docket(revlog, version_header):
616b8f412676 revlogv2: introduce a very basic docket file
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
diff changeset
83 """given a revlog version a new docket object for the given revlog"""
616b8f412676 revlogv2: introduce a very basic docket file
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
diff changeset
84 if (version_header & 0xFFFF) != constants.REVLOGV2:
616b8f412676 revlogv2: introduce a very basic docket file
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
diff changeset
85 return None
616b8f412676 revlogv2: introduce a very basic docket file
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
diff changeset
86 docket = RevlogDocket(revlog, version_header=version_header)
616b8f412676 revlogv2: introduce a very basic docket file
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
diff changeset
87 docket._dirty = True
616b8f412676 revlogv2: introduce a very basic docket file
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
diff changeset
88 return docket
616b8f412676 revlogv2: introduce a very basic docket file
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
diff changeset
89
616b8f412676 revlogv2: introduce a very basic docket file
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
diff changeset
90
616b8f412676 revlogv2: introduce a very basic docket file
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
diff changeset
91 def parse_docket(revlog, data):
616b8f412676 revlogv2: introduce a very basic docket file
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
diff changeset
92 """given some docket data return a docket object for the given revlog"""
616b8f412676 revlogv2: introduce a very basic docket file
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
diff changeset
93 header = S_HEADER.unpack(data[: S_HEADER.size])
47249
6597255a4f94 revlogv2: track current index size in the docket
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 47245
diff changeset
94 version_header, index_size = header
47245
616b8f412676 revlogv2: introduce a very basic docket file
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
diff changeset
95 docket = RevlogDocket(
616b8f412676 revlogv2: introduce a very basic docket file
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
diff changeset
96 revlog,
616b8f412676 revlogv2: introduce a very basic docket file
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
diff changeset
97 version_header=version_header,
47249
6597255a4f94 revlogv2: track current index size in the docket
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 47245
diff changeset
98 index_end=index_size,
47245
616b8f412676 revlogv2: introduce a very basic docket file
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
diff changeset
99 )
616b8f412676 revlogv2: introduce a very basic docket file
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
diff changeset
100 return docket