Mercurial > public > mercurial-scm > hg

annotate mercurial/pure/parsers.py @ 46857:cc65cea90edb

Find changesets by keywords (author, files, the commit message), revision number or hash, or revset expression.
revlog: move the details of revlog "v1" index inside revlog.utils.constants The revlog module is quite large and this kind of format information would handy for other module. So let us start to gather this information about the format in a more appropriate place. We update various reference to this information to use the new "source of truth" in the process. Differential Revision: https://phab.mercurial-scm.org/D10304
author Pierre-Yves David <pierre-yves.david@octobus.net>
date Mon, 05 Apr 2021 12:21:01 +0200
parents d4ba4d51f85f
children 85e3a630cad9
Ignore whitespace changes - Everywhere: Within whitespace: At end of lines:
rev   line source
7700
f410c552fa15 pure Python implementation of parsers.c
Martin Geisler <mg@daimi.au.dk>
parents:
diff changeset
1 # parsers.py - Python implementation of parsers.c
f410c552fa15 pure Python implementation of parsers.c
Martin Geisler <mg@daimi.au.dk>
parents:
diff changeset
2 #
46819
d4ba4d51f85f contributor: change mentions of mpm to olivia
Rapha?l Gom?s <rgomes@octobus.net>
parents: 46717
diff changeset
3 # Copyright 2009 Olivia Mackall <olivia@selenic.com> and others
7700
f410c552fa15 pure Python implementation of parsers.c
Martin Geisler <mg@daimi.au.dk>
parents:
diff changeset
4 #
8225
46293a0c7e9f updated license to be explicit about GPL version 2
Martin Geisler <mg@lazybytes.net>
parents: 7945
diff changeset
5 # This software may be used and distributed according to the terms of the
10263
25e572394f5c Update license to GPLv2+
Matt Mackall <mpm@selenic.com>
parents: 8225
diff changeset
6 # GNU General Public License version 2 or any later version.
7700
f410c552fa15 pure Python implementation of parsers.c
Martin Geisler <mg@daimi.au.dk>
parents:
diff changeset
7
27339
6ab8c6511a6a parsers: use absolute_import
Gregory Szorc <gregory.szorc@gmail.com>
parents: 24634
diff changeset
8 from __future__ import absolute_import
6ab8c6511a6a parsers: use absolute_import
Gregory Szorc <gregory.szorc@gmail.com>
parents: 24634
diff changeset
9
6ab8c6511a6a parsers: use absolute_import
Gregory Szorc <gregory.szorc@gmail.com>
parents: 24634
diff changeset
10 import struct
6ab8c6511a6a parsers: use absolute_import
Gregory Szorc <gregory.szorc@gmail.com>
parents: 24634
diff changeset
11 import zlib
6ab8c6511a6a parsers: use absolute_import
Gregory Szorc <gregory.szorc@gmail.com>
parents: 24634
diff changeset
12
43525
845e5b313783 revlog: move the nodemap into the index object (for pure)
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 43106
diff changeset
13 from ..node import nullid, nullrev
845e5b313783 revlog: move the nodemap into the index object (for pure)
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 43106
diff changeset
14 from .. import (
845e5b313783 revlog: move the nodemap into the index object (for pure)
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 43106
diff changeset
15 pycompat,
845e5b313783 revlog: move the nodemap into the index object (for pure)
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 43106
diff changeset
16 util,
845e5b313783 revlog: move the nodemap into the index object (for pure)
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 43106
diff changeset
17 )
43076
2372284d9457 formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents: 39217
diff changeset
18
44034
ab595920de0e revlogutils: move the NodeMap class in a dedicated nodemap module
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 43619
diff changeset
19 from ..revlogutils import nodemap as nodemaputil
46857
cc65cea90edb revlog: move the details of revlog "v1" index inside revlog.utils.constants
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 46819
diff changeset
20 from ..revlogutils import constants as revlog_constants
44034
ab595920de0e revlogutils: move the NodeMap class in a dedicated nodemap module
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 43619
diff changeset
21
36958
644a02f6b34f util: prefer "bytesio" to "stringio"
Gregory Szorc <gregory.szorc@gmail.com>
parents: 34331
diff changeset
22 stringio = pycompat.bytesio
7700
f410c552fa15 pure Python implementation of parsers.c
Martin Geisler <mg@daimi.au.dk>
parents:
diff changeset
23
31220
37596c980662 parsers: alias long to int on Python 3
Pulkit Goyal <7895pulkit@gmail.com>
parents: 29133
diff changeset
24
7700
f410c552fa15 pure Python implementation of parsers.c
Martin Geisler <mg@daimi.au.dk>
parents:
diff changeset
25 _pack = struct.pack
f410c552fa15 pure Python implementation of parsers.c
Martin Geisler <mg@daimi.au.dk>
parents:
diff changeset
26 _unpack = struct.unpack
f410c552fa15 pure Python implementation of parsers.c
Martin Geisler <mg@daimi.au.dk>
parents:
diff changeset
27 _compress = zlib.compress
f410c552fa15 pure Python implementation of parsers.c
Martin Geisler <mg@daimi.au.dk>
parents:
diff changeset
28 _decompress = zlib.decompress
f410c552fa15 pure Python implementation of parsers.c
Martin Geisler <mg@daimi.au.dk>
parents:
diff changeset
29
21809
e250b8300e6e parsers: inline fields of dirstate values in C version
Siddharth Agarwal <sid0@fb.com>
parents: 19652
diff changeset
30 # Some code below makes tuples directly because it's more convenient. However,
e250b8300e6e parsers: inline fields of dirstate values in C version
Siddharth Agarwal <sid0@fb.com>
parents: 19652
diff changeset
31 # code outside this module should always use dirstatetuple.
e250b8300e6e parsers: inline fields of dirstate values in C version
Siddharth Agarwal <sid0@fb.com>
parents: 19652
diff changeset
32 def dirstatetuple(*x):
e250b8300e6e parsers: inline fields of dirstate values in C version
Siddharth Agarwal <sid0@fb.com>
parents: 19652
diff changeset
33 # x is a tuple
e250b8300e6e parsers: inline fields of dirstate values in C version
Siddharth Agarwal <sid0@fb.com>
parents: 19652
diff changeset
34 return x
e250b8300e6e parsers: inline fields of dirstate values in C version
Siddharth Agarwal <sid0@fb.com>
parents: 19652
diff changeset
35
43076
2372284d9457 formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents: 39217
diff changeset
36
29133
255274719dc1 pure: write a really lazy version of pure indexObject
Maciej Fijalkowski <fijall@gmail.com>
parents: 28861
diff changeset
37 def gettype(q):
255274719dc1 pure: write a really lazy version of pure indexObject
Maciej Fijalkowski <fijall@gmail.com>
parents: 28861
diff changeset
38 return int(q & 0xFFFF)
7945
94d7e14cfa42 pure/parsers: fix circular imports, import mercurial modules properly
Matt Mackall <mpm@selenic.com>
parents: 7873
diff changeset
39
43076
2372284d9457 formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents: 39217
diff changeset
40
29133
255274719dc1 pure: write a really lazy version of pure indexObject
Maciej Fijalkowski <fijall@gmail.com>
parents: 28861
diff changeset
41 def offset_type(offset, type):
31529
61ff3852f6ed pure: use int instead of long
Martin von Zweigbergk <martinvonz@google.com>
parents: 31220
diff changeset
42 return int(int(offset) << 16 | type)
29133
255274719dc1 pure: write a really lazy version of pure indexObject
Maciej Fijalkowski <fijall@gmail.com>
parents: 28861
diff changeset
43
43076
2372284d9457 formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents: 39217
diff changeset
44
29133
255274719dc1 pure: write a really lazy version of pure indexObject
Maciej Fijalkowski <fijall@gmail.com>
parents: 28861
diff changeset
45 class BaseIndexObject(object):
46635
e83327af26f1 pure-parsers: document index class constants
Rapha?l Gom?s <rgomes@octobus.net>
parents: 46407
diff changeset
46 # Format of an index entry according to Python's `struct` language
46857
cc65cea90edb revlog: move the details of revlog "v1" index inside revlog.utils.constants
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 46819
diff changeset
47 index_format = revlog_constants.INDEX_ENTRY_V1.format
46635
e83327af26f1 pure-parsers: document index class constants
Rapha?l Gom?s <rgomes@octobus.net>
parents: 46407
diff changeset
48 # Size of a C unsigned long long int, platform independent
e83327af26f1 pure-parsers: document index class constants
Rapha?l Gom?s <rgomes@octobus.net>
parents: 46407
diff changeset
49 big_int_size = struct.calcsize(b'>Q')
e83327af26f1 pure-parsers: document index class constants
Rapha?l Gom?s <rgomes@octobus.net>
parents: 46407
diff changeset
50 # Size of a C long int, platform independent
e83327af26f1 pure-parsers: document index class constants
Rapha?l Gom?s <rgomes@octobus.net>
parents: 46407
diff changeset
51 int_size = struct.calcsize(b'>i')
e83327af26f1 pure-parsers: document index class constants
Rapha?l Gom?s <rgomes@octobus.net>
parents: 46407
diff changeset
52 # Size of the entire index format
46857
cc65cea90edb revlog: move the details of revlog "v1" index inside revlog.utils.constants
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 46819
diff changeset
53 index_size = revlog_constants.INDEX_ENTRY_V1.size
46635
e83327af26f1 pure-parsers: document index class constants
Rapha?l Gom?s <rgomes@octobus.net>
parents: 46407
diff changeset
54 # An empty index entry, used as a default value to be overridden, or nullrev
46407
095fa99ae5f5 revlog: prepare pure parser for being overloaded
Rapha?l Gom?s <rgomes@octobus.net>
parents: 45942
diff changeset
55 null_item = (0, 0, 0, -1, -1, -1, -1, nullid)
095fa99ae5f5 revlog: prepare pure parser for being overloaded
Rapha?l Gom?s <rgomes@octobus.net>
parents: 45942
diff changeset
56
43574
02802fa87b74 revlog: deprecate direct `nodemap` access
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 43554
diff changeset
57 @property
02802fa87b74 revlog: deprecate direct `nodemap` access
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 43554
diff changeset
58 def nodemap(self):
43619
c207c46a86b9 py3: pass a bytes value for "msg" to nouideprecwarn()
Denis Laxalde <denis.laxalde@logilab.fr>
parents: 43574
diff changeset
59 msg = b"index.nodemap is deprecated, use index.[has_node|rev|get_rev]"
43574
02802fa87b74 revlog: deprecate direct `nodemap` access
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 43554
diff changeset
60 util.nouideprecwarn(msg, b'5.3', stacklevel=2)
02802fa87b74 revlog: deprecate direct `nodemap` access
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 43554
diff changeset
61 return self._nodemap
02802fa87b74 revlog: deprecate direct `nodemap` access
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 43554
diff changeset
62
43525
845e5b313783 revlog: move the nodemap into the index object (for pure)
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 43106
diff changeset
63 @util.propertycache
43574
02802fa87b74 revlog: deprecate direct `nodemap` access
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 43554
diff changeset
64 def _nodemap(self):
44034
ab595920de0e revlogutils: move the NodeMap class in a dedicated nodemap module
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 43619
diff changeset
65 nodemap = nodemaputil.NodeMap({nullid: nullrev})
43525
845e5b313783 revlog: move the nodemap into the index object (for pure)
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 43106
diff changeset
66 for r in range(0, len(self)):
845e5b313783 revlog: move the nodemap into the index object (for pure)
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 43106
diff changeset
67 n = self[r][7]
845e5b313783 revlog: move the nodemap into the index object (for pure)
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 43106
diff changeset
68 nodemap[n] = r
845e5b313783 revlog: move the nodemap into the index object (for pure)
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 43106
diff changeset
69 return nodemap
845e5b313783 revlog: move the nodemap into the index object (for pure)
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 43106
diff changeset
70
43534
0c659fc20207 index: add a `has_node` method (API)
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 43533
diff changeset
71 def has_node(self, node):
0c659fc20207 index: add a `has_node` method (API)
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 43533
diff changeset
72 """return True if the node exist in the index"""
43574
02802fa87b74 revlog: deprecate direct `nodemap` access
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 43554
diff changeset
73 return node in self._nodemap
43534
0c659fc20207 index: add a `has_node` method (API)
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 43533
diff changeset
74
43552
bd87114ce341 index: add a `rev` method (API)
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 43534
diff changeset
75 def rev(self, node):
bd87114ce341 index: add a `rev` method (API)
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 43534
diff changeset
76 """return a revision for a node
bd87114ce341 index: add a `rev` method (API)
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 43534
diff changeset
77
bd87114ce341 index: add a `rev` method (API)
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 43534
diff changeset
78 If the node is unknown, raise a RevlogError"""
43574
02802fa87b74 revlog: deprecate direct `nodemap` access
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 43554
diff changeset
79 return self._nodemap[node]
43552
bd87114ce341 index: add a `rev` method (API)
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 43534
diff changeset
80
43554
b56de57c45ce index: add a `get_rev` method (API)
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 43552
diff changeset
81 def get_rev(self, node):
b56de57c45ce index: add a `get_rev` method (API)
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 43552
diff changeset
82 """return a revision for a node
b56de57c45ce index: add a `get_rev` method (API)
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 43552
diff changeset
83
b56de57c45ce index: add a `get_rev` method (API)
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 43552
diff changeset
84 If the node is unknown, return None"""
43574
02802fa87b74 revlog: deprecate direct `nodemap` access
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 43554
diff changeset
85 return self._nodemap.get(node)
43554
b56de57c45ce index: add a `get_rev` method (API)
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 43552
diff changeset
86
43533
642433629e20 revlog: deal with nodemap deletion within the index
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 43531
diff changeset
87 def _stripnodes(self, start):
43574
02802fa87b74 revlog: deprecate direct `nodemap` access
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 43554
diff changeset
88 if '_nodemap' in vars(self):
43533
642433629e20 revlog: deal with nodemap deletion within the index
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 43531
diff changeset
89 for r in range(start, len(self)):
642433629e20 revlog: deal with nodemap deletion within the index
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 43531
diff changeset
90 n = self[r][7]
43574
02802fa87b74 revlog: deprecate direct `nodemap` access
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 43554
diff changeset
91 del self._nodemap[n]
43533
642433629e20 revlog: deal with nodemap deletion within the index
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 43531
diff changeset
92
43525
845e5b313783 revlog: move the nodemap into the index object (for pure)
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 43106
diff changeset
93 def clearcaches(self):
43574
02802fa87b74 revlog: deprecate direct `nodemap` access
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 43554
diff changeset
94 self.__dict__.pop('_nodemap', None)
43525
845e5b313783 revlog: move the nodemap into the index object (for pure)
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 43106
diff changeset
95
29133
255274719dc1 pure: write a really lazy version of pure indexObject
Maciej Fijalkowski <fijall@gmail.com>
parents: 28861
diff changeset
96 def __len__(self):
38851
781b2720d2ac index: don't include nullid in len()
Martin von Zweigbergk <martinvonz@google.com>
parents: 38850
diff changeset
97 return self._lgt + len(self._extra)
29133
255274719dc1 pure: write a really lazy version of pure indexObject
Maciej Fijalkowski <fijall@gmail.com>
parents: 28861
diff changeset
98
38850
6104b203bec8 index: replace insert(-1, e) method by append(e) method
Martin von Zweigbergk <martinvonz@google.com>
parents: 38848
diff changeset
99 def append(self, tup):
43574
02802fa87b74 revlog: deprecate direct `nodemap` access
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 43554
diff changeset
100 if '_nodemap' in vars(self):
02802fa87b74 revlog: deprecate direct `nodemap` access
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 43554
diff changeset
101 self._nodemap[tup[7]] = len(self)
46407
095fa99ae5f5 revlog: prepare pure parser for being overloaded
Rapha?l Gom?s <rgomes@octobus.net>
parents: 45942
diff changeset
102 data = _pack(self.index_format, *tup)
45936
0ce15a8c7b8b revlog: store new index entries as binary
Joerg Sonnenberger <joerg@bec.de>
parents: 45645
diff changeset
103 self._extra.append(data)
7945
94d7e14cfa42 pure/parsers: fix circular imports, import mercurial modules properly
Matt Mackall <mpm@selenic.com>
parents: 7873
diff changeset
104
39217
5961517fd2a8 index: rename _fix_index() since it no longer fixes the index
Martin von Zweigbergk <martinvonz@google.com>
parents: 39216
diff changeset
105 def _check_index(self, i):
29133
255274719dc1 pure: write a really lazy version of pure indexObject
Maciej Fijalkowski <fijall@gmail.com>
parents: 28861
diff changeset
106 if not isinstance(i, int):
43077
687b865b95ad formatting: byteify all mercurial/ and hgext/ string literals
Augie Fackler <augie@google.com>
parents: 43076
diff changeset
107 raise TypeError(b"expecting int indexes")
39216
ec6d5a9d1631 index: don't include nullid in boundary check in pure code
Martin von Zweigbergk <martinvonz@google.com>
parents: 39047
diff changeset
108 if i < 0 or i >= len(self):
29133
255274719dc1 pure: write a really lazy version of pure indexObject
Maciej Fijalkowski <fijall@gmail.com>
parents: 28861
diff changeset
109 raise IndexError
7945
94d7e14cfa42 pure/parsers: fix circular imports, import mercurial modules properly
Matt Mackall <mpm@selenic.com>
parents: 7873
diff changeset
110
29133
255274719dc1 pure: write a really lazy version of pure indexObject
Maciej Fijalkowski <fijall@gmail.com>
parents: 28861
diff changeset
111 def __getitem__(self, i):
39047
a1f934573c0b parsers: adjust pure-python version to mimic a3dacabd476b
Augie Fackler <augie@google.com>
parents: 38851
diff changeset
112 if i == -1:
46407
095fa99ae5f5 revlog: prepare pure parser for being overloaded
Rapha?l Gom?s <rgomes@octobus.net>
parents: 45942
diff changeset
113 return self.null_item
39217
5961517fd2a8 index: rename _fix_index() since it no longer fixes the index
Martin von Zweigbergk <martinvonz@google.com>
parents: 39216
diff changeset
114 self._check_index(i)
29133
255274719dc1 pure: write a really lazy version of pure indexObject
Maciej Fijalkowski <fijall@gmail.com>
parents: 28861
diff changeset
115 if i >= self._lgt:
45936
0ce15a8c7b8b revlog: store new index entries as binary
Joerg Sonnenberger <joerg@bec.de>
parents: 45645
diff changeset
116 data = self._extra[i - self._lgt]
0ce15a8c7b8b revlog: store new index entries as binary
Joerg Sonnenberger <joerg@bec.de>
parents: 45645
diff changeset
117 else:
0ce15a8c7b8b revlog: store new index entries as binary
Joerg Sonnenberger <joerg@bec.de>
parents: 45645
diff changeset
118 index = self._calculate_index(i)
46407
095fa99ae5f5 revlog: prepare pure parser for being overloaded
Rapha?l Gom?s <rgomes@octobus.net>
parents: 45942
diff changeset
119 data = self._data[index : index + self.index_size]
095fa99ae5f5 revlog: prepare pure parser for being overloaded
Rapha?l Gom?s <rgomes@octobus.net>
parents: 45942
diff changeset
120 r = _unpack(self.index_format, data)
45936
0ce15a8c7b8b revlog: store new index entries as binary
Joerg Sonnenberger <joerg@bec.de>
parents: 45645
diff changeset
121 if self._lgt and i == 0:
0ce15a8c7b8b revlog: store new index entries as binary
Joerg Sonnenberger <joerg@bec.de>
parents: 45645
diff changeset
122 r = (offset_type(0, gettype(r[0])),) + r[1:]
29133
255274719dc1 pure: write a really lazy version of pure indexObject
Maciej Fijalkowski <fijall@gmail.com>
parents: 28861
diff changeset
123 return r
255274719dc1 pure: write a really lazy version of pure indexObject
Maciej Fijalkowski <fijall@gmail.com>
parents: 28861
diff changeset
124
43076
2372284d9457 formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents: 39217
diff changeset
125
29133
255274719dc1 pure: write a really lazy version of pure indexObject
Maciej Fijalkowski <fijall@gmail.com>
parents: 28861
diff changeset
126 class IndexObject(BaseIndexObject):
255274719dc1 pure: write a really lazy version of pure indexObject
Maciej Fijalkowski <fijall@gmail.com>
parents: 28861
diff changeset
127 def __init__(self, data):
46407
095fa99ae5f5 revlog: prepare pure parser for being overloaded
Rapha?l Gom?s <rgomes@octobus.net>
parents: 45942
diff changeset
128 assert len(data) % self.index_size == 0
29133
255274719dc1 pure: write a really lazy version of pure indexObject
Maciej Fijalkowski <fijall@gmail.com>
parents: 28861
diff changeset
129 self._data = data
46407
095fa99ae5f5 revlog: prepare pure parser for being overloaded
Rapha?l Gom?s <rgomes@octobus.net>
parents: 45942
diff changeset
130 self._lgt = len(data) // self.index_size
29133
255274719dc1 pure: write a really lazy version of pure indexObject
Maciej Fijalkowski <fijall@gmail.com>
parents: 28861
diff changeset
131 self._extra = []
255274719dc1 pure: write a really lazy version of pure indexObject
Maciej Fijalkowski <fijall@gmail.com>
parents: 28861
diff changeset
132
255274719dc1 pure: write a really lazy version of pure indexObject
Maciej Fijalkowski <fijall@gmail.com>
parents: 28861
diff changeset
133 def _calculate_index(self, i):
46407
095fa99ae5f5 revlog: prepare pure parser for being overloaded
Rapha?l Gom?s <rgomes@octobus.net>
parents: 45942
diff changeset
134 return i * self.index_size
13253
61c9bc3da402 revlog: remove lazy index
Matt Mackall <mpm@selenic.com>
parents: 10263
diff changeset
135
29133
255274719dc1 pure: write a really lazy version of pure indexObject
Maciej Fijalkowski <fijall@gmail.com>
parents: 28861
diff changeset
136 def __delitem__(self, i):
34331
531332502568 style: always use `x is not None` instead of `not x is None`
Alex Gaynor <agaynor@mozilla.com>
parents: 32372
diff changeset
137 if not isinstance(i, slice) or not i.stop == -1 or i.step is not None:
43077
687b865b95ad formatting: byteify all mercurial/ and hgext/ string literals
Augie Fackler <augie@google.com>
parents: 43076
diff changeset
138 raise ValueError(b"deleting slices only supports a:-1 with step 1")
39217
5961517fd2a8 index: rename _fix_index() since it no longer fixes the index
Martin von Zweigbergk <martinvonz@google.com>
parents: 39216
diff changeset
139 i = i.start
5961517fd2a8 index: rename _fix_index() since it no longer fixes the index
Martin von Zweigbergk <martinvonz@google.com>
parents: 39216
diff changeset
140 self._check_index(i)
43533
642433629e20 revlog: deal with nodemap deletion within the index
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 43531
diff changeset
141 self._stripnodes(i)
29133
255274719dc1 pure: write a really lazy version of pure indexObject
Maciej Fijalkowski <fijall@gmail.com>
parents: 28861
diff changeset
142 if i < self._lgt:
46407
095fa99ae5f5 revlog: prepare pure parser for being overloaded
Rapha?l Gom?s <rgomes@octobus.net>
parents: 45942
diff changeset
143 self._data = self._data[: i * self.index_size]
29133
255274719dc1 pure: write a really lazy version of pure indexObject
Maciej Fijalkowski <fijall@gmail.com>
parents: 28861
diff changeset
144 self._lgt = i
255274719dc1 pure: write a really lazy version of pure indexObject
Maciej Fijalkowski <fijall@gmail.com>
parents: 28861
diff changeset
145 self._extra = []
255274719dc1 pure: write a really lazy version of pure indexObject
Maciej Fijalkowski <fijall@gmail.com>
parents: 28861
diff changeset
146 else:
43076
2372284d9457 formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents: 39217
diff changeset
147 self._extra = self._extra[: i - self._lgt]
2372284d9457 formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents: 39217
diff changeset
148
29133
255274719dc1 pure: write a really lazy version of pure indexObject
Maciej Fijalkowski <fijall@gmail.com>
parents: 28861
diff changeset
149
44313
6f9e8e142cea nodemap: add a (python) index class for persistent nodemap testing
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 44034
diff changeset
150 class PersistentNodeMapIndexObject(IndexObject):
6f9e8e142cea nodemap: add a (python) index class for persistent nodemap testing
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 44034
diff changeset
151 """a Debug oriented class to test persistent nodemap
6f9e8e142cea nodemap: add a (python) index class for persistent nodemap testing
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 44034
diff changeset
152
6f9e8e142cea nodemap: add a (python) index class for persistent nodemap testing
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 44034
diff changeset
153 We need a simple python object to test API and higher level behavior. See
6f9e8e142cea nodemap: add a (python) index class for persistent nodemap testing
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 44034
diff changeset
154 the Rust implementation for more serious usage. This should be used only
6f9e8e142cea nodemap: add a (python) index class for persistent nodemap testing
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 44034
diff changeset
155 through the dedicated `devel.persistent-nodemap` config.
6f9e8e142cea nodemap: add a (python) index class for persistent nodemap testing
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 44034
diff changeset
156 """
6f9e8e142cea nodemap: add a (python) index class for persistent nodemap testing
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 44034
diff changeset
157
44314
7f4f7ef3133e nodemap: add a optional `nodemap_add_full` method on indexes
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 44313
diff changeset
158 def nodemap_data_all(self):
7f4f7ef3133e nodemap: add a optional `nodemap_add_full` method on indexes
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 44313
diff changeset
159 """Return bytes containing a full serialization of a nodemap
7f4f7ef3133e nodemap: add a optional `nodemap_add_full` method on indexes
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 44313
diff changeset
160
7f4f7ef3133e nodemap: add a optional `nodemap_add_full` method on indexes
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 44313
diff changeset
161 The nodemap should be valid for the full set of revisions in the
7f4f7ef3133e nodemap: add a optional `nodemap_add_full` method on indexes
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 44313
diff changeset
162 index."""
7f4f7ef3133e nodemap: add a optional `nodemap_add_full` method on indexes
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 44313
diff changeset
163 return nodemaputil.persistent_data(self)
7f4f7ef3133e nodemap: add a optional `nodemap_add_full` method on indexes
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 44313
diff changeset
164
44333
50ad851efd9b nodemap: introduce append-only incremental update of the persistent data
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 44320
diff changeset
165 def nodemap_data_incremental(self):
50ad851efd9b nodemap: introduce append-only incremental update of the persistent data
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 44320
diff changeset
166 """Return bytes containing a incremental update to persistent nodemap
50ad851efd9b nodemap: introduce append-only incremental update of the persistent data
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 44320
diff changeset
167
50ad851efd9b nodemap: introduce append-only incremental update of the persistent data
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 44320
diff changeset
168 This containst the data for an append-only update of the data provided
50ad851efd9b nodemap: introduce append-only incremental update of the persistent data
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 44320
diff changeset
169 in the last call to `update_nodemap_data`.
50ad851efd9b nodemap: introduce append-only incremental update of the persistent data
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 44320
diff changeset
170 """
50ad851efd9b nodemap: introduce append-only incremental update of the persistent data
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 44320
diff changeset
171 if self._nm_root is None:
50ad851efd9b nodemap: introduce append-only incremental update of the persistent data
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 44320
diff changeset
172 return None
44337
1d2b37def017 nodemap: double check the source docket when doing incremental update
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 44336
diff changeset
173 docket = self._nm_docket
44336
8374b69aef75 nodemap: track the total and unused amount of data in the rawdata file
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 44335
diff changeset
174 changed, data = nodemaputil.update_persistent_data(
44337
1d2b37def017 nodemap: double check the source docket when doing incremental update
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 44336
diff changeset
175 self, self._nm_root, self._nm_max_idx, self._nm_docket.tip_rev
44333
50ad851efd9b nodemap: introduce append-only incremental update of the persistent data
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 44320
diff changeset
176 )
44337
1d2b37def017 nodemap: double check the source docket when doing incremental update
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 44336
diff changeset
177
1d2b37def017 nodemap: double check the source docket when doing incremental update
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 44336
diff changeset
178 self._nm_root = self._nm_max_idx = self._nm_docket = None
1d2b37def017 nodemap: double check the source docket when doing incremental update
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 44336
diff changeset
179 return docket, changed, data
44333
50ad851efd9b nodemap: introduce append-only incremental update of the persistent data
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 44320
diff changeset
180
44335
e41a164db7a9 nodemap: track the maximum revision tracked in the nodemap
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 44333
diff changeset
181 def update_nodemap_data(self, docket, nm_data):
e41a164db7a9 nodemap: track the maximum revision tracked in the nodemap
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 44333
diff changeset
182 """provide full block of persisted binary data for a nodemap
44320
671f9479af0e nodemap: provide the on disk data to indexes who support it
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 44314
diff changeset
183
671f9479af0e nodemap: provide the on disk data to indexes who support it
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 44314
diff changeset
184 The data are expected to come from disk. See `nodemap_data_all` for a
671f9479af0e nodemap: provide the on disk data to indexes who support it
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 44314
diff changeset
185 produceur of such data."""
671f9479af0e nodemap: provide the on disk data to indexes who support it
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 44314
diff changeset
186 if nm_data is not None:
44333
50ad851efd9b nodemap: introduce append-only incremental update of the persistent data
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 44320
diff changeset
187 self._nm_root, self._nm_max_idx = nodemaputil.parse_data(nm_data)
50ad851efd9b nodemap: introduce append-only incremental update of the persistent data
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 44320
diff changeset
188 if self._nm_root:
44337
1d2b37def017 nodemap: double check the source docket when doing incremental update
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 44336
diff changeset
189 self._nm_docket = docket
44333
50ad851efd9b nodemap: introduce append-only incremental update of the persistent data
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 44320
diff changeset
190 else:
44337
1d2b37def017 nodemap: double check the source docket when doing incremental update
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 44336
diff changeset
191 self._nm_root = self._nm_max_idx = self._nm_docket = None
44320
671f9479af0e nodemap: provide the on disk data to indexes who support it
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 44314
diff changeset
192
44313
6f9e8e142cea nodemap: add a (python) index class for persistent nodemap testing
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 44034
diff changeset
193
29133
255274719dc1 pure: write a really lazy version of pure indexObject
Maciej Fijalkowski <fijall@gmail.com>
parents: 28861
diff changeset
194 class InlinedIndexObject(BaseIndexObject):
255274719dc1 pure: write a really lazy version of pure indexObject
Maciej Fijalkowski <fijall@gmail.com>
parents: 28861
diff changeset
195 def __init__(self, data, inline=0):
255274719dc1 pure: write a really lazy version of pure indexObject
Maciej Fijalkowski <fijall@gmail.com>
parents: 28861
diff changeset
196 self._data = data
255274719dc1 pure: write a really lazy version of pure indexObject
Maciej Fijalkowski <fijall@gmail.com>
parents: 28861
diff changeset
197 self._lgt = self._inline_scan(None)
255274719dc1 pure: write a really lazy version of pure indexObject
Maciej Fijalkowski <fijall@gmail.com>
parents: 28861
diff changeset
198 self._inline_scan(self._lgt)
255274719dc1 pure: write a really lazy version of pure indexObject
Maciej Fijalkowski <fijall@gmail.com>
parents: 28861
diff changeset
199 self._extra = []
7700
f410c552fa15 pure Python implementation of parsers.c
Martin Geisler <mg@daimi.au.dk>
parents:
diff changeset
200
29133
255274719dc1 pure: write a really lazy version of pure indexObject
Maciej Fijalkowski <fijall@gmail.com>
parents: 28861
diff changeset
201 def _inline_scan(self, lgt):
255274719dc1 pure: write a really lazy version of pure indexObject
Maciej Fijalkowski <fijall@gmail.com>
parents: 28861
diff changeset
202 off = 0
255274719dc1 pure: write a really lazy version of pure indexObject
Maciej Fijalkowski <fijall@gmail.com>
parents: 28861
diff changeset
203 if lgt is not None:
255274719dc1 pure: write a really lazy version of pure indexObject
Maciej Fijalkowski <fijall@gmail.com>
parents: 28861
diff changeset
204 self._offsets = [0] * lgt
255274719dc1 pure: write a really lazy version of pure indexObject
Maciej Fijalkowski <fijall@gmail.com>
parents: 28861
diff changeset
205 count = 0
46407
095fa99ae5f5 revlog: prepare pure parser for being overloaded
Rapha?l Gom?s <rgomes@octobus.net>
parents: 45942
diff changeset
206 while off <= len(self._data) - self.index_size:
095fa99ae5f5 revlog: prepare pure parser for being overloaded
Rapha?l Gom?s <rgomes@octobus.net>
parents: 45942
diff changeset
207 start = off + self.big_int_size
43076
2372284d9457 formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents: 39217
diff changeset
208 (s,) = struct.unpack(
46407
095fa99ae5f5 revlog: prepare pure parser for being overloaded
Rapha?l Gom?s <rgomes@octobus.net>
parents: 45942
diff changeset
209 b'>i',
095fa99ae5f5 revlog: prepare pure parser for being overloaded
Rapha?l Gom?s <rgomes@octobus.net>
parents: 45942
diff changeset
210 self._data[start : start + self.int_size],
43076
2372284d9457 formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents: 39217
diff changeset
211 )
29133
255274719dc1 pure: write a really lazy version of pure indexObject
Maciej Fijalkowski <fijall@gmail.com>
parents: 28861
diff changeset
212 if lgt is not None:
255274719dc1 pure: write a really lazy version of pure indexObject
Maciej Fijalkowski <fijall@gmail.com>
parents: 28861
diff changeset
213 self._offsets[count] = off
255274719dc1 pure: write a really lazy version of pure indexObject
Maciej Fijalkowski <fijall@gmail.com>
parents: 28861
diff changeset
214 count += 1
46407
095fa99ae5f5 revlog: prepare pure parser for being overloaded
Rapha?l Gom?s <rgomes@octobus.net>
parents: 45942
diff changeset
215 off += self.index_size + s
29133
255274719dc1 pure: write a really lazy version of pure indexObject
Maciej Fijalkowski <fijall@gmail.com>
parents: 28861
diff changeset
216 if off != len(self._data):
43077
687b865b95ad formatting: byteify all mercurial/ and hgext/ string literals
Augie Fackler <augie@google.com>
parents: 43076
diff changeset
217 raise ValueError(b"corrupted data")
29133
255274719dc1 pure: write a really lazy version of pure indexObject
Maciej Fijalkowski <fijall@gmail.com>
parents: 28861
diff changeset
218 return count
14421
639f26cab2f5 pure parsers: properly detect corrupt index files
Augie Fackler <durin42@gmail.com>
parents: 14064
diff changeset
219
29133
255274719dc1 pure: write a really lazy version of pure indexObject
Maciej Fijalkowski <fijall@gmail.com>
parents: 28861
diff changeset
220 def __delitem__(self, i):
34331
531332502568 style: always use `x is not None` instead of `not x is None`
Alex Gaynor <agaynor@mozilla.com>
parents: 32372
diff changeset
221 if not isinstance(i, slice) or not i.stop == -1 or i.step is not None:
43077
687b865b95ad formatting: byteify all mercurial/ and hgext/ string literals
Augie Fackler <augie@google.com>
parents: 43076
diff changeset
222 raise ValueError(b"deleting slices only supports a:-1 with step 1")
39217
5961517fd2a8 index: rename _fix_index() since it no longer fixes the index
Martin von Zweigbergk <martinvonz@google.com>
parents: 39216
diff changeset
223 i = i.start
5961517fd2a8 index: rename _fix_index() since it no longer fixes the index
Martin von Zweigbergk <martinvonz@google.com>
parents: 39216
diff changeset
224 self._check_index(i)
43533
642433629e20 revlog: deal with nodemap deletion within the index
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 43531
diff changeset
225 self._stripnodes(i)
29133
255274719dc1 pure: write a really lazy version of pure indexObject
Maciej Fijalkowski <fijall@gmail.com>
parents: 28861
diff changeset
226 if i < self._lgt:
255274719dc1 pure: write a really lazy version of pure indexObject
Maciej Fijalkowski <fijall@gmail.com>
parents: 28861
diff changeset
227 self._offsets = self._offsets[:i]
255274719dc1 pure: write a really lazy version of pure indexObject
Maciej Fijalkowski <fijall@gmail.com>
parents: 28861
diff changeset
228 self._lgt = i
255274719dc1 pure: write a really lazy version of pure indexObject
Maciej Fijalkowski <fijall@gmail.com>
parents: 28861
diff changeset
229 self._extra = []
255274719dc1 pure: write a really lazy version of pure indexObject
Maciej Fijalkowski <fijall@gmail.com>
parents: 28861
diff changeset
230 else:
43076
2372284d9457 formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents: 39217
diff changeset
231 self._extra = self._extra[: i - self._lgt]
7700
f410c552fa15 pure Python implementation of parsers.c
Martin Geisler <mg@daimi.au.dk>
parents:
diff changeset
232
29133
255274719dc1 pure: write a really lazy version of pure indexObject
Maciej Fijalkowski <fijall@gmail.com>
parents: 28861
diff changeset
233 def _calculate_index(self, i):
255274719dc1 pure: write a really lazy version of pure indexObject
Maciej Fijalkowski <fijall@gmail.com>
parents: 28861
diff changeset
234 return self._offsets[i]
7700
f410c552fa15 pure Python implementation of parsers.c
Martin Geisler <mg@daimi.au.dk>
parents:
diff changeset
235
43076
2372284d9457 formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents: 39217
diff changeset
236
46704
913485776542 revlog: introduce v2 format
Rapha?l Gom?s <rgomes@octobus.net>
parents: 46635
diff changeset
237 def parse_index2(data, inline, revlogv2=False):
29133
255274719dc1 pure: write a really lazy version of pure indexObject
Maciej Fijalkowski <fijall@gmail.com>
parents: 28861
diff changeset
238 if not inline:
46704
913485776542 revlog: introduce v2 format
Rapha?l Gom?s <rgomes@octobus.net>
parents: 46635
diff changeset
239 cls = IndexObject2 if revlogv2 else IndexObject
913485776542 revlog: introduce v2 format
Rapha?l Gom?s <rgomes@octobus.net>
parents: 46635
diff changeset
240 return cls(data), None
913485776542 revlog: introduce v2 format
Rapha?l Gom?s <rgomes@octobus.net>
parents: 46635
diff changeset
241 cls = InlinedIndexObject2 if revlogv2 else InlinedIndexObject
913485776542 revlog: introduce v2 format
Rapha?l Gom?s <rgomes@octobus.net>
parents: 46635
diff changeset
242 return cls(data, inline), (0, data)
913485776542 revlog: introduce v2 format
Rapha?l Gom?s <rgomes@octobus.net>
parents: 46635
diff changeset
243
913485776542 revlog: introduce v2 format
Rapha?l Gom?s <rgomes@octobus.net>
parents: 46635
diff changeset
244
913485776542 revlog: introduce v2 format
Rapha?l Gom?s <rgomes@octobus.net>
parents: 46635
diff changeset
245 class Index2Mixin(object):
913485776542 revlog: introduce v2 format
Rapha?l Gom?s <rgomes@octobus.net>
parents: 46635
diff changeset
246 # 6 bytes: offset
913485776542 revlog: introduce v2 format
Rapha?l Gom?s <rgomes@octobus.net>
parents: 46635
diff changeset
247 # 2 bytes: flags
913485776542 revlog: introduce v2 format
Rapha?l Gom?s <rgomes@octobus.net>
parents: 46635
diff changeset
248 # 4 bytes: compressed length
913485776542 revlog: introduce v2 format
Rapha?l Gom?s <rgomes@octobus.net>
parents: 46635
diff changeset
249 # 4 bytes: uncompressed length
913485776542 revlog: introduce v2 format
Rapha?l Gom?s <rgomes@octobus.net>
parents: 46635
diff changeset
250 # 4 bytes: base rev
913485776542 revlog: introduce v2 format
Rapha?l Gom?s <rgomes@octobus.net>
parents: 46635
diff changeset
251 # 4 bytes: link rev
913485776542 revlog: introduce v2 format
Rapha?l Gom?s <rgomes@octobus.net>
parents: 46635
diff changeset
252 # 4 bytes: parent 1 rev
913485776542 revlog: introduce v2 format
Rapha?l Gom?s <rgomes@octobus.net>
parents: 46635
diff changeset
253 # 4 bytes: parent 2 rev
913485776542 revlog: introduce v2 format
Rapha?l Gom?s <rgomes@octobus.net>
parents: 46635
diff changeset
254 # 32 bytes: nodeid
913485776542 revlog: introduce v2 format
Rapha?l Gom?s <rgomes@octobus.net>
parents: 46635
diff changeset
255 # 8 bytes: sidedata offset
913485776542 revlog: introduce v2 format
Rapha?l Gom?s <rgomes@octobus.net>
parents: 46635
diff changeset
256 # 4 bytes: sidedata compressed length
913485776542 revlog: introduce v2 format
Rapha?l Gom?s <rgomes@octobus.net>
parents: 46635
diff changeset
257 # 20 bytes: Padding to align to 96 bytes (see RevlogV2Plan wiki page)
913485776542 revlog: introduce v2 format
Rapha?l Gom?s <rgomes@octobus.net>
parents: 46635
diff changeset
258 index_format = b">Qiiiiii20s12xQi20x"
913485776542 revlog: introduce v2 format
Rapha?l Gom?s <rgomes@octobus.net>
parents: 46635
diff changeset
259 index_size = struct.calcsize(index_format)
913485776542 revlog: introduce v2 format
Rapha?l Gom?s <rgomes@octobus.net>
parents: 46635
diff changeset
260 assert index_size == 96, index_size
913485776542 revlog: introduce v2 format
Rapha?l Gom?s <rgomes@octobus.net>
parents: 46635
diff changeset
261 null_item = (0, 0, 0, -1, -1, -1, -1, nullid, 0, 0)
913485776542 revlog: introduce v2 format
Rapha?l Gom?s <rgomes@octobus.net>
parents: 46635
diff changeset
262
46717
502e795b55ac revlog-index: add `replace_sidedata_info` method
Rapha?l Gom?s <rgomes@octobus.net>
parents: 46704
diff changeset
263 def replace_sidedata_info(self, i, sidedata_offset, sidedata_length):
502e795b55ac revlog-index: add `replace_sidedata_info` method
Rapha?l Gom?s <rgomes@octobus.net>
parents: 46704
diff changeset
264 """
502e795b55ac revlog-index: add `replace_sidedata_info` method
Rapha?l Gom?s <rgomes@octobus.net>
parents: 46704
diff changeset
265 Replace an existing index entry's sidedata offset and length with new
502e795b55ac revlog-index: add `replace_sidedata_info` method
Rapha?l Gom?s <rgomes@octobus.net>
parents: 46704
diff changeset
266 ones.
502e795b55ac revlog-index: add `replace_sidedata_info` method
Rapha?l Gom?s <rgomes@octobus.net>
parents: 46704
diff changeset
267 This cannot be used outside of the context of sidedata rewriting,
502e795b55ac revlog-index: add `replace_sidedata_info` method
Rapha?l Gom?s <rgomes@octobus.net>
parents: 46704
diff changeset
268 inside the transaction that creates the revision `i`.
502e795b55ac revlog-index: add `replace_sidedata_info` method
Rapha?l Gom?s <rgomes@octobus.net>
parents: 46704
diff changeset
269 """
502e795b55ac revlog-index: add `replace_sidedata_info` method
Rapha?l Gom?s <rgomes@octobus.net>
parents: 46704
diff changeset
270 if i < 0:
502e795b55ac revlog-index: add `replace_sidedata_info` method
Rapha?l Gom?s <rgomes@octobus.net>
parents: 46704
diff changeset
271 raise KeyError
502e795b55ac revlog-index: add `replace_sidedata_info` method
Rapha?l Gom?s <rgomes@octobus.net>
parents: 46704
diff changeset
272 self._check_index(i)
502e795b55ac revlog-index: add `replace_sidedata_info` method
Rapha?l Gom?s <rgomes@octobus.net>
parents: 46704
diff changeset
273 sidedata_format = b">Qi"
502e795b55ac revlog-index: add `replace_sidedata_info` method
Rapha?l Gom?s <rgomes@octobus.net>
parents: 46704
diff changeset
274 packed_size = struct.calcsize(sidedata_format)
502e795b55ac revlog-index: add `replace_sidedata_info` method
Rapha?l Gom?s <rgomes@octobus.net>
parents: 46704
diff changeset
275 if i >= self._lgt:
502e795b55ac revlog-index: add `replace_sidedata_info` method
Rapha?l Gom?s <rgomes@octobus.net>
parents: 46704
diff changeset
276 packed = _pack(sidedata_format, sidedata_offset, sidedata_length)
502e795b55ac revlog-index: add `replace_sidedata_info` method
Rapha?l Gom?s <rgomes@octobus.net>
parents: 46704
diff changeset
277 old = self._extra[i - self._lgt]
502e795b55ac revlog-index: add `replace_sidedata_info` method
Rapha?l Gom?s <rgomes@octobus.net>
parents: 46704
diff changeset
278 new = old[:64] + packed + old[64 + packed_size :]
502e795b55ac revlog-index: add `replace_sidedata_info` method
Rapha?l Gom?s <rgomes@octobus.net>
parents: 46704
diff changeset
279 self._extra[i - self._lgt] = new
502e795b55ac revlog-index: add `replace_sidedata_info` method
Rapha?l Gom?s <rgomes@octobus.net>
parents: 46704
diff changeset
280 else:
502e795b55ac revlog-index: add `replace_sidedata_info` method
Rapha?l Gom?s <rgomes@octobus.net>
parents: 46704
diff changeset
281 msg = b"cannot rewrite entries outside of this transaction"
502e795b55ac revlog-index: add `replace_sidedata_info` method
Rapha?l Gom?s <rgomes@octobus.net>
parents: 46704
diff changeset
282 raise KeyError(msg)
502e795b55ac revlog-index: add `replace_sidedata_info` method
Rapha?l Gom?s <rgomes@octobus.net>
parents: 46704
diff changeset
283
46704
913485776542 revlog: introduce v2 format
Rapha?l Gom?s <rgomes@octobus.net>
parents: 46635
diff changeset
284
913485776542 revlog: introduce v2 format
Rapha?l Gom?s <rgomes@octobus.net>
parents: 46635
diff changeset
285 class IndexObject2(Index2Mixin, IndexObject):
913485776542 revlog: introduce v2 format
Rapha?l Gom?s <rgomes@octobus.net>
parents: 46635
diff changeset
286 pass
913485776542 revlog: introduce v2 format
Rapha?l Gom?s <rgomes@octobus.net>
parents: 46635
diff changeset
287
913485776542 revlog: introduce v2 format
Rapha?l Gom?s <rgomes@octobus.net>
parents: 46635
diff changeset
288
913485776542 revlog: introduce v2 format
Rapha?l Gom?s <rgomes@octobus.net>
parents: 46635
diff changeset
289 class InlinedIndexObject2(Index2Mixin, InlinedIndexObject):
913485776542 revlog: introduce v2 format
Rapha?l Gom?s <rgomes@octobus.net>
parents: 46635
diff changeset
290 def _inline_scan(self, lgt):
913485776542 revlog: introduce v2 format
Rapha?l Gom?s <rgomes@octobus.net>
parents: 46635
diff changeset
291 sidedata_length_pos = 72
913485776542 revlog: introduce v2 format
Rapha?l Gom?s <rgomes@octobus.net>
parents: 46635
diff changeset
292 off = 0
913485776542 revlog: introduce v2 format
Rapha?l Gom?s <rgomes@octobus.net>
parents: 46635
diff changeset
293 if lgt is not None:
913485776542 revlog: introduce v2 format
Rapha?l Gom?s <rgomes@octobus.net>
parents: 46635
diff changeset
294 self._offsets = [0] * lgt
913485776542 revlog: introduce v2 format
Rapha?l Gom?s <rgomes@octobus.net>
parents: 46635
diff changeset
295 count = 0
913485776542 revlog: introduce v2 format
Rapha?l Gom?s <rgomes@octobus.net>
parents: 46635
diff changeset
296 while off <= len(self._data) - self.index_size:
913485776542 revlog: introduce v2 format
Rapha?l Gom?s <rgomes@octobus.net>
parents: 46635
diff changeset
297 start = off + self.big_int_size
913485776542 revlog: introduce v2 format
Rapha?l Gom?s <rgomes@octobus.net>
parents: 46635
diff changeset
298 (data_size,) = struct.unpack(
913485776542 revlog: introduce v2 format
Rapha?l Gom?s <rgomes@octobus.net>
parents: 46635
diff changeset
299 b'>i',
913485776542 revlog: introduce v2 format
Rapha?l Gom?s <rgomes@octobus.net>
parents: 46635
diff changeset
300 self._data[start : start + self.int_size],
913485776542 revlog: introduce v2 format
Rapha?l Gom?s <rgomes@octobus.net>
parents: 46635
diff changeset
301 )
913485776542 revlog: introduce v2 format
Rapha?l Gom?s <rgomes@octobus.net>
parents: 46635
diff changeset
302 start = off + sidedata_length_pos
913485776542 revlog: introduce v2 format
Rapha?l Gom?s <rgomes@octobus.net>
parents: 46635
diff changeset
303 (side_data_size,) = struct.unpack(
913485776542 revlog: introduce v2 format
Rapha?l Gom?s <rgomes@octobus.net>
parents: 46635
diff changeset
304 b'>i', self._data[start : start + self.int_size]
913485776542 revlog: introduce v2 format
Rapha?l Gom?s <rgomes@octobus.net>
parents: 46635
diff changeset
305 )
913485776542 revlog: introduce v2 format
Rapha?l Gom?s <rgomes@octobus.net>
parents: 46635
diff changeset
306 if lgt is not None:
913485776542 revlog: introduce v2 format
Rapha?l Gom?s <rgomes@octobus.net>
parents: 46635
diff changeset
307 self._offsets[count] = off
913485776542 revlog: introduce v2 format
Rapha?l Gom?s <rgomes@octobus.net>
parents: 46635
diff changeset
308 count += 1
913485776542 revlog: introduce v2 format
Rapha?l Gom?s <rgomes@octobus.net>
parents: 46635
diff changeset
309 off += self.index_size + data_size + side_data_size
913485776542 revlog: introduce v2 format
Rapha?l Gom?s <rgomes@octobus.net>
parents: 46635
diff changeset
310 if off != len(self._data):
913485776542 revlog: introduce v2 format
Rapha?l Gom?s <rgomes@octobus.net>
parents: 46635
diff changeset
311 raise ValueError(b"corrupted data")
913485776542 revlog: introduce v2 format
Rapha?l Gom?s <rgomes@octobus.net>
parents: 46635
diff changeset
312 return count
7700
f410c552fa15 pure Python implementation of parsers.c
Martin Geisler <mg@daimi.au.dk>
parents:
diff changeset
313
43076
2372284d9457 formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents: 39217
diff changeset
314
44313
6f9e8e142cea nodemap: add a (python) index class for persistent nodemap testing
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 44034
diff changeset
315 def parse_index_devel_nodemap(data, inline):
45942
89a2afe31e82 formating: upgrade to black 20.8b1
Augie Fackler <raf@durin42.com>
parents: 45936
diff changeset
316 """like parse_index2, but alway return a PersistentNodeMapIndexObject"""
44313
6f9e8e142cea nodemap: add a (python) index class for persistent nodemap testing
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 44034
diff changeset
317 return PersistentNodeMapIndexObject(data), None
6f9e8e142cea nodemap: add a (python) index class for persistent nodemap testing
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 44034
diff changeset
318
6f9e8e142cea nodemap: add a (python) index class for persistent nodemap testing
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 44034
diff changeset
319
7700
f410c552fa15 pure Python implementation of parsers.c
Martin Geisler <mg@daimi.au.dk>
parents:
diff changeset
320 def parse_dirstate(dmap, copymap, st):
43076
2372284d9457 formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents: 39217
diff changeset
321 parents = [st[:20], st[20:40]]
17425
e95ec38f86b0 fix wording and not-completely-trivial spelling errors and bad docstrings
Mads Kiilerich <mads@kiilerich.com>
parents: 14995
diff changeset
322 # dereference fields so they will be local in loop
43077
687b865b95ad formatting: byteify all mercurial/ and hgext/ string literals
Augie Fackler <augie@google.com>
parents: 43076
diff changeset
323 format = b">cllll"
7945
94d7e14cfa42 pure/parsers: fix circular imports, import mercurial modules properly
Matt Mackall <mpm@selenic.com>
parents: 7873
diff changeset
324 e_size = struct.calcsize(format)
7700
f410c552fa15 pure Python implementation of parsers.c
Martin Geisler <mg@daimi.au.dk>
parents:
diff changeset
325 pos1 = 40
f410c552fa15 pure Python implementation of parsers.c
Martin Geisler <mg@daimi.au.dk>
parents:
diff changeset
326 l = len(st)
f410c552fa15 pure Python implementation of parsers.c
Martin Geisler <mg@daimi.au.dk>
parents:
diff changeset
327
f410c552fa15 pure Python implementation of parsers.c
Martin Geisler <mg@daimi.au.dk>
parents:
diff changeset
328 # the inner loop
f410c552fa15 pure Python implementation of parsers.c
Martin Geisler <mg@daimi.au.dk>
parents:
diff changeset
329 while pos1 < l:
f410c552fa15 pure Python implementation of parsers.c
Martin Geisler <mg@daimi.au.dk>
parents:
diff changeset
330 pos2 = pos1 + e_size
43077
687b865b95ad formatting: byteify all mercurial/ and hgext/ string literals
Augie Fackler <augie@google.com>
parents: 43076
diff changeset
331 e = _unpack(b">cllll", st[pos1:pos2]) # a literal here is faster
7700
f410c552fa15 pure Python implementation of parsers.c
Martin Geisler <mg@daimi.au.dk>
parents:
diff changeset
332 pos1 = pos2 + e[4]
f410c552fa15 pure Python implementation of parsers.c
Martin Geisler <mg@daimi.au.dk>
parents:
diff changeset
333 f = st[pos2:pos1]
43077
687b865b95ad formatting: byteify all mercurial/ and hgext/ string literals
Augie Fackler <augie@google.com>
parents: 43076
diff changeset
334 if b'\0' in f:
687b865b95ad formatting: byteify all mercurial/ and hgext/ string literals
Augie Fackler <augie@google.com>
parents: 43076
diff changeset
335 f, c = f.split(b'\0')
7700
f410c552fa15 pure Python implementation of parsers.c
Martin Geisler <mg@daimi.au.dk>
parents:
diff changeset
336 copymap[f] = c
f410c552fa15 pure Python implementation of parsers.c
Martin Geisler <mg@daimi.au.dk>
parents:
diff changeset
337 dmap[f] = e[:4]
f410c552fa15 pure Python implementation of parsers.c
Martin Geisler <mg@daimi.au.dk>
parents:
diff changeset
338 return parents
18567
194e63c1ccb9 dirstate: move pure python dirstate packing to pure/parsers.py
Siddharth Agarwal <sid0@fb.com>
parents: 17425
diff changeset
339
43076
2372284d9457 formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents: 39217
diff changeset
340
18567
194e63c1ccb9 dirstate: move pure python dirstate packing to pure/parsers.py
Siddharth Agarwal <sid0@fb.com>
parents: 17425
diff changeset
341 def pack_dirstate(dmap, copymap, pl, now):
194e63c1ccb9 dirstate: move pure python dirstate packing to pure/parsers.py
Siddharth Agarwal <sid0@fb.com>
parents: 17425
diff changeset
342 now = int(now)
28861
86db5cb55d46 pycompat: switch to util.stringio for py3 compat
timeless <timeless@mozdev.org>
parents: 27339
diff changeset
343 cs = stringio()
18567
194e63c1ccb9 dirstate: move pure python dirstate packing to pure/parsers.py
Siddharth Agarwal <sid0@fb.com>
parents: 17425
diff changeset
344 write = cs.write
43077
687b865b95ad formatting: byteify all mercurial/ and hgext/ string literals
Augie Fackler <augie@google.com>
parents: 43076
diff changeset
345 write(b"".join(pl))
43106
d783f945a701 py3: finish porting iteritems() to pycompat and remove source transformer
Gregory Szorc <gregory.szorc@gmail.com>
parents: 43077
diff changeset
346 for f, e in pycompat.iteritems(dmap):
43077
687b865b95ad formatting: byteify all mercurial/ and hgext/ string literals
Augie Fackler <augie@google.com>
parents: 43076
diff changeset
347 if e[0] == b'n' and e[3] == now:
18567
194e63c1ccb9 dirstate: move pure python dirstate packing to pure/parsers.py
Siddharth Agarwal <sid0@fb.com>
parents: 17425
diff changeset
348 # The file was last modified "simultaneously" with the current
194e63c1ccb9 dirstate: move pure python dirstate packing to pure/parsers.py
Siddharth Agarwal <sid0@fb.com>
parents: 17425
diff changeset
349 # write to dirstate (i.e. within the same second for file-
194e63c1ccb9 dirstate: move pure python dirstate packing to pure/parsers.py
Siddharth Agarwal <sid0@fb.com>
parents: 17425
diff changeset
350 # systems with a granularity of 1 sec). This commonly happens
194e63c1ccb9 dirstate: move pure python dirstate packing to pure/parsers.py
Siddharth Agarwal <sid0@fb.com>
parents: 17425
diff changeset
351 # for at least a couple of files on 'update'.
194e63c1ccb9 dirstate: move pure python dirstate packing to pure/parsers.py
Siddharth Agarwal <sid0@fb.com>
parents: 17425
diff changeset
352 # The user could change the file without changing its size
19652
187bf2dde7c1 pack_dirstate: only invalidate mtime for files written in the last second
Siddharth Agarwal <sid0@fb.com>
parents: 18567
diff changeset
353 # within the same second. Invalidate the file's mtime in
18567
194e63c1ccb9 dirstate: move pure python dirstate packing to pure/parsers.py
Siddharth Agarwal <sid0@fb.com>
parents: 17425
diff changeset
354 # dirstate, forcing future 'status' calls to compare the
19652
187bf2dde7c1 pack_dirstate: only invalidate mtime for files written in the last second
Siddharth Agarwal <sid0@fb.com>
parents: 18567
diff changeset
355 # contents of the file if the size is the same. This prevents
187bf2dde7c1 pack_dirstate: only invalidate mtime for files written in the last second
Siddharth Agarwal <sid0@fb.com>
parents: 18567
diff changeset
356 # mistakenly treating such files as clean.
21809
e250b8300e6e parsers: inline fields of dirstate values in C version
Siddharth Agarwal <sid0@fb.com>
parents: 19652
diff changeset
357 e = dirstatetuple(e[0], e[1], e[2], -1)
18567
194e63c1ccb9 dirstate: move pure python dirstate packing to pure/parsers.py
Siddharth Agarwal <sid0@fb.com>
parents: 17425
diff changeset
358 dmap[f] = e
194e63c1ccb9 dirstate: move pure python dirstate packing to pure/parsers.py
Siddharth Agarwal <sid0@fb.com>
parents: 17425
diff changeset
359
194e63c1ccb9 dirstate: move pure python dirstate packing to pure/parsers.py
Siddharth Agarwal <sid0@fb.com>
parents: 17425
diff changeset
360 if f in copymap:
43077
687b865b95ad formatting: byteify all mercurial/ and hgext/ string literals
Augie Fackler <augie@google.com>
parents: 43076
diff changeset
361 f = b"%s\0%s" % (f, copymap[f])
687b865b95ad formatting: byteify all mercurial/ and hgext/ string literals
Augie Fackler <augie@google.com>
parents: 43076
diff changeset
362 e = _pack(b">cllll", e[0], e[1], e[2], e[3], len(f))
18567
194e63c1ccb9 dirstate: move pure python dirstate packing to pure/parsers.py
Siddharth Agarwal <sid0@fb.com>
parents: 17425
diff changeset
363 write(e)
194e63c1ccb9 dirstate: move pure python dirstate packing to pure/parsers.py
Siddharth Agarwal <sid0@fb.com>
parents: 17425
diff changeset
364 write(f)
194e63c1ccb9 dirstate: move pure python dirstate packing to pure/parsers.py
Siddharth Agarwal <sid0@fb.com>
parents: 17425
diff changeset
365 return cs.getvalue()