Mercurial > public > mercurial-scm > hg

annotate mercurial/dirstateutils/v2.py @ 48221:a32a96079e2d

Find changesets by keywords (author, files, the commit message), revision number or hash, or revset expression.
dirstate-v2: initial Python parser The dirstate-v2 file format should be supported even if Rust extensions are not enabled. This changeset adds parsing code that is not used yet. Differential Revision: https://phab.mercurial-scm.org/D11518
author Simon Sapin <simon.sapin@octobus.net>
date Sun, 03 Oct 2021 13:18:03 +0200
parents
children 7e78c72ee3ea
Ignore whitespace changes - Everywhere: Within whitespace: At end of lines:
rev   line source
48221
a32a96079e2d dirstate-v2: initial Python parser
Simon Sapin <simon.sapin@octobus.net>
parents:
diff changeset
1 # v2.py - Pure-Python implementation of the dirstate-v2 file format
a32a96079e2d dirstate-v2: initial Python parser
Simon Sapin <simon.sapin@octobus.net>
parents:
diff changeset
2 #
a32a96079e2d dirstate-v2: initial Python parser
Simon Sapin <simon.sapin@octobus.net>
parents:
diff changeset
3 # Copyright Mercurial Contributors
a32a96079e2d dirstate-v2: initial Python parser
Simon Sapin <simon.sapin@octobus.net>
parents:
diff changeset
4 #
a32a96079e2d dirstate-v2: initial Python parser
Simon Sapin <simon.sapin@octobus.net>
parents:
diff changeset
5 # This software may be used and distributed according to the terms of the
a32a96079e2d dirstate-v2: initial Python parser
Simon Sapin <simon.sapin@octobus.net>
parents:
diff changeset
6 # GNU General Public License version 2 or any later version.
a32a96079e2d dirstate-v2: initial Python parser
Simon Sapin <simon.sapin@octobus.net>
parents:
diff changeset
7
a32a96079e2d dirstate-v2: initial Python parser
Simon Sapin <simon.sapin@octobus.net>
parents:
diff changeset
8 from __future__ import absolute_import
a32a96079e2d dirstate-v2: initial Python parser
Simon Sapin <simon.sapin@octobus.net>
parents:
diff changeset
9
a32a96079e2d dirstate-v2: initial Python parser
Simon Sapin <simon.sapin@octobus.net>
parents:
diff changeset
10 import struct
a32a96079e2d dirstate-v2: initial Python parser
Simon Sapin <simon.sapin@octobus.net>
parents:
diff changeset
11
a32a96079e2d dirstate-v2: initial Python parser
Simon Sapin <simon.sapin@octobus.net>
parents:
diff changeset
12 from .. import policy
a32a96079e2d dirstate-v2: initial Python parser
Simon Sapin <simon.sapin@octobus.net>
parents:
diff changeset
13
a32a96079e2d dirstate-v2: initial Python parser
Simon Sapin <simon.sapin@octobus.net>
parents:
diff changeset
14 parsers = policy.importmod('parsers')
a32a96079e2d dirstate-v2: initial Python parser
Simon Sapin <simon.sapin@octobus.net>
parents:
diff changeset
15
a32a96079e2d dirstate-v2: initial Python parser
Simon Sapin <simon.sapin@octobus.net>
parents:
diff changeset
16
a32a96079e2d dirstate-v2: initial Python parser
Simon Sapin <simon.sapin@octobus.net>
parents:
diff changeset
17 # Must match the constant of the same name in
a32a96079e2d dirstate-v2: initial Python parser
Simon Sapin <simon.sapin@octobus.net>
parents:
diff changeset
18 # `rust/hg-core/src/dirstate_tree/on_disk.rs`
a32a96079e2d dirstate-v2: initial Python parser
Simon Sapin <simon.sapin@octobus.net>
parents:
diff changeset
19 TREE_METADATA_SIZE = 44
a32a96079e2d dirstate-v2: initial Python parser
Simon Sapin <simon.sapin@octobus.net>
parents:
diff changeset
20 NODE_SIZE = 43
a32a96079e2d dirstate-v2: initial Python parser
Simon Sapin <simon.sapin@octobus.net>
parents:
diff changeset
21
a32a96079e2d dirstate-v2: initial Python parser
Simon Sapin <simon.sapin@octobus.net>
parents:
diff changeset
22
a32a96079e2d dirstate-v2: initial Python parser
Simon Sapin <simon.sapin@octobus.net>
parents:
diff changeset
23 # Must match the `TreeMetadata` Rust struct in
a32a96079e2d dirstate-v2: initial Python parser
Simon Sapin <simon.sapin@octobus.net>
parents:
diff changeset
24 # `rust/hg-core/src/dirstate_tree/on_disk.rs`. See doc-comments there.
a32a96079e2d dirstate-v2: initial Python parser
Simon Sapin <simon.sapin@octobus.net>
parents:
diff changeset
25 #
a32a96079e2d dirstate-v2: initial Python parser
Simon Sapin <simon.sapin@octobus.net>
parents:
diff changeset
26 # * 4 bytes: start offset of root nodes
a32a96079e2d dirstate-v2: initial Python parser
Simon Sapin <simon.sapin@octobus.net>
parents:
diff changeset
27 # * 4 bytes: number of root nodes
a32a96079e2d dirstate-v2: initial Python parser
Simon Sapin <simon.sapin@octobus.net>
parents:
diff changeset
28 # * 4 bytes: total number of nodes in the tree that have an entry
a32a96079e2d dirstate-v2: initial Python parser
Simon Sapin <simon.sapin@octobus.net>
parents:
diff changeset
29 # * 4 bytes: total number of nodes in the tree that have a copy source
a32a96079e2d dirstate-v2: initial Python parser
Simon Sapin <simon.sapin@octobus.net>
parents:
diff changeset
30 # * 4 bytes: number of bytes in the data file that are not used anymore
a32a96079e2d dirstate-v2: initial Python parser
Simon Sapin <simon.sapin@octobus.net>
parents:
diff changeset
31 # * 4 bytes: unused
a32a96079e2d dirstate-v2: initial Python parser
Simon Sapin <simon.sapin@octobus.net>
parents:
diff changeset
32 # * 20 bytes: SHA-1 hash of ignore patterns
a32a96079e2d dirstate-v2: initial Python parser
Simon Sapin <simon.sapin@octobus.net>
parents:
diff changeset
33 TREE_METADATA = struct.Struct('>LLLLL4s20s')
a32a96079e2d dirstate-v2: initial Python parser
Simon Sapin <simon.sapin@octobus.net>
parents:
diff changeset
34
a32a96079e2d dirstate-v2: initial Python parser
Simon Sapin <simon.sapin@octobus.net>
parents:
diff changeset
35
a32a96079e2d dirstate-v2: initial Python parser
Simon Sapin <simon.sapin@octobus.net>
parents:
diff changeset
36 # Must match the `Node` Rust struct in
a32a96079e2d dirstate-v2: initial Python parser
Simon Sapin <simon.sapin@octobus.net>
parents:
diff changeset
37 # `rust/hg-core/src/dirstate_tree/on_disk.rs`. See doc-comments there.
a32a96079e2d dirstate-v2: initial Python parser
Simon Sapin <simon.sapin@octobus.net>
parents:
diff changeset
38 #
a32a96079e2d dirstate-v2: initial Python parser
Simon Sapin <simon.sapin@octobus.net>
parents:
diff changeset
39 # * 4 bytes: start offset of full path
a32a96079e2d dirstate-v2: initial Python parser
Simon Sapin <simon.sapin@octobus.net>
parents:
diff changeset
40 # * 2 bytes: length of the full path
a32a96079e2d dirstate-v2: initial Python parser
Simon Sapin <simon.sapin@octobus.net>
parents:
diff changeset
41 # * 2 bytes: length within the full path before its "base name"
a32a96079e2d dirstate-v2: initial Python parser
Simon Sapin <simon.sapin@octobus.net>
parents:
diff changeset
42 # * 4 bytes: start offset of the copy source if any, or zero for no copy source
a32a96079e2d dirstate-v2: initial Python parser
Simon Sapin <simon.sapin@octobus.net>
parents:
diff changeset
43 # * 2 bytes: length of the copy source if any, or unused
a32a96079e2d dirstate-v2: initial Python parser
Simon Sapin <simon.sapin@octobus.net>
parents:
diff changeset
44 # * 4 bytes: start offset of child nodes
a32a96079e2d dirstate-v2: initial Python parser
Simon Sapin <simon.sapin@octobus.net>
parents:
diff changeset
45 # * 4 bytes: number of child nodes
a32a96079e2d dirstate-v2: initial Python parser
Simon Sapin <simon.sapin@octobus.net>
parents:
diff changeset
46 # * 4 bytes: number of descendant nodes that have an entry
a32a96079e2d dirstate-v2: initial Python parser
Simon Sapin <simon.sapin@octobus.net>
parents:
diff changeset
47 # * 4 bytes: number of descendant nodes that have a "tracked" state
a32a96079e2d dirstate-v2: initial Python parser
Simon Sapin <simon.sapin@octobus.net>
parents:
diff changeset
48 # * 1 byte: flags
a32a96079e2d dirstate-v2: initial Python parser
Simon Sapin <simon.sapin@octobus.net>
parents:
diff changeset
49 # * 4 bytes: expected size
a32a96079e2d dirstate-v2: initial Python parser
Simon Sapin <simon.sapin@octobus.net>
parents:
diff changeset
50 # * 4 bytes: mtime seconds
a32a96079e2d dirstate-v2: initial Python parser
Simon Sapin <simon.sapin@octobus.net>
parents:
diff changeset
51 # * 4 bytes: mtime nanoseconds
a32a96079e2d dirstate-v2: initial Python parser
Simon Sapin <simon.sapin@octobus.net>
parents:
diff changeset
52 NODE = struct.Struct('>LHHLHLLLLBlll')
a32a96079e2d dirstate-v2: initial Python parser
Simon Sapin <simon.sapin@octobus.net>
parents:
diff changeset
53
a32a96079e2d dirstate-v2: initial Python parser
Simon Sapin <simon.sapin@octobus.net>
parents:
diff changeset
54
a32a96079e2d dirstate-v2: initial Python parser
Simon Sapin <simon.sapin@octobus.net>
parents:
diff changeset
55 assert TREE_METADATA_SIZE == TREE_METADATA.size
a32a96079e2d dirstate-v2: initial Python parser
Simon Sapin <simon.sapin@octobus.net>
parents:
diff changeset
56 assert NODE_SIZE == NODE.size
a32a96079e2d dirstate-v2: initial Python parser
Simon Sapin <simon.sapin@octobus.net>
parents:
diff changeset
57
a32a96079e2d dirstate-v2: initial Python parser
Simon Sapin <simon.sapin@octobus.net>
parents:
diff changeset
58
a32a96079e2d dirstate-v2: initial Python parser
Simon Sapin <simon.sapin@octobus.net>
parents:
diff changeset
59 def parse_dirstate(map, copy_map, data, tree_metadata):
a32a96079e2d dirstate-v2: initial Python parser
Simon Sapin <simon.sapin@octobus.net>
parents:
diff changeset
60 """parse a full v2-dirstate from a binary data into dictionnaries:
a32a96079e2d dirstate-v2: initial Python parser
Simon Sapin <simon.sapin@octobus.net>
parents:
diff changeset
61
a32a96079e2d dirstate-v2: initial Python parser
Simon Sapin <simon.sapin@octobus.net>
parents:
diff changeset
62 - map: a {path: entry} mapping that will be filled
a32a96079e2d dirstate-v2: initial Python parser
Simon Sapin <simon.sapin@octobus.net>
parents:
diff changeset
63 - copy_map: a {path: copy-source} mapping that will be filled
a32a96079e2d dirstate-v2: initial Python parser
Simon Sapin <simon.sapin@octobus.net>
parents:
diff changeset
64 - data: a binary blob contains v2 nodes data
a32a96079e2d dirstate-v2: initial Python parser
Simon Sapin <simon.sapin@octobus.net>
parents:
diff changeset
65 - tree_metadata:: a binary blob of the top level node (from the docket)
a32a96079e2d dirstate-v2: initial Python parser
Simon Sapin <simon.sapin@octobus.net>
parents:
diff changeset
66 """
a32a96079e2d dirstate-v2: initial Python parser
Simon Sapin <simon.sapin@octobus.net>
parents:
diff changeset
67 (
a32a96079e2d dirstate-v2: initial Python parser
Simon Sapin <simon.sapin@octobus.net>
parents:
diff changeset
68 root_nodes_start,
a32a96079e2d dirstate-v2: initial Python parser
Simon Sapin <simon.sapin@octobus.net>
parents:
diff changeset
69 root_nodes_len,
a32a96079e2d dirstate-v2: initial Python parser
Simon Sapin <simon.sapin@octobus.net>
parents:
diff changeset
70 _nodes_with_entry_count,
a32a96079e2d dirstate-v2: initial Python parser
Simon Sapin <simon.sapin@octobus.net>
parents:
diff changeset
71 _nodes_with_copy_source_count,
a32a96079e2d dirstate-v2: initial Python parser
Simon Sapin <simon.sapin@octobus.net>
parents:
diff changeset
72 _unreachable_bytes,
a32a96079e2d dirstate-v2: initial Python parser
Simon Sapin <simon.sapin@octobus.net>
parents:
diff changeset
73 _unused,
a32a96079e2d dirstate-v2: initial Python parser
Simon Sapin <simon.sapin@octobus.net>
parents:
diff changeset
74 _ignore_patterns_hash,
a32a96079e2d dirstate-v2: initial Python parser
Simon Sapin <simon.sapin@octobus.net>
parents:
diff changeset
75 ) = TREE_METADATA.unpack(tree_metadata)
a32a96079e2d dirstate-v2: initial Python parser
Simon Sapin <simon.sapin@octobus.net>
parents:
diff changeset
76 parse_nodes(map, copy_map, data, root_nodes_start, root_nodes_len)
a32a96079e2d dirstate-v2: initial Python parser
Simon Sapin <simon.sapin@octobus.net>
parents:
diff changeset
77
a32a96079e2d dirstate-v2: initial Python parser
Simon Sapin <simon.sapin@octobus.net>
parents:
diff changeset
78
a32a96079e2d dirstate-v2: initial Python parser
Simon Sapin <simon.sapin@octobus.net>
parents:
diff changeset
79 def parse_nodes(map, copy_map, data, start, len):
a32a96079e2d dirstate-v2: initial Python parser
Simon Sapin <simon.sapin@octobus.net>
parents:
diff changeset
80 """parse <len> nodes from <data> starting at offset <start>
a32a96079e2d dirstate-v2: initial Python parser
Simon Sapin <simon.sapin@octobus.net>
parents:
diff changeset
81
a32a96079e2d dirstate-v2: initial Python parser
Simon Sapin <simon.sapin@octobus.net>
parents:
diff changeset
82 This is used by parse_dirstate to recursively fill `map` and `copy_map`.
a32a96079e2d dirstate-v2: initial Python parser
Simon Sapin <simon.sapin@octobus.net>
parents:
diff changeset
83 """
a32a96079e2d dirstate-v2: initial Python parser
Simon Sapin <simon.sapin@octobus.net>
parents:
diff changeset
84 for i in range(len):
a32a96079e2d dirstate-v2: initial Python parser
Simon Sapin <simon.sapin@octobus.net>
parents:
diff changeset
85 node_start = start + NODE_SIZE * i
a32a96079e2d dirstate-v2: initial Python parser
Simon Sapin <simon.sapin@octobus.net>
parents:
diff changeset
86 node_bytes = slice_with_len(data, node_start, NODE_SIZE)
a32a96079e2d dirstate-v2: initial Python parser
Simon Sapin <simon.sapin@octobus.net>
parents:
diff changeset
87 (
a32a96079e2d dirstate-v2: initial Python parser
Simon Sapin <simon.sapin@octobus.net>
parents:
diff changeset
88 path_start,
a32a96079e2d dirstate-v2: initial Python parser
Simon Sapin <simon.sapin@octobus.net>
parents:
diff changeset
89 path_len,
a32a96079e2d dirstate-v2: initial Python parser
Simon Sapin <simon.sapin@octobus.net>
parents:
diff changeset
90 _basename_start,
a32a96079e2d dirstate-v2: initial Python parser
Simon Sapin <simon.sapin@octobus.net>
parents:
diff changeset
91 copy_source_start,
a32a96079e2d dirstate-v2: initial Python parser
Simon Sapin <simon.sapin@octobus.net>
parents:
diff changeset
92 copy_source_len,
a32a96079e2d dirstate-v2: initial Python parser
Simon Sapin <simon.sapin@octobus.net>
parents:
diff changeset
93 children_start,
a32a96079e2d dirstate-v2: initial Python parser
Simon Sapin <simon.sapin@octobus.net>
parents:
diff changeset
94 children_count,
a32a96079e2d dirstate-v2: initial Python parser
Simon Sapin <simon.sapin@octobus.net>
parents:
diff changeset
95 _descendants_with_entry_count,
a32a96079e2d dirstate-v2: initial Python parser
Simon Sapin <simon.sapin@octobus.net>
parents:
diff changeset
96 _tracked_descendants_count,
a32a96079e2d dirstate-v2: initial Python parser
Simon Sapin <simon.sapin@octobus.net>
parents:
diff changeset
97 flags,
a32a96079e2d dirstate-v2: initial Python parser
Simon Sapin <simon.sapin@octobus.net>
parents:
diff changeset
98 size,
a32a96079e2d dirstate-v2: initial Python parser
Simon Sapin <simon.sapin@octobus.net>
parents:
diff changeset
99 mtime_s,
a32a96079e2d dirstate-v2: initial Python parser
Simon Sapin <simon.sapin@octobus.net>
parents:
diff changeset
100 _mtime_ns,
a32a96079e2d dirstate-v2: initial Python parser
Simon Sapin <simon.sapin@octobus.net>
parents:
diff changeset
101 ) = NODE.unpack(node_bytes)
a32a96079e2d dirstate-v2: initial Python parser
Simon Sapin <simon.sapin@octobus.net>
parents:
diff changeset
102
a32a96079e2d dirstate-v2: initial Python parser
Simon Sapin <simon.sapin@octobus.net>
parents:
diff changeset
103 # Parse child nodes of this node recursively
a32a96079e2d dirstate-v2: initial Python parser
Simon Sapin <simon.sapin@octobus.net>
parents:
diff changeset
104 parse_nodes(map, copy_map, data, children_start, children_count)
a32a96079e2d dirstate-v2: initial Python parser
Simon Sapin <simon.sapin@octobus.net>
parents:
diff changeset
105
a32a96079e2d dirstate-v2: initial Python parser
Simon Sapin <simon.sapin@octobus.net>
parents:
diff changeset
106 item = parsers.DirstateItem.from_v2_data(flags, size, mtime_s)
a32a96079e2d dirstate-v2: initial Python parser
Simon Sapin <simon.sapin@octobus.net>
parents:
diff changeset
107 if not item.any_tracked:
a32a96079e2d dirstate-v2: initial Python parser
Simon Sapin <simon.sapin@octobus.net>
parents:
diff changeset
108 continue
a32a96079e2d dirstate-v2: initial Python parser
Simon Sapin <simon.sapin@octobus.net>
parents:
diff changeset
109 path = slice_with_len(data, path_start, path_len)
a32a96079e2d dirstate-v2: initial Python parser
Simon Sapin <simon.sapin@octobus.net>
parents:
diff changeset
110 map[path] = item
a32a96079e2d dirstate-v2: initial Python parser
Simon Sapin <simon.sapin@octobus.net>
parents:
diff changeset
111 if copy_source_start:
a32a96079e2d dirstate-v2: initial Python parser
Simon Sapin <simon.sapin@octobus.net>
parents:
diff changeset
112 copy_map[path] = slice_with_len(
a32a96079e2d dirstate-v2: initial Python parser
Simon Sapin <simon.sapin@octobus.net>
parents:
diff changeset
113 data, copy_source_start, copy_source_len
a32a96079e2d dirstate-v2: initial Python parser
Simon Sapin <simon.sapin@octobus.net>
parents:
diff changeset
114 )
a32a96079e2d dirstate-v2: initial Python parser
Simon Sapin <simon.sapin@octobus.net>
parents:
diff changeset
115
a32a96079e2d dirstate-v2: initial Python parser
Simon Sapin <simon.sapin@octobus.net>
parents:
diff changeset
116
a32a96079e2d dirstate-v2: initial Python parser
Simon Sapin <simon.sapin@octobus.net>
parents:
diff changeset
117 def slice_with_len(data, start, len):
a32a96079e2d dirstate-v2: initial Python parser
Simon Sapin <simon.sapin@octobus.net>
parents:
diff changeset
118 return data[start : start + len]