Mercurial > public > mercurial-scm > hg

annotate mercurial/revlogutils/rewrite.py @ 47472:c81a5297f185

Find changesets by keywords (author, files, the commit message), revision number or hash, or revset expression.
censor: migrate the logic to a set of `censor_revs` Instead of considering a special unique censored revision within the code, we consider a set of revision (currently always of size 1). This make the main code less censor-centric and prepare for the usage of a similar approach for stripping changesets. Differential Revision: https://phab.mercurial-scm.org/D10903
author Pierre-Yves David <pierre-yves.david@octobus.net>
date Tue, 22 Jun 2021 23:20:32 +0200
parents aab064416f0c
children 5045ba2a3afd
Ignore whitespace changes - Everywhere: Within whitespace: At end of lines:
rev   line source
47391
33d626910374 revlog: move censoring code in a dedicated module
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 47389
diff changeset
1 # censor code related to censoring revision
47457
f8330a3fc39f censor: implement censoring for revlogv2
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 47392
diff changeset
2 # coding: utf8
8226
8b2cd04a6e97 put license and copyright info into comment blocks
Martin Geisler <mg@lazybytes.net>
parents: 8225
diff changeset
3 #
47391
33d626910374 revlog: move censoring code in a dedicated module
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 47389
diff changeset
4 # Copyright 2021 Pierre-Yves David <pierre-yves.david@octobus.net>
33d626910374 revlog: move censoring code in a dedicated module
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 47389
diff changeset
5 # Copyright 2015 Google, Inc <martinvonz@google.com>
8226
8b2cd04a6e97 put license and copyright info into comment blocks
Martin Geisler <mg@lazybytes.net>
parents: 8225
diff changeset
6 #
8b2cd04a6e97 put license and copyright info into comment blocks
Martin Geisler <mg@lazybytes.net>
parents: 8225
diff changeset
7 # This software may be used and distributed according to the terms of the
10263
25e572394f5c Update license to GPLv2+
Matt Mackall <mpm@selenic.com>
parents: 10047
diff changeset
8 # GNU General Public License version 2 or any later version.
0
9117c6561b0b Add back links from file revisions to changeset revisions
mpm@selenic.com
parents:
diff changeset
9
47457
f8330a3fc39f censor: implement censoring for revlogv2
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 47392
diff changeset
10 import contextlib
f8330a3fc39f censor: implement censoring for revlogv2
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 47392
diff changeset
11 import os
f8330a3fc39f censor: implement censoring for revlogv2
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 47392
diff changeset
12
47391
33d626910374 revlog: move censoring code in a dedicated module
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 47389
diff changeset
13 from ..node import (
27361
29f50344fa83 revlog: use absolute_import
Gregory Szorc <gregory.szorc@gmail.com>
parents: 27251
diff changeset
14 nullrev,
39767
db088e133e91 revlog: define ellipsis flag processors in core
Gregory Szorc <gregory.szorc@gmail.com>
parents: 39333
diff changeset
15 )
47457
f8330a3fc39f censor: implement censoring for revlogv2
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 47392
diff changeset
16 from .constants import (
f8330a3fc39f censor: implement censoring for revlogv2
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 47392
diff changeset
17 COMP_MODE_PLAIN,
f8330a3fc39f censor: implement censoring for revlogv2
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 47392
diff changeset
18 ENTRY_DATA_COMPRESSED_LENGTH,
f8330a3fc39f censor: implement censoring for revlogv2
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 47392
diff changeset
19 ENTRY_DATA_COMPRESSION_MODE,
f8330a3fc39f censor: implement censoring for revlogv2
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 47392
diff changeset
20 ENTRY_DATA_OFFSET,
f8330a3fc39f censor: implement censoring for revlogv2
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 47392
diff changeset
21 ENTRY_DATA_UNCOMPRESSED_LENGTH,
f8330a3fc39f censor: implement censoring for revlogv2
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 47392
diff changeset
22 ENTRY_DELTA_BASE,
f8330a3fc39f censor: implement censoring for revlogv2
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 47392
diff changeset
23 ENTRY_LINK_REV,
f8330a3fc39f censor: implement censoring for revlogv2
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 47392
diff changeset
24 ENTRY_NODE_ID,
f8330a3fc39f censor: implement censoring for revlogv2
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 47392
diff changeset
25 ENTRY_PARENT_1,
f8330a3fc39f censor: implement censoring for revlogv2
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 47392
diff changeset
26 ENTRY_PARENT_2,
f8330a3fc39f censor: implement censoring for revlogv2
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 47392
diff changeset
27 ENTRY_SIDEDATA_COMPRESSED_LENGTH,
f8330a3fc39f censor: implement censoring for revlogv2
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 47392
diff changeset
28 ENTRY_SIDEDATA_COMPRESSION_MODE,
f8330a3fc39f censor: implement censoring for revlogv2
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 47392
diff changeset
29 ENTRY_SIDEDATA_OFFSET,
f8330a3fc39f censor: implement censoring for revlogv2
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 47392
diff changeset
30 REVLOGV0,
f8330a3fc39f censor: implement censoring for revlogv2
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 47392
diff changeset
31 REVLOGV1,
f8330a3fc39f censor: implement censoring for revlogv2
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 47392
diff changeset
32 )
47391
33d626910374 revlog: move censoring code in a dedicated module
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 47389
diff changeset
33 from ..i18n import _
47457
f8330a3fc39f censor: implement censoring for revlogv2
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 47392
diff changeset
34
47391
33d626910374 revlog: move censoring code in a dedicated module
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 47389
diff changeset
35 from .. import (
33d626910374 revlog: move censoring code in a dedicated module
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 47389
diff changeset
36 error,
47457
f8330a3fc39f censor: implement censoring for revlogv2
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 47392
diff changeset
37 pycompat,
f8330a3fc39f censor: implement censoring for revlogv2
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 47392
diff changeset
38 revlogutils,
f8330a3fc39f censor: implement censoring for revlogv2
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 47392
diff changeset
39 util,
46310
fc2d5c0aed7f persistent-nodemap: add a "warn" option to the slow-path config
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 45942
diff changeset
40 )
47391
33d626910374 revlog: move censoring code in a dedicated module
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 47389
diff changeset
41 from ..utils import (
33d626910374 revlog: move censoring code in a dedicated module
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 47389
diff changeset
42 storageutil,
47386
0d0fb091c49f revlog: simplify "partial read" error message
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 47385
diff changeset
43 )
47457
f8330a3fc39f censor: implement censoring for revlogv2
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 47392
diff changeset
44 from . import (
f8330a3fc39f censor: implement censoring for revlogv2
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 47392
diff changeset
45 constants,
f8330a3fc39f censor: implement censoring for revlogv2
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 47392
diff changeset
46 deltas,
f8330a3fc39f censor: implement censoring for revlogv2
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 47392
diff changeset
47 )
47246
02a4463565ea revlog: improve documentation of the entry tuple
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 47245
diff changeset
48
02a4463565ea revlog: improve documentation of the entry tuple
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 47245
diff changeset
49
47391
33d626910374 revlog: move censoring code in a dedicated module
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 47389
diff changeset
50 def v1_censor(rl, tr, censornode, tombstone=b''):
33d626910374 revlog: move censoring code in a dedicated module
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 47389
diff changeset
51 """censors a revision in a "version 1" revlog"""
33d626910374 revlog: move censoring code in a dedicated module
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 47389
diff changeset
52 assert rl._format_version == constants.REVLOGV1, rl._format_version
47151
24be247a13b4 revlog: stop usage of `_indexfile` to computing nodemap path
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 47150
diff changeset
53
47391
33d626910374 revlog: move censoring code in a dedicated module
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 47389
diff changeset
54 # avoid cycle
33d626910374 revlog: move censoring code in a dedicated module
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 47389
diff changeset
55 from .. import revlog
47144
b6e1fe7ac24b revlog: split the option initialisation in its own method
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 47143
diff changeset
56
47391
33d626910374 revlog: move censoring code in a dedicated module
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 47389
diff changeset
57 censorrev = rl.rev(censornode)
33d626910374 revlog: move censoring code in a dedicated module
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 47389
diff changeset
58 tombstone = storageutil.packmeta({b'censored': tombstone}, b'')
30154
5e72129d75ed revlog: add instance variable controlling delta chain use
Gregory Szorc <gregory.szorc@gmail.com>
parents: 30012
diff changeset
59
47391
33d626910374 revlog: move censoring code in a dedicated module
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 47389
diff changeset
60 # Rewriting the revlog in place is hard. Our strategy for censoring is
33d626910374 revlog: move censoring code in a dedicated module
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 47389
diff changeset
61 # to create a new revlog, copy all revisions to it, then replace the
33d626910374 revlog: move censoring code in a dedicated module
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 47389
diff changeset
62 # revlogs on transaction close.
33d626910374 revlog: move censoring code in a dedicated module
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 47389
diff changeset
63 #
33d626910374 revlog: move censoring code in a dedicated module
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 47389
diff changeset
64 # This is a bit dangerous. We could easily have a mismatch of state.
33d626910374 revlog: move censoring code in a dedicated module
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 47389
diff changeset
65 newrl = revlog.revlog(
33d626910374 revlog: move censoring code in a dedicated module
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 47389
diff changeset
66 rl.opener,
33d626910374 revlog: move censoring code in a dedicated module
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 47389
diff changeset
67 target=rl.target,
33d626910374 revlog: move censoring code in a dedicated module
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 47389
diff changeset
68 radix=rl.radix,
33d626910374 revlog: move censoring code in a dedicated module
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 47389
diff changeset
69 postfix=b'tmpcensored',
33d626910374 revlog: move censoring code in a dedicated module
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 47389
diff changeset
70 censorable=True,
33d626910374 revlog: move censoring code in a dedicated module
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 47389
diff changeset
71 )
33d626910374 revlog: move censoring code in a dedicated module
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 47389
diff changeset
72 newrl._format_version = rl._format_version
33d626910374 revlog: move censoring code in a dedicated module
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 47389
diff changeset
73 newrl._format_flags = rl._format_flags
33d626910374 revlog: move censoring code in a dedicated module
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 47389
diff changeset
74 newrl._generaldelta = rl._generaldelta
33d626910374 revlog: move censoring code in a dedicated module
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 47389
diff changeset
75 newrl._parse_index = rl._parse_index
23337
3a8a763f4197 revlog: add a method to get missing revs incrementally
Siddharth Agarwal <sid0@fb.com>
parents: 23328
diff changeset
76
47391
33d626910374 revlog: move censoring code in a dedicated module
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 47389
diff changeset
77 for rev in rl.revs():
33d626910374 revlog: move censoring code in a dedicated module
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 47389
diff changeset
78 node = rl.node(rev)
33d626910374 revlog: move censoring code in a dedicated module
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 47389
diff changeset
79 p1, p2 = rl.parents(node)
1457
518da3c3b6ce This implements the nodesbetween method, and it removes the newer method
Eric Hopper <hopper@omnifarious.org>
parents: 1351
diff changeset
80
47391
33d626910374 revlog: move censoring code in a dedicated module
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 47389
diff changeset
81 if rev == censorrev:
33d626910374 revlog: move censoring code in a dedicated module
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 47389
diff changeset
82 newrl.addrawrevision(
33d626910374 revlog: move censoring code in a dedicated module
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 47389
diff changeset
83 tombstone,
33d626910374 revlog: move censoring code in a dedicated module
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 47389
diff changeset
84 tr,
33d626910374 revlog: move censoring code in a dedicated module
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 47389
diff changeset
85 rl.linkrev(censorrev),
33d626910374 revlog: move censoring code in a dedicated module
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 47389
diff changeset
86 p1,
33d626910374 revlog: move censoring code in a dedicated module
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 47389
diff changeset
87 p2,
33d626910374 revlog: move censoring code in a dedicated module
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 47389
diff changeset
88 censornode,
33d626910374 revlog: move censoring code in a dedicated module
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 47389
diff changeset
89 constants.REVIDX_ISCENSORED,
47294
93a0abe098e7 revlog: avoid raising no-arg RevlogError for internal flow control
Martin von Zweigbergk <martinvonz@google.com>
parents: 47285
diff changeset
90 )
16665
e410be860393 revlog: speed up prefix matching against nodes
Bryan O'Sullivan <bryano@fb.com>
parents: 16533
diff changeset
91
47391
33d626910374 revlog: move censoring code in a dedicated module
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 47389
diff changeset
92 if newrl.deltaparent(rev) != nullrev:
33d626910374 revlog: move censoring code in a dedicated module
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 47389
diff changeset
93 m = _(b'censored revision stored as delta; cannot censor')
33d626910374 revlog: move censoring code in a dedicated module
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 47389
diff changeset
94 h = _(
33d626910374 revlog: move censoring code in a dedicated module
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 47389
diff changeset
95 b'censoring of revlogs is not fully implemented;'
33d626910374 revlog: move censoring code in a dedicated module
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 47389
diff changeset
96 b' please report this bug'
33d626910374 revlog: move censoring code in a dedicated module
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 47389
diff changeset
97 )
33d626910374 revlog: move censoring code in a dedicated module
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 47389
diff changeset
98 raise error.Abort(m, hint=h)
33d626910374 revlog: move censoring code in a dedicated module
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 47389
diff changeset
99 continue
42980
0d1272783f24 revlog: introduce a `sidedata` method
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 42879
diff changeset
100
47391
33d626910374 revlog: move censoring code in a dedicated module
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 47389
diff changeset
101 if rl.iscensored(rev):
33d626910374 revlog: move censoring code in a dedicated module
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 47389
diff changeset
102 if rl.deltaparent(rev) != nullrev:
33d626910374 revlog: move censoring code in a dedicated module
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 47389
diff changeset
103 m = _(
33d626910374 revlog: move censoring code in a dedicated module
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 47389
diff changeset
104 b'cannot censor due to censored '
33d626910374 revlog: move censoring code in a dedicated module
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 47389
diff changeset
105 b'revision having delta stored'
33d626910374 revlog: move censoring code in a dedicated module
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 47389
diff changeset
106 )
33d626910374 revlog: move censoring code in a dedicated module
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 47389
diff changeset
107 raise error.Abort(m)
33d626910374 revlog: move censoring code in a dedicated module
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 47389
diff changeset
108 rawtext = rl._chunk(rev)
42879
4a3efe0febb5 revlog: stop using `_processflags` directly
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 42876
diff changeset
109 else:
47391
33d626910374 revlog: move censoring code in a dedicated module
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 47389
diff changeset
110 rawtext = rl.rawdata(rev)
8315
c8493310ad9b revlog: use index to find index size
Matt Mackall <mpm@selenic.com>
parents: 8314
diff changeset
111
47391
33d626910374 revlog: move censoring code in a dedicated module
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 47389
diff changeset
112 newrl.addrawrevision(
33d626910374 revlog: move censoring code in a dedicated module
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 47389
diff changeset
113 rawtext, tr, rl.linkrev(rev), p1, p2, node, rl.flags(rev)
43076
2372284d9457 formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents: 43039
diff changeset
114 )
2372284d9457 formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents: 43039
diff changeset
115
47391
33d626910374 revlog: move censoring code in a dedicated module
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 47389
diff changeset
116 tr.addbackup(rl._indexfile, location=b'store')
33d626910374 revlog: move censoring code in a dedicated module
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 47389
diff changeset
117 if not rl._inline:
33d626910374 revlog: move censoring code in a dedicated module
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 47389
diff changeset
118 tr.addbackup(rl._datafile, location=b'store')
1667
daff3ef0de8d verify: notice extra data in indices
Matt Mackall <mpm@selenic.com>
parents: 1660
diff changeset
119
47391
33d626910374 revlog: move censoring code in a dedicated module
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 47389
diff changeset
120 rl.opener.rename(newrl._indexfile, rl._indexfile)
33d626910374 revlog: move censoring code in a dedicated module
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 47389
diff changeset
121 if not rl._inline:
33d626910374 revlog: move censoring code in a dedicated module
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 47389
diff changeset
122 rl.opener.rename(newrl._datafile, rl._datafile)
39778
a6b3c4c1019f revlog: move censor logic out of censor extension
Gregory Szorc <gregory.szorc@gmail.com>
parents: 39775
diff changeset
123
47391
33d626910374 revlog: move censoring code in a dedicated module
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 47389
diff changeset
124 rl.clearcaches()
33d626910374 revlog: move censoring code in a dedicated module
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 47389
diff changeset
125 rl._loadindex()
47457
f8330a3fc39f censor: implement censoring for revlogv2
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 47392
diff changeset
126
f8330a3fc39f censor: implement censoring for revlogv2
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 47392
diff changeset
127
47471
aab064416f0c censor: rename `rl` to `revlog` in the main function
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 47470
diff changeset
128 def v2_censor(revlog, tr, censornode, tombstone=b''):
47457
f8330a3fc39f censor: implement censoring for revlogv2
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 47392
diff changeset
129 """censors a revision in a "version 2" revlog"""
f8330a3fc39f censor: implement censoring for revlogv2
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 47392
diff changeset
130 # General principle
f8330a3fc39f censor: implement censoring for revlogv2
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 47392
diff changeset
131 #
f8330a3fc39f censor: implement censoring for revlogv2
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 47392
diff changeset
132 # We create new revlog files (index/data/sidedata) to copy the content of
f8330a3fc39f censor: implement censoring for revlogv2
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 47392
diff changeset
133 # the existing data without the censored data.
f8330a3fc39f censor: implement censoring for revlogv2
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 47392
diff changeset
134 #
f8330a3fc39f censor: implement censoring for revlogv2
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 47392
diff changeset
135 # We need to recompute new delta for any revision that used the censored
f8330a3fc39f censor: implement censoring for revlogv2
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 47392
diff changeset
136 # revision as delta base. As the cumulative size of the new delta may be
f8330a3fc39f censor: implement censoring for revlogv2
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 47392
diff changeset
137 # large, we store them in a temporary file until they are stored in their
f8330a3fc39f censor: implement censoring for revlogv2
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 47392
diff changeset
138 # final destination.
f8330a3fc39f censor: implement censoring for revlogv2
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 47392
diff changeset
139 #
f8330a3fc39f censor: implement censoring for revlogv2
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 47392
diff changeset
140 # All data before the censored data can be blindly copied. The rest needs
f8330a3fc39f censor: implement censoring for revlogv2
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 47392
diff changeset
141 # to be copied as we go and the associated index entry needs adjustement.
f8330a3fc39f censor: implement censoring for revlogv2
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 47392
diff changeset
142
47471
aab064416f0c censor: rename `rl` to `revlog` in the main function
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 47470
diff changeset
143 assert revlog._format_version != REVLOGV0, revlog._format_version
aab064416f0c censor: rename `rl` to `revlog` in the main function
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 47470
diff changeset
144 assert revlog._format_version != REVLOGV1, revlog._format_version
47457
f8330a3fc39f censor: implement censoring for revlogv2
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 47392
diff changeset
145
47471
aab064416f0c censor: rename `rl` to `revlog` in the main function
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 47470
diff changeset
146 old_index = revlog.index
aab064416f0c censor: rename `rl` to `revlog` in the main function
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 47470
diff changeset
147 docket = revlog._docket
47457
f8330a3fc39f censor: implement censoring for revlogv2
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 47392
diff changeset
148
47472
c81a5297f185 censor: migrate the logic to a set of `censor_revs`
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 47471
diff changeset
149 censor_revs = {revlog.rev(censornode)}
47457
f8330a3fc39f censor: implement censoring for revlogv2
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 47392
diff changeset
150 tombstone = storageutil.packmeta({b'censored': tombstone}, b'')
f8330a3fc39f censor: implement censoring for revlogv2
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 47392
diff changeset
151
47472
c81a5297f185 censor: migrate the logic to a set of `censor_revs`
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 47471
diff changeset
152 first_excl_rev = min(censor_revs)
c81a5297f185 censor: migrate the logic to a set of `censor_revs`
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 47471
diff changeset
153
c81a5297f185 censor: migrate the logic to a set of `censor_revs`
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 47471
diff changeset
154 first_excl_entry = revlog.index[first_excl_rev]
c81a5297f185 censor: migrate the logic to a set of `censor_revs`
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 47471
diff changeset
155 index_cutoff = revlog.index.entry_size * first_excl_rev
c81a5297f185 censor: migrate the logic to a set of `censor_revs`
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 47471
diff changeset
156 data_cutoff = first_excl_entry[ENTRY_DATA_OFFSET] >> 16
c81a5297f185 censor: migrate the logic to a set of `censor_revs`
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 47471
diff changeset
157 sidedata_cutoff = revlog.sidedata_cut_off(first_excl_rev)
47457
f8330a3fc39f censor: implement censoring for revlogv2
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 47392
diff changeset
158
f8330a3fc39f censor: implement censoring for revlogv2
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 47392
diff changeset
159 with pycompat.unnamedtempfile(mode=b"w+b") as tmp_storage:
47469
60c48458ee6c censor: extract the part about recomputing delta in a function
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 47468
diff changeset
160 # rev → (new_base, data_start, data_end, compression_mode)
60c48458ee6c censor: extract the part about recomputing delta in a function
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 47468
diff changeset
161 rewritten_entries = _precompute_rewritten_delta(
47471
aab064416f0c censor: rename `rl` to `revlog` in the main function
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 47470
diff changeset
162 revlog,
47469
60c48458ee6c censor: extract the part about recomputing delta in a function
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 47468
diff changeset
163 old_index,
47472
c81a5297f185 censor: migrate the logic to a set of `censor_revs`
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 47471
diff changeset
164 censor_revs,
47469
60c48458ee6c censor: extract the part about recomputing delta in a function
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 47468
diff changeset
165 tmp_storage,
60c48458ee6c censor: extract the part about recomputing delta in a function
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 47468
diff changeset
166 )
47457
f8330a3fc39f censor: implement censoring for revlogv2
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 47392
diff changeset
167
47470
d6afe1478a2a censor: extract the part about creating and opening new files in a function
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 47469
diff changeset
168 all_files = _setup_new_files(
47471
aab064416f0c censor: rename `rl` to `revlog` in the main function
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 47470
diff changeset
169 revlog,
47470
d6afe1478a2a censor: extract the part about creating and opening new files in a function
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 47469
diff changeset
170 index_cutoff,
d6afe1478a2a censor: extract the part about creating and opening new files in a function
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 47469
diff changeset
171 data_cutoff,
d6afe1478a2a censor: extract the part about creating and opening new files in a function
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 47469
diff changeset
172 sidedata_cutoff,
47457
f8330a3fc39f censor: implement censoring for revlogv2
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 47392
diff changeset
173 )
f8330a3fc39f censor: implement censoring for revlogv2
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 47392
diff changeset
174
f8330a3fc39f censor: implement censoring for revlogv2
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 47392
diff changeset
175 # we dont need to open the old index file since its content already
f8330a3fc39f censor: implement censoring for revlogv2
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 47392
diff changeset
176 # exist in a usable form in `old_index`.
47466
f7a94e2d4470 censor: put the tuple of open files in an explicit variable
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 47465
diff changeset
177 with all_files() as open_files:
f7a94e2d4470 censor: put the tuple of open files in an explicit variable
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 47465
diff changeset
178 (
f7a94e2d4470 censor: put the tuple of open files in an explicit variable
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 47465
diff changeset
179 old_data_file,
f7a94e2d4470 censor: put the tuple of open files in an explicit variable
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 47465
diff changeset
180 old_sidedata_file,
f7a94e2d4470 censor: put the tuple of open files in an explicit variable
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 47465
diff changeset
181 new_index_file,
f7a94e2d4470 censor: put the tuple of open files in an explicit variable
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 47465
diff changeset
182 new_data_file,
f7a94e2d4470 censor: put the tuple of open files in an explicit variable
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 47465
diff changeset
183 new_sidedata_file,
f7a94e2d4470 censor: put the tuple of open files in an explicit variable
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 47465
diff changeset
184 ) = open_files
47457
f8330a3fc39f censor: implement censoring for revlogv2
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 47392
diff changeset
185
47467
3ab267f0cbe4 censor: extract the part about writing the censored revision in a function
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 47466
diff changeset
186 # writing the censored revision
47457
f8330a3fc39f censor: implement censoring for revlogv2
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 47392
diff changeset
187
47468
9b70aa7bcbab censor: extract the part about writing the other revision in a function
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 47467
diff changeset
188 # Writing all subsequent revisions
47472
c81a5297f185 censor: migrate the logic to a set of `censor_revs`
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 47471
diff changeset
189 for rev in range(first_excl_rev, len(old_index)):
c81a5297f185 censor: migrate the logic to a set of `censor_revs`
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 47471
diff changeset
190 if rev in censor_revs:
c81a5297f185 censor: migrate the logic to a set of `censor_revs`
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 47471
diff changeset
191 _rewrite_censor(
c81a5297f185 censor: migrate the logic to a set of `censor_revs`
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 47471
diff changeset
192 revlog,
c81a5297f185 censor: migrate the logic to a set of `censor_revs`
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 47471
diff changeset
193 old_index,
c81a5297f185 censor: migrate the logic to a set of `censor_revs`
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 47471
diff changeset
194 open_files,
c81a5297f185 censor: migrate the logic to a set of `censor_revs`
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 47471
diff changeset
195 rev,
c81a5297f185 censor: migrate the logic to a set of `censor_revs`
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 47471
diff changeset
196 tombstone,
c81a5297f185 censor: migrate the logic to a set of `censor_revs`
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 47471
diff changeset
197 )
c81a5297f185 censor: migrate the logic to a set of `censor_revs`
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 47471
diff changeset
198 else:
c81a5297f185 censor: migrate the logic to a set of `censor_revs`
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 47471
diff changeset
199 _rewrite_simple(
c81a5297f185 censor: migrate the logic to a set of `censor_revs`
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 47471
diff changeset
200 revlog,
c81a5297f185 censor: migrate the logic to a set of `censor_revs`
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 47471
diff changeset
201 old_index,
c81a5297f185 censor: migrate the logic to a set of `censor_revs`
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 47471
diff changeset
202 open_files,
c81a5297f185 censor: migrate the logic to a set of `censor_revs`
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 47471
diff changeset
203 rev,
c81a5297f185 censor: migrate the logic to a set of `censor_revs`
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 47471
diff changeset
204 rewritten_entries,
c81a5297f185 censor: migrate the logic to a set of `censor_revs`
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 47471
diff changeset
205 tmp_storage,
c81a5297f185 censor: migrate the logic to a set of `censor_revs`
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 47471
diff changeset
206 )
47468
9b70aa7bcbab censor: extract the part about writing the other revision in a function
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 47467
diff changeset
207 docket.write(transaction=None, stripping=True)
9b70aa7bcbab censor: extract the part about writing the other revision in a function
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 47467
diff changeset
208
47457
f8330a3fc39f censor: implement censoring for revlogv2
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 47392
diff changeset
209
47469
60c48458ee6c censor: extract the part about recomputing delta in a function
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 47468
diff changeset
210 def _precompute_rewritten_delta(
60c48458ee6c censor: extract the part about recomputing delta in a function
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 47468
diff changeset
211 revlog,
60c48458ee6c censor: extract the part about recomputing delta in a function
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 47468
diff changeset
212 old_index,
60c48458ee6c censor: extract the part about recomputing delta in a function
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 47468
diff changeset
213 excluded_revs,
60c48458ee6c censor: extract the part about recomputing delta in a function
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 47468
diff changeset
214 tmp_storage,
60c48458ee6c censor: extract the part about recomputing delta in a function
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 47468
diff changeset
215 ):
60c48458ee6c censor: extract the part about recomputing delta in a function
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 47468
diff changeset
216 """Compute new delta for revisions whose delta is based on revision that
60c48458ee6c censor: extract the part about recomputing delta in a function
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 47468
diff changeset
217 will not survive as is.
60c48458ee6c censor: extract the part about recomputing delta in a function
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 47468
diff changeset
218
60c48458ee6c censor: extract the part about recomputing delta in a function
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 47468
diff changeset
219 Return a mapping: {rev → (new_base, data_start, data_end, compression_mode)}
60c48458ee6c censor: extract the part about recomputing delta in a function
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 47468
diff changeset
220 """
60c48458ee6c censor: extract the part about recomputing delta in a function
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 47468
diff changeset
221 dc = deltas.deltacomputer(revlog)
60c48458ee6c censor: extract the part about recomputing delta in a function
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 47468
diff changeset
222 rewritten_entries = {}
60c48458ee6c censor: extract the part about recomputing delta in a function
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 47468
diff changeset
223 first_excl_rev = min(excluded_revs)
60c48458ee6c censor: extract the part about recomputing delta in a function
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 47468
diff changeset
224 with revlog._segmentfile._open_read() as dfh:
60c48458ee6c censor: extract the part about recomputing delta in a function
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 47468
diff changeset
225 for rev in range(first_excl_rev, len(old_index)):
60c48458ee6c censor: extract the part about recomputing delta in a function
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 47468
diff changeset
226 if rev in excluded_revs:
60c48458ee6c censor: extract the part about recomputing delta in a function
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 47468
diff changeset
227 # this revision will be preserved as is, so we don't need to
60c48458ee6c censor: extract the part about recomputing delta in a function
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 47468
diff changeset
228 # consider recomputing a delta.
60c48458ee6c censor: extract the part about recomputing delta in a function
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 47468
diff changeset
229 continue
60c48458ee6c censor: extract the part about recomputing delta in a function
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 47468
diff changeset
230 entry = old_index[rev]
60c48458ee6c censor: extract the part about recomputing delta in a function
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 47468
diff changeset
231 if entry[ENTRY_DELTA_BASE] not in excluded_revs:
60c48458ee6c censor: extract the part about recomputing delta in a function
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 47468
diff changeset
232 continue
60c48458ee6c censor: extract the part about recomputing delta in a function
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 47468
diff changeset
233 # This is a revision that use the censored revision as the base
60c48458ee6c censor: extract the part about recomputing delta in a function
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 47468
diff changeset
234 # for its delta. We need a need new deltas
60c48458ee6c censor: extract the part about recomputing delta in a function
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 47468
diff changeset
235 if entry[ENTRY_DATA_UNCOMPRESSED_LENGTH] == 0:
60c48458ee6c censor: extract the part about recomputing delta in a function
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 47468
diff changeset
236 # this revision is empty, we can delta against nullrev
60c48458ee6c censor: extract the part about recomputing delta in a function
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 47468
diff changeset
237 rewritten_entries[rev] = (nullrev, 0, 0, COMP_MODE_PLAIN)
60c48458ee6c censor: extract the part about recomputing delta in a function
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 47468
diff changeset
238 else:
60c48458ee6c censor: extract the part about recomputing delta in a function
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 47468
diff changeset
239
60c48458ee6c censor: extract the part about recomputing delta in a function
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 47468
diff changeset
240 text = revlog.rawdata(rev, _df=dfh)
60c48458ee6c censor: extract the part about recomputing delta in a function
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 47468
diff changeset
241 info = revlogutils.revisioninfo(
60c48458ee6c censor: extract the part about recomputing delta in a function
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 47468
diff changeset
242 node=entry[ENTRY_NODE_ID],
60c48458ee6c censor: extract the part about recomputing delta in a function
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 47468
diff changeset
243 p1=revlog.node(entry[ENTRY_PARENT_1]),
60c48458ee6c censor: extract the part about recomputing delta in a function
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 47468
diff changeset
244 p2=revlog.node(entry[ENTRY_PARENT_2]),
60c48458ee6c censor: extract the part about recomputing delta in a function
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 47468
diff changeset
245 btext=[text],
60c48458ee6c censor: extract the part about recomputing delta in a function
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 47468
diff changeset
246 textlen=len(text),
60c48458ee6c censor: extract the part about recomputing delta in a function
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 47468
diff changeset
247 cachedelta=None,
60c48458ee6c censor: extract the part about recomputing delta in a function
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 47468
diff changeset
248 flags=entry[ENTRY_DATA_OFFSET] & 0xFFFF,
60c48458ee6c censor: extract the part about recomputing delta in a function
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 47468
diff changeset
249 )
60c48458ee6c censor: extract the part about recomputing delta in a function
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 47468
diff changeset
250 d = dc.finddeltainfo(
60c48458ee6c censor: extract the part about recomputing delta in a function
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 47468
diff changeset
251 info, dfh, excluded_bases=excluded_revs, target_rev=rev
60c48458ee6c censor: extract the part about recomputing delta in a function
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 47468
diff changeset
252 )
60c48458ee6c censor: extract the part about recomputing delta in a function
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 47468
diff changeset
253 default_comp = revlog._docket.default_compression_header
60c48458ee6c censor: extract the part about recomputing delta in a function
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 47468
diff changeset
254 comp_mode, d = deltas.delta_compression(default_comp, d)
60c48458ee6c censor: extract the part about recomputing delta in a function
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 47468
diff changeset
255 # using `tell` is a bit lazy, but we are not here for speed
60c48458ee6c censor: extract the part about recomputing delta in a function
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 47468
diff changeset
256 start = tmp_storage.tell()
60c48458ee6c censor: extract the part about recomputing delta in a function
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 47468
diff changeset
257 tmp_storage.write(d.data[1])
60c48458ee6c censor: extract the part about recomputing delta in a function
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 47468
diff changeset
258 end = tmp_storage.tell()
60c48458ee6c censor: extract the part about recomputing delta in a function
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 47468
diff changeset
259 rewritten_entries[rev] = (d.base, start, end, comp_mode)
60c48458ee6c censor: extract the part about recomputing delta in a function
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 47468
diff changeset
260 return rewritten_entries
60c48458ee6c censor: extract the part about recomputing delta in a function
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 47468
diff changeset
261
60c48458ee6c censor: extract the part about recomputing delta in a function
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 47468
diff changeset
262
47470
d6afe1478a2a censor: extract the part about creating and opening new files in a function
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 47469
diff changeset
263 def _setup_new_files(
d6afe1478a2a censor: extract the part about creating and opening new files in a function
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 47469
diff changeset
264 revlog,
d6afe1478a2a censor: extract the part about creating and opening new files in a function
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 47469
diff changeset
265 index_cutoff,
d6afe1478a2a censor: extract the part about creating and opening new files in a function
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 47469
diff changeset
266 data_cutoff,
d6afe1478a2a censor: extract the part about creating and opening new files in a function
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 47469
diff changeset
267 sidedata_cutoff,
d6afe1478a2a censor: extract the part about creating and opening new files in a function
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 47469
diff changeset
268 ):
d6afe1478a2a censor: extract the part about creating and opening new files in a function
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 47469
diff changeset
269 """
d6afe1478a2a censor: extract the part about creating and opening new files in a function
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 47469
diff changeset
270
d6afe1478a2a censor: extract the part about creating and opening new files in a function
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 47469
diff changeset
271 return a context manager to open all the relevant files:
d6afe1478a2a censor: extract the part about creating and opening new files in a function
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 47469
diff changeset
272 - old_data_file,
d6afe1478a2a censor: extract the part about creating and opening new files in a function
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 47469
diff changeset
273 - old_sidedata_file,
d6afe1478a2a censor: extract the part about creating and opening new files in a function
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 47469
diff changeset
274 - new_index_file,
d6afe1478a2a censor: extract the part about creating and opening new files in a function
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 47469
diff changeset
275 - new_data_file,
d6afe1478a2a censor: extract the part about creating and opening new files in a function
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 47469
diff changeset
276 - new_sidedata_file,
d6afe1478a2a censor: extract the part about creating and opening new files in a function
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 47469
diff changeset
277
d6afe1478a2a censor: extract the part about creating and opening new files in a function
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 47469
diff changeset
278 The old_index_file is not here because it is accessed through the
d6afe1478a2a censor: extract the part about creating and opening new files in a function
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 47469
diff changeset
279 `old_index` object if the caller function.
d6afe1478a2a censor: extract the part about creating and opening new files in a function
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 47469
diff changeset
280 """
d6afe1478a2a censor: extract the part about creating and opening new files in a function
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 47469
diff changeset
281 docket = revlog._docket
d6afe1478a2a censor: extract the part about creating and opening new files in a function
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 47469
diff changeset
282 old_index_filepath = revlog.opener.join(docket.index_filepath())
d6afe1478a2a censor: extract the part about creating and opening new files in a function
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 47469
diff changeset
283 old_data_filepath = revlog.opener.join(docket.data_filepath())
d6afe1478a2a censor: extract the part about creating and opening new files in a function
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 47469
diff changeset
284 old_sidedata_filepath = revlog.opener.join(docket.sidedata_filepath())
d6afe1478a2a censor: extract the part about creating and opening new files in a function
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 47469
diff changeset
285
d6afe1478a2a censor: extract the part about creating and opening new files in a function
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 47469
diff changeset
286 new_index_filepath = revlog.opener.join(docket.new_index_file())
d6afe1478a2a censor: extract the part about creating and opening new files in a function
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 47469
diff changeset
287 new_data_filepath = revlog.opener.join(docket.new_data_file())
d6afe1478a2a censor: extract the part about creating and opening new files in a function
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 47469
diff changeset
288 new_sidedata_filepath = revlog.opener.join(docket.new_sidedata_file())
d6afe1478a2a censor: extract the part about creating and opening new files in a function
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 47469
diff changeset
289
d6afe1478a2a censor: extract the part about creating and opening new files in a function
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 47469
diff changeset
290 util.copyfile(old_index_filepath, new_index_filepath, nb_bytes=index_cutoff)
d6afe1478a2a censor: extract the part about creating and opening new files in a function
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 47469
diff changeset
291 util.copyfile(old_data_filepath, new_data_filepath, nb_bytes=data_cutoff)
d6afe1478a2a censor: extract the part about creating and opening new files in a function
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 47469
diff changeset
292 util.copyfile(
d6afe1478a2a censor: extract the part about creating and opening new files in a function
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 47469
diff changeset
293 old_sidedata_filepath,
d6afe1478a2a censor: extract the part about creating and opening new files in a function
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 47469
diff changeset
294 new_sidedata_filepath,
d6afe1478a2a censor: extract the part about creating and opening new files in a function
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 47469
diff changeset
295 nb_bytes=sidedata_cutoff,
d6afe1478a2a censor: extract the part about creating and opening new files in a function
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 47469
diff changeset
296 )
d6afe1478a2a censor: extract the part about creating and opening new files in a function
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 47469
diff changeset
297 revlog.opener.register_file(docket.index_filepath())
d6afe1478a2a censor: extract the part about creating and opening new files in a function
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 47469
diff changeset
298 revlog.opener.register_file(docket.data_filepath())
d6afe1478a2a censor: extract the part about creating and opening new files in a function
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 47469
diff changeset
299 revlog.opener.register_file(docket.sidedata_filepath())
d6afe1478a2a censor: extract the part about creating and opening new files in a function
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 47469
diff changeset
300
d6afe1478a2a censor: extract the part about creating and opening new files in a function
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 47469
diff changeset
301 docket.index_end = index_cutoff
d6afe1478a2a censor: extract the part about creating and opening new files in a function
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 47469
diff changeset
302 docket.data_end = data_cutoff
d6afe1478a2a censor: extract the part about creating and opening new files in a function
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 47469
diff changeset
303 docket.sidedata_end = sidedata_cutoff
d6afe1478a2a censor: extract the part about creating and opening new files in a function
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 47469
diff changeset
304
d6afe1478a2a censor: extract the part about creating and opening new files in a function
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 47469
diff changeset
305 # reload the revlog internal information
d6afe1478a2a censor: extract the part about creating and opening new files in a function
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 47469
diff changeset
306 revlog.clearcaches()
d6afe1478a2a censor: extract the part about creating and opening new files in a function
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 47469
diff changeset
307 revlog._loadindex(docket=docket)
d6afe1478a2a censor: extract the part about creating and opening new files in a function
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 47469
diff changeset
308
d6afe1478a2a censor: extract the part about creating and opening new files in a function
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 47469
diff changeset
309 @contextlib.contextmanager
d6afe1478a2a censor: extract the part about creating and opening new files in a function
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 47469
diff changeset
310 def all_files_opener():
d6afe1478a2a censor: extract the part about creating and opening new files in a function
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 47469
diff changeset
311 # hide opening in an helper function to please check-code, black
d6afe1478a2a censor: extract the part about creating and opening new files in a function
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 47469
diff changeset
312 # and various python version at the same time
d6afe1478a2a censor: extract the part about creating and opening new files in a function
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 47469
diff changeset
313 with open(old_data_filepath, 'rb') as old_data_file:
d6afe1478a2a censor: extract the part about creating and opening new files in a function
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 47469
diff changeset
314 with open(old_sidedata_filepath, 'rb') as old_sidedata_file:
d6afe1478a2a censor: extract the part about creating and opening new files in a function
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 47469
diff changeset
315 with open(new_index_filepath, 'r+b') as new_index_file:
d6afe1478a2a censor: extract the part about creating and opening new files in a function
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 47469
diff changeset
316 with open(new_data_filepath, 'r+b') as new_data_file:
d6afe1478a2a censor: extract the part about creating and opening new files in a function
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 47469
diff changeset
317 with open(
d6afe1478a2a censor: extract the part about creating and opening new files in a function
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 47469
diff changeset
318 new_sidedata_filepath, 'r+b'
d6afe1478a2a censor: extract the part about creating and opening new files in a function
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 47469
diff changeset
319 ) as new_sidedata_file:
d6afe1478a2a censor: extract the part about creating and opening new files in a function
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 47469
diff changeset
320 new_index_file.seek(0, os.SEEK_END)
d6afe1478a2a censor: extract the part about creating and opening new files in a function
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 47469
diff changeset
321 assert new_index_file.tell() == index_cutoff
d6afe1478a2a censor: extract the part about creating and opening new files in a function
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 47469
diff changeset
322 new_data_file.seek(0, os.SEEK_END)
d6afe1478a2a censor: extract the part about creating and opening new files in a function
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 47469
diff changeset
323 assert new_data_file.tell() == data_cutoff
d6afe1478a2a censor: extract the part about creating and opening new files in a function
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 47469
diff changeset
324 new_sidedata_file.seek(0, os.SEEK_END)
d6afe1478a2a censor: extract the part about creating and opening new files in a function
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 47469
diff changeset
325 assert new_sidedata_file.tell() == sidedata_cutoff
d6afe1478a2a censor: extract the part about creating and opening new files in a function
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 47469
diff changeset
326 yield (
d6afe1478a2a censor: extract the part about creating and opening new files in a function
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 47469
diff changeset
327 old_data_file,
d6afe1478a2a censor: extract the part about creating and opening new files in a function
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 47469
diff changeset
328 old_sidedata_file,
d6afe1478a2a censor: extract the part about creating and opening new files in a function
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 47469
diff changeset
329 new_index_file,
d6afe1478a2a censor: extract the part about creating and opening new files in a function
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 47469
diff changeset
330 new_data_file,
d6afe1478a2a censor: extract the part about creating and opening new files in a function
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 47469
diff changeset
331 new_sidedata_file,
d6afe1478a2a censor: extract the part about creating and opening new files in a function
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 47469
diff changeset
332 )
d6afe1478a2a censor: extract the part about creating and opening new files in a function
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 47469
diff changeset
333
d6afe1478a2a censor: extract the part about creating and opening new files in a function
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 47469
diff changeset
334 return all_files_opener
d6afe1478a2a censor: extract the part about creating and opening new files in a function
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 47469
diff changeset
335
d6afe1478a2a censor: extract the part about creating and opening new files in a function
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 47469
diff changeset
336
47468
9b70aa7bcbab censor: extract the part about writing the other revision in a function
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 47467
diff changeset
337 def _rewrite_simple(
9b70aa7bcbab censor: extract the part about writing the other revision in a function
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 47467
diff changeset
338 revlog,
9b70aa7bcbab censor: extract the part about writing the other revision in a function
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 47467
diff changeset
339 old_index,
9b70aa7bcbab censor: extract the part about writing the other revision in a function
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 47467
diff changeset
340 all_files,
9b70aa7bcbab censor: extract the part about writing the other revision in a function
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 47467
diff changeset
341 rev,
9b70aa7bcbab censor: extract the part about writing the other revision in a function
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 47467
diff changeset
342 rewritten_entries,
9b70aa7bcbab censor: extract the part about writing the other revision in a function
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 47467
diff changeset
343 tmp_storage,
9b70aa7bcbab censor: extract the part about writing the other revision in a function
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 47467
diff changeset
344 ):
9b70aa7bcbab censor: extract the part about writing the other revision in a function
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 47467
diff changeset
345 """append a normal revision to the index after the rewritten one(s)"""
9b70aa7bcbab censor: extract the part about writing the other revision in a function
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 47467
diff changeset
346 (
9b70aa7bcbab censor: extract the part about writing the other revision in a function
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 47467
diff changeset
347 old_data_file,
9b70aa7bcbab censor: extract the part about writing the other revision in a function
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 47467
diff changeset
348 old_sidedata_file,
9b70aa7bcbab censor: extract the part about writing the other revision in a function
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 47467
diff changeset
349 new_index_file,
9b70aa7bcbab censor: extract the part about writing the other revision in a function
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 47467
diff changeset
350 new_data_file,
9b70aa7bcbab censor: extract the part about writing the other revision in a function
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 47467
diff changeset
351 new_sidedata_file,
9b70aa7bcbab censor: extract the part about writing the other revision in a function
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 47467
diff changeset
352 ) = all_files
9b70aa7bcbab censor: extract the part about writing the other revision in a function
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 47467
diff changeset
353 entry = old_index[rev]
9b70aa7bcbab censor: extract the part about writing the other revision in a function
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 47467
diff changeset
354 flags = entry[ENTRY_DATA_OFFSET] & 0xFFFF
9b70aa7bcbab censor: extract the part about writing the other revision in a function
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 47467
diff changeset
355 old_data_offset = entry[ENTRY_DATA_OFFSET] >> 16
47457
f8330a3fc39f censor: implement censoring for revlogv2
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 47392
diff changeset
356
47468
9b70aa7bcbab censor: extract the part about writing the other revision in a function
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 47467
diff changeset
357 if rev not in rewritten_entries:
9b70aa7bcbab censor: extract the part about writing the other revision in a function
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 47467
diff changeset
358 old_data_file.seek(old_data_offset)
9b70aa7bcbab censor: extract the part about writing the other revision in a function
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 47467
diff changeset
359 new_data_size = entry[ENTRY_DATA_COMPRESSED_LENGTH]
9b70aa7bcbab censor: extract the part about writing the other revision in a function
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 47467
diff changeset
360 new_data = old_data_file.read(new_data_size)
9b70aa7bcbab censor: extract the part about writing the other revision in a function
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 47467
diff changeset
361 data_delta_base = entry[ENTRY_DELTA_BASE]
9b70aa7bcbab censor: extract the part about writing the other revision in a function
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 47467
diff changeset
362 d_comp_mode = entry[ENTRY_DATA_COMPRESSION_MODE]
9b70aa7bcbab censor: extract the part about writing the other revision in a function
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 47467
diff changeset
363 else:
9b70aa7bcbab censor: extract the part about writing the other revision in a function
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 47467
diff changeset
364 (
9b70aa7bcbab censor: extract the part about writing the other revision in a function
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 47467
diff changeset
365 data_delta_base,
9b70aa7bcbab censor: extract the part about writing the other revision in a function
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 47467
diff changeset
366 start,
9b70aa7bcbab censor: extract the part about writing the other revision in a function
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 47467
diff changeset
367 end,
9b70aa7bcbab censor: extract the part about writing the other revision in a function
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 47467
diff changeset
368 d_comp_mode,
9b70aa7bcbab censor: extract the part about writing the other revision in a function
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 47467
diff changeset
369 ) = rewritten_entries[rev]
9b70aa7bcbab censor: extract the part about writing the other revision in a function
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 47467
diff changeset
370 new_data_size = end - start
9b70aa7bcbab censor: extract the part about writing the other revision in a function
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 47467
diff changeset
371 tmp_storage.seek(start)
9b70aa7bcbab censor: extract the part about writing the other revision in a function
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 47467
diff changeset
372 new_data = tmp_storage.read(new_data_size)
47457
f8330a3fc39f censor: implement censoring for revlogv2
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 47392
diff changeset
373
47468
9b70aa7bcbab censor: extract the part about writing the other revision in a function
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 47467
diff changeset
374 # It might be faster to group continuous read/write operation,
9b70aa7bcbab censor: extract the part about writing the other revision in a function
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 47467
diff changeset
375 # however, this is censor, an operation that is not focussed
9b70aa7bcbab censor: extract the part about writing the other revision in a function
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 47467
diff changeset
376 # around stellar performance. So I have not written this
9b70aa7bcbab censor: extract the part about writing the other revision in a function
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 47467
diff changeset
377 # optimisation yet.
9b70aa7bcbab censor: extract the part about writing the other revision in a function
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 47467
diff changeset
378 new_data_offset = new_data_file.tell()
9b70aa7bcbab censor: extract the part about writing the other revision in a function
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 47467
diff changeset
379 new_data_file.write(new_data)
47457
f8330a3fc39f censor: implement censoring for revlogv2
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 47392
diff changeset
380
47468
9b70aa7bcbab censor: extract the part about writing the other revision in a function
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 47467
diff changeset
381 sidedata_size = entry[ENTRY_SIDEDATA_COMPRESSED_LENGTH]
9b70aa7bcbab censor: extract the part about writing the other revision in a function
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 47467
diff changeset
382 new_sidedata_offset = new_sidedata_file.tell()
9b70aa7bcbab censor: extract the part about writing the other revision in a function
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 47467
diff changeset
383 if 0 < sidedata_size:
9b70aa7bcbab censor: extract the part about writing the other revision in a function
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 47467
diff changeset
384 old_sidedata_offset = entry[ENTRY_SIDEDATA_OFFSET]
9b70aa7bcbab censor: extract the part about writing the other revision in a function
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 47467
diff changeset
385 old_sidedata_file.seek(old_sidedata_offset)
9b70aa7bcbab censor: extract the part about writing the other revision in a function
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 47467
diff changeset
386 new_sidedata = old_sidedata_file.read(sidedata_size)
9b70aa7bcbab censor: extract the part about writing the other revision in a function
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 47467
diff changeset
387 new_sidedata_file.write(new_sidedata)
9b70aa7bcbab censor: extract the part about writing the other revision in a function
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 47467
diff changeset
388
9b70aa7bcbab censor: extract the part about writing the other revision in a function
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 47467
diff changeset
389 data_uncompressed_length = entry[ENTRY_DATA_UNCOMPRESSED_LENGTH]
9b70aa7bcbab censor: extract the part about writing the other revision in a function
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 47467
diff changeset
390 sd_com_mode = entry[ENTRY_SIDEDATA_COMPRESSION_MODE]
9b70aa7bcbab censor: extract the part about writing the other revision in a function
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 47467
diff changeset
391 assert data_delta_base <= rev, (data_delta_base, rev)
47457
f8330a3fc39f censor: implement censoring for revlogv2
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 47392
diff changeset
392
47468
9b70aa7bcbab censor: extract the part about writing the other revision in a function
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 47467
diff changeset
393 new_entry = revlogutils.entry(
9b70aa7bcbab censor: extract the part about writing the other revision in a function
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 47467
diff changeset
394 flags=flags,
9b70aa7bcbab censor: extract the part about writing the other revision in a function
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 47467
diff changeset
395 data_offset=new_data_offset,
9b70aa7bcbab censor: extract the part about writing the other revision in a function
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 47467
diff changeset
396 data_compressed_length=new_data_size,
9b70aa7bcbab censor: extract the part about writing the other revision in a function
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 47467
diff changeset
397 data_uncompressed_length=data_uncompressed_length,
9b70aa7bcbab censor: extract the part about writing the other revision in a function
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 47467
diff changeset
398 data_delta_base=data_delta_base,
9b70aa7bcbab censor: extract the part about writing the other revision in a function
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 47467
diff changeset
399 link_rev=entry[ENTRY_LINK_REV],
9b70aa7bcbab censor: extract the part about writing the other revision in a function
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 47467
diff changeset
400 parent_rev_1=entry[ENTRY_PARENT_1],
9b70aa7bcbab censor: extract the part about writing the other revision in a function
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 47467
diff changeset
401 parent_rev_2=entry[ENTRY_PARENT_2],
9b70aa7bcbab censor: extract the part about writing the other revision in a function
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 47467
diff changeset
402 node_id=entry[ENTRY_NODE_ID],
9b70aa7bcbab censor: extract the part about writing the other revision in a function
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 47467
diff changeset
403 sidedata_offset=new_sidedata_offset,
9b70aa7bcbab censor: extract the part about writing the other revision in a function
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 47467
diff changeset
404 sidedata_compressed_length=sidedata_size,
9b70aa7bcbab censor: extract the part about writing the other revision in a function
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 47467
diff changeset
405 data_compression_mode=d_comp_mode,
9b70aa7bcbab censor: extract the part about writing the other revision in a function
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 47467
diff changeset
406 sidedata_compression_mode=sd_com_mode,
9b70aa7bcbab censor: extract the part about writing the other revision in a function
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 47467
diff changeset
407 )
9b70aa7bcbab censor: extract the part about writing the other revision in a function
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 47467
diff changeset
408 revlog.index.append(new_entry)
9b70aa7bcbab censor: extract the part about writing the other revision in a function
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 47467
diff changeset
409 entry_bin = revlog.index.entry_binary(rev)
9b70aa7bcbab censor: extract the part about writing the other revision in a function
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 47467
diff changeset
410 new_index_file.write(entry_bin)
47457
f8330a3fc39f censor: implement censoring for revlogv2
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 47392
diff changeset
411
47468
9b70aa7bcbab censor: extract the part about writing the other revision in a function
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 47467
diff changeset
412 revlog._docket.index_end = new_index_file.tell()
9b70aa7bcbab censor: extract the part about writing the other revision in a function
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 47467
diff changeset
413 revlog._docket.data_end = new_data_file.tell()
9b70aa7bcbab censor: extract the part about writing the other revision in a function
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 47467
diff changeset
414 revlog._docket.sidedata_end = new_sidedata_file.tell()
47467
3ab267f0cbe4 censor: extract the part about writing the censored revision in a function
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 47466
diff changeset
415
3ab267f0cbe4 censor: extract the part about writing the censored revision in a function
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 47466
diff changeset
416
3ab267f0cbe4 censor: extract the part about writing the censored revision in a function
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 47466
diff changeset
417 def _rewrite_censor(
3ab267f0cbe4 censor: extract the part about writing the censored revision in a function
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 47466
diff changeset
418 revlog,
3ab267f0cbe4 censor: extract the part about writing the censored revision in a function
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 47466
diff changeset
419 old_index,
3ab267f0cbe4 censor: extract the part about writing the censored revision in a function
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 47466
diff changeset
420 all_files,
3ab267f0cbe4 censor: extract the part about writing the censored revision in a function
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 47466
diff changeset
421 rev,
3ab267f0cbe4 censor: extract the part about writing the censored revision in a function
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 47466
diff changeset
422 tombstone,
3ab267f0cbe4 censor: extract the part about writing the censored revision in a function
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 47466
diff changeset
423 ):
3ab267f0cbe4 censor: extract the part about writing the censored revision in a function
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 47466
diff changeset
424 """rewrite and append a censored revision"""
3ab267f0cbe4 censor: extract the part about writing the censored revision in a function
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 47466
diff changeset
425 (
3ab267f0cbe4 censor: extract the part about writing the censored revision in a function
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 47466
diff changeset
426 old_data_file,
3ab267f0cbe4 censor: extract the part about writing the censored revision in a function
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 47466
diff changeset
427 old_sidedata_file,
3ab267f0cbe4 censor: extract the part about writing the censored revision in a function
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 47466
diff changeset
428 new_index_file,
3ab267f0cbe4 censor: extract the part about writing the censored revision in a function
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 47466
diff changeset
429 new_data_file,
3ab267f0cbe4 censor: extract the part about writing the censored revision in a function
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 47466
diff changeset
430 new_sidedata_file,
3ab267f0cbe4 censor: extract the part about writing the censored revision in a function
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 47466
diff changeset
431 ) = all_files
3ab267f0cbe4 censor: extract the part about writing the censored revision in a function
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 47466
diff changeset
432 entry = old_index[rev]
3ab267f0cbe4 censor: extract the part about writing the censored revision in a function
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 47466
diff changeset
433
3ab267f0cbe4 censor: extract the part about writing the censored revision in a function
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 47466
diff changeset
434 # XXX consider trying the default compression too
3ab267f0cbe4 censor: extract the part about writing the censored revision in a function
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 47466
diff changeset
435 new_data_size = len(tombstone)
3ab267f0cbe4 censor: extract the part about writing the censored revision in a function
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 47466
diff changeset
436 new_data_offset = new_data_file.tell()
3ab267f0cbe4 censor: extract the part about writing the censored revision in a function
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 47466
diff changeset
437 new_data_file.write(tombstone)
3ab267f0cbe4 censor: extract the part about writing the censored revision in a function
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 47466
diff changeset
438
3ab267f0cbe4 censor: extract the part about writing the censored revision in a function
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 47466
diff changeset
439 # we are not adding any sidedata as they might leak info about the censored version
3ab267f0cbe4 censor: extract the part about writing the censored revision in a function
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 47466
diff changeset
440
3ab267f0cbe4 censor: extract the part about writing the censored revision in a function
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 47466
diff changeset
441 link_rev = entry[ENTRY_LINK_REV]
3ab267f0cbe4 censor: extract the part about writing the censored revision in a function
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 47466
diff changeset
442
3ab267f0cbe4 censor: extract the part about writing the censored revision in a function
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 47466
diff changeset
443 p1 = entry[ENTRY_PARENT_1]
3ab267f0cbe4 censor: extract the part about writing the censored revision in a function
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 47466
diff changeset
444 p2 = entry[ENTRY_PARENT_2]
3ab267f0cbe4 censor: extract the part about writing the censored revision in a function
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 47466
diff changeset
445
3ab267f0cbe4 censor: extract the part about writing the censored revision in a function
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 47466
diff changeset
446 new_entry = revlogutils.entry(
3ab267f0cbe4 censor: extract the part about writing the censored revision in a function
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 47466
diff changeset
447 flags=constants.REVIDX_ISCENSORED,
3ab267f0cbe4 censor: extract the part about writing the censored revision in a function
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 47466
diff changeset
448 data_offset=new_data_offset,
3ab267f0cbe4 censor: extract the part about writing the censored revision in a function
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 47466
diff changeset
449 data_compressed_length=new_data_size,
3ab267f0cbe4 censor: extract the part about writing the censored revision in a function
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 47466
diff changeset
450 data_uncompressed_length=new_data_size,
3ab267f0cbe4 censor: extract the part about writing the censored revision in a function
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 47466
diff changeset
451 data_delta_base=rev,
3ab267f0cbe4 censor: extract the part about writing the censored revision in a function
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 47466
diff changeset
452 link_rev=link_rev,
3ab267f0cbe4 censor: extract the part about writing the censored revision in a function
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 47466
diff changeset
453 parent_rev_1=p1,
3ab267f0cbe4 censor: extract the part about writing the censored revision in a function
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 47466
diff changeset
454 parent_rev_2=p2,
3ab267f0cbe4 censor: extract the part about writing the censored revision in a function
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 47466
diff changeset
455 node_id=entry[ENTRY_NODE_ID],
3ab267f0cbe4 censor: extract the part about writing the censored revision in a function
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 47466
diff changeset
456 sidedata_offset=0,
3ab267f0cbe4 censor: extract the part about writing the censored revision in a function
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 47466
diff changeset
457 sidedata_compressed_length=0,
3ab267f0cbe4 censor: extract the part about writing the censored revision in a function
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 47466
diff changeset
458 data_compression_mode=COMP_MODE_PLAIN,
3ab267f0cbe4 censor: extract the part about writing the censored revision in a function
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 47466
diff changeset
459 sidedata_compression_mode=COMP_MODE_PLAIN,
3ab267f0cbe4 censor: extract the part about writing the censored revision in a function
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 47466
diff changeset
460 )
3ab267f0cbe4 censor: extract the part about writing the censored revision in a function
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 47466
diff changeset
461 revlog.index.append(new_entry)
3ab267f0cbe4 censor: extract the part about writing the censored revision in a function
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 47466
diff changeset
462 entry_bin = revlog.index.entry_binary(rev)
3ab267f0cbe4 censor: extract the part about writing the censored revision in a function
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 47466
diff changeset
463 new_index_file.write(entry_bin)
3ab267f0cbe4 censor: extract the part about writing the censored revision in a function
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 47466
diff changeset
464 revlog._docket.index_end = new_index_file.tell()
3ab267f0cbe4 censor: extract the part about writing the censored revision in a function
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 47466
diff changeset
465 revlog._docket.data_end = new_data_file.tell()