Mercurial > public > mercurial-scm > hg-stable

annotate mercurial/revlogutils/rewrite.py @ 47478:5045ba2a3afd

Find changesets by keywords (author, files, the commit message), revision number or hash, or revset expression.
censor: split the core of the logic into its own function We now have a "generic" rewrite function (only able to do censoring for now) and a thin wrapper that implement the `censor` API with it. We are now ready to start incorporating strip specific changes. Differential Revision: https://phab.mercurial-scm.org/D10904
author Pierre-Yves David <pierre-yves.david@octobus.net>
date Tue, 22 Jun 2021 22:52:08 +0200
parents c81a5297f185
children b30a53ffbf9b
Ignore whitespace changes - Everywhere: Within whitespace: At end of lines:
rev   line source
47397
33d626910374 revlog: move censoring code in a dedicated module
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 47395
diff changeset
1 # censor code related to censoring revision
47463
f8330a3fc39f censor: implement censoring for revlogv2
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 47398
diff changeset
2 # coding: utf8
8226
8b2cd04a6e97 put license and copyright info into comment blocks
Martin Geisler <mg@lazybytes.net>
parents: 8225
diff changeset
3 #
47397
33d626910374 revlog: move censoring code in a dedicated module
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 47395
diff changeset
4 # Copyright 2021 Pierre-Yves David <pierre-yves.david@octobus.net>
33d626910374 revlog: move censoring code in a dedicated module
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 47395
diff changeset
5 # Copyright 2015 Google, Inc <martinvonz@google.com>
8226
8b2cd04a6e97 put license and copyright info into comment blocks
Martin Geisler <mg@lazybytes.net>
parents: 8225
diff changeset
6 #
8b2cd04a6e97 put license and copyright info into comment blocks
Martin Geisler <mg@lazybytes.net>
parents: 8225
diff changeset
7 # This software may be used and distributed according to the terms of the
10263
25e572394f5c Update license to GPLv2+
Matt Mackall <mpm@selenic.com>
parents: 10047
diff changeset
8 # GNU General Public License version 2 or any later version.
0
9117c6561b0b Add back links from file revisions to changeset revisions
mpm@selenic.com
parents:
diff changeset
9
47463
f8330a3fc39f censor: implement censoring for revlogv2
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 47398
diff changeset
10 import contextlib
f8330a3fc39f censor: implement censoring for revlogv2
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 47398
diff changeset
11 import os
f8330a3fc39f censor: implement censoring for revlogv2
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 47398
diff changeset
12
47397
33d626910374 revlog: move censoring code in a dedicated module
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 47395
diff changeset
13 from ..node import (
27361
29f50344fa83 revlog: use absolute_import
Gregory Szorc <gregory.szorc@gmail.com>
parents: 27251
diff changeset
14 nullrev,
39783
db088e133e91 revlog: define ellipsis flag processors in core
Gregory Szorc <gregory.szorc@gmail.com>
parents: 39360
diff changeset
15 )
47463
f8330a3fc39f censor: implement censoring for revlogv2
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 47398
diff changeset
16 from .constants import (
f8330a3fc39f censor: implement censoring for revlogv2
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 47398
diff changeset
17 COMP_MODE_PLAIN,
f8330a3fc39f censor: implement censoring for revlogv2
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 47398
diff changeset
18 ENTRY_DATA_COMPRESSED_LENGTH,
f8330a3fc39f censor: implement censoring for revlogv2
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 47398
diff changeset
19 ENTRY_DATA_COMPRESSION_MODE,
f8330a3fc39f censor: implement censoring for revlogv2
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 47398
diff changeset
20 ENTRY_DATA_OFFSET,
f8330a3fc39f censor: implement censoring for revlogv2
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 47398
diff changeset
21 ENTRY_DATA_UNCOMPRESSED_LENGTH,
f8330a3fc39f censor: implement censoring for revlogv2
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 47398
diff changeset
22 ENTRY_DELTA_BASE,
f8330a3fc39f censor: implement censoring for revlogv2
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 47398
diff changeset
23 ENTRY_LINK_REV,
f8330a3fc39f censor: implement censoring for revlogv2
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 47398
diff changeset
24 ENTRY_NODE_ID,
f8330a3fc39f censor: implement censoring for revlogv2
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 47398
diff changeset
25 ENTRY_PARENT_1,
f8330a3fc39f censor: implement censoring for revlogv2
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 47398
diff changeset
26 ENTRY_PARENT_2,
f8330a3fc39f censor: implement censoring for revlogv2
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 47398
diff changeset
27 ENTRY_SIDEDATA_COMPRESSED_LENGTH,
f8330a3fc39f censor: implement censoring for revlogv2
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 47398
diff changeset
28 ENTRY_SIDEDATA_COMPRESSION_MODE,
f8330a3fc39f censor: implement censoring for revlogv2
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 47398
diff changeset
29 ENTRY_SIDEDATA_OFFSET,
f8330a3fc39f censor: implement censoring for revlogv2
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 47398
diff changeset
30 REVLOGV0,
f8330a3fc39f censor: implement censoring for revlogv2
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 47398
diff changeset
31 REVLOGV1,
f8330a3fc39f censor: implement censoring for revlogv2
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 47398
diff changeset
32 )
47397
33d626910374 revlog: move censoring code in a dedicated module
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 47395
diff changeset
33 from ..i18n import _
47463
f8330a3fc39f censor: implement censoring for revlogv2
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 47398
diff changeset
34
47397
33d626910374 revlog: move censoring code in a dedicated module
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 47395
diff changeset
35 from .. import (
33d626910374 revlog: move censoring code in a dedicated module
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 47395
diff changeset
36 error,
47463
f8330a3fc39f censor: implement censoring for revlogv2
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 47398
diff changeset
37 pycompat,
f8330a3fc39f censor: implement censoring for revlogv2
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 47398
diff changeset
38 revlogutils,
f8330a3fc39f censor: implement censoring for revlogv2
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 47398
diff changeset
39 util,
46310
fc2d5c0aed7f persistent-nodemap: add a "warn" option to the slow-path config
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 45957
diff changeset
40 )
47397
33d626910374 revlog: move censoring code in a dedicated module
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 47395
diff changeset
41 from ..utils import (
33d626910374 revlog: move censoring code in a dedicated module
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 47395
diff changeset
42 storageutil,
47392
0d0fb091c49f revlog: simplify "partial read" error message
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 47391
diff changeset
43 )
47463
f8330a3fc39f censor: implement censoring for revlogv2
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 47398
diff changeset
44 from . import (
f8330a3fc39f censor: implement censoring for revlogv2
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 47398
diff changeset
45 constants,
f8330a3fc39f censor: implement censoring for revlogv2
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 47398
diff changeset
46 deltas,
f8330a3fc39f censor: implement censoring for revlogv2
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 47398
diff changeset
47 )
47257
02a4463565ea revlog: improve documentation of the entry tuple
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 47256
diff changeset
48
02a4463565ea revlog: improve documentation of the entry tuple
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 47256
diff changeset
49
47397
33d626910374 revlog: move censoring code in a dedicated module
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 47395
diff changeset
50 def v1_censor(rl, tr, censornode, tombstone=b''):
33d626910374 revlog: move censoring code in a dedicated module
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 47395
diff changeset
51 """censors a revision in a "version 1" revlog"""
33d626910374 revlog: move censoring code in a dedicated module
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 47395
diff changeset
52 assert rl._format_version == constants.REVLOGV1, rl._format_version
47165
24be247a13b4 revlog: stop usage of `_indexfile` to computing nodemap path
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 47164
diff changeset
53
47397
33d626910374 revlog: move censoring code in a dedicated module
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 47395
diff changeset
54 # avoid cycle
33d626910374 revlog: move censoring code in a dedicated module
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 47395
diff changeset
55 from .. import revlog
47158
b6e1fe7ac24b revlog: split the option initialisation in its own method
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 47157
diff changeset
56
47397
33d626910374 revlog: move censoring code in a dedicated module
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 47395
diff changeset
57 censorrev = rl.rev(censornode)
33d626910374 revlog: move censoring code in a dedicated module
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 47395
diff changeset
58 tombstone = storageutil.packmeta({b'censored': tombstone}, b'')
30154
5e72129d75ed revlog: add instance variable controlling delta chain use
Gregory Szorc <gregory.szorc@gmail.com>
parents: 30014
diff changeset
59
47397
33d626910374 revlog: move censoring code in a dedicated module
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 47395
diff changeset
60 # Rewriting the revlog in place is hard. Our strategy for censoring is
33d626910374 revlog: move censoring code in a dedicated module
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 47395
diff changeset
61 # to create a new revlog, copy all revisions to it, then replace the
33d626910374 revlog: move censoring code in a dedicated module
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 47395
diff changeset
62 # revlogs on transaction close.
33d626910374 revlog: move censoring code in a dedicated module
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 47395
diff changeset
63 #
33d626910374 revlog: move censoring code in a dedicated module
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 47395
diff changeset
64 # This is a bit dangerous. We could easily have a mismatch of state.
33d626910374 revlog: move censoring code in a dedicated module
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 47395
diff changeset
65 newrl = revlog.revlog(
33d626910374 revlog: move censoring code in a dedicated module
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 47395
diff changeset
66 rl.opener,
33d626910374 revlog: move censoring code in a dedicated module
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 47395
diff changeset
67 target=rl.target,
33d626910374 revlog: move censoring code in a dedicated module
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 47395
diff changeset
68 radix=rl.radix,
33d626910374 revlog: move censoring code in a dedicated module
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 47395
diff changeset
69 postfix=b'tmpcensored',
33d626910374 revlog: move censoring code in a dedicated module
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 47395
diff changeset
70 censorable=True,
33d626910374 revlog: move censoring code in a dedicated module
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 47395
diff changeset
71 )
33d626910374 revlog: move censoring code in a dedicated module
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 47395
diff changeset
72 newrl._format_version = rl._format_version
33d626910374 revlog: move censoring code in a dedicated module
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 47395
diff changeset
73 newrl._format_flags = rl._format_flags
33d626910374 revlog: move censoring code in a dedicated module
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 47395
diff changeset
74 newrl._generaldelta = rl._generaldelta
33d626910374 revlog: move censoring code in a dedicated module
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 47395
diff changeset
75 newrl._parse_index = rl._parse_index
23337
3a8a763f4197 revlog: add a method to get missing revs incrementally
Siddharth Agarwal <sid0@fb.com>
parents: 23328
diff changeset
76
47397
33d626910374 revlog: move censoring code in a dedicated module
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 47395
diff changeset
77 for rev in rl.revs():
33d626910374 revlog: move censoring code in a dedicated module
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 47395
diff changeset
78 node = rl.node(rev)
33d626910374 revlog: move censoring code in a dedicated module
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 47395
diff changeset
79 p1, p2 = rl.parents(node)
1457
518da3c3b6ce This implements the nodesbetween method, and it removes the newer method
Eric Hopper <hopper@omnifarious.org>
parents: 1351
diff changeset
80
47397
33d626910374 revlog: move censoring code in a dedicated module
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 47395
diff changeset
81 if rev == censorrev:
33d626910374 revlog: move censoring code in a dedicated module
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 47395
diff changeset
82 newrl.addrawrevision(
33d626910374 revlog: move censoring code in a dedicated module
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 47395
diff changeset
83 tombstone,
33d626910374 revlog: move censoring code in a dedicated module
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 47395
diff changeset
84 tr,
33d626910374 revlog: move censoring code in a dedicated module
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 47395
diff changeset
85 rl.linkrev(censorrev),
33d626910374 revlog: move censoring code in a dedicated module
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 47395
diff changeset
86 p1,
33d626910374 revlog: move censoring code in a dedicated module
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 47395
diff changeset
87 p2,
33d626910374 revlog: move censoring code in a dedicated module
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 47395
diff changeset
88 censornode,
33d626910374 revlog: move censoring code in a dedicated module
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 47395
diff changeset
89 constants.REVIDX_ISCENSORED,
47305
93a0abe098e7 revlog: avoid raising no-arg RevlogError for internal flow control
Martin von Zweigbergk <martinvonz@google.com>
parents: 47296
diff changeset
90 )
16665
e410be860393 revlog: speed up prefix matching against nodes
Bryan O'Sullivan <bryano@fb.com>
parents: 16533
diff changeset
91
47397
33d626910374 revlog: move censoring code in a dedicated module
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 47395
diff changeset
92 if newrl.deltaparent(rev) != nullrev:
33d626910374 revlog: move censoring code in a dedicated module
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 47395
diff changeset
93 m = _(b'censored revision stored as delta; cannot censor')
33d626910374 revlog: move censoring code in a dedicated module
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 47395
diff changeset
94 h = _(
33d626910374 revlog: move censoring code in a dedicated module
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 47395
diff changeset
95 b'censoring of revlogs is not fully implemented;'
33d626910374 revlog: move censoring code in a dedicated module
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 47395
diff changeset
96 b' please report this bug'
33d626910374 revlog: move censoring code in a dedicated module
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 47395
diff changeset
97 )
33d626910374 revlog: move censoring code in a dedicated module
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 47395
diff changeset
98 raise error.Abort(m, hint=h)
33d626910374 revlog: move censoring code in a dedicated module
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 47395
diff changeset
99 continue
42982
0d1272783f24 revlog: introduce a `sidedata` method
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 42885
diff changeset
100
47397
33d626910374 revlog: move censoring code in a dedicated module
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 47395
diff changeset
101 if rl.iscensored(rev):
33d626910374 revlog: move censoring code in a dedicated module
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 47395
diff changeset
102 if rl.deltaparent(rev) != nullrev:
33d626910374 revlog: move censoring code in a dedicated module
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 47395
diff changeset
103 m = _(
33d626910374 revlog: move censoring code in a dedicated module
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 47395
diff changeset
104 b'cannot censor due to censored '
33d626910374 revlog: move censoring code in a dedicated module
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 47395
diff changeset
105 b'revision having delta stored'
33d626910374 revlog: move censoring code in a dedicated module
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 47395
diff changeset
106 )
33d626910374 revlog: move censoring code in a dedicated module
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 47395
diff changeset
107 raise error.Abort(m)
33d626910374 revlog: move censoring code in a dedicated module
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 47395
diff changeset
108 rawtext = rl._chunk(rev)
42885
4a3efe0febb5 revlog: stop using `_processflags` directly
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 42882
diff changeset
109 else:
47397
33d626910374 revlog: move censoring code in a dedicated module
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 47395
diff changeset
110 rawtext = rl.rawdata(rev)
8315
c8493310ad9b revlog: use index to find index size
Matt Mackall <mpm@selenic.com>
parents: 8314
diff changeset
111
47397
33d626910374 revlog: move censoring code in a dedicated module
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 47395
diff changeset
112 newrl.addrawrevision(
33d626910374 revlog: move censoring code in a dedicated module
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 47395
diff changeset
113 rawtext, tr, rl.linkrev(rev), p1, p2, node, rl.flags(rev)
43076
2372284d9457 formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents: 43039
diff changeset
114 )
2372284d9457 formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents: 43039
diff changeset
115
47397
33d626910374 revlog: move censoring code in a dedicated module
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 47395
diff changeset
116 tr.addbackup(rl._indexfile, location=b'store')
33d626910374 revlog: move censoring code in a dedicated module
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 47395
diff changeset
117 if not rl._inline:
33d626910374 revlog: move censoring code in a dedicated module
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 47395
diff changeset
118 tr.addbackup(rl._datafile, location=b'store')
1667
daff3ef0de8d verify: notice extra data in indices
Matt Mackall <mpm@selenic.com>
parents: 1660
diff changeset
119
47397
33d626910374 revlog: move censoring code in a dedicated module
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 47395
diff changeset
120 rl.opener.rename(newrl._indexfile, rl._indexfile)
33d626910374 revlog: move censoring code in a dedicated module
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 47395
diff changeset
121 if not rl._inline:
33d626910374 revlog: move censoring code in a dedicated module
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 47395
diff changeset
122 rl.opener.rename(newrl._datafile, rl._datafile)
39794
a6b3c4c1019f revlog: move censor logic out of censor extension
Gregory Szorc <gregory.szorc@gmail.com>
parents: 39791
diff changeset
123
47397
33d626910374 revlog: move censoring code in a dedicated module
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 47395
diff changeset
124 rl.clearcaches()
33d626910374 revlog: move censoring code in a dedicated module
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 47395
diff changeset
125 rl._loadindex()
47463
f8330a3fc39f censor: implement censoring for revlogv2
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 47398
diff changeset
126
f8330a3fc39f censor: implement censoring for revlogv2
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 47398
diff changeset
127
47476
aab064416f0c censor: rename `rl` to `revlog` in the main function
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 47475
diff changeset
128 def v2_censor(revlog, tr, censornode, tombstone=b''):
47463
f8330a3fc39f censor: implement censoring for revlogv2
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 47398
diff changeset
129 """censors a revision in a "version 2" revlog"""
47478
5045ba2a3afd censor: split the core of the logic into its own function
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 47477
diff changeset
130 assert revlog._format_version != REVLOGV0, revlog._format_version
5045ba2a3afd censor: split the core of the logic into its own function
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 47477
diff changeset
131 assert revlog._format_version != REVLOGV1, revlog._format_version
5045ba2a3afd censor: split the core of the logic into its own function
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 47477
diff changeset
132
5045ba2a3afd censor: split the core of the logic into its own function
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 47477
diff changeset
133 censor_revs = {revlog.rev(censornode)}
5045ba2a3afd censor: split the core of the logic into its own function
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 47477
diff changeset
134 _rewrite_v2(revlog, tr, censor_revs, tombstone)
5045ba2a3afd censor: split the core of the logic into its own function
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 47477
diff changeset
135
5045ba2a3afd censor: split the core of the logic into its own function
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 47477
diff changeset
136
5045ba2a3afd censor: split the core of the logic into its own function
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 47477
diff changeset
137 def _rewrite_v2(revlog, tr, censor_revs, tombstone=b''):
5045ba2a3afd censor: split the core of the logic into its own function
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 47477
diff changeset
138 """rewrite a revlog to censor some of its content
5045ba2a3afd censor: split the core of the logic into its own function
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 47477
diff changeset
139
5045ba2a3afd censor: split the core of the logic into its own function
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 47477
diff changeset
140 General principle
47463
f8330a3fc39f censor: implement censoring for revlogv2
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 47398
diff changeset
141
47478
5045ba2a3afd censor: split the core of the logic into its own function
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 47477
diff changeset
142 We create new revlog files (index/data/sidedata) to copy the content of
5045ba2a3afd censor: split the core of the logic into its own function
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 47477
diff changeset
143 the existing data without the censored data.
5045ba2a3afd censor: split the core of the logic into its own function
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 47477
diff changeset
144
5045ba2a3afd censor: split the core of the logic into its own function
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 47477
diff changeset
145 We need to recompute new delta for any revision that used the censored
5045ba2a3afd censor: split the core of the logic into its own function
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 47477
diff changeset
146 revision as delta base. As the cumulative size of the new delta may be
5045ba2a3afd censor: split the core of the logic into its own function
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 47477
diff changeset
147 large, we store them in a temporary file until they are stored in their
5045ba2a3afd censor: split the core of the logic into its own function
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 47477
diff changeset
148 final destination.
5045ba2a3afd censor: split the core of the logic into its own function
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 47477
diff changeset
149
5045ba2a3afd censor: split the core of the logic into its own function
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 47477
diff changeset
150 All data before the censored data can be blindly copied. The rest needs
5045ba2a3afd censor: split the core of the logic into its own function
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 47477
diff changeset
151 to be copied as we go and the associated index entry needs adjustement.
5045ba2a3afd censor: split the core of the logic into its own function
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 47477
diff changeset
152 """
47476
aab064416f0c censor: rename `rl` to `revlog` in the main function
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 47475
diff changeset
153 assert revlog._format_version != REVLOGV0, revlog._format_version
aab064416f0c censor: rename `rl` to `revlog` in the main function
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 47475
diff changeset
154 assert revlog._format_version != REVLOGV1, revlog._format_version
47463
f8330a3fc39f censor: implement censoring for revlogv2
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 47398
diff changeset
155
47476
aab064416f0c censor: rename `rl` to `revlog` in the main function
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 47475
diff changeset
156 old_index = revlog.index
aab064416f0c censor: rename `rl` to `revlog` in the main function
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 47475
diff changeset
157 docket = revlog._docket
47463
f8330a3fc39f censor: implement censoring for revlogv2
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 47398
diff changeset
158
f8330a3fc39f censor: implement censoring for revlogv2
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 47398
diff changeset
159 tombstone = storageutil.packmeta({b'censored': tombstone}, b'')
f8330a3fc39f censor: implement censoring for revlogv2
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 47398
diff changeset
160
47477
c81a5297f185 censor: migrate the logic to a set of `censor_revs`
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 47476
diff changeset
161 first_excl_rev = min(censor_revs)
c81a5297f185 censor: migrate the logic to a set of `censor_revs`
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 47476
diff changeset
162
c81a5297f185 censor: migrate the logic to a set of `censor_revs`
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 47476
diff changeset
163 first_excl_entry = revlog.index[first_excl_rev]
c81a5297f185 censor: migrate the logic to a set of `censor_revs`
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 47476
diff changeset
164 index_cutoff = revlog.index.entry_size * first_excl_rev
c81a5297f185 censor: migrate the logic to a set of `censor_revs`
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 47476
diff changeset
165 data_cutoff = first_excl_entry[ENTRY_DATA_OFFSET] >> 16
c81a5297f185 censor: migrate the logic to a set of `censor_revs`
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 47476
diff changeset
166 sidedata_cutoff = revlog.sidedata_cut_off(first_excl_rev)
47463
f8330a3fc39f censor: implement censoring for revlogv2
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 47398
diff changeset
167
f8330a3fc39f censor: implement censoring for revlogv2
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 47398
diff changeset
168 with pycompat.unnamedtempfile(mode=b"w+b") as tmp_storage:
47474
60c48458ee6c censor: extract the part about recomputing delta in a function
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 47473
diff changeset
169 # rev → (new_base, data_start, data_end, compression_mode)
60c48458ee6c censor: extract the part about recomputing delta in a function
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 47473
diff changeset
170 rewritten_entries = _precompute_rewritten_delta(
47476
aab064416f0c censor: rename `rl` to `revlog` in the main function
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 47475
diff changeset
171 revlog,
47474
60c48458ee6c censor: extract the part about recomputing delta in a function
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 47473
diff changeset
172 old_index,
47477
c81a5297f185 censor: migrate the logic to a set of `censor_revs`
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 47476
diff changeset
173 censor_revs,
47474
60c48458ee6c censor: extract the part about recomputing delta in a function
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 47473
diff changeset
174 tmp_storage,
60c48458ee6c censor: extract the part about recomputing delta in a function
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 47473
diff changeset
175 )
47463
f8330a3fc39f censor: implement censoring for revlogv2
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 47398
diff changeset
176
47475
d6afe1478a2a censor: extract the part about creating and opening new files in a function
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 47474
diff changeset
177 all_files = _setup_new_files(
47476
aab064416f0c censor: rename `rl` to `revlog` in the main function
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 47475
diff changeset
178 revlog,
47475
d6afe1478a2a censor: extract the part about creating and opening new files in a function
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 47474
diff changeset
179 index_cutoff,
d6afe1478a2a censor: extract the part about creating and opening new files in a function
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 47474
diff changeset
180 data_cutoff,
d6afe1478a2a censor: extract the part about creating and opening new files in a function
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 47474
diff changeset
181 sidedata_cutoff,
47463
f8330a3fc39f censor: implement censoring for revlogv2
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 47398
diff changeset
182 )
f8330a3fc39f censor: implement censoring for revlogv2
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 47398
diff changeset
183
f8330a3fc39f censor: implement censoring for revlogv2
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 47398
diff changeset
184 # we dont need to open the old index file since its content already
f8330a3fc39f censor: implement censoring for revlogv2
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 47398
diff changeset
185 # exist in a usable form in `old_index`.
47471
f7a94e2d4470 censor: put the tuple of open files in an explicit variable
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 47470
diff changeset
186 with all_files() as open_files:
f7a94e2d4470 censor: put the tuple of open files in an explicit variable
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 47470
diff changeset
187 (
f7a94e2d4470 censor: put the tuple of open files in an explicit variable
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 47470
diff changeset
188 old_data_file,
f7a94e2d4470 censor: put the tuple of open files in an explicit variable
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 47470
diff changeset
189 old_sidedata_file,
f7a94e2d4470 censor: put the tuple of open files in an explicit variable
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 47470
diff changeset
190 new_index_file,
f7a94e2d4470 censor: put the tuple of open files in an explicit variable
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 47470
diff changeset
191 new_data_file,
f7a94e2d4470 censor: put the tuple of open files in an explicit variable
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 47470
diff changeset
192 new_sidedata_file,
f7a94e2d4470 censor: put the tuple of open files in an explicit variable
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 47470
diff changeset
193 ) = open_files
47463
f8330a3fc39f censor: implement censoring for revlogv2
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 47398
diff changeset
194
47472
3ab267f0cbe4 censor: extract the part about writing the censored revision in a function
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 47471
diff changeset
195 # writing the censored revision
47463
f8330a3fc39f censor: implement censoring for revlogv2
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 47398
diff changeset
196
47473
9b70aa7bcbab censor: extract the part about writing the other revision in a function
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 47472
diff changeset
197 # Writing all subsequent revisions
47477
c81a5297f185 censor: migrate the logic to a set of `censor_revs`
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 47476
diff changeset
198 for rev in range(first_excl_rev, len(old_index)):
c81a5297f185 censor: migrate the logic to a set of `censor_revs`
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 47476
diff changeset
199 if rev in censor_revs:
c81a5297f185 censor: migrate the logic to a set of `censor_revs`
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 47476
diff changeset
200 _rewrite_censor(
c81a5297f185 censor: migrate the logic to a set of `censor_revs`
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 47476
diff changeset
201 revlog,
c81a5297f185 censor: migrate the logic to a set of `censor_revs`
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 47476
diff changeset
202 old_index,
c81a5297f185 censor: migrate the logic to a set of `censor_revs`
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 47476
diff changeset
203 open_files,
c81a5297f185 censor: migrate the logic to a set of `censor_revs`
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 47476
diff changeset
204 rev,
c81a5297f185 censor: migrate the logic to a set of `censor_revs`
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 47476
diff changeset
205 tombstone,
c81a5297f185 censor: migrate the logic to a set of `censor_revs`
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 47476
diff changeset
206 )
c81a5297f185 censor: migrate the logic to a set of `censor_revs`
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 47476
diff changeset
207 else:
c81a5297f185 censor: migrate the logic to a set of `censor_revs`
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 47476
diff changeset
208 _rewrite_simple(
c81a5297f185 censor: migrate the logic to a set of `censor_revs`
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 47476
diff changeset
209 revlog,
c81a5297f185 censor: migrate the logic to a set of `censor_revs`
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 47476
diff changeset
210 old_index,
c81a5297f185 censor: migrate the logic to a set of `censor_revs`
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 47476
diff changeset
211 open_files,
c81a5297f185 censor: migrate the logic to a set of `censor_revs`
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 47476
diff changeset
212 rev,
c81a5297f185 censor: migrate the logic to a set of `censor_revs`
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 47476
diff changeset
213 rewritten_entries,
c81a5297f185 censor: migrate the logic to a set of `censor_revs`
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 47476
diff changeset
214 tmp_storage,
c81a5297f185 censor: migrate the logic to a set of `censor_revs`
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 47476
diff changeset
215 )
47473
9b70aa7bcbab censor: extract the part about writing the other revision in a function
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 47472
diff changeset
216 docket.write(transaction=None, stripping=True)
9b70aa7bcbab censor: extract the part about writing the other revision in a function
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 47472
diff changeset
217
47463
f8330a3fc39f censor: implement censoring for revlogv2
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 47398
diff changeset
218
47474
60c48458ee6c censor: extract the part about recomputing delta in a function
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 47473
diff changeset
219 def _precompute_rewritten_delta(
60c48458ee6c censor: extract the part about recomputing delta in a function
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 47473
diff changeset
220 revlog,
60c48458ee6c censor: extract the part about recomputing delta in a function
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 47473
diff changeset
221 old_index,
60c48458ee6c censor: extract the part about recomputing delta in a function
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 47473
diff changeset
222 excluded_revs,
60c48458ee6c censor: extract the part about recomputing delta in a function
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 47473
diff changeset
223 tmp_storage,
60c48458ee6c censor: extract the part about recomputing delta in a function
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 47473
diff changeset
224 ):
60c48458ee6c censor: extract the part about recomputing delta in a function
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 47473
diff changeset
225 """Compute new delta for revisions whose delta is based on revision that
60c48458ee6c censor: extract the part about recomputing delta in a function
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 47473
diff changeset
226 will not survive as is.
60c48458ee6c censor: extract the part about recomputing delta in a function
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 47473
diff changeset
227
60c48458ee6c censor: extract the part about recomputing delta in a function
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 47473
diff changeset
228 Return a mapping: {rev → (new_base, data_start, data_end, compression_mode)}
60c48458ee6c censor: extract the part about recomputing delta in a function
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 47473
diff changeset
229 """
60c48458ee6c censor: extract the part about recomputing delta in a function
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 47473
diff changeset
230 dc = deltas.deltacomputer(revlog)
60c48458ee6c censor: extract the part about recomputing delta in a function
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 47473
diff changeset
231 rewritten_entries = {}
60c48458ee6c censor: extract the part about recomputing delta in a function
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 47473
diff changeset
232 first_excl_rev = min(excluded_revs)
60c48458ee6c censor: extract the part about recomputing delta in a function
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 47473
diff changeset
233 with revlog._segmentfile._open_read() as dfh:
60c48458ee6c censor: extract the part about recomputing delta in a function
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 47473
diff changeset
234 for rev in range(first_excl_rev, len(old_index)):
60c48458ee6c censor: extract the part about recomputing delta in a function
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 47473
diff changeset
235 if rev in excluded_revs:
60c48458ee6c censor: extract the part about recomputing delta in a function
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 47473
diff changeset
236 # this revision will be preserved as is, so we don't need to
60c48458ee6c censor: extract the part about recomputing delta in a function
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 47473
diff changeset
237 # consider recomputing a delta.
60c48458ee6c censor: extract the part about recomputing delta in a function
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 47473
diff changeset
238 continue
60c48458ee6c censor: extract the part about recomputing delta in a function
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 47473
diff changeset
239 entry = old_index[rev]
60c48458ee6c censor: extract the part about recomputing delta in a function
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 47473
diff changeset
240 if entry[ENTRY_DELTA_BASE] not in excluded_revs:
60c48458ee6c censor: extract the part about recomputing delta in a function
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 47473
diff changeset
241 continue
60c48458ee6c censor: extract the part about recomputing delta in a function
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 47473
diff changeset
242 # This is a revision that use the censored revision as the base
60c48458ee6c censor: extract the part about recomputing delta in a function
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 47473
diff changeset
243 # for its delta. We need a need new deltas
60c48458ee6c censor: extract the part about recomputing delta in a function
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 47473
diff changeset
244 if entry[ENTRY_DATA_UNCOMPRESSED_LENGTH] == 0:
60c48458ee6c censor: extract the part about recomputing delta in a function
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 47473
diff changeset
245 # this revision is empty, we can delta against nullrev
60c48458ee6c censor: extract the part about recomputing delta in a function
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 47473
diff changeset
246 rewritten_entries[rev] = (nullrev, 0, 0, COMP_MODE_PLAIN)
60c48458ee6c censor: extract the part about recomputing delta in a function
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 47473
diff changeset
247 else:
60c48458ee6c censor: extract the part about recomputing delta in a function
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 47473
diff changeset
248
60c48458ee6c censor: extract the part about recomputing delta in a function
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 47473
diff changeset
249 text = revlog.rawdata(rev, _df=dfh)
60c48458ee6c censor: extract the part about recomputing delta in a function
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 47473
diff changeset
250 info = revlogutils.revisioninfo(
60c48458ee6c censor: extract the part about recomputing delta in a function
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 47473
diff changeset
251 node=entry[ENTRY_NODE_ID],
60c48458ee6c censor: extract the part about recomputing delta in a function
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 47473
diff changeset
252 p1=revlog.node(entry[ENTRY_PARENT_1]),
60c48458ee6c censor: extract the part about recomputing delta in a function
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 47473
diff changeset
253 p2=revlog.node(entry[ENTRY_PARENT_2]),
60c48458ee6c censor: extract the part about recomputing delta in a function
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 47473
diff changeset
254 btext=[text],
60c48458ee6c censor: extract the part about recomputing delta in a function
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 47473
diff changeset
255 textlen=len(text),
60c48458ee6c censor: extract the part about recomputing delta in a function
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 47473
diff changeset
256 cachedelta=None,
60c48458ee6c censor: extract the part about recomputing delta in a function
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 47473
diff changeset
257 flags=entry[ENTRY_DATA_OFFSET] & 0xFFFF,
60c48458ee6c censor: extract the part about recomputing delta in a function
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 47473
diff changeset
258 )
60c48458ee6c censor: extract the part about recomputing delta in a function
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 47473
diff changeset
259 d = dc.finddeltainfo(
60c48458ee6c censor: extract the part about recomputing delta in a function
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 47473
diff changeset
260 info, dfh, excluded_bases=excluded_revs, target_rev=rev
60c48458ee6c censor: extract the part about recomputing delta in a function
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 47473
diff changeset
261 )
60c48458ee6c censor: extract the part about recomputing delta in a function
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 47473
diff changeset
262 default_comp = revlog._docket.default_compression_header
60c48458ee6c censor: extract the part about recomputing delta in a function
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 47473
diff changeset
263 comp_mode, d = deltas.delta_compression(default_comp, d)
60c48458ee6c censor: extract the part about recomputing delta in a function
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 47473
diff changeset
264 # using `tell` is a bit lazy, but we are not here for speed
60c48458ee6c censor: extract the part about recomputing delta in a function
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 47473
diff changeset
265 start = tmp_storage.tell()
60c48458ee6c censor: extract the part about recomputing delta in a function
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 47473
diff changeset
266 tmp_storage.write(d.data[1])
60c48458ee6c censor: extract the part about recomputing delta in a function
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 47473
diff changeset
267 end = tmp_storage.tell()
60c48458ee6c censor: extract the part about recomputing delta in a function
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 47473
diff changeset
268 rewritten_entries[rev] = (d.base, start, end, comp_mode)
60c48458ee6c censor: extract the part about recomputing delta in a function
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 47473
diff changeset
269 return rewritten_entries
60c48458ee6c censor: extract the part about recomputing delta in a function
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 47473
diff changeset
270
60c48458ee6c censor: extract the part about recomputing delta in a function
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 47473
diff changeset
271
47475
d6afe1478a2a censor: extract the part about creating and opening new files in a function
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 47474
diff changeset
272 def _setup_new_files(
d6afe1478a2a censor: extract the part about creating and opening new files in a function
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 47474
diff changeset
273 revlog,
d6afe1478a2a censor: extract the part about creating and opening new files in a function
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 47474
diff changeset
274 index_cutoff,
d6afe1478a2a censor: extract the part about creating and opening new files in a function
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 47474
diff changeset
275 data_cutoff,
d6afe1478a2a censor: extract the part about creating and opening new files in a function
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 47474
diff changeset
276 sidedata_cutoff,
d6afe1478a2a censor: extract the part about creating and opening new files in a function
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 47474
diff changeset
277 ):
d6afe1478a2a censor: extract the part about creating and opening new files in a function
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 47474
diff changeset
278 """
d6afe1478a2a censor: extract the part about creating and opening new files in a function
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 47474
diff changeset
279
d6afe1478a2a censor: extract the part about creating and opening new files in a function
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 47474
diff changeset
280 return a context manager to open all the relevant files:
d6afe1478a2a censor: extract the part about creating and opening new files in a function
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 47474
diff changeset
281 - old_data_file,
d6afe1478a2a censor: extract the part about creating and opening new files in a function
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 47474
diff changeset
282 - old_sidedata_file,
d6afe1478a2a censor: extract the part about creating and opening new files in a function
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 47474
diff changeset
283 - new_index_file,
d6afe1478a2a censor: extract the part about creating and opening new files in a function
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 47474
diff changeset
284 - new_data_file,
d6afe1478a2a censor: extract the part about creating and opening new files in a function
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 47474
diff changeset
285 - new_sidedata_file,
d6afe1478a2a censor: extract the part about creating and opening new files in a function
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 47474
diff changeset
286
d6afe1478a2a censor: extract the part about creating and opening new files in a function
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 47474
diff changeset
287 The old_index_file is not here because it is accessed through the
d6afe1478a2a censor: extract the part about creating and opening new files in a function
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 47474
diff changeset
288 `old_index` object if the caller function.
d6afe1478a2a censor: extract the part about creating and opening new files in a function
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 47474
diff changeset
289 """
d6afe1478a2a censor: extract the part about creating and opening new files in a function
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 47474
diff changeset
290 docket = revlog._docket
d6afe1478a2a censor: extract the part about creating and opening new files in a function
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 47474
diff changeset
291 old_index_filepath = revlog.opener.join(docket.index_filepath())
d6afe1478a2a censor: extract the part about creating and opening new files in a function
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 47474
diff changeset
292 old_data_filepath = revlog.opener.join(docket.data_filepath())
d6afe1478a2a censor: extract the part about creating and opening new files in a function
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 47474
diff changeset
293 old_sidedata_filepath = revlog.opener.join(docket.sidedata_filepath())
d6afe1478a2a censor: extract the part about creating and opening new files in a function
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 47474
diff changeset
294
d6afe1478a2a censor: extract the part about creating and opening new files in a function
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 47474
diff changeset
295 new_index_filepath = revlog.opener.join(docket.new_index_file())
d6afe1478a2a censor: extract the part about creating and opening new files in a function
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 47474
diff changeset
296 new_data_filepath = revlog.opener.join(docket.new_data_file())
d6afe1478a2a censor: extract the part about creating and opening new files in a function
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 47474
diff changeset
297 new_sidedata_filepath = revlog.opener.join(docket.new_sidedata_file())
d6afe1478a2a censor: extract the part about creating and opening new files in a function
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 47474
diff changeset
298
d6afe1478a2a censor: extract the part about creating and opening new files in a function
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 47474
diff changeset
299 util.copyfile(old_index_filepath, new_index_filepath, nb_bytes=index_cutoff)
d6afe1478a2a censor: extract the part about creating and opening new files in a function
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 47474
diff changeset
300 util.copyfile(old_data_filepath, new_data_filepath, nb_bytes=data_cutoff)
d6afe1478a2a censor: extract the part about creating and opening new files in a function
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 47474
diff changeset
301 util.copyfile(
d6afe1478a2a censor: extract the part about creating and opening new files in a function
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 47474
diff changeset
302 old_sidedata_filepath,
d6afe1478a2a censor: extract the part about creating and opening new files in a function
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 47474
diff changeset
303 new_sidedata_filepath,
d6afe1478a2a censor: extract the part about creating and opening new files in a function
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 47474
diff changeset
304 nb_bytes=sidedata_cutoff,
d6afe1478a2a censor: extract the part about creating and opening new files in a function
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 47474
diff changeset
305 )
d6afe1478a2a censor: extract the part about creating and opening new files in a function
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 47474
diff changeset
306 revlog.opener.register_file(docket.index_filepath())
d6afe1478a2a censor: extract the part about creating and opening new files in a function
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 47474
diff changeset
307 revlog.opener.register_file(docket.data_filepath())
d6afe1478a2a censor: extract the part about creating and opening new files in a function
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 47474
diff changeset
308 revlog.opener.register_file(docket.sidedata_filepath())
d6afe1478a2a censor: extract the part about creating and opening new files in a function
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 47474
diff changeset
309
d6afe1478a2a censor: extract the part about creating and opening new files in a function
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 47474
diff changeset
310 docket.index_end = index_cutoff
d6afe1478a2a censor: extract the part about creating and opening new files in a function
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 47474
diff changeset
311 docket.data_end = data_cutoff
d6afe1478a2a censor: extract the part about creating and opening new files in a function
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 47474
diff changeset
312 docket.sidedata_end = sidedata_cutoff
d6afe1478a2a censor: extract the part about creating and opening new files in a function
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 47474
diff changeset
313
d6afe1478a2a censor: extract the part about creating and opening new files in a function
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 47474
diff changeset
314 # reload the revlog internal information
d6afe1478a2a censor: extract the part about creating and opening new files in a function
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 47474
diff changeset
315 revlog.clearcaches()
d6afe1478a2a censor: extract the part about creating and opening new files in a function
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 47474
diff changeset
316 revlog._loadindex(docket=docket)
d6afe1478a2a censor: extract the part about creating and opening new files in a function
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 47474
diff changeset
317
d6afe1478a2a censor: extract the part about creating and opening new files in a function
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 47474
diff changeset
318 @contextlib.contextmanager
d6afe1478a2a censor: extract the part about creating and opening new files in a function
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 47474
diff changeset
319 def all_files_opener():
d6afe1478a2a censor: extract the part about creating and opening new files in a function
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 47474
diff changeset
320 # hide opening in an helper function to please check-code, black
d6afe1478a2a censor: extract the part about creating and opening new files in a function
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 47474
diff changeset
321 # and various python version at the same time
d6afe1478a2a censor: extract the part about creating and opening new files in a function
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 47474
diff changeset
322 with open(old_data_filepath, 'rb') as old_data_file:
d6afe1478a2a censor: extract the part about creating and opening new files in a function
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 47474
diff changeset
323 with open(old_sidedata_filepath, 'rb') as old_sidedata_file:
d6afe1478a2a censor: extract the part about creating and opening new files in a function
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 47474
diff changeset
324 with open(new_index_filepath, 'r+b') as new_index_file:
d6afe1478a2a censor: extract the part about creating and opening new files in a function
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 47474
diff changeset
325 with open(new_data_filepath, 'r+b') as new_data_file:
d6afe1478a2a censor: extract the part about creating and opening new files in a function
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 47474
diff changeset
326 with open(
d6afe1478a2a censor: extract the part about creating and opening new files in a function
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 47474
diff changeset
327 new_sidedata_filepath, 'r+b'
d6afe1478a2a censor: extract the part about creating and opening new files in a function
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 47474
diff changeset
328 ) as new_sidedata_file:
d6afe1478a2a censor: extract the part about creating and opening new files in a function
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 47474
diff changeset
329 new_index_file.seek(0, os.SEEK_END)
d6afe1478a2a censor: extract the part about creating and opening new files in a function
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 47474
diff changeset
330 assert new_index_file.tell() == index_cutoff
d6afe1478a2a censor: extract the part about creating and opening new files in a function
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 47474
diff changeset
331 new_data_file.seek(0, os.SEEK_END)
d6afe1478a2a censor: extract the part about creating and opening new files in a function
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 47474
diff changeset
332 assert new_data_file.tell() == data_cutoff
d6afe1478a2a censor: extract the part about creating and opening new files in a function
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 47474
diff changeset
333 new_sidedata_file.seek(0, os.SEEK_END)
d6afe1478a2a censor: extract the part about creating and opening new files in a function
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 47474
diff changeset
334 assert new_sidedata_file.tell() == sidedata_cutoff
d6afe1478a2a censor: extract the part about creating and opening new files in a function
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 47474
diff changeset
335 yield (
d6afe1478a2a censor: extract the part about creating and opening new files in a function
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 47474
diff changeset
336 old_data_file,
d6afe1478a2a censor: extract the part about creating and opening new files in a function
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 47474
diff changeset
337 old_sidedata_file,
d6afe1478a2a censor: extract the part about creating and opening new files in a function
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 47474
diff changeset
338 new_index_file,
d6afe1478a2a censor: extract the part about creating and opening new files in a function
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 47474
diff changeset
339 new_data_file,
d6afe1478a2a censor: extract the part about creating and opening new files in a function
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 47474
diff changeset
340 new_sidedata_file,
d6afe1478a2a censor: extract the part about creating and opening new files in a function
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 47474
diff changeset
341 )
d6afe1478a2a censor: extract the part about creating and opening new files in a function
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 47474
diff changeset
342
d6afe1478a2a censor: extract the part about creating and opening new files in a function
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 47474
diff changeset
343 return all_files_opener
d6afe1478a2a censor: extract the part about creating and opening new files in a function
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 47474
diff changeset
344
d6afe1478a2a censor: extract the part about creating and opening new files in a function
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 47474
diff changeset
345
47473
9b70aa7bcbab censor: extract the part about writing the other revision in a function
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 47472
diff changeset
346 def _rewrite_simple(
9b70aa7bcbab censor: extract the part about writing the other revision in a function
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 47472
diff changeset
347 revlog,
9b70aa7bcbab censor: extract the part about writing the other revision in a function
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 47472
diff changeset
348 old_index,
9b70aa7bcbab censor: extract the part about writing the other revision in a function
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 47472
diff changeset
349 all_files,
9b70aa7bcbab censor: extract the part about writing the other revision in a function
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 47472
diff changeset
350 rev,
9b70aa7bcbab censor: extract the part about writing the other revision in a function
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 47472
diff changeset
351 rewritten_entries,
9b70aa7bcbab censor: extract the part about writing the other revision in a function
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 47472
diff changeset
352 tmp_storage,
9b70aa7bcbab censor: extract the part about writing the other revision in a function
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 47472
diff changeset
353 ):
9b70aa7bcbab censor: extract the part about writing the other revision in a function
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 47472
diff changeset
354 """append a normal revision to the index after the rewritten one(s)"""
9b70aa7bcbab censor: extract the part about writing the other revision in a function
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 47472
diff changeset
355 (
9b70aa7bcbab censor: extract the part about writing the other revision in a function
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 47472
diff changeset
356 old_data_file,
9b70aa7bcbab censor: extract the part about writing the other revision in a function
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 47472
diff changeset
357 old_sidedata_file,
9b70aa7bcbab censor: extract the part about writing the other revision in a function
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 47472
diff changeset
358 new_index_file,
9b70aa7bcbab censor: extract the part about writing the other revision in a function
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 47472
diff changeset
359 new_data_file,
9b70aa7bcbab censor: extract the part about writing the other revision in a function
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 47472
diff changeset
360 new_sidedata_file,
9b70aa7bcbab censor: extract the part about writing the other revision in a function
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 47472
diff changeset
361 ) = all_files
9b70aa7bcbab censor: extract the part about writing the other revision in a function
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 47472
diff changeset
362 entry = old_index[rev]
9b70aa7bcbab censor: extract the part about writing the other revision in a function
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 47472
diff changeset
363 flags = entry[ENTRY_DATA_OFFSET] & 0xFFFF
9b70aa7bcbab censor: extract the part about writing the other revision in a function
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 47472
diff changeset
364 old_data_offset = entry[ENTRY_DATA_OFFSET] >> 16
47463
f8330a3fc39f censor: implement censoring for revlogv2
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 47398
diff changeset
365
47473
9b70aa7bcbab censor: extract the part about writing the other revision in a function
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 47472
diff changeset
366 if rev not in rewritten_entries:
9b70aa7bcbab censor: extract the part about writing the other revision in a function
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 47472
diff changeset
367 old_data_file.seek(old_data_offset)
9b70aa7bcbab censor: extract the part about writing the other revision in a function
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 47472
diff changeset
368 new_data_size = entry[ENTRY_DATA_COMPRESSED_LENGTH]
9b70aa7bcbab censor: extract the part about writing the other revision in a function
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 47472
diff changeset
369 new_data = old_data_file.read(new_data_size)
9b70aa7bcbab censor: extract the part about writing the other revision in a function
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 47472
diff changeset
370 data_delta_base = entry[ENTRY_DELTA_BASE]
9b70aa7bcbab censor: extract the part about writing the other revision in a function
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 47472
diff changeset
371 d_comp_mode = entry[ENTRY_DATA_COMPRESSION_MODE]
9b70aa7bcbab censor: extract the part about writing the other revision in a function
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 47472
diff changeset
372 else:
9b70aa7bcbab censor: extract the part about writing the other revision in a function
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 47472
diff changeset
373 (
9b70aa7bcbab censor: extract the part about writing the other revision in a function
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 47472
diff changeset
374 data_delta_base,
9b70aa7bcbab censor: extract the part about writing the other revision in a function
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 47472
diff changeset
375 start,
9b70aa7bcbab censor: extract the part about writing the other revision in a function
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 47472
diff changeset
376 end,
9b70aa7bcbab censor: extract the part about writing the other revision in a function
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 47472
diff changeset
377 d_comp_mode,
9b70aa7bcbab censor: extract the part about writing the other revision in a function
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 47472
diff changeset
378 ) = rewritten_entries[rev]
9b70aa7bcbab censor: extract the part about writing the other revision in a function
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 47472
diff changeset
379 new_data_size = end - start
9b70aa7bcbab censor: extract the part about writing the other revision in a function
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 47472
diff changeset
380 tmp_storage.seek(start)
9b70aa7bcbab censor: extract the part about writing the other revision in a function
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 47472
diff changeset
381 new_data = tmp_storage.read(new_data_size)
47463
f8330a3fc39f censor: implement censoring for revlogv2
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 47398
diff changeset
382
47473
9b70aa7bcbab censor: extract the part about writing the other revision in a function
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 47472
diff changeset
383 # It might be faster to group continuous read/write operation,
9b70aa7bcbab censor: extract the part about writing the other revision in a function
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 47472
diff changeset
384 # however, this is censor, an operation that is not focussed
9b70aa7bcbab censor: extract the part about writing the other revision in a function
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 47472
diff changeset
385 # around stellar performance. So I have not written this
9b70aa7bcbab censor: extract the part about writing the other revision in a function
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 47472
diff changeset
386 # optimisation yet.
9b70aa7bcbab censor: extract the part about writing the other revision in a function
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 47472
diff changeset
387 new_data_offset = new_data_file.tell()
9b70aa7bcbab censor: extract the part about writing the other revision in a function
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 47472
diff changeset
388 new_data_file.write(new_data)
47463
f8330a3fc39f censor: implement censoring for revlogv2
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 47398
diff changeset
389
47473
9b70aa7bcbab censor: extract the part about writing the other revision in a function
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 47472
diff changeset
390 sidedata_size = entry[ENTRY_SIDEDATA_COMPRESSED_LENGTH]
9b70aa7bcbab censor: extract the part about writing the other revision in a function
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 47472
diff changeset
391 new_sidedata_offset = new_sidedata_file.tell()
9b70aa7bcbab censor: extract the part about writing the other revision in a function
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 47472
diff changeset
392 if 0 < sidedata_size:
9b70aa7bcbab censor: extract the part about writing the other revision in a function
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 47472
diff changeset
393 old_sidedata_offset = entry[ENTRY_SIDEDATA_OFFSET]
9b70aa7bcbab censor: extract the part about writing the other revision in a function
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 47472
diff changeset
394 old_sidedata_file.seek(old_sidedata_offset)
9b70aa7bcbab censor: extract the part about writing the other revision in a function
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 47472
diff changeset
395 new_sidedata = old_sidedata_file.read(sidedata_size)
9b70aa7bcbab censor: extract the part about writing the other revision in a function
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 47472
diff changeset
396 new_sidedata_file.write(new_sidedata)
9b70aa7bcbab censor: extract the part about writing the other revision in a function
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 47472
diff changeset
397
9b70aa7bcbab censor: extract the part about writing the other revision in a function
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 47472
diff changeset
398 data_uncompressed_length = entry[ENTRY_DATA_UNCOMPRESSED_LENGTH]
9b70aa7bcbab censor: extract the part about writing the other revision in a function
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 47472
diff changeset
399 sd_com_mode = entry[ENTRY_SIDEDATA_COMPRESSION_MODE]
9b70aa7bcbab censor: extract the part about writing the other revision in a function
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 47472
diff changeset
400 assert data_delta_base <= rev, (data_delta_base, rev)
47463
f8330a3fc39f censor: implement censoring for revlogv2
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 47398
diff changeset
401
47473
9b70aa7bcbab censor: extract the part about writing the other revision in a function
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 47472
diff changeset
402 new_entry = revlogutils.entry(
9b70aa7bcbab censor: extract the part about writing the other revision in a function
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 47472
diff changeset
403 flags=flags,
9b70aa7bcbab censor: extract the part about writing the other revision in a function
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 47472
diff changeset
404 data_offset=new_data_offset,
9b70aa7bcbab censor: extract the part about writing the other revision in a function
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 47472
diff changeset
405 data_compressed_length=new_data_size,
9b70aa7bcbab censor: extract the part about writing the other revision in a function
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 47472
diff changeset
406 data_uncompressed_length=data_uncompressed_length,
9b70aa7bcbab censor: extract the part about writing the other revision in a function
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 47472
diff changeset
407 data_delta_base=data_delta_base,
9b70aa7bcbab censor: extract the part about writing the other revision in a function
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 47472
diff changeset
408 link_rev=entry[ENTRY_LINK_REV],
9b70aa7bcbab censor: extract the part about writing the other revision in a function
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 47472
diff changeset
409 parent_rev_1=entry[ENTRY_PARENT_1],
9b70aa7bcbab censor: extract the part about writing the other revision in a function
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 47472
diff changeset
410 parent_rev_2=entry[ENTRY_PARENT_2],
9b70aa7bcbab censor: extract the part about writing the other revision in a function
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 47472
diff changeset
411 node_id=entry[ENTRY_NODE_ID],
9b70aa7bcbab censor: extract the part about writing the other revision in a function
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 47472
diff changeset
412 sidedata_offset=new_sidedata_offset,
9b70aa7bcbab censor: extract the part about writing the other revision in a function
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 47472
diff changeset
413 sidedata_compressed_length=sidedata_size,
9b70aa7bcbab censor: extract the part about writing the other revision in a function
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 47472
diff changeset
414 data_compression_mode=d_comp_mode,
9b70aa7bcbab censor: extract the part about writing the other revision in a function
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 47472
diff changeset
415 sidedata_compression_mode=sd_com_mode,
9b70aa7bcbab censor: extract the part about writing the other revision in a function
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 47472
diff changeset
416 )
9b70aa7bcbab censor: extract the part about writing the other revision in a function
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 47472
diff changeset
417 revlog.index.append(new_entry)
9b70aa7bcbab censor: extract the part about writing the other revision in a function
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 47472
diff changeset
418 entry_bin = revlog.index.entry_binary(rev)
9b70aa7bcbab censor: extract the part about writing the other revision in a function
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 47472
diff changeset
419 new_index_file.write(entry_bin)
47463
f8330a3fc39f censor: implement censoring for revlogv2
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 47398
diff changeset
420
47473
9b70aa7bcbab censor: extract the part about writing the other revision in a function
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 47472
diff changeset
421 revlog._docket.index_end = new_index_file.tell()
9b70aa7bcbab censor: extract the part about writing the other revision in a function
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 47472
diff changeset
422 revlog._docket.data_end = new_data_file.tell()
9b70aa7bcbab censor: extract the part about writing the other revision in a function
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 47472
diff changeset
423 revlog._docket.sidedata_end = new_sidedata_file.tell()
47472
3ab267f0cbe4 censor: extract the part about writing the censored revision in a function
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 47471
diff changeset
424
3ab267f0cbe4 censor: extract the part about writing the censored revision in a function
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 47471
diff changeset
425
3ab267f0cbe4 censor: extract the part about writing the censored revision in a function
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 47471
diff changeset
426 def _rewrite_censor(
3ab267f0cbe4 censor: extract the part about writing the censored revision in a function
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 47471
diff changeset
427 revlog,
3ab267f0cbe4 censor: extract the part about writing the censored revision in a function
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 47471
diff changeset
428 old_index,
3ab267f0cbe4 censor: extract the part about writing the censored revision in a function
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 47471
diff changeset
429 all_files,
3ab267f0cbe4 censor: extract the part about writing the censored revision in a function
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 47471
diff changeset
430 rev,
3ab267f0cbe4 censor: extract the part about writing the censored revision in a function
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 47471
diff changeset
431 tombstone,
3ab267f0cbe4 censor: extract the part about writing the censored revision in a function
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 47471
diff changeset
432 ):
3ab267f0cbe4 censor: extract the part about writing the censored revision in a function
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 47471
diff changeset
433 """rewrite and append a censored revision"""
3ab267f0cbe4 censor: extract the part about writing the censored revision in a function
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 47471
diff changeset
434 (
3ab267f0cbe4 censor: extract the part about writing the censored revision in a function
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 47471
diff changeset
435 old_data_file,
3ab267f0cbe4 censor: extract the part about writing the censored revision in a function
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 47471
diff changeset
436 old_sidedata_file,
3ab267f0cbe4 censor: extract the part about writing the censored revision in a function
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 47471
diff changeset
437 new_index_file,
3ab267f0cbe4 censor: extract the part about writing the censored revision in a function
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 47471
diff changeset
438 new_data_file,
3ab267f0cbe4 censor: extract the part about writing the censored revision in a function
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 47471
diff changeset
439 new_sidedata_file,
3ab267f0cbe4 censor: extract the part about writing the censored revision in a function
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 47471
diff changeset
440 ) = all_files
3ab267f0cbe4 censor: extract the part about writing the censored revision in a function
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 47471
diff changeset
441 entry = old_index[rev]
3ab267f0cbe4 censor: extract the part about writing the censored revision in a function
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 47471
diff changeset
442
3ab267f0cbe4 censor: extract the part about writing the censored revision in a function
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 47471
diff changeset
443 # XXX consider trying the default compression too
3ab267f0cbe4 censor: extract the part about writing the censored revision in a function
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 47471
diff changeset
444 new_data_size = len(tombstone)
3ab267f0cbe4 censor: extract the part about writing the censored revision in a function
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 47471
diff changeset
445 new_data_offset = new_data_file.tell()
3ab267f0cbe4 censor: extract the part about writing the censored revision in a function
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 47471
diff changeset
446 new_data_file.write(tombstone)
3ab267f0cbe4 censor: extract the part about writing the censored revision in a function
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 47471
diff changeset
447
3ab267f0cbe4 censor: extract the part about writing the censored revision in a function
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 47471
diff changeset
448 # we are not adding any sidedata as they might leak info about the censored version
3ab267f0cbe4 censor: extract the part about writing the censored revision in a function
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 47471
diff changeset
449
3ab267f0cbe4 censor: extract the part about writing the censored revision in a function
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 47471
diff changeset
450 link_rev = entry[ENTRY_LINK_REV]
3ab267f0cbe4 censor: extract the part about writing the censored revision in a function
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 47471
diff changeset
451
3ab267f0cbe4 censor: extract the part about writing the censored revision in a function
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 47471
diff changeset
452 p1 = entry[ENTRY_PARENT_1]
3ab267f0cbe4 censor: extract the part about writing the censored revision in a function
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 47471
diff changeset
453 p2 = entry[ENTRY_PARENT_2]
3ab267f0cbe4 censor: extract the part about writing the censored revision in a function
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 47471
diff changeset
454
3ab267f0cbe4 censor: extract the part about writing the censored revision in a function
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 47471
diff changeset
455 new_entry = revlogutils.entry(
3ab267f0cbe4 censor: extract the part about writing the censored revision in a function
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 47471
diff changeset
456 flags=constants.REVIDX_ISCENSORED,
3ab267f0cbe4 censor: extract the part about writing the censored revision in a function
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 47471
diff changeset
457 data_offset=new_data_offset,
3ab267f0cbe4 censor: extract the part about writing the censored revision in a function
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 47471
diff changeset
458 data_compressed_length=new_data_size,
3ab267f0cbe4 censor: extract the part about writing the censored revision in a function
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 47471
diff changeset
459 data_uncompressed_length=new_data_size,
3ab267f0cbe4 censor: extract the part about writing the censored revision in a function
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 47471
diff changeset
460 data_delta_base=rev,
3ab267f0cbe4 censor: extract the part about writing the censored revision in a function
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 47471
diff changeset
461 link_rev=link_rev,
3ab267f0cbe4 censor: extract the part about writing the censored revision in a function
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 47471
diff changeset
462 parent_rev_1=p1,
3ab267f0cbe4 censor: extract the part about writing the censored revision in a function
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 47471
diff changeset
463 parent_rev_2=p2,
3ab267f0cbe4 censor: extract the part about writing the censored revision in a function
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 47471
diff changeset
464 node_id=entry[ENTRY_NODE_ID],
3ab267f0cbe4 censor: extract the part about writing the censored revision in a function
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 47471
diff changeset
465 sidedata_offset=0,
3ab267f0cbe4 censor: extract the part about writing the censored revision in a function
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 47471
diff changeset
466 sidedata_compressed_length=0,
3ab267f0cbe4 censor: extract the part about writing the censored revision in a function
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 47471
diff changeset
467 data_compression_mode=COMP_MODE_PLAIN,
3ab267f0cbe4 censor: extract the part about writing the censored revision in a function
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 47471
diff changeset
468 sidedata_compression_mode=COMP_MODE_PLAIN,
3ab267f0cbe4 censor: extract the part about writing the censored revision in a function
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 47471
diff changeset
469 )
3ab267f0cbe4 censor: extract the part about writing the censored revision in a function
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 47471
diff changeset
470 revlog.index.append(new_entry)
3ab267f0cbe4 censor: extract the part about writing the censored revision in a function
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 47471
diff changeset
471 entry_bin = revlog.index.entry_binary(rev)
3ab267f0cbe4 censor: extract the part about writing the censored revision in a function
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 47471
diff changeset
472 new_index_file.write(entry_bin)
3ab267f0cbe4 censor: extract the part about writing the censored revision in a function
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 47471
diff changeset
473 revlog._docket.index_end = new_index_file.tell()
3ab267f0cbe4 censor: extract the part about writing the censored revision in a function
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 47471
diff changeset
474 revlog._docket.data_end = new_data_file.tell()