Mercurial > public > mercurial-scm > hg-stable

annotate mercurial/revlogutils/censor.py @ 47398:8089d0fa8400

Find changesets by keywords (author, files, the commit message), revision number or hash, or revset expression.
censor: drop size limitation on the tombstone This limitation seems to date back from a previous implementation of censors, were the revision were censored" in place. For a long while the implementation actually involved rewriting the revlog before replacing the original data. So it looks like we can safely remove this limitation. The tests suite agrees. Differential Revision: https://phab.mercurial-scm.org/D10790
author Pierre-Yves David <pierre-yves.david@octobus.net>
date Fri, 21 May 2021 20:08:11 +0200
parents 33d626910374
children f8330a3fc39f
Ignore whitespace changes - Everywhere: Within whitespace: At end of lines:
rev   line source
47397
33d626910374 revlog: move censoring code in a dedicated module
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 47395
diff changeset
1 # censor code related to censoring revision
8226
8b2cd04a6e97 put license and copyright info into comment blocks
Martin Geisler <mg@lazybytes.net>
parents: 8225
diff changeset
2 #
47397
33d626910374 revlog: move censoring code in a dedicated module
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 47395
diff changeset
3 # Copyright 2021 Pierre-Yves David <pierre-yves.david@octobus.net>
33d626910374 revlog: move censoring code in a dedicated module
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 47395
diff changeset
4 # Copyright 2015 Google, Inc <martinvonz@google.com>
8226
8b2cd04a6e97 put license and copyright info into comment blocks
Martin Geisler <mg@lazybytes.net>
parents: 8225
diff changeset
5 #
8b2cd04a6e97 put license and copyright info into comment blocks
Martin Geisler <mg@lazybytes.net>
parents: 8225
diff changeset
6 # This software may be used and distributed according to the terms of the
10263
25e572394f5c Update license to GPLv2+
Matt Mackall <mpm@selenic.com>
parents: 10047
diff changeset
7 # GNU General Public License version 2 or any later version.
0
9117c6561b0b Add back links from file revisions to changeset revisions
mpm@selenic.com
parents:
diff changeset
8
47397
33d626910374 revlog: move censoring code in a dedicated module
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 47395
diff changeset
9 from ..node import (
27361
29f50344fa83 revlog: use absolute_import
Gregory Szorc <gregory.szorc@gmail.com>
parents: 27251
diff changeset
10 nullrev,
39783
db088e133e91 revlog: define ellipsis flag processors in core
Gregory Szorc <gregory.szorc@gmail.com>
parents: 39360
diff changeset
11 )
47397
33d626910374 revlog: move censoring code in a dedicated module
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 47395
diff changeset
12 from ..i18n import _
33d626910374 revlog: move censoring code in a dedicated module
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 47395
diff changeset
13 from .. import (
33d626910374 revlog: move censoring code in a dedicated module
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 47395
diff changeset
14 error,
46310
fc2d5c0aed7f persistent-nodemap: add a "warn" option to the slow-path config
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 45957
diff changeset
15 )
47397
33d626910374 revlog: move censoring code in a dedicated module
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 47395
diff changeset
16 from ..utils import (
33d626910374 revlog: move censoring code in a dedicated module
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 47395
diff changeset
17 storageutil,
47392
0d0fb091c49f revlog: simplify "partial read" error message
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 47391
diff changeset
18 )
47397
33d626910374 revlog: move censoring code in a dedicated module
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 47395
diff changeset
19 from . import constants
47257
02a4463565ea revlog: improve documentation of the entry tuple
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 47256
diff changeset
20
02a4463565ea revlog: improve documentation of the entry tuple
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 47256
diff changeset
21
47397
33d626910374 revlog: move censoring code in a dedicated module
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 47395
diff changeset
22 def v1_censor(rl, tr, censornode, tombstone=b''):
33d626910374 revlog: move censoring code in a dedicated module
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 47395
diff changeset
23 """censors a revision in a "version 1" revlog"""
33d626910374 revlog: move censoring code in a dedicated module
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 47395
diff changeset
24 assert rl._format_version == constants.REVLOGV1, rl._format_version
47165
24be247a13b4 revlog: stop usage of `_indexfile` to computing nodemap path
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 47164
diff changeset
25
47397
33d626910374 revlog: move censoring code in a dedicated module
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 47395
diff changeset
26 # avoid cycle
33d626910374 revlog: move censoring code in a dedicated module
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 47395
diff changeset
27 from .. import revlog
47158
b6e1fe7ac24b revlog: split the option initialisation in its own method
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 47157
diff changeset
28
47397
33d626910374 revlog: move censoring code in a dedicated module
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 47395
diff changeset
29 censorrev = rl.rev(censornode)
33d626910374 revlog: move censoring code in a dedicated module
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 47395
diff changeset
30 tombstone = storageutil.packmeta({b'censored': tombstone}, b'')
30154
5e72129d75ed revlog: add instance variable controlling delta chain use
Gregory Szorc <gregory.szorc@gmail.com>
parents: 30014
diff changeset
31
47397
33d626910374 revlog: move censoring code in a dedicated module
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 47395
diff changeset
32 # Rewriting the revlog in place is hard. Our strategy for censoring is
33d626910374 revlog: move censoring code in a dedicated module
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 47395
diff changeset
33 # to create a new revlog, copy all revisions to it, then replace the
33d626910374 revlog: move censoring code in a dedicated module
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 47395
diff changeset
34 # revlogs on transaction close.
33d626910374 revlog: move censoring code in a dedicated module
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 47395
diff changeset
35 #
33d626910374 revlog: move censoring code in a dedicated module
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 47395
diff changeset
36 # This is a bit dangerous. We could easily have a mismatch of state.
33d626910374 revlog: move censoring code in a dedicated module
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 47395
diff changeset
37 newrl = revlog.revlog(
33d626910374 revlog: move censoring code in a dedicated module
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 47395
diff changeset
38 rl.opener,
33d626910374 revlog: move censoring code in a dedicated module
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 47395
diff changeset
39 target=rl.target,
33d626910374 revlog: move censoring code in a dedicated module
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 47395
diff changeset
40 radix=rl.radix,
33d626910374 revlog: move censoring code in a dedicated module
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 47395
diff changeset
41 postfix=b'tmpcensored',
33d626910374 revlog: move censoring code in a dedicated module
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 47395
diff changeset
42 censorable=True,
33d626910374 revlog: move censoring code in a dedicated module
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 47395
diff changeset
43 )
33d626910374 revlog: move censoring code in a dedicated module
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 47395
diff changeset
44 newrl._format_version = rl._format_version
33d626910374 revlog: move censoring code in a dedicated module
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 47395
diff changeset
45 newrl._format_flags = rl._format_flags
33d626910374 revlog: move censoring code in a dedicated module
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 47395
diff changeset
46 newrl._generaldelta = rl._generaldelta
33d626910374 revlog: move censoring code in a dedicated module
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 47395
diff changeset
47 newrl._parse_index = rl._parse_index
23337
3a8a763f4197 revlog: add a method to get missing revs incrementally
Siddharth Agarwal <sid0@fb.com>
parents: 23328
diff changeset
48
47397
33d626910374 revlog: move censoring code in a dedicated module
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 47395
diff changeset
49 for rev in rl.revs():
33d626910374 revlog: move censoring code in a dedicated module
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 47395
diff changeset
50 node = rl.node(rev)
33d626910374 revlog: move censoring code in a dedicated module
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 47395
diff changeset
51 p1, p2 = rl.parents(node)
1457
518da3c3b6ce This implements the nodesbetween method, and it removes the newer method
Eric Hopper <hopper@omnifarious.org>
parents: 1351
diff changeset
52
47397
33d626910374 revlog: move censoring code in a dedicated module
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 47395
diff changeset
53 if rev == censorrev:
33d626910374 revlog: move censoring code in a dedicated module
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 47395
diff changeset
54 newrl.addrawrevision(
33d626910374 revlog: move censoring code in a dedicated module
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 47395
diff changeset
55 tombstone,
33d626910374 revlog: move censoring code in a dedicated module
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 47395
diff changeset
56 tr,
33d626910374 revlog: move censoring code in a dedicated module
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 47395
diff changeset
57 rl.linkrev(censorrev),
33d626910374 revlog: move censoring code in a dedicated module
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 47395
diff changeset
58 p1,
33d626910374 revlog: move censoring code in a dedicated module
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 47395
diff changeset
59 p2,
33d626910374 revlog: move censoring code in a dedicated module
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 47395
diff changeset
60 censornode,
33d626910374 revlog: move censoring code in a dedicated module
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 47395
diff changeset
61 constants.REVIDX_ISCENSORED,
47305
93a0abe098e7 revlog: avoid raising no-arg RevlogError for internal flow control
Martin von Zweigbergk <martinvonz@google.com>
parents: 47296
diff changeset
62 )
16665
e410be860393 revlog: speed up prefix matching against nodes
Bryan O'Sullivan <bryano@fb.com>
parents: 16533
diff changeset
63
47397
33d626910374 revlog: move censoring code in a dedicated module
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 47395
diff changeset
64 if newrl.deltaparent(rev) != nullrev:
33d626910374 revlog: move censoring code in a dedicated module
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 47395
diff changeset
65 m = _(b'censored revision stored as delta; cannot censor')
33d626910374 revlog: move censoring code in a dedicated module
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 47395
diff changeset
66 h = _(
33d626910374 revlog: move censoring code in a dedicated module
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 47395
diff changeset
67 b'censoring of revlogs is not fully implemented;'
33d626910374 revlog: move censoring code in a dedicated module
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 47395
diff changeset
68 b' please report this bug'
33d626910374 revlog: move censoring code in a dedicated module
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 47395
diff changeset
69 )
33d626910374 revlog: move censoring code in a dedicated module
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 47395
diff changeset
70 raise error.Abort(m, hint=h)
33d626910374 revlog: move censoring code in a dedicated module
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 47395
diff changeset
71 continue
42982
0d1272783f24 revlog: introduce a `sidedata` method
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 42885
diff changeset
72
47397
33d626910374 revlog: move censoring code in a dedicated module
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 47395
diff changeset
73 if rl.iscensored(rev):
33d626910374 revlog: move censoring code in a dedicated module
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 47395
diff changeset
74 if rl.deltaparent(rev) != nullrev:
33d626910374 revlog: move censoring code in a dedicated module
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 47395
diff changeset
75 m = _(
33d626910374 revlog: move censoring code in a dedicated module
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 47395
diff changeset
76 b'cannot censor due to censored '
33d626910374 revlog: move censoring code in a dedicated module
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 47395
diff changeset
77 b'revision having delta stored'
33d626910374 revlog: move censoring code in a dedicated module
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 47395
diff changeset
78 )
33d626910374 revlog: move censoring code in a dedicated module
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 47395
diff changeset
79 raise error.Abort(m)
33d626910374 revlog: move censoring code in a dedicated module
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 47395
diff changeset
80 rawtext = rl._chunk(rev)
42885
4a3efe0febb5 revlog: stop using `_processflags` directly
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 42882
diff changeset
81 else:
47397
33d626910374 revlog: move censoring code in a dedicated module
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 47395
diff changeset
82 rawtext = rl.rawdata(rev)
8315
c8493310ad9b revlog: use index to find index size
Matt Mackall <mpm@selenic.com>
parents: 8314
diff changeset
83
47397
33d626910374 revlog: move censoring code in a dedicated module
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 47395
diff changeset
84 newrl.addrawrevision(
33d626910374 revlog: move censoring code in a dedicated module
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 47395
diff changeset
85 rawtext, tr, rl.linkrev(rev), p1, p2, node, rl.flags(rev)
43076
2372284d9457 formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents: 43039
diff changeset
86 )
2372284d9457 formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents: 43039
diff changeset
87
47397
33d626910374 revlog: move censoring code in a dedicated module
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 47395
diff changeset
88 tr.addbackup(rl._indexfile, location=b'store')
33d626910374 revlog: move censoring code in a dedicated module
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 47395
diff changeset
89 if not rl._inline:
33d626910374 revlog: move censoring code in a dedicated module
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 47395
diff changeset
90 tr.addbackup(rl._datafile, location=b'store')
1667
daff3ef0de8d verify: notice extra data in indices
Matt Mackall <mpm@selenic.com>
parents: 1660
diff changeset
91
47397
33d626910374 revlog: move censoring code in a dedicated module
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 47395
diff changeset
92 rl.opener.rename(newrl._indexfile, rl._indexfile)
33d626910374 revlog: move censoring code in a dedicated module
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 47395
diff changeset
93 if not rl._inline:
33d626910374 revlog: move censoring code in a dedicated module
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 47395
diff changeset
94 rl.opener.rename(newrl._datafile, rl._datafile)
39794
a6b3c4c1019f revlog: move censor logic out of censor extension
Gregory Szorc <gregory.szorc@gmail.com>
parents: 39791
diff changeset
95
47397
33d626910374 revlog: move censoring code in a dedicated module
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 47395
diff changeset
96 rl.clearcaches()
33d626910374 revlog: move censoring code in a dedicated module
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 47395
diff changeset
97 rl._loadindex()