mercurial-scm/hg-stable: mercurial/revlogutils/rewrite.py annotate

annotate mercurial/revlogutils/rewrite.py @ 47810:b30a53ffbf9b stable

debugcommands: introduce a debug command to repair repos affected by issue6528 This command is quite basic and slow, it will loop over the entirety of the filelogs in the repository and check each revision for corruption, then fixes the affected filelogs. It takes under 25 minutes for Mozilla-Central on my not-top-of-the-line laptop, using the `--to-report` and `--from-report` options will make this pretty tolerable to use, I think. This change also introduces a test for the fix. Differential Revision: https://phab.mercurial-scm.org/D11239

author	Rapha?l Gom?s <rgomes@octobus.net>
date	Tue, 27 Jul 2021 21:45:27 +0200
parents	5045ba2a3afd
children	32e21ac3adb1

rev	line source
47397 33d626910374 revlog: move censoring code in a dedicated module Pierre-Yves David <pierre-yves.david@octobus.net> parents: 47395 diff changeset	1 # censor code related to censoring revision
47463 f8330a3fc39f censor: implement censoring for revlogv2 Pierre-Yves David <pierre-yves.david@octobus.net> parents: 47398 diff changeset	2 # coding: utf8
8226 8b2cd04a6e97 put license and copyright info into comment blocks Martin Geisler <mg@lazybytes.net> parents: 8225 diff changeset	3 #
47397 33d626910374 revlog: move censoring code in a dedicated module Pierre-Yves David <pierre-yves.david@octobus.net> parents: 47395 diff changeset	4 # Copyright 2021 Pierre-Yves David <pierre-yves.david@octobus.net>
33d626910374 revlog: move censoring code in a dedicated module Pierre-Yves David <pierre-yves.david@octobus.net> parents: 47395 diff changeset	5 # Copyright 2015 Google, Inc <martinvonz@google.com>
8226 8b2cd04a6e97 put license and copyright info into comment blocks Martin Geisler <mg@lazybytes.net> parents: 8225 diff changeset	6 #
8b2cd04a6e97 put license and copyright info into comment blocks Martin Geisler <mg@lazybytes.net> parents: 8225 diff changeset	7 # This software may be used and distributed according to the terms of the
10263 25e572394f5c Update license to GPLv2+ Matt Mackall <mpm@selenic.com> parents: 10047 diff changeset	8 # GNU General Public License version 2 or any later version.
0 9117c6561b0b Add back links from file revisions to changeset revisions mpm@selenic.com parents: diff changeset	9
47810 b30a53ffbf9b debugcommands: introduce a debug command to repair repos affected by issue6528 Rapha?l Gom?s <rgomes@octobus.net> parents: 47478 diff changeset	10 import binascii
47463 f8330a3fc39f censor: implement censoring for revlogv2 Pierre-Yves David <pierre-yves.david@octobus.net> parents: 47398 diff changeset	11 import contextlib
f8330a3fc39f censor: implement censoring for revlogv2 Pierre-Yves David <pierre-yves.david@octobus.net> parents: 47398 diff changeset	12 import os
f8330a3fc39f censor: implement censoring for revlogv2 Pierre-Yves David <pierre-yves.david@octobus.net> parents: 47398 diff changeset	13
47397 33d626910374 revlog: move censoring code in a dedicated module Pierre-Yves David <pierre-yves.david@octobus.net> parents: 47395 diff changeset	14 from ..node import (
27361 29f50344fa83 revlog: use absolute_import Gregory Szorc <gregory.szorc@gmail.com> parents: 27251 diff changeset	15 nullrev,
39783 db088e133e91 revlog: define ellipsis flag processors in core Gregory Szorc <gregory.szorc@gmail.com> parents: 39360 diff changeset	16 )
47463 f8330a3fc39f censor: implement censoring for revlogv2 Pierre-Yves David <pierre-yves.david@octobus.net> parents: 47398 diff changeset	17 from .constants import (
f8330a3fc39f censor: implement censoring for revlogv2 Pierre-Yves David <pierre-yves.david@octobus.net> parents: 47398 diff changeset	18 COMP_MODE_PLAIN,
f8330a3fc39f censor: implement censoring for revlogv2 Pierre-Yves David <pierre-yves.david@octobus.net> parents: 47398 diff changeset	19 ENTRY_DATA_COMPRESSED_LENGTH,
f8330a3fc39f censor: implement censoring for revlogv2 Pierre-Yves David <pierre-yves.david@octobus.net> parents: 47398 diff changeset	20 ENTRY_DATA_COMPRESSION_MODE,
f8330a3fc39f censor: implement censoring for revlogv2 Pierre-Yves David <pierre-yves.david@octobus.net> parents: 47398 diff changeset	21 ENTRY_DATA_OFFSET,
f8330a3fc39f censor: implement censoring for revlogv2 Pierre-Yves David <pierre-yves.david@octobus.net> parents: 47398 diff changeset	22 ENTRY_DATA_UNCOMPRESSED_LENGTH,
f8330a3fc39f censor: implement censoring for revlogv2 Pierre-Yves David <pierre-yves.david@octobus.net> parents: 47398 diff changeset	23 ENTRY_DELTA_BASE,
f8330a3fc39f censor: implement censoring for revlogv2 Pierre-Yves David <pierre-yves.david@octobus.net> parents: 47398 diff changeset	24 ENTRY_LINK_REV,
f8330a3fc39f censor: implement censoring for revlogv2 Pierre-Yves David <pierre-yves.david@octobus.net> parents: 47398 diff changeset	25 ENTRY_NODE_ID,
f8330a3fc39f censor: implement censoring for revlogv2 Pierre-Yves David <pierre-yves.david@octobus.net> parents: 47398 diff changeset	26 ENTRY_PARENT_1,
f8330a3fc39f censor: implement censoring for revlogv2 Pierre-Yves David <pierre-yves.david@octobus.net> parents: 47398 diff changeset	27 ENTRY_PARENT_2,
f8330a3fc39f censor: implement censoring for revlogv2 Pierre-Yves David <pierre-yves.david@octobus.net> parents: 47398 diff changeset	28 ENTRY_SIDEDATA_COMPRESSED_LENGTH,
f8330a3fc39f censor: implement censoring for revlogv2 Pierre-Yves David <pierre-yves.david@octobus.net> parents: 47398 diff changeset	29 ENTRY_SIDEDATA_COMPRESSION_MODE,
f8330a3fc39f censor: implement censoring for revlogv2 Pierre-Yves David <pierre-yves.david@octobus.net> parents: 47398 diff changeset	30 ENTRY_SIDEDATA_OFFSET,
f8330a3fc39f censor: implement censoring for revlogv2 Pierre-Yves David <pierre-yves.david@octobus.net> parents: 47398 diff changeset	31 REVLOGV0,
f8330a3fc39f censor: implement censoring for revlogv2 Pierre-Yves David <pierre-yves.david@octobus.net> parents: 47398 diff changeset	32 REVLOGV1,
f8330a3fc39f censor: implement censoring for revlogv2 Pierre-Yves David <pierre-yves.david@octobus.net> parents: 47398 diff changeset	33 )
47397 33d626910374 revlog: move censoring code in a dedicated module Pierre-Yves David <pierre-yves.david@octobus.net> parents: 47395 diff changeset	34 from ..i18n import _
47463 f8330a3fc39f censor: implement censoring for revlogv2 Pierre-Yves David <pierre-yves.david@octobus.net> parents: 47398 diff changeset	35
47397 33d626910374 revlog: move censoring code in a dedicated module Pierre-Yves David <pierre-yves.david@octobus.net> parents: 47395 diff changeset	36 from .. import (
33d626910374 revlog: move censoring code in a dedicated module Pierre-Yves David <pierre-yves.david@octobus.net> parents: 47395 diff changeset	37 error,
47463 f8330a3fc39f censor: implement censoring for revlogv2 Pierre-Yves David <pierre-yves.david@octobus.net> parents: 47398 diff changeset	38 pycompat,
f8330a3fc39f censor: implement censoring for revlogv2 Pierre-Yves David <pierre-yves.david@octobus.net> parents: 47398 diff changeset	39 revlogutils,
f8330a3fc39f censor: implement censoring for revlogv2 Pierre-Yves David <pierre-yves.david@octobus.net> parents: 47398 diff changeset	40 util,
46310 fc2d5c0aed7f persistent-nodemap: add a "warn" option to the slow-path config Pierre-Yves David <pierre-yves.david@octobus.net> parents: 45957 diff changeset	41 )
47397 33d626910374 revlog: move censoring code in a dedicated module Pierre-Yves David <pierre-yves.david@octobus.net> parents: 47395 diff changeset	42 from ..utils import (
33d626910374 revlog: move censoring code in a dedicated module Pierre-Yves David <pierre-yves.david@octobus.net> parents: 47395 diff changeset	43 storageutil,
47392 0d0fb091c49f revlog: simplify "partial read" error message Pierre-Yves David <pierre-yves.david@octobus.net> parents: 47391 diff changeset	44 )
47463 f8330a3fc39f censor: implement censoring for revlogv2 Pierre-Yves David <pierre-yves.david@octobus.net> parents: 47398 diff changeset	45 from . import (
f8330a3fc39f censor: implement censoring for revlogv2 Pierre-Yves David <pierre-yves.david@octobus.net> parents: 47398 diff changeset	46 constants,
f8330a3fc39f censor: implement censoring for revlogv2 Pierre-Yves David <pierre-yves.david@octobus.net> parents: 47398 diff changeset	47 deltas,
f8330a3fc39f censor: implement censoring for revlogv2 Pierre-Yves David <pierre-yves.david@octobus.net> parents: 47398 diff changeset	48 )
47257 02a4463565ea revlog: improve documentation of the entry tuple Pierre-Yves David <pierre-yves.david@octobus.net> parents: 47256 diff changeset	49
02a4463565ea revlog: improve documentation of the entry tuple Pierre-Yves David <pierre-yves.david@octobus.net> parents: 47256 diff changeset	50
47397 33d626910374 revlog: move censoring code in a dedicated module Pierre-Yves David <pierre-yves.david@octobus.net> parents: 47395 diff changeset	51 def v1_censor(rl, tr, censornode, tombstone=b''):
33d626910374 revlog: move censoring code in a dedicated module Pierre-Yves David <pierre-yves.david@octobus.net> parents: 47395 diff changeset	52 """censors a revision in a "version 1" revlog"""
33d626910374 revlog: move censoring code in a dedicated module Pierre-Yves David <pierre-yves.david@octobus.net> parents: 47395 diff changeset	53 assert rl._format_version == constants.REVLOGV1, rl._format_version
47165 24be247a13b4 revlog: stop usage of `_indexfile` to computing nodemap path Pierre-Yves David <pierre-yves.david@octobus.net> parents: 47164 diff changeset	54
47397 33d626910374 revlog: move censoring code in a dedicated module Pierre-Yves David <pierre-yves.david@octobus.net> parents: 47395 diff changeset	55 # avoid cycle
33d626910374 revlog: move censoring code in a dedicated module Pierre-Yves David <pierre-yves.david@octobus.net> parents: 47395 diff changeset	56 from .. import revlog
47158 b6e1fe7ac24b revlog: split the option initialisation in its own method Pierre-Yves David <pierre-yves.david@octobus.net> parents: 47157 diff changeset	57
47397 33d626910374 revlog: move censoring code in a dedicated module Pierre-Yves David <pierre-yves.david@octobus.net> parents: 47395 diff changeset	58 censorrev = rl.rev(censornode)
33d626910374 revlog: move censoring code in a dedicated module Pierre-Yves David <pierre-yves.david@octobus.net> parents: 47395 diff changeset	59 tombstone = storageutil.packmeta({b'censored': tombstone}, b'')
30154 5e72129d75ed revlog: add instance variable controlling delta chain use Gregory Szorc <gregory.szorc@gmail.com> parents: 30014 diff changeset	60
47397 33d626910374 revlog: move censoring code in a dedicated module Pierre-Yves David <pierre-yves.david@octobus.net> parents: 47395 diff changeset	61 # Rewriting the revlog in place is hard. Our strategy for censoring is
33d626910374 revlog: move censoring code in a dedicated module Pierre-Yves David <pierre-yves.david@octobus.net> parents: 47395 diff changeset	62 # to create a new revlog, copy all revisions to it, then replace the
33d626910374 revlog: move censoring code in a dedicated module Pierre-Yves David <pierre-yves.david@octobus.net> parents: 47395 diff changeset	63 # revlogs on transaction close.
33d626910374 revlog: move censoring code in a dedicated module Pierre-Yves David <pierre-yves.david@octobus.net> parents: 47395 diff changeset	64 #
33d626910374 revlog: move censoring code in a dedicated module Pierre-Yves David <pierre-yves.david@octobus.net> parents: 47395 diff changeset	65 # This is a bit dangerous. We could easily have a mismatch of state.
33d626910374 revlog: move censoring code in a dedicated module Pierre-Yves David <pierre-yves.david@octobus.net> parents: 47395 diff changeset	66 newrl = revlog.revlog(
33d626910374 revlog: move censoring code in a dedicated module Pierre-Yves David <pierre-yves.david@octobus.net> parents: 47395 diff changeset	67 rl.opener,
33d626910374 revlog: move censoring code in a dedicated module Pierre-Yves David <pierre-yves.david@octobus.net> parents: 47395 diff changeset	68 target=rl.target,
33d626910374 revlog: move censoring code in a dedicated module Pierre-Yves David <pierre-yves.david@octobus.net> parents: 47395 diff changeset	69 radix=rl.radix,
33d626910374 revlog: move censoring code in a dedicated module Pierre-Yves David <pierre-yves.david@octobus.net> parents: 47395 diff changeset	70 postfix=b'tmpcensored',
33d626910374 revlog: move censoring code in a dedicated module Pierre-Yves David <pierre-yves.david@octobus.net> parents: 47395 diff changeset	71 censorable=True,
33d626910374 revlog: move censoring code in a dedicated module Pierre-Yves David <pierre-yves.david@octobus.net> parents: 47395 diff changeset	72 )
33d626910374 revlog: move censoring code in a dedicated module Pierre-Yves David <pierre-yves.david@octobus.net> parents: 47395 diff changeset	73 newrl._format_version = rl._format_version
33d626910374 revlog: move censoring code in a dedicated module Pierre-Yves David <pierre-yves.david@octobus.net> parents: 47395 diff changeset	74 newrl._format_flags = rl._format_flags
33d626910374 revlog: move censoring code in a dedicated module Pierre-Yves David <pierre-yves.david@octobus.net> parents: 47395 diff changeset	75 newrl._generaldelta = rl._generaldelta
33d626910374 revlog: move censoring code in a dedicated module Pierre-Yves David <pierre-yves.david@octobus.net> parents: 47395 diff changeset	76 newrl._parse_index = rl._parse_index
23337 3a8a763f4197 revlog: add a method to get missing revs incrementally Siddharth Agarwal <sid0@fb.com> parents: 23328 diff changeset	77
47397 33d626910374 revlog: move censoring code in a dedicated module Pierre-Yves David <pierre-yves.david@octobus.net> parents: 47395 diff changeset	78 for rev in rl.revs():
33d626910374 revlog: move censoring code in a dedicated module Pierre-Yves David <pierre-yves.david@octobus.net> parents: 47395 diff changeset	79 node = rl.node(rev)
33d626910374 revlog: move censoring code in a dedicated module Pierre-Yves David <pierre-yves.david@octobus.net> parents: 47395 diff changeset	80 p1, p2 = rl.parents(node)
1457 518da3c3b6ce This implements the nodesbetween method, and it removes the newer method Eric Hopper <hopper@omnifarious.org> parents: 1351 diff changeset	81
47397 33d626910374 revlog: move censoring code in a dedicated module Pierre-Yves David <pierre-yves.david@octobus.net> parents: 47395 diff changeset	82 if rev == censorrev:
33d626910374 revlog: move censoring code in a dedicated module Pierre-Yves David <pierre-yves.david@octobus.net> parents: 47395 diff changeset	83 newrl.addrawrevision(
33d626910374 revlog: move censoring code in a dedicated module Pierre-Yves David <pierre-yves.david@octobus.net> parents: 47395 diff changeset	84 tombstone,
33d626910374 revlog: move censoring code in a dedicated module Pierre-Yves David <pierre-yves.david@octobus.net> parents: 47395 diff changeset	85 tr,
33d626910374 revlog: move censoring code in a dedicated module Pierre-Yves David <pierre-yves.david@octobus.net> parents: 47395 diff changeset	86 rl.linkrev(censorrev),
33d626910374 revlog: move censoring code in a dedicated module Pierre-Yves David <pierre-yves.david@octobus.net> parents: 47395 diff changeset	87 p1,
33d626910374 revlog: move censoring code in a dedicated module Pierre-Yves David <pierre-yves.david@octobus.net> parents: 47395 diff changeset	88 p2,
33d626910374 revlog: move censoring code in a dedicated module Pierre-Yves David <pierre-yves.david@octobus.net> parents: 47395 diff changeset	89 censornode,
33d626910374 revlog: move censoring code in a dedicated module Pierre-Yves David <pierre-yves.david@octobus.net> parents: 47395 diff changeset	90 constants.REVIDX_ISCENSORED,
47305 93a0abe098e7 revlog: avoid raising no-arg RevlogError for internal flow control Martin von Zweigbergk <martinvonz@google.com> parents: 47296 diff changeset	91 )
16665 e410be860393 revlog: speed up prefix matching against nodes Bryan O'Sullivan <bryano@fb.com> parents: 16533 diff changeset	92
47397 33d626910374 revlog: move censoring code in a dedicated module Pierre-Yves David <pierre-yves.david@octobus.net> parents: 47395 diff changeset	93 if newrl.deltaparent(rev) != nullrev:
33d626910374 revlog: move censoring code in a dedicated module Pierre-Yves David <pierre-yves.david@octobus.net> parents: 47395 diff changeset	94 m = _(b'censored revision stored as delta; cannot censor')
33d626910374 revlog: move censoring code in a dedicated module Pierre-Yves David <pierre-yves.david@octobus.net> parents: 47395 diff changeset	95 h = _(
33d626910374 revlog: move censoring code in a dedicated module Pierre-Yves David <pierre-yves.david@octobus.net> parents: 47395 diff changeset	96 b'censoring of revlogs is not fully implemented;'
33d626910374 revlog: move censoring code in a dedicated module Pierre-Yves David <pierre-yves.david@octobus.net> parents: 47395 diff changeset	97 b' please report this bug'
33d626910374 revlog: move censoring code in a dedicated module Pierre-Yves David <pierre-yves.david@octobus.net> parents: 47395 diff changeset	98 )
33d626910374 revlog: move censoring code in a dedicated module Pierre-Yves David <pierre-yves.david@octobus.net> parents: 47395 diff changeset	99 raise error.Abort(m, hint=h)
33d626910374 revlog: move censoring code in a dedicated module Pierre-Yves David <pierre-yves.david@octobus.net> parents: 47395 diff changeset	100 continue
42982 0d1272783f24 revlog: introduce a `sidedata` method Pierre-Yves David <pierre-yves.david@octobus.net> parents: 42885 diff changeset	101
47397 33d626910374 revlog: move censoring code in a dedicated module Pierre-Yves David <pierre-yves.david@octobus.net> parents: 47395 diff changeset	102 if rl.iscensored(rev):
33d626910374 revlog: move censoring code in a dedicated module Pierre-Yves David <pierre-yves.david@octobus.net> parents: 47395 diff changeset	103 if rl.deltaparent(rev) != nullrev:
33d626910374 revlog: move censoring code in a dedicated module Pierre-Yves David <pierre-yves.david@octobus.net> parents: 47395 diff changeset	104 m = _(
33d626910374 revlog: move censoring code in a dedicated module Pierre-Yves David <pierre-yves.david@octobus.net> parents: 47395 diff changeset	105 b'cannot censor due to censored '
33d626910374 revlog: move censoring code in a dedicated module Pierre-Yves David <pierre-yves.david@octobus.net> parents: 47395 diff changeset	106 b'revision having delta stored'
33d626910374 revlog: move censoring code in a dedicated module Pierre-Yves David <pierre-yves.david@octobus.net> parents: 47395 diff changeset	107 )
33d626910374 revlog: move censoring code in a dedicated module Pierre-Yves David <pierre-yves.david@octobus.net> parents: 47395 diff changeset	108 raise error.Abort(m)
33d626910374 revlog: move censoring code in a dedicated module Pierre-Yves David <pierre-yves.david@octobus.net> parents: 47395 diff changeset	109 rawtext = rl._chunk(rev)
42885 4a3efe0febb5 revlog: stop using `_processflags` directly Pierre-Yves David <pierre-yves.david@octobus.net> parents: 42882 diff changeset	110 else:
47397 33d626910374 revlog: move censoring code in a dedicated module Pierre-Yves David <pierre-yves.david@octobus.net> parents: 47395 diff changeset	111 rawtext = rl.rawdata(rev)
8315 c8493310ad9b revlog: use index to find index size Matt Mackall <mpm@selenic.com> parents: 8314 diff changeset	112
47397 33d626910374 revlog: move censoring code in a dedicated module Pierre-Yves David <pierre-yves.david@octobus.net> parents: 47395 diff changeset	113 newrl.addrawrevision(
33d626910374 revlog: move censoring code in a dedicated module Pierre-Yves David <pierre-yves.david@octobus.net> parents: 47395 diff changeset	114 rawtext, tr, rl.linkrev(rev), p1, p2, node, rl.flags(rev)
43076 2372284d9457 formatting: blacken the codebase Augie Fackler <augie@google.com> parents: 43039 diff changeset	115 )
2372284d9457 formatting: blacken the codebase Augie Fackler <augie@google.com> parents: 43039 diff changeset	116
47397 33d626910374 revlog: move censoring code in a dedicated module Pierre-Yves David <pierre-yves.david@octobus.net> parents: 47395 diff changeset	117 tr.addbackup(rl._indexfile, location=b'store')
33d626910374 revlog: move censoring code in a dedicated module Pierre-Yves David <pierre-yves.david@octobus.net> parents: 47395 diff changeset	118 if not rl._inline:
33d626910374 revlog: move censoring code in a dedicated module Pierre-Yves David <pierre-yves.david@octobus.net> parents: 47395 diff changeset	119 tr.addbackup(rl._datafile, location=b'store')
1667 daff3ef0de8d verify: notice extra data in indices Matt Mackall <mpm@selenic.com> parents: 1660 diff changeset	120
47397 33d626910374 revlog: move censoring code in a dedicated module Pierre-Yves David <pierre-yves.david@octobus.net> parents: 47395 diff changeset	121 rl.opener.rename(newrl._indexfile, rl._indexfile)
33d626910374 revlog: move censoring code in a dedicated module Pierre-Yves David <pierre-yves.david@octobus.net> parents: 47395 diff changeset	122 if not rl._inline:
33d626910374 revlog: move censoring code in a dedicated module Pierre-Yves David <pierre-yves.david@octobus.net> parents: 47395 diff changeset	123 rl.opener.rename(newrl._datafile, rl._datafile)
39794 a6b3c4c1019f revlog: move censor logic out of censor extension Gregory Szorc <gregory.szorc@gmail.com> parents: 39791 diff changeset	124
47397 33d626910374 revlog: move censoring code in a dedicated module Pierre-Yves David <pierre-yves.david@octobus.net> parents: 47395 diff changeset	125 rl.clearcaches()
33d626910374 revlog: move censoring code in a dedicated module Pierre-Yves David <pierre-yves.david@octobus.net> parents: 47395 diff changeset	126 rl._loadindex()
47463 f8330a3fc39f censor: implement censoring for revlogv2 Pierre-Yves David <pierre-yves.david@octobus.net> parents: 47398 diff changeset	127
f8330a3fc39f censor: implement censoring for revlogv2 Pierre-Yves David <pierre-yves.david@octobus.net> parents: 47398 diff changeset	128
47476 aab064416f0c censor: rename `rl` to `revlog` in the main function Pierre-Yves David <pierre-yves.david@octobus.net> parents: 47475 diff changeset	129 def v2_censor(revlog, tr, censornode, tombstone=b''):
47463 f8330a3fc39f censor: implement censoring for revlogv2 Pierre-Yves David <pierre-yves.david@octobus.net> parents: 47398 diff changeset	130 """censors a revision in a "version 2" revlog"""
47478 5045ba2a3afd censor: split the core of the logic into its own function Pierre-Yves David <pierre-yves.david@octobus.net> parents: 47477 diff changeset	131 assert revlog._format_version != REVLOGV0, revlog._format_version
5045ba2a3afd censor: split the core of the logic into its own function Pierre-Yves David <pierre-yves.david@octobus.net> parents: 47477 diff changeset	132 assert revlog._format_version != REVLOGV1, revlog._format_version
5045ba2a3afd censor: split the core of the logic into its own function Pierre-Yves David <pierre-yves.david@octobus.net> parents: 47477 diff changeset	133
5045ba2a3afd censor: split the core of the logic into its own function Pierre-Yves David <pierre-yves.david@octobus.net> parents: 47477 diff changeset	134 censor_revs = {revlog.rev(censornode)}
5045ba2a3afd censor: split the core of the logic into its own function Pierre-Yves David <pierre-yves.david@octobus.net> parents: 47477 diff changeset	135 _rewrite_v2(revlog, tr, censor_revs, tombstone)
5045ba2a3afd censor: split the core of the logic into its own function Pierre-Yves David <pierre-yves.david@octobus.net> parents: 47477 diff changeset	136
5045ba2a3afd censor: split the core of the logic into its own function Pierre-Yves David <pierre-yves.david@octobus.net> parents: 47477 diff changeset	137
5045ba2a3afd censor: split the core of the logic into its own function Pierre-Yves David <pierre-yves.david@octobus.net> parents: 47477 diff changeset	138 def _rewrite_v2(revlog, tr, censor_revs, tombstone=b''):
5045ba2a3afd censor: split the core of the logic into its own function Pierre-Yves David <pierre-yves.david@octobus.net> parents: 47477 diff changeset	139 """rewrite a revlog to censor some of its content
5045ba2a3afd censor: split the core of the logic into its own function Pierre-Yves David <pierre-yves.david@octobus.net> parents: 47477 diff changeset	140
5045ba2a3afd censor: split the core of the logic into its own function Pierre-Yves David <pierre-yves.david@octobus.net> parents: 47477 diff changeset	141 General principle
47463 f8330a3fc39f censor: implement censoring for revlogv2 Pierre-Yves David <pierre-yves.david@octobus.net> parents: 47398 diff changeset	142
47478 5045ba2a3afd censor: split the core of the logic into its own function Pierre-Yves David <pierre-yves.david@octobus.net> parents: 47477 diff changeset	143 We create new revlog files (index/data/sidedata) to copy the content of
5045ba2a3afd censor: split the core of the logic into its own function Pierre-Yves David <pierre-yves.david@octobus.net> parents: 47477 diff changeset	144 the existing data without the censored data.
5045ba2a3afd censor: split the core of the logic into its own function Pierre-Yves David <pierre-yves.david@octobus.net> parents: 47477 diff changeset	145
5045ba2a3afd censor: split the core of the logic into its own function Pierre-Yves David <pierre-yves.david@octobus.net> parents: 47477 diff changeset	146 We need to recompute new delta for any revision that used the censored
5045ba2a3afd censor: split the core of the logic into its own function Pierre-Yves David <pierre-yves.david@octobus.net> parents: 47477 diff changeset	147 revision as delta base. As the cumulative size of the new delta may be
5045ba2a3afd censor: split the core of the logic into its own function Pierre-Yves David <pierre-yves.david@octobus.net> parents: 47477 diff changeset	148 large, we store them in a temporary file until they are stored in their
5045ba2a3afd censor: split the core of the logic into its own function Pierre-Yves David <pierre-yves.david@octobus.net> parents: 47477 diff changeset	149 final destination.
5045ba2a3afd censor: split the core of the logic into its own function Pierre-Yves David <pierre-yves.david@octobus.net> parents: 47477 diff changeset	150
5045ba2a3afd censor: split the core of the logic into its own function Pierre-Yves David <pierre-yves.david@octobus.net> parents: 47477 diff changeset	151 All data before the censored data can be blindly copied. The rest needs
5045ba2a3afd censor: split the core of the logic into its own function Pierre-Yves David <pierre-yves.david@octobus.net> parents: 47477 diff changeset	152 to be copied as we go and the associated index entry needs adjustement.
5045ba2a3afd censor: split the core of the logic into its own function Pierre-Yves David <pierre-yves.david@octobus.net> parents: 47477 diff changeset	153 """
47476 aab064416f0c censor: rename `rl` to `revlog` in the main function Pierre-Yves David <pierre-yves.david@octobus.net> parents: 47475 diff changeset	154 assert revlog._format_version != REVLOGV0, revlog._format_version
aab064416f0c censor: rename `rl` to `revlog` in the main function Pierre-Yves David <pierre-yves.david@octobus.net> parents: 47475 diff changeset	155 assert revlog._format_version != REVLOGV1, revlog._format_version
47463 f8330a3fc39f censor: implement censoring for revlogv2 Pierre-Yves David <pierre-yves.david@octobus.net> parents: 47398 diff changeset	156
47476 aab064416f0c censor: rename `rl` to `revlog` in the main function Pierre-Yves David <pierre-yves.david@octobus.net> parents: 47475 diff changeset	157 old_index = revlog.index
aab064416f0c censor: rename `rl` to `revlog` in the main function Pierre-Yves David <pierre-yves.david@octobus.net> parents: 47475 diff changeset	158 docket = revlog._docket
47463 f8330a3fc39f censor: implement censoring for revlogv2 Pierre-Yves David <pierre-yves.david@octobus.net> parents: 47398 diff changeset	159
f8330a3fc39f censor: implement censoring for revlogv2 Pierre-Yves David <pierre-yves.david@octobus.net> parents: 47398 diff changeset	160 tombstone = storageutil.packmeta({b'censored': tombstone}, b'')
f8330a3fc39f censor: implement censoring for revlogv2 Pierre-Yves David <pierre-yves.david@octobus.net> parents: 47398 diff changeset	161
47477 c81a5297f185 censor: migrate the logic to a set of `censor_revs` Pierre-Yves David <pierre-yves.david@octobus.net> parents: 47476 diff changeset	162 first_excl_rev = min(censor_revs)
c81a5297f185 censor: migrate the logic to a set of `censor_revs` Pierre-Yves David <pierre-yves.david@octobus.net> parents: 47476 diff changeset	163
c81a5297f185 censor: migrate the logic to a set of `censor_revs` Pierre-Yves David <pierre-yves.david@octobus.net> parents: 47476 diff changeset	164 first_excl_entry = revlog.index[first_excl_rev]
c81a5297f185 censor: migrate the logic to a set of `censor_revs` Pierre-Yves David <pierre-yves.david@octobus.net> parents: 47476 diff changeset	165 index_cutoff = revlog.index.entry_size * first_excl_rev
c81a5297f185 censor: migrate the logic to a set of `censor_revs` Pierre-Yves David <pierre-yves.david@octobus.net> parents: 47476 diff changeset	166 data_cutoff = first_excl_entry[ENTRY_DATA_OFFSET] >> 16
c81a5297f185 censor: migrate the logic to a set of `censor_revs` Pierre-Yves David <pierre-yves.david@octobus.net> parents: 47476 diff changeset	167 sidedata_cutoff = revlog.sidedata_cut_off(first_excl_rev)
47463 f8330a3fc39f censor: implement censoring for revlogv2 Pierre-Yves David <pierre-yves.david@octobus.net> parents: 47398 diff changeset	168
f8330a3fc39f censor: implement censoring for revlogv2 Pierre-Yves David <pierre-yves.david@octobus.net> parents: 47398 diff changeset	169 with pycompat.unnamedtempfile(mode=b"w+b") as tmp_storage:
47474 60c48458ee6c censor: extract the part about recomputing delta in a function Pierre-Yves David <pierre-yves.david@octobus.net> parents: 47473 diff changeset	170 # rev → (new_base, data_start, data_end, compression_mode)
60c48458ee6c censor: extract the part about recomputing delta in a function Pierre-Yves David <pierre-yves.david@octobus.net> parents: 47473 diff changeset	171 rewritten_entries = _precompute_rewritten_delta(
47476 aab064416f0c censor: rename `rl` to `revlog` in the main function Pierre-Yves David <pierre-yves.david@octobus.net> parents: 47475 diff changeset	172 revlog,
47474 60c48458ee6c censor: extract the part about recomputing delta in a function Pierre-Yves David <pierre-yves.david@octobus.net> parents: 47473 diff changeset	173 old_index,
47477 c81a5297f185 censor: migrate the logic to a set of `censor_revs` Pierre-Yves David <pierre-yves.david@octobus.net> parents: 47476 diff changeset	174 censor_revs,
47474 60c48458ee6c censor: extract the part about recomputing delta in a function Pierre-Yves David <pierre-yves.david@octobus.net> parents: 47473 diff changeset	175 tmp_storage,
60c48458ee6c censor: extract the part about recomputing delta in a function Pierre-Yves David <pierre-yves.david@octobus.net> parents: 47473 diff changeset	176 )
47463 f8330a3fc39f censor: implement censoring for revlogv2 Pierre-Yves David <pierre-yves.david@octobus.net> parents: 47398 diff changeset	177
47475 d6afe1478a2a censor: extract the part about creating and opening new files in a function Pierre-Yves David <pierre-yves.david@octobus.net> parents: 47474 diff changeset	178 all_files = _setup_new_files(
47476 aab064416f0c censor: rename `rl` to `revlog` in the main function Pierre-Yves David <pierre-yves.david@octobus.net> parents: 47475 diff changeset	179 revlog,
47475 d6afe1478a2a censor: extract the part about creating and opening new files in a function Pierre-Yves David <pierre-yves.david@octobus.net> parents: 47474 diff changeset	180 index_cutoff,
d6afe1478a2a censor: extract the part about creating and opening new files in a function Pierre-Yves David <pierre-yves.david@octobus.net> parents: 47474 diff changeset	181 data_cutoff,
d6afe1478a2a censor: extract the part about creating and opening new files in a function Pierre-Yves David <pierre-yves.david@octobus.net> parents: 47474 diff changeset	182 sidedata_cutoff,
47463 f8330a3fc39f censor: implement censoring for revlogv2 Pierre-Yves David <pierre-yves.david@octobus.net> parents: 47398 diff changeset	183 )
f8330a3fc39f censor: implement censoring for revlogv2 Pierre-Yves David <pierre-yves.david@octobus.net> parents: 47398 diff changeset	184
f8330a3fc39f censor: implement censoring for revlogv2 Pierre-Yves David <pierre-yves.david@octobus.net> parents: 47398 diff changeset	185 # we dont need to open the old index file since its content already
f8330a3fc39f censor: implement censoring for revlogv2 Pierre-Yves David <pierre-yves.david@octobus.net> parents: 47398 diff changeset	186 # exist in a usable form in `old_index`.
47471 f7a94e2d4470 censor: put the tuple of open files in an explicit variable Pierre-Yves David <pierre-yves.david@octobus.net> parents: 47470 diff changeset	187 with all_files() as open_files:
f7a94e2d4470 censor: put the tuple of open files in an explicit variable Pierre-Yves David <pierre-yves.david@octobus.net> parents: 47470 diff changeset	188 (
f7a94e2d4470 censor: put the tuple of open files in an explicit variable Pierre-Yves David <pierre-yves.david@octobus.net> parents: 47470 diff changeset	189 old_data_file,
f7a94e2d4470 censor: put the tuple of open files in an explicit variable Pierre-Yves David <pierre-yves.david@octobus.net> parents: 47470 diff changeset	190 old_sidedata_file,
f7a94e2d4470 censor: put the tuple of open files in an explicit variable Pierre-Yves David <pierre-yves.david@octobus.net> parents: 47470 diff changeset	191 new_index_file,
f7a94e2d4470 censor: put the tuple of open files in an explicit variable Pierre-Yves David <pierre-yves.david@octobus.net> parents: 47470 diff changeset	192 new_data_file,
f7a94e2d4470 censor: put the tuple of open files in an explicit variable Pierre-Yves David <pierre-yves.david@octobus.net> parents: 47470 diff changeset	193 new_sidedata_file,
f7a94e2d4470 censor: put the tuple of open files in an explicit variable Pierre-Yves David <pierre-yves.david@octobus.net> parents: 47470 diff changeset	194 ) = open_files
47463 f8330a3fc39f censor: implement censoring for revlogv2 Pierre-Yves David <pierre-yves.david@octobus.net> parents: 47398 diff changeset	195
47472 3ab267f0cbe4 censor: extract the part about writing the censored revision in a function Pierre-Yves David <pierre-yves.david@octobus.net> parents: 47471 diff changeset	196 # writing the censored revision
47463 f8330a3fc39f censor: implement censoring for revlogv2 Pierre-Yves David <pierre-yves.david@octobus.net> parents: 47398 diff changeset	197
47473 9b70aa7bcbab censor: extract the part about writing the other revision in a function Pierre-Yves David <pierre-yves.david@octobus.net> parents: 47472 diff changeset	198 # Writing all subsequent revisions
47477 c81a5297f185 censor: migrate the logic to a set of `censor_revs` Pierre-Yves David <pierre-yves.david@octobus.net> parents: 47476 diff changeset	199 for rev in range(first_excl_rev, len(old_index)):
c81a5297f185 censor: migrate the logic to a set of `censor_revs` Pierre-Yves David <pierre-yves.david@octobus.net> parents: 47476 diff changeset	200 if rev in censor_revs:
c81a5297f185 censor: migrate the logic to a set of `censor_revs` Pierre-Yves David <pierre-yves.david@octobus.net> parents: 47476 diff changeset	201 _rewrite_censor(
c81a5297f185 censor: migrate the logic to a set of `censor_revs` Pierre-Yves David <pierre-yves.david@octobus.net> parents: 47476 diff changeset	202 revlog,
c81a5297f185 censor: migrate the logic to a set of `censor_revs` Pierre-Yves David <pierre-yves.david@octobus.net> parents: 47476 diff changeset	203 old_index,
c81a5297f185 censor: migrate the logic to a set of `censor_revs` Pierre-Yves David <pierre-yves.david@octobus.net> parents: 47476 diff changeset	204 open_files,
c81a5297f185 censor: migrate the logic to a set of `censor_revs` Pierre-Yves David <pierre-yves.david@octobus.net> parents: 47476 diff changeset	205 rev,
c81a5297f185 censor: migrate the logic to a set of `censor_revs` Pierre-Yves David <pierre-yves.david@octobus.net> parents: 47476 diff changeset	206 tombstone,
c81a5297f185 censor: migrate the logic to a set of `censor_revs` Pierre-Yves David <pierre-yves.david@octobus.net> parents: 47476 diff changeset	207 )
c81a5297f185 censor: migrate the logic to a set of `censor_revs` Pierre-Yves David <pierre-yves.david@octobus.net> parents: 47476 diff changeset	208 else:
c81a5297f185 censor: migrate the logic to a set of `censor_revs` Pierre-Yves David <pierre-yves.david@octobus.net> parents: 47476 diff changeset	209 _rewrite_simple(
c81a5297f185 censor: migrate the logic to a set of `censor_revs` Pierre-Yves David <pierre-yves.david@octobus.net> parents: 47476 diff changeset	210 revlog,
c81a5297f185 censor: migrate the logic to a set of `censor_revs` Pierre-Yves David <pierre-yves.david@octobus.net> parents: 47476 diff changeset	211 old_index,
c81a5297f185 censor: migrate the logic to a set of `censor_revs` Pierre-Yves David <pierre-yves.david@octobus.net> parents: 47476 diff changeset	212 open_files,
c81a5297f185 censor: migrate the logic to a set of `censor_revs` Pierre-Yves David <pierre-yves.david@octobus.net> parents: 47476 diff changeset	213 rev,
c81a5297f185 censor: migrate the logic to a set of `censor_revs` Pierre-Yves David <pierre-yves.david@octobus.net> parents: 47476 diff changeset	214 rewritten_entries,
c81a5297f185 censor: migrate the logic to a set of `censor_revs` Pierre-Yves David <pierre-yves.david@octobus.net> parents: 47476 diff changeset	215 tmp_storage,
c81a5297f185 censor: migrate the logic to a set of `censor_revs` Pierre-Yves David <pierre-yves.david@octobus.net> parents: 47476 diff changeset	216 )
47473 9b70aa7bcbab censor: extract the part about writing the other revision in a function Pierre-Yves David <pierre-yves.david@octobus.net> parents: 47472 diff changeset	217 docket.write(transaction=None, stripping=True)
9b70aa7bcbab censor: extract the part about writing the other revision in a function Pierre-Yves David <pierre-yves.david@octobus.net> parents: 47472 diff changeset	218
47463 f8330a3fc39f censor: implement censoring for revlogv2 Pierre-Yves David <pierre-yves.david@octobus.net> parents: 47398 diff changeset	219
47474 60c48458ee6c censor: extract the part about recomputing delta in a function Pierre-Yves David <pierre-yves.david@octobus.net> parents: 47473 diff changeset	220 def _precompute_rewritten_delta(
60c48458ee6c censor: extract the part about recomputing delta in a function Pierre-Yves David <pierre-yves.david@octobus.net> parents: 47473 diff changeset	221 revlog,
60c48458ee6c censor: extract the part about recomputing delta in a function Pierre-Yves David <pierre-yves.david@octobus.net> parents: 47473 diff changeset	222 old_index,
60c48458ee6c censor: extract the part about recomputing delta in a function Pierre-Yves David <pierre-yves.david@octobus.net> parents: 47473 diff changeset	223 excluded_revs,
60c48458ee6c censor: extract the part about recomputing delta in a function Pierre-Yves David <pierre-yves.david@octobus.net> parents: 47473 diff changeset	224 tmp_storage,
60c48458ee6c censor: extract the part about recomputing delta in a function Pierre-Yves David <pierre-yves.david@octobus.net> parents: 47473 diff changeset	225 ):
60c48458ee6c censor: extract the part about recomputing delta in a function Pierre-Yves David <pierre-yves.david@octobus.net> parents: 47473 diff changeset	226 """Compute new delta for revisions whose delta is based on revision that
60c48458ee6c censor: extract the part about recomputing delta in a function Pierre-Yves David <pierre-yves.david@octobus.net> parents: 47473 diff changeset	227 will not survive as is.
60c48458ee6c censor: extract the part about recomputing delta in a function Pierre-Yves David <pierre-yves.david@octobus.net> parents: 47473 diff changeset	228
60c48458ee6c censor: extract the part about recomputing delta in a function Pierre-Yves David <pierre-yves.david@octobus.net> parents: 47473 diff changeset	229 Return a mapping: {rev → (new_base, data_start, data_end, compression_mode)}
60c48458ee6c censor: extract the part about recomputing delta in a function Pierre-Yves David <pierre-yves.david@octobus.net> parents: 47473 diff changeset	230 """
60c48458ee6c censor: extract the part about recomputing delta in a function Pierre-Yves David <pierre-yves.david@octobus.net> parents: 47473 diff changeset	231 dc = deltas.deltacomputer(revlog)
60c48458ee6c censor: extract the part about recomputing delta in a function Pierre-Yves David <pierre-yves.david@octobus.net> parents: 47473 diff changeset	232 rewritten_entries = {}
60c48458ee6c censor: extract the part about recomputing delta in a function Pierre-Yves David <pierre-yves.david@octobus.net> parents: 47473 diff changeset	233 first_excl_rev = min(excluded_revs)
60c48458ee6c censor: extract the part about recomputing delta in a function Pierre-Yves David <pierre-yves.david@octobus.net> parents: 47473 diff changeset	234 with revlog._segmentfile._open_read() as dfh:
60c48458ee6c censor: extract the part about recomputing delta in a function Pierre-Yves David <pierre-yves.david@octobus.net> parents: 47473 diff changeset	235 for rev in range(first_excl_rev, len(old_index)):
60c48458ee6c censor: extract the part about recomputing delta in a function Pierre-Yves David <pierre-yves.david@octobus.net> parents: 47473 diff changeset	236 if rev in excluded_revs:
60c48458ee6c censor: extract the part about recomputing delta in a function Pierre-Yves David <pierre-yves.david@octobus.net> parents: 47473 diff changeset	237 # this revision will be preserved as is, so we don't need to
60c48458ee6c censor: extract the part about recomputing delta in a function Pierre-Yves David <pierre-yves.david@octobus.net> parents: 47473 diff changeset	238 # consider recomputing a delta.
60c48458ee6c censor: extract the part about recomputing delta in a function Pierre-Yves David <pierre-yves.david@octobus.net> parents: 47473 diff changeset	239 continue
60c48458ee6c censor: extract the part about recomputing delta in a function Pierre-Yves David <pierre-yves.david@octobus.net> parents: 47473 diff changeset	240 entry = old_index[rev]
60c48458ee6c censor: extract the part about recomputing delta in a function Pierre-Yves David <pierre-yves.david@octobus.net> parents: 47473 diff changeset	241 if entry[ENTRY_DELTA_BASE] not in excluded_revs:
60c48458ee6c censor: extract the part about recomputing delta in a function Pierre-Yves David <pierre-yves.david@octobus.net> parents: 47473 diff changeset	242 continue
60c48458ee6c censor: extract the part about recomputing delta in a function Pierre-Yves David <pierre-yves.david@octobus.net> parents: 47473 diff changeset	243 # This is a revision that use the censored revision as the base
60c48458ee6c censor: extract the part about recomputing delta in a function Pierre-Yves David <pierre-yves.david@octobus.net> parents: 47473 diff changeset	244 # for its delta. We need a need new deltas
60c48458ee6c censor: extract the part about recomputing delta in a function Pierre-Yves David <pierre-yves.david@octobus.net> parents: 47473 diff changeset	245 if entry[ENTRY_DATA_UNCOMPRESSED_LENGTH] == 0:
60c48458ee6c censor: extract the part about recomputing delta in a function Pierre-Yves David <pierre-yves.david@octobus.net> parents: 47473 diff changeset	246 # this revision is empty, we can delta against nullrev
60c48458ee6c censor: extract the part about recomputing delta in a function Pierre-Yves David <pierre-yves.david@octobus.net> parents: 47473 diff changeset	247 rewritten_entries[rev] = (nullrev, 0, 0, COMP_MODE_PLAIN)
60c48458ee6c censor: extract the part about recomputing delta in a function Pierre-Yves David <pierre-yves.david@octobus.net> parents: 47473 diff changeset	248 else:
60c48458ee6c censor: extract the part about recomputing delta in a function Pierre-Yves David <pierre-yves.david@octobus.net> parents: 47473 diff changeset	249
60c48458ee6c censor: extract the part about recomputing delta in a function Pierre-Yves David <pierre-yves.david@octobus.net> parents: 47473 diff changeset	250 text = revlog.rawdata(rev, _df=dfh)
60c48458ee6c censor: extract the part about recomputing delta in a function Pierre-Yves David <pierre-yves.david@octobus.net> parents: 47473 diff changeset	251 info = revlogutils.revisioninfo(
60c48458ee6c censor: extract the part about recomputing delta in a function Pierre-Yves David <pierre-yves.david@octobus.net> parents: 47473 diff changeset	252 node=entry[ENTRY_NODE_ID],
60c48458ee6c censor: extract the part about recomputing delta in a function Pierre-Yves David <pierre-yves.david@octobus.net> parents: 47473 diff changeset	253 p1=revlog.node(entry[ENTRY_PARENT_1]),
60c48458ee6c censor: extract the part about recomputing delta in a function Pierre-Yves David <pierre-yves.david@octobus.net> parents: 47473 diff changeset	254 p2=revlog.node(entry[ENTRY_PARENT_2]),
60c48458ee6c censor: extract the part about recomputing delta in a function Pierre-Yves David <pierre-yves.david@octobus.net> parents: 47473 diff changeset	255 btext=[text],
60c48458ee6c censor: extract the part about recomputing delta in a function Pierre-Yves David <pierre-yves.david@octobus.net> parents: 47473 diff changeset	256 textlen=len(text),
60c48458ee6c censor: extract the part about recomputing delta in a function Pierre-Yves David <pierre-yves.david@octobus.net> parents: 47473 diff changeset	257 cachedelta=None,
60c48458ee6c censor: extract the part about recomputing delta in a function Pierre-Yves David <pierre-yves.david@octobus.net> parents: 47473 diff changeset	258 flags=entry[ENTRY_DATA_OFFSET] & 0xFFFF,
60c48458ee6c censor: extract the part about recomputing delta in a function Pierre-Yves David <pierre-yves.david@octobus.net> parents: 47473 diff changeset	259 )
60c48458ee6c censor: extract the part about recomputing delta in a function Pierre-Yves David <pierre-yves.david@octobus.net> parents: 47473 diff changeset	260 d = dc.finddeltainfo(
60c48458ee6c censor: extract the part about recomputing delta in a function Pierre-Yves David <pierre-yves.david@octobus.net> parents: 47473 diff changeset	261 info, dfh, excluded_bases=excluded_revs, target_rev=rev
60c48458ee6c censor: extract the part about recomputing delta in a function Pierre-Yves David <pierre-yves.david@octobus.net> parents: 47473 diff changeset	262 )
60c48458ee6c censor: extract the part about recomputing delta in a function Pierre-Yves David <pierre-yves.david@octobus.net> parents: 47473 diff changeset	263 default_comp = revlog._docket.default_compression_header
60c48458ee6c censor: extract the part about recomputing delta in a function Pierre-Yves David <pierre-yves.david@octobus.net> parents: 47473 diff changeset	264 comp_mode, d = deltas.delta_compression(default_comp, d)
60c48458ee6c censor: extract the part about recomputing delta in a function Pierre-Yves David <pierre-yves.david@octobus.net> parents: 47473 diff changeset	265 # using `tell` is a bit lazy, but we are not here for speed
60c48458ee6c censor: extract the part about recomputing delta in a function Pierre-Yves David <pierre-yves.david@octobus.net> parents: 47473 diff changeset	266 start = tmp_storage.tell()
60c48458ee6c censor: extract the part about recomputing delta in a function Pierre-Yves David <pierre-yves.david@octobus.net> parents: 47473 diff changeset	267 tmp_storage.write(d.data[1])
60c48458ee6c censor: extract the part about recomputing delta in a function Pierre-Yves David <pierre-yves.david@octobus.net> parents: 47473 diff changeset	268 end = tmp_storage.tell()
60c48458ee6c censor: extract the part about recomputing delta in a function Pierre-Yves David <pierre-yves.david@octobus.net> parents: 47473 diff changeset	269 rewritten_entries[rev] = (d.base, start, end, comp_mode)
60c48458ee6c censor: extract the part about recomputing delta in a function Pierre-Yves David <pierre-yves.david@octobus.net> parents: 47473 diff changeset	270 return rewritten_entries
60c48458ee6c censor: extract the part about recomputing delta in a function Pierre-Yves David <pierre-yves.david@octobus.net> parents: 47473 diff changeset	271
60c48458ee6c censor: extract the part about recomputing delta in a function Pierre-Yves David <pierre-yves.david@octobus.net> parents: 47473 diff changeset	272
47475 d6afe1478a2a censor: extract the part about creating and opening new files in a function Pierre-Yves David <pierre-yves.david@octobus.net> parents: 47474 diff changeset	273 def _setup_new_files(
d6afe1478a2a censor: extract the part about creating and opening new files in a function Pierre-Yves David <pierre-yves.david@octobus.net> parents: 47474 diff changeset	274 revlog,
d6afe1478a2a censor: extract the part about creating and opening new files in a function Pierre-Yves David <pierre-yves.david@octobus.net> parents: 47474 diff changeset	275 index_cutoff,
d6afe1478a2a censor: extract the part about creating and opening new files in a function Pierre-Yves David <pierre-yves.david@octobus.net> parents: 47474 diff changeset	276 data_cutoff,
d6afe1478a2a censor: extract the part about creating and opening new files in a function Pierre-Yves David <pierre-yves.david@octobus.net> parents: 47474 diff changeset	277 sidedata_cutoff,
d6afe1478a2a censor: extract the part about creating and opening new files in a function Pierre-Yves David <pierre-yves.david@octobus.net> parents: 47474 diff changeset	278 ):
d6afe1478a2a censor: extract the part about creating and opening new files in a function Pierre-Yves David <pierre-yves.david@octobus.net> parents: 47474 diff changeset	279 """
d6afe1478a2a censor: extract the part about creating and opening new files in a function Pierre-Yves David <pierre-yves.david@octobus.net> parents: 47474 diff changeset	280
d6afe1478a2a censor: extract the part about creating and opening new files in a function Pierre-Yves David <pierre-yves.david@octobus.net> parents: 47474 diff changeset	281 return a context manager to open all the relevant files:
d6afe1478a2a censor: extract the part about creating and opening new files in a function Pierre-Yves David <pierre-yves.david@octobus.net> parents: 47474 diff changeset	282 - old_data_file,
d6afe1478a2a censor: extract the part about creating and opening new files in a function Pierre-Yves David <pierre-yves.david@octobus.net> parents: 47474 diff changeset	283 - old_sidedata_file,
d6afe1478a2a censor: extract the part about creating and opening new files in a function Pierre-Yves David <pierre-yves.david@octobus.net> parents: 47474 diff changeset	284 - new_index_file,
d6afe1478a2a censor: extract the part about creating and opening new files in a function Pierre-Yves David <pierre-yves.david@octobus.net> parents: 47474 diff changeset	285 - new_data_file,
d6afe1478a2a censor: extract the part about creating and opening new files in a function Pierre-Yves David <pierre-yves.david@octobus.net> parents: 47474 diff changeset	286 - new_sidedata_file,
d6afe1478a2a censor: extract the part about creating and opening new files in a function Pierre-Yves David <pierre-yves.david@octobus.net> parents: 47474 diff changeset	287
d6afe1478a2a censor: extract the part about creating and opening new files in a function Pierre-Yves David <pierre-yves.david@octobus.net> parents: 47474 diff changeset	288 The old_index_file is not here because it is accessed through the
d6afe1478a2a censor: extract the part about creating and opening new files in a function Pierre-Yves David <pierre-yves.david@octobus.net> parents: 47474 diff changeset	289 `old_index` object if the caller function.
d6afe1478a2a censor: extract the part about creating and opening new files in a function Pierre-Yves David <pierre-yves.david@octobus.net> parents: 47474 diff changeset	290 """
d6afe1478a2a censor: extract the part about creating and opening new files in a function Pierre-Yves David <pierre-yves.david@octobus.net> parents: 47474 diff changeset	291 docket = revlog._docket
d6afe1478a2a censor: extract the part about creating and opening new files in a function Pierre-Yves David <pierre-yves.david@octobus.net> parents: 47474 diff changeset	292 old_index_filepath = revlog.opener.join(docket.index_filepath())
d6afe1478a2a censor: extract the part about creating and opening new files in a function Pierre-Yves David <pierre-yves.david@octobus.net> parents: 47474 diff changeset	293 old_data_filepath = revlog.opener.join(docket.data_filepath())
d6afe1478a2a censor: extract the part about creating and opening new files in a function Pierre-Yves David <pierre-yves.david@octobus.net> parents: 47474 diff changeset	294 old_sidedata_filepath = revlog.opener.join(docket.sidedata_filepath())
d6afe1478a2a censor: extract the part about creating and opening new files in a function Pierre-Yves David <pierre-yves.david@octobus.net> parents: 47474 diff changeset	295
d6afe1478a2a censor: extract the part about creating and opening new files in a function Pierre-Yves David <pierre-yves.david@octobus.net> parents: 47474 diff changeset	296 new_index_filepath = revlog.opener.join(docket.new_index_file())
d6afe1478a2a censor: extract the part about creating and opening new files in a function Pierre-Yves David <pierre-yves.david@octobus.net> parents: 47474 diff changeset	297 new_data_filepath = revlog.opener.join(docket.new_data_file())
d6afe1478a2a censor: extract the part about creating and opening new files in a function Pierre-Yves David <pierre-yves.david@octobus.net> parents: 47474 diff changeset	298 new_sidedata_filepath = revlog.opener.join(docket.new_sidedata_file())
d6afe1478a2a censor: extract the part about creating and opening new files in a function Pierre-Yves David <pierre-yves.david@octobus.net> parents: 47474 diff changeset	299
d6afe1478a2a censor: extract the part about creating and opening new files in a function Pierre-Yves David <pierre-yves.david@octobus.net> parents: 47474 diff changeset	300 util.copyfile(old_index_filepath, new_index_filepath, nb_bytes=index_cutoff)
d6afe1478a2a censor: extract the part about creating and opening new files in a function Pierre-Yves David <pierre-yves.david@octobus.net> parents: 47474 diff changeset	301 util.copyfile(old_data_filepath, new_data_filepath, nb_bytes=data_cutoff)
d6afe1478a2a censor: extract the part about creating and opening new files in a function Pierre-Yves David <pierre-yves.david@octobus.net> parents: 47474 diff changeset	302 util.copyfile(
d6afe1478a2a censor: extract the part about creating and opening new files in a function Pierre-Yves David <pierre-yves.david@octobus.net> parents: 47474 diff changeset	303 old_sidedata_filepath,
d6afe1478a2a censor: extract the part about creating and opening new files in a function Pierre-Yves David <pierre-yves.david@octobus.net> parents: 47474 diff changeset	304 new_sidedata_filepath,
d6afe1478a2a censor: extract the part about creating and opening new files in a function Pierre-Yves David <pierre-yves.david@octobus.net> parents: 47474 diff changeset	305 nb_bytes=sidedata_cutoff,
d6afe1478a2a censor: extract the part about creating and opening new files in a function Pierre-Yves David <pierre-yves.david@octobus.net> parents: 47474 diff changeset	306 )
d6afe1478a2a censor: extract the part about creating and opening new files in a function Pierre-Yves David <pierre-yves.david@octobus.net> parents: 47474 diff changeset	307 revlog.opener.register_file(docket.index_filepath())
d6afe1478a2a censor: extract the part about creating and opening new files in a function Pierre-Yves David <pierre-yves.david@octobus.net> parents: 47474 diff changeset	308 revlog.opener.register_file(docket.data_filepath())
d6afe1478a2a censor: extract the part about creating and opening new files in a function Pierre-Yves David <pierre-yves.david@octobus.net> parents: 47474 diff changeset	309 revlog.opener.register_file(docket.sidedata_filepath())
d6afe1478a2a censor: extract the part about creating and opening new files in a function Pierre-Yves David <pierre-yves.david@octobus.net> parents: 47474 diff changeset	310
d6afe1478a2a censor: extract the part about creating and opening new files in a function Pierre-Yves David <pierre-yves.david@octobus.net> parents: 47474 diff changeset	311 docket.index_end = index_cutoff
d6afe1478a2a censor: extract the part about creating and opening new files in a function Pierre-Yves David <pierre-yves.david@octobus.net> parents: 47474 diff changeset	312 docket.data_end = data_cutoff
d6afe1478a2a censor: extract the part about creating and opening new files in a function Pierre-Yves David <pierre-yves.david@octobus.net> parents: 47474 diff changeset	313 docket.sidedata_end = sidedata_cutoff
d6afe1478a2a censor: extract the part about creating and opening new files in a function Pierre-Yves David <pierre-yves.david@octobus.net> parents: 47474 diff changeset	314
d6afe1478a2a censor: extract the part about creating and opening new files in a function Pierre-Yves David <pierre-yves.david@octobus.net> parents: 47474 diff changeset	315 # reload the revlog internal information
d6afe1478a2a censor: extract the part about creating and opening new files in a function Pierre-Yves David <pierre-yves.david@octobus.net> parents: 47474 diff changeset	316 revlog.clearcaches()
d6afe1478a2a censor: extract the part about creating and opening new files in a function Pierre-Yves David <pierre-yves.david@octobus.net> parents: 47474 diff changeset	317 revlog._loadindex(docket=docket)
d6afe1478a2a censor: extract the part about creating and opening new files in a function Pierre-Yves David <pierre-yves.david@octobus.net> parents: 47474 diff changeset	318
d6afe1478a2a censor: extract the part about creating and opening new files in a function Pierre-Yves David <pierre-yves.david@octobus.net> parents: 47474 diff changeset	319 @contextlib.contextmanager
d6afe1478a2a censor: extract the part about creating and opening new files in a function Pierre-Yves David <pierre-yves.david@octobus.net> parents: 47474 diff changeset	320 def all_files_opener():
d6afe1478a2a censor: extract the part about creating and opening new files in a function Pierre-Yves David <pierre-yves.david@octobus.net> parents: 47474 diff changeset	321 # hide opening in an helper function to please check-code, black
d6afe1478a2a censor: extract the part about creating and opening new files in a function Pierre-Yves David <pierre-yves.david@octobus.net> parents: 47474 diff changeset	322 # and various python version at the same time
d6afe1478a2a censor: extract the part about creating and opening new files in a function Pierre-Yves David <pierre-yves.david@octobus.net> parents: 47474 diff changeset	323 with open(old_data_filepath, 'rb') as old_data_file:
d6afe1478a2a censor: extract the part about creating and opening new files in a function Pierre-Yves David <pierre-yves.david@octobus.net> parents: 47474 diff changeset	324 with open(old_sidedata_filepath, 'rb') as old_sidedata_file:
d6afe1478a2a censor: extract the part about creating and opening new files in a function Pierre-Yves David <pierre-yves.david@octobus.net> parents: 47474 diff changeset	325 with open(new_index_filepath, 'r+b') as new_index_file:
d6afe1478a2a censor: extract the part about creating and opening new files in a function Pierre-Yves David <pierre-yves.david@octobus.net> parents: 47474 diff changeset	326 with open(new_data_filepath, 'r+b') as new_data_file:
d6afe1478a2a censor: extract the part about creating and opening new files in a function Pierre-Yves David <pierre-yves.david@octobus.net> parents: 47474 diff changeset	327 with open(
d6afe1478a2a censor: extract the part about creating and opening new files in a function Pierre-Yves David <pierre-yves.david@octobus.net> parents: 47474 diff changeset	328 new_sidedata_filepath, 'r+b'
d6afe1478a2a censor: extract the part about creating and opening new files in a function Pierre-Yves David <pierre-yves.david@octobus.net> parents: 47474 diff changeset	329 ) as new_sidedata_file:
d6afe1478a2a censor: extract the part about creating and opening new files in a function Pierre-Yves David <pierre-yves.david@octobus.net> parents: 47474 diff changeset	330 new_index_file.seek(0, os.SEEK_END)
d6afe1478a2a censor: extract the part about creating and opening new files in a function Pierre-Yves David <pierre-yves.david@octobus.net> parents: 47474 diff changeset	331 assert new_index_file.tell() == index_cutoff
d6afe1478a2a censor: extract the part about creating and opening new files in a function Pierre-Yves David <pierre-yves.david@octobus.net> parents: 47474 diff changeset	332 new_data_file.seek(0, os.SEEK_END)
d6afe1478a2a censor: extract the part about creating and opening new files in a function Pierre-Yves David <pierre-yves.david@octobus.net> parents: 47474 diff changeset	333 assert new_data_file.tell() == data_cutoff
d6afe1478a2a censor: extract the part about creating and opening new files in a function Pierre-Yves David <pierre-yves.david@octobus.net> parents: 47474 diff changeset	334 new_sidedata_file.seek(0, os.SEEK_END)
d6afe1478a2a censor: extract the part about creating and opening new files in a function Pierre-Yves David <pierre-yves.david@octobus.net> parents: 47474 diff changeset	335 assert new_sidedata_file.tell() == sidedata_cutoff
d6afe1478a2a censor: extract the part about creating and opening new files in a function Pierre-Yves David <pierre-yves.david@octobus.net> parents: 47474 diff changeset	336 yield (
d6afe1478a2a censor: extract the part about creating and opening new files in a function Pierre-Yves David <pierre-yves.david@octobus.net> parents: 47474 diff changeset	337 old_data_file,
d6afe1478a2a censor: extract the part about creating and opening new files in a function Pierre-Yves David <pierre-yves.david@octobus.net> parents: 47474 diff changeset	338 old_sidedata_file,
d6afe1478a2a censor: extract the part about creating and opening new files in a function Pierre-Yves David <pierre-yves.david@octobus.net> parents: 47474 diff changeset	339 new_index_file,
d6afe1478a2a censor: extract the part about creating and opening new files in a function Pierre-Yves David <pierre-yves.david@octobus.net> parents: 47474 diff changeset	340 new_data_file,
d6afe1478a2a censor: extract the part about creating and opening new files in a function Pierre-Yves David <pierre-yves.david@octobus.net> parents: 47474 diff changeset	341 new_sidedata_file,
d6afe1478a2a censor: extract the part about creating and opening new files in a function Pierre-Yves David <pierre-yves.david@octobus.net> parents: 47474 diff changeset	342 )
d6afe1478a2a censor: extract the part about creating and opening new files in a function Pierre-Yves David <pierre-yves.david@octobus.net> parents: 47474 diff changeset	343
d6afe1478a2a censor: extract the part about creating and opening new files in a function Pierre-Yves David <pierre-yves.david@octobus.net> parents: 47474 diff changeset	344 return all_files_opener
d6afe1478a2a censor: extract the part about creating and opening new files in a function Pierre-Yves David <pierre-yves.david@octobus.net> parents: 47474 diff changeset	345
d6afe1478a2a censor: extract the part about creating and opening new files in a function Pierre-Yves David <pierre-yves.david@octobus.net> parents: 47474 diff changeset	346
47473 9b70aa7bcbab censor: extract the part about writing the other revision in a function Pierre-Yves David <pierre-yves.david@octobus.net> parents: 47472 diff changeset	347 def _rewrite_simple(
9b70aa7bcbab censor: extract the part about writing the other revision in a function Pierre-Yves David <pierre-yves.david@octobus.net> parents: 47472 diff changeset	348 revlog,
9b70aa7bcbab censor: extract the part about writing the other revision in a function Pierre-Yves David <pierre-yves.david@octobus.net> parents: 47472 diff changeset	349 old_index,
9b70aa7bcbab censor: extract the part about writing the other revision in a function Pierre-Yves David <pierre-yves.david@octobus.net> parents: 47472 diff changeset	350 all_files,
9b70aa7bcbab censor: extract the part about writing the other revision in a function Pierre-Yves David <pierre-yves.david@octobus.net> parents: 47472 diff changeset	351 rev,
9b70aa7bcbab censor: extract the part about writing the other revision in a function Pierre-Yves David <pierre-yves.david@octobus.net> parents: 47472 diff changeset	352 rewritten_entries,
9b70aa7bcbab censor: extract the part about writing the other revision in a function Pierre-Yves David <pierre-yves.david@octobus.net> parents: 47472 diff changeset	353 tmp_storage,
9b70aa7bcbab censor: extract the part about writing the other revision in a function Pierre-Yves David <pierre-yves.david@octobus.net> parents: 47472 diff changeset	354 ):
9b70aa7bcbab censor: extract the part about writing the other revision in a function Pierre-Yves David <pierre-yves.david@octobus.net> parents: 47472 diff changeset	355 """append a normal revision to the index after the rewritten one(s)"""
9b70aa7bcbab censor: extract the part about writing the other revision in a function Pierre-Yves David <pierre-yves.david@octobus.net> parents: 47472 diff changeset	356 (
9b70aa7bcbab censor: extract the part about writing the other revision in a function Pierre-Yves David <pierre-yves.david@octobus.net> parents: 47472 diff changeset	357 old_data_file,
9b70aa7bcbab censor: extract the part about writing the other revision in a function Pierre-Yves David <pierre-yves.david@octobus.net> parents: 47472 diff changeset	358 old_sidedata_file,
9b70aa7bcbab censor: extract the part about writing the other revision in a function Pierre-Yves David <pierre-yves.david@octobus.net> parents: 47472 diff changeset	359 new_index_file,
9b70aa7bcbab censor: extract the part about writing the other revision in a function Pierre-Yves David <pierre-yves.david@octobus.net> parents: 47472 diff changeset	360 new_data_file,
9b70aa7bcbab censor: extract the part about writing the other revision in a function Pierre-Yves David <pierre-yves.david@octobus.net> parents: 47472 diff changeset	361 new_sidedata_file,
9b70aa7bcbab censor: extract the part about writing the other revision in a function Pierre-Yves David <pierre-yves.david@octobus.net> parents: 47472 diff changeset	362 ) = all_files
9b70aa7bcbab censor: extract the part about writing the other revision in a function Pierre-Yves David <pierre-yves.david@octobus.net> parents: 47472 diff changeset	363 entry = old_index[rev]
9b70aa7bcbab censor: extract the part about writing the other revision in a function Pierre-Yves David <pierre-yves.david@octobus.net> parents: 47472 diff changeset	364 flags = entry[ENTRY_DATA_OFFSET] & 0xFFFF
9b70aa7bcbab censor: extract the part about writing the other revision in a function Pierre-Yves David <pierre-yves.david@octobus.net> parents: 47472 diff changeset	365 old_data_offset = entry[ENTRY_DATA_OFFSET] >> 16
47463 f8330a3fc39f censor: implement censoring for revlogv2 Pierre-Yves David <pierre-yves.david@octobus.net> parents: 47398 diff changeset	366
47473 9b70aa7bcbab censor: extract the part about writing the other revision in a function Pierre-Yves David <pierre-yves.david@octobus.net> parents: 47472 diff changeset	367 if rev not in rewritten_entries:
9b70aa7bcbab censor: extract the part about writing the other revision in a function Pierre-Yves David <pierre-yves.david@octobus.net> parents: 47472 diff changeset	368 old_data_file.seek(old_data_offset)
9b70aa7bcbab censor: extract the part about writing the other revision in a function Pierre-Yves David <pierre-yves.david@octobus.net> parents: 47472 diff changeset	369 new_data_size = entry[ENTRY_DATA_COMPRESSED_LENGTH]
9b70aa7bcbab censor: extract the part about writing the other revision in a function Pierre-Yves David <pierre-yves.david@octobus.net> parents: 47472 diff changeset	370 new_data = old_data_file.read(new_data_size)
9b70aa7bcbab censor: extract the part about writing the other revision in a function Pierre-Yves David <pierre-yves.david@octobus.net> parents: 47472 diff changeset	371 data_delta_base = entry[ENTRY_DELTA_BASE]
9b70aa7bcbab censor: extract the part about writing the other revision in a function Pierre-Yves David <pierre-yves.david@octobus.net> parents: 47472 diff changeset	372 d_comp_mode = entry[ENTRY_DATA_COMPRESSION_MODE]
9b70aa7bcbab censor: extract the part about writing the other revision in a function Pierre-Yves David <pierre-yves.david@octobus.net> parents: 47472 diff changeset	373 else:
9b70aa7bcbab censor: extract the part about writing the other revision in a function Pierre-Yves David <pierre-yves.david@octobus.net> parents: 47472 diff changeset	374 (
9b70aa7bcbab censor: extract the part about writing the other revision in a function Pierre-Yves David <pierre-yves.david@octobus.net> parents: 47472 diff changeset	375 data_delta_base,
9b70aa7bcbab censor: extract the part about writing the other revision in a function Pierre-Yves David <pierre-yves.david@octobus.net> parents: 47472 diff changeset	376 start,
9b70aa7bcbab censor: extract the part about writing the other revision in a function Pierre-Yves David <pierre-yves.david@octobus.net> parents: 47472 diff changeset	377 end,
9b70aa7bcbab censor: extract the part about writing the other revision in a function Pierre-Yves David <pierre-yves.david@octobus.net> parents: 47472 diff changeset	378 d_comp_mode,
9b70aa7bcbab censor: extract the part about writing the other revision in a function Pierre-Yves David <pierre-yves.david@octobus.net> parents: 47472 diff changeset	379 ) = rewritten_entries[rev]
9b70aa7bcbab censor: extract the part about writing the other revision in a function Pierre-Yves David <pierre-yves.david@octobus.net> parents: 47472 diff changeset	380 new_data_size = end - start
9b70aa7bcbab censor: extract the part about writing the other revision in a function Pierre-Yves David <pierre-yves.david@octobus.net> parents: 47472 diff changeset	381 tmp_storage.seek(start)
9b70aa7bcbab censor: extract the part about writing the other revision in a function Pierre-Yves David <pierre-yves.david@octobus.net> parents: 47472 diff changeset	382 new_data = tmp_storage.read(new_data_size)
47463 f8330a3fc39f censor: implement censoring for revlogv2 Pierre-Yves David <pierre-yves.david@octobus.net> parents: 47398 diff changeset	383
47473 9b70aa7bcbab censor: extract the part about writing the other revision in a function Pierre-Yves David <pierre-yves.david@octobus.net> parents: 47472 diff changeset	384 # It might be faster to group continuous read/write operation,
9b70aa7bcbab censor: extract the part about writing the other revision in a function Pierre-Yves David <pierre-yves.david@octobus.net> parents: 47472 diff changeset	385 # however, this is censor, an operation that is not focussed
9b70aa7bcbab censor: extract the part about writing the other revision in a function Pierre-Yves David <pierre-yves.david@octobus.net> parents: 47472 diff changeset	386 # around stellar performance. So I have not written this
9b70aa7bcbab censor: extract the part about writing the other revision in a function Pierre-Yves David <pierre-yves.david@octobus.net> parents: 47472 diff changeset	387 # optimisation yet.
9b70aa7bcbab censor: extract the part about writing the other revision in a function Pierre-Yves David <pierre-yves.david@octobus.net> parents: 47472 diff changeset	388 new_data_offset = new_data_file.tell()
9b70aa7bcbab censor: extract the part about writing the other revision in a function Pierre-Yves David <pierre-yves.david@octobus.net> parents: 47472 diff changeset	389 new_data_file.write(new_data)
47463 f8330a3fc39f censor: implement censoring for revlogv2 Pierre-Yves David <pierre-yves.david@octobus.net> parents: 47398 diff changeset	390
47473 9b70aa7bcbab censor: extract the part about writing the other revision in a function Pierre-Yves David <pierre-yves.david@octobus.net> parents: 47472 diff changeset	391 sidedata_size = entry[ENTRY_SIDEDATA_COMPRESSED_LENGTH]
9b70aa7bcbab censor: extract the part about writing the other revision in a function Pierre-Yves David <pierre-yves.david@octobus.net> parents: 47472 diff changeset	392 new_sidedata_offset = new_sidedata_file.tell()
9b70aa7bcbab censor: extract the part about writing the other revision in a function Pierre-Yves David <pierre-yves.david@octobus.net> parents: 47472 diff changeset	393 if 0 < sidedata_size:
9b70aa7bcbab censor: extract the part about writing the other revision in a function Pierre-Yves David <pierre-yves.david@octobus.net> parents: 47472 diff changeset	394 old_sidedata_offset = entry[ENTRY_SIDEDATA_OFFSET]
9b70aa7bcbab censor: extract the part about writing the other revision in a function Pierre-Yves David <pierre-yves.david@octobus.net> parents: 47472 diff changeset	395 old_sidedata_file.seek(old_sidedata_offset)
9b70aa7bcbab censor: extract the part about writing the other revision in a function Pierre-Yves David <pierre-yves.david@octobus.net> parents: 47472 diff changeset	396 new_sidedata = old_sidedata_file.read(sidedata_size)
9b70aa7bcbab censor: extract the part about writing the other revision in a function Pierre-Yves David <pierre-yves.david@octobus.net> parents: 47472 diff changeset	397 new_sidedata_file.write(new_sidedata)
9b70aa7bcbab censor: extract the part about writing the other revision in a function Pierre-Yves David <pierre-yves.david@octobus.net> parents: 47472 diff changeset	398
9b70aa7bcbab censor: extract the part about writing the other revision in a function Pierre-Yves David <pierre-yves.david@octobus.net> parents: 47472 diff changeset	399 data_uncompressed_length = entry[ENTRY_DATA_UNCOMPRESSED_LENGTH]
9b70aa7bcbab censor: extract the part about writing the other revision in a function Pierre-Yves David <pierre-yves.david@octobus.net> parents: 47472 diff changeset	400 sd_com_mode = entry[ENTRY_SIDEDATA_COMPRESSION_MODE]
9b70aa7bcbab censor: extract the part about writing the other revision in a function Pierre-Yves David <pierre-yves.david@octobus.net> parents: 47472 diff changeset	401 assert data_delta_base <= rev, (data_delta_base, rev)
47463 f8330a3fc39f censor: implement censoring for revlogv2 Pierre-Yves David <pierre-yves.david@octobus.net> parents: 47398 diff changeset	402
47473 9b70aa7bcbab censor: extract the part about writing the other revision in a function Pierre-Yves David <pierre-yves.david@octobus.net> parents: 47472 diff changeset	403 new_entry = revlogutils.entry(
9b70aa7bcbab censor: extract the part about writing the other revision in a function Pierre-Yves David <pierre-yves.david@octobus.net> parents: 47472 diff changeset	404 flags=flags,
9b70aa7bcbab censor: extract the part about writing the other revision in a function Pierre-Yves David <pierre-yves.david@octobus.net> parents: 47472 diff changeset	405 data_offset=new_data_offset,
9b70aa7bcbab censor: extract the part about writing the other revision in a function Pierre-Yves David <pierre-yves.david@octobus.net> parents: 47472 diff changeset	406 data_compressed_length=new_data_size,
9b70aa7bcbab censor: extract the part about writing the other revision in a function Pierre-Yves David <pierre-yves.david@octobus.net> parents: 47472 diff changeset	407 data_uncompressed_length=data_uncompressed_length,
9b70aa7bcbab censor: extract the part about writing the other revision in a function Pierre-Yves David <pierre-yves.david@octobus.net> parents: 47472 diff changeset	408 data_delta_base=data_delta_base,
9b70aa7bcbab censor: extract the part about writing the other revision in a function Pierre-Yves David <pierre-yves.david@octobus.net> parents: 47472 diff changeset	409 link_rev=entry[ENTRY_LINK_REV],
9b70aa7bcbab censor: extract the part about writing the other revision in a function Pierre-Yves David <pierre-yves.david@octobus.net> parents: 47472 diff changeset	410 parent_rev_1=entry[ENTRY_PARENT_1],
9b70aa7bcbab censor: extract the part about writing the other revision in a function Pierre-Yves David <pierre-yves.david@octobus.net> parents: 47472 diff changeset	411 parent_rev_2=entry[ENTRY_PARENT_2],
9b70aa7bcbab censor: extract the part about writing the other revision in a function Pierre-Yves David <pierre-yves.david@octobus.net> parents: 47472 diff changeset	412 node_id=entry[ENTRY_NODE_ID],
9b70aa7bcbab censor: extract the part about writing the other revision in a function Pierre-Yves David <pierre-yves.david@octobus.net> parents: 47472 diff changeset	413 sidedata_offset=new_sidedata_offset,
9b70aa7bcbab censor: extract the part about writing the other revision in a function Pierre-Yves David <pierre-yves.david@octobus.net> parents: 47472 diff changeset	414 sidedata_compressed_length=sidedata_size,
9b70aa7bcbab censor: extract the part about writing the other revision in a function Pierre-Yves David <pierre-yves.david@octobus.net> parents: 47472 diff changeset	415 data_compression_mode=d_comp_mode,
9b70aa7bcbab censor: extract the part about writing the other revision in a function Pierre-Yves David <pierre-yves.david@octobus.net> parents: 47472 diff changeset	416 sidedata_compression_mode=sd_com_mode,
9b70aa7bcbab censor: extract the part about writing the other revision in a function Pierre-Yves David <pierre-yves.david@octobus.net> parents: 47472 diff changeset	417 )
9b70aa7bcbab censor: extract the part about writing the other revision in a function Pierre-Yves David <pierre-yves.david@octobus.net> parents: 47472 diff changeset	418 revlog.index.append(new_entry)
9b70aa7bcbab censor: extract the part about writing the other revision in a function Pierre-Yves David <pierre-yves.david@octobus.net> parents: 47472 diff changeset	419 entry_bin = revlog.index.entry_binary(rev)
9b70aa7bcbab censor: extract the part about writing the other revision in a function Pierre-Yves David <pierre-yves.david@octobus.net> parents: 47472 diff changeset	420 new_index_file.write(entry_bin)
47463 f8330a3fc39f censor: implement censoring for revlogv2 Pierre-Yves David <pierre-yves.david@octobus.net> parents: 47398 diff changeset	421
47473 9b70aa7bcbab censor: extract the part about writing the other revision in a function Pierre-Yves David <pierre-yves.david@octobus.net> parents: 47472 diff changeset	422 revlog._docket.index_end = new_index_file.tell()
9b70aa7bcbab censor: extract the part about writing the other revision in a function Pierre-Yves David <pierre-yves.david@octobus.net> parents: 47472 diff changeset	423 revlog._docket.data_end = new_data_file.tell()
9b70aa7bcbab censor: extract the part about writing the other revision in a function Pierre-Yves David <pierre-yves.david@octobus.net> parents: 47472 diff changeset	424 revlog._docket.sidedata_end = new_sidedata_file.tell()
47472 3ab267f0cbe4 censor: extract the part about writing the censored revision in a function Pierre-Yves David <pierre-yves.david@octobus.net> parents: 47471 diff changeset	425
3ab267f0cbe4 censor: extract the part about writing the censored revision in a function Pierre-Yves David <pierre-yves.david@octobus.net> parents: 47471 diff changeset	426
3ab267f0cbe4 censor: extract the part about writing the censored revision in a function Pierre-Yves David <pierre-yves.david@octobus.net> parents: 47471 diff changeset	427 def _rewrite_censor(
3ab267f0cbe4 censor: extract the part about writing the censored revision in a function Pierre-Yves David <pierre-yves.david@octobus.net> parents: 47471 diff changeset	428 revlog,
3ab267f0cbe4 censor: extract the part about writing the censored revision in a function Pierre-Yves David <pierre-yves.david@octobus.net> parents: 47471 diff changeset	429 old_index,
3ab267f0cbe4 censor: extract the part about writing the censored revision in a function Pierre-Yves David <pierre-yves.david@octobus.net> parents: 47471 diff changeset	430 all_files,
3ab267f0cbe4 censor: extract the part about writing the censored revision in a function Pierre-Yves David <pierre-yves.david@octobus.net> parents: 47471 diff changeset	431 rev,
3ab267f0cbe4 censor: extract the part about writing the censored revision in a function Pierre-Yves David <pierre-yves.david@octobus.net> parents: 47471 diff changeset	432 tombstone,
3ab267f0cbe4 censor: extract the part about writing the censored revision in a function Pierre-Yves David <pierre-yves.david@octobus.net> parents: 47471 diff changeset	433 ):
3ab267f0cbe4 censor: extract the part about writing the censored revision in a function Pierre-Yves David <pierre-yves.david@octobus.net> parents: 47471 diff changeset	434 """rewrite and append a censored revision"""
3ab267f0cbe4 censor: extract the part about writing the censored revision in a function Pierre-Yves David <pierre-yves.david@octobus.net> parents: 47471 diff changeset	435 (
3ab267f0cbe4 censor: extract the part about writing the censored revision in a function Pierre-Yves David <pierre-yves.david@octobus.net> parents: 47471 diff changeset	436 old_data_file,
3ab267f0cbe4 censor: extract the part about writing the censored revision in a function Pierre-Yves David <pierre-yves.david@octobus.net> parents: 47471 diff changeset	437 old_sidedata_file,
3ab267f0cbe4 censor: extract the part about writing the censored revision in a function Pierre-Yves David <pierre-yves.david@octobus.net> parents: 47471 diff changeset	438 new_index_file,
3ab267f0cbe4 censor: extract the part about writing the censored revision in a function Pierre-Yves David <pierre-yves.david@octobus.net> parents: 47471 diff changeset	439 new_data_file,
3ab267f0cbe4 censor: extract the part about writing the censored revision in a function Pierre-Yves David <pierre-yves.david@octobus.net> parents: 47471 diff changeset	440 new_sidedata_file,
3ab267f0cbe4 censor: extract the part about writing the censored revision in a function Pierre-Yves David <pierre-yves.david@octobus.net> parents: 47471 diff changeset	441 ) = all_files
3ab267f0cbe4 censor: extract the part about writing the censored revision in a function Pierre-Yves David <pierre-yves.david@octobus.net> parents: 47471 diff changeset	442 entry = old_index[rev]
3ab267f0cbe4 censor: extract the part about writing the censored revision in a function Pierre-Yves David <pierre-yves.david@octobus.net> parents: 47471 diff changeset	443
3ab267f0cbe4 censor: extract the part about writing the censored revision in a function Pierre-Yves David <pierre-yves.david@octobus.net> parents: 47471 diff changeset	444 # XXX consider trying the default compression too
3ab267f0cbe4 censor: extract the part about writing the censored revision in a function Pierre-Yves David <pierre-yves.david@octobus.net> parents: 47471 diff changeset	445 new_data_size = len(tombstone)
3ab267f0cbe4 censor: extract the part about writing the censored revision in a function Pierre-Yves David <pierre-yves.david@octobus.net> parents: 47471 diff changeset	446 new_data_offset = new_data_file.tell()
3ab267f0cbe4 censor: extract the part about writing the censored revision in a function Pierre-Yves David <pierre-yves.david@octobus.net> parents: 47471 diff changeset	447 new_data_file.write(tombstone)
3ab267f0cbe4 censor: extract the part about writing the censored revision in a function Pierre-Yves David <pierre-yves.david@octobus.net> parents: 47471 diff changeset	448
3ab267f0cbe4 censor: extract the part about writing the censored revision in a function Pierre-Yves David <pierre-yves.david@octobus.net> parents: 47471 diff changeset	449 # we are not adding any sidedata as they might leak info about the censored version
3ab267f0cbe4 censor: extract the part about writing the censored revision in a function Pierre-Yves David <pierre-yves.david@octobus.net> parents: 47471 diff changeset	450
3ab267f0cbe4 censor: extract the part about writing the censored revision in a function Pierre-Yves David <pierre-yves.david@octobus.net> parents: 47471 diff changeset	451 link_rev = entry[ENTRY_LINK_REV]
3ab267f0cbe4 censor: extract the part about writing the censored revision in a function Pierre-Yves David <pierre-yves.david@octobus.net> parents: 47471 diff changeset	452
3ab267f0cbe4 censor: extract the part about writing the censored revision in a function Pierre-Yves David <pierre-yves.david@octobus.net> parents: 47471 diff changeset	453 p1 = entry[ENTRY_PARENT_1]
3ab267f0cbe4 censor: extract the part about writing the censored revision in a function Pierre-Yves David <pierre-yves.david@octobus.net> parents: 47471 diff changeset	454 p2 = entry[ENTRY_PARENT_2]
3ab267f0cbe4 censor: extract the part about writing the censored revision in a function Pierre-Yves David <pierre-yves.david@octobus.net> parents: 47471 diff changeset	455
3ab267f0cbe4 censor: extract the part about writing the censored revision in a function Pierre-Yves David <pierre-yves.david@octobus.net> parents: 47471 diff changeset	456 new_entry = revlogutils.entry(
3ab267f0cbe4 censor: extract the part about writing the censored revision in a function Pierre-Yves David <pierre-yves.david@octobus.net> parents: 47471 diff changeset	457 flags=constants.REVIDX_ISCENSORED,
3ab267f0cbe4 censor: extract the part about writing the censored revision in a function Pierre-Yves David <pierre-yves.david@octobus.net> parents: 47471 diff changeset	458 data_offset=new_data_offset,
3ab267f0cbe4 censor: extract the part about writing the censored revision in a function Pierre-Yves David <pierre-yves.david@octobus.net> parents: 47471 diff changeset	459 data_compressed_length=new_data_size,
3ab267f0cbe4 censor: extract the part about writing the censored revision in a function Pierre-Yves David <pierre-yves.david@octobus.net> parents: 47471 diff changeset	460 data_uncompressed_length=new_data_size,
3ab267f0cbe4 censor: extract the part about writing the censored revision in a function Pierre-Yves David <pierre-yves.david@octobus.net> parents: 47471 diff changeset	461 data_delta_base=rev,
3ab267f0cbe4 censor: extract the part about writing the censored revision in a function Pierre-Yves David <pierre-yves.david@octobus.net> parents: 47471 diff changeset	462 link_rev=link_rev,
3ab267f0cbe4 censor: extract the part about writing the censored revision in a function Pierre-Yves David <pierre-yves.david@octobus.net> parents: 47471 diff changeset	463 parent_rev_1=p1,
3ab267f0cbe4 censor: extract the part about writing the censored revision in a function Pierre-Yves David <pierre-yves.david@octobus.net> parents: 47471 diff changeset	464 parent_rev_2=p2,
3ab267f0cbe4 censor: extract the part about writing the censored revision in a function Pierre-Yves David <pierre-yves.david@octobus.net> parents: 47471 diff changeset	465 node_id=entry[ENTRY_NODE_ID],
3ab267f0cbe4 censor: extract the part about writing the censored revision in a function Pierre-Yves David <pierre-yves.david@octobus.net> parents: 47471 diff changeset	466 sidedata_offset=0,
3ab267f0cbe4 censor: extract the part about writing the censored revision in a function Pierre-Yves David <pierre-yves.david@octobus.net> parents: 47471 diff changeset	467 sidedata_compressed_length=0,
3ab267f0cbe4 censor: extract the part about writing the censored revision in a function Pierre-Yves David <pierre-yves.david@octobus.net> parents: 47471 diff changeset	468 data_compression_mode=COMP_MODE_PLAIN,
3ab267f0cbe4 censor: extract the part about writing the censored revision in a function Pierre-Yves David <pierre-yves.david@octobus.net> parents: 47471 diff changeset	469 sidedata_compression_mode=COMP_MODE_PLAIN,
3ab267f0cbe4 censor: extract the part about writing the censored revision in a function Pierre-Yves David <pierre-yves.david@octobus.net> parents: 47471 diff changeset	470 )
3ab267f0cbe4 censor: extract the part about writing the censored revision in a function Pierre-Yves David <pierre-yves.david@octobus.net> parents: 47471 diff changeset	471 revlog.index.append(new_entry)
3ab267f0cbe4 censor: extract the part about writing the censored revision in a function Pierre-Yves David <pierre-yves.david@octobus.net> parents: 47471 diff changeset	472 entry_bin = revlog.index.entry_binary(rev)
3ab267f0cbe4 censor: extract the part about writing the censored revision in a function Pierre-Yves David <pierre-yves.david@octobus.net> parents: 47471 diff changeset	473 new_index_file.write(entry_bin)
3ab267f0cbe4 censor: extract the part about writing the censored revision in a function Pierre-Yves David <pierre-yves.david@octobus.net> parents: 47471 diff changeset	474 revlog._docket.index_end = new_index_file.tell()
3ab267f0cbe4 censor: extract the part about writing the censored revision in a function Pierre-Yves David <pierre-yves.david@octobus.net> parents: 47471 diff changeset	475 revlog._docket.data_end = new_data_file.tell()
47810 b30a53ffbf9b debugcommands: introduce a debug command to repair repos affected by issue6528 Rapha?l Gom?s <rgomes@octobus.net> parents: 47478 diff changeset	476
b30a53ffbf9b debugcommands: introduce a debug command to repair repos affected by issue6528 Rapha?l Gom?s <rgomes@octobus.net> parents: 47478 diff changeset	477
b30a53ffbf9b debugcommands: introduce a debug command to repair repos affected by issue6528 Rapha?l Gom?s <rgomes@octobus.net> parents: 47478 diff changeset	478 def _get_filename_from_filelog_index(path):
b30a53ffbf9b debugcommands: introduce a debug command to repair repos affected by issue6528 Rapha?l Gom?s <rgomes@octobus.net> parents: 47478 diff changeset	479 # Drop the extension and the `data/` prefix
b30a53ffbf9b debugcommands: introduce a debug command to repair repos affected by issue6528 Rapha?l Gom?s <rgomes@octobus.net> parents: 47478 diff changeset	480 path_part = path.rsplit(b'.', 1)[0].split(b'/', 1)
b30a53ffbf9b debugcommands: introduce a debug command to repair repos affected by issue6528 Rapha?l Gom?s <rgomes@octobus.net> parents: 47478 diff changeset	481 if len(path_part) < 2:
b30a53ffbf9b debugcommands: introduce a debug command to repair repos affected by issue6528 Rapha?l Gom?s <rgomes@octobus.net> parents: 47478 diff changeset	482 msg = _(b"cannot recognize filelog from filename: '%s'")
b30a53ffbf9b debugcommands: introduce a debug command to repair repos affected by issue6528 Rapha?l Gom?s <rgomes@octobus.net> parents: 47478 diff changeset	483 msg %= path
b30a53ffbf9b debugcommands: introduce a debug command to repair repos affected by issue6528 Rapha?l Gom?s <rgomes@octobus.net> parents: 47478 diff changeset	484 raise error.Abort(msg)
b30a53ffbf9b debugcommands: introduce a debug command to repair repos affected by issue6528 Rapha?l Gom?s <rgomes@octobus.net> parents: 47478 diff changeset	485
b30a53ffbf9b debugcommands: introduce a debug command to repair repos affected by issue6528 Rapha?l Gom?s <rgomes@octobus.net> parents: 47478 diff changeset	486 return path_part[1]
b30a53ffbf9b debugcommands: introduce a debug command to repair repos affected by issue6528 Rapha?l Gom?s <rgomes@octobus.net> parents: 47478 diff changeset	487
b30a53ffbf9b debugcommands: introduce a debug command to repair repos affected by issue6528 Rapha?l Gom?s <rgomes@octobus.net> parents: 47478 diff changeset	488
b30a53ffbf9b debugcommands: introduce a debug command to repair repos affected by issue6528 Rapha?l Gom?s <rgomes@octobus.net> parents: 47478 diff changeset	489 def _filelog_from_filename(repo, path):
b30a53ffbf9b debugcommands: introduce a debug command to repair repos affected by issue6528 Rapha?l Gom?s <rgomes@octobus.net> parents: 47478 diff changeset	490 """Returns the filelog for the given `path`. Stolen from `engine.py`"""
b30a53ffbf9b debugcommands: introduce a debug command to repair repos affected by issue6528 Rapha?l Gom?s <rgomes@octobus.net> parents: 47478 diff changeset	491
b30a53ffbf9b debugcommands: introduce a debug command to repair repos affected by issue6528 Rapha?l Gom?s <rgomes@octobus.net> parents: 47478 diff changeset	492 from .. import filelog # avoid cycle
b30a53ffbf9b debugcommands: introduce a debug command to repair repos affected by issue6528 Rapha?l Gom?s <rgomes@octobus.net> parents: 47478 diff changeset	493
b30a53ffbf9b debugcommands: introduce a debug command to repair repos affected by issue6528 Rapha?l Gom?s <rgomes@octobus.net> parents: 47478 diff changeset	494 fl = filelog.filelog(repo.svfs, path)
b30a53ffbf9b debugcommands: introduce a debug command to repair repos affected by issue6528 Rapha?l Gom?s <rgomes@octobus.net> parents: 47478 diff changeset	495 return fl
b30a53ffbf9b debugcommands: introduce a debug command to repair repos affected by issue6528 Rapha?l Gom?s <rgomes@octobus.net> parents: 47478 diff changeset	496
b30a53ffbf9b debugcommands: introduce a debug command to repair repos affected by issue6528 Rapha?l Gom?s <rgomes@octobus.net> parents: 47478 diff changeset	497
b30a53ffbf9b debugcommands: introduce a debug command to repair repos affected by issue6528 Rapha?l Gom?s <rgomes@octobus.net> parents: 47478 diff changeset	498 def _write_swapped_parents(repo, rl, rev, offset, fp):
b30a53ffbf9b debugcommands: introduce a debug command to repair repos affected by issue6528 Rapha?l Gom?s <rgomes@octobus.net> parents: 47478 diff changeset	499 """Swaps p1 and p2 and overwrites the revlog entry for `rev` in `fp`"""
b30a53ffbf9b debugcommands: introduce a debug command to repair repos affected by issue6528 Rapha?l Gom?s <rgomes@octobus.net> parents: 47478 diff changeset	500 from ..pure import parsers # avoid cycle
b30a53ffbf9b debugcommands: introduce a debug command to repair repos affected by issue6528 Rapha?l Gom?s <rgomes@octobus.net> parents: 47478 diff changeset	501
b30a53ffbf9b debugcommands: introduce a debug command to repair repos affected by issue6528 Rapha?l Gom?s <rgomes@octobus.net> parents: 47478 diff changeset	502 if repo._currentlock(repo._lockref) is None:
b30a53ffbf9b debugcommands: introduce a debug command to repair repos affected by issue6528 Rapha?l Gom?s <rgomes@octobus.net> parents: 47478 diff changeset	503 # Let's be paranoid about it
b30a53ffbf9b debugcommands: introduce a debug command to repair repos affected by issue6528 Rapha?l Gom?s <rgomes@octobus.net> parents: 47478 diff changeset	504 msg = "repo needs to be locked to rewrite parents"
b30a53ffbf9b debugcommands: introduce a debug command to repair repos affected by issue6528 Rapha?l Gom?s <rgomes@octobus.net> parents: 47478 diff changeset	505 raise error.ProgrammingError(msg)
b30a53ffbf9b debugcommands: introduce a debug command to repair repos affected by issue6528 Rapha?l Gom?s <rgomes@octobus.net> parents: 47478 diff changeset	506
b30a53ffbf9b debugcommands: introduce a debug command to repair repos affected by issue6528 Rapha?l Gom?s <rgomes@octobus.net> parents: 47478 diff changeset	507 index_format = parsers.IndexObject.index_format
b30a53ffbf9b debugcommands: introduce a debug command to repair repos affected by issue6528 Rapha?l Gom?s <rgomes@octobus.net> parents: 47478 diff changeset	508 entry = rl.index[rev]
b30a53ffbf9b debugcommands: introduce a debug command to repair repos affected by issue6528 Rapha?l Gom?s <rgomes@octobus.net> parents: 47478 diff changeset	509 new_entry = list(entry)
b30a53ffbf9b debugcommands: introduce a debug command to repair repos affected by issue6528 Rapha?l Gom?s <rgomes@octobus.net> parents: 47478 diff changeset	510 new_entry[5], new_entry[6] = entry[6], entry[5]
b30a53ffbf9b debugcommands: introduce a debug command to repair repos affected by issue6528 Rapha?l Gom?s <rgomes@octobus.net> parents: 47478 diff changeset	511 packed = index_format.pack(*new_entry[:8])
b30a53ffbf9b debugcommands: introduce a debug command to repair repos affected by issue6528 Rapha?l Gom?s <rgomes@octobus.net> parents: 47478 diff changeset	512 fp.seek(offset)
b30a53ffbf9b debugcommands: introduce a debug command to repair repos affected by issue6528 Rapha?l Gom?s <rgomes@octobus.net> parents: 47478 diff changeset	513 fp.write(packed)
b30a53ffbf9b debugcommands: introduce a debug command to repair repos affected by issue6528 Rapha?l Gom?s <rgomes@octobus.net> parents: 47478 diff changeset	514
b30a53ffbf9b debugcommands: introduce a debug command to repair repos affected by issue6528 Rapha?l Gom?s <rgomes@octobus.net> parents: 47478 diff changeset	515
b30a53ffbf9b debugcommands: introduce a debug command to repair repos affected by issue6528 Rapha?l Gom?s <rgomes@octobus.net> parents: 47478 diff changeset	516 def _reorder_filelog_parents(repo, fl, to_fix):
b30a53ffbf9b debugcommands: introduce a debug command to repair repos affected by issue6528 Rapha?l Gom?s <rgomes@octobus.net> parents: 47478 diff changeset	517 """
b30a53ffbf9b debugcommands: introduce a debug command to repair repos affected by issue6528 Rapha?l Gom?s <rgomes@octobus.net> parents: 47478 diff changeset	518 Swaps p1 and p2 for all `to_fix` revisions of filelog `fl` and writes the
b30a53ffbf9b debugcommands: introduce a debug command to repair repos affected by issue6528 Rapha?l Gom?s <rgomes@octobus.net> parents: 47478 diff changeset	519 new version to disk, overwriting the old one with a rename.
b30a53ffbf9b debugcommands: introduce a debug command to repair repos affected by issue6528 Rapha?l Gom?s <rgomes@octobus.net> parents: 47478 diff changeset	520 """
b30a53ffbf9b debugcommands: introduce a debug command to repair repos affected by issue6528 Rapha?l Gom?s <rgomes@octobus.net> parents: 47478 diff changeset	521 from ..pure import parsers # avoid cycle
b30a53ffbf9b debugcommands: introduce a debug command to repair repos affected by issue6528 Rapha?l Gom?s <rgomes@octobus.net> parents: 47478 diff changeset	522
b30a53ffbf9b debugcommands: introduce a debug command to repair repos affected by issue6528 Rapha?l Gom?s <rgomes@octobus.net> parents: 47478 diff changeset	523 ui = repo.ui
b30a53ffbf9b debugcommands: introduce a debug command to repair repos affected by issue6528 Rapha?l Gom?s <rgomes@octobus.net> parents: 47478 diff changeset	524 assert len(to_fix) > 0
b30a53ffbf9b debugcommands: introduce a debug command to repair repos affected by issue6528 Rapha?l Gom?s <rgomes@octobus.net> parents: 47478 diff changeset	525 rl = fl._revlog
b30a53ffbf9b debugcommands: introduce a debug command to repair repos affected by issue6528 Rapha?l Gom?s <rgomes@octobus.net> parents: 47478 diff changeset	526 if rl._format_version != constants.REVLOGV1:
b30a53ffbf9b debugcommands: introduce a debug command to repair repos affected by issue6528 Rapha?l Gom?s <rgomes@octobus.net> parents: 47478 diff changeset	527 msg = "expected version 1 revlog, got version '%d'" % rl._format_version
b30a53ffbf9b debugcommands: introduce a debug command to repair repos affected by issue6528 Rapha?l Gom?s <rgomes@octobus.net> parents: 47478 diff changeset	528 raise error.ProgrammingError(msg)
b30a53ffbf9b debugcommands: introduce a debug command to repair repos affected by issue6528 Rapha?l Gom?s <rgomes@octobus.net> parents: 47478 diff changeset	529
b30a53ffbf9b debugcommands: introduce a debug command to repair repos affected by issue6528 Rapha?l Gom?s <rgomes@octobus.net> parents: 47478 diff changeset	530 index_file = rl._indexfile
b30a53ffbf9b debugcommands: introduce a debug command to repair repos affected by issue6528 Rapha?l Gom?s <rgomes@octobus.net> parents: 47478 diff changeset	531 new_file_path = index_file + b'.tmp-parents-fix'
b30a53ffbf9b debugcommands: introduce a debug command to repair repos affected by issue6528 Rapha?l Gom?s <rgomes@octobus.net> parents: 47478 diff changeset	532 repaired_msg = _(b"repaired revision %d of 'filelog %s'\n")
b30a53ffbf9b debugcommands: introduce a debug command to repair repos affected by issue6528 Rapha?l Gom?s <rgomes@octobus.net> parents: 47478 diff changeset	533
b30a53ffbf9b debugcommands: introduce a debug command to repair repos affected by issue6528 Rapha?l Gom?s <rgomes@octobus.net> parents: 47478 diff changeset	534 with ui.uninterruptible():
b30a53ffbf9b debugcommands: introduce a debug command to repair repos affected by issue6528 Rapha?l Gom?s <rgomes@octobus.net> parents: 47478 diff changeset	535 try:
b30a53ffbf9b debugcommands: introduce a debug command to repair repos affected by issue6528 Rapha?l Gom?s <rgomes@octobus.net> parents: 47478 diff changeset	536 util.copyfile(
b30a53ffbf9b debugcommands: introduce a debug command to repair repos affected by issue6528 Rapha?l Gom?s <rgomes@octobus.net> parents: 47478 diff changeset	537 rl.opener.join(index_file),
b30a53ffbf9b debugcommands: introduce a debug command to repair repos affected by issue6528 Rapha?l Gom?s <rgomes@octobus.net> parents: 47478 diff changeset	538 rl.opener.join(new_file_path),
b30a53ffbf9b debugcommands: introduce a debug command to repair repos affected by issue6528 Rapha?l Gom?s <rgomes@octobus.net> parents: 47478 diff changeset	539 checkambig=rl._checkambig,
b30a53ffbf9b debugcommands: introduce a debug command to repair repos affected by issue6528 Rapha?l Gom?s <rgomes@octobus.net> parents: 47478 diff changeset	540 )
b30a53ffbf9b debugcommands: introduce a debug command to repair repos affected by issue6528 Rapha?l Gom?s <rgomes@octobus.net> parents: 47478 diff changeset	541
b30a53ffbf9b debugcommands: introduce a debug command to repair repos affected by issue6528 Rapha?l Gom?s <rgomes@octobus.net> parents: 47478 diff changeset	542 with rl.opener(new_file_path, mode=b"r+") as fp:
b30a53ffbf9b debugcommands: introduce a debug command to repair repos affected by issue6528 Rapha?l Gom?s <rgomes@octobus.net> parents: 47478 diff changeset	543 if rl._inline:
b30a53ffbf9b debugcommands: introduce a debug command to repair repos affected by issue6528 Rapha?l Gom?s <rgomes@octobus.net> parents: 47478 diff changeset	544 index = parsers.InlinedIndexObject(fp.read())
b30a53ffbf9b debugcommands: introduce a debug command to repair repos affected by issue6528 Rapha?l Gom?s <rgomes@octobus.net> parents: 47478 diff changeset	545 for rev in fl.revs():
b30a53ffbf9b debugcommands: introduce a debug command to repair repos affected by issue6528 Rapha?l Gom?s <rgomes@octobus.net> parents: 47478 diff changeset	546 if rev in to_fix:
b30a53ffbf9b debugcommands: introduce a debug command to repair repos affected by issue6528 Rapha?l Gom?s <rgomes@octobus.net> parents: 47478 diff changeset	547 offset = index._calculate_index(rev)
b30a53ffbf9b debugcommands: introduce a debug command to repair repos affected by issue6528 Rapha?l Gom?s <rgomes@octobus.net> parents: 47478 diff changeset	548 _write_swapped_parents(repo, rl, rev, offset, fp)
b30a53ffbf9b debugcommands: introduce a debug command to repair repos affected by issue6528 Rapha?l Gom?s <rgomes@octobus.net> parents: 47478 diff changeset	549 ui.write(repaired_msg % (rev, index_file))
b30a53ffbf9b debugcommands: introduce a debug command to repair repos affected by issue6528 Rapha?l Gom?s <rgomes@octobus.net> parents: 47478 diff changeset	550 else:
b30a53ffbf9b debugcommands: introduce a debug command to repair repos affected by issue6528 Rapha?l Gom?s <rgomes@octobus.net> parents: 47478 diff changeset	551 index_format = parsers.IndexObject.index_format
b30a53ffbf9b debugcommands: introduce a debug command to repair repos affected by issue6528 Rapha?l Gom?s <rgomes@octobus.net> parents: 47478 diff changeset	552 for rev in to_fix:
b30a53ffbf9b debugcommands: introduce a debug command to repair repos affected by issue6528 Rapha?l Gom?s <rgomes@octobus.net> parents: 47478 diff changeset	553 offset = rev * index_format.size
b30a53ffbf9b debugcommands: introduce a debug command to repair repos affected by issue6528 Rapha?l Gom?s <rgomes@octobus.net> parents: 47478 diff changeset	554 _write_swapped_parents(repo, rl, rev, offset, fp)
b30a53ffbf9b debugcommands: introduce a debug command to repair repos affected by issue6528 Rapha?l Gom?s <rgomes@octobus.net> parents: 47478 diff changeset	555 ui.write(repaired_msg % (rev, index_file))
b30a53ffbf9b debugcommands: introduce a debug command to repair repos affected by issue6528 Rapha?l Gom?s <rgomes@octobus.net> parents: 47478 diff changeset	556
b30a53ffbf9b debugcommands: introduce a debug command to repair repos affected by issue6528 Rapha?l Gom?s <rgomes@octobus.net> parents: 47478 diff changeset	557 rl.opener.rename(new_file_path, index_file)
b30a53ffbf9b debugcommands: introduce a debug command to repair repos affected by issue6528 Rapha?l Gom?s <rgomes@octobus.net> parents: 47478 diff changeset	558 rl.clearcaches()
b30a53ffbf9b debugcommands: introduce a debug command to repair repos affected by issue6528 Rapha?l Gom?s <rgomes@octobus.net> parents: 47478 diff changeset	559 rl._loadindex()
b30a53ffbf9b debugcommands: introduce a debug command to repair repos affected by issue6528 Rapha?l Gom?s <rgomes@octobus.net> parents: 47478 diff changeset	560 finally:
b30a53ffbf9b debugcommands: introduce a debug command to repair repos affected by issue6528 Rapha?l Gom?s <rgomes@octobus.net> parents: 47478 diff changeset	561 util.tryunlink(new_file_path)
b30a53ffbf9b debugcommands: introduce a debug command to repair repos affected by issue6528 Rapha?l Gom?s <rgomes@octobus.net> parents: 47478 diff changeset	562
b30a53ffbf9b debugcommands: introduce a debug command to repair repos affected by issue6528 Rapha?l Gom?s <rgomes@octobus.net> parents: 47478 diff changeset	563
b30a53ffbf9b debugcommands: introduce a debug command to repair repos affected by issue6528 Rapha?l Gom?s <rgomes@octobus.net> parents: 47478 diff changeset	564 def _is_revision_affected(ui, fl, filerev, path):
b30a53ffbf9b debugcommands: introduce a debug command to repair repos affected by issue6528 Rapha?l Gom?s <rgomes@octobus.net> parents: 47478 diff changeset	565 """Mercurial currently (5.9rc0) uses `p1 == nullrev and p2 != nullrev` as a
b30a53ffbf9b debugcommands: introduce a debug command to repair repos affected by issue6528 Rapha?l Gom?s <rgomes@octobus.net> parents: 47478 diff changeset	566 special meaning compared to the reverse in the context of filelog-based
b30a53ffbf9b debugcommands: introduce a debug command to repair repos affected by issue6528 Rapha?l Gom?s <rgomes@octobus.net> parents: 47478 diff changeset	567 copytracing. issue6528 exists because new code assumed that parent ordering
b30a53ffbf9b debugcommands: introduce a debug command to repair repos affected by issue6528 Rapha?l Gom?s <rgomes@octobus.net> parents: 47478 diff changeset	568 didn't matter, so this detects if the revision contains metadata (since
b30a53ffbf9b debugcommands: introduce a debug command to repair repos affected by issue6528 Rapha?l Gom?s <rgomes@octobus.net> parents: 47478 diff changeset	569 it's only used for filelog-based copytracing) and its parents are in the
b30a53ffbf9b debugcommands: introduce a debug command to repair repos affected by issue6528 Rapha?l Gom?s <rgomes@octobus.net> parents: 47478 diff changeset	570 "wrong" order."""
b30a53ffbf9b debugcommands: introduce a debug command to repair repos affected by issue6528 Rapha?l Gom?s <rgomes@octobus.net> parents: 47478 diff changeset	571 try:
b30a53ffbf9b debugcommands: introduce a debug command to repair repos affected by issue6528 Rapha?l Gom?s <rgomes@octobus.net> parents: 47478 diff changeset	572 raw_text = fl.rawdata(filerev)
b30a53ffbf9b debugcommands: introduce a debug command to repair repos affected by issue6528 Rapha?l Gom?s <rgomes@octobus.net> parents: 47478 diff changeset	573 except error.CensoredNodeError:
b30a53ffbf9b debugcommands: introduce a debug command to repair repos affected by issue6528 Rapha?l Gom?s <rgomes@octobus.net> parents: 47478 diff changeset	574 # We don't care about censored nodes as they never carry metadata
b30a53ffbf9b debugcommands: introduce a debug command to repair repos affected by issue6528 Rapha?l Gom?s <rgomes@octobus.net> parents: 47478 diff changeset	575 return False
b30a53ffbf9b debugcommands: introduce a debug command to repair repos affected by issue6528 Rapha?l Gom?s <rgomes@octobus.net> parents: 47478 diff changeset	576 has_meta = raw_text.startswith(b'\x01\n')
b30a53ffbf9b debugcommands: introduce a debug command to repair repos affected by issue6528 Rapha?l Gom?s <rgomes@octobus.net> parents: 47478 diff changeset	577 if has_meta:
b30a53ffbf9b debugcommands: introduce a debug command to repair repos affected by issue6528 Rapha?l Gom?s <rgomes@octobus.net> parents: 47478 diff changeset	578 (p1, p2) = fl.parentrevs(filerev)
b30a53ffbf9b debugcommands: introduce a debug command to repair repos affected by issue6528 Rapha?l Gom?s <rgomes@octobus.net> parents: 47478 diff changeset	579 if p1 != nullrev and p2 == nullrev:
b30a53ffbf9b debugcommands: introduce a debug command to repair repos affected by issue6528 Rapha?l Gom?s <rgomes@octobus.net> parents: 47478 diff changeset	580 return True
b30a53ffbf9b debugcommands: introduce a debug command to repair repos affected by issue6528 Rapha?l Gom?s <rgomes@octobus.net> parents: 47478 diff changeset	581 return False
b30a53ffbf9b debugcommands: introduce a debug command to repair repos affected by issue6528 Rapha?l Gom?s <rgomes@octobus.net> parents: 47478 diff changeset	582
b30a53ffbf9b debugcommands: introduce a debug command to repair repos affected by issue6528 Rapha?l Gom?s <rgomes@octobus.net> parents: 47478 diff changeset	583
b30a53ffbf9b debugcommands: introduce a debug command to repair repos affected by issue6528 Rapha?l Gom?s <rgomes@octobus.net> parents: 47478 diff changeset	584 def _from_report(ui, repo, context, from_report, dry_run):
b30a53ffbf9b debugcommands: introduce a debug command to repair repos affected by issue6528 Rapha?l Gom?s <rgomes@octobus.net> parents: 47478 diff changeset	585 """
b30a53ffbf9b debugcommands: introduce a debug command to repair repos affected by issue6528 Rapha?l Gom?s <rgomes@octobus.net> parents: 47478 diff changeset	586 Fix the revisions given in the `from_report` file, but still checks if the
b30a53ffbf9b debugcommands: introduce a debug command to repair repos affected by issue6528 Rapha?l Gom?s <rgomes@octobus.net> parents: 47478 diff changeset	587 revisions are indeed affected to prevent an unfortunate cyclic situation
b30a53ffbf9b debugcommands: introduce a debug command to repair repos affected by issue6528 Rapha?l Gom?s <rgomes@octobus.net> parents: 47478 diff changeset	588 where we'd swap well-ordered parents again.
b30a53ffbf9b debugcommands: introduce a debug command to repair repos affected by issue6528 Rapha?l Gom?s <rgomes@octobus.net> parents: 47478 diff changeset	589
b30a53ffbf9b debugcommands: introduce a debug command to repair repos affected by issue6528 Rapha?l Gom?s <rgomes@octobus.net> parents: 47478 diff changeset	590 See the doc for `debug_fix_issue6528` for the format documentation.
b30a53ffbf9b debugcommands: introduce a debug command to repair repos affected by issue6528 Rapha?l Gom?s <rgomes@octobus.net> parents: 47478 diff changeset	591 """
b30a53ffbf9b debugcommands: introduce a debug command to repair repos affected by issue6528 Rapha?l Gom?s <rgomes@octobus.net> parents: 47478 diff changeset	592 ui.write(_(b"loading report file '%s'\n") % from_report)
b30a53ffbf9b debugcommands: introduce a debug command to repair repos affected by issue6528 Rapha?l Gom?s <rgomes@octobus.net> parents: 47478 diff changeset	593
b30a53ffbf9b debugcommands: introduce a debug command to repair repos affected by issue6528 Rapha?l Gom?s <rgomes@octobus.net> parents: 47478 diff changeset	594 with context(), open(from_report, mode='rb') as f:
b30a53ffbf9b debugcommands: introduce a debug command to repair repos affected by issue6528 Rapha?l Gom?s <rgomes@octobus.net> parents: 47478 diff changeset	595 for line in f.read().split(b'\n'):
b30a53ffbf9b debugcommands: introduce a debug command to repair repos affected by issue6528 Rapha?l Gom?s <rgomes@octobus.net> parents: 47478 diff changeset	596 if not line:
b30a53ffbf9b debugcommands: introduce a debug command to repair repos affected by issue6528 Rapha?l Gom?s <rgomes@octobus.net> parents: 47478 diff changeset	597 continue
b30a53ffbf9b debugcommands: introduce a debug command to repair repos affected by issue6528 Rapha?l Gom?s <rgomes@octobus.net> parents: 47478 diff changeset	598 filenodes, filename = line.split(b' ', 1)
b30a53ffbf9b debugcommands: introduce a debug command to repair repos affected by issue6528 Rapha?l Gom?s <rgomes@octobus.net> parents: 47478 diff changeset	599 fl = _filelog_from_filename(repo, filename)
b30a53ffbf9b debugcommands: introduce a debug command to repair repos affected by issue6528 Rapha?l Gom?s <rgomes@octobus.net> parents: 47478 diff changeset	600 to_fix = set(
b30a53ffbf9b debugcommands: introduce a debug command to repair repos affected by issue6528 Rapha?l Gom?s <rgomes@octobus.net> parents: 47478 diff changeset	601 fl.rev(binascii.unhexlify(n)) for n in filenodes.split(b',')
b30a53ffbf9b debugcommands: introduce a debug command to repair repos affected by issue6528 Rapha?l Gom?s <rgomes@octobus.net> parents: 47478 diff changeset	602 )
b30a53ffbf9b debugcommands: introduce a debug command to repair repos affected by issue6528 Rapha?l Gom?s <rgomes@octobus.net> parents: 47478 diff changeset	603 excluded = set()
b30a53ffbf9b debugcommands: introduce a debug command to repair repos affected by issue6528 Rapha?l Gom?s <rgomes@octobus.net> parents: 47478 diff changeset	604
b30a53ffbf9b debugcommands: introduce a debug command to repair repos affected by issue6528 Rapha?l Gom?s <rgomes@octobus.net> parents: 47478 diff changeset	605 for filerev in to_fix:
b30a53ffbf9b debugcommands: introduce a debug command to repair repos affected by issue6528 Rapha?l Gom?s <rgomes@octobus.net> parents: 47478 diff changeset	606 if _is_revision_affected(ui, fl, filerev, filename):
b30a53ffbf9b debugcommands: introduce a debug command to repair repos affected by issue6528 Rapha?l Gom?s <rgomes@octobus.net> parents: 47478 diff changeset	607 msg = b"found affected revision %d for filelog '%s'\n"
b30a53ffbf9b debugcommands: introduce a debug command to repair repos affected by issue6528 Rapha?l Gom?s <rgomes@octobus.net> parents: 47478 diff changeset	608 ui.warn(msg % (filerev, filename))
b30a53ffbf9b debugcommands: introduce a debug command to repair repos affected by issue6528 Rapha?l Gom?s <rgomes@octobus.net> parents: 47478 diff changeset	609 else:
b30a53ffbf9b debugcommands: introduce a debug command to repair repos affected by issue6528 Rapha?l Gom?s <rgomes@octobus.net> parents: 47478 diff changeset	610 msg = _(b"revision %s of file '%s' is not affected\n")
b30a53ffbf9b debugcommands: introduce a debug command to repair repos affected by issue6528 Rapha?l Gom?s <rgomes@octobus.net> parents: 47478 diff changeset	611 msg %= (binascii.hexlify(fl.node(filerev)), filename)
b30a53ffbf9b debugcommands: introduce a debug command to repair repos affected by issue6528 Rapha?l Gom?s <rgomes@octobus.net> parents: 47478 diff changeset	612 ui.warn(msg)
b30a53ffbf9b debugcommands: introduce a debug command to repair repos affected by issue6528 Rapha?l Gom?s <rgomes@octobus.net> parents: 47478 diff changeset	613 excluded.add(filerev)
b30a53ffbf9b debugcommands: introduce a debug command to repair repos affected by issue6528 Rapha?l Gom?s <rgomes@octobus.net> parents: 47478 diff changeset	614
b30a53ffbf9b debugcommands: introduce a debug command to repair repos affected by issue6528 Rapha?l Gom?s <rgomes@octobus.net> parents: 47478 diff changeset	615 to_fix = to_fix - excluded
b30a53ffbf9b debugcommands: introduce a debug command to repair repos affected by issue6528 Rapha?l Gom?s <rgomes@octobus.net> parents: 47478 diff changeset	616 if not to_fix:
b30a53ffbf9b debugcommands: introduce a debug command to repair repos affected by issue6528 Rapha?l Gom?s <rgomes@octobus.net> parents: 47478 diff changeset	617 msg = _(b"no affected revisions were found for '%s'\n")
b30a53ffbf9b debugcommands: introduce a debug command to repair repos affected by issue6528 Rapha?l Gom?s <rgomes@octobus.net> parents: 47478 diff changeset	618 ui.write(msg % filename)
b30a53ffbf9b debugcommands: introduce a debug command to repair repos affected by issue6528 Rapha?l Gom?s <rgomes@octobus.net> parents: 47478 diff changeset	619 continue
b30a53ffbf9b debugcommands: introduce a debug command to repair repos affected by issue6528 Rapha?l Gom?s <rgomes@octobus.net> parents: 47478 diff changeset	620 if not dry_run:
b30a53ffbf9b debugcommands: introduce a debug command to repair repos affected by issue6528 Rapha?l Gom?s <rgomes@octobus.net> parents: 47478 diff changeset	621 _reorder_filelog_parents(repo, fl, sorted(to_fix))
b30a53ffbf9b debugcommands: introduce a debug command to repair repos affected by issue6528 Rapha?l Gom?s <rgomes@octobus.net> parents: 47478 diff changeset	622
b30a53ffbf9b debugcommands: introduce a debug command to repair repos affected by issue6528 Rapha?l Gom?s <rgomes@octobus.net> parents: 47478 diff changeset	623
b30a53ffbf9b debugcommands: introduce a debug command to repair repos affected by issue6528 Rapha?l Gom?s <rgomes@octobus.net> parents: 47478 diff changeset	624 def repair_issue6528(ui, repo, dry_run=False, to_report=None, from_report=None):
b30a53ffbf9b debugcommands: introduce a debug command to repair repos affected by issue6528 Rapha?l Gom?s <rgomes@octobus.net> parents: 47478 diff changeset	625 from .. import store # avoid cycle
b30a53ffbf9b debugcommands: introduce a debug command to repair repos affected by issue6528 Rapha?l Gom?s <rgomes@octobus.net> parents: 47478 diff changeset	626
b30a53ffbf9b debugcommands: introduce a debug command to repair repos affected by issue6528 Rapha?l Gom?s <rgomes@octobus.net> parents: 47478 diff changeset	627 @contextlib.contextmanager
b30a53ffbf9b debugcommands: introduce a debug command to repair repos affected by issue6528 Rapha?l Gom?s <rgomes@octobus.net> parents: 47478 diff changeset	628 def context():
b30a53ffbf9b debugcommands: introduce a debug command to repair repos affected by issue6528 Rapha?l Gom?s <rgomes@octobus.net> parents: 47478 diff changeset	629 if dry_run or to_report: # No need for locking
b30a53ffbf9b debugcommands: introduce a debug command to repair repos affected by issue6528 Rapha?l Gom?s <rgomes@octobus.net> parents: 47478 diff changeset	630 yield
b30a53ffbf9b debugcommands: introduce a debug command to repair repos affected by issue6528 Rapha?l Gom?s <rgomes@octobus.net> parents: 47478 diff changeset	631 else:
b30a53ffbf9b debugcommands: introduce a debug command to repair repos affected by issue6528 Rapha?l Gom?s <rgomes@octobus.net> parents: 47478 diff changeset	632 with repo.wlock(), repo.lock():
b30a53ffbf9b debugcommands: introduce a debug command to repair repos affected by issue6528 Rapha?l Gom?s <rgomes@octobus.net> parents: 47478 diff changeset	633 yield
b30a53ffbf9b debugcommands: introduce a debug command to repair repos affected by issue6528 Rapha?l Gom?s <rgomes@octobus.net> parents: 47478 diff changeset	634
b30a53ffbf9b debugcommands: introduce a debug command to repair repos affected by issue6528 Rapha?l Gom?s <rgomes@octobus.net> parents: 47478 diff changeset	635 if from_report:
b30a53ffbf9b debugcommands: introduce a debug command to repair repos affected by issue6528 Rapha?l Gom?s <rgomes@octobus.net> parents: 47478 diff changeset	636 return _from_report(ui, repo, context, from_report, dry_run)
b30a53ffbf9b debugcommands: introduce a debug command to repair repos affected by issue6528 Rapha?l Gom?s <rgomes@octobus.net> parents: 47478 diff changeset	637
b30a53ffbf9b debugcommands: introduce a debug command to repair repos affected by issue6528 Rapha?l Gom?s <rgomes@octobus.net> parents: 47478 diff changeset	638 report_entries = []
b30a53ffbf9b debugcommands: introduce a debug command to repair repos affected by issue6528 Rapha?l Gom?s <rgomes@octobus.net> parents: 47478 diff changeset	639
b30a53ffbf9b debugcommands: introduce a debug command to repair repos affected by issue6528 Rapha?l Gom?s <rgomes@octobus.net> parents: 47478 diff changeset	640 with context():
b30a53ffbf9b debugcommands: introduce a debug command to repair repos affected by issue6528 Rapha?l Gom?s <rgomes@octobus.net> parents: 47478 diff changeset	641 files = list(
b30a53ffbf9b debugcommands: introduce a debug command to repair repos affected by issue6528 Rapha?l Gom?s <rgomes@octobus.net> parents: 47478 diff changeset	642 (file_type, path)
b30a53ffbf9b debugcommands: introduce a debug command to repair repos affected by issue6528 Rapha?l Gom?s <rgomes@octobus.net> parents: 47478 diff changeset	643 for (file_type, path, _e, _s) in repo.store.datafiles()
b30a53ffbf9b debugcommands: introduce a debug command to repair repos affected by issue6528 Rapha?l Gom?s <rgomes@octobus.net> parents: 47478 diff changeset	644 if path.endswith(b'.i') and file_type & store.FILEFLAGS_FILELOG
b30a53ffbf9b debugcommands: introduce a debug command to repair repos affected by issue6528 Rapha?l Gom?s <rgomes@octobus.net> parents: 47478 diff changeset	645 )
b30a53ffbf9b debugcommands: introduce a debug command to repair repos affected by issue6528 Rapha?l Gom?s <rgomes@octobus.net> parents: 47478 diff changeset	646
b30a53ffbf9b debugcommands: introduce a debug command to repair repos affected by issue6528 Rapha?l Gom?s <rgomes@octobus.net> parents: 47478 diff changeset	647 progress = ui.makeprogress(
b30a53ffbf9b debugcommands: introduce a debug command to repair repos affected by issue6528 Rapha?l Gom?s <rgomes@octobus.net> parents: 47478 diff changeset	648 _(b"looking for affected revisions"),
b30a53ffbf9b debugcommands: introduce a debug command to repair repos affected by issue6528 Rapha?l Gom?s <rgomes@octobus.net> parents: 47478 diff changeset	649 unit=_(b"filelogs"),
b30a53ffbf9b debugcommands: introduce a debug command to repair repos affected by issue6528 Rapha?l Gom?s <rgomes@octobus.net> parents: 47478 diff changeset	650 total=len(files),
b30a53ffbf9b debugcommands: introduce a debug command to repair repos affected by issue6528 Rapha?l Gom?s <rgomes@octobus.net> parents: 47478 diff changeset	651 )
b30a53ffbf9b debugcommands: introduce a debug command to repair repos affected by issue6528 Rapha?l Gom?s <rgomes@octobus.net> parents: 47478 diff changeset	652 found_nothing = True
b30a53ffbf9b debugcommands: introduce a debug command to repair repos affected by issue6528 Rapha?l Gom?s <rgomes@octobus.net> parents: 47478 diff changeset	653
b30a53ffbf9b debugcommands: introduce a debug command to repair repos affected by issue6528 Rapha?l Gom?s <rgomes@octobus.net> parents: 47478 diff changeset	654 for file_type, path in files:
b30a53ffbf9b debugcommands: introduce a debug command to repair repos affected by issue6528 Rapha?l Gom?s <rgomes@octobus.net> parents: 47478 diff changeset	655 if (
b30a53ffbf9b debugcommands: introduce a debug command to repair repos affected by issue6528 Rapha?l Gom?s <rgomes@octobus.net> parents: 47478 diff changeset	656 not path.endswith(b'.i')
b30a53ffbf9b debugcommands: introduce a debug command to repair repos affected by issue6528 Rapha?l Gom?s <rgomes@octobus.net> parents: 47478 diff changeset	657 or not file_type & store.FILEFLAGS_FILELOG
b30a53ffbf9b debugcommands: introduce a debug command to repair repos affected by issue6528 Rapha?l Gom?s <rgomes@octobus.net> parents: 47478 diff changeset	658 ):
b30a53ffbf9b debugcommands: introduce a debug command to repair repos affected by issue6528 Rapha?l Gom?s <rgomes@octobus.net> parents: 47478 diff changeset	659 continue
b30a53ffbf9b debugcommands: introduce a debug command to repair repos affected by issue6528 Rapha?l Gom?s <rgomes@octobus.net> parents: 47478 diff changeset	660 progress.increment()
b30a53ffbf9b debugcommands: introduce a debug command to repair repos affected by issue6528 Rapha?l Gom?s <rgomes@octobus.net> parents: 47478 diff changeset	661 filename = _get_filename_from_filelog_index(path)
b30a53ffbf9b debugcommands: introduce a debug command to repair repos affected by issue6528 Rapha?l Gom?s <rgomes@octobus.net> parents: 47478 diff changeset	662 fl = _filelog_from_filename(repo, filename)
b30a53ffbf9b debugcommands: introduce a debug command to repair repos affected by issue6528 Rapha?l Gom?s <rgomes@octobus.net> parents: 47478 diff changeset	663
b30a53ffbf9b debugcommands: introduce a debug command to repair repos affected by issue6528 Rapha?l Gom?s <rgomes@octobus.net> parents: 47478 diff changeset	664 # Set of filerevs (or hex filenodes if `to_report`) that need fixing
b30a53ffbf9b debugcommands: introduce a debug command to repair repos affected by issue6528 Rapha?l Gom?s <rgomes@octobus.net> parents: 47478 diff changeset	665 to_fix = set()
b30a53ffbf9b debugcommands: introduce a debug command to repair repos affected by issue6528 Rapha?l Gom?s <rgomes@octobus.net> parents: 47478 diff changeset	666 for filerev in fl.revs():
b30a53ffbf9b debugcommands: introduce a debug command to repair repos affected by issue6528 Rapha?l Gom?s <rgomes@octobus.net> parents: 47478 diff changeset	667 # TODO speed up by looking at the start of the delta
b30a53ffbf9b debugcommands: introduce a debug command to repair repos affected by issue6528 Rapha?l Gom?s <rgomes@octobus.net> parents: 47478 diff changeset	668 # If it hasn't changed, it's not worth looking at the other revs
b30a53ffbf9b debugcommands: introduce a debug command to repair repos affected by issue6528 Rapha?l Gom?s <rgomes@octobus.net> parents: 47478 diff changeset	669 # in the same chain
b30a53ffbf9b debugcommands: introduce a debug command to repair repos affected by issue6528 Rapha?l Gom?s <rgomes@octobus.net> parents: 47478 diff changeset	670 affected = _is_revision_affected(ui, fl, filerev, path)
b30a53ffbf9b debugcommands: introduce a debug command to repair repos affected by issue6528 Rapha?l Gom?s <rgomes@octobus.net> parents: 47478 diff changeset	671 if affected:
b30a53ffbf9b debugcommands: introduce a debug command to repair repos affected by issue6528 Rapha?l Gom?s <rgomes@octobus.net> parents: 47478 diff changeset	672 msg = b"found affected revision %d for filelog '%s'\n"
b30a53ffbf9b debugcommands: introduce a debug command to repair repos affected by issue6528 Rapha?l Gom?s <rgomes@octobus.net> parents: 47478 diff changeset	673 ui.warn(msg % (filerev, path))
b30a53ffbf9b debugcommands: introduce a debug command to repair repos affected by issue6528 Rapha?l Gom?s <rgomes@octobus.net> parents: 47478 diff changeset	674 found_nothing = False
b30a53ffbf9b debugcommands: introduce a debug command to repair repos affected by issue6528 Rapha?l Gom?s <rgomes@octobus.net> parents: 47478 diff changeset	675 if not dry_run:
b30a53ffbf9b debugcommands: introduce a debug command to repair repos affected by issue6528 Rapha?l Gom?s <rgomes@octobus.net> parents: 47478 diff changeset	676 if to_report:
b30a53ffbf9b debugcommands: introduce a debug command to repair repos affected by issue6528 Rapha?l Gom?s <rgomes@octobus.net> parents: 47478 diff changeset	677 to_fix.add(binascii.hexlify(fl.node(filerev)))
b30a53ffbf9b debugcommands: introduce a debug command to repair repos affected by issue6528 Rapha?l Gom?s <rgomes@octobus.net> parents: 47478 diff changeset	678 else:
b30a53ffbf9b debugcommands: introduce a debug command to repair repos affected by issue6528 Rapha?l Gom?s <rgomes@octobus.net> parents: 47478 diff changeset	679 to_fix.add(filerev)
b30a53ffbf9b debugcommands: introduce a debug command to repair repos affected by issue6528 Rapha?l Gom?s <rgomes@octobus.net> parents: 47478 diff changeset	680
b30a53ffbf9b debugcommands: introduce a debug command to repair repos affected by issue6528 Rapha?l Gom?s <rgomes@octobus.net> parents: 47478 diff changeset	681 if to_fix:
b30a53ffbf9b debugcommands: introduce a debug command to repair repos affected by issue6528 Rapha?l Gom?s <rgomes@octobus.net> parents: 47478 diff changeset	682 to_fix = sorted(to_fix)
b30a53ffbf9b debugcommands: introduce a debug command to repair repos affected by issue6528 Rapha?l Gom?s <rgomes@octobus.net> parents: 47478 diff changeset	683 if to_report:
b30a53ffbf9b debugcommands: introduce a debug command to repair repos affected by issue6528 Rapha?l Gom?s <rgomes@octobus.net> parents: 47478 diff changeset	684 report_entries.append((filename, to_fix))
b30a53ffbf9b debugcommands: introduce a debug command to repair repos affected by issue6528 Rapha?l Gom?s <rgomes@octobus.net> parents: 47478 diff changeset	685 else:
b30a53ffbf9b debugcommands: introduce a debug command to repair repos affected by issue6528 Rapha?l Gom?s <rgomes@octobus.net> parents: 47478 diff changeset	686 _reorder_filelog_parents(repo, fl, to_fix)
b30a53ffbf9b debugcommands: introduce a debug command to repair repos affected by issue6528 Rapha?l Gom?s <rgomes@octobus.net> parents: 47478 diff changeset	687
b30a53ffbf9b debugcommands: introduce a debug command to repair repos affected by issue6528 Rapha?l Gom?s <rgomes@octobus.net> parents: 47478 diff changeset	688 if found_nothing:
b30a53ffbf9b debugcommands: introduce a debug command to repair repos affected by issue6528 Rapha?l Gom?s <rgomes@octobus.net> parents: 47478 diff changeset	689 ui.write(_(b"no affected revisions were found\n"))
b30a53ffbf9b debugcommands: introduce a debug command to repair repos affected by issue6528 Rapha?l Gom?s <rgomes@octobus.net> parents: 47478 diff changeset	690
b30a53ffbf9b debugcommands: introduce a debug command to repair repos affected by issue6528 Rapha?l Gom?s <rgomes@octobus.net> parents: 47478 diff changeset	691 if to_report and report_entries:
b30a53ffbf9b debugcommands: introduce a debug command to repair repos affected by issue6528 Rapha?l Gom?s <rgomes@octobus.net> parents: 47478 diff changeset	692 with open(to_report, mode="wb") as f:
b30a53ffbf9b debugcommands: introduce a debug command to repair repos affected by issue6528 Rapha?l Gom?s <rgomes@octobus.net> parents: 47478 diff changeset	693 for path, to_fix in report_entries:
b30a53ffbf9b debugcommands: introduce a debug command to repair repos affected by issue6528 Rapha?l Gom?s <rgomes@octobus.net> parents: 47478 diff changeset	694 f.write(b"%s %s\n" % (b",".join(to_fix), path))
b30a53ffbf9b debugcommands: introduce a debug command to repair repos affected by issue6528 Rapha?l Gom?s <rgomes@octobus.net> parents: 47478 diff changeset	695
b30a53ffbf9b debugcommands: introduce a debug command to repair repos affected by issue6528 Rapha?l Gom?s <rgomes@octobus.net> parents: 47478 diff changeset	696 progress.complete()

Mercurial > public > mercurial-scm > hg-stable

annotate mercurial/revlogutils/rewrite.py @ 47810:b30a53ffbf9b stable