Mercurial > public > mercurial-scm > hg

annotate mercurial/verify.py @ 28111:06205989264b

Find changesets by keywords (author, files, the commit message), revision number or hash, or revset expression.
verify: move cross-checking of changeset/manifest out of _crosscheckfiles() Reasons: * _crosscheckfiles(), as the name suggests, is about checking that the set of files files mentioned in changesets match the set of files mentioned in the manifests. * The "checking" in _crosscheckfiles() looked rather strange, as it just emitted an error for *every* entry in mflinkrevs. The reason was that these were the entries remaining after the call to _verifymanifest(). Moving all the processing of mflinkrevs into _verifymanifest() makes it much clearer that it's the remaining entries that are a problem. Functional change: progress is no longer reported for "crosschecking" of missing manifest entries. Since the crosschecking phase takes a tiny fraction of the verification, I don't think this is a problem. Also, any reports of "changeset refers to unknown manifest" will now come before "crosschecking files in changesets and manifests".
author Martin von Zweigbergk <martinvonz@google.com>
date Sun, 31 Jan 2016 00:10:56 -0800
parents fb92927f9775
children 334a3aa677fb
Ignore whitespace changes - Everywhere: Within whitespace: At end of lines:
rev   line source
2778
fdc232d8a193 Move repo.verify
Matt Mackall <mpm@selenic.com>
parents:
diff changeset
1 # verify.py - repository integrity checking for Mercurial
fdc232d8a193 Move repo.verify
Matt Mackall <mpm@selenic.com>
parents:
diff changeset
2 #
4635
63b9d2deed48 Updated copyright notices and add "and others" to "hg version"
Thomas Arendsen Hein <thomas@intevation.de>
parents: 4395
diff changeset
3 # Copyright 2006, 2007 Matt Mackall <mpm@selenic.com>
2778
fdc232d8a193 Move repo.verify
Matt Mackall <mpm@selenic.com>
parents:
diff changeset
4 #
8225
46293a0c7e9f updated license to be explicit about GPL version 2
Martin Geisler <mg@lazybytes.net>
parents: 8209
diff changeset
5 # This software may be used and distributed according to the terms of the
10263
25e572394f5c Update license to GPLv2+
Matt Mackall <mpm@selenic.com>
parents: 9690
diff changeset
6 # GNU General Public License version 2 or any later version.
2778
fdc232d8a193 Move repo.verify
Matt Mackall <mpm@selenic.com>
parents:
diff changeset
7
25991
d21d1774c73b verify: use absolute_import
Gregory Szorc <gregory.szorc@gmail.com>
parents: 25846
diff changeset
8 from __future__ import absolute_import
d21d1774c73b verify: use absolute_import
Gregory Szorc <gregory.szorc@gmail.com>
parents: 25846
diff changeset
9
17860
a45b33f12627 verify: fix all doubled-slash sites (issue3665)
Bryan O'Sullivan <bryano@fb.com>
parents: 17851
diff changeset
10 import os
25991
d21d1774c73b verify: use absolute_import
Gregory Szorc <gregory.szorc@gmail.com>
parents: 25846
diff changeset
11
d21d1774c73b verify: use absolute_import
Gregory Szorc <gregory.szorc@gmail.com>
parents: 25846
diff changeset
12 from .i18n import _
d21d1774c73b verify: use absolute_import
Gregory Szorc <gregory.szorc@gmail.com>
parents: 25846
diff changeset
13 from .node import (
d21d1774c73b verify: use absolute_import
Gregory Szorc <gregory.szorc@gmail.com>
parents: 25846
diff changeset
14 nullid,
d21d1774c73b verify: use absolute_import
Gregory Szorc <gregory.szorc@gmail.com>
parents: 25846
diff changeset
15 short,
d21d1774c73b verify: use absolute_import
Gregory Szorc <gregory.szorc@gmail.com>
parents: 25846
diff changeset
16 )
d21d1774c73b verify: use absolute_import
Gregory Szorc <gregory.szorc@gmail.com>
parents: 25846
diff changeset
17
d21d1774c73b verify: use absolute_import
Gregory Szorc <gregory.szorc@gmail.com>
parents: 25846
diff changeset
18 from . import (
d21d1774c73b verify: use absolute_import
Gregory Szorc <gregory.szorc@gmail.com>
parents: 25846
diff changeset
19 error,
d21d1774c73b verify: use absolute_import
Gregory Szorc <gregory.szorc@gmail.com>
parents: 25846
diff changeset
20 revlog,
d21d1774c73b verify: use absolute_import
Gregory Szorc <gregory.szorc@gmail.com>
parents: 25846
diff changeset
21 util,
d21d1774c73b verify: use absolute_import
Gregory Szorc <gregory.szorc@gmail.com>
parents: 25846
diff changeset
22 )
2778
fdc232d8a193 Move repo.verify
Matt Mackall <mpm@selenic.com>
parents:
diff changeset
23
fdc232d8a193 Move repo.verify
Matt Mackall <mpm@selenic.com>
parents:
diff changeset
24 def verify(repo):
27849
900d36a3e4dd with: use context manager in verify
Bryan O'Sullivan <bryano@fb.com>
parents: 27695
diff changeset
25 with repo.lock():
27444
6647401858ab verify: move widely used variables into class members
Durham Goode <durham@fb.com>
parents: 27443
diff changeset
26 return verifier(repo).verify()
4915
97b734fb9c6f Use try/finally pattern to cleanup locks and transactions
Matt Mackall <mpm@selenic.com>
parents: 4635
diff changeset
27
17860
a45b33f12627 verify: fix all doubled-slash sites (issue3665)
Bryan O'Sullivan <bryano@fb.com>
parents: 17851
diff changeset
28 def _normpath(f):
a45b33f12627 verify: fix all doubled-slash sites (issue3665)
Bryan O'Sullivan <bryano@fb.com>
parents: 17851
diff changeset
29 # under hg < 2.4, convert didn't sanitize paths properly, so a
a45b33f12627 verify: fix all doubled-slash sites (issue3665)
Bryan O'Sullivan <bryano@fb.com>
parents: 17851
diff changeset
30 # converted repo may contain repeated slashes
a45b33f12627 verify: fix all doubled-slash sites (issue3665)
Bryan O'Sullivan <bryano@fb.com>
parents: 17851
diff changeset
31 while '//' in f:
a45b33f12627 verify: fix all doubled-slash sites (issue3665)
Bryan O'Sullivan <bryano@fb.com>
parents: 17851
diff changeset
32 f = f.replace('//', '/')
a45b33f12627 verify: fix all doubled-slash sites (issue3665)
Bryan O'Sullivan <bryano@fb.com>
parents: 17851
diff changeset
33 return f
a45b33f12627 verify: fix all doubled-slash sites (issue3665)
Bryan O'Sullivan <bryano@fb.com>
parents: 17851
diff changeset
34
26900
d1c741644d25 verify: add a hook that can let extensions manipulate file lists
Augie Fackler <augie@google.com>
parents: 26587
diff changeset
35 def _validpath(repo, path):
d1c741644d25 verify: add a hook that can let extensions manipulate file lists
Augie Fackler <augie@google.com>
parents: 26587
diff changeset
36 """Returns False if a path should NOT be treated as part of a repo.
d1c741644d25 verify: add a hook that can let extensions manipulate file lists
Augie Fackler <augie@google.com>
parents: 26587
diff changeset
37
d1c741644d25 verify: add a hook that can let extensions manipulate file lists
Augie Fackler <augie@google.com>
parents: 26587
diff changeset
38 For all in-core cases, this returns True, as we have no way for a
d1c741644d25 verify: add a hook that can let extensions manipulate file lists
Augie Fackler <augie@google.com>
parents: 26587
diff changeset
39 path to be mentioned in the history but not actually be
d1c741644d25 verify: add a hook that can let extensions manipulate file lists
Augie Fackler <augie@google.com>
parents: 26587
diff changeset
40 relevant. For narrow clones, this is important because many
d1c741644d25 verify: add a hook that can let extensions manipulate file lists
Augie Fackler <augie@google.com>
parents: 26587
diff changeset
41 filelogs will be missing, and changelog entries may mention
d1c741644d25 verify: add a hook that can let extensions manipulate file lists
Augie Fackler <augie@google.com>
parents: 26587
diff changeset
42 modified files that are outside the narrow scope.
d1c741644d25 verify: add a hook that can let extensions manipulate file lists
Augie Fackler <augie@google.com>
parents: 26587
diff changeset
43 """
d1c741644d25 verify: add a hook that can let extensions manipulate file lists
Augie Fackler <augie@google.com>
parents: 26587
diff changeset
44 return True
d1c741644d25 verify: add a hook that can let extensions manipulate file lists
Augie Fackler <augie@google.com>
parents: 26587
diff changeset
45
27443
937e73a6e4ff verify: move verify logic into a class
Durham Goode <durham@fb.com>
parents: 26900
diff changeset
46 class verifier(object):
27444
6647401858ab verify: move widely used variables into class members
Durham Goode <durham@fb.com>
parents: 27443
diff changeset
47 def __init__(self, repo):
6647401858ab verify: move widely used variables into class members
Durham Goode <durham@fb.com>
parents: 27443
diff changeset
48 self.repo = repo.unfiltered()
6647401858ab verify: move widely used variables into class members
Durham Goode <durham@fb.com>
parents: 27443
diff changeset
49 self.ui = repo.ui
6647401858ab verify: move widely used variables into class members
Durham Goode <durham@fb.com>
parents: 27443
diff changeset
50 self.badrevs = set()
27453
8462d7f2c4fe verify: clean up weird error/warning lists
Matt Mackall <mpm@selenic.com>
parents: 27450
diff changeset
51 self.errors = 0
8462d7f2c4fe verify: clean up weird error/warning lists
Matt Mackall <mpm@selenic.com>
parents: 27450
diff changeset
52 self.warnings = 0
27444
6647401858ab verify: move widely used variables into class members
Durham Goode <durham@fb.com>
parents: 27443
diff changeset
53 self.havecl = len(repo.changelog) > 0
6647401858ab verify: move widely used variables into class members
Durham Goode <durham@fb.com>
parents: 27443
diff changeset
54 self.havemf = len(repo.manifest) > 0
6647401858ab verify: move widely used variables into class members
Durham Goode <durham@fb.com>
parents: 27443
diff changeset
55 self.revlogv1 = repo.changelog.version != revlog.REVLOGV0
6647401858ab verify: move widely used variables into class members
Durham Goode <durham@fb.com>
parents: 27443
diff changeset
56 self.lrugetctx = util.lrucachefunc(repo.changectx)
6647401858ab verify: move widely used variables into class members
Durham Goode <durham@fb.com>
parents: 27443
diff changeset
57 self.refersmf = False
27445
cc178057ab49 verify: move fncachewarned up to a class variable
Durham Goode <durham@fb.com>
parents: 27444
diff changeset
58 self.fncachewarned = False
27444
6647401858ab verify: move widely used variables into class members
Durham Goode <durham@fb.com>
parents: 27443
diff changeset
59
27446
6b2c1a1871a6 verify: move warn() to a class level function
Durham Goode <durham@fb.com>
parents: 27445
diff changeset
60 def warn(self, msg):
6b2c1a1871a6 verify: move warn() to a class level function
Durham Goode <durham@fb.com>
parents: 27445
diff changeset
61 self.ui.warn(msg + "\n")
27453
8462d7f2c4fe verify: clean up weird error/warning lists
Matt Mackall <mpm@selenic.com>
parents: 27450
diff changeset
62 self.warnings += 1
27446
6b2c1a1871a6 verify: move warn() to a class level function
Durham Goode <durham@fb.com>
parents: 27445
diff changeset
63
27447
d1b91c10ce70 verify: move err() to be a class function
Durham Goode <durham@fb.com>
parents: 27446
diff changeset
64 def err(self, linkrev, msg, filename=None):
d1b91c10ce70 verify: move err() to be a class function
Durham Goode <durham@fb.com>
parents: 27446
diff changeset
65 if linkrev is not None:
d1b91c10ce70 verify: move err() to be a class function
Durham Goode <durham@fb.com>
parents: 27446
diff changeset
66 self.badrevs.add(linkrev)
d1b91c10ce70 verify: move err() to be a class function
Durham Goode <durham@fb.com>
parents: 27446
diff changeset
67 else:
d1b91c10ce70 verify: move err() to be a class function
Durham Goode <durham@fb.com>
parents: 27446
diff changeset
68 linkrev = '?'
d1b91c10ce70 verify: move err() to be a class function
Durham Goode <durham@fb.com>
parents: 27446
diff changeset
69 msg = "%s: %s" % (linkrev, msg)
d1b91c10ce70 verify: move err() to be a class function
Durham Goode <durham@fb.com>
parents: 27446
diff changeset
70 if filename:
d1b91c10ce70 verify: move err() to be a class function
Durham Goode <durham@fb.com>
parents: 27446
diff changeset
71 msg = "%s@%s" % (filename, msg)
d1b91c10ce70 verify: move err() to be a class function
Durham Goode <durham@fb.com>
parents: 27446
diff changeset
72 self.ui.warn(" " + msg + "\n")
27453
8462d7f2c4fe verify: clean up weird error/warning lists
Matt Mackall <mpm@selenic.com>
parents: 27450
diff changeset
73 self.errors += 1
27447
d1b91c10ce70 verify: move err() to be a class function
Durham Goode <durham@fb.com>
parents: 27446
diff changeset
74
27448
f4f2179077cb verify: move exc() function onto class
Durham Goode <durham@fb.com>
parents: 27447
diff changeset
75 def exc(self, linkrev, msg, inst, filename=None):
f4f2179077cb verify: move exc() function onto class
Durham Goode <durham@fb.com>
parents: 27447
diff changeset
76 if not str(inst):
f4f2179077cb verify: move exc() function onto class
Durham Goode <durham@fb.com>
parents: 27447
diff changeset
77 inst = repr(inst)
f4f2179077cb verify: move exc() function onto class
Durham Goode <durham@fb.com>
parents: 27447
diff changeset
78 self.err(linkrev, "%s: %s" % (msg, inst), filename)
f4f2179077cb verify: move exc() function onto class
Durham Goode <durham@fb.com>
parents: 27447
diff changeset
79
27642
f6457349985b verify: move checklog() onto class
Durham Goode <durham@fb.com>
parents: 27453
diff changeset
80 def checklog(self, obj, name, linkrev):
f6457349985b verify: move checklog() onto class
Durham Goode <durham@fb.com>
parents: 27453
diff changeset
81 if not len(obj) and (self.havecl or self.havemf):
f6457349985b verify: move checklog() onto class
Durham Goode <durham@fb.com>
parents: 27453
diff changeset
82 self.err(linkrev, _("empty or missing %s") % name)
f6457349985b verify: move checklog() onto class
Durham Goode <durham@fb.com>
parents: 27453
diff changeset
83 return
f6457349985b verify: move checklog() onto class
Durham Goode <durham@fb.com>
parents: 27453
diff changeset
84
f6457349985b verify: move checklog() onto class
Durham Goode <durham@fb.com>
parents: 27453
diff changeset
85 d = obj.checksize()
f6457349985b verify: move checklog() onto class
Durham Goode <durham@fb.com>
parents: 27453
diff changeset
86 if d[0]:
f6457349985b verify: move checklog() onto class
Durham Goode <durham@fb.com>
parents: 27453
diff changeset
87 self.err(None, _("data length off by %d bytes") % d[0], name)
f6457349985b verify: move checklog() onto class
Durham Goode <durham@fb.com>
parents: 27453
diff changeset
88 if d[1]:
f6457349985b verify: move checklog() onto class
Durham Goode <durham@fb.com>
parents: 27453
diff changeset
89 self.err(None, _("index contains %d extra bytes") % d[1], name)
f6457349985b verify: move checklog() onto class
Durham Goode <durham@fb.com>
parents: 27453
diff changeset
90
f6457349985b verify: move checklog() onto class
Durham Goode <durham@fb.com>
parents: 27453
diff changeset
91 if obj.version != revlog.REVLOGV0:
f6457349985b verify: move checklog() onto class
Durham Goode <durham@fb.com>
parents: 27453
diff changeset
92 if not self.revlogv1:
f6457349985b verify: move checklog() onto class
Durham Goode <durham@fb.com>
parents: 27453
diff changeset
93 self.warn(_("warning: `%s' uses revlog format 1") % name)
f6457349985b verify: move checklog() onto class
Durham Goode <durham@fb.com>
parents: 27453
diff changeset
94 elif self.revlogv1:
f6457349985b verify: move checklog() onto class
Durham Goode <durham@fb.com>
parents: 27453
diff changeset
95 self.warn(_("warning: `%s' uses revlog format 0") % name)
f6457349985b verify: move checklog() onto class
Durham Goode <durham@fb.com>
parents: 27453
diff changeset
96
27643
62ce86fcfd06 verify: move checkentry() to be a class function
Durham Goode <durham@fb.com>
parents: 27642
diff changeset
97 def checkentry(self, obj, i, node, seen, linkrevs, f):
62ce86fcfd06 verify: move checkentry() to be a class function
Durham Goode <durham@fb.com>
parents: 27642
diff changeset
98 lr = obj.linkrev(obj.rev(node))
62ce86fcfd06 verify: move checkentry() to be a class function
Durham Goode <durham@fb.com>
parents: 27642
diff changeset
99 if lr < 0 or (self.havecl and lr not in linkrevs):
62ce86fcfd06 verify: move checkentry() to be a class function
Durham Goode <durham@fb.com>
parents: 27642
diff changeset
100 if lr < 0 or lr >= len(self.repo.changelog):
62ce86fcfd06 verify: move checkentry() to be a class function
Durham Goode <durham@fb.com>
parents: 27642
diff changeset
101 msg = _("rev %d points to nonexistent changeset %d")
62ce86fcfd06 verify: move checkentry() to be a class function
Durham Goode <durham@fb.com>
parents: 27642
diff changeset
102 else:
62ce86fcfd06 verify: move checkentry() to be a class function
Durham Goode <durham@fb.com>
parents: 27642
diff changeset
103 msg = _("rev %d points to unexpected changeset %d")
62ce86fcfd06 verify: move checkentry() to be a class function
Durham Goode <durham@fb.com>
parents: 27642
diff changeset
104 self.err(None, msg % (i, lr), f)
62ce86fcfd06 verify: move checkentry() to be a class function
Durham Goode <durham@fb.com>
parents: 27642
diff changeset
105 if linkrevs:
62ce86fcfd06 verify: move checkentry() to be a class function
Durham Goode <durham@fb.com>
parents: 27642
diff changeset
106 if f and len(linkrevs) > 1:
62ce86fcfd06 verify: move checkentry() to be a class function
Durham Goode <durham@fb.com>
parents: 27642
diff changeset
107 try:
62ce86fcfd06 verify: move checkentry() to be a class function
Durham Goode <durham@fb.com>
parents: 27642
diff changeset
108 # attempt to filter down to real linkrevs
62ce86fcfd06 verify: move checkentry() to be a class function
Durham Goode <durham@fb.com>
parents: 27642
diff changeset
109 linkrevs = [l for l in linkrevs
62ce86fcfd06 verify: move checkentry() to be a class function
Durham Goode <durham@fb.com>
parents: 27642
diff changeset
110 if self.lrugetctx(l)[f].filenode() == node]
62ce86fcfd06 verify: move checkentry() to be a class function
Durham Goode <durham@fb.com>
parents: 27642
diff changeset
111 except Exception:
62ce86fcfd06 verify: move checkentry() to be a class function
Durham Goode <durham@fb.com>
parents: 27642
diff changeset
112 pass
62ce86fcfd06 verify: move checkentry() to be a class function
Durham Goode <durham@fb.com>
parents: 27642
diff changeset
113 self.warn(_(" (expected %s)") % " ".join(map(str, linkrevs)))
62ce86fcfd06 verify: move checkentry() to be a class function
Durham Goode <durham@fb.com>
parents: 27642
diff changeset
114 lr = None # can't be trusted
62ce86fcfd06 verify: move checkentry() to be a class function
Durham Goode <durham@fb.com>
parents: 27642
diff changeset
115
62ce86fcfd06 verify: move checkentry() to be a class function
Durham Goode <durham@fb.com>
parents: 27642
diff changeset
116 try:
62ce86fcfd06 verify: move checkentry() to be a class function
Durham Goode <durham@fb.com>
parents: 27642
diff changeset
117 p1, p2 = obj.parents(node)
62ce86fcfd06 verify: move checkentry() to be a class function
Durham Goode <durham@fb.com>
parents: 27642
diff changeset
118 if p1 not in seen and p1 != nullid:
62ce86fcfd06 verify: move checkentry() to be a class function
Durham Goode <durham@fb.com>
parents: 27642
diff changeset
119 self.err(lr, _("unknown parent 1 %s of %s") %
62ce86fcfd06 verify: move checkentry() to be a class function
Durham Goode <durham@fb.com>
parents: 27642
diff changeset
120 (short(p1), short(node)), f)
62ce86fcfd06 verify: move checkentry() to be a class function
Durham Goode <durham@fb.com>
parents: 27642
diff changeset
121 if p2 not in seen and p2 != nullid:
62ce86fcfd06 verify: move checkentry() to be a class function
Durham Goode <durham@fb.com>
parents: 27642
diff changeset
122 self.err(lr, _("unknown parent 2 %s of %s") %
62ce86fcfd06 verify: move checkentry() to be a class function
Durham Goode <durham@fb.com>
parents: 27642
diff changeset
123 (short(p2), short(node)), f)
62ce86fcfd06 verify: move checkentry() to be a class function
Durham Goode <durham@fb.com>
parents: 27642
diff changeset
124 except Exception as inst:
62ce86fcfd06 verify: move checkentry() to be a class function
Durham Goode <durham@fb.com>
parents: 27642
diff changeset
125 self.exc(lr, _("checking parents of %s") % short(node), inst, f)
62ce86fcfd06 verify: move checkentry() to be a class function
Durham Goode <durham@fb.com>
parents: 27642
diff changeset
126
62ce86fcfd06 verify: move checkentry() to be a class function
Durham Goode <durham@fb.com>
parents: 27642
diff changeset
127 if node in seen:
62ce86fcfd06 verify: move checkentry() to be a class function
Durham Goode <durham@fb.com>
parents: 27642
diff changeset
128 self.err(lr, _("duplicate revision %d (%d)") % (i, seen[node]), f)
62ce86fcfd06 verify: move checkentry() to be a class function
Durham Goode <durham@fb.com>
parents: 27642
diff changeset
129 seen[node] = i
62ce86fcfd06 verify: move checkentry() to be a class function
Durham Goode <durham@fb.com>
parents: 27642
diff changeset
130 return lr
62ce86fcfd06 verify: move checkentry() to be a class function
Durham Goode <durham@fb.com>
parents: 27642
diff changeset
131
27444
6647401858ab verify: move widely used variables into class members
Durham Goode <durham@fb.com>
parents: 27443
diff changeset
132 def verify(self):
6647401858ab verify: move widely used variables into class members
Durham Goode <durham@fb.com>
parents: 27443
diff changeset
133 repo = self.repo
27648
e72e669dd51f verify: get rid of some unnecessary local variables
Durham Goode <durham@fb.com>
parents: 27647
diff changeset
134
27443
937e73a6e4ff verify: move verify logic into a class
Durham Goode <durham@fb.com>
parents: 26900
diff changeset
135 ui = repo.ui
2778
fdc232d8a193 Move repo.verify
Matt Mackall <mpm@selenic.com>
parents:
diff changeset
136
27443
937e73a6e4ff verify: move verify logic into a class
Durham Goode <durham@fb.com>
parents: 26900
diff changeset
137 if not repo.url().startswith('file:'):
937e73a6e4ff verify: move verify logic into a class
Durham Goode <durham@fb.com>
parents: 26900
diff changeset
138 raise error.Abort(_("cannot verify bundle or remote repos"))
6752
e79a8f36c2a5 verify: lots of refactoring
Matt Mackall <mpm@selenic.com>
parents: 6751
diff changeset
139
27443
937e73a6e4ff verify: move verify logic into a class
Durham Goode <durham@fb.com>
parents: 26900
diff changeset
140 if os.path.exists(repo.sjoin("journal")):
937e73a6e4ff verify: move verify logic into a class
Durham Goode <durham@fb.com>
parents: 26900
diff changeset
141 ui.warn(_("abandoned transaction found - run hg recover\n"))
937e73a6e4ff verify: move verify logic into a class
Durham Goode <durham@fb.com>
parents: 26900
diff changeset
142
27648
e72e669dd51f verify: get rid of some unnecessary local variables
Durham Goode <durham@fb.com>
parents: 27647
diff changeset
143 if ui.verbose or not self.revlogv1:
27443
937e73a6e4ff verify: move verify logic into a class
Durham Goode <durham@fb.com>
parents: 26900
diff changeset
144 ui.status(_("repository uses revlog format %d\n") %
27648
e72e669dd51f verify: get rid of some unnecessary local variables
Durham Goode <durham@fb.com>
parents: 27647
diff changeset
145 (self.revlogv1 and 1 or 0))
27443
937e73a6e4ff verify: move verify logic into a class
Durham Goode <durham@fb.com>
parents: 26900
diff changeset
146
27695
fb0cc863d172 verify: replace "output parameters" by return values
Martin von Zweigbergk <martinvonz@google.com>
parents: 27648
diff changeset
147 mflinkrevs, filelinkrevs = self._verifychangelog()
27647
2c2858f3c1bb verify: move changelog verificaiton to its own function
Durham Goode <durham@fb.com>
parents: 27646
diff changeset
148
27695
fb0cc863d172 verify: replace "output parameters" by return values
Martin von Zweigbergk <martinvonz@google.com>
parents: 27648
diff changeset
149 filenodes = self._verifymanifest(mflinkrevs)
28111
06205989264b verify: move cross-checking of changeset/manifest out of _crosscheckfiles()
Martin von Zweigbergk <martinvonz@google.com>
parents: 28007
diff changeset
150 del mflinkrevs
27647
2c2858f3c1bb verify: move changelog verificaiton to its own function
Durham Goode <durham@fb.com>
parents: 27646
diff changeset
151
28111
06205989264b verify: move cross-checking of changeset/manifest out of _crosscheckfiles()
Martin von Zweigbergk <martinvonz@google.com>
parents: 28007
diff changeset
152 self._crosscheckfiles(filelinkrevs, filenodes)
27647
2c2858f3c1bb verify: move changelog verificaiton to its own function
Durham Goode <durham@fb.com>
parents: 27646
diff changeset
153
2c2858f3c1bb verify: move changelog verificaiton to its own function
Durham Goode <durham@fb.com>
parents: 27646
diff changeset
154 totalfiles, filerevisions = self._verifyfiles(filenodes, filelinkrevs)
2c2858f3c1bb verify: move changelog verificaiton to its own function
Durham Goode <durham@fb.com>
parents: 27646
diff changeset
155
2c2858f3c1bb verify: move changelog verificaiton to its own function
Durham Goode <durham@fb.com>
parents: 27646
diff changeset
156 ui.status(_("%d files, %d changesets, %d total revisions\n") %
27648
e72e669dd51f verify: get rid of some unnecessary local variables
Durham Goode <durham@fb.com>
parents: 27647
diff changeset
157 (totalfiles, len(repo.changelog), filerevisions))
27647
2c2858f3c1bb verify: move changelog verificaiton to its own function
Durham Goode <durham@fb.com>
parents: 27646
diff changeset
158 if self.warnings:
2c2858f3c1bb verify: move changelog verificaiton to its own function
Durham Goode <durham@fb.com>
parents: 27646
diff changeset
159 ui.warn(_("%d warnings encountered!\n") % self.warnings)
2c2858f3c1bb verify: move changelog verificaiton to its own function
Durham Goode <durham@fb.com>
parents: 27646
diff changeset
160 if self.fncachewarned:
2c2858f3c1bb verify: move changelog verificaiton to its own function
Durham Goode <durham@fb.com>
parents: 27646
diff changeset
161 ui.warn(_('hint: run "hg debugrebuildfncache" to recover from '
2c2858f3c1bb verify: move changelog verificaiton to its own function
Durham Goode <durham@fb.com>
parents: 27646
diff changeset
162 'corrupt fncache\n'))
2c2858f3c1bb verify: move changelog verificaiton to its own function
Durham Goode <durham@fb.com>
parents: 27646
diff changeset
163 if self.errors:
2c2858f3c1bb verify: move changelog verificaiton to its own function
Durham Goode <durham@fb.com>
parents: 27646
diff changeset
164 ui.warn(_("%d integrity errors encountered!\n") % self.errors)
27648
e72e669dd51f verify: get rid of some unnecessary local variables
Durham Goode <durham@fb.com>
parents: 27647
diff changeset
165 if self.badrevs:
27647
2c2858f3c1bb verify: move changelog verificaiton to its own function
Durham Goode <durham@fb.com>
parents: 27646
diff changeset
166 ui.warn(_("(first damaged changeset appears to be %d)\n")
27648
e72e669dd51f verify: get rid of some unnecessary local variables
Durham Goode <durham@fb.com>
parents: 27647
diff changeset
167 % min(self.badrevs))
27647
2c2858f3c1bb verify: move changelog verificaiton to its own function
Durham Goode <durham@fb.com>
parents: 27646
diff changeset
168 return 1
2c2858f3c1bb verify: move changelog verificaiton to its own function
Durham Goode <durham@fb.com>
parents: 27646
diff changeset
169
27695
fb0cc863d172 verify: replace "output parameters" by return values
Martin von Zweigbergk <martinvonz@google.com>
parents: 27648
diff changeset
170 def _verifychangelog(self):
27647
2c2858f3c1bb verify: move changelog verificaiton to its own function
Durham Goode <durham@fb.com>
parents: 27646
diff changeset
171 ui = self.ui
2c2858f3c1bb verify: move changelog verificaiton to its own function
Durham Goode <durham@fb.com>
parents: 27646
diff changeset
172 repo = self.repo
2c2858f3c1bb verify: move changelog verificaiton to its own function
Durham Goode <durham@fb.com>
parents: 27646
diff changeset
173 cl = repo.changelog
2c2858f3c1bb verify: move changelog verificaiton to its own function
Durham Goode <durham@fb.com>
parents: 27646
diff changeset
174
27443
937e73a6e4ff verify: move verify logic into a class
Durham Goode <durham@fb.com>
parents: 26900
diff changeset
175 ui.status(_("checking changesets\n"))
27695
fb0cc863d172 verify: replace "output parameters" by return values
Martin von Zweigbergk <martinvonz@google.com>
parents: 27648
diff changeset
176 mflinkrevs = {}
fb0cc863d172 verify: replace "output parameters" by return values
Martin von Zweigbergk <martinvonz@google.com>
parents: 27648
diff changeset
177 filelinkrevs = {}
27443
937e73a6e4ff verify: move verify logic into a class
Durham Goode <durham@fb.com>
parents: 26900
diff changeset
178 seen = {}
27642
f6457349985b verify: move checklog() onto class
Durham Goode <durham@fb.com>
parents: 27453
diff changeset
179 self.checklog(cl, "changelog", 0)
27443
937e73a6e4ff verify: move verify logic into a class
Durham Goode <durham@fb.com>
parents: 26900
diff changeset
180 total = len(repo)
937e73a6e4ff verify: move verify logic into a class
Durham Goode <durham@fb.com>
parents: 26900
diff changeset
181 for i in repo:
937e73a6e4ff verify: move verify logic into a class
Durham Goode <durham@fb.com>
parents: 26900
diff changeset
182 ui.progress(_('checking'), i, total=total, unit=_('changesets'))
937e73a6e4ff verify: move verify logic into a class
Durham Goode <durham@fb.com>
parents: 26900
diff changeset
183 n = cl.node(i)
27643
62ce86fcfd06 verify: move checkentry() to be a class function
Durham Goode <durham@fb.com>
parents: 27642
diff changeset
184 self.checkentry(cl, i, n, seen, [i], "changelog")
2778
fdc232d8a193 Move repo.verify
Matt Mackall <mpm@selenic.com>
parents:
diff changeset
185
27443
937e73a6e4ff verify: move verify logic into a class
Durham Goode <durham@fb.com>
parents: 26900
diff changeset
186 try:
937e73a6e4ff verify: move verify logic into a class
Durham Goode <durham@fb.com>
parents: 26900
diff changeset
187 changes = cl.read(n)
937e73a6e4ff verify: move verify logic into a class
Durham Goode <durham@fb.com>
parents: 26900
diff changeset
188 if changes[0] != nullid:
937e73a6e4ff verify: move verify logic into a class
Durham Goode <durham@fb.com>
parents: 26900
diff changeset
189 mflinkrevs.setdefault(changes[0], []).append(i)
27444
6647401858ab verify: move widely used variables into class members
Durham Goode <durham@fb.com>
parents: 27443
diff changeset
190 self.refersmf = True
27443
937e73a6e4ff verify: move verify logic into a class
Durham Goode <durham@fb.com>
parents: 26900
diff changeset
191 for f in changes[3]:
26900
d1c741644d25 verify: add a hook that can let extensions manipulate file lists
Augie Fackler <augie@google.com>
parents: 26587
diff changeset
192 if _validpath(repo, f):
27443
937e73a6e4ff verify: move verify logic into a class
Durham Goode <durham@fb.com>
parents: 26900
diff changeset
193 filelinkrevs.setdefault(_normpath(f), []).append(i)
937e73a6e4ff verify: move verify logic into a class
Durham Goode <durham@fb.com>
parents: 26900
diff changeset
194 except Exception as inst:
27444
6647401858ab verify: move widely used variables into class members
Durham Goode <durham@fb.com>
parents: 27443
diff changeset
195 self.refersmf = True
27448
f4f2179077cb verify: move exc() function onto class
Durham Goode <durham@fb.com>
parents: 27447
diff changeset
196 self.exc(i, _("unpacking changeset %s") % short(n), inst)
27443
937e73a6e4ff verify: move verify logic into a class
Durham Goode <durham@fb.com>
parents: 26900
diff changeset
197 ui.progress(_('checking'), None)
27695
fb0cc863d172 verify: replace "output parameters" by return values
Martin von Zweigbergk <martinvonz@google.com>
parents: 27648
diff changeset
198 return mflinkrevs, filelinkrevs
2778
fdc232d8a193 Move repo.verify
Matt Mackall <mpm@selenic.com>
parents:
diff changeset
199
27695
fb0cc863d172 verify: replace "output parameters" by return values
Martin von Zweigbergk <martinvonz@google.com>
parents: 27648
diff changeset
200 def _verifymanifest(self, mflinkrevs):
27646
8f43793382c6 verify: move manifest verification to its own function
Durham Goode <durham@fb.com>
parents: 27645
diff changeset
201 repo = self.repo
8f43793382c6 verify: move manifest verification to its own function
Durham Goode <durham@fb.com>
parents: 27645
diff changeset
202 ui = self.ui
8f43793382c6 verify: move manifest verification to its own function
Durham Goode <durham@fb.com>
parents: 27645
diff changeset
203 mf = self.repo.manifest
8f43793382c6 verify: move manifest verification to its own function
Durham Goode <durham@fb.com>
parents: 27645
diff changeset
204
27443
937e73a6e4ff verify: move verify logic into a class
Durham Goode <durham@fb.com>
parents: 26900
diff changeset
205 ui.status(_("checking manifests\n"))
27695
fb0cc863d172 verify: replace "output parameters" by return values
Martin von Zweigbergk <martinvonz@google.com>
parents: 27648
diff changeset
206 filenodes = {}
27443
937e73a6e4ff verify: move verify logic into a class
Durham Goode <durham@fb.com>
parents: 26900
diff changeset
207 seen = {}
27444
6647401858ab verify: move widely used variables into class members
Durham Goode <durham@fb.com>
parents: 27443
diff changeset
208 if self.refersmf:
27443
937e73a6e4ff verify: move verify logic into a class
Durham Goode <durham@fb.com>
parents: 26900
diff changeset
209 # Do not check manifest if there are only changelog entries with
937e73a6e4ff verify: move verify logic into a class
Durham Goode <durham@fb.com>
parents: 26900
diff changeset
210 # null manifests.
27642
f6457349985b verify: move checklog() onto class
Durham Goode <durham@fb.com>
parents: 27453
diff changeset
211 self.checklog(mf, "manifest", 0)
27443
937e73a6e4ff verify: move verify logic into a class
Durham Goode <durham@fb.com>
parents: 26900
diff changeset
212 total = len(mf)
937e73a6e4ff verify: move verify logic into a class
Durham Goode <durham@fb.com>
parents: 26900
diff changeset
213 for i in mf:
937e73a6e4ff verify: move verify logic into a class
Durham Goode <durham@fb.com>
parents: 26900
diff changeset
214 ui.progress(_('checking'), i, total=total, unit=_('manifests'))
937e73a6e4ff verify: move verify logic into a class
Durham Goode <durham@fb.com>
parents: 26900
diff changeset
215 n = mf.node(i)
27643
62ce86fcfd06 verify: move checkentry() to be a class function
Durham Goode <durham@fb.com>
parents: 27642
diff changeset
216 lr = self.checkentry(mf, i, n, seen, mflinkrevs.get(n, []),
62ce86fcfd06 verify: move checkentry() to be a class function
Durham Goode <durham@fb.com>
parents: 27642
diff changeset
217 "manifest")
27443
937e73a6e4ff verify: move verify logic into a class
Durham Goode <durham@fb.com>
parents: 26900
diff changeset
218 if n in mflinkrevs:
937e73a6e4ff verify: move verify logic into a class
Durham Goode <durham@fb.com>
parents: 26900
diff changeset
219 del mflinkrevs[n]
937e73a6e4ff verify: move verify logic into a class
Durham Goode <durham@fb.com>
parents: 26900
diff changeset
220 else:
27447
d1b91c10ce70 verify: move err() to be a class function
Durham Goode <durham@fb.com>
parents: 27446
diff changeset
221 self.err(lr, _("%s not in changesets") % short(n), "manifest")
2778
fdc232d8a193 Move repo.verify
Matt Mackall <mpm@selenic.com>
parents:
diff changeset
222
27443
937e73a6e4ff verify: move verify logic into a class
Durham Goode <durham@fb.com>
parents: 26900
diff changeset
223 try:
937e73a6e4ff verify: move verify logic into a class
Durham Goode <durham@fb.com>
parents: 26900
diff changeset
224 for f, fn in mf.readdelta(n).iteritems():
937e73a6e4ff verify: move verify logic into a class
Durham Goode <durham@fb.com>
parents: 26900
diff changeset
225 if not f:
27447
d1b91c10ce70 verify: move err() to be a class function
Durham Goode <durham@fb.com>
parents: 27446
diff changeset
226 self.err(lr, _("file without name in manifest"))
27443
937e73a6e4ff verify: move verify logic into a class
Durham Goode <durham@fb.com>
parents: 26900
diff changeset
227 elif f != "/dev/null": # ignore this in very old repos
937e73a6e4ff verify: move verify logic into a class
Durham Goode <durham@fb.com>
parents: 26900
diff changeset
228 if _validpath(repo, f):
937e73a6e4ff verify: move verify logic into a class
Durham Goode <durham@fb.com>
parents: 26900
diff changeset
229 filenodes.setdefault(
937e73a6e4ff verify: move verify logic into a class
Durham Goode <durham@fb.com>
parents: 26900
diff changeset
230 _normpath(f), {}).setdefault(fn, lr)
937e73a6e4ff verify: move verify logic into a class
Durham Goode <durham@fb.com>
parents: 26900
diff changeset
231 except Exception as inst:
27448
f4f2179077cb verify: move exc() function onto class
Durham Goode <durham@fb.com>
parents: 27447
diff changeset
232 self.exc(lr, _("reading manifest delta %s") % short(n), inst)
27443
937e73a6e4ff verify: move verify logic into a class
Durham Goode <durham@fb.com>
parents: 26900
diff changeset
233 ui.progress(_('checking'), None)
937e73a6e4ff verify: move verify logic into a class
Durham Goode <durham@fb.com>
parents: 26900
diff changeset
234
28111
06205989264b verify: move cross-checking of changeset/manifest out of _crosscheckfiles()
Martin von Zweigbergk <martinvonz@google.com>
parents: 28007
diff changeset
235 if self.havemf:
06205989264b verify: move cross-checking of changeset/manifest out of _crosscheckfiles()
Martin von Zweigbergk <martinvonz@google.com>
parents: 28007
diff changeset
236 for c, m in sorted([(c, m) for m in mflinkrevs
06205989264b verify: move cross-checking of changeset/manifest out of _crosscheckfiles()
Martin von Zweigbergk <martinvonz@google.com>
parents: 28007
diff changeset
237 for c in mflinkrevs[m]]):
06205989264b verify: move cross-checking of changeset/manifest out of _crosscheckfiles()
Martin von Zweigbergk <martinvonz@google.com>
parents: 28007
diff changeset
238 if m == nullid:
06205989264b verify: move cross-checking of changeset/manifest out of _crosscheckfiles()
Martin von Zweigbergk <martinvonz@google.com>
parents: 28007
diff changeset
239 continue
06205989264b verify: move cross-checking of changeset/manifest out of _crosscheckfiles()
Martin von Zweigbergk <martinvonz@google.com>
parents: 28007
diff changeset
240 self.err(c, _("changeset refers to unknown manifest %s") %
06205989264b verify: move cross-checking of changeset/manifest out of _crosscheckfiles()
Martin von Zweigbergk <martinvonz@google.com>
parents: 28007
diff changeset
241 short(m))
06205989264b verify: move cross-checking of changeset/manifest out of _crosscheckfiles()
Martin von Zweigbergk <martinvonz@google.com>
parents: 28007
diff changeset
242
27695
fb0cc863d172 verify: replace "output parameters" by return values
Martin von Zweigbergk <martinvonz@google.com>
parents: 27648
diff changeset
243 return filenodes
27645
df8973e1fb45 verify: move file cross checking to its own function
Durham Goode <durham@fb.com>
parents: 27644
diff changeset
244
28111
06205989264b verify: move cross-checking of changeset/manifest out of _crosscheckfiles()
Martin von Zweigbergk <martinvonz@google.com>
parents: 28007
diff changeset
245 def _crosscheckfiles(self, filelinkrevs, filenodes):
27645
df8973e1fb45 verify: move file cross checking to its own function
Durham Goode <durham@fb.com>
parents: 27644
diff changeset
246 repo = self.repo
df8973e1fb45 verify: move file cross checking to its own function
Durham Goode <durham@fb.com>
parents: 27644
diff changeset
247 ui = self.ui
27443
937e73a6e4ff verify: move verify logic into a class
Durham Goode <durham@fb.com>
parents: 26900
diff changeset
248 ui.status(_("crosschecking files in changesets and manifests\n"))
2778
fdc232d8a193 Move repo.verify
Matt Mackall <mpm@selenic.com>
parents:
diff changeset
249
28111
06205989264b verify: move cross-checking of changeset/manifest out of _crosscheckfiles()
Martin von Zweigbergk <martinvonz@google.com>
parents: 28007
diff changeset
250 total = len(filelinkrevs) + len(filenodes)
27443
937e73a6e4ff verify: move verify logic into a class
Durham Goode <durham@fb.com>
parents: 26900
diff changeset
251 count = 0
27645
df8973e1fb45 verify: move file cross checking to its own function
Durham Goode <durham@fb.com>
parents: 27644
diff changeset
252 if self.havemf:
27443
937e73a6e4ff verify: move verify logic into a class
Durham Goode <durham@fb.com>
parents: 26900
diff changeset
253 for f in sorted(filelinkrevs):
937e73a6e4ff verify: move verify logic into a class
Durham Goode <durham@fb.com>
parents: 26900
diff changeset
254 count += 1
937e73a6e4ff verify: move verify logic into a class
Durham Goode <durham@fb.com>
parents: 26900
diff changeset
255 ui.progress(_('crosschecking'), count, total=total)
937e73a6e4ff verify: move verify logic into a class
Durham Goode <durham@fb.com>
parents: 26900
diff changeset
256 if f not in filenodes:
937e73a6e4ff verify: move verify logic into a class
Durham Goode <durham@fb.com>
parents: 26900
diff changeset
257 lr = filelinkrevs[f][0]
27447
d1b91c10ce70 verify: move err() to be a class function
Durham Goode <durham@fb.com>
parents: 27446
diff changeset
258 self.err(lr, _("in changeset but not in manifest"), f)
6892
dab95717058d verify: check repo.store
Adrian Buehlmann <adrian@cadifra.com>
parents: 6889
diff changeset
259
27645
df8973e1fb45 verify: move file cross checking to its own function
Durham Goode <durham@fb.com>
parents: 27644
diff changeset
260 if self.havecl:
27443
937e73a6e4ff verify: move verify logic into a class
Durham Goode <durham@fb.com>
parents: 26900
diff changeset
261 for f in sorted(filenodes):
937e73a6e4ff verify: move verify logic into a class
Durham Goode <durham@fb.com>
parents: 26900
diff changeset
262 count += 1
937e73a6e4ff verify: move verify logic into a class
Durham Goode <durham@fb.com>
parents: 26900
diff changeset
263 ui.progress(_('crosschecking'), count, total=total)
937e73a6e4ff verify: move verify logic into a class
Durham Goode <durham@fb.com>
parents: 26900
diff changeset
264 if f not in filelinkrevs:
937e73a6e4ff verify: move verify logic into a class
Durham Goode <durham@fb.com>
parents: 26900
diff changeset
265 try:
937e73a6e4ff verify: move verify logic into a class
Durham Goode <durham@fb.com>
parents: 26900
diff changeset
266 fl = repo.file(f)
937e73a6e4ff verify: move verify logic into a class
Durham Goode <durham@fb.com>
parents: 26900
diff changeset
267 lr = min([fl.linkrev(fl.rev(n)) for n in filenodes[f]])
937e73a6e4ff verify: move verify logic into a class
Durham Goode <durham@fb.com>
parents: 26900
diff changeset
268 except Exception:
937e73a6e4ff verify: move verify logic into a class
Durham Goode <durham@fb.com>
parents: 26900
diff changeset
269 lr = None
27447
d1b91c10ce70 verify: move err() to be a class function
Durham Goode <durham@fb.com>
parents: 27446
diff changeset
270 self.err(lr, _("in manifest but not in changeset"), f)
27443
937e73a6e4ff verify: move verify logic into a class
Durham Goode <durham@fb.com>
parents: 26900
diff changeset
271
937e73a6e4ff verify: move verify logic into a class
Durham Goode <durham@fb.com>
parents: 26900
diff changeset
272 ui.progress(_('crosschecking'), None)
8291
f5c1a9094e41 verify: avoid exception on missing file revlog
Henrik Stuart <hg@hstuart.dk>
parents: 8225
diff changeset
273
27644
331e5c28f5f0 verify: move filelog verification to its own function
Durham Goode <durham@fb.com>
parents: 27643
diff changeset
274 def _verifyfiles(self, filenodes, filelinkrevs):
331e5c28f5f0 verify: move filelog verification to its own function
Durham Goode <durham@fb.com>
parents: 27643
diff changeset
275 repo = self.repo
331e5c28f5f0 verify: move filelog verification to its own function
Durham Goode <durham@fb.com>
parents: 27643
diff changeset
276 ui = self.ui
331e5c28f5f0 verify: move filelog verification to its own function
Durham Goode <durham@fb.com>
parents: 27643
diff changeset
277 lrugetctx = self.lrugetctx
331e5c28f5f0 verify: move filelog verification to its own function
Durham Goode <durham@fb.com>
parents: 27643
diff changeset
278 revlogv1 = self.revlogv1
331e5c28f5f0 verify: move filelog verification to its own function
Durham Goode <durham@fb.com>
parents: 27643
diff changeset
279 havemf = self.havemf
27443
937e73a6e4ff verify: move verify logic into a class
Durham Goode <durham@fb.com>
parents: 26900
diff changeset
280 ui.status(_("checking files\n"))
8291
f5c1a9094e41 verify: avoid exception on missing file revlog
Henrik Stuart <hg@hstuart.dk>
parents: 8225
diff changeset
281
27443
937e73a6e4ff verify: move verify logic into a class
Durham Goode <durham@fb.com>
parents: 26900
diff changeset
282 storefiles = set()
937e73a6e4ff verify: move verify logic into a class
Durham Goode <durham@fb.com>
parents: 26900
diff changeset
283 for f, f2, size in repo.store.datafiles():
937e73a6e4ff verify: move verify logic into a class
Durham Goode <durham@fb.com>
parents: 26900
diff changeset
284 if not f:
27447
d1b91c10ce70 verify: move err() to be a class function
Durham Goode <durham@fb.com>
parents: 27446
diff changeset
285 self.err(None, _("cannot decode filename '%s'") % f2)
28007
fb92927f9775 treemanifests: fix streaming clone
Martin von Zweigbergk <martinvonz@google.com>
parents: 27964
diff changeset
286 elif (size > 0 or not revlogv1) and f.startswith('data/'):
27443
937e73a6e4ff verify: move verify logic into a class
Durham Goode <durham@fb.com>
parents: 26900
diff changeset
287 storefiles.add(_normpath(f))
6892
dab95717058d verify: check repo.store
Adrian Buehlmann <adrian@cadifra.com>
parents: 6889
diff changeset
288
27443
937e73a6e4ff verify: move verify logic into a class
Durham Goode <durham@fb.com>
parents: 26900
diff changeset
289 files = sorted(set(filenodes) | set(filelinkrevs))
937e73a6e4ff verify: move verify logic into a class
Durham Goode <durham@fb.com>
parents: 26900
diff changeset
290 total = len(files)
27644
331e5c28f5f0 verify: move filelog verification to its own function
Durham Goode <durham@fb.com>
parents: 27643
diff changeset
291 revisions = 0
27443
937e73a6e4ff verify: move verify logic into a class
Durham Goode <durham@fb.com>
parents: 26900
diff changeset
292 for i, f in enumerate(files):
937e73a6e4ff verify: move verify logic into a class
Durham Goode <durham@fb.com>
parents: 26900
diff changeset
293 ui.progress(_('checking'), i, item=f, total=total)
6892
dab95717058d verify: check repo.store
Adrian Buehlmann <adrian@cadifra.com>
parents: 6889
diff changeset
294 try:
27443
937e73a6e4ff verify: move verify logic into a class
Durham Goode <durham@fb.com>
parents: 26900
diff changeset
295 linkrevs = filelinkrevs[f]
6892
dab95717058d verify: check repo.store
Adrian Buehlmann <adrian@cadifra.com>
parents: 6889
diff changeset
296 except KeyError:
27443
937e73a6e4ff verify: move verify logic into a class
Durham Goode <durham@fb.com>
parents: 26900
diff changeset
297 # in manifest but not in changelog
937e73a6e4ff verify: move verify logic into a class
Durham Goode <durham@fb.com>
parents: 26900
diff changeset
298 linkrevs = []
2778
fdc232d8a193 Move repo.verify
Matt Mackall <mpm@selenic.com>
parents:
diff changeset
299
27443
937e73a6e4ff verify: move verify logic into a class
Durham Goode <durham@fb.com>
parents: 26900
diff changeset
300 if linkrevs:
937e73a6e4ff verify: move verify logic into a class
Durham Goode <durham@fb.com>
parents: 26900
diff changeset
301 lr = linkrevs[0]
937e73a6e4ff verify: move verify logic into a class
Durham Goode <durham@fb.com>
parents: 26900
diff changeset
302 else:
937e73a6e4ff verify: move verify logic into a class
Durham Goode <durham@fb.com>
parents: 26900
diff changeset
303 lr = None
2778
fdc232d8a193 Move repo.verify
Matt Mackall <mpm@selenic.com>
parents:
diff changeset
304
3744
d626fc9e3985 verify: add rename link checking
Matt Mackall <mpm@selenic.com>
parents: 3473
diff changeset
305 try:
27443
937e73a6e4ff verify: move verify logic into a class
Durham Goode <durham@fb.com>
parents: 26900
diff changeset
306 fl = repo.file(f)
937e73a6e4ff verify: move verify logic into a class
Durham Goode <durham@fb.com>
parents: 26900
diff changeset
307 except error.RevlogError as e:
27447
d1b91c10ce70 verify: move err() to be a class function
Durham Goode <durham@fb.com>
parents: 27446
diff changeset
308 self.err(lr, _("broken revlog! (%s)") % e, f)
27443
937e73a6e4ff verify: move verify logic into a class
Durham Goode <durham@fb.com>
parents: 26900
diff changeset
309 continue
937e73a6e4ff verify: move verify logic into a class
Durham Goode <durham@fb.com>
parents: 26900
diff changeset
310
937e73a6e4ff verify: move verify logic into a class
Durham Goode <durham@fb.com>
parents: 26900
diff changeset
311 for ff in fl.files():
937e73a6e4ff verify: move verify logic into a class
Durham Goode <durham@fb.com>
parents: 26900
diff changeset
312 try:
937e73a6e4ff verify: move verify logic into a class
Durham Goode <durham@fb.com>
parents: 26900
diff changeset
313 storefiles.remove(ff)
937e73a6e4ff verify: move verify logic into a class
Durham Goode <durham@fb.com>
parents: 26900
diff changeset
314 except KeyError:
27446
6b2c1a1871a6 verify: move warn() to a class level function
Durham Goode <durham@fb.com>
parents: 27445
diff changeset
315 self.warn(_(" warning: revlog '%s' not in fncache!") % ff)
27445
cc178057ab49 verify: move fncachewarned up to a class variable
Durham Goode <durham@fb.com>
parents: 27444
diff changeset
316 self.fncachewarned = True
27443
937e73a6e4ff verify: move verify logic into a class
Durham Goode <durham@fb.com>
parents: 26900
diff changeset
317
27642
f6457349985b verify: move checklog() onto class
Durham Goode <durham@fb.com>
parents: 27453
diff changeset
318 self.checklog(fl, f, lr)
27443
937e73a6e4ff verify: move verify logic into a class
Durham Goode <durham@fb.com>
parents: 26900
diff changeset
319 seen = {}
937e73a6e4ff verify: move verify logic into a class
Durham Goode <durham@fb.com>
parents: 26900
diff changeset
320 rp = None
937e73a6e4ff verify: move verify logic into a class
Durham Goode <durham@fb.com>
parents: 26900
diff changeset
321 for i in fl:
937e73a6e4ff verify: move verify logic into a class
Durham Goode <durham@fb.com>
parents: 26900
diff changeset
322 revisions += 1
937e73a6e4ff verify: move verify logic into a class
Durham Goode <durham@fb.com>
parents: 26900
diff changeset
323 n = fl.node(i)
27643
62ce86fcfd06 verify: move checkentry() to be a class function
Durham Goode <durham@fb.com>
parents: 27642
diff changeset
324 lr = self.checkentry(fl, i, n, seen, linkrevs, f)
27443
937e73a6e4ff verify: move verify logic into a class
Durham Goode <durham@fb.com>
parents: 26900
diff changeset
325 if f in filenodes:
937e73a6e4ff verify: move verify logic into a class
Durham Goode <durham@fb.com>
parents: 26900
diff changeset
326 if havemf and n not in filenodes[f]:
27447
d1b91c10ce70 verify: move err() to be a class function
Durham Goode <durham@fb.com>
parents: 27446
diff changeset
327 self.err(lr, _("%s not in manifests") % (short(n)), f)
6534
9b35a9f34675 verify: check copy source revlog and nodeid
Patrick Mezard <pmezard@gmail.com>
parents: 6211
diff changeset
328 else:
27443
937e73a6e4ff verify: move verify logic into a class
Durham Goode <durham@fb.com>
parents: 26900
diff changeset
329 del filenodes[f][n]
937e73a6e4ff verify: move verify logic into a class
Durham Goode <durham@fb.com>
parents: 26900
diff changeset
330
937e73a6e4ff verify: move verify logic into a class
Durham Goode <durham@fb.com>
parents: 26900
diff changeset
331 # verify contents
937e73a6e4ff verify: move verify logic into a class
Durham Goode <durham@fb.com>
parents: 26900
diff changeset
332 try:
937e73a6e4ff verify: move verify logic into a class
Durham Goode <durham@fb.com>
parents: 26900
diff changeset
333 l = len(fl.read(n))
937e73a6e4ff verify: move verify logic into a class
Durham Goode <durham@fb.com>
parents: 26900
diff changeset
334 rp = fl.renamed(n)
937e73a6e4ff verify: move verify logic into a class
Durham Goode <durham@fb.com>
parents: 26900
diff changeset
335 if l != fl.size(i):
937e73a6e4ff verify: move verify logic into a class
Durham Goode <durham@fb.com>
parents: 26900
diff changeset
336 if len(fl.revision(n)) != fl.size(i):
27447
d1b91c10ce70 verify: move err() to be a class function
Durham Goode <durham@fb.com>
parents: 27446
diff changeset
337 self.err(lr, _("unpacked size is %s, %s expected") %
d1b91c10ce70 verify: move err() to be a class function
Durham Goode <durham@fb.com>
parents: 27446
diff changeset
338 (l, fl.size(i)), f)
27443
937e73a6e4ff verify: move verify logic into a class
Durham Goode <durham@fb.com>
parents: 26900
diff changeset
339 except error.CensoredNodeError:
937e73a6e4ff verify: move verify logic into a class
Durham Goode <durham@fb.com>
parents: 26900
diff changeset
340 # experimental config: censor.policy
937e73a6e4ff verify: move verify logic into a class
Durham Goode <durham@fb.com>
parents: 26900
diff changeset
341 if ui.config("censor", "policy", "abort") == "abort":
27447
d1b91c10ce70 verify: move err() to be a class function
Durham Goode <durham@fb.com>
parents: 27446
diff changeset
342 self.err(lr, _("censored file data"), f)
27443
937e73a6e4ff verify: move verify logic into a class
Durham Goode <durham@fb.com>
parents: 26900
diff changeset
343 except Exception as inst:
27448
f4f2179077cb verify: move exc() function onto class
Durham Goode <durham@fb.com>
parents: 27447
diff changeset
344 self.exc(lr, _("unpacking %s") % short(n), inst, f)
3744
d626fc9e3985 verify: add rename link checking
Matt Mackall <mpm@selenic.com>
parents: 3473
diff changeset
345
27443
937e73a6e4ff verify: move verify logic into a class
Durham Goode <durham@fb.com>
parents: 26900
diff changeset
346 # check renames
937e73a6e4ff verify: move verify logic into a class
Durham Goode <durham@fb.com>
parents: 26900
diff changeset
347 try:
937e73a6e4ff verify: move verify logic into a class
Durham Goode <durham@fb.com>
parents: 26900
diff changeset
348 if rp:
937e73a6e4ff verify: move verify logic into a class
Durham Goode <durham@fb.com>
parents: 26900
diff changeset
349 if lr is not None and ui.verbose:
937e73a6e4ff verify: move verify logic into a class
Durham Goode <durham@fb.com>
parents: 26900
diff changeset
350 ctx = lrugetctx(lr)
937e73a6e4ff verify: move verify logic into a class
Durham Goode <durham@fb.com>
parents: 26900
diff changeset
351 found = False
937e73a6e4ff verify: move verify logic into a class
Durham Goode <durham@fb.com>
parents: 26900
diff changeset
352 for pctx in ctx.parents():
937e73a6e4ff verify: move verify logic into a class
Durham Goode <durham@fb.com>
parents: 26900
diff changeset
353 if rp[0] in pctx:
937e73a6e4ff verify: move verify logic into a class
Durham Goode <durham@fb.com>
parents: 26900
diff changeset
354 found = True
937e73a6e4ff verify: move verify logic into a class
Durham Goode <durham@fb.com>
parents: 26900
diff changeset
355 break
937e73a6e4ff verify: move verify logic into a class
Durham Goode <durham@fb.com>
parents: 26900
diff changeset
356 if not found:
27446
6b2c1a1871a6 verify: move warn() to a class level function
Durham Goode <durham@fb.com>
parents: 27445
diff changeset
357 self.warn(_("warning: copy source of '%s' not"
6b2c1a1871a6 verify: move warn() to a class level function
Durham Goode <durham@fb.com>
parents: 27445
diff changeset
358 " in parents of %s") % (f, ctx))
27443
937e73a6e4ff verify: move verify logic into a class
Durham Goode <durham@fb.com>
parents: 26900
diff changeset
359 fl2 = repo.file(rp[0])
937e73a6e4ff verify: move verify logic into a class
Durham Goode <durham@fb.com>
parents: 26900
diff changeset
360 if not len(fl2):
27447
d1b91c10ce70 verify: move err() to be a class function
Durham Goode <durham@fb.com>
parents: 27446
diff changeset
361 self.err(lr, _("empty or missing copy source "
d1b91c10ce70 verify: move err() to be a class function
Durham Goode <durham@fb.com>
parents: 27446
diff changeset
362 "revlog %s:%s") % (rp[0], short(rp[1])), f)
27443
937e73a6e4ff verify: move verify logic into a class
Durham Goode <durham@fb.com>
parents: 26900
diff changeset
363 elif rp[1] == nullid:
937e73a6e4ff verify: move verify logic into a class
Durham Goode <durham@fb.com>
parents: 26900
diff changeset
364 ui.note(_("warning: %s@%s: copy source"
937e73a6e4ff verify: move verify logic into a class
Durham Goode <durham@fb.com>
parents: 26900
diff changeset
365 " revision is nullid %s:%s\n")
937e73a6e4ff verify: move verify logic into a class
Durham Goode <durham@fb.com>
parents: 26900
diff changeset
366 % (f, lr, rp[0], short(rp[1])))
937e73a6e4ff verify: move verify logic into a class
Durham Goode <durham@fb.com>
parents: 26900
diff changeset
367 else:
937e73a6e4ff verify: move verify logic into a class
Durham Goode <durham@fb.com>
parents: 26900
diff changeset
368 fl2.rev(rp[1])
937e73a6e4ff verify: move verify logic into a class
Durham Goode <durham@fb.com>
parents: 26900
diff changeset
369 except Exception as inst:
27448
f4f2179077cb verify: move exc() function onto class
Durham Goode <durham@fb.com>
parents: 27447
diff changeset
370 self.exc(lr, _("checking rename of %s") % short(n), inst, f)
6892
dab95717058d verify: check repo.store
Adrian Buehlmann <adrian@cadifra.com>
parents: 6889
diff changeset
371
27443
937e73a6e4ff verify: move verify logic into a class
Durham Goode <durham@fb.com>
parents: 26900
diff changeset
372 # cross-check
937e73a6e4ff verify: move verify logic into a class
Durham Goode <durham@fb.com>
parents: 26900
diff changeset
373 if f in filenodes:
937e73a6e4ff verify: move verify logic into a class
Durham Goode <durham@fb.com>
parents: 26900
diff changeset
374 fns = [(lr, n) for n, lr in filenodes[f].iteritems()]
937e73a6e4ff verify: move verify logic into a class
Durham Goode <durham@fb.com>
parents: 26900
diff changeset
375 for lr, node in sorted(fns):
27447
d1b91c10ce70 verify: move err() to be a class function
Durham Goode <durham@fb.com>
parents: 27446
diff changeset
376 self.err(lr, _("%s in manifests not found") % short(node),
d1b91c10ce70 verify: move err() to be a class function
Durham Goode <durham@fb.com>
parents: 27446
diff changeset
377 f)
27443
937e73a6e4ff verify: move verify logic into a class
Durham Goode <durham@fb.com>
parents: 26900
diff changeset
378 ui.progress(_('checking'), None)
937e73a6e4ff verify: move verify logic into a class
Durham Goode <durham@fb.com>
parents: 26900
diff changeset
379
937e73a6e4ff verify: move verify logic into a class
Durham Goode <durham@fb.com>
parents: 26900
diff changeset
380 for f in storefiles:
27446
6b2c1a1871a6 verify: move warn() to a class level function
Durham Goode <durham@fb.com>
parents: 27445
diff changeset
381 self.warn(_("warning: orphan revlog '%s'") % f)
27443
937e73a6e4ff verify: move verify logic into a class
Durham Goode <durham@fb.com>
parents: 26900
diff changeset
382
27644
331e5c28f5f0 verify: move filelog verification to its own function
Durham Goode <durham@fb.com>
parents: 27643
diff changeset
383 return len(files), revisions