Mercurial > public > mercurial-scm > hg-stable

annotate tests/test-storage.py @ 40052:cdf61ab1f54c

Find changesets by keywords (author, files, the commit message), revision number or hash, or revset expression.
testing: add file storage integration for bad hashes and censoring In order to implement these tests, we need a backdoor to write data into storage backends while bypassing normal checks. We invent a callable to do that. As part of writing the tests, I found a bug with censorrevision() pretty quickly! After calling censorrevision(), attempting to access revision data for an affected node raises a cryptic error related to malformed compression. This appears to be due to the revlog not adjusting delta chains as part of censoring. I also found a bug with regards to hash verification and revision fulltext caching. Essentially, we cache the fulltext before hash verification. If we look up the fulltext after a failed hash verification, we don't get a hash verification exception. Furthermore, the behavior of revision(raw=True) can be inconsistent depending on the order of operations. I'll be fixing both these bugs in subsequent commits. Differential Revision: https://phab.mercurial-scm.org/D4865
author Gregory Szorc <gregory.szorc@gmail.com>
date Wed, 03 Oct 2018 10:56:48 -0700
parents a3a9b93bff80
children 324b4b10351e
Ignore whitespace changes - Everywhere: Within whitespace: At end of lines:
rev   line source
39788
ae531f5e583c testing: add interface unit tests for file storage
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff changeset
1 # This test verifies the conformance of various classes to various
ae531f5e583c testing: add interface unit tests for file storage
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff changeset
2 # storage interfaces.
ae531f5e583c testing: add interface unit tests for file storage
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff changeset
3 from __future__ import absolute_import
ae531f5e583c testing: add interface unit tests for file storage
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff changeset
4
ae531f5e583c testing: add interface unit tests for file storage
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff changeset
5 import silenttestrunner
ae531f5e583c testing: add interface unit tests for file storage
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff changeset
6
ae531f5e583c testing: add interface unit tests for file storage
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff changeset
7 from mercurial import (
40052
cdf61ab1f54c testing: add file storage integration for bad hashes and censoring
Gregory Szorc <gregory.szorc@gmail.com>
parents: 39954
diff changeset
8 error,
39788
ae531f5e583c testing: add interface unit tests for file storage
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff changeset
9 filelog,
40052
cdf61ab1f54c testing: add file storage integration for bad hashes and censoring
Gregory Szorc <gregory.szorc@gmail.com>
parents: 39954
diff changeset
10 revlog,
39788
ae531f5e583c testing: add interface unit tests for file storage
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff changeset
11 transaction,
ae531f5e583c testing: add interface unit tests for file storage
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff changeset
12 ui as uimod,
ae531f5e583c testing: add interface unit tests for file storage
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff changeset
13 vfs as vfsmod,
ae531f5e583c testing: add interface unit tests for file storage
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff changeset
14 )
ae531f5e583c testing: add interface unit tests for file storage
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff changeset
15
ae531f5e583c testing: add interface unit tests for file storage
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff changeset
16 from mercurial.testing import (
ae531f5e583c testing: add interface unit tests for file storage
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff changeset
17 storage as storagetesting,
ae531f5e583c testing: add interface unit tests for file storage
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff changeset
18 )
ae531f5e583c testing: add interface unit tests for file storage
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff changeset
19
ae531f5e583c testing: add interface unit tests for file storage
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff changeset
20 STATE = {
ae531f5e583c testing: add interface unit tests for file storage
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff changeset
21 'lastindex': 0,
ae531f5e583c testing: add interface unit tests for file storage
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff changeset
22 'ui': uimod.ui(),
ae531f5e583c testing: add interface unit tests for file storage
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff changeset
23 'vfs': vfsmod.vfs(b'.', realpath=True),
ae531f5e583c testing: add interface unit tests for file storage
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff changeset
24 }
ae531f5e583c testing: add interface unit tests for file storage
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff changeset
25
ae531f5e583c testing: add interface unit tests for file storage
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff changeset
26 def makefilefn(self):
ae531f5e583c testing: add interface unit tests for file storage
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff changeset
27 """Factory for filelog instances."""
39954
a3a9b93bff80 py3: byteify test-storage.py
Gregory Szorc <gregory.szorc@gmail.com>
parents: 39788
diff changeset
28 fl = filelog.filelog(STATE['vfs'], b'filelog-%d' % STATE['lastindex'])
39788
ae531f5e583c testing: add interface unit tests for file storage
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff changeset
29 STATE['lastindex'] += 1
ae531f5e583c testing: add interface unit tests for file storage
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff changeset
30 return fl
ae531f5e583c testing: add interface unit tests for file storage
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff changeset
31
ae531f5e583c testing: add interface unit tests for file storage
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff changeset
32 def maketransaction(self):
ae531f5e583c testing: add interface unit tests for file storage
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff changeset
33 vfsmap = {'plain': STATE['vfs']}
ae531f5e583c testing: add interface unit tests for file storage
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff changeset
34
ae531f5e583c testing: add interface unit tests for file storage
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff changeset
35 return transaction.transaction(STATE['ui'].warn, STATE['vfs'], vfsmap,
39954
a3a9b93bff80 py3: byteify test-storage.py
Gregory Szorc <gregory.szorc@gmail.com>
parents: 39788
diff changeset
36 b'journal', b'undo')
39788
ae531f5e583c testing: add interface unit tests for file storage
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff changeset
37
40052
cdf61ab1f54c testing: add file storage integration for bad hashes and censoring
Gregory Szorc <gregory.szorc@gmail.com>
parents: 39954
diff changeset
38 def addrawrevision(self, fl, tr, node, p1, p2, linkrev, rawtext=None,
cdf61ab1f54c testing: add file storage integration for bad hashes and censoring
Gregory Szorc <gregory.szorc@gmail.com>
parents: 39954
diff changeset
39 delta=None, censored=False, ellipsis=False, extstored=False):
cdf61ab1f54c testing: add file storage integration for bad hashes and censoring
Gregory Szorc <gregory.szorc@gmail.com>
parents: 39954
diff changeset
40 flags = 0
cdf61ab1f54c testing: add file storage integration for bad hashes and censoring
Gregory Szorc <gregory.szorc@gmail.com>
parents: 39954
diff changeset
41
cdf61ab1f54c testing: add file storage integration for bad hashes and censoring
Gregory Szorc <gregory.szorc@gmail.com>
parents: 39954
diff changeset
42 if censored:
cdf61ab1f54c testing: add file storage integration for bad hashes and censoring
Gregory Szorc <gregory.szorc@gmail.com>
parents: 39954
diff changeset
43 flags |= revlog.REVIDX_ISCENSORED
cdf61ab1f54c testing: add file storage integration for bad hashes and censoring
Gregory Szorc <gregory.szorc@gmail.com>
parents: 39954
diff changeset
44 if ellipsis:
cdf61ab1f54c testing: add file storage integration for bad hashes and censoring
Gregory Szorc <gregory.szorc@gmail.com>
parents: 39954
diff changeset
45 flags |= revlog.REVIDX_ELLIPSIS
cdf61ab1f54c testing: add file storage integration for bad hashes and censoring
Gregory Szorc <gregory.szorc@gmail.com>
parents: 39954
diff changeset
46 if extstored:
cdf61ab1f54c testing: add file storage integration for bad hashes and censoring
Gregory Szorc <gregory.szorc@gmail.com>
parents: 39954
diff changeset
47 flags |= revlog.REVIDX_EXTSTORED
cdf61ab1f54c testing: add file storage integration for bad hashes and censoring
Gregory Szorc <gregory.szorc@gmail.com>
parents: 39954
diff changeset
48
cdf61ab1f54c testing: add file storage integration for bad hashes and censoring
Gregory Szorc <gregory.szorc@gmail.com>
parents: 39954
diff changeset
49 if rawtext is not None:
cdf61ab1f54c testing: add file storage integration for bad hashes and censoring
Gregory Szorc <gregory.szorc@gmail.com>
parents: 39954
diff changeset
50 fl._revlog.addrawrevision(rawtext, tr, linkrev, p1, p2, node, flags)
cdf61ab1f54c testing: add file storage integration for bad hashes and censoring
Gregory Szorc <gregory.szorc@gmail.com>
parents: 39954
diff changeset
51 elif delta is not None:
cdf61ab1f54c testing: add file storage integration for bad hashes and censoring
Gregory Szorc <gregory.szorc@gmail.com>
parents: 39954
diff changeset
52 raise error.Abort('support for storing raw deltas not yet supported')
cdf61ab1f54c testing: add file storage integration for bad hashes and censoring
Gregory Szorc <gregory.szorc@gmail.com>
parents: 39954
diff changeset
53 else:
cdf61ab1f54c testing: add file storage integration for bad hashes and censoring
Gregory Szorc <gregory.szorc@gmail.com>
parents: 39954
diff changeset
54 raise error.Abort('must supply rawtext or delta arguments')
cdf61ab1f54c testing: add file storage integration for bad hashes and censoring
Gregory Szorc <gregory.szorc@gmail.com>
parents: 39954
diff changeset
55
cdf61ab1f54c testing: add file storage integration for bad hashes and censoring
Gregory Szorc <gregory.szorc@gmail.com>
parents: 39954
diff changeset
56 # We may insert bad data. Clear caches to prevent e.g. cache hits to
cdf61ab1f54c testing: add file storage integration for bad hashes and censoring
Gregory Szorc <gregory.szorc@gmail.com>
parents: 39954
diff changeset
57 # bypass hash verification.
cdf61ab1f54c testing: add file storage integration for bad hashes and censoring
Gregory Szorc <gregory.szorc@gmail.com>
parents: 39954
diff changeset
58 fl._revlog.clearcaches()
cdf61ab1f54c testing: add file storage integration for bad hashes and censoring
Gregory Szorc <gregory.szorc@gmail.com>
parents: 39954
diff changeset
59
39788
ae531f5e583c testing: add interface unit tests for file storage
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff changeset
60 # Assigning module-level attributes that inherit from unittest.TestCase
ae531f5e583c testing: add interface unit tests for file storage
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff changeset
61 # is all that is needed to register tests.
ae531f5e583c testing: add interface unit tests for file storage
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff changeset
62 filelogindextests = storagetesting.makeifileindextests(makefilefn,
40052
cdf61ab1f54c testing: add file storage integration for bad hashes and censoring
Gregory Szorc <gregory.szorc@gmail.com>
parents: 39954
diff changeset
63 maketransaction,
cdf61ab1f54c testing: add file storage integration for bad hashes and censoring
Gregory Szorc <gregory.szorc@gmail.com>
parents: 39954
diff changeset
64 addrawrevision)
39788
ae531f5e583c testing: add interface unit tests for file storage
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff changeset
65 filelogdatatests = storagetesting.makeifiledatatests(makefilefn,
40052
cdf61ab1f54c testing: add file storage integration for bad hashes and censoring
Gregory Szorc <gregory.szorc@gmail.com>
parents: 39954
diff changeset
66 maketransaction,
cdf61ab1f54c testing: add file storage integration for bad hashes and censoring
Gregory Szorc <gregory.szorc@gmail.com>
parents: 39954
diff changeset
67 addrawrevision)
39788
ae531f5e583c testing: add interface unit tests for file storage
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff changeset
68 filelogmutationtests = storagetesting.makeifilemutationtests(makefilefn,
40052
cdf61ab1f54c testing: add file storage integration for bad hashes and censoring
Gregory Szorc <gregory.szorc@gmail.com>
parents: 39954
diff changeset
69 maketransaction,
cdf61ab1f54c testing: add file storage integration for bad hashes and censoring
Gregory Szorc <gregory.szorc@gmail.com>
parents: 39954
diff changeset
70 addrawrevision)
39788
ae531f5e583c testing: add interface unit tests for file storage
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff changeset
71
ae531f5e583c testing: add interface unit tests for file storage
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff changeset
72 if __name__ == '__main__':
ae531f5e583c testing: add interface unit tests for file storage
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff changeset
73 silenttestrunner.main(__name__)