Mercurial > public > mercurial-scm > hg-stable
annotate tests/test-verify-repo-operations.py @ 28255:f75f7d39cca3
testing: generate tests operations using Hypothesis
The idea of this patch is to expand the use of Hypothesis
within Mercurial to use its concept of "stateful testing".
The result is a test which runs a sequence of operations
against a Mercurial repository. Each operation is given a
set of allowed ways it can fail. Any other non-zero exit
code is a test failure.
At the end, the whole sequence is then reverified by
generating a .t test and testing it again in pure
mode (this is also useful for catching non-determinism
bugs).
This has proven reasonably effective at finding bugs,
and has identified two problems in the shelve extension
already (issue5113 and issue5112).
author | David R. MacIver <david@drmaciver.com> |
---|---|
date | Wed, 24 Feb 2016 13:05:45 +0000 |
parents | |
children | 55325bdf6c13 |
rev | line source |
---|---|
28255
f75f7d39cca3
testing: generate tests operations using Hypothesis
David R. MacIver <david@drmaciver.com>
parents:
diff
changeset
|
1 from __future__ import print_function, absolute_import |
f75f7d39cca3
testing: generate tests operations using Hypothesis
David R. MacIver <david@drmaciver.com>
parents:
diff
changeset
|
2 |
f75f7d39cca3
testing: generate tests operations using Hypothesis
David R. MacIver <david@drmaciver.com>
parents:
diff
changeset
|
3 """Fuzz testing for operations against a Mercurial repository |
f75f7d39cca3
testing: generate tests operations using Hypothesis
David R. MacIver <david@drmaciver.com>
parents:
diff
changeset
|
4 |
f75f7d39cca3
testing: generate tests operations using Hypothesis
David R. MacIver <david@drmaciver.com>
parents:
diff
changeset
|
5 This uses Hypothesis's stateful testing to generate random repository |
f75f7d39cca3
testing: generate tests operations using Hypothesis
David R. MacIver <david@drmaciver.com>
parents:
diff
changeset
|
6 operations and test Mercurial using them, both to see if there are any |
f75f7d39cca3
testing: generate tests operations using Hypothesis
David R. MacIver <david@drmaciver.com>
parents:
diff
changeset
|
7 unexpected errors and to compare different versions of it.""" |
f75f7d39cca3
testing: generate tests operations using Hypothesis
David R. MacIver <david@drmaciver.com>
parents:
diff
changeset
|
8 |
f75f7d39cca3
testing: generate tests operations using Hypothesis
David R. MacIver <david@drmaciver.com>
parents:
diff
changeset
|
9 import os |
f75f7d39cca3
testing: generate tests operations using Hypothesis
David R. MacIver <david@drmaciver.com>
parents:
diff
changeset
|
10 import sys |
f75f7d39cca3
testing: generate tests operations using Hypothesis
David R. MacIver <david@drmaciver.com>
parents:
diff
changeset
|
11 |
f75f7d39cca3
testing: generate tests operations using Hypothesis
David R. MacIver <david@drmaciver.com>
parents:
diff
changeset
|
12 # These tests require Hypothesis and pytz to be installed. |
f75f7d39cca3
testing: generate tests operations using Hypothesis
David R. MacIver <david@drmaciver.com>
parents:
diff
changeset
|
13 # Running 'pip install hypothesis pytz' will achieve that. |
f75f7d39cca3
testing: generate tests operations using Hypothesis
David R. MacIver <david@drmaciver.com>
parents:
diff
changeset
|
14 # Note: This won't work if you're running Python < 2.7. |
f75f7d39cca3
testing: generate tests operations using Hypothesis
David R. MacIver <david@drmaciver.com>
parents:
diff
changeset
|
15 try: |
f75f7d39cca3
testing: generate tests operations using Hypothesis
David R. MacIver <david@drmaciver.com>
parents:
diff
changeset
|
16 from hypothesis.extra.datetime import datetimes |
f75f7d39cca3
testing: generate tests operations using Hypothesis
David R. MacIver <david@drmaciver.com>
parents:
diff
changeset
|
17 except ImportError: |
f75f7d39cca3
testing: generate tests operations using Hypothesis
David R. MacIver <david@drmaciver.com>
parents:
diff
changeset
|
18 sys.stderr.write("skipped: hypothesis or pytz not installed" + os.linesep) |
f75f7d39cca3
testing: generate tests operations using Hypothesis
David R. MacIver <david@drmaciver.com>
parents:
diff
changeset
|
19 sys.exit(80) |
f75f7d39cca3
testing: generate tests operations using Hypothesis
David R. MacIver <david@drmaciver.com>
parents:
diff
changeset
|
20 |
f75f7d39cca3
testing: generate tests operations using Hypothesis
David R. MacIver <david@drmaciver.com>
parents:
diff
changeset
|
21 # If you are running an old version of pip you may find that the enum34 |
f75f7d39cca3
testing: generate tests operations using Hypothesis
David R. MacIver <david@drmaciver.com>
parents:
diff
changeset
|
22 # backport is not installed automatically. If so 'pip install enum34' will |
f75f7d39cca3
testing: generate tests operations using Hypothesis
David R. MacIver <david@drmaciver.com>
parents:
diff
changeset
|
23 # fix this problem. |
f75f7d39cca3
testing: generate tests operations using Hypothesis
David R. MacIver <david@drmaciver.com>
parents:
diff
changeset
|
24 try: |
f75f7d39cca3
testing: generate tests operations using Hypothesis
David R. MacIver <david@drmaciver.com>
parents:
diff
changeset
|
25 import enum |
f75f7d39cca3
testing: generate tests operations using Hypothesis
David R. MacIver <david@drmaciver.com>
parents:
diff
changeset
|
26 assert enum # Silence pyflakes |
f75f7d39cca3
testing: generate tests operations using Hypothesis
David R. MacIver <david@drmaciver.com>
parents:
diff
changeset
|
27 except ImportError: |
f75f7d39cca3
testing: generate tests operations using Hypothesis
David R. MacIver <david@drmaciver.com>
parents:
diff
changeset
|
28 sys.stderr.write("skipped: enum34 not installed" + os.linesep) |
f75f7d39cca3
testing: generate tests operations using Hypothesis
David R. MacIver <david@drmaciver.com>
parents:
diff
changeset
|
29 sys.exit(80) |
f75f7d39cca3
testing: generate tests operations using Hypothesis
David R. MacIver <david@drmaciver.com>
parents:
diff
changeset
|
30 |
f75f7d39cca3
testing: generate tests operations using Hypothesis
David R. MacIver <david@drmaciver.com>
parents:
diff
changeset
|
31 import binascii |
f75f7d39cca3
testing: generate tests operations using Hypothesis
David R. MacIver <david@drmaciver.com>
parents:
diff
changeset
|
32 from contextlib import contextmanager |
f75f7d39cca3
testing: generate tests operations using Hypothesis
David R. MacIver <david@drmaciver.com>
parents:
diff
changeset
|
33 import errno |
f75f7d39cca3
testing: generate tests operations using Hypothesis
David R. MacIver <david@drmaciver.com>
parents:
diff
changeset
|
34 import pipes |
f75f7d39cca3
testing: generate tests operations using Hypothesis
David R. MacIver <david@drmaciver.com>
parents:
diff
changeset
|
35 import shutil |
f75f7d39cca3
testing: generate tests operations using Hypothesis
David R. MacIver <david@drmaciver.com>
parents:
diff
changeset
|
36 import silenttestrunner |
f75f7d39cca3
testing: generate tests operations using Hypothesis
David R. MacIver <david@drmaciver.com>
parents:
diff
changeset
|
37 import subprocess |
f75f7d39cca3
testing: generate tests operations using Hypothesis
David R. MacIver <david@drmaciver.com>
parents:
diff
changeset
|
38 |
f75f7d39cca3
testing: generate tests operations using Hypothesis
David R. MacIver <david@drmaciver.com>
parents:
diff
changeset
|
39 from hypothesis.errors import HypothesisException |
f75f7d39cca3
testing: generate tests operations using Hypothesis
David R. MacIver <david@drmaciver.com>
parents:
diff
changeset
|
40 from hypothesis.stateful import rule, RuleBasedStateMachine, Bundle |
f75f7d39cca3
testing: generate tests operations using Hypothesis
David R. MacIver <david@drmaciver.com>
parents:
diff
changeset
|
41 from hypothesis import settings, note, strategies as st |
f75f7d39cca3
testing: generate tests operations using Hypothesis
David R. MacIver <david@drmaciver.com>
parents:
diff
changeset
|
42 from hypothesis.configuration import set_hypothesis_home_dir |
f75f7d39cca3
testing: generate tests operations using Hypothesis
David R. MacIver <david@drmaciver.com>
parents:
diff
changeset
|
43 |
f75f7d39cca3
testing: generate tests operations using Hypothesis
David R. MacIver <david@drmaciver.com>
parents:
diff
changeset
|
44 testdir = os.path.abspath(os.environ["TESTDIR"]) |
f75f7d39cca3
testing: generate tests operations using Hypothesis
David R. MacIver <david@drmaciver.com>
parents:
diff
changeset
|
45 |
f75f7d39cca3
testing: generate tests operations using Hypothesis
David R. MacIver <david@drmaciver.com>
parents:
diff
changeset
|
46 # We store Hypothesis examples here rather in the temporary test directory |
f75f7d39cca3
testing: generate tests operations using Hypothesis
David R. MacIver <david@drmaciver.com>
parents:
diff
changeset
|
47 # so that when rerunning a failing test this always results in refinding the |
f75f7d39cca3
testing: generate tests operations using Hypothesis
David R. MacIver <david@drmaciver.com>
parents:
diff
changeset
|
48 # previous failure. This directory is in .hgignore and should not be checked in |
f75f7d39cca3
testing: generate tests operations using Hypothesis
David R. MacIver <david@drmaciver.com>
parents:
diff
changeset
|
49 # but is useful to have for development. |
f75f7d39cca3
testing: generate tests operations using Hypothesis
David R. MacIver <david@drmaciver.com>
parents:
diff
changeset
|
50 set_hypothesis_home_dir(os.path.join(testdir, ".hypothesis")) |
f75f7d39cca3
testing: generate tests operations using Hypothesis
David R. MacIver <david@drmaciver.com>
parents:
diff
changeset
|
51 |
f75f7d39cca3
testing: generate tests operations using Hypothesis
David R. MacIver <david@drmaciver.com>
parents:
diff
changeset
|
52 runtests = os.path.join(os.environ["RUNTESTDIR"], "run-tests.py") |
f75f7d39cca3
testing: generate tests operations using Hypothesis
David R. MacIver <david@drmaciver.com>
parents:
diff
changeset
|
53 testtmp = os.environ["TESTTMP"] |
f75f7d39cca3
testing: generate tests operations using Hypothesis
David R. MacIver <david@drmaciver.com>
parents:
diff
changeset
|
54 assert os.path.isdir(testtmp) |
f75f7d39cca3
testing: generate tests operations using Hypothesis
David R. MacIver <david@drmaciver.com>
parents:
diff
changeset
|
55 |
f75f7d39cca3
testing: generate tests operations using Hypothesis
David R. MacIver <david@drmaciver.com>
parents:
diff
changeset
|
56 generatedtests = os.path.join(testdir, "hypothesis-generated") |
f75f7d39cca3
testing: generate tests operations using Hypothesis
David R. MacIver <david@drmaciver.com>
parents:
diff
changeset
|
57 |
f75f7d39cca3
testing: generate tests operations using Hypothesis
David R. MacIver <david@drmaciver.com>
parents:
diff
changeset
|
58 try: |
f75f7d39cca3
testing: generate tests operations using Hypothesis
David R. MacIver <david@drmaciver.com>
parents:
diff
changeset
|
59 os.makedirs(generatedtests) |
f75f7d39cca3
testing: generate tests operations using Hypothesis
David R. MacIver <david@drmaciver.com>
parents:
diff
changeset
|
60 except OSError: |
f75f7d39cca3
testing: generate tests operations using Hypothesis
David R. MacIver <david@drmaciver.com>
parents:
diff
changeset
|
61 pass |
f75f7d39cca3
testing: generate tests operations using Hypothesis
David R. MacIver <david@drmaciver.com>
parents:
diff
changeset
|
62 |
f75f7d39cca3
testing: generate tests operations using Hypothesis
David R. MacIver <david@drmaciver.com>
parents:
diff
changeset
|
63 # We write out generated .t files to a file in order to ease debugging and to |
f75f7d39cca3
testing: generate tests operations using Hypothesis
David R. MacIver <david@drmaciver.com>
parents:
diff
changeset
|
64 # give a starting point for turning failures Hypothesis finds into normal |
f75f7d39cca3
testing: generate tests operations using Hypothesis
David R. MacIver <david@drmaciver.com>
parents:
diff
changeset
|
65 # tests. In order to ensure that multiple copies of this test can be run in |
f75f7d39cca3
testing: generate tests operations using Hypothesis
David R. MacIver <david@drmaciver.com>
parents:
diff
changeset
|
66 # parallel we use atomic file create to ensure that we always get a unique |
f75f7d39cca3
testing: generate tests operations using Hypothesis
David R. MacIver <david@drmaciver.com>
parents:
diff
changeset
|
67 # name. |
f75f7d39cca3
testing: generate tests operations using Hypothesis
David R. MacIver <david@drmaciver.com>
parents:
diff
changeset
|
68 file_index = 0 |
f75f7d39cca3
testing: generate tests operations using Hypothesis
David R. MacIver <david@drmaciver.com>
parents:
diff
changeset
|
69 while True: |
f75f7d39cca3
testing: generate tests operations using Hypothesis
David R. MacIver <david@drmaciver.com>
parents:
diff
changeset
|
70 file_index += 1 |
f75f7d39cca3
testing: generate tests operations using Hypothesis
David R. MacIver <david@drmaciver.com>
parents:
diff
changeset
|
71 savefile = os.path.join(generatedtests, "test-generated-%d.t" % ( |
f75f7d39cca3
testing: generate tests operations using Hypothesis
David R. MacIver <david@drmaciver.com>
parents:
diff
changeset
|
72 file_index, |
f75f7d39cca3
testing: generate tests operations using Hypothesis
David R. MacIver <david@drmaciver.com>
parents:
diff
changeset
|
73 )) |
f75f7d39cca3
testing: generate tests operations using Hypothesis
David R. MacIver <david@drmaciver.com>
parents:
diff
changeset
|
74 try: |
f75f7d39cca3
testing: generate tests operations using Hypothesis
David R. MacIver <david@drmaciver.com>
parents:
diff
changeset
|
75 os.close(os.open(savefile, os.O_CREAT | os.O_EXCL | os.O_WRONLY)) |
f75f7d39cca3
testing: generate tests operations using Hypothesis
David R. MacIver <david@drmaciver.com>
parents:
diff
changeset
|
76 break |
f75f7d39cca3
testing: generate tests operations using Hypothesis
David R. MacIver <david@drmaciver.com>
parents:
diff
changeset
|
77 except OSError as e: |
f75f7d39cca3
testing: generate tests operations using Hypothesis
David R. MacIver <david@drmaciver.com>
parents:
diff
changeset
|
78 if e.errno != errno.EEXIST: |
f75f7d39cca3
testing: generate tests operations using Hypothesis
David R. MacIver <david@drmaciver.com>
parents:
diff
changeset
|
79 raise |
f75f7d39cca3
testing: generate tests operations using Hypothesis
David R. MacIver <david@drmaciver.com>
parents:
diff
changeset
|
80 assert os.path.exists(savefile) |
f75f7d39cca3
testing: generate tests operations using Hypothesis
David R. MacIver <david@drmaciver.com>
parents:
diff
changeset
|
81 |
f75f7d39cca3
testing: generate tests operations using Hypothesis
David R. MacIver <david@drmaciver.com>
parents:
diff
changeset
|
82 hgrc = os.path.join(".hg", "hgrc") |
f75f7d39cca3
testing: generate tests operations using Hypothesis
David R. MacIver <david@drmaciver.com>
parents:
diff
changeset
|
83 |
f75f7d39cca3
testing: generate tests operations using Hypothesis
David R. MacIver <david@drmaciver.com>
parents:
diff
changeset
|
84 filecharacters = ( |
f75f7d39cca3
testing: generate tests operations using Hypothesis
David R. MacIver <david@drmaciver.com>
parents:
diff
changeset
|
85 "abcdefghijklmnopqrstuvwxyzABCDEFGHIJKLMNOPQRSTUVWXYZ0123456789" |
f75f7d39cca3
testing: generate tests operations using Hypothesis
David R. MacIver <david@drmaciver.com>
parents:
diff
changeset
|
86 "[]^_`;=@{}~ !#$%&'()+,-" |
f75f7d39cca3
testing: generate tests operations using Hypothesis
David R. MacIver <david@drmaciver.com>
parents:
diff
changeset
|
87 ) |
f75f7d39cca3
testing: generate tests operations using Hypothesis
David R. MacIver <david@drmaciver.com>
parents:
diff
changeset
|
88 |
f75f7d39cca3
testing: generate tests operations using Hypothesis
David R. MacIver <david@drmaciver.com>
parents:
diff
changeset
|
89 files = st.text(filecharacters, min_size=1).map(lambda x: x.strip()).filter( |
f75f7d39cca3
testing: generate tests operations using Hypothesis
David R. MacIver <david@drmaciver.com>
parents:
diff
changeset
|
90 bool).map(lambda s: s.encode('ascii')) |
f75f7d39cca3
testing: generate tests operations using Hypothesis
David R. MacIver <david@drmaciver.com>
parents:
diff
changeset
|
91 |
f75f7d39cca3
testing: generate tests operations using Hypothesis
David R. MacIver <david@drmaciver.com>
parents:
diff
changeset
|
92 safetext = st.text(st.characters( |
f75f7d39cca3
testing: generate tests operations using Hypothesis
David R. MacIver <david@drmaciver.com>
parents:
diff
changeset
|
93 min_codepoint=1, max_codepoint=127, |
f75f7d39cca3
testing: generate tests operations using Hypothesis
David R. MacIver <david@drmaciver.com>
parents:
diff
changeset
|
94 blacklist_categories=('Cc', 'Cs')), min_size=1).map( |
f75f7d39cca3
testing: generate tests operations using Hypothesis
David R. MacIver <david@drmaciver.com>
parents:
diff
changeset
|
95 lambda s: s.encode('utf-8') |
f75f7d39cca3
testing: generate tests operations using Hypothesis
David R. MacIver <david@drmaciver.com>
parents:
diff
changeset
|
96 ) |
f75f7d39cca3
testing: generate tests operations using Hypothesis
David R. MacIver <david@drmaciver.com>
parents:
diff
changeset
|
97 |
f75f7d39cca3
testing: generate tests operations using Hypothesis
David R. MacIver <david@drmaciver.com>
parents:
diff
changeset
|
98 @contextmanager |
f75f7d39cca3
testing: generate tests operations using Hypothesis
David R. MacIver <david@drmaciver.com>
parents:
diff
changeset
|
99 def acceptableerrors(*args): |
f75f7d39cca3
testing: generate tests operations using Hypothesis
David R. MacIver <david@drmaciver.com>
parents:
diff
changeset
|
100 """Sometimes we know an operation we're about to perform might fail, and |
f75f7d39cca3
testing: generate tests operations using Hypothesis
David R. MacIver <david@drmaciver.com>
parents:
diff
changeset
|
101 we're OK with some of the failures. In those cases this may be used as a |
f75f7d39cca3
testing: generate tests operations using Hypothesis
David R. MacIver <david@drmaciver.com>
parents:
diff
changeset
|
102 context manager and will swallow expected failures, as identified by |
f75f7d39cca3
testing: generate tests operations using Hypothesis
David R. MacIver <david@drmaciver.com>
parents:
diff
changeset
|
103 substrings of the error message Mercurial emits.""" |
f75f7d39cca3
testing: generate tests operations using Hypothesis
David R. MacIver <david@drmaciver.com>
parents:
diff
changeset
|
104 try: |
f75f7d39cca3
testing: generate tests operations using Hypothesis
David R. MacIver <david@drmaciver.com>
parents:
diff
changeset
|
105 yield |
f75f7d39cca3
testing: generate tests operations using Hypothesis
David R. MacIver <david@drmaciver.com>
parents:
diff
changeset
|
106 except subprocess.CalledProcessError as e: |
f75f7d39cca3
testing: generate tests operations using Hypothesis
David R. MacIver <david@drmaciver.com>
parents:
diff
changeset
|
107 if not any(a in e.output for a in args): |
f75f7d39cca3
testing: generate tests operations using Hypothesis
David R. MacIver <david@drmaciver.com>
parents:
diff
changeset
|
108 note(e.output) |
f75f7d39cca3
testing: generate tests operations using Hypothesis
David R. MacIver <david@drmaciver.com>
parents:
diff
changeset
|
109 raise |
f75f7d39cca3
testing: generate tests operations using Hypothesis
David R. MacIver <david@drmaciver.com>
parents:
diff
changeset
|
110 |
f75f7d39cca3
testing: generate tests operations using Hypothesis
David R. MacIver <david@drmaciver.com>
parents:
diff
changeset
|
111 class verifyingstatemachine(RuleBasedStateMachine): |
f75f7d39cca3
testing: generate tests operations using Hypothesis
David R. MacIver <david@drmaciver.com>
parents:
diff
changeset
|
112 """This defines the set of acceptable operations on a Mercurial repository |
f75f7d39cca3
testing: generate tests operations using Hypothesis
David R. MacIver <david@drmaciver.com>
parents:
diff
changeset
|
113 using Hypothesis's RuleBasedStateMachine. |
f75f7d39cca3
testing: generate tests operations using Hypothesis
David R. MacIver <david@drmaciver.com>
parents:
diff
changeset
|
114 |
f75f7d39cca3
testing: generate tests operations using Hypothesis
David R. MacIver <david@drmaciver.com>
parents:
diff
changeset
|
115 The general concept is that we manage multiple repositories inside a |
f75f7d39cca3
testing: generate tests operations using Hypothesis
David R. MacIver <david@drmaciver.com>
parents:
diff
changeset
|
116 repos/ directory in our temporary test location. Some of these are freshly |
f75f7d39cca3
testing: generate tests operations using Hypothesis
David R. MacIver <david@drmaciver.com>
parents:
diff
changeset
|
117 inited, some are clones of the others. Our current working directory is |
f75f7d39cca3
testing: generate tests operations using Hypothesis
David R. MacIver <david@drmaciver.com>
parents:
diff
changeset
|
118 always inside one of these repositories while the tests are running. |
f75f7d39cca3
testing: generate tests operations using Hypothesis
David R. MacIver <david@drmaciver.com>
parents:
diff
changeset
|
119 |
f75f7d39cca3
testing: generate tests operations using Hypothesis
David R. MacIver <david@drmaciver.com>
parents:
diff
changeset
|
120 Hypothesis then performs a series of operations against these repositories, |
f75f7d39cca3
testing: generate tests operations using Hypothesis
David R. MacIver <david@drmaciver.com>
parents:
diff
changeset
|
121 including hg commands, generating contents and editing the .hgrc file. |
f75f7d39cca3
testing: generate tests operations using Hypothesis
David R. MacIver <david@drmaciver.com>
parents:
diff
changeset
|
122 If these operations fail in unexpected ways or behave differently in |
f75f7d39cca3
testing: generate tests operations using Hypothesis
David R. MacIver <david@drmaciver.com>
parents:
diff
changeset
|
123 different configurations of Mercurial, the test will fail and a minimized |
f75f7d39cca3
testing: generate tests operations using Hypothesis
David R. MacIver <david@drmaciver.com>
parents:
diff
changeset
|
124 .t test file will be written to the hypothesis-generated directory to |
f75f7d39cca3
testing: generate tests operations using Hypothesis
David R. MacIver <david@drmaciver.com>
parents:
diff
changeset
|
125 exhibit that failure. |
f75f7d39cca3
testing: generate tests operations using Hypothesis
David R. MacIver <david@drmaciver.com>
parents:
diff
changeset
|
126 |
f75f7d39cca3
testing: generate tests operations using Hypothesis
David R. MacIver <david@drmaciver.com>
parents:
diff
changeset
|
127 Operations are defined as methods with @rule() decorators. See the |
f75f7d39cca3
testing: generate tests operations using Hypothesis
David R. MacIver <david@drmaciver.com>
parents:
diff
changeset
|
128 Hypothesis documentation at |
f75f7d39cca3
testing: generate tests operations using Hypothesis
David R. MacIver <david@drmaciver.com>
parents:
diff
changeset
|
129 http://hypothesis.readthedocs.org/en/release/stateful.html for more |
f75f7d39cca3
testing: generate tests operations using Hypothesis
David R. MacIver <david@drmaciver.com>
parents:
diff
changeset
|
130 details.""" |
f75f7d39cca3
testing: generate tests operations using Hypothesis
David R. MacIver <david@drmaciver.com>
parents:
diff
changeset
|
131 |
f75f7d39cca3
testing: generate tests operations using Hypothesis
David R. MacIver <david@drmaciver.com>
parents:
diff
changeset
|
132 # A bundle is a reusable collection of previously generated data which may |
f75f7d39cca3
testing: generate tests operations using Hypothesis
David R. MacIver <david@drmaciver.com>
parents:
diff
changeset
|
133 # be provided as arguments to future operations. |
f75f7d39cca3
testing: generate tests operations using Hypothesis
David R. MacIver <david@drmaciver.com>
parents:
diff
changeset
|
134 paths = Bundle('paths') |
f75f7d39cca3
testing: generate tests operations using Hypothesis
David R. MacIver <david@drmaciver.com>
parents:
diff
changeset
|
135 contents = Bundle('contents') |
f75f7d39cca3
testing: generate tests operations using Hypothesis
David R. MacIver <david@drmaciver.com>
parents:
diff
changeset
|
136 committimes = Bundle('committimes') |
f75f7d39cca3
testing: generate tests operations using Hypothesis
David R. MacIver <david@drmaciver.com>
parents:
diff
changeset
|
137 |
f75f7d39cca3
testing: generate tests operations using Hypothesis
David R. MacIver <david@drmaciver.com>
parents:
diff
changeset
|
138 def __init__(self): |
f75f7d39cca3
testing: generate tests operations using Hypothesis
David R. MacIver <david@drmaciver.com>
parents:
diff
changeset
|
139 super(verifyingstatemachine, self).__init__() |
f75f7d39cca3
testing: generate tests operations using Hypothesis
David R. MacIver <david@drmaciver.com>
parents:
diff
changeset
|
140 self.repodir = os.path.join(testtmp, "repo") |
f75f7d39cca3
testing: generate tests operations using Hypothesis
David R. MacIver <david@drmaciver.com>
parents:
diff
changeset
|
141 if os.path.exists(self.repodir): |
f75f7d39cca3
testing: generate tests operations using Hypothesis
David R. MacIver <david@drmaciver.com>
parents:
diff
changeset
|
142 shutil.rmtree(self.repodir) |
f75f7d39cca3
testing: generate tests operations using Hypothesis
David R. MacIver <david@drmaciver.com>
parents:
diff
changeset
|
143 os.chdir(testtmp) |
f75f7d39cca3
testing: generate tests operations using Hypothesis
David R. MacIver <david@drmaciver.com>
parents:
diff
changeset
|
144 self.log = [] |
f75f7d39cca3
testing: generate tests operations using Hypothesis
David R. MacIver <david@drmaciver.com>
parents:
diff
changeset
|
145 self.failed = False |
f75f7d39cca3
testing: generate tests operations using Hypothesis
David R. MacIver <david@drmaciver.com>
parents:
diff
changeset
|
146 |
f75f7d39cca3
testing: generate tests operations using Hypothesis
David R. MacIver <david@drmaciver.com>
parents:
diff
changeset
|
147 self.mkdirp("repo") |
f75f7d39cca3
testing: generate tests operations using Hypothesis
David R. MacIver <david@drmaciver.com>
parents:
diff
changeset
|
148 self.cd("repo") |
f75f7d39cca3
testing: generate tests operations using Hypothesis
David R. MacIver <david@drmaciver.com>
parents:
diff
changeset
|
149 self.hg("init") |
f75f7d39cca3
testing: generate tests operations using Hypothesis
David R. MacIver <david@drmaciver.com>
parents:
diff
changeset
|
150 |
f75f7d39cca3
testing: generate tests operations using Hypothesis
David R. MacIver <david@drmaciver.com>
parents:
diff
changeset
|
151 def teardown(self): |
f75f7d39cca3
testing: generate tests operations using Hypothesis
David R. MacIver <david@drmaciver.com>
parents:
diff
changeset
|
152 """On teardown we clean up after ourselves as usual, but we also |
f75f7d39cca3
testing: generate tests operations using Hypothesis
David R. MacIver <david@drmaciver.com>
parents:
diff
changeset
|
153 do some additional testing: We generate a .t file based on our test |
f75f7d39cca3
testing: generate tests operations using Hypothesis
David R. MacIver <david@drmaciver.com>
parents:
diff
changeset
|
154 run using run-test.py -i to get the correct output. |
f75f7d39cca3
testing: generate tests operations using Hypothesis
David R. MacIver <david@drmaciver.com>
parents:
diff
changeset
|
155 |
f75f7d39cca3
testing: generate tests operations using Hypothesis
David R. MacIver <david@drmaciver.com>
parents:
diff
changeset
|
156 We then test it in a number of other configurations, verifying that |
f75f7d39cca3
testing: generate tests operations using Hypothesis
David R. MacIver <david@drmaciver.com>
parents:
diff
changeset
|
157 each passes the same test.""" |
f75f7d39cca3
testing: generate tests operations using Hypothesis
David R. MacIver <david@drmaciver.com>
parents:
diff
changeset
|
158 super(verifyingstatemachine, self).teardown() |
f75f7d39cca3
testing: generate tests operations using Hypothesis
David R. MacIver <david@drmaciver.com>
parents:
diff
changeset
|
159 try: |
f75f7d39cca3
testing: generate tests operations using Hypothesis
David R. MacIver <david@drmaciver.com>
parents:
diff
changeset
|
160 shutil.rmtree(self.repodir) |
f75f7d39cca3
testing: generate tests operations using Hypothesis
David R. MacIver <david@drmaciver.com>
parents:
diff
changeset
|
161 except OSError: |
f75f7d39cca3
testing: generate tests operations using Hypothesis
David R. MacIver <david@drmaciver.com>
parents:
diff
changeset
|
162 pass |
f75f7d39cca3
testing: generate tests operations using Hypothesis
David R. MacIver <david@drmaciver.com>
parents:
diff
changeset
|
163 ttest = os.linesep.join(" " + l for l in self.log) |
f75f7d39cca3
testing: generate tests operations using Hypothesis
David R. MacIver <david@drmaciver.com>
parents:
diff
changeset
|
164 os.chdir(testtmp) |
f75f7d39cca3
testing: generate tests operations using Hypothesis
David R. MacIver <david@drmaciver.com>
parents:
diff
changeset
|
165 path = os.path.join(testtmp, "test-generated.t") |
f75f7d39cca3
testing: generate tests operations using Hypothesis
David R. MacIver <david@drmaciver.com>
parents:
diff
changeset
|
166 with open(path, 'w') as o: |
f75f7d39cca3
testing: generate tests operations using Hypothesis
David R. MacIver <david@drmaciver.com>
parents:
diff
changeset
|
167 o.write(ttest + os.linesep) |
f75f7d39cca3
testing: generate tests operations using Hypothesis
David R. MacIver <david@drmaciver.com>
parents:
diff
changeset
|
168 with open(os.devnull, "w") as devnull: |
f75f7d39cca3
testing: generate tests operations using Hypothesis
David R. MacIver <david@drmaciver.com>
parents:
diff
changeset
|
169 rewriter = subprocess.Popen( |
f75f7d39cca3
testing: generate tests operations using Hypothesis
David R. MacIver <david@drmaciver.com>
parents:
diff
changeset
|
170 [runtests, "--local", "-i", path], stdin=subprocess.PIPE, |
f75f7d39cca3
testing: generate tests operations using Hypothesis
David R. MacIver <david@drmaciver.com>
parents:
diff
changeset
|
171 stdout=devnull, stderr=devnull, |
f75f7d39cca3
testing: generate tests operations using Hypothesis
David R. MacIver <david@drmaciver.com>
parents:
diff
changeset
|
172 ) |
f75f7d39cca3
testing: generate tests operations using Hypothesis
David R. MacIver <david@drmaciver.com>
parents:
diff
changeset
|
173 rewriter.communicate("yes") |
f75f7d39cca3
testing: generate tests operations using Hypothesis
David R. MacIver <david@drmaciver.com>
parents:
diff
changeset
|
174 with open(path, 'r') as i: |
f75f7d39cca3
testing: generate tests operations using Hypothesis
David R. MacIver <david@drmaciver.com>
parents:
diff
changeset
|
175 ttest = i.read() |
f75f7d39cca3
testing: generate tests operations using Hypothesis
David R. MacIver <david@drmaciver.com>
parents:
diff
changeset
|
176 |
f75f7d39cca3
testing: generate tests operations using Hypothesis
David R. MacIver <david@drmaciver.com>
parents:
diff
changeset
|
177 e = None |
f75f7d39cca3
testing: generate tests operations using Hypothesis
David R. MacIver <david@drmaciver.com>
parents:
diff
changeset
|
178 if not self.failed: |
f75f7d39cca3
testing: generate tests operations using Hypothesis
David R. MacIver <david@drmaciver.com>
parents:
diff
changeset
|
179 try: |
f75f7d39cca3
testing: generate tests operations using Hypothesis
David R. MacIver <david@drmaciver.com>
parents:
diff
changeset
|
180 output = subprocess.check_output([ |
f75f7d39cca3
testing: generate tests operations using Hypothesis
David R. MacIver <david@drmaciver.com>
parents:
diff
changeset
|
181 runtests, path, "--local", "--pure" |
f75f7d39cca3
testing: generate tests operations using Hypothesis
David R. MacIver <david@drmaciver.com>
parents:
diff
changeset
|
182 ], stderr=subprocess.STDOUT) |
f75f7d39cca3
testing: generate tests operations using Hypothesis
David R. MacIver <david@drmaciver.com>
parents:
diff
changeset
|
183 assert "Ran 1 test" in output, output |
f75f7d39cca3
testing: generate tests operations using Hypothesis
David R. MacIver <david@drmaciver.com>
parents:
diff
changeset
|
184 except subprocess.CalledProcessError as e: |
f75f7d39cca3
testing: generate tests operations using Hypothesis
David R. MacIver <david@drmaciver.com>
parents:
diff
changeset
|
185 note(e.output) |
f75f7d39cca3
testing: generate tests operations using Hypothesis
David R. MacIver <david@drmaciver.com>
parents:
diff
changeset
|
186 finally: |
f75f7d39cca3
testing: generate tests operations using Hypothesis
David R. MacIver <david@drmaciver.com>
parents:
diff
changeset
|
187 os.unlink(path) |
f75f7d39cca3
testing: generate tests operations using Hypothesis
David R. MacIver <david@drmaciver.com>
parents:
diff
changeset
|
188 try: |
f75f7d39cca3
testing: generate tests operations using Hypothesis
David R. MacIver <david@drmaciver.com>
parents:
diff
changeset
|
189 os.unlink(path + ".err") |
f75f7d39cca3
testing: generate tests operations using Hypothesis
David R. MacIver <david@drmaciver.com>
parents:
diff
changeset
|
190 except OSError: |
f75f7d39cca3
testing: generate tests operations using Hypothesis
David R. MacIver <david@drmaciver.com>
parents:
diff
changeset
|
191 pass |
f75f7d39cca3
testing: generate tests operations using Hypothesis
David R. MacIver <david@drmaciver.com>
parents:
diff
changeset
|
192 if self.failed or e is not None: |
f75f7d39cca3
testing: generate tests operations using Hypothesis
David R. MacIver <david@drmaciver.com>
parents:
diff
changeset
|
193 with open(savefile, "wb") as o: |
f75f7d39cca3
testing: generate tests operations using Hypothesis
David R. MacIver <david@drmaciver.com>
parents:
diff
changeset
|
194 o.write(ttest) |
f75f7d39cca3
testing: generate tests operations using Hypothesis
David R. MacIver <david@drmaciver.com>
parents:
diff
changeset
|
195 if e is not None: |
f75f7d39cca3
testing: generate tests operations using Hypothesis
David R. MacIver <david@drmaciver.com>
parents:
diff
changeset
|
196 raise e |
f75f7d39cca3
testing: generate tests operations using Hypothesis
David R. MacIver <david@drmaciver.com>
parents:
diff
changeset
|
197 |
f75f7d39cca3
testing: generate tests operations using Hypothesis
David R. MacIver <david@drmaciver.com>
parents:
diff
changeset
|
198 def execute_step(self, step): |
f75f7d39cca3
testing: generate tests operations using Hypothesis
David R. MacIver <david@drmaciver.com>
parents:
diff
changeset
|
199 try: |
f75f7d39cca3
testing: generate tests operations using Hypothesis
David R. MacIver <david@drmaciver.com>
parents:
diff
changeset
|
200 return super(verifyingstatemachine, self).execute_step(step) |
f75f7d39cca3
testing: generate tests operations using Hypothesis
David R. MacIver <david@drmaciver.com>
parents:
diff
changeset
|
201 except (HypothesisException, KeyboardInterrupt): |
f75f7d39cca3
testing: generate tests operations using Hypothesis
David R. MacIver <david@drmaciver.com>
parents:
diff
changeset
|
202 raise |
f75f7d39cca3
testing: generate tests operations using Hypothesis
David R. MacIver <david@drmaciver.com>
parents:
diff
changeset
|
203 except Exception: |
f75f7d39cca3
testing: generate tests operations using Hypothesis
David R. MacIver <david@drmaciver.com>
parents:
diff
changeset
|
204 self.failed = True |
f75f7d39cca3
testing: generate tests operations using Hypothesis
David R. MacIver <david@drmaciver.com>
parents:
diff
changeset
|
205 raise |
f75f7d39cca3
testing: generate tests operations using Hypothesis
David R. MacIver <david@drmaciver.com>
parents:
diff
changeset
|
206 |
f75f7d39cca3
testing: generate tests operations using Hypothesis
David R. MacIver <david@drmaciver.com>
parents:
diff
changeset
|
207 # Section: Basic commands. |
f75f7d39cca3
testing: generate tests operations using Hypothesis
David R. MacIver <david@drmaciver.com>
parents:
diff
changeset
|
208 def mkdirp(self, path): |
f75f7d39cca3
testing: generate tests operations using Hypothesis
David R. MacIver <david@drmaciver.com>
parents:
diff
changeset
|
209 if os.path.exists(path): |
f75f7d39cca3
testing: generate tests operations using Hypothesis
David R. MacIver <david@drmaciver.com>
parents:
diff
changeset
|
210 return |
f75f7d39cca3
testing: generate tests operations using Hypothesis
David R. MacIver <david@drmaciver.com>
parents:
diff
changeset
|
211 self.log.append( |
f75f7d39cca3
testing: generate tests operations using Hypothesis
David R. MacIver <david@drmaciver.com>
parents:
diff
changeset
|
212 "$ mkdir -p -- %s" % (pipes.quote(os.path.relpath(path)),)) |
f75f7d39cca3
testing: generate tests operations using Hypothesis
David R. MacIver <david@drmaciver.com>
parents:
diff
changeset
|
213 os.makedirs(path) |
f75f7d39cca3
testing: generate tests operations using Hypothesis
David R. MacIver <david@drmaciver.com>
parents:
diff
changeset
|
214 |
f75f7d39cca3
testing: generate tests operations using Hypothesis
David R. MacIver <david@drmaciver.com>
parents:
diff
changeset
|
215 def cd(self, path): |
f75f7d39cca3
testing: generate tests operations using Hypothesis
David R. MacIver <david@drmaciver.com>
parents:
diff
changeset
|
216 path = os.path.relpath(path) |
f75f7d39cca3
testing: generate tests operations using Hypothesis
David R. MacIver <david@drmaciver.com>
parents:
diff
changeset
|
217 if path == ".": |
f75f7d39cca3
testing: generate tests operations using Hypothesis
David R. MacIver <david@drmaciver.com>
parents:
diff
changeset
|
218 return |
f75f7d39cca3
testing: generate tests operations using Hypothesis
David R. MacIver <david@drmaciver.com>
parents:
diff
changeset
|
219 os.chdir(path) |
f75f7d39cca3
testing: generate tests operations using Hypothesis
David R. MacIver <david@drmaciver.com>
parents:
diff
changeset
|
220 self.log.append("$ cd -- %s" % (pipes.quote(path),)) |
f75f7d39cca3
testing: generate tests operations using Hypothesis
David R. MacIver <david@drmaciver.com>
parents:
diff
changeset
|
221 |
f75f7d39cca3
testing: generate tests operations using Hypothesis
David R. MacIver <david@drmaciver.com>
parents:
diff
changeset
|
222 def hg(self, *args): |
f75f7d39cca3
testing: generate tests operations using Hypothesis
David R. MacIver <david@drmaciver.com>
parents:
diff
changeset
|
223 self.command("hg", *args) |
f75f7d39cca3
testing: generate tests operations using Hypothesis
David R. MacIver <david@drmaciver.com>
parents:
diff
changeset
|
224 |
f75f7d39cca3
testing: generate tests operations using Hypothesis
David R. MacIver <david@drmaciver.com>
parents:
diff
changeset
|
225 def command(self, *args): |
f75f7d39cca3
testing: generate tests operations using Hypothesis
David R. MacIver <david@drmaciver.com>
parents:
diff
changeset
|
226 self.log.append("$ " + ' '.join(map(pipes.quote, args))) |
f75f7d39cca3
testing: generate tests operations using Hypothesis
David R. MacIver <david@drmaciver.com>
parents:
diff
changeset
|
227 subprocess.check_output(args, stderr=subprocess.STDOUT) |
f75f7d39cca3
testing: generate tests operations using Hypothesis
David R. MacIver <david@drmaciver.com>
parents:
diff
changeset
|
228 |
f75f7d39cca3
testing: generate tests operations using Hypothesis
David R. MacIver <david@drmaciver.com>
parents:
diff
changeset
|
229 # Section: Set up basic data |
f75f7d39cca3
testing: generate tests operations using Hypothesis
David R. MacIver <david@drmaciver.com>
parents:
diff
changeset
|
230 # This section has no side effects but generates data that we will want |
f75f7d39cca3
testing: generate tests operations using Hypothesis
David R. MacIver <david@drmaciver.com>
parents:
diff
changeset
|
231 # to use later. |
f75f7d39cca3
testing: generate tests operations using Hypothesis
David R. MacIver <david@drmaciver.com>
parents:
diff
changeset
|
232 @rule( |
f75f7d39cca3
testing: generate tests operations using Hypothesis
David R. MacIver <david@drmaciver.com>
parents:
diff
changeset
|
233 target=paths, |
f75f7d39cca3
testing: generate tests operations using Hypothesis
David R. MacIver <david@drmaciver.com>
parents:
diff
changeset
|
234 source=st.lists(files, min_size=1).map(lambda l: os.path.join(*l))) |
f75f7d39cca3
testing: generate tests operations using Hypothesis
David R. MacIver <david@drmaciver.com>
parents:
diff
changeset
|
235 def genpath(self, source): |
f75f7d39cca3
testing: generate tests operations using Hypothesis
David R. MacIver <david@drmaciver.com>
parents:
diff
changeset
|
236 return source |
f75f7d39cca3
testing: generate tests operations using Hypothesis
David R. MacIver <david@drmaciver.com>
parents:
diff
changeset
|
237 |
f75f7d39cca3
testing: generate tests operations using Hypothesis
David R. MacIver <david@drmaciver.com>
parents:
diff
changeset
|
238 @rule( |
f75f7d39cca3
testing: generate tests operations using Hypothesis
David R. MacIver <david@drmaciver.com>
parents:
diff
changeset
|
239 target=committimes, |
f75f7d39cca3
testing: generate tests operations using Hypothesis
David R. MacIver <david@drmaciver.com>
parents:
diff
changeset
|
240 when=datetimes(min_year=1970, max_year=2038) | st.none()) |
f75f7d39cca3
testing: generate tests operations using Hypothesis
David R. MacIver <david@drmaciver.com>
parents:
diff
changeset
|
241 def gentime(self, when): |
f75f7d39cca3
testing: generate tests operations using Hypothesis
David R. MacIver <david@drmaciver.com>
parents:
diff
changeset
|
242 return when |
f75f7d39cca3
testing: generate tests operations using Hypothesis
David R. MacIver <david@drmaciver.com>
parents:
diff
changeset
|
243 |
f75f7d39cca3
testing: generate tests operations using Hypothesis
David R. MacIver <david@drmaciver.com>
parents:
diff
changeset
|
244 @rule( |
f75f7d39cca3
testing: generate tests operations using Hypothesis
David R. MacIver <david@drmaciver.com>
parents:
diff
changeset
|
245 target=contents, |
f75f7d39cca3
testing: generate tests operations using Hypothesis
David R. MacIver <david@drmaciver.com>
parents:
diff
changeset
|
246 content=st.one_of( |
f75f7d39cca3
testing: generate tests operations using Hypothesis
David R. MacIver <david@drmaciver.com>
parents:
diff
changeset
|
247 st.binary(), |
f75f7d39cca3
testing: generate tests operations using Hypothesis
David R. MacIver <david@drmaciver.com>
parents:
diff
changeset
|
248 st.text().map(lambda x: x.encode('utf-8')) |
f75f7d39cca3
testing: generate tests operations using Hypothesis
David R. MacIver <david@drmaciver.com>
parents:
diff
changeset
|
249 )) |
f75f7d39cca3
testing: generate tests operations using Hypothesis
David R. MacIver <david@drmaciver.com>
parents:
diff
changeset
|
250 def gencontent(self, content): |
f75f7d39cca3
testing: generate tests operations using Hypothesis
David R. MacIver <david@drmaciver.com>
parents:
diff
changeset
|
251 return content |
f75f7d39cca3
testing: generate tests operations using Hypothesis
David R. MacIver <david@drmaciver.com>
parents:
diff
changeset
|
252 |
f75f7d39cca3
testing: generate tests operations using Hypothesis
David R. MacIver <david@drmaciver.com>
parents:
diff
changeset
|
253 @rule(target=paths, source=paths) |
f75f7d39cca3
testing: generate tests operations using Hypothesis
David R. MacIver <david@drmaciver.com>
parents:
diff
changeset
|
254 def lowerpath(self, source): |
f75f7d39cca3
testing: generate tests operations using Hypothesis
David R. MacIver <david@drmaciver.com>
parents:
diff
changeset
|
255 return source.lower() |
f75f7d39cca3
testing: generate tests operations using Hypothesis
David R. MacIver <david@drmaciver.com>
parents:
diff
changeset
|
256 |
f75f7d39cca3
testing: generate tests operations using Hypothesis
David R. MacIver <david@drmaciver.com>
parents:
diff
changeset
|
257 @rule(target=paths, source=paths) |
f75f7d39cca3
testing: generate tests operations using Hypothesis
David R. MacIver <david@drmaciver.com>
parents:
diff
changeset
|
258 def upperpath(self, source): |
f75f7d39cca3
testing: generate tests operations using Hypothesis
David R. MacIver <david@drmaciver.com>
parents:
diff
changeset
|
259 return source.upper() |
f75f7d39cca3
testing: generate tests operations using Hypothesis
David R. MacIver <david@drmaciver.com>
parents:
diff
changeset
|
260 |
f75f7d39cca3
testing: generate tests operations using Hypothesis
David R. MacIver <david@drmaciver.com>
parents:
diff
changeset
|
261 # Section: Basic path operations |
f75f7d39cca3
testing: generate tests operations using Hypothesis
David R. MacIver <david@drmaciver.com>
parents:
diff
changeset
|
262 @rule(path=paths, content=contents) |
f75f7d39cca3
testing: generate tests operations using Hypothesis
David R. MacIver <david@drmaciver.com>
parents:
diff
changeset
|
263 def writecontent(self, path, content): |
f75f7d39cca3
testing: generate tests operations using Hypothesis
David R. MacIver <david@drmaciver.com>
parents:
diff
changeset
|
264 self.unadded_changes = True |
f75f7d39cca3
testing: generate tests operations using Hypothesis
David R. MacIver <david@drmaciver.com>
parents:
diff
changeset
|
265 if os.path.isdir(path): |
f75f7d39cca3
testing: generate tests operations using Hypothesis
David R. MacIver <david@drmaciver.com>
parents:
diff
changeset
|
266 return |
f75f7d39cca3
testing: generate tests operations using Hypothesis
David R. MacIver <david@drmaciver.com>
parents:
diff
changeset
|
267 parent = os.path.dirname(path) |
f75f7d39cca3
testing: generate tests operations using Hypothesis
David R. MacIver <david@drmaciver.com>
parents:
diff
changeset
|
268 if parent: |
f75f7d39cca3
testing: generate tests operations using Hypothesis
David R. MacIver <david@drmaciver.com>
parents:
diff
changeset
|
269 try: |
f75f7d39cca3
testing: generate tests operations using Hypothesis
David R. MacIver <david@drmaciver.com>
parents:
diff
changeset
|
270 self.mkdirp(parent) |
f75f7d39cca3
testing: generate tests operations using Hypothesis
David R. MacIver <david@drmaciver.com>
parents:
diff
changeset
|
271 except OSError: |
f75f7d39cca3
testing: generate tests operations using Hypothesis
David R. MacIver <david@drmaciver.com>
parents:
diff
changeset
|
272 # It may be the case that there is a regular file that has |
f75f7d39cca3
testing: generate tests operations using Hypothesis
David R. MacIver <david@drmaciver.com>
parents:
diff
changeset
|
273 # previously been created that has the same name as an ancestor |
f75f7d39cca3
testing: generate tests operations using Hypothesis
David R. MacIver <david@drmaciver.com>
parents:
diff
changeset
|
274 # of the current path. This will cause mkdirp to fail with this |
f75f7d39cca3
testing: generate tests operations using Hypothesis
David R. MacIver <david@drmaciver.com>
parents:
diff
changeset
|
275 # error. We just turn this into a no-op in that case. |
f75f7d39cca3
testing: generate tests operations using Hypothesis
David R. MacIver <david@drmaciver.com>
parents:
diff
changeset
|
276 return |
f75f7d39cca3
testing: generate tests operations using Hypothesis
David R. MacIver <david@drmaciver.com>
parents:
diff
changeset
|
277 with open(path, 'wb') as o: |
f75f7d39cca3
testing: generate tests operations using Hypothesis
David R. MacIver <david@drmaciver.com>
parents:
diff
changeset
|
278 o.write(content) |
f75f7d39cca3
testing: generate tests operations using Hypothesis
David R. MacIver <david@drmaciver.com>
parents:
diff
changeset
|
279 self.log.append(( |
f75f7d39cca3
testing: generate tests operations using Hypothesis
David R. MacIver <david@drmaciver.com>
parents:
diff
changeset
|
280 "$ python -c 'import binascii; " |
f75f7d39cca3
testing: generate tests operations using Hypothesis
David R. MacIver <david@drmaciver.com>
parents:
diff
changeset
|
281 "print(binascii.unhexlify(\"%s\"))' > %s") % ( |
f75f7d39cca3
testing: generate tests operations using Hypothesis
David R. MacIver <david@drmaciver.com>
parents:
diff
changeset
|
282 binascii.hexlify(content), |
f75f7d39cca3
testing: generate tests operations using Hypothesis
David R. MacIver <david@drmaciver.com>
parents:
diff
changeset
|
283 pipes.quote(path), |
f75f7d39cca3
testing: generate tests operations using Hypothesis
David R. MacIver <david@drmaciver.com>
parents:
diff
changeset
|
284 )) |
f75f7d39cca3
testing: generate tests operations using Hypothesis
David R. MacIver <david@drmaciver.com>
parents:
diff
changeset
|
285 |
f75f7d39cca3
testing: generate tests operations using Hypothesis
David R. MacIver <david@drmaciver.com>
parents:
diff
changeset
|
286 @rule(path=paths) |
f75f7d39cca3
testing: generate tests operations using Hypothesis
David R. MacIver <david@drmaciver.com>
parents:
diff
changeset
|
287 def addpath(self, path): |
f75f7d39cca3
testing: generate tests operations using Hypothesis
David R. MacIver <david@drmaciver.com>
parents:
diff
changeset
|
288 if os.path.exists(path): |
f75f7d39cca3
testing: generate tests operations using Hypothesis
David R. MacIver <david@drmaciver.com>
parents:
diff
changeset
|
289 self.hg("add", "--", path) |
f75f7d39cca3
testing: generate tests operations using Hypothesis
David R. MacIver <david@drmaciver.com>
parents:
diff
changeset
|
290 |
f75f7d39cca3
testing: generate tests operations using Hypothesis
David R. MacIver <david@drmaciver.com>
parents:
diff
changeset
|
291 @rule(path=paths) |
f75f7d39cca3
testing: generate tests operations using Hypothesis
David R. MacIver <david@drmaciver.com>
parents:
diff
changeset
|
292 def forgetpath(self, path): |
f75f7d39cca3
testing: generate tests operations using Hypothesis
David R. MacIver <david@drmaciver.com>
parents:
diff
changeset
|
293 if os.path.exists(path): |
f75f7d39cca3
testing: generate tests operations using Hypothesis
David R. MacIver <david@drmaciver.com>
parents:
diff
changeset
|
294 with acceptableerrors( |
f75f7d39cca3
testing: generate tests operations using Hypothesis
David R. MacIver <david@drmaciver.com>
parents:
diff
changeset
|
295 "file is already untracked", |
f75f7d39cca3
testing: generate tests operations using Hypothesis
David R. MacIver <david@drmaciver.com>
parents:
diff
changeset
|
296 ): |
f75f7d39cca3
testing: generate tests operations using Hypothesis
David R. MacIver <david@drmaciver.com>
parents:
diff
changeset
|
297 self.hg("forget", "--", path) |
f75f7d39cca3
testing: generate tests operations using Hypothesis
David R. MacIver <david@drmaciver.com>
parents:
diff
changeset
|
298 |
f75f7d39cca3
testing: generate tests operations using Hypothesis
David R. MacIver <david@drmaciver.com>
parents:
diff
changeset
|
299 @rule(s=st.none() | st.integers(0, 100)) |
f75f7d39cca3
testing: generate tests operations using Hypothesis
David R. MacIver <david@drmaciver.com>
parents:
diff
changeset
|
300 def addremove(self, s): |
f75f7d39cca3
testing: generate tests operations using Hypothesis
David R. MacIver <david@drmaciver.com>
parents:
diff
changeset
|
301 args = ["addremove"] |
f75f7d39cca3
testing: generate tests operations using Hypothesis
David R. MacIver <david@drmaciver.com>
parents:
diff
changeset
|
302 if s is not None: |
f75f7d39cca3
testing: generate tests operations using Hypothesis
David R. MacIver <david@drmaciver.com>
parents:
diff
changeset
|
303 args.extend(["-s", str(s)]) |
f75f7d39cca3
testing: generate tests operations using Hypothesis
David R. MacIver <david@drmaciver.com>
parents:
diff
changeset
|
304 self.hg(*args) |
f75f7d39cca3
testing: generate tests operations using Hypothesis
David R. MacIver <david@drmaciver.com>
parents:
diff
changeset
|
305 |
f75f7d39cca3
testing: generate tests operations using Hypothesis
David R. MacIver <david@drmaciver.com>
parents:
diff
changeset
|
306 @rule(path=paths) |
f75f7d39cca3
testing: generate tests operations using Hypothesis
David R. MacIver <david@drmaciver.com>
parents:
diff
changeset
|
307 def removepath(self, path): |
f75f7d39cca3
testing: generate tests operations using Hypothesis
David R. MacIver <david@drmaciver.com>
parents:
diff
changeset
|
308 if os.path.exists(path): |
f75f7d39cca3
testing: generate tests operations using Hypothesis
David R. MacIver <david@drmaciver.com>
parents:
diff
changeset
|
309 with acceptableerrors( |
f75f7d39cca3
testing: generate tests operations using Hypothesis
David R. MacIver <david@drmaciver.com>
parents:
diff
changeset
|
310 'file is untracked', |
f75f7d39cca3
testing: generate tests operations using Hypothesis
David R. MacIver <david@drmaciver.com>
parents:
diff
changeset
|
311 'file has been marked for add', |
f75f7d39cca3
testing: generate tests operations using Hypothesis
David R. MacIver <david@drmaciver.com>
parents:
diff
changeset
|
312 'file is modified', |
f75f7d39cca3
testing: generate tests operations using Hypothesis
David R. MacIver <david@drmaciver.com>
parents:
diff
changeset
|
313 ): |
f75f7d39cca3
testing: generate tests operations using Hypothesis
David R. MacIver <david@drmaciver.com>
parents:
diff
changeset
|
314 self.hg("remove", "--", path) |
f75f7d39cca3
testing: generate tests operations using Hypothesis
David R. MacIver <david@drmaciver.com>
parents:
diff
changeset
|
315 |
f75f7d39cca3
testing: generate tests operations using Hypothesis
David R. MacIver <david@drmaciver.com>
parents:
diff
changeset
|
316 @rule( |
f75f7d39cca3
testing: generate tests operations using Hypothesis
David R. MacIver <david@drmaciver.com>
parents:
diff
changeset
|
317 message=safetext, |
f75f7d39cca3
testing: generate tests operations using Hypothesis
David R. MacIver <david@drmaciver.com>
parents:
diff
changeset
|
318 amend=st.booleans(), |
f75f7d39cca3
testing: generate tests operations using Hypothesis
David R. MacIver <david@drmaciver.com>
parents:
diff
changeset
|
319 when=committimes, |
f75f7d39cca3
testing: generate tests operations using Hypothesis
David R. MacIver <david@drmaciver.com>
parents:
diff
changeset
|
320 addremove=st.booleans(), |
f75f7d39cca3
testing: generate tests operations using Hypothesis
David R. MacIver <david@drmaciver.com>
parents:
diff
changeset
|
321 secret=st.booleans(), |
f75f7d39cca3
testing: generate tests operations using Hypothesis
David R. MacIver <david@drmaciver.com>
parents:
diff
changeset
|
322 close_branch=st.booleans(), |
f75f7d39cca3
testing: generate tests operations using Hypothesis
David R. MacIver <david@drmaciver.com>
parents:
diff
changeset
|
323 ) |
f75f7d39cca3
testing: generate tests operations using Hypothesis
David R. MacIver <david@drmaciver.com>
parents:
diff
changeset
|
324 def maybecommit( |
f75f7d39cca3
testing: generate tests operations using Hypothesis
David R. MacIver <david@drmaciver.com>
parents:
diff
changeset
|
325 self, message, amend, when, addremove, secret, close_branch |
f75f7d39cca3
testing: generate tests operations using Hypothesis
David R. MacIver <david@drmaciver.com>
parents:
diff
changeset
|
326 ): |
f75f7d39cca3
testing: generate tests operations using Hypothesis
David R. MacIver <david@drmaciver.com>
parents:
diff
changeset
|
327 command = ["commit"] |
f75f7d39cca3
testing: generate tests operations using Hypothesis
David R. MacIver <david@drmaciver.com>
parents:
diff
changeset
|
328 errors = ["nothing changed"] |
f75f7d39cca3
testing: generate tests operations using Hypothesis
David R. MacIver <david@drmaciver.com>
parents:
diff
changeset
|
329 if amend: |
f75f7d39cca3
testing: generate tests operations using Hypothesis
David R. MacIver <david@drmaciver.com>
parents:
diff
changeset
|
330 errors.append("cannot amend public changesets") |
f75f7d39cca3
testing: generate tests operations using Hypothesis
David R. MacIver <david@drmaciver.com>
parents:
diff
changeset
|
331 command.append("--amend") |
f75f7d39cca3
testing: generate tests operations using Hypothesis
David R. MacIver <david@drmaciver.com>
parents:
diff
changeset
|
332 command.append("-m" + pipes.quote(message)) |
f75f7d39cca3
testing: generate tests operations using Hypothesis
David R. MacIver <david@drmaciver.com>
parents:
diff
changeset
|
333 if secret: |
f75f7d39cca3
testing: generate tests operations using Hypothesis
David R. MacIver <david@drmaciver.com>
parents:
diff
changeset
|
334 command.append("--secret") |
f75f7d39cca3
testing: generate tests operations using Hypothesis
David R. MacIver <david@drmaciver.com>
parents:
diff
changeset
|
335 if close_branch: |
f75f7d39cca3
testing: generate tests operations using Hypothesis
David R. MacIver <david@drmaciver.com>
parents:
diff
changeset
|
336 command.append("--close-branch") |
f75f7d39cca3
testing: generate tests operations using Hypothesis
David R. MacIver <david@drmaciver.com>
parents:
diff
changeset
|
337 errors.append("can only close branch heads") |
f75f7d39cca3
testing: generate tests operations using Hypothesis
David R. MacIver <david@drmaciver.com>
parents:
diff
changeset
|
338 if addremove: |
f75f7d39cca3
testing: generate tests operations using Hypothesis
David R. MacIver <david@drmaciver.com>
parents:
diff
changeset
|
339 command.append("--addremove") |
f75f7d39cca3
testing: generate tests operations using Hypothesis
David R. MacIver <david@drmaciver.com>
parents:
diff
changeset
|
340 if when is not None: |
f75f7d39cca3
testing: generate tests operations using Hypothesis
David R. MacIver <david@drmaciver.com>
parents:
diff
changeset
|
341 if when.year == 1970: |
f75f7d39cca3
testing: generate tests operations using Hypothesis
David R. MacIver <david@drmaciver.com>
parents:
diff
changeset
|
342 errors.append('negative date value') |
f75f7d39cca3
testing: generate tests operations using Hypothesis
David R. MacIver <david@drmaciver.com>
parents:
diff
changeset
|
343 if when.year == 2038: |
f75f7d39cca3
testing: generate tests operations using Hypothesis
David R. MacIver <david@drmaciver.com>
parents:
diff
changeset
|
344 errors.append('exceeds 32 bits') |
f75f7d39cca3
testing: generate tests operations using Hypothesis
David R. MacIver <david@drmaciver.com>
parents:
diff
changeset
|
345 command.append("--date=%s" % ( |
f75f7d39cca3
testing: generate tests operations using Hypothesis
David R. MacIver <david@drmaciver.com>
parents:
diff
changeset
|
346 when.strftime('%Y-%m-%d %H:%M:%S %z'),)) |
f75f7d39cca3
testing: generate tests operations using Hypothesis
David R. MacIver <david@drmaciver.com>
parents:
diff
changeset
|
347 |
f75f7d39cca3
testing: generate tests operations using Hypothesis
David R. MacIver <david@drmaciver.com>
parents:
diff
changeset
|
348 with acceptableerrors(*errors): |
f75f7d39cca3
testing: generate tests operations using Hypothesis
David R. MacIver <david@drmaciver.com>
parents:
diff
changeset
|
349 self.hg(*command) |
f75f7d39cca3
testing: generate tests operations using Hypothesis
David R. MacIver <david@drmaciver.com>
parents:
diff
changeset
|
350 |
f75f7d39cca3
testing: generate tests operations using Hypothesis
David R. MacIver <david@drmaciver.com>
parents:
diff
changeset
|
351 # Section: Simple side effect free "check" operations |
f75f7d39cca3
testing: generate tests operations using Hypothesis
David R. MacIver <david@drmaciver.com>
parents:
diff
changeset
|
352 @rule() |
f75f7d39cca3
testing: generate tests operations using Hypothesis
David R. MacIver <david@drmaciver.com>
parents:
diff
changeset
|
353 def log(self): |
f75f7d39cca3
testing: generate tests operations using Hypothesis
David R. MacIver <david@drmaciver.com>
parents:
diff
changeset
|
354 self.hg("log") |
f75f7d39cca3
testing: generate tests operations using Hypothesis
David R. MacIver <david@drmaciver.com>
parents:
diff
changeset
|
355 |
f75f7d39cca3
testing: generate tests operations using Hypothesis
David R. MacIver <david@drmaciver.com>
parents:
diff
changeset
|
356 @rule() |
f75f7d39cca3
testing: generate tests operations using Hypothesis
David R. MacIver <david@drmaciver.com>
parents:
diff
changeset
|
357 def verify(self): |
f75f7d39cca3
testing: generate tests operations using Hypothesis
David R. MacIver <david@drmaciver.com>
parents:
diff
changeset
|
358 self.hg("verify") |
f75f7d39cca3
testing: generate tests operations using Hypothesis
David R. MacIver <david@drmaciver.com>
parents:
diff
changeset
|
359 |
f75f7d39cca3
testing: generate tests operations using Hypothesis
David R. MacIver <david@drmaciver.com>
parents:
diff
changeset
|
360 @rule() |
f75f7d39cca3
testing: generate tests operations using Hypothesis
David R. MacIver <david@drmaciver.com>
parents:
diff
changeset
|
361 def diff(self): |
f75f7d39cca3
testing: generate tests operations using Hypothesis
David R. MacIver <david@drmaciver.com>
parents:
diff
changeset
|
362 self.hg("diff", "--nodates") |
f75f7d39cca3
testing: generate tests operations using Hypothesis
David R. MacIver <david@drmaciver.com>
parents:
diff
changeset
|
363 |
f75f7d39cca3
testing: generate tests operations using Hypothesis
David R. MacIver <david@drmaciver.com>
parents:
diff
changeset
|
364 @rule() |
f75f7d39cca3
testing: generate tests operations using Hypothesis
David R. MacIver <david@drmaciver.com>
parents:
diff
changeset
|
365 def status(self): |
f75f7d39cca3
testing: generate tests operations using Hypothesis
David R. MacIver <david@drmaciver.com>
parents:
diff
changeset
|
366 self.hg("status") |
f75f7d39cca3
testing: generate tests operations using Hypothesis
David R. MacIver <david@drmaciver.com>
parents:
diff
changeset
|
367 |
f75f7d39cca3
testing: generate tests operations using Hypothesis
David R. MacIver <david@drmaciver.com>
parents:
diff
changeset
|
368 @rule() |
f75f7d39cca3
testing: generate tests operations using Hypothesis
David R. MacIver <david@drmaciver.com>
parents:
diff
changeset
|
369 def export(self): |
f75f7d39cca3
testing: generate tests operations using Hypothesis
David R. MacIver <david@drmaciver.com>
parents:
diff
changeset
|
370 self.hg("export") |
f75f7d39cca3
testing: generate tests operations using Hypothesis
David R. MacIver <david@drmaciver.com>
parents:
diff
changeset
|
371 |
f75f7d39cca3
testing: generate tests operations using Hypothesis
David R. MacIver <david@drmaciver.com>
parents:
diff
changeset
|
372 settings.register_profile( |
f75f7d39cca3
testing: generate tests operations using Hypothesis
David R. MacIver <david@drmaciver.com>
parents:
diff
changeset
|
373 'default', settings( |
f75f7d39cca3
testing: generate tests operations using Hypothesis
David R. MacIver <david@drmaciver.com>
parents:
diff
changeset
|
374 timeout=300, |
f75f7d39cca3
testing: generate tests operations using Hypothesis
David R. MacIver <david@drmaciver.com>
parents:
diff
changeset
|
375 stateful_step_count=50, |
f75f7d39cca3
testing: generate tests operations using Hypothesis
David R. MacIver <david@drmaciver.com>
parents:
diff
changeset
|
376 max_examples=10, |
f75f7d39cca3
testing: generate tests operations using Hypothesis
David R. MacIver <david@drmaciver.com>
parents:
diff
changeset
|
377 ) |
f75f7d39cca3
testing: generate tests operations using Hypothesis
David R. MacIver <david@drmaciver.com>
parents:
diff
changeset
|
378 ) |
f75f7d39cca3
testing: generate tests operations using Hypothesis
David R. MacIver <david@drmaciver.com>
parents:
diff
changeset
|
379 |
f75f7d39cca3
testing: generate tests operations using Hypothesis
David R. MacIver <david@drmaciver.com>
parents:
diff
changeset
|
380 settings.register_profile( |
f75f7d39cca3
testing: generate tests operations using Hypothesis
David R. MacIver <david@drmaciver.com>
parents:
diff
changeset
|
381 'fast', settings( |
f75f7d39cca3
testing: generate tests operations using Hypothesis
David R. MacIver <david@drmaciver.com>
parents:
diff
changeset
|
382 timeout=10, |
f75f7d39cca3
testing: generate tests operations using Hypothesis
David R. MacIver <david@drmaciver.com>
parents:
diff
changeset
|
383 stateful_step_count=20, |
f75f7d39cca3
testing: generate tests operations using Hypothesis
David R. MacIver <david@drmaciver.com>
parents:
diff
changeset
|
384 max_examples=5, |
f75f7d39cca3
testing: generate tests operations using Hypothesis
David R. MacIver <david@drmaciver.com>
parents:
diff
changeset
|
385 min_satisfying_examples=1, |
f75f7d39cca3
testing: generate tests operations using Hypothesis
David R. MacIver <david@drmaciver.com>
parents:
diff
changeset
|
386 max_shrinks=0, |
f75f7d39cca3
testing: generate tests operations using Hypothesis
David R. MacIver <david@drmaciver.com>
parents:
diff
changeset
|
387 ) |
f75f7d39cca3
testing: generate tests operations using Hypothesis
David R. MacIver <david@drmaciver.com>
parents:
diff
changeset
|
388 ) |
f75f7d39cca3
testing: generate tests operations using Hypothesis
David R. MacIver <david@drmaciver.com>
parents:
diff
changeset
|
389 |
f75f7d39cca3
testing: generate tests operations using Hypothesis
David R. MacIver <david@drmaciver.com>
parents:
diff
changeset
|
390 settings.load_profile(os.getenv('HYPOTHESIS_PROFILE', 'default')) |
f75f7d39cca3
testing: generate tests operations using Hypothesis
David R. MacIver <david@drmaciver.com>
parents:
diff
changeset
|
391 |
f75f7d39cca3
testing: generate tests operations using Hypothesis
David R. MacIver <david@drmaciver.com>
parents:
diff
changeset
|
392 verifyingtest = verifyingstatemachine.TestCase |
f75f7d39cca3
testing: generate tests operations using Hypothesis
David R. MacIver <david@drmaciver.com>
parents:
diff
changeset
|
393 |
f75f7d39cca3
testing: generate tests operations using Hypothesis
David R. MacIver <david@drmaciver.com>
parents:
diff
changeset
|
394 verifyingtest.settings = settings.default |
f75f7d39cca3
testing: generate tests operations using Hypothesis
David R. MacIver <david@drmaciver.com>
parents:
diff
changeset
|
395 |
f75f7d39cca3
testing: generate tests operations using Hypothesis
David R. MacIver <david@drmaciver.com>
parents:
diff
changeset
|
396 if __name__ == '__main__': |
f75f7d39cca3
testing: generate tests operations using Hypothesis
David R. MacIver <david@drmaciver.com>
parents:
diff
changeset
|
397 try: |
f75f7d39cca3
testing: generate tests operations using Hypothesis
David R. MacIver <david@drmaciver.com>
parents:
diff
changeset
|
398 silenttestrunner.main(__name__) |
f75f7d39cca3
testing: generate tests operations using Hypothesis
David R. MacIver <david@drmaciver.com>
parents:
diff
changeset
|
399 finally: |
f75f7d39cca3
testing: generate tests operations using Hypothesis
David R. MacIver <david@drmaciver.com>
parents:
diff
changeset
|
400 # So as to prevent proliferation of useless test files, if we never |
f75f7d39cca3
testing: generate tests operations using Hypothesis
David R. MacIver <david@drmaciver.com>
parents:
diff
changeset
|
401 # actually wrote a failing test we clean up after ourselves and delete |
f75f7d39cca3
testing: generate tests operations using Hypothesis
David R. MacIver <david@drmaciver.com>
parents:
diff
changeset
|
402 # the file for doing so that we owned. |
f75f7d39cca3
testing: generate tests operations using Hypothesis
David R. MacIver <david@drmaciver.com>
parents:
diff
changeset
|
403 if os.path.exists(savefile) and os.path.getsize(savefile) == 0: |
f75f7d39cca3
testing: generate tests operations using Hypothesis
David R. MacIver <david@drmaciver.com>
parents:
diff
changeset
|
404 os.unlink(savefile) |