tests/test-run-tests.py
author Pierre-Yves David <pierre-yves.david@octobus.net>
Mon, 30 Dec 2024 22:10:02 +0100
changeset 52589 484a4d5d360e
parent 48875 6000f5b25c9b
permissions -rw-r--r--
run-tests: no longer rely on global variable for python path We move that value as an attribute to the test runner and the tests, this will allow more flexibility in later changeset, especially to make sure we use the venv in a more traditional way in the future.
Ignore whitespace changes - Everywhere: Within whitespace: At end of lines:
20271
4453d08a616a tests: new test for line matching functions in run-tests
Simon Heimberg <simohe@besonet.ch>
parents:
diff changeset
     1
"""test line matching with some failing examples and some which warn
4453d08a616a tests: new test for line matching functions in run-tests
Simon Heimberg <simohe@besonet.ch>
parents:
diff changeset
     2
4453d08a616a tests: new test for line matching functions in run-tests
Simon Heimberg <simohe@besonet.ch>
parents:
diff changeset
     3
run-test.t only checks positive matches and can not see warnings
4453d08a616a tests: new test for line matching functions in run-tests
Simon Heimberg <simohe@besonet.ch>
parents:
diff changeset
     4
(both by design)
4453d08a616a tests: new test for line matching functions in run-tests
Simon Heimberg <simohe@besonet.ch>
parents:
diff changeset
     5
"""
4453d08a616a tests: new test for line matching functions in run-tests
Simon Heimberg <simohe@besonet.ch>
parents:
diff changeset
     6
28917
f798ffe7cb08 tests: make test-run-tests use absolute_import
Pulkit Goyal <7895pulkit@gmail.com>
parents: 25061
diff changeset
     7
import doctest
f798ffe7cb08 tests: make test-run-tests use absolute_import
Pulkit Goyal <7895pulkit@gmail.com>
parents: 25061
diff changeset
     8
import os
f798ffe7cb08 tests: make test-run-tests use absolute_import
Pulkit Goyal <7895pulkit@gmail.com>
parents: 25061
diff changeset
     9
import re
52589
484a4d5d360e run-tests: no longer rely on global variable for python path
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 48875
diff changeset
    10
import sys
43076
2372284d9457 formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents: 41759
diff changeset
    11
20284
e1e6ddaef299 tests: fix test-run-tests.py on OS X
Simon Heimberg <simohe@besonet.ch>
parents: 20274
diff changeset
    12
# this is hack to make sure no escape characters are inserted into the output
e1e6ddaef299 tests: fix test-run-tests.py on OS X
Simon Heimberg <simohe@besonet.ch>
parents: 20274
diff changeset
    13
if 'TERM' in os.environ:
e1e6ddaef299 tests: fix test-run-tests.py on OS X
Simon Heimberg <simohe@besonet.ch>
parents: 20274
diff changeset
    14
    del os.environ['TERM']
20271
4453d08a616a tests: new test for line matching functions in run-tests
Simon Heimberg <simohe@besonet.ch>
parents:
diff changeset
    15
run_tests = __import__('run-tests')
4453d08a616a tests: new test for line matching functions in run-tests
Simon Heimberg <simohe@besonet.ch>
parents:
diff changeset
    16
43076
2372284d9457 formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents: 41759
diff changeset
    17
25061
625dd917f04f test-run-tests: fix for Python 3.5
Augie Fackler <augie@google.com>
parents: 21315
diff changeset
    18
def prn(ex):
625dd917f04f test-run-tests: fix for Python 3.5
Augie Fackler <augie@google.com>
parents: 21315
diff changeset
    19
    m = ex.args[0]
625dd917f04f test-run-tests: fix for Python 3.5
Augie Fackler <augie@google.com>
parents: 21315
diff changeset
    20
    if isinstance(m, str):
625dd917f04f test-run-tests: fix for Python 3.5
Augie Fackler <augie@google.com>
parents: 21315
diff changeset
    21
        print(m)
625dd917f04f test-run-tests: fix for Python 3.5
Augie Fackler <augie@google.com>
parents: 21315
diff changeset
    22
    else:
625dd917f04f test-run-tests: fix for Python 3.5
Augie Fackler <augie@google.com>
parents: 21315
diff changeset
    23
        print(m.decode('utf-8'))
625dd917f04f test-run-tests: fix for Python 3.5
Augie Fackler <augie@google.com>
parents: 21315
diff changeset
    24
43076
2372284d9457 formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents: 41759
diff changeset
    25
20271
4453d08a616a tests: new test for line matching functions in run-tests
Simon Heimberg <simohe@besonet.ch>
parents:
diff changeset
    26
def lm(expected, output):
4453d08a616a tests: new test for line matching functions in run-tests
Simon Heimberg <simohe@besonet.ch>
parents:
diff changeset
    27
    r"""check if output matches expected
4453d08a616a tests: new test for line matching functions in run-tests
Simon Heimberg <simohe@besonet.ch>
parents:
diff changeset
    28
4453d08a616a tests: new test for line matching functions in run-tests
Simon Heimberg <simohe@besonet.ch>
parents:
diff changeset
    29
    does it generally work?
25061
625dd917f04f test-run-tests: fix for Python 3.5
Augie Fackler <augie@google.com>
parents: 21315
diff changeset
    30
        >>> lm(b'H*e (glob)\n', b'Here\n')
20271
4453d08a616a tests: new test for line matching functions in run-tests
Simon Heimberg <simohe@besonet.ch>
parents:
diff changeset
    31
        True
4453d08a616a tests: new test for line matching functions in run-tests
Simon Heimberg <simohe@besonet.ch>
parents:
diff changeset
    32
4453d08a616a tests: new test for line matching functions in run-tests
Simon Heimberg <simohe@besonet.ch>
parents:
diff changeset
    33
    fail on bad test data
25061
625dd917f04f test-run-tests: fix for Python 3.5
Augie Fackler <augie@google.com>
parents: 21315
diff changeset
    34
        >>> try: lm(b'a\n',b'a')
625dd917f04f test-run-tests: fix for Python 3.5
Augie Fackler <augie@google.com>
parents: 21315
diff changeset
    35
        ... except AssertionError as ex: print(ex)
20271
4453d08a616a tests: new test for line matching functions in run-tests
Simon Heimberg <simohe@besonet.ch>
parents:
diff changeset
    36
        missing newline
25061
625dd917f04f test-run-tests: fix for Python 3.5
Augie Fackler <augie@google.com>
parents: 21315
diff changeset
    37
        >>> try: lm(b'single backslash\n', b'single \backslash\n')
625dd917f04f test-run-tests: fix for Python 3.5
Augie Fackler <augie@google.com>
parents: 21315
diff changeset
    38
        ... except AssertionError as ex: prn(ex)
20271
4453d08a616a tests: new test for line matching functions in run-tests
Simon Heimberg <simohe@besonet.ch>
parents:
diff changeset
    39
        single backslash or unknown char
4453d08a616a tests: new test for line matching functions in run-tests
Simon Heimberg <simohe@besonet.ch>
parents:
diff changeset
    40
    """
43076
2372284d9457 formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents: 41759
diff changeset
    41
    assert expected.endswith(b'\n') and output.endswith(
2372284d9457 formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents: 41759
diff changeset
    42
        b'\n'
2372284d9457 formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents: 41759
diff changeset
    43
    ), 'missing newline'
2372284d9457 formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents: 41759
diff changeset
    44
    assert not re.search(
2372284d9457 formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents: 41759
diff changeset
    45
        br'[^ \w\\/\r\n()*?]', expected + output
2372284d9457 formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents: 41759
diff changeset
    46
    ), b'single backslash or unknown char'
52589
484a4d5d360e run-tests: no longer rely on global variable for python path
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 48875
diff changeset
    47
    test = run_tests.TTest(
484a4d5d360e run-tests: no longer rely on global variable for python path
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 48875
diff changeset
    48
        b'test-run-test.t', b'.', b'.', python=sys.executable
484a4d5d360e run-tests: no longer rely on global variable for python path
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 48875
diff changeset
    49
    )
38554
f83600efa1ca tests: don't allow reodering of glob/re lines across non-glob/re lines
Martin von Zweigbergk <martinvonz@google.com>
parents: 35382
diff changeset
    50
    match, exact = test.linematch(expected, output)
20273
d9d6cbbeef0d run-tests: suggest to append glob when only path sep does not match
Simon Heimberg <simohe@besonet.ch>
parents: 20271
diff changeset
    51
    if isinstance(match, str):
d9d6cbbeef0d run-tests: suggest to append glob when only path sep does not match
Simon Heimberg <simohe@besonet.ch>
parents: 20271
diff changeset
    52
        return 'special: ' + match
25061
625dd917f04f test-run-tests: fix for Python 3.5
Augie Fackler <augie@google.com>
parents: 21315
diff changeset
    53
    elif isinstance(match, bytes):
625dd917f04f test-run-tests: fix for Python 3.5
Augie Fackler <augie@google.com>
parents: 21315
diff changeset
    54
        return 'special: ' + match.decode('utf-8')
20273
d9d6cbbeef0d run-tests: suggest to append glob when only path sep does not match
Simon Heimberg <simohe@besonet.ch>
parents: 20271
diff changeset
    55
    else:
43076
2372284d9457 formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents: 41759
diff changeset
    56
        return bool(match)  # do not return match object
2372284d9457 formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents: 41759
diff changeset
    57
20271
4453d08a616a tests: new test for line matching functions in run-tests
Simon Heimberg <simohe@besonet.ch>
parents:
diff changeset
    58
4453d08a616a tests: new test for line matching functions in run-tests
Simon Heimberg <simohe@besonet.ch>
parents:
diff changeset
    59
def wintests():
4453d08a616a tests: new test for line matching functions in run-tests
Simon Heimberg <simohe@besonet.ch>
parents:
diff changeset
    60
    r"""test matching like running on windows
4453d08a616a tests: new test for line matching functions in run-tests
Simon Heimberg <simohe@besonet.ch>
parents:
diff changeset
    61
4453d08a616a tests: new test for line matching functions in run-tests
Simon Heimberg <simohe@besonet.ch>
parents:
diff changeset
    62
    enable windows matching on any os
4453d08a616a tests: new test for line matching functions in run-tests
Simon Heimberg <simohe@besonet.ch>
parents:
diff changeset
    63
        >>> _osaltsep = os.altsep
4453d08a616a tests: new test for line matching functions in run-tests
Simon Heimberg <simohe@besonet.ch>
parents:
diff changeset
    64
        >>> os.altsep = True
35381
14fd435763ee run-tests: suggest a (glob) for os.path.sep mismatches with '\r\n' EOL too
Matt Harbison <matt_harbison@yahoo.com>
parents: 33710
diff changeset
    65
        >>> _osname = os.name
14fd435763ee run-tests: suggest a (glob) for os.path.sep mismatches with '\r\n' EOL too
Matt Harbison <matt_harbison@yahoo.com>
parents: 33710
diff changeset
    66
        >>> os.name = 'nt'
47573
75b623801f6a run-tests: use a global WINDOWS constant instead of multiple tests
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 43076
diff changeset
    67
        >>> _old_windows = run_tests.WINDOWS
75b623801f6a run-tests: use a global WINDOWS constant instead of multiple tests
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 43076
diff changeset
    68
        >>> run_tests.WINDOWS = True
20271
4453d08a616a tests: new test for line matching functions in run-tests
Simon Heimberg <simohe@besonet.ch>
parents:
diff changeset
    69
4453d08a616a tests: new test for line matching functions in run-tests
Simon Heimberg <simohe@besonet.ch>
parents:
diff changeset
    70
    valid match on windows
25061
625dd917f04f test-run-tests: fix for Python 3.5
Augie Fackler <augie@google.com>
parents: 21315
diff changeset
    71
        >>> lm(b'g/a*/d (glob)\n', b'g\\abc/d\n')
20271
4453d08a616a tests: new test for line matching functions in run-tests
Simon Heimberg <simohe@besonet.ch>
parents:
diff changeset
    72
        True
4453d08a616a tests: new test for line matching functions in run-tests
Simon Heimberg <simohe@besonet.ch>
parents:
diff changeset
    73
4453d08a616a tests: new test for line matching functions in run-tests
Simon Heimberg <simohe@besonet.ch>
parents:
diff changeset
    74
    direct matching, glob unnecessary
25061
625dd917f04f test-run-tests: fix for Python 3.5
Augie Fackler <augie@google.com>
parents: 21315
diff changeset
    75
        >>> lm(b'g/b (glob)\n', b'g/b\n')
20274
7a259dfe24f7 run-tests: print more information on unnecessary glob matching
Simon Heimberg <simohe@besonet.ch>
parents: 20273
diff changeset
    76
        'special: -glob'
20271
4453d08a616a tests: new test for line matching functions in run-tests
Simon Heimberg <simohe@besonet.ch>
parents:
diff changeset
    77
4453d08a616a tests: new test for line matching functions in run-tests
Simon Heimberg <simohe@besonet.ch>
parents:
diff changeset
    78
    missing glob
25061
625dd917f04f test-run-tests: fix for Python 3.5
Augie Fackler <augie@google.com>
parents: 21315
diff changeset
    79
        >>> lm(b'/g/c/d/fg\n', b'\\g\\c\\d/fg\n')
35382
dfae14354660 run-tests: accept '\' vs '/' path differences without '(glob)'
Matt Harbison <matt_harbison@yahoo.com>
parents: 35381
diff changeset
    80
        True
35381
14fd435763ee run-tests: suggest a (glob) for os.path.sep mismatches with '\r\n' EOL too
Matt Harbison <matt_harbison@yahoo.com>
parents: 33710
diff changeset
    81
        >>> lm(b'/g/c/d/fg\n', b'\\g\\c\\d\\fg\r\n')
35382
dfae14354660 run-tests: accept '\' vs '/' path differences without '(glob)'
Matt Harbison <matt_harbison@yahoo.com>
parents: 35381
diff changeset
    82
        True
20271
4453d08a616a tests: new test for line matching functions in run-tests
Simon Heimberg <simohe@besonet.ch>
parents:
diff changeset
    83
4453d08a616a tests: new test for line matching functions in run-tests
Simon Heimberg <simohe@besonet.ch>
parents:
diff changeset
    84
    restore os.altsep
4453d08a616a tests: new test for line matching functions in run-tests
Simon Heimberg <simohe@besonet.ch>
parents:
diff changeset
    85
        >>> os.altsep = _osaltsep
35381
14fd435763ee run-tests: suggest a (glob) for os.path.sep mismatches with '\r\n' EOL too
Matt Harbison <matt_harbison@yahoo.com>
parents: 33710
diff changeset
    86
        >>> os.name = _osname
47573
75b623801f6a run-tests: use a global WINDOWS constant instead of multiple tests
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 43076
diff changeset
    87
        >>> run_tests.WINDOWS = _old_windows
20271
4453d08a616a tests: new test for line matching functions in run-tests
Simon Heimberg <simohe@besonet.ch>
parents:
diff changeset
    88
    """
20284
e1e6ddaef299 tests: fix test-run-tests.py on OS X
Simon Heimberg <simohe@besonet.ch>
parents: 20274
diff changeset
    89
    pass
20271
4453d08a616a tests: new test for line matching functions in run-tests
Simon Heimberg <simohe@besonet.ch>
parents:
diff changeset
    90
43076
2372284d9457 formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents: 41759
diff changeset
    91
20271
4453d08a616a tests: new test for line matching functions in run-tests
Simon Heimberg <simohe@besonet.ch>
parents:
diff changeset
    92
def otherostests():
4453d08a616a tests: new test for line matching functions in run-tests
Simon Heimberg <simohe@besonet.ch>
parents:
diff changeset
    93
    r"""test matching like running on non-windows os
4453d08a616a tests: new test for line matching functions in run-tests
Simon Heimberg <simohe@besonet.ch>
parents:
diff changeset
    94
4453d08a616a tests: new test for line matching functions in run-tests
Simon Heimberg <simohe@besonet.ch>
parents:
diff changeset
    95
    disable windows matching on any os
4453d08a616a tests: new test for line matching functions in run-tests
Simon Heimberg <simohe@besonet.ch>
parents:
diff changeset
    96
        >>> _osaltsep = os.altsep
4453d08a616a tests: new test for line matching functions in run-tests
Simon Heimberg <simohe@besonet.ch>
parents:
diff changeset
    97
        >>> os.altsep = False
35381
14fd435763ee run-tests: suggest a (glob) for os.path.sep mismatches with '\r\n' EOL too
Matt Harbison <matt_harbison@yahoo.com>
parents: 33710
diff changeset
    98
        >>> _osname = os.name
14fd435763ee run-tests: suggest a (glob) for os.path.sep mismatches with '\r\n' EOL too
Matt Harbison <matt_harbison@yahoo.com>
parents: 33710
diff changeset
    99
        >>> os.name = 'nt'
20271
4453d08a616a tests: new test for line matching functions in run-tests
Simon Heimberg <simohe@besonet.ch>
parents:
diff changeset
   100
4453d08a616a tests: new test for line matching functions in run-tests
Simon Heimberg <simohe@besonet.ch>
parents:
diff changeset
   101
    backslash does not match slash
25061
625dd917f04f test-run-tests: fix for Python 3.5
Augie Fackler <augie@google.com>
parents: 21315
diff changeset
   102
        >>> lm(b'h/a* (glob)\n', b'h\\ab\n')
20271
4453d08a616a tests: new test for line matching functions in run-tests
Simon Heimberg <simohe@besonet.ch>
parents:
diff changeset
   103
        False
4453d08a616a tests: new test for line matching functions in run-tests
Simon Heimberg <simohe@besonet.ch>
parents:
diff changeset
   104
4453d08a616a tests: new test for line matching functions in run-tests
Simon Heimberg <simohe@besonet.ch>
parents:
diff changeset
   105
    direct matching glob can not be recognized
25061
625dd917f04f test-run-tests: fix for Python 3.5
Augie Fackler <augie@google.com>
parents: 21315
diff changeset
   106
        >>> lm(b'h/b (glob)\n', b'h/b\n')
20271
4453d08a616a tests: new test for line matching functions in run-tests
Simon Heimberg <simohe@besonet.ch>
parents:
diff changeset
   107
        True
4453d08a616a tests: new test for line matching functions in run-tests
Simon Heimberg <simohe@besonet.ch>
parents:
diff changeset
   108
4453d08a616a tests: new test for line matching functions in run-tests
Simon Heimberg <simohe@besonet.ch>
parents:
diff changeset
   109
    missing glob can not not be recognized
25061
625dd917f04f test-run-tests: fix for Python 3.5
Augie Fackler <augie@google.com>
parents: 21315
diff changeset
   110
        >>> lm(b'/h/c/df/g/\n', b'\\h/c\\df/g\\\n')
20271
4453d08a616a tests: new test for line matching functions in run-tests
Simon Heimberg <simohe@besonet.ch>
parents:
diff changeset
   111
        False
4453d08a616a tests: new test for line matching functions in run-tests
Simon Heimberg <simohe@besonet.ch>
parents:
diff changeset
   112
4453d08a616a tests: new test for line matching functions in run-tests
Simon Heimberg <simohe@besonet.ch>
parents:
diff changeset
   113
    restore os.altsep
4453d08a616a tests: new test for line matching functions in run-tests
Simon Heimberg <simohe@besonet.ch>
parents:
diff changeset
   114
        >>> os.altsep = _osaltsep
35381
14fd435763ee run-tests: suggest a (glob) for os.path.sep mismatches with '\r\n' EOL too
Matt Harbison <matt_harbison@yahoo.com>
parents: 33710
diff changeset
   115
        >>> os.name = _osname
20271
4453d08a616a tests: new test for line matching functions in run-tests
Simon Heimberg <simohe@besonet.ch>
parents:
diff changeset
   116
    """
4453d08a616a tests: new test for line matching functions in run-tests
Simon Heimberg <simohe@besonet.ch>
parents:
diff changeset
   117
    pass
4453d08a616a tests: new test for line matching functions in run-tests
Simon Heimberg <simohe@besonet.ch>
parents:
diff changeset
   118
43076
2372284d9457 formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents: 41759
diff changeset
   119
20271
4453d08a616a tests: new test for line matching functions in run-tests
Simon Heimberg <simohe@besonet.ch>
parents:
diff changeset
   120
if __name__ == '__main__':
4453d08a616a tests: new test for line matching functions in run-tests
Simon Heimberg <simohe@besonet.ch>
parents:
diff changeset
   121
    doctest.testmod()