mercurial-scm/hg-stable: contrib/import-checker.py annotate

annotate contrib/import-checker.py @ 52669:e627cc25b6f3

pyupgrade: rewrite `yield` statements in a loop to `yield from` This is the `legacy` fixer in `pyupgrade`, with the `yield` statement yielding loop commented back in. This seems to help pytype in some cases, and hurt it in others. But that can be manually fixed later. Note that it's possibly buggy in that it aggressively changed `import-checker.py` to `yield from 'fcntl', 'grp', 'pwd', 'select', 'termios': # Unix only`, which is invalid syntax. Possibly it needed help from the token fixer that I've disabled locally (because that wants to make a bunch of unrelated changes). Just change those few places to yield from a list, to avoid having to constantly revert that.

author	Matt Harbison <matt_harbison@yahoo.com>
date	Sun, 05 Jan 2025 22:26:16 -0500
parents	0cc50d9ac6b0
children	b95b12cb31e2

rev	line source
45849 c102b704edb5 global: use python3 in shebangs Gregory Szorc <gregory.szorc@gmail.com> parents: 44440 diff changeset	1 #!/usr/bin/env python3
26954 f804bf27439b import-checker: make it executable for convenience Yuya Nishihara <yuya@tcha.org> parents: 26781 diff changeset	2
28702 e44f671018e3 py3: use absolute_import in import-checker timeless <timeless@mozdev.org> parents: 28700 diff changeset	3
20036 e5d51fa51aba contrib: add an import checker Augie Fackler <raf@durin42.com> parents: diff changeset	4 import ast
26965 1fa66d3ad28d import-checker: reset context to verify convention in function scope Yuya Nishihara <yuya@tcha.org> parents: 26964 diff changeset	5 import collections
43415 a8454e846736 import-checker: open all source files as utf-8 Gregory Szorc <gregory.szorc@gmail.com> parents: 43084 diff changeset	6 import io
20036 e5d51fa51aba contrib: add an import checker Augie Fackler <raf@durin42.com> parents: diff changeset	7 import os
e5d51fa51aba contrib: add an import checker Augie Fackler <raf@durin42.com> parents: diff changeset	8 import sys
e5d51fa51aba contrib: add an import checker Augie Fackler <raf@durin42.com> parents: diff changeset	9
20198 f5393a9dc4e5 import-checker: make test-module-imports.t work using virtualenv (issue4129) Chris Jerdonek <chris.jerdonek@gmail.com> parents: 20197 diff changeset	10 # Import a minimal set of stdlib modules needed for list_stdlib_modules()
f5393a9dc4e5 import-checker: make test-module-imports.t work using virtualenv (issue4129) Chris Jerdonek <chris.jerdonek@gmail.com> parents: 20197 diff changeset	11 # to work when run from a virtualenv. The modules were chosen empirically
f5393a9dc4e5 import-checker: make test-module-imports.t work using virtualenv (issue4129) Chris Jerdonek <chris.jerdonek@gmail.com> parents: 20197 diff changeset	12 # so that the return value matches the return value without virtualenv.
43075 57875cf423c9 style: run a patched black on a subset of mercurial Augie Fackler <augie@google.com> parents: 42823 diff changeset	13 if True: # disable lexical sorting checks
51373 cf1bee12ecdb import-checker: make stdlib path detection work in virtual environments Manuel Jacob <me@manueljacob.de> parents: 50800 diff changeset	14 import argparse
29211 b42c2a66a698 py3: make contrib/import-checker.py get along with itself Yuya Nishihara <yuya@tcha.org> parents: 29208 diff changeset	15 import zlib
20198 f5393a9dc4e5 import-checker: make test-module-imports.t work using virtualenv (issue4129) Chris Jerdonek <chris.jerdonek@gmail.com> parents: 20197 diff changeset	16
40096 7288838bec1f import-checker: use testparseutil.embedded() to centralize detection logic FUJIWARA Katsunori <foozy@lares.dti.ne.jp> parents: 39356 diff changeset	17 import testparseutil
7288838bec1f import-checker: use testparseutil.embedded() to centralize detection logic FUJIWARA Katsunori <foozy@lares.dti.ne.jp> parents: 39356 diff changeset	18
49052 8dec9abf2669 import-checker: allow symbol imports from typing module Gregory Szorc <gregory.szorc@gmail.com> parents: 48970 diff changeset	19 # Allow list of modules that symbols can be directly imported from.
25703 1a6a117d0b95 import-checker: establish modern import convention Gregory Szorc <gregory.szorc@gmail.com> parents: 25702 diff changeset	20 allowsymbolimports = (
1a6a117d0b95 import-checker: establish modern import convention Gregory Szorc <gregory.szorc@gmail.com> parents: 25702 diff changeset	21 '__future__',
47383 26127236b229 convert-bazaar: use breezy package instead of old bzr one Rapha?l Gom?s <rgomes@octobus.net> parents: 45849 diff changeset	22 'breezy',
48925 029b76d645dc imports: allow importing futures from concurrent Augie Fackler <augie@google.com> parents: 47797 diff changeset	23 'concurrent',
33933 2d64b2f1787b contrib: allow symbol imports from hgclient for tests Augie Fackler <raf@durin42.com> parents: 33915 diff changeset	24 'hgclient',
33911 c9cf69d0c3b9 contrib: allow importing "symbols" from mercurial Augie Fackler <raf@durin42.com> parents: 33908 diff changeset	25 'mercurial',
27018 e5be48dd8215 import-checker: allow symbol imports from hgweb.common and .request Yuya Nishihara <yuya@tcha.org> parents: 26965 diff changeset	26 'mercurial.hgweb.common',
e5be48dd8215 import-checker: allow symbol imports from hgweb.common and .request Yuya Nishihara <yuya@tcha.org> parents: 26965 diff changeset	27 'mercurial.hgweb.request',
25703 1a6a117d0b95 import-checker: establish modern import convention Gregory Szorc <gregory.szorc@gmail.com> parents: 25702 diff changeset	28 'mercurial.i18n',
42823 268662aac075 interfaces: create a new folder for interfaces and move repository.py in it Pulkit Goyal <pulkit@yandex-team.ru> parents: 42745 diff changeset	29 'mercurial.interfaces',
25703 1a6a117d0b95 import-checker: establish modern import convention Gregory Szorc <gregory.szorc@gmail.com> parents: 25702 diff changeset	30 'mercurial.node',
43084 c2e284cee333 import-checker: allow symbol imports from mercurial.pycompat Gregory Szorc <gregory.szorc@gmail.com> parents: 43075 diff changeset	31 'mercurial.pycompat',
39356 729082bb9938 revlog: split constants into a new `revlogutils.constants` module Boris Feld <boris.feld@octobus.net> parents: 38837 diff changeset	32 # for revlog to re-export constant to extensions
729082bb9938 revlog: split constants into a new `revlogutils.constants` module Boris Feld <boris.feld@octobus.net> parents: 38837 diff changeset	33 'mercurial.revlogutils.constants',
42745 ca5ca3badd3c flagutil: create a `mercurial.revlogutils.flagutil` module Pierre-Yves David <pierre-yves.david@octobus.net> parents: 42407 diff changeset	34 'mercurial.revlogutils.flagutil',
32540 95085d747db8 import-checker: allow importing symbols from pure modules Yuya Nishihara <yuya@tcha.org> parents: 32458 diff changeset	35 # for cffi modules to re-export pure functions
95085d747db8 import-checker: allow importing symbols from pure modules Yuya Nishihara <yuya@tcha.org> parents: 32458 diff changeset	36 'mercurial.pure.base85',
95085d747db8 import-checker: allow importing symbols from pure modules Yuya Nishihara <yuya@tcha.org> parents: 32458 diff changeset	37 'mercurial.pure.bdiff',
95085d747db8 import-checker: allow importing symbols from pure modules Yuya Nishihara <yuya@tcha.org> parents: 32458 diff changeset	38 'mercurial.pure.mpatch',
95085d747db8 import-checker: allow importing symbols from pure modules Yuya Nishihara <yuya@tcha.org> parents: 32458 diff changeset	39 'mercurial.pure.osutil',
95085d747db8 import-checker: allow importing symbols from pure modules Yuya Nishihara <yuya@tcha.org> parents: 32458 diff changeset	40 'mercurial.pure.parsers',
34396 41401f502c83 tests: disable lints on mercurial/thirdparty Siddharth Agarwal <sid0@fb.com> parents: 34055 diff changeset	41 # third-party imports should be directly imported
41401f502c83 tests: disable lints on mercurial/thirdparty Siddharth Agarwal <sid0@fb.com> parents: 34055 diff changeset	42 'mercurial.thirdparty',
38837 8751d1e2a7ff util: create a context manager to handle timing Martijn Pieters <mj@zopatista.com> parents: 37711 diff changeset	43 'mercurial.thirdparty.attr',
50442 82e5a9b1ef1e extras: re-use Projection from jaraco.collections Jason R. Coombs <jaraco@jaraco.com> parents: 50275 diff changeset	44 'mercurial.thirdparty.jaraco.collections',
50800 2c34c9b61a4f thirdparty: vendor tomli Rapha?l Gom?s <rgomes@octobus.net> parents: 50442 diff changeset	45 'mercurial.thirdparty.tomli',
49052 8dec9abf2669 import-checker: allow symbol imports from typing module Gregory Szorc <gregory.szorc@gmail.com> parents: 48970 diff changeset	46 'typing',
49369 1572f790ee5e convert: remove old ElementTree import cruft from darcs Ian Moody <moz-ian@perix.co.uk> parents: 49052 diff changeset	47 'xml.etree.ElementTree',
25703 1a6a117d0b95 import-checker: establish modern import convention Gregory Szorc <gregory.szorc@gmail.com> parents: 25702 diff changeset	48 )
1a6a117d0b95 import-checker: establish modern import convention Gregory Szorc <gregory.szorc@gmail.com> parents: 25702 diff changeset	49
49052 8dec9abf2669 import-checker: allow symbol imports from typing module Gregory Szorc <gregory.szorc@gmail.com> parents: 48970 diff changeset	50 # Allow list of symbols that can be directly imported.
43075 57875cf423c9 style: run a patched black on a subset of mercurial Augie Fackler <augie@google.com> parents: 42823 diff changeset	51 directsymbols = ('demandimport',)
32457 d02888308235 import-checker: add a way to directly import certain symbols Siddharth Agarwal <sid0@fb.com> parents: 32413 diff changeset	52
25703 1a6a117d0b95 import-checker: establish modern import convention Gregory Szorc <gregory.szorc@gmail.com> parents: 25702 diff changeset	53 # Modules that must be aliased because they are commonly confused with
1a6a117d0b95 import-checker: establish modern import convention Gregory Szorc <gregory.szorc@gmail.com> parents: 25702 diff changeset	54 # common variables and can create aliasing and readability issues.
1a6a117d0b95 import-checker: establish modern import convention Gregory Szorc <gregory.szorc@gmail.com> parents: 25702 diff changeset	55 requirealias = {
1a6a117d0b95 import-checker: establish modern import convention Gregory Szorc <gregory.szorc@gmail.com> parents: 25702 diff changeset	56 'ui': 'uimod',
1a6a117d0b95 import-checker: establish modern import convention Gregory Szorc <gregory.szorc@gmail.com> parents: 25702 diff changeset	57 }
1a6a117d0b95 import-checker: establish modern import convention Gregory Szorc <gregory.szorc@gmail.com> parents: 25702 diff changeset	58
43075 57875cf423c9 style: run a patched black on a subset of mercurial Augie Fackler <augie@google.com> parents: 42823 diff changeset	59
26965 1fa66d3ad28d import-checker: reset context to verify convention in function scope Yuya Nishihara <yuya@tcha.org> parents: 26964 diff changeset	60 def walklocal(root):
1fa66d3ad28d import-checker: reset context to verify convention in function scope Yuya Nishihara <yuya@tcha.org> parents: 26964 diff changeset	61 """Recursively yield all descendant nodes but not in a different scope"""
1fa66d3ad28d import-checker: reset context to verify convention in function scope Yuya Nishihara <yuya@tcha.org> parents: 26964 diff changeset	62 todo = collections.deque(ast.iter_child_nodes(root))
1fa66d3ad28d import-checker: reset context to verify convention in function scope Yuya Nishihara <yuya@tcha.org> parents: 26964 diff changeset	63 yield root, False
1fa66d3ad28d import-checker: reset context to verify convention in function scope Yuya Nishihara <yuya@tcha.org> parents: 26964 diff changeset	64 while todo:
1fa66d3ad28d import-checker: reset context to verify convention in function scope Yuya Nishihara <yuya@tcha.org> parents: 26964 diff changeset	65 node = todo.popleft()
1fa66d3ad28d import-checker: reset context to verify convention in function scope Yuya Nishihara <yuya@tcha.org> parents: 26964 diff changeset	66 newscope = isinstance(node, ast.FunctionDef)
1fa66d3ad28d import-checker: reset context to verify convention in function scope Yuya Nishihara <yuya@tcha.org> parents: 26964 diff changeset	67 if not newscope:
1fa66d3ad28d import-checker: reset context to verify convention in function scope Yuya Nishihara <yuya@tcha.org> parents: 26964 diff changeset	68 todo.extend(ast.iter_child_nodes(node))
1fa66d3ad28d import-checker: reset context to verify convention in function scope Yuya Nishihara <yuya@tcha.org> parents: 26964 diff changeset	69 yield node, newscope
1fa66d3ad28d import-checker: reset context to verify convention in function scope Yuya Nishihara <yuya@tcha.org> parents: 26964 diff changeset	70
43075 57875cf423c9 style: run a patched black on a subset of mercurial Augie Fackler <augie@google.com> parents: 42823 diff changeset	71
32413 194b0f781132 import-checker: drop workaround for pure modules Yuya Nishihara <yuya@tcha.org> parents: 32411 diff changeset	72 def dotted_name_of_path(path):
20036 e5d51fa51aba contrib: add an import checker Augie Fackler <raf@durin42.com> parents: diff changeset	73 """Given a relative path to a source file, return its dotted module name.
e5d51fa51aba contrib: add an import checker Augie Fackler <raf@durin42.com> parents: diff changeset	74
e5d51fa51aba contrib: add an import checker Augie Fackler <raf@durin42.com> parents: diff changeset	75 >>> dotted_name_of_path('mercurial/error.py')
e5d51fa51aba contrib: add an import checker Augie Fackler <raf@durin42.com> parents: diff changeset	76 'mercurial.error'
20383 4990abb4729d import-checker: fix names of dynamically loaded modules Mads Kiilerich <madski@unity3d.com> parents: 20238 diff changeset	77 >>> dotted_name_of_path('zlibmodule.so')
4990abb4729d import-checker: fix names of dynamically loaded modules Mads Kiilerich <madski@unity3d.com> parents: 20238 diff changeset	78 'zlib'
20036 e5d51fa51aba contrib: add an import checker Augie Fackler <raf@durin42.com> parents: diff changeset	79 """
27620 0c60843b55b5 import-checker: normalize directory separator to get module name on Windows Yuya Nishihara <yuya@tcha.org> parents: 27520 diff changeset	80 parts = path.replace(os.sep, '/').split('/')
43075 57875cf423c9 style: run a patched black on a subset of mercurial Augie Fackler <augie@google.com> parents: 42823 diff changeset	81 parts[-1] = parts[-1].split('.', 1)[0] # remove .py and .so and .ARCH.so
20383 4990abb4729d import-checker: fix names of dynamically loaded modules Mads Kiilerich <madski@unity3d.com> parents: 20238 diff changeset	82 if parts[-1].endswith('module'):
4990abb4729d import-checker: fix names of dynamically loaded modules Mads Kiilerich <madski@unity3d.com> parents: 20238 diff changeset	83 parts[-1] = parts[-1][:-6]
20036 e5d51fa51aba contrib: add an import checker Augie Fackler <raf@durin42.com> parents: diff changeset	84 return '.'.join(parts)
e5d51fa51aba contrib: add an import checker Augie Fackler <raf@durin42.com> parents: diff changeset	85
43075 57875cf423c9 style: run a patched black on a subset of mercurial Augie Fackler <augie@google.com> parents: 42823 diff changeset	86
25173 7358b5d9991e import-checker: add utility to examine what module is imported easily FUJIWARA Katsunori <foozy@lares.dti.ne.jp> parents: 25064 diff changeset	87 def fromlocalfunc(modulename, localmods):
7358b5d9991e import-checker: add utility to examine what module is imported easily FUJIWARA Katsunori <foozy@lares.dti.ne.jp> parents: 25064 diff changeset	88 """Get a function to examine which locally defined module the
7358b5d9991e import-checker: add utility to examine what module is imported easily FUJIWARA Katsunori <foozy@lares.dti.ne.jp> parents: 25064 diff changeset	89 target source imports via a specified name.
7358b5d9991e import-checker: add utility to examine what module is imported easily FUJIWARA Katsunori <foozy@lares.dti.ne.jp> parents: 25064 diff changeset	90
7358b5d9991e import-checker: add utility to examine what module is imported easily FUJIWARA Katsunori <foozy@lares.dti.ne.jp> parents: 25064 diff changeset	91 `modulename` is an `dotted_name_of_path()`-ed source file path,
7358b5d9991e import-checker: add utility to examine what module is imported easily FUJIWARA Katsunori <foozy@lares.dti.ne.jp> parents: 25064 diff changeset	92 which may have `.__init__` at the end of it, of the target source.
7358b5d9991e import-checker: add utility to examine what module is imported easily FUJIWARA Katsunori <foozy@lares.dti.ne.jp> parents: 25064 diff changeset	93
32541 4c712b90c60a import-checker: convert localmods to a set of module names Yuya Nishihara <yuya@tcha.org> parents: 32540 diff changeset	94 `localmods` is a set of absolute `dotted_name_of_path()`-ed source file
4c712b90c60a import-checker: convert localmods to a set of module names Yuya Nishihara <yuya@tcha.org> parents: 32540 diff changeset	95 paths of locally defined (= Mercurial specific) modules.
25173 7358b5d9991e import-checker: add utility to examine what module is imported easily FUJIWARA Katsunori <foozy@lares.dti.ne.jp> parents: 25064 diff changeset	96
7358b5d9991e import-checker: add utility to examine what module is imported easily FUJIWARA Katsunori <foozy@lares.dti.ne.jp> parents: 25064 diff changeset	97 This function assumes that module names not existing in
26781 1aee2ab0f902 spelling: trivial spell checking Mads Kiilerich <madski@unity3d.com> parents: 26221 diff changeset	98 `localmods` are from the Python standard library.
25173 7358b5d9991e import-checker: add utility to examine what module is imported easily FUJIWARA Katsunori <foozy@lares.dti.ne.jp> parents: 25064 diff changeset	99
7358b5d9991e import-checker: add utility to examine what module is imported easily FUJIWARA Katsunori <foozy@lares.dti.ne.jp> parents: 25064 diff changeset	100 This function returns the function, which takes `name` argument,
7358b5d9991e import-checker: add utility to examine what module is imported easily FUJIWARA Katsunori <foozy@lares.dti.ne.jp> parents: 25064 diff changeset	101 and returns `(absname, dottedpath, hassubmod)` tuple if `name`
7358b5d9991e import-checker: add utility to examine what module is imported easily FUJIWARA Katsunori <foozy@lares.dti.ne.jp> parents: 25064 diff changeset	102 matches against locally defined module. Otherwise, it returns
7358b5d9991e import-checker: add utility to examine what module is imported easily FUJIWARA Katsunori <foozy@lares.dti.ne.jp> parents: 25064 diff changeset	103 False.
7358b5d9991e import-checker: add utility to examine what module is imported easily FUJIWARA Katsunori <foozy@lares.dti.ne.jp> parents: 25064 diff changeset	104
7358b5d9991e import-checker: add utility to examine what module is imported easily FUJIWARA Katsunori <foozy@lares.dti.ne.jp> parents: 25064 diff changeset	105 It is assumed that `name` doesn't have `.__init__`.
7358b5d9991e import-checker: add utility to examine what module is imported easily FUJIWARA Katsunori <foozy@lares.dti.ne.jp> parents: 25064 diff changeset	106
7358b5d9991e import-checker: add utility to examine what module is imported easily FUJIWARA Katsunori <foozy@lares.dti.ne.jp> parents: 25064 diff changeset	107 `absname` is an absolute module name of specified `name`
7358b5d9991e import-checker: add utility to examine what module is imported easily FUJIWARA Katsunori <foozy@lares.dti.ne.jp> parents: 25064 diff changeset	108 (e.g. "hgext.convert"). This can be used to compose prefix for sub
7358b5d9991e import-checker: add utility to examine what module is imported easily FUJIWARA Katsunori <foozy@lares.dti.ne.jp> parents: 25064 diff changeset	109 modules or so.
7358b5d9991e import-checker: add utility to examine what module is imported easily FUJIWARA Katsunori <foozy@lares.dti.ne.jp> parents: 25064 diff changeset	110
7358b5d9991e import-checker: add utility to examine what module is imported easily FUJIWARA Katsunori <foozy@lares.dti.ne.jp> parents: 25064 diff changeset	111 `dottedpath` is a `dotted_name_of_path()`-ed source file path
7358b5d9991e import-checker: add utility to examine what module is imported easily FUJIWARA Katsunori <foozy@lares.dti.ne.jp> parents: 25064 diff changeset	112 (e.g. "hgext.convert.__init__") of `name`. This is used to look
7358b5d9991e import-checker: add utility to examine what module is imported easily FUJIWARA Katsunori <foozy@lares.dti.ne.jp> parents: 25064 diff changeset	113 module up in `localmods` again.
7358b5d9991e import-checker: add utility to examine what module is imported easily FUJIWARA Katsunori <foozy@lares.dti.ne.jp> parents: 25064 diff changeset	114
7358b5d9991e import-checker: add utility to examine what module is imported easily FUJIWARA Katsunori <foozy@lares.dti.ne.jp> parents: 25064 diff changeset	115 `hassubmod` is whether it may have sub modules under it (for
7358b5d9991e import-checker: add utility to examine what module is imported easily FUJIWARA Katsunori <foozy@lares.dti.ne.jp> parents: 25064 diff changeset	116 convenient, even though this is also equivalent to "absname !=
7358b5d9991e import-checker: add utility to examine what module is imported easily FUJIWARA Katsunori <foozy@lares.dti.ne.jp> parents: 25064 diff changeset	117 dottednpath")
7358b5d9991e import-checker: add utility to examine what module is imported easily FUJIWARA Katsunori <foozy@lares.dti.ne.jp> parents: 25064 diff changeset	118
32541 4c712b90c60a import-checker: convert localmods to a set of module names Yuya Nishihara <yuya@tcha.org> parents: 32540 diff changeset	119 >>> localmods = {'foo.__init__', 'foo.foo1',
4c712b90c60a import-checker: convert localmods to a set of module names Yuya Nishihara <yuya@tcha.org> parents: 32540 diff changeset	120 ... 'foo.bar.__init__', 'foo.bar.bar1',
4c712b90c60a import-checker: convert localmods to a set of module names Yuya Nishihara <yuya@tcha.org> parents: 32540 diff changeset	121 ... 'baz.__init__', 'baz.baz1'}
25173 7358b5d9991e import-checker: add utility to examine what module is imported easily FUJIWARA Katsunori <foozy@lares.dti.ne.jp> parents: 25064 diff changeset	122 >>> fromlocal = fromlocalfunc('foo.xxx', localmods)
7358b5d9991e import-checker: add utility to examine what module is imported easily FUJIWARA Katsunori <foozy@lares.dti.ne.jp> parents: 25064 diff changeset	123 >>> # relative
7358b5d9991e import-checker: add utility to examine what module is imported easily FUJIWARA Katsunori <foozy@lares.dti.ne.jp> parents: 25064 diff changeset	124 >>> fromlocal('foo1')
7358b5d9991e import-checker: add utility to examine what module is imported easily FUJIWARA Katsunori <foozy@lares.dti.ne.jp> parents: 25064 diff changeset	125 ('foo.foo1', 'foo.foo1', False)
7358b5d9991e import-checker: add utility to examine what module is imported easily FUJIWARA Katsunori <foozy@lares.dti.ne.jp> parents: 25064 diff changeset	126 >>> fromlocal('bar')
7358b5d9991e import-checker: add utility to examine what module is imported easily FUJIWARA Katsunori <foozy@lares.dti.ne.jp> parents: 25064 diff changeset	127 ('foo.bar', 'foo.bar.__init__', True)
7358b5d9991e import-checker: add utility to examine what module is imported easily FUJIWARA Katsunori <foozy@lares.dti.ne.jp> parents: 25064 diff changeset	128 >>> fromlocal('bar.bar1')
7358b5d9991e import-checker: add utility to examine what module is imported easily FUJIWARA Katsunori <foozy@lares.dti.ne.jp> parents: 25064 diff changeset	129 ('foo.bar.bar1', 'foo.bar.bar1', False)
7358b5d9991e import-checker: add utility to examine what module is imported easily FUJIWARA Katsunori <foozy@lares.dti.ne.jp> parents: 25064 diff changeset	130 >>> # absolute
7358b5d9991e import-checker: add utility to examine what module is imported easily FUJIWARA Katsunori <foozy@lares.dti.ne.jp> parents: 25064 diff changeset	131 >>> fromlocal('baz')
7358b5d9991e import-checker: add utility to examine what module is imported easily FUJIWARA Katsunori <foozy@lares.dti.ne.jp> parents: 25064 diff changeset	132 ('baz', 'baz.__init__', True)
7358b5d9991e import-checker: add utility to examine what module is imported easily FUJIWARA Katsunori <foozy@lares.dti.ne.jp> parents: 25064 diff changeset	133 >>> fromlocal('baz.baz1')
7358b5d9991e import-checker: add utility to examine what module is imported easily FUJIWARA Katsunori <foozy@lares.dti.ne.jp> parents: 25064 diff changeset	134 ('baz.baz1', 'baz.baz1', False)
7358b5d9991e import-checker: add utility to examine what module is imported easily FUJIWARA Katsunori <foozy@lares.dti.ne.jp> parents: 25064 diff changeset	135 >>> # unknown = maybe standard library
7358b5d9991e import-checker: add utility to examine what module is imported easily FUJIWARA Katsunori <foozy@lares.dti.ne.jp> parents: 25064 diff changeset	136 >>> fromlocal('os')
7358b5d9991e import-checker: add utility to examine what module is imported easily FUJIWARA Katsunori <foozy@lares.dti.ne.jp> parents: 25064 diff changeset	137 False
25701 1f88c0f6ff5a import-checker: resolve relative imports Gregory Szorc <gregory.szorc@gmail.com> parents: 25660 diff changeset	138 >>> fromlocal(None, 1)
1f88c0f6ff5a import-checker: resolve relative imports Gregory Szorc <gregory.szorc@gmail.com> parents: 25660 diff changeset	139 ('foo', 'foo.__init__', True)
29122 660d8d4ec7aa import-checker: recognize relative imports from parents of current package liscju <piotr.listkiewicz@gmail.com> parents: 28922 diff changeset	140 >>> fromlocal('foo1', 1)
660d8d4ec7aa import-checker: recognize relative imports from parents of current package liscju <piotr.listkiewicz@gmail.com> parents: 28922 diff changeset	141 ('foo.foo1', 'foo.foo1', False)
25701 1f88c0f6ff5a import-checker: resolve relative imports Gregory Szorc <gregory.szorc@gmail.com> parents: 25660 diff changeset	142 >>> fromlocal2 = fromlocalfunc('foo.xxx.yyy', localmods)
1f88c0f6ff5a import-checker: resolve relative imports Gregory Szorc <gregory.szorc@gmail.com> parents: 25660 diff changeset	143 >>> fromlocal2(None, 2)
1f88c0f6ff5a import-checker: resolve relative imports Gregory Szorc <gregory.szorc@gmail.com> parents: 25660 diff changeset	144 ('foo', 'foo.__init__', True)
29122 660d8d4ec7aa import-checker: recognize relative imports from parents of current package liscju <piotr.listkiewicz@gmail.com> parents: 28922 diff changeset	145 >>> fromlocal2('bar2', 1)
660d8d4ec7aa import-checker: recognize relative imports from parents of current package liscju <piotr.listkiewicz@gmail.com> parents: 28922 diff changeset	146 False
660d8d4ec7aa import-checker: recognize relative imports from parents of current package liscju <piotr.listkiewicz@gmail.com> parents: 28922 diff changeset	147 >>> fromlocal2('bar', 2)
660d8d4ec7aa import-checker: recognize relative imports from parents of current package liscju <piotr.listkiewicz@gmail.com> parents: 28922 diff changeset	148 ('foo.bar', 'foo.bar.__init__', True)
25173 7358b5d9991e import-checker: add utility to examine what module is imported easily FUJIWARA Katsunori <foozy@lares.dti.ne.jp> parents: 25064 diff changeset	149 """
33908 3de9a2df6411 contrib: have import-checker work mostly with native strings for mod names Augie Fackler <raf@durin42.com> parents: 33896 diff changeset	150 if not isinstance(modulename, str):
3de9a2df6411 contrib: have import-checker work mostly with native strings for mod names Augie Fackler <raf@durin42.com> parents: 33896 diff changeset	151 modulename = modulename.decode('ascii')
25173 7358b5d9991e import-checker: add utility to examine what module is imported easily FUJIWARA Katsunori <foozy@lares.dti.ne.jp> parents: 25064 diff changeset	152 prefix = '.'.join(modulename.split('.')[:-1])
7358b5d9991e import-checker: add utility to examine what module is imported easily FUJIWARA Katsunori <foozy@lares.dti.ne.jp> parents: 25064 diff changeset	153 if prefix:
7358b5d9991e import-checker: add utility to examine what module is imported easily FUJIWARA Katsunori <foozy@lares.dti.ne.jp> parents: 25064 diff changeset	154 prefix += '.'
43075 57875cf423c9 style: run a patched black on a subset of mercurial Augie Fackler <augie@google.com> parents: 42823 diff changeset	155
25701 1f88c0f6ff5a import-checker: resolve relative imports Gregory Szorc <gregory.szorc@gmail.com> parents: 25660 diff changeset	156 def fromlocal(name, level=0):
29374 7712fcde2d56 import-checker: increase portability for python 2.6.x FUJIWARA Katsunori <foozy@lares.dti.ne.jp> parents: 29234 diff changeset	157 # name is false value when relative imports are used.
7712fcde2d56 import-checker: increase portability for python 2.6.x FUJIWARA Katsunori <foozy@lares.dti.ne.jp> parents: 29234 diff changeset	158 if not name:
25701 1f88c0f6ff5a import-checker: resolve relative imports Gregory Szorc <gregory.szorc@gmail.com> parents: 25660 diff changeset	159 # If relative imports are used, level must not be absolute.
1f88c0f6ff5a import-checker: resolve relative imports Gregory Szorc <gregory.szorc@gmail.com> parents: 25660 diff changeset	160 assert level > 0
1f88c0f6ff5a import-checker: resolve relative imports Gregory Szorc <gregory.szorc@gmail.com> parents: 25660 diff changeset	161 candidates = ['.'.join(modulename.split('.')[:-level])]
1f88c0f6ff5a import-checker: resolve relative imports Gregory Szorc <gregory.szorc@gmail.com> parents: 25660 diff changeset	162 else:
29122 660d8d4ec7aa import-checker: recognize relative imports from parents of current package liscju <piotr.listkiewicz@gmail.com> parents: 28922 diff changeset	163 if not level:
660d8d4ec7aa import-checker: recognize relative imports from parents of current package liscju <piotr.listkiewicz@gmail.com> parents: 28922 diff changeset	164 # Check relative name first.
660d8d4ec7aa import-checker: recognize relative imports from parents of current package liscju <piotr.listkiewicz@gmail.com> parents: 28922 diff changeset	165 candidates = [prefix + name, name]
660d8d4ec7aa import-checker: recognize relative imports from parents of current package liscju <piotr.listkiewicz@gmail.com> parents: 28922 diff changeset	166 else:
43075 57875cf423c9 style: run a patched black on a subset of mercurial Augie Fackler <augie@google.com> parents: 42823 diff changeset	167 candidates = [
57875cf423c9 style: run a patched black on a subset of mercurial Augie Fackler <augie@google.com> parents: 42823 diff changeset	168 '.'.join(modulename.split('.')[:-level]) + '.' + name
57875cf423c9 style: run a patched black on a subset of mercurial Augie Fackler <augie@google.com> parents: 42823 diff changeset	169 ]
25701 1f88c0f6ff5a import-checker: resolve relative imports Gregory Szorc <gregory.szorc@gmail.com> parents: 25660 diff changeset	170
1f88c0f6ff5a import-checker: resolve relative imports Gregory Szorc <gregory.szorc@gmail.com> parents: 25660 diff changeset	171 for n in candidates:
25173 7358b5d9991e import-checker: add utility to examine what module is imported easily FUJIWARA Katsunori <foozy@lares.dti.ne.jp> parents: 25064 diff changeset	172 if n in localmods:
7358b5d9991e import-checker: add utility to examine what module is imported easily FUJIWARA Katsunori <foozy@lares.dti.ne.jp> parents: 25064 diff changeset	173 return (n, n, False)
7358b5d9991e import-checker: add utility to examine what module is imported easily FUJIWARA Katsunori <foozy@lares.dti.ne.jp> parents: 25064 diff changeset	174 dottedpath = n + '.__init__'
7358b5d9991e import-checker: add utility to examine what module is imported easily FUJIWARA Katsunori <foozy@lares.dti.ne.jp> parents: 25064 diff changeset	175 if dottedpath in localmods:
7358b5d9991e import-checker: add utility to examine what module is imported easily FUJIWARA Katsunori <foozy@lares.dti.ne.jp> parents: 25064 diff changeset	176 return (n, dottedpath, True)
7358b5d9991e import-checker: add utility to examine what module is imported easily FUJIWARA Katsunori <foozy@lares.dti.ne.jp> parents: 25064 diff changeset	177 return False
43075 57875cf423c9 style: run a patched black on a subset of mercurial Augie Fackler <augie@google.com> parents: 42823 diff changeset	178
25173 7358b5d9991e import-checker: add utility to examine what module is imported easily FUJIWARA Katsunori <foozy@lares.dti.ne.jp> parents: 25064 diff changeset	179 return fromlocal
20036 e5d51fa51aba contrib: add an import checker Augie Fackler <raf@durin42.com> parents: diff changeset	180
43075 57875cf423c9 style: run a patched black on a subset of mercurial Augie Fackler <augie@google.com> parents: 42823 diff changeset	181
32542 a025ec43856c import-checker: guess names of C extension modules Yuya Nishihara <yuya@tcha.org> parents: 32541 diff changeset	182 def populateextmods(localmods):
a025ec43856c import-checker: guess names of C extension modules Yuya Nishihara <yuya@tcha.org> parents: 32541 diff changeset	183 """Populate C extension modules based on pure modules"""
a025ec43856c import-checker: guess names of C extension modules Yuya Nishihara <yuya@tcha.org> parents: 32541 diff changeset	184 newlocalmods = set(localmods)
a025ec43856c import-checker: guess names of C extension modules Yuya Nishihara <yuya@tcha.org> parents: 32541 diff changeset	185 for n in localmods:
a025ec43856c import-checker: guess names of C extension modules Yuya Nishihara <yuya@tcha.org> parents: 32541 diff changeset	186 if n.startswith('mercurial.pure.'):
43075 57875cf423c9 style: run a patched black on a subset of mercurial Augie Fackler <augie@google.com> parents: 42823 diff changeset	187 m = n[len('mercurial.pure.') :]
32542 a025ec43856c import-checker: guess names of C extension modules Yuya Nishihara <yuya@tcha.org> parents: 32541 diff changeset	188 newlocalmods.add('mercurial.cext.' + m)
a025ec43856c import-checker: guess names of C extension modules Yuya Nishihara <yuya@tcha.org> parents: 32541 diff changeset	189 newlocalmods.add('mercurial.cffi._' + m)
a025ec43856c import-checker: guess names of C extension modules Yuya Nishihara <yuya@tcha.org> parents: 32541 diff changeset	190 return newlocalmods
a025ec43856c import-checker: guess names of C extension modules Yuya Nishihara <yuya@tcha.org> parents: 32541 diff changeset	191
43075 57875cf423c9 style: run a patched black on a subset of mercurial Augie Fackler <augie@google.com> parents: 42823 diff changeset	192
20036 e5d51fa51aba contrib: add an import checker Augie Fackler <raf@durin42.com> parents: diff changeset	193 def list_stdlib_modules():
e5d51fa51aba contrib: add an import checker Augie Fackler <raf@durin42.com> parents: diff changeset	194 """List the modules present in the stdlib.
e5d51fa51aba contrib: add an import checker Augie Fackler <raf@durin42.com> parents: diff changeset	195
e5d51fa51aba contrib: add an import checker Augie Fackler <raf@durin42.com> parents: diff changeset	196 >>> mods = set(list_stdlib_modules())
51374 617af10994fb py3: fully port doctest to py3 Manuel Jacob <me@manueljacob.de> parents: 51373 diff changeset	197 >>> 'http' in mods
20036 e5d51fa51aba contrib: add an import checker Augie Fackler <raf@durin42.com> parents: diff changeset	198 True
e5d51fa51aba contrib: add an import checker Augie Fackler <raf@durin42.com> parents: diff changeset	199
e5d51fa51aba contrib: add an import checker Augie Fackler <raf@durin42.com> parents: diff changeset	200 os.path isn't really a module, so it's missing:
e5d51fa51aba contrib: add an import checker Augie Fackler <raf@durin42.com> parents: diff changeset	201
e5d51fa51aba contrib: add an import checker Augie Fackler <raf@durin42.com> parents: diff changeset	202 >>> 'os.path' in mods
e5d51fa51aba contrib: add an import checker Augie Fackler <raf@durin42.com> parents: diff changeset	203 False
e5d51fa51aba contrib: add an import checker Augie Fackler <raf@durin42.com> parents: diff changeset	204
e5d51fa51aba contrib: add an import checker Augie Fackler <raf@durin42.com> parents: diff changeset	205 sys requires special treatment, because it's baked into the
e5d51fa51aba contrib: add an import checker Augie Fackler <raf@durin42.com> parents: diff changeset	206 interpreter, but it should still appear:
e5d51fa51aba contrib: add an import checker Augie Fackler <raf@durin42.com> parents: diff changeset	207
e5d51fa51aba contrib: add an import checker Augie Fackler <raf@durin42.com> parents: diff changeset	208 >>> 'sys' in mods
e5d51fa51aba contrib: add an import checker Augie Fackler <raf@durin42.com> parents: diff changeset	209 True
e5d51fa51aba contrib: add an import checker Augie Fackler <raf@durin42.com> parents: diff changeset	210
e5d51fa51aba contrib: add an import checker Augie Fackler <raf@durin42.com> parents: diff changeset	211 >>> 'collections' in mods
e5d51fa51aba contrib: add an import checker Augie Fackler <raf@durin42.com> parents: diff changeset	212 True
e5d51fa51aba contrib: add an import checker Augie Fackler <raf@durin42.com> parents: diff changeset	213
51374 617af10994fb py3: fully port doctest to py3 Manuel Jacob <me@manueljacob.de> parents: 51373 diff changeset	214 >>> 'array' in mods
20036 e5d51fa51aba contrib: add an import checker Augie Fackler <raf@durin42.com> parents: diff changeset	215 True
29395 4c8026babe8c import-checker: ensure cffi is always a system module Augie Fackler <raf@durin42.com> parents: 29374 diff changeset	216
4c8026babe8c import-checker: ensure cffi is always a system module Augie Fackler <raf@durin42.com> parents: 29374 diff changeset	217 >>> 'cffi' in mods
4c8026babe8c import-checker: ensure cffi is always a system module Augie Fackler <raf@durin42.com> parents: 29374 diff changeset	218 True
20036 e5d51fa51aba contrib: add an import checker Augie Fackler <raf@durin42.com> parents: diff changeset	219 """
52669 e627cc25b6f3 pyupgrade: rewrite `yield` statements in a loop to `yield from` Matt Harbison <matt_harbison@yahoo.com> parents: 52472 diff changeset	220 yield from sys.builtin_module_names
20036 e5d51fa51aba contrib: add an import checker Augie Fackler <raf@durin42.com> parents: diff changeset	221 # These modules only exist on windows, but we should always
e5d51fa51aba contrib: add an import checker Augie Fackler <raf@durin42.com> parents: diff changeset	222 # consider them stdlib.
52669 e627cc25b6f3 pyupgrade: rewrite `yield` statements in a loop to `yield from` Matt Harbison <matt_harbison@yahoo.com> parents: 52472 diff changeset	223 yield from ['msvcrt', '_winreg']
33912 c856cb1c29be contrib: inform import checker that __builtin__ is a thing Augie Fackler <raf@durin42.com> parents: 33911 diff changeset	224 yield '__builtin__'
43075 57875cf423c9 style: run a patched black on a subset of mercurial Augie Fackler <augie@google.com> parents: 42823 diff changeset	225 yield 'builtins' # python3 only
57875cf423c9 style: run a patched black on a subset of mercurial Augie Fackler <augie@google.com> parents: 42823 diff changeset	226 yield 'importlib.abc' # python3 only
57875cf423c9 style: run a patched black on a subset of mercurial Augie Fackler <augie@google.com> parents: 42823 diff changeset	227 yield 'importlib.machinery' # python3 only
57875cf423c9 style: run a patched black on a subset of mercurial Augie Fackler <augie@google.com> parents: 42823 diff changeset	228 yield 'importlib.util' # python3 only
50275 ada9a0245fd7 run-tests: fix a crash when using the coverage options Matt Harbison <matt_harbison@yahoo.com> parents: 49369 diff changeset	229 yield 'packaging.version'
52669 e627cc25b6f3 pyupgrade: rewrite `yield` statements in a loop to `yield from` Matt Harbison <matt_harbison@yahoo.com> parents: 52472 diff changeset	230 yield from ['fcntl', 'grp', 'pwd', 'select', 'termios']
e627cc25b6f3 pyupgrade: rewrite `yield` statements in a loop to `yield from` Matt Harbison <matt_harbison@yahoo.com> parents: 52472 diff changeset	231 yield from ['cPickle', 'datetime']
e627cc25b6f3 pyupgrade: rewrite `yield` statements in a loop to `yield from` Matt Harbison <matt_harbison@yahoo.com> parents: 52472 diff changeset	232 yield from ['cffi']
51659 d9faa71a57f6 test-check: don't report distutils as a local import Pierre-Yves David <pierre-yves.david@octobus.net> parents: 51374 diff changeset	233 yield 'distutils' # in Python < 3.12
d9faa71a57f6 test-check: don't report distutils as a local import Pierre-Yves David <pierre-yves.david@octobus.net> parents: 51374 diff changeset	234 yield 'distutils.version' # in Python < 3.12
32331 bd872f64a8ba cleanup: use set literals Martin von Zweigbergk <martinvonz@google.com> parents: 32252 diff changeset	235 stdlib_prefixes = {sys.prefix, sys.exec_prefix}
20198 f5393a9dc4e5 import-checker: make test-module-imports.t work using virtualenv (issue4129) Chris Jerdonek <chris.jerdonek@gmail.com> parents: 20197 diff changeset	236 # We need to supplement the list of prefixes for the search to work
f5393a9dc4e5 import-checker: make test-module-imports.t work using virtualenv (issue4129) Chris Jerdonek <chris.jerdonek@gmail.com> parents: 20197 diff changeset	237 # when run from within a virtualenv.
51373 cf1bee12ecdb import-checker: make stdlib path detection work in virtual environments Manuel Jacob <me@manueljacob.de> parents: 50800 diff changeset	238 for mod in (argparse, zlib):
33895 998fad4b3072 contrib: work around some modules not existing on Py3 in import checker Augie Fackler <raf@durin42.com> parents: 32621 diff changeset	239 if mod is None:
998fad4b3072 contrib: work around some modules not existing on Py3 in import checker Augie Fackler <raf@durin42.com> parents: 32621 diff changeset	240 continue
20198 f5393a9dc4e5 import-checker: make test-module-imports.t work using virtualenv (issue4129) Chris Jerdonek <chris.jerdonek@gmail.com> parents: 20197 diff changeset	241 try:
f5393a9dc4e5 import-checker: make test-module-imports.t work using virtualenv (issue4129) Chris Jerdonek <chris.jerdonek@gmail.com> parents: 20197 diff changeset	242 # Not all module objects have a __file__ attribute.
f5393a9dc4e5 import-checker: make test-module-imports.t work using virtualenv (issue4129) Chris Jerdonek <chris.jerdonek@gmail.com> parents: 20197 diff changeset	243 filename = mod.__file__
f5393a9dc4e5 import-checker: make test-module-imports.t work using virtualenv (issue4129) Chris Jerdonek <chris.jerdonek@gmail.com> parents: 20197 diff changeset	244 except AttributeError:
f5393a9dc4e5 import-checker: make test-module-imports.t work using virtualenv (issue4129) Chris Jerdonek <chris.jerdonek@gmail.com> parents: 20197 diff changeset	245 continue
f5393a9dc4e5 import-checker: make test-module-imports.t work using virtualenv (issue4129) Chris Jerdonek <chris.jerdonek@gmail.com> parents: 20197 diff changeset	246 dirname = os.path.dirname(filename)
f5393a9dc4e5 import-checker: make test-module-imports.t work using virtualenv (issue4129) Chris Jerdonek <chris.jerdonek@gmail.com> parents: 20197 diff changeset	247 for prefix in stdlib_prefixes:
f5393a9dc4e5 import-checker: make test-module-imports.t work using virtualenv (issue4129) Chris Jerdonek <chris.jerdonek@gmail.com> parents: 20197 diff changeset	248 if dirname.startswith(prefix):
f5393a9dc4e5 import-checker: make test-module-imports.t work using virtualenv (issue4129) Chris Jerdonek <chris.jerdonek@gmail.com> parents: 20197 diff changeset	249 # Then this directory is redundant.
f5393a9dc4e5 import-checker: make test-module-imports.t work using virtualenv (issue4129) Chris Jerdonek <chris.jerdonek@gmail.com> parents: 20197 diff changeset	250 break
f5393a9dc4e5 import-checker: make test-module-imports.t work using virtualenv (issue4129) Chris Jerdonek <chris.jerdonek@gmail.com> parents: 20197 diff changeset	251 else:
f5393a9dc4e5 import-checker: make test-module-imports.t work using virtualenv (issue4129) Chris Jerdonek <chris.jerdonek@gmail.com> parents: 20197 diff changeset	252 stdlib_prefixes.add(dirname)
40726 dd028bca9221 tests: make test-check-module-imports more robust Valentin Gatien-Baron <vgatien-baron@janestreet.com> parents: 40531 diff changeset	253 sourceroot = os.path.abspath(os.path.dirname(os.path.dirname(__file__)))
20036 e5d51fa51aba contrib: add an import checker Augie Fackler <raf@durin42.com> parents: diff changeset	254 for libpath in sys.path:
40726 dd028bca9221 tests: make test-check-module-imports more robust Valentin Gatien-Baron <vgatien-baron@janestreet.com> parents: 40531 diff changeset	255 # We want to walk everything in sys.path that starts with something in
dd028bca9221 tests: make test-check-module-imports more robust Valentin Gatien-Baron <vgatien-baron@janestreet.com> parents: 40531 diff changeset	256 # stdlib_prefixes, but not directories from the hg sources.
43075 57875cf423c9 style: run a patched black on a subset of mercurial Augie Fackler <augie@google.com> parents: 42823 diff changeset	257 if os.path.abspath(libpath).startswith(sourceroot) or not any(
57875cf423c9 style: run a patched black on a subset of mercurial Augie Fackler <augie@google.com> parents: 42823 diff changeset	258 libpath.startswith(p) for p in stdlib_prefixes
57875cf423c9 style: run a patched black on a subset of mercurial Augie Fackler <augie@google.com> parents: 42823 diff changeset	259 ):
20036 e5d51fa51aba contrib: add an import checker Augie Fackler <raf@durin42.com> parents: diff changeset	260 continue
e5d51fa51aba contrib: add an import checker Augie Fackler <raf@durin42.com> parents: diff changeset	261 for top, dirs, files in os.walk(libpath):
47797 42e2cdb50db0 check-module-imports: ignore non-stdlib module installed by distribution Pierre-Yves David <pierre-yves.david@octobus.net> parents: 47383 diff changeset	262 if 'dist-packages' in top.split(os.path.sep):
42e2cdb50db0 check-module-imports: ignore non-stdlib module installed by distribution Pierre-Yves David <pierre-yves.david@octobus.net> parents: 47383 diff changeset	263 continue
25733 f99c066f5f9a import-checker: recurse into subtree of sys.path only if __init__.py exists Yuya Nishihara <yuya@tcha.org> parents: 25731 diff changeset	264 for i, d in reversed(list(enumerate(dirs))):
43075 57875cf423c9 style: run a patched black on a subset of mercurial Augie Fackler <augie@google.com> parents: 42823 diff changeset	265 if (
57875cf423c9 style: run a patched black on a subset of mercurial Augie Fackler <augie@google.com> parents: 42823 diff changeset	266 not os.path.exists(os.path.join(top, d, '__init__.py'))
57875cf423c9 style: run a patched black on a subset of mercurial Augie Fackler <augie@google.com> parents: 42823 diff changeset	267 or top == libpath
57875cf423c9 style: run a patched black on a subset of mercurial Augie Fackler <augie@google.com> parents: 42823 diff changeset	268 and d in ('hgdemandimport', 'hgext', 'mercurial')
57875cf423c9 style: run a patched black on a subset of mercurial Augie Fackler <augie@google.com> parents: 42823 diff changeset	269 ):
25733 f99c066f5f9a import-checker: recurse into subtree of sys.path only if __init__.py exists Yuya Nishihara <yuya@tcha.org> parents: 25731 diff changeset	270 del dirs[i]
20036 e5d51fa51aba contrib: add an import checker Augie Fackler <raf@durin42.com> parents: diff changeset	271 for name in files:
26221 ae65b1b4cb46 import-checker: use modern .endswith for multiple suffixes Augie Fackler <augie@google.com> parents: 26166 diff changeset	272 if not name.endswith(('.py', '.so', '.pyc', '.pyo', '.pyd')):
20036 e5d51fa51aba contrib: add an import checker Augie Fackler <raf@durin42.com> parents: diff changeset	273 continue
27621 39845b064041 import-checker: list package directory as stdlib module Yuya Nishihara <yuya@tcha.org> parents: 27620 diff changeset	274 if name.startswith('__init__.py'):
39845b064041 import-checker: list package directory as stdlib module Yuya Nishihara <yuya@tcha.org> parents: 27620 diff changeset	275 full_path = top
39845b064041 import-checker: list package directory as stdlib module Yuya Nishihara <yuya@tcha.org> parents: 27620 diff changeset	276 else:
39845b064041 import-checker: list package directory as stdlib module Yuya Nishihara <yuya@tcha.org> parents: 27620 diff changeset	277 full_path = os.path.join(top, name)
43075 57875cf423c9 style: run a patched black on a subset of mercurial Augie Fackler <augie@google.com> parents: 42823 diff changeset	278 rel_path = full_path[len(libpath) + 1 :]
20036 e5d51fa51aba contrib: add an import checker Augie Fackler <raf@durin42.com> parents: diff changeset	279 mod = dotted_name_of_path(rel_path)
e5d51fa51aba contrib: add an import checker Augie Fackler <raf@durin42.com> parents: diff changeset	280 yield mod
e5d51fa51aba contrib: add an import checker Augie Fackler <raf@durin42.com> parents: diff changeset	281
43075 57875cf423c9 style: run a patched black on a subset of mercurial Augie Fackler <augie@google.com> parents: 42823 diff changeset	282
20036 e5d51fa51aba contrib: add an import checker Augie Fackler <raf@durin42.com> parents: diff changeset	283 stdlib_modules = set(list_stdlib_modules())
e5d51fa51aba contrib: add an import checker Augie Fackler <raf@durin42.com> parents: diff changeset	284
43075 57875cf423c9 style: run a patched black on a subset of mercurial Augie Fackler <augie@google.com> parents: 42823 diff changeset	285
28921 02ee31a50002 import-checker: track filenames for SyntaxErrors timeless <timeless@mozdev.org> parents: 28920 diff changeset	286 def imported_modules(source, modulename, f, localmods, ignore_nested=False):
20036 e5d51fa51aba contrib: add an import checker Augie Fackler <raf@durin42.com> parents: diff changeset	287 """Given the source of a file as a string, yield the names
e5d51fa51aba contrib: add an import checker Augie Fackler <raf@durin42.com> parents: diff changeset	288 imported by that file.
e5d51fa51aba contrib: add an import checker Augie Fackler <raf@durin42.com> parents: diff changeset	289
20037 957b43371928 import-checker: ignore nested imports Augie Fackler <raf@durin42.com> parents: 20036 diff changeset	290 Args:
957b43371928 import-checker: ignore nested imports Augie Fackler <raf@durin42.com> parents: 20036 diff changeset	291 source: The python source to examine as a string.
25174 86298718b01c import-checker: make imported_modules yield absolute dotted_name_of_path FUJIWARA Katsunori <foozy@lares.dti.ne.jp> parents: 25173 diff changeset	292 modulename: of specified python source (may have `__init__`)
32541 4c712b90c60a import-checker: convert localmods to a set of module names Yuya Nishihara <yuya@tcha.org> parents: 32540 diff changeset	293 localmods: set of locally defined module names (may have `__init__`)
20037 957b43371928 import-checker: ignore nested imports Augie Fackler <raf@durin42.com> parents: 20036 diff changeset	294 ignore_nested: If true, import statements that do not start in
957b43371928 import-checker: ignore nested imports Augie Fackler <raf@durin42.com> parents: 20036 diff changeset	295 column zero will be ignored.
957b43371928 import-checker: ignore nested imports Augie Fackler <raf@durin42.com> parents: 20036 diff changeset	296
957b43371928 import-checker: ignore nested imports Augie Fackler <raf@durin42.com> parents: 20036 diff changeset	297 Returns:
25174 86298718b01c import-checker: make imported_modules yield absolute dotted_name_of_path FUJIWARA Katsunori <foozy@lares.dti.ne.jp> parents: 25173 diff changeset	298 A list of absolute module names imported by the given source.
20037 957b43371928 import-checker: ignore nested imports Augie Fackler <raf@durin42.com> parents: 20036 diff changeset	299
28921 02ee31a50002 import-checker: track filenames for SyntaxErrors timeless <timeless@mozdev.org> parents: 28920 diff changeset	300 >>> f = 'foo/xxx.py'
25174 86298718b01c import-checker: make imported_modules yield absolute dotted_name_of_path FUJIWARA Katsunori <foozy@lares.dti.ne.jp> parents: 25173 diff changeset	301 >>> modulename = 'foo.xxx'
86298718b01c import-checker: make imported_modules yield absolute dotted_name_of_path FUJIWARA Katsunori <foozy@lares.dti.ne.jp> parents: 25173 diff changeset	302 >>> localmods = {'foo.__init__': True,
86298718b01c import-checker: make imported_modules yield absolute dotted_name_of_path FUJIWARA Katsunori <foozy@lares.dti.ne.jp> parents: 25173 diff changeset	303 ... 'foo.foo1': True, 'foo.foo2': True,
86298718b01c import-checker: make imported_modules yield absolute dotted_name_of_path FUJIWARA Katsunori <foozy@lares.dti.ne.jp> parents: 25173 diff changeset	304 ... 'foo.bar.__init__': True, 'foo.bar.bar1': True,
86298718b01c import-checker: make imported_modules yield absolute dotted_name_of_path FUJIWARA Katsunori <foozy@lares.dti.ne.jp> parents: 25173 diff changeset	305 ... 'baz.__init__': True, 'baz.baz1': True }
86298718b01c import-checker: make imported_modules yield absolute dotted_name_of_path FUJIWARA Katsunori <foozy@lares.dti.ne.jp> parents: 25173 diff changeset	306 >>> # standard library (= not locally defined ones)
86298718b01c import-checker: make imported_modules yield absolute dotted_name_of_path FUJIWARA Katsunori <foozy@lares.dti.ne.jp> parents: 25173 diff changeset	307 >>> sorted(imported_modules(
86298718b01c import-checker: make imported_modules yield absolute dotted_name_of_path FUJIWARA Katsunori <foozy@lares.dti.ne.jp> parents: 25173 diff changeset	308 ... 'from stdlib1 import foo, bar; import stdlib2',
28921 02ee31a50002 import-checker: track filenames for SyntaxErrors timeless <timeless@mozdev.org> parents: 28920 diff changeset	309 ... modulename, f, localmods))
25174 86298718b01c import-checker: make imported_modules yield absolute dotted_name_of_path FUJIWARA Katsunori <foozy@lares.dti.ne.jp> parents: 25173 diff changeset	310 []
86298718b01c import-checker: make imported_modules yield absolute dotted_name_of_path FUJIWARA Katsunori <foozy@lares.dti.ne.jp> parents: 25173 diff changeset	311 >>> # relative importing
20037 957b43371928 import-checker: ignore nested imports Augie Fackler <raf@durin42.com> parents: 20036 diff changeset	312 >>> sorted(imported_modules(
25174 86298718b01c import-checker: make imported_modules yield absolute dotted_name_of_path FUJIWARA Katsunori <foozy@lares.dti.ne.jp> parents: 25173 diff changeset	313 ... 'import foo1; from bar import bar1',
28921 02ee31a50002 import-checker: track filenames for SyntaxErrors timeless <timeless@mozdev.org> parents: 28920 diff changeset	314 ... modulename, f, localmods))
26964 5abba2c92da3 import-checker: allow import of child modules from package root Yuya Nishihara <yuya@tcha.org> parents: 26956 diff changeset	315 ['foo.bar.bar1', 'foo.foo1']
25174 86298718b01c import-checker: make imported_modules yield absolute dotted_name_of_path FUJIWARA Katsunori <foozy@lares.dti.ne.jp> parents: 25173 diff changeset	316 >>> sorted(imported_modules(
86298718b01c import-checker: make imported_modules yield absolute dotted_name_of_path FUJIWARA Katsunori <foozy@lares.dti.ne.jp> parents: 25173 diff changeset	317 ... 'from bar.bar1 import name1, name2, name3',
28921 02ee31a50002 import-checker: track filenames for SyntaxErrors timeless <timeless@mozdev.org> parents: 28920 diff changeset	318 ... modulename, f, localmods))
25174 86298718b01c import-checker: make imported_modules yield absolute dotted_name_of_path FUJIWARA Katsunori <foozy@lares.dti.ne.jp> parents: 25173 diff changeset	319 ['foo.bar.bar1']
86298718b01c import-checker: make imported_modules yield absolute dotted_name_of_path FUJIWARA Katsunori <foozy@lares.dti.ne.jp> parents: 25173 diff changeset	320 >>> # absolute importing
86298718b01c import-checker: make imported_modules yield absolute dotted_name_of_path FUJIWARA Katsunori <foozy@lares.dti.ne.jp> parents: 25173 diff changeset	321 >>> sorted(imported_modules(
86298718b01c import-checker: make imported_modules yield absolute dotted_name_of_path FUJIWARA Katsunori <foozy@lares.dti.ne.jp> parents: 25173 diff changeset	322 ... 'from baz import baz1, name1',
28921 02ee31a50002 import-checker: track filenames for SyntaxErrors timeless <timeless@mozdev.org> parents: 28920 diff changeset	323 ... modulename, f, localmods))
25174 86298718b01c import-checker: make imported_modules yield absolute dotted_name_of_path FUJIWARA Katsunori <foozy@lares.dti.ne.jp> parents: 25173 diff changeset	324 ['baz.__init__', 'baz.baz1']
86298718b01c import-checker: make imported_modules yield absolute dotted_name_of_path FUJIWARA Katsunori <foozy@lares.dti.ne.jp> parents: 25173 diff changeset	325 >>> # mixed importing, even though it shouldn't be recommended
86298718b01c import-checker: make imported_modules yield absolute dotted_name_of_path FUJIWARA Katsunori <foozy@lares.dti.ne.jp> parents: 25173 diff changeset	326 >>> sorted(imported_modules(
86298718b01c import-checker: make imported_modules yield absolute dotted_name_of_path FUJIWARA Katsunori <foozy@lares.dti.ne.jp> parents: 25173 diff changeset	327 ... 'import stdlib, foo1, baz',
28921 02ee31a50002 import-checker: track filenames for SyntaxErrors timeless <timeless@mozdev.org> parents: 28920 diff changeset	328 ... modulename, f, localmods))
25174 86298718b01c import-checker: make imported_modules yield absolute dotted_name_of_path FUJIWARA Katsunori <foozy@lares.dti.ne.jp> parents: 25173 diff changeset	329 ['baz.__init__', 'foo.foo1']
86298718b01c import-checker: make imported_modules yield absolute dotted_name_of_path FUJIWARA Katsunori <foozy@lares.dti.ne.jp> parents: 25173 diff changeset	330 >>> # ignore_nested
20037 957b43371928 import-checker: ignore nested imports Augie Fackler <raf@durin42.com> parents: 20036 diff changeset	331 >>> sorted(imported_modules(
957b43371928 import-checker: ignore nested imports Augie Fackler <raf@durin42.com> parents: 20036 diff changeset	332 ... '''import foo
957b43371928 import-checker: ignore nested imports Augie Fackler <raf@durin42.com> parents: 20036 diff changeset	333 ... def wat():
957b43371928 import-checker: ignore nested imports Augie Fackler <raf@durin42.com> parents: 20036 diff changeset	334 ... import bar
28921 02ee31a50002 import-checker: track filenames for SyntaxErrors timeless <timeless@mozdev.org> parents: 28920 diff changeset	335 ... ''', modulename, f, localmods))
25174 86298718b01c import-checker: make imported_modules yield absolute dotted_name_of_path FUJIWARA Katsunori <foozy@lares.dti.ne.jp> parents: 25173 diff changeset	336 ['foo.__init__', 'foo.bar.__init__']
86298718b01c import-checker: make imported_modules yield absolute dotted_name_of_path FUJIWARA Katsunori <foozy@lares.dti.ne.jp> parents: 25173 diff changeset	337 >>> sorted(imported_modules(
86298718b01c import-checker: make imported_modules yield absolute dotted_name_of_path FUJIWARA Katsunori <foozy@lares.dti.ne.jp> parents: 25173 diff changeset	338 ... '''import foo
86298718b01c import-checker: make imported_modules yield absolute dotted_name_of_path FUJIWARA Katsunori <foozy@lares.dti.ne.jp> parents: 25173 diff changeset	339 ... def wat():
86298718b01c import-checker: make imported_modules yield absolute dotted_name_of_path FUJIWARA Katsunori <foozy@lares.dti.ne.jp> parents: 25173 diff changeset	340 ... import bar
28921 02ee31a50002 import-checker: track filenames for SyntaxErrors timeless <timeless@mozdev.org> parents: 28920 diff changeset	341 ... ''', modulename, f, localmods, ignore_nested=True))
25174 86298718b01c import-checker: make imported_modules yield absolute dotted_name_of_path FUJIWARA Katsunori <foozy@lares.dti.ne.jp> parents: 25173 diff changeset	342 ['foo.__init__']
20036 e5d51fa51aba contrib: add an import checker Augie Fackler <raf@durin42.com> parents: diff changeset	343 """
25174 86298718b01c import-checker: make imported_modules yield absolute dotted_name_of_path FUJIWARA Katsunori <foozy@lares.dti.ne.jp> parents: 25173 diff changeset	344 fromlocal = fromlocalfunc(modulename, localmods)
28921 02ee31a50002 import-checker: track filenames for SyntaxErrors timeless <timeless@mozdev.org> parents: 28920 diff changeset	345 for node in ast.walk(ast.parse(source, f)):
20037 957b43371928 import-checker: ignore nested imports Augie Fackler <raf@durin42.com> parents: 20036 diff changeset	346 if ignore_nested and getattr(node, 'col_offset', 0) > 0:
957b43371928 import-checker: ignore nested imports Augie Fackler <raf@durin42.com> parents: 20036 diff changeset	347 continue
20036 e5d51fa51aba contrib: add an import checker Augie Fackler <raf@durin42.com> parents: diff changeset	348 if isinstance(node, ast.Import):
e5d51fa51aba contrib: add an import checker Augie Fackler <raf@durin42.com> parents: diff changeset	349 for n in node.names:
25174 86298718b01c import-checker: make imported_modules yield absolute dotted_name_of_path FUJIWARA Katsunori <foozy@lares.dti.ne.jp> parents: 25173 diff changeset	350 found = fromlocal(n.name)
86298718b01c import-checker: make imported_modules yield absolute dotted_name_of_path FUJIWARA Katsunori <foozy@lares.dti.ne.jp> parents: 25173 diff changeset	351 if not found:
86298718b01c import-checker: make imported_modules yield absolute dotted_name_of_path FUJIWARA Katsunori <foozy@lares.dti.ne.jp> parents: 25173 diff changeset	352 # this should import standard library
86298718b01c import-checker: make imported_modules yield absolute dotted_name_of_path FUJIWARA Katsunori <foozy@lares.dti.ne.jp> parents: 25173 diff changeset	353 continue
86298718b01c import-checker: make imported_modules yield absolute dotted_name_of_path FUJIWARA Katsunori <foozy@lares.dti.ne.jp> parents: 25173 diff changeset	354 yield found[1]
20036 e5d51fa51aba contrib: add an import checker Augie Fackler <raf@durin42.com> parents: diff changeset	355 elif isinstance(node, ast.ImportFrom):
25701 1f88c0f6ff5a import-checker: resolve relative imports Gregory Szorc <gregory.szorc@gmail.com> parents: 25660 diff changeset	356 found = fromlocal(node.module, node.level)
25174 86298718b01c import-checker: make imported_modules yield absolute dotted_name_of_path FUJIWARA Katsunori <foozy@lares.dti.ne.jp> parents: 25173 diff changeset	357 if not found:
86298718b01c import-checker: make imported_modules yield absolute dotted_name_of_path FUJIWARA Katsunori <foozy@lares.dti.ne.jp> parents: 25173 diff changeset	358 # this should import standard library
86298718b01c import-checker: make imported_modules yield absolute dotted_name_of_path FUJIWARA Katsunori <foozy@lares.dti.ne.jp> parents: 25173 diff changeset	359 continue
86298718b01c import-checker: make imported_modules yield absolute dotted_name_of_path FUJIWARA Katsunori <foozy@lares.dti.ne.jp> parents: 25173 diff changeset	360
86298718b01c import-checker: make imported_modules yield absolute dotted_name_of_path FUJIWARA Katsunori <foozy@lares.dti.ne.jp> parents: 25173 diff changeset	361 absname, dottedpath, hassubmod = found
86298718b01c import-checker: make imported_modules yield absolute dotted_name_of_path FUJIWARA Katsunori <foozy@lares.dti.ne.jp> parents: 25173 diff changeset	362 if not hassubmod:
26964 5abba2c92da3 import-checker: allow import of child modules from package root Yuya Nishihara <yuya@tcha.org> parents: 26956 diff changeset	363 # "dottedpath" is not a package; must be imported
5abba2c92da3 import-checker: allow import of child modules from package root Yuya Nishihara <yuya@tcha.org> parents: 26956 diff changeset	364 yield dottedpath
25174 86298718b01c import-checker: make imported_modules yield absolute dotted_name_of_path FUJIWARA Katsunori <foozy@lares.dti.ne.jp> parents: 25173 diff changeset	365 # examination of "node.names" should be redundant
86298718b01c import-checker: make imported_modules yield absolute dotted_name_of_path FUJIWARA Katsunori <foozy@lares.dti.ne.jp> parents: 25173 diff changeset	366 # e.g.: from mercurial.node import nullid, nullrev
86298718b01c import-checker: make imported_modules yield absolute dotted_name_of_path FUJIWARA Katsunori <foozy@lares.dti.ne.jp> parents: 25173 diff changeset	367 continue
86298718b01c import-checker: make imported_modules yield absolute dotted_name_of_path FUJIWARA Katsunori <foozy@lares.dti.ne.jp> parents: 25173 diff changeset	368
26964 5abba2c92da3 import-checker: allow import of child modules from package root Yuya Nishihara <yuya@tcha.org> parents: 26956 diff changeset	369 modnotfound = False
25174 86298718b01c import-checker: make imported_modules yield absolute dotted_name_of_path FUJIWARA Katsunori <foozy@lares.dti.ne.jp> parents: 25173 diff changeset	370 prefix = absname + '.'
20036 e5d51fa51aba contrib: add an import checker Augie Fackler <raf@durin42.com> parents: diff changeset	371 for n in node.names:
25174 86298718b01c import-checker: make imported_modules yield absolute dotted_name_of_path FUJIWARA Katsunori <foozy@lares.dti.ne.jp> parents: 25173 diff changeset	372 found = fromlocal(prefix + n.name)
86298718b01c import-checker: make imported_modules yield absolute dotted_name_of_path FUJIWARA Katsunori <foozy@lares.dti.ne.jp> parents: 25173 diff changeset	373 if not found:
86298718b01c import-checker: make imported_modules yield absolute dotted_name_of_path FUJIWARA Katsunori <foozy@lares.dti.ne.jp> parents: 25173 diff changeset	374 # this should be a function or a property of "node.module"
26964 5abba2c92da3 import-checker: allow import of child modules from package root Yuya Nishihara <yuya@tcha.org> parents: 26956 diff changeset	375 modnotfound = True
25174 86298718b01c import-checker: make imported_modules yield absolute dotted_name_of_path FUJIWARA Katsunori <foozy@lares.dti.ne.jp> parents: 25173 diff changeset	376 continue
86298718b01c import-checker: make imported_modules yield absolute dotted_name_of_path FUJIWARA Katsunori <foozy@lares.dti.ne.jp> parents: 25173 diff changeset	377 yield found[1]
44440 4cabeea6d214 hgext: start building a library for simple hooks Joerg Sonnenberger <joerg@bec.de> parents: 43959 diff changeset	378 if modnotfound and dottedpath != modulename:
26964 5abba2c92da3 import-checker: allow import of child modules from package root Yuya Nishihara <yuya@tcha.org> parents: 26956 diff changeset	379 # "dottedpath" is a package, but imported because of non-module
5abba2c92da3 import-checker: allow import of child modules from package root Yuya Nishihara <yuya@tcha.org> parents: 26956 diff changeset	380 # lookup
44440 4cabeea6d214 hgext: start building a library for simple hooks Joerg Sonnenberger <joerg@bec.de> parents: 43959 diff changeset	381 # specifically allow "from . import foo" from __init__.py
26964 5abba2c92da3 import-checker: allow import of child modules from package root Yuya Nishihara <yuya@tcha.org> parents: 26956 diff changeset	382 yield dottedpath
20036 e5d51fa51aba contrib: add an import checker Augie Fackler <raf@durin42.com> parents: diff changeset	383
43075 57875cf423c9 style: run a patched black on a subset of mercurial Augie Fackler <augie@google.com> parents: 42823 diff changeset	384
27272 69308357ecd1 import-checker: allow absolute imports of sub modules from local packages Yuya Nishihara <yuya@tcha.org> parents: 27018 diff changeset	385 def verify_import_convention(module, source, localmods):
48959 a52f5bfc9358 import-checker: assume absolute and use modern import checker Gregory Szorc <gregory.szorc@gmail.com> parents: 48925 diff changeset	386 """Verify imports match our established coding convention."""
a52f5bfc9358 import-checker: assume absolute and use modern import checker Gregory Szorc <gregory.szorc@gmail.com> parents: 48925 diff changeset	387 root = ast.parse(source)
25703 1a6a117d0b95 import-checker: establish modern import convention Gregory Szorc <gregory.szorc@gmail.com> parents: 25702 diff changeset	388
48959 a52f5bfc9358 import-checker: assume absolute and use modern import checker Gregory Szorc <gregory.szorc@gmail.com> parents: 48925 diff changeset	389 return verify_modern_convention(module, root, localmods)
25703 1a6a117d0b95 import-checker: establish modern import convention Gregory Szorc <gregory.szorc@gmail.com> parents: 25702 diff changeset	390
43075 57875cf423c9 style: run a patched black on a subset of mercurial Augie Fackler <augie@google.com> parents: 42823 diff changeset	391
27272 69308357ecd1 import-checker: allow absolute imports of sub modules from local packages Yuya Nishihara <yuya@tcha.org> parents: 27018 diff changeset	392 def verify_modern_convention(module, root, localmods, root_col_offset=0):
25703 1a6a117d0b95 import-checker: establish modern import convention Gregory Szorc <gregory.szorc@gmail.com> parents: 25702 diff changeset	393 """Verify a file conforms to the modern import convention rules.
1a6a117d0b95 import-checker: establish modern import convention Gregory Szorc <gregory.szorc@gmail.com> parents: 25702 diff changeset	394
1a6a117d0b95 import-checker: establish modern import convention Gregory Szorc <gregory.szorc@gmail.com> parents: 25702 diff changeset	395 The rules of the modern convention are:
1a6a117d0b95 import-checker: establish modern import convention Gregory Szorc <gregory.szorc@gmail.com> parents: 25702 diff changeset	396
1a6a117d0b95 import-checker: establish modern import convention Gregory Szorc <gregory.szorc@gmail.com> parents: 25702 diff changeset	397 * Ordering is stdlib followed by local imports. Each group is lexically
1a6a117d0b95 import-checker: establish modern import convention Gregory Szorc <gregory.szorc@gmail.com> parents: 25702 diff changeset	398 sorted.
1a6a117d0b95 import-checker: establish modern import convention Gregory Szorc <gregory.szorc@gmail.com> parents: 25702 diff changeset	399 * Importing multiple modules via "import X, Y" is not allowed: use
1a6a117d0b95 import-checker: establish modern import convention Gregory Szorc <gregory.szorc@gmail.com> parents: 25702 diff changeset	400 separate import statements.
1a6a117d0b95 import-checker: establish modern import convention Gregory Szorc <gregory.szorc@gmail.com> parents: 25702 diff changeset	401 * Importing multiple modules via "from X import ..." is allowed if using
1a6a117d0b95 import-checker: establish modern import convention Gregory Szorc <gregory.szorc@gmail.com> parents: 25702 diff changeset	402 parenthesis and one entry per line.
1a6a117d0b95 import-checker: establish modern import convention Gregory Szorc <gregory.szorc@gmail.com> parents: 25702 diff changeset	403 * Only 1 relative import statement per import level ("from .", "from ..")
1a6a117d0b95 import-checker: establish modern import convention Gregory Szorc <gregory.szorc@gmail.com> parents: 25702 diff changeset	404 is allowed.
1a6a117d0b95 import-checker: establish modern import convention Gregory Szorc <gregory.szorc@gmail.com> parents: 25702 diff changeset	405 * Relative imports from higher levels must occur before lower levels. e.g.
1a6a117d0b95 import-checker: establish modern import convention Gregory Szorc <gregory.szorc@gmail.com> parents: 25702 diff changeset	406 "from .." must be before "from .".
1a6a117d0b95 import-checker: establish modern import convention Gregory Szorc <gregory.szorc@gmail.com> parents: 25702 diff changeset	407 * Imports from peer packages should use relative import (e.g. do not
1a6a117d0b95 import-checker: establish modern import convention Gregory Szorc <gregory.szorc@gmail.com> parents: 25702 diff changeset	408 "import mercurial.foo" from a "mercurial.*" module).
1a6a117d0b95 import-checker: establish modern import convention Gregory Szorc <gregory.szorc@gmail.com> parents: 25702 diff changeset	409 * Symbols can only be imported from specific modules (see
1a6a117d0b95 import-checker: establish modern import convention Gregory Szorc <gregory.szorc@gmail.com> parents: 25702 diff changeset	410 `allowsymbolimports`). For other modules, first import the module then
1a6a117d0b95 import-checker: establish modern import convention Gregory Szorc <gregory.szorc@gmail.com> parents: 25702 diff changeset	411 assign the symbol to a module-level variable. In addition, these imports
29208 cba8bc11ed10 import-checker: extend check of symbol-import order to all local modules Yuya Nishihara <yuya@tcha.org> parents: 29207 diff changeset	412 must be performed before other local imports. This rule only
25703 1a6a117d0b95 import-checker: establish modern import convention Gregory Szorc <gregory.szorc@gmail.com> parents: 25702 diff changeset	413 applies to import statements outside of any blocks.
34055 bc2535238de2 import-checker: allow relative import a module being checked Jun Wu <quark@fb.com> parents: 33938 diff changeset	414 * Relative imports from the standard library are not allowed, unless that
bc2535238de2 import-checker: allow relative import a module being checked Jun Wu <quark@fb.com> parents: 33938 diff changeset	415 library is also a local module.
25703 1a6a117d0b95 import-checker: establish modern import convention Gregory Szorc <gregory.szorc@gmail.com> parents: 25702 diff changeset	416 * Certain modules must be aliased to alternate names to avoid aliasing
1a6a117d0b95 import-checker: establish modern import convention Gregory Szorc <gregory.szorc@gmail.com> parents: 25702 diff changeset	417 and readability problems. See `requirealias`.
1a6a117d0b95 import-checker: establish modern import convention Gregory Szorc <gregory.szorc@gmail.com> parents: 25702 diff changeset	418 """
33908 3de9a2df6411 contrib: have import-checker work mostly with native strings for mod names Augie Fackler <raf@durin42.com> parents: 33896 diff changeset	419 if not isinstance(module, str):
3de9a2df6411 contrib: have import-checker work mostly with native strings for mod names Augie Fackler <raf@durin42.com> parents: 33896 diff changeset	420 module = module.decode('ascii')
25703 1a6a117d0b95 import-checker: establish modern import convention Gregory Szorc <gregory.szorc@gmail.com> parents: 25702 diff changeset	421 topmodule = module.split('.')[0]
27272 69308357ecd1 import-checker: allow absolute imports of sub modules from local packages Yuya Nishihara <yuya@tcha.org> parents: 27018 diff changeset	422 fromlocal = fromlocalfunc(module, localmods)
25703 1a6a117d0b95 import-checker: establish modern import convention Gregory Szorc <gregory.szorc@gmail.com> parents: 25702 diff changeset	423
1a6a117d0b95 import-checker: establish modern import convention Gregory Szorc <gregory.szorc@gmail.com> parents: 25702 diff changeset	424 # Whether a local/non-stdlib import has been performed.
28330 f3fb24e36d61 import-checker: report local with stdlib late warning timeless <timeless@mozdev.org> parents: 27621 diff changeset	425 seenlocal = None
29208 cba8bc11ed10 import-checker: extend check of symbol-import order to all local modules Yuya Nishihara <yuya@tcha.org> parents: 29207 diff changeset	426 # Whether a local/non-stdlib, non-symbol import has been seen.
cba8bc11ed10 import-checker: extend check of symbol-import order to all local modules Yuya Nishihara <yuya@tcha.org> parents: 29207 diff changeset	427 seennonsymbollocal = False
25703 1a6a117d0b95 import-checker: establish modern import convention Gregory Szorc <gregory.szorc@gmail.com> parents: 25702 diff changeset	428 # The last name to be imported (for sorting).
1a6a117d0b95 import-checker: establish modern import convention Gregory Szorc <gregory.szorc@gmail.com> parents: 25702 diff changeset	429 lastname = None
30595 74eecb93c617 import-checker: do not enforce lexical sort accross stdlib/local boundary Pierre-Yves David <pierre-yves.david@ens-lyon.org> parents: 29395 diff changeset	430 laststdlib = None
25703 1a6a117d0b95 import-checker: establish modern import convention Gregory Szorc <gregory.szorc@gmail.com> parents: 25702 diff changeset	431 # Relative import levels encountered so far.
1a6a117d0b95 import-checker: establish modern import convention Gregory Szorc <gregory.szorc@gmail.com> parents: 25702 diff changeset	432 seenlevels = set()
1a6a117d0b95 import-checker: establish modern import convention Gregory Szorc <gregory.szorc@gmail.com> parents: 25702 diff changeset	433
26965 1fa66d3ad28d import-checker: reset context to verify convention in function scope Yuya Nishihara <yuya@tcha.org> parents: 26964 diff changeset	434 for node, newscope in walklocal(root):
43075 57875cf423c9 style: run a patched black on a subset of mercurial Augie Fackler <augie@google.com> parents: 42823 diff changeset	435
26955 c4114e335b49 import-checker: extract function to generate a formatted warning Yuya Nishihara <yuya@tcha.org> parents: 26954 diff changeset	436 def msg(fmt, *args):
26956 4b56214ebb7a import-checker: include lineno in warning message Yuya Nishihara <yuya@tcha.org> parents: 26955 diff changeset	437 return (fmt % args, node.lineno)
43075 57875cf423c9 style: run a patched black on a subset of mercurial Augie Fackler <augie@google.com> parents: 42823 diff changeset	438
26965 1fa66d3ad28d import-checker: reset context to verify convention in function scope Yuya Nishihara <yuya@tcha.org> parents: 26964 diff changeset	439 if newscope:
1fa66d3ad28d import-checker: reset context to verify convention in function scope Yuya Nishihara <yuya@tcha.org> parents: 26964 diff changeset	440 # Check for local imports in function
52669 e627cc25b6f3 pyupgrade: rewrite `yield` statements in a loop to `yield from` Matt Harbison <matt_harbison@yahoo.com> parents: 52472 diff changeset	441 yield from verify_modern_convention(
43075 57875cf423c9 style: run a patched black on a subset of mercurial Augie Fackler <augie@google.com> parents: 42823 diff changeset	442 module, node, localmods, node.col_offset + 4
52669 e627cc25b6f3 pyupgrade: rewrite `yield` statements in a loop to `yield from` Matt Harbison <matt_harbison@yahoo.com> parents: 52472 diff changeset	443 )
26965 1fa66d3ad28d import-checker: reset context to verify convention in function scope Yuya Nishihara <yuya@tcha.org> parents: 26964 diff changeset	444 elif isinstance(node, ast.Import):
25703 1a6a117d0b95 import-checker: establish modern import convention Gregory Szorc <gregory.szorc@gmail.com> parents: 25702 diff changeset	445 # Disallow "import foo, bar" and require separate imports
1a6a117d0b95 import-checker: establish modern import convention Gregory Szorc <gregory.szorc@gmail.com> parents: 25702 diff changeset	446 # for each module.
1a6a117d0b95 import-checker: establish modern import convention Gregory Szorc <gregory.szorc@gmail.com> parents: 25702 diff changeset	447 if len(node.names) > 1:
43075 57875cf423c9 style: run a patched black on a subset of mercurial Augie Fackler <augie@google.com> parents: 42823 diff changeset	448 yield msg(
57875cf423c9 style: run a patched black on a subset of mercurial Augie Fackler <augie@google.com> parents: 42823 diff changeset	449 'multiple imported names: %s',
57875cf423c9 style: run a patched black on a subset of mercurial Augie Fackler <augie@google.com> parents: 42823 diff changeset	450 ', '.join(n.name for n in node.names),
57875cf423c9 style: run a patched black on a subset of mercurial Augie Fackler <augie@google.com> parents: 42823 diff changeset	451 )
25703 1a6a117d0b95 import-checker: establish modern import convention Gregory Szorc <gregory.szorc@gmail.com> parents: 25702 diff changeset	452
1a6a117d0b95 import-checker: establish modern import convention Gregory Szorc <gregory.szorc@gmail.com> parents: 25702 diff changeset	453 name = node.names[0].name
1a6a117d0b95 import-checker: establish modern import convention Gregory Szorc <gregory.szorc@gmail.com> parents: 25702 diff changeset	454 asname = node.names[0].asname
1a6a117d0b95 import-checker: establish modern import convention Gregory Szorc <gregory.szorc@gmail.com> parents: 25702 diff changeset	455
30595 74eecb93c617 import-checker: do not enforce lexical sort accross stdlib/local boundary Pierre-Yves David <pierre-yves.david@ens-lyon.org> parents: 29395 diff changeset	456 stdlib = name in stdlib_modules
74eecb93c617 import-checker: do not enforce lexical sort accross stdlib/local boundary Pierre-Yves David <pierre-yves.david@ens-lyon.org> parents: 29395 diff changeset	457
25703 1a6a117d0b95 import-checker: establish modern import convention Gregory Szorc <gregory.szorc@gmail.com> parents: 25702 diff changeset	458 # Ignore sorting rules on imports inside blocks.
26965 1fa66d3ad28d import-checker: reset context to verify convention in function scope Yuya Nishihara <yuya@tcha.org> parents: 26964 diff changeset	459 if node.col_offset == root_col_offset:
30595 74eecb93c617 import-checker: do not enforce lexical sort accross stdlib/local boundary Pierre-Yves David <pierre-yves.david@ens-lyon.org> parents: 29395 diff changeset	460 if lastname and name < lastname and laststdlib == stdlib:
43075 57875cf423c9 style: run a patched black on a subset of mercurial Augie Fackler <augie@google.com> parents: 42823 diff changeset	461 yield msg(
57875cf423c9 style: run a patched black on a subset of mercurial Augie Fackler <augie@google.com> parents: 42823 diff changeset	462 'imports not lexically sorted: %s < %s', name, lastname
57875cf423c9 style: run a patched black on a subset of mercurial Augie Fackler <augie@google.com> parents: 42823 diff changeset	463 )
25703 1a6a117d0b95 import-checker: establish modern import convention Gregory Szorc <gregory.szorc@gmail.com> parents: 25702 diff changeset	464
30595 74eecb93c617 import-checker: do not enforce lexical sort accross stdlib/local boundary Pierre-Yves David <pierre-yves.david@ens-lyon.org> parents: 29395 diff changeset	465 lastname = name
74eecb93c617 import-checker: do not enforce lexical sort accross stdlib/local boundary Pierre-Yves David <pierre-yves.david@ens-lyon.org> parents: 29395 diff changeset	466 laststdlib = stdlib
25703 1a6a117d0b95 import-checker: establish modern import convention Gregory Szorc <gregory.szorc@gmail.com> parents: 25702 diff changeset	467
1a6a117d0b95 import-checker: establish modern import convention Gregory Szorc <gregory.szorc@gmail.com> parents: 25702 diff changeset	468 # stdlib imports should be before local imports.
26965 1fa66d3ad28d import-checker: reset context to verify convention in function scope Yuya Nishihara <yuya@tcha.org> parents: 26964 diff changeset	469 if stdlib and seenlocal and node.col_offset == root_col_offset:
43075 57875cf423c9 style: run a patched black on a subset of mercurial Augie Fackler <augie@google.com> parents: 42823 diff changeset	470 yield msg(
57875cf423c9 style: run a patched black on a subset of mercurial Augie Fackler <augie@google.com> parents: 42823 diff changeset	471 'stdlib import "%s" follows local import: %s',
57875cf423c9 style: run a patched black on a subset of mercurial Augie Fackler <augie@google.com> parents: 42823 diff changeset	472 name,
57875cf423c9 style: run a patched black on a subset of mercurial Augie Fackler <augie@google.com> parents: 42823 diff changeset	473 seenlocal,
57875cf423c9 style: run a patched black on a subset of mercurial Augie Fackler <augie@google.com> parents: 42823 diff changeset	474 )
25703 1a6a117d0b95 import-checker: establish modern import convention Gregory Szorc <gregory.szorc@gmail.com> parents: 25702 diff changeset	475
1a6a117d0b95 import-checker: establish modern import convention Gregory Szorc <gregory.szorc@gmail.com> parents: 25702 diff changeset	476 if not stdlib:
28330 f3fb24e36d61 import-checker: report local with stdlib late warning timeless <timeless@mozdev.org> parents: 27621 diff changeset	477 seenlocal = name
25703 1a6a117d0b95 import-checker: establish modern import convention Gregory Szorc <gregory.szorc@gmail.com> parents: 25702 diff changeset	478
1a6a117d0b95 import-checker: establish modern import convention Gregory Szorc <gregory.szorc@gmail.com> parents: 25702 diff changeset	479 # Import of sibling modules should use relative imports.
1a6a117d0b95 import-checker: establish modern import convention Gregory Szorc <gregory.szorc@gmail.com> parents: 25702 diff changeset	480 topname = name.split('.')[0]
1a6a117d0b95 import-checker: establish modern import convention Gregory Szorc <gregory.szorc@gmail.com> parents: 25702 diff changeset	481 if topname == topmodule:
26955 c4114e335b49 import-checker: extract function to generate a formatted warning Yuya Nishihara <yuya@tcha.org> parents: 26954 diff changeset	482 yield msg('import should be relative: %s', name)
25703 1a6a117d0b95 import-checker: establish modern import convention Gregory Szorc <gregory.szorc@gmail.com> parents: 25702 diff changeset	483
1a6a117d0b95 import-checker: establish modern import convention Gregory Szorc <gregory.szorc@gmail.com> parents: 25702 diff changeset	484 if name in requirealias and asname != requirealias[name]:
43075 57875cf423c9 style: run a patched black on a subset of mercurial Augie Fackler <augie@google.com> parents: 42823 diff changeset	485 yield msg(
57875cf423c9 style: run a patched black on a subset of mercurial Augie Fackler <augie@google.com> parents: 42823 diff changeset	486 '%s module must be "as" aliased to %s',
57875cf423c9 style: run a patched black on a subset of mercurial Augie Fackler <augie@google.com> parents: 42823 diff changeset	487 name,
57875cf423c9 style: run a patched black on a subset of mercurial Augie Fackler <augie@google.com> parents: 42823 diff changeset	488 requirealias[name],
57875cf423c9 style: run a patched black on a subset of mercurial Augie Fackler <augie@google.com> parents: 42823 diff changeset	489 )
25703 1a6a117d0b95 import-checker: establish modern import convention Gregory Szorc <gregory.szorc@gmail.com> parents: 25702 diff changeset	490
1a6a117d0b95 import-checker: establish modern import convention Gregory Szorc <gregory.szorc@gmail.com> parents: 25702 diff changeset	491 elif isinstance(node, ast.ImportFrom):
1a6a117d0b95 import-checker: establish modern import convention Gregory Szorc <gregory.szorc@gmail.com> parents: 25702 diff changeset	492 # Resolve the full imported module name.
1a6a117d0b95 import-checker: establish modern import convention Gregory Szorc <gregory.szorc@gmail.com> parents: 25702 diff changeset	493 if node.level > 0:
43075 57875cf423c9 style: run a patched black on a subset of mercurial Augie Fackler <augie@google.com> parents: 42823 diff changeset	494 fullname = '.'.join(module.split('.')[: -node.level])
25703 1a6a117d0b95 import-checker: establish modern import convention Gregory Szorc <gregory.szorc@gmail.com> parents: 25702 diff changeset	495 if node.module:
1a6a117d0b95 import-checker: establish modern import convention Gregory Szorc <gregory.szorc@gmail.com> parents: 25702 diff changeset	496 fullname += '.%s' % node.module
1a6a117d0b95 import-checker: establish modern import convention Gregory Szorc <gregory.szorc@gmail.com> parents: 25702 diff changeset	497 else:
1a6a117d0b95 import-checker: establish modern import convention Gregory Szorc <gregory.szorc@gmail.com> parents: 25702 diff changeset	498 assert node.module
1a6a117d0b95 import-checker: establish modern import convention Gregory Szorc <gregory.szorc@gmail.com> parents: 25702 diff changeset	499 fullname = node.module
1a6a117d0b95 import-checker: establish modern import convention Gregory Szorc <gregory.szorc@gmail.com> parents: 25702 diff changeset	500
1a6a117d0b95 import-checker: establish modern import convention Gregory Szorc <gregory.szorc@gmail.com> parents: 25702 diff changeset	501 topname = fullname.split('.')[0]
1a6a117d0b95 import-checker: establish modern import convention Gregory Szorc <gregory.szorc@gmail.com> parents: 25702 diff changeset	502 if topname == topmodule:
26955 c4114e335b49 import-checker: extract function to generate a formatted warning Yuya Nishihara <yuya@tcha.org> parents: 26954 diff changeset	503 yield msg('import should be relative: %s', fullname)
25703 1a6a117d0b95 import-checker: establish modern import convention Gregory Szorc <gregory.szorc@gmail.com> parents: 25702 diff changeset	504
1a6a117d0b95 import-checker: establish modern import convention Gregory Szorc <gregory.szorc@gmail.com> parents: 25702 diff changeset	505 # __future__ is special since it needs to come first and use
1a6a117d0b95 import-checker: establish modern import convention Gregory Szorc <gregory.szorc@gmail.com> parents: 25702 diff changeset	506 # symbol import.
1a6a117d0b95 import-checker: establish modern import convention Gregory Szorc <gregory.szorc@gmail.com> parents: 25702 diff changeset	507 if fullname != '__future__':
34055 bc2535238de2 import-checker: allow relative import a module being checked Jun Wu <quark@fb.com> parents: 33938 diff changeset	508 if not fullname or (
bc2535238de2 import-checker: allow relative import a module being checked Jun Wu <quark@fb.com> parents: 33938 diff changeset	509 fullname in stdlib_modules
43641 0ad5d6c4bfad import-checker: allow 'from typing import ...' Yuya Nishihara <yuya@tcha.org> parents: 43415 diff changeset	510 # allow standard 'from typing import ...' style
43959 303576116ac1 import-checker: allow all absolute imports of stdlib modules Martin von Zweigbergk <martinvonz@google.com> parents: 43641 diff changeset	511 and fullname.startswith('.')
34055 bc2535238de2 import-checker: allow relative import a module being checked Jun Wu <quark@fb.com> parents: 33938 diff changeset	512 and fullname not in localmods
43075 57875cf423c9 style: run a patched black on a subset of mercurial Augie Fackler <augie@google.com> parents: 42823 diff changeset	513 and fullname + '.__init__' not in localmods
57875cf423c9 style: run a patched black on a subset of mercurial Augie Fackler <augie@google.com> parents: 42823 diff changeset	514 ):
26955 c4114e335b49 import-checker: extract function to generate a formatted warning Yuya Nishihara <yuya@tcha.org> parents: 26954 diff changeset	515 yield msg('relative import of stdlib module')
25703 1a6a117d0b95 import-checker: establish modern import convention Gregory Szorc <gregory.szorc@gmail.com> parents: 25702 diff changeset	516 else:
28330 f3fb24e36d61 import-checker: report local with stdlib late warning timeless <timeless@mozdev.org> parents: 27621 diff changeset	517 seenlocal = fullname
25703 1a6a117d0b95 import-checker: establish modern import convention Gregory Szorc <gregory.szorc@gmail.com> parents: 25702 diff changeset	518
1a6a117d0b95 import-checker: establish modern import convention Gregory Szorc <gregory.szorc@gmail.com> parents: 25702 diff changeset	519 # Direct symbol import is only allowed from certain modules and
1a6a117d0b95 import-checker: establish modern import convention Gregory Szorc <gregory.szorc@gmail.com> parents: 25702 diff changeset	520 # must occur before non-symbol imports.
29207 a09098c61fea import-checker: always build a list of imported symbols Yuya Nishihara <yuya@tcha.org> parents: 29122 diff changeset	521 found = fromlocal(node.module, node.level)
a09098c61fea import-checker: always build a list of imported symbols Yuya Nishihara <yuya@tcha.org> parents: 29122 diff changeset	522 if found and found[2]: # node.module is a package
a09098c61fea import-checker: always build a list of imported symbols Yuya Nishihara <yuya@tcha.org> parents: 29122 diff changeset	523 prefix = found[0] + '.'
43075 57875cf423c9 style: run a patched black on a subset of mercurial Augie Fackler <augie@google.com> parents: 42823 diff changeset	524 symbols = (
57875cf423c9 style: run a patched black on a subset of mercurial Augie Fackler <augie@google.com> parents: 42823 diff changeset	525 n.name for n in node.names if not fromlocal(prefix + n.name)
57875cf423c9 style: run a patched black on a subset of mercurial Augie Fackler <augie@google.com> parents: 42823 diff changeset	526 )
29207 a09098c61fea import-checker: always build a list of imported symbols Yuya Nishihara <yuya@tcha.org> parents: 29122 diff changeset	527 else:
32457 d02888308235 import-checker: add a way to directly import certain symbols Siddharth Agarwal <sid0@fb.com> parents: 32413 diff changeset	528 symbols = (n.name for n in node.names)
d02888308235 import-checker: add a way to directly import certain symbols Siddharth Agarwal <sid0@fb.com> parents: 32413 diff changeset	529 symbols = [sym for sym in symbols if sym not in directsymbols]
26965 1fa66d3ad28d import-checker: reset context to verify convention in function scope Yuya Nishihara <yuya@tcha.org> parents: 26964 diff changeset	530 if node.module and node.col_offset == root_col_offset:
27272 69308357ecd1 import-checker: allow absolute imports of sub modules from local packages Yuya Nishihara <yuya@tcha.org> parents: 27018 diff changeset	531 if symbols and fullname not in allowsymbolimports:
43075 57875cf423c9 style: run a patched black on a subset of mercurial Augie Fackler <augie@google.com> parents: 42823 diff changeset	532 yield msg(
57875cf423c9 style: run a patched black on a subset of mercurial Augie Fackler <augie@google.com> parents: 42823 diff changeset	533 'direct symbol import %s from %s',
57875cf423c9 style: run a patched black on a subset of mercurial Augie Fackler <augie@google.com> parents: 42823 diff changeset	534 ', '.join(symbols),
57875cf423c9 style: run a patched black on a subset of mercurial Augie Fackler <augie@google.com> parents: 42823 diff changeset	535 fullname,
57875cf423c9 style: run a patched black on a subset of mercurial Augie Fackler <augie@google.com> parents: 42823 diff changeset	536 )
25703 1a6a117d0b95 import-checker: establish modern import convention Gregory Szorc <gregory.szorc@gmail.com> parents: 25702 diff changeset	537
29208 cba8bc11ed10 import-checker: extend check of symbol-import order to all local modules Yuya Nishihara <yuya@tcha.org> parents: 29207 diff changeset	538 if symbols and seennonsymbollocal:
43075 57875cf423c9 style: run a patched black on a subset of mercurial Augie Fackler <augie@google.com> parents: 42823 diff changeset	539 yield msg(
57875cf423c9 style: run a patched black on a subset of mercurial Augie Fackler <augie@google.com> parents: 42823 diff changeset	540 'symbol import follows non-symbol import: %s', fullname
57875cf423c9 style: run a patched black on a subset of mercurial Augie Fackler <augie@google.com> parents: 42823 diff changeset	541 )
29208 cba8bc11ed10 import-checker: extend check of symbol-import order to all local modules Yuya Nishihara <yuya@tcha.org> parents: 29207 diff changeset	542 if not symbols and fullname not in stdlib_modules:
cba8bc11ed10 import-checker: extend check of symbol-import order to all local modules Yuya Nishihara <yuya@tcha.org> parents: 29207 diff changeset	543 seennonsymbollocal = True
25703 1a6a117d0b95 import-checker: establish modern import convention Gregory Szorc <gregory.szorc@gmail.com> parents: 25702 diff changeset	544
1a6a117d0b95 import-checker: establish modern import convention Gregory Szorc <gregory.szorc@gmail.com> parents: 25702 diff changeset	545 if not node.module:
1a6a117d0b95 import-checker: establish modern import convention Gregory Szorc <gregory.szorc@gmail.com> parents: 25702 diff changeset	546 assert node.level
1a6a117d0b95 import-checker: establish modern import convention Gregory Szorc <gregory.szorc@gmail.com> parents: 25702 diff changeset	547
1a6a117d0b95 import-checker: establish modern import convention Gregory Szorc <gregory.szorc@gmail.com> parents: 25702 diff changeset	548 # Only allow 1 group per level.
43075 57875cf423c9 style: run a patched black on a subset of mercurial Augie Fackler <augie@google.com> parents: 42823 diff changeset	549 if (
57875cf423c9 style: run a patched black on a subset of mercurial Augie Fackler <augie@google.com> parents: 42823 diff changeset	550 node.level in seenlevels
57875cf423c9 style: run a patched black on a subset of mercurial Augie Fackler <augie@google.com> parents: 42823 diff changeset	551 and node.col_offset == root_col_offset
57875cf423c9 style: run a patched black on a subset of mercurial Augie Fackler <augie@google.com> parents: 42823 diff changeset	552 ):
57875cf423c9 style: run a patched black on a subset of mercurial Augie Fackler <augie@google.com> parents: 42823 diff changeset	553 yield msg(
57875cf423c9 style: run a patched black on a subset of mercurial Augie Fackler <augie@google.com> parents: 42823 diff changeset	554 'multiple "from %s import" statements', '.' * node.level
57875cf423c9 style: run a patched black on a subset of mercurial Augie Fackler <augie@google.com> parents: 42823 diff changeset	555 )
25703 1a6a117d0b95 import-checker: establish modern import convention Gregory Szorc <gregory.szorc@gmail.com> parents: 25702 diff changeset	556
1a6a117d0b95 import-checker: establish modern import convention Gregory Szorc <gregory.szorc@gmail.com> parents: 25702 diff changeset	557 # Higher-level groups come before lower-level groups.
1a6a117d0b95 import-checker: establish modern import convention Gregory Szorc <gregory.szorc@gmail.com> parents: 25702 diff changeset	558 if any(node.level > l for l in seenlevels):
43075 57875cf423c9 style: run a patched black on a subset of mercurial Augie Fackler <augie@google.com> parents: 42823 diff changeset	559 yield msg(
57875cf423c9 style: run a patched black on a subset of mercurial Augie Fackler <augie@google.com> parents: 42823 diff changeset	560 'higher-level import should come first: %s', fullname
57875cf423c9 style: run a patched black on a subset of mercurial Augie Fackler <augie@google.com> parents: 42823 diff changeset	561 )
25703 1a6a117d0b95 import-checker: establish modern import convention Gregory Szorc <gregory.szorc@gmail.com> parents: 25702 diff changeset	562
1a6a117d0b95 import-checker: establish modern import convention Gregory Szorc <gregory.szorc@gmail.com> parents: 25702 diff changeset	563 seenlevels.add(node.level)
1a6a117d0b95 import-checker: establish modern import convention Gregory Szorc <gregory.szorc@gmail.com> parents: 25702 diff changeset	564
1a6a117d0b95 import-checker: establish modern import convention Gregory Szorc <gregory.szorc@gmail.com> parents: 25702 diff changeset	565 # Entries in "from .X import ( ... )" lists must be lexically
1a6a117d0b95 import-checker: establish modern import convention Gregory Szorc <gregory.szorc@gmail.com> parents: 25702 diff changeset	566 # sorted.
1a6a117d0b95 import-checker: establish modern import convention Gregory Szorc <gregory.szorc@gmail.com> parents: 25702 diff changeset	567 lastentryname = None
1a6a117d0b95 import-checker: establish modern import convention Gregory Szorc <gregory.szorc@gmail.com> parents: 25702 diff changeset	568
1a6a117d0b95 import-checker: establish modern import convention Gregory Szorc <gregory.szorc@gmail.com> parents: 25702 diff changeset	569 for n in node.names:
1a6a117d0b95 import-checker: establish modern import convention Gregory Szorc <gregory.szorc@gmail.com> parents: 25702 diff changeset	570 if lastentryname and n.name < lastentryname:
43075 57875cf423c9 style: run a patched black on a subset of mercurial Augie Fackler <augie@google.com> parents: 42823 diff changeset	571 yield msg(
57875cf423c9 style: run a patched black on a subset of mercurial Augie Fackler <augie@google.com> parents: 42823 diff changeset	572 'imports from %s not lexically sorted: %s < %s',
57875cf423c9 style: run a patched black on a subset of mercurial Augie Fackler <augie@google.com> parents: 42823 diff changeset	573 fullname,
57875cf423c9 style: run a patched black on a subset of mercurial Augie Fackler <augie@google.com> parents: 42823 diff changeset	574 n.name,
57875cf423c9 style: run a patched black on a subset of mercurial Augie Fackler <augie@google.com> parents: 42823 diff changeset	575 lastentryname,
57875cf423c9 style: run a patched black on a subset of mercurial Augie Fackler <augie@google.com> parents: 42823 diff changeset	576 )
25703 1a6a117d0b95 import-checker: establish modern import convention Gregory Szorc <gregory.szorc@gmail.com> parents: 25702 diff changeset	577
1a6a117d0b95 import-checker: establish modern import convention Gregory Szorc <gregory.szorc@gmail.com> parents: 25702 diff changeset	578 lastentryname = n.name
1a6a117d0b95 import-checker: establish modern import convention Gregory Szorc <gregory.szorc@gmail.com> parents: 25702 diff changeset	579
1a6a117d0b95 import-checker: establish modern import convention Gregory Szorc <gregory.szorc@gmail.com> parents: 25702 diff changeset	580 if n.name in requirealias and n.asname != requirealias[n.name]:
43075 57875cf423c9 style: run a patched black on a subset of mercurial Augie Fackler <augie@google.com> parents: 42823 diff changeset	581 yield msg(
57875cf423c9 style: run a patched black on a subset of mercurial Augie Fackler <augie@google.com> parents: 42823 diff changeset	582 '%s from %s must be "as" aliased to %s',
57875cf423c9 style: run a patched black on a subset of mercurial Augie Fackler <augie@google.com> parents: 42823 diff changeset	583 n.name,
57875cf423c9 style: run a patched black on a subset of mercurial Augie Fackler <augie@google.com> parents: 42823 diff changeset	584 fullname,
57875cf423c9 style: run a patched black on a subset of mercurial Augie Fackler <augie@google.com> parents: 42823 diff changeset	585 requirealias[n.name],
57875cf423c9 style: run a patched black on a subset of mercurial Augie Fackler <augie@google.com> parents: 42823 diff changeset	586 )
57875cf423c9 style: run a patched black on a subset of mercurial Augie Fackler <augie@google.com> parents: 42823 diff changeset	587
25702 ab2c5163900e import-checker: establish new function for verifying import conventions Gregory Szorc <gregory.szorc@gmail.com> parents: 25701 diff changeset	588
20036 e5d51fa51aba contrib: add an import checker Augie Fackler <raf@durin42.com> parents: diff changeset	589 class CircularImport(Exception):
e5d51fa51aba contrib: add an import checker Augie Fackler <raf@durin42.com> parents: diff changeset	590 pass
e5d51fa51aba contrib: add an import checker Augie Fackler <raf@durin42.com> parents: diff changeset	591
43075 57875cf423c9 style: run a patched black on a subset of mercurial Augie Fackler <augie@google.com> parents: 42823 diff changeset	592
24490 fb4639d5268e import-checker: make search algorithm non-recursive breadth-first Matt Mackall <mpm@selenic.com> parents: 24489 diff changeset	593 def checkmod(mod, imports):
fb4639d5268e import-checker: make search algorithm non-recursive breadth-first Matt Mackall <mpm@selenic.com> parents: 24489 diff changeset	594 shortest = {}
fb4639d5268e import-checker: make search algorithm non-recursive breadth-first Matt Mackall <mpm@selenic.com> parents: 24489 diff changeset	595 visit = [[mod]]
fb4639d5268e import-checker: make search algorithm non-recursive breadth-first Matt Mackall <mpm@selenic.com> parents: 24489 diff changeset	596 while visit:
fb4639d5268e import-checker: make search algorithm non-recursive breadth-first Matt Mackall <mpm@selenic.com> parents: 24489 diff changeset	597 path = visit.pop(0)
fb4639d5268e import-checker: make search algorithm non-recursive breadth-first Matt Mackall <mpm@selenic.com> parents: 24489 diff changeset	598 for i in sorted(imports.get(path[-1], [])):
fb4639d5268e import-checker: make search algorithm non-recursive breadth-first Matt Mackall <mpm@selenic.com> parents: 24489 diff changeset	599 if len(path) < shortest.get(i, 1000):
fb4639d5268e import-checker: make search algorithm non-recursive breadth-first Matt Mackall <mpm@selenic.com> parents: 24489 diff changeset	600 shortest[i] = len(path)
fb4639d5268e import-checker: make search algorithm non-recursive breadth-first Matt Mackall <mpm@selenic.com> parents: 24489 diff changeset	601 if i in path:
fb4639d5268e import-checker: make search algorithm non-recursive breadth-first Matt Mackall <mpm@selenic.com> parents: 24489 diff changeset	602 if i == path[0]:
fb4639d5268e import-checker: make search algorithm non-recursive breadth-first Matt Mackall <mpm@selenic.com> parents: 24489 diff changeset	603 raise CircularImport(path)
fb4639d5268e import-checker: make search algorithm non-recursive breadth-first Matt Mackall <mpm@selenic.com> parents: 24489 diff changeset	604 continue
fb4639d5268e import-checker: make search algorithm non-recursive breadth-first Matt Mackall <mpm@selenic.com> parents: 24489 diff changeset	605 visit.append(path + [i])
20036 e5d51fa51aba contrib: add an import checker Augie Fackler <raf@durin42.com> parents: diff changeset	606
43075 57875cf423c9 style: run a patched black on a subset of mercurial Augie Fackler <augie@google.com> parents: 42823 diff changeset	607
20038 c65a6937b828 import-checker: try a little harder to show fewer cycles Augie Fackler <raf@durin42.com> parents: 20037 diff changeset	608 def rotatecycle(cycle):
c65a6937b828 import-checker: try a little harder to show fewer cycles Augie Fackler <raf@durin42.com> parents: 20037 diff changeset	609 """arrange a cycle so that the lexicographically first module listed first
c65a6937b828 import-checker: try a little harder to show fewer cycles Augie Fackler <raf@durin42.com> parents: 20037 diff changeset	610
24488 4b3fc46097f7 import-checker: drop duplicate element from cycle Matt Mackall <mpm@selenic.com> parents: 24487 diff changeset	611 >>> rotatecycle(['foo', 'bar'])
20038 c65a6937b828 import-checker: try a little harder to show fewer cycles Augie Fackler <raf@durin42.com> parents: 20037 diff changeset	612 ['bar', 'foo', 'bar']
c65a6937b828 import-checker: try a little harder to show fewer cycles Augie Fackler <raf@durin42.com> parents: 20037 diff changeset	613 """
c65a6937b828 import-checker: try a little harder to show fewer cycles Augie Fackler <raf@durin42.com> parents: 20037 diff changeset	614 lowest = min(cycle)
c65a6937b828 import-checker: try a little harder to show fewer cycles Augie Fackler <raf@durin42.com> parents: 20037 diff changeset	615 idx = cycle.index(lowest)
24488 4b3fc46097f7 import-checker: drop duplicate element from cycle Matt Mackall <mpm@selenic.com> parents: 24487 diff changeset	616 return cycle[idx:] + cycle[:idx] + [lowest]
20036 e5d51fa51aba contrib: add an import checker Augie Fackler <raf@durin42.com> parents: diff changeset	617
43075 57875cf423c9 style: run a patched black on a subset of mercurial Augie Fackler <augie@google.com> parents: 42823 diff changeset	618
20036 e5d51fa51aba contrib: add an import checker Augie Fackler <raf@durin42.com> parents: diff changeset	619 def find_cycles(imports):
e5d51fa51aba contrib: add an import checker Augie Fackler <raf@durin42.com> parents: diff changeset	620 """Find cycles in an already-loaded import graph.
e5d51fa51aba contrib: add an import checker Augie Fackler <raf@durin42.com> parents: diff changeset	621
25175 10e6c4b7121b import-checker: don't treat modules as relative one if not found FUJIWARA Katsunori <foozy@lares.dti.ne.jp> parents: 25174 diff changeset	622 All module names recorded in `imports` should be absolute one.
10e6c4b7121b import-checker: don't treat modules as relative one if not found FUJIWARA Katsunori <foozy@lares.dti.ne.jp> parents: 25174 diff changeset	623
10e6c4b7121b import-checker: don't treat modules as relative one if not found FUJIWARA Katsunori <foozy@lares.dti.ne.jp> parents: 25174 diff changeset	624 >>> imports = {'top.foo': ['top.bar', 'os.path', 'top.qux'],
10e6c4b7121b import-checker: don't treat modules as relative one if not found FUJIWARA Katsunori <foozy@lares.dti.ne.jp> parents: 25174 diff changeset	625 ... 'top.bar': ['top.baz', 'sys'],
10e6c4b7121b import-checker: don't treat modules as relative one if not found FUJIWARA Katsunori <foozy@lares.dti.ne.jp> parents: 25174 diff changeset	626 ... 'top.baz': ['top.foo'],
10e6c4b7121b import-checker: don't treat modules as relative one if not found FUJIWARA Katsunori <foozy@lares.dti.ne.jp> parents: 25174 diff changeset	627 ... 'top.qux': ['top.foo']}
28703 a274c4f9087a py3: use print_function in import-checker timeless <timeless@mozdev.org> parents: 28702 diff changeset	628 >>> print('\\n'.join(sorted(find_cycles(imports))))
24487 642d245ff537 import-checker: fix rotatecycle Matt Mackall <mpm@selenic.com> parents: 22975 diff changeset	629 top.bar -> top.baz -> top.foo -> top.bar
642d245ff537 import-checker: fix rotatecycle Matt Mackall <mpm@selenic.com> parents: 22975 diff changeset	630 top.foo -> top.qux -> top.foo
20036 e5d51fa51aba contrib: add an import checker Augie Fackler <raf@durin42.com> parents: diff changeset	631 """
24491 784b278b349c import-checker: rotatecycle is actually the canonical cycle key Matt Mackall <mpm@selenic.com> parents: 24490 diff changeset	632 cycles = set()
28704 1fa6fdb72275 py3: handle iter/iterkeys+iteritems python3 divergence in import-checker timeless <timeless@mozdev.org> parents: 28703 diff changeset	633 for mod in sorted(imports.keys()):
20036 e5d51fa51aba contrib: add an import checker Augie Fackler <raf@durin42.com> parents: diff changeset	634 try:
24490 fb4639d5268e import-checker: make search algorithm non-recursive breadth-first Matt Mackall <mpm@selenic.com> parents: 24489 diff changeset	635 checkmod(mod, imports)
25660 328739ea70c3 global: mass rewrite to use modern exception syntax Gregory Szorc <gregory.szorc@gmail.com> parents: 25175 diff changeset	636 except CircularImport as e:
20036 e5d51fa51aba contrib: add an import checker Augie Fackler <raf@durin42.com> parents: diff changeset	637 cycle = e.args[0]
24491 784b278b349c import-checker: rotatecycle is actually the canonical cycle key Matt Mackall <mpm@selenic.com> parents: 24490 diff changeset	638 cycles.add(" -> ".join(rotatecycle(cycle)))
784b278b349c import-checker: rotatecycle is actually the canonical cycle key Matt Mackall <mpm@selenic.com> parents: 24490 diff changeset	639 return cycles
20036 e5d51fa51aba contrib: add an import checker Augie Fackler <raf@durin42.com> parents: diff changeset	640
43075 57875cf423c9 style: run a patched black on a subset of mercurial Augie Fackler <augie@google.com> parents: 42823 diff changeset	641
20036 e5d51fa51aba contrib: add an import checker Augie Fackler <raf@durin42.com> parents: diff changeset	642 def _cycle_sortkey(c):
e5d51fa51aba contrib: add an import checker Augie Fackler <raf@durin42.com> parents: diff changeset	643 return len(c), c
e5d51fa51aba contrib: add an import checker Augie Fackler <raf@durin42.com> parents: diff changeset	644
43075 57875cf423c9 style: run a patched black on a subset of mercurial Augie Fackler <augie@google.com> parents: 42823 diff changeset	645
28922 4ec62a084e5c import-checker: parse python code from .t files timeless <timeless@mozdev.org> parents: 28921 diff changeset	646 def embedded(f, modname, src):
4ec62a084e5c import-checker: parse python code from .t files timeless <timeless@mozdev.org> parents: 28921 diff changeset	647 """Extract embedded python code
4ec62a084e5c import-checker: parse python code from .t files timeless <timeless@mozdev.org> parents: 28921 diff changeset	648
33896 bcf53149ebce contrib: make import checker always think in terms of bytes Augie Fackler <raf@durin42.com> parents: 33895 diff changeset	649 >>> def _forcestr(thing):
bcf53149ebce contrib: make import checker always think in terms of bytes Augie Fackler <raf@durin42.com> parents: 33895 diff changeset	650 ... if not isinstance(thing, str):
bcf53149ebce contrib: make import checker always think in terms of bytes Augie Fackler <raf@durin42.com> parents: 33895 diff changeset	651 ... return thing.decode('ascii')
bcf53149ebce contrib: make import checker always think in terms of bytes Augie Fackler <raf@durin42.com> parents: 33895 diff changeset	652 ... return thing
28922 4ec62a084e5c import-checker: parse python code from .t files timeless <timeless@mozdev.org> parents: 28921 diff changeset	653 >>> def test(fn, lines):
33896 bcf53149ebce contrib: make import checker always think in terms of bytes Augie Fackler <raf@durin42.com> parents: 33895 diff changeset	654 ... for s, m, f, l in embedded(fn, b"example", lines):
bcf53149ebce contrib: make import checker always think in terms of bytes Augie Fackler <raf@durin42.com> parents: 33895 diff changeset	655 ... print("%s %s %d" % (_forcestr(m), _forcestr(f), l))
bcf53149ebce contrib: make import checker always think in terms of bytes Augie Fackler <raf@durin42.com> parents: 33895 diff changeset	656 ... print(repr(_forcestr(s)))
28922 4ec62a084e5c import-checker: parse python code from .t files timeless <timeless@mozdev.org> parents: 28921 diff changeset	657 >>> lines = [
42407 04eb3c5607af contrib: fix import-checker to operate on str instead of bytes Augie Fackler <augie@google.com> parents: 41161 diff changeset	658 ... 'comment',
04eb3c5607af contrib: fix import-checker to operate on str instead of bytes Augie Fackler <augie@google.com> parents: 41161 diff changeset	659 ... ' >>> from __future__ import print_function',
04eb3c5607af contrib: fix import-checker to operate on str instead of bytes Augie Fackler <augie@google.com> parents: 41161 diff changeset	660 ... " >>> ' multiline",
04eb3c5607af contrib: fix import-checker to operate on str instead of bytes Augie Fackler <augie@google.com> parents: 41161 diff changeset	661 ... " ... string'",
04eb3c5607af contrib: fix import-checker to operate on str instead of bytes Augie Fackler <augie@google.com> parents: 41161 diff changeset	662 ... ' ',
04eb3c5607af contrib: fix import-checker to operate on str instead of bytes Augie Fackler <augie@google.com> parents: 41161 diff changeset	663 ... 'comment',
04eb3c5607af contrib: fix import-checker to operate on str instead of bytes Augie Fackler <augie@google.com> parents: 41161 diff changeset	664 ... ' $ cat > foo.py <<EOF',
04eb3c5607af contrib: fix import-checker to operate on str instead of bytes Augie Fackler <augie@google.com> parents: 41161 diff changeset	665 ... ' > from __future__ import print_function',
04eb3c5607af contrib: fix import-checker to operate on str instead of bytes Augie Fackler <augie@google.com> parents: 41161 diff changeset	666 ... ' > EOF',
28922 4ec62a084e5c import-checker: parse python code from .t files timeless <timeless@mozdev.org> parents: 28921 diff changeset	667 ... ]
33896 bcf53149ebce contrib: make import checker always think in terms of bytes Augie Fackler <raf@durin42.com> parents: 33895 diff changeset	668 >>> test(b"example.t", lines)
40096 7288838bec1f import-checker: use testparseutil.embedded() to centralize detection logic FUJIWARA Katsunori <foozy@lares.dti.ne.jp> parents: 39356 diff changeset	669 example[2] doctest.py 1
7288838bec1f import-checker: use testparseutil.embedded() to centralize detection logic FUJIWARA Katsunori <foozy@lares.dti.ne.jp> parents: 39356 diff changeset	670 "from __future__ import print_function\\n' multiline\\nstring'\\n\\n"
7288838bec1f import-checker: use testparseutil.embedded() to centralize detection logic FUJIWARA Katsunori <foozy@lares.dti.ne.jp> parents: 39356 diff changeset	671 example[8] foo.py 7
28922 4ec62a084e5c import-checker: parse python code from .t files timeless <timeless@mozdev.org> parents: 28921 diff changeset	672 'from __future__ import print_function\\n'
4ec62a084e5c import-checker: parse python code from .t files timeless <timeless@mozdev.org> parents: 28921 diff changeset	673 """
40096 7288838bec1f import-checker: use testparseutil.embedded() to centralize detection logic FUJIWARA Katsunori <foozy@lares.dti.ne.jp> parents: 39356 diff changeset	674 errors = []
7288838bec1f import-checker: use testparseutil.embedded() to centralize detection logic FUJIWARA Katsunori <foozy@lares.dti.ne.jp> parents: 39356 diff changeset	675 for name, starts, ends, code in testparseutil.pyembedded(f, src, errors):
7288838bec1f import-checker: use testparseutil.embedded() to centralize detection logic FUJIWARA Katsunori <foozy@lares.dti.ne.jp> parents: 39356 diff changeset	676 if not name:
7288838bec1f import-checker: use testparseutil.embedded() to centralize detection logic FUJIWARA Katsunori <foozy@lares.dti.ne.jp> parents: 39356 diff changeset	677 # use 'doctest.py', in order to make already existing
7288838bec1f import-checker: use testparseutil.embedded() to centralize detection logic FUJIWARA Katsunori <foozy@lares.dti.ne.jp> parents: 39356 diff changeset	678 # doctest above pass instantly
7288838bec1f import-checker: use testparseutil.embedded() to centralize detection logic FUJIWARA Katsunori <foozy@lares.dti.ne.jp> parents: 39356 diff changeset	679 name = 'doctest.py'
7288838bec1f import-checker: use testparseutil.embedded() to centralize detection logic FUJIWARA Katsunori <foozy@lares.dti.ne.jp> parents: 39356 diff changeset	680 # "starts" is "line number" (1-origin), but embedded() is
7288838bec1f import-checker: use testparseutil.embedded() to centralize detection logic FUJIWARA Katsunori <foozy@lares.dti.ne.jp> parents: 39356 diff changeset	681 # expected to return "line offset" (0-origin). Therefore, this
7288838bec1f import-checker: use testparseutil.embedded() to centralize detection logic FUJIWARA Katsunori <foozy@lares.dti.ne.jp> parents: 39356 diff changeset	682 # yields "starts - 1".
40530 99d5424eedc8 contrib: fix import-checker to not b'' module names on Python 3 Augie Fackler <augie@google.com> parents: 40096 diff changeset	683 if not isinstance(modname, str):
99d5424eedc8 contrib: fix import-checker to not b'' module names on Python 3 Augie Fackler <augie@google.com> parents: 40096 diff changeset	684 modname = modname.decode('utf8')
40096 7288838bec1f import-checker: use testparseutil.embedded() to centralize detection logic FUJIWARA Katsunori <foozy@lares.dti.ne.jp> parents: 39356 diff changeset	685 yield code, "%s[%d]" % (modname, starts), name, starts - 1
28922 4ec62a084e5c import-checker: parse python code from .t files timeless <timeless@mozdev.org> parents: 28921 diff changeset	686
43075 57875cf423c9 style: run a patched black on a subset of mercurial Augie Fackler <augie@google.com> parents: 42823 diff changeset	687
28919 a94f34306bb9 import-checker: refactor source reading timeless <timeless@mozdev.org> parents: 28713 diff changeset	688 def sources(f, modname):
28922 4ec62a084e5c import-checker: parse python code from .t files timeless <timeless@mozdev.org> parents: 28921 diff changeset	689 """Yields possibly multiple sources from a filepath
4ec62a084e5c import-checker: parse python code from .t files timeless <timeless@mozdev.org> parents: 28921 diff changeset	690
4ec62a084e5c import-checker: parse python code from .t files timeless <timeless@mozdev.org> parents: 28921 diff changeset	691 input: filepath, modulename
4ec62a084e5c import-checker: parse python code from .t files timeless <timeless@mozdev.org> parents: 28921 diff changeset	692 yields: script(string), modulename, filepath, linenumber
4ec62a084e5c import-checker: parse python code from .t files timeless <timeless@mozdev.org> parents: 28921 diff changeset	693
4ec62a084e5c import-checker: parse python code from .t files timeless <timeless@mozdev.org> parents: 28921 diff changeset	694 For embedded scripts, the modulename and filepath will be different
4ec62a084e5c import-checker: parse python code from .t files timeless <timeless@mozdev.org> parents: 28921 diff changeset	695 from the function arguments. linenumber is an offset relative to
4ec62a084e5c import-checker: parse python code from .t files timeless <timeless@mozdev.org> parents: 28921 diff changeset	696 the input file.
4ec62a084e5c import-checker: parse python code from .t files timeless <timeless@mozdev.org> parents: 28921 diff changeset	697 """
4ec62a084e5c import-checker: parse python code from .t files timeless <timeless@mozdev.org> parents: 28921 diff changeset	698 py = False
29234 393aef802535 tests: enable import checker for all python files (including no .py files) Yuya Nishihara <yuya@tcha.org> parents: 29211 diff changeset	699 if not f.endswith('.t'):
33896 bcf53149ebce contrib: make import checker always think in terms of bytes Augie Fackler <raf@durin42.com> parents: 33895 diff changeset	700 with open(f, 'rb') as src:
28922 4ec62a084e5c import-checker: parse python code from .t files timeless <timeless@mozdev.org> parents: 28921 diff changeset	701 yield src.read(), modname, f, 0
4ec62a084e5c import-checker: parse python code from .t files timeless <timeless@mozdev.org> parents: 28921 diff changeset	702 py = True
4ec62a084e5c import-checker: parse python code from .t files timeless <timeless@mozdev.org> parents: 28921 diff changeset	703 if py or f.endswith('.t'):
43415 a8454e846736 import-checker: open all source files as utf-8 Gregory Szorc <gregory.szorc@gmail.com> parents: 43084 diff changeset	704 # Strictly speaking we should sniff for the magic header that denotes
a8454e846736 import-checker: open all source files as utf-8 Gregory Szorc <gregory.szorc@gmail.com> parents: 43084 diff changeset	705 # Python source file encoding. But in reality we don't use anything
a8454e846736 import-checker: open all source files as utf-8 Gregory Szorc <gregory.szorc@gmail.com> parents: 43084 diff changeset	706 # other than ASCII (mainly) and UTF-8 (in a few exceptions), so
a8454e846736 import-checker: open all source files as utf-8 Gregory Szorc <gregory.szorc@gmail.com> parents: 43084 diff changeset	707 # simplicity is fine.
a8454e846736 import-checker: open all source files as utf-8 Gregory Szorc <gregory.szorc@gmail.com> parents: 43084 diff changeset	708 with io.open(f, 'r', encoding='utf-8') as src:
28922 4ec62a084e5c import-checker: parse python code from .t files timeless <timeless@mozdev.org> parents: 28921 diff changeset	709 for script, modname, t, line in embedded(f, modname, src):
40531 90517fad4293 contrib: tweak import-checker to always use bytes for module names Augie Fackler <augie@google.com> parents: 40530 diff changeset	710 yield script, modname.encode('utf8'), t, line
28919 a94f34306bb9 import-checker: refactor source reading timeless <timeless@mozdev.org> parents: 28713 diff changeset	711
43075 57875cf423c9 style: run a patched black on a subset of mercurial Augie Fackler <augie@google.com> parents: 42823 diff changeset	712
20036 e5d51fa51aba contrib: add an import checker Augie Fackler <raf@durin42.com> parents: diff changeset	713 def main(argv):
25063 723e364488f4 import-checker: add xargs like mode FUJIWARA Katsunori <foozy@lares.dti.ne.jp> parents: 24669 diff changeset	714 if len(argv) < 2 or (argv[1] == '-' and len(argv) > 2):
28703 a274c4f9087a py3: use print_function in import-checker timeless <timeless@mozdev.org> parents: 28702 diff changeset	715 print('Usage: %s {-\|file [file] [file] ...}')
20036 e5d51fa51aba contrib: add an import checker Augie Fackler <raf@durin42.com> parents: diff changeset	716 return 1
25063 723e364488f4 import-checker: add xargs like mode FUJIWARA Katsunori <foozy@lares.dti.ne.jp> parents: 24669 diff changeset	717 if argv[1] == '-':
723e364488f4 import-checker: add xargs like mode FUJIWARA Katsunori <foozy@lares.dti.ne.jp> parents: 24669 diff changeset	718 argv = argv[:1]
723e364488f4 import-checker: add xargs like mode FUJIWARA Katsunori <foozy@lares.dti.ne.jp> parents: 24669 diff changeset	719 argv.extend(l.rstrip() for l in sys.stdin.readlines())
32541 4c712b90c60a import-checker: convert localmods to a set of module names Yuya Nishihara <yuya@tcha.org> parents: 32540 diff changeset	720 localmodpaths = {}
20036 e5d51fa51aba contrib: add an import checker Augie Fackler <raf@durin42.com> parents: diff changeset	721 used_imports = {}
e5d51fa51aba contrib: add an import checker Augie Fackler <raf@durin42.com> parents: diff changeset	722 any_errors = False
e5d51fa51aba contrib: add an import checker Augie Fackler <raf@durin42.com> parents: diff changeset	723 for source_path in argv[1:]:
32413 194b0f781132 import-checker: drop workaround for pure modules Yuya Nishihara <yuya@tcha.org> parents: 32411 diff changeset	724 modname = dotted_name_of_path(source_path)
32541 4c712b90c60a import-checker: convert localmods to a set of module names Yuya Nishihara <yuya@tcha.org> parents: 32540 diff changeset	725 localmodpaths[modname] = source_path
32542 a025ec43856c import-checker: guess names of C extension modules Yuya Nishihara <yuya@tcha.org> parents: 32541 diff changeset	726 localmods = populateextmods(localmodpaths)
32541 4c712b90c60a import-checker: convert localmods to a set of module names Yuya Nishihara <yuya@tcha.org> parents: 32540 diff changeset	727 for localmodname, source_path in sorted(localmodpaths.items()):
33908 3de9a2df6411 contrib: have import-checker work mostly with native strings for mod names Augie Fackler <raf@durin42.com> parents: 33896 diff changeset	728 if not isinstance(localmodname, bytes):
3de9a2df6411 contrib: have import-checker work mostly with native strings for mod names Augie Fackler <raf@durin42.com> parents: 33896 diff changeset	729 # This is only safe because all hg's files are ascii
3de9a2df6411 contrib: have import-checker work mostly with native strings for mod names Augie Fackler <raf@durin42.com> parents: 33896 diff changeset	730 localmodname = localmodname.encode('ascii')
28922 4ec62a084e5c import-checker: parse python code from .t files timeless <timeless@mozdev.org> parents: 28921 diff changeset	731 for src, modname, name, line in sources(source_path, localmodname):
28920 cdf331b54eb8 import-checker: track SyntaxErrors timeless <timeless@mozdev.org> parents: 28919 diff changeset	732 try:
cdf331b54eb8 import-checker: track SyntaxErrors timeless <timeless@mozdev.org> parents: 28919 diff changeset	733 used_imports[modname] = sorted(
43075 57875cf423c9 style: run a patched black on a subset of mercurial Augie Fackler <augie@google.com> parents: 42823 diff changeset	734 imported_modules(
57875cf423c9 style: run a patched black on a subset of mercurial Augie Fackler <augie@google.com> parents: 42823 diff changeset	735 src, modname, name, localmods, ignore_nested=True
57875cf423c9 style: run a patched black on a subset of mercurial Augie Fackler <augie@google.com> parents: 42823 diff changeset	736 )
57875cf423c9 style: run a patched black on a subset of mercurial Augie Fackler <augie@google.com> parents: 42823 diff changeset	737 )
57875cf423c9 style: run a patched black on a subset of mercurial Augie Fackler <augie@google.com> parents: 42823 diff changeset	738 for error, lineno in verify_import_convention(
57875cf423c9 style: run a patched black on a subset of mercurial Augie Fackler <augie@google.com> parents: 42823 diff changeset	739 modname, src, localmods
57875cf423c9 style: run a patched black on a subset of mercurial Augie Fackler <augie@google.com> parents: 42823 diff changeset	740 ):
28920 cdf331b54eb8 import-checker: track SyntaxErrors timeless <timeless@mozdev.org> parents: 28919 diff changeset	741 any_errors = True
28922 4ec62a084e5c import-checker: parse python code from .t files timeless <timeless@mozdev.org> parents: 28921 diff changeset	742 print('%s:%d: %s' % (source_path, lineno + line, error))
28920 cdf331b54eb8 import-checker: track SyntaxErrors timeless <timeless@mozdev.org> parents: 28919 diff changeset	743 except SyntaxError as e:
43075 57875cf423c9 style: run a patched black on a subset of mercurial Augie Fackler <augie@google.com> parents: 42823 diff changeset	744 print(
57875cf423c9 style: run a patched black on a subset of mercurial Augie Fackler <augie@google.com> parents: 42823 diff changeset	745 '%s:%d: SyntaxError: %s' % (source_path, e.lineno + line, e)
57875cf423c9 style: run a patched black on a subset of mercurial Augie Fackler <augie@google.com> parents: 42823 diff changeset	746 )
20036 e5d51fa51aba contrib: add an import checker Augie Fackler <raf@durin42.com> parents: diff changeset	747 cycles = find_cycles(used_imports)
e5d51fa51aba contrib: add an import checker Augie Fackler <raf@durin42.com> parents: diff changeset	748 if cycles:
e5d51fa51aba contrib: add an import checker Augie Fackler <raf@durin42.com> parents: diff changeset	749 firstmods = set()
e5d51fa51aba contrib: add an import checker Augie Fackler <raf@durin42.com> parents: diff changeset	750 for c in sorted(cycles, key=_cycle_sortkey):
e5d51fa51aba contrib: add an import checker Augie Fackler <raf@durin42.com> parents: diff changeset	751 first = c.split()[0]
e5d51fa51aba contrib: add an import checker Augie Fackler <raf@durin42.com> parents: diff changeset	752 # As a rough cut, ignore any cycle that starts with the
e5d51fa51aba contrib: add an import checker Augie Fackler <raf@durin42.com> parents: diff changeset	753 # same module as some other cycle. Otherwise we see lots
e5d51fa51aba contrib: add an import checker Augie Fackler <raf@durin42.com> parents: diff changeset	754 # of cycles that are effectively duplicates.
e5d51fa51aba contrib: add an import checker Augie Fackler <raf@durin42.com> parents: diff changeset	755 if first in firstmods:
e5d51fa51aba contrib: add an import checker Augie Fackler <raf@durin42.com> parents: diff changeset	756 continue
28703 a274c4f9087a py3: use print_function in import-checker timeless <timeless@mozdev.org> parents: 28702 diff changeset	757 print('Import cycle:', c)
20036 e5d51fa51aba contrib: add an import checker Augie Fackler <raf@durin42.com> parents: diff changeset	758 firstmods.add(first)
e5d51fa51aba contrib: add an import checker Augie Fackler <raf@durin42.com> parents: diff changeset	759 any_errors = True
25731 cd1daab5d036 import-checker.py: exit with code 0 if no error is detected FUJIWARA Katsunori <foozy@lares.dti.ne.jp> parents: 25703 diff changeset	760 return any_errors != 0
20036 e5d51fa51aba contrib: add an import checker Augie Fackler <raf@durin42.com> parents: diff changeset	761
43075 57875cf423c9 style: run a patched black on a subset of mercurial Augie Fackler <augie@google.com> parents: 42823 diff changeset	762
20036 e5d51fa51aba contrib: add an import checker Augie Fackler <raf@durin42.com> parents: diff changeset	763 if __name__ == '__main__':
e5d51fa51aba contrib: add an import checker Augie Fackler <raf@durin42.com> parents: diff changeset	764 sys.exit(int(main(sys.argv)))

Mercurial > public > mercurial-scm > hg-stable

annotate contrib/import-checker.py @ 52669:e627cc25b6f3