mercurial-scm/hg-stable: contrib/import-checker.py annotate

annotate contrib/import-checker.py @ 53040:cdd7bf612c7b stable tip

bundle-spec: properly format boolean parameter (issue6960) This was breaking automatic clone bundle generation. This changeset fixes it and add a test to catch it in the future.

author	Pierre-Yves David <pierre-yves.david@octobus.net>
date	Tue, 11 Mar 2025 02:29:42 +0100
parents	5c48fd4c0e68
children

rev	line source
45849 c102b704edb5 global: use python3 in shebangs Gregory Szorc <gregory.szorc@gmail.com> parents: 44440 diff changeset	1 #!/usr/bin/env python3
26954 f804bf27439b import-checker: make it executable for convenience Yuya Nishihara <yuya@tcha.org> parents: 26781 diff changeset	2
28702 e44f671018e3 py3: use absolute_import in import-checker timeless <timeless@mozdev.org> parents: 28700 diff changeset	3
20036 e5d51fa51aba contrib: add an import checker Augie Fackler <raf@durin42.com> parents: diff changeset	4 import ast
26965 1fa66d3ad28d import-checker: reset context to verify convention in function scope Yuya Nishihara <yuya@tcha.org> parents: 26964 diff changeset	5 import collections
20036 e5d51fa51aba contrib: add an import checker Augie Fackler <raf@durin42.com> parents: diff changeset	6 import os
e5d51fa51aba contrib: add an import checker Augie Fackler <raf@durin42.com> parents: diff changeset	7 import sys
e5d51fa51aba contrib: add an import checker Augie Fackler <raf@durin42.com> parents: diff changeset	8
20198 f5393a9dc4e5 import-checker: make test-module-imports.t work using virtualenv (issue4129) Chris Jerdonek <chris.jerdonek@gmail.com> parents: 20197 diff changeset	9 # Import a minimal set of stdlib modules needed for list_stdlib_modules()
f5393a9dc4e5 import-checker: make test-module-imports.t work using virtualenv (issue4129) Chris Jerdonek <chris.jerdonek@gmail.com> parents: 20197 diff changeset	10 # to work when run from a virtualenv. The modules were chosen empirically
f5393a9dc4e5 import-checker: make test-module-imports.t work using virtualenv (issue4129) Chris Jerdonek <chris.jerdonek@gmail.com> parents: 20197 diff changeset	11 # so that the return value matches the return value without virtualenv.
43075 57875cf423c9 style: run a patched black on a subset of mercurial Augie Fackler <augie@google.com> parents: 42823 diff changeset	12 if True: # disable lexical sorting checks
51373 cf1bee12ecdb import-checker: make stdlib path detection work in virtual environments Manuel Jacob <me@manueljacob.de> parents: 50800 diff changeset	13 import argparse
29211 b42c2a66a698 py3: make contrib/import-checker.py get along with itself Yuya Nishihara <yuya@tcha.org> parents: 29208 diff changeset	14 import zlib
20198 f5393a9dc4e5 import-checker: make test-module-imports.t work using virtualenv (issue4129) Chris Jerdonek <chris.jerdonek@gmail.com> parents: 20197 diff changeset	15
40096 7288838bec1f import-checker: use testparseutil.embedded() to centralize detection logic FUJIWARA Katsunori <foozy@lares.dti.ne.jp> parents: 39356 diff changeset	16 import testparseutil
7288838bec1f import-checker: use testparseutil.embedded() to centralize detection logic FUJIWARA Katsunori <foozy@lares.dti.ne.jp> parents: 39356 diff changeset	17
49052 8dec9abf2669 import-checker: allow symbol imports from typing module Gregory Szorc <gregory.szorc@gmail.com> parents: 48970 diff changeset	18 # Allow list of modules that symbols can be directly imported from.
25703 1a6a117d0b95 import-checker: establish modern import convention Gregory Szorc <gregory.szorc@gmail.com> parents: 25702 diff changeset	19 allowsymbolimports = (
1a6a117d0b95 import-checker: establish modern import convention Gregory Szorc <gregory.szorc@gmail.com> parents: 25702 diff changeset	20 '__future__',
47383 26127236b229 convert-bazaar: use breezy package instead of old bzr one Rapha?l Gom?s <rgomes@octobus.net> parents: 45849 diff changeset	21 'breezy',
48925 029b76d645dc imports: allow importing futures from concurrent Augie Fackler <augie@google.com> parents: 47797 diff changeset	22 'concurrent',
33933 2d64b2f1787b contrib: allow symbol imports from hgclient for tests Augie Fackler <raf@durin42.com> parents: 33915 diff changeset	23 'hgclient',
33911 c9cf69d0c3b9 contrib: allow importing "symbols" from mercurial Augie Fackler <raf@durin42.com> parents: 33908 diff changeset	24 'mercurial',
27018 e5be48dd8215 import-checker: allow symbol imports from hgweb.common and .request Yuya Nishihara <yuya@tcha.org> parents: 26965 diff changeset	25 'mercurial.hgweb.common',
e5be48dd8215 import-checker: allow symbol imports from hgweb.common and .request Yuya Nishihara <yuya@tcha.org> parents: 26965 diff changeset	26 'mercurial.hgweb.request',
25703 1a6a117d0b95 import-checker: establish modern import convention Gregory Szorc <gregory.szorc@gmail.com> parents: 25702 diff changeset	27 'mercurial.i18n',
42823 268662aac075 interfaces: create a new folder for interfaces and move repository.py in it Pulkit Goyal <pulkit@yandex-team.ru> parents: 42745 diff changeset	28 'mercurial.interfaces',
52755 5c48fd4c0e68 typing: introduce a `types` module and a MatcherT alias Pierre-Yves David <pierre-yves.david@octobus.net> parents: 52681 diff changeset	29 'mercurial.interfaces._basetypes',
5c48fd4c0e68 typing: introduce a `types` module and a MatcherT alias Pierre-Yves David <pierre-yves.david@octobus.net> parents: 52681 diff changeset	30 'mercurial.interfaces.types',
25703 1a6a117d0b95 import-checker: establish modern import convention Gregory Szorc <gregory.szorc@gmail.com> parents: 25702 diff changeset	31 'mercurial.node',
43084 c2e284cee333 import-checker: allow symbol imports from mercurial.pycompat Gregory Szorc <gregory.szorc@gmail.com> parents: 43075 diff changeset	32 'mercurial.pycompat',
39356 729082bb9938 revlog: split constants into a new `revlogutils.constants` module Boris Feld <boris.feld@octobus.net> parents: 38837 diff changeset	33 # for revlog to re-export constant to extensions
729082bb9938 revlog: split constants into a new `revlogutils.constants` module Boris Feld <boris.feld@octobus.net> parents: 38837 diff changeset	34 'mercurial.revlogutils.constants',
42745 ca5ca3badd3c flagutil: create a `mercurial.revlogutils.flagutil` module Pierre-Yves David <pierre-yves.david@octobus.net> parents: 42407 diff changeset	35 'mercurial.revlogutils.flagutil',
32540 95085d747db8 import-checker: allow importing symbols from pure modules Yuya Nishihara <yuya@tcha.org> parents: 32458 diff changeset	36 # for cffi modules to re-export pure functions
95085d747db8 import-checker: allow importing symbols from pure modules Yuya Nishihara <yuya@tcha.org> parents: 32458 diff changeset	37 'mercurial.pure.base85',
95085d747db8 import-checker: allow importing symbols from pure modules Yuya Nishihara <yuya@tcha.org> parents: 32458 diff changeset	38 'mercurial.pure.bdiff',
95085d747db8 import-checker: allow importing symbols from pure modules Yuya Nishihara <yuya@tcha.org> parents: 32458 diff changeset	39 'mercurial.pure.mpatch',
95085d747db8 import-checker: allow importing symbols from pure modules Yuya Nishihara <yuya@tcha.org> parents: 32458 diff changeset	40 'mercurial.pure.osutil',
95085d747db8 import-checker: allow importing symbols from pure modules Yuya Nishihara <yuya@tcha.org> parents: 32458 diff changeset	41 'mercurial.pure.parsers',
34396 41401f502c83 tests: disable lints on mercurial/thirdparty Siddharth Agarwal <sid0@fb.com> parents: 34055 diff changeset	42 # third-party imports should be directly imported
41401f502c83 tests: disable lints on mercurial/thirdparty Siddharth Agarwal <sid0@fb.com> parents: 34055 diff changeset	43 'mercurial.thirdparty',
38837 8751d1e2a7ff util: create a context manager to handle timing Martijn Pieters <mj@zopatista.com> parents: 37711 diff changeset	44 'mercurial.thirdparty.attr',
50442 82e5a9b1ef1e extras: re-use Projection from jaraco.collections Jason R. Coombs <jaraco@jaraco.com> parents: 50275 diff changeset	45 'mercurial.thirdparty.jaraco.collections',
50800 2c34c9b61a4f thirdparty: vendor tomli Rapha?l Gom?s <rgomes@octobus.net> parents: 50442 diff changeset	46 'mercurial.thirdparty.tomli',
49052 8dec9abf2669 import-checker: allow symbol imports from typing module Gregory Szorc <gregory.szorc@gmail.com> parents: 48970 diff changeset	47 'typing',
49369 1572f790ee5e convert: remove old ElementTree import cruft from darcs Ian Moody <moz-ian@perix.co.uk> parents: 49052 diff changeset	48 'xml.etree.ElementTree',
25703 1a6a117d0b95 import-checker: establish modern import convention Gregory Szorc <gregory.szorc@gmail.com> parents: 25702 diff changeset	49 )
1a6a117d0b95 import-checker: establish modern import convention Gregory Szorc <gregory.szorc@gmail.com> parents: 25702 diff changeset	50
49052 8dec9abf2669 import-checker: allow symbol imports from typing module Gregory Szorc <gregory.szorc@gmail.com> parents: 48970 diff changeset	51 # Allow list of symbols that can be directly imported.
43075 57875cf423c9 style: run a patched black on a subset of mercurial Augie Fackler <augie@google.com> parents: 42823 diff changeset	52 directsymbols = ('demandimport',)
32457 d02888308235 import-checker: add a way to directly import certain symbols Siddharth Agarwal <sid0@fb.com> parents: 32413 diff changeset	53
25703 1a6a117d0b95 import-checker: establish modern import convention Gregory Szorc <gregory.szorc@gmail.com> parents: 25702 diff changeset	54 # Modules that must be aliased because they are commonly confused with
1a6a117d0b95 import-checker: establish modern import convention Gregory Szorc <gregory.szorc@gmail.com> parents: 25702 diff changeset	55 # common variables and can create aliasing and readability issues.
1a6a117d0b95 import-checker: establish modern import convention Gregory Szorc <gregory.szorc@gmail.com> parents: 25702 diff changeset	56 requirealias = {
1a6a117d0b95 import-checker: establish modern import convention Gregory Szorc <gregory.szorc@gmail.com> parents: 25702 diff changeset	57 'ui': 'uimod',
1a6a117d0b95 import-checker: establish modern import convention Gregory Szorc <gregory.szorc@gmail.com> parents: 25702 diff changeset	58 }
1a6a117d0b95 import-checker: establish modern import convention Gregory Szorc <gregory.szorc@gmail.com> parents: 25702 diff changeset	59
43075 57875cf423c9 style: run a patched black on a subset of mercurial Augie Fackler <augie@google.com> parents: 42823 diff changeset	60
26965 1fa66d3ad28d import-checker: reset context to verify convention in function scope Yuya Nishihara <yuya@tcha.org> parents: 26964 diff changeset	61 def walklocal(root):
1fa66d3ad28d import-checker: reset context to verify convention in function scope Yuya Nishihara <yuya@tcha.org> parents: 26964 diff changeset	62 """Recursively yield all descendant nodes but not in a different scope"""
1fa66d3ad28d import-checker: reset context to verify convention in function scope Yuya Nishihara <yuya@tcha.org> parents: 26964 diff changeset	63 todo = collections.deque(ast.iter_child_nodes(root))
1fa66d3ad28d import-checker: reset context to verify convention in function scope Yuya Nishihara <yuya@tcha.org> parents: 26964 diff changeset	64 yield root, False
1fa66d3ad28d import-checker: reset context to verify convention in function scope Yuya Nishihara <yuya@tcha.org> parents: 26964 diff changeset	65 while todo:
1fa66d3ad28d import-checker: reset context to verify convention in function scope Yuya Nishihara <yuya@tcha.org> parents: 26964 diff changeset	66 node = todo.popleft()
1fa66d3ad28d import-checker: reset context to verify convention in function scope Yuya Nishihara <yuya@tcha.org> parents: 26964 diff changeset	67 newscope = isinstance(node, ast.FunctionDef)
1fa66d3ad28d import-checker: reset context to verify convention in function scope Yuya Nishihara <yuya@tcha.org> parents: 26964 diff changeset	68 if not newscope:
1fa66d3ad28d import-checker: reset context to verify convention in function scope Yuya Nishihara <yuya@tcha.org> parents: 26964 diff changeset	69 todo.extend(ast.iter_child_nodes(node))
1fa66d3ad28d import-checker: reset context to verify convention in function scope Yuya Nishihara <yuya@tcha.org> parents: 26964 diff changeset	70 yield node, newscope
1fa66d3ad28d import-checker: reset context to verify convention in function scope Yuya Nishihara <yuya@tcha.org> parents: 26964 diff changeset	71
43075 57875cf423c9 style: run a patched black on a subset of mercurial Augie Fackler <augie@google.com> parents: 42823 diff changeset	72
32413 194b0f781132 import-checker: drop workaround for pure modules Yuya Nishihara <yuya@tcha.org> parents: 32411 diff changeset	73 def dotted_name_of_path(path):
20036 e5d51fa51aba contrib: add an import checker Augie Fackler <raf@durin42.com> parents: diff changeset	74 """Given a relative path to a source file, return its dotted module name.
e5d51fa51aba contrib: add an import checker Augie Fackler <raf@durin42.com> parents: diff changeset	75
e5d51fa51aba contrib: add an import checker Augie Fackler <raf@durin42.com> parents: diff changeset	76 >>> dotted_name_of_path('mercurial/error.py')
e5d51fa51aba contrib: add an import checker Augie Fackler <raf@durin42.com> parents: diff changeset	77 'mercurial.error'
20383 4990abb4729d import-checker: fix names of dynamically loaded modules Mads Kiilerich <madski@unity3d.com> parents: 20238 diff changeset	78 >>> dotted_name_of_path('zlibmodule.so')
4990abb4729d import-checker: fix names of dynamically loaded modules Mads Kiilerich <madski@unity3d.com> parents: 20238 diff changeset	79 'zlib'
20036 e5d51fa51aba contrib: add an import checker Augie Fackler <raf@durin42.com> parents: diff changeset	80 """
27620 0c60843b55b5 import-checker: normalize directory separator to get module name on Windows Yuya Nishihara <yuya@tcha.org> parents: 27520 diff changeset	81 parts = path.replace(os.sep, '/').split('/')
43075 57875cf423c9 style: run a patched black on a subset of mercurial Augie Fackler <augie@google.com> parents: 42823 diff changeset	82 parts[-1] = parts[-1].split('.', 1)[0] # remove .py and .so and .ARCH.so
20383 4990abb4729d import-checker: fix names of dynamically loaded modules Mads Kiilerich <madski@unity3d.com> parents: 20238 diff changeset	83 if parts[-1].endswith('module'):
4990abb4729d import-checker: fix names of dynamically loaded modules Mads Kiilerich <madski@unity3d.com> parents: 20238 diff changeset	84 parts[-1] = parts[-1][:-6]
20036 e5d51fa51aba contrib: add an import checker Augie Fackler <raf@durin42.com> parents: diff changeset	85 return '.'.join(parts)
e5d51fa51aba contrib: add an import checker Augie Fackler <raf@durin42.com> parents: diff changeset	86
43075 57875cf423c9 style: run a patched black on a subset of mercurial Augie Fackler <augie@google.com> parents: 42823 diff changeset	87
25173 7358b5d9991e import-checker: add utility to examine what module is imported easily FUJIWARA Katsunori <foozy@lares.dti.ne.jp> parents: 25064 diff changeset	88 def fromlocalfunc(modulename, localmods):
7358b5d9991e import-checker: add utility to examine what module is imported easily FUJIWARA Katsunori <foozy@lares.dti.ne.jp> parents: 25064 diff changeset	89 """Get a function to examine which locally defined module the
7358b5d9991e import-checker: add utility to examine what module is imported easily FUJIWARA Katsunori <foozy@lares.dti.ne.jp> parents: 25064 diff changeset	90 target source imports via a specified name.
7358b5d9991e import-checker: add utility to examine what module is imported easily FUJIWARA Katsunori <foozy@lares.dti.ne.jp> parents: 25064 diff changeset	91
7358b5d9991e import-checker: add utility to examine what module is imported easily FUJIWARA Katsunori <foozy@lares.dti.ne.jp> parents: 25064 diff changeset	92 `modulename` is an `dotted_name_of_path()`-ed source file path,
7358b5d9991e import-checker: add utility to examine what module is imported easily FUJIWARA Katsunori <foozy@lares.dti.ne.jp> parents: 25064 diff changeset	93 which may have `.__init__` at the end of it, of the target source.
7358b5d9991e import-checker: add utility to examine what module is imported easily FUJIWARA Katsunori <foozy@lares.dti.ne.jp> parents: 25064 diff changeset	94
32541 4c712b90c60a import-checker: convert localmods to a set of module names Yuya Nishihara <yuya@tcha.org> parents: 32540 diff changeset	95 `localmods` is a set of absolute `dotted_name_of_path()`-ed source file
4c712b90c60a import-checker: convert localmods to a set of module names Yuya Nishihara <yuya@tcha.org> parents: 32540 diff changeset	96 paths of locally defined (= Mercurial specific) modules.
25173 7358b5d9991e import-checker: add utility to examine what module is imported easily FUJIWARA Katsunori <foozy@lares.dti.ne.jp> parents: 25064 diff changeset	97
7358b5d9991e import-checker: add utility to examine what module is imported easily FUJIWARA Katsunori <foozy@lares.dti.ne.jp> parents: 25064 diff changeset	98 This function assumes that module names not existing in
26781 1aee2ab0f902 spelling: trivial spell checking Mads Kiilerich <madski@unity3d.com> parents: 26221 diff changeset	99 `localmods` are from the Python standard library.
25173 7358b5d9991e import-checker: add utility to examine what module is imported easily FUJIWARA Katsunori <foozy@lares.dti.ne.jp> parents: 25064 diff changeset	100
7358b5d9991e import-checker: add utility to examine what module is imported easily FUJIWARA Katsunori <foozy@lares.dti.ne.jp> parents: 25064 diff changeset	101 This function returns the function, which takes `name` argument,
7358b5d9991e import-checker: add utility to examine what module is imported easily FUJIWARA Katsunori <foozy@lares.dti.ne.jp> parents: 25064 diff changeset	102 and returns `(absname, dottedpath, hassubmod)` tuple if `name`
7358b5d9991e import-checker: add utility to examine what module is imported easily FUJIWARA Katsunori <foozy@lares.dti.ne.jp> parents: 25064 diff changeset	103 matches against locally defined module. Otherwise, it returns
7358b5d9991e import-checker: add utility to examine what module is imported easily FUJIWARA Katsunori <foozy@lares.dti.ne.jp> parents: 25064 diff changeset	104 False.
7358b5d9991e import-checker: add utility to examine what module is imported easily FUJIWARA Katsunori <foozy@lares.dti.ne.jp> parents: 25064 diff changeset	105
7358b5d9991e import-checker: add utility to examine what module is imported easily FUJIWARA Katsunori <foozy@lares.dti.ne.jp> parents: 25064 diff changeset	106 It is assumed that `name` doesn't have `.__init__`.
7358b5d9991e import-checker: add utility to examine what module is imported easily FUJIWARA Katsunori <foozy@lares.dti.ne.jp> parents: 25064 diff changeset	107
7358b5d9991e import-checker: add utility to examine what module is imported easily FUJIWARA Katsunori <foozy@lares.dti.ne.jp> parents: 25064 diff changeset	108 `absname` is an absolute module name of specified `name`
7358b5d9991e import-checker: add utility to examine what module is imported easily FUJIWARA Katsunori <foozy@lares.dti.ne.jp> parents: 25064 diff changeset	109 (e.g. "hgext.convert"). This can be used to compose prefix for sub
7358b5d9991e import-checker: add utility to examine what module is imported easily FUJIWARA Katsunori <foozy@lares.dti.ne.jp> parents: 25064 diff changeset	110 modules or so.
7358b5d9991e import-checker: add utility to examine what module is imported easily FUJIWARA Katsunori <foozy@lares.dti.ne.jp> parents: 25064 diff changeset	111
7358b5d9991e import-checker: add utility to examine what module is imported easily FUJIWARA Katsunori <foozy@lares.dti.ne.jp> parents: 25064 diff changeset	112 `dottedpath` is a `dotted_name_of_path()`-ed source file path
7358b5d9991e import-checker: add utility to examine what module is imported easily FUJIWARA Katsunori <foozy@lares.dti.ne.jp> parents: 25064 diff changeset	113 (e.g. "hgext.convert.__init__") of `name`. This is used to look
7358b5d9991e import-checker: add utility to examine what module is imported easily FUJIWARA Katsunori <foozy@lares.dti.ne.jp> parents: 25064 diff changeset	114 module up in `localmods` again.
7358b5d9991e import-checker: add utility to examine what module is imported easily FUJIWARA Katsunori <foozy@lares.dti.ne.jp> parents: 25064 diff changeset	115
7358b5d9991e import-checker: add utility to examine what module is imported easily FUJIWARA Katsunori <foozy@lares.dti.ne.jp> parents: 25064 diff changeset	116 `hassubmod` is whether it may have sub modules under it (for
7358b5d9991e import-checker: add utility to examine what module is imported easily FUJIWARA Katsunori <foozy@lares.dti.ne.jp> parents: 25064 diff changeset	117 convenient, even though this is also equivalent to "absname !=
7358b5d9991e import-checker: add utility to examine what module is imported easily FUJIWARA Katsunori <foozy@lares.dti.ne.jp> parents: 25064 diff changeset	118 dottednpath")
7358b5d9991e import-checker: add utility to examine what module is imported easily FUJIWARA Katsunori <foozy@lares.dti.ne.jp> parents: 25064 diff changeset	119
32541 4c712b90c60a import-checker: convert localmods to a set of module names Yuya Nishihara <yuya@tcha.org> parents: 32540 diff changeset	120 >>> localmods = {'foo.__init__', 'foo.foo1',
4c712b90c60a import-checker: convert localmods to a set of module names Yuya Nishihara <yuya@tcha.org> parents: 32540 diff changeset	121 ... 'foo.bar.__init__', 'foo.bar.bar1',
4c712b90c60a import-checker: convert localmods to a set of module names Yuya Nishihara <yuya@tcha.org> parents: 32540 diff changeset	122 ... 'baz.__init__', 'baz.baz1'}
25173 7358b5d9991e import-checker: add utility to examine what module is imported easily FUJIWARA Katsunori <foozy@lares.dti.ne.jp> parents: 25064 diff changeset	123 >>> fromlocal = fromlocalfunc('foo.xxx', localmods)
7358b5d9991e import-checker: add utility to examine what module is imported easily FUJIWARA Katsunori <foozy@lares.dti.ne.jp> parents: 25064 diff changeset	124 >>> # relative
7358b5d9991e import-checker: add utility to examine what module is imported easily FUJIWARA Katsunori <foozy@lares.dti.ne.jp> parents: 25064 diff changeset	125 >>> fromlocal('foo1')
7358b5d9991e import-checker: add utility to examine what module is imported easily FUJIWARA Katsunori <foozy@lares.dti.ne.jp> parents: 25064 diff changeset	126 ('foo.foo1', 'foo.foo1', False)
7358b5d9991e import-checker: add utility to examine what module is imported easily FUJIWARA Katsunori <foozy@lares.dti.ne.jp> parents: 25064 diff changeset	127 >>> fromlocal('bar')
7358b5d9991e import-checker: add utility to examine what module is imported easily FUJIWARA Katsunori <foozy@lares.dti.ne.jp> parents: 25064 diff changeset	128 ('foo.bar', 'foo.bar.__init__', True)
7358b5d9991e import-checker: add utility to examine what module is imported easily FUJIWARA Katsunori <foozy@lares.dti.ne.jp> parents: 25064 diff changeset	129 >>> fromlocal('bar.bar1')
7358b5d9991e import-checker: add utility to examine what module is imported easily FUJIWARA Katsunori <foozy@lares.dti.ne.jp> parents: 25064 diff changeset	130 ('foo.bar.bar1', 'foo.bar.bar1', False)
7358b5d9991e import-checker: add utility to examine what module is imported easily FUJIWARA Katsunori <foozy@lares.dti.ne.jp> parents: 25064 diff changeset	131 >>> # absolute
7358b5d9991e import-checker: add utility to examine what module is imported easily FUJIWARA Katsunori <foozy@lares.dti.ne.jp> parents: 25064 diff changeset	132 >>> fromlocal('baz')
7358b5d9991e import-checker: add utility to examine what module is imported easily FUJIWARA Katsunori <foozy@lares.dti.ne.jp> parents: 25064 diff changeset	133 ('baz', 'baz.__init__', True)
7358b5d9991e import-checker: add utility to examine what module is imported easily FUJIWARA Katsunori <foozy@lares.dti.ne.jp> parents: 25064 diff changeset	134 >>> fromlocal('baz.baz1')
7358b5d9991e import-checker: add utility to examine what module is imported easily FUJIWARA Katsunori <foozy@lares.dti.ne.jp> parents: 25064 diff changeset	135 ('baz.baz1', 'baz.baz1', False)
7358b5d9991e import-checker: add utility to examine what module is imported easily FUJIWARA Katsunori <foozy@lares.dti.ne.jp> parents: 25064 diff changeset	136 >>> # unknown = maybe standard library
7358b5d9991e import-checker: add utility to examine what module is imported easily FUJIWARA Katsunori <foozy@lares.dti.ne.jp> parents: 25064 diff changeset	137 >>> fromlocal('os')
7358b5d9991e import-checker: add utility to examine what module is imported easily FUJIWARA Katsunori <foozy@lares.dti.ne.jp> parents: 25064 diff changeset	138 False
25701 1f88c0f6ff5a import-checker: resolve relative imports Gregory Szorc <gregory.szorc@gmail.com> parents: 25660 diff changeset	139 >>> fromlocal(None, 1)
1f88c0f6ff5a import-checker: resolve relative imports Gregory Szorc <gregory.szorc@gmail.com> parents: 25660 diff changeset	140 ('foo', 'foo.__init__', True)
29122 660d8d4ec7aa import-checker: recognize relative imports from parents of current package liscju <piotr.listkiewicz@gmail.com> parents: 28922 diff changeset	141 >>> fromlocal('foo1', 1)
660d8d4ec7aa import-checker: recognize relative imports from parents of current package liscju <piotr.listkiewicz@gmail.com> parents: 28922 diff changeset	142 ('foo.foo1', 'foo.foo1', False)
25701 1f88c0f6ff5a import-checker: resolve relative imports Gregory Szorc <gregory.szorc@gmail.com> parents: 25660 diff changeset	143 >>> fromlocal2 = fromlocalfunc('foo.xxx.yyy', localmods)
1f88c0f6ff5a import-checker: resolve relative imports Gregory Szorc <gregory.szorc@gmail.com> parents: 25660 diff changeset	144 >>> fromlocal2(None, 2)
1f88c0f6ff5a import-checker: resolve relative imports Gregory Szorc <gregory.szorc@gmail.com> parents: 25660 diff changeset	145 ('foo', 'foo.__init__', True)
29122 660d8d4ec7aa import-checker: recognize relative imports from parents of current package liscju <piotr.listkiewicz@gmail.com> parents: 28922 diff changeset	146 >>> fromlocal2('bar2', 1)
660d8d4ec7aa import-checker: recognize relative imports from parents of current package liscju <piotr.listkiewicz@gmail.com> parents: 28922 diff changeset	147 False
660d8d4ec7aa import-checker: recognize relative imports from parents of current package liscju <piotr.listkiewicz@gmail.com> parents: 28922 diff changeset	148 >>> fromlocal2('bar', 2)
660d8d4ec7aa import-checker: recognize relative imports from parents of current package liscju <piotr.listkiewicz@gmail.com> parents: 28922 diff changeset	149 ('foo.bar', 'foo.bar.__init__', True)
25173 7358b5d9991e import-checker: add utility to examine what module is imported easily FUJIWARA Katsunori <foozy@lares.dti.ne.jp> parents: 25064 diff changeset	150 """
33908 3de9a2df6411 contrib: have import-checker work mostly with native strings for mod names Augie Fackler <raf@durin42.com> parents: 33896 diff changeset	151 if not isinstance(modulename, str):
3de9a2df6411 contrib: have import-checker work mostly with native strings for mod names Augie Fackler <raf@durin42.com> parents: 33896 diff changeset	152 modulename = modulename.decode('ascii')
25173 7358b5d9991e import-checker: add utility to examine what module is imported easily FUJIWARA Katsunori <foozy@lares.dti.ne.jp> parents: 25064 diff changeset	153 prefix = '.'.join(modulename.split('.')[:-1])
7358b5d9991e import-checker: add utility to examine what module is imported easily FUJIWARA Katsunori <foozy@lares.dti.ne.jp> parents: 25064 diff changeset	154 if prefix:
7358b5d9991e import-checker: add utility to examine what module is imported easily FUJIWARA Katsunori <foozy@lares.dti.ne.jp> parents: 25064 diff changeset	155 prefix += '.'
43075 57875cf423c9 style: run a patched black on a subset of mercurial Augie Fackler <augie@google.com> parents: 42823 diff changeset	156
25701 1f88c0f6ff5a import-checker: resolve relative imports Gregory Szorc <gregory.szorc@gmail.com> parents: 25660 diff changeset	157 def fromlocal(name, level=0):
29374 7712fcde2d56 import-checker: increase portability for python 2.6.x FUJIWARA Katsunori <foozy@lares.dti.ne.jp> parents: 29234 diff changeset	158 # name is false value when relative imports are used.
7712fcde2d56 import-checker: increase portability for python 2.6.x FUJIWARA Katsunori <foozy@lares.dti.ne.jp> parents: 29234 diff changeset	159 if not name:
25701 1f88c0f6ff5a import-checker: resolve relative imports Gregory Szorc <gregory.szorc@gmail.com> parents: 25660 diff changeset	160 # If relative imports are used, level must not be absolute.
1f88c0f6ff5a import-checker: resolve relative imports Gregory Szorc <gregory.szorc@gmail.com> parents: 25660 diff changeset	161 assert level > 0
1f88c0f6ff5a import-checker: resolve relative imports Gregory Szorc <gregory.szorc@gmail.com> parents: 25660 diff changeset	162 candidates = ['.'.join(modulename.split('.')[:-level])]
1f88c0f6ff5a import-checker: resolve relative imports Gregory Szorc <gregory.szorc@gmail.com> parents: 25660 diff changeset	163 else:
29122 660d8d4ec7aa import-checker: recognize relative imports from parents of current package liscju <piotr.listkiewicz@gmail.com> parents: 28922 diff changeset	164 if not level:
660d8d4ec7aa import-checker: recognize relative imports from parents of current package liscju <piotr.listkiewicz@gmail.com> parents: 28922 diff changeset	165 # Check relative name first.
660d8d4ec7aa import-checker: recognize relative imports from parents of current package liscju <piotr.listkiewicz@gmail.com> parents: 28922 diff changeset	166 candidates = [prefix + name, name]
660d8d4ec7aa import-checker: recognize relative imports from parents of current package liscju <piotr.listkiewicz@gmail.com> parents: 28922 diff changeset	167 else:
43075 57875cf423c9 style: run a patched black on a subset of mercurial Augie Fackler <augie@google.com> parents: 42823 diff changeset	168 candidates = [
57875cf423c9 style: run a patched black on a subset of mercurial Augie Fackler <augie@google.com> parents: 42823 diff changeset	169 '.'.join(modulename.split('.')[:-level]) + '.' + name
57875cf423c9 style: run a patched black on a subset of mercurial Augie Fackler <augie@google.com> parents: 42823 diff changeset	170 ]
25701 1f88c0f6ff5a import-checker: resolve relative imports Gregory Szorc <gregory.szorc@gmail.com> parents: 25660 diff changeset	171
1f88c0f6ff5a import-checker: resolve relative imports Gregory Szorc <gregory.szorc@gmail.com> parents: 25660 diff changeset	172 for n in candidates:
25173 7358b5d9991e import-checker: add utility to examine what module is imported easily FUJIWARA Katsunori <foozy@lares.dti.ne.jp> parents: 25064 diff changeset	173 if n in localmods:
7358b5d9991e import-checker: add utility to examine what module is imported easily FUJIWARA Katsunori <foozy@lares.dti.ne.jp> parents: 25064 diff changeset	174 return (n, n, False)
7358b5d9991e import-checker: add utility to examine what module is imported easily FUJIWARA Katsunori <foozy@lares.dti.ne.jp> parents: 25064 diff changeset	175 dottedpath = n + '.__init__'
7358b5d9991e import-checker: add utility to examine what module is imported easily FUJIWARA Katsunori <foozy@lares.dti.ne.jp> parents: 25064 diff changeset	176 if dottedpath in localmods:
7358b5d9991e import-checker: add utility to examine what module is imported easily FUJIWARA Katsunori <foozy@lares.dti.ne.jp> parents: 25064 diff changeset	177 return (n, dottedpath, True)
7358b5d9991e import-checker: add utility to examine what module is imported easily FUJIWARA Katsunori <foozy@lares.dti.ne.jp> parents: 25064 diff changeset	178 return False
43075 57875cf423c9 style: run a patched black on a subset of mercurial Augie Fackler <augie@google.com> parents: 42823 diff changeset	179
25173 7358b5d9991e import-checker: add utility to examine what module is imported easily FUJIWARA Katsunori <foozy@lares.dti.ne.jp> parents: 25064 diff changeset	180 return fromlocal
20036 e5d51fa51aba contrib: add an import checker Augie Fackler <raf@durin42.com> parents: diff changeset	181
43075 57875cf423c9 style: run a patched black on a subset of mercurial Augie Fackler <augie@google.com> parents: 42823 diff changeset	182
32542 a025ec43856c import-checker: guess names of C extension modules Yuya Nishihara <yuya@tcha.org> parents: 32541 diff changeset	183 def populateextmods(localmods):
a025ec43856c import-checker: guess names of C extension modules Yuya Nishihara <yuya@tcha.org> parents: 32541 diff changeset	184 """Populate C extension modules based on pure modules"""
a025ec43856c import-checker: guess names of C extension modules Yuya Nishihara <yuya@tcha.org> parents: 32541 diff changeset	185 newlocalmods = set(localmods)
a025ec43856c import-checker: guess names of C extension modules Yuya Nishihara <yuya@tcha.org> parents: 32541 diff changeset	186 for n in localmods:
a025ec43856c import-checker: guess names of C extension modules Yuya Nishihara <yuya@tcha.org> parents: 32541 diff changeset	187 if n.startswith('mercurial.pure.'):
43075 57875cf423c9 style: run a patched black on a subset of mercurial Augie Fackler <augie@google.com> parents: 42823 diff changeset	188 m = n[len('mercurial.pure.') :]
32542 a025ec43856c import-checker: guess names of C extension modules Yuya Nishihara <yuya@tcha.org> parents: 32541 diff changeset	189 newlocalmods.add('mercurial.cext.' + m)
a025ec43856c import-checker: guess names of C extension modules Yuya Nishihara <yuya@tcha.org> parents: 32541 diff changeset	190 newlocalmods.add('mercurial.cffi._' + m)
a025ec43856c import-checker: guess names of C extension modules Yuya Nishihara <yuya@tcha.org> parents: 32541 diff changeset	191 return newlocalmods
a025ec43856c import-checker: guess names of C extension modules Yuya Nishihara <yuya@tcha.org> parents: 32541 diff changeset	192
43075 57875cf423c9 style: run a patched black on a subset of mercurial Augie Fackler <augie@google.com> parents: 42823 diff changeset	193
20036 e5d51fa51aba contrib: add an import checker Augie Fackler <raf@durin42.com> parents: diff changeset	194 def list_stdlib_modules():
e5d51fa51aba contrib: add an import checker Augie Fackler <raf@durin42.com> parents: diff changeset	195 """List the modules present in the stdlib.
e5d51fa51aba contrib: add an import checker Augie Fackler <raf@durin42.com> parents: diff changeset	196
e5d51fa51aba contrib: add an import checker Augie Fackler <raf@durin42.com> parents: diff changeset	197 >>> mods = set(list_stdlib_modules())
51374 617af10994fb py3: fully port doctest to py3 Manuel Jacob <me@manueljacob.de> parents: 51373 diff changeset	198 >>> 'http' in mods
20036 e5d51fa51aba contrib: add an import checker Augie Fackler <raf@durin42.com> parents: diff changeset	199 True
e5d51fa51aba contrib: add an import checker Augie Fackler <raf@durin42.com> parents: diff changeset	200
e5d51fa51aba contrib: add an import checker Augie Fackler <raf@durin42.com> parents: diff changeset	201 os.path isn't really a module, so it's missing:
e5d51fa51aba contrib: add an import checker Augie Fackler <raf@durin42.com> parents: diff changeset	202
e5d51fa51aba contrib: add an import checker Augie Fackler <raf@durin42.com> parents: diff changeset	203 >>> 'os.path' in mods
e5d51fa51aba contrib: add an import checker Augie Fackler <raf@durin42.com> parents: diff changeset	204 False
e5d51fa51aba contrib: add an import checker Augie Fackler <raf@durin42.com> parents: diff changeset	205
e5d51fa51aba contrib: add an import checker Augie Fackler <raf@durin42.com> parents: diff changeset	206 sys requires special treatment, because it's baked into the
e5d51fa51aba contrib: add an import checker Augie Fackler <raf@durin42.com> parents: diff changeset	207 interpreter, but it should still appear:
e5d51fa51aba contrib: add an import checker Augie Fackler <raf@durin42.com> parents: diff changeset	208
e5d51fa51aba contrib: add an import checker Augie Fackler <raf@durin42.com> parents: diff changeset	209 >>> 'sys' in mods
e5d51fa51aba contrib: add an import checker Augie Fackler <raf@durin42.com> parents: diff changeset	210 True
e5d51fa51aba contrib: add an import checker Augie Fackler <raf@durin42.com> parents: diff changeset	211
e5d51fa51aba contrib: add an import checker Augie Fackler <raf@durin42.com> parents: diff changeset	212 >>> 'collections' in mods
e5d51fa51aba contrib: add an import checker Augie Fackler <raf@durin42.com> parents: diff changeset	213 True
e5d51fa51aba contrib: add an import checker Augie Fackler <raf@durin42.com> parents: diff changeset	214
51374 617af10994fb py3: fully port doctest to py3 Manuel Jacob <me@manueljacob.de> parents: 51373 diff changeset	215 >>> 'array' in mods
20036 e5d51fa51aba contrib: add an import checker Augie Fackler <raf@durin42.com> parents: diff changeset	216 True
29395 4c8026babe8c import-checker: ensure cffi is always a system module Augie Fackler <raf@durin42.com> parents: 29374 diff changeset	217
4c8026babe8c import-checker: ensure cffi is always a system module Augie Fackler <raf@durin42.com> parents: 29374 diff changeset	218 >>> 'cffi' in mods
4c8026babe8c import-checker: ensure cffi is always a system module Augie Fackler <raf@durin42.com> parents: 29374 diff changeset	219 True
20036 e5d51fa51aba contrib: add an import checker Augie Fackler <raf@durin42.com> parents: diff changeset	220 """
52669 e627cc25b6f3 pyupgrade: rewrite `yield` statements in a loop to `yield from` Matt Harbison <matt_harbison@yahoo.com> parents: 52472 diff changeset	221 yield from sys.builtin_module_names
20036 e5d51fa51aba contrib: add an import checker Augie Fackler <raf@durin42.com> parents: diff changeset	222 # These modules only exist on windows, but we should always
e5d51fa51aba contrib: add an import checker Augie Fackler <raf@durin42.com> parents: diff changeset	223 # consider them stdlib.
52669 e627cc25b6f3 pyupgrade: rewrite `yield` statements in a loop to `yield from` Matt Harbison <matt_harbison@yahoo.com> parents: 52472 diff changeset	224 yield from ['msvcrt', '_winreg']
33912 c856cb1c29be contrib: inform import checker that __builtin__ is a thing Augie Fackler <raf@durin42.com> parents: 33911 diff changeset	225 yield '__builtin__'
43075 57875cf423c9 style: run a patched black on a subset of mercurial Augie Fackler <augie@google.com> parents: 42823 diff changeset	226 yield 'builtins' # python3 only
57875cf423c9 style: run a patched black on a subset of mercurial Augie Fackler <augie@google.com> parents: 42823 diff changeset	227 yield 'importlib.abc' # python3 only
57875cf423c9 style: run a patched black on a subset of mercurial Augie Fackler <augie@google.com> parents: 42823 diff changeset	228 yield 'importlib.machinery' # python3 only
57875cf423c9 style: run a patched black on a subset of mercurial Augie Fackler <augie@google.com> parents: 42823 diff changeset	229 yield 'importlib.util' # python3 only
50275 ada9a0245fd7 run-tests: fix a crash when using the coverage options Matt Harbison <matt_harbison@yahoo.com> parents: 49369 diff changeset	230 yield 'packaging.version'
52669 e627cc25b6f3 pyupgrade: rewrite `yield` statements in a loop to `yield from` Matt Harbison <matt_harbison@yahoo.com> parents: 52472 diff changeset	231 yield from ['fcntl', 'grp', 'pwd', 'select', 'termios']
e627cc25b6f3 pyupgrade: rewrite `yield` statements in a loop to `yield from` Matt Harbison <matt_harbison@yahoo.com> parents: 52472 diff changeset	232 yield from ['cPickle', 'datetime']
e627cc25b6f3 pyupgrade: rewrite `yield` statements in a loop to `yield from` Matt Harbison <matt_harbison@yahoo.com> parents: 52472 diff changeset	233 yield from ['cffi']
51659 d9faa71a57f6 test-check: don't report distutils as a local import Pierre-Yves David <pierre-yves.david@octobus.net> parents: 51374 diff changeset	234 yield 'distutils' # in Python < 3.12
d9faa71a57f6 test-check: don't report distutils as a local import Pierre-Yves David <pierre-yves.david@octobus.net> parents: 51374 diff changeset	235 yield 'distutils.version' # in Python < 3.12
32331 bd872f64a8ba cleanup: use set literals Martin von Zweigbergk <martinvonz@google.com> parents: 32252 diff changeset	236 stdlib_prefixes = {sys.prefix, sys.exec_prefix}
20198 f5393a9dc4e5 import-checker: make test-module-imports.t work using virtualenv (issue4129) Chris Jerdonek <chris.jerdonek@gmail.com> parents: 20197 diff changeset	237 # We need to supplement the list of prefixes for the search to work
f5393a9dc4e5 import-checker: make test-module-imports.t work using virtualenv (issue4129) Chris Jerdonek <chris.jerdonek@gmail.com> parents: 20197 diff changeset	238 # when run from within a virtualenv.
51373 cf1bee12ecdb import-checker: make stdlib path detection work in virtual environments Manuel Jacob <me@manueljacob.de> parents: 50800 diff changeset	239 for mod in (argparse, zlib):
33895 998fad4b3072 contrib: work around some modules not existing on Py3 in import checker Augie Fackler <raf@durin42.com> parents: 32621 diff changeset	240 if mod is None:
998fad4b3072 contrib: work around some modules not existing on Py3 in import checker Augie Fackler <raf@durin42.com> parents: 32621 diff changeset	241 continue
20198 f5393a9dc4e5 import-checker: make test-module-imports.t work using virtualenv (issue4129) Chris Jerdonek <chris.jerdonek@gmail.com> parents: 20197 diff changeset	242 try:
f5393a9dc4e5 import-checker: make test-module-imports.t work using virtualenv (issue4129) Chris Jerdonek <chris.jerdonek@gmail.com> parents: 20197 diff changeset	243 # Not all module objects have a __file__ attribute.
f5393a9dc4e5 import-checker: make test-module-imports.t work using virtualenv (issue4129) Chris Jerdonek <chris.jerdonek@gmail.com> parents: 20197 diff changeset	244 filename = mod.__file__
f5393a9dc4e5 import-checker: make test-module-imports.t work using virtualenv (issue4129) Chris Jerdonek <chris.jerdonek@gmail.com> parents: 20197 diff changeset	245 except AttributeError:
f5393a9dc4e5 import-checker: make test-module-imports.t work using virtualenv (issue4129) Chris Jerdonek <chris.jerdonek@gmail.com> parents: 20197 diff changeset	246 continue
f5393a9dc4e5 import-checker: make test-module-imports.t work using virtualenv (issue4129) Chris Jerdonek <chris.jerdonek@gmail.com> parents: 20197 diff changeset	247 dirname = os.path.dirname(filename)
f5393a9dc4e5 import-checker: make test-module-imports.t work using virtualenv (issue4129) Chris Jerdonek <chris.jerdonek@gmail.com> parents: 20197 diff changeset	248 for prefix in stdlib_prefixes:
f5393a9dc4e5 import-checker: make test-module-imports.t work using virtualenv (issue4129) Chris Jerdonek <chris.jerdonek@gmail.com> parents: 20197 diff changeset	249 if dirname.startswith(prefix):
f5393a9dc4e5 import-checker: make test-module-imports.t work using virtualenv (issue4129) Chris Jerdonek <chris.jerdonek@gmail.com> parents: 20197 diff changeset	250 # Then this directory is redundant.
f5393a9dc4e5 import-checker: make test-module-imports.t work using virtualenv (issue4129) Chris Jerdonek <chris.jerdonek@gmail.com> parents: 20197 diff changeset	251 break
f5393a9dc4e5 import-checker: make test-module-imports.t work using virtualenv (issue4129) Chris Jerdonek <chris.jerdonek@gmail.com> parents: 20197 diff changeset	252 else:
f5393a9dc4e5 import-checker: make test-module-imports.t work using virtualenv (issue4129) Chris Jerdonek <chris.jerdonek@gmail.com> parents: 20197 diff changeset	253 stdlib_prefixes.add(dirname)
40726 dd028bca9221 tests: make test-check-module-imports more robust Valentin Gatien-Baron <vgatien-baron@janestreet.com> parents: 40531 diff changeset	254 sourceroot = os.path.abspath(os.path.dirname(os.path.dirname(__file__)))
20036 e5d51fa51aba contrib: add an import checker Augie Fackler <raf@durin42.com> parents: diff changeset	255 for libpath in sys.path:
40726 dd028bca9221 tests: make test-check-module-imports more robust Valentin Gatien-Baron <vgatien-baron@janestreet.com> parents: 40531 diff changeset	256 # We want to walk everything in sys.path that starts with something in
dd028bca9221 tests: make test-check-module-imports more robust Valentin Gatien-Baron <vgatien-baron@janestreet.com> parents: 40531 diff changeset	257 # stdlib_prefixes, but not directories from the hg sources.
43075 57875cf423c9 style: run a patched black on a subset of mercurial Augie Fackler <augie@google.com> parents: 42823 diff changeset	258 if os.path.abspath(libpath).startswith(sourceroot) or not any(
57875cf423c9 style: run a patched black on a subset of mercurial Augie Fackler <augie@google.com> parents: 42823 diff changeset	259 libpath.startswith(p) for p in stdlib_prefixes
57875cf423c9 style: run a patched black on a subset of mercurial Augie Fackler <augie@google.com> parents: 42823 diff changeset	260 ):
20036 e5d51fa51aba contrib: add an import checker Augie Fackler <raf@durin42.com> parents: diff changeset	261 continue
e5d51fa51aba contrib: add an import checker Augie Fackler <raf@durin42.com> parents: diff changeset	262 for top, dirs, files in os.walk(libpath):
47797 42e2cdb50db0 check-module-imports: ignore non-stdlib module installed by distribution Pierre-Yves David <pierre-yves.david@octobus.net> parents: 47383 diff changeset	263 if 'dist-packages' in top.split(os.path.sep):
42e2cdb50db0 check-module-imports: ignore non-stdlib module installed by distribution Pierre-Yves David <pierre-yves.david@octobus.net> parents: 47383 diff changeset	264 continue
25733 f99c066f5f9a import-checker: recurse into subtree of sys.path only if __init__.py exists Yuya Nishihara <yuya@tcha.org> parents: 25731 diff changeset	265 for i, d in reversed(list(enumerate(dirs))):
43075 57875cf423c9 style: run a patched black on a subset of mercurial Augie Fackler <augie@google.com> parents: 42823 diff changeset	266 if (
57875cf423c9 style: run a patched black on a subset of mercurial Augie Fackler <augie@google.com> parents: 42823 diff changeset	267 not os.path.exists(os.path.join(top, d, '__init__.py'))
57875cf423c9 style: run a patched black on a subset of mercurial Augie Fackler <augie@google.com> parents: 42823 diff changeset	268 or top == libpath
57875cf423c9 style: run a patched black on a subset of mercurial Augie Fackler <augie@google.com> parents: 42823 diff changeset	269 and d in ('hgdemandimport', 'hgext', 'mercurial')
57875cf423c9 style: run a patched black on a subset of mercurial Augie Fackler <augie@google.com> parents: 42823 diff changeset	270 ):
25733 f99c066f5f9a import-checker: recurse into subtree of sys.path only if __init__.py exists Yuya Nishihara <yuya@tcha.org> parents: 25731 diff changeset	271 del dirs[i]
20036 e5d51fa51aba contrib: add an import checker Augie Fackler <raf@durin42.com> parents: diff changeset	272 for name in files:
26221 ae65b1b4cb46 import-checker: use modern .endswith for multiple suffixes Augie Fackler <augie@google.com> parents: 26166 diff changeset	273 if not name.endswith(('.py', '.so', '.pyc', '.pyo', '.pyd')):
20036 e5d51fa51aba contrib: add an import checker Augie Fackler <raf@durin42.com> parents: diff changeset	274 continue
27621 39845b064041 import-checker: list package directory as stdlib module Yuya Nishihara <yuya@tcha.org> parents: 27620 diff changeset	275 if name.startswith('__init__.py'):
39845b064041 import-checker: list package directory as stdlib module Yuya Nishihara <yuya@tcha.org> parents: 27620 diff changeset	276 full_path = top
39845b064041 import-checker: list package directory as stdlib module Yuya Nishihara <yuya@tcha.org> parents: 27620 diff changeset	277 else:
39845b064041 import-checker: list package directory as stdlib module Yuya Nishihara <yuya@tcha.org> parents: 27620 diff changeset	278 full_path = os.path.join(top, name)
43075 57875cf423c9 style: run a patched black on a subset of mercurial Augie Fackler <augie@google.com> parents: 42823 diff changeset	279 rel_path = full_path[len(libpath) + 1 :]
20036 e5d51fa51aba contrib: add an import checker Augie Fackler <raf@durin42.com> parents: diff changeset	280 mod = dotted_name_of_path(rel_path)
e5d51fa51aba contrib: add an import checker Augie Fackler <raf@durin42.com> parents: diff changeset	281 yield mod
e5d51fa51aba contrib: add an import checker Augie Fackler <raf@durin42.com> parents: diff changeset	282
43075 57875cf423c9 style: run a patched black on a subset of mercurial Augie Fackler <augie@google.com> parents: 42823 diff changeset	283
20036 e5d51fa51aba contrib: add an import checker Augie Fackler <raf@durin42.com> parents: diff changeset	284 stdlib_modules = set(list_stdlib_modules())
e5d51fa51aba contrib: add an import checker Augie Fackler <raf@durin42.com> parents: diff changeset	285
43075 57875cf423c9 style: run a patched black on a subset of mercurial Augie Fackler <augie@google.com> parents: 42823 diff changeset	286
28921 02ee31a50002 import-checker: track filenames for SyntaxErrors timeless <timeless@mozdev.org> parents: 28920 diff changeset	287 def imported_modules(source, modulename, f, localmods, ignore_nested=False):
20036 e5d51fa51aba contrib: add an import checker Augie Fackler <raf@durin42.com> parents: diff changeset	288 """Given the source of a file as a string, yield the names
e5d51fa51aba contrib: add an import checker Augie Fackler <raf@durin42.com> parents: diff changeset	289 imported by that file.
e5d51fa51aba contrib: add an import checker Augie Fackler <raf@durin42.com> parents: diff changeset	290
20037 957b43371928 import-checker: ignore nested imports Augie Fackler <raf@durin42.com> parents: 20036 diff changeset	291 Args:
957b43371928 import-checker: ignore nested imports Augie Fackler <raf@durin42.com> parents: 20036 diff changeset	292 source: The python source to examine as a string.
25174 86298718b01c import-checker: make imported_modules yield absolute dotted_name_of_path FUJIWARA Katsunori <foozy@lares.dti.ne.jp> parents: 25173 diff changeset	293 modulename: of specified python source (may have `__init__`)
32541 4c712b90c60a import-checker: convert localmods to a set of module names Yuya Nishihara <yuya@tcha.org> parents: 32540 diff changeset	294 localmods: set of locally defined module names (may have `__init__`)
20037 957b43371928 import-checker: ignore nested imports Augie Fackler <raf@durin42.com> parents: 20036 diff changeset	295 ignore_nested: If true, import statements that do not start in
957b43371928 import-checker: ignore nested imports Augie Fackler <raf@durin42.com> parents: 20036 diff changeset	296 column zero will be ignored.
957b43371928 import-checker: ignore nested imports Augie Fackler <raf@durin42.com> parents: 20036 diff changeset	297
957b43371928 import-checker: ignore nested imports Augie Fackler <raf@durin42.com> parents: 20036 diff changeset	298 Returns:
25174 86298718b01c import-checker: make imported_modules yield absolute dotted_name_of_path FUJIWARA Katsunori <foozy@lares.dti.ne.jp> parents: 25173 diff changeset	299 A list of absolute module names imported by the given source.
20037 957b43371928 import-checker: ignore nested imports Augie Fackler <raf@durin42.com> parents: 20036 diff changeset	300
28921 02ee31a50002 import-checker: track filenames for SyntaxErrors timeless <timeless@mozdev.org> parents: 28920 diff changeset	301 >>> f = 'foo/xxx.py'
25174 86298718b01c import-checker: make imported_modules yield absolute dotted_name_of_path FUJIWARA Katsunori <foozy@lares.dti.ne.jp> parents: 25173 diff changeset	302 >>> modulename = 'foo.xxx'
86298718b01c import-checker: make imported_modules yield absolute dotted_name_of_path FUJIWARA Katsunori <foozy@lares.dti.ne.jp> parents: 25173 diff changeset	303 >>> localmods = {'foo.__init__': True,
86298718b01c import-checker: make imported_modules yield absolute dotted_name_of_path FUJIWARA Katsunori <foozy@lares.dti.ne.jp> parents: 25173 diff changeset	304 ... 'foo.foo1': True, 'foo.foo2': True,
86298718b01c import-checker: make imported_modules yield absolute dotted_name_of_path FUJIWARA Katsunori <foozy@lares.dti.ne.jp> parents: 25173 diff changeset	305 ... 'foo.bar.__init__': True, 'foo.bar.bar1': True,
86298718b01c import-checker: make imported_modules yield absolute dotted_name_of_path FUJIWARA Katsunori <foozy@lares.dti.ne.jp> parents: 25173 diff changeset	306 ... 'baz.__init__': True, 'baz.baz1': True }
86298718b01c import-checker: make imported_modules yield absolute dotted_name_of_path FUJIWARA Katsunori <foozy@lares.dti.ne.jp> parents: 25173 diff changeset	307 >>> # standard library (= not locally defined ones)
86298718b01c import-checker: make imported_modules yield absolute dotted_name_of_path FUJIWARA Katsunori <foozy@lares.dti.ne.jp> parents: 25173 diff changeset	308 >>> sorted(imported_modules(
86298718b01c import-checker: make imported_modules yield absolute dotted_name_of_path FUJIWARA Katsunori <foozy@lares.dti.ne.jp> parents: 25173 diff changeset	309 ... 'from stdlib1 import foo, bar; import stdlib2',
28921 02ee31a50002 import-checker: track filenames for SyntaxErrors timeless <timeless@mozdev.org> parents: 28920 diff changeset	310 ... modulename, f, localmods))
25174 86298718b01c import-checker: make imported_modules yield absolute dotted_name_of_path FUJIWARA Katsunori <foozy@lares.dti.ne.jp> parents: 25173 diff changeset	311 []
86298718b01c import-checker: make imported_modules yield absolute dotted_name_of_path FUJIWARA Katsunori <foozy@lares.dti.ne.jp> parents: 25173 diff changeset	312 >>> # relative importing
20037 957b43371928 import-checker: ignore nested imports Augie Fackler <raf@durin42.com> parents: 20036 diff changeset	313 >>> sorted(imported_modules(
25174 86298718b01c import-checker: make imported_modules yield absolute dotted_name_of_path FUJIWARA Katsunori <foozy@lares.dti.ne.jp> parents: 25173 diff changeset	314 ... 'import foo1; from bar import bar1',
28921 02ee31a50002 import-checker: track filenames for SyntaxErrors timeless <timeless@mozdev.org> parents: 28920 diff changeset	315 ... modulename, f, localmods))
26964 5abba2c92da3 import-checker: allow import of child modules from package root Yuya Nishihara <yuya@tcha.org> parents: 26956 diff changeset	316 ['foo.bar.bar1', 'foo.foo1']
25174 86298718b01c import-checker: make imported_modules yield absolute dotted_name_of_path FUJIWARA Katsunori <foozy@lares.dti.ne.jp> parents: 25173 diff changeset	317 >>> sorted(imported_modules(
86298718b01c import-checker: make imported_modules yield absolute dotted_name_of_path FUJIWARA Katsunori <foozy@lares.dti.ne.jp> parents: 25173 diff changeset	318 ... 'from bar.bar1 import name1, name2, name3',
28921 02ee31a50002 import-checker: track filenames for SyntaxErrors timeless <timeless@mozdev.org> parents: 28920 diff changeset	319 ... modulename, f, localmods))
25174 86298718b01c import-checker: make imported_modules yield absolute dotted_name_of_path FUJIWARA Katsunori <foozy@lares.dti.ne.jp> parents: 25173 diff changeset	320 ['foo.bar.bar1']
86298718b01c import-checker: make imported_modules yield absolute dotted_name_of_path FUJIWARA Katsunori <foozy@lares.dti.ne.jp> parents: 25173 diff changeset	321 >>> # absolute importing
86298718b01c import-checker: make imported_modules yield absolute dotted_name_of_path FUJIWARA Katsunori <foozy@lares.dti.ne.jp> parents: 25173 diff changeset	322 >>> sorted(imported_modules(
86298718b01c import-checker: make imported_modules yield absolute dotted_name_of_path FUJIWARA Katsunori <foozy@lares.dti.ne.jp> parents: 25173 diff changeset	323 ... 'from baz import baz1, name1',
28921 02ee31a50002 import-checker: track filenames for SyntaxErrors timeless <timeless@mozdev.org> parents: 28920 diff changeset	324 ... modulename, f, localmods))
25174 86298718b01c import-checker: make imported_modules yield absolute dotted_name_of_path FUJIWARA Katsunori <foozy@lares.dti.ne.jp> parents: 25173 diff changeset	325 ['baz.__init__', 'baz.baz1']
86298718b01c import-checker: make imported_modules yield absolute dotted_name_of_path FUJIWARA Katsunori <foozy@lares.dti.ne.jp> parents: 25173 diff changeset	326 >>> # mixed importing, even though it shouldn't be recommended
86298718b01c import-checker: make imported_modules yield absolute dotted_name_of_path FUJIWARA Katsunori <foozy@lares.dti.ne.jp> parents: 25173 diff changeset	327 >>> sorted(imported_modules(
86298718b01c import-checker: make imported_modules yield absolute dotted_name_of_path FUJIWARA Katsunori <foozy@lares.dti.ne.jp> parents: 25173 diff changeset	328 ... 'import stdlib, foo1, baz',
28921 02ee31a50002 import-checker: track filenames for SyntaxErrors timeless <timeless@mozdev.org> parents: 28920 diff changeset	329 ... modulename, f, localmods))
25174 86298718b01c import-checker: make imported_modules yield absolute dotted_name_of_path FUJIWARA Katsunori <foozy@lares.dti.ne.jp> parents: 25173 diff changeset	330 ['baz.__init__', 'foo.foo1']
86298718b01c import-checker: make imported_modules yield absolute dotted_name_of_path FUJIWARA Katsunori <foozy@lares.dti.ne.jp> parents: 25173 diff changeset	331 >>> # ignore_nested
20037 957b43371928 import-checker: ignore nested imports Augie Fackler <raf@durin42.com> parents: 20036 diff changeset	332 >>> sorted(imported_modules(
957b43371928 import-checker: ignore nested imports Augie Fackler <raf@durin42.com> parents: 20036 diff changeset	333 ... '''import foo
957b43371928 import-checker: ignore nested imports Augie Fackler <raf@durin42.com> parents: 20036 diff changeset	334 ... def wat():
957b43371928 import-checker: ignore nested imports Augie Fackler <raf@durin42.com> parents: 20036 diff changeset	335 ... import bar
28921 02ee31a50002 import-checker: track filenames for SyntaxErrors timeless <timeless@mozdev.org> parents: 28920 diff changeset	336 ... ''', modulename, f, localmods))
25174 86298718b01c import-checker: make imported_modules yield absolute dotted_name_of_path FUJIWARA Katsunori <foozy@lares.dti.ne.jp> parents: 25173 diff changeset	337 ['foo.__init__', 'foo.bar.__init__']
86298718b01c import-checker: make imported_modules yield absolute dotted_name_of_path FUJIWARA Katsunori <foozy@lares.dti.ne.jp> parents: 25173 diff changeset	338 >>> sorted(imported_modules(
86298718b01c import-checker: make imported_modules yield absolute dotted_name_of_path FUJIWARA Katsunori <foozy@lares.dti.ne.jp> parents: 25173 diff changeset	339 ... '''import foo
86298718b01c import-checker: make imported_modules yield absolute dotted_name_of_path FUJIWARA Katsunori <foozy@lares.dti.ne.jp> parents: 25173 diff changeset	340 ... def wat():
86298718b01c import-checker: make imported_modules yield absolute dotted_name_of_path FUJIWARA Katsunori <foozy@lares.dti.ne.jp> parents: 25173 diff changeset	341 ... import bar
28921 02ee31a50002 import-checker: track filenames for SyntaxErrors timeless <timeless@mozdev.org> parents: 28920 diff changeset	342 ... ''', modulename, f, localmods, ignore_nested=True))
25174 86298718b01c import-checker: make imported_modules yield absolute dotted_name_of_path FUJIWARA Katsunori <foozy@lares.dti.ne.jp> parents: 25173 diff changeset	343 ['foo.__init__']
20036 e5d51fa51aba contrib: add an import checker Augie Fackler <raf@durin42.com> parents: diff changeset	344 """
25174 86298718b01c import-checker: make imported_modules yield absolute dotted_name_of_path FUJIWARA Katsunori <foozy@lares.dti.ne.jp> parents: 25173 diff changeset	345 fromlocal = fromlocalfunc(modulename, localmods)
28921 02ee31a50002 import-checker: track filenames for SyntaxErrors timeless <timeless@mozdev.org> parents: 28920 diff changeset	346 for node in ast.walk(ast.parse(source, f)):
20037 957b43371928 import-checker: ignore nested imports Augie Fackler <raf@durin42.com> parents: 20036 diff changeset	347 if ignore_nested and getattr(node, 'col_offset', 0) > 0:
957b43371928 import-checker: ignore nested imports Augie Fackler <raf@durin42.com> parents: 20036 diff changeset	348 continue
20036 e5d51fa51aba contrib: add an import checker Augie Fackler <raf@durin42.com> parents: diff changeset	349 if isinstance(node, ast.Import):
e5d51fa51aba contrib: add an import checker Augie Fackler <raf@durin42.com> parents: diff changeset	350 for n in node.names:
25174 86298718b01c import-checker: make imported_modules yield absolute dotted_name_of_path FUJIWARA Katsunori <foozy@lares.dti.ne.jp> parents: 25173 diff changeset	351 found = fromlocal(n.name)
86298718b01c import-checker: make imported_modules yield absolute dotted_name_of_path FUJIWARA Katsunori <foozy@lares.dti.ne.jp> parents: 25173 diff changeset	352 if not found:
86298718b01c import-checker: make imported_modules yield absolute dotted_name_of_path FUJIWARA Katsunori <foozy@lares.dti.ne.jp> parents: 25173 diff changeset	353 # this should import standard library
86298718b01c import-checker: make imported_modules yield absolute dotted_name_of_path FUJIWARA Katsunori <foozy@lares.dti.ne.jp> parents: 25173 diff changeset	354 continue
86298718b01c import-checker: make imported_modules yield absolute dotted_name_of_path FUJIWARA Katsunori <foozy@lares.dti.ne.jp> parents: 25173 diff changeset	355 yield found[1]
20036 e5d51fa51aba contrib: add an import checker Augie Fackler <raf@durin42.com> parents: diff changeset	356 elif isinstance(node, ast.ImportFrom):
25701 1f88c0f6ff5a import-checker: resolve relative imports Gregory Szorc <gregory.szorc@gmail.com> parents: 25660 diff changeset	357 found = fromlocal(node.module, node.level)
25174 86298718b01c import-checker: make imported_modules yield absolute dotted_name_of_path FUJIWARA Katsunori <foozy@lares.dti.ne.jp> parents: 25173 diff changeset	358 if not found:
86298718b01c import-checker: make imported_modules yield absolute dotted_name_of_path FUJIWARA Katsunori <foozy@lares.dti.ne.jp> parents: 25173 diff changeset	359 # this should import standard library
86298718b01c import-checker: make imported_modules yield absolute dotted_name_of_path FUJIWARA Katsunori <foozy@lares.dti.ne.jp> parents: 25173 diff changeset	360 continue
86298718b01c import-checker: make imported_modules yield absolute dotted_name_of_path FUJIWARA Katsunori <foozy@lares.dti.ne.jp> parents: 25173 diff changeset	361
86298718b01c import-checker: make imported_modules yield absolute dotted_name_of_path FUJIWARA Katsunori <foozy@lares.dti.ne.jp> parents: 25173 diff changeset	362 absname, dottedpath, hassubmod = found
86298718b01c import-checker: make imported_modules yield absolute dotted_name_of_path FUJIWARA Katsunori <foozy@lares.dti.ne.jp> parents: 25173 diff changeset	363 if not hassubmod:
26964 5abba2c92da3 import-checker: allow import of child modules from package root Yuya Nishihara <yuya@tcha.org> parents: 26956 diff changeset	364 # "dottedpath" is not a package; must be imported
5abba2c92da3 import-checker: allow import of child modules from package root Yuya Nishihara <yuya@tcha.org> parents: 26956 diff changeset	365 yield dottedpath
25174 86298718b01c import-checker: make imported_modules yield absolute dotted_name_of_path FUJIWARA Katsunori <foozy@lares.dti.ne.jp> parents: 25173 diff changeset	366 # examination of "node.names" should be redundant
86298718b01c import-checker: make imported_modules yield absolute dotted_name_of_path FUJIWARA Katsunori <foozy@lares.dti.ne.jp> parents: 25173 diff changeset	367 # e.g.: from mercurial.node import nullid, nullrev
86298718b01c import-checker: make imported_modules yield absolute dotted_name_of_path FUJIWARA Katsunori <foozy@lares.dti.ne.jp> parents: 25173 diff changeset	368 continue
86298718b01c import-checker: make imported_modules yield absolute dotted_name_of_path FUJIWARA Katsunori <foozy@lares.dti.ne.jp> parents: 25173 diff changeset	369
26964 5abba2c92da3 import-checker: allow import of child modules from package root Yuya Nishihara <yuya@tcha.org> parents: 26956 diff changeset	370 modnotfound = False
25174 86298718b01c import-checker: make imported_modules yield absolute dotted_name_of_path FUJIWARA Katsunori <foozy@lares.dti.ne.jp> parents: 25173 diff changeset	371 prefix = absname + '.'
20036 e5d51fa51aba contrib: add an import checker Augie Fackler <raf@durin42.com> parents: diff changeset	372 for n in node.names:
25174 86298718b01c import-checker: make imported_modules yield absolute dotted_name_of_path FUJIWARA Katsunori <foozy@lares.dti.ne.jp> parents: 25173 diff changeset	373 found = fromlocal(prefix + n.name)
86298718b01c import-checker: make imported_modules yield absolute dotted_name_of_path FUJIWARA Katsunori <foozy@lares.dti.ne.jp> parents: 25173 diff changeset	374 if not found:
86298718b01c import-checker: make imported_modules yield absolute dotted_name_of_path FUJIWARA Katsunori <foozy@lares.dti.ne.jp> parents: 25173 diff changeset	375 # this should be a function or a property of "node.module"
26964 5abba2c92da3 import-checker: allow import of child modules from package root Yuya Nishihara <yuya@tcha.org> parents: 26956 diff changeset	376 modnotfound = True
25174 86298718b01c import-checker: make imported_modules yield absolute dotted_name_of_path FUJIWARA Katsunori <foozy@lares.dti.ne.jp> parents: 25173 diff changeset	377 continue
86298718b01c import-checker: make imported_modules yield absolute dotted_name_of_path FUJIWARA Katsunori <foozy@lares.dti.ne.jp> parents: 25173 diff changeset	378 yield found[1]
44440 4cabeea6d214 hgext: start building a library for simple hooks Joerg Sonnenberger <joerg@bec.de> parents: 43959 diff changeset	379 if modnotfound and dottedpath != modulename:
26964 5abba2c92da3 import-checker: allow import of child modules from package root Yuya Nishihara <yuya@tcha.org> parents: 26956 diff changeset	380 # "dottedpath" is a package, but imported because of non-module
5abba2c92da3 import-checker: allow import of child modules from package root Yuya Nishihara <yuya@tcha.org> parents: 26956 diff changeset	381 # lookup
44440 4cabeea6d214 hgext: start building a library for simple hooks Joerg Sonnenberger <joerg@bec.de> parents: 43959 diff changeset	382 # specifically allow "from . import foo" from __init__.py
26964 5abba2c92da3 import-checker: allow import of child modules from package root Yuya Nishihara <yuya@tcha.org> parents: 26956 diff changeset	383 yield dottedpath
20036 e5d51fa51aba contrib: add an import checker Augie Fackler <raf@durin42.com> parents: diff changeset	384
43075 57875cf423c9 style: run a patched black on a subset of mercurial Augie Fackler <augie@google.com> parents: 42823 diff changeset	385
27272 69308357ecd1 import-checker: allow absolute imports of sub modules from local packages Yuya Nishihara <yuya@tcha.org> parents: 27018 diff changeset	386 def verify_import_convention(module, source, localmods):
48959 a52f5bfc9358 import-checker: assume absolute and use modern import checker Gregory Szorc <gregory.szorc@gmail.com> parents: 48925 diff changeset	387 """Verify imports match our established coding convention."""
a52f5bfc9358 import-checker: assume absolute and use modern import checker Gregory Szorc <gregory.szorc@gmail.com> parents: 48925 diff changeset	388 root = ast.parse(source)
25703 1a6a117d0b95 import-checker: establish modern import convention Gregory Szorc <gregory.szorc@gmail.com> parents: 25702 diff changeset	389
48959 a52f5bfc9358 import-checker: assume absolute and use modern import checker Gregory Szorc <gregory.szorc@gmail.com> parents: 48925 diff changeset	390 return verify_modern_convention(module, root, localmods)
25703 1a6a117d0b95 import-checker: establish modern import convention Gregory Szorc <gregory.szorc@gmail.com> parents: 25702 diff changeset	391
43075 57875cf423c9 style: run a patched black on a subset of mercurial Augie Fackler <augie@google.com> parents: 42823 diff changeset	392
27272 69308357ecd1 import-checker: allow absolute imports of sub modules from local packages Yuya Nishihara <yuya@tcha.org> parents: 27018 diff changeset	393 def verify_modern_convention(module, root, localmods, root_col_offset=0):
25703 1a6a117d0b95 import-checker: establish modern import convention Gregory Szorc <gregory.szorc@gmail.com> parents: 25702 diff changeset	394 """Verify a file conforms to the modern import convention rules.
1a6a117d0b95 import-checker: establish modern import convention Gregory Szorc <gregory.szorc@gmail.com> parents: 25702 diff changeset	395
1a6a117d0b95 import-checker: establish modern import convention Gregory Szorc <gregory.szorc@gmail.com> parents: 25702 diff changeset	396 The rules of the modern convention are:
1a6a117d0b95 import-checker: establish modern import convention Gregory Szorc <gregory.szorc@gmail.com> parents: 25702 diff changeset	397
1a6a117d0b95 import-checker: establish modern import convention Gregory Szorc <gregory.szorc@gmail.com> parents: 25702 diff changeset	398 * Ordering is stdlib followed by local imports. Each group is lexically
1a6a117d0b95 import-checker: establish modern import convention Gregory Szorc <gregory.szorc@gmail.com> parents: 25702 diff changeset	399 sorted.
1a6a117d0b95 import-checker: establish modern import convention Gregory Szorc <gregory.szorc@gmail.com> parents: 25702 diff changeset	400 * Importing multiple modules via "import X, Y" is not allowed: use
1a6a117d0b95 import-checker: establish modern import convention Gregory Szorc <gregory.szorc@gmail.com> parents: 25702 diff changeset	401 separate import statements.
1a6a117d0b95 import-checker: establish modern import convention Gregory Szorc <gregory.szorc@gmail.com> parents: 25702 diff changeset	402 * Importing multiple modules via "from X import ..." is allowed if using
1a6a117d0b95 import-checker: establish modern import convention Gregory Szorc <gregory.szorc@gmail.com> parents: 25702 diff changeset	403 parenthesis and one entry per line.
1a6a117d0b95 import-checker: establish modern import convention Gregory Szorc <gregory.szorc@gmail.com> parents: 25702 diff changeset	404 * Only 1 relative import statement per import level ("from .", "from ..")
1a6a117d0b95 import-checker: establish modern import convention Gregory Szorc <gregory.szorc@gmail.com> parents: 25702 diff changeset	405 is allowed.
1a6a117d0b95 import-checker: establish modern import convention Gregory Szorc <gregory.szorc@gmail.com> parents: 25702 diff changeset	406 * Relative imports from higher levels must occur before lower levels. e.g.
1a6a117d0b95 import-checker: establish modern import convention Gregory Szorc <gregory.szorc@gmail.com> parents: 25702 diff changeset	407 "from .." must be before "from .".
1a6a117d0b95 import-checker: establish modern import convention Gregory Szorc <gregory.szorc@gmail.com> parents: 25702 diff changeset	408 * Imports from peer packages should use relative import (e.g. do not
1a6a117d0b95 import-checker: establish modern import convention Gregory Szorc <gregory.szorc@gmail.com> parents: 25702 diff changeset	409 "import mercurial.foo" from a "mercurial.*" module).
1a6a117d0b95 import-checker: establish modern import convention Gregory Szorc <gregory.szorc@gmail.com> parents: 25702 diff changeset	410 * Symbols can only be imported from specific modules (see
1a6a117d0b95 import-checker: establish modern import convention Gregory Szorc <gregory.szorc@gmail.com> parents: 25702 diff changeset	411 `allowsymbolimports`). For other modules, first import the module then
1a6a117d0b95 import-checker: establish modern import convention Gregory Szorc <gregory.szorc@gmail.com> parents: 25702 diff changeset	412 assign the symbol to a module-level variable. In addition, these imports
29208 cba8bc11ed10 import-checker: extend check of symbol-import order to all local modules Yuya Nishihara <yuya@tcha.org> parents: 29207 diff changeset	413 must be performed before other local imports. This rule only
25703 1a6a117d0b95 import-checker: establish modern import convention Gregory Szorc <gregory.szorc@gmail.com> parents: 25702 diff changeset	414 applies to import statements outside of any blocks.
34055 bc2535238de2 import-checker: allow relative import a module being checked Jun Wu <quark@fb.com> parents: 33938 diff changeset	415 * Relative imports from the standard library are not allowed, unless that
bc2535238de2 import-checker: allow relative import a module being checked Jun Wu <quark@fb.com> parents: 33938 diff changeset	416 library is also a local module.
25703 1a6a117d0b95 import-checker: establish modern import convention Gregory Szorc <gregory.szorc@gmail.com> parents: 25702 diff changeset	417 * Certain modules must be aliased to alternate names to avoid aliasing
1a6a117d0b95 import-checker: establish modern import convention Gregory Szorc <gregory.szorc@gmail.com> parents: 25702 diff changeset	418 and readability problems. See `requirealias`.
1a6a117d0b95 import-checker: establish modern import convention Gregory Szorc <gregory.szorc@gmail.com> parents: 25702 diff changeset	419 """
33908 3de9a2df6411 contrib: have import-checker work mostly with native strings for mod names Augie Fackler <raf@durin42.com> parents: 33896 diff changeset	420 if not isinstance(module, str):
3de9a2df6411 contrib: have import-checker work mostly with native strings for mod names Augie Fackler <raf@durin42.com> parents: 33896 diff changeset	421 module = module.decode('ascii')
25703 1a6a117d0b95 import-checker: establish modern import convention Gregory Szorc <gregory.szorc@gmail.com> parents: 25702 diff changeset	422 topmodule = module.split('.')[0]
27272 69308357ecd1 import-checker: allow absolute imports of sub modules from local packages Yuya Nishihara <yuya@tcha.org> parents: 27018 diff changeset	423 fromlocal = fromlocalfunc(module, localmods)
25703 1a6a117d0b95 import-checker: establish modern import convention Gregory Szorc <gregory.szorc@gmail.com> parents: 25702 diff changeset	424
1a6a117d0b95 import-checker: establish modern import convention Gregory Szorc <gregory.szorc@gmail.com> parents: 25702 diff changeset	425 # Whether a local/non-stdlib import has been performed.
28330 f3fb24e36d61 import-checker: report local with stdlib late warning timeless <timeless@mozdev.org> parents: 27621 diff changeset	426 seenlocal = None
29208 cba8bc11ed10 import-checker: extend check of symbol-import order to all local modules Yuya Nishihara <yuya@tcha.org> parents: 29207 diff changeset	427 # Whether a local/non-stdlib, non-symbol import has been seen.
cba8bc11ed10 import-checker: extend check of symbol-import order to all local modules Yuya Nishihara <yuya@tcha.org> parents: 29207 diff changeset	428 seennonsymbollocal = False
25703 1a6a117d0b95 import-checker: establish modern import convention Gregory Szorc <gregory.szorc@gmail.com> parents: 25702 diff changeset	429 # The last name to be imported (for sorting).
1a6a117d0b95 import-checker: establish modern import convention Gregory Szorc <gregory.szorc@gmail.com> parents: 25702 diff changeset	430 lastname = None
30595 74eecb93c617 import-checker: do not enforce lexical sort accross stdlib/local boundary Pierre-Yves David <pierre-yves.david@ens-lyon.org> parents: 29395 diff changeset	431 laststdlib = None
25703 1a6a117d0b95 import-checker: establish modern import convention Gregory Szorc <gregory.szorc@gmail.com> parents: 25702 diff changeset	432 # Relative import levels encountered so far.
1a6a117d0b95 import-checker: establish modern import convention Gregory Szorc <gregory.szorc@gmail.com> parents: 25702 diff changeset	433 seenlevels = set()
1a6a117d0b95 import-checker: establish modern import convention Gregory Szorc <gregory.szorc@gmail.com> parents: 25702 diff changeset	434
26965 1fa66d3ad28d import-checker: reset context to verify convention in function scope Yuya Nishihara <yuya@tcha.org> parents: 26964 diff changeset	435 for node, newscope in walklocal(root):
43075 57875cf423c9 style: run a patched black on a subset of mercurial Augie Fackler <augie@google.com> parents: 42823 diff changeset	436
26955 c4114e335b49 import-checker: extract function to generate a formatted warning Yuya Nishihara <yuya@tcha.org> parents: 26954 diff changeset	437 def msg(fmt, *args):
26956 4b56214ebb7a import-checker: include lineno in warning message Yuya Nishihara <yuya@tcha.org> parents: 26955 diff changeset	438 return (fmt % args, node.lineno)
43075 57875cf423c9 style: run a patched black on a subset of mercurial Augie Fackler <augie@google.com> parents: 42823 diff changeset	439
26965 1fa66d3ad28d import-checker: reset context to verify convention in function scope Yuya Nishihara <yuya@tcha.org> parents: 26964 diff changeset	440 if newscope:
1fa66d3ad28d import-checker: reset context to verify convention in function scope Yuya Nishihara <yuya@tcha.org> parents: 26964 diff changeset	441 # Check for local imports in function
52669 e627cc25b6f3 pyupgrade: rewrite `yield` statements in a loop to `yield from` Matt Harbison <matt_harbison@yahoo.com> parents: 52472 diff changeset	442 yield from verify_modern_convention(
43075 57875cf423c9 style: run a patched black on a subset of mercurial Augie Fackler <augie@google.com> parents: 42823 diff changeset	443 module, node, localmods, node.col_offset + 4
52669 e627cc25b6f3 pyupgrade: rewrite `yield` statements in a loop to `yield from` Matt Harbison <matt_harbison@yahoo.com> parents: 52472 diff changeset	444 )
26965 1fa66d3ad28d import-checker: reset context to verify convention in function scope Yuya Nishihara <yuya@tcha.org> parents: 26964 diff changeset	445 elif isinstance(node, ast.Import):
25703 1a6a117d0b95 import-checker: establish modern import convention Gregory Szorc <gregory.szorc@gmail.com> parents: 25702 diff changeset	446 # Disallow "import foo, bar" and require separate imports
1a6a117d0b95 import-checker: establish modern import convention Gregory Szorc <gregory.szorc@gmail.com> parents: 25702 diff changeset	447 # for each module.
1a6a117d0b95 import-checker: establish modern import convention Gregory Szorc <gregory.szorc@gmail.com> parents: 25702 diff changeset	448 if len(node.names) > 1:
43075 57875cf423c9 style: run a patched black on a subset of mercurial Augie Fackler <augie@google.com> parents: 42823 diff changeset	449 yield msg(
57875cf423c9 style: run a patched black on a subset of mercurial Augie Fackler <augie@google.com> parents: 42823 diff changeset	450 'multiple imported names: %s',
57875cf423c9 style: run a patched black on a subset of mercurial Augie Fackler <augie@google.com> parents: 42823 diff changeset	451 ', '.join(n.name for n in node.names),
57875cf423c9 style: run a patched black on a subset of mercurial Augie Fackler <augie@google.com> parents: 42823 diff changeset	452 )
25703 1a6a117d0b95 import-checker: establish modern import convention Gregory Szorc <gregory.szorc@gmail.com> parents: 25702 diff changeset	453
1a6a117d0b95 import-checker: establish modern import convention Gregory Szorc <gregory.szorc@gmail.com> parents: 25702 diff changeset	454 name = node.names[0].name
1a6a117d0b95 import-checker: establish modern import convention Gregory Szorc <gregory.szorc@gmail.com> parents: 25702 diff changeset	455 asname = node.names[0].asname
1a6a117d0b95 import-checker: establish modern import convention Gregory Szorc <gregory.szorc@gmail.com> parents: 25702 diff changeset	456
30595 74eecb93c617 import-checker: do not enforce lexical sort accross stdlib/local boundary Pierre-Yves David <pierre-yves.david@ens-lyon.org> parents: 29395 diff changeset	457 stdlib = name in stdlib_modules
74eecb93c617 import-checker: do not enforce lexical sort accross stdlib/local boundary Pierre-Yves David <pierre-yves.david@ens-lyon.org> parents: 29395 diff changeset	458
25703 1a6a117d0b95 import-checker: establish modern import convention Gregory Szorc <gregory.szorc@gmail.com> parents: 25702 diff changeset	459 # Ignore sorting rules on imports inside blocks.
26965 1fa66d3ad28d import-checker: reset context to verify convention in function scope Yuya Nishihara <yuya@tcha.org> parents: 26964 diff changeset	460 if node.col_offset == root_col_offset:
30595 74eecb93c617 import-checker: do not enforce lexical sort accross stdlib/local boundary Pierre-Yves David <pierre-yves.david@ens-lyon.org> parents: 29395 diff changeset	461 if lastname and name < lastname and laststdlib == stdlib:
43075 57875cf423c9 style: run a patched black on a subset of mercurial Augie Fackler <augie@google.com> parents: 42823 diff changeset	462 yield msg(
57875cf423c9 style: run a patched black on a subset of mercurial Augie Fackler <augie@google.com> parents: 42823 diff changeset	463 'imports not lexically sorted: %s < %s', name, lastname
57875cf423c9 style: run a patched black on a subset of mercurial Augie Fackler <augie@google.com> parents: 42823 diff changeset	464 )
25703 1a6a117d0b95 import-checker: establish modern import convention Gregory Szorc <gregory.szorc@gmail.com> parents: 25702 diff changeset	465
30595 74eecb93c617 import-checker: do not enforce lexical sort accross stdlib/local boundary Pierre-Yves David <pierre-yves.david@ens-lyon.org> parents: 29395 diff changeset	466 lastname = name
74eecb93c617 import-checker: do not enforce lexical sort accross stdlib/local boundary Pierre-Yves David <pierre-yves.david@ens-lyon.org> parents: 29395 diff changeset	467 laststdlib = stdlib
25703 1a6a117d0b95 import-checker: establish modern import convention Gregory Szorc <gregory.szorc@gmail.com> parents: 25702 diff changeset	468
1a6a117d0b95 import-checker: establish modern import convention Gregory Szorc <gregory.szorc@gmail.com> parents: 25702 diff changeset	469 # stdlib imports should be before local imports.
26965 1fa66d3ad28d import-checker: reset context to verify convention in function scope Yuya Nishihara <yuya@tcha.org> parents: 26964 diff changeset	470 if stdlib and seenlocal and node.col_offset == root_col_offset:
43075 57875cf423c9 style: run a patched black on a subset of mercurial Augie Fackler <augie@google.com> parents: 42823 diff changeset	471 yield msg(
57875cf423c9 style: run a patched black on a subset of mercurial Augie Fackler <augie@google.com> parents: 42823 diff changeset	472 'stdlib import "%s" follows local import: %s',
57875cf423c9 style: run a patched black on a subset of mercurial Augie Fackler <augie@google.com> parents: 42823 diff changeset	473 name,
57875cf423c9 style: run a patched black on a subset of mercurial Augie Fackler <augie@google.com> parents: 42823 diff changeset	474 seenlocal,
57875cf423c9 style: run a patched black on a subset of mercurial Augie Fackler <augie@google.com> parents: 42823 diff changeset	475 )
25703 1a6a117d0b95 import-checker: establish modern import convention Gregory Szorc <gregory.szorc@gmail.com> parents: 25702 diff changeset	476
1a6a117d0b95 import-checker: establish modern import convention Gregory Szorc <gregory.szorc@gmail.com> parents: 25702 diff changeset	477 if not stdlib:
28330 f3fb24e36d61 import-checker: report local with stdlib late warning timeless <timeless@mozdev.org> parents: 27621 diff changeset	478 seenlocal = name
25703 1a6a117d0b95 import-checker: establish modern import convention Gregory Szorc <gregory.szorc@gmail.com> parents: 25702 diff changeset	479
1a6a117d0b95 import-checker: establish modern import convention Gregory Szorc <gregory.szorc@gmail.com> parents: 25702 diff changeset	480 # Import of sibling modules should use relative imports.
1a6a117d0b95 import-checker: establish modern import convention Gregory Szorc <gregory.szorc@gmail.com> parents: 25702 diff changeset	481 topname = name.split('.')[0]
1a6a117d0b95 import-checker: establish modern import convention Gregory Szorc <gregory.szorc@gmail.com> parents: 25702 diff changeset	482 if topname == topmodule:
26955 c4114e335b49 import-checker: extract function to generate a formatted warning Yuya Nishihara <yuya@tcha.org> parents: 26954 diff changeset	483 yield msg('import should be relative: %s', name)
25703 1a6a117d0b95 import-checker: establish modern import convention Gregory Szorc <gregory.szorc@gmail.com> parents: 25702 diff changeset	484
1a6a117d0b95 import-checker: establish modern import convention Gregory Szorc <gregory.szorc@gmail.com> parents: 25702 diff changeset	485 if name in requirealias and asname != requirealias[name]:
43075 57875cf423c9 style: run a patched black on a subset of mercurial Augie Fackler <augie@google.com> parents: 42823 diff changeset	486 yield msg(
57875cf423c9 style: run a patched black on a subset of mercurial Augie Fackler <augie@google.com> parents: 42823 diff changeset	487 '%s module must be "as" aliased to %s',
57875cf423c9 style: run a patched black on a subset of mercurial Augie Fackler <augie@google.com> parents: 42823 diff changeset	488 name,
57875cf423c9 style: run a patched black on a subset of mercurial Augie Fackler <augie@google.com> parents: 42823 diff changeset	489 requirealias[name],
57875cf423c9 style: run a patched black on a subset of mercurial Augie Fackler <augie@google.com> parents: 42823 diff changeset	490 )
25703 1a6a117d0b95 import-checker: establish modern import convention Gregory Szorc <gregory.szorc@gmail.com> parents: 25702 diff changeset	491
1a6a117d0b95 import-checker: establish modern import convention Gregory Szorc <gregory.szorc@gmail.com> parents: 25702 diff changeset	492 elif isinstance(node, ast.ImportFrom):
1a6a117d0b95 import-checker: establish modern import convention Gregory Szorc <gregory.szorc@gmail.com> parents: 25702 diff changeset	493 # Resolve the full imported module name.
1a6a117d0b95 import-checker: establish modern import convention Gregory Szorc <gregory.szorc@gmail.com> parents: 25702 diff changeset	494 if node.level > 0:
43075 57875cf423c9 style: run a patched black on a subset of mercurial Augie Fackler <augie@google.com> parents: 42823 diff changeset	495 fullname = '.'.join(module.split('.')[: -node.level])
25703 1a6a117d0b95 import-checker: establish modern import convention Gregory Szorc <gregory.szorc@gmail.com> parents: 25702 diff changeset	496 if node.module:
1a6a117d0b95 import-checker: establish modern import convention Gregory Szorc <gregory.szorc@gmail.com> parents: 25702 diff changeset	497 fullname += '.%s' % node.module
1a6a117d0b95 import-checker: establish modern import convention Gregory Szorc <gregory.szorc@gmail.com> parents: 25702 diff changeset	498 else:
1a6a117d0b95 import-checker: establish modern import convention Gregory Szorc <gregory.szorc@gmail.com> parents: 25702 diff changeset	499 assert node.module
1a6a117d0b95 import-checker: establish modern import convention Gregory Szorc <gregory.szorc@gmail.com> parents: 25702 diff changeset	500 fullname = node.module
1a6a117d0b95 import-checker: establish modern import convention Gregory Szorc <gregory.szorc@gmail.com> parents: 25702 diff changeset	501
1a6a117d0b95 import-checker: establish modern import convention Gregory Szorc <gregory.szorc@gmail.com> parents: 25702 diff changeset	502 topname = fullname.split('.')[0]
1a6a117d0b95 import-checker: establish modern import convention Gregory Szorc <gregory.szorc@gmail.com> parents: 25702 diff changeset	503 if topname == topmodule:
26955 c4114e335b49 import-checker: extract function to generate a formatted warning Yuya Nishihara <yuya@tcha.org> parents: 26954 diff changeset	504 yield msg('import should be relative: %s', fullname)
25703 1a6a117d0b95 import-checker: establish modern import convention Gregory Szorc <gregory.szorc@gmail.com> parents: 25702 diff changeset	505
1a6a117d0b95 import-checker: establish modern import convention Gregory Szorc <gregory.szorc@gmail.com> parents: 25702 diff changeset	506 # __future__ is special since it needs to come first and use
1a6a117d0b95 import-checker: establish modern import convention Gregory Szorc <gregory.szorc@gmail.com> parents: 25702 diff changeset	507 # symbol import.
1a6a117d0b95 import-checker: establish modern import convention Gregory Szorc <gregory.szorc@gmail.com> parents: 25702 diff changeset	508 if fullname != '__future__':
34055 bc2535238de2 import-checker: allow relative import a module being checked Jun Wu <quark@fb.com> parents: 33938 diff changeset	509 if not fullname or (
bc2535238de2 import-checker: allow relative import a module being checked Jun Wu <quark@fb.com> parents: 33938 diff changeset	510 fullname in stdlib_modules
43641 0ad5d6c4bfad import-checker: allow 'from typing import ...' Yuya Nishihara <yuya@tcha.org> parents: 43415 diff changeset	511 # allow standard 'from typing import ...' style
43959 303576116ac1 import-checker: allow all absolute imports of stdlib modules Martin von Zweigbergk <martinvonz@google.com> parents: 43641 diff changeset	512 and fullname.startswith('.')
34055 bc2535238de2 import-checker: allow relative import a module being checked Jun Wu <quark@fb.com> parents: 33938 diff changeset	513 and fullname not in localmods
43075 57875cf423c9 style: run a patched black on a subset of mercurial Augie Fackler <augie@google.com> parents: 42823 diff changeset	514 and fullname + '.__init__' not in localmods
57875cf423c9 style: run a patched black on a subset of mercurial Augie Fackler <augie@google.com> parents: 42823 diff changeset	515 ):
26955 c4114e335b49 import-checker: extract function to generate a formatted warning Yuya Nishihara <yuya@tcha.org> parents: 26954 diff changeset	516 yield msg('relative import of stdlib module')
25703 1a6a117d0b95 import-checker: establish modern import convention Gregory Szorc <gregory.szorc@gmail.com> parents: 25702 diff changeset	517 else:
28330 f3fb24e36d61 import-checker: report local with stdlib late warning timeless <timeless@mozdev.org> parents: 27621 diff changeset	518 seenlocal = fullname
25703 1a6a117d0b95 import-checker: establish modern import convention Gregory Szorc <gregory.szorc@gmail.com> parents: 25702 diff changeset	519
1a6a117d0b95 import-checker: establish modern import convention Gregory Szorc <gregory.szorc@gmail.com> parents: 25702 diff changeset	520 # Direct symbol import is only allowed from certain modules and
1a6a117d0b95 import-checker: establish modern import convention Gregory Szorc <gregory.szorc@gmail.com> parents: 25702 diff changeset	521 # must occur before non-symbol imports.
29207 a09098c61fea import-checker: always build a list of imported symbols Yuya Nishihara <yuya@tcha.org> parents: 29122 diff changeset	522 found = fromlocal(node.module, node.level)
a09098c61fea import-checker: always build a list of imported symbols Yuya Nishihara <yuya@tcha.org> parents: 29122 diff changeset	523 if found and found[2]: # node.module is a package
a09098c61fea import-checker: always build a list of imported symbols Yuya Nishihara <yuya@tcha.org> parents: 29122 diff changeset	524 prefix = found[0] + '.'
43075 57875cf423c9 style: run a patched black on a subset of mercurial Augie Fackler <augie@google.com> parents: 42823 diff changeset	525 symbols = (
57875cf423c9 style: run a patched black on a subset of mercurial Augie Fackler <augie@google.com> parents: 42823 diff changeset	526 n.name for n in node.names if not fromlocal(prefix + n.name)
57875cf423c9 style: run a patched black on a subset of mercurial Augie Fackler <augie@google.com> parents: 42823 diff changeset	527 )
29207 a09098c61fea import-checker: always build a list of imported symbols Yuya Nishihara <yuya@tcha.org> parents: 29122 diff changeset	528 else:
32457 d02888308235 import-checker: add a way to directly import certain symbols Siddharth Agarwal <sid0@fb.com> parents: 32413 diff changeset	529 symbols = (n.name for n in node.names)
d02888308235 import-checker: add a way to directly import certain symbols Siddharth Agarwal <sid0@fb.com> parents: 32413 diff changeset	530 symbols = [sym for sym in symbols if sym not in directsymbols]
26965 1fa66d3ad28d import-checker: reset context to verify convention in function scope Yuya Nishihara <yuya@tcha.org> parents: 26964 diff changeset	531 if node.module and node.col_offset == root_col_offset:
27272 69308357ecd1 import-checker: allow absolute imports of sub modules from local packages Yuya Nishihara <yuya@tcha.org> parents: 27018 diff changeset	532 if symbols and fullname not in allowsymbolimports:
43075 57875cf423c9 style: run a patched black on a subset of mercurial Augie Fackler <augie@google.com> parents: 42823 diff changeset	533 yield msg(
57875cf423c9 style: run a patched black on a subset of mercurial Augie Fackler <augie@google.com> parents: 42823 diff changeset	534 'direct symbol import %s from %s',
57875cf423c9 style: run a patched black on a subset of mercurial Augie Fackler <augie@google.com> parents: 42823 diff changeset	535 ', '.join(symbols),
57875cf423c9 style: run a patched black on a subset of mercurial Augie Fackler <augie@google.com> parents: 42823 diff changeset	536 fullname,
57875cf423c9 style: run a patched black on a subset of mercurial Augie Fackler <augie@google.com> parents: 42823 diff changeset	537 )
25703 1a6a117d0b95 import-checker: establish modern import convention Gregory Szorc <gregory.szorc@gmail.com> parents: 25702 diff changeset	538
29208 cba8bc11ed10 import-checker: extend check of symbol-import order to all local modules Yuya Nishihara <yuya@tcha.org> parents: 29207 diff changeset	539 if symbols and seennonsymbollocal:
43075 57875cf423c9 style: run a patched black on a subset of mercurial Augie Fackler <augie@google.com> parents: 42823 diff changeset	540 yield msg(
57875cf423c9 style: run a patched black on a subset of mercurial Augie Fackler <augie@google.com> parents: 42823 diff changeset	541 'symbol import follows non-symbol import: %s', fullname
57875cf423c9 style: run a patched black on a subset of mercurial Augie Fackler <augie@google.com> parents: 42823 diff changeset	542 )
29208 cba8bc11ed10 import-checker: extend check of symbol-import order to all local modules Yuya Nishihara <yuya@tcha.org> parents: 29207 diff changeset	543 if not symbols and fullname not in stdlib_modules:
cba8bc11ed10 import-checker: extend check of symbol-import order to all local modules Yuya Nishihara <yuya@tcha.org> parents: 29207 diff changeset	544 seennonsymbollocal = True
25703 1a6a117d0b95 import-checker: establish modern import convention Gregory Szorc <gregory.szorc@gmail.com> parents: 25702 diff changeset	545
1a6a117d0b95 import-checker: establish modern import convention Gregory Szorc <gregory.szorc@gmail.com> parents: 25702 diff changeset	546 if not node.module:
1a6a117d0b95 import-checker: establish modern import convention Gregory Szorc <gregory.szorc@gmail.com> parents: 25702 diff changeset	547 assert node.level
1a6a117d0b95 import-checker: establish modern import convention Gregory Szorc <gregory.szorc@gmail.com> parents: 25702 diff changeset	548
1a6a117d0b95 import-checker: establish modern import convention Gregory Szorc <gregory.szorc@gmail.com> parents: 25702 diff changeset	549 # Only allow 1 group per level.
43075 57875cf423c9 style: run a patched black on a subset of mercurial Augie Fackler <augie@google.com> parents: 42823 diff changeset	550 if (
57875cf423c9 style: run a patched black on a subset of mercurial Augie Fackler <augie@google.com> parents: 42823 diff changeset	551 node.level in seenlevels
57875cf423c9 style: run a patched black on a subset of mercurial Augie Fackler <augie@google.com> parents: 42823 diff changeset	552 and node.col_offset == root_col_offset
57875cf423c9 style: run a patched black on a subset of mercurial Augie Fackler <augie@google.com> parents: 42823 diff changeset	553 ):
57875cf423c9 style: run a patched black on a subset of mercurial Augie Fackler <augie@google.com> parents: 42823 diff changeset	554 yield msg(
57875cf423c9 style: run a patched black on a subset of mercurial Augie Fackler <augie@google.com> parents: 42823 diff changeset	555 'multiple "from %s import" statements', '.' * node.level
57875cf423c9 style: run a patched black on a subset of mercurial Augie Fackler <augie@google.com> parents: 42823 diff changeset	556 )
25703 1a6a117d0b95 import-checker: establish modern import convention Gregory Szorc <gregory.szorc@gmail.com> parents: 25702 diff changeset	557
1a6a117d0b95 import-checker: establish modern import convention Gregory Szorc <gregory.szorc@gmail.com> parents: 25702 diff changeset	558 # Higher-level groups come before lower-level groups.
1a6a117d0b95 import-checker: establish modern import convention Gregory Szorc <gregory.szorc@gmail.com> parents: 25702 diff changeset	559 if any(node.level > l for l in seenlevels):
43075 57875cf423c9 style: run a patched black on a subset of mercurial Augie Fackler <augie@google.com> parents: 42823 diff changeset	560 yield msg(
57875cf423c9 style: run a patched black on a subset of mercurial Augie Fackler <augie@google.com> parents: 42823 diff changeset	561 'higher-level import should come first: %s', fullname
57875cf423c9 style: run a patched black on a subset of mercurial Augie Fackler <augie@google.com> parents: 42823 diff changeset	562 )
25703 1a6a117d0b95 import-checker: establish modern import convention Gregory Szorc <gregory.szorc@gmail.com> parents: 25702 diff changeset	563
1a6a117d0b95 import-checker: establish modern import convention Gregory Szorc <gregory.szorc@gmail.com> parents: 25702 diff changeset	564 seenlevels.add(node.level)
1a6a117d0b95 import-checker: establish modern import convention Gregory Szorc <gregory.szorc@gmail.com> parents: 25702 diff changeset	565
1a6a117d0b95 import-checker: establish modern import convention Gregory Szorc <gregory.szorc@gmail.com> parents: 25702 diff changeset	566 # Entries in "from .X import ( ... )" lists must be lexically
1a6a117d0b95 import-checker: establish modern import convention Gregory Szorc <gregory.szorc@gmail.com> parents: 25702 diff changeset	567 # sorted.
1a6a117d0b95 import-checker: establish modern import convention Gregory Szorc <gregory.szorc@gmail.com> parents: 25702 diff changeset	568 lastentryname = None
1a6a117d0b95 import-checker: establish modern import convention Gregory Szorc <gregory.szorc@gmail.com> parents: 25702 diff changeset	569
1a6a117d0b95 import-checker: establish modern import convention Gregory Szorc <gregory.szorc@gmail.com> parents: 25702 diff changeset	570 for n in node.names:
1a6a117d0b95 import-checker: establish modern import convention Gregory Szorc <gregory.szorc@gmail.com> parents: 25702 diff changeset	571 if lastentryname and n.name < lastentryname:
43075 57875cf423c9 style: run a patched black on a subset of mercurial Augie Fackler <augie@google.com> parents: 42823 diff changeset	572 yield msg(
57875cf423c9 style: run a patched black on a subset of mercurial Augie Fackler <augie@google.com> parents: 42823 diff changeset	573 'imports from %s not lexically sorted: %s < %s',
57875cf423c9 style: run a patched black on a subset of mercurial Augie Fackler <augie@google.com> parents: 42823 diff changeset	574 fullname,
57875cf423c9 style: run a patched black on a subset of mercurial Augie Fackler <augie@google.com> parents: 42823 diff changeset	575 n.name,
57875cf423c9 style: run a patched black on a subset of mercurial Augie Fackler <augie@google.com> parents: 42823 diff changeset	576 lastentryname,
57875cf423c9 style: run a patched black on a subset of mercurial Augie Fackler <augie@google.com> parents: 42823 diff changeset	577 )
25703 1a6a117d0b95 import-checker: establish modern import convention Gregory Szorc <gregory.szorc@gmail.com> parents: 25702 diff changeset	578
1a6a117d0b95 import-checker: establish modern import convention Gregory Szorc <gregory.szorc@gmail.com> parents: 25702 diff changeset	579 lastentryname = n.name
1a6a117d0b95 import-checker: establish modern import convention Gregory Szorc <gregory.szorc@gmail.com> parents: 25702 diff changeset	580
1a6a117d0b95 import-checker: establish modern import convention Gregory Szorc <gregory.szorc@gmail.com> parents: 25702 diff changeset	581 if n.name in requirealias and n.asname != requirealias[n.name]:
43075 57875cf423c9 style: run a patched black on a subset of mercurial Augie Fackler <augie@google.com> parents: 42823 diff changeset	582 yield msg(
57875cf423c9 style: run a patched black on a subset of mercurial Augie Fackler <augie@google.com> parents: 42823 diff changeset	583 '%s from %s must be "as" aliased to %s',
57875cf423c9 style: run a patched black on a subset of mercurial Augie Fackler <augie@google.com> parents: 42823 diff changeset	584 n.name,
57875cf423c9 style: run a patched black on a subset of mercurial Augie Fackler <augie@google.com> parents: 42823 diff changeset	585 fullname,
57875cf423c9 style: run a patched black on a subset of mercurial Augie Fackler <augie@google.com> parents: 42823 diff changeset	586 requirealias[n.name],
57875cf423c9 style: run a patched black on a subset of mercurial Augie Fackler <augie@google.com> parents: 42823 diff changeset	587 )
57875cf423c9 style: run a patched black on a subset of mercurial Augie Fackler <augie@google.com> parents: 42823 diff changeset	588
25702 ab2c5163900e import-checker: establish new function for verifying import conventions Gregory Szorc <gregory.szorc@gmail.com> parents: 25701 diff changeset	589
20036 e5d51fa51aba contrib: add an import checker Augie Fackler <raf@durin42.com> parents: diff changeset	590 class CircularImport(Exception):
e5d51fa51aba contrib: add an import checker Augie Fackler <raf@durin42.com> parents: diff changeset	591 pass
e5d51fa51aba contrib: add an import checker Augie Fackler <raf@durin42.com> parents: diff changeset	592
43075 57875cf423c9 style: run a patched black on a subset of mercurial Augie Fackler <augie@google.com> parents: 42823 diff changeset	593
24490 fb4639d5268e import-checker: make search algorithm non-recursive breadth-first Matt Mackall <mpm@selenic.com> parents: 24489 diff changeset	594 def checkmod(mod, imports):
fb4639d5268e import-checker: make search algorithm non-recursive breadth-first Matt Mackall <mpm@selenic.com> parents: 24489 diff changeset	595 shortest = {}
fb4639d5268e import-checker: make search algorithm non-recursive breadth-first Matt Mackall <mpm@selenic.com> parents: 24489 diff changeset	596 visit = [[mod]]
fb4639d5268e import-checker: make search algorithm non-recursive breadth-first Matt Mackall <mpm@selenic.com> parents: 24489 diff changeset	597 while visit:
fb4639d5268e import-checker: make search algorithm non-recursive breadth-first Matt Mackall <mpm@selenic.com> parents: 24489 diff changeset	598 path = visit.pop(0)
fb4639d5268e import-checker: make search algorithm non-recursive breadth-first Matt Mackall <mpm@selenic.com> parents: 24489 diff changeset	599 for i in sorted(imports.get(path[-1], [])):
fb4639d5268e import-checker: make search algorithm non-recursive breadth-first Matt Mackall <mpm@selenic.com> parents: 24489 diff changeset	600 if len(path) < shortest.get(i, 1000):
fb4639d5268e import-checker: make search algorithm non-recursive breadth-first Matt Mackall <mpm@selenic.com> parents: 24489 diff changeset	601 shortest[i] = len(path)
fb4639d5268e import-checker: make search algorithm non-recursive breadth-first Matt Mackall <mpm@selenic.com> parents: 24489 diff changeset	602 if i in path:
fb4639d5268e import-checker: make search algorithm non-recursive breadth-first Matt Mackall <mpm@selenic.com> parents: 24489 diff changeset	603 if i == path[0]:
fb4639d5268e import-checker: make search algorithm non-recursive breadth-first Matt Mackall <mpm@selenic.com> parents: 24489 diff changeset	604 raise CircularImport(path)
fb4639d5268e import-checker: make search algorithm non-recursive breadth-first Matt Mackall <mpm@selenic.com> parents: 24489 diff changeset	605 continue
fb4639d5268e import-checker: make search algorithm non-recursive breadth-first Matt Mackall <mpm@selenic.com> parents: 24489 diff changeset	606 visit.append(path + [i])
20036 e5d51fa51aba contrib: add an import checker Augie Fackler <raf@durin42.com> parents: diff changeset	607
43075 57875cf423c9 style: run a patched black on a subset of mercurial Augie Fackler <augie@google.com> parents: 42823 diff changeset	608
20038 c65a6937b828 import-checker: try a little harder to show fewer cycles Augie Fackler <raf@durin42.com> parents: 20037 diff changeset	609 def rotatecycle(cycle):
c65a6937b828 import-checker: try a little harder to show fewer cycles Augie Fackler <raf@durin42.com> parents: 20037 diff changeset	610 """arrange a cycle so that the lexicographically first module listed first
c65a6937b828 import-checker: try a little harder to show fewer cycles Augie Fackler <raf@durin42.com> parents: 20037 diff changeset	611
24488 4b3fc46097f7 import-checker: drop duplicate element from cycle Matt Mackall <mpm@selenic.com> parents: 24487 diff changeset	612 >>> rotatecycle(['foo', 'bar'])
20038 c65a6937b828 import-checker: try a little harder to show fewer cycles Augie Fackler <raf@durin42.com> parents: 20037 diff changeset	613 ['bar', 'foo', 'bar']
c65a6937b828 import-checker: try a little harder to show fewer cycles Augie Fackler <raf@durin42.com> parents: 20037 diff changeset	614 """
c65a6937b828 import-checker: try a little harder to show fewer cycles Augie Fackler <raf@durin42.com> parents: 20037 diff changeset	615 lowest = min(cycle)
c65a6937b828 import-checker: try a little harder to show fewer cycles Augie Fackler <raf@durin42.com> parents: 20037 diff changeset	616 idx = cycle.index(lowest)
24488 4b3fc46097f7 import-checker: drop duplicate element from cycle Matt Mackall <mpm@selenic.com> parents: 24487 diff changeset	617 return cycle[idx:] + cycle[:idx] + [lowest]
20036 e5d51fa51aba contrib: add an import checker Augie Fackler <raf@durin42.com> parents: diff changeset	618
43075 57875cf423c9 style: run a patched black on a subset of mercurial Augie Fackler <augie@google.com> parents: 42823 diff changeset	619
20036 e5d51fa51aba contrib: add an import checker Augie Fackler <raf@durin42.com> parents: diff changeset	620 def find_cycles(imports):
e5d51fa51aba contrib: add an import checker Augie Fackler <raf@durin42.com> parents: diff changeset	621 """Find cycles in an already-loaded import graph.
e5d51fa51aba contrib: add an import checker Augie Fackler <raf@durin42.com> parents: diff changeset	622
25175 10e6c4b7121b import-checker: don't treat modules as relative one if not found FUJIWARA Katsunori <foozy@lares.dti.ne.jp> parents: 25174 diff changeset	623 All module names recorded in `imports` should be absolute one.
10e6c4b7121b import-checker: don't treat modules as relative one if not found FUJIWARA Katsunori <foozy@lares.dti.ne.jp> parents: 25174 diff changeset	624
10e6c4b7121b import-checker: don't treat modules as relative one if not found FUJIWARA Katsunori <foozy@lares.dti.ne.jp> parents: 25174 diff changeset	625 >>> imports = {'top.foo': ['top.bar', 'os.path', 'top.qux'],
10e6c4b7121b import-checker: don't treat modules as relative one if not found FUJIWARA Katsunori <foozy@lares.dti.ne.jp> parents: 25174 diff changeset	626 ... 'top.bar': ['top.baz', 'sys'],
10e6c4b7121b import-checker: don't treat modules as relative one if not found FUJIWARA Katsunori <foozy@lares.dti.ne.jp> parents: 25174 diff changeset	627 ... 'top.baz': ['top.foo'],
10e6c4b7121b import-checker: don't treat modules as relative one if not found FUJIWARA Katsunori <foozy@lares.dti.ne.jp> parents: 25174 diff changeset	628 ... 'top.qux': ['top.foo']}
28703 a274c4f9087a py3: use print_function in import-checker timeless <timeless@mozdev.org> parents: 28702 diff changeset	629 >>> print('\\n'.join(sorted(find_cycles(imports))))
24487 642d245ff537 import-checker: fix rotatecycle Matt Mackall <mpm@selenic.com> parents: 22975 diff changeset	630 top.bar -> top.baz -> top.foo -> top.bar
642d245ff537 import-checker: fix rotatecycle Matt Mackall <mpm@selenic.com> parents: 22975 diff changeset	631 top.foo -> top.qux -> top.foo
20036 e5d51fa51aba contrib: add an import checker Augie Fackler <raf@durin42.com> parents: diff changeset	632 """
24491 784b278b349c import-checker: rotatecycle is actually the canonical cycle key Matt Mackall <mpm@selenic.com> parents: 24490 diff changeset	633 cycles = set()
28704 1fa6fdb72275 py3: handle iter/iterkeys+iteritems python3 divergence in import-checker timeless <timeless@mozdev.org> parents: 28703 diff changeset	634 for mod in sorted(imports.keys()):
20036 e5d51fa51aba contrib: add an import checker Augie Fackler <raf@durin42.com> parents: diff changeset	635 try:
24490 fb4639d5268e import-checker: make search algorithm non-recursive breadth-first Matt Mackall <mpm@selenic.com> parents: 24489 diff changeset	636 checkmod(mod, imports)
25660 328739ea70c3 global: mass rewrite to use modern exception syntax Gregory Szorc <gregory.szorc@gmail.com> parents: 25175 diff changeset	637 except CircularImport as e:
20036 e5d51fa51aba contrib: add an import checker Augie Fackler <raf@durin42.com> parents: diff changeset	638 cycle = e.args[0]
24491 784b278b349c import-checker: rotatecycle is actually the canonical cycle key Matt Mackall <mpm@selenic.com> parents: 24490 diff changeset	639 cycles.add(" -> ".join(rotatecycle(cycle)))
784b278b349c import-checker: rotatecycle is actually the canonical cycle key Matt Mackall <mpm@selenic.com> parents: 24490 diff changeset	640 return cycles
20036 e5d51fa51aba contrib: add an import checker Augie Fackler <raf@durin42.com> parents: diff changeset	641
43075 57875cf423c9 style: run a patched black on a subset of mercurial Augie Fackler <augie@google.com> parents: 42823 diff changeset	642
20036 e5d51fa51aba contrib: add an import checker Augie Fackler <raf@durin42.com> parents: diff changeset	643 def _cycle_sortkey(c):
e5d51fa51aba contrib: add an import checker Augie Fackler <raf@durin42.com> parents: diff changeset	644 return len(c), c
e5d51fa51aba contrib: add an import checker Augie Fackler <raf@durin42.com> parents: diff changeset	645
43075 57875cf423c9 style: run a patched black on a subset of mercurial Augie Fackler <augie@google.com> parents: 42823 diff changeset	646
28922 4ec62a084e5c import-checker: parse python code from .t files timeless <timeless@mozdev.org> parents: 28921 diff changeset	647 def embedded(f, modname, src):
4ec62a084e5c import-checker: parse python code from .t files timeless <timeless@mozdev.org> parents: 28921 diff changeset	648 """Extract embedded python code
4ec62a084e5c import-checker: parse python code from .t files timeless <timeless@mozdev.org> parents: 28921 diff changeset	649
33896 bcf53149ebce contrib: make import checker always think in terms of bytes Augie Fackler <raf@durin42.com> parents: 33895 diff changeset	650 >>> def _forcestr(thing):
bcf53149ebce contrib: make import checker always think in terms of bytes Augie Fackler <raf@durin42.com> parents: 33895 diff changeset	651 ... if not isinstance(thing, str):
bcf53149ebce contrib: make import checker always think in terms of bytes Augie Fackler <raf@durin42.com> parents: 33895 diff changeset	652 ... return thing.decode('ascii')
bcf53149ebce contrib: make import checker always think in terms of bytes Augie Fackler <raf@durin42.com> parents: 33895 diff changeset	653 ... return thing
28922 4ec62a084e5c import-checker: parse python code from .t files timeless <timeless@mozdev.org> parents: 28921 diff changeset	654 >>> def test(fn, lines):
33896 bcf53149ebce contrib: make import checker always think in terms of bytes Augie Fackler <raf@durin42.com> parents: 33895 diff changeset	655 ... for s, m, f, l in embedded(fn, b"example", lines):
bcf53149ebce contrib: make import checker always think in terms of bytes Augie Fackler <raf@durin42.com> parents: 33895 diff changeset	656 ... print("%s %s %d" % (_forcestr(m), _forcestr(f), l))
bcf53149ebce contrib: make import checker always think in terms of bytes Augie Fackler <raf@durin42.com> parents: 33895 diff changeset	657 ... print(repr(_forcestr(s)))
28922 4ec62a084e5c import-checker: parse python code from .t files timeless <timeless@mozdev.org> parents: 28921 diff changeset	658 >>> lines = [
42407 04eb3c5607af contrib: fix import-checker to operate on str instead of bytes Augie Fackler <augie@google.com> parents: 41161 diff changeset	659 ... 'comment',
04eb3c5607af contrib: fix import-checker to operate on str instead of bytes Augie Fackler <augie@google.com> parents: 41161 diff changeset	660 ... ' >>> from __future__ import print_function',
04eb3c5607af contrib: fix import-checker to operate on str instead of bytes Augie Fackler <augie@google.com> parents: 41161 diff changeset	661 ... " >>> ' multiline",
04eb3c5607af contrib: fix import-checker to operate on str instead of bytes Augie Fackler <augie@google.com> parents: 41161 diff changeset	662 ... " ... string'",
04eb3c5607af contrib: fix import-checker to operate on str instead of bytes Augie Fackler <augie@google.com> parents: 41161 diff changeset	663 ... ' ',
04eb3c5607af contrib: fix import-checker to operate on str instead of bytes Augie Fackler <augie@google.com> parents: 41161 diff changeset	664 ... 'comment',
04eb3c5607af contrib: fix import-checker to operate on str instead of bytes Augie Fackler <augie@google.com> parents: 41161 diff changeset	665 ... ' $ cat > foo.py <<EOF',
04eb3c5607af contrib: fix import-checker to operate on str instead of bytes Augie Fackler <augie@google.com> parents: 41161 diff changeset	666 ... ' > from __future__ import print_function',
04eb3c5607af contrib: fix import-checker to operate on str instead of bytes Augie Fackler <augie@google.com> parents: 41161 diff changeset	667 ... ' > EOF',
28922 4ec62a084e5c import-checker: parse python code from .t files timeless <timeless@mozdev.org> parents: 28921 diff changeset	668 ... ]
33896 bcf53149ebce contrib: make import checker always think in terms of bytes Augie Fackler <raf@durin42.com> parents: 33895 diff changeset	669 >>> test(b"example.t", lines)
40096 7288838bec1f import-checker: use testparseutil.embedded() to centralize detection logic FUJIWARA Katsunori <foozy@lares.dti.ne.jp> parents: 39356 diff changeset	670 example[2] doctest.py 1
7288838bec1f import-checker: use testparseutil.embedded() to centralize detection logic FUJIWARA Katsunori <foozy@lares.dti.ne.jp> parents: 39356 diff changeset	671 "from __future__ import print_function\\n' multiline\\nstring'\\n\\n"
7288838bec1f import-checker: use testparseutil.embedded() to centralize detection logic FUJIWARA Katsunori <foozy@lares.dti.ne.jp> parents: 39356 diff changeset	672 example[8] foo.py 7
28922 4ec62a084e5c import-checker: parse python code from .t files timeless <timeless@mozdev.org> parents: 28921 diff changeset	673 'from __future__ import print_function\\n'
4ec62a084e5c import-checker: parse python code from .t files timeless <timeless@mozdev.org> parents: 28921 diff changeset	674 """
40096 7288838bec1f import-checker: use testparseutil.embedded() to centralize detection logic FUJIWARA Katsunori <foozy@lares.dti.ne.jp> parents: 39356 diff changeset	675 errors = []
7288838bec1f import-checker: use testparseutil.embedded() to centralize detection logic FUJIWARA Katsunori <foozy@lares.dti.ne.jp> parents: 39356 diff changeset	676 for name, starts, ends, code in testparseutil.pyembedded(f, src, errors):
7288838bec1f import-checker: use testparseutil.embedded() to centralize detection logic FUJIWARA Katsunori <foozy@lares.dti.ne.jp> parents: 39356 diff changeset	677 if not name:
7288838bec1f import-checker: use testparseutil.embedded() to centralize detection logic FUJIWARA Katsunori <foozy@lares.dti.ne.jp> parents: 39356 diff changeset	678 # use 'doctest.py', in order to make already existing
7288838bec1f import-checker: use testparseutil.embedded() to centralize detection logic FUJIWARA Katsunori <foozy@lares.dti.ne.jp> parents: 39356 diff changeset	679 # doctest above pass instantly
7288838bec1f import-checker: use testparseutil.embedded() to centralize detection logic FUJIWARA Katsunori <foozy@lares.dti.ne.jp> parents: 39356 diff changeset	680 name = 'doctest.py'
7288838bec1f import-checker: use testparseutil.embedded() to centralize detection logic FUJIWARA Katsunori <foozy@lares.dti.ne.jp> parents: 39356 diff changeset	681 # "starts" is "line number" (1-origin), but embedded() is
7288838bec1f import-checker: use testparseutil.embedded() to centralize detection logic FUJIWARA Katsunori <foozy@lares.dti.ne.jp> parents: 39356 diff changeset	682 # expected to return "line offset" (0-origin). Therefore, this
7288838bec1f import-checker: use testparseutil.embedded() to centralize detection logic FUJIWARA Katsunori <foozy@lares.dti.ne.jp> parents: 39356 diff changeset	683 # yields "starts - 1".
40530 99d5424eedc8 contrib: fix import-checker to not b'' module names on Python 3 Augie Fackler <augie@google.com> parents: 40096 diff changeset	684 if not isinstance(modname, str):
99d5424eedc8 contrib: fix import-checker to not b'' module names on Python 3 Augie Fackler <augie@google.com> parents: 40096 diff changeset	685 modname = modname.decode('utf8')
40096 7288838bec1f import-checker: use testparseutil.embedded() to centralize detection logic FUJIWARA Katsunori <foozy@lares.dti.ne.jp> parents: 39356 diff changeset	686 yield code, "%s[%d]" % (modname, starts), name, starts - 1
28922 4ec62a084e5c import-checker: parse python code from .t files timeless <timeless@mozdev.org> parents: 28921 diff changeset	687
43075 57875cf423c9 style: run a patched black on a subset of mercurial Augie Fackler <augie@google.com> parents: 42823 diff changeset	688
28919 a94f34306bb9 import-checker: refactor source reading timeless <timeless@mozdev.org> parents: 28713 diff changeset	689 def sources(f, modname):
28922 4ec62a084e5c import-checker: parse python code from .t files timeless <timeless@mozdev.org> parents: 28921 diff changeset	690 """Yields possibly multiple sources from a filepath
4ec62a084e5c import-checker: parse python code from .t files timeless <timeless@mozdev.org> parents: 28921 diff changeset	691
4ec62a084e5c import-checker: parse python code from .t files timeless <timeless@mozdev.org> parents: 28921 diff changeset	692 input: filepath, modulename
4ec62a084e5c import-checker: parse python code from .t files timeless <timeless@mozdev.org> parents: 28921 diff changeset	693 yields: script(string), modulename, filepath, linenumber
4ec62a084e5c import-checker: parse python code from .t files timeless <timeless@mozdev.org> parents: 28921 diff changeset	694
4ec62a084e5c import-checker: parse python code from .t files timeless <timeless@mozdev.org> parents: 28921 diff changeset	695 For embedded scripts, the modulename and filepath will be different
4ec62a084e5c import-checker: parse python code from .t files timeless <timeless@mozdev.org> parents: 28921 diff changeset	696 from the function arguments. linenumber is an offset relative to
4ec62a084e5c import-checker: parse python code from .t files timeless <timeless@mozdev.org> parents: 28921 diff changeset	697 the input file.
4ec62a084e5c import-checker: parse python code from .t files timeless <timeless@mozdev.org> parents: 28921 diff changeset	698 """
4ec62a084e5c import-checker: parse python code from .t files timeless <timeless@mozdev.org> parents: 28921 diff changeset	699 py = False
29234 393aef802535 tests: enable import checker for all python files (including no .py files) Yuya Nishihara <yuya@tcha.org> parents: 29211 diff changeset	700 if not f.endswith('.t'):
33896 bcf53149ebce contrib: make import checker always think in terms of bytes Augie Fackler <raf@durin42.com> parents: 33895 diff changeset	701 with open(f, 'rb') as src:
28922 4ec62a084e5c import-checker: parse python code from .t files timeless <timeless@mozdev.org> parents: 28921 diff changeset	702 yield src.read(), modname, f, 0
4ec62a084e5c import-checker: parse python code from .t files timeless <timeless@mozdev.org> parents: 28921 diff changeset	703 py = True
4ec62a084e5c import-checker: parse python code from .t files timeless <timeless@mozdev.org> parents: 28921 diff changeset	704 if py or f.endswith('.t'):
43415 a8454e846736 import-checker: open all source files as utf-8 Gregory Szorc <gregory.szorc@gmail.com> parents: 43084 diff changeset	705 # Strictly speaking we should sniff for the magic header that denotes
a8454e846736 import-checker: open all source files as utf-8 Gregory Szorc <gregory.szorc@gmail.com> parents: 43084 diff changeset	706 # Python source file encoding. But in reality we don't use anything
a8454e846736 import-checker: open all source files as utf-8 Gregory Szorc <gregory.szorc@gmail.com> parents: 43084 diff changeset	707 # other than ASCII (mainly) and UTF-8 (in a few exceptions), so
a8454e846736 import-checker: open all source files as utf-8 Gregory Szorc <gregory.szorc@gmail.com> parents: 43084 diff changeset	708 # simplicity is fine.
52681 3e84e001b6c1 pyupgrade: drop redundant `open()` modes Matt Harbison <matt_harbison@yahoo.com> parents: 52675 diff changeset	709 with open(f, encoding='utf-8') as src:
28922 4ec62a084e5c import-checker: parse python code from .t files timeless <timeless@mozdev.org> parents: 28921 diff changeset	710 for script, modname, t, line in embedded(f, modname, src):
40531 90517fad4293 contrib: tweak import-checker to always use bytes for module names Augie Fackler <augie@google.com> parents: 40530 diff changeset	711 yield script, modname.encode('utf8'), t, line
28919 a94f34306bb9 import-checker: refactor source reading timeless <timeless@mozdev.org> parents: 28713 diff changeset	712
43075 57875cf423c9 style: run a patched black on a subset of mercurial Augie Fackler <augie@google.com> parents: 42823 diff changeset	713
20036 e5d51fa51aba contrib: add an import checker Augie Fackler <raf@durin42.com> parents: diff changeset	714 def main(argv):
25063 723e364488f4 import-checker: add xargs like mode FUJIWARA Katsunori <foozy@lares.dti.ne.jp> parents: 24669 diff changeset	715 if len(argv) < 2 or (argv[1] == '-' and len(argv) > 2):
28703 a274c4f9087a py3: use print_function in import-checker timeless <timeless@mozdev.org> parents: 28702 diff changeset	716 print('Usage: %s {-\|file [file] [file] ...}')
20036 e5d51fa51aba contrib: add an import checker Augie Fackler <raf@durin42.com> parents: diff changeset	717 return 1
25063 723e364488f4 import-checker: add xargs like mode FUJIWARA Katsunori <foozy@lares.dti.ne.jp> parents: 24669 diff changeset	718 if argv[1] == '-':
723e364488f4 import-checker: add xargs like mode FUJIWARA Katsunori <foozy@lares.dti.ne.jp> parents: 24669 diff changeset	719 argv = argv[:1]
723e364488f4 import-checker: add xargs like mode FUJIWARA Katsunori <foozy@lares.dti.ne.jp> parents: 24669 diff changeset	720 argv.extend(l.rstrip() for l in sys.stdin.readlines())
32541 4c712b90c60a import-checker: convert localmods to a set of module names Yuya Nishihara <yuya@tcha.org> parents: 32540 diff changeset	721 localmodpaths = {}
20036 e5d51fa51aba contrib: add an import checker Augie Fackler <raf@durin42.com> parents: diff changeset	722 used_imports = {}
e5d51fa51aba contrib: add an import checker Augie Fackler <raf@durin42.com> parents: diff changeset	723 any_errors = False
e5d51fa51aba contrib: add an import checker Augie Fackler <raf@durin42.com> parents: diff changeset	724 for source_path in argv[1:]:
32413 194b0f781132 import-checker: drop workaround for pure modules Yuya Nishihara <yuya@tcha.org> parents: 32411 diff changeset	725 modname = dotted_name_of_path(source_path)
32541 4c712b90c60a import-checker: convert localmods to a set of module names Yuya Nishihara <yuya@tcha.org> parents: 32540 diff changeset	726 localmodpaths[modname] = source_path
32542 a025ec43856c import-checker: guess names of C extension modules Yuya Nishihara <yuya@tcha.org> parents: 32541 diff changeset	727 localmods = populateextmods(localmodpaths)
32541 4c712b90c60a import-checker: convert localmods to a set of module names Yuya Nishihara <yuya@tcha.org> parents: 32540 diff changeset	728 for localmodname, source_path in sorted(localmodpaths.items()):
33908 3de9a2df6411 contrib: have import-checker work mostly with native strings for mod names Augie Fackler <raf@durin42.com> parents: 33896 diff changeset	729 if not isinstance(localmodname, bytes):
3de9a2df6411 contrib: have import-checker work mostly with native strings for mod names Augie Fackler <raf@durin42.com> parents: 33896 diff changeset	730 # This is only safe because all hg's files are ascii
3de9a2df6411 contrib: have import-checker work mostly with native strings for mod names Augie Fackler <raf@durin42.com> parents: 33896 diff changeset	731 localmodname = localmodname.encode('ascii')
28922 4ec62a084e5c import-checker: parse python code from .t files timeless <timeless@mozdev.org> parents: 28921 diff changeset	732 for src, modname, name, line in sources(source_path, localmodname):
28920 cdf331b54eb8 import-checker: track SyntaxErrors timeless <timeless@mozdev.org> parents: 28919 diff changeset	733 try:
cdf331b54eb8 import-checker: track SyntaxErrors timeless <timeless@mozdev.org> parents: 28919 diff changeset	734 used_imports[modname] = sorted(
43075 57875cf423c9 style: run a patched black on a subset of mercurial Augie Fackler <augie@google.com> parents: 42823 diff changeset	735 imported_modules(
57875cf423c9 style: run a patched black on a subset of mercurial Augie Fackler <augie@google.com> parents: 42823 diff changeset	736 src, modname, name, localmods, ignore_nested=True
57875cf423c9 style: run a patched black on a subset of mercurial Augie Fackler <augie@google.com> parents: 42823 diff changeset	737 )
57875cf423c9 style: run a patched black on a subset of mercurial Augie Fackler <augie@google.com> parents: 42823 diff changeset	738 )
57875cf423c9 style: run a patched black on a subset of mercurial Augie Fackler <augie@google.com> parents: 42823 diff changeset	739 for error, lineno in verify_import_convention(
57875cf423c9 style: run a patched black on a subset of mercurial Augie Fackler <augie@google.com> parents: 42823 diff changeset	740 modname, src, localmods
57875cf423c9 style: run a patched black on a subset of mercurial Augie Fackler <augie@google.com> parents: 42823 diff changeset	741 ):
28920 cdf331b54eb8 import-checker: track SyntaxErrors timeless <timeless@mozdev.org> parents: 28919 diff changeset	742 any_errors = True
28922 4ec62a084e5c import-checker: parse python code from .t files timeless <timeless@mozdev.org> parents: 28921 diff changeset	743 print('%s:%d: %s' % (source_path, lineno + line, error))
28920 cdf331b54eb8 import-checker: track SyntaxErrors timeless <timeless@mozdev.org> parents: 28919 diff changeset	744 except SyntaxError as e:
43075 57875cf423c9 style: run a patched black on a subset of mercurial Augie Fackler <augie@google.com> parents: 42823 diff changeset	745 print(
57875cf423c9 style: run a patched black on a subset of mercurial Augie Fackler <augie@google.com> parents: 42823 diff changeset	746 '%s:%d: SyntaxError: %s' % (source_path, e.lineno + line, e)
57875cf423c9 style: run a patched black on a subset of mercurial Augie Fackler <augie@google.com> parents: 42823 diff changeset	747 )
20036 e5d51fa51aba contrib: add an import checker Augie Fackler <raf@durin42.com> parents: diff changeset	748 cycles = find_cycles(used_imports)
e5d51fa51aba contrib: add an import checker Augie Fackler <raf@durin42.com> parents: diff changeset	749 if cycles:
e5d51fa51aba contrib: add an import checker Augie Fackler <raf@durin42.com> parents: diff changeset	750 firstmods = set()
e5d51fa51aba contrib: add an import checker Augie Fackler <raf@durin42.com> parents: diff changeset	751 for c in sorted(cycles, key=_cycle_sortkey):
e5d51fa51aba contrib: add an import checker Augie Fackler <raf@durin42.com> parents: diff changeset	752 first = c.split()[0]
e5d51fa51aba contrib: add an import checker Augie Fackler <raf@durin42.com> parents: diff changeset	753 # As a rough cut, ignore any cycle that starts with the
e5d51fa51aba contrib: add an import checker Augie Fackler <raf@durin42.com> parents: diff changeset	754 # same module as some other cycle. Otherwise we see lots
e5d51fa51aba contrib: add an import checker Augie Fackler <raf@durin42.com> parents: diff changeset	755 # of cycles that are effectively duplicates.
e5d51fa51aba contrib: add an import checker Augie Fackler <raf@durin42.com> parents: diff changeset	756 if first in firstmods:
e5d51fa51aba contrib: add an import checker Augie Fackler <raf@durin42.com> parents: diff changeset	757 continue
28703 a274c4f9087a py3: use print_function in import-checker timeless <timeless@mozdev.org> parents: 28702 diff changeset	758 print('Import cycle:', c)
20036 e5d51fa51aba contrib: add an import checker Augie Fackler <raf@durin42.com> parents: diff changeset	759 firstmods.add(first)
e5d51fa51aba contrib: add an import checker Augie Fackler <raf@durin42.com> parents: diff changeset	760 any_errors = True
25731 cd1daab5d036 import-checker.py: exit with code 0 if no error is detected FUJIWARA Katsunori <foozy@lares.dti.ne.jp> parents: 25703 diff changeset	761 return any_errors != 0
20036 e5d51fa51aba contrib: add an import checker Augie Fackler <raf@durin42.com> parents: diff changeset	762
43075 57875cf423c9 style: run a patched black on a subset of mercurial Augie Fackler <augie@google.com> parents: 42823 diff changeset	763
20036 e5d51fa51aba contrib: add an import checker Augie Fackler <raf@durin42.com> parents: diff changeset	764 if __name__ == '__main__':
e5d51fa51aba contrib: add an import checker Augie Fackler <raf@durin42.com> parents: diff changeset	765 sys.exit(int(main(sys.argv)))

Mercurial > public > mercurial-scm > hg-stable

annotate contrib/import-checker.py @ 53040:cdd7bf612c7b stable tip