mercurial-scm/hg: contrib/import-checker.py annotate

annotate contrib/import-checker.py @ 28713:806d260c6f3b

tests: fix builtin module test on pypy On pypy datetime and cProfile are modules written in Python, not in C. For the purpose of this test, just list them explicitely as builtins, which silences warnings about them being imported before stdlib modules.

author	Maciej Fijalkowski <fijall@gmail.com>
date	Wed, 30 Mar 2016 21:54:26 +0200
parents	1fa6fdb72275
children	a94f34306bb9

rev	line source
26954 f804bf27439b import-checker: make it executable for convenience Yuya Nishihara <yuya@tcha.org> parents: 26781 diff changeset	1 #!/usr/bin/env python
f804bf27439b import-checker: make it executable for convenience Yuya Nishihara <yuya@tcha.org> parents: 26781 diff changeset	2
28703 a274c4f9087a py3: use print_function in import-checker timeless <timeless@mozdev.org> parents: 28702 diff changeset	3 from __future__ import absolute_import, print_function
28702 e44f671018e3 py3: use absolute_import in import-checker timeless <timeless@mozdev.org> parents: 28700 diff changeset	4
20036 e5d51fa51aba contrib: add an import checker Augie Fackler <raf@durin42.com> parents: diff changeset	5 import ast
26965 1fa66d3ad28d import-checker: reset context to verify convention in function scope Yuya Nishihara <yuya@tcha.org> parents: 26964 diff changeset	6 import collections
20036 e5d51fa51aba contrib: add an import checker Augie Fackler <raf@durin42.com> parents: diff changeset	7 import os
e5d51fa51aba contrib: add an import checker Augie Fackler <raf@durin42.com> parents: diff changeset	8 import sys
e5d51fa51aba contrib: add an import checker Augie Fackler <raf@durin42.com> parents: diff changeset	9
20198 f5393a9dc4e5 import-checker: make test-module-imports.t work using virtualenv (issue4129) Chris Jerdonek <chris.jerdonek@gmail.com> parents: 20197 diff changeset	10 # Import a minimal set of stdlib modules needed for list_stdlib_modules()
f5393a9dc4e5 import-checker: make test-module-imports.t work using virtualenv (issue4129) Chris Jerdonek <chris.jerdonek@gmail.com> parents: 20197 diff changeset	11 # to work when run from a virtualenv. The modules were chosen empirically
f5393a9dc4e5 import-checker: make test-module-imports.t work using virtualenv (issue4129) Chris Jerdonek <chris.jerdonek@gmail.com> parents: 20197 diff changeset	12 # so that the return value matches the return value without virtualenv.
f5393a9dc4e5 import-checker: make test-module-imports.t work using virtualenv (issue4129) Chris Jerdonek <chris.jerdonek@gmail.com> parents: 20197 diff changeset	13 import BaseHTTPServer
f5393a9dc4e5 import-checker: make test-module-imports.t work using virtualenv (issue4129) Chris Jerdonek <chris.jerdonek@gmail.com> parents: 20197 diff changeset	14 import zlib
f5393a9dc4e5 import-checker: make test-module-imports.t work using virtualenv (issue4129) Chris Jerdonek <chris.jerdonek@gmail.com> parents: 20197 diff changeset	15
25703 1a6a117d0b95 import-checker: establish modern import convention Gregory Szorc <gregory.szorc@gmail.com> parents: 25702 diff changeset	16 # Whitelist of modules that symbols can be directly imported from.
1a6a117d0b95 import-checker: establish modern import convention Gregory Szorc <gregory.szorc@gmail.com> parents: 25702 diff changeset	17 allowsymbolimports = (
1a6a117d0b95 import-checker: establish modern import convention Gregory Szorc <gregory.szorc@gmail.com> parents: 25702 diff changeset	18 '__future__',
27018 e5be48dd8215 import-checker: allow symbol imports from hgweb.common and .request Yuya Nishihara <yuya@tcha.org> parents: 26965 diff changeset	19 'mercurial.hgweb.common',
e5be48dd8215 import-checker: allow symbol imports from hgweb.common and .request Yuya Nishihara <yuya@tcha.org> parents: 26965 diff changeset	20 'mercurial.hgweb.request',
25703 1a6a117d0b95 import-checker: establish modern import convention Gregory Szorc <gregory.szorc@gmail.com> parents: 25702 diff changeset	21 'mercurial.i18n',
1a6a117d0b95 import-checker: establish modern import convention Gregory Szorc <gregory.szorc@gmail.com> parents: 25702 diff changeset	22 'mercurial.node',
1a6a117d0b95 import-checker: establish modern import convention Gregory Szorc <gregory.szorc@gmail.com> parents: 25702 diff changeset	23 )
1a6a117d0b95 import-checker: establish modern import convention Gregory Szorc <gregory.szorc@gmail.com> parents: 25702 diff changeset	24
1a6a117d0b95 import-checker: establish modern import convention Gregory Szorc <gregory.szorc@gmail.com> parents: 25702 diff changeset	25 # Modules that must be aliased because they are commonly confused with
1a6a117d0b95 import-checker: establish modern import convention Gregory Szorc <gregory.szorc@gmail.com> parents: 25702 diff changeset	26 # common variables and can create aliasing and readability issues.
1a6a117d0b95 import-checker: establish modern import convention Gregory Szorc <gregory.szorc@gmail.com> parents: 25702 diff changeset	27 requirealias = {
1a6a117d0b95 import-checker: establish modern import convention Gregory Szorc <gregory.szorc@gmail.com> parents: 25702 diff changeset	28 'ui': 'uimod',
1a6a117d0b95 import-checker: establish modern import convention Gregory Szorc <gregory.szorc@gmail.com> parents: 25702 diff changeset	29 }
1a6a117d0b95 import-checker: establish modern import convention Gregory Szorc <gregory.szorc@gmail.com> parents: 25702 diff changeset	30
1a6a117d0b95 import-checker: establish modern import convention Gregory Szorc <gregory.szorc@gmail.com> parents: 25702 diff changeset	31 def usingabsolute(root):
1a6a117d0b95 import-checker: establish modern import convention Gregory Szorc <gregory.szorc@gmail.com> parents: 25702 diff changeset	32 """Whether absolute imports are being used."""
1a6a117d0b95 import-checker: establish modern import convention Gregory Szorc <gregory.szorc@gmail.com> parents: 25702 diff changeset	33 if sys.version_info[0] >= 3:
1a6a117d0b95 import-checker: establish modern import convention Gregory Szorc <gregory.szorc@gmail.com> parents: 25702 diff changeset	34 return True
1a6a117d0b95 import-checker: establish modern import convention Gregory Szorc <gregory.szorc@gmail.com> parents: 25702 diff changeset	35
1a6a117d0b95 import-checker: establish modern import convention Gregory Szorc <gregory.szorc@gmail.com> parents: 25702 diff changeset	36 for node in ast.walk(root):
1a6a117d0b95 import-checker: establish modern import convention Gregory Szorc <gregory.szorc@gmail.com> parents: 25702 diff changeset	37 if isinstance(node, ast.ImportFrom):
1a6a117d0b95 import-checker: establish modern import convention Gregory Szorc <gregory.szorc@gmail.com> parents: 25702 diff changeset	38 if node.module == '__future__':
1a6a117d0b95 import-checker: establish modern import convention Gregory Szorc <gregory.szorc@gmail.com> parents: 25702 diff changeset	39 for n in node.names:
1a6a117d0b95 import-checker: establish modern import convention Gregory Szorc <gregory.szorc@gmail.com> parents: 25702 diff changeset	40 if n.name == 'absolute_import':
1a6a117d0b95 import-checker: establish modern import convention Gregory Szorc <gregory.szorc@gmail.com> parents: 25702 diff changeset	41 return True
1a6a117d0b95 import-checker: establish modern import convention Gregory Szorc <gregory.szorc@gmail.com> parents: 25702 diff changeset	42
1a6a117d0b95 import-checker: establish modern import convention Gregory Szorc <gregory.szorc@gmail.com> parents: 25702 diff changeset	43 return False
1a6a117d0b95 import-checker: establish modern import convention Gregory Szorc <gregory.szorc@gmail.com> parents: 25702 diff changeset	44
26965 1fa66d3ad28d import-checker: reset context to verify convention in function scope Yuya Nishihara <yuya@tcha.org> parents: 26964 diff changeset	45 def walklocal(root):
1fa66d3ad28d import-checker: reset context to verify convention in function scope Yuya Nishihara <yuya@tcha.org> parents: 26964 diff changeset	46 """Recursively yield all descendant nodes but not in a different scope"""
1fa66d3ad28d import-checker: reset context to verify convention in function scope Yuya Nishihara <yuya@tcha.org> parents: 26964 diff changeset	47 todo = collections.deque(ast.iter_child_nodes(root))
1fa66d3ad28d import-checker: reset context to verify convention in function scope Yuya Nishihara <yuya@tcha.org> parents: 26964 diff changeset	48 yield root, False
1fa66d3ad28d import-checker: reset context to verify convention in function scope Yuya Nishihara <yuya@tcha.org> parents: 26964 diff changeset	49 while todo:
1fa66d3ad28d import-checker: reset context to verify convention in function scope Yuya Nishihara <yuya@tcha.org> parents: 26964 diff changeset	50 node = todo.popleft()
1fa66d3ad28d import-checker: reset context to verify convention in function scope Yuya Nishihara <yuya@tcha.org> parents: 26964 diff changeset	51 newscope = isinstance(node, ast.FunctionDef)
1fa66d3ad28d import-checker: reset context to verify convention in function scope Yuya Nishihara <yuya@tcha.org> parents: 26964 diff changeset	52 if not newscope:
1fa66d3ad28d import-checker: reset context to verify convention in function scope Yuya Nishihara <yuya@tcha.org> parents: 26964 diff changeset	53 todo.extend(ast.iter_child_nodes(node))
1fa66d3ad28d import-checker: reset context to verify convention in function scope Yuya Nishihara <yuya@tcha.org> parents: 26964 diff changeset	54 yield node, newscope
1fa66d3ad28d import-checker: reset context to verify convention in function scope Yuya Nishihara <yuya@tcha.org> parents: 26964 diff changeset	55
22975 461342e1c8aa import-checker: check modules for pure Python build correctly FUJIWARA Katsunori <foozy@lares.dti.ne.jp> parents: 22974 diff changeset	56 def dotted_name_of_path(path, trimpure=False):
20036 e5d51fa51aba contrib: add an import checker Augie Fackler <raf@durin42.com> parents: diff changeset	57 """Given a relative path to a source file, return its dotted module name.
e5d51fa51aba contrib: add an import checker Augie Fackler <raf@durin42.com> parents: diff changeset	58
e5d51fa51aba contrib: add an import checker Augie Fackler <raf@durin42.com> parents: diff changeset	59 >>> dotted_name_of_path('mercurial/error.py')
e5d51fa51aba contrib: add an import checker Augie Fackler <raf@durin42.com> parents: diff changeset	60 'mercurial.error'
22975 461342e1c8aa import-checker: check modules for pure Python build correctly FUJIWARA Katsunori <foozy@lares.dti.ne.jp> parents: 22974 diff changeset	61 >>> dotted_name_of_path('mercurial/pure/parsers.py', trimpure=True)
461342e1c8aa import-checker: check modules for pure Python build correctly FUJIWARA Katsunori <foozy@lares.dti.ne.jp> parents: 22974 diff changeset	62 'mercurial.parsers'
20383 4990abb4729d import-checker: fix names of dynamically loaded modules Mads Kiilerich <madski@unity3d.com> parents: 20238 diff changeset	63 >>> dotted_name_of_path('zlibmodule.so')
4990abb4729d import-checker: fix names of dynamically loaded modules Mads Kiilerich <madski@unity3d.com> parents: 20238 diff changeset	64 'zlib'
20036 e5d51fa51aba contrib: add an import checker Augie Fackler <raf@durin42.com> parents: diff changeset	65 """
27620 0c60843b55b5 import-checker: normalize directory separator to get module name on Windows Yuya Nishihara <yuya@tcha.org> parents: 27520 diff changeset	66 parts = path.replace(os.sep, '/').split('/')
20391 466e4c574db0 import-checker: handle standard modules with arch in the filename Mads Kiilerich <madski@unity3d.com> parents: 20386 diff changeset	67 parts[-1] = parts[-1].split('.', 1)[0] # remove .py and .so and .ARCH.so
20383 4990abb4729d import-checker: fix names of dynamically loaded modules Mads Kiilerich <madski@unity3d.com> parents: 20238 diff changeset	68 if parts[-1].endswith('module'):
4990abb4729d import-checker: fix names of dynamically loaded modules Mads Kiilerich <madski@unity3d.com> parents: 20238 diff changeset	69 parts[-1] = parts[-1][:-6]
22975 461342e1c8aa import-checker: check modules for pure Python build correctly FUJIWARA Katsunori <foozy@lares.dti.ne.jp> parents: 22974 diff changeset	70 if trimpure:
461342e1c8aa import-checker: check modules for pure Python build correctly FUJIWARA Katsunori <foozy@lares.dti.ne.jp> parents: 22974 diff changeset	71 return '.'.join(p for p in parts if p != 'pure')
20036 e5d51fa51aba contrib: add an import checker Augie Fackler <raf@durin42.com> parents: diff changeset	72 return '.'.join(parts)
e5d51fa51aba contrib: add an import checker Augie Fackler <raf@durin42.com> parents: diff changeset	73
25173 7358b5d9991e import-checker: add utility to examine what module is imported easily FUJIWARA Katsunori <foozy@lares.dti.ne.jp> parents: 25064 diff changeset	74 def fromlocalfunc(modulename, localmods):
7358b5d9991e import-checker: add utility to examine what module is imported easily FUJIWARA Katsunori <foozy@lares.dti.ne.jp> parents: 25064 diff changeset	75 """Get a function to examine which locally defined module the
7358b5d9991e import-checker: add utility to examine what module is imported easily FUJIWARA Katsunori <foozy@lares.dti.ne.jp> parents: 25064 diff changeset	76 target source imports via a specified name.
7358b5d9991e import-checker: add utility to examine what module is imported easily FUJIWARA Katsunori <foozy@lares.dti.ne.jp> parents: 25064 diff changeset	77
7358b5d9991e import-checker: add utility to examine what module is imported easily FUJIWARA Katsunori <foozy@lares.dti.ne.jp> parents: 25064 diff changeset	78 `modulename` is an `dotted_name_of_path()`-ed source file path,
7358b5d9991e import-checker: add utility to examine what module is imported easily FUJIWARA Katsunori <foozy@lares.dti.ne.jp> parents: 25064 diff changeset	79 which may have `.__init__` at the end of it, of the target source.
7358b5d9991e import-checker: add utility to examine what module is imported easily FUJIWARA Katsunori <foozy@lares.dti.ne.jp> parents: 25064 diff changeset	80
7358b5d9991e import-checker: add utility to examine what module is imported easily FUJIWARA Katsunori <foozy@lares.dti.ne.jp> parents: 25064 diff changeset	81 `localmods` is a dict (or set), of which key is an absolute
7358b5d9991e import-checker: add utility to examine what module is imported easily FUJIWARA Katsunori <foozy@lares.dti.ne.jp> parents: 25064 diff changeset	82 `dotted_name_of_path()`-ed source file path of locally defined (=
7358b5d9991e import-checker: add utility to examine what module is imported easily FUJIWARA Katsunori <foozy@lares.dti.ne.jp> parents: 25064 diff changeset	83 Mercurial specific) modules.
7358b5d9991e import-checker: add utility to examine what module is imported easily FUJIWARA Katsunori <foozy@lares.dti.ne.jp> parents: 25064 diff changeset	84
7358b5d9991e import-checker: add utility to examine what module is imported easily FUJIWARA Katsunori <foozy@lares.dti.ne.jp> parents: 25064 diff changeset	85 This function assumes that module names not existing in
26781 1aee2ab0f902 spelling: trivial spell checking Mads Kiilerich <madski@unity3d.com> parents: 26221 diff changeset	86 `localmods` are from the Python standard library.
25173 7358b5d9991e import-checker: add utility to examine what module is imported easily FUJIWARA Katsunori <foozy@lares.dti.ne.jp> parents: 25064 diff changeset	87
7358b5d9991e import-checker: add utility to examine what module is imported easily FUJIWARA Katsunori <foozy@lares.dti.ne.jp> parents: 25064 diff changeset	88 This function returns the function, which takes `name` argument,
7358b5d9991e import-checker: add utility to examine what module is imported easily FUJIWARA Katsunori <foozy@lares.dti.ne.jp> parents: 25064 diff changeset	89 and returns `(absname, dottedpath, hassubmod)` tuple if `name`
7358b5d9991e import-checker: add utility to examine what module is imported easily FUJIWARA Katsunori <foozy@lares.dti.ne.jp> parents: 25064 diff changeset	90 matches against locally defined module. Otherwise, it returns
7358b5d9991e import-checker: add utility to examine what module is imported easily FUJIWARA Katsunori <foozy@lares.dti.ne.jp> parents: 25064 diff changeset	91 False.
7358b5d9991e import-checker: add utility to examine what module is imported easily FUJIWARA Katsunori <foozy@lares.dti.ne.jp> parents: 25064 diff changeset	92
7358b5d9991e import-checker: add utility to examine what module is imported easily FUJIWARA Katsunori <foozy@lares.dti.ne.jp> parents: 25064 diff changeset	93 It is assumed that `name` doesn't have `.__init__`.
7358b5d9991e import-checker: add utility to examine what module is imported easily FUJIWARA Katsunori <foozy@lares.dti.ne.jp> parents: 25064 diff changeset	94
7358b5d9991e import-checker: add utility to examine what module is imported easily FUJIWARA Katsunori <foozy@lares.dti.ne.jp> parents: 25064 diff changeset	95 `absname` is an absolute module name of specified `name`
7358b5d9991e import-checker: add utility to examine what module is imported easily FUJIWARA Katsunori <foozy@lares.dti.ne.jp> parents: 25064 diff changeset	96 (e.g. "hgext.convert"). This can be used to compose prefix for sub
7358b5d9991e import-checker: add utility to examine what module is imported easily FUJIWARA Katsunori <foozy@lares.dti.ne.jp> parents: 25064 diff changeset	97 modules or so.
7358b5d9991e import-checker: add utility to examine what module is imported easily FUJIWARA Katsunori <foozy@lares.dti.ne.jp> parents: 25064 diff changeset	98
7358b5d9991e import-checker: add utility to examine what module is imported easily FUJIWARA Katsunori <foozy@lares.dti.ne.jp> parents: 25064 diff changeset	99 `dottedpath` is a `dotted_name_of_path()`-ed source file path
7358b5d9991e import-checker: add utility to examine what module is imported easily FUJIWARA Katsunori <foozy@lares.dti.ne.jp> parents: 25064 diff changeset	100 (e.g. "hgext.convert.__init__") of `name`. This is used to look
7358b5d9991e import-checker: add utility to examine what module is imported easily FUJIWARA Katsunori <foozy@lares.dti.ne.jp> parents: 25064 diff changeset	101 module up in `localmods` again.
7358b5d9991e import-checker: add utility to examine what module is imported easily FUJIWARA Katsunori <foozy@lares.dti.ne.jp> parents: 25064 diff changeset	102
7358b5d9991e import-checker: add utility to examine what module is imported easily FUJIWARA Katsunori <foozy@lares.dti.ne.jp> parents: 25064 diff changeset	103 `hassubmod` is whether it may have sub modules under it (for
7358b5d9991e import-checker: add utility to examine what module is imported easily FUJIWARA Katsunori <foozy@lares.dti.ne.jp> parents: 25064 diff changeset	104 convenient, even though this is also equivalent to "absname !=
7358b5d9991e import-checker: add utility to examine what module is imported easily FUJIWARA Katsunori <foozy@lares.dti.ne.jp> parents: 25064 diff changeset	105 dottednpath")
7358b5d9991e import-checker: add utility to examine what module is imported easily FUJIWARA Katsunori <foozy@lares.dti.ne.jp> parents: 25064 diff changeset	106
7358b5d9991e import-checker: add utility to examine what module is imported easily FUJIWARA Katsunori <foozy@lares.dti.ne.jp> parents: 25064 diff changeset	107 >>> localmods = {'foo.__init__': True, 'foo.foo1': True,
7358b5d9991e import-checker: add utility to examine what module is imported easily FUJIWARA Katsunori <foozy@lares.dti.ne.jp> parents: 25064 diff changeset	108 ... 'foo.bar.__init__': True, 'foo.bar.bar1': True,
7358b5d9991e import-checker: add utility to examine what module is imported easily FUJIWARA Katsunori <foozy@lares.dti.ne.jp> parents: 25064 diff changeset	109 ... 'baz.__init__': True, 'baz.baz1': True }
7358b5d9991e import-checker: add utility to examine what module is imported easily FUJIWARA Katsunori <foozy@lares.dti.ne.jp> parents: 25064 diff changeset	110 >>> fromlocal = fromlocalfunc('foo.xxx', localmods)
7358b5d9991e import-checker: add utility to examine what module is imported easily FUJIWARA Katsunori <foozy@lares.dti.ne.jp> parents: 25064 diff changeset	111 >>> # relative
7358b5d9991e import-checker: add utility to examine what module is imported easily FUJIWARA Katsunori <foozy@lares.dti.ne.jp> parents: 25064 diff changeset	112 >>> fromlocal('foo1')
7358b5d9991e import-checker: add utility to examine what module is imported easily FUJIWARA Katsunori <foozy@lares.dti.ne.jp> parents: 25064 diff changeset	113 ('foo.foo1', 'foo.foo1', False)
7358b5d9991e import-checker: add utility to examine what module is imported easily FUJIWARA Katsunori <foozy@lares.dti.ne.jp> parents: 25064 diff changeset	114 >>> fromlocal('bar')
7358b5d9991e import-checker: add utility to examine what module is imported easily FUJIWARA Katsunori <foozy@lares.dti.ne.jp> parents: 25064 diff changeset	115 ('foo.bar', 'foo.bar.__init__', True)
7358b5d9991e import-checker: add utility to examine what module is imported easily FUJIWARA Katsunori <foozy@lares.dti.ne.jp> parents: 25064 diff changeset	116 >>> fromlocal('bar.bar1')
7358b5d9991e import-checker: add utility to examine what module is imported easily FUJIWARA Katsunori <foozy@lares.dti.ne.jp> parents: 25064 diff changeset	117 ('foo.bar.bar1', 'foo.bar.bar1', False)
7358b5d9991e import-checker: add utility to examine what module is imported easily FUJIWARA Katsunori <foozy@lares.dti.ne.jp> parents: 25064 diff changeset	118 >>> # absolute
7358b5d9991e import-checker: add utility to examine what module is imported easily FUJIWARA Katsunori <foozy@lares.dti.ne.jp> parents: 25064 diff changeset	119 >>> fromlocal('baz')
7358b5d9991e import-checker: add utility to examine what module is imported easily FUJIWARA Katsunori <foozy@lares.dti.ne.jp> parents: 25064 diff changeset	120 ('baz', 'baz.__init__', True)
7358b5d9991e import-checker: add utility to examine what module is imported easily FUJIWARA Katsunori <foozy@lares.dti.ne.jp> parents: 25064 diff changeset	121 >>> fromlocal('baz.baz1')
7358b5d9991e import-checker: add utility to examine what module is imported easily FUJIWARA Katsunori <foozy@lares.dti.ne.jp> parents: 25064 diff changeset	122 ('baz.baz1', 'baz.baz1', False)
7358b5d9991e import-checker: add utility to examine what module is imported easily FUJIWARA Katsunori <foozy@lares.dti.ne.jp> parents: 25064 diff changeset	123 >>> # unknown = maybe standard library
7358b5d9991e import-checker: add utility to examine what module is imported easily FUJIWARA Katsunori <foozy@lares.dti.ne.jp> parents: 25064 diff changeset	124 >>> fromlocal('os')
7358b5d9991e import-checker: add utility to examine what module is imported easily FUJIWARA Katsunori <foozy@lares.dti.ne.jp> parents: 25064 diff changeset	125 False
25701 1f88c0f6ff5a import-checker: resolve relative imports Gregory Szorc <gregory.szorc@gmail.com> parents: 25660 diff changeset	126 >>> fromlocal(None, 1)
1f88c0f6ff5a import-checker: resolve relative imports Gregory Szorc <gregory.szorc@gmail.com> parents: 25660 diff changeset	127 ('foo', 'foo.__init__', True)
1f88c0f6ff5a import-checker: resolve relative imports Gregory Szorc <gregory.szorc@gmail.com> parents: 25660 diff changeset	128 >>> fromlocal2 = fromlocalfunc('foo.xxx.yyy', localmods)
1f88c0f6ff5a import-checker: resolve relative imports Gregory Szorc <gregory.szorc@gmail.com> parents: 25660 diff changeset	129 >>> fromlocal2(None, 2)
1f88c0f6ff5a import-checker: resolve relative imports Gregory Szorc <gregory.szorc@gmail.com> parents: 25660 diff changeset	130 ('foo', 'foo.__init__', True)
25173 7358b5d9991e import-checker: add utility to examine what module is imported easily FUJIWARA Katsunori <foozy@lares.dti.ne.jp> parents: 25064 diff changeset	131 """
7358b5d9991e import-checker: add utility to examine what module is imported easily FUJIWARA Katsunori <foozy@lares.dti.ne.jp> parents: 25064 diff changeset	132 prefix = '.'.join(modulename.split('.')[:-1])
7358b5d9991e import-checker: add utility to examine what module is imported easily FUJIWARA Katsunori <foozy@lares.dti.ne.jp> parents: 25064 diff changeset	133 if prefix:
7358b5d9991e import-checker: add utility to examine what module is imported easily FUJIWARA Katsunori <foozy@lares.dti.ne.jp> parents: 25064 diff changeset	134 prefix += '.'
25701 1f88c0f6ff5a import-checker: resolve relative imports Gregory Szorc <gregory.szorc@gmail.com> parents: 25660 diff changeset	135 def fromlocal(name, level=0):
1f88c0f6ff5a import-checker: resolve relative imports Gregory Szorc <gregory.szorc@gmail.com> parents: 25660 diff changeset	136 # name is None when relative imports are used.
1f88c0f6ff5a import-checker: resolve relative imports Gregory Szorc <gregory.szorc@gmail.com> parents: 25660 diff changeset	137 if name is None:
1f88c0f6ff5a import-checker: resolve relative imports Gregory Szorc <gregory.szorc@gmail.com> parents: 25660 diff changeset	138 # If relative imports are used, level must not be absolute.
1f88c0f6ff5a import-checker: resolve relative imports Gregory Szorc <gregory.szorc@gmail.com> parents: 25660 diff changeset	139 assert level > 0
1f88c0f6ff5a import-checker: resolve relative imports Gregory Szorc <gregory.szorc@gmail.com> parents: 25660 diff changeset	140 candidates = ['.'.join(modulename.split('.')[:-level])]
1f88c0f6ff5a import-checker: resolve relative imports Gregory Szorc <gregory.szorc@gmail.com> parents: 25660 diff changeset	141 else:
1f88c0f6ff5a import-checker: resolve relative imports Gregory Szorc <gregory.szorc@gmail.com> parents: 25660 diff changeset	142 # Check relative name first.
1f88c0f6ff5a import-checker: resolve relative imports Gregory Szorc <gregory.szorc@gmail.com> parents: 25660 diff changeset	143 candidates = [prefix + name, name]
1f88c0f6ff5a import-checker: resolve relative imports Gregory Szorc <gregory.szorc@gmail.com> parents: 25660 diff changeset	144
1f88c0f6ff5a import-checker: resolve relative imports Gregory Szorc <gregory.szorc@gmail.com> parents: 25660 diff changeset	145 for n in candidates:
25173 7358b5d9991e import-checker: add utility to examine what module is imported easily FUJIWARA Katsunori <foozy@lares.dti.ne.jp> parents: 25064 diff changeset	146 if n in localmods:
7358b5d9991e import-checker: add utility to examine what module is imported easily FUJIWARA Katsunori <foozy@lares.dti.ne.jp> parents: 25064 diff changeset	147 return (n, n, False)
7358b5d9991e import-checker: add utility to examine what module is imported easily FUJIWARA Katsunori <foozy@lares.dti.ne.jp> parents: 25064 diff changeset	148 dottedpath = n + '.__init__'
7358b5d9991e import-checker: add utility to examine what module is imported easily FUJIWARA Katsunori <foozy@lares.dti.ne.jp> parents: 25064 diff changeset	149 if dottedpath in localmods:
7358b5d9991e import-checker: add utility to examine what module is imported easily FUJIWARA Katsunori <foozy@lares.dti.ne.jp> parents: 25064 diff changeset	150 return (n, dottedpath, True)
7358b5d9991e import-checker: add utility to examine what module is imported easily FUJIWARA Katsunori <foozy@lares.dti.ne.jp> parents: 25064 diff changeset	151 return False
7358b5d9991e import-checker: add utility to examine what module is imported easily FUJIWARA Katsunori <foozy@lares.dti.ne.jp> parents: 25064 diff changeset	152 return fromlocal
20036 e5d51fa51aba contrib: add an import checker Augie Fackler <raf@durin42.com> parents: diff changeset	153
e5d51fa51aba contrib: add an import checker Augie Fackler <raf@durin42.com> parents: diff changeset	154 def list_stdlib_modules():
e5d51fa51aba contrib: add an import checker Augie Fackler <raf@durin42.com> parents: diff changeset	155 """List the modules present in the stdlib.
e5d51fa51aba contrib: add an import checker Augie Fackler <raf@durin42.com> parents: diff changeset	156
e5d51fa51aba contrib: add an import checker Augie Fackler <raf@durin42.com> parents: diff changeset	157 >>> mods = set(list_stdlib_modules())
e5d51fa51aba contrib: add an import checker Augie Fackler <raf@durin42.com> parents: diff changeset	158 >>> 'BaseHTTPServer' in mods
e5d51fa51aba contrib: add an import checker Augie Fackler <raf@durin42.com> parents: diff changeset	159 True
e5d51fa51aba contrib: add an import checker Augie Fackler <raf@durin42.com> parents: diff changeset	160
e5d51fa51aba contrib: add an import checker Augie Fackler <raf@durin42.com> parents: diff changeset	161 os.path isn't really a module, so it's missing:
e5d51fa51aba contrib: add an import checker Augie Fackler <raf@durin42.com> parents: diff changeset	162
e5d51fa51aba contrib: add an import checker Augie Fackler <raf@durin42.com> parents: diff changeset	163 >>> 'os.path' in mods
e5d51fa51aba contrib: add an import checker Augie Fackler <raf@durin42.com> parents: diff changeset	164 False
e5d51fa51aba contrib: add an import checker Augie Fackler <raf@durin42.com> parents: diff changeset	165
e5d51fa51aba contrib: add an import checker Augie Fackler <raf@durin42.com> parents: diff changeset	166 sys requires special treatment, because it's baked into the
e5d51fa51aba contrib: add an import checker Augie Fackler <raf@durin42.com> parents: diff changeset	167 interpreter, but it should still appear:
e5d51fa51aba contrib: add an import checker Augie Fackler <raf@durin42.com> parents: diff changeset	168
e5d51fa51aba contrib: add an import checker Augie Fackler <raf@durin42.com> parents: diff changeset	169 >>> 'sys' in mods
e5d51fa51aba contrib: add an import checker Augie Fackler <raf@durin42.com> parents: diff changeset	170 True
e5d51fa51aba contrib: add an import checker Augie Fackler <raf@durin42.com> parents: diff changeset	171
e5d51fa51aba contrib: add an import checker Augie Fackler <raf@durin42.com> parents: diff changeset	172 >>> 'collections' in mods
e5d51fa51aba contrib: add an import checker Augie Fackler <raf@durin42.com> parents: diff changeset	173 True
e5d51fa51aba contrib: add an import checker Augie Fackler <raf@durin42.com> parents: diff changeset	174
e5d51fa51aba contrib: add an import checker Augie Fackler <raf@durin42.com> parents: diff changeset	175 >>> 'cStringIO' in mods
e5d51fa51aba contrib: add an import checker Augie Fackler <raf@durin42.com> parents: diff changeset	176 True
e5d51fa51aba contrib: add an import checker Augie Fackler <raf@durin42.com> parents: diff changeset	177 """
e5d51fa51aba contrib: add an import checker Augie Fackler <raf@durin42.com> parents: diff changeset	178 for m in sys.builtin_module_names:
e5d51fa51aba contrib: add an import checker Augie Fackler <raf@durin42.com> parents: diff changeset	179 yield m
e5d51fa51aba contrib: add an import checker Augie Fackler <raf@durin42.com> parents: diff changeset	180 # These modules only exist on windows, but we should always
e5d51fa51aba contrib: add an import checker Augie Fackler <raf@durin42.com> parents: diff changeset	181 # consider them stdlib.
e5d51fa51aba contrib: add an import checker Augie Fackler <raf@durin42.com> parents: diff changeset	182 for m in ['msvcrt', '_winreg']:
e5d51fa51aba contrib: add an import checker Augie Fackler <raf@durin42.com> parents: diff changeset	183 yield m
e5d51fa51aba contrib: add an import checker Augie Fackler <raf@durin42.com> parents: diff changeset	184 yield 'builtins' # python3 only
24669 fbdbff1b486a import-checker: force 'fcntl', 'grp', 'pwd', and 'termios' to stdlib modules Matt Harbison <matt_harbison@yahoo.com> parents: 24668 diff changeset	185 for m in 'fcntl', 'grp', 'pwd', 'termios': # Unix only
fbdbff1b486a import-checker: force 'fcntl', 'grp', 'pwd', and 'termios' to stdlib modules Matt Harbison <matt_harbison@yahoo.com> parents: 24668 diff changeset	186 yield m
28713 806d260c6f3b tests: fix builtin module test on pypy Maciej Fijalkowski <fijall@gmail.com> parents: 28704 diff changeset	187 for m in 'cPickle', 'datetime': # in Python (not C) on PyPy
806d260c6f3b tests: fix builtin module test on pypy Maciej Fijalkowski <fijall@gmail.com> parents: 28704 diff changeset	188 yield m
20197 761f2929a6ad import-checker: refactor sys.path prefix check (issue4129) Chris Jerdonek <chris.jerdonek@gmail.com> parents: 20038 diff changeset	189 stdlib_prefixes = set([sys.prefix, sys.exec_prefix])
20198 f5393a9dc4e5 import-checker: make test-module-imports.t work using virtualenv (issue4129) Chris Jerdonek <chris.jerdonek@gmail.com> parents: 20197 diff changeset	190 # We need to supplement the list of prefixes for the search to work
f5393a9dc4e5 import-checker: make test-module-imports.t work using virtualenv (issue4129) Chris Jerdonek <chris.jerdonek@gmail.com> parents: 20197 diff changeset	191 # when run from within a virtualenv.
f5393a9dc4e5 import-checker: make test-module-imports.t work using virtualenv (issue4129) Chris Jerdonek <chris.jerdonek@gmail.com> parents: 20197 diff changeset	192 for mod in (BaseHTTPServer, zlib):
f5393a9dc4e5 import-checker: make test-module-imports.t work using virtualenv (issue4129) Chris Jerdonek <chris.jerdonek@gmail.com> parents: 20197 diff changeset	193 try:
f5393a9dc4e5 import-checker: make test-module-imports.t work using virtualenv (issue4129) Chris Jerdonek <chris.jerdonek@gmail.com> parents: 20197 diff changeset	194 # Not all module objects have a __file__ attribute.
f5393a9dc4e5 import-checker: make test-module-imports.t work using virtualenv (issue4129) Chris Jerdonek <chris.jerdonek@gmail.com> parents: 20197 diff changeset	195 filename = mod.__file__
f5393a9dc4e5 import-checker: make test-module-imports.t work using virtualenv (issue4129) Chris Jerdonek <chris.jerdonek@gmail.com> parents: 20197 diff changeset	196 except AttributeError:
f5393a9dc4e5 import-checker: make test-module-imports.t work using virtualenv (issue4129) Chris Jerdonek <chris.jerdonek@gmail.com> parents: 20197 diff changeset	197 continue
f5393a9dc4e5 import-checker: make test-module-imports.t work using virtualenv (issue4129) Chris Jerdonek <chris.jerdonek@gmail.com> parents: 20197 diff changeset	198 dirname = os.path.dirname(filename)
f5393a9dc4e5 import-checker: make test-module-imports.t work using virtualenv (issue4129) Chris Jerdonek <chris.jerdonek@gmail.com> parents: 20197 diff changeset	199 for prefix in stdlib_prefixes:
f5393a9dc4e5 import-checker: make test-module-imports.t work using virtualenv (issue4129) Chris Jerdonek <chris.jerdonek@gmail.com> parents: 20197 diff changeset	200 if dirname.startswith(prefix):
f5393a9dc4e5 import-checker: make test-module-imports.t work using virtualenv (issue4129) Chris Jerdonek <chris.jerdonek@gmail.com> parents: 20197 diff changeset	201 # Then this directory is redundant.
f5393a9dc4e5 import-checker: make test-module-imports.t work using virtualenv (issue4129) Chris Jerdonek <chris.jerdonek@gmail.com> parents: 20197 diff changeset	202 break
f5393a9dc4e5 import-checker: make test-module-imports.t work using virtualenv (issue4129) Chris Jerdonek <chris.jerdonek@gmail.com> parents: 20197 diff changeset	203 else:
f5393a9dc4e5 import-checker: make test-module-imports.t work using virtualenv (issue4129) Chris Jerdonek <chris.jerdonek@gmail.com> parents: 20197 diff changeset	204 stdlib_prefixes.add(dirname)
20036 e5d51fa51aba contrib: add an import checker Augie Fackler <raf@durin42.com> parents: diff changeset	205 for libpath in sys.path:
20201 bc3b48b0f5c8 import-checker: suppress check-code about any() Augie Fackler <raf@durin42.com> parents: 20200 diff changeset	206 # We want to walk everything in sys.path that starts with
28700 35ad5bcdeb7e py24: remove check-code py24 notation timeless <timeless@mozdev.org> parents: 28400 diff changeset	207 # something in stdlib_prefixes.
35ad5bcdeb7e py24: remove check-code py24 notation timeless <timeless@mozdev.org> parents: 28400 diff changeset	208 if not any(libpath.startswith(p) for p in stdlib_prefixes):
20036 e5d51fa51aba contrib: add an import checker Augie Fackler <raf@durin42.com> parents: diff changeset	209 continue
e5d51fa51aba contrib: add an import checker Augie Fackler <raf@durin42.com> parents: diff changeset	210 for top, dirs, files in os.walk(libpath):
25733 f99c066f5f9a import-checker: recurse into subtree of sys.path only if __init__.py exists Yuya Nishihara <yuya@tcha.org> parents: 25731 diff changeset	211 for i, d in reversed(list(enumerate(dirs))):
25734 9086d0c1def3 import-checker: exclude mercurial packages installed into the system path Yuya Nishihara <yuya@tcha.org> parents: 25733 diff changeset	212 if (not os.path.exists(os.path.join(top, d, '__init__.py'))
9086d0c1def3 import-checker: exclude mercurial packages installed into the system path Yuya Nishihara <yuya@tcha.org> parents: 25733 diff changeset	213 or top == libpath and d in ('hgext', 'mercurial')):
25733 f99c066f5f9a import-checker: recurse into subtree of sys.path only if __init__.py exists Yuya Nishihara <yuya@tcha.org> parents: 25731 diff changeset	214 del dirs[i]
20036 e5d51fa51aba contrib: add an import checker Augie Fackler <raf@durin42.com> parents: diff changeset	215 for name in files:
26221 ae65b1b4cb46 import-checker: use modern .endswith for multiple suffixes Augie Fackler <augie@google.com> parents: 26166 diff changeset	216 if not name.endswith(('.py', '.so', '.pyc', '.pyo', '.pyd')):
20036 e5d51fa51aba contrib: add an import checker Augie Fackler <raf@durin42.com> parents: diff changeset	217 continue
27621 39845b064041 import-checker: list package directory as stdlib module Yuya Nishihara <yuya@tcha.org> parents: 27620 diff changeset	218 if name.startswith('__init__.py'):
39845b064041 import-checker: list package directory as stdlib module Yuya Nishihara <yuya@tcha.org> parents: 27620 diff changeset	219 full_path = top
39845b064041 import-checker: list package directory as stdlib module Yuya Nishihara <yuya@tcha.org> parents: 27620 diff changeset	220 else:
39845b064041 import-checker: list package directory as stdlib module Yuya Nishihara <yuya@tcha.org> parents: 27620 diff changeset	221 full_path = os.path.join(top, name)
20036 e5d51fa51aba contrib: add an import checker Augie Fackler <raf@durin42.com> parents: diff changeset	222 rel_path = full_path[len(libpath) + 1:]
e5d51fa51aba contrib: add an import checker Augie Fackler <raf@durin42.com> parents: diff changeset	223 mod = dotted_name_of_path(rel_path)
e5d51fa51aba contrib: add an import checker Augie Fackler <raf@durin42.com> parents: diff changeset	224 yield mod
e5d51fa51aba contrib: add an import checker Augie Fackler <raf@durin42.com> parents: diff changeset	225
e5d51fa51aba contrib: add an import checker Augie Fackler <raf@durin42.com> parents: diff changeset	226 stdlib_modules = set(list_stdlib_modules())
e5d51fa51aba contrib: add an import checker Augie Fackler <raf@durin42.com> parents: diff changeset	227
25174 86298718b01c import-checker: make imported_modules yield absolute dotted_name_of_path FUJIWARA Katsunori <foozy@lares.dti.ne.jp> parents: 25173 diff changeset	228 def imported_modules(source, modulename, localmods, ignore_nested=False):
20036 e5d51fa51aba contrib: add an import checker Augie Fackler <raf@durin42.com> parents: diff changeset	229 """Given the source of a file as a string, yield the names
e5d51fa51aba contrib: add an import checker Augie Fackler <raf@durin42.com> parents: diff changeset	230 imported by that file.
e5d51fa51aba contrib: add an import checker Augie Fackler <raf@durin42.com> parents: diff changeset	231
20037 957b43371928 import-checker: ignore nested imports Augie Fackler <raf@durin42.com> parents: 20036 diff changeset	232 Args:
957b43371928 import-checker: ignore nested imports Augie Fackler <raf@durin42.com> parents: 20036 diff changeset	233 source: The python source to examine as a string.
25174 86298718b01c import-checker: make imported_modules yield absolute dotted_name_of_path FUJIWARA Katsunori <foozy@lares.dti.ne.jp> parents: 25173 diff changeset	234 modulename: of specified python source (may have `__init__`)
86298718b01c import-checker: make imported_modules yield absolute dotted_name_of_path FUJIWARA Katsunori <foozy@lares.dti.ne.jp> parents: 25173 diff changeset	235 localmods: dict of locally defined module names (may have `__init__`)
20037 957b43371928 import-checker: ignore nested imports Augie Fackler <raf@durin42.com> parents: 20036 diff changeset	236 ignore_nested: If true, import statements that do not start in
957b43371928 import-checker: ignore nested imports Augie Fackler <raf@durin42.com> parents: 20036 diff changeset	237 column zero will be ignored.
957b43371928 import-checker: ignore nested imports Augie Fackler <raf@durin42.com> parents: 20036 diff changeset	238
957b43371928 import-checker: ignore nested imports Augie Fackler <raf@durin42.com> parents: 20036 diff changeset	239 Returns:
25174 86298718b01c import-checker: make imported_modules yield absolute dotted_name_of_path FUJIWARA Katsunori <foozy@lares.dti.ne.jp> parents: 25173 diff changeset	240 A list of absolute module names imported by the given source.
20037 957b43371928 import-checker: ignore nested imports Augie Fackler <raf@durin42.com> parents: 20036 diff changeset	241
25174 86298718b01c import-checker: make imported_modules yield absolute dotted_name_of_path FUJIWARA Katsunori <foozy@lares.dti.ne.jp> parents: 25173 diff changeset	242 >>> modulename = 'foo.xxx'
86298718b01c import-checker: make imported_modules yield absolute dotted_name_of_path FUJIWARA Katsunori <foozy@lares.dti.ne.jp> parents: 25173 diff changeset	243 >>> localmods = {'foo.__init__': True,
86298718b01c import-checker: make imported_modules yield absolute dotted_name_of_path FUJIWARA Katsunori <foozy@lares.dti.ne.jp> parents: 25173 diff changeset	244 ... 'foo.foo1': True, 'foo.foo2': True,
86298718b01c import-checker: make imported_modules yield absolute dotted_name_of_path FUJIWARA Katsunori <foozy@lares.dti.ne.jp> parents: 25173 diff changeset	245 ... 'foo.bar.__init__': True, 'foo.bar.bar1': True,
86298718b01c import-checker: make imported_modules yield absolute dotted_name_of_path FUJIWARA Katsunori <foozy@lares.dti.ne.jp> parents: 25173 diff changeset	246 ... 'baz.__init__': True, 'baz.baz1': True }
86298718b01c import-checker: make imported_modules yield absolute dotted_name_of_path FUJIWARA Katsunori <foozy@lares.dti.ne.jp> parents: 25173 diff changeset	247 >>> # standard library (= not locally defined ones)
86298718b01c import-checker: make imported_modules yield absolute dotted_name_of_path FUJIWARA Katsunori <foozy@lares.dti.ne.jp> parents: 25173 diff changeset	248 >>> sorted(imported_modules(
86298718b01c import-checker: make imported_modules yield absolute dotted_name_of_path FUJIWARA Katsunori <foozy@lares.dti.ne.jp> parents: 25173 diff changeset	249 ... 'from stdlib1 import foo, bar; import stdlib2',
86298718b01c import-checker: make imported_modules yield absolute dotted_name_of_path FUJIWARA Katsunori <foozy@lares.dti.ne.jp> parents: 25173 diff changeset	250 ... modulename, localmods))
86298718b01c import-checker: make imported_modules yield absolute dotted_name_of_path FUJIWARA Katsunori <foozy@lares.dti.ne.jp> parents: 25173 diff changeset	251 []
86298718b01c import-checker: make imported_modules yield absolute dotted_name_of_path FUJIWARA Katsunori <foozy@lares.dti.ne.jp> parents: 25173 diff changeset	252 >>> # relative importing
20037 957b43371928 import-checker: ignore nested imports Augie Fackler <raf@durin42.com> parents: 20036 diff changeset	253 >>> sorted(imported_modules(
25174 86298718b01c import-checker: make imported_modules yield absolute dotted_name_of_path FUJIWARA Katsunori <foozy@lares.dti.ne.jp> parents: 25173 diff changeset	254 ... 'import foo1; from bar import bar1',
86298718b01c import-checker: make imported_modules yield absolute dotted_name_of_path FUJIWARA Katsunori <foozy@lares.dti.ne.jp> parents: 25173 diff changeset	255 ... modulename, localmods))
26964 5abba2c92da3 import-checker: allow import of child modules from package root Yuya Nishihara <yuya@tcha.org> parents: 26956 diff changeset	256 ['foo.bar.bar1', 'foo.foo1']
25174 86298718b01c import-checker: make imported_modules yield absolute dotted_name_of_path FUJIWARA Katsunori <foozy@lares.dti.ne.jp> parents: 25173 diff changeset	257 >>> sorted(imported_modules(
86298718b01c import-checker: make imported_modules yield absolute dotted_name_of_path FUJIWARA Katsunori <foozy@lares.dti.ne.jp> parents: 25173 diff changeset	258 ... 'from bar.bar1 import name1, name2, name3',
86298718b01c import-checker: make imported_modules yield absolute dotted_name_of_path FUJIWARA Katsunori <foozy@lares.dti.ne.jp> parents: 25173 diff changeset	259 ... modulename, localmods))
86298718b01c import-checker: make imported_modules yield absolute dotted_name_of_path FUJIWARA Katsunori <foozy@lares.dti.ne.jp> parents: 25173 diff changeset	260 ['foo.bar.bar1']
86298718b01c import-checker: make imported_modules yield absolute dotted_name_of_path FUJIWARA Katsunori <foozy@lares.dti.ne.jp> parents: 25173 diff changeset	261 >>> # absolute importing
86298718b01c import-checker: make imported_modules yield absolute dotted_name_of_path FUJIWARA Katsunori <foozy@lares.dti.ne.jp> parents: 25173 diff changeset	262 >>> sorted(imported_modules(
86298718b01c import-checker: make imported_modules yield absolute dotted_name_of_path FUJIWARA Katsunori <foozy@lares.dti.ne.jp> parents: 25173 diff changeset	263 ... 'from baz import baz1, name1',
86298718b01c import-checker: make imported_modules yield absolute dotted_name_of_path FUJIWARA Katsunori <foozy@lares.dti.ne.jp> parents: 25173 diff changeset	264 ... modulename, localmods))
86298718b01c import-checker: make imported_modules yield absolute dotted_name_of_path FUJIWARA Katsunori <foozy@lares.dti.ne.jp> parents: 25173 diff changeset	265 ['baz.__init__', 'baz.baz1']
86298718b01c import-checker: make imported_modules yield absolute dotted_name_of_path FUJIWARA Katsunori <foozy@lares.dti.ne.jp> parents: 25173 diff changeset	266 >>> # mixed importing, even though it shouldn't be recommended
86298718b01c import-checker: make imported_modules yield absolute dotted_name_of_path FUJIWARA Katsunori <foozy@lares.dti.ne.jp> parents: 25173 diff changeset	267 >>> sorted(imported_modules(
86298718b01c import-checker: make imported_modules yield absolute dotted_name_of_path FUJIWARA Katsunori <foozy@lares.dti.ne.jp> parents: 25173 diff changeset	268 ... 'import stdlib, foo1, baz',
86298718b01c import-checker: make imported_modules yield absolute dotted_name_of_path FUJIWARA Katsunori <foozy@lares.dti.ne.jp> parents: 25173 diff changeset	269 ... modulename, localmods))
86298718b01c import-checker: make imported_modules yield absolute dotted_name_of_path FUJIWARA Katsunori <foozy@lares.dti.ne.jp> parents: 25173 diff changeset	270 ['baz.__init__', 'foo.foo1']
86298718b01c import-checker: make imported_modules yield absolute dotted_name_of_path FUJIWARA Katsunori <foozy@lares.dti.ne.jp> parents: 25173 diff changeset	271 >>> # ignore_nested
20037 957b43371928 import-checker: ignore nested imports Augie Fackler <raf@durin42.com> parents: 20036 diff changeset	272 >>> sorted(imported_modules(
957b43371928 import-checker: ignore nested imports Augie Fackler <raf@durin42.com> parents: 20036 diff changeset	273 ... '''import foo
957b43371928 import-checker: ignore nested imports Augie Fackler <raf@durin42.com> parents: 20036 diff changeset	274 ... def wat():
957b43371928 import-checker: ignore nested imports Augie Fackler <raf@durin42.com> parents: 20036 diff changeset	275 ... import bar
25174 86298718b01c import-checker: make imported_modules yield absolute dotted_name_of_path FUJIWARA Katsunori <foozy@lares.dti.ne.jp> parents: 25173 diff changeset	276 ... ''', modulename, localmods))
86298718b01c import-checker: make imported_modules yield absolute dotted_name_of_path FUJIWARA Katsunori <foozy@lares.dti.ne.jp> parents: 25173 diff changeset	277 ['foo.__init__', 'foo.bar.__init__']
86298718b01c import-checker: make imported_modules yield absolute dotted_name_of_path FUJIWARA Katsunori <foozy@lares.dti.ne.jp> parents: 25173 diff changeset	278 >>> sorted(imported_modules(
86298718b01c import-checker: make imported_modules yield absolute dotted_name_of_path FUJIWARA Katsunori <foozy@lares.dti.ne.jp> parents: 25173 diff changeset	279 ... '''import foo
86298718b01c import-checker: make imported_modules yield absolute dotted_name_of_path FUJIWARA Katsunori <foozy@lares.dti.ne.jp> parents: 25173 diff changeset	280 ... def wat():
86298718b01c import-checker: make imported_modules yield absolute dotted_name_of_path FUJIWARA Katsunori <foozy@lares.dti.ne.jp> parents: 25173 diff changeset	281 ... import bar
86298718b01c import-checker: make imported_modules yield absolute dotted_name_of_path FUJIWARA Katsunori <foozy@lares.dti.ne.jp> parents: 25173 diff changeset	282 ... ''', modulename, localmods, ignore_nested=True))
86298718b01c import-checker: make imported_modules yield absolute dotted_name_of_path FUJIWARA Katsunori <foozy@lares.dti.ne.jp> parents: 25173 diff changeset	283 ['foo.__init__']
20036 e5d51fa51aba contrib: add an import checker Augie Fackler <raf@durin42.com> parents: diff changeset	284 """
25174 86298718b01c import-checker: make imported_modules yield absolute dotted_name_of_path FUJIWARA Katsunori <foozy@lares.dti.ne.jp> parents: 25173 diff changeset	285 fromlocal = fromlocalfunc(modulename, localmods)
20036 e5d51fa51aba contrib: add an import checker Augie Fackler <raf@durin42.com> parents: diff changeset	286 for node in ast.walk(ast.parse(source)):
20037 957b43371928 import-checker: ignore nested imports Augie Fackler <raf@durin42.com> parents: 20036 diff changeset	287 if ignore_nested and getattr(node, 'col_offset', 0) > 0:
957b43371928 import-checker: ignore nested imports Augie Fackler <raf@durin42.com> parents: 20036 diff changeset	288 continue
20036 e5d51fa51aba contrib: add an import checker Augie Fackler <raf@durin42.com> parents: diff changeset	289 if isinstance(node, ast.Import):
e5d51fa51aba contrib: add an import checker Augie Fackler <raf@durin42.com> parents: diff changeset	290 for n in node.names:
25174 86298718b01c import-checker: make imported_modules yield absolute dotted_name_of_path FUJIWARA Katsunori <foozy@lares.dti.ne.jp> parents: 25173 diff changeset	291 found = fromlocal(n.name)
86298718b01c import-checker: make imported_modules yield absolute dotted_name_of_path FUJIWARA Katsunori <foozy@lares.dti.ne.jp> parents: 25173 diff changeset	292 if not found:
86298718b01c import-checker: make imported_modules yield absolute dotted_name_of_path FUJIWARA Katsunori <foozy@lares.dti.ne.jp> parents: 25173 diff changeset	293 # this should import standard library
86298718b01c import-checker: make imported_modules yield absolute dotted_name_of_path FUJIWARA Katsunori <foozy@lares.dti.ne.jp> parents: 25173 diff changeset	294 continue
86298718b01c import-checker: make imported_modules yield absolute dotted_name_of_path FUJIWARA Katsunori <foozy@lares.dti.ne.jp> parents: 25173 diff changeset	295 yield found[1]
20036 e5d51fa51aba contrib: add an import checker Augie Fackler <raf@durin42.com> parents: diff changeset	296 elif isinstance(node, ast.ImportFrom):
25701 1f88c0f6ff5a import-checker: resolve relative imports Gregory Szorc <gregory.szorc@gmail.com> parents: 25660 diff changeset	297 found = fromlocal(node.module, node.level)
25174 86298718b01c import-checker: make imported_modules yield absolute dotted_name_of_path FUJIWARA Katsunori <foozy@lares.dti.ne.jp> parents: 25173 diff changeset	298 if not found:
86298718b01c import-checker: make imported_modules yield absolute dotted_name_of_path FUJIWARA Katsunori <foozy@lares.dti.ne.jp> parents: 25173 diff changeset	299 # this should import standard library
86298718b01c import-checker: make imported_modules yield absolute dotted_name_of_path FUJIWARA Katsunori <foozy@lares.dti.ne.jp> parents: 25173 diff changeset	300 continue
86298718b01c import-checker: make imported_modules yield absolute dotted_name_of_path FUJIWARA Katsunori <foozy@lares.dti.ne.jp> parents: 25173 diff changeset	301
86298718b01c import-checker: make imported_modules yield absolute dotted_name_of_path FUJIWARA Katsunori <foozy@lares.dti.ne.jp> parents: 25173 diff changeset	302 absname, dottedpath, hassubmod = found
86298718b01c import-checker: make imported_modules yield absolute dotted_name_of_path FUJIWARA Katsunori <foozy@lares.dti.ne.jp> parents: 25173 diff changeset	303 if not hassubmod:
26964 5abba2c92da3 import-checker: allow import of child modules from package root Yuya Nishihara <yuya@tcha.org> parents: 26956 diff changeset	304 # "dottedpath" is not a package; must be imported
5abba2c92da3 import-checker: allow import of child modules from package root Yuya Nishihara <yuya@tcha.org> parents: 26956 diff changeset	305 yield dottedpath
25174 86298718b01c import-checker: make imported_modules yield absolute dotted_name_of_path FUJIWARA Katsunori <foozy@lares.dti.ne.jp> parents: 25173 diff changeset	306 # examination of "node.names" should be redundant
86298718b01c import-checker: make imported_modules yield absolute dotted_name_of_path FUJIWARA Katsunori <foozy@lares.dti.ne.jp> parents: 25173 diff changeset	307 # e.g.: from mercurial.node import nullid, nullrev
86298718b01c import-checker: make imported_modules yield absolute dotted_name_of_path FUJIWARA Katsunori <foozy@lares.dti.ne.jp> parents: 25173 diff changeset	308 continue
86298718b01c import-checker: make imported_modules yield absolute dotted_name_of_path FUJIWARA Katsunori <foozy@lares.dti.ne.jp> parents: 25173 diff changeset	309
26964 5abba2c92da3 import-checker: allow import of child modules from package root Yuya Nishihara <yuya@tcha.org> parents: 26956 diff changeset	310 modnotfound = False
25174 86298718b01c import-checker: make imported_modules yield absolute dotted_name_of_path FUJIWARA Katsunori <foozy@lares.dti.ne.jp> parents: 25173 diff changeset	311 prefix = absname + '.'
20036 e5d51fa51aba contrib: add an import checker Augie Fackler <raf@durin42.com> parents: diff changeset	312 for n in node.names:
25174 86298718b01c import-checker: make imported_modules yield absolute dotted_name_of_path FUJIWARA Katsunori <foozy@lares.dti.ne.jp> parents: 25173 diff changeset	313 found = fromlocal(prefix + n.name)
86298718b01c import-checker: make imported_modules yield absolute dotted_name_of_path FUJIWARA Katsunori <foozy@lares.dti.ne.jp> parents: 25173 diff changeset	314 if not found:
86298718b01c import-checker: make imported_modules yield absolute dotted_name_of_path FUJIWARA Katsunori <foozy@lares.dti.ne.jp> parents: 25173 diff changeset	315 # this should be a function or a property of "node.module"
26964 5abba2c92da3 import-checker: allow import of child modules from package root Yuya Nishihara <yuya@tcha.org> parents: 26956 diff changeset	316 modnotfound = True
25174 86298718b01c import-checker: make imported_modules yield absolute dotted_name_of_path FUJIWARA Katsunori <foozy@lares.dti.ne.jp> parents: 25173 diff changeset	317 continue
86298718b01c import-checker: make imported_modules yield absolute dotted_name_of_path FUJIWARA Katsunori <foozy@lares.dti.ne.jp> parents: 25173 diff changeset	318 yield found[1]
26964 5abba2c92da3 import-checker: allow import of child modules from package root Yuya Nishihara <yuya@tcha.org> parents: 26956 diff changeset	319 if modnotfound:
5abba2c92da3 import-checker: allow import of child modules from package root Yuya Nishihara <yuya@tcha.org> parents: 26956 diff changeset	320 # "dottedpath" is a package, but imported because of non-module
5abba2c92da3 import-checker: allow import of child modules from package root Yuya Nishihara <yuya@tcha.org> parents: 26956 diff changeset	321 # lookup
5abba2c92da3 import-checker: allow import of child modules from package root Yuya Nishihara <yuya@tcha.org> parents: 26956 diff changeset	322 yield dottedpath
20036 e5d51fa51aba contrib: add an import checker Augie Fackler <raf@durin42.com> parents: diff changeset	323
27272 69308357ecd1 import-checker: allow absolute imports of sub modules from local packages Yuya Nishihara <yuya@tcha.org> parents: 27018 diff changeset	324 def verify_import_convention(module, source, localmods):
25703 1a6a117d0b95 import-checker: establish modern import convention Gregory Szorc <gregory.szorc@gmail.com> parents: 25702 diff changeset	325 """Verify imports match our established coding convention.
1a6a117d0b95 import-checker: establish modern import convention Gregory Szorc <gregory.szorc@gmail.com> parents: 25702 diff changeset	326
1a6a117d0b95 import-checker: establish modern import convention Gregory Szorc <gregory.szorc@gmail.com> parents: 25702 diff changeset	327 We have 2 conventions: legacy and modern. The modern convention is in
1a6a117d0b95 import-checker: establish modern import convention Gregory Szorc <gregory.szorc@gmail.com> parents: 25702 diff changeset	328 effect when using absolute imports.
1a6a117d0b95 import-checker: establish modern import convention Gregory Szorc <gregory.szorc@gmail.com> parents: 25702 diff changeset	329
1a6a117d0b95 import-checker: establish modern import convention Gregory Szorc <gregory.szorc@gmail.com> parents: 25702 diff changeset	330 The legacy convention only looks for mixed imports. The modern convention
1a6a117d0b95 import-checker: establish modern import convention Gregory Szorc <gregory.szorc@gmail.com> parents: 25702 diff changeset	331 is much more thorough.
1a6a117d0b95 import-checker: establish modern import convention Gregory Szorc <gregory.szorc@gmail.com> parents: 25702 diff changeset	332 """
25702 ab2c5163900e import-checker: establish new function for verifying import conventions Gregory Szorc <gregory.szorc@gmail.com> parents: 25701 diff changeset	333 root = ast.parse(source)
25703 1a6a117d0b95 import-checker: establish modern import convention Gregory Szorc <gregory.szorc@gmail.com> parents: 25702 diff changeset	334 absolute = usingabsolute(root)
25702 ab2c5163900e import-checker: establish new function for verifying import conventions Gregory Szorc <gregory.szorc@gmail.com> parents: 25701 diff changeset	335
25703 1a6a117d0b95 import-checker: establish modern import convention Gregory Szorc <gregory.szorc@gmail.com> parents: 25702 diff changeset	336 if absolute:
27272 69308357ecd1 import-checker: allow absolute imports of sub modules from local packages Yuya Nishihara <yuya@tcha.org> parents: 27018 diff changeset	337 return verify_modern_convention(module, root, localmods)
25703 1a6a117d0b95 import-checker: establish modern import convention Gregory Szorc <gregory.szorc@gmail.com> parents: 25702 diff changeset	338 else:
1a6a117d0b95 import-checker: establish modern import convention Gregory Szorc <gregory.szorc@gmail.com> parents: 25702 diff changeset	339 return verify_stdlib_on_own_line(root)
1a6a117d0b95 import-checker: establish modern import convention Gregory Szorc <gregory.szorc@gmail.com> parents: 25702 diff changeset	340
27272 69308357ecd1 import-checker: allow absolute imports of sub modules from local packages Yuya Nishihara <yuya@tcha.org> parents: 27018 diff changeset	341 def verify_modern_convention(module, root, localmods, root_col_offset=0):
25703 1a6a117d0b95 import-checker: establish modern import convention Gregory Szorc <gregory.szorc@gmail.com> parents: 25702 diff changeset	342 """Verify a file conforms to the modern import convention rules.
1a6a117d0b95 import-checker: establish modern import convention Gregory Szorc <gregory.szorc@gmail.com> parents: 25702 diff changeset	343
1a6a117d0b95 import-checker: establish modern import convention Gregory Szorc <gregory.szorc@gmail.com> parents: 25702 diff changeset	344 The rules of the modern convention are:
1a6a117d0b95 import-checker: establish modern import convention Gregory Szorc <gregory.szorc@gmail.com> parents: 25702 diff changeset	345
1a6a117d0b95 import-checker: establish modern import convention Gregory Szorc <gregory.szorc@gmail.com> parents: 25702 diff changeset	346 * Ordering is stdlib followed by local imports. Each group is lexically
1a6a117d0b95 import-checker: establish modern import convention Gregory Szorc <gregory.szorc@gmail.com> parents: 25702 diff changeset	347 sorted.
1a6a117d0b95 import-checker: establish modern import convention Gregory Szorc <gregory.szorc@gmail.com> parents: 25702 diff changeset	348 * Importing multiple modules via "import X, Y" is not allowed: use
1a6a117d0b95 import-checker: establish modern import convention Gregory Szorc <gregory.szorc@gmail.com> parents: 25702 diff changeset	349 separate import statements.
1a6a117d0b95 import-checker: establish modern import convention Gregory Szorc <gregory.szorc@gmail.com> parents: 25702 diff changeset	350 * Importing multiple modules via "from X import ..." is allowed if using
1a6a117d0b95 import-checker: establish modern import convention Gregory Szorc <gregory.szorc@gmail.com> parents: 25702 diff changeset	351 parenthesis and one entry per line.
1a6a117d0b95 import-checker: establish modern import convention Gregory Szorc <gregory.szorc@gmail.com> parents: 25702 diff changeset	352 * Only 1 relative import statement per import level ("from .", "from ..")
1a6a117d0b95 import-checker: establish modern import convention Gregory Szorc <gregory.szorc@gmail.com> parents: 25702 diff changeset	353 is allowed.
1a6a117d0b95 import-checker: establish modern import convention Gregory Szorc <gregory.szorc@gmail.com> parents: 25702 diff changeset	354 * Relative imports from higher levels must occur before lower levels. e.g.
1a6a117d0b95 import-checker: establish modern import convention Gregory Szorc <gregory.szorc@gmail.com> parents: 25702 diff changeset	355 "from .." must be before "from .".
1a6a117d0b95 import-checker: establish modern import convention Gregory Szorc <gregory.szorc@gmail.com> parents: 25702 diff changeset	356 * Imports from peer packages should use relative import (e.g. do not
1a6a117d0b95 import-checker: establish modern import convention Gregory Szorc <gregory.szorc@gmail.com> parents: 25702 diff changeset	357 "import mercurial.foo" from a "mercurial.*" module).
1a6a117d0b95 import-checker: establish modern import convention Gregory Szorc <gregory.szorc@gmail.com> parents: 25702 diff changeset	358 * Symbols can only be imported from specific modules (see
1a6a117d0b95 import-checker: establish modern import convention Gregory Szorc <gregory.szorc@gmail.com> parents: 25702 diff changeset	359 `allowsymbolimports`). For other modules, first import the module then
1a6a117d0b95 import-checker: establish modern import convention Gregory Szorc <gregory.szorc@gmail.com> parents: 25702 diff changeset	360 assign the symbol to a module-level variable. In addition, these imports
1a6a117d0b95 import-checker: establish modern import convention Gregory Szorc <gregory.szorc@gmail.com> parents: 25702 diff changeset	361 must be performed before other relative imports. This rule only
1a6a117d0b95 import-checker: establish modern import convention Gregory Szorc <gregory.szorc@gmail.com> parents: 25702 diff changeset	362 applies to import statements outside of any blocks.
1a6a117d0b95 import-checker: establish modern import convention Gregory Szorc <gregory.szorc@gmail.com> parents: 25702 diff changeset	363 * Relative imports from the standard library are not allowed.
1a6a117d0b95 import-checker: establish modern import convention Gregory Szorc <gregory.szorc@gmail.com> parents: 25702 diff changeset	364 * Certain modules must be aliased to alternate names to avoid aliasing
1a6a117d0b95 import-checker: establish modern import convention Gregory Szorc <gregory.szorc@gmail.com> parents: 25702 diff changeset	365 and readability problems. See `requirealias`.
1a6a117d0b95 import-checker: establish modern import convention Gregory Szorc <gregory.szorc@gmail.com> parents: 25702 diff changeset	366 """
1a6a117d0b95 import-checker: establish modern import convention Gregory Szorc <gregory.szorc@gmail.com> parents: 25702 diff changeset	367 topmodule = module.split('.')[0]
27272 69308357ecd1 import-checker: allow absolute imports of sub modules from local packages Yuya Nishihara <yuya@tcha.org> parents: 27018 diff changeset	368 fromlocal = fromlocalfunc(module, localmods)
25703 1a6a117d0b95 import-checker: establish modern import convention Gregory Szorc <gregory.szorc@gmail.com> parents: 25702 diff changeset	369
1a6a117d0b95 import-checker: establish modern import convention Gregory Szorc <gregory.szorc@gmail.com> parents: 25702 diff changeset	370 # Whether a local/non-stdlib import has been performed.
28330 f3fb24e36d61 import-checker: report local with stdlib late warning timeless <timeless@mozdev.org> parents: 27621 diff changeset	371 seenlocal = None
25703 1a6a117d0b95 import-checker: establish modern import convention Gregory Szorc <gregory.szorc@gmail.com> parents: 25702 diff changeset	372 # Whether a relative, non-symbol import has been seen.
1a6a117d0b95 import-checker: establish modern import convention Gregory Szorc <gregory.szorc@gmail.com> parents: 25702 diff changeset	373 seennonsymbolrelative = False
1a6a117d0b95 import-checker: establish modern import convention Gregory Szorc <gregory.szorc@gmail.com> parents: 25702 diff changeset	374 # The last name to be imported (for sorting).
1a6a117d0b95 import-checker: establish modern import convention Gregory Szorc <gregory.szorc@gmail.com> parents: 25702 diff changeset	375 lastname = None
1a6a117d0b95 import-checker: establish modern import convention Gregory Szorc <gregory.szorc@gmail.com> parents: 25702 diff changeset	376 # Relative import levels encountered so far.
1a6a117d0b95 import-checker: establish modern import convention Gregory Szorc <gregory.szorc@gmail.com> parents: 25702 diff changeset	377 seenlevels = set()
1a6a117d0b95 import-checker: establish modern import convention Gregory Szorc <gregory.szorc@gmail.com> parents: 25702 diff changeset	378
26965 1fa66d3ad28d import-checker: reset context to verify convention in function scope Yuya Nishihara <yuya@tcha.org> parents: 26964 diff changeset	379 for node, newscope in walklocal(root):
26955 c4114e335b49 import-checker: extract function to generate a formatted warning Yuya Nishihara <yuya@tcha.org> parents: 26954 diff changeset	380 def msg(fmt, *args):
26956 4b56214ebb7a import-checker: include lineno in warning message Yuya Nishihara <yuya@tcha.org> parents: 26955 diff changeset	381 return (fmt % args, node.lineno)
26965 1fa66d3ad28d import-checker: reset context to verify convention in function scope Yuya Nishihara <yuya@tcha.org> parents: 26964 diff changeset	382 if newscope:
1fa66d3ad28d import-checker: reset context to verify convention in function scope Yuya Nishihara <yuya@tcha.org> parents: 26964 diff changeset	383 # Check for local imports in function
27272 69308357ecd1 import-checker: allow absolute imports of sub modules from local packages Yuya Nishihara <yuya@tcha.org> parents: 27018 diff changeset	384 for r in verify_modern_convention(module, node, localmods,
26965 1fa66d3ad28d import-checker: reset context to verify convention in function scope Yuya Nishihara <yuya@tcha.org> parents: 26964 diff changeset	385 node.col_offset + 4):
1fa66d3ad28d import-checker: reset context to verify convention in function scope Yuya Nishihara <yuya@tcha.org> parents: 26964 diff changeset	386 yield r
1fa66d3ad28d import-checker: reset context to verify convention in function scope Yuya Nishihara <yuya@tcha.org> parents: 26964 diff changeset	387 elif isinstance(node, ast.Import):
25703 1a6a117d0b95 import-checker: establish modern import convention Gregory Szorc <gregory.szorc@gmail.com> parents: 25702 diff changeset	388 # Disallow "import foo, bar" and require separate imports
1a6a117d0b95 import-checker: establish modern import convention Gregory Szorc <gregory.szorc@gmail.com> parents: 25702 diff changeset	389 # for each module.
1a6a117d0b95 import-checker: establish modern import convention Gregory Szorc <gregory.szorc@gmail.com> parents: 25702 diff changeset	390 if len(node.names) > 1:
26955 c4114e335b49 import-checker: extract function to generate a formatted warning Yuya Nishihara <yuya@tcha.org> parents: 26954 diff changeset	391 yield msg('multiple imported names: %s',
c4114e335b49 import-checker: extract function to generate a formatted warning Yuya Nishihara <yuya@tcha.org> parents: 26954 diff changeset	392 ', '.join(n.name for n in node.names))
25703 1a6a117d0b95 import-checker: establish modern import convention Gregory Szorc <gregory.szorc@gmail.com> parents: 25702 diff changeset	393
1a6a117d0b95 import-checker: establish modern import convention Gregory Szorc <gregory.szorc@gmail.com> parents: 25702 diff changeset	394 name = node.names[0].name
1a6a117d0b95 import-checker: establish modern import convention Gregory Szorc <gregory.szorc@gmail.com> parents: 25702 diff changeset	395 asname = node.names[0].asname
1a6a117d0b95 import-checker: establish modern import convention Gregory Szorc <gregory.szorc@gmail.com> parents: 25702 diff changeset	396
1a6a117d0b95 import-checker: establish modern import convention Gregory Szorc <gregory.szorc@gmail.com> parents: 25702 diff changeset	397 # Ignore sorting rules on imports inside blocks.
26965 1fa66d3ad28d import-checker: reset context to verify convention in function scope Yuya Nishihara <yuya@tcha.org> parents: 26964 diff changeset	398 if node.col_offset == root_col_offset:
25703 1a6a117d0b95 import-checker: establish modern import convention Gregory Szorc <gregory.szorc@gmail.com> parents: 25702 diff changeset	399 if lastname and name < lastname:
26955 c4114e335b49 import-checker: extract function to generate a formatted warning Yuya Nishihara <yuya@tcha.org> parents: 26954 diff changeset	400 yield msg('imports not lexically sorted: %s < %s',
c4114e335b49 import-checker: extract function to generate a formatted warning Yuya Nishihara <yuya@tcha.org> parents: 26954 diff changeset	401 name, lastname)
25703 1a6a117d0b95 import-checker: establish modern import convention Gregory Szorc <gregory.szorc@gmail.com> parents: 25702 diff changeset	402
1a6a117d0b95 import-checker: establish modern import convention Gregory Szorc <gregory.szorc@gmail.com> parents: 25702 diff changeset	403 lastname = name
1a6a117d0b95 import-checker: establish modern import convention Gregory Szorc <gregory.szorc@gmail.com> parents: 25702 diff changeset	404
1a6a117d0b95 import-checker: establish modern import convention Gregory Szorc <gregory.szorc@gmail.com> parents: 25702 diff changeset	405 # stdlib imports should be before local imports.
1a6a117d0b95 import-checker: establish modern import convention Gregory Szorc <gregory.szorc@gmail.com> parents: 25702 diff changeset	406 stdlib = name in stdlib_modules
26965 1fa66d3ad28d import-checker: reset context to verify convention in function scope Yuya Nishihara <yuya@tcha.org> parents: 26964 diff changeset	407 if stdlib and seenlocal and node.col_offset == root_col_offset:
28330 f3fb24e36d61 import-checker: report local with stdlib late warning timeless <timeless@mozdev.org> parents: 27621 diff changeset	408 yield msg('stdlib import "%s" follows local import: %s',
f3fb24e36d61 import-checker: report local with stdlib late warning timeless <timeless@mozdev.org> parents: 27621 diff changeset	409 name, seenlocal)
25703 1a6a117d0b95 import-checker: establish modern import convention Gregory Szorc <gregory.szorc@gmail.com> parents: 25702 diff changeset	410
1a6a117d0b95 import-checker: establish modern import convention Gregory Szorc <gregory.szorc@gmail.com> parents: 25702 diff changeset	411 if not stdlib:
28330 f3fb24e36d61 import-checker: report local with stdlib late warning timeless <timeless@mozdev.org> parents: 27621 diff changeset	412 seenlocal = name
25703 1a6a117d0b95 import-checker: establish modern import convention Gregory Szorc <gregory.szorc@gmail.com> parents: 25702 diff changeset	413
1a6a117d0b95 import-checker: establish modern import convention Gregory Szorc <gregory.szorc@gmail.com> parents: 25702 diff changeset	414 # Import of sibling modules should use relative imports.
1a6a117d0b95 import-checker: establish modern import convention Gregory Szorc <gregory.szorc@gmail.com> parents: 25702 diff changeset	415 topname = name.split('.')[0]
1a6a117d0b95 import-checker: establish modern import convention Gregory Szorc <gregory.szorc@gmail.com> parents: 25702 diff changeset	416 if topname == topmodule:
26955 c4114e335b49 import-checker: extract function to generate a formatted warning Yuya Nishihara <yuya@tcha.org> parents: 26954 diff changeset	417 yield msg('import should be relative: %s', name)
25703 1a6a117d0b95 import-checker: establish modern import convention Gregory Szorc <gregory.szorc@gmail.com> parents: 25702 diff changeset	418
1a6a117d0b95 import-checker: establish modern import convention Gregory Szorc <gregory.szorc@gmail.com> parents: 25702 diff changeset	419 if name in requirealias and asname != requirealias[name]:
26955 c4114e335b49 import-checker: extract function to generate a formatted warning Yuya Nishihara <yuya@tcha.org> parents: 26954 diff changeset	420 yield msg('%s module must be "as" aliased to %s',
c4114e335b49 import-checker: extract function to generate a formatted warning Yuya Nishihara <yuya@tcha.org> parents: 26954 diff changeset	421 name, requirealias[name])
25703 1a6a117d0b95 import-checker: establish modern import convention Gregory Szorc <gregory.szorc@gmail.com> parents: 25702 diff changeset	422
1a6a117d0b95 import-checker: establish modern import convention Gregory Szorc <gregory.szorc@gmail.com> parents: 25702 diff changeset	423 elif isinstance(node, ast.ImportFrom):
1a6a117d0b95 import-checker: establish modern import convention Gregory Szorc <gregory.szorc@gmail.com> parents: 25702 diff changeset	424 # Resolve the full imported module name.
1a6a117d0b95 import-checker: establish modern import convention Gregory Szorc <gregory.szorc@gmail.com> parents: 25702 diff changeset	425 if node.level > 0:
1a6a117d0b95 import-checker: establish modern import convention Gregory Szorc <gregory.szorc@gmail.com> parents: 25702 diff changeset	426 fullname = '.'.join(module.split('.')[:-node.level])
1a6a117d0b95 import-checker: establish modern import convention Gregory Szorc <gregory.szorc@gmail.com> parents: 25702 diff changeset	427 if node.module:
1a6a117d0b95 import-checker: establish modern import convention Gregory Szorc <gregory.szorc@gmail.com> parents: 25702 diff changeset	428 fullname += '.%s' % node.module
1a6a117d0b95 import-checker: establish modern import convention Gregory Szorc <gregory.szorc@gmail.com> parents: 25702 diff changeset	429 else:
1a6a117d0b95 import-checker: establish modern import convention Gregory Szorc <gregory.szorc@gmail.com> parents: 25702 diff changeset	430 assert node.module
1a6a117d0b95 import-checker: establish modern import convention Gregory Szorc <gregory.szorc@gmail.com> parents: 25702 diff changeset	431 fullname = node.module
1a6a117d0b95 import-checker: establish modern import convention Gregory Szorc <gregory.szorc@gmail.com> parents: 25702 diff changeset	432
1a6a117d0b95 import-checker: establish modern import convention Gregory Szorc <gregory.szorc@gmail.com> parents: 25702 diff changeset	433 topname = fullname.split('.')[0]
1a6a117d0b95 import-checker: establish modern import convention Gregory Szorc <gregory.szorc@gmail.com> parents: 25702 diff changeset	434 if topname == topmodule:
26955 c4114e335b49 import-checker: extract function to generate a formatted warning Yuya Nishihara <yuya@tcha.org> parents: 26954 diff changeset	435 yield msg('import should be relative: %s', fullname)
25703 1a6a117d0b95 import-checker: establish modern import convention Gregory Szorc <gregory.szorc@gmail.com> parents: 25702 diff changeset	436
1a6a117d0b95 import-checker: establish modern import convention Gregory Szorc <gregory.szorc@gmail.com> parents: 25702 diff changeset	437 # __future__ is special since it needs to come first and use
1a6a117d0b95 import-checker: establish modern import convention Gregory Szorc <gregory.szorc@gmail.com> parents: 25702 diff changeset	438 # symbol import.
1a6a117d0b95 import-checker: establish modern import convention Gregory Szorc <gregory.szorc@gmail.com> parents: 25702 diff changeset	439 if fullname != '__future__':
1a6a117d0b95 import-checker: establish modern import convention Gregory Szorc <gregory.szorc@gmail.com> parents: 25702 diff changeset	440 if not fullname or fullname in stdlib_modules:
26955 c4114e335b49 import-checker: extract function to generate a formatted warning Yuya Nishihara <yuya@tcha.org> parents: 26954 diff changeset	441 yield msg('relative import of stdlib module')
25703 1a6a117d0b95 import-checker: establish modern import convention Gregory Szorc <gregory.szorc@gmail.com> parents: 25702 diff changeset	442 else:
28330 f3fb24e36d61 import-checker: report local with stdlib late warning timeless <timeless@mozdev.org> parents: 27621 diff changeset	443 seenlocal = fullname
25703 1a6a117d0b95 import-checker: establish modern import convention Gregory Szorc <gregory.szorc@gmail.com> parents: 25702 diff changeset	444
1a6a117d0b95 import-checker: establish modern import convention Gregory Szorc <gregory.szorc@gmail.com> parents: 25702 diff changeset	445 # Direct symbol import is only allowed from certain modules and
1a6a117d0b95 import-checker: establish modern import convention Gregory Szorc <gregory.szorc@gmail.com> parents: 25702 diff changeset	446 # must occur before non-symbol imports.
26965 1fa66d3ad28d import-checker: reset context to verify convention in function scope Yuya Nishihara <yuya@tcha.org> parents: 26964 diff changeset	447 if node.module and node.col_offset == root_col_offset:
27272 69308357ecd1 import-checker: allow absolute imports of sub modules from local packages Yuya Nishihara <yuya@tcha.org> parents: 27018 diff changeset	448 found = fromlocal(node.module, node.level)
69308357ecd1 import-checker: allow absolute imports of sub modules from local packages Yuya Nishihara <yuya@tcha.org> parents: 27018 diff changeset	449 if found and found[2]: # node.module is a package
69308357ecd1 import-checker: allow absolute imports of sub modules from local packages Yuya Nishihara <yuya@tcha.org> parents: 27018 diff changeset	450 prefix = found[0] + '.'
69308357ecd1 import-checker: allow absolute imports of sub modules from local packages Yuya Nishihara <yuya@tcha.org> parents: 27018 diff changeset	451 symbols = [n.name for n in node.names
69308357ecd1 import-checker: allow absolute imports of sub modules from local packages Yuya Nishihara <yuya@tcha.org> parents: 27018 diff changeset	452 if not fromlocal(prefix + n.name)]
69308357ecd1 import-checker: allow absolute imports of sub modules from local packages Yuya Nishihara <yuya@tcha.org> parents: 27018 diff changeset	453 else:
69308357ecd1 import-checker: allow absolute imports of sub modules from local packages Yuya Nishihara <yuya@tcha.org> parents: 27018 diff changeset	454 symbols = [n.name for n in node.names]
69308357ecd1 import-checker: allow absolute imports of sub modules from local packages Yuya Nishihara <yuya@tcha.org> parents: 27018 diff changeset	455
69308357ecd1 import-checker: allow absolute imports of sub modules from local packages Yuya Nishihara <yuya@tcha.org> parents: 27018 diff changeset	456 if symbols and fullname not in allowsymbolimports:
27273 5d5b98346fc2 import-checker: tell which symbol causes "direct symbol import" Yuya Nishihara <yuya@tcha.org> parents: 27272 diff changeset	457 yield msg('direct symbol import %s from %s',
5d5b98346fc2 import-checker: tell which symbol causes "direct symbol import" Yuya Nishihara <yuya@tcha.org> parents: 27272 diff changeset	458 ', '.join(symbols), fullname)
25703 1a6a117d0b95 import-checker: establish modern import convention Gregory Szorc <gregory.szorc@gmail.com> parents: 25702 diff changeset	459
27272 69308357ecd1 import-checker: allow absolute imports of sub modules from local packages Yuya Nishihara <yuya@tcha.org> parents: 27018 diff changeset	460 if symbols and seennonsymbolrelative:
26955 c4114e335b49 import-checker: extract function to generate a formatted warning Yuya Nishihara <yuya@tcha.org> parents: 26954 diff changeset	461 yield msg('symbol import follows non-symbol import: %s',
c4114e335b49 import-checker: extract function to generate a formatted warning Yuya Nishihara <yuya@tcha.org> parents: 26954 diff changeset	462 fullname)
25703 1a6a117d0b95 import-checker: establish modern import convention Gregory Szorc <gregory.szorc@gmail.com> parents: 25702 diff changeset	463
1a6a117d0b95 import-checker: establish modern import convention Gregory Szorc <gregory.szorc@gmail.com> parents: 25702 diff changeset	464 if not node.module:
1a6a117d0b95 import-checker: establish modern import convention Gregory Szorc <gregory.szorc@gmail.com> parents: 25702 diff changeset	465 assert node.level
1a6a117d0b95 import-checker: establish modern import convention Gregory Szorc <gregory.szorc@gmail.com> parents: 25702 diff changeset	466 seennonsymbolrelative = True
1a6a117d0b95 import-checker: establish modern import convention Gregory Szorc <gregory.szorc@gmail.com> parents: 25702 diff changeset	467
1a6a117d0b95 import-checker: establish modern import convention Gregory Szorc <gregory.szorc@gmail.com> parents: 25702 diff changeset	468 # Only allow 1 group per level.
26965 1fa66d3ad28d import-checker: reset context to verify convention in function scope Yuya Nishihara <yuya@tcha.org> parents: 26964 diff changeset	469 if (node.level in seenlevels
1fa66d3ad28d import-checker: reset context to verify convention in function scope Yuya Nishihara <yuya@tcha.org> parents: 26964 diff changeset	470 and node.col_offset == root_col_offset):
26955 c4114e335b49 import-checker: extract function to generate a formatted warning Yuya Nishihara <yuya@tcha.org> parents: 26954 diff changeset	471 yield msg('multiple "from %s import" statements',
c4114e335b49 import-checker: extract function to generate a formatted warning Yuya Nishihara <yuya@tcha.org> parents: 26954 diff changeset	472 '.' * node.level)
25703 1a6a117d0b95 import-checker: establish modern import convention Gregory Szorc <gregory.szorc@gmail.com> parents: 25702 diff changeset	473
1a6a117d0b95 import-checker: establish modern import convention Gregory Szorc <gregory.szorc@gmail.com> parents: 25702 diff changeset	474 # Higher-level groups come before lower-level groups.
1a6a117d0b95 import-checker: establish modern import convention Gregory Szorc <gregory.szorc@gmail.com> parents: 25702 diff changeset	475 if any(node.level > l for l in seenlevels):
26955 c4114e335b49 import-checker: extract function to generate a formatted warning Yuya Nishihara <yuya@tcha.org> parents: 26954 diff changeset	476 yield msg('higher-level import should come first: %s',
c4114e335b49 import-checker: extract function to generate a formatted warning Yuya Nishihara <yuya@tcha.org> parents: 26954 diff changeset	477 fullname)
25703 1a6a117d0b95 import-checker: establish modern import convention Gregory Szorc <gregory.szorc@gmail.com> parents: 25702 diff changeset	478
1a6a117d0b95 import-checker: establish modern import convention Gregory Szorc <gregory.szorc@gmail.com> parents: 25702 diff changeset	479 seenlevels.add(node.level)
1a6a117d0b95 import-checker: establish modern import convention Gregory Szorc <gregory.szorc@gmail.com> parents: 25702 diff changeset	480
1a6a117d0b95 import-checker: establish modern import convention Gregory Szorc <gregory.szorc@gmail.com> parents: 25702 diff changeset	481 # Entries in "from .X import ( ... )" lists must be lexically
1a6a117d0b95 import-checker: establish modern import convention Gregory Szorc <gregory.szorc@gmail.com> parents: 25702 diff changeset	482 # sorted.
1a6a117d0b95 import-checker: establish modern import convention Gregory Szorc <gregory.szorc@gmail.com> parents: 25702 diff changeset	483 lastentryname = None
1a6a117d0b95 import-checker: establish modern import convention Gregory Szorc <gregory.szorc@gmail.com> parents: 25702 diff changeset	484
1a6a117d0b95 import-checker: establish modern import convention Gregory Szorc <gregory.szorc@gmail.com> parents: 25702 diff changeset	485 for n in node.names:
1a6a117d0b95 import-checker: establish modern import convention Gregory Szorc <gregory.szorc@gmail.com> parents: 25702 diff changeset	486 if lastentryname and n.name < lastentryname:
26955 c4114e335b49 import-checker: extract function to generate a formatted warning Yuya Nishihara <yuya@tcha.org> parents: 26954 diff changeset	487 yield msg('imports from %s not lexically sorted: %s < %s',
c4114e335b49 import-checker: extract function to generate a formatted warning Yuya Nishihara <yuya@tcha.org> parents: 26954 diff changeset	488 fullname, n.name, lastentryname)
25703 1a6a117d0b95 import-checker: establish modern import convention Gregory Szorc <gregory.szorc@gmail.com> parents: 25702 diff changeset	489
1a6a117d0b95 import-checker: establish modern import convention Gregory Szorc <gregory.szorc@gmail.com> parents: 25702 diff changeset	490 lastentryname = n.name
1a6a117d0b95 import-checker: establish modern import convention Gregory Szorc <gregory.szorc@gmail.com> parents: 25702 diff changeset	491
1a6a117d0b95 import-checker: establish modern import convention Gregory Szorc <gregory.szorc@gmail.com> parents: 25702 diff changeset	492 if n.name in requirealias and n.asname != requirealias[n.name]:
26955 c4114e335b49 import-checker: extract function to generate a formatted warning Yuya Nishihara <yuya@tcha.org> parents: 26954 diff changeset	493 yield msg('%s from %s must be "as" aliased to %s',
c4114e335b49 import-checker: extract function to generate a formatted warning Yuya Nishihara <yuya@tcha.org> parents: 26954 diff changeset	494 n.name, fullname, requirealias[n.name])
25702 ab2c5163900e import-checker: establish new function for verifying import conventions Gregory Szorc <gregory.szorc@gmail.com> parents: 25701 diff changeset	495
ab2c5163900e import-checker: establish new function for verifying import conventions Gregory Szorc <gregory.szorc@gmail.com> parents: 25701 diff changeset	496 def verify_stdlib_on_own_line(root):
20036 e5d51fa51aba contrib: add an import checker Augie Fackler <raf@durin42.com> parents: diff changeset	497 """Given some python source, verify that stdlib imports are done
e5d51fa51aba contrib: add an import checker Augie Fackler <raf@durin42.com> parents: diff changeset	498 in separate statements from relative local module imports.
e5d51fa51aba contrib: add an import checker Augie Fackler <raf@durin42.com> parents: diff changeset	499
25702 ab2c5163900e import-checker: establish new function for verifying import conventions Gregory Szorc <gregory.szorc@gmail.com> parents: 25701 diff changeset	500 >>> list(verify_stdlib_on_own_line(ast.parse('import sys, foo')))
26956 4b56214ebb7a import-checker: include lineno in warning message Yuya Nishihara <yuya@tcha.org> parents: 26955 diff changeset	501 [('mixed imports\\n stdlib: sys\\n relative: foo', 1)]
25702 ab2c5163900e import-checker: establish new function for verifying import conventions Gregory Szorc <gregory.szorc@gmail.com> parents: 25701 diff changeset	502 >>> list(verify_stdlib_on_own_line(ast.parse('import sys, os')))
20036 e5d51fa51aba contrib: add an import checker Augie Fackler <raf@durin42.com> parents: diff changeset	503 []
25702 ab2c5163900e import-checker: establish new function for verifying import conventions Gregory Szorc <gregory.szorc@gmail.com> parents: 25701 diff changeset	504 >>> list(verify_stdlib_on_own_line(ast.parse('import foo, bar')))
20036 e5d51fa51aba contrib: add an import checker Augie Fackler <raf@durin42.com> parents: diff changeset	505 []
e5d51fa51aba contrib: add an import checker Augie Fackler <raf@durin42.com> parents: diff changeset	506 """
25702 ab2c5163900e import-checker: establish new function for verifying import conventions Gregory Szorc <gregory.szorc@gmail.com> parents: 25701 diff changeset	507 for node in ast.walk(root):
20036 e5d51fa51aba contrib: add an import checker Augie Fackler <raf@durin42.com> parents: diff changeset	508 if isinstance(node, ast.Import):
20386 a05d31b040d7 import-checker: show stdlib and relative imports separately Mads Kiilerich <madski@unity3d.com> parents: 20383 diff changeset	509 from_stdlib = {False: [], True: []}
20036 e5d51fa51aba contrib: add an import checker Augie Fackler <raf@durin42.com> parents: diff changeset	510 for n in node.names:
20386 a05d31b040d7 import-checker: show stdlib and relative imports separately Mads Kiilerich <madski@unity3d.com> parents: 20383 diff changeset	511 from_stdlib[n.name in stdlib_modules].append(n.name)
a05d31b040d7 import-checker: show stdlib and relative imports separately Mads Kiilerich <madski@unity3d.com> parents: 20383 diff changeset	512 if from_stdlib[True] and from_stdlib[False]:
a05d31b040d7 import-checker: show stdlib and relative imports separately Mads Kiilerich <madski@unity3d.com> parents: 20383 diff changeset	513 yield ('mixed imports\n stdlib: %s\n relative: %s' %
a05d31b040d7 import-checker: show stdlib and relative imports separately Mads Kiilerich <madski@unity3d.com> parents: 20383 diff changeset	514 (', '.join(sorted(from_stdlib[True])),
26956 4b56214ebb7a import-checker: include lineno in warning message Yuya Nishihara <yuya@tcha.org> parents: 26955 diff changeset	515 ', '.join(sorted(from_stdlib[False]))), node.lineno)
20036 e5d51fa51aba contrib: add an import checker Augie Fackler <raf@durin42.com> parents: diff changeset	516
e5d51fa51aba contrib: add an import checker Augie Fackler <raf@durin42.com> parents: diff changeset	517 class CircularImport(Exception):
e5d51fa51aba contrib: add an import checker Augie Fackler <raf@durin42.com> parents: diff changeset	518 pass
e5d51fa51aba contrib: add an import checker Augie Fackler <raf@durin42.com> parents: diff changeset	519
24490 fb4639d5268e import-checker: make search algorithm non-recursive breadth-first Matt Mackall <mpm@selenic.com> parents: 24489 diff changeset	520 def checkmod(mod, imports):
fb4639d5268e import-checker: make search algorithm non-recursive breadth-first Matt Mackall <mpm@selenic.com> parents: 24489 diff changeset	521 shortest = {}
fb4639d5268e import-checker: make search algorithm non-recursive breadth-first Matt Mackall <mpm@selenic.com> parents: 24489 diff changeset	522 visit = [[mod]]
fb4639d5268e import-checker: make search algorithm non-recursive breadth-first Matt Mackall <mpm@selenic.com> parents: 24489 diff changeset	523 while visit:
fb4639d5268e import-checker: make search algorithm non-recursive breadth-first Matt Mackall <mpm@selenic.com> parents: 24489 diff changeset	524 path = visit.pop(0)
fb4639d5268e import-checker: make search algorithm non-recursive breadth-first Matt Mackall <mpm@selenic.com> parents: 24489 diff changeset	525 for i in sorted(imports.get(path[-1], [])):
fb4639d5268e import-checker: make search algorithm non-recursive breadth-first Matt Mackall <mpm@selenic.com> parents: 24489 diff changeset	526 if len(path) < shortest.get(i, 1000):
fb4639d5268e import-checker: make search algorithm non-recursive breadth-first Matt Mackall <mpm@selenic.com> parents: 24489 diff changeset	527 shortest[i] = len(path)
fb4639d5268e import-checker: make search algorithm non-recursive breadth-first Matt Mackall <mpm@selenic.com> parents: 24489 diff changeset	528 if i in path:
fb4639d5268e import-checker: make search algorithm non-recursive breadth-first Matt Mackall <mpm@selenic.com> parents: 24489 diff changeset	529 if i == path[0]:
fb4639d5268e import-checker: make search algorithm non-recursive breadth-first Matt Mackall <mpm@selenic.com> parents: 24489 diff changeset	530 raise CircularImport(path)
fb4639d5268e import-checker: make search algorithm non-recursive breadth-first Matt Mackall <mpm@selenic.com> parents: 24489 diff changeset	531 continue
fb4639d5268e import-checker: make search algorithm non-recursive breadth-first Matt Mackall <mpm@selenic.com> parents: 24489 diff changeset	532 visit.append(path + [i])
20036 e5d51fa51aba contrib: add an import checker Augie Fackler <raf@durin42.com> parents: diff changeset	533
20038 c65a6937b828 import-checker: try a little harder to show fewer cycles Augie Fackler <raf@durin42.com> parents: 20037 diff changeset	534 def rotatecycle(cycle):
c65a6937b828 import-checker: try a little harder to show fewer cycles Augie Fackler <raf@durin42.com> parents: 20037 diff changeset	535 """arrange a cycle so that the lexicographically first module listed first
c65a6937b828 import-checker: try a little harder to show fewer cycles Augie Fackler <raf@durin42.com> parents: 20037 diff changeset	536
24488 4b3fc46097f7 import-checker: drop duplicate element from cycle Matt Mackall <mpm@selenic.com> parents: 24487 diff changeset	537 >>> rotatecycle(['foo', 'bar'])
20038 c65a6937b828 import-checker: try a little harder to show fewer cycles Augie Fackler <raf@durin42.com> parents: 20037 diff changeset	538 ['bar', 'foo', 'bar']
c65a6937b828 import-checker: try a little harder to show fewer cycles Augie Fackler <raf@durin42.com> parents: 20037 diff changeset	539 """
c65a6937b828 import-checker: try a little harder to show fewer cycles Augie Fackler <raf@durin42.com> parents: 20037 diff changeset	540 lowest = min(cycle)
c65a6937b828 import-checker: try a little harder to show fewer cycles Augie Fackler <raf@durin42.com> parents: 20037 diff changeset	541 idx = cycle.index(lowest)
24488 4b3fc46097f7 import-checker: drop duplicate element from cycle Matt Mackall <mpm@selenic.com> parents: 24487 diff changeset	542 return cycle[idx:] + cycle[:idx] + [lowest]
20036 e5d51fa51aba contrib: add an import checker Augie Fackler <raf@durin42.com> parents: diff changeset	543
e5d51fa51aba contrib: add an import checker Augie Fackler <raf@durin42.com> parents: diff changeset	544 def find_cycles(imports):
e5d51fa51aba contrib: add an import checker Augie Fackler <raf@durin42.com> parents: diff changeset	545 """Find cycles in an already-loaded import graph.
e5d51fa51aba contrib: add an import checker Augie Fackler <raf@durin42.com> parents: diff changeset	546
25175 10e6c4b7121b import-checker: don't treat modules as relative one if not found FUJIWARA Katsunori <foozy@lares.dti.ne.jp> parents: 25174 diff changeset	547 All module names recorded in `imports` should be absolute one.
10e6c4b7121b import-checker: don't treat modules as relative one if not found FUJIWARA Katsunori <foozy@lares.dti.ne.jp> parents: 25174 diff changeset	548
28703 a274c4f9087a py3: use print_function in import-checker timeless <timeless@mozdev.org> parents: 28702 diff changeset	549 >>> from __future__ import print_function
25175 10e6c4b7121b import-checker: don't treat modules as relative one if not found FUJIWARA Katsunori <foozy@lares.dti.ne.jp> parents: 25174 diff changeset	550 >>> imports = {'top.foo': ['top.bar', 'os.path', 'top.qux'],
10e6c4b7121b import-checker: don't treat modules as relative one if not found FUJIWARA Katsunori <foozy@lares.dti.ne.jp> parents: 25174 diff changeset	551 ... 'top.bar': ['top.baz', 'sys'],
10e6c4b7121b import-checker: don't treat modules as relative one if not found FUJIWARA Katsunori <foozy@lares.dti.ne.jp> parents: 25174 diff changeset	552 ... 'top.baz': ['top.foo'],
10e6c4b7121b import-checker: don't treat modules as relative one if not found FUJIWARA Katsunori <foozy@lares.dti.ne.jp> parents: 25174 diff changeset	553 ... 'top.qux': ['top.foo']}
28703 a274c4f9087a py3: use print_function in import-checker timeless <timeless@mozdev.org> parents: 28702 diff changeset	554 >>> print('\\n'.join(sorted(find_cycles(imports))))
24487 642d245ff537 import-checker: fix rotatecycle Matt Mackall <mpm@selenic.com> parents: 22975 diff changeset	555 top.bar -> top.baz -> top.foo -> top.bar
642d245ff537 import-checker: fix rotatecycle Matt Mackall <mpm@selenic.com> parents: 22975 diff changeset	556 top.foo -> top.qux -> top.foo
20036 e5d51fa51aba contrib: add an import checker Augie Fackler <raf@durin42.com> parents: diff changeset	557 """
24491 784b278b349c import-checker: rotatecycle is actually the canonical cycle key Matt Mackall <mpm@selenic.com> parents: 24490 diff changeset	558 cycles = set()
28704 1fa6fdb72275 py3: handle iter/iterkeys+iteritems python3 divergence in import-checker timeless <timeless@mozdev.org> parents: 28703 diff changeset	559 for mod in sorted(imports.keys()):
20036 e5d51fa51aba contrib: add an import checker Augie Fackler <raf@durin42.com> parents: diff changeset	560 try:
24490 fb4639d5268e import-checker: make search algorithm non-recursive breadth-first Matt Mackall <mpm@selenic.com> parents: 24489 diff changeset	561 checkmod(mod, imports)
25660 328739ea70c3 global: mass rewrite to use modern exception syntax Gregory Szorc <gregory.szorc@gmail.com> parents: 25175 diff changeset	562 except CircularImport as e:
20036 e5d51fa51aba contrib: add an import checker Augie Fackler <raf@durin42.com> parents: diff changeset	563 cycle = e.args[0]
24491 784b278b349c import-checker: rotatecycle is actually the canonical cycle key Matt Mackall <mpm@selenic.com> parents: 24490 diff changeset	564 cycles.add(" -> ".join(rotatecycle(cycle)))
784b278b349c import-checker: rotatecycle is actually the canonical cycle key Matt Mackall <mpm@selenic.com> parents: 24490 diff changeset	565 return cycles
20036 e5d51fa51aba contrib: add an import checker Augie Fackler <raf@durin42.com> parents: diff changeset	566
e5d51fa51aba contrib: add an import checker Augie Fackler <raf@durin42.com> parents: diff changeset	567 def _cycle_sortkey(c):
e5d51fa51aba contrib: add an import checker Augie Fackler <raf@durin42.com> parents: diff changeset	568 return len(c), c
e5d51fa51aba contrib: add an import checker Augie Fackler <raf@durin42.com> parents: diff changeset	569
e5d51fa51aba contrib: add an import checker Augie Fackler <raf@durin42.com> parents: diff changeset	570 def main(argv):
25063 723e364488f4 import-checker: add xargs like mode FUJIWARA Katsunori <foozy@lares.dti.ne.jp> parents: 24669 diff changeset	571 if len(argv) < 2 or (argv[1] == '-' and len(argv) > 2):
28703 a274c4f9087a py3: use print_function in import-checker timeless <timeless@mozdev.org> parents: 28702 diff changeset	572 print('Usage: %s {-\|file [file] [file] ...}')
20036 e5d51fa51aba contrib: add an import checker Augie Fackler <raf@durin42.com> parents: diff changeset	573 return 1
25063 723e364488f4 import-checker: add xargs like mode FUJIWARA Katsunori <foozy@lares.dti.ne.jp> parents: 24669 diff changeset	574 if argv[1] == '-':
723e364488f4 import-checker: add xargs like mode FUJIWARA Katsunori <foozy@lares.dti.ne.jp> parents: 24669 diff changeset	575 argv = argv[:1]
723e364488f4 import-checker: add xargs like mode FUJIWARA Katsunori <foozy@lares.dti.ne.jp> parents: 24669 diff changeset	576 argv.extend(l.rstrip() for l in sys.stdin.readlines())
25064 3bbbadf69d0a import-checker: loop to get list of locally defined modules at first FUJIWARA Katsunori <foozy@lares.dti.ne.jp> parents: 25063 diff changeset	577 localmods = {}
20036 e5d51fa51aba contrib: add an import checker Augie Fackler <raf@durin42.com> parents: diff changeset	578 used_imports = {}
e5d51fa51aba contrib: add an import checker Augie Fackler <raf@durin42.com> parents: diff changeset	579 any_errors = False
e5d51fa51aba contrib: add an import checker Augie Fackler <raf@durin42.com> parents: diff changeset	580 for source_path in argv[1:]:
25064 3bbbadf69d0a import-checker: loop to get list of locally defined modules at first FUJIWARA Katsunori <foozy@lares.dti.ne.jp> parents: 25063 diff changeset	581 modname = dotted_name_of_path(source_path, trimpure=True)
3bbbadf69d0a import-checker: loop to get list of locally defined modules at first FUJIWARA Katsunori <foozy@lares.dti.ne.jp> parents: 25063 diff changeset	582 localmods[modname] = source_path
28704 1fa6fdb72275 py3: handle iter/iterkeys+iteritems python3 divergence in import-checker timeless <timeless@mozdev.org> parents: 28703 diff changeset	583 for modname, source_path in sorted(localmods.items()):
20036 e5d51fa51aba contrib: add an import checker Augie Fackler <raf@durin42.com> parents: diff changeset	584 f = open(source_path)
e5d51fa51aba contrib: add an import checker Augie Fackler <raf@durin42.com> parents: diff changeset	585 src = f.read()
20037 957b43371928 import-checker: ignore nested imports Augie Fackler <raf@durin42.com> parents: 20036 diff changeset	586 used_imports[modname] = sorted(
25174 86298718b01c import-checker: make imported_modules yield absolute dotted_name_of_path FUJIWARA Katsunori <foozy@lares.dti.ne.jp> parents: 25173 diff changeset	587 imported_modules(src, modname, localmods, ignore_nested=True))
27272 69308357ecd1 import-checker: allow absolute imports of sub modules from local packages Yuya Nishihara <yuya@tcha.org> parents: 27018 diff changeset	588 for error, lineno in verify_import_convention(modname, src, localmods):
20036 e5d51fa51aba contrib: add an import checker Augie Fackler <raf@durin42.com> parents: diff changeset	589 any_errors = True
28703 a274c4f9087a py3: use print_function in import-checker timeless <timeless@mozdev.org> parents: 28702 diff changeset	590 print('%s:%d: %s' % (source_path, lineno, error))
20036 e5d51fa51aba contrib: add an import checker Augie Fackler <raf@durin42.com> parents: diff changeset	591 f.close()
e5d51fa51aba contrib: add an import checker Augie Fackler <raf@durin42.com> parents: diff changeset	592 cycles = find_cycles(used_imports)
e5d51fa51aba contrib: add an import checker Augie Fackler <raf@durin42.com> parents: diff changeset	593 if cycles:
e5d51fa51aba contrib: add an import checker Augie Fackler <raf@durin42.com> parents: diff changeset	594 firstmods = set()
e5d51fa51aba contrib: add an import checker Augie Fackler <raf@durin42.com> parents: diff changeset	595 for c in sorted(cycles, key=_cycle_sortkey):
e5d51fa51aba contrib: add an import checker Augie Fackler <raf@durin42.com> parents: diff changeset	596 first = c.split()[0]
e5d51fa51aba contrib: add an import checker Augie Fackler <raf@durin42.com> parents: diff changeset	597 # As a rough cut, ignore any cycle that starts with the
e5d51fa51aba contrib: add an import checker Augie Fackler <raf@durin42.com> parents: diff changeset	598 # same module as some other cycle. Otherwise we see lots
e5d51fa51aba contrib: add an import checker Augie Fackler <raf@durin42.com> parents: diff changeset	599 # of cycles that are effectively duplicates.
e5d51fa51aba contrib: add an import checker Augie Fackler <raf@durin42.com> parents: diff changeset	600 if first in firstmods:
e5d51fa51aba contrib: add an import checker Augie Fackler <raf@durin42.com> parents: diff changeset	601 continue
28703 a274c4f9087a py3: use print_function in import-checker timeless <timeless@mozdev.org> parents: 28702 diff changeset	602 print('Import cycle:', c)
20036 e5d51fa51aba contrib: add an import checker Augie Fackler <raf@durin42.com> parents: diff changeset	603 firstmods.add(first)
e5d51fa51aba contrib: add an import checker Augie Fackler <raf@durin42.com> parents: diff changeset	604 any_errors = True
25731 cd1daab5d036 import-checker.py: exit with code 0 if no error is detected FUJIWARA Katsunori <foozy@lares.dti.ne.jp> parents: 25703 diff changeset	605 return any_errors != 0
20036 e5d51fa51aba contrib: add an import checker Augie Fackler <raf@durin42.com> parents: diff changeset	606
e5d51fa51aba contrib: add an import checker Augie Fackler <raf@durin42.com> parents: diff changeset	607 if __name__ == '__main__':
e5d51fa51aba contrib: add an import checker Augie Fackler <raf@durin42.com> parents: diff changeset	608 sys.exit(int(main(sys.argv)))

Mercurial > public > mercurial-scm > hg

annotate contrib/import-checker.py @ 28713:806d260c6f3b