mercurial-scm/hg-stable: contrib/import-checker.py annotate

annotate contrib/import-checker.py @ 28921:02ee31a50002

import-checker: track filenames for SyntaxErrors

author	timeless <timeless@mozdev.org>
date	Wed, 13 Apr 2016 16:34:59 +0000
parents	cdf331b54eb8
children	4ec62a084e5c

rev	line source
26954 f804bf27439b import-checker: make it executable for convenience Yuya Nishihara <yuya@tcha.org> parents: 26781 diff changeset	1 #!/usr/bin/env python
f804bf27439b import-checker: make it executable for convenience Yuya Nishihara <yuya@tcha.org> parents: 26781 diff changeset	2
28703 a274c4f9087a py3: use print_function in import-checker timeless <timeless@mozdev.org> parents: 28702 diff changeset	3 from __future__ import absolute_import, print_function
28702 e44f671018e3 py3: use absolute_import in import-checker timeless <timeless@mozdev.org> parents: 28700 diff changeset	4
20036 e5d51fa51aba contrib: add an import checker Augie Fackler <raf@durin42.com> parents: diff changeset	5 import ast
26965 1fa66d3ad28d import-checker: reset context to verify convention in function scope Yuya Nishihara <yuya@tcha.org> parents: 26964 diff changeset	6 import collections
20036 e5d51fa51aba contrib: add an import checker Augie Fackler <raf@durin42.com> parents: diff changeset	7 import os
e5d51fa51aba contrib: add an import checker Augie Fackler <raf@durin42.com> parents: diff changeset	8 import sys
e5d51fa51aba contrib: add an import checker Augie Fackler <raf@durin42.com> parents: diff changeset	9
20198 f5393a9dc4e5 import-checker: make test-module-imports.t work using virtualenv (issue4129) Chris Jerdonek <chris.jerdonek@gmail.com> parents: 20197 diff changeset	10 # Import a minimal set of stdlib modules needed for list_stdlib_modules()
f5393a9dc4e5 import-checker: make test-module-imports.t work using virtualenv (issue4129) Chris Jerdonek <chris.jerdonek@gmail.com> parents: 20197 diff changeset	11 # to work when run from a virtualenv. The modules were chosen empirically
f5393a9dc4e5 import-checker: make test-module-imports.t work using virtualenv (issue4129) Chris Jerdonek <chris.jerdonek@gmail.com> parents: 20197 diff changeset	12 # so that the return value matches the return value without virtualenv.
f5393a9dc4e5 import-checker: make test-module-imports.t work using virtualenv (issue4129) Chris Jerdonek <chris.jerdonek@gmail.com> parents: 20197 diff changeset	13 import BaseHTTPServer
f5393a9dc4e5 import-checker: make test-module-imports.t work using virtualenv (issue4129) Chris Jerdonek <chris.jerdonek@gmail.com> parents: 20197 diff changeset	14 import zlib
f5393a9dc4e5 import-checker: make test-module-imports.t work using virtualenv (issue4129) Chris Jerdonek <chris.jerdonek@gmail.com> parents: 20197 diff changeset	15
25703 1a6a117d0b95 import-checker: establish modern import convention Gregory Szorc <gregory.szorc@gmail.com> parents: 25702 diff changeset	16 # Whitelist of modules that symbols can be directly imported from.
1a6a117d0b95 import-checker: establish modern import convention Gregory Szorc <gregory.szorc@gmail.com> parents: 25702 diff changeset	17 allowsymbolimports = (
1a6a117d0b95 import-checker: establish modern import convention Gregory Szorc <gregory.szorc@gmail.com> parents: 25702 diff changeset	18 '__future__',
27018 e5be48dd8215 import-checker: allow symbol imports from hgweb.common and .request Yuya Nishihara <yuya@tcha.org> parents: 26965 diff changeset	19 'mercurial.hgweb.common',
e5be48dd8215 import-checker: allow symbol imports from hgweb.common and .request Yuya Nishihara <yuya@tcha.org> parents: 26965 diff changeset	20 'mercurial.hgweb.request',
25703 1a6a117d0b95 import-checker: establish modern import convention Gregory Szorc <gregory.szorc@gmail.com> parents: 25702 diff changeset	21 'mercurial.i18n',
1a6a117d0b95 import-checker: establish modern import convention Gregory Szorc <gregory.szorc@gmail.com> parents: 25702 diff changeset	22 'mercurial.node',
1a6a117d0b95 import-checker: establish modern import convention Gregory Szorc <gregory.szorc@gmail.com> parents: 25702 diff changeset	23 )
1a6a117d0b95 import-checker: establish modern import convention Gregory Szorc <gregory.szorc@gmail.com> parents: 25702 diff changeset	24
1a6a117d0b95 import-checker: establish modern import convention Gregory Szorc <gregory.szorc@gmail.com> parents: 25702 diff changeset	25 # Modules that must be aliased because they are commonly confused with
1a6a117d0b95 import-checker: establish modern import convention Gregory Szorc <gregory.szorc@gmail.com> parents: 25702 diff changeset	26 # common variables and can create aliasing and readability issues.
1a6a117d0b95 import-checker: establish modern import convention Gregory Szorc <gregory.szorc@gmail.com> parents: 25702 diff changeset	27 requirealias = {
1a6a117d0b95 import-checker: establish modern import convention Gregory Szorc <gregory.szorc@gmail.com> parents: 25702 diff changeset	28 'ui': 'uimod',
1a6a117d0b95 import-checker: establish modern import convention Gregory Szorc <gregory.szorc@gmail.com> parents: 25702 diff changeset	29 }
1a6a117d0b95 import-checker: establish modern import convention Gregory Szorc <gregory.szorc@gmail.com> parents: 25702 diff changeset	30
1a6a117d0b95 import-checker: establish modern import convention Gregory Szorc <gregory.szorc@gmail.com> parents: 25702 diff changeset	31 def usingabsolute(root):
1a6a117d0b95 import-checker: establish modern import convention Gregory Szorc <gregory.szorc@gmail.com> parents: 25702 diff changeset	32 """Whether absolute imports are being used."""
1a6a117d0b95 import-checker: establish modern import convention Gregory Szorc <gregory.szorc@gmail.com> parents: 25702 diff changeset	33 if sys.version_info[0] >= 3:
1a6a117d0b95 import-checker: establish modern import convention Gregory Szorc <gregory.szorc@gmail.com> parents: 25702 diff changeset	34 return True
1a6a117d0b95 import-checker: establish modern import convention Gregory Szorc <gregory.szorc@gmail.com> parents: 25702 diff changeset	35
1a6a117d0b95 import-checker: establish modern import convention Gregory Szorc <gregory.szorc@gmail.com> parents: 25702 diff changeset	36 for node in ast.walk(root):
1a6a117d0b95 import-checker: establish modern import convention Gregory Szorc <gregory.szorc@gmail.com> parents: 25702 diff changeset	37 if isinstance(node, ast.ImportFrom):
1a6a117d0b95 import-checker: establish modern import convention Gregory Szorc <gregory.szorc@gmail.com> parents: 25702 diff changeset	38 if node.module == '__future__':
1a6a117d0b95 import-checker: establish modern import convention Gregory Szorc <gregory.szorc@gmail.com> parents: 25702 diff changeset	39 for n in node.names:
1a6a117d0b95 import-checker: establish modern import convention Gregory Szorc <gregory.szorc@gmail.com> parents: 25702 diff changeset	40 if n.name == 'absolute_import':
1a6a117d0b95 import-checker: establish modern import convention Gregory Szorc <gregory.szorc@gmail.com> parents: 25702 diff changeset	41 return True
1a6a117d0b95 import-checker: establish modern import convention Gregory Szorc <gregory.szorc@gmail.com> parents: 25702 diff changeset	42
1a6a117d0b95 import-checker: establish modern import convention Gregory Szorc <gregory.szorc@gmail.com> parents: 25702 diff changeset	43 return False
1a6a117d0b95 import-checker: establish modern import convention Gregory Szorc <gregory.szorc@gmail.com> parents: 25702 diff changeset	44
26965 1fa66d3ad28d import-checker: reset context to verify convention in function scope Yuya Nishihara <yuya@tcha.org> parents: 26964 diff changeset	45 def walklocal(root):
1fa66d3ad28d import-checker: reset context to verify convention in function scope Yuya Nishihara <yuya@tcha.org> parents: 26964 diff changeset	46 """Recursively yield all descendant nodes but not in a different scope"""
1fa66d3ad28d import-checker: reset context to verify convention in function scope Yuya Nishihara <yuya@tcha.org> parents: 26964 diff changeset	47 todo = collections.deque(ast.iter_child_nodes(root))
1fa66d3ad28d import-checker: reset context to verify convention in function scope Yuya Nishihara <yuya@tcha.org> parents: 26964 diff changeset	48 yield root, False
1fa66d3ad28d import-checker: reset context to verify convention in function scope Yuya Nishihara <yuya@tcha.org> parents: 26964 diff changeset	49 while todo:
1fa66d3ad28d import-checker: reset context to verify convention in function scope Yuya Nishihara <yuya@tcha.org> parents: 26964 diff changeset	50 node = todo.popleft()
1fa66d3ad28d import-checker: reset context to verify convention in function scope Yuya Nishihara <yuya@tcha.org> parents: 26964 diff changeset	51 newscope = isinstance(node, ast.FunctionDef)
1fa66d3ad28d import-checker: reset context to verify convention in function scope Yuya Nishihara <yuya@tcha.org> parents: 26964 diff changeset	52 if not newscope:
1fa66d3ad28d import-checker: reset context to verify convention in function scope Yuya Nishihara <yuya@tcha.org> parents: 26964 diff changeset	53 todo.extend(ast.iter_child_nodes(node))
1fa66d3ad28d import-checker: reset context to verify convention in function scope Yuya Nishihara <yuya@tcha.org> parents: 26964 diff changeset	54 yield node, newscope
1fa66d3ad28d import-checker: reset context to verify convention in function scope Yuya Nishihara <yuya@tcha.org> parents: 26964 diff changeset	55
22975 461342e1c8aa import-checker: check modules for pure Python build correctly FUJIWARA Katsunori <foozy@lares.dti.ne.jp> parents: 22974 diff changeset	56 def dotted_name_of_path(path, trimpure=False):
20036 e5d51fa51aba contrib: add an import checker Augie Fackler <raf@durin42.com> parents: diff changeset	57 """Given a relative path to a source file, return its dotted module name.
e5d51fa51aba contrib: add an import checker Augie Fackler <raf@durin42.com> parents: diff changeset	58
e5d51fa51aba contrib: add an import checker Augie Fackler <raf@durin42.com> parents: diff changeset	59 >>> dotted_name_of_path('mercurial/error.py')
e5d51fa51aba contrib: add an import checker Augie Fackler <raf@durin42.com> parents: diff changeset	60 'mercurial.error'
22975 461342e1c8aa import-checker: check modules for pure Python build correctly FUJIWARA Katsunori <foozy@lares.dti.ne.jp> parents: 22974 diff changeset	61 >>> dotted_name_of_path('mercurial/pure/parsers.py', trimpure=True)
461342e1c8aa import-checker: check modules for pure Python build correctly FUJIWARA Katsunori <foozy@lares.dti.ne.jp> parents: 22974 diff changeset	62 'mercurial.parsers'
20383 4990abb4729d import-checker: fix names of dynamically loaded modules Mads Kiilerich <madski@unity3d.com> parents: 20238 diff changeset	63 >>> dotted_name_of_path('zlibmodule.so')
4990abb4729d import-checker: fix names of dynamically loaded modules Mads Kiilerich <madski@unity3d.com> parents: 20238 diff changeset	64 'zlib'
20036 e5d51fa51aba contrib: add an import checker Augie Fackler <raf@durin42.com> parents: diff changeset	65 """
27620 0c60843b55b5 import-checker: normalize directory separator to get module name on Windows Yuya Nishihara <yuya@tcha.org> parents: 27520 diff changeset	66 parts = path.replace(os.sep, '/').split('/')
20391 466e4c574db0 import-checker: handle standard modules with arch in the filename Mads Kiilerich <madski@unity3d.com> parents: 20386 diff changeset	67 parts[-1] = parts[-1].split('.', 1)[0] # remove .py and .so and .ARCH.so
20383 4990abb4729d import-checker: fix names of dynamically loaded modules Mads Kiilerich <madski@unity3d.com> parents: 20238 diff changeset	68 if parts[-1].endswith('module'):
4990abb4729d import-checker: fix names of dynamically loaded modules Mads Kiilerich <madski@unity3d.com> parents: 20238 diff changeset	69 parts[-1] = parts[-1][:-6]
22975 461342e1c8aa import-checker: check modules for pure Python build correctly FUJIWARA Katsunori <foozy@lares.dti.ne.jp> parents: 22974 diff changeset	70 if trimpure:
461342e1c8aa import-checker: check modules for pure Python build correctly FUJIWARA Katsunori <foozy@lares.dti.ne.jp> parents: 22974 diff changeset	71 return '.'.join(p for p in parts if p != 'pure')
20036 e5d51fa51aba contrib: add an import checker Augie Fackler <raf@durin42.com> parents: diff changeset	72 return '.'.join(parts)
e5d51fa51aba contrib: add an import checker Augie Fackler <raf@durin42.com> parents: diff changeset	73
25173 7358b5d9991e import-checker: add utility to examine what module is imported easily FUJIWARA Katsunori <foozy@lares.dti.ne.jp> parents: 25064 diff changeset	74 def fromlocalfunc(modulename, localmods):
7358b5d9991e import-checker: add utility to examine what module is imported easily FUJIWARA Katsunori <foozy@lares.dti.ne.jp> parents: 25064 diff changeset	75 """Get a function to examine which locally defined module the
7358b5d9991e import-checker: add utility to examine what module is imported easily FUJIWARA Katsunori <foozy@lares.dti.ne.jp> parents: 25064 diff changeset	76 target source imports via a specified name.
7358b5d9991e import-checker: add utility to examine what module is imported easily FUJIWARA Katsunori <foozy@lares.dti.ne.jp> parents: 25064 diff changeset	77
7358b5d9991e import-checker: add utility to examine what module is imported easily FUJIWARA Katsunori <foozy@lares.dti.ne.jp> parents: 25064 diff changeset	78 `modulename` is an `dotted_name_of_path()`-ed source file path,
7358b5d9991e import-checker: add utility to examine what module is imported easily FUJIWARA Katsunori <foozy@lares.dti.ne.jp> parents: 25064 diff changeset	79 which may have `.__init__` at the end of it, of the target source.
7358b5d9991e import-checker: add utility to examine what module is imported easily FUJIWARA Katsunori <foozy@lares.dti.ne.jp> parents: 25064 diff changeset	80
7358b5d9991e import-checker: add utility to examine what module is imported easily FUJIWARA Katsunori <foozy@lares.dti.ne.jp> parents: 25064 diff changeset	81 `localmods` is a dict (or set), of which key is an absolute
7358b5d9991e import-checker: add utility to examine what module is imported easily FUJIWARA Katsunori <foozy@lares.dti.ne.jp> parents: 25064 diff changeset	82 `dotted_name_of_path()`-ed source file path of locally defined (=
7358b5d9991e import-checker: add utility to examine what module is imported easily FUJIWARA Katsunori <foozy@lares.dti.ne.jp> parents: 25064 diff changeset	83 Mercurial specific) modules.
7358b5d9991e import-checker: add utility to examine what module is imported easily FUJIWARA Katsunori <foozy@lares.dti.ne.jp> parents: 25064 diff changeset	84
7358b5d9991e import-checker: add utility to examine what module is imported easily FUJIWARA Katsunori <foozy@lares.dti.ne.jp> parents: 25064 diff changeset	85 This function assumes that module names not existing in
26781 1aee2ab0f902 spelling: trivial spell checking Mads Kiilerich <madski@unity3d.com> parents: 26221 diff changeset	86 `localmods` are from the Python standard library.
25173 7358b5d9991e import-checker: add utility to examine what module is imported easily FUJIWARA Katsunori <foozy@lares.dti.ne.jp> parents: 25064 diff changeset	87
7358b5d9991e import-checker: add utility to examine what module is imported easily FUJIWARA Katsunori <foozy@lares.dti.ne.jp> parents: 25064 diff changeset	88 This function returns the function, which takes `name` argument,
7358b5d9991e import-checker: add utility to examine what module is imported easily FUJIWARA Katsunori <foozy@lares.dti.ne.jp> parents: 25064 diff changeset	89 and returns `(absname, dottedpath, hassubmod)` tuple if `name`
7358b5d9991e import-checker: add utility to examine what module is imported easily FUJIWARA Katsunori <foozy@lares.dti.ne.jp> parents: 25064 diff changeset	90 matches against locally defined module. Otherwise, it returns
7358b5d9991e import-checker: add utility to examine what module is imported easily FUJIWARA Katsunori <foozy@lares.dti.ne.jp> parents: 25064 diff changeset	91 False.
7358b5d9991e import-checker: add utility to examine what module is imported easily FUJIWARA Katsunori <foozy@lares.dti.ne.jp> parents: 25064 diff changeset	92
7358b5d9991e import-checker: add utility to examine what module is imported easily FUJIWARA Katsunori <foozy@lares.dti.ne.jp> parents: 25064 diff changeset	93 It is assumed that `name` doesn't have `.__init__`.
7358b5d9991e import-checker: add utility to examine what module is imported easily FUJIWARA Katsunori <foozy@lares.dti.ne.jp> parents: 25064 diff changeset	94
7358b5d9991e import-checker: add utility to examine what module is imported easily FUJIWARA Katsunori <foozy@lares.dti.ne.jp> parents: 25064 diff changeset	95 `absname` is an absolute module name of specified `name`
7358b5d9991e import-checker: add utility to examine what module is imported easily FUJIWARA Katsunori <foozy@lares.dti.ne.jp> parents: 25064 diff changeset	96 (e.g. "hgext.convert"). This can be used to compose prefix for sub
7358b5d9991e import-checker: add utility to examine what module is imported easily FUJIWARA Katsunori <foozy@lares.dti.ne.jp> parents: 25064 diff changeset	97 modules or so.
7358b5d9991e import-checker: add utility to examine what module is imported easily FUJIWARA Katsunori <foozy@lares.dti.ne.jp> parents: 25064 diff changeset	98
7358b5d9991e import-checker: add utility to examine what module is imported easily FUJIWARA Katsunori <foozy@lares.dti.ne.jp> parents: 25064 diff changeset	99 `dottedpath` is a `dotted_name_of_path()`-ed source file path
7358b5d9991e import-checker: add utility to examine what module is imported easily FUJIWARA Katsunori <foozy@lares.dti.ne.jp> parents: 25064 diff changeset	100 (e.g. "hgext.convert.__init__") of `name`. This is used to look
7358b5d9991e import-checker: add utility to examine what module is imported easily FUJIWARA Katsunori <foozy@lares.dti.ne.jp> parents: 25064 diff changeset	101 module up in `localmods` again.
7358b5d9991e import-checker: add utility to examine what module is imported easily FUJIWARA Katsunori <foozy@lares.dti.ne.jp> parents: 25064 diff changeset	102
7358b5d9991e import-checker: add utility to examine what module is imported easily FUJIWARA Katsunori <foozy@lares.dti.ne.jp> parents: 25064 diff changeset	103 `hassubmod` is whether it may have sub modules under it (for
7358b5d9991e import-checker: add utility to examine what module is imported easily FUJIWARA Katsunori <foozy@lares.dti.ne.jp> parents: 25064 diff changeset	104 convenient, even though this is also equivalent to "absname !=
7358b5d9991e import-checker: add utility to examine what module is imported easily FUJIWARA Katsunori <foozy@lares.dti.ne.jp> parents: 25064 diff changeset	105 dottednpath")
7358b5d9991e import-checker: add utility to examine what module is imported easily FUJIWARA Katsunori <foozy@lares.dti.ne.jp> parents: 25064 diff changeset	106
7358b5d9991e import-checker: add utility to examine what module is imported easily FUJIWARA Katsunori <foozy@lares.dti.ne.jp> parents: 25064 diff changeset	107 >>> localmods = {'foo.__init__': True, 'foo.foo1': True,
7358b5d9991e import-checker: add utility to examine what module is imported easily FUJIWARA Katsunori <foozy@lares.dti.ne.jp> parents: 25064 diff changeset	108 ... 'foo.bar.__init__': True, 'foo.bar.bar1': True,
7358b5d9991e import-checker: add utility to examine what module is imported easily FUJIWARA Katsunori <foozy@lares.dti.ne.jp> parents: 25064 diff changeset	109 ... 'baz.__init__': True, 'baz.baz1': True }
7358b5d9991e import-checker: add utility to examine what module is imported easily FUJIWARA Katsunori <foozy@lares.dti.ne.jp> parents: 25064 diff changeset	110 >>> fromlocal = fromlocalfunc('foo.xxx', localmods)
7358b5d9991e import-checker: add utility to examine what module is imported easily FUJIWARA Katsunori <foozy@lares.dti.ne.jp> parents: 25064 diff changeset	111 >>> # relative
7358b5d9991e import-checker: add utility to examine what module is imported easily FUJIWARA Katsunori <foozy@lares.dti.ne.jp> parents: 25064 diff changeset	112 >>> fromlocal('foo1')
7358b5d9991e import-checker: add utility to examine what module is imported easily FUJIWARA Katsunori <foozy@lares.dti.ne.jp> parents: 25064 diff changeset	113 ('foo.foo1', 'foo.foo1', False)
7358b5d9991e import-checker: add utility to examine what module is imported easily FUJIWARA Katsunori <foozy@lares.dti.ne.jp> parents: 25064 diff changeset	114 >>> fromlocal('bar')
7358b5d9991e import-checker: add utility to examine what module is imported easily FUJIWARA Katsunori <foozy@lares.dti.ne.jp> parents: 25064 diff changeset	115 ('foo.bar', 'foo.bar.__init__', True)
7358b5d9991e import-checker: add utility to examine what module is imported easily FUJIWARA Katsunori <foozy@lares.dti.ne.jp> parents: 25064 diff changeset	116 >>> fromlocal('bar.bar1')
7358b5d9991e import-checker: add utility to examine what module is imported easily FUJIWARA Katsunori <foozy@lares.dti.ne.jp> parents: 25064 diff changeset	117 ('foo.bar.bar1', 'foo.bar.bar1', False)
7358b5d9991e import-checker: add utility to examine what module is imported easily FUJIWARA Katsunori <foozy@lares.dti.ne.jp> parents: 25064 diff changeset	118 >>> # absolute
7358b5d9991e import-checker: add utility to examine what module is imported easily FUJIWARA Katsunori <foozy@lares.dti.ne.jp> parents: 25064 diff changeset	119 >>> fromlocal('baz')
7358b5d9991e import-checker: add utility to examine what module is imported easily FUJIWARA Katsunori <foozy@lares.dti.ne.jp> parents: 25064 diff changeset	120 ('baz', 'baz.__init__', True)
7358b5d9991e import-checker: add utility to examine what module is imported easily FUJIWARA Katsunori <foozy@lares.dti.ne.jp> parents: 25064 diff changeset	121 >>> fromlocal('baz.baz1')
7358b5d9991e import-checker: add utility to examine what module is imported easily FUJIWARA Katsunori <foozy@lares.dti.ne.jp> parents: 25064 diff changeset	122 ('baz.baz1', 'baz.baz1', False)
7358b5d9991e import-checker: add utility to examine what module is imported easily FUJIWARA Katsunori <foozy@lares.dti.ne.jp> parents: 25064 diff changeset	123 >>> # unknown = maybe standard library
7358b5d9991e import-checker: add utility to examine what module is imported easily FUJIWARA Katsunori <foozy@lares.dti.ne.jp> parents: 25064 diff changeset	124 >>> fromlocal('os')
7358b5d9991e import-checker: add utility to examine what module is imported easily FUJIWARA Katsunori <foozy@lares.dti.ne.jp> parents: 25064 diff changeset	125 False
25701 1f88c0f6ff5a import-checker: resolve relative imports Gregory Szorc <gregory.szorc@gmail.com> parents: 25660 diff changeset	126 >>> fromlocal(None, 1)
1f88c0f6ff5a import-checker: resolve relative imports Gregory Szorc <gregory.szorc@gmail.com> parents: 25660 diff changeset	127 ('foo', 'foo.__init__', True)
1f88c0f6ff5a import-checker: resolve relative imports Gregory Szorc <gregory.szorc@gmail.com> parents: 25660 diff changeset	128 >>> fromlocal2 = fromlocalfunc('foo.xxx.yyy', localmods)
1f88c0f6ff5a import-checker: resolve relative imports Gregory Szorc <gregory.szorc@gmail.com> parents: 25660 diff changeset	129 >>> fromlocal2(None, 2)
1f88c0f6ff5a import-checker: resolve relative imports Gregory Szorc <gregory.szorc@gmail.com> parents: 25660 diff changeset	130 ('foo', 'foo.__init__', True)
25173 7358b5d9991e import-checker: add utility to examine what module is imported easily FUJIWARA Katsunori <foozy@lares.dti.ne.jp> parents: 25064 diff changeset	131 """
7358b5d9991e import-checker: add utility to examine what module is imported easily FUJIWARA Katsunori <foozy@lares.dti.ne.jp> parents: 25064 diff changeset	132 prefix = '.'.join(modulename.split('.')[:-1])
7358b5d9991e import-checker: add utility to examine what module is imported easily FUJIWARA Katsunori <foozy@lares.dti.ne.jp> parents: 25064 diff changeset	133 if prefix:
7358b5d9991e import-checker: add utility to examine what module is imported easily FUJIWARA Katsunori <foozy@lares.dti.ne.jp> parents: 25064 diff changeset	134 prefix += '.'
25701 1f88c0f6ff5a import-checker: resolve relative imports Gregory Szorc <gregory.szorc@gmail.com> parents: 25660 diff changeset	135 def fromlocal(name, level=0):
1f88c0f6ff5a import-checker: resolve relative imports Gregory Szorc <gregory.szorc@gmail.com> parents: 25660 diff changeset	136 # name is None when relative imports are used.
1f88c0f6ff5a import-checker: resolve relative imports Gregory Szorc <gregory.szorc@gmail.com> parents: 25660 diff changeset	137 if name is None:
1f88c0f6ff5a import-checker: resolve relative imports Gregory Szorc <gregory.szorc@gmail.com> parents: 25660 diff changeset	138 # If relative imports are used, level must not be absolute.
1f88c0f6ff5a import-checker: resolve relative imports Gregory Szorc <gregory.szorc@gmail.com> parents: 25660 diff changeset	139 assert level > 0
1f88c0f6ff5a import-checker: resolve relative imports Gregory Szorc <gregory.szorc@gmail.com> parents: 25660 diff changeset	140 candidates = ['.'.join(modulename.split('.')[:-level])]
1f88c0f6ff5a import-checker: resolve relative imports Gregory Szorc <gregory.szorc@gmail.com> parents: 25660 diff changeset	141 else:
1f88c0f6ff5a import-checker: resolve relative imports Gregory Szorc <gregory.szorc@gmail.com> parents: 25660 diff changeset	142 # Check relative name first.
1f88c0f6ff5a import-checker: resolve relative imports Gregory Szorc <gregory.szorc@gmail.com> parents: 25660 diff changeset	143 candidates = [prefix + name, name]
1f88c0f6ff5a import-checker: resolve relative imports Gregory Szorc <gregory.szorc@gmail.com> parents: 25660 diff changeset	144
1f88c0f6ff5a import-checker: resolve relative imports Gregory Szorc <gregory.szorc@gmail.com> parents: 25660 diff changeset	145 for n in candidates:
25173 7358b5d9991e import-checker: add utility to examine what module is imported easily FUJIWARA Katsunori <foozy@lares.dti.ne.jp> parents: 25064 diff changeset	146 if n in localmods:
7358b5d9991e import-checker: add utility to examine what module is imported easily FUJIWARA Katsunori <foozy@lares.dti.ne.jp> parents: 25064 diff changeset	147 return (n, n, False)
7358b5d9991e import-checker: add utility to examine what module is imported easily FUJIWARA Katsunori <foozy@lares.dti.ne.jp> parents: 25064 diff changeset	148 dottedpath = n + '.__init__'
7358b5d9991e import-checker: add utility to examine what module is imported easily FUJIWARA Katsunori <foozy@lares.dti.ne.jp> parents: 25064 diff changeset	149 if dottedpath in localmods:
7358b5d9991e import-checker: add utility to examine what module is imported easily FUJIWARA Katsunori <foozy@lares.dti.ne.jp> parents: 25064 diff changeset	150 return (n, dottedpath, True)
7358b5d9991e import-checker: add utility to examine what module is imported easily FUJIWARA Katsunori <foozy@lares.dti.ne.jp> parents: 25064 diff changeset	151 return False
7358b5d9991e import-checker: add utility to examine what module is imported easily FUJIWARA Katsunori <foozy@lares.dti.ne.jp> parents: 25064 diff changeset	152 return fromlocal
20036 e5d51fa51aba contrib: add an import checker Augie Fackler <raf@durin42.com> parents: diff changeset	153
e5d51fa51aba contrib: add an import checker Augie Fackler <raf@durin42.com> parents: diff changeset	154 def list_stdlib_modules():
e5d51fa51aba contrib: add an import checker Augie Fackler <raf@durin42.com> parents: diff changeset	155 """List the modules present in the stdlib.
e5d51fa51aba contrib: add an import checker Augie Fackler <raf@durin42.com> parents: diff changeset	156
e5d51fa51aba contrib: add an import checker Augie Fackler <raf@durin42.com> parents: diff changeset	157 >>> mods = set(list_stdlib_modules())
e5d51fa51aba contrib: add an import checker Augie Fackler <raf@durin42.com> parents: diff changeset	158 >>> 'BaseHTTPServer' in mods
e5d51fa51aba contrib: add an import checker Augie Fackler <raf@durin42.com> parents: diff changeset	159 True
e5d51fa51aba contrib: add an import checker Augie Fackler <raf@durin42.com> parents: diff changeset	160
e5d51fa51aba contrib: add an import checker Augie Fackler <raf@durin42.com> parents: diff changeset	161 os.path isn't really a module, so it's missing:
e5d51fa51aba contrib: add an import checker Augie Fackler <raf@durin42.com> parents: diff changeset	162
e5d51fa51aba contrib: add an import checker Augie Fackler <raf@durin42.com> parents: diff changeset	163 >>> 'os.path' in mods
e5d51fa51aba contrib: add an import checker Augie Fackler <raf@durin42.com> parents: diff changeset	164 False
e5d51fa51aba contrib: add an import checker Augie Fackler <raf@durin42.com> parents: diff changeset	165
e5d51fa51aba contrib: add an import checker Augie Fackler <raf@durin42.com> parents: diff changeset	166 sys requires special treatment, because it's baked into the
e5d51fa51aba contrib: add an import checker Augie Fackler <raf@durin42.com> parents: diff changeset	167 interpreter, but it should still appear:
e5d51fa51aba contrib: add an import checker Augie Fackler <raf@durin42.com> parents: diff changeset	168
e5d51fa51aba contrib: add an import checker Augie Fackler <raf@durin42.com> parents: diff changeset	169 >>> 'sys' in mods
e5d51fa51aba contrib: add an import checker Augie Fackler <raf@durin42.com> parents: diff changeset	170 True
e5d51fa51aba contrib: add an import checker Augie Fackler <raf@durin42.com> parents: diff changeset	171
e5d51fa51aba contrib: add an import checker Augie Fackler <raf@durin42.com> parents: diff changeset	172 >>> 'collections' in mods
e5d51fa51aba contrib: add an import checker Augie Fackler <raf@durin42.com> parents: diff changeset	173 True
e5d51fa51aba contrib: add an import checker Augie Fackler <raf@durin42.com> parents: diff changeset	174
e5d51fa51aba contrib: add an import checker Augie Fackler <raf@durin42.com> parents: diff changeset	175 >>> 'cStringIO' in mods
e5d51fa51aba contrib: add an import checker Augie Fackler <raf@durin42.com> parents: diff changeset	176 True
e5d51fa51aba contrib: add an import checker Augie Fackler <raf@durin42.com> parents: diff changeset	177 """
e5d51fa51aba contrib: add an import checker Augie Fackler <raf@durin42.com> parents: diff changeset	178 for m in sys.builtin_module_names:
e5d51fa51aba contrib: add an import checker Augie Fackler <raf@durin42.com> parents: diff changeset	179 yield m
e5d51fa51aba contrib: add an import checker Augie Fackler <raf@durin42.com> parents: diff changeset	180 # These modules only exist on windows, but we should always
e5d51fa51aba contrib: add an import checker Augie Fackler <raf@durin42.com> parents: diff changeset	181 # consider them stdlib.
e5d51fa51aba contrib: add an import checker Augie Fackler <raf@durin42.com> parents: diff changeset	182 for m in ['msvcrt', '_winreg']:
e5d51fa51aba contrib: add an import checker Augie Fackler <raf@durin42.com> parents: diff changeset	183 yield m
e5d51fa51aba contrib: add an import checker Augie Fackler <raf@durin42.com> parents: diff changeset	184 yield 'builtins' # python3 only
24669 fbdbff1b486a import-checker: force 'fcntl', 'grp', 'pwd', and 'termios' to stdlib modules Matt Harbison <matt_harbison@yahoo.com> parents: 24668 diff changeset	185 for m in 'fcntl', 'grp', 'pwd', 'termios': # Unix only
fbdbff1b486a import-checker: force 'fcntl', 'grp', 'pwd', and 'termios' to stdlib modules Matt Harbison <matt_harbison@yahoo.com> parents: 24668 diff changeset	186 yield m
28713 806d260c6f3b tests: fix builtin module test on pypy Maciej Fijalkowski <fijall@gmail.com> parents: 28704 diff changeset	187 for m in 'cPickle', 'datetime': # in Python (not C) on PyPy
806d260c6f3b tests: fix builtin module test on pypy Maciej Fijalkowski <fijall@gmail.com> parents: 28704 diff changeset	188 yield m
20197 761f2929a6ad import-checker: refactor sys.path prefix check (issue4129) Chris Jerdonek <chris.jerdonek@gmail.com> parents: 20038 diff changeset	189 stdlib_prefixes = set([sys.prefix, sys.exec_prefix])
20198 f5393a9dc4e5 import-checker: make test-module-imports.t work using virtualenv (issue4129) Chris Jerdonek <chris.jerdonek@gmail.com> parents: 20197 diff changeset	190 # We need to supplement the list of prefixes for the search to work
f5393a9dc4e5 import-checker: make test-module-imports.t work using virtualenv (issue4129) Chris Jerdonek <chris.jerdonek@gmail.com> parents: 20197 diff changeset	191 # when run from within a virtualenv.
f5393a9dc4e5 import-checker: make test-module-imports.t work using virtualenv (issue4129) Chris Jerdonek <chris.jerdonek@gmail.com> parents: 20197 diff changeset	192 for mod in (BaseHTTPServer, zlib):
f5393a9dc4e5 import-checker: make test-module-imports.t work using virtualenv (issue4129) Chris Jerdonek <chris.jerdonek@gmail.com> parents: 20197 diff changeset	193 try:
f5393a9dc4e5 import-checker: make test-module-imports.t work using virtualenv (issue4129) Chris Jerdonek <chris.jerdonek@gmail.com> parents: 20197 diff changeset	194 # Not all module objects have a __file__ attribute.
f5393a9dc4e5 import-checker: make test-module-imports.t work using virtualenv (issue4129) Chris Jerdonek <chris.jerdonek@gmail.com> parents: 20197 diff changeset	195 filename = mod.__file__
f5393a9dc4e5 import-checker: make test-module-imports.t work using virtualenv (issue4129) Chris Jerdonek <chris.jerdonek@gmail.com> parents: 20197 diff changeset	196 except AttributeError:
f5393a9dc4e5 import-checker: make test-module-imports.t work using virtualenv (issue4129) Chris Jerdonek <chris.jerdonek@gmail.com> parents: 20197 diff changeset	197 continue
f5393a9dc4e5 import-checker: make test-module-imports.t work using virtualenv (issue4129) Chris Jerdonek <chris.jerdonek@gmail.com> parents: 20197 diff changeset	198 dirname = os.path.dirname(filename)
f5393a9dc4e5 import-checker: make test-module-imports.t work using virtualenv (issue4129) Chris Jerdonek <chris.jerdonek@gmail.com> parents: 20197 diff changeset	199 for prefix in stdlib_prefixes:
f5393a9dc4e5 import-checker: make test-module-imports.t work using virtualenv (issue4129) Chris Jerdonek <chris.jerdonek@gmail.com> parents: 20197 diff changeset	200 if dirname.startswith(prefix):
f5393a9dc4e5 import-checker: make test-module-imports.t work using virtualenv (issue4129) Chris Jerdonek <chris.jerdonek@gmail.com> parents: 20197 diff changeset	201 # Then this directory is redundant.
f5393a9dc4e5 import-checker: make test-module-imports.t work using virtualenv (issue4129) Chris Jerdonek <chris.jerdonek@gmail.com> parents: 20197 diff changeset	202 break
f5393a9dc4e5 import-checker: make test-module-imports.t work using virtualenv (issue4129) Chris Jerdonek <chris.jerdonek@gmail.com> parents: 20197 diff changeset	203 else:
f5393a9dc4e5 import-checker: make test-module-imports.t work using virtualenv (issue4129) Chris Jerdonek <chris.jerdonek@gmail.com> parents: 20197 diff changeset	204 stdlib_prefixes.add(dirname)
20036 e5d51fa51aba contrib: add an import checker Augie Fackler <raf@durin42.com> parents: diff changeset	205 for libpath in sys.path:
20201 bc3b48b0f5c8 import-checker: suppress check-code about any() Augie Fackler <raf@durin42.com> parents: 20200 diff changeset	206 # We want to walk everything in sys.path that starts with
28700 35ad5bcdeb7e py24: remove check-code py24 notation timeless <timeless@mozdev.org> parents: 28400 diff changeset	207 # something in stdlib_prefixes.
35ad5bcdeb7e py24: remove check-code py24 notation timeless <timeless@mozdev.org> parents: 28400 diff changeset	208 if not any(libpath.startswith(p) for p in stdlib_prefixes):
20036 e5d51fa51aba contrib: add an import checker Augie Fackler <raf@durin42.com> parents: diff changeset	209 continue
e5d51fa51aba contrib: add an import checker Augie Fackler <raf@durin42.com> parents: diff changeset	210 for top, dirs, files in os.walk(libpath):
25733 f99c066f5f9a import-checker: recurse into subtree of sys.path only if __init__.py exists Yuya Nishihara <yuya@tcha.org> parents: 25731 diff changeset	211 for i, d in reversed(list(enumerate(dirs))):
25734 9086d0c1def3 import-checker: exclude mercurial packages installed into the system path Yuya Nishihara <yuya@tcha.org> parents: 25733 diff changeset	212 if (not os.path.exists(os.path.join(top, d, '__init__.py'))
9086d0c1def3 import-checker: exclude mercurial packages installed into the system path Yuya Nishihara <yuya@tcha.org> parents: 25733 diff changeset	213 or top == libpath and d in ('hgext', 'mercurial')):
25733 f99c066f5f9a import-checker: recurse into subtree of sys.path only if __init__.py exists Yuya Nishihara <yuya@tcha.org> parents: 25731 diff changeset	214 del dirs[i]
20036 e5d51fa51aba contrib: add an import checker Augie Fackler <raf@durin42.com> parents: diff changeset	215 for name in files:
26221 ae65b1b4cb46 import-checker: use modern .endswith for multiple suffixes Augie Fackler <augie@google.com> parents: 26166 diff changeset	216 if not name.endswith(('.py', '.so', '.pyc', '.pyo', '.pyd')):
20036 e5d51fa51aba contrib: add an import checker Augie Fackler <raf@durin42.com> parents: diff changeset	217 continue
27621 39845b064041 import-checker: list package directory as stdlib module Yuya Nishihara <yuya@tcha.org> parents: 27620 diff changeset	218 if name.startswith('__init__.py'):
39845b064041 import-checker: list package directory as stdlib module Yuya Nishihara <yuya@tcha.org> parents: 27620 diff changeset	219 full_path = top
39845b064041 import-checker: list package directory as stdlib module Yuya Nishihara <yuya@tcha.org> parents: 27620 diff changeset	220 else:
39845b064041 import-checker: list package directory as stdlib module Yuya Nishihara <yuya@tcha.org> parents: 27620 diff changeset	221 full_path = os.path.join(top, name)
20036 e5d51fa51aba contrib: add an import checker Augie Fackler <raf@durin42.com> parents: diff changeset	222 rel_path = full_path[len(libpath) + 1:]
e5d51fa51aba contrib: add an import checker Augie Fackler <raf@durin42.com> parents: diff changeset	223 mod = dotted_name_of_path(rel_path)
e5d51fa51aba contrib: add an import checker Augie Fackler <raf@durin42.com> parents: diff changeset	224 yield mod
e5d51fa51aba contrib: add an import checker Augie Fackler <raf@durin42.com> parents: diff changeset	225
e5d51fa51aba contrib: add an import checker Augie Fackler <raf@durin42.com> parents: diff changeset	226 stdlib_modules = set(list_stdlib_modules())
e5d51fa51aba contrib: add an import checker Augie Fackler <raf@durin42.com> parents: diff changeset	227
28921 02ee31a50002 import-checker: track filenames for SyntaxErrors timeless <timeless@mozdev.org> parents: 28920 diff changeset	228 def imported_modules(source, modulename, f, localmods, ignore_nested=False):
20036 e5d51fa51aba contrib: add an import checker Augie Fackler <raf@durin42.com> parents: diff changeset	229 """Given the source of a file as a string, yield the names
e5d51fa51aba contrib: add an import checker Augie Fackler <raf@durin42.com> parents: diff changeset	230 imported by that file.
e5d51fa51aba contrib: add an import checker Augie Fackler <raf@durin42.com> parents: diff changeset	231
20037 957b43371928 import-checker: ignore nested imports Augie Fackler <raf@durin42.com> parents: 20036 diff changeset	232 Args:
957b43371928 import-checker: ignore nested imports Augie Fackler <raf@durin42.com> parents: 20036 diff changeset	233 source: The python source to examine as a string.
25174 86298718b01c import-checker: make imported_modules yield absolute dotted_name_of_path FUJIWARA Katsunori <foozy@lares.dti.ne.jp> parents: 25173 diff changeset	234 modulename: of specified python source (may have `__init__`)
86298718b01c import-checker: make imported_modules yield absolute dotted_name_of_path FUJIWARA Katsunori <foozy@lares.dti.ne.jp> parents: 25173 diff changeset	235 localmods: dict of locally defined module names (may have `__init__`)
20037 957b43371928 import-checker: ignore nested imports Augie Fackler <raf@durin42.com> parents: 20036 diff changeset	236 ignore_nested: If true, import statements that do not start in
957b43371928 import-checker: ignore nested imports Augie Fackler <raf@durin42.com> parents: 20036 diff changeset	237 column zero will be ignored.
957b43371928 import-checker: ignore nested imports Augie Fackler <raf@durin42.com> parents: 20036 diff changeset	238
957b43371928 import-checker: ignore nested imports Augie Fackler <raf@durin42.com> parents: 20036 diff changeset	239 Returns:
25174 86298718b01c import-checker: make imported_modules yield absolute dotted_name_of_path FUJIWARA Katsunori <foozy@lares.dti.ne.jp> parents: 25173 diff changeset	240 A list of absolute module names imported by the given source.
20037 957b43371928 import-checker: ignore nested imports Augie Fackler <raf@durin42.com> parents: 20036 diff changeset	241
28921 02ee31a50002 import-checker: track filenames for SyntaxErrors timeless <timeless@mozdev.org> parents: 28920 diff changeset	242 >>> f = 'foo/xxx.py'
25174 86298718b01c import-checker: make imported_modules yield absolute dotted_name_of_path FUJIWARA Katsunori <foozy@lares.dti.ne.jp> parents: 25173 diff changeset	243 >>> modulename = 'foo.xxx'
86298718b01c import-checker: make imported_modules yield absolute dotted_name_of_path FUJIWARA Katsunori <foozy@lares.dti.ne.jp> parents: 25173 diff changeset	244 >>> localmods = {'foo.__init__': True,
86298718b01c import-checker: make imported_modules yield absolute dotted_name_of_path FUJIWARA Katsunori <foozy@lares.dti.ne.jp> parents: 25173 diff changeset	245 ... 'foo.foo1': True, 'foo.foo2': True,
86298718b01c import-checker: make imported_modules yield absolute dotted_name_of_path FUJIWARA Katsunori <foozy@lares.dti.ne.jp> parents: 25173 diff changeset	246 ... 'foo.bar.__init__': True, 'foo.bar.bar1': True,
86298718b01c import-checker: make imported_modules yield absolute dotted_name_of_path FUJIWARA Katsunori <foozy@lares.dti.ne.jp> parents: 25173 diff changeset	247 ... 'baz.__init__': True, 'baz.baz1': True }
86298718b01c import-checker: make imported_modules yield absolute dotted_name_of_path FUJIWARA Katsunori <foozy@lares.dti.ne.jp> parents: 25173 diff changeset	248 >>> # standard library (= not locally defined ones)
86298718b01c import-checker: make imported_modules yield absolute dotted_name_of_path FUJIWARA Katsunori <foozy@lares.dti.ne.jp> parents: 25173 diff changeset	249 >>> sorted(imported_modules(
86298718b01c import-checker: make imported_modules yield absolute dotted_name_of_path FUJIWARA Katsunori <foozy@lares.dti.ne.jp> parents: 25173 diff changeset	250 ... 'from stdlib1 import foo, bar; import stdlib2',
28921 02ee31a50002 import-checker: track filenames for SyntaxErrors timeless <timeless@mozdev.org> parents: 28920 diff changeset	251 ... modulename, f, localmods))
25174 86298718b01c import-checker: make imported_modules yield absolute dotted_name_of_path FUJIWARA Katsunori <foozy@lares.dti.ne.jp> parents: 25173 diff changeset	252 []
86298718b01c import-checker: make imported_modules yield absolute dotted_name_of_path FUJIWARA Katsunori <foozy@lares.dti.ne.jp> parents: 25173 diff changeset	253 >>> # relative importing
20037 957b43371928 import-checker: ignore nested imports Augie Fackler <raf@durin42.com> parents: 20036 diff changeset	254 >>> sorted(imported_modules(
25174 86298718b01c import-checker: make imported_modules yield absolute dotted_name_of_path FUJIWARA Katsunori <foozy@lares.dti.ne.jp> parents: 25173 diff changeset	255 ... 'import foo1; from bar import bar1',
28921 02ee31a50002 import-checker: track filenames for SyntaxErrors timeless <timeless@mozdev.org> parents: 28920 diff changeset	256 ... modulename, f, localmods))
26964 5abba2c92da3 import-checker: allow import of child modules from package root Yuya Nishihara <yuya@tcha.org> parents: 26956 diff changeset	257 ['foo.bar.bar1', 'foo.foo1']
25174 86298718b01c import-checker: make imported_modules yield absolute dotted_name_of_path FUJIWARA Katsunori <foozy@lares.dti.ne.jp> parents: 25173 diff changeset	258 >>> sorted(imported_modules(
86298718b01c import-checker: make imported_modules yield absolute dotted_name_of_path FUJIWARA Katsunori <foozy@lares.dti.ne.jp> parents: 25173 diff changeset	259 ... 'from bar.bar1 import name1, name2, name3',
28921 02ee31a50002 import-checker: track filenames for SyntaxErrors timeless <timeless@mozdev.org> parents: 28920 diff changeset	260 ... modulename, f, localmods))
25174 86298718b01c import-checker: make imported_modules yield absolute dotted_name_of_path FUJIWARA Katsunori <foozy@lares.dti.ne.jp> parents: 25173 diff changeset	261 ['foo.bar.bar1']
86298718b01c import-checker: make imported_modules yield absolute dotted_name_of_path FUJIWARA Katsunori <foozy@lares.dti.ne.jp> parents: 25173 diff changeset	262 >>> # absolute importing
86298718b01c import-checker: make imported_modules yield absolute dotted_name_of_path FUJIWARA Katsunori <foozy@lares.dti.ne.jp> parents: 25173 diff changeset	263 >>> sorted(imported_modules(
86298718b01c import-checker: make imported_modules yield absolute dotted_name_of_path FUJIWARA Katsunori <foozy@lares.dti.ne.jp> parents: 25173 diff changeset	264 ... 'from baz import baz1, name1',
28921 02ee31a50002 import-checker: track filenames for SyntaxErrors timeless <timeless@mozdev.org> parents: 28920 diff changeset	265 ... modulename, f, localmods))
25174 86298718b01c import-checker: make imported_modules yield absolute dotted_name_of_path FUJIWARA Katsunori <foozy@lares.dti.ne.jp> parents: 25173 diff changeset	266 ['baz.__init__', 'baz.baz1']
86298718b01c import-checker: make imported_modules yield absolute dotted_name_of_path FUJIWARA Katsunori <foozy@lares.dti.ne.jp> parents: 25173 diff changeset	267 >>> # mixed importing, even though it shouldn't be recommended
86298718b01c import-checker: make imported_modules yield absolute dotted_name_of_path FUJIWARA Katsunori <foozy@lares.dti.ne.jp> parents: 25173 diff changeset	268 >>> sorted(imported_modules(
86298718b01c import-checker: make imported_modules yield absolute dotted_name_of_path FUJIWARA Katsunori <foozy@lares.dti.ne.jp> parents: 25173 diff changeset	269 ... 'import stdlib, foo1, baz',
28921 02ee31a50002 import-checker: track filenames for SyntaxErrors timeless <timeless@mozdev.org> parents: 28920 diff changeset	270 ... modulename, f, localmods))
25174 86298718b01c import-checker: make imported_modules yield absolute dotted_name_of_path FUJIWARA Katsunori <foozy@lares.dti.ne.jp> parents: 25173 diff changeset	271 ['baz.__init__', 'foo.foo1']
86298718b01c import-checker: make imported_modules yield absolute dotted_name_of_path FUJIWARA Katsunori <foozy@lares.dti.ne.jp> parents: 25173 diff changeset	272 >>> # ignore_nested
20037 957b43371928 import-checker: ignore nested imports Augie Fackler <raf@durin42.com> parents: 20036 diff changeset	273 >>> sorted(imported_modules(
957b43371928 import-checker: ignore nested imports Augie Fackler <raf@durin42.com> parents: 20036 diff changeset	274 ... '''import foo
957b43371928 import-checker: ignore nested imports Augie Fackler <raf@durin42.com> parents: 20036 diff changeset	275 ... def wat():
957b43371928 import-checker: ignore nested imports Augie Fackler <raf@durin42.com> parents: 20036 diff changeset	276 ... import bar
28921 02ee31a50002 import-checker: track filenames for SyntaxErrors timeless <timeless@mozdev.org> parents: 28920 diff changeset	277 ... ''', modulename, f, localmods))
25174 86298718b01c import-checker: make imported_modules yield absolute dotted_name_of_path FUJIWARA Katsunori <foozy@lares.dti.ne.jp> parents: 25173 diff changeset	278 ['foo.__init__', 'foo.bar.__init__']
86298718b01c import-checker: make imported_modules yield absolute dotted_name_of_path FUJIWARA Katsunori <foozy@lares.dti.ne.jp> parents: 25173 diff changeset	279 >>> sorted(imported_modules(
86298718b01c import-checker: make imported_modules yield absolute dotted_name_of_path FUJIWARA Katsunori <foozy@lares.dti.ne.jp> parents: 25173 diff changeset	280 ... '''import foo
86298718b01c import-checker: make imported_modules yield absolute dotted_name_of_path FUJIWARA Katsunori <foozy@lares.dti.ne.jp> parents: 25173 diff changeset	281 ... def wat():
86298718b01c import-checker: make imported_modules yield absolute dotted_name_of_path FUJIWARA Katsunori <foozy@lares.dti.ne.jp> parents: 25173 diff changeset	282 ... import bar
28921 02ee31a50002 import-checker: track filenames for SyntaxErrors timeless <timeless@mozdev.org> parents: 28920 diff changeset	283 ... ''', modulename, f, localmods, ignore_nested=True))
25174 86298718b01c import-checker: make imported_modules yield absolute dotted_name_of_path FUJIWARA Katsunori <foozy@lares.dti.ne.jp> parents: 25173 diff changeset	284 ['foo.__init__']
20036 e5d51fa51aba contrib: add an import checker Augie Fackler <raf@durin42.com> parents: diff changeset	285 """
25174 86298718b01c import-checker: make imported_modules yield absolute dotted_name_of_path FUJIWARA Katsunori <foozy@lares.dti.ne.jp> parents: 25173 diff changeset	286 fromlocal = fromlocalfunc(modulename, localmods)
28921 02ee31a50002 import-checker: track filenames for SyntaxErrors timeless <timeless@mozdev.org> parents: 28920 diff changeset	287 for node in ast.walk(ast.parse(source, f)):
20037 957b43371928 import-checker: ignore nested imports Augie Fackler <raf@durin42.com> parents: 20036 diff changeset	288 if ignore_nested and getattr(node, 'col_offset', 0) > 0:
957b43371928 import-checker: ignore nested imports Augie Fackler <raf@durin42.com> parents: 20036 diff changeset	289 continue
20036 e5d51fa51aba contrib: add an import checker Augie Fackler <raf@durin42.com> parents: diff changeset	290 if isinstance(node, ast.Import):
e5d51fa51aba contrib: add an import checker Augie Fackler <raf@durin42.com> parents: diff changeset	291 for n in node.names:
25174 86298718b01c import-checker: make imported_modules yield absolute dotted_name_of_path FUJIWARA Katsunori <foozy@lares.dti.ne.jp> parents: 25173 diff changeset	292 found = fromlocal(n.name)
86298718b01c import-checker: make imported_modules yield absolute dotted_name_of_path FUJIWARA Katsunori <foozy@lares.dti.ne.jp> parents: 25173 diff changeset	293 if not found:
86298718b01c import-checker: make imported_modules yield absolute dotted_name_of_path FUJIWARA Katsunori <foozy@lares.dti.ne.jp> parents: 25173 diff changeset	294 # this should import standard library
86298718b01c import-checker: make imported_modules yield absolute dotted_name_of_path FUJIWARA Katsunori <foozy@lares.dti.ne.jp> parents: 25173 diff changeset	295 continue
86298718b01c import-checker: make imported_modules yield absolute dotted_name_of_path FUJIWARA Katsunori <foozy@lares.dti.ne.jp> parents: 25173 diff changeset	296 yield found[1]
20036 e5d51fa51aba contrib: add an import checker Augie Fackler <raf@durin42.com> parents: diff changeset	297 elif isinstance(node, ast.ImportFrom):
25701 1f88c0f6ff5a import-checker: resolve relative imports Gregory Szorc <gregory.szorc@gmail.com> parents: 25660 diff changeset	298 found = fromlocal(node.module, node.level)
25174 86298718b01c import-checker: make imported_modules yield absolute dotted_name_of_path FUJIWARA Katsunori <foozy@lares.dti.ne.jp> parents: 25173 diff changeset	299 if not found:
86298718b01c import-checker: make imported_modules yield absolute dotted_name_of_path FUJIWARA Katsunori <foozy@lares.dti.ne.jp> parents: 25173 diff changeset	300 # this should import standard library
86298718b01c import-checker: make imported_modules yield absolute dotted_name_of_path FUJIWARA Katsunori <foozy@lares.dti.ne.jp> parents: 25173 diff changeset	301 continue
86298718b01c import-checker: make imported_modules yield absolute dotted_name_of_path FUJIWARA Katsunori <foozy@lares.dti.ne.jp> parents: 25173 diff changeset	302
86298718b01c import-checker: make imported_modules yield absolute dotted_name_of_path FUJIWARA Katsunori <foozy@lares.dti.ne.jp> parents: 25173 diff changeset	303 absname, dottedpath, hassubmod = found
86298718b01c import-checker: make imported_modules yield absolute dotted_name_of_path FUJIWARA Katsunori <foozy@lares.dti.ne.jp> parents: 25173 diff changeset	304 if not hassubmod:
26964 5abba2c92da3 import-checker: allow import of child modules from package root Yuya Nishihara <yuya@tcha.org> parents: 26956 diff changeset	305 # "dottedpath" is not a package; must be imported
5abba2c92da3 import-checker: allow import of child modules from package root Yuya Nishihara <yuya@tcha.org> parents: 26956 diff changeset	306 yield dottedpath
25174 86298718b01c import-checker: make imported_modules yield absolute dotted_name_of_path FUJIWARA Katsunori <foozy@lares.dti.ne.jp> parents: 25173 diff changeset	307 # examination of "node.names" should be redundant
86298718b01c import-checker: make imported_modules yield absolute dotted_name_of_path FUJIWARA Katsunori <foozy@lares.dti.ne.jp> parents: 25173 diff changeset	308 # e.g.: from mercurial.node import nullid, nullrev
86298718b01c import-checker: make imported_modules yield absolute dotted_name_of_path FUJIWARA Katsunori <foozy@lares.dti.ne.jp> parents: 25173 diff changeset	309 continue
86298718b01c import-checker: make imported_modules yield absolute dotted_name_of_path FUJIWARA Katsunori <foozy@lares.dti.ne.jp> parents: 25173 diff changeset	310
26964 5abba2c92da3 import-checker: allow import of child modules from package root Yuya Nishihara <yuya@tcha.org> parents: 26956 diff changeset	311 modnotfound = False
25174 86298718b01c import-checker: make imported_modules yield absolute dotted_name_of_path FUJIWARA Katsunori <foozy@lares.dti.ne.jp> parents: 25173 diff changeset	312 prefix = absname + '.'
20036 e5d51fa51aba contrib: add an import checker Augie Fackler <raf@durin42.com> parents: diff changeset	313 for n in node.names:
25174 86298718b01c import-checker: make imported_modules yield absolute dotted_name_of_path FUJIWARA Katsunori <foozy@lares.dti.ne.jp> parents: 25173 diff changeset	314 found = fromlocal(prefix + n.name)
86298718b01c import-checker: make imported_modules yield absolute dotted_name_of_path FUJIWARA Katsunori <foozy@lares.dti.ne.jp> parents: 25173 diff changeset	315 if not found:
86298718b01c import-checker: make imported_modules yield absolute dotted_name_of_path FUJIWARA Katsunori <foozy@lares.dti.ne.jp> parents: 25173 diff changeset	316 # this should be a function or a property of "node.module"
26964 5abba2c92da3 import-checker: allow import of child modules from package root Yuya Nishihara <yuya@tcha.org> parents: 26956 diff changeset	317 modnotfound = True
25174 86298718b01c import-checker: make imported_modules yield absolute dotted_name_of_path FUJIWARA Katsunori <foozy@lares.dti.ne.jp> parents: 25173 diff changeset	318 continue
86298718b01c import-checker: make imported_modules yield absolute dotted_name_of_path FUJIWARA Katsunori <foozy@lares.dti.ne.jp> parents: 25173 diff changeset	319 yield found[1]
26964 5abba2c92da3 import-checker: allow import of child modules from package root Yuya Nishihara <yuya@tcha.org> parents: 26956 diff changeset	320 if modnotfound:
5abba2c92da3 import-checker: allow import of child modules from package root Yuya Nishihara <yuya@tcha.org> parents: 26956 diff changeset	321 # "dottedpath" is a package, but imported because of non-module
5abba2c92da3 import-checker: allow import of child modules from package root Yuya Nishihara <yuya@tcha.org> parents: 26956 diff changeset	322 # lookup
5abba2c92da3 import-checker: allow import of child modules from package root Yuya Nishihara <yuya@tcha.org> parents: 26956 diff changeset	323 yield dottedpath
20036 e5d51fa51aba contrib: add an import checker Augie Fackler <raf@durin42.com> parents: diff changeset	324
27272 69308357ecd1 import-checker: allow absolute imports of sub modules from local packages Yuya Nishihara <yuya@tcha.org> parents: 27018 diff changeset	325 def verify_import_convention(module, source, localmods):
25703 1a6a117d0b95 import-checker: establish modern import convention Gregory Szorc <gregory.szorc@gmail.com> parents: 25702 diff changeset	326 """Verify imports match our established coding convention.
1a6a117d0b95 import-checker: establish modern import convention Gregory Szorc <gregory.szorc@gmail.com> parents: 25702 diff changeset	327
1a6a117d0b95 import-checker: establish modern import convention Gregory Szorc <gregory.szorc@gmail.com> parents: 25702 diff changeset	328 We have 2 conventions: legacy and modern. The modern convention is in
1a6a117d0b95 import-checker: establish modern import convention Gregory Szorc <gregory.szorc@gmail.com> parents: 25702 diff changeset	329 effect when using absolute imports.
1a6a117d0b95 import-checker: establish modern import convention Gregory Szorc <gregory.szorc@gmail.com> parents: 25702 diff changeset	330
1a6a117d0b95 import-checker: establish modern import convention Gregory Szorc <gregory.szorc@gmail.com> parents: 25702 diff changeset	331 The legacy convention only looks for mixed imports. The modern convention
1a6a117d0b95 import-checker: establish modern import convention Gregory Szorc <gregory.szorc@gmail.com> parents: 25702 diff changeset	332 is much more thorough.
1a6a117d0b95 import-checker: establish modern import convention Gregory Szorc <gregory.szorc@gmail.com> parents: 25702 diff changeset	333 """
25702 ab2c5163900e import-checker: establish new function for verifying import conventions Gregory Szorc <gregory.szorc@gmail.com> parents: 25701 diff changeset	334 root = ast.parse(source)
25703 1a6a117d0b95 import-checker: establish modern import convention Gregory Szorc <gregory.szorc@gmail.com> parents: 25702 diff changeset	335 absolute = usingabsolute(root)
25702 ab2c5163900e import-checker: establish new function for verifying import conventions Gregory Szorc <gregory.szorc@gmail.com> parents: 25701 diff changeset	336
25703 1a6a117d0b95 import-checker: establish modern import convention Gregory Szorc <gregory.szorc@gmail.com> parents: 25702 diff changeset	337 if absolute:
27272 69308357ecd1 import-checker: allow absolute imports of sub modules from local packages Yuya Nishihara <yuya@tcha.org> parents: 27018 diff changeset	338 return verify_modern_convention(module, root, localmods)
25703 1a6a117d0b95 import-checker: establish modern import convention Gregory Szorc <gregory.szorc@gmail.com> parents: 25702 diff changeset	339 else:
1a6a117d0b95 import-checker: establish modern import convention Gregory Szorc <gregory.szorc@gmail.com> parents: 25702 diff changeset	340 return verify_stdlib_on_own_line(root)
1a6a117d0b95 import-checker: establish modern import convention Gregory Szorc <gregory.szorc@gmail.com> parents: 25702 diff changeset	341
27272 69308357ecd1 import-checker: allow absolute imports of sub modules from local packages Yuya Nishihara <yuya@tcha.org> parents: 27018 diff changeset	342 def verify_modern_convention(module, root, localmods, root_col_offset=0):
25703 1a6a117d0b95 import-checker: establish modern import convention Gregory Szorc <gregory.szorc@gmail.com> parents: 25702 diff changeset	343 """Verify a file conforms to the modern import convention rules.
1a6a117d0b95 import-checker: establish modern import convention Gregory Szorc <gregory.szorc@gmail.com> parents: 25702 diff changeset	344
1a6a117d0b95 import-checker: establish modern import convention Gregory Szorc <gregory.szorc@gmail.com> parents: 25702 diff changeset	345 The rules of the modern convention are:
1a6a117d0b95 import-checker: establish modern import convention Gregory Szorc <gregory.szorc@gmail.com> parents: 25702 diff changeset	346
1a6a117d0b95 import-checker: establish modern import convention Gregory Szorc <gregory.szorc@gmail.com> parents: 25702 diff changeset	347 * Ordering is stdlib followed by local imports. Each group is lexically
1a6a117d0b95 import-checker: establish modern import convention Gregory Szorc <gregory.szorc@gmail.com> parents: 25702 diff changeset	348 sorted.
1a6a117d0b95 import-checker: establish modern import convention Gregory Szorc <gregory.szorc@gmail.com> parents: 25702 diff changeset	349 * Importing multiple modules via "import X, Y" is not allowed: use
1a6a117d0b95 import-checker: establish modern import convention Gregory Szorc <gregory.szorc@gmail.com> parents: 25702 diff changeset	350 separate import statements.
1a6a117d0b95 import-checker: establish modern import convention Gregory Szorc <gregory.szorc@gmail.com> parents: 25702 diff changeset	351 * Importing multiple modules via "from X import ..." is allowed if using
1a6a117d0b95 import-checker: establish modern import convention Gregory Szorc <gregory.szorc@gmail.com> parents: 25702 diff changeset	352 parenthesis and one entry per line.
1a6a117d0b95 import-checker: establish modern import convention Gregory Szorc <gregory.szorc@gmail.com> parents: 25702 diff changeset	353 * Only 1 relative import statement per import level ("from .", "from ..")
1a6a117d0b95 import-checker: establish modern import convention Gregory Szorc <gregory.szorc@gmail.com> parents: 25702 diff changeset	354 is allowed.
1a6a117d0b95 import-checker: establish modern import convention Gregory Szorc <gregory.szorc@gmail.com> parents: 25702 diff changeset	355 * Relative imports from higher levels must occur before lower levels. e.g.
1a6a117d0b95 import-checker: establish modern import convention Gregory Szorc <gregory.szorc@gmail.com> parents: 25702 diff changeset	356 "from .." must be before "from .".
1a6a117d0b95 import-checker: establish modern import convention Gregory Szorc <gregory.szorc@gmail.com> parents: 25702 diff changeset	357 * Imports from peer packages should use relative import (e.g. do not
1a6a117d0b95 import-checker: establish modern import convention Gregory Szorc <gregory.szorc@gmail.com> parents: 25702 diff changeset	358 "import mercurial.foo" from a "mercurial.*" module).
1a6a117d0b95 import-checker: establish modern import convention Gregory Szorc <gregory.szorc@gmail.com> parents: 25702 diff changeset	359 * Symbols can only be imported from specific modules (see
1a6a117d0b95 import-checker: establish modern import convention Gregory Szorc <gregory.szorc@gmail.com> parents: 25702 diff changeset	360 `allowsymbolimports`). For other modules, first import the module then
1a6a117d0b95 import-checker: establish modern import convention Gregory Szorc <gregory.szorc@gmail.com> parents: 25702 diff changeset	361 assign the symbol to a module-level variable. In addition, these imports
1a6a117d0b95 import-checker: establish modern import convention Gregory Szorc <gregory.szorc@gmail.com> parents: 25702 diff changeset	362 must be performed before other relative imports. This rule only
1a6a117d0b95 import-checker: establish modern import convention Gregory Szorc <gregory.szorc@gmail.com> parents: 25702 diff changeset	363 applies to import statements outside of any blocks.
1a6a117d0b95 import-checker: establish modern import convention Gregory Szorc <gregory.szorc@gmail.com> parents: 25702 diff changeset	364 * Relative imports from the standard library are not allowed.
1a6a117d0b95 import-checker: establish modern import convention Gregory Szorc <gregory.szorc@gmail.com> parents: 25702 diff changeset	365 * Certain modules must be aliased to alternate names to avoid aliasing
1a6a117d0b95 import-checker: establish modern import convention Gregory Szorc <gregory.szorc@gmail.com> parents: 25702 diff changeset	366 and readability problems. See `requirealias`.
1a6a117d0b95 import-checker: establish modern import convention Gregory Szorc <gregory.szorc@gmail.com> parents: 25702 diff changeset	367 """
1a6a117d0b95 import-checker: establish modern import convention Gregory Szorc <gregory.szorc@gmail.com> parents: 25702 diff changeset	368 topmodule = module.split('.')[0]
27272 69308357ecd1 import-checker: allow absolute imports of sub modules from local packages Yuya Nishihara <yuya@tcha.org> parents: 27018 diff changeset	369 fromlocal = fromlocalfunc(module, localmods)
25703 1a6a117d0b95 import-checker: establish modern import convention Gregory Szorc <gregory.szorc@gmail.com> parents: 25702 diff changeset	370
1a6a117d0b95 import-checker: establish modern import convention Gregory Szorc <gregory.szorc@gmail.com> parents: 25702 diff changeset	371 # Whether a local/non-stdlib import has been performed.
28330 f3fb24e36d61 import-checker: report local with stdlib late warning timeless <timeless@mozdev.org> parents: 27621 diff changeset	372 seenlocal = None
25703 1a6a117d0b95 import-checker: establish modern import convention Gregory Szorc <gregory.szorc@gmail.com> parents: 25702 diff changeset	373 # Whether a relative, non-symbol import has been seen.
1a6a117d0b95 import-checker: establish modern import convention Gregory Szorc <gregory.szorc@gmail.com> parents: 25702 diff changeset	374 seennonsymbolrelative = False
1a6a117d0b95 import-checker: establish modern import convention Gregory Szorc <gregory.szorc@gmail.com> parents: 25702 diff changeset	375 # The last name to be imported (for sorting).
1a6a117d0b95 import-checker: establish modern import convention Gregory Szorc <gregory.szorc@gmail.com> parents: 25702 diff changeset	376 lastname = None
1a6a117d0b95 import-checker: establish modern import convention Gregory Szorc <gregory.szorc@gmail.com> parents: 25702 diff changeset	377 # Relative import levels encountered so far.
1a6a117d0b95 import-checker: establish modern import convention Gregory Szorc <gregory.szorc@gmail.com> parents: 25702 diff changeset	378 seenlevels = set()
1a6a117d0b95 import-checker: establish modern import convention Gregory Szorc <gregory.szorc@gmail.com> parents: 25702 diff changeset	379
26965 1fa66d3ad28d import-checker: reset context to verify convention in function scope Yuya Nishihara <yuya@tcha.org> parents: 26964 diff changeset	380 for node, newscope in walklocal(root):
26955 c4114e335b49 import-checker: extract function to generate a formatted warning Yuya Nishihara <yuya@tcha.org> parents: 26954 diff changeset	381 def msg(fmt, *args):
26956 4b56214ebb7a import-checker: include lineno in warning message Yuya Nishihara <yuya@tcha.org> parents: 26955 diff changeset	382 return (fmt % args, node.lineno)
26965 1fa66d3ad28d import-checker: reset context to verify convention in function scope Yuya Nishihara <yuya@tcha.org> parents: 26964 diff changeset	383 if newscope:
1fa66d3ad28d import-checker: reset context to verify convention in function scope Yuya Nishihara <yuya@tcha.org> parents: 26964 diff changeset	384 # Check for local imports in function
27272 69308357ecd1 import-checker: allow absolute imports of sub modules from local packages Yuya Nishihara <yuya@tcha.org> parents: 27018 diff changeset	385 for r in verify_modern_convention(module, node, localmods,
26965 1fa66d3ad28d import-checker: reset context to verify convention in function scope Yuya Nishihara <yuya@tcha.org> parents: 26964 diff changeset	386 node.col_offset + 4):
1fa66d3ad28d import-checker: reset context to verify convention in function scope Yuya Nishihara <yuya@tcha.org> parents: 26964 diff changeset	387 yield r
1fa66d3ad28d import-checker: reset context to verify convention in function scope Yuya Nishihara <yuya@tcha.org> parents: 26964 diff changeset	388 elif isinstance(node, ast.Import):
25703 1a6a117d0b95 import-checker: establish modern import convention Gregory Szorc <gregory.szorc@gmail.com> parents: 25702 diff changeset	389 # Disallow "import foo, bar" and require separate imports
1a6a117d0b95 import-checker: establish modern import convention Gregory Szorc <gregory.szorc@gmail.com> parents: 25702 diff changeset	390 # for each module.
1a6a117d0b95 import-checker: establish modern import convention Gregory Szorc <gregory.szorc@gmail.com> parents: 25702 diff changeset	391 if len(node.names) > 1:
26955 c4114e335b49 import-checker: extract function to generate a formatted warning Yuya Nishihara <yuya@tcha.org> parents: 26954 diff changeset	392 yield msg('multiple imported names: %s',
c4114e335b49 import-checker: extract function to generate a formatted warning Yuya Nishihara <yuya@tcha.org> parents: 26954 diff changeset	393 ', '.join(n.name for n in node.names))
25703 1a6a117d0b95 import-checker: establish modern import convention Gregory Szorc <gregory.szorc@gmail.com> parents: 25702 diff changeset	394
1a6a117d0b95 import-checker: establish modern import convention Gregory Szorc <gregory.szorc@gmail.com> parents: 25702 diff changeset	395 name = node.names[0].name
1a6a117d0b95 import-checker: establish modern import convention Gregory Szorc <gregory.szorc@gmail.com> parents: 25702 diff changeset	396 asname = node.names[0].asname
1a6a117d0b95 import-checker: establish modern import convention Gregory Szorc <gregory.szorc@gmail.com> parents: 25702 diff changeset	397
1a6a117d0b95 import-checker: establish modern import convention Gregory Szorc <gregory.szorc@gmail.com> parents: 25702 diff changeset	398 # Ignore sorting rules on imports inside blocks.
26965 1fa66d3ad28d import-checker: reset context to verify convention in function scope Yuya Nishihara <yuya@tcha.org> parents: 26964 diff changeset	399 if node.col_offset == root_col_offset:
25703 1a6a117d0b95 import-checker: establish modern import convention Gregory Szorc <gregory.szorc@gmail.com> parents: 25702 diff changeset	400 if lastname and name < lastname:
26955 c4114e335b49 import-checker: extract function to generate a formatted warning Yuya Nishihara <yuya@tcha.org> parents: 26954 diff changeset	401 yield msg('imports not lexically sorted: %s < %s',
c4114e335b49 import-checker: extract function to generate a formatted warning Yuya Nishihara <yuya@tcha.org> parents: 26954 diff changeset	402 name, lastname)
25703 1a6a117d0b95 import-checker: establish modern import convention Gregory Szorc <gregory.szorc@gmail.com> parents: 25702 diff changeset	403
1a6a117d0b95 import-checker: establish modern import convention Gregory Szorc <gregory.szorc@gmail.com> parents: 25702 diff changeset	404 lastname = name
1a6a117d0b95 import-checker: establish modern import convention Gregory Szorc <gregory.szorc@gmail.com> parents: 25702 diff changeset	405
1a6a117d0b95 import-checker: establish modern import convention Gregory Szorc <gregory.szorc@gmail.com> parents: 25702 diff changeset	406 # stdlib imports should be before local imports.
1a6a117d0b95 import-checker: establish modern import convention Gregory Szorc <gregory.szorc@gmail.com> parents: 25702 diff changeset	407 stdlib = name in stdlib_modules
26965 1fa66d3ad28d import-checker: reset context to verify convention in function scope Yuya Nishihara <yuya@tcha.org> parents: 26964 diff changeset	408 if stdlib and seenlocal and node.col_offset == root_col_offset:
28330 f3fb24e36d61 import-checker: report local with stdlib late warning timeless <timeless@mozdev.org> parents: 27621 diff changeset	409 yield msg('stdlib import "%s" follows local import: %s',
f3fb24e36d61 import-checker: report local with stdlib late warning timeless <timeless@mozdev.org> parents: 27621 diff changeset	410 name, seenlocal)
25703 1a6a117d0b95 import-checker: establish modern import convention Gregory Szorc <gregory.szorc@gmail.com> parents: 25702 diff changeset	411
1a6a117d0b95 import-checker: establish modern import convention Gregory Szorc <gregory.szorc@gmail.com> parents: 25702 diff changeset	412 if not stdlib:
28330 f3fb24e36d61 import-checker: report local with stdlib late warning timeless <timeless@mozdev.org> parents: 27621 diff changeset	413 seenlocal = name
25703 1a6a117d0b95 import-checker: establish modern import convention Gregory Szorc <gregory.szorc@gmail.com> parents: 25702 diff changeset	414
1a6a117d0b95 import-checker: establish modern import convention Gregory Szorc <gregory.szorc@gmail.com> parents: 25702 diff changeset	415 # Import of sibling modules should use relative imports.
1a6a117d0b95 import-checker: establish modern import convention Gregory Szorc <gregory.szorc@gmail.com> parents: 25702 diff changeset	416 topname = name.split('.')[0]
1a6a117d0b95 import-checker: establish modern import convention Gregory Szorc <gregory.szorc@gmail.com> parents: 25702 diff changeset	417 if topname == topmodule:
26955 c4114e335b49 import-checker: extract function to generate a formatted warning Yuya Nishihara <yuya@tcha.org> parents: 26954 diff changeset	418 yield msg('import should be relative: %s', name)
25703 1a6a117d0b95 import-checker: establish modern import convention Gregory Szorc <gregory.szorc@gmail.com> parents: 25702 diff changeset	419
1a6a117d0b95 import-checker: establish modern import convention Gregory Szorc <gregory.szorc@gmail.com> parents: 25702 diff changeset	420 if name in requirealias and asname != requirealias[name]:
26955 c4114e335b49 import-checker: extract function to generate a formatted warning Yuya Nishihara <yuya@tcha.org> parents: 26954 diff changeset	421 yield msg('%s module must be "as" aliased to %s',
c4114e335b49 import-checker: extract function to generate a formatted warning Yuya Nishihara <yuya@tcha.org> parents: 26954 diff changeset	422 name, requirealias[name])
25703 1a6a117d0b95 import-checker: establish modern import convention Gregory Szorc <gregory.szorc@gmail.com> parents: 25702 diff changeset	423
1a6a117d0b95 import-checker: establish modern import convention Gregory Szorc <gregory.szorc@gmail.com> parents: 25702 diff changeset	424 elif isinstance(node, ast.ImportFrom):
1a6a117d0b95 import-checker: establish modern import convention Gregory Szorc <gregory.szorc@gmail.com> parents: 25702 diff changeset	425 # Resolve the full imported module name.
1a6a117d0b95 import-checker: establish modern import convention Gregory Szorc <gregory.szorc@gmail.com> parents: 25702 diff changeset	426 if node.level > 0:
1a6a117d0b95 import-checker: establish modern import convention Gregory Szorc <gregory.szorc@gmail.com> parents: 25702 diff changeset	427 fullname = '.'.join(module.split('.')[:-node.level])
1a6a117d0b95 import-checker: establish modern import convention Gregory Szorc <gregory.szorc@gmail.com> parents: 25702 diff changeset	428 if node.module:
1a6a117d0b95 import-checker: establish modern import convention Gregory Szorc <gregory.szorc@gmail.com> parents: 25702 diff changeset	429 fullname += '.%s' % node.module
1a6a117d0b95 import-checker: establish modern import convention Gregory Szorc <gregory.szorc@gmail.com> parents: 25702 diff changeset	430 else:
1a6a117d0b95 import-checker: establish modern import convention Gregory Szorc <gregory.szorc@gmail.com> parents: 25702 diff changeset	431 assert node.module
1a6a117d0b95 import-checker: establish modern import convention Gregory Szorc <gregory.szorc@gmail.com> parents: 25702 diff changeset	432 fullname = node.module
1a6a117d0b95 import-checker: establish modern import convention Gregory Szorc <gregory.szorc@gmail.com> parents: 25702 diff changeset	433
1a6a117d0b95 import-checker: establish modern import convention Gregory Szorc <gregory.szorc@gmail.com> parents: 25702 diff changeset	434 topname = fullname.split('.')[0]
1a6a117d0b95 import-checker: establish modern import convention Gregory Szorc <gregory.szorc@gmail.com> parents: 25702 diff changeset	435 if topname == topmodule:
26955 c4114e335b49 import-checker: extract function to generate a formatted warning Yuya Nishihara <yuya@tcha.org> parents: 26954 diff changeset	436 yield msg('import should be relative: %s', fullname)
25703 1a6a117d0b95 import-checker: establish modern import convention Gregory Szorc <gregory.szorc@gmail.com> parents: 25702 diff changeset	437
1a6a117d0b95 import-checker: establish modern import convention Gregory Szorc <gregory.szorc@gmail.com> parents: 25702 diff changeset	438 # __future__ is special since it needs to come first and use
1a6a117d0b95 import-checker: establish modern import convention Gregory Szorc <gregory.szorc@gmail.com> parents: 25702 diff changeset	439 # symbol import.
1a6a117d0b95 import-checker: establish modern import convention Gregory Szorc <gregory.szorc@gmail.com> parents: 25702 diff changeset	440 if fullname != '__future__':
1a6a117d0b95 import-checker: establish modern import convention Gregory Szorc <gregory.szorc@gmail.com> parents: 25702 diff changeset	441 if not fullname or fullname in stdlib_modules:
26955 c4114e335b49 import-checker: extract function to generate a formatted warning Yuya Nishihara <yuya@tcha.org> parents: 26954 diff changeset	442 yield msg('relative import of stdlib module')
25703 1a6a117d0b95 import-checker: establish modern import convention Gregory Szorc <gregory.szorc@gmail.com> parents: 25702 diff changeset	443 else:
28330 f3fb24e36d61 import-checker: report local with stdlib late warning timeless <timeless@mozdev.org> parents: 27621 diff changeset	444 seenlocal = fullname
25703 1a6a117d0b95 import-checker: establish modern import convention Gregory Szorc <gregory.szorc@gmail.com> parents: 25702 diff changeset	445
1a6a117d0b95 import-checker: establish modern import convention Gregory Szorc <gregory.szorc@gmail.com> parents: 25702 diff changeset	446 # Direct symbol import is only allowed from certain modules and
1a6a117d0b95 import-checker: establish modern import convention Gregory Szorc <gregory.szorc@gmail.com> parents: 25702 diff changeset	447 # must occur before non-symbol imports.
26965 1fa66d3ad28d import-checker: reset context to verify convention in function scope Yuya Nishihara <yuya@tcha.org> parents: 26964 diff changeset	448 if node.module and node.col_offset == root_col_offset:
27272 69308357ecd1 import-checker: allow absolute imports of sub modules from local packages Yuya Nishihara <yuya@tcha.org> parents: 27018 diff changeset	449 found = fromlocal(node.module, node.level)
69308357ecd1 import-checker: allow absolute imports of sub modules from local packages Yuya Nishihara <yuya@tcha.org> parents: 27018 diff changeset	450 if found and found[2]: # node.module is a package
69308357ecd1 import-checker: allow absolute imports of sub modules from local packages Yuya Nishihara <yuya@tcha.org> parents: 27018 diff changeset	451 prefix = found[0] + '.'
69308357ecd1 import-checker: allow absolute imports of sub modules from local packages Yuya Nishihara <yuya@tcha.org> parents: 27018 diff changeset	452 symbols = [n.name for n in node.names
69308357ecd1 import-checker: allow absolute imports of sub modules from local packages Yuya Nishihara <yuya@tcha.org> parents: 27018 diff changeset	453 if not fromlocal(prefix + n.name)]
69308357ecd1 import-checker: allow absolute imports of sub modules from local packages Yuya Nishihara <yuya@tcha.org> parents: 27018 diff changeset	454 else:
69308357ecd1 import-checker: allow absolute imports of sub modules from local packages Yuya Nishihara <yuya@tcha.org> parents: 27018 diff changeset	455 symbols = [n.name for n in node.names]
69308357ecd1 import-checker: allow absolute imports of sub modules from local packages Yuya Nishihara <yuya@tcha.org> parents: 27018 diff changeset	456
69308357ecd1 import-checker: allow absolute imports of sub modules from local packages Yuya Nishihara <yuya@tcha.org> parents: 27018 diff changeset	457 if symbols and fullname not in allowsymbolimports:
27273 5d5b98346fc2 import-checker: tell which symbol causes "direct symbol import" Yuya Nishihara <yuya@tcha.org> parents: 27272 diff changeset	458 yield msg('direct symbol import %s from %s',
5d5b98346fc2 import-checker: tell which symbol causes "direct symbol import" Yuya Nishihara <yuya@tcha.org> parents: 27272 diff changeset	459 ', '.join(symbols), fullname)
25703 1a6a117d0b95 import-checker: establish modern import convention Gregory Szorc <gregory.szorc@gmail.com> parents: 25702 diff changeset	460
27272 69308357ecd1 import-checker: allow absolute imports of sub modules from local packages Yuya Nishihara <yuya@tcha.org> parents: 27018 diff changeset	461 if symbols and seennonsymbolrelative:
26955 c4114e335b49 import-checker: extract function to generate a formatted warning Yuya Nishihara <yuya@tcha.org> parents: 26954 diff changeset	462 yield msg('symbol import follows non-symbol import: %s',
c4114e335b49 import-checker: extract function to generate a formatted warning Yuya Nishihara <yuya@tcha.org> parents: 26954 diff changeset	463 fullname)
25703 1a6a117d0b95 import-checker: establish modern import convention Gregory Szorc <gregory.szorc@gmail.com> parents: 25702 diff changeset	464
1a6a117d0b95 import-checker: establish modern import convention Gregory Szorc <gregory.szorc@gmail.com> parents: 25702 diff changeset	465 if not node.module:
1a6a117d0b95 import-checker: establish modern import convention Gregory Szorc <gregory.szorc@gmail.com> parents: 25702 diff changeset	466 assert node.level
1a6a117d0b95 import-checker: establish modern import convention Gregory Szorc <gregory.szorc@gmail.com> parents: 25702 diff changeset	467 seennonsymbolrelative = True
1a6a117d0b95 import-checker: establish modern import convention Gregory Szorc <gregory.szorc@gmail.com> parents: 25702 diff changeset	468
1a6a117d0b95 import-checker: establish modern import convention Gregory Szorc <gregory.szorc@gmail.com> parents: 25702 diff changeset	469 # Only allow 1 group per level.
26965 1fa66d3ad28d import-checker: reset context to verify convention in function scope Yuya Nishihara <yuya@tcha.org> parents: 26964 diff changeset	470 if (node.level in seenlevels
1fa66d3ad28d import-checker: reset context to verify convention in function scope Yuya Nishihara <yuya@tcha.org> parents: 26964 diff changeset	471 and node.col_offset == root_col_offset):
26955 c4114e335b49 import-checker: extract function to generate a formatted warning Yuya Nishihara <yuya@tcha.org> parents: 26954 diff changeset	472 yield msg('multiple "from %s import" statements',
c4114e335b49 import-checker: extract function to generate a formatted warning Yuya Nishihara <yuya@tcha.org> parents: 26954 diff changeset	473 '.' * node.level)
25703 1a6a117d0b95 import-checker: establish modern import convention Gregory Szorc <gregory.szorc@gmail.com> parents: 25702 diff changeset	474
1a6a117d0b95 import-checker: establish modern import convention Gregory Szorc <gregory.szorc@gmail.com> parents: 25702 diff changeset	475 # Higher-level groups come before lower-level groups.
1a6a117d0b95 import-checker: establish modern import convention Gregory Szorc <gregory.szorc@gmail.com> parents: 25702 diff changeset	476 if any(node.level > l for l in seenlevels):
26955 c4114e335b49 import-checker: extract function to generate a formatted warning Yuya Nishihara <yuya@tcha.org> parents: 26954 diff changeset	477 yield msg('higher-level import should come first: %s',
c4114e335b49 import-checker: extract function to generate a formatted warning Yuya Nishihara <yuya@tcha.org> parents: 26954 diff changeset	478 fullname)
25703 1a6a117d0b95 import-checker: establish modern import convention Gregory Szorc <gregory.szorc@gmail.com> parents: 25702 diff changeset	479
1a6a117d0b95 import-checker: establish modern import convention Gregory Szorc <gregory.szorc@gmail.com> parents: 25702 diff changeset	480 seenlevels.add(node.level)
1a6a117d0b95 import-checker: establish modern import convention Gregory Szorc <gregory.szorc@gmail.com> parents: 25702 diff changeset	481
1a6a117d0b95 import-checker: establish modern import convention Gregory Szorc <gregory.szorc@gmail.com> parents: 25702 diff changeset	482 # Entries in "from .X import ( ... )" lists must be lexically
1a6a117d0b95 import-checker: establish modern import convention Gregory Szorc <gregory.szorc@gmail.com> parents: 25702 diff changeset	483 # sorted.
1a6a117d0b95 import-checker: establish modern import convention Gregory Szorc <gregory.szorc@gmail.com> parents: 25702 diff changeset	484 lastentryname = None
1a6a117d0b95 import-checker: establish modern import convention Gregory Szorc <gregory.szorc@gmail.com> parents: 25702 diff changeset	485
1a6a117d0b95 import-checker: establish modern import convention Gregory Szorc <gregory.szorc@gmail.com> parents: 25702 diff changeset	486 for n in node.names:
1a6a117d0b95 import-checker: establish modern import convention Gregory Szorc <gregory.szorc@gmail.com> parents: 25702 diff changeset	487 if lastentryname and n.name < lastentryname:
26955 c4114e335b49 import-checker: extract function to generate a formatted warning Yuya Nishihara <yuya@tcha.org> parents: 26954 diff changeset	488 yield msg('imports from %s not lexically sorted: %s < %s',
c4114e335b49 import-checker: extract function to generate a formatted warning Yuya Nishihara <yuya@tcha.org> parents: 26954 diff changeset	489 fullname, n.name, lastentryname)
25703 1a6a117d0b95 import-checker: establish modern import convention Gregory Szorc <gregory.szorc@gmail.com> parents: 25702 diff changeset	490
1a6a117d0b95 import-checker: establish modern import convention Gregory Szorc <gregory.szorc@gmail.com> parents: 25702 diff changeset	491 lastentryname = n.name
1a6a117d0b95 import-checker: establish modern import convention Gregory Szorc <gregory.szorc@gmail.com> parents: 25702 diff changeset	492
1a6a117d0b95 import-checker: establish modern import convention Gregory Szorc <gregory.szorc@gmail.com> parents: 25702 diff changeset	493 if n.name in requirealias and n.asname != requirealias[n.name]:
26955 c4114e335b49 import-checker: extract function to generate a formatted warning Yuya Nishihara <yuya@tcha.org> parents: 26954 diff changeset	494 yield msg('%s from %s must be "as" aliased to %s',
c4114e335b49 import-checker: extract function to generate a formatted warning Yuya Nishihara <yuya@tcha.org> parents: 26954 diff changeset	495 n.name, fullname, requirealias[n.name])
25702 ab2c5163900e import-checker: establish new function for verifying import conventions Gregory Szorc <gregory.szorc@gmail.com> parents: 25701 diff changeset	496
ab2c5163900e import-checker: establish new function for verifying import conventions Gregory Szorc <gregory.szorc@gmail.com> parents: 25701 diff changeset	497 def verify_stdlib_on_own_line(root):
20036 e5d51fa51aba contrib: add an import checker Augie Fackler <raf@durin42.com> parents: diff changeset	498 """Given some python source, verify that stdlib imports are done
e5d51fa51aba contrib: add an import checker Augie Fackler <raf@durin42.com> parents: diff changeset	499 in separate statements from relative local module imports.
e5d51fa51aba contrib: add an import checker Augie Fackler <raf@durin42.com> parents: diff changeset	500
25702 ab2c5163900e import-checker: establish new function for verifying import conventions Gregory Szorc <gregory.szorc@gmail.com> parents: 25701 diff changeset	501 >>> list(verify_stdlib_on_own_line(ast.parse('import sys, foo')))
26956 4b56214ebb7a import-checker: include lineno in warning message Yuya Nishihara <yuya@tcha.org> parents: 26955 diff changeset	502 [('mixed imports\\n stdlib: sys\\n relative: foo', 1)]
25702 ab2c5163900e import-checker: establish new function for verifying import conventions Gregory Szorc <gregory.szorc@gmail.com> parents: 25701 diff changeset	503 >>> list(verify_stdlib_on_own_line(ast.parse('import sys, os')))
20036 e5d51fa51aba contrib: add an import checker Augie Fackler <raf@durin42.com> parents: diff changeset	504 []
25702 ab2c5163900e import-checker: establish new function for verifying import conventions Gregory Szorc <gregory.szorc@gmail.com> parents: 25701 diff changeset	505 >>> list(verify_stdlib_on_own_line(ast.parse('import foo, bar')))
20036 e5d51fa51aba contrib: add an import checker Augie Fackler <raf@durin42.com> parents: diff changeset	506 []
e5d51fa51aba contrib: add an import checker Augie Fackler <raf@durin42.com> parents: diff changeset	507 """
25702 ab2c5163900e import-checker: establish new function for verifying import conventions Gregory Szorc <gregory.szorc@gmail.com> parents: 25701 diff changeset	508 for node in ast.walk(root):
20036 e5d51fa51aba contrib: add an import checker Augie Fackler <raf@durin42.com> parents: diff changeset	509 if isinstance(node, ast.Import):
20386 a05d31b040d7 import-checker: show stdlib and relative imports separately Mads Kiilerich <madski@unity3d.com> parents: 20383 diff changeset	510 from_stdlib = {False: [], True: []}
20036 e5d51fa51aba contrib: add an import checker Augie Fackler <raf@durin42.com> parents: diff changeset	511 for n in node.names:
20386 a05d31b040d7 import-checker: show stdlib and relative imports separately Mads Kiilerich <madski@unity3d.com> parents: 20383 diff changeset	512 from_stdlib[n.name in stdlib_modules].append(n.name)
a05d31b040d7 import-checker: show stdlib and relative imports separately Mads Kiilerich <madski@unity3d.com> parents: 20383 diff changeset	513 if from_stdlib[True] and from_stdlib[False]:
a05d31b040d7 import-checker: show stdlib and relative imports separately Mads Kiilerich <madski@unity3d.com> parents: 20383 diff changeset	514 yield ('mixed imports\n stdlib: %s\n relative: %s' %
a05d31b040d7 import-checker: show stdlib and relative imports separately Mads Kiilerich <madski@unity3d.com> parents: 20383 diff changeset	515 (', '.join(sorted(from_stdlib[True])),
26956 4b56214ebb7a import-checker: include lineno in warning message Yuya Nishihara <yuya@tcha.org> parents: 26955 diff changeset	516 ', '.join(sorted(from_stdlib[False]))), node.lineno)
20036 e5d51fa51aba contrib: add an import checker Augie Fackler <raf@durin42.com> parents: diff changeset	517
e5d51fa51aba contrib: add an import checker Augie Fackler <raf@durin42.com> parents: diff changeset	518 class CircularImport(Exception):
e5d51fa51aba contrib: add an import checker Augie Fackler <raf@durin42.com> parents: diff changeset	519 pass
e5d51fa51aba contrib: add an import checker Augie Fackler <raf@durin42.com> parents: diff changeset	520
24490 fb4639d5268e import-checker: make search algorithm non-recursive breadth-first Matt Mackall <mpm@selenic.com> parents: 24489 diff changeset	521 def checkmod(mod, imports):
fb4639d5268e import-checker: make search algorithm non-recursive breadth-first Matt Mackall <mpm@selenic.com> parents: 24489 diff changeset	522 shortest = {}
fb4639d5268e import-checker: make search algorithm non-recursive breadth-first Matt Mackall <mpm@selenic.com> parents: 24489 diff changeset	523 visit = [[mod]]
fb4639d5268e import-checker: make search algorithm non-recursive breadth-first Matt Mackall <mpm@selenic.com> parents: 24489 diff changeset	524 while visit:
fb4639d5268e import-checker: make search algorithm non-recursive breadth-first Matt Mackall <mpm@selenic.com> parents: 24489 diff changeset	525 path = visit.pop(0)
fb4639d5268e import-checker: make search algorithm non-recursive breadth-first Matt Mackall <mpm@selenic.com> parents: 24489 diff changeset	526 for i in sorted(imports.get(path[-1], [])):
fb4639d5268e import-checker: make search algorithm non-recursive breadth-first Matt Mackall <mpm@selenic.com> parents: 24489 diff changeset	527 if len(path) < shortest.get(i, 1000):
fb4639d5268e import-checker: make search algorithm non-recursive breadth-first Matt Mackall <mpm@selenic.com> parents: 24489 diff changeset	528 shortest[i] = len(path)
fb4639d5268e import-checker: make search algorithm non-recursive breadth-first Matt Mackall <mpm@selenic.com> parents: 24489 diff changeset	529 if i in path:
fb4639d5268e import-checker: make search algorithm non-recursive breadth-first Matt Mackall <mpm@selenic.com> parents: 24489 diff changeset	530 if i == path[0]:
fb4639d5268e import-checker: make search algorithm non-recursive breadth-first Matt Mackall <mpm@selenic.com> parents: 24489 diff changeset	531 raise CircularImport(path)
fb4639d5268e import-checker: make search algorithm non-recursive breadth-first Matt Mackall <mpm@selenic.com> parents: 24489 diff changeset	532 continue
fb4639d5268e import-checker: make search algorithm non-recursive breadth-first Matt Mackall <mpm@selenic.com> parents: 24489 diff changeset	533 visit.append(path + [i])
20036 e5d51fa51aba contrib: add an import checker Augie Fackler <raf@durin42.com> parents: diff changeset	534
20038 c65a6937b828 import-checker: try a little harder to show fewer cycles Augie Fackler <raf@durin42.com> parents: 20037 diff changeset	535 def rotatecycle(cycle):
c65a6937b828 import-checker: try a little harder to show fewer cycles Augie Fackler <raf@durin42.com> parents: 20037 diff changeset	536 """arrange a cycle so that the lexicographically first module listed first
c65a6937b828 import-checker: try a little harder to show fewer cycles Augie Fackler <raf@durin42.com> parents: 20037 diff changeset	537
24488 4b3fc46097f7 import-checker: drop duplicate element from cycle Matt Mackall <mpm@selenic.com> parents: 24487 diff changeset	538 >>> rotatecycle(['foo', 'bar'])
20038 c65a6937b828 import-checker: try a little harder to show fewer cycles Augie Fackler <raf@durin42.com> parents: 20037 diff changeset	539 ['bar', 'foo', 'bar']
c65a6937b828 import-checker: try a little harder to show fewer cycles Augie Fackler <raf@durin42.com> parents: 20037 diff changeset	540 """
c65a6937b828 import-checker: try a little harder to show fewer cycles Augie Fackler <raf@durin42.com> parents: 20037 diff changeset	541 lowest = min(cycle)
c65a6937b828 import-checker: try a little harder to show fewer cycles Augie Fackler <raf@durin42.com> parents: 20037 diff changeset	542 idx = cycle.index(lowest)
24488 4b3fc46097f7 import-checker: drop duplicate element from cycle Matt Mackall <mpm@selenic.com> parents: 24487 diff changeset	543 return cycle[idx:] + cycle[:idx] + [lowest]
20036 e5d51fa51aba contrib: add an import checker Augie Fackler <raf@durin42.com> parents: diff changeset	544
e5d51fa51aba contrib: add an import checker Augie Fackler <raf@durin42.com> parents: diff changeset	545 def find_cycles(imports):
e5d51fa51aba contrib: add an import checker Augie Fackler <raf@durin42.com> parents: diff changeset	546 """Find cycles in an already-loaded import graph.
e5d51fa51aba contrib: add an import checker Augie Fackler <raf@durin42.com> parents: diff changeset	547
25175 10e6c4b7121b import-checker: don't treat modules as relative one if not found FUJIWARA Katsunori <foozy@lares.dti.ne.jp> parents: 25174 diff changeset	548 All module names recorded in `imports` should be absolute one.
10e6c4b7121b import-checker: don't treat modules as relative one if not found FUJIWARA Katsunori <foozy@lares.dti.ne.jp> parents: 25174 diff changeset	549
28703 a274c4f9087a py3: use print_function in import-checker timeless <timeless@mozdev.org> parents: 28702 diff changeset	550 >>> from __future__ import print_function
25175 10e6c4b7121b import-checker: don't treat modules as relative one if not found FUJIWARA Katsunori <foozy@lares.dti.ne.jp> parents: 25174 diff changeset	551 >>> imports = {'top.foo': ['top.bar', 'os.path', 'top.qux'],
10e6c4b7121b import-checker: don't treat modules as relative one if not found FUJIWARA Katsunori <foozy@lares.dti.ne.jp> parents: 25174 diff changeset	552 ... 'top.bar': ['top.baz', 'sys'],
10e6c4b7121b import-checker: don't treat modules as relative one if not found FUJIWARA Katsunori <foozy@lares.dti.ne.jp> parents: 25174 diff changeset	553 ... 'top.baz': ['top.foo'],
10e6c4b7121b import-checker: don't treat modules as relative one if not found FUJIWARA Katsunori <foozy@lares.dti.ne.jp> parents: 25174 diff changeset	554 ... 'top.qux': ['top.foo']}
28703 a274c4f9087a py3: use print_function in import-checker timeless <timeless@mozdev.org> parents: 28702 diff changeset	555 >>> print('\\n'.join(sorted(find_cycles(imports))))
24487 642d245ff537 import-checker: fix rotatecycle Matt Mackall <mpm@selenic.com> parents: 22975 diff changeset	556 top.bar -> top.baz -> top.foo -> top.bar
642d245ff537 import-checker: fix rotatecycle Matt Mackall <mpm@selenic.com> parents: 22975 diff changeset	557 top.foo -> top.qux -> top.foo
20036 e5d51fa51aba contrib: add an import checker Augie Fackler <raf@durin42.com> parents: diff changeset	558 """
24491 784b278b349c import-checker: rotatecycle is actually the canonical cycle key Matt Mackall <mpm@selenic.com> parents: 24490 diff changeset	559 cycles = set()
28704 1fa6fdb72275 py3: handle iter/iterkeys+iteritems python3 divergence in import-checker timeless <timeless@mozdev.org> parents: 28703 diff changeset	560 for mod in sorted(imports.keys()):
20036 e5d51fa51aba contrib: add an import checker Augie Fackler <raf@durin42.com> parents: diff changeset	561 try:
24490 fb4639d5268e import-checker: make search algorithm non-recursive breadth-first Matt Mackall <mpm@selenic.com> parents: 24489 diff changeset	562 checkmod(mod, imports)
25660 328739ea70c3 global: mass rewrite to use modern exception syntax Gregory Szorc <gregory.szorc@gmail.com> parents: 25175 diff changeset	563 except CircularImport as e:
20036 e5d51fa51aba contrib: add an import checker Augie Fackler <raf@durin42.com> parents: diff changeset	564 cycle = e.args[0]
24491 784b278b349c import-checker: rotatecycle is actually the canonical cycle key Matt Mackall <mpm@selenic.com> parents: 24490 diff changeset	565 cycles.add(" -> ".join(rotatecycle(cycle)))
784b278b349c import-checker: rotatecycle is actually the canonical cycle key Matt Mackall <mpm@selenic.com> parents: 24490 diff changeset	566 return cycles
20036 e5d51fa51aba contrib: add an import checker Augie Fackler <raf@durin42.com> parents: diff changeset	567
e5d51fa51aba contrib: add an import checker Augie Fackler <raf@durin42.com> parents: diff changeset	568 def _cycle_sortkey(c):
e5d51fa51aba contrib: add an import checker Augie Fackler <raf@durin42.com> parents: diff changeset	569 return len(c), c
e5d51fa51aba contrib: add an import checker Augie Fackler <raf@durin42.com> parents: diff changeset	570
28919 a94f34306bb9 import-checker: refactor source reading timeless <timeless@mozdev.org> parents: 28713 diff changeset	571 def sources(f, modname):
a94f34306bb9 import-checker: refactor source reading timeless <timeless@mozdev.org> parents: 28713 diff changeset	572 if f.endswith('.py'):
a94f34306bb9 import-checker: refactor source reading timeless <timeless@mozdev.org> parents: 28713 diff changeset	573 with open(f) as src:
a94f34306bb9 import-checker: refactor source reading timeless <timeless@mozdev.org> parents: 28713 diff changeset	574 yield src.read(), modname
a94f34306bb9 import-checker: refactor source reading timeless <timeless@mozdev.org> parents: 28713 diff changeset	575
20036 e5d51fa51aba contrib: add an import checker Augie Fackler <raf@durin42.com> parents: diff changeset	576 def main(argv):
25063 723e364488f4 import-checker: add xargs like mode FUJIWARA Katsunori <foozy@lares.dti.ne.jp> parents: 24669 diff changeset	577 if len(argv) < 2 or (argv[1] == '-' and len(argv) > 2):
28703 a274c4f9087a py3: use print_function in import-checker timeless <timeless@mozdev.org> parents: 28702 diff changeset	578 print('Usage: %s {-\|file [file] [file] ...}')
20036 e5d51fa51aba contrib: add an import checker Augie Fackler <raf@durin42.com> parents: diff changeset	579 return 1
25063 723e364488f4 import-checker: add xargs like mode FUJIWARA Katsunori <foozy@lares.dti.ne.jp> parents: 24669 diff changeset	580 if argv[1] == '-':
723e364488f4 import-checker: add xargs like mode FUJIWARA Katsunori <foozy@lares.dti.ne.jp> parents: 24669 diff changeset	581 argv = argv[:1]
723e364488f4 import-checker: add xargs like mode FUJIWARA Katsunori <foozy@lares.dti.ne.jp> parents: 24669 diff changeset	582 argv.extend(l.rstrip() for l in sys.stdin.readlines())
25064 3bbbadf69d0a import-checker: loop to get list of locally defined modules at first FUJIWARA Katsunori <foozy@lares.dti.ne.jp> parents: 25063 diff changeset	583 localmods = {}
20036 e5d51fa51aba contrib: add an import checker Augie Fackler <raf@durin42.com> parents: diff changeset	584 used_imports = {}
e5d51fa51aba contrib: add an import checker Augie Fackler <raf@durin42.com> parents: diff changeset	585 any_errors = False
e5d51fa51aba contrib: add an import checker Augie Fackler <raf@durin42.com> parents: diff changeset	586 for source_path in argv[1:]:
25064 3bbbadf69d0a import-checker: loop to get list of locally defined modules at first FUJIWARA Katsunori <foozy@lares.dti.ne.jp> parents: 25063 diff changeset	587 modname = dotted_name_of_path(source_path, trimpure=True)
3bbbadf69d0a import-checker: loop to get list of locally defined modules at first FUJIWARA Katsunori <foozy@lares.dti.ne.jp> parents: 25063 diff changeset	588 localmods[modname] = source_path
28919 a94f34306bb9 import-checker: refactor source reading timeless <timeless@mozdev.org> parents: 28713 diff changeset	589 for localmodname, source_path in sorted(localmods.items()):
a94f34306bb9 import-checker: refactor source reading timeless <timeless@mozdev.org> parents: 28713 diff changeset	590 for src, modname in sources(source_path, localmodname):
28920 cdf331b54eb8 import-checker: track SyntaxErrors timeless <timeless@mozdev.org> parents: 28919 diff changeset	591 try:
cdf331b54eb8 import-checker: track SyntaxErrors timeless <timeless@mozdev.org> parents: 28919 diff changeset	592 used_imports[modname] = sorted(
28921 02ee31a50002 import-checker: track filenames for SyntaxErrors timeless <timeless@mozdev.org> parents: 28920 diff changeset	593 imported_modules(src, modname, source_path, localmods,
28920 cdf331b54eb8 import-checker: track SyntaxErrors timeless <timeless@mozdev.org> parents: 28919 diff changeset	594 ignore_nested=True))
cdf331b54eb8 import-checker: track SyntaxErrors timeless <timeless@mozdev.org> parents: 28919 diff changeset	595 for error, lineno in verify_import_convention(modname, src,
cdf331b54eb8 import-checker: track SyntaxErrors timeless <timeless@mozdev.org> parents: 28919 diff changeset	596 localmods):
cdf331b54eb8 import-checker: track SyntaxErrors timeless <timeless@mozdev.org> parents: 28919 diff changeset	597 any_errors = True
cdf331b54eb8 import-checker: track SyntaxErrors timeless <timeless@mozdev.org> parents: 28919 diff changeset	598 print('%s:%d: %s' % (source_path, lineno, error))
cdf331b54eb8 import-checker: track SyntaxErrors timeless <timeless@mozdev.org> parents: 28919 diff changeset	599 except SyntaxError as e:
cdf331b54eb8 import-checker: track SyntaxErrors timeless <timeless@mozdev.org> parents: 28919 diff changeset	600 print('%s:%d: SyntaxError: %s' %
cdf331b54eb8 import-checker: track SyntaxErrors timeless <timeless@mozdev.org> parents: 28919 diff changeset	601 (source_path, e.lineno, e))
20036 e5d51fa51aba contrib: add an import checker Augie Fackler <raf@durin42.com> parents: diff changeset	602 cycles = find_cycles(used_imports)
e5d51fa51aba contrib: add an import checker Augie Fackler <raf@durin42.com> parents: diff changeset	603 if cycles:
e5d51fa51aba contrib: add an import checker Augie Fackler <raf@durin42.com> parents: diff changeset	604 firstmods = set()
e5d51fa51aba contrib: add an import checker Augie Fackler <raf@durin42.com> parents: diff changeset	605 for c in sorted(cycles, key=_cycle_sortkey):
e5d51fa51aba contrib: add an import checker Augie Fackler <raf@durin42.com> parents: diff changeset	606 first = c.split()[0]
e5d51fa51aba contrib: add an import checker Augie Fackler <raf@durin42.com> parents: diff changeset	607 # As a rough cut, ignore any cycle that starts with the
e5d51fa51aba contrib: add an import checker Augie Fackler <raf@durin42.com> parents: diff changeset	608 # same module as some other cycle. Otherwise we see lots
e5d51fa51aba contrib: add an import checker Augie Fackler <raf@durin42.com> parents: diff changeset	609 # of cycles that are effectively duplicates.
e5d51fa51aba contrib: add an import checker Augie Fackler <raf@durin42.com> parents: diff changeset	610 if first in firstmods:
e5d51fa51aba contrib: add an import checker Augie Fackler <raf@durin42.com> parents: diff changeset	611 continue
28703 a274c4f9087a py3: use print_function in import-checker timeless <timeless@mozdev.org> parents: 28702 diff changeset	612 print('Import cycle:', c)
20036 e5d51fa51aba contrib: add an import checker Augie Fackler <raf@durin42.com> parents: diff changeset	613 firstmods.add(first)
e5d51fa51aba contrib: add an import checker Augie Fackler <raf@durin42.com> parents: diff changeset	614 any_errors = True
25731 cd1daab5d036 import-checker.py: exit with code 0 if no error is detected FUJIWARA Katsunori <foozy@lares.dti.ne.jp> parents: 25703 diff changeset	615 return any_errors != 0
20036 e5d51fa51aba contrib: add an import checker Augie Fackler <raf@durin42.com> parents: diff changeset	616
e5d51fa51aba contrib: add an import checker Augie Fackler <raf@durin42.com> parents: diff changeset	617 if __name__ == '__main__':
e5d51fa51aba contrib: add an import checker Augie Fackler <raf@durin42.com> parents: diff changeset	618 sys.exit(int(main(sys.argv)))

Mercurial > public > mercurial-scm > hg-stable

annotate contrib/import-checker.py @ 28921:02ee31a50002