mercurial-scm/hg: contrib/import-checker.py annotate

annotate contrib/import-checker.py @ 39329:729082bb9938

revlog: split constants into a new `revlogutils.constants` module We want to split some logic out of the main revlog file (the delta computing logic). However, this logic needs access to multiple constants related to the revlog. So we move all revlog related constants into a new module that could be imported from multiple places. We don't copy the file (preserving blame history) because there are only a few moving lines. Also, copying the file would result in annoying merge conflicts with ongoing work from others contributors.

author	Boris Feld <boris.feld@octobus.net>
date	Thu, 16 Aug 2018 02:08:13 +0200
parents	8751d1e2a7ff
children	7288838bec1f

rev	line source
26954 f804bf27439b import-checker: make it executable for convenience Yuya Nishihara <yuya@tcha.org> parents: 26781 diff changeset	1 #!/usr/bin/env python
f804bf27439b import-checker: make it executable for convenience Yuya Nishihara <yuya@tcha.org> parents: 26781 diff changeset	2
28703 a274c4f9087a py3: use print_function in import-checker timeless <timeless@mozdev.org> parents: 28702 diff changeset	3 from __future__ import absolute_import, print_function
28702 e44f671018e3 py3: use absolute_import in import-checker timeless <timeless@mozdev.org> parents: 28700 diff changeset	4
20036 e5d51fa51aba contrib: add an import checker Augie Fackler <raf@durin42.com> parents: diff changeset	5 import ast
26965 1fa66d3ad28d import-checker: reset context to verify convention in function scope Yuya Nishihara <yuya@tcha.org> parents: 26964 diff changeset	6 import collections
20036 e5d51fa51aba contrib: add an import checker Augie Fackler <raf@durin42.com> parents: diff changeset	7 import os
28922 4ec62a084e5c import-checker: parse python code from .t files timeless <timeless@mozdev.org> parents: 28921 diff changeset	8 import re
20036 e5d51fa51aba contrib: add an import checker Augie Fackler <raf@durin42.com> parents: diff changeset	9 import sys
e5d51fa51aba contrib: add an import checker Augie Fackler <raf@durin42.com> parents: diff changeset	10
20198 f5393a9dc4e5 import-checker: make test-module-imports.t work using virtualenv (issue4129) Chris Jerdonek <chris.jerdonek@gmail.com> parents: 20197 diff changeset	11 # Import a minimal set of stdlib modules needed for list_stdlib_modules()
f5393a9dc4e5 import-checker: make test-module-imports.t work using virtualenv (issue4129) Chris Jerdonek <chris.jerdonek@gmail.com> parents: 20197 diff changeset	12 # to work when run from a virtualenv. The modules were chosen empirically
f5393a9dc4e5 import-checker: make test-module-imports.t work using virtualenv (issue4129) Chris Jerdonek <chris.jerdonek@gmail.com> parents: 20197 diff changeset	13 # so that the return value matches the return value without virtualenv.
29211 b42c2a66a698 py3: make contrib/import-checker.py get along with itself Yuya Nishihara <yuya@tcha.org> parents: 29208 diff changeset	14 if True: # disable lexical sorting checks
33876 998fad4b3072 contrib: work around some modules not existing on Py3 in import checker Augie Fackler <raf@durin42.com> parents: 32595 diff changeset	15 try:
998fad4b3072 contrib: work around some modules not existing on Py3 in import checker Augie Fackler <raf@durin42.com> parents: 32595 diff changeset	16 import BaseHTTPServer as basehttpserver
998fad4b3072 contrib: work around some modules not existing on Py3 in import checker Augie Fackler <raf@durin42.com> parents: 32595 diff changeset	17 except ImportError:
998fad4b3072 contrib: work around some modules not existing on Py3 in import checker Augie Fackler <raf@durin42.com> parents: 32595 diff changeset	18 basehttpserver = None
29211 b42c2a66a698 py3: make contrib/import-checker.py get along with itself Yuya Nishihara <yuya@tcha.org> parents: 29208 diff changeset	19 import zlib
20198 f5393a9dc4e5 import-checker: make test-module-imports.t work using virtualenv (issue4129) Chris Jerdonek <chris.jerdonek@gmail.com> parents: 20197 diff changeset	20
25703 1a6a117d0b95 import-checker: establish modern import convention Gregory Szorc <gregory.szorc@gmail.com> parents: 25702 diff changeset	21 # Whitelist of modules that symbols can be directly imported from.
1a6a117d0b95 import-checker: establish modern import convention Gregory Szorc <gregory.szorc@gmail.com> parents: 25702 diff changeset	22 allowsymbolimports = (
1a6a117d0b95 import-checker: establish modern import convention Gregory Szorc <gregory.szorc@gmail.com> parents: 25702 diff changeset	23 '__future__',
33919 5ed0be4d9df9 contrib: add bzrlib to list of packages from which we import symbols Augie Fackler <raf@durin42.com> parents: 33915 diff changeset	24 'bzrlib',
33915 2d64b2f1787b contrib: allow symbol imports from hgclient for tests Augie Fackler <raf@durin42.com> parents: 33897 diff changeset	25 'hgclient',
33893 c9cf69d0c3b9 contrib: allow importing "symbols" from mercurial Augie Fackler <raf@durin42.com> parents: 33890 diff changeset	26 'mercurial',
27018 e5be48dd8215 import-checker: allow symbol imports from hgweb.common and .request Yuya Nishihara <yuya@tcha.org> parents: 26965 diff changeset	27 'mercurial.hgweb.common',
e5be48dd8215 import-checker: allow symbol imports from hgweb.common and .request Yuya Nishihara <yuya@tcha.org> parents: 26965 diff changeset	28 'mercurial.hgweb.request',
25703 1a6a117d0b95 import-checker: establish modern import convention Gregory Szorc <gregory.szorc@gmail.com> parents: 25702 diff changeset	29 'mercurial.i18n',
1a6a117d0b95 import-checker: establish modern import convention Gregory Szorc <gregory.szorc@gmail.com> parents: 25702 diff changeset	30 'mercurial.node',
39329 729082bb9938 revlog: split constants into a new `revlogutils.constants` module Boris Feld <boris.feld@octobus.net> parents: 38797 diff changeset	31 # for revlog to re-export constant to extensions
729082bb9938 revlog: split constants into a new `revlogutils.constants` module Boris Feld <boris.feld@octobus.net> parents: 38797 diff changeset	32 'mercurial.revlogutils.constants',
32507 95085d747db8 import-checker: allow importing symbols from pure modules Yuya Nishihara <yuya@tcha.org> parents: 32420 diff changeset	33 # for cffi modules to re-export pure functions
95085d747db8 import-checker: allow importing symbols from pure modules Yuya Nishihara <yuya@tcha.org> parents: 32420 diff changeset	34 'mercurial.pure.base85',
95085d747db8 import-checker: allow importing symbols from pure modules Yuya Nishihara <yuya@tcha.org> parents: 32420 diff changeset	35 'mercurial.pure.bdiff',
95085d747db8 import-checker: allow importing symbols from pure modules Yuya Nishihara <yuya@tcha.org> parents: 32420 diff changeset	36 'mercurial.pure.mpatch',
95085d747db8 import-checker: allow importing symbols from pure modules Yuya Nishihara <yuya@tcha.org> parents: 32420 diff changeset	37 'mercurial.pure.osutil',
95085d747db8 import-checker: allow importing symbols from pure modules Yuya Nishihara <yuya@tcha.org> parents: 32420 diff changeset	38 'mercurial.pure.parsers',
34395 41401f502c83 tests: disable lints on mercurial/thirdparty Siddharth Agarwal <sid0@fb.com> parents: 34038 diff changeset	39 # third-party imports should be directly imported
41401f502c83 tests: disable lints on mercurial/thirdparty Siddharth Agarwal <sid0@fb.com> parents: 34038 diff changeset	40 'mercurial.thirdparty',
38797 8751d1e2a7ff util: create a context manager to handle timing Martijn Pieters <mj@zopatista.com> parents: 37711 diff changeset	41 'mercurial.thirdparty.attr',
37711 65a23cc8e75b cborutil: implement support for streaming encoding, bytestring decoding Gregory Szorc <gregory.szorc@gmail.com> parents: 37569 diff changeset	42 'mercurial.thirdparty.cbor',
65a23cc8e75b cborutil: implement support for streaming encoding, bytestring decoding Gregory Szorc <gregory.szorc@gmail.com> parents: 37569 diff changeset	43 'mercurial.thirdparty.cbor.cbor2',
37180 922b3fae9c7d setup: register zope.interface packages and compile C extension Gregory Szorc <gregory.szorc@gmail.com> parents: 34395 diff changeset	44 'mercurial.thirdparty.zope',
922b3fae9c7d setup: register zope.interface packages and compile C extension Gregory Szorc <gregory.szorc@gmail.com> parents: 34395 diff changeset	45 'mercurial.thirdparty.zope.interface',
25703 1a6a117d0b95 import-checker: establish modern import convention Gregory Szorc <gregory.szorc@gmail.com> parents: 25702 diff changeset	46 )
1a6a117d0b95 import-checker: establish modern import convention Gregory Szorc <gregory.szorc@gmail.com> parents: 25702 diff changeset	47
32419 d02888308235 import-checker: add a way to directly import certain symbols Siddharth Agarwal <sid0@fb.com> parents: 32374 diff changeset	48 # Whitelist of symbols that can be directly imported.
32420 0906b85bf222 demandimport: move to separate package Siddharth Agarwal <sid0@fb.com> parents: 32419 diff changeset	49 directsymbols = (
0906b85bf222 demandimport: move to separate package Siddharth Agarwal <sid0@fb.com> parents: 32419 diff changeset	50 'demandimport',
0906b85bf222 demandimport: move to separate package Siddharth Agarwal <sid0@fb.com> parents: 32419 diff changeset	51 )
32419 d02888308235 import-checker: add a way to directly import certain symbols Siddharth Agarwal <sid0@fb.com> parents: 32374 diff changeset	52
25703 1a6a117d0b95 import-checker: establish modern import convention Gregory Szorc <gregory.szorc@gmail.com> parents: 25702 diff changeset	53 # Modules that must be aliased because they are commonly confused with
1a6a117d0b95 import-checker: establish modern import convention Gregory Szorc <gregory.szorc@gmail.com> parents: 25702 diff changeset	54 # common variables and can create aliasing and readability issues.
1a6a117d0b95 import-checker: establish modern import convention Gregory Szorc <gregory.szorc@gmail.com> parents: 25702 diff changeset	55 requirealias = {
1a6a117d0b95 import-checker: establish modern import convention Gregory Szorc <gregory.szorc@gmail.com> parents: 25702 diff changeset	56 'ui': 'uimod',
1a6a117d0b95 import-checker: establish modern import convention Gregory Szorc <gregory.szorc@gmail.com> parents: 25702 diff changeset	57 }
1a6a117d0b95 import-checker: establish modern import convention Gregory Szorc <gregory.szorc@gmail.com> parents: 25702 diff changeset	58
1a6a117d0b95 import-checker: establish modern import convention Gregory Szorc <gregory.szorc@gmail.com> parents: 25702 diff changeset	59 def usingabsolute(root):
1a6a117d0b95 import-checker: establish modern import convention Gregory Szorc <gregory.szorc@gmail.com> parents: 25702 diff changeset	60 """Whether absolute imports are being used."""
1a6a117d0b95 import-checker: establish modern import convention Gregory Szorc <gregory.szorc@gmail.com> parents: 25702 diff changeset	61 if sys.version_info[0] >= 3:
1a6a117d0b95 import-checker: establish modern import convention Gregory Szorc <gregory.szorc@gmail.com> parents: 25702 diff changeset	62 return True
1a6a117d0b95 import-checker: establish modern import convention Gregory Szorc <gregory.szorc@gmail.com> parents: 25702 diff changeset	63
1a6a117d0b95 import-checker: establish modern import convention Gregory Szorc <gregory.szorc@gmail.com> parents: 25702 diff changeset	64 for node in ast.walk(root):
1a6a117d0b95 import-checker: establish modern import convention Gregory Szorc <gregory.szorc@gmail.com> parents: 25702 diff changeset	65 if isinstance(node, ast.ImportFrom):
1a6a117d0b95 import-checker: establish modern import convention Gregory Szorc <gregory.szorc@gmail.com> parents: 25702 diff changeset	66 if node.module == '__future__':
1a6a117d0b95 import-checker: establish modern import convention Gregory Szorc <gregory.szorc@gmail.com> parents: 25702 diff changeset	67 for n in node.names:
1a6a117d0b95 import-checker: establish modern import convention Gregory Szorc <gregory.szorc@gmail.com> parents: 25702 diff changeset	68 if n.name == 'absolute_import':
1a6a117d0b95 import-checker: establish modern import convention Gregory Szorc <gregory.szorc@gmail.com> parents: 25702 diff changeset	69 return True
1a6a117d0b95 import-checker: establish modern import convention Gregory Szorc <gregory.szorc@gmail.com> parents: 25702 diff changeset	70
1a6a117d0b95 import-checker: establish modern import convention Gregory Szorc <gregory.szorc@gmail.com> parents: 25702 diff changeset	71 return False
1a6a117d0b95 import-checker: establish modern import convention Gregory Szorc <gregory.szorc@gmail.com> parents: 25702 diff changeset	72
26965 1fa66d3ad28d import-checker: reset context to verify convention in function scope Yuya Nishihara <yuya@tcha.org> parents: 26964 diff changeset	73 def walklocal(root):
1fa66d3ad28d import-checker: reset context to verify convention in function scope Yuya Nishihara <yuya@tcha.org> parents: 26964 diff changeset	74 """Recursively yield all descendant nodes but not in a different scope"""
1fa66d3ad28d import-checker: reset context to verify convention in function scope Yuya Nishihara <yuya@tcha.org> parents: 26964 diff changeset	75 todo = collections.deque(ast.iter_child_nodes(root))
1fa66d3ad28d import-checker: reset context to verify convention in function scope Yuya Nishihara <yuya@tcha.org> parents: 26964 diff changeset	76 yield root, False
1fa66d3ad28d import-checker: reset context to verify convention in function scope Yuya Nishihara <yuya@tcha.org> parents: 26964 diff changeset	77 while todo:
1fa66d3ad28d import-checker: reset context to verify convention in function scope Yuya Nishihara <yuya@tcha.org> parents: 26964 diff changeset	78 node = todo.popleft()
1fa66d3ad28d import-checker: reset context to verify convention in function scope Yuya Nishihara <yuya@tcha.org> parents: 26964 diff changeset	79 newscope = isinstance(node, ast.FunctionDef)
1fa66d3ad28d import-checker: reset context to verify convention in function scope Yuya Nishihara <yuya@tcha.org> parents: 26964 diff changeset	80 if not newscope:
1fa66d3ad28d import-checker: reset context to verify convention in function scope Yuya Nishihara <yuya@tcha.org> parents: 26964 diff changeset	81 todo.extend(ast.iter_child_nodes(node))
1fa66d3ad28d import-checker: reset context to verify convention in function scope Yuya Nishihara <yuya@tcha.org> parents: 26964 diff changeset	82 yield node, newscope
1fa66d3ad28d import-checker: reset context to verify convention in function scope Yuya Nishihara <yuya@tcha.org> parents: 26964 diff changeset	83
32374 194b0f781132 import-checker: drop workaround for pure modules Yuya Nishihara <yuya@tcha.org> parents: 32372 diff changeset	84 def dotted_name_of_path(path):
20036 e5d51fa51aba contrib: add an import checker Augie Fackler <raf@durin42.com> parents: diff changeset	85 """Given a relative path to a source file, return its dotted module name.
e5d51fa51aba contrib: add an import checker Augie Fackler <raf@durin42.com> parents: diff changeset	86
e5d51fa51aba contrib: add an import checker Augie Fackler <raf@durin42.com> parents: diff changeset	87 >>> dotted_name_of_path('mercurial/error.py')
e5d51fa51aba contrib: add an import checker Augie Fackler <raf@durin42.com> parents: diff changeset	88 'mercurial.error'
20383 4990abb4729d import-checker: fix names of dynamically loaded modules Mads Kiilerich <madski@unity3d.com> parents: 20238 diff changeset	89 >>> dotted_name_of_path('zlibmodule.so')
4990abb4729d import-checker: fix names of dynamically loaded modules Mads Kiilerich <madski@unity3d.com> parents: 20238 diff changeset	90 'zlib'
20036 e5d51fa51aba contrib: add an import checker Augie Fackler <raf@durin42.com> parents: diff changeset	91 """
27620 0c60843b55b5 import-checker: normalize directory separator to get module name on Windows Yuya Nishihara <yuya@tcha.org> parents: 27520 diff changeset	92 parts = path.replace(os.sep, '/').split('/')
20391 466e4c574db0 import-checker: handle standard modules with arch in the filename Mads Kiilerich <madski@unity3d.com> parents: 20386 diff changeset	93 parts[-1] = parts[-1].split('.', 1)[0] # remove .py and .so and .ARCH.so
20383 4990abb4729d import-checker: fix names of dynamically loaded modules Mads Kiilerich <madski@unity3d.com> parents: 20238 diff changeset	94 if parts[-1].endswith('module'):
4990abb4729d import-checker: fix names of dynamically loaded modules Mads Kiilerich <madski@unity3d.com> parents: 20238 diff changeset	95 parts[-1] = parts[-1][:-6]
20036 e5d51fa51aba contrib: add an import checker Augie Fackler <raf@durin42.com> parents: diff changeset	96 return '.'.join(parts)
e5d51fa51aba contrib: add an import checker Augie Fackler <raf@durin42.com> parents: diff changeset	97
25173 7358b5d9991e import-checker: add utility to examine what module is imported easily FUJIWARA Katsunori <foozy@lares.dti.ne.jp> parents: 25064 diff changeset	98 def fromlocalfunc(modulename, localmods):
7358b5d9991e import-checker: add utility to examine what module is imported easily FUJIWARA Katsunori <foozy@lares.dti.ne.jp> parents: 25064 diff changeset	99 """Get a function to examine which locally defined module the
7358b5d9991e import-checker: add utility to examine what module is imported easily FUJIWARA Katsunori <foozy@lares.dti.ne.jp> parents: 25064 diff changeset	100 target source imports via a specified name.
7358b5d9991e import-checker: add utility to examine what module is imported easily FUJIWARA Katsunori <foozy@lares.dti.ne.jp> parents: 25064 diff changeset	101
7358b5d9991e import-checker: add utility to examine what module is imported easily FUJIWARA Katsunori <foozy@lares.dti.ne.jp> parents: 25064 diff changeset	102 `modulename` is an `dotted_name_of_path()`-ed source file path,
7358b5d9991e import-checker: add utility to examine what module is imported easily FUJIWARA Katsunori <foozy@lares.dti.ne.jp> parents: 25064 diff changeset	103 which may have `.__init__` at the end of it, of the target source.
7358b5d9991e import-checker: add utility to examine what module is imported easily FUJIWARA Katsunori <foozy@lares.dti.ne.jp> parents: 25064 diff changeset	104
32508 4c712b90c60a import-checker: convert localmods to a set of module names Yuya Nishihara <yuya@tcha.org> parents: 32507 diff changeset	105 `localmods` is a set of absolute `dotted_name_of_path()`-ed source file
4c712b90c60a import-checker: convert localmods to a set of module names Yuya Nishihara <yuya@tcha.org> parents: 32507 diff changeset	106 paths of locally defined (= Mercurial specific) modules.
25173 7358b5d9991e import-checker: add utility to examine what module is imported easily FUJIWARA Katsunori <foozy@lares.dti.ne.jp> parents: 25064 diff changeset	107
7358b5d9991e import-checker: add utility to examine what module is imported easily FUJIWARA Katsunori <foozy@lares.dti.ne.jp> parents: 25064 diff changeset	108 This function assumes that module names not existing in
26781 1aee2ab0f902 spelling: trivial spell checking Mads Kiilerich <madski@unity3d.com> parents: 26221 diff changeset	109 `localmods` are from the Python standard library.
25173 7358b5d9991e import-checker: add utility to examine what module is imported easily FUJIWARA Katsunori <foozy@lares.dti.ne.jp> parents: 25064 diff changeset	110
7358b5d9991e import-checker: add utility to examine what module is imported easily FUJIWARA Katsunori <foozy@lares.dti.ne.jp> parents: 25064 diff changeset	111 This function returns the function, which takes `name` argument,
7358b5d9991e import-checker: add utility to examine what module is imported easily FUJIWARA Katsunori <foozy@lares.dti.ne.jp> parents: 25064 diff changeset	112 and returns `(absname, dottedpath, hassubmod)` tuple if `name`
7358b5d9991e import-checker: add utility to examine what module is imported easily FUJIWARA Katsunori <foozy@lares.dti.ne.jp> parents: 25064 diff changeset	113 matches against locally defined module. Otherwise, it returns
7358b5d9991e import-checker: add utility to examine what module is imported easily FUJIWARA Katsunori <foozy@lares.dti.ne.jp> parents: 25064 diff changeset	114 False.
7358b5d9991e import-checker: add utility to examine what module is imported easily FUJIWARA Katsunori <foozy@lares.dti.ne.jp> parents: 25064 diff changeset	115
7358b5d9991e import-checker: add utility to examine what module is imported easily FUJIWARA Katsunori <foozy@lares.dti.ne.jp> parents: 25064 diff changeset	116 It is assumed that `name` doesn't have `.__init__`.
7358b5d9991e import-checker: add utility to examine what module is imported easily FUJIWARA Katsunori <foozy@lares.dti.ne.jp> parents: 25064 diff changeset	117
7358b5d9991e import-checker: add utility to examine what module is imported easily FUJIWARA Katsunori <foozy@lares.dti.ne.jp> parents: 25064 diff changeset	118 `absname` is an absolute module name of specified `name`
7358b5d9991e import-checker: add utility to examine what module is imported easily FUJIWARA Katsunori <foozy@lares.dti.ne.jp> parents: 25064 diff changeset	119 (e.g. "hgext.convert"). This can be used to compose prefix for sub
7358b5d9991e import-checker: add utility to examine what module is imported easily FUJIWARA Katsunori <foozy@lares.dti.ne.jp> parents: 25064 diff changeset	120 modules or so.
7358b5d9991e import-checker: add utility to examine what module is imported easily FUJIWARA Katsunori <foozy@lares.dti.ne.jp> parents: 25064 diff changeset	121
7358b5d9991e import-checker: add utility to examine what module is imported easily FUJIWARA Katsunori <foozy@lares.dti.ne.jp> parents: 25064 diff changeset	122 `dottedpath` is a `dotted_name_of_path()`-ed source file path
7358b5d9991e import-checker: add utility to examine what module is imported easily FUJIWARA Katsunori <foozy@lares.dti.ne.jp> parents: 25064 diff changeset	123 (e.g. "hgext.convert.__init__") of `name`. This is used to look
7358b5d9991e import-checker: add utility to examine what module is imported easily FUJIWARA Katsunori <foozy@lares.dti.ne.jp> parents: 25064 diff changeset	124 module up in `localmods` again.
7358b5d9991e import-checker: add utility to examine what module is imported easily FUJIWARA Katsunori <foozy@lares.dti.ne.jp> parents: 25064 diff changeset	125
7358b5d9991e import-checker: add utility to examine what module is imported easily FUJIWARA Katsunori <foozy@lares.dti.ne.jp> parents: 25064 diff changeset	126 `hassubmod` is whether it may have sub modules under it (for
7358b5d9991e import-checker: add utility to examine what module is imported easily FUJIWARA Katsunori <foozy@lares.dti.ne.jp> parents: 25064 diff changeset	127 convenient, even though this is also equivalent to "absname !=
7358b5d9991e import-checker: add utility to examine what module is imported easily FUJIWARA Katsunori <foozy@lares.dti.ne.jp> parents: 25064 diff changeset	128 dottednpath")
7358b5d9991e import-checker: add utility to examine what module is imported easily FUJIWARA Katsunori <foozy@lares.dti.ne.jp> parents: 25064 diff changeset	129
32508 4c712b90c60a import-checker: convert localmods to a set of module names Yuya Nishihara <yuya@tcha.org> parents: 32507 diff changeset	130 >>> localmods = {'foo.__init__', 'foo.foo1',
4c712b90c60a import-checker: convert localmods to a set of module names Yuya Nishihara <yuya@tcha.org> parents: 32507 diff changeset	131 ... 'foo.bar.__init__', 'foo.bar.bar1',
4c712b90c60a import-checker: convert localmods to a set of module names Yuya Nishihara <yuya@tcha.org> parents: 32507 diff changeset	132 ... 'baz.__init__', 'baz.baz1'}
25173 7358b5d9991e import-checker: add utility to examine what module is imported easily FUJIWARA Katsunori <foozy@lares.dti.ne.jp> parents: 25064 diff changeset	133 >>> fromlocal = fromlocalfunc('foo.xxx', localmods)
7358b5d9991e import-checker: add utility to examine what module is imported easily FUJIWARA Katsunori <foozy@lares.dti.ne.jp> parents: 25064 diff changeset	134 >>> # relative
7358b5d9991e import-checker: add utility to examine what module is imported easily FUJIWARA Katsunori <foozy@lares.dti.ne.jp> parents: 25064 diff changeset	135 >>> fromlocal('foo1')
7358b5d9991e import-checker: add utility to examine what module is imported easily FUJIWARA Katsunori <foozy@lares.dti.ne.jp> parents: 25064 diff changeset	136 ('foo.foo1', 'foo.foo1', False)
7358b5d9991e import-checker: add utility to examine what module is imported easily FUJIWARA Katsunori <foozy@lares.dti.ne.jp> parents: 25064 diff changeset	137 >>> fromlocal('bar')
7358b5d9991e import-checker: add utility to examine what module is imported easily FUJIWARA Katsunori <foozy@lares.dti.ne.jp> parents: 25064 diff changeset	138 ('foo.bar', 'foo.bar.__init__', True)
7358b5d9991e import-checker: add utility to examine what module is imported easily FUJIWARA Katsunori <foozy@lares.dti.ne.jp> parents: 25064 diff changeset	139 >>> fromlocal('bar.bar1')
7358b5d9991e import-checker: add utility to examine what module is imported easily FUJIWARA Katsunori <foozy@lares.dti.ne.jp> parents: 25064 diff changeset	140 ('foo.bar.bar1', 'foo.bar.bar1', False)
7358b5d9991e import-checker: add utility to examine what module is imported easily FUJIWARA Katsunori <foozy@lares.dti.ne.jp> parents: 25064 diff changeset	141 >>> # absolute
7358b5d9991e import-checker: add utility to examine what module is imported easily FUJIWARA Katsunori <foozy@lares.dti.ne.jp> parents: 25064 diff changeset	142 >>> fromlocal('baz')
7358b5d9991e import-checker: add utility to examine what module is imported easily FUJIWARA Katsunori <foozy@lares.dti.ne.jp> parents: 25064 diff changeset	143 ('baz', 'baz.__init__', True)
7358b5d9991e import-checker: add utility to examine what module is imported easily FUJIWARA Katsunori <foozy@lares.dti.ne.jp> parents: 25064 diff changeset	144 >>> fromlocal('baz.baz1')
7358b5d9991e import-checker: add utility to examine what module is imported easily FUJIWARA Katsunori <foozy@lares.dti.ne.jp> parents: 25064 diff changeset	145 ('baz.baz1', 'baz.baz1', False)
7358b5d9991e import-checker: add utility to examine what module is imported easily FUJIWARA Katsunori <foozy@lares.dti.ne.jp> parents: 25064 diff changeset	146 >>> # unknown = maybe standard library
7358b5d9991e import-checker: add utility to examine what module is imported easily FUJIWARA Katsunori <foozy@lares.dti.ne.jp> parents: 25064 diff changeset	147 >>> fromlocal('os')
7358b5d9991e import-checker: add utility to examine what module is imported easily FUJIWARA Katsunori <foozy@lares.dti.ne.jp> parents: 25064 diff changeset	148 False
25701 1f88c0f6ff5a import-checker: resolve relative imports Gregory Szorc <gregory.szorc@gmail.com> parents: 25660 diff changeset	149 >>> fromlocal(None, 1)
1f88c0f6ff5a import-checker: resolve relative imports Gregory Szorc <gregory.szorc@gmail.com> parents: 25660 diff changeset	150 ('foo', 'foo.__init__', True)
29122 660d8d4ec7aa import-checker: recognize relative imports from parents of current package liscju <piotr.listkiewicz@gmail.com> parents: 28922 diff changeset	151 >>> fromlocal('foo1', 1)
660d8d4ec7aa import-checker: recognize relative imports from parents of current package liscju <piotr.listkiewicz@gmail.com> parents: 28922 diff changeset	152 ('foo.foo1', 'foo.foo1', False)
25701 1f88c0f6ff5a import-checker: resolve relative imports Gregory Szorc <gregory.szorc@gmail.com> parents: 25660 diff changeset	153 >>> fromlocal2 = fromlocalfunc('foo.xxx.yyy', localmods)
1f88c0f6ff5a import-checker: resolve relative imports Gregory Szorc <gregory.szorc@gmail.com> parents: 25660 diff changeset	154 >>> fromlocal2(None, 2)
1f88c0f6ff5a import-checker: resolve relative imports Gregory Szorc <gregory.szorc@gmail.com> parents: 25660 diff changeset	155 ('foo', 'foo.__init__', True)
29122 660d8d4ec7aa import-checker: recognize relative imports from parents of current package liscju <piotr.listkiewicz@gmail.com> parents: 28922 diff changeset	156 >>> fromlocal2('bar2', 1)
660d8d4ec7aa import-checker: recognize relative imports from parents of current package liscju <piotr.listkiewicz@gmail.com> parents: 28922 diff changeset	157 False
660d8d4ec7aa import-checker: recognize relative imports from parents of current package liscju <piotr.listkiewicz@gmail.com> parents: 28922 diff changeset	158 >>> fromlocal2('bar', 2)
660d8d4ec7aa import-checker: recognize relative imports from parents of current package liscju <piotr.listkiewicz@gmail.com> parents: 28922 diff changeset	159 ('foo.bar', 'foo.bar.__init__', True)
25173 7358b5d9991e import-checker: add utility to examine what module is imported easily FUJIWARA Katsunori <foozy@lares.dti.ne.jp> parents: 25064 diff changeset	160 """
33890 3de9a2df6411 contrib: have import-checker work mostly with native strings for mod names Augie Fackler <raf@durin42.com> parents: 33877 diff changeset	161 if not isinstance(modulename, str):
3de9a2df6411 contrib: have import-checker work mostly with native strings for mod names Augie Fackler <raf@durin42.com> parents: 33877 diff changeset	162 modulename = modulename.decode('ascii')
25173 7358b5d9991e import-checker: add utility to examine what module is imported easily FUJIWARA Katsunori <foozy@lares.dti.ne.jp> parents: 25064 diff changeset	163 prefix = '.'.join(modulename.split('.')[:-1])
7358b5d9991e import-checker: add utility to examine what module is imported easily FUJIWARA Katsunori <foozy@lares.dti.ne.jp> parents: 25064 diff changeset	164 if prefix:
7358b5d9991e import-checker: add utility to examine what module is imported easily FUJIWARA Katsunori <foozy@lares.dti.ne.jp> parents: 25064 diff changeset	165 prefix += '.'
25701 1f88c0f6ff5a import-checker: resolve relative imports Gregory Szorc <gregory.szorc@gmail.com> parents: 25660 diff changeset	166 def fromlocal(name, level=0):
29374 7712fcde2d56 import-checker: increase portability for python 2.6.x FUJIWARA Katsunori <foozy@lares.dti.ne.jp> parents: 29234 diff changeset	167 # name is false value when relative imports are used.
7712fcde2d56 import-checker: increase portability for python 2.6.x FUJIWARA Katsunori <foozy@lares.dti.ne.jp> parents: 29234 diff changeset	168 if not name:
25701 1f88c0f6ff5a import-checker: resolve relative imports Gregory Szorc <gregory.szorc@gmail.com> parents: 25660 diff changeset	169 # If relative imports are used, level must not be absolute.
1f88c0f6ff5a import-checker: resolve relative imports Gregory Szorc <gregory.szorc@gmail.com> parents: 25660 diff changeset	170 assert level > 0
1f88c0f6ff5a import-checker: resolve relative imports Gregory Szorc <gregory.szorc@gmail.com> parents: 25660 diff changeset	171 candidates = ['.'.join(modulename.split('.')[:-level])]
1f88c0f6ff5a import-checker: resolve relative imports Gregory Szorc <gregory.szorc@gmail.com> parents: 25660 diff changeset	172 else:
29122 660d8d4ec7aa import-checker: recognize relative imports from parents of current package liscju <piotr.listkiewicz@gmail.com> parents: 28922 diff changeset	173 if not level:
660d8d4ec7aa import-checker: recognize relative imports from parents of current package liscju <piotr.listkiewicz@gmail.com> parents: 28922 diff changeset	174 # Check relative name first.
660d8d4ec7aa import-checker: recognize relative imports from parents of current package liscju <piotr.listkiewicz@gmail.com> parents: 28922 diff changeset	175 candidates = [prefix + name, name]
660d8d4ec7aa import-checker: recognize relative imports from parents of current package liscju <piotr.listkiewicz@gmail.com> parents: 28922 diff changeset	176 else:
660d8d4ec7aa import-checker: recognize relative imports from parents of current package liscju <piotr.listkiewicz@gmail.com> parents: 28922 diff changeset	177 candidates = ['.'.join(modulename.split('.')[:-level]) +
660d8d4ec7aa import-checker: recognize relative imports from parents of current package liscju <piotr.listkiewicz@gmail.com> parents: 28922 diff changeset	178 '.' + name]
25701 1f88c0f6ff5a import-checker: resolve relative imports Gregory Szorc <gregory.szorc@gmail.com> parents: 25660 diff changeset	179
1f88c0f6ff5a import-checker: resolve relative imports Gregory Szorc <gregory.szorc@gmail.com> parents: 25660 diff changeset	180 for n in candidates:
25173 7358b5d9991e import-checker: add utility to examine what module is imported easily FUJIWARA Katsunori <foozy@lares.dti.ne.jp> parents: 25064 diff changeset	181 if n in localmods:
7358b5d9991e import-checker: add utility to examine what module is imported easily FUJIWARA Katsunori <foozy@lares.dti.ne.jp> parents: 25064 diff changeset	182 return (n, n, False)
7358b5d9991e import-checker: add utility to examine what module is imported easily FUJIWARA Katsunori <foozy@lares.dti.ne.jp> parents: 25064 diff changeset	183 dottedpath = n + '.__init__'
7358b5d9991e import-checker: add utility to examine what module is imported easily FUJIWARA Katsunori <foozy@lares.dti.ne.jp> parents: 25064 diff changeset	184 if dottedpath in localmods:
7358b5d9991e import-checker: add utility to examine what module is imported easily FUJIWARA Katsunori <foozy@lares.dti.ne.jp> parents: 25064 diff changeset	185 return (n, dottedpath, True)
7358b5d9991e import-checker: add utility to examine what module is imported easily FUJIWARA Katsunori <foozy@lares.dti.ne.jp> parents: 25064 diff changeset	186 return False
7358b5d9991e import-checker: add utility to examine what module is imported easily FUJIWARA Katsunori <foozy@lares.dti.ne.jp> parents: 25064 diff changeset	187 return fromlocal
20036 e5d51fa51aba contrib: add an import checker Augie Fackler <raf@durin42.com> parents: diff changeset	188
32509 a025ec43856c import-checker: guess names of C extension modules Yuya Nishihara <yuya@tcha.org> parents: 32508 diff changeset	189 def populateextmods(localmods):
a025ec43856c import-checker: guess names of C extension modules Yuya Nishihara <yuya@tcha.org> parents: 32508 diff changeset	190 """Populate C extension modules based on pure modules"""
a025ec43856c import-checker: guess names of C extension modules Yuya Nishihara <yuya@tcha.org> parents: 32508 diff changeset	191 newlocalmods = set(localmods)
a025ec43856c import-checker: guess names of C extension modules Yuya Nishihara <yuya@tcha.org> parents: 32508 diff changeset	192 for n in localmods:
a025ec43856c import-checker: guess names of C extension modules Yuya Nishihara <yuya@tcha.org> parents: 32508 diff changeset	193 if n.startswith('mercurial.pure.'):
a025ec43856c import-checker: guess names of C extension modules Yuya Nishihara <yuya@tcha.org> parents: 32508 diff changeset	194 m = n[len('mercurial.pure.'):]
a025ec43856c import-checker: guess names of C extension modules Yuya Nishihara <yuya@tcha.org> parents: 32508 diff changeset	195 newlocalmods.add('mercurial.cext.' + m)
a025ec43856c import-checker: guess names of C extension modules Yuya Nishihara <yuya@tcha.org> parents: 32508 diff changeset	196 newlocalmods.add('mercurial.cffi._' + m)
a025ec43856c import-checker: guess names of C extension modules Yuya Nishihara <yuya@tcha.org> parents: 32508 diff changeset	197 return newlocalmods
a025ec43856c import-checker: guess names of C extension modules Yuya Nishihara <yuya@tcha.org> parents: 32508 diff changeset	198
20036 e5d51fa51aba contrib: add an import checker Augie Fackler <raf@durin42.com> parents: diff changeset	199 def list_stdlib_modules():
e5d51fa51aba contrib: add an import checker Augie Fackler <raf@durin42.com> parents: diff changeset	200 """List the modules present in the stdlib.
e5d51fa51aba contrib: add an import checker Augie Fackler <raf@durin42.com> parents: diff changeset	201
33876 998fad4b3072 contrib: work around some modules not existing on Py3 in import checker Augie Fackler <raf@durin42.com> parents: 32595 diff changeset	202 >>> py3 = sys.version_info[0] >= 3
20036 e5d51fa51aba contrib: add an import checker Augie Fackler <raf@durin42.com> parents: diff changeset	203 >>> mods = set(list_stdlib_modules())
33876 998fad4b3072 contrib: work around some modules not existing on Py3 in import checker Augie Fackler <raf@durin42.com> parents: 32595 diff changeset	204 >>> 'BaseHTTPServer' in mods or py3
20036 e5d51fa51aba contrib: add an import checker Augie Fackler <raf@durin42.com> parents: diff changeset	205 True
e5d51fa51aba contrib: add an import checker Augie Fackler <raf@durin42.com> parents: diff changeset	206
e5d51fa51aba contrib: add an import checker Augie Fackler <raf@durin42.com> parents: diff changeset	207 os.path isn't really a module, so it's missing:
e5d51fa51aba contrib: add an import checker Augie Fackler <raf@durin42.com> parents: diff changeset	208
e5d51fa51aba contrib: add an import checker Augie Fackler <raf@durin42.com> parents: diff changeset	209 >>> 'os.path' in mods
e5d51fa51aba contrib: add an import checker Augie Fackler <raf@durin42.com> parents: diff changeset	210 False
e5d51fa51aba contrib: add an import checker Augie Fackler <raf@durin42.com> parents: diff changeset	211
e5d51fa51aba contrib: add an import checker Augie Fackler <raf@durin42.com> parents: diff changeset	212 sys requires special treatment, because it's baked into the
e5d51fa51aba contrib: add an import checker Augie Fackler <raf@durin42.com> parents: diff changeset	213 interpreter, but it should still appear:
e5d51fa51aba contrib: add an import checker Augie Fackler <raf@durin42.com> parents: diff changeset	214
e5d51fa51aba contrib: add an import checker Augie Fackler <raf@durin42.com> parents: diff changeset	215 >>> 'sys' in mods
e5d51fa51aba contrib: add an import checker Augie Fackler <raf@durin42.com> parents: diff changeset	216 True
e5d51fa51aba contrib: add an import checker Augie Fackler <raf@durin42.com> parents: diff changeset	217
e5d51fa51aba contrib: add an import checker Augie Fackler <raf@durin42.com> parents: diff changeset	218 >>> 'collections' in mods
e5d51fa51aba contrib: add an import checker Augie Fackler <raf@durin42.com> parents: diff changeset	219 True
e5d51fa51aba contrib: add an import checker Augie Fackler <raf@durin42.com> parents: diff changeset	220
33876 998fad4b3072 contrib: work around some modules not existing on Py3 in import checker Augie Fackler <raf@durin42.com> parents: 32595 diff changeset	221 >>> 'cStringIO' in mods or py3
20036 e5d51fa51aba contrib: add an import checker Augie Fackler <raf@durin42.com> parents: diff changeset	222 True
29395 4c8026babe8c import-checker: ensure cffi is always a system module Augie Fackler <raf@durin42.com> parents: 29374 diff changeset	223
4c8026babe8c import-checker: ensure cffi is always a system module Augie Fackler <raf@durin42.com> parents: 29374 diff changeset	224 >>> 'cffi' in mods
4c8026babe8c import-checker: ensure cffi is always a system module Augie Fackler <raf@durin42.com> parents: 29374 diff changeset	225 True
20036 e5d51fa51aba contrib: add an import checker Augie Fackler <raf@durin42.com> parents: diff changeset	226 """
e5d51fa51aba contrib: add an import checker Augie Fackler <raf@durin42.com> parents: diff changeset	227 for m in sys.builtin_module_names:
e5d51fa51aba contrib: add an import checker Augie Fackler <raf@durin42.com> parents: diff changeset	228 yield m
e5d51fa51aba contrib: add an import checker Augie Fackler <raf@durin42.com> parents: diff changeset	229 # These modules only exist on windows, but we should always
e5d51fa51aba contrib: add an import checker Augie Fackler <raf@durin42.com> parents: diff changeset	230 # consider them stdlib.
e5d51fa51aba contrib: add an import checker Augie Fackler <raf@durin42.com> parents: diff changeset	231 for m in ['msvcrt', '_winreg']:
e5d51fa51aba contrib: add an import checker Augie Fackler <raf@durin42.com> parents: diff changeset	232 yield m
33894 c856cb1c29be contrib: inform import checker that __builtin__ is a thing Augie Fackler <raf@durin42.com> parents: 33893 diff changeset	233 yield '__builtin__'
20036 e5d51fa51aba contrib: add an import checker Augie Fackler <raf@durin42.com> parents: diff changeset	234 yield 'builtins' # python3 only
33897 b4294ff13392 contrib: always treat importlib.* as stdlib Augie Fackler <raf@durin42.com> parents: 33894 diff changeset	235 yield 'importlib.abc' # python3 only
b4294ff13392 contrib: always treat importlib.* as stdlib Augie Fackler <raf@durin42.com> parents: 33894 diff changeset	236 yield 'importlib.machinery' # python3 only
b4294ff13392 contrib: always treat importlib.* as stdlib Augie Fackler <raf@durin42.com> parents: 33894 diff changeset	237 yield 'importlib.util' # python3 only
24669 fbdbff1b486a import-checker: force 'fcntl', 'grp', 'pwd', and 'termios' to stdlib modules Matt Harbison <matt_harbison@yahoo.com> parents: 24668 diff changeset	238 for m in 'fcntl', 'grp', 'pwd', 'termios': # Unix only
fbdbff1b486a import-checker: force 'fcntl', 'grp', 'pwd', and 'termios' to stdlib modules Matt Harbison <matt_harbison@yahoo.com> parents: 24668 diff changeset	239 yield m
28713 806d260c6f3b tests: fix builtin module test on pypy Maciej Fijalkowski <fijall@gmail.com> parents: 28704 diff changeset	240 for m in 'cPickle', 'datetime': # in Python (not C) on PyPy
806d260c6f3b tests: fix builtin module test on pypy Maciej Fijalkowski <fijall@gmail.com> parents: 28704 diff changeset	241 yield m
29395 4c8026babe8c import-checker: ensure cffi is always a system module Augie Fackler <raf@durin42.com> parents: 29374 diff changeset	242 for m in ['cffi']:
4c8026babe8c import-checker: ensure cffi is always a system module Augie Fackler <raf@durin42.com> parents: 29374 diff changeset	243 yield m
32291 bd872f64a8ba cleanup: use set literals Martin von Zweigbergk <martinvonz@google.com> parents: 32212 diff changeset	244 stdlib_prefixes = {sys.prefix, sys.exec_prefix}
20198 f5393a9dc4e5 import-checker: make test-module-imports.t work using virtualenv (issue4129) Chris Jerdonek <chris.jerdonek@gmail.com> parents: 20197 diff changeset	245 # We need to supplement the list of prefixes for the search to work
f5393a9dc4e5 import-checker: make test-module-imports.t work using virtualenv (issue4129) Chris Jerdonek <chris.jerdonek@gmail.com> parents: 20197 diff changeset	246 # when run from within a virtualenv.
33876 998fad4b3072 contrib: work around some modules not existing on Py3 in import checker Augie Fackler <raf@durin42.com> parents: 32595 diff changeset	247 for mod in (basehttpserver, zlib):
998fad4b3072 contrib: work around some modules not existing on Py3 in import checker Augie Fackler <raf@durin42.com> parents: 32595 diff changeset	248 if mod is None:
998fad4b3072 contrib: work around some modules not existing on Py3 in import checker Augie Fackler <raf@durin42.com> parents: 32595 diff changeset	249 continue
20198 f5393a9dc4e5 import-checker: make test-module-imports.t work using virtualenv (issue4129) Chris Jerdonek <chris.jerdonek@gmail.com> parents: 20197 diff changeset	250 try:
f5393a9dc4e5 import-checker: make test-module-imports.t work using virtualenv (issue4129) Chris Jerdonek <chris.jerdonek@gmail.com> parents: 20197 diff changeset	251 # Not all module objects have a __file__ attribute.
f5393a9dc4e5 import-checker: make test-module-imports.t work using virtualenv (issue4129) Chris Jerdonek <chris.jerdonek@gmail.com> parents: 20197 diff changeset	252 filename = mod.__file__
f5393a9dc4e5 import-checker: make test-module-imports.t work using virtualenv (issue4129) Chris Jerdonek <chris.jerdonek@gmail.com> parents: 20197 diff changeset	253 except AttributeError:
f5393a9dc4e5 import-checker: make test-module-imports.t work using virtualenv (issue4129) Chris Jerdonek <chris.jerdonek@gmail.com> parents: 20197 diff changeset	254 continue
f5393a9dc4e5 import-checker: make test-module-imports.t work using virtualenv (issue4129) Chris Jerdonek <chris.jerdonek@gmail.com> parents: 20197 diff changeset	255 dirname = os.path.dirname(filename)
f5393a9dc4e5 import-checker: make test-module-imports.t work using virtualenv (issue4129) Chris Jerdonek <chris.jerdonek@gmail.com> parents: 20197 diff changeset	256 for prefix in stdlib_prefixes:
f5393a9dc4e5 import-checker: make test-module-imports.t work using virtualenv (issue4129) Chris Jerdonek <chris.jerdonek@gmail.com> parents: 20197 diff changeset	257 if dirname.startswith(prefix):
f5393a9dc4e5 import-checker: make test-module-imports.t work using virtualenv (issue4129) Chris Jerdonek <chris.jerdonek@gmail.com> parents: 20197 diff changeset	258 # Then this directory is redundant.
f5393a9dc4e5 import-checker: make test-module-imports.t work using virtualenv (issue4129) Chris Jerdonek <chris.jerdonek@gmail.com> parents: 20197 diff changeset	259 break
f5393a9dc4e5 import-checker: make test-module-imports.t work using virtualenv (issue4129) Chris Jerdonek <chris.jerdonek@gmail.com> parents: 20197 diff changeset	260 else:
f5393a9dc4e5 import-checker: make test-module-imports.t work using virtualenv (issue4129) Chris Jerdonek <chris.jerdonek@gmail.com> parents: 20197 diff changeset	261 stdlib_prefixes.add(dirname)
20036 e5d51fa51aba contrib: add an import checker Augie Fackler <raf@durin42.com> parents: diff changeset	262 for libpath in sys.path:
20201 bc3b48b0f5c8 import-checker: suppress check-code about any() Augie Fackler <raf@durin42.com> parents: 20200 diff changeset	263 # We want to walk everything in sys.path that starts with
28700 35ad5bcdeb7e py24: remove check-code py24 notation timeless <timeless@mozdev.org> parents: 28400 diff changeset	264 # something in stdlib_prefixes.
35ad5bcdeb7e py24: remove check-code py24 notation timeless <timeless@mozdev.org> parents: 28400 diff changeset	265 if not any(libpath.startswith(p) for p in stdlib_prefixes):
20036 e5d51fa51aba contrib: add an import checker Augie Fackler <raf@durin42.com> parents: diff changeset	266 continue
e5d51fa51aba contrib: add an import checker Augie Fackler <raf@durin42.com> parents: diff changeset	267 for top, dirs, files in os.walk(libpath):
25733 f99c066f5f9a import-checker: recurse into subtree of sys.path only if __init__.py exists Yuya Nishihara <yuya@tcha.org> parents: 25731 diff changeset	268 for i, d in reversed(list(enumerate(dirs))):
25734 9086d0c1def3 import-checker: exclude mercurial packages installed into the system path Yuya Nishihara <yuya@tcha.org> parents: 25733 diff changeset	269 if (not os.path.exists(os.path.join(top, d, '__init__.py'))
32595 9e46627baa3c import-checker: add hgdemandimport to local modules Siddharth Agarwal <sid0@fb.com> parents: 32509 diff changeset	270 or top == libpath and d in ('hgdemandimport', 'hgext',
9e46627baa3c import-checker: add hgdemandimport to local modules Siddharth Agarwal <sid0@fb.com> parents: 32509 diff changeset	271 'mercurial')):
25733 f99c066f5f9a import-checker: recurse into subtree of sys.path only if __init__.py exists Yuya Nishihara <yuya@tcha.org> parents: 25731 diff changeset	272 del dirs[i]
20036 e5d51fa51aba contrib: add an import checker Augie Fackler <raf@durin42.com> parents: diff changeset	273 for name in files:
26221 ae65b1b4cb46 import-checker: use modern .endswith for multiple suffixes Augie Fackler <augie@google.com> parents: 26166 diff changeset	274 if not name.endswith(('.py', '.so', '.pyc', '.pyo', '.pyd')):
20036 e5d51fa51aba contrib: add an import checker Augie Fackler <raf@durin42.com> parents: diff changeset	275 continue
27621 39845b064041 import-checker: list package directory as stdlib module Yuya Nishihara <yuya@tcha.org> parents: 27620 diff changeset	276 if name.startswith('__init__.py'):
39845b064041 import-checker: list package directory as stdlib module Yuya Nishihara <yuya@tcha.org> parents: 27620 diff changeset	277 full_path = top
39845b064041 import-checker: list package directory as stdlib module Yuya Nishihara <yuya@tcha.org> parents: 27620 diff changeset	278 else:
39845b064041 import-checker: list package directory as stdlib module Yuya Nishihara <yuya@tcha.org> parents: 27620 diff changeset	279 full_path = os.path.join(top, name)
20036 e5d51fa51aba contrib: add an import checker Augie Fackler <raf@durin42.com> parents: diff changeset	280 rel_path = full_path[len(libpath) + 1:]
e5d51fa51aba contrib: add an import checker Augie Fackler <raf@durin42.com> parents: diff changeset	281 mod = dotted_name_of_path(rel_path)
e5d51fa51aba contrib: add an import checker Augie Fackler <raf@durin42.com> parents: diff changeset	282 yield mod
e5d51fa51aba contrib: add an import checker Augie Fackler <raf@durin42.com> parents: diff changeset	283
e5d51fa51aba contrib: add an import checker Augie Fackler <raf@durin42.com> parents: diff changeset	284 stdlib_modules = set(list_stdlib_modules())
e5d51fa51aba contrib: add an import checker Augie Fackler <raf@durin42.com> parents: diff changeset	285
28921 02ee31a50002 import-checker: track filenames for SyntaxErrors timeless <timeless@mozdev.org> parents: 28920 diff changeset	286 def imported_modules(source, modulename, f, localmods, ignore_nested=False):
20036 e5d51fa51aba contrib: add an import checker Augie Fackler <raf@durin42.com> parents: diff changeset	287 """Given the source of a file as a string, yield the names
e5d51fa51aba contrib: add an import checker Augie Fackler <raf@durin42.com> parents: diff changeset	288 imported by that file.
e5d51fa51aba contrib: add an import checker Augie Fackler <raf@durin42.com> parents: diff changeset	289
20037 957b43371928 import-checker: ignore nested imports Augie Fackler <raf@durin42.com> parents: 20036 diff changeset	290 Args:
957b43371928 import-checker: ignore nested imports Augie Fackler <raf@durin42.com> parents: 20036 diff changeset	291 source: The python source to examine as a string.
25174 86298718b01c import-checker: make imported_modules yield absolute dotted_name_of_path FUJIWARA Katsunori <foozy@lares.dti.ne.jp> parents: 25173 diff changeset	292 modulename: of specified python source (may have `__init__`)
32508 4c712b90c60a import-checker: convert localmods to a set of module names Yuya Nishihara <yuya@tcha.org> parents: 32507 diff changeset	293 localmods: set of locally defined module names (may have `__init__`)
20037 957b43371928 import-checker: ignore nested imports Augie Fackler <raf@durin42.com> parents: 20036 diff changeset	294 ignore_nested: If true, import statements that do not start in
957b43371928 import-checker: ignore nested imports Augie Fackler <raf@durin42.com> parents: 20036 diff changeset	295 column zero will be ignored.
957b43371928 import-checker: ignore nested imports Augie Fackler <raf@durin42.com> parents: 20036 diff changeset	296
957b43371928 import-checker: ignore nested imports Augie Fackler <raf@durin42.com> parents: 20036 diff changeset	297 Returns:
25174 86298718b01c import-checker: make imported_modules yield absolute dotted_name_of_path FUJIWARA Katsunori <foozy@lares.dti.ne.jp> parents: 25173 diff changeset	298 A list of absolute module names imported by the given source.
20037 957b43371928 import-checker: ignore nested imports Augie Fackler <raf@durin42.com> parents: 20036 diff changeset	299
28921 02ee31a50002 import-checker: track filenames for SyntaxErrors timeless <timeless@mozdev.org> parents: 28920 diff changeset	300 >>> f = 'foo/xxx.py'
25174 86298718b01c import-checker: make imported_modules yield absolute dotted_name_of_path FUJIWARA Katsunori <foozy@lares.dti.ne.jp> parents: 25173 diff changeset	301 >>> modulename = 'foo.xxx'
86298718b01c import-checker: make imported_modules yield absolute dotted_name_of_path FUJIWARA Katsunori <foozy@lares.dti.ne.jp> parents: 25173 diff changeset	302 >>> localmods = {'foo.__init__': True,
86298718b01c import-checker: make imported_modules yield absolute dotted_name_of_path FUJIWARA Katsunori <foozy@lares.dti.ne.jp> parents: 25173 diff changeset	303 ... 'foo.foo1': True, 'foo.foo2': True,
86298718b01c import-checker: make imported_modules yield absolute dotted_name_of_path FUJIWARA Katsunori <foozy@lares.dti.ne.jp> parents: 25173 diff changeset	304 ... 'foo.bar.__init__': True, 'foo.bar.bar1': True,
86298718b01c import-checker: make imported_modules yield absolute dotted_name_of_path FUJIWARA Katsunori <foozy@lares.dti.ne.jp> parents: 25173 diff changeset	305 ... 'baz.__init__': True, 'baz.baz1': True }
86298718b01c import-checker: make imported_modules yield absolute dotted_name_of_path FUJIWARA Katsunori <foozy@lares.dti.ne.jp> parents: 25173 diff changeset	306 >>> # standard library (= not locally defined ones)
86298718b01c import-checker: make imported_modules yield absolute dotted_name_of_path FUJIWARA Katsunori <foozy@lares.dti.ne.jp> parents: 25173 diff changeset	307 >>> sorted(imported_modules(
86298718b01c import-checker: make imported_modules yield absolute dotted_name_of_path FUJIWARA Katsunori <foozy@lares.dti.ne.jp> parents: 25173 diff changeset	308 ... 'from stdlib1 import foo, bar; import stdlib2',
28921 02ee31a50002 import-checker: track filenames for SyntaxErrors timeless <timeless@mozdev.org> parents: 28920 diff changeset	309 ... modulename, f, localmods))
25174 86298718b01c import-checker: make imported_modules yield absolute dotted_name_of_path FUJIWARA Katsunori <foozy@lares.dti.ne.jp> parents: 25173 diff changeset	310 []
86298718b01c import-checker: make imported_modules yield absolute dotted_name_of_path FUJIWARA Katsunori <foozy@lares.dti.ne.jp> parents: 25173 diff changeset	311 >>> # relative importing
20037 957b43371928 import-checker: ignore nested imports Augie Fackler <raf@durin42.com> parents: 20036 diff changeset	312 >>> sorted(imported_modules(
25174 86298718b01c import-checker: make imported_modules yield absolute dotted_name_of_path FUJIWARA Katsunori <foozy@lares.dti.ne.jp> parents: 25173 diff changeset	313 ... 'import foo1; from bar import bar1',
28921 02ee31a50002 import-checker: track filenames for SyntaxErrors timeless <timeless@mozdev.org> parents: 28920 diff changeset	314 ... modulename, f, localmods))
26964 5abba2c92da3 import-checker: allow import of child modules from package root Yuya Nishihara <yuya@tcha.org> parents: 26956 diff changeset	315 ['foo.bar.bar1', 'foo.foo1']
25174 86298718b01c import-checker: make imported_modules yield absolute dotted_name_of_path FUJIWARA Katsunori <foozy@lares.dti.ne.jp> parents: 25173 diff changeset	316 >>> sorted(imported_modules(
86298718b01c import-checker: make imported_modules yield absolute dotted_name_of_path FUJIWARA Katsunori <foozy@lares.dti.ne.jp> parents: 25173 diff changeset	317 ... 'from bar.bar1 import name1, name2, name3',
28921 02ee31a50002 import-checker: track filenames for SyntaxErrors timeless <timeless@mozdev.org> parents: 28920 diff changeset	318 ... modulename, f, localmods))
25174 86298718b01c import-checker: make imported_modules yield absolute dotted_name_of_path FUJIWARA Katsunori <foozy@lares.dti.ne.jp> parents: 25173 diff changeset	319 ['foo.bar.bar1']
86298718b01c import-checker: make imported_modules yield absolute dotted_name_of_path FUJIWARA Katsunori <foozy@lares.dti.ne.jp> parents: 25173 diff changeset	320 >>> # absolute importing
86298718b01c import-checker: make imported_modules yield absolute dotted_name_of_path FUJIWARA Katsunori <foozy@lares.dti.ne.jp> parents: 25173 diff changeset	321 >>> sorted(imported_modules(
86298718b01c import-checker: make imported_modules yield absolute dotted_name_of_path FUJIWARA Katsunori <foozy@lares.dti.ne.jp> parents: 25173 diff changeset	322 ... 'from baz import baz1, name1',
28921 02ee31a50002 import-checker: track filenames for SyntaxErrors timeless <timeless@mozdev.org> parents: 28920 diff changeset	323 ... modulename, f, localmods))
25174 86298718b01c import-checker: make imported_modules yield absolute dotted_name_of_path FUJIWARA Katsunori <foozy@lares.dti.ne.jp> parents: 25173 diff changeset	324 ['baz.__init__', 'baz.baz1']
86298718b01c import-checker: make imported_modules yield absolute dotted_name_of_path FUJIWARA Katsunori <foozy@lares.dti.ne.jp> parents: 25173 diff changeset	325 >>> # mixed importing, even though it shouldn't be recommended
86298718b01c import-checker: make imported_modules yield absolute dotted_name_of_path FUJIWARA Katsunori <foozy@lares.dti.ne.jp> parents: 25173 diff changeset	326 >>> sorted(imported_modules(
86298718b01c import-checker: make imported_modules yield absolute dotted_name_of_path FUJIWARA Katsunori <foozy@lares.dti.ne.jp> parents: 25173 diff changeset	327 ... 'import stdlib, foo1, baz',
28921 02ee31a50002 import-checker: track filenames for SyntaxErrors timeless <timeless@mozdev.org> parents: 28920 diff changeset	328 ... modulename, f, localmods))
25174 86298718b01c import-checker: make imported_modules yield absolute dotted_name_of_path FUJIWARA Katsunori <foozy@lares.dti.ne.jp> parents: 25173 diff changeset	329 ['baz.__init__', 'foo.foo1']
86298718b01c import-checker: make imported_modules yield absolute dotted_name_of_path FUJIWARA Katsunori <foozy@lares.dti.ne.jp> parents: 25173 diff changeset	330 >>> # ignore_nested
20037 957b43371928 import-checker: ignore nested imports Augie Fackler <raf@durin42.com> parents: 20036 diff changeset	331 >>> sorted(imported_modules(
957b43371928 import-checker: ignore nested imports Augie Fackler <raf@durin42.com> parents: 20036 diff changeset	332 ... '''import foo
957b43371928 import-checker: ignore nested imports Augie Fackler <raf@durin42.com> parents: 20036 diff changeset	333 ... def wat():
957b43371928 import-checker: ignore nested imports Augie Fackler <raf@durin42.com> parents: 20036 diff changeset	334 ... import bar
28921 02ee31a50002 import-checker: track filenames for SyntaxErrors timeless <timeless@mozdev.org> parents: 28920 diff changeset	335 ... ''', modulename, f, localmods))
25174 86298718b01c import-checker: make imported_modules yield absolute dotted_name_of_path FUJIWARA Katsunori <foozy@lares.dti.ne.jp> parents: 25173 diff changeset	336 ['foo.__init__', 'foo.bar.__init__']
86298718b01c import-checker: make imported_modules yield absolute dotted_name_of_path FUJIWARA Katsunori <foozy@lares.dti.ne.jp> parents: 25173 diff changeset	337 >>> sorted(imported_modules(
86298718b01c import-checker: make imported_modules yield absolute dotted_name_of_path FUJIWARA Katsunori <foozy@lares.dti.ne.jp> parents: 25173 diff changeset	338 ... '''import foo
86298718b01c import-checker: make imported_modules yield absolute dotted_name_of_path FUJIWARA Katsunori <foozy@lares.dti.ne.jp> parents: 25173 diff changeset	339 ... def wat():
86298718b01c import-checker: make imported_modules yield absolute dotted_name_of_path FUJIWARA Katsunori <foozy@lares.dti.ne.jp> parents: 25173 diff changeset	340 ... import bar
28921 02ee31a50002 import-checker: track filenames for SyntaxErrors timeless <timeless@mozdev.org> parents: 28920 diff changeset	341 ... ''', modulename, f, localmods, ignore_nested=True))
25174 86298718b01c import-checker: make imported_modules yield absolute dotted_name_of_path FUJIWARA Katsunori <foozy@lares.dti.ne.jp> parents: 25173 diff changeset	342 ['foo.__init__']
20036 e5d51fa51aba contrib: add an import checker Augie Fackler <raf@durin42.com> parents: diff changeset	343 """
25174 86298718b01c import-checker: make imported_modules yield absolute dotted_name_of_path FUJIWARA Katsunori <foozy@lares.dti.ne.jp> parents: 25173 diff changeset	344 fromlocal = fromlocalfunc(modulename, localmods)
28921 02ee31a50002 import-checker: track filenames for SyntaxErrors timeless <timeless@mozdev.org> parents: 28920 diff changeset	345 for node in ast.walk(ast.parse(source, f)):
20037 957b43371928 import-checker: ignore nested imports Augie Fackler <raf@durin42.com> parents: 20036 diff changeset	346 if ignore_nested and getattr(node, 'col_offset', 0) > 0:
957b43371928 import-checker: ignore nested imports Augie Fackler <raf@durin42.com> parents: 20036 diff changeset	347 continue
20036 e5d51fa51aba contrib: add an import checker Augie Fackler <raf@durin42.com> parents: diff changeset	348 if isinstance(node, ast.Import):
e5d51fa51aba contrib: add an import checker Augie Fackler <raf@durin42.com> parents: diff changeset	349 for n in node.names:
25174 86298718b01c import-checker: make imported_modules yield absolute dotted_name_of_path FUJIWARA Katsunori <foozy@lares.dti.ne.jp> parents: 25173 diff changeset	350 found = fromlocal(n.name)
86298718b01c import-checker: make imported_modules yield absolute dotted_name_of_path FUJIWARA Katsunori <foozy@lares.dti.ne.jp> parents: 25173 diff changeset	351 if not found:
86298718b01c import-checker: make imported_modules yield absolute dotted_name_of_path FUJIWARA Katsunori <foozy@lares.dti.ne.jp> parents: 25173 diff changeset	352 # this should import standard library
86298718b01c import-checker: make imported_modules yield absolute dotted_name_of_path FUJIWARA Katsunori <foozy@lares.dti.ne.jp> parents: 25173 diff changeset	353 continue
86298718b01c import-checker: make imported_modules yield absolute dotted_name_of_path FUJIWARA Katsunori <foozy@lares.dti.ne.jp> parents: 25173 diff changeset	354 yield found[1]
20036 e5d51fa51aba contrib: add an import checker Augie Fackler <raf@durin42.com> parents: diff changeset	355 elif isinstance(node, ast.ImportFrom):
25701 1f88c0f6ff5a import-checker: resolve relative imports Gregory Szorc <gregory.szorc@gmail.com> parents: 25660 diff changeset	356 found = fromlocal(node.module, node.level)
25174 86298718b01c import-checker: make imported_modules yield absolute dotted_name_of_path FUJIWARA Katsunori <foozy@lares.dti.ne.jp> parents: 25173 diff changeset	357 if not found:
86298718b01c import-checker: make imported_modules yield absolute dotted_name_of_path FUJIWARA Katsunori <foozy@lares.dti.ne.jp> parents: 25173 diff changeset	358 # this should import standard library
86298718b01c import-checker: make imported_modules yield absolute dotted_name_of_path FUJIWARA Katsunori <foozy@lares.dti.ne.jp> parents: 25173 diff changeset	359 continue
86298718b01c import-checker: make imported_modules yield absolute dotted_name_of_path FUJIWARA Katsunori <foozy@lares.dti.ne.jp> parents: 25173 diff changeset	360
86298718b01c import-checker: make imported_modules yield absolute dotted_name_of_path FUJIWARA Katsunori <foozy@lares.dti.ne.jp> parents: 25173 diff changeset	361 absname, dottedpath, hassubmod = found
86298718b01c import-checker: make imported_modules yield absolute dotted_name_of_path FUJIWARA Katsunori <foozy@lares.dti.ne.jp> parents: 25173 diff changeset	362 if not hassubmod:
26964 5abba2c92da3 import-checker: allow import of child modules from package root Yuya Nishihara <yuya@tcha.org> parents: 26956 diff changeset	363 # "dottedpath" is not a package; must be imported
5abba2c92da3 import-checker: allow import of child modules from package root Yuya Nishihara <yuya@tcha.org> parents: 26956 diff changeset	364 yield dottedpath
25174 86298718b01c import-checker: make imported_modules yield absolute dotted_name_of_path FUJIWARA Katsunori <foozy@lares.dti.ne.jp> parents: 25173 diff changeset	365 # examination of "node.names" should be redundant
86298718b01c import-checker: make imported_modules yield absolute dotted_name_of_path FUJIWARA Katsunori <foozy@lares.dti.ne.jp> parents: 25173 diff changeset	366 # e.g.: from mercurial.node import nullid, nullrev
86298718b01c import-checker: make imported_modules yield absolute dotted_name_of_path FUJIWARA Katsunori <foozy@lares.dti.ne.jp> parents: 25173 diff changeset	367 continue
86298718b01c import-checker: make imported_modules yield absolute dotted_name_of_path FUJIWARA Katsunori <foozy@lares.dti.ne.jp> parents: 25173 diff changeset	368
26964 5abba2c92da3 import-checker: allow import of child modules from package root Yuya Nishihara <yuya@tcha.org> parents: 26956 diff changeset	369 modnotfound = False
25174 86298718b01c import-checker: make imported_modules yield absolute dotted_name_of_path FUJIWARA Katsunori <foozy@lares.dti.ne.jp> parents: 25173 diff changeset	370 prefix = absname + '.'
20036 e5d51fa51aba contrib: add an import checker Augie Fackler <raf@durin42.com> parents: diff changeset	371 for n in node.names:
25174 86298718b01c import-checker: make imported_modules yield absolute dotted_name_of_path FUJIWARA Katsunori <foozy@lares.dti.ne.jp> parents: 25173 diff changeset	372 found = fromlocal(prefix + n.name)
86298718b01c import-checker: make imported_modules yield absolute dotted_name_of_path FUJIWARA Katsunori <foozy@lares.dti.ne.jp> parents: 25173 diff changeset	373 if not found:
86298718b01c import-checker: make imported_modules yield absolute dotted_name_of_path FUJIWARA Katsunori <foozy@lares.dti.ne.jp> parents: 25173 diff changeset	374 # this should be a function or a property of "node.module"
26964 5abba2c92da3 import-checker: allow import of child modules from package root Yuya Nishihara <yuya@tcha.org> parents: 26956 diff changeset	375 modnotfound = True
25174 86298718b01c import-checker: make imported_modules yield absolute dotted_name_of_path FUJIWARA Katsunori <foozy@lares.dti.ne.jp> parents: 25173 diff changeset	376 continue
86298718b01c import-checker: make imported_modules yield absolute dotted_name_of_path FUJIWARA Katsunori <foozy@lares.dti.ne.jp> parents: 25173 diff changeset	377 yield found[1]
26964 5abba2c92da3 import-checker: allow import of child modules from package root Yuya Nishihara <yuya@tcha.org> parents: 26956 diff changeset	378 if modnotfound:
5abba2c92da3 import-checker: allow import of child modules from package root Yuya Nishihara <yuya@tcha.org> parents: 26956 diff changeset	379 # "dottedpath" is a package, but imported because of non-module
5abba2c92da3 import-checker: allow import of child modules from package root Yuya Nishihara <yuya@tcha.org> parents: 26956 diff changeset	380 # lookup
5abba2c92da3 import-checker: allow import of child modules from package root Yuya Nishihara <yuya@tcha.org> parents: 26956 diff changeset	381 yield dottedpath
20036 e5d51fa51aba contrib: add an import checker Augie Fackler <raf@durin42.com> parents: diff changeset	382
27272 69308357ecd1 import-checker: allow absolute imports of sub modules from local packages Yuya Nishihara <yuya@tcha.org> parents: 27018 diff changeset	383 def verify_import_convention(module, source, localmods):
25703 1a6a117d0b95 import-checker: establish modern import convention Gregory Szorc <gregory.szorc@gmail.com> parents: 25702 diff changeset	384 """Verify imports match our established coding convention.
1a6a117d0b95 import-checker: establish modern import convention Gregory Szorc <gregory.szorc@gmail.com> parents: 25702 diff changeset	385
1a6a117d0b95 import-checker: establish modern import convention Gregory Szorc <gregory.szorc@gmail.com> parents: 25702 diff changeset	386 We have 2 conventions: legacy and modern. The modern convention is in
1a6a117d0b95 import-checker: establish modern import convention Gregory Szorc <gregory.szorc@gmail.com> parents: 25702 diff changeset	387 effect when using absolute imports.
1a6a117d0b95 import-checker: establish modern import convention Gregory Szorc <gregory.szorc@gmail.com> parents: 25702 diff changeset	388
1a6a117d0b95 import-checker: establish modern import convention Gregory Szorc <gregory.szorc@gmail.com> parents: 25702 diff changeset	389 The legacy convention only looks for mixed imports. The modern convention
1a6a117d0b95 import-checker: establish modern import convention Gregory Szorc <gregory.szorc@gmail.com> parents: 25702 diff changeset	390 is much more thorough.
1a6a117d0b95 import-checker: establish modern import convention Gregory Szorc <gregory.szorc@gmail.com> parents: 25702 diff changeset	391 """
25702 ab2c5163900e import-checker: establish new function for verifying import conventions Gregory Szorc <gregory.szorc@gmail.com> parents: 25701 diff changeset	392 root = ast.parse(source)
25703 1a6a117d0b95 import-checker: establish modern import convention Gregory Szorc <gregory.szorc@gmail.com> parents: 25702 diff changeset	393 absolute = usingabsolute(root)
25702 ab2c5163900e import-checker: establish new function for verifying import conventions Gregory Szorc <gregory.szorc@gmail.com> parents: 25701 diff changeset	394
25703 1a6a117d0b95 import-checker: establish modern import convention Gregory Szorc <gregory.szorc@gmail.com> parents: 25702 diff changeset	395 if absolute:
27272 69308357ecd1 import-checker: allow absolute imports of sub modules from local packages Yuya Nishihara <yuya@tcha.org> parents: 27018 diff changeset	396 return verify_modern_convention(module, root, localmods)
25703 1a6a117d0b95 import-checker: establish modern import convention Gregory Szorc <gregory.szorc@gmail.com> parents: 25702 diff changeset	397 else:
1a6a117d0b95 import-checker: establish modern import convention Gregory Szorc <gregory.szorc@gmail.com> parents: 25702 diff changeset	398 return verify_stdlib_on_own_line(root)
1a6a117d0b95 import-checker: establish modern import convention Gregory Szorc <gregory.szorc@gmail.com> parents: 25702 diff changeset	399
27272 69308357ecd1 import-checker: allow absolute imports of sub modules from local packages Yuya Nishihara <yuya@tcha.org> parents: 27018 diff changeset	400 def verify_modern_convention(module, root, localmods, root_col_offset=0):
25703 1a6a117d0b95 import-checker: establish modern import convention Gregory Szorc <gregory.szorc@gmail.com> parents: 25702 diff changeset	401 """Verify a file conforms to the modern import convention rules.
1a6a117d0b95 import-checker: establish modern import convention Gregory Szorc <gregory.szorc@gmail.com> parents: 25702 diff changeset	402
1a6a117d0b95 import-checker: establish modern import convention Gregory Szorc <gregory.szorc@gmail.com> parents: 25702 diff changeset	403 The rules of the modern convention are:
1a6a117d0b95 import-checker: establish modern import convention Gregory Szorc <gregory.szorc@gmail.com> parents: 25702 diff changeset	404
1a6a117d0b95 import-checker: establish modern import convention Gregory Szorc <gregory.szorc@gmail.com> parents: 25702 diff changeset	405 * Ordering is stdlib followed by local imports. Each group is lexically
1a6a117d0b95 import-checker: establish modern import convention Gregory Szorc <gregory.szorc@gmail.com> parents: 25702 diff changeset	406 sorted.
1a6a117d0b95 import-checker: establish modern import convention Gregory Szorc <gregory.szorc@gmail.com> parents: 25702 diff changeset	407 * Importing multiple modules via "import X, Y" is not allowed: use
1a6a117d0b95 import-checker: establish modern import convention Gregory Szorc <gregory.szorc@gmail.com> parents: 25702 diff changeset	408 separate import statements.
1a6a117d0b95 import-checker: establish modern import convention Gregory Szorc <gregory.szorc@gmail.com> parents: 25702 diff changeset	409 * Importing multiple modules via "from X import ..." is allowed if using
1a6a117d0b95 import-checker: establish modern import convention Gregory Szorc <gregory.szorc@gmail.com> parents: 25702 diff changeset	410 parenthesis and one entry per line.
1a6a117d0b95 import-checker: establish modern import convention Gregory Szorc <gregory.szorc@gmail.com> parents: 25702 diff changeset	411 * Only 1 relative import statement per import level ("from .", "from ..")
1a6a117d0b95 import-checker: establish modern import convention Gregory Szorc <gregory.szorc@gmail.com> parents: 25702 diff changeset	412 is allowed.
1a6a117d0b95 import-checker: establish modern import convention Gregory Szorc <gregory.szorc@gmail.com> parents: 25702 diff changeset	413 * Relative imports from higher levels must occur before lower levels. e.g.
1a6a117d0b95 import-checker: establish modern import convention Gregory Szorc <gregory.szorc@gmail.com> parents: 25702 diff changeset	414 "from .." must be before "from .".
1a6a117d0b95 import-checker: establish modern import convention Gregory Szorc <gregory.szorc@gmail.com> parents: 25702 diff changeset	415 * Imports from peer packages should use relative import (e.g. do not
1a6a117d0b95 import-checker: establish modern import convention Gregory Szorc <gregory.szorc@gmail.com> parents: 25702 diff changeset	416 "import mercurial.foo" from a "mercurial.*" module).
1a6a117d0b95 import-checker: establish modern import convention Gregory Szorc <gregory.szorc@gmail.com> parents: 25702 diff changeset	417 * Symbols can only be imported from specific modules (see
1a6a117d0b95 import-checker: establish modern import convention Gregory Szorc <gregory.szorc@gmail.com> parents: 25702 diff changeset	418 `allowsymbolimports`). For other modules, first import the module then
1a6a117d0b95 import-checker: establish modern import convention Gregory Szorc <gregory.szorc@gmail.com> parents: 25702 diff changeset	419 assign the symbol to a module-level variable. In addition, these imports
29208 cba8bc11ed10 import-checker: extend check of symbol-import order to all local modules Yuya Nishihara <yuya@tcha.org> parents: 29207 diff changeset	420 must be performed before other local imports. This rule only
25703 1a6a117d0b95 import-checker: establish modern import convention Gregory Szorc <gregory.szorc@gmail.com> parents: 25702 diff changeset	421 applies to import statements outside of any blocks.
34038 bc2535238de2 import-checker: allow relative import a module being checked Jun Wu <quark@fb.com> parents: 33920 diff changeset	422 * Relative imports from the standard library are not allowed, unless that
bc2535238de2 import-checker: allow relative import a module being checked Jun Wu <quark@fb.com> parents: 33920 diff changeset	423 library is also a local module.
25703 1a6a117d0b95 import-checker: establish modern import convention Gregory Szorc <gregory.szorc@gmail.com> parents: 25702 diff changeset	424 * Certain modules must be aliased to alternate names to avoid aliasing
1a6a117d0b95 import-checker: establish modern import convention Gregory Szorc <gregory.szorc@gmail.com> parents: 25702 diff changeset	425 and readability problems. See `requirealias`.
1a6a117d0b95 import-checker: establish modern import convention Gregory Szorc <gregory.szorc@gmail.com> parents: 25702 diff changeset	426 """
33890 3de9a2df6411 contrib: have import-checker work mostly with native strings for mod names Augie Fackler <raf@durin42.com> parents: 33877 diff changeset	427 if not isinstance(module, str):
3de9a2df6411 contrib: have import-checker work mostly with native strings for mod names Augie Fackler <raf@durin42.com> parents: 33877 diff changeset	428 module = module.decode('ascii')
25703 1a6a117d0b95 import-checker: establish modern import convention Gregory Szorc <gregory.szorc@gmail.com> parents: 25702 diff changeset	429 topmodule = module.split('.')[0]
27272 69308357ecd1 import-checker: allow absolute imports of sub modules from local packages Yuya Nishihara <yuya@tcha.org> parents: 27018 diff changeset	430 fromlocal = fromlocalfunc(module, localmods)
25703 1a6a117d0b95 import-checker: establish modern import convention Gregory Szorc <gregory.szorc@gmail.com> parents: 25702 diff changeset	431
1a6a117d0b95 import-checker: establish modern import convention Gregory Szorc <gregory.szorc@gmail.com> parents: 25702 diff changeset	432 # Whether a local/non-stdlib import has been performed.
28330 f3fb24e36d61 import-checker: report local with stdlib late warning timeless <timeless@mozdev.org> parents: 27621 diff changeset	433 seenlocal = None
29208 cba8bc11ed10 import-checker: extend check of symbol-import order to all local modules Yuya Nishihara <yuya@tcha.org> parents: 29207 diff changeset	434 # Whether a local/non-stdlib, non-symbol import has been seen.
cba8bc11ed10 import-checker: extend check of symbol-import order to all local modules Yuya Nishihara <yuya@tcha.org> parents: 29207 diff changeset	435 seennonsymbollocal = False
25703 1a6a117d0b95 import-checker: establish modern import convention Gregory Szorc <gregory.szorc@gmail.com> parents: 25702 diff changeset	436 # The last name to be imported (for sorting).
1a6a117d0b95 import-checker: establish modern import convention Gregory Szorc <gregory.szorc@gmail.com> parents: 25702 diff changeset	437 lastname = None
30590 74eecb93c617 import-checker: do not enforce lexical sort accross stdlib/local boundary Pierre-Yves David <pierre-yves.david@ens-lyon.org> parents: 29395 diff changeset	438 laststdlib = None
25703 1a6a117d0b95 import-checker: establish modern import convention Gregory Szorc <gregory.szorc@gmail.com> parents: 25702 diff changeset	439 # Relative import levels encountered so far.
1a6a117d0b95 import-checker: establish modern import convention Gregory Szorc <gregory.szorc@gmail.com> parents: 25702 diff changeset	440 seenlevels = set()
1a6a117d0b95 import-checker: establish modern import convention Gregory Szorc <gregory.szorc@gmail.com> parents: 25702 diff changeset	441
26965 1fa66d3ad28d import-checker: reset context to verify convention in function scope Yuya Nishihara <yuya@tcha.org> parents: 26964 diff changeset	442 for node, newscope in walklocal(root):
26955 c4114e335b49 import-checker: extract function to generate a formatted warning Yuya Nishihara <yuya@tcha.org> parents: 26954 diff changeset	443 def msg(fmt, *args):
26956 4b56214ebb7a import-checker: include lineno in warning message Yuya Nishihara <yuya@tcha.org> parents: 26955 diff changeset	444 return (fmt % args, node.lineno)
26965 1fa66d3ad28d import-checker: reset context to verify convention in function scope Yuya Nishihara <yuya@tcha.org> parents: 26964 diff changeset	445 if newscope:
1fa66d3ad28d import-checker: reset context to verify convention in function scope Yuya Nishihara <yuya@tcha.org> parents: 26964 diff changeset	446 # Check for local imports in function
27272 69308357ecd1 import-checker: allow absolute imports of sub modules from local packages Yuya Nishihara <yuya@tcha.org> parents: 27018 diff changeset	447 for r in verify_modern_convention(module, node, localmods,
26965 1fa66d3ad28d import-checker: reset context to verify convention in function scope Yuya Nishihara <yuya@tcha.org> parents: 26964 diff changeset	448 node.col_offset + 4):
1fa66d3ad28d import-checker: reset context to verify convention in function scope Yuya Nishihara <yuya@tcha.org> parents: 26964 diff changeset	449 yield r
1fa66d3ad28d import-checker: reset context to verify convention in function scope Yuya Nishihara <yuya@tcha.org> parents: 26964 diff changeset	450 elif isinstance(node, ast.Import):
25703 1a6a117d0b95 import-checker: establish modern import convention Gregory Szorc <gregory.szorc@gmail.com> parents: 25702 diff changeset	451 # Disallow "import foo, bar" and require separate imports
1a6a117d0b95 import-checker: establish modern import convention Gregory Szorc <gregory.szorc@gmail.com> parents: 25702 diff changeset	452 # for each module.
1a6a117d0b95 import-checker: establish modern import convention Gregory Szorc <gregory.szorc@gmail.com> parents: 25702 diff changeset	453 if len(node.names) > 1:
26955 c4114e335b49 import-checker: extract function to generate a formatted warning Yuya Nishihara <yuya@tcha.org> parents: 26954 diff changeset	454 yield msg('multiple imported names: %s',
c4114e335b49 import-checker: extract function to generate a formatted warning Yuya Nishihara <yuya@tcha.org> parents: 26954 diff changeset	455 ', '.join(n.name for n in node.names))
25703 1a6a117d0b95 import-checker: establish modern import convention Gregory Szorc <gregory.szorc@gmail.com> parents: 25702 diff changeset	456
1a6a117d0b95 import-checker: establish modern import convention Gregory Szorc <gregory.szorc@gmail.com> parents: 25702 diff changeset	457 name = node.names[0].name
1a6a117d0b95 import-checker: establish modern import convention Gregory Szorc <gregory.szorc@gmail.com> parents: 25702 diff changeset	458 asname = node.names[0].asname
1a6a117d0b95 import-checker: establish modern import convention Gregory Szorc <gregory.szorc@gmail.com> parents: 25702 diff changeset	459
30590 74eecb93c617 import-checker: do not enforce lexical sort accross stdlib/local boundary Pierre-Yves David <pierre-yves.david@ens-lyon.org> parents: 29395 diff changeset	460 stdlib = name in stdlib_modules
74eecb93c617 import-checker: do not enforce lexical sort accross stdlib/local boundary Pierre-Yves David <pierre-yves.david@ens-lyon.org> parents: 29395 diff changeset	461
25703 1a6a117d0b95 import-checker: establish modern import convention Gregory Szorc <gregory.szorc@gmail.com> parents: 25702 diff changeset	462 # Ignore sorting rules on imports inside blocks.
26965 1fa66d3ad28d import-checker: reset context to verify convention in function scope Yuya Nishihara <yuya@tcha.org> parents: 26964 diff changeset	463 if node.col_offset == root_col_offset:
30590 74eecb93c617 import-checker: do not enforce lexical sort accross stdlib/local boundary Pierre-Yves David <pierre-yves.david@ens-lyon.org> parents: 29395 diff changeset	464 if lastname and name < lastname and laststdlib == stdlib:
26955 c4114e335b49 import-checker: extract function to generate a formatted warning Yuya Nishihara <yuya@tcha.org> parents: 26954 diff changeset	465 yield msg('imports not lexically sorted: %s < %s',
c4114e335b49 import-checker: extract function to generate a formatted warning Yuya Nishihara <yuya@tcha.org> parents: 26954 diff changeset	466 name, lastname)
25703 1a6a117d0b95 import-checker: establish modern import convention Gregory Szorc <gregory.szorc@gmail.com> parents: 25702 diff changeset	467
30590 74eecb93c617 import-checker: do not enforce lexical sort accross stdlib/local boundary Pierre-Yves David <pierre-yves.david@ens-lyon.org> parents: 29395 diff changeset	468 lastname = name
74eecb93c617 import-checker: do not enforce lexical sort accross stdlib/local boundary Pierre-Yves David <pierre-yves.david@ens-lyon.org> parents: 29395 diff changeset	469 laststdlib = stdlib
25703 1a6a117d0b95 import-checker: establish modern import convention Gregory Szorc <gregory.szorc@gmail.com> parents: 25702 diff changeset	470
1a6a117d0b95 import-checker: establish modern import convention Gregory Szorc <gregory.szorc@gmail.com> parents: 25702 diff changeset	471 # stdlib imports should be before local imports.
26965 1fa66d3ad28d import-checker: reset context to verify convention in function scope Yuya Nishihara <yuya@tcha.org> parents: 26964 diff changeset	472 if stdlib and seenlocal and node.col_offset == root_col_offset:
28330 f3fb24e36d61 import-checker: report local with stdlib late warning timeless <timeless@mozdev.org> parents: 27621 diff changeset	473 yield msg('stdlib import "%s" follows local import: %s',
f3fb24e36d61 import-checker: report local with stdlib late warning timeless <timeless@mozdev.org> parents: 27621 diff changeset	474 name, seenlocal)
25703 1a6a117d0b95 import-checker: establish modern import convention Gregory Szorc <gregory.szorc@gmail.com> parents: 25702 diff changeset	475
1a6a117d0b95 import-checker: establish modern import convention Gregory Szorc <gregory.szorc@gmail.com> parents: 25702 diff changeset	476 if not stdlib:
28330 f3fb24e36d61 import-checker: report local with stdlib late warning timeless <timeless@mozdev.org> parents: 27621 diff changeset	477 seenlocal = name
25703 1a6a117d0b95 import-checker: establish modern import convention Gregory Szorc <gregory.szorc@gmail.com> parents: 25702 diff changeset	478
1a6a117d0b95 import-checker: establish modern import convention Gregory Szorc <gregory.szorc@gmail.com> parents: 25702 diff changeset	479 # Import of sibling modules should use relative imports.
1a6a117d0b95 import-checker: establish modern import convention Gregory Szorc <gregory.szorc@gmail.com> parents: 25702 diff changeset	480 topname = name.split('.')[0]
1a6a117d0b95 import-checker: establish modern import convention Gregory Szorc <gregory.szorc@gmail.com> parents: 25702 diff changeset	481 if topname == topmodule:
26955 c4114e335b49 import-checker: extract function to generate a formatted warning Yuya Nishihara <yuya@tcha.org> parents: 26954 diff changeset	482 yield msg('import should be relative: %s', name)
25703 1a6a117d0b95 import-checker: establish modern import convention Gregory Szorc <gregory.szorc@gmail.com> parents: 25702 diff changeset	483
1a6a117d0b95 import-checker: establish modern import convention Gregory Szorc <gregory.szorc@gmail.com> parents: 25702 diff changeset	484 if name in requirealias and asname != requirealias[name]:
26955 c4114e335b49 import-checker: extract function to generate a formatted warning Yuya Nishihara <yuya@tcha.org> parents: 26954 diff changeset	485 yield msg('%s module must be "as" aliased to %s',
c4114e335b49 import-checker: extract function to generate a formatted warning Yuya Nishihara <yuya@tcha.org> parents: 26954 diff changeset	486 name, requirealias[name])
25703 1a6a117d0b95 import-checker: establish modern import convention Gregory Szorc <gregory.szorc@gmail.com> parents: 25702 diff changeset	487
1a6a117d0b95 import-checker: establish modern import convention Gregory Szorc <gregory.szorc@gmail.com> parents: 25702 diff changeset	488 elif isinstance(node, ast.ImportFrom):
1a6a117d0b95 import-checker: establish modern import convention Gregory Szorc <gregory.szorc@gmail.com> parents: 25702 diff changeset	489 # Resolve the full imported module name.
1a6a117d0b95 import-checker: establish modern import convention Gregory Szorc <gregory.szorc@gmail.com> parents: 25702 diff changeset	490 if node.level > 0:
1a6a117d0b95 import-checker: establish modern import convention Gregory Szorc <gregory.szorc@gmail.com> parents: 25702 diff changeset	491 fullname = '.'.join(module.split('.')[:-node.level])
1a6a117d0b95 import-checker: establish modern import convention Gregory Szorc <gregory.szorc@gmail.com> parents: 25702 diff changeset	492 if node.module:
1a6a117d0b95 import-checker: establish modern import convention Gregory Szorc <gregory.szorc@gmail.com> parents: 25702 diff changeset	493 fullname += '.%s' % node.module
1a6a117d0b95 import-checker: establish modern import convention Gregory Szorc <gregory.szorc@gmail.com> parents: 25702 diff changeset	494 else:
1a6a117d0b95 import-checker: establish modern import convention Gregory Szorc <gregory.szorc@gmail.com> parents: 25702 diff changeset	495 assert node.module
1a6a117d0b95 import-checker: establish modern import convention Gregory Szorc <gregory.szorc@gmail.com> parents: 25702 diff changeset	496 fullname = node.module
1a6a117d0b95 import-checker: establish modern import convention Gregory Szorc <gregory.szorc@gmail.com> parents: 25702 diff changeset	497
1a6a117d0b95 import-checker: establish modern import convention Gregory Szorc <gregory.szorc@gmail.com> parents: 25702 diff changeset	498 topname = fullname.split('.')[0]
1a6a117d0b95 import-checker: establish modern import convention Gregory Szorc <gregory.szorc@gmail.com> parents: 25702 diff changeset	499 if topname == topmodule:
26955 c4114e335b49 import-checker: extract function to generate a formatted warning Yuya Nishihara <yuya@tcha.org> parents: 26954 diff changeset	500 yield msg('import should be relative: %s', fullname)
25703 1a6a117d0b95 import-checker: establish modern import convention Gregory Szorc <gregory.szorc@gmail.com> parents: 25702 diff changeset	501
1a6a117d0b95 import-checker: establish modern import convention Gregory Szorc <gregory.szorc@gmail.com> parents: 25702 diff changeset	502 # __future__ is special since it needs to come first and use
1a6a117d0b95 import-checker: establish modern import convention Gregory Szorc <gregory.szorc@gmail.com> parents: 25702 diff changeset	503 # symbol import.
1a6a117d0b95 import-checker: establish modern import convention Gregory Szorc <gregory.szorc@gmail.com> parents: 25702 diff changeset	504 if fullname != '__future__':
34038 bc2535238de2 import-checker: allow relative import a module being checked Jun Wu <quark@fb.com> parents: 33920 diff changeset	505 if not fullname or (
bc2535238de2 import-checker: allow relative import a module being checked Jun Wu <quark@fb.com> parents: 33920 diff changeset	506 fullname in stdlib_modules
bc2535238de2 import-checker: allow relative import a module being checked Jun Wu <quark@fb.com> parents: 33920 diff changeset	507 and fullname not in localmods
bc2535238de2 import-checker: allow relative import a module being checked Jun Wu <quark@fb.com> parents: 33920 diff changeset	508 and fullname + '.__init__' not in localmods):
26955 c4114e335b49 import-checker: extract function to generate a formatted warning Yuya Nishihara <yuya@tcha.org> parents: 26954 diff changeset	509 yield msg('relative import of stdlib module')
25703 1a6a117d0b95 import-checker: establish modern import convention Gregory Szorc <gregory.szorc@gmail.com> parents: 25702 diff changeset	510 else:
28330 f3fb24e36d61 import-checker: report local with stdlib late warning timeless <timeless@mozdev.org> parents: 27621 diff changeset	511 seenlocal = fullname
25703 1a6a117d0b95 import-checker: establish modern import convention Gregory Szorc <gregory.szorc@gmail.com> parents: 25702 diff changeset	512
1a6a117d0b95 import-checker: establish modern import convention Gregory Szorc <gregory.szorc@gmail.com> parents: 25702 diff changeset	513 # Direct symbol import is only allowed from certain modules and
1a6a117d0b95 import-checker: establish modern import convention Gregory Szorc <gregory.szorc@gmail.com> parents: 25702 diff changeset	514 # must occur before non-symbol imports.
29207 a09098c61fea import-checker: always build a list of imported symbols Yuya Nishihara <yuya@tcha.org> parents: 29122 diff changeset	515 found = fromlocal(node.module, node.level)
a09098c61fea import-checker: always build a list of imported symbols Yuya Nishihara <yuya@tcha.org> parents: 29122 diff changeset	516 if found and found[2]: # node.module is a package
a09098c61fea import-checker: always build a list of imported symbols Yuya Nishihara <yuya@tcha.org> parents: 29122 diff changeset	517 prefix = found[0] + '.'
32419 d02888308235 import-checker: add a way to directly import certain symbols Siddharth Agarwal <sid0@fb.com> parents: 32374 diff changeset	518 symbols = (n.name for n in node.names
d02888308235 import-checker: add a way to directly import certain symbols Siddharth Agarwal <sid0@fb.com> parents: 32374 diff changeset	519 if not fromlocal(prefix + n.name))
29207 a09098c61fea import-checker: always build a list of imported symbols Yuya Nishihara <yuya@tcha.org> parents: 29122 diff changeset	520 else:
32419 d02888308235 import-checker: add a way to directly import certain symbols Siddharth Agarwal <sid0@fb.com> parents: 32374 diff changeset	521 symbols = (n.name for n in node.names)
d02888308235 import-checker: add a way to directly import certain symbols Siddharth Agarwal <sid0@fb.com> parents: 32374 diff changeset	522 symbols = [sym for sym in symbols if sym not in directsymbols]
26965 1fa66d3ad28d import-checker: reset context to verify convention in function scope Yuya Nishihara <yuya@tcha.org> parents: 26964 diff changeset	523 if node.module and node.col_offset == root_col_offset:
27272 69308357ecd1 import-checker: allow absolute imports of sub modules from local packages Yuya Nishihara <yuya@tcha.org> parents: 27018 diff changeset	524 if symbols and fullname not in allowsymbolimports:
27273 5d5b98346fc2 import-checker: tell which symbol causes "direct symbol import" Yuya Nishihara <yuya@tcha.org> parents: 27272 diff changeset	525 yield msg('direct symbol import %s from %s',
5d5b98346fc2 import-checker: tell which symbol causes "direct symbol import" Yuya Nishihara <yuya@tcha.org> parents: 27272 diff changeset	526 ', '.join(symbols), fullname)
25703 1a6a117d0b95 import-checker: establish modern import convention Gregory Szorc <gregory.szorc@gmail.com> parents: 25702 diff changeset	527
29208 cba8bc11ed10 import-checker: extend check of symbol-import order to all local modules Yuya Nishihara <yuya@tcha.org> parents: 29207 diff changeset	528 if symbols and seennonsymbollocal:
26955 c4114e335b49 import-checker: extract function to generate a formatted warning Yuya Nishihara <yuya@tcha.org> parents: 26954 diff changeset	529 yield msg('symbol import follows non-symbol import: %s',
c4114e335b49 import-checker: extract function to generate a formatted warning Yuya Nishihara <yuya@tcha.org> parents: 26954 diff changeset	530 fullname)
29208 cba8bc11ed10 import-checker: extend check of symbol-import order to all local modules Yuya Nishihara <yuya@tcha.org> parents: 29207 diff changeset	531 if not symbols and fullname not in stdlib_modules:
cba8bc11ed10 import-checker: extend check of symbol-import order to all local modules Yuya Nishihara <yuya@tcha.org> parents: 29207 diff changeset	532 seennonsymbollocal = True
25703 1a6a117d0b95 import-checker: establish modern import convention Gregory Szorc <gregory.szorc@gmail.com> parents: 25702 diff changeset	533
1a6a117d0b95 import-checker: establish modern import convention Gregory Szorc <gregory.szorc@gmail.com> parents: 25702 diff changeset	534 if not node.module:
1a6a117d0b95 import-checker: establish modern import convention Gregory Szorc <gregory.szorc@gmail.com> parents: 25702 diff changeset	535 assert node.level
1a6a117d0b95 import-checker: establish modern import convention Gregory Szorc <gregory.szorc@gmail.com> parents: 25702 diff changeset	536
1a6a117d0b95 import-checker: establish modern import convention Gregory Szorc <gregory.szorc@gmail.com> parents: 25702 diff changeset	537 # Only allow 1 group per level.
26965 1fa66d3ad28d import-checker: reset context to verify convention in function scope Yuya Nishihara <yuya@tcha.org> parents: 26964 diff changeset	538 if (node.level in seenlevels
1fa66d3ad28d import-checker: reset context to verify convention in function scope Yuya Nishihara <yuya@tcha.org> parents: 26964 diff changeset	539 and node.col_offset == root_col_offset):
26955 c4114e335b49 import-checker: extract function to generate a formatted warning Yuya Nishihara <yuya@tcha.org> parents: 26954 diff changeset	540 yield msg('multiple "from %s import" statements',
c4114e335b49 import-checker: extract function to generate a formatted warning Yuya Nishihara <yuya@tcha.org> parents: 26954 diff changeset	541 '.' * node.level)
25703 1a6a117d0b95 import-checker: establish modern import convention Gregory Szorc <gregory.szorc@gmail.com> parents: 25702 diff changeset	542
1a6a117d0b95 import-checker: establish modern import convention Gregory Szorc <gregory.szorc@gmail.com> parents: 25702 diff changeset	543 # Higher-level groups come before lower-level groups.
1a6a117d0b95 import-checker: establish modern import convention Gregory Szorc <gregory.szorc@gmail.com> parents: 25702 diff changeset	544 if any(node.level > l for l in seenlevels):
26955 c4114e335b49 import-checker: extract function to generate a formatted warning Yuya Nishihara <yuya@tcha.org> parents: 26954 diff changeset	545 yield msg('higher-level import should come first: %s',
c4114e335b49 import-checker: extract function to generate a formatted warning Yuya Nishihara <yuya@tcha.org> parents: 26954 diff changeset	546 fullname)
25703 1a6a117d0b95 import-checker: establish modern import convention Gregory Szorc <gregory.szorc@gmail.com> parents: 25702 diff changeset	547
1a6a117d0b95 import-checker: establish modern import convention Gregory Szorc <gregory.szorc@gmail.com> parents: 25702 diff changeset	548 seenlevels.add(node.level)
1a6a117d0b95 import-checker: establish modern import convention Gregory Szorc <gregory.szorc@gmail.com> parents: 25702 diff changeset	549
1a6a117d0b95 import-checker: establish modern import convention Gregory Szorc <gregory.szorc@gmail.com> parents: 25702 diff changeset	550 # Entries in "from .X import ( ... )" lists must be lexically
1a6a117d0b95 import-checker: establish modern import convention Gregory Szorc <gregory.szorc@gmail.com> parents: 25702 diff changeset	551 # sorted.
1a6a117d0b95 import-checker: establish modern import convention Gregory Szorc <gregory.szorc@gmail.com> parents: 25702 diff changeset	552 lastentryname = None
1a6a117d0b95 import-checker: establish modern import convention Gregory Szorc <gregory.szorc@gmail.com> parents: 25702 diff changeset	553
1a6a117d0b95 import-checker: establish modern import convention Gregory Szorc <gregory.szorc@gmail.com> parents: 25702 diff changeset	554 for n in node.names:
1a6a117d0b95 import-checker: establish modern import convention Gregory Szorc <gregory.szorc@gmail.com> parents: 25702 diff changeset	555 if lastentryname and n.name < lastentryname:
26955 c4114e335b49 import-checker: extract function to generate a formatted warning Yuya Nishihara <yuya@tcha.org> parents: 26954 diff changeset	556 yield msg('imports from %s not lexically sorted: %s < %s',
c4114e335b49 import-checker: extract function to generate a formatted warning Yuya Nishihara <yuya@tcha.org> parents: 26954 diff changeset	557 fullname, n.name, lastentryname)
25703 1a6a117d0b95 import-checker: establish modern import convention Gregory Szorc <gregory.szorc@gmail.com> parents: 25702 diff changeset	558
1a6a117d0b95 import-checker: establish modern import convention Gregory Szorc <gregory.szorc@gmail.com> parents: 25702 diff changeset	559 lastentryname = n.name
1a6a117d0b95 import-checker: establish modern import convention Gregory Szorc <gregory.szorc@gmail.com> parents: 25702 diff changeset	560
1a6a117d0b95 import-checker: establish modern import convention Gregory Szorc <gregory.szorc@gmail.com> parents: 25702 diff changeset	561 if n.name in requirealias and n.asname != requirealias[n.name]:
26955 c4114e335b49 import-checker: extract function to generate a formatted warning Yuya Nishihara <yuya@tcha.org> parents: 26954 diff changeset	562 yield msg('%s from %s must be "as" aliased to %s',
c4114e335b49 import-checker: extract function to generate a formatted warning Yuya Nishihara <yuya@tcha.org> parents: 26954 diff changeset	563 n.name, fullname, requirealias[n.name])
25702 ab2c5163900e import-checker: establish new function for verifying import conventions Gregory Szorc <gregory.szorc@gmail.com> parents: 25701 diff changeset	564
ab2c5163900e import-checker: establish new function for verifying import conventions Gregory Szorc <gregory.szorc@gmail.com> parents: 25701 diff changeset	565 def verify_stdlib_on_own_line(root):
20036 e5d51fa51aba contrib: add an import checker Augie Fackler <raf@durin42.com> parents: diff changeset	566 """Given some python source, verify that stdlib imports are done
e5d51fa51aba contrib: add an import checker Augie Fackler <raf@durin42.com> parents: diff changeset	567 in separate statements from relative local module imports.
e5d51fa51aba contrib: add an import checker Augie Fackler <raf@durin42.com> parents: diff changeset	568
25702 ab2c5163900e import-checker: establish new function for verifying import conventions Gregory Szorc <gregory.szorc@gmail.com> parents: 25701 diff changeset	569 >>> list(verify_stdlib_on_own_line(ast.parse('import sys, foo')))
26956 4b56214ebb7a import-checker: include lineno in warning message Yuya Nishihara <yuya@tcha.org> parents: 26955 diff changeset	570 [('mixed imports\\n stdlib: sys\\n relative: foo', 1)]
25702 ab2c5163900e import-checker: establish new function for verifying import conventions Gregory Szorc <gregory.szorc@gmail.com> parents: 25701 diff changeset	571 >>> list(verify_stdlib_on_own_line(ast.parse('import sys, os')))
20036 e5d51fa51aba contrib: add an import checker Augie Fackler <raf@durin42.com> parents: diff changeset	572 []
25702 ab2c5163900e import-checker: establish new function for verifying import conventions Gregory Szorc <gregory.szorc@gmail.com> parents: 25701 diff changeset	573 >>> list(verify_stdlib_on_own_line(ast.parse('import foo, bar')))
20036 e5d51fa51aba contrib: add an import checker Augie Fackler <raf@durin42.com> parents: diff changeset	574 []
e5d51fa51aba contrib: add an import checker Augie Fackler <raf@durin42.com> parents: diff changeset	575 """
25702 ab2c5163900e import-checker: establish new function for verifying import conventions Gregory Szorc <gregory.szorc@gmail.com> parents: 25701 diff changeset	576 for node in ast.walk(root):
20036 e5d51fa51aba contrib: add an import checker Augie Fackler <raf@durin42.com> parents: diff changeset	577 if isinstance(node, ast.Import):
20386 a05d31b040d7 import-checker: show stdlib and relative imports separately Mads Kiilerich <madski@unity3d.com> parents: 20383 diff changeset	578 from_stdlib = {False: [], True: []}
20036 e5d51fa51aba contrib: add an import checker Augie Fackler <raf@durin42.com> parents: diff changeset	579 for n in node.names:
20386 a05d31b040d7 import-checker: show stdlib and relative imports separately Mads Kiilerich <madski@unity3d.com> parents: 20383 diff changeset	580 from_stdlib[n.name in stdlib_modules].append(n.name)
a05d31b040d7 import-checker: show stdlib and relative imports separately Mads Kiilerich <madski@unity3d.com> parents: 20383 diff changeset	581 if from_stdlib[True] and from_stdlib[False]:
a05d31b040d7 import-checker: show stdlib and relative imports separately Mads Kiilerich <madski@unity3d.com> parents: 20383 diff changeset	582 yield ('mixed imports\n stdlib: %s\n relative: %s' %
a05d31b040d7 import-checker: show stdlib and relative imports separately Mads Kiilerich <madski@unity3d.com> parents: 20383 diff changeset	583 (', '.join(sorted(from_stdlib[True])),
26956 4b56214ebb7a import-checker: include lineno in warning message Yuya Nishihara <yuya@tcha.org> parents: 26955 diff changeset	584 ', '.join(sorted(from_stdlib[False]))), node.lineno)
20036 e5d51fa51aba contrib: add an import checker Augie Fackler <raf@durin42.com> parents: diff changeset	585
e5d51fa51aba contrib: add an import checker Augie Fackler <raf@durin42.com> parents: diff changeset	586 class CircularImport(Exception):
e5d51fa51aba contrib: add an import checker Augie Fackler <raf@durin42.com> parents: diff changeset	587 pass
e5d51fa51aba contrib: add an import checker Augie Fackler <raf@durin42.com> parents: diff changeset	588
24490 fb4639d5268e import-checker: make search algorithm non-recursive breadth-first Matt Mackall <mpm@selenic.com> parents: 24489 diff changeset	589 def checkmod(mod, imports):
fb4639d5268e import-checker: make search algorithm non-recursive breadth-first Matt Mackall <mpm@selenic.com> parents: 24489 diff changeset	590 shortest = {}
fb4639d5268e import-checker: make search algorithm non-recursive breadth-first Matt Mackall <mpm@selenic.com> parents: 24489 diff changeset	591 visit = [[mod]]
fb4639d5268e import-checker: make search algorithm non-recursive breadth-first Matt Mackall <mpm@selenic.com> parents: 24489 diff changeset	592 while visit:
fb4639d5268e import-checker: make search algorithm non-recursive breadth-first Matt Mackall <mpm@selenic.com> parents: 24489 diff changeset	593 path = visit.pop(0)
fb4639d5268e import-checker: make search algorithm non-recursive breadth-first Matt Mackall <mpm@selenic.com> parents: 24489 diff changeset	594 for i in sorted(imports.get(path[-1], [])):
fb4639d5268e import-checker: make search algorithm non-recursive breadth-first Matt Mackall <mpm@selenic.com> parents: 24489 diff changeset	595 if len(path) < shortest.get(i, 1000):
fb4639d5268e import-checker: make search algorithm non-recursive breadth-first Matt Mackall <mpm@selenic.com> parents: 24489 diff changeset	596 shortest[i] = len(path)
fb4639d5268e import-checker: make search algorithm non-recursive breadth-first Matt Mackall <mpm@selenic.com> parents: 24489 diff changeset	597 if i in path:
fb4639d5268e import-checker: make search algorithm non-recursive breadth-first Matt Mackall <mpm@selenic.com> parents: 24489 diff changeset	598 if i == path[0]:
fb4639d5268e import-checker: make search algorithm non-recursive breadth-first Matt Mackall <mpm@selenic.com> parents: 24489 diff changeset	599 raise CircularImport(path)
fb4639d5268e import-checker: make search algorithm non-recursive breadth-first Matt Mackall <mpm@selenic.com> parents: 24489 diff changeset	600 continue
fb4639d5268e import-checker: make search algorithm non-recursive breadth-first Matt Mackall <mpm@selenic.com> parents: 24489 diff changeset	601 visit.append(path + [i])
20036 e5d51fa51aba contrib: add an import checker Augie Fackler <raf@durin42.com> parents: diff changeset	602
20038 c65a6937b828 import-checker: try a little harder to show fewer cycles Augie Fackler <raf@durin42.com> parents: 20037 diff changeset	603 def rotatecycle(cycle):
c65a6937b828 import-checker: try a little harder to show fewer cycles Augie Fackler <raf@durin42.com> parents: 20037 diff changeset	604 """arrange a cycle so that the lexicographically first module listed first
c65a6937b828 import-checker: try a little harder to show fewer cycles Augie Fackler <raf@durin42.com> parents: 20037 diff changeset	605
24488 4b3fc46097f7 import-checker: drop duplicate element from cycle Matt Mackall <mpm@selenic.com> parents: 24487 diff changeset	606 >>> rotatecycle(['foo', 'bar'])
20038 c65a6937b828 import-checker: try a little harder to show fewer cycles Augie Fackler <raf@durin42.com> parents: 20037 diff changeset	607 ['bar', 'foo', 'bar']
c65a6937b828 import-checker: try a little harder to show fewer cycles Augie Fackler <raf@durin42.com> parents: 20037 diff changeset	608 """
c65a6937b828 import-checker: try a little harder to show fewer cycles Augie Fackler <raf@durin42.com> parents: 20037 diff changeset	609 lowest = min(cycle)
c65a6937b828 import-checker: try a little harder to show fewer cycles Augie Fackler <raf@durin42.com> parents: 20037 diff changeset	610 idx = cycle.index(lowest)
24488 4b3fc46097f7 import-checker: drop duplicate element from cycle Matt Mackall <mpm@selenic.com> parents: 24487 diff changeset	611 return cycle[idx:] + cycle[:idx] + [lowest]
20036 e5d51fa51aba contrib: add an import checker Augie Fackler <raf@durin42.com> parents: diff changeset	612
e5d51fa51aba contrib: add an import checker Augie Fackler <raf@durin42.com> parents: diff changeset	613 def find_cycles(imports):
e5d51fa51aba contrib: add an import checker Augie Fackler <raf@durin42.com> parents: diff changeset	614 """Find cycles in an already-loaded import graph.
e5d51fa51aba contrib: add an import checker Augie Fackler <raf@durin42.com> parents: diff changeset	615
25175 10e6c4b7121b import-checker: don't treat modules as relative one if not found FUJIWARA Katsunori <foozy@lares.dti.ne.jp> parents: 25174 diff changeset	616 All module names recorded in `imports` should be absolute one.
10e6c4b7121b import-checker: don't treat modules as relative one if not found FUJIWARA Katsunori <foozy@lares.dti.ne.jp> parents: 25174 diff changeset	617
28703 a274c4f9087a py3: use print_function in import-checker timeless <timeless@mozdev.org> parents: 28702 diff changeset	618 >>> from __future__ import print_function
25175 10e6c4b7121b import-checker: don't treat modules as relative one if not found FUJIWARA Katsunori <foozy@lares.dti.ne.jp> parents: 25174 diff changeset	619 >>> imports = {'top.foo': ['top.bar', 'os.path', 'top.qux'],
10e6c4b7121b import-checker: don't treat modules as relative one if not found FUJIWARA Katsunori <foozy@lares.dti.ne.jp> parents: 25174 diff changeset	620 ... 'top.bar': ['top.baz', 'sys'],
10e6c4b7121b import-checker: don't treat modules as relative one if not found FUJIWARA Katsunori <foozy@lares.dti.ne.jp> parents: 25174 diff changeset	621 ... 'top.baz': ['top.foo'],
10e6c4b7121b import-checker: don't treat modules as relative one if not found FUJIWARA Katsunori <foozy@lares.dti.ne.jp> parents: 25174 diff changeset	622 ... 'top.qux': ['top.foo']}
28703 a274c4f9087a py3: use print_function in import-checker timeless <timeless@mozdev.org> parents: 28702 diff changeset	623 >>> print('\\n'.join(sorted(find_cycles(imports))))
24487 642d245ff537 import-checker: fix rotatecycle Matt Mackall <mpm@selenic.com> parents: 22975 diff changeset	624 top.bar -> top.baz -> top.foo -> top.bar
642d245ff537 import-checker: fix rotatecycle Matt Mackall <mpm@selenic.com> parents: 22975 diff changeset	625 top.foo -> top.qux -> top.foo
20036 e5d51fa51aba contrib: add an import checker Augie Fackler <raf@durin42.com> parents: diff changeset	626 """
24491 784b278b349c import-checker: rotatecycle is actually the canonical cycle key Matt Mackall <mpm@selenic.com> parents: 24490 diff changeset	627 cycles = set()
28704 1fa6fdb72275 py3: handle iter/iterkeys+iteritems python3 divergence in import-checker timeless <timeless@mozdev.org> parents: 28703 diff changeset	628 for mod in sorted(imports.keys()):
20036 e5d51fa51aba contrib: add an import checker Augie Fackler <raf@durin42.com> parents: diff changeset	629 try:
24490 fb4639d5268e import-checker: make search algorithm non-recursive breadth-first Matt Mackall <mpm@selenic.com> parents: 24489 diff changeset	630 checkmod(mod, imports)
25660 328739ea70c3 global: mass rewrite to use modern exception syntax Gregory Szorc <gregory.szorc@gmail.com> parents: 25175 diff changeset	631 except CircularImport as e:
20036 e5d51fa51aba contrib: add an import checker Augie Fackler <raf@durin42.com> parents: diff changeset	632 cycle = e.args[0]
24491 784b278b349c import-checker: rotatecycle is actually the canonical cycle key Matt Mackall <mpm@selenic.com> parents: 24490 diff changeset	633 cycles.add(" -> ".join(rotatecycle(cycle)))
784b278b349c import-checker: rotatecycle is actually the canonical cycle key Matt Mackall <mpm@selenic.com> parents: 24490 diff changeset	634 return cycles
20036 e5d51fa51aba contrib: add an import checker Augie Fackler <raf@durin42.com> parents: diff changeset	635
e5d51fa51aba contrib: add an import checker Augie Fackler <raf@durin42.com> parents: diff changeset	636 def _cycle_sortkey(c):
e5d51fa51aba contrib: add an import checker Augie Fackler <raf@durin42.com> parents: diff changeset	637 return len(c), c
e5d51fa51aba contrib: add an import checker Augie Fackler <raf@durin42.com> parents: diff changeset	638
28922 4ec62a084e5c import-checker: parse python code from .t files timeless <timeless@mozdev.org> parents: 28921 diff changeset	639 def embedded(f, modname, src):
4ec62a084e5c import-checker: parse python code from .t files timeless <timeless@mozdev.org> parents: 28921 diff changeset	640 """Extract embedded python code
4ec62a084e5c import-checker: parse python code from .t files timeless <timeless@mozdev.org> parents: 28921 diff changeset	641
33877 bcf53149ebce contrib: make import checker always think in terms of bytes Augie Fackler <raf@durin42.com> parents: 33876 diff changeset	642 >>> def _forcestr(thing):
bcf53149ebce contrib: make import checker always think in terms of bytes Augie Fackler <raf@durin42.com> parents: 33876 diff changeset	643 ... if not isinstance(thing, str):
bcf53149ebce contrib: make import checker always think in terms of bytes Augie Fackler <raf@durin42.com> parents: 33876 diff changeset	644 ... return thing.decode('ascii')
bcf53149ebce contrib: make import checker always think in terms of bytes Augie Fackler <raf@durin42.com> parents: 33876 diff changeset	645 ... return thing
28922 4ec62a084e5c import-checker: parse python code from .t files timeless <timeless@mozdev.org> parents: 28921 diff changeset	646 >>> def test(fn, lines):
33877 bcf53149ebce contrib: make import checker always think in terms of bytes Augie Fackler <raf@durin42.com> parents: 33876 diff changeset	647 ... for s, m, f, l in embedded(fn, b"example", lines):
bcf53149ebce contrib: make import checker always think in terms of bytes Augie Fackler <raf@durin42.com> parents: 33876 diff changeset	648 ... print("%s %s %d" % (_forcestr(m), _forcestr(f), l))
bcf53149ebce contrib: make import checker always think in terms of bytes Augie Fackler <raf@durin42.com> parents: 33876 diff changeset	649 ... print(repr(_forcestr(s)))
28922 4ec62a084e5c import-checker: parse python code from .t files timeless <timeless@mozdev.org> parents: 28921 diff changeset	650 >>> lines = [
33877 bcf53149ebce contrib: make import checker always think in terms of bytes Augie Fackler <raf@durin42.com> parents: 33876 diff changeset	651 ... b'comment',
bcf53149ebce contrib: make import checker always think in terms of bytes Augie Fackler <raf@durin42.com> parents: 33876 diff changeset	652 ... b' >>> from __future__ import print_function',
bcf53149ebce contrib: make import checker always think in terms of bytes Augie Fackler <raf@durin42.com> parents: 33876 diff changeset	653 ... b" >>> ' multiline",
bcf53149ebce contrib: make import checker always think in terms of bytes Augie Fackler <raf@durin42.com> parents: 33876 diff changeset	654 ... b" ... string'",
bcf53149ebce contrib: make import checker always think in terms of bytes Augie Fackler <raf@durin42.com> parents: 33876 diff changeset	655 ... b' ',
bcf53149ebce contrib: make import checker always think in terms of bytes Augie Fackler <raf@durin42.com> parents: 33876 diff changeset	656 ... b'comment',
bcf53149ebce contrib: make import checker always think in terms of bytes Augie Fackler <raf@durin42.com> parents: 33876 diff changeset	657 ... b' $ cat > foo.py <<EOF',
bcf53149ebce contrib: make import checker always think in terms of bytes Augie Fackler <raf@durin42.com> parents: 33876 diff changeset	658 ... b' > from __future__ import print_function',
bcf53149ebce contrib: make import checker always think in terms of bytes Augie Fackler <raf@durin42.com> parents: 33876 diff changeset	659 ... b' > EOF',
28922 4ec62a084e5c import-checker: parse python code from .t files timeless <timeless@mozdev.org> parents: 28921 diff changeset	660 ... ]
33877 bcf53149ebce contrib: make import checker always think in terms of bytes Augie Fackler <raf@durin42.com> parents: 33876 diff changeset	661 >>> test(b"example.t", lines)
28922 4ec62a084e5c import-checker: parse python code from .t files timeless <timeless@mozdev.org> parents: 28921 diff changeset	662 example[2] doctest.py 2
4ec62a084e5c import-checker: parse python code from .t files timeless <timeless@mozdev.org> parents: 28921 diff changeset	663 "from __future__ import print_function\\n' multiline\\nstring'\\n"
4ec62a084e5c import-checker: parse python code from .t files timeless <timeless@mozdev.org> parents: 28921 diff changeset	664 example[7] foo.py 7
4ec62a084e5c import-checker: parse python code from .t files timeless <timeless@mozdev.org> parents: 28921 diff changeset	665 'from __future__ import print_function\\n'
4ec62a084e5c import-checker: parse python code from .t files timeless <timeless@mozdev.org> parents: 28921 diff changeset	666 """
4ec62a084e5c import-checker: parse python code from .t files timeless <timeless@mozdev.org> parents: 28921 diff changeset	667 inlinepython = 0
4ec62a084e5c import-checker: parse python code from .t files timeless <timeless@mozdev.org> parents: 28921 diff changeset	668 shpython = 0
4ec62a084e5c import-checker: parse python code from .t files timeless <timeless@mozdev.org> parents: 28921 diff changeset	669 script = []
4ec62a084e5c import-checker: parse python code from .t files timeless <timeless@mozdev.org> parents: 28921 diff changeset	670 prefix = 6
4ec62a084e5c import-checker: parse python code from .t files timeless <timeless@mozdev.org> parents: 28921 diff changeset	671 t = ''
4ec62a084e5c import-checker: parse python code from .t files timeless <timeless@mozdev.org> parents: 28921 diff changeset	672 n = 0
4ec62a084e5c import-checker: parse python code from .t files timeless <timeless@mozdev.org> parents: 28921 diff changeset	673 for l in src:
4ec62a084e5c import-checker: parse python code from .t files timeless <timeless@mozdev.org> parents: 28921 diff changeset	674 n += 1
4ec62a084e5c import-checker: parse python code from .t files timeless <timeless@mozdev.org> parents: 28921 diff changeset	675 if not l.endswith(b'\n'):
4ec62a084e5c import-checker: parse python code from .t files timeless <timeless@mozdev.org> parents: 28921 diff changeset	676 l += b'\n'
4ec62a084e5c import-checker: parse python code from .t files timeless <timeless@mozdev.org> parents: 28921 diff changeset	677 if l.startswith(b' >>> '): # python inlines
4ec62a084e5c import-checker: parse python code from .t files timeless <timeless@mozdev.org> parents: 28921 diff changeset	678 if shpython:
4ec62a084e5c import-checker: parse python code from .t files timeless <timeless@mozdev.org> parents: 28921 diff changeset	679 print("%s:%d: Parse Error" % (f, n))
4ec62a084e5c import-checker: parse python code from .t files timeless <timeless@mozdev.org> parents: 28921 diff changeset	680 if not inlinepython:
4ec62a084e5c import-checker: parse python code from .t files timeless <timeless@mozdev.org> parents: 28921 diff changeset	681 # We've just entered a Python block.
4ec62a084e5c import-checker: parse python code from .t files timeless <timeless@mozdev.org> parents: 28921 diff changeset	682 inlinepython = n
33877 bcf53149ebce contrib: make import checker always think in terms of bytes Augie Fackler <raf@durin42.com> parents: 33876 diff changeset	683 t = b'doctest.py'
28922 4ec62a084e5c import-checker: parse python code from .t files timeless <timeless@mozdev.org> parents: 28921 diff changeset	684 script.append(l[prefix:])
4ec62a084e5c import-checker: parse python code from .t files timeless <timeless@mozdev.org> parents: 28921 diff changeset	685 continue
4ec62a084e5c import-checker: parse python code from .t files timeless <timeless@mozdev.org> parents: 28921 diff changeset	686 if l.startswith(b' ... '): # python inlines
4ec62a084e5c import-checker: parse python code from .t files timeless <timeless@mozdev.org> parents: 28921 diff changeset	687 script.append(l[prefix:])
4ec62a084e5c import-checker: parse python code from .t files timeless <timeless@mozdev.org> parents: 28921 diff changeset	688 continue
33877 bcf53149ebce contrib: make import checker always think in terms of bytes Augie Fackler <raf@durin42.com> parents: 33876 diff changeset	689 cat = re.search(br"\$ \scat\s>\s(\S+\.py)\s<<\s*EOF", l)
28922 4ec62a084e5c import-checker: parse python code from .t files timeless <timeless@mozdev.org> parents: 28921 diff changeset	690 if cat:
4ec62a084e5c import-checker: parse python code from .t files timeless <timeless@mozdev.org> parents: 28921 diff changeset	691 if inlinepython:
33877 bcf53149ebce contrib: make import checker always think in terms of bytes Augie Fackler <raf@durin42.com> parents: 33876 diff changeset	692 yield b''.join(script), (b"%s[%d]" %
28922 4ec62a084e5c import-checker: parse python code from .t files timeless <timeless@mozdev.org> parents: 28921 diff changeset	693 (modname, inlinepython)), t, inlinepython
4ec62a084e5c import-checker: parse python code from .t files timeless <timeless@mozdev.org> parents: 28921 diff changeset	694 script = []
4ec62a084e5c import-checker: parse python code from .t files timeless <timeless@mozdev.org> parents: 28921 diff changeset	695 inlinepython = 0
4ec62a084e5c import-checker: parse python code from .t files timeless <timeless@mozdev.org> parents: 28921 diff changeset	696 shpython = n
4ec62a084e5c import-checker: parse python code from .t files timeless <timeless@mozdev.org> parents: 28921 diff changeset	697 t = cat.group(1)
4ec62a084e5c import-checker: parse python code from .t files timeless <timeless@mozdev.org> parents: 28921 diff changeset	698 continue
4ec62a084e5c import-checker: parse python code from .t files timeless <timeless@mozdev.org> parents: 28921 diff changeset	699 if shpython and l.startswith(b' > '): # sh continuation
4ec62a084e5c import-checker: parse python code from .t files timeless <timeless@mozdev.org> parents: 28921 diff changeset	700 if l == b' > EOF\n':
33877 bcf53149ebce contrib: make import checker always think in terms of bytes Augie Fackler <raf@durin42.com> parents: 33876 diff changeset	701 yield b''.join(script), (b"%s[%d]" %
28922 4ec62a084e5c import-checker: parse python code from .t files timeless <timeless@mozdev.org> parents: 28921 diff changeset	702 (modname, shpython)), t, shpython
4ec62a084e5c import-checker: parse python code from .t files timeless <timeless@mozdev.org> parents: 28921 diff changeset	703 script = []
4ec62a084e5c import-checker: parse python code from .t files timeless <timeless@mozdev.org> parents: 28921 diff changeset	704 shpython = 0
4ec62a084e5c import-checker: parse python code from .t files timeless <timeless@mozdev.org> parents: 28921 diff changeset	705 else:
4ec62a084e5c import-checker: parse python code from .t files timeless <timeless@mozdev.org> parents: 28921 diff changeset	706 script.append(l[4:])
4ec62a084e5c import-checker: parse python code from .t files timeless <timeless@mozdev.org> parents: 28921 diff changeset	707 continue
33920 8a8dd6e4a97a contrib: make import-checker agree more with run-tests about heredocpy Augie Fackler <raf@durin42.com> parents: 33919 diff changeset	708 # If we have an empty line or a command for sh, we end the
8a8dd6e4a97a contrib: make import-checker agree more with run-tests about heredocpy Augie Fackler <raf@durin42.com> parents: 33919 diff changeset	709 # inline script.
8a8dd6e4a97a contrib: make import-checker agree more with run-tests about heredocpy Augie Fackler <raf@durin42.com> parents: 33919 diff changeset	710 if inlinepython and (l == b' \n'
8a8dd6e4a97a contrib: make import-checker agree more with run-tests about heredocpy Augie Fackler <raf@durin42.com> parents: 33919 diff changeset	711 or l.startswith(b' $ ')):
33877 bcf53149ebce contrib: make import checker always think in terms of bytes Augie Fackler <raf@durin42.com> parents: 33876 diff changeset	712 yield b''.join(script), (b"%s[%d]" %
28922 4ec62a084e5c import-checker: parse python code from .t files timeless <timeless@mozdev.org> parents: 28921 diff changeset	713 (modname, inlinepython)), t, inlinepython
4ec62a084e5c import-checker: parse python code from .t files timeless <timeless@mozdev.org> parents: 28921 diff changeset	714 script = []
4ec62a084e5c import-checker: parse python code from .t files timeless <timeless@mozdev.org> parents: 28921 diff changeset	715 inlinepython = 0
4ec62a084e5c import-checker: parse python code from .t files timeless <timeless@mozdev.org> parents: 28921 diff changeset	716 continue
4ec62a084e5c import-checker: parse python code from .t files timeless <timeless@mozdev.org> parents: 28921 diff changeset	717
28919 a94f34306bb9 import-checker: refactor source reading timeless <timeless@mozdev.org> parents: 28713 diff changeset	718 def sources(f, modname):
28922 4ec62a084e5c import-checker: parse python code from .t files timeless <timeless@mozdev.org> parents: 28921 diff changeset	719 """Yields possibly multiple sources from a filepath
4ec62a084e5c import-checker: parse python code from .t files timeless <timeless@mozdev.org> parents: 28921 diff changeset	720
4ec62a084e5c import-checker: parse python code from .t files timeless <timeless@mozdev.org> parents: 28921 diff changeset	721 input: filepath, modulename
4ec62a084e5c import-checker: parse python code from .t files timeless <timeless@mozdev.org> parents: 28921 diff changeset	722 yields: script(string), modulename, filepath, linenumber
4ec62a084e5c import-checker: parse python code from .t files timeless <timeless@mozdev.org> parents: 28921 diff changeset	723
4ec62a084e5c import-checker: parse python code from .t files timeless <timeless@mozdev.org> parents: 28921 diff changeset	724 For embedded scripts, the modulename and filepath will be different
4ec62a084e5c import-checker: parse python code from .t files timeless <timeless@mozdev.org> parents: 28921 diff changeset	725 from the function arguments. linenumber is an offset relative to
4ec62a084e5c import-checker: parse python code from .t files timeless <timeless@mozdev.org> parents: 28921 diff changeset	726 the input file.
4ec62a084e5c import-checker: parse python code from .t files timeless <timeless@mozdev.org> parents: 28921 diff changeset	727 """
4ec62a084e5c import-checker: parse python code from .t files timeless <timeless@mozdev.org> parents: 28921 diff changeset	728 py = False
29234 393aef802535 tests: enable import checker for all python files (including no .py files) Yuya Nishihara <yuya@tcha.org> parents: 29211 diff changeset	729 if not f.endswith('.t'):
33877 bcf53149ebce contrib: make import checker always think in terms of bytes Augie Fackler <raf@durin42.com> parents: 33876 diff changeset	730 with open(f, 'rb') as src:
28922 4ec62a084e5c import-checker: parse python code from .t files timeless <timeless@mozdev.org> parents: 28921 diff changeset	731 yield src.read(), modname, f, 0
4ec62a084e5c import-checker: parse python code from .t files timeless <timeless@mozdev.org> parents: 28921 diff changeset	732 py = True
4ec62a084e5c import-checker: parse python code from .t files timeless <timeless@mozdev.org> parents: 28921 diff changeset	733 if py or f.endswith('.t'):
33877 bcf53149ebce contrib: make import checker always think in terms of bytes Augie Fackler <raf@durin42.com> parents: 33876 diff changeset	734 with open(f, 'rb') as src:
28922 4ec62a084e5c import-checker: parse python code from .t files timeless <timeless@mozdev.org> parents: 28921 diff changeset	735 for script, modname, t, line in embedded(f, modname, src):
4ec62a084e5c import-checker: parse python code from .t files timeless <timeless@mozdev.org> parents: 28921 diff changeset	736 yield script, modname, t, line
28919 a94f34306bb9 import-checker: refactor source reading timeless <timeless@mozdev.org> parents: 28713 diff changeset	737
20036 e5d51fa51aba contrib: add an import checker Augie Fackler <raf@durin42.com> parents: diff changeset	738 def main(argv):
25063 723e364488f4 import-checker: add xargs like mode FUJIWARA Katsunori <foozy@lares.dti.ne.jp> parents: 24669 diff changeset	739 if len(argv) < 2 or (argv[1] == '-' and len(argv) > 2):
28703 a274c4f9087a py3: use print_function in import-checker timeless <timeless@mozdev.org> parents: 28702 diff changeset	740 print('Usage: %s {-\|file [file] [file] ...}')
20036 e5d51fa51aba contrib: add an import checker Augie Fackler <raf@durin42.com> parents: diff changeset	741 return 1
25063 723e364488f4 import-checker: add xargs like mode FUJIWARA Katsunori <foozy@lares.dti.ne.jp> parents: 24669 diff changeset	742 if argv[1] == '-':
723e364488f4 import-checker: add xargs like mode FUJIWARA Katsunori <foozy@lares.dti.ne.jp> parents: 24669 diff changeset	743 argv = argv[:1]
723e364488f4 import-checker: add xargs like mode FUJIWARA Katsunori <foozy@lares.dti.ne.jp> parents: 24669 diff changeset	744 argv.extend(l.rstrip() for l in sys.stdin.readlines())
32508 4c712b90c60a import-checker: convert localmods to a set of module names Yuya Nishihara <yuya@tcha.org> parents: 32507 diff changeset	745 localmodpaths = {}
20036 e5d51fa51aba contrib: add an import checker Augie Fackler <raf@durin42.com> parents: diff changeset	746 used_imports = {}
e5d51fa51aba contrib: add an import checker Augie Fackler <raf@durin42.com> parents: diff changeset	747 any_errors = False
e5d51fa51aba contrib: add an import checker Augie Fackler <raf@durin42.com> parents: diff changeset	748 for source_path in argv[1:]:
32374 194b0f781132 import-checker: drop workaround for pure modules Yuya Nishihara <yuya@tcha.org> parents: 32372 diff changeset	749 modname = dotted_name_of_path(source_path)
32508 4c712b90c60a import-checker: convert localmods to a set of module names Yuya Nishihara <yuya@tcha.org> parents: 32507 diff changeset	750 localmodpaths[modname] = source_path
32509 a025ec43856c import-checker: guess names of C extension modules Yuya Nishihara <yuya@tcha.org> parents: 32508 diff changeset	751 localmods = populateextmods(localmodpaths)
32508 4c712b90c60a import-checker: convert localmods to a set of module names Yuya Nishihara <yuya@tcha.org> parents: 32507 diff changeset	752 for localmodname, source_path in sorted(localmodpaths.items()):
33890 3de9a2df6411 contrib: have import-checker work mostly with native strings for mod names Augie Fackler <raf@durin42.com> parents: 33877 diff changeset	753 if not isinstance(localmodname, bytes):
3de9a2df6411 contrib: have import-checker work mostly with native strings for mod names Augie Fackler <raf@durin42.com> parents: 33877 diff changeset	754 # This is only safe because all hg's files are ascii
3de9a2df6411 contrib: have import-checker work mostly with native strings for mod names Augie Fackler <raf@durin42.com> parents: 33877 diff changeset	755 localmodname = localmodname.encode('ascii')
28922 4ec62a084e5c import-checker: parse python code from .t files timeless <timeless@mozdev.org> parents: 28921 diff changeset	756 for src, modname, name, line in sources(source_path, localmodname):
28920 cdf331b54eb8 import-checker: track SyntaxErrors timeless <timeless@mozdev.org> parents: 28919 diff changeset	757 try:
cdf331b54eb8 import-checker: track SyntaxErrors timeless <timeless@mozdev.org> parents: 28919 diff changeset	758 used_imports[modname] = sorted(
28922 4ec62a084e5c import-checker: parse python code from .t files timeless <timeless@mozdev.org> parents: 28921 diff changeset	759 imported_modules(src, modname, name, localmods,
28920 cdf331b54eb8 import-checker: track SyntaxErrors timeless <timeless@mozdev.org> parents: 28919 diff changeset	760 ignore_nested=True))
cdf331b54eb8 import-checker: track SyntaxErrors timeless <timeless@mozdev.org> parents: 28919 diff changeset	761 for error, lineno in verify_import_convention(modname, src,
cdf331b54eb8 import-checker: track SyntaxErrors timeless <timeless@mozdev.org> parents: 28919 diff changeset	762 localmods):
cdf331b54eb8 import-checker: track SyntaxErrors timeless <timeless@mozdev.org> parents: 28919 diff changeset	763 any_errors = True
28922 4ec62a084e5c import-checker: parse python code from .t files timeless <timeless@mozdev.org> parents: 28921 diff changeset	764 print('%s:%d: %s' % (source_path, lineno + line, error))
28920 cdf331b54eb8 import-checker: track SyntaxErrors timeless <timeless@mozdev.org> parents: 28919 diff changeset	765 except SyntaxError as e:
cdf331b54eb8 import-checker: track SyntaxErrors timeless <timeless@mozdev.org> parents: 28919 diff changeset	766 print('%s:%d: SyntaxError: %s' %
28922 4ec62a084e5c import-checker: parse python code from .t files timeless <timeless@mozdev.org> parents: 28921 diff changeset	767 (source_path, e.lineno + line, e))
20036 e5d51fa51aba contrib: add an import checker Augie Fackler <raf@durin42.com> parents: diff changeset	768 cycles = find_cycles(used_imports)
e5d51fa51aba contrib: add an import checker Augie Fackler <raf@durin42.com> parents: diff changeset	769 if cycles:
e5d51fa51aba contrib: add an import checker Augie Fackler <raf@durin42.com> parents: diff changeset	770 firstmods = set()
e5d51fa51aba contrib: add an import checker Augie Fackler <raf@durin42.com> parents: diff changeset	771 for c in sorted(cycles, key=_cycle_sortkey):
e5d51fa51aba contrib: add an import checker Augie Fackler <raf@durin42.com> parents: diff changeset	772 first = c.split()[0]
e5d51fa51aba contrib: add an import checker Augie Fackler <raf@durin42.com> parents: diff changeset	773 # As a rough cut, ignore any cycle that starts with the
e5d51fa51aba contrib: add an import checker Augie Fackler <raf@durin42.com> parents: diff changeset	774 # same module as some other cycle. Otherwise we see lots
e5d51fa51aba contrib: add an import checker Augie Fackler <raf@durin42.com> parents: diff changeset	775 # of cycles that are effectively duplicates.
e5d51fa51aba contrib: add an import checker Augie Fackler <raf@durin42.com> parents: diff changeset	776 if first in firstmods:
e5d51fa51aba contrib: add an import checker Augie Fackler <raf@durin42.com> parents: diff changeset	777 continue
28703 a274c4f9087a py3: use print_function in import-checker timeless <timeless@mozdev.org> parents: 28702 diff changeset	778 print('Import cycle:', c)
20036 e5d51fa51aba contrib: add an import checker Augie Fackler <raf@durin42.com> parents: diff changeset	779 firstmods.add(first)
e5d51fa51aba contrib: add an import checker Augie Fackler <raf@durin42.com> parents: diff changeset	780 any_errors = True
25731 cd1daab5d036 import-checker.py: exit with code 0 if no error is detected FUJIWARA Katsunori <foozy@lares.dti.ne.jp> parents: 25703 diff changeset	781 return any_errors != 0
20036 e5d51fa51aba contrib: add an import checker Augie Fackler <raf@durin42.com> parents: diff changeset	782
e5d51fa51aba contrib: add an import checker Augie Fackler <raf@durin42.com> parents: diff changeset	783 if __name__ == '__main__':
e5d51fa51aba contrib: add an import checker Augie Fackler <raf@durin42.com> parents: diff changeset	784 sys.exit(int(main(sys.argv)))

Mercurial > public > mercurial-scm > hg

annotate contrib/import-checker.py @ 39329:729082bb9938