Mercurial > public > mercurial-scm > hg-stable
annotate mercurial/demandimport.py @ 26117:4dc5b51f38fe
revlog: change generaldelta delta parent heuristic
The old generaldelta heuristic was "if p1 (or p2) was closer than the last full text,
use it, otherwise use prev". This was problematic when a repo contained multiple
branches that were very different. If commits to branch A were pushed, and the
last full text was branch B, it would generate a fulltext. Then if branch B was
pushed, it would generate another fulltext. The problem is that the last
fulltext (and delta'ing against `prev` in general) has no correlation with the
contents of the incoming revision, and therefore will always have degenerate
cases.
According to the blame, that algorithm was chosen to minimize the chain length.
Since there is already code that protects against that (the delta-vs-fulltext
code), and since it has been improved since the original generaldelta algorithm
went in (2011), I believe the chain length criteria will still be preserved.
The new algorithm always diffs against p1 (or p2 if it's closer), unless the
resulting delta will fail the delta-vs-fulltext check, in which case we delta
against prev.
Some before and after stats on manifest.d size.
internal large repo
old heuristic - 2.0 GB
new heuristic - 1.2 GB
mozilla-central
old heuristic - 242 MB
new heuristic - 261 MB
The regression in mozilla central is due to the new heuristic choosing p2r as
the delta when it's closer to the tip. Switching the algorithm to always prefer
p1r brings the size back down (242 MB). This is result of the way in which
mozilla does merges and pushes, and the result could easily swing the other
direction in other repos (depending on if they merge X into Y or Y into X), but
will never be as degenerate as before.
I future patch will address the regression by introducing an optional, even more
aggressive delta heuristic which will knock the mozilla manifest size down
dramatically.
author | Durham Goode <durham@fb.com> |
---|---|
date | Sun, 30 Aug 2015 13:58:11 -0700 |
parents | 3beed01daff9 |
children | f2bf76d3d567 |
rev | line source |
---|---|
3877
abaee83ce0a6
Replace demandload with new demandimport
Matt Mackall <mpm@selenic.com>
parents:
diff
changeset
|
1 # demandimport.py - global demand-loading of modules for Mercurial |
abaee83ce0a6
Replace demandload with new demandimport
Matt Mackall <mpm@selenic.com>
parents:
diff
changeset
|
2 # |
4635
63b9d2deed48
Updated copyright notices and add "and others" to "hg version"
Thomas Arendsen Hein <thomas@intevation.de>
parents:
4631
diff
changeset
|
3 # Copyright 2006, 2007 Matt Mackall <mpm@selenic.com> |
3877
abaee83ce0a6
Replace demandload with new demandimport
Matt Mackall <mpm@selenic.com>
parents:
diff
changeset
|
4 # |
8225
46293a0c7e9f
updated license to be explicit about GPL version 2
Martin Geisler <mg@lazybytes.net>
parents:
7861
diff
changeset
|
5 # This software may be used and distributed according to the terms of the |
10263 | 6 # GNU General Public License version 2 or any later version. |
3877
abaee83ce0a6
Replace demandload with new demandimport
Matt Mackall <mpm@selenic.com>
parents:
diff
changeset
|
7 |
abaee83ce0a6
Replace demandload with new demandimport
Matt Mackall <mpm@selenic.com>
parents:
diff
changeset
|
8 ''' |
abaee83ce0a6
Replace demandload with new demandimport
Matt Mackall <mpm@selenic.com>
parents:
diff
changeset
|
9 demandimport - automatic demandloading of modules |
abaee83ce0a6
Replace demandload with new demandimport
Matt Mackall <mpm@selenic.com>
parents:
diff
changeset
|
10 |
abaee83ce0a6
Replace demandload with new demandimport
Matt Mackall <mpm@selenic.com>
parents:
diff
changeset
|
11 To enable this module, do: |
abaee83ce0a6
Replace demandload with new demandimport
Matt Mackall <mpm@selenic.com>
parents:
diff
changeset
|
12 |
abaee83ce0a6
Replace demandload with new demandimport
Matt Mackall <mpm@selenic.com>
parents:
diff
changeset
|
13 import demandimport; demandimport.enable() |
abaee83ce0a6
Replace demandload with new demandimport
Matt Mackall <mpm@selenic.com>
parents:
diff
changeset
|
14 |
abaee83ce0a6
Replace demandload with new demandimport
Matt Mackall <mpm@selenic.com>
parents:
diff
changeset
|
15 Imports of the following forms will be demand-loaded: |
abaee83ce0a6
Replace demandload with new demandimport
Matt Mackall <mpm@selenic.com>
parents:
diff
changeset
|
16 |
abaee83ce0a6
Replace demandload with new demandimport
Matt Mackall <mpm@selenic.com>
parents:
diff
changeset
|
17 import a, b.c |
abaee83ce0a6
Replace demandload with new demandimport
Matt Mackall <mpm@selenic.com>
parents:
diff
changeset
|
18 import a.b as c |
abaee83ce0a6
Replace demandload with new demandimport
Matt Mackall <mpm@selenic.com>
parents:
diff
changeset
|
19 from a import b,c # a will be loaded immediately |
abaee83ce0a6
Replace demandload with new demandimport
Matt Mackall <mpm@selenic.com>
parents:
diff
changeset
|
20 |
abaee83ce0a6
Replace demandload with new demandimport
Matt Mackall <mpm@selenic.com>
parents:
diff
changeset
|
21 These imports will not be delayed: |
abaee83ce0a6
Replace demandload with new demandimport
Matt Mackall <mpm@selenic.com>
parents:
diff
changeset
|
22 |
abaee83ce0a6
Replace demandload with new demandimport
Matt Mackall <mpm@selenic.com>
parents:
diff
changeset
|
23 from a import * |
abaee83ce0a6
Replace demandload with new demandimport
Matt Mackall <mpm@selenic.com>
parents:
diff
changeset
|
24 b = __import__(a) |
abaee83ce0a6
Replace demandload with new demandimport
Matt Mackall <mpm@selenic.com>
parents:
diff
changeset
|
25 ''' |
abaee83ce0a6
Replace demandload with new demandimport
Matt Mackall <mpm@selenic.com>
parents:
diff
changeset
|
26 |
25943
3beed01daff9
demandimport: use absolute_import
Gregory Szorc <gregory.szorc@gmail.com>
parents:
25937
diff
changeset
|
27 from __future__ import absolute_import |
3beed01daff9
demandimport: use absolute_import
Gregory Szorc <gregory.szorc@gmail.com>
parents:
25937
diff
changeset
|
28 |
3beed01daff9
demandimport: use absolute_import
Gregory Szorc <gregory.szorc@gmail.com>
parents:
25937
diff
changeset
|
29 import contextlib |
3beed01daff9
demandimport: use absolute_import
Gregory Szorc <gregory.szorc@gmail.com>
parents:
25937
diff
changeset
|
30 import os |
3beed01daff9
demandimport: use absolute_import
Gregory Szorc <gregory.szorc@gmail.com>
parents:
25937
diff
changeset
|
31 import sys |
25674
5d0847cd1587
demandimport: support importing builtins for Python 3
Gregory Szorc <gregory.szorc@gmail.com>
parents:
25673
diff
changeset
|
32 |
5d0847cd1587
demandimport: support importing builtins for Python 3
Gregory Szorc <gregory.szorc@gmail.com>
parents:
25673
diff
changeset
|
33 # __builtin__ in Python 2, builtins in Python 3. |
5d0847cd1587
demandimport: support importing builtins for Python 3
Gregory Szorc <gregory.szorc@gmail.com>
parents:
25673
diff
changeset
|
34 try: |
5d0847cd1587
demandimport: support importing builtins for Python 3
Gregory Szorc <gregory.szorc@gmail.com>
parents:
25673
diff
changeset
|
35 import __builtin__ as builtins |
5d0847cd1587
demandimport: support importing builtins for Python 3
Gregory Szorc <gregory.szorc@gmail.com>
parents:
25673
diff
changeset
|
36 except ImportError: |
5d0847cd1587
demandimport: support importing builtins for Python 3
Gregory Szorc <gregory.szorc@gmail.com>
parents:
25673
diff
changeset
|
37 import builtins |
25327
2e7804110b14
demandimport: define a `deactivated` context manager
Jordi Guti?rrez Hermoso <jordigh@octave.org>
parents:
23643
diff
changeset
|
38 |
25943
3beed01daff9
demandimport: use absolute_import
Gregory Szorc <gregory.szorc@gmail.com>
parents:
25937
diff
changeset
|
39 contextmanager = contextlib.contextmanager |
3beed01daff9
demandimport: use absolute_import
Gregory Szorc <gregory.szorc@gmail.com>
parents:
25937
diff
changeset
|
40 |
3877
abaee83ce0a6
Replace demandload with new demandimport
Matt Mackall <mpm@selenic.com>
parents:
diff
changeset
|
41 _origimport = __import__ |
abaee83ce0a6
Replace demandload with new demandimport
Matt Mackall <mpm@selenic.com>
parents:
diff
changeset
|
42 |
14977
1dbd42a02153
demandimport: use getattr instead of hasattr
Augie Fackler <durin42@gmail.com>
parents:
14976
diff
changeset
|
43 nothing = object() |
1dbd42a02153
demandimport: use getattr instead of hasattr
Augie Fackler <durin42@gmail.com>
parents:
14976
diff
changeset
|
44 |
25933
1fc6c02782ab
demandimport: remove support for Python < 2.5
Gregory Szorc <gregory.szorc@gmail.com>
parents:
25674
diff
changeset
|
45 # Python 3 doesn't have relative imports nor level -1. |
1fc6c02782ab
demandimport: remove support for Python < 2.5
Gregory Szorc <gregory.szorc@gmail.com>
parents:
25674
diff
changeset
|
46 level = -1 |
1fc6c02782ab
demandimport: remove support for Python < 2.5
Gregory Szorc <gregory.szorc@gmail.com>
parents:
25674
diff
changeset
|
47 if sys.version_info[0] >= 3: |
1fc6c02782ab
demandimport: remove support for Python < 2.5
Gregory Szorc <gregory.szorc@gmail.com>
parents:
25674
diff
changeset
|
48 level = 0 |
1fc6c02782ab
demandimport: remove support for Python < 2.5
Gregory Szorc <gregory.szorc@gmail.com>
parents:
25674
diff
changeset
|
49 _import = _origimport |
15096
868282fa29d8
demandimport: determine at load time if __import__ has level argument
Simon Heimberg <simohe@besonet.ch>
parents:
14977
diff
changeset
|
50 |
25936
f90bb2002bcf
demandimport: support keyword arguments on _hgextimport
Gregory Szorc <gregory.szorc@gmail.com>
parents:
25935
diff
changeset
|
51 def _hgextimport(importfunc, name, globals, *args, **kwargs): |
19933
621a26eb3a99
demandimport: allow extensions to import own modules by absolute name
FUJIWARA Katsunori <foozy@lares.dti.ne.jp>
parents:
19932
diff
changeset
|
52 try: |
25936
f90bb2002bcf
demandimport: support keyword arguments on _hgextimport
Gregory Szorc <gregory.szorc@gmail.com>
parents:
25935
diff
changeset
|
53 return importfunc(name, globals, *args, **kwargs) |
19933
621a26eb3a99
demandimport: allow extensions to import own modules by absolute name
FUJIWARA Katsunori <foozy@lares.dti.ne.jp>
parents:
19932
diff
changeset
|
54 except ImportError: |
621a26eb3a99
demandimport: allow extensions to import own modules by absolute name
FUJIWARA Katsunori <foozy@lares.dti.ne.jp>
parents:
19932
diff
changeset
|
55 if not globals: |
621a26eb3a99
demandimport: allow extensions to import own modules by absolute name
FUJIWARA Katsunori <foozy@lares.dti.ne.jp>
parents:
19932
diff
changeset
|
56 raise |
621a26eb3a99
demandimport: allow extensions to import own modules by absolute name
FUJIWARA Katsunori <foozy@lares.dti.ne.jp>
parents:
19932
diff
changeset
|
57 # extensions are loaded with "hgext_" prefix |
621a26eb3a99
demandimport: allow extensions to import own modules by absolute name
FUJIWARA Katsunori <foozy@lares.dti.ne.jp>
parents:
19932
diff
changeset
|
58 hgextname = 'hgext_%s' % name |
621a26eb3a99
demandimport: allow extensions to import own modules by absolute name
FUJIWARA Katsunori <foozy@lares.dti.ne.jp>
parents:
19932
diff
changeset
|
59 nameroot = hgextname.split('.', 1)[0] |
621a26eb3a99
demandimport: allow extensions to import own modules by absolute name
FUJIWARA Katsunori <foozy@lares.dti.ne.jp>
parents:
19932
diff
changeset
|
60 contextroot = globals.get('__name__', '').split('.', 1)[0] |
621a26eb3a99
demandimport: allow extensions to import own modules by absolute name
FUJIWARA Katsunori <foozy@lares.dti.ne.jp>
parents:
19932
diff
changeset
|
61 if nameroot != contextroot: |
621a26eb3a99
demandimport: allow extensions to import own modules by absolute name
FUJIWARA Katsunori <foozy@lares.dti.ne.jp>
parents:
19932
diff
changeset
|
62 raise |
621a26eb3a99
demandimport: allow extensions to import own modules by absolute name
FUJIWARA Katsunori <foozy@lares.dti.ne.jp>
parents:
19932
diff
changeset
|
63 # retry to import with "hgext_" prefix |
25936
f90bb2002bcf
demandimport: support keyword arguments on _hgextimport
Gregory Szorc <gregory.szorc@gmail.com>
parents:
25935
diff
changeset
|
64 return importfunc(hgextname, globals, *args, **kwargs) |
19933
621a26eb3a99
demandimport: allow extensions to import own modules by absolute name
FUJIWARA Katsunori <foozy@lares.dti.ne.jp>
parents:
19932
diff
changeset
|
65 |
3877
abaee83ce0a6
Replace demandload with new demandimport
Matt Mackall <mpm@selenic.com>
parents:
diff
changeset
|
66 class _demandmod(object): |
abaee83ce0a6
Replace demandload with new demandimport
Matt Mackall <mpm@selenic.com>
parents:
diff
changeset
|
67 """module demand-loader and proxy""" |
21290
74be3fb1e3b8
demandimport: pass proper level to __import__ in Python 3
Gregory Szorc <gregory.szorc@gmail.com>
parents:
21025
diff
changeset
|
68 def __init__(self, name, globals, locals, level=level): |
3877
abaee83ce0a6
Replace demandload with new demandimport
Matt Mackall <mpm@selenic.com>
parents:
diff
changeset
|
69 if '.' in name: |
abaee83ce0a6
Replace demandload with new demandimport
Matt Mackall <mpm@selenic.com>
parents:
diff
changeset
|
70 head, rest = name.split('.', 1) |
abaee83ce0a6
Replace demandload with new demandimport
Matt Mackall <mpm@selenic.com>
parents:
diff
changeset
|
71 after = [rest] |
abaee83ce0a6
Replace demandload with new demandimport
Matt Mackall <mpm@selenic.com>
parents:
diff
changeset
|
72 else: |
abaee83ce0a6
Replace demandload with new demandimport
Matt Mackall <mpm@selenic.com>
parents:
diff
changeset
|
73 head = name |
abaee83ce0a6
Replace demandload with new demandimport
Matt Mackall <mpm@selenic.com>
parents:
diff
changeset
|
74 after = [] |
19932
e3a5922e18c3
demandimport: support "absolute_import" for external libraries (issue4029)
FUJIWARA Katsunori <foozy@lares.dti.ne.jp>
parents:
15096
diff
changeset
|
75 object.__setattr__(self, "_data", |
e3a5922e18c3
demandimport: support "absolute_import" for external libraries (issue4029)
FUJIWARA Katsunori <foozy@lares.dti.ne.jp>
parents:
15096
diff
changeset
|
76 (head, globals, locals, after, level)) |
3896
3b628b5da9e9
use parent.__setattr__ instead of __dict__
Benoit Boissinot <benoit.boissinot@ens-lyon.org>
parents:
3877
diff
changeset
|
77 object.__setattr__(self, "_module", None) |
3877
abaee83ce0a6
Replace demandload with new demandimport
Matt Mackall <mpm@selenic.com>
parents:
diff
changeset
|
78 def _extend(self, name): |
abaee83ce0a6
Replace demandload with new demandimport
Matt Mackall <mpm@selenic.com>
parents:
diff
changeset
|
79 """add to the list of submodules to load""" |
abaee83ce0a6
Replace demandload with new demandimport
Matt Mackall <mpm@selenic.com>
parents:
diff
changeset
|
80 self._data[3].append(name) |
abaee83ce0a6
Replace demandload with new demandimport
Matt Mackall <mpm@selenic.com>
parents:
diff
changeset
|
81 def _load(self): |
abaee83ce0a6
Replace demandload with new demandimport
Matt Mackall <mpm@selenic.com>
parents:
diff
changeset
|
82 if not self._module: |
19932
e3a5922e18c3
demandimport: support "absolute_import" for external libraries (issue4029)
FUJIWARA Katsunori <foozy@lares.dti.ne.jp>
parents:
15096
diff
changeset
|
83 head, globals, locals, after, level = self._data |
19933
621a26eb3a99
demandimport: allow extensions to import own modules by absolute name
FUJIWARA Katsunori <foozy@lares.dti.ne.jp>
parents:
19932
diff
changeset
|
84 mod = _hgextimport(_import, head, globals, locals, None, level) |
3877
abaee83ce0a6
Replace demandload with new demandimport
Matt Mackall <mpm@selenic.com>
parents:
diff
changeset
|
85 # load submodules |
3921
6d0d025e125a
demandimport: fix import x.y.z as a when x.y is already imported.
Matt Mackall <mpm@selenic.com>
parents:
3903
diff
changeset
|
86 def subload(mod, p): |
6d0d025e125a
demandimport: fix import x.y.z as a when x.y is already imported.
Matt Mackall <mpm@selenic.com>
parents:
3903
diff
changeset
|
87 h, t = p, None |
6d0d025e125a
demandimport: fix import x.y.z as a when x.y is already imported.
Matt Mackall <mpm@selenic.com>
parents:
3903
diff
changeset
|
88 if '.' in p: |
6d0d025e125a
demandimport: fix import x.y.z as a when x.y is already imported.
Matt Mackall <mpm@selenic.com>
parents:
3903
diff
changeset
|
89 h, t = p.split('.', 1) |
14977
1dbd42a02153
demandimport: use getattr instead of hasattr
Augie Fackler <durin42@gmail.com>
parents:
14976
diff
changeset
|
90 if getattr(mod, h, nothing) is nothing: |
12894
bc91a79fa3d0
demandimport: back out 50a4e55aa278 (issue2467)
Matt Mackall <mpm@selenic.com>
parents:
12801
diff
changeset
|
91 setattr(mod, h, _demandmod(p, mod.__dict__, mod.__dict__)) |
3926
de6ae8f016af
demandimport: handle already-loaded nested modules in subload
Brendan Cully <brendan@kublai.com>
parents:
3921
diff
changeset
|
92 elif t: |
3921
6d0d025e125a
demandimport: fix import x.y.z as a when x.y is already imported.
Matt Mackall <mpm@selenic.com>
parents:
3903
diff
changeset
|
93 subload(getattr(mod, h), t) |
6d0d025e125a
demandimport: fix import x.y.z as a when x.y is already imported.
Matt Mackall <mpm@selenic.com>
parents:
3903
diff
changeset
|
94 |
3877
abaee83ce0a6
Replace demandload with new demandimport
Matt Mackall <mpm@selenic.com>
parents:
diff
changeset
|
95 for x in after: |
3921
6d0d025e125a
demandimport: fix import x.y.z as a when x.y is already imported.
Matt Mackall <mpm@selenic.com>
parents:
3903
diff
changeset
|
96 subload(mod, x) |
6d0d025e125a
demandimport: fix import x.y.z as a when x.y is already imported.
Matt Mackall <mpm@selenic.com>
parents:
3903
diff
changeset
|
97 |
3877
abaee83ce0a6
Replace demandload with new demandimport
Matt Mackall <mpm@selenic.com>
parents:
diff
changeset
|
98 # are we in the locals dictionary still? |
abaee83ce0a6
Replace demandload with new demandimport
Matt Mackall <mpm@selenic.com>
parents:
diff
changeset
|
99 if locals and locals.get(head) == self: |
abaee83ce0a6
Replace demandload with new demandimport
Matt Mackall <mpm@selenic.com>
parents:
diff
changeset
|
100 locals[head] = mod |
3896
3b628b5da9e9
use parent.__setattr__ instead of __dict__
Benoit Boissinot <benoit.boissinot@ens-lyon.org>
parents:
3877
diff
changeset
|
101 object.__setattr__(self, "_module", mod) |
4631
e3afa670e484
demandimport: fix issue579 and add a test
Matt Mackall <mpm@selenic.com>
parents:
4626
diff
changeset
|
102 |
3877
abaee83ce0a6
Replace demandload with new demandimport
Matt Mackall <mpm@selenic.com>
parents:
diff
changeset
|
103 def __repr__(self): |
4631
e3afa670e484
demandimport: fix issue579 and add a test
Matt Mackall <mpm@selenic.com>
parents:
4626
diff
changeset
|
104 if self._module: |
e3afa670e484
demandimport: fix issue579 and add a test
Matt Mackall <mpm@selenic.com>
parents:
4626
diff
changeset
|
105 return "<proxied module '%s'>" % self._data[0] |
3877
abaee83ce0a6
Replace demandload with new demandimport
Matt Mackall <mpm@selenic.com>
parents:
diff
changeset
|
106 return "<unloaded module '%s'>" % self._data[0] |
abaee83ce0a6
Replace demandload with new demandimport
Matt Mackall <mpm@selenic.com>
parents:
diff
changeset
|
107 def __call__(self, *args, **kwargs): |
5639
7dd5cf9d1e09
demandload: give better diagnostic for call of an unloaded module
Matt Mackall <mpm@selenic.com>
parents:
5098
diff
changeset
|
108 raise TypeError("%s object is not callable" % repr(self)) |
3903
f9136599700f
Make demandimport pass all tests on python2.5.
Brendan Cully <brendan@kublai.com>
parents:
3898
diff
changeset
|
109 def __getattribute__(self, attr): |
f9136599700f
Make demandimport pass all tests on python2.5.
Brendan Cully <brendan@kublai.com>
parents:
3898
diff
changeset
|
110 if attr in ('_data', '_extend', '_load', '_module'): |
f9136599700f
Make demandimport pass all tests on python2.5.
Brendan Cully <brendan@kublai.com>
parents:
3898
diff
changeset
|
111 return object.__getattribute__(self, attr) |
3877
abaee83ce0a6
Replace demandload with new demandimport
Matt Mackall <mpm@selenic.com>
parents:
diff
changeset
|
112 self._load() |
abaee83ce0a6
Replace demandload with new demandimport
Matt Mackall <mpm@selenic.com>
parents:
diff
changeset
|
113 return getattr(self._module, attr) |
abaee83ce0a6
Replace demandload with new demandimport
Matt Mackall <mpm@selenic.com>
parents:
diff
changeset
|
114 def __setattr__(self, attr, val): |
abaee83ce0a6
Replace demandload with new demandimport
Matt Mackall <mpm@selenic.com>
parents:
diff
changeset
|
115 self._load() |
abaee83ce0a6
Replace demandload with new demandimport
Matt Mackall <mpm@selenic.com>
parents:
diff
changeset
|
116 setattr(self._module, attr, val) |
abaee83ce0a6
Replace demandload with new demandimport
Matt Mackall <mpm@selenic.com>
parents:
diff
changeset
|
117 |
21290
74be3fb1e3b8
demandimport: pass proper level to __import__ in Python 3
Gregory Szorc <gregory.szorc@gmail.com>
parents:
21025
diff
changeset
|
118 def _demandimport(name, globals=None, locals=None, fromlist=None, level=level): |
3877
abaee83ce0a6
Replace demandload with new demandimport
Matt Mackall <mpm@selenic.com>
parents:
diff
changeset
|
119 if not locals or name in ignore or fromlist == ('*',): |
abaee83ce0a6
Replace demandload with new demandimport
Matt Mackall <mpm@selenic.com>
parents:
diff
changeset
|
120 # these cases we can't really delay |
19933
621a26eb3a99
demandimport: allow extensions to import own modules by absolute name
FUJIWARA Katsunori <foozy@lares.dti.ne.jp>
parents:
19932
diff
changeset
|
121 return _hgextimport(_import, name, globals, locals, fromlist, level) |
3877
abaee83ce0a6
Replace demandload with new demandimport
Matt Mackall <mpm@selenic.com>
parents:
diff
changeset
|
122 elif not fromlist: |
abaee83ce0a6
Replace demandload with new demandimport
Matt Mackall <mpm@selenic.com>
parents:
diff
changeset
|
123 # import a [as b] |
abaee83ce0a6
Replace demandload with new demandimport
Matt Mackall <mpm@selenic.com>
parents:
diff
changeset
|
124 if '.' in name: # a.b |
abaee83ce0a6
Replace demandload with new demandimport
Matt Mackall <mpm@selenic.com>
parents:
diff
changeset
|
125 base, rest = name.split('.', 1) |
3903
f9136599700f
Make demandimport pass all tests on python2.5.
Brendan Cully <brendan@kublai.com>
parents:
3898
diff
changeset
|
126 # email.__init__ loading email.mime |
f9136599700f
Make demandimport pass all tests on python2.5.
Brendan Cully <brendan@kublai.com>
parents:
3898
diff
changeset
|
127 if globals and globals.get('__name__', None) == base: |
15096
868282fa29d8
demandimport: determine at load time if __import__ has level argument
Simon Heimberg <simohe@besonet.ch>
parents:
14977
diff
changeset
|
128 return _import(name, globals, locals, fromlist, level) |
3877
abaee83ce0a6
Replace demandload with new demandimport
Matt Mackall <mpm@selenic.com>
parents:
diff
changeset
|
129 # if a is already demand-loaded, add b to its submodule list |
abaee83ce0a6
Replace demandload with new demandimport
Matt Mackall <mpm@selenic.com>
parents:
diff
changeset
|
130 if base in locals: |
abaee83ce0a6
Replace demandload with new demandimport
Matt Mackall <mpm@selenic.com>
parents:
diff
changeset
|
131 if isinstance(locals[base], _demandmod): |
abaee83ce0a6
Replace demandload with new demandimport
Matt Mackall <mpm@selenic.com>
parents:
diff
changeset
|
132 locals[base]._extend(rest) |
abaee83ce0a6
Replace demandload with new demandimport
Matt Mackall <mpm@selenic.com>
parents:
diff
changeset
|
133 return locals[base] |
19932
e3a5922e18c3
demandimport: support "absolute_import" for external libraries (issue4029)
FUJIWARA Katsunori <foozy@lares.dti.ne.jp>
parents:
15096
diff
changeset
|
134 return _demandmod(name, globals, locals, level) |
3877
abaee83ce0a6
Replace demandload with new demandimport
Matt Mackall <mpm@selenic.com>
parents:
diff
changeset
|
135 else: |
25935
49dd4fd3f283
demandimport: refactor logic and add documentation
Gregory Szorc <gregory.szorc@gmail.com>
parents:
25934
diff
changeset
|
136 # There is a fromlist. |
49dd4fd3f283
demandimport: refactor logic and add documentation
Gregory Szorc <gregory.szorc@gmail.com>
parents:
25934
diff
changeset
|
137 # from a import b,c,d |
49dd4fd3f283
demandimport: refactor logic and add documentation
Gregory Szorc <gregory.szorc@gmail.com>
parents:
25934
diff
changeset
|
138 # from . import b,c,d |
49dd4fd3f283
demandimport: refactor logic and add documentation
Gregory Szorc <gregory.szorc@gmail.com>
parents:
25934
diff
changeset
|
139 # from .a import b,c,d |
49dd4fd3f283
demandimport: refactor logic and add documentation
Gregory Szorc <gregory.szorc@gmail.com>
parents:
25934
diff
changeset
|
140 |
49dd4fd3f283
demandimport: refactor logic and add documentation
Gregory Szorc <gregory.szorc@gmail.com>
parents:
25934
diff
changeset
|
141 # level == -1: relative and absolute attempted (Python 2 only). |
49dd4fd3f283
demandimport: refactor logic and add documentation
Gregory Szorc <gregory.szorc@gmail.com>
parents:
25934
diff
changeset
|
142 # level >= 0: absolute only (Python 2 w/ absolute_import and Python 3). |
49dd4fd3f283
demandimport: refactor logic and add documentation
Gregory Szorc <gregory.szorc@gmail.com>
parents:
25934
diff
changeset
|
143 # The modern Mercurial convention is to use absolute_import everywhere, |
49dd4fd3f283
demandimport: refactor logic and add documentation
Gregory Szorc <gregory.szorc@gmail.com>
parents:
25934
diff
changeset
|
144 # so modern Mercurial code will have level >= 0. |
49dd4fd3f283
demandimport: refactor logic and add documentation
Gregory Szorc <gregory.szorc@gmail.com>
parents:
25934
diff
changeset
|
145 |
49dd4fd3f283
demandimport: refactor logic and add documentation
Gregory Szorc <gregory.szorc@gmail.com>
parents:
25934
diff
changeset
|
146 if level >= 0: |
25937
4f1144c3c72b
demandimport: support lazy loading for absolute_import
Gregory Szorc <gregory.szorc@gmail.com>
parents:
25936
diff
changeset
|
147 # Mercurial's enforced import style does not use |
4f1144c3c72b
demandimport: support lazy loading for absolute_import
Gregory Szorc <gregory.szorc@gmail.com>
parents:
25936
diff
changeset
|
148 # "from a import b,c,d" or "from .a import b,c,d" syntax. In |
4f1144c3c72b
demandimport: support lazy loading for absolute_import
Gregory Szorc <gregory.szorc@gmail.com>
parents:
25936
diff
changeset
|
149 # addition, this appears to be giving errors with some modules |
4f1144c3c72b
demandimport: support lazy loading for absolute_import
Gregory Szorc <gregory.szorc@gmail.com>
parents:
25936
diff
changeset
|
150 # for unknown reasons. Since we shouldn't be using this syntax |
4f1144c3c72b
demandimport: support lazy loading for absolute_import
Gregory Szorc <gregory.szorc@gmail.com>
parents:
25936
diff
changeset
|
151 # much, work around the problems. |
4f1144c3c72b
demandimport: support lazy loading for absolute_import
Gregory Szorc <gregory.szorc@gmail.com>
parents:
25936
diff
changeset
|
152 if name: |
4f1144c3c72b
demandimport: support lazy loading for absolute_import
Gregory Szorc <gregory.szorc@gmail.com>
parents:
25936
diff
changeset
|
153 return _hgextimport(_origimport, name, globals, locals, |
4f1144c3c72b
demandimport: support lazy loading for absolute_import
Gregory Szorc <gregory.szorc@gmail.com>
parents:
25936
diff
changeset
|
154 fromlist, level) |
4f1144c3c72b
demandimport: support lazy loading for absolute_import
Gregory Szorc <gregory.szorc@gmail.com>
parents:
25936
diff
changeset
|
155 |
4f1144c3c72b
demandimport: support lazy loading for absolute_import
Gregory Szorc <gregory.szorc@gmail.com>
parents:
25936
diff
changeset
|
156 mod = _hgextimport(_origimport, name, globals, locals, level=level) |
4f1144c3c72b
demandimport: support lazy loading for absolute_import
Gregory Szorc <gregory.szorc@gmail.com>
parents:
25936
diff
changeset
|
157 for x in fromlist: |
4f1144c3c72b
demandimport: support lazy loading for absolute_import
Gregory Szorc <gregory.szorc@gmail.com>
parents:
25936
diff
changeset
|
158 # Missing symbols mean they weren't defined in the module |
4f1144c3c72b
demandimport: support lazy loading for absolute_import
Gregory Szorc <gregory.szorc@gmail.com>
parents:
25936
diff
changeset
|
159 # itself which means they are sub-modules. |
4f1144c3c72b
demandimport: support lazy loading for absolute_import
Gregory Szorc <gregory.szorc@gmail.com>
parents:
25936
diff
changeset
|
160 if getattr(mod, x, nothing) is nothing: |
4f1144c3c72b
demandimport: support lazy loading for absolute_import
Gregory Szorc <gregory.szorc@gmail.com>
parents:
25936
diff
changeset
|
161 setattr(mod, x, |
4f1144c3c72b
demandimport: support lazy loading for absolute_import
Gregory Szorc <gregory.szorc@gmail.com>
parents:
25936
diff
changeset
|
162 _demandmod(x, mod.__dict__, locals, level=level)) |
4f1144c3c72b
demandimport: support lazy loading for absolute_import
Gregory Szorc <gregory.szorc@gmail.com>
parents:
25936
diff
changeset
|
163 |
4f1144c3c72b
demandimport: support lazy loading for absolute_import
Gregory Szorc <gregory.szorc@gmail.com>
parents:
25936
diff
changeset
|
164 return mod |
25935
49dd4fd3f283
demandimport: refactor logic and add documentation
Gregory Szorc <gregory.szorc@gmail.com>
parents:
25934
diff
changeset
|
165 |
49dd4fd3f283
demandimport: refactor logic and add documentation
Gregory Szorc <gregory.szorc@gmail.com>
parents:
25934
diff
changeset
|
166 # But, we still need to support lazy loading of standard library and 3rd |
49dd4fd3f283
demandimport: refactor logic and add documentation
Gregory Szorc <gregory.szorc@gmail.com>
parents:
25934
diff
changeset
|
167 # party modules. So handle level == -1. |
19933
621a26eb3a99
demandimport: allow extensions to import own modules by absolute name
FUJIWARA Katsunori <foozy@lares.dti.ne.jp>
parents:
19932
diff
changeset
|
168 mod = _hgextimport(_origimport, name, globals, locals) |
3877
abaee83ce0a6
Replace demandload with new demandimport
Matt Mackall <mpm@selenic.com>
parents:
diff
changeset
|
169 # recurse down the module chain |
abaee83ce0a6
Replace demandload with new demandimport
Matt Mackall <mpm@selenic.com>
parents:
diff
changeset
|
170 for comp in name.split('.')[1:]: |
14977
1dbd42a02153
demandimport: use getattr instead of hasattr
Augie Fackler <durin42@gmail.com>
parents:
14976
diff
changeset
|
171 if getattr(mod, comp, nothing) is nothing: |
25935
49dd4fd3f283
demandimport: refactor logic and add documentation
Gregory Szorc <gregory.szorc@gmail.com>
parents:
25934
diff
changeset
|
172 setattr(mod, comp, |
49dd4fd3f283
demandimport: refactor logic and add documentation
Gregory Szorc <gregory.szorc@gmail.com>
parents:
25934
diff
changeset
|
173 _demandmod(comp, mod.__dict__, mod.__dict__)) |
3877
abaee83ce0a6
Replace demandload with new demandimport
Matt Mackall <mpm@selenic.com>
parents:
diff
changeset
|
174 mod = getattr(mod, comp) |
abaee83ce0a6
Replace demandload with new demandimport
Matt Mackall <mpm@selenic.com>
parents:
diff
changeset
|
175 for x in fromlist: |
abaee83ce0a6
Replace demandload with new demandimport
Matt Mackall <mpm@selenic.com>
parents:
diff
changeset
|
176 # set requested submodules for demand load |
14977
1dbd42a02153
demandimport: use getattr instead of hasattr
Augie Fackler <durin42@gmail.com>
parents:
14976
diff
changeset
|
177 if getattr(mod, x, nothing) is nothing: |
12894
bc91a79fa3d0
demandimport: back out 50a4e55aa278 (issue2467)
Matt Mackall <mpm@selenic.com>
parents:
12801
diff
changeset
|
178 setattr(mod, x, _demandmod(x, mod.__dict__, locals)) |
3877
abaee83ce0a6
Replace demandload with new demandimport
Matt Mackall <mpm@selenic.com>
parents:
diff
changeset
|
179 return mod |
abaee83ce0a6
Replace demandload with new demandimport
Matt Mackall <mpm@selenic.com>
parents:
diff
changeset
|
180 |
5098
0bbd86b847dd
demandimport: ignore resource module, not available under Windows.
Patrick Mezard <pmezard@gmail.com>
parents:
5097
diff
changeset
|
181 ignore = [ |
25934
e283c5d922db
demandimport: add __future__ to ignore list
Gregory Szorc <gregory.szorc@gmail.com>
parents:
25933
diff
changeset
|
182 '__future__', |
5098
0bbd86b847dd
demandimport: ignore resource module, not available under Windows.
Patrick Mezard <pmezard@gmail.com>
parents:
5097
diff
changeset
|
183 '_hashlib', |
0bbd86b847dd
demandimport: ignore resource module, not available under Windows.
Patrick Mezard <pmezard@gmail.com>
parents:
5097
diff
changeset
|
184 '_xmlplus', |
0bbd86b847dd
demandimport: ignore resource module, not available under Windows.
Patrick Mezard <pmezard@gmail.com>
parents:
5097
diff
changeset
|
185 'fcntl', |
0bbd86b847dd
demandimport: ignore resource module, not available under Windows.
Patrick Mezard <pmezard@gmail.com>
parents:
5097
diff
changeset
|
186 'win32com.gen_py', |
10242
ecd0a5c8bbe5
demandimport: ignore _winreg (used in python-2.7 mimetypes)
Dirkjan Ochtman <dirkjan@ochtman.nl>
parents:
9458
diff
changeset
|
187 '_winreg', # 2.7 mimetypes needs immediate ImportError |
7861
2bc14da14992
demandimport: blacklist pythoncom
Steve Borho <steve@borho.org>
parents:
7727
diff
changeset
|
188 'pythoncom', |
5098
0bbd86b847dd
demandimport: ignore resource module, not available under Windows.
Patrick Mezard <pmezard@gmail.com>
parents:
5097
diff
changeset
|
189 # imported by tarfile, not available under Windows |
0bbd86b847dd
demandimport: ignore resource module, not available under Windows.
Patrick Mezard <pmezard@gmail.com>
parents:
5097
diff
changeset
|
190 'pwd', |
0bbd86b847dd
demandimport: ignore resource module, not available under Windows.
Patrick Mezard <pmezard@gmail.com>
parents:
5097
diff
changeset
|
191 'grp', |
0bbd86b847dd
demandimport: ignore resource module, not available under Windows.
Patrick Mezard <pmezard@gmail.com>
parents:
5097
diff
changeset
|
192 # imported by profile, itself imported by hotshot.stats, |
0bbd86b847dd
demandimport: ignore resource module, not available under Windows.
Patrick Mezard <pmezard@gmail.com>
parents:
5097
diff
changeset
|
193 # not available under Windows |
0bbd86b847dd
demandimport: ignore resource module, not available under Windows.
Patrick Mezard <pmezard@gmail.com>
parents:
5097
diff
changeset
|
194 'resource', |
9458 | 195 # this trips up many extension authors |
196 'gtk', | |
10598
1037bd445768
demandimport: add __main__ to the blacklist (because of setuptools)
Greg Ward <greg-hg@gerg.ca>
parents:
10263
diff
changeset
|
197 # setuptools' pkg_resources.py expects "from __main__ import x" to |
1037bd445768
demandimport: add __main__ to the blacklist (because of setuptools)
Greg Ward <greg-hg@gerg.ca>
parents:
10263
diff
changeset
|
198 # raise ImportError if x not defined |
1037bd445768
demandimport: add __main__ to the blacklist (because of setuptools)
Greg Ward <greg-hg@gerg.ca>
parents:
10263
diff
changeset
|
199 '__main__', |
10612
30553ac3e355
demandimport: blacklist _ssl (issue1964)
Dirkjan Ochtman <dirkjan@ochtman.nl>
parents:
10598
diff
changeset
|
200 '_ssl', # conditional imports in the stdlib, issue1964 |
14976
04a950b1c2ad
demandimport: blacklist rfc822 and mimetools to prevent spurious warnings
Augie Fackler <durin42@gmail.com>
parents:
13082
diff
changeset
|
201 'rfc822', |
04a950b1c2ad
demandimport: blacklist rfc822 and mimetools to prevent spurious warnings
Augie Fackler <durin42@gmail.com>
parents:
13082
diff
changeset
|
202 'mimetools', |
23643
2205d00b6d2b
demandimport: blacklist distutils.msvc9compiler (issue4475)
Augie Fackler <raf@durin42.com>
parents:
21290
diff
changeset
|
203 # setuptools 8 expects this module to explode early when not on windows |
2205d00b6d2b
demandimport: blacklist distutils.msvc9compiler (issue4475)
Augie Fackler <raf@durin42.com>
parents:
21290
diff
changeset
|
204 'distutils.msvc9compiler' |
5098
0bbd86b847dd
demandimport: ignore resource module, not available under Windows.
Patrick Mezard <pmezard@gmail.com>
parents:
5097
diff
changeset
|
205 ] |
3877
abaee83ce0a6
Replace demandload with new demandimport
Matt Mackall <mpm@selenic.com>
parents:
diff
changeset
|
206 |
20422
aac87f70f38e
hooks: only disable/re-enable demandimport when it's already enabled
Brodie Rao <brodie@sf.io>
parents:
19933
diff
changeset
|
207 def isenabled(): |
25673
fa1f04529775
demandimport: alias __builtin__ as builtins
Gregory Szorc <gregory.szorc@gmail.com>
parents:
25327
diff
changeset
|
208 return builtins.__import__ == _demandimport |
20422
aac87f70f38e
hooks: only disable/re-enable demandimport when it's already enabled
Brodie Rao <brodie@sf.io>
parents:
19933
diff
changeset
|
209 |
3877
abaee83ce0a6
Replace demandload with new demandimport
Matt Mackall <mpm@selenic.com>
parents:
diff
changeset
|
210 def enable(): |
abaee83ce0a6
Replace demandload with new demandimport
Matt Mackall <mpm@selenic.com>
parents:
diff
changeset
|
211 "enable global demand-loading of modules" |
21025
54af51c18c4c
demandimport: make it possible to disable by setting HGDEMANDIMPORT=disable
Mads Kiilerich <madski@unity3d.com>
parents:
20422
diff
changeset
|
212 if os.environ.get('HGDEMANDIMPORT') != 'disable': |
25673
fa1f04529775
demandimport: alias __builtin__ as builtins
Gregory Szorc <gregory.szorc@gmail.com>
parents:
25327
diff
changeset
|
213 builtins.__import__ = _demandimport |
3877
abaee83ce0a6
Replace demandload with new demandimport
Matt Mackall <mpm@selenic.com>
parents:
diff
changeset
|
214 |
abaee83ce0a6
Replace demandload with new demandimport
Matt Mackall <mpm@selenic.com>
parents:
diff
changeset
|
215 def disable(): |
abaee83ce0a6
Replace demandload with new demandimport
Matt Mackall <mpm@selenic.com>
parents:
diff
changeset
|
216 "disable global demand-loading of modules" |
25673
fa1f04529775
demandimport: alias __builtin__ as builtins
Gregory Szorc <gregory.szorc@gmail.com>
parents:
25327
diff
changeset
|
217 builtins.__import__ = _origimport |
25327
2e7804110b14
demandimport: define a `deactivated` context manager
Jordi Guti?rrez Hermoso <jordigh@octave.org>
parents:
23643
diff
changeset
|
218 |
2e7804110b14
demandimport: define a `deactivated` context manager
Jordi Guti?rrez Hermoso <jordigh@octave.org>
parents:
23643
diff
changeset
|
219 @contextmanager |
2e7804110b14
demandimport: define a `deactivated` context manager
Jordi Guti?rrez Hermoso <jordigh@octave.org>
parents:
23643
diff
changeset
|
220 def deactivated(): |
2e7804110b14
demandimport: define a `deactivated` context manager
Jordi Guti?rrez Hermoso <jordigh@octave.org>
parents:
23643
diff
changeset
|
221 "context manager for disabling demandimport in 'with' blocks" |
2e7804110b14
demandimport: define a `deactivated` context manager
Jordi Guti?rrez Hermoso <jordigh@octave.org>
parents:
23643
diff
changeset
|
222 demandenabled = isenabled() |
2e7804110b14
demandimport: define a `deactivated` context manager
Jordi Guti?rrez Hermoso <jordigh@octave.org>
parents:
23643
diff
changeset
|
223 if demandenabled: |
2e7804110b14
demandimport: define a `deactivated` context manager
Jordi Guti?rrez Hermoso <jordigh@octave.org>
parents:
23643
diff
changeset
|
224 disable() |
2e7804110b14
demandimport: define a `deactivated` context manager
Jordi Guti?rrez Hermoso <jordigh@octave.org>
parents:
23643
diff
changeset
|
225 |
2e7804110b14
demandimport: define a `deactivated` context manager
Jordi Guti?rrez Hermoso <jordigh@octave.org>
parents:
23643
diff
changeset
|
226 try: |
2e7804110b14
demandimport: define a `deactivated` context manager
Jordi Guti?rrez Hermoso <jordigh@octave.org>
parents:
23643
diff
changeset
|
227 yield |
2e7804110b14
demandimport: define a `deactivated` context manager
Jordi Guti?rrez Hermoso <jordigh@octave.org>
parents:
23643
diff
changeset
|
228 finally: |
2e7804110b14
demandimport: define a `deactivated` context manager
Jordi Guti?rrez Hermoso <jordigh@octave.org>
parents:
23643
diff
changeset
|
229 if demandenabled: |
2e7804110b14
demandimport: define a `deactivated` context manager
Jordi Guti?rrez Hermoso <jordigh@octave.org>
parents:
23643
diff
changeset
|
230 enable() |