Mercurial > public > mercurial-scm > hg-stable

annotate hgdemandimport/demandimportpy3.py @ 53028:89ab2459f62a stable 6.9.3

Find changesets by keywords (author, files, the commit message), revision number or hash, or revset expression.
stream-bundle: properly process requirements When the manifest bundle constain some known requirement that does not affect the stream clone, we used to crash. However since we know them and know they don't affect the stream clone, we can ignore them. Mozilla generated such buggy manifest bundle for a time which allowed us to catch this error. The issue was not caught until 961900fbd67c (released in 6.9.2) as the requirements information were ignored for stream-v2 until then. We fix the issue, refactor the code for robustness and adds more tests to better catch this kind of issue in the future.
author Pierre-Yves David <pierre-yves.david@octobus.net>
date Wed, 05 Mar 2025 11:41:45 +0100
parents f4733654f144
children
Ignore whitespace changes - Everywhere: Within whitespace: At end of lines:
rev   line source
32461
859496bb6db3 demandimport: add python 3 implementation
Siddharth Agarwal <sid0@fb.com>
parents:
diff changeset
1 # demandimportpy3 - global demand-loading of modules for Mercurial
859496bb6db3 demandimport: add python 3 implementation
Siddharth Agarwal <sid0@fb.com>
parents:
diff changeset
2 #
859496bb6db3 demandimport: add python 3 implementation
Siddharth Agarwal <sid0@fb.com>
parents:
diff changeset
3 # Copyright 2017 Facebook Inc.
859496bb6db3 demandimport: add python 3 implementation
Siddharth Agarwal <sid0@fb.com>
parents:
diff changeset
4 #
859496bb6db3 demandimport: add python 3 implementation
Siddharth Agarwal <sid0@fb.com>
parents:
diff changeset
5 # This software may be used and distributed according to the terms of the
859496bb6db3 demandimport: add python 3 implementation
Siddharth Agarwal <sid0@fb.com>
parents:
diff changeset
6 # GNU General Public License version 2 or any later version.
859496bb6db3 demandimport: add python 3 implementation
Siddharth Agarwal <sid0@fb.com>
parents:
diff changeset
7
859496bb6db3 demandimport: add python 3 implementation
Siddharth Agarwal <sid0@fb.com>
parents:
diff changeset
8 """Lazy loading for Python 3.6 and above.
859496bb6db3 demandimport: add python 3 implementation
Siddharth Agarwal <sid0@fb.com>
parents:
diff changeset
9
859496bb6db3 demandimport: add python 3 implementation
Siddharth Agarwal <sid0@fb.com>
parents:
diff changeset
10 This uses the new importlib finder/loader functionality available in Python 3.5
859496bb6db3 demandimport: add python 3 implementation
Siddharth Agarwal <sid0@fb.com>
parents:
diff changeset
11 and up. The code reuses most of the mechanics implemented inside importlib.util,
859496bb6db3 demandimport: add python 3 implementation
Siddharth Agarwal <sid0@fb.com>
parents:
diff changeset
12 but with a few additions:
859496bb6db3 demandimport: add python 3 implementation
Siddharth Agarwal <sid0@fb.com>
parents:
diff changeset
13
859496bb6db3 demandimport: add python 3 implementation
Siddharth Agarwal <sid0@fb.com>
parents:
diff changeset
14 * Allow excluding certain modules from lazy imports.
859496bb6db3 demandimport: add python 3 implementation
Siddharth Agarwal <sid0@fb.com>
parents:
diff changeset
15 * Expose an interface that's substantially the same as demandimport for
859496bb6db3 demandimport: add python 3 implementation
Siddharth Agarwal <sid0@fb.com>
parents:
diff changeset
16 Python 2.
859496bb6db3 demandimport: add python 3 implementation
Siddharth Agarwal <sid0@fb.com>
parents:
diff changeset
17
859496bb6db3 demandimport: add python 3 implementation
Siddharth Agarwal <sid0@fb.com>
parents:
diff changeset
18 This also has some limitations compared to the Python 2 implementation:
859496bb6db3 demandimport: add python 3 implementation
Siddharth Agarwal <sid0@fb.com>
parents:
diff changeset
19
859496bb6db3 demandimport: add python 3 implementation
Siddharth Agarwal <sid0@fb.com>
parents:
diff changeset
20 * Much of the logic is per-package, not per-module, so any packages loaded
859496bb6db3 demandimport: add python 3 implementation
Siddharth Agarwal <sid0@fb.com>
parents:
diff changeset
21 before demandimport is enabled will not be lazily imported in the future. In
859496bb6db3 demandimport: add python 3 implementation
Siddharth Agarwal <sid0@fb.com>
parents:
diff changeset
22 practice, we only expect builtins to be loaded before demandimport is
859496bb6db3 demandimport: add python 3 implementation
Siddharth Agarwal <sid0@fb.com>
parents:
diff changeset
23 enabled.
859496bb6db3 demandimport: add python 3 implementation
Siddharth Agarwal <sid0@fb.com>
parents:
diff changeset
24 """
859496bb6db3 demandimport: add python 3 implementation
Siddharth Agarwal <sid0@fb.com>
parents:
diff changeset
25
51901
f4733654f144 typing: add `from __future__ import annotations` to most files
Matt Harbison <matt_harbison@yahoo.com>
parents: 49929
diff changeset
26 from __future__ import annotations
f4733654f144 typing: add `from __future__ import annotations` to most files
Matt Harbison <matt_harbison@yahoo.com>
parents: 49929
diff changeset
27
32461
859496bb6db3 demandimport: add python 3 implementation
Siddharth Agarwal <sid0@fb.com>
parents:
diff changeset
28 import contextlib
859496bb6db3 demandimport: add python 3 implementation
Siddharth Agarwal <sid0@fb.com>
parents:
diff changeset
29 import importlib.util
33916
3595e4e0ae57 demandimportpy3: update to pass import checker
Augie Fackler <raf@durin42.com>
parents: 33880
diff changeset
30 import sys
32461
859496bb6db3 demandimport: add python 3 implementation
Siddharth Agarwal <sid0@fb.com>
parents:
diff changeset
31
42490
adb636392b3f demandimport: add tracing coverage for Python 3
Augie Fackler <augie@google.com>
parents: 37889
diff changeset
32 from . import tracing
adb636392b3f demandimport: add tracing coverage for Python 3
Augie Fackler <augie@google.com>
parents: 37889
diff changeset
33
32461
859496bb6db3 demandimport: add python 3 implementation
Siddharth Agarwal <sid0@fb.com>
parents:
diff changeset
34 _deactivated = False
859496bb6db3 demandimport: add python 3 implementation
Siddharth Agarwal <sid0@fb.com>
parents:
diff changeset
35
43076
2372284d9457 formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents: 42490
diff changeset
36
32461
859496bb6db3 demandimport: add python 3 implementation
Siddharth Agarwal <sid0@fb.com>
parents:
diff changeset
37 class _lazyloaderex(importlib.util.LazyLoader):
859496bb6db3 demandimport: add python 3 implementation
Siddharth Agarwal <sid0@fb.com>
parents:
diff changeset
38 """This is a LazyLoader except it also follows the _deactivated global and
859496bb6db3 demandimport: add python 3 implementation
Siddharth Agarwal <sid0@fb.com>
parents:
diff changeset
39 the ignore list.
859496bb6db3 demandimport: add python 3 implementation
Siddharth Agarwal <sid0@fb.com>
parents:
diff changeset
40 """
43076
2372284d9457 formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents: 42490
diff changeset
41
49929
31bbf7a28a75 pytype: add coverage for hgdemandimport
Matt Harbison <matt_harbison@yahoo.com>
parents: 49927
diff changeset
42 _HAS_DYNAMIC_ATTRIBUTES = True # help pytype not flag self.loader
31bbf7a28a75 pytype: add coverage for hgdemandimport
Matt Harbison <matt_harbison@yahoo.com>
parents: 49927
diff changeset
43
32461
859496bb6db3 demandimport: add python 3 implementation
Siddharth Agarwal <sid0@fb.com>
parents:
diff changeset
44 def exec_module(self, module):
859496bb6db3 demandimport: add python 3 implementation
Siddharth Agarwal <sid0@fb.com>
parents:
diff changeset
45 """Make the module load lazily."""
42490
adb636392b3f demandimport: add tracing coverage for Python 3
Augie Fackler <augie@google.com>
parents: 37889
diff changeset
46 with tracing.log('demandimport %s', module):
adb636392b3f demandimport: add tracing coverage for Python 3
Augie Fackler <augie@google.com>
parents: 37889
diff changeset
47 if _deactivated or module.__name__ in ignores:
49729
7236f11db0c3 demandimport: ensure lazyloaderex sets loader attributes (issue6725)
Jason R. Coombs <jaraco@jaraco.com>
parents: 49714
diff changeset
48 # Reset the loader on the module as super() does (issue6725)
7236f11db0c3 demandimport: ensure lazyloaderex sets loader attributes (issue6725)
Jason R. Coombs <jaraco@jaraco.com>
parents: 49714
diff changeset
49 module.__spec__.loader = self.loader
7236f11db0c3 demandimport: ensure lazyloaderex sets loader attributes (issue6725)
Jason R. Coombs <jaraco@jaraco.com>
parents: 49714
diff changeset
50 module.__loader__ = self.loader
7236f11db0c3 demandimport: ensure lazyloaderex sets loader attributes (issue6725)
Jason R. Coombs <jaraco@jaraco.com>
parents: 49714
diff changeset
51
42490
adb636392b3f demandimport: add tracing coverage for Python 3
Augie Fackler <augie@google.com>
parents: 37889
diff changeset
52 self.loader.exec_module(module)
adb636392b3f demandimport: add tracing coverage for Python 3
Augie Fackler <augie@google.com>
parents: 37889
diff changeset
53 else:
adb636392b3f demandimport: add tracing coverage for Python 3
Augie Fackler <augie@google.com>
parents: 37889
diff changeset
54 super().exec_module(module)
32461
859496bb6db3 demandimport: add python 3 implementation
Siddharth Agarwal <sid0@fb.com>
parents:
diff changeset
55
43076
2372284d9457 formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents: 42490
diff changeset
56
49037
642e31cb55f0 py3: use class X: instead of class X(object):
Gregory Szorc <gregory.szorc@gmail.com>
parents: 48966
diff changeset
57 class LazyFinder:
44118
f81c17ec303c hgdemandimport: apply lazy module loading to sys.meta_path finders
Gregory Szorc <gregory.szorc@gmail.com>
parents: 44117
diff changeset
58 """A wrapper around a ``MetaPathFinder`` that makes loaders lazy.
f81c17ec303c hgdemandimport: apply lazy module loading to sys.meta_path finders
Gregory Szorc <gregory.szorc@gmail.com>
parents: 44117
diff changeset
59
f81c17ec303c hgdemandimport: apply lazy module loading to sys.meta_path finders
Gregory Szorc <gregory.szorc@gmail.com>
parents: 44117
diff changeset
60 ``sys.meta_path`` finders have their ``find_spec()`` called to locate a
f81c17ec303c hgdemandimport: apply lazy module loading to sys.meta_path finders
Gregory Szorc <gregory.szorc@gmail.com>
parents: 44117
diff changeset
61 module. This returns a ``ModuleSpec`` if found or ``None``. The
f81c17ec303c hgdemandimport: apply lazy module loading to sys.meta_path finders
Gregory Szorc <gregory.szorc@gmail.com>
parents: 44117
diff changeset
62 ``ModuleSpec`` has a ``loader`` attribute, which is called to actually
f81c17ec303c hgdemandimport: apply lazy module loading to sys.meta_path finders
Gregory Szorc <gregory.szorc@gmail.com>
parents: 44117
diff changeset
63 load a module.
f81c17ec303c hgdemandimport: apply lazy module loading to sys.meta_path finders
Gregory Szorc <gregory.szorc@gmail.com>
parents: 44117
diff changeset
64
f81c17ec303c hgdemandimport: apply lazy module loading to sys.meta_path finders
Gregory Szorc <gregory.szorc@gmail.com>
parents: 44117
diff changeset
65 Our class wraps an existing finder and overloads its ``find_spec()`` to
f81c17ec303c hgdemandimport: apply lazy module loading to sys.meta_path finders
Gregory Szorc <gregory.szorc@gmail.com>
parents: 44117
diff changeset
66 replace the ``loader`` with our lazy loader proxy.
32461
859496bb6db3 demandimport: add python 3 implementation
Siddharth Agarwal <sid0@fb.com>
parents:
diff changeset
67
44118
f81c17ec303c hgdemandimport: apply lazy module loading to sys.meta_path finders
Gregory Szorc <gregory.szorc@gmail.com>
parents: 44117
diff changeset
68 We have to use __getattribute__ to proxy the instance because some meta
f81c17ec303c hgdemandimport: apply lazy module loading to sys.meta_path finders
Gregory Szorc <gregory.szorc@gmail.com>
parents: 44117
diff changeset
69 path finders don't support monkeypatching.
f81c17ec303c hgdemandimport: apply lazy module loading to sys.meta_path finders
Gregory Szorc <gregory.szorc@gmail.com>
parents: 44117
diff changeset
70 """
f81c17ec303c hgdemandimport: apply lazy module loading to sys.meta_path finders
Gregory Szorc <gregory.szorc@gmail.com>
parents: 44117
diff changeset
71
f81c17ec303c hgdemandimport: apply lazy module loading to sys.meta_path finders
Gregory Szorc <gregory.szorc@gmail.com>
parents: 44117
diff changeset
72 __slots__ = ("_finder",)
f81c17ec303c hgdemandimport: apply lazy module loading to sys.meta_path finders
Gregory Szorc <gregory.szorc@gmail.com>
parents: 44117
diff changeset
73
f81c17ec303c hgdemandimport: apply lazy module loading to sys.meta_path finders
Gregory Szorc <gregory.szorc@gmail.com>
parents: 44117
diff changeset
74 def __init__(self, finder):
f81c17ec303c hgdemandimport: apply lazy module loading to sys.meta_path finders
Gregory Szorc <gregory.szorc@gmail.com>
parents: 44117
diff changeset
75 object.__setattr__(self, "_finder", finder)
f81c17ec303c hgdemandimport: apply lazy module loading to sys.meta_path finders
Gregory Szorc <gregory.szorc@gmail.com>
parents: 44117
diff changeset
76
f81c17ec303c hgdemandimport: apply lazy module loading to sys.meta_path finders
Gregory Szorc <gregory.szorc@gmail.com>
parents: 44117
diff changeset
77 def __repr__(self):
f81c17ec303c hgdemandimport: apply lazy module loading to sys.meta_path finders
Gregory Szorc <gregory.szorc@gmail.com>
parents: 44117
diff changeset
78 return "<LazyFinder for %r>" % object.__getattribute__(self, "_finder")
f81c17ec303c hgdemandimport: apply lazy module loading to sys.meta_path finders
Gregory Szorc <gregory.szorc@gmail.com>
parents: 44117
diff changeset
79
f81c17ec303c hgdemandimport: apply lazy module loading to sys.meta_path finders
Gregory Szorc <gregory.szorc@gmail.com>
parents: 44117
diff changeset
80 # __bool__ is canonical Python 3. But check-code insists on __nonzero__ being
f81c17ec303c hgdemandimport: apply lazy module loading to sys.meta_path finders
Gregory Szorc <gregory.szorc@gmail.com>
parents: 44117
diff changeset
81 # defined via `def`.
f81c17ec303c hgdemandimport: apply lazy module loading to sys.meta_path finders
Gregory Szorc <gregory.szorc@gmail.com>
parents: 44117
diff changeset
82 def __nonzero__(self):
f81c17ec303c hgdemandimport: apply lazy module loading to sys.meta_path finders
Gregory Szorc <gregory.szorc@gmail.com>
parents: 44117
diff changeset
83 return bool(object.__getattribute__(self, "_finder"))
43076
2372284d9457 formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents: 42490
diff changeset
84
44118
f81c17ec303c hgdemandimport: apply lazy module loading to sys.meta_path finders
Gregory Szorc <gregory.szorc@gmail.com>
parents: 44117
diff changeset
85 __bool__ = __nonzero__
f81c17ec303c hgdemandimport: apply lazy module loading to sys.meta_path finders
Gregory Szorc <gregory.szorc@gmail.com>
parents: 44117
diff changeset
86
f81c17ec303c hgdemandimport: apply lazy module loading to sys.meta_path finders
Gregory Szorc <gregory.szorc@gmail.com>
parents: 44117
diff changeset
87 def __getattribute__(self, name):
f81c17ec303c hgdemandimport: apply lazy module loading to sys.meta_path finders
Gregory Szorc <gregory.szorc@gmail.com>
parents: 44117
diff changeset
88 if name in ("_finder", "find_spec"):
f81c17ec303c hgdemandimport: apply lazy module loading to sys.meta_path finders
Gregory Szorc <gregory.szorc@gmail.com>
parents: 44117
diff changeset
89 return object.__getattribute__(self, name)
f81c17ec303c hgdemandimport: apply lazy module loading to sys.meta_path finders
Gregory Szorc <gregory.szorc@gmail.com>
parents: 44117
diff changeset
90
f81c17ec303c hgdemandimport: apply lazy module loading to sys.meta_path finders
Gregory Szorc <gregory.szorc@gmail.com>
parents: 44117
diff changeset
91 return getattr(object.__getattribute__(self, "_finder"), name)
f81c17ec303c hgdemandimport: apply lazy module loading to sys.meta_path finders
Gregory Szorc <gregory.szorc@gmail.com>
parents: 44117
diff changeset
92
f81c17ec303c hgdemandimport: apply lazy module loading to sys.meta_path finders
Gregory Szorc <gregory.szorc@gmail.com>
parents: 44117
diff changeset
93 def __delattr__(self, name):
49599
48e38b179106 demandimport: fix a crash in LazyFinder.__delattr__
Matt Harbison <matt_harbison@yahoo.com>
parents: 49049
diff changeset
94 return delattr(object.__getattribute__(self, "_finder"), name)
44118
f81c17ec303c hgdemandimport: apply lazy module loading to sys.meta_path finders
Gregory Szorc <gregory.szorc@gmail.com>
parents: 44117
diff changeset
95
f81c17ec303c hgdemandimport: apply lazy module loading to sys.meta_path finders
Gregory Szorc <gregory.szorc@gmail.com>
parents: 44117
diff changeset
96 def __setattr__(self, name, value):
f81c17ec303c hgdemandimport: apply lazy module loading to sys.meta_path finders
Gregory Szorc <gregory.szorc@gmail.com>
parents: 44117
diff changeset
97 return setattr(object.__getattribute__(self, "_finder"), name, value)
f81c17ec303c hgdemandimport: apply lazy module loading to sys.meta_path finders
Gregory Szorc <gregory.szorc@gmail.com>
parents: 44117
diff changeset
98
44775
a6e12d477595 demandimport: fix compatibility with meta path finders w/o find_spec() method
Manuel Jacob <me@manueljacob.de>
parents: 44118
diff changeset
99 def find_spec(self, fullname, path, target=None):
44118
f81c17ec303c hgdemandimport: apply lazy module loading to sys.meta_path finders
Gregory Szorc <gregory.szorc@gmail.com>
parents: 44117
diff changeset
100 finder = object.__getattribute__(self, "_finder")
44775
a6e12d477595 demandimport: fix compatibility with meta path finders w/o find_spec() method
Manuel Jacob <me@manueljacob.de>
parents: 44118
diff changeset
101 try:
a6e12d477595 demandimport: fix compatibility with meta path finders w/o find_spec() method
Manuel Jacob <me@manueljacob.de>
parents: 44118
diff changeset
102 find_spec = finder.find_spec
a6e12d477595 demandimport: fix compatibility with meta path finders w/o find_spec() method
Manuel Jacob <me@manueljacob.de>
parents: 44118
diff changeset
103 except AttributeError:
a6e12d477595 demandimport: fix compatibility with meta path finders w/o find_spec() method
Manuel Jacob <me@manueljacob.de>
parents: 44118
diff changeset
104 loader = finder.find_module(fullname, path)
a6e12d477595 demandimport: fix compatibility with meta path finders w/o find_spec() method
Manuel Jacob <me@manueljacob.de>
parents: 44118
diff changeset
105 if loader is None:
a6e12d477595 demandimport: fix compatibility with meta path finders w/o find_spec() method
Manuel Jacob <me@manueljacob.de>
parents: 44118
diff changeset
106 spec = None
a6e12d477595 demandimport: fix compatibility with meta path finders w/o find_spec() method
Manuel Jacob <me@manueljacob.de>
parents: 44118
diff changeset
107 else:
a6e12d477595 demandimport: fix compatibility with meta path finders w/o find_spec() method
Manuel Jacob <me@manueljacob.de>
parents: 44118
diff changeset
108 spec = importlib.util.spec_from_loader(fullname, loader)
a6e12d477595 demandimport: fix compatibility with meta path finders w/o find_spec() method
Manuel Jacob <me@manueljacob.de>
parents: 44118
diff changeset
109 else:
a6e12d477595 demandimport: fix compatibility with meta path finders w/o find_spec() method
Manuel Jacob <me@manueljacob.de>
parents: 44118
diff changeset
110 spec = find_spec(fullname, path, target)
44118
f81c17ec303c hgdemandimport: apply lazy module loading to sys.meta_path finders
Gregory Szorc <gregory.szorc@gmail.com>
parents: 44117
diff changeset
111
f81c17ec303c hgdemandimport: apply lazy module loading to sys.meta_path finders
Gregory Szorc <gregory.szorc@gmail.com>
parents: 44117
diff changeset
112 # Lazy loader requires exec_module().
f81c17ec303c hgdemandimport: apply lazy module loading to sys.meta_path finders
Gregory Szorc <gregory.szorc@gmail.com>
parents: 44117
diff changeset
113 if (
f81c17ec303c hgdemandimport: apply lazy module loading to sys.meta_path finders
Gregory Szorc <gregory.szorc@gmail.com>
parents: 44117
diff changeset
114 spec is not None
f81c17ec303c hgdemandimport: apply lazy module loading to sys.meta_path finders
Gregory Szorc <gregory.szorc@gmail.com>
parents: 44117
diff changeset
115 and spec.loader is not None
45755
8ed69bd42f10 demandimport: don't raise AttributeError if `exec_module` is missing
Matt Harbison <matt_harbison@yahoo.com>
parents: 44775
diff changeset
116 and getattr(spec.loader, "exec_module", None)
44118
f81c17ec303c hgdemandimport: apply lazy module loading to sys.meta_path finders
Gregory Szorc <gregory.szorc@gmail.com>
parents: 44117
diff changeset
117 ):
f81c17ec303c hgdemandimport: apply lazy module loading to sys.meta_path finders
Gregory Szorc <gregory.szorc@gmail.com>
parents: 44117
diff changeset
118 spec.loader = _lazyloaderex(spec.loader)
f81c17ec303c hgdemandimport: apply lazy module loading to sys.meta_path finders
Gregory Szorc <gregory.szorc@gmail.com>
parents: 44117
diff changeset
119
f81c17ec303c hgdemandimport: apply lazy module loading to sys.meta_path finders
Gregory Szorc <gregory.szorc@gmail.com>
parents: 44117
diff changeset
120 return spec
32461
859496bb6db3 demandimport: add python 3 implementation
Siddharth Agarwal <sid0@fb.com>
parents:
diff changeset
121
43076
2372284d9457 formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents: 42490
diff changeset
122
37889
670eb4fa1b86 demandimport: make module ignores a set (API)
Gregory Szorc <gregory.szorc@gmail.com>
parents: 35528
diff changeset
123 ignores = set()
32461
859496bb6db3 demandimport: add python 3 implementation
Siddharth Agarwal <sid0@fb.com>
parents:
diff changeset
124
43076
2372284d9457 formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents: 42490
diff changeset
125
37889
670eb4fa1b86 demandimport: make module ignores a set (API)
Gregory Szorc <gregory.szorc@gmail.com>
parents: 35528
diff changeset
126 def init(ignoreset):
670eb4fa1b86 demandimport: make module ignores a set (API)
Gregory Szorc <gregory.szorc@gmail.com>
parents: 35528
diff changeset
127 global ignores
670eb4fa1b86 demandimport: make module ignores a set (API)
Gregory Szorc <gregory.szorc@gmail.com>
parents: 35528
diff changeset
128 ignores = ignoreset
32461
859496bb6db3 demandimport: add python 3 implementation
Siddharth Agarwal <sid0@fb.com>
parents:
diff changeset
129
43076
2372284d9457 formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents: 42490
diff changeset
130
32461
859496bb6db3 demandimport: add python 3 implementation
Siddharth Agarwal <sid0@fb.com>
parents:
diff changeset
131 def isenabled():
44118
f81c17ec303c hgdemandimport: apply lazy module loading to sys.meta_path finders
Gregory Szorc <gregory.szorc@gmail.com>
parents: 44117
diff changeset
132 return not _deactivated and any(
f81c17ec303c hgdemandimport: apply lazy module loading to sys.meta_path finders
Gregory Szorc <gregory.szorc@gmail.com>
parents: 44117
diff changeset
133 isinstance(finder, LazyFinder) for finder in sys.meta_path
f81c17ec303c hgdemandimport: apply lazy module loading to sys.meta_path finders
Gregory Szorc <gregory.szorc@gmail.com>
parents: 44117
diff changeset
134 )
32461
859496bb6db3 demandimport: add python 3 implementation
Siddharth Agarwal <sid0@fb.com>
parents:
diff changeset
135
43076
2372284d9457 formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents: 42490
diff changeset
136
32461
859496bb6db3 demandimport: add python 3 implementation
Siddharth Agarwal <sid0@fb.com>
parents:
diff changeset
137 def disable():
44118
f81c17ec303c hgdemandimport: apply lazy module loading to sys.meta_path finders
Gregory Szorc <gregory.szorc@gmail.com>
parents: 44117
diff changeset
138 new_finders = []
f81c17ec303c hgdemandimport: apply lazy module loading to sys.meta_path finders
Gregory Szorc <gregory.szorc@gmail.com>
parents: 44117
diff changeset
139 for finder in sys.meta_path:
f81c17ec303c hgdemandimport: apply lazy module loading to sys.meta_path finders
Gregory Szorc <gregory.szorc@gmail.com>
parents: 44117
diff changeset
140 new_finders.append(
f81c17ec303c hgdemandimport: apply lazy module loading to sys.meta_path finders
Gregory Szorc <gregory.szorc@gmail.com>
parents: 44117
diff changeset
141 finder._finder if isinstance(finder, LazyFinder) else finder
f81c17ec303c hgdemandimport: apply lazy module loading to sys.meta_path finders
Gregory Szorc <gregory.szorc@gmail.com>
parents: 44117
diff changeset
142 )
f81c17ec303c hgdemandimport: apply lazy module loading to sys.meta_path finders
Gregory Szorc <gregory.szorc@gmail.com>
parents: 44117
diff changeset
143 sys.meta_path[:] = new_finders
32461
859496bb6db3 demandimport: add python 3 implementation
Siddharth Agarwal <sid0@fb.com>
parents:
diff changeset
144
43076
2372284d9457 formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents: 42490
diff changeset
145
32461
859496bb6db3 demandimport: add python 3 implementation
Siddharth Agarwal <sid0@fb.com>
parents:
diff changeset
146 def enable():
44118
f81c17ec303c hgdemandimport: apply lazy module loading to sys.meta_path finders
Gregory Szorc <gregory.szorc@gmail.com>
parents: 44117
diff changeset
147 new_finders = []
f81c17ec303c hgdemandimport: apply lazy module loading to sys.meta_path finders
Gregory Szorc <gregory.szorc@gmail.com>
parents: 44117
diff changeset
148 for finder in sys.meta_path:
f81c17ec303c hgdemandimport: apply lazy module loading to sys.meta_path finders
Gregory Szorc <gregory.szorc@gmail.com>
parents: 44117
diff changeset
149 new_finders.append(
f81c17ec303c hgdemandimport: apply lazy module loading to sys.meta_path finders
Gregory Szorc <gregory.szorc@gmail.com>
parents: 44117
diff changeset
150 LazyFinder(finder) if not isinstance(finder, LazyFinder) else finder
f81c17ec303c hgdemandimport: apply lazy module loading to sys.meta_path finders
Gregory Szorc <gregory.szorc@gmail.com>
parents: 44117
diff changeset
151 )
f81c17ec303c hgdemandimport: apply lazy module loading to sys.meta_path finders
Gregory Szorc <gregory.szorc@gmail.com>
parents: 44117
diff changeset
152 sys.meta_path[:] = new_finders
32461
859496bb6db3 demandimport: add python 3 implementation
Siddharth Agarwal <sid0@fb.com>
parents:
diff changeset
153
43076
2372284d9457 formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents: 42490
diff changeset
154
32461
859496bb6db3 demandimport: add python 3 implementation
Siddharth Agarwal <sid0@fb.com>
parents:
diff changeset
155 @contextlib.contextmanager
859496bb6db3 demandimport: add python 3 implementation
Siddharth Agarwal <sid0@fb.com>
parents:
diff changeset
156 def deactivated():
859496bb6db3 demandimport: add python 3 implementation
Siddharth Agarwal <sid0@fb.com>
parents:
diff changeset
157 # This implementation is a bit different from Python 2's. Python 3
859496bb6db3 demandimport: add python 3 implementation
Siddharth Agarwal <sid0@fb.com>
parents:
diff changeset
158 # maintains a per-package finder cache in sys.path_importer_cache (see
859496bb6db3 demandimport: add python 3 implementation
Siddharth Agarwal <sid0@fb.com>
parents:
diff changeset
159 # PEP 302). This means that we can't just call disable + enable.
859496bb6db3 demandimport: add python 3 implementation
Siddharth Agarwal <sid0@fb.com>
parents:
diff changeset
160 # If we do that, in situations like:
859496bb6db3 demandimport: add python 3 implementation
Siddharth Agarwal <sid0@fb.com>
parents:
diff changeset
161 #
859496bb6db3 demandimport: add python 3 implementation
Siddharth Agarwal <sid0@fb.com>
parents:
diff changeset
162 # demandimport.enable()
859496bb6db3 demandimport: add python 3 implementation
Siddharth Agarwal <sid0@fb.com>
parents:
diff changeset
163 # ...
859496bb6db3 demandimport: add python 3 implementation
Siddharth Agarwal <sid0@fb.com>
parents:
diff changeset
164 # from foo.bar import mod1
859496bb6db3 demandimport: add python 3 implementation
Siddharth Agarwal <sid0@fb.com>
parents:
diff changeset
165 # with demandimport.deactivated():
859496bb6db3 demandimport: add python 3 implementation
Siddharth Agarwal <sid0@fb.com>
parents:
diff changeset
166 # from foo.bar import mod2
859496bb6db3 demandimport: add python 3 implementation
Siddharth Agarwal <sid0@fb.com>
parents:
diff changeset
167 #
859496bb6db3 demandimport: add python 3 implementation
Siddharth Agarwal <sid0@fb.com>
parents:
diff changeset
168 # mod2 will be imported lazily. (The converse also holds -- whatever finder
859496bb6db3 demandimport: add python 3 implementation
Siddharth Agarwal <sid0@fb.com>
parents:
diff changeset
169 # first gets cached will be used.)
859496bb6db3 demandimport: add python 3 implementation
Siddharth Agarwal <sid0@fb.com>
parents:
diff changeset
170 #
859496bb6db3 demandimport: add python 3 implementation
Siddharth Agarwal <sid0@fb.com>
parents:
diff changeset
171 # Instead, have a global flag the LazyLoader can use.
859496bb6db3 demandimport: add python 3 implementation
Siddharth Agarwal <sid0@fb.com>
parents:
diff changeset
172 global _deactivated
859496bb6db3 demandimport: add python 3 implementation
Siddharth Agarwal <sid0@fb.com>
parents:
diff changeset
173 demandenabled = isenabled()
859496bb6db3 demandimport: add python 3 implementation
Siddharth Agarwal <sid0@fb.com>
parents:
diff changeset
174 if demandenabled:
859496bb6db3 demandimport: add python 3 implementation
Siddharth Agarwal <sid0@fb.com>
parents:
diff changeset
175 _deactivated = True
859496bb6db3 demandimport: add python 3 implementation
Siddharth Agarwal <sid0@fb.com>
parents:
diff changeset
176 try:
859496bb6db3 demandimport: add python 3 implementation
Siddharth Agarwal <sid0@fb.com>
parents:
diff changeset
177 yield
859496bb6db3 demandimport: add python 3 implementation
Siddharth Agarwal <sid0@fb.com>
parents:
diff changeset
178 finally:
859496bb6db3 demandimport: add python 3 implementation
Siddharth Agarwal <sid0@fb.com>
parents:
diff changeset
179 if demandenabled:
859496bb6db3 demandimport: add python 3 implementation
Siddharth Agarwal <sid0@fb.com>
parents:
diff changeset
180 _deactivated = False