Mercurial > public > mercurial-scm > hg-stable

annotate mercurial/packagescan.py @ 1598:14d1f1868bf6

Find changesets by keywords (author, files, the commit message), revision number or hash, or revset expression.
cleanup of revlog.group when repository is local revlog.group cached every chunk from the revlog, the behaviour was needed to minimize the roundtrip with old-http. We now cache the revlog data ~4MB at a time. The memory used server side when pulling goes down to 35Mo maximum whereas without the patch more than 160Mo was used when cloning the linux kernel repository. The time used by cloning is higher mainly because of the check in revlog.revision. before 110.25user 20.90system 2:52.00elapsed 76%CPU (0avgtext+0avgdata 0maxresident)k 0inputs+0outputs (0major+708707minor)pagefaults 0swaps after 117.56user 18.86system 2:50.43elapsed 80%CPU (0avgtext+0avgdata 0maxresident)k 0inputs+0outputs (0major+748366minor)pagefaults 0swaps
author Benoit Boissinot <benoit.boissinot@ens-lyon.org>
date Tue, 27 Dec 2005 13:09:49 -0600
parents 2073e5a71008
children 7f12a63568ae
Ignore whitespace changes - Everywhere: Within whitespace: At end of lines:
rev   line source
1308
2073e5a71008 Cleanup of tabs and trailing spaces.
Thomas Arendsen Hein <thomas@intevation.de>
parents: 1283
diff changeset
1 # packagescan.py - Helper module for identifing used modules.
1283
f5faab34f32e Support for the distutils extention 'py2exe' added.
Volker.Kleinfeld@gmx.de
parents:
diff changeset
2 # Used for the py2exe distutil.
f5faab34f32e Support for the distutils extention 'py2exe' added.
Volker.Kleinfeld@gmx.de
parents:
diff changeset
3 #
f5faab34f32e Support for the distutils extention 'py2exe' added.
Volker.Kleinfeld@gmx.de
parents:
diff changeset
4 # Copyright 2005 Volker Kleinfeld <Volker.Kleinfeld@gmx.de>
f5faab34f32e Support for the distutils extention 'py2exe' added.
Volker.Kleinfeld@gmx.de
parents:
diff changeset
5 #
f5faab34f32e Support for the distutils extention 'py2exe' added.
Volker.Kleinfeld@gmx.de
parents:
diff changeset
6 # This software may be used and distributed according to the terms
f5faab34f32e Support for the distutils extention 'py2exe' added.
Volker.Kleinfeld@gmx.de
parents:
diff changeset
7 # of the GNU General Public License, incorporated herein by reference.
f5faab34f32e Support for the distutils extention 'py2exe' added.
Volker.Kleinfeld@gmx.de
parents:
diff changeset
8 import glob
f5faab34f32e Support for the distutils extention 'py2exe' added.
Volker.Kleinfeld@gmx.de
parents:
diff changeset
9 import os
f5faab34f32e Support for the distutils extention 'py2exe' added.
Volker.Kleinfeld@gmx.de
parents:
diff changeset
10 import sys
f5faab34f32e Support for the distutils extention 'py2exe' added.
Volker.Kleinfeld@gmx.de
parents:
diff changeset
11 import demandload
f5faab34f32e Support for the distutils extention 'py2exe' added.
Volker.Kleinfeld@gmx.de
parents:
diff changeset
12 import ihooks
f5faab34f32e Support for the distutils extention 'py2exe' added.
Volker.Kleinfeld@gmx.de
parents:
diff changeset
13
f5faab34f32e Support for the distutils extention 'py2exe' added.
Volker.Kleinfeld@gmx.de
parents:
diff changeset
14 requiredmodules = {} # Will contain the modules imported by demandload
f5faab34f32e Support for the distutils extention 'py2exe' added.
Volker.Kleinfeld@gmx.de
parents:
diff changeset
15 def demandload(scope, modules):
f5faab34f32e Support for the distutils extention 'py2exe' added.
Volker.Kleinfeld@gmx.de
parents:
diff changeset
16 """ fake demandload function that collects the required modules """
f5faab34f32e Support for the distutils extention 'py2exe' added.
Volker.Kleinfeld@gmx.de
parents:
diff changeset
17 for m in modules.split():
f5faab34f32e Support for the distutils extention 'py2exe' added.
Volker.Kleinfeld@gmx.de
parents:
diff changeset
18 mod = None
f5faab34f32e Support for the distutils extention 'py2exe' added.
Volker.Kleinfeld@gmx.de
parents:
diff changeset
19 mod = __import__(m,scope,scope)
f5faab34f32e Support for the distutils extention 'py2exe' added.
Volker.Kleinfeld@gmx.de
parents:
diff changeset
20 scope[m] = mod
f5faab34f32e Support for the distutils extention 'py2exe' added.
Volker.Kleinfeld@gmx.de
parents:
diff changeset
21 requiredmodules[mod.__name__] = 1
f5faab34f32e Support for the distutils extention 'py2exe' added.
Volker.Kleinfeld@gmx.de
parents:
diff changeset
22
f5faab34f32e Support for the distutils extention 'py2exe' added.
Volker.Kleinfeld@gmx.de
parents:
diff changeset
23 def getmodules(libpath,packagename):
f5faab34f32e Support for the distutils extention 'py2exe' added.
Volker.Kleinfeld@gmx.de
parents:
diff changeset
24 """ helper for finding all required modules of package <packagename> """
f5faab34f32e Support for the distutils extention 'py2exe' added.
Volker.Kleinfeld@gmx.de
parents:
diff changeset
25 # Use the package in the build directory
f5faab34f32e Support for the distutils extention 'py2exe' added.
Volker.Kleinfeld@gmx.de
parents:
diff changeset
26 libpath = os.path.abspath(libpath)
f5faab34f32e Support for the distutils extention 'py2exe' added.
Volker.Kleinfeld@gmx.de
parents:
diff changeset
27 sys.path.insert(0,libpath)
f5faab34f32e Support for the distutils extention 'py2exe' added.
Volker.Kleinfeld@gmx.de
parents:
diff changeset
28 packdir = os.path.join(libpath,packagename)
1308
2073e5a71008 Cleanup of tabs and trailing spaces.
Thomas Arendsen Hein <thomas@intevation.de>
parents: 1283
diff changeset
29 # A normal import would not find the package in
1283
f5faab34f32e Support for the distutils extention 'py2exe' added.
Volker.Kleinfeld@gmx.de
parents:
diff changeset
30 # the build directory. ihook is used to force the import.
1308
2073e5a71008 Cleanup of tabs and trailing spaces.
Thomas Arendsen Hein <thomas@intevation.de>
parents: 1283
diff changeset
31 # After the package is imported the import scope for
1283
f5faab34f32e Support for the distutils extention 'py2exe' added.
Volker.Kleinfeld@gmx.de
parents:
diff changeset
32 # the following imports is settled.
f5faab34f32e Support for the distutils extention 'py2exe' added.
Volker.Kleinfeld@gmx.de
parents:
diff changeset
33 p = importfrom(packdir)
f5faab34f32e Support for the distutils extention 'py2exe' added.
Volker.Kleinfeld@gmx.de
parents:
diff changeset
34 globals()[packagename] = p
f5faab34f32e Support for the distutils extention 'py2exe' added.
Volker.Kleinfeld@gmx.de
parents:
diff changeset
35 sys.modules[packagename] = p
f5faab34f32e Support for the distutils extention 'py2exe' added.
Volker.Kleinfeld@gmx.de
parents:
diff changeset
36 # Fetch the python modules in the package
f5faab34f32e Support for the distutils extention 'py2exe' added.
Volker.Kleinfeld@gmx.de
parents:
diff changeset
37 cwd = os.getcwd()
f5faab34f32e Support for the distutils extention 'py2exe' added.
Volker.Kleinfeld@gmx.de
parents:
diff changeset
38 os.chdir(packdir)
f5faab34f32e Support for the distutils extention 'py2exe' added.
Volker.Kleinfeld@gmx.de
parents:
diff changeset
39 pymodulefiles = glob.glob('*.py')
f5faab34f32e Support for the distutils extention 'py2exe' added.
Volker.Kleinfeld@gmx.de
parents:
diff changeset
40 extmodulefiles = glob.glob('*.pyd')
f5faab34f32e Support for the distutils extention 'py2exe' added.
Volker.Kleinfeld@gmx.de
parents:
diff changeset
41 os.chdir(cwd)
f5faab34f32e Support for the distutils extention 'py2exe' added.
Volker.Kleinfeld@gmx.de
parents:
diff changeset
42 # Install a fake demandload module
f5faab34f32e Support for the distutils extention 'py2exe' added.
Volker.Kleinfeld@gmx.de
parents:
diff changeset
43 sys.modules['mercurial.demandload'] = sys.modules['mercurial.packagescan']
f5faab34f32e Support for the distutils extention 'py2exe' added.
Volker.Kleinfeld@gmx.de
parents:
diff changeset
44 # Import all python modules and by that run the fake demandload
f5faab34f32e Support for the distutils extention 'py2exe' added.
Volker.Kleinfeld@gmx.de
parents:
diff changeset
45 for m in pymodulefiles:
f5faab34f32e Support for the distutils extention 'py2exe' added.
Volker.Kleinfeld@gmx.de
parents:
diff changeset
46 if m == '__init__.py': continue
f5faab34f32e Support for the distutils extention 'py2exe' added.
Volker.Kleinfeld@gmx.de
parents:
diff changeset
47 tmp = {}
f5faab34f32e Support for the distutils extention 'py2exe' added.
Volker.Kleinfeld@gmx.de
parents:
diff changeset
48 mname,ext = os.path.splitext(m)
1308
2073e5a71008 Cleanup of tabs and trailing spaces.
Thomas Arendsen Hein <thomas@intevation.de>
parents: 1283
diff changeset
49 fullname = packagename+'.'+mname
1283
f5faab34f32e Support for the distutils extention 'py2exe' added.
Volker.Kleinfeld@gmx.de
parents:
diff changeset
50 __import__(fullname,tmp,tmp)
f5faab34f32e Support for the distutils extention 'py2exe' added.
Volker.Kleinfeld@gmx.de
parents:
diff changeset
51 requiredmodules[fullname] = 1
f5faab34f32e Support for the distutils extention 'py2exe' added.
Volker.Kleinfeld@gmx.de
parents:
diff changeset
52 # Import all extension modules and by that run the fake demandload
f5faab34f32e Support for the distutils extention 'py2exe' added.
Volker.Kleinfeld@gmx.de
parents:
diff changeset
53 for m in extmodulefiles:
f5faab34f32e Support for the distutils extention 'py2exe' added.
Volker.Kleinfeld@gmx.de
parents:
diff changeset
54 tmp = {}
f5faab34f32e Support for the distutils extention 'py2exe' added.
Volker.Kleinfeld@gmx.de
parents:
diff changeset
55 mname,ext = os.path.splitext(m)
f5faab34f32e Support for the distutils extention 'py2exe' added.
Volker.Kleinfeld@gmx.de
parents:
diff changeset
56 fullname = packagename+'.'+mname
f5faab34f32e Support for the distutils extention 'py2exe' added.
Volker.Kleinfeld@gmx.de
parents:
diff changeset
57 __import__(fullname,tmp,tmp)
f5faab34f32e Support for the distutils extention 'py2exe' added.
Volker.Kleinfeld@gmx.de
parents:
diff changeset
58 requiredmodules[fullname] = 1
f5faab34f32e Support for the distutils extention 'py2exe' added.
Volker.Kleinfeld@gmx.de
parents:
diff changeset
59 includes = requiredmodules.keys()
f5faab34f32e Support for the distutils extention 'py2exe' added.
Volker.Kleinfeld@gmx.de
parents:
diff changeset
60 return includes
f5faab34f32e Support for the distutils extention 'py2exe' added.
Volker.Kleinfeld@gmx.de
parents:
diff changeset
61
f5faab34f32e Support for the distutils extention 'py2exe' added.
Volker.Kleinfeld@gmx.de
parents:
diff changeset
62 def importfrom(filename):
f5faab34f32e Support for the distutils extention 'py2exe' added.
Volker.Kleinfeld@gmx.de
parents:
diff changeset
63 """
f5faab34f32e Support for the distutils extention 'py2exe' added.
Volker.Kleinfeld@gmx.de
parents:
diff changeset
64 import module/package from a named file and returns the module.
f5faab34f32e Support for the distutils extention 'py2exe' added.
Volker.Kleinfeld@gmx.de
parents:
diff changeset
65 It does not check on sys.modules or includes the module in the scope.
f5faab34f32e Support for the distutils extention 'py2exe' added.
Volker.Kleinfeld@gmx.de
parents:
diff changeset
66 """
f5faab34f32e Support for the distutils extention 'py2exe' added.
Volker.Kleinfeld@gmx.de
parents:
diff changeset
67 loader = ihooks.BasicModuleLoader()
f5faab34f32e Support for the distutils extention 'py2exe' added.
Volker.Kleinfeld@gmx.de
parents:
diff changeset
68 path, file = os.path.split(filename)
f5faab34f32e Support for the distutils extention 'py2exe' added.
Volker.Kleinfeld@gmx.de
parents:
diff changeset
69 name, ext = os.path.splitext(file)
f5faab34f32e Support for the distutils extention 'py2exe' added.
Volker.Kleinfeld@gmx.de
parents:
diff changeset
70 m = loader.find_module_in_dir(name, path)
f5faab34f32e Support for the distutils extention 'py2exe' added.
Volker.Kleinfeld@gmx.de
parents:
diff changeset
71 if not m:
f5faab34f32e Support for the distutils extention 'py2exe' added.
Volker.Kleinfeld@gmx.de
parents:
diff changeset
72 raise ImportError, name
f5faab34f32e Support for the distutils extention 'py2exe' added.
Volker.Kleinfeld@gmx.de
parents:
diff changeset
73 m = loader.load_module(name, m)
f5faab34f32e Support for the distutils extention 'py2exe' added.
Volker.Kleinfeld@gmx.de
parents:
diff changeset
74 return m