Mercurial > public > mercurial-scm > hg-stable
view mercurial/packagescan.py @ 1598:14d1f1868bf6
cleanup of revlog.group when repository is local
revlog.group cached every chunk from the revlog, the behaviour was
needed to minimize the roundtrip with old-http.
We now cache the revlog data ~4MB at a time.
The memory used server side when pulling goes down to 35Mo maximum
whereas without the patch more than 160Mo was used when cloning the linux kernel
repository.
The time used by cloning is higher mainly because of the check in revlog.revision.
before
110.25user 20.90system 2:52.00elapsed 76%CPU (0avgtext+0avgdata 0maxresident)k
0inputs+0outputs (0major+708707minor)pagefaults 0swaps
after
117.56user 18.86system 2:50.43elapsed 80%CPU (0avgtext+0avgdata 0maxresident)k
0inputs+0outputs (0major+748366minor)pagefaults 0swaps
author | Benoit Boissinot <benoit.boissinot@ens-lyon.org> |
---|---|
date | Tue, 27 Dec 2005 13:09:49 -0600 |
parents | 2073e5a71008 |
children | 7f12a63568ae |
line wrap: on
line source
# packagescan.py - Helper module for identifing used modules. # Used for the py2exe distutil. # # Copyright 2005 Volker Kleinfeld <Volker.Kleinfeld@gmx.de> # # This software may be used and distributed according to the terms # of the GNU General Public License, incorporated herein by reference. import glob import os import sys import demandload import ihooks requiredmodules = {} # Will contain the modules imported by demandload def demandload(scope, modules): """ fake demandload function that collects the required modules """ for m in modules.split(): mod = None mod = __import__(m,scope,scope) scope[m] = mod requiredmodules[mod.__name__] = 1 def getmodules(libpath,packagename): """ helper for finding all required modules of package <packagename> """ # Use the package in the build directory libpath = os.path.abspath(libpath) sys.path.insert(0,libpath) packdir = os.path.join(libpath,packagename) # A normal import would not find the package in # the build directory. ihook is used to force the import. # After the package is imported the import scope for # the following imports is settled. p = importfrom(packdir) globals()[packagename] = p sys.modules[packagename] = p # Fetch the python modules in the package cwd = os.getcwd() os.chdir(packdir) pymodulefiles = glob.glob('*.py') extmodulefiles = glob.glob('*.pyd') os.chdir(cwd) # Install a fake demandload module sys.modules['mercurial.demandload'] = sys.modules['mercurial.packagescan'] # Import all python modules and by that run the fake demandload for m in pymodulefiles: if m == '__init__.py': continue tmp = {} mname,ext = os.path.splitext(m) fullname = packagename+'.'+mname __import__(fullname,tmp,tmp) requiredmodules[fullname] = 1 # Import all extension modules and by that run the fake demandload for m in extmodulefiles: tmp = {} mname,ext = os.path.splitext(m) fullname = packagename+'.'+mname __import__(fullname,tmp,tmp) requiredmodules[fullname] = 1 includes = requiredmodules.keys() return includes def importfrom(filename): """ import module/package from a named file and returns the module. It does not check on sys.modules or includes the module in the scope. """ loader = ihooks.BasicModuleLoader() path, file = os.path.split(filename) name, ext = os.path.splitext(file) m = loader.find_module_in_dir(name, path) if not m: raise ImportError, name m = loader.load_module(name, m) return m