Mercurial > public > mercurial-scm > hg

annotate contrib/benchmarks/__init__.py @ 30559:d83ca854fa21

Find changesets by keywords (author, files, the commit message), revision number or hash, or revset expression.
ui: factor out ui.load() to create a ui without loading configs (API) This allows us to write doctests depending on a ui object, but not on global configs. ui.load() is a class method so we can do wsgiui.load(). All ui() calls but for doctests are replaced with ui.load(). Some of them could be changed to not load configs later.
author Yuya Nishihara <yuya@tcha.org>
date Sat, 22 Oct 2016 14:35:10 +0900
parents cff0f5926797
children be0e7af80543
Ignore whitespace changes - Everywhere: Within whitespace: At end of lines:
rev   line source
30406
cff0f5926797 perf: add asv benchmarks
Philippe Pepiot <philippe.pepiot@logilab.fr>
parents:
diff changeset
1 # __init__.py - asv benchmark suite
cff0f5926797 perf: add asv benchmarks
Philippe Pepiot <philippe.pepiot@logilab.fr>
parents:
diff changeset
2 #
cff0f5926797 perf: add asv benchmarks
Philippe Pepiot <philippe.pepiot@logilab.fr>
parents:
diff changeset
3 # Copyright 2016 Logilab SA <contact@logilab.fr>
cff0f5926797 perf: add asv benchmarks
Philippe Pepiot <philippe.pepiot@logilab.fr>
parents:
diff changeset
4 #
cff0f5926797 perf: add asv benchmarks
Philippe Pepiot <philippe.pepiot@logilab.fr>
parents:
diff changeset
5 # This software may be used and distributed according to the terms of the
cff0f5926797 perf: add asv benchmarks
Philippe Pepiot <philippe.pepiot@logilab.fr>
parents:
diff changeset
6 # GNU General Public License version 2 or any later version.
cff0f5926797 perf: add asv benchmarks
Philippe Pepiot <philippe.pepiot@logilab.fr>
parents:
diff changeset
7
cff0f5926797 perf: add asv benchmarks
Philippe Pepiot <philippe.pepiot@logilab.fr>
parents:
diff changeset
8 '''ASV (https://asv.readthedocs.io) benchmark suite
cff0f5926797 perf: add asv benchmarks
Philippe Pepiot <philippe.pepiot@logilab.fr>
parents:
diff changeset
9
cff0f5926797 perf: add asv benchmarks
Philippe Pepiot <philippe.pepiot@logilab.fr>
parents:
diff changeset
10 Benchmark are parameterized against reference repositories found in the
cff0f5926797 perf: add asv benchmarks
Philippe Pepiot <philippe.pepiot@logilab.fr>
parents:
diff changeset
11 directory pointed by the REPOS_DIR environment variable.
cff0f5926797 perf: add asv benchmarks
Philippe Pepiot <philippe.pepiot@logilab.fr>
parents:
diff changeset
12
cff0f5926797 perf: add asv benchmarks
Philippe Pepiot <philippe.pepiot@logilab.fr>
parents:
diff changeset
13 Invocation example:
cff0f5926797 perf: add asv benchmarks
Philippe Pepiot <philippe.pepiot@logilab.fr>
parents:
diff changeset
14
cff0f5926797 perf: add asv benchmarks
Philippe Pepiot <philippe.pepiot@logilab.fr>
parents:
diff changeset
15 $ export REPOS_DIR=~/hgperf/repos
cff0f5926797 perf: add asv benchmarks
Philippe Pepiot <philippe.pepiot@logilab.fr>
parents:
diff changeset
16 # run suite on given revision
cff0f5926797 perf: add asv benchmarks
Philippe Pepiot <philippe.pepiot@logilab.fr>
parents:
diff changeset
17 $ asv --config contrib/asv.conf.json run REV
cff0f5926797 perf: add asv benchmarks
Philippe Pepiot <philippe.pepiot@logilab.fr>
parents:
diff changeset
18 # run suite on new changesets found in stable and default branch
cff0f5926797 perf: add asv benchmarks
Philippe Pepiot <philippe.pepiot@logilab.fr>
parents:
diff changeset
19 $ asv --config contrib/asv.conf.json run NEW
cff0f5926797 perf: add asv benchmarks
Philippe Pepiot <philippe.pepiot@logilab.fr>
parents:
diff changeset
20 # display a comparative result table of benchmark results between two given
cff0f5926797 perf: add asv benchmarks
Philippe Pepiot <philippe.pepiot@logilab.fr>
parents:
diff changeset
21 # revisions
cff0f5926797 perf: add asv benchmarks
Philippe Pepiot <philippe.pepiot@logilab.fr>
parents:
diff changeset
22 $ asv --config contrib/asv.conf.json compare REV1 REV2
cff0f5926797 perf: add asv benchmarks
Philippe Pepiot <philippe.pepiot@logilab.fr>
parents:
diff changeset
23 # compute regression detection and generate ASV static website
cff0f5926797 perf: add asv benchmarks
Philippe Pepiot <philippe.pepiot@logilab.fr>
parents:
diff changeset
24 $ asv --config contrib/asv.conf.json publish
cff0f5926797 perf: add asv benchmarks
Philippe Pepiot <philippe.pepiot@logilab.fr>
parents:
diff changeset
25 # serve the static website
cff0f5926797 perf: add asv benchmarks
Philippe Pepiot <philippe.pepiot@logilab.fr>
parents:
diff changeset
26 $ asv --config contrib/asv.conf.json preview
cff0f5926797 perf: add asv benchmarks
Philippe Pepiot <philippe.pepiot@logilab.fr>
parents:
diff changeset
27 '''
cff0f5926797 perf: add asv benchmarks
Philippe Pepiot <philippe.pepiot@logilab.fr>
parents:
diff changeset
28
cff0f5926797 perf: add asv benchmarks
Philippe Pepiot <philippe.pepiot@logilab.fr>
parents:
diff changeset
29 from __future__ import absolute_import
cff0f5926797 perf: add asv benchmarks
Philippe Pepiot <philippe.pepiot@logilab.fr>
parents:
diff changeset
30
cff0f5926797 perf: add asv benchmarks
Philippe Pepiot <philippe.pepiot@logilab.fr>
parents:
diff changeset
31 import functools
cff0f5926797 perf: add asv benchmarks
Philippe Pepiot <philippe.pepiot@logilab.fr>
parents:
diff changeset
32 import os
cff0f5926797 perf: add asv benchmarks
Philippe Pepiot <philippe.pepiot@logilab.fr>
parents:
diff changeset
33 import re
cff0f5926797 perf: add asv benchmarks
Philippe Pepiot <philippe.pepiot@logilab.fr>
parents:
diff changeset
34
cff0f5926797 perf: add asv benchmarks
Philippe Pepiot <philippe.pepiot@logilab.fr>
parents:
diff changeset
35 from mercurial import (
cff0f5926797 perf: add asv benchmarks
Philippe Pepiot <philippe.pepiot@logilab.fr>
parents:
diff changeset
36 extensions,
cff0f5926797 perf: add asv benchmarks
Philippe Pepiot <philippe.pepiot@logilab.fr>
parents:
diff changeset
37 hg,
cff0f5926797 perf: add asv benchmarks
Philippe Pepiot <philippe.pepiot@logilab.fr>
parents:
diff changeset
38 ui as uimod,
cff0f5926797 perf: add asv benchmarks
Philippe Pepiot <philippe.pepiot@logilab.fr>
parents:
diff changeset
39 )
cff0f5926797 perf: add asv benchmarks
Philippe Pepiot <philippe.pepiot@logilab.fr>
parents:
diff changeset
40
cff0f5926797 perf: add asv benchmarks
Philippe Pepiot <philippe.pepiot@logilab.fr>
parents:
diff changeset
41 basedir = os.path.abspath(os.path.join(os.path.dirname(__file__),
cff0f5926797 perf: add asv benchmarks
Philippe Pepiot <philippe.pepiot@logilab.fr>
parents:
diff changeset
42 os.path.pardir, os.path.pardir))
cff0f5926797 perf: add asv benchmarks
Philippe Pepiot <philippe.pepiot@logilab.fr>
parents:
diff changeset
43 reposdir = os.environ['REPOS_DIR']
cff0f5926797 perf: add asv benchmarks
Philippe Pepiot <philippe.pepiot@logilab.fr>
parents:
diff changeset
44 reposnames = [name for name in os.listdir(reposdir)
cff0f5926797 perf: add asv benchmarks
Philippe Pepiot <philippe.pepiot@logilab.fr>
parents:
diff changeset
45 if os.path.isdir(os.path.join(reposdir, name, ".hg"))]
cff0f5926797 perf: add asv benchmarks
Philippe Pepiot <philippe.pepiot@logilab.fr>
parents:
diff changeset
46 if not reposnames:
cff0f5926797 perf: add asv benchmarks
Philippe Pepiot <philippe.pepiot@logilab.fr>
parents:
diff changeset
47 raise ValueError("No repositories found in $REPO_DIR")
cff0f5926797 perf: add asv benchmarks
Philippe Pepiot <philippe.pepiot@logilab.fr>
parents:
diff changeset
48 outputre = re.compile((r'! wall (\d+.\d+) comb \d+.\d+ user \d+.\d+ sys '
cff0f5926797 perf: add asv benchmarks
Philippe Pepiot <philippe.pepiot@logilab.fr>
parents:
diff changeset
49 r'\d+.\d+ \(best of \d+\)'))
cff0f5926797 perf: add asv benchmarks
Philippe Pepiot <philippe.pepiot@logilab.fr>
parents:
diff changeset
50
cff0f5926797 perf: add asv benchmarks
Philippe Pepiot <philippe.pepiot@logilab.fr>
parents:
diff changeset
51 def runperfcommand(reponame, command, *args, **kwargs):
cff0f5926797 perf: add asv benchmarks
Philippe Pepiot <philippe.pepiot@logilab.fr>
parents:
diff changeset
52 os.environ["HGRCPATH"] = os.environ.get("ASVHGRCPATH", "")
30559
d83ca854fa21 ui: factor out ui.load() to create a ui without loading configs (API)
Yuya Nishihara <yuya@tcha.org>
parents: 30406
diff changeset
53 ui = uimod.ui.load()
30406
cff0f5926797 perf: add asv benchmarks
Philippe Pepiot <philippe.pepiot@logilab.fr>
parents:
diff changeset
54 repo = hg.repository(ui, os.path.join(reposdir, reponame))
cff0f5926797 perf: add asv benchmarks
Philippe Pepiot <philippe.pepiot@logilab.fr>
parents:
diff changeset
55 perfext = extensions.load(ui, 'perfext',
cff0f5926797 perf: add asv benchmarks
Philippe Pepiot <philippe.pepiot@logilab.fr>
parents:
diff changeset
56 os.path.join(basedir, 'contrib', 'perf.py'))
cff0f5926797 perf: add asv benchmarks
Philippe Pepiot <philippe.pepiot@logilab.fr>
parents:
diff changeset
57 cmd = getattr(perfext, command)
cff0f5926797 perf: add asv benchmarks
Philippe Pepiot <philippe.pepiot@logilab.fr>
parents:
diff changeset
58 ui.pushbuffer()
cff0f5926797 perf: add asv benchmarks
Philippe Pepiot <philippe.pepiot@logilab.fr>
parents:
diff changeset
59 cmd(ui, repo, *args, **kwargs)
cff0f5926797 perf: add asv benchmarks
Philippe Pepiot <philippe.pepiot@logilab.fr>
parents:
diff changeset
60 output = ui.popbuffer()
cff0f5926797 perf: add asv benchmarks
Philippe Pepiot <philippe.pepiot@logilab.fr>
parents:
diff changeset
61 match = outputre.search(output)
cff0f5926797 perf: add asv benchmarks
Philippe Pepiot <philippe.pepiot@logilab.fr>
parents:
diff changeset
62 if not match:
cff0f5926797 perf: add asv benchmarks
Philippe Pepiot <philippe.pepiot@logilab.fr>
parents:
diff changeset
63 raise ValueError("Invalid output {0}".format(output))
cff0f5926797 perf: add asv benchmarks
Philippe Pepiot <philippe.pepiot@logilab.fr>
parents:
diff changeset
64 return float(match.group(1))
cff0f5926797 perf: add asv benchmarks
Philippe Pepiot <philippe.pepiot@logilab.fr>
parents:
diff changeset
65
cff0f5926797 perf: add asv benchmarks
Philippe Pepiot <philippe.pepiot@logilab.fr>
parents:
diff changeset
66 def perfbench(repos=reposnames, name=None, params=None):
cff0f5926797 perf: add asv benchmarks
Philippe Pepiot <philippe.pepiot@logilab.fr>
parents:
diff changeset
67 """decorator to declare ASV benchmark based on contrib/perf.py extension
cff0f5926797 perf: add asv benchmarks
Philippe Pepiot <philippe.pepiot@logilab.fr>
parents:
diff changeset
68
cff0f5926797 perf: add asv benchmarks
Philippe Pepiot <philippe.pepiot@logilab.fr>
parents:
diff changeset
69 An ASV benchmark is a python function with the given attributes:
cff0f5926797 perf: add asv benchmarks
Philippe Pepiot <philippe.pepiot@logilab.fr>
parents:
diff changeset
70
cff0f5926797 perf: add asv benchmarks
Philippe Pepiot <philippe.pepiot@logilab.fr>
parents:
diff changeset
71 __name__: should start with track_, time_ or mem_ to be collected by ASV
cff0f5926797 perf: add asv benchmarks
Philippe Pepiot <philippe.pepiot@logilab.fr>
parents:
diff changeset
72 params and param_name: parameter matrix to display multiple graphs on the
cff0f5926797 perf: add asv benchmarks
Philippe Pepiot <philippe.pepiot@logilab.fr>
parents:
diff changeset
73 same page.
cff0f5926797 perf: add asv benchmarks
Philippe Pepiot <philippe.pepiot@logilab.fr>
parents:
diff changeset
74 pretty_name: If defined it's displayed in web-ui instead of __name__
cff0f5926797 perf: add asv benchmarks
Philippe Pepiot <philippe.pepiot@logilab.fr>
parents:
diff changeset
75 (useful for revsets)
cff0f5926797 perf: add asv benchmarks
Philippe Pepiot <philippe.pepiot@logilab.fr>
parents:
diff changeset
76 the module name is prepended to the benchmark name and displayed as
cff0f5926797 perf: add asv benchmarks
Philippe Pepiot <philippe.pepiot@logilab.fr>
parents:
diff changeset
77 "category" in webui.
cff0f5926797 perf: add asv benchmarks
Philippe Pepiot <philippe.pepiot@logilab.fr>
parents:
diff changeset
78
cff0f5926797 perf: add asv benchmarks
Philippe Pepiot <philippe.pepiot@logilab.fr>
parents:
diff changeset
79 Benchmarks are automatically parameterized with repositories found in the
cff0f5926797 perf: add asv benchmarks
Philippe Pepiot <philippe.pepiot@logilab.fr>
parents:
diff changeset
80 REPOS_DIR environment variable.
cff0f5926797 perf: add asv benchmarks
Philippe Pepiot <philippe.pepiot@logilab.fr>
parents:
diff changeset
81
cff0f5926797 perf: add asv benchmarks
Philippe Pepiot <philippe.pepiot@logilab.fr>
parents:
diff changeset
82 `params` is the param matrix in the form of a list of tuple
cff0f5926797 perf: add asv benchmarks
Philippe Pepiot <philippe.pepiot@logilab.fr>
parents:
diff changeset
83 (param_name, [value0, value1])
cff0f5926797 perf: add asv benchmarks
Philippe Pepiot <philippe.pepiot@logilab.fr>
parents:
diff changeset
84
cff0f5926797 perf: add asv benchmarks
Philippe Pepiot <philippe.pepiot@logilab.fr>
parents:
diff changeset
85 For example [(x, [a, b]), (y, [c, d])] declare benchmarks for
cff0f5926797 perf: add asv benchmarks
Philippe Pepiot <philippe.pepiot@logilab.fr>
parents:
diff changeset
86 (a, c), (a, d), (b, c) and (b, d).
cff0f5926797 perf: add asv benchmarks
Philippe Pepiot <philippe.pepiot@logilab.fr>
parents:
diff changeset
87 """
cff0f5926797 perf: add asv benchmarks
Philippe Pepiot <philippe.pepiot@logilab.fr>
parents:
diff changeset
88 params = list(params or [])
cff0f5926797 perf: add asv benchmarks
Philippe Pepiot <philippe.pepiot@logilab.fr>
parents:
diff changeset
89 params.insert(0, ("repo", repos))
cff0f5926797 perf: add asv benchmarks
Philippe Pepiot <philippe.pepiot@logilab.fr>
parents:
diff changeset
90
cff0f5926797 perf: add asv benchmarks
Philippe Pepiot <philippe.pepiot@logilab.fr>
parents:
diff changeset
91 def decorator(func):
cff0f5926797 perf: add asv benchmarks
Philippe Pepiot <philippe.pepiot@logilab.fr>
parents:
diff changeset
92 @functools.wraps(func)
cff0f5926797 perf: add asv benchmarks
Philippe Pepiot <philippe.pepiot@logilab.fr>
parents:
diff changeset
93 def wrapped(repo, *args):
cff0f5926797 perf: add asv benchmarks
Philippe Pepiot <philippe.pepiot@logilab.fr>
parents:
diff changeset
94 def perf(command, *a, **kw):
cff0f5926797 perf: add asv benchmarks
Philippe Pepiot <philippe.pepiot@logilab.fr>
parents:
diff changeset
95 return runperfcommand(repo, command, *a, **kw)
cff0f5926797 perf: add asv benchmarks
Philippe Pepiot <philippe.pepiot@logilab.fr>
parents:
diff changeset
96 return func(perf, *args)
cff0f5926797 perf: add asv benchmarks
Philippe Pepiot <philippe.pepiot@logilab.fr>
parents:
diff changeset
97
cff0f5926797 perf: add asv benchmarks
Philippe Pepiot <philippe.pepiot@logilab.fr>
parents:
diff changeset
98 wrapped.params = [p[1] for p in params]
cff0f5926797 perf: add asv benchmarks
Philippe Pepiot <philippe.pepiot@logilab.fr>
parents:
diff changeset
99 wrapped.param_names = [p[0] for p in params]
cff0f5926797 perf: add asv benchmarks
Philippe Pepiot <philippe.pepiot@logilab.fr>
parents:
diff changeset
100 wrapped.pretty_name = name
cff0f5926797 perf: add asv benchmarks
Philippe Pepiot <philippe.pepiot@logilab.fr>
parents:
diff changeset
101 return wrapped
cff0f5926797 perf: add asv benchmarks
Philippe Pepiot <philippe.pepiot@logilab.fr>
parents:
diff changeset
102 return decorator