annotate hgext/fastannotate/support.py @ 39210:1ddb296e0dee

fastannotate: initial import from Facebook's hg-experimental I made as few changes as I could to get the tests to pass, but this was a bit involved due to some churn in the blame code since someone last gave fastannotate any TLC. There's still follow-up work here to rip out support for old versions of hg and to integrate the protocol with modern standards. Some performance numbers (all on my 2016 MacBook Pro with a 2.6Ghz i7): Mercurial mercurial/manifest.py traditional blame time: real 1.050 secs (user 0.990+0.000 sys 0.060+0.000) build cache time: real 5.900 secs (user 5.720+0.000 sys 0.110+0.000) fastannotate time: real 0.120 secs (user 0.100+0.000 sys 0.020+0.000) Mercurial mercurial/localrepo.py traditional blame time: real 3.330 secs (user 3.220+0.000 sys 0.070+0.000) build cache time: real 30.610 secs (user 30.190+0.000 sys 0.230+0.000) fastannotate time: real 0.180 secs (user 0.160+0.000 sys 0.020+0.000) mozilla-central dom/ipc/ContentParent.cpp traditional blame time: real 7.640 secs (user 7.210+0.000 sys 0.380+0.000) build cache time: real 98.650 secs (user 97.000+0.000 sys 0.950+0.000) fastannotate time: real 1.580 secs (user 1.340+0.000 sys 0.240+0.000) mozilla-central dom/base/nsDocument.cpp traditional blame time: real 17.110 secs (user 16.490+0.000 sys 0.500+0.000) build cache time: real 399.750 secs (user 394.520+0.000 sys 2.610+0.000) fastannotate time: real 1.780 secs (user 1.530+0.000 sys 0.240+0.000) So building the cache is expensive (but might be faster with xdiff enabled), but the blame results are *way* faster. Differential Revision: https://phab.mercurial-scm.org/D3994
author Augie Fackler <augie@google.com>
date Mon, 30 Jul 2018 22:50:00 -0400
parents
children 303dae0136b0
Ignore whitespace changes - Everywhere: Within whitespace: At end of lines:
rev   line source
39210
1ddb296e0dee fastannotate: initial import from Facebook's hg-experimental
Augie Fackler <augie@google.com>
parents:
diff changeset
1 # Copyright 2016-present Facebook. All Rights Reserved.
1ddb296e0dee fastannotate: initial import from Facebook's hg-experimental
Augie Fackler <augie@google.com>
parents:
diff changeset
2 #
1ddb296e0dee fastannotate: initial import from Facebook's hg-experimental
Augie Fackler <augie@google.com>
parents:
diff changeset
3 # support: fastannotate support for hgweb, and filectx
1ddb296e0dee fastannotate: initial import from Facebook's hg-experimental
Augie Fackler <augie@google.com>
parents:
diff changeset
4 #
1ddb296e0dee fastannotate: initial import from Facebook's hg-experimental
Augie Fackler <augie@google.com>
parents:
diff changeset
5 # This software may be used and distributed according to the terms of the
1ddb296e0dee fastannotate: initial import from Facebook's hg-experimental
Augie Fackler <augie@google.com>
parents:
diff changeset
6 # GNU General Public License version 2 or any later version.
1ddb296e0dee fastannotate: initial import from Facebook's hg-experimental
Augie Fackler <augie@google.com>
parents:
diff changeset
7
1ddb296e0dee fastannotate: initial import from Facebook's hg-experimental
Augie Fackler <augie@google.com>
parents:
diff changeset
8 from __future__ import absolute_import
1ddb296e0dee fastannotate: initial import from Facebook's hg-experimental
Augie Fackler <augie@google.com>
parents:
diff changeset
9
1ddb296e0dee fastannotate: initial import from Facebook's hg-experimental
Augie Fackler <augie@google.com>
parents:
diff changeset
10 from mercurial import (
1ddb296e0dee fastannotate: initial import from Facebook's hg-experimental
Augie Fackler <augie@google.com>
parents:
diff changeset
11 context as hgcontext,
1ddb296e0dee fastannotate: initial import from Facebook's hg-experimental
Augie Fackler <augie@google.com>
parents:
diff changeset
12 dagop,
1ddb296e0dee fastannotate: initial import from Facebook's hg-experimental
Augie Fackler <augie@google.com>
parents:
diff changeset
13 extensions,
1ddb296e0dee fastannotate: initial import from Facebook's hg-experimental
Augie Fackler <augie@google.com>
parents:
diff changeset
14 hgweb,
1ddb296e0dee fastannotate: initial import from Facebook's hg-experimental
Augie Fackler <augie@google.com>
parents:
diff changeset
15 patch,
1ddb296e0dee fastannotate: initial import from Facebook's hg-experimental
Augie Fackler <augie@google.com>
parents:
diff changeset
16 util,
1ddb296e0dee fastannotate: initial import from Facebook's hg-experimental
Augie Fackler <augie@google.com>
parents:
diff changeset
17 )
1ddb296e0dee fastannotate: initial import from Facebook's hg-experimental
Augie Fackler <augie@google.com>
parents:
diff changeset
18
1ddb296e0dee fastannotate: initial import from Facebook's hg-experimental
Augie Fackler <augie@google.com>
parents:
diff changeset
19 from . import (
1ddb296e0dee fastannotate: initial import from Facebook's hg-experimental
Augie Fackler <augie@google.com>
parents:
diff changeset
20 context,
1ddb296e0dee fastannotate: initial import from Facebook's hg-experimental
Augie Fackler <augie@google.com>
parents:
diff changeset
21 revmap,
1ddb296e0dee fastannotate: initial import from Facebook's hg-experimental
Augie Fackler <augie@google.com>
parents:
diff changeset
22 )
1ddb296e0dee fastannotate: initial import from Facebook's hg-experimental
Augie Fackler <augie@google.com>
parents:
diff changeset
23
1ddb296e0dee fastannotate: initial import from Facebook's hg-experimental
Augie Fackler <augie@google.com>
parents:
diff changeset
24 class _lazyfctx(object):
1ddb296e0dee fastannotate: initial import from Facebook's hg-experimental
Augie Fackler <augie@google.com>
parents:
diff changeset
25 """delegates to fctx but do not construct fctx when unnecessary"""
1ddb296e0dee fastannotate: initial import from Facebook's hg-experimental
Augie Fackler <augie@google.com>
parents:
diff changeset
26
1ddb296e0dee fastannotate: initial import from Facebook's hg-experimental
Augie Fackler <augie@google.com>
parents:
diff changeset
27 def __init__(self, repo, node, path):
1ddb296e0dee fastannotate: initial import from Facebook's hg-experimental
Augie Fackler <augie@google.com>
parents:
diff changeset
28 self._node = node
1ddb296e0dee fastannotate: initial import from Facebook's hg-experimental
Augie Fackler <augie@google.com>
parents:
diff changeset
29 self._path = path
1ddb296e0dee fastannotate: initial import from Facebook's hg-experimental
Augie Fackler <augie@google.com>
parents:
diff changeset
30 self._repo = repo
1ddb296e0dee fastannotate: initial import from Facebook's hg-experimental
Augie Fackler <augie@google.com>
parents:
diff changeset
31
1ddb296e0dee fastannotate: initial import from Facebook's hg-experimental
Augie Fackler <augie@google.com>
parents:
diff changeset
32 def node(self):
1ddb296e0dee fastannotate: initial import from Facebook's hg-experimental
Augie Fackler <augie@google.com>
parents:
diff changeset
33 return self._node
1ddb296e0dee fastannotate: initial import from Facebook's hg-experimental
Augie Fackler <augie@google.com>
parents:
diff changeset
34
1ddb296e0dee fastannotate: initial import from Facebook's hg-experimental
Augie Fackler <augie@google.com>
parents:
diff changeset
35 def path(self):
1ddb296e0dee fastannotate: initial import from Facebook's hg-experimental
Augie Fackler <augie@google.com>
parents:
diff changeset
36 return self._path
1ddb296e0dee fastannotate: initial import from Facebook's hg-experimental
Augie Fackler <augie@google.com>
parents:
diff changeset
37
1ddb296e0dee fastannotate: initial import from Facebook's hg-experimental
Augie Fackler <augie@google.com>
parents:
diff changeset
38 @util.propertycache
1ddb296e0dee fastannotate: initial import from Facebook's hg-experimental
Augie Fackler <augie@google.com>
parents:
diff changeset
39 def _fctx(self):
1ddb296e0dee fastannotate: initial import from Facebook's hg-experimental
Augie Fackler <augie@google.com>
parents:
diff changeset
40 return context.resolvefctx(self._repo, self._node, self._path)
1ddb296e0dee fastannotate: initial import from Facebook's hg-experimental
Augie Fackler <augie@google.com>
parents:
diff changeset
41
1ddb296e0dee fastannotate: initial import from Facebook's hg-experimental
Augie Fackler <augie@google.com>
parents:
diff changeset
42 def __getattr__(self, name):
1ddb296e0dee fastannotate: initial import from Facebook's hg-experimental
Augie Fackler <augie@google.com>
parents:
diff changeset
43 return getattr(self._fctx, name)
1ddb296e0dee fastannotate: initial import from Facebook's hg-experimental
Augie Fackler <augie@google.com>
parents:
diff changeset
44
1ddb296e0dee fastannotate: initial import from Facebook's hg-experimental
Augie Fackler <augie@google.com>
parents:
diff changeset
45 def _convertoutputs(repo, annotated, contents):
1ddb296e0dee fastannotate: initial import from Facebook's hg-experimental
Augie Fackler <augie@google.com>
parents:
diff changeset
46 """convert fastannotate outputs to vanilla annotate format"""
1ddb296e0dee fastannotate: initial import from Facebook's hg-experimental
Augie Fackler <augie@google.com>
parents:
diff changeset
47 # fastannotate returns: [(nodeid, linenum, path)], [linecontent]
1ddb296e0dee fastannotate: initial import from Facebook's hg-experimental
Augie Fackler <augie@google.com>
parents:
diff changeset
48 # convert to what fctx.annotate returns: [annotateline]
1ddb296e0dee fastannotate: initial import from Facebook's hg-experimental
Augie Fackler <augie@google.com>
parents:
diff changeset
49 results = []
1ddb296e0dee fastannotate: initial import from Facebook's hg-experimental
Augie Fackler <augie@google.com>
parents:
diff changeset
50 fctxmap = {}
1ddb296e0dee fastannotate: initial import from Facebook's hg-experimental
Augie Fackler <augie@google.com>
parents:
diff changeset
51 annotateline = dagop.annotateline
1ddb296e0dee fastannotate: initial import from Facebook's hg-experimental
Augie Fackler <augie@google.com>
parents:
diff changeset
52 for i, (hsh, linenum, path) in enumerate(annotated):
1ddb296e0dee fastannotate: initial import from Facebook's hg-experimental
Augie Fackler <augie@google.com>
parents:
diff changeset
53 if (hsh, path) not in fctxmap:
1ddb296e0dee fastannotate: initial import from Facebook's hg-experimental
Augie Fackler <augie@google.com>
parents:
diff changeset
54 fctxmap[(hsh, path)] = _lazyfctx(repo, hsh, path)
1ddb296e0dee fastannotate: initial import from Facebook's hg-experimental
Augie Fackler <augie@google.com>
parents:
diff changeset
55 # linenum: the user wants 1-based, we have 0-based.
1ddb296e0dee fastannotate: initial import from Facebook's hg-experimental
Augie Fackler <augie@google.com>
parents:
diff changeset
56 lineno = linenum + 1
1ddb296e0dee fastannotate: initial import from Facebook's hg-experimental
Augie Fackler <augie@google.com>
parents:
diff changeset
57 fctx = fctxmap[(hsh, path)]
1ddb296e0dee fastannotate: initial import from Facebook's hg-experimental
Augie Fackler <augie@google.com>
parents:
diff changeset
58 line = contents[i]
1ddb296e0dee fastannotate: initial import from Facebook's hg-experimental
Augie Fackler <augie@google.com>
parents:
diff changeset
59 results.append(annotateline(fctx=fctx, lineno=lineno, text=line))
1ddb296e0dee fastannotate: initial import from Facebook's hg-experimental
Augie Fackler <augie@google.com>
parents:
diff changeset
60 return results
1ddb296e0dee fastannotate: initial import from Facebook's hg-experimental
Augie Fackler <augie@google.com>
parents:
diff changeset
61
1ddb296e0dee fastannotate: initial import from Facebook's hg-experimental
Augie Fackler <augie@google.com>
parents:
diff changeset
62 def _getmaster(fctx):
1ddb296e0dee fastannotate: initial import from Facebook's hg-experimental
Augie Fackler <augie@google.com>
parents:
diff changeset
63 """(fctx) -> str"""
1ddb296e0dee fastannotate: initial import from Facebook's hg-experimental
Augie Fackler <augie@google.com>
parents:
diff changeset
64 return fctx._repo.ui.config('fastannotate', 'mainbranch') or 'default'
1ddb296e0dee fastannotate: initial import from Facebook's hg-experimental
Augie Fackler <augie@google.com>
parents:
diff changeset
65
1ddb296e0dee fastannotate: initial import from Facebook's hg-experimental
Augie Fackler <augie@google.com>
parents:
diff changeset
66 def _doannotate(fctx, follow=True, diffopts=None):
1ddb296e0dee fastannotate: initial import from Facebook's hg-experimental
Augie Fackler <augie@google.com>
parents:
diff changeset
67 """like the vanilla fctx.annotate, but do it via fastannotate, and make
1ddb296e0dee fastannotate: initial import from Facebook's hg-experimental
Augie Fackler <augie@google.com>
parents:
diff changeset
68 the output format compatible with the vanilla fctx.annotate.
1ddb296e0dee fastannotate: initial import from Facebook's hg-experimental
Augie Fackler <augie@google.com>
parents:
diff changeset
69 may raise Exception, and always return line numbers.
1ddb296e0dee fastannotate: initial import from Facebook's hg-experimental
Augie Fackler <augie@google.com>
parents:
diff changeset
70 """
1ddb296e0dee fastannotate: initial import from Facebook's hg-experimental
Augie Fackler <augie@google.com>
parents:
diff changeset
71 master = _getmaster(fctx)
1ddb296e0dee fastannotate: initial import from Facebook's hg-experimental
Augie Fackler <augie@google.com>
parents:
diff changeset
72 annotated = contents = None
1ddb296e0dee fastannotate: initial import from Facebook's hg-experimental
Augie Fackler <augie@google.com>
parents:
diff changeset
73
1ddb296e0dee fastannotate: initial import from Facebook's hg-experimental
Augie Fackler <augie@google.com>
parents:
diff changeset
74 with context.fctxannotatecontext(fctx, follow, diffopts) as ac:
1ddb296e0dee fastannotate: initial import from Facebook's hg-experimental
Augie Fackler <augie@google.com>
parents:
diff changeset
75 try:
1ddb296e0dee fastannotate: initial import from Facebook's hg-experimental
Augie Fackler <augie@google.com>
parents:
diff changeset
76 annotated, contents = ac.annotate(fctx.rev(), master=master,
1ddb296e0dee fastannotate: initial import from Facebook's hg-experimental
Augie Fackler <augie@google.com>
parents:
diff changeset
77 showpath=True, showlines=True)
1ddb296e0dee fastannotate: initial import from Facebook's hg-experimental
Augie Fackler <augie@google.com>
parents:
diff changeset
78 except Exception:
1ddb296e0dee fastannotate: initial import from Facebook's hg-experimental
Augie Fackler <augie@google.com>
parents:
diff changeset
79 ac.rebuild() # try rebuild once
1ddb296e0dee fastannotate: initial import from Facebook's hg-experimental
Augie Fackler <augie@google.com>
parents:
diff changeset
80 fctx._repo.ui.debug('fastannotate: %s: rebuilding broken cache\n'
1ddb296e0dee fastannotate: initial import from Facebook's hg-experimental
Augie Fackler <augie@google.com>
parents:
diff changeset
81 % fctx._path)
1ddb296e0dee fastannotate: initial import from Facebook's hg-experimental
Augie Fackler <augie@google.com>
parents:
diff changeset
82 try:
1ddb296e0dee fastannotate: initial import from Facebook's hg-experimental
Augie Fackler <augie@google.com>
parents:
diff changeset
83 annotated, contents = ac.annotate(fctx.rev(), master=master,
1ddb296e0dee fastannotate: initial import from Facebook's hg-experimental
Augie Fackler <augie@google.com>
parents:
diff changeset
84 showpath=True, showlines=True)
1ddb296e0dee fastannotate: initial import from Facebook's hg-experimental
Augie Fackler <augie@google.com>
parents:
diff changeset
85 except Exception:
1ddb296e0dee fastannotate: initial import from Facebook's hg-experimental
Augie Fackler <augie@google.com>
parents:
diff changeset
86 raise
1ddb296e0dee fastannotate: initial import from Facebook's hg-experimental
Augie Fackler <augie@google.com>
parents:
diff changeset
87
1ddb296e0dee fastannotate: initial import from Facebook's hg-experimental
Augie Fackler <augie@google.com>
parents:
diff changeset
88 assert annotated and contents
1ddb296e0dee fastannotate: initial import from Facebook's hg-experimental
Augie Fackler <augie@google.com>
parents:
diff changeset
89 return _convertoutputs(fctx._repo, annotated, contents)
1ddb296e0dee fastannotate: initial import from Facebook's hg-experimental
Augie Fackler <augie@google.com>
parents:
diff changeset
90
1ddb296e0dee fastannotate: initial import from Facebook's hg-experimental
Augie Fackler <augie@google.com>
parents:
diff changeset
91 def _hgwebannotate(orig, fctx, ui):
1ddb296e0dee fastannotate: initial import from Facebook's hg-experimental
Augie Fackler <augie@google.com>
parents:
diff changeset
92 diffopts = patch.difffeatureopts(ui, untrusted=True,
1ddb296e0dee fastannotate: initial import from Facebook's hg-experimental
Augie Fackler <augie@google.com>
parents:
diff changeset
93 section='annotate', whitespace=True)
1ddb296e0dee fastannotate: initial import from Facebook's hg-experimental
Augie Fackler <augie@google.com>
parents:
diff changeset
94 return _doannotate(fctx, diffopts=diffopts)
1ddb296e0dee fastannotate: initial import from Facebook's hg-experimental
Augie Fackler <augie@google.com>
parents:
diff changeset
95
1ddb296e0dee fastannotate: initial import from Facebook's hg-experimental
Augie Fackler <augie@google.com>
parents:
diff changeset
96 def _fctxannotate(orig, self, follow=False, linenumber=False, skiprevs=None,
1ddb296e0dee fastannotate: initial import from Facebook's hg-experimental
Augie Fackler <augie@google.com>
parents:
diff changeset
97 diffopts=None):
1ddb296e0dee fastannotate: initial import from Facebook's hg-experimental
Augie Fackler <augie@google.com>
parents:
diff changeset
98 if skiprevs:
1ddb296e0dee fastannotate: initial import from Facebook's hg-experimental
Augie Fackler <augie@google.com>
parents:
diff changeset
99 # skiprevs is not supported yet
1ddb296e0dee fastannotate: initial import from Facebook's hg-experimental
Augie Fackler <augie@google.com>
parents:
diff changeset
100 return orig(self, follow, linenumber, skiprevs=skiprevs,
1ddb296e0dee fastannotate: initial import from Facebook's hg-experimental
Augie Fackler <augie@google.com>
parents:
diff changeset
101 diffopts=diffopts)
1ddb296e0dee fastannotate: initial import from Facebook's hg-experimental
Augie Fackler <augie@google.com>
parents:
diff changeset
102 try:
1ddb296e0dee fastannotate: initial import from Facebook's hg-experimental
Augie Fackler <augie@google.com>
parents:
diff changeset
103 return _doannotate(self, follow, diffopts)
1ddb296e0dee fastannotate: initial import from Facebook's hg-experimental
Augie Fackler <augie@google.com>
parents:
diff changeset
104 except Exception as ex:
1ddb296e0dee fastannotate: initial import from Facebook's hg-experimental
Augie Fackler <augie@google.com>
parents:
diff changeset
105 self._repo.ui.debug('fastannotate: falling back to the vanilla '
1ddb296e0dee fastannotate: initial import from Facebook's hg-experimental
Augie Fackler <augie@google.com>
parents:
diff changeset
106 'annotate: %r\n' % ex)
1ddb296e0dee fastannotate: initial import from Facebook's hg-experimental
Augie Fackler <augie@google.com>
parents:
diff changeset
107 return orig(self, follow=follow, skiprevs=skiprevs,
1ddb296e0dee fastannotate: initial import from Facebook's hg-experimental
Augie Fackler <augie@google.com>
parents:
diff changeset
108 diffopts=diffopts)
1ddb296e0dee fastannotate: initial import from Facebook's hg-experimental
Augie Fackler <augie@google.com>
parents:
diff changeset
109
1ddb296e0dee fastannotate: initial import from Facebook's hg-experimental
Augie Fackler <augie@google.com>
parents:
diff changeset
110 def _remotefctxannotate(orig, self, follow=False, skiprevs=None, diffopts=None):
1ddb296e0dee fastannotate: initial import from Facebook's hg-experimental
Augie Fackler <augie@google.com>
parents:
diff changeset
111 # skipset: a set-like used to test if a fctx needs to be downloaded
1ddb296e0dee fastannotate: initial import from Facebook's hg-experimental
Augie Fackler <augie@google.com>
parents:
diff changeset
112 skipset = None
1ddb296e0dee fastannotate: initial import from Facebook's hg-experimental
Augie Fackler <augie@google.com>
parents:
diff changeset
113 with context.fctxannotatecontext(self, follow, diffopts) as ac:
1ddb296e0dee fastannotate: initial import from Facebook's hg-experimental
Augie Fackler <augie@google.com>
parents:
diff changeset
114 skipset = revmap.revmap(ac.revmappath)
1ddb296e0dee fastannotate: initial import from Facebook's hg-experimental
Augie Fackler <augie@google.com>
parents:
diff changeset
115 return orig(self, follow, skiprevs=skiprevs, diffopts=diffopts,
1ddb296e0dee fastannotate: initial import from Facebook's hg-experimental
Augie Fackler <augie@google.com>
parents:
diff changeset
116 prefetchskip=skipset)
1ddb296e0dee fastannotate: initial import from Facebook's hg-experimental
Augie Fackler <augie@google.com>
parents:
diff changeset
117
1ddb296e0dee fastannotate: initial import from Facebook's hg-experimental
Augie Fackler <augie@google.com>
parents:
diff changeset
118 def replacehgwebannotate():
1ddb296e0dee fastannotate: initial import from Facebook's hg-experimental
Augie Fackler <augie@google.com>
parents:
diff changeset
119 extensions.wrapfunction(hgweb.webutil, 'annotate', _hgwebannotate)
1ddb296e0dee fastannotate: initial import from Facebook's hg-experimental
Augie Fackler <augie@google.com>
parents:
diff changeset
120
1ddb296e0dee fastannotate: initial import from Facebook's hg-experimental
Augie Fackler <augie@google.com>
parents:
diff changeset
121 def replacefctxannotate():
1ddb296e0dee fastannotate: initial import from Facebook's hg-experimental
Augie Fackler <augie@google.com>
parents:
diff changeset
122 extensions.wrapfunction(hgcontext.basefilectx, 'annotate', _fctxannotate)
1ddb296e0dee fastannotate: initial import from Facebook's hg-experimental
Augie Fackler <augie@google.com>
parents:
diff changeset
123
1ddb296e0dee fastannotate: initial import from Facebook's hg-experimental
Augie Fackler <augie@google.com>
parents:
diff changeset
124 def replaceremotefctxannotate():
1ddb296e0dee fastannotate: initial import from Facebook's hg-experimental
Augie Fackler <augie@google.com>
parents:
diff changeset
125 try:
1ddb296e0dee fastannotate: initial import from Facebook's hg-experimental
Augie Fackler <augie@google.com>
parents:
diff changeset
126 r = extensions.find('remotefilelog')
1ddb296e0dee fastannotate: initial import from Facebook's hg-experimental
Augie Fackler <augie@google.com>
parents:
diff changeset
127 except KeyError:
1ddb296e0dee fastannotate: initial import from Facebook's hg-experimental
Augie Fackler <augie@google.com>
parents:
diff changeset
128 return
1ddb296e0dee fastannotate: initial import from Facebook's hg-experimental
Augie Fackler <augie@google.com>
parents:
diff changeset
129 else:
1ddb296e0dee fastannotate: initial import from Facebook's hg-experimental
Augie Fackler <augie@google.com>
parents:
diff changeset
130 extensions.wrapfunction(r.remotefilectx.remotefilectx, 'annotate',
1ddb296e0dee fastannotate: initial import from Facebook's hg-experimental
Augie Fackler <augie@google.com>
parents:
diff changeset
131 _remotefctxannotate)