Mercurial > public > mercurial-scm > hg-stable

annotate hgext/fastannotate/protocol.py @ 39731:d8a7690ccc74

Find changesets by keywords (author, files, the commit message), revision number or hash, or revset expression.
fastannotate: process files as they arrive peer.commandexecutor()'s context manager waits for all responses to arrive in its __exit__() method. We want to process the results as they arrive, so we should do that inside the context manager scope. Note that the futures' result() methods have been replaced to make sure that the command executor's sendcommands() method is called when the first future's result is requested, so we don't need to do that. A minor side-effect is that we can no longer easily tell when the server has started sending us responses, so that long statement was lost. Differential Revision: https://phab.mercurial-scm.org/D4666
author Martin von Zweigbergk <martinvonz@google.com>
date Wed, 19 Sep 2018 17:34:36 -0700
parents b220851999b5
children e40b7a504b1d
Ignore whitespace changes - Everywhere: Within whitespace: At end of lines:
rev   line source
39238
1ddb296e0dee fastannotate: initial import from Facebook's hg-experimental
Augie Fackler <augie@google.com>
parents:
diff changeset
1 # Copyright 2016-present Facebook. All Rights Reserved.
1ddb296e0dee fastannotate: initial import from Facebook's hg-experimental
Augie Fackler <augie@google.com>
parents:
diff changeset
2 #
1ddb296e0dee fastannotate: initial import from Facebook's hg-experimental
Augie Fackler <augie@google.com>
parents:
diff changeset
3 # protocol: logic for a server providing fastannotate support
1ddb296e0dee fastannotate: initial import from Facebook's hg-experimental
Augie Fackler <augie@google.com>
parents:
diff changeset
4 #
1ddb296e0dee fastannotate: initial import from Facebook's hg-experimental
Augie Fackler <augie@google.com>
parents:
diff changeset
5 # This software may be used and distributed according to the terms of the
1ddb296e0dee fastannotate: initial import from Facebook's hg-experimental
Augie Fackler <augie@google.com>
parents:
diff changeset
6 # GNU General Public License version 2 or any later version.
1ddb296e0dee fastannotate: initial import from Facebook's hg-experimental
Augie Fackler <augie@google.com>
parents:
diff changeset
7 from __future__ import absolute_import
1ddb296e0dee fastannotate: initial import from Facebook's hg-experimental
Augie Fackler <augie@google.com>
parents:
diff changeset
8
1ddb296e0dee fastannotate: initial import from Facebook's hg-experimental
Augie Fackler <augie@google.com>
parents:
diff changeset
9 import contextlib
1ddb296e0dee fastannotate: initial import from Facebook's hg-experimental
Augie Fackler <augie@google.com>
parents:
diff changeset
10 import os
1ddb296e0dee fastannotate: initial import from Facebook's hg-experimental
Augie Fackler <augie@google.com>
parents:
diff changeset
11
1ddb296e0dee fastannotate: initial import from Facebook's hg-experimental
Augie Fackler <augie@google.com>
parents:
diff changeset
12 from mercurial.i18n import _
1ddb296e0dee fastannotate: initial import from Facebook's hg-experimental
Augie Fackler <augie@google.com>
parents:
diff changeset
13 from mercurial import (
1ddb296e0dee fastannotate: initial import from Facebook's hg-experimental
Augie Fackler <augie@google.com>
parents:
diff changeset
14 error,
1ddb296e0dee fastannotate: initial import from Facebook's hg-experimental
Augie Fackler <augie@google.com>
parents:
diff changeset
15 extensions,
1ddb296e0dee fastannotate: initial import from Facebook's hg-experimental
Augie Fackler <augie@google.com>
parents:
diff changeset
16 hg,
39279
bb2b462f81da fastannotate: pconvert paths from the server for Windows
Matt Harbison <matt_harbison@yahoo.com>
parents: 39243
diff changeset
17 util,
39238
1ddb296e0dee fastannotate: initial import from Facebook's hg-experimental
Augie Fackler <augie@google.com>
parents:
diff changeset
18 wireprotov1peer,
1ddb296e0dee fastannotate: initial import from Facebook's hg-experimental
Augie Fackler <augie@google.com>
parents:
diff changeset
19 wireprotov1server,
1ddb296e0dee fastannotate: initial import from Facebook's hg-experimental
Augie Fackler <augie@google.com>
parents:
diff changeset
20 )
1ddb296e0dee fastannotate: initial import from Facebook's hg-experimental
Augie Fackler <augie@google.com>
parents:
diff changeset
21 from . import context
1ddb296e0dee fastannotate: initial import from Facebook's hg-experimental
Augie Fackler <augie@google.com>
parents:
diff changeset
22
1ddb296e0dee fastannotate: initial import from Facebook's hg-experimental
Augie Fackler <augie@google.com>
parents:
diff changeset
23 # common
1ddb296e0dee fastannotate: initial import from Facebook's hg-experimental
Augie Fackler <augie@google.com>
parents:
diff changeset
24
1ddb296e0dee fastannotate: initial import from Facebook's hg-experimental
Augie Fackler <augie@google.com>
parents:
diff changeset
25 def _getmaster(ui):
1ddb296e0dee fastannotate: initial import from Facebook's hg-experimental
Augie Fackler <augie@google.com>
parents:
diff changeset
26 """get the mainbranch, and enforce it is set"""
1ddb296e0dee fastannotate: initial import from Facebook's hg-experimental
Augie Fackler <augie@google.com>
parents:
diff changeset
27 master = ui.config('fastannotate', 'mainbranch')
1ddb296e0dee fastannotate: initial import from Facebook's hg-experimental
Augie Fackler <augie@google.com>
parents:
diff changeset
28 if not master:
1ddb296e0dee fastannotate: initial import from Facebook's hg-experimental
Augie Fackler <augie@google.com>
parents:
diff changeset
29 raise error.Abort(_('fastannotate.mainbranch is required '
1ddb296e0dee fastannotate: initial import from Facebook's hg-experimental
Augie Fackler <augie@google.com>
parents:
diff changeset
30 'for both the client and the server'))
1ddb296e0dee fastannotate: initial import from Facebook's hg-experimental
Augie Fackler <augie@google.com>
parents:
diff changeset
31 return master
1ddb296e0dee fastannotate: initial import from Facebook's hg-experimental
Augie Fackler <augie@google.com>
parents:
diff changeset
32
1ddb296e0dee fastannotate: initial import from Facebook's hg-experimental
Augie Fackler <augie@google.com>
parents:
diff changeset
33 # server-side
1ddb296e0dee fastannotate: initial import from Facebook's hg-experimental
Augie Fackler <augie@google.com>
parents:
diff changeset
34
1ddb296e0dee fastannotate: initial import from Facebook's hg-experimental
Augie Fackler <augie@google.com>
parents:
diff changeset
35 def _capabilities(orig, repo, proto):
1ddb296e0dee fastannotate: initial import from Facebook's hg-experimental
Augie Fackler <augie@google.com>
parents:
diff changeset
36 result = orig(repo, proto)
1ddb296e0dee fastannotate: initial import from Facebook's hg-experimental
Augie Fackler <augie@google.com>
parents:
diff changeset
37 result.append('getannotate')
1ddb296e0dee fastannotate: initial import from Facebook's hg-experimental
Augie Fackler <augie@google.com>
parents:
diff changeset
38 return result
1ddb296e0dee fastannotate: initial import from Facebook's hg-experimental
Augie Fackler <augie@google.com>
parents:
diff changeset
39
1ddb296e0dee fastannotate: initial import from Facebook's hg-experimental
Augie Fackler <augie@google.com>
parents:
diff changeset
40 def _getannotate(repo, proto, path, lastnode):
1ddb296e0dee fastannotate: initial import from Facebook's hg-experimental
Augie Fackler <augie@google.com>
parents:
diff changeset
41 # output:
1ddb296e0dee fastannotate: initial import from Facebook's hg-experimental
Augie Fackler <augie@google.com>
parents:
diff changeset
42 # FILE := vfspath + '\0' + str(size) + '\0' + content
1ddb296e0dee fastannotate: initial import from Facebook's hg-experimental
Augie Fackler <augie@google.com>
parents:
diff changeset
43 # OUTPUT := '' | FILE + OUTPUT
1ddb296e0dee fastannotate: initial import from Facebook's hg-experimental
Augie Fackler <augie@google.com>
parents:
diff changeset
44 result = ''
1ddb296e0dee fastannotate: initial import from Facebook's hg-experimental
Augie Fackler <augie@google.com>
parents:
diff changeset
45 buildondemand = repo.ui.configbool('fastannotate', 'serverbuildondemand',
1ddb296e0dee fastannotate: initial import from Facebook's hg-experimental
Augie Fackler <augie@google.com>
parents:
diff changeset
46 True)
1ddb296e0dee fastannotate: initial import from Facebook's hg-experimental
Augie Fackler <augie@google.com>
parents:
diff changeset
47 with context.annotatecontext(repo, path) as actx:
1ddb296e0dee fastannotate: initial import from Facebook's hg-experimental
Augie Fackler <augie@google.com>
parents:
diff changeset
48 if buildondemand:
1ddb296e0dee fastannotate: initial import from Facebook's hg-experimental
Augie Fackler <augie@google.com>
parents:
diff changeset
49 # update before responding to the client
1ddb296e0dee fastannotate: initial import from Facebook's hg-experimental
Augie Fackler <augie@google.com>
parents:
diff changeset
50 master = _getmaster(repo.ui)
1ddb296e0dee fastannotate: initial import from Facebook's hg-experimental
Augie Fackler <augie@google.com>
parents:
diff changeset
51 try:
1ddb296e0dee fastannotate: initial import from Facebook's hg-experimental
Augie Fackler <augie@google.com>
parents:
diff changeset
52 if not actx.isuptodate(master):
1ddb296e0dee fastannotate: initial import from Facebook's hg-experimental
Augie Fackler <augie@google.com>
parents:
diff changeset
53 actx.annotate(master, master)
1ddb296e0dee fastannotate: initial import from Facebook's hg-experimental
Augie Fackler <augie@google.com>
parents:
diff changeset
54 except Exception:
1ddb296e0dee fastannotate: initial import from Facebook's hg-experimental
Augie Fackler <augie@google.com>
parents:
diff changeset
55 # non-fast-forward move or corrupted. rebuild automically.
1ddb296e0dee fastannotate: initial import from Facebook's hg-experimental
Augie Fackler <augie@google.com>
parents:
diff changeset
56 actx.rebuild()
1ddb296e0dee fastannotate: initial import from Facebook's hg-experimental
Augie Fackler <augie@google.com>
parents:
diff changeset
57 try:
1ddb296e0dee fastannotate: initial import from Facebook's hg-experimental
Augie Fackler <augie@google.com>
parents:
diff changeset
58 actx.annotate(master, master)
1ddb296e0dee fastannotate: initial import from Facebook's hg-experimental
Augie Fackler <augie@google.com>
parents:
diff changeset
59 except Exception:
1ddb296e0dee fastannotate: initial import from Facebook's hg-experimental
Augie Fackler <augie@google.com>
parents:
diff changeset
60 actx.rebuild() # delete files
1ddb296e0dee fastannotate: initial import from Facebook's hg-experimental
Augie Fackler <augie@google.com>
parents:
diff changeset
61 finally:
1ddb296e0dee fastannotate: initial import from Facebook's hg-experimental
Augie Fackler <augie@google.com>
parents:
diff changeset
62 # although the "with" context will also do a close/flush, we
1ddb296e0dee fastannotate: initial import from Facebook's hg-experimental
Augie Fackler <augie@google.com>
parents:
diff changeset
63 # need to do it early so we can send the correct respond to
1ddb296e0dee fastannotate: initial import from Facebook's hg-experimental
Augie Fackler <augie@google.com>
parents:
diff changeset
64 # client.
1ddb296e0dee fastannotate: initial import from Facebook's hg-experimental
Augie Fackler <augie@google.com>
parents:
diff changeset
65 actx.close()
1ddb296e0dee fastannotate: initial import from Facebook's hg-experimental
Augie Fackler <augie@google.com>
parents:
diff changeset
66 # send back the full content of revmap and linelog, in the future we
1ddb296e0dee fastannotate: initial import from Facebook's hg-experimental
Augie Fackler <augie@google.com>
parents:
diff changeset
67 # may want to do some rsync-like fancy updating.
1ddb296e0dee fastannotate: initial import from Facebook's hg-experimental
Augie Fackler <augie@google.com>
parents:
diff changeset
68 # the lastnode check is not necessary if the client and the server
1ddb296e0dee fastannotate: initial import from Facebook's hg-experimental
Augie Fackler <augie@google.com>
parents:
diff changeset
69 # agree where the main branch is.
1ddb296e0dee fastannotate: initial import from Facebook's hg-experimental
Augie Fackler <augie@google.com>
parents:
diff changeset
70 if actx.lastnode != lastnode:
1ddb296e0dee fastannotate: initial import from Facebook's hg-experimental
Augie Fackler <augie@google.com>
parents:
diff changeset
71 for p in [actx.revmappath, actx.linelogpath]:
1ddb296e0dee fastannotate: initial import from Facebook's hg-experimental
Augie Fackler <augie@google.com>
parents:
diff changeset
72 if not os.path.exists(p):
1ddb296e0dee fastannotate: initial import from Facebook's hg-experimental
Augie Fackler <augie@google.com>
parents:
diff changeset
73 continue
1ddb296e0dee fastannotate: initial import from Facebook's hg-experimental
Augie Fackler <augie@google.com>
parents:
diff changeset
74 content = ''
1ddb296e0dee fastannotate: initial import from Facebook's hg-experimental
Augie Fackler <augie@google.com>
parents:
diff changeset
75 with open(p, 'rb') as f:
1ddb296e0dee fastannotate: initial import from Facebook's hg-experimental
Augie Fackler <augie@google.com>
parents:
diff changeset
76 content = f.read()
1ddb296e0dee fastannotate: initial import from Facebook's hg-experimental
Augie Fackler <augie@google.com>
parents:
diff changeset
77 vfsbaselen = len(repo.vfs.base + '/')
1ddb296e0dee fastannotate: initial import from Facebook's hg-experimental
Augie Fackler <augie@google.com>
parents:
diff changeset
78 relpath = p[vfsbaselen:]
39636
b220851999b5 py3: use '%d' for integers instead of '%s'
Pulkit Goyal <pulkit@yandex-team.ru>
parents: 39560
diff changeset
79 result += '%s\0%d\0%s' % (relpath, len(content), content)
39238
1ddb296e0dee fastannotate: initial import from Facebook's hg-experimental
Augie Fackler <augie@google.com>
parents:
diff changeset
80 return result
1ddb296e0dee fastannotate: initial import from Facebook's hg-experimental
Augie Fackler <augie@google.com>
parents:
diff changeset
81
1ddb296e0dee fastannotate: initial import from Facebook's hg-experimental
Augie Fackler <augie@google.com>
parents:
diff changeset
82 def _registerwireprotocommand():
1ddb296e0dee fastannotate: initial import from Facebook's hg-experimental
Augie Fackler <augie@google.com>
parents:
diff changeset
83 if 'getannotate' in wireprotov1server.commands:
1ddb296e0dee fastannotate: initial import from Facebook's hg-experimental
Augie Fackler <augie@google.com>
parents:
diff changeset
84 return
1ddb296e0dee fastannotate: initial import from Facebook's hg-experimental
Augie Fackler <augie@google.com>
parents:
diff changeset
85 wireprotov1server.wireprotocommand(
1ddb296e0dee fastannotate: initial import from Facebook's hg-experimental
Augie Fackler <augie@google.com>
parents:
diff changeset
86 'getannotate', 'path lastnode')(_getannotate)
1ddb296e0dee fastannotate: initial import from Facebook's hg-experimental
Augie Fackler <augie@google.com>
parents:
diff changeset
87
1ddb296e0dee fastannotate: initial import from Facebook's hg-experimental
Augie Fackler <augie@google.com>
parents:
diff changeset
88 def serveruisetup(ui):
1ddb296e0dee fastannotate: initial import from Facebook's hg-experimental
Augie Fackler <augie@google.com>
parents:
diff changeset
89 _registerwireprotocommand()
1ddb296e0dee fastannotate: initial import from Facebook's hg-experimental
Augie Fackler <augie@google.com>
parents:
diff changeset
90 extensions.wrapfunction(wireprotov1server, '_capabilities', _capabilities)
1ddb296e0dee fastannotate: initial import from Facebook's hg-experimental
Augie Fackler <augie@google.com>
parents:
diff changeset
91
1ddb296e0dee fastannotate: initial import from Facebook's hg-experimental
Augie Fackler <augie@google.com>
parents:
diff changeset
92 # client-side
1ddb296e0dee fastannotate: initial import from Facebook's hg-experimental
Augie Fackler <augie@google.com>
parents:
diff changeset
93
1ddb296e0dee fastannotate: initial import from Facebook's hg-experimental
Augie Fackler <augie@google.com>
parents:
diff changeset
94 def _parseresponse(payload):
1ddb296e0dee fastannotate: initial import from Facebook's hg-experimental
Augie Fackler <augie@google.com>
parents:
diff changeset
95 result = {}
1ddb296e0dee fastannotate: initial import from Facebook's hg-experimental
Augie Fackler <augie@google.com>
parents:
diff changeset
96 i = 0
1ddb296e0dee fastannotate: initial import from Facebook's hg-experimental
Augie Fackler <augie@google.com>
parents:
diff changeset
97 l = len(payload) - 1
1ddb296e0dee fastannotate: initial import from Facebook's hg-experimental
Augie Fackler <augie@google.com>
parents:
diff changeset
98 state = 0 # 0: vfspath, 1: size
1ddb296e0dee fastannotate: initial import from Facebook's hg-experimental
Augie Fackler <augie@google.com>
parents:
diff changeset
99 vfspath = size = ''
1ddb296e0dee fastannotate: initial import from Facebook's hg-experimental
Augie Fackler <augie@google.com>
parents:
diff changeset
100 while i < l:
1ddb296e0dee fastannotate: initial import from Facebook's hg-experimental
Augie Fackler <augie@google.com>
parents:
diff changeset
101 ch = payload[i]
1ddb296e0dee fastannotate: initial import from Facebook's hg-experimental
Augie Fackler <augie@google.com>
parents:
diff changeset
102 if ch == '\0':
1ddb296e0dee fastannotate: initial import from Facebook's hg-experimental
Augie Fackler <augie@google.com>
parents:
diff changeset
103 if state == 1:
1ddb296e0dee fastannotate: initial import from Facebook's hg-experimental
Augie Fackler <augie@google.com>
parents:
diff changeset
104 result[vfspath] = buffer(payload, i + 1, int(size))
1ddb296e0dee fastannotate: initial import from Facebook's hg-experimental
Augie Fackler <augie@google.com>
parents:
diff changeset
105 i += int(size)
1ddb296e0dee fastannotate: initial import from Facebook's hg-experimental
Augie Fackler <augie@google.com>
parents:
diff changeset
106 state = 0
1ddb296e0dee fastannotate: initial import from Facebook's hg-experimental
Augie Fackler <augie@google.com>
parents:
diff changeset
107 vfspath = size = ''
1ddb296e0dee fastannotate: initial import from Facebook's hg-experimental
Augie Fackler <augie@google.com>
parents:
diff changeset
108 elif state == 0:
1ddb296e0dee fastannotate: initial import from Facebook's hg-experimental
Augie Fackler <augie@google.com>
parents:
diff changeset
109 state = 1
1ddb296e0dee fastannotate: initial import from Facebook's hg-experimental
Augie Fackler <augie@google.com>
parents:
diff changeset
110 else:
1ddb296e0dee fastannotate: initial import from Facebook's hg-experimental
Augie Fackler <augie@google.com>
parents:
diff changeset
111 if state == 1:
1ddb296e0dee fastannotate: initial import from Facebook's hg-experimental
Augie Fackler <augie@google.com>
parents:
diff changeset
112 size += ch
1ddb296e0dee fastannotate: initial import from Facebook's hg-experimental
Augie Fackler <augie@google.com>
parents:
diff changeset
113 elif state == 0:
1ddb296e0dee fastannotate: initial import from Facebook's hg-experimental
Augie Fackler <augie@google.com>
parents:
diff changeset
114 vfspath += ch
1ddb296e0dee fastannotate: initial import from Facebook's hg-experimental
Augie Fackler <augie@google.com>
parents:
diff changeset
115 i += 1
1ddb296e0dee fastannotate: initial import from Facebook's hg-experimental
Augie Fackler <augie@google.com>
parents:
diff changeset
116 return result
1ddb296e0dee fastannotate: initial import from Facebook's hg-experimental
Augie Fackler <augie@google.com>
parents:
diff changeset
117
1ddb296e0dee fastannotate: initial import from Facebook's hg-experimental
Augie Fackler <augie@google.com>
parents:
diff changeset
118 def peersetup(ui, peer):
1ddb296e0dee fastannotate: initial import from Facebook's hg-experimental
Augie Fackler <augie@google.com>
parents:
diff changeset
119 class fastannotatepeer(peer.__class__):
1ddb296e0dee fastannotate: initial import from Facebook's hg-experimental
Augie Fackler <augie@google.com>
parents:
diff changeset
120 @wireprotov1peer.batchable
1ddb296e0dee fastannotate: initial import from Facebook's hg-experimental
Augie Fackler <augie@google.com>
parents:
diff changeset
121 def getannotate(self, path, lastnode=None):
1ddb296e0dee fastannotate: initial import from Facebook's hg-experimental
Augie Fackler <augie@google.com>
parents:
diff changeset
122 if not self.capable('getannotate'):
1ddb296e0dee fastannotate: initial import from Facebook's hg-experimental
Augie Fackler <augie@google.com>
parents:
diff changeset
123 ui.warn(_('remote peer cannot provide annotate cache\n'))
1ddb296e0dee fastannotate: initial import from Facebook's hg-experimental
Augie Fackler <augie@google.com>
parents:
diff changeset
124 yield None, None
1ddb296e0dee fastannotate: initial import from Facebook's hg-experimental
Augie Fackler <augie@google.com>
parents:
diff changeset
125 else:
1ddb296e0dee fastannotate: initial import from Facebook's hg-experimental
Augie Fackler <augie@google.com>
parents:
diff changeset
126 args = {'path': path, 'lastnode': lastnode or ''}
1ddb296e0dee fastannotate: initial import from Facebook's hg-experimental
Augie Fackler <augie@google.com>
parents:
diff changeset
127 f = wireprotov1peer.future()
1ddb296e0dee fastannotate: initial import from Facebook's hg-experimental
Augie Fackler <augie@google.com>
parents:
diff changeset
128 yield args, f
1ddb296e0dee fastannotate: initial import from Facebook's hg-experimental
Augie Fackler <augie@google.com>
parents:
diff changeset
129 yield _parseresponse(f.value)
1ddb296e0dee fastannotate: initial import from Facebook's hg-experimental
Augie Fackler <augie@google.com>
parents:
diff changeset
130 peer.__class__ = fastannotatepeer
1ddb296e0dee fastannotate: initial import from Facebook's hg-experimental
Augie Fackler <augie@google.com>
parents:
diff changeset
131
1ddb296e0dee fastannotate: initial import from Facebook's hg-experimental
Augie Fackler <augie@google.com>
parents:
diff changeset
132 @contextlib.contextmanager
1ddb296e0dee fastannotate: initial import from Facebook's hg-experimental
Augie Fackler <augie@google.com>
parents:
diff changeset
133 def annotatepeer(repo):
1ddb296e0dee fastannotate: initial import from Facebook's hg-experimental
Augie Fackler <augie@google.com>
parents:
diff changeset
134 ui = repo.ui
1ddb296e0dee fastannotate: initial import from Facebook's hg-experimental
Augie Fackler <augie@google.com>
parents:
diff changeset
135
39241
303dae0136b0 fastannotate: rip out specialized support for remotefilelog
Augie Fackler <augie@google.com>
parents: 39239
diff changeset
136 remotepath = ui.expandpath(
303dae0136b0 fastannotate: rip out specialized support for remotefilelog
Augie Fackler <augie@google.com>
parents: 39239
diff changeset
137 ui.config('fastannotate', 'remotepath', 'default'))
303dae0136b0 fastannotate: rip out specialized support for remotefilelog
Augie Fackler <augie@google.com>
parents: 39239
diff changeset
138 peer = hg.peer(ui, {}, remotepath)
39238
1ddb296e0dee fastannotate: initial import from Facebook's hg-experimental
Augie Fackler <augie@google.com>
parents:
diff changeset
139
1ddb296e0dee fastannotate: initial import from Facebook's hg-experimental
Augie Fackler <augie@google.com>
parents:
diff changeset
140 try:
1ddb296e0dee fastannotate: initial import from Facebook's hg-experimental
Augie Fackler <augie@google.com>
parents:
diff changeset
141 yield peer
1ddb296e0dee fastannotate: initial import from Facebook's hg-experimental
Augie Fackler <augie@google.com>
parents:
diff changeset
142 finally:
39241
303dae0136b0 fastannotate: rip out specialized support for remotefilelog
Augie Fackler <augie@google.com>
parents: 39239
diff changeset
143 peer.close()
39238
1ddb296e0dee fastannotate: initial import from Facebook's hg-experimental
Augie Fackler <augie@google.com>
parents:
diff changeset
144
1ddb296e0dee fastannotate: initial import from Facebook's hg-experimental
Augie Fackler <augie@google.com>
parents:
diff changeset
145 def clientfetch(repo, paths, lastnodemap=None, peer=None):
1ddb296e0dee fastannotate: initial import from Facebook's hg-experimental
Augie Fackler <augie@google.com>
parents:
diff changeset
146 """download annotate cache from the server for paths"""
1ddb296e0dee fastannotate: initial import from Facebook's hg-experimental
Augie Fackler <augie@google.com>
parents:
diff changeset
147 if not paths:
1ddb296e0dee fastannotate: initial import from Facebook's hg-experimental
Augie Fackler <augie@google.com>
parents:
diff changeset
148 return
1ddb296e0dee fastannotate: initial import from Facebook's hg-experimental
Augie Fackler <augie@google.com>
parents:
diff changeset
149
1ddb296e0dee fastannotate: initial import from Facebook's hg-experimental
Augie Fackler <augie@google.com>
parents:
diff changeset
150 if peer is None:
1ddb296e0dee fastannotate: initial import from Facebook's hg-experimental
Augie Fackler <augie@google.com>
parents:
diff changeset
151 with annotatepeer(repo) as peer:
1ddb296e0dee fastannotate: initial import from Facebook's hg-experimental
Augie Fackler <augie@google.com>
parents:
diff changeset
152 return clientfetch(repo, paths, lastnodemap, peer)
1ddb296e0dee fastannotate: initial import from Facebook's hg-experimental
Augie Fackler <augie@google.com>
parents:
diff changeset
153
1ddb296e0dee fastannotate: initial import from Facebook's hg-experimental
Augie Fackler <augie@google.com>
parents:
diff changeset
154 if lastnodemap is None:
1ddb296e0dee fastannotate: initial import from Facebook's hg-experimental
Augie Fackler <augie@google.com>
parents:
diff changeset
155 lastnodemap = {}
1ddb296e0dee fastannotate: initial import from Facebook's hg-experimental
Augie Fackler <augie@google.com>
parents:
diff changeset
156
1ddb296e0dee fastannotate: initial import from Facebook's hg-experimental
Augie Fackler <augie@google.com>
parents:
diff changeset
157 ui = repo.ui
1ddb296e0dee fastannotate: initial import from Facebook's hg-experimental
Augie Fackler <augie@google.com>
parents:
diff changeset
158 results = []
1ddb296e0dee fastannotate: initial import from Facebook's hg-experimental
Augie Fackler <augie@google.com>
parents:
diff changeset
159 with peer.commandexecutor() as batcher:
1ddb296e0dee fastannotate: initial import from Facebook's hg-experimental
Augie Fackler <augie@google.com>
parents:
diff changeset
160 ui.debug('fastannotate: requesting %d files\n' % len(paths))
1ddb296e0dee fastannotate: initial import from Facebook's hg-experimental
Augie Fackler <augie@google.com>
parents:
diff changeset
161 for p in paths:
1ddb296e0dee fastannotate: initial import from Facebook's hg-experimental
Augie Fackler <augie@google.com>
parents:
diff changeset
162 results.append(batcher.callcommand(
1ddb296e0dee fastannotate: initial import from Facebook's hg-experimental
Augie Fackler <augie@google.com>
parents:
diff changeset
163 'getannotate',
1ddb296e0dee fastannotate: initial import from Facebook's hg-experimental
Augie Fackler <augie@google.com>
parents:
diff changeset
164 {'path': p, 'lastnode':lastnodemap.get(p)}))
1ddb296e0dee fastannotate: initial import from Facebook's hg-experimental
Augie Fackler <augie@google.com>
parents:
diff changeset
165
39731
d8a7690ccc74 fastannotate: process files as they arrive
Martin von Zweigbergk <martinvonz@google.com>
parents: 39636
diff changeset
166 for result in results:
d8a7690ccc74 fastannotate: process files as they arrive
Martin von Zweigbergk <martinvonz@google.com>
parents: 39636
diff changeset
167 r = result.result()
d8a7690ccc74 fastannotate: process files as they arrive
Martin von Zweigbergk <martinvonz@google.com>
parents: 39636
diff changeset
168 # TODO: pconvert these paths on the server?
d8a7690ccc74 fastannotate: process files as they arrive
Martin von Zweigbergk <martinvonz@google.com>
parents: 39636
diff changeset
169 r = {util.pconvert(p): v for p, v in r.iteritems()}
d8a7690ccc74 fastannotate: process files as they arrive
Martin von Zweigbergk <martinvonz@google.com>
parents: 39636
diff changeset
170 for path in sorted(r):
d8a7690ccc74 fastannotate: process files as they arrive
Martin von Zweigbergk <martinvonz@google.com>
parents: 39636
diff changeset
171 # ignore malicious paths
d8a7690ccc74 fastannotate: process files as they arrive
Martin von Zweigbergk <martinvonz@google.com>
parents: 39636
diff changeset
172 if (not path.startswith('fastannotate/')
d8a7690ccc74 fastannotate: process files as they arrive
Martin von Zweigbergk <martinvonz@google.com>
parents: 39636
diff changeset
173 or '/../' in (path + '/')):
d8a7690ccc74 fastannotate: process files as they arrive
Martin von Zweigbergk <martinvonz@google.com>
parents: 39636
diff changeset
174 ui.debug('fastannotate: ignored malicious path %s\n' % path)
d8a7690ccc74 fastannotate: process files as they arrive
Martin von Zweigbergk <martinvonz@google.com>
parents: 39636
diff changeset
175 continue
d8a7690ccc74 fastannotate: process files as they arrive
Martin von Zweigbergk <martinvonz@google.com>
parents: 39636
diff changeset
176 content = r[path]
d8a7690ccc74 fastannotate: process files as they arrive
Martin von Zweigbergk <martinvonz@google.com>
parents: 39636
diff changeset
177 if ui.debugflag:
d8a7690ccc74 fastannotate: process files as they arrive
Martin von Zweigbergk <martinvonz@google.com>
parents: 39636
diff changeset
178 ui.debug('fastannotate: writing %d bytes to %s\n'
d8a7690ccc74 fastannotate: process files as they arrive
Martin von Zweigbergk <martinvonz@google.com>
parents: 39636
diff changeset
179 % (len(content), path))
d8a7690ccc74 fastannotate: process files as they arrive
Martin von Zweigbergk <martinvonz@google.com>
parents: 39636
diff changeset
180 repo.vfs.makedirs(os.path.dirname(path))
d8a7690ccc74 fastannotate: process files as they arrive
Martin von Zweigbergk <martinvonz@google.com>
parents: 39636
diff changeset
181 with repo.vfs(path, 'wb') as f:
d8a7690ccc74 fastannotate: process files as they arrive
Martin von Zweigbergk <martinvonz@google.com>
parents: 39636
diff changeset
182 f.write(content)
39238
1ddb296e0dee fastannotate: initial import from Facebook's hg-experimental
Augie Fackler <augie@google.com>
parents:
diff changeset
183
1ddb296e0dee fastannotate: initial import from Facebook's hg-experimental
Augie Fackler <augie@google.com>
parents:
diff changeset
184 def _filterfetchpaths(repo, paths):
1ddb296e0dee fastannotate: initial import from Facebook's hg-experimental
Augie Fackler <augie@google.com>
parents:
diff changeset
185 """return a subset of paths whose history is long and need to fetch linelog
1ddb296e0dee fastannotate: initial import from Facebook's hg-experimental
Augie Fackler <augie@google.com>
parents:
diff changeset
186 from the server. works with remotefilelog and non-remotefilelog repos.
1ddb296e0dee fastannotate: initial import from Facebook's hg-experimental
Augie Fackler <augie@google.com>
parents:
diff changeset
187 """
1ddb296e0dee fastannotate: initial import from Facebook's hg-experimental
Augie Fackler <augie@google.com>
parents:
diff changeset
188 threshold = repo.ui.configint('fastannotate', 'clientfetchthreshold', 10)
1ddb296e0dee fastannotate: initial import from Facebook's hg-experimental
Augie Fackler <augie@google.com>
parents:
diff changeset
189 if threshold <= 0:
1ddb296e0dee fastannotate: initial import from Facebook's hg-experimental
Augie Fackler <augie@google.com>
parents:
diff changeset
190 return paths
1ddb296e0dee fastannotate: initial import from Facebook's hg-experimental
Augie Fackler <augie@google.com>
parents:
diff changeset
191
1ddb296e0dee fastannotate: initial import from Facebook's hg-experimental
Augie Fackler <augie@google.com>
parents:
diff changeset
192 result = []
1ddb296e0dee fastannotate: initial import from Facebook's hg-experimental
Augie Fackler <augie@google.com>
parents:
diff changeset
193 for path in paths:
1ddb296e0dee fastannotate: initial import from Facebook's hg-experimental
Augie Fackler <augie@google.com>
parents:
diff changeset
194 try:
39241
303dae0136b0 fastannotate: rip out specialized support for remotefilelog
Augie Fackler <augie@google.com>
parents: 39239
diff changeset
195 if len(repo.file(path)) >= threshold:
39238
1ddb296e0dee fastannotate: initial import from Facebook's hg-experimental
Augie Fackler <augie@google.com>
parents:
diff changeset
196 result.append(path)
1ddb296e0dee fastannotate: initial import from Facebook's hg-experimental
Augie Fackler <augie@google.com>
parents:
diff changeset
197 except Exception: # file not found etc.
1ddb296e0dee fastannotate: initial import from Facebook's hg-experimental
Augie Fackler <augie@google.com>
parents:
diff changeset
198 result.append(path)
1ddb296e0dee fastannotate: initial import from Facebook's hg-experimental
Augie Fackler <augie@google.com>
parents:
diff changeset
199
1ddb296e0dee fastannotate: initial import from Facebook's hg-experimental
Augie Fackler <augie@google.com>
parents:
diff changeset
200 return result
1ddb296e0dee fastannotate: initial import from Facebook's hg-experimental
Augie Fackler <augie@google.com>
parents:
diff changeset
201
1ddb296e0dee fastannotate: initial import from Facebook's hg-experimental
Augie Fackler <augie@google.com>
parents:
diff changeset
202 def localreposetup(ui, repo):
1ddb296e0dee fastannotate: initial import from Facebook's hg-experimental
Augie Fackler <augie@google.com>
parents:
diff changeset
203 class fastannotaterepo(repo.__class__):
1ddb296e0dee fastannotate: initial import from Facebook's hg-experimental
Augie Fackler <augie@google.com>
parents:
diff changeset
204 def prefetchfastannotate(self, paths, peer=None):
1ddb296e0dee fastannotate: initial import from Facebook's hg-experimental
Augie Fackler <augie@google.com>
parents:
diff changeset
205 master = _getmaster(self.ui)
1ddb296e0dee fastannotate: initial import from Facebook's hg-experimental
Augie Fackler <augie@google.com>
parents:
diff changeset
206 needupdatepaths = []
1ddb296e0dee fastannotate: initial import from Facebook's hg-experimental
Augie Fackler <augie@google.com>
parents:
diff changeset
207 lastnodemap = {}
1ddb296e0dee fastannotate: initial import from Facebook's hg-experimental
Augie Fackler <augie@google.com>
parents:
diff changeset
208 try:
1ddb296e0dee fastannotate: initial import from Facebook's hg-experimental
Augie Fackler <augie@google.com>
parents:
diff changeset
209 for path in _filterfetchpaths(self, paths):
1ddb296e0dee fastannotate: initial import from Facebook's hg-experimental
Augie Fackler <augie@google.com>
parents:
diff changeset
210 with context.annotatecontext(self, path) as actx:
1ddb296e0dee fastannotate: initial import from Facebook's hg-experimental
Augie Fackler <augie@google.com>
parents:
diff changeset
211 if not actx.isuptodate(master, strict=False):
1ddb296e0dee fastannotate: initial import from Facebook's hg-experimental
Augie Fackler <augie@google.com>
parents:
diff changeset
212 needupdatepaths.append(path)
1ddb296e0dee fastannotate: initial import from Facebook's hg-experimental
Augie Fackler <augie@google.com>
parents:
diff changeset
213 lastnodemap[path] = actx.lastnode
1ddb296e0dee fastannotate: initial import from Facebook's hg-experimental
Augie Fackler <augie@google.com>
parents:
diff changeset
214 if needupdatepaths:
1ddb296e0dee fastannotate: initial import from Facebook's hg-experimental
Augie Fackler <augie@google.com>
parents:
diff changeset
215 clientfetch(self, needupdatepaths, lastnodemap, peer)
1ddb296e0dee fastannotate: initial import from Facebook's hg-experimental
Augie Fackler <augie@google.com>
parents:
diff changeset
216 except Exception as ex:
1ddb296e0dee fastannotate: initial import from Facebook's hg-experimental
Augie Fackler <augie@google.com>
parents:
diff changeset
217 # could be directory not writable or so, not fatal
1ddb296e0dee fastannotate: initial import from Facebook's hg-experimental
Augie Fackler <augie@google.com>
parents:
diff changeset
218 self.ui.debug('fastannotate: prefetch failed: %r\n' % ex)
1ddb296e0dee fastannotate: initial import from Facebook's hg-experimental
Augie Fackler <augie@google.com>
parents:
diff changeset
219 repo.__class__ = fastannotaterepo
1ddb296e0dee fastannotate: initial import from Facebook's hg-experimental
Augie Fackler <augie@google.com>
parents:
diff changeset
220
1ddb296e0dee fastannotate: initial import from Facebook's hg-experimental
Augie Fackler <augie@google.com>
parents:
diff changeset
221 def clientreposetup(ui, repo):
1ddb296e0dee fastannotate: initial import from Facebook's hg-experimental
Augie Fackler <augie@google.com>
parents:
diff changeset
222 _registerwireprotocommand()
39560
2182e67ea912 fastannotate: use repo.local()
Gregory Szorc <gregory.szorc@gmail.com>
parents: 39279
diff changeset
223 if repo.local():
39238
1ddb296e0dee fastannotate: initial import from Facebook's hg-experimental
Augie Fackler <augie@google.com>
parents:
diff changeset
224 localreposetup(ui, repo)
39242
8da20fc9fc07 fastannotate: move some global state mutation to extsetup()
Augie Fackler <augie@google.com>
parents: 39241
diff changeset
225 # TODO: this mutates global state, but only if at least one repo
8da20fc9fc07 fastannotate: move some global state mutation to extsetup()
Augie Fackler <augie@google.com>
parents: 39241
diff changeset
226 # has the extension enabled. This is probably bad for hgweb.
39238
1ddb296e0dee fastannotate: initial import from Facebook's hg-experimental
Augie Fackler <augie@google.com>
parents:
diff changeset
227 if peersetup not in hg.wirepeersetupfuncs:
1ddb296e0dee fastannotate: initial import from Facebook's hg-experimental
Augie Fackler <augie@google.com>
parents:
diff changeset
228 hg.wirepeersetupfuncs.append(peersetup)