Mercurial > public > mercurial-scm > hg-stable

annotate hgext/fastannotate/protocol.py @ 48578:28f0092ec89f

Find changesets by keywords (author, files, the commit message), revision number or hash, or revset expression.
exchange: add fast path for subrepo check on push Try to check if .hgsub and .hgsubstate exist at all before looking for them in every changeset to be pushed. The latter can be quite expensive for large repositories and the existance check is almost free. Differential Revision: https://phab.mercurial-scm.org/D11956
author Joerg Sonnenberger <joerg@bec.de>
date Mon, 03 Jan 2022 01:09:56 +0100
parents c424ff4807e6
children 6000f5b25c9b
Ignore whitespace changes - Everywhere: Within whitespace: At end of lines:
rev   line source
39238
1ddb296e0dee fastannotate: initial import from Facebook's hg-experimental
Augie Fackler <augie@google.com>
parents:
diff changeset
1 # Copyright 2016-present Facebook. All Rights Reserved.
1ddb296e0dee fastannotate: initial import from Facebook's hg-experimental
Augie Fackler <augie@google.com>
parents:
diff changeset
2 #
1ddb296e0dee fastannotate: initial import from Facebook's hg-experimental
Augie Fackler <augie@google.com>
parents:
diff changeset
3 # protocol: logic for a server providing fastannotate support
1ddb296e0dee fastannotate: initial import from Facebook's hg-experimental
Augie Fackler <augie@google.com>
parents:
diff changeset
4 #
1ddb296e0dee fastannotate: initial import from Facebook's hg-experimental
Augie Fackler <augie@google.com>
parents:
diff changeset
5 # This software may be used and distributed according to the terms of the
1ddb296e0dee fastannotate: initial import from Facebook's hg-experimental
Augie Fackler <augie@google.com>
parents:
diff changeset
6 # GNU General Public License version 2 or any later version.
1ddb296e0dee fastannotate: initial import from Facebook's hg-experimental
Augie Fackler <augie@google.com>
parents:
diff changeset
7 from __future__ import absolute_import
1ddb296e0dee fastannotate: initial import from Facebook's hg-experimental
Augie Fackler <augie@google.com>
parents:
diff changeset
8
1ddb296e0dee fastannotate: initial import from Facebook's hg-experimental
Augie Fackler <augie@google.com>
parents:
diff changeset
9 import contextlib
1ddb296e0dee fastannotate: initial import from Facebook's hg-experimental
Augie Fackler <augie@google.com>
parents:
diff changeset
10 import os
1ddb296e0dee fastannotate: initial import from Facebook's hg-experimental
Augie Fackler <augie@google.com>
parents:
diff changeset
11
1ddb296e0dee fastannotate: initial import from Facebook's hg-experimental
Augie Fackler <augie@google.com>
parents:
diff changeset
12 from mercurial.i18n import _
43085
eef9a2d67051 py3: manually import pycompat.open into files that need it
Gregory Szorc <gregory.szorc@gmail.com>
parents: 43077
diff changeset
13 from mercurial.pycompat import open
39238
1ddb296e0dee fastannotate: initial import from Facebook's hg-experimental
Augie Fackler <augie@google.com>
parents:
diff changeset
14 from mercurial import (
1ddb296e0dee fastannotate: initial import from Facebook's hg-experimental
Augie Fackler <augie@google.com>
parents:
diff changeset
15 error,
1ddb296e0dee fastannotate: initial import from Facebook's hg-experimental
Augie Fackler <augie@google.com>
parents:
diff changeset
16 extensions,
1ddb296e0dee fastannotate: initial import from Facebook's hg-experimental
Augie Fackler <augie@google.com>
parents:
diff changeset
17 hg,
43105
649d3ac37a12 py3: define and use pycompat.iteritems() for hgext/
Gregory Szorc <gregory.szorc@gmail.com>
parents: 43085
diff changeset
18 pycompat,
39279
bb2b462f81da fastannotate: pconvert paths from the server for Windows
Matt Harbison <matt_harbison@yahoo.com>
parents: 39243
diff changeset
19 util,
39238
1ddb296e0dee fastannotate: initial import from Facebook's hg-experimental
Augie Fackler <augie@google.com>
parents:
diff changeset
20 wireprotov1peer,
1ddb296e0dee fastannotate: initial import from Facebook's hg-experimental
Augie Fackler <augie@google.com>
parents:
diff changeset
21 wireprotov1server,
1ddb296e0dee fastannotate: initial import from Facebook's hg-experimental
Augie Fackler <augie@google.com>
parents:
diff changeset
22 )
46946
b6b696442a4d fastannotate: use `get_unique_pull_path`
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 43105
diff changeset
23 from mercurial.utils import (
b6b696442a4d fastannotate: use `get_unique_pull_path`
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 43105
diff changeset
24 urlutil,
b6b696442a4d fastannotate: use `get_unique_pull_path`
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 43105
diff changeset
25 )
39238
1ddb296e0dee fastannotate: initial import from Facebook's hg-experimental
Augie Fackler <augie@google.com>
parents:
diff changeset
26 from . import context
1ddb296e0dee fastannotate: initial import from Facebook's hg-experimental
Augie Fackler <augie@google.com>
parents:
diff changeset
27
1ddb296e0dee fastannotate: initial import from Facebook's hg-experimental
Augie Fackler <augie@google.com>
parents:
diff changeset
28 # common
1ddb296e0dee fastannotate: initial import from Facebook's hg-experimental
Augie Fackler <augie@google.com>
parents:
diff changeset
29
43076
2372284d9457 formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents: 41387
diff changeset
30
39238
1ddb296e0dee fastannotate: initial import from Facebook's hg-experimental
Augie Fackler <augie@google.com>
parents:
diff changeset
31 def _getmaster(ui):
1ddb296e0dee fastannotate: initial import from Facebook's hg-experimental
Augie Fackler <augie@google.com>
parents:
diff changeset
32 """get the mainbranch, and enforce it is set"""
43077
687b865b95ad formatting: byteify all mercurial/ and hgext/ string literals
Augie Fackler <augie@google.com>
parents: 43076
diff changeset
33 master = ui.config(b'fastannotate', b'mainbranch')
39238
1ddb296e0dee fastannotate: initial import from Facebook's hg-experimental
Augie Fackler <augie@google.com>
parents:
diff changeset
34 if not master:
43076
2372284d9457 formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents: 41387
diff changeset
35 raise error.Abort(
2372284d9457 formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents: 41387
diff changeset
36 _(
43077
687b865b95ad formatting: byteify all mercurial/ and hgext/ string literals
Augie Fackler <augie@google.com>
parents: 43076
diff changeset
37 b'fastannotate.mainbranch is required '
687b865b95ad formatting: byteify all mercurial/ and hgext/ string literals
Augie Fackler <augie@google.com>
parents: 43076
diff changeset
38 b'for both the client and the server'
43076
2372284d9457 formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents: 41387
diff changeset
39 )
2372284d9457 formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents: 41387
diff changeset
40 )
39238
1ddb296e0dee fastannotate: initial import from Facebook's hg-experimental
Augie Fackler <augie@google.com>
parents:
diff changeset
41 return master
1ddb296e0dee fastannotate: initial import from Facebook's hg-experimental
Augie Fackler <augie@google.com>
parents:
diff changeset
42
43076
2372284d9457 formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents: 41387
diff changeset
43
39238
1ddb296e0dee fastannotate: initial import from Facebook's hg-experimental
Augie Fackler <augie@google.com>
parents:
diff changeset
44 # server-side
1ddb296e0dee fastannotate: initial import from Facebook's hg-experimental
Augie Fackler <augie@google.com>
parents:
diff changeset
45
43076
2372284d9457 formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents: 41387
diff changeset
46
39238
1ddb296e0dee fastannotate: initial import from Facebook's hg-experimental
Augie Fackler <augie@google.com>
parents:
diff changeset
47 def _capabilities(orig, repo, proto):
1ddb296e0dee fastannotate: initial import from Facebook's hg-experimental
Augie Fackler <augie@google.com>
parents:
diff changeset
48 result = orig(repo, proto)
43077
687b865b95ad formatting: byteify all mercurial/ and hgext/ string literals
Augie Fackler <augie@google.com>
parents: 43076
diff changeset
49 result.append(b'getannotate')
39238
1ddb296e0dee fastannotate: initial import from Facebook's hg-experimental
Augie Fackler <augie@google.com>
parents:
diff changeset
50 return result
1ddb296e0dee fastannotate: initial import from Facebook's hg-experimental
Augie Fackler <augie@google.com>
parents:
diff changeset
51
43076
2372284d9457 formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents: 41387
diff changeset
52
39238
1ddb296e0dee fastannotate: initial import from Facebook's hg-experimental
Augie Fackler <augie@google.com>
parents:
diff changeset
53 def _getannotate(repo, proto, path, lastnode):
1ddb296e0dee fastannotate: initial import from Facebook's hg-experimental
Augie Fackler <augie@google.com>
parents:
diff changeset
54 # output:
1ddb296e0dee fastannotate: initial import from Facebook's hg-experimental
Augie Fackler <augie@google.com>
parents:
diff changeset
55 # FILE := vfspath + '\0' + str(size) + '\0' + content
1ddb296e0dee fastannotate: initial import from Facebook's hg-experimental
Augie Fackler <augie@google.com>
parents:
diff changeset
56 # OUTPUT := '' | FILE + OUTPUT
43077
687b865b95ad formatting: byteify all mercurial/ and hgext/ string literals
Augie Fackler <augie@google.com>
parents: 43076
diff changeset
57 result = b''
43076
2372284d9457 formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents: 41387
diff changeset
58 buildondemand = repo.ui.configbool(
43077
687b865b95ad formatting: byteify all mercurial/ and hgext/ string literals
Augie Fackler <augie@google.com>
parents: 43076
diff changeset
59 b'fastannotate', b'serverbuildondemand', True
43076
2372284d9457 formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents: 41387
diff changeset
60 )
39238
1ddb296e0dee fastannotate: initial import from Facebook's hg-experimental
Augie Fackler <augie@google.com>
parents:
diff changeset
61 with context.annotatecontext(repo, path) as actx:
1ddb296e0dee fastannotate: initial import from Facebook's hg-experimental
Augie Fackler <augie@google.com>
parents:
diff changeset
62 if buildondemand:
1ddb296e0dee fastannotate: initial import from Facebook's hg-experimental
Augie Fackler <augie@google.com>
parents:
diff changeset
63 # update before responding to the client
1ddb296e0dee fastannotate: initial import from Facebook's hg-experimental
Augie Fackler <augie@google.com>
parents:
diff changeset
64 master = _getmaster(repo.ui)
1ddb296e0dee fastannotate: initial import from Facebook's hg-experimental
Augie Fackler <augie@google.com>
parents:
diff changeset
65 try:
1ddb296e0dee fastannotate: initial import from Facebook's hg-experimental
Augie Fackler <augie@google.com>
parents:
diff changeset
66 if not actx.isuptodate(master):
1ddb296e0dee fastannotate: initial import from Facebook's hg-experimental
Augie Fackler <augie@google.com>
parents:
diff changeset
67 actx.annotate(master, master)
1ddb296e0dee fastannotate: initial import from Facebook's hg-experimental
Augie Fackler <augie@google.com>
parents:
diff changeset
68 except Exception:
1ddb296e0dee fastannotate: initial import from Facebook's hg-experimental
Augie Fackler <augie@google.com>
parents:
diff changeset
69 # non-fast-forward move or corrupted. rebuild automically.
1ddb296e0dee fastannotate: initial import from Facebook's hg-experimental
Augie Fackler <augie@google.com>
parents:
diff changeset
70 actx.rebuild()
1ddb296e0dee fastannotate: initial import from Facebook's hg-experimental
Augie Fackler <augie@google.com>
parents:
diff changeset
71 try:
1ddb296e0dee fastannotate: initial import from Facebook's hg-experimental
Augie Fackler <augie@google.com>
parents:
diff changeset
72 actx.annotate(master, master)
1ddb296e0dee fastannotate: initial import from Facebook's hg-experimental
Augie Fackler <augie@google.com>
parents:
diff changeset
73 except Exception:
43076
2372284d9457 formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents: 41387
diff changeset
74 actx.rebuild() # delete files
39238
1ddb296e0dee fastannotate: initial import from Facebook's hg-experimental
Augie Fackler <augie@google.com>
parents:
diff changeset
75 finally:
1ddb296e0dee fastannotate: initial import from Facebook's hg-experimental
Augie Fackler <augie@google.com>
parents:
diff changeset
76 # although the "with" context will also do a close/flush, we
1ddb296e0dee fastannotate: initial import from Facebook's hg-experimental
Augie Fackler <augie@google.com>
parents:
diff changeset
77 # need to do it early so we can send the correct respond to
1ddb296e0dee fastannotate: initial import from Facebook's hg-experimental
Augie Fackler <augie@google.com>
parents:
diff changeset
78 # client.
1ddb296e0dee fastannotate: initial import from Facebook's hg-experimental
Augie Fackler <augie@google.com>
parents:
diff changeset
79 actx.close()
1ddb296e0dee fastannotate: initial import from Facebook's hg-experimental
Augie Fackler <augie@google.com>
parents:
diff changeset
80 # send back the full content of revmap and linelog, in the future we
1ddb296e0dee fastannotate: initial import from Facebook's hg-experimental
Augie Fackler <augie@google.com>
parents:
diff changeset
81 # may want to do some rsync-like fancy updating.
1ddb296e0dee fastannotate: initial import from Facebook's hg-experimental
Augie Fackler <augie@google.com>
parents:
diff changeset
82 # the lastnode check is not necessary if the client and the server
1ddb296e0dee fastannotate: initial import from Facebook's hg-experimental
Augie Fackler <augie@google.com>
parents:
diff changeset
83 # agree where the main branch is.
1ddb296e0dee fastannotate: initial import from Facebook's hg-experimental
Augie Fackler <augie@google.com>
parents:
diff changeset
84 if actx.lastnode != lastnode:
1ddb296e0dee fastannotate: initial import from Facebook's hg-experimental
Augie Fackler <augie@google.com>
parents:
diff changeset
85 for p in [actx.revmappath, actx.linelogpath]:
1ddb296e0dee fastannotate: initial import from Facebook's hg-experimental
Augie Fackler <augie@google.com>
parents:
diff changeset
86 if not os.path.exists(p):
1ddb296e0dee fastannotate: initial import from Facebook's hg-experimental
Augie Fackler <augie@google.com>
parents:
diff changeset
87 continue
43077
687b865b95ad formatting: byteify all mercurial/ and hgext/ string literals
Augie Fackler <augie@google.com>
parents: 43076
diff changeset
88 with open(p, b'rb') as f:
39238
1ddb296e0dee fastannotate: initial import from Facebook's hg-experimental
Augie Fackler <augie@google.com>
parents:
diff changeset
89 content = f.read()
43077
687b865b95ad formatting: byteify all mercurial/ and hgext/ string literals
Augie Fackler <augie@google.com>
parents: 43076
diff changeset
90 vfsbaselen = len(repo.vfs.base + b'/')
39238
1ddb296e0dee fastannotate: initial import from Facebook's hg-experimental
Augie Fackler <augie@google.com>
parents:
diff changeset
91 relpath = p[vfsbaselen:]
43077
687b865b95ad formatting: byteify all mercurial/ and hgext/ string literals
Augie Fackler <augie@google.com>
parents: 43076
diff changeset
92 result += b'%s\0%d\0%s' % (relpath, len(content), content)
39238
1ddb296e0dee fastannotate: initial import from Facebook's hg-experimental
Augie Fackler <augie@google.com>
parents:
diff changeset
93 return result
1ddb296e0dee fastannotate: initial import from Facebook's hg-experimental
Augie Fackler <augie@google.com>
parents:
diff changeset
94
43076
2372284d9457 formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents: 41387
diff changeset
95
39238
1ddb296e0dee fastannotate: initial import from Facebook's hg-experimental
Augie Fackler <augie@google.com>
parents:
diff changeset
96 def _registerwireprotocommand():
43077
687b865b95ad formatting: byteify all mercurial/ and hgext/ string literals
Augie Fackler <augie@google.com>
parents: 43076
diff changeset
97 if b'getannotate' in wireprotov1server.commands:
39238
1ddb296e0dee fastannotate: initial import from Facebook's hg-experimental
Augie Fackler <augie@google.com>
parents:
diff changeset
98 return
43077
687b865b95ad formatting: byteify all mercurial/ and hgext/ string literals
Augie Fackler <augie@google.com>
parents: 43076
diff changeset
99 wireprotov1server.wireprotocommand(b'getannotate', b'path lastnode')(
43076
2372284d9457 formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents: 41387
diff changeset
100 _getannotate
2372284d9457 formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents: 41387
diff changeset
101 )
2372284d9457 formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents: 41387
diff changeset
102
39238
1ddb296e0dee fastannotate: initial import from Facebook's hg-experimental
Augie Fackler <augie@google.com>
parents:
diff changeset
103
1ddb296e0dee fastannotate: initial import from Facebook's hg-experimental
Augie Fackler <augie@google.com>
parents:
diff changeset
104 def serveruisetup(ui):
1ddb296e0dee fastannotate: initial import from Facebook's hg-experimental
Augie Fackler <augie@google.com>
parents:
diff changeset
105 _registerwireprotocommand()
43077
687b865b95ad formatting: byteify all mercurial/ and hgext/ string literals
Augie Fackler <augie@google.com>
parents: 43076
diff changeset
106 extensions.wrapfunction(wireprotov1server, b'_capabilities', _capabilities)
39238
1ddb296e0dee fastannotate: initial import from Facebook's hg-experimental
Augie Fackler <augie@google.com>
parents:
diff changeset
107
43076
2372284d9457 formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents: 41387
diff changeset
108
39238
1ddb296e0dee fastannotate: initial import from Facebook's hg-experimental
Augie Fackler <augie@google.com>
parents:
diff changeset
109 # client-side
1ddb296e0dee fastannotate: initial import from Facebook's hg-experimental
Augie Fackler <augie@google.com>
parents:
diff changeset
110
43076
2372284d9457 formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents: 41387
diff changeset
111
39238
1ddb296e0dee fastannotate: initial import from Facebook's hg-experimental
Augie Fackler <augie@google.com>
parents:
diff changeset
112 def _parseresponse(payload):
1ddb296e0dee fastannotate: initial import from Facebook's hg-experimental
Augie Fackler <augie@google.com>
parents:
diff changeset
113 result = {}
1ddb296e0dee fastannotate: initial import from Facebook's hg-experimental
Augie Fackler <augie@google.com>
parents:
diff changeset
114 i = 0
1ddb296e0dee fastannotate: initial import from Facebook's hg-experimental
Augie Fackler <augie@google.com>
parents:
diff changeset
115 l = len(payload) - 1
43076
2372284d9457 formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents: 41387
diff changeset
116 state = 0 # 0: vfspath, 1: size
43077
687b865b95ad formatting: byteify all mercurial/ and hgext/ string literals
Augie Fackler <augie@google.com>
parents: 43076
diff changeset
117 vfspath = size = b''
39238
1ddb296e0dee fastannotate: initial import from Facebook's hg-experimental
Augie Fackler <augie@google.com>
parents:
diff changeset
118 while i < l:
43076
2372284d9457 formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents: 41387
diff changeset
119 ch = payload[i : i + 1]
43077
687b865b95ad formatting: byteify all mercurial/ and hgext/ string literals
Augie Fackler <augie@google.com>
parents: 43076
diff changeset
120 if ch == b'\0':
39238
1ddb296e0dee fastannotate: initial import from Facebook's hg-experimental
Augie Fackler <augie@google.com>
parents:
diff changeset
121 if state == 1:
43076
2372284d9457 formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents: 41387
diff changeset
122 result[vfspath] = payload[i + 1 : i + 1 + int(size)]
39238
1ddb296e0dee fastannotate: initial import from Facebook's hg-experimental
Augie Fackler <augie@google.com>
parents:
diff changeset
123 i += int(size)
1ddb296e0dee fastannotate: initial import from Facebook's hg-experimental
Augie Fackler <augie@google.com>
parents:
diff changeset
124 state = 0
43077
687b865b95ad formatting: byteify all mercurial/ and hgext/ string literals
Augie Fackler <augie@google.com>
parents: 43076
diff changeset
125 vfspath = size = b''
39238
1ddb296e0dee fastannotate: initial import from Facebook's hg-experimental
Augie Fackler <augie@google.com>
parents:
diff changeset
126 elif state == 0:
1ddb296e0dee fastannotate: initial import from Facebook's hg-experimental
Augie Fackler <augie@google.com>
parents:
diff changeset
127 state = 1
1ddb296e0dee fastannotate: initial import from Facebook's hg-experimental
Augie Fackler <augie@google.com>
parents:
diff changeset
128 else:
1ddb296e0dee fastannotate: initial import from Facebook's hg-experimental
Augie Fackler <augie@google.com>
parents:
diff changeset
129 if state == 1:
1ddb296e0dee fastannotate: initial import from Facebook's hg-experimental
Augie Fackler <augie@google.com>
parents:
diff changeset
130 size += ch
1ddb296e0dee fastannotate: initial import from Facebook's hg-experimental
Augie Fackler <augie@google.com>
parents:
diff changeset
131 elif state == 0:
1ddb296e0dee fastannotate: initial import from Facebook's hg-experimental
Augie Fackler <augie@google.com>
parents:
diff changeset
132 vfspath += ch
1ddb296e0dee fastannotate: initial import from Facebook's hg-experimental
Augie Fackler <augie@google.com>
parents:
diff changeset
133 i += 1
1ddb296e0dee fastannotate: initial import from Facebook's hg-experimental
Augie Fackler <augie@google.com>
parents:
diff changeset
134 return result
1ddb296e0dee fastannotate: initial import from Facebook's hg-experimental
Augie Fackler <augie@google.com>
parents:
diff changeset
135
43076
2372284d9457 formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents: 41387
diff changeset
136
39238
1ddb296e0dee fastannotate: initial import from Facebook's hg-experimental
Augie Fackler <augie@google.com>
parents:
diff changeset
137 def peersetup(ui, peer):
1ddb296e0dee fastannotate: initial import from Facebook's hg-experimental
Augie Fackler <augie@google.com>
parents:
diff changeset
138 class fastannotatepeer(peer.__class__):
1ddb296e0dee fastannotate: initial import from Facebook's hg-experimental
Augie Fackler <augie@google.com>
parents:
diff changeset
139 @wireprotov1peer.batchable
1ddb296e0dee fastannotate: initial import from Facebook's hg-experimental
Augie Fackler <augie@google.com>
parents:
diff changeset
140 def getannotate(self, path, lastnode=None):
43077
687b865b95ad formatting: byteify all mercurial/ and hgext/ string literals
Augie Fackler <augie@google.com>
parents: 43076
diff changeset
141 if not self.capable(b'getannotate'):
687b865b95ad formatting: byteify all mercurial/ and hgext/ string literals
Augie Fackler <augie@google.com>
parents: 43076
diff changeset
142 ui.warn(_(b'remote peer cannot provide annotate cache\n'))
47914
c424ff4807e6 wireprotov1peer: update all rpcs to use the new batchable scheme
Valentin Gatien-Baron <valentin.gatienbaron@gmail.com>
parents: 46946
diff changeset
143 return None, None
39238
1ddb296e0dee fastannotate: initial import from Facebook's hg-experimental
Augie Fackler <augie@google.com>
parents:
diff changeset
144 else:
43077
687b865b95ad formatting: byteify all mercurial/ and hgext/ string literals
Augie Fackler <augie@google.com>
parents: 43076
diff changeset
145 args = {b'path': path, b'lastnode': lastnode or b''}
47914
c424ff4807e6 wireprotov1peer: update all rpcs to use the new batchable scheme
Valentin Gatien-Baron <valentin.gatienbaron@gmail.com>
parents: 46946
diff changeset
146 return args, _parseresponse
43076
2372284d9457 formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents: 41387
diff changeset
147
39238
1ddb296e0dee fastannotate: initial import from Facebook's hg-experimental
Augie Fackler <augie@google.com>
parents:
diff changeset
148 peer.__class__ = fastannotatepeer
1ddb296e0dee fastannotate: initial import from Facebook's hg-experimental
Augie Fackler <augie@google.com>
parents:
diff changeset
149
43076
2372284d9457 formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents: 41387
diff changeset
150
39238
1ddb296e0dee fastannotate: initial import from Facebook's hg-experimental
Augie Fackler <augie@google.com>
parents:
diff changeset
151 @contextlib.contextmanager
1ddb296e0dee fastannotate: initial import from Facebook's hg-experimental
Augie Fackler <augie@google.com>
parents:
diff changeset
152 def annotatepeer(repo):
1ddb296e0dee fastannotate: initial import from Facebook's hg-experimental
Augie Fackler <augie@google.com>
parents:
diff changeset
153 ui = repo.ui
1ddb296e0dee fastannotate: initial import from Facebook's hg-experimental
Augie Fackler <augie@google.com>
parents:
diff changeset
154
46946
b6b696442a4d fastannotate: use `get_unique_pull_path`
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 43105
diff changeset
155 remotedest = ui.config(b'fastannotate', b'remotepath', b'default')
b6b696442a4d fastannotate: use `get_unique_pull_path`
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 43105
diff changeset
156 r = urlutil.get_unique_pull_path(b'fastannotate', repo, ui, remotedest)
b6b696442a4d fastannotate: use `get_unique_pull_path`
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 43105
diff changeset
157 remotepath = r[0]
39241
303dae0136b0 fastannotate: rip out specialized support for remotefilelog
Augie Fackler <augie@google.com>
parents: 39239
diff changeset
158 peer = hg.peer(ui, {}, remotepath)
39238
1ddb296e0dee fastannotate: initial import from Facebook's hg-experimental
Augie Fackler <augie@google.com>
parents:
diff changeset
159
1ddb296e0dee fastannotate: initial import from Facebook's hg-experimental
Augie Fackler <augie@google.com>
parents:
diff changeset
160 try:
1ddb296e0dee fastannotate: initial import from Facebook's hg-experimental
Augie Fackler <augie@google.com>
parents:
diff changeset
161 yield peer
1ddb296e0dee fastannotate: initial import from Facebook's hg-experimental
Augie Fackler <augie@google.com>
parents:
diff changeset
162 finally:
39241
303dae0136b0 fastannotate: rip out specialized support for remotefilelog
Augie Fackler <augie@google.com>
parents: 39239
diff changeset
163 peer.close()
39238
1ddb296e0dee fastannotate: initial import from Facebook's hg-experimental
Augie Fackler <augie@google.com>
parents:
diff changeset
164
43076
2372284d9457 formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents: 41387
diff changeset
165
39238
1ddb296e0dee fastannotate: initial import from Facebook's hg-experimental
Augie Fackler <augie@google.com>
parents:
diff changeset
166 def clientfetch(repo, paths, lastnodemap=None, peer=None):
1ddb296e0dee fastannotate: initial import from Facebook's hg-experimental
Augie Fackler <augie@google.com>
parents:
diff changeset
167 """download annotate cache from the server for paths"""
1ddb296e0dee fastannotate: initial import from Facebook's hg-experimental
Augie Fackler <augie@google.com>
parents:
diff changeset
168 if not paths:
1ddb296e0dee fastannotate: initial import from Facebook's hg-experimental
Augie Fackler <augie@google.com>
parents:
diff changeset
169 return
1ddb296e0dee fastannotate: initial import from Facebook's hg-experimental
Augie Fackler <augie@google.com>
parents:
diff changeset
170
1ddb296e0dee fastannotate: initial import from Facebook's hg-experimental
Augie Fackler <augie@google.com>
parents:
diff changeset
171 if peer is None:
1ddb296e0dee fastannotate: initial import from Facebook's hg-experimental
Augie Fackler <augie@google.com>
parents:
diff changeset
172 with annotatepeer(repo) as peer:
1ddb296e0dee fastannotate: initial import from Facebook's hg-experimental
Augie Fackler <augie@google.com>
parents:
diff changeset
173 return clientfetch(repo, paths, lastnodemap, peer)
1ddb296e0dee fastannotate: initial import from Facebook's hg-experimental
Augie Fackler <augie@google.com>
parents:
diff changeset
174
1ddb296e0dee fastannotate: initial import from Facebook's hg-experimental
Augie Fackler <augie@google.com>
parents:
diff changeset
175 if lastnodemap is None:
1ddb296e0dee fastannotate: initial import from Facebook's hg-experimental
Augie Fackler <augie@google.com>
parents:
diff changeset
176 lastnodemap = {}
1ddb296e0dee fastannotate: initial import from Facebook's hg-experimental
Augie Fackler <augie@google.com>
parents:
diff changeset
177
1ddb296e0dee fastannotate: initial import from Facebook's hg-experimental
Augie Fackler <augie@google.com>
parents:
diff changeset
178 ui = repo.ui
1ddb296e0dee fastannotate: initial import from Facebook's hg-experimental
Augie Fackler <augie@google.com>
parents:
diff changeset
179 results = []
1ddb296e0dee fastannotate: initial import from Facebook's hg-experimental
Augie Fackler <augie@google.com>
parents:
diff changeset
180 with peer.commandexecutor() as batcher:
43077
687b865b95ad formatting: byteify all mercurial/ and hgext/ string literals
Augie Fackler <augie@google.com>
parents: 43076
diff changeset
181 ui.debug(b'fastannotate: requesting %d files\n' % len(paths))
39238
1ddb296e0dee fastannotate: initial import from Facebook's hg-experimental
Augie Fackler <augie@google.com>
parents:
diff changeset
182 for p in paths:
43076
2372284d9457 formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents: 41387
diff changeset
183 results.append(
2372284d9457 formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents: 41387
diff changeset
184 batcher.callcommand(
43077
687b865b95ad formatting: byteify all mercurial/ and hgext/ string literals
Augie Fackler <augie@google.com>
parents: 43076
diff changeset
185 b'getannotate',
687b865b95ad formatting: byteify all mercurial/ and hgext/ string literals
Augie Fackler <augie@google.com>
parents: 43076
diff changeset
186 {b'path': p, b'lastnode': lastnodemap.get(p)},
43076
2372284d9457 formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents: 41387
diff changeset
187 )
2372284d9457 formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents: 41387
diff changeset
188 )
39238
1ddb296e0dee fastannotate: initial import from Facebook's hg-experimental
Augie Fackler <augie@google.com>
parents:
diff changeset
189
39731
d8a7690ccc74 fastannotate: process files as they arrive
Martin von Zweigbergk <martinvonz@google.com>
parents: 39636
diff changeset
190 for result in results:
d8a7690ccc74 fastannotate: process files as they arrive
Martin von Zweigbergk <martinvonz@google.com>
parents: 39636
diff changeset
191 r = result.result()
d8a7690ccc74 fastannotate: process files as they arrive
Martin von Zweigbergk <martinvonz@google.com>
parents: 39636
diff changeset
192 # TODO: pconvert these paths on the server?
43105
649d3ac37a12 py3: define and use pycompat.iteritems() for hgext/
Gregory Szorc <gregory.szorc@gmail.com>
parents: 43085
diff changeset
193 r = {util.pconvert(p): v for p, v in pycompat.iteritems(r)}
39731
d8a7690ccc74 fastannotate: process files as they arrive
Martin von Zweigbergk <martinvonz@google.com>
parents: 39636
diff changeset
194 for path in sorted(r):
d8a7690ccc74 fastannotate: process files as they arrive
Martin von Zweigbergk <martinvonz@google.com>
parents: 39636
diff changeset
195 # ignore malicious paths
43077
687b865b95ad formatting: byteify all mercurial/ and hgext/ string literals
Augie Fackler <augie@google.com>
parents: 43076
diff changeset
196 if not path.startswith(b'fastannotate/') or b'/../' in (
687b865b95ad formatting: byteify all mercurial/ and hgext/ string literals
Augie Fackler <augie@google.com>
parents: 43076
diff changeset
197 path + b'/'
43076
2372284d9457 formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents: 41387
diff changeset
198 ):
43077
687b865b95ad formatting: byteify all mercurial/ and hgext/ string literals
Augie Fackler <augie@google.com>
parents: 43076
diff changeset
199 ui.debug(
687b865b95ad formatting: byteify all mercurial/ and hgext/ string literals
Augie Fackler <augie@google.com>
parents: 43076
diff changeset
200 b'fastannotate: ignored malicious path %s\n' % path
687b865b95ad formatting: byteify all mercurial/ and hgext/ string literals
Augie Fackler <augie@google.com>
parents: 43076
diff changeset
201 )
39731
d8a7690ccc74 fastannotate: process files as they arrive
Martin von Zweigbergk <martinvonz@google.com>
parents: 39636
diff changeset
202 continue
d8a7690ccc74 fastannotate: process files as they arrive
Martin von Zweigbergk <martinvonz@google.com>
parents: 39636
diff changeset
203 content = r[path]
d8a7690ccc74 fastannotate: process files as they arrive
Martin von Zweigbergk <martinvonz@google.com>
parents: 39636
diff changeset
204 if ui.debugflag:
43076
2372284d9457 formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents: 41387
diff changeset
205 ui.debug(
43077
687b865b95ad formatting: byteify all mercurial/ and hgext/ string literals
Augie Fackler <augie@google.com>
parents: 43076
diff changeset
206 b'fastannotate: writing %d bytes to %s\n'
43076
2372284d9457 formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents: 41387
diff changeset
207 % (len(content), path)
2372284d9457 formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents: 41387
diff changeset
208 )
39731
d8a7690ccc74 fastannotate: process files as they arrive
Martin von Zweigbergk <martinvonz@google.com>
parents: 39636
diff changeset
209 repo.vfs.makedirs(os.path.dirname(path))
43077
687b865b95ad formatting: byteify all mercurial/ and hgext/ string literals
Augie Fackler <augie@google.com>
parents: 43076
diff changeset
210 with repo.vfs(path, b'wb') as f:
39731
d8a7690ccc74 fastannotate: process files as they arrive
Martin von Zweigbergk <martinvonz@google.com>
parents: 39636
diff changeset
211 f.write(content)
39238
1ddb296e0dee fastannotate: initial import from Facebook's hg-experimental
Augie Fackler <augie@google.com>
parents:
diff changeset
212
43076
2372284d9457 formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents: 41387
diff changeset
213
39238
1ddb296e0dee fastannotate: initial import from Facebook's hg-experimental
Augie Fackler <augie@google.com>
parents:
diff changeset
214 def _filterfetchpaths(repo, paths):
1ddb296e0dee fastannotate: initial import from Facebook's hg-experimental
Augie Fackler <augie@google.com>
parents:
diff changeset
215 """return a subset of paths whose history is long and need to fetch linelog
1ddb296e0dee fastannotate: initial import from Facebook's hg-experimental
Augie Fackler <augie@google.com>
parents:
diff changeset
216 from the server. works with remotefilelog and non-remotefilelog repos.
1ddb296e0dee fastannotate: initial import from Facebook's hg-experimental
Augie Fackler <augie@google.com>
parents:
diff changeset
217 """
43077
687b865b95ad formatting: byteify all mercurial/ and hgext/ string literals
Augie Fackler <augie@google.com>
parents: 43076
diff changeset
218 threshold = repo.ui.configint(b'fastannotate', b'clientfetchthreshold', 10)
39238
1ddb296e0dee fastannotate: initial import from Facebook's hg-experimental
Augie Fackler <augie@google.com>
parents:
diff changeset
219 if threshold <= 0:
1ddb296e0dee fastannotate: initial import from Facebook's hg-experimental
Augie Fackler <augie@google.com>
parents:
diff changeset
220 return paths
1ddb296e0dee fastannotate: initial import from Facebook's hg-experimental
Augie Fackler <augie@google.com>
parents:
diff changeset
221
1ddb296e0dee fastannotate: initial import from Facebook's hg-experimental
Augie Fackler <augie@google.com>
parents:
diff changeset
222 result = []
1ddb296e0dee fastannotate: initial import from Facebook's hg-experimental
Augie Fackler <augie@google.com>
parents:
diff changeset
223 for path in paths:
1ddb296e0dee fastannotate: initial import from Facebook's hg-experimental
Augie Fackler <augie@google.com>
parents:
diff changeset
224 try:
39241
303dae0136b0 fastannotate: rip out specialized support for remotefilelog
Augie Fackler <augie@google.com>
parents: 39239
diff changeset
225 if len(repo.file(path)) >= threshold:
39238
1ddb296e0dee fastannotate: initial import from Facebook's hg-experimental
Augie Fackler <augie@google.com>
parents:
diff changeset
226 result.append(path)
43076
2372284d9457 formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents: 41387
diff changeset
227 except Exception: # file not found etc.
39238
1ddb296e0dee fastannotate: initial import from Facebook's hg-experimental
Augie Fackler <augie@google.com>
parents:
diff changeset
228 result.append(path)
1ddb296e0dee fastannotate: initial import from Facebook's hg-experimental
Augie Fackler <augie@google.com>
parents:
diff changeset
229
1ddb296e0dee fastannotate: initial import from Facebook's hg-experimental
Augie Fackler <augie@google.com>
parents:
diff changeset
230 return result
1ddb296e0dee fastannotate: initial import from Facebook's hg-experimental
Augie Fackler <augie@google.com>
parents:
diff changeset
231
43076
2372284d9457 formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents: 41387
diff changeset
232
39238
1ddb296e0dee fastannotate: initial import from Facebook's hg-experimental
Augie Fackler <augie@google.com>
parents:
diff changeset
233 def localreposetup(ui, repo):
1ddb296e0dee fastannotate: initial import from Facebook's hg-experimental
Augie Fackler <augie@google.com>
parents:
diff changeset
234 class fastannotaterepo(repo.__class__):
1ddb296e0dee fastannotate: initial import from Facebook's hg-experimental
Augie Fackler <augie@google.com>
parents:
diff changeset
235 def prefetchfastannotate(self, paths, peer=None):
1ddb296e0dee fastannotate: initial import from Facebook's hg-experimental
Augie Fackler <augie@google.com>
parents:
diff changeset
236 master = _getmaster(self.ui)
1ddb296e0dee fastannotate: initial import from Facebook's hg-experimental
Augie Fackler <augie@google.com>
parents:
diff changeset
237 needupdatepaths = []
1ddb296e0dee fastannotate: initial import from Facebook's hg-experimental
Augie Fackler <augie@google.com>
parents:
diff changeset
238 lastnodemap = {}
1ddb296e0dee fastannotate: initial import from Facebook's hg-experimental
Augie Fackler <augie@google.com>
parents:
diff changeset
239 try:
1ddb296e0dee fastannotate: initial import from Facebook's hg-experimental
Augie Fackler <augie@google.com>
parents:
diff changeset
240 for path in _filterfetchpaths(self, paths):
1ddb296e0dee fastannotate: initial import from Facebook's hg-experimental
Augie Fackler <augie@google.com>
parents:
diff changeset
241 with context.annotatecontext(self, path) as actx:
1ddb296e0dee fastannotate: initial import from Facebook's hg-experimental
Augie Fackler <augie@google.com>
parents:
diff changeset
242 if not actx.isuptodate(master, strict=False):
1ddb296e0dee fastannotate: initial import from Facebook's hg-experimental
Augie Fackler <augie@google.com>
parents:
diff changeset
243 needupdatepaths.append(path)
1ddb296e0dee fastannotate: initial import from Facebook's hg-experimental
Augie Fackler <augie@google.com>
parents:
diff changeset
244 lastnodemap[path] = actx.lastnode
1ddb296e0dee fastannotate: initial import from Facebook's hg-experimental
Augie Fackler <augie@google.com>
parents:
diff changeset
245 if needupdatepaths:
1ddb296e0dee fastannotate: initial import from Facebook's hg-experimental
Augie Fackler <augie@google.com>
parents:
diff changeset
246 clientfetch(self, needupdatepaths, lastnodemap, peer)
1ddb296e0dee fastannotate: initial import from Facebook's hg-experimental
Augie Fackler <augie@google.com>
parents:
diff changeset
247 except Exception as ex:
1ddb296e0dee fastannotate: initial import from Facebook's hg-experimental
Augie Fackler <augie@google.com>
parents:
diff changeset
248 # could be directory not writable or so, not fatal
43077
687b865b95ad formatting: byteify all mercurial/ and hgext/ string literals
Augie Fackler <augie@google.com>
parents: 43076
diff changeset
249 self.ui.debug(b'fastannotate: prefetch failed: %r\n' % ex)
43076
2372284d9457 formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents: 41387
diff changeset
250
39238
1ddb296e0dee fastannotate: initial import from Facebook's hg-experimental
Augie Fackler <augie@google.com>
parents:
diff changeset
251 repo.__class__ = fastannotaterepo
1ddb296e0dee fastannotate: initial import from Facebook's hg-experimental
Augie Fackler <augie@google.com>
parents:
diff changeset
252
43076
2372284d9457 formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents: 41387
diff changeset
253
39238
1ddb296e0dee fastannotate: initial import from Facebook's hg-experimental
Augie Fackler <augie@google.com>
parents:
diff changeset
254 def clientreposetup(ui, repo):
1ddb296e0dee fastannotate: initial import from Facebook's hg-experimental
Augie Fackler <augie@google.com>
parents:
diff changeset
255 _registerwireprotocommand()
39560
2182e67ea912 fastannotate: use repo.local()
Gregory Szorc <gregory.szorc@gmail.com>
parents: 39279
diff changeset
256 if repo.local():
39238
1ddb296e0dee fastannotate: initial import from Facebook's hg-experimental
Augie Fackler <augie@google.com>
parents:
diff changeset
257 localreposetup(ui, repo)
39242
8da20fc9fc07 fastannotate: move some global state mutation to extsetup()
Augie Fackler <augie@google.com>
parents: 39241
diff changeset
258 # TODO: this mutates global state, but only if at least one repo
8da20fc9fc07 fastannotate: move some global state mutation to extsetup()
Augie Fackler <augie@google.com>
parents: 39241
diff changeset
259 # has the extension enabled. This is probably bad for hgweb.
39238
1ddb296e0dee fastannotate: initial import from Facebook's hg-experimental
Augie Fackler <augie@google.com>
parents:
diff changeset
260 if peersetup not in hg.wirepeersetupfuncs:
1ddb296e0dee fastannotate: initial import from Facebook's hg-experimental
Augie Fackler <augie@google.com>
parents:
diff changeset
261 hg.wirepeersetupfuncs.append(peersetup)