Mercurial > public > mercurial-scm > hg-stable

annotate mercurial/sshpeer.py @ 35440:31d21309635b

Find changesets by keywords (author, files, the commit message), revision number or hash, or revset expression.
sshpeer: allow for additional environment passing to ssh exe We already have the ability to customize the ssh command line arguments, let's add the ability to customize its environment as well. Example use-case is ssh.exe from Git on Windows. If `HOME` enviroment variable is present and has some non-empty value, ssh.exe will try to access that location for some stuff (for example, it seems for resolving `~` in `.ssh/config`). Git for Windows seems to sometimess set this variable to the value of `/home/username` which probably works under Git Bash, but does not work in a native `cmd.exe` or `powershell`. Whatever the root cause, setting `HOME` to be an empty string heals things. Therefore, some distributors might want to set `sshenv.HOME=` in the configuration (seems less intrusive that forcing everyone to tweak their env). Test Plan: - rt Differential Revision: https://phab.mercurial-scm.org/D1683
author Kostia Balytskyi <ikostia@fb.com>
date Thu, 14 Dec 2017 14:31:57 +0000
parents 8b1c887d52e7
children b520c8f98e1e
Ignore whitespace changes - Everywhere: Within whitespace: At end of lines:
rev   line source
17192
1ac628cd7113 peer: introduce real peer classes
Peter Arrenbrecht <peter.arrenbrecht@gmail.com>
parents: 16688
diff changeset
1 # sshpeer.py - ssh repository proxy class for mercurial
0
9117c6561b0b Add back links from file revisions to changeset revisions
mpm@selenic.com
parents:
diff changeset
2 #
2859
345bac2bc4ec update copyrights.
Vadim Gelfer <vadim.gelfer@gmail.com>
parents: 2740
diff changeset
3 # Copyright 2005, 2006 Matt Mackall <mpm@selenic.com>
0
9117c6561b0b Add back links from file revisions to changeset revisions
mpm@selenic.com
parents:
diff changeset
4 #
8225
46293a0c7e9f updated license to be explicit about GPL version 2
Martin Geisler <mg@lazybytes.net>
parents: 8210
diff changeset
5 # This software may be used and distributed according to the terms of the
10263
25e572394f5c Update license to GPLv2+
Matt Mackall <mpm@selenic.com>
parents: 9878
diff changeset
6 # GNU General Public License version 2 or any later version.
0
9117c6561b0b Add back links from file revisions to changeset revisions
mpm@selenic.com
parents:
diff changeset
7
25975
de7a3893ee65 sshpeer: use absolute_import
Gregory Szorc <gregory.szorc@gmail.com>
parents: 25458
diff changeset
8 from __future__ import absolute_import
de7a3893ee65 sshpeer: use absolute_import
Gregory Szorc <gregory.szorc@gmail.com>
parents: 25458
diff changeset
9
15622
86fc364ca5f8 sshrepo: don't quote obviously safe strings (issue2983)
Mads Kiilerich <mads@kiilerich.com>
parents: 15581
diff changeset
10 import re
25975
de7a3893ee65 sshpeer: use absolute_import
Gregory Szorc <gregory.szorc@gmail.com>
parents: 25458
diff changeset
11
de7a3893ee65 sshpeer: use absolute_import
Gregory Szorc <gregory.szorc@gmail.com>
parents: 25458
diff changeset
12 from .i18n import _
de7a3893ee65 sshpeer: use absolute_import
Gregory Szorc <gregory.szorc@gmail.com>
parents: 25458
diff changeset
13 from . import (
de7a3893ee65 sshpeer: use absolute_import
Gregory Szorc <gregory.szorc@gmail.com>
parents: 25458
diff changeset
14 error,
33112
05906b8e1d23 py3: use pycompat.byteskwargs() to convert kwargs' keys to bytes
Pulkit Goyal <7895pulkit@gmail.com>
parents: 32062
diff changeset
15 pycompat,
25975
de7a3893ee65 sshpeer: use absolute_import
Gregory Szorc <gregory.szorc@gmail.com>
parents: 25458
diff changeset
16 util,
de7a3893ee65 sshpeer: use absolute_import
Gregory Szorc <gregory.szorc@gmail.com>
parents: 25458
diff changeset
17 wireproto,
de7a3893ee65 sshpeer: use absolute_import
Gregory Szorc <gregory.szorc@gmail.com>
parents: 25458
diff changeset
18 )
60
e32fdbd97839 Add hg:// protocol
mpm@selenic.com
parents: 56
diff changeset
19
15581
d8fa35c28335 ssh: quote remote paths (issue2983)
Mads Kiilerich <mads@kiilerich.com>
parents: 15017
diff changeset
20 def _serverquote(s):
23671
e3f30068d2eb sshpeer: more thorough shell quoting
Matt Mackall <mpm@selenic.com>
parents: 22935
diff changeset
21 if not s:
e3f30068d2eb sshpeer: more thorough shell quoting
Matt Mackall <mpm@selenic.com>
parents: 22935
diff changeset
22 return s
15581
d8fa35c28335 ssh: quote remote paths (issue2983)
Mads Kiilerich <mads@kiilerich.com>
parents: 15017
diff changeset
23 '''quote a string for the remote shell ... which we assume is sh'''
15624
be43234a6d60 sshrepo: add more safe characters (issue2983)
Thomas Arendsen Hein <thomas@intevation.de>
parents: 15622
diff changeset
24 if re.match('[a-zA-Z0-9@%_+=:,./-]*$', s):
15622
86fc364ca5f8 sshrepo: don't quote obviously safe strings (issue2983)
Mads Kiilerich <mads@kiilerich.com>
parents: 15581
diff changeset
25 return s
15581
d8fa35c28335 ssh: quote remote paths (issue2983)
Mads Kiilerich <mads@kiilerich.com>
parents: 15017
diff changeset
26 return "'%s'" % s.replace("'", "'\\''")
d8fa35c28335 ssh: quote remote paths (issue2983)
Mads Kiilerich <mads@kiilerich.com>
parents: 15017
diff changeset
27
25244
cf90764f40a4 sshpeer: extract the forward output logic
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 25243
diff changeset
28 def _forwardoutput(ui, pipe):
cf90764f40a4 sshpeer: extract the forward output logic
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 25243
diff changeset
29 """display all data currently available on pipe as remote output.
cf90764f40a4 sshpeer: extract the forward output logic
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 25243
diff changeset
30
cf90764f40a4 sshpeer: extract the forward output logic
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 25243
diff changeset
31 This is non blocking."""
cf90764f40a4 sshpeer: extract the forward output logic
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 25243
diff changeset
32 s = util.readpipe(pipe)
cf90764f40a4 sshpeer: extract the forward output logic
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 25243
diff changeset
33 if s:
cf90764f40a4 sshpeer: extract the forward output logic
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 25243
diff changeset
34 for l in s.splitlines():
cf90764f40a4 sshpeer: extract the forward output logic
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 25243
diff changeset
35 ui.status(_("remote: "), l, '\n')
cf90764f40a4 sshpeer: extract the forward output logic
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 25243
diff changeset
36
25421
3dd3ccf7b807 sshpeer: introduce a "doublepipe" class
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 25408
diff changeset
37 class doublepipe(object):
3dd3ccf7b807 sshpeer: introduce a "doublepipe" class
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 25408
diff changeset
38 """Operate a side-channel pipe in addition of a main one
3dd3ccf7b807 sshpeer: introduce a "doublepipe" class
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 25408
diff changeset
39
3dd3ccf7b807 sshpeer: introduce a "doublepipe" class
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 25408
diff changeset
40 The side-channel pipe contains server output to be forwarded to the user
3dd3ccf7b807 sshpeer: introduce a "doublepipe" class
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 25408
diff changeset
41 input. The double pipe will behave as the "main" pipe, but will ensure the
3dd3ccf7b807 sshpeer: introduce a "doublepipe" class
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 25408
diff changeset
42 content of the "side" pipe is properly processed while we wait for blocking
3dd3ccf7b807 sshpeer: introduce a "doublepipe" class
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 25408
diff changeset
43 call on the "main" pipe.
3dd3ccf7b807 sshpeer: introduce a "doublepipe" class
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 25408
diff changeset
44
3dd3ccf7b807 sshpeer: introduce a "doublepipe" class
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 25408
diff changeset
45 If large amounts of data are read from "main", the forward will cease after
3dd3ccf7b807 sshpeer: introduce a "doublepipe" class
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 25408
diff changeset
46 the first bytes start to appear. This simplifies the implementation
3dd3ccf7b807 sshpeer: introduce a "doublepipe" class
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 25408
diff changeset
47 without affecting actual output of sshpeer too much as we rarely issue
3dd3ccf7b807 sshpeer: introduce a "doublepipe" class
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 25408
diff changeset
48 large read for data not yet emitted by the server.
3dd3ccf7b807 sshpeer: introduce a "doublepipe" class
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 25408
diff changeset
49
3dd3ccf7b807 sshpeer: introduce a "doublepipe" class
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 25408
diff changeset
50 The main pipe is expected to be a 'bufferedinputpipe' from the util module
31953
cc2382b60007 sshpeer: fix docstring typo
Augie Fackler <augie@google.com>
parents: 31207
diff changeset
51 that handle all the os specific bits. This class lives in this module
26781
1aee2ab0f902 spelling: trivial spell checking
Mads Kiilerich <madski@unity3d.com>
parents: 26587
diff changeset
52 because it focus on behavior specific to the ssh protocol."""
25421
3dd3ccf7b807 sshpeer: introduce a "doublepipe" class
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 25408
diff changeset
53
3dd3ccf7b807 sshpeer: introduce a "doublepipe" class
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 25408
diff changeset
54 def __init__(self, ui, main, side):
3dd3ccf7b807 sshpeer: introduce a "doublepipe" class
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 25408
diff changeset
55 self._ui = ui
3dd3ccf7b807 sshpeer: introduce a "doublepipe" class
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 25408
diff changeset
56 self._main = main
3dd3ccf7b807 sshpeer: introduce a "doublepipe" class
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 25408
diff changeset
57 self._side = side
3dd3ccf7b807 sshpeer: introduce a "doublepipe" class
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 25408
diff changeset
58
3dd3ccf7b807 sshpeer: introduce a "doublepipe" class
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 25408
diff changeset
59 def _wait(self):
3dd3ccf7b807 sshpeer: introduce a "doublepipe" class
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 25408
diff changeset
60 """wait until some data are available on main or side
3dd3ccf7b807 sshpeer: introduce a "doublepipe" class
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 25408
diff changeset
61
3dd3ccf7b807 sshpeer: introduce a "doublepipe" class
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 25408
diff changeset
62 return a pair of boolean (ismainready, issideready)
3dd3ccf7b807 sshpeer: introduce a "doublepipe" class
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 25408
diff changeset
63
3dd3ccf7b807 sshpeer: introduce a "doublepipe" class
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 25408
diff changeset
64 (This will only wait for data if the setup is supported by `util.poll`)
3dd3ccf7b807 sshpeer: introduce a "doublepipe" class
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 25408
diff changeset
65 """
25457
2afa748138e0 sshpeer: allow doublepipe on unbuffered main pipe
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 25456
diff changeset
66 if getattr(self._main, 'hasbuffer', False): # getattr for classic pipe
25421
3dd3ccf7b807 sshpeer: introduce a "doublepipe" class
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 25408
diff changeset
67 return (True, True) # main has data, assume side is worth poking at.
3dd3ccf7b807 sshpeer: introduce a "doublepipe" class
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 25408
diff changeset
68 fds = [self._main.fileno(), self._side.fileno()]
3dd3ccf7b807 sshpeer: introduce a "doublepipe" class
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 25408
diff changeset
69 try:
3dd3ccf7b807 sshpeer: introduce a "doublepipe" class
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 25408
diff changeset
70 act = util.poll(fds)
3dd3ccf7b807 sshpeer: introduce a "doublepipe" class
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 25408
diff changeset
71 except NotImplementedError:
3dd3ccf7b807 sshpeer: introduce a "doublepipe" class
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 25408
diff changeset
72 # non supported yet case, assume all have data.
3dd3ccf7b807 sshpeer: introduce a "doublepipe" class
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 25408
diff changeset
73 act = fds
3dd3ccf7b807 sshpeer: introduce a "doublepipe" class
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 25408
diff changeset
74 return (self._main.fileno() in act, self._side.fileno() in act)
3dd3ccf7b807 sshpeer: introduce a "doublepipe" class
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 25408
diff changeset
75
25456
408b7979bf03 sshpeer: allow write operations through double pipe
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 25455
diff changeset
76 def write(self, data):
408b7979bf03 sshpeer: allow write operations through double pipe
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 25455
diff changeset
77 return self._call('write', data)
408b7979bf03 sshpeer: allow write operations through double pipe
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 25455
diff changeset
78
25421
3dd3ccf7b807 sshpeer: introduce a "doublepipe" class
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 25408
diff changeset
79 def read(self, size):
32062
ad6c5497cd15 sshpeer: try harder to snag stderr when stdout closes unexpectedly
Augie Fackler <augie@google.com>
parents: 31953
diff changeset
80 r = self._call('read', size)
ad6c5497cd15 sshpeer: try harder to snag stderr when stdout closes unexpectedly
Augie Fackler <augie@google.com>
parents: 31953
diff changeset
81 if size != 0 and not r:
ad6c5497cd15 sshpeer: try harder to snag stderr when stdout closes unexpectedly
Augie Fackler <augie@google.com>
parents: 31953
diff changeset
82 # We've observed a condition that indicates the
ad6c5497cd15 sshpeer: try harder to snag stderr when stdout closes unexpectedly
Augie Fackler <augie@google.com>
parents: 31953
diff changeset
83 # stdout closed unexpectedly. Check stderr one
ad6c5497cd15 sshpeer: try harder to snag stderr when stdout closes unexpectedly
Augie Fackler <augie@google.com>
parents: 31953
diff changeset
84 # more time and snag anything that's there before
ad6c5497cd15 sshpeer: try harder to snag stderr when stdout closes unexpectedly
Augie Fackler <augie@google.com>
parents: 31953
diff changeset
85 # letting anyone know the main part of the pipe
ad6c5497cd15 sshpeer: try harder to snag stderr when stdout closes unexpectedly
Augie Fackler <augie@google.com>
parents: 31953
diff changeset
86 # closed prematurely.
ad6c5497cd15 sshpeer: try harder to snag stderr when stdout closes unexpectedly
Augie Fackler <augie@google.com>
parents: 31953
diff changeset
87 _forwardoutput(self._ui, self._side)
ad6c5497cd15 sshpeer: try harder to snag stderr when stdout closes unexpectedly
Augie Fackler <augie@google.com>
parents: 31953
diff changeset
88 return r
25421
3dd3ccf7b807 sshpeer: introduce a "doublepipe" class
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 25408
diff changeset
89
3dd3ccf7b807 sshpeer: introduce a "doublepipe" class
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 25408
diff changeset
90 def readline(self):
3dd3ccf7b807 sshpeer: introduce a "doublepipe" class
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 25408
diff changeset
91 return self._call('readline')
3dd3ccf7b807 sshpeer: introduce a "doublepipe" class
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 25408
diff changeset
92
25455
dc02a284e034 sshpeer: rename 'size' to 'data' in doublepipe
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 25422
diff changeset
93 def _call(self, methname, data=None):
25421
3dd3ccf7b807 sshpeer: introduce a "doublepipe" class
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 25408
diff changeset
94 """call <methname> on "main", forward output of "side" while blocking
3dd3ccf7b807 sshpeer: introduce a "doublepipe" class
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 25408
diff changeset
95 """
25455
dc02a284e034 sshpeer: rename 'size' to 'data' in doublepipe
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 25422
diff changeset
96 # data can be '' or 0
dc02a284e034 sshpeer: rename 'size' to 'data' in doublepipe
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 25422
diff changeset
97 if (data is not None and not data) or self._main.closed:
25421
3dd3ccf7b807 sshpeer: introduce a "doublepipe" class
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 25408
diff changeset
98 _forwardoutput(self._ui, self._side)
3dd3ccf7b807 sshpeer: introduce a "doublepipe" class
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 25408
diff changeset
99 return ''
3dd3ccf7b807 sshpeer: introduce a "doublepipe" class
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 25408
diff changeset
100 while True:
3dd3ccf7b807 sshpeer: introduce a "doublepipe" class
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 25408
diff changeset
101 mainready, sideready = self._wait()
3dd3ccf7b807 sshpeer: introduce a "doublepipe" class
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 25408
diff changeset
102 if sideready:
3dd3ccf7b807 sshpeer: introduce a "doublepipe" class
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 25408
diff changeset
103 _forwardoutput(self._ui, self._side)
3dd3ccf7b807 sshpeer: introduce a "doublepipe" class
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 25408
diff changeset
104 if mainready:
3dd3ccf7b807 sshpeer: introduce a "doublepipe" class
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 25408
diff changeset
105 meth = getattr(self._main, methname)
25455
dc02a284e034 sshpeer: rename 'size' to 'data' in doublepipe
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 25422
diff changeset
106 if data is None:
25421
3dd3ccf7b807 sshpeer: introduce a "doublepipe" class
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 25408
diff changeset
107 return meth()
3dd3ccf7b807 sshpeer: introduce a "doublepipe" class
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 25408
diff changeset
108 else:
25455
dc02a284e034 sshpeer: rename 'size' to 'data' in doublepipe
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 25422
diff changeset
109 return meth(data)
25421
3dd3ccf7b807 sshpeer: introduce a "doublepipe" class
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 25408
diff changeset
110
3dd3ccf7b807 sshpeer: introduce a "doublepipe" class
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 25408
diff changeset
111 def close(self):
3dd3ccf7b807 sshpeer: introduce a "doublepipe" class
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 25408
diff changeset
112 return self._main.close()
3dd3ccf7b807 sshpeer: introduce a "doublepipe" class
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 25408
diff changeset
113
25456
408b7979bf03 sshpeer: allow write operations through double pipe
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 25455
diff changeset
114 def flush(self):
408b7979bf03 sshpeer: allow write operations through double pipe
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 25455
diff changeset
115 return self._main.flush()
408b7979bf03 sshpeer: allow write operations through double pipe
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 25455
diff changeset
116
17192
1ac628cd7113 peer: introduce real peer classes
Peter Arrenbrecht <peter.arrenbrecht@gmail.com>
parents: 16688
diff changeset
117 class sshpeer(wireproto.wirepeer):
14363
82f3b0f3f0a5 localrepo, sshrepo: use Boolean create argument in __init__
Martin Geisler <mg@lazybytes.net>
parents: 14076
diff changeset
118 def __init__(self, ui, path, create=False):
2673
109a22f5434a hooks: add url to changegroup, incoming, prechangegroup, pretxnchangegroup hooks
Vadim Gelfer <vadim.gelfer@gmail.com>
parents: 2612
diff changeset
119 self._url = path
33825
1f8460b55986 sshpeer: use peer interface
Gregory Szorc <gregory.szorc@gmail.com>
parents: 33789
diff changeset
120 self._ui = ui
33789
82d564d5ac4f sshpeer: make instance attributes and methods internal
Gregory Szorc <gregory.szorc@gmail.com>
parents: 33763
diff changeset
121 self._pipeo = self._pipei = self._pipee = None
624
876333a295ff Add an sshrepository class and hg serve --stdio
Matt Mackall <mpm@selenic.com>
parents: 623
diff changeset
122
14076
924c82157d46 url: move URL parsing functions into util to improve startup time
Brodie Rao <brodie@bitheap.org>
parents: 14004
diff changeset
123 u = util.url(path, parsequery=False, parsefragment=False)
13819
d16894e29f91 httprepo/sshrepo: use url.url
Brodie Rao <brodie@bitheap.org>
parents: 13721
diff changeset
124 if u.scheme != 'ssh' or not u.host or u.path is None:
11590
0b15aee0a306 protocol: convert some ssh abort calls
Matt Mackall <mpm@selenic.com>
parents: 11589
diff changeset
125 self._abort(error.RepoError(_("couldn't parse location %s") % path))
624
876333a295ff Add an sshrepository class and hg serve --stdio
Matt Mackall <mpm@selenic.com>
parents: 623
diff changeset
126
33650
d7a1c4c1181d sshpeer: check for safe ssh url (SEC)
Sean Farley <sean@farley.io>
parents: 33499
diff changeset
127 util.checksafessh(path)
d7a1c4c1181d sshpeer: check for safe ssh url (SEC)
Sean Farley <sean@farley.io>
parents: 33499
diff changeset
128
13819
d16894e29f91 httprepo/sshrepo: use url.url
Brodie Rao <brodie@bitheap.org>
parents: 13721
diff changeset
129 if u.passwd is not None:
13464
da0ddd62b9d8 sshrepo: catch passwords in ssh urls
Adrian Buehlmann <adrian@cadifra.com>
parents: 13084
diff changeset
130 self._abort(error.RepoError(_("password in URL not supported")))
33789
82d564d5ac4f sshpeer: make instance attributes and methods internal
Gregory Szorc <gregory.szorc@gmail.com>
parents: 33763
diff changeset
131
82d564d5ac4f sshpeer: make instance attributes and methods internal
Gregory Szorc <gregory.szorc@gmail.com>
parents: 33763
diff changeset
132 self._user = u.user
82d564d5ac4f sshpeer: make instance attributes and methods internal
Gregory Szorc <gregory.szorc@gmail.com>
parents: 33763
diff changeset
133 self._host = u.host
82d564d5ac4f sshpeer: make instance attributes and methods internal
Gregory Szorc <gregory.szorc@gmail.com>
parents: 33763
diff changeset
134 self._port = u.port
82d564d5ac4f sshpeer: make instance attributes and methods internal
Gregory Szorc <gregory.szorc@gmail.com>
parents: 33763
diff changeset
135 self._path = u.path or '.'
624
876333a295ff Add an sshrepository class and hg serve --stdio
Matt Mackall <mpm@selenic.com>
parents: 623
diff changeset
136
33499
0407a51b9d8c codemod: register core configitems using a script
Jun Wu <quark@fb.com>
parents: 33112
diff changeset
137 sshcmd = self.ui.config("ui", "ssh")
0407a51b9d8c codemod: register core configitems using a script
Jun Wu <quark@fb.com>
parents: 33112
diff changeset
138 remotecmd = self.ui.config("ui", "remotecmd")
35440
31d21309635b sshpeer: allow for additional environment passing to ssh exe
Kostia Balytskyi <ikostia@fb.com>
parents: 35131
diff changeset
139 sshaddenv = dict(self.ui.configitems("sshenv"))
31d21309635b sshpeer: allow for additional environment passing to ssh exe
Kostia Balytskyi <ikostia@fb.com>
parents: 35131
diff changeset
140 sshenv = util.shellenviron(sshaddenv)
2549
e1831f06eef1 Added ability to clone from a local repository to a (new) remote one.
Sean Meiners <sean.meiners@linspire.com>
parents: 2484
diff changeset
141
33789
82d564d5ac4f sshpeer: make instance attributes and methods internal
Gregory Szorc <gregory.szorc@gmail.com>
parents: 33763
diff changeset
142 args = util.sshargs(sshcmd, self._host, self._user, self._port)
5644
e2e8e977a6cb win32: fix ssh://host:port when using Plink
Steve Borho <steve@borho.org>
parents: 5293
diff changeset
143
2549
e1831f06eef1 Added ability to clone from a local repository to a (new) remote one.
Sean Meiners <sean.meiners@linspire.com>
parents: 2484
diff changeset
144 if create:
15581
d8fa35c28335 ssh: quote remote paths (issue2983)
Mads Kiilerich <mads@kiilerich.com>
parents: 15017
diff changeset
145 cmd = '%s %s %s' % (sshcmd, args,
d8fa35c28335 ssh: quote remote paths (issue2983)
Mads Kiilerich <mads@kiilerich.com>
parents: 15017
diff changeset
146 util.shellquote("%s init %s" %
33789
82d564d5ac4f sshpeer: make instance attributes and methods internal
Gregory Szorc <gregory.szorc@gmail.com>
parents: 33763
diff changeset
147 (_serverquote(remotecmd), _serverquote(self._path))))
20794
8b0e3a8982ea sshpeer: only print out 'running ssh' messages in debug mode (BC)
Siddharth Agarwal <sid0@fb.com>
parents: 19405
diff changeset
148 ui.debug('running %s\n' % cmd)
35440
31d21309635b sshpeer: allow for additional environment passing to ssh exe
Kostia Balytskyi <ikostia@fb.com>
parents: 35131
diff changeset
149 res = ui.system(cmd, blockedtag='sshpeer', environ=sshenv)
2549
e1831f06eef1 Added ability to clone from a local repository to a (new) remote one.
Sean Meiners <sean.meiners@linspire.com>
parents: 2484
diff changeset
150 if res != 0:
11590
0b15aee0a306 protocol: convert some ssh abort calls
Matt Mackall <mpm@selenic.com>
parents: 11589
diff changeset
151 self._abort(error.RepoError(_("could not create remote repo")))
2549
e1831f06eef1 Added ability to clone from a local repository to a (new) remote one.
Sean Meiners <sean.meiners@linspire.com>
parents: 2484
diff changeset
152
35440
31d21309635b sshpeer: allow for additional environment passing to ssh exe
Kostia Balytskyi <ikostia@fb.com>
parents: 35131
diff changeset
153 self._validaterepo(sshcmd, args, remotecmd, sshenv)
2549
e1831f06eef1 Added ability to clone from a local repository to a (new) remote one.
Sean Meiners <sean.meiners@linspire.com>
parents: 2484
diff changeset
154
33825
1f8460b55986 sshpeer: use peer interface
Gregory Szorc <gregory.szorc@gmail.com>
parents: 33789
diff changeset
155 # Begin of _basepeer interface.
1f8460b55986 sshpeer: use peer interface
Gregory Szorc <gregory.szorc@gmail.com>
parents: 33789
diff changeset
156
1f8460b55986 sshpeer: use peer interface
Gregory Szorc <gregory.szorc@gmail.com>
parents: 33789
diff changeset
157 @util.propertycache
1f8460b55986 sshpeer: use peer interface
Gregory Szorc <gregory.szorc@gmail.com>
parents: 33789
diff changeset
158 def ui(self):
1f8460b55986 sshpeer: use peer interface
Gregory Szorc <gregory.szorc@gmail.com>
parents: 33789
diff changeset
159 return self._ui
1f8460b55986 sshpeer: use peer interface
Gregory Szorc <gregory.szorc@gmail.com>
parents: 33789
diff changeset
160
2673
109a22f5434a hooks: add url to changegroup, incoming, prechangegroup, pretxnchangegroup hooks
Vadim Gelfer <vadim.gelfer@gmail.com>
parents: 2612
diff changeset
161 def url(self):
109a22f5434a hooks: add url to changegroup, incoming, prechangegroup, pretxnchangegroup hooks
Vadim Gelfer <vadim.gelfer@gmail.com>
parents: 2612
diff changeset
162 return self._url
109a22f5434a hooks: add url to changegroup, incoming, prechangegroup, pretxnchangegroup hooks
Vadim Gelfer <vadim.gelfer@gmail.com>
parents: 2612
diff changeset
163
33825
1f8460b55986 sshpeer: use peer interface
Gregory Szorc <gregory.szorc@gmail.com>
parents: 33789
diff changeset
164 def local(self):
1f8460b55986 sshpeer: use peer interface
Gregory Szorc <gregory.szorc@gmail.com>
parents: 33789
diff changeset
165 return None
1f8460b55986 sshpeer: use peer interface
Gregory Szorc <gregory.szorc@gmail.com>
parents: 33789
diff changeset
166
1f8460b55986 sshpeer: use peer interface
Gregory Szorc <gregory.szorc@gmail.com>
parents: 33789
diff changeset
167 def peer(self):
1f8460b55986 sshpeer: use peer interface
Gregory Szorc <gregory.szorc@gmail.com>
parents: 33789
diff changeset
168 return self
1f8460b55986 sshpeer: use peer interface
Gregory Szorc <gregory.szorc@gmail.com>
parents: 33789
diff changeset
169
1f8460b55986 sshpeer: use peer interface
Gregory Szorc <gregory.szorc@gmail.com>
parents: 33789
diff changeset
170 def canpush(self):
1f8460b55986 sshpeer: use peer interface
Gregory Szorc <gregory.szorc@gmail.com>
parents: 33789
diff changeset
171 return True
1f8460b55986 sshpeer: use peer interface
Gregory Szorc <gregory.szorc@gmail.com>
parents: 33789
diff changeset
172
1f8460b55986 sshpeer: use peer interface
Gregory Szorc <gregory.szorc@gmail.com>
parents: 33789
diff changeset
173 def close(self):
1f8460b55986 sshpeer: use peer interface
Gregory Szorc <gregory.szorc@gmail.com>
parents: 33789
diff changeset
174 pass
1f8460b55986 sshpeer: use peer interface
Gregory Szorc <gregory.szorc@gmail.com>
parents: 33789
diff changeset
175
1f8460b55986 sshpeer: use peer interface
Gregory Szorc <gregory.szorc@gmail.com>
parents: 33789
diff changeset
176 # End of _basepeer interface.
1f8460b55986 sshpeer: use peer interface
Gregory Szorc <gregory.szorc@gmail.com>
parents: 33789
diff changeset
177
1f8460b55986 sshpeer: use peer interface
Gregory Szorc <gregory.szorc@gmail.com>
parents: 33789
diff changeset
178 # Begin of _basewirecommands interface.
1f8460b55986 sshpeer: use peer interface
Gregory Szorc <gregory.szorc@gmail.com>
parents: 33789
diff changeset
179
1f8460b55986 sshpeer: use peer interface
Gregory Szorc <gregory.szorc@gmail.com>
parents: 33789
diff changeset
180 def capabilities(self):
1f8460b55986 sshpeer: use peer interface
Gregory Szorc <gregory.szorc@gmail.com>
parents: 33789
diff changeset
181 return self._caps
1f8460b55986 sshpeer: use peer interface
Gregory Szorc <gregory.szorc@gmail.com>
parents: 33789
diff changeset
182
1f8460b55986 sshpeer: use peer interface
Gregory Szorc <gregory.szorc@gmail.com>
parents: 33789
diff changeset
183 # End of _basewirecommands interface.
1f8460b55986 sshpeer: use peer interface
Gregory Szorc <gregory.szorc@gmail.com>
parents: 33789
diff changeset
184
35440
31d21309635b sshpeer: allow for additional environment passing to ssh exe
Kostia Balytskyi <ikostia@fb.com>
parents: 35131
diff changeset
185 def _validaterepo(self, sshcmd, args, remotecmd, sshenv=None):
3034
2b0bc36a48d8 sshrepo: flush stderr before connecting to the hg server
Benoit Boissinot <benoit.boissinot@ens-lyon.org>
parents: 2859
diff changeset
186 # cleanup up previous run
33789
82d564d5ac4f sshpeer: make instance attributes and methods internal
Gregory Szorc <gregory.szorc@gmail.com>
parents: 33763
diff changeset
187 self._cleanup()
3034
2b0bc36a48d8 sshrepo: flush stderr before connecting to the hg server
Benoit Boissinot <benoit.boissinot@ens-lyon.org>
parents: 2859
diff changeset
188
15581
d8fa35c28335 ssh: quote remote paths (issue2983)
Mads Kiilerich <mads@kiilerich.com>
parents: 15017
diff changeset
189 cmd = '%s %s %s' % (sshcmd, args,
d8fa35c28335 ssh: quote remote paths (issue2983)
Mads Kiilerich <mads@kiilerich.com>
parents: 15017
diff changeset
190 util.shellquote("%s -R %s serve --stdio" %
33789
82d564d5ac4f sshpeer: make instance attributes and methods internal
Gregory Szorc <gregory.szorc@gmail.com>
parents: 33763
diff changeset
191 (_serverquote(remotecmd), _serverquote(self._path))))
20794
8b0e3a8982ea sshpeer: only print out 'running ssh' messages in debug mode (BC)
Siddharth Agarwal <sid0@fb.com>
parents: 19405
diff changeset
192 self.ui.debug('running %s\n' % cmd)
5292
5a65d870871d sshrepo: fix Windows command quoting
Alexis S. L. Carvalho <alexis@cecm.usp.br>
parents: 5190
diff changeset
193 cmd = util.quotecommand(cmd)
18759
9baf4330d88f sshpeer: store subprocess so it cleans up correctly
Durham Goode <durham@fb.com>
parents: 17192
diff changeset
194
33789
82d564d5ac4f sshpeer: make instance attributes and methods internal
Gregory Szorc <gregory.szorc@gmail.com>
parents: 33763
diff changeset
195 # while self._subprocess isn't used, having it allows the subprocess to
18759
9baf4330d88f sshpeer: store subprocess so it cleans up correctly
Durham Goode <durham@fb.com>
parents: 17192
diff changeset
196 # to clean up correctly later
25408
c88975a4d264 sshpeer: run the ssh command unbuffered
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 25407
diff changeset
197 #
c88975a4d264 sshpeer: run the ssh command unbuffered
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 25407
diff changeset
198 # no buffer allow the use of 'select'
c88975a4d264 sshpeer: run the ssh command unbuffered
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 25407
diff changeset
199 # feel free to remove buffering and select usage when we ultimately
c88975a4d264 sshpeer: run the ssh command unbuffered
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 25407
diff changeset
200 # move to threading.
35440
31d21309635b sshpeer: allow for additional environment passing to ssh exe
Kostia Balytskyi <ikostia@fb.com>
parents: 35131
diff changeset
201 sub = util.popen4(cmd, bufsize=0, env=sshenv)
33789
82d564d5ac4f sshpeer: make instance attributes and methods internal
Gregory Szorc <gregory.szorc@gmail.com>
parents: 33763
diff changeset
202 self._pipeo, self._pipei, self._pipee, self._subprocess = sub
646
342927da4f4c Show remote client output with "remote:"
Matt Mackall <mpm@selenic.com>
parents: 644
diff changeset
203
33789
82d564d5ac4f sshpeer: make instance attributes and methods internal
Gregory Szorc <gregory.szorc@gmail.com>
parents: 33763
diff changeset
204 self._pipei = util.bufferedinputpipe(self._pipei)
82d564d5ac4f sshpeer: make instance attributes and methods internal
Gregory Szorc <gregory.szorc@gmail.com>
parents: 33763
diff changeset
205 self._pipei = doublepipe(self.ui, self._pipei, self._pipee)
82d564d5ac4f sshpeer: make instance attributes and methods internal
Gregory Szorc <gregory.szorc@gmail.com>
parents: 33763
diff changeset
206 self._pipeo = doublepipe(self.ui, self._pipeo, self._pipee)
25407
e461230cc95b sshpeer: use a 'bufferedinputpipe' for standard output of the ssh process
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 25244
diff changeset
207
33677
1908dc958639 ssh: fix flakey ssh errors on BSD systems
Durham Goode <durham@fb.com>
parents: 33657
diff changeset
208 def badresponse():
35131
8b1c887d52e7 sshpeer: add a configurable hint for the ssh error message
Zuzanna Mroczek <zuza@fb.com>
parents: 34264
diff changeset
209 msg = _("no suitable response from remote hg")
8b1c887d52e7 sshpeer: add a configurable hint for the ssh error message
Zuzanna Mroczek <zuza@fb.com>
parents: 34264
diff changeset
210 hint = self.ui.config("ui", "ssherrorhint")
8b1c887d52e7 sshpeer: add a configurable hint for the ssh error message
Zuzanna Mroczek <zuza@fb.com>
parents: 34264
diff changeset
211 self._abort(error.RepoError(msg, hint=hint))
33677
1908dc958639 ssh: fix flakey ssh errors on BSD systems
Durham Goode <durham@fb.com>
parents: 33657
diff changeset
212
1908dc958639 ssh: fix flakey ssh errors on BSD systems
Durham Goode <durham@fb.com>
parents: 33657
diff changeset
213 try:
1908dc958639 ssh: fix flakey ssh errors on BSD systems
Durham Goode <durham@fb.com>
parents: 33657
diff changeset
214 # skip any noise generated by remote shell
1908dc958639 ssh: fix flakey ssh errors on BSD systems
Durham Goode <durham@fb.com>
parents: 33657
diff changeset
215 self._callstream("hello")
1908dc958639 ssh: fix flakey ssh errors on BSD systems
Durham Goode <durham@fb.com>
parents: 33657
diff changeset
216 r = self._callstream("between", pairs=("%s-%s" % ("0"*40, "0"*40)))
1908dc958639 ssh: fix flakey ssh errors on BSD systems
Durham Goode <durham@fb.com>
parents: 33657
diff changeset
217 except IOError:
1908dc958639 ssh: fix flakey ssh errors on BSD systems
Durham Goode <durham@fb.com>
parents: 33657
diff changeset
218 badresponse()
1908dc958639 ssh: fix flakey ssh errors on BSD systems
Durham Goode <durham@fb.com>
parents: 33657
diff changeset
219
2420
144280f1578f ssh: gather initial output so we can do capability detection
Matt Mackall <mpm@selenic.com>
parents: 2230
diff changeset
220 lines = ["", "dummy"]
2046
d14497cbd668 Show remote ssh noise only with --debug and increase the limit to 500 lines.
Thomas Arendsen Hein <thomas@intevation.de>
parents: 2040
diff changeset
221 max_noise = 500
2420
144280f1578f ssh: gather initial output so we can do capability detection
Matt Mackall <mpm@selenic.com>
parents: 2230
diff changeset
222 while lines[-1] and max_noise:
33677
1908dc958639 ssh: fix flakey ssh errors on BSD systems
Durham Goode <durham@fb.com>
parents: 33657
diff changeset
223 try:
1908dc958639 ssh: fix flakey ssh errors on BSD systems
Durham Goode <durham@fb.com>
parents: 33657
diff changeset
224 l = r.readline()
34120
c037fd655b47 ssh: fix flakey ssh errors on BSD systems
Durham Goode <durham@fb.com>
parents: 33827
diff changeset
225 self._readerr()
33677
1908dc958639 ssh: fix flakey ssh errors on BSD systems
Durham Goode <durham@fb.com>
parents: 33657
diff changeset
226 if lines[-1] == "1\n" and l == "\n":
1908dc958639 ssh: fix flakey ssh errors on BSD systems
Durham Goode <durham@fb.com>
parents: 33657
diff changeset
227 break
1908dc958639 ssh: fix flakey ssh errors on BSD systems
Durham Goode <durham@fb.com>
parents: 33657
diff changeset
228 if l:
1908dc958639 ssh: fix flakey ssh errors on BSD systems
Durham Goode <durham@fb.com>
parents: 33657
diff changeset
229 self.ui.debug("remote: ", l)
1908dc958639 ssh: fix flakey ssh errors on BSD systems
Durham Goode <durham@fb.com>
parents: 33657
diff changeset
230 lines.append(l)
1908dc958639 ssh: fix flakey ssh errors on BSD systems
Durham Goode <durham@fb.com>
parents: 33657
diff changeset
231 max_noise -= 1
1908dc958639 ssh: fix flakey ssh errors on BSD systems
Durham Goode <durham@fb.com>
parents: 33657
diff changeset
232 except IOError:
1908dc958639 ssh: fix flakey ssh errors on BSD systems
Durham Goode <durham@fb.com>
parents: 33657
diff changeset
233 badresponse()
2040
cd7711268774 Don't enter an endless loop if remote hg doesn't answer, show remote noise.
Thomas Arendsen Hein <thomas@intevation.de>
parents: 2028
diff changeset
234 else:
33677
1908dc958639 ssh: fix flakey ssh errors on BSD systems
Durham Goode <durham@fb.com>
parents: 33657
diff changeset
235 badresponse()
2028
1f1fc418a96c ssh: skip noise generated by remote shell
Matt Mackall <mpm@selenic.com>
parents: 2019
diff changeset
236
17192
1ac628cd7113 peer: introduce real peer classes
Peter Arrenbrecht <peter.arrenbrecht@gmail.com>
parents: 16688
diff changeset
237 self._caps = set()
8210
344751cd8cb8 replace various uses of list.reverse()
Matt Mackall <mpm@selenic.com>
parents: 8150
diff changeset
238 for l in reversed(lines):
2421
a1cfe679192c ssh: add capability detection at startup
Matt Mackall <mpm@selenic.com>
parents: 2420
diff changeset
239 if l.startswith("capabilities:"):
17192
1ac628cd7113 peer: introduce real peer classes
Peter Arrenbrecht <peter.arrenbrecht@gmail.com>
parents: 16688
diff changeset
240 self._caps.update(l[:-1].split(":")[1].split())
2421
a1cfe679192c ssh: add capability detection at startup
Matt Mackall <mpm@selenic.com>
parents: 2420
diff changeset
241 break
a1cfe679192c ssh: add capability detection at startup
Matt Mackall <mpm@selenic.com>
parents: 2420
diff changeset
242
33789
82d564d5ac4f sshpeer: make instance attributes and methods internal
Gregory Szorc <gregory.szorc@gmail.com>
parents: 33763
diff changeset
243 def _readerr(self):
82d564d5ac4f sshpeer: make instance attributes and methods internal
Gregory Szorc <gregory.szorc@gmail.com>
parents: 33763
diff changeset
244 _forwardoutput(self.ui, self._pipee)
624
876333a295ff Add an sshrepository class and hg serve --stdio
Matt Mackall <mpm@selenic.com>
parents: 623
diff changeset
245
11586
ddaaaa23bb8f protocol: move basic ssh client commands to wirerepository
Matt Mackall <mpm@selenic.com>
parents: 11369
diff changeset
246 def _abort(self, exception):
33789
82d564d5ac4f sshpeer: make instance attributes and methods internal
Gregory Szorc <gregory.szorc@gmail.com>
parents: 33763
diff changeset
247 self._cleanup()
11586
ddaaaa23bb8f protocol: move basic ssh client commands to wirerepository
Matt Mackall <mpm@selenic.com>
parents: 11369
diff changeset
248 raise exception
ddaaaa23bb8f protocol: move basic ssh client commands to wirerepository
Matt Mackall <mpm@selenic.com>
parents: 11369
diff changeset
249
33789
82d564d5ac4f sshpeer: make instance attributes and methods internal
Gregory Szorc <gregory.szorc@gmail.com>
parents: 33763
diff changeset
250 def _cleanup(self):
82d564d5ac4f sshpeer: make instance attributes and methods internal
Gregory Szorc <gregory.szorc@gmail.com>
parents: 33763
diff changeset
251 if self._pipeo is None:
16688
cfb6682961b8 cleanup: replace naked excepts with more specific ones
Brodie Rao <brodie@sf.io>
parents: 16683
diff changeset
252 return
33789
82d564d5ac4f sshpeer: make instance attributes and methods internal
Gregory Szorc <gregory.szorc@gmail.com>
parents: 33763
diff changeset
253 self._pipeo.close()
82d564d5ac4f sshpeer: make instance attributes and methods internal
Gregory Szorc <gregory.szorc@gmail.com>
parents: 33763
diff changeset
254 self._pipei.close()
817
cf1d9a01dd92 Make ssh URL parsing more robust
mpm@selenic.com
parents: 816
diff changeset
255 try:
1358
20abfd48e21c Partially revert ssh change so we read all of remote ssh stream
Matt Mackall <mpm@selenic.com>
parents: 1357
diff changeset
256 # read the error descriptor until EOF
33789
82d564d5ac4f sshpeer: make instance attributes and methods internal
Gregory Szorc <gregory.szorc@gmail.com>
parents: 33763
diff changeset
257 for l in self._pipee:
1402
9d2c2e6b32b5 i18n part2: use '_' for all strings who are part of the user interface
Benoit Boissinot <benoit.boissinot@ens-lyon.org>
parents: 1400
diff changeset
258 self.ui.status(_("remote: "), l)
16688
cfb6682961b8 cleanup: replace naked excepts with more specific ones
Brodie Rao <brodie@sf.io>
parents: 16683
diff changeset
259 except (IOError, ValueError):
817
cf1d9a01dd92 Make ssh URL parsing more robust
mpm@selenic.com
parents: 816
diff changeset
260 pass
33789
82d564d5ac4f sshpeer: make instance attributes and methods internal
Gregory Szorc <gregory.szorc@gmail.com>
parents: 33763
diff changeset
261 self._pipee.close()
624
876333a295ff Add an sshrepository class and hg serve --stdio
Matt Mackall <mpm@selenic.com>
parents: 623
diff changeset
262
33789
82d564d5ac4f sshpeer: make instance attributes and methods internal
Gregory Szorc <gregory.szorc@gmail.com>
parents: 33763
diff changeset
263 __del__ = _cleanup
3034
2b0bc36a48d8 sshrepo: flush stderr before connecting to the hg server
Benoit Boissinot <benoit.boissinot@ens-lyon.org>
parents: 2859
diff changeset
264
28438
48fd02dac1d4 wireproto: make iterbatcher behave streamily over http(s)
Augie Fackler <augie@google.com>
parents: 27798
diff changeset
265 def _submitbatch(self, req):
29745
bb04f96df51c wireproto: consolidate code for obtaining "cmds" argument value
Gregory Szorc <gregory.szorc@gmail.com>
parents: 29739
diff changeset
266 rsp = self._callstream("batch", cmds=wireproto.encodebatchcmds(req))
28438
48fd02dac1d4 wireproto: make iterbatcher behave streamily over http(s)
Augie Fackler <augie@google.com>
parents: 27798
diff changeset
267 available = self._getamount()
48fd02dac1d4 wireproto: make iterbatcher behave streamily over http(s)
Augie Fackler <augie@google.com>
parents: 27798
diff changeset
268 # TODO this response parsing is probably suboptimal for large
48fd02dac1d4 wireproto: make iterbatcher behave streamily over http(s)
Augie Fackler <augie@google.com>
parents: 27798
diff changeset
269 # batches with large responses.
48fd02dac1d4 wireproto: make iterbatcher behave streamily over http(s)
Augie Fackler <augie@google.com>
parents: 27798
diff changeset
270 toread = min(available, 1024)
48fd02dac1d4 wireproto: make iterbatcher behave streamily over http(s)
Augie Fackler <augie@google.com>
parents: 27798
diff changeset
271 work = rsp.read(toread)
48fd02dac1d4 wireproto: make iterbatcher behave streamily over http(s)
Augie Fackler <augie@google.com>
parents: 27798
diff changeset
272 available -= toread
48fd02dac1d4 wireproto: make iterbatcher behave streamily over http(s)
Augie Fackler <augie@google.com>
parents: 27798
diff changeset
273 chunk = work
48fd02dac1d4 wireproto: make iterbatcher behave streamily over http(s)
Augie Fackler <augie@google.com>
parents: 27798
diff changeset
274 while chunk:
48fd02dac1d4 wireproto: make iterbatcher behave streamily over http(s)
Augie Fackler <augie@google.com>
parents: 27798
diff changeset
275 while ';' in work:
48fd02dac1d4 wireproto: make iterbatcher behave streamily over http(s)
Augie Fackler <augie@google.com>
parents: 27798
diff changeset
276 one, work = work.split(';', 1)
48fd02dac1d4 wireproto: make iterbatcher behave streamily over http(s)
Augie Fackler <augie@google.com>
parents: 27798
diff changeset
277 yield wireproto.unescapearg(one)
48fd02dac1d4 wireproto: make iterbatcher behave streamily over http(s)
Augie Fackler <augie@google.com>
parents: 27798
diff changeset
278 toread = min(available, 1024)
48fd02dac1d4 wireproto: make iterbatcher behave streamily over http(s)
Augie Fackler <augie@google.com>
parents: 27798
diff changeset
279 chunk = rsp.read(toread)
48fd02dac1d4 wireproto: make iterbatcher behave streamily over http(s)
Augie Fackler <augie@google.com>
parents: 27798
diff changeset
280 available -= toread
48fd02dac1d4 wireproto: make iterbatcher behave streamily over http(s)
Augie Fackler <augie@google.com>
parents: 27798
diff changeset
281 work += chunk
48fd02dac1d4 wireproto: make iterbatcher behave streamily over http(s)
Augie Fackler <augie@google.com>
parents: 27798
diff changeset
282 yield wireproto.unescapearg(work)
48fd02dac1d4 wireproto: make iterbatcher behave streamily over http(s)
Augie Fackler <augie@google.com>
parents: 27798
diff changeset
283
11589
e8d22fe2ddab protocol: clean up call-like functions in http and ssh clients
Matt Mackall <mpm@selenic.com>
parents: 11588
diff changeset
284 def _callstream(self, cmd, **args):
33112
05906b8e1d23 py3: use pycompat.byteskwargs() to convert kwargs' keys to bytes
Pulkit Goyal <7895pulkit@gmail.com>
parents: 32062
diff changeset
285 args = pycompat.byteskwargs(args)
9467
4c041f1ee1b4 do not attempt to translate ui.debug output
Martin Geisler <mg@lazybytes.net>
parents: 8563
diff changeset
286 self.ui.debug("sending %s command\n" % cmd)
33789
82d564d5ac4f sshpeer: make instance attributes and methods internal
Gregory Szorc <gregory.szorc@gmail.com>
parents: 33763
diff changeset
287 self._pipeo.write("%s\n" % cmd)
13721
3458c15ab2f0 wireproto: fix handling of '*' args for HTTP and SSH
Peter Arrenbrecht <peter.arrenbrecht@gmail.com>
parents: 13464
diff changeset
288 _func, names = wireproto.commands[cmd]
3458c15ab2f0 wireproto: fix handling of '*' args for HTTP and SSH
Peter Arrenbrecht <peter.arrenbrecht@gmail.com>
parents: 13464
diff changeset
289 keys = names.split()
3458c15ab2f0 wireproto: fix handling of '*' args for HTTP and SSH
Peter Arrenbrecht <peter.arrenbrecht@gmail.com>
parents: 13464
diff changeset
290 wireargs = {}
3458c15ab2f0 wireproto: fix handling of '*' args for HTTP and SSH
Peter Arrenbrecht <peter.arrenbrecht@gmail.com>
parents: 13464
diff changeset
291 for k in keys:
3458c15ab2f0 wireproto: fix handling of '*' args for HTTP and SSH
Peter Arrenbrecht <peter.arrenbrecht@gmail.com>
parents: 13464
diff changeset
292 if k == '*':
3458c15ab2f0 wireproto: fix handling of '*' args for HTTP and SSH
Peter Arrenbrecht <peter.arrenbrecht@gmail.com>
parents: 13464
diff changeset
293 wireargs['*'] = args
3458c15ab2f0 wireproto: fix handling of '*' args for HTTP and SSH
Peter Arrenbrecht <peter.arrenbrecht@gmail.com>
parents: 13464
diff changeset
294 break
3458c15ab2f0 wireproto: fix handling of '*' args for HTTP and SSH
Peter Arrenbrecht <peter.arrenbrecht@gmail.com>
parents: 13464
diff changeset
295 else:
3458c15ab2f0 wireproto: fix handling of '*' args for HTTP and SSH
Peter Arrenbrecht <peter.arrenbrecht@gmail.com>
parents: 13464
diff changeset
296 wireargs[k] = args[k]
3458c15ab2f0 wireproto: fix handling of '*' args for HTTP and SSH
Peter Arrenbrecht <peter.arrenbrecht@gmail.com>
parents: 13464
diff changeset
297 del args[k]
3458c15ab2f0 wireproto: fix handling of '*' args for HTTP and SSH
Peter Arrenbrecht <peter.arrenbrecht@gmail.com>
parents: 13464
diff changeset
298 for k, v in sorted(wireargs.iteritems()):
33789
82d564d5ac4f sshpeer: make instance attributes and methods internal
Gregory Szorc <gregory.szorc@gmail.com>
parents: 33763
diff changeset
299 self._pipeo.write("%s %d\n" % (k, len(v)))
13721
3458c15ab2f0 wireproto: fix handling of '*' args for HTTP and SSH
Peter Arrenbrecht <peter.arrenbrecht@gmail.com>
parents: 13464
diff changeset
300 if isinstance(v, dict):
3458c15ab2f0 wireproto: fix handling of '*' args for HTTP and SSH
Peter Arrenbrecht <peter.arrenbrecht@gmail.com>
parents: 13464
diff changeset
301 for dk, dv in v.iteritems():
33789
82d564d5ac4f sshpeer: make instance attributes and methods internal
Gregory Szorc <gregory.szorc@gmail.com>
parents: 33763
diff changeset
302 self._pipeo.write("%s %d\n" % (dk, len(dv)))
82d564d5ac4f sshpeer: make instance attributes and methods internal
Gregory Szorc <gregory.szorc@gmail.com>
parents: 33763
diff changeset
303 self._pipeo.write(dv)
13721
3458c15ab2f0 wireproto: fix handling of '*' args for HTTP and SSH
Peter Arrenbrecht <peter.arrenbrecht@gmail.com>
parents: 13464
diff changeset
304 else:
33789
82d564d5ac4f sshpeer: make instance attributes and methods internal
Gregory Szorc <gregory.szorc@gmail.com>
parents: 33763
diff changeset
305 self._pipeo.write(v)
82d564d5ac4f sshpeer: make instance attributes and methods internal
Gregory Szorc <gregory.szorc@gmail.com>
parents: 33763
diff changeset
306 self._pipeo.flush()
624
876333a295ff Add an sshrepository class and hg serve --stdio
Matt Mackall <mpm@selenic.com>
parents: 623
diff changeset
307
33789
82d564d5ac4f sshpeer: make instance attributes and methods internal
Gregory Szorc <gregory.szorc@gmail.com>
parents: 33763
diff changeset
308 return self._pipei
624
876333a295ff Add an sshrepository class and hg serve --stdio
Matt Mackall <mpm@selenic.com>
parents: 623
diff changeset
309
20905
167047ba3cfa wireproto: drop the _decompress method in favor a new call type
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 20794
diff changeset
310 def _callcompressable(self, cmd, **args):
167047ba3cfa wireproto: drop the _decompress method in favor a new call type
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 20794
diff changeset
311 return self._callstream(cmd, **args)
167047ba3cfa wireproto: drop the _decompress method in favor a new call type
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 20794
diff changeset
312
11589
e8d22fe2ddab protocol: clean up call-like functions in http and ssh clients
Matt Mackall <mpm@selenic.com>
parents: 11588
diff changeset
313 def _call(self, cmd, **args):
e8d22fe2ddab protocol: clean up call-like functions in http and ssh clients
Matt Mackall <mpm@selenic.com>
parents: 11588
diff changeset
314 self._callstream(cmd, **args)
5978
7939c71f3132 sshrepo: be more careful while reading data
Alexis S. L. Carvalho <alexis@cecm.usp.br>
parents: 5293
diff changeset
315 return self._recv()
7939c71f3132 sshrepo: be more careful while reading data
Alexis S. L. Carvalho <alexis@cecm.usp.br>
parents: 5293
diff changeset
316
11592
26e0782b8380 protocol: unify client unbundle support
Matt Mackall <mpm@selenic.com>
parents: 11591
diff changeset
317 def _callpush(self, cmd, fp, **args):
26e0782b8380 protocol: unify client unbundle support
Matt Mackall <mpm@selenic.com>
parents: 11591
diff changeset
318 r = self._call(cmd, **args)
26e0782b8380 protocol: unify client unbundle support
Matt Mackall <mpm@selenic.com>
parents: 11591
diff changeset
319 if r:
26e0782b8380 protocol: unify client unbundle support
Matt Mackall <mpm@selenic.com>
parents: 11591
diff changeset
320 return '', r
29739
0dbd788a2dfd sshpeer: use `iter(callable, sentinel)` instead of while True
Augie Fackler <augie@google.com>
parents: 29389
diff changeset
321 for d in iter(lambda: fp.read(4096), ''):
11592
26e0782b8380 protocol: unify client unbundle support
Matt Mackall <mpm@selenic.com>
parents: 11591
diff changeset
322 self._send(d)
26e0782b8380 protocol: unify client unbundle support
Matt Mackall <mpm@selenic.com>
parents: 11591
diff changeset
323 self._send("", flush=True)
26e0782b8380 protocol: unify client unbundle support
Matt Mackall <mpm@selenic.com>
parents: 11591
diff changeset
324 r = self._recv()
26e0782b8380 protocol: unify client unbundle support
Matt Mackall <mpm@selenic.com>
parents: 11591
diff changeset
325 if r:
26e0782b8380 protocol: unify client unbundle support
Matt Mackall <mpm@selenic.com>
parents: 11591
diff changeset
326 return '', r
26e0782b8380 protocol: unify client unbundle support
Matt Mackall <mpm@selenic.com>
parents: 11591
diff changeset
327 return self._recv(), ''
26e0782b8380 protocol: unify client unbundle support
Matt Mackall <mpm@selenic.com>
parents: 11591
diff changeset
328
21073
83ce71ef7804 sshpeer: add implementation of _calltwowaystream
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 20905
diff changeset
329 def _calltwowaystream(self, cmd, fp, **args):
83ce71ef7804 sshpeer: add implementation of _calltwowaystream
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 20905
diff changeset
330 r = self._call(cmd, **args)
83ce71ef7804 sshpeer: add implementation of _calltwowaystream
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 20905
diff changeset
331 if r:
83ce71ef7804 sshpeer: add implementation of _calltwowaystream
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 20905
diff changeset
332 # XXX needs to be made better
29389
98e8313dcd9e i18n: translate abort messages
liscju <piotr.listkiewicz@gmail.com>
parents: 28438
diff changeset
333 raise error.Abort(_('unexpected remote reply: %s') % r)
29739
0dbd788a2dfd sshpeer: use `iter(callable, sentinel)` instead of while True
Augie Fackler <augie@google.com>
parents: 29389
diff changeset
334 for d in iter(lambda: fp.read(4096), ''):
21073
83ce71ef7804 sshpeer: add implementation of _calltwowaystream
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 20905
diff changeset
335 self._send(d)
83ce71ef7804 sshpeer: add implementation of _calltwowaystream
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 20905
diff changeset
336 self._send("", flush=True)
33789
82d564d5ac4f sshpeer: make instance attributes and methods internal
Gregory Szorc <gregory.szorc@gmail.com>
parents: 33763
diff changeset
337 return self._pipei
11591
0d9cb3f3b0a1 protocol: unify client changegroup methods
Matt Mackall <mpm@selenic.com>
parents: 11590
diff changeset
338
28438
48fd02dac1d4 wireproto: make iterbatcher behave streamily over http(s)
Augie Fackler <augie@google.com>
parents: 27798
diff changeset
339 def _getamount(self):
33789
82d564d5ac4f sshpeer: make instance attributes and methods internal
Gregory Szorc <gregory.szorc@gmail.com>
parents: 33763
diff changeset
340 l = self._pipei.readline()
15017
f4522df38c65 wireproto: add out-of-band error class to allow remote repo to report errors
Andrew Pritchard <andrewp@fogcreek.com>
parents: 14537
diff changeset
341 if l == '\n':
33789
82d564d5ac4f sshpeer: make instance attributes and methods internal
Gregory Szorc <gregory.szorc@gmail.com>
parents: 33763
diff changeset
342 self._readerr()
25243
d65243d28749 sshpeer: break "OutOfBandError" feature for ssh (BC)
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 23684
diff changeset
343 msg = _('check previous remote output')
d65243d28749 sshpeer: break "OutOfBandError" feature for ssh (BC)
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 23684
diff changeset
344 self._abort(error.OutOfBandError(hint=msg))
33789
82d564d5ac4f sshpeer: make instance attributes and methods internal
Gregory Szorc <gregory.szorc@gmail.com>
parents: 33763
diff changeset
345 self._readerr()
646
342927da4f4c Show remote client output with "remote:"
Matt Mackall <mpm@selenic.com>
parents: 644
diff changeset
346 try:
28438
48fd02dac1d4 wireproto: make iterbatcher behave streamily over http(s)
Augie Fackler <augie@google.com>
parents: 27798
diff changeset
347 return int(l)
14004
97ed99d1f419 eliminate various naked except clauses
Idan Kamara <idankk86@gmail.com>
parents: 13827
diff changeset
348 except ValueError:
11590
0b15aee0a306 protocol: convert some ssh abort calls
Matt Mackall <mpm@selenic.com>
parents: 11589
diff changeset
349 self._abort(error.ResponseError(_("unexpected response:"), l))
28438
48fd02dac1d4 wireproto: make iterbatcher behave streamily over http(s)
Augie Fackler <augie@google.com>
parents: 27798
diff changeset
350
48fd02dac1d4 wireproto: make iterbatcher behave streamily over http(s)
Augie Fackler <augie@google.com>
parents: 27798
diff changeset
351 def _recv(self):
33789
82d564d5ac4f sshpeer: make instance attributes and methods internal
Gregory Szorc <gregory.szorc@gmail.com>
parents: 33763
diff changeset
352 return self._pipei.read(self._getamount())
5978
7939c71f3132 sshrepo: be more careful while reading data
Alexis S. L. Carvalho <alexis@cecm.usp.br>
parents: 5293
diff changeset
353
7939c71f3132 sshrepo: be more careful while reading data
Alexis S. L. Carvalho <alexis@cecm.usp.br>
parents: 5293
diff changeset
354 def _send(self, data, flush=False):
33789
82d564d5ac4f sshpeer: make instance attributes and methods internal
Gregory Szorc <gregory.szorc@gmail.com>
parents: 33763
diff changeset
355 self._pipeo.write("%d\n" % len(data))
5978
7939c71f3132 sshrepo: be more careful while reading data
Alexis S. L. Carvalho <alexis@cecm.usp.br>
parents: 5293
diff changeset
356 if data:
33789
82d564d5ac4f sshpeer: make instance attributes and methods internal
Gregory Szorc <gregory.szorc@gmail.com>
parents: 33763
diff changeset
357 self._pipeo.write(data)
5978
7939c71f3132 sshrepo: be more careful while reading data
Alexis S. L. Carvalho <alexis@cecm.usp.br>
parents: 5293
diff changeset
358 if flush:
33789
82d564d5ac4f sshpeer: make instance attributes and methods internal
Gregory Szorc <gregory.szorc@gmail.com>
parents: 33763
diff changeset
359 self._pipeo.flush()
82d564d5ac4f sshpeer: make instance attributes and methods internal
Gregory Szorc <gregory.szorc@gmail.com>
parents: 33763
diff changeset
360 self._readerr()
2612
ffb895f16925 add support for streaming clone.
Vadim Gelfer <vadim.gelfer@gmail.com>
parents: 2549
diff changeset
361
17192
1ac628cd7113 peer: introduce real peer classes
Peter Arrenbrecht <peter.arrenbrecht@gmail.com>
parents: 16688
diff changeset
362 instance = sshpeer