Mercurial > public > mercurial-scm > hg-stable

annotate mercurial/httppeer.py @ 35924:197d10e157ce

Find changesets by keywords (author, files, the commit message), revision number or hash, or revset expression.
httppeer: remove support for connecting to <0.9.1 servers (BC) Previously, HTTP wire protocol clients would attempt a "capabilities" wire protocol command. If that failed, they would fall back to issuing a "between" command. The "capabilities" command was added in Mercurial 0.9.1 (released July 2006). The "between" command has been present for as long as the wire protocol has existed. So if the "between" command failed, it was safe to assume that the remote could not speak any version of the Mercurial wire protocol. The "between" fallback was added in 395a84f78736 in 2011. Before that changeset, Mercurial would *always* issue the "between" command and would issue "capabilities" if capabilities were requested. At that time, many connections would issue "capabilities" eventually, so it was decided to issue "capabilities" by default and fall back to "between" if that failed. This saved a round trip when connecting to modern servers while still preserving compatibility with legacy servers. Fast forward ~7 years. Mercurial servers supporting "capabilities" have been around for over a decade. If modern clients are connecting to <0.9.1 servers, they are getting a bad experience. They may even be getting bad data (an old server is vulnerable to numerous security issues and could have been p0wned, leading to a Mercurial repository serving backdoors or other badness). In addition, the fallback can harm experience for modern servers. If a client experiences an intermittent HTTP request failure (due to bad network, etc) and falls back to a "between" that works, it would assume an empty capability set and would attempt to communicate with the repository using a very ancient wire protocol. Auditing HTTP logs for hg.mozilla.org, I did find a handful of requests for the null range of the "between" command. However, requests can be days apart. And when I do see requests, they come in batches. Those batches seem to correlate to spikes of HTTP 500 or other server/network events. So I think these requests are fallbacks from failed "capabilities" requests and not from old clients. If you need even more evidence to discontinue support, apparently we have no test coverage for communicating with servers not supporting "capabilities." I know this because all tests pass with the "between" fallback removed. Finally, server-side support for <0.9.1 pushing (the "addchangegroup" wire protocol command along with locking-related commands) was dropped from the HTTP client in fda0867cfe03 in 2017 and the SSH client in 9f6e0e7ef828 in 2015. I think this all adds up to enough justification for removing client support for communicating with servers not supporting "capabilities." So this commit removes that fallback. Differential Revision: https://phab.mercurial-scm.org/D2001
author Gregory Szorc <gregory.szorc@gmail.com>
date Fri, 02 Feb 2018 13:13:46 -0800
parents 0c4b23ccf1a5
children e4ccd7a69f77
Ignore whitespace changes - Everywhere: Within whitespace: At end of lines:
rev   line source
17192
1ac628cd7113 peer: introduce real peer classes
Peter Arrenbrecht <peter.arrenbrecht@gmail.com>
parents: 15246
diff changeset
1 # httppeer.py - HTTP repository proxy classes for mercurial
0
9117c6561b0b Add back links from file revisions to changeset revisions
mpm@selenic.com
parents:
diff changeset
2 #
2859
345bac2bc4ec update copyrights.
Vadim Gelfer <vadim.gelfer@gmail.com>
parents: 2740
diff changeset
3 # Copyright 2005, 2006 Matt Mackall <mpm@selenic.com>
345bac2bc4ec update copyrights.
Vadim Gelfer <vadim.gelfer@gmail.com>
parents: 2740
diff changeset
4 # Copyright 2006 Vadim Gelfer <vadim.gelfer@gmail.com>
0
9117c6561b0b Add back links from file revisions to changeset revisions
mpm@selenic.com
parents:
diff changeset
5 #
8225
46293a0c7e9f updated license to be explicit about GPL version 2
Martin Geisler <mg@lazybytes.net>
parents: 8206
diff changeset
6 # This software may be used and distributed according to the terms of the
10263
25e572394f5c Update license to GPLv2+
Matt Mackall <mpm@selenic.com>
parents: 9878
diff changeset
7 # GNU General Public License version 2 or any later version.
0
9117c6561b0b Add back links from file revisions to changeset revisions
mpm@selenic.com
parents:
diff changeset
8
25954
7bbdb78d2842 httppeer: use absolute_import
Gregory Szorc <gregory.szorc@gmail.com>
parents: 25660
diff changeset
9 from __future__ import absolute_import
7bbdb78d2842 httppeer: use absolute_import
Gregory Szorc <gregory.szorc@gmail.com>
parents: 25660
diff changeset
10
7bbdb78d2842 httppeer: use absolute_import
Gregory Szorc <gregory.szorc@gmail.com>
parents: 25660
diff changeset
11 import errno
33842
3c91cc0c5fde httppeer: add support for httppostargs when we're sending a file
Augie Fackler <augie@google.com>
parents: 33827
diff changeset
12 import io
25954
7bbdb78d2842 httppeer: use absolute_import
Gregory Szorc <gregory.szorc@gmail.com>
parents: 25660
diff changeset
13 import os
7bbdb78d2842 httppeer: use absolute_import
Gregory Szorc <gregory.szorc@gmail.com>
parents: 25660
diff changeset
14 import socket
30763
a520aefb96f1 httppeer: advertise and support application/mercurial-0.2
Gregory Szorc <gregory.szorc@gmail.com>
parents: 30759
diff changeset
15 import struct
21074
f8a0d82b0463 httppeer: support for _calltwowaystream
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 20905
diff changeset
16 import tempfile
25954
7bbdb78d2842 httppeer: use absolute_import
Gregory Szorc <gregory.szorc@gmail.com>
parents: 25660
diff changeset
17
7bbdb78d2842 httppeer: use absolute_import
Gregory Szorc <gregory.szorc@gmail.com>
parents: 25660
diff changeset
18 from .i18n import _
7bbdb78d2842 httppeer: use absolute_import
Gregory Szorc <gregory.szorc@gmail.com>
parents: 25660
diff changeset
19 from . import (
28666
ae53ecc47414 bundle: move writebundle() from changegroup.py to bundle2.py (API)
Martin von Zweigbergk <martinvonz@google.com>
parents: 28530
diff changeset
20 bundle2,
25954
7bbdb78d2842 httppeer: use absolute_import
Gregory Szorc <gregory.szorc@gmail.com>
parents: 25660
diff changeset
21 error,
7bbdb78d2842 httppeer: use absolute_import
Gregory Szorc <gregory.szorc@gmail.com>
parents: 25660
diff changeset
22 httpconnection,
30944
48dea083f66d py3: convert the mode argument of os.fdopen to unicodes (1 of 2)
Pulkit Goyal <7895pulkit@gmail.com>
parents: 30763
diff changeset
23 pycompat,
25954
7bbdb78d2842 httppeer: use absolute_import
Gregory Szorc <gregory.szorc@gmail.com>
parents: 25660
diff changeset
24 statichttprepo,
7bbdb78d2842 httppeer: use absolute_import
Gregory Szorc <gregory.szorc@gmail.com>
parents: 25660
diff changeset
25 url,
7bbdb78d2842 httppeer: use absolute_import
Gregory Szorc <gregory.szorc@gmail.com>
parents: 25660
diff changeset
26 util,
7bbdb78d2842 httppeer: use absolute_import
Gregory Szorc <gregory.szorc@gmail.com>
parents: 25660
diff changeset
27 wireproto,
7bbdb78d2842 httppeer: use absolute_import
Gregory Szorc <gregory.szorc@gmail.com>
parents: 25660
diff changeset
28 )
4678
a814a5b11fff Work around urllib2 digest auth bug with Python < 2.5
Alexis S. L. Carvalho <alexis@cecm.usp.br>
parents: 4633
diff changeset
29
29455
0c741fd6158a py3: conditionalize httplib import
Pulkit Goyal <7895pulkit@gmail.com>
parents: 29241
diff changeset
30 httplib = util.httplib
28883
032c4c2f802a pycompat: switch to util.urlreq/util.urlerr for py3 compat
timeless <timeless@mozdev.org>
parents: 28666
diff changeset
31 urlerr = util.urlerr
032c4c2f802a pycompat: switch to util.urlreq/util.urlerr for py3 compat
timeless <timeless@mozdev.org>
parents: 28666
diff changeset
32 urlreq = util.urlreq
032c4c2f802a pycompat: switch to util.urlreq/util.urlerr for py3 compat
timeless <timeless@mozdev.org>
parents: 28666
diff changeset
33
30759
3f5f0c98cd18 httppeer: extract code for HTTP header spanning
Gregory Szorc <gregory.szorc@gmail.com>
parents: 30569
diff changeset
34 def encodevalueinheaders(value, header, limit):
3f5f0c98cd18 httppeer: extract code for HTTP header spanning
Gregory Szorc <gregory.szorc@gmail.com>
parents: 30569
diff changeset
35 """Encode a string value into multiple HTTP headers.
3f5f0c98cd18 httppeer: extract code for HTTP header spanning
Gregory Szorc <gregory.szorc@gmail.com>
parents: 30569
diff changeset
36
3f5f0c98cd18 httppeer: extract code for HTTP header spanning
Gregory Szorc <gregory.szorc@gmail.com>
parents: 30569
diff changeset
37 ``value`` will be encoded into 1 or more HTTP headers with the names
3f5f0c98cd18 httppeer: extract code for HTTP header spanning
Gregory Szorc <gregory.szorc@gmail.com>
parents: 30569
diff changeset
38 ``header-<N>`` where ``<N>`` is an integer starting at 1. Each header
3f5f0c98cd18 httppeer: extract code for HTTP header spanning
Gregory Szorc <gregory.szorc@gmail.com>
parents: 30569
diff changeset
39 name + value will be at most ``limit`` bytes long.
3f5f0c98cd18 httppeer: extract code for HTTP header spanning
Gregory Szorc <gregory.szorc@gmail.com>
parents: 30569
diff changeset
40
34732
67e9678efd98 httppeer: always produce native str header keys and values
Augie Fackler <augie@google.com>
parents: 34725
diff changeset
41 Returns an iterable of 2-tuples consisting of header names and
67e9678efd98 httppeer: always produce native str header keys and values
Augie Fackler <augie@google.com>
parents: 34725
diff changeset
42 values as native strings.
30759
3f5f0c98cd18 httppeer: extract code for HTTP header spanning
Gregory Szorc <gregory.szorc@gmail.com>
parents: 30569
diff changeset
43 """
34732
67e9678efd98 httppeer: always produce native str header keys and values
Augie Fackler <augie@google.com>
parents: 34725
diff changeset
44 # HTTP Headers are ASCII. Python 3 requires them to be unicodes,
67e9678efd98 httppeer: always produce native str header keys and values
Augie Fackler <augie@google.com>
parents: 34725
diff changeset
45 # not bytes. This function always takes bytes in as arguments.
67e9678efd98 httppeer: always produce native str header keys and values
Augie Fackler <augie@google.com>
parents: 34725
diff changeset
46 fmt = pycompat.strurl(header) + r'-%s'
67e9678efd98 httppeer: always produce native str header keys and values
Augie Fackler <augie@google.com>
parents: 34725
diff changeset
47 # Note: it is *NOT* a bug that the last bit here is a bytestring
67e9678efd98 httppeer: always produce native str header keys and values
Augie Fackler <augie@google.com>
parents: 34725
diff changeset
48 # and not a unicode: we're just getting the encoded length anyway,
67e9678efd98 httppeer: always produce native str header keys and values
Augie Fackler <augie@google.com>
parents: 34725
diff changeset
49 # and using an r-string to make it portable between Python 2 and 3
67e9678efd98 httppeer: always produce native str header keys and values
Augie Fackler <augie@google.com>
parents: 34725
diff changeset
50 # doesn't work because then the \r is a literal backslash-r
67e9678efd98 httppeer: always produce native str header keys and values
Augie Fackler <augie@google.com>
parents: 34725
diff changeset
51 # instead of a carriage return.
67e9678efd98 httppeer: always produce native str header keys and values
Augie Fackler <augie@google.com>
parents: 34725
diff changeset
52 valuelen = limit - len(fmt % r'000') - len(': \r\n')
30759
3f5f0c98cd18 httppeer: extract code for HTTP header spanning
Gregory Szorc <gregory.szorc@gmail.com>
parents: 30569
diff changeset
53 result = []
3f5f0c98cd18 httppeer: extract code for HTTP header spanning
Gregory Szorc <gregory.szorc@gmail.com>
parents: 30569
diff changeset
54
3f5f0c98cd18 httppeer: extract code for HTTP header spanning
Gregory Szorc <gregory.szorc@gmail.com>
parents: 30569
diff changeset
55 n = 0
3f5f0c98cd18 httppeer: extract code for HTTP header spanning
Gregory Szorc <gregory.szorc@gmail.com>
parents: 30569
diff changeset
56 for i in xrange(0, len(value), valuelen):
3f5f0c98cd18 httppeer: extract code for HTTP header spanning
Gregory Szorc <gregory.szorc@gmail.com>
parents: 30569
diff changeset
57 n += 1
34732
67e9678efd98 httppeer: always produce native str header keys and values
Augie Fackler <augie@google.com>
parents: 34725
diff changeset
58 result.append((fmt % str(n), pycompat.strurl(value[i:i + valuelen])))
30759
3f5f0c98cd18 httppeer: extract code for HTTP header spanning
Gregory Szorc <gregory.szorc@gmail.com>
parents: 30569
diff changeset
59
3f5f0c98cd18 httppeer: extract code for HTTP header spanning
Gregory Szorc <gregory.szorc@gmail.com>
parents: 30569
diff changeset
60 return result
3f5f0c98cd18 httppeer: extract code for HTTP header spanning
Gregory Szorc <gregory.szorc@gmail.com>
parents: 30569
diff changeset
61
32002
bf855efe5664 httppeer: wrap HTTPResponse.read() globally
Gregory Szorc <gregory.szorc@gmail.com>
parents: 30944
diff changeset
62 def _wraphttpresponse(resp):
bf855efe5664 httppeer: wrap HTTPResponse.read() globally
Gregory Szorc <gregory.szorc@gmail.com>
parents: 30944
diff changeset
63 """Wrap an HTTPResponse with common error handlers.
bf855efe5664 httppeer: wrap HTTPResponse.read() globally
Gregory Szorc <gregory.szorc@gmail.com>
parents: 30944
diff changeset
64
bf855efe5664 httppeer: wrap HTTPResponse.read() globally
Gregory Szorc <gregory.szorc@gmail.com>
parents: 30944
diff changeset
65 This ensures that any I/O from any consumer raises the appropriate
bf855efe5664 httppeer: wrap HTTPResponse.read() globally
Gregory Szorc <gregory.szorc@gmail.com>
parents: 30944
diff changeset
66 error and messaging.
bf855efe5664 httppeer: wrap HTTPResponse.read() globally
Gregory Szorc <gregory.szorc@gmail.com>
parents: 30944
diff changeset
67 """
bf855efe5664 httppeer: wrap HTTPResponse.read() globally
Gregory Szorc <gregory.szorc@gmail.com>
parents: 30944
diff changeset
68 origread = resp.read
bf855efe5664 httppeer: wrap HTTPResponse.read() globally
Gregory Szorc <gregory.szorc@gmail.com>
parents: 30944
diff changeset
69
bf855efe5664 httppeer: wrap HTTPResponse.read() globally
Gregory Szorc <gregory.szorc@gmail.com>
parents: 30944
diff changeset
70 class readerproxy(resp.__class__):
bf855efe5664 httppeer: wrap HTTPResponse.read() globally
Gregory Szorc <gregory.szorc@gmail.com>
parents: 30944
diff changeset
71 def read(self, size=None):
bf855efe5664 httppeer: wrap HTTPResponse.read() globally
Gregory Szorc <gregory.szorc@gmail.com>
parents: 30944
diff changeset
72 try:
bf855efe5664 httppeer: wrap HTTPResponse.read() globally
Gregory Szorc <gregory.szorc@gmail.com>
parents: 30944
diff changeset
73 return origread(size)
bf855efe5664 httppeer: wrap HTTPResponse.read() globally
Gregory Szorc <gregory.szorc@gmail.com>
parents: 30944
diff changeset
74 except httplib.IncompleteRead as e:
bf855efe5664 httppeer: wrap HTTPResponse.read() globally
Gregory Szorc <gregory.szorc@gmail.com>
parents: 30944
diff changeset
75 # e.expected is an integer if length known or None otherwise.
bf855efe5664 httppeer: wrap HTTPResponse.read() globally
Gregory Szorc <gregory.szorc@gmail.com>
parents: 30944
diff changeset
76 if e.expected:
bf855efe5664 httppeer: wrap HTTPResponse.read() globally
Gregory Szorc <gregory.szorc@gmail.com>
parents: 30944
diff changeset
77 msg = _('HTTP request error (incomplete response; '
bf855efe5664 httppeer: wrap HTTPResponse.read() globally
Gregory Szorc <gregory.szorc@gmail.com>
parents: 30944
diff changeset
78 'expected %d bytes got %d)') % (e.expected,
bf855efe5664 httppeer: wrap HTTPResponse.read() globally
Gregory Szorc <gregory.szorc@gmail.com>
parents: 30944
diff changeset
79 len(e.partial))
bf855efe5664 httppeer: wrap HTTPResponse.read() globally
Gregory Szorc <gregory.szorc@gmail.com>
parents: 30944
diff changeset
80 else:
bf855efe5664 httppeer: wrap HTTPResponse.read() globally
Gregory Szorc <gregory.szorc@gmail.com>
parents: 30944
diff changeset
81 msg = _('HTTP request error (incomplete response)')
bf855efe5664 httppeer: wrap HTTPResponse.read() globally
Gregory Szorc <gregory.szorc@gmail.com>
parents: 30944
diff changeset
82
32023
a29580905771 error: rename RichIOError to PeerTransportError
Gregory Szorc <gregory.szorc@gmail.com>
parents: 32022
diff changeset
83 raise error.PeerTransportError(
32002
bf855efe5664 httppeer: wrap HTTPResponse.read() globally
Gregory Szorc <gregory.szorc@gmail.com>
parents: 30944
diff changeset
84 msg,
bf855efe5664 httppeer: wrap HTTPResponse.read() globally
Gregory Szorc <gregory.szorc@gmail.com>
parents: 30944
diff changeset
85 hint=_('this may be an intermittent network failure; '
bf855efe5664 httppeer: wrap HTTPResponse.read() globally
Gregory Szorc <gregory.szorc@gmail.com>
parents: 30944
diff changeset
86 'if the error persists, consider contacting the '
bf855efe5664 httppeer: wrap HTTPResponse.read() globally
Gregory Szorc <gregory.szorc@gmail.com>
parents: 30944
diff changeset
87 'network or server operator'))
bf855efe5664 httppeer: wrap HTTPResponse.read() globally
Gregory Szorc <gregory.szorc@gmail.com>
parents: 30944
diff changeset
88 except httplib.HTTPException as e:
32023
a29580905771 error: rename RichIOError to PeerTransportError
Gregory Szorc <gregory.szorc@gmail.com>
parents: 32022
diff changeset
89 raise error.PeerTransportError(
32002
bf855efe5664 httppeer: wrap HTTPResponse.read() globally
Gregory Szorc <gregory.szorc@gmail.com>
parents: 30944
diff changeset
90 _('HTTP request error (%s)') % e,
32086
b59a292d0a53 httppeer: unify hint message for PeerTransportError
FUJIWARA Katsunori <foozy@lares.dti.ne.jp>
parents: 32023
diff changeset
91 hint=_('this may be an intermittent network failure; '
32002
bf855efe5664 httppeer: wrap HTTPResponse.read() globally
Gregory Szorc <gregory.szorc@gmail.com>
parents: 30944
diff changeset
92 'if the error persists, consider contacting the '
bf855efe5664 httppeer: wrap HTTPResponse.read() globally
Gregory Szorc <gregory.szorc@gmail.com>
parents: 30944
diff changeset
93 'network or server operator'))
bf855efe5664 httppeer: wrap HTTPResponse.read() globally
Gregory Szorc <gregory.szorc@gmail.com>
parents: 30944
diff changeset
94
bf855efe5664 httppeer: wrap HTTPResponse.read() globally
Gregory Szorc <gregory.szorc@gmail.com>
parents: 30944
diff changeset
95 resp.__class__ = readerproxy
bf855efe5664 httppeer: wrap HTTPResponse.read() globally
Gregory Szorc <gregory.szorc@gmail.com>
parents: 30944
diff changeset
96
33842
3c91cc0c5fde httppeer: add support for httppostargs when we're sending a file
Augie Fackler <augie@google.com>
parents: 33827
diff changeset
97 class _multifile(object):
3c91cc0c5fde httppeer: add support for httppostargs when we're sending a file
Augie Fackler <augie@google.com>
parents: 33827
diff changeset
98 def __init__(self, *fileobjs):
3c91cc0c5fde httppeer: add support for httppostargs when we're sending a file
Augie Fackler <augie@google.com>
parents: 33827
diff changeset
99 for f in fileobjs:
3c91cc0c5fde httppeer: add support for httppostargs when we're sending a file
Augie Fackler <augie@google.com>
parents: 33827
diff changeset
100 if not util.safehasattr(f, 'length'):
3c91cc0c5fde httppeer: add support for httppostargs when we're sending a file
Augie Fackler <augie@google.com>
parents: 33827
diff changeset
101 raise ValueError(
3c91cc0c5fde httppeer: add support for httppostargs when we're sending a file
Augie Fackler <augie@google.com>
parents: 33827
diff changeset
102 '_multifile only supports file objects that '
3c91cc0c5fde httppeer: add support for httppostargs when we're sending a file
Augie Fackler <augie@google.com>
parents: 33827
diff changeset
103 'have a length but this one does not:', type(f), f)
3c91cc0c5fde httppeer: add support for httppostargs when we're sending a file
Augie Fackler <augie@google.com>
parents: 33827
diff changeset
104 self._fileobjs = fileobjs
3c91cc0c5fde httppeer: add support for httppostargs when we're sending a file
Augie Fackler <augie@google.com>
parents: 33827
diff changeset
105 self._index = 0
3c91cc0c5fde httppeer: add support for httppostargs when we're sending a file
Augie Fackler <augie@google.com>
parents: 33827
diff changeset
106
3c91cc0c5fde httppeer: add support for httppostargs when we're sending a file
Augie Fackler <augie@google.com>
parents: 33827
diff changeset
107 @property
3c91cc0c5fde httppeer: add support for httppostargs when we're sending a file
Augie Fackler <augie@google.com>
parents: 33827
diff changeset
108 def length(self):
3c91cc0c5fde httppeer: add support for httppostargs when we're sending a file
Augie Fackler <augie@google.com>
parents: 33827
diff changeset
109 return sum(f.length for f in self._fileobjs)
3c91cc0c5fde httppeer: add support for httppostargs when we're sending a file
Augie Fackler <augie@google.com>
parents: 33827
diff changeset
110
3c91cc0c5fde httppeer: add support for httppostargs when we're sending a file
Augie Fackler <augie@google.com>
parents: 33827
diff changeset
111 def read(self, amt=None):
3c91cc0c5fde httppeer: add support for httppostargs when we're sending a file
Augie Fackler <augie@google.com>
parents: 33827
diff changeset
112 if amt <= 0:
3c91cc0c5fde httppeer: add support for httppostargs when we're sending a file
Augie Fackler <augie@google.com>
parents: 33827
diff changeset
113 return ''.join(f.read() for f in self._fileobjs)
3c91cc0c5fde httppeer: add support for httppostargs when we're sending a file
Augie Fackler <augie@google.com>
parents: 33827
diff changeset
114 parts = []
3c91cc0c5fde httppeer: add support for httppostargs when we're sending a file
Augie Fackler <augie@google.com>
parents: 33827
diff changeset
115 while amt and self._index < len(self._fileobjs):
3c91cc0c5fde httppeer: add support for httppostargs when we're sending a file
Augie Fackler <augie@google.com>
parents: 33827
diff changeset
116 parts.append(self._fileobjs[self._index].read(amt))
3c91cc0c5fde httppeer: add support for httppostargs when we're sending a file
Augie Fackler <augie@google.com>
parents: 33827
diff changeset
117 got = len(parts[-1])
3c91cc0c5fde httppeer: add support for httppostargs when we're sending a file
Augie Fackler <augie@google.com>
parents: 33827
diff changeset
118 if got < amt:
3c91cc0c5fde httppeer: add support for httppostargs when we're sending a file
Augie Fackler <augie@google.com>
parents: 33827
diff changeset
119 self._index += 1
3c91cc0c5fde httppeer: add support for httppostargs when we're sending a file
Augie Fackler <augie@google.com>
parents: 33827
diff changeset
120 amt -= got
3c91cc0c5fde httppeer: add support for httppostargs when we're sending a file
Augie Fackler <augie@google.com>
parents: 33827
diff changeset
121 return ''.join(parts)
3c91cc0c5fde httppeer: add support for httppostargs when we're sending a file
Augie Fackler <augie@google.com>
parents: 33827
diff changeset
122
3c91cc0c5fde httppeer: add support for httppostargs when we're sending a file
Augie Fackler <augie@google.com>
parents: 33827
diff changeset
123 def seek(self, offset, whence=os.SEEK_SET):
3c91cc0c5fde httppeer: add support for httppostargs when we're sending a file
Augie Fackler <augie@google.com>
parents: 33827
diff changeset
124 if whence != os.SEEK_SET:
3c91cc0c5fde httppeer: add support for httppostargs when we're sending a file
Augie Fackler <augie@google.com>
parents: 33827
diff changeset
125 raise NotImplementedError(
3c91cc0c5fde httppeer: add support for httppostargs when we're sending a file
Augie Fackler <augie@google.com>
parents: 33827
diff changeset
126 '_multifile does not support anything other'
3c91cc0c5fde httppeer: add support for httppostargs when we're sending a file
Augie Fackler <augie@google.com>
parents: 33827
diff changeset
127 ' than os.SEEK_SET for whence on seek()')
3c91cc0c5fde httppeer: add support for httppostargs when we're sending a file
Augie Fackler <augie@google.com>
parents: 33827
diff changeset
128 if offset != 0:
3c91cc0c5fde httppeer: add support for httppostargs when we're sending a file
Augie Fackler <augie@google.com>
parents: 33827
diff changeset
129 raise NotImplementedError(
3c91cc0c5fde httppeer: add support for httppostargs when we're sending a file
Augie Fackler <augie@google.com>
parents: 33827
diff changeset
130 '_multifile only supports seeking to start, but that '
3c91cc0c5fde httppeer: add support for httppostargs when we're sending a file
Augie Fackler <augie@google.com>
parents: 33827
diff changeset
131 'could be fixed if you need it')
3c91cc0c5fde httppeer: add support for httppostargs when we're sending a file
Augie Fackler <augie@google.com>
parents: 33827
diff changeset
132 for f in self._fileobjs:
3c91cc0c5fde httppeer: add support for httppostargs when we're sending a file
Augie Fackler <augie@google.com>
parents: 33827
diff changeset
133 f.seek(0)
3c91cc0c5fde httppeer: add support for httppostargs when we're sending a file
Augie Fackler <augie@google.com>
parents: 33827
diff changeset
134 self._index = 0
3c91cc0c5fde httppeer: add support for httppostargs when we're sending a file
Augie Fackler <augie@google.com>
parents: 33827
diff changeset
135
33827
dedab036215d wireproto: use new peer interface
Gregory Szorc <gregory.szorc@gmail.com>
parents: 33826
diff changeset
136 class httppeer(wireproto.wirepeer):
60
e32fdbd97839 Add hg:// protocol
mpm@selenic.com
parents: 56
diff changeset
137 def __init__(self, ui, path):
33731
73fd395ee29e httppeer: make several instance attributes internal (API)
Gregory Szorc <gregory.szorc@gmail.com>
parents: 33730
diff changeset
138 self._path = path
73fd395ee29e httppeer: make several instance attributes internal (API)
Gregory Szorc <gregory.szorc@gmail.com>
parents: 33730
diff changeset
139 self._caps = None
73fd395ee29e httppeer: make several instance attributes internal (API)
Gregory Szorc <gregory.szorc@gmail.com>
parents: 33730
diff changeset
140 self._urlopener = None
73fd395ee29e httppeer: make several instance attributes internal (API)
Gregory Szorc <gregory.szorc@gmail.com>
parents: 33730
diff changeset
141 self._requestbuilder = None
14076
924c82157d46 url: move URL parsing functions into util to improve startup time
Brodie Rao <brodie@bitheap.org>
parents: 14060
diff changeset
142 u = util.url(path)
13819
d16894e29f91 httprepo/sshrepo: use url.url
Brodie Rao <brodie@bitheap.org>
parents: 13603
diff changeset
143 if u.query or u.fragment:
26587
56b2bcea2529 error: get Abort from 'error' instead of 'util'
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 25954
diff changeset
144 raise error.Abort(_('unsupported URL component: "%s"') %
13819
d16894e29f91 httprepo/sshrepo: use url.url
Brodie Rao <brodie@bitheap.org>
parents: 13603
diff changeset
145 (u.query or u.fragment))
2337
3f24bc5dee81 http: fix many problems with url parsing and auth. added proxy test.
Vadim Gelfer <vadim.gelfer@gmail.com>
parents: 2336
diff changeset
146
3f24bc5dee81 http: fix many problems with url parsing and auth. added proxy test.
Vadim Gelfer <vadim.gelfer@gmail.com>
parents: 2336
diff changeset
147 # urllib cannot handle URLs with embedded user or passwd
13819
d16894e29f91 httprepo/sshrepo: use url.url
Brodie Rao <brodie@bitheap.org>
parents: 13603
diff changeset
148 self._url, authinfo = u.authinfo()
7270
2db33c1a5654 factor out the url handling from httprepo
Benoit Boissinot <benoit.boissinot@ens-lyon.org>
parents: 7269
diff changeset
149
33826
f913e90f15a0 httppeer: use peer interface
Gregory Szorc <gregory.szorc@gmail.com>
parents: 33731
diff changeset
150 self._ui = ui
f913e90f15a0 httppeer: use peer interface
Gregory Szorc <gregory.szorc@gmail.com>
parents: 33731
diff changeset
151 ui.debug('using %s\n' % self._url)
2337
3f24bc5dee81 http: fix many problems with url parsing and auth. added proxy test.
Vadim Gelfer <vadim.gelfer@gmail.com>
parents: 2336
diff changeset
152
33731
73fd395ee29e httppeer: make several instance attributes internal (API)
Gregory Szorc <gregory.szorc@gmail.com>
parents: 33730
diff changeset
153 self._urlopener = url.opener(ui, authinfo)
73fd395ee29e httppeer: make several instance attributes internal (API)
Gregory Szorc <gregory.szorc@gmail.com>
parents: 33730
diff changeset
154 self._requestbuilder = urlreq.request
4516
96d8a56d4ef9 Removed trailing whitespace and tabs from python files
Thomas Arendsen Hein <thomas@intevation.de>
parents: 4369
diff changeset
155
7752
998fc8f62539 close sockets on httprepository deletion (issue1487)
Steve Borho <steve@borho.org>
parents: 7641
diff changeset
156 def __del__(self):
33731
73fd395ee29e httppeer: make several instance attributes internal (API)
Gregory Szorc <gregory.szorc@gmail.com>
parents: 33730
diff changeset
157 urlopener = getattr(self, '_urlopener', None)
30241
cac4ca036dff httppeer: make __del__ access to self.urlopener more safe
Mads Kiilerich <madski@unity3d.com>
parents: 29455
diff changeset
158 if urlopener:
cac4ca036dff httppeer: make __del__ access to self.urlopener more safe
Mads Kiilerich <madski@unity3d.com>
parents: 29455
diff changeset
159 for h in urlopener.handlers:
15246
7b15dd9125b3 httprepo: make __del__ more stable in error situations
Mads Kiilerich <mads@kiilerich.com>
parents: 15159
diff changeset
160 h.close()
34486
a57c938e7ac8 style: never use a space before a colon or comma
Alex Gaynor <agaynor@mozilla.com>
parents: 33842
diff changeset
161 getattr(h, "close_all", lambda: None)()
7752
998fc8f62539 close sockets on httprepository deletion (issue1487)
Steve Borho <steve@borho.org>
parents: 7641
diff changeset
162
35697
5a7906ed78d4 httppeer: move url opening in its own method
Boris Feld <boris.feld@octobus.net>
parents: 35368
diff changeset
163 def _openurl(self, req):
35698
0c4b23ccf1a5 httppeer: add support for tracing all http request made by the peer
Boris Feld <boris.feld@octobus.net>
parents: 35697
diff changeset
164 if (self._ui.debugflag
0c4b23ccf1a5 httppeer: add support for tracing all http request made by the peer
Boris Feld <boris.feld@octobus.net>
parents: 35697
diff changeset
165 and self._ui.configbool('devel', 'debug.peer-request')):
0c4b23ccf1a5 httppeer: add support for tracing all http request made by the peer
Boris Feld <boris.feld@octobus.net>
parents: 35697
diff changeset
166 dbg = self._ui.debug
0c4b23ccf1a5 httppeer: add support for tracing all http request made by the peer
Boris Feld <boris.feld@octobus.net>
parents: 35697
diff changeset
167 line = 'devel-peer-request: %s\n'
0c4b23ccf1a5 httppeer: add support for tracing all http request made by the peer
Boris Feld <boris.feld@octobus.net>
parents: 35697
diff changeset
168 dbg(line % '%s %s' % (req.get_method(), req.get_full_url()))
0c4b23ccf1a5 httppeer: add support for tracing all http request made by the peer
Boris Feld <boris.feld@octobus.net>
parents: 35697
diff changeset
169 hgargssize = None
0c4b23ccf1a5 httppeer: add support for tracing all http request made by the peer
Boris Feld <boris.feld@octobus.net>
parents: 35697
diff changeset
170
0c4b23ccf1a5 httppeer: add support for tracing all http request made by the peer
Boris Feld <boris.feld@octobus.net>
parents: 35697
diff changeset
171 for header, value in sorted(req.header_items()):
0c4b23ccf1a5 httppeer: add support for tracing all http request made by the peer
Boris Feld <boris.feld@octobus.net>
parents: 35697
diff changeset
172 if header.startswith('X-hgarg-'):
0c4b23ccf1a5 httppeer: add support for tracing all http request made by the peer
Boris Feld <boris.feld@octobus.net>
parents: 35697
diff changeset
173 if hgargssize is None:
0c4b23ccf1a5 httppeer: add support for tracing all http request made by the peer
Boris Feld <boris.feld@octobus.net>
parents: 35697
diff changeset
174 hgargssize = 0
0c4b23ccf1a5 httppeer: add support for tracing all http request made by the peer
Boris Feld <boris.feld@octobus.net>
parents: 35697
diff changeset
175 hgargssize += len(value)
0c4b23ccf1a5 httppeer: add support for tracing all http request made by the peer
Boris Feld <boris.feld@octobus.net>
parents: 35697
diff changeset
176 else:
0c4b23ccf1a5 httppeer: add support for tracing all http request made by the peer
Boris Feld <boris.feld@octobus.net>
parents: 35697
diff changeset
177 dbg(line % ' %s %s' % (header, value))
0c4b23ccf1a5 httppeer: add support for tracing all http request made by the peer
Boris Feld <boris.feld@octobus.net>
parents: 35697
diff changeset
178
0c4b23ccf1a5 httppeer: add support for tracing all http request made by the peer
Boris Feld <boris.feld@octobus.net>
parents: 35697
diff changeset
179 if hgargssize is not None:
0c4b23ccf1a5 httppeer: add support for tracing all http request made by the peer
Boris Feld <boris.feld@octobus.net>
parents: 35697
diff changeset
180 dbg(line % ' %d bytes of commands arguments in headers'
0c4b23ccf1a5 httppeer: add support for tracing all http request made by the peer
Boris Feld <boris.feld@octobus.net>
parents: 35697
diff changeset
181 % hgargssize)
0c4b23ccf1a5 httppeer: add support for tracing all http request made by the peer
Boris Feld <boris.feld@octobus.net>
parents: 35697
diff changeset
182
0c4b23ccf1a5 httppeer: add support for tracing all http request made by the peer
Boris Feld <boris.feld@octobus.net>
parents: 35697
diff changeset
183 if req.has_data():
0c4b23ccf1a5 httppeer: add support for tracing all http request made by the peer
Boris Feld <boris.feld@octobus.net>
parents: 35697
diff changeset
184 data = req.get_data()
0c4b23ccf1a5 httppeer: add support for tracing all http request made by the peer
Boris Feld <boris.feld@octobus.net>
parents: 35697
diff changeset
185 length = getattr(data, 'length', None)
0c4b23ccf1a5 httppeer: add support for tracing all http request made by the peer
Boris Feld <boris.feld@octobus.net>
parents: 35697
diff changeset
186 if length is None:
0c4b23ccf1a5 httppeer: add support for tracing all http request made by the peer
Boris Feld <boris.feld@octobus.net>
parents: 35697
diff changeset
187 length = len(data)
0c4b23ccf1a5 httppeer: add support for tracing all http request made by the peer
Boris Feld <boris.feld@octobus.net>
parents: 35697
diff changeset
188 dbg(line % ' %d bytes of data' % length)
0c4b23ccf1a5 httppeer: add support for tracing all http request made by the peer
Boris Feld <boris.feld@octobus.net>
parents: 35697
diff changeset
189
0c4b23ccf1a5 httppeer: add support for tracing all http request made by the peer
Boris Feld <boris.feld@octobus.net>
parents: 35697
diff changeset
190 start = util.timer()
0c4b23ccf1a5 httppeer: add support for tracing all http request made by the peer
Boris Feld <boris.feld@octobus.net>
parents: 35697
diff changeset
191
0c4b23ccf1a5 httppeer: add support for tracing all http request made by the peer
Boris Feld <boris.feld@octobus.net>
parents: 35697
diff changeset
192 ret = self._urlopener.open(req)
0c4b23ccf1a5 httppeer: add support for tracing all http request made by the peer
Boris Feld <boris.feld@octobus.net>
parents: 35697
diff changeset
193 if self._ui.configbool('devel', 'debug.peer-request'):
0c4b23ccf1a5 httppeer: add support for tracing all http request made by the peer
Boris Feld <boris.feld@octobus.net>
parents: 35697
diff changeset
194 dbg(line % ' finished in %.4f seconds (%s)'
0c4b23ccf1a5 httppeer: add support for tracing all http request made by the peer
Boris Feld <boris.feld@octobus.net>
parents: 35697
diff changeset
195 % (util.timer() - start, ret.code))
0c4b23ccf1a5 httppeer: add support for tracing all http request made by the peer
Boris Feld <boris.feld@octobus.net>
parents: 35697
diff changeset
196 return ret
35697
5a7906ed78d4 httppeer: move url opening in its own method
Boris Feld <boris.feld@octobus.net>
parents: 35368
diff changeset
197
33826
f913e90f15a0 httppeer: use peer interface
Gregory Szorc <gregory.szorc@gmail.com>
parents: 33731
diff changeset
198 # Begin of _basepeer interface.
f913e90f15a0 httppeer: use peer interface
Gregory Szorc <gregory.szorc@gmail.com>
parents: 33731
diff changeset
199
f913e90f15a0 httppeer: use peer interface
Gregory Szorc <gregory.szorc@gmail.com>
parents: 33731
diff changeset
200 @util.propertycache
f913e90f15a0 httppeer: use peer interface
Gregory Szorc <gregory.szorc@gmail.com>
parents: 33731
diff changeset
201 def ui(self):
f913e90f15a0 httppeer: use peer interface
Gregory Szorc <gregory.szorc@gmail.com>
parents: 33731
diff changeset
202 return self._ui
f913e90f15a0 httppeer: use peer interface
Gregory Szorc <gregory.szorc@gmail.com>
parents: 33731
diff changeset
203
2673
109a22f5434a hooks: add url to changegroup, incoming, prechangegroup, pretxnchangegroup hooks
Vadim Gelfer <vadim.gelfer@gmail.com>
parents: 2612
diff changeset
204 def url(self):
33731
73fd395ee29e httppeer: make several instance attributes internal (API)
Gregory Szorc <gregory.szorc@gmail.com>
parents: 33730
diff changeset
205 return self._path
2673
109a22f5434a hooks: add url to changegroup, incoming, prechangegroup, pretxnchangegroup hooks
Vadim Gelfer <vadim.gelfer@gmail.com>
parents: 2612
diff changeset
206
33826
f913e90f15a0 httppeer: use peer interface
Gregory Szorc <gregory.szorc@gmail.com>
parents: 33731
diff changeset
207 def local(self):
f913e90f15a0 httppeer: use peer interface
Gregory Szorc <gregory.szorc@gmail.com>
parents: 33731
diff changeset
208 return None
f913e90f15a0 httppeer: use peer interface
Gregory Szorc <gregory.szorc@gmail.com>
parents: 33731
diff changeset
209
f913e90f15a0 httppeer: use peer interface
Gregory Szorc <gregory.szorc@gmail.com>
parents: 33731
diff changeset
210 def peer(self):
f913e90f15a0 httppeer: use peer interface
Gregory Szorc <gregory.szorc@gmail.com>
parents: 33731
diff changeset
211 return self
f913e90f15a0 httppeer: use peer interface
Gregory Szorc <gregory.szorc@gmail.com>
parents: 33731
diff changeset
212
f913e90f15a0 httppeer: use peer interface
Gregory Szorc <gregory.szorc@gmail.com>
parents: 33731
diff changeset
213 def canpush(self):
f913e90f15a0 httppeer: use peer interface
Gregory Szorc <gregory.szorc@gmail.com>
parents: 33731
diff changeset
214 return True
2442
c660691fb45d http: query server for capabilities
Vadim Gelfer <vadim.gelfer@gmail.com>
parents: 2439
diff changeset
215
33826
f913e90f15a0 httppeer: use peer interface
Gregory Szorc <gregory.szorc@gmail.com>
parents: 33731
diff changeset
216 def close(self):
f913e90f15a0 httppeer: use peer interface
Gregory Szorc <gregory.szorc@gmail.com>
parents: 33731
diff changeset
217 pass
13603
395a84f78736 httprepo: use caps instead of between for compat check
Peter Arrenbrecht <peter.arrenbrecht@gmail.com>
parents: 13555
diff changeset
218
33826
f913e90f15a0 httppeer: use peer interface
Gregory Szorc <gregory.szorc@gmail.com>
parents: 33731
diff changeset
219 # End of _basepeer interface.
f913e90f15a0 httppeer: use peer interface
Gregory Szorc <gregory.szorc@gmail.com>
parents: 33731
diff changeset
220
f913e90f15a0 httppeer: use peer interface
Gregory Szorc <gregory.szorc@gmail.com>
parents: 33731
diff changeset
221 # Begin of _basewirepeer interface.
f913e90f15a0 httppeer: use peer interface
Gregory Szorc <gregory.szorc@gmail.com>
parents: 33731
diff changeset
222
f913e90f15a0 httppeer: use peer interface
Gregory Szorc <gregory.szorc@gmail.com>
parents: 33731
diff changeset
223 def capabilities(self):
33731
73fd395ee29e httppeer: make several instance attributes internal (API)
Gregory Szorc <gregory.szorc@gmail.com>
parents: 33730
diff changeset
224 if self._caps is None:
2442
c660691fb45d http: query server for capabilities
Vadim Gelfer <vadim.gelfer@gmail.com>
parents: 2439
diff changeset
225 try:
13603
395a84f78736 httprepo: use caps instead of between for compat check
Peter Arrenbrecht <peter.arrenbrecht@gmail.com>
parents: 13555
diff changeset
226 self._fetchcaps()
7637
1d54e2f6c0b7 error: move repo errors
Matt Mackall <mpm@selenic.com>
parents: 7342
diff changeset
227 except error.RepoError:
33731
73fd395ee29e httppeer: make several instance attributes internal (API)
Gregory Szorc <gregory.szorc@gmail.com>
parents: 33730
diff changeset
228 self._caps = set()
9467
4c041f1ee1b4 do not attempt to translate ui.debug output
Martin Geisler <mg@lazybytes.net>
parents: 8563
diff changeset
229 self.ui.debug('capabilities: %s\n' %
33731
73fd395ee29e httppeer: make several instance attributes internal (API)
Gregory Szorc <gregory.szorc@gmail.com>
parents: 33730
diff changeset
230 (' '.join(self._caps or ['none'])))
73fd395ee29e httppeer: make several instance attributes internal (API)
Gregory Szorc <gregory.szorc@gmail.com>
parents: 33730
diff changeset
231 return self._caps
2442
c660691fb45d http: query server for capabilities
Vadim Gelfer <vadim.gelfer@gmail.com>
parents: 2439
diff changeset
232
33826
f913e90f15a0 httppeer: use peer interface
Gregory Szorc <gregory.szorc@gmail.com>
parents: 33731
diff changeset
233 # End of _basewirepeer interface.
f913e90f15a0 httppeer: use peer interface
Gregory Szorc <gregory.szorc@gmail.com>
parents: 33731
diff changeset
234
f913e90f15a0 httppeer: use peer interface
Gregory Szorc <gregory.szorc@gmail.com>
parents: 33731
diff changeset
235 # look up capabilities only when needed
f913e90f15a0 httppeer: use peer interface
Gregory Szorc <gregory.szorc@gmail.com>
parents: 33731
diff changeset
236
f913e90f15a0 httppeer: use peer interface
Gregory Szorc <gregory.szorc@gmail.com>
parents: 33731
diff changeset
237 def _fetchcaps(self):
f913e90f15a0 httppeer: use peer interface
Gregory Szorc <gregory.szorc@gmail.com>
parents: 33731
diff changeset
238 self._caps = set(self._call('capabilities').split())
f913e90f15a0 httppeer: use peer interface
Gregory Szorc <gregory.szorc@gmail.com>
parents: 33731
diff changeset
239
30473
e16e234b9ca3 httppeer: do decompression inside _callstream
Gregory Szorc <gregory.szorc@gmail.com>
parents: 30241
diff changeset
240 def _callstream(self, cmd, _compressible=False, **args):
35368
98bc4c43f570 py3: handle keyword arguments correctly in httppeer.py
Pulkit Goyal <7895pulkit@gmail.com>
parents: 35208
diff changeset
241 args = pycompat.byteskwargs(args)
13006
ea68947ad0ce httprepo: remove is-comparison with string literal
Dan Villiom Podlaski Christiansen <danchr@gmail.com>
parents: 12969
diff changeset
242 if cmd == 'pushkey':
12969
6bd9778ae749 pushkey: force HTTP POST on push and add tests (issue2489)
Matt Mackall <mpm@selenic.com>
parents: 12062
diff changeset
243 args['data'] = ''
2465
c91118f425d0 push over http: client support.
Vadim Gelfer <vadim.gelfer@gmail.com>
parents: 2447
diff changeset
244 data = args.pop('data', None)
c91118f425d0 push over http: client support.
Vadim Gelfer <vadim.gelfer@gmail.com>
parents: 2447
diff changeset
245 headers = args.pop('headers', {})
14245
13d44e4235f8 httprepo: send 100-continue on POSTs if using http2
Augie Fackler <durin42@gmail.com>
parents: 14244
diff changeset
246
9467
4c041f1ee1b4 do not attempt to translate ui.debug output
Martin Geisler <mg@lazybytes.net>
parents: 8563
diff changeset
247 self.ui.debug("sending %s command\n" % cmd)
14093
ce99d887585f httprepo: long arguments support (issue2126)
Steven Brown <StevenGBrown@gmail.com>
parents: 14076
diff changeset
248 q = [('cmd', cmd)]
ce99d887585f httprepo: long arguments support (issue2126)
Steven Brown <StevenGBrown@gmail.com>
parents: 14076
diff changeset
249 headersize = 0
30569
07bcd1bf6151 httppeer: assign Vary request header last
Gregory Szorc <gregory.szorc@gmail.com>
parents: 30493
diff changeset
250 varyheaders = []
28530
fd2acc5046f6 http: support sending hgargs via POST body instead of in GET or headers
Augie Fackler <augie@google.com>
parents: 28486
diff changeset
251 # Important: don't use self.capable() here or else you end up
fd2acc5046f6 http: support sending hgargs via POST body instead of in GET or headers
Augie Fackler <augie@google.com>
parents: 28486
diff changeset
252 # with infinite recursion when trying to look up capabilities
fd2acc5046f6 http: support sending hgargs via POST body instead of in GET or headers
Augie Fackler <augie@google.com>
parents: 28486
diff changeset
253 # for the first time.
33731
73fd395ee29e httppeer: make several instance attributes internal (API)
Gregory Szorc <gregory.szorc@gmail.com>
parents: 33730
diff changeset
254 postargsok = self._caps is not None and 'httppostargs' in self._caps
33842
3c91cc0c5fde httppeer: add support for httppostargs when we're sending a file
Augie Fackler <augie@google.com>
parents: 33827
diff changeset
255 if postargsok and args:
28883
032c4c2f802a pycompat: switch to util.urlreq/util.urlerr for py3 compat
timeless <timeless@mozdev.org>
parents: 28666
diff changeset
256 strargs = urlreq.urlencode(sorted(args.items()))
33842
3c91cc0c5fde httppeer: add support for httppostargs when we're sending a file
Augie Fackler <augie@google.com>
parents: 33827
diff changeset
257 if not data:
3c91cc0c5fde httppeer: add support for httppostargs when we're sending a file
Augie Fackler <augie@google.com>
parents: 33827
diff changeset
258 data = strargs
3c91cc0c5fde httppeer: add support for httppostargs when we're sending a file
Augie Fackler <augie@google.com>
parents: 33827
diff changeset
259 else:
35208
bfd072c52e03 py3: use bytes in place of basestring
Pulkit Goyal <7895pulkit@gmail.com>
parents: 34732
diff changeset
260 if isinstance(data, bytes):
33842
3c91cc0c5fde httppeer: add support for httppostargs when we're sending a file
Augie Fackler <augie@google.com>
parents: 33827
diff changeset
261 i = io.BytesIO(data)
3c91cc0c5fde httppeer: add support for httppostargs when we're sending a file
Augie Fackler <augie@google.com>
parents: 33827
diff changeset
262 i.length = len(data)
3c91cc0c5fde httppeer: add support for httppostargs when we're sending a file
Augie Fackler <augie@google.com>
parents: 33827
diff changeset
263 data = i
3c91cc0c5fde httppeer: add support for httppostargs when we're sending a file
Augie Fackler <augie@google.com>
parents: 33827
diff changeset
264 argsio = io.BytesIO(strargs)
3c91cc0c5fde httppeer: add support for httppostargs when we're sending a file
Augie Fackler <augie@google.com>
parents: 33827
diff changeset
265 argsio.length = len(strargs)
3c91cc0c5fde httppeer: add support for httppostargs when we're sending a file
Augie Fackler <augie@google.com>
parents: 33827
diff changeset
266 data = _multifile(argsio, data)
34701
6db536bed7ec httppeer: use native strings for headers
Augie Fackler <augie@google.com>
parents: 34699
diff changeset
267 headers[r'X-HgArgs-Post'] = len(strargs)
28530
fd2acc5046f6 http: support sending hgargs via POST body instead of in GET or headers
Augie Fackler <augie@google.com>
parents: 28486
diff changeset
268 else:
28485
d3893900f6c8 httppeer: indent existing argument handling with if True
Augie Fackler <augie@google.com>
parents: 28484
diff changeset
269 if len(args) > 0:
d3893900f6c8 httppeer: indent existing argument handling with if True
Augie Fackler <augie@google.com>
parents: 28484
diff changeset
270 httpheader = self.capable('httpheader')
d3893900f6c8 httppeer: indent existing argument handling with if True
Augie Fackler <augie@google.com>
parents: 28484
diff changeset
271 if httpheader:
d3893900f6c8 httppeer: indent existing argument handling with if True
Augie Fackler <augie@google.com>
parents: 28484
diff changeset
272 headersize = int(httpheader.split(',', 1)[0])
d3893900f6c8 httppeer: indent existing argument handling with if True
Augie Fackler <augie@google.com>
parents: 28484
diff changeset
273 if headersize > 0:
d3893900f6c8 httppeer: indent existing argument handling with if True
Augie Fackler <augie@google.com>
parents: 28484
diff changeset
274 # The headers can typically carry more data than the URL.
28883
032c4c2f802a pycompat: switch to util.urlreq/util.urlerr for py3 compat
timeless <timeless@mozdev.org>
parents: 28666
diff changeset
275 encargs = urlreq.urlencode(sorted(args.items()))
30759
3f5f0c98cd18 httppeer: extract code for HTTP header spanning
Gregory Szorc <gregory.szorc@gmail.com>
parents: 30569
diff changeset
276 for header, value in encodevalueinheaders(encargs, 'X-HgArg',
3f5f0c98cd18 httppeer: extract code for HTTP header spanning
Gregory Szorc <gregory.szorc@gmail.com>
parents: 30569
diff changeset
277 headersize):
3f5f0c98cd18 httppeer: extract code for HTTP header spanning
Gregory Szorc <gregory.szorc@gmail.com>
parents: 30569
diff changeset
278 headers[header] = value
28486
50314dc3ae4e httppeer: compute header names only once
Augie Fackler <augie@google.com>
parents: 28485
diff changeset
279 varyheaders.append(header)
28485
d3893900f6c8 httppeer: indent existing argument handling with if True
Augie Fackler <augie@google.com>
parents: 28484
diff changeset
280 else:
d3893900f6c8 httppeer: indent existing argument handling with if True
Augie Fackler <augie@google.com>
parents: 28484
diff changeset
281 q += sorted(args.items())
28883
032c4c2f802a pycompat: switch to util.urlreq/util.urlerr for py3 compat
timeless <timeless@mozdev.org>
parents: 28666
diff changeset
282 qs = '?%s' % urlreq.urlencode(q)
3562
88b4755fa48f httprepo: record the url after a request, makes pull + redirect works
Benoit Boissinot <benoit.boissinot@ens-lyon.org>
parents: 3445
diff changeset
283 cu = "%s%s" % (self._url, qs)
28484
da6f713ab480 httppeer: move size computation later in _callstream
Augie Fackler <augie@google.com>
parents: 28483
diff changeset
284 size = 0
da6f713ab480 httppeer: move size computation later in _callstream
Augie Fackler <augie@google.com>
parents: 28483
diff changeset
285 if util.safehasattr(data, 'length'):
da6f713ab480 httppeer: move size computation later in _callstream
Augie Fackler <augie@google.com>
parents: 28483
diff changeset
286 size = data.length
da6f713ab480 httppeer: move size computation later in _callstream
Augie Fackler <augie@google.com>
parents: 28483
diff changeset
287 elif data is not None:
da6f713ab480 httppeer: move size computation later in _callstream
Augie Fackler <augie@google.com>
parents: 28483
diff changeset
288 size = len(data)
33499
0407a51b9d8c codemod: register core configitems using a script
Jun Wu <quark@fb.com>
parents: 32086
diff changeset
289 if size and self.ui.configbool('ui', 'usehttp2'):
34701
6db536bed7ec httppeer: use native strings for headers
Augie Fackler <augie@google.com>
parents: 34699
diff changeset
290 headers[r'Expect'] = r'100-Continue'
6db536bed7ec httppeer: use native strings for headers
Augie Fackler <augie@google.com>
parents: 34699
diff changeset
291 headers[r'X-HgHttp2'] = r'1'
6db536bed7ec httppeer: use native strings for headers
Augie Fackler <augie@google.com>
parents: 34699
diff changeset
292 if data is not None and r'Content-Type' not in headers:
6db536bed7ec httppeer: use native strings for headers
Augie Fackler <augie@google.com>
parents: 34699
diff changeset
293 headers[r'Content-Type'] = r'application/mercurial-0.1'
30569
07bcd1bf6151 httppeer: assign Vary request header last
Gregory Szorc <gregory.szorc@gmail.com>
parents: 30493
diff changeset
294
30763
a520aefb96f1 httppeer: advertise and support application/mercurial-0.2
Gregory Szorc <gregory.szorc@gmail.com>
parents: 30759
diff changeset
295 # Tell the server we accept application/mercurial-0.2 and multiple
a520aefb96f1 httppeer: advertise and support application/mercurial-0.2
Gregory Szorc <gregory.szorc@gmail.com>
parents: 30759
diff changeset
296 # compression formats if the server is capable of emitting those
a520aefb96f1 httppeer: advertise and support application/mercurial-0.2
Gregory Szorc <gregory.szorc@gmail.com>
parents: 30759
diff changeset
297 # payloads.
a520aefb96f1 httppeer: advertise and support application/mercurial-0.2
Gregory Szorc <gregory.szorc@gmail.com>
parents: 30759
diff changeset
298 protoparams = []
a520aefb96f1 httppeer: advertise and support application/mercurial-0.2
Gregory Szorc <gregory.szorc@gmail.com>
parents: 30759
diff changeset
299
a520aefb96f1 httppeer: advertise and support application/mercurial-0.2
Gregory Szorc <gregory.szorc@gmail.com>
parents: 30759
diff changeset
300 mediatypes = set()
33731
73fd395ee29e httppeer: make several instance attributes internal (API)
Gregory Szorc <gregory.szorc@gmail.com>
parents: 33730
diff changeset
301 if self._caps is not None:
30763
a520aefb96f1 httppeer: advertise and support application/mercurial-0.2
Gregory Szorc <gregory.szorc@gmail.com>
parents: 30759
diff changeset
302 mt = self.capable('httpmediatype')
a520aefb96f1 httppeer: advertise and support application/mercurial-0.2
Gregory Szorc <gregory.szorc@gmail.com>
parents: 30759
diff changeset
303 if mt:
a520aefb96f1 httppeer: advertise and support application/mercurial-0.2
Gregory Szorc <gregory.szorc@gmail.com>
parents: 30759
diff changeset
304 protoparams.append('0.1')
a520aefb96f1 httppeer: advertise and support application/mercurial-0.2
Gregory Szorc <gregory.szorc@gmail.com>
parents: 30759
diff changeset
305 mediatypes = set(mt.split(','))
a520aefb96f1 httppeer: advertise and support application/mercurial-0.2
Gregory Szorc <gregory.szorc@gmail.com>
parents: 30759
diff changeset
306
a520aefb96f1 httppeer: advertise and support application/mercurial-0.2
Gregory Szorc <gregory.szorc@gmail.com>
parents: 30759
diff changeset
307 if '0.2tx' in mediatypes:
a520aefb96f1 httppeer: advertise and support application/mercurial-0.2
Gregory Szorc <gregory.szorc@gmail.com>
parents: 30759
diff changeset
308 protoparams.append('0.2')
a520aefb96f1 httppeer: advertise and support application/mercurial-0.2
Gregory Szorc <gregory.szorc@gmail.com>
parents: 30759
diff changeset
309
a520aefb96f1 httppeer: advertise and support application/mercurial-0.2
Gregory Szorc <gregory.szorc@gmail.com>
parents: 30759
diff changeset
310 if '0.2tx' in mediatypes and self.capable('compression'):
a520aefb96f1 httppeer: advertise and support application/mercurial-0.2
Gregory Szorc <gregory.szorc@gmail.com>
parents: 30759
diff changeset
311 # We /could/ compare supported compression formats and prune
a520aefb96f1 httppeer: advertise and support application/mercurial-0.2
Gregory Szorc <gregory.szorc@gmail.com>
parents: 30759
diff changeset
312 # non-mutually supported or error if nothing is mutually supported.
a520aefb96f1 httppeer: advertise and support application/mercurial-0.2
Gregory Szorc <gregory.szorc@gmail.com>
parents: 30759
diff changeset
313 # For now, send the full list to the server and have it error.
a520aefb96f1 httppeer: advertise and support application/mercurial-0.2
Gregory Szorc <gregory.szorc@gmail.com>
parents: 30759
diff changeset
314 comps = [e.wireprotosupport().name for e in
a520aefb96f1 httppeer: advertise and support application/mercurial-0.2
Gregory Szorc <gregory.szorc@gmail.com>
parents: 30759
diff changeset
315 util.compengines.supportedwireengines(util.CLIENTROLE)]
a520aefb96f1 httppeer: advertise and support application/mercurial-0.2
Gregory Szorc <gregory.szorc@gmail.com>
parents: 30759
diff changeset
316 protoparams.append('comp=%s' % ','.join(comps))
a520aefb96f1 httppeer: advertise and support application/mercurial-0.2
Gregory Szorc <gregory.szorc@gmail.com>
parents: 30759
diff changeset
317
a520aefb96f1 httppeer: advertise and support application/mercurial-0.2
Gregory Szorc <gregory.szorc@gmail.com>
parents: 30759
diff changeset
318 if protoparams:
a520aefb96f1 httppeer: advertise and support application/mercurial-0.2
Gregory Szorc <gregory.szorc@gmail.com>
parents: 30759
diff changeset
319 protoheaders = encodevalueinheaders(' '.join(protoparams),
a520aefb96f1 httppeer: advertise and support application/mercurial-0.2
Gregory Szorc <gregory.szorc@gmail.com>
parents: 30759
diff changeset
320 'X-HgProto',
a520aefb96f1 httppeer: advertise and support application/mercurial-0.2
Gregory Szorc <gregory.szorc@gmail.com>
parents: 30759
diff changeset
321 headersize or 1024)
a520aefb96f1 httppeer: advertise and support application/mercurial-0.2
Gregory Szorc <gregory.szorc@gmail.com>
parents: 30759
diff changeset
322 for header, value in protoheaders:
a520aefb96f1 httppeer: advertise and support application/mercurial-0.2
Gregory Szorc <gregory.szorc@gmail.com>
parents: 30759
diff changeset
323 headers[header] = value
a520aefb96f1 httppeer: advertise and support application/mercurial-0.2
Gregory Szorc <gregory.szorc@gmail.com>
parents: 30759
diff changeset
324 varyheaders.append(header)
a520aefb96f1 httppeer: advertise and support application/mercurial-0.2
Gregory Szorc <gregory.szorc@gmail.com>
parents: 30759
diff changeset
325
32022
e5d7f99a3063 httppeer: don't send empty Vary request header
Gregory Szorc <gregory.szorc@gmail.com>
parents: 32003
diff changeset
326 if varyheaders:
34701
6db536bed7ec httppeer: use native strings for headers
Augie Fackler <augie@google.com>
parents: 34699
diff changeset
327 headers[r'Vary'] = r','.join(varyheaders)
32022
e5d7f99a3063 httppeer: don't send empty Vary request header
Gregory Szorc <gregory.szorc@gmail.com>
parents: 32003
diff changeset
328
34699
375c8debe336 httppeer: pass url to urllib as native str, not bytes
Augie Fackler <augie@google.com>
parents: 34486
diff changeset
329 req = self._requestbuilder(pycompat.strurl(cu), data, headers)
30569
07bcd1bf6151 httppeer: assign Vary request header last
Gregory Szorc <gregory.szorc@gmail.com>
parents: 30493
diff changeset
330
10491
d7e582cab6b6 http: len(x) fails if it doesn't fit into an int, use __len__() instead
Benoit Boissinot <benoit.boissinot@ens-lyon.org>
parents: 10282
diff changeset
331 if data is not None:
d7e582cab6b6 http: len(x) fails if it doesn't fit into an int, use __len__() instead
Benoit Boissinot <benoit.boissinot@ens-lyon.org>
parents: 10282
diff changeset
332 self.ui.debug("sending %s bytes\n" % size)
d7e582cab6b6 http: len(x) fails if it doesn't fit into an int, use __len__() instead
Benoit Boissinot <benoit.boissinot@ens-lyon.org>
parents: 10282
diff changeset
333 req.add_unredirected_header('Content-Length', '%d' % size)
2294
ce67fa312f61 Catch urllib's HTTPException and give a meaningful error message to the user.
Thomas Arendsen Hein <thomas@intevation.de>
parents: 2281
diff changeset
334 try:
35697
5a7906ed78d4 httppeer: move url opening in its own method
Boris Feld <boris.feld@octobus.net>
parents: 35368
diff changeset
335 resp = self._openurl(req)
28883
032c4c2f802a pycompat: switch to util.urlreq/util.urlerr for py3 compat
timeless <timeless@mozdev.org>
parents: 28666
diff changeset
336 except urlerr.httperror as inst:
2467
4e78dc71d946 http client: better work with authorization errors, broken sockets.
Vadim Gelfer <vadim.gelfer@gmail.com>
parents: 2465
diff changeset
337 if inst.code == 401:
26587
56b2bcea2529 error: get Abort from 'error' instead of 'util'
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 25954
diff changeset
338 raise error.Abort(_('authorization failed'))
2467
4e78dc71d946 http client: better work with authorization errors, broken sockets.
Vadim Gelfer <vadim.gelfer@gmail.com>
parents: 2465
diff changeset
339 raise
25660
328739ea70c3 global: mass rewrite to use modern exception syntax
Gregory Szorc <gregory.szorc@gmail.com>
parents: 25500
diff changeset
340 except httplib.HTTPException as inst:
9467
4c041f1ee1b4 do not attempt to translate ui.debug output
Martin Geisler <mg@lazybytes.net>
parents: 8563
diff changeset
341 self.ui.debug('http error while sending %s command\n' % cmd)
8206
cce63ef1045b ui: print_exc() -> traceback()
Matt Mackall <mpm@selenic.com>
parents: 8150
diff changeset
342 self.ui.traceback()
2336
f77edcffb837 http: print better error if exception happens.
Vadim Gelfer <vadim.gelfer@gmail.com>
parents: 2294
diff changeset
343 raise IOError(None, inst)
32002
bf855efe5664 httppeer: wrap HTTPResponse.read() globally
Gregory Szorc <gregory.szorc@gmail.com>
parents: 30944
diff changeset
344
bf855efe5664 httppeer: wrap HTTPResponse.read() globally
Gregory Szorc <gregory.szorc@gmail.com>
parents: 30944
diff changeset
345 # Insert error handlers for common I/O failures.
bf855efe5664 httppeer: wrap HTTPResponse.read() globally
Gregory Szorc <gregory.szorc@gmail.com>
parents: 30944
diff changeset
346 _wraphttpresponse(resp)
bf855efe5664 httppeer: wrap HTTPResponse.read() globally
Gregory Szorc <gregory.szorc@gmail.com>
parents: 30944
diff changeset
347
3562
88b4755fa48f httprepo: record the url after a request, makes pull + redirect works
Benoit Boissinot <benoit.boissinot@ens-lyon.org>
parents: 3445
diff changeset
348 # record the url we got redirected to
34724
9c3dcaf648ef httppeer: convert request url back to bytes before inspecting it
Augie Fackler <augie@google.com>
parents: 34701
diff changeset
349 resp_url = pycompat.bytesurl(resp.geturl())
3570
c141d07198b9 Inform the user about the new URL when being redirected via http.
Thomas Arendsen Hein <thomas@intevation.de>
parents: 3569
diff changeset
350 if resp_url.endswith(qs):
c141d07198b9 Inform the user about the new URL when being redirected via http.
Thomas Arendsen Hein <thomas@intevation.de>
parents: 3569
diff changeset
351 resp_url = resp_url[:-len(qs)]
9881
54b518fc6671 httprepo: suppress the `real URL is...' message in safe, common cases.
Dan Villiom Podlaski Christiansen <danchr@gmail.com>
parents: 9878
diff changeset
352 if self._url.rstrip('/') != resp_url.rstrip('/'):
14504
c59968e8b579 httprepo: send URL redirection notices to stderr (issue2828)
Matt Mackall <mpm@selenic.com>
parents: 14503
diff changeset
353 if not self.ui.quiet:
c59968e8b579 httprepo: send URL redirection notices to stderr (issue2828)
Matt Mackall <mpm@selenic.com>
parents: 14503
diff changeset
354 self.ui.warn(_('real URL is %s\n') % resp_url)
10208
37c4ce51a12d httprepo: always store the response url (issue1968)
Steve Borho <steve@borho.org>
parents: 9881
diff changeset
355 self._url = resp_url
2435
ff2bac730b99 http client: support persistent connections.
Vadim Gelfer <vadim.gelfer@gmail.com>
parents: 2337
diff changeset
356 try:
34725
a288712d86d5 httppeer: extract content-type from headers using native str
Augie Fackler <augie@google.com>
parents: 34724
diff changeset
357 proto = pycompat.bytesurl(resp.getheader(r'content-type', r''))
2435
ff2bac730b99 http client: support persistent connections.
Vadim Gelfer <vadim.gelfer@gmail.com>
parents: 2337
diff changeset
358 except AttributeError:
34725
a288712d86d5 httppeer: extract content-type from headers using native str
Augie Fackler <augie@google.com>
parents: 34724
diff changeset
359 proto = pycompat.bytesurl(resp.headers.get(r'content-type', r''))
752
c693eafd5967 Simplify content type checking
mpm@selenic.com
parents: 751
diff changeset
360
14076
924c82157d46 url: move URL parsing functions into util to improve startup time
Brodie Rao <brodie@bitheap.org>
parents: 14060
diff changeset
361 safeurl = util.hidepassword(self._url)
15017
f4522df38c65 wireproto: add out-of-band error class to allow remote repo to report errors
Andrew Pritchard <andrewp@fogcreek.com>
parents: 14999
diff changeset
362 if proto.startswith('application/hg-error'):
f4522df38c65 wireproto: add out-of-band error class to allow remote repo to report errors
Andrew Pritchard <andrewp@fogcreek.com>
parents: 14999
diff changeset
363 raise error.OutOfBandError(resp.read())
753
8760d0c83b9b Check protocol versions
mpm@selenic.com
parents: 752
diff changeset
364 # accept old "text/plain" and "application/hg-changegroup" for now
4633
ff7253a0d1da Cleanup of whitespace, indentation and line continuation.
Thomas Arendsen Hein <thomas@intevation.de>
parents: 4516
diff changeset
365 if not (proto.startswith('application/mercurial-') or
18737
56f8522c3591 httppeer: improve protocol check
Matt Mackall <mpm@selenic.com>
parents: 17221
diff changeset
366 (proto.startswith('text/plain')
56f8522c3591 httppeer: improve protocol check
Matt Mackall <mpm@selenic.com>
parents: 17221
diff changeset
367 and not resp.headers.get('content-length')) or
4633
ff7253a0d1da Cleanup of whitespace, indentation and line continuation.
Thomas Arendsen Hein <thomas@intevation.de>
parents: 4516
diff changeset
368 proto.startswith('application/hg-changegroup')):
14076
924c82157d46 url: move URL parsing functions into util to improve startup time
Brodie Rao <brodie@bitheap.org>
parents: 14060
diff changeset
369 self.ui.debug("requested URL: '%s'\n" % util.hidepassword(cu))
10282
08a0f04b56bd many, many trivial check-code fixups
Matt Mackall <mpm@selenic.com>
parents: 10264
diff changeset
370 raise error.RepoError(
08a0f04b56bd many, many trivial check-code fixups
Matt Mackall <mpm@selenic.com>
parents: 10264
diff changeset
371 _("'%s' does not appear to be an hg repository:\n"
08a0f04b56bd many, many trivial check-code fixups
Matt Mackall <mpm@selenic.com>
parents: 10264
diff changeset
372 "---%%<--- (%s)\n%s\n---%%<---\n")
18738
b376e8f91c16 httppeer: avoid large dumps when we don't see an hgweb repo
Matt Mackall <mpm@selenic.com>
parents: 18737
diff changeset
373 % (safeurl, proto or 'no content-type', resp.read(1024)))
752
c693eafd5967 Simplify content type checking
mpm@selenic.com
parents: 751
diff changeset
374
4012
d1e31d7f7d44 fix handling of multiple Content-type headers
Benoit Boissinot <benoit.boissinot@ens-lyon.org>
parents: 3703
diff changeset
375 if proto.startswith('application/mercurial-'):
d1e31d7f7d44 fix handling of multiple Content-type headers
Benoit Boissinot <benoit.boissinot@ens-lyon.org>
parents: 3703
diff changeset
376 try:
4356
aed9e6dceb85 Avoid float rounding errors when checking http protocol version.
Thomas Arendsen Hein <thomas@intevation.de>
parents: 4226
diff changeset
377 version = proto.split('-', 1)[1]
aed9e6dceb85 Avoid float rounding errors when checking http protocol version.
Thomas Arendsen Hein <thomas@intevation.de>
parents: 4226
diff changeset
378 version_info = tuple([int(n) for n in version.split('.')])
4012
d1e31d7f7d44 fix handling of multiple Content-type headers
Benoit Boissinot <benoit.boissinot@ens-lyon.org>
parents: 3703
diff changeset
379 except ValueError:
7637
1d54e2f6c0b7 error: move repo errors
Matt Mackall <mpm@selenic.com>
parents: 7342
diff changeset
380 raise error.RepoError(_("'%s' sent a broken Content-Type "
8053
976170068286 hide passwords in httprepo error messages
Steve Borho <steve@borho.org>
parents: 7752
diff changeset
381 "header (%s)") % (safeurl, proto))
30763
a520aefb96f1 httppeer: advertise and support application/mercurial-0.2
Gregory Szorc <gregory.szorc@gmail.com>
parents: 30759
diff changeset
382
32003
84569d2b3fb7 httppeer: eliminate decompressresponse() proxy
Gregory Szorc <gregory.szorc@gmail.com>
parents: 32002
diff changeset
383 # TODO consider switching to a decompression reader that uses
84569d2b3fb7 httppeer: eliminate decompressresponse() proxy
Gregory Szorc <gregory.szorc@gmail.com>
parents: 32002
diff changeset
384 # generators.
30763
a520aefb96f1 httppeer: advertise and support application/mercurial-0.2
Gregory Szorc <gregory.szorc@gmail.com>
parents: 30759
diff changeset
385 if version_info == (0, 1):
a520aefb96f1 httppeer: advertise and support application/mercurial-0.2
Gregory Szorc <gregory.szorc@gmail.com>
parents: 30759
diff changeset
386 if _compressible:
32003
84569d2b3fb7 httppeer: eliminate decompressresponse() proxy
Gregory Szorc <gregory.szorc@gmail.com>
parents: 32002
diff changeset
387 return util.compengines['zlib'].decompressorreader(resp)
30763
a520aefb96f1 httppeer: advertise and support application/mercurial-0.2
Gregory Szorc <gregory.szorc@gmail.com>
parents: 30759
diff changeset
388 return resp
a520aefb96f1 httppeer: advertise and support application/mercurial-0.2
Gregory Szorc <gregory.szorc@gmail.com>
parents: 30759
diff changeset
389 elif version_info == (0, 2):
a520aefb96f1 httppeer: advertise and support application/mercurial-0.2
Gregory Szorc <gregory.szorc@gmail.com>
parents: 30759
diff changeset
390 # application/mercurial-0.2 always identifies the compression
a520aefb96f1 httppeer: advertise and support application/mercurial-0.2
Gregory Szorc <gregory.szorc@gmail.com>
parents: 30759
diff changeset
391 # engine in the payload header.
a520aefb96f1 httppeer: advertise and support application/mercurial-0.2
Gregory Szorc <gregory.szorc@gmail.com>
parents: 30759
diff changeset
392 elen = struct.unpack('B', resp.read(1))[0]
a520aefb96f1 httppeer: advertise and support application/mercurial-0.2
Gregory Szorc <gregory.szorc@gmail.com>
parents: 30759
diff changeset
393 ename = resp.read(elen)
a520aefb96f1 httppeer: advertise and support application/mercurial-0.2
Gregory Szorc <gregory.szorc@gmail.com>
parents: 30759
diff changeset
394 engine = util.compengines.forwiretype(ename)
32003
84569d2b3fb7 httppeer: eliminate decompressresponse() proxy
Gregory Szorc <gregory.szorc@gmail.com>
parents: 32002
diff changeset
395 return engine.decompressorreader(resp)
30763
a520aefb96f1 httppeer: advertise and support application/mercurial-0.2
Gregory Szorc <gregory.szorc@gmail.com>
parents: 30759
diff changeset
396 else:
7637
1d54e2f6c0b7 error: move repo errors
Matt Mackall <mpm@selenic.com>
parents: 7342
diff changeset
397 raise error.RepoError(_("'%s' uses newer protocol %s") %
8053
976170068286 hide passwords in httprepo error messages
Steve Borho <steve@borho.org>
parents: 7752
diff changeset
398 (safeurl, version))
753
8760d0c83b9b Check protocol versions
mpm@selenic.com
parents: 752
diff changeset
399
30473
e16e234b9ca3 httppeer: do decompression inside _callstream
Gregory Szorc <gregory.szorc@gmail.com>
parents: 30241
diff changeset
400 if _compressible:
32003
84569d2b3fb7 httppeer: eliminate decompressresponse() proxy
Gregory Szorc <gregory.szorc@gmail.com>
parents: 32002
diff changeset
401 return util.compengines['zlib'].decompressorreader(resp)
30473
e16e234b9ca3 httppeer: do decompression inside _callstream
Gregory Szorc <gregory.szorc@gmail.com>
parents: 30241
diff changeset
402
752
c693eafd5967 Simplify content type checking
mpm@selenic.com
parents: 751
diff changeset
403 return resp
60
e32fdbd97839 Add hg:// protocol
mpm@selenic.com
parents: 56
diff changeset
404
11589
e8d22fe2ddab protocol: clean up call-like functions in http and ssh clients
Matt Mackall <mpm@selenic.com>
parents: 11588
diff changeset
405 def _call(self, cmd, **args):
e8d22fe2ddab protocol: clean up call-like functions in http and ssh clients
Matt Mackall <mpm@selenic.com>
parents: 11588
diff changeset
406 fp = self._callstream(cmd, **args)
2435
ff2bac730b99 http client: support persistent connections.
Vadim Gelfer <vadim.gelfer@gmail.com>
parents: 2337
diff changeset
407 try:
ff2bac730b99 http client: support persistent connections.
Vadim Gelfer <vadim.gelfer@gmail.com>
parents: 2337
diff changeset
408 return fp.read()
ff2bac730b99 http client: support persistent connections.
Vadim Gelfer <vadim.gelfer@gmail.com>
parents: 2337
diff changeset
409 finally:
ff2bac730b99 http client: support persistent connections.
Vadim Gelfer <vadim.gelfer@gmail.com>
parents: 2337
diff changeset
410 # if using keepalive, allow connection to be reused
ff2bac730b99 http client: support persistent connections.
Vadim Gelfer <vadim.gelfer@gmail.com>
parents: 2337
diff changeset
411 fp.close()
ff2bac730b99 http client: support persistent connections.
Vadim Gelfer <vadim.gelfer@gmail.com>
parents: 2337
diff changeset
412
11592
26e0782b8380 protocol: unify client unbundle support
Matt Mackall <mpm@selenic.com>
parents: 11591
diff changeset
413 def _callpush(self, cmd, cg, **args):
2465
c91118f425d0 push over http: client support.
Vadim Gelfer <vadim.gelfer@gmail.com>
parents: 2447
diff changeset
414 # have to stream bundle to a temp file because we do not have
c91118f425d0 push over http: client support.
Vadim Gelfer <vadim.gelfer@gmail.com>
parents: 2447
diff changeset
415 # http 1.1 chunked transfer.
c91118f425d0 push over http: client support.
Vadim Gelfer <vadim.gelfer@gmail.com>
parents: 2447
diff changeset
416
3662
f4dc02d7fb71 unduplicate bundle writing code from httprepo
Matt Mackall <mpm@selenic.com>
parents: 3661
diff changeset
417 types = self.capable('unbundle')
3703
e674cae8efee fix push over HTTP to older servers
Alexis S. L. Carvalho <alexis@cecm.usp.br>
parents: 3662
diff changeset
418 try:
e674cae8efee fix push over HTTP to older servers
Alexis S. L. Carvalho <alexis@cecm.usp.br>
parents: 3662
diff changeset
419 types = types.split(',')
e674cae8efee fix push over HTTP to older servers
Alexis S. L. Carvalho <alexis@cecm.usp.br>
parents: 3662
diff changeset
420 except AttributeError:
14060
aaa9a5989405 bundle: more comments about the different header types, remove useless if
Benoit Boissinot <benoit.boissinot@ens-lyon.org>
parents: 13819
diff changeset
421 # servers older than d1b16a746db6 will send 'unbundle' as a
aaa9a5989405 bundle: more comments about the different header types, remove useless if
Benoit Boissinot <benoit.boissinot@ens-lyon.org>
parents: 13819
diff changeset
422 # boolean capability. They only support headerless/uncompressed
aaa9a5989405 bundle: more comments about the different header types, remove useless if
Benoit Boissinot <benoit.boissinot@ens-lyon.org>
parents: 13819
diff changeset
423 # bundles.
3703
e674cae8efee fix push over HTTP to older servers
Alexis S. L. Carvalho <alexis@cecm.usp.br>
parents: 3662
diff changeset
424 types = [""]
14060
aaa9a5989405 bundle: more comments about the different header types, remove useless if
Benoit Boissinot <benoit.boissinot@ens-lyon.org>
parents: 13819
diff changeset
425 for x in types:
28666
ae53ecc47414 bundle: move writebundle() from changegroup.py to bundle2.py (API)
Martin von Zweigbergk <martinvonz@google.com>
parents: 28530
diff changeset
426 if x in bundle2.bundletypes:
14060
aaa9a5989405 bundle: more comments about the different header types, remove useless if
Benoit Boissinot <benoit.boissinot@ens-lyon.org>
parents: 13819
diff changeset
427 type = x
aaa9a5989405 bundle: more comments about the different header types, remove useless if
Benoit Boissinot <benoit.boissinot@ens-lyon.org>
parents: 13819
diff changeset
428 break
3613
cbf352b9a3cd Client support for hgweb unbundle with versions.
Thomas Arendsen Hein <thomas@intevation.de>
parents: 3609
diff changeset
429
28666
ae53ecc47414 bundle: move writebundle() from changegroup.py to bundle2.py (API)
Martin von Zweigbergk <martinvonz@google.com>
parents: 28530
diff changeset
430 tempname = bundle2.writebundle(self.ui, cg, None, type)
14244
e7525a555a64 url: use new http support if requested by the user
Augie Fackler <durin42@gmail.com>
parents: 14149
diff changeset
431 fp = httpconnection.httpsendfile(self.ui, tempname, "rb")
11592
26e0782b8380 protocol: unify client unbundle support
Matt Mackall <mpm@selenic.com>
parents: 11591
diff changeset
432 headers = {'Content-Type': 'application/mercurial-0.1'}
26e0782b8380 protocol: unify client unbundle support
Matt Mackall <mpm@selenic.com>
parents: 11591
diff changeset
433
2465
c91118f425d0 push over http: client support.
Vadim Gelfer <vadim.gelfer@gmail.com>
parents: 2447
diff changeset
434 try:
25085
e05734cd7902 httppeer: use try/except/finally
Matt Mackall <mpm@selenic.com>
parents: 23895
diff changeset
435 r = self._call(cmd, data=fp, headers=headers, **args)
e05734cd7902 httppeer: use try/except/finally
Matt Mackall <mpm@selenic.com>
parents: 23895
diff changeset
436 vals = r.split('\n', 1)
e05734cd7902 httppeer: use try/except/finally
Matt Mackall <mpm@selenic.com>
parents: 23895
diff changeset
437 if len(vals) < 2:
e05734cd7902 httppeer: use try/except/finally
Matt Mackall <mpm@selenic.com>
parents: 23895
diff changeset
438 raise error.ResponseError(_("unexpected response:"), r)
e05734cd7902 httppeer: use try/except/finally
Matt Mackall <mpm@selenic.com>
parents: 23895
diff changeset
439 return vals
25660
328739ea70c3 global: mass rewrite to use modern exception syntax
Gregory Szorc <gregory.szorc@gmail.com>
parents: 25500
diff changeset
440 except socket.error as err:
25085
e05734cd7902 httppeer: use try/except/finally
Matt Mackall <mpm@selenic.com>
parents: 23895
diff changeset
441 if err.args[0] in (errno.ECONNRESET, errno.EPIPE):
26587
56b2bcea2529 error: get Abort from 'error' instead of 'util'
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 25954
diff changeset
442 raise error.Abort(_('push failed: %s') % err.args[1])
56b2bcea2529 error: get Abort from 'error' instead of 'util'
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 25954
diff changeset
443 raise error.Abort(err.args[1])
2465
c91118f425d0 push over http: client support.
Vadim Gelfer <vadim.gelfer@gmail.com>
parents: 2447
diff changeset
444 finally:
c91118f425d0 push over http: client support.
Vadim Gelfer <vadim.gelfer@gmail.com>
parents: 2447
diff changeset
445 fp.close()
c91118f425d0 push over http: client support.
Vadim Gelfer <vadim.gelfer@gmail.com>
parents: 2447
diff changeset
446 os.unlink(tempname)
2439
e8c4f3d3df8c extend network protocol to stop clients from locking servers
Vadim Gelfer <vadim.gelfer@gmail.com>
parents: 2435
diff changeset
447
21074
f8a0d82b0463 httppeer: support for _calltwowaystream
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 20905
diff changeset
448 def _calltwowaystream(self, cmd, fp, **args):
f8a0d82b0463 httppeer: support for _calltwowaystream
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 20905
diff changeset
449 fh = None
23086
cde6904f1992 httppeer: close the temporary bundle file after two-way streaming it
Matt Harbison <matt_harbison@yahoo.com>
parents: 21188
diff changeset
450 fp_ = None
21074
f8a0d82b0463 httppeer: support for _calltwowaystream
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 20905
diff changeset
451 filename = None
f8a0d82b0463 httppeer: support for _calltwowaystream
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 20905
diff changeset
452 try:
f8a0d82b0463 httppeer: support for _calltwowaystream
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 20905
diff changeset
453 # dump bundle to disk
f8a0d82b0463 httppeer: support for _calltwowaystream
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 20905
diff changeset
454 fd, filename = tempfile.mkstemp(prefix="hg-bundle-", suffix=".hg")
30944
48dea083f66d py3: convert the mode argument of os.fdopen to unicodes (1 of 2)
Pulkit Goyal <7895pulkit@gmail.com>
parents: 30763
diff changeset
455 fh = os.fdopen(fd, pycompat.sysstr("wb"))
21074
f8a0d82b0463 httppeer: support for _calltwowaystream
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 20905
diff changeset
456 d = fp.read(4096)
f8a0d82b0463 httppeer: support for _calltwowaystream
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 20905
diff changeset
457 while d:
f8a0d82b0463 httppeer: support for _calltwowaystream
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 20905
diff changeset
458 fh.write(d)
f8a0d82b0463 httppeer: support for _calltwowaystream
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 20905
diff changeset
459 d = fp.read(4096)
f8a0d82b0463 httppeer: support for _calltwowaystream
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 20905
diff changeset
460 fh.close()
f8a0d82b0463 httppeer: support for _calltwowaystream
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 20905
diff changeset
461 # start http push
23086
cde6904f1992 httppeer: close the temporary bundle file after two-way streaming it
Matt Harbison <matt_harbison@yahoo.com>
parents: 21188
diff changeset
462 fp_ = httpconnection.httpsendfile(self.ui, filename, "rb")
21074
f8a0d82b0463 httppeer: support for _calltwowaystream
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 20905
diff changeset
463 headers = {'Content-Type': 'application/mercurial-0.1'}
23086
cde6904f1992 httppeer: close the temporary bundle file after two-way streaming it
Matt Harbison <matt_harbison@yahoo.com>
parents: 21188
diff changeset
464 return self._callstream(cmd, data=fp_, headers=headers, **args)
21074
f8a0d82b0463 httppeer: support for _calltwowaystream
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 20905
diff changeset
465 finally:
23086
cde6904f1992 httppeer: close the temporary bundle file after two-way streaming it
Matt Harbison <matt_harbison@yahoo.com>
parents: 21188
diff changeset
466 if fp_ is not None:
cde6904f1992 httppeer: close the temporary bundle file after two-way streaming it
Matt Harbison <matt_harbison@yahoo.com>
parents: 21188
diff changeset
467 fp_.close()
21074
f8a0d82b0463 httppeer: support for _calltwowaystream
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 20905
diff changeset
468 if fh is not None:
f8a0d82b0463 httppeer: support for _calltwowaystream
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 20905
diff changeset
469 fh.close()
f8a0d82b0463 httppeer: support for _calltwowaystream
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 20905
diff changeset
470 os.unlink(filename)
f8a0d82b0463 httppeer: support for _calltwowaystream
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 20905
diff changeset
471
20905
167047ba3cfa wireproto: drop the _decompress method in favor a new call type
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 18742
diff changeset
472 def _callcompressable(self, cmd, **args):
30473
e16e234b9ca3 httppeer: do decompression inside _callstream
Gregory Szorc <gregory.szorc@gmail.com>
parents: 30241
diff changeset
473 return self._callstream(cmd, _compressible=True, **args)
11370
db3f6f0e4e7d pushkey: add http support
Matt Mackall <mpm@selenic.com>
parents: 11153
diff changeset
474
21188
d36440d84328 httppeer: reintroduce _abort that accidentally was removed in 167047ba3cfa
Mads Kiilerich <madski@unity3d.com>
parents: 21074
diff changeset
475 def _abort(self, exception):
d36440d84328 httppeer: reintroduce _abort that accidentally was removed in 167047ba3cfa
Mads Kiilerich <madski@unity3d.com>
parents: 21074
diff changeset
476 raise exception
d36440d84328 httppeer: reintroduce _abort that accidentally was removed in 167047ba3cfa
Mads Kiilerich <madski@unity3d.com>
parents: 21074
diff changeset
477
17192
1ac628cd7113 peer: introduce real peer classes
Peter Arrenbrecht <peter.arrenbrecht@gmail.com>
parents: 15246
diff changeset
478 class httpspeer(httppeer):
2569
52ce0d6bc375 HTTPS: fix python2.3, persistent connections, don't explode if SSL is not available
Alexis S. L. Carvalho <alexis@cecm.usp.br>
parents: 2557
diff changeset
479 def __init__(self, ui, path):
7279
1f0f84660dea Fix https availability checking
Benoit Boissinot <benoit.boissinot@ens-lyon.org>
parents: 7270
diff changeset
480 if not url.has_https:
26587
56b2bcea2529 error: get Abort from 'error' instead of 'util'
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 25954
diff changeset
481 raise error.Abort(_('Python support for SSL and HTTPS '
2569
52ce0d6bc375 HTTPS: fix python2.3, persistent connections, don't explode if SSL is not available
Alexis S. L. Carvalho <alexis@cecm.usp.br>
parents: 2557
diff changeset
482 'is not installed'))
17192
1ac628cd7113 peer: introduce real peer classes
Peter Arrenbrecht <peter.arrenbrecht@gmail.com>
parents: 15246
diff changeset
483 httppeer.__init__(self, ui, path)
2740
386f04d6ecb3 clean up hg.py: move repo constructor code into each repo module
Vadim Gelfer <vadim.gelfer@gmail.com>
parents: 2673
diff changeset
484
386f04d6ecb3 clean up hg.py: move repo constructor code into each repo module
Vadim Gelfer <vadim.gelfer@gmail.com>
parents: 2673
diff changeset
485 def instance(ui, path, create):
386f04d6ecb3 clean up hg.py: move repo constructor code into each repo module
Vadim Gelfer <vadim.gelfer@gmail.com>
parents: 2673
diff changeset
486 if create:
26587
56b2bcea2529 error: get Abort from 'error' instead of 'util'
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 25954
diff changeset
487 raise error.Abort(_('cannot create new http repository'))
7211
25c0dee16ee0 Autodetect static-http
Matt Mackall <mpm@selenic.com>
parents: 7207
diff changeset
488 try:
25c0dee16ee0 Autodetect static-http
Matt Mackall <mpm@selenic.com>
parents: 7207
diff changeset
489 if path.startswith('https:'):
17192
1ac628cd7113 peer: introduce real peer classes
Peter Arrenbrecht <peter.arrenbrecht@gmail.com>
parents: 15246
diff changeset
490 inst = httpspeer(ui, path)
7211
25c0dee16ee0 Autodetect static-http
Matt Mackall <mpm@selenic.com>
parents: 7207
diff changeset
491 else:
17192
1ac628cd7113 peer: introduce real peer classes
Peter Arrenbrecht <peter.arrenbrecht@gmail.com>
parents: 15246
diff changeset
492 inst = httppeer(ui, path)
35924
197d10e157ce httppeer: remove support for connecting to <0.9.1 servers (BC)
Gregory Szorc <gregory.szorc@gmail.com>
parents: 35698
diff changeset
493
197d10e157ce httppeer: remove support for connecting to <0.9.1 servers (BC)
Gregory Szorc <gregory.szorc@gmail.com>
parents: 35698
diff changeset
494 inst._fetchcaps()
197d10e157ce httppeer: remove support for connecting to <0.9.1 servers (BC)
Gregory Szorc <gregory.szorc@gmail.com>
parents: 35698
diff changeset
495
7211
25c0dee16ee0 Autodetect static-http
Matt Mackall <mpm@selenic.com>
parents: 7207
diff changeset
496 return inst
25660
328739ea70c3 global: mass rewrite to use modern exception syntax
Gregory Szorc <gregory.szorc@gmail.com>
parents: 25500
diff changeset
497 except error.RepoError as httpexception:
14148
cc9366a3751b httprepo: use the original exception after falling back to static-http failed
Mads Kiilerich <mads@kiilerich.com>
parents: 14094
diff changeset
498 try:
cc9366a3751b httprepo: use the original exception after falling back to static-http failed
Mads Kiilerich <mads@kiilerich.com>
parents: 14094
diff changeset
499 r = statichttprepo.instance(ui, "static-" + path, create)
29241
269f7ea08983 httppeer: make a message translatable
FUJIWARA Katsunori <foozy@lares.dti.ne.jp>
parents: 28883
diff changeset
500 ui.note(_('(falling back to static-http)\n'))
14148
cc9366a3751b httprepo: use the original exception after falling back to static-http failed
Mads Kiilerich <mads@kiilerich.com>
parents: 14094
diff changeset
501 return r
cc9366a3751b httprepo: use the original exception after falling back to static-http failed
Mads Kiilerich <mads@kiilerich.com>
parents: 14094
diff changeset
502 except error.RepoError:
cc9366a3751b httprepo: use the original exception after falling back to static-http failed
Mads Kiilerich <mads@kiilerich.com>
parents: 14094
diff changeset
503 raise httpexception # use the original http RepoError instead