Mercurial > public > mercurial-scm > hg-stable

annotate mercurial/httpconnection.py @ 29248:e6de6ef3e426

Find changesets by keywords (author, files, the commit message), revision number or hash, or revset expression.
sslutil: remove ui from sslkwargs (API) Arguments to sslutil.wrapsocket() are partially determined by calling sslutil.sslkwargs(). This function receives a ui and a hostname and determines what settings, if any, need to be applied when the socket is wrapped. Both the ui and hostname are passed into wrapsocket(). The other arguments to wrapsocket() provided by sslkwargs() (ca_certs and cert_reqs) are not looked at or modified anywhere outside of sslutil.py. So, sslkwargs() doesn't need to exist as a separate public API called before wrapsocket(). This commit starts the process of removing external consumers of sslkwargs() by removing the "ui" key/argument from its return. All callers now pass the ui argument explicitly.
author Gregory Szorc <gregory.szorc@gmail.com>
date Wed, 25 May 2016 19:43:22 -0700
parents dffe78d80a6c
children d6b9468eebee
Ignore whitespace changes - Everywhere: Within whitespace: At end of lines:
rev   line source
14244
e7525a555a64 url: use new http support if requested by the user
Augie Fackler <durin42@gmail.com>
parents:
diff changeset
1 # httpconnection.py - urllib2 handler for new http support
e7525a555a64 url: use new http support if requested by the user
Augie Fackler <durin42@gmail.com>
parents:
diff changeset
2 #
e7525a555a64 url: use new http support if requested by the user
Augie Fackler <durin42@gmail.com>
parents:
diff changeset
3 # Copyright 2005, 2006, 2007, 2008 Matt Mackall <mpm@selenic.com>
e7525a555a64 url: use new http support if requested by the user
Augie Fackler <durin42@gmail.com>
parents:
diff changeset
4 # Copyright 2006, 2007 Alexis S. L. Carvalho <alexis@cecm.usp.br>
e7525a555a64 url: use new http support if requested by the user
Augie Fackler <durin42@gmail.com>
parents:
diff changeset
5 # Copyright 2006 Vadim Gelfer <vadim.gelfer@gmail.com>
e7525a555a64 url: use new http support if requested by the user
Augie Fackler <durin42@gmail.com>
parents:
diff changeset
6 # Copyright 2011 Google, Inc.
e7525a555a64 url: use new http support if requested by the user
Augie Fackler <durin42@gmail.com>
parents:
diff changeset
7 #
e7525a555a64 url: use new http support if requested by the user
Augie Fackler <durin42@gmail.com>
parents:
diff changeset
8 # This software may be used and distributed according to the terms of the
e7525a555a64 url: use new http support if requested by the user
Augie Fackler <durin42@gmail.com>
parents:
diff changeset
9 # GNU General Public License version 2 or any later version.
27521
b1adf32b0605 httpconnection: use absolute_import
Gregory Szorc <gregory.szorc@gmail.com>
parents: 26347
diff changeset
10
b1adf32b0605 httpconnection: use absolute_import
Gregory Szorc <gregory.szorc@gmail.com>
parents: 26347
diff changeset
11 from __future__ import absolute_import
b1adf32b0605 httpconnection: use absolute_import
Gregory Szorc <gregory.szorc@gmail.com>
parents: 26347
diff changeset
12
14244
e7525a555a64 url: use new http support if requested by the user
Augie Fackler <durin42@gmail.com>
parents:
diff changeset
13 import logging
27521
b1adf32b0605 httpconnection: use absolute_import
Gregory Szorc <gregory.szorc@gmail.com>
parents: 26347
diff changeset
14 import os
14244
e7525a555a64 url: use new http support if requested by the user
Augie Fackler <durin42@gmail.com>
parents:
diff changeset
15 import socket
e7525a555a64 url: use new http support if requested by the user
Augie Fackler <durin42@gmail.com>
parents:
diff changeset
16
27521
b1adf32b0605 httpconnection: use absolute_import
Gregory Szorc <gregory.szorc@gmail.com>
parents: 26347
diff changeset
17 from .i18n import _
b1adf32b0605 httpconnection: use absolute_import
Gregory Szorc <gregory.szorc@gmail.com>
parents: 26347
diff changeset
18 from . import (
b1adf32b0605 httpconnection: use absolute_import
Gregory Szorc <gregory.szorc@gmail.com>
parents: 26347
diff changeset
19 httpclient,
b1adf32b0605 httpconnection: use absolute_import
Gregory Szorc <gregory.szorc@gmail.com>
parents: 26347
diff changeset
20 sslutil,
b1adf32b0605 httpconnection: use absolute_import
Gregory Szorc <gregory.szorc@gmail.com>
parents: 26347
diff changeset
21 util,
b1adf32b0605 httpconnection: use absolute_import
Gregory Szorc <gregory.szorc@gmail.com>
parents: 26347
diff changeset
22 )
14244
e7525a555a64 url: use new http support if requested by the user
Augie Fackler <durin42@gmail.com>
parents:
diff changeset
23
28883
032c4c2f802a pycompat: switch to util.urlreq/util.urlerr for py3 compat
timeless <timeless@mozdev.org>
parents: 28526
diff changeset
24 urlerr = util.urlerr
032c4c2f802a pycompat: switch to util.urlreq/util.urlerr for py3 compat
timeless <timeless@mozdev.org>
parents: 28526
diff changeset
25 urlreq = util.urlreq
032c4c2f802a pycompat: switch to util.urlreq/util.urlerr for py3 compat
timeless <timeless@mozdev.org>
parents: 28526
diff changeset
26
14244
e7525a555a64 url: use new http support if requested by the user
Augie Fackler <durin42@gmail.com>
parents:
diff changeset
27 # moved here from url.py to avoid a cycle
e7525a555a64 url: use new http support if requested by the user
Augie Fackler <durin42@gmail.com>
parents:
diff changeset
28 class httpsendfile(object):
e7525a555a64 url: use new http support if requested by the user
Augie Fackler <durin42@gmail.com>
parents:
diff changeset
29 """This is a wrapper around the objects returned by python's "open".
e7525a555a64 url: use new http support if requested by the user
Augie Fackler <durin42@gmail.com>
parents:
diff changeset
30
15152
94b200a11cf7 http: handle push of bundles > 2 GB again (issue3017)
Mads Kiilerich <mads@kiilerich.com>
parents: 15025
diff changeset
31 Its purpose is to send file-like objects via HTTP.
94b200a11cf7 http: handle push of bundles > 2 GB again (issue3017)
Mads Kiilerich <mads@kiilerich.com>
parents: 15025
diff changeset
32 It do however not define a __len__ attribute because the length
94b200a11cf7 http: handle push of bundles > 2 GB again (issue3017)
Mads Kiilerich <mads@kiilerich.com>
parents: 15025
diff changeset
33 might be more than Py_ssize_t can handle.
14244
e7525a555a64 url: use new http support if requested by the user
Augie Fackler <durin42@gmail.com>
parents:
diff changeset
34 """
e7525a555a64 url: use new http support if requested by the user
Augie Fackler <durin42@gmail.com>
parents:
diff changeset
35
e7525a555a64 url: use new http support if requested by the user
Augie Fackler <durin42@gmail.com>
parents:
diff changeset
36 def __init__(self, ui, *args, **kwargs):
e7525a555a64 url: use new http support if requested by the user
Augie Fackler <durin42@gmail.com>
parents:
diff changeset
37 self.ui = ui
e7525a555a64 url: use new http support if requested by the user
Augie Fackler <durin42@gmail.com>
parents:
diff changeset
38 self._data = open(*args, **kwargs)
e7525a555a64 url: use new http support if requested by the user
Augie Fackler <durin42@gmail.com>
parents:
diff changeset
39 self.seek = self._data.seek
e7525a555a64 url: use new http support if requested by the user
Augie Fackler <durin42@gmail.com>
parents:
diff changeset
40 self.close = self._data.close
e7525a555a64 url: use new http support if requested by the user
Augie Fackler <durin42@gmail.com>
parents:
diff changeset
41 self.write = self._data.write
15152
94b200a11cf7 http: handle push of bundles > 2 GB again (issue3017)
Mads Kiilerich <mads@kiilerich.com>
parents: 15025
diff changeset
42 self.length = os.fstat(self._data.fileno()).st_size
14244
e7525a555a64 url: use new http support if requested by the user
Augie Fackler <durin42@gmail.com>
parents:
diff changeset
43 self._pos = 0
15791
a814f8fcc65a Use explicit integer division
Martin Geisler <mg@aragost.com>
parents: 15288
diff changeset
44 self._total = self.length // 1024 * 2
14244
e7525a555a64 url: use new http support if requested by the user
Augie Fackler <durin42@gmail.com>
parents:
diff changeset
45
e7525a555a64 url: use new http support if requested by the user
Augie Fackler <durin42@gmail.com>
parents:
diff changeset
46 def read(self, *args, **kwargs):
e7525a555a64 url: use new http support if requested by the user
Augie Fackler <durin42@gmail.com>
parents:
diff changeset
47 try:
e7525a555a64 url: use new http support if requested by the user
Augie Fackler <durin42@gmail.com>
parents:
diff changeset
48 ret = self._data.read(*args, **kwargs)
e7525a555a64 url: use new http support if requested by the user
Augie Fackler <durin42@gmail.com>
parents:
diff changeset
49 except EOFError:
e7525a555a64 url: use new http support if requested by the user
Augie Fackler <durin42@gmail.com>
parents:
diff changeset
50 self.ui.progress(_('sending'), None)
e7525a555a64 url: use new http support if requested by the user
Augie Fackler <durin42@gmail.com>
parents:
diff changeset
51 self._pos += len(ret)
e7525a555a64 url: use new http support if requested by the user
Augie Fackler <durin42@gmail.com>
parents:
diff changeset
52 # We pass double the max for total because we currently have
e7525a555a64 url: use new http support if requested by the user
Augie Fackler <durin42@gmail.com>
parents:
diff changeset
53 # to send the bundle twice in the case of a server that
e7525a555a64 url: use new http support if requested by the user
Augie Fackler <durin42@gmail.com>
parents:
diff changeset
54 # requires authentication. Since we can't know until we try
e7525a555a64 url: use new http support if requested by the user
Augie Fackler <durin42@gmail.com>
parents:
diff changeset
55 # once whether authentication will be required, just lie to
e7525a555a64 url: use new http support if requested by the user
Augie Fackler <durin42@gmail.com>
parents:
diff changeset
56 # the user and maybe the push succeeds suddenly at 50%.
15791
a814f8fcc65a Use explicit integer division
Martin Geisler <mg@aragost.com>
parents: 15288
diff changeset
57 self.ui.progress(_('sending'), self._pos // 1024,
14244
e7525a555a64 url: use new http support if requested by the user
Augie Fackler <durin42@gmail.com>
parents:
diff changeset
58 unit=_('kb'), total=self._total)
e7525a555a64 url: use new http support if requested by the user
Augie Fackler <durin42@gmail.com>
parents:
diff changeset
59 return ret
e7525a555a64 url: use new http support if requested by the user
Augie Fackler <durin42@gmail.com>
parents:
diff changeset
60
e7525a555a64 url: use new http support if requested by the user
Augie Fackler <durin42@gmail.com>
parents:
diff changeset
61 # moved here from url.py to avoid a cycle
15025
0593e8f81c71 http: pass user to readauthforuri() (fix 4a43e23b8c55)
Patrick Mezard <pmezard@gmail.com>
parents: 15005
diff changeset
62 def readauthforuri(ui, uri, user):
14244
e7525a555a64 url: use new http support if requested by the user
Augie Fackler <durin42@gmail.com>
parents:
diff changeset
63 # Read configuration
e7525a555a64 url: use new http support if requested by the user
Augie Fackler <durin42@gmail.com>
parents:
diff changeset
64 config = dict()
e7525a555a64 url: use new http support if requested by the user
Augie Fackler <durin42@gmail.com>
parents:
diff changeset
65 for key, val in ui.configitems('auth'):
e7525a555a64 url: use new http support if requested by the user
Augie Fackler <durin42@gmail.com>
parents:
diff changeset
66 if '.' not in key:
e7525a555a64 url: use new http support if requested by the user
Augie Fackler <durin42@gmail.com>
parents:
diff changeset
67 ui.warn(_("ignoring invalid [auth] key '%s'\n") % key)
e7525a555a64 url: use new http support if requested by the user
Augie Fackler <durin42@gmail.com>
parents:
diff changeset
68 continue
e7525a555a64 url: use new http support if requested by the user
Augie Fackler <durin42@gmail.com>
parents:
diff changeset
69 group, setting = key.rsplit('.', 1)
e7525a555a64 url: use new http support if requested by the user
Augie Fackler <durin42@gmail.com>
parents:
diff changeset
70 gdict = config.setdefault(group, dict())
e7525a555a64 url: use new http support if requested by the user
Augie Fackler <durin42@gmail.com>
parents:
diff changeset
71 if setting in ('username', 'cert', 'key'):
e7525a555a64 url: use new http support if requested by the user
Augie Fackler <durin42@gmail.com>
parents:
diff changeset
72 val = util.expandpath(val)
e7525a555a64 url: use new http support if requested by the user
Augie Fackler <durin42@gmail.com>
parents:
diff changeset
73 gdict[setting] = val
e7525a555a64 url: use new http support if requested by the user
Augie Fackler <durin42@gmail.com>
parents:
diff changeset
74
e7525a555a64 url: use new http support if requested by the user
Augie Fackler <durin42@gmail.com>
parents:
diff changeset
75 # Find the best match
25206
18a032704f0a httpconnection: drop Python 2.4 specify hack
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 19809
diff changeset
76 scheme, hostpath = uri.split('://', 1)
15005
4a43e23b8c55 hgweb: do not ignore [auth] if url has a username (issue2822)
Patrick Mezard <pmezard@gmail.com>
parents: 14430
diff changeset
77 bestuser = None
14244
e7525a555a64 url: use new http support if requested by the user
Augie Fackler <durin42@gmail.com>
parents:
diff changeset
78 bestlen = 0
e7525a555a64 url: use new http support if requested by the user
Augie Fackler <durin42@gmail.com>
parents:
diff changeset
79 bestauth = None
e7525a555a64 url: use new http support if requested by the user
Augie Fackler <durin42@gmail.com>
parents:
diff changeset
80 for group, auth in config.iteritems():
15005
4a43e23b8c55 hgweb: do not ignore [auth] if url has a username (issue2822)
Patrick Mezard <pmezard@gmail.com>
parents: 14430
diff changeset
81 if user and user != auth.get('username', user):
4a43e23b8c55 hgweb: do not ignore [auth] if url has a username (issue2822)
Patrick Mezard <pmezard@gmail.com>
parents: 14430
diff changeset
82 # If a username was set in the URI, the entry username
4a43e23b8c55 hgweb: do not ignore [auth] if url has a username (issue2822)
Patrick Mezard <pmezard@gmail.com>
parents: 14430
diff changeset
83 # must either match it or be unset
4a43e23b8c55 hgweb: do not ignore [auth] if url has a username (issue2822)
Patrick Mezard <pmezard@gmail.com>
parents: 14430
diff changeset
84 continue
14244
e7525a555a64 url: use new http support if requested by the user
Augie Fackler <durin42@gmail.com>
parents:
diff changeset
85 prefix = auth.get('prefix')
e7525a555a64 url: use new http support if requested by the user
Augie Fackler <durin42@gmail.com>
parents:
diff changeset
86 if not prefix:
e7525a555a64 url: use new http support if requested by the user
Augie Fackler <durin42@gmail.com>
parents:
diff changeset
87 continue
e7525a555a64 url: use new http support if requested by the user
Augie Fackler <durin42@gmail.com>
parents:
diff changeset
88 p = prefix.split('://', 1)
e7525a555a64 url: use new http support if requested by the user
Augie Fackler <durin42@gmail.com>
parents:
diff changeset
89 if len(p) > 1:
e7525a555a64 url: use new http support if requested by the user
Augie Fackler <durin42@gmail.com>
parents:
diff changeset
90 schemes, prefix = [p[0]], p[1]
e7525a555a64 url: use new http support if requested by the user
Augie Fackler <durin42@gmail.com>
parents:
diff changeset
91 else:
e7525a555a64 url: use new http support if requested by the user
Augie Fackler <durin42@gmail.com>
parents:
diff changeset
92 schemes = (auth.get('schemes') or 'https').split()
e7525a555a64 url: use new http support if requested by the user
Augie Fackler <durin42@gmail.com>
parents:
diff changeset
93 if (prefix == '*' or hostpath.startswith(prefix)) and \
15005
4a43e23b8c55 hgweb: do not ignore [auth] if url has a username (issue2822)
Patrick Mezard <pmezard@gmail.com>
parents: 14430
diff changeset
94 (len(prefix) > bestlen or (len(prefix) == bestlen and \
4a43e23b8c55 hgweb: do not ignore [auth] if url has a username (issue2822)
Patrick Mezard <pmezard@gmail.com>
parents: 14430
diff changeset
95 not bestuser and 'username' in auth)) \
4a43e23b8c55 hgweb: do not ignore [auth] if url has a username (issue2822)
Patrick Mezard <pmezard@gmail.com>
parents: 14430
diff changeset
96 and scheme in schemes:
14244
e7525a555a64 url: use new http support if requested by the user
Augie Fackler <durin42@gmail.com>
parents:
diff changeset
97 bestlen = len(prefix)
e7525a555a64 url: use new http support if requested by the user
Augie Fackler <durin42@gmail.com>
parents:
diff changeset
98 bestauth = group, auth
15005
4a43e23b8c55 hgweb: do not ignore [auth] if url has a username (issue2822)
Patrick Mezard <pmezard@gmail.com>
parents: 14430
diff changeset
99 bestuser = auth.get('username')
4a43e23b8c55 hgweb: do not ignore [auth] if url has a username (issue2822)
Patrick Mezard <pmezard@gmail.com>
parents: 14430
diff changeset
100 if user and not bestuser:
4a43e23b8c55 hgweb: do not ignore [auth] if url has a username (issue2822)
Patrick Mezard <pmezard@gmail.com>
parents: 14430
diff changeset
101 auth['username'] = user
14244
e7525a555a64 url: use new http support if requested by the user
Augie Fackler <durin42@gmail.com>
parents:
diff changeset
102 return bestauth
e7525a555a64 url: use new http support if requested by the user
Augie Fackler <durin42@gmail.com>
parents:
diff changeset
103
e7525a555a64 url: use new http support if requested by the user
Augie Fackler <durin42@gmail.com>
parents:
diff changeset
104 # Mercurial (at least until we can remove the old codepath) requires
e7525a555a64 url: use new http support if requested by the user
Augie Fackler <durin42@gmail.com>
parents:
diff changeset
105 # that the http response object be sufficiently file-like, so we
e7525a555a64 url: use new http support if requested by the user
Augie Fackler <durin42@gmail.com>
parents:
diff changeset
106 # provide a close() method here.
e7525a555a64 url: use new http support if requested by the user
Augie Fackler <durin42@gmail.com>
parents:
diff changeset
107 class HTTPResponse(httpclient.HTTPResponse):
e7525a555a64 url: use new http support if requested by the user
Augie Fackler <durin42@gmail.com>
parents:
diff changeset
108 def close(self):
e7525a555a64 url: use new http support if requested by the user
Augie Fackler <durin42@gmail.com>
parents:
diff changeset
109 pass
e7525a555a64 url: use new http support if requested by the user
Augie Fackler <durin42@gmail.com>
parents:
diff changeset
110
e7525a555a64 url: use new http support if requested by the user
Augie Fackler <durin42@gmail.com>
parents:
diff changeset
111 class HTTPConnection(httpclient.HTTPConnection):
e7525a555a64 url: use new http support if requested by the user
Augie Fackler <durin42@gmail.com>
parents:
diff changeset
112 response_class = HTTPResponse
26347
e9a35411bbbc httpconnection: remove a mutable default argument
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 25837
diff changeset
113 def request(self, method, uri, body=None, headers=None):
e9a35411bbbc httpconnection: remove a mutable default argument
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 25837
diff changeset
114 if headers is None:
e9a35411bbbc httpconnection: remove a mutable default argument
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 25837
diff changeset
115 headers = {}
14244
e7525a555a64 url: use new http support if requested by the user
Augie Fackler <durin42@gmail.com>
parents:
diff changeset
116 if isinstance(body, httpsendfile):
e7525a555a64 url: use new http support if requested by the user
Augie Fackler <durin42@gmail.com>
parents:
diff changeset
117 body.seek(0)
e7525a555a64 url: use new http support if requested by the user
Augie Fackler <durin42@gmail.com>
parents:
diff changeset
118 httpclient.HTTPConnection.request(self, method, uri, body=body,
e7525a555a64 url: use new http support if requested by the user
Augie Fackler <durin42@gmail.com>
parents:
diff changeset
119 headers=headers)
e7525a555a64 url: use new http support if requested by the user
Augie Fackler <durin42@gmail.com>
parents:
diff changeset
120
e7525a555a64 url: use new http support if requested by the user
Augie Fackler <durin42@gmail.com>
parents:
diff changeset
121
e7525a555a64 url: use new http support if requested by the user
Augie Fackler <durin42@gmail.com>
parents:
diff changeset
122 _configuredlogging = False
14375
436e5379d7ba httpconnection: improved logging formatting
Augie Fackler <durin42@gmail.com>
parents: 14346
diff changeset
123 LOGFMT = '%(levelname)s:%(name)s:%(lineno)d:%(message)s'
14244
e7525a555a64 url: use new http support if requested by the user
Augie Fackler <durin42@gmail.com>
parents:
diff changeset
124 # Subclass BOTH of these because otherwise urllib2 "helpfully"
e7525a555a64 url: use new http support if requested by the user
Augie Fackler <durin42@gmail.com>
parents:
diff changeset
125 # reinserts them since it notices we don't include any subclasses of
e7525a555a64 url: use new http support if requested by the user
Augie Fackler <durin42@gmail.com>
parents:
diff changeset
126 # them.
28883
032c4c2f802a pycompat: switch to util.urlreq/util.urlerr for py3 compat
timeless <timeless@mozdev.org>
parents: 28526
diff changeset
127 class http2handler(urlreq.httphandler, urlreq.httpshandler):
14244
e7525a555a64 url: use new http support if requested by the user
Augie Fackler <durin42@gmail.com>
parents:
diff changeset
128 def __init__(self, ui, pwmgr):
e7525a555a64 url: use new http support if requested by the user
Augie Fackler <durin42@gmail.com>
parents:
diff changeset
129 global _configuredlogging
28883
032c4c2f802a pycompat: switch to util.urlreq/util.urlerr for py3 compat
timeless <timeless@mozdev.org>
parents: 28526
diff changeset
130 urlreq.abstracthttphandler.__init__(self)
14244
e7525a555a64 url: use new http support if requested by the user
Augie Fackler <durin42@gmail.com>
parents:
diff changeset
131 self.ui = ui
e7525a555a64 url: use new http support if requested by the user
Augie Fackler <durin42@gmail.com>
parents:
diff changeset
132 self.pwmgr = pwmgr
e7525a555a64 url: use new http support if requested by the user
Augie Fackler <durin42@gmail.com>
parents:
diff changeset
133 self._connections = {}
25837
d343806dcf68 http2: mark experimental and developer options
Matt Mackall <mpm@selenic.com>
parents: 25660
diff changeset
134 # developer config: ui.http2debuglevel
14244
e7525a555a64 url: use new http support if requested by the user
Augie Fackler <durin42@gmail.com>
parents:
diff changeset
135 loglevel = ui.config('ui', 'http2debuglevel', default=None)
e7525a555a64 url: use new http support if requested by the user
Augie Fackler <durin42@gmail.com>
parents:
diff changeset
136 if loglevel and not _configuredlogging:
e7525a555a64 url: use new http support if requested by the user
Augie Fackler <durin42@gmail.com>
parents:
diff changeset
137 _configuredlogging = True
14294
84256ba2fbf7 httpconnection: fix debug logging option for httpclient
Augie Fackler <durin42@gmail.com>
parents: 14244
diff changeset
138 logger = logging.getLogger('mercurial.httpclient')
14244
e7525a555a64 url: use new http support if requested by the user
Augie Fackler <durin42@gmail.com>
parents:
diff changeset
139 logger.setLevel(getattr(logging, loglevel.upper()))
14375
436e5379d7ba httpconnection: improved logging formatting
Augie Fackler <durin42@gmail.com>
parents: 14346
diff changeset
140 handler = logging.StreamHandler()
436e5379d7ba httpconnection: improved logging formatting
Augie Fackler <durin42@gmail.com>
parents: 14346
diff changeset
141 handler.setFormatter(logging.Formatter(LOGFMT))
436e5379d7ba httpconnection: improved logging formatting
Augie Fackler <durin42@gmail.com>
parents: 14346
diff changeset
142 logger.addHandler(handler)
14244
e7525a555a64 url: use new http support if requested by the user
Augie Fackler <durin42@gmail.com>
parents:
diff changeset
143
e7525a555a64 url: use new http support if requested by the user
Augie Fackler <durin42@gmail.com>
parents:
diff changeset
144 def close_all(self):
e7525a555a64 url: use new http support if requested by the user
Augie Fackler <durin42@gmail.com>
parents:
diff changeset
145 """Close and remove all connection objects being kept for reuse."""
e7525a555a64 url: use new http support if requested by the user
Augie Fackler <durin42@gmail.com>
parents:
diff changeset
146 for openconns in self._connections.values():
e7525a555a64 url: use new http support if requested by the user
Augie Fackler <durin42@gmail.com>
parents:
diff changeset
147 for conn in openconns:
e7525a555a64 url: use new http support if requested by the user
Augie Fackler <durin42@gmail.com>
parents:
diff changeset
148 conn.close()
e7525a555a64 url: use new http support if requested by the user
Augie Fackler <durin42@gmail.com>
parents:
diff changeset
149 self._connections = {}
e7525a555a64 url: use new http support if requested by the user
Augie Fackler <durin42@gmail.com>
parents:
diff changeset
150
e7525a555a64 url: use new http support if requested by the user
Augie Fackler <durin42@gmail.com>
parents:
diff changeset
151 # shamelessly borrowed from urllib2.AbstractHTTPHandler
14346
bf85c2639700 httpconnection: correctly handle redirects from http to https
Augie Fackler <durin42@gmail.com>
parents: 14294
diff changeset
152 def do_open(self, http_class, req, use_ssl):
14244
e7525a555a64 url: use new http support if requested by the user
Augie Fackler <durin42@gmail.com>
parents:
diff changeset
153 """Return an addinfourl object for the request, using http_class.
e7525a555a64 url: use new http support if requested by the user
Augie Fackler <durin42@gmail.com>
parents:
diff changeset
154
e7525a555a64 url: use new http support if requested by the user
Augie Fackler <durin42@gmail.com>
parents:
diff changeset
155 http_class must implement the HTTPConnection API from httplib.
e7525a555a64 url: use new http support if requested by the user
Augie Fackler <durin42@gmail.com>
parents:
diff changeset
156 The addinfourl return value is a file-like object. It also
e7525a555a64 url: use new http support if requested by the user
Augie Fackler <durin42@gmail.com>
parents:
diff changeset
157 has methods and attributes including:
e7525a555a64 url: use new http support if requested by the user
Augie Fackler <durin42@gmail.com>
parents:
diff changeset
158 - info(): return a mimetools.Message object for the headers
e7525a555a64 url: use new http support if requested by the user
Augie Fackler <durin42@gmail.com>
parents:
diff changeset
159 - geturl(): return the original request URL
e7525a555a64 url: use new http support if requested by the user
Augie Fackler <durin42@gmail.com>
parents:
diff changeset
160 - code: HTTP status code
e7525a555a64 url: use new http support if requested by the user
Augie Fackler <durin42@gmail.com>
parents:
diff changeset
161 """
e7525a555a64 url: use new http support if requested by the user
Augie Fackler <durin42@gmail.com>
parents:
diff changeset
162 # If using a proxy, the host returned by get_host() is
e7525a555a64 url: use new http support if requested by the user
Augie Fackler <durin42@gmail.com>
parents:
diff changeset
163 # actually the proxy. On Python 2.6.1, the real destination
e7525a555a64 url: use new http support if requested by the user
Augie Fackler <durin42@gmail.com>
parents:
diff changeset
164 # hostname is encoded in the URI in the urllib2 request
e7525a555a64 url: use new http support if requested by the user
Augie Fackler <durin42@gmail.com>
parents:
diff changeset
165 # object. On Python 2.6.5, it's stored in the _tunnel_host
e7525a555a64 url: use new http support if requested by the user
Augie Fackler <durin42@gmail.com>
parents:
diff changeset
166 # attribute which has no accessor.
e7525a555a64 url: use new http support if requested by the user
Augie Fackler <durin42@gmail.com>
parents:
diff changeset
167 tunhost = getattr(req, '_tunnel_host', None)
e7525a555a64 url: use new http support if requested by the user
Augie Fackler <durin42@gmail.com>
parents:
diff changeset
168 host = req.get_host()
e7525a555a64 url: use new http support if requested by the user
Augie Fackler <durin42@gmail.com>
parents:
diff changeset
169 if tunhost:
e7525a555a64 url: use new http support if requested by the user
Augie Fackler <durin42@gmail.com>
parents:
diff changeset
170 proxyhost = host
e7525a555a64 url: use new http support if requested by the user
Augie Fackler <durin42@gmail.com>
parents:
diff changeset
171 host = tunhost
e7525a555a64 url: use new http support if requested by the user
Augie Fackler <durin42@gmail.com>
parents:
diff changeset
172 elif req.has_proxy():
e7525a555a64 url: use new http support if requested by the user
Augie Fackler <durin42@gmail.com>
parents:
diff changeset
173 proxyhost = req.get_host()
e7525a555a64 url: use new http support if requested by the user
Augie Fackler <durin42@gmail.com>
parents:
diff changeset
174 host = req.get_selector().split('://', 1)[1].split('/', 1)[0]
e7525a555a64 url: use new http support if requested by the user
Augie Fackler <durin42@gmail.com>
parents:
diff changeset
175 else:
e7525a555a64 url: use new http support if requested by the user
Augie Fackler <durin42@gmail.com>
parents:
diff changeset
176 proxyhost = None
e7525a555a64 url: use new http support if requested by the user
Augie Fackler <durin42@gmail.com>
parents:
diff changeset
177
e7525a555a64 url: use new http support if requested by the user
Augie Fackler <durin42@gmail.com>
parents:
diff changeset
178 if proxyhost:
e7525a555a64 url: use new http support if requested by the user
Augie Fackler <durin42@gmail.com>
parents:
diff changeset
179 if ':' in proxyhost:
e7525a555a64 url: use new http support if requested by the user
Augie Fackler <durin42@gmail.com>
parents:
diff changeset
180 # Note: this means we'll explode if we try and use an
e7525a555a64 url: use new http support if requested by the user
Augie Fackler <durin42@gmail.com>
parents:
diff changeset
181 # IPv6 http proxy. This isn't a regression, so we
e7525a555a64 url: use new http support if requested by the user
Augie Fackler <durin42@gmail.com>
parents:
diff changeset
182 # won't worry about it for now.
e7525a555a64 url: use new http support if requested by the user
Augie Fackler <durin42@gmail.com>
parents:
diff changeset
183 proxyhost, proxyport = proxyhost.rsplit(':', 1)
e7525a555a64 url: use new http support if requested by the user
Augie Fackler <durin42@gmail.com>
parents:
diff changeset
184 else:
e7525a555a64 url: use new http support if requested by the user
Augie Fackler <durin42@gmail.com>
parents:
diff changeset
185 proxyport = 3128 # squid default
e7525a555a64 url: use new http support if requested by the user
Augie Fackler <durin42@gmail.com>
parents:
diff changeset
186 proxy = (proxyhost, proxyport)
e7525a555a64 url: use new http support if requested by the user
Augie Fackler <durin42@gmail.com>
parents:
diff changeset
187 else:
e7525a555a64 url: use new http support if requested by the user
Augie Fackler <durin42@gmail.com>
parents:
diff changeset
188 proxy = None
e7525a555a64 url: use new http support if requested by the user
Augie Fackler <durin42@gmail.com>
parents:
diff changeset
189
e7525a555a64 url: use new http support if requested by the user
Augie Fackler <durin42@gmail.com>
parents:
diff changeset
190 if not host:
28883
032c4c2f802a pycompat: switch to util.urlreq/util.urlerr for py3 compat
timeless <timeless@mozdev.org>
parents: 28526
diff changeset
191 raise urlerr.urlerror('no host given')
14244
e7525a555a64 url: use new http support if requested by the user
Augie Fackler <durin42@gmail.com>
parents:
diff changeset
192
14346
bf85c2639700 httpconnection: correctly handle redirects from http to https
Augie Fackler <durin42@gmail.com>
parents: 14294
diff changeset
193 connkey = use_ssl, host, proxy
bf85c2639700 httpconnection: correctly handle redirects from http to https
Augie Fackler <durin42@gmail.com>
parents: 14294
diff changeset
194 allconns = self._connections.get(connkey, [])
14244
e7525a555a64 url: use new http support if requested by the user
Augie Fackler <durin42@gmail.com>
parents:
diff changeset
195 conns = [c for c in allconns if not c.busy()]
e7525a555a64 url: use new http support if requested by the user
Augie Fackler <durin42@gmail.com>
parents:
diff changeset
196 if conns:
e7525a555a64 url: use new http support if requested by the user
Augie Fackler <durin42@gmail.com>
parents:
diff changeset
197 h = conns[0]
e7525a555a64 url: use new http support if requested by the user
Augie Fackler <durin42@gmail.com>
parents:
diff changeset
198 else:
e7525a555a64 url: use new http support if requested by the user
Augie Fackler <durin42@gmail.com>
parents:
diff changeset
199 if allconns:
e7525a555a64 url: use new http support if requested by the user
Augie Fackler <durin42@gmail.com>
parents:
diff changeset
200 self.ui.debug('all connections for %s busy, making a new '
e7525a555a64 url: use new http support if requested by the user
Augie Fackler <durin42@gmail.com>
parents:
diff changeset
201 'one\n' % host)
e7525a555a64 url: use new http support if requested by the user
Augie Fackler <durin42@gmail.com>
parents:
diff changeset
202 timeout = None
e7525a555a64 url: use new http support if requested by the user
Augie Fackler <durin42@gmail.com>
parents:
diff changeset
203 if req.timeout is not socket._GLOBAL_DEFAULT_TIMEOUT:
e7525a555a64 url: use new http support if requested by the user
Augie Fackler <durin42@gmail.com>
parents:
diff changeset
204 timeout = req.timeout
e7525a555a64 url: use new http support if requested by the user
Augie Fackler <durin42@gmail.com>
parents:
diff changeset
205 h = http_class(host, timeout=timeout, proxy_hostport=proxy)
14346
bf85c2639700 httpconnection: correctly handle redirects from http to https
Augie Fackler <durin42@gmail.com>
parents: 14294
diff changeset
206 self._connections.setdefault(connkey, []).append(h)
14244
e7525a555a64 url: use new http support if requested by the user
Augie Fackler <durin42@gmail.com>
parents:
diff changeset
207
e7525a555a64 url: use new http support if requested by the user
Augie Fackler <durin42@gmail.com>
parents:
diff changeset
208 headers = dict(req.headers)
e7525a555a64 url: use new http support if requested by the user
Augie Fackler <durin42@gmail.com>
parents:
diff changeset
209 headers.update(req.unredirected_hdrs)
e7525a555a64 url: use new http support if requested by the user
Augie Fackler <durin42@gmail.com>
parents:
diff changeset
210 headers = dict(
e7525a555a64 url: use new http support if requested by the user
Augie Fackler <durin42@gmail.com>
parents:
diff changeset
211 (name.title(), val) for name, val in headers.items())
e7525a555a64 url: use new http support if requested by the user
Augie Fackler <durin42@gmail.com>
parents:
diff changeset
212 try:
e7525a555a64 url: use new http support if requested by the user
Augie Fackler <durin42@gmail.com>
parents:
diff changeset
213 path = req.get_selector()
e7525a555a64 url: use new http support if requested by the user
Augie Fackler <durin42@gmail.com>
parents:
diff changeset
214 if '://' in path:
e7525a555a64 url: use new http support if requested by the user
Augie Fackler <durin42@gmail.com>
parents:
diff changeset
215 path = path.split('://', 1)[1].split('/', 1)[1]
e7525a555a64 url: use new http support if requested by the user
Augie Fackler <durin42@gmail.com>
parents:
diff changeset
216 if path[0] != '/':
e7525a555a64 url: use new http support if requested by the user
Augie Fackler <durin42@gmail.com>
parents:
diff changeset
217 path = '/' + path
e7525a555a64 url: use new http support if requested by the user
Augie Fackler <durin42@gmail.com>
parents:
diff changeset
218 h.request(req.get_method(), path, req.data, headers)
e7525a555a64 url: use new http support if requested by the user
Augie Fackler <durin42@gmail.com>
parents:
diff changeset
219 r = h.getresponse()
25660
328739ea70c3 global: mass rewrite to use modern exception syntax
Gregory Szorc <gregory.szorc@gmail.com>
parents: 25429
diff changeset
220 except socket.error as err: # XXX what error?
28883
032c4c2f802a pycompat: switch to util.urlreq/util.urlerr for py3 compat
timeless <timeless@mozdev.org>
parents: 28526
diff changeset
221 raise urlerr.urlerror(err)
14244
e7525a555a64 url: use new http support if requested by the user
Augie Fackler <durin42@gmail.com>
parents:
diff changeset
222
e7525a555a64 url: use new http support if requested by the user
Augie Fackler <durin42@gmail.com>
parents:
diff changeset
223 # Pick apart the HTTPResponse object to get the addinfourl
e7525a555a64 url: use new http support if requested by the user
Augie Fackler <durin42@gmail.com>
parents:
diff changeset
224 # object initialized properly.
e7525a555a64 url: use new http support if requested by the user
Augie Fackler <durin42@gmail.com>
parents:
diff changeset
225 r.recv = r.read
e7525a555a64 url: use new http support if requested by the user
Augie Fackler <durin42@gmail.com>
parents:
diff changeset
226
28883
032c4c2f802a pycompat: switch to util.urlreq/util.urlerr for py3 compat
timeless <timeless@mozdev.org>
parents: 28526
diff changeset
227 resp = urlreq.addinfourl(r, r.headers, req.get_full_url())
14244
e7525a555a64 url: use new http support if requested by the user
Augie Fackler <durin42@gmail.com>
parents:
diff changeset
228 resp.code = r.status
e7525a555a64 url: use new http support if requested by the user
Augie Fackler <durin42@gmail.com>
parents:
diff changeset
229 resp.msg = r.reason
e7525a555a64 url: use new http support if requested by the user
Augie Fackler <durin42@gmail.com>
parents:
diff changeset
230 return resp
e7525a555a64 url: use new http support if requested by the user
Augie Fackler <durin42@gmail.com>
parents:
diff changeset
231
e7525a555a64 url: use new http support if requested by the user
Augie Fackler <durin42@gmail.com>
parents:
diff changeset
232 # httplib always uses the given host/port as the socket connect
e7525a555a64 url: use new http support if requested by the user
Augie Fackler <durin42@gmail.com>
parents:
diff changeset
233 # target, and then allows full URIs in the request path, which it
e7525a555a64 url: use new http support if requested by the user
Augie Fackler <durin42@gmail.com>
parents:
diff changeset
234 # then observes and treats as a signal to do proxying instead.
e7525a555a64 url: use new http support if requested by the user
Augie Fackler <durin42@gmail.com>
parents:
diff changeset
235 def http_open(self, req):
e7525a555a64 url: use new http support if requested by the user
Augie Fackler <durin42@gmail.com>
parents:
diff changeset
236 if req.get_full_url().startswith('https'):
e7525a555a64 url: use new http support if requested by the user
Augie Fackler <durin42@gmail.com>
parents:
diff changeset
237 return self.https_open(req)
17787
21503aa02d4f http2: make it possible to connect w/o ssl on port 443
Augie Fackler <raf@durin42.com>
parents: 17428
diff changeset
238 def makehttpcon(*args, **kwargs):
17836
98347af64593 httpclient: fix calling convention violation
Matt Mackall <mpm@selenic.com>
parents: 17787
diff changeset
239 k2 = dict(kwargs)
98347af64593 httpclient: fix calling convention violation
Matt Mackall <mpm@selenic.com>
parents: 17787
diff changeset
240 k2['use_ssl'] = False
98347af64593 httpclient: fix calling convention violation
Matt Mackall <mpm@selenic.com>
parents: 17787
diff changeset
241 return HTTPConnection(*args, **k2)
17787
21503aa02d4f http2: make it possible to connect w/o ssl on port 443
Augie Fackler <raf@durin42.com>
parents: 17428
diff changeset
242 return self.do_open(makehttpcon, req, False)
14244
e7525a555a64 url: use new http support if requested by the user
Augie Fackler <durin42@gmail.com>
parents:
diff changeset
243
e7525a555a64 url: use new http support if requested by the user
Augie Fackler <durin42@gmail.com>
parents:
diff changeset
244 def https_open(self, req):
15025
0593e8f81c71 http: pass user to readauthforuri() (fix 4a43e23b8c55)
Patrick Mezard <pmezard@gmail.com>
parents: 15005
diff changeset
245 # req.get_full_url() does not contain credentials and we may
0593e8f81c71 http: pass user to readauthforuri() (fix 4a43e23b8c55)
Patrick Mezard <pmezard@gmail.com>
parents: 15005
diff changeset
246 # need them to match the certificates.
0593e8f81c71 http: pass user to readauthforuri() (fix 4a43e23b8c55)
Patrick Mezard <pmezard@gmail.com>
parents: 15005
diff changeset
247 url = req.get_full_url()
0593e8f81c71 http: pass user to readauthforuri() (fix 4a43e23b8c55)
Patrick Mezard <pmezard@gmail.com>
parents: 15005
diff changeset
248 user, password = self.pwmgr.find_stored_password(url)
0593e8f81c71 http: pass user to readauthforuri() (fix 4a43e23b8c55)
Patrick Mezard <pmezard@gmail.com>
parents: 15005
diff changeset
249 res = readauthforuri(self.ui, url, user)
14244
e7525a555a64 url: use new http support if requested by the user
Augie Fackler <durin42@gmail.com>
parents:
diff changeset
250 if res:
e7525a555a64 url: use new http support if requested by the user
Augie Fackler <durin42@gmail.com>
parents:
diff changeset
251 group, auth = res
e7525a555a64 url: use new http support if requested by the user
Augie Fackler <durin42@gmail.com>
parents:
diff changeset
252 self.auth = auth
e7525a555a64 url: use new http support if requested by the user
Augie Fackler <durin42@gmail.com>
parents:
diff changeset
253 self.ui.debug("using auth.%s.* for authentication\n" % group)
e7525a555a64 url: use new http support if requested by the user
Augie Fackler <durin42@gmail.com>
parents:
diff changeset
254 else:
e7525a555a64 url: use new http support if requested by the user
Augie Fackler <durin42@gmail.com>
parents:
diff changeset
255 self.auth = None
14346
bf85c2639700 httpconnection: correctly handle redirects from http to https
Augie Fackler <durin42@gmail.com>
parents: 14294
diff changeset
256 return self.do_open(self._makesslconnection, req, True)
14244
e7525a555a64 url: use new http support if requested by the user
Augie Fackler <durin42@gmail.com>
parents:
diff changeset
257
e7525a555a64 url: use new http support if requested by the user
Augie Fackler <durin42@gmail.com>
parents:
diff changeset
258 def _makesslconnection(self, host, port=443, *args, **kwargs):
e7525a555a64 url: use new http support if requested by the user
Augie Fackler <durin42@gmail.com>
parents:
diff changeset
259 keyfile = None
e7525a555a64 url: use new http support if requested by the user
Augie Fackler <durin42@gmail.com>
parents:
diff changeset
260 certfile = None
e7525a555a64 url: use new http support if requested by the user
Augie Fackler <durin42@gmail.com>
parents:
diff changeset
261
e7525a555a64 url: use new http support if requested by the user
Augie Fackler <durin42@gmail.com>
parents:
diff changeset
262 if args: # key_file
e7525a555a64 url: use new http support if requested by the user
Augie Fackler <durin42@gmail.com>
parents:
diff changeset
263 keyfile = args.pop(0)
e7525a555a64 url: use new http support if requested by the user
Augie Fackler <durin42@gmail.com>
parents:
diff changeset
264 if args: # cert_file
e7525a555a64 url: use new http support if requested by the user
Augie Fackler <durin42@gmail.com>
parents:
diff changeset
265 certfile = args.pop(0)
e7525a555a64 url: use new http support if requested by the user
Augie Fackler <durin42@gmail.com>
parents:
diff changeset
266
e7525a555a64 url: use new http support if requested by the user
Augie Fackler <durin42@gmail.com>
parents:
diff changeset
267 # if the user has specified different key/cert files in
e7525a555a64 url: use new http support if requested by the user
Augie Fackler <durin42@gmail.com>
parents:
diff changeset
268 # hgrc, we prefer these
e7525a555a64 url: use new http support if requested by the user
Augie Fackler <durin42@gmail.com>
parents:
diff changeset
269 if self.auth and 'key' in self.auth and 'cert' in self.auth:
e7525a555a64 url: use new http support if requested by the user
Augie Fackler <durin42@gmail.com>
parents:
diff changeset
270 keyfile = self.auth['key']
e7525a555a64 url: use new http support if requested by the user
Augie Fackler <durin42@gmail.com>
parents:
diff changeset
271 certfile = self.auth['cert']
e7525a555a64 url: use new http support if requested by the user
Augie Fackler <durin42@gmail.com>
parents:
diff changeset
272
e7525a555a64 url: use new http support if requested by the user
Augie Fackler <durin42@gmail.com>
parents:
diff changeset
273 # let host port take precedence
e7525a555a64 url: use new http support if requested by the user
Augie Fackler <durin42@gmail.com>
parents:
diff changeset
274 if ':' in host and '[' not in host or ']:' in host:
e7525a555a64 url: use new http support if requested by the user
Augie Fackler <durin42@gmail.com>
parents:
diff changeset
275 host, port = host.rsplit(':', 1)
e7525a555a64 url: use new http support if requested by the user
Augie Fackler <durin42@gmail.com>
parents:
diff changeset
276 port = int(port)
e7525a555a64 url: use new http support if requested by the user
Augie Fackler <durin42@gmail.com>
parents:
diff changeset
277 if '[' in host:
e7525a555a64 url: use new http support if requested by the user
Augie Fackler <durin42@gmail.com>
parents:
diff changeset
278 host = host[1:-1]
e7525a555a64 url: use new http support if requested by the user
Augie Fackler <durin42@gmail.com>
parents:
diff changeset
279
19809
50d721553198 httpconnection: properly inject ssl_wrap_socket into httpclient (issue4038)
Augie Fackler <raf@durin42.com>
parents: 17836
diff changeset
280 kwargs['keyfile'] = keyfile
50d721553198 httpconnection: properly inject ssl_wrap_socket into httpclient (issue4038)
Augie Fackler <raf@durin42.com>
parents: 17836
diff changeset
281 kwargs['certfile'] = certfile
14244
e7525a555a64 url: use new http support if requested by the user
Augie Fackler <durin42@gmail.com>
parents:
diff changeset
282
e7525a555a64 url: use new http support if requested by the user
Augie Fackler <durin42@gmail.com>
parents:
diff changeset
283 kwargs.update(sslutil.sslkwargs(self.ui, host))
e7525a555a64 url: use new http support if requested by the user
Augie Fackler <durin42@gmail.com>
parents:
diff changeset
284
e7525a555a64 url: use new http support if requested by the user
Augie Fackler <durin42@gmail.com>
parents:
diff changeset
285 con = HTTPConnection(host, port, use_ssl=True,
25429
9d1c61715939 ssl: rename ssl_wrap_socket() to conform to our naming convention
Yuya Nishihara <yuya@tcha.org>
parents: 25206
diff changeset
286 ssl_wrap_socket=sslutil.wrapsocket,
29227
dffe78d80a6c sslutil: convert socket validation from a class to a function (API)
Gregory Szorc <gregory.szorc@gmail.com>
parents: 28883
diff changeset
287 ssl_validator=sslutil.validatesocket,
29248
e6de6ef3e426 sslutil: remove ui from sslkwargs (API)
Gregory Szorc <gregory.szorc@gmail.com>
parents: 29227
diff changeset
288 ui=self.ui,
14244
e7525a555a64 url: use new http support if requested by the user
Augie Fackler <durin42@gmail.com>
parents:
diff changeset
289 **kwargs)
e7525a555a64 url: use new http support if requested by the user
Augie Fackler <durin42@gmail.com>
parents:
diff changeset
290 return con