Mercurial > public > mercurial-scm > hg

annotate mercurial/httpconnection.py @ 15063:c20688b7c061 stable

Find changesets by keywords (author, files, the commit message), revision number or hash, or revset expression.
setdiscovery: fix hang when #heads>200 (issue2971) When setting up the next sample, we always add all of the heads, regardless of the desired max sample size. But if the number of heads exceeds this size, then we don't add any more nodes from the still undecided set. (This is debatable per se, and I'll investigate it, but it's how we designed it at the moment.) The bug was that we always added the overall heads, not the heads of the remaining undecided set. Thus, if #heads>200 (desired sample size), we did not make progress any longer.
author Peter Arrenbrecht <peter.arrenbrecht@gmail.com>
date Thu, 25 Aug 2011 21:25:14 +0200
parents 0593e8f81c71
children 94b200a11cf7
Ignore whitespace changes - Everywhere: Within whitespace: At end of lines:
rev   line source
14244
e7525a555a64 url: use new http support if requested by the user
Augie Fackler <durin42@gmail.com>
parents:
diff changeset
1 # httpconnection.py - urllib2 handler for new http support
e7525a555a64 url: use new http support if requested by the user
Augie Fackler <durin42@gmail.com>
parents:
diff changeset
2 #
e7525a555a64 url: use new http support if requested by the user
Augie Fackler <durin42@gmail.com>
parents:
diff changeset
3 # Copyright 2005, 2006, 2007, 2008 Matt Mackall <mpm@selenic.com>
e7525a555a64 url: use new http support if requested by the user
Augie Fackler <durin42@gmail.com>
parents:
diff changeset
4 # Copyright 2006, 2007 Alexis S. L. Carvalho <alexis@cecm.usp.br>
e7525a555a64 url: use new http support if requested by the user
Augie Fackler <durin42@gmail.com>
parents:
diff changeset
5 # Copyright 2006 Vadim Gelfer <vadim.gelfer@gmail.com>
e7525a555a64 url: use new http support if requested by the user
Augie Fackler <durin42@gmail.com>
parents:
diff changeset
6 # Copyright 2011 Google, Inc.
e7525a555a64 url: use new http support if requested by the user
Augie Fackler <durin42@gmail.com>
parents:
diff changeset
7 #
e7525a555a64 url: use new http support if requested by the user
Augie Fackler <durin42@gmail.com>
parents:
diff changeset
8 # This software may be used and distributed according to the terms of the
e7525a555a64 url: use new http support if requested by the user
Augie Fackler <durin42@gmail.com>
parents:
diff changeset
9 # GNU General Public License version 2 or any later version.
e7525a555a64 url: use new http support if requested by the user
Augie Fackler <durin42@gmail.com>
parents:
diff changeset
10 import logging
e7525a555a64 url: use new http support if requested by the user
Augie Fackler <durin42@gmail.com>
parents:
diff changeset
11 import socket
e7525a555a64 url: use new http support if requested by the user
Augie Fackler <durin42@gmail.com>
parents:
diff changeset
12 import urllib
e7525a555a64 url: use new http support if requested by the user
Augie Fackler <durin42@gmail.com>
parents:
diff changeset
13 import urllib2
e7525a555a64 url: use new http support if requested by the user
Augie Fackler <durin42@gmail.com>
parents:
diff changeset
14 import os
e7525a555a64 url: use new http support if requested by the user
Augie Fackler <durin42@gmail.com>
parents:
diff changeset
15
e7525a555a64 url: use new http support if requested by the user
Augie Fackler <durin42@gmail.com>
parents:
diff changeset
16 from mercurial import httpclient
e7525a555a64 url: use new http support if requested by the user
Augie Fackler <durin42@gmail.com>
parents:
diff changeset
17 from mercurial import sslutil
e7525a555a64 url: use new http support if requested by the user
Augie Fackler <durin42@gmail.com>
parents:
diff changeset
18 from mercurial import util
e7525a555a64 url: use new http support if requested by the user
Augie Fackler <durin42@gmail.com>
parents:
diff changeset
19 from mercurial.i18n import _
e7525a555a64 url: use new http support if requested by the user
Augie Fackler <durin42@gmail.com>
parents:
diff changeset
20
e7525a555a64 url: use new http support if requested by the user
Augie Fackler <durin42@gmail.com>
parents:
diff changeset
21 # moved here from url.py to avoid a cycle
e7525a555a64 url: use new http support if requested by the user
Augie Fackler <durin42@gmail.com>
parents:
diff changeset
22 class httpsendfile(object):
e7525a555a64 url: use new http support if requested by the user
Augie Fackler <durin42@gmail.com>
parents:
diff changeset
23 """This is a wrapper around the objects returned by python's "open".
e7525a555a64 url: use new http support if requested by the user
Augie Fackler <durin42@gmail.com>
parents:
diff changeset
24
e7525a555a64 url: use new http support if requested by the user
Augie Fackler <durin42@gmail.com>
parents:
diff changeset
25 Its purpose is to send file-like objects via HTTP and, to do so, it
e7525a555a64 url: use new http support if requested by the user
Augie Fackler <durin42@gmail.com>
parents:
diff changeset
26 defines a __len__ attribute to feed the Content-Length header.
e7525a555a64 url: use new http support if requested by the user
Augie Fackler <durin42@gmail.com>
parents:
diff changeset
27 """
e7525a555a64 url: use new http support if requested by the user
Augie Fackler <durin42@gmail.com>
parents:
diff changeset
28
e7525a555a64 url: use new http support if requested by the user
Augie Fackler <durin42@gmail.com>
parents:
diff changeset
29 def __init__(self, ui, *args, **kwargs):
e7525a555a64 url: use new http support if requested by the user
Augie Fackler <durin42@gmail.com>
parents:
diff changeset
30 # We can't just "self._data = open(*args, **kwargs)" here because there
e7525a555a64 url: use new http support if requested by the user
Augie Fackler <durin42@gmail.com>
parents:
diff changeset
31 # is an "open" function defined in this module that shadows the global
e7525a555a64 url: use new http support if requested by the user
Augie Fackler <durin42@gmail.com>
parents:
diff changeset
32 # one
e7525a555a64 url: use new http support if requested by the user
Augie Fackler <durin42@gmail.com>
parents:
diff changeset
33 self.ui = ui
e7525a555a64 url: use new http support if requested by the user
Augie Fackler <durin42@gmail.com>
parents:
diff changeset
34 self._data = open(*args, **kwargs)
e7525a555a64 url: use new http support if requested by the user
Augie Fackler <durin42@gmail.com>
parents:
diff changeset
35 self.seek = self._data.seek
e7525a555a64 url: use new http support if requested by the user
Augie Fackler <durin42@gmail.com>
parents:
diff changeset
36 self.close = self._data.close
e7525a555a64 url: use new http support if requested by the user
Augie Fackler <durin42@gmail.com>
parents:
diff changeset
37 self.write = self._data.write
e7525a555a64 url: use new http support if requested by the user
Augie Fackler <durin42@gmail.com>
parents:
diff changeset
38 self._len = os.fstat(self._data.fileno()).st_size
e7525a555a64 url: use new http support if requested by the user
Augie Fackler <durin42@gmail.com>
parents:
diff changeset
39 self._pos = 0
14430
c864f5e743ef httprepo: handle large lengths by bypassing the len() operator
Matt Mackall <mpm@selenic.com>
parents: 14375
diff changeset
40 self._total = self._len / 1024 * 2
14244
e7525a555a64 url: use new http support if requested by the user
Augie Fackler <durin42@gmail.com>
parents:
diff changeset
41
e7525a555a64 url: use new http support if requested by the user
Augie Fackler <durin42@gmail.com>
parents:
diff changeset
42 def read(self, *args, **kwargs):
e7525a555a64 url: use new http support if requested by the user
Augie Fackler <durin42@gmail.com>
parents:
diff changeset
43 try:
e7525a555a64 url: use new http support if requested by the user
Augie Fackler <durin42@gmail.com>
parents:
diff changeset
44 ret = self._data.read(*args, **kwargs)
e7525a555a64 url: use new http support if requested by the user
Augie Fackler <durin42@gmail.com>
parents:
diff changeset
45 except EOFError:
e7525a555a64 url: use new http support if requested by the user
Augie Fackler <durin42@gmail.com>
parents:
diff changeset
46 self.ui.progress(_('sending'), None)
e7525a555a64 url: use new http support if requested by the user
Augie Fackler <durin42@gmail.com>
parents:
diff changeset
47 self._pos += len(ret)
e7525a555a64 url: use new http support if requested by the user
Augie Fackler <durin42@gmail.com>
parents:
diff changeset
48 # We pass double the max for total because we currently have
e7525a555a64 url: use new http support if requested by the user
Augie Fackler <durin42@gmail.com>
parents:
diff changeset
49 # to send the bundle twice in the case of a server that
e7525a555a64 url: use new http support if requested by the user
Augie Fackler <durin42@gmail.com>
parents:
diff changeset
50 # requires authentication. Since we can't know until we try
e7525a555a64 url: use new http support if requested by the user
Augie Fackler <durin42@gmail.com>
parents:
diff changeset
51 # once whether authentication will be required, just lie to
e7525a555a64 url: use new http support if requested by the user
Augie Fackler <durin42@gmail.com>
parents:
diff changeset
52 # the user and maybe the push succeeds suddenly at 50%.
e7525a555a64 url: use new http support if requested by the user
Augie Fackler <durin42@gmail.com>
parents:
diff changeset
53 self.ui.progress(_('sending'), self._pos / 1024,
e7525a555a64 url: use new http support if requested by the user
Augie Fackler <durin42@gmail.com>
parents:
diff changeset
54 unit=_('kb'), total=self._total)
e7525a555a64 url: use new http support if requested by the user
Augie Fackler <durin42@gmail.com>
parents:
diff changeset
55 return ret
e7525a555a64 url: use new http support if requested by the user
Augie Fackler <durin42@gmail.com>
parents:
diff changeset
56
e7525a555a64 url: use new http support if requested by the user
Augie Fackler <durin42@gmail.com>
parents:
diff changeset
57 def __len__(self):
e7525a555a64 url: use new http support if requested by the user
Augie Fackler <durin42@gmail.com>
parents:
diff changeset
58 return self._len
e7525a555a64 url: use new http support if requested by the user
Augie Fackler <durin42@gmail.com>
parents:
diff changeset
59
e7525a555a64 url: use new http support if requested by the user
Augie Fackler <durin42@gmail.com>
parents:
diff changeset
60 # moved here from url.py to avoid a cycle
15025
0593e8f81c71 http: pass user to readauthforuri() (fix 4a43e23b8c55)
Patrick Mezard <pmezard@gmail.com>
parents: 15005
diff changeset
61 def readauthforuri(ui, uri, user):
14244
e7525a555a64 url: use new http support if requested by the user
Augie Fackler <durin42@gmail.com>
parents:
diff changeset
62 # Read configuration
e7525a555a64 url: use new http support if requested by the user
Augie Fackler <durin42@gmail.com>
parents:
diff changeset
63 config = dict()
e7525a555a64 url: use new http support if requested by the user
Augie Fackler <durin42@gmail.com>
parents:
diff changeset
64 for key, val in ui.configitems('auth'):
e7525a555a64 url: use new http support if requested by the user
Augie Fackler <durin42@gmail.com>
parents:
diff changeset
65 if '.' not in key:
e7525a555a64 url: use new http support if requested by the user
Augie Fackler <durin42@gmail.com>
parents:
diff changeset
66 ui.warn(_("ignoring invalid [auth] key '%s'\n") % key)
e7525a555a64 url: use new http support if requested by the user
Augie Fackler <durin42@gmail.com>
parents:
diff changeset
67 continue
e7525a555a64 url: use new http support if requested by the user
Augie Fackler <durin42@gmail.com>
parents:
diff changeset
68 group, setting = key.rsplit('.', 1)
e7525a555a64 url: use new http support if requested by the user
Augie Fackler <durin42@gmail.com>
parents:
diff changeset
69 gdict = config.setdefault(group, dict())
e7525a555a64 url: use new http support if requested by the user
Augie Fackler <durin42@gmail.com>
parents:
diff changeset
70 if setting in ('username', 'cert', 'key'):
e7525a555a64 url: use new http support if requested by the user
Augie Fackler <durin42@gmail.com>
parents:
diff changeset
71 val = util.expandpath(val)
e7525a555a64 url: use new http support if requested by the user
Augie Fackler <durin42@gmail.com>
parents:
diff changeset
72 gdict[setting] = val
e7525a555a64 url: use new http support if requested by the user
Augie Fackler <durin42@gmail.com>
parents:
diff changeset
73
e7525a555a64 url: use new http support if requested by the user
Augie Fackler <durin42@gmail.com>
parents:
diff changeset
74 # Find the best match
e7525a555a64 url: use new http support if requested by the user
Augie Fackler <durin42@gmail.com>
parents:
diff changeset
75 scheme, hostpath = uri.split('://', 1)
15005
4a43e23b8c55 hgweb: do not ignore [auth] if url has a username (issue2822)
Patrick Mezard <pmezard@gmail.com>
parents: 14430
diff changeset
76 bestuser = None
14244
e7525a555a64 url: use new http support if requested by the user
Augie Fackler <durin42@gmail.com>
parents:
diff changeset
77 bestlen = 0
e7525a555a64 url: use new http support if requested by the user
Augie Fackler <durin42@gmail.com>
parents:
diff changeset
78 bestauth = None
e7525a555a64 url: use new http support if requested by the user
Augie Fackler <durin42@gmail.com>
parents:
diff changeset
79 for group, auth in config.iteritems():
15005
4a43e23b8c55 hgweb: do not ignore [auth] if url has a username (issue2822)
Patrick Mezard <pmezard@gmail.com>
parents: 14430
diff changeset
80 if user and user != auth.get('username', user):
4a43e23b8c55 hgweb: do not ignore [auth] if url has a username (issue2822)
Patrick Mezard <pmezard@gmail.com>
parents: 14430
diff changeset
81 # If a username was set in the URI, the entry username
4a43e23b8c55 hgweb: do not ignore [auth] if url has a username (issue2822)
Patrick Mezard <pmezard@gmail.com>
parents: 14430
diff changeset
82 # must either match it or be unset
4a43e23b8c55 hgweb: do not ignore [auth] if url has a username (issue2822)
Patrick Mezard <pmezard@gmail.com>
parents: 14430
diff changeset
83 continue
14244
e7525a555a64 url: use new http support if requested by the user
Augie Fackler <durin42@gmail.com>
parents:
diff changeset
84 prefix = auth.get('prefix')
e7525a555a64 url: use new http support if requested by the user
Augie Fackler <durin42@gmail.com>
parents:
diff changeset
85 if not prefix:
e7525a555a64 url: use new http support if requested by the user
Augie Fackler <durin42@gmail.com>
parents:
diff changeset
86 continue
e7525a555a64 url: use new http support if requested by the user
Augie Fackler <durin42@gmail.com>
parents:
diff changeset
87 p = prefix.split('://', 1)
e7525a555a64 url: use new http support if requested by the user
Augie Fackler <durin42@gmail.com>
parents:
diff changeset
88 if len(p) > 1:
e7525a555a64 url: use new http support if requested by the user
Augie Fackler <durin42@gmail.com>
parents:
diff changeset
89 schemes, prefix = [p[0]], p[1]
e7525a555a64 url: use new http support if requested by the user
Augie Fackler <durin42@gmail.com>
parents:
diff changeset
90 else:
e7525a555a64 url: use new http support if requested by the user
Augie Fackler <durin42@gmail.com>
parents:
diff changeset
91 schemes = (auth.get('schemes') or 'https').split()
e7525a555a64 url: use new http support if requested by the user
Augie Fackler <durin42@gmail.com>
parents:
diff changeset
92 if (prefix == '*' or hostpath.startswith(prefix)) and \
15005
4a43e23b8c55 hgweb: do not ignore [auth] if url has a username (issue2822)
Patrick Mezard <pmezard@gmail.com>
parents: 14430
diff changeset
93 (len(prefix) > bestlen or (len(prefix) == bestlen and \
4a43e23b8c55 hgweb: do not ignore [auth] if url has a username (issue2822)
Patrick Mezard <pmezard@gmail.com>
parents: 14430
diff changeset
94 not bestuser and 'username' in auth)) \
4a43e23b8c55 hgweb: do not ignore [auth] if url has a username (issue2822)
Patrick Mezard <pmezard@gmail.com>
parents: 14430
diff changeset
95 and scheme in schemes:
14244
e7525a555a64 url: use new http support if requested by the user
Augie Fackler <durin42@gmail.com>
parents:
diff changeset
96 bestlen = len(prefix)
e7525a555a64 url: use new http support if requested by the user
Augie Fackler <durin42@gmail.com>
parents:
diff changeset
97 bestauth = group, auth
15005
4a43e23b8c55 hgweb: do not ignore [auth] if url has a username (issue2822)
Patrick Mezard <pmezard@gmail.com>
parents: 14430
diff changeset
98 bestuser = auth.get('username')
4a43e23b8c55 hgweb: do not ignore [auth] if url has a username (issue2822)
Patrick Mezard <pmezard@gmail.com>
parents: 14430
diff changeset
99 if user and not bestuser:
4a43e23b8c55 hgweb: do not ignore [auth] if url has a username (issue2822)
Patrick Mezard <pmezard@gmail.com>
parents: 14430
diff changeset
100 auth['username'] = user
14244
e7525a555a64 url: use new http support if requested by the user
Augie Fackler <durin42@gmail.com>
parents:
diff changeset
101 return bestauth
e7525a555a64 url: use new http support if requested by the user
Augie Fackler <durin42@gmail.com>
parents:
diff changeset
102
e7525a555a64 url: use new http support if requested by the user
Augie Fackler <durin42@gmail.com>
parents:
diff changeset
103 # Mercurial (at least until we can remove the old codepath) requires
e7525a555a64 url: use new http support if requested by the user
Augie Fackler <durin42@gmail.com>
parents:
diff changeset
104 # that the http response object be sufficiently file-like, so we
e7525a555a64 url: use new http support if requested by the user
Augie Fackler <durin42@gmail.com>
parents:
diff changeset
105 # provide a close() method here.
e7525a555a64 url: use new http support if requested by the user
Augie Fackler <durin42@gmail.com>
parents:
diff changeset
106 class HTTPResponse(httpclient.HTTPResponse):
e7525a555a64 url: use new http support if requested by the user
Augie Fackler <durin42@gmail.com>
parents:
diff changeset
107 def close(self):
e7525a555a64 url: use new http support if requested by the user
Augie Fackler <durin42@gmail.com>
parents:
diff changeset
108 pass
e7525a555a64 url: use new http support if requested by the user
Augie Fackler <durin42@gmail.com>
parents:
diff changeset
109
e7525a555a64 url: use new http support if requested by the user
Augie Fackler <durin42@gmail.com>
parents:
diff changeset
110 class HTTPConnection(httpclient.HTTPConnection):
e7525a555a64 url: use new http support if requested by the user
Augie Fackler <durin42@gmail.com>
parents:
diff changeset
111 response_class = HTTPResponse
e7525a555a64 url: use new http support if requested by the user
Augie Fackler <durin42@gmail.com>
parents:
diff changeset
112 def request(self, method, uri, body=None, headers={}):
e7525a555a64 url: use new http support if requested by the user
Augie Fackler <durin42@gmail.com>
parents:
diff changeset
113 if isinstance(body, httpsendfile):
e7525a555a64 url: use new http support if requested by the user
Augie Fackler <durin42@gmail.com>
parents:
diff changeset
114 body.seek(0)
e7525a555a64 url: use new http support if requested by the user
Augie Fackler <durin42@gmail.com>
parents:
diff changeset
115 httpclient.HTTPConnection.request(self, method, uri, body=body,
e7525a555a64 url: use new http support if requested by the user
Augie Fackler <durin42@gmail.com>
parents:
diff changeset
116 headers=headers)
e7525a555a64 url: use new http support if requested by the user
Augie Fackler <durin42@gmail.com>
parents:
diff changeset
117
e7525a555a64 url: use new http support if requested by the user
Augie Fackler <durin42@gmail.com>
parents:
diff changeset
118
e7525a555a64 url: use new http support if requested by the user
Augie Fackler <durin42@gmail.com>
parents:
diff changeset
119 _configuredlogging = False
14375
436e5379d7ba httpconnection: improved logging formatting
Augie Fackler <durin42@gmail.com>
parents: 14346
diff changeset
120 LOGFMT = '%(levelname)s:%(name)s:%(lineno)d:%(message)s'
14244
e7525a555a64 url: use new http support if requested by the user
Augie Fackler <durin42@gmail.com>
parents:
diff changeset
121 # Subclass BOTH of these because otherwise urllib2 "helpfully"
e7525a555a64 url: use new http support if requested by the user
Augie Fackler <durin42@gmail.com>
parents:
diff changeset
122 # reinserts them since it notices we don't include any subclasses of
e7525a555a64 url: use new http support if requested by the user
Augie Fackler <durin42@gmail.com>
parents:
diff changeset
123 # them.
e7525a555a64 url: use new http support if requested by the user
Augie Fackler <durin42@gmail.com>
parents:
diff changeset
124 class http2handler(urllib2.HTTPHandler, urllib2.HTTPSHandler):
e7525a555a64 url: use new http support if requested by the user
Augie Fackler <durin42@gmail.com>
parents:
diff changeset
125 def __init__(self, ui, pwmgr):
e7525a555a64 url: use new http support if requested by the user
Augie Fackler <durin42@gmail.com>
parents:
diff changeset
126 global _configuredlogging
e7525a555a64 url: use new http support if requested by the user
Augie Fackler <durin42@gmail.com>
parents:
diff changeset
127 urllib2.AbstractHTTPHandler.__init__(self)
e7525a555a64 url: use new http support if requested by the user
Augie Fackler <durin42@gmail.com>
parents:
diff changeset
128 self.ui = ui
e7525a555a64 url: use new http support if requested by the user
Augie Fackler <durin42@gmail.com>
parents:
diff changeset
129 self.pwmgr = pwmgr
e7525a555a64 url: use new http support if requested by the user
Augie Fackler <durin42@gmail.com>
parents:
diff changeset
130 self._connections = {}
e7525a555a64 url: use new http support if requested by the user
Augie Fackler <durin42@gmail.com>
parents:
diff changeset
131 loglevel = ui.config('ui', 'http2debuglevel', default=None)
e7525a555a64 url: use new http support if requested by the user
Augie Fackler <durin42@gmail.com>
parents:
diff changeset
132 if loglevel and not _configuredlogging:
e7525a555a64 url: use new http support if requested by the user
Augie Fackler <durin42@gmail.com>
parents:
diff changeset
133 _configuredlogging = True
14294
84256ba2fbf7 httpconnection: fix debug logging option for httpclient
Augie Fackler <durin42@gmail.com>
parents: 14244
diff changeset
134 logger = logging.getLogger('mercurial.httpclient')
14244
e7525a555a64 url: use new http support if requested by the user
Augie Fackler <durin42@gmail.com>
parents:
diff changeset
135 logger.setLevel(getattr(logging, loglevel.upper()))
14375
436e5379d7ba httpconnection: improved logging formatting
Augie Fackler <durin42@gmail.com>
parents: 14346
diff changeset
136 handler = logging.StreamHandler()
436e5379d7ba httpconnection: improved logging formatting
Augie Fackler <durin42@gmail.com>
parents: 14346
diff changeset
137 handler.setFormatter(logging.Formatter(LOGFMT))
436e5379d7ba httpconnection: improved logging formatting
Augie Fackler <durin42@gmail.com>
parents: 14346
diff changeset
138 logger.addHandler(handler)
14244
e7525a555a64 url: use new http support if requested by the user
Augie Fackler <durin42@gmail.com>
parents:
diff changeset
139
e7525a555a64 url: use new http support if requested by the user
Augie Fackler <durin42@gmail.com>
parents:
diff changeset
140 def close_all(self):
e7525a555a64 url: use new http support if requested by the user
Augie Fackler <durin42@gmail.com>
parents:
diff changeset
141 """Close and remove all connection objects being kept for reuse."""
e7525a555a64 url: use new http support if requested by the user
Augie Fackler <durin42@gmail.com>
parents:
diff changeset
142 for openconns in self._connections.values():
e7525a555a64 url: use new http support if requested by the user
Augie Fackler <durin42@gmail.com>
parents:
diff changeset
143 for conn in openconns:
e7525a555a64 url: use new http support if requested by the user
Augie Fackler <durin42@gmail.com>
parents:
diff changeset
144 conn.close()
e7525a555a64 url: use new http support if requested by the user
Augie Fackler <durin42@gmail.com>
parents:
diff changeset
145 self._connections = {}
e7525a555a64 url: use new http support if requested by the user
Augie Fackler <durin42@gmail.com>
parents:
diff changeset
146
e7525a555a64 url: use new http support if requested by the user
Augie Fackler <durin42@gmail.com>
parents:
diff changeset
147 # shamelessly borrowed from urllib2.AbstractHTTPHandler
14346
bf85c2639700 httpconnection: correctly handle redirects from http to https
Augie Fackler <durin42@gmail.com>
parents: 14294
diff changeset
148 def do_open(self, http_class, req, use_ssl):
14244
e7525a555a64 url: use new http support if requested by the user
Augie Fackler <durin42@gmail.com>
parents:
diff changeset
149 """Return an addinfourl object for the request, using http_class.
e7525a555a64 url: use new http support if requested by the user
Augie Fackler <durin42@gmail.com>
parents:
diff changeset
150
e7525a555a64 url: use new http support if requested by the user
Augie Fackler <durin42@gmail.com>
parents:
diff changeset
151 http_class must implement the HTTPConnection API from httplib.
e7525a555a64 url: use new http support if requested by the user
Augie Fackler <durin42@gmail.com>
parents:
diff changeset
152 The addinfourl return value is a file-like object. It also
e7525a555a64 url: use new http support if requested by the user
Augie Fackler <durin42@gmail.com>
parents:
diff changeset
153 has methods and attributes including:
e7525a555a64 url: use new http support if requested by the user
Augie Fackler <durin42@gmail.com>
parents:
diff changeset
154 - info(): return a mimetools.Message object for the headers
e7525a555a64 url: use new http support if requested by the user
Augie Fackler <durin42@gmail.com>
parents:
diff changeset
155 - geturl(): return the original request URL
e7525a555a64 url: use new http support if requested by the user
Augie Fackler <durin42@gmail.com>
parents:
diff changeset
156 - code: HTTP status code
e7525a555a64 url: use new http support if requested by the user
Augie Fackler <durin42@gmail.com>
parents:
diff changeset
157 """
e7525a555a64 url: use new http support if requested by the user
Augie Fackler <durin42@gmail.com>
parents:
diff changeset
158 # If using a proxy, the host returned by get_host() is
e7525a555a64 url: use new http support if requested by the user
Augie Fackler <durin42@gmail.com>
parents:
diff changeset
159 # actually the proxy. On Python 2.6.1, the real destination
e7525a555a64 url: use new http support if requested by the user
Augie Fackler <durin42@gmail.com>
parents:
diff changeset
160 # hostname is encoded in the URI in the urllib2 request
e7525a555a64 url: use new http support if requested by the user
Augie Fackler <durin42@gmail.com>
parents:
diff changeset
161 # object. On Python 2.6.5, it's stored in the _tunnel_host
e7525a555a64 url: use new http support if requested by the user
Augie Fackler <durin42@gmail.com>
parents:
diff changeset
162 # attribute which has no accessor.
e7525a555a64 url: use new http support if requested by the user
Augie Fackler <durin42@gmail.com>
parents:
diff changeset
163 tunhost = getattr(req, '_tunnel_host', None)
e7525a555a64 url: use new http support if requested by the user
Augie Fackler <durin42@gmail.com>
parents:
diff changeset
164 host = req.get_host()
e7525a555a64 url: use new http support if requested by the user
Augie Fackler <durin42@gmail.com>
parents:
diff changeset
165 if tunhost:
e7525a555a64 url: use new http support if requested by the user
Augie Fackler <durin42@gmail.com>
parents:
diff changeset
166 proxyhost = host
e7525a555a64 url: use new http support if requested by the user
Augie Fackler <durin42@gmail.com>
parents:
diff changeset
167 host = tunhost
e7525a555a64 url: use new http support if requested by the user
Augie Fackler <durin42@gmail.com>
parents:
diff changeset
168 elif req.has_proxy():
e7525a555a64 url: use new http support if requested by the user
Augie Fackler <durin42@gmail.com>
parents:
diff changeset
169 proxyhost = req.get_host()
e7525a555a64 url: use new http support if requested by the user
Augie Fackler <durin42@gmail.com>
parents:
diff changeset
170 host = req.get_selector().split('://', 1)[1].split('/', 1)[0]
e7525a555a64 url: use new http support if requested by the user
Augie Fackler <durin42@gmail.com>
parents:
diff changeset
171 else:
e7525a555a64 url: use new http support if requested by the user
Augie Fackler <durin42@gmail.com>
parents:
diff changeset
172 proxyhost = None
e7525a555a64 url: use new http support if requested by the user
Augie Fackler <durin42@gmail.com>
parents:
diff changeset
173
e7525a555a64 url: use new http support if requested by the user
Augie Fackler <durin42@gmail.com>
parents:
diff changeset
174 if proxyhost:
e7525a555a64 url: use new http support if requested by the user
Augie Fackler <durin42@gmail.com>
parents:
diff changeset
175 if ':' in proxyhost:
e7525a555a64 url: use new http support if requested by the user
Augie Fackler <durin42@gmail.com>
parents:
diff changeset
176 # Note: this means we'll explode if we try and use an
e7525a555a64 url: use new http support if requested by the user
Augie Fackler <durin42@gmail.com>
parents:
diff changeset
177 # IPv6 http proxy. This isn't a regression, so we
e7525a555a64 url: use new http support if requested by the user
Augie Fackler <durin42@gmail.com>
parents:
diff changeset
178 # won't worry about it for now.
e7525a555a64 url: use new http support if requested by the user
Augie Fackler <durin42@gmail.com>
parents:
diff changeset
179 proxyhost, proxyport = proxyhost.rsplit(':', 1)
e7525a555a64 url: use new http support if requested by the user
Augie Fackler <durin42@gmail.com>
parents:
diff changeset
180 else:
e7525a555a64 url: use new http support if requested by the user
Augie Fackler <durin42@gmail.com>
parents:
diff changeset
181 proxyport = 3128 # squid default
e7525a555a64 url: use new http support if requested by the user
Augie Fackler <durin42@gmail.com>
parents:
diff changeset
182 proxy = (proxyhost, proxyport)
e7525a555a64 url: use new http support if requested by the user
Augie Fackler <durin42@gmail.com>
parents:
diff changeset
183 else:
e7525a555a64 url: use new http support if requested by the user
Augie Fackler <durin42@gmail.com>
parents:
diff changeset
184 proxy = None
e7525a555a64 url: use new http support if requested by the user
Augie Fackler <durin42@gmail.com>
parents:
diff changeset
185
e7525a555a64 url: use new http support if requested by the user
Augie Fackler <durin42@gmail.com>
parents:
diff changeset
186 if not host:
e7525a555a64 url: use new http support if requested by the user
Augie Fackler <durin42@gmail.com>
parents:
diff changeset
187 raise urllib2.URLError('no host given')
e7525a555a64 url: use new http support if requested by the user
Augie Fackler <durin42@gmail.com>
parents:
diff changeset
188
14346
bf85c2639700 httpconnection: correctly handle redirects from http to https
Augie Fackler <durin42@gmail.com>
parents: 14294
diff changeset
189 connkey = use_ssl, host, proxy
bf85c2639700 httpconnection: correctly handle redirects from http to https
Augie Fackler <durin42@gmail.com>
parents: 14294
diff changeset
190 allconns = self._connections.get(connkey, [])
14244
e7525a555a64 url: use new http support if requested by the user
Augie Fackler <durin42@gmail.com>
parents:
diff changeset
191 conns = [c for c in allconns if not c.busy()]
e7525a555a64 url: use new http support if requested by the user
Augie Fackler <durin42@gmail.com>
parents:
diff changeset
192 if conns:
e7525a555a64 url: use new http support if requested by the user
Augie Fackler <durin42@gmail.com>
parents:
diff changeset
193 h = conns[0]
e7525a555a64 url: use new http support if requested by the user
Augie Fackler <durin42@gmail.com>
parents:
diff changeset
194 else:
e7525a555a64 url: use new http support if requested by the user
Augie Fackler <durin42@gmail.com>
parents:
diff changeset
195 if allconns:
e7525a555a64 url: use new http support if requested by the user
Augie Fackler <durin42@gmail.com>
parents:
diff changeset
196 self.ui.debug('all connections for %s busy, making a new '
e7525a555a64 url: use new http support if requested by the user
Augie Fackler <durin42@gmail.com>
parents:
diff changeset
197 'one\n' % host)
e7525a555a64 url: use new http support if requested by the user
Augie Fackler <durin42@gmail.com>
parents:
diff changeset
198 timeout = None
e7525a555a64 url: use new http support if requested by the user
Augie Fackler <durin42@gmail.com>
parents:
diff changeset
199 if req.timeout is not socket._GLOBAL_DEFAULT_TIMEOUT:
e7525a555a64 url: use new http support if requested by the user
Augie Fackler <durin42@gmail.com>
parents:
diff changeset
200 timeout = req.timeout
e7525a555a64 url: use new http support if requested by the user
Augie Fackler <durin42@gmail.com>
parents:
diff changeset
201 h = http_class(host, timeout=timeout, proxy_hostport=proxy)
14346
bf85c2639700 httpconnection: correctly handle redirects from http to https
Augie Fackler <durin42@gmail.com>
parents: 14294
diff changeset
202 self._connections.setdefault(connkey, []).append(h)
14244
e7525a555a64 url: use new http support if requested by the user
Augie Fackler <durin42@gmail.com>
parents:
diff changeset
203
e7525a555a64 url: use new http support if requested by the user
Augie Fackler <durin42@gmail.com>
parents:
diff changeset
204 headers = dict(req.headers)
e7525a555a64 url: use new http support if requested by the user
Augie Fackler <durin42@gmail.com>
parents:
diff changeset
205 headers.update(req.unredirected_hdrs)
e7525a555a64 url: use new http support if requested by the user
Augie Fackler <durin42@gmail.com>
parents:
diff changeset
206 headers = dict(
e7525a555a64 url: use new http support if requested by the user
Augie Fackler <durin42@gmail.com>
parents:
diff changeset
207 (name.title(), val) for name, val in headers.items())
e7525a555a64 url: use new http support if requested by the user
Augie Fackler <durin42@gmail.com>
parents:
diff changeset
208 try:
e7525a555a64 url: use new http support if requested by the user
Augie Fackler <durin42@gmail.com>
parents:
diff changeset
209 path = req.get_selector()
e7525a555a64 url: use new http support if requested by the user
Augie Fackler <durin42@gmail.com>
parents:
diff changeset
210 if '://' in path:
e7525a555a64 url: use new http support if requested by the user
Augie Fackler <durin42@gmail.com>
parents:
diff changeset
211 path = path.split('://', 1)[1].split('/', 1)[1]
e7525a555a64 url: use new http support if requested by the user
Augie Fackler <durin42@gmail.com>
parents:
diff changeset
212 if path[0] != '/':
e7525a555a64 url: use new http support if requested by the user
Augie Fackler <durin42@gmail.com>
parents:
diff changeset
213 path = '/' + path
e7525a555a64 url: use new http support if requested by the user
Augie Fackler <durin42@gmail.com>
parents:
diff changeset
214 h.request(req.get_method(), path, req.data, headers)
e7525a555a64 url: use new http support if requested by the user
Augie Fackler <durin42@gmail.com>
parents:
diff changeset
215 r = h.getresponse()
e7525a555a64 url: use new http support if requested by the user
Augie Fackler <durin42@gmail.com>
parents:
diff changeset
216 except socket.error, err: # XXX what error?
e7525a555a64 url: use new http support if requested by the user
Augie Fackler <durin42@gmail.com>
parents:
diff changeset
217 raise urllib2.URLError(err)
e7525a555a64 url: use new http support if requested by the user
Augie Fackler <durin42@gmail.com>
parents:
diff changeset
218
e7525a555a64 url: use new http support if requested by the user
Augie Fackler <durin42@gmail.com>
parents:
diff changeset
219 # Pick apart the HTTPResponse object to get the addinfourl
e7525a555a64 url: use new http support if requested by the user
Augie Fackler <durin42@gmail.com>
parents:
diff changeset
220 # object initialized properly.
e7525a555a64 url: use new http support if requested by the user
Augie Fackler <durin42@gmail.com>
parents:
diff changeset
221 r.recv = r.read
e7525a555a64 url: use new http support if requested by the user
Augie Fackler <durin42@gmail.com>
parents:
diff changeset
222
e7525a555a64 url: use new http support if requested by the user
Augie Fackler <durin42@gmail.com>
parents:
diff changeset
223 resp = urllib.addinfourl(r, r.headers, req.get_full_url())
e7525a555a64 url: use new http support if requested by the user
Augie Fackler <durin42@gmail.com>
parents:
diff changeset
224 resp.code = r.status
e7525a555a64 url: use new http support if requested by the user
Augie Fackler <durin42@gmail.com>
parents:
diff changeset
225 resp.msg = r.reason
e7525a555a64 url: use new http support if requested by the user
Augie Fackler <durin42@gmail.com>
parents:
diff changeset
226 return resp
e7525a555a64 url: use new http support if requested by the user
Augie Fackler <durin42@gmail.com>
parents:
diff changeset
227
e7525a555a64 url: use new http support if requested by the user
Augie Fackler <durin42@gmail.com>
parents:
diff changeset
228 # httplib always uses the given host/port as the socket connect
e7525a555a64 url: use new http support if requested by the user
Augie Fackler <durin42@gmail.com>
parents:
diff changeset
229 # target, and then allows full URIs in the request path, which it
e7525a555a64 url: use new http support if requested by the user
Augie Fackler <durin42@gmail.com>
parents:
diff changeset
230 # then observes and treats as a signal to do proxying instead.
e7525a555a64 url: use new http support if requested by the user
Augie Fackler <durin42@gmail.com>
parents:
diff changeset
231 def http_open(self, req):
e7525a555a64 url: use new http support if requested by the user
Augie Fackler <durin42@gmail.com>
parents:
diff changeset
232 if req.get_full_url().startswith('https'):
e7525a555a64 url: use new http support if requested by the user
Augie Fackler <durin42@gmail.com>
parents:
diff changeset
233 return self.https_open(req)
14346
bf85c2639700 httpconnection: correctly handle redirects from http to https
Augie Fackler <durin42@gmail.com>
parents: 14294
diff changeset
234 return self.do_open(HTTPConnection, req, False)
14244
e7525a555a64 url: use new http support if requested by the user
Augie Fackler <durin42@gmail.com>
parents:
diff changeset
235
e7525a555a64 url: use new http support if requested by the user
Augie Fackler <durin42@gmail.com>
parents:
diff changeset
236 def https_open(self, req):
15025
0593e8f81c71 http: pass user to readauthforuri() (fix 4a43e23b8c55)
Patrick Mezard <pmezard@gmail.com>
parents: 15005
diff changeset
237 # req.get_full_url() does not contain credentials and we may
0593e8f81c71 http: pass user to readauthforuri() (fix 4a43e23b8c55)
Patrick Mezard <pmezard@gmail.com>
parents: 15005
diff changeset
238 # need them to match the certificates.
0593e8f81c71 http: pass user to readauthforuri() (fix 4a43e23b8c55)
Patrick Mezard <pmezard@gmail.com>
parents: 15005
diff changeset
239 url = req.get_full_url()
0593e8f81c71 http: pass user to readauthforuri() (fix 4a43e23b8c55)
Patrick Mezard <pmezard@gmail.com>
parents: 15005
diff changeset
240 user, password = self.pwmgr.find_stored_password(url)
0593e8f81c71 http: pass user to readauthforuri() (fix 4a43e23b8c55)
Patrick Mezard <pmezard@gmail.com>
parents: 15005
diff changeset
241 res = readauthforuri(self.ui, url, user)
14244
e7525a555a64 url: use new http support if requested by the user
Augie Fackler <durin42@gmail.com>
parents:
diff changeset
242 if res:
e7525a555a64 url: use new http support if requested by the user
Augie Fackler <durin42@gmail.com>
parents:
diff changeset
243 group, auth = res
e7525a555a64 url: use new http support if requested by the user
Augie Fackler <durin42@gmail.com>
parents:
diff changeset
244 self.auth = auth
e7525a555a64 url: use new http support if requested by the user
Augie Fackler <durin42@gmail.com>
parents:
diff changeset
245 self.ui.debug("using auth.%s.* for authentication\n" % group)
e7525a555a64 url: use new http support if requested by the user
Augie Fackler <durin42@gmail.com>
parents:
diff changeset
246 else:
e7525a555a64 url: use new http support if requested by the user
Augie Fackler <durin42@gmail.com>
parents:
diff changeset
247 self.auth = None
14346
bf85c2639700 httpconnection: correctly handle redirects from http to https
Augie Fackler <durin42@gmail.com>
parents: 14294
diff changeset
248 return self.do_open(self._makesslconnection, req, True)
14244
e7525a555a64 url: use new http support if requested by the user
Augie Fackler <durin42@gmail.com>
parents:
diff changeset
249
e7525a555a64 url: use new http support if requested by the user
Augie Fackler <durin42@gmail.com>
parents:
diff changeset
250 def _makesslconnection(self, host, port=443, *args, **kwargs):
e7525a555a64 url: use new http support if requested by the user
Augie Fackler <durin42@gmail.com>
parents:
diff changeset
251 keyfile = None
e7525a555a64 url: use new http support if requested by the user
Augie Fackler <durin42@gmail.com>
parents:
diff changeset
252 certfile = None
e7525a555a64 url: use new http support if requested by the user
Augie Fackler <durin42@gmail.com>
parents:
diff changeset
253
e7525a555a64 url: use new http support if requested by the user
Augie Fackler <durin42@gmail.com>
parents:
diff changeset
254 if args: # key_file
e7525a555a64 url: use new http support if requested by the user
Augie Fackler <durin42@gmail.com>
parents:
diff changeset
255 keyfile = args.pop(0)
e7525a555a64 url: use new http support if requested by the user
Augie Fackler <durin42@gmail.com>
parents:
diff changeset
256 if args: # cert_file
e7525a555a64 url: use new http support if requested by the user
Augie Fackler <durin42@gmail.com>
parents:
diff changeset
257 certfile = args.pop(0)
e7525a555a64 url: use new http support if requested by the user
Augie Fackler <durin42@gmail.com>
parents:
diff changeset
258
e7525a555a64 url: use new http support if requested by the user
Augie Fackler <durin42@gmail.com>
parents:
diff changeset
259 # if the user has specified different key/cert files in
e7525a555a64 url: use new http support if requested by the user
Augie Fackler <durin42@gmail.com>
parents:
diff changeset
260 # hgrc, we prefer these
e7525a555a64 url: use new http support if requested by the user
Augie Fackler <durin42@gmail.com>
parents:
diff changeset
261 if self.auth and 'key' in self.auth and 'cert' in self.auth:
e7525a555a64 url: use new http support if requested by the user
Augie Fackler <durin42@gmail.com>
parents:
diff changeset
262 keyfile = self.auth['key']
e7525a555a64 url: use new http support if requested by the user
Augie Fackler <durin42@gmail.com>
parents:
diff changeset
263 certfile = self.auth['cert']
e7525a555a64 url: use new http support if requested by the user
Augie Fackler <durin42@gmail.com>
parents:
diff changeset
264
e7525a555a64 url: use new http support if requested by the user
Augie Fackler <durin42@gmail.com>
parents:
diff changeset
265 # let host port take precedence
e7525a555a64 url: use new http support if requested by the user
Augie Fackler <durin42@gmail.com>
parents:
diff changeset
266 if ':' in host and '[' not in host or ']:' in host:
e7525a555a64 url: use new http support if requested by the user
Augie Fackler <durin42@gmail.com>
parents:
diff changeset
267 host, port = host.rsplit(':', 1)
e7525a555a64 url: use new http support if requested by the user
Augie Fackler <durin42@gmail.com>
parents:
diff changeset
268 port = int(port)
e7525a555a64 url: use new http support if requested by the user
Augie Fackler <durin42@gmail.com>
parents:
diff changeset
269 if '[' in host:
e7525a555a64 url: use new http support if requested by the user
Augie Fackler <durin42@gmail.com>
parents:
diff changeset
270 host = host[1:-1]
e7525a555a64 url: use new http support if requested by the user
Augie Fackler <durin42@gmail.com>
parents:
diff changeset
271
e7525a555a64 url: use new http support if requested by the user
Augie Fackler <durin42@gmail.com>
parents:
diff changeset
272 if keyfile:
e7525a555a64 url: use new http support if requested by the user
Augie Fackler <durin42@gmail.com>
parents:
diff changeset
273 kwargs['keyfile'] = keyfile
e7525a555a64 url: use new http support if requested by the user
Augie Fackler <durin42@gmail.com>
parents:
diff changeset
274 if certfile:
e7525a555a64 url: use new http support if requested by the user
Augie Fackler <durin42@gmail.com>
parents:
diff changeset
275 kwargs['certfile'] = certfile
e7525a555a64 url: use new http support if requested by the user
Augie Fackler <durin42@gmail.com>
parents:
diff changeset
276
e7525a555a64 url: use new http support if requested by the user
Augie Fackler <durin42@gmail.com>
parents:
diff changeset
277 kwargs.update(sslutil.sslkwargs(self.ui, host))
e7525a555a64 url: use new http support if requested by the user
Augie Fackler <durin42@gmail.com>
parents:
diff changeset
278
e7525a555a64 url: use new http support if requested by the user
Augie Fackler <durin42@gmail.com>
parents:
diff changeset
279 con = HTTPConnection(host, port, use_ssl=True,
e7525a555a64 url: use new http support if requested by the user
Augie Fackler <durin42@gmail.com>
parents:
diff changeset
280 ssl_validator=sslutil.validator(self.ui, host),
e7525a555a64 url: use new http support if requested by the user
Augie Fackler <durin42@gmail.com>
parents:
diff changeset
281 **kwargs)
e7525a555a64 url: use new http support if requested by the user
Augie Fackler <durin42@gmail.com>
parents:
diff changeset
282 return con