mercurial-scm/hg-stable: mercurial/url.py annotate

annotate mercurial/url.py @ 8590:59acb9c7d90f

url: use CONNECT for HTTPS connections through HTTP proxy (issue967) urllib2 and httplib does not support using CONNECT proxy requests, but only regular requests over the proxy. This does not work with HTTPS requests as they typically require that the client issues a CONNECT to the proxy to give a direct connection to the remote HTTPS server. This is solved by duplicating some of the httplib functionality and tying it together with the keepalive library such that a HTTPS connection that need to be proxied can be proxied by letting a connection be established to the proxy server and then subsequently performing the normal request to the specified server through the proxy server. As it stands, the code also purports to support HTTPS proxies, i.e. proxies that you connect to using SSL. These are extremely rare and nothing is done to ensure that CONNECT requests can be made to these as that would require multiple SSL handshakes. This use case is also not supported by most other contemporary web tools like curl and Firefox3.

author	Henrik Stuart <hg@hstuart.dk>
date	Fri, 22 May 2009 08:56:43 +0200
parents	873429914ec5
children	7951f385fcb7

rev	line source
7270 2db33c1a5654 factor out the url handling from httprepo Benoit Boissinot <benoit.boissinot@ens-lyon.org> parents: diff changeset	1 # url.py - HTTP handling for mercurial
2db33c1a5654 factor out the url handling from httprepo Benoit Boissinot <benoit.boissinot@ens-lyon.org> parents: diff changeset	2 #
2db33c1a5654 factor out the url handling from httprepo Benoit Boissinot <benoit.boissinot@ens-lyon.org> parents: diff changeset	3 # Copyright 2005, 2006, 2007, 2008 Matt Mackall <mpm@selenic.com>
2db33c1a5654 factor out the url handling from httprepo Benoit Boissinot <benoit.boissinot@ens-lyon.org> parents: diff changeset	4 # Copyright 2006, 2007 Alexis S. L. Carvalho <alexis@cecm.usp.br>
2db33c1a5654 factor out the url handling from httprepo Benoit Boissinot <benoit.boissinot@ens-lyon.org> parents: diff changeset	5 # Copyright 2006 Vadim Gelfer <vadim.gelfer@gmail.com>
2db33c1a5654 factor out the url handling from httprepo Benoit Boissinot <benoit.boissinot@ens-lyon.org> parents: diff changeset	6 #
8225 46293a0c7e9f updated license to be explicit about GPL version 2 Martin Geisler <mg@lazybytes.net> parents: 8208 diff changeset	7 # This software may be used and distributed according to the terms of the
46293a0c7e9f updated license to be explicit about GPL version 2 Martin Geisler <mg@lazybytes.net> parents: 8208 diff changeset	8 # GNU General Public License version 2, incorporated herein by reference.
7270 2db33c1a5654 factor out the url handling from httprepo Benoit Boissinot <benoit.boissinot@ens-lyon.org> parents: diff changeset	9
8590 59acb9c7d90f url: use CONNECT for HTTPS connections through HTTP proxy (issue967) Henrik Stuart <hg@hstuart.dk> parents: 8344 diff changeset	10 import urllib, urllib2, urlparse, httplib, os, re, socket, cStringIO
7270 2db33c1a5654 factor out the url handling from httprepo Benoit Boissinot <benoit.boissinot@ens-lyon.org> parents: diff changeset	11 from i18n import _
2db33c1a5654 factor out the url handling from httprepo Benoit Boissinot <benoit.boissinot@ens-lyon.org> parents: diff changeset	12 import keepalive, util
2db33c1a5654 factor out the url handling from httprepo Benoit Boissinot <benoit.boissinot@ens-lyon.org> parents: diff changeset	13
2db33c1a5654 factor out the url handling from httprepo Benoit Boissinot <benoit.boissinot@ens-lyon.org> parents: diff changeset	14 def hidepassword(url):
2db33c1a5654 factor out the url handling from httprepo Benoit Boissinot <benoit.boissinot@ens-lyon.org> parents: diff changeset	15 '''hide user credential in a url string'''
2db33c1a5654 factor out the url handling from httprepo Benoit Boissinot <benoit.boissinot@ens-lyon.org> parents: diff changeset	16 scheme, netloc, path, params, query, fragment = urlparse.urlparse(url)
2db33c1a5654 factor out the url handling from httprepo Benoit Boissinot <benoit.boissinot@ens-lyon.org> parents: diff changeset	17 netloc = re.sub('([^:]):([^@])@(.)', r'\1:**@\3', netloc)
2db33c1a5654 factor out the url handling from httprepo Benoit Boissinot <benoit.boissinot@ens-lyon.org> parents: diff changeset	18 return urlparse.urlunparse((scheme, netloc, path, params, query, fragment))
2db33c1a5654 factor out the url handling from httprepo Benoit Boissinot <benoit.boissinot@ens-lyon.org> parents: diff changeset	19
2db33c1a5654 factor out the url handling from httprepo Benoit Boissinot <benoit.boissinot@ens-lyon.org> parents: diff changeset	20 def removeauth(url):
2db33c1a5654 factor out the url handling from httprepo Benoit Boissinot <benoit.boissinot@ens-lyon.org> parents: diff changeset	21 '''remove all authentication information from a url string'''
2db33c1a5654 factor out the url handling from httprepo Benoit Boissinot <benoit.boissinot@ens-lyon.org> parents: diff changeset	22 scheme, netloc, path, params, query, fragment = urlparse.urlparse(url)
2db33c1a5654 factor out the url handling from httprepo Benoit Boissinot <benoit.boissinot@ens-lyon.org> parents: diff changeset	23 netloc = netloc[netloc.find('@')+1:]
2db33c1a5654 factor out the url handling from httprepo Benoit Boissinot <benoit.boissinot@ens-lyon.org> parents: diff changeset	24 return urlparse.urlunparse((scheme, netloc, path, params, query, fragment))
2db33c1a5654 factor out the url handling from httprepo Benoit Boissinot <benoit.boissinot@ens-lyon.org> parents: diff changeset	25
2db33c1a5654 factor out the url handling from httprepo Benoit Boissinot <benoit.boissinot@ens-lyon.org> parents: diff changeset	26 def netlocsplit(netloc):
2db33c1a5654 factor out the url handling from httprepo Benoit Boissinot <benoit.boissinot@ens-lyon.org> parents: diff changeset	27 '''split [user[:passwd]@]host[:port] into 4-tuple.'''
2db33c1a5654 factor out the url handling from httprepo Benoit Boissinot <benoit.boissinot@ens-lyon.org> parents: diff changeset	28
2db33c1a5654 factor out the url handling from httprepo Benoit Boissinot <benoit.boissinot@ens-lyon.org> parents: diff changeset	29 a = netloc.find('@')
2db33c1a5654 factor out the url handling from httprepo Benoit Boissinot <benoit.boissinot@ens-lyon.org> parents: diff changeset	30 if a == -1:
2db33c1a5654 factor out the url handling from httprepo Benoit Boissinot <benoit.boissinot@ens-lyon.org> parents: diff changeset	31 user, passwd = None, None
2db33c1a5654 factor out the url handling from httprepo Benoit Boissinot <benoit.boissinot@ens-lyon.org> parents: diff changeset	32 else:
2db33c1a5654 factor out the url handling from httprepo Benoit Boissinot <benoit.boissinot@ens-lyon.org> parents: diff changeset	33 userpass, netloc = netloc[:a], netloc[a+1:]
2db33c1a5654 factor out the url handling from httprepo Benoit Boissinot <benoit.boissinot@ens-lyon.org> parents: diff changeset	34 c = userpass.find(':')
2db33c1a5654 factor out the url handling from httprepo Benoit Boissinot <benoit.boissinot@ens-lyon.org> parents: diff changeset	35 if c == -1:
2db33c1a5654 factor out the url handling from httprepo Benoit Boissinot <benoit.boissinot@ens-lyon.org> parents: diff changeset	36 user, passwd = urllib.unquote(userpass), None
2db33c1a5654 factor out the url handling from httprepo Benoit Boissinot <benoit.boissinot@ens-lyon.org> parents: diff changeset	37 else:
2db33c1a5654 factor out the url handling from httprepo Benoit Boissinot <benoit.boissinot@ens-lyon.org> parents: diff changeset	38 user = urllib.unquote(userpass[:c])
2db33c1a5654 factor out the url handling from httprepo Benoit Boissinot <benoit.boissinot@ens-lyon.org> parents: diff changeset	39 passwd = urllib.unquote(userpass[c+1:])
2db33c1a5654 factor out the url handling from httprepo Benoit Boissinot <benoit.boissinot@ens-lyon.org> parents: diff changeset	40 c = netloc.find(':')
2db33c1a5654 factor out the url handling from httprepo Benoit Boissinot <benoit.boissinot@ens-lyon.org> parents: diff changeset	41 if c == -1:
2db33c1a5654 factor out the url handling from httprepo Benoit Boissinot <benoit.boissinot@ens-lyon.org> parents: diff changeset	42 host, port = netloc, None
2db33c1a5654 factor out the url handling from httprepo Benoit Boissinot <benoit.boissinot@ens-lyon.org> parents: diff changeset	43 else:
2db33c1a5654 factor out the url handling from httprepo Benoit Boissinot <benoit.boissinot@ens-lyon.org> parents: diff changeset	44 host, port = netloc[:c], netloc[c+1:]
2db33c1a5654 factor out the url handling from httprepo Benoit Boissinot <benoit.boissinot@ens-lyon.org> parents: diff changeset	45 return host, port, user, passwd
2db33c1a5654 factor out the url handling from httprepo Benoit Boissinot <benoit.boissinot@ens-lyon.org> parents: diff changeset	46
2db33c1a5654 factor out the url handling from httprepo Benoit Boissinot <benoit.boissinot@ens-lyon.org> parents: diff changeset	47 def netlocunsplit(host, port, user=None, passwd=None):
2db33c1a5654 factor out the url handling from httprepo Benoit Boissinot <benoit.boissinot@ens-lyon.org> parents: diff changeset	48 '''turn host, port, user, passwd into [user[:passwd]@]host[:port].'''
2db33c1a5654 factor out the url handling from httprepo Benoit Boissinot <benoit.boissinot@ens-lyon.org> parents: diff changeset	49 if port:
2db33c1a5654 factor out the url handling from httprepo Benoit Boissinot <benoit.boissinot@ens-lyon.org> parents: diff changeset	50 hostport = host + ':' + port
2db33c1a5654 factor out the url handling from httprepo Benoit Boissinot <benoit.boissinot@ens-lyon.org> parents: diff changeset	51 else:
2db33c1a5654 factor out the url handling from httprepo Benoit Boissinot <benoit.boissinot@ens-lyon.org> parents: diff changeset	52 hostport = host
2db33c1a5654 factor out the url handling from httprepo Benoit Boissinot <benoit.boissinot@ens-lyon.org> parents: diff changeset	53 if user:
2db33c1a5654 factor out the url handling from httprepo Benoit Boissinot <benoit.boissinot@ens-lyon.org> parents: diff changeset	54 if passwd:
2db33c1a5654 factor out the url handling from httprepo Benoit Boissinot <benoit.boissinot@ens-lyon.org> parents: diff changeset	55 userpass = urllib.quote(user) + ':' + urllib.quote(passwd)
2db33c1a5654 factor out the url handling from httprepo Benoit Boissinot <benoit.boissinot@ens-lyon.org> parents: diff changeset	56 else:
2db33c1a5654 factor out the url handling from httprepo Benoit Boissinot <benoit.boissinot@ens-lyon.org> parents: diff changeset	57 userpass = urllib.quote(user)
2db33c1a5654 factor out the url handling from httprepo Benoit Boissinot <benoit.boissinot@ens-lyon.org> parents: diff changeset	58 return userpass + '@' + hostport
2db33c1a5654 factor out the url handling from httprepo Benoit Boissinot <benoit.boissinot@ens-lyon.org> parents: diff changeset	59 return hostport
2db33c1a5654 factor out the url handling from httprepo Benoit Boissinot <benoit.boissinot@ens-lyon.org> parents: diff changeset	60
2db33c1a5654 factor out the url handling from httprepo Benoit Boissinot <benoit.boissinot@ens-lyon.org> parents: diff changeset	61 _safe = ('abcdefghijklmnopqrstuvwxyz'
2db33c1a5654 factor out the url handling from httprepo Benoit Boissinot <benoit.boissinot@ens-lyon.org> parents: diff changeset	62 'ABCDEFGHIJKLMNOPQRSTUVWXYZ'
2db33c1a5654 factor out the url handling from httprepo Benoit Boissinot <benoit.boissinot@ens-lyon.org> parents: diff changeset	63 '0123456789' '_.-/')
2db33c1a5654 factor out the url handling from httprepo Benoit Boissinot <benoit.boissinot@ens-lyon.org> parents: diff changeset	64 _safeset = None
2db33c1a5654 factor out the url handling from httprepo Benoit Boissinot <benoit.boissinot@ens-lyon.org> parents: diff changeset	65 _hex = None
2db33c1a5654 factor out the url handling from httprepo Benoit Boissinot <benoit.boissinot@ens-lyon.org> parents: diff changeset	66 def quotepath(path):
2db33c1a5654 factor out the url handling from httprepo Benoit Boissinot <benoit.boissinot@ens-lyon.org> parents: diff changeset	67 '''quote the path part of a URL
2db33c1a5654 factor out the url handling from httprepo Benoit Boissinot <benoit.boissinot@ens-lyon.org> parents: diff changeset	68
2db33c1a5654 factor out the url handling from httprepo Benoit Boissinot <benoit.boissinot@ens-lyon.org> parents: diff changeset	69 This is similar to urllib.quote, but it also tries to avoid
2db33c1a5654 factor out the url handling from httprepo Benoit Boissinot <benoit.boissinot@ens-lyon.org> parents: diff changeset	70 quoting things twice (inspired by wget):
2db33c1a5654 factor out the url handling from httprepo Benoit Boissinot <benoit.boissinot@ens-lyon.org> parents: diff changeset	71
2db33c1a5654 factor out the url handling from httprepo Benoit Boissinot <benoit.boissinot@ens-lyon.org> parents: diff changeset	72 >>> quotepath('abc def')
2db33c1a5654 factor out the url handling from httprepo Benoit Boissinot <benoit.boissinot@ens-lyon.org> parents: diff changeset	73 'abc%20def'
2db33c1a5654 factor out the url handling from httprepo Benoit Boissinot <benoit.boissinot@ens-lyon.org> parents: diff changeset	74 >>> quotepath('abc%20def')
2db33c1a5654 factor out the url handling from httprepo Benoit Boissinot <benoit.boissinot@ens-lyon.org> parents: diff changeset	75 'abc%20def'
2db33c1a5654 factor out the url handling from httprepo Benoit Boissinot <benoit.boissinot@ens-lyon.org> parents: diff changeset	76 >>> quotepath('abc%20 def')
2db33c1a5654 factor out the url handling from httprepo Benoit Boissinot <benoit.boissinot@ens-lyon.org> parents: diff changeset	77 'abc%20%20def'
2db33c1a5654 factor out the url handling from httprepo Benoit Boissinot <benoit.boissinot@ens-lyon.org> parents: diff changeset	78 >>> quotepath('abc def%20')
2db33c1a5654 factor out the url handling from httprepo Benoit Boissinot <benoit.boissinot@ens-lyon.org> parents: diff changeset	79 'abc%20def%20'
2db33c1a5654 factor out the url handling from httprepo Benoit Boissinot <benoit.boissinot@ens-lyon.org> parents: diff changeset	80 >>> quotepath('abc def%2')
2db33c1a5654 factor out the url handling from httprepo Benoit Boissinot <benoit.boissinot@ens-lyon.org> parents: diff changeset	81 'abc%20def%252'
2db33c1a5654 factor out the url handling from httprepo Benoit Boissinot <benoit.boissinot@ens-lyon.org> parents: diff changeset	82 >>> quotepath('abc def%')
2db33c1a5654 factor out the url handling from httprepo Benoit Boissinot <benoit.boissinot@ens-lyon.org> parents: diff changeset	83 'abc%20def%25'
2db33c1a5654 factor out the url handling from httprepo Benoit Boissinot <benoit.boissinot@ens-lyon.org> parents: diff changeset	84 '''
2db33c1a5654 factor out the url handling from httprepo Benoit Boissinot <benoit.boissinot@ens-lyon.org> parents: diff changeset	85 global _safeset, _hex
2db33c1a5654 factor out the url handling from httprepo Benoit Boissinot <benoit.boissinot@ens-lyon.org> parents: diff changeset	86 if _safeset is None:
8150 bbc24c0753a0 util: use built-in set and frozenset Martin Geisler <mg@lazybytes.net> parents: 7285 diff changeset	87 _safeset = set(_safe)
bbc24c0753a0 util: use built-in set and frozenset Martin Geisler <mg@lazybytes.net> parents: 7285 diff changeset	88 _hex = set('abcdefABCDEF0123456789')
7270 2db33c1a5654 factor out the url handling from httprepo Benoit Boissinot <benoit.boissinot@ens-lyon.org> parents: diff changeset	89 l = list(path)
2db33c1a5654 factor out the url handling from httprepo Benoit Boissinot <benoit.boissinot@ens-lyon.org> parents: diff changeset	90 for i in xrange(len(l)):
2db33c1a5654 factor out the url handling from httprepo Benoit Boissinot <benoit.boissinot@ens-lyon.org> parents: diff changeset	91 c = l[i]
2db33c1a5654 factor out the url handling from httprepo Benoit Boissinot <benoit.boissinot@ens-lyon.org> parents: diff changeset	92 if c == '%' and i + 2 < len(l) and (l[i+1] in _hex and l[i+2] in _hex):
2db33c1a5654 factor out the url handling from httprepo Benoit Boissinot <benoit.boissinot@ens-lyon.org> parents: diff changeset	93 pass
2db33c1a5654 factor out the url handling from httprepo Benoit Boissinot <benoit.boissinot@ens-lyon.org> parents: diff changeset	94 elif c not in _safeset:
2db33c1a5654 factor out the url handling from httprepo Benoit Boissinot <benoit.boissinot@ens-lyon.org> parents: diff changeset	95 l[i] = '%%%02X' % ord(c)
2db33c1a5654 factor out the url handling from httprepo Benoit Boissinot <benoit.boissinot@ens-lyon.org> parents: diff changeset	96 return ''.join(l)
2db33c1a5654 factor out the url handling from httprepo Benoit Boissinot <benoit.boissinot@ens-lyon.org> parents: diff changeset	97
2db33c1a5654 factor out the url handling from httprepo Benoit Boissinot <benoit.boissinot@ens-lyon.org> parents: diff changeset	98 class passwordmgr(urllib2.HTTPPasswordMgrWithDefaultRealm):
2db33c1a5654 factor out the url handling from httprepo Benoit Boissinot <benoit.boissinot@ens-lyon.org> parents: diff changeset	99 def __init__(self, ui):
2db33c1a5654 factor out the url handling from httprepo Benoit Boissinot <benoit.boissinot@ens-lyon.org> parents: diff changeset	100 urllib2.HTTPPasswordMgrWithDefaultRealm.__init__(self)
2db33c1a5654 factor out the url handling from httprepo Benoit Boissinot <benoit.boissinot@ens-lyon.org> parents: diff changeset	101 self.ui = ui
2db33c1a5654 factor out the url handling from httprepo Benoit Boissinot <benoit.boissinot@ens-lyon.org> parents: diff changeset	102
2db33c1a5654 factor out the url handling from httprepo Benoit Boissinot <benoit.boissinot@ens-lyon.org> parents: diff changeset	103 def find_user_password(self, realm, authuri):
2db33c1a5654 factor out the url handling from httprepo Benoit Boissinot <benoit.boissinot@ens-lyon.org> parents: diff changeset	104 authinfo = urllib2.HTTPPasswordMgrWithDefaultRealm.find_user_password(
2db33c1a5654 factor out the url handling from httprepo Benoit Boissinot <benoit.boissinot@ens-lyon.org> parents: diff changeset	105 self, realm, authuri)
2db33c1a5654 factor out the url handling from httprepo Benoit Boissinot <benoit.boissinot@ens-lyon.org> parents: diff changeset	106 user, passwd = authinfo
2db33c1a5654 factor out the url handling from httprepo Benoit Boissinot <benoit.boissinot@ens-lyon.org> parents: diff changeset	107 if user and passwd:
8333 89c80c3dc584 allow http authentication information to be specified in the configuration Sune Foldager <cryo@cyanite.org> parents: 8225 diff changeset	108 self._writedebug(user, passwd)
7270 2db33c1a5654 factor out the url handling from httprepo Benoit Boissinot <benoit.boissinot@ens-lyon.org> parents: diff changeset	109 return (user, passwd)
2db33c1a5654 factor out the url handling from httprepo Benoit Boissinot <benoit.boissinot@ens-lyon.org> parents: diff changeset	110
8344 873429914ec5 url: fix bug in passwordmgr related to auth configuration Sune Foldager <cryo@cyanite.org> parents: 8333 diff changeset	111 if not user:
873429914ec5 url: fix bug in passwordmgr related to auth configuration Sune Foldager <cryo@cyanite.org> parents: 8333 diff changeset	112 user, passwd = self._readauthtoken(authuri)
8333 89c80c3dc584 allow http authentication information to be specified in the configuration Sune Foldager <cryo@cyanite.org> parents: 8225 diff changeset	113 if not user or not passwd:
89c80c3dc584 allow http authentication information to be specified in the configuration Sune Foldager <cryo@cyanite.org> parents: 8225 diff changeset	114 if not self.ui.interactive():
89c80c3dc584 allow http authentication information to be specified in the configuration Sune Foldager <cryo@cyanite.org> parents: 8225 diff changeset	115 raise util.Abort(_('http authorization required'))
7270 2db33c1a5654 factor out the url handling from httprepo Benoit Boissinot <benoit.boissinot@ens-lyon.org> parents: diff changeset	116
8333 89c80c3dc584 allow http authentication information to be specified in the configuration Sune Foldager <cryo@cyanite.org> parents: 8225 diff changeset	117 self.ui.write(_("http authorization required\n"))
89c80c3dc584 allow http authentication information to be specified in the configuration Sune Foldager <cryo@cyanite.org> parents: 8225 diff changeset	118 self.ui.status(_("realm: %s\n") % realm)
89c80c3dc584 allow http authentication information to be specified in the configuration Sune Foldager <cryo@cyanite.org> parents: 8225 diff changeset	119 if user:
89c80c3dc584 allow http authentication information to be specified in the configuration Sune Foldager <cryo@cyanite.org> parents: 8225 diff changeset	120 self.ui.status(_("user: %s\n") % user)
89c80c3dc584 allow http authentication information to be specified in the configuration Sune Foldager <cryo@cyanite.org> parents: 8225 diff changeset	121 else:
89c80c3dc584 allow http authentication information to be specified in the configuration Sune Foldager <cryo@cyanite.org> parents: 8225 diff changeset	122 user = self.ui.prompt(_("user:"), default=None)
7270 2db33c1a5654 factor out the url handling from httprepo Benoit Boissinot <benoit.boissinot@ens-lyon.org> parents: diff changeset	123
8333 89c80c3dc584 allow http authentication information to be specified in the configuration Sune Foldager <cryo@cyanite.org> parents: 8225 diff changeset	124 if not passwd:
89c80c3dc584 allow http authentication information to be specified in the configuration Sune Foldager <cryo@cyanite.org> parents: 8225 diff changeset	125 passwd = self.ui.getpass()
7270 2db33c1a5654 factor out the url handling from httprepo Benoit Boissinot <benoit.boissinot@ens-lyon.org> parents: diff changeset	126
2db33c1a5654 factor out the url handling from httprepo Benoit Boissinot <benoit.boissinot@ens-lyon.org> parents: diff changeset	127 self.add_password(realm, authuri, user, passwd)
8333 89c80c3dc584 allow http authentication information to be specified in the configuration Sune Foldager <cryo@cyanite.org> parents: 8225 diff changeset	128 self._writedebug(user, passwd)
7270 2db33c1a5654 factor out the url handling from httprepo Benoit Boissinot <benoit.boissinot@ens-lyon.org> parents: diff changeset	129 return (user, passwd)
2db33c1a5654 factor out the url handling from httprepo Benoit Boissinot <benoit.boissinot@ens-lyon.org> parents: diff changeset	130
8333 89c80c3dc584 allow http authentication information to be specified in the configuration Sune Foldager <cryo@cyanite.org> parents: 8225 diff changeset	131 def _writedebug(self, user, passwd):
89c80c3dc584 allow http authentication information to be specified in the configuration Sune Foldager <cryo@cyanite.org> parents: 8225 diff changeset	132 msg = _('http auth: user %s, password %s\n')
89c80c3dc584 allow http authentication information to be specified in the configuration Sune Foldager <cryo@cyanite.org> parents: 8225 diff changeset	133 self.ui.debug(msg % (user, passwd and '' len(passwd) or 'not set'))
89c80c3dc584 allow http authentication information to be specified in the configuration Sune Foldager <cryo@cyanite.org> parents: 8225 diff changeset	134
89c80c3dc584 allow http authentication information to be specified in the configuration Sune Foldager <cryo@cyanite.org> parents: 8225 diff changeset	135 def _readauthtoken(self, uri):
89c80c3dc584 allow http authentication information to be specified in the configuration Sune Foldager <cryo@cyanite.org> parents: 8225 diff changeset	136 # Read configuration
89c80c3dc584 allow http authentication information to be specified in the configuration Sune Foldager <cryo@cyanite.org> parents: 8225 diff changeset	137 config = dict()
89c80c3dc584 allow http authentication information to be specified in the configuration Sune Foldager <cryo@cyanite.org> parents: 8225 diff changeset	138 for key, val in self.ui.configitems('auth'):
89c80c3dc584 allow http authentication information to be specified in the configuration Sune Foldager <cryo@cyanite.org> parents: 8225 diff changeset	139 group, setting = key.split('.', 1)
89c80c3dc584 allow http authentication information to be specified in the configuration Sune Foldager <cryo@cyanite.org> parents: 8225 diff changeset	140 gdict = config.setdefault(group, dict())
89c80c3dc584 allow http authentication information to be specified in the configuration Sune Foldager <cryo@cyanite.org> parents: 8225 diff changeset	141 gdict[setting] = val
89c80c3dc584 allow http authentication information to be specified in the configuration Sune Foldager <cryo@cyanite.org> parents: 8225 diff changeset	142
89c80c3dc584 allow http authentication information to be specified in the configuration Sune Foldager <cryo@cyanite.org> parents: 8225 diff changeset	143 # Find the best match
89c80c3dc584 allow http authentication information to be specified in the configuration Sune Foldager <cryo@cyanite.org> parents: 8225 diff changeset	144 scheme, hostpath = uri.split('://', 1)
89c80c3dc584 allow http authentication information to be specified in the configuration Sune Foldager <cryo@cyanite.org> parents: 8225 diff changeset	145 bestlen = 0
89c80c3dc584 allow http authentication information to be specified in the configuration Sune Foldager <cryo@cyanite.org> parents: 8225 diff changeset	146 bestauth = None, None
89c80c3dc584 allow http authentication information to be specified in the configuration Sune Foldager <cryo@cyanite.org> parents: 8225 diff changeset	147 for auth in config.itervalues():
89c80c3dc584 allow http authentication information to be specified in the configuration Sune Foldager <cryo@cyanite.org> parents: 8225 diff changeset	148 prefix = auth.get('prefix')
89c80c3dc584 allow http authentication information to be specified in the configuration Sune Foldager <cryo@cyanite.org> parents: 8225 diff changeset	149 if not prefix: continue
89c80c3dc584 allow http authentication information to be specified in the configuration Sune Foldager <cryo@cyanite.org> parents: 8225 diff changeset	150 p = prefix.split('://', 1)
89c80c3dc584 allow http authentication information to be specified in the configuration Sune Foldager <cryo@cyanite.org> parents: 8225 diff changeset	151 if len(p) > 1:
89c80c3dc584 allow http authentication information to be specified in the configuration Sune Foldager <cryo@cyanite.org> parents: 8225 diff changeset	152 schemes, prefix = [p[0]], p[1]
89c80c3dc584 allow http authentication information to be specified in the configuration Sune Foldager <cryo@cyanite.org> parents: 8225 diff changeset	153 else:
89c80c3dc584 allow http authentication information to be specified in the configuration Sune Foldager <cryo@cyanite.org> parents: 8225 diff changeset	154 schemes = (auth.get('schemes') or 'https').split()
89c80c3dc584 allow http authentication information to be specified in the configuration Sune Foldager <cryo@cyanite.org> parents: 8225 diff changeset	155 if (prefix == '*' or hostpath.startswith(prefix)) and \
89c80c3dc584 allow http authentication information to be specified in the configuration Sune Foldager <cryo@cyanite.org> parents: 8225 diff changeset	156 len(prefix) > bestlen and scheme in schemes:
89c80c3dc584 allow http authentication information to be specified in the configuration Sune Foldager <cryo@cyanite.org> parents: 8225 diff changeset	157 bestlen = len(prefix)
89c80c3dc584 allow http authentication information to be specified in the configuration Sune Foldager <cryo@cyanite.org> parents: 8225 diff changeset	158 bestauth = auth.get('username'), auth.get('password')
89c80c3dc584 allow http authentication information to be specified in the configuration Sune Foldager <cryo@cyanite.org> parents: 8225 diff changeset	159 return bestauth
89c80c3dc584 allow http authentication information to be specified in the configuration Sune Foldager <cryo@cyanite.org> parents: 8225 diff changeset	160
7270 2db33c1a5654 factor out the url handling from httprepo Benoit Boissinot <benoit.boissinot@ens-lyon.org> parents: diff changeset	161 class proxyhandler(urllib2.ProxyHandler):
2db33c1a5654 factor out the url handling from httprepo Benoit Boissinot <benoit.boissinot@ens-lyon.org> parents: diff changeset	162 def __init__(self, ui):
2db33c1a5654 factor out the url handling from httprepo Benoit Boissinot <benoit.boissinot@ens-lyon.org> parents: diff changeset	163 proxyurl = ui.config("http_proxy", "host") or os.getenv('http_proxy')
2db33c1a5654 factor out the url handling from httprepo Benoit Boissinot <benoit.boissinot@ens-lyon.org> parents: diff changeset	164 # XXX proxyauthinfo = None
2db33c1a5654 factor out the url handling from httprepo Benoit Boissinot <benoit.boissinot@ens-lyon.org> parents: diff changeset	165
2db33c1a5654 factor out the url handling from httprepo Benoit Boissinot <benoit.boissinot@ens-lyon.org> parents: diff changeset	166 if proxyurl:
2db33c1a5654 factor out the url handling from httprepo Benoit Boissinot <benoit.boissinot@ens-lyon.org> parents: diff changeset	167 # proxy can be proper url or host[:port]
2db33c1a5654 factor out the url handling from httprepo Benoit Boissinot <benoit.boissinot@ens-lyon.org> parents: diff changeset	168 if not (proxyurl.startswith('http:') or
2db33c1a5654 factor out the url handling from httprepo Benoit Boissinot <benoit.boissinot@ens-lyon.org> parents: diff changeset	169 proxyurl.startswith('https:')):
2db33c1a5654 factor out the url handling from httprepo Benoit Boissinot <benoit.boissinot@ens-lyon.org> parents: diff changeset	170 proxyurl = 'http://' + proxyurl + '/'
2db33c1a5654 factor out the url handling from httprepo Benoit Boissinot <benoit.boissinot@ens-lyon.org> parents: diff changeset	171 snpqf = urlparse.urlsplit(proxyurl)
2db33c1a5654 factor out the url handling from httprepo Benoit Boissinot <benoit.boissinot@ens-lyon.org> parents: diff changeset	172 proxyscheme, proxynetloc, proxypath, proxyquery, proxyfrag = snpqf
2db33c1a5654 factor out the url handling from httprepo Benoit Boissinot <benoit.boissinot@ens-lyon.org> parents: diff changeset	173 hpup = netlocsplit(proxynetloc)
2db33c1a5654 factor out the url handling from httprepo Benoit Boissinot <benoit.boissinot@ens-lyon.org> parents: diff changeset	174
2db33c1a5654 factor out the url handling from httprepo Benoit Boissinot <benoit.boissinot@ens-lyon.org> parents: diff changeset	175 proxyhost, proxyport, proxyuser, proxypasswd = hpup
2db33c1a5654 factor out the url handling from httprepo Benoit Boissinot <benoit.boissinot@ens-lyon.org> parents: diff changeset	176 if not proxyuser:
2db33c1a5654 factor out the url handling from httprepo Benoit Boissinot <benoit.boissinot@ens-lyon.org> parents: diff changeset	177 proxyuser = ui.config("http_proxy", "user")
2db33c1a5654 factor out the url handling from httprepo Benoit Boissinot <benoit.boissinot@ens-lyon.org> parents: diff changeset	178 proxypasswd = ui.config("http_proxy", "passwd")
2db33c1a5654 factor out the url handling from httprepo Benoit Boissinot <benoit.boissinot@ens-lyon.org> parents: diff changeset	179
2db33c1a5654 factor out the url handling from httprepo Benoit Boissinot <benoit.boissinot@ens-lyon.org> parents: diff changeset	180 # see if we should use a proxy for this url
2db33c1a5654 factor out the url handling from httprepo Benoit Boissinot <benoit.boissinot@ens-lyon.org> parents: diff changeset	181 no_list = [ "localhost", "127.0.0.1" ]
2db33c1a5654 factor out the url handling from httprepo Benoit Boissinot <benoit.boissinot@ens-lyon.org> parents: diff changeset	182 no_list.extend([p.lower() for
2db33c1a5654 factor out the url handling from httprepo Benoit Boissinot <benoit.boissinot@ens-lyon.org> parents: diff changeset	183 p in ui.configlist("http_proxy", "no")])
2db33c1a5654 factor out the url handling from httprepo Benoit Boissinot <benoit.boissinot@ens-lyon.org> parents: diff changeset	184 no_list.extend([p.strip().lower() for
2db33c1a5654 factor out the url handling from httprepo Benoit Boissinot <benoit.boissinot@ens-lyon.org> parents: diff changeset	185 p in os.getenv("no_proxy", '').split(',')
2db33c1a5654 factor out the url handling from httprepo Benoit Boissinot <benoit.boissinot@ens-lyon.org> parents: diff changeset	186 if p.strip()])
2db33c1a5654 factor out the url handling from httprepo Benoit Boissinot <benoit.boissinot@ens-lyon.org> parents: diff changeset	187 # "http_proxy.always" config is for running tests on localhost
2db33c1a5654 factor out the url handling from httprepo Benoit Boissinot <benoit.boissinot@ens-lyon.org> parents: diff changeset	188 if ui.configbool("http_proxy", "always"):
2db33c1a5654 factor out the url handling from httprepo Benoit Boissinot <benoit.boissinot@ens-lyon.org> parents: diff changeset	189 self.no_list = []
2db33c1a5654 factor out the url handling from httprepo Benoit Boissinot <benoit.boissinot@ens-lyon.org> parents: diff changeset	190 else:
2db33c1a5654 factor out the url handling from httprepo Benoit Boissinot <benoit.boissinot@ens-lyon.org> parents: diff changeset	191 self.no_list = no_list
2db33c1a5654 factor out the url handling from httprepo Benoit Boissinot <benoit.boissinot@ens-lyon.org> parents: diff changeset	192
2db33c1a5654 factor out the url handling from httprepo Benoit Boissinot <benoit.boissinot@ens-lyon.org> parents: diff changeset	193 proxyurl = urlparse.urlunsplit((
2db33c1a5654 factor out the url handling from httprepo Benoit Boissinot <benoit.boissinot@ens-lyon.org> parents: diff changeset	194 proxyscheme, netlocunsplit(proxyhost, proxyport,
2db33c1a5654 factor out the url handling from httprepo Benoit Boissinot <benoit.boissinot@ens-lyon.org> parents: diff changeset	195 proxyuser, proxypasswd or ''),
2db33c1a5654 factor out the url handling from httprepo Benoit Boissinot <benoit.boissinot@ens-lyon.org> parents: diff changeset	196 proxypath, proxyquery, proxyfrag))
2db33c1a5654 factor out the url handling from httprepo Benoit Boissinot <benoit.boissinot@ens-lyon.org> parents: diff changeset	197 proxies = {'http': proxyurl, 'https': proxyurl}
2db33c1a5654 factor out the url handling from httprepo Benoit Boissinot <benoit.boissinot@ens-lyon.org> parents: diff changeset	198 ui.debug(_('proxying through http://%s:%s\n') %
2db33c1a5654 factor out the url handling from httprepo Benoit Boissinot <benoit.boissinot@ens-lyon.org> parents: diff changeset	199 (proxyhost, proxyport))
2db33c1a5654 factor out the url handling from httprepo Benoit Boissinot <benoit.boissinot@ens-lyon.org> parents: diff changeset	200 else:
2db33c1a5654 factor out the url handling from httprepo Benoit Boissinot <benoit.boissinot@ens-lyon.org> parents: diff changeset	201 proxies = {}
2db33c1a5654 factor out the url handling from httprepo Benoit Boissinot <benoit.boissinot@ens-lyon.org> parents: diff changeset	202
2db33c1a5654 factor out the url handling from httprepo Benoit Boissinot <benoit.boissinot@ens-lyon.org> parents: diff changeset	203 # urllib2 takes proxy values from the environment and those
2db33c1a5654 factor out the url handling from httprepo Benoit Boissinot <benoit.boissinot@ens-lyon.org> parents: diff changeset	204 # will take precedence if found, so drop them
2db33c1a5654 factor out the url handling from httprepo Benoit Boissinot <benoit.boissinot@ens-lyon.org> parents: diff changeset	205 for env in ["HTTP_PROXY", "http_proxy", "no_proxy"]:
2db33c1a5654 factor out the url handling from httprepo Benoit Boissinot <benoit.boissinot@ens-lyon.org> parents: diff changeset	206 try:
2db33c1a5654 factor out the url handling from httprepo Benoit Boissinot <benoit.boissinot@ens-lyon.org> parents: diff changeset	207 if env in os.environ:
2db33c1a5654 factor out the url handling from httprepo Benoit Boissinot <benoit.boissinot@ens-lyon.org> parents: diff changeset	208 del os.environ[env]
2db33c1a5654 factor out the url handling from httprepo Benoit Boissinot <benoit.boissinot@ens-lyon.org> parents: diff changeset	209 except OSError:
2db33c1a5654 factor out the url handling from httprepo Benoit Boissinot <benoit.boissinot@ens-lyon.org> parents: diff changeset	210 pass
2db33c1a5654 factor out the url handling from httprepo Benoit Boissinot <benoit.boissinot@ens-lyon.org> parents: diff changeset	211
2db33c1a5654 factor out the url handling from httprepo Benoit Boissinot <benoit.boissinot@ens-lyon.org> parents: diff changeset	212 urllib2.ProxyHandler.__init__(self, proxies)
2db33c1a5654 factor out the url handling from httprepo Benoit Boissinot <benoit.boissinot@ens-lyon.org> parents: diff changeset	213 self.ui = ui
2db33c1a5654 factor out the url handling from httprepo Benoit Boissinot <benoit.boissinot@ens-lyon.org> parents: diff changeset	214
2db33c1a5654 factor out the url handling from httprepo Benoit Boissinot <benoit.boissinot@ens-lyon.org> parents: diff changeset	215 def proxy_open(self, req, proxy, type_):
2db33c1a5654 factor out the url handling from httprepo Benoit Boissinot <benoit.boissinot@ens-lyon.org> parents: diff changeset	216 host = req.get_host().split(':')[0]
2db33c1a5654 factor out the url handling from httprepo Benoit Boissinot <benoit.boissinot@ens-lyon.org> parents: diff changeset	217 if host in self.no_list:
2db33c1a5654 factor out the url handling from httprepo Benoit Boissinot <benoit.boissinot@ens-lyon.org> parents: diff changeset	218 return None
2db33c1a5654 factor out the url handling from httprepo Benoit Boissinot <benoit.boissinot@ens-lyon.org> parents: diff changeset	219
2db33c1a5654 factor out the url handling from httprepo Benoit Boissinot <benoit.boissinot@ens-lyon.org> parents: diff changeset	220 # work around a bug in Python < 2.4.2
2db33c1a5654 factor out the url handling from httprepo Benoit Boissinot <benoit.boissinot@ens-lyon.org> parents: diff changeset	221 # (it leaves a "\n" at the end of Proxy-authorization headers)
2db33c1a5654 factor out the url handling from httprepo Benoit Boissinot <benoit.boissinot@ens-lyon.org> parents: diff changeset	222 baseclass = req.__class__
2db33c1a5654 factor out the url handling from httprepo Benoit Boissinot <benoit.boissinot@ens-lyon.org> parents: diff changeset	223 class _request(baseclass):
2db33c1a5654 factor out the url handling from httprepo Benoit Boissinot <benoit.boissinot@ens-lyon.org> parents: diff changeset	224 def add_header(self, key, val):
2db33c1a5654 factor out the url handling from httprepo Benoit Boissinot <benoit.boissinot@ens-lyon.org> parents: diff changeset	225 if key.lower() == 'proxy-authorization':
2db33c1a5654 factor out the url handling from httprepo Benoit Boissinot <benoit.boissinot@ens-lyon.org> parents: diff changeset	226 val = val.strip()
2db33c1a5654 factor out the url handling from httprepo Benoit Boissinot <benoit.boissinot@ens-lyon.org> parents: diff changeset	227 return baseclass.add_header(self, key, val)
2db33c1a5654 factor out the url handling from httprepo Benoit Boissinot <benoit.boissinot@ens-lyon.org> parents: diff changeset	228 req.__class__ = _request
2db33c1a5654 factor out the url handling from httprepo Benoit Boissinot <benoit.boissinot@ens-lyon.org> parents: diff changeset	229
2db33c1a5654 factor out the url handling from httprepo Benoit Boissinot <benoit.boissinot@ens-lyon.org> parents: diff changeset	230 return urllib2.ProxyHandler.proxy_open(self, req, proxy, type_)
2db33c1a5654 factor out the url handling from httprepo Benoit Boissinot <benoit.boissinot@ens-lyon.org> parents: diff changeset	231
2db33c1a5654 factor out the url handling from httprepo Benoit Boissinot <benoit.boissinot@ens-lyon.org> parents: diff changeset	232 class httpsendfile(file):
2db33c1a5654 factor out the url handling from httprepo Benoit Boissinot <benoit.boissinot@ens-lyon.org> parents: diff changeset	233 def __len__(self):
2db33c1a5654 factor out the url handling from httprepo Benoit Boissinot <benoit.boissinot@ens-lyon.org> parents: diff changeset	234 return os.fstat(self.fileno()).st_size
2db33c1a5654 factor out the url handling from httprepo Benoit Boissinot <benoit.boissinot@ens-lyon.org> parents: diff changeset	235
2db33c1a5654 factor out the url handling from httprepo Benoit Boissinot <benoit.boissinot@ens-lyon.org> parents: diff changeset	236 def _gen_sendfile(connection):
2db33c1a5654 factor out the url handling from httprepo Benoit Boissinot <benoit.boissinot@ens-lyon.org> parents: diff changeset	237 def _sendfile(self, data):
2db33c1a5654 factor out the url handling from httprepo Benoit Boissinot <benoit.boissinot@ens-lyon.org> parents: diff changeset	238 # send a file
2db33c1a5654 factor out the url handling from httprepo Benoit Boissinot <benoit.boissinot@ens-lyon.org> parents: diff changeset	239 if isinstance(data, httpsendfile):
2db33c1a5654 factor out the url handling from httprepo Benoit Boissinot <benoit.boissinot@ens-lyon.org> parents: diff changeset	240 # if auth required, some data sent twice, so rewind here
2db33c1a5654 factor out the url handling from httprepo Benoit Boissinot <benoit.boissinot@ens-lyon.org> parents: diff changeset	241 data.seek(0)
2db33c1a5654 factor out the url handling from httprepo Benoit Boissinot <benoit.boissinot@ens-lyon.org> parents: diff changeset	242 for chunk in util.filechunkiter(data):
2db33c1a5654 factor out the url handling from httprepo Benoit Boissinot <benoit.boissinot@ens-lyon.org> parents: diff changeset	243 connection.send(self, chunk)
2db33c1a5654 factor out the url handling from httprepo Benoit Boissinot <benoit.boissinot@ens-lyon.org> parents: diff changeset	244 else:
2db33c1a5654 factor out the url handling from httprepo Benoit Boissinot <benoit.boissinot@ens-lyon.org> parents: diff changeset	245 connection.send(self, data)
2db33c1a5654 factor out the url handling from httprepo Benoit Boissinot <benoit.boissinot@ens-lyon.org> parents: diff changeset	246 return _sendfile
2db33c1a5654 factor out the url handling from httprepo Benoit Boissinot <benoit.boissinot@ens-lyon.org> parents: diff changeset	247
8590 59acb9c7d90f url: use CONNECT for HTTPS connections through HTTP proxy (issue967) Henrik Stuart <hg@hstuart.dk> parents: 8344 diff changeset	248 has_https = hasattr(urllib2, 'HTTPSHandler')
59acb9c7d90f url: use CONNECT for HTTPS connections through HTTP proxy (issue967) Henrik Stuart <hg@hstuart.dk> parents: 8344 diff changeset	249 if has_https:
59acb9c7d90f url: use CONNECT for HTTPS connections through HTTP proxy (issue967) Henrik Stuart <hg@hstuart.dk> parents: 8344 diff changeset	250 try:
59acb9c7d90f url: use CONNECT for HTTPS connections through HTTP proxy (issue967) Henrik Stuart <hg@hstuart.dk> parents: 8344 diff changeset	251 # avoid using deprecated/broken FakeSocket in python 2.6
59acb9c7d90f url: use CONNECT for HTTPS connections through HTTP proxy (issue967) Henrik Stuart <hg@hstuart.dk> parents: 8344 diff changeset	252 import ssl
59acb9c7d90f url: use CONNECT for HTTPS connections through HTTP proxy (issue967) Henrik Stuart <hg@hstuart.dk> parents: 8344 diff changeset	253 _ssl_wrap_socket = ssl.wrap_socket
59acb9c7d90f url: use CONNECT for HTTPS connections through HTTP proxy (issue967) Henrik Stuart <hg@hstuart.dk> parents: 8344 diff changeset	254 except ImportError:
59acb9c7d90f url: use CONNECT for HTTPS connections through HTTP proxy (issue967) Henrik Stuart <hg@hstuart.dk> parents: 8344 diff changeset	255 def _ssl_wrap_socket(sock, key_file, cert_file):
59acb9c7d90f url: use CONNECT for HTTPS connections through HTTP proxy (issue967) Henrik Stuart <hg@hstuart.dk> parents: 8344 diff changeset	256 ssl = socket.ssl(sock, key_file, cert_file)
59acb9c7d90f url: use CONNECT for HTTPS connections through HTTP proxy (issue967) Henrik Stuart <hg@hstuart.dk> parents: 8344 diff changeset	257 return httplib.FakeSocket(sock, ssl)
59acb9c7d90f url: use CONNECT for HTTPS connections through HTTP proxy (issue967) Henrik Stuart <hg@hstuart.dk> parents: 8344 diff changeset	258
7270 2db33c1a5654 factor out the url handling from httprepo Benoit Boissinot <benoit.boissinot@ens-lyon.org> parents: diff changeset	259 class httpconnection(keepalive.HTTPConnection):
2db33c1a5654 factor out the url handling from httprepo Benoit Boissinot <benoit.boissinot@ens-lyon.org> parents: diff changeset	260 # must be able to send big bundle as stream.
2db33c1a5654 factor out the url handling from httprepo Benoit Boissinot <benoit.boissinot@ens-lyon.org> parents: diff changeset	261 send = _gen_sendfile(keepalive.HTTPConnection)
2db33c1a5654 factor out the url handling from httprepo Benoit Boissinot <benoit.boissinot@ens-lyon.org> parents: diff changeset	262
8590 59acb9c7d90f url: use CONNECT for HTTPS connections through HTTP proxy (issue967) Henrik Stuart <hg@hstuart.dk> parents: 8344 diff changeset	263 def _proxytunnel(self):
59acb9c7d90f url: use CONNECT for HTTPS connections through HTTP proxy (issue967) Henrik Stuart <hg@hstuart.dk> parents: 8344 diff changeset	264 proxyheaders = dict(
59acb9c7d90f url: use CONNECT for HTTPS connections through HTTP proxy (issue967) Henrik Stuart <hg@hstuart.dk> parents: 8344 diff changeset	265 [(x, self.headers[x]) for x in self.headers
59acb9c7d90f url: use CONNECT for HTTPS connections through HTTP proxy (issue967) Henrik Stuart <hg@hstuart.dk> parents: 8344 diff changeset	266 if x.lower().startswith('proxy-')])
59acb9c7d90f url: use CONNECT for HTTPS connections through HTTP proxy (issue967) Henrik Stuart <hg@hstuart.dk> parents: 8344 diff changeset	267 self._set_hostport(self.host, self.port)
59acb9c7d90f url: use CONNECT for HTTPS connections through HTTP proxy (issue967) Henrik Stuart <hg@hstuart.dk> parents: 8344 diff changeset	268 self.send('CONNECT %s:%d HTTP/1.0\r\n' % (self.realhost, self.realport))
59acb9c7d90f url: use CONNECT for HTTPS connections through HTTP proxy (issue967) Henrik Stuart <hg@hstuart.dk> parents: 8344 diff changeset	269 for header in proxyheaders.iteritems():
59acb9c7d90f url: use CONNECT for HTTPS connections through HTTP proxy (issue967) Henrik Stuart <hg@hstuart.dk> parents: 8344 diff changeset	270 self.send('%s: %s\r\n' % header)
59acb9c7d90f url: use CONNECT for HTTPS connections through HTTP proxy (issue967) Henrik Stuart <hg@hstuart.dk> parents: 8344 diff changeset	271 self.send('\r\n')
59acb9c7d90f url: use CONNECT for HTTPS connections through HTTP proxy (issue967) Henrik Stuart <hg@hstuart.dk> parents: 8344 diff changeset	272
59acb9c7d90f url: use CONNECT for HTTPS connections through HTTP proxy (issue967) Henrik Stuart <hg@hstuart.dk> parents: 8344 diff changeset	273 # majority of the following code is duplicated from
59acb9c7d90f url: use CONNECT for HTTPS connections through HTTP proxy (issue967) Henrik Stuart <hg@hstuart.dk> parents: 8344 diff changeset	274 # httplib.HTTPConnection as there are no adequate places to
59acb9c7d90f url: use CONNECT for HTTPS connections through HTTP proxy (issue967) Henrik Stuart <hg@hstuart.dk> parents: 8344 diff changeset	275 # override functions to provide the needed functionality
59acb9c7d90f url: use CONNECT for HTTPS connections through HTTP proxy (issue967) Henrik Stuart <hg@hstuart.dk> parents: 8344 diff changeset	276 res = self.response_class(self.sock,
59acb9c7d90f url: use CONNECT for HTTPS connections through HTTP proxy (issue967) Henrik Stuart <hg@hstuart.dk> parents: 8344 diff changeset	277 strict=self.strict,
59acb9c7d90f url: use CONNECT for HTTPS connections through HTTP proxy (issue967) Henrik Stuart <hg@hstuart.dk> parents: 8344 diff changeset	278 method=self._method)
59acb9c7d90f url: use CONNECT for HTTPS connections through HTTP proxy (issue967) Henrik Stuart <hg@hstuart.dk> parents: 8344 diff changeset	279
59acb9c7d90f url: use CONNECT for HTTPS connections through HTTP proxy (issue967) Henrik Stuart <hg@hstuart.dk> parents: 8344 diff changeset	280 while True:
59acb9c7d90f url: use CONNECT for HTTPS connections through HTTP proxy (issue967) Henrik Stuart <hg@hstuart.dk> parents: 8344 diff changeset	281 version, status, reason = res._read_status()
59acb9c7d90f url: use CONNECT for HTTPS connections through HTTP proxy (issue967) Henrik Stuart <hg@hstuart.dk> parents: 8344 diff changeset	282 if status != httplib.CONTINUE:
59acb9c7d90f url: use CONNECT for HTTPS connections through HTTP proxy (issue967) Henrik Stuart <hg@hstuart.dk> parents: 8344 diff changeset	283 break
59acb9c7d90f url: use CONNECT for HTTPS connections through HTTP proxy (issue967) Henrik Stuart <hg@hstuart.dk> parents: 8344 diff changeset	284 while True:
59acb9c7d90f url: use CONNECT for HTTPS connections through HTTP proxy (issue967) Henrik Stuart <hg@hstuart.dk> parents: 8344 diff changeset	285 skip = res.fp.readline().strip()
59acb9c7d90f url: use CONNECT for HTTPS connections through HTTP proxy (issue967) Henrik Stuart <hg@hstuart.dk> parents: 8344 diff changeset	286 if not skip:
59acb9c7d90f url: use CONNECT for HTTPS connections through HTTP proxy (issue967) Henrik Stuart <hg@hstuart.dk> parents: 8344 diff changeset	287 break
59acb9c7d90f url: use CONNECT for HTTPS connections through HTTP proxy (issue967) Henrik Stuart <hg@hstuart.dk> parents: 8344 diff changeset	288 res.status = status
59acb9c7d90f url: use CONNECT for HTTPS connections through HTTP proxy (issue967) Henrik Stuart <hg@hstuart.dk> parents: 8344 diff changeset	289 res.reason = reason.strip()
59acb9c7d90f url: use CONNECT for HTTPS connections through HTTP proxy (issue967) Henrik Stuart <hg@hstuart.dk> parents: 8344 diff changeset	290
59acb9c7d90f url: use CONNECT for HTTPS connections through HTTP proxy (issue967) Henrik Stuart <hg@hstuart.dk> parents: 8344 diff changeset	291 if res.status == 200:
59acb9c7d90f url: use CONNECT for HTTPS connections through HTTP proxy (issue967) Henrik Stuart <hg@hstuart.dk> parents: 8344 diff changeset	292 while True:
59acb9c7d90f url: use CONNECT for HTTPS connections through HTTP proxy (issue967) Henrik Stuart <hg@hstuart.dk> parents: 8344 diff changeset	293 line = res.fp.readline()
59acb9c7d90f url: use CONNECT for HTTPS connections through HTTP proxy (issue967) Henrik Stuart <hg@hstuart.dk> parents: 8344 diff changeset	294 if line == '\r\n':
59acb9c7d90f url: use CONNECT for HTTPS connections through HTTP proxy (issue967) Henrik Stuart <hg@hstuart.dk> parents: 8344 diff changeset	295 break
59acb9c7d90f url: use CONNECT for HTTPS connections through HTTP proxy (issue967) Henrik Stuart <hg@hstuart.dk> parents: 8344 diff changeset	296 return True
59acb9c7d90f url: use CONNECT for HTTPS connections through HTTP proxy (issue967) Henrik Stuart <hg@hstuart.dk> parents: 8344 diff changeset	297
59acb9c7d90f url: use CONNECT for HTTPS connections through HTTP proxy (issue967) Henrik Stuart <hg@hstuart.dk> parents: 8344 diff changeset	298 if version == 'HTTP/1.0':
59acb9c7d90f url: use CONNECT for HTTPS connections through HTTP proxy (issue967) Henrik Stuart <hg@hstuart.dk> parents: 8344 diff changeset	299 res.version = 10
59acb9c7d90f url: use CONNECT for HTTPS connections through HTTP proxy (issue967) Henrik Stuart <hg@hstuart.dk> parents: 8344 diff changeset	300 elif version.startswith('HTTP/1.'):
59acb9c7d90f url: use CONNECT for HTTPS connections through HTTP proxy (issue967) Henrik Stuart <hg@hstuart.dk> parents: 8344 diff changeset	301 res.version = 11
59acb9c7d90f url: use CONNECT for HTTPS connections through HTTP proxy (issue967) Henrik Stuart <hg@hstuart.dk> parents: 8344 diff changeset	302 elif version == 'HTTP/0.9':
59acb9c7d90f url: use CONNECT for HTTPS connections through HTTP proxy (issue967) Henrik Stuart <hg@hstuart.dk> parents: 8344 diff changeset	303 res.version = 9
59acb9c7d90f url: use CONNECT for HTTPS connections through HTTP proxy (issue967) Henrik Stuart <hg@hstuart.dk> parents: 8344 diff changeset	304 else:
59acb9c7d90f url: use CONNECT for HTTPS connections through HTTP proxy (issue967) Henrik Stuart <hg@hstuart.dk> parents: 8344 diff changeset	305 raise httplib.UnknownProtocol(version)
59acb9c7d90f url: use CONNECT for HTTPS connections through HTTP proxy (issue967) Henrik Stuart <hg@hstuart.dk> parents: 8344 diff changeset	306
59acb9c7d90f url: use CONNECT for HTTPS connections through HTTP proxy (issue967) Henrik Stuart <hg@hstuart.dk> parents: 8344 diff changeset	307 if res.version == 9:
59acb9c7d90f url: use CONNECT for HTTPS connections through HTTP proxy (issue967) Henrik Stuart <hg@hstuart.dk> parents: 8344 diff changeset	308 res.length = None
59acb9c7d90f url: use CONNECT for HTTPS connections through HTTP proxy (issue967) Henrik Stuart <hg@hstuart.dk> parents: 8344 diff changeset	309 res.chunked = 0
59acb9c7d90f url: use CONNECT for HTTPS connections through HTTP proxy (issue967) Henrik Stuart <hg@hstuart.dk> parents: 8344 diff changeset	310 res.will_close = 1
59acb9c7d90f url: use CONNECT for HTTPS connections through HTTP proxy (issue967) Henrik Stuart <hg@hstuart.dk> parents: 8344 diff changeset	311 res.msg = httplib.HTTPMessage(cStringIO.StringIO())
59acb9c7d90f url: use CONNECT for HTTPS connections through HTTP proxy (issue967) Henrik Stuart <hg@hstuart.dk> parents: 8344 diff changeset	312 return False
59acb9c7d90f url: use CONNECT for HTTPS connections through HTTP proxy (issue967) Henrik Stuart <hg@hstuart.dk> parents: 8344 diff changeset	313
59acb9c7d90f url: use CONNECT for HTTPS connections through HTTP proxy (issue967) Henrik Stuart <hg@hstuart.dk> parents: 8344 diff changeset	314 res.msg = httplib.HTTPMessage(res.fp)
59acb9c7d90f url: use CONNECT for HTTPS connections through HTTP proxy (issue967) Henrik Stuart <hg@hstuart.dk> parents: 8344 diff changeset	315 res.msg.fp = None
59acb9c7d90f url: use CONNECT for HTTPS connections through HTTP proxy (issue967) Henrik Stuart <hg@hstuart.dk> parents: 8344 diff changeset	316
59acb9c7d90f url: use CONNECT for HTTPS connections through HTTP proxy (issue967) Henrik Stuart <hg@hstuart.dk> parents: 8344 diff changeset	317 # are we using the chunked-style of transfer encoding?
59acb9c7d90f url: use CONNECT for HTTPS connections through HTTP proxy (issue967) Henrik Stuart <hg@hstuart.dk> parents: 8344 diff changeset	318 trenc = res.msg.getheader('transfer-encoding')
59acb9c7d90f url: use CONNECT for HTTPS connections through HTTP proxy (issue967) Henrik Stuart <hg@hstuart.dk> parents: 8344 diff changeset	319 if trenc and trenc.lower() == "chunked":
59acb9c7d90f url: use CONNECT for HTTPS connections through HTTP proxy (issue967) Henrik Stuart <hg@hstuart.dk> parents: 8344 diff changeset	320 res.chunked = 1
59acb9c7d90f url: use CONNECT for HTTPS connections through HTTP proxy (issue967) Henrik Stuart <hg@hstuart.dk> parents: 8344 diff changeset	321 res.chunk_left = None
59acb9c7d90f url: use CONNECT for HTTPS connections through HTTP proxy (issue967) Henrik Stuart <hg@hstuart.dk> parents: 8344 diff changeset	322 else:
59acb9c7d90f url: use CONNECT for HTTPS connections through HTTP proxy (issue967) Henrik Stuart <hg@hstuart.dk> parents: 8344 diff changeset	323 res.chunked = 0
59acb9c7d90f url: use CONNECT for HTTPS connections through HTTP proxy (issue967) Henrik Stuart <hg@hstuart.dk> parents: 8344 diff changeset	324
59acb9c7d90f url: use CONNECT for HTTPS connections through HTTP proxy (issue967) Henrik Stuart <hg@hstuart.dk> parents: 8344 diff changeset	325 # will the connection close at the end of the response?
59acb9c7d90f url: use CONNECT for HTTPS connections through HTTP proxy (issue967) Henrik Stuart <hg@hstuart.dk> parents: 8344 diff changeset	326 res.will_close = res._check_close()
59acb9c7d90f url: use CONNECT for HTTPS connections through HTTP proxy (issue967) Henrik Stuart <hg@hstuart.dk> parents: 8344 diff changeset	327
59acb9c7d90f url: use CONNECT for HTTPS connections through HTTP proxy (issue967) Henrik Stuart <hg@hstuart.dk> parents: 8344 diff changeset	328 # do we have a Content-Length?
59acb9c7d90f url: use CONNECT for HTTPS connections through HTTP proxy (issue967) Henrik Stuart <hg@hstuart.dk> parents: 8344 diff changeset	329 # NOTE: RFC 2616, S4.4, #3 says we ignore this if tr_enc is "chunked"
59acb9c7d90f url: use CONNECT for HTTPS connections through HTTP proxy (issue967) Henrik Stuart <hg@hstuart.dk> parents: 8344 diff changeset	330 length = res.msg.getheader('content-length')
59acb9c7d90f url: use CONNECT for HTTPS connections through HTTP proxy (issue967) Henrik Stuart <hg@hstuart.dk> parents: 8344 diff changeset	331 if length and not res.chunked:
59acb9c7d90f url: use CONNECT for HTTPS connections through HTTP proxy (issue967) Henrik Stuart <hg@hstuart.dk> parents: 8344 diff changeset	332 try:
59acb9c7d90f url: use CONNECT for HTTPS connections through HTTP proxy (issue967) Henrik Stuart <hg@hstuart.dk> parents: 8344 diff changeset	333 res.length = int(length)
59acb9c7d90f url: use CONNECT for HTTPS connections through HTTP proxy (issue967) Henrik Stuart <hg@hstuart.dk> parents: 8344 diff changeset	334 except ValueError:
59acb9c7d90f url: use CONNECT for HTTPS connections through HTTP proxy (issue967) Henrik Stuart <hg@hstuart.dk> parents: 8344 diff changeset	335 res.length = None
59acb9c7d90f url: use CONNECT for HTTPS connections through HTTP proxy (issue967) Henrik Stuart <hg@hstuart.dk> parents: 8344 diff changeset	336 else:
59acb9c7d90f url: use CONNECT for HTTPS connections through HTTP proxy (issue967) Henrik Stuart <hg@hstuart.dk> parents: 8344 diff changeset	337 if res.length < 0: # ignore nonsensical negative lengths
59acb9c7d90f url: use CONNECT for HTTPS connections through HTTP proxy (issue967) Henrik Stuart <hg@hstuart.dk> parents: 8344 diff changeset	338 res.length = None
59acb9c7d90f url: use CONNECT for HTTPS connections through HTTP proxy (issue967) Henrik Stuart <hg@hstuart.dk> parents: 8344 diff changeset	339 else:
59acb9c7d90f url: use CONNECT for HTTPS connections through HTTP proxy (issue967) Henrik Stuart <hg@hstuart.dk> parents: 8344 diff changeset	340 res.length = None
59acb9c7d90f url: use CONNECT for HTTPS connections through HTTP proxy (issue967) Henrik Stuart <hg@hstuart.dk> parents: 8344 diff changeset	341
59acb9c7d90f url: use CONNECT for HTTPS connections through HTTP proxy (issue967) Henrik Stuart <hg@hstuart.dk> parents: 8344 diff changeset	342 # does the body have a fixed length? (of zero)
59acb9c7d90f url: use CONNECT for HTTPS connections through HTTP proxy (issue967) Henrik Stuart <hg@hstuart.dk> parents: 8344 diff changeset	343 if (status == httplib.NO_CONTENT or status == httplib.NOT_MODIFIED or
59acb9c7d90f url: use CONNECT for HTTPS connections through HTTP proxy (issue967) Henrik Stuart <hg@hstuart.dk> parents: 8344 diff changeset	344 100 <= status < 200 or # 1xx codes
59acb9c7d90f url: use CONNECT for HTTPS connections through HTTP proxy (issue967) Henrik Stuart <hg@hstuart.dk> parents: 8344 diff changeset	345 res._method == 'HEAD'):
59acb9c7d90f url: use CONNECT for HTTPS connections through HTTP proxy (issue967) Henrik Stuart <hg@hstuart.dk> parents: 8344 diff changeset	346 res.length = 0
59acb9c7d90f url: use CONNECT for HTTPS connections through HTTP proxy (issue967) Henrik Stuart <hg@hstuart.dk> parents: 8344 diff changeset	347
59acb9c7d90f url: use CONNECT for HTTPS connections through HTTP proxy (issue967) Henrik Stuart <hg@hstuart.dk> parents: 8344 diff changeset	348 # if the connection remains open, and we aren't using chunked, and
59acb9c7d90f url: use CONNECT for HTTPS connections through HTTP proxy (issue967) Henrik Stuart <hg@hstuart.dk> parents: 8344 diff changeset	349 # a content-length was not provided, then assume that the connection
59acb9c7d90f url: use CONNECT for HTTPS connections through HTTP proxy (issue967) Henrik Stuart <hg@hstuart.dk> parents: 8344 diff changeset	350 # WILL close.
59acb9c7d90f url: use CONNECT for HTTPS connections through HTTP proxy (issue967) Henrik Stuart <hg@hstuart.dk> parents: 8344 diff changeset	351 if (not res.will_close and
59acb9c7d90f url: use CONNECT for HTTPS connections through HTTP proxy (issue967) Henrik Stuart <hg@hstuart.dk> parents: 8344 diff changeset	352 not res.chunked and
59acb9c7d90f url: use CONNECT for HTTPS connections through HTTP proxy (issue967) Henrik Stuart <hg@hstuart.dk> parents: 8344 diff changeset	353 res.length is None):
59acb9c7d90f url: use CONNECT for HTTPS connections through HTTP proxy (issue967) Henrik Stuart <hg@hstuart.dk> parents: 8344 diff changeset	354 res.will_close = 1
59acb9c7d90f url: use CONNECT for HTTPS connections through HTTP proxy (issue967) Henrik Stuart <hg@hstuart.dk> parents: 8344 diff changeset	355
59acb9c7d90f url: use CONNECT for HTTPS connections through HTTP proxy (issue967) Henrik Stuart <hg@hstuart.dk> parents: 8344 diff changeset	356 self.proxyres = res
59acb9c7d90f url: use CONNECT for HTTPS connections through HTTP proxy (issue967) Henrik Stuart <hg@hstuart.dk> parents: 8344 diff changeset	357
59acb9c7d90f url: use CONNECT for HTTPS connections through HTTP proxy (issue967) Henrik Stuart <hg@hstuart.dk> parents: 8344 diff changeset	358 return False
59acb9c7d90f url: use CONNECT for HTTPS connections through HTTP proxy (issue967) Henrik Stuart <hg@hstuart.dk> parents: 8344 diff changeset	359
59acb9c7d90f url: use CONNECT for HTTPS connections through HTTP proxy (issue967) Henrik Stuart <hg@hstuart.dk> parents: 8344 diff changeset	360 def connect(self):
59acb9c7d90f url: use CONNECT for HTTPS connections through HTTP proxy (issue967) Henrik Stuart <hg@hstuart.dk> parents: 8344 diff changeset	361 if has_https and self.realhost: # use CONNECT proxy
59acb9c7d90f url: use CONNECT for HTTPS connections through HTTP proxy (issue967) Henrik Stuart <hg@hstuart.dk> parents: 8344 diff changeset	362 self.sock = socket.socket(socket.AF_INET, socket.SOCK_STREAM)
59acb9c7d90f url: use CONNECT for HTTPS connections through HTTP proxy (issue967) Henrik Stuart <hg@hstuart.dk> parents: 8344 diff changeset	363 self.sock.connect((self.host, self.port))
59acb9c7d90f url: use CONNECT for HTTPS connections through HTTP proxy (issue967) Henrik Stuart <hg@hstuart.dk> parents: 8344 diff changeset	364 if self._proxytunnel():
59acb9c7d90f url: use CONNECT for HTTPS connections through HTTP proxy (issue967) Henrik Stuart <hg@hstuart.dk> parents: 8344 diff changeset	365 # we do not support client x509 certificates
59acb9c7d90f url: use CONNECT for HTTPS connections through HTTP proxy (issue967) Henrik Stuart <hg@hstuart.dk> parents: 8344 diff changeset	366 self.sock = _ssl_wrap_socket(self.sock, None, None)
59acb9c7d90f url: use CONNECT for HTTPS connections through HTTP proxy (issue967) Henrik Stuart <hg@hstuart.dk> parents: 8344 diff changeset	367 else:
59acb9c7d90f url: use CONNECT for HTTPS connections through HTTP proxy (issue967) Henrik Stuart <hg@hstuart.dk> parents: 8344 diff changeset	368 keepalive.HTTPConnection.connect(self)
59acb9c7d90f url: use CONNECT for HTTPS connections through HTTP proxy (issue967) Henrik Stuart <hg@hstuart.dk> parents: 8344 diff changeset	369
59acb9c7d90f url: use CONNECT for HTTPS connections through HTTP proxy (issue967) Henrik Stuart <hg@hstuart.dk> parents: 8344 diff changeset	370 def getresponse(self):
59acb9c7d90f url: use CONNECT for HTTPS connections through HTTP proxy (issue967) Henrik Stuart <hg@hstuart.dk> parents: 8344 diff changeset	371 proxyres = getattr(self, 'proxyres', None)
59acb9c7d90f url: use CONNECT for HTTPS connections through HTTP proxy (issue967) Henrik Stuart <hg@hstuart.dk> parents: 8344 diff changeset	372 if proxyres:
59acb9c7d90f url: use CONNECT for HTTPS connections through HTTP proxy (issue967) Henrik Stuart <hg@hstuart.dk> parents: 8344 diff changeset	373 if proxyres.will_close:
59acb9c7d90f url: use CONNECT for HTTPS connections through HTTP proxy (issue967) Henrik Stuart <hg@hstuart.dk> parents: 8344 diff changeset	374 self.close()
59acb9c7d90f url: use CONNECT for HTTPS connections through HTTP proxy (issue967) Henrik Stuart <hg@hstuart.dk> parents: 8344 diff changeset	375 self.proxyres = None
59acb9c7d90f url: use CONNECT for HTTPS connections through HTTP proxy (issue967) Henrik Stuart <hg@hstuart.dk> parents: 8344 diff changeset	376 return proxyres
59acb9c7d90f url: use CONNECT for HTTPS connections through HTTP proxy (issue967) Henrik Stuart <hg@hstuart.dk> parents: 8344 diff changeset	377 return keepalive.HTTPConnection.getresponse(self)
59acb9c7d90f url: use CONNECT for HTTPS connections through HTTP proxy (issue967) Henrik Stuart <hg@hstuart.dk> parents: 8344 diff changeset	378
7270 2db33c1a5654 factor out the url handling from httprepo Benoit Boissinot <benoit.boissinot@ens-lyon.org> parents: diff changeset	379 class httphandler(keepalive.HTTPHandler):
2db33c1a5654 factor out the url handling from httprepo Benoit Boissinot <benoit.boissinot@ens-lyon.org> parents: diff changeset	380 def http_open(self, req):
2db33c1a5654 factor out the url handling from httprepo Benoit Boissinot <benoit.boissinot@ens-lyon.org> parents: diff changeset	381 return self.do_open(httpconnection, req)
2db33c1a5654 factor out the url handling from httprepo Benoit Boissinot <benoit.boissinot@ens-lyon.org> parents: diff changeset	382
8590 59acb9c7d90f url: use CONNECT for HTTPS connections through HTTP proxy (issue967) Henrik Stuart <hg@hstuart.dk> parents: 8344 diff changeset	383 def _start_transaction(self, h, req):
59acb9c7d90f url: use CONNECT for HTTPS connections through HTTP proxy (issue967) Henrik Stuart <hg@hstuart.dk> parents: 8344 diff changeset	384 if req.get_selector() == req.get_full_url(): # has proxy
59acb9c7d90f url: use CONNECT for HTTPS connections through HTTP proxy (issue967) Henrik Stuart <hg@hstuart.dk> parents: 8344 diff changeset	385 urlparts = urlparse.urlparse(req.get_selector())
59acb9c7d90f url: use CONNECT for HTTPS connections through HTTP proxy (issue967) Henrik Stuart <hg@hstuart.dk> parents: 8344 diff changeset	386 if urlparts[0] == 'https': # only use CONNECT for HTTPS
59acb9c7d90f url: use CONNECT for HTTPS connections through HTTP proxy (issue967) Henrik Stuart <hg@hstuart.dk> parents: 8344 diff changeset	387 if ':' in urlparts[1]:
59acb9c7d90f url: use CONNECT for HTTPS connections through HTTP proxy (issue967) Henrik Stuart <hg@hstuart.dk> parents: 8344 diff changeset	388 realhost, realport = urlparts[1].split(':')
59acb9c7d90f url: use CONNECT for HTTPS connections through HTTP proxy (issue967) Henrik Stuart <hg@hstuart.dk> parents: 8344 diff changeset	389 else:
59acb9c7d90f url: use CONNECT for HTTPS connections through HTTP proxy (issue967) Henrik Stuart <hg@hstuart.dk> parents: 8344 diff changeset	390 realhost = urlparts[1]
59acb9c7d90f url: use CONNECT for HTTPS connections through HTTP proxy (issue967) Henrik Stuart <hg@hstuart.dk> parents: 8344 diff changeset	391 realport = 443
59acb9c7d90f url: use CONNECT for HTTPS connections through HTTP proxy (issue967) Henrik Stuart <hg@hstuart.dk> parents: 8344 diff changeset	392
59acb9c7d90f url: use CONNECT for HTTPS connections through HTTP proxy (issue967) Henrik Stuart <hg@hstuart.dk> parents: 8344 diff changeset	393 h.realhost = realhost
59acb9c7d90f url: use CONNECT for HTTPS connections through HTTP proxy (issue967) Henrik Stuart <hg@hstuart.dk> parents: 8344 diff changeset	394 h.realport = realport
59acb9c7d90f url: use CONNECT for HTTPS connections through HTTP proxy (issue967) Henrik Stuart <hg@hstuart.dk> parents: 8344 diff changeset	395 h.headers = req.headers.copy()
59acb9c7d90f url: use CONNECT for HTTPS connections through HTTP proxy (issue967) Henrik Stuart <hg@hstuart.dk> parents: 8344 diff changeset	396 h.headers.update(self.parent.addheaders)
59acb9c7d90f url: use CONNECT for HTTPS connections through HTTP proxy (issue967) Henrik Stuart <hg@hstuart.dk> parents: 8344 diff changeset	397 return keepalive.HTTPHandler._start_transaction(self, h, req)
59acb9c7d90f url: use CONNECT for HTTPS connections through HTTP proxy (issue967) Henrik Stuart <hg@hstuart.dk> parents: 8344 diff changeset	398
59acb9c7d90f url: use CONNECT for HTTPS connections through HTTP proxy (issue967) Henrik Stuart <hg@hstuart.dk> parents: 8344 diff changeset	399 h.realhost = None
59acb9c7d90f url: use CONNECT for HTTPS connections through HTTP proxy (issue967) Henrik Stuart <hg@hstuart.dk> parents: 8344 diff changeset	400 h.realport = None
59acb9c7d90f url: use CONNECT for HTTPS connections through HTTP proxy (issue967) Henrik Stuart <hg@hstuart.dk> parents: 8344 diff changeset	401 h.headers = None
59acb9c7d90f url: use CONNECT for HTTPS connections through HTTP proxy (issue967) Henrik Stuart <hg@hstuart.dk> parents: 8344 diff changeset	402 return keepalive.HTTPHandler._start_transaction(self, h, req)
59acb9c7d90f url: use CONNECT for HTTPS connections through HTTP proxy (issue967) Henrik Stuart <hg@hstuart.dk> parents: 8344 diff changeset	403
7270 2db33c1a5654 factor out the url handling from httprepo Benoit Boissinot <benoit.boissinot@ens-lyon.org> parents: diff changeset	404 def __del__(self):
2db33c1a5654 factor out the url handling from httprepo Benoit Boissinot <benoit.boissinot@ens-lyon.org> parents: diff changeset	405 self.close_all()
2db33c1a5654 factor out the url handling from httprepo Benoit Boissinot <benoit.boissinot@ens-lyon.org> parents: diff changeset	406
2db33c1a5654 factor out the url handling from httprepo Benoit Boissinot <benoit.boissinot@ens-lyon.org> parents: diff changeset	407 if has_https:
2db33c1a5654 factor out the url handling from httprepo Benoit Boissinot <benoit.boissinot@ens-lyon.org> parents: diff changeset	408 class httpsconnection(httplib.HTTPSConnection):
2db33c1a5654 factor out the url handling from httprepo Benoit Boissinot <benoit.boissinot@ens-lyon.org> parents: diff changeset	409 response_class = keepalive.HTTPResponse
2db33c1a5654 factor out the url handling from httprepo Benoit Boissinot <benoit.boissinot@ens-lyon.org> parents: diff changeset	410 # must be able to send big bundle as stream.
2db33c1a5654 factor out the url handling from httprepo Benoit Boissinot <benoit.boissinot@ens-lyon.org> parents: diff changeset	411 send = _gen_sendfile(httplib.HTTPSConnection)
2db33c1a5654 factor out the url handling from httprepo Benoit Boissinot <benoit.boissinot@ens-lyon.org> parents: diff changeset	412
2db33c1a5654 factor out the url handling from httprepo Benoit Boissinot <benoit.boissinot@ens-lyon.org> parents: diff changeset	413 class httpshandler(keepalive.KeepAliveHandler, urllib2.HTTPSHandler):
2db33c1a5654 factor out the url handling from httprepo Benoit Boissinot <benoit.boissinot@ens-lyon.org> parents: diff changeset	414 def https_open(self, req):
2db33c1a5654 factor out the url handling from httprepo Benoit Boissinot <benoit.boissinot@ens-lyon.org> parents: diff changeset	415 return self.do_open(httpsconnection, req)
2db33c1a5654 factor out the url handling from httprepo Benoit Boissinot <benoit.boissinot@ens-lyon.org> parents: diff changeset	416
2db33c1a5654 factor out the url handling from httprepo Benoit Boissinot <benoit.boissinot@ens-lyon.org> parents: diff changeset	417 # In python < 2.5 AbstractDigestAuthHandler raises a ValueError if
2db33c1a5654 factor out the url handling from httprepo Benoit Boissinot <benoit.boissinot@ens-lyon.org> parents: diff changeset	418 # it doesn't know about the auth type requested. This can happen if
2db33c1a5654 factor out the url handling from httprepo Benoit Boissinot <benoit.boissinot@ens-lyon.org> parents: diff changeset	419 # somebody is using BasicAuth and types a bad password.
2db33c1a5654 factor out the url handling from httprepo Benoit Boissinot <benoit.boissinot@ens-lyon.org> parents: diff changeset	420 class httpdigestauthhandler(urllib2.HTTPDigestAuthHandler):
2db33c1a5654 factor out the url handling from httprepo Benoit Boissinot <benoit.boissinot@ens-lyon.org> parents: diff changeset	421 def http_error_auth_reqed(self, auth_header, host, req, headers):
2db33c1a5654 factor out the url handling from httprepo Benoit Boissinot <benoit.boissinot@ens-lyon.org> parents: diff changeset	422 try:
2db33c1a5654 factor out the url handling from httprepo Benoit Boissinot <benoit.boissinot@ens-lyon.org> parents: diff changeset	423 return urllib2.HTTPDigestAuthHandler.http_error_auth_reqed(
2db33c1a5654 factor out the url handling from httprepo Benoit Boissinot <benoit.boissinot@ens-lyon.org> parents: diff changeset	424 self, auth_header, host, req, headers)
2db33c1a5654 factor out the url handling from httprepo Benoit Boissinot <benoit.boissinot@ens-lyon.org> parents: diff changeset	425 except ValueError, inst:
2db33c1a5654 factor out the url handling from httprepo Benoit Boissinot <benoit.boissinot@ens-lyon.org> parents: diff changeset	426 arg = inst.args[0]
2db33c1a5654 factor out the url handling from httprepo Benoit Boissinot <benoit.boissinot@ens-lyon.org> parents: diff changeset	427 if arg.startswith("AbstractDigestAuthHandler doesn't know "):
2db33c1a5654 factor out the url handling from httprepo Benoit Boissinot <benoit.boissinot@ens-lyon.org> parents: diff changeset	428 return
2db33c1a5654 factor out the url handling from httprepo Benoit Boissinot <benoit.boissinot@ens-lyon.org> parents: diff changeset	429 raise
2db33c1a5654 factor out the url handling from httprepo Benoit Boissinot <benoit.boissinot@ens-lyon.org> parents: diff changeset	430
2db33c1a5654 factor out the url handling from httprepo Benoit Boissinot <benoit.boissinot@ens-lyon.org> parents: diff changeset	431 def getauthinfo(path):
2db33c1a5654 factor out the url handling from httprepo Benoit Boissinot <benoit.boissinot@ens-lyon.org> parents: diff changeset	432 scheme, netloc, urlpath, query, frag = urlparse.urlsplit(path)
2db33c1a5654 factor out the url handling from httprepo Benoit Boissinot <benoit.boissinot@ens-lyon.org> parents: diff changeset	433 if not urlpath:
2db33c1a5654 factor out the url handling from httprepo Benoit Boissinot <benoit.boissinot@ens-lyon.org> parents: diff changeset	434 urlpath = '/'
7284 ac81ffac0f35 url: fix file:// URL handling Patrick Mezard <pmezard@gmail.com> parents: 7270 diff changeset	435 if scheme != 'file':
ac81ffac0f35 url: fix file:// URL handling Patrick Mezard <pmezard@gmail.com> parents: 7270 diff changeset	436 # XXX: why are we quoting the path again with some smart
ac81ffac0f35 url: fix file:// URL handling Patrick Mezard <pmezard@gmail.com> parents: 7270 diff changeset	437 # heuristic here? Anyway, it cannot be done with file://
ac81ffac0f35 url: fix file:// URL handling Patrick Mezard <pmezard@gmail.com> parents: 7270 diff changeset	438 # urls since path encoding is os/fs dependent (see
ac81ffac0f35 url: fix file:// URL handling Patrick Mezard <pmezard@gmail.com> parents: 7270 diff changeset	439 # urllib.pathname2url() for details).
ac81ffac0f35 url: fix file:// URL handling Patrick Mezard <pmezard@gmail.com> parents: 7270 diff changeset	440 urlpath = quotepath(urlpath)
7270 2db33c1a5654 factor out the url handling from httprepo Benoit Boissinot <benoit.boissinot@ens-lyon.org> parents: diff changeset	441 host, port, user, passwd = netlocsplit(netloc)
2db33c1a5654 factor out the url handling from httprepo Benoit Boissinot <benoit.boissinot@ens-lyon.org> parents: diff changeset	442
2db33c1a5654 factor out the url handling from httprepo Benoit Boissinot <benoit.boissinot@ens-lyon.org> parents: diff changeset	443 # urllib cannot handle URLs with embedded user or passwd
2db33c1a5654 factor out the url handling from httprepo Benoit Boissinot <benoit.boissinot@ens-lyon.org> parents: diff changeset	444 url = urlparse.urlunsplit((scheme, netlocunsplit(host, port),
2db33c1a5654 factor out the url handling from httprepo Benoit Boissinot <benoit.boissinot@ens-lyon.org> parents: diff changeset	445 urlpath, query, frag))
2db33c1a5654 factor out the url handling from httprepo Benoit Boissinot <benoit.boissinot@ens-lyon.org> parents: diff changeset	446 if user:
2db33c1a5654 factor out the url handling from httprepo Benoit Boissinot <benoit.boissinot@ens-lyon.org> parents: diff changeset	447 netloc = host
2db33c1a5654 factor out the url handling from httprepo Benoit Boissinot <benoit.boissinot@ens-lyon.org> parents: diff changeset	448 if port:
2db33c1a5654 factor out the url handling from httprepo Benoit Boissinot <benoit.boissinot@ens-lyon.org> parents: diff changeset	449 netloc += ':' + port
2db33c1a5654 factor out the url handling from httprepo Benoit Boissinot <benoit.boissinot@ens-lyon.org> parents: diff changeset	450 # Python < 2.4.3 uses only the netloc to search for a password
2db33c1a5654 factor out the url handling from httprepo Benoit Boissinot <benoit.boissinot@ens-lyon.org> parents: diff changeset	451 authinfo = (None, (url, netloc), user, passwd or '')
2db33c1a5654 factor out the url handling from httprepo Benoit Boissinot <benoit.boissinot@ens-lyon.org> parents: diff changeset	452 else:
2db33c1a5654 factor out the url handling from httprepo Benoit Boissinot <benoit.boissinot@ens-lyon.org> parents: diff changeset	453 authinfo = None
2db33c1a5654 factor out the url handling from httprepo Benoit Boissinot <benoit.boissinot@ens-lyon.org> parents: diff changeset	454 return url, authinfo
2db33c1a5654 factor out the url handling from httprepo Benoit Boissinot <benoit.boissinot@ens-lyon.org> parents: diff changeset	455
2db33c1a5654 factor out the url handling from httprepo Benoit Boissinot <benoit.boissinot@ens-lyon.org> parents: diff changeset	456 def opener(ui, authinfo=None):
2db33c1a5654 factor out the url handling from httprepo Benoit Boissinot <benoit.boissinot@ens-lyon.org> parents: diff changeset	457 '''
2db33c1a5654 factor out the url handling from httprepo Benoit Boissinot <benoit.boissinot@ens-lyon.org> parents: diff changeset	458 construct an opener suitable for urllib2
2db33c1a5654 factor out the url handling from httprepo Benoit Boissinot <benoit.boissinot@ens-lyon.org> parents: diff changeset	459 authinfo will be added to the password manager
2db33c1a5654 factor out the url handling from httprepo Benoit Boissinot <benoit.boissinot@ens-lyon.org> parents: diff changeset	460 '''
2db33c1a5654 factor out the url handling from httprepo Benoit Boissinot <benoit.boissinot@ens-lyon.org> parents: diff changeset	461 handlers = [httphandler()]
2db33c1a5654 factor out the url handling from httprepo Benoit Boissinot <benoit.boissinot@ens-lyon.org> parents: diff changeset	462 if has_https:
2db33c1a5654 factor out the url handling from httprepo Benoit Boissinot <benoit.boissinot@ens-lyon.org> parents: diff changeset	463 handlers.append(httpshandler())
2db33c1a5654 factor out the url handling from httprepo Benoit Boissinot <benoit.boissinot@ens-lyon.org> parents: diff changeset	464
2db33c1a5654 factor out the url handling from httprepo Benoit Boissinot <benoit.boissinot@ens-lyon.org> parents: diff changeset	465 handlers.append(proxyhandler(ui))
2db33c1a5654 factor out the url handling from httprepo Benoit Boissinot <benoit.boissinot@ens-lyon.org> parents: diff changeset	466
2db33c1a5654 factor out the url handling from httprepo Benoit Boissinot <benoit.boissinot@ens-lyon.org> parents: diff changeset	467 passmgr = passwordmgr(ui)
2db33c1a5654 factor out the url handling from httprepo Benoit Boissinot <benoit.boissinot@ens-lyon.org> parents: diff changeset	468 if authinfo is not None:
2db33c1a5654 factor out the url handling from httprepo Benoit Boissinot <benoit.boissinot@ens-lyon.org> parents: diff changeset	469 passmgr.add_password(*authinfo)
2db33c1a5654 factor out the url handling from httprepo Benoit Boissinot <benoit.boissinot@ens-lyon.org> parents: diff changeset	470 user, passwd = authinfo[2:4]
2db33c1a5654 factor out the url handling from httprepo Benoit Boissinot <benoit.boissinot@ens-lyon.org> parents: diff changeset	471 ui.debug(_('http auth: user %s, password %s\n') %
2db33c1a5654 factor out the url handling from httprepo Benoit Boissinot <benoit.boissinot@ens-lyon.org> parents: diff changeset	472 (user, passwd and '' len(passwd) or 'not set'))
2db33c1a5654 factor out the url handling from httprepo Benoit Boissinot <benoit.boissinot@ens-lyon.org> parents: diff changeset	473
2db33c1a5654 factor out the url handling from httprepo Benoit Boissinot <benoit.boissinot@ens-lyon.org> parents: diff changeset	474 handlers.extend((urllib2.HTTPBasicAuthHandler(passmgr),
2db33c1a5654 factor out the url handling from httprepo Benoit Boissinot <benoit.boissinot@ens-lyon.org> parents: diff changeset	475 httpdigestauthhandler(passmgr)))
2db33c1a5654 factor out the url handling from httprepo Benoit Boissinot <benoit.boissinot@ens-lyon.org> parents: diff changeset	476 opener = urllib2.build_opener(*handlers)
2db33c1a5654 factor out the url handling from httprepo Benoit Boissinot <benoit.boissinot@ens-lyon.org> parents: diff changeset	477
2db33c1a5654 factor out the url handling from httprepo Benoit Boissinot <benoit.boissinot@ens-lyon.org> parents: diff changeset	478 # 1.0 here is the _protocol_ version
2db33c1a5654 factor out the url handling from httprepo Benoit Boissinot <benoit.boissinot@ens-lyon.org> parents: diff changeset	479 opener.addheaders = [('User-agent', 'mercurial/proto-1.0')]
2db33c1a5654 factor out the url handling from httprepo Benoit Boissinot <benoit.boissinot@ens-lyon.org> parents: diff changeset	480 opener.addheaders.append(('Accept', 'application/mercurial-0.1'))
2db33c1a5654 factor out the url handling from httprepo Benoit Boissinot <benoit.boissinot@ens-lyon.org> parents: diff changeset	481 return opener
2db33c1a5654 factor out the url handling from httprepo Benoit Boissinot <benoit.boissinot@ens-lyon.org> parents: diff changeset	482
7285 5ad99abfab79 url: detect scheme with a regexp instead of urlsplit() Patrick Mezard <pmezard@gmail.com> parents: 7284 diff changeset	483 scheme_re = re.compile(r'^([a-zA-Z0-9+-.]+)://')
5ad99abfab79 url: detect scheme with a regexp instead of urlsplit() Patrick Mezard <pmezard@gmail.com> parents: 7284 diff changeset	484
7270 2db33c1a5654 factor out the url handling from httprepo Benoit Boissinot <benoit.boissinot@ens-lyon.org> parents: diff changeset	485 def open(ui, url, data=None):
7285 5ad99abfab79 url: detect scheme with a regexp instead of urlsplit() Patrick Mezard <pmezard@gmail.com> parents: 7284 diff changeset	486 scheme = None
5ad99abfab79 url: detect scheme with a regexp instead of urlsplit() Patrick Mezard <pmezard@gmail.com> parents: 7284 diff changeset	487 m = scheme_re.search(url)
5ad99abfab79 url: detect scheme with a regexp instead of urlsplit() Patrick Mezard <pmezard@gmail.com> parents: 7284 diff changeset	488 if m:
5ad99abfab79 url: detect scheme with a regexp instead of urlsplit() Patrick Mezard <pmezard@gmail.com> parents: 7284 diff changeset	489 scheme = m.group(1).lower()
7270 2db33c1a5654 factor out the url handling from httprepo Benoit Boissinot <benoit.boissinot@ens-lyon.org> parents: diff changeset	490 if not scheme:
7284 ac81ffac0f35 url: fix file:// URL handling Patrick Mezard <pmezard@gmail.com> parents: 7270 diff changeset	491 path = util.normpath(os.path.abspath(url))
ac81ffac0f35 url: fix file:// URL handling Patrick Mezard <pmezard@gmail.com> parents: 7270 diff changeset	492 url = 'file://' + urllib.pathname2url(path)
ac81ffac0f35 url: fix file:// URL handling Patrick Mezard <pmezard@gmail.com> parents: 7270 diff changeset	493 authinfo = None
7270 2db33c1a5654 factor out the url handling from httprepo Benoit Boissinot <benoit.boissinot@ens-lyon.org> parents: diff changeset	494 else:
2db33c1a5654 factor out the url handling from httprepo Benoit Boissinot <benoit.boissinot@ens-lyon.org> parents: diff changeset	495 url, authinfo = getauthinfo(url)
2db33c1a5654 factor out the url handling from httprepo Benoit Boissinot <benoit.boissinot@ens-lyon.org> parents: diff changeset	496 return opener(ui, authinfo).open(url, data)

Mercurial > public > mercurial-scm > hg-stable

annotate mercurial/url.py @ 8590:59acb9c7d90f