mercurial-scm/hg-stable: mercurial/url.py annotate

annotate mercurial/url.py @ 8847:7951f385fcb7

url: support client certificate files over HTTPS (issue643) This extends the httpshandler with the means to utilise the auth section to provide it with a PEM encoded certificate key file and certificate chain file. This works also with sites that both require client certificate authentication and basic or digest password authentication, although the latter situation may require the user to enter the PEM password multiple times.

author	Henrik Stuart <hg@hstuart.dk>
date	Sat, 20 Jun 2009 10:58:57 +0200
parents	59acb9c7d90f
children	89b71acdac9a

rev	line source
7270 2db33c1a5654 factor out the url handling from httprepo Benoit Boissinot <benoit.boissinot@ens-lyon.org> parents: diff changeset	1 # url.py - HTTP handling for mercurial
2db33c1a5654 factor out the url handling from httprepo Benoit Boissinot <benoit.boissinot@ens-lyon.org> parents: diff changeset	2 #
2db33c1a5654 factor out the url handling from httprepo Benoit Boissinot <benoit.boissinot@ens-lyon.org> parents: diff changeset	3 # Copyright 2005, 2006, 2007, 2008 Matt Mackall <mpm@selenic.com>
2db33c1a5654 factor out the url handling from httprepo Benoit Boissinot <benoit.boissinot@ens-lyon.org> parents: diff changeset	4 # Copyright 2006, 2007 Alexis S. L. Carvalho <alexis@cecm.usp.br>
2db33c1a5654 factor out the url handling from httprepo Benoit Boissinot <benoit.boissinot@ens-lyon.org> parents: diff changeset	5 # Copyright 2006 Vadim Gelfer <vadim.gelfer@gmail.com>
2db33c1a5654 factor out the url handling from httprepo Benoit Boissinot <benoit.boissinot@ens-lyon.org> parents: diff changeset	6 #
8225 46293a0c7e9f updated license to be explicit about GPL version 2 Martin Geisler <mg@lazybytes.net> parents: 8208 diff changeset	7 # This software may be used and distributed according to the terms of the
46293a0c7e9f updated license to be explicit about GPL version 2 Martin Geisler <mg@lazybytes.net> parents: 8208 diff changeset	8 # GNU General Public License version 2, incorporated herein by reference.
7270 2db33c1a5654 factor out the url handling from httprepo Benoit Boissinot <benoit.boissinot@ens-lyon.org> parents: diff changeset	9
8590 59acb9c7d90f url: use CONNECT for HTTPS connections through HTTP proxy (issue967) Henrik Stuart <hg@hstuart.dk> parents: 8344 diff changeset	10 import urllib, urllib2, urlparse, httplib, os, re, socket, cStringIO
7270 2db33c1a5654 factor out the url handling from httprepo Benoit Boissinot <benoit.boissinot@ens-lyon.org> parents: diff changeset	11 from i18n import _
2db33c1a5654 factor out the url handling from httprepo Benoit Boissinot <benoit.boissinot@ens-lyon.org> parents: diff changeset	12 import keepalive, util
2db33c1a5654 factor out the url handling from httprepo Benoit Boissinot <benoit.boissinot@ens-lyon.org> parents: diff changeset	13
2db33c1a5654 factor out the url handling from httprepo Benoit Boissinot <benoit.boissinot@ens-lyon.org> parents: diff changeset	14 def hidepassword(url):
2db33c1a5654 factor out the url handling from httprepo Benoit Boissinot <benoit.boissinot@ens-lyon.org> parents: diff changeset	15 '''hide user credential in a url string'''
2db33c1a5654 factor out the url handling from httprepo Benoit Boissinot <benoit.boissinot@ens-lyon.org> parents: diff changeset	16 scheme, netloc, path, params, query, fragment = urlparse.urlparse(url)
2db33c1a5654 factor out the url handling from httprepo Benoit Boissinot <benoit.boissinot@ens-lyon.org> parents: diff changeset	17 netloc = re.sub('([^:]):([^@])@(.)', r'\1:**@\3', netloc)
2db33c1a5654 factor out the url handling from httprepo Benoit Boissinot <benoit.boissinot@ens-lyon.org> parents: diff changeset	18 return urlparse.urlunparse((scheme, netloc, path, params, query, fragment))
2db33c1a5654 factor out the url handling from httprepo Benoit Boissinot <benoit.boissinot@ens-lyon.org> parents: diff changeset	19
2db33c1a5654 factor out the url handling from httprepo Benoit Boissinot <benoit.boissinot@ens-lyon.org> parents: diff changeset	20 def removeauth(url):
2db33c1a5654 factor out the url handling from httprepo Benoit Boissinot <benoit.boissinot@ens-lyon.org> parents: diff changeset	21 '''remove all authentication information from a url string'''
2db33c1a5654 factor out the url handling from httprepo Benoit Boissinot <benoit.boissinot@ens-lyon.org> parents: diff changeset	22 scheme, netloc, path, params, query, fragment = urlparse.urlparse(url)
2db33c1a5654 factor out the url handling from httprepo Benoit Boissinot <benoit.boissinot@ens-lyon.org> parents: diff changeset	23 netloc = netloc[netloc.find('@')+1:]
2db33c1a5654 factor out the url handling from httprepo Benoit Boissinot <benoit.boissinot@ens-lyon.org> parents: diff changeset	24 return urlparse.urlunparse((scheme, netloc, path, params, query, fragment))
2db33c1a5654 factor out the url handling from httprepo Benoit Boissinot <benoit.boissinot@ens-lyon.org> parents: diff changeset	25
2db33c1a5654 factor out the url handling from httprepo Benoit Boissinot <benoit.boissinot@ens-lyon.org> parents: diff changeset	26 def netlocsplit(netloc):
2db33c1a5654 factor out the url handling from httprepo Benoit Boissinot <benoit.boissinot@ens-lyon.org> parents: diff changeset	27 '''split [user[:passwd]@]host[:port] into 4-tuple.'''
2db33c1a5654 factor out the url handling from httprepo Benoit Boissinot <benoit.boissinot@ens-lyon.org> parents: diff changeset	28
2db33c1a5654 factor out the url handling from httprepo Benoit Boissinot <benoit.boissinot@ens-lyon.org> parents: diff changeset	29 a = netloc.find('@')
2db33c1a5654 factor out the url handling from httprepo Benoit Boissinot <benoit.boissinot@ens-lyon.org> parents: diff changeset	30 if a == -1:
2db33c1a5654 factor out the url handling from httprepo Benoit Boissinot <benoit.boissinot@ens-lyon.org> parents: diff changeset	31 user, passwd = None, None
2db33c1a5654 factor out the url handling from httprepo Benoit Boissinot <benoit.boissinot@ens-lyon.org> parents: diff changeset	32 else:
2db33c1a5654 factor out the url handling from httprepo Benoit Boissinot <benoit.boissinot@ens-lyon.org> parents: diff changeset	33 userpass, netloc = netloc[:a], netloc[a+1:]
2db33c1a5654 factor out the url handling from httprepo Benoit Boissinot <benoit.boissinot@ens-lyon.org> parents: diff changeset	34 c = userpass.find(':')
2db33c1a5654 factor out the url handling from httprepo Benoit Boissinot <benoit.boissinot@ens-lyon.org> parents: diff changeset	35 if c == -1:
2db33c1a5654 factor out the url handling from httprepo Benoit Boissinot <benoit.boissinot@ens-lyon.org> parents: diff changeset	36 user, passwd = urllib.unquote(userpass), None
2db33c1a5654 factor out the url handling from httprepo Benoit Boissinot <benoit.boissinot@ens-lyon.org> parents: diff changeset	37 else:
2db33c1a5654 factor out the url handling from httprepo Benoit Boissinot <benoit.boissinot@ens-lyon.org> parents: diff changeset	38 user = urllib.unquote(userpass[:c])
2db33c1a5654 factor out the url handling from httprepo Benoit Boissinot <benoit.boissinot@ens-lyon.org> parents: diff changeset	39 passwd = urllib.unquote(userpass[c+1:])
2db33c1a5654 factor out the url handling from httprepo Benoit Boissinot <benoit.boissinot@ens-lyon.org> parents: diff changeset	40 c = netloc.find(':')
2db33c1a5654 factor out the url handling from httprepo Benoit Boissinot <benoit.boissinot@ens-lyon.org> parents: diff changeset	41 if c == -1:
2db33c1a5654 factor out the url handling from httprepo Benoit Boissinot <benoit.boissinot@ens-lyon.org> parents: diff changeset	42 host, port = netloc, None
2db33c1a5654 factor out the url handling from httprepo Benoit Boissinot <benoit.boissinot@ens-lyon.org> parents: diff changeset	43 else:
2db33c1a5654 factor out the url handling from httprepo Benoit Boissinot <benoit.boissinot@ens-lyon.org> parents: diff changeset	44 host, port = netloc[:c], netloc[c+1:]
2db33c1a5654 factor out the url handling from httprepo Benoit Boissinot <benoit.boissinot@ens-lyon.org> parents: diff changeset	45 return host, port, user, passwd
2db33c1a5654 factor out the url handling from httprepo Benoit Boissinot <benoit.boissinot@ens-lyon.org> parents: diff changeset	46
2db33c1a5654 factor out the url handling from httprepo Benoit Boissinot <benoit.boissinot@ens-lyon.org> parents: diff changeset	47 def netlocunsplit(host, port, user=None, passwd=None):
2db33c1a5654 factor out the url handling from httprepo Benoit Boissinot <benoit.boissinot@ens-lyon.org> parents: diff changeset	48 '''turn host, port, user, passwd into [user[:passwd]@]host[:port].'''
2db33c1a5654 factor out the url handling from httprepo Benoit Boissinot <benoit.boissinot@ens-lyon.org> parents: diff changeset	49 if port:
2db33c1a5654 factor out the url handling from httprepo Benoit Boissinot <benoit.boissinot@ens-lyon.org> parents: diff changeset	50 hostport = host + ':' + port
2db33c1a5654 factor out the url handling from httprepo Benoit Boissinot <benoit.boissinot@ens-lyon.org> parents: diff changeset	51 else:
2db33c1a5654 factor out the url handling from httprepo Benoit Boissinot <benoit.boissinot@ens-lyon.org> parents: diff changeset	52 hostport = host
2db33c1a5654 factor out the url handling from httprepo Benoit Boissinot <benoit.boissinot@ens-lyon.org> parents: diff changeset	53 if user:
2db33c1a5654 factor out the url handling from httprepo Benoit Boissinot <benoit.boissinot@ens-lyon.org> parents: diff changeset	54 if passwd:
2db33c1a5654 factor out the url handling from httprepo Benoit Boissinot <benoit.boissinot@ens-lyon.org> parents: diff changeset	55 userpass = urllib.quote(user) + ':' + urllib.quote(passwd)
2db33c1a5654 factor out the url handling from httprepo Benoit Boissinot <benoit.boissinot@ens-lyon.org> parents: diff changeset	56 else:
2db33c1a5654 factor out the url handling from httprepo Benoit Boissinot <benoit.boissinot@ens-lyon.org> parents: diff changeset	57 userpass = urllib.quote(user)
2db33c1a5654 factor out the url handling from httprepo Benoit Boissinot <benoit.boissinot@ens-lyon.org> parents: diff changeset	58 return userpass + '@' + hostport
2db33c1a5654 factor out the url handling from httprepo Benoit Boissinot <benoit.boissinot@ens-lyon.org> parents: diff changeset	59 return hostport
2db33c1a5654 factor out the url handling from httprepo Benoit Boissinot <benoit.boissinot@ens-lyon.org> parents: diff changeset	60
2db33c1a5654 factor out the url handling from httprepo Benoit Boissinot <benoit.boissinot@ens-lyon.org> parents: diff changeset	61 _safe = ('abcdefghijklmnopqrstuvwxyz'
2db33c1a5654 factor out the url handling from httprepo Benoit Boissinot <benoit.boissinot@ens-lyon.org> parents: diff changeset	62 'ABCDEFGHIJKLMNOPQRSTUVWXYZ'
2db33c1a5654 factor out the url handling from httprepo Benoit Boissinot <benoit.boissinot@ens-lyon.org> parents: diff changeset	63 '0123456789' '_.-/')
2db33c1a5654 factor out the url handling from httprepo Benoit Boissinot <benoit.boissinot@ens-lyon.org> parents: diff changeset	64 _safeset = None
2db33c1a5654 factor out the url handling from httprepo Benoit Boissinot <benoit.boissinot@ens-lyon.org> parents: diff changeset	65 _hex = None
2db33c1a5654 factor out the url handling from httprepo Benoit Boissinot <benoit.boissinot@ens-lyon.org> parents: diff changeset	66 def quotepath(path):
2db33c1a5654 factor out the url handling from httprepo Benoit Boissinot <benoit.boissinot@ens-lyon.org> parents: diff changeset	67 '''quote the path part of a URL
2db33c1a5654 factor out the url handling from httprepo Benoit Boissinot <benoit.boissinot@ens-lyon.org> parents: diff changeset	68
2db33c1a5654 factor out the url handling from httprepo Benoit Boissinot <benoit.boissinot@ens-lyon.org> parents: diff changeset	69 This is similar to urllib.quote, but it also tries to avoid
2db33c1a5654 factor out the url handling from httprepo Benoit Boissinot <benoit.boissinot@ens-lyon.org> parents: diff changeset	70 quoting things twice (inspired by wget):
2db33c1a5654 factor out the url handling from httprepo Benoit Boissinot <benoit.boissinot@ens-lyon.org> parents: diff changeset	71
2db33c1a5654 factor out the url handling from httprepo Benoit Boissinot <benoit.boissinot@ens-lyon.org> parents: diff changeset	72 >>> quotepath('abc def')
2db33c1a5654 factor out the url handling from httprepo Benoit Boissinot <benoit.boissinot@ens-lyon.org> parents: diff changeset	73 'abc%20def'
2db33c1a5654 factor out the url handling from httprepo Benoit Boissinot <benoit.boissinot@ens-lyon.org> parents: diff changeset	74 >>> quotepath('abc%20def')
2db33c1a5654 factor out the url handling from httprepo Benoit Boissinot <benoit.boissinot@ens-lyon.org> parents: diff changeset	75 'abc%20def'
2db33c1a5654 factor out the url handling from httprepo Benoit Boissinot <benoit.boissinot@ens-lyon.org> parents: diff changeset	76 >>> quotepath('abc%20 def')
2db33c1a5654 factor out the url handling from httprepo Benoit Boissinot <benoit.boissinot@ens-lyon.org> parents: diff changeset	77 'abc%20%20def'
2db33c1a5654 factor out the url handling from httprepo Benoit Boissinot <benoit.boissinot@ens-lyon.org> parents: diff changeset	78 >>> quotepath('abc def%20')
2db33c1a5654 factor out the url handling from httprepo Benoit Boissinot <benoit.boissinot@ens-lyon.org> parents: diff changeset	79 'abc%20def%20'
2db33c1a5654 factor out the url handling from httprepo Benoit Boissinot <benoit.boissinot@ens-lyon.org> parents: diff changeset	80 >>> quotepath('abc def%2')
2db33c1a5654 factor out the url handling from httprepo Benoit Boissinot <benoit.boissinot@ens-lyon.org> parents: diff changeset	81 'abc%20def%252'
2db33c1a5654 factor out the url handling from httprepo Benoit Boissinot <benoit.boissinot@ens-lyon.org> parents: diff changeset	82 >>> quotepath('abc def%')
2db33c1a5654 factor out the url handling from httprepo Benoit Boissinot <benoit.boissinot@ens-lyon.org> parents: diff changeset	83 'abc%20def%25'
2db33c1a5654 factor out the url handling from httprepo Benoit Boissinot <benoit.boissinot@ens-lyon.org> parents: diff changeset	84 '''
2db33c1a5654 factor out the url handling from httprepo Benoit Boissinot <benoit.boissinot@ens-lyon.org> parents: diff changeset	85 global _safeset, _hex
2db33c1a5654 factor out the url handling from httprepo Benoit Boissinot <benoit.boissinot@ens-lyon.org> parents: diff changeset	86 if _safeset is None:
8150 bbc24c0753a0 util: use built-in set and frozenset Martin Geisler <mg@lazybytes.net> parents: 7285 diff changeset	87 _safeset = set(_safe)
bbc24c0753a0 util: use built-in set and frozenset Martin Geisler <mg@lazybytes.net> parents: 7285 diff changeset	88 _hex = set('abcdefABCDEF0123456789')
7270 2db33c1a5654 factor out the url handling from httprepo Benoit Boissinot <benoit.boissinot@ens-lyon.org> parents: diff changeset	89 l = list(path)
2db33c1a5654 factor out the url handling from httprepo Benoit Boissinot <benoit.boissinot@ens-lyon.org> parents: diff changeset	90 for i in xrange(len(l)):
2db33c1a5654 factor out the url handling from httprepo Benoit Boissinot <benoit.boissinot@ens-lyon.org> parents: diff changeset	91 c = l[i]
2db33c1a5654 factor out the url handling from httprepo Benoit Boissinot <benoit.boissinot@ens-lyon.org> parents: diff changeset	92 if c == '%' and i + 2 < len(l) and (l[i+1] in _hex and l[i+2] in _hex):
2db33c1a5654 factor out the url handling from httprepo Benoit Boissinot <benoit.boissinot@ens-lyon.org> parents: diff changeset	93 pass
2db33c1a5654 factor out the url handling from httprepo Benoit Boissinot <benoit.boissinot@ens-lyon.org> parents: diff changeset	94 elif c not in _safeset:
2db33c1a5654 factor out the url handling from httprepo Benoit Boissinot <benoit.boissinot@ens-lyon.org> parents: diff changeset	95 l[i] = '%%%02X' % ord(c)
2db33c1a5654 factor out the url handling from httprepo Benoit Boissinot <benoit.boissinot@ens-lyon.org> parents: diff changeset	96 return ''.join(l)
2db33c1a5654 factor out the url handling from httprepo Benoit Boissinot <benoit.boissinot@ens-lyon.org> parents: diff changeset	97
2db33c1a5654 factor out the url handling from httprepo Benoit Boissinot <benoit.boissinot@ens-lyon.org> parents: diff changeset	98 class passwordmgr(urllib2.HTTPPasswordMgrWithDefaultRealm):
2db33c1a5654 factor out the url handling from httprepo Benoit Boissinot <benoit.boissinot@ens-lyon.org> parents: diff changeset	99 def __init__(self, ui):
2db33c1a5654 factor out the url handling from httprepo Benoit Boissinot <benoit.boissinot@ens-lyon.org> parents: diff changeset	100 urllib2.HTTPPasswordMgrWithDefaultRealm.__init__(self)
2db33c1a5654 factor out the url handling from httprepo Benoit Boissinot <benoit.boissinot@ens-lyon.org> parents: diff changeset	101 self.ui = ui
2db33c1a5654 factor out the url handling from httprepo Benoit Boissinot <benoit.boissinot@ens-lyon.org> parents: diff changeset	102
2db33c1a5654 factor out the url handling from httprepo Benoit Boissinot <benoit.boissinot@ens-lyon.org> parents: diff changeset	103 def find_user_password(self, realm, authuri):
2db33c1a5654 factor out the url handling from httprepo Benoit Boissinot <benoit.boissinot@ens-lyon.org> parents: diff changeset	104 authinfo = urllib2.HTTPPasswordMgrWithDefaultRealm.find_user_password(
2db33c1a5654 factor out the url handling from httprepo Benoit Boissinot <benoit.boissinot@ens-lyon.org> parents: diff changeset	105 self, realm, authuri)
2db33c1a5654 factor out the url handling from httprepo Benoit Boissinot <benoit.boissinot@ens-lyon.org> parents: diff changeset	106 user, passwd = authinfo
2db33c1a5654 factor out the url handling from httprepo Benoit Boissinot <benoit.boissinot@ens-lyon.org> parents: diff changeset	107 if user and passwd:
8333 89c80c3dc584 allow http authentication information to be specified in the configuration Sune Foldager <cryo@cyanite.org> parents: 8225 diff changeset	108 self._writedebug(user, passwd)
7270 2db33c1a5654 factor out the url handling from httprepo Benoit Boissinot <benoit.boissinot@ens-lyon.org> parents: diff changeset	109 return (user, passwd)
2db33c1a5654 factor out the url handling from httprepo Benoit Boissinot <benoit.boissinot@ens-lyon.org> parents: diff changeset	110
8344 873429914ec5 url: fix bug in passwordmgr related to auth configuration Sune Foldager <cryo@cyanite.org> parents: 8333 diff changeset	111 if not user:
8847 7951f385fcb7 url: support client certificate files over HTTPS (issue643) Henrik Stuart <hg@hstuart.dk> parents: 8590 diff changeset	112 auth = self.readauthtoken(authuri)
7951f385fcb7 url: support client certificate files over HTTPS (issue643) Henrik Stuart <hg@hstuart.dk> parents: 8590 diff changeset	113 if auth:
7951f385fcb7 url: support client certificate files over HTTPS (issue643) Henrik Stuart <hg@hstuart.dk> parents: 8590 diff changeset	114 user, passwd = auth.get('username'), auth.get('password')
8333 89c80c3dc584 allow http authentication information to be specified in the configuration Sune Foldager <cryo@cyanite.org> parents: 8225 diff changeset	115 if not user or not passwd:
89c80c3dc584 allow http authentication information to be specified in the configuration Sune Foldager <cryo@cyanite.org> parents: 8225 diff changeset	116 if not self.ui.interactive():
89c80c3dc584 allow http authentication information to be specified in the configuration Sune Foldager <cryo@cyanite.org> parents: 8225 diff changeset	117 raise util.Abort(_('http authorization required'))
7270 2db33c1a5654 factor out the url handling from httprepo Benoit Boissinot <benoit.boissinot@ens-lyon.org> parents: diff changeset	118
8333 89c80c3dc584 allow http authentication information to be specified in the configuration Sune Foldager <cryo@cyanite.org> parents: 8225 diff changeset	119 self.ui.write(_("http authorization required\n"))
89c80c3dc584 allow http authentication information to be specified in the configuration Sune Foldager <cryo@cyanite.org> parents: 8225 diff changeset	120 self.ui.status(_("realm: %s\n") % realm)
89c80c3dc584 allow http authentication information to be specified in the configuration Sune Foldager <cryo@cyanite.org> parents: 8225 diff changeset	121 if user:
89c80c3dc584 allow http authentication information to be specified in the configuration Sune Foldager <cryo@cyanite.org> parents: 8225 diff changeset	122 self.ui.status(_("user: %s\n") % user)
89c80c3dc584 allow http authentication information to be specified in the configuration Sune Foldager <cryo@cyanite.org> parents: 8225 diff changeset	123 else:
89c80c3dc584 allow http authentication information to be specified in the configuration Sune Foldager <cryo@cyanite.org> parents: 8225 diff changeset	124 user = self.ui.prompt(_("user:"), default=None)
7270 2db33c1a5654 factor out the url handling from httprepo Benoit Boissinot <benoit.boissinot@ens-lyon.org> parents: diff changeset	125
8333 89c80c3dc584 allow http authentication information to be specified in the configuration Sune Foldager <cryo@cyanite.org> parents: 8225 diff changeset	126 if not passwd:
89c80c3dc584 allow http authentication information to be specified in the configuration Sune Foldager <cryo@cyanite.org> parents: 8225 diff changeset	127 passwd = self.ui.getpass()
7270 2db33c1a5654 factor out the url handling from httprepo Benoit Boissinot <benoit.boissinot@ens-lyon.org> parents: diff changeset	128
2db33c1a5654 factor out the url handling from httprepo Benoit Boissinot <benoit.boissinot@ens-lyon.org> parents: diff changeset	129 self.add_password(realm, authuri, user, passwd)
8333 89c80c3dc584 allow http authentication information to be specified in the configuration Sune Foldager <cryo@cyanite.org> parents: 8225 diff changeset	130 self._writedebug(user, passwd)
7270 2db33c1a5654 factor out the url handling from httprepo Benoit Boissinot <benoit.boissinot@ens-lyon.org> parents: diff changeset	131 return (user, passwd)
2db33c1a5654 factor out the url handling from httprepo Benoit Boissinot <benoit.boissinot@ens-lyon.org> parents: diff changeset	132
8333 89c80c3dc584 allow http authentication information to be specified in the configuration Sune Foldager <cryo@cyanite.org> parents: 8225 diff changeset	133 def _writedebug(self, user, passwd):
89c80c3dc584 allow http authentication information to be specified in the configuration Sune Foldager <cryo@cyanite.org> parents: 8225 diff changeset	134 msg = _('http auth: user %s, password %s\n')
89c80c3dc584 allow http authentication information to be specified in the configuration Sune Foldager <cryo@cyanite.org> parents: 8225 diff changeset	135 self.ui.debug(msg % (user, passwd and '' len(passwd) or 'not set'))
89c80c3dc584 allow http authentication information to be specified in the configuration Sune Foldager <cryo@cyanite.org> parents: 8225 diff changeset	136
8847 7951f385fcb7 url: support client certificate files over HTTPS (issue643) Henrik Stuart <hg@hstuart.dk> parents: 8590 diff changeset	137 def readauthtoken(self, uri):
8333 89c80c3dc584 allow http authentication information to be specified in the configuration Sune Foldager <cryo@cyanite.org> parents: 8225 diff changeset	138 # Read configuration
89c80c3dc584 allow http authentication information to be specified in the configuration Sune Foldager <cryo@cyanite.org> parents: 8225 diff changeset	139 config = dict()
89c80c3dc584 allow http authentication information to be specified in the configuration Sune Foldager <cryo@cyanite.org> parents: 8225 diff changeset	140 for key, val in self.ui.configitems('auth'):
89c80c3dc584 allow http authentication information to be specified in the configuration Sune Foldager <cryo@cyanite.org> parents: 8225 diff changeset	141 group, setting = key.split('.', 1)
89c80c3dc584 allow http authentication information to be specified in the configuration Sune Foldager <cryo@cyanite.org> parents: 8225 diff changeset	142 gdict = config.setdefault(group, dict())
89c80c3dc584 allow http authentication information to be specified in the configuration Sune Foldager <cryo@cyanite.org> parents: 8225 diff changeset	143 gdict[setting] = val
89c80c3dc584 allow http authentication information to be specified in the configuration Sune Foldager <cryo@cyanite.org> parents: 8225 diff changeset	144
89c80c3dc584 allow http authentication information to be specified in the configuration Sune Foldager <cryo@cyanite.org> parents: 8225 diff changeset	145 # Find the best match
89c80c3dc584 allow http authentication information to be specified in the configuration Sune Foldager <cryo@cyanite.org> parents: 8225 diff changeset	146 scheme, hostpath = uri.split('://', 1)
89c80c3dc584 allow http authentication information to be specified in the configuration Sune Foldager <cryo@cyanite.org> parents: 8225 diff changeset	147 bestlen = 0
8847 7951f385fcb7 url: support client certificate files over HTTPS (issue643) Henrik Stuart <hg@hstuart.dk> parents: 8590 diff changeset	148 bestauth = None
8333 89c80c3dc584 allow http authentication information to be specified in the configuration Sune Foldager <cryo@cyanite.org> parents: 8225 diff changeset	149 for auth in config.itervalues():
89c80c3dc584 allow http authentication information to be specified in the configuration Sune Foldager <cryo@cyanite.org> parents: 8225 diff changeset	150 prefix = auth.get('prefix')
89c80c3dc584 allow http authentication information to be specified in the configuration Sune Foldager <cryo@cyanite.org> parents: 8225 diff changeset	151 if not prefix: continue
89c80c3dc584 allow http authentication information to be specified in the configuration Sune Foldager <cryo@cyanite.org> parents: 8225 diff changeset	152 p = prefix.split('://', 1)
89c80c3dc584 allow http authentication information to be specified in the configuration Sune Foldager <cryo@cyanite.org> parents: 8225 diff changeset	153 if len(p) > 1:
89c80c3dc584 allow http authentication information to be specified in the configuration Sune Foldager <cryo@cyanite.org> parents: 8225 diff changeset	154 schemes, prefix = [p[0]], p[1]
89c80c3dc584 allow http authentication information to be specified in the configuration Sune Foldager <cryo@cyanite.org> parents: 8225 diff changeset	155 else:
89c80c3dc584 allow http authentication information to be specified in the configuration Sune Foldager <cryo@cyanite.org> parents: 8225 diff changeset	156 schemes = (auth.get('schemes') or 'https').split()
89c80c3dc584 allow http authentication information to be specified in the configuration Sune Foldager <cryo@cyanite.org> parents: 8225 diff changeset	157 if (prefix == '*' or hostpath.startswith(prefix)) and \
89c80c3dc584 allow http authentication information to be specified in the configuration Sune Foldager <cryo@cyanite.org> parents: 8225 diff changeset	158 len(prefix) > bestlen and scheme in schemes:
89c80c3dc584 allow http authentication information to be specified in the configuration Sune Foldager <cryo@cyanite.org> parents: 8225 diff changeset	159 bestlen = len(prefix)
8847 7951f385fcb7 url: support client certificate files over HTTPS (issue643) Henrik Stuart <hg@hstuart.dk> parents: 8590 diff changeset	160 bestauth = auth
8333 89c80c3dc584 allow http authentication information to be specified in the configuration Sune Foldager <cryo@cyanite.org> parents: 8225 diff changeset	161 return bestauth
89c80c3dc584 allow http authentication information to be specified in the configuration Sune Foldager <cryo@cyanite.org> parents: 8225 diff changeset	162
7270 2db33c1a5654 factor out the url handling from httprepo Benoit Boissinot <benoit.boissinot@ens-lyon.org> parents: diff changeset	163 class proxyhandler(urllib2.ProxyHandler):
2db33c1a5654 factor out the url handling from httprepo Benoit Boissinot <benoit.boissinot@ens-lyon.org> parents: diff changeset	164 def __init__(self, ui):
2db33c1a5654 factor out the url handling from httprepo Benoit Boissinot <benoit.boissinot@ens-lyon.org> parents: diff changeset	165 proxyurl = ui.config("http_proxy", "host") or os.getenv('http_proxy')
2db33c1a5654 factor out the url handling from httprepo Benoit Boissinot <benoit.boissinot@ens-lyon.org> parents: diff changeset	166 # XXX proxyauthinfo = None
2db33c1a5654 factor out the url handling from httprepo Benoit Boissinot <benoit.boissinot@ens-lyon.org> parents: diff changeset	167
2db33c1a5654 factor out the url handling from httprepo Benoit Boissinot <benoit.boissinot@ens-lyon.org> parents: diff changeset	168 if proxyurl:
2db33c1a5654 factor out the url handling from httprepo Benoit Boissinot <benoit.boissinot@ens-lyon.org> parents: diff changeset	169 # proxy can be proper url or host[:port]
2db33c1a5654 factor out the url handling from httprepo Benoit Boissinot <benoit.boissinot@ens-lyon.org> parents: diff changeset	170 if not (proxyurl.startswith('http:') or
2db33c1a5654 factor out the url handling from httprepo Benoit Boissinot <benoit.boissinot@ens-lyon.org> parents: diff changeset	171 proxyurl.startswith('https:')):
2db33c1a5654 factor out the url handling from httprepo Benoit Boissinot <benoit.boissinot@ens-lyon.org> parents: diff changeset	172 proxyurl = 'http://' + proxyurl + '/'
2db33c1a5654 factor out the url handling from httprepo Benoit Boissinot <benoit.boissinot@ens-lyon.org> parents: diff changeset	173 snpqf = urlparse.urlsplit(proxyurl)
2db33c1a5654 factor out the url handling from httprepo Benoit Boissinot <benoit.boissinot@ens-lyon.org> parents: diff changeset	174 proxyscheme, proxynetloc, proxypath, proxyquery, proxyfrag = snpqf
2db33c1a5654 factor out the url handling from httprepo Benoit Boissinot <benoit.boissinot@ens-lyon.org> parents: diff changeset	175 hpup = netlocsplit(proxynetloc)
2db33c1a5654 factor out the url handling from httprepo Benoit Boissinot <benoit.boissinot@ens-lyon.org> parents: diff changeset	176
2db33c1a5654 factor out the url handling from httprepo Benoit Boissinot <benoit.boissinot@ens-lyon.org> parents: diff changeset	177 proxyhost, proxyport, proxyuser, proxypasswd = hpup
2db33c1a5654 factor out the url handling from httprepo Benoit Boissinot <benoit.boissinot@ens-lyon.org> parents: diff changeset	178 if not proxyuser:
2db33c1a5654 factor out the url handling from httprepo Benoit Boissinot <benoit.boissinot@ens-lyon.org> parents: diff changeset	179 proxyuser = ui.config("http_proxy", "user")
2db33c1a5654 factor out the url handling from httprepo Benoit Boissinot <benoit.boissinot@ens-lyon.org> parents: diff changeset	180 proxypasswd = ui.config("http_proxy", "passwd")
2db33c1a5654 factor out the url handling from httprepo Benoit Boissinot <benoit.boissinot@ens-lyon.org> parents: diff changeset	181
2db33c1a5654 factor out the url handling from httprepo Benoit Boissinot <benoit.boissinot@ens-lyon.org> parents: diff changeset	182 # see if we should use a proxy for this url
2db33c1a5654 factor out the url handling from httprepo Benoit Boissinot <benoit.boissinot@ens-lyon.org> parents: diff changeset	183 no_list = [ "localhost", "127.0.0.1" ]
2db33c1a5654 factor out the url handling from httprepo Benoit Boissinot <benoit.boissinot@ens-lyon.org> parents: diff changeset	184 no_list.extend([p.lower() for
2db33c1a5654 factor out the url handling from httprepo Benoit Boissinot <benoit.boissinot@ens-lyon.org> parents: diff changeset	185 p in ui.configlist("http_proxy", "no")])
2db33c1a5654 factor out the url handling from httprepo Benoit Boissinot <benoit.boissinot@ens-lyon.org> parents: diff changeset	186 no_list.extend([p.strip().lower() for
2db33c1a5654 factor out the url handling from httprepo Benoit Boissinot <benoit.boissinot@ens-lyon.org> parents: diff changeset	187 p in os.getenv("no_proxy", '').split(',')
2db33c1a5654 factor out the url handling from httprepo Benoit Boissinot <benoit.boissinot@ens-lyon.org> parents: diff changeset	188 if p.strip()])
2db33c1a5654 factor out the url handling from httprepo Benoit Boissinot <benoit.boissinot@ens-lyon.org> parents: diff changeset	189 # "http_proxy.always" config is for running tests on localhost
2db33c1a5654 factor out the url handling from httprepo Benoit Boissinot <benoit.boissinot@ens-lyon.org> parents: diff changeset	190 if ui.configbool("http_proxy", "always"):
2db33c1a5654 factor out the url handling from httprepo Benoit Boissinot <benoit.boissinot@ens-lyon.org> parents: diff changeset	191 self.no_list = []
2db33c1a5654 factor out the url handling from httprepo Benoit Boissinot <benoit.boissinot@ens-lyon.org> parents: diff changeset	192 else:
2db33c1a5654 factor out the url handling from httprepo Benoit Boissinot <benoit.boissinot@ens-lyon.org> parents: diff changeset	193 self.no_list = no_list
2db33c1a5654 factor out the url handling from httprepo Benoit Boissinot <benoit.boissinot@ens-lyon.org> parents: diff changeset	194
2db33c1a5654 factor out the url handling from httprepo Benoit Boissinot <benoit.boissinot@ens-lyon.org> parents: diff changeset	195 proxyurl = urlparse.urlunsplit((
2db33c1a5654 factor out the url handling from httprepo Benoit Boissinot <benoit.boissinot@ens-lyon.org> parents: diff changeset	196 proxyscheme, netlocunsplit(proxyhost, proxyport,
2db33c1a5654 factor out the url handling from httprepo Benoit Boissinot <benoit.boissinot@ens-lyon.org> parents: diff changeset	197 proxyuser, proxypasswd or ''),
2db33c1a5654 factor out the url handling from httprepo Benoit Boissinot <benoit.boissinot@ens-lyon.org> parents: diff changeset	198 proxypath, proxyquery, proxyfrag))
2db33c1a5654 factor out the url handling from httprepo Benoit Boissinot <benoit.boissinot@ens-lyon.org> parents: diff changeset	199 proxies = {'http': proxyurl, 'https': proxyurl}
2db33c1a5654 factor out the url handling from httprepo Benoit Boissinot <benoit.boissinot@ens-lyon.org> parents: diff changeset	200 ui.debug(_('proxying through http://%s:%s\n') %
2db33c1a5654 factor out the url handling from httprepo Benoit Boissinot <benoit.boissinot@ens-lyon.org> parents: diff changeset	201 (proxyhost, proxyport))
2db33c1a5654 factor out the url handling from httprepo Benoit Boissinot <benoit.boissinot@ens-lyon.org> parents: diff changeset	202 else:
2db33c1a5654 factor out the url handling from httprepo Benoit Boissinot <benoit.boissinot@ens-lyon.org> parents: diff changeset	203 proxies = {}
2db33c1a5654 factor out the url handling from httprepo Benoit Boissinot <benoit.boissinot@ens-lyon.org> parents: diff changeset	204
2db33c1a5654 factor out the url handling from httprepo Benoit Boissinot <benoit.boissinot@ens-lyon.org> parents: diff changeset	205 # urllib2 takes proxy values from the environment and those
2db33c1a5654 factor out the url handling from httprepo Benoit Boissinot <benoit.boissinot@ens-lyon.org> parents: diff changeset	206 # will take precedence if found, so drop them
2db33c1a5654 factor out the url handling from httprepo Benoit Boissinot <benoit.boissinot@ens-lyon.org> parents: diff changeset	207 for env in ["HTTP_PROXY", "http_proxy", "no_proxy"]:
2db33c1a5654 factor out the url handling from httprepo Benoit Boissinot <benoit.boissinot@ens-lyon.org> parents: diff changeset	208 try:
2db33c1a5654 factor out the url handling from httprepo Benoit Boissinot <benoit.boissinot@ens-lyon.org> parents: diff changeset	209 if env in os.environ:
2db33c1a5654 factor out the url handling from httprepo Benoit Boissinot <benoit.boissinot@ens-lyon.org> parents: diff changeset	210 del os.environ[env]
2db33c1a5654 factor out the url handling from httprepo Benoit Boissinot <benoit.boissinot@ens-lyon.org> parents: diff changeset	211 except OSError:
2db33c1a5654 factor out the url handling from httprepo Benoit Boissinot <benoit.boissinot@ens-lyon.org> parents: diff changeset	212 pass
2db33c1a5654 factor out the url handling from httprepo Benoit Boissinot <benoit.boissinot@ens-lyon.org> parents: diff changeset	213
2db33c1a5654 factor out the url handling from httprepo Benoit Boissinot <benoit.boissinot@ens-lyon.org> parents: diff changeset	214 urllib2.ProxyHandler.__init__(self, proxies)
2db33c1a5654 factor out the url handling from httprepo Benoit Boissinot <benoit.boissinot@ens-lyon.org> parents: diff changeset	215 self.ui = ui
2db33c1a5654 factor out the url handling from httprepo Benoit Boissinot <benoit.boissinot@ens-lyon.org> parents: diff changeset	216
2db33c1a5654 factor out the url handling from httprepo Benoit Boissinot <benoit.boissinot@ens-lyon.org> parents: diff changeset	217 def proxy_open(self, req, proxy, type_):
2db33c1a5654 factor out the url handling from httprepo Benoit Boissinot <benoit.boissinot@ens-lyon.org> parents: diff changeset	218 host = req.get_host().split(':')[0]
2db33c1a5654 factor out the url handling from httprepo Benoit Boissinot <benoit.boissinot@ens-lyon.org> parents: diff changeset	219 if host in self.no_list:
2db33c1a5654 factor out the url handling from httprepo Benoit Boissinot <benoit.boissinot@ens-lyon.org> parents: diff changeset	220 return None
2db33c1a5654 factor out the url handling from httprepo Benoit Boissinot <benoit.boissinot@ens-lyon.org> parents: diff changeset	221
2db33c1a5654 factor out the url handling from httprepo Benoit Boissinot <benoit.boissinot@ens-lyon.org> parents: diff changeset	222 # work around a bug in Python < 2.4.2
2db33c1a5654 factor out the url handling from httprepo Benoit Boissinot <benoit.boissinot@ens-lyon.org> parents: diff changeset	223 # (it leaves a "\n" at the end of Proxy-authorization headers)
2db33c1a5654 factor out the url handling from httprepo Benoit Boissinot <benoit.boissinot@ens-lyon.org> parents: diff changeset	224 baseclass = req.__class__
2db33c1a5654 factor out the url handling from httprepo Benoit Boissinot <benoit.boissinot@ens-lyon.org> parents: diff changeset	225 class _request(baseclass):
2db33c1a5654 factor out the url handling from httprepo Benoit Boissinot <benoit.boissinot@ens-lyon.org> parents: diff changeset	226 def add_header(self, key, val):
2db33c1a5654 factor out the url handling from httprepo Benoit Boissinot <benoit.boissinot@ens-lyon.org> parents: diff changeset	227 if key.lower() == 'proxy-authorization':
2db33c1a5654 factor out the url handling from httprepo Benoit Boissinot <benoit.boissinot@ens-lyon.org> parents: diff changeset	228 val = val.strip()
2db33c1a5654 factor out the url handling from httprepo Benoit Boissinot <benoit.boissinot@ens-lyon.org> parents: diff changeset	229 return baseclass.add_header(self, key, val)
2db33c1a5654 factor out the url handling from httprepo Benoit Boissinot <benoit.boissinot@ens-lyon.org> parents: diff changeset	230 req.__class__ = _request
2db33c1a5654 factor out the url handling from httprepo Benoit Boissinot <benoit.boissinot@ens-lyon.org> parents: diff changeset	231
2db33c1a5654 factor out the url handling from httprepo Benoit Boissinot <benoit.boissinot@ens-lyon.org> parents: diff changeset	232 return urllib2.ProxyHandler.proxy_open(self, req, proxy, type_)
2db33c1a5654 factor out the url handling from httprepo Benoit Boissinot <benoit.boissinot@ens-lyon.org> parents: diff changeset	233
2db33c1a5654 factor out the url handling from httprepo Benoit Boissinot <benoit.boissinot@ens-lyon.org> parents: diff changeset	234 class httpsendfile(file):
2db33c1a5654 factor out the url handling from httprepo Benoit Boissinot <benoit.boissinot@ens-lyon.org> parents: diff changeset	235 def __len__(self):
2db33c1a5654 factor out the url handling from httprepo Benoit Boissinot <benoit.boissinot@ens-lyon.org> parents: diff changeset	236 return os.fstat(self.fileno()).st_size
2db33c1a5654 factor out the url handling from httprepo Benoit Boissinot <benoit.boissinot@ens-lyon.org> parents: diff changeset	237
2db33c1a5654 factor out the url handling from httprepo Benoit Boissinot <benoit.boissinot@ens-lyon.org> parents: diff changeset	238 def _gen_sendfile(connection):
2db33c1a5654 factor out the url handling from httprepo Benoit Boissinot <benoit.boissinot@ens-lyon.org> parents: diff changeset	239 def _sendfile(self, data):
2db33c1a5654 factor out the url handling from httprepo Benoit Boissinot <benoit.boissinot@ens-lyon.org> parents: diff changeset	240 # send a file
2db33c1a5654 factor out the url handling from httprepo Benoit Boissinot <benoit.boissinot@ens-lyon.org> parents: diff changeset	241 if isinstance(data, httpsendfile):
2db33c1a5654 factor out the url handling from httprepo Benoit Boissinot <benoit.boissinot@ens-lyon.org> parents: diff changeset	242 # if auth required, some data sent twice, so rewind here
2db33c1a5654 factor out the url handling from httprepo Benoit Boissinot <benoit.boissinot@ens-lyon.org> parents: diff changeset	243 data.seek(0)
2db33c1a5654 factor out the url handling from httprepo Benoit Boissinot <benoit.boissinot@ens-lyon.org> parents: diff changeset	244 for chunk in util.filechunkiter(data):
2db33c1a5654 factor out the url handling from httprepo Benoit Boissinot <benoit.boissinot@ens-lyon.org> parents: diff changeset	245 connection.send(self, chunk)
2db33c1a5654 factor out the url handling from httprepo Benoit Boissinot <benoit.boissinot@ens-lyon.org> parents: diff changeset	246 else:
2db33c1a5654 factor out the url handling from httprepo Benoit Boissinot <benoit.boissinot@ens-lyon.org> parents: diff changeset	247 connection.send(self, data)
2db33c1a5654 factor out the url handling from httprepo Benoit Boissinot <benoit.boissinot@ens-lyon.org> parents: diff changeset	248 return _sendfile
2db33c1a5654 factor out the url handling from httprepo Benoit Boissinot <benoit.boissinot@ens-lyon.org> parents: diff changeset	249
8590 59acb9c7d90f url: use CONNECT for HTTPS connections through HTTP proxy (issue967) Henrik Stuart <hg@hstuart.dk> parents: 8344 diff changeset	250 has_https = hasattr(urllib2, 'HTTPSHandler')
59acb9c7d90f url: use CONNECT for HTTPS connections through HTTP proxy (issue967) Henrik Stuart <hg@hstuart.dk> parents: 8344 diff changeset	251 if has_https:
59acb9c7d90f url: use CONNECT for HTTPS connections through HTTP proxy (issue967) Henrik Stuart <hg@hstuart.dk> parents: 8344 diff changeset	252 try:
59acb9c7d90f url: use CONNECT for HTTPS connections through HTTP proxy (issue967) Henrik Stuart <hg@hstuart.dk> parents: 8344 diff changeset	253 # avoid using deprecated/broken FakeSocket in python 2.6
59acb9c7d90f url: use CONNECT for HTTPS connections through HTTP proxy (issue967) Henrik Stuart <hg@hstuart.dk> parents: 8344 diff changeset	254 import ssl
59acb9c7d90f url: use CONNECT for HTTPS connections through HTTP proxy (issue967) Henrik Stuart <hg@hstuart.dk> parents: 8344 diff changeset	255 _ssl_wrap_socket = ssl.wrap_socket
59acb9c7d90f url: use CONNECT for HTTPS connections through HTTP proxy (issue967) Henrik Stuart <hg@hstuart.dk> parents: 8344 diff changeset	256 except ImportError:
59acb9c7d90f url: use CONNECT for HTTPS connections through HTTP proxy (issue967) Henrik Stuart <hg@hstuart.dk> parents: 8344 diff changeset	257 def _ssl_wrap_socket(sock, key_file, cert_file):
59acb9c7d90f url: use CONNECT for HTTPS connections through HTTP proxy (issue967) Henrik Stuart <hg@hstuart.dk> parents: 8344 diff changeset	258 ssl = socket.ssl(sock, key_file, cert_file)
59acb9c7d90f url: use CONNECT for HTTPS connections through HTTP proxy (issue967) Henrik Stuart <hg@hstuart.dk> parents: 8344 diff changeset	259 return httplib.FakeSocket(sock, ssl)
59acb9c7d90f url: use CONNECT for HTTPS connections through HTTP proxy (issue967) Henrik Stuart <hg@hstuart.dk> parents: 8344 diff changeset	260
7270 2db33c1a5654 factor out the url handling from httprepo Benoit Boissinot <benoit.boissinot@ens-lyon.org> parents: diff changeset	261 class httpconnection(keepalive.HTTPConnection):
2db33c1a5654 factor out the url handling from httprepo Benoit Boissinot <benoit.boissinot@ens-lyon.org> parents: diff changeset	262 # must be able to send big bundle as stream.
2db33c1a5654 factor out the url handling from httprepo Benoit Boissinot <benoit.boissinot@ens-lyon.org> parents: diff changeset	263 send = _gen_sendfile(keepalive.HTTPConnection)
2db33c1a5654 factor out the url handling from httprepo Benoit Boissinot <benoit.boissinot@ens-lyon.org> parents: diff changeset	264
8590 59acb9c7d90f url: use CONNECT for HTTPS connections through HTTP proxy (issue967) Henrik Stuart <hg@hstuart.dk> parents: 8344 diff changeset	265 def _proxytunnel(self):
59acb9c7d90f url: use CONNECT for HTTPS connections through HTTP proxy (issue967) Henrik Stuart <hg@hstuart.dk> parents: 8344 diff changeset	266 proxyheaders = dict(
59acb9c7d90f url: use CONNECT for HTTPS connections through HTTP proxy (issue967) Henrik Stuart <hg@hstuart.dk> parents: 8344 diff changeset	267 [(x, self.headers[x]) for x in self.headers
59acb9c7d90f url: use CONNECT for HTTPS connections through HTTP proxy (issue967) Henrik Stuart <hg@hstuart.dk> parents: 8344 diff changeset	268 if x.lower().startswith('proxy-')])
59acb9c7d90f url: use CONNECT for HTTPS connections through HTTP proxy (issue967) Henrik Stuart <hg@hstuart.dk> parents: 8344 diff changeset	269 self._set_hostport(self.host, self.port)
59acb9c7d90f url: use CONNECT for HTTPS connections through HTTP proxy (issue967) Henrik Stuart <hg@hstuart.dk> parents: 8344 diff changeset	270 self.send('CONNECT %s:%d HTTP/1.0\r\n' % (self.realhost, self.realport))
59acb9c7d90f url: use CONNECT for HTTPS connections through HTTP proxy (issue967) Henrik Stuart <hg@hstuart.dk> parents: 8344 diff changeset	271 for header in proxyheaders.iteritems():
59acb9c7d90f url: use CONNECT for HTTPS connections through HTTP proxy (issue967) Henrik Stuart <hg@hstuart.dk> parents: 8344 diff changeset	272 self.send('%s: %s\r\n' % header)
59acb9c7d90f url: use CONNECT for HTTPS connections through HTTP proxy (issue967) Henrik Stuart <hg@hstuart.dk> parents: 8344 diff changeset	273 self.send('\r\n')
59acb9c7d90f url: use CONNECT for HTTPS connections through HTTP proxy (issue967) Henrik Stuart <hg@hstuart.dk> parents: 8344 diff changeset	274
59acb9c7d90f url: use CONNECT for HTTPS connections through HTTP proxy (issue967) Henrik Stuart <hg@hstuart.dk> parents: 8344 diff changeset	275 # majority of the following code is duplicated from
59acb9c7d90f url: use CONNECT for HTTPS connections through HTTP proxy (issue967) Henrik Stuart <hg@hstuart.dk> parents: 8344 diff changeset	276 # httplib.HTTPConnection as there are no adequate places to
59acb9c7d90f url: use CONNECT for HTTPS connections through HTTP proxy (issue967) Henrik Stuart <hg@hstuart.dk> parents: 8344 diff changeset	277 # override functions to provide the needed functionality
59acb9c7d90f url: use CONNECT for HTTPS connections through HTTP proxy (issue967) Henrik Stuart <hg@hstuart.dk> parents: 8344 diff changeset	278 res = self.response_class(self.sock,
59acb9c7d90f url: use CONNECT for HTTPS connections through HTTP proxy (issue967) Henrik Stuart <hg@hstuart.dk> parents: 8344 diff changeset	279 strict=self.strict,
59acb9c7d90f url: use CONNECT for HTTPS connections through HTTP proxy (issue967) Henrik Stuart <hg@hstuart.dk> parents: 8344 diff changeset	280 method=self._method)
59acb9c7d90f url: use CONNECT for HTTPS connections through HTTP proxy (issue967) Henrik Stuart <hg@hstuart.dk> parents: 8344 diff changeset	281
59acb9c7d90f url: use CONNECT for HTTPS connections through HTTP proxy (issue967) Henrik Stuart <hg@hstuart.dk> parents: 8344 diff changeset	282 while True:
59acb9c7d90f url: use CONNECT for HTTPS connections through HTTP proxy (issue967) Henrik Stuart <hg@hstuart.dk> parents: 8344 diff changeset	283 version, status, reason = res._read_status()
59acb9c7d90f url: use CONNECT for HTTPS connections through HTTP proxy (issue967) Henrik Stuart <hg@hstuart.dk> parents: 8344 diff changeset	284 if status != httplib.CONTINUE:
59acb9c7d90f url: use CONNECT for HTTPS connections through HTTP proxy (issue967) Henrik Stuart <hg@hstuart.dk> parents: 8344 diff changeset	285 break
59acb9c7d90f url: use CONNECT for HTTPS connections through HTTP proxy (issue967) Henrik Stuart <hg@hstuart.dk> parents: 8344 diff changeset	286 while True:
59acb9c7d90f url: use CONNECT for HTTPS connections through HTTP proxy (issue967) Henrik Stuart <hg@hstuart.dk> parents: 8344 diff changeset	287 skip = res.fp.readline().strip()
59acb9c7d90f url: use CONNECT for HTTPS connections through HTTP proxy (issue967) Henrik Stuart <hg@hstuart.dk> parents: 8344 diff changeset	288 if not skip:
59acb9c7d90f url: use CONNECT for HTTPS connections through HTTP proxy (issue967) Henrik Stuart <hg@hstuart.dk> parents: 8344 diff changeset	289 break
59acb9c7d90f url: use CONNECT for HTTPS connections through HTTP proxy (issue967) Henrik Stuart <hg@hstuart.dk> parents: 8344 diff changeset	290 res.status = status
59acb9c7d90f url: use CONNECT for HTTPS connections through HTTP proxy (issue967) Henrik Stuart <hg@hstuart.dk> parents: 8344 diff changeset	291 res.reason = reason.strip()
59acb9c7d90f url: use CONNECT for HTTPS connections through HTTP proxy (issue967) Henrik Stuart <hg@hstuart.dk> parents: 8344 diff changeset	292
59acb9c7d90f url: use CONNECT for HTTPS connections through HTTP proxy (issue967) Henrik Stuart <hg@hstuart.dk> parents: 8344 diff changeset	293 if res.status == 200:
59acb9c7d90f url: use CONNECT for HTTPS connections through HTTP proxy (issue967) Henrik Stuart <hg@hstuart.dk> parents: 8344 diff changeset	294 while True:
59acb9c7d90f url: use CONNECT for HTTPS connections through HTTP proxy (issue967) Henrik Stuart <hg@hstuart.dk> parents: 8344 diff changeset	295 line = res.fp.readline()
59acb9c7d90f url: use CONNECT for HTTPS connections through HTTP proxy (issue967) Henrik Stuart <hg@hstuart.dk> parents: 8344 diff changeset	296 if line == '\r\n':
59acb9c7d90f url: use CONNECT for HTTPS connections through HTTP proxy (issue967) Henrik Stuart <hg@hstuart.dk> parents: 8344 diff changeset	297 break
59acb9c7d90f url: use CONNECT for HTTPS connections through HTTP proxy (issue967) Henrik Stuart <hg@hstuart.dk> parents: 8344 diff changeset	298 return True
59acb9c7d90f url: use CONNECT for HTTPS connections through HTTP proxy (issue967) Henrik Stuart <hg@hstuart.dk> parents: 8344 diff changeset	299
59acb9c7d90f url: use CONNECT for HTTPS connections through HTTP proxy (issue967) Henrik Stuart <hg@hstuart.dk> parents: 8344 diff changeset	300 if version == 'HTTP/1.0':
59acb9c7d90f url: use CONNECT for HTTPS connections through HTTP proxy (issue967) Henrik Stuart <hg@hstuart.dk> parents: 8344 diff changeset	301 res.version = 10
59acb9c7d90f url: use CONNECT for HTTPS connections through HTTP proxy (issue967) Henrik Stuart <hg@hstuart.dk> parents: 8344 diff changeset	302 elif version.startswith('HTTP/1.'):
59acb9c7d90f url: use CONNECT for HTTPS connections through HTTP proxy (issue967) Henrik Stuart <hg@hstuart.dk> parents: 8344 diff changeset	303 res.version = 11
59acb9c7d90f url: use CONNECT for HTTPS connections through HTTP proxy (issue967) Henrik Stuart <hg@hstuart.dk> parents: 8344 diff changeset	304 elif version == 'HTTP/0.9':
59acb9c7d90f url: use CONNECT for HTTPS connections through HTTP proxy (issue967) Henrik Stuart <hg@hstuart.dk> parents: 8344 diff changeset	305 res.version = 9
59acb9c7d90f url: use CONNECT for HTTPS connections through HTTP proxy (issue967) Henrik Stuart <hg@hstuart.dk> parents: 8344 diff changeset	306 else:
59acb9c7d90f url: use CONNECT for HTTPS connections through HTTP proxy (issue967) Henrik Stuart <hg@hstuart.dk> parents: 8344 diff changeset	307 raise httplib.UnknownProtocol(version)
59acb9c7d90f url: use CONNECT for HTTPS connections through HTTP proxy (issue967) Henrik Stuart <hg@hstuart.dk> parents: 8344 diff changeset	308
59acb9c7d90f url: use CONNECT for HTTPS connections through HTTP proxy (issue967) Henrik Stuart <hg@hstuart.dk> parents: 8344 diff changeset	309 if res.version == 9:
59acb9c7d90f url: use CONNECT for HTTPS connections through HTTP proxy (issue967) Henrik Stuart <hg@hstuart.dk> parents: 8344 diff changeset	310 res.length = None
59acb9c7d90f url: use CONNECT for HTTPS connections through HTTP proxy (issue967) Henrik Stuart <hg@hstuart.dk> parents: 8344 diff changeset	311 res.chunked = 0
59acb9c7d90f url: use CONNECT for HTTPS connections through HTTP proxy (issue967) Henrik Stuart <hg@hstuart.dk> parents: 8344 diff changeset	312 res.will_close = 1
59acb9c7d90f url: use CONNECT for HTTPS connections through HTTP proxy (issue967) Henrik Stuart <hg@hstuart.dk> parents: 8344 diff changeset	313 res.msg = httplib.HTTPMessage(cStringIO.StringIO())
59acb9c7d90f url: use CONNECT for HTTPS connections through HTTP proxy (issue967) Henrik Stuart <hg@hstuart.dk> parents: 8344 diff changeset	314 return False
59acb9c7d90f url: use CONNECT for HTTPS connections through HTTP proxy (issue967) Henrik Stuart <hg@hstuart.dk> parents: 8344 diff changeset	315
59acb9c7d90f url: use CONNECT for HTTPS connections through HTTP proxy (issue967) Henrik Stuart <hg@hstuart.dk> parents: 8344 diff changeset	316 res.msg = httplib.HTTPMessage(res.fp)
59acb9c7d90f url: use CONNECT for HTTPS connections through HTTP proxy (issue967) Henrik Stuart <hg@hstuart.dk> parents: 8344 diff changeset	317 res.msg.fp = None
59acb9c7d90f url: use CONNECT for HTTPS connections through HTTP proxy (issue967) Henrik Stuart <hg@hstuart.dk> parents: 8344 diff changeset	318
59acb9c7d90f url: use CONNECT for HTTPS connections through HTTP proxy (issue967) Henrik Stuart <hg@hstuart.dk> parents: 8344 diff changeset	319 # are we using the chunked-style of transfer encoding?
59acb9c7d90f url: use CONNECT for HTTPS connections through HTTP proxy (issue967) Henrik Stuart <hg@hstuart.dk> parents: 8344 diff changeset	320 trenc = res.msg.getheader('transfer-encoding')
59acb9c7d90f url: use CONNECT for HTTPS connections through HTTP proxy (issue967) Henrik Stuart <hg@hstuart.dk> parents: 8344 diff changeset	321 if trenc and trenc.lower() == "chunked":
59acb9c7d90f url: use CONNECT for HTTPS connections through HTTP proxy (issue967) Henrik Stuart <hg@hstuart.dk> parents: 8344 diff changeset	322 res.chunked = 1
59acb9c7d90f url: use CONNECT for HTTPS connections through HTTP proxy (issue967) Henrik Stuart <hg@hstuart.dk> parents: 8344 diff changeset	323 res.chunk_left = None
59acb9c7d90f url: use CONNECT for HTTPS connections through HTTP proxy (issue967) Henrik Stuart <hg@hstuart.dk> parents: 8344 diff changeset	324 else:
59acb9c7d90f url: use CONNECT for HTTPS connections through HTTP proxy (issue967) Henrik Stuart <hg@hstuart.dk> parents: 8344 diff changeset	325 res.chunked = 0
59acb9c7d90f url: use CONNECT for HTTPS connections through HTTP proxy (issue967) Henrik Stuart <hg@hstuart.dk> parents: 8344 diff changeset	326
59acb9c7d90f url: use CONNECT for HTTPS connections through HTTP proxy (issue967) Henrik Stuart <hg@hstuart.dk> parents: 8344 diff changeset	327 # will the connection close at the end of the response?
59acb9c7d90f url: use CONNECT for HTTPS connections through HTTP proxy (issue967) Henrik Stuart <hg@hstuart.dk> parents: 8344 diff changeset	328 res.will_close = res._check_close()
59acb9c7d90f url: use CONNECT for HTTPS connections through HTTP proxy (issue967) Henrik Stuart <hg@hstuart.dk> parents: 8344 diff changeset	329
59acb9c7d90f url: use CONNECT for HTTPS connections through HTTP proxy (issue967) Henrik Stuart <hg@hstuart.dk> parents: 8344 diff changeset	330 # do we have a Content-Length?
59acb9c7d90f url: use CONNECT for HTTPS connections through HTTP proxy (issue967) Henrik Stuart <hg@hstuart.dk> parents: 8344 diff changeset	331 # NOTE: RFC 2616, S4.4, #3 says we ignore this if tr_enc is "chunked"
59acb9c7d90f url: use CONNECT for HTTPS connections through HTTP proxy (issue967) Henrik Stuart <hg@hstuart.dk> parents: 8344 diff changeset	332 length = res.msg.getheader('content-length')
59acb9c7d90f url: use CONNECT for HTTPS connections through HTTP proxy (issue967) Henrik Stuart <hg@hstuart.dk> parents: 8344 diff changeset	333 if length and not res.chunked:
59acb9c7d90f url: use CONNECT for HTTPS connections through HTTP proxy (issue967) Henrik Stuart <hg@hstuart.dk> parents: 8344 diff changeset	334 try:
59acb9c7d90f url: use CONNECT for HTTPS connections through HTTP proxy (issue967) Henrik Stuart <hg@hstuart.dk> parents: 8344 diff changeset	335 res.length = int(length)
59acb9c7d90f url: use CONNECT for HTTPS connections through HTTP proxy (issue967) Henrik Stuart <hg@hstuart.dk> parents: 8344 diff changeset	336 except ValueError:
59acb9c7d90f url: use CONNECT for HTTPS connections through HTTP proxy (issue967) Henrik Stuart <hg@hstuart.dk> parents: 8344 diff changeset	337 res.length = None
59acb9c7d90f url: use CONNECT for HTTPS connections through HTTP proxy (issue967) Henrik Stuart <hg@hstuart.dk> parents: 8344 diff changeset	338 else:
59acb9c7d90f url: use CONNECT for HTTPS connections through HTTP proxy (issue967) Henrik Stuart <hg@hstuart.dk> parents: 8344 diff changeset	339 if res.length < 0: # ignore nonsensical negative lengths
59acb9c7d90f url: use CONNECT for HTTPS connections through HTTP proxy (issue967) Henrik Stuart <hg@hstuart.dk> parents: 8344 diff changeset	340 res.length = None
59acb9c7d90f url: use CONNECT for HTTPS connections through HTTP proxy (issue967) Henrik Stuart <hg@hstuart.dk> parents: 8344 diff changeset	341 else:
59acb9c7d90f url: use CONNECT for HTTPS connections through HTTP proxy (issue967) Henrik Stuart <hg@hstuart.dk> parents: 8344 diff changeset	342 res.length = None
59acb9c7d90f url: use CONNECT for HTTPS connections through HTTP proxy (issue967) Henrik Stuart <hg@hstuart.dk> parents: 8344 diff changeset	343
59acb9c7d90f url: use CONNECT for HTTPS connections through HTTP proxy (issue967) Henrik Stuart <hg@hstuart.dk> parents: 8344 diff changeset	344 # does the body have a fixed length? (of zero)
59acb9c7d90f url: use CONNECT for HTTPS connections through HTTP proxy (issue967) Henrik Stuart <hg@hstuart.dk> parents: 8344 diff changeset	345 if (status == httplib.NO_CONTENT or status == httplib.NOT_MODIFIED or
59acb9c7d90f url: use CONNECT for HTTPS connections through HTTP proxy (issue967) Henrik Stuart <hg@hstuart.dk> parents: 8344 diff changeset	346 100 <= status < 200 or # 1xx codes
59acb9c7d90f url: use CONNECT for HTTPS connections through HTTP proxy (issue967) Henrik Stuart <hg@hstuart.dk> parents: 8344 diff changeset	347 res._method == 'HEAD'):
59acb9c7d90f url: use CONNECT for HTTPS connections through HTTP proxy (issue967) Henrik Stuart <hg@hstuart.dk> parents: 8344 diff changeset	348 res.length = 0
59acb9c7d90f url: use CONNECT for HTTPS connections through HTTP proxy (issue967) Henrik Stuart <hg@hstuart.dk> parents: 8344 diff changeset	349
59acb9c7d90f url: use CONNECT for HTTPS connections through HTTP proxy (issue967) Henrik Stuart <hg@hstuart.dk> parents: 8344 diff changeset	350 # if the connection remains open, and we aren't using chunked, and
59acb9c7d90f url: use CONNECT for HTTPS connections through HTTP proxy (issue967) Henrik Stuart <hg@hstuart.dk> parents: 8344 diff changeset	351 # a content-length was not provided, then assume that the connection
59acb9c7d90f url: use CONNECT for HTTPS connections through HTTP proxy (issue967) Henrik Stuart <hg@hstuart.dk> parents: 8344 diff changeset	352 # WILL close.
59acb9c7d90f url: use CONNECT for HTTPS connections through HTTP proxy (issue967) Henrik Stuart <hg@hstuart.dk> parents: 8344 diff changeset	353 if (not res.will_close and
59acb9c7d90f url: use CONNECT for HTTPS connections through HTTP proxy (issue967) Henrik Stuart <hg@hstuart.dk> parents: 8344 diff changeset	354 not res.chunked and
59acb9c7d90f url: use CONNECT for HTTPS connections through HTTP proxy (issue967) Henrik Stuart <hg@hstuart.dk> parents: 8344 diff changeset	355 res.length is None):
59acb9c7d90f url: use CONNECT for HTTPS connections through HTTP proxy (issue967) Henrik Stuart <hg@hstuart.dk> parents: 8344 diff changeset	356 res.will_close = 1
59acb9c7d90f url: use CONNECT for HTTPS connections through HTTP proxy (issue967) Henrik Stuart <hg@hstuart.dk> parents: 8344 diff changeset	357
59acb9c7d90f url: use CONNECT for HTTPS connections through HTTP proxy (issue967) Henrik Stuart <hg@hstuart.dk> parents: 8344 diff changeset	358 self.proxyres = res
59acb9c7d90f url: use CONNECT for HTTPS connections through HTTP proxy (issue967) Henrik Stuart <hg@hstuart.dk> parents: 8344 diff changeset	359
59acb9c7d90f url: use CONNECT for HTTPS connections through HTTP proxy (issue967) Henrik Stuart <hg@hstuart.dk> parents: 8344 diff changeset	360 return False
59acb9c7d90f url: use CONNECT for HTTPS connections through HTTP proxy (issue967) Henrik Stuart <hg@hstuart.dk> parents: 8344 diff changeset	361
59acb9c7d90f url: use CONNECT for HTTPS connections through HTTP proxy (issue967) Henrik Stuart <hg@hstuart.dk> parents: 8344 diff changeset	362 def connect(self):
59acb9c7d90f url: use CONNECT for HTTPS connections through HTTP proxy (issue967) Henrik Stuart <hg@hstuart.dk> parents: 8344 diff changeset	363 if has_https and self.realhost: # use CONNECT proxy
59acb9c7d90f url: use CONNECT for HTTPS connections through HTTP proxy (issue967) Henrik Stuart <hg@hstuart.dk> parents: 8344 diff changeset	364 self.sock = socket.socket(socket.AF_INET, socket.SOCK_STREAM)
59acb9c7d90f url: use CONNECT for HTTPS connections through HTTP proxy (issue967) Henrik Stuart <hg@hstuart.dk> parents: 8344 diff changeset	365 self.sock.connect((self.host, self.port))
59acb9c7d90f url: use CONNECT for HTTPS connections through HTTP proxy (issue967) Henrik Stuart <hg@hstuart.dk> parents: 8344 diff changeset	366 if self._proxytunnel():
59acb9c7d90f url: use CONNECT for HTTPS connections through HTTP proxy (issue967) Henrik Stuart <hg@hstuart.dk> parents: 8344 diff changeset	367 # we do not support client x509 certificates
59acb9c7d90f url: use CONNECT for HTTPS connections through HTTP proxy (issue967) Henrik Stuart <hg@hstuart.dk> parents: 8344 diff changeset	368 self.sock = _ssl_wrap_socket(self.sock, None, None)
59acb9c7d90f url: use CONNECT for HTTPS connections through HTTP proxy (issue967) Henrik Stuart <hg@hstuart.dk> parents: 8344 diff changeset	369 else:
59acb9c7d90f url: use CONNECT for HTTPS connections through HTTP proxy (issue967) Henrik Stuart <hg@hstuart.dk> parents: 8344 diff changeset	370 keepalive.HTTPConnection.connect(self)
59acb9c7d90f url: use CONNECT for HTTPS connections through HTTP proxy (issue967) Henrik Stuart <hg@hstuart.dk> parents: 8344 diff changeset	371
59acb9c7d90f url: use CONNECT for HTTPS connections through HTTP proxy (issue967) Henrik Stuart <hg@hstuart.dk> parents: 8344 diff changeset	372 def getresponse(self):
59acb9c7d90f url: use CONNECT for HTTPS connections through HTTP proxy (issue967) Henrik Stuart <hg@hstuart.dk> parents: 8344 diff changeset	373 proxyres = getattr(self, 'proxyres', None)
59acb9c7d90f url: use CONNECT for HTTPS connections through HTTP proxy (issue967) Henrik Stuart <hg@hstuart.dk> parents: 8344 diff changeset	374 if proxyres:
59acb9c7d90f url: use CONNECT for HTTPS connections through HTTP proxy (issue967) Henrik Stuart <hg@hstuart.dk> parents: 8344 diff changeset	375 if proxyres.will_close:
59acb9c7d90f url: use CONNECT for HTTPS connections through HTTP proxy (issue967) Henrik Stuart <hg@hstuart.dk> parents: 8344 diff changeset	376 self.close()
59acb9c7d90f url: use CONNECT for HTTPS connections through HTTP proxy (issue967) Henrik Stuart <hg@hstuart.dk> parents: 8344 diff changeset	377 self.proxyres = None
59acb9c7d90f url: use CONNECT for HTTPS connections through HTTP proxy (issue967) Henrik Stuart <hg@hstuart.dk> parents: 8344 diff changeset	378 return proxyres
59acb9c7d90f url: use CONNECT for HTTPS connections through HTTP proxy (issue967) Henrik Stuart <hg@hstuart.dk> parents: 8344 diff changeset	379 return keepalive.HTTPConnection.getresponse(self)
59acb9c7d90f url: use CONNECT for HTTPS connections through HTTP proxy (issue967) Henrik Stuart <hg@hstuart.dk> parents: 8344 diff changeset	380
7270 2db33c1a5654 factor out the url handling from httprepo Benoit Boissinot <benoit.boissinot@ens-lyon.org> parents: diff changeset	381 class httphandler(keepalive.HTTPHandler):
2db33c1a5654 factor out the url handling from httprepo Benoit Boissinot <benoit.boissinot@ens-lyon.org> parents: diff changeset	382 def http_open(self, req):
2db33c1a5654 factor out the url handling from httprepo Benoit Boissinot <benoit.boissinot@ens-lyon.org> parents: diff changeset	383 return self.do_open(httpconnection, req)
2db33c1a5654 factor out the url handling from httprepo Benoit Boissinot <benoit.boissinot@ens-lyon.org> parents: diff changeset	384
8590 59acb9c7d90f url: use CONNECT for HTTPS connections through HTTP proxy (issue967) Henrik Stuart <hg@hstuart.dk> parents: 8344 diff changeset	385 def _start_transaction(self, h, req):
59acb9c7d90f url: use CONNECT for HTTPS connections through HTTP proxy (issue967) Henrik Stuart <hg@hstuart.dk> parents: 8344 diff changeset	386 if req.get_selector() == req.get_full_url(): # has proxy
59acb9c7d90f url: use CONNECT for HTTPS connections through HTTP proxy (issue967) Henrik Stuart <hg@hstuart.dk> parents: 8344 diff changeset	387 urlparts = urlparse.urlparse(req.get_selector())
59acb9c7d90f url: use CONNECT for HTTPS connections through HTTP proxy (issue967) Henrik Stuart <hg@hstuart.dk> parents: 8344 diff changeset	388 if urlparts[0] == 'https': # only use CONNECT for HTTPS
59acb9c7d90f url: use CONNECT for HTTPS connections through HTTP proxy (issue967) Henrik Stuart <hg@hstuart.dk> parents: 8344 diff changeset	389 if ':' in urlparts[1]:
59acb9c7d90f url: use CONNECT for HTTPS connections through HTTP proxy (issue967) Henrik Stuart <hg@hstuart.dk> parents: 8344 diff changeset	390 realhost, realport = urlparts[1].split(':')
59acb9c7d90f url: use CONNECT for HTTPS connections through HTTP proxy (issue967) Henrik Stuart <hg@hstuart.dk> parents: 8344 diff changeset	391 else:
59acb9c7d90f url: use CONNECT for HTTPS connections through HTTP proxy (issue967) Henrik Stuart <hg@hstuart.dk> parents: 8344 diff changeset	392 realhost = urlparts[1]
59acb9c7d90f url: use CONNECT for HTTPS connections through HTTP proxy (issue967) Henrik Stuart <hg@hstuart.dk> parents: 8344 diff changeset	393 realport = 443
59acb9c7d90f url: use CONNECT for HTTPS connections through HTTP proxy (issue967) Henrik Stuart <hg@hstuart.dk> parents: 8344 diff changeset	394
59acb9c7d90f url: use CONNECT for HTTPS connections through HTTP proxy (issue967) Henrik Stuart <hg@hstuart.dk> parents: 8344 diff changeset	395 h.realhost = realhost
59acb9c7d90f url: use CONNECT for HTTPS connections through HTTP proxy (issue967) Henrik Stuart <hg@hstuart.dk> parents: 8344 diff changeset	396 h.realport = realport
59acb9c7d90f url: use CONNECT for HTTPS connections through HTTP proxy (issue967) Henrik Stuart <hg@hstuart.dk> parents: 8344 diff changeset	397 h.headers = req.headers.copy()
59acb9c7d90f url: use CONNECT for HTTPS connections through HTTP proxy (issue967) Henrik Stuart <hg@hstuart.dk> parents: 8344 diff changeset	398 h.headers.update(self.parent.addheaders)
59acb9c7d90f url: use CONNECT for HTTPS connections through HTTP proxy (issue967) Henrik Stuart <hg@hstuart.dk> parents: 8344 diff changeset	399 return keepalive.HTTPHandler._start_transaction(self, h, req)
59acb9c7d90f url: use CONNECT for HTTPS connections through HTTP proxy (issue967) Henrik Stuart <hg@hstuart.dk> parents: 8344 diff changeset	400
59acb9c7d90f url: use CONNECT for HTTPS connections through HTTP proxy (issue967) Henrik Stuart <hg@hstuart.dk> parents: 8344 diff changeset	401 h.realhost = None
59acb9c7d90f url: use CONNECT for HTTPS connections through HTTP proxy (issue967) Henrik Stuart <hg@hstuart.dk> parents: 8344 diff changeset	402 h.realport = None
59acb9c7d90f url: use CONNECT for HTTPS connections through HTTP proxy (issue967) Henrik Stuart <hg@hstuart.dk> parents: 8344 diff changeset	403 h.headers = None
59acb9c7d90f url: use CONNECT for HTTPS connections through HTTP proxy (issue967) Henrik Stuart <hg@hstuart.dk> parents: 8344 diff changeset	404 return keepalive.HTTPHandler._start_transaction(self, h, req)
59acb9c7d90f url: use CONNECT for HTTPS connections through HTTP proxy (issue967) Henrik Stuart <hg@hstuart.dk> parents: 8344 diff changeset	405
7270 2db33c1a5654 factor out the url handling from httprepo Benoit Boissinot <benoit.boissinot@ens-lyon.org> parents: diff changeset	406 def __del__(self):
2db33c1a5654 factor out the url handling from httprepo Benoit Boissinot <benoit.boissinot@ens-lyon.org> parents: diff changeset	407 self.close_all()
2db33c1a5654 factor out the url handling from httprepo Benoit Boissinot <benoit.boissinot@ens-lyon.org> parents: diff changeset	408
2db33c1a5654 factor out the url handling from httprepo Benoit Boissinot <benoit.boissinot@ens-lyon.org> parents: diff changeset	409 if has_https:
2db33c1a5654 factor out the url handling from httprepo Benoit Boissinot <benoit.boissinot@ens-lyon.org> parents: diff changeset	410 class httpsconnection(httplib.HTTPSConnection):
2db33c1a5654 factor out the url handling from httprepo Benoit Boissinot <benoit.boissinot@ens-lyon.org> parents: diff changeset	411 response_class = keepalive.HTTPResponse
2db33c1a5654 factor out the url handling from httprepo Benoit Boissinot <benoit.boissinot@ens-lyon.org> parents: diff changeset	412 # must be able to send big bundle as stream.
2db33c1a5654 factor out the url handling from httprepo Benoit Boissinot <benoit.boissinot@ens-lyon.org> parents: diff changeset	413 send = _gen_sendfile(httplib.HTTPSConnection)
2db33c1a5654 factor out the url handling from httprepo Benoit Boissinot <benoit.boissinot@ens-lyon.org> parents: diff changeset	414
2db33c1a5654 factor out the url handling from httprepo Benoit Boissinot <benoit.boissinot@ens-lyon.org> parents: diff changeset	415 class httpshandler(keepalive.KeepAliveHandler, urllib2.HTTPSHandler):
8847 7951f385fcb7 url: support client certificate files over HTTPS (issue643) Henrik Stuart <hg@hstuart.dk> parents: 8590 diff changeset	416 def __init__(self, ui):
7951f385fcb7 url: support client certificate files over HTTPS (issue643) Henrik Stuart <hg@hstuart.dk> parents: 8590 diff changeset	417 keepalive.KeepAliveHandler.__init__(self)
7951f385fcb7 url: support client certificate files over HTTPS (issue643) Henrik Stuart <hg@hstuart.dk> parents: 8590 diff changeset	418 urllib2.HTTPSHandler.__init__(self)
7951f385fcb7 url: support client certificate files over HTTPS (issue643) Henrik Stuart <hg@hstuart.dk> parents: 8590 diff changeset	419 self.ui = ui
7951f385fcb7 url: support client certificate files over HTTPS (issue643) Henrik Stuart <hg@hstuart.dk> parents: 8590 diff changeset	420 self.pwmgr = passwordmgr(self.ui)
7951f385fcb7 url: support client certificate files over HTTPS (issue643) Henrik Stuart <hg@hstuart.dk> parents: 8590 diff changeset	421
7270 2db33c1a5654 factor out the url handling from httprepo Benoit Boissinot <benoit.boissinot@ens-lyon.org> parents: diff changeset	422 def https_open(self, req):
8847 7951f385fcb7 url: support client certificate files over HTTPS (issue643) Henrik Stuart <hg@hstuart.dk> parents: 8590 diff changeset	423 self.auth = self.pwmgr.readauthtoken(req.get_full_url())
7951f385fcb7 url: support client certificate files over HTTPS (issue643) Henrik Stuart <hg@hstuart.dk> parents: 8590 diff changeset	424 return self.do_open(self._makeconnection, req)
7951f385fcb7 url: support client certificate files over HTTPS (issue643) Henrik Stuart <hg@hstuart.dk> parents: 8590 diff changeset	425
7951f385fcb7 url: support client certificate files over HTTPS (issue643) Henrik Stuart <hg@hstuart.dk> parents: 8590 diff changeset	426 def _makeconnection(self, host, port=443, args, *kwargs):
7951f385fcb7 url: support client certificate files over HTTPS (issue643) Henrik Stuart <hg@hstuart.dk> parents: 8590 diff changeset	427 keyfile = None
7951f385fcb7 url: support client certificate files over HTTPS (issue643) Henrik Stuart <hg@hstuart.dk> parents: 8590 diff changeset	428 certfile = None
7951f385fcb7 url: support client certificate files over HTTPS (issue643) Henrik Stuart <hg@hstuart.dk> parents: 8590 diff changeset	429
7951f385fcb7 url: support client certificate files over HTTPS (issue643) Henrik Stuart <hg@hstuart.dk> parents: 8590 diff changeset	430 if args: # key_file
7951f385fcb7 url: support client certificate files over HTTPS (issue643) Henrik Stuart <hg@hstuart.dk> parents: 8590 diff changeset	431 keyfile = args.pop(0)
7951f385fcb7 url: support client certificate files over HTTPS (issue643) Henrik Stuart <hg@hstuart.dk> parents: 8590 diff changeset	432 if args: # cert_file
7951f385fcb7 url: support client certificate files over HTTPS (issue643) Henrik Stuart <hg@hstuart.dk> parents: 8590 diff changeset	433 certfile = args.pop(0)
7951f385fcb7 url: support client certificate files over HTTPS (issue643) Henrik Stuart <hg@hstuart.dk> parents: 8590 diff changeset	434
7951f385fcb7 url: support client certificate files over HTTPS (issue643) Henrik Stuart <hg@hstuart.dk> parents: 8590 diff changeset	435 # if the user has specified different key/cert files in
7951f385fcb7 url: support client certificate files over HTTPS (issue643) Henrik Stuart <hg@hstuart.dk> parents: 8590 diff changeset	436 # hgrc, we prefer these
7951f385fcb7 url: support client certificate files over HTTPS (issue643) Henrik Stuart <hg@hstuart.dk> parents: 8590 diff changeset	437 if self.auth and 'key' in self.auth and 'cert' in self.auth:
7951f385fcb7 url: support client certificate files over HTTPS (issue643) Henrik Stuart <hg@hstuart.dk> parents: 8590 diff changeset	438 keyfile = self.auth['key']
7951f385fcb7 url: support client certificate files over HTTPS (issue643) Henrik Stuart <hg@hstuart.dk> parents: 8590 diff changeset	439 certfile = self.auth['cert']
7951f385fcb7 url: support client certificate files over HTTPS (issue643) Henrik Stuart <hg@hstuart.dk> parents: 8590 diff changeset	440
7951f385fcb7 url: support client certificate files over HTTPS (issue643) Henrik Stuart <hg@hstuart.dk> parents: 8590 diff changeset	441 return httpsconnection(host, port, keyfile, certfile, args, *kwargs)
7270 2db33c1a5654 factor out the url handling from httprepo Benoit Boissinot <benoit.boissinot@ens-lyon.org> parents: diff changeset	442
2db33c1a5654 factor out the url handling from httprepo Benoit Boissinot <benoit.boissinot@ens-lyon.org> parents: diff changeset	443 # In python < 2.5 AbstractDigestAuthHandler raises a ValueError if
2db33c1a5654 factor out the url handling from httprepo Benoit Boissinot <benoit.boissinot@ens-lyon.org> parents: diff changeset	444 # it doesn't know about the auth type requested. This can happen if
2db33c1a5654 factor out the url handling from httprepo Benoit Boissinot <benoit.boissinot@ens-lyon.org> parents: diff changeset	445 # somebody is using BasicAuth and types a bad password.
2db33c1a5654 factor out the url handling from httprepo Benoit Boissinot <benoit.boissinot@ens-lyon.org> parents: diff changeset	446 class httpdigestauthhandler(urllib2.HTTPDigestAuthHandler):
2db33c1a5654 factor out the url handling from httprepo Benoit Boissinot <benoit.boissinot@ens-lyon.org> parents: diff changeset	447 def http_error_auth_reqed(self, auth_header, host, req, headers):
2db33c1a5654 factor out the url handling from httprepo Benoit Boissinot <benoit.boissinot@ens-lyon.org> parents: diff changeset	448 try:
2db33c1a5654 factor out the url handling from httprepo Benoit Boissinot <benoit.boissinot@ens-lyon.org> parents: diff changeset	449 return urllib2.HTTPDigestAuthHandler.http_error_auth_reqed(
2db33c1a5654 factor out the url handling from httprepo Benoit Boissinot <benoit.boissinot@ens-lyon.org> parents: diff changeset	450 self, auth_header, host, req, headers)
2db33c1a5654 factor out the url handling from httprepo Benoit Boissinot <benoit.boissinot@ens-lyon.org> parents: diff changeset	451 except ValueError, inst:
2db33c1a5654 factor out the url handling from httprepo Benoit Boissinot <benoit.boissinot@ens-lyon.org> parents: diff changeset	452 arg = inst.args[0]
2db33c1a5654 factor out the url handling from httprepo Benoit Boissinot <benoit.boissinot@ens-lyon.org> parents: diff changeset	453 if arg.startswith("AbstractDigestAuthHandler doesn't know "):
2db33c1a5654 factor out the url handling from httprepo Benoit Boissinot <benoit.boissinot@ens-lyon.org> parents: diff changeset	454 return
2db33c1a5654 factor out the url handling from httprepo Benoit Boissinot <benoit.boissinot@ens-lyon.org> parents: diff changeset	455 raise
2db33c1a5654 factor out the url handling from httprepo Benoit Boissinot <benoit.boissinot@ens-lyon.org> parents: diff changeset	456
2db33c1a5654 factor out the url handling from httprepo Benoit Boissinot <benoit.boissinot@ens-lyon.org> parents: diff changeset	457 def getauthinfo(path):
2db33c1a5654 factor out the url handling from httprepo Benoit Boissinot <benoit.boissinot@ens-lyon.org> parents: diff changeset	458 scheme, netloc, urlpath, query, frag = urlparse.urlsplit(path)
2db33c1a5654 factor out the url handling from httprepo Benoit Boissinot <benoit.boissinot@ens-lyon.org> parents: diff changeset	459 if not urlpath:
2db33c1a5654 factor out the url handling from httprepo Benoit Boissinot <benoit.boissinot@ens-lyon.org> parents: diff changeset	460 urlpath = '/'
7284 ac81ffac0f35 url: fix file:// URL handling Patrick Mezard <pmezard@gmail.com> parents: 7270 diff changeset	461 if scheme != 'file':
ac81ffac0f35 url: fix file:// URL handling Patrick Mezard <pmezard@gmail.com> parents: 7270 diff changeset	462 # XXX: why are we quoting the path again with some smart
ac81ffac0f35 url: fix file:// URL handling Patrick Mezard <pmezard@gmail.com> parents: 7270 diff changeset	463 # heuristic here? Anyway, it cannot be done with file://
ac81ffac0f35 url: fix file:// URL handling Patrick Mezard <pmezard@gmail.com> parents: 7270 diff changeset	464 # urls since path encoding is os/fs dependent (see
ac81ffac0f35 url: fix file:// URL handling Patrick Mezard <pmezard@gmail.com> parents: 7270 diff changeset	465 # urllib.pathname2url() for details).
ac81ffac0f35 url: fix file:// URL handling Patrick Mezard <pmezard@gmail.com> parents: 7270 diff changeset	466 urlpath = quotepath(urlpath)
7270 2db33c1a5654 factor out the url handling from httprepo Benoit Boissinot <benoit.boissinot@ens-lyon.org> parents: diff changeset	467 host, port, user, passwd = netlocsplit(netloc)
2db33c1a5654 factor out the url handling from httprepo Benoit Boissinot <benoit.boissinot@ens-lyon.org> parents: diff changeset	468
2db33c1a5654 factor out the url handling from httprepo Benoit Boissinot <benoit.boissinot@ens-lyon.org> parents: diff changeset	469 # urllib cannot handle URLs with embedded user or passwd
2db33c1a5654 factor out the url handling from httprepo Benoit Boissinot <benoit.boissinot@ens-lyon.org> parents: diff changeset	470 url = urlparse.urlunsplit((scheme, netlocunsplit(host, port),
2db33c1a5654 factor out the url handling from httprepo Benoit Boissinot <benoit.boissinot@ens-lyon.org> parents: diff changeset	471 urlpath, query, frag))
2db33c1a5654 factor out the url handling from httprepo Benoit Boissinot <benoit.boissinot@ens-lyon.org> parents: diff changeset	472 if user:
2db33c1a5654 factor out the url handling from httprepo Benoit Boissinot <benoit.boissinot@ens-lyon.org> parents: diff changeset	473 netloc = host
2db33c1a5654 factor out the url handling from httprepo Benoit Boissinot <benoit.boissinot@ens-lyon.org> parents: diff changeset	474 if port:
2db33c1a5654 factor out the url handling from httprepo Benoit Boissinot <benoit.boissinot@ens-lyon.org> parents: diff changeset	475 netloc += ':' + port
2db33c1a5654 factor out the url handling from httprepo Benoit Boissinot <benoit.boissinot@ens-lyon.org> parents: diff changeset	476 # Python < 2.4.3 uses only the netloc to search for a password
2db33c1a5654 factor out the url handling from httprepo Benoit Boissinot <benoit.boissinot@ens-lyon.org> parents: diff changeset	477 authinfo = (None, (url, netloc), user, passwd or '')
2db33c1a5654 factor out the url handling from httprepo Benoit Boissinot <benoit.boissinot@ens-lyon.org> parents: diff changeset	478 else:
2db33c1a5654 factor out the url handling from httprepo Benoit Boissinot <benoit.boissinot@ens-lyon.org> parents: diff changeset	479 authinfo = None
2db33c1a5654 factor out the url handling from httprepo Benoit Boissinot <benoit.boissinot@ens-lyon.org> parents: diff changeset	480 return url, authinfo
2db33c1a5654 factor out the url handling from httprepo Benoit Boissinot <benoit.boissinot@ens-lyon.org> parents: diff changeset	481
2db33c1a5654 factor out the url handling from httprepo Benoit Boissinot <benoit.boissinot@ens-lyon.org> parents: diff changeset	482 def opener(ui, authinfo=None):
2db33c1a5654 factor out the url handling from httprepo Benoit Boissinot <benoit.boissinot@ens-lyon.org> parents: diff changeset	483 '''
2db33c1a5654 factor out the url handling from httprepo Benoit Boissinot <benoit.boissinot@ens-lyon.org> parents: diff changeset	484 construct an opener suitable for urllib2
2db33c1a5654 factor out the url handling from httprepo Benoit Boissinot <benoit.boissinot@ens-lyon.org> parents: diff changeset	485 authinfo will be added to the password manager
2db33c1a5654 factor out the url handling from httprepo Benoit Boissinot <benoit.boissinot@ens-lyon.org> parents: diff changeset	486 '''
2db33c1a5654 factor out the url handling from httprepo Benoit Boissinot <benoit.boissinot@ens-lyon.org> parents: diff changeset	487 handlers = [httphandler()]
2db33c1a5654 factor out the url handling from httprepo Benoit Boissinot <benoit.boissinot@ens-lyon.org> parents: diff changeset	488 if has_https:
8847 7951f385fcb7 url: support client certificate files over HTTPS (issue643) Henrik Stuart <hg@hstuart.dk> parents: 8590 diff changeset	489 handlers.append(httpshandler(ui))
7270 2db33c1a5654 factor out the url handling from httprepo Benoit Boissinot <benoit.boissinot@ens-lyon.org> parents: diff changeset	490
2db33c1a5654 factor out the url handling from httprepo Benoit Boissinot <benoit.boissinot@ens-lyon.org> parents: diff changeset	491 handlers.append(proxyhandler(ui))
2db33c1a5654 factor out the url handling from httprepo Benoit Boissinot <benoit.boissinot@ens-lyon.org> parents: diff changeset	492
2db33c1a5654 factor out the url handling from httprepo Benoit Boissinot <benoit.boissinot@ens-lyon.org> parents: diff changeset	493 passmgr = passwordmgr(ui)
2db33c1a5654 factor out the url handling from httprepo Benoit Boissinot <benoit.boissinot@ens-lyon.org> parents: diff changeset	494 if authinfo is not None:
2db33c1a5654 factor out the url handling from httprepo Benoit Boissinot <benoit.boissinot@ens-lyon.org> parents: diff changeset	495 passmgr.add_password(*authinfo)
2db33c1a5654 factor out the url handling from httprepo Benoit Boissinot <benoit.boissinot@ens-lyon.org> parents: diff changeset	496 user, passwd = authinfo[2:4]
2db33c1a5654 factor out the url handling from httprepo Benoit Boissinot <benoit.boissinot@ens-lyon.org> parents: diff changeset	497 ui.debug(_('http auth: user %s, password %s\n') %
2db33c1a5654 factor out the url handling from httprepo Benoit Boissinot <benoit.boissinot@ens-lyon.org> parents: diff changeset	498 (user, passwd and '' len(passwd) or 'not set'))
2db33c1a5654 factor out the url handling from httprepo Benoit Boissinot <benoit.boissinot@ens-lyon.org> parents: diff changeset	499
2db33c1a5654 factor out the url handling from httprepo Benoit Boissinot <benoit.boissinot@ens-lyon.org> parents: diff changeset	500 handlers.extend((urllib2.HTTPBasicAuthHandler(passmgr),
2db33c1a5654 factor out the url handling from httprepo Benoit Boissinot <benoit.boissinot@ens-lyon.org> parents: diff changeset	501 httpdigestauthhandler(passmgr)))
2db33c1a5654 factor out the url handling from httprepo Benoit Boissinot <benoit.boissinot@ens-lyon.org> parents: diff changeset	502 opener = urllib2.build_opener(*handlers)
2db33c1a5654 factor out the url handling from httprepo Benoit Boissinot <benoit.boissinot@ens-lyon.org> parents: diff changeset	503
2db33c1a5654 factor out the url handling from httprepo Benoit Boissinot <benoit.boissinot@ens-lyon.org> parents: diff changeset	504 # 1.0 here is the _protocol_ version
2db33c1a5654 factor out the url handling from httprepo Benoit Boissinot <benoit.boissinot@ens-lyon.org> parents: diff changeset	505 opener.addheaders = [('User-agent', 'mercurial/proto-1.0')]
2db33c1a5654 factor out the url handling from httprepo Benoit Boissinot <benoit.boissinot@ens-lyon.org> parents: diff changeset	506 opener.addheaders.append(('Accept', 'application/mercurial-0.1'))
2db33c1a5654 factor out the url handling from httprepo Benoit Boissinot <benoit.boissinot@ens-lyon.org> parents: diff changeset	507 return opener
2db33c1a5654 factor out the url handling from httprepo Benoit Boissinot <benoit.boissinot@ens-lyon.org> parents: diff changeset	508
7285 5ad99abfab79 url: detect scheme with a regexp instead of urlsplit() Patrick Mezard <pmezard@gmail.com> parents: 7284 diff changeset	509 scheme_re = re.compile(r'^([a-zA-Z0-9+-.]+)://')
5ad99abfab79 url: detect scheme with a regexp instead of urlsplit() Patrick Mezard <pmezard@gmail.com> parents: 7284 diff changeset	510
7270 2db33c1a5654 factor out the url handling from httprepo Benoit Boissinot <benoit.boissinot@ens-lyon.org> parents: diff changeset	511 def open(ui, url, data=None):
7285 5ad99abfab79 url: detect scheme with a regexp instead of urlsplit() Patrick Mezard <pmezard@gmail.com> parents: 7284 diff changeset	512 scheme = None
5ad99abfab79 url: detect scheme with a regexp instead of urlsplit() Patrick Mezard <pmezard@gmail.com> parents: 7284 diff changeset	513 m = scheme_re.search(url)
5ad99abfab79 url: detect scheme with a regexp instead of urlsplit() Patrick Mezard <pmezard@gmail.com> parents: 7284 diff changeset	514 if m:
5ad99abfab79 url: detect scheme with a regexp instead of urlsplit() Patrick Mezard <pmezard@gmail.com> parents: 7284 diff changeset	515 scheme = m.group(1).lower()
7270 2db33c1a5654 factor out the url handling from httprepo Benoit Boissinot <benoit.boissinot@ens-lyon.org> parents: diff changeset	516 if not scheme:
7284 ac81ffac0f35 url: fix file:// URL handling Patrick Mezard <pmezard@gmail.com> parents: 7270 diff changeset	517 path = util.normpath(os.path.abspath(url))
ac81ffac0f35 url: fix file:// URL handling Patrick Mezard <pmezard@gmail.com> parents: 7270 diff changeset	518 url = 'file://' + urllib.pathname2url(path)
ac81ffac0f35 url: fix file:// URL handling Patrick Mezard <pmezard@gmail.com> parents: 7270 diff changeset	519 authinfo = None
7270 2db33c1a5654 factor out the url handling from httprepo Benoit Boissinot <benoit.boissinot@ens-lyon.org> parents: diff changeset	520 else:
2db33c1a5654 factor out the url handling from httprepo Benoit Boissinot <benoit.boissinot@ens-lyon.org> parents: diff changeset	521 url, authinfo = getauthinfo(url)
2db33c1a5654 factor out the url handling from httprepo Benoit Boissinot <benoit.boissinot@ens-lyon.org> parents: diff changeset	522 return opener(ui, authinfo).open(url, data)

Mercurial > public > mercurial-scm > hg-stable

annotate mercurial/url.py @ 8847:7951f385fcb7