mercurial-scm/hg: mercurial/url.py annotate

annotate mercurial/url.py @ 13770:4e8f2310f310

url: provide url object This adds a url object that re-implements urlsplit() and unsplit(). The implementation splits out usernames, passwords, and ports. The implementation is based on the behavior specified by RFC 2396[1]. However, it is much more forgiving than the RFC's specification; it places no specific restrictions on what characters are allowed in each segment of the URL other than what is necessary to split the URL into its constituent parts. [1]: http://www.ietf.org/rfc/rfc2396.txt

author	Brodie Rao <brodie@bitheap.org>
date	Fri, 25 Mar 2011 22:58:56 -0700
parents	66d65bccbf06
children	463aca32a937

rev	line source
7270 2db33c1a5654 factor out the url handling from httprepo Benoit Boissinot <benoit.boissinot@ens-lyon.org> parents: diff changeset	1 # url.py - HTTP handling for mercurial
2db33c1a5654 factor out the url handling from httprepo Benoit Boissinot <benoit.boissinot@ens-lyon.org> parents: diff changeset	2 #
2db33c1a5654 factor out the url handling from httprepo Benoit Boissinot <benoit.boissinot@ens-lyon.org> parents: diff changeset	3 # Copyright 2005, 2006, 2007, 2008 Matt Mackall <mpm@selenic.com>
2db33c1a5654 factor out the url handling from httprepo Benoit Boissinot <benoit.boissinot@ens-lyon.org> parents: diff changeset	4 # Copyright 2006, 2007 Alexis S. L. Carvalho <alexis@cecm.usp.br>
2db33c1a5654 factor out the url handling from httprepo Benoit Boissinot <benoit.boissinot@ens-lyon.org> parents: diff changeset	5 # Copyright 2006 Vadim Gelfer <vadim.gelfer@gmail.com>
2db33c1a5654 factor out the url handling from httprepo Benoit Boissinot <benoit.boissinot@ens-lyon.org> parents: diff changeset	6 #
8225 46293a0c7e9f updated license to be explicit about GPL version 2 Martin Geisler <mg@lazybytes.net> parents: 8208 diff changeset	7 # This software may be used and distributed according to the terms of the
10263 25e572394f5c Update license to GPLv2+ Matt Mackall <mpm@selenic.com> parents: 9852 diff changeset	8 # GNU General Public License version 2 or any later version.
7270 2db33c1a5654 factor out the url handling from httprepo Benoit Boissinot <benoit.boissinot@ens-lyon.org> parents: diff changeset	9
12742 6ab4a7d3c179 url: validity (notBefore/notAfter) is checked by OpenSSL (issue2407) Mads Kiilerich <mads@kiilerich.com> parents: 12607 diff changeset	10 import urllib, urllib2, urlparse, httplib, os, re, socket, cStringIO
11880 e3526634d5a3 url.py: removed 'file' inheritance in the httpsendfile class Renato Cunha <renatoc@gmail.com> parents: 11876 diff changeset	11 import __builtin__
7270 2db33c1a5654 factor out the url handling from httprepo Benoit Boissinot <benoit.boissinot@ens-lyon.org> parents: diff changeset	12 from i18n import _
2db33c1a5654 factor out the url handling from httprepo Benoit Boissinot <benoit.boissinot@ens-lyon.org> parents: diff changeset	13 import keepalive, util
2db33c1a5654 factor out the url handling from httprepo Benoit Boissinot <benoit.boissinot@ens-lyon.org> parents: diff changeset	14
11035 e4f911ce21de schemes: fix // breakage with Python 2.6.5 (issue2111) Michael Glassford <glassfordmjg@gmail.com> parents: 10794 diff changeset	15 def _urlunparse(scheme, netloc, path, params, query, fragment, url):
e4f911ce21de schemes: fix // breakage with Python 2.6.5 (issue2111) Michael Glassford <glassfordmjg@gmail.com> parents: 10794 diff changeset	16 '''Handle cases where urlunparse(urlparse(x://)) doesn't preserve the "//"'''
e4f911ce21de schemes: fix // breakage with Python 2.6.5 (issue2111) Michael Glassford <glassfordmjg@gmail.com> parents: 10794 diff changeset	17 result = urlparse.urlunparse((scheme, netloc, path, params, query, fragment))
e4f911ce21de schemes: fix // breakage with Python 2.6.5 (issue2111) Michael Glassford <glassfordmjg@gmail.com> parents: 10794 diff changeset	18 if (scheme and
e4f911ce21de schemes: fix // breakage with Python 2.6.5 (issue2111) Michael Glassford <glassfordmjg@gmail.com> parents: 10794 diff changeset	19 result.startswith(scheme + ':') and
e4f911ce21de schemes: fix // breakage with Python 2.6.5 (issue2111) Michael Glassford <glassfordmjg@gmail.com> parents: 10794 diff changeset	20 not result.startswith(scheme + '://') and
e4f911ce21de schemes: fix // breakage with Python 2.6.5 (issue2111) Michael Glassford <glassfordmjg@gmail.com> parents: 10794 diff changeset	21 url.startswith(scheme + '://')
e4f911ce21de schemes: fix // breakage with Python 2.6.5 (issue2111) Michael Glassford <glassfordmjg@gmail.com> parents: 10794 diff changeset	22 ):
e4f911ce21de schemes: fix // breakage with Python 2.6.5 (issue2111) Michael Glassford <glassfordmjg@gmail.com> parents: 10794 diff changeset	23 result = scheme + '://' + result[len(scheme + ':'):]
e4f911ce21de schemes: fix // breakage with Python 2.6.5 (issue2111) Michael Glassford <glassfordmjg@gmail.com> parents: 10794 diff changeset	24 return result
e4f911ce21de schemes: fix // breakage with Python 2.6.5 (issue2111) Michael Glassford <glassfordmjg@gmail.com> parents: 10794 diff changeset	25
13770 4e8f2310f310 url: provide url object Brodie Rao <brodie@bitheap.org> parents: 13544 diff changeset	26 class url(object):
4e8f2310f310 url: provide url object Brodie Rao <brodie@bitheap.org> parents: 13544 diff changeset	27 """Reliable URL parser.
4e8f2310f310 url: provide url object Brodie Rao <brodie@bitheap.org> parents: 13544 diff changeset	28
4e8f2310f310 url: provide url object Brodie Rao <brodie@bitheap.org> parents: 13544 diff changeset	29 This parses URLs and provides attributes for the following
4e8f2310f310 url: provide url object Brodie Rao <brodie@bitheap.org> parents: 13544 diff changeset	30 components:
4e8f2310f310 url: provide url object Brodie Rao <brodie@bitheap.org> parents: 13544 diff changeset	31
4e8f2310f310 url: provide url object Brodie Rao <brodie@bitheap.org> parents: 13544 diff changeset	32 <scheme>://<user>:<passwd>@<host>:<port>/<path>?<query>#<fragment>
4e8f2310f310 url: provide url object Brodie Rao <brodie@bitheap.org> parents: 13544 diff changeset	33
4e8f2310f310 url: provide url object Brodie Rao <brodie@bitheap.org> parents: 13544 diff changeset	34 Missing components are set to None. The only exception is
4e8f2310f310 url: provide url object Brodie Rao <brodie@bitheap.org> parents: 13544 diff changeset	35 fragment, which is set to '' if present but empty.
4e8f2310f310 url: provide url object Brodie Rao <brodie@bitheap.org> parents: 13544 diff changeset	36
4e8f2310f310 url: provide url object Brodie Rao <brodie@bitheap.org> parents: 13544 diff changeset	37 If parse_fragment is False, fragment is included in query. If
4e8f2310f310 url: provide url object Brodie Rao <brodie@bitheap.org> parents: 13544 diff changeset	38 parse_query is False, query is included in path. If both are
4e8f2310f310 url: provide url object Brodie Rao <brodie@bitheap.org> parents: 13544 diff changeset	39 False, both fragment and query are included in path.
4e8f2310f310 url: provide url object Brodie Rao <brodie@bitheap.org> parents: 13544 diff changeset	40
4e8f2310f310 url: provide url object Brodie Rao <brodie@bitheap.org> parents: 13544 diff changeset	41 See http://www.ietf.org/rfc/rfc2396.txt for more information.
4e8f2310f310 url: provide url object Brodie Rao <brodie@bitheap.org> parents: 13544 diff changeset	42
4e8f2310f310 url: provide url object Brodie Rao <brodie@bitheap.org> parents: 13544 diff changeset	43 Examples:
4e8f2310f310 url: provide url object Brodie Rao <brodie@bitheap.org> parents: 13544 diff changeset	44
4e8f2310f310 url: provide url object Brodie Rao <brodie@bitheap.org> parents: 13544 diff changeset	45 >>> url('http://www.ietf.org/rfc/rfc2396.txt')
4e8f2310f310 url: provide url object Brodie Rao <brodie@bitheap.org> parents: 13544 diff changeset	46 <url scheme: 'http', host: 'www.ietf.org', path: 'rfc/rfc2396.txt'>
4e8f2310f310 url: provide url object Brodie Rao <brodie@bitheap.org> parents: 13544 diff changeset	47 >>> url('ssh://[::1]:2200//home/joe/repo')
4e8f2310f310 url: provide url object Brodie Rao <brodie@bitheap.org> parents: 13544 diff changeset	48 <url scheme: 'ssh', host: '[::1]', port: '2200', path: '/home/joe/repo'>
4e8f2310f310 url: provide url object Brodie Rao <brodie@bitheap.org> parents: 13544 diff changeset	49 >>> url('file:///home/joe/repo')
4e8f2310f310 url: provide url object Brodie Rao <brodie@bitheap.org> parents: 13544 diff changeset	50 <url scheme: 'file', path: '/home/joe/repo'>
4e8f2310f310 url: provide url object Brodie Rao <brodie@bitheap.org> parents: 13544 diff changeset	51 >>> url('bundle:foo')
4e8f2310f310 url: provide url object Brodie Rao <brodie@bitheap.org> parents: 13544 diff changeset	52 <url scheme: 'bundle', path: 'foo'>
4e8f2310f310 url: provide url object Brodie Rao <brodie@bitheap.org> parents: 13544 diff changeset	53
4e8f2310f310 url: provide url object Brodie Rao <brodie@bitheap.org> parents: 13544 diff changeset	54 Authentication credentials:
4e8f2310f310 url: provide url object Brodie Rao <brodie@bitheap.org> parents: 13544 diff changeset	55
4e8f2310f310 url: provide url object Brodie Rao <brodie@bitheap.org> parents: 13544 diff changeset	56 >>> url('ssh://joe:xyz@x/repo')
4e8f2310f310 url: provide url object Brodie Rao <brodie@bitheap.org> parents: 13544 diff changeset	57 <url scheme: 'ssh', user: 'joe', passwd: 'xyz', host: 'x', path: 'repo'>
4e8f2310f310 url: provide url object Brodie Rao <brodie@bitheap.org> parents: 13544 diff changeset	58 >>> url('ssh://joe@x/repo')
4e8f2310f310 url: provide url object Brodie Rao <brodie@bitheap.org> parents: 13544 diff changeset	59 <url scheme: 'ssh', user: 'joe', host: 'x', path: 'repo'>
4e8f2310f310 url: provide url object Brodie Rao <brodie@bitheap.org> parents: 13544 diff changeset	60
4e8f2310f310 url: provide url object Brodie Rao <brodie@bitheap.org> parents: 13544 diff changeset	61 Query strings and fragments:
4e8f2310f310 url: provide url object Brodie Rao <brodie@bitheap.org> parents: 13544 diff changeset	62
4e8f2310f310 url: provide url object Brodie Rao <brodie@bitheap.org> parents: 13544 diff changeset	63 >>> url('http://host/a?b#c')
4e8f2310f310 url: provide url object Brodie Rao <brodie@bitheap.org> parents: 13544 diff changeset	64 <url scheme: 'http', host: 'host', path: 'a', query: 'b', fragment: 'c'>
4e8f2310f310 url: provide url object Brodie Rao <brodie@bitheap.org> parents: 13544 diff changeset	65 >>> url('http://host/a?b#c', parse_query=False, parse_fragment=False)
4e8f2310f310 url: provide url object Brodie Rao <brodie@bitheap.org> parents: 13544 diff changeset	66 <url scheme: 'http', host: 'host', path: 'a?b#c'>
4e8f2310f310 url: provide url object Brodie Rao <brodie@bitheap.org> parents: 13544 diff changeset	67 """
4e8f2310f310 url: provide url object Brodie Rao <brodie@bitheap.org> parents: 13544 diff changeset	68
4e8f2310f310 url: provide url object Brodie Rao <brodie@bitheap.org> parents: 13544 diff changeset	69 _safechars = "!~*'()+"
4e8f2310f310 url: provide url object Brodie Rao <brodie@bitheap.org> parents: 13544 diff changeset	70 _safepchars = "/!~*'()+"
4e8f2310f310 url: provide url object Brodie Rao <brodie@bitheap.org> parents: 13544 diff changeset	71
4e8f2310f310 url: provide url object Brodie Rao <brodie@bitheap.org> parents: 13544 diff changeset	72 def __init__(self, path, parse_query=True, parse_fragment=True):
4e8f2310f310 url: provide url object Brodie Rao <brodie@bitheap.org> parents: 13544 diff changeset	73 # We slowly chomp away at path until we have only the path left
4e8f2310f310 url: provide url object Brodie Rao <brodie@bitheap.org> parents: 13544 diff changeset	74 self.scheme = self.user = self.passwd = self.host = None
4e8f2310f310 url: provide url object Brodie Rao <brodie@bitheap.org> parents: 13544 diff changeset	75 self.port = self.path = self.query = self.fragment = None
4e8f2310f310 url: provide url object Brodie Rao <brodie@bitheap.org> parents: 13544 diff changeset	76 self._localpath = True
4e8f2310f310 url: provide url object Brodie Rao <brodie@bitheap.org> parents: 13544 diff changeset	77
4e8f2310f310 url: provide url object Brodie Rao <brodie@bitheap.org> parents: 13544 diff changeset	78 if not path.startswith('/') and ':' in path:
4e8f2310f310 url: provide url object Brodie Rao <brodie@bitheap.org> parents: 13544 diff changeset	79 parts = path.split(':', 1)
4e8f2310f310 url: provide url object Brodie Rao <brodie@bitheap.org> parents: 13544 diff changeset	80 if parts[0]:
4e8f2310f310 url: provide url object Brodie Rao <brodie@bitheap.org> parents: 13544 diff changeset	81 self.scheme, path = parts
4e8f2310f310 url: provide url object Brodie Rao <brodie@bitheap.org> parents: 13544 diff changeset	82 self._localpath = False
4e8f2310f310 url: provide url object Brodie Rao <brodie@bitheap.org> parents: 13544 diff changeset	83
4e8f2310f310 url: provide url object Brodie Rao <brodie@bitheap.org> parents: 13544 diff changeset	84 if not path:
4e8f2310f310 url: provide url object Brodie Rao <brodie@bitheap.org> parents: 13544 diff changeset	85 path = None
4e8f2310f310 url: provide url object Brodie Rao <brodie@bitheap.org> parents: 13544 diff changeset	86 if self._localpath:
4e8f2310f310 url: provide url object Brodie Rao <brodie@bitheap.org> parents: 13544 diff changeset	87 self.path = ''
4e8f2310f310 url: provide url object Brodie Rao <brodie@bitheap.org> parents: 13544 diff changeset	88 return
4e8f2310f310 url: provide url object Brodie Rao <brodie@bitheap.org> parents: 13544 diff changeset	89 else:
4e8f2310f310 url: provide url object Brodie Rao <brodie@bitheap.org> parents: 13544 diff changeset	90 if parse_fragment and '#' in path:
4e8f2310f310 url: provide url object Brodie Rao <brodie@bitheap.org> parents: 13544 diff changeset	91 path, self.fragment = path.split('#', 1)
4e8f2310f310 url: provide url object Brodie Rao <brodie@bitheap.org> parents: 13544 diff changeset	92 if not path:
4e8f2310f310 url: provide url object Brodie Rao <brodie@bitheap.org> parents: 13544 diff changeset	93 path = None
4e8f2310f310 url: provide url object Brodie Rao <brodie@bitheap.org> parents: 13544 diff changeset	94 if self._localpath:
4e8f2310f310 url: provide url object Brodie Rao <brodie@bitheap.org> parents: 13544 diff changeset	95 self.path = path
4e8f2310f310 url: provide url object Brodie Rao <brodie@bitheap.org> parents: 13544 diff changeset	96 return
4e8f2310f310 url: provide url object Brodie Rao <brodie@bitheap.org> parents: 13544 diff changeset	97
4e8f2310f310 url: provide url object Brodie Rao <brodie@bitheap.org> parents: 13544 diff changeset	98 if parse_query and '?' in path:
4e8f2310f310 url: provide url object Brodie Rao <brodie@bitheap.org> parents: 13544 diff changeset	99 path, self.query = path.split('?', 1)
4e8f2310f310 url: provide url object Brodie Rao <brodie@bitheap.org> parents: 13544 diff changeset	100 if not path:
4e8f2310f310 url: provide url object Brodie Rao <brodie@bitheap.org> parents: 13544 diff changeset	101 path = None
4e8f2310f310 url: provide url object Brodie Rao <brodie@bitheap.org> parents: 13544 diff changeset	102 if not self.query:
4e8f2310f310 url: provide url object Brodie Rao <brodie@bitheap.org> parents: 13544 diff changeset	103 self.query = None
4e8f2310f310 url: provide url object Brodie Rao <brodie@bitheap.org> parents: 13544 diff changeset	104
4e8f2310f310 url: provide url object Brodie Rao <brodie@bitheap.org> parents: 13544 diff changeset	105 # // is required to specify a host/authority
4e8f2310f310 url: provide url object Brodie Rao <brodie@bitheap.org> parents: 13544 diff changeset	106 if path and path.startswith('//'):
4e8f2310f310 url: provide url object Brodie Rao <brodie@bitheap.org> parents: 13544 diff changeset	107 parts = path[2:].split('/', 1)
4e8f2310f310 url: provide url object Brodie Rao <brodie@bitheap.org> parents: 13544 diff changeset	108 if len(parts) > 1:
4e8f2310f310 url: provide url object Brodie Rao <brodie@bitheap.org> parents: 13544 diff changeset	109 self.host, path = parts
4e8f2310f310 url: provide url object Brodie Rao <brodie@bitheap.org> parents: 13544 diff changeset	110 path = path
4e8f2310f310 url: provide url object Brodie Rao <brodie@bitheap.org> parents: 13544 diff changeset	111 else:
4e8f2310f310 url: provide url object Brodie Rao <brodie@bitheap.org> parents: 13544 diff changeset	112 self.host = parts[0]
4e8f2310f310 url: provide url object Brodie Rao <brodie@bitheap.org> parents: 13544 diff changeset	113 path = None
4e8f2310f310 url: provide url object Brodie Rao <brodie@bitheap.org> parents: 13544 diff changeset	114 if not self.host:
4e8f2310f310 url: provide url object Brodie Rao <brodie@bitheap.org> parents: 13544 diff changeset	115 self.host = None
4e8f2310f310 url: provide url object Brodie Rao <brodie@bitheap.org> parents: 13544 diff changeset	116 if path:
4e8f2310f310 url: provide url object Brodie Rao <brodie@bitheap.org> parents: 13544 diff changeset	117 path = '/' + path
4e8f2310f310 url: provide url object Brodie Rao <brodie@bitheap.org> parents: 13544 diff changeset	118
4e8f2310f310 url: provide url object Brodie Rao <brodie@bitheap.org> parents: 13544 diff changeset	119 if self.host and '@' in self.host:
4e8f2310f310 url: provide url object Brodie Rao <brodie@bitheap.org> parents: 13544 diff changeset	120 self.user, self.host = self.host.rsplit('@', 1)
4e8f2310f310 url: provide url object Brodie Rao <brodie@bitheap.org> parents: 13544 diff changeset	121 if ':' in self.user:
4e8f2310f310 url: provide url object Brodie Rao <brodie@bitheap.org> parents: 13544 diff changeset	122 self.user, self.passwd = self.user.split(':', 1)
4e8f2310f310 url: provide url object Brodie Rao <brodie@bitheap.org> parents: 13544 diff changeset	123 if not self.host:
4e8f2310f310 url: provide url object Brodie Rao <brodie@bitheap.org> parents: 13544 diff changeset	124 self.host = None
4e8f2310f310 url: provide url object Brodie Rao <brodie@bitheap.org> parents: 13544 diff changeset	125
4e8f2310f310 url: provide url object Brodie Rao <brodie@bitheap.org> parents: 13544 diff changeset	126 # Don't split on colons in IPv6 addresses without ports
4e8f2310f310 url: provide url object Brodie Rao <brodie@bitheap.org> parents: 13544 diff changeset	127 if (self.host and ':' in self.host and
4e8f2310f310 url: provide url object Brodie Rao <brodie@bitheap.org> parents: 13544 diff changeset	128 not (self.host.startswith('[') and self.host.endswith(']'))):
4e8f2310f310 url: provide url object Brodie Rao <brodie@bitheap.org> parents: 13544 diff changeset	129 self.host, self.port = self.host.rsplit(':', 1)
4e8f2310f310 url: provide url object Brodie Rao <brodie@bitheap.org> parents: 13544 diff changeset	130 if not self.host:
4e8f2310f310 url: provide url object Brodie Rao <brodie@bitheap.org> parents: 13544 diff changeset	131 self.host = None
4e8f2310f310 url: provide url object Brodie Rao <brodie@bitheap.org> parents: 13544 diff changeset	132 self.path = path
4e8f2310f310 url: provide url object Brodie Rao <brodie@bitheap.org> parents: 13544 diff changeset	133
4e8f2310f310 url: provide url object Brodie Rao <brodie@bitheap.org> parents: 13544 diff changeset	134 for a in ('user', 'passwd', 'host', 'port',
4e8f2310f310 url: provide url object Brodie Rao <brodie@bitheap.org> parents: 13544 diff changeset	135 'path', 'query', 'fragment'):
4e8f2310f310 url: provide url object Brodie Rao <brodie@bitheap.org> parents: 13544 diff changeset	136 v = getattr(self, a)
4e8f2310f310 url: provide url object Brodie Rao <brodie@bitheap.org> parents: 13544 diff changeset	137 if v is not None:
4e8f2310f310 url: provide url object Brodie Rao <brodie@bitheap.org> parents: 13544 diff changeset	138 setattr(self, a, urllib.unquote(v))
4e8f2310f310 url: provide url object Brodie Rao <brodie@bitheap.org> parents: 13544 diff changeset	139
4e8f2310f310 url: provide url object Brodie Rao <brodie@bitheap.org> parents: 13544 diff changeset	140 def __repr__(self):
4e8f2310f310 url: provide url object Brodie Rao <brodie@bitheap.org> parents: 13544 diff changeset	141 attrs = []
4e8f2310f310 url: provide url object Brodie Rao <brodie@bitheap.org> parents: 13544 diff changeset	142 for a in ('scheme', 'user', 'passwd', 'host', 'port', 'path',
4e8f2310f310 url: provide url object Brodie Rao <brodie@bitheap.org> parents: 13544 diff changeset	143 'query', 'fragment'):
4e8f2310f310 url: provide url object Brodie Rao <brodie@bitheap.org> parents: 13544 diff changeset	144 v = getattr(self, a)
4e8f2310f310 url: provide url object Brodie Rao <brodie@bitheap.org> parents: 13544 diff changeset	145 if v is not None:
4e8f2310f310 url: provide url object Brodie Rao <brodie@bitheap.org> parents: 13544 diff changeset	146 attrs.append('%s: %r' % (a, v))
4e8f2310f310 url: provide url object Brodie Rao <brodie@bitheap.org> parents: 13544 diff changeset	147 return '<url %s>' % ', '.join(attrs)
4e8f2310f310 url: provide url object Brodie Rao <brodie@bitheap.org> parents: 13544 diff changeset	148
4e8f2310f310 url: provide url object Brodie Rao <brodie@bitheap.org> parents: 13544 diff changeset	149 def __str__(self):
4e8f2310f310 url: provide url object Brodie Rao <brodie@bitheap.org> parents: 13544 diff changeset	150 """Join the URL's components back into a URL string.
4e8f2310f310 url: provide url object Brodie Rao <brodie@bitheap.org> parents: 13544 diff changeset	151
4e8f2310f310 url: provide url object Brodie Rao <brodie@bitheap.org> parents: 13544 diff changeset	152 Examples:
4e8f2310f310 url: provide url object Brodie Rao <brodie@bitheap.org> parents: 13544 diff changeset	153
4e8f2310f310 url: provide url object Brodie Rao <brodie@bitheap.org> parents: 13544 diff changeset	154 >>> str(url('http://user:pw@host:80/?foo#bar'))
4e8f2310f310 url: provide url object Brodie Rao <brodie@bitheap.org> parents: 13544 diff changeset	155 'http://user:pw@host:80/?foo#bar'
4e8f2310f310 url: provide url object Brodie Rao <brodie@bitheap.org> parents: 13544 diff changeset	156 >>> str(url('ssh://user:pw@[::1]:2200//home/joe#'))
4e8f2310f310 url: provide url object Brodie Rao <brodie@bitheap.org> parents: 13544 diff changeset	157 'ssh://user:pw@[::1]:2200//home/joe#'
4e8f2310f310 url: provide url object Brodie Rao <brodie@bitheap.org> parents: 13544 diff changeset	158 >>> str(url('http://localhost:80//'))
4e8f2310f310 url: provide url object Brodie Rao <brodie@bitheap.org> parents: 13544 diff changeset	159 'http://localhost:80//'
4e8f2310f310 url: provide url object Brodie Rao <brodie@bitheap.org> parents: 13544 diff changeset	160 >>> str(url('http://localhost:80/'))
4e8f2310f310 url: provide url object Brodie Rao <brodie@bitheap.org> parents: 13544 diff changeset	161 'http://localhost:80/'
4e8f2310f310 url: provide url object Brodie Rao <brodie@bitheap.org> parents: 13544 diff changeset	162 >>> str(url('http://localhost:80'))
4e8f2310f310 url: provide url object Brodie Rao <brodie@bitheap.org> parents: 13544 diff changeset	163 'http://localhost:80'
4e8f2310f310 url: provide url object Brodie Rao <brodie@bitheap.org> parents: 13544 diff changeset	164 >>> str(url('bundle:foo'))
4e8f2310f310 url: provide url object Brodie Rao <brodie@bitheap.org> parents: 13544 diff changeset	165 'bundle:foo'
4e8f2310f310 url: provide url object Brodie Rao <brodie@bitheap.org> parents: 13544 diff changeset	166 >>> str(url('path'))
4e8f2310f310 url: provide url object Brodie Rao <brodie@bitheap.org> parents: 13544 diff changeset	167 'path'
4e8f2310f310 url: provide url object Brodie Rao <brodie@bitheap.org> parents: 13544 diff changeset	168 """
4e8f2310f310 url: provide url object Brodie Rao <brodie@bitheap.org> parents: 13544 diff changeset	169 if self._localpath:
4e8f2310f310 url: provide url object Brodie Rao <brodie@bitheap.org> parents: 13544 diff changeset	170 s = self.path
4e8f2310f310 url: provide url object Brodie Rao <brodie@bitheap.org> parents: 13544 diff changeset	171 if self.fragment:
4e8f2310f310 url: provide url object Brodie Rao <brodie@bitheap.org> parents: 13544 diff changeset	172 s += '#' + self.fragment
4e8f2310f310 url: provide url object Brodie Rao <brodie@bitheap.org> parents: 13544 diff changeset	173 return s
4e8f2310f310 url: provide url object Brodie Rao <brodie@bitheap.org> parents: 13544 diff changeset	174
4e8f2310f310 url: provide url object Brodie Rao <brodie@bitheap.org> parents: 13544 diff changeset	175 s = self.scheme + ':'
4e8f2310f310 url: provide url object Brodie Rao <brodie@bitheap.org> parents: 13544 diff changeset	176 if (self.user or self.passwd or self.host or
4e8f2310f310 url: provide url object Brodie Rao <brodie@bitheap.org> parents: 13544 diff changeset	177 self.scheme and not self.path):
4e8f2310f310 url: provide url object Brodie Rao <brodie@bitheap.org> parents: 13544 diff changeset	178 s += '//'
4e8f2310f310 url: provide url object Brodie Rao <brodie@bitheap.org> parents: 13544 diff changeset	179 if self.user:
4e8f2310f310 url: provide url object Brodie Rao <brodie@bitheap.org> parents: 13544 diff changeset	180 s += urllib.quote(self.user, safe=self._safechars)
4e8f2310f310 url: provide url object Brodie Rao <brodie@bitheap.org> parents: 13544 diff changeset	181 if self.passwd:
4e8f2310f310 url: provide url object Brodie Rao <brodie@bitheap.org> parents: 13544 diff changeset	182 s += ':' + urllib.quote(self.passwd, safe=self._safechars)
4e8f2310f310 url: provide url object Brodie Rao <brodie@bitheap.org> parents: 13544 diff changeset	183 if self.user or self.passwd:
4e8f2310f310 url: provide url object Brodie Rao <brodie@bitheap.org> parents: 13544 diff changeset	184 s += '@'
4e8f2310f310 url: provide url object Brodie Rao <brodie@bitheap.org> parents: 13544 diff changeset	185 if self.host:
4e8f2310f310 url: provide url object Brodie Rao <brodie@bitheap.org> parents: 13544 diff changeset	186 if not (self.host.startswith('[') and self.host.endswith(']')):
4e8f2310f310 url: provide url object Brodie Rao <brodie@bitheap.org> parents: 13544 diff changeset	187 s += urllib.quote(self.host)
4e8f2310f310 url: provide url object Brodie Rao <brodie@bitheap.org> parents: 13544 diff changeset	188 else:
4e8f2310f310 url: provide url object Brodie Rao <brodie@bitheap.org> parents: 13544 diff changeset	189 s += self.host
4e8f2310f310 url: provide url object Brodie Rao <brodie@bitheap.org> parents: 13544 diff changeset	190 if self.port:
4e8f2310f310 url: provide url object Brodie Rao <brodie@bitheap.org> parents: 13544 diff changeset	191 s += ':' + urllib.quote(self.port)
4e8f2310f310 url: provide url object Brodie Rao <brodie@bitheap.org> parents: 13544 diff changeset	192 if ((self.host and self.path is not None) or
4e8f2310f310 url: provide url object Brodie Rao <brodie@bitheap.org> parents: 13544 diff changeset	193 (self.host and self.query or self.fragment)):
4e8f2310f310 url: provide url object Brodie Rao <brodie@bitheap.org> parents: 13544 diff changeset	194 s += '/'
4e8f2310f310 url: provide url object Brodie Rao <brodie@bitheap.org> parents: 13544 diff changeset	195 if self.path:
4e8f2310f310 url: provide url object Brodie Rao <brodie@bitheap.org> parents: 13544 diff changeset	196 s += urllib.quote(self.path, safe=self._safepchars)
4e8f2310f310 url: provide url object Brodie Rao <brodie@bitheap.org> parents: 13544 diff changeset	197 if self.query:
4e8f2310f310 url: provide url object Brodie Rao <brodie@bitheap.org> parents: 13544 diff changeset	198 s += '?' + urllib.quote(self.query, safe=self._safepchars)
4e8f2310f310 url: provide url object Brodie Rao <brodie@bitheap.org> parents: 13544 diff changeset	199 if self.fragment is not None:
4e8f2310f310 url: provide url object Brodie Rao <brodie@bitheap.org> parents: 13544 diff changeset	200 s += '#' + urllib.quote(self.fragment, safe=self._safepchars)
4e8f2310f310 url: provide url object Brodie Rao <brodie@bitheap.org> parents: 13544 diff changeset	201 return s
4e8f2310f310 url: provide url object Brodie Rao <brodie@bitheap.org> parents: 13544 diff changeset	202
4e8f2310f310 url: provide url object Brodie Rao <brodie@bitheap.org> parents: 13544 diff changeset	203 def authinfo(self):
4e8f2310f310 url: provide url object Brodie Rao <brodie@bitheap.org> parents: 13544 diff changeset	204 user, passwd = self.user, self.passwd
4e8f2310f310 url: provide url object Brodie Rao <brodie@bitheap.org> parents: 13544 diff changeset	205 try:
4e8f2310f310 url: provide url object Brodie Rao <brodie@bitheap.org> parents: 13544 diff changeset	206 self.user, self.passwd = None, None
4e8f2310f310 url: provide url object Brodie Rao <brodie@bitheap.org> parents: 13544 diff changeset	207 s = str(self)
4e8f2310f310 url: provide url object Brodie Rao <brodie@bitheap.org> parents: 13544 diff changeset	208 finally:
4e8f2310f310 url: provide url object Brodie Rao <brodie@bitheap.org> parents: 13544 diff changeset	209 self.user, self.passwd = user, passwd
4e8f2310f310 url: provide url object Brodie Rao <brodie@bitheap.org> parents: 13544 diff changeset	210 if not self.user:
4e8f2310f310 url: provide url object Brodie Rao <brodie@bitheap.org> parents: 13544 diff changeset	211 return (s, None)
4e8f2310f310 url: provide url object Brodie Rao <brodie@bitheap.org> parents: 13544 diff changeset	212 return (s, (None, (str(self), self.host),
4e8f2310f310 url: provide url object Brodie Rao <brodie@bitheap.org> parents: 13544 diff changeset	213 self.user, self.passwd or ''))
4e8f2310f310 url: provide url object Brodie Rao <brodie@bitheap.org> parents: 13544 diff changeset	214
4e8f2310f310 url: provide url object Brodie Rao <brodie@bitheap.org> parents: 13544 diff changeset	215 def has_scheme(path):
4e8f2310f310 url: provide url object Brodie Rao <brodie@bitheap.org> parents: 13544 diff changeset	216 return bool(url(path).scheme)
4e8f2310f310 url: provide url object Brodie Rao <brodie@bitheap.org> parents: 13544 diff changeset	217
7270 2db33c1a5654 factor out the url handling from httprepo Benoit Boissinot <benoit.boissinot@ens-lyon.org> parents: diff changeset	218 def hidepassword(url):
2db33c1a5654 factor out the url handling from httprepo Benoit Boissinot <benoit.boissinot@ens-lyon.org> parents: diff changeset	219 '''hide user credential in a url string'''
2db33c1a5654 factor out the url handling from httprepo Benoit Boissinot <benoit.boissinot@ens-lyon.org> parents: diff changeset	220 scheme, netloc, path, params, query, fragment = urlparse.urlparse(url)
2db33c1a5654 factor out the url handling from httprepo Benoit Boissinot <benoit.boissinot@ens-lyon.org> parents: diff changeset	221 netloc = re.sub('([^:]):([^@])@(.)', r'\1:**@\3', netloc)
11035 e4f911ce21de schemes: fix // breakage with Python 2.6.5 (issue2111) Michael Glassford <glassfordmjg@gmail.com> parents: 10794 diff changeset	222 return _urlunparse(scheme, netloc, path, params, query, fragment, url)
7270 2db33c1a5654 factor out the url handling from httprepo Benoit Boissinot <benoit.boissinot@ens-lyon.org> parents: diff changeset	223
2db33c1a5654 factor out the url handling from httprepo Benoit Boissinot <benoit.boissinot@ens-lyon.org> parents: diff changeset	224 def removeauth(url):
2db33c1a5654 factor out the url handling from httprepo Benoit Boissinot <benoit.boissinot@ens-lyon.org> parents: diff changeset	225 '''remove all authentication information from a url string'''
2db33c1a5654 factor out the url handling from httprepo Benoit Boissinot <benoit.boissinot@ens-lyon.org> parents: diff changeset	226 scheme, netloc, path, params, query, fragment = urlparse.urlparse(url)
2db33c1a5654 factor out the url handling from httprepo Benoit Boissinot <benoit.boissinot@ens-lyon.org> parents: diff changeset	227 netloc = netloc[netloc.find('@')+1:]
11035 e4f911ce21de schemes: fix // breakage with Python 2.6.5 (issue2111) Michael Glassford <glassfordmjg@gmail.com> parents: 10794 diff changeset	228 return _urlunparse(scheme, netloc, path, params, query, fragment, url)
7270 2db33c1a5654 factor out the url handling from httprepo Benoit Boissinot <benoit.boissinot@ens-lyon.org> parents: diff changeset	229
2db33c1a5654 factor out the url handling from httprepo Benoit Boissinot <benoit.boissinot@ens-lyon.org> parents: diff changeset	230 def netlocsplit(netloc):
2db33c1a5654 factor out the url handling from httprepo Benoit Boissinot <benoit.boissinot@ens-lyon.org> parents: diff changeset	231 '''split [user[:passwd]@]host[:port] into 4-tuple.'''
2db33c1a5654 factor out the url handling from httprepo Benoit Boissinot <benoit.boissinot@ens-lyon.org> parents: diff changeset	232
2db33c1a5654 factor out the url handling from httprepo Benoit Boissinot <benoit.boissinot@ens-lyon.org> parents: diff changeset	233 a = netloc.find('@')
2db33c1a5654 factor out the url handling from httprepo Benoit Boissinot <benoit.boissinot@ens-lyon.org> parents: diff changeset	234 if a == -1:
2db33c1a5654 factor out the url handling from httprepo Benoit Boissinot <benoit.boissinot@ens-lyon.org> parents: diff changeset	235 user, passwd = None, None
2db33c1a5654 factor out the url handling from httprepo Benoit Boissinot <benoit.boissinot@ens-lyon.org> parents: diff changeset	236 else:
10282 08a0f04b56bd many, many trivial check-code fixups Matt Mackall <mpm@selenic.com> parents: 10263 diff changeset	237 userpass, netloc = netloc[:a], netloc[a + 1:]
7270 2db33c1a5654 factor out the url handling from httprepo Benoit Boissinot <benoit.boissinot@ens-lyon.org> parents: diff changeset	238 c = userpass.find(':')
2db33c1a5654 factor out the url handling from httprepo Benoit Boissinot <benoit.boissinot@ens-lyon.org> parents: diff changeset	239 if c == -1:
2db33c1a5654 factor out the url handling from httprepo Benoit Boissinot <benoit.boissinot@ens-lyon.org> parents: diff changeset	240 user, passwd = urllib.unquote(userpass), None
2db33c1a5654 factor out the url handling from httprepo Benoit Boissinot <benoit.boissinot@ens-lyon.org> parents: diff changeset	241 else:
2db33c1a5654 factor out the url handling from httprepo Benoit Boissinot <benoit.boissinot@ens-lyon.org> parents: diff changeset	242 user = urllib.unquote(userpass[:c])
10282 08a0f04b56bd many, many trivial check-code fixups Matt Mackall <mpm@selenic.com> parents: 10263 diff changeset	243 passwd = urllib.unquote(userpass[c + 1:])
7270 2db33c1a5654 factor out the url handling from httprepo Benoit Boissinot <benoit.boissinot@ens-lyon.org> parents: diff changeset	244 c = netloc.find(':')
2db33c1a5654 factor out the url handling from httprepo Benoit Boissinot <benoit.boissinot@ens-lyon.org> parents: diff changeset	245 if c == -1:
2db33c1a5654 factor out the url handling from httprepo Benoit Boissinot <benoit.boissinot@ens-lyon.org> parents: diff changeset	246 host, port = netloc, None
2db33c1a5654 factor out the url handling from httprepo Benoit Boissinot <benoit.boissinot@ens-lyon.org> parents: diff changeset	247 else:
10282 08a0f04b56bd many, many trivial check-code fixups Matt Mackall <mpm@selenic.com> parents: 10263 diff changeset	248 host, port = netloc[:c], netloc[c + 1:]
7270 2db33c1a5654 factor out the url handling from httprepo Benoit Boissinot <benoit.boissinot@ens-lyon.org> parents: diff changeset	249 return host, port, user, passwd
2db33c1a5654 factor out the url handling from httprepo Benoit Boissinot <benoit.boissinot@ens-lyon.org> parents: diff changeset	250
2db33c1a5654 factor out the url handling from httprepo Benoit Boissinot <benoit.boissinot@ens-lyon.org> parents: diff changeset	251 def netlocunsplit(host, port, user=None, passwd=None):
2db33c1a5654 factor out the url handling from httprepo Benoit Boissinot <benoit.boissinot@ens-lyon.org> parents: diff changeset	252 '''turn host, port, user, passwd into [user[:passwd]@]host[:port].'''
2db33c1a5654 factor out the url handling from httprepo Benoit Boissinot <benoit.boissinot@ens-lyon.org> parents: diff changeset	253 if port:
2db33c1a5654 factor out the url handling from httprepo Benoit Boissinot <benoit.boissinot@ens-lyon.org> parents: diff changeset	254 hostport = host + ':' + port
2db33c1a5654 factor out the url handling from httprepo Benoit Boissinot <benoit.boissinot@ens-lyon.org> parents: diff changeset	255 else:
2db33c1a5654 factor out the url handling from httprepo Benoit Boissinot <benoit.boissinot@ens-lyon.org> parents: diff changeset	256 hostport = host
2db33c1a5654 factor out the url handling from httprepo Benoit Boissinot <benoit.boissinot@ens-lyon.org> parents: diff changeset	257 if user:
10484 cadd7e076b69 url: correctly quote '/' in user and password embedded in urls Benoit Boissinot <benoit.boissinot@ens-lyon.org> parents: 10482 diff changeset	258 quote = lambda s: urllib.quote(s, safe='')
7270 2db33c1a5654 factor out the url handling from httprepo Benoit Boissinot <benoit.boissinot@ens-lyon.org> parents: diff changeset	259 if passwd:
10484 cadd7e076b69 url: correctly quote '/' in user and password embedded in urls Benoit Boissinot <benoit.boissinot@ens-lyon.org> parents: 10482 diff changeset	260 userpass = quote(user) + ':' + quote(passwd)
7270 2db33c1a5654 factor out the url handling from httprepo Benoit Boissinot <benoit.boissinot@ens-lyon.org> parents: diff changeset	261 else:
10484 cadd7e076b69 url: correctly quote '/' in user and password embedded in urls Benoit Boissinot <benoit.boissinot@ens-lyon.org> parents: 10482 diff changeset	262 userpass = quote(user)
7270 2db33c1a5654 factor out the url handling from httprepo Benoit Boissinot <benoit.boissinot@ens-lyon.org> parents: diff changeset	263 return userpass + '@' + hostport
2db33c1a5654 factor out the url handling from httprepo Benoit Boissinot <benoit.boissinot@ens-lyon.org> parents: diff changeset	264 return hostport
2db33c1a5654 factor out the url handling from httprepo Benoit Boissinot <benoit.boissinot@ens-lyon.org> parents: diff changeset	265
13371 c691cfdc6b4d url: move [auth] parsing out into a utility function Steve Borho <steve@borho.org> parents: 13370 diff changeset	266 def readauthforuri(ui, uri):
c691cfdc6b4d url: move [auth] parsing out into a utility function Steve Borho <steve@borho.org> parents: 13370 diff changeset	267 # Read configuration
c691cfdc6b4d url: move [auth] parsing out into a utility function Steve Borho <steve@borho.org> parents: 13370 diff changeset	268 config = dict()
c691cfdc6b4d url: move [auth] parsing out into a utility function Steve Borho <steve@borho.org> parents: 13370 diff changeset	269 for key, val in ui.configitems('auth'):
c691cfdc6b4d url: move [auth] parsing out into a utility function Steve Borho <steve@borho.org> parents: 13370 diff changeset	270 if '.' not in key:
c691cfdc6b4d url: move [auth] parsing out into a utility function Steve Borho <steve@borho.org> parents: 13370 diff changeset	271 ui.warn(_("ignoring invalid [auth] key '%s'\n") % key)
c691cfdc6b4d url: move [auth] parsing out into a utility function Steve Borho <steve@borho.org> parents: 13370 diff changeset	272 continue
c691cfdc6b4d url: move [auth] parsing out into a utility function Steve Borho <steve@borho.org> parents: 13370 diff changeset	273 group, setting = key.rsplit('.', 1)
c691cfdc6b4d url: move [auth] parsing out into a utility function Steve Borho <steve@borho.org> parents: 13370 diff changeset	274 gdict = config.setdefault(group, dict())
c691cfdc6b4d url: move [auth] parsing out into a utility function Steve Borho <steve@borho.org> parents: 13370 diff changeset	275 if setting in ('username', 'cert', 'key'):
c691cfdc6b4d url: move [auth] parsing out into a utility function Steve Borho <steve@borho.org> parents: 13370 diff changeset	276 val = util.expandpath(val)
c691cfdc6b4d url: move [auth] parsing out into a utility function Steve Borho <steve@borho.org> parents: 13370 diff changeset	277 gdict[setting] = val
c691cfdc6b4d url: move [auth] parsing out into a utility function Steve Borho <steve@borho.org> parents: 13370 diff changeset	278
c691cfdc6b4d url: move [auth] parsing out into a utility function Steve Borho <steve@borho.org> parents: 13370 diff changeset	279 # Find the best match
c691cfdc6b4d url: move [auth] parsing out into a utility function Steve Borho <steve@borho.org> parents: 13370 diff changeset	280 scheme, hostpath = uri.split('://', 1)
c691cfdc6b4d url: move [auth] parsing out into a utility function Steve Borho <steve@borho.org> parents: 13370 diff changeset	281 bestlen = 0
c691cfdc6b4d url: move [auth] parsing out into a utility function Steve Borho <steve@borho.org> parents: 13370 diff changeset	282 bestauth = None
13372 5bced0d28a39 url: return the matched authentication group name from readauthforuri() Steve Borho <steve@borho.org> parents: 13371 diff changeset	283 for group, auth in config.iteritems():
13371 c691cfdc6b4d url: move [auth] parsing out into a utility function Steve Borho <steve@borho.org> parents: 13370 diff changeset	284 prefix = auth.get('prefix')
c691cfdc6b4d url: move [auth] parsing out into a utility function Steve Borho <steve@borho.org> parents: 13370 diff changeset	285 if not prefix:
c691cfdc6b4d url: move [auth] parsing out into a utility function Steve Borho <steve@borho.org> parents: 13370 diff changeset	286 continue
c691cfdc6b4d url: move [auth] parsing out into a utility function Steve Borho <steve@borho.org> parents: 13370 diff changeset	287 p = prefix.split('://', 1)
c691cfdc6b4d url: move [auth] parsing out into a utility function Steve Borho <steve@borho.org> parents: 13370 diff changeset	288 if len(p) > 1:
c691cfdc6b4d url: move [auth] parsing out into a utility function Steve Borho <steve@borho.org> parents: 13370 diff changeset	289 schemes, prefix = [p[0]], p[1]
c691cfdc6b4d url: move [auth] parsing out into a utility function Steve Borho <steve@borho.org> parents: 13370 diff changeset	290 else:
c691cfdc6b4d url: move [auth] parsing out into a utility function Steve Borho <steve@borho.org> parents: 13370 diff changeset	291 schemes = (auth.get('schemes') or 'https').split()
c691cfdc6b4d url: move [auth] parsing out into a utility function Steve Borho <steve@borho.org> parents: 13370 diff changeset	292 if (prefix == '*' or hostpath.startswith(prefix)) and \
c691cfdc6b4d url: move [auth] parsing out into a utility function Steve Borho <steve@borho.org> parents: 13370 diff changeset	293 len(prefix) > bestlen and scheme in schemes:
c691cfdc6b4d url: move [auth] parsing out into a utility function Steve Borho <steve@borho.org> parents: 13370 diff changeset	294 bestlen = len(prefix)
13372 5bced0d28a39 url: return the matched authentication group name from readauthforuri() Steve Borho <steve@borho.org> parents: 13371 diff changeset	295 bestauth = group, auth
13371 c691cfdc6b4d url: move [auth] parsing out into a utility function Steve Borho <steve@borho.org> parents: 13370 diff changeset	296 return bestauth
c691cfdc6b4d url: move [auth] parsing out into a utility function Steve Borho <steve@borho.org> parents: 13370 diff changeset	297
7270 2db33c1a5654 factor out the url handling from httprepo Benoit Boissinot <benoit.boissinot@ens-lyon.org> parents: diff changeset	298 _safe = ('abcdefghijklmnopqrstuvwxyz'
2db33c1a5654 factor out the url handling from httprepo Benoit Boissinot <benoit.boissinot@ens-lyon.org> parents: diff changeset	299 'ABCDEFGHIJKLMNOPQRSTUVWXYZ'
2db33c1a5654 factor out the url handling from httprepo Benoit Boissinot <benoit.boissinot@ens-lyon.org> parents: diff changeset	300 '0123456789' '_.-/')
2db33c1a5654 factor out the url handling from httprepo Benoit Boissinot <benoit.boissinot@ens-lyon.org> parents: diff changeset	301 _safeset = None
2db33c1a5654 factor out the url handling from httprepo Benoit Boissinot <benoit.boissinot@ens-lyon.org> parents: diff changeset	302 _hex = None
2db33c1a5654 factor out the url handling from httprepo Benoit Boissinot <benoit.boissinot@ens-lyon.org> parents: diff changeset	303 def quotepath(path):
2db33c1a5654 factor out the url handling from httprepo Benoit Boissinot <benoit.boissinot@ens-lyon.org> parents: diff changeset	304 '''quote the path part of a URL
2db33c1a5654 factor out the url handling from httprepo Benoit Boissinot <benoit.boissinot@ens-lyon.org> parents: diff changeset	305
2db33c1a5654 factor out the url handling from httprepo Benoit Boissinot <benoit.boissinot@ens-lyon.org> parents: diff changeset	306 This is similar to urllib.quote, but it also tries to avoid
2db33c1a5654 factor out the url handling from httprepo Benoit Boissinot <benoit.boissinot@ens-lyon.org> parents: diff changeset	307 quoting things twice (inspired by wget):
2db33c1a5654 factor out the url handling from httprepo Benoit Boissinot <benoit.boissinot@ens-lyon.org> parents: diff changeset	308
2db33c1a5654 factor out the url handling from httprepo Benoit Boissinot <benoit.boissinot@ens-lyon.org> parents: diff changeset	309 >>> quotepath('abc def')
2db33c1a5654 factor out the url handling from httprepo Benoit Boissinot <benoit.boissinot@ens-lyon.org> parents: diff changeset	310 'abc%20def'
2db33c1a5654 factor out the url handling from httprepo Benoit Boissinot <benoit.boissinot@ens-lyon.org> parents: diff changeset	311 >>> quotepath('abc%20def')
2db33c1a5654 factor out the url handling from httprepo Benoit Boissinot <benoit.boissinot@ens-lyon.org> parents: diff changeset	312 'abc%20def'
2db33c1a5654 factor out the url handling from httprepo Benoit Boissinot <benoit.boissinot@ens-lyon.org> parents: diff changeset	313 >>> quotepath('abc%20 def')
2db33c1a5654 factor out the url handling from httprepo Benoit Boissinot <benoit.boissinot@ens-lyon.org> parents: diff changeset	314 'abc%20%20def'
2db33c1a5654 factor out the url handling from httprepo Benoit Boissinot <benoit.boissinot@ens-lyon.org> parents: diff changeset	315 >>> quotepath('abc def%20')
2db33c1a5654 factor out the url handling from httprepo Benoit Boissinot <benoit.boissinot@ens-lyon.org> parents: diff changeset	316 'abc%20def%20'
2db33c1a5654 factor out the url handling from httprepo Benoit Boissinot <benoit.boissinot@ens-lyon.org> parents: diff changeset	317 >>> quotepath('abc def%2')
2db33c1a5654 factor out the url handling from httprepo Benoit Boissinot <benoit.boissinot@ens-lyon.org> parents: diff changeset	318 'abc%20def%252'
2db33c1a5654 factor out the url handling from httprepo Benoit Boissinot <benoit.boissinot@ens-lyon.org> parents: diff changeset	319 >>> quotepath('abc def%')
2db33c1a5654 factor out the url handling from httprepo Benoit Boissinot <benoit.boissinot@ens-lyon.org> parents: diff changeset	320 'abc%20def%25'
2db33c1a5654 factor out the url handling from httprepo Benoit Boissinot <benoit.boissinot@ens-lyon.org> parents: diff changeset	321 '''
2db33c1a5654 factor out the url handling from httprepo Benoit Boissinot <benoit.boissinot@ens-lyon.org> parents: diff changeset	322 global _safeset, _hex
2db33c1a5654 factor out the url handling from httprepo Benoit Boissinot <benoit.boissinot@ens-lyon.org> parents: diff changeset	323 if _safeset is None:
8150 bbc24c0753a0 util: use built-in set and frozenset Martin Geisler <mg@lazybytes.net> parents: 7285 diff changeset	324 _safeset = set(_safe)
bbc24c0753a0 util: use built-in set and frozenset Martin Geisler <mg@lazybytes.net> parents: 7285 diff changeset	325 _hex = set('abcdefABCDEF0123456789')
7270 2db33c1a5654 factor out the url handling from httprepo Benoit Boissinot <benoit.boissinot@ens-lyon.org> parents: diff changeset	326 l = list(path)
2db33c1a5654 factor out the url handling from httprepo Benoit Boissinot <benoit.boissinot@ens-lyon.org> parents: diff changeset	327 for i in xrange(len(l)):
2db33c1a5654 factor out the url handling from httprepo Benoit Boissinot <benoit.boissinot@ens-lyon.org> parents: diff changeset	328 c = l[i]
10282 08a0f04b56bd many, many trivial check-code fixups Matt Mackall <mpm@selenic.com> parents: 10263 diff changeset	329 if (c == '%' and i + 2 < len(l) and
08a0f04b56bd many, many trivial check-code fixups Matt Mackall <mpm@selenic.com> parents: 10263 diff changeset	330 l[i + 1] in _hex and l[i + 2] in _hex):
7270 2db33c1a5654 factor out the url handling from httprepo Benoit Boissinot <benoit.boissinot@ens-lyon.org> parents: diff changeset	331 pass
2db33c1a5654 factor out the url handling from httprepo Benoit Boissinot <benoit.boissinot@ens-lyon.org> parents: diff changeset	332 elif c not in _safeset:
2db33c1a5654 factor out the url handling from httprepo Benoit Boissinot <benoit.boissinot@ens-lyon.org> parents: diff changeset	333 l[i] = '%%%02X' % ord(c)
2db33c1a5654 factor out the url handling from httprepo Benoit Boissinot <benoit.boissinot@ens-lyon.org> parents: diff changeset	334 return ''.join(l)
2db33c1a5654 factor out the url handling from httprepo Benoit Boissinot <benoit.boissinot@ens-lyon.org> parents: diff changeset	335
2db33c1a5654 factor out the url handling from httprepo Benoit Boissinot <benoit.boissinot@ens-lyon.org> parents: diff changeset	336 class passwordmgr(urllib2.HTTPPasswordMgrWithDefaultRealm):
2db33c1a5654 factor out the url handling from httprepo Benoit Boissinot <benoit.boissinot@ens-lyon.org> parents: diff changeset	337 def __init__(self, ui):
2db33c1a5654 factor out the url handling from httprepo Benoit Boissinot <benoit.boissinot@ens-lyon.org> parents: diff changeset	338 urllib2.HTTPPasswordMgrWithDefaultRealm.__init__(self)
2db33c1a5654 factor out the url handling from httprepo Benoit Boissinot <benoit.boissinot@ens-lyon.org> parents: diff changeset	339 self.ui = ui
2db33c1a5654 factor out the url handling from httprepo Benoit Boissinot <benoit.boissinot@ens-lyon.org> parents: diff changeset	340
2db33c1a5654 factor out the url handling from httprepo Benoit Boissinot <benoit.boissinot@ens-lyon.org> parents: diff changeset	341 def find_user_password(self, realm, authuri):
2db33c1a5654 factor out the url handling from httprepo Benoit Boissinot <benoit.boissinot@ens-lyon.org> parents: diff changeset	342 authinfo = urllib2.HTTPPasswordMgrWithDefaultRealm.find_user_password(
2db33c1a5654 factor out the url handling from httprepo Benoit Boissinot <benoit.boissinot@ens-lyon.org> parents: diff changeset	343 self, realm, authuri)
2db33c1a5654 factor out the url handling from httprepo Benoit Boissinot <benoit.boissinot@ens-lyon.org> parents: diff changeset	344 user, passwd = authinfo
2db33c1a5654 factor out the url handling from httprepo Benoit Boissinot <benoit.boissinot@ens-lyon.org> parents: diff changeset	345 if user and passwd:
8333 89c80c3dc584 allow http authentication information to be specified in the configuration Sune Foldager <cryo@cyanite.org> parents: 8225 diff changeset	346 self._writedebug(user, passwd)
7270 2db33c1a5654 factor out the url handling from httprepo Benoit Boissinot <benoit.boissinot@ens-lyon.org> parents: diff changeset	347 return (user, passwd)
2db33c1a5654 factor out the url handling from httprepo Benoit Boissinot <benoit.boissinot@ens-lyon.org> parents: diff changeset	348
8344 873429914ec5 url: fix bug in passwordmgr related to auth configuration Sune Foldager <cryo@cyanite.org> parents: 8333 diff changeset	349 if not user:
13372 5bced0d28a39 url: return the matched authentication group name from readauthforuri() Steve Borho <steve@borho.org> parents: 13371 diff changeset	350 res = readauthforuri(self.ui, authuri)
5bced0d28a39 url: return the matched authentication group name from readauthforuri() Steve Borho <steve@borho.org> parents: 13371 diff changeset	351 if res:
5bced0d28a39 url: return the matched authentication group name from readauthforuri() Steve Borho <steve@borho.org> parents: 13371 diff changeset	352 group, auth = res
8847 7951f385fcb7 url: support client certificate files over HTTPS (issue643) Henrik Stuart <hg@hstuart.dk> parents: 8590 diff changeset	353 user, passwd = auth.get('username'), auth.get('password')
13372 5bced0d28a39 url: return the matched authentication group name from readauthforuri() Steve Borho <steve@borho.org> parents: 13371 diff changeset	354 self.ui.debug("using auth.%s.* for authentication\n" % group)
8333 89c80c3dc584 allow http authentication information to be specified in the configuration Sune Foldager <cryo@cyanite.org> parents: 8225 diff changeset	355 if not user or not passwd:
89c80c3dc584 allow http authentication information to be specified in the configuration Sune Foldager <cryo@cyanite.org> parents: 8225 diff changeset	356 if not self.ui.interactive():
89c80c3dc584 allow http authentication information to be specified in the configuration Sune Foldager <cryo@cyanite.org> parents: 8225 diff changeset	357 raise util.Abort(_('http authorization required'))
7270 2db33c1a5654 factor out the url handling from httprepo Benoit Boissinot <benoit.boissinot@ens-lyon.org> parents: diff changeset	358
8333 89c80c3dc584 allow http authentication information to be specified in the configuration Sune Foldager <cryo@cyanite.org> parents: 8225 diff changeset	359 self.ui.write(_("http authorization required\n"))
12862 9d6adddc8eea url: show realm/user when asking for username/password timeless <timeless@gmail.com> parents: 12770 diff changeset	360 self.ui.write(_("realm: %s\n") % realm)
8333 89c80c3dc584 allow http authentication information to be specified in the configuration Sune Foldager <cryo@cyanite.org> parents: 8225 diff changeset	361 if user:
12862 9d6adddc8eea url: show realm/user when asking for username/password timeless <timeless@gmail.com> parents: 12770 diff changeset	362 self.ui.write(_("user: %s\n") % user)
8333 89c80c3dc584 allow http authentication information to be specified in the configuration Sune Foldager <cryo@cyanite.org> parents: 8225 diff changeset	363 else:
89c80c3dc584 allow http authentication information to be specified in the configuration Sune Foldager <cryo@cyanite.org> parents: 8225 diff changeset	364 user = self.ui.prompt(_("user:"), default=None)
7270 2db33c1a5654 factor out the url handling from httprepo Benoit Boissinot <benoit.boissinot@ens-lyon.org> parents: diff changeset	365
8333 89c80c3dc584 allow http authentication information to be specified in the configuration Sune Foldager <cryo@cyanite.org> parents: 8225 diff changeset	366 if not passwd:
89c80c3dc584 allow http authentication information to be specified in the configuration Sune Foldager <cryo@cyanite.org> parents: 8225 diff changeset	367 passwd = self.ui.getpass()
7270 2db33c1a5654 factor out the url handling from httprepo Benoit Boissinot <benoit.boissinot@ens-lyon.org> parents: diff changeset	368
2db33c1a5654 factor out the url handling from httprepo Benoit Boissinot <benoit.boissinot@ens-lyon.org> parents: diff changeset	369 self.add_password(realm, authuri, user, passwd)
8333 89c80c3dc584 allow http authentication information to be specified in the configuration Sune Foldager <cryo@cyanite.org> parents: 8225 diff changeset	370 self._writedebug(user, passwd)
7270 2db33c1a5654 factor out the url handling from httprepo Benoit Boissinot <benoit.boissinot@ens-lyon.org> parents: diff changeset	371 return (user, passwd)
2db33c1a5654 factor out the url handling from httprepo Benoit Boissinot <benoit.boissinot@ens-lyon.org> parents: diff changeset	372
8333 89c80c3dc584 allow http authentication information to be specified in the configuration Sune Foldager <cryo@cyanite.org> parents: 8225 diff changeset	373 def _writedebug(self, user, passwd):
89c80c3dc584 allow http authentication information to be specified in the configuration Sune Foldager <cryo@cyanite.org> parents: 8225 diff changeset	374 msg = _('http auth: user %s, password %s\n')
89c80c3dc584 allow http authentication information to be specified in the configuration Sune Foldager <cryo@cyanite.org> parents: 8225 diff changeset	375 self.ui.debug(msg % (user, passwd and '' len(passwd) or 'not set'))
89c80c3dc584 allow http authentication information to be specified in the configuration Sune Foldager <cryo@cyanite.org> parents: 8225 diff changeset	376
7270 2db33c1a5654 factor out the url handling from httprepo Benoit Boissinot <benoit.boissinot@ens-lyon.org> parents: diff changeset	377 class proxyhandler(urllib2.ProxyHandler):
2db33c1a5654 factor out the url handling from httprepo Benoit Boissinot <benoit.boissinot@ens-lyon.org> parents: diff changeset	378 def __init__(self, ui):
2db33c1a5654 factor out the url handling from httprepo Benoit Boissinot <benoit.boissinot@ens-lyon.org> parents: diff changeset	379 proxyurl = ui.config("http_proxy", "host") or os.getenv('http_proxy')
2db33c1a5654 factor out the url handling from httprepo Benoit Boissinot <benoit.boissinot@ens-lyon.org> parents: diff changeset	380 # XXX proxyauthinfo = None
2db33c1a5654 factor out the url handling from httprepo Benoit Boissinot <benoit.boissinot@ens-lyon.org> parents: diff changeset	381
2db33c1a5654 factor out the url handling from httprepo Benoit Boissinot <benoit.boissinot@ens-lyon.org> parents: diff changeset	382 if proxyurl:
2db33c1a5654 factor out the url handling from httprepo Benoit Boissinot <benoit.boissinot@ens-lyon.org> parents: diff changeset	383 # proxy can be proper url or host[:port]
2db33c1a5654 factor out the url handling from httprepo Benoit Boissinot <benoit.boissinot@ens-lyon.org> parents: diff changeset	384 if not (proxyurl.startswith('http:') or
2db33c1a5654 factor out the url handling from httprepo Benoit Boissinot <benoit.boissinot@ens-lyon.org> parents: diff changeset	385 proxyurl.startswith('https:')):
2db33c1a5654 factor out the url handling from httprepo Benoit Boissinot <benoit.boissinot@ens-lyon.org> parents: diff changeset	386 proxyurl = 'http://' + proxyurl + '/'
2db33c1a5654 factor out the url handling from httprepo Benoit Boissinot <benoit.boissinot@ens-lyon.org> parents: diff changeset	387 snpqf = urlparse.urlsplit(proxyurl)
2db33c1a5654 factor out the url handling from httprepo Benoit Boissinot <benoit.boissinot@ens-lyon.org> parents: diff changeset	388 proxyscheme, proxynetloc, proxypath, proxyquery, proxyfrag = snpqf
2db33c1a5654 factor out the url handling from httprepo Benoit Boissinot <benoit.boissinot@ens-lyon.org> parents: diff changeset	389 hpup = netlocsplit(proxynetloc)
2db33c1a5654 factor out the url handling from httprepo Benoit Boissinot <benoit.boissinot@ens-lyon.org> parents: diff changeset	390
2db33c1a5654 factor out the url handling from httprepo Benoit Boissinot <benoit.boissinot@ens-lyon.org> parents: diff changeset	391 proxyhost, proxyport, proxyuser, proxypasswd = hpup
2db33c1a5654 factor out the url handling from httprepo Benoit Boissinot <benoit.boissinot@ens-lyon.org> parents: diff changeset	392 if not proxyuser:
2db33c1a5654 factor out the url handling from httprepo Benoit Boissinot <benoit.boissinot@ens-lyon.org> parents: diff changeset	393 proxyuser = ui.config("http_proxy", "user")
2db33c1a5654 factor out the url handling from httprepo Benoit Boissinot <benoit.boissinot@ens-lyon.org> parents: diff changeset	394 proxypasswd = ui.config("http_proxy", "passwd")
2db33c1a5654 factor out the url handling from httprepo Benoit Boissinot <benoit.boissinot@ens-lyon.org> parents: diff changeset	395
2db33c1a5654 factor out the url handling from httprepo Benoit Boissinot <benoit.boissinot@ens-lyon.org> parents: diff changeset	396 # see if we should use a proxy for this url
10282 08a0f04b56bd many, many trivial check-code fixups Matt Mackall <mpm@selenic.com> parents: 10263 diff changeset	397 no_list = ["localhost", "127.0.0.1"]
7270 2db33c1a5654 factor out the url handling from httprepo Benoit Boissinot <benoit.boissinot@ens-lyon.org> parents: diff changeset	398 no_list.extend([p.lower() for
2db33c1a5654 factor out the url handling from httprepo Benoit Boissinot <benoit.boissinot@ens-lyon.org> parents: diff changeset	399 p in ui.configlist("http_proxy", "no")])
2db33c1a5654 factor out the url handling from httprepo Benoit Boissinot <benoit.boissinot@ens-lyon.org> parents: diff changeset	400 no_list.extend([p.strip().lower() for
2db33c1a5654 factor out the url handling from httprepo Benoit Boissinot <benoit.boissinot@ens-lyon.org> parents: diff changeset	401 p in os.getenv("no_proxy", '').split(',')
2db33c1a5654 factor out the url handling from httprepo Benoit Boissinot <benoit.boissinot@ens-lyon.org> parents: diff changeset	402 if p.strip()])
2db33c1a5654 factor out the url handling from httprepo Benoit Boissinot <benoit.boissinot@ens-lyon.org> parents: diff changeset	403 # "http_proxy.always" config is for running tests on localhost
2db33c1a5654 factor out the url handling from httprepo Benoit Boissinot <benoit.boissinot@ens-lyon.org> parents: diff changeset	404 if ui.configbool("http_proxy", "always"):
2db33c1a5654 factor out the url handling from httprepo Benoit Boissinot <benoit.boissinot@ens-lyon.org> parents: diff changeset	405 self.no_list = []
2db33c1a5654 factor out the url handling from httprepo Benoit Boissinot <benoit.boissinot@ens-lyon.org> parents: diff changeset	406 else:
2db33c1a5654 factor out the url handling from httprepo Benoit Boissinot <benoit.boissinot@ens-lyon.org> parents: diff changeset	407 self.no_list = no_list
2db33c1a5654 factor out the url handling from httprepo Benoit Boissinot <benoit.boissinot@ens-lyon.org> parents: diff changeset	408
2db33c1a5654 factor out the url handling from httprepo Benoit Boissinot <benoit.boissinot@ens-lyon.org> parents: diff changeset	409 proxyurl = urlparse.urlunsplit((
2db33c1a5654 factor out the url handling from httprepo Benoit Boissinot <benoit.boissinot@ens-lyon.org> parents: diff changeset	410 proxyscheme, netlocunsplit(proxyhost, proxyport,
2db33c1a5654 factor out the url handling from httprepo Benoit Boissinot <benoit.boissinot@ens-lyon.org> parents: diff changeset	411 proxyuser, proxypasswd or ''),
2db33c1a5654 factor out the url handling from httprepo Benoit Boissinot <benoit.boissinot@ens-lyon.org> parents: diff changeset	412 proxypath, proxyquery, proxyfrag))
2db33c1a5654 factor out the url handling from httprepo Benoit Boissinot <benoit.boissinot@ens-lyon.org> parents: diff changeset	413 proxies = {'http': proxyurl, 'https': proxyurl}
9467 4c041f1ee1b4 do not attempt to translate ui.debug output Martin Geisler <mg@lazybytes.net> parents: 9347 diff changeset	414 ui.debug('proxying through http://%s:%s\n' %
7270 2db33c1a5654 factor out the url handling from httprepo Benoit Boissinot <benoit.boissinot@ens-lyon.org> parents: diff changeset	415 (proxyhost, proxyport))
2db33c1a5654 factor out the url handling from httprepo Benoit Boissinot <benoit.boissinot@ens-lyon.org> parents: diff changeset	416 else:
2db33c1a5654 factor out the url handling from httprepo Benoit Boissinot <benoit.boissinot@ens-lyon.org> parents: diff changeset	417 proxies = {}
2db33c1a5654 factor out the url handling from httprepo Benoit Boissinot <benoit.boissinot@ens-lyon.org> parents: diff changeset	418
2db33c1a5654 factor out the url handling from httprepo Benoit Boissinot <benoit.boissinot@ens-lyon.org> parents: diff changeset	419 # urllib2 takes proxy values from the environment and those
2db33c1a5654 factor out the url handling from httprepo Benoit Boissinot <benoit.boissinot@ens-lyon.org> parents: diff changeset	420 # will take precedence if found, so drop them
2db33c1a5654 factor out the url handling from httprepo Benoit Boissinot <benoit.boissinot@ens-lyon.org> parents: diff changeset	421 for env in ["HTTP_PROXY", "http_proxy", "no_proxy"]:
2db33c1a5654 factor out the url handling from httprepo Benoit Boissinot <benoit.boissinot@ens-lyon.org> parents: diff changeset	422 try:
2db33c1a5654 factor out the url handling from httprepo Benoit Boissinot <benoit.boissinot@ens-lyon.org> parents: diff changeset	423 if env in os.environ:
2db33c1a5654 factor out the url handling from httprepo Benoit Boissinot <benoit.boissinot@ens-lyon.org> parents: diff changeset	424 del os.environ[env]
2db33c1a5654 factor out the url handling from httprepo Benoit Boissinot <benoit.boissinot@ens-lyon.org> parents: diff changeset	425 except OSError:
2db33c1a5654 factor out the url handling from httprepo Benoit Boissinot <benoit.boissinot@ens-lyon.org> parents: diff changeset	426 pass
2db33c1a5654 factor out the url handling from httprepo Benoit Boissinot <benoit.boissinot@ens-lyon.org> parents: diff changeset	427
2db33c1a5654 factor out the url handling from httprepo Benoit Boissinot <benoit.boissinot@ens-lyon.org> parents: diff changeset	428 urllib2.ProxyHandler.__init__(self, proxies)
2db33c1a5654 factor out the url handling from httprepo Benoit Boissinot <benoit.boissinot@ens-lyon.org> parents: diff changeset	429 self.ui = ui
2db33c1a5654 factor out the url handling from httprepo Benoit Boissinot <benoit.boissinot@ens-lyon.org> parents: diff changeset	430
2db33c1a5654 factor out the url handling from httprepo Benoit Boissinot <benoit.boissinot@ens-lyon.org> parents: diff changeset	431 def proxy_open(self, req, proxy, type_):
2db33c1a5654 factor out the url handling from httprepo Benoit Boissinot <benoit.boissinot@ens-lyon.org> parents: diff changeset	432 host = req.get_host().split(':')[0]
2db33c1a5654 factor out the url handling from httprepo Benoit Boissinot <benoit.boissinot@ens-lyon.org> parents: diff changeset	433 if host in self.no_list:
2db33c1a5654 factor out the url handling from httprepo Benoit Boissinot <benoit.boissinot@ens-lyon.org> parents: diff changeset	434 return None
2db33c1a5654 factor out the url handling from httprepo Benoit Boissinot <benoit.boissinot@ens-lyon.org> parents: diff changeset	435
2db33c1a5654 factor out the url handling from httprepo Benoit Boissinot <benoit.boissinot@ens-lyon.org> parents: diff changeset	436 # work around a bug in Python < 2.4.2
2db33c1a5654 factor out the url handling from httprepo Benoit Boissinot <benoit.boissinot@ens-lyon.org> parents: diff changeset	437 # (it leaves a "\n" at the end of Proxy-authorization headers)
2db33c1a5654 factor out the url handling from httprepo Benoit Boissinot <benoit.boissinot@ens-lyon.org> parents: diff changeset	438 baseclass = req.__class__
2db33c1a5654 factor out the url handling from httprepo Benoit Boissinot <benoit.boissinot@ens-lyon.org> parents: diff changeset	439 class _request(baseclass):
2db33c1a5654 factor out the url handling from httprepo Benoit Boissinot <benoit.boissinot@ens-lyon.org> parents: diff changeset	440 def add_header(self, key, val):
2db33c1a5654 factor out the url handling from httprepo Benoit Boissinot <benoit.boissinot@ens-lyon.org> parents: diff changeset	441 if key.lower() == 'proxy-authorization':
2db33c1a5654 factor out the url handling from httprepo Benoit Boissinot <benoit.boissinot@ens-lyon.org> parents: diff changeset	442 val = val.strip()
2db33c1a5654 factor out the url handling from httprepo Benoit Boissinot <benoit.boissinot@ens-lyon.org> parents: diff changeset	443 return baseclass.add_header(self, key, val)
2db33c1a5654 factor out the url handling from httprepo Benoit Boissinot <benoit.boissinot@ens-lyon.org> parents: diff changeset	444 req.__class__ = _request
2db33c1a5654 factor out the url handling from httprepo Benoit Boissinot <benoit.boissinot@ens-lyon.org> parents: diff changeset	445
2db33c1a5654 factor out the url handling from httprepo Benoit Boissinot <benoit.boissinot@ens-lyon.org> parents: diff changeset	446 return urllib2.ProxyHandler.proxy_open(self, req, proxy, type_)
2db33c1a5654 factor out the url handling from httprepo Benoit Boissinot <benoit.boissinot@ens-lyon.org> parents: diff changeset	447
11880 e3526634d5a3 url.py: removed 'file' inheritance in the httpsendfile class Renato Cunha <renatoc@gmail.com> parents: 11876 diff changeset	448 class httpsendfile(object):
e3526634d5a3 url.py: removed 'file' inheritance in the httpsendfile class Renato Cunha <renatoc@gmail.com> parents: 11876 diff changeset	449 """This is a wrapper around the objects returned by python's "open".
e3526634d5a3 url.py: removed 'file' inheritance in the httpsendfile class Renato Cunha <renatoc@gmail.com> parents: 11876 diff changeset	450
e3526634d5a3 url.py: removed 'file' inheritance in the httpsendfile class Renato Cunha <renatoc@gmail.com> parents: 11876 diff changeset	451 Its purpose is to send file-like objects via HTTP and, to do so, it
e3526634d5a3 url.py: removed 'file' inheritance in the httpsendfile class Renato Cunha <renatoc@gmail.com> parents: 11876 diff changeset	452 defines a __len__ attribute to feed the Content-Length header.
e3526634d5a3 url.py: removed 'file' inheritance in the httpsendfile class Renato Cunha <renatoc@gmail.com> parents: 11876 diff changeset	453 """
e3526634d5a3 url.py: removed 'file' inheritance in the httpsendfile class Renato Cunha <renatoc@gmail.com> parents: 11876 diff changeset	454
13115 bda5f35fbf67 httpsendfile: record progress information during read() Augie Fackler <durin42@gmail.com> parents: 12906 diff changeset	455 def __init__(self, ui, args, *kwargs):
11880 e3526634d5a3 url.py: removed 'file' inheritance in the httpsendfile class Renato Cunha <renatoc@gmail.com> parents: 11876 diff changeset	456 # We can't just "self._data = open(args, *kwargs)" here because there
e3526634d5a3 url.py: removed 'file' inheritance in the httpsendfile class Renato Cunha <renatoc@gmail.com> parents: 11876 diff changeset	457 # is an "open" function defined in this module that shadows the global
e3526634d5a3 url.py: removed 'file' inheritance in the httpsendfile class Renato Cunha <renatoc@gmail.com> parents: 11876 diff changeset	458 # one
13115 bda5f35fbf67 httpsendfile: record progress information during read() Augie Fackler <durin42@gmail.com> parents: 12906 diff changeset	459 self.ui = ui
11880 e3526634d5a3 url.py: removed 'file' inheritance in the httpsendfile class Renato Cunha <renatoc@gmail.com> parents: 11876 diff changeset	460 self._data = __builtin__.open(args, *kwargs)
e3526634d5a3 url.py: removed 'file' inheritance in the httpsendfile class Renato Cunha <renatoc@gmail.com> parents: 11876 diff changeset	461 self.seek = self._data.seek
e3526634d5a3 url.py: removed 'file' inheritance in the httpsendfile class Renato Cunha <renatoc@gmail.com> parents: 11876 diff changeset	462 self.close = self._data.close
e3526634d5a3 url.py: removed 'file' inheritance in the httpsendfile class Renato Cunha <renatoc@gmail.com> parents: 11876 diff changeset	463 self.write = self._data.write
13115 bda5f35fbf67 httpsendfile: record progress information during read() Augie Fackler <durin42@gmail.com> parents: 12906 diff changeset	464 self._len = os.fstat(self._data.fileno()).st_size
bda5f35fbf67 httpsendfile: record progress information during read() Augie Fackler <durin42@gmail.com> parents: 12906 diff changeset	465 self._pos = 0
bda5f35fbf67 httpsendfile: record progress information during read() Augie Fackler <durin42@gmail.com> parents: 12906 diff changeset	466 self._total = len(self) / 1024 * 2
bda5f35fbf67 httpsendfile: record progress information during read() Augie Fackler <durin42@gmail.com> parents: 12906 diff changeset	467
bda5f35fbf67 httpsendfile: record progress information during read() Augie Fackler <durin42@gmail.com> parents: 12906 diff changeset	468 def read(self, args, *kwargs):
bda5f35fbf67 httpsendfile: record progress information during read() Augie Fackler <durin42@gmail.com> parents: 12906 diff changeset	469 try:
bda5f35fbf67 httpsendfile: record progress information during read() Augie Fackler <durin42@gmail.com> parents: 12906 diff changeset	470 ret = self._data.read(args, *kwargs)
bda5f35fbf67 httpsendfile: record progress information during read() Augie Fackler <durin42@gmail.com> parents: 12906 diff changeset	471 except EOFError:
bda5f35fbf67 httpsendfile: record progress information during read() Augie Fackler <durin42@gmail.com> parents: 12906 diff changeset	472 self.ui.progress(_('sending'), None)
bda5f35fbf67 httpsendfile: record progress information during read() Augie Fackler <durin42@gmail.com> parents: 12906 diff changeset	473 self._pos += len(ret)
bda5f35fbf67 httpsendfile: record progress information during read() Augie Fackler <durin42@gmail.com> parents: 12906 diff changeset	474 # We pass double the max for total because we currently have
bda5f35fbf67 httpsendfile: record progress information during read() Augie Fackler <durin42@gmail.com> parents: 12906 diff changeset	475 # to send the bundle twice in the case of a server that
bda5f35fbf67 httpsendfile: record progress information during read() Augie Fackler <durin42@gmail.com> parents: 12906 diff changeset	476 # requires authentication. Since we can't know until we try
bda5f35fbf67 httpsendfile: record progress information during read() Augie Fackler <durin42@gmail.com> parents: 12906 diff changeset	477 # once whether authentication will be required, just lie to
bda5f35fbf67 httpsendfile: record progress information during read() Augie Fackler <durin42@gmail.com> parents: 12906 diff changeset	478 # the user and maybe the push succeeds suddenly at 50%.
bda5f35fbf67 httpsendfile: record progress information during read() Augie Fackler <durin42@gmail.com> parents: 12906 diff changeset	479 self.ui.progress(_('sending'), self._pos / 1024,
bda5f35fbf67 httpsendfile: record progress information during read() Augie Fackler <durin42@gmail.com> parents: 12906 diff changeset	480 unit=_('kb'), total=self._total)
bda5f35fbf67 httpsendfile: record progress information during read() Augie Fackler <durin42@gmail.com> parents: 12906 diff changeset	481 return ret
11880 e3526634d5a3 url.py: removed 'file' inheritance in the httpsendfile class Renato Cunha <renatoc@gmail.com> parents: 11876 diff changeset	482
7270 2db33c1a5654 factor out the url handling from httprepo Benoit Boissinot <benoit.boissinot@ens-lyon.org> parents: diff changeset	483 def __len__(self):
13115 bda5f35fbf67 httpsendfile: record progress information during read() Augie Fackler <durin42@gmail.com> parents: 12906 diff changeset	484 return self._len
7270 2db33c1a5654 factor out the url handling from httprepo Benoit Boissinot <benoit.boissinot@ens-lyon.org> parents: diff changeset	485
13420 051f498628f7 url: refactor _gen_sendfile Mads Kiilerich <mads@kiilerich.com> parents: 13419 diff changeset	486 def _gen_sendfile(orgsend):
7270 2db33c1a5654 factor out the url handling from httprepo Benoit Boissinot <benoit.boissinot@ens-lyon.org> parents: diff changeset	487 def _sendfile(self, data):
2db33c1a5654 factor out the url handling from httprepo Benoit Boissinot <benoit.boissinot@ens-lyon.org> parents: diff changeset	488 # send a file
2db33c1a5654 factor out the url handling from httprepo Benoit Boissinot <benoit.boissinot@ens-lyon.org> parents: diff changeset	489 if isinstance(data, httpsendfile):
2db33c1a5654 factor out the url handling from httprepo Benoit Boissinot <benoit.boissinot@ens-lyon.org> parents: diff changeset	490 # if auth required, some data sent twice, so rewind here
2db33c1a5654 factor out the url handling from httprepo Benoit Boissinot <benoit.boissinot@ens-lyon.org> parents: diff changeset	491 data.seek(0)
2db33c1a5654 factor out the url handling from httprepo Benoit Boissinot <benoit.boissinot@ens-lyon.org> parents: diff changeset	492 for chunk in util.filechunkiter(data):
13420 051f498628f7 url: refactor _gen_sendfile Mads Kiilerich <mads@kiilerich.com> parents: 13419 diff changeset	493 orgsend(self, chunk)
7270 2db33c1a5654 factor out the url handling from httprepo Benoit Boissinot <benoit.boissinot@ens-lyon.org> parents: diff changeset	494 else:
13420 051f498628f7 url: refactor _gen_sendfile Mads Kiilerich <mads@kiilerich.com> parents: 13419 diff changeset	495 orgsend(self, data)
7270 2db33c1a5654 factor out the url handling from httprepo Benoit Boissinot <benoit.boissinot@ens-lyon.org> parents: diff changeset	496 return _sendfile
2db33c1a5654 factor out the url handling from httprepo Benoit Boissinot <benoit.boissinot@ens-lyon.org> parents: diff changeset	497
8590 59acb9c7d90f url: use CONNECT for HTTPS connections through HTTP proxy (issue967) Henrik Stuart <hg@hstuart.dk> parents: 8344 diff changeset	498 has_https = hasattr(urllib2, 'HTTPSHandler')
59acb9c7d90f url: use CONNECT for HTTPS connections through HTTP proxy (issue967) Henrik Stuart <hg@hstuart.dk> parents: 8344 diff changeset	499 if has_https:
59acb9c7d90f url: use CONNECT for HTTPS connections through HTTP proxy (issue967) Henrik Stuart <hg@hstuart.dk> parents: 8344 diff changeset	500 try:
59acb9c7d90f url: use CONNECT for HTTPS connections through HTTP proxy (issue967) Henrik Stuart <hg@hstuart.dk> parents: 8344 diff changeset	501 # avoid using deprecated/broken FakeSocket in python 2.6
59acb9c7d90f url: use CONNECT for HTTPS connections through HTTP proxy (issue967) Henrik Stuart <hg@hstuart.dk> parents: 8344 diff changeset	502 import ssl
59acb9c7d90f url: use CONNECT for HTTPS connections through HTTP proxy (issue967) Henrik Stuart <hg@hstuart.dk> parents: 8344 diff changeset	503 _ssl_wrap_socket = ssl.wrap_socket
10409 4c94a3df4b10 url: SSL server certificate verification using web.cacerts file (issue1174) Henrik Stuart <hg@hstuart.dk> parents: 10408 diff changeset	504 CERT_REQUIRED = ssl.CERT_REQUIRED
8590 59acb9c7d90f url: use CONNECT for HTTPS connections through HTTP proxy (issue967) Henrik Stuart <hg@hstuart.dk> parents: 8344 diff changeset	505 except ImportError:
10409 4c94a3df4b10 url: SSL server certificate verification using web.cacerts file (issue1174) Henrik Stuart <hg@hstuart.dk> parents: 10408 diff changeset	506 CERT_REQUIRED = 2
4c94a3df4b10 url: SSL server certificate verification using web.cacerts file (issue1174) Henrik Stuart <hg@hstuart.dk> parents: 10408 diff changeset	507
4c94a3df4b10 url: SSL server certificate verification using web.cacerts file (issue1174) Henrik Stuart <hg@hstuart.dk> parents: 10408 diff changeset	508 def _ssl_wrap_socket(sock, key_file, cert_file,
4c94a3df4b10 url: SSL server certificate verification using web.cacerts file (issue1174) Henrik Stuart <hg@hstuart.dk> parents: 10408 diff changeset	509 cert_reqs=CERT_REQUIRED, ca_certs=None):
4c94a3df4b10 url: SSL server certificate verification using web.cacerts file (issue1174) Henrik Stuart <hg@hstuart.dk> parents: 10408 diff changeset	510 if ca_certs:
4c94a3df4b10 url: SSL server certificate verification using web.cacerts file (issue1174) Henrik Stuart <hg@hstuart.dk> parents: 10408 diff changeset	511 raise util.Abort(_(
4c94a3df4b10 url: SSL server certificate verification using web.cacerts file (issue1174) Henrik Stuart <hg@hstuart.dk> parents: 10408 diff changeset	512 'certificate checking requires Python 2.6'))
4c94a3df4b10 url: SSL server certificate verification using web.cacerts file (issue1174) Henrik Stuart <hg@hstuart.dk> parents: 10408 diff changeset	513
8590 59acb9c7d90f url: use CONNECT for HTTPS connections through HTTP proxy (issue967) Henrik Stuart <hg@hstuart.dk> parents: 8344 diff changeset	514 ssl = socket.ssl(sock, key_file, cert_file)
59acb9c7d90f url: use CONNECT for HTTPS connections through HTTP proxy (issue967) Henrik Stuart <hg@hstuart.dk> parents: 8344 diff changeset	515 return httplib.FakeSocket(sock, ssl)
59acb9c7d90f url: use CONNECT for HTTPS connections through HTTP proxy (issue967) Henrik Stuart <hg@hstuart.dk> parents: 8344 diff changeset	516
10409 4c94a3df4b10 url: SSL server certificate verification using web.cacerts file (issue1174) Henrik Stuart <hg@hstuart.dk> parents: 10408 diff changeset	517 try:
4c94a3df4b10 url: SSL server certificate verification using web.cacerts file (issue1174) Henrik Stuart <hg@hstuart.dk> parents: 10408 diff changeset	518 _create_connection = socket.create_connection
10411 af4c42ec19ed ssl: fix compatibility with pre-2.6 Python Matt Mackall <mpm@selenic.com> parents: 10409 diff changeset	519 except AttributeError:
10482 95265afff99f url: fix python < 2.6 with ssl installed Benoit Boissinot <benoit.boissinot@ens-lyon.org> parents: 10415 diff changeset	520 _GLOBAL_DEFAULT_TIMEOUT = object()
95265afff99f url: fix python < 2.6 with ssl installed Benoit Boissinot <benoit.boissinot@ens-lyon.org> parents: 10415 diff changeset	521
10409 4c94a3df4b10 url: SSL server certificate verification using web.cacerts file (issue1174) Henrik Stuart <hg@hstuart.dk> parents: 10408 diff changeset	522 def _create_connection(address, timeout=_GLOBAL_DEFAULT_TIMEOUT,
4c94a3df4b10 url: SSL server certificate verification using web.cacerts file (issue1174) Henrik Stuart <hg@hstuart.dk> parents: 10408 diff changeset	523 source_address=None):
4c94a3df4b10 url: SSL server certificate verification using web.cacerts file (issue1174) Henrik Stuart <hg@hstuart.dk> parents: 10408 diff changeset	524 # lifted from Python 2.6
4c94a3df4b10 url: SSL server certificate verification using web.cacerts file (issue1174) Henrik Stuart <hg@hstuart.dk> parents: 10408 diff changeset	525
4c94a3df4b10 url: SSL server certificate verification using web.cacerts file (issue1174) Henrik Stuart <hg@hstuart.dk> parents: 10408 diff changeset	526 msg = "getaddrinfo returns an empty list"
4c94a3df4b10 url: SSL server certificate verification using web.cacerts file (issue1174) Henrik Stuart <hg@hstuart.dk> parents: 10408 diff changeset	527 host, port = address
4c94a3df4b10 url: SSL server certificate verification using web.cacerts file (issue1174) Henrik Stuart <hg@hstuart.dk> parents: 10408 diff changeset	528 for res in socket.getaddrinfo(host, port, 0, socket.SOCK_STREAM):
4c94a3df4b10 url: SSL server certificate verification using web.cacerts file (issue1174) Henrik Stuart <hg@hstuart.dk> parents: 10408 diff changeset	529 af, socktype, proto, canonname, sa = res
4c94a3df4b10 url: SSL server certificate verification using web.cacerts file (issue1174) Henrik Stuart <hg@hstuart.dk> parents: 10408 diff changeset	530 sock = None
4c94a3df4b10 url: SSL server certificate verification using web.cacerts file (issue1174) Henrik Stuart <hg@hstuart.dk> parents: 10408 diff changeset	531 try:
4c94a3df4b10 url: SSL server certificate verification using web.cacerts file (issue1174) Henrik Stuart <hg@hstuart.dk> parents: 10408 diff changeset	532 sock = socket.socket(af, socktype, proto)
4c94a3df4b10 url: SSL server certificate verification using web.cacerts file (issue1174) Henrik Stuart <hg@hstuart.dk> parents: 10408 diff changeset	533 if timeout is not _GLOBAL_DEFAULT_TIMEOUT:
4c94a3df4b10 url: SSL server certificate verification using web.cacerts file (issue1174) Henrik Stuart <hg@hstuart.dk> parents: 10408 diff changeset	534 sock.settimeout(timeout)
4c94a3df4b10 url: SSL server certificate verification using web.cacerts file (issue1174) Henrik Stuart <hg@hstuart.dk> parents: 10408 diff changeset	535 if source_address:
4c94a3df4b10 url: SSL server certificate verification using web.cacerts file (issue1174) Henrik Stuart <hg@hstuart.dk> parents: 10408 diff changeset	536 sock.bind(source_address)
4c94a3df4b10 url: SSL server certificate verification using web.cacerts file (issue1174) Henrik Stuart <hg@hstuart.dk> parents: 10408 diff changeset	537 sock.connect(sa)
4c94a3df4b10 url: SSL server certificate verification using web.cacerts file (issue1174) Henrik Stuart <hg@hstuart.dk> parents: 10408 diff changeset	538 return sock
4c94a3df4b10 url: SSL server certificate verification using web.cacerts file (issue1174) Henrik Stuart <hg@hstuart.dk> parents: 10408 diff changeset	539
4c94a3df4b10 url: SSL server certificate verification using web.cacerts file (issue1174) Henrik Stuart <hg@hstuart.dk> parents: 10408 diff changeset	540 except socket.error, msg:
4c94a3df4b10 url: SSL server certificate verification using web.cacerts file (issue1174) Henrik Stuart <hg@hstuart.dk> parents: 10408 diff changeset	541 if sock is not None:
4c94a3df4b10 url: SSL server certificate verification using web.cacerts file (issue1174) Henrik Stuart <hg@hstuart.dk> parents: 10408 diff changeset	542 sock.close()
4c94a3df4b10 url: SSL server certificate verification using web.cacerts file (issue1174) Henrik Stuart <hg@hstuart.dk> parents: 10408 diff changeset	543
4c94a3df4b10 url: SSL server certificate verification using web.cacerts file (issue1174) Henrik Stuart <hg@hstuart.dk> parents: 10408 diff changeset	544 raise socket.error, msg
4c94a3df4b10 url: SSL server certificate verification using web.cacerts file (issue1174) Henrik Stuart <hg@hstuart.dk> parents: 10408 diff changeset	545
7270 2db33c1a5654 factor out the url handling from httprepo Benoit Boissinot <benoit.boissinot@ens-lyon.org> parents: diff changeset	546 class httpconnection(keepalive.HTTPConnection):
2db33c1a5654 factor out the url handling from httprepo Benoit Boissinot <benoit.boissinot@ens-lyon.org> parents: diff changeset	547 # must be able to send big bundle as stream.
13420 051f498628f7 url: refactor _gen_sendfile Mads Kiilerich <mads@kiilerich.com> parents: 13419 diff changeset	548 send = _gen_sendfile(keepalive.HTTPConnection.send)
7270 2db33c1a5654 factor out the url handling from httprepo Benoit Boissinot <benoit.boissinot@ens-lyon.org> parents: diff changeset	549
8590 59acb9c7d90f url: use CONNECT for HTTPS connections through HTTP proxy (issue967) Henrik Stuart <hg@hstuart.dk> parents: 8344 diff changeset	550 def connect(self):
10415 677f15da38c1 url: proxy handling, simplify and correctly deal with IPv6 Benoit Boissinot <benoit.boissinot@ens-lyon.org> parents: 10411 diff changeset	551 if has_https and self.realhostport: # use CONNECT proxy
8590 59acb9c7d90f url: use CONNECT for HTTPS connections through HTTP proxy (issue967) Henrik Stuart <hg@hstuart.dk> parents: 8344 diff changeset	552 self.sock = socket.socket(socket.AF_INET, socket.SOCK_STREAM)
59acb9c7d90f url: use CONNECT for HTTPS connections through HTTP proxy (issue967) Henrik Stuart <hg@hstuart.dk> parents: 8344 diff changeset	553 self.sock.connect((self.host, self.port))
9852 917cf6bb6d0c url: generalise HTTPS proxy handling to accomodate Python changes Henrik Stuart <hg@hstuart.dk> parents: 9726 diff changeset	554 if _generic_proxytunnel(self):
8590 59acb9c7d90f url: use CONNECT for HTTPS connections through HTTP proxy (issue967) Henrik Stuart <hg@hstuart.dk> parents: 8344 diff changeset	555 # we do not support client x509 certificates
59acb9c7d90f url: use CONNECT for HTTPS connections through HTTP proxy (issue967) Henrik Stuart <hg@hstuart.dk> parents: 8344 diff changeset	556 self.sock = _ssl_wrap_socket(self.sock, None, None)
59acb9c7d90f url: use CONNECT for HTTPS connections through HTTP proxy (issue967) Henrik Stuart <hg@hstuart.dk> parents: 8344 diff changeset	557 else:
59acb9c7d90f url: use CONNECT for HTTPS connections through HTTP proxy (issue967) Henrik Stuart <hg@hstuart.dk> parents: 8344 diff changeset	558 keepalive.HTTPConnection.connect(self)
59acb9c7d90f url: use CONNECT for HTTPS connections through HTTP proxy (issue967) Henrik Stuart <hg@hstuart.dk> parents: 8344 diff changeset	559
59acb9c7d90f url: use CONNECT for HTTPS connections through HTTP proxy (issue967) Henrik Stuart <hg@hstuart.dk> parents: 8344 diff changeset	560 def getresponse(self):
59acb9c7d90f url: use CONNECT for HTTPS connections through HTTP proxy (issue967) Henrik Stuart <hg@hstuart.dk> parents: 8344 diff changeset	561 proxyres = getattr(self, 'proxyres', None)
59acb9c7d90f url: use CONNECT for HTTPS connections through HTTP proxy (issue967) Henrik Stuart <hg@hstuart.dk> parents: 8344 diff changeset	562 if proxyres:
59acb9c7d90f url: use CONNECT for HTTPS connections through HTTP proxy (issue967) Henrik Stuart <hg@hstuart.dk> parents: 8344 diff changeset	563 if proxyres.will_close:
59acb9c7d90f url: use CONNECT for HTTPS connections through HTTP proxy (issue967) Henrik Stuart <hg@hstuart.dk> parents: 8344 diff changeset	564 self.close()
59acb9c7d90f url: use CONNECT for HTTPS connections through HTTP proxy (issue967) Henrik Stuart <hg@hstuart.dk> parents: 8344 diff changeset	565 self.proxyres = None
59acb9c7d90f url: use CONNECT for HTTPS connections through HTTP proxy (issue967) Henrik Stuart <hg@hstuart.dk> parents: 8344 diff changeset	566 return proxyres
59acb9c7d90f url: use CONNECT for HTTPS connections through HTTP proxy (issue967) Henrik Stuart <hg@hstuart.dk> parents: 8344 diff changeset	567 return keepalive.HTTPConnection.getresponse(self)
59acb9c7d90f url: use CONNECT for HTTPS connections through HTTP proxy (issue967) Henrik Stuart <hg@hstuart.dk> parents: 8344 diff changeset	568
9852 917cf6bb6d0c url: generalise HTTPS proxy handling to accomodate Python changes Henrik Stuart <hg@hstuart.dk> parents: 9726 diff changeset	569 # general transaction handler to support different ways to handle
917cf6bb6d0c url: generalise HTTPS proxy handling to accomodate Python changes Henrik Stuart <hg@hstuart.dk> parents: 9726 diff changeset	570 # HTTPS proxying before and after Python 2.6.3.
917cf6bb6d0c url: generalise HTTPS proxy handling to accomodate Python changes Henrik Stuart <hg@hstuart.dk> parents: 9726 diff changeset	571 def _generic_start_transaction(handler, h, req):
917cf6bb6d0c url: generalise HTTPS proxy handling to accomodate Python changes Henrik Stuart <hg@hstuart.dk> parents: 9726 diff changeset	572 if hasattr(req, '_tunnel_host') and req._tunnel_host:
917cf6bb6d0c url: generalise HTTPS proxy handling to accomodate Python changes Henrik Stuart <hg@hstuart.dk> parents: 9726 diff changeset	573 tunnel_host = req._tunnel_host
917cf6bb6d0c url: generalise HTTPS proxy handling to accomodate Python changes Henrik Stuart <hg@hstuart.dk> parents: 9726 diff changeset	574 if tunnel_host[:7] not in ['http://', 'https:/']:
917cf6bb6d0c url: generalise HTTPS proxy handling to accomodate Python changes Henrik Stuart <hg@hstuart.dk> parents: 9726 diff changeset	575 tunnel_host = 'https://' + tunnel_host
917cf6bb6d0c url: generalise HTTPS proxy handling to accomodate Python changes Henrik Stuart <hg@hstuart.dk> parents: 9726 diff changeset	576 new_tunnel = True
917cf6bb6d0c url: generalise HTTPS proxy handling to accomodate Python changes Henrik Stuart <hg@hstuart.dk> parents: 9726 diff changeset	577 else:
917cf6bb6d0c url: generalise HTTPS proxy handling to accomodate Python changes Henrik Stuart <hg@hstuart.dk> parents: 9726 diff changeset	578 tunnel_host = req.get_selector()
917cf6bb6d0c url: generalise HTTPS proxy handling to accomodate Python changes Henrik Stuart <hg@hstuart.dk> parents: 9726 diff changeset	579 new_tunnel = False
917cf6bb6d0c url: generalise HTTPS proxy handling to accomodate Python changes Henrik Stuart <hg@hstuart.dk> parents: 9726 diff changeset	580
917cf6bb6d0c url: generalise HTTPS proxy handling to accomodate Python changes Henrik Stuart <hg@hstuart.dk> parents: 9726 diff changeset	581 if new_tunnel or tunnel_host == req.get_full_url(): # has proxy
917cf6bb6d0c url: generalise HTTPS proxy handling to accomodate Python changes Henrik Stuart <hg@hstuart.dk> parents: 9726 diff changeset	582 urlparts = urlparse.urlparse(tunnel_host)
917cf6bb6d0c url: generalise HTTPS proxy handling to accomodate Python changes Henrik Stuart <hg@hstuart.dk> parents: 9726 diff changeset	583 if new_tunnel or urlparts[0] == 'https': # only use CONNECT for HTTPS
10415 677f15da38c1 url: proxy handling, simplify and correctly deal with IPv6 Benoit Boissinot <benoit.boissinot@ens-lyon.org> parents: 10411 diff changeset	584 realhostport = urlparts[1]
677f15da38c1 url: proxy handling, simplify and correctly deal with IPv6 Benoit Boissinot <benoit.boissinot@ens-lyon.org> parents: 10411 diff changeset	585 if realhostport[-1] == ']' or ':' not in realhostport:
677f15da38c1 url: proxy handling, simplify and correctly deal with IPv6 Benoit Boissinot <benoit.boissinot@ens-lyon.org> parents: 10411 diff changeset	586 realhostport += ':443'
9852 917cf6bb6d0c url: generalise HTTPS proxy handling to accomodate Python changes Henrik Stuart <hg@hstuart.dk> parents: 9726 diff changeset	587
10415 677f15da38c1 url: proxy handling, simplify and correctly deal with IPv6 Benoit Boissinot <benoit.boissinot@ens-lyon.org> parents: 10411 diff changeset	588 h.realhostport = realhostport
9852 917cf6bb6d0c url: generalise HTTPS proxy handling to accomodate Python changes Henrik Stuart <hg@hstuart.dk> parents: 9726 diff changeset	589 h.headers = req.headers.copy()
917cf6bb6d0c url: generalise HTTPS proxy handling to accomodate Python changes Henrik Stuart <hg@hstuart.dk> parents: 9726 diff changeset	590 h.headers.update(handler.parent.addheaders)
917cf6bb6d0c url: generalise HTTPS proxy handling to accomodate Python changes Henrik Stuart <hg@hstuart.dk> parents: 9726 diff changeset	591 return
917cf6bb6d0c url: generalise HTTPS proxy handling to accomodate Python changes Henrik Stuart <hg@hstuart.dk> parents: 9726 diff changeset	592
10415 677f15da38c1 url: proxy handling, simplify and correctly deal with IPv6 Benoit Boissinot <benoit.boissinot@ens-lyon.org> parents: 10411 diff changeset	593 h.realhostport = None
9852 917cf6bb6d0c url: generalise HTTPS proxy handling to accomodate Python changes Henrik Stuart <hg@hstuart.dk> parents: 9726 diff changeset	594 h.headers = None
917cf6bb6d0c url: generalise HTTPS proxy handling to accomodate Python changes Henrik Stuart <hg@hstuart.dk> parents: 9726 diff changeset	595
917cf6bb6d0c url: generalise HTTPS proxy handling to accomodate Python changes Henrik Stuart <hg@hstuart.dk> parents: 9726 diff changeset	596 def _generic_proxytunnel(self):
917cf6bb6d0c url: generalise HTTPS proxy handling to accomodate Python changes Henrik Stuart <hg@hstuart.dk> parents: 9726 diff changeset	597 proxyheaders = dict(
917cf6bb6d0c url: generalise HTTPS proxy handling to accomodate Python changes Henrik Stuart <hg@hstuart.dk> parents: 9726 diff changeset	598 [(x, self.headers[x]) for x in self.headers
917cf6bb6d0c url: generalise HTTPS proxy handling to accomodate Python changes Henrik Stuart <hg@hstuart.dk> parents: 9726 diff changeset	599 if x.lower().startswith('proxy-')])
917cf6bb6d0c url: generalise HTTPS proxy handling to accomodate Python changes Henrik Stuart <hg@hstuart.dk> parents: 9726 diff changeset	600 self._set_hostport(self.host, self.port)
10415 677f15da38c1 url: proxy handling, simplify and correctly deal with IPv6 Benoit Boissinot <benoit.boissinot@ens-lyon.org> parents: 10411 diff changeset	601 self.send('CONNECT %s HTTP/1.0\r\n' % self.realhostport)
9852 917cf6bb6d0c url: generalise HTTPS proxy handling to accomodate Python changes Henrik Stuart <hg@hstuart.dk> parents: 9726 diff changeset	602 for header in proxyheaders.iteritems():
917cf6bb6d0c url: generalise HTTPS proxy handling to accomodate Python changes Henrik Stuart <hg@hstuart.dk> parents: 9726 diff changeset	603 self.send('%s: %s\r\n' % header)
917cf6bb6d0c url: generalise HTTPS proxy handling to accomodate Python changes Henrik Stuart <hg@hstuart.dk> parents: 9726 diff changeset	604 self.send('\r\n')
917cf6bb6d0c url: generalise HTTPS proxy handling to accomodate Python changes Henrik Stuart <hg@hstuart.dk> parents: 9726 diff changeset	605
917cf6bb6d0c url: generalise HTTPS proxy handling to accomodate Python changes Henrik Stuart <hg@hstuart.dk> parents: 9726 diff changeset	606 # majority of the following code is duplicated from
917cf6bb6d0c url: generalise HTTPS proxy handling to accomodate Python changes Henrik Stuart <hg@hstuart.dk> parents: 9726 diff changeset	607 # httplib.HTTPConnection as there are no adequate places to
917cf6bb6d0c url: generalise HTTPS proxy handling to accomodate Python changes Henrik Stuart <hg@hstuart.dk> parents: 9726 diff changeset	608 # override functions to provide the needed functionality
917cf6bb6d0c url: generalise HTTPS proxy handling to accomodate Python changes Henrik Stuart <hg@hstuart.dk> parents: 9726 diff changeset	609 res = self.response_class(self.sock,
917cf6bb6d0c url: generalise HTTPS proxy handling to accomodate Python changes Henrik Stuart <hg@hstuart.dk> parents: 9726 diff changeset	610 strict=self.strict,
917cf6bb6d0c url: generalise HTTPS proxy handling to accomodate Python changes Henrik Stuart <hg@hstuart.dk> parents: 9726 diff changeset	611 method=self._method)
917cf6bb6d0c url: generalise HTTPS proxy handling to accomodate Python changes Henrik Stuart <hg@hstuart.dk> parents: 9726 diff changeset	612
917cf6bb6d0c url: generalise HTTPS proxy handling to accomodate Python changes Henrik Stuart <hg@hstuart.dk> parents: 9726 diff changeset	613 while True:
917cf6bb6d0c url: generalise HTTPS proxy handling to accomodate Python changes Henrik Stuart <hg@hstuart.dk> parents: 9726 diff changeset	614 version, status, reason = res._read_status()
917cf6bb6d0c url: generalise HTTPS proxy handling to accomodate Python changes Henrik Stuart <hg@hstuart.dk> parents: 9726 diff changeset	615 if status != httplib.CONTINUE:
917cf6bb6d0c url: generalise HTTPS proxy handling to accomodate Python changes Henrik Stuart <hg@hstuart.dk> parents: 9726 diff changeset	616 break
917cf6bb6d0c url: generalise HTTPS proxy handling to accomodate Python changes Henrik Stuart <hg@hstuart.dk> parents: 9726 diff changeset	617 while True:
917cf6bb6d0c url: generalise HTTPS proxy handling to accomodate Python changes Henrik Stuart <hg@hstuart.dk> parents: 9726 diff changeset	618 skip = res.fp.readline().strip()
917cf6bb6d0c url: generalise HTTPS proxy handling to accomodate Python changes Henrik Stuart <hg@hstuart.dk> parents: 9726 diff changeset	619 if not skip:
917cf6bb6d0c url: generalise HTTPS proxy handling to accomodate Python changes Henrik Stuart <hg@hstuart.dk> parents: 9726 diff changeset	620 break
917cf6bb6d0c url: generalise HTTPS proxy handling to accomodate Python changes Henrik Stuart <hg@hstuart.dk> parents: 9726 diff changeset	621 res.status = status
917cf6bb6d0c url: generalise HTTPS proxy handling to accomodate Python changes Henrik Stuart <hg@hstuart.dk> parents: 9726 diff changeset	622 res.reason = reason.strip()
917cf6bb6d0c url: generalise HTTPS proxy handling to accomodate Python changes Henrik Stuart <hg@hstuart.dk> parents: 9726 diff changeset	623
917cf6bb6d0c url: generalise HTTPS proxy handling to accomodate Python changes Henrik Stuart <hg@hstuart.dk> parents: 9726 diff changeset	624 if res.status == 200:
917cf6bb6d0c url: generalise HTTPS proxy handling to accomodate Python changes Henrik Stuart <hg@hstuart.dk> parents: 9726 diff changeset	625 while True:
917cf6bb6d0c url: generalise HTTPS proxy handling to accomodate Python changes Henrik Stuart <hg@hstuart.dk> parents: 9726 diff changeset	626 line = res.fp.readline()
917cf6bb6d0c url: generalise HTTPS proxy handling to accomodate Python changes Henrik Stuart <hg@hstuart.dk> parents: 9726 diff changeset	627 if line == '\r\n':
917cf6bb6d0c url: generalise HTTPS proxy handling to accomodate Python changes Henrik Stuart <hg@hstuart.dk> parents: 9726 diff changeset	628 break
917cf6bb6d0c url: generalise HTTPS proxy handling to accomodate Python changes Henrik Stuart <hg@hstuart.dk> parents: 9726 diff changeset	629 return True
917cf6bb6d0c url: generalise HTTPS proxy handling to accomodate Python changes Henrik Stuart <hg@hstuart.dk> parents: 9726 diff changeset	630
917cf6bb6d0c url: generalise HTTPS proxy handling to accomodate Python changes Henrik Stuart <hg@hstuart.dk> parents: 9726 diff changeset	631 if version == 'HTTP/1.0':
917cf6bb6d0c url: generalise HTTPS proxy handling to accomodate Python changes Henrik Stuart <hg@hstuart.dk> parents: 9726 diff changeset	632 res.version = 10
917cf6bb6d0c url: generalise HTTPS proxy handling to accomodate Python changes Henrik Stuart <hg@hstuart.dk> parents: 9726 diff changeset	633 elif version.startswith('HTTP/1.'):
917cf6bb6d0c url: generalise HTTPS proxy handling to accomodate Python changes Henrik Stuart <hg@hstuart.dk> parents: 9726 diff changeset	634 res.version = 11
917cf6bb6d0c url: generalise HTTPS proxy handling to accomodate Python changes Henrik Stuart <hg@hstuart.dk> parents: 9726 diff changeset	635 elif version == 'HTTP/0.9':
917cf6bb6d0c url: generalise HTTPS proxy handling to accomodate Python changes Henrik Stuart <hg@hstuart.dk> parents: 9726 diff changeset	636 res.version = 9
917cf6bb6d0c url: generalise HTTPS proxy handling to accomodate Python changes Henrik Stuart <hg@hstuart.dk> parents: 9726 diff changeset	637 else:
917cf6bb6d0c url: generalise HTTPS proxy handling to accomodate Python changes Henrik Stuart <hg@hstuart.dk> parents: 9726 diff changeset	638 raise httplib.UnknownProtocol(version)
917cf6bb6d0c url: generalise HTTPS proxy handling to accomodate Python changes Henrik Stuart <hg@hstuart.dk> parents: 9726 diff changeset	639
917cf6bb6d0c url: generalise HTTPS proxy handling to accomodate Python changes Henrik Stuart <hg@hstuart.dk> parents: 9726 diff changeset	640 if res.version == 9:
917cf6bb6d0c url: generalise HTTPS proxy handling to accomodate Python changes Henrik Stuart <hg@hstuart.dk> parents: 9726 diff changeset	641 res.length = None
917cf6bb6d0c url: generalise HTTPS proxy handling to accomodate Python changes Henrik Stuart <hg@hstuart.dk> parents: 9726 diff changeset	642 res.chunked = 0
917cf6bb6d0c url: generalise HTTPS proxy handling to accomodate Python changes Henrik Stuart <hg@hstuart.dk> parents: 9726 diff changeset	643 res.will_close = 1
917cf6bb6d0c url: generalise HTTPS proxy handling to accomodate Python changes Henrik Stuart <hg@hstuart.dk> parents: 9726 diff changeset	644 res.msg = httplib.HTTPMessage(cStringIO.StringIO())
917cf6bb6d0c url: generalise HTTPS proxy handling to accomodate Python changes Henrik Stuart <hg@hstuart.dk> parents: 9726 diff changeset	645 return False
917cf6bb6d0c url: generalise HTTPS proxy handling to accomodate Python changes Henrik Stuart <hg@hstuart.dk> parents: 9726 diff changeset	646
917cf6bb6d0c url: generalise HTTPS proxy handling to accomodate Python changes Henrik Stuart <hg@hstuart.dk> parents: 9726 diff changeset	647 res.msg = httplib.HTTPMessage(res.fp)
917cf6bb6d0c url: generalise HTTPS proxy handling to accomodate Python changes Henrik Stuart <hg@hstuart.dk> parents: 9726 diff changeset	648 res.msg.fp = None
917cf6bb6d0c url: generalise HTTPS proxy handling to accomodate Python changes Henrik Stuart <hg@hstuart.dk> parents: 9726 diff changeset	649
917cf6bb6d0c url: generalise HTTPS proxy handling to accomodate Python changes Henrik Stuart <hg@hstuart.dk> parents: 9726 diff changeset	650 # are we using the chunked-style of transfer encoding?
917cf6bb6d0c url: generalise HTTPS proxy handling to accomodate Python changes Henrik Stuart <hg@hstuart.dk> parents: 9726 diff changeset	651 trenc = res.msg.getheader('transfer-encoding')
917cf6bb6d0c url: generalise HTTPS proxy handling to accomodate Python changes Henrik Stuart <hg@hstuart.dk> parents: 9726 diff changeset	652 if trenc and trenc.lower() == "chunked":
917cf6bb6d0c url: generalise HTTPS proxy handling to accomodate Python changes Henrik Stuart <hg@hstuart.dk> parents: 9726 diff changeset	653 res.chunked = 1
917cf6bb6d0c url: generalise HTTPS proxy handling to accomodate Python changes Henrik Stuart <hg@hstuart.dk> parents: 9726 diff changeset	654 res.chunk_left = None
917cf6bb6d0c url: generalise HTTPS proxy handling to accomodate Python changes Henrik Stuart <hg@hstuart.dk> parents: 9726 diff changeset	655 else:
917cf6bb6d0c url: generalise HTTPS proxy handling to accomodate Python changes Henrik Stuart <hg@hstuart.dk> parents: 9726 diff changeset	656 res.chunked = 0
917cf6bb6d0c url: generalise HTTPS proxy handling to accomodate Python changes Henrik Stuart <hg@hstuart.dk> parents: 9726 diff changeset	657
917cf6bb6d0c url: generalise HTTPS proxy handling to accomodate Python changes Henrik Stuart <hg@hstuart.dk> parents: 9726 diff changeset	658 # will the connection close at the end of the response?
917cf6bb6d0c url: generalise HTTPS proxy handling to accomodate Python changes Henrik Stuart <hg@hstuart.dk> parents: 9726 diff changeset	659 res.will_close = res._check_close()
917cf6bb6d0c url: generalise HTTPS proxy handling to accomodate Python changes Henrik Stuart <hg@hstuart.dk> parents: 9726 diff changeset	660
917cf6bb6d0c url: generalise HTTPS proxy handling to accomodate Python changes Henrik Stuart <hg@hstuart.dk> parents: 9726 diff changeset	661 # do we have a Content-Length?
917cf6bb6d0c url: generalise HTTPS proxy handling to accomodate Python changes Henrik Stuart <hg@hstuart.dk> parents: 9726 diff changeset	662 # NOTE: RFC 2616, S4.4, #3 says we ignore this if tr_enc is "chunked"
917cf6bb6d0c url: generalise HTTPS proxy handling to accomodate Python changes Henrik Stuart <hg@hstuart.dk> parents: 9726 diff changeset	663 length = res.msg.getheader('content-length')
917cf6bb6d0c url: generalise HTTPS proxy handling to accomodate Python changes Henrik Stuart <hg@hstuart.dk> parents: 9726 diff changeset	664 if length and not res.chunked:
917cf6bb6d0c url: generalise HTTPS proxy handling to accomodate Python changes Henrik Stuart <hg@hstuart.dk> parents: 9726 diff changeset	665 try:
917cf6bb6d0c url: generalise HTTPS proxy handling to accomodate Python changes Henrik Stuart <hg@hstuart.dk> parents: 9726 diff changeset	666 res.length = int(length)
917cf6bb6d0c url: generalise HTTPS proxy handling to accomodate Python changes Henrik Stuart <hg@hstuart.dk> parents: 9726 diff changeset	667 except ValueError:
917cf6bb6d0c url: generalise HTTPS proxy handling to accomodate Python changes Henrik Stuart <hg@hstuart.dk> parents: 9726 diff changeset	668 res.length = None
917cf6bb6d0c url: generalise HTTPS proxy handling to accomodate Python changes Henrik Stuart <hg@hstuart.dk> parents: 9726 diff changeset	669 else:
917cf6bb6d0c url: generalise HTTPS proxy handling to accomodate Python changes Henrik Stuart <hg@hstuart.dk> parents: 9726 diff changeset	670 if res.length < 0: # ignore nonsensical negative lengths
917cf6bb6d0c url: generalise HTTPS proxy handling to accomodate Python changes Henrik Stuart <hg@hstuart.dk> parents: 9726 diff changeset	671 res.length = None
917cf6bb6d0c url: generalise HTTPS proxy handling to accomodate Python changes Henrik Stuart <hg@hstuart.dk> parents: 9726 diff changeset	672 else:
917cf6bb6d0c url: generalise HTTPS proxy handling to accomodate Python changes Henrik Stuart <hg@hstuart.dk> parents: 9726 diff changeset	673 res.length = None
917cf6bb6d0c url: generalise HTTPS proxy handling to accomodate Python changes Henrik Stuart <hg@hstuart.dk> parents: 9726 diff changeset	674
917cf6bb6d0c url: generalise HTTPS proxy handling to accomodate Python changes Henrik Stuart <hg@hstuart.dk> parents: 9726 diff changeset	675 # does the body have a fixed length? (of zero)
917cf6bb6d0c url: generalise HTTPS proxy handling to accomodate Python changes Henrik Stuart <hg@hstuart.dk> parents: 9726 diff changeset	676 if (status == httplib.NO_CONTENT or status == httplib.NOT_MODIFIED or
917cf6bb6d0c url: generalise HTTPS proxy handling to accomodate Python changes Henrik Stuart <hg@hstuart.dk> parents: 9726 diff changeset	677 100 <= status < 200 or # 1xx codes
917cf6bb6d0c url: generalise HTTPS proxy handling to accomodate Python changes Henrik Stuart <hg@hstuart.dk> parents: 9726 diff changeset	678 res._method == 'HEAD'):
917cf6bb6d0c url: generalise HTTPS proxy handling to accomodate Python changes Henrik Stuart <hg@hstuart.dk> parents: 9726 diff changeset	679 res.length = 0
917cf6bb6d0c url: generalise HTTPS proxy handling to accomodate Python changes Henrik Stuart <hg@hstuart.dk> parents: 9726 diff changeset	680
917cf6bb6d0c url: generalise HTTPS proxy handling to accomodate Python changes Henrik Stuart <hg@hstuart.dk> parents: 9726 diff changeset	681 # if the connection remains open, and we aren't using chunked, and
917cf6bb6d0c url: generalise HTTPS proxy handling to accomodate Python changes Henrik Stuart <hg@hstuart.dk> parents: 9726 diff changeset	682 # a content-length was not provided, then assume that the connection
917cf6bb6d0c url: generalise HTTPS proxy handling to accomodate Python changes Henrik Stuart <hg@hstuart.dk> parents: 9726 diff changeset	683 # WILL close.
917cf6bb6d0c url: generalise HTTPS proxy handling to accomodate Python changes Henrik Stuart <hg@hstuart.dk> parents: 9726 diff changeset	684 if (not res.will_close and
917cf6bb6d0c url: generalise HTTPS proxy handling to accomodate Python changes Henrik Stuart <hg@hstuart.dk> parents: 9726 diff changeset	685 not res.chunked and
917cf6bb6d0c url: generalise HTTPS proxy handling to accomodate Python changes Henrik Stuart <hg@hstuart.dk> parents: 9726 diff changeset	686 res.length is None):
917cf6bb6d0c url: generalise HTTPS proxy handling to accomodate Python changes Henrik Stuart <hg@hstuart.dk> parents: 9726 diff changeset	687 res.will_close = 1
917cf6bb6d0c url: generalise HTTPS proxy handling to accomodate Python changes Henrik Stuart <hg@hstuart.dk> parents: 9726 diff changeset	688
917cf6bb6d0c url: generalise HTTPS proxy handling to accomodate Python changes Henrik Stuart <hg@hstuart.dk> parents: 9726 diff changeset	689 self.proxyres = res
917cf6bb6d0c url: generalise HTTPS proxy handling to accomodate Python changes Henrik Stuart <hg@hstuart.dk> parents: 9726 diff changeset	690
917cf6bb6d0c url: generalise HTTPS proxy handling to accomodate Python changes Henrik Stuart <hg@hstuart.dk> parents: 9726 diff changeset	691 return False
917cf6bb6d0c url: generalise HTTPS proxy handling to accomodate Python changes Henrik Stuart <hg@hstuart.dk> parents: 9726 diff changeset	692
7270 2db33c1a5654 factor out the url handling from httprepo Benoit Boissinot <benoit.boissinot@ens-lyon.org> parents: diff changeset	693 class httphandler(keepalive.HTTPHandler):
2db33c1a5654 factor out the url handling from httprepo Benoit Boissinot <benoit.boissinot@ens-lyon.org> parents: diff changeset	694 def http_open(self, req):
2db33c1a5654 factor out the url handling from httprepo Benoit Boissinot <benoit.boissinot@ens-lyon.org> parents: diff changeset	695 return self.do_open(httpconnection, req)
2db33c1a5654 factor out the url handling from httprepo Benoit Boissinot <benoit.boissinot@ens-lyon.org> parents: diff changeset	696
8590 59acb9c7d90f url: use CONNECT for HTTPS connections through HTTP proxy (issue967) Henrik Stuart <hg@hstuart.dk> parents: 8344 diff changeset	697 def _start_transaction(self, h, req):
9852 917cf6bb6d0c url: generalise HTTPS proxy handling to accomodate Python changes Henrik Stuart <hg@hstuart.dk> parents: 9726 diff changeset	698 _generic_start_transaction(self, h, req)
8590 59acb9c7d90f url: use CONNECT for HTTPS connections through HTTP proxy (issue967) Henrik Stuart <hg@hstuart.dk> parents: 8344 diff changeset	699 return keepalive.HTTPHandler._start_transaction(self, h, req)
59acb9c7d90f url: use CONNECT for HTTPS connections through HTTP proxy (issue967) Henrik Stuart <hg@hstuart.dk> parents: 8344 diff changeset	700
12592 f2937d6492c5 url: verify correctness of https server certificates (issue2407) Mads Kiilerich <mads@kiilerich.com> parents: 12391 diff changeset	701 def _verifycert(cert, hostname):
12742 6ab4a7d3c179 url: validity (notBefore/notAfter) is checked by OpenSSL (issue2407) Mads Kiilerich <mads@kiilerich.com> parents: 12607 diff changeset	702 '''Verify that cert (in socket.getpeercert() format) matches hostname.
13249 75d0c38a0bca url: check subjectAltName when verifying ssl certificate Yuya Nishihara <yuya@tcha.org> parents: 13248 diff changeset	703 CRLs is not handled.
12770 614f0d8724ab check-code: find trailing whitespace Martin Geisler <mg@lazybytes.net> parents: 12742 diff changeset	704
12592 f2937d6492c5 url: verify correctness of https server certificates (issue2407) Mads Kiilerich <mads@kiilerich.com> parents: 12391 diff changeset	705 Returns error message if any problems are found and None on success.
f2937d6492c5 url: verify correctness of https server certificates (issue2407) Mads Kiilerich <mads@kiilerich.com> parents: 12391 diff changeset	706 '''
f2937d6492c5 url: verify correctness of https server certificates (issue2407) Mads Kiilerich <mads@kiilerich.com> parents: 12391 diff changeset	707 if not cert:
f2937d6492c5 url: verify correctness of https server certificates (issue2407) Mads Kiilerich <mads@kiilerich.com> parents: 12391 diff changeset	708 return _('no certificate received')
f2937d6492c5 url: verify correctness of https server certificates (issue2407) Mads Kiilerich <mads@kiilerich.com> parents: 12391 diff changeset	709 dnsname = hostname.lower()
13249 75d0c38a0bca url: check subjectAltName when verifying ssl certificate Yuya Nishihara <yuya@tcha.org> parents: 13248 diff changeset	710 def matchdnsname(certname):
75d0c38a0bca url: check subjectAltName when verifying ssl certificate Yuya Nishihara <yuya@tcha.org> parents: 13248 diff changeset	711 return (certname == dnsname or
75d0c38a0bca url: check subjectAltName when verifying ssl certificate Yuya Nishihara <yuya@tcha.org> parents: 13248 diff changeset	712 '.' in dnsname and certname == '*.' + dnsname.split('.', 1)[1])
75d0c38a0bca url: check subjectAltName when verifying ssl certificate Yuya Nishihara <yuya@tcha.org> parents: 13248 diff changeset	713
75d0c38a0bca url: check subjectAltName when verifying ssl certificate Yuya Nishihara <yuya@tcha.org> parents: 13248 diff changeset	714 san = cert.get('subjectAltName', [])
75d0c38a0bca url: check subjectAltName when verifying ssl certificate Yuya Nishihara <yuya@tcha.org> parents: 13248 diff changeset	715 if san:
75d0c38a0bca url: check subjectAltName when verifying ssl certificate Yuya Nishihara <yuya@tcha.org> parents: 13248 diff changeset	716 certnames = [value.lower() for key, value in san if key == 'DNS']
75d0c38a0bca url: check subjectAltName when verifying ssl certificate Yuya Nishihara <yuya@tcha.org> parents: 13248 diff changeset	717 for name in certnames:
75d0c38a0bca url: check subjectAltName when verifying ssl certificate Yuya Nishihara <yuya@tcha.org> parents: 13248 diff changeset	718 if matchdnsname(name):
75d0c38a0bca url: check subjectAltName when verifying ssl certificate Yuya Nishihara <yuya@tcha.org> parents: 13248 diff changeset	719 return None
75d0c38a0bca url: check subjectAltName when verifying ssl certificate Yuya Nishihara <yuya@tcha.org> parents: 13248 diff changeset	720 return _('certificate is for %s') % ', '.join(certnames)
75d0c38a0bca url: check subjectAltName when verifying ssl certificate Yuya Nishihara <yuya@tcha.org> parents: 13248 diff changeset	721
75d0c38a0bca url: check subjectAltName when verifying ssl certificate Yuya Nishihara <yuya@tcha.org> parents: 13248 diff changeset	722 # subject is only checked when subjectAltName is empty
12592 f2937d6492c5 url: verify correctness of https server certificates (issue2407) Mads Kiilerich <mads@kiilerich.com> parents: 12391 diff changeset	723 for s in cert.get('subject', []):
f2937d6492c5 url: verify correctness of https server certificates (issue2407) Mads Kiilerich <mads@kiilerich.com> parents: 12391 diff changeset	724 key, value = s[0]
f2937d6492c5 url: verify correctness of https server certificates (issue2407) Mads Kiilerich <mads@kiilerich.com> parents: 12391 diff changeset	725 if key == 'commonName':
13248 00411a4fa1bb url: fix UnicodeDecodeError on certificate verification error Yuya Nishihara <yuya@tcha.org> parents: 13231 diff changeset	726 try:
00411a4fa1bb url: fix UnicodeDecodeError on certificate verification error Yuya Nishihara <yuya@tcha.org> parents: 13231 diff changeset	727 # 'subject' entries are unicode
00411a4fa1bb url: fix UnicodeDecodeError on certificate verification error Yuya Nishihara <yuya@tcha.org> parents: 13231 diff changeset	728 certname = value.lower().encode('ascii')
00411a4fa1bb url: fix UnicodeDecodeError on certificate verification error Yuya Nishihara <yuya@tcha.org> parents: 13231 diff changeset	729 except UnicodeEncodeError:
00411a4fa1bb url: fix UnicodeDecodeError on certificate verification error Yuya Nishihara <yuya@tcha.org> parents: 13231 diff changeset	730 return _('IDN in certificate not supported')
13249 75d0c38a0bca url: check subjectAltName when verifying ssl certificate Yuya Nishihara <yuya@tcha.org> parents: 13248 diff changeset	731 if matchdnsname(certname):
12592 f2937d6492c5 url: verify correctness of https server certificates (issue2407) Mads Kiilerich <mads@kiilerich.com> parents: 12391 diff changeset	732 return None
f2937d6492c5 url: verify correctness of https server certificates (issue2407) Mads Kiilerich <mads@kiilerich.com> parents: 12391 diff changeset	733 return _('certificate is for %s') % certname
13249 75d0c38a0bca url: check subjectAltName when verifying ssl certificate Yuya Nishihara <yuya@tcha.org> parents: 13248 diff changeset	734 return _('no commonName or subjectAltName found in certificate')
12592 f2937d6492c5 url: verify correctness of https server certificates (issue2407) Mads Kiilerich <mads@kiilerich.com> parents: 12391 diff changeset	735
7270 2db33c1a5654 factor out the url handling from httprepo Benoit Boissinot <benoit.boissinot@ens-lyon.org> parents: diff changeset	736 if has_https:
13424 08f9c587141f url: merge BetterHTTPS with httpsconnection to get some proxy https validation Mads Kiilerich <mads@kiilerich.com> parents: 13422 diff changeset	737 class httpsconnection(httplib.HTTPSConnection):
08f9c587141f url: merge BetterHTTPS with httpsconnection to get some proxy https validation Mads Kiilerich <mads@kiilerich.com> parents: 13422 diff changeset	738 response_class = keepalive.HTTPResponse
08f9c587141f url: merge BetterHTTPS with httpsconnection to get some proxy https validation Mads Kiilerich <mads@kiilerich.com> parents: 13422 diff changeset	739 # must be able to send big bundle as stream.
08f9c587141f url: merge BetterHTTPS with httpsconnection to get some proxy https validation Mads Kiilerich <mads@kiilerich.com> parents: 13422 diff changeset	740 send = _gen_sendfile(keepalive.safesend)
08f9c587141f url: merge BetterHTTPS with httpsconnection to get some proxy https validation Mads Kiilerich <mads@kiilerich.com> parents: 13422 diff changeset	741 getresponse = keepalive.wrapgetresponse(httplib.HTTPSConnection)
9726 430e59ff3437 keepalive: handle broken pipes gracefully during large POSTs Augie Fackler <durin42@gmail.com> parents: 9467 diff changeset	742
10409 4c94a3df4b10 url: SSL server certificate verification using web.cacerts file (issue1174) Henrik Stuart <hg@hstuart.dk> parents: 10408 diff changeset	743 def connect(self):
13422 ebce5196b9db url: always create BetterHTTPS connections the same way Mads Kiilerich <mads@kiilerich.com> parents: 13421 diff changeset	744 self.sock = _create_connection((self.host, self.port))
ebce5196b9db url: always create BetterHTTPS connections the same way Mads Kiilerich <mads@kiilerich.com> parents: 13421 diff changeset	745
13421 bd8bfa85d5a5 url: refactor BetterHTTPS.connect Mads Kiilerich <mads@kiilerich.com> parents: 13420 diff changeset	746 host = self.host
13424 08f9c587141f url: merge BetterHTTPS with httpsconnection to get some proxy https validation Mads Kiilerich <mads@kiilerich.com> parents: 13422 diff changeset	747 if self.realhostport: # use CONNECT proxy
08f9c587141f url: merge BetterHTTPS with httpsconnection to get some proxy https validation Mads Kiilerich <mads@kiilerich.com> parents: 13422 diff changeset	748 something = _generic_proxytunnel(self)
08f9c587141f url: merge BetterHTTPS with httpsconnection to get some proxy https validation Mads Kiilerich <mads@kiilerich.com> parents: 13422 diff changeset	749 host = self.realhostport.rsplit(':', 1)[0]
08f9c587141f url: merge BetterHTTPS with httpsconnection to get some proxy https validation Mads Kiilerich <mads@kiilerich.com> parents: 13422 diff changeset	750
13419 1cc73868c740 url: remove test for self.ui in BetterHTTPS Mads Kiilerich <mads@kiilerich.com> parents: 13372 diff changeset	751 cacerts = self.ui.config('web', 'cacerts')
13421 bd8bfa85d5a5 url: refactor BetterHTTPS.connect Mads Kiilerich <mads@kiilerich.com> parents: 13420 diff changeset	752 hostfingerprint = self.ui.config('hostfingerprints', host)
10409 4c94a3df4b10 url: SSL server certificate verification using web.cacerts file (issue1174) Henrik Stuart <hg@hstuart.dk> parents: 10408 diff changeset	753
13314 8dc488dfcdb4 url: 'ssh known host'-like checking of fingerprints of HTTPS certificates Mads Kiilerich <mads@kiilerich.com> parents: 13249 diff changeset	754 if cacerts and not hostfingerprint:
13544 66d65bccbf06 cacert: improve error report when web.cacert file does not exist timeless <timeless@gmail.com> parents: 13424 diff changeset	755 cacerts = util.expandpath(cacerts)
66d65bccbf06 cacert: improve error report when web.cacert file does not exist timeless <timeless@gmail.com> parents: 13424 diff changeset	756 if not os.path.exists(cacerts):
66d65bccbf06 cacert: improve error report when web.cacert file does not exist timeless <timeless@gmail.com> parents: 13424 diff changeset	757 raise util.Abort(_('could not find '
66d65bccbf06 cacert: improve error report when web.cacert file does not exist timeless <timeless@gmail.com> parents: 13424 diff changeset	758 'web.cacerts: %s') % cacerts)
13421 bd8bfa85d5a5 url: refactor BetterHTTPS.connect Mads Kiilerich <mads@kiilerich.com> parents: 13420 diff changeset	759 self.sock = _ssl_wrap_socket(self.sock, self.key_file,
bd8bfa85d5a5 url: refactor BetterHTTPS.connect Mads Kiilerich <mads@kiilerich.com> parents: 13420 diff changeset	760 self.cert_file, cert_reqs=CERT_REQUIRED,
13544 66d65bccbf06 cacert: improve error report when web.cacert file does not exist timeless <timeless@gmail.com> parents: 13424 diff changeset	761 ca_certs=cacerts)
13421 bd8bfa85d5a5 url: refactor BetterHTTPS.connect Mads Kiilerich <mads@kiilerich.com> parents: 13420 diff changeset	762 msg = _verifycert(self.sock.getpeercert(), host)
12592 f2937d6492c5 url: verify correctness of https server certificates (issue2407) Mads Kiilerich <mads@kiilerich.com> parents: 12391 diff changeset	763 if msg:
13328 a939f08fae9c url: add --insecure option to bypass verification of ssl certificates Yuya Nishihara <yuya@tcha.org> parents: 13314 diff changeset	764 raise util.Abort(_('%s certificate error: %s '
a939f08fae9c url: add --insecure option to bypass verification of ssl certificates Yuya Nishihara <yuya@tcha.org> parents: 13314 diff changeset	765 '(use --insecure to connect '
13421 bd8bfa85d5a5 url: refactor BetterHTTPS.connect Mads Kiilerich <mads@kiilerich.com> parents: 13420 diff changeset	766 'insecurely)') % (host, msg))
bd8bfa85d5a5 url: refactor BetterHTTPS.connect Mads Kiilerich <mads@kiilerich.com> parents: 13420 diff changeset	767 self.ui.debug('%s certificate successfully verified\n' % host)
10409 4c94a3df4b10 url: SSL server certificate verification using web.cacerts file (issue1174) Henrik Stuart <hg@hstuart.dk> parents: 10408 diff changeset	768 else:
13422 ebce5196b9db url: always create BetterHTTPS connections the same way Mads Kiilerich <mads@kiilerich.com> parents: 13421 diff changeset	769 self.sock = _ssl_wrap_socket(self.sock, self.key_file,
ebce5196b9db url: always create BetterHTTPS connections the same way Mads Kiilerich <mads@kiilerich.com> parents: 13421 diff changeset	770 self.cert_file)
13314 8dc488dfcdb4 url: 'ssh known host'-like checking of fingerprints of HTTPS certificates Mads Kiilerich <mads@kiilerich.com> parents: 13249 diff changeset	771 if hasattr(self.sock, 'getpeercert'):
8dc488dfcdb4 url: 'ssh known host'-like checking of fingerprints of HTTPS certificates Mads Kiilerich <mads@kiilerich.com> parents: 13249 diff changeset	772 peercert = self.sock.getpeercert(True)
8dc488dfcdb4 url: 'ssh known host'-like checking of fingerprints of HTTPS certificates Mads Kiilerich <mads@kiilerich.com> parents: 13249 diff changeset	773 peerfingerprint = util.sha1(peercert).hexdigest()
8dc488dfcdb4 url: 'ssh known host'-like checking of fingerprints of HTTPS certificates Mads Kiilerich <mads@kiilerich.com> parents: 13249 diff changeset	774 nicefingerprint = ":".join([peerfingerprint[x:x + 2]
8dc488dfcdb4 url: 'ssh known host'-like checking of fingerprints of HTTPS certificates Mads Kiilerich <mads@kiilerich.com> parents: 13249 diff changeset	775 for x in xrange(0, len(peerfingerprint), 2)])
8dc488dfcdb4 url: 'ssh known host'-like checking of fingerprints of HTTPS certificates Mads Kiilerich <mads@kiilerich.com> parents: 13249 diff changeset	776 if hostfingerprint:
8dc488dfcdb4 url: 'ssh known host'-like checking of fingerprints of HTTPS certificates Mads Kiilerich <mads@kiilerich.com> parents: 13249 diff changeset	777 if peerfingerprint.lower() != \
8dc488dfcdb4 url: 'ssh known host'-like checking of fingerprints of HTTPS certificates Mads Kiilerich <mads@kiilerich.com> parents: 13249 diff changeset	778 hostfingerprint.replace(':', '').lower():
8dc488dfcdb4 url: 'ssh known host'-like checking of fingerprints of HTTPS certificates Mads Kiilerich <mads@kiilerich.com> parents: 13249 diff changeset	779 raise util.Abort(_('invalid certificate for %s '
8dc488dfcdb4 url: 'ssh known host'-like checking of fingerprints of HTTPS certificates Mads Kiilerich <mads@kiilerich.com> parents: 13249 diff changeset	780 'with fingerprint %s') %
13421 bd8bfa85d5a5 url: refactor BetterHTTPS.connect Mads Kiilerich <mads@kiilerich.com> parents: 13420 diff changeset	781 (host, nicefingerprint))
13314 8dc488dfcdb4 url: 'ssh known host'-like checking of fingerprints of HTTPS certificates Mads Kiilerich <mads@kiilerich.com> parents: 13249 diff changeset	782 self.ui.debug('%s certificate matched fingerprint %s\n' %
13421 bd8bfa85d5a5 url: refactor BetterHTTPS.connect Mads Kiilerich <mads@kiilerich.com> parents: 13420 diff changeset	783 (host, nicefingerprint))
13314 8dc488dfcdb4 url: 'ssh known host'-like checking of fingerprints of HTTPS certificates Mads Kiilerich <mads@kiilerich.com> parents: 13249 diff changeset	784 else:
8dc488dfcdb4 url: 'ssh known host'-like checking of fingerprints of HTTPS certificates Mads Kiilerich <mads@kiilerich.com> parents: 13249 diff changeset	785 self.ui.warn(_('warning: %s certificate '
8dc488dfcdb4 url: 'ssh known host'-like checking of fingerprints of HTTPS certificates Mads Kiilerich <mads@kiilerich.com> parents: 13249 diff changeset	786 'with fingerprint %s not verified '
8dc488dfcdb4 url: 'ssh known host'-like checking of fingerprints of HTTPS certificates Mads Kiilerich <mads@kiilerich.com> parents: 13249 diff changeset	787 '(check hostfingerprints or web.cacerts '
8dc488dfcdb4 url: 'ssh known host'-like checking of fingerprints of HTTPS certificates Mads Kiilerich <mads@kiilerich.com> parents: 13249 diff changeset	788 'config setting)\n') %
13421 bd8bfa85d5a5 url: refactor BetterHTTPS.connect Mads Kiilerich <mads@kiilerich.com> parents: 13420 diff changeset	789 (host, nicefingerprint))
13314 8dc488dfcdb4 url: 'ssh known host'-like checking of fingerprints of HTTPS certificates Mads Kiilerich <mads@kiilerich.com> parents: 13249 diff changeset	790 else: # python 2.5 ?
8dc488dfcdb4 url: 'ssh known host'-like checking of fingerprints of HTTPS certificates Mads Kiilerich <mads@kiilerich.com> parents: 13249 diff changeset	791 if hostfingerprint:
13421 bd8bfa85d5a5 url: refactor BetterHTTPS.connect Mads Kiilerich <mads@kiilerich.com> parents: 13420 diff changeset	792 raise util.Abort(_('no certificate for %s with '
bd8bfa85d5a5 url: refactor BetterHTTPS.connect Mads Kiilerich <mads@kiilerich.com> parents: 13420 diff changeset	793 'configured hostfingerprint') % host)
13314 8dc488dfcdb4 url: 'ssh known host'-like checking of fingerprints of HTTPS certificates Mads Kiilerich <mads@kiilerich.com> parents: 13249 diff changeset	794 self.ui.warn(_('warning: %s certificate not verified '
8dc488dfcdb4 url: 'ssh known host'-like checking of fingerprints of HTTPS certificates Mads Kiilerich <mads@kiilerich.com> parents: 13249 diff changeset	795 '(check web.cacerts config setting)\n') %
13421 bd8bfa85d5a5 url: refactor BetterHTTPS.connect Mads Kiilerich <mads@kiilerich.com> parents: 13420 diff changeset	796 host)
10409 4c94a3df4b10 url: SSL server certificate verification using web.cacerts file (issue1174) Henrik Stuart <hg@hstuart.dk> parents: 10408 diff changeset	797
7270 2db33c1a5654 factor out the url handling from httprepo Benoit Boissinot <benoit.boissinot@ens-lyon.org> parents: diff changeset	798 class httpshandler(keepalive.KeepAliveHandler, urllib2.HTTPSHandler):
8847 7951f385fcb7 url: support client certificate files over HTTPS (issue643) Henrik Stuart <hg@hstuart.dk> parents: 8590 diff changeset	799 def __init__(self, ui):
7951f385fcb7 url: support client certificate files over HTTPS (issue643) Henrik Stuart <hg@hstuart.dk> parents: 8590 diff changeset	800 keepalive.KeepAliveHandler.__init__(self)
7951f385fcb7 url: support client certificate files over HTTPS (issue643) Henrik Stuart <hg@hstuart.dk> parents: 8590 diff changeset	801 urllib2.HTTPSHandler.__init__(self)
7951f385fcb7 url: support client certificate files over HTTPS (issue643) Henrik Stuart <hg@hstuart.dk> parents: 8590 diff changeset	802 self.ui = ui
7951f385fcb7 url: support client certificate files over HTTPS (issue643) Henrik Stuart <hg@hstuart.dk> parents: 8590 diff changeset	803 self.pwmgr = passwordmgr(self.ui)
7951f385fcb7 url: support client certificate files over HTTPS (issue643) Henrik Stuart <hg@hstuart.dk> parents: 8590 diff changeset	804
9852 917cf6bb6d0c url: generalise HTTPS proxy handling to accomodate Python changes Henrik Stuart <hg@hstuart.dk> parents: 9726 diff changeset	805 def _start_transaction(self, h, req):
917cf6bb6d0c url: generalise HTTPS proxy handling to accomodate Python changes Henrik Stuart <hg@hstuart.dk> parents: 9726 diff changeset	806 _generic_start_transaction(self, h, req)
917cf6bb6d0c url: generalise HTTPS proxy handling to accomodate Python changes Henrik Stuart <hg@hstuart.dk> parents: 9726 diff changeset	807 return keepalive.KeepAliveHandler._start_transaction(self, h, req)
917cf6bb6d0c url: generalise HTTPS proxy handling to accomodate Python changes Henrik Stuart <hg@hstuart.dk> parents: 9726 diff changeset	808
7270 2db33c1a5654 factor out the url handling from httprepo Benoit Boissinot <benoit.boissinot@ens-lyon.org> parents: diff changeset	809 def https_open(self, req):
13372 5bced0d28a39 url: return the matched authentication group name from readauthforuri() Steve Borho <steve@borho.org> parents: 13371 diff changeset	810 res = readauthforuri(self.ui, req.get_full_url())
5bced0d28a39 url: return the matched authentication group name from readauthforuri() Steve Borho <steve@borho.org> parents: 13371 diff changeset	811 if res:
5bced0d28a39 url: return the matched authentication group name from readauthforuri() Steve Borho <steve@borho.org> parents: 13371 diff changeset	812 group, auth = res
5bced0d28a39 url: return the matched authentication group name from readauthforuri() Steve Borho <steve@borho.org> parents: 13371 diff changeset	813 self.auth = auth
5bced0d28a39 url: return the matched authentication group name from readauthforuri() Steve Borho <steve@borho.org> parents: 13371 diff changeset	814 self.ui.debug("using auth.%s.* for authentication\n" % group)
5bced0d28a39 url: return the matched authentication group name from readauthforuri() Steve Borho <steve@borho.org> parents: 13371 diff changeset	815 else:
5bced0d28a39 url: return the matched authentication group name from readauthforuri() Steve Borho <steve@borho.org> parents: 13371 diff changeset	816 self.auth = None
8847 7951f385fcb7 url: support client certificate files over HTTPS (issue643) Henrik Stuart <hg@hstuart.dk> parents: 8590 diff changeset	817 return self.do_open(self._makeconnection, req)
7951f385fcb7 url: support client certificate files over HTTPS (issue643) Henrik Stuart <hg@hstuart.dk> parents: 8590 diff changeset	818
10408 50fb1fe143ff url: httplib.HTTPSConnection already handles IPv6 and port parsing fine Benoit Boissinot <benoit.boissinot@ens-lyon.org> parents: 10282 diff changeset	819 def _makeconnection(self, host, port=None, args, *kwargs):
8847 7951f385fcb7 url: support client certificate files over HTTPS (issue643) Henrik Stuart <hg@hstuart.dk> parents: 8590 diff changeset	820 keyfile = None
7951f385fcb7 url: support client certificate files over HTTPS (issue643) Henrik Stuart <hg@hstuart.dk> parents: 8590 diff changeset	821 certfile = None
7951f385fcb7 url: support client certificate files over HTTPS (issue643) Henrik Stuart <hg@hstuart.dk> parents: 8590 diff changeset	822
10511 6f61c480f51c url: args argument is a tuple, not a list (found by pylint) Benoit Boissinot <benoit.boissinot@ens-lyon.org>* parents: 10484 diff changeset	823 if len(args) >= 1: # key_file
6f61c480f51c url: args argument is a tuple, not a list (found by pylint) Benoit Boissinot <benoit.boissinot@ens-lyon.org>* parents: 10484 diff changeset	824 keyfile = args[0]
6f61c480f51c url: args argument is a tuple, not a list (found by pylint) Benoit Boissinot <benoit.boissinot@ens-lyon.org>* parents: 10484 diff changeset	825 if len(args) >= 2: # cert_file
6f61c480f51c url: args argument is a tuple, not a list (found by pylint) Benoit Boissinot <benoit.boissinot@ens-lyon.org>* parents: 10484 diff changeset	826 certfile = args[1]
6f61c480f51c url: args argument is a tuple, not a list (found by pylint) Benoit Boissinot <benoit.boissinot@ens-lyon.org>* parents: 10484 diff changeset	827 args = args[2:]
8847 7951f385fcb7 url: support client certificate files over HTTPS (issue643) Henrik Stuart <hg@hstuart.dk> parents: 8590 diff changeset	828
7951f385fcb7 url: support client certificate files over HTTPS (issue643) Henrik Stuart <hg@hstuart.dk> parents: 8590 diff changeset	829 # if the user has specified different key/cert files in
7951f385fcb7 url: support client certificate files over HTTPS (issue643) Henrik Stuart <hg@hstuart.dk> parents: 8590 diff changeset	830 # hgrc, we prefer these
7951f385fcb7 url: support client certificate files over HTTPS (issue643) Henrik Stuart <hg@hstuart.dk> parents: 8590 diff changeset	831 if self.auth and 'key' in self.auth and 'cert' in self.auth:
7951f385fcb7 url: support client certificate files over HTTPS (issue643) Henrik Stuart <hg@hstuart.dk> parents: 8590 diff changeset	832 keyfile = self.auth['key']
7951f385fcb7 url: support client certificate files over HTTPS (issue643) Henrik Stuart <hg@hstuart.dk> parents: 8590 diff changeset	833 certfile = self.auth['cert']
7951f385fcb7 url: support client certificate files over HTTPS (issue643) Henrik Stuart <hg@hstuart.dk> parents: 8590 diff changeset	834
10409 4c94a3df4b10 url: SSL server certificate verification using web.cacerts file (issue1174) Henrik Stuart <hg@hstuart.dk> parents: 10408 diff changeset	835 conn = httpsconnection(host, port, keyfile, certfile, args, *kwargs)
4c94a3df4b10 url: SSL server certificate verification using web.cacerts file (issue1174) Henrik Stuart <hg@hstuart.dk> parents: 10408 diff changeset	836 conn.ui = self.ui
4c94a3df4b10 url: SSL server certificate verification using web.cacerts file (issue1174) Henrik Stuart <hg@hstuart.dk> parents: 10408 diff changeset	837 return conn
7270 2db33c1a5654 factor out the url handling from httprepo Benoit Boissinot <benoit.boissinot@ens-lyon.org> parents: diff changeset	838
2db33c1a5654 factor out the url handling from httprepo Benoit Boissinot <benoit.boissinot@ens-lyon.org> parents: diff changeset	839 class httpdigestauthhandler(urllib2.HTTPDigestAuthHandler):
11457 2ec346160783 http digest auth: reset redirect counter on new requests (issue2255) Mads Kiilerich <mads@kiilerich.com> parents: 11415 diff changeset	840 def __init__(self, args, *kwargs):
2ec346160783 http digest auth: reset redirect counter on new requests (issue2255) Mads Kiilerich <mads@kiilerich.com> parents: 11415 diff changeset	841 urllib2.HTTPDigestAuthHandler.__init__(self, args, *kwargs)
2ec346160783 http digest auth: reset redirect counter on new requests (issue2255) Mads Kiilerich <mads@kiilerich.com> parents: 11415 diff changeset	842 self.retried_req = None
2ec346160783 http digest auth: reset redirect counter on new requests (issue2255) Mads Kiilerich <mads@kiilerich.com> parents: 11415 diff changeset	843
2ec346160783 http digest auth: reset redirect counter on new requests (issue2255) Mads Kiilerich <mads@kiilerich.com> parents: 11415 diff changeset	844 def reset_retry_count(self):
2ec346160783 http digest auth: reset redirect counter on new requests (issue2255) Mads Kiilerich <mads@kiilerich.com> parents: 11415 diff changeset	845 # Python 2.6.5 will call this on 401 or 407 errors and thus loop
2ec346160783 http digest auth: reset redirect counter on new requests (issue2255) Mads Kiilerich <mads@kiilerich.com> parents: 11415 diff changeset	846 # forever. We disable reset_retry_count completely and reset in
2ec346160783 http digest auth: reset redirect counter on new requests (issue2255) Mads Kiilerich <mads@kiilerich.com> parents: 11415 diff changeset	847 # http_error_auth_reqed instead.
2ec346160783 http digest auth: reset redirect counter on new requests (issue2255) Mads Kiilerich <mads@kiilerich.com> parents: 11415 diff changeset	848 pass
2ec346160783 http digest auth: reset redirect counter on new requests (issue2255) Mads Kiilerich <mads@kiilerich.com> parents: 11415 diff changeset	849
7270 2db33c1a5654 factor out the url handling from httprepo Benoit Boissinot <benoit.boissinot@ens-lyon.org> parents: diff changeset	850 def http_error_auth_reqed(self, auth_header, host, req, headers):
11457 2ec346160783 http digest auth: reset redirect counter on new requests (issue2255) Mads Kiilerich <mads@kiilerich.com> parents: 11415 diff changeset	851 # Reset the retry counter once for each request.
2ec346160783 http digest auth: reset redirect counter on new requests (issue2255) Mads Kiilerich <mads@kiilerich.com> parents: 11415 diff changeset	852 if req is not self.retried_req:
2ec346160783 http digest auth: reset redirect counter on new requests (issue2255) Mads Kiilerich <mads@kiilerich.com> parents: 11415 diff changeset	853 self.retried_req = req
2ec346160783 http digest auth: reset redirect counter on new requests (issue2255) Mads Kiilerich <mads@kiilerich.com> parents: 11415 diff changeset	854 self.retried = 0
2ec346160783 http digest auth: reset redirect counter on new requests (issue2255) Mads Kiilerich <mads@kiilerich.com> parents: 11415 diff changeset	855 # In python < 2.5 AbstractDigestAuthHandler raises a ValueError if
2ec346160783 http digest auth: reset redirect counter on new requests (issue2255) Mads Kiilerich <mads@kiilerich.com> parents: 11415 diff changeset	856 # it doesn't know about the auth type requested. This can happen if
2ec346160783 http digest auth: reset redirect counter on new requests (issue2255) Mads Kiilerich <mads@kiilerich.com> parents: 11415 diff changeset	857 # somebody is using BasicAuth and types a bad password.
7270 2db33c1a5654 factor out the url handling from httprepo Benoit Boissinot <benoit.boissinot@ens-lyon.org> parents: diff changeset	858 try:
2db33c1a5654 factor out the url handling from httprepo Benoit Boissinot <benoit.boissinot@ens-lyon.org> parents: diff changeset	859 return urllib2.HTTPDigestAuthHandler.http_error_auth_reqed(
2db33c1a5654 factor out the url handling from httprepo Benoit Boissinot <benoit.boissinot@ens-lyon.org> parents: diff changeset	860 self, auth_header, host, req, headers)
2db33c1a5654 factor out the url handling from httprepo Benoit Boissinot <benoit.boissinot@ens-lyon.org> parents: diff changeset	861 except ValueError, inst:
2db33c1a5654 factor out the url handling from httprepo Benoit Boissinot <benoit.boissinot@ens-lyon.org> parents: diff changeset	862 arg = inst.args[0]
2db33c1a5654 factor out the url handling from httprepo Benoit Boissinot <benoit.boissinot@ens-lyon.org> parents: diff changeset	863 if arg.startswith("AbstractDigestAuthHandler doesn't know "):
2db33c1a5654 factor out the url handling from httprepo Benoit Boissinot <benoit.boissinot@ens-lyon.org> parents: diff changeset	864 return
2db33c1a5654 factor out the url handling from httprepo Benoit Boissinot <benoit.boissinot@ens-lyon.org> parents: diff changeset	865 raise
2db33c1a5654 factor out the url handling from httprepo Benoit Boissinot <benoit.boissinot@ens-lyon.org> parents: diff changeset	866
11844 6c51a5056020 http basic auth: reset redirect counter on new requests (issue2255) Wagner Bruna <wbruna@softwareexpress.com.br> parents: 11457 diff changeset	867 class httpbasicauthhandler(urllib2.HTTPBasicAuthHandler):
6c51a5056020 http basic auth: reset redirect counter on new requests (issue2255) Wagner Bruna <wbruna@softwareexpress.com.br> parents: 11457 diff changeset	868 def __init__(self, args, *kwargs):
6c51a5056020 http basic auth: reset redirect counter on new requests (issue2255) Wagner Bruna <wbruna@softwareexpress.com.br> parents: 11457 diff changeset	869 urllib2.HTTPBasicAuthHandler.__init__(self, args, *kwargs)
6c51a5056020 http basic auth: reset redirect counter on new requests (issue2255) Wagner Bruna <wbruna@softwareexpress.com.br> parents: 11457 diff changeset	870 self.retried_req = None
6c51a5056020 http basic auth: reset redirect counter on new requests (issue2255) Wagner Bruna <wbruna@softwareexpress.com.br> parents: 11457 diff changeset	871
6c51a5056020 http basic auth: reset redirect counter on new requests (issue2255) Wagner Bruna <wbruna@softwareexpress.com.br> parents: 11457 diff changeset	872 def reset_retry_count(self):
6c51a5056020 http basic auth: reset redirect counter on new requests (issue2255) Wagner Bruna <wbruna@softwareexpress.com.br> parents: 11457 diff changeset	873 # Python 2.6.5 will call this on 401 or 407 errors and thus loop
6c51a5056020 http basic auth: reset redirect counter on new requests (issue2255) Wagner Bruna <wbruna@softwareexpress.com.br> parents: 11457 diff changeset	874 # forever. We disable reset_retry_count completely and reset in
6c51a5056020 http basic auth: reset redirect counter on new requests (issue2255) Wagner Bruna <wbruna@softwareexpress.com.br> parents: 11457 diff changeset	875 # http_error_auth_reqed instead.
6c51a5056020 http basic auth: reset redirect counter on new requests (issue2255) Wagner Bruna <wbruna@softwareexpress.com.br> parents: 11457 diff changeset	876 pass
6c51a5056020 http basic auth: reset redirect counter on new requests (issue2255) Wagner Bruna <wbruna@softwareexpress.com.br> parents: 11457 diff changeset	877
6c51a5056020 http basic auth: reset redirect counter on new requests (issue2255) Wagner Bruna <wbruna@softwareexpress.com.br> parents: 11457 diff changeset	878 def http_error_auth_reqed(self, auth_header, host, req, headers):
6c51a5056020 http basic auth: reset redirect counter on new requests (issue2255) Wagner Bruna <wbruna@softwareexpress.com.br> parents: 11457 diff changeset	879 # Reset the retry counter once for each request.
6c51a5056020 http basic auth: reset redirect counter on new requests (issue2255) Wagner Bruna <wbruna@softwareexpress.com.br> parents: 11457 diff changeset	880 if req is not self.retried_req:
6c51a5056020 http basic auth: reset redirect counter on new requests (issue2255) Wagner Bruna <wbruna@softwareexpress.com.br> parents: 11457 diff changeset	881 self.retried_req = req
6c51a5056020 http basic auth: reset redirect counter on new requests (issue2255) Wagner Bruna <wbruna@softwareexpress.com.br> parents: 11457 diff changeset	882 self.retried = 0
6c51a5056020 http basic auth: reset redirect counter on new requests (issue2255) Wagner Bruna <wbruna@softwareexpress.com.br> parents: 11457 diff changeset	883 return urllib2.HTTPBasicAuthHandler.http_error_auth_reqed(
6c51a5056020 http basic auth: reset redirect counter on new requests (issue2255) Wagner Bruna <wbruna@softwareexpress.com.br> parents: 11457 diff changeset	884 self, auth_header, host, req, headers)
6c51a5056020 http basic auth: reset redirect counter on new requests (issue2255) Wagner Bruna <wbruna@softwareexpress.com.br> parents: 11457 diff changeset	885
7270 2db33c1a5654 factor out the url handling from httprepo Benoit Boissinot <benoit.boissinot@ens-lyon.org> parents: diff changeset	886 def getauthinfo(path):
2db33c1a5654 factor out the url handling from httprepo Benoit Boissinot <benoit.boissinot@ens-lyon.org> parents: diff changeset	887 scheme, netloc, urlpath, query, frag = urlparse.urlsplit(path)
2db33c1a5654 factor out the url handling from httprepo Benoit Boissinot <benoit.boissinot@ens-lyon.org> parents: diff changeset	888 if not urlpath:
2db33c1a5654 factor out the url handling from httprepo Benoit Boissinot <benoit.boissinot@ens-lyon.org> parents: diff changeset	889 urlpath = '/'
7284 ac81ffac0f35 url: fix file:// URL handling Patrick Mezard <pmezard@gmail.com> parents: 7270 diff changeset	890 if scheme != 'file':
ac81ffac0f35 url: fix file:// URL handling Patrick Mezard <pmezard@gmail.com> parents: 7270 diff changeset	891 # XXX: why are we quoting the path again with some smart
ac81ffac0f35 url: fix file:// URL handling Patrick Mezard <pmezard@gmail.com> parents: 7270 diff changeset	892 # heuristic here? Anyway, it cannot be done with file://
ac81ffac0f35 url: fix file:// URL handling Patrick Mezard <pmezard@gmail.com> parents: 7270 diff changeset	893 # urls since path encoding is os/fs dependent (see
ac81ffac0f35 url: fix file:// URL handling Patrick Mezard <pmezard@gmail.com> parents: 7270 diff changeset	894 # urllib.pathname2url() for details).
ac81ffac0f35 url: fix file:// URL handling Patrick Mezard <pmezard@gmail.com> parents: 7270 diff changeset	895 urlpath = quotepath(urlpath)
7270 2db33c1a5654 factor out the url handling from httprepo Benoit Boissinot <benoit.boissinot@ens-lyon.org> parents: diff changeset	896 host, port, user, passwd = netlocsplit(netloc)
2db33c1a5654 factor out the url handling from httprepo Benoit Boissinot <benoit.boissinot@ens-lyon.org> parents: diff changeset	897
2db33c1a5654 factor out the url handling from httprepo Benoit Boissinot <benoit.boissinot@ens-lyon.org> parents: diff changeset	898 # urllib cannot handle URLs with embedded user or passwd
2db33c1a5654 factor out the url handling from httprepo Benoit Boissinot <benoit.boissinot@ens-lyon.org> parents: diff changeset	899 url = urlparse.urlunsplit((scheme, netlocunsplit(host, port),
2db33c1a5654 factor out the url handling from httprepo Benoit Boissinot <benoit.boissinot@ens-lyon.org> parents: diff changeset	900 urlpath, query, frag))
2db33c1a5654 factor out the url handling from httprepo Benoit Boissinot <benoit.boissinot@ens-lyon.org> parents: diff changeset	901 if user:
2db33c1a5654 factor out the url handling from httprepo Benoit Boissinot <benoit.boissinot@ens-lyon.org> parents: diff changeset	902 netloc = host
2db33c1a5654 factor out the url handling from httprepo Benoit Boissinot <benoit.boissinot@ens-lyon.org> parents: diff changeset	903 if port:
2db33c1a5654 factor out the url handling from httprepo Benoit Boissinot <benoit.boissinot@ens-lyon.org> parents: diff changeset	904 netloc += ':' + port
2db33c1a5654 factor out the url handling from httprepo Benoit Boissinot <benoit.boissinot@ens-lyon.org> parents: diff changeset	905 # Python < 2.4.3 uses only the netloc to search for a password
2db33c1a5654 factor out the url handling from httprepo Benoit Boissinot <benoit.boissinot@ens-lyon.org> parents: diff changeset	906 authinfo = (None, (url, netloc), user, passwd or '')
2db33c1a5654 factor out the url handling from httprepo Benoit Boissinot <benoit.boissinot@ens-lyon.org> parents: diff changeset	907 else:
2db33c1a5654 factor out the url handling from httprepo Benoit Boissinot <benoit.boissinot@ens-lyon.org> parents: diff changeset	908 authinfo = None
2db33c1a5654 factor out the url handling from httprepo Benoit Boissinot <benoit.boissinot@ens-lyon.org> parents: diff changeset	909 return url, authinfo
2db33c1a5654 factor out the url handling from httprepo Benoit Boissinot <benoit.boissinot@ens-lyon.org> parents: diff changeset	910
9347 d0474b184347 url: add support for custom handlers in extensions Henrik Stuart <hg@hstuart.dk> parents: 9122 diff changeset	911 handlerfuncs = []
d0474b184347 url: add support for custom handlers in extensions Henrik Stuart <hg@hstuart.dk> parents: 9122 diff changeset	912
7270 2db33c1a5654 factor out the url handling from httprepo Benoit Boissinot <benoit.boissinot@ens-lyon.org> parents: diff changeset	913 def opener(ui, authinfo=None):
2db33c1a5654 factor out the url handling from httprepo Benoit Boissinot <benoit.boissinot@ens-lyon.org> parents: diff changeset	914 '''
2db33c1a5654 factor out the url handling from httprepo Benoit Boissinot <benoit.boissinot@ens-lyon.org> parents: diff changeset	915 construct an opener suitable for urllib2
2db33c1a5654 factor out the url handling from httprepo Benoit Boissinot <benoit.boissinot@ens-lyon.org> parents: diff changeset	916 authinfo will be added to the password manager
2db33c1a5654 factor out the url handling from httprepo Benoit Boissinot <benoit.boissinot@ens-lyon.org> parents: diff changeset	917 '''
2db33c1a5654 factor out the url handling from httprepo Benoit Boissinot <benoit.boissinot@ens-lyon.org> parents: diff changeset	918 handlers = [httphandler()]
2db33c1a5654 factor out the url handling from httprepo Benoit Boissinot <benoit.boissinot@ens-lyon.org> parents: diff changeset	919 if has_https:
8847 7951f385fcb7 url: support client certificate files over HTTPS (issue643) Henrik Stuart <hg@hstuart.dk> parents: 8590 diff changeset	920 handlers.append(httpshandler(ui))
7270 2db33c1a5654 factor out the url handling from httprepo Benoit Boissinot <benoit.boissinot@ens-lyon.org> parents: diff changeset	921
2db33c1a5654 factor out the url handling from httprepo Benoit Boissinot <benoit.boissinot@ens-lyon.org> parents: diff changeset	922 handlers.append(proxyhandler(ui))
2db33c1a5654 factor out the url handling from httprepo Benoit Boissinot <benoit.boissinot@ens-lyon.org> parents: diff changeset	923
2db33c1a5654 factor out the url handling from httprepo Benoit Boissinot <benoit.boissinot@ens-lyon.org> parents: diff changeset	924 passmgr = passwordmgr(ui)
2db33c1a5654 factor out the url handling from httprepo Benoit Boissinot <benoit.boissinot@ens-lyon.org> parents: diff changeset	925 if authinfo is not None:
2db33c1a5654 factor out the url handling from httprepo Benoit Boissinot <benoit.boissinot@ens-lyon.org> parents: diff changeset	926 passmgr.add_password(*authinfo)
2db33c1a5654 factor out the url handling from httprepo Benoit Boissinot <benoit.boissinot@ens-lyon.org> parents: diff changeset	927 user, passwd = authinfo[2:4]
9467 4c041f1ee1b4 do not attempt to translate ui.debug output Martin Geisler <mg@lazybytes.net> parents: 9347 diff changeset	928 ui.debug('http auth: user %s, password %s\n' %
7270 2db33c1a5654 factor out the url handling from httprepo Benoit Boissinot <benoit.boissinot@ens-lyon.org> parents: diff changeset	929 (user, passwd and '' len(passwd) or 'not set'))
2db33c1a5654 factor out the url handling from httprepo Benoit Boissinot <benoit.boissinot@ens-lyon.org> parents: diff changeset	930
11844 6c51a5056020 http basic auth: reset redirect counter on new requests (issue2255) Wagner Bruna <wbruna@softwareexpress.com.br> parents: 11457 diff changeset	931 handlers.extend((httpbasicauthhandler(passmgr),
7270 2db33c1a5654 factor out the url handling from httprepo Benoit Boissinot <benoit.boissinot@ens-lyon.org> parents: diff changeset	932 httpdigestauthhandler(passmgr)))
9347 d0474b184347 url: add support for custom handlers in extensions Henrik Stuart <hg@hstuart.dk> parents: 9122 diff changeset	933 handlers.extend([h(ui, passmgr) for h in handlerfuncs])
7270 2db33c1a5654 factor out the url handling from httprepo Benoit Boissinot <benoit.boissinot@ens-lyon.org> parents: diff changeset	934 opener = urllib2.build_opener(*handlers)
2db33c1a5654 factor out the url handling from httprepo Benoit Boissinot <benoit.boissinot@ens-lyon.org> parents: diff changeset	935
2db33c1a5654 factor out the url handling from httprepo Benoit Boissinot <benoit.boissinot@ens-lyon.org> parents: diff changeset	936 # 1.0 here is the _protocol_ version
2db33c1a5654 factor out the url handling from httprepo Benoit Boissinot <benoit.boissinot@ens-lyon.org> parents: diff changeset	937 opener.addheaders = [('User-agent', 'mercurial/proto-1.0')]
2db33c1a5654 factor out the url handling from httprepo Benoit Boissinot <benoit.boissinot@ens-lyon.org> parents: diff changeset	938 opener.addheaders.append(('Accept', 'application/mercurial-0.1'))
2db33c1a5654 factor out the url handling from httprepo Benoit Boissinot <benoit.boissinot@ens-lyon.org> parents: diff changeset	939 return opener
2db33c1a5654 factor out the url handling from httprepo Benoit Boissinot <benoit.boissinot@ens-lyon.org> parents: diff changeset	940
7285 5ad99abfab79 url: detect scheme with a regexp instead of urlsplit() Patrick Mezard <pmezard@gmail.com> parents: 7284 diff changeset	941 scheme_re = re.compile(r'^([a-zA-Z0-9+-.]+)://')
5ad99abfab79 url: detect scheme with a regexp instead of urlsplit() Patrick Mezard <pmezard@gmail.com> parents: 7284 diff changeset	942
7270 2db33c1a5654 factor out the url handling from httprepo Benoit Boissinot <benoit.boissinot@ens-lyon.org> parents: diff changeset	943 def open(ui, url, data=None):
7285 5ad99abfab79 url: detect scheme with a regexp instead of urlsplit() Patrick Mezard <pmezard@gmail.com> parents: 7284 diff changeset	944 scheme = None
5ad99abfab79 url: detect scheme with a regexp instead of urlsplit() Patrick Mezard <pmezard@gmail.com> parents: 7284 diff changeset	945 m = scheme_re.search(url)
5ad99abfab79 url: detect scheme with a regexp instead of urlsplit() Patrick Mezard <pmezard@gmail.com> parents: 7284 diff changeset	946 if m:
5ad99abfab79 url: detect scheme with a regexp instead of urlsplit() Patrick Mezard <pmezard@gmail.com> parents: 7284 diff changeset	947 scheme = m.group(1).lower()
7270 2db33c1a5654 factor out the url handling from httprepo Benoit Boissinot <benoit.boissinot@ens-lyon.org> parents: diff changeset	948 if not scheme:
7284 ac81ffac0f35 url: fix file:// URL handling Patrick Mezard <pmezard@gmail.com> parents: 7270 diff changeset	949 path = util.normpath(os.path.abspath(url))
ac81ffac0f35 url: fix file:// URL handling Patrick Mezard <pmezard@gmail.com> parents: 7270 diff changeset	950 url = 'file://' + urllib.pathname2url(path)
ac81ffac0f35 url: fix file:// URL handling Patrick Mezard <pmezard@gmail.com> parents: 7270 diff changeset	951 authinfo = None
7270 2db33c1a5654 factor out the url handling from httprepo Benoit Boissinot <benoit.boissinot@ens-lyon.org> parents: diff changeset	952 else:
2db33c1a5654 factor out the url handling from httprepo Benoit Boissinot <benoit.boissinot@ens-lyon.org> parents: diff changeset	953 url, authinfo = getauthinfo(url)
2db33c1a5654 factor out the url handling from httprepo Benoit Boissinot <benoit.boissinot@ens-lyon.org> parents: diff changeset	954 return opener(ui, authinfo).open(url, data)

Mercurial > public > mercurial-scm > hg

annotate mercurial/url.py @ 13770:4e8f2310f310