mercurial-scm/hg-stable: mercurial/url.py annotate

annotate mercurial/url.py @ 41721:4028897dfa05

url: always use str for proxy configuration Previously, proxies didn't work on Python 3 for various reasons. First, the keys to the "proxies" dict are fed into a `setattr(self, "%s_open", ...)` call and passing bytestrings results in setting an oddly named attribute due to the b'' in %s formatting. This resulted in "http_open" and "https_open" not being properly overridden and proxies not being used. Second, the standard library was expecting proxy URLs to be str. And various operations (including our custom code in url.py) would fail to account for the str/bytes mismatch. This commit normalizes everything to str and adjusts our proxy code in url.py to account for the presence of str on Python 3. Differential Revision: https://phab.mercurial-scm.org/D5952

author	Gregory Szorc <gregory.szorc@gmail.com>
date	Fri, 15 Feb 2019 13:16:07 -0800
parents	d20f1594ff4a
children	544035846830

rev	line source
7270 2db33c1a5654 factor out the url handling from httprepo Benoit Boissinot <benoit.boissinot@ens-lyon.org> parents: diff changeset	1 # url.py - HTTP handling for mercurial
2db33c1a5654 factor out the url handling from httprepo Benoit Boissinot <benoit.boissinot@ens-lyon.org> parents: diff changeset	2 #
2db33c1a5654 factor out the url handling from httprepo Benoit Boissinot <benoit.boissinot@ens-lyon.org> parents: diff changeset	3 # Copyright 2005, 2006, 2007, 2008 Matt Mackall <mpm@selenic.com>
2db33c1a5654 factor out the url handling from httprepo Benoit Boissinot <benoit.boissinot@ens-lyon.org> parents: diff changeset	4 # Copyright 2006, 2007 Alexis S. L. Carvalho <alexis@cecm.usp.br>
2db33c1a5654 factor out the url handling from httprepo Benoit Boissinot <benoit.boissinot@ens-lyon.org> parents: diff changeset	5 # Copyright 2006 Vadim Gelfer <vadim.gelfer@gmail.com>
2db33c1a5654 factor out the url handling from httprepo Benoit Boissinot <benoit.boissinot@ens-lyon.org> parents: diff changeset	6 #
8225 46293a0c7e9f updated license to be explicit about GPL version 2 Martin Geisler <mg@lazybytes.net> parents: 8208 diff changeset	7 # This software may be used and distributed according to the terms of the
10263 25e572394f5c Update license to GPLv2+ Matt Mackall <mpm@selenic.com> parents: 9852 diff changeset	8 # GNU General Public License version 2 or any later version.
7270 2db33c1a5654 factor out the url handling from httprepo Benoit Boissinot <benoit.boissinot@ens-lyon.org> parents: diff changeset	9
25990 c3efcdba08ea url: use absolute_import Gregory Szorc <gregory.szorc@gmail.com> parents: 25837 diff changeset	10 from __future__ import absolute_import
c3efcdba08ea url: use absolute_import Gregory Szorc <gregory.szorc@gmail.com> parents: 25837 diff changeset	11
c3efcdba08ea url: use absolute_import Gregory Szorc <gregory.szorc@gmail.com> parents: 25837 diff changeset	12 import base64
c3efcdba08ea url: use absolute_import Gregory Szorc <gregory.szorc@gmail.com> parents: 25837 diff changeset	13 import os
c3efcdba08ea url: use absolute_import Gregory Szorc <gregory.szorc@gmail.com> parents: 25837 diff changeset	14 import socket
37103 9e6d3465f17e url: make logginghttphandler compatible with Python 2.7.6 Yuya Nishihara <yuya@tcha.org> parents: 37087 diff changeset	15 import sys
25990 c3efcdba08ea url: use absolute_import Gregory Szorc <gregory.szorc@gmail.com> parents: 25837 diff changeset	16
c3efcdba08ea url: use absolute_import Gregory Szorc <gregory.szorc@gmail.com> parents: 25837 diff changeset	17 from .i18n import _
c3efcdba08ea url: use absolute_import Gregory Szorc <gregory.szorc@gmail.com> parents: 25837 diff changeset	18 from . import (
30820 6a70cf94d1b5 py3: replace pycompat.getenv with encoding.environ.get Pulkit Goyal <7895pulkit@gmail.com> parents: 30763 diff changeset	19 encoding,
26587 56b2bcea2529 error: get Abort from 'error' instead of 'util' Pierre-Yves David <pierre-yves.david@fb.com> parents: 25990 diff changeset	20 error,
25990 c3efcdba08ea url: use absolute_import Gregory Szorc <gregory.szorc@gmail.com> parents: 25837 diff changeset	21 httpconnection as httpconnectionmod,
c3efcdba08ea url: use absolute_import Gregory Szorc <gregory.szorc@gmail.com> parents: 25837 diff changeset	22 keepalive,
34429 0ee9cf8d054a url: use native strings for header values Augie Fackler <augie@google.com> parents: 33499 diff changeset	23 pycompat,
25990 c3efcdba08ea url: use absolute_import Gregory Szorc <gregory.szorc@gmail.com> parents: 25837 diff changeset	24 sslutil,
34467 1232f7fa00c3 cleanup: use urllibcompat for renamed methods on urllib request objects Augie Fackler <augie@google.com> parents: 34429 diff changeset	25 urllibcompat,
25990 c3efcdba08ea url: use absolute_import Gregory Szorc <gregory.szorc@gmail.com> parents: 25837 diff changeset	26 util,
c3efcdba08ea url: use absolute_import Gregory Szorc <gregory.szorc@gmail.com> parents: 25837 diff changeset	27 )
37087 f0b6fbea00cf stringutil: bulk-replace call sites to point to new module Yuya Nishihara <yuya@tcha.org> parents: 37048 diff changeset	28 from .utils import (
f0b6fbea00cf stringutil: bulk-replace call sites to point to new module Yuya Nishihara <yuya@tcha.org> parents: 37048 diff changeset	29 stringutil,
f0b6fbea00cf stringutil: bulk-replace call sites to point to new module Yuya Nishihara <yuya@tcha.org> parents: 37048 diff changeset	30 )
29455 0c741fd6158a py3: conditionalize httplib import Pulkit Goyal <7895pulkit@gmail.com> parents: 29379 diff changeset	31
0c741fd6158a py3: conditionalize httplib import Pulkit Goyal <7895pulkit@gmail.com> parents: 29379 diff changeset	32 httplib = util.httplib
28861 86db5cb55d46 pycompat: switch to util.stringio for py3 compat timeless <timeless@mozdev.org> parents: 26806 diff changeset	33 stringio = util.stringio
28883 032c4c2f802a pycompat: switch to util.urlreq/util.urlerr for py3 compat timeless <timeless@mozdev.org> parents: 28861 diff changeset	34 urlerr = util.urlerr
032c4c2f802a pycompat: switch to util.urlreq/util.urlerr for py3 compat timeless <timeless@mozdev.org> parents: 28861 diff changeset	35 urlreq = util.urlreq
032c4c2f802a pycompat: switch to util.urlreq/util.urlerr for py3 compat timeless <timeless@mozdev.org> parents: 28861 diff changeset	36
34694 2976cf87a60a url: add cgi.escape equivalent for bytestrings Augie Fackler <augie@google.com> parents: 34467 diff changeset	37 def escape(s, quote=None):
2976cf87a60a url: add cgi.escape equivalent for bytestrings Augie Fackler <augie@google.com> parents: 34467 diff changeset	38 '''Replace special characters "&", "<" and ">" to HTML-safe sequences.
2976cf87a60a url: add cgi.escape equivalent for bytestrings Augie Fackler <augie@google.com> parents: 34467 diff changeset	39 If the optional flag quote is true, the quotation mark character (")
2976cf87a60a url: add cgi.escape equivalent for bytestrings Augie Fackler <augie@google.com> parents: 34467 diff changeset	40 is also translated.
2976cf87a60a url: add cgi.escape equivalent for bytestrings Augie Fackler <augie@google.com> parents: 34467 diff changeset	41
2976cf87a60a url: add cgi.escape equivalent for bytestrings Augie Fackler <augie@google.com> parents: 34467 diff changeset	42 This is the same as cgi.escape in Python, but always operates on
2976cf87a60a url: add cgi.escape equivalent for bytestrings Augie Fackler <augie@google.com> parents: 34467 diff changeset	43 bytes, whereas cgi.escape in Python 3 only works on unicodes.
2976cf87a60a url: add cgi.escape equivalent for bytestrings Augie Fackler <augie@google.com> parents: 34467 diff changeset	44 '''
2976cf87a60a url: add cgi.escape equivalent for bytestrings Augie Fackler <augie@google.com> parents: 34467 diff changeset	45 s = s.replace(b"&", b"&")
2976cf87a60a url: add cgi.escape equivalent for bytestrings Augie Fackler <augie@google.com> parents: 34467 diff changeset	46 s = s.replace(b"<", b"<")
2976cf87a60a url: add cgi.escape equivalent for bytestrings Augie Fackler <augie@google.com> parents: 34467 diff changeset	47 s = s.replace(b">", b">")
2976cf87a60a url: add cgi.escape equivalent for bytestrings Augie Fackler <augie@google.com> parents: 34467 diff changeset	48 if quote:
2976cf87a60a url: add cgi.escape equivalent for bytestrings Augie Fackler <augie@google.com> parents: 34467 diff changeset	49 s = s.replace(b'"', b""")
2976cf87a60a url: add cgi.escape equivalent for bytestrings Augie Fackler <augie@google.com> parents: 34467 diff changeset	50 return s
2976cf87a60a url: add cgi.escape equivalent for bytestrings Augie Fackler <augie@google.com> parents: 34467 diff changeset	51
29377 2c019aac6b99 url: extract password database from password manager liscju <piotr.listkiewicz@gmail.com> parents: 29252 diff changeset	52 class passwordmgr(object):
2c019aac6b99 url: extract password database from password manager liscju <piotr.listkiewicz@gmail.com> parents: 29252 diff changeset	53 def __init__(self, ui, passwddb):
7270 2db33c1a5654 factor out the url handling from httprepo Benoit Boissinot <benoit.boissinot@ens-lyon.org> parents: diff changeset	54 self.ui = ui
29377 2c019aac6b99 url: extract password database from password manager liscju <piotr.listkiewicz@gmail.com> parents: 29252 diff changeset	55 self.passwddb = passwddb
2c019aac6b99 url: extract password database from password manager liscju <piotr.listkiewicz@gmail.com> parents: 29252 diff changeset	56
2c019aac6b99 url: extract password database from password manager liscju <piotr.listkiewicz@gmail.com> parents: 29252 diff changeset	57 def add_password(self, realm, uri, user, passwd):
2c019aac6b99 url: extract password database from password manager liscju <piotr.listkiewicz@gmail.com> parents: 29252 diff changeset	58 return self.passwddb.add_password(realm, uri, user, passwd)
7270 2db33c1a5654 factor out the url handling from httprepo Benoit Boissinot <benoit.boissinot@ens-lyon.org> parents: diff changeset	59
2db33c1a5654 factor out the url handling from httprepo Benoit Boissinot <benoit.boissinot@ens-lyon.org> parents: diff changeset	60 def find_user_password(self, realm, authuri):
41469 bc776c31c093 url: add some defensive asserts on expected incoming types Augie Fackler <augie@google.com> parents: 40044 diff changeset	61 assert isinstance(realm, (type(None), str))
bc776c31c093 url: add some defensive asserts on expected incoming types Augie Fackler <augie@google.com> parents: 40044 diff changeset	62 assert isinstance(authuri, str)
29377 2c019aac6b99 url: extract password database from password manager liscju <piotr.listkiewicz@gmail.com> parents: 29252 diff changeset	63 authinfo = self.passwddb.find_user_password(realm, authuri)
7270 2db33c1a5654 factor out the url handling from httprepo Benoit Boissinot <benoit.boissinot@ens-lyon.org> parents: diff changeset	64 user, passwd = authinfo
41470 d437d1e2a711 url: convert some variables back to bytes Augie Fackler <augie@google.com> parents: 41469 diff changeset	65 user, passwd = pycompat.bytesurl(user), pycompat.bytesurl(passwd)
7270 2db33c1a5654 factor out the url handling from httprepo Benoit Boissinot <benoit.boissinot@ens-lyon.org> parents: diff changeset	66 if user and passwd:
8333 89c80c3dc584 allow http authentication information to be specified in the configuration Sune Foldager <cryo@cyanite.org> parents: 8225 diff changeset	67 self._writedebug(user, passwd)
41603 349c8879becd py3: ensure the HTTP password manager returns strings, not bytes Matt Harbison <matt_harbison@yahoo.com> parents: 41470 diff changeset	68 return (pycompat.strurl(user), pycompat.strurl(passwd))
7270 2db33c1a5654 factor out the url handling from httprepo Benoit Boissinot <benoit.boissinot@ens-lyon.org> parents: diff changeset	69
15005 4a43e23b8c55 hgweb: do not ignore [auth] if url has a username (issue2822) Patrick Mezard <pmezard@gmail.com> parents: 14244 diff changeset	70 if not user or not passwd:
15025 0593e8f81c71 http: pass user to readauthforuri() (fix 4a43e23b8c55) Patrick Mezard <pmezard@gmail.com> parents: 15005 diff changeset	71 res = httpconnectionmod.readauthforuri(self.ui, authuri, user)
13372 5bced0d28a39 url: return the matched authentication group name from readauthforuri() Steve Borho <steve@borho.org> parents: 13371 diff changeset	72 if res:
5bced0d28a39 url: return the matched authentication group name from readauthforuri() Steve Borho <steve@borho.org> parents: 13371 diff changeset	73 group, auth = res
8847 7951f385fcb7 url: support client certificate files over HTTPS (issue643) Henrik Stuart <hg@hstuart.dk> parents: 8590 diff changeset	74 user, passwd = auth.get('username'), auth.get('password')
13372 5bced0d28a39 url: return the matched authentication group name from readauthforuri() Steve Borho <steve@borho.org> parents: 13371 diff changeset	75 self.ui.debug("using auth.%s.* for authentication\n" % group)
8333 89c80c3dc584 allow http authentication information to be specified in the configuration Sune Foldager <cryo@cyanite.org> parents: 8225 diff changeset	76 if not user or not passwd:
36681 8381126bf43c url: more bytes/unicodes fussing in url.py around auth handling Augie Fackler <augie@google.com> parents: 36484 diff changeset	77 u = util.url(pycompat.bytesurl(authuri))
20291 7d589d923b8a url: added authuri when login information is requested (issue3209) Lucas Moscovicz <lmoscovicz@fb.com> parents: 20174 diff changeset	78 u.query = None
8333 89c80c3dc584 allow http authentication information to be specified in the configuration Sune Foldager <cryo@cyanite.org> parents: 8225 diff changeset	79 if not self.ui.interactive():
26587 56b2bcea2529 error: get Abort from 'error' instead of 'util' Pierre-Yves David <pierre-yves.david@fb.com> parents: 25990 diff changeset	80 raise error.Abort(_('http authorization required for %s') %
36341 548316560a05 py3: use bytes() to byte-stringify url object in url.py Yuya Nishihara <yuya@tcha.org> parents: 36340 diff changeset	81 util.hidepassword(bytes(u)))
7270 2db33c1a5654 factor out the url handling from httprepo Benoit Boissinot <benoit.boissinot@ens-lyon.org> parents: diff changeset	82
20291 7d589d923b8a url: added authuri when login information is requested (issue3209) Lucas Moscovicz <lmoscovicz@fb.com> parents: 20174 diff changeset	83 self.ui.write(_("http authorization required for %s\n") %
36341 548316560a05 py3: use bytes() to byte-stringify url object in url.py Yuya Nishihara <yuya@tcha.org> parents: 36340 diff changeset	84 util.hidepassword(bytes(u)))
36681 8381126bf43c url: more bytes/unicodes fussing in url.py around auth handling Augie Fackler <augie@google.com> parents: 36484 diff changeset	85 self.ui.write(_("realm: %s\n") % pycompat.bytesurl(realm))
8333 89c80c3dc584 allow http authentication information to be specified in the configuration Sune Foldager <cryo@cyanite.org> parents: 8225 diff changeset	86 if user:
12862 9d6adddc8eea url: show realm/user when asking for username/password timeless <timeless@gmail.com> parents: 12770 diff changeset	87 self.ui.write(_("user: %s\n") % user)
8333 89c80c3dc584 allow http authentication information to be specified in the configuration Sune Foldager <cryo@cyanite.org> parents: 8225 diff changeset	88 else:
89c80c3dc584 allow http authentication information to be specified in the configuration Sune Foldager <cryo@cyanite.org> parents: 8225 diff changeset	89 user = self.ui.prompt(_("user:"), default=None)
7270 2db33c1a5654 factor out the url handling from httprepo Benoit Boissinot <benoit.boissinot@ens-lyon.org> parents: diff changeset	90
8333 89c80c3dc584 allow http authentication information to be specified in the configuration Sune Foldager <cryo@cyanite.org> parents: 8225 diff changeset	91 if not passwd:
89c80c3dc584 allow http authentication information to be specified in the configuration Sune Foldager <cryo@cyanite.org> parents: 8225 diff changeset	92 passwd = self.ui.getpass()
7270 2db33c1a5654 factor out the url handling from httprepo Benoit Boissinot <benoit.boissinot@ens-lyon.org> parents: diff changeset	93
29377 2c019aac6b99 url: extract password database from password manager liscju <piotr.listkiewicz@gmail.com> parents: 29252 diff changeset	94 self.passwddb.add_password(realm, authuri, user, passwd)
8333 89c80c3dc584 allow http authentication information to be specified in the configuration Sune Foldager <cryo@cyanite.org> parents: 8225 diff changeset	95 self._writedebug(user, passwd)
41603 349c8879becd py3: ensure the HTTP password manager returns strings, not bytes Matt Harbison <matt_harbison@yahoo.com> parents: 41470 diff changeset	96 return (pycompat.strurl(user), pycompat.strurl(passwd))
7270 2db33c1a5654 factor out the url handling from httprepo Benoit Boissinot <benoit.boissinot@ens-lyon.org> parents: diff changeset	97
8333 89c80c3dc584 allow http authentication information to be specified in the configuration Sune Foldager <cryo@cyanite.org> parents: 8225 diff changeset	98 def _writedebug(self, user, passwd):
89c80c3dc584 allow http authentication information to be specified in the configuration Sune Foldager <cryo@cyanite.org> parents: 8225 diff changeset	99 msg = _('http auth: user %s, password %s\n')
89c80c3dc584 allow http authentication information to be specified in the configuration Sune Foldager <cryo@cyanite.org> parents: 8225 diff changeset	100 self.ui.debug(msg % (user, passwd and '' len(passwd) or 'not set'))
89c80c3dc584 allow http authentication information to be specified in the configuration Sune Foldager <cryo@cyanite.org> parents: 8225 diff changeset	101
15025 0593e8f81c71 http: pass user to readauthforuri() (fix 4a43e23b8c55) Patrick Mezard <pmezard@gmail.com> parents: 15005 diff changeset	102 def find_stored_password(self, authuri):
29377 2c019aac6b99 url: extract password database from password manager liscju <piotr.listkiewicz@gmail.com> parents: 29252 diff changeset	103 return self.passwddb.find_user_password(None, authuri)
15025 0593e8f81c71 http: pass user to readauthforuri() (fix 4a43e23b8c55) Patrick Mezard <pmezard@gmail.com> parents: 15005 diff changeset	104
28883 032c4c2f802a pycompat: switch to util.urlreq/util.urlerr for py3 compat timeless <timeless@mozdev.org> parents: 28861 diff changeset	105 class proxyhandler(urlreq.proxyhandler):
7270 2db33c1a5654 factor out the url handling from httprepo Benoit Boissinot <benoit.boissinot@ens-lyon.org> parents: diff changeset	106 def __init__(self, ui):
30667 69acfd2ca11e py3: replace os.getenv with pycompat.osgetenv Pulkit Goyal <7895pulkit@gmail.com> parents: 30665 diff changeset	107 proxyurl = (ui.config("http_proxy", "host") or
30820 6a70cf94d1b5 py3: replace pycompat.getenv with encoding.environ.get Pulkit Goyal <7895pulkit@gmail.com> parents: 30763 diff changeset	108 encoding.environ.get('http_proxy'))
7270 2db33c1a5654 factor out the url handling from httprepo Benoit Boissinot <benoit.boissinot@ens-lyon.org> parents: diff changeset	109 # XXX proxyauthinfo = None
2db33c1a5654 factor out the url handling from httprepo Benoit Boissinot <benoit.boissinot@ens-lyon.org> parents: diff changeset	110
2db33c1a5654 factor out the url handling from httprepo Benoit Boissinot <benoit.boissinot@ens-lyon.org> parents: diff changeset	111 if proxyurl:
2db33c1a5654 factor out the url handling from httprepo Benoit Boissinot <benoit.boissinot@ens-lyon.org> parents: diff changeset	112 # proxy can be proper url or host[:port]
2db33c1a5654 factor out the url handling from httprepo Benoit Boissinot <benoit.boissinot@ens-lyon.org> parents: diff changeset	113 if not (proxyurl.startswith('http:') or
2db33c1a5654 factor out the url handling from httprepo Benoit Boissinot <benoit.boissinot@ens-lyon.org> parents: diff changeset	114 proxyurl.startswith('https:')):
2db33c1a5654 factor out the url handling from httprepo Benoit Boissinot <benoit.boissinot@ens-lyon.org> parents: diff changeset	115 proxyurl = 'http://' + proxyurl + '/'
14076 924c82157d46 url: move URL parsing functions into util to improve startup time Brodie Rao <brodie@bitheap.org> parents: 14071 diff changeset	116 proxy = util.url(proxyurl)
13820 65b89e80f892 url: use url.url in proxyhandler Brodie Rao <brodie@bitheap.org> parents: 13819 diff changeset	117 if not proxy.user:
65b89e80f892 url: use url.url in proxyhandler Brodie Rao <brodie@bitheap.org> parents: 13819 diff changeset	118 proxy.user = ui.config("http_proxy", "user")
65b89e80f892 url: use url.url in proxyhandler Brodie Rao <brodie@bitheap.org> parents: 13819 diff changeset	119 proxy.passwd = ui.config("http_proxy", "passwd")
7270 2db33c1a5654 factor out the url handling from httprepo Benoit Boissinot <benoit.boissinot@ens-lyon.org> parents: diff changeset	120
2db33c1a5654 factor out the url handling from httprepo Benoit Boissinot <benoit.boissinot@ens-lyon.org> parents: diff changeset	121 # see if we should use a proxy for this url
10282 08a0f04b56bd many, many trivial check-code fixups Matt Mackall <mpm@selenic.com> parents: 10263 diff changeset	122 no_list = ["localhost", "127.0.0.1"]
7270 2db33c1a5654 factor out the url handling from httprepo Benoit Boissinot <benoit.boissinot@ens-lyon.org> parents: diff changeset	123 no_list.extend([p.lower() for
2db33c1a5654 factor out the url handling from httprepo Benoit Boissinot <benoit.boissinot@ens-lyon.org> parents: diff changeset	124 p in ui.configlist("http_proxy", "no")])
2db33c1a5654 factor out the url handling from httprepo Benoit Boissinot <benoit.boissinot@ens-lyon.org> parents: diff changeset	125 no_list.extend([p.strip().lower() for
30820 6a70cf94d1b5 py3: replace pycompat.getenv with encoding.environ.get Pulkit Goyal <7895pulkit@gmail.com> parents: 30763 diff changeset	126 p in encoding.environ.get("no_proxy", '').split(',')
7270 2db33c1a5654 factor out the url handling from httprepo Benoit Boissinot <benoit.boissinot@ens-lyon.org> parents: diff changeset	127 if p.strip()])
2db33c1a5654 factor out the url handling from httprepo Benoit Boissinot <benoit.boissinot@ens-lyon.org> parents: diff changeset	128 # "http_proxy.always" config is for running tests on localhost
2db33c1a5654 factor out the url handling from httprepo Benoit Boissinot <benoit.boissinot@ens-lyon.org> parents: diff changeset	129 if ui.configbool("http_proxy", "always"):
2db33c1a5654 factor out the url handling from httprepo Benoit Boissinot <benoit.boissinot@ens-lyon.org> parents: diff changeset	130 self.no_list = []
2db33c1a5654 factor out the url handling from httprepo Benoit Boissinot <benoit.boissinot@ens-lyon.org> parents: diff changeset	131 else:
2db33c1a5654 factor out the url handling from httprepo Benoit Boissinot <benoit.boissinot@ens-lyon.org> parents: diff changeset	132 self.no_list = no_list
2db33c1a5654 factor out the url handling from httprepo Benoit Boissinot <benoit.boissinot@ens-lyon.org> parents: diff changeset	133
41721 4028897dfa05 url: always use str for proxy configuration Gregory Szorc <gregory.szorc@gmail.com> parents: 41719 diff changeset	134 # Keys and values need to be str because the standard library
4028897dfa05 url: always use str for proxy configuration Gregory Szorc <gregory.szorc@gmail.com> parents: 41719 diff changeset	135 # expects them to be.
4028897dfa05 url: always use str for proxy configuration Gregory Szorc <gregory.szorc@gmail.com> parents: 41719 diff changeset	136 proxyurl = str(proxy)
4028897dfa05 url: always use str for proxy configuration Gregory Szorc <gregory.szorc@gmail.com> parents: 41719 diff changeset	137 proxies = {r'http': proxyurl, r'https': proxyurl}
4028897dfa05 url: always use str for proxy configuration Gregory Szorc <gregory.szorc@gmail.com> parents: 41719 diff changeset	138 ui.debug('proxying through %s\n' % util.hidepassword(bytes(proxy)))
7270 2db33c1a5654 factor out the url handling from httprepo Benoit Boissinot <benoit.boissinot@ens-lyon.org> parents: diff changeset	139 else:
2db33c1a5654 factor out the url handling from httprepo Benoit Boissinot <benoit.boissinot@ens-lyon.org> parents: diff changeset	140 proxies = {}
2db33c1a5654 factor out the url handling from httprepo Benoit Boissinot <benoit.boissinot@ens-lyon.org> parents: diff changeset	141
28883 032c4c2f802a pycompat: switch to util.urlreq/util.urlerr for py3 compat timeless <timeless@mozdev.org> parents: 28861 diff changeset	142 urlreq.proxyhandler.__init__(self, proxies)
7270 2db33c1a5654 factor out the url handling from httprepo Benoit Boissinot <benoit.boissinot@ens-lyon.org> parents: diff changeset	143 self.ui = ui
2db33c1a5654 factor out the url handling from httprepo Benoit Boissinot <benoit.boissinot@ens-lyon.org> parents: diff changeset	144
2db33c1a5654 factor out the url handling from httprepo Benoit Boissinot <benoit.boissinot@ens-lyon.org> parents: diff changeset	145 def proxy_open(self, req, proxy, type_):
41721 4028897dfa05 url: always use str for proxy configuration Gregory Szorc <gregory.szorc@gmail.com> parents: 41719 diff changeset	146 host = pycompat.bytesurl(urllibcompat.gethost(req)).split(':')[0]
19535 df2155ebf502 proxy: allow wildcards in the no proxy list (issue1821) Matt Mackall <mpm@selenic.com> parents: 18176 diff changeset	147 for e in self.no_list:
df2155ebf502 proxy: allow wildcards in the no proxy list (issue1821) Matt Mackall <mpm@selenic.com> parents: 18176 diff changeset	148 if host == e:
df2155ebf502 proxy: allow wildcards in the no proxy list (issue1821) Matt Mackall <mpm@selenic.com> parents: 18176 diff changeset	149 return None
df2155ebf502 proxy: allow wildcards in the no proxy list (issue1821) Matt Mackall <mpm@selenic.com> parents: 18176 diff changeset	150 if e.startswith('*.') and host.endswith(e[2:]):
df2155ebf502 proxy: allow wildcards in the no proxy list (issue1821) Matt Mackall <mpm@selenic.com> parents: 18176 diff changeset	151 return None
df2155ebf502 proxy: allow wildcards in the no proxy list (issue1821) Matt Mackall <mpm@selenic.com> parents: 18176 diff changeset	152 if e.startswith('.') and host.endswith(e[1:]):
df2155ebf502 proxy: allow wildcards in the no proxy list (issue1821) Matt Mackall <mpm@selenic.com> parents: 18176 diff changeset	153 return None
7270 2db33c1a5654 factor out the url handling from httprepo Benoit Boissinot <benoit.boissinot@ens-lyon.org> parents: diff changeset	154
28883 032c4c2f802a pycompat: switch to util.urlreq/util.urlerr for py3 compat timeless <timeless@mozdev.org> parents: 28861 diff changeset	155 return urlreq.proxyhandler.proxy_open(self, req, proxy, type_)
7270 2db33c1a5654 factor out the url handling from httprepo Benoit Boissinot <benoit.boissinot@ens-lyon.org> parents: diff changeset	156
13420 051f498628f7 url: refactor _gen_sendfile Mads Kiilerich <mads@kiilerich.com> parents: 13419 diff changeset	157 def _gen_sendfile(orgsend):
7270 2db33c1a5654 factor out the url handling from httprepo Benoit Boissinot <benoit.boissinot@ens-lyon.org> parents: diff changeset	158 def _sendfile(self, data):
2db33c1a5654 factor out the url handling from httprepo Benoit Boissinot <benoit.boissinot@ens-lyon.org> parents: diff changeset	159 # send a file
14244 e7525a555a64 url: use new http support if requested by the user Augie Fackler <durin42@gmail.com> parents: 14204 diff changeset	160 if isinstance(data, httpconnectionmod.httpsendfile):
7270 2db33c1a5654 factor out the url handling from httprepo Benoit Boissinot <benoit.boissinot@ens-lyon.org> parents: diff changeset	161 # if auth required, some data sent twice, so rewind here
2db33c1a5654 factor out the url handling from httprepo Benoit Boissinot <benoit.boissinot@ens-lyon.org> parents: diff changeset	162 data.seek(0)
2db33c1a5654 factor out the url handling from httprepo Benoit Boissinot <benoit.boissinot@ens-lyon.org> parents: diff changeset	163 for chunk in util.filechunkiter(data):
13420 051f498628f7 url: refactor _gen_sendfile Mads Kiilerich <mads@kiilerich.com> parents: 13419 diff changeset	164 orgsend(self, chunk)
7270 2db33c1a5654 factor out the url handling from httprepo Benoit Boissinot <benoit.boissinot@ens-lyon.org> parents: diff changeset	165 else:
13420 051f498628f7 url: refactor _gen_sendfile Mads Kiilerich <mads@kiilerich.com> parents: 13419 diff changeset	166 orgsend(self, data)
7270 2db33c1a5654 factor out the url handling from httprepo Benoit Boissinot <benoit.boissinot@ens-lyon.org> parents: diff changeset	167 return _sendfile
2db33c1a5654 factor out the url handling from httprepo Benoit Boissinot <benoit.boissinot@ens-lyon.org> parents: diff changeset	168
28883 032c4c2f802a pycompat: switch to util.urlreq/util.urlerr for py3 compat timeless <timeless@mozdev.org> parents: 28861 diff changeset	169 has_https = util.safehasattr(urlreq, 'httpshandler')
10409 4c94a3df4b10 url: SSL server certificate verification using web.cacerts file (issue1174) Henrik Stuart <hg@hstuart.dk> parents: 10408 diff changeset	170
7270 2db33c1a5654 factor out the url handling from httprepo Benoit Boissinot <benoit.boissinot@ens-lyon.org> parents: diff changeset	171 class httpconnection(keepalive.HTTPConnection):
2db33c1a5654 factor out the url handling from httprepo Benoit Boissinot <benoit.boissinot@ens-lyon.org> parents: diff changeset	172 # must be able to send big bundle as stream.
13420 051f498628f7 url: refactor _gen_sendfile Mads Kiilerich <mads@kiilerich.com> parents: 13419 diff changeset	173 send = _gen_sendfile(keepalive.HTTPConnection.send)
7270 2db33c1a5654 factor out the url handling from httprepo Benoit Boissinot <benoit.boissinot@ens-lyon.org> parents: diff changeset	174
8590 59acb9c7d90f url: use CONNECT for HTTPS connections through HTTP proxy (issue967) Henrik Stuart <hg@hstuart.dk> parents: 8344 diff changeset	175 def getresponse(self):
59acb9c7d90f url: use CONNECT for HTTPS connections through HTTP proxy (issue967) Henrik Stuart <hg@hstuart.dk> parents: 8344 diff changeset	176 proxyres = getattr(self, 'proxyres', None)
59acb9c7d90f url: use CONNECT for HTTPS connections through HTTP proxy (issue967) Henrik Stuart <hg@hstuart.dk> parents: 8344 diff changeset	177 if proxyres:
59acb9c7d90f url: use CONNECT for HTTPS connections through HTTP proxy (issue967) Henrik Stuart <hg@hstuart.dk> parents: 8344 diff changeset	178 if proxyres.will_close:
59acb9c7d90f url: use CONNECT for HTTPS connections through HTTP proxy (issue967) Henrik Stuart <hg@hstuart.dk> parents: 8344 diff changeset	179 self.close()
59acb9c7d90f url: use CONNECT for HTTPS connections through HTTP proxy (issue967) Henrik Stuart <hg@hstuart.dk> parents: 8344 diff changeset	180 self.proxyres = None
59acb9c7d90f url: use CONNECT for HTTPS connections through HTTP proxy (issue967) Henrik Stuart <hg@hstuart.dk> parents: 8344 diff changeset	181 return proxyres
59acb9c7d90f url: use CONNECT for HTTPS connections through HTTP proxy (issue967) Henrik Stuart <hg@hstuart.dk> parents: 8344 diff changeset	182 return keepalive.HTTPConnection.getresponse(self)
59acb9c7d90f url: use CONNECT for HTTPS connections through HTTP proxy (issue967) Henrik Stuart <hg@hstuart.dk> parents: 8344 diff changeset	183
41719 d20f1594ff4a url: always access req._tunnel_host Gregory Szorc <gregory.szorc@gmail.com> parents: 41603 diff changeset	184 # Large parts of this function have their origin from before Python 2.6
d20f1594ff4a url: always access req._tunnel_host Gregory Szorc <gregory.szorc@gmail.com> parents: 41603 diff changeset	185 # and could potentially be removed.
9852 917cf6bb6d0c url: generalise HTTPS proxy handling to accomodate Python changes Henrik Stuart <hg@hstuart.dk> parents: 9726 diff changeset	186 def _generic_start_transaction(handler, h, req):
41719 d20f1594ff4a url: always access req._tunnel_host Gregory Szorc <gregory.szorc@gmail.com> parents: 41603 diff changeset	187 tunnel_host = req._tunnel_host
14964 376c32a5ccdc url: replace uses of hasattr with safehasattr or getattr Augie Fackler <durin42@gmail.com> parents: 14244 diff changeset	188 if tunnel_host:
41721 4028897dfa05 url: always use str for proxy configuration Gregory Szorc <gregory.szorc@gmail.com> parents: 41719 diff changeset	189 if tunnel_host[:7] not in [r'http://', r'https:/']:
4028897dfa05 url: always use str for proxy configuration Gregory Szorc <gregory.szorc@gmail.com> parents: 41719 diff changeset	190 tunnel_host = r'https://' + tunnel_host
9852 917cf6bb6d0c url: generalise HTTPS proxy handling to accomodate Python changes Henrik Stuart <hg@hstuart.dk> parents: 9726 diff changeset	191 new_tunnel = True
917cf6bb6d0c url: generalise HTTPS proxy handling to accomodate Python changes Henrik Stuart <hg@hstuart.dk> parents: 9726 diff changeset	192 else:
34467 1232f7fa00c3 cleanup: use urllibcompat for renamed methods on urllib request objects Augie Fackler <augie@google.com> parents: 34429 diff changeset	193 tunnel_host = urllibcompat.getselector(req)
9852 917cf6bb6d0c url: generalise HTTPS proxy handling to accomodate Python changes Henrik Stuart <hg@hstuart.dk> parents: 9726 diff changeset	194 new_tunnel = False
917cf6bb6d0c url: generalise HTTPS proxy handling to accomodate Python changes Henrik Stuart <hg@hstuart.dk> parents: 9726 diff changeset	195
34467 1232f7fa00c3 cleanup: use urllibcompat for renamed methods on urllib request objects Augie Fackler <augie@google.com> parents: 34429 diff changeset	196 if new_tunnel or tunnel_host == urllibcompat.getfullurl(req): # has proxy
41721 4028897dfa05 url: always use str for proxy configuration Gregory Szorc <gregory.szorc@gmail.com> parents: 41719 diff changeset	197 u = util.url(pycompat.bytesurl(tunnel_host))
13820 65b89e80f892 url: use url.url in proxyhandler Brodie Rao <brodie@bitheap.org> parents: 13819 diff changeset	198 if new_tunnel or u.scheme == 'https': # only use CONNECT for HTTPS
65b89e80f892 url: use url.url in proxyhandler Brodie Rao <brodie@bitheap.org> parents: 13819 diff changeset	199 h.realhostport = ':'.join([u.host, (u.port or '443')])
9852 917cf6bb6d0c url: generalise HTTPS proxy handling to accomodate Python changes Henrik Stuart <hg@hstuart.dk> parents: 9726 diff changeset	200 h.headers = req.headers.copy()
917cf6bb6d0c url: generalise HTTPS proxy handling to accomodate Python changes Henrik Stuart <hg@hstuart.dk> parents: 9726 diff changeset	201 h.headers.update(handler.parent.addheaders)
917cf6bb6d0c url: generalise HTTPS proxy handling to accomodate Python changes Henrik Stuart <hg@hstuart.dk> parents: 9726 diff changeset	202 return
917cf6bb6d0c url: generalise HTTPS proxy handling to accomodate Python changes Henrik Stuart <hg@hstuart.dk> parents: 9726 diff changeset	203
10415 677f15da38c1 url: proxy handling, simplify and correctly deal with IPv6 Benoit Boissinot <benoit.boissinot@ens-lyon.org> parents: 10411 diff changeset	204 h.realhostport = None
9852 917cf6bb6d0c url: generalise HTTPS proxy handling to accomodate Python changes Henrik Stuart <hg@hstuart.dk> parents: 9726 diff changeset	205 h.headers = None
917cf6bb6d0c url: generalise HTTPS proxy handling to accomodate Python changes Henrik Stuart <hg@hstuart.dk> parents: 9726 diff changeset	206
917cf6bb6d0c url: generalise HTTPS proxy handling to accomodate Python changes Henrik Stuart <hg@hstuart.dk> parents: 9726 diff changeset	207 def _generic_proxytunnel(self):
917cf6bb6d0c url: generalise HTTPS proxy handling to accomodate Python changes Henrik Stuart <hg@hstuart.dk> parents: 9726 diff changeset	208 proxyheaders = dict(
917cf6bb6d0c url: generalise HTTPS proxy handling to accomodate Python changes Henrik Stuart <hg@hstuart.dk> parents: 9726 diff changeset	209 [(x, self.headers[x]) for x in self.headers
41721 4028897dfa05 url: always use str for proxy configuration Gregory Szorc <gregory.szorc@gmail.com> parents: 41719 diff changeset	210 if x.lower().startswith(r'proxy-')])
10415 677f15da38c1 url: proxy handling, simplify and correctly deal with IPv6 Benoit Boissinot <benoit.boissinot@ens-lyon.org> parents: 10411 diff changeset	211 self.send('CONNECT %s HTTP/1.0\r\n' % self.realhostport)
9852 917cf6bb6d0c url: generalise HTTPS proxy handling to accomodate Python changes Henrik Stuart <hg@hstuart.dk> parents: 9726 diff changeset	212 for header in proxyheaders.iteritems():
917cf6bb6d0c url: generalise HTTPS proxy handling to accomodate Python changes Henrik Stuart <hg@hstuart.dk> parents: 9726 diff changeset	213 self.send('%s: %s\r\n' % header)
917cf6bb6d0c url: generalise HTTPS proxy handling to accomodate Python changes Henrik Stuart <hg@hstuart.dk> parents: 9726 diff changeset	214 self.send('\r\n')
917cf6bb6d0c url: generalise HTTPS proxy handling to accomodate Python changes Henrik Stuart <hg@hstuart.dk> parents: 9726 diff changeset	215
917cf6bb6d0c url: generalise HTTPS proxy handling to accomodate Python changes Henrik Stuart <hg@hstuart.dk> parents: 9726 diff changeset	216 # majority of the following code is duplicated from
917cf6bb6d0c url: generalise HTTPS proxy handling to accomodate Python changes Henrik Stuart <hg@hstuart.dk> parents: 9726 diff changeset	217 # httplib.HTTPConnection as there are no adequate places to
917cf6bb6d0c url: generalise HTTPS proxy handling to accomodate Python changes Henrik Stuart <hg@hstuart.dk> parents: 9726 diff changeset	218 # override functions to provide the needed functionality
917cf6bb6d0c url: generalise HTTPS proxy handling to accomodate Python changes Henrik Stuart <hg@hstuart.dk> parents: 9726 diff changeset	219 res = self.response_class(self.sock,
917cf6bb6d0c url: generalise HTTPS proxy handling to accomodate Python changes Henrik Stuart <hg@hstuart.dk> parents: 9726 diff changeset	220 strict=self.strict,
917cf6bb6d0c url: generalise HTTPS proxy handling to accomodate Python changes Henrik Stuart <hg@hstuart.dk> parents: 9726 diff changeset	221 method=self._method)
917cf6bb6d0c url: generalise HTTPS proxy handling to accomodate Python changes Henrik Stuart <hg@hstuart.dk> parents: 9726 diff changeset	222
917cf6bb6d0c url: generalise HTTPS proxy handling to accomodate Python changes Henrik Stuart <hg@hstuart.dk> parents: 9726 diff changeset	223 while True:
917cf6bb6d0c url: generalise HTTPS proxy handling to accomodate Python changes Henrik Stuart <hg@hstuart.dk> parents: 9726 diff changeset	224 version, status, reason = res._read_status()
917cf6bb6d0c url: generalise HTTPS proxy handling to accomodate Python changes Henrik Stuart <hg@hstuart.dk> parents: 9726 diff changeset	225 if status != httplib.CONTINUE:
917cf6bb6d0c url: generalise HTTPS proxy handling to accomodate Python changes Henrik Stuart <hg@hstuart.dk> parents: 9726 diff changeset	226 break
29741 44ea12756fef url: use `iter(callable, sentinel)` instead of while True Augie Fackler <augie@google.com> parents: 29682 diff changeset	227 # skip lines that are all whitespace
44ea12756fef url: use `iter(callable, sentinel)` instead of while True Augie Fackler <augie@google.com> parents: 29682 diff changeset	228 list(iter(lambda: res.fp.readline().strip(), ''))
9852 917cf6bb6d0c url: generalise HTTPS proxy handling to accomodate Python changes Henrik Stuart <hg@hstuart.dk> parents: 9726 diff changeset	229 res.status = status
917cf6bb6d0c url: generalise HTTPS proxy handling to accomodate Python changes Henrik Stuart <hg@hstuart.dk> parents: 9726 diff changeset	230 res.reason = reason.strip()
917cf6bb6d0c url: generalise HTTPS proxy handling to accomodate Python changes Henrik Stuart <hg@hstuart.dk> parents: 9726 diff changeset	231
917cf6bb6d0c url: generalise HTTPS proxy handling to accomodate Python changes Henrik Stuart <hg@hstuart.dk> parents: 9726 diff changeset	232 if res.status == 200:
29741 44ea12756fef url: use `iter(callable, sentinel)` instead of while True Augie Fackler <augie@google.com> parents: 29682 diff changeset	233 # skip lines until we find a blank line
44ea12756fef url: use `iter(callable, sentinel)` instead of while True Augie Fackler <augie@google.com> parents: 29682 diff changeset	234 list(iter(res.fp.readline, '\r\n'))
9852 917cf6bb6d0c url: generalise HTTPS proxy handling to accomodate Python changes Henrik Stuart <hg@hstuart.dk> parents: 9726 diff changeset	235 return True
917cf6bb6d0c url: generalise HTTPS proxy handling to accomodate Python changes Henrik Stuart <hg@hstuart.dk> parents: 9726 diff changeset	236
917cf6bb6d0c url: generalise HTTPS proxy handling to accomodate Python changes Henrik Stuart <hg@hstuart.dk> parents: 9726 diff changeset	237 if version == 'HTTP/1.0':
917cf6bb6d0c url: generalise HTTPS proxy handling to accomodate Python changes Henrik Stuart <hg@hstuart.dk> parents: 9726 diff changeset	238 res.version = 10
917cf6bb6d0c url: generalise HTTPS proxy handling to accomodate Python changes Henrik Stuart <hg@hstuart.dk> parents: 9726 diff changeset	239 elif version.startswith('HTTP/1.'):
917cf6bb6d0c url: generalise HTTPS proxy handling to accomodate Python changes Henrik Stuart <hg@hstuart.dk> parents: 9726 diff changeset	240 res.version = 11
917cf6bb6d0c url: generalise HTTPS proxy handling to accomodate Python changes Henrik Stuart <hg@hstuart.dk> parents: 9726 diff changeset	241 elif version == 'HTTP/0.9':
917cf6bb6d0c url: generalise HTTPS proxy handling to accomodate Python changes Henrik Stuart <hg@hstuart.dk> parents: 9726 diff changeset	242 res.version = 9
917cf6bb6d0c url: generalise HTTPS proxy handling to accomodate Python changes Henrik Stuart <hg@hstuart.dk> parents: 9726 diff changeset	243 else:
917cf6bb6d0c url: generalise HTTPS proxy handling to accomodate Python changes Henrik Stuart <hg@hstuart.dk> parents: 9726 diff changeset	244 raise httplib.UnknownProtocol(version)
917cf6bb6d0c url: generalise HTTPS proxy handling to accomodate Python changes Henrik Stuart <hg@hstuart.dk> parents: 9726 diff changeset	245
917cf6bb6d0c url: generalise HTTPS proxy handling to accomodate Python changes Henrik Stuart <hg@hstuart.dk> parents: 9726 diff changeset	246 if res.version == 9:
917cf6bb6d0c url: generalise HTTPS proxy handling to accomodate Python changes Henrik Stuart <hg@hstuart.dk> parents: 9726 diff changeset	247 res.length = None
917cf6bb6d0c url: generalise HTTPS proxy handling to accomodate Python changes Henrik Stuart <hg@hstuart.dk> parents: 9726 diff changeset	248 res.chunked = 0
917cf6bb6d0c url: generalise HTTPS proxy handling to accomodate Python changes Henrik Stuart <hg@hstuart.dk> parents: 9726 diff changeset	249 res.will_close = 1
28861 86db5cb55d46 pycompat: switch to util.stringio for py3 compat timeless <timeless@mozdev.org> parents: 26806 diff changeset	250 res.msg = httplib.HTTPMessage(stringio())
9852 917cf6bb6d0c url: generalise HTTPS proxy handling to accomodate Python changes Henrik Stuart <hg@hstuart.dk> parents: 9726 diff changeset	251 return False
917cf6bb6d0c url: generalise HTTPS proxy handling to accomodate Python changes Henrik Stuart <hg@hstuart.dk> parents: 9726 diff changeset	252
917cf6bb6d0c url: generalise HTTPS proxy handling to accomodate Python changes Henrik Stuart <hg@hstuart.dk> parents: 9726 diff changeset	253 res.msg = httplib.HTTPMessage(res.fp)
917cf6bb6d0c url: generalise HTTPS proxy handling to accomodate Python changes Henrik Stuart <hg@hstuart.dk> parents: 9726 diff changeset	254 res.msg.fp = None
917cf6bb6d0c url: generalise HTTPS proxy handling to accomodate Python changes Henrik Stuart <hg@hstuart.dk> parents: 9726 diff changeset	255
917cf6bb6d0c url: generalise HTTPS proxy handling to accomodate Python changes Henrik Stuart <hg@hstuart.dk> parents: 9726 diff changeset	256 # are we using the chunked-style of transfer encoding?
917cf6bb6d0c url: generalise HTTPS proxy handling to accomodate Python changes Henrik Stuart <hg@hstuart.dk> parents: 9726 diff changeset	257 trenc = res.msg.getheader('transfer-encoding')
917cf6bb6d0c url: generalise HTTPS proxy handling to accomodate Python changes Henrik Stuart <hg@hstuart.dk> parents: 9726 diff changeset	258 if trenc and trenc.lower() == "chunked":
917cf6bb6d0c url: generalise HTTPS proxy handling to accomodate Python changes Henrik Stuart <hg@hstuart.dk> parents: 9726 diff changeset	259 res.chunked = 1
917cf6bb6d0c url: generalise HTTPS proxy handling to accomodate Python changes Henrik Stuart <hg@hstuart.dk> parents: 9726 diff changeset	260 res.chunk_left = None
917cf6bb6d0c url: generalise HTTPS proxy handling to accomodate Python changes Henrik Stuart <hg@hstuart.dk> parents: 9726 diff changeset	261 else:
917cf6bb6d0c url: generalise HTTPS proxy handling to accomodate Python changes Henrik Stuart <hg@hstuart.dk> parents: 9726 diff changeset	262 res.chunked = 0
917cf6bb6d0c url: generalise HTTPS proxy handling to accomodate Python changes Henrik Stuart <hg@hstuart.dk> parents: 9726 diff changeset	263
917cf6bb6d0c url: generalise HTTPS proxy handling to accomodate Python changes Henrik Stuart <hg@hstuart.dk> parents: 9726 diff changeset	264 # will the connection close at the end of the response?
917cf6bb6d0c url: generalise HTTPS proxy handling to accomodate Python changes Henrik Stuart <hg@hstuart.dk> parents: 9726 diff changeset	265 res.will_close = res._check_close()
917cf6bb6d0c url: generalise HTTPS proxy handling to accomodate Python changes Henrik Stuart <hg@hstuart.dk> parents: 9726 diff changeset	266
917cf6bb6d0c url: generalise HTTPS proxy handling to accomodate Python changes Henrik Stuart <hg@hstuart.dk> parents: 9726 diff changeset	267 # do we have a Content-Length?
17428 72803c8edaa4 avoid using abbreviations that look like spelling errors Mads Kiilerich <mads@kiilerich.com> parents: 17424 diff changeset	268 # NOTE: RFC 2616, section 4.4, #3 says we ignore this if
72803c8edaa4 avoid using abbreviations that look like spelling errors Mads Kiilerich <mads@kiilerich.com> parents: 17424 diff changeset	269 # transfer-encoding is "chunked"
9852 917cf6bb6d0c url: generalise HTTPS proxy handling to accomodate Python changes Henrik Stuart <hg@hstuart.dk> parents: 9726 diff changeset	270 length = res.msg.getheader('content-length')
917cf6bb6d0c url: generalise HTTPS proxy handling to accomodate Python changes Henrik Stuart <hg@hstuart.dk> parents: 9726 diff changeset	271 if length and not res.chunked:
917cf6bb6d0c url: generalise HTTPS proxy handling to accomodate Python changes Henrik Stuart <hg@hstuart.dk> parents: 9726 diff changeset	272 try:
917cf6bb6d0c url: generalise HTTPS proxy handling to accomodate Python changes Henrik Stuart <hg@hstuart.dk> parents: 9726 diff changeset	273 res.length = int(length)
917cf6bb6d0c url: generalise HTTPS proxy handling to accomodate Python changes Henrik Stuart <hg@hstuart.dk> parents: 9726 diff changeset	274 except ValueError:
917cf6bb6d0c url: generalise HTTPS proxy handling to accomodate Python changes Henrik Stuart <hg@hstuart.dk> parents: 9726 diff changeset	275 res.length = None
917cf6bb6d0c url: generalise HTTPS proxy handling to accomodate Python changes Henrik Stuart <hg@hstuart.dk> parents: 9726 diff changeset	276 else:
917cf6bb6d0c url: generalise HTTPS proxy handling to accomodate Python changes Henrik Stuart <hg@hstuart.dk> parents: 9726 diff changeset	277 if res.length < 0: # ignore nonsensical negative lengths
917cf6bb6d0c url: generalise HTTPS proxy handling to accomodate Python changes Henrik Stuart <hg@hstuart.dk> parents: 9726 diff changeset	278 res.length = None
917cf6bb6d0c url: generalise HTTPS proxy handling to accomodate Python changes Henrik Stuart <hg@hstuart.dk> parents: 9726 diff changeset	279 else:
917cf6bb6d0c url: generalise HTTPS proxy handling to accomodate Python changes Henrik Stuart <hg@hstuart.dk> parents: 9726 diff changeset	280 res.length = None
917cf6bb6d0c url: generalise HTTPS proxy handling to accomodate Python changes Henrik Stuart <hg@hstuart.dk> parents: 9726 diff changeset	281
917cf6bb6d0c url: generalise HTTPS proxy handling to accomodate Python changes Henrik Stuart <hg@hstuart.dk> parents: 9726 diff changeset	282 # does the body have a fixed length? (of zero)
917cf6bb6d0c url: generalise HTTPS proxy handling to accomodate Python changes Henrik Stuart <hg@hstuart.dk> parents: 9726 diff changeset	283 if (status == httplib.NO_CONTENT or status == httplib.NOT_MODIFIED or
917cf6bb6d0c url: generalise HTTPS proxy handling to accomodate Python changes Henrik Stuart <hg@hstuart.dk> parents: 9726 diff changeset	284 100 <= status < 200 or # 1xx codes
917cf6bb6d0c url: generalise HTTPS proxy handling to accomodate Python changes Henrik Stuart <hg@hstuart.dk> parents: 9726 diff changeset	285 res._method == 'HEAD'):
917cf6bb6d0c url: generalise HTTPS proxy handling to accomodate Python changes Henrik Stuart <hg@hstuart.dk> parents: 9726 diff changeset	286 res.length = 0
917cf6bb6d0c url: generalise HTTPS proxy handling to accomodate Python changes Henrik Stuart <hg@hstuart.dk> parents: 9726 diff changeset	287
917cf6bb6d0c url: generalise HTTPS proxy handling to accomodate Python changes Henrik Stuart <hg@hstuart.dk> parents: 9726 diff changeset	288 # if the connection remains open, and we aren't using chunked, and
917cf6bb6d0c url: generalise HTTPS proxy handling to accomodate Python changes Henrik Stuart <hg@hstuart.dk> parents: 9726 diff changeset	289 # a content-length was not provided, then assume that the connection
917cf6bb6d0c url: generalise HTTPS proxy handling to accomodate Python changes Henrik Stuart <hg@hstuart.dk> parents: 9726 diff changeset	290 # WILL close.
917cf6bb6d0c url: generalise HTTPS proxy handling to accomodate Python changes Henrik Stuart <hg@hstuart.dk> parents: 9726 diff changeset	291 if (not res.will_close and
917cf6bb6d0c url: generalise HTTPS proxy handling to accomodate Python changes Henrik Stuart <hg@hstuart.dk> parents: 9726 diff changeset	292 not res.chunked and
917cf6bb6d0c url: generalise HTTPS proxy handling to accomodate Python changes Henrik Stuart <hg@hstuart.dk> parents: 9726 diff changeset	293 res.length is None):
917cf6bb6d0c url: generalise HTTPS proxy handling to accomodate Python changes Henrik Stuart <hg@hstuart.dk> parents: 9726 diff changeset	294 res.will_close = 1
917cf6bb6d0c url: generalise HTTPS proxy handling to accomodate Python changes Henrik Stuart <hg@hstuart.dk> parents: 9726 diff changeset	295
917cf6bb6d0c url: generalise HTTPS proxy handling to accomodate Python changes Henrik Stuart <hg@hstuart.dk> parents: 9726 diff changeset	296 self.proxyres = res
917cf6bb6d0c url: generalise HTTPS proxy handling to accomodate Python changes Henrik Stuart <hg@hstuart.dk> parents: 9726 diff changeset	297
917cf6bb6d0c url: generalise HTTPS proxy handling to accomodate Python changes Henrik Stuart <hg@hstuart.dk> parents: 9726 diff changeset	298 return False
917cf6bb6d0c url: generalise HTTPS proxy handling to accomodate Python changes Henrik Stuart <hg@hstuart.dk> parents: 9726 diff changeset	299
7270 2db33c1a5654 factor out the url handling from httprepo Benoit Boissinot <benoit.boissinot@ens-lyon.org> parents: diff changeset	300 class httphandler(keepalive.HTTPHandler):
2db33c1a5654 factor out the url handling from httprepo Benoit Boissinot <benoit.boissinot@ens-lyon.org> parents: diff changeset	301 def http_open(self, req):
2db33c1a5654 factor out the url handling from httprepo Benoit Boissinot <benoit.boissinot@ens-lyon.org> parents: diff changeset	302 return self.do_open(httpconnection, req)
2db33c1a5654 factor out the url handling from httprepo Benoit Boissinot <benoit.boissinot@ens-lyon.org> parents: diff changeset	303
8590 59acb9c7d90f url: use CONNECT for HTTPS connections through HTTP proxy (issue967) Henrik Stuart <hg@hstuart.dk> parents: 8344 diff changeset	304 def _start_transaction(self, h, req):
9852 917cf6bb6d0c url: generalise HTTPS proxy handling to accomodate Python changes Henrik Stuart <hg@hstuart.dk> parents: 9726 diff changeset	305 _generic_start_transaction(self, h, req)
8590 59acb9c7d90f url: use CONNECT for HTTPS connections through HTTP proxy (issue967) Henrik Stuart <hg@hstuart.dk> parents: 8344 diff changeset	306 return keepalive.HTTPHandler._start_transaction(self, h, req)
59acb9c7d90f url: use CONNECT for HTTPS connections through HTTP proxy (issue967) Henrik Stuart <hg@hstuart.dk> parents: 8344 diff changeset	307
37014 02221d6fb041 url: add HTTP handler that uses a proxied socket Gregory Szorc <gregory.szorc@gmail.com> parents: 36681 diff changeset	308 class logginghttpconnection(keepalive.HTTPConnection):
02221d6fb041 url: add HTTP handler that uses a proxied socket Gregory Szorc <gregory.szorc@gmail.com> parents: 36681 diff changeset	309 def __init__(self, createconn, args, *kwargs):
02221d6fb041 url: add HTTP handler that uses a proxied socket Gregory Szorc <gregory.szorc@gmail.com> parents: 36681 diff changeset	310 keepalive.HTTPConnection.__init__(self, args, *kwargs)
02221d6fb041 url: add HTTP handler that uses a proxied socket Gregory Szorc <gregory.szorc@gmail.com> parents: 36681 diff changeset	311 self._create_connection = createconn
02221d6fb041 url: add HTTP handler that uses a proxied socket Gregory Szorc <gregory.szorc@gmail.com> parents: 36681 diff changeset	312
37103 9e6d3465f17e url: make logginghttphandler compatible with Python 2.7.6 Yuya Nishihara <yuya@tcha.org> parents: 37087 diff changeset	313 if sys.version_info < (2, 7, 7):
9e6d3465f17e url: make logginghttphandler compatible with Python 2.7.6 Yuya Nishihara <yuya@tcha.org> parents: 37087 diff changeset	314 # copied from 2.7.14, since old implementations directly call
9e6d3465f17e url: make logginghttphandler compatible with Python 2.7.6 Yuya Nishihara <yuya@tcha.org> parents: 37087 diff changeset	315 # socket.create_connection()
9e6d3465f17e url: make logginghttphandler compatible with Python 2.7.6 Yuya Nishihara <yuya@tcha.org> parents: 37087 diff changeset	316 def connect(self):
9e6d3465f17e url: make logginghttphandler compatible with Python 2.7.6 Yuya Nishihara <yuya@tcha.org> parents: 37087 diff changeset	317 self.sock = self._create_connection((self.host, self.port),
9e6d3465f17e url: make logginghttphandler compatible with Python 2.7.6 Yuya Nishihara <yuya@tcha.org> parents: 37087 diff changeset	318 self.timeout,
9e6d3465f17e url: make logginghttphandler compatible with Python 2.7.6 Yuya Nishihara <yuya@tcha.org> parents: 37087 diff changeset	319 self.source_address)
9e6d3465f17e url: make logginghttphandler compatible with Python 2.7.6 Yuya Nishihara <yuya@tcha.org> parents: 37087 diff changeset	320 if self._tunnel_host:
9e6d3465f17e url: make logginghttphandler compatible with Python 2.7.6 Yuya Nishihara <yuya@tcha.org> parents: 37087 diff changeset	321 self._tunnel()
9e6d3465f17e url: make logginghttphandler compatible with Python 2.7.6 Yuya Nishihara <yuya@tcha.org> parents: 37087 diff changeset	322
37014 02221d6fb041 url: add HTTP handler that uses a proxied socket Gregory Szorc <gregory.szorc@gmail.com> parents: 36681 diff changeset	323 class logginghttphandler(httphandler):
02221d6fb041 url: add HTTP handler that uses a proxied socket Gregory Szorc <gregory.szorc@gmail.com> parents: 36681 diff changeset	324 """HTTP handler that logs socket I/O."""
40044 6509fcec830c url: allow to configure timeout on http connection C?dric Krier <ced@b2ck.com> parents: 40035 diff changeset	325 def __init__(self, logfh, name, observeropts, timeout=None):
6509fcec830c url: allow to configure timeout on http connection C?dric Krier <ced@b2ck.com> parents: 40035 diff changeset	326 super(logginghttphandler, self).__init__(timeout=timeout)
37014 02221d6fb041 url: add HTTP handler that uses a proxied socket Gregory Szorc <gregory.szorc@gmail.com> parents: 36681 diff changeset	327
02221d6fb041 url: add HTTP handler that uses a proxied socket Gregory Szorc <gregory.szorc@gmail.com> parents: 36681 diff changeset	328 self._logfh = logfh
02221d6fb041 url: add HTTP handler that uses a proxied socket Gregory Szorc <gregory.szorc@gmail.com> parents: 36681 diff changeset	329 self._logname = name
02221d6fb041 url: add HTTP handler that uses a proxied socket Gregory Szorc <gregory.szorc@gmail.com> parents: 36681 diff changeset	330 self._observeropts = observeropts
02221d6fb041 url: add HTTP handler that uses a proxied socket Gregory Szorc <gregory.szorc@gmail.com> parents: 36681 diff changeset	331
02221d6fb041 url: add HTTP handler that uses a proxied socket Gregory Szorc <gregory.szorc@gmail.com> parents: 36681 diff changeset	332 # do_open() calls the passed class to instantiate an HTTPConnection. We
02221d6fb041 url: add HTTP handler that uses a proxied socket Gregory Szorc <gregory.szorc@gmail.com> parents: 36681 diff changeset	333 # pass in a callable method that creates a custom HTTPConnection instance
02221d6fb041 url: add HTTP handler that uses a proxied socket Gregory Szorc <gregory.szorc@gmail.com> parents: 36681 diff changeset	334 # whose callback to create the socket knows how to proxy the socket.
02221d6fb041 url: add HTTP handler that uses a proxied socket Gregory Szorc <gregory.szorc@gmail.com> parents: 36681 diff changeset	335 def http_open(self, req):
02221d6fb041 url: add HTTP handler that uses a proxied socket Gregory Szorc <gregory.szorc@gmail.com> parents: 36681 diff changeset	336 return self.do_open(self._makeconnection, req)
02221d6fb041 url: add HTTP handler that uses a proxied socket Gregory Szorc <gregory.szorc@gmail.com> parents: 36681 diff changeset	337
02221d6fb041 url: add HTTP handler that uses a proxied socket Gregory Szorc <gregory.szorc@gmail.com> parents: 36681 diff changeset	338 def _makeconnection(self, args, *kwargs):
02221d6fb041 url: add HTTP handler that uses a proxied socket Gregory Szorc <gregory.szorc@gmail.com> parents: 36681 diff changeset	339 def createconnection(args, *kwargs):
02221d6fb041 url: add HTTP handler that uses a proxied socket Gregory Szorc <gregory.szorc@gmail.com> parents: 36681 diff changeset	340 sock = socket.create_connection(args, *kwargs)
02221d6fb041 url: add HTTP handler that uses a proxied socket Gregory Szorc <gregory.szorc@gmail.com> parents: 36681 diff changeset	341 return util.makeloggingsocket(self._logfh, sock, self._logname,
02221d6fb041 url: add HTTP handler that uses a proxied socket Gregory Szorc <gregory.szorc@gmail.com> parents: 36681 diff changeset	342 **self._observeropts)
02221d6fb041 url: add HTTP handler that uses a proxied socket Gregory Szorc <gregory.szorc@gmail.com> parents: 36681 diff changeset	343
02221d6fb041 url: add HTTP handler that uses a proxied socket Gregory Szorc <gregory.szorc@gmail.com> parents: 36681 diff changeset	344 return logginghttpconnection(createconnection, args, *kwargs)
02221d6fb041 url: add HTTP handler that uses a proxied socket Gregory Szorc <gregory.szorc@gmail.com> parents: 36681 diff changeset	345
7270 2db33c1a5654 factor out the url handling from httprepo Benoit Boissinot <benoit.boissinot@ens-lyon.org> parents: diff changeset	346 if has_https:
40032 f2dffa1359c6 url: have httpsconnection inherit from our custom HTTPConnection Gregory Szorc <gregory.szorc@gmail.com> parents: 40019 diff changeset	347 class httpsconnection(keepalive.HTTPConnection):
13424 08f9c587141f url: merge BetterHTTPS with httpsconnection to get some proxy https validation Mads Kiilerich <mads@kiilerich.com> parents: 13422 diff changeset	348 response_class = keepalive.HTTPResponse
25414 f7ccbc2776b7 https: do not inherit httplib.HTTPSConnection that creates unused SSLContext Yuya Nishihara <yuya@tcha.org> parents: 25207 diff changeset	349 default_port = httplib.HTTPS_PORT
13424 08f9c587141f url: merge BetterHTTPS with httpsconnection to get some proxy https validation Mads Kiilerich <mads@kiilerich.com> parents: 13422 diff changeset	350 # must be able to send big bundle as stream.
08f9c587141f url: merge BetterHTTPS with httpsconnection to get some proxy https validation Mads Kiilerich <mads@kiilerich.com> parents: 13422 diff changeset	351 send = _gen_sendfile(keepalive.safesend)
25414 f7ccbc2776b7 https: do not inherit httplib.HTTPSConnection that creates unused SSLContext Yuya Nishihara <yuya@tcha.org> parents: 25207 diff changeset	352 getresponse = keepalive.wrapgetresponse(httplib.HTTPConnection)
f7ccbc2776b7 https: do not inherit httplib.HTTPSConnection that creates unused SSLContext Yuya Nishihara <yuya@tcha.org> parents: 25207 diff changeset	353
f7ccbc2776b7 https: do not inherit httplib.HTTPSConnection that creates unused SSLContext Yuya Nishihara <yuya@tcha.org> parents: 25207 diff changeset	354 def __init__(self, host, port=None, key_file=None, cert_file=None,
f7ccbc2776b7 https: do not inherit httplib.HTTPSConnection that creates unused SSLContext Yuya Nishihara <yuya@tcha.org> parents: 25207 diff changeset	355 args, *kwargs):
40032 f2dffa1359c6 url: have httpsconnection inherit from our custom HTTPConnection Gregory Szorc <gregory.szorc@gmail.com> parents: 40019 diff changeset	356 keepalive.HTTPConnection.__init__(self, host, port, args, *kwargs)
25414 f7ccbc2776b7 https: do not inherit httplib.HTTPSConnection that creates unused SSLContext Yuya Nishihara <yuya@tcha.org> parents: 25207 diff changeset	357 self.key_file = key_file
f7ccbc2776b7 https: do not inherit httplib.HTTPSConnection that creates unused SSLContext Yuya Nishihara <yuya@tcha.org> parents: 25207 diff changeset	358 self.cert_file = cert_file
9726 430e59ff3437 keepalive: handle broken pipes gracefully during large POSTs Augie Fackler <durin42@gmail.com> parents: 9467 diff changeset	359
10409 4c94a3df4b10 url: SSL server certificate verification using web.cacerts file (issue1174) Henrik Stuart <hg@hstuart.dk> parents: 10408 diff changeset	360 def connect(self):
29682 b76ea1384bf2 url: drop compatibility wrapper of socket.create_connection() Yuya Nishihara <yuya@tcha.org> parents: 29639 diff changeset	361 self.sock = socket.create_connection((self.host, self.port))
13422 ebce5196b9db url: always create BetterHTTPS connections the same way Mads Kiilerich <mads@kiilerich.com> parents: 13421 diff changeset	362
13421 bd8bfa85d5a5 url: refactor BetterHTTPS.connect Mads Kiilerich <mads@kiilerich.com> parents: 13420 diff changeset	363 host = self.host
13424 08f9c587141f url: merge BetterHTTPS with httpsconnection to get some proxy https validation Mads Kiilerich <mads@kiilerich.com> parents: 13422 diff changeset	364 if self.realhostport: # use CONNECT proxy
14064 e4bfb9c337f3 remove unused imports and variables Alexander Solovyov <alexander@solovyov.net> parents: 13902 diff changeset	365 _generic_proxytunnel(self)
13424 08f9c587141f url: merge BetterHTTPS with httpsconnection to get some proxy https validation Mads Kiilerich <mads@kiilerich.com> parents: 13422 diff changeset	366 host = self.realhostport.rsplit(':', 1)[0]
25429 9d1c61715939 ssl: rename ssl_wrap_socket() to conform to our naming convention Yuya Nishihara <yuya@tcha.org> parents: 25415 diff changeset	367 self.sock = sslutil.wrapsocket(
29248 e6de6ef3e426 sslutil: remove ui from sslkwargs (API) Gregory Szorc <gregory.szorc@gmail.com> parents: 29227 diff changeset	368 self.sock, self.key_file, self.cert_file, ui=self.ui,
29252 f1fe92c6c03c url: remove use of sslkwargs Gregory Szorc <gregory.szorc@gmail.com> parents: 29248 diff changeset	369 serverhostname=host)
29227 dffe78d80a6c sslutil: convert socket validation from a class to a function (API) Gregory Szorc <gregory.szorc@gmail.com> parents: 28883 diff changeset	370 sslutil.validatesocket(self.sock)
10409 4c94a3df4b10 url: SSL server certificate verification using web.cacerts file (issue1174) Henrik Stuart <hg@hstuart.dk> parents: 10408 diff changeset	371
28883 032c4c2f802a pycompat: switch to util.urlreq/util.urlerr for py3 compat timeless <timeless@mozdev.org> parents: 28861 diff changeset	372 class httpshandler(keepalive.KeepAliveHandler, urlreq.httpshandler):
40044 6509fcec830c url: allow to configure timeout on http connection C?dric Krier <ced@b2ck.com> parents: 40035 diff changeset	373 def __init__(self, ui, timeout=None):
6509fcec830c url: allow to configure timeout on http connection C?dric Krier <ced@b2ck.com> parents: 40035 diff changeset	374 keepalive.KeepAliveHandler.__init__(self, timeout=timeout)
28883 032c4c2f802a pycompat: switch to util.urlreq/util.urlerr for py3 compat timeless <timeless@mozdev.org> parents: 28861 diff changeset	375 urlreq.httpshandler.__init__(self)
8847 7951f385fcb7 url: support client certificate files over HTTPS (issue643) Henrik Stuart <hg@hstuart.dk> parents: 8590 diff changeset	376 self.ui = ui
29377 2c019aac6b99 url: extract password database from password manager liscju <piotr.listkiewicz@gmail.com> parents: 29252 diff changeset	377 self.pwmgr = passwordmgr(self.ui,
29378 fea71f66ebff url: remember http password database in ui object liscju <piotr.listkiewicz@gmail.com> parents: 29377 diff changeset	378 self.ui.httppasswordmgrdb)
8847 7951f385fcb7 url: support client certificate files over HTTPS (issue643) Henrik Stuart <hg@hstuart.dk> parents: 8590 diff changeset	379
9852 917cf6bb6d0c url: generalise HTTPS proxy handling to accomodate Python changes Henrik Stuart <hg@hstuart.dk> parents: 9726 diff changeset	380 def _start_transaction(self, h, req):
917cf6bb6d0c url: generalise HTTPS proxy handling to accomodate Python changes Henrik Stuart <hg@hstuart.dk> parents: 9726 diff changeset	381 _generic_start_transaction(self, h, req)
917cf6bb6d0c url: generalise HTTPS proxy handling to accomodate Python changes Henrik Stuart <hg@hstuart.dk> parents: 9726 diff changeset	382 return keepalive.KeepAliveHandler._start_transaction(self, h, req)
917cf6bb6d0c url: generalise HTTPS proxy handling to accomodate Python changes Henrik Stuart <hg@hstuart.dk> parents: 9726 diff changeset	383
7270 2db33c1a5654 factor out the url handling from httprepo Benoit Boissinot <benoit.boissinot@ens-lyon.org> parents: diff changeset	384 def https_open(self, req):
34467 1232f7fa00c3 cleanup: use urllibcompat for renamed methods on urllib request objects Augie Fackler <augie@google.com> parents: 34429 diff changeset	385 # urllibcompat.getfullurl() does not contain credentials
1232f7fa00c3 cleanup: use urllibcompat for renamed methods on urllib request objects Augie Fackler <augie@google.com> parents: 34429 diff changeset	386 # and we may need them to match the certificates.
1232f7fa00c3 cleanup: use urllibcompat for renamed methods on urllib request objects Augie Fackler <augie@google.com> parents: 34429 diff changeset	387 url = urllibcompat.getfullurl(req)
15025 0593e8f81c71 http: pass user to readauthforuri() (fix 4a43e23b8c55) Patrick Mezard <pmezard@gmail.com> parents: 15005 diff changeset	388 user, password = self.pwmgr.find_stored_password(url)
0593e8f81c71 http: pass user to readauthforuri() (fix 4a43e23b8c55) Patrick Mezard <pmezard@gmail.com> parents: 15005 diff changeset	389 res = httpconnectionmod.readauthforuri(self.ui, url, user)
13372 5bced0d28a39 url: return the matched authentication group name from readauthforuri() Steve Borho <steve@borho.org> parents: 13371 diff changeset	390 if res:
5bced0d28a39 url: return the matched authentication group name from readauthforuri() Steve Borho <steve@borho.org> parents: 13371 diff changeset	391 group, auth = res
5bced0d28a39 url: return the matched authentication group name from readauthforuri() Steve Borho <steve@borho.org> parents: 13371 diff changeset	392 self.auth = auth
5bced0d28a39 url: return the matched authentication group name from readauthforuri() Steve Borho <steve@borho.org> parents: 13371 diff changeset	393 self.ui.debug("using auth.%s.* for authentication\n" % group)
5bced0d28a39 url: return the matched authentication group name from readauthforuri() Steve Borho <steve@borho.org> parents: 13371 diff changeset	394 else:
5bced0d28a39 url: return the matched authentication group name from readauthforuri() Steve Borho <steve@borho.org> parents: 13371 diff changeset	395 self.auth = None
8847 7951f385fcb7 url: support client certificate files over HTTPS (issue643) Henrik Stuart <hg@hstuart.dk> parents: 8590 diff changeset	396 return self.do_open(self._makeconnection, req)
7951f385fcb7 url: support client certificate files over HTTPS (issue643) Henrik Stuart <hg@hstuart.dk> parents: 8590 diff changeset	397
10408 50fb1fe143ff url: httplib.HTTPSConnection already handles IPv6 and port parsing fine Benoit Boissinot <benoit.boissinot@ens-lyon.org> parents: 10282 diff changeset	398 def _makeconnection(self, host, port=None, args, *kwargs):
8847 7951f385fcb7 url: support client certificate files over HTTPS (issue643) Henrik Stuart <hg@hstuart.dk> parents: 8590 diff changeset	399 keyfile = None
7951f385fcb7 url: support client certificate files over HTTPS (issue643) Henrik Stuart <hg@hstuart.dk> parents: 8590 diff changeset	400 certfile = None
7951f385fcb7 url: support client certificate files over HTTPS (issue643) Henrik Stuart <hg@hstuart.dk> parents: 8590 diff changeset	401
10511 6f61c480f51c url: args argument is a tuple, not a list (found by pylint) Benoit Boissinot <benoit.boissinot@ens-lyon.org>* parents: 10484 diff changeset	402 if len(args) >= 1: # key_file
6f61c480f51c url: args argument is a tuple, not a list (found by pylint) Benoit Boissinot <benoit.boissinot@ens-lyon.org>* parents: 10484 diff changeset	403 keyfile = args[0]
6f61c480f51c url: args argument is a tuple, not a list (found by pylint) Benoit Boissinot <benoit.boissinot@ens-lyon.org>* parents: 10484 diff changeset	404 if len(args) >= 2: # cert_file
6f61c480f51c url: args argument is a tuple, not a list (found by pylint) Benoit Boissinot <benoit.boissinot@ens-lyon.org>* parents: 10484 diff changeset	405 certfile = args[1]
6f61c480f51c url: args argument is a tuple, not a list (found by pylint) Benoit Boissinot <benoit.boissinot@ens-lyon.org>* parents: 10484 diff changeset	406 args = args[2:]
8847 7951f385fcb7 url: support client certificate files over HTTPS (issue643) Henrik Stuart <hg@hstuart.dk> parents: 8590 diff changeset	407
7951f385fcb7 url: support client certificate files over HTTPS (issue643) Henrik Stuart <hg@hstuart.dk> parents: 8590 diff changeset	408 # if the user has specified different key/cert files in
7951f385fcb7 url: support client certificate files over HTTPS (issue643) Henrik Stuart <hg@hstuart.dk> parents: 8590 diff changeset	409 # hgrc, we prefer these
7951f385fcb7 url: support client certificate files over HTTPS (issue643) Henrik Stuart <hg@hstuart.dk> parents: 8590 diff changeset	410 if self.auth and 'key' in self.auth and 'cert' in self.auth:
7951f385fcb7 url: support client certificate files over HTTPS (issue643) Henrik Stuart <hg@hstuart.dk> parents: 8590 diff changeset	411 keyfile = self.auth['key']
7951f385fcb7 url: support client certificate files over HTTPS (issue643) Henrik Stuart <hg@hstuart.dk> parents: 8590 diff changeset	412 certfile = self.auth['cert']
7951f385fcb7 url: support client certificate files over HTTPS (issue643) Henrik Stuart <hg@hstuart.dk> parents: 8590 diff changeset	413
16683 525fdb738975 cleanup: eradicate long lines Brodie Rao <brodie@sf.io> parents: 15081 diff changeset	414 conn = httpsconnection(host, port, keyfile, certfile, *args,
525fdb738975 cleanup: eradicate long lines Brodie Rao <brodie@sf.io> parents: 15081 diff changeset	415 **kwargs)
10409 4c94a3df4b10 url: SSL server certificate verification using web.cacerts file (issue1174) Henrik Stuart <hg@hstuart.dk> parents: 10408 diff changeset	416 conn.ui = self.ui
4c94a3df4b10 url: SSL server certificate verification using web.cacerts file (issue1174) Henrik Stuart <hg@hstuart.dk> parents: 10408 diff changeset	417 return conn
7270 2db33c1a5654 factor out the url handling from httprepo Benoit Boissinot <benoit.boissinot@ens-lyon.org> parents: diff changeset	418
28883 032c4c2f802a pycompat: switch to util.urlreq/util.urlerr for py3 compat timeless <timeless@mozdev.org> parents: 28861 diff changeset	419 class httpdigestauthhandler(urlreq.httpdigestauthhandler):
11457 2ec346160783 http digest auth: reset redirect counter on new requests (issue2255) Mads Kiilerich <mads@kiilerich.com> parents: 11415 diff changeset	420 def __init__(self, args, *kwargs):
28883 032c4c2f802a pycompat: switch to util.urlreq/util.urlerr for py3 compat timeless <timeless@mozdev.org> parents: 28861 diff changeset	421 urlreq.httpdigestauthhandler.__init__(self, args, *kwargs)
11457 2ec346160783 http digest auth: reset redirect counter on new requests (issue2255) Mads Kiilerich <mads@kiilerich.com> parents: 11415 diff changeset	422 self.retried_req = None
2ec346160783 http digest auth: reset redirect counter on new requests (issue2255) Mads Kiilerich <mads@kiilerich.com> parents: 11415 diff changeset	423
2ec346160783 http digest auth: reset redirect counter on new requests (issue2255) Mads Kiilerich <mads@kiilerich.com> parents: 11415 diff changeset	424 def reset_retry_count(self):
2ec346160783 http digest auth: reset redirect counter on new requests (issue2255) Mads Kiilerich <mads@kiilerich.com> parents: 11415 diff changeset	425 # Python 2.6.5 will call this on 401 or 407 errors and thus loop
2ec346160783 http digest auth: reset redirect counter on new requests (issue2255) Mads Kiilerich <mads@kiilerich.com> parents: 11415 diff changeset	426 # forever. We disable reset_retry_count completely and reset in
2ec346160783 http digest auth: reset redirect counter on new requests (issue2255) Mads Kiilerich <mads@kiilerich.com> parents: 11415 diff changeset	427 # http_error_auth_reqed instead.
2ec346160783 http digest auth: reset redirect counter on new requests (issue2255) Mads Kiilerich <mads@kiilerich.com> parents: 11415 diff changeset	428 pass
2ec346160783 http digest auth: reset redirect counter on new requests (issue2255) Mads Kiilerich <mads@kiilerich.com> parents: 11415 diff changeset	429
7270 2db33c1a5654 factor out the url handling from httprepo Benoit Boissinot <benoit.boissinot@ens-lyon.org> parents: diff changeset	430 def http_error_auth_reqed(self, auth_header, host, req, headers):
11457 2ec346160783 http digest auth: reset redirect counter on new requests (issue2255) Mads Kiilerich <mads@kiilerich.com> parents: 11415 diff changeset	431 # Reset the retry counter once for each request.
2ec346160783 http digest auth: reset redirect counter on new requests (issue2255) Mads Kiilerich <mads@kiilerich.com> parents: 11415 diff changeset	432 if req is not self.retried_req:
2ec346160783 http digest auth: reset redirect counter on new requests (issue2255) Mads Kiilerich <mads@kiilerich.com> parents: 11415 diff changeset	433 self.retried_req = req
2ec346160783 http digest auth: reset redirect counter on new requests (issue2255) Mads Kiilerich <mads@kiilerich.com> parents: 11415 diff changeset	434 self.retried = 0
28883 032c4c2f802a pycompat: switch to util.urlreq/util.urlerr for py3 compat timeless <timeless@mozdev.org> parents: 28861 diff changeset	435 return urlreq.httpdigestauthhandler.http_error_auth_reqed(
26806 ec12ebe20200 url: drop support for python2.5 timeless <timeless@mozdev.org> parents: 26587 diff changeset	436 self, auth_header, host, req, headers)
7270 2db33c1a5654 factor out the url handling from httprepo Benoit Boissinot <benoit.boissinot@ens-lyon.org> parents: diff changeset	437
28883 032c4c2f802a pycompat: switch to util.urlreq/util.urlerr for py3 compat timeless <timeless@mozdev.org> parents: 28861 diff changeset	438 class httpbasicauthhandler(urlreq.httpbasicauthhandler):
11844 6c51a5056020 http basic auth: reset redirect counter on new requests (issue2255) Wagner Bruna <wbruna@softwareexpress.com.br> parents: 11457 diff changeset	439 def __init__(self, args, *kwargs):
20964 a939eeb94833 http: reuse authentication info after the first failed request (issue3567) St?phane Klein <contact@stephane-klein.info> parents: 20291 diff changeset	440 self.auth = None
28883 032c4c2f802a pycompat: switch to util.urlreq/util.urlerr for py3 compat timeless <timeless@mozdev.org> parents: 28861 diff changeset	441 urlreq.httpbasicauthhandler.__init__(self, args, *kwargs)
11844 6c51a5056020 http basic auth: reset redirect counter on new requests (issue2255) Wagner Bruna <wbruna@softwareexpress.com.br> parents: 11457 diff changeset	442 self.retried_req = None
6c51a5056020 http basic auth: reset redirect counter on new requests (issue2255) Wagner Bruna <wbruna@softwareexpress.com.br> parents: 11457 diff changeset	443
20964 a939eeb94833 http: reuse authentication info after the first failed request (issue3567) St?phane Klein <contact@stephane-klein.info> parents: 20291 diff changeset	444 def http_request(self, request):
a939eeb94833 http: reuse authentication info after the first failed request (issue3567) St?phane Klein <contact@stephane-klein.info> parents: 20291 diff changeset	445 if self.auth:
a939eeb94833 http: reuse authentication info after the first failed request (issue3567) St?phane Klein <contact@stephane-klein.info> parents: 20291 diff changeset	446 request.add_unredirected_header(self.auth_header, self.auth)
a939eeb94833 http: reuse authentication info after the first failed request (issue3567) St?phane Klein <contact@stephane-klein.info> parents: 20291 diff changeset	447
a939eeb94833 http: reuse authentication info after the first failed request (issue3567) St?phane Klein <contact@stephane-klein.info> parents: 20291 diff changeset	448 return request
a939eeb94833 http: reuse authentication info after the first failed request (issue3567) St?phane Klein <contact@stephane-klein.info> parents: 20291 diff changeset	449
a939eeb94833 http: reuse authentication info after the first failed request (issue3567) St?phane Klein <contact@stephane-klein.info> parents: 20291 diff changeset	450 def https_request(self, request):
a939eeb94833 http: reuse authentication info after the first failed request (issue3567) St?phane Klein <contact@stephane-klein.info> parents: 20291 diff changeset	451 if self.auth:
a939eeb94833 http: reuse authentication info after the first failed request (issue3567) St?phane Klein <contact@stephane-klein.info> parents: 20291 diff changeset	452 request.add_unredirected_header(self.auth_header, self.auth)
a939eeb94833 http: reuse authentication info after the first failed request (issue3567) St?phane Klein <contact@stephane-klein.info> parents: 20291 diff changeset	453
a939eeb94833 http: reuse authentication info after the first failed request (issue3567) St?phane Klein <contact@stephane-klein.info> parents: 20291 diff changeset	454 return request
a939eeb94833 http: reuse authentication info after the first failed request (issue3567) St?phane Klein <contact@stephane-klein.info> parents: 20291 diff changeset	455
11844 6c51a5056020 http basic auth: reset redirect counter on new requests (issue2255) Wagner Bruna <wbruna@softwareexpress.com.br> parents: 11457 diff changeset	456 def reset_retry_count(self):
6c51a5056020 http basic auth: reset redirect counter on new requests (issue2255) Wagner Bruna <wbruna@softwareexpress.com.br> parents: 11457 diff changeset	457 # Python 2.6.5 will call this on 401 or 407 errors and thus loop
6c51a5056020 http basic auth: reset redirect counter on new requests (issue2255) Wagner Bruna <wbruna@softwareexpress.com.br> parents: 11457 diff changeset	458 # forever. We disable reset_retry_count completely and reset in
6c51a5056020 http basic auth: reset redirect counter on new requests (issue2255) Wagner Bruna <wbruna@softwareexpress.com.br> parents: 11457 diff changeset	459 # http_error_auth_reqed instead.
6c51a5056020 http basic auth: reset redirect counter on new requests (issue2255) Wagner Bruna <wbruna@softwareexpress.com.br> parents: 11457 diff changeset	460 pass
6c51a5056020 http basic auth: reset redirect counter on new requests (issue2255) Wagner Bruna <wbruna@softwareexpress.com.br> parents: 11457 diff changeset	461
6c51a5056020 http basic auth: reset redirect counter on new requests (issue2255) Wagner Bruna <wbruna@softwareexpress.com.br> parents: 11457 diff changeset	462 def http_error_auth_reqed(self, auth_header, host, req, headers):
6c51a5056020 http basic auth: reset redirect counter on new requests (issue2255) Wagner Bruna <wbruna@softwareexpress.com.br> parents: 11457 diff changeset	463 # Reset the retry counter once for each request.
6c51a5056020 http basic auth: reset redirect counter on new requests (issue2255) Wagner Bruna <wbruna@softwareexpress.com.br> parents: 11457 diff changeset	464 if req is not self.retried_req:
6c51a5056020 http basic auth: reset redirect counter on new requests (issue2255) Wagner Bruna <wbruna@softwareexpress.com.br> parents: 11457 diff changeset	465 self.retried_req = req
6c51a5056020 http basic auth: reset redirect counter on new requests (issue2255) Wagner Bruna <wbruna@softwareexpress.com.br> parents: 11457 diff changeset	466 self.retried = 0
28883 032c4c2f802a pycompat: switch to util.urlreq/util.urlerr for py3 compat timeless <timeless@mozdev.org> parents: 28861 diff changeset	467 return urlreq.httpbasicauthhandler.http_error_auth_reqed(
11844 6c51a5056020 http basic auth: reset redirect counter on new requests (issue2255) Wagner Bruna <wbruna@softwareexpress.com.br> parents: 11457 diff changeset	468 self, auth_header, host, req, headers)
6c51a5056020 http basic auth: reset redirect counter on new requests (issue2255) Wagner Bruna <wbruna@softwareexpress.com.br> parents: 11457 diff changeset	469
20964 a939eeb94833 http: reuse authentication info after the first failed request (issue3567) St?phane Klein <contact@stephane-klein.info> parents: 20291 diff changeset	470 def retry_http_basic_auth(self, host, req, realm):
34467 1232f7fa00c3 cleanup: use urllibcompat for renamed methods on urllib request objects Augie Fackler <augie@google.com> parents: 34429 diff changeset	471 user, pw = self.passwd.find_user_password(
1232f7fa00c3 cleanup: use urllibcompat for renamed methods on urllib request objects Augie Fackler <augie@google.com> parents: 34429 diff changeset	472 realm, urllibcompat.getfullurl(req))
20964 a939eeb94833 http: reuse authentication info after the first failed request (issue3567) St?phane Klein <contact@stephane-klein.info> parents: 20291 diff changeset	473 if pw is not None:
36681 8381126bf43c url: more bytes/unicodes fussing in url.py around auth handling Augie Fackler <augie@google.com> parents: 36484 diff changeset	474 raw = "%s:%s" % (pycompat.bytesurl(user), pycompat.bytesurl(pw))
8381126bf43c url: more bytes/unicodes fussing in url.py around auth handling Augie Fackler <augie@google.com> parents: 36484 diff changeset	475 auth = r'Basic %s' % pycompat.strurl(base64.b64encode(raw).strip())
29639 6fd751fa58d3 url: avoid re-issuing incorrect password (issue3210) Kim Randell <Kim.Randell@vicon.com> parents: 29599 diff changeset	476 if req.get_header(self.auth_header, None) == auth:
20964 a939eeb94833 http: reuse authentication info after the first failed request (issue3567) St?phane Klein <contact@stephane-klein.info> parents: 20291 diff changeset	477 return None
a939eeb94833 http: reuse authentication info after the first failed request (issue3567) St?phane Klein <contact@stephane-klein.info> parents: 20291 diff changeset	478 self.auth = auth
a939eeb94833 http: reuse authentication info after the first failed request (issue3567) St?phane Klein <contact@stephane-klein.info> parents: 20291 diff changeset	479 req.add_unredirected_header(self.auth_header, auth)
a939eeb94833 http: reuse authentication info after the first failed request (issue3567) St?phane Klein <contact@stephane-klein.info> parents: 20291 diff changeset	480 return self.parent.open(req)
a939eeb94833 http: reuse authentication info after the first failed request (issue3567) St?phane Klein <contact@stephane-klein.info> parents: 20291 diff changeset	481 else:
a939eeb94833 http: reuse authentication info after the first failed request (issue3567) St?phane Klein <contact@stephane-klein.info> parents: 20291 diff changeset	482 return None
a939eeb94833 http: reuse authentication info after the first failed request (issue3567) St?phane Klein <contact@stephane-klein.info> parents: 20291 diff changeset	483
31936 806f9a883b4f url: support auth.cookiesfile for adding cookies to HTTP requests Gregory Szorc <gregory.szorc@gmail.com> parents: 30820 diff changeset	484 class cookiehandler(urlreq.basehandler):
806f9a883b4f url: support auth.cookiesfile for adding cookies to HTTP requests Gregory Szorc <gregory.szorc@gmail.com> parents: 30820 diff changeset	485 def __init__(self, ui):
806f9a883b4f url: support auth.cookiesfile for adding cookies to HTTP requests Gregory Szorc <gregory.szorc@gmail.com> parents: 30820 diff changeset	486 self.cookiejar = None
806f9a883b4f url: support auth.cookiesfile for adding cookies to HTTP requests Gregory Szorc <gregory.szorc@gmail.com> parents: 30820 diff changeset	487
806f9a883b4f url: support auth.cookiesfile for adding cookies to HTTP requests Gregory Szorc <gregory.szorc@gmail.com> parents: 30820 diff changeset	488 cookiefile = ui.config('auth', 'cookiefile')
806f9a883b4f url: support auth.cookiesfile for adding cookies to HTTP requests Gregory Szorc <gregory.szorc@gmail.com> parents: 30820 diff changeset	489 if not cookiefile:
806f9a883b4f url: support auth.cookiesfile for adding cookies to HTTP requests Gregory Szorc <gregory.szorc@gmail.com> parents: 30820 diff changeset	490 return
806f9a883b4f url: support auth.cookiesfile for adding cookies to HTTP requests Gregory Szorc <gregory.szorc@gmail.com> parents: 30820 diff changeset	491
806f9a883b4f url: support auth.cookiesfile for adding cookies to HTTP requests Gregory Szorc <gregory.szorc@gmail.com> parents: 30820 diff changeset	492 cookiefile = util.expandpath(cookiefile)
806f9a883b4f url: support auth.cookiesfile for adding cookies to HTTP requests Gregory Szorc <gregory.szorc@gmail.com> parents: 30820 diff changeset	493 try:
37735 126998dcfb08 url: some bytes/str cleanup where we interface with stdlib funcs Augie Fackler <augie@google.com> parents: 37103 diff changeset	494 cookiejar = util.cookielib.MozillaCookieJar(
126998dcfb08 url: some bytes/str cleanup where we interface with stdlib funcs Augie Fackler <augie@google.com> parents: 37103 diff changeset	495 pycompat.fsdecode(cookiefile))
31936 806f9a883b4f url: support auth.cookiesfile for adding cookies to HTTP requests Gregory Szorc <gregory.szorc@gmail.com> parents: 30820 diff changeset	496 cookiejar.load()
806f9a883b4f url: support auth.cookiesfile for adding cookies to HTTP requests Gregory Szorc <gregory.szorc@gmail.com> parents: 30820 diff changeset	497 self.cookiejar = cookiejar
806f9a883b4f url: support auth.cookiesfile for adding cookies to HTTP requests Gregory Szorc <gregory.szorc@gmail.com> parents: 30820 diff changeset	498 except util.cookielib.LoadError as e:
806f9a883b4f url: support auth.cookiesfile for adding cookies to HTTP requests Gregory Szorc <gregory.szorc@gmail.com> parents: 30820 diff changeset	499 ui.warn(_('(error loading cookie file %s: %s; continuing without '
37087 f0b6fbea00cf stringutil: bulk-replace call sites to point to new module Yuya Nishihara <yuya@tcha.org> parents: 37048 diff changeset	500 'cookies)\n') % (cookiefile, stringutil.forcebytestr(e)))
31936 806f9a883b4f url: support auth.cookiesfile for adding cookies to HTTP requests Gregory Szorc <gregory.szorc@gmail.com> parents: 30820 diff changeset	501
806f9a883b4f url: support auth.cookiesfile for adding cookies to HTTP requests Gregory Szorc <gregory.szorc@gmail.com> parents: 30820 diff changeset	502 def http_request(self, request):
806f9a883b4f url: support auth.cookiesfile for adding cookies to HTTP requests Gregory Szorc <gregory.szorc@gmail.com> parents: 30820 diff changeset	503 if self.cookiejar:
806f9a883b4f url: support auth.cookiesfile for adding cookies to HTTP requests Gregory Szorc <gregory.szorc@gmail.com> parents: 30820 diff changeset	504 self.cookiejar.add_cookie_header(request)
806f9a883b4f url: support auth.cookiesfile for adding cookies to HTTP requests Gregory Szorc <gregory.szorc@gmail.com> parents: 30820 diff changeset	505
806f9a883b4f url: support auth.cookiesfile for adding cookies to HTTP requests Gregory Szorc <gregory.szorc@gmail.com> parents: 30820 diff changeset	506 return request
806f9a883b4f url: support auth.cookiesfile for adding cookies to HTTP requests Gregory Szorc <gregory.szorc@gmail.com> parents: 30820 diff changeset	507
806f9a883b4f url: support auth.cookiesfile for adding cookies to HTTP requests Gregory Szorc <gregory.szorc@gmail.com> parents: 30820 diff changeset	508 def https_request(self, request):
806f9a883b4f url: support auth.cookiesfile for adding cookies to HTTP requests Gregory Szorc <gregory.szorc@gmail.com> parents: 30820 diff changeset	509 if self.cookiejar:
806f9a883b4f url: support auth.cookiesfile for adding cookies to HTTP requests Gregory Szorc <gregory.szorc@gmail.com> parents: 30820 diff changeset	510 self.cookiejar.add_cookie_header(request)
806f9a883b4f url: support auth.cookiesfile for adding cookies to HTTP requests Gregory Szorc <gregory.szorc@gmail.com> parents: 30820 diff changeset	511
806f9a883b4f url: support auth.cookiesfile for adding cookies to HTTP requests Gregory Szorc <gregory.szorc@gmail.com> parents: 30820 diff changeset	512 return request
806f9a883b4f url: support auth.cookiesfile for adding cookies to HTTP requests Gregory Szorc <gregory.szorc@gmail.com> parents: 30820 diff changeset	513
9347 d0474b184347 url: add support for custom handlers in extensions Henrik Stuart <hg@hstuart.dk> parents: 9122 diff changeset	514 handlerfuncs = []
d0474b184347 url: add support for custom handlers in extensions Henrik Stuart <hg@hstuart.dk> parents: 9122 diff changeset	515
37014 02221d6fb041 url: add HTTP handler that uses a proxied socket Gregory Szorc <gregory.szorc@gmail.com> parents: 36681 diff changeset	516 def opener(ui, authinfo=None, useragent=None, loggingfh=None,
37048 a708e1e4d7a8 url: support suppressing Accept header Gregory Szorc <gregory.szorc@gmail.com> parents: 37014 diff changeset	517 loggingname=b's', loggingopts=None, sendaccept=True):
7270 2db33c1a5654 factor out the url handling from httprepo Benoit Boissinot <benoit.boissinot@ens-lyon.org> parents: diff changeset	518 '''
2db33c1a5654 factor out the url handling from httprepo Benoit Boissinot <benoit.boissinot@ens-lyon.org> parents: diff changeset	519 construct an opener suitable for urllib2
2db33c1a5654 factor out the url handling from httprepo Benoit Boissinot <benoit.boissinot@ens-lyon.org> parents: diff changeset	520 authinfo will be added to the password manager
37014 02221d6fb041 url: add HTTP handler that uses a proxied socket Gregory Szorc <gregory.szorc@gmail.com> parents: 36681 diff changeset	521
02221d6fb041 url: add HTTP handler that uses a proxied socket Gregory Szorc <gregory.szorc@gmail.com> parents: 36681 diff changeset	522 The opener can be configured to log socket events if the various
02221d6fb041 url: add HTTP handler that uses a proxied socket Gregory Szorc <gregory.szorc@gmail.com> parents: 36681 diff changeset	523 ``logging*`` arguments are specified.
02221d6fb041 url: add HTTP handler that uses a proxied socket Gregory Szorc <gregory.szorc@gmail.com> parents: 36681 diff changeset	524
02221d6fb041 url: add HTTP handler that uses a proxied socket Gregory Szorc <gregory.szorc@gmail.com> parents: 36681 diff changeset	525 ``loggingfh`` denotes a file object to log events to.
02221d6fb041 url: add HTTP handler that uses a proxied socket Gregory Szorc <gregory.szorc@gmail.com> parents: 36681 diff changeset	526 ``loggingname`` denotes the name of the to print when logging.
02221d6fb041 url: add HTTP handler that uses a proxied socket Gregory Szorc <gregory.szorc@gmail.com> parents: 36681 diff changeset	527 ``loggingopts`` is a dict of keyword arguments to pass to the constructed
02221d6fb041 url: add HTTP handler that uses a proxied socket Gregory Szorc <gregory.szorc@gmail.com> parents: 36681 diff changeset	528 ``util.socketobserver`` instance.
37048 a708e1e4d7a8 url: support suppressing Accept header Gregory Szorc <gregory.szorc@gmail.com> parents: 37014 diff changeset	529
a708e1e4d7a8 url: support suppressing Accept header Gregory Szorc <gregory.szorc@gmail.com> parents: 37014 diff changeset	530 ``sendaccept`` allows controlling whether the ``Accept`` request header
a708e1e4d7a8 url: support suppressing Accept header Gregory Szorc <gregory.szorc@gmail.com> parents: 37014 diff changeset	531 is sent. The header is sent by default.
7270 2db33c1a5654 factor out the url handling from httprepo Benoit Boissinot <benoit.boissinot@ens-lyon.org> parents: diff changeset	532 '''
40044 6509fcec830c url: allow to configure timeout on http connection C?dric Krier <ced@b2ck.com> parents: 40035 diff changeset	533 timeout = ui.configwith(float, 'http', 'timeout')
37014 02221d6fb041 url: add HTTP handler that uses a proxied socket Gregory Szorc <gregory.szorc@gmail.com> parents: 36681 diff changeset	534 handlers = []
02221d6fb041 url: add HTTP handler that uses a proxied socket Gregory Szorc <gregory.szorc@gmail.com> parents: 36681 diff changeset	535
02221d6fb041 url: add HTTP handler that uses a proxied socket Gregory Szorc <gregory.szorc@gmail.com> parents: 36681 diff changeset	536 if loggingfh:
02221d6fb041 url: add HTTP handler that uses a proxied socket Gregory Szorc <gregory.szorc@gmail.com> parents: 36681 diff changeset	537 handlers.append(logginghttphandler(loggingfh, loggingname,
40044 6509fcec830c url: allow to configure timeout on http connection C?dric Krier <ced@b2ck.com> parents: 40035 diff changeset	538 loggingopts or {}, timeout=timeout))
37014 02221d6fb041 url: add HTTP handler that uses a proxied socket Gregory Szorc <gregory.szorc@gmail.com> parents: 36681 diff changeset	539 # We don't yet support HTTPS when logging I/O. If we attempt to open
02221d6fb041 url: add HTTP handler that uses a proxied socket Gregory Szorc <gregory.szorc@gmail.com> parents: 36681 diff changeset	540 # an HTTPS URL, we'll likely fail due to unknown protocol.
02221d6fb041 url: add HTTP handler that uses a proxied socket Gregory Szorc <gregory.szorc@gmail.com> parents: 36681 diff changeset	541
02221d6fb041 url: add HTTP handler that uses a proxied socket Gregory Szorc <gregory.szorc@gmail.com> parents: 36681 diff changeset	542 else:
40044 6509fcec830c url: allow to configure timeout on http connection C?dric Krier <ced@b2ck.com> parents: 40035 diff changeset	543 handlers.append(httphandler(timeout=timeout))
37014 02221d6fb041 url: add HTTP handler that uses a proxied socket Gregory Szorc <gregory.szorc@gmail.com> parents: 36681 diff changeset	544 if has_https:
40044 6509fcec830c url: allow to configure timeout on http connection C?dric Krier <ced@b2ck.com> parents: 40035 diff changeset	545 handlers.append(httpshandler(ui, timeout=timeout))
7270 2db33c1a5654 factor out the url handling from httprepo Benoit Boissinot <benoit.boissinot@ens-lyon.org> parents: diff changeset	546
2db33c1a5654 factor out the url handling from httprepo Benoit Boissinot <benoit.boissinot@ens-lyon.org> parents: diff changeset	547 handlers.append(proxyhandler(ui))
2db33c1a5654 factor out the url handling from httprepo Benoit Boissinot <benoit.boissinot@ens-lyon.org> parents: diff changeset	548
29378 fea71f66ebff url: remember http password database in ui object liscju <piotr.listkiewicz@gmail.com> parents: 29377 diff changeset	549 passmgr = passwordmgr(ui, ui.httppasswordmgrdb)
7270 2db33c1a5654 factor out the url handling from httprepo Benoit Boissinot <benoit.boissinot@ens-lyon.org> parents: diff changeset	550 if authinfo is not None:
29379 fc777c855d66 largefiles: make cloning not ask two times about password (issue4883) liscju <piotr.listkiewicz@gmail.com> parents: 29378 diff changeset	551 realm, uris, user, passwd = authinfo
fc777c855d66 largefiles: make cloning not ask two times about password (issue4883) liscju <piotr.listkiewicz@gmail.com> parents: 29378 diff changeset	552 saveduser, savedpass = passmgr.find_stored_password(uris[0])
fc777c855d66 largefiles: make cloning not ask two times about password (issue4883) liscju <piotr.listkiewicz@gmail.com> parents: 29378 diff changeset	553 if user != saveduser or passwd:
fc777c855d66 largefiles: make cloning not ask two times about password (issue4883) liscju <piotr.listkiewicz@gmail.com> parents: 29378 diff changeset	554 passmgr.add_password(realm, uris, user, passwd)
9467 4c041f1ee1b4 do not attempt to translate ui.debug output Martin Geisler <mg@lazybytes.net> parents: 9347 diff changeset	555 ui.debug('http auth: user %s, password %s\n' %
7270 2db33c1a5654 factor out the url handling from httprepo Benoit Boissinot <benoit.boissinot@ens-lyon.org> parents: diff changeset	556 (user, passwd and '' len(passwd) or 'not set'))
2db33c1a5654 factor out the url handling from httprepo Benoit Boissinot <benoit.boissinot@ens-lyon.org> parents: diff changeset	557
11844 6c51a5056020 http basic auth: reset redirect counter on new requests (issue2255) Wagner Bruna <wbruna@softwareexpress.com.br> parents: 11457 diff changeset	558 handlers.extend((httpbasicauthhandler(passmgr),
7270 2db33c1a5654 factor out the url handling from httprepo Benoit Boissinot <benoit.boissinot@ens-lyon.org> parents: diff changeset	559 httpdigestauthhandler(passmgr)))
9347 d0474b184347 url: add support for custom handlers in extensions Henrik Stuart <hg@hstuart.dk> parents: 9122 diff changeset	560 handlers.extend([h(ui, passmgr) for h in handlerfuncs])
31936 806f9a883b4f url: support auth.cookiesfile for adding cookies to HTTP requests Gregory Szorc <gregory.szorc@gmail.com> parents: 30820 diff changeset	561 handlers.append(cookiehandler(ui))
28883 032c4c2f802a pycompat: switch to util.urlreq/util.urlerr for py3 compat timeless <timeless@mozdev.org> parents: 28861 diff changeset	562 opener = urlreq.buildopener(*handlers)
7270 2db33c1a5654 factor out the url handling from httprepo Benoit Boissinot <benoit.boissinot@ens-lyon.org> parents: diff changeset	563
40035 393e44324037 httppeer: report http statistics Gregory Szorc <gregory.szorc@gmail.com> parents: 40032 diff changeset	564 # keepalive.py's handlers will populate these attributes if they exist.
393e44324037 httppeer: report http statistics Gregory Szorc <gregory.szorc@gmail.com> parents: 40032 diff changeset	565 opener.requestscount = 0
393e44324037 httppeer: report http statistics Gregory Szorc <gregory.szorc@gmail.com> parents: 40032 diff changeset	566 opener.sentbytescount = 0
393e44324037 httppeer: report http statistics Gregory Szorc <gregory.szorc@gmail.com> parents: 40032 diff changeset	567 opener.receivedbytescount = 0
393e44324037 httppeer: report http statistics Gregory Szorc <gregory.szorc@gmail.com> parents: 40032 diff changeset	568
29589 486de14eb394 url: add distribution and version to user-agent request header (BC) Gregory Szorc <gregory.szorc@gmail.com> parents: 29455 diff changeset	569 # The user agent should should NOT be used by servers for e.g.
486de14eb394 url: add distribution and version to user-agent request header (BC) Gregory Szorc <gregory.szorc@gmail.com> parents: 29455 diff changeset	570 # protocol detection or feature negotiation: there are other
486de14eb394 url: add distribution and version to user-agent request header (BC) Gregory Szorc <gregory.szorc@gmail.com> parents: 29455 diff changeset	571 # facilities for that.
486de14eb394 url: add distribution and version to user-agent request header (BC) Gregory Szorc <gregory.szorc@gmail.com> parents: 29455 diff changeset	572 #
486de14eb394 url: add distribution and version to user-agent request header (BC) Gregory Szorc <gregory.szorc@gmail.com> parents: 29455 diff changeset	573 # "mercurial/proto-1.0" was the original user agent string and
486de14eb394 url: add distribution and version to user-agent request header (BC) Gregory Szorc <gregory.szorc@gmail.com> parents: 29455 diff changeset	574 # exists for backwards compatibility reasons.
486de14eb394 url: add distribution and version to user-agent request header (BC) Gregory Szorc <gregory.szorc@gmail.com> parents: 29455 diff changeset	575 #
486de14eb394 url: add distribution and version to user-agent request header (BC) Gregory Szorc <gregory.szorc@gmail.com> parents: 29455 diff changeset	576 # The "(Mercurial %s)" string contains the distribution
486de14eb394 url: add distribution and version to user-agent request header (BC) Gregory Szorc <gregory.szorc@gmail.com> parents: 29455 diff changeset	577 # name and version. Other client implementations should choose their
486de14eb394 url: add distribution and version to user-agent request header (BC) Gregory Szorc <gregory.szorc@gmail.com> parents: 29455 diff changeset	578 # own distribution name. Since servers should not be using the user
486de14eb394 url: add distribution and version to user-agent request header (BC) Gregory Szorc <gregory.szorc@gmail.com> parents: 29455 diff changeset	579 # agent string for anything, clients should be able to define whatever
486de14eb394 url: add distribution and version to user-agent request header (BC) Gregory Szorc <gregory.szorc@gmail.com> parents: 29455 diff changeset	580 # user agent they deem appropriate.
35443 e7bb5fc4570c lfs: add git to the User-Agent header for blob transfers Matt Harbison <matt_harbison@yahoo.com> parents: 34694 diff changeset	581 #
e7bb5fc4570c lfs: add git to the User-Agent header for blob transfers Matt Harbison <matt_harbison@yahoo.com> parents: 34694 diff changeset	582 # The custom user agent is for lfs, because unfortunately some servers
e7bb5fc4570c lfs: add git to the User-Agent header for blob transfers Matt Harbison <matt_harbison@yahoo.com> parents: 34694 diff changeset	583 # do look at this value.
e7bb5fc4570c lfs: add git to the User-Agent header for blob transfers Matt Harbison <matt_harbison@yahoo.com> parents: 34694 diff changeset	584 if not useragent:
e7bb5fc4570c lfs: add git to the User-Agent header for blob transfers Matt Harbison <matt_harbison@yahoo.com> parents: 34694 diff changeset	585 agent = 'mercurial/proto-1.0 (Mercurial %s)' % util.version()
e7bb5fc4570c lfs: add git to the User-Agent header for blob transfers Matt Harbison <matt_harbison@yahoo.com> parents: 34694 diff changeset	586 opener.addheaders = [(r'User-agent', pycompat.sysstr(agent))]
e7bb5fc4570c lfs: add git to the User-Agent header for blob transfers Matt Harbison <matt_harbison@yahoo.com> parents: 34694 diff changeset	587 else:
e7bb5fc4570c lfs: add git to the User-Agent header for blob transfers Matt Harbison <matt_harbison@yahoo.com> parents: 34694 diff changeset	588 opener.addheaders = [(r'User-agent', pycompat.sysstr(useragent))]
30763 a520aefb96f1 httppeer: advertise and support application/mercurial-0.2 Gregory Szorc <gregory.szorc@gmail.com> parents: 30667 diff changeset	589
a520aefb96f1 httppeer: advertise and support application/mercurial-0.2 Gregory Szorc <gregory.szorc@gmail.com> parents: 30667 diff changeset	590 # This header should only be needed by wire protocol requests. But it has
a520aefb96f1 httppeer: advertise and support application/mercurial-0.2 Gregory Szorc <gregory.szorc@gmail.com> parents: 30667 diff changeset	591 # been sent on all requests since forever. We keep sending it for backwards
a520aefb96f1 httppeer: advertise and support application/mercurial-0.2 Gregory Szorc <gregory.szorc@gmail.com> parents: 30667 diff changeset	592 # compatibility reasons. Modern versions of the wire protocol use
a520aefb96f1 httppeer: advertise and support application/mercurial-0.2 Gregory Szorc <gregory.szorc@gmail.com> parents: 30667 diff changeset	593 # X-HgProto-<N> for advertising client support.
37048 a708e1e4d7a8 url: support suppressing Accept header Gregory Szorc <gregory.szorc@gmail.com> parents: 37014 diff changeset	594 if sendaccept:
a708e1e4d7a8 url: support suppressing Accept header Gregory Szorc <gregory.szorc@gmail.com> parents: 37014 diff changeset	595 opener.addheaders.append((r'Accept', r'application/mercurial-0.1'))
a708e1e4d7a8 url: support suppressing Accept header Gregory Szorc <gregory.szorc@gmail.com> parents: 37014 diff changeset	596
7270 2db33c1a5654 factor out the url handling from httprepo Benoit Boissinot <benoit.boissinot@ens-lyon.org> parents: diff changeset	597 return opener
2db33c1a5654 factor out the url handling from httprepo Benoit Boissinot <benoit.boissinot@ens-lyon.org> parents: diff changeset	598
13818 bf6156bab41b url: use url.url in url.open() Brodie Rao <brodie@bitheap.org> parents: 13817 diff changeset	599 def open(ui, url_, data=None):
14076 924c82157d46 url: move URL parsing functions into util to improve startup time Brodie Rao <brodie@bitheap.org> parents: 14071 diff changeset	600 u = util.url(url_)
13818 bf6156bab41b url: use url.url in url.open() Brodie Rao <brodie@bitheap.org> parents: 13817 diff changeset	601 if u.scheme:
bf6156bab41b url: use url.url in url.open() Brodie Rao <brodie@bitheap.org> parents: 13817 diff changeset	602 u.scheme = u.scheme.lower()
bf6156bab41b url: use url.url in url.open() Brodie Rao <brodie@bitheap.org> parents: 13817 diff changeset	603 url_, authinfo = u.authinfo()
bf6156bab41b url: use url.url in url.open() Brodie Rao <brodie@bitheap.org> parents: 13817 diff changeset	604 else:
bf6156bab41b url: use url.url in url.open() Brodie Rao <brodie@bitheap.org> parents: 13817 diff changeset	605 path = util.normpath(os.path.abspath(url_))
37735 126998dcfb08 url: some bytes/str cleanup where we interface with stdlib funcs Augie Fackler <augie@google.com> parents: 37103 diff changeset	606 url_ = 'file://' + pycompat.bytesurl(urlreq.pathname2url(path))
7284 ac81ffac0f35 url: fix file:// URL handling Patrick Mezard <pmezard@gmail.com> parents: 7270 diff changeset	607 authinfo = None
36484 698fe0f6eb5c py3: use pycompat.strurl to convert url to str Pulkit Goyal <7895pulkit@gmail.com> parents: 36456 diff changeset	608 return opener(ui, authinfo).open(pycompat.strurl(url_), data)
40019 f80db6adabbe url: move _wraphttpresponse() from httpeer Gregory Szorc <gregory.szorc@gmail.com> parents: 37735 diff changeset	609
f80db6adabbe url: move _wraphttpresponse() from httpeer Gregory Szorc <gregory.szorc@gmail.com> parents: 37735 diff changeset	610 def wrapresponse(resp):
f80db6adabbe url: move _wraphttpresponse() from httpeer Gregory Szorc <gregory.szorc@gmail.com> parents: 37735 diff changeset	611 """Wrap a response object with common error handlers.
f80db6adabbe url: move _wraphttpresponse() from httpeer Gregory Szorc <gregory.szorc@gmail.com> parents: 37735 diff changeset	612
f80db6adabbe url: move _wraphttpresponse() from httpeer Gregory Szorc <gregory.szorc@gmail.com> parents: 37735 diff changeset	613 This ensures that any I/O from any consumer raises the appropriate
f80db6adabbe url: move _wraphttpresponse() from httpeer Gregory Szorc <gregory.szorc@gmail.com> parents: 37735 diff changeset	614 error and messaging.
f80db6adabbe url: move _wraphttpresponse() from httpeer Gregory Szorc <gregory.szorc@gmail.com> parents: 37735 diff changeset	615 """
f80db6adabbe url: move _wraphttpresponse() from httpeer Gregory Szorc <gregory.szorc@gmail.com> parents: 37735 diff changeset	616 origread = resp.read
f80db6adabbe url: move _wraphttpresponse() from httpeer Gregory Szorc <gregory.szorc@gmail.com> parents: 37735 diff changeset	617
f80db6adabbe url: move _wraphttpresponse() from httpeer Gregory Szorc <gregory.szorc@gmail.com> parents: 37735 diff changeset	618 class readerproxy(resp.__class__):
f80db6adabbe url: move _wraphttpresponse() from httpeer Gregory Szorc <gregory.szorc@gmail.com> parents: 37735 diff changeset	619 def read(self, size=None):
f80db6adabbe url: move _wraphttpresponse() from httpeer Gregory Szorc <gregory.szorc@gmail.com> parents: 37735 diff changeset	620 try:
f80db6adabbe url: move _wraphttpresponse() from httpeer Gregory Szorc <gregory.szorc@gmail.com> parents: 37735 diff changeset	621 return origread(size)
f80db6adabbe url: move _wraphttpresponse() from httpeer Gregory Szorc <gregory.szorc@gmail.com> parents: 37735 diff changeset	622 except httplib.IncompleteRead as e:
f80db6adabbe url: move _wraphttpresponse() from httpeer Gregory Szorc <gregory.szorc@gmail.com> parents: 37735 diff changeset	623 # e.expected is an integer if length known or None otherwise.
f80db6adabbe url: move _wraphttpresponse() from httpeer Gregory Szorc <gregory.szorc@gmail.com> parents: 37735 diff changeset	624 if e.expected:
f80db6adabbe url: move _wraphttpresponse() from httpeer Gregory Szorc <gregory.szorc@gmail.com> parents: 37735 diff changeset	625 got = len(e.partial)
f80db6adabbe url: move _wraphttpresponse() from httpeer Gregory Szorc <gregory.szorc@gmail.com> parents: 37735 diff changeset	626 total = e.expected + got
f80db6adabbe url: move _wraphttpresponse() from httpeer Gregory Szorc <gregory.szorc@gmail.com> parents: 37735 diff changeset	627 msg = _('HTTP request error (incomplete response; '
f80db6adabbe url: move _wraphttpresponse() from httpeer Gregory Szorc <gregory.szorc@gmail.com> parents: 37735 diff changeset	628 'expected %d bytes got %d)') % (total, got)
f80db6adabbe url: move _wraphttpresponse() from httpeer Gregory Szorc <gregory.szorc@gmail.com> parents: 37735 diff changeset	629 else:
f80db6adabbe url: move _wraphttpresponse() from httpeer Gregory Szorc <gregory.szorc@gmail.com> parents: 37735 diff changeset	630 msg = _('HTTP request error (incomplete response)')
f80db6adabbe url: move _wraphttpresponse() from httpeer Gregory Szorc <gregory.szorc@gmail.com> parents: 37735 diff changeset	631
f80db6adabbe url: move _wraphttpresponse() from httpeer Gregory Szorc <gregory.szorc@gmail.com> parents: 37735 diff changeset	632 raise error.PeerTransportError(
f80db6adabbe url: move _wraphttpresponse() from httpeer Gregory Szorc <gregory.szorc@gmail.com> parents: 37735 diff changeset	633 msg,
f80db6adabbe url: move _wraphttpresponse() from httpeer Gregory Szorc <gregory.szorc@gmail.com> parents: 37735 diff changeset	634 hint=_('this may be an intermittent network failure; '
f80db6adabbe url: move _wraphttpresponse() from httpeer Gregory Szorc <gregory.szorc@gmail.com> parents: 37735 diff changeset	635 'if the error persists, consider contacting the '
f80db6adabbe url: move _wraphttpresponse() from httpeer Gregory Szorc <gregory.szorc@gmail.com> parents: 37735 diff changeset	636 'network or server operator'))
f80db6adabbe url: move _wraphttpresponse() from httpeer Gregory Szorc <gregory.szorc@gmail.com> parents: 37735 diff changeset	637 except httplib.HTTPException as e:
f80db6adabbe url: move _wraphttpresponse() from httpeer Gregory Szorc <gregory.szorc@gmail.com> parents: 37735 diff changeset	638 raise error.PeerTransportError(
f80db6adabbe url: move _wraphttpresponse() from httpeer Gregory Szorc <gregory.szorc@gmail.com> parents: 37735 diff changeset	639 _('HTTP request error (%s)') % e,
f80db6adabbe url: move _wraphttpresponse() from httpeer Gregory Szorc <gregory.szorc@gmail.com> parents: 37735 diff changeset	640 hint=_('this may be an intermittent network failure; '
f80db6adabbe url: move _wraphttpresponse() from httpeer Gregory Szorc <gregory.szorc@gmail.com> parents: 37735 diff changeset	641 'if the error persists, consider contacting the '
f80db6adabbe url: move _wraphttpresponse() from httpeer Gregory Szorc <gregory.szorc@gmail.com> parents: 37735 diff changeset	642 'network or server operator'))
f80db6adabbe url: move _wraphttpresponse() from httpeer Gregory Szorc <gregory.szorc@gmail.com> parents: 37735 diff changeset	643
f80db6adabbe url: move _wraphttpresponse() from httpeer Gregory Szorc <gregory.szorc@gmail.com> parents: 37735 diff changeset	644 resp.__class__ = readerproxy

Mercurial > public > mercurial-scm > hg-stable

annotate mercurial/url.py @ 41721:4028897dfa05