mercurial-scm/hg: mercurial/keepalive.py annotate

annotate mercurial/keepalive.py @ 30473:39d13b8c101d

py3: bulk replace sys.stdin/out/err by util's Almost all sys.stdin/out/err in hgext/ and mercurial/ are replaced by util's. There are a few exceptions: - lsprof.py and statprof.py are untouched since they are a kind of vendor code and they never import mercurial modules right now. - ui._readline() needs to replace sys.stdin and stdout to pass them to raw_input(). We'll need another workaround here.

author	Yuya Nishihara <yuya@tcha.org>
date	Thu, 20 Oct 2016 23:53:36 +0900
parents	bc0def54c17d
children	88a448a12ae8

rev	line source
2435 ff2bac730b99 http client: support persistent connections. Vadim Gelfer <vadim.gelfer@gmail.com> parents: diff changeset	1 # This library is free software; you can redistribute it and/or
ff2bac730b99 http client: support persistent connections. Vadim Gelfer <vadim.gelfer@gmail.com> parents: diff changeset	2 # modify it under the terms of the GNU Lesser General Public
ff2bac730b99 http client: support persistent connections. Vadim Gelfer <vadim.gelfer@gmail.com> parents: diff changeset	3 # License as published by the Free Software Foundation; either
ff2bac730b99 http client: support persistent connections. Vadim Gelfer <vadim.gelfer@gmail.com> parents: diff changeset	4 # version 2.1 of the License, or (at your option) any later version.
ff2bac730b99 http client: support persistent connections. Vadim Gelfer <vadim.gelfer@gmail.com> parents: diff changeset	5 #
ff2bac730b99 http client: support persistent connections. Vadim Gelfer <vadim.gelfer@gmail.com> parents: diff changeset	6 # This library is distributed in the hope that it will be useful,
ff2bac730b99 http client: support persistent connections. Vadim Gelfer <vadim.gelfer@gmail.com> parents: diff changeset	7 # but WITHOUT ANY WARRANTY; without even the implied warranty of
ff2bac730b99 http client: support persistent connections. Vadim Gelfer <vadim.gelfer@gmail.com> parents: diff changeset	8 # MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the GNU
ff2bac730b99 http client: support persistent connections. Vadim Gelfer <vadim.gelfer@gmail.com> parents: diff changeset	9 # Lesser General Public License for more details.
ff2bac730b99 http client: support persistent connections. Vadim Gelfer <vadim.gelfer@gmail.com> parents: diff changeset	10 #
ff2bac730b99 http client: support persistent connections. Vadim Gelfer <vadim.gelfer@gmail.com> parents: diff changeset	11 # You should have received a copy of the GNU Lesser General Public
15782 7de7630053cb Remove FSF mailing address from GPL headers Martin Geisler <mg@aragost.com> parents: 14958 diff changeset	12 # License along with this library; if not, see
7de7630053cb Remove FSF mailing address from GPL headers Martin Geisler <mg@aragost.com> parents: 14958 diff changeset	13 # <http://www.gnu.org/licenses/>.
2435 ff2bac730b99 http client: support persistent connections. Vadim Gelfer <vadim.gelfer@gmail.com> parents: diff changeset	14
ff2bac730b99 http client: support persistent connections. Vadim Gelfer <vadim.gelfer@gmail.com> parents: diff changeset	15 # This file is part of urlgrabber, a high-level cross-protocol url-grabber
ff2bac730b99 http client: support persistent connections. Vadim Gelfer <vadim.gelfer@gmail.com> parents: diff changeset	16 # Copyright 2002-2004 Michael D. Stenner, Ryan Tomayko
ff2bac730b99 http client: support persistent connections. Vadim Gelfer <vadim.gelfer@gmail.com> parents: diff changeset	17
4026 8520a773a141 fix for digest auth when using keepalive.py Benoit Boissinot <benoit.boissinot@ens-lyon.org> parents: 2600 diff changeset	18 # Modified by Benoit Boissinot:
8520a773a141 fix for digest auth when using keepalive.py Benoit Boissinot <benoit.boissinot@ens-lyon.org> parents: 2600 diff changeset	19 # - fix for digest auth (inspired from urllib2.py @ Python v2.4)
6470 ac0bcd951c2c python 2.6 compatibility: compatibility wrappers for hash functions Dirkjan Ochtman <dirkjan@ochtman.nl> parents: 6001 diff changeset	20 # Modified by Dirkjan Ochtman:
ac0bcd951c2c python 2.6 compatibility: compatibility wrappers for hash functions Dirkjan Ochtman <dirkjan@ochtman.nl> parents: 6001 diff changeset	21 # - import md5 function from a local util module
9726 430e59ff3437 keepalive: handle broken pipes gracefully during large POSTs Augie Fackler <durin42@gmail.com> parents: 8296 diff changeset	22 # Modified by Augie Fackler:
430e59ff3437 keepalive: handle broken pipes gracefully during large POSTs Augie Fackler <durin42@gmail.com> parents: 8296 diff changeset	23 # - add safesend method and use it to prevent broken pipe errors
430e59ff3437 keepalive: handle broken pipes gracefully during large POSTs Augie Fackler <durin42@gmail.com> parents: 8296 diff changeset	24 # on large POST requests
4026 8520a773a141 fix for digest auth when using keepalive.py Benoit Boissinot <benoit.boissinot@ens-lyon.org> parents: 2600 diff changeset	25
2435 ff2bac730b99 http client: support persistent connections. Vadim Gelfer <vadim.gelfer@gmail.com> parents: diff changeset	26 """An HTTP handler for urllib2 that supports HTTP 1.1 and keepalive.
ff2bac730b99 http client: support persistent connections. Vadim Gelfer <vadim.gelfer@gmail.com> parents: diff changeset	27
ff2bac730b99 http client: support persistent connections. Vadim Gelfer <vadim.gelfer@gmail.com> parents: diff changeset	28 >>> import urllib2
ff2bac730b99 http client: support persistent connections. Vadim Gelfer <vadim.gelfer@gmail.com> parents: diff changeset	29 >>> from keepalive import HTTPHandler
ff2bac730b99 http client: support persistent connections. Vadim Gelfer <vadim.gelfer@gmail.com> parents: diff changeset	30 >>> keepalive_handler = HTTPHandler()
28883 032c4c2f802a pycompat: switch to util.urlreq/util.urlerr for py3 compat timeless <timeless@mozdev.org> parents: 28278 diff changeset	31 >>> opener = urlreq.buildopener(keepalive_handler)
032c4c2f802a pycompat: switch to util.urlreq/util.urlerr for py3 compat timeless <timeless@mozdev.org> parents: 28278 diff changeset	32 >>> urlreq.installopener(opener)
2600 c4325f0a9b91 clean up trailing white space. Vadim Gelfer <vadim.gelfer@gmail.com> parents: 2444 diff changeset	33 >>>
28883 032c4c2f802a pycompat: switch to util.urlreq/util.urlerr for py3 compat timeless <timeless@mozdev.org> parents: 28278 diff changeset	34 >>> fo = urlreq.urlopen('http://www.python.org')
2435 ff2bac730b99 http client: support persistent connections. Vadim Gelfer <vadim.gelfer@gmail.com> parents: diff changeset	35
ff2bac730b99 http client: support persistent connections. Vadim Gelfer <vadim.gelfer@gmail.com> parents: diff changeset	36 If a connection to a given host is requested, and all of the existing
ff2bac730b99 http client: support persistent connections. Vadim Gelfer <vadim.gelfer@gmail.com> parents: diff changeset	37 connections are still in use, another connection will be opened. If
ff2bac730b99 http client: support persistent connections. Vadim Gelfer <vadim.gelfer@gmail.com> parents: diff changeset	38 the handler tries to use an existing connection but it fails in some
ff2bac730b99 http client: support persistent connections. Vadim Gelfer <vadim.gelfer@gmail.com> parents: diff changeset	39 way, it will be closed and removed from the pool.
ff2bac730b99 http client: support persistent connections. Vadim Gelfer <vadim.gelfer@gmail.com> parents: diff changeset	40
ff2bac730b99 http client: support persistent connections. Vadim Gelfer <vadim.gelfer@gmail.com> parents: diff changeset	41 To remove the handler, simply re-run build_opener with no arguments, and
ff2bac730b99 http client: support persistent connections. Vadim Gelfer <vadim.gelfer@gmail.com> parents: diff changeset	42 install that opener.
ff2bac730b99 http client: support persistent connections. Vadim Gelfer <vadim.gelfer@gmail.com> parents: diff changeset	43
ff2bac730b99 http client: support persistent connections. Vadim Gelfer <vadim.gelfer@gmail.com> parents: diff changeset	44 You can explicitly close connections by using the close_connection()
ff2bac730b99 http client: support persistent connections. Vadim Gelfer <vadim.gelfer@gmail.com> parents: diff changeset	45 method of the returned file-like object (described below) or you can
ff2bac730b99 http client: support persistent connections. Vadim Gelfer <vadim.gelfer@gmail.com> parents: diff changeset	46 use the handler methods:
ff2bac730b99 http client: support persistent connections. Vadim Gelfer <vadim.gelfer@gmail.com> parents: diff changeset	47
ff2bac730b99 http client: support persistent connections. Vadim Gelfer <vadim.gelfer@gmail.com> parents: diff changeset	48 close_connection(host)
ff2bac730b99 http client: support persistent connections. Vadim Gelfer <vadim.gelfer@gmail.com> parents: diff changeset	49 close_all()
ff2bac730b99 http client: support persistent connections. Vadim Gelfer <vadim.gelfer@gmail.com> parents: diff changeset	50 open_connections()
ff2bac730b99 http client: support persistent connections. Vadim Gelfer <vadim.gelfer@gmail.com> parents: diff changeset	51
ff2bac730b99 http client: support persistent connections. Vadim Gelfer <vadim.gelfer@gmail.com> parents: diff changeset	52 NOTE: using the close_connection and close_all methods of the handler
ff2bac730b99 http client: support persistent connections. Vadim Gelfer <vadim.gelfer@gmail.com> parents: diff changeset	53 should be done with care when using multiple threads.
ff2bac730b99 http client: support persistent connections. Vadim Gelfer <vadim.gelfer@gmail.com> parents: diff changeset	54 * there is nothing that prevents another thread from creating new
ff2bac730b99 http client: support persistent connections. Vadim Gelfer <vadim.gelfer@gmail.com> parents: diff changeset	55 connections immediately after connections are closed
ff2bac730b99 http client: support persistent connections. Vadim Gelfer <vadim.gelfer@gmail.com> parents: diff changeset	56 * no checks are done to prevent in-use connections from being closed
ff2bac730b99 http client: support persistent connections. Vadim Gelfer <vadim.gelfer@gmail.com> parents: diff changeset	57
ff2bac730b99 http client: support persistent connections. Vadim Gelfer <vadim.gelfer@gmail.com> parents: diff changeset	58 >>> keepalive_handler.close_all()
ff2bac730b99 http client: support persistent connections. Vadim Gelfer <vadim.gelfer@gmail.com> parents: diff changeset	59
ff2bac730b99 http client: support persistent connections. Vadim Gelfer <vadim.gelfer@gmail.com> parents: diff changeset	60 EXTRA ATTRIBUTES AND METHODS
ff2bac730b99 http client: support persistent connections. Vadim Gelfer <vadim.gelfer@gmail.com> parents: diff changeset	61
ff2bac730b99 http client: support persistent connections. Vadim Gelfer <vadim.gelfer@gmail.com> parents: diff changeset	62 Upon a status of 200, the object returned has a few additional
ff2bac730b99 http client: support persistent connections. Vadim Gelfer <vadim.gelfer@gmail.com> parents: diff changeset	63 attributes and methods, which should not be used if you want to
ff2bac730b99 http client: support persistent connections. Vadim Gelfer <vadim.gelfer@gmail.com> parents: diff changeset	64 remain consistent with the normal urllib2-returned objects:
ff2bac730b99 http client: support persistent connections. Vadim Gelfer <vadim.gelfer@gmail.com> parents: diff changeset	65
ff2bac730b99 http client: support persistent connections. Vadim Gelfer <vadim.gelfer@gmail.com> parents: diff changeset	66 close_connection() - close the connection to the host
ff2bac730b99 http client: support persistent connections. Vadim Gelfer <vadim.gelfer@gmail.com> parents: diff changeset	67 readlines() - you know, readlines()
17424 e7cfe3587ea4 fix trivial spelling errors Mads Kiilerich <mads@kiilerich.com> parents: 17251 diff changeset	68 status - the return status (i.e. 404)
e7cfe3587ea4 fix trivial spelling errors Mads Kiilerich <mads@kiilerich.com> parents: 17251 diff changeset	69 reason - english translation of status (i.e. 'File not found')
2435 ff2bac730b99 http client: support persistent connections. Vadim Gelfer <vadim.gelfer@gmail.com> parents: diff changeset	70
ff2bac730b99 http client: support persistent connections. Vadim Gelfer <vadim.gelfer@gmail.com> parents: diff changeset	71 If you want the best of both worlds, use this inside an
ff2bac730b99 http client: support persistent connections. Vadim Gelfer <vadim.gelfer@gmail.com> parents: diff changeset	72 AttributeError-catching try:
ff2bac730b99 http client: support persistent connections. Vadim Gelfer <vadim.gelfer@gmail.com> parents: diff changeset	73
ff2bac730b99 http client: support persistent connections. Vadim Gelfer <vadim.gelfer@gmail.com> parents: diff changeset	74 >>> try: status = fo.status
ff2bac730b99 http client: support persistent connections. Vadim Gelfer <vadim.gelfer@gmail.com> parents: diff changeset	75 >>> except AttributeError: status = None
ff2bac730b99 http client: support persistent connections. Vadim Gelfer <vadim.gelfer@gmail.com> parents: diff changeset	76
ff2bac730b99 http client: support persistent connections. Vadim Gelfer <vadim.gelfer@gmail.com> parents: diff changeset	77 Unfortunately, these are ONLY there if status == 200, so it's not
ff2bac730b99 http client: support persistent connections. Vadim Gelfer <vadim.gelfer@gmail.com> parents: diff changeset	78 easy to distinguish between non-200 responses. The reason is that
ff2bac730b99 http client: support persistent connections. Vadim Gelfer <vadim.gelfer@gmail.com> parents: diff changeset	79 urllib2 tries to do clever things with error codes 301, 302, 401,
ff2bac730b99 http client: support persistent connections. Vadim Gelfer <vadim.gelfer@gmail.com> parents: diff changeset	80 and 407, and it wraps the object upon return.
ff2bac730b99 http client: support persistent connections. Vadim Gelfer <vadim.gelfer@gmail.com> parents: diff changeset	81
ff2bac730b99 http client: support persistent connections. Vadim Gelfer <vadim.gelfer@gmail.com> parents: diff changeset	82 For python versions earlier than 2.4, you can avoid this fancy error
ff2bac730b99 http client: support persistent connections. Vadim Gelfer <vadim.gelfer@gmail.com> parents: diff changeset	83 handling by setting the module-level global HANDLE_ERRORS to zero.
ff2bac730b99 http client: support persistent connections. Vadim Gelfer <vadim.gelfer@gmail.com> parents: diff changeset	84 You see, prior to 2.4, it's the HTTP Handler's job to determine what
ff2bac730b99 http client: support persistent connections. Vadim Gelfer <vadim.gelfer@gmail.com> parents: diff changeset	85 to handle specially, and what to just pass up. HANDLE_ERRORS == 0
ff2bac730b99 http client: support persistent connections. Vadim Gelfer <vadim.gelfer@gmail.com> parents: diff changeset	86 means "pass everything up". In python 2.4, however, this job no
ff2bac730b99 http client: support persistent connections. Vadim Gelfer <vadim.gelfer@gmail.com> parents: diff changeset	87 longer belongs to the HTTP Handler and is now done by a NEW handler,
ff2bac730b99 http client: support persistent connections. Vadim Gelfer <vadim.gelfer@gmail.com> parents: diff changeset	88 HTTPErrorProcessor. Here's the bottom line:
ff2bac730b99 http client: support persistent connections. Vadim Gelfer <vadim.gelfer@gmail.com> parents: diff changeset	89
ff2bac730b99 http client: support persistent connections. Vadim Gelfer <vadim.gelfer@gmail.com> parents: diff changeset	90 python version < 2.4
ff2bac730b99 http client: support persistent connections. Vadim Gelfer <vadim.gelfer@gmail.com> parents: diff changeset	91 HANDLE_ERRORS == 1 (default) pass up 200, treat the rest as
ff2bac730b99 http client: support persistent connections. Vadim Gelfer <vadim.gelfer@gmail.com> parents: diff changeset	92 errors
ff2bac730b99 http client: support persistent connections. Vadim Gelfer <vadim.gelfer@gmail.com> parents: diff changeset	93 HANDLE_ERRORS == 0 pass everything up, error processing is
ff2bac730b99 http client: support persistent connections. Vadim Gelfer <vadim.gelfer@gmail.com> parents: diff changeset	94 left to the calling code
ff2bac730b99 http client: support persistent connections. Vadim Gelfer <vadim.gelfer@gmail.com> parents: diff changeset	95 python version >= 2.4
ff2bac730b99 http client: support persistent connections. Vadim Gelfer <vadim.gelfer@gmail.com> parents: diff changeset	96 HANDLE_ERRORS == 1 pass up 200, treat the rest as errors
ff2bac730b99 http client: support persistent connections. Vadim Gelfer <vadim.gelfer@gmail.com> parents: diff changeset	97 HANDLE_ERRORS == 0 (default) pass everything up, let the
ff2bac730b99 http client: support persistent connections. Vadim Gelfer <vadim.gelfer@gmail.com> parents: diff changeset	98 other handlers (specifically,
ff2bac730b99 http client: support persistent connections. Vadim Gelfer <vadim.gelfer@gmail.com> parents: diff changeset	99 HTTPErrorProcessor) decide what to do
ff2bac730b99 http client: support persistent connections. Vadim Gelfer <vadim.gelfer@gmail.com> parents: diff changeset	100
ff2bac730b99 http client: support persistent connections. Vadim Gelfer <vadim.gelfer@gmail.com> parents: diff changeset	101 In practice, setting the variable either way makes little difference
ff2bac730b99 http client: support persistent connections. Vadim Gelfer <vadim.gelfer@gmail.com> parents: diff changeset	102 in python 2.4, so for the most consistent behavior across versions,
ff2bac730b99 http client: support persistent connections. Vadim Gelfer <vadim.gelfer@gmail.com> parents: diff changeset	103 you probably just want to use the defaults, which will give you
ff2bac730b99 http client: support persistent connections. Vadim Gelfer <vadim.gelfer@gmail.com> parents: diff changeset	104 exceptions on errors.
ff2bac730b99 http client: support persistent connections. Vadim Gelfer <vadim.gelfer@gmail.com> parents: diff changeset	105
ff2bac730b99 http client: support persistent connections. Vadim Gelfer <vadim.gelfer@gmail.com> parents: diff changeset	106 """
ff2bac730b99 http client: support persistent connections. Vadim Gelfer <vadim.gelfer@gmail.com> parents: diff changeset	107
2444 5eb02f9ed804 Update keepalive.py to current CVS version of urlgrabber. Alexis S. L. Carvalho <alexis@cecm.usp.br> parents: 2435 diff changeset	108 # $Id: keepalive.py,v 1.14 2006/04/04 21:00:32 mstenner Exp $
2435 ff2bac730b99 http client: support persistent connections. Vadim Gelfer <vadim.gelfer@gmail.com> parents: diff changeset	109
27616 0765d8423fbc keepalive: use print function Gregory Szorc <gregory.szorc@gmail.com> parents: 27507 diff changeset	110 from __future__ import absolute_import, print_function
27507 a16489f9132d keepalive: use absolute_import Gregory Szorc <gregory.szorc@gmail.com> parents: 25660 diff changeset	111
9726 430e59ff3437 keepalive: handle broken pipes gracefully during large POSTs Augie Fackler <durin42@gmail.com> parents: 8296 diff changeset	112 import errno
29341 0d83ad967bf8 cleanup: replace uses of util.(md5\|sha1\|sha256\|sha512) with hashlib.\1 Augie Fackler <raf@durin42.com> parents: 28883 diff changeset	113 import hashlib
2435 ff2bac730b99 http client: support persistent connections. Vadim Gelfer <vadim.gelfer@gmail.com> parents: diff changeset	114 import socket
27507 a16489f9132d keepalive: use absolute_import Gregory Szorc <gregory.szorc@gmail.com> parents: 25660 diff changeset	115 import sys
29456 e61d384e3be4 keepalive: switch from thread to threading module Pulkit Goyal <7895pulkit@gmail.com> parents: 29455 diff changeset	116 import threading
28883 032c4c2f802a pycompat: switch to util.urlreq/util.urlerr for py3 compat timeless <timeless@mozdev.org> parents: 28278 diff changeset	117
032c4c2f802a pycompat: switch to util.urlreq/util.urlerr for py3 compat timeless <timeless@mozdev.org> parents: 28278 diff changeset	118 from . import (
032c4c2f802a pycompat: switch to util.urlreq/util.urlerr for py3 compat timeless <timeless@mozdev.org> parents: 28278 diff changeset	119 util,
032c4c2f802a pycompat: switch to util.urlreq/util.urlerr for py3 compat timeless <timeless@mozdev.org> parents: 28278 diff changeset	120 )
032c4c2f802a pycompat: switch to util.urlreq/util.urlerr for py3 compat timeless <timeless@mozdev.org> parents: 28278 diff changeset	121
29455 0c741fd6158a py3: conditionalize httplib import Pulkit Goyal <7895pulkit@gmail.com> parents: 29341 diff changeset	122 httplib = util.httplib
28883 032c4c2f802a pycompat: switch to util.urlreq/util.urlerr for py3 compat timeless <timeless@mozdev.org> parents: 28278 diff changeset	123 urlerr = util.urlerr
032c4c2f802a pycompat: switch to util.urlreq/util.urlerr for py3 compat timeless <timeless@mozdev.org> parents: 28278 diff changeset	124 urlreq = util.urlreq
2435 ff2bac730b99 http client: support persistent connections. Vadim Gelfer <vadim.gelfer@gmail.com> parents: diff changeset	125
ff2bac730b99 http client: support persistent connections. Vadim Gelfer <vadim.gelfer@gmail.com> parents: diff changeset	126 DEBUG = None
ff2bac730b99 http client: support persistent connections. Vadim Gelfer <vadim.gelfer@gmail.com> parents: diff changeset	127
10282 08a0f04b56bd many, many trivial check-code fixups Matt Mackall <mpm@selenic.com> parents: 9726 diff changeset	128 if sys.version_info < (2, 4):
08a0f04b56bd many, many trivial check-code fixups Matt Mackall <mpm@selenic.com> parents: 9726 diff changeset	129 HANDLE_ERRORS = 1
2435 ff2bac730b99 http client: support persistent connections. Vadim Gelfer <vadim.gelfer@gmail.com> parents: diff changeset	130 else: HANDLE_ERRORS = 0
2600 c4325f0a9b91 clean up trailing white space. Vadim Gelfer <vadim.gelfer@gmail.com> parents: 2444 diff changeset	131
14764 a7d5816087a9 classes: fix class style problems found by b071cd58af50 Thomas Arendsen Hein <thomas@intevation.de> parents: 14494 diff changeset	132 class ConnectionManager(object):
2435 ff2bac730b99 http client: support persistent connections. Vadim Gelfer <vadim.gelfer@gmail.com> parents: diff changeset	133 """
ff2bac730b99 http client: support persistent connections. Vadim Gelfer <vadim.gelfer@gmail.com> parents: diff changeset	134 The connection manager must be able to:
ff2bac730b99 http client: support persistent connections. Vadim Gelfer <vadim.gelfer@gmail.com> parents: diff changeset	135 * keep track of all existing
ff2bac730b99 http client: support persistent connections. Vadim Gelfer <vadim.gelfer@gmail.com> parents: diff changeset	136 """
ff2bac730b99 http client: support persistent connections. Vadim Gelfer <vadim.gelfer@gmail.com> parents: diff changeset	137 def __init__(self):
29456 e61d384e3be4 keepalive: switch from thread to threading module Pulkit Goyal <7895pulkit@gmail.com> parents: 29455 diff changeset	138 self._lock = threading.Lock()
2435 ff2bac730b99 http client: support persistent connections. Vadim Gelfer <vadim.gelfer@gmail.com> parents: diff changeset	139 self._hostmap = {} # map hosts to a list of connections
ff2bac730b99 http client: support persistent connections. Vadim Gelfer <vadim.gelfer@gmail.com> parents: diff changeset	140 self._connmap = {} # map connections to host
ff2bac730b99 http client: support persistent connections. Vadim Gelfer <vadim.gelfer@gmail.com> parents: diff changeset	141 self._readymap = {} # map connection to ready state
ff2bac730b99 http client: support persistent connections. Vadim Gelfer <vadim.gelfer@gmail.com> parents: diff changeset	142
ff2bac730b99 http client: support persistent connections. Vadim Gelfer <vadim.gelfer@gmail.com> parents: diff changeset	143 def add(self, host, connection, ready):
ff2bac730b99 http client: support persistent connections. Vadim Gelfer <vadim.gelfer@gmail.com> parents: diff changeset	144 self._lock.acquire()
ff2bac730b99 http client: support persistent connections. Vadim Gelfer <vadim.gelfer@gmail.com> parents: diff changeset	145 try:
16686 67964cda8701 cleanup: "not x in y" -> "x not in y" Brodie Rao <brodie@sf.io> parents: 15782 diff changeset	146 if host not in self._hostmap:
10282 08a0f04b56bd many, many trivial check-code fixups Matt Mackall <mpm@selenic.com> parents: 9726 diff changeset	147 self._hostmap[host] = []
2435 ff2bac730b99 http client: support persistent connections. Vadim Gelfer <vadim.gelfer@gmail.com> parents: diff changeset	148 self._hostmap[host].append(connection)
ff2bac730b99 http client: support persistent connections. Vadim Gelfer <vadim.gelfer@gmail.com> parents: diff changeset	149 self._connmap[connection] = host
ff2bac730b99 http client: support persistent connections. Vadim Gelfer <vadim.gelfer@gmail.com> parents: diff changeset	150 self._readymap[connection] = ready
ff2bac730b99 http client: support persistent connections. Vadim Gelfer <vadim.gelfer@gmail.com> parents: diff changeset	151 finally:
ff2bac730b99 http client: support persistent connections. Vadim Gelfer <vadim.gelfer@gmail.com> parents: diff changeset	152 self._lock.release()
ff2bac730b99 http client: support persistent connections. Vadim Gelfer <vadim.gelfer@gmail.com> parents: diff changeset	153
ff2bac730b99 http client: support persistent connections. Vadim Gelfer <vadim.gelfer@gmail.com> parents: diff changeset	154 def remove(self, connection):
ff2bac730b99 http client: support persistent connections. Vadim Gelfer <vadim.gelfer@gmail.com> parents: diff changeset	155 self._lock.acquire()
ff2bac730b99 http client: support persistent connections. Vadim Gelfer <vadim.gelfer@gmail.com> parents: diff changeset	156 try:
ff2bac730b99 http client: support persistent connections. Vadim Gelfer <vadim.gelfer@gmail.com> parents: diff changeset	157 try:
ff2bac730b99 http client: support persistent connections. Vadim Gelfer <vadim.gelfer@gmail.com> parents: diff changeset	158 host = self._connmap[connection]
ff2bac730b99 http client: support persistent connections. Vadim Gelfer <vadim.gelfer@gmail.com> parents: diff changeset	159 except KeyError:
ff2bac730b99 http client: support persistent connections. Vadim Gelfer <vadim.gelfer@gmail.com> parents: diff changeset	160 pass
ff2bac730b99 http client: support persistent connections. Vadim Gelfer <vadim.gelfer@gmail.com> parents: diff changeset	161 else:
ff2bac730b99 http client: support persistent connections. Vadim Gelfer <vadim.gelfer@gmail.com> parents: diff changeset	162 del self._connmap[connection]
ff2bac730b99 http client: support persistent connections. Vadim Gelfer <vadim.gelfer@gmail.com> parents: diff changeset	163 del self._readymap[connection]
ff2bac730b99 http client: support persistent connections. Vadim Gelfer <vadim.gelfer@gmail.com> parents: diff changeset	164 self._hostmap[host].remove(connection)
ff2bac730b99 http client: support persistent connections. Vadim Gelfer <vadim.gelfer@gmail.com> parents: diff changeset	165 if not self._hostmap[host]: del self._hostmap[host]
ff2bac730b99 http client: support persistent connections. Vadim Gelfer <vadim.gelfer@gmail.com> parents: diff changeset	166 finally:
ff2bac730b99 http client: support persistent connections. Vadim Gelfer <vadim.gelfer@gmail.com> parents: diff changeset	167 self._lock.release()
ff2bac730b99 http client: support persistent connections. Vadim Gelfer <vadim.gelfer@gmail.com> parents: diff changeset	168
ff2bac730b99 http client: support persistent connections. Vadim Gelfer <vadim.gelfer@gmail.com> parents: diff changeset	169 def set_ready(self, connection, ready):
10282 08a0f04b56bd many, many trivial check-code fixups Matt Mackall <mpm@selenic.com> parents: 9726 diff changeset	170 try:
08a0f04b56bd many, many trivial check-code fixups Matt Mackall <mpm@selenic.com> parents: 9726 diff changeset	171 self._readymap[connection] = ready
08a0f04b56bd many, many trivial check-code fixups Matt Mackall <mpm@selenic.com> parents: 9726 diff changeset	172 except KeyError:
08a0f04b56bd many, many trivial check-code fixups Matt Mackall <mpm@selenic.com> parents: 9726 diff changeset	173 pass
2600 c4325f0a9b91 clean up trailing white space. Vadim Gelfer <vadim.gelfer@gmail.com> parents: 2444 diff changeset	174
2435 ff2bac730b99 http client: support persistent connections. Vadim Gelfer <vadim.gelfer@gmail.com> parents: diff changeset	175 def get_ready_conn(self, host):
ff2bac730b99 http client: support persistent connections. Vadim Gelfer <vadim.gelfer@gmail.com> parents: diff changeset	176 conn = None
ff2bac730b99 http client: support persistent connections. Vadim Gelfer <vadim.gelfer@gmail.com> parents: diff changeset	177 self._lock.acquire()
ff2bac730b99 http client: support persistent connections. Vadim Gelfer <vadim.gelfer@gmail.com> parents: diff changeset	178 try:
5915 d0576d065993 Prefer i in d over d.has_key(i) Christian Ebert <blacktrash@gmx.net> parents: 4026 diff changeset	179 if host in self._hostmap:
2435 ff2bac730b99 http client: support persistent connections. Vadim Gelfer <vadim.gelfer@gmail.com> parents: diff changeset	180 for c in self._hostmap[host]:
ff2bac730b99 http client: support persistent connections. Vadim Gelfer <vadim.gelfer@gmail.com> parents: diff changeset	181 if self._readymap[c]:
ff2bac730b99 http client: support persistent connections. Vadim Gelfer <vadim.gelfer@gmail.com> parents: diff changeset	182 self._readymap[c] = 0
ff2bac730b99 http client: support persistent connections. Vadim Gelfer <vadim.gelfer@gmail.com> parents: diff changeset	183 conn = c
ff2bac730b99 http client: support persistent connections. Vadim Gelfer <vadim.gelfer@gmail.com> parents: diff changeset	184 break
ff2bac730b99 http client: support persistent connections. Vadim Gelfer <vadim.gelfer@gmail.com> parents: diff changeset	185 finally:
ff2bac730b99 http client: support persistent connections. Vadim Gelfer <vadim.gelfer@gmail.com> parents: diff changeset	186 self._lock.release()
ff2bac730b99 http client: support persistent connections. Vadim Gelfer <vadim.gelfer@gmail.com> parents: diff changeset	187 return conn
ff2bac730b99 http client: support persistent connections. Vadim Gelfer <vadim.gelfer@gmail.com> parents: diff changeset	188
ff2bac730b99 http client: support persistent connections. Vadim Gelfer <vadim.gelfer@gmail.com> parents: diff changeset	189 def get_all(self, host=None):
ff2bac730b99 http client: support persistent connections. Vadim Gelfer <vadim.gelfer@gmail.com> parents: diff changeset	190 if host:
ff2bac730b99 http client: support persistent connections. Vadim Gelfer <vadim.gelfer@gmail.com> parents: diff changeset	191 return list(self._hostmap.get(host, []))
ff2bac730b99 http client: support persistent connections. Vadim Gelfer <vadim.gelfer@gmail.com> parents: diff changeset	192 else:
ff2bac730b99 http client: support persistent connections. Vadim Gelfer <vadim.gelfer@gmail.com> parents: diff changeset	193 return dict(self._hostmap)
ff2bac730b99 http client: support persistent connections. Vadim Gelfer <vadim.gelfer@gmail.com> parents: diff changeset	194
14764 a7d5816087a9 classes: fix class style problems found by b071cd58af50 Thomas Arendsen Hein <thomas@intevation.de> parents: 14494 diff changeset	195 class KeepAliveHandler(object):
2435 ff2bac730b99 http client: support persistent connections. Vadim Gelfer <vadim.gelfer@gmail.com> parents: diff changeset	196 def __init__(self):
ff2bac730b99 http client: support persistent connections. Vadim Gelfer <vadim.gelfer@gmail.com> parents: diff changeset	197 self._cm = ConnectionManager()
2600 c4325f0a9b91 clean up trailing white space. Vadim Gelfer <vadim.gelfer@gmail.com> parents: 2444 diff changeset	198
2435 ff2bac730b99 http client: support persistent connections. Vadim Gelfer <vadim.gelfer@gmail.com> parents: diff changeset	199 #### Connection Management
ff2bac730b99 http client: support persistent connections. Vadim Gelfer <vadim.gelfer@gmail.com> parents: diff changeset	200 def open_connections(self):
ff2bac730b99 http client: support persistent connections. Vadim Gelfer <vadim.gelfer@gmail.com> parents: diff changeset	201 """return a list of connected hosts and the number of connections
ff2bac730b99 http client: support persistent connections. Vadim Gelfer <vadim.gelfer@gmail.com> parents: diff changeset	202 to each. [('foo.com:80', 2), ('bar.org', 1)]"""
ff2bac730b99 http client: support persistent connections. Vadim Gelfer <vadim.gelfer@gmail.com> parents: diff changeset	203 return [(host, len(li)) for (host, li) in self._cm.get_all().items()]
ff2bac730b99 http client: support persistent connections. Vadim Gelfer <vadim.gelfer@gmail.com> parents: diff changeset	204
ff2bac730b99 http client: support persistent connections. Vadim Gelfer <vadim.gelfer@gmail.com> parents: diff changeset	205 def close_connection(self, host):
ff2bac730b99 http client: support persistent connections. Vadim Gelfer <vadim.gelfer@gmail.com> parents: diff changeset	206 """close connection(s) to <host>
ff2bac730b99 http client: support persistent connections. Vadim Gelfer <vadim.gelfer@gmail.com> parents: diff changeset	207 host is the host:port spec, as in 'www.cnn.com:8080' as passed in.
ff2bac730b99 http client: support persistent connections. Vadim Gelfer <vadim.gelfer@gmail.com> parents: diff changeset	208 no error occurs if there is no connection to that host."""
ff2bac730b99 http client: support persistent connections. Vadim Gelfer <vadim.gelfer@gmail.com> parents: diff changeset	209 for h in self._cm.get_all(host):
ff2bac730b99 http client: support persistent connections. Vadim Gelfer <vadim.gelfer@gmail.com> parents: diff changeset	210 self._cm.remove(h)
ff2bac730b99 http client: support persistent connections. Vadim Gelfer <vadim.gelfer@gmail.com> parents: diff changeset	211 h.close()
2600 c4325f0a9b91 clean up trailing white space. Vadim Gelfer <vadim.gelfer@gmail.com> parents: 2444 diff changeset	212
2435 ff2bac730b99 http client: support persistent connections. Vadim Gelfer <vadim.gelfer@gmail.com> parents: diff changeset	213 def close_all(self):
ff2bac730b99 http client: support persistent connections. Vadim Gelfer <vadim.gelfer@gmail.com> parents: diff changeset	214 """close all open connections"""
7622 4dd7b28003d2 use dict.iteritems() rather than dict.items() Dirkjan Ochtman <dirkjan@ochtman.nl> parents: 6470 diff changeset	215 for host, conns in self._cm.get_all().iteritems():
2435 ff2bac730b99 http client: support persistent connections. Vadim Gelfer <vadim.gelfer@gmail.com> parents: diff changeset	216 for h in conns:
ff2bac730b99 http client: support persistent connections. Vadim Gelfer <vadim.gelfer@gmail.com> parents: diff changeset	217 self._cm.remove(h)
ff2bac730b99 http client: support persistent connections. Vadim Gelfer <vadim.gelfer@gmail.com> parents: diff changeset	218 h.close()
2600 c4325f0a9b91 clean up trailing white space. Vadim Gelfer <vadim.gelfer@gmail.com> parents: 2444 diff changeset	219
2435 ff2bac730b99 http client: support persistent connections. Vadim Gelfer <vadim.gelfer@gmail.com> parents: diff changeset	220 def _request_closed(self, request, host, connection):
17251 98166640b356 help: fix some instances of 'the the' Mads Kiilerich <mads@kiilerich.com> parents: 16705 diff changeset	221 """tells us that this request is now closed and that the
2435 ff2bac730b99 http client: support persistent connections. Vadim Gelfer <vadim.gelfer@gmail.com> parents: diff changeset	222 connection is ready for another request"""
ff2bac730b99 http client: support persistent connections. Vadim Gelfer <vadim.gelfer@gmail.com> parents: diff changeset	223 self._cm.set_ready(connection, 1)
ff2bac730b99 http client: support persistent connections. Vadim Gelfer <vadim.gelfer@gmail.com> parents: diff changeset	224
ff2bac730b99 http client: support persistent connections. Vadim Gelfer <vadim.gelfer@gmail.com> parents: diff changeset	225 def _remove_connection(self, host, connection, close=0):
10282 08a0f04b56bd many, many trivial check-code fixups Matt Mackall <mpm@selenic.com> parents: 9726 diff changeset	226 if close:
08a0f04b56bd many, many trivial check-code fixups Matt Mackall <mpm@selenic.com> parents: 9726 diff changeset	227 connection.close()
2435 ff2bac730b99 http client: support persistent connections. Vadim Gelfer <vadim.gelfer@gmail.com> parents: diff changeset	228 self._cm.remove(connection)
2600 c4325f0a9b91 clean up trailing white space. Vadim Gelfer <vadim.gelfer@gmail.com> parents: 2444 diff changeset	229
2435 ff2bac730b99 http client: support persistent connections. Vadim Gelfer <vadim.gelfer@gmail.com> parents: diff changeset	230 #### Transaction Execution
ff2bac730b99 http client: support persistent connections. Vadim Gelfer <vadim.gelfer@gmail.com> parents: diff changeset	231 def http_open(self, req):
ff2bac730b99 http client: support persistent connections. Vadim Gelfer <vadim.gelfer@gmail.com> parents: diff changeset	232 return self.do_open(HTTPConnection, req)
ff2bac730b99 http client: support persistent connections. Vadim Gelfer <vadim.gelfer@gmail.com> parents: diff changeset	233
ff2bac730b99 http client: support persistent connections. Vadim Gelfer <vadim.gelfer@gmail.com> parents: diff changeset	234 def do_open(self, http_class, req):
ff2bac730b99 http client: support persistent connections. Vadim Gelfer <vadim.gelfer@gmail.com> parents: diff changeset	235 host = req.get_host()
ff2bac730b99 http client: support persistent connections. Vadim Gelfer <vadim.gelfer@gmail.com> parents: diff changeset	236 if not host:
28883 032c4c2f802a pycompat: switch to util.urlreq/util.urlerr for py3 compat timeless <timeless@mozdev.org> parents: 28278 diff changeset	237 raise urlerr.urlerror('no host given')
2435 ff2bac730b99 http client: support persistent connections. Vadim Gelfer <vadim.gelfer@gmail.com> parents: diff changeset	238
ff2bac730b99 http client: support persistent connections. Vadim Gelfer <vadim.gelfer@gmail.com> parents: diff changeset	239 try:
ff2bac730b99 http client: support persistent connections. Vadim Gelfer <vadim.gelfer@gmail.com> parents: diff changeset	240 h = self._cm.get_ready_conn(host)
ff2bac730b99 http client: support persistent connections. Vadim Gelfer <vadim.gelfer@gmail.com> parents: diff changeset	241 while h:
ff2bac730b99 http client: support persistent connections. Vadim Gelfer <vadim.gelfer@gmail.com> parents: diff changeset	242 r = self._reuse_connection(h, req, host)
ff2bac730b99 http client: support persistent connections. Vadim Gelfer <vadim.gelfer@gmail.com> parents: diff changeset	243
ff2bac730b99 http client: support persistent connections. Vadim Gelfer <vadim.gelfer@gmail.com> parents: diff changeset	244 # if this response is non-None, then it worked and we're
ff2bac730b99 http client: support persistent connections. Vadim Gelfer <vadim.gelfer@gmail.com> parents: diff changeset	245 # done. Break out, skipping the else block.
10282 08a0f04b56bd many, many trivial check-code fixups Matt Mackall <mpm@selenic.com> parents: 9726 diff changeset	246 if r:
08a0f04b56bd many, many trivial check-code fixups Matt Mackall <mpm@selenic.com> parents: 9726 diff changeset	247 break
2435 ff2bac730b99 http client: support persistent connections. Vadim Gelfer <vadim.gelfer@gmail.com> parents: diff changeset	248
ff2bac730b99 http client: support persistent connections. Vadim Gelfer <vadim.gelfer@gmail.com> parents: diff changeset	249 # connection is bad - possibly closed by server
ff2bac730b99 http client: support persistent connections. Vadim Gelfer <vadim.gelfer@gmail.com> parents: diff changeset	250 # discard it and ask for the next free connection
ff2bac730b99 http client: support persistent connections. Vadim Gelfer <vadim.gelfer@gmail.com> parents: diff changeset	251 h.close()
ff2bac730b99 http client: support persistent connections. Vadim Gelfer <vadim.gelfer@gmail.com> parents: diff changeset	252 self._cm.remove(h)
ff2bac730b99 http client: support persistent connections. Vadim Gelfer <vadim.gelfer@gmail.com> parents: diff changeset	253 h = self._cm.get_ready_conn(host)
ff2bac730b99 http client: support persistent connections. Vadim Gelfer <vadim.gelfer@gmail.com> parents: diff changeset	254 else:
ff2bac730b99 http client: support persistent connections. Vadim Gelfer <vadim.gelfer@gmail.com> parents: diff changeset	255 # no (working) free connections were found. Create a new one.
ff2bac730b99 http client: support persistent connections. Vadim Gelfer <vadim.gelfer@gmail.com> parents: diff changeset	256 h = http_class(host)
10282 08a0f04b56bd many, many trivial check-code fixups Matt Mackall <mpm@selenic.com> parents: 9726 diff changeset	257 if DEBUG:
08a0f04b56bd many, many trivial check-code fixups Matt Mackall <mpm@selenic.com> parents: 9726 diff changeset	258 DEBUG.info("creating new connection to %s (%d)",
08a0f04b56bd many, many trivial check-code fixups Matt Mackall <mpm@selenic.com> parents: 9726 diff changeset	259 host, id(h))
2435 ff2bac730b99 http client: support persistent connections. Vadim Gelfer <vadim.gelfer@gmail.com> parents: diff changeset	260 self._cm.add(host, h, 0)
ff2bac730b99 http client: support persistent connections. Vadim Gelfer <vadim.gelfer@gmail.com> parents: diff changeset	261 self._start_transaction(h, req)
ff2bac730b99 http client: support persistent connections. Vadim Gelfer <vadim.gelfer@gmail.com> parents: diff changeset	262 r = h.getresponse()
25660 328739ea70c3 global: mass rewrite to use modern exception syntax Gregory Szorc <gregory.szorc@gmail.com> parents: 22958 diff changeset	263 except (socket.error, httplib.HTTPException) as err:
28883 032c4c2f802a pycompat: switch to util.urlreq/util.urlerr for py3 compat timeless <timeless@mozdev.org> parents: 28278 diff changeset	264 raise urlerr.urlerror(err)
2600 c4325f0a9b91 clean up trailing white space. Vadim Gelfer <vadim.gelfer@gmail.com> parents: 2444 diff changeset	265
2435 ff2bac730b99 http client: support persistent connections. Vadim Gelfer <vadim.gelfer@gmail.com> parents: diff changeset	266 # if not a persistent connection, don't try to reuse it
10282 08a0f04b56bd many, many trivial check-code fixups Matt Mackall <mpm@selenic.com> parents: 9726 diff changeset	267 if r.will_close:
08a0f04b56bd many, many trivial check-code fixups Matt Mackall <mpm@selenic.com> parents: 9726 diff changeset	268 self._cm.remove(h)
2435 ff2bac730b99 http client: support persistent connections. Vadim Gelfer <vadim.gelfer@gmail.com> parents: diff changeset	269
10282 08a0f04b56bd many, many trivial check-code fixups Matt Mackall <mpm@selenic.com> parents: 9726 diff changeset	270 if DEBUG:
08a0f04b56bd many, many trivial check-code fixups Matt Mackall <mpm@selenic.com> parents: 9726 diff changeset	271 DEBUG.info("STATUS: %s, %s", r.status, r.reason)
2435 ff2bac730b99 http client: support persistent connections. Vadim Gelfer <vadim.gelfer@gmail.com> parents: diff changeset	272 r._handler = self
ff2bac730b99 http client: support persistent connections. Vadim Gelfer <vadim.gelfer@gmail.com> parents: diff changeset	273 r._host = host
ff2bac730b99 http client: support persistent connections. Vadim Gelfer <vadim.gelfer@gmail.com> parents: diff changeset	274 r._url = req.get_full_url()
ff2bac730b99 http client: support persistent connections. Vadim Gelfer <vadim.gelfer@gmail.com> parents: diff changeset	275 r._connection = h
ff2bac730b99 http client: support persistent connections. Vadim Gelfer <vadim.gelfer@gmail.com> parents: diff changeset	276 r.code = r.status
2444 5eb02f9ed804 Update keepalive.py to current CVS version of urlgrabber. Alexis S. L. Carvalho <alexis@cecm.usp.br> parents: 2435 diff changeset	277 r.headers = r.msg
5eb02f9ed804 Update keepalive.py to current CVS version of urlgrabber. Alexis S. L. Carvalho <alexis@cecm.usp.br> parents: 2435 diff changeset	278 r.msg = r.reason
2600 c4325f0a9b91 clean up trailing white space. Vadim Gelfer <vadim.gelfer@gmail.com> parents: 2444 diff changeset	279
2435 ff2bac730b99 http client: support persistent connections. Vadim Gelfer <vadim.gelfer@gmail.com> parents: diff changeset	280 if r.status == 200 or not HANDLE_ERRORS:
ff2bac730b99 http client: support persistent connections. Vadim Gelfer <vadim.gelfer@gmail.com> parents: diff changeset	281 return r
ff2bac730b99 http client: support persistent connections. Vadim Gelfer <vadim.gelfer@gmail.com> parents: diff changeset	282 else:
2444 5eb02f9ed804 Update keepalive.py to current CVS version of urlgrabber. Alexis S. L. Carvalho <alexis@cecm.usp.br> parents: 2435 diff changeset	283 return self.parent.error('http', req, r,
5eb02f9ed804 Update keepalive.py to current CVS version of urlgrabber. Alexis S. L. Carvalho <alexis@cecm.usp.br> parents: 2435 diff changeset	284 r.status, r.msg, r.headers)
2435 ff2bac730b99 http client: support persistent connections. Vadim Gelfer <vadim.gelfer@gmail.com> parents: diff changeset	285
ff2bac730b99 http client: support persistent connections. Vadim Gelfer <vadim.gelfer@gmail.com> parents: diff changeset	286 def _reuse_connection(self, h, req, host):
ff2bac730b99 http client: support persistent connections. Vadim Gelfer <vadim.gelfer@gmail.com> parents: diff changeset	287 """start the transaction with a re-used connection
ff2bac730b99 http client: support persistent connections. Vadim Gelfer <vadim.gelfer@gmail.com> parents: diff changeset	288 return a response object (r) upon success or None on failure.
ff2bac730b99 http client: support persistent connections. Vadim Gelfer <vadim.gelfer@gmail.com> parents: diff changeset	289 This DOES not close or remove bad connections in cases where
ff2bac730b99 http client: support persistent connections. Vadim Gelfer <vadim.gelfer@gmail.com> parents: diff changeset	290 it returns. However, if an unexpected exception occurs, it
ff2bac730b99 http client: support persistent connections. Vadim Gelfer <vadim.gelfer@gmail.com> parents: diff changeset	291 will close and remove the connection before re-raising.
ff2bac730b99 http client: support persistent connections. Vadim Gelfer <vadim.gelfer@gmail.com> parents: diff changeset	292 """
ff2bac730b99 http client: support persistent connections. Vadim Gelfer <vadim.gelfer@gmail.com> parents: diff changeset	293 try:
ff2bac730b99 http client: support persistent connections. Vadim Gelfer <vadim.gelfer@gmail.com> parents: diff changeset	294 self._start_transaction(h, req)
ff2bac730b99 http client: support persistent connections. Vadim Gelfer <vadim.gelfer@gmail.com> parents: diff changeset	295 r = h.getresponse()
ff2bac730b99 http client: support persistent connections. Vadim Gelfer <vadim.gelfer@gmail.com> parents: diff changeset	296 # note: just because we got something back doesn't mean it
ff2bac730b99 http client: support persistent connections. Vadim Gelfer <vadim.gelfer@gmail.com> parents: diff changeset	297 # worked. We'll check the version below, too.
ff2bac730b99 http client: support persistent connections. Vadim Gelfer <vadim.gelfer@gmail.com> parents: diff changeset	298 except (socket.error, httplib.HTTPException):
ff2bac730b99 http client: support persistent connections. Vadim Gelfer <vadim.gelfer@gmail.com> parents: diff changeset	299 r = None
16705 c2d9ef43ff6c check-code: ignore naked excepts with a "re-raise" comment Brodie Rao <brodie@sf.io> parents: 16688 diff changeset	300 except: # re-raises
2435 ff2bac730b99 http client: support persistent connections. Vadim Gelfer <vadim.gelfer@gmail.com> parents: diff changeset	301 # adding this block just in case we've missed
ff2bac730b99 http client: support persistent connections. Vadim Gelfer <vadim.gelfer@gmail.com> parents: diff changeset	302 # something we will still raise the exception, but
ff2bac730b99 http client: support persistent connections. Vadim Gelfer <vadim.gelfer@gmail.com> parents: diff changeset	303 # lets try and close the connection and remove it
ff2bac730b99 http client: support persistent connections. Vadim Gelfer <vadim.gelfer@gmail.com> parents: diff changeset	304 # first. We previously got into a nasty loop
ff2bac730b99 http client: support persistent connections. Vadim Gelfer <vadim.gelfer@gmail.com> parents: diff changeset	305 # where an exception was uncaught, and so the
ff2bac730b99 http client: support persistent connections. Vadim Gelfer <vadim.gelfer@gmail.com> parents: diff changeset	306 # connection stayed open. On the next try, the
17424 e7cfe3587ea4 fix trivial spelling errors Mads Kiilerich <mads@kiilerich.com> parents: 17251 diff changeset	307 # same exception was raised, etc. The trade-off is
2435 ff2bac730b99 http client: support persistent connections. Vadim Gelfer <vadim.gelfer@gmail.com> parents: diff changeset	308 # that it's now possible this call will raise
ff2bac730b99 http client: support persistent connections. Vadim Gelfer <vadim.gelfer@gmail.com> parents: diff changeset	309 # a DIFFERENT exception
10282 08a0f04b56bd many, many trivial check-code fixups Matt Mackall <mpm@selenic.com> parents: 9726 diff changeset	310 if DEBUG:
08a0f04b56bd many, many trivial check-code fixups Matt Mackall <mpm@selenic.com> parents: 9726 diff changeset	311 DEBUG.error("unexpected exception - closing "
08a0f04b56bd many, many trivial check-code fixups Matt Mackall <mpm@selenic.com> parents: 9726 diff changeset	312 "connection to %s (%d)", host, id(h))
2435 ff2bac730b99 http client: support persistent connections. Vadim Gelfer <vadim.gelfer@gmail.com> parents: diff changeset	313 self._cm.remove(h)
ff2bac730b99 http client: support persistent connections. Vadim Gelfer <vadim.gelfer@gmail.com> parents: diff changeset	314 h.close()
ff2bac730b99 http client: support persistent connections. Vadim Gelfer <vadim.gelfer@gmail.com> parents: diff changeset	315 raise
2600 c4325f0a9b91 clean up trailing white space. Vadim Gelfer <vadim.gelfer@gmail.com> parents: 2444 diff changeset	316
2435 ff2bac730b99 http client: support persistent connections. Vadim Gelfer <vadim.gelfer@gmail.com> parents: diff changeset	317 if r is None or r.version == 9:
ff2bac730b99 http client: support persistent connections. Vadim Gelfer <vadim.gelfer@gmail.com> parents: diff changeset	318 # httplib falls back to assuming HTTP 0.9 if it gets a
ff2bac730b99 http client: support persistent connections. Vadim Gelfer <vadim.gelfer@gmail.com> parents: diff changeset	319 # bad header back. This is most likely to happen if
ff2bac730b99 http client: support persistent connections. Vadim Gelfer <vadim.gelfer@gmail.com> parents: diff changeset	320 # the socket has been closed by the server since we
ff2bac730b99 http client: support persistent connections. Vadim Gelfer <vadim.gelfer@gmail.com> parents: diff changeset	321 # last used the connection.
10282 08a0f04b56bd many, many trivial check-code fixups Matt Mackall <mpm@selenic.com> parents: 9726 diff changeset	322 if DEBUG:
08a0f04b56bd many, many trivial check-code fixups Matt Mackall <mpm@selenic.com> parents: 9726 diff changeset	323 DEBUG.info("failed to re-use connection to %s (%d)",
08a0f04b56bd many, many trivial check-code fixups Matt Mackall <mpm@selenic.com> parents: 9726 diff changeset	324 host, id(h))
2435 ff2bac730b99 http client: support persistent connections. Vadim Gelfer <vadim.gelfer@gmail.com> parents: diff changeset	325 r = None
ff2bac730b99 http client: support persistent connections. Vadim Gelfer <vadim.gelfer@gmail.com> parents: diff changeset	326 else:
10282 08a0f04b56bd many, many trivial check-code fixups Matt Mackall <mpm@selenic.com> parents: 9726 diff changeset	327 if DEBUG:
08a0f04b56bd many, many trivial check-code fixups Matt Mackall <mpm@selenic.com> parents: 9726 diff changeset	328 DEBUG.info("re-using connection to %s (%d)", host, id(h))
2435 ff2bac730b99 http client: support persistent connections. Vadim Gelfer <vadim.gelfer@gmail.com> parents: diff changeset	329
ff2bac730b99 http client: support persistent connections. Vadim Gelfer <vadim.gelfer@gmail.com> parents: diff changeset	330 return r
ff2bac730b99 http client: support persistent connections. Vadim Gelfer <vadim.gelfer@gmail.com> parents: diff changeset	331
ff2bac730b99 http client: support persistent connections. Vadim Gelfer <vadim.gelfer@gmail.com> parents: diff changeset	332 def _start_transaction(self, h, req):
8233 655c435efe92 keepalive: fix 4f13ed6ee544, reintroduce unredirected_hdrs Patrick Mezard <pmezard@gmail.com> parents: 8146 diff changeset	333 # What follows mostly reimplements HTTPConnection.request()
655c435efe92 keepalive: fix 4f13ed6ee544, reintroduce unredirected_hdrs Patrick Mezard <pmezard@gmail.com> parents: 8146 diff changeset	334 # except it adds self.parent.addheaders in the mix.
30463 bc0def54c17d keepalive: reorder header precedence Gregory Szorc <gregory.szorc@gmail.com> parents: 29456 diff changeset	335 headers = dict(self.parent.addheaders)
bc0def54c17d keepalive: reorder header precedence Gregory Szorc <gregory.szorc@gmail.com> parents: 29456 diff changeset	336 headers.update(req.headers)
bc0def54c17d keepalive: reorder header precedence Gregory Szorc <gregory.szorc@gmail.com> parents: 29456 diff changeset	337 headers.update(req.unredirected_hdrs)
10282 08a0f04b56bd many, many trivial check-code fixups Matt Mackall <mpm@selenic.com> parents: 9726 diff changeset	338 headers = dict((n.lower(), v) for n, v in headers.items())
8233 655c435efe92 keepalive: fix 4f13ed6ee544, reintroduce unredirected_hdrs Patrick Mezard <pmezard@gmail.com> parents: 8146 diff changeset	339 skipheaders = {}
655c435efe92 keepalive: fix 4f13ed6ee544, reintroduce unredirected_hdrs Patrick Mezard <pmezard@gmail.com> parents: 8146 diff changeset	340 for n in ('host', 'accept-encoding'):
655c435efe92 keepalive: fix 4f13ed6ee544, reintroduce unredirected_hdrs Patrick Mezard <pmezard@gmail.com> parents: 8146 diff changeset	341 if n in headers:
655c435efe92 keepalive: fix 4f13ed6ee544, reintroduce unredirected_hdrs Patrick Mezard <pmezard@gmail.com> parents: 8146 diff changeset	342 skipheaders['skip_' + n.replace('-', '_')] = 1
2435 ff2bac730b99 http client: support persistent connections. Vadim Gelfer <vadim.gelfer@gmail.com> parents: diff changeset	343 try:
8146 4f13ed6ee544 keepalive: attempt to fix issue1003 Matt Mackall <mpm@selenic.com> parents: 7875 diff changeset	344 if req.has_data():
4f13ed6ee544 keepalive: attempt to fix issue1003 Matt Mackall <mpm@selenic.com> parents: 7875 diff changeset	345 data = req.get_data()
8233 655c435efe92 keepalive: fix 4f13ed6ee544, reintroduce unredirected_hdrs Patrick Mezard <pmezard@gmail.com> parents: 8146 diff changeset	346 h.putrequest('POST', req.get_selector(), **skipheaders)
655c435efe92 keepalive: fix 4f13ed6ee544, reintroduce unredirected_hdrs Patrick Mezard <pmezard@gmail.com> parents: 8146 diff changeset	347 if 'content-type' not in headers:
8146 4f13ed6ee544 keepalive: attempt to fix issue1003 Matt Mackall <mpm@selenic.com> parents: 7875 diff changeset	348 h.putheader('Content-type',
4f13ed6ee544 keepalive: attempt to fix issue1003 Matt Mackall <mpm@selenic.com> parents: 7875 diff changeset	349 'application/x-www-form-urlencoded')
8233 655c435efe92 keepalive: fix 4f13ed6ee544, reintroduce unredirected_hdrs Patrick Mezard <pmezard@gmail.com> parents: 8146 diff changeset	350 if 'content-length' not in headers:
8146 4f13ed6ee544 keepalive: attempt to fix issue1003 Matt Mackall <mpm@selenic.com> parents: 7875 diff changeset	351 h.putheader('Content-length', '%d' % len(data))
4f13ed6ee544 keepalive: attempt to fix issue1003 Matt Mackall <mpm@selenic.com> parents: 7875 diff changeset	352 else:
8233 655c435efe92 keepalive: fix 4f13ed6ee544, reintroduce unredirected_hdrs Patrick Mezard <pmezard@gmail.com> parents: 8146 diff changeset	353 h.putrequest('GET', req.get_selector(), **skipheaders)
28278 b1b22185c764 keepalive: remove useless parentheses around exception type Gregory Szorc <gregory.szorc@gmail.com> parents: 27616 diff changeset	354 except socket.error as err:
28883 032c4c2f802a pycompat: switch to util.urlreq/util.urlerr for py3 compat timeless <timeless@mozdev.org> parents: 28278 diff changeset	355 raise urlerr.urlerror(err)
8233 655c435efe92 keepalive: fix 4f13ed6ee544, reintroduce unredirected_hdrs Patrick Mezard <pmezard@gmail.com> parents: 8146 diff changeset	356 for k, v in headers.items():
8146 4f13ed6ee544 keepalive: attempt to fix issue1003 Matt Mackall <mpm@selenic.com> parents: 7875 diff changeset	357 h.putheader(k, v)
4f13ed6ee544 keepalive: attempt to fix issue1003 Matt Mackall <mpm@selenic.com> parents: 7875 diff changeset	358 h.endheaders()
4f13ed6ee544 keepalive: attempt to fix issue1003 Matt Mackall <mpm@selenic.com> parents: 7875 diff changeset	359 if req.has_data():
4f13ed6ee544 keepalive: attempt to fix issue1003 Matt Mackall <mpm@selenic.com> parents: 7875 diff changeset	360 h.send(data)
4f13ed6ee544 keepalive: attempt to fix issue1003 Matt Mackall <mpm@selenic.com> parents: 7875 diff changeset	361
28883 032c4c2f802a pycompat: switch to util.urlreq/util.urlerr for py3 compat timeless <timeless@mozdev.org> parents: 28278 diff changeset	362 class HTTPHandler(KeepAliveHandler, urlreq.httphandler):
5983 6f1fcbc58efa httprepo: use separate handlers for HTTP and HTTPS Alexis S. L. Carvalho <alexis@cecm.usp.br> parents: 4026 diff changeset	363 pass
6f1fcbc58efa httprepo: use separate handlers for HTTP and HTTPS Alexis S. L. Carvalho <alexis@cecm.usp.br> parents: 4026 diff changeset	364
2435 ff2bac730b99 http client: support persistent connections. Vadim Gelfer <vadim.gelfer@gmail.com> parents: diff changeset	365 class HTTPResponse(httplib.HTTPResponse):
ff2bac730b99 http client: support persistent connections. Vadim Gelfer <vadim.gelfer@gmail.com> parents: diff changeset	366 # we need to subclass HTTPResponse in order to
ff2bac730b99 http client: support persistent connections. Vadim Gelfer <vadim.gelfer@gmail.com> parents: diff changeset	367 # 1) add readline() and readlines() methods
ff2bac730b99 http client: support persistent connections. Vadim Gelfer <vadim.gelfer@gmail.com> parents: diff changeset	368 # 2) add close_connection() methods
ff2bac730b99 http client: support persistent connections. Vadim Gelfer <vadim.gelfer@gmail.com> parents: diff changeset	369 # 3) add info() and geturl() methods
ff2bac730b99 http client: support persistent connections. Vadim Gelfer <vadim.gelfer@gmail.com> parents: diff changeset	370
ff2bac730b99 http client: support persistent connections. Vadim Gelfer <vadim.gelfer@gmail.com> parents: diff changeset	371 # in order to add readline(), read must be modified to deal with a
ff2bac730b99 http client: support persistent connections. Vadim Gelfer <vadim.gelfer@gmail.com> parents: diff changeset	372 # buffer. example: readline must read a buffer and then spit back
ff2bac730b99 http client: support persistent connections. Vadim Gelfer <vadim.gelfer@gmail.com> parents: diff changeset	373 # one line at a time. The only real alternative is to read one
ff2bac730b99 http client: support persistent connections. Vadim Gelfer <vadim.gelfer@gmail.com> parents: diff changeset	374 # BYTE at a time (ick). Once something has been read, it can't be
ff2bac730b99 http client: support persistent connections. Vadim Gelfer <vadim.gelfer@gmail.com> parents: diff changeset	375 # put back (ok, maybe it can, but that's even uglier than this),
ff2bac730b99 http client: support persistent connections. Vadim Gelfer <vadim.gelfer@gmail.com> parents: diff changeset	376 # so if you THEN do a normal read, you must first take stuff from
ff2bac730b99 http client: support persistent connections. Vadim Gelfer <vadim.gelfer@gmail.com> parents: diff changeset	377 # the buffer.
ff2bac730b99 http client: support persistent connections. Vadim Gelfer <vadim.gelfer@gmail.com> parents: diff changeset	378
17424 e7cfe3587ea4 fix trivial spelling errors Mads Kiilerich <mads@kiilerich.com> parents: 17251 diff changeset	379 # the read method wraps the original to accommodate buffering,
2435 ff2bac730b99 http client: support persistent connections. Vadim Gelfer <vadim.gelfer@gmail.com> parents: diff changeset	380 # although read() never adds to the buffer.
ff2bac730b99 http client: support persistent connections. Vadim Gelfer <vadim.gelfer@gmail.com> parents: diff changeset	381 # Both readline and readlines have been stolen with almost no
ff2bac730b99 http client: support persistent connections. Vadim Gelfer <vadim.gelfer@gmail.com> parents: diff changeset	382 # modification from socket.py
2600 c4325f0a9b91 clean up trailing white space. Vadim Gelfer <vadim.gelfer@gmail.com> parents: 2444 diff changeset	383
2435 ff2bac730b99 http client: support persistent connections. Vadim Gelfer <vadim.gelfer@gmail.com> parents: diff changeset	384
ff2bac730b99 http client: support persistent connections. Vadim Gelfer <vadim.gelfer@gmail.com> parents: diff changeset	385 def __init__(self, sock, debuglevel=0, strict=0, method=None):
17700 5b1b0e4e6902 keepalive: drop python 2.2 legacy code Bryan O'Sullivan <bryano@fb.com> parents: 17424 diff changeset	386 httplib.HTTPResponse.__init__(self, sock, debuglevel, method)
2435 ff2bac730b99 http client: support persistent connections. Vadim Gelfer <vadim.gelfer@gmail.com> parents: diff changeset	387 self.fileno = sock.fileno
ff2bac730b99 http client: support persistent connections. Vadim Gelfer <vadim.gelfer@gmail.com> parents: diff changeset	388 self.code = None
ff2bac730b99 http client: support persistent connections. Vadim Gelfer <vadim.gelfer@gmail.com> parents: diff changeset	389 self._rbuf = ''
ff2bac730b99 http client: support persistent connections. Vadim Gelfer <vadim.gelfer@gmail.com> parents: diff changeset	390 self._rbufsize = 8096
ff2bac730b99 http client: support persistent connections. Vadim Gelfer <vadim.gelfer@gmail.com> parents: diff changeset	391 self._handler = None # inserted by the handler later
ff2bac730b99 http client: support persistent connections. Vadim Gelfer <vadim.gelfer@gmail.com> parents: diff changeset	392 self._host = None # (same)
ff2bac730b99 http client: support persistent connections. Vadim Gelfer <vadim.gelfer@gmail.com> parents: diff changeset	393 self._url = None # (same)
ff2bac730b99 http client: support persistent connections. Vadim Gelfer <vadim.gelfer@gmail.com> parents: diff changeset	394 self._connection = None # (same)
ff2bac730b99 http client: support persistent connections. Vadim Gelfer <vadim.gelfer@gmail.com> parents: diff changeset	395
ff2bac730b99 http client: support persistent connections. Vadim Gelfer <vadim.gelfer@gmail.com> parents: diff changeset	396 _raw_read = httplib.HTTPResponse.read
ff2bac730b99 http client: support persistent connections. Vadim Gelfer <vadim.gelfer@gmail.com> parents: diff changeset	397
ff2bac730b99 http client: support persistent connections. Vadim Gelfer <vadim.gelfer@gmail.com> parents: diff changeset	398 def close(self):
ff2bac730b99 http client: support persistent connections. Vadim Gelfer <vadim.gelfer@gmail.com> parents: diff changeset	399 if self.fp:
ff2bac730b99 http client: support persistent connections. Vadim Gelfer <vadim.gelfer@gmail.com> parents: diff changeset	400 self.fp.close()
ff2bac730b99 http client: support persistent connections. Vadim Gelfer <vadim.gelfer@gmail.com> parents: diff changeset	401 self.fp = None
ff2bac730b99 http client: support persistent connections. Vadim Gelfer <vadim.gelfer@gmail.com> parents: diff changeset	402 if self._handler:
ff2bac730b99 http client: support persistent connections. Vadim Gelfer <vadim.gelfer@gmail.com> parents: diff changeset	403 self._handler._request_closed(self, self._host,
ff2bac730b99 http client: support persistent connections. Vadim Gelfer <vadim.gelfer@gmail.com> parents: diff changeset	404 self._connection)
ff2bac730b99 http client: support persistent connections. Vadim Gelfer <vadim.gelfer@gmail.com> parents: diff changeset	405
ff2bac730b99 http client: support persistent connections. Vadim Gelfer <vadim.gelfer@gmail.com> parents: diff changeset	406 def close_connection(self):
ff2bac730b99 http client: support persistent connections. Vadim Gelfer <vadim.gelfer@gmail.com> parents: diff changeset	407 self._handler._remove_connection(self._host, self._connection, close=1)
ff2bac730b99 http client: support persistent connections. Vadim Gelfer <vadim.gelfer@gmail.com> parents: diff changeset	408 self.close()
2600 c4325f0a9b91 clean up trailing white space. Vadim Gelfer <vadim.gelfer@gmail.com> parents: 2444 diff changeset	409
2435 ff2bac730b99 http client: support persistent connections. Vadim Gelfer <vadim.gelfer@gmail.com> parents: diff changeset	410 def info(self):
2444 5eb02f9ed804 Update keepalive.py to current CVS version of urlgrabber. Alexis S. L. Carvalho <alexis@cecm.usp.br> parents: 2435 diff changeset	411 return self.headers
2435 ff2bac730b99 http client: support persistent connections. Vadim Gelfer <vadim.gelfer@gmail.com> parents: diff changeset	412
ff2bac730b99 http client: support persistent connections. Vadim Gelfer <vadim.gelfer@gmail.com> parents: diff changeset	413 def geturl(self):
ff2bac730b99 http client: support persistent connections. Vadim Gelfer <vadim.gelfer@gmail.com> parents: diff changeset	414 return self._url
ff2bac730b99 http client: support persistent connections. Vadim Gelfer <vadim.gelfer@gmail.com> parents: diff changeset	415
ff2bac730b99 http client: support persistent connections. Vadim Gelfer <vadim.gelfer@gmail.com> parents: diff changeset	416 def read(self, amt=None):
ff2bac730b99 http client: support persistent connections. Vadim Gelfer <vadim.gelfer@gmail.com> parents: diff changeset	417 # the _rbuf test is only in this first if for speed. It's not
ff2bac730b99 http client: support persistent connections. Vadim Gelfer <vadim.gelfer@gmail.com> parents: diff changeset	418 # logically necessary
ff2bac730b99 http client: support persistent connections. Vadim Gelfer <vadim.gelfer@gmail.com> parents: diff changeset	419 if self._rbuf and not amt is None:
ff2bac730b99 http client: support persistent connections. Vadim Gelfer <vadim.gelfer@gmail.com> parents: diff changeset	420 L = len(self._rbuf)
ff2bac730b99 http client: support persistent connections. Vadim Gelfer <vadim.gelfer@gmail.com> parents: diff changeset	421 if amt > L:
ff2bac730b99 http client: support persistent connections. Vadim Gelfer <vadim.gelfer@gmail.com> parents: diff changeset	422 amt -= L
ff2bac730b99 http client: support persistent connections. Vadim Gelfer <vadim.gelfer@gmail.com> parents: diff changeset	423 else:
ff2bac730b99 http client: support persistent connections. Vadim Gelfer <vadim.gelfer@gmail.com> parents: diff changeset	424 s = self._rbuf[:amt]
ff2bac730b99 http client: support persistent connections. Vadim Gelfer <vadim.gelfer@gmail.com> parents: diff changeset	425 self._rbuf = self._rbuf[amt:]
ff2bac730b99 http client: support persistent connections. Vadim Gelfer <vadim.gelfer@gmail.com> parents: diff changeset	426 return s
ff2bac730b99 http client: support persistent connections. Vadim Gelfer <vadim.gelfer@gmail.com> parents: diff changeset	427
ff2bac730b99 http client: support persistent connections. Vadim Gelfer <vadim.gelfer@gmail.com> parents: diff changeset	428 s = self._rbuf + self._raw_read(amt)
ff2bac730b99 http client: support persistent connections. Vadim Gelfer <vadim.gelfer@gmail.com> parents: diff changeset	429 self._rbuf = ''
ff2bac730b99 http client: support persistent connections. Vadim Gelfer <vadim.gelfer@gmail.com> parents: diff changeset	430 return s
ff2bac730b99 http client: support persistent connections. Vadim Gelfer <vadim.gelfer@gmail.com> parents: diff changeset	431
7781 a45206455d85 keepalive: borrow code from newer httplib to patch ValueError (issue1088) Matt Mackall <mpm@selenic.com> parents: 7622 diff changeset	432 # stolen from Python SVN #68532 to fix issue1088
a45206455d85 keepalive: borrow code from newer httplib to patch ValueError (issue1088) Matt Mackall <mpm@selenic.com> parents: 7622 diff changeset	433 def _read_chunked(self, amt):
a45206455d85 keepalive: borrow code from newer httplib to patch ValueError (issue1088) Matt Mackall <mpm@selenic.com> parents: 7622 diff changeset	434 chunk_left = self.chunk_left
a45206455d85 keepalive: borrow code from newer httplib to patch ValueError (issue1088) Matt Mackall <mpm@selenic.com> parents: 7622 diff changeset	435 value = ''
a45206455d85 keepalive: borrow code from newer httplib to patch ValueError (issue1088) Matt Mackall <mpm@selenic.com> parents: 7622 diff changeset	436
a45206455d85 keepalive: borrow code from newer httplib to patch ValueError (issue1088) Matt Mackall <mpm@selenic.com> parents: 7622 diff changeset	437 # XXX This accumulates chunks by repeated string concatenation,
a45206455d85 keepalive: borrow code from newer httplib to patch ValueError (issue1088) Matt Mackall <mpm@selenic.com> parents: 7622 diff changeset	438 # which is not efficient as the number or size of chunks gets big.
a45206455d85 keepalive: borrow code from newer httplib to patch ValueError (issue1088) Matt Mackall <mpm@selenic.com> parents: 7622 diff changeset	439 while True:
a45206455d85 keepalive: borrow code from newer httplib to patch ValueError (issue1088) Matt Mackall <mpm@selenic.com> parents: 7622 diff changeset	440 if chunk_left is None:
a45206455d85 keepalive: borrow code from newer httplib to patch ValueError (issue1088) Matt Mackall <mpm@selenic.com> parents: 7622 diff changeset	441 line = self.fp.readline()
a45206455d85 keepalive: borrow code from newer httplib to patch ValueError (issue1088) Matt Mackall <mpm@selenic.com> parents: 7622 diff changeset	442 i = line.find(';')
a45206455d85 keepalive: borrow code from newer httplib to patch ValueError (issue1088) Matt Mackall <mpm@selenic.com> parents: 7622 diff changeset	443 if i >= 0:
a45206455d85 keepalive: borrow code from newer httplib to patch ValueError (issue1088) Matt Mackall <mpm@selenic.com> parents: 7622 diff changeset	444 line = line[:i] # strip chunk-extensions
a45206455d85 keepalive: borrow code from newer httplib to patch ValueError (issue1088) Matt Mackall <mpm@selenic.com> parents: 7622 diff changeset	445 try:
a45206455d85 keepalive: borrow code from newer httplib to patch ValueError (issue1088) Matt Mackall <mpm@selenic.com> parents: 7622 diff changeset	446 chunk_left = int(line, 16)
a45206455d85 keepalive: borrow code from newer httplib to patch ValueError (issue1088) Matt Mackall <mpm@selenic.com> parents: 7622 diff changeset	447 except ValueError:
17424 e7cfe3587ea4 fix trivial spelling errors Mads Kiilerich <mads@kiilerich.com> parents: 17251 diff changeset	448 # close the connection as protocol synchronization is
7781 a45206455d85 keepalive: borrow code from newer httplib to patch ValueError (issue1088) Matt Mackall <mpm@selenic.com> parents: 7622 diff changeset	449 # probably lost
a45206455d85 keepalive: borrow code from newer httplib to patch ValueError (issue1088) Matt Mackall <mpm@selenic.com> parents: 7622 diff changeset	450 self.close()
7868 2b901f9b37aa keepalive: fix reference to IncompleteRead Peter Arrenbrecht <peter.arrenbrecht@gmail.com> parents: 7781 diff changeset	451 raise httplib.IncompleteRead(value)
7781 a45206455d85 keepalive: borrow code from newer httplib to patch ValueError (issue1088) Matt Mackall <mpm@selenic.com> parents: 7622 diff changeset	452 if chunk_left == 0:
a45206455d85 keepalive: borrow code from newer httplib to patch ValueError (issue1088) Matt Mackall <mpm@selenic.com> parents: 7622 diff changeset	453 break
a45206455d85 keepalive: borrow code from newer httplib to patch ValueError (issue1088) Matt Mackall <mpm@selenic.com> parents: 7622 diff changeset	454 if amt is None:
a45206455d85 keepalive: borrow code from newer httplib to patch ValueError (issue1088) Matt Mackall <mpm@selenic.com> parents: 7622 diff changeset	455 value += self._safe_read(chunk_left)
a45206455d85 keepalive: borrow code from newer httplib to patch ValueError (issue1088) Matt Mackall <mpm@selenic.com> parents: 7622 diff changeset	456 elif amt < chunk_left:
a45206455d85 keepalive: borrow code from newer httplib to patch ValueError (issue1088) Matt Mackall <mpm@selenic.com> parents: 7622 diff changeset	457 value += self._safe_read(amt)
a45206455d85 keepalive: borrow code from newer httplib to patch ValueError (issue1088) Matt Mackall <mpm@selenic.com> parents: 7622 diff changeset	458 self.chunk_left = chunk_left - amt
a45206455d85 keepalive: borrow code from newer httplib to patch ValueError (issue1088) Matt Mackall <mpm@selenic.com> parents: 7622 diff changeset	459 return value
a45206455d85 keepalive: borrow code from newer httplib to patch ValueError (issue1088) Matt Mackall <mpm@selenic.com> parents: 7622 diff changeset	460 elif amt == chunk_left:
a45206455d85 keepalive: borrow code from newer httplib to patch ValueError (issue1088) Matt Mackall <mpm@selenic.com> parents: 7622 diff changeset	461 value += self._safe_read(amt)
a45206455d85 keepalive: borrow code from newer httplib to patch ValueError (issue1088) Matt Mackall <mpm@selenic.com> parents: 7622 diff changeset	462 self._safe_read(2) # toss the CRLF at the end of the chunk
a45206455d85 keepalive: borrow code from newer httplib to patch ValueError (issue1088) Matt Mackall <mpm@selenic.com> parents: 7622 diff changeset	463 self.chunk_left = None
a45206455d85 keepalive: borrow code from newer httplib to patch ValueError (issue1088) Matt Mackall <mpm@selenic.com> parents: 7622 diff changeset	464 return value
a45206455d85 keepalive: borrow code from newer httplib to patch ValueError (issue1088) Matt Mackall <mpm@selenic.com> parents: 7622 diff changeset	465 else:
a45206455d85 keepalive: borrow code from newer httplib to patch ValueError (issue1088) Matt Mackall <mpm@selenic.com> parents: 7622 diff changeset	466 value += self._safe_read(chunk_left)
a45206455d85 keepalive: borrow code from newer httplib to patch ValueError (issue1088) Matt Mackall <mpm@selenic.com> parents: 7622 diff changeset	467 amt -= chunk_left
a45206455d85 keepalive: borrow code from newer httplib to patch ValueError (issue1088) Matt Mackall <mpm@selenic.com> parents: 7622 diff changeset	468
a45206455d85 keepalive: borrow code from newer httplib to patch ValueError (issue1088) Matt Mackall <mpm@selenic.com> parents: 7622 diff changeset	469 # we read the whole chunk, get another
a45206455d85 keepalive: borrow code from newer httplib to patch ValueError (issue1088) Matt Mackall <mpm@selenic.com> parents: 7622 diff changeset	470 self._safe_read(2) # toss the CRLF at the end of the chunk
a45206455d85 keepalive: borrow code from newer httplib to patch ValueError (issue1088) Matt Mackall <mpm@selenic.com> parents: 7622 diff changeset	471 chunk_left = None
a45206455d85 keepalive: borrow code from newer httplib to patch ValueError (issue1088) Matt Mackall <mpm@selenic.com> parents: 7622 diff changeset	472
a45206455d85 keepalive: borrow code from newer httplib to patch ValueError (issue1088) Matt Mackall <mpm@selenic.com> parents: 7622 diff changeset	473 # read and discard trailer up to the CRLF terminator
a45206455d85 keepalive: borrow code from newer httplib to patch ValueError (issue1088) Matt Mackall <mpm@selenic.com> parents: 7622 diff changeset	474 ### note: we shouldn't have any trailers!
a45206455d85 keepalive: borrow code from newer httplib to patch ValueError (issue1088) Matt Mackall <mpm@selenic.com> parents: 7622 diff changeset	475 while True:
a45206455d85 keepalive: borrow code from newer httplib to patch ValueError (issue1088) Matt Mackall <mpm@selenic.com> parents: 7622 diff changeset	476 line = self.fp.readline()
a45206455d85 keepalive: borrow code from newer httplib to patch ValueError (issue1088) Matt Mackall <mpm@selenic.com> parents: 7622 diff changeset	477 if not line:
a45206455d85 keepalive: borrow code from newer httplib to patch ValueError (issue1088) Matt Mackall <mpm@selenic.com> parents: 7622 diff changeset	478 # a vanishingly small number of sites EOF without
a45206455d85 keepalive: borrow code from newer httplib to patch ValueError (issue1088) Matt Mackall <mpm@selenic.com> parents: 7622 diff changeset	479 # sending the trailer
a45206455d85 keepalive: borrow code from newer httplib to patch ValueError (issue1088) Matt Mackall <mpm@selenic.com> parents: 7622 diff changeset	480 break
a45206455d85 keepalive: borrow code from newer httplib to patch ValueError (issue1088) Matt Mackall <mpm@selenic.com> parents: 7622 diff changeset	481 if line == '\r\n':
a45206455d85 keepalive: borrow code from newer httplib to patch ValueError (issue1088) Matt Mackall <mpm@selenic.com> parents: 7622 diff changeset	482 break
a45206455d85 keepalive: borrow code from newer httplib to patch ValueError (issue1088) Matt Mackall <mpm@selenic.com> parents: 7622 diff changeset	483
a45206455d85 keepalive: borrow code from newer httplib to patch ValueError (issue1088) Matt Mackall <mpm@selenic.com> parents: 7622 diff changeset	484 # we read everything; close the "file"
a45206455d85 keepalive: borrow code from newer httplib to patch ValueError (issue1088) Matt Mackall <mpm@selenic.com> parents: 7622 diff changeset	485 self.close()
a45206455d85 keepalive: borrow code from newer httplib to patch ValueError (issue1088) Matt Mackall <mpm@selenic.com> parents: 7622 diff changeset	486
a45206455d85 keepalive: borrow code from newer httplib to patch ValueError (issue1088) Matt Mackall <mpm@selenic.com> parents: 7622 diff changeset	487 return value
a45206455d85 keepalive: borrow code from newer httplib to patch ValueError (issue1088) Matt Mackall <mpm@selenic.com> parents: 7622 diff changeset	488
2435 ff2bac730b99 http client: support persistent connections. Vadim Gelfer <vadim.gelfer@gmail.com> parents: diff changeset	489 def readline(self, limit=-1):
ff2bac730b99 http client: support persistent connections. Vadim Gelfer <vadim.gelfer@gmail.com> parents: diff changeset	490 i = self._rbuf.find('\n')
ff2bac730b99 http client: support persistent connections. Vadim Gelfer <vadim.gelfer@gmail.com> parents: diff changeset	491 while i < 0 and not (0 < limit <= len(self._rbuf)):
ff2bac730b99 http client: support persistent connections. Vadim Gelfer <vadim.gelfer@gmail.com> parents: diff changeset	492 new = self._raw_read(self._rbufsize)
10282 08a0f04b56bd many, many trivial check-code fixups Matt Mackall <mpm@selenic.com> parents: 9726 diff changeset	493 if not new:
08a0f04b56bd many, many trivial check-code fixups Matt Mackall <mpm@selenic.com> parents: 9726 diff changeset	494 break
2435 ff2bac730b99 http client: support persistent connections. Vadim Gelfer <vadim.gelfer@gmail.com> parents: diff changeset	495 i = new.find('\n')
10282 08a0f04b56bd many, many trivial check-code fixups Matt Mackall <mpm@selenic.com> parents: 9726 diff changeset	496 if i >= 0:
08a0f04b56bd many, many trivial check-code fixups Matt Mackall <mpm@selenic.com> parents: 9726 diff changeset	497 i = i + len(self._rbuf)
2435 ff2bac730b99 http client: support persistent connections. Vadim Gelfer <vadim.gelfer@gmail.com> parents: diff changeset	498 self._rbuf = self._rbuf + new
10282 08a0f04b56bd many, many trivial check-code fixups Matt Mackall <mpm@selenic.com> parents: 9726 diff changeset	499 if i < 0:
08a0f04b56bd many, many trivial check-code fixups Matt Mackall <mpm@selenic.com> parents: 9726 diff changeset	500 i = len(self._rbuf)
08a0f04b56bd many, many trivial check-code fixups Matt Mackall <mpm@selenic.com> parents: 9726 diff changeset	501 else:
08a0f04b56bd many, many trivial check-code fixups Matt Mackall <mpm@selenic.com> parents: 9726 diff changeset	502 i = i + 1
08a0f04b56bd many, many trivial check-code fixups Matt Mackall <mpm@selenic.com> parents: 9726 diff changeset	503 if 0 <= limit < len(self._rbuf):
08a0f04b56bd many, many trivial check-code fixups Matt Mackall <mpm@selenic.com> parents: 9726 diff changeset	504 i = limit
2435 ff2bac730b99 http client: support persistent connections. Vadim Gelfer <vadim.gelfer@gmail.com> parents: diff changeset	505 data, self._rbuf = self._rbuf[:i], self._rbuf[i:]
ff2bac730b99 http client: support persistent connections. Vadim Gelfer <vadim.gelfer@gmail.com> parents: diff changeset	506 return data
ff2bac730b99 http client: support persistent connections. Vadim Gelfer <vadim.gelfer@gmail.com> parents: diff changeset	507
19872 681f7b9213a4 check-code: check for spaces around = for named parameters Mads Kiilerich <madski@unity3d.com> parents: 17700 diff changeset	508 def readlines(self, sizehint=0):
2435 ff2bac730b99 http client: support persistent connections. Vadim Gelfer <vadim.gelfer@gmail.com> parents: diff changeset	509 total = 0
ff2bac730b99 http client: support persistent connections. Vadim Gelfer <vadim.gelfer@gmail.com> parents: diff changeset	510 list = []
14494 1ffeeb91c55d check-code: flag 0/1 used as constant Boolean expression Martin Geisler <mg@lazybytes.net> parents: 10394 diff changeset	511 while True:
2435 ff2bac730b99 http client: support persistent connections. Vadim Gelfer <vadim.gelfer@gmail.com> parents: diff changeset	512 line = self.readline()
10282 08a0f04b56bd many, many trivial check-code fixups Matt Mackall <mpm@selenic.com> parents: 9726 diff changeset	513 if not line:
08a0f04b56bd many, many trivial check-code fixups Matt Mackall <mpm@selenic.com> parents: 9726 diff changeset	514 break
2435 ff2bac730b99 http client: support persistent connections. Vadim Gelfer <vadim.gelfer@gmail.com> parents: diff changeset	515 list.append(line)
ff2bac730b99 http client: support persistent connections. Vadim Gelfer <vadim.gelfer@gmail.com> parents: diff changeset	516 total += len(line)
ff2bac730b99 http client: support persistent connections. Vadim Gelfer <vadim.gelfer@gmail.com> parents: diff changeset	517 if sizehint and total >= sizehint:
ff2bac730b99 http client: support persistent connections. Vadim Gelfer <vadim.gelfer@gmail.com> parents: diff changeset	518 break
ff2bac730b99 http client: support persistent connections. Vadim Gelfer <vadim.gelfer@gmail.com> parents: diff changeset	519 return list
ff2bac730b99 http client: support persistent connections. Vadim Gelfer <vadim.gelfer@gmail.com> parents: diff changeset	520
9726 430e59ff3437 keepalive: handle broken pipes gracefully during large POSTs Augie Fackler <durin42@gmail.com> parents: 8296 diff changeset	521 def safesend(self, str):
430e59ff3437 keepalive: handle broken pipes gracefully during large POSTs Augie Fackler <durin42@gmail.com> parents: 8296 diff changeset	522 """Send `str' to the server.
430e59ff3437 keepalive: handle broken pipes gracefully during large POSTs Augie Fackler <durin42@gmail.com> parents: 8296 diff changeset	523
430e59ff3437 keepalive: handle broken pipes gracefully during large POSTs Augie Fackler <durin42@gmail.com> parents: 8296 diff changeset	524 Shamelessly ripped off from httplib to patch a bad behavior.
430e59ff3437 keepalive: handle broken pipes gracefully during large POSTs Augie Fackler <durin42@gmail.com> parents: 8296 diff changeset	525 """
430e59ff3437 keepalive: handle broken pipes gracefully during large POSTs Augie Fackler <durin42@gmail.com> parents: 8296 diff changeset	526 # _broken_pipe_resp is an attribute we set in this function
430e59ff3437 keepalive: handle broken pipes gracefully during large POSTs Augie Fackler <durin42@gmail.com> parents: 8296 diff changeset	527 # if the socket is closed while we're sending data but
430e59ff3437 keepalive: handle broken pipes gracefully during large POSTs Augie Fackler <durin42@gmail.com> parents: 8296 diff changeset	528 # the server sent us a response before hanging up.
430e59ff3437 keepalive: handle broken pipes gracefully during large POSTs Augie Fackler <durin42@gmail.com> parents: 8296 diff changeset	529 # In that case, we want to pretend to send the rest of the
430e59ff3437 keepalive: handle broken pipes gracefully during large POSTs Augie Fackler <durin42@gmail.com> parents: 8296 diff changeset	530 # outgoing data, and then let the user use getresponse()
430e59ff3437 keepalive: handle broken pipes gracefully during large POSTs Augie Fackler <durin42@gmail.com> parents: 8296 diff changeset	531 # (which we wrap) to get this last response before
430e59ff3437 keepalive: handle broken pipes gracefully during large POSTs Augie Fackler <durin42@gmail.com> parents: 8296 diff changeset	532 # opening a new socket.
430e59ff3437 keepalive: handle broken pipes gracefully during large POSTs Augie Fackler <durin42@gmail.com> parents: 8296 diff changeset	533 if getattr(self, '_broken_pipe_resp', None) is not None:
430e59ff3437 keepalive: handle broken pipes gracefully during large POSTs Augie Fackler <durin42@gmail.com> parents: 8296 diff changeset	534 return
430e59ff3437 keepalive: handle broken pipes gracefully during large POSTs Augie Fackler <durin42@gmail.com> parents: 8296 diff changeset	535
430e59ff3437 keepalive: handle broken pipes gracefully during large POSTs Augie Fackler <durin42@gmail.com> parents: 8296 diff changeset	536 if self.sock is None:
430e59ff3437 keepalive: handle broken pipes gracefully during large POSTs Augie Fackler <durin42@gmail.com> parents: 8296 diff changeset	537 if self.auto_open:
430e59ff3437 keepalive: handle broken pipes gracefully during large POSTs Augie Fackler <durin42@gmail.com> parents: 8296 diff changeset	538 self.connect()
430e59ff3437 keepalive: handle broken pipes gracefully during large POSTs Augie Fackler <durin42@gmail.com> parents: 8296 diff changeset	539 else:
16687 e34106fa0dc3 cleanup: "raise SomeException()" -> "raise SomeException" Brodie Rao <brodie@sf.io> parents: 16686 diff changeset	540 raise httplib.NotConnected
9726 430e59ff3437 keepalive: handle broken pipes gracefully during large POSTs Augie Fackler <durin42@gmail.com> parents: 8296 diff changeset	541
430e59ff3437 keepalive: handle broken pipes gracefully during large POSTs Augie Fackler <durin42@gmail.com> parents: 8296 diff changeset	542 # send the data to the server. if we get a broken pipe, then close
430e59ff3437 keepalive: handle broken pipes gracefully during large POSTs Augie Fackler <durin42@gmail.com> parents: 8296 diff changeset	543 # the socket. we want to reconnect when somebody tries to send again.
430e59ff3437 keepalive: handle broken pipes gracefully during large POSTs Augie Fackler <durin42@gmail.com> parents: 8296 diff changeset	544 #
430e59ff3437 keepalive: handle broken pipes gracefully during large POSTs Augie Fackler <durin42@gmail.com> parents: 8296 diff changeset	545 # NOTE: we DO propagate the error, though, because we cannot simply
430e59ff3437 keepalive: handle broken pipes gracefully during large POSTs Augie Fackler <durin42@gmail.com> parents: 8296 diff changeset	546 # ignore the error... the caller will know if they can retry.
430e59ff3437 keepalive: handle broken pipes gracefully during large POSTs Augie Fackler <durin42@gmail.com> parents: 8296 diff changeset	547 if self.debuglevel > 0:
27616 0765d8423fbc keepalive: use print function Gregory Szorc <gregory.szorc@gmail.com> parents: 27507 diff changeset	548 print("send:", repr(str))
9726 430e59ff3437 keepalive: handle broken pipes gracefully during large POSTs Augie Fackler <durin42@gmail.com> parents: 8296 diff changeset	549 try:
10282 08a0f04b56bd many, many trivial check-code fixups Matt Mackall <mpm@selenic.com> parents: 9726 diff changeset	550 blocksize = 8192
14958 fd246aefedd3 keepalive: use safehasattr instead of hasattr Augie Fackler <durin42@gmail.com> parents: 14764 diff changeset	551 read = getattr(str, 'read', None)
fd246aefedd3 keepalive: use safehasattr instead of hasattr Augie Fackler <durin42@gmail.com> parents: 14764 diff changeset	552 if read is not None:
10282 08a0f04b56bd many, many trivial check-code fixups Matt Mackall <mpm@selenic.com> parents: 9726 diff changeset	553 if self.debuglevel > 0:
27616 0765d8423fbc keepalive: use print function Gregory Szorc <gregory.szorc@gmail.com> parents: 27507 diff changeset	554 print("sending a read()able")
14958 fd246aefedd3 keepalive: use safehasattr instead of hasattr Augie Fackler <durin42@gmail.com> parents: 14764 diff changeset	555 data = read(blocksize)
9726 430e59ff3437 keepalive: handle broken pipes gracefully during large POSTs Augie Fackler <durin42@gmail.com> parents: 8296 diff changeset	556 while data:
430e59ff3437 keepalive: handle broken pipes gracefully during large POSTs Augie Fackler <durin42@gmail.com> parents: 8296 diff changeset	557 self.sock.sendall(data)
14958 fd246aefedd3 keepalive: use safehasattr instead of hasattr Augie Fackler <durin42@gmail.com> parents: 14764 diff changeset	558 data = read(blocksize)
9726 430e59ff3437 keepalive: handle broken pipes gracefully during large POSTs Augie Fackler <durin42@gmail.com> parents: 8296 diff changeset	559 else:
430e59ff3437 keepalive: handle broken pipes gracefully during large POSTs Augie Fackler <durin42@gmail.com> parents: 8296 diff changeset	560 self.sock.sendall(str)
25660 328739ea70c3 global: mass rewrite to use modern exception syntax Gregory Szorc <gregory.szorc@gmail.com> parents: 22958 diff changeset	561 except socket.error as v:
9726 430e59ff3437 keepalive: handle broken pipes gracefully during large POSTs Augie Fackler <durin42@gmail.com> parents: 8296 diff changeset	562 reraise = True
430e59ff3437 keepalive: handle broken pipes gracefully during large POSTs Augie Fackler <durin42@gmail.com> parents: 8296 diff changeset	563 if v[0] == errno.EPIPE: # Broken pipe
430e59ff3437 keepalive: handle broken pipes gracefully during large POSTs Augie Fackler <durin42@gmail.com> parents: 8296 diff changeset	564 if self._HTTPConnection__state == httplib._CS_REQ_SENT:
430e59ff3437 keepalive: handle broken pipes gracefully during large POSTs Augie Fackler <durin42@gmail.com> parents: 8296 diff changeset	565 self._broken_pipe_resp = None
430e59ff3437 keepalive: handle broken pipes gracefully during large POSTs Augie Fackler <durin42@gmail.com> parents: 8296 diff changeset	566 self._broken_pipe_resp = self.getresponse()
430e59ff3437 keepalive: handle broken pipes gracefully during large POSTs Augie Fackler <durin42@gmail.com> parents: 8296 diff changeset	567 reraise = False
430e59ff3437 keepalive: handle broken pipes gracefully during large POSTs Augie Fackler <durin42@gmail.com> parents: 8296 diff changeset	568 self.close()
430e59ff3437 keepalive: handle broken pipes gracefully during large POSTs Augie Fackler <durin42@gmail.com> parents: 8296 diff changeset	569 if reraise:
430e59ff3437 keepalive: handle broken pipes gracefully during large POSTs Augie Fackler <durin42@gmail.com> parents: 8296 diff changeset	570 raise
430e59ff3437 keepalive: handle broken pipes gracefully during large POSTs Augie Fackler <durin42@gmail.com> parents: 8296 diff changeset	571
430e59ff3437 keepalive: handle broken pipes gracefully during large POSTs Augie Fackler <durin42@gmail.com> parents: 8296 diff changeset	572 def wrapgetresponse(cls):
430e59ff3437 keepalive: handle broken pipes gracefully during large POSTs Augie Fackler <durin42@gmail.com> parents: 8296 diff changeset	573 """Wraps getresponse in cls with a broken-pipe sane version.
430e59ff3437 keepalive: handle broken pipes gracefully during large POSTs Augie Fackler <durin42@gmail.com> parents: 8296 diff changeset	574 """
430e59ff3437 keepalive: handle broken pipes gracefully during large POSTs Augie Fackler <durin42@gmail.com> parents: 8296 diff changeset	575 def safegetresponse(self):
430e59ff3437 keepalive: handle broken pipes gracefully during large POSTs Augie Fackler <durin42@gmail.com> parents: 8296 diff changeset	576 # In safesend() we might set the _broken_pipe_resp
430e59ff3437 keepalive: handle broken pipes gracefully during large POSTs Augie Fackler <durin42@gmail.com> parents: 8296 diff changeset	577 # attribute, in which case the socket has already
430e59ff3437 keepalive: handle broken pipes gracefully during large POSTs Augie Fackler <durin42@gmail.com> parents: 8296 diff changeset	578 # been closed and we just need to give them the response
430e59ff3437 keepalive: handle broken pipes gracefully during large POSTs Augie Fackler <durin42@gmail.com> parents: 8296 diff changeset	579 # back. Otherwise, we use the normal response path.
430e59ff3437 keepalive: handle broken pipes gracefully during large POSTs Augie Fackler <durin42@gmail.com> parents: 8296 diff changeset	580 r = getattr(self, '_broken_pipe_resp', None)
430e59ff3437 keepalive: handle broken pipes gracefully during large POSTs Augie Fackler <durin42@gmail.com> parents: 8296 diff changeset	581 if r is not None:
430e59ff3437 keepalive: handle broken pipes gracefully during large POSTs Augie Fackler <durin42@gmail.com> parents: 8296 diff changeset	582 return r
430e59ff3437 keepalive: handle broken pipes gracefully during large POSTs Augie Fackler <durin42@gmail.com> parents: 8296 diff changeset	583 return cls.getresponse(self)
430e59ff3437 keepalive: handle broken pipes gracefully during large POSTs Augie Fackler <durin42@gmail.com> parents: 8296 diff changeset	584 safegetresponse.__doc__ = cls.getresponse.__doc__
430e59ff3437 keepalive: handle broken pipes gracefully during large POSTs Augie Fackler <durin42@gmail.com> parents: 8296 diff changeset	585 return safegetresponse
2435 ff2bac730b99 http client: support persistent connections. Vadim Gelfer <vadim.gelfer@gmail.com> parents: diff changeset	586
ff2bac730b99 http client: support persistent connections. Vadim Gelfer <vadim.gelfer@gmail.com> parents: diff changeset	587 class HTTPConnection(httplib.HTTPConnection):
ff2bac730b99 http client: support persistent connections. Vadim Gelfer <vadim.gelfer@gmail.com> parents: diff changeset	588 # use the modified response class
ff2bac730b99 http client: support persistent connections. Vadim Gelfer <vadim.gelfer@gmail.com> parents: diff changeset	589 response_class = HTTPResponse
9726 430e59ff3437 keepalive: handle broken pipes gracefully during large POSTs Augie Fackler <durin42@gmail.com> parents: 8296 diff changeset	590 send = safesend
430e59ff3437 keepalive: handle broken pipes gracefully during large POSTs Augie Fackler <durin42@gmail.com> parents: 8296 diff changeset	591 getresponse = wrapgetresponse(httplib.HTTPConnection)
430e59ff3437 keepalive: handle broken pipes gracefully during large POSTs Augie Fackler <durin42@gmail.com> parents: 8296 diff changeset	592
2600 c4325f0a9b91 clean up trailing white space. Vadim Gelfer <vadim.gelfer@gmail.com> parents: 2444 diff changeset	593
2435 ff2bac730b99 http client: support persistent connections. Vadim Gelfer <vadim.gelfer@gmail.com> parents: diff changeset	594 #########################################################################
ff2bac730b99 http client: support persistent connections. Vadim Gelfer <vadim.gelfer@gmail.com> parents: diff changeset	595 ##### TEST FUNCTIONS
ff2bac730b99 http client: support persistent connections. Vadim Gelfer <vadim.gelfer@gmail.com> parents: diff changeset	596 #########################################################################
ff2bac730b99 http client: support persistent connections. Vadim Gelfer <vadim.gelfer@gmail.com> parents: diff changeset	597
ff2bac730b99 http client: support persistent connections. Vadim Gelfer <vadim.gelfer@gmail.com> parents: diff changeset	598 def error_handler(url):
ff2bac730b99 http client: support persistent connections. Vadim Gelfer <vadim.gelfer@gmail.com> parents: diff changeset	599 global HANDLE_ERRORS
ff2bac730b99 http client: support persistent connections. Vadim Gelfer <vadim.gelfer@gmail.com> parents: diff changeset	600 orig = HANDLE_ERRORS
ff2bac730b99 http client: support persistent connections. Vadim Gelfer <vadim.gelfer@gmail.com> parents: diff changeset	601 keepalive_handler = HTTPHandler()
28883 032c4c2f802a pycompat: switch to util.urlreq/util.urlerr for py3 compat timeless <timeless@mozdev.org> parents: 28278 diff changeset	602 opener = urlreq.buildopener(keepalive_handler)
032c4c2f802a pycompat: switch to util.urlreq/util.urlerr for py3 compat timeless <timeless@mozdev.org> parents: 28278 diff changeset	603 urlreq.installopener(opener)
2435 ff2bac730b99 http client: support persistent connections. Vadim Gelfer <vadim.gelfer@gmail.com> parents: diff changeset	604 pos = {0: 'off', 1: 'on'}
ff2bac730b99 http client: support persistent connections. Vadim Gelfer <vadim.gelfer@gmail.com> parents: diff changeset	605 for i in (0, 1):
27616 0765d8423fbc keepalive: use print function Gregory Szorc <gregory.szorc@gmail.com> parents: 27507 diff changeset	606 print(" fancy error handling %s (HANDLE_ERRORS = %i)" % (pos[i], i))
2435 ff2bac730b99 http client: support persistent connections. Vadim Gelfer <vadim.gelfer@gmail.com> parents: diff changeset	607 HANDLE_ERRORS = i
ff2bac730b99 http client: support persistent connections. Vadim Gelfer <vadim.gelfer@gmail.com> parents: diff changeset	608 try:
28883 032c4c2f802a pycompat: switch to util.urlreq/util.urlerr for py3 compat timeless <timeless@mozdev.org> parents: 28278 diff changeset	609 fo = urlreq.urlopen(url)
7874 d812029cda85 cleanup: drop variables for unused return values Peter Arrenbrecht <peter.arrenbrecht@gmail.com> parents: 7868 diff changeset	610 fo.read()
2435 ff2bac730b99 http client: support persistent connections. Vadim Gelfer <vadim.gelfer@gmail.com> parents: diff changeset	611 fo.close()
10282 08a0f04b56bd many, many trivial check-code fixups Matt Mackall <mpm@selenic.com> parents: 9726 diff changeset	612 try:
08a0f04b56bd many, many trivial check-code fixups Matt Mackall <mpm@selenic.com> parents: 9726 diff changeset	613 status, reason = fo.status, fo.reason
08a0f04b56bd many, many trivial check-code fixups Matt Mackall <mpm@selenic.com> parents: 9726 diff changeset	614 except AttributeError:
08a0f04b56bd many, many trivial check-code fixups Matt Mackall <mpm@selenic.com> parents: 9726 diff changeset	615 status, reason = None, None
25660 328739ea70c3 global: mass rewrite to use modern exception syntax Gregory Szorc <gregory.szorc@gmail.com> parents: 22958 diff changeset	616 except IOError as e:
27616 0765d8423fbc keepalive: use print function Gregory Szorc <gregory.szorc@gmail.com> parents: 27507 diff changeset	617 print(" EXCEPTION: %s" % e)
2435 ff2bac730b99 http client: support persistent connections. Vadim Gelfer <vadim.gelfer@gmail.com> parents: diff changeset	618 raise
ff2bac730b99 http client: support persistent connections. Vadim Gelfer <vadim.gelfer@gmail.com> parents: diff changeset	619 else:
27616 0765d8423fbc keepalive: use print function Gregory Szorc <gregory.szorc@gmail.com> parents: 27507 diff changeset	620 print(" status = %s, reason = %s" % (status, reason))
2435 ff2bac730b99 http client: support persistent connections. Vadim Gelfer <vadim.gelfer@gmail.com> parents: diff changeset	621 HANDLE_ERRORS = orig
ff2bac730b99 http client: support persistent connections. Vadim Gelfer <vadim.gelfer@gmail.com> parents: diff changeset	622 hosts = keepalive_handler.open_connections()
27616 0765d8423fbc keepalive: use print function Gregory Szorc <gregory.szorc@gmail.com> parents: 27507 diff changeset	623 print("open connections:", hosts)
2435 ff2bac730b99 http client: support persistent connections. Vadim Gelfer <vadim.gelfer@gmail.com> parents: diff changeset	624 keepalive_handler.close_all()
ff2bac730b99 http client: support persistent connections. Vadim Gelfer <vadim.gelfer@gmail.com> parents: diff changeset	625
ff2bac730b99 http client: support persistent connections. Vadim Gelfer <vadim.gelfer@gmail.com> parents: diff changeset	626 def continuity(url):
29341 0d83ad967bf8 cleanup: replace uses of util.(md5\|sha1\|sha256\|sha512) with hashlib.\1 Augie Fackler <raf@durin42.com> parents: 28883 diff changeset	627 md5 = hashlib.md5
2435 ff2bac730b99 http client: support persistent connections. Vadim Gelfer <vadim.gelfer@gmail.com> parents: diff changeset	628 format = '%25s: %s'
2600 c4325f0a9b91 clean up trailing white space. Vadim Gelfer <vadim.gelfer@gmail.com> parents: 2444 diff changeset	629
2435 ff2bac730b99 http client: support persistent connections. Vadim Gelfer <vadim.gelfer@gmail.com> parents: diff changeset	630 # first fetch the file with the normal http handler
28883 032c4c2f802a pycompat: switch to util.urlreq/util.urlerr for py3 compat timeless <timeless@mozdev.org> parents: 28278 diff changeset	631 opener = urlreq.buildopener()
032c4c2f802a pycompat: switch to util.urlreq/util.urlerr for py3 compat timeless <timeless@mozdev.org> parents: 28278 diff changeset	632 urlreq.installopener(opener)
032c4c2f802a pycompat: switch to util.urlreq/util.urlerr for py3 compat timeless <timeless@mozdev.org> parents: 28278 diff changeset	633 fo = urlreq.urlopen(url)
2435 ff2bac730b99 http client: support persistent connections. Vadim Gelfer <vadim.gelfer@gmail.com> parents: diff changeset	634 foo = fo.read()
ff2bac730b99 http client: support persistent connections. Vadim Gelfer <vadim.gelfer@gmail.com> parents: diff changeset	635 fo.close()
22505 232d437af120 keepalive: fix how md5 is used Mike Hommey <mh@glandium.org> parents: 19872 diff changeset	636 m = md5(foo)
27616 0765d8423fbc keepalive: use print function Gregory Szorc <gregory.szorc@gmail.com> parents: 27507 diff changeset	637 print(format % ('normal urllib', m.hexdigest()))
2435 ff2bac730b99 http client: support persistent connections. Vadim Gelfer <vadim.gelfer@gmail.com> parents: diff changeset	638
ff2bac730b99 http client: support persistent connections. Vadim Gelfer <vadim.gelfer@gmail.com> parents: diff changeset	639 # now install the keepalive handler and try again
28883 032c4c2f802a pycompat: switch to util.urlreq/util.urlerr for py3 compat timeless <timeless@mozdev.org> parents: 28278 diff changeset	640 opener = urlreq.buildopener(HTTPHandler())
032c4c2f802a pycompat: switch to util.urlreq/util.urlerr for py3 compat timeless <timeless@mozdev.org> parents: 28278 diff changeset	641 urlreq.installopener(opener)
2435 ff2bac730b99 http client: support persistent connections. Vadim Gelfer <vadim.gelfer@gmail.com> parents: diff changeset	642
28883 032c4c2f802a pycompat: switch to util.urlreq/util.urlerr for py3 compat timeless <timeless@mozdev.org> parents: 28278 diff changeset	643 fo = urlreq.urlopen(url)
2435 ff2bac730b99 http client: support persistent connections. Vadim Gelfer <vadim.gelfer@gmail.com> parents: diff changeset	644 foo = fo.read()
ff2bac730b99 http client: support persistent connections. Vadim Gelfer <vadim.gelfer@gmail.com> parents: diff changeset	645 fo.close()
22505 232d437af120 keepalive: fix how md5 is used Mike Hommey <mh@glandium.org> parents: 19872 diff changeset	646 m = md5(foo)
27616 0765d8423fbc keepalive: use print function Gregory Szorc <gregory.szorc@gmail.com> parents: 27507 diff changeset	647 print(format % ('keepalive read', m.hexdigest()))
2435 ff2bac730b99 http client: support persistent connections. Vadim Gelfer <vadim.gelfer@gmail.com> parents: diff changeset	648
28883 032c4c2f802a pycompat: switch to util.urlreq/util.urlerr for py3 compat timeless <timeless@mozdev.org> parents: 28278 diff changeset	649 fo = urlreq.urlopen(url)
2435 ff2bac730b99 http client: support persistent connections. Vadim Gelfer <vadim.gelfer@gmail.com> parents: diff changeset	650 foo = ''
14494 1ffeeb91c55d check-code: flag 0/1 used as constant Boolean expression Martin Geisler <mg@lazybytes.net> parents: 10394 diff changeset	651 while True:
2435 ff2bac730b99 http client: support persistent connections. Vadim Gelfer <vadim.gelfer@gmail.com> parents: diff changeset	652 f = fo.readline()
10282 08a0f04b56bd many, many trivial check-code fixups Matt Mackall <mpm@selenic.com> parents: 9726 diff changeset	653 if f:
08a0f04b56bd many, many trivial check-code fixups Matt Mackall <mpm@selenic.com> parents: 9726 diff changeset	654 foo = foo + f
2435 ff2bac730b99 http client: support persistent connections. Vadim Gelfer <vadim.gelfer@gmail.com> parents: diff changeset	655 else: break
ff2bac730b99 http client: support persistent connections. Vadim Gelfer <vadim.gelfer@gmail.com> parents: diff changeset	656 fo.close()
22505 232d437af120 keepalive: fix how md5 is used Mike Hommey <mh@glandium.org> parents: 19872 diff changeset	657 m = md5(foo)
27616 0765d8423fbc keepalive: use print function Gregory Szorc <gregory.szorc@gmail.com> parents: 27507 diff changeset	658 print(format % ('keepalive readline', m.hexdigest()))
2435 ff2bac730b99 http client: support persistent connections. Vadim Gelfer <vadim.gelfer@gmail.com> parents: diff changeset	659
ff2bac730b99 http client: support persistent connections. Vadim Gelfer <vadim.gelfer@gmail.com> parents: diff changeset	660 def comp(N, url):
27616 0765d8423fbc keepalive: use print function Gregory Szorc <gregory.szorc@gmail.com> parents: 27507 diff changeset	661 print(' making %i connections to:\n %s' % (N, url))
2435 ff2bac730b99 http client: support persistent connections. Vadim Gelfer <vadim.gelfer@gmail.com> parents: diff changeset	662
30473 39d13b8c101d py3: bulk replace sys.stdin/out/err by util's Yuya Nishihara <yuya@tcha.org> parents: 30463 diff changeset	663 util.stdout.write(' first using the normal urllib handlers')
2435 ff2bac730b99 http client: support persistent connections. Vadim Gelfer <vadim.gelfer@gmail.com> parents: diff changeset	664 # first use normal opener
28883 032c4c2f802a pycompat: switch to util.urlreq/util.urlerr for py3 compat timeless <timeless@mozdev.org> parents: 28278 diff changeset	665 opener = urlreq.buildopener()
032c4c2f802a pycompat: switch to util.urlreq/util.urlerr for py3 compat timeless <timeless@mozdev.org> parents: 28278 diff changeset	666 urlreq.installopener(opener)
2435 ff2bac730b99 http client: support persistent connections. Vadim Gelfer <vadim.gelfer@gmail.com> parents: diff changeset	667 t1 = fetch(N, url)
27616 0765d8423fbc keepalive: use print function Gregory Szorc <gregory.szorc@gmail.com> parents: 27507 diff changeset	668 print(' TIME: %.3f s' % t1)
2435 ff2bac730b99 http client: support persistent connections. Vadim Gelfer <vadim.gelfer@gmail.com> parents: diff changeset	669
30473 39d13b8c101d py3: bulk replace sys.stdin/out/err by util's Yuya Nishihara <yuya@tcha.org> parents: 30463 diff changeset	670 util.stdout.write(' now using the keepalive handler ')
2435 ff2bac730b99 http client: support persistent connections. Vadim Gelfer <vadim.gelfer@gmail.com> parents: diff changeset	671 # now install the keepalive handler and try again
28883 032c4c2f802a pycompat: switch to util.urlreq/util.urlerr for py3 compat timeless <timeless@mozdev.org> parents: 28278 diff changeset	672 opener = urlreq.buildopener(HTTPHandler())
032c4c2f802a pycompat: switch to util.urlreq/util.urlerr for py3 compat timeless <timeless@mozdev.org> parents: 28278 diff changeset	673 urlreq.installopener(opener)
2435 ff2bac730b99 http client: support persistent connections. Vadim Gelfer <vadim.gelfer@gmail.com> parents: diff changeset	674 t2 = fetch(N, url)
27616 0765d8423fbc keepalive: use print function Gregory Szorc <gregory.szorc@gmail.com> parents: 27507 diff changeset	675 print(' TIME: %.3f s' % t2)
0765d8423fbc keepalive: use print function Gregory Szorc <gregory.szorc@gmail.com> parents: 27507 diff changeset	676 print(' improvement factor: %.2f' % (t1 / t2))
2600 c4325f0a9b91 clean up trailing white space. Vadim Gelfer <vadim.gelfer@gmail.com> parents: 2444 diff changeset	677
2435 ff2bac730b99 http client: support persistent connections. Vadim Gelfer <vadim.gelfer@gmail.com> parents: diff changeset	678 def fetch(N, url, delay=0):
ff2bac730b99 http client: support persistent connections. Vadim Gelfer <vadim.gelfer@gmail.com> parents: diff changeset	679 import time
ff2bac730b99 http client: support persistent connections. Vadim Gelfer <vadim.gelfer@gmail.com> parents: diff changeset	680 lens = []
ff2bac730b99 http client: support persistent connections. Vadim Gelfer <vadim.gelfer@gmail.com> parents: diff changeset	681 starttime = time.time()
ff2bac730b99 http client: support persistent connections. Vadim Gelfer <vadim.gelfer@gmail.com> parents: diff changeset	682 for i in range(N):
10282 08a0f04b56bd many, many trivial check-code fixups Matt Mackall <mpm@selenic.com> parents: 9726 diff changeset	683 if delay and i > 0:
08a0f04b56bd many, many trivial check-code fixups Matt Mackall <mpm@selenic.com> parents: 9726 diff changeset	684 time.sleep(delay)
28883 032c4c2f802a pycompat: switch to util.urlreq/util.urlerr for py3 compat timeless <timeless@mozdev.org> parents: 28278 diff changeset	685 fo = urlreq.urlopen(url)
2435 ff2bac730b99 http client: support persistent connections. Vadim Gelfer <vadim.gelfer@gmail.com> parents: diff changeset	686 foo = fo.read()
ff2bac730b99 http client: support persistent connections. Vadim Gelfer <vadim.gelfer@gmail.com> parents: diff changeset	687 fo.close()
ff2bac730b99 http client: support persistent connections. Vadim Gelfer <vadim.gelfer@gmail.com> parents: diff changeset	688 lens.append(len(foo))
ff2bac730b99 http client: support persistent connections. Vadim Gelfer <vadim.gelfer@gmail.com> parents: diff changeset	689 diff = time.time() - starttime
ff2bac730b99 http client: support persistent connections. Vadim Gelfer <vadim.gelfer@gmail.com> parents: diff changeset	690
ff2bac730b99 http client: support persistent connections. Vadim Gelfer <vadim.gelfer@gmail.com> parents: diff changeset	691 j = 0
ff2bac730b99 http client: support persistent connections. Vadim Gelfer <vadim.gelfer@gmail.com> parents: diff changeset	692 for i in lens[1:]:
ff2bac730b99 http client: support persistent connections. Vadim Gelfer <vadim.gelfer@gmail.com> parents: diff changeset	693 j = j + 1
ff2bac730b99 http client: support persistent connections. Vadim Gelfer <vadim.gelfer@gmail.com> parents: diff changeset	694 if not i == lens[0]:
27616 0765d8423fbc keepalive: use print function Gregory Szorc <gregory.szorc@gmail.com> parents: 27507 diff changeset	695 print("WARNING: inconsistent length on read %i: %i" % (j, i))
2435 ff2bac730b99 http client: support persistent connections. Vadim Gelfer <vadim.gelfer@gmail.com> parents: diff changeset	696
ff2bac730b99 http client: support persistent connections. Vadim Gelfer <vadim.gelfer@gmail.com> parents: diff changeset	697 return diff
ff2bac730b99 http client: support persistent connections. Vadim Gelfer <vadim.gelfer@gmail.com> parents: diff changeset	698
ff2bac730b99 http client: support persistent connections. Vadim Gelfer <vadim.gelfer@gmail.com> parents: diff changeset	699 def test_timeout(url):
ff2bac730b99 http client: support persistent connections. Vadim Gelfer <vadim.gelfer@gmail.com> parents: diff changeset	700 global DEBUG
ff2bac730b99 http client: support persistent connections. Vadim Gelfer <vadim.gelfer@gmail.com> parents: diff changeset	701 dbbackup = DEBUG
14764 a7d5816087a9 classes: fix class style problems found by b071cd58af50 Thomas Arendsen Hein <thomas@intevation.de> parents: 14494 diff changeset	702 class FakeLogger(object):
10282 08a0f04b56bd many, many trivial check-code fixups Matt Mackall <mpm@selenic.com> parents: 9726 diff changeset	703 def debug(self, msg, *args):
27616 0765d8423fbc keepalive: use print function Gregory Szorc <gregory.szorc@gmail.com> parents: 27507 diff changeset	704 print(msg % args)
2435 ff2bac730b99 http client: support persistent connections. Vadim Gelfer <vadim.gelfer@gmail.com> parents: diff changeset	705 info = warning = error = debug
ff2bac730b99 http client: support persistent connections. Vadim Gelfer <vadim.gelfer@gmail.com> parents: diff changeset	706 DEBUG = FakeLogger()
27616 0765d8423fbc keepalive: use print function Gregory Szorc <gregory.szorc@gmail.com> parents: 27507 diff changeset	707 print(" fetching the file to establish a connection")
28883 032c4c2f802a pycompat: switch to util.urlreq/util.urlerr for py3 compat timeless <timeless@mozdev.org> parents: 28278 diff changeset	708 fo = urlreq.urlopen(url)
2435 ff2bac730b99 http client: support persistent connections. Vadim Gelfer <vadim.gelfer@gmail.com> parents: diff changeset	709 data1 = fo.read()
ff2bac730b99 http client: support persistent connections. Vadim Gelfer <vadim.gelfer@gmail.com> parents: diff changeset	710 fo.close()
2600 c4325f0a9b91 clean up trailing white space. Vadim Gelfer <vadim.gelfer@gmail.com> parents: 2444 diff changeset	711
2435 ff2bac730b99 http client: support persistent connections. Vadim Gelfer <vadim.gelfer@gmail.com> parents: diff changeset	712 i = 20
27616 0765d8423fbc keepalive: use print function Gregory Szorc <gregory.szorc@gmail.com> parents: 27507 diff changeset	713 print(" waiting %i seconds for the server to close the connection" % i)
2435 ff2bac730b99 http client: support persistent connections. Vadim Gelfer <vadim.gelfer@gmail.com> parents: diff changeset	714 while i > 0:
30473 39d13b8c101d py3: bulk replace sys.stdin/out/err by util's Yuya Nishihara <yuya@tcha.org> parents: 30463 diff changeset	715 util.stdout.write('\r %2i' % i)
39d13b8c101d py3: bulk replace sys.stdin/out/err by util's Yuya Nishihara <yuya@tcha.org> parents: 30463 diff changeset	716 util.stdout.flush()
2435 ff2bac730b99 http client: support persistent connections. Vadim Gelfer <vadim.gelfer@gmail.com> parents: diff changeset	717 time.sleep(1)
ff2bac730b99 http client: support persistent connections. Vadim Gelfer <vadim.gelfer@gmail.com> parents: diff changeset	718 i -= 1
30473 39d13b8c101d py3: bulk replace sys.stdin/out/err by util's Yuya Nishihara <yuya@tcha.org> parents: 30463 diff changeset	719 util.stderr.write('\r')
2435 ff2bac730b99 http client: support persistent connections. Vadim Gelfer <vadim.gelfer@gmail.com> parents: diff changeset	720
27616 0765d8423fbc keepalive: use print function Gregory Szorc <gregory.szorc@gmail.com> parents: 27507 diff changeset	721 print(" fetching the file a second time")
28883 032c4c2f802a pycompat: switch to util.urlreq/util.urlerr for py3 compat timeless <timeless@mozdev.org> parents: 28278 diff changeset	722 fo = urlreq.urlopen(url)
2435 ff2bac730b99 http client: support persistent connections. Vadim Gelfer <vadim.gelfer@gmail.com> parents: diff changeset	723 data2 = fo.read()
ff2bac730b99 http client: support persistent connections. Vadim Gelfer <vadim.gelfer@gmail.com> parents: diff changeset	724 fo.close()
ff2bac730b99 http client: support persistent connections. Vadim Gelfer <vadim.gelfer@gmail.com> parents: diff changeset	725
ff2bac730b99 http client: support persistent connections. Vadim Gelfer <vadim.gelfer@gmail.com> parents: diff changeset	726 if data1 == data2:
27616 0765d8423fbc keepalive: use print function Gregory Szorc <gregory.szorc@gmail.com> parents: 27507 diff changeset	727 print(' data are identical')
2435 ff2bac730b99 http client: support persistent connections. Vadim Gelfer <vadim.gelfer@gmail.com> parents: diff changeset	728 else:
27616 0765d8423fbc keepalive: use print function Gregory Szorc <gregory.szorc@gmail.com> parents: 27507 diff changeset	729 print(' ERROR: DATA DIFFER')
2435 ff2bac730b99 http client: support persistent connections. Vadim Gelfer <vadim.gelfer@gmail.com> parents: diff changeset	730
ff2bac730b99 http client: support persistent connections. Vadim Gelfer <vadim.gelfer@gmail.com> parents: diff changeset	731 DEBUG = dbbackup
ff2bac730b99 http client: support persistent connections. Vadim Gelfer <vadim.gelfer@gmail.com> parents: diff changeset	732
2600 c4325f0a9b91 clean up trailing white space. Vadim Gelfer <vadim.gelfer@gmail.com> parents: 2444 diff changeset	733
2435 ff2bac730b99 http client: support persistent connections. Vadim Gelfer <vadim.gelfer@gmail.com> parents: diff changeset	734 def test(url, N=10):
27616 0765d8423fbc keepalive: use print function Gregory Szorc <gregory.szorc@gmail.com> parents: 27507 diff changeset	735 print("checking error handler (do this on a non-200)")
2435 ff2bac730b99 http client: support persistent connections. Vadim Gelfer <vadim.gelfer@gmail.com> parents: diff changeset	736 try: error_handler(url)
7875 553aa0cbeab6 cleanup: drop unused assignments Peter Arrenbrecht <peter.arrenbrecht@gmail.com> parents: 7874 diff changeset	737 except IOError:
27616 0765d8423fbc keepalive: use print function Gregory Szorc <gregory.szorc@gmail.com> parents: 27507 diff changeset	738 print("exiting - exception will prevent further tests")
2435 ff2bac730b99 http client: support persistent connections. Vadim Gelfer <vadim.gelfer@gmail.com> parents: diff changeset	739 sys.exit()
27616 0765d8423fbc keepalive: use print function Gregory Szorc <gregory.szorc@gmail.com> parents: 27507 diff changeset	740 print('')
0765d8423fbc keepalive: use print function Gregory Szorc <gregory.szorc@gmail.com> parents: 27507 diff changeset	741 print("performing continuity test (making sure stuff isn't corrupted)")
2435 ff2bac730b99 http client: support persistent connections. Vadim Gelfer <vadim.gelfer@gmail.com> parents: diff changeset	742 continuity(url)
27616 0765d8423fbc keepalive: use print function Gregory Szorc <gregory.szorc@gmail.com> parents: 27507 diff changeset	743 print('')
0765d8423fbc keepalive: use print function Gregory Szorc <gregory.szorc@gmail.com> parents: 27507 diff changeset	744 print("performing speed comparison")
2435 ff2bac730b99 http client: support persistent connections. Vadim Gelfer <vadim.gelfer@gmail.com> parents: diff changeset	745 comp(N, url)
27616 0765d8423fbc keepalive: use print function Gregory Szorc <gregory.szorc@gmail.com> parents: 27507 diff changeset	746 print('')
0765d8423fbc keepalive: use print function Gregory Szorc <gregory.szorc@gmail.com> parents: 27507 diff changeset	747 print("performing dropped-connection check")
2435 ff2bac730b99 http client: support persistent connections. Vadim Gelfer <vadim.gelfer@gmail.com> parents: diff changeset	748 test_timeout(url)
2600 c4325f0a9b91 clean up trailing white space. Vadim Gelfer <vadim.gelfer@gmail.com> parents: 2444 diff changeset	749
2435 ff2bac730b99 http client: support persistent connections. Vadim Gelfer <vadim.gelfer@gmail.com> parents: diff changeset	750 if __name__ == '__main__':
ff2bac730b99 http client: support persistent connections. Vadim Gelfer <vadim.gelfer@gmail.com> parents: diff changeset	751 import time
ff2bac730b99 http client: support persistent connections. Vadim Gelfer <vadim.gelfer@gmail.com> parents: diff changeset	752 try:
ff2bac730b99 http client: support persistent connections. Vadim Gelfer <vadim.gelfer@gmail.com> parents: diff changeset	753 N = int(sys.argv[1])
ff2bac730b99 http client: support persistent connections. Vadim Gelfer <vadim.gelfer@gmail.com> parents: diff changeset	754 url = sys.argv[2]
16688 cfb6682961b8 cleanup: replace naked excepts with more specific ones Brodie Rao <brodie@sf.io> parents: 16687 diff changeset	755 except (IndexError, ValueError):
27616 0765d8423fbc keepalive: use print function Gregory Szorc <gregory.szorc@gmail.com> parents: 27507 diff changeset	756 print("%s <integer> <url>" % sys.argv[0])
2435 ff2bac730b99 http client: support persistent connections. Vadim Gelfer <vadim.gelfer@gmail.com> parents: diff changeset	757 else:
ff2bac730b99 http client: support persistent connections. Vadim Gelfer <vadim.gelfer@gmail.com> parents: diff changeset	758 test(url, N)

Mercurial > public > mercurial-scm > hg

annotate mercurial/keepalive.py @ 30473:39d13b8c101d