mercurial-scm/hg: mercurial/httpclient/__init_

annotate mercurial/httpclient/init.py @ 26534:e4f27fb65da7

merge: drop special parent assignment in the obsolete case We can safely drop this because the very same assignment is enforcement later in the function. Dropping it will make it simpler to extract the default destination logic in its own function.

author	Pierre-Yves David <pierre-yves.david@fb.com>
date	Mon, 05 Oct 2015 01:47:33 -0700
parents	328739ea70c3
children	1ad9da968a2e

rev	line source
14243 861f28212398 Import new http library as mercurial.httpclient. Augie Fackler <durin42@gmail.com> parents: diff changeset	1 # Copyright 2010, Google Inc.
861f28212398 Import new http library as mercurial.httpclient. Augie Fackler <durin42@gmail.com> parents: diff changeset	2 # All rights reserved.
861f28212398 Import new http library as mercurial.httpclient. Augie Fackler <durin42@gmail.com> parents: diff changeset	3 #
861f28212398 Import new http library as mercurial.httpclient. Augie Fackler <durin42@gmail.com> parents: diff changeset	4 # Redistribution and use in source and binary forms, with or without
861f28212398 Import new http library as mercurial.httpclient. Augie Fackler <durin42@gmail.com> parents: diff changeset	5 # modification, are permitted provided that the following conditions are
861f28212398 Import new http library as mercurial.httpclient. Augie Fackler <durin42@gmail.com> parents: diff changeset	6 # met:
861f28212398 Import new http library as mercurial.httpclient. Augie Fackler <durin42@gmail.com> parents: diff changeset	7 #
861f28212398 Import new http library as mercurial.httpclient. Augie Fackler <durin42@gmail.com> parents: diff changeset	8 # * Redistributions of source code must retain the above copyright
861f28212398 Import new http library as mercurial.httpclient. Augie Fackler <durin42@gmail.com> parents: diff changeset	9 # notice, this list of conditions and the following disclaimer.
861f28212398 Import new http library as mercurial.httpclient. Augie Fackler <durin42@gmail.com> parents: diff changeset	10 # * Redistributions in binary form must reproduce the above
861f28212398 Import new http library as mercurial.httpclient. Augie Fackler <durin42@gmail.com> parents: diff changeset	11 # copyright notice, this list of conditions and the following disclaimer
861f28212398 Import new http library as mercurial.httpclient. Augie Fackler <durin42@gmail.com> parents: diff changeset	12 # in the documentation and/or other materials provided with the
861f28212398 Import new http library as mercurial.httpclient. Augie Fackler <durin42@gmail.com> parents: diff changeset	13 # distribution.
861f28212398 Import new http library as mercurial.httpclient. Augie Fackler <durin42@gmail.com> parents: diff changeset	14 # * Neither the name of Google Inc. nor the names of its
861f28212398 Import new http library as mercurial.httpclient. Augie Fackler <durin42@gmail.com> parents: diff changeset	15 # contributors may be used to endorse or promote products derived from
861f28212398 Import new http library as mercurial.httpclient. Augie Fackler <durin42@gmail.com> parents: diff changeset	16 # this software without specific prior written permission.
861f28212398 Import new http library as mercurial.httpclient. Augie Fackler <durin42@gmail.com> parents: diff changeset	17
861f28212398 Import new http library as mercurial.httpclient. Augie Fackler <durin42@gmail.com> parents: diff changeset	18 # THIS SOFTWARE IS PROVIDED BY THE COPYRIGHT HOLDERS AND CONTRIBUTORS
861f28212398 Import new http library as mercurial.httpclient. Augie Fackler <durin42@gmail.com> parents: diff changeset	19 # "AS IS" AND ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT
861f28212398 Import new http library as mercurial.httpclient. Augie Fackler <durin42@gmail.com> parents: diff changeset	20 # LIMITED TO, THE IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR
861f28212398 Import new http library as mercurial.httpclient. Augie Fackler <durin42@gmail.com> parents: diff changeset	21 # A PARTICULAR PURPOSE ARE DISCLAIMED. IN NO EVENT SHALL THE COPYRIGHT
861f28212398 Import new http library as mercurial.httpclient. Augie Fackler <durin42@gmail.com> parents: diff changeset	22 # OWNER OR CONTRIBUTORS BE LIABLE FOR ANY DIRECT, INDIRECT, INCIDENTAL,
861f28212398 Import new http library as mercurial.httpclient. Augie Fackler <durin42@gmail.com> parents: diff changeset	23 # SPECIAL, EXEMPLARY, OR CONSEQUENTIAL DAMAGES (INCLUDING, BUT NOT
861f28212398 Import new http library as mercurial.httpclient. Augie Fackler <durin42@gmail.com> parents: diff changeset	24 # LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS OR SERVICES; LOSS OF USE,
861f28212398 Import new http library as mercurial.httpclient. Augie Fackler <durin42@gmail.com> parents: diff changeset	25 # DATA, OR PROFITS; OR BUSINESS INTERRUPTION) HOWEVER CAUSED AND ON ANY
861f28212398 Import new http library as mercurial.httpclient. Augie Fackler <durin42@gmail.com> parents: diff changeset	26 # THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY, OR TORT
861f28212398 Import new http library as mercurial.httpclient. Augie Fackler <durin42@gmail.com> parents: diff changeset	27 # (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF THE USE
861f28212398 Import new http library as mercurial.httpclient. Augie Fackler <durin42@gmail.com> parents: diff changeset	28 # OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE.
861f28212398 Import new http library as mercurial.httpclient. Augie Fackler <durin42@gmail.com> parents: diff changeset	29 """Improved HTTP/1.1 client library
861f28212398 Import new http library as mercurial.httpclient. Augie Fackler <durin42@gmail.com> parents: diff changeset	30
861f28212398 Import new http library as mercurial.httpclient. Augie Fackler <durin42@gmail.com> parents: diff changeset	31 This library contains an HTTPConnection which is similar to the one in
861f28212398 Import new http library as mercurial.httpclient. Augie Fackler <durin42@gmail.com> parents: diff changeset	32 httplib, but has several additional features:
861f28212398 Import new http library as mercurial.httpclient. Augie Fackler <durin42@gmail.com> parents: diff changeset	33
861f28212398 Import new http library as mercurial.httpclient. Augie Fackler <durin42@gmail.com> parents: diff changeset	34 * supports keepalives natively
861f28212398 Import new http library as mercurial.httpclient. Augie Fackler <durin42@gmail.com> parents: diff changeset	35 * uses select() to block for incoming data
861f28212398 Import new http library as mercurial.httpclient. Augie Fackler <durin42@gmail.com> parents: diff changeset	36 * notices when the server responds early to a request
861f28212398 Import new http library as mercurial.httpclient. Augie Fackler <durin42@gmail.com> parents: diff changeset	37 * implements ssl inline instead of in a different class
861f28212398 Import new http library as mercurial.httpclient. Augie Fackler <durin42@gmail.com> parents: diff changeset	38 """
861f28212398 Import new http library as mercurial.httpclient. Augie Fackler <durin42@gmail.com> parents: diff changeset	39
19182 fae47ecaa952 httpclient: upgrade to fe8c09e4db64 of httpplus Augie Fackler <raf@durin42.com> parents: 19038 diff changeset	40 # Many functions in this file have too many arguments.
fae47ecaa952 httpclient: upgrade to fe8c09e4db64 of httpplus Augie Fackler <raf@durin42.com> parents: 19038 diff changeset	41 # pylint: disable=R0913
fae47ecaa952 httpclient: upgrade to fe8c09e4db64 of httpplus Augie Fackler <raf@durin42.com> parents: 19038 diff changeset	42
14243 861f28212398 Import new http library as mercurial.httpclient. Augie Fackler <durin42@gmail.com> parents: diff changeset	43 import cStringIO
861f28212398 Import new http library as mercurial.httpclient. Augie Fackler <durin42@gmail.com> parents: diff changeset	44 import errno
861f28212398 Import new http library as mercurial.httpclient. Augie Fackler <durin42@gmail.com> parents: diff changeset	45 import httplib
861f28212398 Import new http library as mercurial.httpclient. Augie Fackler <durin42@gmail.com> parents: diff changeset	46 import logging
861f28212398 Import new http library as mercurial.httpclient. Augie Fackler <durin42@gmail.com> parents: diff changeset	47 import rfc822
861f28212398 Import new http library as mercurial.httpclient. Augie Fackler <durin42@gmail.com> parents: diff changeset	48 import select
861f28212398 Import new http library as mercurial.httpclient. Augie Fackler <durin42@gmail.com> parents: diff changeset	49 import socket
861f28212398 Import new http library as mercurial.httpclient. Augie Fackler <durin42@gmail.com> parents: diff changeset	50
16643 24dbef11f477 httpclient: update to revision 892730fe7f46 of httpplus Augie Fackler <raf@durin42.com> parents: 15218 diff changeset	51 import _readers
14243 861f28212398 Import new http library as mercurial.httpclient. Augie Fackler <durin42@gmail.com> parents: diff changeset	52 import socketutil
861f28212398 Import new http library as mercurial.httpclient. Augie Fackler <durin42@gmail.com> parents: diff changeset	53
861f28212398 Import new http library as mercurial.httpclient. Augie Fackler <durin42@gmail.com> parents: diff changeset	54 logger = logging.getLogger(__name__)
861f28212398 Import new http library as mercurial.httpclient. Augie Fackler <durin42@gmail.com> parents: diff changeset	55
861f28212398 Import new http library as mercurial.httpclient. Augie Fackler <durin42@gmail.com> parents: diff changeset	56 __all__ = ['HTTPConnection', 'HTTPResponse']
861f28212398 Import new http library as mercurial.httpclient. Augie Fackler <durin42@gmail.com> parents: diff changeset	57
861f28212398 Import new http library as mercurial.httpclient. Augie Fackler <durin42@gmail.com> parents: diff changeset	58 HTTP_VER_1_0 = 'HTTP/1.0'
861f28212398 Import new http library as mercurial.httpclient. Augie Fackler <durin42@gmail.com> parents: diff changeset	59 HTTP_VER_1_1 = 'HTTP/1.1'
861f28212398 Import new http library as mercurial.httpclient. Augie Fackler <durin42@gmail.com> parents: diff changeset	60
861f28212398 Import new http library as mercurial.httpclient. Augie Fackler <durin42@gmail.com> parents: diff changeset	61 OUTGOING_BUFFER_SIZE = 1 << 15
861f28212398 Import new http library as mercurial.httpclient. Augie Fackler <durin42@gmail.com> parents: diff changeset	62 INCOMING_BUFFER_SIZE = 1 << 20
861f28212398 Import new http library as mercurial.httpclient. Augie Fackler <durin42@gmail.com> parents: diff changeset	63
861f28212398 Import new http library as mercurial.httpclient. Augie Fackler <durin42@gmail.com> parents: diff changeset	64 HDR_ACCEPT_ENCODING = 'accept-encoding'
861f28212398 Import new http library as mercurial.httpclient. Augie Fackler <durin42@gmail.com> parents: diff changeset	65 HDR_CONNECTION_CTRL = 'connection'
861f28212398 Import new http library as mercurial.httpclient. Augie Fackler <durin42@gmail.com> parents: diff changeset	66 HDR_CONTENT_LENGTH = 'content-length'
861f28212398 Import new http library as mercurial.httpclient. Augie Fackler <durin42@gmail.com> parents: diff changeset	67 HDR_XFER_ENCODING = 'transfer-encoding'
861f28212398 Import new http library as mercurial.httpclient. Augie Fackler <durin42@gmail.com> parents: diff changeset	68
861f28212398 Import new http library as mercurial.httpclient. Augie Fackler <durin42@gmail.com> parents: diff changeset	69 XFER_ENCODING_CHUNKED = 'chunked'
861f28212398 Import new http library as mercurial.httpclient. Augie Fackler <durin42@gmail.com> parents: diff changeset	70
861f28212398 Import new http library as mercurial.httpclient. Augie Fackler <durin42@gmail.com> parents: diff changeset	71 CONNECTION_CLOSE = 'close'
861f28212398 Import new http library as mercurial.httpclient. Augie Fackler <durin42@gmail.com> parents: diff changeset	72
861f28212398 Import new http library as mercurial.httpclient. Augie Fackler <durin42@gmail.com> parents: diff changeset	73 EOL = '\r\n'
861f28212398 Import new http library as mercurial.httpclient. Augie Fackler <durin42@gmail.com> parents: diff changeset	74 _END_HEADERS = EOL * 2
861f28212398 Import new http library as mercurial.httpclient. Augie Fackler <durin42@gmail.com> parents: diff changeset	75
861f28212398 Import new http library as mercurial.httpclient. Augie Fackler <durin42@gmail.com> parents: diff changeset	76 # Based on some searching around, 1 second seems like a reasonable
861f28212398 Import new http library as mercurial.httpclient. Augie Fackler <durin42@gmail.com> parents: diff changeset	77 # default here.
861f28212398 Import new http library as mercurial.httpclient. Augie Fackler <durin42@gmail.com> parents: diff changeset	78 TIMEOUT_ASSUME_CONTINUE = 1
861f28212398 Import new http library as mercurial.httpclient. Augie Fackler <durin42@gmail.com> parents: diff changeset	79 TIMEOUT_DEFAULT = None
861f28212398 Import new http library as mercurial.httpclient. Augie Fackler <durin42@gmail.com> parents: diff changeset	80
861f28212398 Import new http library as mercurial.httpclient. Augie Fackler <durin42@gmail.com> parents: diff changeset	81
861f28212398 Import new http library as mercurial.httpclient. Augie Fackler <durin42@gmail.com> parents: diff changeset	82 class HTTPResponse(object):
861f28212398 Import new http library as mercurial.httpclient. Augie Fackler <durin42@gmail.com> parents: diff changeset	83 """Response from an HTTP server.
861f28212398 Import new http library as mercurial.httpclient. Augie Fackler <durin42@gmail.com> parents: diff changeset	84
861f28212398 Import new http library as mercurial.httpclient. Augie Fackler <durin42@gmail.com> parents: diff changeset	85 The response will continue to load as available. If you need the
861f28212398 Import new http library as mercurial.httpclient. Augie Fackler <durin42@gmail.com> parents: diff changeset	86 complete response before continuing, check the .complete() method.
861f28212398 Import new http library as mercurial.httpclient. Augie Fackler <durin42@gmail.com> parents: diff changeset	87 """
16643 24dbef11f477 httpclient: update to revision 892730fe7f46 of httpplus Augie Fackler <raf@durin42.com> parents: 15218 diff changeset	88 def __init__(self, sock, timeout, method):
14243 861f28212398 Import new http library as mercurial.httpclient. Augie Fackler <durin42@gmail.com> parents: diff changeset	89 self.sock = sock
16643 24dbef11f477 httpclient: update to revision 892730fe7f46 of httpplus Augie Fackler <raf@durin42.com> parents: 15218 diff changeset	90 self.method = method
14243 861f28212398 Import new http library as mercurial.httpclient. Augie Fackler <durin42@gmail.com> parents: diff changeset	91 self.raw_response = ''
861f28212398 Import new http library as mercurial.httpclient. Augie Fackler <durin42@gmail.com> parents: diff changeset	92 self._headers_len = 0
861f28212398 Import new http library as mercurial.httpclient. Augie Fackler <durin42@gmail.com> parents: diff changeset	93 self.headers = None
861f28212398 Import new http library as mercurial.httpclient. Augie Fackler <durin42@gmail.com> parents: diff changeset	94 self.will_close = False
861f28212398 Import new http library as mercurial.httpclient. Augie Fackler <durin42@gmail.com> parents: diff changeset	95 self.status_line = ''
861f28212398 Import new http library as mercurial.httpclient. Augie Fackler <durin42@gmail.com> parents: diff changeset	96 self.status = None
16643 24dbef11f477 httpclient: update to revision 892730fe7f46 of httpplus Augie Fackler <raf@durin42.com> parents: 15218 diff changeset	97 self.continued = False
14243 861f28212398 Import new http library as mercurial.httpclient. Augie Fackler <durin42@gmail.com> parents: diff changeset	98 self.http_version = None
861f28212398 Import new http library as mercurial.httpclient. Augie Fackler <durin42@gmail.com> parents: diff changeset	99 self.reason = None
16643 24dbef11f477 httpclient: update to revision 892730fe7f46 of httpplus Augie Fackler <raf@durin42.com> parents: 15218 diff changeset	100 self._reader = None
14243 861f28212398 Import new http library as mercurial.httpclient. Augie Fackler <durin42@gmail.com> parents: diff changeset	101
861f28212398 Import new http library as mercurial.httpclient. Augie Fackler <durin42@gmail.com> parents: diff changeset	102 self._read_location = 0
861f28212398 Import new http library as mercurial.httpclient. Augie Fackler <durin42@gmail.com> parents: diff changeset	103 self._eol = EOL
861f28212398 Import new http library as mercurial.httpclient. Augie Fackler <durin42@gmail.com> parents: diff changeset	104
861f28212398 Import new http library as mercurial.httpclient. Augie Fackler <durin42@gmail.com> parents: diff changeset	105 self._timeout = timeout
861f28212398 Import new http library as mercurial.httpclient. Augie Fackler <durin42@gmail.com> parents: diff changeset	106
861f28212398 Import new http library as mercurial.httpclient. Augie Fackler <durin42@gmail.com> parents: diff changeset	107 @property
861f28212398 Import new http library as mercurial.httpclient. Augie Fackler <durin42@gmail.com> parents: diff changeset	108 def _end_headers(self):
861f28212398 Import new http library as mercurial.httpclient. Augie Fackler <durin42@gmail.com> parents: diff changeset	109 return self._eol * 2
861f28212398 Import new http library as mercurial.httpclient. Augie Fackler <durin42@gmail.com> parents: diff changeset	110
861f28212398 Import new http library as mercurial.httpclient. Augie Fackler <durin42@gmail.com> parents: diff changeset	111 def complete(self):
861f28212398 Import new http library as mercurial.httpclient. Augie Fackler <durin42@gmail.com> parents: diff changeset	112 """Returns true if this response is completely loaded.
14376 a75e0f4ba0ab httpclient: import revision fc731618702a of py-nonblocking-http Augie Fackler <durin42@gmail.com> parents: 14341 diff changeset	113
a75e0f4ba0ab httpclient: import revision fc731618702a of py-nonblocking-http Augie Fackler <durin42@gmail.com> parents: 14341 diff changeset	114 Note that if this is a connection where complete means the
a75e0f4ba0ab httpclient: import revision fc731618702a of py-nonblocking-http Augie Fackler <durin42@gmail.com> parents: 14341 diff changeset	115 socket is closed, this will nearly always return False, even
a75e0f4ba0ab httpclient: import revision fc731618702a of py-nonblocking-http Augie Fackler <durin42@gmail.com> parents: 14341 diff changeset	116 in cases where all the data has actually been loaded.
14243 861f28212398 Import new http library as mercurial.httpclient. Augie Fackler <durin42@gmail.com> parents: diff changeset	117 """
16643 24dbef11f477 httpclient: update to revision 892730fe7f46 of httpplus Augie Fackler <raf@durin42.com> parents: 15218 diff changeset	118 if self._reader:
24dbef11f477 httpclient: update to revision 892730fe7f46 of httpplus Augie Fackler <raf@durin42.com> parents: 15218 diff changeset	119 return self._reader.done()
24dbef11f477 httpclient: update to revision 892730fe7f46 of httpplus Augie Fackler <raf@durin42.com> parents: 15218 diff changeset	120
24dbef11f477 httpclient: update to revision 892730fe7f46 of httpplus Augie Fackler <raf@durin42.com> parents: 15218 diff changeset	121 def _close(self):
24dbef11f477 httpclient: update to revision 892730fe7f46 of httpplus Augie Fackler <raf@durin42.com> parents: 15218 diff changeset	122 if self._reader is not None:
19182 fae47ecaa952 httpclient: upgrade to fe8c09e4db64 of httpplus Augie Fackler <raf@durin42.com> parents: 19038 diff changeset	123 # We're a friend of the reader class here.
fae47ecaa952 httpclient: upgrade to fe8c09e4db64 of httpplus Augie Fackler <raf@durin42.com> parents: 19038 diff changeset	124 # pylint: disable=W0212
16643 24dbef11f477 httpclient: update to revision 892730fe7f46 of httpplus Augie Fackler <raf@durin42.com> parents: 15218 diff changeset	125 self._reader._close()
14243 861f28212398 Import new http library as mercurial.httpclient. Augie Fackler <durin42@gmail.com> parents: diff changeset	126
861f28212398 Import new http library as mercurial.httpclient. Augie Fackler <durin42@gmail.com> parents: diff changeset	127 def readline(self):
861f28212398 Import new http library as mercurial.httpclient. Augie Fackler <durin42@gmail.com> parents: diff changeset	128 """Read a single line from the response body.
861f28212398 Import new http library as mercurial.httpclient. Augie Fackler <durin42@gmail.com> parents: diff changeset	129
861f28212398 Import new http library as mercurial.httpclient. Augie Fackler <durin42@gmail.com> parents: diff changeset	130 This may block until either a line ending is found or the
861f28212398 Import new http library as mercurial.httpclient. Augie Fackler <durin42@gmail.com> parents: diff changeset	131 response is complete.
861f28212398 Import new http library as mercurial.httpclient. Augie Fackler <durin42@gmail.com> parents: diff changeset	132 """
19038 36733ab7fa05 http2: sane readline Brendan Cully <brendan@kublai.com> parents: 17537 diff changeset	133 blocks = []
36733ab7fa05 http2: sane readline Brendan Cully <brendan@kublai.com> parents: 17537 diff changeset	134 while True:
36733ab7fa05 http2: sane readline Brendan Cully <brendan@kublai.com> parents: 17537 diff changeset	135 self._reader.readto('\n', blocks)
36733ab7fa05 http2: sane readline Brendan Cully <brendan@kublai.com> parents: 17537 diff changeset	136
36733ab7fa05 http2: sane readline Brendan Cully <brendan@kublai.com> parents: 17537 diff changeset	137 if blocks and blocks[-1][-1] == '\n' or self.complete():
36733ab7fa05 http2: sane readline Brendan Cully <brendan@kublai.com> parents: 17537 diff changeset	138 break
36733ab7fa05 http2: sane readline Brendan Cully <brendan@kublai.com> parents: 17537 diff changeset	139
14243 861f28212398 Import new http library as mercurial.httpclient. Augie Fackler <durin42@gmail.com> parents: diff changeset	140 self._select()
19038 36733ab7fa05 http2: sane readline Brendan Cully <brendan@kublai.com> parents: 17537 diff changeset	141
36733ab7fa05 http2: sane readline Brendan Cully <brendan@kublai.com> parents: 17537 diff changeset	142 return ''.join(blocks)
14243 861f28212398 Import new http library as mercurial.httpclient. Augie Fackler <durin42@gmail.com> parents: diff changeset	143
861f28212398 Import new http library as mercurial.httpclient. Augie Fackler <durin42@gmail.com> parents: diff changeset	144 def read(self, length=None):
19182 fae47ecaa952 httpclient: upgrade to fe8c09e4db64 of httpplus Augie Fackler <raf@durin42.com> parents: 19038 diff changeset	145 """Read data from the response body."""
14243 861f28212398 Import new http library as mercurial.httpclient. Augie Fackler <durin42@gmail.com> parents: diff changeset	146 # if length is None, unbounded read
861f28212398 Import new http library as mercurial.httpclient. Augie Fackler <durin42@gmail.com> parents: diff changeset	147 while (not self.complete() # never select on a finished read
861f28212398 Import new http library as mercurial.httpclient. Augie Fackler <durin42@gmail.com> parents: diff changeset	148 and (not length # unbounded, so we wait for complete()
16643 24dbef11f477 httpclient: update to revision 892730fe7f46 of httpplus Augie Fackler <raf@durin42.com> parents: 15218 diff changeset	149 or length > self._reader.available_data)):
14243 861f28212398 Import new http library as mercurial.httpclient. Augie Fackler <durin42@gmail.com> parents: diff changeset	150 self._select()
861f28212398 Import new http library as mercurial.httpclient. Augie Fackler <durin42@gmail.com> parents: diff changeset	151 if not length:
16643 24dbef11f477 httpclient: update to revision 892730fe7f46 of httpplus Augie Fackler <raf@durin42.com> parents: 15218 diff changeset	152 length = self._reader.available_data
24dbef11f477 httpclient: update to revision 892730fe7f46 of httpplus Augie Fackler <raf@durin42.com> parents: 15218 diff changeset	153 r = self._reader.read(length)
14243 861f28212398 Import new http library as mercurial.httpclient. Augie Fackler <durin42@gmail.com> parents: diff changeset	154 if self.complete() and self.will_close:
861f28212398 Import new http library as mercurial.httpclient. Augie Fackler <durin42@gmail.com> parents: diff changeset	155 self.sock.close()
861f28212398 Import new http library as mercurial.httpclient. Augie Fackler <durin42@gmail.com> parents: diff changeset	156 return r
861f28212398 Import new http library as mercurial.httpclient. Augie Fackler <durin42@gmail.com> parents: diff changeset	157
861f28212398 Import new http library as mercurial.httpclient. Augie Fackler <durin42@gmail.com> parents: diff changeset	158 def _select(self):
19182 fae47ecaa952 httpclient: upgrade to fe8c09e4db64 of httpplus Augie Fackler <raf@durin42.com> parents: 19038 diff changeset	159 r, unused_write, unused_err = select.select(
fae47ecaa952 httpclient: upgrade to fe8c09e4db64 of httpplus Augie Fackler <raf@durin42.com> parents: 19038 diff changeset	160 [self.sock], [], [], self._timeout)
14243 861f28212398 Import new http library as mercurial.httpclient. Augie Fackler <durin42@gmail.com> parents: diff changeset	161 if not r:
16643 24dbef11f477 httpclient: update to revision 892730fe7f46 of httpplus Augie Fackler <raf@durin42.com> parents: 15218 diff changeset	162 # socket was not readable. If the response is not
24dbef11f477 httpclient: update to revision 892730fe7f46 of httpplus Augie Fackler <raf@durin42.com> parents: 15218 diff changeset	163 # complete, raise a timeout.
24dbef11f477 httpclient: update to revision 892730fe7f46 of httpplus Augie Fackler <raf@durin42.com> parents: 15218 diff changeset	164 if not self.complete():
14243 861f28212398 Import new http library as mercurial.httpclient. Augie Fackler <durin42@gmail.com> parents: diff changeset	165 logger.info('timed out with timeout of %s', self._timeout)
861f28212398 Import new http library as mercurial.httpclient. Augie Fackler <durin42@gmail.com> parents: diff changeset	166 raise HTTPTimeoutException('timeout reading data')
14341 5c3de67e7402 httpclient: import revision b8c3511a8cae from py-nonblocking-http Augie Fackler <durin42@gmail.com> parents: 14293 diff changeset	167 try:
5c3de67e7402 httpclient: import revision b8c3511a8cae from py-nonblocking-http Augie Fackler <durin42@gmail.com> parents: 14293 diff changeset	168 data = self.sock.recv(INCOMING_BUFFER_SIZE)
25660 328739ea70c3 global: mass rewrite to use modern exception syntax Gregory Szorc <gregory.szorc@gmail.com> parents: 24306 diff changeset	169 except socket.sslerror as e:
14341 5c3de67e7402 httpclient: import revision b8c3511a8cae from py-nonblocking-http Augie Fackler <durin42@gmail.com> parents: 14293 diff changeset	170 if e.args[0] != socket.SSL_ERROR_WANT_READ:
5c3de67e7402 httpclient: import revision b8c3511a8cae from py-nonblocking-http Augie Fackler <durin42@gmail.com> parents: 14293 diff changeset	171 raise
17428 72803c8edaa4 avoid using abbreviations that look like spelling errors Mads Kiilerich <mads@kiilerich.com> parents: 17424 diff changeset	172 logger.debug('SSL_ERROR_WANT_READ in _select, should retry later')
14341 5c3de67e7402 httpclient: import revision b8c3511a8cae from py-nonblocking-http Augie Fackler <durin42@gmail.com> parents: 14293 diff changeset	173 return True
14243 861f28212398 Import new http library as mercurial.httpclient. Augie Fackler <durin42@gmail.com> parents: diff changeset	174 logger.debug('response read %d data during _select', len(data))
16643 24dbef11f477 httpclient: update to revision 892730fe7f46 of httpplus Augie Fackler <raf@durin42.com> parents: 15218 diff changeset	175 # If the socket was readable and no data was read, that means
24dbef11f477 httpclient: update to revision 892730fe7f46 of httpplus Augie Fackler <raf@durin42.com> parents: 15218 diff changeset	176 # the socket was closed. Inform the reader (if any) so it can
24dbef11f477 httpclient: update to revision 892730fe7f46 of httpplus Augie Fackler <raf@durin42.com> parents: 15218 diff changeset	177 # raise an exception if this is an invalid situation.
14243 861f28212398 Import new http library as mercurial.httpclient. Augie Fackler <durin42@gmail.com> parents: diff changeset	178 if not data:
16643 24dbef11f477 httpclient: update to revision 892730fe7f46 of httpplus Augie Fackler <raf@durin42.com> parents: 15218 diff changeset	179 if self._reader:
19182 fae47ecaa952 httpclient: upgrade to fe8c09e4db64 of httpplus Augie Fackler <raf@durin42.com> parents: 19038 diff changeset	180 # We're a friend of the reader class here.
fae47ecaa952 httpclient: upgrade to fe8c09e4db64 of httpplus Augie Fackler <raf@durin42.com> parents: 19038 diff changeset	181 # pylint: disable=W0212
16643 24dbef11f477 httpclient: update to revision 892730fe7f46 of httpplus Augie Fackler <raf@durin42.com> parents: 15218 diff changeset	182 self._reader._close()
14243 861f28212398 Import new http library as mercurial.httpclient. Augie Fackler <durin42@gmail.com> parents: diff changeset	183 return False
861f28212398 Import new http library as mercurial.httpclient. Augie Fackler <durin42@gmail.com> parents: diff changeset	184 else:
861f28212398 Import new http library as mercurial.httpclient. Augie Fackler <durin42@gmail.com> parents: diff changeset	185 self._load_response(data)
861f28212398 Import new http library as mercurial.httpclient. Augie Fackler <durin42@gmail.com> parents: diff changeset	186 return True
861f28212398 Import new http library as mercurial.httpclient. Augie Fackler <durin42@gmail.com> parents: diff changeset	187
19182 fae47ecaa952 httpclient: upgrade to fe8c09e4db64 of httpplus Augie Fackler <raf@durin42.com> parents: 19038 diff changeset	188 # This method gets replaced by _load later, which confuses pylint.
fae47ecaa952 httpclient: upgrade to fe8c09e4db64 of httpplus Augie Fackler <raf@durin42.com> parents: 19038 diff changeset	189 def _load_response(self, data): # pylint: disable=E0202
16643 24dbef11f477 httpclient: update to revision 892730fe7f46 of httpplus Augie Fackler <raf@durin42.com> parents: 15218 diff changeset	190 # Being here implies we're not at the end of the headers yet,
24dbef11f477 httpclient: update to revision 892730fe7f46 of httpplus Augie Fackler <raf@durin42.com> parents: 15218 diff changeset	191 # since at the end of this method if headers were completely
24dbef11f477 httpclient: update to revision 892730fe7f46 of httpplus Augie Fackler <raf@durin42.com> parents: 15218 diff changeset	192 # loaded we replace this method with the load() method of the
24dbef11f477 httpclient: update to revision 892730fe7f46 of httpplus Augie Fackler <raf@durin42.com> parents: 15218 diff changeset	193 # reader we created.
14243 861f28212398 Import new http library as mercurial.httpclient. Augie Fackler <durin42@gmail.com> parents: diff changeset	194 self.raw_response += data
861f28212398 Import new http library as mercurial.httpclient. Augie Fackler <durin42@gmail.com> parents: diff changeset	195 # This is a bogus server with bad line endings
861f28212398 Import new http library as mercurial.httpclient. Augie Fackler <durin42@gmail.com> parents: diff changeset	196 if self._eol not in self.raw_response:
861f28212398 Import new http library as mercurial.httpclient. Augie Fackler <durin42@gmail.com> parents: diff changeset	197 for bad_eol in ('\n', '\r'):
861f28212398 Import new http library as mercurial.httpclient. Augie Fackler <durin42@gmail.com> parents: diff changeset	198 if (bad_eol in self.raw_response
861f28212398 Import new http library as mercurial.httpclient. Augie Fackler <durin42@gmail.com> parents: diff changeset	199 # verify that bad_eol is not the end of the incoming data
861f28212398 Import new http library as mercurial.httpclient. Augie Fackler <durin42@gmail.com> parents: diff changeset	200 # as this could be a response line that just got
861f28212398 Import new http library as mercurial.httpclient. Augie Fackler <durin42@gmail.com> parents: diff changeset	201 # split between \r and \n.
861f28212398 Import new http library as mercurial.httpclient. Augie Fackler <durin42@gmail.com> parents: diff changeset	202 and (self.raw_response.index(bad_eol) <
861f28212398 Import new http library as mercurial.httpclient. Augie Fackler <durin42@gmail.com> parents: diff changeset	203 (len(self.raw_response) - 1))):
861f28212398 Import new http library as mercurial.httpclient. Augie Fackler <durin42@gmail.com> parents: diff changeset	204 logger.info('bogus line endings detected, '
861f28212398 Import new http library as mercurial.httpclient. Augie Fackler <durin42@gmail.com> parents: diff changeset	205 'using %r for EOL', bad_eol)
861f28212398 Import new http library as mercurial.httpclient. Augie Fackler <durin42@gmail.com> parents: diff changeset	206 self._eol = bad_eol
861f28212398 Import new http library as mercurial.httpclient. Augie Fackler <durin42@gmail.com> parents: diff changeset	207 break
861f28212398 Import new http library as mercurial.httpclient. Augie Fackler <durin42@gmail.com> parents: diff changeset	208 # exit early if not at end of headers
861f28212398 Import new http library as mercurial.httpclient. Augie Fackler <durin42@gmail.com> parents: diff changeset	209 if self._end_headers not in self.raw_response or self.headers:
861f28212398 Import new http library as mercurial.httpclient. Augie Fackler <durin42@gmail.com> parents: diff changeset	210 return
861f28212398 Import new http library as mercurial.httpclient. Augie Fackler <durin42@gmail.com> parents: diff changeset	211
861f28212398 Import new http library as mercurial.httpclient. Augie Fackler <durin42@gmail.com> parents: diff changeset	212 # handle 100-continue response
861f28212398 Import new http library as mercurial.httpclient. Augie Fackler <durin42@gmail.com> parents: diff changeset	213 hdrs, body = self.raw_response.split(self._end_headers, 1)
19182 fae47ecaa952 httpclient: upgrade to fe8c09e4db64 of httpplus Augie Fackler <raf@durin42.com> parents: 19038 diff changeset	214 unused_http_ver, status = hdrs.split(' ', 1)
14243 861f28212398 Import new http library as mercurial.httpclient. Augie Fackler <durin42@gmail.com> parents: diff changeset	215 if status.startswith('100'):
861f28212398 Import new http library as mercurial.httpclient. Augie Fackler <durin42@gmail.com> parents: diff changeset	216 self.raw_response = body
16643 24dbef11f477 httpclient: update to revision 892730fe7f46 of httpplus Augie Fackler <raf@durin42.com> parents: 15218 diff changeset	217 self.continued = True
14243 861f28212398 Import new http library as mercurial.httpclient. Augie Fackler <durin42@gmail.com> parents: diff changeset	218 logger.debug('continue seen, setting body to %r', body)
861f28212398 Import new http library as mercurial.httpclient. Augie Fackler <durin42@gmail.com> parents: diff changeset	219 return
861f28212398 Import new http library as mercurial.httpclient. Augie Fackler <durin42@gmail.com> parents: diff changeset	220
861f28212398 Import new http library as mercurial.httpclient. Augie Fackler <durin42@gmail.com> parents: diff changeset	221 # arriving here means we should parse response headers
861f28212398 Import new http library as mercurial.httpclient. Augie Fackler <durin42@gmail.com> parents: diff changeset	222 # as all headers have arrived completely
861f28212398 Import new http library as mercurial.httpclient. Augie Fackler <durin42@gmail.com> parents: diff changeset	223 hdrs, body = self.raw_response.split(self._end_headers, 1)
861f28212398 Import new http library as mercurial.httpclient. Augie Fackler <durin42@gmail.com> parents: diff changeset	224 del self.raw_response
861f28212398 Import new http library as mercurial.httpclient. Augie Fackler <durin42@gmail.com> parents: diff changeset	225 if self._eol in hdrs:
861f28212398 Import new http library as mercurial.httpclient. Augie Fackler <durin42@gmail.com> parents: diff changeset	226 self.status_line, hdrs = hdrs.split(self._eol, 1)
861f28212398 Import new http library as mercurial.httpclient. Augie Fackler <durin42@gmail.com> parents: diff changeset	227 else:
861f28212398 Import new http library as mercurial.httpclient. Augie Fackler <durin42@gmail.com> parents: diff changeset	228 self.status_line = hdrs
861f28212398 Import new http library as mercurial.httpclient. Augie Fackler <durin42@gmail.com> parents: diff changeset	229 hdrs = ''
861f28212398 Import new http library as mercurial.httpclient. Augie Fackler <durin42@gmail.com> parents: diff changeset	230 # TODO HTTP < 1.0 support
861f28212398 Import new http library as mercurial.httpclient. Augie Fackler <durin42@gmail.com> parents: diff changeset	231 (self.http_version, self.status,
861f28212398 Import new http library as mercurial.httpclient. Augie Fackler <durin42@gmail.com> parents: diff changeset	232 self.reason) = self.status_line.split(' ', 2)
861f28212398 Import new http library as mercurial.httpclient. Augie Fackler <durin42@gmail.com> parents: diff changeset	233 self.status = int(self.status)
861f28212398 Import new http library as mercurial.httpclient. Augie Fackler <durin42@gmail.com> parents: diff changeset	234 if self._eol != EOL:
861f28212398 Import new http library as mercurial.httpclient. Augie Fackler <durin42@gmail.com> parents: diff changeset	235 hdrs = hdrs.replace(self._eol, '\r\n')
861f28212398 Import new http library as mercurial.httpclient. Augie Fackler <durin42@gmail.com> parents: diff changeset	236 headers = rfc822.Message(cStringIO.StringIO(hdrs))
16643 24dbef11f477 httpclient: update to revision 892730fe7f46 of httpplus Augie Fackler <raf@durin42.com> parents: 15218 diff changeset	237 content_len = None
14243 861f28212398 Import new http library as mercurial.httpclient. Augie Fackler <durin42@gmail.com> parents: diff changeset	238 if HDR_CONTENT_LENGTH in headers:
16643 24dbef11f477 httpclient: update to revision 892730fe7f46 of httpplus Augie Fackler <raf@durin42.com> parents: 15218 diff changeset	239 content_len = int(headers[HDR_CONTENT_LENGTH])
14243 861f28212398 Import new http library as mercurial.httpclient. Augie Fackler <durin42@gmail.com> parents: diff changeset	240 if self.http_version == HTTP_VER_1_0:
861f28212398 Import new http library as mercurial.httpclient. Augie Fackler <durin42@gmail.com> parents: diff changeset	241 self.will_close = True
861f28212398 Import new http library as mercurial.httpclient. Augie Fackler <durin42@gmail.com> parents: diff changeset	242 elif HDR_CONNECTION_CTRL in headers:
861f28212398 Import new http library as mercurial.httpclient. Augie Fackler <durin42@gmail.com> parents: diff changeset	243 self.will_close = (
861f28212398 Import new http library as mercurial.httpclient. Augie Fackler <durin42@gmail.com> parents: diff changeset	244 headers[HDR_CONNECTION_CTRL].lower() == CONNECTION_CLOSE)
861f28212398 Import new http library as mercurial.httpclient. Augie Fackler <durin42@gmail.com> parents: diff changeset	245 if (HDR_XFER_ENCODING in headers
861f28212398 Import new http library as mercurial.httpclient. Augie Fackler <durin42@gmail.com> parents: diff changeset	246 and headers[HDR_XFER_ENCODING].lower() == XFER_ENCODING_CHUNKED):
16643 24dbef11f477 httpclient: update to revision 892730fe7f46 of httpplus Augie Fackler <raf@durin42.com> parents: 15218 diff changeset	247 self._reader = _readers.ChunkedReader(self._eol)
24dbef11f477 httpclient: update to revision 892730fe7f46 of httpplus Augie Fackler <raf@durin42.com> parents: 15218 diff changeset	248 logger.debug('using a chunked reader')
24dbef11f477 httpclient: update to revision 892730fe7f46 of httpplus Augie Fackler <raf@durin42.com> parents: 15218 diff changeset	249 else:
24dbef11f477 httpclient: update to revision 892730fe7f46 of httpplus Augie Fackler <raf@durin42.com> parents: 15218 diff changeset	250 # HEAD responses are forbidden from returning a body, and
24dbef11f477 httpclient: update to revision 892730fe7f46 of httpplus Augie Fackler <raf@durin42.com> parents: 15218 diff changeset	251 # it's implausible for a CONNECT response to use
24dbef11f477 httpclient: update to revision 892730fe7f46 of httpplus Augie Fackler <raf@durin42.com> parents: 15218 diff changeset	252 # close-is-end logic for an OK response.
24dbef11f477 httpclient: update to revision 892730fe7f46 of httpplus Augie Fackler <raf@durin42.com> parents: 15218 diff changeset	253 if (self.method == 'HEAD' or
24dbef11f477 httpclient: update to revision 892730fe7f46 of httpplus Augie Fackler <raf@durin42.com> parents: 15218 diff changeset	254 (self.method == 'CONNECT' and content_len is None)):
24dbef11f477 httpclient: update to revision 892730fe7f46 of httpplus Augie Fackler <raf@durin42.com> parents: 15218 diff changeset	255 content_len = 0
24dbef11f477 httpclient: update to revision 892730fe7f46 of httpplus Augie Fackler <raf@durin42.com> parents: 15218 diff changeset	256 if content_len is not None:
24dbef11f477 httpclient: update to revision 892730fe7f46 of httpplus Augie Fackler <raf@durin42.com> parents: 15218 diff changeset	257 logger.debug('using a content-length reader with length %d',
24dbef11f477 httpclient: update to revision 892730fe7f46 of httpplus Augie Fackler <raf@durin42.com> parents: 15218 diff changeset	258 content_len)
24dbef11f477 httpclient: update to revision 892730fe7f46 of httpplus Augie Fackler <raf@durin42.com> parents: 15218 diff changeset	259 self._reader = _readers.ContentLengthReader(content_len)
24dbef11f477 httpclient: update to revision 892730fe7f46 of httpplus Augie Fackler <raf@durin42.com> parents: 15218 diff changeset	260 else:
24dbef11f477 httpclient: update to revision 892730fe7f46 of httpplus Augie Fackler <raf@durin42.com> parents: 15218 diff changeset	261 # Response body had no length specified and is not
24dbef11f477 httpclient: update to revision 892730fe7f46 of httpplus Augie Fackler <raf@durin42.com> parents: 15218 diff changeset	262 # chunked, so the end of the body will only be
24dbef11f477 httpclient: update to revision 892730fe7f46 of httpplus Augie Fackler <raf@durin42.com> parents: 15218 diff changeset	263 # identifiable by the termination of the socket by the
24dbef11f477 httpclient: update to revision 892730fe7f46 of httpplus Augie Fackler <raf@durin42.com> parents: 15218 diff changeset	264 # server. My interpretation of the spec means that we
24dbef11f477 httpclient: update to revision 892730fe7f46 of httpplus Augie Fackler <raf@durin42.com> parents: 15218 diff changeset	265 # are correct in hitting this case if
24dbef11f477 httpclient: update to revision 892730fe7f46 of httpplus Augie Fackler <raf@durin42.com> parents: 15218 diff changeset	266 # transfer-encoding, content-length, and
24dbef11f477 httpclient: update to revision 892730fe7f46 of httpplus Augie Fackler <raf@durin42.com> parents: 15218 diff changeset	267 # connection-control were left unspecified.
24dbef11f477 httpclient: update to revision 892730fe7f46 of httpplus Augie Fackler <raf@durin42.com> parents: 15218 diff changeset	268 self._reader = _readers.CloseIsEndReader()
24dbef11f477 httpclient: update to revision 892730fe7f46 of httpplus Augie Fackler <raf@durin42.com> parents: 15218 diff changeset	269 logger.debug('using a close-is-end reader')
24dbef11f477 httpclient: update to revision 892730fe7f46 of httpplus Augie Fackler <raf@durin42.com> parents: 15218 diff changeset	270 self.will_close = True
24dbef11f477 httpclient: update to revision 892730fe7f46 of httpplus Augie Fackler <raf@durin42.com> parents: 15218 diff changeset	271
24dbef11f477 httpclient: update to revision 892730fe7f46 of httpplus Augie Fackler <raf@durin42.com> parents: 15218 diff changeset	272 if body:
19182 fae47ecaa952 httpclient: upgrade to fe8c09e4db64 of httpplus Augie Fackler <raf@durin42.com> parents: 19038 diff changeset	273 # We're a friend of the reader class here.
fae47ecaa952 httpclient: upgrade to fe8c09e4db64 of httpplus Augie Fackler <raf@durin42.com> parents: 19038 diff changeset	274 # pylint: disable=W0212
16643 24dbef11f477 httpclient: update to revision 892730fe7f46 of httpplus Augie Fackler <raf@durin42.com> parents: 15218 diff changeset	275 self._reader._load(body)
24dbef11f477 httpclient: update to revision 892730fe7f46 of httpplus Augie Fackler <raf@durin42.com> parents: 15218 diff changeset	276 logger.debug('headers complete')
14243 861f28212398 Import new http library as mercurial.httpclient. Augie Fackler <durin42@gmail.com> parents: diff changeset	277 self.headers = headers
19182 fae47ecaa952 httpclient: upgrade to fe8c09e4db64 of httpplus Augie Fackler <raf@durin42.com> parents: 19038 diff changeset	278 # We're a friend of the reader class here.
fae47ecaa952 httpclient: upgrade to fe8c09e4db64 of httpplus Augie Fackler <raf@durin42.com> parents: 19038 diff changeset	279 # pylint: disable=W0212
16643 24dbef11f477 httpclient: update to revision 892730fe7f46 of httpplus Augie Fackler <raf@durin42.com> parents: 15218 diff changeset	280 self._load_response = self._reader._load
14243 861f28212398 Import new http library as mercurial.httpclient. Augie Fackler <durin42@gmail.com> parents: diff changeset	281
861f28212398 Import new http library as mercurial.httpclient. Augie Fackler <durin42@gmail.com> parents: diff changeset	282
861f28212398 Import new http library as mercurial.httpclient. Augie Fackler <durin42@gmail.com> parents: diff changeset	283 class HTTPConnection(object):
861f28212398 Import new http library as mercurial.httpclient. Augie Fackler <durin42@gmail.com> parents: diff changeset	284 """Connection to a single http server.
861f28212398 Import new http library as mercurial.httpclient. Augie Fackler <durin42@gmail.com> parents: diff changeset	285
861f28212398 Import new http library as mercurial.httpclient. Augie Fackler <durin42@gmail.com> parents: diff changeset	286 Supports 100-continue and keepalives natively. Uses select() for
861f28212398 Import new http library as mercurial.httpclient. Augie Fackler <durin42@gmail.com> parents: diff changeset	287 non-blocking socket operations.
861f28212398 Import new http library as mercurial.httpclient. Augie Fackler <durin42@gmail.com> parents: diff changeset	288 """
861f28212398 Import new http library as mercurial.httpclient. Augie Fackler <durin42@gmail.com> parents: diff changeset	289 http_version = HTTP_VER_1_1
861f28212398 Import new http library as mercurial.httpclient. Augie Fackler <durin42@gmail.com> parents: diff changeset	290 response_class = HTTPResponse
861f28212398 Import new http library as mercurial.httpclient. Augie Fackler <durin42@gmail.com> parents: diff changeset	291
861f28212398 Import new http library as mercurial.httpclient. Augie Fackler <durin42@gmail.com> parents: diff changeset	292 def __init__(self, host, port=None, use_ssl=None, ssl_validator=None,
861f28212398 Import new http library as mercurial.httpclient. Augie Fackler <durin42@gmail.com> parents: diff changeset	293 timeout=TIMEOUT_DEFAULT,
861f28212398 Import new http library as mercurial.httpclient. Augie Fackler <durin42@gmail.com> parents: diff changeset	294 continue_timeout=TIMEOUT_ASSUME_CONTINUE,
19807 c48df403caae httpclient: import 4bb625347d4a to provide SSL wrapper injection Augie Fackler <raf@durin42.com> parents: 19620 diff changeset	295 proxy_hostport=None, ssl_wrap_socket=None, **ssl_opts):
14243 861f28212398 Import new http library as mercurial.httpclient. Augie Fackler <durin42@gmail.com> parents: diff changeset	296 """Create a new HTTPConnection.
861f28212398 Import new http library as mercurial.httpclient. Augie Fackler <durin42@gmail.com> parents: diff changeset	297
861f28212398 Import new http library as mercurial.httpclient. Augie Fackler <durin42@gmail.com> parents: diff changeset	298 Args:
861f28212398 Import new http library as mercurial.httpclient. Augie Fackler <durin42@gmail.com> parents: diff changeset	299 host: The host to which we'll connect.
861f28212398 Import new http library as mercurial.httpclient. Augie Fackler <durin42@gmail.com> parents: diff changeset	300 port: Optional. The port over which we'll connect. Default 80 for
861f28212398 Import new http library as mercurial.httpclient. Augie Fackler <durin42@gmail.com> parents: diff changeset	301 non-ssl, 443 for ssl.
17424 e7cfe3587ea4 fix trivial spelling errors Mads Kiilerich <mads@kiilerich.com> parents: 16774 diff changeset	302 use_ssl: Optional. Whether to use ssl. Defaults to False if port is
14243 861f28212398 Import new http library as mercurial.httpclient. Augie Fackler <durin42@gmail.com> parents: diff changeset	303 not 443, true if port is 443.
861f28212398 Import new http library as mercurial.httpclient. Augie Fackler <durin42@gmail.com> parents: diff changeset	304 ssl_validator: a function(socket) to validate the ssl cert
861f28212398 Import new http library as mercurial.httpclient. Augie Fackler <durin42@gmail.com> parents: diff changeset	305 timeout: Optional. Connection timeout, default is TIMEOUT_DEFAULT.
861f28212398 Import new http library as mercurial.httpclient. Augie Fackler <durin42@gmail.com> parents: diff changeset	306 continue_timeout: Optional. Timeout for waiting on an expected
861f28212398 Import new http library as mercurial.httpclient. Augie Fackler <durin42@gmail.com> parents: diff changeset	307 "100 Continue" response. Default is TIMEOUT_ASSUME_CONTINUE.
861f28212398 Import new http library as mercurial.httpclient. Augie Fackler <durin42@gmail.com> parents: diff changeset	308 proxy_hostport: Optional. Tuple of (host, port) to use as an http
861f28212398 Import new http library as mercurial.httpclient. Augie Fackler <durin42@gmail.com> parents: diff changeset	309 proxy for the connection. Default is to not use a proxy.
19807 c48df403caae httpclient: import 4bb625347d4a to provide SSL wrapper injection Augie Fackler <raf@durin42.com> parents: 19620 diff changeset	310 ssl_wrap_socket: Optional function to use for wrapping
c48df403caae httpclient: import 4bb625347d4a to provide SSL wrapper injection Augie Fackler <raf@durin42.com> parents: 19620 diff changeset	311 sockets. If unspecified, the one from the ssl module will
c48df403caae httpclient: import 4bb625347d4a to provide SSL wrapper injection Augie Fackler <raf@durin42.com> parents: 19620 diff changeset	312 be used if available, or something that's compatible with
c48df403caae httpclient: import 4bb625347d4a to provide SSL wrapper injection Augie Fackler <raf@durin42.com> parents: 19620 diff changeset	313 it if on a Python older than 2.6.
c48df403caae httpclient: import 4bb625347d4a to provide SSL wrapper injection Augie Fackler <raf@durin42.com> parents: 19620 diff changeset	314
c48df403caae httpclient: import 4bb625347d4a to provide SSL wrapper injection Augie Fackler <raf@durin42.com> parents: 19620 diff changeset	315 Any extra keyword arguments to this function will be provided
c48df403caae httpclient: import 4bb625347d4a to provide SSL wrapper injection Augie Fackler <raf@durin42.com> parents: 19620 diff changeset	316 to the ssl_wrap_socket method. If no ssl
14243 861f28212398 Import new http library as mercurial.httpclient. Augie Fackler <durin42@gmail.com> parents: diff changeset	317 """
861f28212398 Import new http library as mercurial.httpclient. Augie Fackler <durin42@gmail.com> parents: diff changeset	318 if port is None and host.count(':') == 1 or ']:' in host:
861f28212398 Import new http library as mercurial.httpclient. Augie Fackler <durin42@gmail.com> parents: diff changeset	319 host, port = host.rsplit(':', 1)
861f28212398 Import new http library as mercurial.httpclient. Augie Fackler <durin42@gmail.com> parents: diff changeset	320 port = int(port)
861f28212398 Import new http library as mercurial.httpclient. Augie Fackler <durin42@gmail.com> parents: diff changeset	321 if '[' in host:
861f28212398 Import new http library as mercurial.httpclient. Augie Fackler <durin42@gmail.com> parents: diff changeset	322 host = host[1:-1]
19807 c48df403caae httpclient: import 4bb625347d4a to provide SSL wrapper injection Augie Fackler <raf@durin42.com> parents: 19620 diff changeset	323 if ssl_wrap_socket is not None:
c48df403caae httpclient: import 4bb625347d4a to provide SSL wrapper injection Augie Fackler <raf@durin42.com> parents: 19620 diff changeset	324 self._ssl_wrap_socket = ssl_wrap_socket
c48df403caae httpclient: import 4bb625347d4a to provide SSL wrapper injection Augie Fackler <raf@durin42.com> parents: 19620 diff changeset	325 else:
c48df403caae httpclient: import 4bb625347d4a to provide SSL wrapper injection Augie Fackler <raf@durin42.com> parents: 19620 diff changeset	326 self._ssl_wrap_socket = socketutil.wrap_socket
14243 861f28212398 Import new http library as mercurial.httpclient. Augie Fackler <durin42@gmail.com> parents: diff changeset	327 if use_ssl is None and port is None:
861f28212398 Import new http library as mercurial.httpclient. Augie Fackler <durin42@gmail.com> parents: diff changeset	328 use_ssl = False
861f28212398 Import new http library as mercurial.httpclient. Augie Fackler <durin42@gmail.com> parents: diff changeset	329 port = 80
861f28212398 Import new http library as mercurial.httpclient. Augie Fackler <durin42@gmail.com> parents: diff changeset	330 elif use_ssl is None:
861f28212398 Import new http library as mercurial.httpclient. Augie Fackler <durin42@gmail.com> parents: diff changeset	331 use_ssl = (port == 443)
861f28212398 Import new http library as mercurial.httpclient. Augie Fackler <durin42@gmail.com> parents: diff changeset	332 elif port is None:
24306 6ddc86eedc3b style: kill ersatz if-else ternary operators Jordi Guti?rrez Hermoso <jordigh@octave.org> parents: 19807 diff changeset	333 if use_ssl:
6ddc86eedc3b style: kill ersatz if-else ternary operators Jordi Guti?rrez Hermoso <jordigh@octave.org> parents: 19807 diff changeset	334 port = 443
6ddc86eedc3b style: kill ersatz if-else ternary operators Jordi Guti?rrez Hermoso <jordigh@octave.org> parents: 19807 diff changeset	335 else:
6ddc86eedc3b style: kill ersatz if-else ternary operators Jordi Guti?rrez Hermoso <jordigh@octave.org> parents: 19807 diff changeset	336 port = 80
14243 861f28212398 Import new http library as mercurial.httpclient. Augie Fackler <durin42@gmail.com> parents: diff changeset	337 self.port = port
861f28212398 Import new http library as mercurial.httpclient. Augie Fackler <durin42@gmail.com> parents: diff changeset	338 if use_ssl and not socketutil.have_ssl:
861f28212398 Import new http library as mercurial.httpclient. Augie Fackler <durin42@gmail.com> parents: diff changeset	339 raise Exception('ssl requested but unavailable on this Python')
861f28212398 Import new http library as mercurial.httpclient. Augie Fackler <durin42@gmail.com> parents: diff changeset	340 self.ssl = use_ssl
861f28212398 Import new http library as mercurial.httpclient. Augie Fackler <durin42@gmail.com> parents: diff changeset	341 self.ssl_opts = ssl_opts
861f28212398 Import new http library as mercurial.httpclient. Augie Fackler <durin42@gmail.com> parents: diff changeset	342 self._ssl_validator = ssl_validator
861f28212398 Import new http library as mercurial.httpclient. Augie Fackler <durin42@gmail.com> parents: diff changeset	343 self.host = host
861f28212398 Import new http library as mercurial.httpclient. Augie Fackler <durin42@gmail.com> parents: diff changeset	344 self.sock = None
861f28212398 Import new http library as mercurial.httpclient. Augie Fackler <durin42@gmail.com> parents: diff changeset	345 self._current_response = None
861f28212398 Import new http library as mercurial.httpclient. Augie Fackler <durin42@gmail.com> parents: diff changeset	346 self._current_response_taken = False
861f28212398 Import new http library as mercurial.httpclient. Augie Fackler <durin42@gmail.com> parents: diff changeset	347 if proxy_hostport is None:
861f28212398 Import new http library as mercurial.httpclient. Augie Fackler <durin42@gmail.com> parents: diff changeset	348 self._proxy_host = self._proxy_port = None
861f28212398 Import new http library as mercurial.httpclient. Augie Fackler <durin42@gmail.com> parents: diff changeset	349 else:
861f28212398 Import new http library as mercurial.httpclient. Augie Fackler <durin42@gmail.com> parents: diff changeset	350 self._proxy_host, self._proxy_port = proxy_hostport
861f28212398 Import new http library as mercurial.httpclient. Augie Fackler <durin42@gmail.com> parents: diff changeset	351
861f28212398 Import new http library as mercurial.httpclient. Augie Fackler <durin42@gmail.com> parents: diff changeset	352 self.timeout = timeout
861f28212398 Import new http library as mercurial.httpclient. Augie Fackler <durin42@gmail.com> parents: diff changeset	353 self.continue_timeout = continue_timeout
861f28212398 Import new http library as mercurial.httpclient. Augie Fackler <durin42@gmail.com> parents: diff changeset	354
861f28212398 Import new http library as mercurial.httpclient. Augie Fackler <durin42@gmail.com> parents: diff changeset	355 def _connect(self):
861f28212398 Import new http library as mercurial.httpclient. Augie Fackler <durin42@gmail.com> parents: diff changeset	356 """Connect to the host and port specified in __init__."""
861f28212398 Import new http library as mercurial.httpclient. Augie Fackler <durin42@gmail.com> parents: diff changeset	357 if self.sock:
861f28212398 Import new http library as mercurial.httpclient. Augie Fackler <durin42@gmail.com> parents: diff changeset	358 return
861f28212398 Import new http library as mercurial.httpclient. Augie Fackler <durin42@gmail.com> parents: diff changeset	359 if self._proxy_host is not None:
861f28212398 Import new http library as mercurial.httpclient. Augie Fackler <durin42@gmail.com> parents: diff changeset	360 logger.info('Connecting to http proxy %s:%s',
861f28212398 Import new http library as mercurial.httpclient. Augie Fackler <durin42@gmail.com> parents: diff changeset	361 self._proxy_host, self._proxy_port)
861f28212398 Import new http library as mercurial.httpclient. Augie Fackler <durin42@gmail.com> parents: diff changeset	362 sock = socketutil.create_connection((self._proxy_host,
861f28212398 Import new http library as mercurial.httpclient. Augie Fackler <durin42@gmail.com> parents: diff changeset	363 self._proxy_port))
861f28212398 Import new http library as mercurial.httpclient. Augie Fackler <durin42@gmail.com> parents: diff changeset	364 if self.ssl:
861f28212398 Import new http library as mercurial.httpclient. Augie Fackler <durin42@gmail.com> parents: diff changeset	365 # TODO proxy header support
19182 fae47ecaa952 httpclient: upgrade to fe8c09e4db64 of httpplus Augie Fackler <raf@durin42.com> parents: 19038 diff changeset	366 data = self._buildheaders('CONNECT', '%s:%d' % (self.host,
fae47ecaa952 httpclient: upgrade to fe8c09e4db64 of httpplus Augie Fackler <raf@durin42.com> parents: 19038 diff changeset	367 self.port),
fae47ecaa952 httpclient: upgrade to fe8c09e4db64 of httpplus Augie Fackler <raf@durin42.com> parents: 19038 diff changeset	368 {}, HTTP_VER_1_0)
14243 861f28212398 Import new http library as mercurial.httpclient. Augie Fackler <durin42@gmail.com> parents: diff changeset	369 sock.send(data)
861f28212398 Import new http library as mercurial.httpclient. Augie Fackler <durin42@gmail.com> parents: diff changeset	370 sock.setblocking(0)
16643 24dbef11f477 httpclient: update to revision 892730fe7f46 of httpplus Augie Fackler <raf@durin42.com> parents: 15218 diff changeset	371 r = self.response_class(sock, self.timeout, 'CONNECT')
14243 861f28212398 Import new http library as mercurial.httpclient. Augie Fackler <durin42@gmail.com> parents: diff changeset	372 timeout_exc = HTTPTimeoutException(
861f28212398 Import new http library as mercurial.httpclient. Augie Fackler <durin42@gmail.com> parents: diff changeset	373 'Timed out waiting for CONNECT response from proxy')
861f28212398 Import new http library as mercurial.httpclient. Augie Fackler <durin42@gmail.com> parents: diff changeset	374 while not r.complete():
861f28212398 Import new http library as mercurial.httpclient. Augie Fackler <durin42@gmail.com> parents: diff changeset	375 try:
19182 fae47ecaa952 httpclient: upgrade to fe8c09e4db64 of httpplus Augie Fackler <raf@durin42.com> parents: 19038 diff changeset	376 # We're a friend of the response class, so let
fae47ecaa952 httpclient: upgrade to fe8c09e4db64 of httpplus Augie Fackler <raf@durin42.com> parents: 19038 diff changeset	377 # us use the private attribute.
fae47ecaa952 httpclient: upgrade to fe8c09e4db64 of httpplus Augie Fackler <raf@durin42.com> parents: 19038 diff changeset	378 # pylint: disable=W0212
14243 861f28212398 Import new http library as mercurial.httpclient. Augie Fackler <durin42@gmail.com> parents: diff changeset	379 if not r._select():
16643 24dbef11f477 httpclient: update to revision 892730fe7f46 of httpplus Augie Fackler <raf@durin42.com> parents: 15218 diff changeset	380 if not r.complete():
24dbef11f477 httpclient: update to revision 892730fe7f46 of httpplus Augie Fackler <raf@durin42.com> parents: 15218 diff changeset	381 raise timeout_exc
14243 861f28212398 Import new http library as mercurial.httpclient. Augie Fackler <durin42@gmail.com> parents: diff changeset	382 except HTTPTimeoutException:
861f28212398 Import new http library as mercurial.httpclient. Augie Fackler <durin42@gmail.com> parents: diff changeset	383 # This raise/except pattern looks goofy, but
861f28212398 Import new http library as mercurial.httpclient. Augie Fackler <durin42@gmail.com> parents: diff changeset	384 # _select can raise the timeout as well as the
861f28212398 Import new http library as mercurial.httpclient. Augie Fackler <durin42@gmail.com> parents: diff changeset	385 # loop body. I wish it wasn't this convoluted,
861f28212398 Import new http library as mercurial.httpclient. Augie Fackler <durin42@gmail.com> parents: diff changeset	386 # but I don't have a better solution
861f28212398 Import new http library as mercurial.httpclient. Augie Fackler <durin42@gmail.com> parents: diff changeset	387 # immediately handy.
861f28212398 Import new http library as mercurial.httpclient. Augie Fackler <durin42@gmail.com> parents: diff changeset	388 raise timeout_exc
861f28212398 Import new http library as mercurial.httpclient. Augie Fackler <durin42@gmail.com> parents: diff changeset	389 if r.status != 200:
861f28212398 Import new http library as mercurial.httpclient. Augie Fackler <durin42@gmail.com> parents: diff changeset	390 raise HTTPProxyConnectFailedException(
861f28212398 Import new http library as mercurial.httpclient. Augie Fackler <durin42@gmail.com> parents: diff changeset	391 'Proxy connection failed: %d %s' % (r.status,
861f28212398 Import new http library as mercurial.httpclient. Augie Fackler <durin42@gmail.com> parents: diff changeset	392 r.read()))
861f28212398 Import new http library as mercurial.httpclient. Augie Fackler <durin42@gmail.com> parents: diff changeset	393 logger.info('CONNECT (for SSL) to %s:%s via proxy succeeded.',
861f28212398 Import new http library as mercurial.httpclient. Augie Fackler <durin42@gmail.com> parents: diff changeset	394 self.host, self.port)
861f28212398 Import new http library as mercurial.httpclient. Augie Fackler <durin42@gmail.com> parents: diff changeset	395 else:
861f28212398 Import new http library as mercurial.httpclient. Augie Fackler <durin42@gmail.com> parents: diff changeset	396 sock = socketutil.create_connection((self.host, self.port))
861f28212398 Import new http library as mercurial.httpclient. Augie Fackler <durin42@gmail.com> parents: diff changeset	397 if self.ssl:
16774 69af967b6d6f httpclient: update to c5abd358e543 of httpplus Augie Fackler <raf@durin42.com> parents: 16643 diff changeset	398 # This is the default, but in the case of proxied SSL
69af967b6d6f httpclient: update to c5abd358e543 of httpplus Augie Fackler <raf@durin42.com> parents: 16643 diff changeset	399 # requests the proxy logic above will have cleared
17424 e7cfe3587ea4 fix trivial spelling errors Mads Kiilerich <mads@kiilerich.com> parents: 16774 diff changeset	400 # blocking mode, so re-enable it just to be safe.
16774 69af967b6d6f httpclient: update to c5abd358e543 of httpplus Augie Fackler <raf@durin42.com> parents: 16643 diff changeset	401 sock.setblocking(1)
14243 861f28212398 Import new http library as mercurial.httpclient. Augie Fackler <durin42@gmail.com> parents: diff changeset	402 logger.debug('wrapping socket for ssl with options %r',
861f28212398 Import new http library as mercurial.httpclient. Augie Fackler <durin42@gmail.com> parents: diff changeset	403 self.ssl_opts)
19807 c48df403caae httpclient: import 4bb625347d4a to provide SSL wrapper injection Augie Fackler <raf@durin42.com> parents: 19620 diff changeset	404 sock = self._ssl_wrap_socket(sock, **self.ssl_opts)
14243 861f28212398 Import new http library as mercurial.httpclient. Augie Fackler <durin42@gmail.com> parents: diff changeset	405 if self._ssl_validator:
861f28212398 Import new http library as mercurial.httpclient. Augie Fackler <durin42@gmail.com> parents: diff changeset	406 self._ssl_validator(sock)
861f28212398 Import new http library as mercurial.httpclient. Augie Fackler <durin42@gmail.com> parents: diff changeset	407 sock.setblocking(0)
861f28212398 Import new http library as mercurial.httpclient. Augie Fackler <durin42@gmail.com> parents: diff changeset	408 self.sock = sock
861f28212398 Import new http library as mercurial.httpclient. Augie Fackler <durin42@gmail.com> parents: diff changeset	409
19182 fae47ecaa952 httpclient: upgrade to fe8c09e4db64 of httpplus Augie Fackler <raf@durin42.com> parents: 19038 diff changeset	410 def _buildheaders(self, method, path, headers, http_ver):
14243 861f28212398 Import new http library as mercurial.httpclient. Augie Fackler <durin42@gmail.com> parents: diff changeset	411 if self.ssl and self.port == 443 or self.port == 80:
861f28212398 Import new http library as mercurial.httpclient. Augie Fackler <durin42@gmail.com> parents: diff changeset	412 # default port for protocol, so leave it out
861f28212398 Import new http library as mercurial.httpclient. Augie Fackler <durin42@gmail.com> parents: diff changeset	413 hdrhost = self.host
861f28212398 Import new http library as mercurial.httpclient. Augie Fackler <durin42@gmail.com> parents: diff changeset	414 else:
861f28212398 Import new http library as mercurial.httpclient. Augie Fackler <durin42@gmail.com> parents: diff changeset	415 # include nonstandard port in header
861f28212398 Import new http library as mercurial.httpclient. Augie Fackler <durin42@gmail.com> parents: diff changeset	416 if ':' in self.host: # must be IPv6
861f28212398 Import new http library as mercurial.httpclient. Augie Fackler <durin42@gmail.com> parents: diff changeset	417 hdrhost = '[%s]:%d' % (self.host, self.port)
861f28212398 Import new http library as mercurial.httpclient. Augie Fackler <durin42@gmail.com> parents: diff changeset	418 else:
861f28212398 Import new http library as mercurial.httpclient. Augie Fackler <durin42@gmail.com> parents: diff changeset	419 hdrhost = '%s:%d' % (self.host, self.port)
861f28212398 Import new http library as mercurial.httpclient. Augie Fackler <durin42@gmail.com> parents: diff changeset	420 if self._proxy_host and not self.ssl:
861f28212398 Import new http library as mercurial.httpclient. Augie Fackler <durin42@gmail.com> parents: diff changeset	421 # When talking to a regular http proxy we must send the
861f28212398 Import new http library as mercurial.httpclient. Augie Fackler <durin42@gmail.com> parents: diff changeset	422 # full URI, but in all other cases we must not (although
861f28212398 Import new http library as mercurial.httpclient. Augie Fackler <durin42@gmail.com> parents: diff changeset	423 # technically RFC 2616 says servers must accept our
861f28212398 Import new http library as mercurial.httpclient. Augie Fackler <durin42@gmail.com> parents: diff changeset	424 # request if we screw up, experimentally few do that
861f28212398 Import new http library as mercurial.httpclient. Augie Fackler <durin42@gmail.com> parents: diff changeset	425 # correctly.)
861f28212398 Import new http library as mercurial.httpclient. Augie Fackler <durin42@gmail.com> parents: diff changeset	426 assert path[0] == '/', 'path must start with a /'
861f28212398 Import new http library as mercurial.httpclient. Augie Fackler <durin42@gmail.com> parents: diff changeset	427 path = 'http://%s%s' % (hdrhost, path)
861f28212398 Import new http library as mercurial.httpclient. Augie Fackler <durin42@gmail.com> parents: diff changeset	428 outgoing = ['%s %s %s%s' % (method, path, http_ver, EOL)]
861f28212398 Import new http library as mercurial.httpclient. Augie Fackler <durin42@gmail.com> parents: diff changeset	429 headers['host'] = ('Host', hdrhost)
861f28212398 Import new http library as mercurial.httpclient. Augie Fackler <durin42@gmail.com> parents: diff changeset	430 headers[HDR_ACCEPT_ENCODING] = (HDR_ACCEPT_ENCODING, 'identity')
861f28212398 Import new http library as mercurial.httpclient. Augie Fackler <durin42@gmail.com> parents: diff changeset	431 for hdr, val in headers.itervalues():
861f28212398 Import new http library as mercurial.httpclient. Augie Fackler <durin42@gmail.com> parents: diff changeset	432 outgoing.append('%s: %s%s' % (hdr, val, EOL))
861f28212398 Import new http library as mercurial.httpclient. Augie Fackler <durin42@gmail.com> parents: diff changeset	433 outgoing.append(EOL)
861f28212398 Import new http library as mercurial.httpclient. Augie Fackler <durin42@gmail.com> parents: diff changeset	434 return ''.join(outgoing)
861f28212398 Import new http library as mercurial.httpclient. Augie Fackler <durin42@gmail.com> parents: diff changeset	435
861f28212398 Import new http library as mercurial.httpclient. Augie Fackler <durin42@gmail.com> parents: diff changeset	436 def close(self):
861f28212398 Import new http library as mercurial.httpclient. Augie Fackler <durin42@gmail.com> parents: diff changeset	437 """Close the connection to the server.
861f28212398 Import new http library as mercurial.httpclient. Augie Fackler <durin42@gmail.com> parents: diff changeset	438
861f28212398 Import new http library as mercurial.httpclient. Augie Fackler <durin42@gmail.com> parents: diff changeset	439 This is a no-op if the connection is already closed. The
17536 dc6364a81e42 spelling: requested timeless@mozdev.org parents: 16774 diff changeset	440 connection may automatically close if requested by the server
14243 861f28212398 Import new http library as mercurial.httpclient. Augie Fackler <durin42@gmail.com> parents: diff changeset	441 or required by the nature of a response.
861f28212398 Import new http library as mercurial.httpclient. Augie Fackler <durin42@gmail.com> parents: diff changeset	442 """
861f28212398 Import new http library as mercurial.httpclient. Augie Fackler <durin42@gmail.com> parents: diff changeset	443 if self.sock is None:
861f28212398 Import new http library as mercurial.httpclient. Augie Fackler <durin42@gmail.com> parents: diff changeset	444 return
861f28212398 Import new http library as mercurial.httpclient. Augie Fackler <durin42@gmail.com> parents: diff changeset	445 self.sock.close()
861f28212398 Import new http library as mercurial.httpclient. Augie Fackler <durin42@gmail.com> parents: diff changeset	446 self.sock = None
861f28212398 Import new http library as mercurial.httpclient. Augie Fackler <durin42@gmail.com> parents: diff changeset	447 logger.info('closed connection to %s on %s', self.host, self.port)
861f28212398 Import new http library as mercurial.httpclient. Augie Fackler <durin42@gmail.com> parents: diff changeset	448
861f28212398 Import new http library as mercurial.httpclient. Augie Fackler <durin42@gmail.com> parents: diff changeset	449 def busy(self):
861f28212398 Import new http library as mercurial.httpclient. Augie Fackler <durin42@gmail.com> parents: diff changeset	450 """Returns True if this connection object is currently in use.
861f28212398 Import new http library as mercurial.httpclient. Augie Fackler <durin42@gmail.com> parents: diff changeset	451
861f28212398 Import new http library as mercurial.httpclient. Augie Fackler <durin42@gmail.com> parents: diff changeset	452 If a response is still pending, this will return True, even if
861f28212398 Import new http library as mercurial.httpclient. Augie Fackler <durin42@gmail.com> parents: diff changeset	453 the request has finished sending. In the future,
861f28212398 Import new http library as mercurial.httpclient. Augie Fackler <durin42@gmail.com> parents: diff changeset	454 HTTPConnection may transparently juggle multiple connections
861f28212398 Import new http library as mercurial.httpclient. Augie Fackler <durin42@gmail.com> parents: diff changeset	455 to the server, in which case this will be useful to detect if
861f28212398 Import new http library as mercurial.httpclient. Augie Fackler <durin42@gmail.com> parents: diff changeset	456 any of those connections is ready for use.
861f28212398 Import new http library as mercurial.httpclient. Augie Fackler <durin42@gmail.com> parents: diff changeset	457 """
861f28212398 Import new http library as mercurial.httpclient. Augie Fackler <durin42@gmail.com> parents: diff changeset	458 cr = self._current_response
861f28212398 Import new http library as mercurial.httpclient. Augie Fackler <durin42@gmail.com> parents: diff changeset	459 if cr is not None:
861f28212398 Import new http library as mercurial.httpclient. Augie Fackler <durin42@gmail.com> parents: diff changeset	460 if self._current_response_taken:
861f28212398 Import new http library as mercurial.httpclient. Augie Fackler <durin42@gmail.com> parents: diff changeset	461 if cr.will_close:
861f28212398 Import new http library as mercurial.httpclient. Augie Fackler <durin42@gmail.com> parents: diff changeset	462 self.sock = None
861f28212398 Import new http library as mercurial.httpclient. Augie Fackler <durin42@gmail.com> parents: diff changeset	463 self._current_response = None
861f28212398 Import new http library as mercurial.httpclient. Augie Fackler <durin42@gmail.com> parents: diff changeset	464 return False
861f28212398 Import new http library as mercurial.httpclient. Augie Fackler <durin42@gmail.com> parents: diff changeset	465 elif cr.complete():
861f28212398 Import new http library as mercurial.httpclient. Augie Fackler <durin42@gmail.com> parents: diff changeset	466 self._current_response = None
861f28212398 Import new http library as mercurial.httpclient. Augie Fackler <durin42@gmail.com> parents: diff changeset	467 return False
861f28212398 Import new http library as mercurial.httpclient. Augie Fackler <durin42@gmail.com> parents: diff changeset	468 return True
861f28212398 Import new http library as mercurial.httpclient. Augie Fackler <durin42@gmail.com> parents: diff changeset	469 return False
861f28212398 Import new http library as mercurial.httpclient. Augie Fackler <durin42@gmail.com> parents: diff changeset	470
19182 fae47ecaa952 httpclient: upgrade to fe8c09e4db64 of httpplus Augie Fackler <raf@durin42.com> parents: 19038 diff changeset	471 def _reconnect(self, where):
fae47ecaa952 httpclient: upgrade to fe8c09e4db64 of httpplus Augie Fackler <raf@durin42.com> parents: 19038 diff changeset	472 logger.info('reconnecting during %s', where)
fae47ecaa952 httpclient: upgrade to fe8c09e4db64 of httpplus Augie Fackler <raf@durin42.com> parents: 19038 diff changeset	473 self.close()
fae47ecaa952 httpclient: upgrade to fe8c09e4db64 of httpplus Augie Fackler <raf@durin42.com> parents: 19038 diff changeset	474 self._connect()
fae47ecaa952 httpclient: upgrade to fe8c09e4db64 of httpplus Augie Fackler <raf@durin42.com> parents: 19038 diff changeset	475
14243 861f28212398 Import new http library as mercurial.httpclient. Augie Fackler <durin42@gmail.com> parents: diff changeset	476 def request(self, method, path, body=None, headers={},
861f28212398 Import new http library as mercurial.httpclient. Augie Fackler <durin42@gmail.com> parents: diff changeset	477 expect_continue=False):
861f28212398 Import new http library as mercurial.httpclient. Augie Fackler <durin42@gmail.com> parents: diff changeset	478 """Send a request to the server.
861f28212398 Import new http library as mercurial.httpclient. Augie Fackler <durin42@gmail.com> parents: diff changeset	479
861f28212398 Import new http library as mercurial.httpclient. Augie Fackler <durin42@gmail.com> parents: diff changeset	480 For increased flexibility, this does not return the response
861f28212398 Import new http library as mercurial.httpclient. Augie Fackler <durin42@gmail.com> parents: diff changeset	481 object. Future versions of HTTPConnection that juggle multiple
861f28212398 Import new http library as mercurial.httpclient. Augie Fackler <durin42@gmail.com> parents: diff changeset	482 sockets will be able to send (for example) 5 requests all at
861f28212398 Import new http library as mercurial.httpclient. Augie Fackler <durin42@gmail.com> parents: diff changeset	483 once, and then let the requests arrive as data is
861f28212398 Import new http library as mercurial.httpclient. Augie Fackler <durin42@gmail.com> parents: diff changeset	484 available. Use the `getresponse()` method to retrieve the
861f28212398 Import new http library as mercurial.httpclient. Augie Fackler <durin42@gmail.com> parents: diff changeset	485 response.
861f28212398 Import new http library as mercurial.httpclient. Augie Fackler <durin42@gmail.com> parents: diff changeset	486 """
861f28212398 Import new http library as mercurial.httpclient. Augie Fackler <durin42@gmail.com> parents: diff changeset	487 if self.busy():
861f28212398 Import new http library as mercurial.httpclient. Augie Fackler <durin42@gmail.com> parents: diff changeset	488 raise httplib.CannotSendRequest(
861f28212398 Import new http library as mercurial.httpclient. Augie Fackler <durin42@gmail.com> parents: diff changeset	489 'Can not send another request before '
861f28212398 Import new http library as mercurial.httpclient. Augie Fackler <durin42@gmail.com> parents: diff changeset	490 'current response is read!')
861f28212398 Import new http library as mercurial.httpclient. Augie Fackler <durin42@gmail.com> parents: diff changeset	491 self._current_response_taken = False
861f28212398 Import new http library as mercurial.httpclient. Augie Fackler <durin42@gmail.com> parents: diff changeset	492
861f28212398 Import new http library as mercurial.httpclient. Augie Fackler <durin42@gmail.com> parents: diff changeset	493 logger.info('sending %s request for %s to %s on port %s',
861f28212398 Import new http library as mercurial.httpclient. Augie Fackler <durin42@gmail.com> parents: diff changeset	494 method, path, self.host, self.port)
861f28212398 Import new http library as mercurial.httpclient. Augie Fackler <durin42@gmail.com> parents: diff changeset	495 hdrs = dict((k.lower(), (k, v)) for k, v in headers.iteritems())
861f28212398 Import new http library as mercurial.httpclient. Augie Fackler <durin42@gmail.com> parents: diff changeset	496 if hdrs.get('expect', ('', ''))[1].lower() == '100-continue':
861f28212398 Import new http library as mercurial.httpclient. Augie Fackler <durin42@gmail.com> parents: diff changeset	497 expect_continue = True
861f28212398 Import new http library as mercurial.httpclient. Augie Fackler <durin42@gmail.com> parents: diff changeset	498 elif expect_continue:
861f28212398 Import new http library as mercurial.httpclient. Augie Fackler <durin42@gmail.com> parents: diff changeset	499 hdrs['expect'] = ('Expect', '100-Continue')
861f28212398 Import new http library as mercurial.httpclient. Augie Fackler <durin42@gmail.com> parents: diff changeset	500
861f28212398 Import new http library as mercurial.httpclient. Augie Fackler <durin42@gmail.com> parents: diff changeset	501 chunked = False
861f28212398 Import new http library as mercurial.httpclient. Augie Fackler <durin42@gmail.com> parents: diff changeset	502 if body and HDR_CONTENT_LENGTH not in hdrs:
861f28212398 Import new http library as mercurial.httpclient. Augie Fackler <durin42@gmail.com> parents: diff changeset	503 if getattr(body, '__len__', False):
861f28212398 Import new http library as mercurial.httpclient. Augie Fackler <durin42@gmail.com> parents: diff changeset	504 hdrs[HDR_CONTENT_LENGTH] = (HDR_CONTENT_LENGTH, len(body))
861f28212398 Import new http library as mercurial.httpclient. Augie Fackler <durin42@gmail.com> parents: diff changeset	505 elif getattr(body, 'read', False):
861f28212398 Import new http library as mercurial.httpclient. Augie Fackler <durin42@gmail.com> parents: diff changeset	506 hdrs[HDR_XFER_ENCODING] = (HDR_XFER_ENCODING,
861f28212398 Import new http library as mercurial.httpclient. Augie Fackler <durin42@gmail.com> parents: diff changeset	507 XFER_ENCODING_CHUNKED)
861f28212398 Import new http library as mercurial.httpclient. Augie Fackler <durin42@gmail.com> parents: diff changeset	508 chunked = True
861f28212398 Import new http library as mercurial.httpclient. Augie Fackler <durin42@gmail.com> parents: diff changeset	509 else:
861f28212398 Import new http library as mercurial.httpclient. Augie Fackler <durin42@gmail.com> parents: diff changeset	510 raise BadRequestData('body has no __len__() nor read()')
861f28212398 Import new http library as mercurial.httpclient. Augie Fackler <durin42@gmail.com> parents: diff changeset	511
19620 d4a0055af149 httpclient: import 0d1b0a8bc549 to fix bug involving late-arriving RST after a response Augie Fackler <raf@durin42.com> parents: 19489 diff changeset	512 # If we're reusing the underlying socket, there are some
d4a0055af149 httpclient: import 0d1b0a8bc549 to fix bug involving late-arriving RST after a response Augie Fackler <raf@durin42.com> parents: 19489 diff changeset	513 # conditions where we'll want to retry, so make a note of the
d4a0055af149 httpclient: import 0d1b0a8bc549 to fix bug involving late-arriving RST after a response Augie Fackler <raf@durin42.com> parents: 19489 diff changeset	514 # state of self.sock
d4a0055af149 httpclient: import 0d1b0a8bc549 to fix bug involving late-arriving RST after a response Augie Fackler <raf@durin42.com> parents: 19489 diff changeset	515 fresh_socket = self.sock is None
14243 861f28212398 Import new http library as mercurial.httpclient. Augie Fackler <durin42@gmail.com> parents: diff changeset	516 self._connect()
19182 fae47ecaa952 httpclient: upgrade to fe8c09e4db64 of httpplus Augie Fackler <raf@durin42.com> parents: 19038 diff changeset	517 outgoing_headers = self._buildheaders(
14243 861f28212398 Import new http library as mercurial.httpclient. Augie Fackler <durin42@gmail.com> parents: diff changeset	518 method, path, hdrs, self.http_version)
861f28212398 Import new http library as mercurial.httpclient. Augie Fackler <durin42@gmail.com> parents: diff changeset	519 response = None
861f28212398 Import new http library as mercurial.httpclient. Augie Fackler <durin42@gmail.com> parents: diff changeset	520 first = True
861f28212398 Import new http library as mercurial.httpclient. Augie Fackler <durin42@gmail.com> parents: diff changeset	521
861f28212398 Import new http library as mercurial.httpclient. Augie Fackler <durin42@gmail.com> parents: diff changeset	522 while ((outgoing_headers or body)
861f28212398 Import new http library as mercurial.httpclient. Augie Fackler <durin42@gmail.com> parents: diff changeset	523 and not (response and response.complete())):
861f28212398 Import new http library as mercurial.httpclient. Augie Fackler <durin42@gmail.com> parents: diff changeset	524 select_timeout = self.timeout
861f28212398 Import new http library as mercurial.httpclient. Augie Fackler <durin42@gmail.com> parents: diff changeset	525 out = outgoing_headers or body
861f28212398 Import new http library as mercurial.httpclient. Augie Fackler <durin42@gmail.com> parents: diff changeset	526 blocking_on_continue = False
861f28212398 Import new http library as mercurial.httpclient. Augie Fackler <durin42@gmail.com> parents: diff changeset	527 if expect_continue and not outgoing_headers and not (
16643 24dbef11f477 httpclient: update to revision 892730fe7f46 of httpplus Augie Fackler <raf@durin42.com> parents: 15218 diff changeset	528 response and (response.headers or response.continued)):
14243 861f28212398 Import new http library as mercurial.httpclient. Augie Fackler <durin42@gmail.com> parents: diff changeset	529 logger.info(
861f28212398 Import new http library as mercurial.httpclient. Augie Fackler <durin42@gmail.com> parents: diff changeset	530 'waiting up to %s seconds for'
861f28212398 Import new http library as mercurial.httpclient. Augie Fackler <durin42@gmail.com> parents: diff changeset	531 ' continue response from server',
861f28212398 Import new http library as mercurial.httpclient. Augie Fackler <durin42@gmail.com> parents: diff changeset	532 self.continue_timeout)
861f28212398 Import new http library as mercurial.httpclient. Augie Fackler <durin42@gmail.com> parents: diff changeset	533 select_timeout = self.continue_timeout
861f28212398 Import new http library as mercurial.httpclient. Augie Fackler <durin42@gmail.com> parents: diff changeset	534 blocking_on_continue = True
861f28212398 Import new http library as mercurial.httpclient. Augie Fackler <durin42@gmail.com> parents: diff changeset	535 out = False
861f28212398 Import new http library as mercurial.httpclient. Augie Fackler <durin42@gmail.com> parents: diff changeset	536 if out:
861f28212398 Import new http library as mercurial.httpclient. Augie Fackler <durin42@gmail.com> parents: diff changeset	537 w = [self.sock]
861f28212398 Import new http library as mercurial.httpclient. Augie Fackler <durin42@gmail.com> parents: diff changeset	538 else:
861f28212398 Import new http library as mercurial.httpclient. Augie Fackler <durin42@gmail.com> parents: diff changeset	539 w = []
861f28212398 Import new http library as mercurial.httpclient. Augie Fackler <durin42@gmail.com> parents: diff changeset	540 r, w, x = select.select([self.sock], w, [], select_timeout)
861f28212398 Import new http library as mercurial.httpclient. Augie Fackler <durin42@gmail.com> parents: diff changeset	541 # if we were expecting a 100 continue and it's been long
861f28212398 Import new http library as mercurial.httpclient. Augie Fackler <durin42@gmail.com> parents: diff changeset	542 # enough, just go ahead and assume it's ok. This is the
861f28212398 Import new http library as mercurial.httpclient. Augie Fackler <durin42@gmail.com> parents: diff changeset	543 # recommended behavior from the RFC.
861f28212398 Import new http library as mercurial.httpclient. Augie Fackler <durin42@gmail.com> parents: diff changeset	544 if r == w == x == []:
861f28212398 Import new http library as mercurial.httpclient. Augie Fackler <durin42@gmail.com> parents: diff changeset	545 if blocking_on_continue:
861f28212398 Import new http library as mercurial.httpclient. Augie Fackler <durin42@gmail.com> parents: diff changeset	546 expect_continue = False
861f28212398 Import new http library as mercurial.httpclient. Augie Fackler <durin42@gmail.com> parents: diff changeset	547 logger.info('no response to continue expectation from '
861f28212398 Import new http library as mercurial.httpclient. Augie Fackler <durin42@gmail.com> parents: diff changeset	548 'server, optimistically sending request body')
861f28212398 Import new http library as mercurial.httpclient. Augie Fackler <durin42@gmail.com> parents: diff changeset	549 else:
861f28212398 Import new http library as mercurial.httpclient. Augie Fackler <durin42@gmail.com> parents: diff changeset	550 raise HTTPTimeoutException('timeout sending data')
861f28212398 Import new http library as mercurial.httpclient. Augie Fackler <durin42@gmail.com> parents: diff changeset	551 was_first = first
861f28212398 Import new http library as mercurial.httpclient. Augie Fackler <durin42@gmail.com> parents: diff changeset	552
861f28212398 Import new http library as mercurial.httpclient. Augie Fackler <durin42@gmail.com> parents: diff changeset	553 # incoming data
861f28212398 Import new http library as mercurial.httpclient. Augie Fackler <durin42@gmail.com> parents: diff changeset	554 if r:
861f28212398 Import new http library as mercurial.httpclient. Augie Fackler <durin42@gmail.com> parents: diff changeset	555 try:
14341 5c3de67e7402 httpclient: import revision b8c3511a8cae from py-nonblocking-http Augie Fackler <durin42@gmail.com> parents: 14293 diff changeset	556 try:
5c3de67e7402 httpclient: import revision b8c3511a8cae from py-nonblocking-http Augie Fackler <durin42@gmail.com> parents: 14293 diff changeset	557 data = r[0].recv(INCOMING_BUFFER_SIZE)
25660 328739ea70c3 global: mass rewrite to use modern exception syntax Gregory Szorc <gregory.szorc@gmail.com> parents: 24306 diff changeset	558 except socket.sslerror as e:
14341 5c3de67e7402 httpclient: import revision b8c3511a8cae from py-nonblocking-http Augie Fackler <durin42@gmail.com> parents: 14293 diff changeset	559 if e.args[0] != socket.SSL_ERROR_WANT_READ:
5c3de67e7402 httpclient: import revision b8c3511a8cae from py-nonblocking-http Augie Fackler <durin42@gmail.com> parents: 14293 diff changeset	560 raise
19182 fae47ecaa952 httpclient: upgrade to fe8c09e4db64 of httpplus Augie Fackler <raf@durin42.com> parents: 19038 diff changeset	561 logger.debug('SSL_ERROR_WANT_READ while sending '
fae47ecaa952 httpclient: upgrade to fe8c09e4db64 of httpplus Augie Fackler <raf@durin42.com> parents: 19038 diff changeset	562 'data, retrying...')
14341 5c3de67e7402 httpclient: import revision b8c3511a8cae from py-nonblocking-http Augie Fackler <durin42@gmail.com> parents: 14293 diff changeset	563 continue
14243 861f28212398 Import new http library as mercurial.httpclient. Augie Fackler <durin42@gmail.com> parents: diff changeset	564 if not data:
861f28212398 Import new http library as mercurial.httpclient. Augie Fackler <durin42@gmail.com> parents: diff changeset	565 logger.info('socket appears closed in read')
14376 a75e0f4ba0ab httpclient: import revision fc731618702a of py-nonblocking-http Augie Fackler <durin42@gmail.com> parents: 14341 diff changeset	566 self.sock = None
a75e0f4ba0ab httpclient: import revision fc731618702a of py-nonblocking-http Augie Fackler <durin42@gmail.com> parents: 14341 diff changeset	567 self._current_response = None
16643 24dbef11f477 httpclient: update to revision 892730fe7f46 of httpplus Augie Fackler <raf@durin42.com> parents: 15218 diff changeset	568 if response is not None:
19182 fae47ecaa952 httpclient: upgrade to fe8c09e4db64 of httpplus Augie Fackler <raf@durin42.com> parents: 19038 diff changeset	569 # We're a friend of the response class, so let
fae47ecaa952 httpclient: upgrade to fe8c09e4db64 of httpplus Augie Fackler <raf@durin42.com> parents: 19038 diff changeset	570 # us use the private attribute.
fae47ecaa952 httpclient: upgrade to fe8c09e4db64 of httpplus Augie Fackler <raf@durin42.com> parents: 19038 diff changeset	571 # pylint: disable=W0212
16643 24dbef11f477 httpclient: update to revision 892730fe7f46 of httpplus Augie Fackler <raf@durin42.com> parents: 15218 diff changeset	572 response._close()
14376 a75e0f4ba0ab httpclient: import revision fc731618702a of py-nonblocking-http Augie Fackler <durin42@gmail.com> parents: 14341 diff changeset	573 # This if/elif ladder is a bit subtle,
a75e0f4ba0ab httpclient: import revision fc731618702a of py-nonblocking-http Augie Fackler <durin42@gmail.com> parents: 14341 diff changeset	574 # comments in each branch should help.
16643 24dbef11f477 httpclient: update to revision 892730fe7f46 of httpplus Augie Fackler <raf@durin42.com> parents: 15218 diff changeset	575 if response is not None and response.complete():
14376 a75e0f4ba0ab httpclient: import revision fc731618702a of py-nonblocking-http Augie Fackler <durin42@gmail.com> parents: 14341 diff changeset	576 # Server responded completely and then
a75e0f4ba0ab httpclient: import revision fc731618702a of py-nonblocking-http Augie Fackler <durin42@gmail.com> parents: 14341 diff changeset	577 # closed the socket. We should just shut
a75e0f4ba0ab httpclient: import revision fc731618702a of py-nonblocking-http Augie Fackler <durin42@gmail.com> parents: 14341 diff changeset	578 # things down and let the caller get their
a75e0f4ba0ab httpclient: import revision fc731618702a of py-nonblocking-http Augie Fackler <durin42@gmail.com> parents: 14341 diff changeset	579 # response.
a75e0f4ba0ab httpclient: import revision fc731618702a of py-nonblocking-http Augie Fackler <durin42@gmail.com> parents: 14341 diff changeset	580 logger.info('Got an early response, '
a75e0f4ba0ab httpclient: import revision fc731618702a of py-nonblocking-http Augie Fackler <durin42@gmail.com> parents: 14341 diff changeset	581 'aborting remaining request.')
a75e0f4ba0ab httpclient: import revision fc731618702a of py-nonblocking-http Augie Fackler <durin42@gmail.com> parents: 14341 diff changeset	582 break
a75e0f4ba0ab httpclient: import revision fc731618702a of py-nonblocking-http Augie Fackler <durin42@gmail.com> parents: 14341 diff changeset	583 elif was_first and response is None:
a75e0f4ba0ab httpclient: import revision fc731618702a of py-nonblocking-http Augie Fackler <durin42@gmail.com> parents: 14341 diff changeset	584 # Most likely a keepalive that got killed
a75e0f4ba0ab httpclient: import revision fc731618702a of py-nonblocking-http Augie Fackler <durin42@gmail.com> parents: 14341 diff changeset	585 # on the server's end. Commonly happens
a75e0f4ba0ab httpclient: import revision fc731618702a of py-nonblocking-http Augie Fackler <durin42@gmail.com> parents: 14341 diff changeset	586 # after getting a really large response
a75e0f4ba0ab httpclient: import revision fc731618702a of py-nonblocking-http Augie Fackler <durin42@gmail.com> parents: 14341 diff changeset	587 # from the server.
a75e0f4ba0ab httpclient: import revision fc731618702a of py-nonblocking-http Augie Fackler <durin42@gmail.com> parents: 14341 diff changeset	588 logger.info(
a75e0f4ba0ab httpclient: import revision fc731618702a of py-nonblocking-http Augie Fackler <durin42@gmail.com> parents: 14341 diff changeset	589 'Connection appeared closed in read on first'
a75e0f4ba0ab httpclient: import revision fc731618702a of py-nonblocking-http Augie Fackler <durin42@gmail.com> parents: 14341 diff changeset	590 ' request loop iteration, will retry.')
19182 fae47ecaa952 httpclient: upgrade to fe8c09e4db64 of httpplus Augie Fackler <raf@durin42.com> parents: 19038 diff changeset	591 self._reconnect('read')
14376 a75e0f4ba0ab httpclient: import revision fc731618702a of py-nonblocking-http Augie Fackler <durin42@gmail.com> parents: 14341 diff changeset	592 continue
a75e0f4ba0ab httpclient: import revision fc731618702a of py-nonblocking-http Augie Fackler <durin42@gmail.com> parents: 14341 diff changeset	593 else:
a75e0f4ba0ab httpclient: import revision fc731618702a of py-nonblocking-http Augie Fackler <durin42@gmail.com> parents: 14341 diff changeset	594 # We didn't just send the first data hunk,
a75e0f4ba0ab httpclient: import revision fc731618702a of py-nonblocking-http Augie Fackler <durin42@gmail.com> parents: 14341 diff changeset	595 # and either have a partial response or no
a75e0f4ba0ab httpclient: import revision fc731618702a of py-nonblocking-http Augie Fackler <durin42@gmail.com> parents: 14341 diff changeset	596 # response at all. There's really nothing
a75e0f4ba0ab httpclient: import revision fc731618702a of py-nonblocking-http Augie Fackler <durin42@gmail.com> parents: 14341 diff changeset	597 # meaningful we can do here.
a75e0f4ba0ab httpclient: import revision fc731618702a of py-nonblocking-http Augie Fackler <durin42@gmail.com> parents: 14341 diff changeset	598 raise HTTPStateError(
a75e0f4ba0ab httpclient: import revision fc731618702a of py-nonblocking-http Augie Fackler <durin42@gmail.com> parents: 14341 diff changeset	599 'Connection appears closed after '
a75e0f4ba0ab httpclient: import revision fc731618702a of py-nonblocking-http Augie Fackler <durin42@gmail.com> parents: 14341 diff changeset	600 'some request data was written, but the '
a75e0f4ba0ab httpclient: import revision fc731618702a of py-nonblocking-http Augie Fackler <durin42@gmail.com> parents: 14341 diff changeset	601 'response was missing or incomplete!')
a75e0f4ba0ab httpclient: import revision fc731618702a of py-nonblocking-http Augie Fackler <durin42@gmail.com> parents: 14341 diff changeset	602 logger.debug('read %d bytes in request()', len(data))
14243 861f28212398 Import new http library as mercurial.httpclient. Augie Fackler <durin42@gmail.com> parents: diff changeset	603 if response is None:
19182 fae47ecaa952 httpclient: upgrade to fe8c09e4db64 of httpplus Augie Fackler <raf@durin42.com> parents: 19038 diff changeset	604 response = self.response_class(
fae47ecaa952 httpclient: upgrade to fe8c09e4db64 of httpplus Augie Fackler <raf@durin42.com> parents: 19038 diff changeset	605 r[0], self.timeout, method)
fae47ecaa952 httpclient: upgrade to fe8c09e4db64 of httpplus Augie Fackler <raf@durin42.com> parents: 19038 diff changeset	606 # We're a friend of the response class, so let us
fae47ecaa952 httpclient: upgrade to fe8c09e4db64 of httpplus Augie Fackler <raf@durin42.com> parents: 19038 diff changeset	607 # use the private attribute.
fae47ecaa952 httpclient: upgrade to fe8c09e4db64 of httpplus Augie Fackler <raf@durin42.com> parents: 19038 diff changeset	608 # pylint: disable=W0212
14243 861f28212398 Import new http library as mercurial.httpclient. Augie Fackler <durin42@gmail.com> parents: diff changeset	609 response._load_response(data)
14376 a75e0f4ba0ab httpclient: import revision fc731618702a of py-nonblocking-http Augie Fackler <durin42@gmail.com> parents: 14341 diff changeset	610 # Jump to the next select() call so we load more
a75e0f4ba0ab httpclient: import revision fc731618702a of py-nonblocking-http Augie Fackler <durin42@gmail.com> parents: 14341 diff changeset	611 # data if the server is still sending us content.
a75e0f4ba0ab httpclient: import revision fc731618702a of py-nonblocking-http Augie Fackler <durin42@gmail.com> parents: 14341 diff changeset	612 continue
25660 328739ea70c3 global: mass rewrite to use modern exception syntax Gregory Szorc <gregory.szorc@gmail.com> parents: 24306 diff changeset	613 except socket.error as e:
14243 861f28212398 Import new http library as mercurial.httpclient. Augie Fackler <durin42@gmail.com> parents: diff changeset	614 if e[0] != errno.EPIPE and not was_first:
861f28212398 Import new http library as mercurial.httpclient. Augie Fackler <durin42@gmail.com> parents: diff changeset	615 raise
861f28212398 Import new http library as mercurial.httpclient. Augie Fackler <durin42@gmail.com> parents: diff changeset	616
861f28212398 Import new http library as mercurial.httpclient. Augie Fackler <durin42@gmail.com> parents: diff changeset	617 # outgoing data
861f28212398 Import new http library as mercurial.httpclient. Augie Fackler <durin42@gmail.com> parents: diff changeset	618 if w and out:
861f28212398 Import new http library as mercurial.httpclient. Augie Fackler <durin42@gmail.com> parents: diff changeset	619 try:
861f28212398 Import new http library as mercurial.httpclient. Augie Fackler <durin42@gmail.com> parents: diff changeset	620 if getattr(out, 'read', False):
19182 fae47ecaa952 httpclient: upgrade to fe8c09e4db64 of httpplus Augie Fackler <raf@durin42.com> parents: 19038 diff changeset	621 # pylint guesses the type of out incorrectly here
fae47ecaa952 httpclient: upgrade to fe8c09e4db64 of httpplus Augie Fackler <raf@durin42.com> parents: 19038 diff changeset	622 # pylint: disable=E1103
14243 861f28212398 Import new http library as mercurial.httpclient. Augie Fackler <durin42@gmail.com> parents: diff changeset	623 data = out.read(OUTGOING_BUFFER_SIZE)
861f28212398 Import new http library as mercurial.httpclient. Augie Fackler <durin42@gmail.com> parents: diff changeset	624 if not data:
861f28212398 Import new http library as mercurial.httpclient. Augie Fackler <durin42@gmail.com> parents: diff changeset	625 continue
861f28212398 Import new http library as mercurial.httpclient. Augie Fackler <durin42@gmail.com> parents: diff changeset	626 if len(data) < OUTGOING_BUFFER_SIZE:
861f28212398 Import new http library as mercurial.httpclient. Augie Fackler <durin42@gmail.com> parents: diff changeset	627 if chunked:
861f28212398 Import new http library as mercurial.httpclient. Augie Fackler <durin42@gmail.com> parents: diff changeset	628 body = '0' + EOL + EOL
861f28212398 Import new http library as mercurial.httpclient. Augie Fackler <durin42@gmail.com> parents: diff changeset	629 else:
861f28212398 Import new http library as mercurial.httpclient. Augie Fackler <durin42@gmail.com> parents: diff changeset	630 body = None
861f28212398 Import new http library as mercurial.httpclient. Augie Fackler <durin42@gmail.com> parents: diff changeset	631 if chunked:
861f28212398 Import new http library as mercurial.httpclient. Augie Fackler <durin42@gmail.com> parents: diff changeset	632 out = hex(len(data))[2:] + EOL + data + EOL
861f28212398 Import new http library as mercurial.httpclient. Augie Fackler <durin42@gmail.com> parents: diff changeset	633 else:
861f28212398 Import new http library as mercurial.httpclient. Augie Fackler <durin42@gmail.com> parents: diff changeset	634 out = data
861f28212398 Import new http library as mercurial.httpclient. Augie Fackler <durin42@gmail.com> parents: diff changeset	635 amt = w[0].send(out)
25660 328739ea70c3 global: mass rewrite to use modern exception syntax Gregory Szorc <gregory.szorc@gmail.com> parents: 24306 diff changeset	636 except socket.error as e:
14243 861f28212398 Import new http library as mercurial.httpclient. Augie Fackler <durin42@gmail.com> parents: diff changeset	637 if e[0] == socket.SSL_ERROR_WANT_WRITE and self.ssl:
861f28212398 Import new http library as mercurial.httpclient. Augie Fackler <durin42@gmail.com> parents: diff changeset	638 # This means that SSL hasn't flushed its buffer into
861f28212398 Import new http library as mercurial.httpclient. Augie Fackler <durin42@gmail.com> parents: diff changeset	639 # the socket yet.
861f28212398 Import new http library as mercurial.httpclient. Augie Fackler <durin42@gmail.com> parents: diff changeset	640 # TODO: find a way to block on ssl flushing its buffer
861f28212398 Import new http library as mercurial.httpclient. Augie Fackler <durin42@gmail.com> parents: diff changeset	641 # similar to selecting on a raw socket.
861f28212398 Import new http library as mercurial.httpclient. Augie Fackler <durin42@gmail.com> parents: diff changeset	642 continue
19489 42fcb2f7787d httpclient: update to revision 9517a2b56fe9 of httpplus (issue3905) Augie Fackler <raf@durin42.com> parents: 19182 diff changeset	643 if e[0] == errno.EWOULDBLOCK or e[0] == errno.EAGAIN:
42fcb2f7787d httpclient: update to revision 9517a2b56fe9 of httpplus (issue3905) Augie Fackler <raf@durin42.com> parents: 19182 diff changeset	644 continue
14243 861f28212398 Import new http library as mercurial.httpclient. Augie Fackler <durin42@gmail.com> parents: diff changeset	645 elif (e[0] not in (errno.ECONNRESET, errno.EPIPE)
861f28212398 Import new http library as mercurial.httpclient. Augie Fackler <durin42@gmail.com> parents: diff changeset	646 and not first):
861f28212398 Import new http library as mercurial.httpclient. Augie Fackler <durin42@gmail.com> parents: diff changeset	647 raise
19182 fae47ecaa952 httpclient: upgrade to fe8c09e4db64 of httpplus Augie Fackler <raf@durin42.com> parents: 19038 diff changeset	648 self._reconnect('write')
14243 861f28212398 Import new http library as mercurial.httpclient. Augie Fackler <durin42@gmail.com> parents: diff changeset	649 amt = self.sock.send(out)
861f28212398 Import new http library as mercurial.httpclient. Augie Fackler <durin42@gmail.com> parents: diff changeset	650 logger.debug('sent %d', amt)
861f28212398 Import new http library as mercurial.httpclient. Augie Fackler <durin42@gmail.com> parents: diff changeset	651 first = False
861f28212398 Import new http library as mercurial.httpclient. Augie Fackler <durin42@gmail.com> parents: diff changeset	652 if out is body:
861f28212398 Import new http library as mercurial.httpclient. Augie Fackler <durin42@gmail.com> parents: diff changeset	653 body = out[amt:]
861f28212398 Import new http library as mercurial.httpclient. Augie Fackler <durin42@gmail.com> parents: diff changeset	654 else:
861f28212398 Import new http library as mercurial.httpclient. Augie Fackler <durin42@gmail.com> parents: diff changeset	655 outgoing_headers = out[amt:]
861f28212398 Import new http library as mercurial.httpclient. Augie Fackler <durin42@gmail.com> parents: diff changeset	656
861f28212398 Import new http library as mercurial.httpclient. Augie Fackler <durin42@gmail.com> parents: diff changeset	657 # close if the server response said to or responded before eating
861f28212398 Import new http library as mercurial.httpclient. Augie Fackler <durin42@gmail.com> parents: diff changeset	658 # the whole request
861f28212398 Import new http library as mercurial.httpclient. Augie Fackler <durin42@gmail.com> parents: diff changeset	659 if response is None:
16643 24dbef11f477 httpclient: update to revision 892730fe7f46 of httpplus Augie Fackler <raf@durin42.com> parents: 15218 diff changeset	660 response = self.response_class(self.sock, self.timeout, method)
19620 d4a0055af149 httpclient: import 0d1b0a8bc549 to fix bug involving late-arriving RST after a response Augie Fackler <raf@durin42.com> parents: 19489 diff changeset	661 if not fresh_socket:
d4a0055af149 httpclient: import 0d1b0a8bc549 to fix bug involving late-arriving RST after a response Augie Fackler <raf@durin42.com> parents: 19489 diff changeset	662 if not response._select():
d4a0055af149 httpclient: import 0d1b0a8bc549 to fix bug involving late-arriving RST after a response Augie Fackler <raf@durin42.com> parents: 19489 diff changeset	663 # This means the response failed to get any response
d4a0055af149 httpclient: import 0d1b0a8bc549 to fix bug involving late-arriving RST after a response Augie Fackler <raf@durin42.com> parents: 19489 diff changeset	664 # data at all, and in all probability the socket was
d4a0055af149 httpclient: import 0d1b0a8bc549 to fix bug involving late-arriving RST after a response Augie Fackler <raf@durin42.com> parents: 19489 diff changeset	665 # closed before the server even saw our request. Try
d4a0055af149 httpclient: import 0d1b0a8bc549 to fix bug involving late-arriving RST after a response Augie Fackler <raf@durin42.com> parents: 19489 diff changeset	666 # the request again on a fresh socket.
d4a0055af149 httpclient: import 0d1b0a8bc549 to fix bug involving late-arriving RST after a response Augie Fackler <raf@durin42.com> parents: 19489 diff changeset	667 logging.debug('response._select() failed during request().'
d4a0055af149 httpclient: import 0d1b0a8bc549 to fix bug involving late-arriving RST after a response Augie Fackler <raf@durin42.com> parents: 19489 diff changeset	668 ' Assuming request needs to be retried.')
d4a0055af149 httpclient: import 0d1b0a8bc549 to fix bug involving late-arriving RST after a response Augie Fackler <raf@durin42.com> parents: 19489 diff changeset	669 self.sock = None
d4a0055af149 httpclient: import 0d1b0a8bc549 to fix bug involving late-arriving RST after a response Augie Fackler <raf@durin42.com> parents: 19489 diff changeset	670 # Call this method explicitly to re-try the
d4a0055af149 httpclient: import 0d1b0a8bc549 to fix bug involving late-arriving RST after a response Augie Fackler <raf@durin42.com> parents: 19489 diff changeset	671 # request. We don't use self.request() because
d4a0055af149 httpclient: import 0d1b0a8bc549 to fix bug involving late-arriving RST after a response Augie Fackler <raf@durin42.com> parents: 19489 diff changeset	672 # some tools (notably Mercurial) expect to be able
d4a0055af149 httpclient: import 0d1b0a8bc549 to fix bug involving late-arriving RST after a response Augie Fackler <raf@durin42.com> parents: 19489 diff changeset	673 # to subclass and redefine request(), and they
d4a0055af149 httpclient: import 0d1b0a8bc549 to fix bug involving late-arriving RST after a response Augie Fackler <raf@durin42.com> parents: 19489 diff changeset	674 # don't have the same argspec as we do.
d4a0055af149 httpclient: import 0d1b0a8bc549 to fix bug involving late-arriving RST after a response Augie Fackler <raf@durin42.com> parents: 19489 diff changeset	675 #
d4a0055af149 httpclient: import 0d1b0a8bc549 to fix bug involving late-arriving RST after a response Augie Fackler <raf@durin42.com> parents: 19489 diff changeset	676 # TODO restructure sending of requests to avoid
d4a0055af149 httpclient: import 0d1b0a8bc549 to fix bug involving late-arriving RST after a response Augie Fackler <raf@durin42.com> parents: 19489 diff changeset	677 # this recursion
d4a0055af149 httpclient: import 0d1b0a8bc549 to fix bug involving late-arriving RST after a response Augie Fackler <raf@durin42.com> parents: 19489 diff changeset	678 return HTTPConnection.request(
d4a0055af149 httpclient: import 0d1b0a8bc549 to fix bug involving late-arriving RST after a response Augie Fackler <raf@durin42.com> parents: 19489 diff changeset	679 self, method, path, body=body, headers=headers,
d4a0055af149 httpclient: import 0d1b0a8bc549 to fix bug involving late-arriving RST after a response Augie Fackler <raf@durin42.com> parents: 19489 diff changeset	680 expect_continue=expect_continue)
14243 861f28212398 Import new http library as mercurial.httpclient. Augie Fackler <durin42@gmail.com> parents: diff changeset	681 data_left = bool(outgoing_headers or body)
861f28212398 Import new http library as mercurial.httpclient. Augie Fackler <durin42@gmail.com> parents: diff changeset	682 if data_left:
861f28212398 Import new http library as mercurial.httpclient. Augie Fackler <durin42@gmail.com> parents: diff changeset	683 logger.info('stopped sending request early, '
861f28212398 Import new http library as mercurial.httpclient. Augie Fackler <durin42@gmail.com> parents: diff changeset	684 'will close the socket to be safe.')
861f28212398 Import new http library as mercurial.httpclient. Augie Fackler <durin42@gmail.com> parents: diff changeset	685 response.will_close = True
861f28212398 Import new http library as mercurial.httpclient. Augie Fackler <durin42@gmail.com> parents: diff changeset	686 if response.will_close:
861f28212398 Import new http library as mercurial.httpclient. Augie Fackler <durin42@gmail.com> parents: diff changeset	687 # The socket will be closed by the response, so we disown
861f28212398 Import new http library as mercurial.httpclient. Augie Fackler <durin42@gmail.com> parents: diff changeset	688 # the socket
861f28212398 Import new http library as mercurial.httpclient. Augie Fackler <durin42@gmail.com> parents: diff changeset	689 self.sock = None
861f28212398 Import new http library as mercurial.httpclient. Augie Fackler <durin42@gmail.com> parents: diff changeset	690 self._current_response = response
861f28212398 Import new http library as mercurial.httpclient. Augie Fackler <durin42@gmail.com> parents: diff changeset	691
861f28212398 Import new http library as mercurial.httpclient. Augie Fackler <durin42@gmail.com> parents: diff changeset	692 def getresponse(self):
19182 fae47ecaa952 httpclient: upgrade to fe8c09e4db64 of httpplus Augie Fackler <raf@durin42.com> parents: 19038 diff changeset	693 """Returns the response to the most recent request."""
14243 861f28212398 Import new http library as mercurial.httpclient. Augie Fackler <durin42@gmail.com> parents: diff changeset	694 if self._current_response is None:
861f28212398 Import new http library as mercurial.httpclient. Augie Fackler <durin42@gmail.com> parents: diff changeset	695 raise httplib.ResponseNotReady()
861f28212398 Import new http library as mercurial.httpclient. Augie Fackler <durin42@gmail.com> parents: diff changeset	696 r = self._current_response
861f28212398 Import new http library as mercurial.httpclient. Augie Fackler <durin42@gmail.com> parents: diff changeset	697 while r.headers is None:
19182 fae47ecaa952 httpclient: upgrade to fe8c09e4db64 of httpplus Augie Fackler <raf@durin42.com> parents: 19038 diff changeset	698 # We're a friend of the response class, so let us use the
fae47ecaa952 httpclient: upgrade to fe8c09e4db64 of httpplus Augie Fackler <raf@durin42.com> parents: 19038 diff changeset	699 # private attribute.
fae47ecaa952 httpclient: upgrade to fe8c09e4db64 of httpplus Augie Fackler <raf@durin42.com> parents: 19038 diff changeset	700 # pylint: disable=W0212
16643 24dbef11f477 httpclient: update to revision 892730fe7f46 of httpplus Augie Fackler <raf@durin42.com> parents: 15218 diff changeset	701 if not r._select() and not r.complete():
24dbef11f477 httpclient: update to revision 892730fe7f46 of httpplus Augie Fackler <raf@durin42.com> parents: 15218 diff changeset	702 raise _readers.HTTPRemoteClosedError()
14293 9adbb5ef0964 httpclient: import f4c380237fd5 to fix keepalive not working Augie Fackler <durin42@gmail.com> parents: 14243 diff changeset	703 if r.will_close:
14243 861f28212398 Import new http library as mercurial.httpclient. Augie Fackler <durin42@gmail.com> parents: diff changeset	704 self.sock = None
861f28212398 Import new http library as mercurial.httpclient. Augie Fackler <durin42@gmail.com> parents: diff changeset	705 self._current_response = None
14293 9adbb5ef0964 httpclient: import f4c380237fd5 to fix keepalive not working Augie Fackler <durin42@gmail.com> parents: 14243 diff changeset	706 elif r.complete():
9adbb5ef0964 httpclient: import f4c380237fd5 to fix keepalive not working Augie Fackler <durin42@gmail.com> parents: 14243 diff changeset	707 self._current_response = None
14243 861f28212398 Import new http library as mercurial.httpclient. Augie Fackler <durin42@gmail.com> parents: diff changeset	708 else:
861f28212398 Import new http library as mercurial.httpclient. Augie Fackler <durin42@gmail.com> parents: diff changeset	709 self._current_response_taken = True
861f28212398 Import new http library as mercurial.httpclient. Augie Fackler <durin42@gmail.com> parents: diff changeset	710 return r
861f28212398 Import new http library as mercurial.httpclient. Augie Fackler <durin42@gmail.com> parents: diff changeset	711
861f28212398 Import new http library as mercurial.httpclient. Augie Fackler <durin42@gmail.com> parents: diff changeset	712
861f28212398 Import new http library as mercurial.httpclient. Augie Fackler <durin42@gmail.com> parents: diff changeset	713 class HTTPTimeoutException(httplib.HTTPException):
861f28212398 Import new http library as mercurial.httpclient. Augie Fackler <durin42@gmail.com> parents: diff changeset	714 """A timeout occurred while waiting on the server."""
861f28212398 Import new http library as mercurial.httpclient. Augie Fackler <durin42@gmail.com> parents: diff changeset	715
861f28212398 Import new http library as mercurial.httpclient. Augie Fackler <durin42@gmail.com> parents: diff changeset	716
861f28212398 Import new http library as mercurial.httpclient. Augie Fackler <durin42@gmail.com> parents: diff changeset	717 class BadRequestData(httplib.HTTPException):
861f28212398 Import new http library as mercurial.httpclient. Augie Fackler <durin42@gmail.com> parents: diff changeset	718 """Request body object has neither __len__ nor read."""
861f28212398 Import new http library as mercurial.httpclient. Augie Fackler <durin42@gmail.com> parents: diff changeset	719
861f28212398 Import new http library as mercurial.httpclient. Augie Fackler <durin42@gmail.com> parents: diff changeset	720
861f28212398 Import new http library as mercurial.httpclient. Augie Fackler <durin42@gmail.com> parents: diff changeset	721 class HTTPProxyConnectFailedException(httplib.HTTPException):
861f28212398 Import new http library as mercurial.httpclient. Augie Fackler <durin42@gmail.com> parents: diff changeset	722 """Connecting to the HTTP proxy failed."""
14376 a75e0f4ba0ab httpclient: import revision fc731618702a of py-nonblocking-http Augie Fackler <durin42@gmail.com> parents: 14341 diff changeset	723
15218 c81dce8a7bb6 httpclient: update to 07d8c356f4d1 of py-nonblocking-http Augie Fackler <durin42@gmail.com> parents: 14376 diff changeset	724
14376 a75e0f4ba0ab httpclient: import revision fc731618702a of py-nonblocking-http Augie Fackler <durin42@gmail.com> parents: 14341 diff changeset	725 class HTTPStateError(httplib.HTTPException):
a75e0f4ba0ab httpclient: import revision fc731618702a of py-nonblocking-http Augie Fackler <durin42@gmail.com> parents: 14341 diff changeset	726 """Invalid internal state encountered."""
15218 c81dce8a7bb6 httpclient: update to 07d8c356f4d1 of py-nonblocking-http Augie Fackler <durin42@gmail.com> parents: 14376 diff changeset	727
16643 24dbef11f477 httpclient: update to revision 892730fe7f46 of httpplus Augie Fackler <raf@durin42.com> parents: 15218 diff changeset	728 # Forward this exception type from _readers since it needs to be part
24dbef11f477 httpclient: update to revision 892730fe7f46 of httpplus Augie Fackler <raf@durin42.com> parents: 15218 diff changeset	729 # of the public API.
24dbef11f477 httpclient: update to revision 892730fe7f46 of httpplus Augie Fackler <raf@durin42.com> parents: 15218 diff changeset	730 HTTPRemoteClosedError = _readers.HTTPRemoteClosedError
14243 861f28212398 Import new http library as mercurial.httpclient. Augie Fackler <durin42@gmail.com> parents: diff changeset	731 # no-check-code

Mercurial > public > mercurial-scm > hg

annotate mercurial/httpclient/__init__.py @ 26534:e4f27fb65da7

annotate mercurial/httpclient/init.py @ 26534:e4f27fb65da7