mercurial-scm/hg: mercurial/httpclient/__init_

annotate mercurial/httpclient/init.py @ 15218:c81dce8a7bb6

httpclient: update to 07d8c356f4d1 of py-nonblocking-http This addresses a defect when the server closes the socket before finishing a response (if it crashes, for example) first spotted in Issue2951.

author	Augie Fackler <durin42@gmail.com>
date	Mon, 10 Oct 2011 17:57:40 -0500
parents	a75e0f4ba0ab
children	24dbef11f477

rev	line source
14243 861f28212398 Import new http library as mercurial.httpclient. Augie Fackler <durin42@gmail.com> parents: diff changeset	1 # Copyright 2010, Google Inc.
861f28212398 Import new http library as mercurial.httpclient. Augie Fackler <durin42@gmail.com> parents: diff changeset	2 # All rights reserved.
861f28212398 Import new http library as mercurial.httpclient. Augie Fackler <durin42@gmail.com> parents: diff changeset	3 #
861f28212398 Import new http library as mercurial.httpclient. Augie Fackler <durin42@gmail.com> parents: diff changeset	4 # Redistribution and use in source and binary forms, with or without
861f28212398 Import new http library as mercurial.httpclient. Augie Fackler <durin42@gmail.com> parents: diff changeset	5 # modification, are permitted provided that the following conditions are
861f28212398 Import new http library as mercurial.httpclient. Augie Fackler <durin42@gmail.com> parents: diff changeset	6 # met:
861f28212398 Import new http library as mercurial.httpclient. Augie Fackler <durin42@gmail.com> parents: diff changeset	7 #
861f28212398 Import new http library as mercurial.httpclient. Augie Fackler <durin42@gmail.com> parents: diff changeset	8 # * Redistributions of source code must retain the above copyright
861f28212398 Import new http library as mercurial.httpclient. Augie Fackler <durin42@gmail.com> parents: diff changeset	9 # notice, this list of conditions and the following disclaimer.
861f28212398 Import new http library as mercurial.httpclient. Augie Fackler <durin42@gmail.com> parents: diff changeset	10 # * Redistributions in binary form must reproduce the above
861f28212398 Import new http library as mercurial.httpclient. Augie Fackler <durin42@gmail.com> parents: diff changeset	11 # copyright notice, this list of conditions and the following disclaimer
861f28212398 Import new http library as mercurial.httpclient. Augie Fackler <durin42@gmail.com> parents: diff changeset	12 # in the documentation and/or other materials provided with the
861f28212398 Import new http library as mercurial.httpclient. Augie Fackler <durin42@gmail.com> parents: diff changeset	13 # distribution.
861f28212398 Import new http library as mercurial.httpclient. Augie Fackler <durin42@gmail.com> parents: diff changeset	14 # * Neither the name of Google Inc. nor the names of its
861f28212398 Import new http library as mercurial.httpclient. Augie Fackler <durin42@gmail.com> parents: diff changeset	15 # contributors may be used to endorse or promote products derived from
861f28212398 Import new http library as mercurial.httpclient. Augie Fackler <durin42@gmail.com> parents: diff changeset	16 # this software without specific prior written permission.
861f28212398 Import new http library as mercurial.httpclient. Augie Fackler <durin42@gmail.com> parents: diff changeset	17
861f28212398 Import new http library as mercurial.httpclient. Augie Fackler <durin42@gmail.com> parents: diff changeset	18 # THIS SOFTWARE IS PROVIDED BY THE COPYRIGHT HOLDERS AND CONTRIBUTORS
861f28212398 Import new http library as mercurial.httpclient. Augie Fackler <durin42@gmail.com> parents: diff changeset	19 # "AS IS" AND ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT
861f28212398 Import new http library as mercurial.httpclient. Augie Fackler <durin42@gmail.com> parents: diff changeset	20 # LIMITED TO, THE IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR
861f28212398 Import new http library as mercurial.httpclient. Augie Fackler <durin42@gmail.com> parents: diff changeset	21 # A PARTICULAR PURPOSE ARE DISCLAIMED. IN NO EVENT SHALL THE COPYRIGHT
861f28212398 Import new http library as mercurial.httpclient. Augie Fackler <durin42@gmail.com> parents: diff changeset	22 # OWNER OR CONTRIBUTORS BE LIABLE FOR ANY DIRECT, INDIRECT, INCIDENTAL,
861f28212398 Import new http library as mercurial.httpclient. Augie Fackler <durin42@gmail.com> parents: diff changeset	23 # SPECIAL, EXEMPLARY, OR CONSEQUENTIAL DAMAGES (INCLUDING, BUT NOT
861f28212398 Import new http library as mercurial.httpclient. Augie Fackler <durin42@gmail.com> parents: diff changeset	24 # LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS OR SERVICES; LOSS OF USE,
861f28212398 Import new http library as mercurial.httpclient. Augie Fackler <durin42@gmail.com> parents: diff changeset	25 # DATA, OR PROFITS; OR BUSINESS INTERRUPTION) HOWEVER CAUSED AND ON ANY
861f28212398 Import new http library as mercurial.httpclient. Augie Fackler <durin42@gmail.com> parents: diff changeset	26 # THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY, OR TORT
861f28212398 Import new http library as mercurial.httpclient. Augie Fackler <durin42@gmail.com> parents: diff changeset	27 # (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF THE USE
861f28212398 Import new http library as mercurial.httpclient. Augie Fackler <durin42@gmail.com> parents: diff changeset	28 # OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE.
861f28212398 Import new http library as mercurial.httpclient. Augie Fackler <durin42@gmail.com> parents: diff changeset	29 """Improved HTTP/1.1 client library
861f28212398 Import new http library as mercurial.httpclient. Augie Fackler <durin42@gmail.com> parents: diff changeset	30
861f28212398 Import new http library as mercurial.httpclient. Augie Fackler <durin42@gmail.com> parents: diff changeset	31 This library contains an HTTPConnection which is similar to the one in
861f28212398 Import new http library as mercurial.httpclient. Augie Fackler <durin42@gmail.com> parents: diff changeset	32 httplib, but has several additional features:
861f28212398 Import new http library as mercurial.httpclient. Augie Fackler <durin42@gmail.com> parents: diff changeset	33
861f28212398 Import new http library as mercurial.httpclient. Augie Fackler <durin42@gmail.com> parents: diff changeset	34 * supports keepalives natively
861f28212398 Import new http library as mercurial.httpclient. Augie Fackler <durin42@gmail.com> parents: diff changeset	35 * uses select() to block for incoming data
861f28212398 Import new http library as mercurial.httpclient. Augie Fackler <durin42@gmail.com> parents: diff changeset	36 * notices when the server responds early to a request
861f28212398 Import new http library as mercurial.httpclient. Augie Fackler <durin42@gmail.com> parents: diff changeset	37 * implements ssl inline instead of in a different class
861f28212398 Import new http library as mercurial.httpclient. Augie Fackler <durin42@gmail.com> parents: diff changeset	38 """
861f28212398 Import new http library as mercurial.httpclient. Augie Fackler <durin42@gmail.com> parents: diff changeset	39
861f28212398 Import new http library as mercurial.httpclient. Augie Fackler <durin42@gmail.com> parents: diff changeset	40 import cStringIO
861f28212398 Import new http library as mercurial.httpclient. Augie Fackler <durin42@gmail.com> parents: diff changeset	41 import errno
861f28212398 Import new http library as mercurial.httpclient. Augie Fackler <durin42@gmail.com> parents: diff changeset	42 import httplib
861f28212398 Import new http library as mercurial.httpclient. Augie Fackler <durin42@gmail.com> parents: diff changeset	43 import logging
861f28212398 Import new http library as mercurial.httpclient. Augie Fackler <durin42@gmail.com> parents: diff changeset	44 import rfc822
861f28212398 Import new http library as mercurial.httpclient. Augie Fackler <durin42@gmail.com> parents: diff changeset	45 import select
861f28212398 Import new http library as mercurial.httpclient. Augie Fackler <durin42@gmail.com> parents: diff changeset	46 import socket
861f28212398 Import new http library as mercurial.httpclient. Augie Fackler <durin42@gmail.com> parents: diff changeset	47
861f28212398 Import new http library as mercurial.httpclient. Augie Fackler <durin42@gmail.com> parents: diff changeset	48 import socketutil
861f28212398 Import new http library as mercurial.httpclient. Augie Fackler <durin42@gmail.com> parents: diff changeset	49
861f28212398 Import new http library as mercurial.httpclient. Augie Fackler <durin42@gmail.com> parents: diff changeset	50 logger = logging.getLogger(__name__)
861f28212398 Import new http library as mercurial.httpclient. Augie Fackler <durin42@gmail.com> parents: diff changeset	51
861f28212398 Import new http library as mercurial.httpclient. Augie Fackler <durin42@gmail.com> parents: diff changeset	52 __all__ = ['HTTPConnection', 'HTTPResponse']
861f28212398 Import new http library as mercurial.httpclient. Augie Fackler <durin42@gmail.com> parents: diff changeset	53
861f28212398 Import new http library as mercurial.httpclient. Augie Fackler <durin42@gmail.com> parents: diff changeset	54 HTTP_VER_1_0 = 'HTTP/1.0'
861f28212398 Import new http library as mercurial.httpclient. Augie Fackler <durin42@gmail.com> parents: diff changeset	55 HTTP_VER_1_1 = 'HTTP/1.1'
861f28212398 Import new http library as mercurial.httpclient. Augie Fackler <durin42@gmail.com> parents: diff changeset	56
861f28212398 Import new http library as mercurial.httpclient. Augie Fackler <durin42@gmail.com> parents: diff changeset	57 _LEN_CLOSE_IS_END = -1
861f28212398 Import new http library as mercurial.httpclient. Augie Fackler <durin42@gmail.com> parents: diff changeset	58
861f28212398 Import new http library as mercurial.httpclient. Augie Fackler <durin42@gmail.com> parents: diff changeset	59 OUTGOING_BUFFER_SIZE = 1 << 15
861f28212398 Import new http library as mercurial.httpclient. Augie Fackler <durin42@gmail.com> parents: diff changeset	60 INCOMING_BUFFER_SIZE = 1 << 20
861f28212398 Import new http library as mercurial.httpclient. Augie Fackler <durin42@gmail.com> parents: diff changeset	61
861f28212398 Import new http library as mercurial.httpclient. Augie Fackler <durin42@gmail.com> parents: diff changeset	62 HDR_ACCEPT_ENCODING = 'accept-encoding'
861f28212398 Import new http library as mercurial.httpclient. Augie Fackler <durin42@gmail.com> parents: diff changeset	63 HDR_CONNECTION_CTRL = 'connection'
861f28212398 Import new http library as mercurial.httpclient. Augie Fackler <durin42@gmail.com> parents: diff changeset	64 HDR_CONTENT_LENGTH = 'content-length'
861f28212398 Import new http library as mercurial.httpclient. Augie Fackler <durin42@gmail.com> parents: diff changeset	65 HDR_XFER_ENCODING = 'transfer-encoding'
861f28212398 Import new http library as mercurial.httpclient. Augie Fackler <durin42@gmail.com> parents: diff changeset	66
861f28212398 Import new http library as mercurial.httpclient. Augie Fackler <durin42@gmail.com> parents: diff changeset	67 XFER_ENCODING_CHUNKED = 'chunked'
861f28212398 Import new http library as mercurial.httpclient. Augie Fackler <durin42@gmail.com> parents: diff changeset	68
861f28212398 Import new http library as mercurial.httpclient. Augie Fackler <durin42@gmail.com> parents: diff changeset	69 CONNECTION_CLOSE = 'close'
861f28212398 Import new http library as mercurial.httpclient. Augie Fackler <durin42@gmail.com> parents: diff changeset	70
861f28212398 Import new http library as mercurial.httpclient. Augie Fackler <durin42@gmail.com> parents: diff changeset	71 EOL = '\r\n'
861f28212398 Import new http library as mercurial.httpclient. Augie Fackler <durin42@gmail.com> parents: diff changeset	72 _END_HEADERS = EOL * 2
861f28212398 Import new http library as mercurial.httpclient. Augie Fackler <durin42@gmail.com> parents: diff changeset	73
861f28212398 Import new http library as mercurial.httpclient. Augie Fackler <durin42@gmail.com> parents: diff changeset	74 # Based on some searching around, 1 second seems like a reasonable
861f28212398 Import new http library as mercurial.httpclient. Augie Fackler <durin42@gmail.com> parents: diff changeset	75 # default here.
861f28212398 Import new http library as mercurial.httpclient. Augie Fackler <durin42@gmail.com> parents: diff changeset	76 TIMEOUT_ASSUME_CONTINUE = 1
861f28212398 Import new http library as mercurial.httpclient. Augie Fackler <durin42@gmail.com> parents: diff changeset	77 TIMEOUT_DEFAULT = None
861f28212398 Import new http library as mercurial.httpclient. Augie Fackler <durin42@gmail.com> parents: diff changeset	78
861f28212398 Import new http library as mercurial.httpclient. Augie Fackler <durin42@gmail.com> parents: diff changeset	79
861f28212398 Import new http library as mercurial.httpclient. Augie Fackler <durin42@gmail.com> parents: diff changeset	80 class HTTPResponse(object):
861f28212398 Import new http library as mercurial.httpclient. Augie Fackler <durin42@gmail.com> parents: diff changeset	81 """Response from an HTTP server.
861f28212398 Import new http library as mercurial.httpclient. Augie Fackler <durin42@gmail.com> parents: diff changeset	82
861f28212398 Import new http library as mercurial.httpclient. Augie Fackler <durin42@gmail.com> parents: diff changeset	83 The response will continue to load as available. If you need the
861f28212398 Import new http library as mercurial.httpclient. Augie Fackler <durin42@gmail.com> parents: diff changeset	84 complete response before continuing, check the .complete() method.
861f28212398 Import new http library as mercurial.httpclient. Augie Fackler <durin42@gmail.com> parents: diff changeset	85 """
861f28212398 Import new http library as mercurial.httpclient. Augie Fackler <durin42@gmail.com> parents: diff changeset	86 def __init__(self, sock, timeout):
861f28212398 Import new http library as mercurial.httpclient. Augie Fackler <durin42@gmail.com> parents: diff changeset	87 self.sock = sock
861f28212398 Import new http library as mercurial.httpclient. Augie Fackler <durin42@gmail.com> parents: diff changeset	88 self.raw_response = ''
861f28212398 Import new http library as mercurial.httpclient. Augie Fackler <durin42@gmail.com> parents: diff changeset	89 self._body = None
861f28212398 Import new http library as mercurial.httpclient. Augie Fackler <durin42@gmail.com> parents: diff changeset	90 self._headers_len = 0
861f28212398 Import new http library as mercurial.httpclient. Augie Fackler <durin42@gmail.com> parents: diff changeset	91 self._content_len = 0
861f28212398 Import new http library as mercurial.httpclient. Augie Fackler <durin42@gmail.com> parents: diff changeset	92 self.headers = None
861f28212398 Import new http library as mercurial.httpclient. Augie Fackler <durin42@gmail.com> parents: diff changeset	93 self.will_close = False
861f28212398 Import new http library as mercurial.httpclient. Augie Fackler <durin42@gmail.com> parents: diff changeset	94 self.status_line = ''
861f28212398 Import new http library as mercurial.httpclient. Augie Fackler <durin42@gmail.com> parents: diff changeset	95 self.status = None
861f28212398 Import new http library as mercurial.httpclient. Augie Fackler <durin42@gmail.com> parents: diff changeset	96 self.http_version = None
861f28212398 Import new http library as mercurial.httpclient. Augie Fackler <durin42@gmail.com> parents: diff changeset	97 self.reason = None
861f28212398 Import new http library as mercurial.httpclient. Augie Fackler <durin42@gmail.com> parents: diff changeset	98 self._chunked = False
861f28212398 Import new http library as mercurial.httpclient. Augie Fackler <durin42@gmail.com> parents: diff changeset	99 self._chunked_done = False
861f28212398 Import new http library as mercurial.httpclient. Augie Fackler <durin42@gmail.com> parents: diff changeset	100 self._chunked_until_next = 0
861f28212398 Import new http library as mercurial.httpclient. Augie Fackler <durin42@gmail.com> parents: diff changeset	101 self._chunked_skip_bytes = 0
861f28212398 Import new http library as mercurial.httpclient. Augie Fackler <durin42@gmail.com> parents: diff changeset	102 self._chunked_preloaded_block = None
861f28212398 Import new http library as mercurial.httpclient. Augie Fackler <durin42@gmail.com> parents: diff changeset	103
861f28212398 Import new http library as mercurial.httpclient. Augie Fackler <durin42@gmail.com> parents: diff changeset	104 self._read_location = 0
861f28212398 Import new http library as mercurial.httpclient. Augie Fackler <durin42@gmail.com> parents: diff changeset	105 self._eol = EOL
861f28212398 Import new http library as mercurial.httpclient. Augie Fackler <durin42@gmail.com> parents: diff changeset	106
861f28212398 Import new http library as mercurial.httpclient. Augie Fackler <durin42@gmail.com> parents: diff changeset	107 self._timeout = timeout
861f28212398 Import new http library as mercurial.httpclient. Augie Fackler <durin42@gmail.com> parents: diff changeset	108
861f28212398 Import new http library as mercurial.httpclient. Augie Fackler <durin42@gmail.com> parents: diff changeset	109 @property
861f28212398 Import new http library as mercurial.httpclient. Augie Fackler <durin42@gmail.com> parents: diff changeset	110 def _end_headers(self):
861f28212398 Import new http library as mercurial.httpclient. Augie Fackler <durin42@gmail.com> parents: diff changeset	111 return self._eol * 2
861f28212398 Import new http library as mercurial.httpclient. Augie Fackler <durin42@gmail.com> parents: diff changeset	112
861f28212398 Import new http library as mercurial.httpclient. Augie Fackler <durin42@gmail.com> parents: diff changeset	113 def complete(self):
861f28212398 Import new http library as mercurial.httpclient. Augie Fackler <durin42@gmail.com> parents: diff changeset	114 """Returns true if this response is completely loaded.
14376 a75e0f4ba0ab httpclient: import revision fc731618702a of py-nonblocking-http Augie Fackler <durin42@gmail.com> parents: 14341 diff changeset	115
a75e0f4ba0ab httpclient: import revision fc731618702a of py-nonblocking-http Augie Fackler <durin42@gmail.com> parents: 14341 diff changeset	116 Note that if this is a connection where complete means the
a75e0f4ba0ab httpclient: import revision fc731618702a of py-nonblocking-http Augie Fackler <durin42@gmail.com> parents: 14341 diff changeset	117 socket is closed, this will nearly always return False, even
a75e0f4ba0ab httpclient: import revision fc731618702a of py-nonblocking-http Augie Fackler <durin42@gmail.com> parents: 14341 diff changeset	118 in cases where all the data has actually been loaded.
14243 861f28212398 Import new http library as mercurial.httpclient. Augie Fackler <durin42@gmail.com> parents: diff changeset	119 """
861f28212398 Import new http library as mercurial.httpclient. Augie Fackler <durin42@gmail.com> parents: diff changeset	120 if self._chunked:
861f28212398 Import new http library as mercurial.httpclient. Augie Fackler <durin42@gmail.com> parents: diff changeset	121 return self._chunked_done
861f28212398 Import new http library as mercurial.httpclient. Augie Fackler <durin42@gmail.com> parents: diff changeset	122 if self._content_len == _LEN_CLOSE_IS_END:
861f28212398 Import new http library as mercurial.httpclient. Augie Fackler <durin42@gmail.com> parents: diff changeset	123 return False
861f28212398 Import new http library as mercurial.httpclient. Augie Fackler <durin42@gmail.com> parents: diff changeset	124 return self._body is not None and len(self._body) >= self._content_len
861f28212398 Import new http library as mercurial.httpclient. Augie Fackler <durin42@gmail.com> parents: diff changeset	125
861f28212398 Import new http library as mercurial.httpclient. Augie Fackler <durin42@gmail.com> parents: diff changeset	126 def readline(self):
861f28212398 Import new http library as mercurial.httpclient. Augie Fackler <durin42@gmail.com> parents: diff changeset	127 """Read a single line from the response body.
861f28212398 Import new http library as mercurial.httpclient. Augie Fackler <durin42@gmail.com> parents: diff changeset	128
861f28212398 Import new http library as mercurial.httpclient. Augie Fackler <durin42@gmail.com> parents: diff changeset	129 This may block until either a line ending is found or the
861f28212398 Import new http library as mercurial.httpclient. Augie Fackler <durin42@gmail.com> parents: diff changeset	130 response is complete.
861f28212398 Import new http library as mercurial.httpclient. Augie Fackler <durin42@gmail.com> parents: diff changeset	131 """
861f28212398 Import new http library as mercurial.httpclient. Augie Fackler <durin42@gmail.com> parents: diff changeset	132 eol = self._body.find('\n', self._read_location)
861f28212398 Import new http library as mercurial.httpclient. Augie Fackler <durin42@gmail.com> parents: diff changeset	133 while eol == -1 and not self.complete():
861f28212398 Import new http library as mercurial.httpclient. Augie Fackler <durin42@gmail.com> parents: diff changeset	134 self._select()
861f28212398 Import new http library as mercurial.httpclient. Augie Fackler <durin42@gmail.com> parents: diff changeset	135 eol = self._body.find('\n', self._read_location)
861f28212398 Import new http library as mercurial.httpclient. Augie Fackler <durin42@gmail.com> parents: diff changeset	136 if eol != -1:
861f28212398 Import new http library as mercurial.httpclient. Augie Fackler <durin42@gmail.com> parents: diff changeset	137 eol += 1
861f28212398 Import new http library as mercurial.httpclient. Augie Fackler <durin42@gmail.com> parents: diff changeset	138 else:
861f28212398 Import new http library as mercurial.httpclient. Augie Fackler <durin42@gmail.com> parents: diff changeset	139 eol = len(self._body)
861f28212398 Import new http library as mercurial.httpclient. Augie Fackler <durin42@gmail.com> parents: diff changeset	140 data = self._body[self._read_location:eol]
861f28212398 Import new http library as mercurial.httpclient. Augie Fackler <durin42@gmail.com> parents: diff changeset	141 self._read_location = eol
861f28212398 Import new http library as mercurial.httpclient. Augie Fackler <durin42@gmail.com> parents: diff changeset	142 return data
861f28212398 Import new http library as mercurial.httpclient. Augie Fackler <durin42@gmail.com> parents: diff changeset	143
861f28212398 Import new http library as mercurial.httpclient. Augie Fackler <durin42@gmail.com> parents: diff changeset	144 def read(self, length=None):
861f28212398 Import new http library as mercurial.httpclient. Augie Fackler <durin42@gmail.com> parents: diff changeset	145 # if length is None, unbounded read
861f28212398 Import new http library as mercurial.httpclient. Augie Fackler <durin42@gmail.com> parents: diff changeset	146 while (not self.complete() # never select on a finished read
861f28212398 Import new http library as mercurial.httpclient. Augie Fackler <durin42@gmail.com> parents: diff changeset	147 and (not length # unbounded, so we wait for complete()
861f28212398 Import new http library as mercurial.httpclient. Augie Fackler <durin42@gmail.com> parents: diff changeset	148 or (self._read_location + length) > len(self._body))):
861f28212398 Import new http library as mercurial.httpclient. Augie Fackler <durin42@gmail.com> parents: diff changeset	149 self._select()
861f28212398 Import new http library as mercurial.httpclient. Augie Fackler <durin42@gmail.com> parents: diff changeset	150 if not length:
861f28212398 Import new http library as mercurial.httpclient. Augie Fackler <durin42@gmail.com> parents: diff changeset	151 length = len(self._body) - self._read_location
861f28212398 Import new http library as mercurial.httpclient. Augie Fackler <durin42@gmail.com> parents: diff changeset	152 elif len(self._body) < (self._read_location + length):
861f28212398 Import new http library as mercurial.httpclient. Augie Fackler <durin42@gmail.com> parents: diff changeset	153 length = len(self._body) - self._read_location
861f28212398 Import new http library as mercurial.httpclient. Augie Fackler <durin42@gmail.com> parents: diff changeset	154 r = self._body[self._read_location:self._read_location + length]
861f28212398 Import new http library as mercurial.httpclient. Augie Fackler <durin42@gmail.com> parents: diff changeset	155 self._read_location += len(r)
861f28212398 Import new http library as mercurial.httpclient. Augie Fackler <durin42@gmail.com> parents: diff changeset	156 if self.complete() and self.will_close:
861f28212398 Import new http library as mercurial.httpclient. Augie Fackler <durin42@gmail.com> parents: diff changeset	157 self.sock.close()
861f28212398 Import new http library as mercurial.httpclient. Augie Fackler <durin42@gmail.com> parents: diff changeset	158 return r
861f28212398 Import new http library as mercurial.httpclient. Augie Fackler <durin42@gmail.com> parents: diff changeset	159
861f28212398 Import new http library as mercurial.httpclient. Augie Fackler <durin42@gmail.com> parents: diff changeset	160 def _select(self):
861f28212398 Import new http library as mercurial.httpclient. Augie Fackler <durin42@gmail.com> parents: diff changeset	161 r, _, _ = select.select([self.sock], [], [], self._timeout)
861f28212398 Import new http library as mercurial.httpclient. Augie Fackler <durin42@gmail.com> parents: diff changeset	162 if not r:
861f28212398 Import new http library as mercurial.httpclient. Augie Fackler <durin42@gmail.com> parents: diff changeset	163 # socket was not readable. If the response is not complete
861f28212398 Import new http library as mercurial.httpclient. Augie Fackler <durin42@gmail.com> parents: diff changeset	164 # and we're not a _LEN_CLOSE_IS_END response, raise a timeout.
861f28212398 Import new http library as mercurial.httpclient. Augie Fackler <durin42@gmail.com> parents: diff changeset	165 # If we are a _LEN_CLOSE_IS_END response and we have no data,
861f28212398 Import new http library as mercurial.httpclient. Augie Fackler <durin42@gmail.com> parents: diff changeset	166 # raise a timeout.
861f28212398 Import new http library as mercurial.httpclient. Augie Fackler <durin42@gmail.com> parents: diff changeset	167 if not (self.complete() or
861f28212398 Import new http library as mercurial.httpclient. Augie Fackler <durin42@gmail.com> parents: diff changeset	168 (self._content_len == _LEN_CLOSE_IS_END and self._body)):
861f28212398 Import new http library as mercurial.httpclient. Augie Fackler <durin42@gmail.com> parents: diff changeset	169 logger.info('timed out with timeout of %s', self._timeout)
861f28212398 Import new http library as mercurial.httpclient. Augie Fackler <durin42@gmail.com> parents: diff changeset	170 raise HTTPTimeoutException('timeout reading data')
861f28212398 Import new http library as mercurial.httpclient. Augie Fackler <durin42@gmail.com> parents: diff changeset	171 logger.info('cl: %r body: %r', self._content_len, self._body)
14341 5c3de67e7402 httpclient: import revision b8c3511a8cae from py-nonblocking-http Augie Fackler <durin42@gmail.com> parents: 14293 diff changeset	172 try:
5c3de67e7402 httpclient: import revision b8c3511a8cae from py-nonblocking-http Augie Fackler <durin42@gmail.com> parents: 14293 diff changeset	173 data = self.sock.recv(INCOMING_BUFFER_SIZE)
15218 c81dce8a7bb6 httpclient: update to 07d8c356f4d1 of py-nonblocking-http Augie Fackler <durin42@gmail.com> parents: 14376 diff changeset	174 # If the socket was readable and no data was read, that
c81dce8a7bb6 httpclient: update to 07d8c356f4d1 of py-nonblocking-http Augie Fackler <durin42@gmail.com> parents: 14376 diff changeset	175 # means the socket was closed. If this isn't a
c81dce8a7bb6 httpclient: update to 07d8c356f4d1 of py-nonblocking-http Augie Fackler <durin42@gmail.com> parents: 14376 diff changeset	176 # _CLOSE_IS_END socket, then something is wrong if we're
c81dce8a7bb6 httpclient: update to 07d8c356f4d1 of py-nonblocking-http Augie Fackler <durin42@gmail.com> parents: 14376 diff changeset	177 # here (we shouldn't enter _select() if the response is
c81dce8a7bb6 httpclient: update to 07d8c356f4d1 of py-nonblocking-http Augie Fackler <durin42@gmail.com> parents: 14376 diff changeset	178 # complete), so abort.
c81dce8a7bb6 httpclient: update to 07d8c356f4d1 of py-nonblocking-http Augie Fackler <durin42@gmail.com> parents: 14376 diff changeset	179 if not data and self._content_len != _LEN_CLOSE_IS_END:
c81dce8a7bb6 httpclient: update to 07d8c356f4d1 of py-nonblocking-http Augie Fackler <durin42@gmail.com> parents: 14376 diff changeset	180 raise HTTPRemoteClosedError(
c81dce8a7bb6 httpclient: update to 07d8c356f4d1 of py-nonblocking-http Augie Fackler <durin42@gmail.com> parents: 14376 diff changeset	181 'server appears to have closed the socket mid-response')
14341 5c3de67e7402 httpclient: import revision b8c3511a8cae from py-nonblocking-http Augie Fackler <durin42@gmail.com> parents: 14293 diff changeset	182 except socket.sslerror, e:
5c3de67e7402 httpclient: import revision b8c3511a8cae from py-nonblocking-http Augie Fackler <durin42@gmail.com> parents: 14293 diff changeset	183 if e.args[0] != socket.SSL_ERROR_WANT_READ:
5c3de67e7402 httpclient: import revision b8c3511a8cae from py-nonblocking-http Augie Fackler <durin42@gmail.com> parents: 14293 diff changeset	184 raise
5c3de67e7402 httpclient: import revision b8c3511a8cae from py-nonblocking-http Augie Fackler <durin42@gmail.com> parents: 14293 diff changeset	185 logger.debug('SSL_WANT_READ in _select, should retry later')
5c3de67e7402 httpclient: import revision b8c3511a8cae from py-nonblocking-http Augie Fackler <durin42@gmail.com> parents: 14293 diff changeset	186 return True
14243 861f28212398 Import new http library as mercurial.httpclient. Augie Fackler <durin42@gmail.com> parents: diff changeset	187 logger.debug('response read %d data during _select', len(data))
861f28212398 Import new http library as mercurial.httpclient. Augie Fackler <durin42@gmail.com> parents: diff changeset	188 if not data:
14376 a75e0f4ba0ab httpclient: import revision fc731618702a of py-nonblocking-http Augie Fackler <durin42@gmail.com> parents: 14341 diff changeset	189 if self.headers and self._content_len == _LEN_CLOSE_IS_END:
14243 861f28212398 Import new http library as mercurial.httpclient. Augie Fackler <durin42@gmail.com> parents: diff changeset	190 self._content_len = len(self._body)
861f28212398 Import new http library as mercurial.httpclient. Augie Fackler <durin42@gmail.com> parents: diff changeset	191 return False
861f28212398 Import new http library as mercurial.httpclient. Augie Fackler <durin42@gmail.com> parents: diff changeset	192 else:
861f28212398 Import new http library as mercurial.httpclient. Augie Fackler <durin42@gmail.com> parents: diff changeset	193 self._load_response(data)
861f28212398 Import new http library as mercurial.httpclient. Augie Fackler <durin42@gmail.com> parents: diff changeset	194 return True
861f28212398 Import new http library as mercurial.httpclient. Augie Fackler <durin42@gmail.com> parents: diff changeset	195
861f28212398 Import new http library as mercurial.httpclient. Augie Fackler <durin42@gmail.com> parents: diff changeset	196 def _chunked_parsedata(self, data):
861f28212398 Import new http library as mercurial.httpclient. Augie Fackler <durin42@gmail.com> parents: diff changeset	197 if self._chunked_preloaded_block:
861f28212398 Import new http library as mercurial.httpclient. Augie Fackler <durin42@gmail.com> parents: diff changeset	198 data = self._chunked_preloaded_block + data
861f28212398 Import new http library as mercurial.httpclient. Augie Fackler <durin42@gmail.com> parents: diff changeset	199 self._chunked_preloaded_block = None
861f28212398 Import new http library as mercurial.httpclient. Augie Fackler <durin42@gmail.com> parents: diff changeset	200 while data:
861f28212398 Import new http library as mercurial.httpclient. Augie Fackler <durin42@gmail.com> parents: diff changeset	201 logger.debug('looping with %d data remaining', len(data))
861f28212398 Import new http library as mercurial.httpclient. Augie Fackler <durin42@gmail.com> parents: diff changeset	202 # Slice out anything we should skip
861f28212398 Import new http library as mercurial.httpclient. Augie Fackler <durin42@gmail.com> parents: diff changeset	203 if self._chunked_skip_bytes:
861f28212398 Import new http library as mercurial.httpclient. Augie Fackler <durin42@gmail.com> parents: diff changeset	204 if len(data) <= self._chunked_skip_bytes:
861f28212398 Import new http library as mercurial.httpclient. Augie Fackler <durin42@gmail.com> parents: diff changeset	205 self._chunked_skip_bytes -= len(data)
861f28212398 Import new http library as mercurial.httpclient. Augie Fackler <durin42@gmail.com> parents: diff changeset	206 data = ''
861f28212398 Import new http library as mercurial.httpclient. Augie Fackler <durin42@gmail.com> parents: diff changeset	207 break
861f28212398 Import new http library as mercurial.httpclient. Augie Fackler <durin42@gmail.com> parents: diff changeset	208 else:
861f28212398 Import new http library as mercurial.httpclient. Augie Fackler <durin42@gmail.com> parents: diff changeset	209 data = data[self._chunked_skip_bytes:]
861f28212398 Import new http library as mercurial.httpclient. Augie Fackler <durin42@gmail.com> parents: diff changeset	210 self._chunked_skip_bytes = 0
861f28212398 Import new http library as mercurial.httpclient. Augie Fackler <durin42@gmail.com> parents: diff changeset	211
861f28212398 Import new http library as mercurial.httpclient. Augie Fackler <durin42@gmail.com> parents: diff changeset	212 # determine how much is until the next chunk
861f28212398 Import new http library as mercurial.httpclient. Augie Fackler <durin42@gmail.com> parents: diff changeset	213 if self._chunked_until_next:
861f28212398 Import new http library as mercurial.httpclient. Augie Fackler <durin42@gmail.com> parents: diff changeset	214 amt = self._chunked_until_next
861f28212398 Import new http library as mercurial.httpclient. Augie Fackler <durin42@gmail.com> parents: diff changeset	215 logger.debug('reading remaining %d of existing chunk', amt)
861f28212398 Import new http library as mercurial.httpclient. Augie Fackler <durin42@gmail.com> parents: diff changeset	216 self._chunked_until_next = 0
861f28212398 Import new http library as mercurial.httpclient. Augie Fackler <durin42@gmail.com> parents: diff changeset	217 body = data
861f28212398 Import new http library as mercurial.httpclient. Augie Fackler <durin42@gmail.com> parents: diff changeset	218 else:
861f28212398 Import new http library as mercurial.httpclient. Augie Fackler <durin42@gmail.com> parents: diff changeset	219 try:
861f28212398 Import new http library as mercurial.httpclient. Augie Fackler <durin42@gmail.com> parents: diff changeset	220 amt, body = data.split(self._eol, 1)
861f28212398 Import new http library as mercurial.httpclient. Augie Fackler <durin42@gmail.com> parents: diff changeset	221 except ValueError:
861f28212398 Import new http library as mercurial.httpclient. Augie Fackler <durin42@gmail.com> parents: diff changeset	222 self._chunked_preloaded_block = data
861f28212398 Import new http library as mercurial.httpclient. Augie Fackler <durin42@gmail.com> parents: diff changeset	223 logger.debug('saving %r as a preloaded block for chunked',
861f28212398 Import new http library as mercurial.httpclient. Augie Fackler <durin42@gmail.com> parents: diff changeset	224 self._chunked_preloaded_block)
861f28212398 Import new http library as mercurial.httpclient. Augie Fackler <durin42@gmail.com> parents: diff changeset	225 return
861f28212398 Import new http library as mercurial.httpclient. Augie Fackler <durin42@gmail.com> parents: diff changeset	226 amt = int(amt, base=16)
861f28212398 Import new http library as mercurial.httpclient. Augie Fackler <durin42@gmail.com> parents: diff changeset	227 logger.debug('reading chunk of length %d', amt)
861f28212398 Import new http library as mercurial.httpclient. Augie Fackler <durin42@gmail.com> parents: diff changeset	228 if amt == 0:
861f28212398 Import new http library as mercurial.httpclient. Augie Fackler <durin42@gmail.com> parents: diff changeset	229 self._chunked_done = True
861f28212398 Import new http library as mercurial.httpclient. Augie Fackler <durin42@gmail.com> parents: diff changeset	230
861f28212398 Import new http library as mercurial.httpclient. Augie Fackler <durin42@gmail.com> parents: diff changeset	231 # read through end of what we have or the chunk
861f28212398 Import new http library as mercurial.httpclient. Augie Fackler <durin42@gmail.com> parents: diff changeset	232 self._body += body[:amt]
861f28212398 Import new http library as mercurial.httpclient. Augie Fackler <durin42@gmail.com> parents: diff changeset	233 if len(body) >= amt:
861f28212398 Import new http library as mercurial.httpclient. Augie Fackler <durin42@gmail.com> parents: diff changeset	234 data = body[amt:]
861f28212398 Import new http library as mercurial.httpclient. Augie Fackler <durin42@gmail.com> parents: diff changeset	235 self._chunked_skip_bytes = len(self._eol)
861f28212398 Import new http library as mercurial.httpclient. Augie Fackler <durin42@gmail.com> parents: diff changeset	236 else:
861f28212398 Import new http library as mercurial.httpclient. Augie Fackler <durin42@gmail.com> parents: diff changeset	237 self._chunked_until_next = amt - len(body)
861f28212398 Import new http library as mercurial.httpclient. Augie Fackler <durin42@gmail.com> parents: diff changeset	238 self._chunked_skip_bytes = 0
861f28212398 Import new http library as mercurial.httpclient. Augie Fackler <durin42@gmail.com> parents: diff changeset	239 data = ''
861f28212398 Import new http library as mercurial.httpclient. Augie Fackler <durin42@gmail.com> parents: diff changeset	240
861f28212398 Import new http library as mercurial.httpclient. Augie Fackler <durin42@gmail.com> parents: diff changeset	241 def _load_response(self, data):
861f28212398 Import new http library as mercurial.httpclient. Augie Fackler <durin42@gmail.com> parents: diff changeset	242 if self._chunked:
861f28212398 Import new http library as mercurial.httpclient. Augie Fackler <durin42@gmail.com> parents: diff changeset	243 self._chunked_parsedata(data)
861f28212398 Import new http library as mercurial.httpclient. Augie Fackler <durin42@gmail.com> parents: diff changeset	244 return
861f28212398 Import new http library as mercurial.httpclient. Augie Fackler <durin42@gmail.com> parents: diff changeset	245 elif self._body is not None:
861f28212398 Import new http library as mercurial.httpclient. Augie Fackler <durin42@gmail.com> parents: diff changeset	246 self._body += data
861f28212398 Import new http library as mercurial.httpclient. Augie Fackler <durin42@gmail.com> parents: diff changeset	247 return
861f28212398 Import new http library as mercurial.httpclient. Augie Fackler <durin42@gmail.com> parents: diff changeset	248
861f28212398 Import new http library as mercurial.httpclient. Augie Fackler <durin42@gmail.com> parents: diff changeset	249 # We haven't seen end of headers yet
861f28212398 Import new http library as mercurial.httpclient. Augie Fackler <durin42@gmail.com> parents: diff changeset	250 self.raw_response += data
861f28212398 Import new http library as mercurial.httpclient. Augie Fackler <durin42@gmail.com> parents: diff changeset	251 # This is a bogus server with bad line endings
861f28212398 Import new http library as mercurial.httpclient. Augie Fackler <durin42@gmail.com> parents: diff changeset	252 if self._eol not in self.raw_response:
861f28212398 Import new http library as mercurial.httpclient. Augie Fackler <durin42@gmail.com> parents: diff changeset	253 for bad_eol in ('\n', '\r'):
861f28212398 Import new http library as mercurial.httpclient. Augie Fackler <durin42@gmail.com> parents: diff changeset	254 if (bad_eol in self.raw_response
861f28212398 Import new http library as mercurial.httpclient. Augie Fackler <durin42@gmail.com> parents: diff changeset	255 # verify that bad_eol is not the end of the incoming data
861f28212398 Import new http library as mercurial.httpclient. Augie Fackler <durin42@gmail.com> parents: diff changeset	256 # as this could be a response line that just got
861f28212398 Import new http library as mercurial.httpclient. Augie Fackler <durin42@gmail.com> parents: diff changeset	257 # split between \r and \n.
861f28212398 Import new http library as mercurial.httpclient. Augie Fackler <durin42@gmail.com> parents: diff changeset	258 and (self.raw_response.index(bad_eol) <
861f28212398 Import new http library as mercurial.httpclient. Augie Fackler <durin42@gmail.com> parents: diff changeset	259 (len(self.raw_response) - 1))):
861f28212398 Import new http library as mercurial.httpclient. Augie Fackler <durin42@gmail.com> parents: diff changeset	260 logger.info('bogus line endings detected, '
861f28212398 Import new http library as mercurial.httpclient. Augie Fackler <durin42@gmail.com> parents: diff changeset	261 'using %r for EOL', bad_eol)
861f28212398 Import new http library as mercurial.httpclient. Augie Fackler <durin42@gmail.com> parents: diff changeset	262 self._eol = bad_eol
861f28212398 Import new http library as mercurial.httpclient. Augie Fackler <durin42@gmail.com> parents: diff changeset	263 break
861f28212398 Import new http library as mercurial.httpclient. Augie Fackler <durin42@gmail.com> parents: diff changeset	264 # exit early if not at end of headers
861f28212398 Import new http library as mercurial.httpclient. Augie Fackler <durin42@gmail.com> parents: diff changeset	265 if self._end_headers not in self.raw_response or self.headers:
861f28212398 Import new http library as mercurial.httpclient. Augie Fackler <durin42@gmail.com> parents: diff changeset	266 return
861f28212398 Import new http library as mercurial.httpclient. Augie Fackler <durin42@gmail.com> parents: diff changeset	267
861f28212398 Import new http library as mercurial.httpclient. Augie Fackler <durin42@gmail.com> parents: diff changeset	268 # handle 100-continue response
861f28212398 Import new http library as mercurial.httpclient. Augie Fackler <durin42@gmail.com> parents: diff changeset	269 hdrs, body = self.raw_response.split(self._end_headers, 1)
861f28212398 Import new http library as mercurial.httpclient. Augie Fackler <durin42@gmail.com> parents: diff changeset	270 http_ver, status = hdrs.split(' ', 1)
861f28212398 Import new http library as mercurial.httpclient. Augie Fackler <durin42@gmail.com> parents: diff changeset	271 if status.startswith('100'):
861f28212398 Import new http library as mercurial.httpclient. Augie Fackler <durin42@gmail.com> parents: diff changeset	272 self.raw_response = body
861f28212398 Import new http library as mercurial.httpclient. Augie Fackler <durin42@gmail.com> parents: diff changeset	273 logger.debug('continue seen, setting body to %r', body)
861f28212398 Import new http library as mercurial.httpclient. Augie Fackler <durin42@gmail.com> parents: diff changeset	274 return
861f28212398 Import new http library as mercurial.httpclient. Augie Fackler <durin42@gmail.com> parents: diff changeset	275
861f28212398 Import new http library as mercurial.httpclient. Augie Fackler <durin42@gmail.com> parents: diff changeset	276 # arriving here means we should parse response headers
861f28212398 Import new http library as mercurial.httpclient. Augie Fackler <durin42@gmail.com> parents: diff changeset	277 # as all headers have arrived completely
861f28212398 Import new http library as mercurial.httpclient. Augie Fackler <durin42@gmail.com> parents: diff changeset	278 hdrs, body = self.raw_response.split(self._end_headers, 1)
861f28212398 Import new http library as mercurial.httpclient. Augie Fackler <durin42@gmail.com> parents: diff changeset	279 del self.raw_response
861f28212398 Import new http library as mercurial.httpclient. Augie Fackler <durin42@gmail.com> parents: diff changeset	280 if self._eol in hdrs:
861f28212398 Import new http library as mercurial.httpclient. Augie Fackler <durin42@gmail.com> parents: diff changeset	281 self.status_line, hdrs = hdrs.split(self._eol, 1)
861f28212398 Import new http library as mercurial.httpclient. Augie Fackler <durin42@gmail.com> parents: diff changeset	282 else:
861f28212398 Import new http library as mercurial.httpclient. Augie Fackler <durin42@gmail.com> parents: diff changeset	283 self.status_line = hdrs
861f28212398 Import new http library as mercurial.httpclient. Augie Fackler <durin42@gmail.com> parents: diff changeset	284 hdrs = ''
861f28212398 Import new http library as mercurial.httpclient. Augie Fackler <durin42@gmail.com> parents: diff changeset	285 # TODO HTTP < 1.0 support
861f28212398 Import new http library as mercurial.httpclient. Augie Fackler <durin42@gmail.com> parents: diff changeset	286 (self.http_version, self.status,
861f28212398 Import new http library as mercurial.httpclient. Augie Fackler <durin42@gmail.com> parents: diff changeset	287 self.reason) = self.status_line.split(' ', 2)
861f28212398 Import new http library as mercurial.httpclient. Augie Fackler <durin42@gmail.com> parents: diff changeset	288 self.status = int(self.status)
861f28212398 Import new http library as mercurial.httpclient. Augie Fackler <durin42@gmail.com> parents: diff changeset	289 if self._eol != EOL:
861f28212398 Import new http library as mercurial.httpclient. Augie Fackler <durin42@gmail.com> parents: diff changeset	290 hdrs = hdrs.replace(self._eol, '\r\n')
861f28212398 Import new http library as mercurial.httpclient. Augie Fackler <durin42@gmail.com> parents: diff changeset	291 headers = rfc822.Message(cStringIO.StringIO(hdrs))
861f28212398 Import new http library as mercurial.httpclient. Augie Fackler <durin42@gmail.com> parents: diff changeset	292 if HDR_CONTENT_LENGTH in headers:
861f28212398 Import new http library as mercurial.httpclient. Augie Fackler <durin42@gmail.com> parents: diff changeset	293 self._content_len = int(headers[HDR_CONTENT_LENGTH])
861f28212398 Import new http library as mercurial.httpclient. Augie Fackler <durin42@gmail.com> parents: diff changeset	294 if self.http_version == HTTP_VER_1_0:
861f28212398 Import new http library as mercurial.httpclient. Augie Fackler <durin42@gmail.com> parents: diff changeset	295 self.will_close = True
861f28212398 Import new http library as mercurial.httpclient. Augie Fackler <durin42@gmail.com> parents: diff changeset	296 elif HDR_CONNECTION_CTRL in headers:
861f28212398 Import new http library as mercurial.httpclient. Augie Fackler <durin42@gmail.com> parents: diff changeset	297 self.will_close = (
861f28212398 Import new http library as mercurial.httpclient. Augie Fackler <durin42@gmail.com> parents: diff changeset	298 headers[HDR_CONNECTION_CTRL].lower() == CONNECTION_CLOSE)
861f28212398 Import new http library as mercurial.httpclient. Augie Fackler <durin42@gmail.com> parents: diff changeset	299 if self._content_len == 0:
861f28212398 Import new http library as mercurial.httpclient. Augie Fackler <durin42@gmail.com> parents: diff changeset	300 self._content_len = _LEN_CLOSE_IS_END
861f28212398 Import new http library as mercurial.httpclient. Augie Fackler <durin42@gmail.com> parents: diff changeset	301 if (HDR_XFER_ENCODING in headers
861f28212398 Import new http library as mercurial.httpclient. Augie Fackler <durin42@gmail.com> parents: diff changeset	302 and headers[HDR_XFER_ENCODING].lower() == XFER_ENCODING_CHUNKED):
861f28212398 Import new http library as mercurial.httpclient. Augie Fackler <durin42@gmail.com> parents: diff changeset	303 self._body = ''
861f28212398 Import new http library as mercurial.httpclient. Augie Fackler <durin42@gmail.com> parents: diff changeset	304 self._chunked_parsedata(body)
861f28212398 Import new http library as mercurial.httpclient. Augie Fackler <durin42@gmail.com> parents: diff changeset	305 self._chunked = True
861f28212398 Import new http library as mercurial.httpclient. Augie Fackler <durin42@gmail.com> parents: diff changeset	306 if self._body is None:
861f28212398 Import new http library as mercurial.httpclient. Augie Fackler <durin42@gmail.com> parents: diff changeset	307 self._body = body
861f28212398 Import new http library as mercurial.httpclient. Augie Fackler <durin42@gmail.com> parents: diff changeset	308 self.headers = headers
861f28212398 Import new http library as mercurial.httpclient. Augie Fackler <durin42@gmail.com> parents: diff changeset	309
861f28212398 Import new http library as mercurial.httpclient. Augie Fackler <durin42@gmail.com> parents: diff changeset	310
861f28212398 Import new http library as mercurial.httpclient. Augie Fackler <durin42@gmail.com> parents: diff changeset	311 class HTTPConnection(object):
861f28212398 Import new http library as mercurial.httpclient. Augie Fackler <durin42@gmail.com> parents: diff changeset	312 """Connection to a single http server.
861f28212398 Import new http library as mercurial.httpclient. Augie Fackler <durin42@gmail.com> parents: diff changeset	313
861f28212398 Import new http library as mercurial.httpclient. Augie Fackler <durin42@gmail.com> parents: diff changeset	314 Supports 100-continue and keepalives natively. Uses select() for
861f28212398 Import new http library as mercurial.httpclient. Augie Fackler <durin42@gmail.com> parents: diff changeset	315 non-blocking socket operations.
861f28212398 Import new http library as mercurial.httpclient. Augie Fackler <durin42@gmail.com> parents: diff changeset	316 """
861f28212398 Import new http library as mercurial.httpclient. Augie Fackler <durin42@gmail.com> parents: diff changeset	317 http_version = HTTP_VER_1_1
861f28212398 Import new http library as mercurial.httpclient. Augie Fackler <durin42@gmail.com> parents: diff changeset	318 response_class = HTTPResponse
861f28212398 Import new http library as mercurial.httpclient. Augie Fackler <durin42@gmail.com> parents: diff changeset	319
861f28212398 Import new http library as mercurial.httpclient. Augie Fackler <durin42@gmail.com> parents: diff changeset	320 def __init__(self, host, port=None, use_ssl=None, ssl_validator=None,
861f28212398 Import new http library as mercurial.httpclient. Augie Fackler <durin42@gmail.com> parents: diff changeset	321 timeout=TIMEOUT_DEFAULT,
861f28212398 Import new http library as mercurial.httpclient. Augie Fackler <durin42@gmail.com> parents: diff changeset	322 continue_timeout=TIMEOUT_ASSUME_CONTINUE,
861f28212398 Import new http library as mercurial.httpclient. Augie Fackler <durin42@gmail.com> parents: diff changeset	323 proxy_hostport=None, **ssl_opts):
861f28212398 Import new http library as mercurial.httpclient. Augie Fackler <durin42@gmail.com> parents: diff changeset	324 """Create a new HTTPConnection.
861f28212398 Import new http library as mercurial.httpclient. Augie Fackler <durin42@gmail.com> parents: diff changeset	325
861f28212398 Import new http library as mercurial.httpclient. Augie Fackler <durin42@gmail.com> parents: diff changeset	326 Args:
861f28212398 Import new http library as mercurial.httpclient. Augie Fackler <durin42@gmail.com> parents: diff changeset	327 host: The host to which we'll connect.
861f28212398 Import new http library as mercurial.httpclient. Augie Fackler <durin42@gmail.com> parents: diff changeset	328 port: Optional. The port over which we'll connect. Default 80 for
861f28212398 Import new http library as mercurial.httpclient. Augie Fackler <durin42@gmail.com> parents: diff changeset	329 non-ssl, 443 for ssl.
861f28212398 Import new http library as mercurial.httpclient. Augie Fackler <durin42@gmail.com> parents: diff changeset	330 use_ssl: Optional. Wether to use ssl. Defaults to False if port is
861f28212398 Import new http library as mercurial.httpclient. Augie Fackler <durin42@gmail.com> parents: diff changeset	331 not 443, true if port is 443.
861f28212398 Import new http library as mercurial.httpclient. Augie Fackler <durin42@gmail.com> parents: diff changeset	332 ssl_validator: a function(socket) to validate the ssl cert
861f28212398 Import new http library as mercurial.httpclient. Augie Fackler <durin42@gmail.com> parents: diff changeset	333 timeout: Optional. Connection timeout, default is TIMEOUT_DEFAULT.
861f28212398 Import new http library as mercurial.httpclient. Augie Fackler <durin42@gmail.com> parents: diff changeset	334 continue_timeout: Optional. Timeout for waiting on an expected
861f28212398 Import new http library as mercurial.httpclient. Augie Fackler <durin42@gmail.com> parents: diff changeset	335 "100 Continue" response. Default is TIMEOUT_ASSUME_CONTINUE.
861f28212398 Import new http library as mercurial.httpclient. Augie Fackler <durin42@gmail.com> parents: diff changeset	336 proxy_hostport: Optional. Tuple of (host, port) to use as an http
861f28212398 Import new http library as mercurial.httpclient. Augie Fackler <durin42@gmail.com> parents: diff changeset	337 proxy for the connection. Default is to not use a proxy.
861f28212398 Import new http library as mercurial.httpclient. Augie Fackler <durin42@gmail.com> parents: diff changeset	338 """
861f28212398 Import new http library as mercurial.httpclient. Augie Fackler <durin42@gmail.com> parents: diff changeset	339 if port is None and host.count(':') == 1 or ']:' in host:
861f28212398 Import new http library as mercurial.httpclient. Augie Fackler <durin42@gmail.com> parents: diff changeset	340 host, port = host.rsplit(':', 1)
861f28212398 Import new http library as mercurial.httpclient. Augie Fackler <durin42@gmail.com> parents: diff changeset	341 port = int(port)
861f28212398 Import new http library as mercurial.httpclient. Augie Fackler <durin42@gmail.com> parents: diff changeset	342 if '[' in host:
861f28212398 Import new http library as mercurial.httpclient. Augie Fackler <durin42@gmail.com> parents: diff changeset	343 host = host[1:-1]
861f28212398 Import new http library as mercurial.httpclient. Augie Fackler <durin42@gmail.com> parents: diff changeset	344 if use_ssl is None and port is None:
861f28212398 Import new http library as mercurial.httpclient. Augie Fackler <durin42@gmail.com> parents: diff changeset	345 use_ssl = False
861f28212398 Import new http library as mercurial.httpclient. Augie Fackler <durin42@gmail.com> parents: diff changeset	346 port = 80
861f28212398 Import new http library as mercurial.httpclient. Augie Fackler <durin42@gmail.com> parents: diff changeset	347 elif use_ssl is None:
861f28212398 Import new http library as mercurial.httpclient. Augie Fackler <durin42@gmail.com> parents: diff changeset	348 use_ssl = (port == 443)
861f28212398 Import new http library as mercurial.httpclient. Augie Fackler <durin42@gmail.com> parents: diff changeset	349 elif port is None:
861f28212398 Import new http library as mercurial.httpclient. Augie Fackler <durin42@gmail.com> parents: diff changeset	350 port = (use_ssl and 443 or 80)
861f28212398 Import new http library as mercurial.httpclient. Augie Fackler <durin42@gmail.com> parents: diff changeset	351 self.port = port
861f28212398 Import new http library as mercurial.httpclient. Augie Fackler <durin42@gmail.com> parents: diff changeset	352 if use_ssl and not socketutil.have_ssl:
861f28212398 Import new http library as mercurial.httpclient. Augie Fackler <durin42@gmail.com> parents: diff changeset	353 raise Exception('ssl requested but unavailable on this Python')
861f28212398 Import new http library as mercurial.httpclient. Augie Fackler <durin42@gmail.com> parents: diff changeset	354 self.ssl = use_ssl
861f28212398 Import new http library as mercurial.httpclient. Augie Fackler <durin42@gmail.com> parents: diff changeset	355 self.ssl_opts = ssl_opts
861f28212398 Import new http library as mercurial.httpclient. Augie Fackler <durin42@gmail.com> parents: diff changeset	356 self._ssl_validator = ssl_validator
861f28212398 Import new http library as mercurial.httpclient. Augie Fackler <durin42@gmail.com> parents: diff changeset	357 self.host = host
861f28212398 Import new http library as mercurial.httpclient. Augie Fackler <durin42@gmail.com> parents: diff changeset	358 self.sock = None
861f28212398 Import new http library as mercurial.httpclient. Augie Fackler <durin42@gmail.com> parents: diff changeset	359 self._current_response = None
861f28212398 Import new http library as mercurial.httpclient. Augie Fackler <durin42@gmail.com> parents: diff changeset	360 self._current_response_taken = False
861f28212398 Import new http library as mercurial.httpclient. Augie Fackler <durin42@gmail.com> parents: diff changeset	361 if proxy_hostport is None:
861f28212398 Import new http library as mercurial.httpclient. Augie Fackler <durin42@gmail.com> parents: diff changeset	362 self._proxy_host = self._proxy_port = None
861f28212398 Import new http library as mercurial.httpclient. Augie Fackler <durin42@gmail.com> parents: diff changeset	363 else:
861f28212398 Import new http library as mercurial.httpclient. Augie Fackler <durin42@gmail.com> parents: diff changeset	364 self._proxy_host, self._proxy_port = proxy_hostport
861f28212398 Import new http library as mercurial.httpclient. Augie Fackler <durin42@gmail.com> parents: diff changeset	365
861f28212398 Import new http library as mercurial.httpclient. Augie Fackler <durin42@gmail.com> parents: diff changeset	366 self.timeout = timeout
861f28212398 Import new http library as mercurial.httpclient. Augie Fackler <durin42@gmail.com> parents: diff changeset	367 self.continue_timeout = continue_timeout
861f28212398 Import new http library as mercurial.httpclient. Augie Fackler <durin42@gmail.com> parents: diff changeset	368
861f28212398 Import new http library as mercurial.httpclient. Augie Fackler <durin42@gmail.com> parents: diff changeset	369 def _connect(self):
861f28212398 Import new http library as mercurial.httpclient. Augie Fackler <durin42@gmail.com> parents: diff changeset	370 """Connect to the host and port specified in __init__."""
861f28212398 Import new http library as mercurial.httpclient. Augie Fackler <durin42@gmail.com> parents: diff changeset	371 if self.sock:
861f28212398 Import new http library as mercurial.httpclient. Augie Fackler <durin42@gmail.com> parents: diff changeset	372 return
861f28212398 Import new http library as mercurial.httpclient. Augie Fackler <durin42@gmail.com> parents: diff changeset	373 if self._proxy_host is not None:
861f28212398 Import new http library as mercurial.httpclient. Augie Fackler <durin42@gmail.com> parents: diff changeset	374 logger.info('Connecting to http proxy %s:%s',
861f28212398 Import new http library as mercurial.httpclient. Augie Fackler <durin42@gmail.com> parents: diff changeset	375 self._proxy_host, self._proxy_port)
861f28212398 Import new http library as mercurial.httpclient. Augie Fackler <durin42@gmail.com> parents: diff changeset	376 sock = socketutil.create_connection((self._proxy_host,
861f28212398 Import new http library as mercurial.httpclient. Augie Fackler <durin42@gmail.com> parents: diff changeset	377 self._proxy_port))
861f28212398 Import new http library as mercurial.httpclient. Augie Fackler <durin42@gmail.com> parents: diff changeset	378 if self.ssl:
861f28212398 Import new http library as mercurial.httpclient. Augie Fackler <durin42@gmail.com> parents: diff changeset	379 # TODO proxy header support
861f28212398 Import new http library as mercurial.httpclient. Augie Fackler <durin42@gmail.com> parents: diff changeset	380 data = self.buildheaders('CONNECT', '%s:%d' % (self.host,
861f28212398 Import new http library as mercurial.httpclient. Augie Fackler <durin42@gmail.com> parents: diff changeset	381 self.port),
861f28212398 Import new http library as mercurial.httpclient. Augie Fackler <durin42@gmail.com> parents: diff changeset	382 {}, HTTP_VER_1_0)
861f28212398 Import new http library as mercurial.httpclient. Augie Fackler <durin42@gmail.com> parents: diff changeset	383 sock.send(data)
861f28212398 Import new http library as mercurial.httpclient. Augie Fackler <durin42@gmail.com> parents: diff changeset	384 sock.setblocking(0)
861f28212398 Import new http library as mercurial.httpclient. Augie Fackler <durin42@gmail.com> parents: diff changeset	385 r = self.response_class(sock, self.timeout)
861f28212398 Import new http library as mercurial.httpclient. Augie Fackler <durin42@gmail.com> parents: diff changeset	386 timeout_exc = HTTPTimeoutException(
861f28212398 Import new http library as mercurial.httpclient. Augie Fackler <durin42@gmail.com> parents: diff changeset	387 'Timed out waiting for CONNECT response from proxy')
861f28212398 Import new http library as mercurial.httpclient. Augie Fackler <durin42@gmail.com> parents: diff changeset	388 while not r.complete():
861f28212398 Import new http library as mercurial.httpclient. Augie Fackler <durin42@gmail.com> parents: diff changeset	389 try:
861f28212398 Import new http library as mercurial.httpclient. Augie Fackler <durin42@gmail.com> parents: diff changeset	390 if not r._select():
861f28212398 Import new http library as mercurial.httpclient. Augie Fackler <durin42@gmail.com> parents: diff changeset	391 raise timeout_exc
861f28212398 Import new http library as mercurial.httpclient. Augie Fackler <durin42@gmail.com> parents: diff changeset	392 except HTTPTimeoutException:
861f28212398 Import new http library as mercurial.httpclient. Augie Fackler <durin42@gmail.com> parents: diff changeset	393 # This raise/except pattern looks goofy, but
861f28212398 Import new http library as mercurial.httpclient. Augie Fackler <durin42@gmail.com> parents: diff changeset	394 # _select can raise the timeout as well as the
861f28212398 Import new http library as mercurial.httpclient. Augie Fackler <durin42@gmail.com> parents: diff changeset	395 # loop body. I wish it wasn't this convoluted,
861f28212398 Import new http library as mercurial.httpclient. Augie Fackler <durin42@gmail.com> parents: diff changeset	396 # but I don't have a better solution
861f28212398 Import new http library as mercurial.httpclient. Augie Fackler <durin42@gmail.com> parents: diff changeset	397 # immediately handy.
861f28212398 Import new http library as mercurial.httpclient. Augie Fackler <durin42@gmail.com> parents: diff changeset	398 raise timeout_exc
861f28212398 Import new http library as mercurial.httpclient. Augie Fackler <durin42@gmail.com> parents: diff changeset	399 if r.status != 200:
861f28212398 Import new http library as mercurial.httpclient. Augie Fackler <durin42@gmail.com> parents: diff changeset	400 raise HTTPProxyConnectFailedException(
861f28212398 Import new http library as mercurial.httpclient. Augie Fackler <durin42@gmail.com> parents: diff changeset	401 'Proxy connection failed: %d %s' % (r.status,
861f28212398 Import new http library as mercurial.httpclient. Augie Fackler <durin42@gmail.com> parents: diff changeset	402 r.read()))
861f28212398 Import new http library as mercurial.httpclient. Augie Fackler <durin42@gmail.com> parents: diff changeset	403 logger.info('CONNECT (for SSL) to %s:%s via proxy succeeded.',
861f28212398 Import new http library as mercurial.httpclient. Augie Fackler <durin42@gmail.com> parents: diff changeset	404 self.host, self.port)
861f28212398 Import new http library as mercurial.httpclient. Augie Fackler <durin42@gmail.com> parents: diff changeset	405 else:
861f28212398 Import new http library as mercurial.httpclient. Augie Fackler <durin42@gmail.com> parents: diff changeset	406 sock = socketutil.create_connection((self.host, self.port))
861f28212398 Import new http library as mercurial.httpclient. Augie Fackler <durin42@gmail.com> parents: diff changeset	407 if self.ssl:
861f28212398 Import new http library as mercurial.httpclient. Augie Fackler <durin42@gmail.com> parents: diff changeset	408 logger.debug('wrapping socket for ssl with options %r',
861f28212398 Import new http library as mercurial.httpclient. Augie Fackler <durin42@gmail.com> parents: diff changeset	409 self.ssl_opts)
861f28212398 Import new http library as mercurial.httpclient. Augie Fackler <durin42@gmail.com> parents: diff changeset	410 sock = socketutil.wrap_socket(sock, **self.ssl_opts)
861f28212398 Import new http library as mercurial.httpclient. Augie Fackler <durin42@gmail.com> parents: diff changeset	411 if self._ssl_validator:
861f28212398 Import new http library as mercurial.httpclient. Augie Fackler <durin42@gmail.com> parents: diff changeset	412 self._ssl_validator(sock)
861f28212398 Import new http library as mercurial.httpclient. Augie Fackler <durin42@gmail.com> parents: diff changeset	413 sock.setblocking(0)
861f28212398 Import new http library as mercurial.httpclient. Augie Fackler <durin42@gmail.com> parents: diff changeset	414 self.sock = sock
861f28212398 Import new http library as mercurial.httpclient. Augie Fackler <durin42@gmail.com> parents: diff changeset	415
861f28212398 Import new http library as mercurial.httpclient. Augie Fackler <durin42@gmail.com> parents: diff changeset	416 def buildheaders(self, method, path, headers, http_ver):
861f28212398 Import new http library as mercurial.httpclient. Augie Fackler <durin42@gmail.com> parents: diff changeset	417 if self.ssl and self.port == 443 or self.port == 80:
861f28212398 Import new http library as mercurial.httpclient. Augie Fackler <durin42@gmail.com> parents: diff changeset	418 # default port for protocol, so leave it out
861f28212398 Import new http library as mercurial.httpclient. Augie Fackler <durin42@gmail.com> parents: diff changeset	419 hdrhost = self.host
861f28212398 Import new http library as mercurial.httpclient. Augie Fackler <durin42@gmail.com> parents: diff changeset	420 else:
861f28212398 Import new http library as mercurial.httpclient. Augie Fackler <durin42@gmail.com> parents: diff changeset	421 # include nonstandard port in header
861f28212398 Import new http library as mercurial.httpclient. Augie Fackler <durin42@gmail.com> parents: diff changeset	422 if ':' in self.host: # must be IPv6
861f28212398 Import new http library as mercurial.httpclient. Augie Fackler <durin42@gmail.com> parents: diff changeset	423 hdrhost = '[%s]:%d' % (self.host, self.port)
861f28212398 Import new http library as mercurial.httpclient. Augie Fackler <durin42@gmail.com> parents: diff changeset	424 else:
861f28212398 Import new http library as mercurial.httpclient. Augie Fackler <durin42@gmail.com> parents: diff changeset	425 hdrhost = '%s:%d' % (self.host, self.port)
861f28212398 Import new http library as mercurial.httpclient. Augie Fackler <durin42@gmail.com> parents: diff changeset	426 if self._proxy_host and not self.ssl:
861f28212398 Import new http library as mercurial.httpclient. Augie Fackler <durin42@gmail.com> parents: diff changeset	427 # When talking to a regular http proxy we must send the
861f28212398 Import new http library as mercurial.httpclient. Augie Fackler <durin42@gmail.com> parents: diff changeset	428 # full URI, but in all other cases we must not (although
861f28212398 Import new http library as mercurial.httpclient. Augie Fackler <durin42@gmail.com> parents: diff changeset	429 # technically RFC 2616 says servers must accept our
861f28212398 Import new http library as mercurial.httpclient. Augie Fackler <durin42@gmail.com> parents: diff changeset	430 # request if we screw up, experimentally few do that
861f28212398 Import new http library as mercurial.httpclient. Augie Fackler <durin42@gmail.com> parents: diff changeset	431 # correctly.)
861f28212398 Import new http library as mercurial.httpclient. Augie Fackler <durin42@gmail.com> parents: diff changeset	432 assert path[0] == '/', 'path must start with a /'
861f28212398 Import new http library as mercurial.httpclient. Augie Fackler <durin42@gmail.com> parents: diff changeset	433 path = 'http://%s%s' % (hdrhost, path)
861f28212398 Import new http library as mercurial.httpclient. Augie Fackler <durin42@gmail.com> parents: diff changeset	434 outgoing = ['%s %s %s%s' % (method, path, http_ver, EOL)]
861f28212398 Import new http library as mercurial.httpclient. Augie Fackler <durin42@gmail.com> parents: diff changeset	435 headers['host'] = ('Host', hdrhost)
861f28212398 Import new http library as mercurial.httpclient. Augie Fackler <durin42@gmail.com> parents: diff changeset	436 headers[HDR_ACCEPT_ENCODING] = (HDR_ACCEPT_ENCODING, 'identity')
861f28212398 Import new http library as mercurial.httpclient. Augie Fackler <durin42@gmail.com> parents: diff changeset	437 for hdr, val in headers.itervalues():
861f28212398 Import new http library as mercurial.httpclient. Augie Fackler <durin42@gmail.com> parents: diff changeset	438 outgoing.append('%s: %s%s' % (hdr, val, EOL))
861f28212398 Import new http library as mercurial.httpclient. Augie Fackler <durin42@gmail.com> parents: diff changeset	439 outgoing.append(EOL)
861f28212398 Import new http library as mercurial.httpclient. Augie Fackler <durin42@gmail.com> parents: diff changeset	440 return ''.join(outgoing)
861f28212398 Import new http library as mercurial.httpclient. Augie Fackler <durin42@gmail.com> parents: diff changeset	441
861f28212398 Import new http library as mercurial.httpclient. Augie Fackler <durin42@gmail.com> parents: diff changeset	442 def close(self):
861f28212398 Import new http library as mercurial.httpclient. Augie Fackler <durin42@gmail.com> parents: diff changeset	443 """Close the connection to the server.
861f28212398 Import new http library as mercurial.httpclient. Augie Fackler <durin42@gmail.com> parents: diff changeset	444
861f28212398 Import new http library as mercurial.httpclient. Augie Fackler <durin42@gmail.com> parents: diff changeset	445 This is a no-op if the connection is already closed. The
861f28212398 Import new http library as mercurial.httpclient. Augie Fackler <durin42@gmail.com> parents: diff changeset	446 connection may automatically close if requessted by the server
861f28212398 Import new http library as mercurial.httpclient. Augie Fackler <durin42@gmail.com> parents: diff changeset	447 or required by the nature of a response.
861f28212398 Import new http library as mercurial.httpclient. Augie Fackler <durin42@gmail.com> parents: diff changeset	448 """
861f28212398 Import new http library as mercurial.httpclient. Augie Fackler <durin42@gmail.com> parents: diff changeset	449 if self.sock is None:
861f28212398 Import new http library as mercurial.httpclient. Augie Fackler <durin42@gmail.com> parents: diff changeset	450 return
861f28212398 Import new http library as mercurial.httpclient. Augie Fackler <durin42@gmail.com> parents: diff changeset	451 self.sock.close()
861f28212398 Import new http library as mercurial.httpclient. Augie Fackler <durin42@gmail.com> parents: diff changeset	452 self.sock = None
861f28212398 Import new http library as mercurial.httpclient. Augie Fackler <durin42@gmail.com> parents: diff changeset	453 logger.info('closed connection to %s on %s', self.host, self.port)
861f28212398 Import new http library as mercurial.httpclient. Augie Fackler <durin42@gmail.com> parents: diff changeset	454
861f28212398 Import new http library as mercurial.httpclient. Augie Fackler <durin42@gmail.com> parents: diff changeset	455 def busy(self):
861f28212398 Import new http library as mercurial.httpclient. Augie Fackler <durin42@gmail.com> parents: diff changeset	456 """Returns True if this connection object is currently in use.
861f28212398 Import new http library as mercurial.httpclient. Augie Fackler <durin42@gmail.com> parents: diff changeset	457
861f28212398 Import new http library as mercurial.httpclient. Augie Fackler <durin42@gmail.com> parents: diff changeset	458 If a response is still pending, this will return True, even if
861f28212398 Import new http library as mercurial.httpclient. Augie Fackler <durin42@gmail.com> parents: diff changeset	459 the request has finished sending. In the future,
861f28212398 Import new http library as mercurial.httpclient. Augie Fackler <durin42@gmail.com> parents: diff changeset	460 HTTPConnection may transparently juggle multiple connections
861f28212398 Import new http library as mercurial.httpclient. Augie Fackler <durin42@gmail.com> parents: diff changeset	461 to the server, in which case this will be useful to detect if
861f28212398 Import new http library as mercurial.httpclient. Augie Fackler <durin42@gmail.com> parents: diff changeset	462 any of those connections is ready for use.
861f28212398 Import new http library as mercurial.httpclient. Augie Fackler <durin42@gmail.com> parents: diff changeset	463 """
861f28212398 Import new http library as mercurial.httpclient. Augie Fackler <durin42@gmail.com> parents: diff changeset	464 cr = self._current_response
861f28212398 Import new http library as mercurial.httpclient. Augie Fackler <durin42@gmail.com> parents: diff changeset	465 if cr is not None:
861f28212398 Import new http library as mercurial.httpclient. Augie Fackler <durin42@gmail.com> parents: diff changeset	466 if self._current_response_taken:
861f28212398 Import new http library as mercurial.httpclient. Augie Fackler <durin42@gmail.com> parents: diff changeset	467 if cr.will_close:
861f28212398 Import new http library as mercurial.httpclient. Augie Fackler <durin42@gmail.com> parents: diff changeset	468 self.sock = None
861f28212398 Import new http library as mercurial.httpclient. Augie Fackler <durin42@gmail.com> parents: diff changeset	469 self._current_response = None
861f28212398 Import new http library as mercurial.httpclient. Augie Fackler <durin42@gmail.com> parents: diff changeset	470 return False
861f28212398 Import new http library as mercurial.httpclient. Augie Fackler <durin42@gmail.com> parents: diff changeset	471 elif cr.complete():
861f28212398 Import new http library as mercurial.httpclient. Augie Fackler <durin42@gmail.com> parents: diff changeset	472 self._current_response = None
861f28212398 Import new http library as mercurial.httpclient. Augie Fackler <durin42@gmail.com> parents: diff changeset	473 return False
861f28212398 Import new http library as mercurial.httpclient. Augie Fackler <durin42@gmail.com> parents: diff changeset	474 return True
861f28212398 Import new http library as mercurial.httpclient. Augie Fackler <durin42@gmail.com> parents: diff changeset	475 return False
861f28212398 Import new http library as mercurial.httpclient. Augie Fackler <durin42@gmail.com> parents: diff changeset	476
861f28212398 Import new http library as mercurial.httpclient. Augie Fackler <durin42@gmail.com> parents: diff changeset	477 def request(self, method, path, body=None, headers={},
861f28212398 Import new http library as mercurial.httpclient. Augie Fackler <durin42@gmail.com> parents: diff changeset	478 expect_continue=False):
861f28212398 Import new http library as mercurial.httpclient. Augie Fackler <durin42@gmail.com> parents: diff changeset	479 """Send a request to the server.
861f28212398 Import new http library as mercurial.httpclient. Augie Fackler <durin42@gmail.com> parents: diff changeset	480
861f28212398 Import new http library as mercurial.httpclient. Augie Fackler <durin42@gmail.com> parents: diff changeset	481 For increased flexibility, this does not return the response
861f28212398 Import new http library as mercurial.httpclient. Augie Fackler <durin42@gmail.com> parents: diff changeset	482 object. Future versions of HTTPConnection that juggle multiple
861f28212398 Import new http library as mercurial.httpclient. Augie Fackler <durin42@gmail.com> parents: diff changeset	483 sockets will be able to send (for example) 5 requests all at
861f28212398 Import new http library as mercurial.httpclient. Augie Fackler <durin42@gmail.com> parents: diff changeset	484 once, and then let the requests arrive as data is
861f28212398 Import new http library as mercurial.httpclient. Augie Fackler <durin42@gmail.com> parents: diff changeset	485 available. Use the `getresponse()` method to retrieve the
861f28212398 Import new http library as mercurial.httpclient. Augie Fackler <durin42@gmail.com> parents: diff changeset	486 response.
861f28212398 Import new http library as mercurial.httpclient. Augie Fackler <durin42@gmail.com> parents: diff changeset	487 """
861f28212398 Import new http library as mercurial.httpclient. Augie Fackler <durin42@gmail.com> parents: diff changeset	488 if self.busy():
861f28212398 Import new http library as mercurial.httpclient. Augie Fackler <durin42@gmail.com> parents: diff changeset	489 raise httplib.CannotSendRequest(
861f28212398 Import new http library as mercurial.httpclient. Augie Fackler <durin42@gmail.com> parents: diff changeset	490 'Can not send another request before '
861f28212398 Import new http library as mercurial.httpclient. Augie Fackler <durin42@gmail.com> parents: diff changeset	491 'current response is read!')
861f28212398 Import new http library as mercurial.httpclient. Augie Fackler <durin42@gmail.com> parents: diff changeset	492 self._current_response_taken = False
861f28212398 Import new http library as mercurial.httpclient. Augie Fackler <durin42@gmail.com> parents: diff changeset	493
861f28212398 Import new http library as mercurial.httpclient. Augie Fackler <durin42@gmail.com> parents: diff changeset	494 logger.info('sending %s request for %s to %s on port %s',
861f28212398 Import new http library as mercurial.httpclient. Augie Fackler <durin42@gmail.com> parents: diff changeset	495 method, path, self.host, self.port)
861f28212398 Import new http library as mercurial.httpclient. Augie Fackler <durin42@gmail.com> parents: diff changeset	496 hdrs = dict((k.lower(), (k, v)) for k, v in headers.iteritems())
861f28212398 Import new http library as mercurial.httpclient. Augie Fackler <durin42@gmail.com> parents: diff changeset	497 if hdrs.get('expect', ('', ''))[1].lower() == '100-continue':
861f28212398 Import new http library as mercurial.httpclient. Augie Fackler <durin42@gmail.com> parents: diff changeset	498 expect_continue = True
861f28212398 Import new http library as mercurial.httpclient. Augie Fackler <durin42@gmail.com> parents: diff changeset	499 elif expect_continue:
861f28212398 Import new http library as mercurial.httpclient. Augie Fackler <durin42@gmail.com> parents: diff changeset	500 hdrs['expect'] = ('Expect', '100-Continue')
861f28212398 Import new http library as mercurial.httpclient. Augie Fackler <durin42@gmail.com> parents: diff changeset	501
861f28212398 Import new http library as mercurial.httpclient. Augie Fackler <durin42@gmail.com> parents: diff changeset	502 chunked = False
861f28212398 Import new http library as mercurial.httpclient. Augie Fackler <durin42@gmail.com> parents: diff changeset	503 if body and HDR_CONTENT_LENGTH not in hdrs:
861f28212398 Import new http library as mercurial.httpclient. Augie Fackler <durin42@gmail.com> parents: diff changeset	504 if getattr(body, '__len__', False):
861f28212398 Import new http library as mercurial.httpclient. Augie Fackler <durin42@gmail.com> parents: diff changeset	505 hdrs[HDR_CONTENT_LENGTH] = (HDR_CONTENT_LENGTH, len(body))
861f28212398 Import new http library as mercurial.httpclient. Augie Fackler <durin42@gmail.com> parents: diff changeset	506 elif getattr(body, 'read', False):
861f28212398 Import new http library as mercurial.httpclient. Augie Fackler <durin42@gmail.com> parents: diff changeset	507 hdrs[HDR_XFER_ENCODING] = (HDR_XFER_ENCODING,
861f28212398 Import new http library as mercurial.httpclient. Augie Fackler <durin42@gmail.com> parents: diff changeset	508 XFER_ENCODING_CHUNKED)
861f28212398 Import new http library as mercurial.httpclient. Augie Fackler <durin42@gmail.com> parents: diff changeset	509 chunked = True
861f28212398 Import new http library as mercurial.httpclient. Augie Fackler <durin42@gmail.com> parents: diff changeset	510 else:
861f28212398 Import new http library as mercurial.httpclient. Augie Fackler <durin42@gmail.com> parents: diff changeset	511 raise BadRequestData('body has no __len__() nor read()')
861f28212398 Import new http library as mercurial.httpclient. Augie Fackler <durin42@gmail.com> parents: diff changeset	512
861f28212398 Import new http library as mercurial.httpclient. Augie Fackler <durin42@gmail.com> parents: diff changeset	513 self._connect()
861f28212398 Import new http library as mercurial.httpclient. Augie Fackler <durin42@gmail.com> parents: diff changeset	514 outgoing_headers = self.buildheaders(
861f28212398 Import new http library as mercurial.httpclient. Augie Fackler <durin42@gmail.com> parents: diff changeset	515 method, path, hdrs, self.http_version)
861f28212398 Import new http library as mercurial.httpclient. Augie Fackler <durin42@gmail.com> parents: diff changeset	516 response = None
861f28212398 Import new http library as mercurial.httpclient. Augie Fackler <durin42@gmail.com> parents: diff changeset	517 first = True
861f28212398 Import new http library as mercurial.httpclient. Augie Fackler <durin42@gmail.com> parents: diff changeset	518
861f28212398 Import new http library as mercurial.httpclient. Augie Fackler <durin42@gmail.com> parents: diff changeset	519 def reconnect(where):
861f28212398 Import new http library as mercurial.httpclient. Augie Fackler <durin42@gmail.com> parents: diff changeset	520 logger.info('reconnecting during %s', where)
861f28212398 Import new http library as mercurial.httpclient. Augie Fackler <durin42@gmail.com> parents: diff changeset	521 self.close()
861f28212398 Import new http library as mercurial.httpclient. Augie Fackler <durin42@gmail.com> parents: diff changeset	522 self._connect()
861f28212398 Import new http library as mercurial.httpclient. Augie Fackler <durin42@gmail.com> parents: diff changeset	523
861f28212398 Import new http library as mercurial.httpclient. Augie Fackler <durin42@gmail.com> parents: diff changeset	524 while ((outgoing_headers or body)
861f28212398 Import new http library as mercurial.httpclient. Augie Fackler <durin42@gmail.com> parents: diff changeset	525 and not (response and response.complete())):
861f28212398 Import new http library as mercurial.httpclient. Augie Fackler <durin42@gmail.com> parents: diff changeset	526 select_timeout = self.timeout
861f28212398 Import new http library as mercurial.httpclient. Augie Fackler <durin42@gmail.com> parents: diff changeset	527 out = outgoing_headers or body
861f28212398 Import new http library as mercurial.httpclient. Augie Fackler <durin42@gmail.com> parents: diff changeset	528 blocking_on_continue = False
861f28212398 Import new http library as mercurial.httpclient. Augie Fackler <durin42@gmail.com> parents: diff changeset	529 if expect_continue and not outgoing_headers and not (
861f28212398 Import new http library as mercurial.httpclient. Augie Fackler <durin42@gmail.com> parents: diff changeset	530 response and response.headers):
861f28212398 Import new http library as mercurial.httpclient. Augie Fackler <durin42@gmail.com> parents: diff changeset	531 logger.info(
861f28212398 Import new http library as mercurial.httpclient. Augie Fackler <durin42@gmail.com> parents: diff changeset	532 'waiting up to %s seconds for'
861f28212398 Import new http library as mercurial.httpclient. Augie Fackler <durin42@gmail.com> parents: diff changeset	533 ' continue response from server',
861f28212398 Import new http library as mercurial.httpclient. Augie Fackler <durin42@gmail.com> parents: diff changeset	534 self.continue_timeout)
861f28212398 Import new http library as mercurial.httpclient. Augie Fackler <durin42@gmail.com> parents: diff changeset	535 select_timeout = self.continue_timeout
861f28212398 Import new http library as mercurial.httpclient. Augie Fackler <durin42@gmail.com> parents: diff changeset	536 blocking_on_continue = True
861f28212398 Import new http library as mercurial.httpclient. Augie Fackler <durin42@gmail.com> parents: diff changeset	537 out = False
861f28212398 Import new http library as mercurial.httpclient. Augie Fackler <durin42@gmail.com> parents: diff changeset	538 if out:
861f28212398 Import new http library as mercurial.httpclient. Augie Fackler <durin42@gmail.com> parents: diff changeset	539 w = [self.sock]
861f28212398 Import new http library as mercurial.httpclient. Augie Fackler <durin42@gmail.com> parents: diff changeset	540 else:
861f28212398 Import new http library as mercurial.httpclient. Augie Fackler <durin42@gmail.com> parents: diff changeset	541 w = []
861f28212398 Import new http library as mercurial.httpclient. Augie Fackler <durin42@gmail.com> parents: diff changeset	542 r, w, x = select.select([self.sock], w, [], select_timeout)
861f28212398 Import new http library as mercurial.httpclient. Augie Fackler <durin42@gmail.com> parents: diff changeset	543 # if we were expecting a 100 continue and it's been long
861f28212398 Import new http library as mercurial.httpclient. Augie Fackler <durin42@gmail.com> parents: diff changeset	544 # enough, just go ahead and assume it's ok. This is the
861f28212398 Import new http library as mercurial.httpclient. Augie Fackler <durin42@gmail.com> parents: diff changeset	545 # recommended behavior from the RFC.
861f28212398 Import new http library as mercurial.httpclient. Augie Fackler <durin42@gmail.com> parents: diff changeset	546 if r == w == x == []:
861f28212398 Import new http library as mercurial.httpclient. Augie Fackler <durin42@gmail.com> parents: diff changeset	547 if blocking_on_continue:
861f28212398 Import new http library as mercurial.httpclient. Augie Fackler <durin42@gmail.com> parents: diff changeset	548 expect_continue = False
861f28212398 Import new http library as mercurial.httpclient. Augie Fackler <durin42@gmail.com> parents: diff changeset	549 logger.info('no response to continue expectation from '
861f28212398 Import new http library as mercurial.httpclient. Augie Fackler <durin42@gmail.com> parents: diff changeset	550 'server, optimistically sending request body')
861f28212398 Import new http library as mercurial.httpclient. Augie Fackler <durin42@gmail.com> parents: diff changeset	551 else:
861f28212398 Import new http library as mercurial.httpclient. Augie Fackler <durin42@gmail.com> parents: diff changeset	552 raise HTTPTimeoutException('timeout sending data')
861f28212398 Import new http library as mercurial.httpclient. Augie Fackler <durin42@gmail.com> parents: diff changeset	553 # TODO exceptional conditions with select? (what are those be?)
861f28212398 Import new http library as mercurial.httpclient. Augie Fackler <durin42@gmail.com> parents: diff changeset	554 # TODO if the response is loading, must we finish sending at all?
861f28212398 Import new http library as mercurial.httpclient. Augie Fackler <durin42@gmail.com> parents: diff changeset	555 #
861f28212398 Import new http library as mercurial.httpclient. Augie Fackler <durin42@gmail.com> parents: diff changeset	556 # Certainly not if it's going to close the connection and/or
861f28212398 Import new http library as mercurial.httpclient. Augie Fackler <durin42@gmail.com> parents: diff changeset	557 # the response is already done...I think.
861f28212398 Import new http library as mercurial.httpclient. Augie Fackler <durin42@gmail.com> parents: diff changeset	558 was_first = first
861f28212398 Import new http library as mercurial.httpclient. Augie Fackler <durin42@gmail.com> parents: diff changeset	559
861f28212398 Import new http library as mercurial.httpclient. Augie Fackler <durin42@gmail.com> parents: diff changeset	560 # incoming data
861f28212398 Import new http library as mercurial.httpclient. Augie Fackler <durin42@gmail.com> parents: diff changeset	561 if r:
861f28212398 Import new http library as mercurial.httpclient. Augie Fackler <durin42@gmail.com> parents: diff changeset	562 try:
14341 5c3de67e7402 httpclient: import revision b8c3511a8cae from py-nonblocking-http Augie Fackler <durin42@gmail.com> parents: 14293 diff changeset	563 try:
5c3de67e7402 httpclient: import revision b8c3511a8cae from py-nonblocking-http Augie Fackler <durin42@gmail.com> parents: 14293 diff changeset	564 data = r[0].recv(INCOMING_BUFFER_SIZE)
5c3de67e7402 httpclient: import revision b8c3511a8cae from py-nonblocking-http Augie Fackler <durin42@gmail.com> parents: 14293 diff changeset	565 except socket.sslerror, e:
5c3de67e7402 httpclient: import revision b8c3511a8cae from py-nonblocking-http Augie Fackler <durin42@gmail.com> parents: 14293 diff changeset	566 if e.args[0] != socket.SSL_ERROR_WANT_READ:
5c3de67e7402 httpclient: import revision b8c3511a8cae from py-nonblocking-http Augie Fackler <durin42@gmail.com> parents: 14293 diff changeset	567 raise
5c3de67e7402 httpclient: import revision b8c3511a8cae from py-nonblocking-http Augie Fackler <durin42@gmail.com> parents: 14293 diff changeset	568 logger.debug(
5c3de67e7402 httpclient: import revision b8c3511a8cae from py-nonblocking-http Augie Fackler <durin42@gmail.com> parents: 14293 diff changeset	569 'SSL_WANT_READ while sending data, retrying...')
5c3de67e7402 httpclient: import revision b8c3511a8cae from py-nonblocking-http Augie Fackler <durin42@gmail.com> parents: 14293 diff changeset	570 continue
14243 861f28212398 Import new http library as mercurial.httpclient. Augie Fackler <durin42@gmail.com> parents: diff changeset	571 if not data:
861f28212398 Import new http library as mercurial.httpclient. Augie Fackler <durin42@gmail.com> parents: diff changeset	572 logger.info('socket appears closed in read')
14376 a75e0f4ba0ab httpclient: import revision fc731618702a of py-nonblocking-http Augie Fackler <durin42@gmail.com> parents: 14341 diff changeset	573 self.sock = None
a75e0f4ba0ab httpclient: import revision fc731618702a of py-nonblocking-http Augie Fackler <durin42@gmail.com> parents: 14341 diff changeset	574 self._current_response = None
a75e0f4ba0ab httpclient: import revision fc731618702a of py-nonblocking-http Augie Fackler <durin42@gmail.com> parents: 14341 diff changeset	575 # This if/elif ladder is a bit subtle,
a75e0f4ba0ab httpclient: import revision fc731618702a of py-nonblocking-http Augie Fackler <durin42@gmail.com> parents: 14341 diff changeset	576 # comments in each branch should help.
a75e0f4ba0ab httpclient: import revision fc731618702a of py-nonblocking-http Augie Fackler <durin42@gmail.com> parents: 14341 diff changeset	577 if response is not None and (
a75e0f4ba0ab httpclient: import revision fc731618702a of py-nonblocking-http Augie Fackler <durin42@gmail.com> parents: 14341 diff changeset	578 response.complete() or
a75e0f4ba0ab httpclient: import revision fc731618702a of py-nonblocking-http Augie Fackler <durin42@gmail.com> parents: 14341 diff changeset	579 response._content_len == _LEN_CLOSE_IS_END):
a75e0f4ba0ab httpclient: import revision fc731618702a of py-nonblocking-http Augie Fackler <durin42@gmail.com> parents: 14341 diff changeset	580 # Server responded completely and then
a75e0f4ba0ab httpclient: import revision fc731618702a of py-nonblocking-http Augie Fackler <durin42@gmail.com> parents: 14341 diff changeset	581 # closed the socket. We should just shut
a75e0f4ba0ab httpclient: import revision fc731618702a of py-nonblocking-http Augie Fackler <durin42@gmail.com> parents: 14341 diff changeset	582 # things down and let the caller get their
a75e0f4ba0ab httpclient: import revision fc731618702a of py-nonblocking-http Augie Fackler <durin42@gmail.com> parents: 14341 diff changeset	583 # response.
a75e0f4ba0ab httpclient: import revision fc731618702a of py-nonblocking-http Augie Fackler <durin42@gmail.com> parents: 14341 diff changeset	584 logger.info('Got an early response, '
a75e0f4ba0ab httpclient: import revision fc731618702a of py-nonblocking-http Augie Fackler <durin42@gmail.com> parents: 14341 diff changeset	585 'aborting remaining request.')
a75e0f4ba0ab httpclient: import revision fc731618702a of py-nonblocking-http Augie Fackler <durin42@gmail.com> parents: 14341 diff changeset	586 break
a75e0f4ba0ab httpclient: import revision fc731618702a of py-nonblocking-http Augie Fackler <durin42@gmail.com> parents: 14341 diff changeset	587 elif was_first and response is None:
a75e0f4ba0ab httpclient: import revision fc731618702a of py-nonblocking-http Augie Fackler <durin42@gmail.com> parents: 14341 diff changeset	588 # Most likely a keepalive that got killed
a75e0f4ba0ab httpclient: import revision fc731618702a of py-nonblocking-http Augie Fackler <durin42@gmail.com> parents: 14341 diff changeset	589 # on the server's end. Commonly happens
a75e0f4ba0ab httpclient: import revision fc731618702a of py-nonblocking-http Augie Fackler <durin42@gmail.com> parents: 14341 diff changeset	590 # after getting a really large response
a75e0f4ba0ab httpclient: import revision fc731618702a of py-nonblocking-http Augie Fackler <durin42@gmail.com> parents: 14341 diff changeset	591 # from the server.
a75e0f4ba0ab httpclient: import revision fc731618702a of py-nonblocking-http Augie Fackler <durin42@gmail.com> parents: 14341 diff changeset	592 logger.info(
a75e0f4ba0ab httpclient: import revision fc731618702a of py-nonblocking-http Augie Fackler <durin42@gmail.com> parents: 14341 diff changeset	593 'Connection appeared closed in read on first'
a75e0f4ba0ab httpclient: import revision fc731618702a of py-nonblocking-http Augie Fackler <durin42@gmail.com> parents: 14341 diff changeset	594 ' request loop iteration, will retry.')
a75e0f4ba0ab httpclient: import revision fc731618702a of py-nonblocking-http Augie Fackler <durin42@gmail.com> parents: 14341 diff changeset	595 reconnect('read')
a75e0f4ba0ab httpclient: import revision fc731618702a of py-nonblocking-http Augie Fackler <durin42@gmail.com> parents: 14341 diff changeset	596 continue
a75e0f4ba0ab httpclient: import revision fc731618702a of py-nonblocking-http Augie Fackler <durin42@gmail.com> parents: 14341 diff changeset	597 else:
a75e0f4ba0ab httpclient: import revision fc731618702a of py-nonblocking-http Augie Fackler <durin42@gmail.com> parents: 14341 diff changeset	598 # We didn't just send the first data hunk,
a75e0f4ba0ab httpclient: import revision fc731618702a of py-nonblocking-http Augie Fackler <durin42@gmail.com> parents: 14341 diff changeset	599 # and either have a partial response or no
a75e0f4ba0ab httpclient: import revision fc731618702a of py-nonblocking-http Augie Fackler <durin42@gmail.com> parents: 14341 diff changeset	600 # response at all. There's really nothing
a75e0f4ba0ab httpclient: import revision fc731618702a of py-nonblocking-http Augie Fackler <durin42@gmail.com> parents: 14341 diff changeset	601 # meaningful we can do here.
a75e0f4ba0ab httpclient: import revision fc731618702a of py-nonblocking-http Augie Fackler <durin42@gmail.com> parents: 14341 diff changeset	602 raise HTTPStateError(
a75e0f4ba0ab httpclient: import revision fc731618702a of py-nonblocking-http Augie Fackler <durin42@gmail.com> parents: 14341 diff changeset	603 'Connection appears closed after '
a75e0f4ba0ab httpclient: import revision fc731618702a of py-nonblocking-http Augie Fackler <durin42@gmail.com> parents: 14341 diff changeset	604 'some request data was written, but the '
a75e0f4ba0ab httpclient: import revision fc731618702a of py-nonblocking-http Augie Fackler <durin42@gmail.com> parents: 14341 diff changeset	605 'response was missing or incomplete!')
a75e0f4ba0ab httpclient: import revision fc731618702a of py-nonblocking-http Augie Fackler <durin42@gmail.com> parents: 14341 diff changeset	606 logger.debug('read %d bytes in request()', len(data))
14243 861f28212398 Import new http library as mercurial.httpclient. Augie Fackler <durin42@gmail.com> parents: diff changeset	607 if response is None:
861f28212398 Import new http library as mercurial.httpclient. Augie Fackler <durin42@gmail.com> parents: diff changeset	608 response = self.response_class(r[0], self.timeout)
861f28212398 Import new http library as mercurial.httpclient. Augie Fackler <durin42@gmail.com> parents: diff changeset	609 response._load_response(data)
14376 a75e0f4ba0ab httpclient: import revision fc731618702a of py-nonblocking-http Augie Fackler <durin42@gmail.com> parents: 14341 diff changeset	610 # Jump to the next select() call so we load more
a75e0f4ba0ab httpclient: import revision fc731618702a of py-nonblocking-http Augie Fackler <durin42@gmail.com> parents: 14341 diff changeset	611 # data if the server is still sending us content.
a75e0f4ba0ab httpclient: import revision fc731618702a of py-nonblocking-http Augie Fackler <durin42@gmail.com> parents: 14341 diff changeset	612 continue
14243 861f28212398 Import new http library as mercurial.httpclient. Augie Fackler <durin42@gmail.com> parents: diff changeset	613 except socket.error, e:
861f28212398 Import new http library as mercurial.httpclient. Augie Fackler <durin42@gmail.com> parents: diff changeset	614 if e[0] != errno.EPIPE and not was_first:
861f28212398 Import new http library as mercurial.httpclient. Augie Fackler <durin42@gmail.com> parents: diff changeset	615 raise
861f28212398 Import new http library as mercurial.httpclient. Augie Fackler <durin42@gmail.com> parents: diff changeset	616 if (response._content_len
861f28212398 Import new http library as mercurial.httpclient. Augie Fackler <durin42@gmail.com> parents: diff changeset	617 and response._content_len != _LEN_CLOSE_IS_END):
861f28212398 Import new http library as mercurial.httpclient. Augie Fackler <durin42@gmail.com> parents: diff changeset	618 outgoing_headers = sent_data + outgoing_headers
861f28212398 Import new http library as mercurial.httpclient. Augie Fackler <durin42@gmail.com> parents: diff changeset	619 reconnect('read')
861f28212398 Import new http library as mercurial.httpclient. Augie Fackler <durin42@gmail.com> parents: diff changeset	620
861f28212398 Import new http library as mercurial.httpclient. Augie Fackler <durin42@gmail.com> parents: diff changeset	621 # outgoing data
861f28212398 Import new http library as mercurial.httpclient. Augie Fackler <durin42@gmail.com> parents: diff changeset	622 if w and out:
861f28212398 Import new http library as mercurial.httpclient. Augie Fackler <durin42@gmail.com> parents: diff changeset	623 try:
861f28212398 Import new http library as mercurial.httpclient. Augie Fackler <durin42@gmail.com> parents: diff changeset	624 if getattr(out, 'read', False):
861f28212398 Import new http library as mercurial.httpclient. Augie Fackler <durin42@gmail.com> parents: diff changeset	625 data = out.read(OUTGOING_BUFFER_SIZE)
861f28212398 Import new http library as mercurial.httpclient. Augie Fackler <durin42@gmail.com> parents: diff changeset	626 if not data:
861f28212398 Import new http library as mercurial.httpclient. Augie Fackler <durin42@gmail.com> parents: diff changeset	627 continue
861f28212398 Import new http library as mercurial.httpclient. Augie Fackler <durin42@gmail.com> parents: diff changeset	628 if len(data) < OUTGOING_BUFFER_SIZE:
861f28212398 Import new http library as mercurial.httpclient. Augie Fackler <durin42@gmail.com> parents: diff changeset	629 if chunked:
861f28212398 Import new http library as mercurial.httpclient. Augie Fackler <durin42@gmail.com> parents: diff changeset	630 body = '0' + EOL + EOL
861f28212398 Import new http library as mercurial.httpclient. Augie Fackler <durin42@gmail.com> parents: diff changeset	631 else:
861f28212398 Import new http library as mercurial.httpclient. Augie Fackler <durin42@gmail.com> parents: diff changeset	632 body = None
861f28212398 Import new http library as mercurial.httpclient. Augie Fackler <durin42@gmail.com> parents: diff changeset	633 if chunked:
861f28212398 Import new http library as mercurial.httpclient. Augie Fackler <durin42@gmail.com> parents: diff changeset	634 out = hex(len(data))[2:] + EOL + data + EOL
861f28212398 Import new http library as mercurial.httpclient. Augie Fackler <durin42@gmail.com> parents: diff changeset	635 else:
861f28212398 Import new http library as mercurial.httpclient. Augie Fackler <durin42@gmail.com> parents: diff changeset	636 out = data
861f28212398 Import new http library as mercurial.httpclient. Augie Fackler <durin42@gmail.com> parents: diff changeset	637 amt = w[0].send(out)
861f28212398 Import new http library as mercurial.httpclient. Augie Fackler <durin42@gmail.com> parents: diff changeset	638 except socket.error, e:
861f28212398 Import new http library as mercurial.httpclient. Augie Fackler <durin42@gmail.com> parents: diff changeset	639 if e[0] == socket.SSL_ERROR_WANT_WRITE and self.ssl:
861f28212398 Import new http library as mercurial.httpclient. Augie Fackler <durin42@gmail.com> parents: diff changeset	640 # This means that SSL hasn't flushed its buffer into
861f28212398 Import new http library as mercurial.httpclient. Augie Fackler <durin42@gmail.com> parents: diff changeset	641 # the socket yet.
861f28212398 Import new http library as mercurial.httpclient. Augie Fackler <durin42@gmail.com> parents: diff changeset	642 # TODO: find a way to block on ssl flushing its buffer
861f28212398 Import new http library as mercurial.httpclient. Augie Fackler <durin42@gmail.com> parents: diff changeset	643 # similar to selecting on a raw socket.
861f28212398 Import new http library as mercurial.httpclient. Augie Fackler <durin42@gmail.com> parents: diff changeset	644 continue
861f28212398 Import new http library as mercurial.httpclient. Augie Fackler <durin42@gmail.com> parents: diff changeset	645 elif (e[0] not in (errno.ECONNRESET, errno.EPIPE)
861f28212398 Import new http library as mercurial.httpclient. Augie Fackler <durin42@gmail.com> parents: diff changeset	646 and not first):
861f28212398 Import new http library as mercurial.httpclient. Augie Fackler <durin42@gmail.com> parents: diff changeset	647 raise
861f28212398 Import new http library as mercurial.httpclient. Augie Fackler <durin42@gmail.com> parents: diff changeset	648 reconnect('write')
861f28212398 Import new http library as mercurial.httpclient. Augie Fackler <durin42@gmail.com> parents: diff changeset	649 amt = self.sock.send(out)
861f28212398 Import new http library as mercurial.httpclient. Augie Fackler <durin42@gmail.com> parents: diff changeset	650 logger.debug('sent %d', amt)
861f28212398 Import new http library as mercurial.httpclient. Augie Fackler <durin42@gmail.com> parents: diff changeset	651 first = False
861f28212398 Import new http library as mercurial.httpclient. Augie Fackler <durin42@gmail.com> parents: diff changeset	652 # stash data we think we sent in case the socket breaks
861f28212398 Import new http library as mercurial.httpclient. Augie Fackler <durin42@gmail.com> parents: diff changeset	653 # when we read from it
861f28212398 Import new http library as mercurial.httpclient. Augie Fackler <durin42@gmail.com> parents: diff changeset	654 if was_first:
861f28212398 Import new http library as mercurial.httpclient. Augie Fackler <durin42@gmail.com> parents: diff changeset	655 sent_data = out[:amt]
861f28212398 Import new http library as mercurial.httpclient. Augie Fackler <durin42@gmail.com> parents: diff changeset	656 if out is body:
861f28212398 Import new http library as mercurial.httpclient. Augie Fackler <durin42@gmail.com> parents: diff changeset	657 body = out[amt:]
861f28212398 Import new http library as mercurial.httpclient. Augie Fackler <durin42@gmail.com> parents: diff changeset	658 else:
861f28212398 Import new http library as mercurial.httpclient. Augie Fackler <durin42@gmail.com> parents: diff changeset	659 outgoing_headers = out[amt:]
861f28212398 Import new http library as mercurial.httpclient. Augie Fackler <durin42@gmail.com> parents: diff changeset	660
861f28212398 Import new http library as mercurial.httpclient. Augie Fackler <durin42@gmail.com> parents: diff changeset	661 # close if the server response said to or responded before eating
861f28212398 Import new http library as mercurial.httpclient. Augie Fackler <durin42@gmail.com> parents: diff changeset	662 # the whole request
861f28212398 Import new http library as mercurial.httpclient. Augie Fackler <durin42@gmail.com> parents: diff changeset	663 if response is None:
861f28212398 Import new http library as mercurial.httpclient. Augie Fackler <durin42@gmail.com> parents: diff changeset	664 response = self.response_class(self.sock, self.timeout)
861f28212398 Import new http library as mercurial.httpclient. Augie Fackler <durin42@gmail.com> parents: diff changeset	665 complete = response.complete()
861f28212398 Import new http library as mercurial.httpclient. Augie Fackler <durin42@gmail.com> parents: diff changeset	666 data_left = bool(outgoing_headers or body)
861f28212398 Import new http library as mercurial.httpclient. Augie Fackler <durin42@gmail.com> parents: diff changeset	667 if data_left:
861f28212398 Import new http library as mercurial.httpclient. Augie Fackler <durin42@gmail.com> parents: diff changeset	668 logger.info('stopped sending request early, '
861f28212398 Import new http library as mercurial.httpclient. Augie Fackler <durin42@gmail.com> parents: diff changeset	669 'will close the socket to be safe.')
861f28212398 Import new http library as mercurial.httpclient. Augie Fackler <durin42@gmail.com> parents: diff changeset	670 response.will_close = True
861f28212398 Import new http library as mercurial.httpclient. Augie Fackler <durin42@gmail.com> parents: diff changeset	671 if response.will_close:
861f28212398 Import new http library as mercurial.httpclient. Augie Fackler <durin42@gmail.com> parents: diff changeset	672 # The socket will be closed by the response, so we disown
861f28212398 Import new http library as mercurial.httpclient. Augie Fackler <durin42@gmail.com> parents: diff changeset	673 # the socket
861f28212398 Import new http library as mercurial.httpclient. Augie Fackler <durin42@gmail.com> parents: diff changeset	674 self.sock = None
861f28212398 Import new http library as mercurial.httpclient. Augie Fackler <durin42@gmail.com> parents: diff changeset	675 self._current_response = response
861f28212398 Import new http library as mercurial.httpclient. Augie Fackler <durin42@gmail.com> parents: diff changeset	676
861f28212398 Import new http library as mercurial.httpclient. Augie Fackler <durin42@gmail.com> parents: diff changeset	677 def getresponse(self):
861f28212398 Import new http library as mercurial.httpclient. Augie Fackler <durin42@gmail.com> parents: diff changeset	678 if self._current_response is None:
861f28212398 Import new http library as mercurial.httpclient. Augie Fackler <durin42@gmail.com> parents: diff changeset	679 raise httplib.ResponseNotReady()
861f28212398 Import new http library as mercurial.httpclient. Augie Fackler <durin42@gmail.com> parents: diff changeset	680 r = self._current_response
861f28212398 Import new http library as mercurial.httpclient. Augie Fackler <durin42@gmail.com> parents: diff changeset	681 while r.headers is None:
861f28212398 Import new http library as mercurial.httpclient. Augie Fackler <durin42@gmail.com> parents: diff changeset	682 r._select()
14293 9adbb5ef0964 httpclient: import f4c380237fd5 to fix keepalive not working Augie Fackler <durin42@gmail.com> parents: 14243 diff changeset	683 if r.will_close:
14243 861f28212398 Import new http library as mercurial.httpclient. Augie Fackler <durin42@gmail.com> parents: diff changeset	684 self.sock = None
861f28212398 Import new http library as mercurial.httpclient. Augie Fackler <durin42@gmail.com> parents: diff changeset	685 self._current_response = None
14293 9adbb5ef0964 httpclient: import f4c380237fd5 to fix keepalive not working Augie Fackler <durin42@gmail.com> parents: 14243 diff changeset	686 elif r.complete():
9adbb5ef0964 httpclient: import f4c380237fd5 to fix keepalive not working Augie Fackler <durin42@gmail.com> parents: 14243 diff changeset	687 self._current_response = None
14243 861f28212398 Import new http library as mercurial.httpclient. Augie Fackler <durin42@gmail.com> parents: diff changeset	688 else:
861f28212398 Import new http library as mercurial.httpclient. Augie Fackler <durin42@gmail.com> parents: diff changeset	689 self._current_response_taken = True
861f28212398 Import new http library as mercurial.httpclient. Augie Fackler <durin42@gmail.com> parents: diff changeset	690 return r
861f28212398 Import new http library as mercurial.httpclient. Augie Fackler <durin42@gmail.com> parents: diff changeset	691
861f28212398 Import new http library as mercurial.httpclient. Augie Fackler <durin42@gmail.com> parents: diff changeset	692
861f28212398 Import new http library as mercurial.httpclient. Augie Fackler <durin42@gmail.com> parents: diff changeset	693 class HTTPTimeoutException(httplib.HTTPException):
861f28212398 Import new http library as mercurial.httpclient. Augie Fackler <durin42@gmail.com> parents: diff changeset	694 """A timeout occurred while waiting on the server."""
861f28212398 Import new http library as mercurial.httpclient. Augie Fackler <durin42@gmail.com> parents: diff changeset	695
861f28212398 Import new http library as mercurial.httpclient. Augie Fackler <durin42@gmail.com> parents: diff changeset	696
861f28212398 Import new http library as mercurial.httpclient. Augie Fackler <durin42@gmail.com> parents: diff changeset	697 class BadRequestData(httplib.HTTPException):
861f28212398 Import new http library as mercurial.httpclient. Augie Fackler <durin42@gmail.com> parents: diff changeset	698 """Request body object has neither __len__ nor read."""
861f28212398 Import new http library as mercurial.httpclient. Augie Fackler <durin42@gmail.com> parents: diff changeset	699
861f28212398 Import new http library as mercurial.httpclient. Augie Fackler <durin42@gmail.com> parents: diff changeset	700
861f28212398 Import new http library as mercurial.httpclient. Augie Fackler <durin42@gmail.com> parents: diff changeset	701 class HTTPProxyConnectFailedException(httplib.HTTPException):
861f28212398 Import new http library as mercurial.httpclient. Augie Fackler <durin42@gmail.com> parents: diff changeset	702 """Connecting to the HTTP proxy failed."""
14376 a75e0f4ba0ab httpclient: import revision fc731618702a of py-nonblocking-http Augie Fackler <durin42@gmail.com> parents: 14341 diff changeset	703
15218 c81dce8a7bb6 httpclient: update to 07d8c356f4d1 of py-nonblocking-http Augie Fackler <durin42@gmail.com> parents: 14376 diff changeset	704
14376 a75e0f4ba0ab httpclient: import revision fc731618702a of py-nonblocking-http Augie Fackler <durin42@gmail.com> parents: 14341 diff changeset	705 class HTTPStateError(httplib.HTTPException):
a75e0f4ba0ab httpclient: import revision fc731618702a of py-nonblocking-http Augie Fackler <durin42@gmail.com> parents: 14341 diff changeset	706 """Invalid internal state encountered."""
15218 c81dce8a7bb6 httpclient: update to 07d8c356f4d1 of py-nonblocking-http Augie Fackler <durin42@gmail.com> parents: 14376 diff changeset	707
c81dce8a7bb6 httpclient: update to 07d8c356f4d1 of py-nonblocking-http Augie Fackler <durin42@gmail.com> parents: 14376 diff changeset	708
c81dce8a7bb6 httpclient: update to 07d8c356f4d1 of py-nonblocking-http Augie Fackler <durin42@gmail.com> parents: 14376 diff changeset	709 class HTTPRemoteClosedError(httplib.HTTPException):
c81dce8a7bb6 httpclient: update to 07d8c356f4d1 of py-nonblocking-http Augie Fackler <durin42@gmail.com> parents: 14376 diff changeset	710 """The server closed the remote socket in the middle of a response."""
14243 861f28212398 Import new http library as mercurial.httpclient. Augie Fackler <durin42@gmail.com> parents: diff changeset	711 # no-check-code

Mercurial > public > mercurial-scm > hg

annotate mercurial/httpclient/__init__.py @ 15218:c81dce8a7bb6

annotate mercurial/httpclient/init.py @ 15218:c81dce8a7bb6