mercurial-scm/hg: hgext/lfs/wireprotolfsserver.py annotate

annotate hgext/lfs/wireprotolfsserver.py @ 37692:10e5bb9678f4

lfs: gracefully handle aborts on the server when corrupt blobs are detected The aborts weren't killing the server, but this seems cleaner. I'm not sure if it matters to handle the remaining IOError in the test like this, for consistency. The error code still feels wrong (especially if the client is trying to download a corrupt blob) but I don't see anything better in the RFCs, and this is already used elsewhere because the Batch API spec specifically mentioned this as a "Validation Error".

author	Matt Harbison <matt_harbison@yahoo.com>
date	Sun, 25 Feb 2018 14:07:13 -0500
parents	726c4102db9e
children	31a0d47d69b3

rev	line source
37147 a2566597acb5 lfs: add basic routing for the server side wire protocol processing Matt Harbison <matt_harbison@yahoo.com> parents: diff changeset	1 # wireprotolfsserver.py - lfs protocol server side implementation
a2566597acb5 lfs: add basic routing for the server side wire protocol processing Matt Harbison <matt_harbison@yahoo.com> parents: diff changeset	2 #
a2566597acb5 lfs: add basic routing for the server side wire protocol processing Matt Harbison <matt_harbison@yahoo.com> parents: diff changeset	3 # Copyright 2018 Matt Harbison <matt_harbison@yahoo.com>
a2566597acb5 lfs: add basic routing for the server side wire protocol processing Matt Harbison <matt_harbison@yahoo.com> parents: diff changeset	4 #
a2566597acb5 lfs: add basic routing for the server side wire protocol processing Matt Harbison <matt_harbison@yahoo.com> parents: diff changeset	5 # This software may be used and distributed according to the terms of the
a2566597acb5 lfs: add basic routing for the server side wire protocol processing Matt Harbison <matt_harbison@yahoo.com> parents: diff changeset	6 # GNU General Public License version 2 or any later version.
a2566597acb5 lfs: add basic routing for the server side wire protocol processing Matt Harbison <matt_harbison@yahoo.com> parents: diff changeset	7
a2566597acb5 lfs: add basic routing for the server side wire protocol processing Matt Harbison <matt_harbison@yahoo.com> parents: diff changeset	8 from __future__ import absolute_import
a2566597acb5 lfs: add basic routing for the server side wire protocol processing Matt Harbison <matt_harbison@yahoo.com> parents: diff changeset	9
37148 ea6fc58524d7 lfs: add server side support for the Batch API Matt Harbison <matt_harbison@yahoo.com> parents: 37147 diff changeset	10 import datetime
ea6fc58524d7 lfs: add server side support for the Batch API Matt Harbison <matt_harbison@yahoo.com> parents: 37147 diff changeset	11 import errno
ea6fc58524d7 lfs: add server side support for the Batch API Matt Harbison <matt_harbison@yahoo.com> parents: 37147 diff changeset	12 import json
37690 726c4102db9e lfs: log information about Internal Server Errors reported in the Batch API Matt Harbison <matt_harbison@yahoo.com> parents: 37617 diff changeset	13 import traceback
37148 ea6fc58524d7 lfs: add server side support for the Batch API Matt Harbison <matt_harbison@yahoo.com> parents: 37147 diff changeset	14
37147 a2566597acb5 lfs: add basic routing for the server side wire protocol processing Matt Harbison <matt_harbison@yahoo.com> parents: diff changeset	15 from mercurial.hgweb import (
a2566597acb5 lfs: add basic routing for the server side wire protocol processing Matt Harbison <matt_harbison@yahoo.com> parents: diff changeset	16 common as hgwebcommon,
a2566597acb5 lfs: add basic routing for the server side wire protocol processing Matt Harbison <matt_harbison@yahoo.com> parents: diff changeset	17 )
a2566597acb5 lfs: add basic routing for the server side wire protocol processing Matt Harbison <matt_harbison@yahoo.com> parents: diff changeset	18
a2566597acb5 lfs: add basic routing for the server side wire protocol processing Matt Harbison <matt_harbison@yahoo.com> parents: diff changeset	19 from mercurial import (
a2566597acb5 lfs: add basic routing for the server side wire protocol processing Matt Harbison <matt_harbison@yahoo.com> parents: diff changeset	20 pycompat,
a2566597acb5 lfs: add basic routing for the server side wire protocol processing Matt Harbison <matt_harbison@yahoo.com> parents: diff changeset	21 )
a2566597acb5 lfs: add basic routing for the server side wire protocol processing Matt Harbison <matt_harbison@yahoo.com> parents: diff changeset	22
37692 10e5bb9678f4 lfs: gracefully handle aborts on the server when corrupt blobs are detected Matt Harbison <matt_harbison@yahoo.com> parents: 37690 diff changeset	23 from . import blobstore
10e5bb9678f4 lfs: gracefully handle aborts on the server when corrupt blobs are detected Matt Harbison <matt_harbison@yahoo.com> parents: 37690 diff changeset	24
37148 ea6fc58524d7 lfs: add server side support for the Batch API Matt Harbison <matt_harbison@yahoo.com> parents: 37147 diff changeset	25 HTTP_OK = hgwebcommon.HTTP_OK
37149 cc0a6ea95d98 lfs: add support for serving blob files Matt Harbison <matt_harbison@yahoo.com> parents: 37148 diff changeset	26 HTTP_CREATED = hgwebcommon.HTTP_CREATED
37148 ea6fc58524d7 lfs: add server side support for the Batch API Matt Harbison <matt_harbison@yahoo.com> parents: 37147 diff changeset	27 HTTP_BAD_REQUEST = hgwebcommon.HTTP_BAD_REQUEST
37250 9640ccf44ac0 lfs: ensure the transfer request is for a known URI Matt Harbison <matt_harbison@yahoo.com> parents: 37249 diff changeset	28 HTTP_NOT_FOUND = hgwebcommon.HTTP_NOT_FOUND
37148 ea6fc58524d7 lfs: add server side support for the Batch API Matt Harbison <matt_harbison@yahoo.com> parents: 37147 diff changeset	29
37147 a2566597acb5 lfs: add basic routing for the server side wire protocol processing Matt Harbison <matt_harbison@yahoo.com> parents: diff changeset	30 def handlewsgirequest(orig, rctx, req, res, checkperm):
a2566597acb5 lfs: add basic routing for the server side wire protocol processing Matt Harbison <matt_harbison@yahoo.com> parents: diff changeset	31 """Wrap wireprotoserver.handlewsgirequest() to possibly process an LFS
a2566597acb5 lfs: add basic routing for the server side wire protocol processing Matt Harbison <matt_harbison@yahoo.com> parents: diff changeset	32 request if it is left unprocessed by the wrapped method.
a2566597acb5 lfs: add basic routing for the server side wire protocol processing Matt Harbison <matt_harbison@yahoo.com> parents: diff changeset	33 """
a2566597acb5 lfs: add basic routing for the server side wire protocol processing Matt Harbison <matt_harbison@yahoo.com> parents: diff changeset	34 if orig(rctx, req, res, checkperm):
a2566597acb5 lfs: add basic routing for the server side wire protocol processing Matt Harbison <matt_harbison@yahoo.com> parents: diff changeset	35 return True
a2566597acb5 lfs: add basic routing for the server side wire protocol processing Matt Harbison <matt_harbison@yahoo.com> parents: diff changeset	36
37248 dfb38c4850a9 lfs: add an experimental knob to disable blob serving Matt Harbison <matt_harbison@yahoo.com> parents: 37149 diff changeset	37 if not rctx.repo.ui.configbool('experimental', 'lfs.serve'):
dfb38c4850a9 lfs: add an experimental knob to disable blob serving Matt Harbison <matt_harbison@yahoo.com> parents: 37149 diff changeset	38 return False
dfb38c4850a9 lfs: add an experimental knob to disable blob serving Matt Harbison <matt_harbison@yahoo.com> parents: 37149 diff changeset	39
37147 a2566597acb5 lfs: add basic routing for the server side wire protocol processing Matt Harbison <matt_harbison@yahoo.com> parents: diff changeset	40 if not req.dispatchpath:
a2566597acb5 lfs: add basic routing for the server side wire protocol processing Matt Harbison <matt_harbison@yahoo.com> parents: diff changeset	41 return False
a2566597acb5 lfs: add basic routing for the server side wire protocol processing Matt Harbison <matt_harbison@yahoo.com> parents: diff changeset	42
a2566597acb5 lfs: add basic routing for the server side wire protocol processing Matt Harbison <matt_harbison@yahoo.com> parents: diff changeset	43 try:
a2566597acb5 lfs: add basic routing for the server side wire protocol processing Matt Harbison <matt_harbison@yahoo.com> parents: diff changeset	44 if req.dispatchpath == b'.git/info/lfs/objects/batch':
a2566597acb5 lfs: add basic routing for the server side wire protocol processing Matt Harbison <matt_harbison@yahoo.com> parents: diff changeset	45 checkperm(rctx, req, 'pull')
a2566597acb5 lfs: add basic routing for the server side wire protocol processing Matt Harbison <matt_harbison@yahoo.com> parents: diff changeset	46 return _processbatchrequest(rctx.repo, req, res)
a2566597acb5 lfs: add basic routing for the server side wire protocol processing Matt Harbison <matt_harbison@yahoo.com> parents: diff changeset	47 # TODO: reserve and use a path in the proposed http wireprotocol /api/
a2566597acb5 lfs: add basic routing for the server side wire protocol processing Matt Harbison <matt_harbison@yahoo.com> parents: diff changeset	48 # namespace?
a2566597acb5 lfs: add basic routing for the server side wire protocol processing Matt Harbison <matt_harbison@yahoo.com> parents: diff changeset	49 elif req.dispatchpath.startswith(b'.hg/lfs/objects'):
a2566597acb5 lfs: add basic routing for the server side wire protocol processing Matt Harbison <matt_harbison@yahoo.com> parents: diff changeset	50 return _processbasictransfer(rctx.repo, req, res,
a2566597acb5 lfs: add basic routing for the server side wire protocol processing Matt Harbison <matt_harbison@yahoo.com> parents: diff changeset	51 lambda perm:
a2566597acb5 lfs: add basic routing for the server side wire protocol processing Matt Harbison <matt_harbison@yahoo.com> parents: diff changeset	52 checkperm(rctx, req, perm))
a2566597acb5 lfs: add basic routing for the server side wire protocol processing Matt Harbison <matt_harbison@yahoo.com> parents: diff changeset	53 return False
a2566597acb5 lfs: add basic routing for the server side wire protocol processing Matt Harbison <matt_harbison@yahoo.com> parents: diff changeset	54 except hgwebcommon.ErrorResponse as e:
a2566597acb5 lfs: add basic routing for the server side wire protocol processing Matt Harbison <matt_harbison@yahoo.com> parents: diff changeset	55 # XXX: copied from the handler surrounding wireprotoserver._callhttp()
a2566597acb5 lfs: add basic routing for the server side wire protocol processing Matt Harbison <matt_harbison@yahoo.com> parents: diff changeset	56 # in the wrapped function. Should this be moved back to hgweb to
a2566597acb5 lfs: add basic routing for the server side wire protocol processing Matt Harbison <matt_harbison@yahoo.com> parents: diff changeset	57 # be a common handler?
a2566597acb5 lfs: add basic routing for the server side wire protocol processing Matt Harbison <matt_harbison@yahoo.com> parents: diff changeset	58 for k, v in e.headers:
a2566597acb5 lfs: add basic routing for the server side wire protocol processing Matt Harbison <matt_harbison@yahoo.com> parents: diff changeset	59 res.headers[k] = v
a2566597acb5 lfs: add basic routing for the server side wire protocol processing Matt Harbison <matt_harbison@yahoo.com> parents: diff changeset	60 res.status = hgwebcommon.statusmessage(e.code, pycompat.bytestr(e))
a2566597acb5 lfs: add basic routing for the server side wire protocol processing Matt Harbison <matt_harbison@yahoo.com> parents: diff changeset	61 res.setbodybytes(b'0\n%s\n' % pycompat.bytestr(e))
a2566597acb5 lfs: add basic routing for the server side wire protocol processing Matt Harbison <matt_harbison@yahoo.com> parents: diff changeset	62 return True
a2566597acb5 lfs: add basic routing for the server side wire protocol processing Matt Harbison <matt_harbison@yahoo.com> parents: diff changeset	63
37148 ea6fc58524d7 lfs: add server side support for the Batch API Matt Harbison <matt_harbison@yahoo.com> parents: 37147 diff changeset	64 def _sethttperror(res, code, message=None):
ea6fc58524d7 lfs: add server side support for the Batch API Matt Harbison <matt_harbison@yahoo.com> parents: 37147 diff changeset	65 res.status = hgwebcommon.statusmessage(code, message=message)
ea6fc58524d7 lfs: add server side support for the Batch API Matt Harbison <matt_harbison@yahoo.com> parents: 37147 diff changeset	66 res.headers[b'Content-Type'] = b'text/plain; charset=utf-8'
ea6fc58524d7 lfs: add server side support for the Batch API Matt Harbison <matt_harbison@yahoo.com> parents: 37147 diff changeset	67 res.setbodybytes(b'')
ea6fc58524d7 lfs: add server side support for the Batch API Matt Harbison <matt_harbison@yahoo.com> parents: 37147 diff changeset	68
37690 726c4102db9e lfs: log information about Internal Server Errors reported in the Batch API Matt Harbison <matt_harbison@yahoo.com> parents: 37617 diff changeset	69 def _logexception(req):
726c4102db9e lfs: log information about Internal Server Errors reported in the Batch API Matt Harbison <matt_harbison@yahoo.com> parents: 37617 diff changeset	70 """Write information about the current exception to wsgi.errors."""
726c4102db9e lfs: log information about Internal Server Errors reported in the Batch API Matt Harbison <matt_harbison@yahoo.com> parents: 37617 diff changeset	71 tb = pycompat.sysbytes(traceback.format_exc())
726c4102db9e lfs: log information about Internal Server Errors reported in the Batch API Matt Harbison <matt_harbison@yahoo.com> parents: 37617 diff changeset	72 errorlog = req.rawenv[r'wsgi.errors']
726c4102db9e lfs: log information about Internal Server Errors reported in the Batch API Matt Harbison <matt_harbison@yahoo.com> parents: 37617 diff changeset	73
726c4102db9e lfs: log information about Internal Server Errors reported in the Batch API Matt Harbison <matt_harbison@yahoo.com> parents: 37617 diff changeset	74 uri = b''
726c4102db9e lfs: log information about Internal Server Errors reported in the Batch API Matt Harbison <matt_harbison@yahoo.com> parents: 37617 diff changeset	75 if req.apppath:
726c4102db9e lfs: log information about Internal Server Errors reported in the Batch API Matt Harbison <matt_harbison@yahoo.com> parents: 37617 diff changeset	76 uri += req.apppath
726c4102db9e lfs: log information about Internal Server Errors reported in the Batch API Matt Harbison <matt_harbison@yahoo.com> parents: 37617 diff changeset	77 uri += b'/' + req.dispatchpath
726c4102db9e lfs: log information about Internal Server Errors reported in the Batch API Matt Harbison <matt_harbison@yahoo.com> parents: 37617 diff changeset	78
726c4102db9e lfs: log information about Internal Server Errors reported in the Batch API Matt Harbison <matt_harbison@yahoo.com> parents: 37617 diff changeset	79 errorlog.write(b"Exception happened while processing request '%s':\n%s" %
726c4102db9e lfs: log information about Internal Server Errors reported in the Batch API Matt Harbison <matt_harbison@yahoo.com> parents: 37617 diff changeset	80 (uri, tb))
726c4102db9e lfs: log information about Internal Server Errors reported in the Batch API Matt Harbison <matt_harbison@yahoo.com> parents: 37617 diff changeset	81
37147 a2566597acb5 lfs: add basic routing for the server side wire protocol processing Matt Harbison <matt_harbison@yahoo.com> parents: diff changeset	82 def _processbatchrequest(repo, req, res):
a2566597acb5 lfs: add basic routing for the server side wire protocol processing Matt Harbison <matt_harbison@yahoo.com> parents: diff changeset	83 """Handle a request for the Batch API, which is the gateway to granting file
a2566597acb5 lfs: add basic routing for the server side wire protocol processing Matt Harbison <matt_harbison@yahoo.com> parents: diff changeset	84 access.
a2566597acb5 lfs: add basic routing for the server side wire protocol processing Matt Harbison <matt_harbison@yahoo.com> parents: diff changeset	85
a2566597acb5 lfs: add basic routing for the server side wire protocol processing Matt Harbison <matt_harbison@yahoo.com> parents: diff changeset	86 https://github.com/git-lfs/git-lfs/blob/master/docs/api/batch.md
a2566597acb5 lfs: add basic routing for the server side wire protocol processing Matt Harbison <matt_harbison@yahoo.com> parents: diff changeset	87 """
37148 ea6fc58524d7 lfs: add server side support for the Batch API Matt Harbison <matt_harbison@yahoo.com> parents: 37147 diff changeset	88
ea6fc58524d7 lfs: add server side support for the Batch API Matt Harbison <matt_harbison@yahoo.com> parents: 37147 diff changeset	89 # Mercurial client request:
ea6fc58524d7 lfs: add server side support for the Batch API Matt Harbison <matt_harbison@yahoo.com> parents: 37147 diff changeset	90 #
ea6fc58524d7 lfs: add server side support for the Batch API Matt Harbison <matt_harbison@yahoo.com> parents: 37147 diff changeset	91 # HOST: localhost:$HGPORT
ea6fc58524d7 lfs: add server side support for the Batch API Matt Harbison <matt_harbison@yahoo.com> parents: 37147 diff changeset	92 # ACCEPT: application/vnd.git-lfs+json
ea6fc58524d7 lfs: add server side support for the Batch API Matt Harbison <matt_harbison@yahoo.com> parents: 37147 diff changeset	93 # ACCEPT-ENCODING: identity
ea6fc58524d7 lfs: add server side support for the Batch API Matt Harbison <matt_harbison@yahoo.com> parents: 37147 diff changeset	94 # USER-AGENT: git-lfs/2.3.4 (Mercurial 4.5.2+1114-f48b9754f04c+20180316)
ea6fc58524d7 lfs: add server side support for the Batch API Matt Harbison <matt_harbison@yahoo.com> parents: 37147 diff changeset	95 # Content-Length: 125
ea6fc58524d7 lfs: add server side support for the Batch API Matt Harbison <matt_harbison@yahoo.com> parents: 37147 diff changeset	96 # Content-Type: application/vnd.git-lfs+json
ea6fc58524d7 lfs: add server side support for the Batch API Matt Harbison <matt_harbison@yahoo.com> parents: 37147 diff changeset	97 #
ea6fc58524d7 lfs: add server side support for the Batch API Matt Harbison <matt_harbison@yahoo.com> parents: 37147 diff changeset	98 # {
ea6fc58524d7 lfs: add server side support for the Batch API Matt Harbison <matt_harbison@yahoo.com> parents: 37147 diff changeset	99 # "objects": [
ea6fc58524d7 lfs: add server side support for the Batch API Matt Harbison <matt_harbison@yahoo.com> parents: 37147 diff changeset	100 # {
ea6fc58524d7 lfs: add server side support for the Batch API Matt Harbison <matt_harbison@yahoo.com> parents: 37147 diff changeset	101 # "oid": "31cf...8e5b"
ea6fc58524d7 lfs: add server side support for the Batch API Matt Harbison <matt_harbison@yahoo.com> parents: 37147 diff changeset	102 # "size": 12
ea6fc58524d7 lfs: add server side support for the Batch API Matt Harbison <matt_harbison@yahoo.com> parents: 37147 diff changeset	103 # }
ea6fc58524d7 lfs: add server side support for the Batch API Matt Harbison <matt_harbison@yahoo.com> parents: 37147 diff changeset	104 # ]
ea6fc58524d7 lfs: add server side support for the Batch API Matt Harbison <matt_harbison@yahoo.com> parents: 37147 diff changeset	105 # "operation": "upload"
ea6fc58524d7 lfs: add server side support for the Batch API Matt Harbison <matt_harbison@yahoo.com> parents: 37147 diff changeset	106 # }
ea6fc58524d7 lfs: add server side support for the Batch API Matt Harbison <matt_harbison@yahoo.com> parents: 37147 diff changeset	107
ea6fc58524d7 lfs: add server side support for the Batch API Matt Harbison <matt_harbison@yahoo.com> parents: 37147 diff changeset	108 if (req.method != b'POST'
ea6fc58524d7 lfs: add server side support for the Batch API Matt Harbison <matt_harbison@yahoo.com> parents: 37147 diff changeset	109 or req.headers[b'Content-Type'] != b'application/vnd.git-lfs+json'
ea6fc58524d7 lfs: add server side support for the Batch API Matt Harbison <matt_harbison@yahoo.com> parents: 37147 diff changeset	110 or req.headers[b'Accept'] != b'application/vnd.git-lfs+json'):
ea6fc58524d7 lfs: add server side support for the Batch API Matt Harbison <matt_harbison@yahoo.com> parents: 37147 diff changeset	111 # TODO: figure out what the proper handling for a bad request to the
ea6fc58524d7 lfs: add server side support for the Batch API Matt Harbison <matt_harbison@yahoo.com> parents: 37147 diff changeset	112 # Batch API is.
ea6fc58524d7 lfs: add server side support for the Batch API Matt Harbison <matt_harbison@yahoo.com> parents: 37147 diff changeset	113 _sethttperror(res, HTTP_BAD_REQUEST, b'Invalid Batch API request')
ea6fc58524d7 lfs: add server side support for the Batch API Matt Harbison <matt_harbison@yahoo.com> parents: 37147 diff changeset	114 return True
ea6fc58524d7 lfs: add server side support for the Batch API Matt Harbison <matt_harbison@yahoo.com> parents: 37147 diff changeset	115
ea6fc58524d7 lfs: add server side support for the Batch API Matt Harbison <matt_harbison@yahoo.com> parents: 37147 diff changeset	116 # XXX: specify an encoding?
ea6fc58524d7 lfs: add server side support for the Batch API Matt Harbison <matt_harbison@yahoo.com> parents: 37147 diff changeset	117 lfsreq = json.loads(req.bodyfh.read())
ea6fc58524d7 lfs: add server side support for the Batch API Matt Harbison <matt_harbison@yahoo.com> parents: 37147 diff changeset	118
ea6fc58524d7 lfs: add server side support for the Batch API Matt Harbison <matt_harbison@yahoo.com> parents: 37147 diff changeset	119 # If no transfer handlers are explicitly requested, 'basic' is assumed.
ea6fc58524d7 lfs: add server side support for the Batch API Matt Harbison <matt_harbison@yahoo.com> parents: 37147 diff changeset	120 if 'basic' not in lfsreq.get('transfers', ['basic']):
ea6fc58524d7 lfs: add server side support for the Batch API Matt Harbison <matt_harbison@yahoo.com> parents: 37147 diff changeset	121 _sethttperror(res, HTTP_BAD_REQUEST,
ea6fc58524d7 lfs: add server side support for the Batch API Matt Harbison <matt_harbison@yahoo.com> parents: 37147 diff changeset	122 b'Only the basic LFS transfer handler is supported')
ea6fc58524d7 lfs: add server side support for the Batch API Matt Harbison <matt_harbison@yahoo.com> parents: 37147 diff changeset	123 return True
ea6fc58524d7 lfs: add server side support for the Batch API Matt Harbison <matt_harbison@yahoo.com> parents: 37147 diff changeset	124
ea6fc58524d7 lfs: add server side support for the Batch API Matt Harbison <matt_harbison@yahoo.com> parents: 37147 diff changeset	125 operation = lfsreq.get('operation')
ea6fc58524d7 lfs: add server side support for the Batch API Matt Harbison <matt_harbison@yahoo.com> parents: 37147 diff changeset	126 if operation not in ('upload', 'download'):
ea6fc58524d7 lfs: add server side support for the Batch API Matt Harbison <matt_harbison@yahoo.com> parents: 37147 diff changeset	127 _sethttperror(res, HTTP_BAD_REQUEST,
ea6fc58524d7 lfs: add server side support for the Batch API Matt Harbison <matt_harbison@yahoo.com> parents: 37147 diff changeset	128 b'Unsupported LFS transfer operation: %s' % operation)
ea6fc58524d7 lfs: add server side support for the Batch API Matt Harbison <matt_harbison@yahoo.com> parents: 37147 diff changeset	129 return True
ea6fc58524d7 lfs: add server side support for the Batch API Matt Harbison <matt_harbison@yahoo.com> parents: 37147 diff changeset	130
ea6fc58524d7 lfs: add server side support for the Batch API Matt Harbison <matt_harbison@yahoo.com> parents: 37147 diff changeset	131 localstore = repo.svfs.lfslocalblobstore
ea6fc58524d7 lfs: add server side support for the Batch API Matt Harbison <matt_harbison@yahoo.com> parents: 37147 diff changeset	132
ea6fc58524d7 lfs: add server side support for the Batch API Matt Harbison <matt_harbison@yahoo.com> parents: 37147 diff changeset	133 objects = [p for p in _batchresponseobjects(req, lfsreq.get('objects', []),
ea6fc58524d7 lfs: add server side support for the Batch API Matt Harbison <matt_harbison@yahoo.com> parents: 37147 diff changeset	134 operation, localstore)]
ea6fc58524d7 lfs: add server side support for the Batch API Matt Harbison <matt_harbison@yahoo.com> parents: 37147 diff changeset	135
ea6fc58524d7 lfs: add server side support for the Batch API Matt Harbison <matt_harbison@yahoo.com> parents: 37147 diff changeset	136 rsp = {
ea6fc58524d7 lfs: add server side support for the Batch API Matt Harbison <matt_harbison@yahoo.com> parents: 37147 diff changeset	137 'transfer': 'basic',
ea6fc58524d7 lfs: add server side support for the Batch API Matt Harbison <matt_harbison@yahoo.com> parents: 37147 diff changeset	138 'objects': objects,
ea6fc58524d7 lfs: add server side support for the Batch API Matt Harbison <matt_harbison@yahoo.com> parents: 37147 diff changeset	139 }
ea6fc58524d7 lfs: add server side support for the Batch API Matt Harbison <matt_harbison@yahoo.com> parents: 37147 diff changeset	140
ea6fc58524d7 lfs: add server side support for the Batch API Matt Harbison <matt_harbison@yahoo.com> parents: 37147 diff changeset	141 res.status = hgwebcommon.statusmessage(HTTP_OK)
ea6fc58524d7 lfs: add server side support for the Batch API Matt Harbison <matt_harbison@yahoo.com> parents: 37147 diff changeset	142 res.headers[b'Content-Type'] = b'application/vnd.git-lfs+json'
ea6fc58524d7 lfs: add server side support for the Batch API Matt Harbison <matt_harbison@yahoo.com> parents: 37147 diff changeset	143 res.setbodybytes(pycompat.bytestr(json.dumps(rsp)))
ea6fc58524d7 lfs: add server side support for the Batch API Matt Harbison <matt_harbison@yahoo.com> parents: 37147 diff changeset	144
ea6fc58524d7 lfs: add server side support for the Batch API Matt Harbison <matt_harbison@yahoo.com> parents: 37147 diff changeset	145 return True
ea6fc58524d7 lfs: add server side support for the Batch API Matt Harbison <matt_harbison@yahoo.com> parents: 37147 diff changeset	146
ea6fc58524d7 lfs: add server side support for the Batch API Matt Harbison <matt_harbison@yahoo.com> parents: 37147 diff changeset	147 def _batchresponseobjects(req, objects, action, store):
ea6fc58524d7 lfs: add server side support for the Batch API Matt Harbison <matt_harbison@yahoo.com> parents: 37147 diff changeset	148 """Yield one dictionary of attributes for the Batch API response for each
ea6fc58524d7 lfs: add server side support for the Batch API Matt Harbison <matt_harbison@yahoo.com> parents: 37147 diff changeset	149 object in the list.
ea6fc58524d7 lfs: add server side support for the Batch API Matt Harbison <matt_harbison@yahoo.com> parents: 37147 diff changeset	150
ea6fc58524d7 lfs: add server side support for the Batch API Matt Harbison <matt_harbison@yahoo.com> parents: 37147 diff changeset	151 req: The parsedrequest for the Batch API request
ea6fc58524d7 lfs: add server side support for the Batch API Matt Harbison <matt_harbison@yahoo.com> parents: 37147 diff changeset	152 objects: The list of objects in the Batch API object request list
ea6fc58524d7 lfs: add server side support for the Batch API Matt Harbison <matt_harbison@yahoo.com> parents: 37147 diff changeset	153 action: 'upload' or 'download'
ea6fc58524d7 lfs: add server side support for the Batch API Matt Harbison <matt_harbison@yahoo.com> parents: 37147 diff changeset	154 store: The local blob store for servicing requests"""
ea6fc58524d7 lfs: add server side support for the Batch API Matt Harbison <matt_harbison@yahoo.com> parents: 37147 diff changeset	155
ea6fc58524d7 lfs: add server side support for the Batch API Matt Harbison <matt_harbison@yahoo.com> parents: 37147 diff changeset	156 # Successful lfs-test-server response to solict an upload:
ea6fc58524d7 lfs: add server side support for the Batch API Matt Harbison <matt_harbison@yahoo.com> parents: 37147 diff changeset	157 # {
ea6fc58524d7 lfs: add server side support for the Batch API Matt Harbison <matt_harbison@yahoo.com> parents: 37147 diff changeset	158 # u'objects': [{
ea6fc58524d7 lfs: add server side support for the Batch API Matt Harbison <matt_harbison@yahoo.com> parents: 37147 diff changeset	159 # u'size': 12,
ea6fc58524d7 lfs: add server side support for the Batch API Matt Harbison <matt_harbison@yahoo.com> parents: 37147 diff changeset	160 # u'oid': u'31cf...8e5b',
ea6fc58524d7 lfs: add server side support for the Batch API Matt Harbison <matt_harbison@yahoo.com> parents: 37147 diff changeset	161 # u'actions': {
ea6fc58524d7 lfs: add server side support for the Batch API Matt Harbison <matt_harbison@yahoo.com> parents: 37147 diff changeset	162 # u'upload': {
ea6fc58524d7 lfs: add server side support for the Batch API Matt Harbison <matt_harbison@yahoo.com> parents: 37147 diff changeset	163 # u'href': u'http://localhost:$HGPORT/objects/31cf...8e5b',
ea6fc58524d7 lfs: add server side support for the Batch API Matt Harbison <matt_harbison@yahoo.com> parents: 37147 diff changeset	164 # u'expires_at': u'0001-01-01T00:00:00Z',
ea6fc58524d7 lfs: add server side support for the Batch API Matt Harbison <matt_harbison@yahoo.com> parents: 37147 diff changeset	165 # u'header': {
ea6fc58524d7 lfs: add server side support for the Batch API Matt Harbison <matt_harbison@yahoo.com> parents: 37147 diff changeset	166 # u'Accept': u'application/vnd.git-lfs'
ea6fc58524d7 lfs: add server side support for the Batch API Matt Harbison <matt_harbison@yahoo.com> parents: 37147 diff changeset	167 # }
ea6fc58524d7 lfs: add server side support for the Batch API Matt Harbison <matt_harbison@yahoo.com> parents: 37147 diff changeset	168 # }
ea6fc58524d7 lfs: add server side support for the Batch API Matt Harbison <matt_harbison@yahoo.com> parents: 37147 diff changeset	169 # }
ea6fc58524d7 lfs: add server side support for the Batch API Matt Harbison <matt_harbison@yahoo.com> parents: 37147 diff changeset	170 # }]
ea6fc58524d7 lfs: add server side support for the Batch API Matt Harbison <matt_harbison@yahoo.com> parents: 37147 diff changeset	171 # }
ea6fc58524d7 lfs: add server side support for the Batch API Matt Harbison <matt_harbison@yahoo.com> parents: 37147 diff changeset	172
ea6fc58524d7 lfs: add server side support for the Batch API Matt Harbison <matt_harbison@yahoo.com> parents: 37147 diff changeset	173 # TODO: Sort out the expires_at/expires_in/authenticated keys.
ea6fc58524d7 lfs: add server side support for the Batch API Matt Harbison <matt_harbison@yahoo.com> parents: 37147 diff changeset	174
ea6fc58524d7 lfs: add server side support for the Batch API Matt Harbison <matt_harbison@yahoo.com> parents: 37147 diff changeset	175 for obj in objects:
ea6fc58524d7 lfs: add server side support for the Batch API Matt Harbison <matt_harbison@yahoo.com> parents: 37147 diff changeset	176 # Convert unicode to ASCII to create a filesystem path
ea6fc58524d7 lfs: add server side support for the Batch API Matt Harbison <matt_harbison@yahoo.com> parents: 37147 diff changeset	177 oid = obj.get('oid').encode('ascii')
ea6fc58524d7 lfs: add server side support for the Batch API Matt Harbison <matt_harbison@yahoo.com> parents: 37147 diff changeset	178 rsp = {
ea6fc58524d7 lfs: add server side support for the Batch API Matt Harbison <matt_harbison@yahoo.com> parents: 37147 diff changeset	179 'oid': oid,
ea6fc58524d7 lfs: add server side support for the Batch API Matt Harbison <matt_harbison@yahoo.com> parents: 37147 diff changeset	180 'size': obj.get('size'), # XXX: should this check the local size?
ea6fc58524d7 lfs: add server side support for the Batch API Matt Harbison <matt_harbison@yahoo.com> parents: 37147 diff changeset	181 #'authenticated': True,
ea6fc58524d7 lfs: add server side support for the Batch API Matt Harbison <matt_harbison@yahoo.com> parents: 37147 diff changeset	182 }
ea6fc58524d7 lfs: add server side support for the Batch API Matt Harbison <matt_harbison@yahoo.com> parents: 37147 diff changeset	183
ea6fc58524d7 lfs: add server side support for the Batch API Matt Harbison <matt_harbison@yahoo.com> parents: 37147 diff changeset	184 exists = True
ea6fc58524d7 lfs: add server side support for the Batch API Matt Harbison <matt_harbison@yahoo.com> parents: 37147 diff changeset	185 verifies = False
ea6fc58524d7 lfs: add server side support for the Batch API Matt Harbison <matt_harbison@yahoo.com> parents: 37147 diff changeset	186
ea6fc58524d7 lfs: add server side support for the Batch API Matt Harbison <matt_harbison@yahoo.com> parents: 37147 diff changeset	187 # Verify an existing file on the upload request, so that the client is
ea6fc58524d7 lfs: add server side support for the Batch API Matt Harbison <matt_harbison@yahoo.com> parents: 37147 diff changeset	188 # solicited to re-upload if it corrupt locally. Download requests are
ea6fc58524d7 lfs: add server side support for the Batch API Matt Harbison <matt_harbison@yahoo.com> parents: 37147 diff changeset	189 # also verified, so the error can be flagged in the Batch API response.
ea6fc58524d7 lfs: add server side support for the Batch API Matt Harbison <matt_harbison@yahoo.com> parents: 37147 diff changeset	190 # (Maybe we can use this to short circuit the download for `hg verify`,
ea6fc58524d7 lfs: add server side support for the Batch API Matt Harbison <matt_harbison@yahoo.com> parents: 37147 diff changeset	191 # IFF the client can assert that the remote end is an hg server.)
ea6fc58524d7 lfs: add server side support for the Batch API Matt Harbison <matt_harbison@yahoo.com> parents: 37147 diff changeset	192 # Otherwise, it's potentially overkill on download, since it is also
ea6fc58524d7 lfs: add server side support for the Batch API Matt Harbison <matt_harbison@yahoo.com> parents: 37147 diff changeset	193 # verified as the file is streamed to the caller.
ea6fc58524d7 lfs: add server side support for the Batch API Matt Harbison <matt_harbison@yahoo.com> parents: 37147 diff changeset	194 try:
ea6fc58524d7 lfs: add server side support for the Batch API Matt Harbison <matt_harbison@yahoo.com> parents: 37147 diff changeset	195 verifies = store.verify(oid)
ea6fc58524d7 lfs: add server side support for the Batch API Matt Harbison <matt_harbison@yahoo.com> parents: 37147 diff changeset	196 except IOError as inst:
ea6fc58524d7 lfs: add server side support for the Batch API Matt Harbison <matt_harbison@yahoo.com> parents: 37147 diff changeset	197 if inst.errno != errno.ENOENT:
37690 726c4102db9e lfs: log information about Internal Server Errors reported in the Batch API Matt Harbison <matt_harbison@yahoo.com> parents: 37617 diff changeset	198 _logexception(req)
726c4102db9e lfs: log information about Internal Server Errors reported in the Batch API Matt Harbison <matt_harbison@yahoo.com> parents: 37617 diff changeset	199
37148 ea6fc58524d7 lfs: add server side support for the Batch API Matt Harbison <matt_harbison@yahoo.com> parents: 37147 diff changeset	200 rsp['error'] = {
ea6fc58524d7 lfs: add server side support for the Batch API Matt Harbison <matt_harbison@yahoo.com> parents: 37147 diff changeset	201 'code': 500,
ea6fc58524d7 lfs: add server side support for the Batch API Matt Harbison <matt_harbison@yahoo.com> parents: 37147 diff changeset	202 'message': inst.strerror or 'Internal Server Server'
ea6fc58524d7 lfs: add server side support for the Batch API Matt Harbison <matt_harbison@yahoo.com> parents: 37147 diff changeset	203 }
ea6fc58524d7 lfs: add server side support for the Batch API Matt Harbison <matt_harbison@yahoo.com> parents: 37147 diff changeset	204 yield rsp
ea6fc58524d7 lfs: add server side support for the Batch API Matt Harbison <matt_harbison@yahoo.com> parents: 37147 diff changeset	205 continue
ea6fc58524d7 lfs: add server side support for the Batch API Matt Harbison <matt_harbison@yahoo.com> parents: 37147 diff changeset	206
ea6fc58524d7 lfs: add server side support for the Batch API Matt Harbison <matt_harbison@yahoo.com> parents: 37147 diff changeset	207 exists = False
ea6fc58524d7 lfs: add server side support for the Batch API Matt Harbison <matt_harbison@yahoo.com> parents: 37147 diff changeset	208
ea6fc58524d7 lfs: add server side support for the Batch API Matt Harbison <matt_harbison@yahoo.com> parents: 37147 diff changeset	209 # Items are always listed for downloads. They are dropped for uploads
ea6fc58524d7 lfs: add server side support for the Batch API Matt Harbison <matt_harbison@yahoo.com> parents: 37147 diff changeset	210 # IFF they already exist locally.
ea6fc58524d7 lfs: add server side support for the Batch API Matt Harbison <matt_harbison@yahoo.com> parents: 37147 diff changeset	211 if action == 'download':
ea6fc58524d7 lfs: add server side support for the Batch API Matt Harbison <matt_harbison@yahoo.com> parents: 37147 diff changeset	212 if not exists:
ea6fc58524d7 lfs: add server side support for the Batch API Matt Harbison <matt_harbison@yahoo.com> parents: 37147 diff changeset	213 rsp['error'] = {
ea6fc58524d7 lfs: add server side support for the Batch API Matt Harbison <matt_harbison@yahoo.com> parents: 37147 diff changeset	214 'code': 404,
ea6fc58524d7 lfs: add server side support for the Batch API Matt Harbison <matt_harbison@yahoo.com> parents: 37147 diff changeset	215 'message': "The object does not exist"
ea6fc58524d7 lfs: add server side support for the Batch API Matt Harbison <matt_harbison@yahoo.com> parents: 37147 diff changeset	216 }
ea6fc58524d7 lfs: add server side support for the Batch API Matt Harbison <matt_harbison@yahoo.com> parents: 37147 diff changeset	217 yield rsp
ea6fc58524d7 lfs: add server side support for the Batch API Matt Harbison <matt_harbison@yahoo.com> parents: 37147 diff changeset	218 continue
ea6fc58524d7 lfs: add server side support for the Batch API Matt Harbison <matt_harbison@yahoo.com> parents: 37147 diff changeset	219
ea6fc58524d7 lfs: add server side support for the Batch API Matt Harbison <matt_harbison@yahoo.com> parents: 37147 diff changeset	220 elif not verifies:
ea6fc58524d7 lfs: add server side support for the Batch API Matt Harbison <matt_harbison@yahoo.com> parents: 37147 diff changeset	221 rsp['error'] = {
ea6fc58524d7 lfs: add server side support for the Batch API Matt Harbison <matt_harbison@yahoo.com> parents: 37147 diff changeset	222 'code': 422, # XXX: is this the right code?
ea6fc58524d7 lfs: add server side support for the Batch API Matt Harbison <matt_harbison@yahoo.com> parents: 37147 diff changeset	223 'message': "The object is corrupt"
ea6fc58524d7 lfs: add server side support for the Batch API Matt Harbison <matt_harbison@yahoo.com> parents: 37147 diff changeset	224 }
ea6fc58524d7 lfs: add server side support for the Batch API Matt Harbison <matt_harbison@yahoo.com> parents: 37147 diff changeset	225 yield rsp
ea6fc58524d7 lfs: add server side support for the Batch API Matt Harbison <matt_harbison@yahoo.com> parents: 37147 diff changeset	226 continue
ea6fc58524d7 lfs: add server side support for the Batch API Matt Harbison <matt_harbison@yahoo.com> parents: 37147 diff changeset	227
ea6fc58524d7 lfs: add server side support for the Batch API Matt Harbison <matt_harbison@yahoo.com> parents: 37147 diff changeset	228 elif verifies:
ea6fc58524d7 lfs: add server side support for the Batch API Matt Harbison <matt_harbison@yahoo.com> parents: 37147 diff changeset	229 yield rsp # Skip 'actions': already uploaded
ea6fc58524d7 lfs: add server side support for the Batch API Matt Harbison <matt_harbison@yahoo.com> parents: 37147 diff changeset	230 continue
ea6fc58524d7 lfs: add server side support for the Batch API Matt Harbison <matt_harbison@yahoo.com> parents: 37147 diff changeset	231
ea6fc58524d7 lfs: add server side support for the Batch API Matt Harbison <matt_harbison@yahoo.com> parents: 37147 diff changeset	232 expiresat = datetime.datetime.now() + datetime.timedelta(minutes=10)
ea6fc58524d7 lfs: add server side support for the Batch API Matt Harbison <matt_harbison@yahoo.com> parents: 37147 diff changeset	233
ea6fc58524d7 lfs: add server side support for the Batch API Matt Harbison <matt_harbison@yahoo.com> parents: 37147 diff changeset	234 rsp['actions'] = {
ea6fc58524d7 lfs: add server side support for the Batch API Matt Harbison <matt_harbison@yahoo.com> parents: 37147 diff changeset	235 '%s' % action: {
37617 b03f2e0fdb88 lfs: teach the blob server to handle --prefix Matt Harbison <matt_harbison@yahoo.com> parents: 37250 diff changeset	236 'href': '%s%s/.hg/lfs/objects/%s'
b03f2e0fdb88 lfs: teach the blob server to handle --prefix Matt Harbison <matt_harbison@yahoo.com> parents: 37250 diff changeset	237 % (req.baseurl, req.apppath, oid),
37148 ea6fc58524d7 lfs: add server side support for the Batch API Matt Harbison <matt_harbison@yahoo.com> parents: 37147 diff changeset	238 # datetime.isoformat() doesn't include the 'Z' suffix
ea6fc58524d7 lfs: add server side support for the Batch API Matt Harbison <matt_harbison@yahoo.com> parents: 37147 diff changeset	239 "expires_at": expiresat.strftime('%Y-%m-%dT%H:%M:%SZ'),
ea6fc58524d7 lfs: add server side support for the Batch API Matt Harbison <matt_harbison@yahoo.com> parents: 37147 diff changeset	240 'header': {
ea6fc58524d7 lfs: add server side support for the Batch API Matt Harbison <matt_harbison@yahoo.com> parents: 37147 diff changeset	241 # The spec doesn't mention the Accept header here, but avoid
ea6fc58524d7 lfs: add server side support for the Batch API Matt Harbison <matt_harbison@yahoo.com> parents: 37147 diff changeset	242 # a gratuitous deviation from lfs-test-server in the test
ea6fc58524d7 lfs: add server side support for the Batch API Matt Harbison <matt_harbison@yahoo.com> parents: 37147 diff changeset	243 # output.
ea6fc58524d7 lfs: add server side support for the Batch API Matt Harbison <matt_harbison@yahoo.com> parents: 37147 diff changeset	244 'Accept': 'application/vnd.git-lfs'
ea6fc58524d7 lfs: add server side support for the Batch API Matt Harbison <matt_harbison@yahoo.com> parents: 37147 diff changeset	245 }
ea6fc58524d7 lfs: add server side support for the Batch API Matt Harbison <matt_harbison@yahoo.com> parents: 37147 diff changeset	246 }
ea6fc58524d7 lfs: add server side support for the Batch API Matt Harbison <matt_harbison@yahoo.com> parents: 37147 diff changeset	247 }
ea6fc58524d7 lfs: add server side support for the Batch API Matt Harbison <matt_harbison@yahoo.com> parents: 37147 diff changeset	248
ea6fc58524d7 lfs: add server side support for the Batch API Matt Harbison <matt_harbison@yahoo.com> parents: 37147 diff changeset	249 yield rsp
37147 a2566597acb5 lfs: add basic routing for the server side wire protocol processing Matt Harbison <matt_harbison@yahoo.com> parents: diff changeset	250
a2566597acb5 lfs: add basic routing for the server side wire protocol processing Matt Harbison <matt_harbison@yahoo.com> parents: diff changeset	251 def _processbasictransfer(repo, req, res, checkperm):
a2566597acb5 lfs: add basic routing for the server side wire protocol processing Matt Harbison <matt_harbison@yahoo.com> parents: diff changeset	252 """Handle a single file upload (PUT) or download (GET) action for the Basic
a2566597acb5 lfs: add basic routing for the server side wire protocol processing Matt Harbison <matt_harbison@yahoo.com> parents: diff changeset	253 Transfer Adapter.
a2566597acb5 lfs: add basic routing for the server side wire protocol processing Matt Harbison <matt_harbison@yahoo.com> parents: diff changeset	254
a2566597acb5 lfs: add basic routing for the server side wire protocol processing Matt Harbison <matt_harbison@yahoo.com> parents: diff changeset	255 After determining if the request is for an upload or download, the access
a2566597acb5 lfs: add basic routing for the server side wire protocol processing Matt Harbison <matt_harbison@yahoo.com> parents: diff changeset	256 must be checked by calling ``checkperm()`` with either 'pull' or 'upload'
a2566597acb5 lfs: add basic routing for the server side wire protocol processing Matt Harbison <matt_harbison@yahoo.com> parents: diff changeset	257 before accessing the files.
a2566597acb5 lfs: add basic routing for the server side wire protocol processing Matt Harbison <matt_harbison@yahoo.com> parents: diff changeset	258
a2566597acb5 lfs: add basic routing for the server side wire protocol processing Matt Harbison <matt_harbison@yahoo.com> parents: diff changeset	259 https://github.com/git-lfs/git-lfs/blob/master/docs/api/basic-transfers.md
a2566597acb5 lfs: add basic routing for the server side wire protocol processing Matt Harbison <matt_harbison@yahoo.com> parents: diff changeset	260 """
a2566597acb5 lfs: add basic routing for the server side wire protocol processing Matt Harbison <matt_harbison@yahoo.com> parents: diff changeset	261
a2566597acb5 lfs: add basic routing for the server side wire protocol processing Matt Harbison <matt_harbison@yahoo.com> parents: diff changeset	262 method = req.method
37249 fe061e47a2cf lfs: avoid an improper usage of os.path.basename() to parse a URI Matt Harbison <matt_harbison@yahoo.com> parents: 37248 diff changeset	263 oid = req.dispatchparts[-1]
37149 cc0a6ea95d98 lfs: add support for serving blob files Matt Harbison <matt_harbison@yahoo.com> parents: 37148 diff changeset	264 localstore = repo.svfs.lfslocalblobstore
37147 a2566597acb5 lfs: add basic routing for the server side wire protocol processing Matt Harbison <matt_harbison@yahoo.com> parents: diff changeset	265
37250 9640ccf44ac0 lfs: ensure the transfer request is for a known URI Matt Harbison <matt_harbison@yahoo.com> parents: 37249 diff changeset	266 if len(req.dispatchparts) != 4:
9640ccf44ac0 lfs: ensure the transfer request is for a known URI Matt Harbison <matt_harbison@yahoo.com> parents: 37249 diff changeset	267 _sethttperror(res, HTTP_NOT_FOUND)
9640ccf44ac0 lfs: ensure the transfer request is for a known URI Matt Harbison <matt_harbison@yahoo.com> parents: 37249 diff changeset	268 return True
9640ccf44ac0 lfs: ensure the transfer request is for a known URI Matt Harbison <matt_harbison@yahoo.com> parents: 37249 diff changeset	269
37147 a2566597acb5 lfs: add basic routing for the server side wire protocol processing Matt Harbison <matt_harbison@yahoo.com> parents: diff changeset	270 if method == b'PUT':
a2566597acb5 lfs: add basic routing for the server side wire protocol processing Matt Harbison <matt_harbison@yahoo.com> parents: diff changeset	271 checkperm('upload')
37149 cc0a6ea95d98 lfs: add support for serving blob files Matt Harbison <matt_harbison@yahoo.com> parents: 37148 diff changeset	272
cc0a6ea95d98 lfs: add support for serving blob files Matt Harbison <matt_harbison@yahoo.com> parents: 37148 diff changeset	273 # TODO: verify Content-Type?
cc0a6ea95d98 lfs: add support for serving blob files Matt Harbison <matt_harbison@yahoo.com> parents: 37148 diff changeset	274
cc0a6ea95d98 lfs: add support for serving blob files Matt Harbison <matt_harbison@yahoo.com> parents: 37148 diff changeset	275 existed = localstore.has(oid)
cc0a6ea95d98 lfs: add support for serving blob files Matt Harbison <matt_harbison@yahoo.com> parents: 37148 diff changeset	276
cc0a6ea95d98 lfs: add support for serving blob files Matt Harbison <matt_harbison@yahoo.com> parents: 37148 diff changeset	277 # TODO: how to handle timeouts? The body proxy handles limiting to
cc0a6ea95d98 lfs: add support for serving blob files Matt Harbison <matt_harbison@yahoo.com> parents: 37148 diff changeset	278 # Content-Length, but what happens if a client sends less than it
cc0a6ea95d98 lfs: add support for serving blob files Matt Harbison <matt_harbison@yahoo.com> parents: 37148 diff changeset	279 # says it will?
cc0a6ea95d98 lfs: add support for serving blob files Matt Harbison <matt_harbison@yahoo.com> parents: 37148 diff changeset	280
37692 10e5bb9678f4 lfs: gracefully handle aborts on the server when corrupt blobs are detected Matt Harbison <matt_harbison@yahoo.com> parents: 37690 diff changeset	281 statusmessage = hgwebcommon.statusmessage
10e5bb9678f4 lfs: gracefully handle aborts on the server when corrupt blobs are detected Matt Harbison <matt_harbison@yahoo.com> parents: 37690 diff changeset	282 try:
10e5bb9678f4 lfs: gracefully handle aborts on the server when corrupt blobs are detected Matt Harbison <matt_harbison@yahoo.com> parents: 37690 diff changeset	283 localstore.download(oid, req.bodyfh)
10e5bb9678f4 lfs: gracefully handle aborts on the server when corrupt blobs are detected Matt Harbison <matt_harbison@yahoo.com> parents: 37690 diff changeset	284 res.status = statusmessage(HTTP_OK if existed else HTTP_CREATED)
10e5bb9678f4 lfs: gracefully handle aborts on the server when corrupt blobs are detected Matt Harbison <matt_harbison@yahoo.com> parents: 37690 diff changeset	285 except blobstore.LfsCorruptionError:
10e5bb9678f4 lfs: gracefully handle aborts on the server when corrupt blobs are detected Matt Harbison <matt_harbison@yahoo.com> parents: 37690 diff changeset	286 _logexception(req)
37149 cc0a6ea95d98 lfs: add support for serving blob files Matt Harbison <matt_harbison@yahoo.com> parents: 37148 diff changeset	287
37692 10e5bb9678f4 lfs: gracefully handle aborts on the server when corrupt blobs are detected Matt Harbison <matt_harbison@yahoo.com> parents: 37690 diff changeset	288 # XXX: Is this the right code?
10e5bb9678f4 lfs: gracefully handle aborts on the server when corrupt blobs are detected Matt Harbison <matt_harbison@yahoo.com> parents: 37690 diff changeset	289 res.status = statusmessage(422, b'corrupt blob')
37149 cc0a6ea95d98 lfs: add support for serving blob files Matt Harbison <matt_harbison@yahoo.com> parents: 37148 diff changeset	290
cc0a6ea95d98 lfs: add support for serving blob files Matt Harbison <matt_harbison@yahoo.com> parents: 37148 diff changeset	291 # There's no payload here, but this is the header that lfs-test-server
cc0a6ea95d98 lfs: add support for serving blob files Matt Harbison <matt_harbison@yahoo.com> parents: 37148 diff changeset	292 # sends back. This eliminates some gratuitous test output conditionals.
cc0a6ea95d98 lfs: add support for serving blob files Matt Harbison <matt_harbison@yahoo.com> parents: 37148 diff changeset	293 res.headers[b'Content-Type'] = b'text/plain; charset=utf-8'
cc0a6ea95d98 lfs: add support for serving blob files Matt Harbison <matt_harbison@yahoo.com> parents: 37148 diff changeset	294 res.setbodybytes(b'')
cc0a6ea95d98 lfs: add support for serving blob files Matt Harbison <matt_harbison@yahoo.com> parents: 37148 diff changeset	295
cc0a6ea95d98 lfs: add support for serving blob files Matt Harbison <matt_harbison@yahoo.com> parents: 37148 diff changeset	296 return True
37147 a2566597acb5 lfs: add basic routing for the server side wire protocol processing Matt Harbison <matt_harbison@yahoo.com> parents: diff changeset	297 elif method == b'GET':
a2566597acb5 lfs: add basic routing for the server side wire protocol processing Matt Harbison <matt_harbison@yahoo.com> parents: diff changeset	298 checkperm('pull')
a2566597acb5 lfs: add basic routing for the server side wire protocol processing Matt Harbison <matt_harbison@yahoo.com> parents: diff changeset	299
37149 cc0a6ea95d98 lfs: add support for serving blob files Matt Harbison <matt_harbison@yahoo.com> parents: 37148 diff changeset	300 res.status = hgwebcommon.statusmessage(HTTP_OK)
cc0a6ea95d98 lfs: add support for serving blob files Matt Harbison <matt_harbison@yahoo.com> parents: 37148 diff changeset	301 res.headers[b'Content-Type'] = b'application/octet-stream'
cc0a6ea95d98 lfs: add support for serving blob files Matt Harbison <matt_harbison@yahoo.com> parents: 37148 diff changeset	302
37692 10e5bb9678f4 lfs: gracefully handle aborts on the server when corrupt blobs are detected Matt Harbison <matt_harbison@yahoo.com> parents: 37690 diff changeset	303 try:
10e5bb9678f4 lfs: gracefully handle aborts on the server when corrupt blobs are detected Matt Harbison <matt_harbison@yahoo.com> parents: 37690 diff changeset	304 # TODO: figure out how to send back the file in chunks, instead of
10e5bb9678f4 lfs: gracefully handle aborts on the server when corrupt blobs are detected Matt Harbison <matt_harbison@yahoo.com> parents: 37690 diff changeset	305 # reading the whole thing. (Also figure out how to send back
10e5bb9678f4 lfs: gracefully handle aborts on the server when corrupt blobs are detected Matt Harbison <matt_harbison@yahoo.com> parents: 37690 diff changeset	306 # an error status if an IOError occurs after a partial write
10e5bb9678f4 lfs: gracefully handle aborts on the server when corrupt blobs are detected Matt Harbison <matt_harbison@yahoo.com> parents: 37690 diff changeset	307 # in that case. Here, everything is read before starting.)
10e5bb9678f4 lfs: gracefully handle aborts on the server when corrupt blobs are detected Matt Harbison <matt_harbison@yahoo.com> parents: 37690 diff changeset	308 res.setbodybytes(localstore.read(oid))
10e5bb9678f4 lfs: gracefully handle aborts on the server when corrupt blobs are detected Matt Harbison <matt_harbison@yahoo.com> parents: 37690 diff changeset	309 except blobstore.LfsCorruptionError:
10e5bb9678f4 lfs: gracefully handle aborts on the server when corrupt blobs are detected Matt Harbison <matt_harbison@yahoo.com> parents: 37690 diff changeset	310 _logexception(req)
10e5bb9678f4 lfs: gracefully handle aborts on the server when corrupt blobs are detected Matt Harbison <matt_harbison@yahoo.com> parents: 37690 diff changeset	311
10e5bb9678f4 lfs: gracefully handle aborts on the server when corrupt blobs are detected Matt Harbison <matt_harbison@yahoo.com> parents: 37690 diff changeset	312 # XXX: Is this the right code?
10e5bb9678f4 lfs: gracefully handle aborts on the server when corrupt blobs are detected Matt Harbison <matt_harbison@yahoo.com> parents: 37690 diff changeset	313 res.status = hgwebcommon.statusmessage(422, b'corrupt blob')
10e5bb9678f4 lfs: gracefully handle aborts on the server when corrupt blobs are detected Matt Harbison <matt_harbison@yahoo.com> parents: 37690 diff changeset	314 res.setbodybytes(b'')
37149 cc0a6ea95d98 lfs: add support for serving blob files Matt Harbison <matt_harbison@yahoo.com> parents: 37148 diff changeset	315
cc0a6ea95d98 lfs: add support for serving blob files Matt Harbison <matt_harbison@yahoo.com> parents: 37148 diff changeset	316 return True
cc0a6ea95d98 lfs: add support for serving blob files Matt Harbison <matt_harbison@yahoo.com> parents: 37148 diff changeset	317 else:
cc0a6ea95d98 lfs: add support for serving blob files Matt Harbison <matt_harbison@yahoo.com> parents: 37148 diff changeset	318 _sethttperror(res, HTTP_BAD_REQUEST,
cc0a6ea95d98 lfs: add support for serving blob files Matt Harbison <matt_harbison@yahoo.com> parents: 37148 diff changeset	319 message=b'Unsupported LFS transfer method: %s' % method)
cc0a6ea95d98 lfs: add support for serving blob files Matt Harbison <matt_harbison@yahoo.com> parents: 37148 diff changeset	320 return True

Mercurial > public > mercurial-scm > hg

annotate hgext/lfs/wireprotolfsserver.py @ 37692:10e5bb9678f4