mercurial-scm/hg-stable: mercurial/wireprotoframing.py annotate

annotate mercurial/wireprotoframing.py @ 37544:55b5ba8d4e68

wireproto: client reactor support for receiving frames We can now feed received frames into the client reactor and it will validate their sanity, dispatch them appropriately. The hacky HTTP peer has been updated to use the new code. No existing tests changed, somewhat proving the code works as expected. Rudimentary unit tests for the new functionality have been implemented. Differential Revision: https://phab.mercurial-scm.org/D3224

author	Gregory Szorc <gregory.szorc@gmail.com>
date	Mon, 09 Apr 2018 16:54:20 -0700
parents	01361be9e2dc
children	b9502b5f2066

rev	line source
37054 40206e227412 wireproto: define and implement protocol for issuing requests Gregory Szorc <gregory.szorc@gmail.com> parents: diff changeset	1 # wireprotoframing.py - unified framing protocol for wire protocol
40206e227412 wireproto: define and implement protocol for issuing requests Gregory Szorc <gregory.szorc@gmail.com> parents: diff changeset	2 #
40206e227412 wireproto: define and implement protocol for issuing requests Gregory Szorc <gregory.szorc@gmail.com> parents: diff changeset	3 # Copyright 2018 Gregory Szorc <gregory.szorc@gmail.com>
40206e227412 wireproto: define and implement protocol for issuing requests Gregory Szorc <gregory.szorc@gmail.com> parents: diff changeset	4 #
40206e227412 wireproto: define and implement protocol for issuing requests Gregory Szorc <gregory.szorc@gmail.com> parents: diff changeset	5 # This software may be used and distributed according to the terms of the
40206e227412 wireproto: define and implement protocol for issuing requests Gregory Szorc <gregory.szorc@gmail.com> parents: diff changeset	6 # GNU General Public License version 2 or any later version.
40206e227412 wireproto: define and implement protocol for issuing requests Gregory Szorc <gregory.szorc@gmail.com> parents: diff changeset	7
40206e227412 wireproto: define and implement protocol for issuing requests Gregory Szorc <gregory.szorc@gmail.com> parents: diff changeset	8 # This file contains functionality to support the unified frame-based wire
40206e227412 wireproto: define and implement protocol for issuing requests Gregory Szorc <gregory.szorc@gmail.com> parents: diff changeset	9 # protocol. For details about the protocol, see
40206e227412 wireproto: define and implement protocol for issuing requests Gregory Szorc <gregory.szorc@gmail.com> parents: diff changeset	10 # `hg help internals.wireprotocol`.
40206e227412 wireproto: define and implement protocol for issuing requests Gregory Szorc <gregory.szorc@gmail.com> parents: diff changeset	11
40206e227412 wireproto: define and implement protocol for issuing requests Gregory Szorc <gregory.szorc@gmail.com> parents: diff changeset	12 from __future__ import absolute_import
40206e227412 wireproto: define and implement protocol for issuing requests Gregory Szorc <gregory.szorc@gmail.com> parents: diff changeset	13
37543 01361be9e2dc wireproto: introduce a reactor for client-side state Gregory Szorc <gregory.szorc@gmail.com> parents: 37485 diff changeset	14 import collections
37054 40206e227412 wireproto: define and implement protocol for issuing requests Gregory Szorc <gregory.szorc@gmail.com> parents: diff changeset	15 import struct
40206e227412 wireproto: define and implement protocol for issuing requests Gregory Szorc <gregory.szorc@gmail.com> parents: diff changeset	16
37055 8c3c47362934 wireproto: implement basic frame reading and processing Gregory Szorc <gregory.szorc@gmail.com> parents: 37054 diff changeset	17 from .i18n import _
37064 884a0c1604ad wireproto: define attr-based classes for representing frames Gregory Szorc <gregory.szorc@gmail.com> parents: 37063 diff changeset	18 from .thirdparty import (
884a0c1604ad wireproto: define attr-based classes for representing frames Gregory Szorc <gregory.szorc@gmail.com> parents: 37063 diff changeset	19 attr,
37290 cc5a040fe150 wireproto: syntax for encoding CBOR into frames Gregory Szorc <gregory.szorc@gmail.com> parents: 37289 diff changeset	20 cbor,
37064 884a0c1604ad wireproto: define attr-based classes for representing frames Gregory Szorc <gregory.szorc@gmail.com> parents: 37063 diff changeset	21 )
37054 40206e227412 wireproto: define and implement protocol for issuing requests Gregory Szorc <gregory.szorc@gmail.com> parents: diff changeset	22 from . import (
37474 d33997123ea5 py3: system-stringify repr(frame) Yuya Nishihara <yuya@tcha.org> parents: 37473 diff changeset	23 encoding,
37055 8c3c47362934 wireproto: implement basic frame reading and processing Gregory Szorc <gregory.szorc@gmail.com> parents: 37054 diff changeset	24 error,
37054 40206e227412 wireproto: define and implement protocol for issuing requests Gregory Szorc <gregory.szorc@gmail.com> parents: diff changeset	25 util,
40206e227412 wireproto: define and implement protocol for issuing requests Gregory Szorc <gregory.szorc@gmail.com> parents: diff changeset	26 )
37087 f0b6fbea00cf stringutil: bulk-replace call sites to point to new module Yuya Nishihara <yuya@tcha.org> parents: 37066 diff changeset	27 from .utils import (
f0b6fbea00cf stringutil: bulk-replace call sites to point to new module Yuya Nishihara <yuya@tcha.org> parents: 37066 diff changeset	28 stringutil,
f0b6fbea00cf stringutil: bulk-replace call sites to point to new module Yuya Nishihara <yuya@tcha.org> parents: 37066 diff changeset	29 )
37054 40206e227412 wireproto: define and implement protocol for issuing requests Gregory Szorc <gregory.szorc@gmail.com> parents: diff changeset	30
37288 9bfcbe4f4745 wireproto: add streams to frame-based protocol Gregory Szorc <gregory.szorc@gmail.com> parents: 37287 diff changeset	31 FRAME_HEADER_SIZE = 8
37054 40206e227412 wireproto: define and implement protocol for issuing requests Gregory Szorc <gregory.szorc@gmail.com> parents: diff changeset	32 DEFAULT_MAX_FRAME_SIZE = 32768
40206e227412 wireproto: define and implement protocol for issuing requests Gregory Szorc <gregory.szorc@gmail.com> parents: diff changeset	33
37288 9bfcbe4f4745 wireproto: add streams to frame-based protocol Gregory Szorc <gregory.szorc@gmail.com> parents: 37287 diff changeset	34 STREAM_FLAG_BEGIN_STREAM = 0x01
9bfcbe4f4745 wireproto: add streams to frame-based protocol Gregory Szorc <gregory.szorc@gmail.com> parents: 37287 diff changeset	35 STREAM_FLAG_END_STREAM = 0x02
9bfcbe4f4745 wireproto: add streams to frame-based protocol Gregory Szorc <gregory.szorc@gmail.com> parents: 37287 diff changeset	36 STREAM_FLAG_ENCODING_APPLIED = 0x04
9bfcbe4f4745 wireproto: add streams to frame-based protocol Gregory Szorc <gregory.szorc@gmail.com> parents: 37287 diff changeset	37
9bfcbe4f4745 wireproto: add streams to frame-based protocol Gregory Szorc <gregory.szorc@gmail.com> parents: 37287 diff changeset	38 STREAM_FLAGS = {
9bfcbe4f4745 wireproto: add streams to frame-based protocol Gregory Szorc <gregory.szorc@gmail.com> parents: 37287 diff changeset	39 b'stream-begin': STREAM_FLAG_BEGIN_STREAM,
9bfcbe4f4745 wireproto: add streams to frame-based protocol Gregory Szorc <gregory.szorc@gmail.com> parents: 37287 diff changeset	40 b'stream-end': STREAM_FLAG_END_STREAM,
9bfcbe4f4745 wireproto: add streams to frame-based protocol Gregory Szorc <gregory.szorc@gmail.com> parents: 37287 diff changeset	41 b'encoded': STREAM_FLAG_ENCODING_APPLIED,
9bfcbe4f4745 wireproto: add streams to frame-based protocol Gregory Szorc <gregory.szorc@gmail.com> parents: 37287 diff changeset	42 }
9bfcbe4f4745 wireproto: add streams to frame-based protocol Gregory Szorc <gregory.szorc@gmail.com> parents: 37287 diff changeset	43
37292 3d0e2cd86e05 wireproto: use CBOR for command requests Gregory Szorc <gregory.szorc@gmail.com> parents: 37291 diff changeset	44 FRAME_TYPE_COMMAND_REQUEST = 0x01
37054 40206e227412 wireproto: define and implement protocol for issuing requests Gregory Szorc <gregory.szorc@gmail.com> parents: diff changeset	45 FRAME_TYPE_COMMAND_DATA = 0x03
37058 61393f888dfe wireproto: define and implement responses in framing protocol Gregory Szorc <gregory.szorc@gmail.com> parents: 37055 diff changeset	46 FRAME_TYPE_BYTES_RESPONSE = 0x04
61393f888dfe wireproto: define and implement responses in framing protocol Gregory Szorc <gregory.szorc@gmail.com> parents: 37055 diff changeset	47 FRAME_TYPE_ERROR_RESPONSE = 0x05
37063 0a6c5cc09a88 wireproto: define human output side channel frame Gregory Szorc <gregory.szorc@gmail.com> parents: 37061 diff changeset	48 FRAME_TYPE_TEXT_OUTPUT = 0x06
37291 b0041036214e wireproto: define frame to represent progress updates Gregory Szorc <gregory.szorc@gmail.com> parents: 37290 diff changeset	49 FRAME_TYPE_PROGRESS = 0x07
37288 9bfcbe4f4745 wireproto: add streams to frame-based protocol Gregory Szorc <gregory.szorc@gmail.com> parents: 37287 diff changeset	50 FRAME_TYPE_STREAM_SETTINGS = 0x08
37054 40206e227412 wireproto: define and implement protocol for issuing requests Gregory Szorc <gregory.szorc@gmail.com> parents: diff changeset	51
40206e227412 wireproto: define and implement protocol for issuing requests Gregory Szorc <gregory.szorc@gmail.com> parents: diff changeset	52 FRAME_TYPES = {
37292 3d0e2cd86e05 wireproto: use CBOR for command requests Gregory Szorc <gregory.szorc@gmail.com> parents: 37291 diff changeset	53 b'command-request': FRAME_TYPE_COMMAND_REQUEST,
37054 40206e227412 wireproto: define and implement protocol for issuing requests Gregory Szorc <gregory.szorc@gmail.com> parents: diff changeset	54 b'command-data': FRAME_TYPE_COMMAND_DATA,
37058 61393f888dfe wireproto: define and implement responses in framing protocol Gregory Szorc <gregory.szorc@gmail.com> parents: 37055 diff changeset	55 b'bytes-response': FRAME_TYPE_BYTES_RESPONSE,
61393f888dfe wireproto: define and implement responses in framing protocol Gregory Szorc <gregory.szorc@gmail.com> parents: 37055 diff changeset	56 b'error-response': FRAME_TYPE_ERROR_RESPONSE,
37063 0a6c5cc09a88 wireproto: define human output side channel frame Gregory Szorc <gregory.szorc@gmail.com> parents: 37061 diff changeset	57 b'text-output': FRAME_TYPE_TEXT_OUTPUT,
37291 b0041036214e wireproto: define frame to represent progress updates Gregory Szorc <gregory.szorc@gmail.com> parents: 37290 diff changeset	58 b'progress': FRAME_TYPE_PROGRESS,
37288 9bfcbe4f4745 wireproto: add streams to frame-based protocol Gregory Szorc <gregory.szorc@gmail.com> parents: 37287 diff changeset	59 b'stream-settings': FRAME_TYPE_STREAM_SETTINGS,
37054 40206e227412 wireproto: define and implement protocol for issuing requests Gregory Szorc <gregory.szorc@gmail.com> parents: diff changeset	60 }
40206e227412 wireproto: define and implement protocol for issuing requests Gregory Szorc <gregory.szorc@gmail.com> parents: diff changeset	61
37292 3d0e2cd86e05 wireproto: use CBOR for command requests Gregory Szorc <gregory.szorc@gmail.com> parents: 37291 diff changeset	62 FLAG_COMMAND_REQUEST_NEW = 0x01
3d0e2cd86e05 wireproto: use CBOR for command requests Gregory Szorc <gregory.szorc@gmail.com> parents: 37291 diff changeset	63 FLAG_COMMAND_REQUEST_CONTINUATION = 0x02
3d0e2cd86e05 wireproto: use CBOR for command requests Gregory Szorc <gregory.szorc@gmail.com> parents: 37291 diff changeset	64 FLAG_COMMAND_REQUEST_MORE_FRAMES = 0x04
3d0e2cd86e05 wireproto: use CBOR for command requests Gregory Szorc <gregory.szorc@gmail.com> parents: 37291 diff changeset	65 FLAG_COMMAND_REQUEST_EXPECT_DATA = 0x08
37054 40206e227412 wireproto: define and implement protocol for issuing requests Gregory Szorc <gregory.szorc@gmail.com> parents: diff changeset	66
37292 3d0e2cd86e05 wireproto: use CBOR for command requests Gregory Szorc <gregory.szorc@gmail.com> parents: 37291 diff changeset	67 FLAGS_COMMAND_REQUEST = {
3d0e2cd86e05 wireproto: use CBOR for command requests Gregory Szorc <gregory.szorc@gmail.com> parents: 37291 diff changeset	68 b'new': FLAG_COMMAND_REQUEST_NEW,
3d0e2cd86e05 wireproto: use CBOR for command requests Gregory Szorc <gregory.szorc@gmail.com> parents: 37291 diff changeset	69 b'continuation': FLAG_COMMAND_REQUEST_CONTINUATION,
3d0e2cd86e05 wireproto: use CBOR for command requests Gregory Szorc <gregory.szorc@gmail.com> parents: 37291 diff changeset	70 b'more': FLAG_COMMAND_REQUEST_MORE_FRAMES,
3d0e2cd86e05 wireproto: use CBOR for command requests Gregory Szorc <gregory.szorc@gmail.com> parents: 37291 diff changeset	71 b'have-data': FLAG_COMMAND_REQUEST_EXPECT_DATA,
37054 40206e227412 wireproto: define and implement protocol for issuing requests Gregory Szorc <gregory.szorc@gmail.com> parents: diff changeset	72 }
40206e227412 wireproto: define and implement protocol for issuing requests Gregory Szorc <gregory.szorc@gmail.com> parents: diff changeset	73
40206e227412 wireproto: define and implement protocol for issuing requests Gregory Szorc <gregory.szorc@gmail.com> parents: diff changeset	74 FLAG_COMMAND_DATA_CONTINUATION = 0x01
40206e227412 wireproto: define and implement protocol for issuing requests Gregory Szorc <gregory.szorc@gmail.com> parents: diff changeset	75 FLAG_COMMAND_DATA_EOS = 0x02
40206e227412 wireproto: define and implement protocol for issuing requests Gregory Szorc <gregory.szorc@gmail.com> parents: diff changeset	76
40206e227412 wireproto: define and implement protocol for issuing requests Gregory Szorc <gregory.szorc@gmail.com> parents: diff changeset	77 FLAGS_COMMAND_DATA = {
40206e227412 wireproto: define and implement protocol for issuing requests Gregory Szorc <gregory.szorc@gmail.com> parents: diff changeset	78 b'continuation': FLAG_COMMAND_DATA_CONTINUATION,
40206e227412 wireproto: define and implement protocol for issuing requests Gregory Szorc <gregory.szorc@gmail.com> parents: diff changeset	79 b'eos': FLAG_COMMAND_DATA_EOS,
40206e227412 wireproto: define and implement protocol for issuing requests Gregory Szorc <gregory.szorc@gmail.com> parents: diff changeset	80 }
40206e227412 wireproto: define and implement protocol for issuing requests Gregory Szorc <gregory.szorc@gmail.com> parents: diff changeset	81
37058 61393f888dfe wireproto: define and implement responses in framing protocol Gregory Szorc <gregory.szorc@gmail.com> parents: 37055 diff changeset	82 FLAG_BYTES_RESPONSE_CONTINUATION = 0x01
61393f888dfe wireproto: define and implement responses in framing protocol Gregory Szorc <gregory.szorc@gmail.com> parents: 37055 diff changeset	83 FLAG_BYTES_RESPONSE_EOS = 0x02
37299 e9aadee698cf wireproto: add frame flag to denote payloads as CBOR Gregory Szorc <gregory.szorc@gmail.com> parents: 37298 diff changeset	84 FLAG_BYTES_RESPONSE_CBOR = 0x04
37058 61393f888dfe wireproto: define and implement responses in framing protocol Gregory Szorc <gregory.szorc@gmail.com> parents: 37055 diff changeset	85
61393f888dfe wireproto: define and implement responses in framing protocol Gregory Szorc <gregory.szorc@gmail.com> parents: 37055 diff changeset	86 FLAGS_BYTES_RESPONSE = {
61393f888dfe wireproto: define and implement responses in framing protocol Gregory Szorc <gregory.szorc@gmail.com> parents: 37055 diff changeset	87 b'continuation': FLAG_BYTES_RESPONSE_CONTINUATION,
61393f888dfe wireproto: define and implement responses in framing protocol Gregory Szorc <gregory.szorc@gmail.com> parents: 37055 diff changeset	88 b'eos': FLAG_BYTES_RESPONSE_EOS,
37299 e9aadee698cf wireproto: add frame flag to denote payloads as CBOR Gregory Szorc <gregory.szorc@gmail.com> parents: 37298 diff changeset	89 b'cbor': FLAG_BYTES_RESPONSE_CBOR,
37058 61393f888dfe wireproto: define and implement responses in framing protocol Gregory Szorc <gregory.szorc@gmail.com> parents: 37055 diff changeset	90 }
61393f888dfe wireproto: define and implement responses in framing protocol Gregory Szorc <gregory.szorc@gmail.com> parents: 37055 diff changeset	91
61393f888dfe wireproto: define and implement responses in framing protocol Gregory Szorc <gregory.szorc@gmail.com> parents: 37055 diff changeset	92 FLAG_ERROR_RESPONSE_PROTOCOL = 0x01
61393f888dfe wireproto: define and implement responses in framing protocol Gregory Szorc <gregory.szorc@gmail.com> parents: 37055 diff changeset	93 FLAG_ERROR_RESPONSE_APPLICATION = 0x02
61393f888dfe wireproto: define and implement responses in framing protocol Gregory Szorc <gregory.szorc@gmail.com> parents: 37055 diff changeset	94
61393f888dfe wireproto: define and implement responses in framing protocol Gregory Szorc <gregory.szorc@gmail.com> parents: 37055 diff changeset	95 FLAGS_ERROR_RESPONSE = {
61393f888dfe wireproto: define and implement responses in framing protocol Gregory Szorc <gregory.szorc@gmail.com> parents: 37055 diff changeset	96 b'protocol': FLAG_ERROR_RESPONSE_PROTOCOL,
61393f888dfe wireproto: define and implement responses in framing protocol Gregory Szorc <gregory.szorc@gmail.com> parents: 37055 diff changeset	97 b'application': FLAG_ERROR_RESPONSE_APPLICATION,
61393f888dfe wireproto: define and implement responses in framing protocol Gregory Szorc <gregory.szorc@gmail.com> parents: 37055 diff changeset	98 }
61393f888dfe wireproto: define and implement responses in framing protocol Gregory Szorc <gregory.szorc@gmail.com> parents: 37055 diff changeset	99
37054 40206e227412 wireproto: define and implement protocol for issuing requests Gregory Szorc <gregory.szorc@gmail.com> parents: diff changeset	100 # Maps frame types to their available flags.
40206e227412 wireproto: define and implement protocol for issuing requests Gregory Szorc <gregory.szorc@gmail.com> parents: diff changeset	101 FRAME_TYPE_FLAGS = {
37292 3d0e2cd86e05 wireproto: use CBOR for command requests Gregory Szorc <gregory.szorc@gmail.com> parents: 37291 diff changeset	102 FRAME_TYPE_COMMAND_REQUEST: FLAGS_COMMAND_REQUEST,
37054 40206e227412 wireproto: define and implement protocol for issuing requests Gregory Szorc <gregory.szorc@gmail.com> parents: diff changeset	103 FRAME_TYPE_COMMAND_DATA: FLAGS_COMMAND_DATA,
37058 61393f888dfe wireproto: define and implement responses in framing protocol Gregory Szorc <gregory.szorc@gmail.com> parents: 37055 diff changeset	104 FRAME_TYPE_BYTES_RESPONSE: FLAGS_BYTES_RESPONSE,
61393f888dfe wireproto: define and implement responses in framing protocol Gregory Szorc <gregory.szorc@gmail.com> parents: 37055 diff changeset	105 FRAME_TYPE_ERROR_RESPONSE: FLAGS_ERROR_RESPONSE,
37063 0a6c5cc09a88 wireproto: define human output side channel frame Gregory Szorc <gregory.szorc@gmail.com> parents: 37061 diff changeset	106 FRAME_TYPE_TEXT_OUTPUT: {},
37291 b0041036214e wireproto: define frame to represent progress updates Gregory Szorc <gregory.szorc@gmail.com> parents: 37290 diff changeset	107 FRAME_TYPE_PROGRESS: {},
37288 9bfcbe4f4745 wireproto: add streams to frame-based protocol Gregory Szorc <gregory.szorc@gmail.com> parents: 37287 diff changeset	108 FRAME_TYPE_STREAM_SETTINGS: {},
37054 40206e227412 wireproto: define and implement protocol for issuing requests Gregory Szorc <gregory.szorc@gmail.com> parents: diff changeset	109 }
40206e227412 wireproto: define and implement protocol for issuing requests Gregory Szorc <gregory.szorc@gmail.com> parents: diff changeset	110
37292 3d0e2cd86e05 wireproto: use CBOR for command requests Gregory Szorc <gregory.szorc@gmail.com> parents: 37291 diff changeset	111 ARGUMENT_RECORD_HEADER = struct.Struct(r'<HH')
37054 40206e227412 wireproto: define and implement protocol for issuing requests Gregory Szorc <gregory.szorc@gmail.com> parents: diff changeset	112
37298 5ef2da00e935 wireproto: implement custom __repr__ for frame Gregory Szorc <gregory.szorc@gmail.com> parents: 37292 diff changeset	113 def humanflags(mapping, value):
5ef2da00e935 wireproto: implement custom __repr__ for frame Gregory Szorc <gregory.szorc@gmail.com> parents: 37292 diff changeset	114 """Convert a numeric flags value to a human value, using a mapping table."""
37473 7c2c7c749411 wireproto: show unknown id and flags in repr(frame) Yuya Nishihara <yuya@tcha.org> parents: 37472 diff changeset	115 namemap = {v: k for k, v in mapping.iteritems()}
37298 5ef2da00e935 wireproto: implement custom __repr__ for frame Gregory Szorc <gregory.szorc@gmail.com> parents: 37292 diff changeset	116 flags = []
37473 7c2c7c749411 wireproto: show unknown id and flags in repr(frame) Yuya Nishihara <yuya@tcha.org> parents: 37472 diff changeset	117 val = 1
7c2c7c749411 wireproto: show unknown id and flags in repr(frame) Yuya Nishihara <yuya@tcha.org> parents: 37472 diff changeset	118 while value >= val:
37298 5ef2da00e935 wireproto: implement custom __repr__ for frame Gregory Szorc <gregory.szorc@gmail.com> parents: 37292 diff changeset	119 if value & val:
37473 7c2c7c749411 wireproto: show unknown id and flags in repr(frame) Yuya Nishihara <yuya@tcha.org> parents: 37472 diff changeset	120 flags.append(namemap.get(val, '<unknown 0x%02x>' % val))
7c2c7c749411 wireproto: show unknown id and flags in repr(frame) Yuya Nishihara <yuya@tcha.org> parents: 37472 diff changeset	121 val <<= 1
37298 5ef2da00e935 wireproto: implement custom __repr__ for frame Gregory Szorc <gregory.szorc@gmail.com> parents: 37292 diff changeset	122
5ef2da00e935 wireproto: implement custom __repr__ for frame Gregory Szorc <gregory.szorc@gmail.com> parents: 37292 diff changeset	123 return b'\|'.join(flags)
5ef2da00e935 wireproto: implement custom __repr__ for frame Gregory Szorc <gregory.szorc@gmail.com> parents: 37292 diff changeset	124
37064 884a0c1604ad wireproto: define attr-based classes for representing frames Gregory Szorc <gregory.szorc@gmail.com> parents: 37063 diff changeset	125 @attr.s(slots=True)
884a0c1604ad wireproto: define attr-based classes for representing frames Gregory Szorc <gregory.szorc@gmail.com> parents: 37063 diff changeset	126 class frameheader(object):
884a0c1604ad wireproto: define attr-based classes for representing frames Gregory Szorc <gregory.szorc@gmail.com> parents: 37063 diff changeset	127 """Represents the data in a frame header."""
884a0c1604ad wireproto: define attr-based classes for representing frames Gregory Szorc <gregory.szorc@gmail.com> parents: 37063 diff changeset	128
884a0c1604ad wireproto: define attr-based classes for representing frames Gregory Szorc <gregory.szorc@gmail.com> parents: 37063 diff changeset	129 length = attr.ib()
884a0c1604ad wireproto: define attr-based classes for representing frames Gregory Szorc <gregory.szorc@gmail.com> parents: 37063 diff changeset	130 requestid = attr.ib()
37288 9bfcbe4f4745 wireproto: add streams to frame-based protocol Gregory Szorc <gregory.szorc@gmail.com> parents: 37287 diff changeset	131 streamid = attr.ib()
9bfcbe4f4745 wireproto: add streams to frame-based protocol Gregory Szorc <gregory.szorc@gmail.com> parents: 37287 diff changeset	132 streamflags = attr.ib()
37064 884a0c1604ad wireproto: define attr-based classes for representing frames Gregory Szorc <gregory.szorc@gmail.com> parents: 37063 diff changeset	133 typeid = attr.ib()
884a0c1604ad wireproto: define attr-based classes for representing frames Gregory Szorc <gregory.szorc@gmail.com> parents: 37063 diff changeset	134 flags = attr.ib()
884a0c1604ad wireproto: define attr-based classes for representing frames Gregory Szorc <gregory.szorc@gmail.com> parents: 37063 diff changeset	135
37298 5ef2da00e935 wireproto: implement custom __repr__ for frame Gregory Szorc <gregory.szorc@gmail.com> parents: 37292 diff changeset	136 @attr.s(slots=True, repr=False)
37064 884a0c1604ad wireproto: define attr-based classes for representing frames Gregory Szorc <gregory.szorc@gmail.com> parents: 37063 diff changeset	137 class frame(object):
884a0c1604ad wireproto: define attr-based classes for representing frames Gregory Szorc <gregory.szorc@gmail.com> parents: 37063 diff changeset	138 """Represents a parsed frame."""
884a0c1604ad wireproto: define attr-based classes for representing frames Gregory Szorc <gregory.szorc@gmail.com> parents: 37063 diff changeset	139
884a0c1604ad wireproto: define attr-based classes for representing frames Gregory Szorc <gregory.szorc@gmail.com> parents: 37063 diff changeset	140 requestid = attr.ib()
37288 9bfcbe4f4745 wireproto: add streams to frame-based protocol Gregory Szorc <gregory.szorc@gmail.com> parents: 37287 diff changeset	141 streamid = attr.ib()
9bfcbe4f4745 wireproto: add streams to frame-based protocol Gregory Szorc <gregory.szorc@gmail.com> parents: 37287 diff changeset	142 streamflags = attr.ib()
37064 884a0c1604ad wireproto: define attr-based classes for representing frames Gregory Szorc <gregory.szorc@gmail.com> parents: 37063 diff changeset	143 typeid = attr.ib()
884a0c1604ad wireproto: define attr-based classes for representing frames Gregory Szorc <gregory.szorc@gmail.com> parents: 37063 diff changeset	144 flags = attr.ib()
884a0c1604ad wireproto: define attr-based classes for representing frames Gregory Szorc <gregory.szorc@gmail.com> parents: 37063 diff changeset	145 payload = attr.ib()
884a0c1604ad wireproto: define attr-based classes for representing frames Gregory Szorc <gregory.szorc@gmail.com> parents: 37063 diff changeset	146
37474 d33997123ea5 py3: system-stringify repr(frame) Yuya Nishihara <yuya@tcha.org> parents: 37473 diff changeset	147 @encoding.strmethod
37298 5ef2da00e935 wireproto: implement custom __repr__ for frame Gregory Szorc <gregory.szorc@gmail.com> parents: 37292 diff changeset	148 def __repr__(self):
37473 7c2c7c749411 wireproto: show unknown id and flags in repr(frame) Yuya Nishihara <yuya@tcha.org> parents: 37472 diff changeset	149 typename = '<unknown 0x%02x>' % self.typeid
37298 5ef2da00e935 wireproto: implement custom __repr__ for frame Gregory Szorc <gregory.szorc@gmail.com> parents: 37292 diff changeset	150 for name, value in FRAME_TYPES.iteritems():
5ef2da00e935 wireproto: implement custom __repr__ for frame Gregory Szorc <gregory.szorc@gmail.com> parents: 37292 diff changeset	151 if value == self.typeid:
5ef2da00e935 wireproto: implement custom __repr__ for frame Gregory Szorc <gregory.szorc@gmail.com> parents: 37292 diff changeset	152 typename = name
5ef2da00e935 wireproto: implement custom __repr__ for frame Gregory Szorc <gregory.szorc@gmail.com> parents: 37292 diff changeset	153 break
5ef2da00e935 wireproto: implement custom __repr__ for frame Gregory Szorc <gregory.szorc@gmail.com> parents: 37292 diff changeset	154
5ef2da00e935 wireproto: implement custom __repr__ for frame Gregory Szorc <gregory.szorc@gmail.com> parents: 37292 diff changeset	155 return ('frame(size=%d; request=%d; stream=%d; streamflags=%s; '
5ef2da00e935 wireproto: implement custom __repr__ for frame Gregory Szorc <gregory.szorc@gmail.com> parents: 37292 diff changeset	156 'type=%s; flags=%s)' % (
5ef2da00e935 wireproto: implement custom __repr__ for frame Gregory Szorc <gregory.szorc@gmail.com> parents: 37292 diff changeset	157 len(self.payload), self.requestid, self.streamid,
5ef2da00e935 wireproto: implement custom __repr__ for frame Gregory Szorc <gregory.szorc@gmail.com> parents: 37292 diff changeset	158 humanflags(STREAM_FLAGS, self.streamflags), typename,
37472 2f81926c7f89 wireproto: fix repr(frame) to not crash by unknown type id Yuya Nishihara <yuya@tcha.org> parents: 37319 diff changeset	159 humanflags(FRAME_TYPE_FLAGS.get(self.typeid, {}), self.flags)))
37298 5ef2da00e935 wireproto: implement custom __repr__ for frame Gregory Szorc <gregory.szorc@gmail.com> parents: 37292 diff changeset	160
37288 9bfcbe4f4745 wireproto: add streams to frame-based protocol Gregory Szorc <gregory.szorc@gmail.com> parents: 37287 diff changeset	161 def makeframe(requestid, streamid, streamflags, typeid, flags, payload):
37054 40206e227412 wireproto: define and implement protocol for issuing requests Gregory Szorc <gregory.szorc@gmail.com> parents: diff changeset	162 """Assemble a frame into a byte array."""
40206e227412 wireproto: define and implement protocol for issuing requests Gregory Szorc <gregory.szorc@gmail.com> parents: diff changeset	163 # TODO assert size of payload.
40206e227412 wireproto: define and implement protocol for issuing requests Gregory Szorc <gregory.szorc@gmail.com> parents: diff changeset	164 frame = bytearray(FRAME_HEADER_SIZE + len(payload))
40206e227412 wireproto: define and implement protocol for issuing requests Gregory Szorc <gregory.szorc@gmail.com> parents: diff changeset	165
37060 2ec1fb9de638 wireproto: add request IDs to frames Gregory Szorc <gregory.szorc@gmail.com> parents: 37059 diff changeset	166 # 24 bits length
2ec1fb9de638 wireproto: add request IDs to frames Gregory Szorc <gregory.szorc@gmail.com> parents: 37059 diff changeset	167 # 16 bits request id
37288 9bfcbe4f4745 wireproto: add streams to frame-based protocol Gregory Szorc <gregory.szorc@gmail.com> parents: 37287 diff changeset	168 # 8 bits stream id
9bfcbe4f4745 wireproto: add streams to frame-based protocol Gregory Szorc <gregory.szorc@gmail.com> parents: 37287 diff changeset	169 # 8 bits stream flags
37060 2ec1fb9de638 wireproto: add request IDs to frames Gregory Szorc <gregory.szorc@gmail.com> parents: 37059 diff changeset	170 # 4 bits type
2ec1fb9de638 wireproto: add request IDs to frames Gregory Szorc <gregory.szorc@gmail.com> parents: 37059 diff changeset	171 # 4 bits flags
2ec1fb9de638 wireproto: add request IDs to frames Gregory Szorc <gregory.szorc@gmail.com> parents: 37059 diff changeset	172
37054 40206e227412 wireproto: define and implement protocol for issuing requests Gregory Szorc <gregory.szorc@gmail.com> parents: diff changeset	173 l = struct.pack(r'<I', len(payload))
40206e227412 wireproto: define and implement protocol for issuing requests Gregory Szorc <gregory.szorc@gmail.com> parents: diff changeset	174 frame[0:3] = l[0:3]
37288 9bfcbe4f4745 wireproto: add streams to frame-based protocol Gregory Szorc <gregory.szorc@gmail.com> parents: 37287 diff changeset	175 struct.pack_into(r'<HBB', frame, 3, requestid, streamid, streamflags)
9bfcbe4f4745 wireproto: add streams to frame-based protocol Gregory Szorc <gregory.szorc@gmail.com> parents: 37287 diff changeset	176 frame[7] = (typeid << 4) \| flags
9bfcbe4f4745 wireproto: add streams to frame-based protocol Gregory Szorc <gregory.szorc@gmail.com> parents: 37287 diff changeset	177 frame[8:] = payload
37054 40206e227412 wireproto: define and implement protocol for issuing requests Gregory Szorc <gregory.szorc@gmail.com> parents: diff changeset	178
40206e227412 wireproto: define and implement protocol for issuing requests Gregory Szorc <gregory.szorc@gmail.com> parents: diff changeset	179 return frame
40206e227412 wireproto: define and implement protocol for issuing requests Gregory Szorc <gregory.szorc@gmail.com> parents: diff changeset	180
40206e227412 wireproto: define and implement protocol for issuing requests Gregory Szorc <gregory.szorc@gmail.com> parents: diff changeset	181 def makeframefromhumanstring(s):
37060 2ec1fb9de638 wireproto: add request IDs to frames Gregory Szorc <gregory.szorc@gmail.com> parents: 37059 diff changeset	182 """Create a frame from a human readable string
2ec1fb9de638 wireproto: add request IDs to frames Gregory Szorc <gregory.szorc@gmail.com> parents: 37059 diff changeset	183
2ec1fb9de638 wireproto: add request IDs to frames Gregory Szorc <gregory.szorc@gmail.com> parents: 37059 diff changeset	184 Strings have the form:
2ec1fb9de638 wireproto: add request IDs to frames Gregory Szorc <gregory.szorc@gmail.com> parents: 37059 diff changeset	185
37288 9bfcbe4f4745 wireproto: add streams to frame-based protocol Gregory Szorc <gregory.szorc@gmail.com> parents: 37287 diff changeset	186 <request-id> <stream-id> <stream-flags> <type> <flags> <payload>
37054 40206e227412 wireproto: define and implement protocol for issuing requests Gregory Szorc <gregory.szorc@gmail.com> parents: diff changeset	187
40206e227412 wireproto: define and implement protocol for issuing requests Gregory Szorc <gregory.szorc@gmail.com> parents: diff changeset	188 This can be used by user-facing applications and tests for creating
40206e227412 wireproto: define and implement protocol for issuing requests Gregory Szorc <gregory.szorc@gmail.com> parents: diff changeset	189 frames easily without having to type out a bunch of constants.
40206e227412 wireproto: define and implement protocol for issuing requests Gregory Szorc <gregory.szorc@gmail.com> parents: diff changeset	190
37288 9bfcbe4f4745 wireproto: add streams to frame-based protocol Gregory Szorc <gregory.szorc@gmail.com> parents: 37287 diff changeset	191 Request ID and stream IDs are integers.
37060 2ec1fb9de638 wireproto: add request IDs to frames Gregory Szorc <gregory.szorc@gmail.com> parents: 37059 diff changeset	192
37288 9bfcbe4f4745 wireproto: add streams to frame-based protocol Gregory Szorc <gregory.szorc@gmail.com> parents: 37287 diff changeset	193 Stream flags, frame type, and flags can be specified by integer or
9bfcbe4f4745 wireproto: add streams to frame-based protocol Gregory Szorc <gregory.szorc@gmail.com> parents: 37287 diff changeset	194 named constant.
37060 2ec1fb9de638 wireproto: add request IDs to frames Gregory Szorc <gregory.szorc@gmail.com> parents: 37059 diff changeset	195
37054 40206e227412 wireproto: define and implement protocol for issuing requests Gregory Szorc <gregory.szorc@gmail.com> parents: diff changeset	196 Flags can be delimited by `\|` to bitwise OR them together.
37290 cc5a040fe150 wireproto: syntax for encoding CBOR into frames Gregory Szorc <gregory.szorc@gmail.com> parents: 37289 diff changeset	197
cc5a040fe150 wireproto: syntax for encoding CBOR into frames Gregory Szorc <gregory.szorc@gmail.com> parents: 37289 diff changeset	198 If the payload begins with ``cbor:``, the following string will be
37476 e9dea82ea1f3 wireproto: convert python literal to object without using unsafe eval() Yuya Nishihara <yuya@tcha.org> parents: 37474 diff changeset	199 evaluated as Python literal and the resulting object will be fed into
37290 cc5a040fe150 wireproto: syntax for encoding CBOR into frames Gregory Szorc <gregory.szorc@gmail.com> parents: 37289 diff changeset	200 a CBOR encoder. Otherwise, the payload is interpreted as a Python
cc5a040fe150 wireproto: syntax for encoding CBOR into frames Gregory Szorc <gregory.szorc@gmail.com> parents: 37289 diff changeset	201 byte string literal.
37054 40206e227412 wireproto: define and implement protocol for issuing requests Gregory Szorc <gregory.szorc@gmail.com> parents: diff changeset	202 """
37288 9bfcbe4f4745 wireproto: add streams to frame-based protocol Gregory Szorc <gregory.szorc@gmail.com> parents: 37287 diff changeset	203 fields = s.split(b' ', 5)
9bfcbe4f4745 wireproto: add streams to frame-based protocol Gregory Szorc <gregory.szorc@gmail.com> parents: 37287 diff changeset	204 requestid, streamid, streamflags, frametype, frameflags, payload = fields
37060 2ec1fb9de638 wireproto: add request IDs to frames Gregory Szorc <gregory.szorc@gmail.com> parents: 37059 diff changeset	205
2ec1fb9de638 wireproto: add request IDs to frames Gregory Szorc <gregory.szorc@gmail.com> parents: 37059 diff changeset	206 requestid = int(requestid)
37288 9bfcbe4f4745 wireproto: add streams to frame-based protocol Gregory Szorc <gregory.szorc@gmail.com> parents: 37287 diff changeset	207 streamid = int(streamid)
9bfcbe4f4745 wireproto: add streams to frame-based protocol Gregory Szorc <gregory.szorc@gmail.com> parents: 37287 diff changeset	208
9bfcbe4f4745 wireproto: add streams to frame-based protocol Gregory Szorc <gregory.szorc@gmail.com> parents: 37287 diff changeset	209 finalstreamflags = 0
9bfcbe4f4745 wireproto: add streams to frame-based protocol Gregory Szorc <gregory.szorc@gmail.com> parents: 37287 diff changeset	210 for flag in streamflags.split(b'\|'):
9bfcbe4f4745 wireproto: add streams to frame-based protocol Gregory Szorc <gregory.szorc@gmail.com> parents: 37287 diff changeset	211 if flag in STREAM_FLAGS:
9bfcbe4f4745 wireproto: add streams to frame-based protocol Gregory Szorc <gregory.szorc@gmail.com> parents: 37287 diff changeset	212 finalstreamflags \|= STREAM_FLAGS[flag]
9bfcbe4f4745 wireproto: add streams to frame-based protocol Gregory Szorc <gregory.szorc@gmail.com> parents: 37287 diff changeset	213 else:
9bfcbe4f4745 wireproto: add streams to frame-based protocol Gregory Szorc <gregory.szorc@gmail.com> parents: 37287 diff changeset	214 finalstreamflags \|= int(flag)
37054 40206e227412 wireproto: define and implement protocol for issuing requests Gregory Szorc <gregory.szorc@gmail.com> parents: diff changeset	215
40206e227412 wireproto: define and implement protocol for issuing requests Gregory Szorc <gregory.szorc@gmail.com> parents: diff changeset	216 if frametype in FRAME_TYPES:
40206e227412 wireproto: define and implement protocol for issuing requests Gregory Szorc <gregory.szorc@gmail.com> parents: diff changeset	217 frametype = FRAME_TYPES[frametype]
40206e227412 wireproto: define and implement protocol for issuing requests Gregory Szorc <gregory.szorc@gmail.com> parents: diff changeset	218 else:
40206e227412 wireproto: define and implement protocol for issuing requests Gregory Szorc <gregory.szorc@gmail.com> parents: diff changeset	219 frametype = int(frametype)
40206e227412 wireproto: define and implement protocol for issuing requests Gregory Szorc <gregory.szorc@gmail.com> parents: diff changeset	220
40206e227412 wireproto: define and implement protocol for issuing requests Gregory Szorc <gregory.szorc@gmail.com> parents: diff changeset	221 finalflags = 0
40206e227412 wireproto: define and implement protocol for issuing requests Gregory Szorc <gregory.szorc@gmail.com> parents: diff changeset	222 validflags = FRAME_TYPE_FLAGS[frametype]
40206e227412 wireproto: define and implement protocol for issuing requests Gregory Szorc <gregory.szorc@gmail.com> parents: diff changeset	223 for flag in frameflags.split(b'\|'):
40206e227412 wireproto: define and implement protocol for issuing requests Gregory Szorc <gregory.szorc@gmail.com> parents: diff changeset	224 if flag in validflags:
40206e227412 wireproto: define and implement protocol for issuing requests Gregory Szorc <gregory.szorc@gmail.com> parents: diff changeset	225 finalflags \|= validflags[flag]
40206e227412 wireproto: define and implement protocol for issuing requests Gregory Szorc <gregory.szorc@gmail.com> parents: diff changeset	226 else:
40206e227412 wireproto: define and implement protocol for issuing requests Gregory Szorc <gregory.szorc@gmail.com> parents: diff changeset	227 finalflags \|= int(flag)
40206e227412 wireproto: define and implement protocol for issuing requests Gregory Szorc <gregory.szorc@gmail.com> parents: diff changeset	228
37290 cc5a040fe150 wireproto: syntax for encoding CBOR into frames Gregory Szorc <gregory.szorc@gmail.com> parents: 37289 diff changeset	229 if payload.startswith(b'cbor:'):
37476 e9dea82ea1f3 wireproto: convert python literal to object without using unsafe eval() Yuya Nishihara <yuya@tcha.org> parents: 37474 diff changeset	230 payload = cbor.dumps(stringutil.evalpythonliteral(payload[5:]),
e9dea82ea1f3 wireproto: convert python literal to object without using unsafe eval() Yuya Nishihara <yuya@tcha.org> parents: 37474 diff changeset	231 canonical=True)
37290 cc5a040fe150 wireproto: syntax for encoding CBOR into frames Gregory Szorc <gregory.szorc@gmail.com> parents: 37289 diff changeset	232
cc5a040fe150 wireproto: syntax for encoding CBOR into frames Gregory Szorc <gregory.szorc@gmail.com> parents: 37289 diff changeset	233 else:
cc5a040fe150 wireproto: syntax for encoding CBOR into frames Gregory Szorc <gregory.szorc@gmail.com> parents: 37289 diff changeset	234 payload = stringutil.unescapestr(payload)
37054 40206e227412 wireproto: define and implement protocol for issuing requests Gregory Szorc <gregory.szorc@gmail.com> parents: diff changeset	235
37288 9bfcbe4f4745 wireproto: add streams to frame-based protocol Gregory Szorc <gregory.szorc@gmail.com> parents: 37287 diff changeset	236 return makeframe(requestid=requestid, streamid=streamid,
9bfcbe4f4745 wireproto: add streams to frame-based protocol Gregory Szorc <gregory.szorc@gmail.com> parents: 37287 diff changeset	237 streamflags=finalstreamflags, typeid=frametype,
37065 fe4c944f95bb wireproto: use named arguments when passing around frame data Gregory Szorc <gregory.szorc@gmail.com> parents: 37064 diff changeset	238 flags=finalflags, payload=payload)
37054 40206e227412 wireproto: define and implement protocol for issuing requests Gregory Szorc <gregory.szorc@gmail.com> parents: diff changeset	239
37055 8c3c47362934 wireproto: implement basic frame reading and processing Gregory Szorc <gregory.szorc@gmail.com> parents: 37054 diff changeset	240 def parseheader(data):
8c3c47362934 wireproto: implement basic frame reading and processing Gregory Szorc <gregory.szorc@gmail.com> parents: 37054 diff changeset	241 """Parse a unified framing protocol frame header from a buffer.
8c3c47362934 wireproto: implement basic frame reading and processing Gregory Szorc <gregory.szorc@gmail.com> parents: 37054 diff changeset	242
8c3c47362934 wireproto: implement basic frame reading and processing Gregory Szorc <gregory.szorc@gmail.com> parents: 37054 diff changeset	243 The header is expected to be in the buffer at offset 0 and the
8c3c47362934 wireproto: implement basic frame reading and processing Gregory Szorc <gregory.szorc@gmail.com> parents: 37054 diff changeset	244 buffer is expected to be large enough to hold a full header.
8c3c47362934 wireproto: implement basic frame reading and processing Gregory Szorc <gregory.szorc@gmail.com> parents: 37054 diff changeset	245 """
8c3c47362934 wireproto: implement basic frame reading and processing Gregory Szorc <gregory.szorc@gmail.com> parents: 37054 diff changeset	246 # 24 bits payload length (little endian)
37288 9bfcbe4f4745 wireproto: add streams to frame-based protocol Gregory Szorc <gregory.szorc@gmail.com> parents: 37287 diff changeset	247 # 16 bits request ID
9bfcbe4f4745 wireproto: add streams to frame-based protocol Gregory Szorc <gregory.szorc@gmail.com> parents: 37287 diff changeset	248 # 8 bits stream ID
9bfcbe4f4745 wireproto: add streams to frame-based protocol Gregory Szorc <gregory.szorc@gmail.com> parents: 37287 diff changeset	249 # 8 bits stream flags
37055 8c3c47362934 wireproto: implement basic frame reading and processing Gregory Szorc <gregory.szorc@gmail.com> parents: 37054 diff changeset	250 # 4 bits frame type
8c3c47362934 wireproto: implement basic frame reading and processing Gregory Szorc <gregory.szorc@gmail.com> parents: 37054 diff changeset	251 # 4 bits frame flags
8c3c47362934 wireproto: implement basic frame reading and processing Gregory Szorc <gregory.szorc@gmail.com> parents: 37054 diff changeset	252 # ... payload
8c3c47362934 wireproto: implement basic frame reading and processing Gregory Szorc <gregory.szorc@gmail.com> parents: 37054 diff changeset	253 framelength = data[0] + 256 * data[1] + 16384 * data[2]
37288 9bfcbe4f4745 wireproto: add streams to frame-based protocol Gregory Szorc <gregory.szorc@gmail.com> parents: 37287 diff changeset	254 requestid, streamid, streamflags = struct.unpack_from(r'<HBB', data, 3)
9bfcbe4f4745 wireproto: add streams to frame-based protocol Gregory Szorc <gregory.szorc@gmail.com> parents: 37287 diff changeset	255 typeflags = data[7]
37055 8c3c47362934 wireproto: implement basic frame reading and processing Gregory Szorc <gregory.szorc@gmail.com> parents: 37054 diff changeset	256
8c3c47362934 wireproto: implement basic frame reading and processing Gregory Szorc <gregory.szorc@gmail.com> parents: 37054 diff changeset	257 frametype = (typeflags & 0xf0) >> 4
8c3c47362934 wireproto: implement basic frame reading and processing Gregory Szorc <gregory.szorc@gmail.com> parents: 37054 diff changeset	258 frameflags = typeflags & 0x0f
8c3c47362934 wireproto: implement basic frame reading and processing Gregory Szorc <gregory.szorc@gmail.com> parents: 37054 diff changeset	259
37288 9bfcbe4f4745 wireproto: add streams to frame-based protocol Gregory Szorc <gregory.szorc@gmail.com> parents: 37287 diff changeset	260 return frameheader(framelength, requestid, streamid, streamflags,
9bfcbe4f4745 wireproto: add streams to frame-based protocol Gregory Szorc <gregory.szorc@gmail.com> parents: 37287 diff changeset	261 frametype, frameflags)
37055 8c3c47362934 wireproto: implement basic frame reading and processing Gregory Szorc <gregory.szorc@gmail.com> parents: 37054 diff changeset	262
8c3c47362934 wireproto: implement basic frame reading and processing Gregory Szorc <gregory.szorc@gmail.com> parents: 37054 diff changeset	263 def readframe(fh):
8c3c47362934 wireproto: implement basic frame reading and processing Gregory Szorc <gregory.szorc@gmail.com> parents: 37054 diff changeset	264 """Read a unified framing protocol frame from a file object.
8c3c47362934 wireproto: implement basic frame reading and processing Gregory Szorc <gregory.szorc@gmail.com> parents: 37054 diff changeset	265
8c3c47362934 wireproto: implement basic frame reading and processing Gregory Szorc <gregory.szorc@gmail.com> parents: 37054 diff changeset	266 Returns a 3-tuple of (type, flags, payload) for the decoded frame or
8c3c47362934 wireproto: implement basic frame reading and processing Gregory Szorc <gregory.szorc@gmail.com> parents: 37054 diff changeset	267 None if no frame is available. May raise if a malformed frame is
8c3c47362934 wireproto: implement basic frame reading and processing Gregory Szorc <gregory.szorc@gmail.com> parents: 37054 diff changeset	268 seen.
8c3c47362934 wireproto: implement basic frame reading and processing Gregory Szorc <gregory.szorc@gmail.com> parents: 37054 diff changeset	269 """
8c3c47362934 wireproto: implement basic frame reading and processing Gregory Szorc <gregory.szorc@gmail.com> parents: 37054 diff changeset	270 header = bytearray(FRAME_HEADER_SIZE)
8c3c47362934 wireproto: implement basic frame reading and processing Gregory Szorc <gregory.szorc@gmail.com> parents: 37054 diff changeset	271
8c3c47362934 wireproto: implement basic frame reading and processing Gregory Szorc <gregory.szorc@gmail.com> parents: 37054 diff changeset	272 readcount = fh.readinto(header)
8c3c47362934 wireproto: implement basic frame reading and processing Gregory Szorc <gregory.szorc@gmail.com> parents: 37054 diff changeset	273
8c3c47362934 wireproto: implement basic frame reading and processing Gregory Szorc <gregory.szorc@gmail.com> parents: 37054 diff changeset	274 if readcount == 0:
8c3c47362934 wireproto: implement basic frame reading and processing Gregory Szorc <gregory.szorc@gmail.com> parents: 37054 diff changeset	275 return None
8c3c47362934 wireproto: implement basic frame reading and processing Gregory Szorc <gregory.szorc@gmail.com> parents: 37054 diff changeset	276
8c3c47362934 wireproto: implement basic frame reading and processing Gregory Szorc <gregory.szorc@gmail.com> parents: 37054 diff changeset	277 if readcount != FRAME_HEADER_SIZE:
8c3c47362934 wireproto: implement basic frame reading and processing Gregory Szorc <gregory.szorc@gmail.com> parents: 37054 diff changeset	278 raise error.Abort(_('received incomplete frame: got %d bytes: %s') %
8c3c47362934 wireproto: implement basic frame reading and processing Gregory Szorc <gregory.szorc@gmail.com> parents: 37054 diff changeset	279 (readcount, header))
8c3c47362934 wireproto: implement basic frame reading and processing Gregory Szorc <gregory.szorc@gmail.com> parents: 37054 diff changeset	280
37064 884a0c1604ad wireproto: define attr-based classes for representing frames Gregory Szorc <gregory.szorc@gmail.com> parents: 37063 diff changeset	281 h = parseheader(header)
37055 8c3c47362934 wireproto: implement basic frame reading and processing Gregory Szorc <gregory.szorc@gmail.com> parents: 37054 diff changeset	282
37064 884a0c1604ad wireproto: define attr-based classes for representing frames Gregory Szorc <gregory.szorc@gmail.com> parents: 37063 diff changeset	283 payload = fh.read(h.length)
884a0c1604ad wireproto: define attr-based classes for representing frames Gregory Szorc <gregory.szorc@gmail.com> parents: 37063 diff changeset	284 if len(payload) != h.length:
37055 8c3c47362934 wireproto: implement basic frame reading and processing Gregory Szorc <gregory.szorc@gmail.com> parents: 37054 diff changeset	285 raise error.Abort(_('frame length error: expected %d; got %d') %
37064 884a0c1604ad wireproto: define attr-based classes for representing frames Gregory Szorc <gregory.szorc@gmail.com> parents: 37063 diff changeset	286 (h.length, len(payload)))
37055 8c3c47362934 wireproto: implement basic frame reading and processing Gregory Szorc <gregory.szorc@gmail.com> parents: 37054 diff changeset	287
37288 9bfcbe4f4745 wireproto: add streams to frame-based protocol Gregory Szorc <gregory.szorc@gmail.com> parents: 37287 diff changeset	288 return frame(h.requestid, h.streamid, h.streamflags, h.typeid, h.flags,
9bfcbe4f4745 wireproto: add streams to frame-based protocol Gregory Szorc <gregory.szorc@gmail.com> parents: 37287 diff changeset	289 payload)
37055 8c3c47362934 wireproto: implement basic frame reading and processing Gregory Szorc <gregory.szorc@gmail.com> parents: 37054 diff changeset	290
37292 3d0e2cd86e05 wireproto: use CBOR for command requests Gregory Szorc <gregory.szorc@gmail.com> parents: 37291 diff changeset	291 def createcommandframes(stream, requestid, cmd, args, datafh=None,
3d0e2cd86e05 wireproto: use CBOR for command requests Gregory Szorc <gregory.szorc@gmail.com> parents: 37291 diff changeset	292 maxframesize=DEFAULT_MAX_FRAME_SIZE):
37054 40206e227412 wireproto: define and implement protocol for issuing requests Gregory Szorc <gregory.szorc@gmail.com> parents: diff changeset	293 """Create frames necessary to transmit a request to run a command.
40206e227412 wireproto: define and implement protocol for issuing requests Gregory Szorc <gregory.szorc@gmail.com> parents: diff changeset	294
40206e227412 wireproto: define and implement protocol for issuing requests Gregory Szorc <gregory.szorc@gmail.com> parents: diff changeset	295 This is a generator of bytearrays. Each item represents a frame
40206e227412 wireproto: define and implement protocol for issuing requests Gregory Szorc <gregory.szorc@gmail.com> parents: diff changeset	296 ready to be sent over the wire to a peer.
40206e227412 wireproto: define and implement protocol for issuing requests Gregory Szorc <gregory.szorc@gmail.com> parents: diff changeset	297 """
37292 3d0e2cd86e05 wireproto: use CBOR for command requests Gregory Szorc <gregory.szorc@gmail.com> parents: 37291 diff changeset	298 data = {b'name': cmd}
37054 40206e227412 wireproto: define and implement protocol for issuing requests Gregory Szorc <gregory.szorc@gmail.com> parents: diff changeset	299 if args:
37292 3d0e2cd86e05 wireproto: use CBOR for command requests Gregory Szorc <gregory.szorc@gmail.com> parents: 37291 diff changeset	300 data[b'args'] = args
37054 40206e227412 wireproto: define and implement protocol for issuing requests Gregory Szorc <gregory.szorc@gmail.com> parents: diff changeset	301
37292 3d0e2cd86e05 wireproto: use CBOR for command requests Gregory Szorc <gregory.szorc@gmail.com> parents: 37291 diff changeset	302 data = cbor.dumps(data, canonical=True)
37054 40206e227412 wireproto: define and implement protocol for issuing requests Gregory Szorc <gregory.szorc@gmail.com> parents: diff changeset	303
37292 3d0e2cd86e05 wireproto: use CBOR for command requests Gregory Szorc <gregory.szorc@gmail.com> parents: 37291 diff changeset	304 offset = 0
3d0e2cd86e05 wireproto: use CBOR for command requests Gregory Szorc <gregory.szorc@gmail.com> parents: 37291 diff changeset	305
3d0e2cd86e05 wireproto: use CBOR for command requests Gregory Szorc <gregory.szorc@gmail.com> parents: 37291 diff changeset	306 while True:
3d0e2cd86e05 wireproto: use CBOR for command requests Gregory Szorc <gregory.szorc@gmail.com> parents: 37291 diff changeset	307 flags = 0
37054 40206e227412 wireproto: define and implement protocol for issuing requests Gregory Szorc <gregory.szorc@gmail.com> parents: diff changeset	308
37292 3d0e2cd86e05 wireproto: use CBOR for command requests Gregory Szorc <gregory.szorc@gmail.com> parents: 37291 diff changeset	309 # Must set new or continuation flag.
3d0e2cd86e05 wireproto: use CBOR for command requests Gregory Szorc <gregory.szorc@gmail.com> parents: 37291 diff changeset	310 if not offset:
3d0e2cd86e05 wireproto: use CBOR for command requests Gregory Szorc <gregory.szorc@gmail.com> parents: 37291 diff changeset	311 flags \|= FLAG_COMMAND_REQUEST_NEW
3d0e2cd86e05 wireproto: use CBOR for command requests Gregory Szorc <gregory.szorc@gmail.com> parents: 37291 diff changeset	312 else:
3d0e2cd86e05 wireproto: use CBOR for command requests Gregory Szorc <gregory.szorc@gmail.com> parents: 37291 diff changeset	313 flags \|= FLAG_COMMAND_REQUEST_CONTINUATION
37054 40206e227412 wireproto: define and implement protocol for issuing requests Gregory Szorc <gregory.szorc@gmail.com> parents: diff changeset	314
37292 3d0e2cd86e05 wireproto: use CBOR for command requests Gregory Szorc <gregory.szorc@gmail.com> parents: 37291 diff changeset	315 # Data frames is set on all frames.
3d0e2cd86e05 wireproto: use CBOR for command requests Gregory Szorc <gregory.szorc@gmail.com> parents: 37291 diff changeset	316 if datafh:
3d0e2cd86e05 wireproto: use CBOR for command requests Gregory Szorc <gregory.szorc@gmail.com> parents: 37291 diff changeset	317 flags \|= FLAG_COMMAND_REQUEST_EXPECT_DATA
37054 40206e227412 wireproto: define and implement protocol for issuing requests Gregory Szorc <gregory.szorc@gmail.com> parents: diff changeset	318
37292 3d0e2cd86e05 wireproto: use CBOR for command requests Gregory Szorc <gregory.szorc@gmail.com> parents: 37291 diff changeset	319 payload = data[offset:offset + maxframesize]
3d0e2cd86e05 wireproto: use CBOR for command requests Gregory Szorc <gregory.szorc@gmail.com> parents: 37291 diff changeset	320 offset += len(payload)
3d0e2cd86e05 wireproto: use CBOR for command requests Gregory Szorc <gregory.szorc@gmail.com> parents: 37291 diff changeset	321
3d0e2cd86e05 wireproto: use CBOR for command requests Gregory Szorc <gregory.szorc@gmail.com> parents: 37291 diff changeset	322 if len(payload) == maxframesize and offset < len(data):
3d0e2cd86e05 wireproto: use CBOR for command requests Gregory Szorc <gregory.szorc@gmail.com> parents: 37291 diff changeset	323 flags \|= FLAG_COMMAND_REQUEST_MORE_FRAMES
3d0e2cd86e05 wireproto: use CBOR for command requests Gregory Szorc <gregory.szorc@gmail.com> parents: 37291 diff changeset	324
37287 3ed344546d9e wireproto: start to associate frame generation with a stream Gregory Szorc <gregory.szorc@gmail.com> parents: 37087 diff changeset	325 yield stream.makeframe(requestid=requestid,
37292 3d0e2cd86e05 wireproto: use CBOR for command requests Gregory Szorc <gregory.szorc@gmail.com> parents: 37291 diff changeset	326 typeid=FRAME_TYPE_COMMAND_REQUEST,
37287 3ed344546d9e wireproto: start to associate frame generation with a stream Gregory Szorc <gregory.szorc@gmail.com> parents: 37087 diff changeset	327 flags=flags,
3ed344546d9e wireproto: start to associate frame generation with a stream Gregory Szorc <gregory.szorc@gmail.com> parents: 37087 diff changeset	328 payload=payload)
37054 40206e227412 wireproto: define and implement protocol for issuing requests Gregory Szorc <gregory.szorc@gmail.com> parents: diff changeset	329
37292 3d0e2cd86e05 wireproto: use CBOR for command requests Gregory Szorc <gregory.szorc@gmail.com> parents: 37291 diff changeset	330 if not (flags & FLAG_COMMAND_REQUEST_MORE_FRAMES):
3d0e2cd86e05 wireproto: use CBOR for command requests Gregory Szorc <gregory.szorc@gmail.com> parents: 37291 diff changeset	331 break
3d0e2cd86e05 wireproto: use CBOR for command requests Gregory Szorc <gregory.szorc@gmail.com> parents: 37291 diff changeset	332
37054 40206e227412 wireproto: define and implement protocol for issuing requests Gregory Szorc <gregory.szorc@gmail.com> parents: diff changeset	333 if datafh:
40206e227412 wireproto: define and implement protocol for issuing requests Gregory Szorc <gregory.szorc@gmail.com> parents: diff changeset	334 while True:
40206e227412 wireproto: define and implement protocol for issuing requests Gregory Szorc <gregory.szorc@gmail.com> parents: diff changeset	335 data = datafh.read(DEFAULT_MAX_FRAME_SIZE)
40206e227412 wireproto: define and implement protocol for issuing requests Gregory Szorc <gregory.szorc@gmail.com> parents: diff changeset	336
40206e227412 wireproto: define and implement protocol for issuing requests Gregory Szorc <gregory.szorc@gmail.com> parents: diff changeset	337 done = False
40206e227412 wireproto: define and implement protocol for issuing requests Gregory Szorc <gregory.szorc@gmail.com> parents: diff changeset	338 if len(data) == DEFAULT_MAX_FRAME_SIZE:
40206e227412 wireproto: define and implement protocol for issuing requests Gregory Szorc <gregory.szorc@gmail.com> parents: diff changeset	339 flags = FLAG_COMMAND_DATA_CONTINUATION
40206e227412 wireproto: define and implement protocol for issuing requests Gregory Szorc <gregory.szorc@gmail.com> parents: diff changeset	340 else:
40206e227412 wireproto: define and implement protocol for issuing requests Gregory Szorc <gregory.szorc@gmail.com> parents: diff changeset	341 flags = FLAG_COMMAND_DATA_EOS
40206e227412 wireproto: define and implement protocol for issuing requests Gregory Szorc <gregory.szorc@gmail.com> parents: diff changeset	342 assert datafh.read(1) == b''
40206e227412 wireproto: define and implement protocol for issuing requests Gregory Szorc <gregory.szorc@gmail.com> parents: diff changeset	343 done = True
40206e227412 wireproto: define and implement protocol for issuing requests Gregory Szorc <gregory.szorc@gmail.com> parents: diff changeset	344
37287 3ed344546d9e wireproto: start to associate frame generation with a stream Gregory Szorc <gregory.szorc@gmail.com> parents: 37087 diff changeset	345 yield stream.makeframe(requestid=requestid,
3ed344546d9e wireproto: start to associate frame generation with a stream Gregory Szorc <gregory.szorc@gmail.com> parents: 37087 diff changeset	346 typeid=FRAME_TYPE_COMMAND_DATA,
3ed344546d9e wireproto: start to associate frame generation with a stream Gregory Szorc <gregory.szorc@gmail.com> parents: 37087 diff changeset	347 flags=flags,
3ed344546d9e wireproto: start to associate frame generation with a stream Gregory Szorc <gregory.szorc@gmail.com> parents: 37087 diff changeset	348 payload=data)
37054 40206e227412 wireproto: define and implement protocol for issuing requests Gregory Szorc <gregory.szorc@gmail.com> parents: diff changeset	349
40206e227412 wireproto: define and implement protocol for issuing requests Gregory Szorc <gregory.szorc@gmail.com> parents: diff changeset	350 if done:
40206e227412 wireproto: define and implement protocol for issuing requests Gregory Szorc <gregory.szorc@gmail.com> parents: diff changeset	351 break
37055 8c3c47362934 wireproto: implement basic frame reading and processing Gregory Szorc <gregory.szorc@gmail.com> parents: 37054 diff changeset	352
37485 0b7475ea38cf wireproto: port heads command to wire protocol v2 Gregory Szorc <gregory.szorc@gmail.com> parents: 37476 diff changeset	353 def createbytesresponseframesfrombytes(stream, requestid, data, iscbor=False,
37058 61393f888dfe wireproto: define and implement responses in framing protocol Gregory Szorc <gregory.szorc@gmail.com> parents: 37055 diff changeset	354 maxframesize=DEFAULT_MAX_FRAME_SIZE):
61393f888dfe wireproto: define and implement responses in framing protocol Gregory Szorc <gregory.szorc@gmail.com> parents: 37055 diff changeset	355 """Create a raw frame to send a bytes response from static bytes input.
61393f888dfe wireproto: define and implement responses in framing protocol Gregory Szorc <gregory.szorc@gmail.com> parents: 37055 diff changeset	356
61393f888dfe wireproto: define and implement responses in framing protocol Gregory Szorc <gregory.szorc@gmail.com> parents: 37055 diff changeset	357 Returns a generator of bytearrays.
61393f888dfe wireproto: define and implement responses in framing protocol Gregory Szorc <gregory.szorc@gmail.com> parents: 37055 diff changeset	358 """
61393f888dfe wireproto: define and implement responses in framing protocol Gregory Szorc <gregory.szorc@gmail.com> parents: 37055 diff changeset	359
61393f888dfe wireproto: define and implement responses in framing protocol Gregory Szorc <gregory.szorc@gmail.com> parents: 37055 diff changeset	360 # Simple case of a single frame.
61393f888dfe wireproto: define and implement responses in framing protocol Gregory Szorc <gregory.szorc@gmail.com> parents: 37055 diff changeset	361 if len(data) <= maxframesize:
37485 0b7475ea38cf wireproto: port heads command to wire protocol v2 Gregory Szorc <gregory.szorc@gmail.com> parents: 37476 diff changeset	362 flags = FLAG_BYTES_RESPONSE_EOS
0b7475ea38cf wireproto: port heads command to wire protocol v2 Gregory Szorc <gregory.szorc@gmail.com> parents: 37476 diff changeset	363 if iscbor:
0b7475ea38cf wireproto: port heads command to wire protocol v2 Gregory Szorc <gregory.szorc@gmail.com> parents: 37476 diff changeset	364 flags \|= FLAG_BYTES_RESPONSE_CBOR
0b7475ea38cf wireproto: port heads command to wire protocol v2 Gregory Szorc <gregory.szorc@gmail.com> parents: 37476 diff changeset	365
37287 3ed344546d9e wireproto: start to associate frame generation with a stream Gregory Szorc <gregory.szorc@gmail.com> parents: 37087 diff changeset	366 yield stream.makeframe(requestid=requestid,
3ed344546d9e wireproto: start to associate frame generation with a stream Gregory Szorc <gregory.szorc@gmail.com> parents: 37087 diff changeset	367 typeid=FRAME_TYPE_BYTES_RESPONSE,
37485 0b7475ea38cf wireproto: port heads command to wire protocol v2 Gregory Szorc <gregory.szorc@gmail.com> parents: 37476 diff changeset	368 flags=flags,
37287 3ed344546d9e wireproto: start to associate frame generation with a stream Gregory Szorc <gregory.szorc@gmail.com> parents: 37087 diff changeset	369 payload=data)
37058 61393f888dfe wireproto: define and implement responses in framing protocol Gregory Szorc <gregory.szorc@gmail.com> parents: 37055 diff changeset	370 return
61393f888dfe wireproto: define and implement responses in framing protocol Gregory Szorc <gregory.szorc@gmail.com> parents: 37055 diff changeset	371
61393f888dfe wireproto: define and implement responses in framing protocol Gregory Szorc <gregory.szorc@gmail.com> parents: 37055 diff changeset	372 offset = 0
61393f888dfe wireproto: define and implement responses in framing protocol Gregory Szorc <gregory.szorc@gmail.com> parents: 37055 diff changeset	373 while True:
61393f888dfe wireproto: define and implement responses in framing protocol Gregory Szorc <gregory.szorc@gmail.com> parents: 37055 diff changeset	374 chunk = data[offset:offset + maxframesize]
61393f888dfe wireproto: define and implement responses in framing protocol Gregory Szorc <gregory.szorc@gmail.com> parents: 37055 diff changeset	375 offset += len(chunk)
61393f888dfe wireproto: define and implement responses in framing protocol Gregory Szorc <gregory.szorc@gmail.com> parents: 37055 diff changeset	376 done = offset == len(data)
61393f888dfe wireproto: define and implement responses in framing protocol Gregory Szorc <gregory.szorc@gmail.com> parents: 37055 diff changeset	377
61393f888dfe wireproto: define and implement responses in framing protocol Gregory Szorc <gregory.szorc@gmail.com> parents: 37055 diff changeset	378 if done:
61393f888dfe wireproto: define and implement responses in framing protocol Gregory Szorc <gregory.szorc@gmail.com> parents: 37055 diff changeset	379 flags = FLAG_BYTES_RESPONSE_EOS
61393f888dfe wireproto: define and implement responses in framing protocol Gregory Szorc <gregory.szorc@gmail.com> parents: 37055 diff changeset	380 else:
61393f888dfe wireproto: define and implement responses in framing protocol Gregory Szorc <gregory.szorc@gmail.com> parents: 37055 diff changeset	381 flags = FLAG_BYTES_RESPONSE_CONTINUATION
61393f888dfe wireproto: define and implement responses in framing protocol Gregory Szorc <gregory.szorc@gmail.com> parents: 37055 diff changeset	382
37485 0b7475ea38cf wireproto: port heads command to wire protocol v2 Gregory Szorc <gregory.szorc@gmail.com> parents: 37476 diff changeset	383 if iscbor:
0b7475ea38cf wireproto: port heads command to wire protocol v2 Gregory Szorc <gregory.szorc@gmail.com> parents: 37476 diff changeset	384 flags \|= FLAG_BYTES_RESPONSE_CBOR
0b7475ea38cf wireproto: port heads command to wire protocol v2 Gregory Szorc <gregory.szorc@gmail.com> parents: 37476 diff changeset	385
37287 3ed344546d9e wireproto: start to associate frame generation with a stream Gregory Szorc <gregory.szorc@gmail.com> parents: 37087 diff changeset	386 yield stream.makeframe(requestid=requestid,
3ed344546d9e wireproto: start to associate frame generation with a stream Gregory Szorc <gregory.szorc@gmail.com> parents: 37087 diff changeset	387 typeid=FRAME_TYPE_BYTES_RESPONSE,
3ed344546d9e wireproto: start to associate frame generation with a stream Gregory Szorc <gregory.szorc@gmail.com> parents: 37087 diff changeset	388 flags=flags,
3ed344546d9e wireproto: start to associate frame generation with a stream Gregory Szorc <gregory.szorc@gmail.com> parents: 37087 diff changeset	389 payload=chunk)
37058 61393f888dfe wireproto: define and implement responses in framing protocol Gregory Szorc <gregory.szorc@gmail.com> parents: 37055 diff changeset	390
61393f888dfe wireproto: define and implement responses in framing protocol Gregory Szorc <gregory.szorc@gmail.com> parents: 37055 diff changeset	391 if done:
61393f888dfe wireproto: define and implement responses in framing protocol Gregory Szorc <gregory.szorc@gmail.com> parents: 37055 diff changeset	392 break
61393f888dfe wireproto: define and implement responses in framing protocol Gregory Szorc <gregory.szorc@gmail.com> parents: 37055 diff changeset	393
37287 3ed344546d9e wireproto: start to associate frame generation with a stream Gregory Szorc <gregory.szorc@gmail.com> parents: 37087 diff changeset	394 def createerrorframe(stream, requestid, msg, protocol=False, application=False):
37058 61393f888dfe wireproto: define and implement responses in framing protocol Gregory Szorc <gregory.szorc@gmail.com> parents: 37055 diff changeset	395 # TODO properly handle frame size limits.
61393f888dfe wireproto: define and implement responses in framing protocol Gregory Szorc <gregory.szorc@gmail.com> parents: 37055 diff changeset	396 assert len(msg) <= DEFAULT_MAX_FRAME_SIZE
61393f888dfe wireproto: define and implement responses in framing protocol Gregory Szorc <gregory.szorc@gmail.com> parents: 37055 diff changeset	397
61393f888dfe wireproto: define and implement responses in framing protocol Gregory Szorc <gregory.szorc@gmail.com> parents: 37055 diff changeset	398 flags = 0
61393f888dfe wireproto: define and implement responses in framing protocol Gregory Szorc <gregory.szorc@gmail.com> parents: 37055 diff changeset	399 if protocol:
61393f888dfe wireproto: define and implement responses in framing protocol Gregory Szorc <gregory.szorc@gmail.com> parents: 37055 diff changeset	400 flags \|= FLAG_ERROR_RESPONSE_PROTOCOL
61393f888dfe wireproto: define and implement responses in framing protocol Gregory Szorc <gregory.szorc@gmail.com> parents: 37055 diff changeset	401 if application:
61393f888dfe wireproto: define and implement responses in framing protocol Gregory Szorc <gregory.szorc@gmail.com> parents: 37055 diff changeset	402 flags \|= FLAG_ERROR_RESPONSE_APPLICATION
61393f888dfe wireproto: define and implement responses in framing protocol Gregory Szorc <gregory.szorc@gmail.com> parents: 37055 diff changeset	403
37287 3ed344546d9e wireproto: start to associate frame generation with a stream Gregory Szorc <gregory.szorc@gmail.com> parents: 37087 diff changeset	404 yield stream.makeframe(requestid=requestid,
3ed344546d9e wireproto: start to associate frame generation with a stream Gregory Szorc <gregory.szorc@gmail.com> parents: 37087 diff changeset	405 typeid=FRAME_TYPE_ERROR_RESPONSE,
3ed344546d9e wireproto: start to associate frame generation with a stream Gregory Szorc <gregory.szorc@gmail.com> parents: 37087 diff changeset	406 flags=flags,
3ed344546d9e wireproto: start to associate frame generation with a stream Gregory Szorc <gregory.szorc@gmail.com> parents: 37087 diff changeset	407 payload=msg)
37058 61393f888dfe wireproto: define and implement responses in framing protocol Gregory Szorc <gregory.szorc@gmail.com> parents: 37055 diff changeset	408
37319 36d17f37db91 wireproto: convert human output frames to CBOR Gregory Szorc <gregory.szorc@gmail.com> parents: 37299 diff changeset	409 def createtextoutputframe(stream, requestid, atoms,
36d17f37db91 wireproto: convert human output frames to CBOR Gregory Szorc <gregory.szorc@gmail.com> parents: 37299 diff changeset	410 maxframesize=DEFAULT_MAX_FRAME_SIZE):
37063 0a6c5cc09a88 wireproto: define human output side channel frame Gregory Szorc <gregory.szorc@gmail.com> parents: 37061 diff changeset	411 """Create a text output frame to render text to people.
0a6c5cc09a88 wireproto: define human output side channel frame Gregory Szorc <gregory.szorc@gmail.com> parents: 37061 diff changeset	412
0a6c5cc09a88 wireproto: define human output side channel frame Gregory Szorc <gregory.szorc@gmail.com> parents: 37061 diff changeset	413 ``atoms`` is a 3-tuple of (formatting string, args, labels).
0a6c5cc09a88 wireproto: define human output side channel frame Gregory Szorc <gregory.szorc@gmail.com> parents: 37061 diff changeset	414
0a6c5cc09a88 wireproto: define human output side channel frame Gregory Szorc <gregory.szorc@gmail.com> parents: 37061 diff changeset	415 The formatting string contains ``%s`` tokens to be replaced by the
0a6c5cc09a88 wireproto: define human output side channel frame Gregory Szorc <gregory.szorc@gmail.com> parents: 37061 diff changeset	416 corresponding indexed entry in ``args``. ``labels`` is an iterable of
0a6c5cc09a88 wireproto: define human output side channel frame Gregory Szorc <gregory.szorc@gmail.com> parents: 37061 diff changeset	417 formatters to be applied at rendering time. In terms of the ``ui``
0a6c5cc09a88 wireproto: define human output side channel frame Gregory Szorc <gregory.szorc@gmail.com> parents: 37061 diff changeset	418 class, each atom corresponds to a ``ui.write()``.
0a6c5cc09a88 wireproto: define human output side channel frame Gregory Szorc <gregory.szorc@gmail.com> parents: 37061 diff changeset	419 """
37319 36d17f37db91 wireproto: convert human output frames to CBOR Gregory Szorc <gregory.szorc@gmail.com> parents: 37299 diff changeset	420 atomdicts = []
37063 0a6c5cc09a88 wireproto: define human output side channel frame Gregory Szorc <gregory.szorc@gmail.com> parents: 37061 diff changeset	421
0a6c5cc09a88 wireproto: define human output side channel frame Gregory Szorc <gregory.szorc@gmail.com> parents: 37061 diff changeset	422 for (formatting, args, labels) in atoms:
0a6c5cc09a88 wireproto: define human output side channel frame Gregory Szorc <gregory.szorc@gmail.com> parents: 37061 diff changeset	423 # TODO look for localstr, other types here?
0a6c5cc09a88 wireproto: define human output side channel frame Gregory Szorc <gregory.szorc@gmail.com> parents: 37061 diff changeset	424
0a6c5cc09a88 wireproto: define human output side channel frame Gregory Szorc <gregory.szorc@gmail.com> parents: 37061 diff changeset	425 if not isinstance(formatting, bytes):
0a6c5cc09a88 wireproto: define human output side channel frame Gregory Szorc <gregory.szorc@gmail.com> parents: 37061 diff changeset	426 raise ValueError('must use bytes formatting strings')
0a6c5cc09a88 wireproto: define human output side channel frame Gregory Szorc <gregory.szorc@gmail.com> parents: 37061 diff changeset	427 for arg in args:
0a6c5cc09a88 wireproto: define human output side channel frame Gregory Szorc <gregory.szorc@gmail.com> parents: 37061 diff changeset	428 if not isinstance(arg, bytes):
0a6c5cc09a88 wireproto: define human output side channel frame Gregory Szorc <gregory.szorc@gmail.com> parents: 37061 diff changeset	429 raise ValueError('must use bytes for arguments')
0a6c5cc09a88 wireproto: define human output side channel frame Gregory Szorc <gregory.szorc@gmail.com> parents: 37061 diff changeset	430 for label in labels:
0a6c5cc09a88 wireproto: define human output side channel frame Gregory Szorc <gregory.szorc@gmail.com> parents: 37061 diff changeset	431 if not isinstance(label, bytes):
0a6c5cc09a88 wireproto: define human output side channel frame Gregory Szorc <gregory.szorc@gmail.com> parents: 37061 diff changeset	432 raise ValueError('must use bytes for labels')
0a6c5cc09a88 wireproto: define human output side channel frame Gregory Szorc <gregory.szorc@gmail.com> parents: 37061 diff changeset	433
37319 36d17f37db91 wireproto: convert human output frames to CBOR Gregory Szorc <gregory.szorc@gmail.com> parents: 37299 diff changeset	434 # Formatting string must be ASCII.
36d17f37db91 wireproto: convert human output frames to CBOR Gregory Szorc <gregory.szorc@gmail.com> parents: 37299 diff changeset	435 formatting = formatting.decode(r'ascii', r'replace').encode(r'ascii')
37063 0a6c5cc09a88 wireproto: define human output side channel frame Gregory Szorc <gregory.szorc@gmail.com> parents: 37061 diff changeset	436
0a6c5cc09a88 wireproto: define human output side channel frame Gregory Szorc <gregory.szorc@gmail.com> parents: 37061 diff changeset	437 # Arguments must be UTF-8.
0a6c5cc09a88 wireproto: define human output side channel frame Gregory Szorc <gregory.szorc@gmail.com> parents: 37061 diff changeset	438 args = [a.decode(r'utf-8', r'replace').encode(r'utf-8') for a in args]
0a6c5cc09a88 wireproto: define human output side channel frame Gregory Szorc <gregory.szorc@gmail.com> parents: 37061 diff changeset	439
0a6c5cc09a88 wireproto: define human output side channel frame Gregory Szorc <gregory.szorc@gmail.com> parents: 37061 diff changeset	440 # Labels must be ASCII.
0a6c5cc09a88 wireproto: define human output side channel frame Gregory Szorc <gregory.szorc@gmail.com> parents: 37061 diff changeset	441 labels = [l.decode(r'ascii', r'strict').encode(r'ascii')
0a6c5cc09a88 wireproto: define human output side channel frame Gregory Szorc <gregory.szorc@gmail.com> parents: 37061 diff changeset	442 for l in labels]
0a6c5cc09a88 wireproto: define human output side channel frame Gregory Szorc <gregory.szorc@gmail.com> parents: 37061 diff changeset	443
37319 36d17f37db91 wireproto: convert human output frames to CBOR Gregory Szorc <gregory.szorc@gmail.com> parents: 37299 diff changeset	444 atom = {b'msg': formatting}
36d17f37db91 wireproto: convert human output frames to CBOR Gregory Szorc <gregory.szorc@gmail.com> parents: 37299 diff changeset	445 if args:
36d17f37db91 wireproto: convert human output frames to CBOR Gregory Szorc <gregory.szorc@gmail.com> parents: 37299 diff changeset	446 atom[b'args'] = args
36d17f37db91 wireproto: convert human output frames to CBOR Gregory Szorc <gregory.szorc@gmail.com> parents: 37299 diff changeset	447 if labels:
36d17f37db91 wireproto: convert human output frames to CBOR Gregory Szorc <gregory.szorc@gmail.com> parents: 37299 diff changeset	448 atom[b'labels'] = labels
37063 0a6c5cc09a88 wireproto: define human output side channel frame Gregory Szorc <gregory.szorc@gmail.com> parents: 37061 diff changeset	449
37319 36d17f37db91 wireproto: convert human output frames to CBOR Gregory Szorc <gregory.szorc@gmail.com> parents: 37299 diff changeset	450 atomdicts.append(atom)
37063 0a6c5cc09a88 wireproto: define human output side channel frame Gregory Szorc <gregory.szorc@gmail.com> parents: 37061 diff changeset	451
37319 36d17f37db91 wireproto: convert human output frames to CBOR Gregory Szorc <gregory.szorc@gmail.com> parents: 37299 diff changeset	452 payload = cbor.dumps(atomdicts, canonical=True)
37063 0a6c5cc09a88 wireproto: define human output side channel frame Gregory Szorc <gregory.szorc@gmail.com> parents: 37061 diff changeset	453
37319 36d17f37db91 wireproto: convert human output frames to CBOR Gregory Szorc <gregory.szorc@gmail.com> parents: 37299 diff changeset	454 if len(payload) > maxframesize:
37063 0a6c5cc09a88 wireproto: define human output side channel frame Gregory Szorc <gregory.szorc@gmail.com> parents: 37061 diff changeset	455 raise ValueError('cannot encode data in a single frame')
0a6c5cc09a88 wireproto: define human output side channel frame Gregory Szorc <gregory.szorc@gmail.com> parents: 37061 diff changeset	456
37287 3ed344546d9e wireproto: start to associate frame generation with a stream Gregory Szorc <gregory.szorc@gmail.com> parents: 37087 diff changeset	457 yield stream.makeframe(requestid=requestid,
3ed344546d9e wireproto: start to associate frame generation with a stream Gregory Szorc <gregory.szorc@gmail.com> parents: 37087 diff changeset	458 typeid=FRAME_TYPE_TEXT_OUTPUT,
3ed344546d9e wireproto: start to associate frame generation with a stream Gregory Szorc <gregory.szorc@gmail.com> parents: 37087 diff changeset	459 flags=0,
37319 36d17f37db91 wireproto: convert human output frames to CBOR Gregory Szorc <gregory.szorc@gmail.com> parents: 37299 diff changeset	460 payload=payload)
37287 3ed344546d9e wireproto: start to associate frame generation with a stream Gregory Szorc <gregory.szorc@gmail.com> parents: 37087 diff changeset	461
3ed344546d9e wireproto: start to associate frame generation with a stream Gregory Szorc <gregory.szorc@gmail.com> parents: 37087 diff changeset	462 class stream(object):
3ed344546d9e wireproto: start to associate frame generation with a stream Gregory Szorc <gregory.szorc@gmail.com> parents: 37087 diff changeset	463 """Represents a logical unidirectional series of frames."""
3ed344546d9e wireproto: start to associate frame generation with a stream Gregory Szorc <gregory.szorc@gmail.com> parents: 37087 diff changeset	464
37288 9bfcbe4f4745 wireproto: add streams to frame-based protocol Gregory Szorc <gregory.szorc@gmail.com> parents: 37287 diff changeset	465 def __init__(self, streamid, active=False):
9bfcbe4f4745 wireproto: add streams to frame-based protocol Gregory Szorc <gregory.szorc@gmail.com> parents: 37287 diff changeset	466 self.streamid = streamid
9bfcbe4f4745 wireproto: add streams to frame-based protocol Gregory Szorc <gregory.szorc@gmail.com> parents: 37287 diff changeset	467 self._active = False
9bfcbe4f4745 wireproto: add streams to frame-based protocol Gregory Szorc <gregory.szorc@gmail.com> parents: 37287 diff changeset	468
37287 3ed344546d9e wireproto: start to associate frame generation with a stream Gregory Szorc <gregory.szorc@gmail.com> parents: 37087 diff changeset	469 def makeframe(self, requestid, typeid, flags, payload):
3ed344546d9e wireproto: start to associate frame generation with a stream Gregory Szorc <gregory.szorc@gmail.com> parents: 37087 diff changeset	470 """Create a frame to be sent out over this stream.
3ed344546d9e wireproto: start to associate frame generation with a stream Gregory Szorc <gregory.szorc@gmail.com> parents: 37087 diff changeset	471
3ed344546d9e wireproto: start to associate frame generation with a stream Gregory Szorc <gregory.szorc@gmail.com> parents: 37087 diff changeset	472 Only returns the frame instance. Does not actually send it.
3ed344546d9e wireproto: start to associate frame generation with a stream Gregory Szorc <gregory.szorc@gmail.com> parents: 37087 diff changeset	473 """
37288 9bfcbe4f4745 wireproto: add streams to frame-based protocol Gregory Szorc <gregory.szorc@gmail.com> parents: 37287 diff changeset	474 streamflags = 0
9bfcbe4f4745 wireproto: add streams to frame-based protocol Gregory Szorc <gregory.szorc@gmail.com> parents: 37287 diff changeset	475 if not self._active:
9bfcbe4f4745 wireproto: add streams to frame-based protocol Gregory Szorc <gregory.szorc@gmail.com> parents: 37287 diff changeset	476 streamflags \|= STREAM_FLAG_BEGIN_STREAM
9bfcbe4f4745 wireproto: add streams to frame-based protocol Gregory Szorc <gregory.szorc@gmail.com> parents: 37287 diff changeset	477 self._active = True
9bfcbe4f4745 wireproto: add streams to frame-based protocol Gregory Szorc <gregory.szorc@gmail.com> parents: 37287 diff changeset	478
9bfcbe4f4745 wireproto: add streams to frame-based protocol Gregory Szorc <gregory.szorc@gmail.com> parents: 37287 diff changeset	479 return makeframe(requestid, self.streamid, streamflags, typeid, flags,
9bfcbe4f4745 wireproto: add streams to frame-based protocol Gregory Szorc <gregory.szorc@gmail.com> parents: 37287 diff changeset	480 payload)
9bfcbe4f4745 wireproto: add streams to frame-based protocol Gregory Szorc <gregory.szorc@gmail.com> parents: 37287 diff changeset	481
9bfcbe4f4745 wireproto: add streams to frame-based protocol Gregory Szorc <gregory.szorc@gmail.com> parents: 37287 diff changeset	482 def ensureserverstream(stream):
9bfcbe4f4745 wireproto: add streams to frame-based protocol Gregory Szorc <gregory.szorc@gmail.com> parents: 37287 diff changeset	483 if stream.streamid % 2:
9bfcbe4f4745 wireproto: add streams to frame-based protocol Gregory Szorc <gregory.szorc@gmail.com> parents: 37287 diff changeset	484 raise error.ProgrammingError('server should only write to even '
9bfcbe4f4745 wireproto: add streams to frame-based protocol Gregory Szorc <gregory.szorc@gmail.com> parents: 37287 diff changeset	485 'numbered streams; %d is not even' %
9bfcbe4f4745 wireproto: add streams to frame-based protocol Gregory Szorc <gregory.szorc@gmail.com> parents: 37287 diff changeset	486 stream.streamid)
37063 0a6c5cc09a88 wireproto: define human output side channel frame Gregory Szorc <gregory.szorc@gmail.com> parents: 37061 diff changeset	487
37055 8c3c47362934 wireproto: implement basic frame reading and processing Gregory Szorc <gregory.szorc@gmail.com> parents: 37054 diff changeset	488 class serverreactor(object):
8c3c47362934 wireproto: implement basic frame reading and processing Gregory Szorc <gregory.szorc@gmail.com> parents: 37054 diff changeset	489 """Holds state of a server handling frame-based protocol requests.
8c3c47362934 wireproto: implement basic frame reading and processing Gregory Szorc <gregory.szorc@gmail.com> parents: 37054 diff changeset	490
8c3c47362934 wireproto: implement basic frame reading and processing Gregory Szorc <gregory.szorc@gmail.com> parents: 37054 diff changeset	491 This class is the "brain" of the unified frame-based protocol server
8c3c47362934 wireproto: implement basic frame reading and processing Gregory Szorc <gregory.szorc@gmail.com> parents: 37054 diff changeset	492 component. While the protocol is stateless from the perspective of
8c3c47362934 wireproto: implement basic frame reading and processing Gregory Szorc <gregory.szorc@gmail.com> parents: 37054 diff changeset	493 requests/commands, something needs to track which frames have been
8c3c47362934 wireproto: implement basic frame reading and processing Gregory Szorc <gregory.szorc@gmail.com> parents: 37054 diff changeset	494 received, what frames to expect, etc. This class is that thing.
8c3c47362934 wireproto: implement basic frame reading and processing Gregory Szorc <gregory.szorc@gmail.com> parents: 37054 diff changeset	495
8c3c47362934 wireproto: implement basic frame reading and processing Gregory Szorc <gregory.szorc@gmail.com> parents: 37054 diff changeset	496 Instances are modeled as a state machine of sorts. Instances are also
8c3c47362934 wireproto: implement basic frame reading and processing Gregory Szorc <gregory.szorc@gmail.com> parents: 37054 diff changeset	497 reactionary to external events. The point of this class is to encapsulate
8c3c47362934 wireproto: implement basic frame reading and processing Gregory Szorc <gregory.szorc@gmail.com> parents: 37054 diff changeset	498 the state of the connection and the exchange of frames, not to perform
8c3c47362934 wireproto: implement basic frame reading and processing Gregory Szorc <gregory.szorc@gmail.com> parents: 37054 diff changeset	499 work. Instead, callers tell this class when something occurs, like a
8c3c47362934 wireproto: implement basic frame reading and processing Gregory Szorc <gregory.szorc@gmail.com> parents: 37054 diff changeset	500 frame arriving. If that activity is worthy of a follow-up action (say
8c3c47362934 wireproto: implement basic frame reading and processing Gregory Szorc <gregory.szorc@gmail.com> parents: 37054 diff changeset	501 run a command), the return value of that handler will say so.
8c3c47362934 wireproto: implement basic frame reading and processing Gregory Szorc <gregory.szorc@gmail.com> parents: 37054 diff changeset	502
8c3c47362934 wireproto: implement basic frame reading and processing Gregory Szorc <gregory.szorc@gmail.com> parents: 37054 diff changeset	503 I/O and CPU intensive operations are purposefully delegated outside of
8c3c47362934 wireproto: implement basic frame reading and processing Gregory Szorc <gregory.szorc@gmail.com> parents: 37054 diff changeset	504 this class.
8c3c47362934 wireproto: implement basic frame reading and processing Gregory Szorc <gregory.szorc@gmail.com> parents: 37054 diff changeset	505
8c3c47362934 wireproto: implement basic frame reading and processing Gregory Szorc <gregory.szorc@gmail.com> parents: 37054 diff changeset	506 Consumers are expected to tell instances when events occur. They do so by
8c3c47362934 wireproto: implement basic frame reading and processing Gregory Szorc <gregory.szorc@gmail.com> parents: 37054 diff changeset	507 calling the various ``on*`` methods. These methods return a 2-tuple
8c3c47362934 wireproto: implement basic frame reading and processing Gregory Szorc <gregory.szorc@gmail.com> parents: 37054 diff changeset	508 describing any follow-up action(s) to take. The first element is the
8c3c47362934 wireproto: implement basic frame reading and processing Gregory Szorc <gregory.szorc@gmail.com> parents: 37054 diff changeset	509 name of an action to perform. The second is a data structure (usually
8c3c47362934 wireproto: implement basic frame reading and processing Gregory Szorc <gregory.szorc@gmail.com> parents: 37054 diff changeset	510 a dict) specific to that action that contains more information. e.g.
8c3c47362934 wireproto: implement basic frame reading and processing Gregory Szorc <gregory.szorc@gmail.com> parents: 37054 diff changeset	511 if the server wants to send frames back to the client, the data structure
8c3c47362934 wireproto: implement basic frame reading and processing Gregory Szorc <gregory.szorc@gmail.com> parents: 37054 diff changeset	512 will contain a reference to those frames.
8c3c47362934 wireproto: implement basic frame reading and processing Gregory Szorc <gregory.szorc@gmail.com> parents: 37054 diff changeset	513
8c3c47362934 wireproto: implement basic frame reading and processing Gregory Szorc <gregory.szorc@gmail.com> parents: 37054 diff changeset	514 Valid actions that consumers can be instructed to take are:
8c3c47362934 wireproto: implement basic frame reading and processing Gregory Szorc <gregory.szorc@gmail.com> parents: 37054 diff changeset	515
37058 61393f888dfe wireproto: define and implement responses in framing protocol Gregory Szorc <gregory.szorc@gmail.com> parents: 37055 diff changeset	516 sendframes
61393f888dfe wireproto: define and implement responses in framing protocol Gregory Szorc <gregory.szorc@gmail.com> parents: 37055 diff changeset	517 Indicates that frames should be sent to the client. The ``framegen``
61393f888dfe wireproto: define and implement responses in framing protocol Gregory Szorc <gregory.szorc@gmail.com> parents: 37055 diff changeset	518 key contains a generator of frames that should be sent. The server
61393f888dfe wireproto: define and implement responses in framing protocol Gregory Szorc <gregory.szorc@gmail.com> parents: 37055 diff changeset	519 assumes that all frames are sent to the client.
61393f888dfe wireproto: define and implement responses in framing protocol Gregory Szorc <gregory.szorc@gmail.com> parents: 37055 diff changeset	520
37055 8c3c47362934 wireproto: implement basic frame reading and processing Gregory Szorc <gregory.szorc@gmail.com> parents: 37054 diff changeset	521 error
8c3c47362934 wireproto: implement basic frame reading and processing Gregory Szorc <gregory.szorc@gmail.com> parents: 37054 diff changeset	522 Indicates that an error occurred. Consumer should probably abort.
8c3c47362934 wireproto: implement basic frame reading and processing Gregory Szorc <gregory.szorc@gmail.com> parents: 37054 diff changeset	523
8c3c47362934 wireproto: implement basic frame reading and processing Gregory Szorc <gregory.szorc@gmail.com> parents: 37054 diff changeset	524 runcommand
8c3c47362934 wireproto: implement basic frame reading and processing Gregory Szorc <gregory.szorc@gmail.com> parents: 37054 diff changeset	525 Indicates that the consumer should run a wire protocol command. Details
8c3c47362934 wireproto: implement basic frame reading and processing Gregory Szorc <gregory.szorc@gmail.com> parents: 37054 diff changeset	526 of the command to run are given in the data structure.
8c3c47362934 wireproto: implement basic frame reading and processing Gregory Szorc <gregory.szorc@gmail.com> parents: 37054 diff changeset	527
8c3c47362934 wireproto: implement basic frame reading and processing Gregory Szorc <gregory.szorc@gmail.com> parents: 37054 diff changeset	528 wantframe
8c3c47362934 wireproto: implement basic frame reading and processing Gregory Szorc <gregory.szorc@gmail.com> parents: 37054 diff changeset	529 Indicates that nothing of interest happened and the server is waiting on
8c3c47362934 wireproto: implement basic frame reading and processing Gregory Szorc <gregory.szorc@gmail.com> parents: 37054 diff changeset	530 more frames from the client before anything interesting can be done.
37059 861e9d37e56e wireproto: buffer output frames when in half duplex mode Gregory Szorc <gregory.szorc@gmail.com> parents: 37058 diff changeset	531
861e9d37e56e wireproto: buffer output frames when in half duplex mode Gregory Szorc <gregory.szorc@gmail.com> parents: 37058 diff changeset	532 noop
861e9d37e56e wireproto: buffer output frames when in half duplex mode Gregory Szorc <gregory.szorc@gmail.com> parents: 37058 diff changeset	533 Indicates no additional action is required.
37061 c5e9c3b47366 wireproto: support for receiving multiple requests Gregory Szorc <gregory.szorc@gmail.com> parents: 37060 diff changeset	534
c5e9c3b47366 wireproto: support for receiving multiple requests Gregory Szorc <gregory.szorc@gmail.com> parents: 37060 diff changeset	535 Known Issues
c5e9c3b47366 wireproto: support for receiving multiple requests Gregory Szorc <gregory.szorc@gmail.com> parents: 37060 diff changeset	536 ------------
c5e9c3b47366 wireproto: support for receiving multiple requests Gregory Szorc <gregory.szorc@gmail.com> parents: 37060 diff changeset	537
c5e9c3b47366 wireproto: support for receiving multiple requests Gregory Szorc <gregory.szorc@gmail.com> parents: 37060 diff changeset	538 There are no limits to the number of partially received commands or their
c5e9c3b47366 wireproto: support for receiving multiple requests Gregory Szorc <gregory.szorc@gmail.com> parents: 37060 diff changeset	539 size. A malicious client could stream command request data and exhaust the
c5e9c3b47366 wireproto: support for receiving multiple requests Gregory Szorc <gregory.szorc@gmail.com> parents: 37060 diff changeset	540 server's memory.
c5e9c3b47366 wireproto: support for receiving multiple requests Gregory Szorc <gregory.szorc@gmail.com> parents: 37060 diff changeset	541
c5e9c3b47366 wireproto: support for receiving multiple requests Gregory Szorc <gregory.szorc@gmail.com> parents: 37060 diff changeset	542 Partially received commands are not acted upon when end of input is
c5e9c3b47366 wireproto: support for receiving multiple requests Gregory Szorc <gregory.szorc@gmail.com> parents: 37060 diff changeset	543 reached. Should the server error if it receives a partial request?
c5e9c3b47366 wireproto: support for receiving multiple requests Gregory Szorc <gregory.szorc@gmail.com> parents: 37060 diff changeset	544 Should the client send a message to abort a partially transmitted request
c5e9c3b47366 wireproto: support for receiving multiple requests Gregory Szorc <gregory.szorc@gmail.com> parents: 37060 diff changeset	545 to facilitate graceful shutdown?
c5e9c3b47366 wireproto: support for receiving multiple requests Gregory Szorc <gregory.szorc@gmail.com> parents: 37060 diff changeset	546
c5e9c3b47366 wireproto: support for receiving multiple requests Gregory Szorc <gregory.szorc@gmail.com> parents: 37060 diff changeset	547 Active requests that haven't been responded to aren't tracked. This means
c5e9c3b47366 wireproto: support for receiving multiple requests Gregory Szorc <gregory.szorc@gmail.com> parents: 37060 diff changeset	548 that if we receive a command and instruct its dispatch, another command
c5e9c3b47366 wireproto: support for receiving multiple requests Gregory Szorc <gregory.szorc@gmail.com> parents: 37060 diff changeset	549 with its request ID can come in over the wire and there will be a race
c5e9c3b47366 wireproto: support for receiving multiple requests Gregory Szorc <gregory.szorc@gmail.com> parents: 37060 diff changeset	550 between who responds to what.
37055 8c3c47362934 wireproto: implement basic frame reading and processing Gregory Szorc <gregory.szorc@gmail.com> parents: 37054 diff changeset	551 """
8c3c47362934 wireproto: implement basic frame reading and processing Gregory Szorc <gregory.szorc@gmail.com> parents: 37054 diff changeset	552
37059 861e9d37e56e wireproto: buffer output frames when in half duplex mode Gregory Szorc <gregory.szorc@gmail.com> parents: 37058 diff changeset	553 def __init__(self, deferoutput=False):
861e9d37e56e wireproto: buffer output frames when in half duplex mode Gregory Szorc <gregory.szorc@gmail.com> parents: 37058 diff changeset	554 """Construct a new server reactor.
861e9d37e56e wireproto: buffer output frames when in half duplex mode Gregory Szorc <gregory.szorc@gmail.com> parents: 37058 diff changeset	555
861e9d37e56e wireproto: buffer output frames when in half duplex mode Gregory Szorc <gregory.szorc@gmail.com> parents: 37058 diff changeset	556 ``deferoutput`` can be used to indicate that no output frames should be
861e9d37e56e wireproto: buffer output frames when in half duplex mode Gregory Szorc <gregory.szorc@gmail.com> parents: 37058 diff changeset	557 instructed to be sent until input has been exhausted. In this mode,
861e9d37e56e wireproto: buffer output frames when in half duplex mode Gregory Szorc <gregory.szorc@gmail.com> parents: 37058 diff changeset	558 events that would normally generate output frames (such as a command
861e9d37e56e wireproto: buffer output frames when in half duplex mode Gregory Szorc <gregory.szorc@gmail.com> parents: 37058 diff changeset	559 response being ready) will instead defer instructing the consumer to
861e9d37e56e wireproto: buffer output frames when in half duplex mode Gregory Szorc <gregory.szorc@gmail.com> parents: 37058 diff changeset	560 send those frames. This is useful for half-duplex transports where the
861e9d37e56e wireproto: buffer output frames when in half duplex mode Gregory Szorc <gregory.szorc@gmail.com> parents: 37058 diff changeset	561 sender cannot receive until all data has been transmitted.
861e9d37e56e wireproto: buffer output frames when in half duplex mode Gregory Szorc <gregory.szorc@gmail.com> parents: 37058 diff changeset	562 """
861e9d37e56e wireproto: buffer output frames when in half duplex mode Gregory Szorc <gregory.szorc@gmail.com> parents: 37058 diff changeset	563 self._deferoutput = deferoutput
37055 8c3c47362934 wireproto: implement basic frame reading and processing Gregory Szorc <gregory.szorc@gmail.com> parents: 37054 diff changeset	564 self._state = 'idle'
37289 5fadc63ac99f wireproto: explicit API to create outgoing streams Gregory Szorc <gregory.szorc@gmail.com> parents: 37288 diff changeset	565 self._nextoutgoingstreamid = 2
37059 861e9d37e56e wireproto: buffer output frames when in half duplex mode Gregory Szorc <gregory.szorc@gmail.com> parents: 37058 diff changeset	566 self._bufferedframegens = []
37288 9bfcbe4f4745 wireproto: add streams to frame-based protocol Gregory Szorc <gregory.szorc@gmail.com> parents: 37287 diff changeset	567 # stream id -> stream instance for all active streams from the client.
9bfcbe4f4745 wireproto: add streams to frame-based protocol Gregory Szorc <gregory.szorc@gmail.com> parents: 37287 diff changeset	568 self._incomingstreams = {}
37289 5fadc63ac99f wireproto: explicit API to create outgoing streams Gregory Szorc <gregory.szorc@gmail.com> parents: 37288 diff changeset	569 self._outgoingstreams = {}
37061 c5e9c3b47366 wireproto: support for receiving multiple requests Gregory Szorc <gregory.szorc@gmail.com> parents: 37060 diff changeset	570 # request id -> dict of commands that are actively being received.
c5e9c3b47366 wireproto: support for receiving multiple requests Gregory Szorc <gregory.szorc@gmail.com> parents: 37060 diff changeset	571 self._receivingcommands = {}
37066 39304dd63589 wireproto: explicitly track which requests are active Gregory Szorc <gregory.szorc@gmail.com> parents: 37065 diff changeset	572 # Request IDs that have been received and are actively being processed.
39304dd63589 wireproto: explicitly track which requests are active Gregory Szorc <gregory.szorc@gmail.com> parents: 37065 diff changeset	573 # Once all output for a request has been sent, it is removed from this
39304dd63589 wireproto: explicitly track which requests are active Gregory Szorc <gregory.szorc@gmail.com> parents: 37065 diff changeset	574 # set.
39304dd63589 wireproto: explicitly track which requests are active Gregory Szorc <gregory.szorc@gmail.com> parents: 37065 diff changeset	575 self._activecommands = set()
37055 8c3c47362934 wireproto: implement basic frame reading and processing Gregory Szorc <gregory.szorc@gmail.com> parents: 37054 diff changeset	576
37064 884a0c1604ad wireproto: define attr-based classes for representing frames Gregory Szorc <gregory.szorc@gmail.com> parents: 37063 diff changeset	577 def onframerecv(self, frame):
37055 8c3c47362934 wireproto: implement basic frame reading and processing Gregory Szorc <gregory.szorc@gmail.com> parents: 37054 diff changeset	578 """Process a frame that has been received off the wire.
8c3c47362934 wireproto: implement basic frame reading and processing Gregory Szorc <gregory.szorc@gmail.com> parents: 37054 diff changeset	579
8c3c47362934 wireproto: implement basic frame reading and processing Gregory Szorc <gregory.szorc@gmail.com> parents: 37054 diff changeset	580 Returns a dict with an ``action`` key that details what action,
8c3c47362934 wireproto: implement basic frame reading and processing Gregory Szorc <gregory.szorc@gmail.com> parents: 37054 diff changeset	581 if any, the consumer should take next.
8c3c47362934 wireproto: implement basic frame reading and processing Gregory Szorc <gregory.szorc@gmail.com> parents: 37054 diff changeset	582 """
37288 9bfcbe4f4745 wireproto: add streams to frame-based protocol Gregory Szorc <gregory.szorc@gmail.com> parents: 37287 diff changeset	583 if not frame.streamid % 2:
9bfcbe4f4745 wireproto: add streams to frame-based protocol Gregory Szorc <gregory.szorc@gmail.com> parents: 37287 diff changeset	584 self._state = 'errored'
9bfcbe4f4745 wireproto: add streams to frame-based protocol Gregory Szorc <gregory.szorc@gmail.com> parents: 37287 diff changeset	585 return self._makeerrorresult(
9bfcbe4f4745 wireproto: add streams to frame-based protocol Gregory Szorc <gregory.szorc@gmail.com> parents: 37287 diff changeset	586 _('received frame with even numbered stream ID: %d') %
9bfcbe4f4745 wireproto: add streams to frame-based protocol Gregory Szorc <gregory.szorc@gmail.com> parents: 37287 diff changeset	587 frame.streamid)
9bfcbe4f4745 wireproto: add streams to frame-based protocol Gregory Szorc <gregory.szorc@gmail.com> parents: 37287 diff changeset	588
9bfcbe4f4745 wireproto: add streams to frame-based protocol Gregory Szorc <gregory.szorc@gmail.com> parents: 37287 diff changeset	589 if frame.streamid not in self._incomingstreams:
9bfcbe4f4745 wireproto: add streams to frame-based protocol Gregory Szorc <gregory.szorc@gmail.com> parents: 37287 diff changeset	590 if not frame.streamflags & STREAM_FLAG_BEGIN_STREAM:
9bfcbe4f4745 wireproto: add streams to frame-based protocol Gregory Szorc <gregory.szorc@gmail.com> parents: 37287 diff changeset	591 self._state = 'errored'
9bfcbe4f4745 wireproto: add streams to frame-based protocol Gregory Szorc <gregory.szorc@gmail.com> parents: 37287 diff changeset	592 return self._makeerrorresult(
9bfcbe4f4745 wireproto: add streams to frame-based protocol Gregory Szorc <gregory.szorc@gmail.com> parents: 37287 diff changeset	593 _('received frame on unknown inactive stream without '
9bfcbe4f4745 wireproto: add streams to frame-based protocol Gregory Szorc <gregory.szorc@gmail.com> parents: 37287 diff changeset	594 'beginning of stream flag set'))
9bfcbe4f4745 wireproto: add streams to frame-based protocol Gregory Szorc <gregory.szorc@gmail.com> parents: 37287 diff changeset	595
9bfcbe4f4745 wireproto: add streams to frame-based protocol Gregory Szorc <gregory.szorc@gmail.com> parents: 37287 diff changeset	596 self._incomingstreams[frame.streamid] = stream(frame.streamid)
9bfcbe4f4745 wireproto: add streams to frame-based protocol Gregory Szorc <gregory.szorc@gmail.com> parents: 37287 diff changeset	597
9bfcbe4f4745 wireproto: add streams to frame-based protocol Gregory Szorc <gregory.szorc@gmail.com> parents: 37287 diff changeset	598 if frame.streamflags & STREAM_FLAG_ENCODING_APPLIED:
9bfcbe4f4745 wireproto: add streams to frame-based protocol Gregory Szorc <gregory.szorc@gmail.com> parents: 37287 diff changeset	599 # TODO handle decoding frames
9bfcbe4f4745 wireproto: add streams to frame-based protocol Gregory Szorc <gregory.szorc@gmail.com> parents: 37287 diff changeset	600 self._state = 'errored'
9bfcbe4f4745 wireproto: add streams to frame-based protocol Gregory Szorc <gregory.szorc@gmail.com> parents: 37287 diff changeset	601 raise error.ProgrammingError('support for decoding stream payloads '
9bfcbe4f4745 wireproto: add streams to frame-based protocol Gregory Szorc <gregory.szorc@gmail.com> parents: 37287 diff changeset	602 'not yet implemented')
9bfcbe4f4745 wireproto: add streams to frame-based protocol Gregory Szorc <gregory.szorc@gmail.com> parents: 37287 diff changeset	603
9bfcbe4f4745 wireproto: add streams to frame-based protocol Gregory Szorc <gregory.szorc@gmail.com> parents: 37287 diff changeset	604 if frame.streamflags & STREAM_FLAG_END_STREAM:
9bfcbe4f4745 wireproto: add streams to frame-based protocol Gregory Szorc <gregory.szorc@gmail.com> parents: 37287 diff changeset	605 del self._incomingstreams[frame.streamid]
9bfcbe4f4745 wireproto: add streams to frame-based protocol Gregory Szorc <gregory.szorc@gmail.com> parents: 37287 diff changeset	606
37055 8c3c47362934 wireproto: implement basic frame reading and processing Gregory Szorc <gregory.szorc@gmail.com> parents: 37054 diff changeset	607 handlers = {
8c3c47362934 wireproto: implement basic frame reading and processing Gregory Szorc <gregory.szorc@gmail.com> parents: 37054 diff changeset	608 'idle': self._onframeidle,
37061 c5e9c3b47366 wireproto: support for receiving multiple requests Gregory Szorc <gregory.szorc@gmail.com> parents: 37060 diff changeset	609 'command-receiving': self._onframecommandreceiving,
37055 8c3c47362934 wireproto: implement basic frame reading and processing Gregory Szorc <gregory.szorc@gmail.com> parents: 37054 diff changeset	610 'errored': self._onframeerrored,
8c3c47362934 wireproto: implement basic frame reading and processing Gregory Szorc <gregory.szorc@gmail.com> parents: 37054 diff changeset	611 }
8c3c47362934 wireproto: implement basic frame reading and processing Gregory Szorc <gregory.szorc@gmail.com> parents: 37054 diff changeset	612
8c3c47362934 wireproto: implement basic frame reading and processing Gregory Szorc <gregory.szorc@gmail.com> parents: 37054 diff changeset	613 meth = handlers.get(self._state)
8c3c47362934 wireproto: implement basic frame reading and processing Gregory Szorc <gregory.szorc@gmail.com> parents: 37054 diff changeset	614 if not meth:
8c3c47362934 wireproto: implement basic frame reading and processing Gregory Szorc <gregory.szorc@gmail.com> parents: 37054 diff changeset	615 raise error.ProgrammingError('unhandled state: %s' % self._state)
8c3c47362934 wireproto: implement basic frame reading and processing Gregory Szorc <gregory.szorc@gmail.com> parents: 37054 diff changeset	616
37064 884a0c1604ad wireproto: define attr-based classes for representing frames Gregory Szorc <gregory.szorc@gmail.com> parents: 37063 diff changeset	617 return meth(frame)
37055 8c3c47362934 wireproto: implement basic frame reading and processing Gregory Szorc <gregory.szorc@gmail.com> parents: 37054 diff changeset	618
37485 0b7475ea38cf wireproto: port heads command to wire protocol v2 Gregory Szorc <gregory.szorc@gmail.com> parents: 37476 diff changeset	619 def onbytesresponseready(self, stream, requestid, data, iscbor=False):
37058 61393f888dfe wireproto: define and implement responses in framing protocol Gregory Szorc <gregory.szorc@gmail.com> parents: 37055 diff changeset	620 """Signal that a bytes response is ready to be sent to the client.
61393f888dfe wireproto: define and implement responses in framing protocol Gregory Szorc <gregory.szorc@gmail.com> parents: 37055 diff changeset	621
61393f888dfe wireproto: define and implement responses in framing protocol Gregory Szorc <gregory.szorc@gmail.com> parents: 37055 diff changeset	622 The raw bytes response is passed as an argument.
61393f888dfe wireproto: define and implement responses in framing protocol Gregory Szorc <gregory.szorc@gmail.com> parents: 37055 diff changeset	623 """
37288 9bfcbe4f4745 wireproto: add streams to frame-based protocol Gregory Szorc <gregory.szorc@gmail.com> parents: 37287 diff changeset	624 ensureserverstream(stream)
9bfcbe4f4745 wireproto: add streams to frame-based protocol Gregory Szorc <gregory.szorc@gmail.com> parents: 37287 diff changeset	625
37066 39304dd63589 wireproto: explicitly track which requests are active Gregory Szorc <gregory.szorc@gmail.com> parents: 37065 diff changeset	626 def sendframes():
37287 3ed344546d9e wireproto: start to associate frame generation with a stream Gregory Szorc <gregory.szorc@gmail.com> parents: 37087 diff changeset	627 for frame in createbytesresponseframesfrombytes(stream, requestid,
37485 0b7475ea38cf wireproto: port heads command to wire protocol v2 Gregory Szorc <gregory.szorc@gmail.com> parents: 37476 diff changeset	628 data,
0b7475ea38cf wireproto: port heads command to wire protocol v2 Gregory Szorc <gregory.szorc@gmail.com> parents: 37476 diff changeset	629 iscbor=iscbor):
37066 39304dd63589 wireproto: explicitly track which requests are active Gregory Szorc <gregory.szorc@gmail.com> parents: 37065 diff changeset	630 yield frame
39304dd63589 wireproto: explicitly track which requests are active Gregory Szorc <gregory.szorc@gmail.com> parents: 37065 diff changeset	631
39304dd63589 wireproto: explicitly track which requests are active Gregory Szorc <gregory.szorc@gmail.com> parents: 37065 diff changeset	632 self._activecommands.remove(requestid)
39304dd63589 wireproto: explicitly track which requests are active Gregory Szorc <gregory.szorc@gmail.com> parents: 37065 diff changeset	633
39304dd63589 wireproto: explicitly track which requests are active Gregory Szorc <gregory.szorc@gmail.com> parents: 37065 diff changeset	634 result = sendframes()
37059 861e9d37e56e wireproto: buffer output frames when in half duplex mode Gregory Szorc <gregory.szorc@gmail.com> parents: 37058 diff changeset	635
861e9d37e56e wireproto: buffer output frames when in half duplex mode Gregory Szorc <gregory.szorc@gmail.com> parents: 37058 diff changeset	636 if self._deferoutput:
37066 39304dd63589 wireproto: explicitly track which requests are active Gregory Szorc <gregory.szorc@gmail.com> parents: 37065 diff changeset	637 self._bufferedframegens.append(result)
37059 861e9d37e56e wireproto: buffer output frames when in half duplex mode Gregory Szorc <gregory.szorc@gmail.com> parents: 37058 diff changeset	638 return 'noop', {}
861e9d37e56e wireproto: buffer output frames when in half duplex mode Gregory Szorc <gregory.szorc@gmail.com> parents: 37058 diff changeset	639 else:
861e9d37e56e wireproto: buffer output frames when in half duplex mode Gregory Szorc <gregory.szorc@gmail.com> parents: 37058 diff changeset	640 return 'sendframes', {
37066 39304dd63589 wireproto: explicitly track which requests are active Gregory Szorc <gregory.szorc@gmail.com> parents: 37065 diff changeset	641 'framegen': result,
37059 861e9d37e56e wireproto: buffer output frames when in half duplex mode Gregory Szorc <gregory.szorc@gmail.com> parents: 37058 diff changeset	642 }
861e9d37e56e wireproto: buffer output frames when in half duplex mode Gregory Szorc <gregory.szorc@gmail.com> parents: 37058 diff changeset	643
861e9d37e56e wireproto: buffer output frames when in half duplex mode Gregory Szorc <gregory.szorc@gmail.com> parents: 37058 diff changeset	644 def oninputeof(self):
861e9d37e56e wireproto: buffer output frames when in half duplex mode Gregory Szorc <gregory.szorc@gmail.com> parents: 37058 diff changeset	645 """Signals that end of input has been received.
861e9d37e56e wireproto: buffer output frames when in half duplex mode Gregory Szorc <gregory.szorc@gmail.com> parents: 37058 diff changeset	646
861e9d37e56e wireproto: buffer output frames when in half duplex mode Gregory Szorc <gregory.szorc@gmail.com> parents: 37058 diff changeset	647 No more frames will be received. All pending activity should be
861e9d37e56e wireproto: buffer output frames when in half duplex mode Gregory Szorc <gregory.szorc@gmail.com> parents: 37058 diff changeset	648 completed.
861e9d37e56e wireproto: buffer output frames when in half duplex mode Gregory Szorc <gregory.szorc@gmail.com> parents: 37058 diff changeset	649 """
37061 c5e9c3b47366 wireproto: support for receiving multiple requests Gregory Szorc <gregory.szorc@gmail.com> parents: 37060 diff changeset	650 # TODO should we do anything about in-flight commands?
c5e9c3b47366 wireproto: support for receiving multiple requests Gregory Szorc <gregory.szorc@gmail.com> parents: 37060 diff changeset	651
37059 861e9d37e56e wireproto: buffer output frames when in half duplex mode Gregory Szorc <gregory.szorc@gmail.com> parents: 37058 diff changeset	652 if not self._deferoutput or not self._bufferedframegens:
861e9d37e56e wireproto: buffer output frames when in half duplex mode Gregory Szorc <gregory.szorc@gmail.com> parents: 37058 diff changeset	653 return 'noop', {}
861e9d37e56e wireproto: buffer output frames when in half duplex mode Gregory Szorc <gregory.szorc@gmail.com> parents: 37058 diff changeset	654
861e9d37e56e wireproto: buffer output frames when in half duplex mode Gregory Szorc <gregory.szorc@gmail.com> parents: 37058 diff changeset	655 # If we buffered all our responses, emit those.
861e9d37e56e wireproto: buffer output frames when in half duplex mode Gregory Szorc <gregory.szorc@gmail.com> parents: 37058 diff changeset	656 def makegen():
861e9d37e56e wireproto: buffer output frames when in half duplex mode Gregory Szorc <gregory.szorc@gmail.com> parents: 37058 diff changeset	657 for gen in self._bufferedframegens:
861e9d37e56e wireproto: buffer output frames when in half duplex mode Gregory Szorc <gregory.szorc@gmail.com> parents: 37058 diff changeset	658 for frame in gen:
861e9d37e56e wireproto: buffer output frames when in half duplex mode Gregory Szorc <gregory.szorc@gmail.com> parents: 37058 diff changeset	659 yield frame
861e9d37e56e wireproto: buffer output frames when in half duplex mode Gregory Szorc <gregory.szorc@gmail.com> parents: 37058 diff changeset	660
37058 61393f888dfe wireproto: define and implement responses in framing protocol Gregory Szorc <gregory.szorc@gmail.com> parents: 37055 diff changeset	661 return 'sendframes', {
37059 861e9d37e56e wireproto: buffer output frames when in half duplex mode Gregory Szorc <gregory.szorc@gmail.com> parents: 37058 diff changeset	662 'framegen': makegen(),
37058 61393f888dfe wireproto: define and implement responses in framing protocol Gregory Szorc <gregory.szorc@gmail.com> parents: 37055 diff changeset	663 }
61393f888dfe wireproto: define and implement responses in framing protocol Gregory Szorc <gregory.szorc@gmail.com> parents: 37055 diff changeset	664
37287 3ed344546d9e wireproto: start to associate frame generation with a stream Gregory Szorc <gregory.szorc@gmail.com> parents: 37087 diff changeset	665 def onapplicationerror(self, stream, requestid, msg):
37288 9bfcbe4f4745 wireproto: add streams to frame-based protocol Gregory Szorc <gregory.szorc@gmail.com> parents: 37287 diff changeset	666 ensureserverstream(stream)
9bfcbe4f4745 wireproto: add streams to frame-based protocol Gregory Szorc <gregory.szorc@gmail.com> parents: 37287 diff changeset	667
37058 61393f888dfe wireproto: define and implement responses in framing protocol Gregory Szorc <gregory.szorc@gmail.com> parents: 37055 diff changeset	668 return 'sendframes', {
37287 3ed344546d9e wireproto: start to associate frame generation with a stream Gregory Szorc <gregory.szorc@gmail.com> parents: 37087 diff changeset	669 'framegen': createerrorframe(stream, requestid, msg,
3ed344546d9e wireproto: start to associate frame generation with a stream Gregory Szorc <gregory.szorc@gmail.com> parents: 37087 diff changeset	670 application=True),
37058 61393f888dfe wireproto: define and implement responses in framing protocol Gregory Szorc <gregory.szorc@gmail.com> parents: 37055 diff changeset	671 }
61393f888dfe wireproto: define and implement responses in framing protocol Gregory Szorc <gregory.szorc@gmail.com> parents: 37055 diff changeset	672
37289 5fadc63ac99f wireproto: explicit API to create outgoing streams Gregory Szorc <gregory.szorc@gmail.com> parents: 37288 diff changeset	673 def makeoutputstream(self):
5fadc63ac99f wireproto: explicit API to create outgoing streams Gregory Szorc <gregory.szorc@gmail.com> parents: 37288 diff changeset	674 """Create a stream to be used for sending data to the client."""
5fadc63ac99f wireproto: explicit API to create outgoing streams Gregory Szorc <gregory.szorc@gmail.com> parents: 37288 diff changeset	675 streamid = self._nextoutgoingstreamid
5fadc63ac99f wireproto: explicit API to create outgoing streams Gregory Szorc <gregory.szorc@gmail.com> parents: 37288 diff changeset	676 self._nextoutgoingstreamid += 2
5fadc63ac99f wireproto: explicit API to create outgoing streams Gregory Szorc <gregory.szorc@gmail.com> parents: 37288 diff changeset	677
5fadc63ac99f wireproto: explicit API to create outgoing streams Gregory Szorc <gregory.szorc@gmail.com> parents: 37288 diff changeset	678 s = stream(streamid)
5fadc63ac99f wireproto: explicit API to create outgoing streams Gregory Szorc <gregory.szorc@gmail.com> parents: 37288 diff changeset	679 self._outgoingstreams[streamid] = s
5fadc63ac99f wireproto: explicit API to create outgoing streams Gregory Szorc <gregory.szorc@gmail.com> parents: 37288 diff changeset	680
5fadc63ac99f wireproto: explicit API to create outgoing streams Gregory Szorc <gregory.szorc@gmail.com> parents: 37288 diff changeset	681 return s
5fadc63ac99f wireproto: explicit API to create outgoing streams Gregory Szorc <gregory.szorc@gmail.com> parents: 37288 diff changeset	682
37055 8c3c47362934 wireproto: implement basic frame reading and processing Gregory Szorc <gregory.szorc@gmail.com> parents: 37054 diff changeset	683 def _makeerrorresult(self, msg):
8c3c47362934 wireproto: implement basic frame reading and processing Gregory Szorc <gregory.szorc@gmail.com> parents: 37054 diff changeset	684 return 'error', {
8c3c47362934 wireproto: implement basic frame reading and processing Gregory Szorc <gregory.szorc@gmail.com> parents: 37054 diff changeset	685 'message': msg,
8c3c47362934 wireproto: implement basic frame reading and processing Gregory Szorc <gregory.szorc@gmail.com> parents: 37054 diff changeset	686 }
8c3c47362934 wireproto: implement basic frame reading and processing Gregory Szorc <gregory.szorc@gmail.com> parents: 37054 diff changeset	687
37061 c5e9c3b47366 wireproto: support for receiving multiple requests Gregory Szorc <gregory.szorc@gmail.com> parents: 37060 diff changeset	688 def _makeruncommandresult(self, requestid):
c5e9c3b47366 wireproto: support for receiving multiple requests Gregory Szorc <gregory.szorc@gmail.com> parents: 37060 diff changeset	689 entry = self._receivingcommands[requestid]
37292 3d0e2cd86e05 wireproto: use CBOR for command requests Gregory Szorc <gregory.szorc@gmail.com> parents: 37291 diff changeset	690
3d0e2cd86e05 wireproto: use CBOR for command requests Gregory Szorc <gregory.szorc@gmail.com> parents: 37291 diff changeset	691 if not entry['requestdone']:
3d0e2cd86e05 wireproto: use CBOR for command requests Gregory Szorc <gregory.szorc@gmail.com> parents: 37291 diff changeset	692 self._state = 'errored'
3d0e2cd86e05 wireproto: use CBOR for command requests Gregory Szorc <gregory.szorc@gmail.com> parents: 37291 diff changeset	693 raise error.ProgrammingError('should not be called without '
3d0e2cd86e05 wireproto: use CBOR for command requests Gregory Szorc <gregory.szorc@gmail.com> parents: 37291 diff changeset	694 'requestdone set')
3d0e2cd86e05 wireproto: use CBOR for command requests Gregory Szorc <gregory.szorc@gmail.com> parents: 37291 diff changeset	695
37061 c5e9c3b47366 wireproto: support for receiving multiple requests Gregory Szorc <gregory.szorc@gmail.com> parents: 37060 diff changeset	696 del self._receivingcommands[requestid]
c5e9c3b47366 wireproto: support for receiving multiple requests Gregory Szorc <gregory.szorc@gmail.com> parents: 37060 diff changeset	697
c5e9c3b47366 wireproto: support for receiving multiple requests Gregory Szorc <gregory.szorc@gmail.com> parents: 37060 diff changeset	698 if self._receivingcommands:
c5e9c3b47366 wireproto: support for receiving multiple requests Gregory Szorc <gregory.szorc@gmail.com> parents: 37060 diff changeset	699 self._state = 'command-receiving'
c5e9c3b47366 wireproto: support for receiving multiple requests Gregory Szorc <gregory.szorc@gmail.com> parents: 37060 diff changeset	700 else:
c5e9c3b47366 wireproto: support for receiving multiple requests Gregory Szorc <gregory.szorc@gmail.com> parents: 37060 diff changeset	701 self._state = 'idle'
c5e9c3b47366 wireproto: support for receiving multiple requests Gregory Szorc <gregory.szorc@gmail.com> parents: 37060 diff changeset	702
37292 3d0e2cd86e05 wireproto: use CBOR for command requests Gregory Szorc <gregory.szorc@gmail.com> parents: 37291 diff changeset	703 # Decode the payloads as CBOR.
3d0e2cd86e05 wireproto: use CBOR for command requests Gregory Szorc <gregory.szorc@gmail.com> parents: 37291 diff changeset	704 entry['payload'].seek(0)
3d0e2cd86e05 wireproto: use CBOR for command requests Gregory Szorc <gregory.szorc@gmail.com> parents: 37291 diff changeset	705 request = cbor.load(entry['payload'])
3d0e2cd86e05 wireproto: use CBOR for command requests Gregory Szorc <gregory.szorc@gmail.com> parents: 37291 diff changeset	706
3d0e2cd86e05 wireproto: use CBOR for command requests Gregory Szorc <gregory.szorc@gmail.com> parents: 37291 diff changeset	707 if b'name' not in request:
3d0e2cd86e05 wireproto: use CBOR for command requests Gregory Szorc <gregory.szorc@gmail.com> parents: 37291 diff changeset	708 self._state = 'errored'
3d0e2cd86e05 wireproto: use CBOR for command requests Gregory Szorc <gregory.szorc@gmail.com> parents: 37291 diff changeset	709 return self._makeerrorresult(
3d0e2cd86e05 wireproto: use CBOR for command requests Gregory Szorc <gregory.szorc@gmail.com> parents: 37291 diff changeset	710 _('command request missing "name" field'))
3d0e2cd86e05 wireproto: use CBOR for command requests Gregory Szorc <gregory.szorc@gmail.com> parents: 37291 diff changeset	711
3d0e2cd86e05 wireproto: use CBOR for command requests Gregory Szorc <gregory.szorc@gmail.com> parents: 37291 diff changeset	712 if b'args' not in request:
3d0e2cd86e05 wireproto: use CBOR for command requests Gregory Szorc <gregory.szorc@gmail.com> parents: 37291 diff changeset	713 request[b'args'] = {}
3d0e2cd86e05 wireproto: use CBOR for command requests Gregory Szorc <gregory.szorc@gmail.com> parents: 37291 diff changeset	714
37066 39304dd63589 wireproto: explicitly track which requests are active Gregory Szorc <gregory.szorc@gmail.com> parents: 37065 diff changeset	715 assert requestid not in self._activecommands
39304dd63589 wireproto: explicitly track which requests are active Gregory Szorc <gregory.szorc@gmail.com> parents: 37065 diff changeset	716 self._activecommands.add(requestid)
39304dd63589 wireproto: explicitly track which requests are active Gregory Szorc <gregory.szorc@gmail.com> parents: 37065 diff changeset	717
37055 8c3c47362934 wireproto: implement basic frame reading and processing Gregory Szorc <gregory.szorc@gmail.com> parents: 37054 diff changeset	718 return 'runcommand', {
37061 c5e9c3b47366 wireproto: support for receiving multiple requests Gregory Szorc <gregory.szorc@gmail.com> parents: 37060 diff changeset	719 'requestid': requestid,
37292 3d0e2cd86e05 wireproto: use CBOR for command requests Gregory Szorc <gregory.szorc@gmail.com> parents: 37291 diff changeset	720 'command': request[b'name'],
3d0e2cd86e05 wireproto: use CBOR for command requests Gregory Szorc <gregory.szorc@gmail.com> parents: 37291 diff changeset	721 'args': request[b'args'],
37061 c5e9c3b47366 wireproto: support for receiving multiple requests Gregory Szorc <gregory.szorc@gmail.com> parents: 37060 diff changeset	722 'data': entry['data'].getvalue() if entry['data'] else None,
37055 8c3c47362934 wireproto: implement basic frame reading and processing Gregory Szorc <gregory.szorc@gmail.com> parents: 37054 diff changeset	723 }
8c3c47362934 wireproto: implement basic frame reading and processing Gregory Szorc <gregory.szorc@gmail.com> parents: 37054 diff changeset	724
8c3c47362934 wireproto: implement basic frame reading and processing Gregory Szorc <gregory.szorc@gmail.com> parents: 37054 diff changeset	725 def _makewantframeresult(self):
8c3c47362934 wireproto: implement basic frame reading and processing Gregory Szorc <gregory.szorc@gmail.com> parents: 37054 diff changeset	726 return 'wantframe', {
8c3c47362934 wireproto: implement basic frame reading and processing Gregory Szorc <gregory.szorc@gmail.com> parents: 37054 diff changeset	727 'state': self._state,
8c3c47362934 wireproto: implement basic frame reading and processing Gregory Szorc <gregory.szorc@gmail.com> parents: 37054 diff changeset	728 }
8c3c47362934 wireproto: implement basic frame reading and processing Gregory Szorc <gregory.szorc@gmail.com> parents: 37054 diff changeset	729
37292 3d0e2cd86e05 wireproto: use CBOR for command requests Gregory Szorc <gregory.szorc@gmail.com> parents: 37291 diff changeset	730 def _validatecommandrequestframe(self, frame):
3d0e2cd86e05 wireproto: use CBOR for command requests Gregory Szorc <gregory.szorc@gmail.com> parents: 37291 diff changeset	731 new = frame.flags & FLAG_COMMAND_REQUEST_NEW
3d0e2cd86e05 wireproto: use CBOR for command requests Gregory Szorc <gregory.szorc@gmail.com> parents: 37291 diff changeset	732 continuation = frame.flags & FLAG_COMMAND_REQUEST_CONTINUATION
3d0e2cd86e05 wireproto: use CBOR for command requests Gregory Szorc <gregory.szorc@gmail.com> parents: 37291 diff changeset	733
3d0e2cd86e05 wireproto: use CBOR for command requests Gregory Szorc <gregory.szorc@gmail.com> parents: 37291 diff changeset	734 if new and continuation:
3d0e2cd86e05 wireproto: use CBOR for command requests Gregory Szorc <gregory.szorc@gmail.com> parents: 37291 diff changeset	735 self._state = 'errored'
3d0e2cd86e05 wireproto: use CBOR for command requests Gregory Szorc <gregory.szorc@gmail.com> parents: 37291 diff changeset	736 return self._makeerrorresult(
3d0e2cd86e05 wireproto: use CBOR for command requests Gregory Szorc <gregory.szorc@gmail.com> parents: 37291 diff changeset	737 _('received command request frame with both new and '
3d0e2cd86e05 wireproto: use CBOR for command requests Gregory Szorc <gregory.szorc@gmail.com> parents: 37291 diff changeset	738 'continuation flags set'))
3d0e2cd86e05 wireproto: use CBOR for command requests Gregory Szorc <gregory.szorc@gmail.com> parents: 37291 diff changeset	739
3d0e2cd86e05 wireproto: use CBOR for command requests Gregory Szorc <gregory.szorc@gmail.com> parents: 37291 diff changeset	740 if not new and not continuation:
3d0e2cd86e05 wireproto: use CBOR for command requests Gregory Szorc <gregory.szorc@gmail.com> parents: 37291 diff changeset	741 self._state = 'errored'
3d0e2cd86e05 wireproto: use CBOR for command requests Gregory Szorc <gregory.szorc@gmail.com> parents: 37291 diff changeset	742 return self._makeerrorresult(
3d0e2cd86e05 wireproto: use CBOR for command requests Gregory Szorc <gregory.szorc@gmail.com> parents: 37291 diff changeset	743 _('received command request frame with neither new nor '
3d0e2cd86e05 wireproto: use CBOR for command requests Gregory Szorc <gregory.szorc@gmail.com> parents: 37291 diff changeset	744 'continuation flags set'))
3d0e2cd86e05 wireproto: use CBOR for command requests Gregory Szorc <gregory.szorc@gmail.com> parents: 37291 diff changeset	745
37064 884a0c1604ad wireproto: define attr-based classes for representing frames Gregory Szorc <gregory.szorc@gmail.com> parents: 37063 diff changeset	746 def _onframeidle(self, frame):
37055 8c3c47362934 wireproto: implement basic frame reading and processing Gregory Szorc <gregory.szorc@gmail.com> parents: 37054 diff changeset	747 # The only frame type that should be received in this state is a
8c3c47362934 wireproto: implement basic frame reading and processing Gregory Szorc <gregory.szorc@gmail.com> parents: 37054 diff changeset	748 # command request.
37292 3d0e2cd86e05 wireproto: use CBOR for command requests Gregory Szorc <gregory.szorc@gmail.com> parents: 37291 diff changeset	749 if frame.typeid != FRAME_TYPE_COMMAND_REQUEST:
37055 8c3c47362934 wireproto: implement basic frame reading and processing Gregory Szorc <gregory.szorc@gmail.com> parents: 37054 diff changeset	750 self._state = 'errored'
8c3c47362934 wireproto: implement basic frame reading and processing Gregory Szorc <gregory.szorc@gmail.com> parents: 37054 diff changeset	751 return self._makeerrorresult(
37292 3d0e2cd86e05 wireproto: use CBOR for command requests Gregory Szorc <gregory.szorc@gmail.com> parents: 37291 diff changeset	752 _('expected command request frame; got %d') % frame.typeid)
3d0e2cd86e05 wireproto: use CBOR for command requests Gregory Szorc <gregory.szorc@gmail.com> parents: 37291 diff changeset	753
3d0e2cd86e05 wireproto: use CBOR for command requests Gregory Szorc <gregory.szorc@gmail.com> parents: 37291 diff changeset	754 res = self._validatecommandrequestframe(frame)
3d0e2cd86e05 wireproto: use CBOR for command requests Gregory Szorc <gregory.szorc@gmail.com> parents: 37291 diff changeset	755 if res:
3d0e2cd86e05 wireproto: use CBOR for command requests Gregory Szorc <gregory.szorc@gmail.com> parents: 37291 diff changeset	756 return res
37055 8c3c47362934 wireproto: implement basic frame reading and processing Gregory Szorc <gregory.szorc@gmail.com> parents: 37054 diff changeset	757
37064 884a0c1604ad wireproto: define attr-based classes for representing frames Gregory Szorc <gregory.szorc@gmail.com> parents: 37063 diff changeset	758 if frame.requestid in self._receivingcommands:
37061 c5e9c3b47366 wireproto: support for receiving multiple requests Gregory Szorc <gregory.szorc@gmail.com> parents: 37060 diff changeset	759 self._state = 'errored'
c5e9c3b47366 wireproto: support for receiving multiple requests Gregory Szorc <gregory.szorc@gmail.com> parents: 37060 diff changeset	760 return self._makeerrorresult(
37064 884a0c1604ad wireproto: define attr-based classes for representing frames Gregory Szorc <gregory.szorc@gmail.com> parents: 37063 diff changeset	761 _('request with ID %d already received') % frame.requestid)
37061 c5e9c3b47366 wireproto: support for receiving multiple requests Gregory Szorc <gregory.szorc@gmail.com> parents: 37060 diff changeset	762
37066 39304dd63589 wireproto: explicitly track which requests are active Gregory Szorc <gregory.szorc@gmail.com> parents: 37065 diff changeset	763 if frame.requestid in self._activecommands:
39304dd63589 wireproto: explicitly track which requests are active Gregory Szorc <gregory.szorc@gmail.com> parents: 37065 diff changeset	764 self._state = 'errored'
37292 3d0e2cd86e05 wireproto: use CBOR for command requests Gregory Szorc <gregory.szorc@gmail.com> parents: 37291 diff changeset	765 return self._makeerrorresult(
3d0e2cd86e05 wireproto: use CBOR for command requests Gregory Szorc <gregory.szorc@gmail.com> parents: 37291 diff changeset	766 _('request with ID %d is already active') % frame.requestid)
3d0e2cd86e05 wireproto: use CBOR for command requests Gregory Szorc <gregory.szorc@gmail.com> parents: 37291 diff changeset	767
3d0e2cd86e05 wireproto: use CBOR for command requests Gregory Szorc <gregory.szorc@gmail.com> parents: 37291 diff changeset	768 new = frame.flags & FLAG_COMMAND_REQUEST_NEW
3d0e2cd86e05 wireproto: use CBOR for command requests Gregory Szorc <gregory.szorc@gmail.com> parents: 37291 diff changeset	769 moreframes = frame.flags & FLAG_COMMAND_REQUEST_MORE_FRAMES
3d0e2cd86e05 wireproto: use CBOR for command requests Gregory Szorc <gregory.szorc@gmail.com> parents: 37291 diff changeset	770 expectingdata = frame.flags & FLAG_COMMAND_REQUEST_EXPECT_DATA
37066 39304dd63589 wireproto: explicitly track which requests are active Gregory Szorc <gregory.szorc@gmail.com> parents: 37065 diff changeset	771
37292 3d0e2cd86e05 wireproto: use CBOR for command requests Gregory Szorc <gregory.szorc@gmail.com> parents: 37291 diff changeset	772 if not new:
3d0e2cd86e05 wireproto: use CBOR for command requests Gregory Szorc <gregory.szorc@gmail.com> parents: 37291 diff changeset	773 self._state = 'errored'
3d0e2cd86e05 wireproto: use CBOR for command requests Gregory Szorc <gregory.szorc@gmail.com> parents: 37291 diff changeset	774 return self._makeerrorresult(
3d0e2cd86e05 wireproto: use CBOR for command requests Gregory Szorc <gregory.szorc@gmail.com> parents: 37291 diff changeset	775 _('received command request frame without new flag set'))
3d0e2cd86e05 wireproto: use CBOR for command requests Gregory Szorc <gregory.szorc@gmail.com> parents: 37291 diff changeset	776
3d0e2cd86e05 wireproto: use CBOR for command requests Gregory Szorc <gregory.szorc@gmail.com> parents: 37291 diff changeset	777 payload = util.bytesio()
3d0e2cd86e05 wireproto: use CBOR for command requests Gregory Szorc <gregory.szorc@gmail.com> parents: 37291 diff changeset	778 payload.write(frame.payload)
37061 c5e9c3b47366 wireproto: support for receiving multiple requests Gregory Szorc <gregory.szorc@gmail.com> parents: 37060 diff changeset	779
37064 884a0c1604ad wireproto: define attr-based classes for representing frames Gregory Szorc <gregory.szorc@gmail.com> parents: 37063 diff changeset	780 self._receivingcommands[frame.requestid] = {
37292 3d0e2cd86e05 wireproto: use CBOR for command requests Gregory Szorc <gregory.szorc@gmail.com> parents: 37291 diff changeset	781 'payload': payload,
37061 c5e9c3b47366 wireproto: support for receiving multiple requests Gregory Szorc <gregory.szorc@gmail.com> parents: 37060 diff changeset	782 'data': None,
37292 3d0e2cd86e05 wireproto: use CBOR for command requests Gregory Szorc <gregory.szorc@gmail.com> parents: 37291 diff changeset	783 'requestdone': not moreframes,
3d0e2cd86e05 wireproto: use CBOR for command requests Gregory Szorc <gregory.szorc@gmail.com> parents: 37291 diff changeset	784 'expectingdata': bool(expectingdata),
37061 c5e9c3b47366 wireproto: support for receiving multiple requests Gregory Szorc <gregory.szorc@gmail.com> parents: 37060 diff changeset	785 }
37055 8c3c47362934 wireproto: implement basic frame reading and processing Gregory Szorc <gregory.szorc@gmail.com> parents: 37054 diff changeset	786
37292 3d0e2cd86e05 wireproto: use CBOR for command requests Gregory Szorc <gregory.szorc@gmail.com> parents: 37291 diff changeset	787 # This is the final frame for this request. Dispatch it.
3d0e2cd86e05 wireproto: use CBOR for command requests Gregory Szorc <gregory.szorc@gmail.com> parents: 37291 diff changeset	788 if not moreframes and not expectingdata:
37064 884a0c1604ad wireproto: define attr-based classes for representing frames Gregory Szorc <gregory.szorc@gmail.com> parents: 37063 diff changeset	789 return self._makeruncommandresult(frame.requestid)
37055 8c3c47362934 wireproto: implement basic frame reading and processing Gregory Szorc <gregory.szorc@gmail.com> parents: 37054 diff changeset	790
37292 3d0e2cd86e05 wireproto: use CBOR for command requests Gregory Szorc <gregory.szorc@gmail.com> parents: 37291 diff changeset	791 assert moreframes or expectingdata
3d0e2cd86e05 wireproto: use CBOR for command requests Gregory Szorc <gregory.szorc@gmail.com> parents: 37291 diff changeset	792 self._state = 'command-receiving'
3d0e2cd86e05 wireproto: use CBOR for command requests Gregory Szorc <gregory.szorc@gmail.com> parents: 37291 diff changeset	793 return self._makewantframeresult()
37055 8c3c47362934 wireproto: implement basic frame reading and processing Gregory Szorc <gregory.szorc@gmail.com> parents: 37054 diff changeset	794
37064 884a0c1604ad wireproto: define attr-based classes for representing frames Gregory Szorc <gregory.szorc@gmail.com> parents: 37063 diff changeset	795 def _onframecommandreceiving(self, frame):
37292 3d0e2cd86e05 wireproto: use CBOR for command requests Gregory Szorc <gregory.szorc@gmail.com> parents: 37291 diff changeset	796 if frame.typeid == FRAME_TYPE_COMMAND_REQUEST:
3d0e2cd86e05 wireproto: use CBOR for command requests Gregory Szorc <gregory.szorc@gmail.com> parents: 37291 diff changeset	797 # Process new command requests as such.
3d0e2cd86e05 wireproto: use CBOR for command requests Gregory Szorc <gregory.szorc@gmail.com> parents: 37291 diff changeset	798 if frame.flags & FLAG_COMMAND_REQUEST_NEW:
3d0e2cd86e05 wireproto: use CBOR for command requests Gregory Szorc <gregory.szorc@gmail.com> parents: 37291 diff changeset	799 return self._onframeidle(frame)
3d0e2cd86e05 wireproto: use CBOR for command requests Gregory Szorc <gregory.szorc@gmail.com> parents: 37291 diff changeset	800
3d0e2cd86e05 wireproto: use CBOR for command requests Gregory Szorc <gregory.szorc@gmail.com> parents: 37291 diff changeset	801 res = self._validatecommandrequestframe(frame)
3d0e2cd86e05 wireproto: use CBOR for command requests Gregory Szorc <gregory.szorc@gmail.com> parents: 37291 diff changeset	802 if res:
3d0e2cd86e05 wireproto: use CBOR for command requests Gregory Szorc <gregory.szorc@gmail.com> parents: 37291 diff changeset	803 return res
37061 c5e9c3b47366 wireproto: support for receiving multiple requests Gregory Szorc <gregory.szorc@gmail.com> parents: 37060 diff changeset	804
c5e9c3b47366 wireproto: support for receiving multiple requests Gregory Szorc <gregory.szorc@gmail.com> parents: 37060 diff changeset	805 # All other frames should be related to a command that is currently
37066 39304dd63589 wireproto: explicitly track which requests are active Gregory Szorc <gregory.szorc@gmail.com> parents: 37065 diff changeset	806 # receiving but is not active.
39304dd63589 wireproto: explicitly track which requests are active Gregory Szorc <gregory.szorc@gmail.com> parents: 37065 diff changeset	807 if frame.requestid in self._activecommands:
39304dd63589 wireproto: explicitly track which requests are active Gregory Szorc <gregory.szorc@gmail.com> parents: 37065 diff changeset	808 self._state = 'errored'
39304dd63589 wireproto: explicitly track which requests are active Gregory Szorc <gregory.szorc@gmail.com> parents: 37065 diff changeset	809 return self._makeerrorresult(
39304dd63589 wireproto: explicitly track which requests are active Gregory Szorc <gregory.szorc@gmail.com> parents: 37065 diff changeset	810 _('received frame for request that is still active: %d') %
39304dd63589 wireproto: explicitly track which requests are active Gregory Szorc <gregory.szorc@gmail.com> parents: 37065 diff changeset	811 frame.requestid)
39304dd63589 wireproto: explicitly track which requests are active Gregory Szorc <gregory.szorc@gmail.com> parents: 37065 diff changeset	812
37064 884a0c1604ad wireproto: define attr-based classes for representing frames Gregory Szorc <gregory.szorc@gmail.com> parents: 37063 diff changeset	813 if frame.requestid not in self._receivingcommands:
37055 8c3c47362934 wireproto: implement basic frame reading and processing Gregory Szorc <gregory.szorc@gmail.com> parents: 37054 diff changeset	814 self._state = 'errored'
37061 c5e9c3b47366 wireproto: support for receiving multiple requests Gregory Szorc <gregory.szorc@gmail.com> parents: 37060 diff changeset	815 return self._makeerrorresult(
c5e9c3b47366 wireproto: support for receiving multiple requests Gregory Szorc <gregory.szorc@gmail.com> parents: 37060 diff changeset	816 _('received frame for request that is not receiving: %d') %
37064 884a0c1604ad wireproto: define attr-based classes for representing frames Gregory Szorc <gregory.szorc@gmail.com> parents: 37063 diff changeset	817 frame.requestid)
37061 c5e9c3b47366 wireproto: support for receiving multiple requests Gregory Szorc <gregory.szorc@gmail.com> parents: 37060 diff changeset	818
37064 884a0c1604ad wireproto: define attr-based classes for representing frames Gregory Szorc <gregory.szorc@gmail.com> parents: 37063 diff changeset	819 entry = self._receivingcommands[frame.requestid]
37061 c5e9c3b47366 wireproto: support for receiving multiple requests Gregory Szorc <gregory.szorc@gmail.com> parents: 37060 diff changeset	820
37292 3d0e2cd86e05 wireproto: use CBOR for command requests Gregory Szorc <gregory.szorc@gmail.com> parents: 37291 diff changeset	821 if frame.typeid == FRAME_TYPE_COMMAND_REQUEST:
3d0e2cd86e05 wireproto: use CBOR for command requests Gregory Szorc <gregory.szorc@gmail.com> parents: 37291 diff changeset	822 moreframes = frame.flags & FLAG_COMMAND_REQUEST_MORE_FRAMES
3d0e2cd86e05 wireproto: use CBOR for command requests Gregory Szorc <gregory.szorc@gmail.com> parents: 37291 diff changeset	823 expectingdata = bool(frame.flags & FLAG_COMMAND_REQUEST_EXPECT_DATA)
3d0e2cd86e05 wireproto: use CBOR for command requests Gregory Szorc <gregory.szorc@gmail.com> parents: 37291 diff changeset	824
3d0e2cd86e05 wireproto: use CBOR for command requests Gregory Szorc <gregory.szorc@gmail.com> parents: 37291 diff changeset	825 if entry['requestdone']:
3d0e2cd86e05 wireproto: use CBOR for command requests Gregory Szorc <gregory.szorc@gmail.com> parents: 37291 diff changeset	826 self._state = 'errored'
3d0e2cd86e05 wireproto: use CBOR for command requests Gregory Szorc <gregory.szorc@gmail.com> parents: 37291 diff changeset	827 return self._makeerrorresult(
3d0e2cd86e05 wireproto: use CBOR for command requests Gregory Szorc <gregory.szorc@gmail.com> parents: 37291 diff changeset	828 _('received command request frame when request frames '
3d0e2cd86e05 wireproto: use CBOR for command requests Gregory Szorc <gregory.szorc@gmail.com> parents: 37291 diff changeset	829 'were supposedly done'))
3d0e2cd86e05 wireproto: use CBOR for command requests Gregory Szorc <gregory.szorc@gmail.com> parents: 37291 diff changeset	830
3d0e2cd86e05 wireproto: use CBOR for command requests Gregory Szorc <gregory.szorc@gmail.com> parents: 37291 diff changeset	831 if expectingdata != entry['expectingdata']:
37061 c5e9c3b47366 wireproto: support for receiving multiple requests Gregory Szorc <gregory.szorc@gmail.com> parents: 37060 diff changeset	832 self._state = 'errored'
37292 3d0e2cd86e05 wireproto: use CBOR for command requests Gregory Szorc <gregory.szorc@gmail.com> parents: 37291 diff changeset	833 return self._makeerrorresult(
3d0e2cd86e05 wireproto: use CBOR for command requests Gregory Szorc <gregory.szorc@gmail.com> parents: 37291 diff changeset	834 _('mismatch between expect data flag and previous frame'))
3d0e2cd86e05 wireproto: use CBOR for command requests Gregory Szorc <gregory.szorc@gmail.com> parents: 37291 diff changeset	835
3d0e2cd86e05 wireproto: use CBOR for command requests Gregory Szorc <gregory.szorc@gmail.com> parents: 37291 diff changeset	836 entry['payload'].write(frame.payload)
37061 c5e9c3b47366 wireproto: support for receiving multiple requests Gregory Szorc <gregory.szorc@gmail.com> parents: 37060 diff changeset	837
37292 3d0e2cd86e05 wireproto: use CBOR for command requests Gregory Szorc <gregory.szorc@gmail.com> parents: 37291 diff changeset	838 if not moreframes:
3d0e2cd86e05 wireproto: use CBOR for command requests Gregory Szorc <gregory.szorc@gmail.com> parents: 37291 diff changeset	839 entry['requestdone'] = True
3d0e2cd86e05 wireproto: use CBOR for command requests Gregory Szorc <gregory.szorc@gmail.com> parents: 37291 diff changeset	840
3d0e2cd86e05 wireproto: use CBOR for command requests Gregory Szorc <gregory.szorc@gmail.com> parents: 37291 diff changeset	841 if not moreframes and not expectingdata:
3d0e2cd86e05 wireproto: use CBOR for command requests Gregory Szorc <gregory.szorc@gmail.com> parents: 37291 diff changeset	842 return self._makeruncommandresult(frame.requestid)
3d0e2cd86e05 wireproto: use CBOR for command requests Gregory Szorc <gregory.szorc@gmail.com> parents: 37291 diff changeset	843
3d0e2cd86e05 wireproto: use CBOR for command requests Gregory Szorc <gregory.szorc@gmail.com> parents: 37291 diff changeset	844 return self._makewantframeresult()
37061 c5e9c3b47366 wireproto: support for receiving multiple requests Gregory Szorc <gregory.szorc@gmail.com> parents: 37060 diff changeset	845
37064 884a0c1604ad wireproto: define attr-based classes for representing frames Gregory Szorc <gregory.szorc@gmail.com> parents: 37063 diff changeset	846 elif frame.typeid == FRAME_TYPE_COMMAND_DATA:
37061 c5e9c3b47366 wireproto: support for receiving multiple requests Gregory Szorc <gregory.szorc@gmail.com> parents: 37060 diff changeset	847 if not entry['expectingdata']:
c5e9c3b47366 wireproto: support for receiving multiple requests Gregory Szorc <gregory.szorc@gmail.com> parents: 37060 diff changeset	848 self._state = 'errored'
c5e9c3b47366 wireproto: support for receiving multiple requests Gregory Szorc <gregory.szorc@gmail.com> parents: 37060 diff changeset	849 return self._makeerrorresult(_(
c5e9c3b47366 wireproto: support for receiving multiple requests Gregory Szorc <gregory.szorc@gmail.com> parents: 37060 diff changeset	850 'received command data frame for request that is not '
37064 884a0c1604ad wireproto: define attr-based classes for representing frames Gregory Szorc <gregory.szorc@gmail.com> parents: 37063 diff changeset	851 'expecting data: %d') % frame.requestid)
37061 c5e9c3b47366 wireproto: support for receiving multiple requests Gregory Szorc <gregory.szorc@gmail.com> parents: 37060 diff changeset	852
c5e9c3b47366 wireproto: support for receiving multiple requests Gregory Szorc <gregory.szorc@gmail.com> parents: 37060 diff changeset	853 if entry['data'] is None:
c5e9c3b47366 wireproto: support for receiving multiple requests Gregory Szorc <gregory.szorc@gmail.com> parents: 37060 diff changeset	854 entry['data'] = util.bytesio()
c5e9c3b47366 wireproto: support for receiving multiple requests Gregory Szorc <gregory.szorc@gmail.com> parents: 37060 diff changeset	855
37064 884a0c1604ad wireproto: define attr-based classes for representing frames Gregory Szorc <gregory.szorc@gmail.com> parents: 37063 diff changeset	856 return self._handlecommanddataframe(frame, entry)
37292 3d0e2cd86e05 wireproto: use CBOR for command requests Gregory Szorc <gregory.szorc@gmail.com> parents: 37291 diff changeset	857 else:
37055 8c3c47362934 wireproto: implement basic frame reading and processing Gregory Szorc <gregory.szorc@gmail.com> parents: 37054 diff changeset	858 self._state = 'errored'
37292 3d0e2cd86e05 wireproto: use CBOR for command requests Gregory Szorc <gregory.szorc@gmail.com> parents: 37291 diff changeset	859 return self._makeerrorresult(_(
3d0e2cd86e05 wireproto: use CBOR for command requests Gregory Szorc <gregory.szorc@gmail.com> parents: 37291 diff changeset	860 'received unexpected frame type: %d') % frame.typeid)
37055 8c3c47362934 wireproto: implement basic frame reading and processing Gregory Szorc <gregory.szorc@gmail.com> parents: 37054 diff changeset	861
37064 884a0c1604ad wireproto: define attr-based classes for representing frames Gregory Szorc <gregory.szorc@gmail.com> parents: 37063 diff changeset	862 def _handlecommanddataframe(self, frame, entry):
884a0c1604ad wireproto: define attr-based classes for representing frames Gregory Szorc <gregory.szorc@gmail.com> parents: 37063 diff changeset	863 assert frame.typeid == FRAME_TYPE_COMMAND_DATA
37055 8c3c47362934 wireproto: implement basic frame reading and processing Gregory Szorc <gregory.szorc@gmail.com> parents: 37054 diff changeset	864
8c3c47362934 wireproto: implement basic frame reading and processing Gregory Szorc <gregory.szorc@gmail.com> parents: 37054 diff changeset	865 # TODO support streaming data instead of buffering it.
37064 884a0c1604ad wireproto: define attr-based classes for representing frames Gregory Szorc <gregory.szorc@gmail.com> parents: 37063 diff changeset	866 entry['data'].write(frame.payload)
37055 8c3c47362934 wireproto: implement basic frame reading and processing Gregory Szorc <gregory.szorc@gmail.com> parents: 37054 diff changeset	867
37064 884a0c1604ad wireproto: define attr-based classes for representing frames Gregory Szorc <gregory.szorc@gmail.com> parents: 37063 diff changeset	868 if frame.flags & FLAG_COMMAND_DATA_CONTINUATION:
37055 8c3c47362934 wireproto: implement basic frame reading and processing Gregory Szorc <gregory.szorc@gmail.com> parents: 37054 diff changeset	869 return self._makewantframeresult()
37064 884a0c1604ad wireproto: define attr-based classes for representing frames Gregory Szorc <gregory.szorc@gmail.com> parents: 37063 diff changeset	870 elif frame.flags & FLAG_COMMAND_DATA_EOS:
37061 c5e9c3b47366 wireproto: support for receiving multiple requests Gregory Szorc <gregory.szorc@gmail.com> parents: 37060 diff changeset	871 entry['data'].seek(0)
37064 884a0c1604ad wireproto: define attr-based classes for representing frames Gregory Szorc <gregory.szorc@gmail.com> parents: 37063 diff changeset	872 return self._makeruncommandresult(frame.requestid)
37055 8c3c47362934 wireproto: implement basic frame reading and processing Gregory Szorc <gregory.szorc@gmail.com> parents: 37054 diff changeset	873 else:
8c3c47362934 wireproto: implement basic frame reading and processing Gregory Szorc <gregory.szorc@gmail.com> parents: 37054 diff changeset	874 self._state = 'errored'
8c3c47362934 wireproto: implement basic frame reading and processing Gregory Szorc <gregory.szorc@gmail.com> parents: 37054 diff changeset	875 return self._makeerrorresult(_('command data frame without '
8c3c47362934 wireproto: implement basic frame reading and processing Gregory Szorc <gregory.szorc@gmail.com> parents: 37054 diff changeset	876 'flags'))
8c3c47362934 wireproto: implement basic frame reading and processing Gregory Szorc <gregory.szorc@gmail.com> parents: 37054 diff changeset	877
37064 884a0c1604ad wireproto: define attr-based classes for representing frames Gregory Szorc <gregory.szorc@gmail.com> parents: 37063 diff changeset	878 def _onframeerrored(self, frame):
37055 8c3c47362934 wireproto: implement basic frame reading and processing Gregory Szorc <gregory.szorc@gmail.com> parents: 37054 diff changeset	879 return self._makeerrorresult(_('server already errored'))
37543 01361be9e2dc wireproto: introduce a reactor for client-side state Gregory Szorc <gregory.szorc@gmail.com> parents: 37485 diff changeset	880
01361be9e2dc wireproto: introduce a reactor for client-side state Gregory Szorc <gregory.szorc@gmail.com> parents: 37485 diff changeset	881 class commandrequest(object):
01361be9e2dc wireproto: introduce a reactor for client-side state Gregory Szorc <gregory.szorc@gmail.com> parents: 37485 diff changeset	882 """Represents a request to run a command."""
01361be9e2dc wireproto: introduce a reactor for client-side state Gregory Szorc <gregory.szorc@gmail.com> parents: 37485 diff changeset	883
01361be9e2dc wireproto: introduce a reactor for client-side state Gregory Szorc <gregory.szorc@gmail.com> parents: 37485 diff changeset	884 def __init__(self, requestid, name, args, datafh=None):
01361be9e2dc wireproto: introduce a reactor for client-side state Gregory Szorc <gregory.szorc@gmail.com> parents: 37485 diff changeset	885 self.requestid = requestid
01361be9e2dc wireproto: introduce a reactor for client-side state Gregory Szorc <gregory.szorc@gmail.com> parents: 37485 diff changeset	886 self.name = name
01361be9e2dc wireproto: introduce a reactor for client-side state Gregory Szorc <gregory.szorc@gmail.com> parents: 37485 diff changeset	887 self.args = args
01361be9e2dc wireproto: introduce a reactor for client-side state Gregory Szorc <gregory.szorc@gmail.com> parents: 37485 diff changeset	888 self.datafh = datafh
01361be9e2dc wireproto: introduce a reactor for client-side state Gregory Szorc <gregory.szorc@gmail.com> parents: 37485 diff changeset	889 self.state = 'pending'
01361be9e2dc wireproto: introduce a reactor for client-side state Gregory Szorc <gregory.szorc@gmail.com> parents: 37485 diff changeset	890
01361be9e2dc wireproto: introduce a reactor for client-side state Gregory Szorc <gregory.szorc@gmail.com> parents: 37485 diff changeset	891 class clientreactor(object):
01361be9e2dc wireproto: introduce a reactor for client-side state Gregory Szorc <gregory.szorc@gmail.com> parents: 37485 diff changeset	892 """Holds state of a client issuing frame-based protocol requests.
01361be9e2dc wireproto: introduce a reactor for client-side state Gregory Szorc <gregory.szorc@gmail.com> parents: 37485 diff changeset	893
01361be9e2dc wireproto: introduce a reactor for client-side state Gregory Szorc <gregory.szorc@gmail.com> parents: 37485 diff changeset	894 This is like ``serverreactor`` but for client-side state.
01361be9e2dc wireproto: introduce a reactor for client-side state Gregory Szorc <gregory.szorc@gmail.com> parents: 37485 diff changeset	895
01361be9e2dc wireproto: introduce a reactor for client-side state Gregory Szorc <gregory.szorc@gmail.com> parents: 37485 diff changeset	896 Each instance is bound to the lifetime of a connection. For persistent
01361be9e2dc wireproto: introduce a reactor for client-side state Gregory Szorc <gregory.szorc@gmail.com> parents: 37485 diff changeset	897 connection transports using e.g. TCP sockets and speaking the raw
01361be9e2dc wireproto: introduce a reactor for client-side state Gregory Szorc <gregory.szorc@gmail.com> parents: 37485 diff changeset	898 framing protocol, there will be a single instance for the lifetime of
01361be9e2dc wireproto: introduce a reactor for client-side state Gregory Szorc <gregory.szorc@gmail.com> parents: 37485 diff changeset	899 the TCP socket. For transports where there are multiple discrete
01361be9e2dc wireproto: introduce a reactor for client-side state Gregory Szorc <gregory.szorc@gmail.com> parents: 37485 diff changeset	900 interactions (say tunneled within in HTTP request), there will be a
01361be9e2dc wireproto: introduce a reactor for client-side state Gregory Szorc <gregory.szorc@gmail.com> parents: 37485 diff changeset	901 separate instance for each distinct interaction.
01361be9e2dc wireproto: introduce a reactor for client-side state Gregory Szorc <gregory.szorc@gmail.com> parents: 37485 diff changeset	902 """
01361be9e2dc wireproto: introduce a reactor for client-side state Gregory Szorc <gregory.szorc@gmail.com> parents: 37485 diff changeset	903 def __init__(self, hasmultiplesend=False, buffersends=True):
01361be9e2dc wireproto: introduce a reactor for client-side state Gregory Szorc <gregory.szorc@gmail.com> parents: 37485 diff changeset	904 """Create a new instance.
01361be9e2dc wireproto: introduce a reactor for client-side state Gregory Szorc <gregory.szorc@gmail.com> parents: 37485 diff changeset	905
01361be9e2dc wireproto: introduce a reactor for client-side state Gregory Szorc <gregory.szorc@gmail.com> parents: 37485 diff changeset	906 ``hasmultiplesend`` indicates whether multiple sends are supported
01361be9e2dc wireproto: introduce a reactor for client-side state Gregory Szorc <gregory.szorc@gmail.com> parents: 37485 diff changeset	907 by the transport. When True, it is possible to send commands immediately
01361be9e2dc wireproto: introduce a reactor for client-side state Gregory Szorc <gregory.szorc@gmail.com> parents: 37485 diff changeset	908 instead of buffering until the caller signals an intent to finish a
01361be9e2dc wireproto: introduce a reactor for client-side state Gregory Szorc <gregory.szorc@gmail.com> parents: 37485 diff changeset	909 send operation.
01361be9e2dc wireproto: introduce a reactor for client-side state Gregory Szorc <gregory.szorc@gmail.com> parents: 37485 diff changeset	910
01361be9e2dc wireproto: introduce a reactor for client-side state Gregory Szorc <gregory.szorc@gmail.com> parents: 37485 diff changeset	911 ``buffercommands`` indicates whether sends should be buffered until the
01361be9e2dc wireproto: introduce a reactor for client-side state Gregory Szorc <gregory.szorc@gmail.com> parents: 37485 diff changeset	912 last request has been issued.
01361be9e2dc wireproto: introduce a reactor for client-side state Gregory Szorc <gregory.szorc@gmail.com> parents: 37485 diff changeset	913 """
01361be9e2dc wireproto: introduce a reactor for client-side state Gregory Szorc <gregory.szorc@gmail.com> parents: 37485 diff changeset	914 self._hasmultiplesend = hasmultiplesend
01361be9e2dc wireproto: introduce a reactor for client-side state Gregory Szorc <gregory.szorc@gmail.com> parents: 37485 diff changeset	915 self._buffersends = buffersends
01361be9e2dc wireproto: introduce a reactor for client-side state Gregory Szorc <gregory.szorc@gmail.com> parents: 37485 diff changeset	916
01361be9e2dc wireproto: introduce a reactor for client-side state Gregory Szorc <gregory.szorc@gmail.com> parents: 37485 diff changeset	917 self._canissuecommands = True
01361be9e2dc wireproto: introduce a reactor for client-side state Gregory Szorc <gregory.szorc@gmail.com> parents: 37485 diff changeset	918 self._cansend = True
01361be9e2dc wireproto: introduce a reactor for client-side state Gregory Szorc <gregory.szorc@gmail.com> parents: 37485 diff changeset	919
01361be9e2dc wireproto: introduce a reactor for client-side state Gregory Szorc <gregory.szorc@gmail.com> parents: 37485 diff changeset	920 self._nextrequestid = 1
01361be9e2dc wireproto: introduce a reactor for client-side state Gregory Szorc <gregory.szorc@gmail.com> parents: 37485 diff changeset	921 # We only support a single outgoing stream for now.
01361be9e2dc wireproto: introduce a reactor for client-side state Gregory Szorc <gregory.szorc@gmail.com> parents: 37485 diff changeset	922 self._outgoingstream = stream(1)
01361be9e2dc wireproto: introduce a reactor for client-side state Gregory Szorc <gregory.szorc@gmail.com> parents: 37485 diff changeset	923 self._pendingrequests = collections.deque()
01361be9e2dc wireproto: introduce a reactor for client-side state Gregory Szorc <gregory.szorc@gmail.com> parents: 37485 diff changeset	924 self._activerequests = {}
37544 55b5ba8d4e68 wireproto: client reactor support for receiving frames Gregory Szorc <gregory.szorc@gmail.com> parents: 37543 diff changeset	925 self._incomingstreams = {}
37543 01361be9e2dc wireproto: introduce a reactor for client-side state Gregory Szorc <gregory.szorc@gmail.com> parents: 37485 diff changeset	926
01361be9e2dc wireproto: introduce a reactor for client-side state Gregory Szorc <gregory.szorc@gmail.com> parents: 37485 diff changeset	927 def callcommand(self, name, args, datafh=None):
01361be9e2dc wireproto: introduce a reactor for client-side state Gregory Szorc <gregory.szorc@gmail.com> parents: 37485 diff changeset	928 """Request that a command be executed.
01361be9e2dc wireproto: introduce a reactor for client-side state Gregory Szorc <gregory.szorc@gmail.com> parents: 37485 diff changeset	929
01361be9e2dc wireproto: introduce a reactor for client-side state Gregory Szorc <gregory.szorc@gmail.com> parents: 37485 diff changeset	930 Receives the command name, a dict of arguments to pass to the command,
01361be9e2dc wireproto: introduce a reactor for client-side state Gregory Szorc <gregory.szorc@gmail.com> parents: 37485 diff changeset	931 and an optional file object containing the raw data for the command.
01361be9e2dc wireproto: introduce a reactor for client-side state Gregory Szorc <gregory.szorc@gmail.com> parents: 37485 diff changeset	932
01361be9e2dc wireproto: introduce a reactor for client-side state Gregory Szorc <gregory.szorc@gmail.com> parents: 37485 diff changeset	933 Returns a 3-tuple of (request, action, action data).
01361be9e2dc wireproto: introduce a reactor for client-side state Gregory Szorc <gregory.szorc@gmail.com> parents: 37485 diff changeset	934 """
01361be9e2dc wireproto: introduce a reactor for client-side state Gregory Szorc <gregory.szorc@gmail.com> parents: 37485 diff changeset	935 if not self._canissuecommands:
01361be9e2dc wireproto: introduce a reactor for client-side state Gregory Szorc <gregory.szorc@gmail.com> parents: 37485 diff changeset	936 raise error.ProgrammingError('cannot issue new commands')
01361be9e2dc wireproto: introduce a reactor for client-side state Gregory Szorc <gregory.szorc@gmail.com> parents: 37485 diff changeset	937
01361be9e2dc wireproto: introduce a reactor for client-side state Gregory Szorc <gregory.szorc@gmail.com> parents: 37485 diff changeset	938 requestid = self._nextrequestid
01361be9e2dc wireproto: introduce a reactor for client-side state Gregory Szorc <gregory.szorc@gmail.com> parents: 37485 diff changeset	939 self._nextrequestid += 2
01361be9e2dc wireproto: introduce a reactor for client-side state Gregory Szorc <gregory.szorc@gmail.com> parents: 37485 diff changeset	940
01361be9e2dc wireproto: introduce a reactor for client-side state Gregory Szorc <gregory.szorc@gmail.com> parents: 37485 diff changeset	941 request = commandrequest(requestid, name, args, datafh=datafh)
01361be9e2dc wireproto: introduce a reactor for client-side state Gregory Szorc <gregory.szorc@gmail.com> parents: 37485 diff changeset	942
01361be9e2dc wireproto: introduce a reactor for client-side state Gregory Szorc <gregory.szorc@gmail.com> parents: 37485 diff changeset	943 if self._buffersends:
01361be9e2dc wireproto: introduce a reactor for client-side state Gregory Szorc <gregory.szorc@gmail.com> parents: 37485 diff changeset	944 self._pendingrequests.append(request)
01361be9e2dc wireproto: introduce a reactor for client-side state Gregory Szorc <gregory.szorc@gmail.com> parents: 37485 diff changeset	945 return request, 'noop', {}
01361be9e2dc wireproto: introduce a reactor for client-side state Gregory Szorc <gregory.szorc@gmail.com> parents: 37485 diff changeset	946 else:
01361be9e2dc wireproto: introduce a reactor for client-side state Gregory Szorc <gregory.szorc@gmail.com> parents: 37485 diff changeset	947 if not self._cansend:
01361be9e2dc wireproto: introduce a reactor for client-side state Gregory Szorc <gregory.szorc@gmail.com> parents: 37485 diff changeset	948 raise error.ProgrammingError('sends cannot be performed on '
01361be9e2dc wireproto: introduce a reactor for client-side state Gregory Szorc <gregory.szorc@gmail.com> parents: 37485 diff changeset	949 'this instance')
01361be9e2dc wireproto: introduce a reactor for client-side state Gregory Szorc <gregory.szorc@gmail.com> parents: 37485 diff changeset	950
01361be9e2dc wireproto: introduce a reactor for client-side state Gregory Szorc <gregory.szorc@gmail.com> parents: 37485 diff changeset	951 if not self._hasmultiplesend:
01361be9e2dc wireproto: introduce a reactor for client-side state Gregory Szorc <gregory.szorc@gmail.com> parents: 37485 diff changeset	952 self._cansend = False
01361be9e2dc wireproto: introduce a reactor for client-side state Gregory Szorc <gregory.szorc@gmail.com> parents: 37485 diff changeset	953 self._canissuecommands = False
01361be9e2dc wireproto: introduce a reactor for client-side state Gregory Szorc <gregory.szorc@gmail.com> parents: 37485 diff changeset	954
01361be9e2dc wireproto: introduce a reactor for client-side state Gregory Szorc <gregory.szorc@gmail.com> parents: 37485 diff changeset	955 return request, 'sendframes', {
01361be9e2dc wireproto: introduce a reactor for client-side state Gregory Szorc <gregory.szorc@gmail.com> parents: 37485 diff changeset	956 'framegen': self._makecommandframes(request),
01361be9e2dc wireproto: introduce a reactor for client-side state Gregory Szorc <gregory.szorc@gmail.com> parents: 37485 diff changeset	957 }
01361be9e2dc wireproto: introduce a reactor for client-side state Gregory Szorc <gregory.szorc@gmail.com> parents: 37485 diff changeset	958
01361be9e2dc wireproto: introduce a reactor for client-side state Gregory Szorc <gregory.szorc@gmail.com> parents: 37485 diff changeset	959 def flushcommands(self):
01361be9e2dc wireproto: introduce a reactor for client-side state Gregory Szorc <gregory.szorc@gmail.com> parents: 37485 diff changeset	960 """Request that all queued commands be sent.
01361be9e2dc wireproto: introduce a reactor for client-side state Gregory Szorc <gregory.szorc@gmail.com> parents: 37485 diff changeset	961
01361be9e2dc wireproto: introduce a reactor for client-side state Gregory Szorc <gregory.szorc@gmail.com> parents: 37485 diff changeset	962 If any commands are buffered, this will instruct the caller to send
01361be9e2dc wireproto: introduce a reactor for client-side state Gregory Szorc <gregory.szorc@gmail.com> parents: 37485 diff changeset	963 them over the wire. If no commands are buffered it instructs the client
01361be9e2dc wireproto: introduce a reactor for client-side state Gregory Szorc <gregory.szorc@gmail.com> parents: 37485 diff changeset	964 to no-op.
01361be9e2dc wireproto: introduce a reactor for client-side state Gregory Szorc <gregory.szorc@gmail.com> parents: 37485 diff changeset	965
01361be9e2dc wireproto: introduce a reactor for client-side state Gregory Szorc <gregory.szorc@gmail.com> parents: 37485 diff changeset	966 If instances aren't configured for multiple sends, no new command
01361be9e2dc wireproto: introduce a reactor for client-side state Gregory Szorc <gregory.szorc@gmail.com> parents: 37485 diff changeset	967 requests are allowed after this is called.
01361be9e2dc wireproto: introduce a reactor for client-side state Gregory Szorc <gregory.szorc@gmail.com> parents: 37485 diff changeset	968 """
01361be9e2dc wireproto: introduce a reactor for client-side state Gregory Szorc <gregory.szorc@gmail.com> parents: 37485 diff changeset	969 if not self._pendingrequests:
01361be9e2dc wireproto: introduce a reactor for client-side state Gregory Szorc <gregory.szorc@gmail.com> parents: 37485 diff changeset	970 return 'noop', {}
01361be9e2dc wireproto: introduce a reactor for client-side state Gregory Szorc <gregory.szorc@gmail.com> parents: 37485 diff changeset	971
01361be9e2dc wireproto: introduce a reactor for client-side state Gregory Szorc <gregory.szorc@gmail.com> parents: 37485 diff changeset	972 if not self._cansend:
01361be9e2dc wireproto: introduce a reactor for client-side state Gregory Szorc <gregory.szorc@gmail.com> parents: 37485 diff changeset	973 raise error.ProgrammingError('sends cannot be performed on this '
01361be9e2dc wireproto: introduce a reactor for client-side state Gregory Szorc <gregory.szorc@gmail.com> parents: 37485 diff changeset	974 'instance')
01361be9e2dc wireproto: introduce a reactor for client-side state Gregory Szorc <gregory.szorc@gmail.com> parents: 37485 diff changeset	975
01361be9e2dc wireproto: introduce a reactor for client-side state Gregory Szorc <gregory.szorc@gmail.com> parents: 37485 diff changeset	976 # If the instance only allows sending once, mark that we have fired
01361be9e2dc wireproto: introduce a reactor for client-side state Gregory Szorc <gregory.szorc@gmail.com> parents: 37485 diff changeset	977 # our one shot.
01361be9e2dc wireproto: introduce a reactor for client-side state Gregory Szorc <gregory.szorc@gmail.com> parents: 37485 diff changeset	978 if not self._hasmultiplesend:
01361be9e2dc wireproto: introduce a reactor for client-side state Gregory Szorc <gregory.szorc@gmail.com> parents: 37485 diff changeset	979 self._canissuecommands = False
01361be9e2dc wireproto: introduce a reactor for client-side state Gregory Szorc <gregory.szorc@gmail.com> parents: 37485 diff changeset	980 self._cansend = False
01361be9e2dc wireproto: introduce a reactor for client-side state Gregory Szorc <gregory.szorc@gmail.com> parents: 37485 diff changeset	981
01361be9e2dc wireproto: introduce a reactor for client-side state Gregory Szorc <gregory.szorc@gmail.com> parents: 37485 diff changeset	982 def makeframes():
01361be9e2dc wireproto: introduce a reactor for client-side state Gregory Szorc <gregory.szorc@gmail.com> parents: 37485 diff changeset	983 while self._pendingrequests:
01361be9e2dc wireproto: introduce a reactor for client-side state Gregory Szorc <gregory.szorc@gmail.com> parents: 37485 diff changeset	984 request = self._pendingrequests.popleft()
01361be9e2dc wireproto: introduce a reactor for client-side state Gregory Szorc <gregory.szorc@gmail.com> parents: 37485 diff changeset	985 for frame in self._makecommandframes(request):
01361be9e2dc wireproto: introduce a reactor for client-side state Gregory Szorc <gregory.szorc@gmail.com> parents: 37485 diff changeset	986 yield frame
01361be9e2dc wireproto: introduce a reactor for client-side state Gregory Szorc <gregory.szorc@gmail.com> parents: 37485 diff changeset	987
01361be9e2dc wireproto: introduce a reactor for client-side state Gregory Szorc <gregory.szorc@gmail.com> parents: 37485 diff changeset	988 return 'sendframes', {
01361be9e2dc wireproto: introduce a reactor for client-side state Gregory Szorc <gregory.szorc@gmail.com> parents: 37485 diff changeset	989 'framegen': makeframes(),
01361be9e2dc wireproto: introduce a reactor for client-side state Gregory Szorc <gregory.szorc@gmail.com> parents: 37485 diff changeset	990 }
01361be9e2dc wireproto: introduce a reactor for client-side state Gregory Szorc <gregory.szorc@gmail.com> parents: 37485 diff changeset	991
01361be9e2dc wireproto: introduce a reactor for client-side state Gregory Szorc <gregory.szorc@gmail.com> parents: 37485 diff changeset	992 def _makecommandframes(self, request):
01361be9e2dc wireproto: introduce a reactor for client-side state Gregory Szorc <gregory.szorc@gmail.com> parents: 37485 diff changeset	993 """Emit frames to issue a command request.
01361be9e2dc wireproto: introduce a reactor for client-side state Gregory Szorc <gregory.szorc@gmail.com> parents: 37485 diff changeset	994
01361be9e2dc wireproto: introduce a reactor for client-side state Gregory Szorc <gregory.szorc@gmail.com> parents: 37485 diff changeset	995 As a side-effect, update request accounting to reflect its changed
01361be9e2dc wireproto: introduce a reactor for client-side state Gregory Szorc <gregory.szorc@gmail.com> parents: 37485 diff changeset	996 state.
01361be9e2dc wireproto: introduce a reactor for client-side state Gregory Szorc <gregory.szorc@gmail.com> parents: 37485 diff changeset	997 """
01361be9e2dc wireproto: introduce a reactor for client-side state Gregory Szorc <gregory.szorc@gmail.com> parents: 37485 diff changeset	998 self._activerequests[request.requestid] = request
01361be9e2dc wireproto: introduce a reactor for client-side state Gregory Szorc <gregory.szorc@gmail.com> parents: 37485 diff changeset	999 request.state = 'sending'
01361be9e2dc wireproto: introduce a reactor for client-side state Gregory Szorc <gregory.szorc@gmail.com> parents: 37485 diff changeset	1000
01361be9e2dc wireproto: introduce a reactor for client-side state Gregory Szorc <gregory.szorc@gmail.com> parents: 37485 diff changeset	1001 res = createcommandframes(self._outgoingstream,
01361be9e2dc wireproto: introduce a reactor for client-side state Gregory Szorc <gregory.szorc@gmail.com> parents: 37485 diff changeset	1002 request.requestid,
01361be9e2dc wireproto: introduce a reactor for client-side state Gregory Szorc <gregory.szorc@gmail.com> parents: 37485 diff changeset	1003 request.name,
01361be9e2dc wireproto: introduce a reactor for client-side state Gregory Szorc <gregory.szorc@gmail.com> parents: 37485 diff changeset	1004 request.args,
01361be9e2dc wireproto: introduce a reactor for client-side state Gregory Szorc <gregory.szorc@gmail.com> parents: 37485 diff changeset	1005 request.datafh)
01361be9e2dc wireproto: introduce a reactor for client-side state Gregory Szorc <gregory.szorc@gmail.com> parents: 37485 diff changeset	1006
01361be9e2dc wireproto: introduce a reactor for client-side state Gregory Szorc <gregory.szorc@gmail.com> parents: 37485 diff changeset	1007 for frame in res:
01361be9e2dc wireproto: introduce a reactor for client-side state Gregory Szorc <gregory.szorc@gmail.com> parents: 37485 diff changeset	1008 yield frame
01361be9e2dc wireproto: introduce a reactor for client-side state Gregory Szorc <gregory.szorc@gmail.com> parents: 37485 diff changeset	1009
01361be9e2dc wireproto: introduce a reactor for client-side state Gregory Szorc <gregory.szorc@gmail.com> parents: 37485 diff changeset	1010 request.state = 'sent'
37544 55b5ba8d4e68 wireproto: client reactor support for receiving frames Gregory Szorc <gregory.szorc@gmail.com> parents: 37543 diff changeset	1011
55b5ba8d4e68 wireproto: client reactor support for receiving frames Gregory Szorc <gregory.szorc@gmail.com> parents: 37543 diff changeset	1012 def onframerecv(self, frame):
55b5ba8d4e68 wireproto: client reactor support for receiving frames Gregory Szorc <gregory.szorc@gmail.com> parents: 37543 diff changeset	1013 """Process a frame that has been received off the wire.
55b5ba8d4e68 wireproto: client reactor support for receiving frames Gregory Szorc <gregory.szorc@gmail.com> parents: 37543 diff changeset	1014
55b5ba8d4e68 wireproto: client reactor support for receiving frames Gregory Szorc <gregory.szorc@gmail.com> parents: 37543 diff changeset	1015 Returns a 2-tuple of (action, meta) describing further action the
55b5ba8d4e68 wireproto: client reactor support for receiving frames Gregory Szorc <gregory.szorc@gmail.com> parents: 37543 diff changeset	1016 caller needs to take as a result of receiving this frame.
55b5ba8d4e68 wireproto: client reactor support for receiving frames Gregory Szorc <gregory.szorc@gmail.com> parents: 37543 diff changeset	1017 """
55b5ba8d4e68 wireproto: client reactor support for receiving frames Gregory Szorc <gregory.szorc@gmail.com> parents: 37543 diff changeset	1018 if frame.streamid % 2:
55b5ba8d4e68 wireproto: client reactor support for receiving frames Gregory Szorc <gregory.szorc@gmail.com> parents: 37543 diff changeset	1019 return 'error', {
55b5ba8d4e68 wireproto: client reactor support for receiving frames Gregory Szorc <gregory.szorc@gmail.com> parents: 37543 diff changeset	1020 'message': (
55b5ba8d4e68 wireproto: client reactor support for receiving frames Gregory Szorc <gregory.szorc@gmail.com> parents: 37543 diff changeset	1021 _('received frame with odd numbered stream ID: %d') %
55b5ba8d4e68 wireproto: client reactor support for receiving frames Gregory Szorc <gregory.szorc@gmail.com> parents: 37543 diff changeset	1022 frame.streamid),
55b5ba8d4e68 wireproto: client reactor support for receiving frames Gregory Szorc <gregory.szorc@gmail.com> parents: 37543 diff changeset	1023 }
55b5ba8d4e68 wireproto: client reactor support for receiving frames Gregory Szorc <gregory.szorc@gmail.com> parents: 37543 diff changeset	1024
55b5ba8d4e68 wireproto: client reactor support for receiving frames Gregory Szorc <gregory.szorc@gmail.com> parents: 37543 diff changeset	1025 if frame.streamid not in self._incomingstreams:
55b5ba8d4e68 wireproto: client reactor support for receiving frames Gregory Szorc <gregory.szorc@gmail.com> parents: 37543 diff changeset	1026 if not frame.streamflags & STREAM_FLAG_BEGIN_STREAM:
55b5ba8d4e68 wireproto: client reactor support for receiving frames Gregory Szorc <gregory.szorc@gmail.com> parents: 37543 diff changeset	1027 return 'error', {
55b5ba8d4e68 wireproto: client reactor support for receiving frames Gregory Szorc <gregory.szorc@gmail.com> parents: 37543 diff changeset	1028 'message': _('received frame on unknown stream '
55b5ba8d4e68 wireproto: client reactor support for receiving frames Gregory Szorc <gregory.szorc@gmail.com> parents: 37543 diff changeset	1029 'without beginning of stream flag set'),
55b5ba8d4e68 wireproto: client reactor support for receiving frames Gregory Szorc <gregory.szorc@gmail.com> parents: 37543 diff changeset	1030 }
55b5ba8d4e68 wireproto: client reactor support for receiving frames Gregory Szorc <gregory.szorc@gmail.com> parents: 37543 diff changeset	1031
55b5ba8d4e68 wireproto: client reactor support for receiving frames Gregory Szorc <gregory.szorc@gmail.com> parents: 37543 diff changeset	1032 if frame.streamflags & STREAM_FLAG_ENCODING_APPLIED:
55b5ba8d4e68 wireproto: client reactor support for receiving frames Gregory Szorc <gregory.szorc@gmail.com> parents: 37543 diff changeset	1033 raise error.ProgrammingError('support for decoding stream '
55b5ba8d4e68 wireproto: client reactor support for receiving frames Gregory Szorc <gregory.szorc@gmail.com> parents: 37543 diff changeset	1034 'payloads not yet implemneted')
55b5ba8d4e68 wireproto: client reactor support for receiving frames Gregory Szorc <gregory.szorc@gmail.com> parents: 37543 diff changeset	1035
55b5ba8d4e68 wireproto: client reactor support for receiving frames Gregory Szorc <gregory.szorc@gmail.com> parents: 37543 diff changeset	1036 if frame.streamflags & STREAM_FLAG_END_STREAM:
55b5ba8d4e68 wireproto: client reactor support for receiving frames Gregory Szorc <gregory.szorc@gmail.com> parents: 37543 diff changeset	1037 del self._incomingstreams[frame.streamid]
55b5ba8d4e68 wireproto: client reactor support for receiving frames Gregory Szorc <gregory.szorc@gmail.com> parents: 37543 diff changeset	1038
55b5ba8d4e68 wireproto: client reactor support for receiving frames Gregory Szorc <gregory.szorc@gmail.com> parents: 37543 diff changeset	1039 if frame.requestid not in self._activerequests:
55b5ba8d4e68 wireproto: client reactor support for receiving frames Gregory Szorc <gregory.szorc@gmail.com> parents: 37543 diff changeset	1040 return 'error', {
55b5ba8d4e68 wireproto: client reactor support for receiving frames Gregory Szorc <gregory.szorc@gmail.com> parents: 37543 diff changeset	1041 'message': (_('received frame for inactive request ID: %d') %
55b5ba8d4e68 wireproto: client reactor support for receiving frames Gregory Szorc <gregory.szorc@gmail.com> parents: 37543 diff changeset	1042 frame.requestid),
55b5ba8d4e68 wireproto: client reactor support for receiving frames Gregory Szorc <gregory.szorc@gmail.com> parents: 37543 diff changeset	1043 }
55b5ba8d4e68 wireproto: client reactor support for receiving frames Gregory Szorc <gregory.szorc@gmail.com> parents: 37543 diff changeset	1044
55b5ba8d4e68 wireproto: client reactor support for receiving frames Gregory Szorc <gregory.szorc@gmail.com> parents: 37543 diff changeset	1045 request = self._activerequests[frame.requestid]
55b5ba8d4e68 wireproto: client reactor support for receiving frames Gregory Szorc <gregory.szorc@gmail.com> parents: 37543 diff changeset	1046 request.state = 'receiving'
55b5ba8d4e68 wireproto: client reactor support for receiving frames Gregory Szorc <gregory.szorc@gmail.com> parents: 37543 diff changeset	1047
55b5ba8d4e68 wireproto: client reactor support for receiving frames Gregory Szorc <gregory.szorc@gmail.com> parents: 37543 diff changeset	1048 handlers = {
55b5ba8d4e68 wireproto: client reactor support for receiving frames Gregory Szorc <gregory.szorc@gmail.com> parents: 37543 diff changeset	1049 FRAME_TYPE_BYTES_RESPONSE: self._onbytesresponseframe,
55b5ba8d4e68 wireproto: client reactor support for receiving frames Gregory Szorc <gregory.szorc@gmail.com> parents: 37543 diff changeset	1050 }
55b5ba8d4e68 wireproto: client reactor support for receiving frames Gregory Szorc <gregory.szorc@gmail.com> parents: 37543 diff changeset	1051
55b5ba8d4e68 wireproto: client reactor support for receiving frames Gregory Szorc <gregory.szorc@gmail.com> parents: 37543 diff changeset	1052 meth = handlers.get(frame.typeid)
55b5ba8d4e68 wireproto: client reactor support for receiving frames Gregory Szorc <gregory.szorc@gmail.com> parents: 37543 diff changeset	1053 if not meth:
55b5ba8d4e68 wireproto: client reactor support for receiving frames Gregory Szorc <gregory.szorc@gmail.com> parents: 37543 diff changeset	1054 raise error.ProgrammingError('unhandled frame type: %d' %
55b5ba8d4e68 wireproto: client reactor support for receiving frames Gregory Szorc <gregory.szorc@gmail.com> parents: 37543 diff changeset	1055 frame.typeid)
55b5ba8d4e68 wireproto: client reactor support for receiving frames Gregory Szorc <gregory.szorc@gmail.com> parents: 37543 diff changeset	1056
55b5ba8d4e68 wireproto: client reactor support for receiving frames Gregory Szorc <gregory.szorc@gmail.com> parents: 37543 diff changeset	1057 return meth(request, frame)
55b5ba8d4e68 wireproto: client reactor support for receiving frames Gregory Szorc <gregory.szorc@gmail.com> parents: 37543 diff changeset	1058
55b5ba8d4e68 wireproto: client reactor support for receiving frames Gregory Szorc <gregory.szorc@gmail.com> parents: 37543 diff changeset	1059 def _onbytesresponseframe(self, request, frame):
55b5ba8d4e68 wireproto: client reactor support for receiving frames Gregory Szorc <gregory.szorc@gmail.com> parents: 37543 diff changeset	1060 if frame.flags & FLAG_BYTES_RESPONSE_EOS:
55b5ba8d4e68 wireproto: client reactor support for receiving frames Gregory Szorc <gregory.szorc@gmail.com> parents: 37543 diff changeset	1061 request.state = 'received'
55b5ba8d4e68 wireproto: client reactor support for receiving frames Gregory Szorc <gregory.szorc@gmail.com> parents: 37543 diff changeset	1062 del self._activerequests[request.requestid]
55b5ba8d4e68 wireproto: client reactor support for receiving frames Gregory Szorc <gregory.szorc@gmail.com> parents: 37543 diff changeset	1063
55b5ba8d4e68 wireproto: client reactor support for receiving frames Gregory Szorc <gregory.szorc@gmail.com> parents: 37543 diff changeset	1064 return 'responsedata', {
55b5ba8d4e68 wireproto: client reactor support for receiving frames Gregory Szorc <gregory.szorc@gmail.com> parents: 37543 diff changeset	1065 'request': request,
55b5ba8d4e68 wireproto: client reactor support for receiving frames Gregory Szorc <gregory.szorc@gmail.com> parents: 37543 diff changeset	1066 'expectmore': frame.flags & FLAG_BYTES_RESPONSE_CONTINUATION,
55b5ba8d4e68 wireproto: client reactor support for receiving frames Gregory Szorc <gregory.szorc@gmail.com> parents: 37543 diff changeset	1067 'eos': frame.flags & FLAG_BYTES_RESPONSE_EOS,
55b5ba8d4e68 wireproto: client reactor support for receiving frames Gregory Szorc <gregory.szorc@gmail.com> parents: 37543 diff changeset	1068 'cbor': frame.flags & FLAG_BYTES_RESPONSE_CBOR,
55b5ba8d4e68 wireproto: client reactor support for receiving frames Gregory Szorc <gregory.szorc@gmail.com> parents: 37543 diff changeset	1069 'data': frame.payload,
55b5ba8d4e68 wireproto: client reactor support for receiving frames Gregory Szorc <gregory.szorc@gmail.com> parents: 37543 diff changeset	1070 }

Mercurial > public > mercurial-scm > hg-stable

annotate mercurial/wireprotoframing.py @ 37544:55b5ba8d4e68