mercurial-scm/hg-stable: mercurial/wireprotoframing.py annotate

annotate mercurial/wireprotoframing.py @ 37725:3ea8323d6f95

wireprotov2: change command response protocol to include a leading map The error handling mechanism for the new wire protocol isn't very well-defined. This commit takes us a step in the right direction by introducing a leading CBOR map for command responses. This map will contain an overall result of the command. Currently, the map indicates whether the command was overall successful or if an error occurred. And if an error occurred, that error is present in the map. There is still a dedicated error frame. My intent is to use that for protocol-level errors and for errors that are encountered after the initial response frame has been sent. This will be clarified in a later commit. Differential Revision: https://phab.mercurial-scm.org/D3385

author	Gregory Szorc <gregory.szorc@gmail.com>
date	Sat, 14 Apr 2018 15:19:36 -0700
parents	deff7cf7eefd
children	0c184ca594bb

rev	line source
37054 40206e227412 wireproto: define and implement protocol for issuing requests Gregory Szorc <gregory.szorc@gmail.com> parents: diff changeset	1 # wireprotoframing.py - unified framing protocol for wire protocol
40206e227412 wireproto: define and implement protocol for issuing requests Gregory Szorc <gregory.szorc@gmail.com> parents: diff changeset	2 #
40206e227412 wireproto: define and implement protocol for issuing requests Gregory Szorc <gregory.szorc@gmail.com> parents: diff changeset	3 # Copyright 2018 Gregory Szorc <gregory.szorc@gmail.com>
40206e227412 wireproto: define and implement protocol for issuing requests Gregory Szorc <gregory.szorc@gmail.com> parents: diff changeset	4 #
40206e227412 wireproto: define and implement protocol for issuing requests Gregory Szorc <gregory.szorc@gmail.com> parents: diff changeset	5 # This software may be used and distributed according to the terms of the
40206e227412 wireproto: define and implement protocol for issuing requests Gregory Szorc <gregory.szorc@gmail.com> parents: diff changeset	6 # GNU General Public License version 2 or any later version.
40206e227412 wireproto: define and implement protocol for issuing requests Gregory Szorc <gregory.szorc@gmail.com> parents: diff changeset	7
40206e227412 wireproto: define and implement protocol for issuing requests Gregory Szorc <gregory.szorc@gmail.com> parents: diff changeset	8 # This file contains functionality to support the unified frame-based wire
40206e227412 wireproto: define and implement protocol for issuing requests Gregory Szorc <gregory.szorc@gmail.com> parents: diff changeset	9 # protocol. For details about the protocol, see
40206e227412 wireproto: define and implement protocol for issuing requests Gregory Szorc <gregory.szorc@gmail.com> parents: diff changeset	10 # `hg help internals.wireprotocol`.
40206e227412 wireproto: define and implement protocol for issuing requests Gregory Szorc <gregory.szorc@gmail.com> parents: diff changeset	11
40206e227412 wireproto: define and implement protocol for issuing requests Gregory Szorc <gregory.szorc@gmail.com> parents: diff changeset	12 from __future__ import absolute_import
40206e227412 wireproto: define and implement protocol for issuing requests Gregory Szorc <gregory.szorc@gmail.com> parents: diff changeset	13
37543 01361be9e2dc wireproto: introduce a reactor for client-side state Gregory Szorc <gregory.szorc@gmail.com> parents: 37485 diff changeset	14 import collections
37054 40206e227412 wireproto: define and implement protocol for issuing requests Gregory Szorc <gregory.szorc@gmail.com> parents: diff changeset	15 import struct
40206e227412 wireproto: define and implement protocol for issuing requests Gregory Szorc <gregory.szorc@gmail.com> parents: diff changeset	16
37055 8c3c47362934 wireproto: implement basic frame reading and processing Gregory Szorc <gregory.szorc@gmail.com> parents: 37054 diff changeset	17 from .i18n import _
37064 884a0c1604ad wireproto: define attr-based classes for representing frames Gregory Szorc <gregory.szorc@gmail.com> parents: 37063 diff changeset	18 from .thirdparty import (
884a0c1604ad wireproto: define attr-based classes for representing frames Gregory Szorc <gregory.szorc@gmail.com> parents: 37063 diff changeset	19 attr,
37290 cc5a040fe150 wireproto: syntax for encoding CBOR into frames Gregory Szorc <gregory.szorc@gmail.com> parents: 37289 diff changeset	20 cbor,
37064 884a0c1604ad wireproto: define attr-based classes for representing frames Gregory Szorc <gregory.szorc@gmail.com> parents: 37063 diff changeset	21 )
37054 40206e227412 wireproto: define and implement protocol for issuing requests Gregory Szorc <gregory.szorc@gmail.com> parents: diff changeset	22 from . import (
37474 d33997123ea5 py3: system-stringify repr(frame) Yuya Nishihara <yuya@tcha.org> parents: 37473 diff changeset	23 encoding,
37055 8c3c47362934 wireproto: implement basic frame reading and processing Gregory Szorc <gregory.szorc@gmail.com> parents: 37054 diff changeset	24 error,
37054 40206e227412 wireproto: define and implement protocol for issuing requests Gregory Szorc <gregory.szorc@gmail.com> parents: diff changeset	25 util,
40206e227412 wireproto: define and implement protocol for issuing requests Gregory Szorc <gregory.szorc@gmail.com> parents: diff changeset	26 )
37087 f0b6fbea00cf stringutil: bulk-replace call sites to point to new module Yuya Nishihara <yuya@tcha.org> parents: 37066 diff changeset	27 from .utils import (
f0b6fbea00cf stringutil: bulk-replace call sites to point to new module Yuya Nishihara <yuya@tcha.org> parents: 37066 diff changeset	28 stringutil,
f0b6fbea00cf stringutil: bulk-replace call sites to point to new module Yuya Nishihara <yuya@tcha.org> parents: 37066 diff changeset	29 )
37054 40206e227412 wireproto: define and implement protocol for issuing requests Gregory Szorc <gregory.szorc@gmail.com> parents: diff changeset	30
37288 9bfcbe4f4745 wireproto: add streams to frame-based protocol Gregory Szorc <gregory.szorc@gmail.com> parents: 37287 diff changeset	31 FRAME_HEADER_SIZE = 8
37054 40206e227412 wireproto: define and implement protocol for issuing requests Gregory Szorc <gregory.szorc@gmail.com> parents: diff changeset	32 DEFAULT_MAX_FRAME_SIZE = 32768
40206e227412 wireproto: define and implement protocol for issuing requests Gregory Szorc <gregory.szorc@gmail.com> parents: diff changeset	33
37288 9bfcbe4f4745 wireproto: add streams to frame-based protocol Gregory Szorc <gregory.szorc@gmail.com> parents: 37287 diff changeset	34 STREAM_FLAG_BEGIN_STREAM = 0x01
9bfcbe4f4745 wireproto: add streams to frame-based protocol Gregory Szorc <gregory.szorc@gmail.com> parents: 37287 diff changeset	35 STREAM_FLAG_END_STREAM = 0x02
9bfcbe4f4745 wireproto: add streams to frame-based protocol Gregory Szorc <gregory.szorc@gmail.com> parents: 37287 diff changeset	36 STREAM_FLAG_ENCODING_APPLIED = 0x04
9bfcbe4f4745 wireproto: add streams to frame-based protocol Gregory Szorc <gregory.szorc@gmail.com> parents: 37287 diff changeset	37
9bfcbe4f4745 wireproto: add streams to frame-based protocol Gregory Szorc <gregory.szorc@gmail.com> parents: 37287 diff changeset	38 STREAM_FLAGS = {
9bfcbe4f4745 wireproto: add streams to frame-based protocol Gregory Szorc <gregory.szorc@gmail.com> parents: 37287 diff changeset	39 b'stream-begin': STREAM_FLAG_BEGIN_STREAM,
9bfcbe4f4745 wireproto: add streams to frame-based protocol Gregory Szorc <gregory.szorc@gmail.com> parents: 37287 diff changeset	40 b'stream-end': STREAM_FLAG_END_STREAM,
9bfcbe4f4745 wireproto: add streams to frame-based protocol Gregory Szorc <gregory.szorc@gmail.com> parents: 37287 diff changeset	41 b'encoded': STREAM_FLAG_ENCODING_APPLIED,
9bfcbe4f4745 wireproto: add streams to frame-based protocol Gregory Szorc <gregory.szorc@gmail.com> parents: 37287 diff changeset	42 }
9bfcbe4f4745 wireproto: add streams to frame-based protocol Gregory Szorc <gregory.szorc@gmail.com> parents: 37287 diff changeset	43
37292 3d0e2cd86e05 wireproto: use CBOR for command requests Gregory Szorc <gregory.szorc@gmail.com> parents: 37291 diff changeset	44 FRAME_TYPE_COMMAND_REQUEST = 0x01
37723 e8fba6d578f0 wireprotov2: change frame type value for command data Gregory Szorc <gregory.szorc@gmail.com> parents: 37722 diff changeset	45 FRAME_TYPE_COMMAND_DATA = 0x02
37724 deff7cf7eefd wireprotov2: change frame type and name for command response Gregory Szorc <gregory.szorc@gmail.com> parents: 37723 diff changeset	46 FRAME_TYPE_COMMAND_RESPONSE = 0x03
37058 61393f888dfe wireproto: define and implement responses in framing protocol Gregory Szorc <gregory.szorc@gmail.com> parents: 37055 diff changeset	47 FRAME_TYPE_ERROR_RESPONSE = 0x05
37063 0a6c5cc09a88 wireproto: define human output side channel frame Gregory Szorc <gregory.szorc@gmail.com> parents: 37061 diff changeset	48 FRAME_TYPE_TEXT_OUTPUT = 0x06
37291 b0041036214e wireproto: define frame to represent progress updates Gregory Szorc <gregory.szorc@gmail.com> parents: 37290 diff changeset	49 FRAME_TYPE_PROGRESS = 0x07
37288 9bfcbe4f4745 wireproto: add streams to frame-based protocol Gregory Szorc <gregory.szorc@gmail.com> parents: 37287 diff changeset	50 FRAME_TYPE_STREAM_SETTINGS = 0x08
37054 40206e227412 wireproto: define and implement protocol for issuing requests Gregory Szorc <gregory.szorc@gmail.com> parents: diff changeset	51
40206e227412 wireproto: define and implement protocol for issuing requests Gregory Szorc <gregory.szorc@gmail.com> parents: diff changeset	52 FRAME_TYPES = {
37292 3d0e2cd86e05 wireproto: use CBOR for command requests Gregory Szorc <gregory.szorc@gmail.com> parents: 37291 diff changeset	53 b'command-request': FRAME_TYPE_COMMAND_REQUEST,
37054 40206e227412 wireproto: define and implement protocol for issuing requests Gregory Szorc <gregory.szorc@gmail.com> parents: diff changeset	54 b'command-data': FRAME_TYPE_COMMAND_DATA,
37724 deff7cf7eefd wireprotov2: change frame type and name for command response Gregory Szorc <gregory.szorc@gmail.com> parents: 37723 diff changeset	55 b'command-response': FRAME_TYPE_COMMAND_RESPONSE,
37058 61393f888dfe wireproto: define and implement responses in framing protocol Gregory Szorc <gregory.szorc@gmail.com> parents: 37055 diff changeset	56 b'error-response': FRAME_TYPE_ERROR_RESPONSE,
37063 0a6c5cc09a88 wireproto: define human output side channel frame Gregory Szorc <gregory.szorc@gmail.com> parents: 37061 diff changeset	57 b'text-output': FRAME_TYPE_TEXT_OUTPUT,
37291 b0041036214e wireproto: define frame to represent progress updates Gregory Szorc <gregory.szorc@gmail.com> parents: 37290 diff changeset	58 b'progress': FRAME_TYPE_PROGRESS,
37288 9bfcbe4f4745 wireproto: add streams to frame-based protocol Gregory Szorc <gregory.szorc@gmail.com> parents: 37287 diff changeset	59 b'stream-settings': FRAME_TYPE_STREAM_SETTINGS,
37054 40206e227412 wireproto: define and implement protocol for issuing requests Gregory Szorc <gregory.szorc@gmail.com> parents: diff changeset	60 }
40206e227412 wireproto: define and implement protocol for issuing requests Gregory Szorc <gregory.szorc@gmail.com> parents: diff changeset	61
37292 3d0e2cd86e05 wireproto: use CBOR for command requests Gregory Szorc <gregory.szorc@gmail.com> parents: 37291 diff changeset	62 FLAG_COMMAND_REQUEST_NEW = 0x01
3d0e2cd86e05 wireproto: use CBOR for command requests Gregory Szorc <gregory.szorc@gmail.com> parents: 37291 diff changeset	63 FLAG_COMMAND_REQUEST_CONTINUATION = 0x02
3d0e2cd86e05 wireproto: use CBOR for command requests Gregory Szorc <gregory.szorc@gmail.com> parents: 37291 diff changeset	64 FLAG_COMMAND_REQUEST_MORE_FRAMES = 0x04
3d0e2cd86e05 wireproto: use CBOR for command requests Gregory Szorc <gregory.szorc@gmail.com> parents: 37291 diff changeset	65 FLAG_COMMAND_REQUEST_EXPECT_DATA = 0x08
37054 40206e227412 wireproto: define and implement protocol for issuing requests Gregory Szorc <gregory.szorc@gmail.com> parents: diff changeset	66
37292 3d0e2cd86e05 wireproto: use CBOR for command requests Gregory Szorc <gregory.szorc@gmail.com> parents: 37291 diff changeset	67 FLAGS_COMMAND_REQUEST = {
3d0e2cd86e05 wireproto: use CBOR for command requests Gregory Szorc <gregory.szorc@gmail.com> parents: 37291 diff changeset	68 b'new': FLAG_COMMAND_REQUEST_NEW,
3d0e2cd86e05 wireproto: use CBOR for command requests Gregory Szorc <gregory.szorc@gmail.com> parents: 37291 diff changeset	69 b'continuation': FLAG_COMMAND_REQUEST_CONTINUATION,
3d0e2cd86e05 wireproto: use CBOR for command requests Gregory Szorc <gregory.szorc@gmail.com> parents: 37291 diff changeset	70 b'more': FLAG_COMMAND_REQUEST_MORE_FRAMES,
3d0e2cd86e05 wireproto: use CBOR for command requests Gregory Szorc <gregory.szorc@gmail.com> parents: 37291 diff changeset	71 b'have-data': FLAG_COMMAND_REQUEST_EXPECT_DATA,
37054 40206e227412 wireproto: define and implement protocol for issuing requests Gregory Szorc <gregory.szorc@gmail.com> parents: diff changeset	72 }
40206e227412 wireproto: define and implement protocol for issuing requests Gregory Szorc <gregory.szorc@gmail.com> parents: diff changeset	73
40206e227412 wireproto: define and implement protocol for issuing requests Gregory Szorc <gregory.szorc@gmail.com> parents: diff changeset	74 FLAG_COMMAND_DATA_CONTINUATION = 0x01
40206e227412 wireproto: define and implement protocol for issuing requests Gregory Szorc <gregory.szorc@gmail.com> parents: diff changeset	75 FLAG_COMMAND_DATA_EOS = 0x02
40206e227412 wireproto: define and implement protocol for issuing requests Gregory Szorc <gregory.szorc@gmail.com> parents: diff changeset	76
40206e227412 wireproto: define and implement protocol for issuing requests Gregory Szorc <gregory.szorc@gmail.com> parents: diff changeset	77 FLAGS_COMMAND_DATA = {
40206e227412 wireproto: define and implement protocol for issuing requests Gregory Szorc <gregory.szorc@gmail.com> parents: diff changeset	78 b'continuation': FLAG_COMMAND_DATA_CONTINUATION,
40206e227412 wireproto: define and implement protocol for issuing requests Gregory Szorc <gregory.szorc@gmail.com> parents: diff changeset	79 b'eos': FLAG_COMMAND_DATA_EOS,
40206e227412 wireproto: define and implement protocol for issuing requests Gregory Szorc <gregory.szorc@gmail.com> parents: diff changeset	80 }
40206e227412 wireproto: define and implement protocol for issuing requests Gregory Szorc <gregory.szorc@gmail.com> parents: diff changeset	81
37724 deff7cf7eefd wireprotov2: change frame type and name for command response Gregory Szorc <gregory.szorc@gmail.com> parents: 37723 diff changeset	82 FLAG_COMMAND_RESPONSE_CONTINUATION = 0x01
deff7cf7eefd wireprotov2: change frame type and name for command response Gregory Szorc <gregory.szorc@gmail.com> parents: 37723 diff changeset	83 FLAG_COMMAND_RESPONSE_EOS = 0x02
37058 61393f888dfe wireproto: define and implement responses in framing protocol Gregory Szorc <gregory.szorc@gmail.com> parents: 37055 diff changeset	84
37724 deff7cf7eefd wireprotov2: change frame type and name for command response Gregory Szorc <gregory.szorc@gmail.com> parents: 37723 diff changeset	85 FLAGS_COMMAND_RESPONSE = {
deff7cf7eefd wireprotov2: change frame type and name for command response Gregory Szorc <gregory.szorc@gmail.com> parents: 37723 diff changeset	86 b'continuation': FLAG_COMMAND_RESPONSE_CONTINUATION,
deff7cf7eefd wireprotov2: change frame type and name for command response Gregory Szorc <gregory.szorc@gmail.com> parents: 37723 diff changeset	87 b'eos': FLAG_COMMAND_RESPONSE_EOS,
37058 61393f888dfe wireproto: define and implement responses in framing protocol Gregory Szorc <gregory.szorc@gmail.com> parents: 37055 diff changeset	88 }
61393f888dfe wireproto: define and implement responses in framing protocol Gregory Szorc <gregory.szorc@gmail.com> parents: 37055 diff changeset	89
61393f888dfe wireproto: define and implement responses in framing protocol Gregory Szorc <gregory.szorc@gmail.com> parents: 37055 diff changeset	90 FLAG_ERROR_RESPONSE_PROTOCOL = 0x01
61393f888dfe wireproto: define and implement responses in framing protocol Gregory Szorc <gregory.szorc@gmail.com> parents: 37055 diff changeset	91 FLAG_ERROR_RESPONSE_APPLICATION = 0x02
61393f888dfe wireproto: define and implement responses in framing protocol Gregory Szorc <gregory.szorc@gmail.com> parents: 37055 diff changeset	92
61393f888dfe wireproto: define and implement responses in framing protocol Gregory Szorc <gregory.szorc@gmail.com> parents: 37055 diff changeset	93 FLAGS_ERROR_RESPONSE = {
61393f888dfe wireproto: define and implement responses in framing protocol Gregory Szorc <gregory.szorc@gmail.com> parents: 37055 diff changeset	94 b'protocol': FLAG_ERROR_RESPONSE_PROTOCOL,
61393f888dfe wireproto: define and implement responses in framing protocol Gregory Szorc <gregory.szorc@gmail.com> parents: 37055 diff changeset	95 b'application': FLAG_ERROR_RESPONSE_APPLICATION,
61393f888dfe wireproto: define and implement responses in framing protocol Gregory Szorc <gregory.szorc@gmail.com> parents: 37055 diff changeset	96 }
61393f888dfe wireproto: define and implement responses in framing protocol Gregory Szorc <gregory.szorc@gmail.com> parents: 37055 diff changeset	97
37054 40206e227412 wireproto: define and implement protocol for issuing requests Gregory Szorc <gregory.szorc@gmail.com> parents: diff changeset	98 # Maps frame types to their available flags.
40206e227412 wireproto: define and implement protocol for issuing requests Gregory Szorc <gregory.szorc@gmail.com> parents: diff changeset	99 FRAME_TYPE_FLAGS = {
37292 3d0e2cd86e05 wireproto: use CBOR for command requests Gregory Szorc <gregory.szorc@gmail.com> parents: 37291 diff changeset	100 FRAME_TYPE_COMMAND_REQUEST: FLAGS_COMMAND_REQUEST,
37054 40206e227412 wireproto: define and implement protocol for issuing requests Gregory Szorc <gregory.szorc@gmail.com> parents: diff changeset	101 FRAME_TYPE_COMMAND_DATA: FLAGS_COMMAND_DATA,
37724 deff7cf7eefd wireprotov2: change frame type and name for command response Gregory Szorc <gregory.szorc@gmail.com> parents: 37723 diff changeset	102 FRAME_TYPE_COMMAND_RESPONSE: FLAGS_COMMAND_RESPONSE,
37058 61393f888dfe wireproto: define and implement responses in framing protocol Gregory Szorc <gregory.szorc@gmail.com> parents: 37055 diff changeset	103 FRAME_TYPE_ERROR_RESPONSE: FLAGS_ERROR_RESPONSE,
37063 0a6c5cc09a88 wireproto: define human output side channel frame Gregory Szorc <gregory.szorc@gmail.com> parents: 37061 diff changeset	104 FRAME_TYPE_TEXT_OUTPUT: {},
37291 b0041036214e wireproto: define frame to represent progress updates Gregory Szorc <gregory.szorc@gmail.com> parents: 37290 diff changeset	105 FRAME_TYPE_PROGRESS: {},
37288 9bfcbe4f4745 wireproto: add streams to frame-based protocol Gregory Szorc <gregory.szorc@gmail.com> parents: 37287 diff changeset	106 FRAME_TYPE_STREAM_SETTINGS: {},
37054 40206e227412 wireproto: define and implement protocol for issuing requests Gregory Szorc <gregory.szorc@gmail.com> parents: diff changeset	107 }
40206e227412 wireproto: define and implement protocol for issuing requests Gregory Szorc <gregory.szorc@gmail.com> parents: diff changeset	108
37292 3d0e2cd86e05 wireproto: use CBOR for command requests Gregory Szorc <gregory.szorc@gmail.com> parents: 37291 diff changeset	109 ARGUMENT_RECORD_HEADER = struct.Struct(r'<HH')
37054 40206e227412 wireproto: define and implement protocol for issuing requests Gregory Szorc <gregory.szorc@gmail.com> parents: diff changeset	110
37298 5ef2da00e935 wireproto: implement custom __repr__ for frame Gregory Szorc <gregory.szorc@gmail.com> parents: 37292 diff changeset	111 def humanflags(mapping, value):
5ef2da00e935 wireproto: implement custom __repr__ for frame Gregory Szorc <gregory.szorc@gmail.com> parents: 37292 diff changeset	112 """Convert a numeric flags value to a human value, using a mapping table."""
37473 7c2c7c749411 wireproto: show unknown id and flags in repr(frame) Yuya Nishihara <yuya@tcha.org> parents: 37472 diff changeset	113 namemap = {v: k for k, v in mapping.iteritems()}
37298 5ef2da00e935 wireproto: implement custom __repr__ for frame Gregory Szorc <gregory.szorc@gmail.com> parents: 37292 diff changeset	114 flags = []
37473 7c2c7c749411 wireproto: show unknown id and flags in repr(frame) Yuya Nishihara <yuya@tcha.org> parents: 37472 diff changeset	115 val = 1
7c2c7c749411 wireproto: show unknown id and flags in repr(frame) Yuya Nishihara <yuya@tcha.org> parents: 37472 diff changeset	116 while value >= val:
37298 5ef2da00e935 wireproto: implement custom __repr__ for frame Gregory Szorc <gregory.szorc@gmail.com> parents: 37292 diff changeset	117 if value & val:
37473 7c2c7c749411 wireproto: show unknown id and flags in repr(frame) Yuya Nishihara <yuya@tcha.org> parents: 37472 diff changeset	118 flags.append(namemap.get(val, '<unknown 0x%02x>' % val))
7c2c7c749411 wireproto: show unknown id and flags in repr(frame) Yuya Nishihara <yuya@tcha.org> parents: 37472 diff changeset	119 val <<= 1
37298 5ef2da00e935 wireproto: implement custom __repr__ for frame Gregory Szorc <gregory.szorc@gmail.com> parents: 37292 diff changeset	120
5ef2da00e935 wireproto: implement custom __repr__ for frame Gregory Szorc <gregory.szorc@gmail.com> parents: 37292 diff changeset	121 return b'\|'.join(flags)
5ef2da00e935 wireproto: implement custom __repr__ for frame Gregory Szorc <gregory.szorc@gmail.com> parents: 37292 diff changeset	122
37064 884a0c1604ad wireproto: define attr-based classes for representing frames Gregory Szorc <gregory.szorc@gmail.com> parents: 37063 diff changeset	123 @attr.s(slots=True)
884a0c1604ad wireproto: define attr-based classes for representing frames Gregory Szorc <gregory.szorc@gmail.com> parents: 37063 diff changeset	124 class frameheader(object):
884a0c1604ad wireproto: define attr-based classes for representing frames Gregory Szorc <gregory.szorc@gmail.com> parents: 37063 diff changeset	125 """Represents the data in a frame header."""
884a0c1604ad wireproto: define attr-based classes for representing frames Gregory Szorc <gregory.szorc@gmail.com> parents: 37063 diff changeset	126
884a0c1604ad wireproto: define attr-based classes for representing frames Gregory Szorc <gregory.szorc@gmail.com> parents: 37063 diff changeset	127 length = attr.ib()
884a0c1604ad wireproto: define attr-based classes for representing frames Gregory Szorc <gregory.szorc@gmail.com> parents: 37063 diff changeset	128 requestid = attr.ib()
37288 9bfcbe4f4745 wireproto: add streams to frame-based protocol Gregory Szorc <gregory.szorc@gmail.com> parents: 37287 diff changeset	129 streamid = attr.ib()
9bfcbe4f4745 wireproto: add streams to frame-based protocol Gregory Szorc <gregory.szorc@gmail.com> parents: 37287 diff changeset	130 streamflags = attr.ib()
37064 884a0c1604ad wireproto: define attr-based classes for representing frames Gregory Szorc <gregory.szorc@gmail.com> parents: 37063 diff changeset	131 typeid = attr.ib()
884a0c1604ad wireproto: define attr-based classes for representing frames Gregory Szorc <gregory.szorc@gmail.com> parents: 37063 diff changeset	132 flags = attr.ib()
884a0c1604ad wireproto: define attr-based classes for representing frames Gregory Szorc <gregory.szorc@gmail.com> parents: 37063 diff changeset	133
37298 5ef2da00e935 wireproto: implement custom __repr__ for frame Gregory Szorc <gregory.szorc@gmail.com> parents: 37292 diff changeset	134 @attr.s(slots=True, repr=False)
37064 884a0c1604ad wireproto: define attr-based classes for representing frames Gregory Szorc <gregory.szorc@gmail.com> parents: 37063 diff changeset	135 class frame(object):
884a0c1604ad wireproto: define attr-based classes for representing frames Gregory Szorc <gregory.szorc@gmail.com> parents: 37063 diff changeset	136 """Represents a parsed frame."""
884a0c1604ad wireproto: define attr-based classes for representing frames Gregory Szorc <gregory.szorc@gmail.com> parents: 37063 diff changeset	137
884a0c1604ad wireproto: define attr-based classes for representing frames Gregory Szorc <gregory.szorc@gmail.com> parents: 37063 diff changeset	138 requestid = attr.ib()
37288 9bfcbe4f4745 wireproto: add streams to frame-based protocol Gregory Szorc <gregory.szorc@gmail.com> parents: 37287 diff changeset	139 streamid = attr.ib()
9bfcbe4f4745 wireproto: add streams to frame-based protocol Gregory Szorc <gregory.szorc@gmail.com> parents: 37287 diff changeset	140 streamflags = attr.ib()
37064 884a0c1604ad wireproto: define attr-based classes for representing frames Gregory Szorc <gregory.szorc@gmail.com> parents: 37063 diff changeset	141 typeid = attr.ib()
884a0c1604ad wireproto: define attr-based classes for representing frames Gregory Szorc <gregory.szorc@gmail.com> parents: 37063 diff changeset	142 flags = attr.ib()
884a0c1604ad wireproto: define attr-based classes for representing frames Gregory Szorc <gregory.szorc@gmail.com> parents: 37063 diff changeset	143 payload = attr.ib()
884a0c1604ad wireproto: define attr-based classes for representing frames Gregory Szorc <gregory.szorc@gmail.com> parents: 37063 diff changeset	144
37474 d33997123ea5 py3: system-stringify repr(frame) Yuya Nishihara <yuya@tcha.org> parents: 37473 diff changeset	145 @encoding.strmethod
37298 5ef2da00e935 wireproto: implement custom __repr__ for frame Gregory Szorc <gregory.szorc@gmail.com> parents: 37292 diff changeset	146 def __repr__(self):
37473 7c2c7c749411 wireproto: show unknown id and flags in repr(frame) Yuya Nishihara <yuya@tcha.org> parents: 37472 diff changeset	147 typename = '<unknown 0x%02x>' % self.typeid
37298 5ef2da00e935 wireproto: implement custom __repr__ for frame Gregory Szorc <gregory.szorc@gmail.com> parents: 37292 diff changeset	148 for name, value in FRAME_TYPES.iteritems():
5ef2da00e935 wireproto: implement custom __repr__ for frame Gregory Szorc <gregory.szorc@gmail.com> parents: 37292 diff changeset	149 if value == self.typeid:
5ef2da00e935 wireproto: implement custom __repr__ for frame Gregory Szorc <gregory.szorc@gmail.com> parents: 37292 diff changeset	150 typename = name
5ef2da00e935 wireproto: implement custom __repr__ for frame Gregory Szorc <gregory.szorc@gmail.com> parents: 37292 diff changeset	151 break
5ef2da00e935 wireproto: implement custom __repr__ for frame Gregory Szorc <gregory.szorc@gmail.com> parents: 37292 diff changeset	152
5ef2da00e935 wireproto: implement custom __repr__ for frame Gregory Szorc <gregory.szorc@gmail.com> parents: 37292 diff changeset	153 return ('frame(size=%d; request=%d; stream=%d; streamflags=%s; '
5ef2da00e935 wireproto: implement custom __repr__ for frame Gregory Szorc <gregory.szorc@gmail.com> parents: 37292 diff changeset	154 'type=%s; flags=%s)' % (
5ef2da00e935 wireproto: implement custom __repr__ for frame Gregory Szorc <gregory.szorc@gmail.com> parents: 37292 diff changeset	155 len(self.payload), self.requestid, self.streamid,
5ef2da00e935 wireproto: implement custom __repr__ for frame Gregory Szorc <gregory.szorc@gmail.com> parents: 37292 diff changeset	156 humanflags(STREAM_FLAGS, self.streamflags), typename,
37472 2f81926c7f89 wireproto: fix repr(frame) to not crash by unknown type id Yuya Nishihara <yuya@tcha.org> parents: 37319 diff changeset	157 humanflags(FRAME_TYPE_FLAGS.get(self.typeid, {}), self.flags)))
37298 5ef2da00e935 wireproto: implement custom __repr__ for frame Gregory Szorc <gregory.szorc@gmail.com> parents: 37292 diff changeset	158
37288 9bfcbe4f4745 wireproto: add streams to frame-based protocol Gregory Szorc <gregory.szorc@gmail.com> parents: 37287 diff changeset	159 def makeframe(requestid, streamid, streamflags, typeid, flags, payload):
37054 40206e227412 wireproto: define and implement protocol for issuing requests Gregory Szorc <gregory.szorc@gmail.com> parents: diff changeset	160 """Assemble a frame into a byte array."""
40206e227412 wireproto: define and implement protocol for issuing requests Gregory Szorc <gregory.szorc@gmail.com> parents: diff changeset	161 # TODO assert size of payload.
40206e227412 wireproto: define and implement protocol for issuing requests Gregory Szorc <gregory.szorc@gmail.com> parents: diff changeset	162 frame = bytearray(FRAME_HEADER_SIZE + len(payload))
40206e227412 wireproto: define and implement protocol for issuing requests Gregory Szorc <gregory.szorc@gmail.com> parents: diff changeset	163
37060 2ec1fb9de638 wireproto: add request IDs to frames Gregory Szorc <gregory.szorc@gmail.com> parents: 37059 diff changeset	164 # 24 bits length
2ec1fb9de638 wireproto: add request IDs to frames Gregory Szorc <gregory.szorc@gmail.com> parents: 37059 diff changeset	165 # 16 bits request id
37288 9bfcbe4f4745 wireproto: add streams to frame-based protocol Gregory Szorc <gregory.szorc@gmail.com> parents: 37287 diff changeset	166 # 8 bits stream id
9bfcbe4f4745 wireproto: add streams to frame-based protocol Gregory Szorc <gregory.szorc@gmail.com> parents: 37287 diff changeset	167 # 8 bits stream flags
37060 2ec1fb9de638 wireproto: add request IDs to frames Gregory Szorc <gregory.szorc@gmail.com> parents: 37059 diff changeset	168 # 4 bits type
2ec1fb9de638 wireproto: add request IDs to frames Gregory Szorc <gregory.szorc@gmail.com> parents: 37059 diff changeset	169 # 4 bits flags
2ec1fb9de638 wireproto: add request IDs to frames Gregory Szorc <gregory.szorc@gmail.com> parents: 37059 diff changeset	170
37054 40206e227412 wireproto: define and implement protocol for issuing requests Gregory Szorc <gregory.szorc@gmail.com> parents: diff changeset	171 l = struct.pack(r'<I', len(payload))
40206e227412 wireproto: define and implement protocol for issuing requests Gregory Szorc <gregory.szorc@gmail.com> parents: diff changeset	172 frame[0:3] = l[0:3]
37288 9bfcbe4f4745 wireproto: add streams to frame-based protocol Gregory Szorc <gregory.szorc@gmail.com> parents: 37287 diff changeset	173 struct.pack_into(r'<HBB', frame, 3, requestid, streamid, streamflags)
9bfcbe4f4745 wireproto: add streams to frame-based protocol Gregory Szorc <gregory.szorc@gmail.com> parents: 37287 diff changeset	174 frame[7] = (typeid << 4) \| flags
9bfcbe4f4745 wireproto: add streams to frame-based protocol Gregory Szorc <gregory.szorc@gmail.com> parents: 37287 diff changeset	175 frame[8:] = payload
37054 40206e227412 wireproto: define and implement protocol for issuing requests Gregory Szorc <gregory.szorc@gmail.com> parents: diff changeset	176
40206e227412 wireproto: define and implement protocol for issuing requests Gregory Szorc <gregory.szorc@gmail.com> parents: diff changeset	177 return frame
40206e227412 wireproto: define and implement protocol for issuing requests Gregory Szorc <gregory.szorc@gmail.com> parents: diff changeset	178
40206e227412 wireproto: define and implement protocol for issuing requests Gregory Szorc <gregory.szorc@gmail.com> parents: diff changeset	179 def makeframefromhumanstring(s):
37060 2ec1fb9de638 wireproto: add request IDs to frames Gregory Szorc <gregory.szorc@gmail.com> parents: 37059 diff changeset	180 """Create a frame from a human readable string
2ec1fb9de638 wireproto: add request IDs to frames Gregory Szorc <gregory.szorc@gmail.com> parents: 37059 diff changeset	181
2ec1fb9de638 wireproto: add request IDs to frames Gregory Szorc <gregory.szorc@gmail.com> parents: 37059 diff changeset	182 Strings have the form:
2ec1fb9de638 wireproto: add request IDs to frames Gregory Szorc <gregory.szorc@gmail.com> parents: 37059 diff changeset	183
37288 9bfcbe4f4745 wireproto: add streams to frame-based protocol Gregory Szorc <gregory.szorc@gmail.com> parents: 37287 diff changeset	184 <request-id> <stream-id> <stream-flags> <type> <flags> <payload>
37054 40206e227412 wireproto: define and implement protocol for issuing requests Gregory Szorc <gregory.szorc@gmail.com> parents: diff changeset	185
40206e227412 wireproto: define and implement protocol for issuing requests Gregory Szorc <gregory.szorc@gmail.com> parents: diff changeset	186 This can be used by user-facing applications and tests for creating
40206e227412 wireproto: define and implement protocol for issuing requests Gregory Szorc <gregory.szorc@gmail.com> parents: diff changeset	187 frames easily without having to type out a bunch of constants.
40206e227412 wireproto: define and implement protocol for issuing requests Gregory Szorc <gregory.szorc@gmail.com> parents: diff changeset	188
37288 9bfcbe4f4745 wireproto: add streams to frame-based protocol Gregory Szorc <gregory.szorc@gmail.com> parents: 37287 diff changeset	189 Request ID and stream IDs are integers.
37060 2ec1fb9de638 wireproto: add request IDs to frames Gregory Szorc <gregory.szorc@gmail.com> parents: 37059 diff changeset	190
37288 9bfcbe4f4745 wireproto: add streams to frame-based protocol Gregory Szorc <gregory.szorc@gmail.com> parents: 37287 diff changeset	191 Stream flags, frame type, and flags can be specified by integer or
9bfcbe4f4745 wireproto: add streams to frame-based protocol Gregory Szorc <gregory.szorc@gmail.com> parents: 37287 diff changeset	192 named constant.
37060 2ec1fb9de638 wireproto: add request IDs to frames Gregory Szorc <gregory.szorc@gmail.com> parents: 37059 diff changeset	193
37054 40206e227412 wireproto: define and implement protocol for issuing requests Gregory Szorc <gregory.szorc@gmail.com> parents: diff changeset	194 Flags can be delimited by `\|` to bitwise OR them together.
37290 cc5a040fe150 wireproto: syntax for encoding CBOR into frames Gregory Szorc <gregory.szorc@gmail.com> parents: 37289 diff changeset	195
cc5a040fe150 wireproto: syntax for encoding CBOR into frames Gregory Szorc <gregory.szorc@gmail.com> parents: 37289 diff changeset	196 If the payload begins with ``cbor:``, the following string will be
37476 e9dea82ea1f3 wireproto: convert python literal to object without using unsafe eval() Yuya Nishihara <yuya@tcha.org> parents: 37474 diff changeset	197 evaluated as Python literal and the resulting object will be fed into
37290 cc5a040fe150 wireproto: syntax for encoding CBOR into frames Gregory Szorc <gregory.szorc@gmail.com> parents: 37289 diff changeset	198 a CBOR encoder. Otherwise, the payload is interpreted as a Python
cc5a040fe150 wireproto: syntax for encoding CBOR into frames Gregory Szorc <gregory.szorc@gmail.com> parents: 37289 diff changeset	199 byte string literal.
37054 40206e227412 wireproto: define and implement protocol for issuing requests Gregory Szorc <gregory.szorc@gmail.com> parents: diff changeset	200 """
37288 9bfcbe4f4745 wireproto: add streams to frame-based protocol Gregory Szorc <gregory.szorc@gmail.com> parents: 37287 diff changeset	201 fields = s.split(b' ', 5)
9bfcbe4f4745 wireproto: add streams to frame-based protocol Gregory Szorc <gregory.szorc@gmail.com> parents: 37287 diff changeset	202 requestid, streamid, streamflags, frametype, frameflags, payload = fields
37060 2ec1fb9de638 wireproto: add request IDs to frames Gregory Szorc <gregory.szorc@gmail.com> parents: 37059 diff changeset	203
2ec1fb9de638 wireproto: add request IDs to frames Gregory Szorc <gregory.szorc@gmail.com> parents: 37059 diff changeset	204 requestid = int(requestid)
37288 9bfcbe4f4745 wireproto: add streams to frame-based protocol Gregory Szorc <gregory.szorc@gmail.com> parents: 37287 diff changeset	205 streamid = int(streamid)
9bfcbe4f4745 wireproto: add streams to frame-based protocol Gregory Szorc <gregory.szorc@gmail.com> parents: 37287 diff changeset	206
9bfcbe4f4745 wireproto: add streams to frame-based protocol Gregory Szorc <gregory.szorc@gmail.com> parents: 37287 diff changeset	207 finalstreamflags = 0
9bfcbe4f4745 wireproto: add streams to frame-based protocol Gregory Szorc <gregory.szorc@gmail.com> parents: 37287 diff changeset	208 for flag in streamflags.split(b'\|'):
9bfcbe4f4745 wireproto: add streams to frame-based protocol Gregory Szorc <gregory.szorc@gmail.com> parents: 37287 diff changeset	209 if flag in STREAM_FLAGS:
9bfcbe4f4745 wireproto: add streams to frame-based protocol Gregory Szorc <gregory.szorc@gmail.com> parents: 37287 diff changeset	210 finalstreamflags \|= STREAM_FLAGS[flag]
9bfcbe4f4745 wireproto: add streams to frame-based protocol Gregory Szorc <gregory.szorc@gmail.com> parents: 37287 diff changeset	211 else:
9bfcbe4f4745 wireproto: add streams to frame-based protocol Gregory Szorc <gregory.szorc@gmail.com> parents: 37287 diff changeset	212 finalstreamflags \|= int(flag)
37054 40206e227412 wireproto: define and implement protocol for issuing requests Gregory Szorc <gregory.szorc@gmail.com> parents: diff changeset	213
40206e227412 wireproto: define and implement protocol for issuing requests Gregory Szorc <gregory.szorc@gmail.com> parents: diff changeset	214 if frametype in FRAME_TYPES:
40206e227412 wireproto: define and implement protocol for issuing requests Gregory Szorc <gregory.szorc@gmail.com> parents: diff changeset	215 frametype = FRAME_TYPES[frametype]
40206e227412 wireproto: define and implement protocol for issuing requests Gregory Szorc <gregory.szorc@gmail.com> parents: diff changeset	216 else:
40206e227412 wireproto: define and implement protocol for issuing requests Gregory Szorc <gregory.szorc@gmail.com> parents: diff changeset	217 frametype = int(frametype)
40206e227412 wireproto: define and implement protocol for issuing requests Gregory Szorc <gregory.szorc@gmail.com> parents: diff changeset	218
40206e227412 wireproto: define and implement protocol for issuing requests Gregory Szorc <gregory.szorc@gmail.com> parents: diff changeset	219 finalflags = 0
40206e227412 wireproto: define and implement protocol for issuing requests Gregory Szorc <gregory.szorc@gmail.com> parents: diff changeset	220 validflags = FRAME_TYPE_FLAGS[frametype]
40206e227412 wireproto: define and implement protocol for issuing requests Gregory Szorc <gregory.szorc@gmail.com> parents: diff changeset	221 for flag in frameflags.split(b'\|'):
40206e227412 wireproto: define and implement protocol for issuing requests Gregory Szorc <gregory.szorc@gmail.com> parents: diff changeset	222 if flag in validflags:
40206e227412 wireproto: define and implement protocol for issuing requests Gregory Szorc <gregory.szorc@gmail.com> parents: diff changeset	223 finalflags \|= validflags[flag]
40206e227412 wireproto: define and implement protocol for issuing requests Gregory Szorc <gregory.szorc@gmail.com> parents: diff changeset	224 else:
40206e227412 wireproto: define and implement protocol for issuing requests Gregory Szorc <gregory.szorc@gmail.com> parents: diff changeset	225 finalflags \|= int(flag)
40206e227412 wireproto: define and implement protocol for issuing requests Gregory Szorc <gregory.szorc@gmail.com> parents: diff changeset	226
37290 cc5a040fe150 wireproto: syntax for encoding CBOR into frames Gregory Szorc <gregory.szorc@gmail.com> parents: 37289 diff changeset	227 if payload.startswith(b'cbor:'):
37476 e9dea82ea1f3 wireproto: convert python literal to object without using unsafe eval() Yuya Nishihara <yuya@tcha.org> parents: 37474 diff changeset	228 payload = cbor.dumps(stringutil.evalpythonliteral(payload[5:]),
e9dea82ea1f3 wireproto: convert python literal to object without using unsafe eval() Yuya Nishihara <yuya@tcha.org> parents: 37474 diff changeset	229 canonical=True)
37290 cc5a040fe150 wireproto: syntax for encoding CBOR into frames Gregory Szorc <gregory.szorc@gmail.com> parents: 37289 diff changeset	230
cc5a040fe150 wireproto: syntax for encoding CBOR into frames Gregory Szorc <gregory.szorc@gmail.com> parents: 37289 diff changeset	231 else:
cc5a040fe150 wireproto: syntax for encoding CBOR into frames Gregory Szorc <gregory.szorc@gmail.com> parents: 37289 diff changeset	232 payload = stringutil.unescapestr(payload)
37054 40206e227412 wireproto: define and implement protocol for issuing requests Gregory Szorc <gregory.szorc@gmail.com> parents: diff changeset	233
37288 9bfcbe4f4745 wireproto: add streams to frame-based protocol Gregory Szorc <gregory.szorc@gmail.com> parents: 37287 diff changeset	234 return makeframe(requestid=requestid, streamid=streamid,
9bfcbe4f4745 wireproto: add streams to frame-based protocol Gregory Szorc <gregory.szorc@gmail.com> parents: 37287 diff changeset	235 streamflags=finalstreamflags, typeid=frametype,
37065 fe4c944f95bb wireproto: use named arguments when passing around frame data Gregory Szorc <gregory.szorc@gmail.com> parents: 37064 diff changeset	236 flags=finalflags, payload=payload)
37054 40206e227412 wireproto: define and implement protocol for issuing requests Gregory Szorc <gregory.szorc@gmail.com> parents: diff changeset	237
37055 8c3c47362934 wireproto: implement basic frame reading and processing Gregory Szorc <gregory.szorc@gmail.com> parents: 37054 diff changeset	238 def parseheader(data):
8c3c47362934 wireproto: implement basic frame reading and processing Gregory Szorc <gregory.szorc@gmail.com> parents: 37054 diff changeset	239 """Parse a unified framing protocol frame header from a buffer.
8c3c47362934 wireproto: implement basic frame reading and processing Gregory Szorc <gregory.szorc@gmail.com> parents: 37054 diff changeset	240
8c3c47362934 wireproto: implement basic frame reading and processing Gregory Szorc <gregory.szorc@gmail.com> parents: 37054 diff changeset	241 The header is expected to be in the buffer at offset 0 and the
8c3c47362934 wireproto: implement basic frame reading and processing Gregory Szorc <gregory.szorc@gmail.com> parents: 37054 diff changeset	242 buffer is expected to be large enough to hold a full header.
8c3c47362934 wireproto: implement basic frame reading and processing Gregory Szorc <gregory.szorc@gmail.com> parents: 37054 diff changeset	243 """
8c3c47362934 wireproto: implement basic frame reading and processing Gregory Szorc <gregory.szorc@gmail.com> parents: 37054 diff changeset	244 # 24 bits payload length (little endian)
37288 9bfcbe4f4745 wireproto: add streams to frame-based protocol Gregory Szorc <gregory.szorc@gmail.com> parents: 37287 diff changeset	245 # 16 bits request ID
9bfcbe4f4745 wireproto: add streams to frame-based protocol Gregory Szorc <gregory.szorc@gmail.com> parents: 37287 diff changeset	246 # 8 bits stream ID
9bfcbe4f4745 wireproto: add streams to frame-based protocol Gregory Szorc <gregory.szorc@gmail.com> parents: 37287 diff changeset	247 # 8 bits stream flags
37055 8c3c47362934 wireproto: implement basic frame reading and processing Gregory Szorc <gregory.szorc@gmail.com> parents: 37054 diff changeset	248 # 4 bits frame type
8c3c47362934 wireproto: implement basic frame reading and processing Gregory Szorc <gregory.szorc@gmail.com> parents: 37054 diff changeset	249 # 4 bits frame flags
8c3c47362934 wireproto: implement basic frame reading and processing Gregory Szorc <gregory.szorc@gmail.com> parents: 37054 diff changeset	250 # ... payload
8c3c47362934 wireproto: implement basic frame reading and processing Gregory Szorc <gregory.szorc@gmail.com> parents: 37054 diff changeset	251 framelength = data[0] + 256 * data[1] + 16384 * data[2]
37288 9bfcbe4f4745 wireproto: add streams to frame-based protocol Gregory Szorc <gregory.szorc@gmail.com> parents: 37287 diff changeset	252 requestid, streamid, streamflags = struct.unpack_from(r'<HBB', data, 3)
9bfcbe4f4745 wireproto: add streams to frame-based protocol Gregory Szorc <gregory.szorc@gmail.com> parents: 37287 diff changeset	253 typeflags = data[7]
37055 8c3c47362934 wireproto: implement basic frame reading and processing Gregory Szorc <gregory.szorc@gmail.com> parents: 37054 diff changeset	254
8c3c47362934 wireproto: implement basic frame reading and processing Gregory Szorc <gregory.szorc@gmail.com> parents: 37054 diff changeset	255 frametype = (typeflags & 0xf0) >> 4
8c3c47362934 wireproto: implement basic frame reading and processing Gregory Szorc <gregory.szorc@gmail.com> parents: 37054 diff changeset	256 frameflags = typeflags & 0x0f
8c3c47362934 wireproto: implement basic frame reading and processing Gregory Szorc <gregory.szorc@gmail.com> parents: 37054 diff changeset	257
37288 9bfcbe4f4745 wireproto: add streams to frame-based protocol Gregory Szorc <gregory.szorc@gmail.com> parents: 37287 diff changeset	258 return frameheader(framelength, requestid, streamid, streamflags,
9bfcbe4f4745 wireproto: add streams to frame-based protocol Gregory Szorc <gregory.szorc@gmail.com> parents: 37287 diff changeset	259 frametype, frameflags)
37055 8c3c47362934 wireproto: implement basic frame reading and processing Gregory Szorc <gregory.szorc@gmail.com> parents: 37054 diff changeset	260
8c3c47362934 wireproto: implement basic frame reading and processing Gregory Szorc <gregory.szorc@gmail.com> parents: 37054 diff changeset	261 def readframe(fh):
8c3c47362934 wireproto: implement basic frame reading and processing Gregory Szorc <gregory.szorc@gmail.com> parents: 37054 diff changeset	262 """Read a unified framing protocol frame from a file object.
8c3c47362934 wireproto: implement basic frame reading and processing Gregory Szorc <gregory.szorc@gmail.com> parents: 37054 diff changeset	263
8c3c47362934 wireproto: implement basic frame reading and processing Gregory Szorc <gregory.szorc@gmail.com> parents: 37054 diff changeset	264 Returns a 3-tuple of (type, flags, payload) for the decoded frame or
8c3c47362934 wireproto: implement basic frame reading and processing Gregory Szorc <gregory.szorc@gmail.com> parents: 37054 diff changeset	265 None if no frame is available. May raise if a malformed frame is
8c3c47362934 wireproto: implement basic frame reading and processing Gregory Szorc <gregory.szorc@gmail.com> parents: 37054 diff changeset	266 seen.
8c3c47362934 wireproto: implement basic frame reading and processing Gregory Szorc <gregory.szorc@gmail.com> parents: 37054 diff changeset	267 """
8c3c47362934 wireproto: implement basic frame reading and processing Gregory Szorc <gregory.szorc@gmail.com> parents: 37054 diff changeset	268 header = bytearray(FRAME_HEADER_SIZE)
8c3c47362934 wireproto: implement basic frame reading and processing Gregory Szorc <gregory.szorc@gmail.com> parents: 37054 diff changeset	269
8c3c47362934 wireproto: implement basic frame reading and processing Gregory Szorc <gregory.szorc@gmail.com> parents: 37054 diff changeset	270 readcount = fh.readinto(header)
8c3c47362934 wireproto: implement basic frame reading and processing Gregory Szorc <gregory.szorc@gmail.com> parents: 37054 diff changeset	271
8c3c47362934 wireproto: implement basic frame reading and processing Gregory Szorc <gregory.szorc@gmail.com> parents: 37054 diff changeset	272 if readcount == 0:
8c3c47362934 wireproto: implement basic frame reading and processing Gregory Szorc <gregory.szorc@gmail.com> parents: 37054 diff changeset	273 return None
8c3c47362934 wireproto: implement basic frame reading and processing Gregory Szorc <gregory.szorc@gmail.com> parents: 37054 diff changeset	274
8c3c47362934 wireproto: implement basic frame reading and processing Gregory Szorc <gregory.szorc@gmail.com> parents: 37054 diff changeset	275 if readcount != FRAME_HEADER_SIZE:
8c3c47362934 wireproto: implement basic frame reading and processing Gregory Szorc <gregory.szorc@gmail.com> parents: 37054 diff changeset	276 raise error.Abort(_('received incomplete frame: got %d bytes: %s') %
8c3c47362934 wireproto: implement basic frame reading and processing Gregory Szorc <gregory.szorc@gmail.com> parents: 37054 diff changeset	277 (readcount, header))
8c3c47362934 wireproto: implement basic frame reading and processing Gregory Szorc <gregory.szorc@gmail.com> parents: 37054 diff changeset	278
37064 884a0c1604ad wireproto: define attr-based classes for representing frames Gregory Szorc <gregory.szorc@gmail.com> parents: 37063 diff changeset	279 h = parseheader(header)
37055 8c3c47362934 wireproto: implement basic frame reading and processing Gregory Szorc <gregory.szorc@gmail.com> parents: 37054 diff changeset	280
37064 884a0c1604ad wireproto: define attr-based classes for representing frames Gregory Szorc <gregory.szorc@gmail.com> parents: 37063 diff changeset	281 payload = fh.read(h.length)
884a0c1604ad wireproto: define attr-based classes for representing frames Gregory Szorc <gregory.szorc@gmail.com> parents: 37063 diff changeset	282 if len(payload) != h.length:
37055 8c3c47362934 wireproto: implement basic frame reading and processing Gregory Szorc <gregory.szorc@gmail.com> parents: 37054 diff changeset	283 raise error.Abort(_('frame length error: expected %d; got %d') %
37064 884a0c1604ad wireproto: define attr-based classes for representing frames Gregory Szorc <gregory.szorc@gmail.com> parents: 37063 diff changeset	284 (h.length, len(payload)))
37055 8c3c47362934 wireproto: implement basic frame reading and processing Gregory Szorc <gregory.szorc@gmail.com> parents: 37054 diff changeset	285
37288 9bfcbe4f4745 wireproto: add streams to frame-based protocol Gregory Szorc <gregory.szorc@gmail.com> parents: 37287 diff changeset	286 return frame(h.requestid, h.streamid, h.streamflags, h.typeid, h.flags,
9bfcbe4f4745 wireproto: add streams to frame-based protocol Gregory Szorc <gregory.szorc@gmail.com> parents: 37287 diff changeset	287 payload)
37055 8c3c47362934 wireproto: implement basic frame reading and processing Gregory Szorc <gregory.szorc@gmail.com> parents: 37054 diff changeset	288
37292 3d0e2cd86e05 wireproto: use CBOR for command requests Gregory Szorc <gregory.szorc@gmail.com> parents: 37291 diff changeset	289 def createcommandframes(stream, requestid, cmd, args, datafh=None,
3d0e2cd86e05 wireproto: use CBOR for command requests Gregory Szorc <gregory.szorc@gmail.com> parents: 37291 diff changeset	290 maxframesize=DEFAULT_MAX_FRAME_SIZE):
37054 40206e227412 wireproto: define and implement protocol for issuing requests Gregory Szorc <gregory.szorc@gmail.com> parents: diff changeset	291 """Create frames necessary to transmit a request to run a command.
40206e227412 wireproto: define and implement protocol for issuing requests Gregory Szorc <gregory.szorc@gmail.com> parents: diff changeset	292
40206e227412 wireproto: define and implement protocol for issuing requests Gregory Szorc <gregory.szorc@gmail.com> parents: diff changeset	293 This is a generator of bytearrays. Each item represents a frame
40206e227412 wireproto: define and implement protocol for issuing requests Gregory Szorc <gregory.szorc@gmail.com> parents: diff changeset	294 ready to be sent over the wire to a peer.
40206e227412 wireproto: define and implement protocol for issuing requests Gregory Szorc <gregory.szorc@gmail.com> parents: diff changeset	295 """
37292 3d0e2cd86e05 wireproto: use CBOR for command requests Gregory Szorc <gregory.szorc@gmail.com> parents: 37291 diff changeset	296 data = {b'name': cmd}
37054 40206e227412 wireproto: define and implement protocol for issuing requests Gregory Szorc <gregory.szorc@gmail.com> parents: diff changeset	297 if args:
37292 3d0e2cd86e05 wireproto: use CBOR for command requests Gregory Szorc <gregory.szorc@gmail.com> parents: 37291 diff changeset	298 data[b'args'] = args
37054 40206e227412 wireproto: define and implement protocol for issuing requests Gregory Szorc <gregory.szorc@gmail.com> parents: diff changeset	299
37292 3d0e2cd86e05 wireproto: use CBOR for command requests Gregory Szorc <gregory.szorc@gmail.com> parents: 37291 diff changeset	300 data = cbor.dumps(data, canonical=True)
37054 40206e227412 wireproto: define and implement protocol for issuing requests Gregory Szorc <gregory.szorc@gmail.com> parents: diff changeset	301
37292 3d0e2cd86e05 wireproto: use CBOR for command requests Gregory Szorc <gregory.szorc@gmail.com> parents: 37291 diff changeset	302 offset = 0
3d0e2cd86e05 wireproto: use CBOR for command requests Gregory Szorc <gregory.szorc@gmail.com> parents: 37291 diff changeset	303
3d0e2cd86e05 wireproto: use CBOR for command requests Gregory Szorc <gregory.szorc@gmail.com> parents: 37291 diff changeset	304 while True:
3d0e2cd86e05 wireproto: use CBOR for command requests Gregory Szorc <gregory.szorc@gmail.com> parents: 37291 diff changeset	305 flags = 0
37054 40206e227412 wireproto: define and implement protocol for issuing requests Gregory Szorc <gregory.szorc@gmail.com> parents: diff changeset	306
37292 3d0e2cd86e05 wireproto: use CBOR for command requests Gregory Szorc <gregory.szorc@gmail.com> parents: 37291 diff changeset	307 # Must set new or continuation flag.
3d0e2cd86e05 wireproto: use CBOR for command requests Gregory Szorc <gregory.szorc@gmail.com> parents: 37291 diff changeset	308 if not offset:
3d0e2cd86e05 wireproto: use CBOR for command requests Gregory Szorc <gregory.szorc@gmail.com> parents: 37291 diff changeset	309 flags \|= FLAG_COMMAND_REQUEST_NEW
3d0e2cd86e05 wireproto: use CBOR for command requests Gregory Szorc <gregory.szorc@gmail.com> parents: 37291 diff changeset	310 else:
3d0e2cd86e05 wireproto: use CBOR for command requests Gregory Szorc <gregory.szorc@gmail.com> parents: 37291 diff changeset	311 flags \|= FLAG_COMMAND_REQUEST_CONTINUATION
37054 40206e227412 wireproto: define and implement protocol for issuing requests Gregory Szorc <gregory.szorc@gmail.com> parents: diff changeset	312
37292 3d0e2cd86e05 wireproto: use CBOR for command requests Gregory Szorc <gregory.szorc@gmail.com> parents: 37291 diff changeset	313 # Data frames is set on all frames.
3d0e2cd86e05 wireproto: use CBOR for command requests Gregory Szorc <gregory.szorc@gmail.com> parents: 37291 diff changeset	314 if datafh:
3d0e2cd86e05 wireproto: use CBOR for command requests Gregory Szorc <gregory.szorc@gmail.com> parents: 37291 diff changeset	315 flags \|= FLAG_COMMAND_REQUEST_EXPECT_DATA
37054 40206e227412 wireproto: define and implement protocol for issuing requests Gregory Szorc <gregory.szorc@gmail.com> parents: diff changeset	316
37292 3d0e2cd86e05 wireproto: use CBOR for command requests Gregory Szorc <gregory.szorc@gmail.com> parents: 37291 diff changeset	317 payload = data[offset:offset + maxframesize]
3d0e2cd86e05 wireproto: use CBOR for command requests Gregory Szorc <gregory.szorc@gmail.com> parents: 37291 diff changeset	318 offset += len(payload)
3d0e2cd86e05 wireproto: use CBOR for command requests Gregory Szorc <gregory.szorc@gmail.com> parents: 37291 diff changeset	319
3d0e2cd86e05 wireproto: use CBOR for command requests Gregory Szorc <gregory.szorc@gmail.com> parents: 37291 diff changeset	320 if len(payload) == maxframesize and offset < len(data):
3d0e2cd86e05 wireproto: use CBOR for command requests Gregory Szorc <gregory.szorc@gmail.com> parents: 37291 diff changeset	321 flags \|= FLAG_COMMAND_REQUEST_MORE_FRAMES
3d0e2cd86e05 wireproto: use CBOR for command requests Gregory Szorc <gregory.szorc@gmail.com> parents: 37291 diff changeset	322
37287 3ed344546d9e wireproto: start to associate frame generation with a stream Gregory Szorc <gregory.szorc@gmail.com> parents: 37087 diff changeset	323 yield stream.makeframe(requestid=requestid,
37292 3d0e2cd86e05 wireproto: use CBOR for command requests Gregory Szorc <gregory.szorc@gmail.com> parents: 37291 diff changeset	324 typeid=FRAME_TYPE_COMMAND_REQUEST,
37287 3ed344546d9e wireproto: start to associate frame generation with a stream Gregory Szorc <gregory.szorc@gmail.com> parents: 37087 diff changeset	325 flags=flags,
3ed344546d9e wireproto: start to associate frame generation with a stream Gregory Szorc <gregory.szorc@gmail.com> parents: 37087 diff changeset	326 payload=payload)
37054 40206e227412 wireproto: define and implement protocol for issuing requests Gregory Szorc <gregory.szorc@gmail.com> parents: diff changeset	327
37292 3d0e2cd86e05 wireproto: use CBOR for command requests Gregory Szorc <gregory.szorc@gmail.com> parents: 37291 diff changeset	328 if not (flags & FLAG_COMMAND_REQUEST_MORE_FRAMES):
3d0e2cd86e05 wireproto: use CBOR for command requests Gregory Szorc <gregory.szorc@gmail.com> parents: 37291 diff changeset	329 break
3d0e2cd86e05 wireproto: use CBOR for command requests Gregory Szorc <gregory.szorc@gmail.com> parents: 37291 diff changeset	330
37054 40206e227412 wireproto: define and implement protocol for issuing requests Gregory Szorc <gregory.szorc@gmail.com> parents: diff changeset	331 if datafh:
40206e227412 wireproto: define and implement protocol for issuing requests Gregory Szorc <gregory.szorc@gmail.com> parents: diff changeset	332 while True:
40206e227412 wireproto: define and implement protocol for issuing requests Gregory Szorc <gregory.szorc@gmail.com> parents: diff changeset	333 data = datafh.read(DEFAULT_MAX_FRAME_SIZE)
40206e227412 wireproto: define and implement protocol for issuing requests Gregory Szorc <gregory.szorc@gmail.com> parents: diff changeset	334
40206e227412 wireproto: define and implement protocol for issuing requests Gregory Szorc <gregory.szorc@gmail.com> parents: diff changeset	335 done = False
40206e227412 wireproto: define and implement protocol for issuing requests Gregory Szorc <gregory.szorc@gmail.com> parents: diff changeset	336 if len(data) == DEFAULT_MAX_FRAME_SIZE:
40206e227412 wireproto: define and implement protocol for issuing requests Gregory Szorc <gregory.szorc@gmail.com> parents: diff changeset	337 flags = FLAG_COMMAND_DATA_CONTINUATION
40206e227412 wireproto: define and implement protocol for issuing requests Gregory Szorc <gregory.szorc@gmail.com> parents: diff changeset	338 else:
40206e227412 wireproto: define and implement protocol for issuing requests Gregory Szorc <gregory.szorc@gmail.com> parents: diff changeset	339 flags = FLAG_COMMAND_DATA_EOS
40206e227412 wireproto: define and implement protocol for issuing requests Gregory Szorc <gregory.szorc@gmail.com> parents: diff changeset	340 assert datafh.read(1) == b''
40206e227412 wireproto: define and implement protocol for issuing requests Gregory Szorc <gregory.szorc@gmail.com> parents: diff changeset	341 done = True
40206e227412 wireproto: define and implement protocol for issuing requests Gregory Szorc <gregory.szorc@gmail.com> parents: diff changeset	342
37287 3ed344546d9e wireproto: start to associate frame generation with a stream Gregory Szorc <gregory.szorc@gmail.com> parents: 37087 diff changeset	343 yield stream.makeframe(requestid=requestid,
3ed344546d9e wireproto: start to associate frame generation with a stream Gregory Szorc <gregory.szorc@gmail.com> parents: 37087 diff changeset	344 typeid=FRAME_TYPE_COMMAND_DATA,
3ed344546d9e wireproto: start to associate frame generation with a stream Gregory Szorc <gregory.szorc@gmail.com> parents: 37087 diff changeset	345 flags=flags,
3ed344546d9e wireproto: start to associate frame generation with a stream Gregory Szorc <gregory.szorc@gmail.com> parents: 37087 diff changeset	346 payload=data)
37054 40206e227412 wireproto: define and implement protocol for issuing requests Gregory Szorc <gregory.szorc@gmail.com> parents: diff changeset	347
40206e227412 wireproto: define and implement protocol for issuing requests Gregory Szorc <gregory.szorc@gmail.com> parents: diff changeset	348 if done:
40206e227412 wireproto: define and implement protocol for issuing requests Gregory Szorc <gregory.szorc@gmail.com> parents: diff changeset	349 break
37055 8c3c47362934 wireproto: implement basic frame reading and processing Gregory Szorc <gregory.szorc@gmail.com> parents: 37054 diff changeset	350
37724 deff7cf7eefd wireprotov2: change frame type and name for command response Gregory Szorc <gregory.szorc@gmail.com> parents: 37723 diff changeset	351 def createcommandresponseframesfrombytes(stream, requestid, data,
deff7cf7eefd wireprotov2: change frame type and name for command response Gregory Szorc <gregory.szorc@gmail.com> parents: 37723 diff changeset	352 maxframesize=DEFAULT_MAX_FRAME_SIZE):
37058 61393f888dfe wireproto: define and implement responses in framing protocol Gregory Szorc <gregory.szorc@gmail.com> parents: 37055 diff changeset	353 """Create a raw frame to send a bytes response from static bytes input.
61393f888dfe wireproto: define and implement responses in framing protocol Gregory Szorc <gregory.szorc@gmail.com> parents: 37055 diff changeset	354
61393f888dfe wireproto: define and implement responses in framing protocol Gregory Szorc <gregory.szorc@gmail.com> parents: 37055 diff changeset	355 Returns a generator of bytearrays.
61393f888dfe wireproto: define and implement responses in framing protocol Gregory Szorc <gregory.szorc@gmail.com> parents: 37055 diff changeset	356 """
37725 3ea8323d6f95 wireprotov2: change command response protocol to include a leading map Gregory Szorc <gregory.szorc@gmail.com> parents: 37724 diff changeset	357 # Automatically send the overall CBOR response map.
3ea8323d6f95 wireprotov2: change command response protocol to include a leading map Gregory Szorc <gregory.szorc@gmail.com> parents: 37724 diff changeset	358 overall = cbor.dumps({b'status': b'ok'}, canonical=True)
3ea8323d6f95 wireprotov2: change command response protocol to include a leading map Gregory Szorc <gregory.szorc@gmail.com> parents: 37724 diff changeset	359 if len(overall) > maxframesize:
3ea8323d6f95 wireprotov2: change command response protocol to include a leading map Gregory Szorc <gregory.szorc@gmail.com> parents: 37724 diff changeset	360 raise error.ProgrammingError('not yet implemented')
37058 61393f888dfe wireproto: define and implement responses in framing protocol Gregory Szorc <gregory.szorc@gmail.com> parents: 37055 diff changeset	361
37725 3ea8323d6f95 wireprotov2: change command response protocol to include a leading map Gregory Szorc <gregory.szorc@gmail.com> parents: 37724 diff changeset	362 # Simple case where we can fit the full response in a single frame.
3ea8323d6f95 wireprotov2: change command response protocol to include a leading map Gregory Szorc <gregory.szorc@gmail.com> parents: 37724 diff changeset	363 if len(overall) + len(data) <= maxframesize:
37724 deff7cf7eefd wireprotov2: change frame type and name for command response Gregory Szorc <gregory.szorc@gmail.com> parents: 37723 diff changeset	364 flags = FLAG_COMMAND_RESPONSE_EOS
37287 3ed344546d9e wireproto: start to associate frame generation with a stream Gregory Szorc <gregory.szorc@gmail.com> parents: 37087 diff changeset	365 yield stream.makeframe(requestid=requestid,
37724 deff7cf7eefd wireprotov2: change frame type and name for command response Gregory Szorc <gregory.szorc@gmail.com> parents: 37723 diff changeset	366 typeid=FRAME_TYPE_COMMAND_RESPONSE,
37485 0b7475ea38cf wireproto: port heads command to wire protocol v2 Gregory Szorc <gregory.szorc@gmail.com> parents: 37476 diff changeset	367 flags=flags,
37725 3ea8323d6f95 wireprotov2: change command response protocol to include a leading map Gregory Szorc <gregory.szorc@gmail.com> parents: 37724 diff changeset	368 payload=overall + data)
37058 61393f888dfe wireproto: define and implement responses in framing protocol Gregory Szorc <gregory.szorc@gmail.com> parents: 37055 diff changeset	369 return
61393f888dfe wireproto: define and implement responses in framing protocol Gregory Szorc <gregory.szorc@gmail.com> parents: 37055 diff changeset	370
37725 3ea8323d6f95 wireprotov2: change command response protocol to include a leading map Gregory Szorc <gregory.szorc@gmail.com> parents: 37724 diff changeset	371 # It's easier to send the overall CBOR map in its own frame than to track
3ea8323d6f95 wireprotov2: change command response protocol to include a leading map Gregory Szorc <gregory.szorc@gmail.com> parents: 37724 diff changeset	372 # offsets.
3ea8323d6f95 wireprotov2: change command response protocol to include a leading map Gregory Szorc <gregory.szorc@gmail.com> parents: 37724 diff changeset	373 yield stream.makeframe(requestid=requestid,
3ea8323d6f95 wireprotov2: change command response protocol to include a leading map Gregory Szorc <gregory.szorc@gmail.com> parents: 37724 diff changeset	374 typeid=FRAME_TYPE_COMMAND_RESPONSE,
3ea8323d6f95 wireprotov2: change command response protocol to include a leading map Gregory Szorc <gregory.szorc@gmail.com> parents: 37724 diff changeset	375 flags=FLAG_COMMAND_RESPONSE_CONTINUATION,
3ea8323d6f95 wireprotov2: change command response protocol to include a leading map Gregory Szorc <gregory.szorc@gmail.com> parents: 37724 diff changeset	376 payload=overall)
3ea8323d6f95 wireprotov2: change command response protocol to include a leading map Gregory Szorc <gregory.szorc@gmail.com> parents: 37724 diff changeset	377
37058 61393f888dfe wireproto: define and implement responses in framing protocol Gregory Szorc <gregory.szorc@gmail.com> parents: 37055 diff changeset	378 offset = 0
61393f888dfe wireproto: define and implement responses in framing protocol Gregory Szorc <gregory.szorc@gmail.com> parents: 37055 diff changeset	379 while True:
61393f888dfe wireproto: define and implement responses in framing protocol Gregory Szorc <gregory.szorc@gmail.com> parents: 37055 diff changeset	380 chunk = data[offset:offset + maxframesize]
61393f888dfe wireproto: define and implement responses in framing protocol Gregory Szorc <gregory.szorc@gmail.com> parents: 37055 diff changeset	381 offset += len(chunk)
61393f888dfe wireproto: define and implement responses in framing protocol Gregory Szorc <gregory.szorc@gmail.com> parents: 37055 diff changeset	382 done = offset == len(data)
61393f888dfe wireproto: define and implement responses in framing protocol Gregory Szorc <gregory.szorc@gmail.com> parents: 37055 diff changeset	383
61393f888dfe wireproto: define and implement responses in framing protocol Gregory Szorc <gregory.szorc@gmail.com> parents: 37055 diff changeset	384 if done:
37724 deff7cf7eefd wireprotov2: change frame type and name for command response Gregory Szorc <gregory.szorc@gmail.com> parents: 37723 diff changeset	385 flags = FLAG_COMMAND_RESPONSE_EOS
37058 61393f888dfe wireproto: define and implement responses in framing protocol Gregory Szorc <gregory.szorc@gmail.com> parents: 37055 diff changeset	386 else:
37724 deff7cf7eefd wireprotov2: change frame type and name for command response Gregory Szorc <gregory.szorc@gmail.com> parents: 37723 diff changeset	387 flags = FLAG_COMMAND_RESPONSE_CONTINUATION
37058 61393f888dfe wireproto: define and implement responses in framing protocol Gregory Szorc <gregory.szorc@gmail.com> parents: 37055 diff changeset	388
37287 3ed344546d9e wireproto: start to associate frame generation with a stream Gregory Szorc <gregory.szorc@gmail.com> parents: 37087 diff changeset	389 yield stream.makeframe(requestid=requestid,
37724 deff7cf7eefd wireprotov2: change frame type and name for command response Gregory Szorc <gregory.szorc@gmail.com> parents: 37723 diff changeset	390 typeid=FRAME_TYPE_COMMAND_RESPONSE,
37287 3ed344546d9e wireproto: start to associate frame generation with a stream Gregory Szorc <gregory.szorc@gmail.com> parents: 37087 diff changeset	391 flags=flags,
3ed344546d9e wireproto: start to associate frame generation with a stream Gregory Szorc <gregory.szorc@gmail.com> parents: 37087 diff changeset	392 payload=chunk)
37058 61393f888dfe wireproto: define and implement responses in framing protocol Gregory Szorc <gregory.szorc@gmail.com> parents: 37055 diff changeset	393
61393f888dfe wireproto: define and implement responses in framing protocol Gregory Szorc <gregory.szorc@gmail.com> parents: 37055 diff changeset	394 if done:
61393f888dfe wireproto: define and implement responses in framing protocol Gregory Szorc <gregory.szorc@gmail.com> parents: 37055 diff changeset	395 break
61393f888dfe wireproto: define and implement responses in framing protocol Gregory Szorc <gregory.szorc@gmail.com> parents: 37055 diff changeset	396
37287 3ed344546d9e wireproto: start to associate frame generation with a stream Gregory Szorc <gregory.szorc@gmail.com> parents: 37087 diff changeset	397 def createerrorframe(stream, requestid, msg, protocol=False, application=False):
37058 61393f888dfe wireproto: define and implement responses in framing protocol Gregory Szorc <gregory.szorc@gmail.com> parents: 37055 diff changeset	398 # TODO properly handle frame size limits.
61393f888dfe wireproto: define and implement responses in framing protocol Gregory Szorc <gregory.szorc@gmail.com> parents: 37055 diff changeset	399 assert len(msg) <= DEFAULT_MAX_FRAME_SIZE
61393f888dfe wireproto: define and implement responses in framing protocol Gregory Szorc <gregory.szorc@gmail.com> parents: 37055 diff changeset	400
61393f888dfe wireproto: define and implement responses in framing protocol Gregory Szorc <gregory.szorc@gmail.com> parents: 37055 diff changeset	401 flags = 0
61393f888dfe wireproto: define and implement responses in framing protocol Gregory Szorc <gregory.szorc@gmail.com> parents: 37055 diff changeset	402 if protocol:
61393f888dfe wireproto: define and implement responses in framing protocol Gregory Szorc <gregory.szorc@gmail.com> parents: 37055 diff changeset	403 flags \|= FLAG_ERROR_RESPONSE_PROTOCOL
61393f888dfe wireproto: define and implement responses in framing protocol Gregory Szorc <gregory.szorc@gmail.com> parents: 37055 diff changeset	404 if application:
61393f888dfe wireproto: define and implement responses in framing protocol Gregory Szorc <gregory.szorc@gmail.com> parents: 37055 diff changeset	405 flags \|= FLAG_ERROR_RESPONSE_APPLICATION
61393f888dfe wireproto: define and implement responses in framing protocol Gregory Szorc <gregory.szorc@gmail.com> parents: 37055 diff changeset	406
37287 3ed344546d9e wireproto: start to associate frame generation with a stream Gregory Szorc <gregory.szorc@gmail.com> parents: 37087 diff changeset	407 yield stream.makeframe(requestid=requestid,
3ed344546d9e wireproto: start to associate frame generation with a stream Gregory Szorc <gregory.szorc@gmail.com> parents: 37087 diff changeset	408 typeid=FRAME_TYPE_ERROR_RESPONSE,
3ed344546d9e wireproto: start to associate frame generation with a stream Gregory Szorc <gregory.szorc@gmail.com> parents: 37087 diff changeset	409 flags=flags,
3ed344546d9e wireproto: start to associate frame generation with a stream Gregory Szorc <gregory.szorc@gmail.com> parents: 37087 diff changeset	410 payload=msg)
37058 61393f888dfe wireproto: define and implement responses in framing protocol Gregory Szorc <gregory.szorc@gmail.com> parents: 37055 diff changeset	411
37319 36d17f37db91 wireproto: convert human output frames to CBOR Gregory Szorc <gregory.szorc@gmail.com> parents: 37299 diff changeset	412 def createtextoutputframe(stream, requestid, atoms,
36d17f37db91 wireproto: convert human output frames to CBOR Gregory Szorc <gregory.szorc@gmail.com> parents: 37299 diff changeset	413 maxframesize=DEFAULT_MAX_FRAME_SIZE):
37063 0a6c5cc09a88 wireproto: define human output side channel frame Gregory Szorc <gregory.szorc@gmail.com> parents: 37061 diff changeset	414 """Create a text output frame to render text to people.
0a6c5cc09a88 wireproto: define human output side channel frame Gregory Szorc <gregory.szorc@gmail.com> parents: 37061 diff changeset	415
0a6c5cc09a88 wireproto: define human output side channel frame Gregory Szorc <gregory.szorc@gmail.com> parents: 37061 diff changeset	416 ``atoms`` is a 3-tuple of (formatting string, args, labels).
0a6c5cc09a88 wireproto: define human output side channel frame Gregory Szorc <gregory.szorc@gmail.com> parents: 37061 diff changeset	417
0a6c5cc09a88 wireproto: define human output side channel frame Gregory Szorc <gregory.szorc@gmail.com> parents: 37061 diff changeset	418 The formatting string contains ``%s`` tokens to be replaced by the
0a6c5cc09a88 wireproto: define human output side channel frame Gregory Szorc <gregory.szorc@gmail.com> parents: 37061 diff changeset	419 corresponding indexed entry in ``args``. ``labels`` is an iterable of
0a6c5cc09a88 wireproto: define human output side channel frame Gregory Szorc <gregory.szorc@gmail.com> parents: 37061 diff changeset	420 formatters to be applied at rendering time. In terms of the ``ui``
0a6c5cc09a88 wireproto: define human output side channel frame Gregory Szorc <gregory.szorc@gmail.com> parents: 37061 diff changeset	421 class, each atom corresponds to a ``ui.write()``.
0a6c5cc09a88 wireproto: define human output side channel frame Gregory Szorc <gregory.szorc@gmail.com> parents: 37061 diff changeset	422 """
37319 36d17f37db91 wireproto: convert human output frames to CBOR Gregory Szorc <gregory.szorc@gmail.com> parents: 37299 diff changeset	423 atomdicts = []
37063 0a6c5cc09a88 wireproto: define human output side channel frame Gregory Szorc <gregory.szorc@gmail.com> parents: 37061 diff changeset	424
0a6c5cc09a88 wireproto: define human output side channel frame Gregory Szorc <gregory.szorc@gmail.com> parents: 37061 diff changeset	425 for (formatting, args, labels) in atoms:
0a6c5cc09a88 wireproto: define human output side channel frame Gregory Szorc <gregory.szorc@gmail.com> parents: 37061 diff changeset	426 # TODO look for localstr, other types here?
0a6c5cc09a88 wireproto: define human output side channel frame Gregory Szorc <gregory.szorc@gmail.com> parents: 37061 diff changeset	427
0a6c5cc09a88 wireproto: define human output side channel frame Gregory Szorc <gregory.szorc@gmail.com> parents: 37061 diff changeset	428 if not isinstance(formatting, bytes):
0a6c5cc09a88 wireproto: define human output side channel frame Gregory Szorc <gregory.szorc@gmail.com> parents: 37061 diff changeset	429 raise ValueError('must use bytes formatting strings')
0a6c5cc09a88 wireproto: define human output side channel frame Gregory Szorc <gregory.szorc@gmail.com> parents: 37061 diff changeset	430 for arg in args:
0a6c5cc09a88 wireproto: define human output side channel frame Gregory Szorc <gregory.szorc@gmail.com> parents: 37061 diff changeset	431 if not isinstance(arg, bytes):
0a6c5cc09a88 wireproto: define human output side channel frame Gregory Szorc <gregory.szorc@gmail.com> parents: 37061 diff changeset	432 raise ValueError('must use bytes for arguments')
0a6c5cc09a88 wireproto: define human output side channel frame Gregory Szorc <gregory.szorc@gmail.com> parents: 37061 diff changeset	433 for label in labels:
0a6c5cc09a88 wireproto: define human output side channel frame Gregory Szorc <gregory.szorc@gmail.com> parents: 37061 diff changeset	434 if not isinstance(label, bytes):
0a6c5cc09a88 wireproto: define human output side channel frame Gregory Szorc <gregory.szorc@gmail.com> parents: 37061 diff changeset	435 raise ValueError('must use bytes for labels')
0a6c5cc09a88 wireproto: define human output side channel frame Gregory Szorc <gregory.szorc@gmail.com> parents: 37061 diff changeset	436
37319 36d17f37db91 wireproto: convert human output frames to CBOR Gregory Szorc <gregory.szorc@gmail.com> parents: 37299 diff changeset	437 # Formatting string must be ASCII.
36d17f37db91 wireproto: convert human output frames to CBOR Gregory Szorc <gregory.szorc@gmail.com> parents: 37299 diff changeset	438 formatting = formatting.decode(r'ascii', r'replace').encode(r'ascii')
37063 0a6c5cc09a88 wireproto: define human output side channel frame Gregory Szorc <gregory.szorc@gmail.com> parents: 37061 diff changeset	439
0a6c5cc09a88 wireproto: define human output side channel frame Gregory Szorc <gregory.szorc@gmail.com> parents: 37061 diff changeset	440 # Arguments must be UTF-8.
0a6c5cc09a88 wireproto: define human output side channel frame Gregory Szorc <gregory.szorc@gmail.com> parents: 37061 diff changeset	441 args = [a.decode(r'utf-8', r'replace').encode(r'utf-8') for a in args]
0a6c5cc09a88 wireproto: define human output side channel frame Gregory Szorc <gregory.szorc@gmail.com> parents: 37061 diff changeset	442
0a6c5cc09a88 wireproto: define human output side channel frame Gregory Szorc <gregory.szorc@gmail.com> parents: 37061 diff changeset	443 # Labels must be ASCII.
0a6c5cc09a88 wireproto: define human output side channel frame Gregory Szorc <gregory.szorc@gmail.com> parents: 37061 diff changeset	444 labels = [l.decode(r'ascii', r'strict').encode(r'ascii')
0a6c5cc09a88 wireproto: define human output side channel frame Gregory Szorc <gregory.szorc@gmail.com> parents: 37061 diff changeset	445 for l in labels]
0a6c5cc09a88 wireproto: define human output side channel frame Gregory Szorc <gregory.szorc@gmail.com> parents: 37061 diff changeset	446
37319 36d17f37db91 wireproto: convert human output frames to CBOR Gregory Szorc <gregory.szorc@gmail.com> parents: 37299 diff changeset	447 atom = {b'msg': formatting}
36d17f37db91 wireproto: convert human output frames to CBOR Gregory Szorc <gregory.szorc@gmail.com> parents: 37299 diff changeset	448 if args:
36d17f37db91 wireproto: convert human output frames to CBOR Gregory Szorc <gregory.szorc@gmail.com> parents: 37299 diff changeset	449 atom[b'args'] = args
36d17f37db91 wireproto: convert human output frames to CBOR Gregory Szorc <gregory.szorc@gmail.com> parents: 37299 diff changeset	450 if labels:
36d17f37db91 wireproto: convert human output frames to CBOR Gregory Szorc <gregory.szorc@gmail.com> parents: 37299 diff changeset	451 atom[b'labels'] = labels
37063 0a6c5cc09a88 wireproto: define human output side channel frame Gregory Szorc <gregory.szorc@gmail.com> parents: 37061 diff changeset	452
37319 36d17f37db91 wireproto: convert human output frames to CBOR Gregory Szorc <gregory.szorc@gmail.com> parents: 37299 diff changeset	453 atomdicts.append(atom)
37063 0a6c5cc09a88 wireproto: define human output side channel frame Gregory Szorc <gregory.szorc@gmail.com> parents: 37061 diff changeset	454
37319 36d17f37db91 wireproto: convert human output frames to CBOR Gregory Szorc <gregory.szorc@gmail.com> parents: 37299 diff changeset	455 payload = cbor.dumps(atomdicts, canonical=True)
37063 0a6c5cc09a88 wireproto: define human output side channel frame Gregory Szorc <gregory.szorc@gmail.com> parents: 37061 diff changeset	456
37319 36d17f37db91 wireproto: convert human output frames to CBOR Gregory Szorc <gregory.szorc@gmail.com> parents: 37299 diff changeset	457 if len(payload) > maxframesize:
37063 0a6c5cc09a88 wireproto: define human output side channel frame Gregory Szorc <gregory.szorc@gmail.com> parents: 37061 diff changeset	458 raise ValueError('cannot encode data in a single frame')
0a6c5cc09a88 wireproto: define human output side channel frame Gregory Szorc <gregory.szorc@gmail.com> parents: 37061 diff changeset	459
37287 3ed344546d9e wireproto: start to associate frame generation with a stream Gregory Szorc <gregory.szorc@gmail.com> parents: 37087 diff changeset	460 yield stream.makeframe(requestid=requestid,
3ed344546d9e wireproto: start to associate frame generation with a stream Gregory Szorc <gregory.szorc@gmail.com> parents: 37087 diff changeset	461 typeid=FRAME_TYPE_TEXT_OUTPUT,
3ed344546d9e wireproto: start to associate frame generation with a stream Gregory Szorc <gregory.szorc@gmail.com> parents: 37087 diff changeset	462 flags=0,
37319 36d17f37db91 wireproto: convert human output frames to CBOR Gregory Szorc <gregory.szorc@gmail.com> parents: 37299 diff changeset	463 payload=payload)
37287 3ed344546d9e wireproto: start to associate frame generation with a stream Gregory Szorc <gregory.szorc@gmail.com> parents: 37087 diff changeset	464
3ed344546d9e wireproto: start to associate frame generation with a stream Gregory Szorc <gregory.szorc@gmail.com> parents: 37087 diff changeset	465 class stream(object):
3ed344546d9e wireproto: start to associate frame generation with a stream Gregory Szorc <gregory.szorc@gmail.com> parents: 37087 diff changeset	466 """Represents a logical unidirectional series of frames."""
3ed344546d9e wireproto: start to associate frame generation with a stream Gregory Szorc <gregory.szorc@gmail.com> parents: 37087 diff changeset	467
37288 9bfcbe4f4745 wireproto: add streams to frame-based protocol Gregory Szorc <gregory.szorc@gmail.com> parents: 37287 diff changeset	468 def __init__(self, streamid, active=False):
9bfcbe4f4745 wireproto: add streams to frame-based protocol Gregory Szorc <gregory.szorc@gmail.com> parents: 37287 diff changeset	469 self.streamid = streamid
37655 b9502b5f2066 wireprotoframing: use value passed into function Gregory Szorc <gregory.szorc@gmail.com> parents: 37544 diff changeset	470 self._active = active
37288 9bfcbe4f4745 wireproto: add streams to frame-based protocol Gregory Szorc <gregory.szorc@gmail.com> parents: 37287 diff changeset	471
37287 3ed344546d9e wireproto: start to associate frame generation with a stream Gregory Szorc <gregory.szorc@gmail.com> parents: 37087 diff changeset	472 def makeframe(self, requestid, typeid, flags, payload):
3ed344546d9e wireproto: start to associate frame generation with a stream Gregory Szorc <gregory.szorc@gmail.com> parents: 37087 diff changeset	473 """Create a frame to be sent out over this stream.
3ed344546d9e wireproto: start to associate frame generation with a stream Gregory Szorc <gregory.szorc@gmail.com> parents: 37087 diff changeset	474
3ed344546d9e wireproto: start to associate frame generation with a stream Gregory Szorc <gregory.szorc@gmail.com> parents: 37087 diff changeset	475 Only returns the frame instance. Does not actually send it.
3ed344546d9e wireproto: start to associate frame generation with a stream Gregory Szorc <gregory.szorc@gmail.com> parents: 37087 diff changeset	476 """
37288 9bfcbe4f4745 wireproto: add streams to frame-based protocol Gregory Szorc <gregory.szorc@gmail.com> parents: 37287 diff changeset	477 streamflags = 0
9bfcbe4f4745 wireproto: add streams to frame-based protocol Gregory Szorc <gregory.szorc@gmail.com> parents: 37287 diff changeset	478 if not self._active:
9bfcbe4f4745 wireproto: add streams to frame-based protocol Gregory Szorc <gregory.szorc@gmail.com> parents: 37287 diff changeset	479 streamflags \|= STREAM_FLAG_BEGIN_STREAM
9bfcbe4f4745 wireproto: add streams to frame-based protocol Gregory Szorc <gregory.szorc@gmail.com> parents: 37287 diff changeset	480 self._active = True
9bfcbe4f4745 wireproto: add streams to frame-based protocol Gregory Szorc <gregory.szorc@gmail.com> parents: 37287 diff changeset	481
9bfcbe4f4745 wireproto: add streams to frame-based protocol Gregory Szorc <gregory.szorc@gmail.com> parents: 37287 diff changeset	482 return makeframe(requestid, self.streamid, streamflags, typeid, flags,
9bfcbe4f4745 wireproto: add streams to frame-based protocol Gregory Szorc <gregory.szorc@gmail.com> parents: 37287 diff changeset	483 payload)
9bfcbe4f4745 wireproto: add streams to frame-based protocol Gregory Szorc <gregory.szorc@gmail.com> parents: 37287 diff changeset	484
9bfcbe4f4745 wireproto: add streams to frame-based protocol Gregory Szorc <gregory.szorc@gmail.com> parents: 37287 diff changeset	485 def ensureserverstream(stream):
9bfcbe4f4745 wireproto: add streams to frame-based protocol Gregory Szorc <gregory.szorc@gmail.com> parents: 37287 diff changeset	486 if stream.streamid % 2:
9bfcbe4f4745 wireproto: add streams to frame-based protocol Gregory Szorc <gregory.szorc@gmail.com> parents: 37287 diff changeset	487 raise error.ProgrammingError('server should only write to even '
9bfcbe4f4745 wireproto: add streams to frame-based protocol Gregory Szorc <gregory.szorc@gmail.com> parents: 37287 diff changeset	488 'numbered streams; %d is not even' %
9bfcbe4f4745 wireproto: add streams to frame-based protocol Gregory Szorc <gregory.szorc@gmail.com> parents: 37287 diff changeset	489 stream.streamid)
37063 0a6c5cc09a88 wireproto: define human output side channel frame Gregory Szorc <gregory.szorc@gmail.com> parents: 37061 diff changeset	490
37055 8c3c47362934 wireproto: implement basic frame reading and processing Gregory Szorc <gregory.szorc@gmail.com> parents: 37054 diff changeset	491 class serverreactor(object):
8c3c47362934 wireproto: implement basic frame reading and processing Gregory Szorc <gregory.szorc@gmail.com> parents: 37054 diff changeset	492 """Holds state of a server handling frame-based protocol requests.
8c3c47362934 wireproto: implement basic frame reading and processing Gregory Szorc <gregory.szorc@gmail.com> parents: 37054 diff changeset	493
8c3c47362934 wireproto: implement basic frame reading and processing Gregory Szorc <gregory.szorc@gmail.com> parents: 37054 diff changeset	494 This class is the "brain" of the unified frame-based protocol server
8c3c47362934 wireproto: implement basic frame reading and processing Gregory Szorc <gregory.szorc@gmail.com> parents: 37054 diff changeset	495 component. While the protocol is stateless from the perspective of
8c3c47362934 wireproto: implement basic frame reading and processing Gregory Szorc <gregory.szorc@gmail.com> parents: 37054 diff changeset	496 requests/commands, something needs to track which frames have been
8c3c47362934 wireproto: implement basic frame reading and processing Gregory Szorc <gregory.szorc@gmail.com> parents: 37054 diff changeset	497 received, what frames to expect, etc. This class is that thing.
8c3c47362934 wireproto: implement basic frame reading and processing Gregory Szorc <gregory.szorc@gmail.com> parents: 37054 diff changeset	498
8c3c47362934 wireproto: implement basic frame reading and processing Gregory Szorc <gregory.szorc@gmail.com> parents: 37054 diff changeset	499 Instances are modeled as a state machine of sorts. Instances are also
8c3c47362934 wireproto: implement basic frame reading and processing Gregory Szorc <gregory.szorc@gmail.com> parents: 37054 diff changeset	500 reactionary to external events. The point of this class is to encapsulate
8c3c47362934 wireproto: implement basic frame reading and processing Gregory Szorc <gregory.szorc@gmail.com> parents: 37054 diff changeset	501 the state of the connection and the exchange of frames, not to perform
8c3c47362934 wireproto: implement basic frame reading and processing Gregory Szorc <gregory.szorc@gmail.com> parents: 37054 diff changeset	502 work. Instead, callers tell this class when something occurs, like a
8c3c47362934 wireproto: implement basic frame reading and processing Gregory Szorc <gregory.szorc@gmail.com> parents: 37054 diff changeset	503 frame arriving. If that activity is worthy of a follow-up action (say
8c3c47362934 wireproto: implement basic frame reading and processing Gregory Szorc <gregory.szorc@gmail.com> parents: 37054 diff changeset	504 run a command), the return value of that handler will say so.
8c3c47362934 wireproto: implement basic frame reading and processing Gregory Szorc <gregory.szorc@gmail.com> parents: 37054 diff changeset	505
8c3c47362934 wireproto: implement basic frame reading and processing Gregory Szorc <gregory.szorc@gmail.com> parents: 37054 diff changeset	506 I/O and CPU intensive operations are purposefully delegated outside of
8c3c47362934 wireproto: implement basic frame reading and processing Gregory Szorc <gregory.szorc@gmail.com> parents: 37054 diff changeset	507 this class.
8c3c47362934 wireproto: implement basic frame reading and processing Gregory Szorc <gregory.szorc@gmail.com> parents: 37054 diff changeset	508
8c3c47362934 wireproto: implement basic frame reading and processing Gregory Szorc <gregory.szorc@gmail.com> parents: 37054 diff changeset	509 Consumers are expected to tell instances when events occur. They do so by
8c3c47362934 wireproto: implement basic frame reading and processing Gregory Szorc <gregory.szorc@gmail.com> parents: 37054 diff changeset	510 calling the various ``on*`` methods. These methods return a 2-tuple
8c3c47362934 wireproto: implement basic frame reading and processing Gregory Szorc <gregory.szorc@gmail.com> parents: 37054 diff changeset	511 describing any follow-up action(s) to take. The first element is the
8c3c47362934 wireproto: implement basic frame reading and processing Gregory Szorc <gregory.szorc@gmail.com> parents: 37054 diff changeset	512 name of an action to perform. The second is a data structure (usually
8c3c47362934 wireproto: implement basic frame reading and processing Gregory Szorc <gregory.szorc@gmail.com> parents: 37054 diff changeset	513 a dict) specific to that action that contains more information. e.g.
8c3c47362934 wireproto: implement basic frame reading and processing Gregory Szorc <gregory.szorc@gmail.com> parents: 37054 diff changeset	514 if the server wants to send frames back to the client, the data structure
8c3c47362934 wireproto: implement basic frame reading and processing Gregory Szorc <gregory.szorc@gmail.com> parents: 37054 diff changeset	515 will contain a reference to those frames.
8c3c47362934 wireproto: implement basic frame reading and processing Gregory Szorc <gregory.szorc@gmail.com> parents: 37054 diff changeset	516
8c3c47362934 wireproto: implement basic frame reading and processing Gregory Szorc <gregory.szorc@gmail.com> parents: 37054 diff changeset	517 Valid actions that consumers can be instructed to take are:
8c3c47362934 wireproto: implement basic frame reading and processing Gregory Szorc <gregory.szorc@gmail.com> parents: 37054 diff changeset	518
37058 61393f888dfe wireproto: define and implement responses in framing protocol Gregory Szorc <gregory.szorc@gmail.com> parents: 37055 diff changeset	519 sendframes
61393f888dfe wireproto: define and implement responses in framing protocol Gregory Szorc <gregory.szorc@gmail.com> parents: 37055 diff changeset	520 Indicates that frames should be sent to the client. The ``framegen``
61393f888dfe wireproto: define and implement responses in framing protocol Gregory Szorc <gregory.szorc@gmail.com> parents: 37055 diff changeset	521 key contains a generator of frames that should be sent. The server
61393f888dfe wireproto: define and implement responses in framing protocol Gregory Szorc <gregory.szorc@gmail.com> parents: 37055 diff changeset	522 assumes that all frames are sent to the client.
61393f888dfe wireproto: define and implement responses in framing protocol Gregory Szorc <gregory.szorc@gmail.com> parents: 37055 diff changeset	523
37055 8c3c47362934 wireproto: implement basic frame reading and processing Gregory Szorc <gregory.szorc@gmail.com> parents: 37054 diff changeset	524 error
8c3c47362934 wireproto: implement basic frame reading and processing Gregory Szorc <gregory.szorc@gmail.com> parents: 37054 diff changeset	525 Indicates that an error occurred. Consumer should probably abort.
8c3c47362934 wireproto: implement basic frame reading and processing Gregory Szorc <gregory.szorc@gmail.com> parents: 37054 diff changeset	526
8c3c47362934 wireproto: implement basic frame reading and processing Gregory Szorc <gregory.szorc@gmail.com> parents: 37054 diff changeset	527 runcommand
8c3c47362934 wireproto: implement basic frame reading and processing Gregory Szorc <gregory.szorc@gmail.com> parents: 37054 diff changeset	528 Indicates that the consumer should run a wire protocol command. Details
8c3c47362934 wireproto: implement basic frame reading and processing Gregory Szorc <gregory.szorc@gmail.com> parents: 37054 diff changeset	529 of the command to run are given in the data structure.
8c3c47362934 wireproto: implement basic frame reading and processing Gregory Szorc <gregory.szorc@gmail.com> parents: 37054 diff changeset	530
8c3c47362934 wireproto: implement basic frame reading and processing Gregory Szorc <gregory.szorc@gmail.com> parents: 37054 diff changeset	531 wantframe
8c3c47362934 wireproto: implement basic frame reading and processing Gregory Szorc <gregory.szorc@gmail.com> parents: 37054 diff changeset	532 Indicates that nothing of interest happened and the server is waiting on
8c3c47362934 wireproto: implement basic frame reading and processing Gregory Szorc <gregory.szorc@gmail.com> parents: 37054 diff changeset	533 more frames from the client before anything interesting can be done.
37059 861e9d37e56e wireproto: buffer output frames when in half duplex mode Gregory Szorc <gregory.szorc@gmail.com> parents: 37058 diff changeset	534
861e9d37e56e wireproto: buffer output frames when in half duplex mode Gregory Szorc <gregory.szorc@gmail.com> parents: 37058 diff changeset	535 noop
861e9d37e56e wireproto: buffer output frames when in half duplex mode Gregory Szorc <gregory.szorc@gmail.com> parents: 37058 diff changeset	536 Indicates no additional action is required.
37061 c5e9c3b47366 wireproto: support for receiving multiple requests Gregory Szorc <gregory.szorc@gmail.com> parents: 37060 diff changeset	537
c5e9c3b47366 wireproto: support for receiving multiple requests Gregory Szorc <gregory.szorc@gmail.com> parents: 37060 diff changeset	538 Known Issues
c5e9c3b47366 wireproto: support for receiving multiple requests Gregory Szorc <gregory.szorc@gmail.com> parents: 37060 diff changeset	539 ------------
c5e9c3b47366 wireproto: support for receiving multiple requests Gregory Szorc <gregory.szorc@gmail.com> parents: 37060 diff changeset	540
c5e9c3b47366 wireproto: support for receiving multiple requests Gregory Szorc <gregory.szorc@gmail.com> parents: 37060 diff changeset	541 There are no limits to the number of partially received commands or their
c5e9c3b47366 wireproto: support for receiving multiple requests Gregory Szorc <gregory.szorc@gmail.com> parents: 37060 diff changeset	542 size. A malicious client could stream command request data and exhaust the
c5e9c3b47366 wireproto: support for receiving multiple requests Gregory Szorc <gregory.szorc@gmail.com> parents: 37060 diff changeset	543 server's memory.
c5e9c3b47366 wireproto: support for receiving multiple requests Gregory Szorc <gregory.szorc@gmail.com> parents: 37060 diff changeset	544
c5e9c3b47366 wireproto: support for receiving multiple requests Gregory Szorc <gregory.szorc@gmail.com> parents: 37060 diff changeset	545 Partially received commands are not acted upon when end of input is
c5e9c3b47366 wireproto: support for receiving multiple requests Gregory Szorc <gregory.szorc@gmail.com> parents: 37060 diff changeset	546 reached. Should the server error if it receives a partial request?
c5e9c3b47366 wireproto: support for receiving multiple requests Gregory Szorc <gregory.szorc@gmail.com> parents: 37060 diff changeset	547 Should the client send a message to abort a partially transmitted request
c5e9c3b47366 wireproto: support for receiving multiple requests Gregory Szorc <gregory.szorc@gmail.com> parents: 37060 diff changeset	548 to facilitate graceful shutdown?
c5e9c3b47366 wireproto: support for receiving multiple requests Gregory Szorc <gregory.szorc@gmail.com> parents: 37060 diff changeset	549
c5e9c3b47366 wireproto: support for receiving multiple requests Gregory Szorc <gregory.szorc@gmail.com> parents: 37060 diff changeset	550 Active requests that haven't been responded to aren't tracked. This means
c5e9c3b47366 wireproto: support for receiving multiple requests Gregory Szorc <gregory.szorc@gmail.com> parents: 37060 diff changeset	551 that if we receive a command and instruct its dispatch, another command
c5e9c3b47366 wireproto: support for receiving multiple requests Gregory Szorc <gregory.szorc@gmail.com> parents: 37060 diff changeset	552 with its request ID can come in over the wire and there will be a race
c5e9c3b47366 wireproto: support for receiving multiple requests Gregory Szorc <gregory.szorc@gmail.com> parents: 37060 diff changeset	553 between who responds to what.
37055 8c3c47362934 wireproto: implement basic frame reading and processing Gregory Szorc <gregory.szorc@gmail.com> parents: 37054 diff changeset	554 """
8c3c47362934 wireproto: implement basic frame reading and processing Gregory Szorc <gregory.szorc@gmail.com> parents: 37054 diff changeset	555
37059 861e9d37e56e wireproto: buffer output frames when in half duplex mode Gregory Szorc <gregory.szorc@gmail.com> parents: 37058 diff changeset	556 def __init__(self, deferoutput=False):
861e9d37e56e wireproto: buffer output frames when in half duplex mode Gregory Szorc <gregory.szorc@gmail.com> parents: 37058 diff changeset	557 """Construct a new server reactor.
861e9d37e56e wireproto: buffer output frames when in half duplex mode Gregory Szorc <gregory.szorc@gmail.com> parents: 37058 diff changeset	558
861e9d37e56e wireproto: buffer output frames when in half duplex mode Gregory Szorc <gregory.szorc@gmail.com> parents: 37058 diff changeset	559 ``deferoutput`` can be used to indicate that no output frames should be
861e9d37e56e wireproto: buffer output frames when in half duplex mode Gregory Szorc <gregory.szorc@gmail.com> parents: 37058 diff changeset	560 instructed to be sent until input has been exhausted. In this mode,
861e9d37e56e wireproto: buffer output frames when in half duplex mode Gregory Szorc <gregory.szorc@gmail.com> parents: 37058 diff changeset	561 events that would normally generate output frames (such as a command
861e9d37e56e wireproto: buffer output frames when in half duplex mode Gregory Szorc <gregory.szorc@gmail.com> parents: 37058 diff changeset	562 response being ready) will instead defer instructing the consumer to
861e9d37e56e wireproto: buffer output frames when in half duplex mode Gregory Szorc <gregory.szorc@gmail.com> parents: 37058 diff changeset	563 send those frames. This is useful for half-duplex transports where the
861e9d37e56e wireproto: buffer output frames when in half duplex mode Gregory Szorc <gregory.szorc@gmail.com> parents: 37058 diff changeset	564 sender cannot receive until all data has been transmitted.
861e9d37e56e wireproto: buffer output frames when in half duplex mode Gregory Szorc <gregory.szorc@gmail.com> parents: 37058 diff changeset	565 """
861e9d37e56e wireproto: buffer output frames when in half duplex mode Gregory Szorc <gregory.szorc@gmail.com> parents: 37058 diff changeset	566 self._deferoutput = deferoutput
37055 8c3c47362934 wireproto: implement basic frame reading and processing Gregory Szorc <gregory.szorc@gmail.com> parents: 37054 diff changeset	567 self._state = 'idle'
37289 5fadc63ac99f wireproto: explicit API to create outgoing streams Gregory Szorc <gregory.szorc@gmail.com> parents: 37288 diff changeset	568 self._nextoutgoingstreamid = 2
37059 861e9d37e56e wireproto: buffer output frames when in half duplex mode Gregory Szorc <gregory.szorc@gmail.com> parents: 37058 diff changeset	569 self._bufferedframegens = []
37288 9bfcbe4f4745 wireproto: add streams to frame-based protocol Gregory Szorc <gregory.szorc@gmail.com> parents: 37287 diff changeset	570 # stream id -> stream instance for all active streams from the client.
9bfcbe4f4745 wireproto: add streams to frame-based protocol Gregory Szorc <gregory.szorc@gmail.com> parents: 37287 diff changeset	571 self._incomingstreams = {}
37289 5fadc63ac99f wireproto: explicit API to create outgoing streams Gregory Szorc <gregory.szorc@gmail.com> parents: 37288 diff changeset	572 self._outgoingstreams = {}
37061 c5e9c3b47366 wireproto: support for receiving multiple requests Gregory Szorc <gregory.szorc@gmail.com> parents: 37060 diff changeset	573 # request id -> dict of commands that are actively being received.
c5e9c3b47366 wireproto: support for receiving multiple requests Gregory Szorc <gregory.szorc@gmail.com> parents: 37060 diff changeset	574 self._receivingcommands = {}
37066 39304dd63589 wireproto: explicitly track which requests are active Gregory Szorc <gregory.szorc@gmail.com> parents: 37065 diff changeset	575 # Request IDs that have been received and are actively being processed.
39304dd63589 wireproto: explicitly track which requests are active Gregory Szorc <gregory.szorc@gmail.com> parents: 37065 diff changeset	576 # Once all output for a request has been sent, it is removed from this
39304dd63589 wireproto: explicitly track which requests are active Gregory Szorc <gregory.szorc@gmail.com> parents: 37065 diff changeset	577 # set.
39304dd63589 wireproto: explicitly track which requests are active Gregory Szorc <gregory.szorc@gmail.com> parents: 37065 diff changeset	578 self._activecommands = set()
37055 8c3c47362934 wireproto: implement basic frame reading and processing Gregory Szorc <gregory.szorc@gmail.com> parents: 37054 diff changeset	579
37064 884a0c1604ad wireproto: define attr-based classes for representing frames Gregory Szorc <gregory.szorc@gmail.com> parents: 37063 diff changeset	580 def onframerecv(self, frame):
37055 8c3c47362934 wireproto: implement basic frame reading and processing Gregory Szorc <gregory.szorc@gmail.com> parents: 37054 diff changeset	581 """Process a frame that has been received off the wire.
8c3c47362934 wireproto: implement basic frame reading and processing Gregory Szorc <gregory.szorc@gmail.com> parents: 37054 diff changeset	582
8c3c47362934 wireproto: implement basic frame reading and processing Gregory Szorc <gregory.szorc@gmail.com> parents: 37054 diff changeset	583 Returns a dict with an ``action`` key that details what action,
8c3c47362934 wireproto: implement basic frame reading and processing Gregory Szorc <gregory.szorc@gmail.com> parents: 37054 diff changeset	584 if any, the consumer should take next.
8c3c47362934 wireproto: implement basic frame reading and processing Gregory Szorc <gregory.szorc@gmail.com> parents: 37054 diff changeset	585 """
37288 9bfcbe4f4745 wireproto: add streams to frame-based protocol Gregory Szorc <gregory.szorc@gmail.com> parents: 37287 diff changeset	586 if not frame.streamid % 2:
9bfcbe4f4745 wireproto: add streams to frame-based protocol Gregory Szorc <gregory.szorc@gmail.com> parents: 37287 diff changeset	587 self._state = 'errored'
9bfcbe4f4745 wireproto: add streams to frame-based protocol Gregory Szorc <gregory.szorc@gmail.com> parents: 37287 diff changeset	588 return self._makeerrorresult(
9bfcbe4f4745 wireproto: add streams to frame-based protocol Gregory Szorc <gregory.szorc@gmail.com> parents: 37287 diff changeset	589 _('received frame with even numbered stream ID: %d') %
9bfcbe4f4745 wireproto: add streams to frame-based protocol Gregory Szorc <gregory.szorc@gmail.com> parents: 37287 diff changeset	590 frame.streamid)
9bfcbe4f4745 wireproto: add streams to frame-based protocol Gregory Szorc <gregory.szorc@gmail.com> parents: 37287 diff changeset	591
9bfcbe4f4745 wireproto: add streams to frame-based protocol Gregory Szorc <gregory.szorc@gmail.com> parents: 37287 diff changeset	592 if frame.streamid not in self._incomingstreams:
9bfcbe4f4745 wireproto: add streams to frame-based protocol Gregory Szorc <gregory.szorc@gmail.com> parents: 37287 diff changeset	593 if not frame.streamflags & STREAM_FLAG_BEGIN_STREAM:
9bfcbe4f4745 wireproto: add streams to frame-based protocol Gregory Szorc <gregory.szorc@gmail.com> parents: 37287 diff changeset	594 self._state = 'errored'
9bfcbe4f4745 wireproto: add streams to frame-based protocol Gregory Szorc <gregory.szorc@gmail.com> parents: 37287 diff changeset	595 return self._makeerrorresult(
9bfcbe4f4745 wireproto: add streams to frame-based protocol Gregory Szorc <gregory.szorc@gmail.com> parents: 37287 diff changeset	596 _('received frame on unknown inactive stream without '
9bfcbe4f4745 wireproto: add streams to frame-based protocol Gregory Szorc <gregory.szorc@gmail.com> parents: 37287 diff changeset	597 'beginning of stream flag set'))
9bfcbe4f4745 wireproto: add streams to frame-based protocol Gregory Szorc <gregory.szorc@gmail.com> parents: 37287 diff changeset	598
9bfcbe4f4745 wireproto: add streams to frame-based protocol Gregory Szorc <gregory.szorc@gmail.com> parents: 37287 diff changeset	599 self._incomingstreams[frame.streamid] = stream(frame.streamid)
9bfcbe4f4745 wireproto: add streams to frame-based protocol Gregory Szorc <gregory.szorc@gmail.com> parents: 37287 diff changeset	600
9bfcbe4f4745 wireproto: add streams to frame-based protocol Gregory Szorc <gregory.szorc@gmail.com> parents: 37287 diff changeset	601 if frame.streamflags & STREAM_FLAG_ENCODING_APPLIED:
9bfcbe4f4745 wireproto: add streams to frame-based protocol Gregory Szorc <gregory.szorc@gmail.com> parents: 37287 diff changeset	602 # TODO handle decoding frames
9bfcbe4f4745 wireproto: add streams to frame-based protocol Gregory Szorc <gregory.szorc@gmail.com> parents: 37287 diff changeset	603 self._state = 'errored'
9bfcbe4f4745 wireproto: add streams to frame-based protocol Gregory Szorc <gregory.szorc@gmail.com> parents: 37287 diff changeset	604 raise error.ProgrammingError('support for decoding stream payloads '
9bfcbe4f4745 wireproto: add streams to frame-based protocol Gregory Szorc <gregory.szorc@gmail.com> parents: 37287 diff changeset	605 'not yet implemented')
9bfcbe4f4745 wireproto: add streams to frame-based protocol Gregory Szorc <gregory.szorc@gmail.com> parents: 37287 diff changeset	606
9bfcbe4f4745 wireproto: add streams to frame-based protocol Gregory Szorc <gregory.szorc@gmail.com> parents: 37287 diff changeset	607 if frame.streamflags & STREAM_FLAG_END_STREAM:
9bfcbe4f4745 wireproto: add streams to frame-based protocol Gregory Szorc <gregory.szorc@gmail.com> parents: 37287 diff changeset	608 del self._incomingstreams[frame.streamid]
9bfcbe4f4745 wireproto: add streams to frame-based protocol Gregory Szorc <gregory.szorc@gmail.com> parents: 37287 diff changeset	609
37055 8c3c47362934 wireproto: implement basic frame reading and processing Gregory Szorc <gregory.szorc@gmail.com> parents: 37054 diff changeset	610 handlers = {
8c3c47362934 wireproto: implement basic frame reading and processing Gregory Szorc <gregory.szorc@gmail.com> parents: 37054 diff changeset	611 'idle': self._onframeidle,
37061 c5e9c3b47366 wireproto: support for receiving multiple requests Gregory Szorc <gregory.szorc@gmail.com> parents: 37060 diff changeset	612 'command-receiving': self._onframecommandreceiving,
37055 8c3c47362934 wireproto: implement basic frame reading and processing Gregory Szorc <gregory.szorc@gmail.com> parents: 37054 diff changeset	613 'errored': self._onframeerrored,
8c3c47362934 wireproto: implement basic frame reading and processing Gregory Szorc <gregory.szorc@gmail.com> parents: 37054 diff changeset	614 }
8c3c47362934 wireproto: implement basic frame reading and processing Gregory Szorc <gregory.szorc@gmail.com> parents: 37054 diff changeset	615
8c3c47362934 wireproto: implement basic frame reading and processing Gregory Szorc <gregory.szorc@gmail.com> parents: 37054 diff changeset	616 meth = handlers.get(self._state)
8c3c47362934 wireproto: implement basic frame reading and processing Gregory Szorc <gregory.szorc@gmail.com> parents: 37054 diff changeset	617 if not meth:
8c3c47362934 wireproto: implement basic frame reading and processing Gregory Szorc <gregory.szorc@gmail.com> parents: 37054 diff changeset	618 raise error.ProgrammingError('unhandled state: %s' % self._state)
8c3c47362934 wireproto: implement basic frame reading and processing Gregory Szorc <gregory.szorc@gmail.com> parents: 37054 diff changeset	619
37064 884a0c1604ad wireproto: define attr-based classes for representing frames Gregory Szorc <gregory.szorc@gmail.com> parents: 37063 diff changeset	620 return meth(frame)
37055 8c3c47362934 wireproto: implement basic frame reading and processing Gregory Szorc <gregory.szorc@gmail.com> parents: 37054 diff changeset	621
37724 deff7cf7eefd wireprotov2: change frame type and name for command response Gregory Szorc <gregory.szorc@gmail.com> parents: 37723 diff changeset	622 def oncommandresponseready(self, stream, requestid, data):
37058 61393f888dfe wireproto: define and implement responses in framing protocol Gregory Szorc <gregory.szorc@gmail.com> parents: 37055 diff changeset	623 """Signal that a bytes response is ready to be sent to the client.
61393f888dfe wireproto: define and implement responses in framing protocol Gregory Szorc <gregory.szorc@gmail.com> parents: 37055 diff changeset	624
61393f888dfe wireproto: define and implement responses in framing protocol Gregory Szorc <gregory.szorc@gmail.com> parents: 37055 diff changeset	625 The raw bytes response is passed as an argument.
61393f888dfe wireproto: define and implement responses in framing protocol Gregory Szorc <gregory.szorc@gmail.com> parents: 37055 diff changeset	626 """
37288 9bfcbe4f4745 wireproto: add streams to frame-based protocol Gregory Szorc <gregory.szorc@gmail.com> parents: 37287 diff changeset	627 ensureserverstream(stream)
9bfcbe4f4745 wireproto: add streams to frame-based protocol Gregory Szorc <gregory.szorc@gmail.com> parents: 37287 diff changeset	628
37066 39304dd63589 wireproto: explicitly track which requests are active Gregory Szorc <gregory.szorc@gmail.com> parents: 37065 diff changeset	629 def sendframes():
37724 deff7cf7eefd wireprotov2: change frame type and name for command response Gregory Szorc <gregory.szorc@gmail.com> parents: 37723 diff changeset	630 for frame in createcommandresponseframesfrombytes(stream, requestid,
deff7cf7eefd wireprotov2: change frame type and name for command response Gregory Szorc <gregory.szorc@gmail.com> parents: 37723 diff changeset	631 data):
37066 39304dd63589 wireproto: explicitly track which requests are active Gregory Szorc <gregory.szorc@gmail.com> parents: 37065 diff changeset	632 yield frame
39304dd63589 wireproto: explicitly track which requests are active Gregory Szorc <gregory.szorc@gmail.com> parents: 37065 diff changeset	633
39304dd63589 wireproto: explicitly track which requests are active Gregory Szorc <gregory.szorc@gmail.com> parents: 37065 diff changeset	634 self._activecommands.remove(requestid)
39304dd63589 wireproto: explicitly track which requests are active Gregory Szorc <gregory.szorc@gmail.com> parents: 37065 diff changeset	635
39304dd63589 wireproto: explicitly track which requests are active Gregory Szorc <gregory.szorc@gmail.com> parents: 37065 diff changeset	636 result = sendframes()
37059 861e9d37e56e wireproto: buffer output frames when in half duplex mode Gregory Szorc <gregory.szorc@gmail.com> parents: 37058 diff changeset	637
861e9d37e56e wireproto: buffer output frames when in half duplex mode Gregory Szorc <gregory.szorc@gmail.com> parents: 37058 diff changeset	638 if self._deferoutput:
37066 39304dd63589 wireproto: explicitly track which requests are active Gregory Szorc <gregory.szorc@gmail.com> parents: 37065 diff changeset	639 self._bufferedframegens.append(result)
37059 861e9d37e56e wireproto: buffer output frames when in half duplex mode Gregory Szorc <gregory.szorc@gmail.com> parents: 37058 diff changeset	640 return 'noop', {}
861e9d37e56e wireproto: buffer output frames when in half duplex mode Gregory Szorc <gregory.szorc@gmail.com> parents: 37058 diff changeset	641 else:
861e9d37e56e wireproto: buffer output frames when in half duplex mode Gregory Szorc <gregory.szorc@gmail.com> parents: 37058 diff changeset	642 return 'sendframes', {
37066 39304dd63589 wireproto: explicitly track which requests are active Gregory Szorc <gregory.szorc@gmail.com> parents: 37065 diff changeset	643 'framegen': result,
37059 861e9d37e56e wireproto: buffer output frames when in half duplex mode Gregory Szorc <gregory.szorc@gmail.com> parents: 37058 diff changeset	644 }
861e9d37e56e wireproto: buffer output frames when in half duplex mode Gregory Szorc <gregory.szorc@gmail.com> parents: 37058 diff changeset	645
861e9d37e56e wireproto: buffer output frames when in half duplex mode Gregory Szorc <gregory.szorc@gmail.com> parents: 37058 diff changeset	646 def oninputeof(self):
861e9d37e56e wireproto: buffer output frames when in half duplex mode Gregory Szorc <gregory.szorc@gmail.com> parents: 37058 diff changeset	647 """Signals that end of input has been received.
861e9d37e56e wireproto: buffer output frames when in half duplex mode Gregory Szorc <gregory.szorc@gmail.com> parents: 37058 diff changeset	648
861e9d37e56e wireproto: buffer output frames when in half duplex mode Gregory Szorc <gregory.szorc@gmail.com> parents: 37058 diff changeset	649 No more frames will be received. All pending activity should be
861e9d37e56e wireproto: buffer output frames when in half duplex mode Gregory Szorc <gregory.szorc@gmail.com> parents: 37058 diff changeset	650 completed.
861e9d37e56e wireproto: buffer output frames when in half duplex mode Gregory Szorc <gregory.szorc@gmail.com> parents: 37058 diff changeset	651 """
37061 c5e9c3b47366 wireproto: support for receiving multiple requests Gregory Szorc <gregory.szorc@gmail.com> parents: 37060 diff changeset	652 # TODO should we do anything about in-flight commands?
c5e9c3b47366 wireproto: support for receiving multiple requests Gregory Szorc <gregory.szorc@gmail.com> parents: 37060 diff changeset	653
37059 861e9d37e56e wireproto: buffer output frames when in half duplex mode Gregory Szorc <gregory.szorc@gmail.com> parents: 37058 diff changeset	654 if not self._deferoutput or not self._bufferedframegens:
861e9d37e56e wireproto: buffer output frames when in half duplex mode Gregory Szorc <gregory.szorc@gmail.com> parents: 37058 diff changeset	655 return 'noop', {}
861e9d37e56e wireproto: buffer output frames when in half duplex mode Gregory Szorc <gregory.szorc@gmail.com> parents: 37058 diff changeset	656
861e9d37e56e wireproto: buffer output frames when in half duplex mode Gregory Szorc <gregory.szorc@gmail.com> parents: 37058 diff changeset	657 # If we buffered all our responses, emit those.
861e9d37e56e wireproto: buffer output frames when in half duplex mode Gregory Szorc <gregory.szorc@gmail.com> parents: 37058 diff changeset	658 def makegen():
861e9d37e56e wireproto: buffer output frames when in half duplex mode Gregory Szorc <gregory.szorc@gmail.com> parents: 37058 diff changeset	659 for gen in self._bufferedframegens:
861e9d37e56e wireproto: buffer output frames when in half duplex mode Gregory Szorc <gregory.szorc@gmail.com> parents: 37058 diff changeset	660 for frame in gen:
861e9d37e56e wireproto: buffer output frames when in half duplex mode Gregory Szorc <gregory.szorc@gmail.com> parents: 37058 diff changeset	661 yield frame
861e9d37e56e wireproto: buffer output frames when in half duplex mode Gregory Szorc <gregory.szorc@gmail.com> parents: 37058 diff changeset	662
37058 61393f888dfe wireproto: define and implement responses in framing protocol Gregory Szorc <gregory.szorc@gmail.com> parents: 37055 diff changeset	663 return 'sendframes', {
37059 861e9d37e56e wireproto: buffer output frames when in half duplex mode Gregory Szorc <gregory.szorc@gmail.com> parents: 37058 diff changeset	664 'framegen': makegen(),
37058 61393f888dfe wireproto: define and implement responses in framing protocol Gregory Szorc <gregory.szorc@gmail.com> parents: 37055 diff changeset	665 }
61393f888dfe wireproto: define and implement responses in framing protocol Gregory Szorc <gregory.szorc@gmail.com> parents: 37055 diff changeset	666
37287 3ed344546d9e wireproto: start to associate frame generation with a stream Gregory Szorc <gregory.szorc@gmail.com> parents: 37087 diff changeset	667 def onapplicationerror(self, stream, requestid, msg):
37288 9bfcbe4f4745 wireproto: add streams to frame-based protocol Gregory Szorc <gregory.szorc@gmail.com> parents: 37287 diff changeset	668 ensureserverstream(stream)
9bfcbe4f4745 wireproto: add streams to frame-based protocol Gregory Szorc <gregory.szorc@gmail.com> parents: 37287 diff changeset	669
37058 61393f888dfe wireproto: define and implement responses in framing protocol Gregory Szorc <gregory.szorc@gmail.com> parents: 37055 diff changeset	670 return 'sendframes', {
37287 3ed344546d9e wireproto: start to associate frame generation with a stream Gregory Szorc <gregory.szorc@gmail.com> parents: 37087 diff changeset	671 'framegen': createerrorframe(stream, requestid, msg,
3ed344546d9e wireproto: start to associate frame generation with a stream Gregory Szorc <gregory.szorc@gmail.com> parents: 37087 diff changeset	672 application=True),
37058 61393f888dfe wireproto: define and implement responses in framing protocol Gregory Szorc <gregory.szorc@gmail.com> parents: 37055 diff changeset	673 }
61393f888dfe wireproto: define and implement responses in framing protocol Gregory Szorc <gregory.szorc@gmail.com> parents: 37055 diff changeset	674
37289 5fadc63ac99f wireproto: explicit API to create outgoing streams Gregory Szorc <gregory.szorc@gmail.com> parents: 37288 diff changeset	675 def makeoutputstream(self):
5fadc63ac99f wireproto: explicit API to create outgoing streams Gregory Szorc <gregory.szorc@gmail.com> parents: 37288 diff changeset	676 """Create a stream to be used for sending data to the client."""
5fadc63ac99f wireproto: explicit API to create outgoing streams Gregory Szorc <gregory.szorc@gmail.com> parents: 37288 diff changeset	677 streamid = self._nextoutgoingstreamid
5fadc63ac99f wireproto: explicit API to create outgoing streams Gregory Szorc <gregory.szorc@gmail.com> parents: 37288 diff changeset	678 self._nextoutgoingstreamid += 2
5fadc63ac99f wireproto: explicit API to create outgoing streams Gregory Szorc <gregory.szorc@gmail.com> parents: 37288 diff changeset	679
5fadc63ac99f wireproto: explicit API to create outgoing streams Gregory Szorc <gregory.szorc@gmail.com> parents: 37288 diff changeset	680 s = stream(streamid)
5fadc63ac99f wireproto: explicit API to create outgoing streams Gregory Szorc <gregory.szorc@gmail.com> parents: 37288 diff changeset	681 self._outgoingstreams[streamid] = s
5fadc63ac99f wireproto: explicit API to create outgoing streams Gregory Szorc <gregory.szorc@gmail.com> parents: 37288 diff changeset	682
5fadc63ac99f wireproto: explicit API to create outgoing streams Gregory Szorc <gregory.szorc@gmail.com> parents: 37288 diff changeset	683 return s
5fadc63ac99f wireproto: explicit API to create outgoing streams Gregory Szorc <gregory.szorc@gmail.com> parents: 37288 diff changeset	684
37055 8c3c47362934 wireproto: implement basic frame reading and processing Gregory Szorc <gregory.szorc@gmail.com> parents: 37054 diff changeset	685 def _makeerrorresult(self, msg):
8c3c47362934 wireproto: implement basic frame reading and processing Gregory Szorc <gregory.szorc@gmail.com> parents: 37054 diff changeset	686 return 'error', {
8c3c47362934 wireproto: implement basic frame reading and processing Gregory Szorc <gregory.szorc@gmail.com> parents: 37054 diff changeset	687 'message': msg,
8c3c47362934 wireproto: implement basic frame reading and processing Gregory Szorc <gregory.szorc@gmail.com> parents: 37054 diff changeset	688 }
8c3c47362934 wireproto: implement basic frame reading and processing Gregory Szorc <gregory.szorc@gmail.com> parents: 37054 diff changeset	689
37061 c5e9c3b47366 wireproto: support for receiving multiple requests Gregory Szorc <gregory.szorc@gmail.com> parents: 37060 diff changeset	690 def _makeruncommandresult(self, requestid):
c5e9c3b47366 wireproto: support for receiving multiple requests Gregory Szorc <gregory.szorc@gmail.com> parents: 37060 diff changeset	691 entry = self._receivingcommands[requestid]
37292 3d0e2cd86e05 wireproto: use CBOR for command requests Gregory Szorc <gregory.szorc@gmail.com> parents: 37291 diff changeset	692
3d0e2cd86e05 wireproto: use CBOR for command requests Gregory Szorc <gregory.szorc@gmail.com> parents: 37291 diff changeset	693 if not entry['requestdone']:
3d0e2cd86e05 wireproto: use CBOR for command requests Gregory Szorc <gregory.szorc@gmail.com> parents: 37291 diff changeset	694 self._state = 'errored'
3d0e2cd86e05 wireproto: use CBOR for command requests Gregory Szorc <gregory.szorc@gmail.com> parents: 37291 diff changeset	695 raise error.ProgrammingError('should not be called without '
3d0e2cd86e05 wireproto: use CBOR for command requests Gregory Szorc <gregory.szorc@gmail.com> parents: 37291 diff changeset	696 'requestdone set')
3d0e2cd86e05 wireproto: use CBOR for command requests Gregory Szorc <gregory.szorc@gmail.com> parents: 37291 diff changeset	697
37061 c5e9c3b47366 wireproto: support for receiving multiple requests Gregory Szorc <gregory.szorc@gmail.com> parents: 37060 diff changeset	698 del self._receivingcommands[requestid]
c5e9c3b47366 wireproto: support for receiving multiple requests Gregory Szorc <gregory.szorc@gmail.com> parents: 37060 diff changeset	699
c5e9c3b47366 wireproto: support for receiving multiple requests Gregory Szorc <gregory.szorc@gmail.com> parents: 37060 diff changeset	700 if self._receivingcommands:
c5e9c3b47366 wireproto: support for receiving multiple requests Gregory Szorc <gregory.szorc@gmail.com> parents: 37060 diff changeset	701 self._state = 'command-receiving'
c5e9c3b47366 wireproto: support for receiving multiple requests Gregory Szorc <gregory.szorc@gmail.com> parents: 37060 diff changeset	702 else:
c5e9c3b47366 wireproto: support for receiving multiple requests Gregory Szorc <gregory.szorc@gmail.com> parents: 37060 diff changeset	703 self._state = 'idle'
c5e9c3b47366 wireproto: support for receiving multiple requests Gregory Szorc <gregory.szorc@gmail.com> parents: 37060 diff changeset	704
37292 3d0e2cd86e05 wireproto: use CBOR for command requests Gregory Szorc <gregory.szorc@gmail.com> parents: 37291 diff changeset	705 # Decode the payloads as CBOR.
3d0e2cd86e05 wireproto: use CBOR for command requests Gregory Szorc <gregory.szorc@gmail.com> parents: 37291 diff changeset	706 entry['payload'].seek(0)
3d0e2cd86e05 wireproto: use CBOR for command requests Gregory Szorc <gregory.szorc@gmail.com> parents: 37291 diff changeset	707 request = cbor.load(entry['payload'])
3d0e2cd86e05 wireproto: use CBOR for command requests Gregory Szorc <gregory.szorc@gmail.com> parents: 37291 diff changeset	708
3d0e2cd86e05 wireproto: use CBOR for command requests Gregory Szorc <gregory.szorc@gmail.com> parents: 37291 diff changeset	709 if b'name' not in request:
3d0e2cd86e05 wireproto: use CBOR for command requests Gregory Szorc <gregory.szorc@gmail.com> parents: 37291 diff changeset	710 self._state = 'errored'
3d0e2cd86e05 wireproto: use CBOR for command requests Gregory Szorc <gregory.szorc@gmail.com> parents: 37291 diff changeset	711 return self._makeerrorresult(
3d0e2cd86e05 wireproto: use CBOR for command requests Gregory Szorc <gregory.szorc@gmail.com> parents: 37291 diff changeset	712 _('command request missing "name" field'))
3d0e2cd86e05 wireproto: use CBOR for command requests Gregory Szorc <gregory.szorc@gmail.com> parents: 37291 diff changeset	713
3d0e2cd86e05 wireproto: use CBOR for command requests Gregory Szorc <gregory.szorc@gmail.com> parents: 37291 diff changeset	714 if b'args' not in request:
3d0e2cd86e05 wireproto: use CBOR for command requests Gregory Szorc <gregory.szorc@gmail.com> parents: 37291 diff changeset	715 request[b'args'] = {}
3d0e2cd86e05 wireproto: use CBOR for command requests Gregory Szorc <gregory.szorc@gmail.com> parents: 37291 diff changeset	716
37066 39304dd63589 wireproto: explicitly track which requests are active Gregory Szorc <gregory.szorc@gmail.com> parents: 37065 diff changeset	717 assert requestid not in self._activecommands
39304dd63589 wireproto: explicitly track which requests are active Gregory Szorc <gregory.szorc@gmail.com> parents: 37065 diff changeset	718 self._activecommands.add(requestid)
39304dd63589 wireproto: explicitly track which requests are active Gregory Szorc <gregory.szorc@gmail.com> parents: 37065 diff changeset	719
37055 8c3c47362934 wireproto: implement basic frame reading and processing Gregory Szorc <gregory.szorc@gmail.com> parents: 37054 diff changeset	720 return 'runcommand', {
37061 c5e9c3b47366 wireproto: support for receiving multiple requests Gregory Szorc <gregory.szorc@gmail.com> parents: 37060 diff changeset	721 'requestid': requestid,
37292 3d0e2cd86e05 wireproto: use CBOR for command requests Gregory Szorc <gregory.szorc@gmail.com> parents: 37291 diff changeset	722 'command': request[b'name'],
3d0e2cd86e05 wireproto: use CBOR for command requests Gregory Szorc <gregory.szorc@gmail.com> parents: 37291 diff changeset	723 'args': request[b'args'],
37061 c5e9c3b47366 wireproto: support for receiving multiple requests Gregory Szorc <gregory.szorc@gmail.com> parents: 37060 diff changeset	724 'data': entry['data'].getvalue() if entry['data'] else None,
37055 8c3c47362934 wireproto: implement basic frame reading and processing Gregory Szorc <gregory.szorc@gmail.com> parents: 37054 diff changeset	725 }
8c3c47362934 wireproto: implement basic frame reading and processing Gregory Szorc <gregory.szorc@gmail.com> parents: 37054 diff changeset	726
8c3c47362934 wireproto: implement basic frame reading and processing Gregory Szorc <gregory.szorc@gmail.com> parents: 37054 diff changeset	727 def _makewantframeresult(self):
8c3c47362934 wireproto: implement basic frame reading and processing Gregory Szorc <gregory.szorc@gmail.com> parents: 37054 diff changeset	728 return 'wantframe', {
8c3c47362934 wireproto: implement basic frame reading and processing Gregory Szorc <gregory.szorc@gmail.com> parents: 37054 diff changeset	729 'state': self._state,
8c3c47362934 wireproto: implement basic frame reading and processing Gregory Szorc <gregory.szorc@gmail.com> parents: 37054 diff changeset	730 }
8c3c47362934 wireproto: implement basic frame reading and processing Gregory Szorc <gregory.szorc@gmail.com> parents: 37054 diff changeset	731
37292 3d0e2cd86e05 wireproto: use CBOR for command requests Gregory Szorc <gregory.szorc@gmail.com> parents: 37291 diff changeset	732 def _validatecommandrequestframe(self, frame):
3d0e2cd86e05 wireproto: use CBOR for command requests Gregory Szorc <gregory.szorc@gmail.com> parents: 37291 diff changeset	733 new = frame.flags & FLAG_COMMAND_REQUEST_NEW
3d0e2cd86e05 wireproto: use CBOR for command requests Gregory Szorc <gregory.szorc@gmail.com> parents: 37291 diff changeset	734 continuation = frame.flags & FLAG_COMMAND_REQUEST_CONTINUATION
3d0e2cd86e05 wireproto: use CBOR for command requests Gregory Szorc <gregory.szorc@gmail.com> parents: 37291 diff changeset	735
3d0e2cd86e05 wireproto: use CBOR for command requests Gregory Szorc <gregory.szorc@gmail.com> parents: 37291 diff changeset	736 if new and continuation:
3d0e2cd86e05 wireproto: use CBOR for command requests Gregory Szorc <gregory.szorc@gmail.com> parents: 37291 diff changeset	737 self._state = 'errored'
3d0e2cd86e05 wireproto: use CBOR for command requests Gregory Szorc <gregory.szorc@gmail.com> parents: 37291 diff changeset	738 return self._makeerrorresult(
3d0e2cd86e05 wireproto: use CBOR for command requests Gregory Szorc <gregory.szorc@gmail.com> parents: 37291 diff changeset	739 _('received command request frame with both new and '
3d0e2cd86e05 wireproto: use CBOR for command requests Gregory Szorc <gregory.szorc@gmail.com> parents: 37291 diff changeset	740 'continuation flags set'))
3d0e2cd86e05 wireproto: use CBOR for command requests Gregory Szorc <gregory.szorc@gmail.com> parents: 37291 diff changeset	741
3d0e2cd86e05 wireproto: use CBOR for command requests Gregory Szorc <gregory.szorc@gmail.com> parents: 37291 diff changeset	742 if not new and not continuation:
3d0e2cd86e05 wireproto: use CBOR for command requests Gregory Szorc <gregory.szorc@gmail.com> parents: 37291 diff changeset	743 self._state = 'errored'
3d0e2cd86e05 wireproto: use CBOR for command requests Gregory Szorc <gregory.szorc@gmail.com> parents: 37291 diff changeset	744 return self._makeerrorresult(
3d0e2cd86e05 wireproto: use CBOR for command requests Gregory Szorc <gregory.szorc@gmail.com> parents: 37291 diff changeset	745 _('received command request frame with neither new nor '
3d0e2cd86e05 wireproto: use CBOR for command requests Gregory Szorc <gregory.szorc@gmail.com> parents: 37291 diff changeset	746 'continuation flags set'))
3d0e2cd86e05 wireproto: use CBOR for command requests Gregory Szorc <gregory.szorc@gmail.com> parents: 37291 diff changeset	747
37064 884a0c1604ad wireproto: define attr-based classes for representing frames Gregory Szorc <gregory.szorc@gmail.com> parents: 37063 diff changeset	748 def _onframeidle(self, frame):
37055 8c3c47362934 wireproto: implement basic frame reading and processing Gregory Szorc <gregory.szorc@gmail.com> parents: 37054 diff changeset	749 # The only frame type that should be received in this state is a
8c3c47362934 wireproto: implement basic frame reading and processing Gregory Szorc <gregory.szorc@gmail.com> parents: 37054 diff changeset	750 # command request.
37292 3d0e2cd86e05 wireproto: use CBOR for command requests Gregory Szorc <gregory.szorc@gmail.com> parents: 37291 diff changeset	751 if frame.typeid != FRAME_TYPE_COMMAND_REQUEST:
37055 8c3c47362934 wireproto: implement basic frame reading and processing Gregory Szorc <gregory.szorc@gmail.com> parents: 37054 diff changeset	752 self._state = 'errored'
8c3c47362934 wireproto: implement basic frame reading and processing Gregory Szorc <gregory.szorc@gmail.com> parents: 37054 diff changeset	753 return self._makeerrorresult(
37292 3d0e2cd86e05 wireproto: use CBOR for command requests Gregory Szorc <gregory.szorc@gmail.com> parents: 37291 diff changeset	754 _('expected command request frame; got %d') % frame.typeid)
3d0e2cd86e05 wireproto: use CBOR for command requests Gregory Szorc <gregory.szorc@gmail.com> parents: 37291 diff changeset	755
3d0e2cd86e05 wireproto: use CBOR for command requests Gregory Szorc <gregory.szorc@gmail.com> parents: 37291 diff changeset	756 res = self._validatecommandrequestframe(frame)
3d0e2cd86e05 wireproto: use CBOR for command requests Gregory Szorc <gregory.szorc@gmail.com> parents: 37291 diff changeset	757 if res:
3d0e2cd86e05 wireproto: use CBOR for command requests Gregory Szorc <gregory.szorc@gmail.com> parents: 37291 diff changeset	758 return res
37055 8c3c47362934 wireproto: implement basic frame reading and processing Gregory Szorc <gregory.szorc@gmail.com> parents: 37054 diff changeset	759
37064 884a0c1604ad wireproto: define attr-based classes for representing frames Gregory Szorc <gregory.szorc@gmail.com> parents: 37063 diff changeset	760 if frame.requestid in self._receivingcommands:
37061 c5e9c3b47366 wireproto: support for receiving multiple requests Gregory Szorc <gregory.szorc@gmail.com> parents: 37060 diff changeset	761 self._state = 'errored'
c5e9c3b47366 wireproto: support for receiving multiple requests Gregory Szorc <gregory.szorc@gmail.com> parents: 37060 diff changeset	762 return self._makeerrorresult(
37064 884a0c1604ad wireproto: define attr-based classes for representing frames Gregory Szorc <gregory.szorc@gmail.com> parents: 37063 diff changeset	763 _('request with ID %d already received') % frame.requestid)
37061 c5e9c3b47366 wireproto: support for receiving multiple requests Gregory Szorc <gregory.szorc@gmail.com> parents: 37060 diff changeset	764
37066 39304dd63589 wireproto: explicitly track which requests are active Gregory Szorc <gregory.szorc@gmail.com> parents: 37065 diff changeset	765 if frame.requestid in self._activecommands:
39304dd63589 wireproto: explicitly track which requests are active Gregory Szorc <gregory.szorc@gmail.com> parents: 37065 diff changeset	766 self._state = 'errored'
37292 3d0e2cd86e05 wireproto: use CBOR for command requests Gregory Szorc <gregory.szorc@gmail.com> parents: 37291 diff changeset	767 return self._makeerrorresult(
3d0e2cd86e05 wireproto: use CBOR for command requests Gregory Szorc <gregory.szorc@gmail.com> parents: 37291 diff changeset	768 _('request with ID %d is already active') % frame.requestid)
3d0e2cd86e05 wireproto: use CBOR for command requests Gregory Szorc <gregory.szorc@gmail.com> parents: 37291 diff changeset	769
3d0e2cd86e05 wireproto: use CBOR for command requests Gregory Szorc <gregory.szorc@gmail.com> parents: 37291 diff changeset	770 new = frame.flags & FLAG_COMMAND_REQUEST_NEW
3d0e2cd86e05 wireproto: use CBOR for command requests Gregory Szorc <gregory.szorc@gmail.com> parents: 37291 diff changeset	771 moreframes = frame.flags & FLAG_COMMAND_REQUEST_MORE_FRAMES
3d0e2cd86e05 wireproto: use CBOR for command requests Gregory Szorc <gregory.szorc@gmail.com> parents: 37291 diff changeset	772 expectingdata = frame.flags & FLAG_COMMAND_REQUEST_EXPECT_DATA
37066 39304dd63589 wireproto: explicitly track which requests are active Gregory Szorc <gregory.szorc@gmail.com> parents: 37065 diff changeset	773
37292 3d0e2cd86e05 wireproto: use CBOR for command requests Gregory Szorc <gregory.szorc@gmail.com> parents: 37291 diff changeset	774 if not new:
3d0e2cd86e05 wireproto: use CBOR for command requests Gregory Szorc <gregory.szorc@gmail.com> parents: 37291 diff changeset	775 self._state = 'errored'
3d0e2cd86e05 wireproto: use CBOR for command requests Gregory Szorc <gregory.szorc@gmail.com> parents: 37291 diff changeset	776 return self._makeerrorresult(
3d0e2cd86e05 wireproto: use CBOR for command requests Gregory Szorc <gregory.szorc@gmail.com> parents: 37291 diff changeset	777 _('received command request frame without new flag set'))
3d0e2cd86e05 wireproto: use CBOR for command requests Gregory Szorc <gregory.szorc@gmail.com> parents: 37291 diff changeset	778
3d0e2cd86e05 wireproto: use CBOR for command requests Gregory Szorc <gregory.szorc@gmail.com> parents: 37291 diff changeset	779 payload = util.bytesio()
3d0e2cd86e05 wireproto: use CBOR for command requests Gregory Szorc <gregory.szorc@gmail.com> parents: 37291 diff changeset	780 payload.write(frame.payload)
37061 c5e9c3b47366 wireproto: support for receiving multiple requests Gregory Szorc <gregory.szorc@gmail.com> parents: 37060 diff changeset	781
37064 884a0c1604ad wireproto: define attr-based classes for representing frames Gregory Szorc <gregory.szorc@gmail.com> parents: 37063 diff changeset	782 self._receivingcommands[frame.requestid] = {
37292 3d0e2cd86e05 wireproto: use CBOR for command requests Gregory Szorc <gregory.szorc@gmail.com> parents: 37291 diff changeset	783 'payload': payload,
37061 c5e9c3b47366 wireproto: support for receiving multiple requests Gregory Szorc <gregory.szorc@gmail.com> parents: 37060 diff changeset	784 'data': None,
37292 3d0e2cd86e05 wireproto: use CBOR for command requests Gregory Szorc <gregory.szorc@gmail.com> parents: 37291 diff changeset	785 'requestdone': not moreframes,
3d0e2cd86e05 wireproto: use CBOR for command requests Gregory Szorc <gregory.szorc@gmail.com> parents: 37291 diff changeset	786 'expectingdata': bool(expectingdata),
37061 c5e9c3b47366 wireproto: support for receiving multiple requests Gregory Szorc <gregory.szorc@gmail.com> parents: 37060 diff changeset	787 }
37055 8c3c47362934 wireproto: implement basic frame reading and processing Gregory Szorc <gregory.szorc@gmail.com> parents: 37054 diff changeset	788
37292 3d0e2cd86e05 wireproto: use CBOR for command requests Gregory Szorc <gregory.szorc@gmail.com> parents: 37291 diff changeset	789 # This is the final frame for this request. Dispatch it.
3d0e2cd86e05 wireproto: use CBOR for command requests Gregory Szorc <gregory.szorc@gmail.com> parents: 37291 diff changeset	790 if not moreframes and not expectingdata:
37064 884a0c1604ad wireproto: define attr-based classes for representing frames Gregory Szorc <gregory.szorc@gmail.com> parents: 37063 diff changeset	791 return self._makeruncommandresult(frame.requestid)
37055 8c3c47362934 wireproto: implement basic frame reading and processing Gregory Szorc <gregory.szorc@gmail.com> parents: 37054 diff changeset	792
37292 3d0e2cd86e05 wireproto: use CBOR for command requests Gregory Szorc <gregory.szorc@gmail.com> parents: 37291 diff changeset	793 assert moreframes or expectingdata
3d0e2cd86e05 wireproto: use CBOR for command requests Gregory Szorc <gregory.szorc@gmail.com> parents: 37291 diff changeset	794 self._state = 'command-receiving'
3d0e2cd86e05 wireproto: use CBOR for command requests Gregory Szorc <gregory.szorc@gmail.com> parents: 37291 diff changeset	795 return self._makewantframeresult()
37055 8c3c47362934 wireproto: implement basic frame reading and processing Gregory Szorc <gregory.szorc@gmail.com> parents: 37054 diff changeset	796
37064 884a0c1604ad wireproto: define attr-based classes for representing frames Gregory Szorc <gregory.szorc@gmail.com> parents: 37063 diff changeset	797 def _onframecommandreceiving(self, frame):
37292 3d0e2cd86e05 wireproto: use CBOR for command requests Gregory Szorc <gregory.szorc@gmail.com> parents: 37291 diff changeset	798 if frame.typeid == FRAME_TYPE_COMMAND_REQUEST:
3d0e2cd86e05 wireproto: use CBOR for command requests Gregory Szorc <gregory.szorc@gmail.com> parents: 37291 diff changeset	799 # Process new command requests as such.
3d0e2cd86e05 wireproto: use CBOR for command requests Gregory Szorc <gregory.szorc@gmail.com> parents: 37291 diff changeset	800 if frame.flags & FLAG_COMMAND_REQUEST_NEW:
3d0e2cd86e05 wireproto: use CBOR for command requests Gregory Szorc <gregory.szorc@gmail.com> parents: 37291 diff changeset	801 return self._onframeidle(frame)
3d0e2cd86e05 wireproto: use CBOR for command requests Gregory Szorc <gregory.szorc@gmail.com> parents: 37291 diff changeset	802
3d0e2cd86e05 wireproto: use CBOR for command requests Gregory Szorc <gregory.szorc@gmail.com> parents: 37291 diff changeset	803 res = self._validatecommandrequestframe(frame)
3d0e2cd86e05 wireproto: use CBOR for command requests Gregory Szorc <gregory.szorc@gmail.com> parents: 37291 diff changeset	804 if res:
3d0e2cd86e05 wireproto: use CBOR for command requests Gregory Szorc <gregory.szorc@gmail.com> parents: 37291 diff changeset	805 return res
37061 c5e9c3b47366 wireproto: support for receiving multiple requests Gregory Szorc <gregory.szorc@gmail.com> parents: 37060 diff changeset	806
c5e9c3b47366 wireproto: support for receiving multiple requests Gregory Szorc <gregory.szorc@gmail.com> parents: 37060 diff changeset	807 # All other frames should be related to a command that is currently
37066 39304dd63589 wireproto: explicitly track which requests are active Gregory Szorc <gregory.szorc@gmail.com> parents: 37065 diff changeset	808 # receiving but is not active.
39304dd63589 wireproto: explicitly track which requests are active Gregory Szorc <gregory.szorc@gmail.com> parents: 37065 diff changeset	809 if frame.requestid in self._activecommands:
39304dd63589 wireproto: explicitly track which requests are active Gregory Szorc <gregory.szorc@gmail.com> parents: 37065 diff changeset	810 self._state = 'errored'
39304dd63589 wireproto: explicitly track which requests are active Gregory Szorc <gregory.szorc@gmail.com> parents: 37065 diff changeset	811 return self._makeerrorresult(
39304dd63589 wireproto: explicitly track which requests are active Gregory Szorc <gregory.szorc@gmail.com> parents: 37065 diff changeset	812 _('received frame for request that is still active: %d') %
39304dd63589 wireproto: explicitly track which requests are active Gregory Szorc <gregory.szorc@gmail.com> parents: 37065 diff changeset	813 frame.requestid)
39304dd63589 wireproto: explicitly track which requests are active Gregory Szorc <gregory.szorc@gmail.com> parents: 37065 diff changeset	814
37064 884a0c1604ad wireproto: define attr-based classes for representing frames Gregory Szorc <gregory.szorc@gmail.com> parents: 37063 diff changeset	815 if frame.requestid not in self._receivingcommands:
37055 8c3c47362934 wireproto: implement basic frame reading and processing Gregory Szorc <gregory.szorc@gmail.com> parents: 37054 diff changeset	816 self._state = 'errored'
37061 c5e9c3b47366 wireproto: support for receiving multiple requests Gregory Szorc <gregory.szorc@gmail.com> parents: 37060 diff changeset	817 return self._makeerrorresult(
c5e9c3b47366 wireproto: support for receiving multiple requests Gregory Szorc <gregory.szorc@gmail.com> parents: 37060 diff changeset	818 _('received frame for request that is not receiving: %d') %
37064 884a0c1604ad wireproto: define attr-based classes for representing frames Gregory Szorc <gregory.szorc@gmail.com> parents: 37063 diff changeset	819 frame.requestid)
37061 c5e9c3b47366 wireproto: support for receiving multiple requests Gregory Szorc <gregory.szorc@gmail.com> parents: 37060 diff changeset	820
37064 884a0c1604ad wireproto: define attr-based classes for representing frames Gregory Szorc <gregory.szorc@gmail.com> parents: 37063 diff changeset	821 entry = self._receivingcommands[frame.requestid]
37061 c5e9c3b47366 wireproto: support for receiving multiple requests Gregory Szorc <gregory.szorc@gmail.com> parents: 37060 diff changeset	822
37292 3d0e2cd86e05 wireproto: use CBOR for command requests Gregory Szorc <gregory.szorc@gmail.com> parents: 37291 diff changeset	823 if frame.typeid == FRAME_TYPE_COMMAND_REQUEST:
3d0e2cd86e05 wireproto: use CBOR for command requests Gregory Szorc <gregory.szorc@gmail.com> parents: 37291 diff changeset	824 moreframes = frame.flags & FLAG_COMMAND_REQUEST_MORE_FRAMES
3d0e2cd86e05 wireproto: use CBOR for command requests Gregory Szorc <gregory.szorc@gmail.com> parents: 37291 diff changeset	825 expectingdata = bool(frame.flags & FLAG_COMMAND_REQUEST_EXPECT_DATA)
3d0e2cd86e05 wireproto: use CBOR for command requests Gregory Szorc <gregory.szorc@gmail.com> parents: 37291 diff changeset	826
3d0e2cd86e05 wireproto: use CBOR for command requests Gregory Szorc <gregory.szorc@gmail.com> parents: 37291 diff changeset	827 if entry['requestdone']:
3d0e2cd86e05 wireproto: use CBOR for command requests Gregory Szorc <gregory.szorc@gmail.com> parents: 37291 diff changeset	828 self._state = 'errored'
3d0e2cd86e05 wireproto: use CBOR for command requests Gregory Szorc <gregory.szorc@gmail.com> parents: 37291 diff changeset	829 return self._makeerrorresult(
3d0e2cd86e05 wireproto: use CBOR for command requests Gregory Szorc <gregory.szorc@gmail.com> parents: 37291 diff changeset	830 _('received command request frame when request frames '
3d0e2cd86e05 wireproto: use CBOR for command requests Gregory Szorc <gregory.szorc@gmail.com> parents: 37291 diff changeset	831 'were supposedly done'))
3d0e2cd86e05 wireproto: use CBOR for command requests Gregory Szorc <gregory.szorc@gmail.com> parents: 37291 diff changeset	832
3d0e2cd86e05 wireproto: use CBOR for command requests Gregory Szorc <gregory.szorc@gmail.com> parents: 37291 diff changeset	833 if expectingdata != entry['expectingdata']:
37061 c5e9c3b47366 wireproto: support for receiving multiple requests Gregory Szorc <gregory.szorc@gmail.com> parents: 37060 diff changeset	834 self._state = 'errored'
37292 3d0e2cd86e05 wireproto: use CBOR for command requests Gregory Szorc <gregory.szorc@gmail.com> parents: 37291 diff changeset	835 return self._makeerrorresult(
3d0e2cd86e05 wireproto: use CBOR for command requests Gregory Szorc <gregory.szorc@gmail.com> parents: 37291 diff changeset	836 _('mismatch between expect data flag and previous frame'))
3d0e2cd86e05 wireproto: use CBOR for command requests Gregory Szorc <gregory.szorc@gmail.com> parents: 37291 diff changeset	837
3d0e2cd86e05 wireproto: use CBOR for command requests Gregory Szorc <gregory.szorc@gmail.com> parents: 37291 diff changeset	838 entry['payload'].write(frame.payload)
37061 c5e9c3b47366 wireproto: support for receiving multiple requests Gregory Szorc <gregory.szorc@gmail.com> parents: 37060 diff changeset	839
37292 3d0e2cd86e05 wireproto: use CBOR for command requests Gregory Szorc <gregory.szorc@gmail.com> parents: 37291 diff changeset	840 if not moreframes:
3d0e2cd86e05 wireproto: use CBOR for command requests Gregory Szorc <gregory.szorc@gmail.com> parents: 37291 diff changeset	841 entry['requestdone'] = True
3d0e2cd86e05 wireproto: use CBOR for command requests Gregory Szorc <gregory.szorc@gmail.com> parents: 37291 diff changeset	842
3d0e2cd86e05 wireproto: use CBOR for command requests Gregory Szorc <gregory.szorc@gmail.com> parents: 37291 diff changeset	843 if not moreframes and not expectingdata:
3d0e2cd86e05 wireproto: use CBOR for command requests Gregory Szorc <gregory.szorc@gmail.com> parents: 37291 diff changeset	844 return self._makeruncommandresult(frame.requestid)
3d0e2cd86e05 wireproto: use CBOR for command requests Gregory Szorc <gregory.szorc@gmail.com> parents: 37291 diff changeset	845
3d0e2cd86e05 wireproto: use CBOR for command requests Gregory Szorc <gregory.szorc@gmail.com> parents: 37291 diff changeset	846 return self._makewantframeresult()
37061 c5e9c3b47366 wireproto: support for receiving multiple requests Gregory Szorc <gregory.szorc@gmail.com> parents: 37060 diff changeset	847
37064 884a0c1604ad wireproto: define attr-based classes for representing frames Gregory Szorc <gregory.szorc@gmail.com> parents: 37063 diff changeset	848 elif frame.typeid == FRAME_TYPE_COMMAND_DATA:
37061 c5e9c3b47366 wireproto: support for receiving multiple requests Gregory Szorc <gregory.szorc@gmail.com> parents: 37060 diff changeset	849 if not entry['expectingdata']:
c5e9c3b47366 wireproto: support for receiving multiple requests Gregory Szorc <gregory.szorc@gmail.com> parents: 37060 diff changeset	850 self._state = 'errored'
c5e9c3b47366 wireproto: support for receiving multiple requests Gregory Szorc <gregory.szorc@gmail.com> parents: 37060 diff changeset	851 return self._makeerrorresult(_(
c5e9c3b47366 wireproto: support for receiving multiple requests Gregory Szorc <gregory.szorc@gmail.com> parents: 37060 diff changeset	852 'received command data frame for request that is not '
37064 884a0c1604ad wireproto: define attr-based classes for representing frames Gregory Szorc <gregory.szorc@gmail.com> parents: 37063 diff changeset	853 'expecting data: %d') % frame.requestid)
37061 c5e9c3b47366 wireproto: support for receiving multiple requests Gregory Szorc <gregory.szorc@gmail.com> parents: 37060 diff changeset	854
c5e9c3b47366 wireproto: support for receiving multiple requests Gregory Szorc <gregory.szorc@gmail.com> parents: 37060 diff changeset	855 if entry['data'] is None:
c5e9c3b47366 wireproto: support for receiving multiple requests Gregory Szorc <gregory.szorc@gmail.com> parents: 37060 diff changeset	856 entry['data'] = util.bytesio()
c5e9c3b47366 wireproto: support for receiving multiple requests Gregory Szorc <gregory.szorc@gmail.com> parents: 37060 diff changeset	857
37064 884a0c1604ad wireproto: define attr-based classes for representing frames Gregory Szorc <gregory.szorc@gmail.com> parents: 37063 diff changeset	858 return self._handlecommanddataframe(frame, entry)
37292 3d0e2cd86e05 wireproto: use CBOR for command requests Gregory Szorc <gregory.szorc@gmail.com> parents: 37291 diff changeset	859 else:
37055 8c3c47362934 wireproto: implement basic frame reading and processing Gregory Szorc <gregory.szorc@gmail.com> parents: 37054 diff changeset	860 self._state = 'errored'
37292 3d0e2cd86e05 wireproto: use CBOR for command requests Gregory Szorc <gregory.szorc@gmail.com> parents: 37291 diff changeset	861 return self._makeerrorresult(_(
3d0e2cd86e05 wireproto: use CBOR for command requests Gregory Szorc <gregory.szorc@gmail.com> parents: 37291 diff changeset	862 'received unexpected frame type: %d') % frame.typeid)
37055 8c3c47362934 wireproto: implement basic frame reading and processing Gregory Szorc <gregory.szorc@gmail.com> parents: 37054 diff changeset	863
37064 884a0c1604ad wireproto: define attr-based classes for representing frames Gregory Szorc <gregory.szorc@gmail.com> parents: 37063 diff changeset	864 def _handlecommanddataframe(self, frame, entry):
884a0c1604ad wireproto: define attr-based classes for representing frames Gregory Szorc <gregory.szorc@gmail.com> parents: 37063 diff changeset	865 assert frame.typeid == FRAME_TYPE_COMMAND_DATA
37055 8c3c47362934 wireproto: implement basic frame reading and processing Gregory Szorc <gregory.szorc@gmail.com> parents: 37054 diff changeset	866
8c3c47362934 wireproto: implement basic frame reading and processing Gregory Szorc <gregory.szorc@gmail.com> parents: 37054 diff changeset	867 # TODO support streaming data instead of buffering it.
37064 884a0c1604ad wireproto: define attr-based classes for representing frames Gregory Szorc <gregory.szorc@gmail.com> parents: 37063 diff changeset	868 entry['data'].write(frame.payload)
37055 8c3c47362934 wireproto: implement basic frame reading and processing Gregory Szorc <gregory.szorc@gmail.com> parents: 37054 diff changeset	869
37064 884a0c1604ad wireproto: define attr-based classes for representing frames Gregory Szorc <gregory.szorc@gmail.com> parents: 37063 diff changeset	870 if frame.flags & FLAG_COMMAND_DATA_CONTINUATION:
37055 8c3c47362934 wireproto: implement basic frame reading and processing Gregory Szorc <gregory.szorc@gmail.com> parents: 37054 diff changeset	871 return self._makewantframeresult()
37064 884a0c1604ad wireproto: define attr-based classes for representing frames Gregory Szorc <gregory.szorc@gmail.com> parents: 37063 diff changeset	872 elif frame.flags & FLAG_COMMAND_DATA_EOS:
37061 c5e9c3b47366 wireproto: support for receiving multiple requests Gregory Szorc <gregory.szorc@gmail.com> parents: 37060 diff changeset	873 entry['data'].seek(0)
37064 884a0c1604ad wireproto: define attr-based classes for representing frames Gregory Szorc <gregory.szorc@gmail.com> parents: 37063 diff changeset	874 return self._makeruncommandresult(frame.requestid)
37055 8c3c47362934 wireproto: implement basic frame reading and processing Gregory Szorc <gregory.szorc@gmail.com> parents: 37054 diff changeset	875 else:
8c3c47362934 wireproto: implement basic frame reading and processing Gregory Szorc <gregory.szorc@gmail.com> parents: 37054 diff changeset	876 self._state = 'errored'
8c3c47362934 wireproto: implement basic frame reading and processing Gregory Szorc <gregory.szorc@gmail.com> parents: 37054 diff changeset	877 return self._makeerrorresult(_('command data frame without '
8c3c47362934 wireproto: implement basic frame reading and processing Gregory Szorc <gregory.szorc@gmail.com> parents: 37054 diff changeset	878 'flags'))
8c3c47362934 wireproto: implement basic frame reading and processing Gregory Szorc <gregory.szorc@gmail.com> parents: 37054 diff changeset	879
37064 884a0c1604ad wireproto: define attr-based classes for representing frames Gregory Szorc <gregory.szorc@gmail.com> parents: 37063 diff changeset	880 def _onframeerrored(self, frame):
37055 8c3c47362934 wireproto: implement basic frame reading and processing Gregory Szorc <gregory.szorc@gmail.com> parents: 37054 diff changeset	881 return self._makeerrorresult(_('server already errored'))
37543 01361be9e2dc wireproto: introduce a reactor for client-side state Gregory Szorc <gregory.szorc@gmail.com> parents: 37485 diff changeset	882
01361be9e2dc wireproto: introduce a reactor for client-side state Gregory Szorc <gregory.szorc@gmail.com> parents: 37485 diff changeset	883 class commandrequest(object):
01361be9e2dc wireproto: introduce a reactor for client-side state Gregory Szorc <gregory.szorc@gmail.com> parents: 37485 diff changeset	884 """Represents a request to run a command."""
01361be9e2dc wireproto: introduce a reactor for client-side state Gregory Szorc <gregory.szorc@gmail.com> parents: 37485 diff changeset	885
01361be9e2dc wireproto: introduce a reactor for client-side state Gregory Szorc <gregory.szorc@gmail.com> parents: 37485 diff changeset	886 def __init__(self, requestid, name, args, datafh=None):
01361be9e2dc wireproto: introduce a reactor for client-side state Gregory Szorc <gregory.szorc@gmail.com> parents: 37485 diff changeset	887 self.requestid = requestid
01361be9e2dc wireproto: introduce a reactor for client-side state Gregory Szorc <gregory.szorc@gmail.com> parents: 37485 diff changeset	888 self.name = name
01361be9e2dc wireproto: introduce a reactor for client-side state Gregory Szorc <gregory.szorc@gmail.com> parents: 37485 diff changeset	889 self.args = args
01361be9e2dc wireproto: introduce a reactor for client-side state Gregory Szorc <gregory.szorc@gmail.com> parents: 37485 diff changeset	890 self.datafh = datafh
01361be9e2dc wireproto: introduce a reactor for client-side state Gregory Szorc <gregory.szorc@gmail.com> parents: 37485 diff changeset	891 self.state = 'pending'
01361be9e2dc wireproto: introduce a reactor for client-side state Gregory Szorc <gregory.szorc@gmail.com> parents: 37485 diff changeset	892
01361be9e2dc wireproto: introduce a reactor for client-side state Gregory Szorc <gregory.szorc@gmail.com> parents: 37485 diff changeset	893 class clientreactor(object):
01361be9e2dc wireproto: introduce a reactor for client-side state Gregory Szorc <gregory.szorc@gmail.com> parents: 37485 diff changeset	894 """Holds state of a client issuing frame-based protocol requests.
01361be9e2dc wireproto: introduce a reactor for client-side state Gregory Szorc <gregory.szorc@gmail.com> parents: 37485 diff changeset	895
01361be9e2dc wireproto: introduce a reactor for client-side state Gregory Szorc <gregory.szorc@gmail.com> parents: 37485 diff changeset	896 This is like ``serverreactor`` but for client-side state.
01361be9e2dc wireproto: introduce a reactor for client-side state Gregory Szorc <gregory.szorc@gmail.com> parents: 37485 diff changeset	897
01361be9e2dc wireproto: introduce a reactor for client-side state Gregory Szorc <gregory.szorc@gmail.com> parents: 37485 diff changeset	898 Each instance is bound to the lifetime of a connection. For persistent
01361be9e2dc wireproto: introduce a reactor for client-side state Gregory Szorc <gregory.szorc@gmail.com> parents: 37485 diff changeset	899 connection transports using e.g. TCP sockets and speaking the raw
01361be9e2dc wireproto: introduce a reactor for client-side state Gregory Szorc <gregory.szorc@gmail.com> parents: 37485 diff changeset	900 framing protocol, there will be a single instance for the lifetime of
01361be9e2dc wireproto: introduce a reactor for client-side state Gregory Szorc <gregory.szorc@gmail.com> parents: 37485 diff changeset	901 the TCP socket. For transports where there are multiple discrete
01361be9e2dc wireproto: introduce a reactor for client-side state Gregory Szorc <gregory.szorc@gmail.com> parents: 37485 diff changeset	902 interactions (say tunneled within in HTTP request), there will be a
01361be9e2dc wireproto: introduce a reactor for client-side state Gregory Szorc <gregory.szorc@gmail.com> parents: 37485 diff changeset	903 separate instance for each distinct interaction.
01361be9e2dc wireproto: introduce a reactor for client-side state Gregory Szorc <gregory.szorc@gmail.com> parents: 37485 diff changeset	904 """
01361be9e2dc wireproto: introduce a reactor for client-side state Gregory Szorc <gregory.szorc@gmail.com> parents: 37485 diff changeset	905 def __init__(self, hasmultiplesend=False, buffersends=True):
01361be9e2dc wireproto: introduce a reactor for client-side state Gregory Szorc <gregory.szorc@gmail.com> parents: 37485 diff changeset	906 """Create a new instance.
01361be9e2dc wireproto: introduce a reactor for client-side state Gregory Szorc <gregory.szorc@gmail.com> parents: 37485 diff changeset	907
01361be9e2dc wireproto: introduce a reactor for client-side state Gregory Szorc <gregory.szorc@gmail.com> parents: 37485 diff changeset	908 ``hasmultiplesend`` indicates whether multiple sends are supported
01361be9e2dc wireproto: introduce a reactor for client-side state Gregory Szorc <gregory.szorc@gmail.com> parents: 37485 diff changeset	909 by the transport. When True, it is possible to send commands immediately
01361be9e2dc wireproto: introduce a reactor for client-side state Gregory Szorc <gregory.szorc@gmail.com> parents: 37485 diff changeset	910 instead of buffering until the caller signals an intent to finish a
01361be9e2dc wireproto: introduce a reactor for client-side state Gregory Szorc <gregory.szorc@gmail.com> parents: 37485 diff changeset	911 send operation.
01361be9e2dc wireproto: introduce a reactor for client-side state Gregory Szorc <gregory.szorc@gmail.com> parents: 37485 diff changeset	912
01361be9e2dc wireproto: introduce a reactor for client-side state Gregory Szorc <gregory.szorc@gmail.com> parents: 37485 diff changeset	913 ``buffercommands`` indicates whether sends should be buffered until the
01361be9e2dc wireproto: introduce a reactor for client-side state Gregory Szorc <gregory.szorc@gmail.com> parents: 37485 diff changeset	914 last request has been issued.
01361be9e2dc wireproto: introduce a reactor for client-side state Gregory Szorc <gregory.szorc@gmail.com> parents: 37485 diff changeset	915 """
01361be9e2dc wireproto: introduce a reactor for client-side state Gregory Szorc <gregory.szorc@gmail.com> parents: 37485 diff changeset	916 self._hasmultiplesend = hasmultiplesend
01361be9e2dc wireproto: introduce a reactor for client-side state Gregory Szorc <gregory.szorc@gmail.com> parents: 37485 diff changeset	917 self._buffersends = buffersends
01361be9e2dc wireproto: introduce a reactor for client-side state Gregory Szorc <gregory.szorc@gmail.com> parents: 37485 diff changeset	918
01361be9e2dc wireproto: introduce a reactor for client-side state Gregory Szorc <gregory.szorc@gmail.com> parents: 37485 diff changeset	919 self._canissuecommands = True
01361be9e2dc wireproto: introduce a reactor for client-side state Gregory Szorc <gregory.szorc@gmail.com> parents: 37485 diff changeset	920 self._cansend = True
01361be9e2dc wireproto: introduce a reactor for client-side state Gregory Szorc <gregory.szorc@gmail.com> parents: 37485 diff changeset	921
01361be9e2dc wireproto: introduce a reactor for client-side state Gregory Szorc <gregory.szorc@gmail.com> parents: 37485 diff changeset	922 self._nextrequestid = 1
01361be9e2dc wireproto: introduce a reactor for client-side state Gregory Szorc <gregory.szorc@gmail.com> parents: 37485 diff changeset	923 # We only support a single outgoing stream for now.
01361be9e2dc wireproto: introduce a reactor for client-side state Gregory Szorc <gregory.szorc@gmail.com> parents: 37485 diff changeset	924 self._outgoingstream = stream(1)
01361be9e2dc wireproto: introduce a reactor for client-side state Gregory Szorc <gregory.szorc@gmail.com> parents: 37485 diff changeset	925 self._pendingrequests = collections.deque()
01361be9e2dc wireproto: introduce a reactor for client-side state Gregory Szorc <gregory.szorc@gmail.com> parents: 37485 diff changeset	926 self._activerequests = {}
37544 55b5ba8d4e68 wireproto: client reactor support for receiving frames Gregory Szorc <gregory.szorc@gmail.com> parents: 37543 diff changeset	927 self._incomingstreams = {}
37543 01361be9e2dc wireproto: introduce a reactor for client-side state Gregory Szorc <gregory.szorc@gmail.com> parents: 37485 diff changeset	928
01361be9e2dc wireproto: introduce a reactor for client-side state Gregory Szorc <gregory.szorc@gmail.com> parents: 37485 diff changeset	929 def callcommand(self, name, args, datafh=None):
01361be9e2dc wireproto: introduce a reactor for client-side state Gregory Szorc <gregory.szorc@gmail.com> parents: 37485 diff changeset	930 """Request that a command be executed.
01361be9e2dc wireproto: introduce a reactor for client-side state Gregory Szorc <gregory.szorc@gmail.com> parents: 37485 diff changeset	931
01361be9e2dc wireproto: introduce a reactor for client-side state Gregory Szorc <gregory.szorc@gmail.com> parents: 37485 diff changeset	932 Receives the command name, a dict of arguments to pass to the command,
01361be9e2dc wireproto: introduce a reactor for client-side state Gregory Szorc <gregory.szorc@gmail.com> parents: 37485 diff changeset	933 and an optional file object containing the raw data for the command.
01361be9e2dc wireproto: introduce a reactor for client-side state Gregory Szorc <gregory.szorc@gmail.com> parents: 37485 diff changeset	934
01361be9e2dc wireproto: introduce a reactor for client-side state Gregory Szorc <gregory.szorc@gmail.com> parents: 37485 diff changeset	935 Returns a 3-tuple of (request, action, action data).
01361be9e2dc wireproto: introduce a reactor for client-side state Gregory Szorc <gregory.szorc@gmail.com> parents: 37485 diff changeset	936 """
01361be9e2dc wireproto: introduce a reactor for client-side state Gregory Szorc <gregory.szorc@gmail.com> parents: 37485 diff changeset	937 if not self._canissuecommands:
01361be9e2dc wireproto: introduce a reactor for client-side state Gregory Szorc <gregory.szorc@gmail.com> parents: 37485 diff changeset	938 raise error.ProgrammingError('cannot issue new commands')
01361be9e2dc wireproto: introduce a reactor for client-side state Gregory Szorc <gregory.szorc@gmail.com> parents: 37485 diff changeset	939
01361be9e2dc wireproto: introduce a reactor for client-side state Gregory Szorc <gregory.szorc@gmail.com> parents: 37485 diff changeset	940 requestid = self._nextrequestid
01361be9e2dc wireproto: introduce a reactor for client-side state Gregory Szorc <gregory.szorc@gmail.com> parents: 37485 diff changeset	941 self._nextrequestid += 2
01361be9e2dc wireproto: introduce a reactor for client-side state Gregory Szorc <gregory.szorc@gmail.com> parents: 37485 diff changeset	942
01361be9e2dc wireproto: introduce a reactor for client-side state Gregory Szorc <gregory.szorc@gmail.com> parents: 37485 diff changeset	943 request = commandrequest(requestid, name, args, datafh=datafh)
01361be9e2dc wireproto: introduce a reactor for client-side state Gregory Szorc <gregory.szorc@gmail.com> parents: 37485 diff changeset	944
01361be9e2dc wireproto: introduce a reactor for client-side state Gregory Szorc <gregory.szorc@gmail.com> parents: 37485 diff changeset	945 if self._buffersends:
01361be9e2dc wireproto: introduce a reactor for client-side state Gregory Szorc <gregory.szorc@gmail.com> parents: 37485 diff changeset	946 self._pendingrequests.append(request)
01361be9e2dc wireproto: introduce a reactor for client-side state Gregory Szorc <gregory.szorc@gmail.com> parents: 37485 diff changeset	947 return request, 'noop', {}
01361be9e2dc wireproto: introduce a reactor for client-side state Gregory Szorc <gregory.szorc@gmail.com> parents: 37485 diff changeset	948 else:
01361be9e2dc wireproto: introduce a reactor for client-side state Gregory Szorc <gregory.szorc@gmail.com> parents: 37485 diff changeset	949 if not self._cansend:
01361be9e2dc wireproto: introduce a reactor for client-side state Gregory Szorc <gregory.szorc@gmail.com> parents: 37485 diff changeset	950 raise error.ProgrammingError('sends cannot be performed on '
01361be9e2dc wireproto: introduce a reactor for client-side state Gregory Szorc <gregory.szorc@gmail.com> parents: 37485 diff changeset	951 'this instance')
01361be9e2dc wireproto: introduce a reactor for client-side state Gregory Szorc <gregory.szorc@gmail.com> parents: 37485 diff changeset	952
01361be9e2dc wireproto: introduce a reactor for client-side state Gregory Szorc <gregory.szorc@gmail.com> parents: 37485 diff changeset	953 if not self._hasmultiplesend:
01361be9e2dc wireproto: introduce a reactor for client-side state Gregory Szorc <gregory.szorc@gmail.com> parents: 37485 diff changeset	954 self._cansend = False
01361be9e2dc wireproto: introduce a reactor for client-side state Gregory Szorc <gregory.szorc@gmail.com> parents: 37485 diff changeset	955 self._canissuecommands = False
01361be9e2dc wireproto: introduce a reactor for client-side state Gregory Szorc <gregory.szorc@gmail.com> parents: 37485 diff changeset	956
01361be9e2dc wireproto: introduce a reactor for client-side state Gregory Szorc <gregory.szorc@gmail.com> parents: 37485 diff changeset	957 return request, 'sendframes', {
01361be9e2dc wireproto: introduce a reactor for client-side state Gregory Szorc <gregory.szorc@gmail.com> parents: 37485 diff changeset	958 'framegen': self._makecommandframes(request),
01361be9e2dc wireproto: introduce a reactor for client-side state Gregory Szorc <gregory.szorc@gmail.com> parents: 37485 diff changeset	959 }
01361be9e2dc wireproto: introduce a reactor for client-side state Gregory Szorc <gregory.szorc@gmail.com> parents: 37485 diff changeset	960
01361be9e2dc wireproto: introduce a reactor for client-side state Gregory Szorc <gregory.szorc@gmail.com> parents: 37485 diff changeset	961 def flushcommands(self):
01361be9e2dc wireproto: introduce a reactor for client-side state Gregory Szorc <gregory.szorc@gmail.com> parents: 37485 diff changeset	962 """Request that all queued commands be sent.
01361be9e2dc wireproto: introduce a reactor for client-side state Gregory Szorc <gregory.szorc@gmail.com> parents: 37485 diff changeset	963
01361be9e2dc wireproto: introduce a reactor for client-side state Gregory Szorc <gregory.szorc@gmail.com> parents: 37485 diff changeset	964 If any commands are buffered, this will instruct the caller to send
01361be9e2dc wireproto: introduce a reactor for client-side state Gregory Szorc <gregory.szorc@gmail.com> parents: 37485 diff changeset	965 them over the wire. If no commands are buffered it instructs the client
01361be9e2dc wireproto: introduce a reactor for client-side state Gregory Szorc <gregory.szorc@gmail.com> parents: 37485 diff changeset	966 to no-op.
01361be9e2dc wireproto: introduce a reactor for client-side state Gregory Szorc <gregory.szorc@gmail.com> parents: 37485 diff changeset	967
01361be9e2dc wireproto: introduce a reactor for client-side state Gregory Szorc <gregory.szorc@gmail.com> parents: 37485 diff changeset	968 If instances aren't configured for multiple sends, no new command
01361be9e2dc wireproto: introduce a reactor for client-side state Gregory Szorc <gregory.szorc@gmail.com> parents: 37485 diff changeset	969 requests are allowed after this is called.
01361be9e2dc wireproto: introduce a reactor for client-side state Gregory Szorc <gregory.szorc@gmail.com> parents: 37485 diff changeset	970 """
01361be9e2dc wireproto: introduce a reactor for client-side state Gregory Szorc <gregory.szorc@gmail.com> parents: 37485 diff changeset	971 if not self._pendingrequests:
01361be9e2dc wireproto: introduce a reactor for client-side state Gregory Szorc <gregory.szorc@gmail.com> parents: 37485 diff changeset	972 return 'noop', {}
01361be9e2dc wireproto: introduce a reactor for client-side state Gregory Szorc <gregory.szorc@gmail.com> parents: 37485 diff changeset	973
01361be9e2dc wireproto: introduce a reactor for client-side state Gregory Szorc <gregory.szorc@gmail.com> parents: 37485 diff changeset	974 if not self._cansend:
01361be9e2dc wireproto: introduce a reactor for client-side state Gregory Szorc <gregory.szorc@gmail.com> parents: 37485 diff changeset	975 raise error.ProgrammingError('sends cannot be performed on this '
01361be9e2dc wireproto: introduce a reactor for client-side state Gregory Szorc <gregory.szorc@gmail.com> parents: 37485 diff changeset	976 'instance')
01361be9e2dc wireproto: introduce a reactor for client-side state Gregory Szorc <gregory.szorc@gmail.com> parents: 37485 diff changeset	977
01361be9e2dc wireproto: introduce a reactor for client-side state Gregory Szorc <gregory.szorc@gmail.com> parents: 37485 diff changeset	978 # If the instance only allows sending once, mark that we have fired
01361be9e2dc wireproto: introduce a reactor for client-side state Gregory Szorc <gregory.szorc@gmail.com> parents: 37485 diff changeset	979 # our one shot.
01361be9e2dc wireproto: introduce a reactor for client-side state Gregory Szorc <gregory.szorc@gmail.com> parents: 37485 diff changeset	980 if not self._hasmultiplesend:
01361be9e2dc wireproto: introduce a reactor for client-side state Gregory Szorc <gregory.szorc@gmail.com> parents: 37485 diff changeset	981 self._canissuecommands = False
01361be9e2dc wireproto: introduce a reactor for client-side state Gregory Szorc <gregory.szorc@gmail.com> parents: 37485 diff changeset	982 self._cansend = False
01361be9e2dc wireproto: introduce a reactor for client-side state Gregory Szorc <gregory.szorc@gmail.com> parents: 37485 diff changeset	983
01361be9e2dc wireproto: introduce a reactor for client-side state Gregory Szorc <gregory.szorc@gmail.com> parents: 37485 diff changeset	984 def makeframes():
01361be9e2dc wireproto: introduce a reactor for client-side state Gregory Szorc <gregory.szorc@gmail.com> parents: 37485 diff changeset	985 while self._pendingrequests:
01361be9e2dc wireproto: introduce a reactor for client-side state Gregory Szorc <gregory.szorc@gmail.com> parents: 37485 diff changeset	986 request = self._pendingrequests.popleft()
01361be9e2dc wireproto: introduce a reactor for client-side state Gregory Szorc <gregory.szorc@gmail.com> parents: 37485 diff changeset	987 for frame in self._makecommandframes(request):
01361be9e2dc wireproto: introduce a reactor for client-side state Gregory Szorc <gregory.szorc@gmail.com> parents: 37485 diff changeset	988 yield frame
01361be9e2dc wireproto: introduce a reactor for client-side state Gregory Szorc <gregory.szorc@gmail.com> parents: 37485 diff changeset	989
01361be9e2dc wireproto: introduce a reactor for client-side state Gregory Szorc <gregory.szorc@gmail.com> parents: 37485 diff changeset	990 return 'sendframes', {
01361be9e2dc wireproto: introduce a reactor for client-side state Gregory Szorc <gregory.szorc@gmail.com> parents: 37485 diff changeset	991 'framegen': makeframes(),
01361be9e2dc wireproto: introduce a reactor for client-side state Gregory Szorc <gregory.szorc@gmail.com> parents: 37485 diff changeset	992 }
01361be9e2dc wireproto: introduce a reactor for client-side state Gregory Szorc <gregory.szorc@gmail.com> parents: 37485 diff changeset	993
01361be9e2dc wireproto: introduce a reactor for client-side state Gregory Szorc <gregory.szorc@gmail.com> parents: 37485 diff changeset	994 def _makecommandframes(self, request):
01361be9e2dc wireproto: introduce a reactor for client-side state Gregory Szorc <gregory.szorc@gmail.com> parents: 37485 diff changeset	995 """Emit frames to issue a command request.
01361be9e2dc wireproto: introduce a reactor for client-side state Gregory Szorc <gregory.szorc@gmail.com> parents: 37485 diff changeset	996
01361be9e2dc wireproto: introduce a reactor for client-side state Gregory Szorc <gregory.szorc@gmail.com> parents: 37485 diff changeset	997 As a side-effect, update request accounting to reflect its changed
01361be9e2dc wireproto: introduce a reactor for client-side state Gregory Szorc <gregory.szorc@gmail.com> parents: 37485 diff changeset	998 state.
01361be9e2dc wireproto: introduce a reactor for client-side state Gregory Szorc <gregory.szorc@gmail.com> parents: 37485 diff changeset	999 """
01361be9e2dc wireproto: introduce a reactor for client-side state Gregory Szorc <gregory.szorc@gmail.com> parents: 37485 diff changeset	1000 self._activerequests[request.requestid] = request
01361be9e2dc wireproto: introduce a reactor for client-side state Gregory Szorc <gregory.szorc@gmail.com> parents: 37485 diff changeset	1001 request.state = 'sending'
01361be9e2dc wireproto: introduce a reactor for client-side state Gregory Szorc <gregory.szorc@gmail.com> parents: 37485 diff changeset	1002
01361be9e2dc wireproto: introduce a reactor for client-side state Gregory Szorc <gregory.szorc@gmail.com> parents: 37485 diff changeset	1003 res = createcommandframes(self._outgoingstream,
01361be9e2dc wireproto: introduce a reactor for client-side state Gregory Szorc <gregory.szorc@gmail.com> parents: 37485 diff changeset	1004 request.requestid,
01361be9e2dc wireproto: introduce a reactor for client-side state Gregory Szorc <gregory.szorc@gmail.com> parents: 37485 diff changeset	1005 request.name,
01361be9e2dc wireproto: introduce a reactor for client-side state Gregory Szorc <gregory.szorc@gmail.com> parents: 37485 diff changeset	1006 request.args,
01361be9e2dc wireproto: introduce a reactor for client-side state Gregory Szorc <gregory.szorc@gmail.com> parents: 37485 diff changeset	1007 request.datafh)
01361be9e2dc wireproto: introduce a reactor for client-side state Gregory Szorc <gregory.szorc@gmail.com> parents: 37485 diff changeset	1008
01361be9e2dc wireproto: introduce a reactor for client-side state Gregory Szorc <gregory.szorc@gmail.com> parents: 37485 diff changeset	1009 for frame in res:
01361be9e2dc wireproto: introduce a reactor for client-side state Gregory Szorc <gregory.szorc@gmail.com> parents: 37485 diff changeset	1010 yield frame
01361be9e2dc wireproto: introduce a reactor for client-side state Gregory Szorc <gregory.szorc@gmail.com> parents: 37485 diff changeset	1011
01361be9e2dc wireproto: introduce a reactor for client-side state Gregory Szorc <gregory.szorc@gmail.com> parents: 37485 diff changeset	1012 request.state = 'sent'
37544 55b5ba8d4e68 wireproto: client reactor support for receiving frames Gregory Szorc <gregory.szorc@gmail.com> parents: 37543 diff changeset	1013
55b5ba8d4e68 wireproto: client reactor support for receiving frames Gregory Szorc <gregory.szorc@gmail.com> parents: 37543 diff changeset	1014 def onframerecv(self, frame):
55b5ba8d4e68 wireproto: client reactor support for receiving frames Gregory Szorc <gregory.szorc@gmail.com> parents: 37543 diff changeset	1015 """Process a frame that has been received off the wire.
55b5ba8d4e68 wireproto: client reactor support for receiving frames Gregory Szorc <gregory.szorc@gmail.com> parents: 37543 diff changeset	1016
55b5ba8d4e68 wireproto: client reactor support for receiving frames Gregory Szorc <gregory.szorc@gmail.com> parents: 37543 diff changeset	1017 Returns a 2-tuple of (action, meta) describing further action the
55b5ba8d4e68 wireproto: client reactor support for receiving frames Gregory Szorc <gregory.szorc@gmail.com> parents: 37543 diff changeset	1018 caller needs to take as a result of receiving this frame.
55b5ba8d4e68 wireproto: client reactor support for receiving frames Gregory Szorc <gregory.szorc@gmail.com> parents: 37543 diff changeset	1019 """
55b5ba8d4e68 wireproto: client reactor support for receiving frames Gregory Szorc <gregory.szorc@gmail.com> parents: 37543 diff changeset	1020 if frame.streamid % 2:
55b5ba8d4e68 wireproto: client reactor support for receiving frames Gregory Szorc <gregory.szorc@gmail.com> parents: 37543 diff changeset	1021 return 'error', {
55b5ba8d4e68 wireproto: client reactor support for receiving frames Gregory Szorc <gregory.szorc@gmail.com> parents: 37543 diff changeset	1022 'message': (
55b5ba8d4e68 wireproto: client reactor support for receiving frames Gregory Szorc <gregory.szorc@gmail.com> parents: 37543 diff changeset	1023 _('received frame with odd numbered stream ID: %d') %
55b5ba8d4e68 wireproto: client reactor support for receiving frames Gregory Szorc <gregory.szorc@gmail.com> parents: 37543 diff changeset	1024 frame.streamid),
55b5ba8d4e68 wireproto: client reactor support for receiving frames Gregory Szorc <gregory.szorc@gmail.com> parents: 37543 diff changeset	1025 }
55b5ba8d4e68 wireproto: client reactor support for receiving frames Gregory Szorc <gregory.szorc@gmail.com> parents: 37543 diff changeset	1026
55b5ba8d4e68 wireproto: client reactor support for receiving frames Gregory Szorc <gregory.szorc@gmail.com> parents: 37543 diff changeset	1027 if frame.streamid not in self._incomingstreams:
55b5ba8d4e68 wireproto: client reactor support for receiving frames Gregory Szorc <gregory.szorc@gmail.com> parents: 37543 diff changeset	1028 if not frame.streamflags & STREAM_FLAG_BEGIN_STREAM:
55b5ba8d4e68 wireproto: client reactor support for receiving frames Gregory Szorc <gregory.szorc@gmail.com> parents: 37543 diff changeset	1029 return 'error', {
55b5ba8d4e68 wireproto: client reactor support for receiving frames Gregory Szorc <gregory.szorc@gmail.com> parents: 37543 diff changeset	1030 'message': _('received frame on unknown stream '
55b5ba8d4e68 wireproto: client reactor support for receiving frames Gregory Szorc <gregory.szorc@gmail.com> parents: 37543 diff changeset	1031 'without beginning of stream flag set'),
55b5ba8d4e68 wireproto: client reactor support for receiving frames Gregory Szorc <gregory.szorc@gmail.com> parents: 37543 diff changeset	1032 }
55b5ba8d4e68 wireproto: client reactor support for receiving frames Gregory Szorc <gregory.szorc@gmail.com> parents: 37543 diff changeset	1033
37656 e6870bca1f47 wireprotoframing: record when new stream is encountered Gregory Szorc <gregory.szorc@gmail.com> parents: 37655 diff changeset	1034 self._incomingstreams[frame.streamid] = stream(frame.streamid)
e6870bca1f47 wireprotoframing: record when new stream is encountered Gregory Szorc <gregory.szorc@gmail.com> parents: 37655 diff changeset	1035
37544 55b5ba8d4e68 wireproto: client reactor support for receiving frames Gregory Szorc <gregory.szorc@gmail.com> parents: 37543 diff changeset	1036 if frame.streamflags & STREAM_FLAG_ENCODING_APPLIED:
55b5ba8d4e68 wireproto: client reactor support for receiving frames Gregory Szorc <gregory.szorc@gmail.com> parents: 37543 diff changeset	1037 raise error.ProgrammingError('support for decoding stream '
55b5ba8d4e68 wireproto: client reactor support for receiving frames Gregory Szorc <gregory.szorc@gmail.com> parents: 37543 diff changeset	1038 'payloads not yet implemneted')
55b5ba8d4e68 wireproto: client reactor support for receiving frames Gregory Szorc <gregory.szorc@gmail.com> parents: 37543 diff changeset	1039
55b5ba8d4e68 wireproto: client reactor support for receiving frames Gregory Szorc <gregory.szorc@gmail.com> parents: 37543 diff changeset	1040 if frame.streamflags & STREAM_FLAG_END_STREAM:
55b5ba8d4e68 wireproto: client reactor support for receiving frames Gregory Szorc <gregory.szorc@gmail.com> parents: 37543 diff changeset	1041 del self._incomingstreams[frame.streamid]
55b5ba8d4e68 wireproto: client reactor support for receiving frames Gregory Szorc <gregory.szorc@gmail.com> parents: 37543 diff changeset	1042
55b5ba8d4e68 wireproto: client reactor support for receiving frames Gregory Szorc <gregory.szorc@gmail.com> parents: 37543 diff changeset	1043 if frame.requestid not in self._activerequests:
55b5ba8d4e68 wireproto: client reactor support for receiving frames Gregory Szorc <gregory.szorc@gmail.com> parents: 37543 diff changeset	1044 return 'error', {
55b5ba8d4e68 wireproto: client reactor support for receiving frames Gregory Szorc <gregory.szorc@gmail.com> parents: 37543 diff changeset	1045 'message': (_('received frame for inactive request ID: %d') %
55b5ba8d4e68 wireproto: client reactor support for receiving frames Gregory Szorc <gregory.szorc@gmail.com> parents: 37543 diff changeset	1046 frame.requestid),
55b5ba8d4e68 wireproto: client reactor support for receiving frames Gregory Szorc <gregory.szorc@gmail.com> parents: 37543 diff changeset	1047 }
55b5ba8d4e68 wireproto: client reactor support for receiving frames Gregory Szorc <gregory.szorc@gmail.com> parents: 37543 diff changeset	1048
55b5ba8d4e68 wireproto: client reactor support for receiving frames Gregory Szorc <gregory.szorc@gmail.com> parents: 37543 diff changeset	1049 request = self._activerequests[frame.requestid]
55b5ba8d4e68 wireproto: client reactor support for receiving frames Gregory Szorc <gregory.szorc@gmail.com> parents: 37543 diff changeset	1050 request.state = 'receiving'
55b5ba8d4e68 wireproto: client reactor support for receiving frames Gregory Szorc <gregory.szorc@gmail.com> parents: 37543 diff changeset	1051
55b5ba8d4e68 wireproto: client reactor support for receiving frames Gregory Szorc <gregory.szorc@gmail.com> parents: 37543 diff changeset	1052 handlers = {
37724 deff7cf7eefd wireprotov2: change frame type and name for command response Gregory Szorc <gregory.szorc@gmail.com> parents: 37723 diff changeset	1053 FRAME_TYPE_COMMAND_RESPONSE: self._oncommandresponseframe,
37544 55b5ba8d4e68 wireproto: client reactor support for receiving frames Gregory Szorc <gregory.szorc@gmail.com> parents: 37543 diff changeset	1054 }
55b5ba8d4e68 wireproto: client reactor support for receiving frames Gregory Szorc <gregory.szorc@gmail.com> parents: 37543 diff changeset	1055
55b5ba8d4e68 wireproto: client reactor support for receiving frames Gregory Szorc <gregory.szorc@gmail.com> parents: 37543 diff changeset	1056 meth = handlers.get(frame.typeid)
55b5ba8d4e68 wireproto: client reactor support for receiving frames Gregory Szorc <gregory.szorc@gmail.com> parents: 37543 diff changeset	1057 if not meth:
55b5ba8d4e68 wireproto: client reactor support for receiving frames Gregory Szorc <gregory.szorc@gmail.com> parents: 37543 diff changeset	1058 raise error.ProgrammingError('unhandled frame type: %d' %
55b5ba8d4e68 wireproto: client reactor support for receiving frames Gregory Szorc <gregory.szorc@gmail.com> parents: 37543 diff changeset	1059 frame.typeid)
55b5ba8d4e68 wireproto: client reactor support for receiving frames Gregory Szorc <gregory.szorc@gmail.com> parents: 37543 diff changeset	1060
55b5ba8d4e68 wireproto: client reactor support for receiving frames Gregory Szorc <gregory.szorc@gmail.com> parents: 37543 diff changeset	1061 return meth(request, frame)
55b5ba8d4e68 wireproto: client reactor support for receiving frames Gregory Szorc <gregory.szorc@gmail.com> parents: 37543 diff changeset	1062
37724 deff7cf7eefd wireprotov2: change frame type and name for command response Gregory Szorc <gregory.szorc@gmail.com> parents: 37723 diff changeset	1063 def _oncommandresponseframe(self, request, frame):
deff7cf7eefd wireprotov2: change frame type and name for command response Gregory Szorc <gregory.szorc@gmail.com> parents: 37723 diff changeset	1064 if frame.flags & FLAG_COMMAND_RESPONSE_EOS:
37544 55b5ba8d4e68 wireproto: client reactor support for receiving frames Gregory Szorc <gregory.szorc@gmail.com> parents: 37543 diff changeset	1065 request.state = 'received'
55b5ba8d4e68 wireproto: client reactor support for receiving frames Gregory Szorc <gregory.szorc@gmail.com> parents: 37543 diff changeset	1066 del self._activerequests[request.requestid]
55b5ba8d4e68 wireproto: client reactor support for receiving frames Gregory Szorc <gregory.szorc@gmail.com> parents: 37543 diff changeset	1067
55b5ba8d4e68 wireproto: client reactor support for receiving frames Gregory Szorc <gregory.szorc@gmail.com> parents: 37543 diff changeset	1068 return 'responsedata', {
55b5ba8d4e68 wireproto: client reactor support for receiving frames Gregory Szorc <gregory.szorc@gmail.com> parents: 37543 diff changeset	1069 'request': request,
37724 deff7cf7eefd wireprotov2: change frame type and name for command response Gregory Szorc <gregory.szorc@gmail.com> parents: 37723 diff changeset	1070 'expectmore': frame.flags & FLAG_COMMAND_RESPONSE_CONTINUATION,
deff7cf7eefd wireprotov2: change frame type and name for command response Gregory Szorc <gregory.szorc@gmail.com> parents: 37723 diff changeset	1071 'eos': frame.flags & FLAG_COMMAND_RESPONSE_EOS,
37544 55b5ba8d4e68 wireproto: client reactor support for receiving frames Gregory Szorc <gregory.szorc@gmail.com> parents: 37543 diff changeset	1072 'data': frame.payload,
55b5ba8d4e68 wireproto: client reactor support for receiving frames Gregory Szorc <gregory.szorc@gmail.com> parents: 37543 diff changeset	1073 }

Mercurial > public > mercurial-scm > hg-stable

annotate mercurial/wireprotoframing.py @ 37725:3ea8323d6f95