mercurial-scm/hg-stable: mercurial/wireprotoframing.py annotate

annotate mercurial/wireprotoframing.py @ 37298:5ef2da00e935

wireproto: implement custom __repr__ for frame This version won't print the full payload (which could be large). It also prints human friendly values for types and flags. Differential Revision: https://phab.mercurial-scm.org/D2985

author	Gregory Szorc <gregory.szorc@gmail.com>
date	Wed, 28 Mar 2018 13:01:28 -0700
parents	3d0e2cd86e05
children	e9aadee698cf

rev	line source
37054 40206e227412 wireproto: define and implement protocol for issuing requests Gregory Szorc <gregory.szorc@gmail.com> parents: diff changeset	1 # wireprotoframing.py - unified framing protocol for wire protocol
40206e227412 wireproto: define and implement protocol for issuing requests Gregory Szorc <gregory.szorc@gmail.com> parents: diff changeset	2 #
40206e227412 wireproto: define and implement protocol for issuing requests Gregory Szorc <gregory.szorc@gmail.com> parents: diff changeset	3 # Copyright 2018 Gregory Szorc <gregory.szorc@gmail.com>
40206e227412 wireproto: define and implement protocol for issuing requests Gregory Szorc <gregory.szorc@gmail.com> parents: diff changeset	4 #
40206e227412 wireproto: define and implement protocol for issuing requests Gregory Szorc <gregory.szorc@gmail.com> parents: diff changeset	5 # This software may be used and distributed according to the terms of the
40206e227412 wireproto: define and implement protocol for issuing requests Gregory Szorc <gregory.szorc@gmail.com> parents: diff changeset	6 # GNU General Public License version 2 or any later version.
40206e227412 wireproto: define and implement protocol for issuing requests Gregory Szorc <gregory.szorc@gmail.com> parents: diff changeset	7
40206e227412 wireproto: define and implement protocol for issuing requests Gregory Szorc <gregory.szorc@gmail.com> parents: diff changeset	8 # This file contains functionality to support the unified frame-based wire
40206e227412 wireproto: define and implement protocol for issuing requests Gregory Szorc <gregory.szorc@gmail.com> parents: diff changeset	9 # protocol. For details about the protocol, see
40206e227412 wireproto: define and implement protocol for issuing requests Gregory Szorc <gregory.szorc@gmail.com> parents: diff changeset	10 # `hg help internals.wireprotocol`.
40206e227412 wireproto: define and implement protocol for issuing requests Gregory Szorc <gregory.szorc@gmail.com> parents: diff changeset	11
40206e227412 wireproto: define and implement protocol for issuing requests Gregory Szorc <gregory.szorc@gmail.com> parents: diff changeset	12 from __future__ import absolute_import
40206e227412 wireproto: define and implement protocol for issuing requests Gregory Szorc <gregory.szorc@gmail.com> parents: diff changeset	13
40206e227412 wireproto: define and implement protocol for issuing requests Gregory Szorc <gregory.szorc@gmail.com> parents: diff changeset	14 import struct
40206e227412 wireproto: define and implement protocol for issuing requests Gregory Szorc <gregory.szorc@gmail.com> parents: diff changeset	15
37055 8c3c47362934 wireproto: implement basic frame reading and processing Gregory Szorc <gregory.szorc@gmail.com> parents: 37054 diff changeset	16 from .i18n import _
37064 884a0c1604ad wireproto: define attr-based classes for representing frames Gregory Szorc <gregory.szorc@gmail.com> parents: 37063 diff changeset	17 from .thirdparty import (
884a0c1604ad wireproto: define attr-based classes for representing frames Gregory Szorc <gregory.szorc@gmail.com> parents: 37063 diff changeset	18 attr,
37290 cc5a040fe150 wireproto: syntax for encoding CBOR into frames Gregory Szorc <gregory.szorc@gmail.com> parents: 37289 diff changeset	19 cbor,
37064 884a0c1604ad wireproto: define attr-based classes for representing frames Gregory Szorc <gregory.szorc@gmail.com> parents: 37063 diff changeset	20 )
37054 40206e227412 wireproto: define and implement protocol for issuing requests Gregory Szorc <gregory.szorc@gmail.com> parents: diff changeset	21 from . import (
37055 8c3c47362934 wireproto: implement basic frame reading and processing Gregory Szorc <gregory.szorc@gmail.com> parents: 37054 diff changeset	22 error,
37054 40206e227412 wireproto: define and implement protocol for issuing requests Gregory Szorc <gregory.szorc@gmail.com> parents: diff changeset	23 util,
40206e227412 wireproto: define and implement protocol for issuing requests Gregory Szorc <gregory.szorc@gmail.com> parents: diff changeset	24 )
37087 f0b6fbea00cf stringutil: bulk-replace call sites to point to new module Yuya Nishihara <yuya@tcha.org> parents: 37066 diff changeset	25 from .utils import (
f0b6fbea00cf stringutil: bulk-replace call sites to point to new module Yuya Nishihara <yuya@tcha.org> parents: 37066 diff changeset	26 stringutil,
f0b6fbea00cf stringutil: bulk-replace call sites to point to new module Yuya Nishihara <yuya@tcha.org> parents: 37066 diff changeset	27 )
37054 40206e227412 wireproto: define and implement protocol for issuing requests Gregory Szorc <gregory.szorc@gmail.com> parents: diff changeset	28
37288 9bfcbe4f4745 wireproto: add streams to frame-based protocol Gregory Szorc <gregory.szorc@gmail.com> parents: 37287 diff changeset	29 FRAME_HEADER_SIZE = 8
37054 40206e227412 wireproto: define and implement protocol for issuing requests Gregory Szorc <gregory.szorc@gmail.com> parents: diff changeset	30 DEFAULT_MAX_FRAME_SIZE = 32768
40206e227412 wireproto: define and implement protocol for issuing requests Gregory Szorc <gregory.szorc@gmail.com> parents: diff changeset	31
37288 9bfcbe4f4745 wireproto: add streams to frame-based protocol Gregory Szorc <gregory.szorc@gmail.com> parents: 37287 diff changeset	32 STREAM_FLAG_BEGIN_STREAM = 0x01
9bfcbe4f4745 wireproto: add streams to frame-based protocol Gregory Szorc <gregory.szorc@gmail.com> parents: 37287 diff changeset	33 STREAM_FLAG_END_STREAM = 0x02
9bfcbe4f4745 wireproto: add streams to frame-based protocol Gregory Szorc <gregory.szorc@gmail.com> parents: 37287 diff changeset	34 STREAM_FLAG_ENCODING_APPLIED = 0x04
9bfcbe4f4745 wireproto: add streams to frame-based protocol Gregory Szorc <gregory.szorc@gmail.com> parents: 37287 diff changeset	35
9bfcbe4f4745 wireproto: add streams to frame-based protocol Gregory Szorc <gregory.szorc@gmail.com> parents: 37287 diff changeset	36 STREAM_FLAGS = {
9bfcbe4f4745 wireproto: add streams to frame-based protocol Gregory Szorc <gregory.szorc@gmail.com> parents: 37287 diff changeset	37 b'stream-begin': STREAM_FLAG_BEGIN_STREAM,
9bfcbe4f4745 wireproto: add streams to frame-based protocol Gregory Szorc <gregory.szorc@gmail.com> parents: 37287 diff changeset	38 b'stream-end': STREAM_FLAG_END_STREAM,
9bfcbe4f4745 wireproto: add streams to frame-based protocol Gregory Szorc <gregory.szorc@gmail.com> parents: 37287 diff changeset	39 b'encoded': STREAM_FLAG_ENCODING_APPLIED,
9bfcbe4f4745 wireproto: add streams to frame-based protocol Gregory Szorc <gregory.szorc@gmail.com> parents: 37287 diff changeset	40 }
9bfcbe4f4745 wireproto: add streams to frame-based protocol Gregory Szorc <gregory.szorc@gmail.com> parents: 37287 diff changeset	41
37292 3d0e2cd86e05 wireproto: use CBOR for command requests Gregory Szorc <gregory.szorc@gmail.com> parents: 37291 diff changeset	42 FRAME_TYPE_COMMAND_REQUEST = 0x01
37054 40206e227412 wireproto: define and implement protocol for issuing requests Gregory Szorc <gregory.szorc@gmail.com> parents: diff changeset	43 FRAME_TYPE_COMMAND_DATA = 0x03
37058 61393f888dfe wireproto: define and implement responses in framing protocol Gregory Szorc <gregory.szorc@gmail.com> parents: 37055 diff changeset	44 FRAME_TYPE_BYTES_RESPONSE = 0x04
61393f888dfe wireproto: define and implement responses in framing protocol Gregory Szorc <gregory.szorc@gmail.com> parents: 37055 diff changeset	45 FRAME_TYPE_ERROR_RESPONSE = 0x05
37063 0a6c5cc09a88 wireproto: define human output side channel frame Gregory Szorc <gregory.szorc@gmail.com> parents: 37061 diff changeset	46 FRAME_TYPE_TEXT_OUTPUT = 0x06
37291 b0041036214e wireproto: define frame to represent progress updates Gregory Szorc <gregory.szorc@gmail.com> parents: 37290 diff changeset	47 FRAME_TYPE_PROGRESS = 0x07
37288 9bfcbe4f4745 wireproto: add streams to frame-based protocol Gregory Szorc <gregory.szorc@gmail.com> parents: 37287 diff changeset	48 FRAME_TYPE_STREAM_SETTINGS = 0x08
37054 40206e227412 wireproto: define and implement protocol for issuing requests Gregory Szorc <gregory.szorc@gmail.com> parents: diff changeset	49
40206e227412 wireproto: define and implement protocol for issuing requests Gregory Szorc <gregory.szorc@gmail.com> parents: diff changeset	50 FRAME_TYPES = {
37292 3d0e2cd86e05 wireproto: use CBOR for command requests Gregory Szorc <gregory.szorc@gmail.com> parents: 37291 diff changeset	51 b'command-request': FRAME_TYPE_COMMAND_REQUEST,
37054 40206e227412 wireproto: define and implement protocol for issuing requests Gregory Szorc <gregory.szorc@gmail.com> parents: diff changeset	52 b'command-data': FRAME_TYPE_COMMAND_DATA,
37058 61393f888dfe wireproto: define and implement responses in framing protocol Gregory Szorc <gregory.szorc@gmail.com> parents: 37055 diff changeset	53 b'bytes-response': FRAME_TYPE_BYTES_RESPONSE,
61393f888dfe wireproto: define and implement responses in framing protocol Gregory Szorc <gregory.szorc@gmail.com> parents: 37055 diff changeset	54 b'error-response': FRAME_TYPE_ERROR_RESPONSE,
37063 0a6c5cc09a88 wireproto: define human output side channel frame Gregory Szorc <gregory.szorc@gmail.com> parents: 37061 diff changeset	55 b'text-output': FRAME_TYPE_TEXT_OUTPUT,
37291 b0041036214e wireproto: define frame to represent progress updates Gregory Szorc <gregory.szorc@gmail.com> parents: 37290 diff changeset	56 b'progress': FRAME_TYPE_PROGRESS,
37288 9bfcbe4f4745 wireproto: add streams to frame-based protocol Gregory Szorc <gregory.szorc@gmail.com> parents: 37287 diff changeset	57 b'stream-settings': FRAME_TYPE_STREAM_SETTINGS,
37054 40206e227412 wireproto: define and implement protocol for issuing requests Gregory Szorc <gregory.szorc@gmail.com> parents: diff changeset	58 }
40206e227412 wireproto: define and implement protocol for issuing requests Gregory Szorc <gregory.szorc@gmail.com> parents: diff changeset	59
37292 3d0e2cd86e05 wireproto: use CBOR for command requests Gregory Szorc <gregory.szorc@gmail.com> parents: 37291 diff changeset	60 FLAG_COMMAND_REQUEST_NEW = 0x01
3d0e2cd86e05 wireproto: use CBOR for command requests Gregory Szorc <gregory.szorc@gmail.com> parents: 37291 diff changeset	61 FLAG_COMMAND_REQUEST_CONTINUATION = 0x02
3d0e2cd86e05 wireproto: use CBOR for command requests Gregory Szorc <gregory.szorc@gmail.com> parents: 37291 diff changeset	62 FLAG_COMMAND_REQUEST_MORE_FRAMES = 0x04
3d0e2cd86e05 wireproto: use CBOR for command requests Gregory Szorc <gregory.szorc@gmail.com> parents: 37291 diff changeset	63 FLAG_COMMAND_REQUEST_EXPECT_DATA = 0x08
37054 40206e227412 wireproto: define and implement protocol for issuing requests Gregory Szorc <gregory.szorc@gmail.com> parents: diff changeset	64
37292 3d0e2cd86e05 wireproto: use CBOR for command requests Gregory Szorc <gregory.szorc@gmail.com> parents: 37291 diff changeset	65 FLAGS_COMMAND_REQUEST = {
3d0e2cd86e05 wireproto: use CBOR for command requests Gregory Szorc <gregory.szorc@gmail.com> parents: 37291 diff changeset	66 b'new': FLAG_COMMAND_REQUEST_NEW,
3d0e2cd86e05 wireproto: use CBOR for command requests Gregory Szorc <gregory.szorc@gmail.com> parents: 37291 diff changeset	67 b'continuation': FLAG_COMMAND_REQUEST_CONTINUATION,
3d0e2cd86e05 wireproto: use CBOR for command requests Gregory Szorc <gregory.szorc@gmail.com> parents: 37291 diff changeset	68 b'more': FLAG_COMMAND_REQUEST_MORE_FRAMES,
3d0e2cd86e05 wireproto: use CBOR for command requests Gregory Szorc <gregory.szorc@gmail.com> parents: 37291 diff changeset	69 b'have-data': FLAG_COMMAND_REQUEST_EXPECT_DATA,
37054 40206e227412 wireproto: define and implement protocol for issuing requests Gregory Szorc <gregory.szorc@gmail.com> parents: diff changeset	70 }
40206e227412 wireproto: define and implement protocol for issuing requests Gregory Szorc <gregory.szorc@gmail.com> parents: diff changeset	71
40206e227412 wireproto: define and implement protocol for issuing requests Gregory Szorc <gregory.szorc@gmail.com> parents: diff changeset	72 FLAG_COMMAND_DATA_CONTINUATION = 0x01
40206e227412 wireproto: define and implement protocol for issuing requests Gregory Szorc <gregory.szorc@gmail.com> parents: diff changeset	73 FLAG_COMMAND_DATA_EOS = 0x02
40206e227412 wireproto: define and implement protocol for issuing requests Gregory Szorc <gregory.szorc@gmail.com> parents: diff changeset	74
40206e227412 wireproto: define and implement protocol for issuing requests Gregory Szorc <gregory.szorc@gmail.com> parents: diff changeset	75 FLAGS_COMMAND_DATA = {
40206e227412 wireproto: define and implement protocol for issuing requests Gregory Szorc <gregory.szorc@gmail.com> parents: diff changeset	76 b'continuation': FLAG_COMMAND_DATA_CONTINUATION,
40206e227412 wireproto: define and implement protocol for issuing requests Gregory Szorc <gregory.szorc@gmail.com> parents: diff changeset	77 b'eos': FLAG_COMMAND_DATA_EOS,
40206e227412 wireproto: define and implement protocol for issuing requests Gregory Szorc <gregory.szorc@gmail.com> parents: diff changeset	78 }
40206e227412 wireproto: define and implement protocol for issuing requests Gregory Szorc <gregory.szorc@gmail.com> parents: diff changeset	79
37058 61393f888dfe wireproto: define and implement responses in framing protocol Gregory Szorc <gregory.szorc@gmail.com> parents: 37055 diff changeset	80 FLAG_BYTES_RESPONSE_CONTINUATION = 0x01
61393f888dfe wireproto: define and implement responses in framing protocol Gregory Szorc <gregory.szorc@gmail.com> parents: 37055 diff changeset	81 FLAG_BYTES_RESPONSE_EOS = 0x02
61393f888dfe wireproto: define and implement responses in framing protocol Gregory Szorc <gregory.szorc@gmail.com> parents: 37055 diff changeset	82
61393f888dfe wireproto: define and implement responses in framing protocol Gregory Szorc <gregory.szorc@gmail.com> parents: 37055 diff changeset	83 FLAGS_BYTES_RESPONSE = {
61393f888dfe wireproto: define and implement responses in framing protocol Gregory Szorc <gregory.szorc@gmail.com> parents: 37055 diff changeset	84 b'continuation': FLAG_BYTES_RESPONSE_CONTINUATION,
61393f888dfe wireproto: define and implement responses in framing protocol Gregory Szorc <gregory.szorc@gmail.com> parents: 37055 diff changeset	85 b'eos': FLAG_BYTES_RESPONSE_EOS,
61393f888dfe wireproto: define and implement responses in framing protocol Gregory Szorc <gregory.szorc@gmail.com> parents: 37055 diff changeset	86 }
61393f888dfe wireproto: define and implement responses in framing protocol Gregory Szorc <gregory.szorc@gmail.com> parents: 37055 diff changeset	87
61393f888dfe wireproto: define and implement responses in framing protocol Gregory Szorc <gregory.szorc@gmail.com> parents: 37055 diff changeset	88 FLAG_ERROR_RESPONSE_PROTOCOL = 0x01
61393f888dfe wireproto: define and implement responses in framing protocol Gregory Szorc <gregory.szorc@gmail.com> parents: 37055 diff changeset	89 FLAG_ERROR_RESPONSE_APPLICATION = 0x02
61393f888dfe wireproto: define and implement responses in framing protocol Gregory Szorc <gregory.szorc@gmail.com> parents: 37055 diff changeset	90
61393f888dfe wireproto: define and implement responses in framing protocol Gregory Szorc <gregory.szorc@gmail.com> parents: 37055 diff changeset	91 FLAGS_ERROR_RESPONSE = {
61393f888dfe wireproto: define and implement responses in framing protocol Gregory Szorc <gregory.szorc@gmail.com> parents: 37055 diff changeset	92 b'protocol': FLAG_ERROR_RESPONSE_PROTOCOL,
61393f888dfe wireproto: define and implement responses in framing protocol Gregory Szorc <gregory.szorc@gmail.com> parents: 37055 diff changeset	93 b'application': FLAG_ERROR_RESPONSE_APPLICATION,
61393f888dfe wireproto: define and implement responses in framing protocol Gregory Szorc <gregory.szorc@gmail.com> parents: 37055 diff changeset	94 }
61393f888dfe wireproto: define and implement responses in framing protocol Gregory Szorc <gregory.szorc@gmail.com> parents: 37055 diff changeset	95
37054 40206e227412 wireproto: define and implement protocol for issuing requests Gregory Szorc <gregory.szorc@gmail.com> parents: diff changeset	96 # Maps frame types to their available flags.
40206e227412 wireproto: define and implement protocol for issuing requests Gregory Szorc <gregory.szorc@gmail.com> parents: diff changeset	97 FRAME_TYPE_FLAGS = {
37292 3d0e2cd86e05 wireproto: use CBOR for command requests Gregory Szorc <gregory.szorc@gmail.com> parents: 37291 diff changeset	98 FRAME_TYPE_COMMAND_REQUEST: FLAGS_COMMAND_REQUEST,
37054 40206e227412 wireproto: define and implement protocol for issuing requests Gregory Szorc <gregory.szorc@gmail.com> parents: diff changeset	99 FRAME_TYPE_COMMAND_DATA: FLAGS_COMMAND_DATA,
37058 61393f888dfe wireproto: define and implement responses in framing protocol Gregory Szorc <gregory.szorc@gmail.com> parents: 37055 diff changeset	100 FRAME_TYPE_BYTES_RESPONSE: FLAGS_BYTES_RESPONSE,
61393f888dfe wireproto: define and implement responses in framing protocol Gregory Szorc <gregory.szorc@gmail.com> parents: 37055 diff changeset	101 FRAME_TYPE_ERROR_RESPONSE: FLAGS_ERROR_RESPONSE,
37063 0a6c5cc09a88 wireproto: define human output side channel frame Gregory Szorc <gregory.szorc@gmail.com> parents: 37061 diff changeset	102 FRAME_TYPE_TEXT_OUTPUT: {},
37291 b0041036214e wireproto: define frame to represent progress updates Gregory Szorc <gregory.szorc@gmail.com> parents: 37290 diff changeset	103 FRAME_TYPE_PROGRESS: {},
37288 9bfcbe4f4745 wireproto: add streams to frame-based protocol Gregory Szorc <gregory.szorc@gmail.com> parents: 37287 diff changeset	104 FRAME_TYPE_STREAM_SETTINGS: {},
37054 40206e227412 wireproto: define and implement protocol for issuing requests Gregory Szorc <gregory.szorc@gmail.com> parents: diff changeset	105 }
40206e227412 wireproto: define and implement protocol for issuing requests Gregory Szorc <gregory.szorc@gmail.com> parents: diff changeset	106
37292 3d0e2cd86e05 wireproto: use CBOR for command requests Gregory Szorc <gregory.szorc@gmail.com> parents: 37291 diff changeset	107 ARGUMENT_RECORD_HEADER = struct.Struct(r'<HH')
37054 40206e227412 wireproto: define and implement protocol for issuing requests Gregory Szorc <gregory.szorc@gmail.com> parents: diff changeset	108
37298 5ef2da00e935 wireproto: implement custom __repr__ for frame Gregory Szorc <gregory.szorc@gmail.com> parents: 37292 diff changeset	109 def humanflags(mapping, value):
5ef2da00e935 wireproto: implement custom __repr__ for frame Gregory Szorc <gregory.szorc@gmail.com> parents: 37292 diff changeset	110 """Convert a numeric flags value to a human value, using a mapping table."""
5ef2da00e935 wireproto: implement custom __repr__ for frame Gregory Szorc <gregory.szorc@gmail.com> parents: 37292 diff changeset	111 flags = []
5ef2da00e935 wireproto: implement custom __repr__ for frame Gregory Szorc <gregory.szorc@gmail.com> parents: 37292 diff changeset	112 for val, name in sorted({v: k for k, v in mapping.iteritems()}.iteritems()):
5ef2da00e935 wireproto: implement custom __repr__ for frame Gregory Szorc <gregory.szorc@gmail.com> parents: 37292 diff changeset	113 if value & val:
5ef2da00e935 wireproto: implement custom __repr__ for frame Gregory Szorc <gregory.szorc@gmail.com> parents: 37292 diff changeset	114 flags.append(name)
5ef2da00e935 wireproto: implement custom __repr__ for frame Gregory Szorc <gregory.szorc@gmail.com> parents: 37292 diff changeset	115
5ef2da00e935 wireproto: implement custom __repr__ for frame Gregory Szorc <gregory.szorc@gmail.com> parents: 37292 diff changeset	116 return b'\|'.join(flags)
5ef2da00e935 wireproto: implement custom __repr__ for frame Gregory Szorc <gregory.szorc@gmail.com> parents: 37292 diff changeset	117
37064 884a0c1604ad wireproto: define attr-based classes for representing frames Gregory Szorc <gregory.szorc@gmail.com> parents: 37063 diff changeset	118 @attr.s(slots=True)
884a0c1604ad wireproto: define attr-based classes for representing frames Gregory Szorc <gregory.szorc@gmail.com> parents: 37063 diff changeset	119 class frameheader(object):
884a0c1604ad wireproto: define attr-based classes for representing frames Gregory Szorc <gregory.szorc@gmail.com> parents: 37063 diff changeset	120 """Represents the data in a frame header."""
884a0c1604ad wireproto: define attr-based classes for representing frames Gregory Szorc <gregory.szorc@gmail.com> parents: 37063 diff changeset	121
884a0c1604ad wireproto: define attr-based classes for representing frames Gregory Szorc <gregory.szorc@gmail.com> parents: 37063 diff changeset	122 length = attr.ib()
884a0c1604ad wireproto: define attr-based classes for representing frames Gregory Szorc <gregory.szorc@gmail.com> parents: 37063 diff changeset	123 requestid = attr.ib()
37288 9bfcbe4f4745 wireproto: add streams to frame-based protocol Gregory Szorc <gregory.szorc@gmail.com> parents: 37287 diff changeset	124 streamid = attr.ib()
9bfcbe4f4745 wireproto: add streams to frame-based protocol Gregory Szorc <gregory.szorc@gmail.com> parents: 37287 diff changeset	125 streamflags = attr.ib()
37064 884a0c1604ad wireproto: define attr-based classes for representing frames Gregory Szorc <gregory.szorc@gmail.com> parents: 37063 diff changeset	126 typeid = attr.ib()
884a0c1604ad wireproto: define attr-based classes for representing frames Gregory Szorc <gregory.szorc@gmail.com> parents: 37063 diff changeset	127 flags = attr.ib()
884a0c1604ad wireproto: define attr-based classes for representing frames Gregory Szorc <gregory.szorc@gmail.com> parents: 37063 diff changeset	128
37298 5ef2da00e935 wireproto: implement custom __repr__ for frame Gregory Szorc <gregory.szorc@gmail.com> parents: 37292 diff changeset	129 @attr.s(slots=True, repr=False)
37064 884a0c1604ad wireproto: define attr-based classes for representing frames Gregory Szorc <gregory.szorc@gmail.com> parents: 37063 diff changeset	130 class frame(object):
884a0c1604ad wireproto: define attr-based classes for representing frames Gregory Szorc <gregory.szorc@gmail.com> parents: 37063 diff changeset	131 """Represents a parsed frame."""
884a0c1604ad wireproto: define attr-based classes for representing frames Gregory Szorc <gregory.szorc@gmail.com> parents: 37063 diff changeset	132
884a0c1604ad wireproto: define attr-based classes for representing frames Gregory Szorc <gregory.szorc@gmail.com> parents: 37063 diff changeset	133 requestid = attr.ib()
37288 9bfcbe4f4745 wireproto: add streams to frame-based protocol Gregory Szorc <gregory.szorc@gmail.com> parents: 37287 diff changeset	134 streamid = attr.ib()
9bfcbe4f4745 wireproto: add streams to frame-based protocol Gregory Szorc <gregory.szorc@gmail.com> parents: 37287 diff changeset	135 streamflags = attr.ib()
37064 884a0c1604ad wireproto: define attr-based classes for representing frames Gregory Szorc <gregory.szorc@gmail.com> parents: 37063 diff changeset	136 typeid = attr.ib()
884a0c1604ad wireproto: define attr-based classes for representing frames Gregory Szorc <gregory.szorc@gmail.com> parents: 37063 diff changeset	137 flags = attr.ib()
884a0c1604ad wireproto: define attr-based classes for representing frames Gregory Szorc <gregory.szorc@gmail.com> parents: 37063 diff changeset	138 payload = attr.ib()
884a0c1604ad wireproto: define attr-based classes for representing frames Gregory Szorc <gregory.szorc@gmail.com> parents: 37063 diff changeset	139
37298 5ef2da00e935 wireproto: implement custom __repr__ for frame Gregory Szorc <gregory.szorc@gmail.com> parents: 37292 diff changeset	140 def __repr__(self):
5ef2da00e935 wireproto: implement custom __repr__ for frame Gregory Szorc <gregory.szorc@gmail.com> parents: 37292 diff changeset	141 typename = '<unknown>'
5ef2da00e935 wireproto: implement custom __repr__ for frame Gregory Szorc <gregory.szorc@gmail.com> parents: 37292 diff changeset	142 for name, value in FRAME_TYPES.iteritems():
5ef2da00e935 wireproto: implement custom __repr__ for frame Gregory Szorc <gregory.szorc@gmail.com> parents: 37292 diff changeset	143 if value == self.typeid:
5ef2da00e935 wireproto: implement custom __repr__ for frame Gregory Szorc <gregory.szorc@gmail.com> parents: 37292 diff changeset	144 typename = name
5ef2da00e935 wireproto: implement custom __repr__ for frame Gregory Szorc <gregory.szorc@gmail.com> parents: 37292 diff changeset	145 break
5ef2da00e935 wireproto: implement custom __repr__ for frame Gregory Szorc <gregory.szorc@gmail.com> parents: 37292 diff changeset	146
5ef2da00e935 wireproto: implement custom __repr__ for frame Gregory Szorc <gregory.szorc@gmail.com> parents: 37292 diff changeset	147 return ('frame(size=%d; request=%d; stream=%d; streamflags=%s; '
5ef2da00e935 wireproto: implement custom __repr__ for frame Gregory Szorc <gregory.szorc@gmail.com> parents: 37292 diff changeset	148 'type=%s; flags=%s)' % (
5ef2da00e935 wireproto: implement custom __repr__ for frame Gregory Szorc <gregory.szorc@gmail.com> parents: 37292 diff changeset	149 len(self.payload), self.requestid, self.streamid,
5ef2da00e935 wireproto: implement custom __repr__ for frame Gregory Szorc <gregory.szorc@gmail.com> parents: 37292 diff changeset	150 humanflags(STREAM_FLAGS, self.streamflags), typename,
5ef2da00e935 wireproto: implement custom __repr__ for frame Gregory Szorc <gregory.szorc@gmail.com> parents: 37292 diff changeset	151 humanflags(FRAME_TYPE_FLAGS[self.typeid], self.flags)))
5ef2da00e935 wireproto: implement custom __repr__ for frame Gregory Szorc <gregory.szorc@gmail.com> parents: 37292 diff changeset	152
37288 9bfcbe4f4745 wireproto: add streams to frame-based protocol Gregory Szorc <gregory.szorc@gmail.com> parents: 37287 diff changeset	153 def makeframe(requestid, streamid, streamflags, typeid, flags, payload):
37054 40206e227412 wireproto: define and implement protocol for issuing requests Gregory Szorc <gregory.szorc@gmail.com> parents: diff changeset	154 """Assemble a frame into a byte array."""
40206e227412 wireproto: define and implement protocol for issuing requests Gregory Szorc <gregory.szorc@gmail.com> parents: diff changeset	155 # TODO assert size of payload.
40206e227412 wireproto: define and implement protocol for issuing requests Gregory Szorc <gregory.szorc@gmail.com> parents: diff changeset	156 frame = bytearray(FRAME_HEADER_SIZE + len(payload))
40206e227412 wireproto: define and implement protocol for issuing requests Gregory Szorc <gregory.szorc@gmail.com> parents: diff changeset	157
37060 2ec1fb9de638 wireproto: add request IDs to frames Gregory Szorc <gregory.szorc@gmail.com> parents: 37059 diff changeset	158 # 24 bits length
2ec1fb9de638 wireproto: add request IDs to frames Gregory Szorc <gregory.szorc@gmail.com> parents: 37059 diff changeset	159 # 16 bits request id
37288 9bfcbe4f4745 wireproto: add streams to frame-based protocol Gregory Szorc <gregory.szorc@gmail.com> parents: 37287 diff changeset	160 # 8 bits stream id
9bfcbe4f4745 wireproto: add streams to frame-based protocol Gregory Szorc <gregory.szorc@gmail.com> parents: 37287 diff changeset	161 # 8 bits stream flags
37060 2ec1fb9de638 wireproto: add request IDs to frames Gregory Szorc <gregory.szorc@gmail.com> parents: 37059 diff changeset	162 # 4 bits type
2ec1fb9de638 wireproto: add request IDs to frames Gregory Szorc <gregory.szorc@gmail.com> parents: 37059 diff changeset	163 # 4 bits flags
2ec1fb9de638 wireproto: add request IDs to frames Gregory Szorc <gregory.szorc@gmail.com> parents: 37059 diff changeset	164
37054 40206e227412 wireproto: define and implement protocol for issuing requests Gregory Szorc <gregory.szorc@gmail.com> parents: diff changeset	165 l = struct.pack(r'<I', len(payload))
40206e227412 wireproto: define and implement protocol for issuing requests Gregory Szorc <gregory.szorc@gmail.com> parents: diff changeset	166 frame[0:3] = l[0:3]
37288 9bfcbe4f4745 wireproto: add streams to frame-based protocol Gregory Szorc <gregory.szorc@gmail.com> parents: 37287 diff changeset	167 struct.pack_into(r'<HBB', frame, 3, requestid, streamid, streamflags)
9bfcbe4f4745 wireproto: add streams to frame-based protocol Gregory Szorc <gregory.szorc@gmail.com> parents: 37287 diff changeset	168 frame[7] = (typeid << 4) \| flags
9bfcbe4f4745 wireproto: add streams to frame-based protocol Gregory Szorc <gregory.szorc@gmail.com> parents: 37287 diff changeset	169 frame[8:] = payload
37054 40206e227412 wireproto: define and implement protocol for issuing requests Gregory Szorc <gregory.szorc@gmail.com> parents: diff changeset	170
40206e227412 wireproto: define and implement protocol for issuing requests Gregory Szorc <gregory.szorc@gmail.com> parents: diff changeset	171 return frame
40206e227412 wireproto: define and implement protocol for issuing requests Gregory Szorc <gregory.szorc@gmail.com> parents: diff changeset	172
40206e227412 wireproto: define and implement protocol for issuing requests Gregory Szorc <gregory.szorc@gmail.com> parents: diff changeset	173 def makeframefromhumanstring(s):
37060 2ec1fb9de638 wireproto: add request IDs to frames Gregory Szorc <gregory.szorc@gmail.com> parents: 37059 diff changeset	174 """Create a frame from a human readable string
2ec1fb9de638 wireproto: add request IDs to frames Gregory Szorc <gregory.szorc@gmail.com> parents: 37059 diff changeset	175
37290 cc5a040fe150 wireproto: syntax for encoding CBOR into frames Gregory Szorc <gregory.szorc@gmail.com> parents: 37289 diff changeset	176 DANGER: NOT SAFE TO USE WITH UNTRUSTED INPUT BECAUSE OF POTENTIAL
cc5a040fe150 wireproto: syntax for encoding CBOR into frames Gregory Szorc <gregory.szorc@gmail.com> parents: 37289 diff changeset	177 eval() USAGE. DO NOT USE IN CORE.
cc5a040fe150 wireproto: syntax for encoding CBOR into frames Gregory Szorc <gregory.szorc@gmail.com> parents: 37289 diff changeset	178
37060 2ec1fb9de638 wireproto: add request IDs to frames Gregory Szorc <gregory.szorc@gmail.com> parents: 37059 diff changeset	179 Strings have the form:
2ec1fb9de638 wireproto: add request IDs to frames Gregory Szorc <gregory.szorc@gmail.com> parents: 37059 diff changeset	180
37288 9bfcbe4f4745 wireproto: add streams to frame-based protocol Gregory Szorc <gregory.szorc@gmail.com> parents: 37287 diff changeset	181 <request-id> <stream-id> <stream-flags> <type> <flags> <payload>
37054 40206e227412 wireproto: define and implement protocol for issuing requests Gregory Szorc <gregory.szorc@gmail.com> parents: diff changeset	182
40206e227412 wireproto: define and implement protocol for issuing requests Gregory Szorc <gregory.szorc@gmail.com> parents: diff changeset	183 This can be used by user-facing applications and tests for creating
40206e227412 wireproto: define and implement protocol for issuing requests Gregory Szorc <gregory.szorc@gmail.com> parents: diff changeset	184 frames easily without having to type out a bunch of constants.
40206e227412 wireproto: define and implement protocol for issuing requests Gregory Szorc <gregory.szorc@gmail.com> parents: diff changeset	185
37288 9bfcbe4f4745 wireproto: add streams to frame-based protocol Gregory Szorc <gregory.szorc@gmail.com> parents: 37287 diff changeset	186 Request ID and stream IDs are integers.
37060 2ec1fb9de638 wireproto: add request IDs to frames Gregory Szorc <gregory.szorc@gmail.com> parents: 37059 diff changeset	187
37288 9bfcbe4f4745 wireproto: add streams to frame-based protocol Gregory Szorc <gregory.szorc@gmail.com> parents: 37287 diff changeset	188 Stream flags, frame type, and flags can be specified by integer or
9bfcbe4f4745 wireproto: add streams to frame-based protocol Gregory Szorc <gregory.szorc@gmail.com> parents: 37287 diff changeset	189 named constant.
37060 2ec1fb9de638 wireproto: add request IDs to frames Gregory Szorc <gregory.szorc@gmail.com> parents: 37059 diff changeset	190
37054 40206e227412 wireproto: define and implement protocol for issuing requests Gregory Szorc <gregory.szorc@gmail.com> parents: diff changeset	191 Flags can be delimited by `\|` to bitwise OR them together.
37290 cc5a040fe150 wireproto: syntax for encoding CBOR into frames Gregory Szorc <gregory.szorc@gmail.com> parents: 37289 diff changeset	192
cc5a040fe150 wireproto: syntax for encoding CBOR into frames Gregory Szorc <gregory.szorc@gmail.com> parents: 37289 diff changeset	193 If the payload begins with ``cbor:``, the following string will be
cc5a040fe150 wireproto: syntax for encoding CBOR into frames Gregory Szorc <gregory.szorc@gmail.com> parents: 37289 diff changeset	194 evaluated as Python code and the resulting object will be fed into
cc5a040fe150 wireproto: syntax for encoding CBOR into frames Gregory Szorc <gregory.szorc@gmail.com> parents: 37289 diff changeset	195 a CBOR encoder. Otherwise, the payload is interpreted as a Python
cc5a040fe150 wireproto: syntax for encoding CBOR into frames Gregory Szorc <gregory.szorc@gmail.com> parents: 37289 diff changeset	196 byte string literal.
37054 40206e227412 wireproto: define and implement protocol for issuing requests Gregory Szorc <gregory.szorc@gmail.com> parents: diff changeset	197 """
37288 9bfcbe4f4745 wireproto: add streams to frame-based protocol Gregory Szorc <gregory.szorc@gmail.com> parents: 37287 diff changeset	198 fields = s.split(b' ', 5)
9bfcbe4f4745 wireproto: add streams to frame-based protocol Gregory Szorc <gregory.szorc@gmail.com> parents: 37287 diff changeset	199 requestid, streamid, streamflags, frametype, frameflags, payload = fields
37060 2ec1fb9de638 wireproto: add request IDs to frames Gregory Szorc <gregory.szorc@gmail.com> parents: 37059 diff changeset	200
2ec1fb9de638 wireproto: add request IDs to frames Gregory Szorc <gregory.szorc@gmail.com> parents: 37059 diff changeset	201 requestid = int(requestid)
37288 9bfcbe4f4745 wireproto: add streams to frame-based protocol Gregory Szorc <gregory.szorc@gmail.com> parents: 37287 diff changeset	202 streamid = int(streamid)
9bfcbe4f4745 wireproto: add streams to frame-based protocol Gregory Szorc <gregory.szorc@gmail.com> parents: 37287 diff changeset	203
9bfcbe4f4745 wireproto: add streams to frame-based protocol Gregory Szorc <gregory.szorc@gmail.com> parents: 37287 diff changeset	204 finalstreamflags = 0
9bfcbe4f4745 wireproto: add streams to frame-based protocol Gregory Szorc <gregory.szorc@gmail.com> parents: 37287 diff changeset	205 for flag in streamflags.split(b'\|'):
9bfcbe4f4745 wireproto: add streams to frame-based protocol Gregory Szorc <gregory.szorc@gmail.com> parents: 37287 diff changeset	206 if flag in STREAM_FLAGS:
9bfcbe4f4745 wireproto: add streams to frame-based protocol Gregory Szorc <gregory.szorc@gmail.com> parents: 37287 diff changeset	207 finalstreamflags \|= STREAM_FLAGS[flag]
9bfcbe4f4745 wireproto: add streams to frame-based protocol Gregory Szorc <gregory.szorc@gmail.com> parents: 37287 diff changeset	208 else:
9bfcbe4f4745 wireproto: add streams to frame-based protocol Gregory Szorc <gregory.szorc@gmail.com> parents: 37287 diff changeset	209 finalstreamflags \|= int(flag)
37054 40206e227412 wireproto: define and implement protocol for issuing requests Gregory Szorc <gregory.szorc@gmail.com> parents: diff changeset	210
40206e227412 wireproto: define and implement protocol for issuing requests Gregory Szorc <gregory.szorc@gmail.com> parents: diff changeset	211 if frametype in FRAME_TYPES:
40206e227412 wireproto: define and implement protocol for issuing requests Gregory Szorc <gregory.szorc@gmail.com> parents: diff changeset	212 frametype = FRAME_TYPES[frametype]
40206e227412 wireproto: define and implement protocol for issuing requests Gregory Szorc <gregory.szorc@gmail.com> parents: diff changeset	213 else:
40206e227412 wireproto: define and implement protocol for issuing requests Gregory Szorc <gregory.szorc@gmail.com> parents: diff changeset	214 frametype = int(frametype)
40206e227412 wireproto: define and implement protocol for issuing requests Gregory Szorc <gregory.szorc@gmail.com> parents: diff changeset	215
40206e227412 wireproto: define and implement protocol for issuing requests Gregory Szorc <gregory.szorc@gmail.com> parents: diff changeset	216 finalflags = 0
40206e227412 wireproto: define and implement protocol for issuing requests Gregory Szorc <gregory.szorc@gmail.com> parents: diff changeset	217 validflags = FRAME_TYPE_FLAGS[frametype]
40206e227412 wireproto: define and implement protocol for issuing requests Gregory Szorc <gregory.szorc@gmail.com> parents: diff changeset	218 for flag in frameflags.split(b'\|'):
40206e227412 wireproto: define and implement protocol for issuing requests Gregory Szorc <gregory.szorc@gmail.com> parents: diff changeset	219 if flag in validflags:
40206e227412 wireproto: define and implement protocol for issuing requests Gregory Szorc <gregory.szorc@gmail.com> parents: diff changeset	220 finalflags \|= validflags[flag]
40206e227412 wireproto: define and implement protocol for issuing requests Gregory Szorc <gregory.szorc@gmail.com> parents: diff changeset	221 else:
40206e227412 wireproto: define and implement protocol for issuing requests Gregory Szorc <gregory.szorc@gmail.com> parents: diff changeset	222 finalflags \|= int(flag)
40206e227412 wireproto: define and implement protocol for issuing requests Gregory Szorc <gregory.szorc@gmail.com> parents: diff changeset	223
37290 cc5a040fe150 wireproto: syntax for encoding CBOR into frames Gregory Szorc <gregory.szorc@gmail.com> parents: 37289 diff changeset	224 if payload.startswith(b'cbor:'):
cc5a040fe150 wireproto: syntax for encoding CBOR into frames Gregory Szorc <gregory.szorc@gmail.com> parents: 37289 diff changeset	225 payload = cbor.dumps(stringutil.evalpython(payload[5:]), canonical=True)
cc5a040fe150 wireproto: syntax for encoding CBOR into frames Gregory Szorc <gregory.szorc@gmail.com> parents: 37289 diff changeset	226
cc5a040fe150 wireproto: syntax for encoding CBOR into frames Gregory Szorc <gregory.szorc@gmail.com> parents: 37289 diff changeset	227 else:
cc5a040fe150 wireproto: syntax for encoding CBOR into frames Gregory Szorc <gregory.szorc@gmail.com> parents: 37289 diff changeset	228 payload = stringutil.unescapestr(payload)
37054 40206e227412 wireproto: define and implement protocol for issuing requests Gregory Szorc <gregory.szorc@gmail.com> parents: diff changeset	229
37288 9bfcbe4f4745 wireproto: add streams to frame-based protocol Gregory Szorc <gregory.szorc@gmail.com> parents: 37287 diff changeset	230 return makeframe(requestid=requestid, streamid=streamid,
9bfcbe4f4745 wireproto: add streams to frame-based protocol Gregory Szorc <gregory.szorc@gmail.com> parents: 37287 diff changeset	231 streamflags=finalstreamflags, typeid=frametype,
37065 fe4c944f95bb wireproto: use named arguments when passing around frame data Gregory Szorc <gregory.szorc@gmail.com> parents: 37064 diff changeset	232 flags=finalflags, payload=payload)
37054 40206e227412 wireproto: define and implement protocol for issuing requests Gregory Szorc <gregory.szorc@gmail.com> parents: diff changeset	233
37055 8c3c47362934 wireproto: implement basic frame reading and processing Gregory Szorc <gregory.szorc@gmail.com> parents: 37054 diff changeset	234 def parseheader(data):
8c3c47362934 wireproto: implement basic frame reading and processing Gregory Szorc <gregory.szorc@gmail.com> parents: 37054 diff changeset	235 """Parse a unified framing protocol frame header from a buffer.
8c3c47362934 wireproto: implement basic frame reading and processing Gregory Szorc <gregory.szorc@gmail.com> parents: 37054 diff changeset	236
8c3c47362934 wireproto: implement basic frame reading and processing Gregory Szorc <gregory.szorc@gmail.com> parents: 37054 diff changeset	237 The header is expected to be in the buffer at offset 0 and the
8c3c47362934 wireproto: implement basic frame reading and processing Gregory Szorc <gregory.szorc@gmail.com> parents: 37054 diff changeset	238 buffer is expected to be large enough to hold a full header.
8c3c47362934 wireproto: implement basic frame reading and processing Gregory Szorc <gregory.szorc@gmail.com> parents: 37054 diff changeset	239 """
8c3c47362934 wireproto: implement basic frame reading and processing Gregory Szorc <gregory.szorc@gmail.com> parents: 37054 diff changeset	240 # 24 bits payload length (little endian)
37288 9bfcbe4f4745 wireproto: add streams to frame-based protocol Gregory Szorc <gregory.szorc@gmail.com> parents: 37287 diff changeset	241 # 16 bits request ID
9bfcbe4f4745 wireproto: add streams to frame-based protocol Gregory Szorc <gregory.szorc@gmail.com> parents: 37287 diff changeset	242 # 8 bits stream ID
9bfcbe4f4745 wireproto: add streams to frame-based protocol Gregory Szorc <gregory.szorc@gmail.com> parents: 37287 diff changeset	243 # 8 bits stream flags
37055 8c3c47362934 wireproto: implement basic frame reading and processing Gregory Szorc <gregory.szorc@gmail.com> parents: 37054 diff changeset	244 # 4 bits frame type
8c3c47362934 wireproto: implement basic frame reading and processing Gregory Szorc <gregory.szorc@gmail.com> parents: 37054 diff changeset	245 # 4 bits frame flags
8c3c47362934 wireproto: implement basic frame reading and processing Gregory Szorc <gregory.szorc@gmail.com> parents: 37054 diff changeset	246 # ... payload
8c3c47362934 wireproto: implement basic frame reading and processing Gregory Szorc <gregory.szorc@gmail.com> parents: 37054 diff changeset	247 framelength = data[0] + 256 * data[1] + 16384 * data[2]
37288 9bfcbe4f4745 wireproto: add streams to frame-based protocol Gregory Szorc <gregory.szorc@gmail.com> parents: 37287 diff changeset	248 requestid, streamid, streamflags = struct.unpack_from(r'<HBB', data, 3)
9bfcbe4f4745 wireproto: add streams to frame-based protocol Gregory Szorc <gregory.szorc@gmail.com> parents: 37287 diff changeset	249 typeflags = data[7]
37055 8c3c47362934 wireproto: implement basic frame reading and processing Gregory Szorc <gregory.szorc@gmail.com> parents: 37054 diff changeset	250
8c3c47362934 wireproto: implement basic frame reading and processing Gregory Szorc <gregory.szorc@gmail.com> parents: 37054 diff changeset	251 frametype = (typeflags & 0xf0) >> 4
8c3c47362934 wireproto: implement basic frame reading and processing Gregory Szorc <gregory.szorc@gmail.com> parents: 37054 diff changeset	252 frameflags = typeflags & 0x0f
8c3c47362934 wireproto: implement basic frame reading and processing Gregory Szorc <gregory.szorc@gmail.com> parents: 37054 diff changeset	253
37288 9bfcbe4f4745 wireproto: add streams to frame-based protocol Gregory Szorc <gregory.szorc@gmail.com> parents: 37287 diff changeset	254 return frameheader(framelength, requestid, streamid, streamflags,
9bfcbe4f4745 wireproto: add streams to frame-based protocol Gregory Szorc <gregory.szorc@gmail.com> parents: 37287 diff changeset	255 frametype, frameflags)
37055 8c3c47362934 wireproto: implement basic frame reading and processing Gregory Szorc <gregory.szorc@gmail.com> parents: 37054 diff changeset	256
8c3c47362934 wireproto: implement basic frame reading and processing Gregory Szorc <gregory.szorc@gmail.com> parents: 37054 diff changeset	257 def readframe(fh):
8c3c47362934 wireproto: implement basic frame reading and processing Gregory Szorc <gregory.szorc@gmail.com> parents: 37054 diff changeset	258 """Read a unified framing protocol frame from a file object.
8c3c47362934 wireproto: implement basic frame reading and processing Gregory Szorc <gregory.szorc@gmail.com> parents: 37054 diff changeset	259
8c3c47362934 wireproto: implement basic frame reading and processing Gregory Szorc <gregory.szorc@gmail.com> parents: 37054 diff changeset	260 Returns a 3-tuple of (type, flags, payload) for the decoded frame or
8c3c47362934 wireproto: implement basic frame reading and processing Gregory Szorc <gregory.szorc@gmail.com> parents: 37054 diff changeset	261 None if no frame is available. May raise if a malformed frame is
8c3c47362934 wireproto: implement basic frame reading and processing Gregory Szorc <gregory.szorc@gmail.com> parents: 37054 diff changeset	262 seen.
8c3c47362934 wireproto: implement basic frame reading and processing Gregory Szorc <gregory.szorc@gmail.com> parents: 37054 diff changeset	263 """
8c3c47362934 wireproto: implement basic frame reading and processing Gregory Szorc <gregory.szorc@gmail.com> parents: 37054 diff changeset	264 header = bytearray(FRAME_HEADER_SIZE)
8c3c47362934 wireproto: implement basic frame reading and processing Gregory Szorc <gregory.szorc@gmail.com> parents: 37054 diff changeset	265
8c3c47362934 wireproto: implement basic frame reading and processing Gregory Szorc <gregory.szorc@gmail.com> parents: 37054 diff changeset	266 readcount = fh.readinto(header)
8c3c47362934 wireproto: implement basic frame reading and processing Gregory Szorc <gregory.szorc@gmail.com> parents: 37054 diff changeset	267
8c3c47362934 wireproto: implement basic frame reading and processing Gregory Szorc <gregory.szorc@gmail.com> parents: 37054 diff changeset	268 if readcount == 0:
8c3c47362934 wireproto: implement basic frame reading and processing Gregory Szorc <gregory.szorc@gmail.com> parents: 37054 diff changeset	269 return None
8c3c47362934 wireproto: implement basic frame reading and processing Gregory Szorc <gregory.szorc@gmail.com> parents: 37054 diff changeset	270
8c3c47362934 wireproto: implement basic frame reading and processing Gregory Szorc <gregory.szorc@gmail.com> parents: 37054 diff changeset	271 if readcount != FRAME_HEADER_SIZE:
8c3c47362934 wireproto: implement basic frame reading and processing Gregory Szorc <gregory.szorc@gmail.com> parents: 37054 diff changeset	272 raise error.Abort(_('received incomplete frame: got %d bytes: %s') %
8c3c47362934 wireproto: implement basic frame reading and processing Gregory Szorc <gregory.szorc@gmail.com> parents: 37054 diff changeset	273 (readcount, header))
8c3c47362934 wireproto: implement basic frame reading and processing Gregory Szorc <gregory.szorc@gmail.com> parents: 37054 diff changeset	274
37064 884a0c1604ad wireproto: define attr-based classes for representing frames Gregory Szorc <gregory.szorc@gmail.com> parents: 37063 diff changeset	275 h = parseheader(header)
37055 8c3c47362934 wireproto: implement basic frame reading and processing Gregory Szorc <gregory.szorc@gmail.com> parents: 37054 diff changeset	276
37064 884a0c1604ad wireproto: define attr-based classes for representing frames Gregory Szorc <gregory.szorc@gmail.com> parents: 37063 diff changeset	277 payload = fh.read(h.length)
884a0c1604ad wireproto: define attr-based classes for representing frames Gregory Szorc <gregory.szorc@gmail.com> parents: 37063 diff changeset	278 if len(payload) != h.length:
37055 8c3c47362934 wireproto: implement basic frame reading and processing Gregory Szorc <gregory.szorc@gmail.com> parents: 37054 diff changeset	279 raise error.Abort(_('frame length error: expected %d; got %d') %
37064 884a0c1604ad wireproto: define attr-based classes for representing frames Gregory Szorc <gregory.szorc@gmail.com> parents: 37063 diff changeset	280 (h.length, len(payload)))
37055 8c3c47362934 wireproto: implement basic frame reading and processing Gregory Szorc <gregory.szorc@gmail.com> parents: 37054 diff changeset	281
37288 9bfcbe4f4745 wireproto: add streams to frame-based protocol Gregory Szorc <gregory.szorc@gmail.com> parents: 37287 diff changeset	282 return frame(h.requestid, h.streamid, h.streamflags, h.typeid, h.flags,
9bfcbe4f4745 wireproto: add streams to frame-based protocol Gregory Szorc <gregory.szorc@gmail.com> parents: 37287 diff changeset	283 payload)
37055 8c3c47362934 wireproto: implement basic frame reading and processing Gregory Szorc <gregory.szorc@gmail.com> parents: 37054 diff changeset	284
37292 3d0e2cd86e05 wireproto: use CBOR for command requests Gregory Szorc <gregory.szorc@gmail.com> parents: 37291 diff changeset	285 def createcommandframes(stream, requestid, cmd, args, datafh=None,
3d0e2cd86e05 wireproto: use CBOR for command requests Gregory Szorc <gregory.szorc@gmail.com> parents: 37291 diff changeset	286 maxframesize=DEFAULT_MAX_FRAME_SIZE):
37054 40206e227412 wireproto: define and implement protocol for issuing requests Gregory Szorc <gregory.szorc@gmail.com> parents: diff changeset	287 """Create frames necessary to transmit a request to run a command.
40206e227412 wireproto: define and implement protocol for issuing requests Gregory Szorc <gregory.szorc@gmail.com> parents: diff changeset	288
40206e227412 wireproto: define and implement protocol for issuing requests Gregory Szorc <gregory.szorc@gmail.com> parents: diff changeset	289 This is a generator of bytearrays. Each item represents a frame
40206e227412 wireproto: define and implement protocol for issuing requests Gregory Szorc <gregory.szorc@gmail.com> parents: diff changeset	290 ready to be sent over the wire to a peer.
40206e227412 wireproto: define and implement protocol for issuing requests Gregory Szorc <gregory.szorc@gmail.com> parents: diff changeset	291 """
37292 3d0e2cd86e05 wireproto: use CBOR for command requests Gregory Szorc <gregory.szorc@gmail.com> parents: 37291 diff changeset	292 data = {b'name': cmd}
37054 40206e227412 wireproto: define and implement protocol for issuing requests Gregory Szorc <gregory.szorc@gmail.com> parents: diff changeset	293 if args:
37292 3d0e2cd86e05 wireproto: use CBOR for command requests Gregory Szorc <gregory.szorc@gmail.com> parents: 37291 diff changeset	294 data[b'args'] = args
37054 40206e227412 wireproto: define and implement protocol for issuing requests Gregory Szorc <gregory.szorc@gmail.com> parents: diff changeset	295
37292 3d0e2cd86e05 wireproto: use CBOR for command requests Gregory Szorc <gregory.szorc@gmail.com> parents: 37291 diff changeset	296 data = cbor.dumps(data, canonical=True)
37054 40206e227412 wireproto: define and implement protocol for issuing requests Gregory Szorc <gregory.szorc@gmail.com> parents: diff changeset	297
37292 3d0e2cd86e05 wireproto: use CBOR for command requests Gregory Szorc <gregory.szorc@gmail.com> parents: 37291 diff changeset	298 offset = 0
3d0e2cd86e05 wireproto: use CBOR for command requests Gregory Szorc <gregory.szorc@gmail.com> parents: 37291 diff changeset	299
3d0e2cd86e05 wireproto: use CBOR for command requests Gregory Szorc <gregory.szorc@gmail.com> parents: 37291 diff changeset	300 while True:
3d0e2cd86e05 wireproto: use CBOR for command requests Gregory Szorc <gregory.szorc@gmail.com> parents: 37291 diff changeset	301 flags = 0
37054 40206e227412 wireproto: define and implement protocol for issuing requests Gregory Szorc <gregory.szorc@gmail.com> parents: diff changeset	302
37292 3d0e2cd86e05 wireproto: use CBOR for command requests Gregory Szorc <gregory.szorc@gmail.com> parents: 37291 diff changeset	303 # Must set new or continuation flag.
3d0e2cd86e05 wireproto: use CBOR for command requests Gregory Szorc <gregory.szorc@gmail.com> parents: 37291 diff changeset	304 if not offset:
3d0e2cd86e05 wireproto: use CBOR for command requests Gregory Szorc <gregory.szorc@gmail.com> parents: 37291 diff changeset	305 flags \|= FLAG_COMMAND_REQUEST_NEW
3d0e2cd86e05 wireproto: use CBOR for command requests Gregory Szorc <gregory.szorc@gmail.com> parents: 37291 diff changeset	306 else:
3d0e2cd86e05 wireproto: use CBOR for command requests Gregory Szorc <gregory.szorc@gmail.com> parents: 37291 diff changeset	307 flags \|= FLAG_COMMAND_REQUEST_CONTINUATION
37054 40206e227412 wireproto: define and implement protocol for issuing requests Gregory Szorc <gregory.szorc@gmail.com> parents: diff changeset	308
37292 3d0e2cd86e05 wireproto: use CBOR for command requests Gregory Szorc <gregory.szorc@gmail.com> parents: 37291 diff changeset	309 # Data frames is set on all frames.
3d0e2cd86e05 wireproto: use CBOR for command requests Gregory Szorc <gregory.szorc@gmail.com> parents: 37291 diff changeset	310 if datafh:
3d0e2cd86e05 wireproto: use CBOR for command requests Gregory Szorc <gregory.szorc@gmail.com> parents: 37291 diff changeset	311 flags \|= FLAG_COMMAND_REQUEST_EXPECT_DATA
37054 40206e227412 wireproto: define and implement protocol for issuing requests Gregory Szorc <gregory.szorc@gmail.com> parents: diff changeset	312
37292 3d0e2cd86e05 wireproto: use CBOR for command requests Gregory Szorc <gregory.szorc@gmail.com> parents: 37291 diff changeset	313 payload = data[offset:offset + maxframesize]
3d0e2cd86e05 wireproto: use CBOR for command requests Gregory Szorc <gregory.szorc@gmail.com> parents: 37291 diff changeset	314 offset += len(payload)
3d0e2cd86e05 wireproto: use CBOR for command requests Gregory Szorc <gregory.szorc@gmail.com> parents: 37291 diff changeset	315
3d0e2cd86e05 wireproto: use CBOR for command requests Gregory Szorc <gregory.szorc@gmail.com> parents: 37291 diff changeset	316 if len(payload) == maxframesize and offset < len(data):
3d0e2cd86e05 wireproto: use CBOR for command requests Gregory Szorc <gregory.szorc@gmail.com> parents: 37291 diff changeset	317 flags \|= FLAG_COMMAND_REQUEST_MORE_FRAMES
3d0e2cd86e05 wireproto: use CBOR for command requests Gregory Szorc <gregory.szorc@gmail.com> parents: 37291 diff changeset	318
37287 3ed344546d9e wireproto: start to associate frame generation with a stream Gregory Szorc <gregory.szorc@gmail.com> parents: 37087 diff changeset	319 yield stream.makeframe(requestid=requestid,
37292 3d0e2cd86e05 wireproto: use CBOR for command requests Gregory Szorc <gregory.szorc@gmail.com> parents: 37291 diff changeset	320 typeid=FRAME_TYPE_COMMAND_REQUEST,
37287 3ed344546d9e wireproto: start to associate frame generation with a stream Gregory Szorc <gregory.szorc@gmail.com> parents: 37087 diff changeset	321 flags=flags,
3ed344546d9e wireproto: start to associate frame generation with a stream Gregory Szorc <gregory.szorc@gmail.com> parents: 37087 diff changeset	322 payload=payload)
37054 40206e227412 wireproto: define and implement protocol for issuing requests Gregory Szorc <gregory.szorc@gmail.com> parents: diff changeset	323
37292 3d0e2cd86e05 wireproto: use CBOR for command requests Gregory Szorc <gregory.szorc@gmail.com> parents: 37291 diff changeset	324 if not (flags & FLAG_COMMAND_REQUEST_MORE_FRAMES):
3d0e2cd86e05 wireproto: use CBOR for command requests Gregory Szorc <gregory.szorc@gmail.com> parents: 37291 diff changeset	325 break
3d0e2cd86e05 wireproto: use CBOR for command requests Gregory Szorc <gregory.szorc@gmail.com> parents: 37291 diff changeset	326
37054 40206e227412 wireproto: define and implement protocol for issuing requests Gregory Szorc <gregory.szorc@gmail.com> parents: diff changeset	327 if datafh:
40206e227412 wireproto: define and implement protocol for issuing requests Gregory Szorc <gregory.szorc@gmail.com> parents: diff changeset	328 while True:
40206e227412 wireproto: define and implement protocol for issuing requests Gregory Szorc <gregory.szorc@gmail.com> parents: diff changeset	329 data = datafh.read(DEFAULT_MAX_FRAME_SIZE)
40206e227412 wireproto: define and implement protocol for issuing requests Gregory Szorc <gregory.szorc@gmail.com> parents: diff changeset	330
40206e227412 wireproto: define and implement protocol for issuing requests Gregory Szorc <gregory.szorc@gmail.com> parents: diff changeset	331 done = False
40206e227412 wireproto: define and implement protocol for issuing requests Gregory Szorc <gregory.szorc@gmail.com> parents: diff changeset	332 if len(data) == DEFAULT_MAX_FRAME_SIZE:
40206e227412 wireproto: define and implement protocol for issuing requests Gregory Szorc <gregory.szorc@gmail.com> parents: diff changeset	333 flags = FLAG_COMMAND_DATA_CONTINUATION
40206e227412 wireproto: define and implement protocol for issuing requests Gregory Szorc <gregory.szorc@gmail.com> parents: diff changeset	334 else:
40206e227412 wireproto: define and implement protocol for issuing requests Gregory Szorc <gregory.szorc@gmail.com> parents: diff changeset	335 flags = FLAG_COMMAND_DATA_EOS
40206e227412 wireproto: define and implement protocol for issuing requests Gregory Szorc <gregory.szorc@gmail.com> parents: diff changeset	336 assert datafh.read(1) == b''
40206e227412 wireproto: define and implement protocol for issuing requests Gregory Szorc <gregory.szorc@gmail.com> parents: diff changeset	337 done = True
40206e227412 wireproto: define and implement protocol for issuing requests Gregory Szorc <gregory.szorc@gmail.com> parents: diff changeset	338
37287 3ed344546d9e wireproto: start to associate frame generation with a stream Gregory Szorc <gregory.szorc@gmail.com> parents: 37087 diff changeset	339 yield stream.makeframe(requestid=requestid,
3ed344546d9e wireproto: start to associate frame generation with a stream Gregory Szorc <gregory.szorc@gmail.com> parents: 37087 diff changeset	340 typeid=FRAME_TYPE_COMMAND_DATA,
3ed344546d9e wireproto: start to associate frame generation with a stream Gregory Szorc <gregory.szorc@gmail.com> parents: 37087 diff changeset	341 flags=flags,
3ed344546d9e wireproto: start to associate frame generation with a stream Gregory Szorc <gregory.szorc@gmail.com> parents: 37087 diff changeset	342 payload=data)
37054 40206e227412 wireproto: define and implement protocol for issuing requests Gregory Szorc <gregory.szorc@gmail.com> parents: diff changeset	343
40206e227412 wireproto: define and implement protocol for issuing requests Gregory Szorc <gregory.szorc@gmail.com> parents: diff changeset	344 if done:
40206e227412 wireproto: define and implement protocol for issuing requests Gregory Szorc <gregory.szorc@gmail.com> parents: diff changeset	345 break
37055 8c3c47362934 wireproto: implement basic frame reading and processing Gregory Szorc <gregory.szorc@gmail.com> parents: 37054 diff changeset	346
37287 3ed344546d9e wireproto: start to associate frame generation with a stream Gregory Szorc <gregory.szorc@gmail.com> parents: 37087 diff changeset	347 def createbytesresponseframesfrombytes(stream, requestid, data,
37058 61393f888dfe wireproto: define and implement responses in framing protocol Gregory Szorc <gregory.szorc@gmail.com> parents: 37055 diff changeset	348 maxframesize=DEFAULT_MAX_FRAME_SIZE):
61393f888dfe wireproto: define and implement responses in framing protocol Gregory Szorc <gregory.szorc@gmail.com> parents: 37055 diff changeset	349 """Create a raw frame to send a bytes response from static bytes input.
61393f888dfe wireproto: define and implement responses in framing protocol Gregory Szorc <gregory.szorc@gmail.com> parents: 37055 diff changeset	350
61393f888dfe wireproto: define and implement responses in framing protocol Gregory Szorc <gregory.szorc@gmail.com> parents: 37055 diff changeset	351 Returns a generator of bytearrays.
61393f888dfe wireproto: define and implement responses in framing protocol Gregory Szorc <gregory.szorc@gmail.com> parents: 37055 diff changeset	352 """
61393f888dfe wireproto: define and implement responses in framing protocol Gregory Szorc <gregory.szorc@gmail.com> parents: 37055 diff changeset	353
61393f888dfe wireproto: define and implement responses in framing protocol Gregory Szorc <gregory.szorc@gmail.com> parents: 37055 diff changeset	354 # Simple case of a single frame.
61393f888dfe wireproto: define and implement responses in framing protocol Gregory Szorc <gregory.szorc@gmail.com> parents: 37055 diff changeset	355 if len(data) <= maxframesize:
37287 3ed344546d9e wireproto: start to associate frame generation with a stream Gregory Szorc <gregory.szorc@gmail.com> parents: 37087 diff changeset	356 yield stream.makeframe(requestid=requestid,
3ed344546d9e wireproto: start to associate frame generation with a stream Gregory Szorc <gregory.szorc@gmail.com> parents: 37087 diff changeset	357 typeid=FRAME_TYPE_BYTES_RESPONSE,
3ed344546d9e wireproto: start to associate frame generation with a stream Gregory Szorc <gregory.szorc@gmail.com> parents: 37087 diff changeset	358 flags=FLAG_BYTES_RESPONSE_EOS,
3ed344546d9e wireproto: start to associate frame generation with a stream Gregory Szorc <gregory.szorc@gmail.com> parents: 37087 diff changeset	359 payload=data)
37058 61393f888dfe wireproto: define and implement responses in framing protocol Gregory Szorc <gregory.szorc@gmail.com> parents: 37055 diff changeset	360 return
61393f888dfe wireproto: define and implement responses in framing protocol Gregory Szorc <gregory.szorc@gmail.com> parents: 37055 diff changeset	361
61393f888dfe wireproto: define and implement responses in framing protocol Gregory Szorc <gregory.szorc@gmail.com> parents: 37055 diff changeset	362 offset = 0
61393f888dfe wireproto: define and implement responses in framing protocol Gregory Szorc <gregory.szorc@gmail.com> parents: 37055 diff changeset	363 while True:
61393f888dfe wireproto: define and implement responses in framing protocol Gregory Szorc <gregory.szorc@gmail.com> parents: 37055 diff changeset	364 chunk = data[offset:offset + maxframesize]
61393f888dfe wireproto: define and implement responses in framing protocol Gregory Szorc <gregory.szorc@gmail.com> parents: 37055 diff changeset	365 offset += len(chunk)
61393f888dfe wireproto: define and implement responses in framing protocol Gregory Szorc <gregory.szorc@gmail.com> parents: 37055 diff changeset	366 done = offset == len(data)
61393f888dfe wireproto: define and implement responses in framing protocol Gregory Szorc <gregory.szorc@gmail.com> parents: 37055 diff changeset	367
61393f888dfe wireproto: define and implement responses in framing protocol Gregory Szorc <gregory.szorc@gmail.com> parents: 37055 diff changeset	368 if done:
61393f888dfe wireproto: define and implement responses in framing protocol Gregory Szorc <gregory.szorc@gmail.com> parents: 37055 diff changeset	369 flags = FLAG_BYTES_RESPONSE_EOS
61393f888dfe wireproto: define and implement responses in framing protocol Gregory Szorc <gregory.szorc@gmail.com> parents: 37055 diff changeset	370 else:
61393f888dfe wireproto: define and implement responses in framing protocol Gregory Szorc <gregory.szorc@gmail.com> parents: 37055 diff changeset	371 flags = FLAG_BYTES_RESPONSE_CONTINUATION
61393f888dfe wireproto: define and implement responses in framing protocol Gregory Szorc <gregory.szorc@gmail.com> parents: 37055 diff changeset	372
37287 3ed344546d9e wireproto: start to associate frame generation with a stream Gregory Szorc <gregory.szorc@gmail.com> parents: 37087 diff changeset	373 yield stream.makeframe(requestid=requestid,
3ed344546d9e wireproto: start to associate frame generation with a stream Gregory Szorc <gregory.szorc@gmail.com> parents: 37087 diff changeset	374 typeid=FRAME_TYPE_BYTES_RESPONSE,
3ed344546d9e wireproto: start to associate frame generation with a stream Gregory Szorc <gregory.szorc@gmail.com> parents: 37087 diff changeset	375 flags=flags,
3ed344546d9e wireproto: start to associate frame generation with a stream Gregory Szorc <gregory.szorc@gmail.com> parents: 37087 diff changeset	376 payload=chunk)
37058 61393f888dfe wireproto: define and implement responses in framing protocol Gregory Szorc <gregory.szorc@gmail.com> parents: 37055 diff changeset	377
61393f888dfe wireproto: define and implement responses in framing protocol Gregory Szorc <gregory.szorc@gmail.com> parents: 37055 diff changeset	378 if done:
61393f888dfe wireproto: define and implement responses in framing protocol Gregory Szorc <gregory.szorc@gmail.com> parents: 37055 diff changeset	379 break
61393f888dfe wireproto: define and implement responses in framing protocol Gregory Szorc <gregory.szorc@gmail.com> parents: 37055 diff changeset	380
37287 3ed344546d9e wireproto: start to associate frame generation with a stream Gregory Szorc <gregory.szorc@gmail.com> parents: 37087 diff changeset	381 def createerrorframe(stream, requestid, msg, protocol=False, application=False):
37058 61393f888dfe wireproto: define and implement responses in framing protocol Gregory Szorc <gregory.szorc@gmail.com> parents: 37055 diff changeset	382 # TODO properly handle frame size limits.
61393f888dfe wireproto: define and implement responses in framing protocol Gregory Szorc <gregory.szorc@gmail.com> parents: 37055 diff changeset	383 assert len(msg) <= DEFAULT_MAX_FRAME_SIZE
61393f888dfe wireproto: define and implement responses in framing protocol Gregory Szorc <gregory.szorc@gmail.com> parents: 37055 diff changeset	384
61393f888dfe wireproto: define and implement responses in framing protocol Gregory Szorc <gregory.szorc@gmail.com> parents: 37055 diff changeset	385 flags = 0
61393f888dfe wireproto: define and implement responses in framing protocol Gregory Szorc <gregory.szorc@gmail.com> parents: 37055 diff changeset	386 if protocol:
61393f888dfe wireproto: define and implement responses in framing protocol Gregory Szorc <gregory.szorc@gmail.com> parents: 37055 diff changeset	387 flags \|= FLAG_ERROR_RESPONSE_PROTOCOL
61393f888dfe wireproto: define and implement responses in framing protocol Gregory Szorc <gregory.szorc@gmail.com> parents: 37055 diff changeset	388 if application:
61393f888dfe wireproto: define and implement responses in framing protocol Gregory Szorc <gregory.szorc@gmail.com> parents: 37055 diff changeset	389 flags \|= FLAG_ERROR_RESPONSE_APPLICATION
61393f888dfe wireproto: define and implement responses in framing protocol Gregory Szorc <gregory.szorc@gmail.com> parents: 37055 diff changeset	390
37287 3ed344546d9e wireproto: start to associate frame generation with a stream Gregory Szorc <gregory.szorc@gmail.com> parents: 37087 diff changeset	391 yield stream.makeframe(requestid=requestid,
3ed344546d9e wireproto: start to associate frame generation with a stream Gregory Szorc <gregory.szorc@gmail.com> parents: 37087 diff changeset	392 typeid=FRAME_TYPE_ERROR_RESPONSE,
3ed344546d9e wireproto: start to associate frame generation with a stream Gregory Szorc <gregory.szorc@gmail.com> parents: 37087 diff changeset	393 flags=flags,
3ed344546d9e wireproto: start to associate frame generation with a stream Gregory Szorc <gregory.szorc@gmail.com> parents: 37087 diff changeset	394 payload=msg)
37058 61393f888dfe wireproto: define and implement responses in framing protocol Gregory Szorc <gregory.szorc@gmail.com> parents: 37055 diff changeset	395
37287 3ed344546d9e wireproto: start to associate frame generation with a stream Gregory Szorc <gregory.szorc@gmail.com> parents: 37087 diff changeset	396 def createtextoutputframe(stream, requestid, atoms):
37063 0a6c5cc09a88 wireproto: define human output side channel frame Gregory Szorc <gregory.szorc@gmail.com> parents: 37061 diff changeset	397 """Create a text output frame to render text to people.
0a6c5cc09a88 wireproto: define human output side channel frame Gregory Szorc <gregory.szorc@gmail.com> parents: 37061 diff changeset	398
0a6c5cc09a88 wireproto: define human output side channel frame Gregory Szorc <gregory.szorc@gmail.com> parents: 37061 diff changeset	399 ``atoms`` is a 3-tuple of (formatting string, args, labels).
0a6c5cc09a88 wireproto: define human output side channel frame Gregory Szorc <gregory.szorc@gmail.com> parents: 37061 diff changeset	400
0a6c5cc09a88 wireproto: define human output side channel frame Gregory Szorc <gregory.szorc@gmail.com> parents: 37061 diff changeset	401 The formatting string contains ``%s`` tokens to be replaced by the
0a6c5cc09a88 wireproto: define human output side channel frame Gregory Szorc <gregory.szorc@gmail.com> parents: 37061 diff changeset	402 corresponding indexed entry in ``args``. ``labels`` is an iterable of
0a6c5cc09a88 wireproto: define human output side channel frame Gregory Szorc <gregory.szorc@gmail.com> parents: 37061 diff changeset	403 formatters to be applied at rendering time. In terms of the ``ui``
0a6c5cc09a88 wireproto: define human output side channel frame Gregory Szorc <gregory.szorc@gmail.com> parents: 37061 diff changeset	404 class, each atom corresponds to a ``ui.write()``.
0a6c5cc09a88 wireproto: define human output side channel frame Gregory Szorc <gregory.szorc@gmail.com> parents: 37061 diff changeset	405 """
0a6c5cc09a88 wireproto: define human output side channel frame Gregory Szorc <gregory.szorc@gmail.com> parents: 37061 diff changeset	406 bytesleft = DEFAULT_MAX_FRAME_SIZE
0a6c5cc09a88 wireproto: define human output side channel frame Gregory Szorc <gregory.szorc@gmail.com> parents: 37061 diff changeset	407 atomchunks = []
0a6c5cc09a88 wireproto: define human output side channel frame Gregory Szorc <gregory.szorc@gmail.com> parents: 37061 diff changeset	408
0a6c5cc09a88 wireproto: define human output side channel frame Gregory Szorc <gregory.szorc@gmail.com> parents: 37061 diff changeset	409 for (formatting, args, labels) in atoms:
0a6c5cc09a88 wireproto: define human output side channel frame Gregory Szorc <gregory.szorc@gmail.com> parents: 37061 diff changeset	410 if len(args) > 255:
0a6c5cc09a88 wireproto: define human output side channel frame Gregory Szorc <gregory.szorc@gmail.com> parents: 37061 diff changeset	411 raise ValueError('cannot use more than 255 formatting arguments')
0a6c5cc09a88 wireproto: define human output side channel frame Gregory Szorc <gregory.szorc@gmail.com> parents: 37061 diff changeset	412 if len(labels) > 255:
0a6c5cc09a88 wireproto: define human output side channel frame Gregory Szorc <gregory.szorc@gmail.com> parents: 37061 diff changeset	413 raise ValueError('cannot use more than 255 labels')
0a6c5cc09a88 wireproto: define human output side channel frame Gregory Szorc <gregory.szorc@gmail.com> parents: 37061 diff changeset	414
0a6c5cc09a88 wireproto: define human output side channel frame Gregory Szorc <gregory.szorc@gmail.com> parents: 37061 diff changeset	415 # TODO look for localstr, other types here?
0a6c5cc09a88 wireproto: define human output side channel frame Gregory Szorc <gregory.szorc@gmail.com> parents: 37061 diff changeset	416
0a6c5cc09a88 wireproto: define human output side channel frame Gregory Szorc <gregory.szorc@gmail.com> parents: 37061 diff changeset	417 if not isinstance(formatting, bytes):
0a6c5cc09a88 wireproto: define human output side channel frame Gregory Szorc <gregory.szorc@gmail.com> parents: 37061 diff changeset	418 raise ValueError('must use bytes formatting strings')
0a6c5cc09a88 wireproto: define human output side channel frame Gregory Szorc <gregory.szorc@gmail.com> parents: 37061 diff changeset	419 for arg in args:
0a6c5cc09a88 wireproto: define human output side channel frame Gregory Szorc <gregory.szorc@gmail.com> parents: 37061 diff changeset	420 if not isinstance(arg, bytes):
0a6c5cc09a88 wireproto: define human output side channel frame Gregory Szorc <gregory.szorc@gmail.com> parents: 37061 diff changeset	421 raise ValueError('must use bytes for arguments')
0a6c5cc09a88 wireproto: define human output side channel frame Gregory Szorc <gregory.szorc@gmail.com> parents: 37061 diff changeset	422 for label in labels:
0a6c5cc09a88 wireproto: define human output side channel frame Gregory Szorc <gregory.szorc@gmail.com> parents: 37061 diff changeset	423 if not isinstance(label, bytes):
0a6c5cc09a88 wireproto: define human output side channel frame Gregory Szorc <gregory.szorc@gmail.com> parents: 37061 diff changeset	424 raise ValueError('must use bytes for labels')
0a6c5cc09a88 wireproto: define human output side channel frame Gregory Szorc <gregory.szorc@gmail.com> parents: 37061 diff changeset	425
0a6c5cc09a88 wireproto: define human output side channel frame Gregory Szorc <gregory.szorc@gmail.com> parents: 37061 diff changeset	426 # Formatting string must be UTF-8.
0a6c5cc09a88 wireproto: define human output side channel frame Gregory Szorc <gregory.szorc@gmail.com> parents: 37061 diff changeset	427 formatting = formatting.decode(r'utf-8', r'replace').encode(r'utf-8')
0a6c5cc09a88 wireproto: define human output side channel frame Gregory Szorc <gregory.szorc@gmail.com> parents: 37061 diff changeset	428
0a6c5cc09a88 wireproto: define human output side channel frame Gregory Szorc <gregory.szorc@gmail.com> parents: 37061 diff changeset	429 # Arguments must be UTF-8.
0a6c5cc09a88 wireproto: define human output side channel frame Gregory Szorc <gregory.szorc@gmail.com> parents: 37061 diff changeset	430 args = [a.decode(r'utf-8', r'replace').encode(r'utf-8') for a in args]
0a6c5cc09a88 wireproto: define human output side channel frame Gregory Szorc <gregory.szorc@gmail.com> parents: 37061 diff changeset	431
0a6c5cc09a88 wireproto: define human output side channel frame Gregory Szorc <gregory.szorc@gmail.com> parents: 37061 diff changeset	432 # Labels must be ASCII.
0a6c5cc09a88 wireproto: define human output side channel frame Gregory Szorc <gregory.szorc@gmail.com> parents: 37061 diff changeset	433 labels = [l.decode(r'ascii', r'strict').encode(r'ascii')
0a6c5cc09a88 wireproto: define human output side channel frame Gregory Szorc <gregory.szorc@gmail.com> parents: 37061 diff changeset	434 for l in labels]
0a6c5cc09a88 wireproto: define human output side channel frame Gregory Szorc <gregory.szorc@gmail.com> parents: 37061 diff changeset	435
0a6c5cc09a88 wireproto: define human output side channel frame Gregory Szorc <gregory.szorc@gmail.com> parents: 37061 diff changeset	436 if len(formatting) > 65535:
0a6c5cc09a88 wireproto: define human output side channel frame Gregory Szorc <gregory.szorc@gmail.com> parents: 37061 diff changeset	437 raise ValueError('formatting string cannot be longer than 64k')
0a6c5cc09a88 wireproto: define human output side channel frame Gregory Szorc <gregory.szorc@gmail.com> parents: 37061 diff changeset	438
0a6c5cc09a88 wireproto: define human output side channel frame Gregory Szorc <gregory.szorc@gmail.com> parents: 37061 diff changeset	439 if any(len(a) > 65535 for a in args):
0a6c5cc09a88 wireproto: define human output side channel frame Gregory Szorc <gregory.szorc@gmail.com> parents: 37061 diff changeset	440 raise ValueError('argument string cannot be longer than 64k')
0a6c5cc09a88 wireproto: define human output side channel frame Gregory Szorc <gregory.szorc@gmail.com> parents: 37061 diff changeset	441
0a6c5cc09a88 wireproto: define human output side channel frame Gregory Szorc <gregory.szorc@gmail.com> parents: 37061 diff changeset	442 if any(len(l) > 255 for l in labels):
0a6c5cc09a88 wireproto: define human output side channel frame Gregory Szorc <gregory.szorc@gmail.com> parents: 37061 diff changeset	443 raise ValueError('label string cannot be longer than 255 bytes')
0a6c5cc09a88 wireproto: define human output side channel frame Gregory Szorc <gregory.szorc@gmail.com> parents: 37061 diff changeset	444
0a6c5cc09a88 wireproto: define human output side channel frame Gregory Szorc <gregory.szorc@gmail.com> parents: 37061 diff changeset	445 chunks = [
0a6c5cc09a88 wireproto: define human output side channel frame Gregory Szorc <gregory.szorc@gmail.com> parents: 37061 diff changeset	446 struct.pack(r'<H', len(formatting)),
0a6c5cc09a88 wireproto: define human output side channel frame Gregory Szorc <gregory.szorc@gmail.com> parents: 37061 diff changeset	447 struct.pack(r'<BB', len(labels), len(args)),
0a6c5cc09a88 wireproto: define human output side channel frame Gregory Szorc <gregory.szorc@gmail.com> parents: 37061 diff changeset	448 struct.pack(r'<' + r'B' * len(labels), *map(len, labels)),
0a6c5cc09a88 wireproto: define human output side channel frame Gregory Szorc <gregory.szorc@gmail.com> parents: 37061 diff changeset	449 struct.pack(r'<' + r'H' * len(args), *map(len, args)),
0a6c5cc09a88 wireproto: define human output side channel frame Gregory Szorc <gregory.szorc@gmail.com> parents: 37061 diff changeset	450 ]
0a6c5cc09a88 wireproto: define human output side channel frame Gregory Szorc <gregory.szorc@gmail.com> parents: 37061 diff changeset	451 chunks.append(formatting)
0a6c5cc09a88 wireproto: define human output side channel frame Gregory Szorc <gregory.szorc@gmail.com> parents: 37061 diff changeset	452 chunks.extend(labels)
0a6c5cc09a88 wireproto: define human output side channel frame Gregory Szorc <gregory.szorc@gmail.com> parents: 37061 diff changeset	453 chunks.extend(args)
0a6c5cc09a88 wireproto: define human output side channel frame Gregory Szorc <gregory.szorc@gmail.com> parents: 37061 diff changeset	454
0a6c5cc09a88 wireproto: define human output side channel frame Gregory Szorc <gregory.szorc@gmail.com> parents: 37061 diff changeset	455 atom = b''.join(chunks)
0a6c5cc09a88 wireproto: define human output side channel frame Gregory Szorc <gregory.szorc@gmail.com> parents: 37061 diff changeset	456 atomchunks.append(atom)
0a6c5cc09a88 wireproto: define human output side channel frame Gregory Szorc <gregory.szorc@gmail.com> parents: 37061 diff changeset	457 bytesleft -= len(atom)
0a6c5cc09a88 wireproto: define human output side channel frame Gregory Szorc <gregory.szorc@gmail.com> parents: 37061 diff changeset	458
0a6c5cc09a88 wireproto: define human output side channel frame Gregory Szorc <gregory.szorc@gmail.com> parents: 37061 diff changeset	459 if bytesleft < 0:
0a6c5cc09a88 wireproto: define human output side channel frame Gregory Szorc <gregory.szorc@gmail.com> parents: 37061 diff changeset	460 raise ValueError('cannot encode data in a single frame')
0a6c5cc09a88 wireproto: define human output side channel frame Gregory Szorc <gregory.szorc@gmail.com> parents: 37061 diff changeset	461
37287 3ed344546d9e wireproto: start to associate frame generation with a stream Gregory Szorc <gregory.szorc@gmail.com> parents: 37087 diff changeset	462 yield stream.makeframe(requestid=requestid,
3ed344546d9e wireproto: start to associate frame generation with a stream Gregory Szorc <gregory.szorc@gmail.com> parents: 37087 diff changeset	463 typeid=FRAME_TYPE_TEXT_OUTPUT,
3ed344546d9e wireproto: start to associate frame generation with a stream Gregory Szorc <gregory.szorc@gmail.com> parents: 37087 diff changeset	464 flags=0,
3ed344546d9e wireproto: start to associate frame generation with a stream Gregory Szorc <gregory.szorc@gmail.com> parents: 37087 diff changeset	465 payload=b''.join(atomchunks))
3ed344546d9e wireproto: start to associate frame generation with a stream Gregory Szorc <gregory.szorc@gmail.com> parents: 37087 diff changeset	466
3ed344546d9e wireproto: start to associate frame generation with a stream Gregory Szorc <gregory.szorc@gmail.com> parents: 37087 diff changeset	467 class stream(object):
3ed344546d9e wireproto: start to associate frame generation with a stream Gregory Szorc <gregory.szorc@gmail.com> parents: 37087 diff changeset	468 """Represents a logical unidirectional series of frames."""
3ed344546d9e wireproto: start to associate frame generation with a stream Gregory Szorc <gregory.szorc@gmail.com> parents: 37087 diff changeset	469
37288 9bfcbe4f4745 wireproto: add streams to frame-based protocol Gregory Szorc <gregory.szorc@gmail.com> parents: 37287 diff changeset	470 def __init__(self, streamid, active=False):
9bfcbe4f4745 wireproto: add streams to frame-based protocol Gregory Szorc <gregory.szorc@gmail.com> parents: 37287 diff changeset	471 self.streamid = streamid
9bfcbe4f4745 wireproto: add streams to frame-based protocol Gregory Szorc <gregory.szorc@gmail.com> parents: 37287 diff changeset	472 self._active = False
9bfcbe4f4745 wireproto: add streams to frame-based protocol Gregory Szorc <gregory.szorc@gmail.com> parents: 37287 diff changeset	473
37287 3ed344546d9e wireproto: start to associate frame generation with a stream Gregory Szorc <gregory.szorc@gmail.com> parents: 37087 diff changeset	474 def makeframe(self, requestid, typeid, flags, payload):
3ed344546d9e wireproto: start to associate frame generation with a stream Gregory Szorc <gregory.szorc@gmail.com> parents: 37087 diff changeset	475 """Create a frame to be sent out over this stream.
3ed344546d9e wireproto: start to associate frame generation with a stream Gregory Szorc <gregory.szorc@gmail.com> parents: 37087 diff changeset	476
3ed344546d9e wireproto: start to associate frame generation with a stream Gregory Szorc <gregory.szorc@gmail.com> parents: 37087 diff changeset	477 Only returns the frame instance. Does not actually send it.
3ed344546d9e wireproto: start to associate frame generation with a stream Gregory Szorc <gregory.szorc@gmail.com> parents: 37087 diff changeset	478 """
37288 9bfcbe4f4745 wireproto: add streams to frame-based protocol Gregory Szorc <gregory.szorc@gmail.com> parents: 37287 diff changeset	479 streamflags = 0
9bfcbe4f4745 wireproto: add streams to frame-based protocol Gregory Szorc <gregory.szorc@gmail.com> parents: 37287 diff changeset	480 if not self._active:
9bfcbe4f4745 wireproto: add streams to frame-based protocol Gregory Szorc <gregory.szorc@gmail.com> parents: 37287 diff changeset	481 streamflags \|= STREAM_FLAG_BEGIN_STREAM
9bfcbe4f4745 wireproto: add streams to frame-based protocol Gregory Szorc <gregory.szorc@gmail.com> parents: 37287 diff changeset	482 self._active = True
9bfcbe4f4745 wireproto: add streams to frame-based protocol Gregory Szorc <gregory.szorc@gmail.com> parents: 37287 diff changeset	483
9bfcbe4f4745 wireproto: add streams to frame-based protocol Gregory Szorc <gregory.szorc@gmail.com> parents: 37287 diff changeset	484 return makeframe(requestid, self.streamid, streamflags, typeid, flags,
9bfcbe4f4745 wireproto: add streams to frame-based protocol Gregory Szorc <gregory.szorc@gmail.com> parents: 37287 diff changeset	485 payload)
9bfcbe4f4745 wireproto: add streams to frame-based protocol Gregory Szorc <gregory.szorc@gmail.com> parents: 37287 diff changeset	486
9bfcbe4f4745 wireproto: add streams to frame-based protocol Gregory Szorc <gregory.szorc@gmail.com> parents: 37287 diff changeset	487 def ensureserverstream(stream):
9bfcbe4f4745 wireproto: add streams to frame-based protocol Gregory Szorc <gregory.szorc@gmail.com> parents: 37287 diff changeset	488 if stream.streamid % 2:
9bfcbe4f4745 wireproto: add streams to frame-based protocol Gregory Szorc <gregory.szorc@gmail.com> parents: 37287 diff changeset	489 raise error.ProgrammingError('server should only write to even '
9bfcbe4f4745 wireproto: add streams to frame-based protocol Gregory Szorc <gregory.szorc@gmail.com> parents: 37287 diff changeset	490 'numbered streams; %d is not even' %
9bfcbe4f4745 wireproto: add streams to frame-based protocol Gregory Szorc <gregory.szorc@gmail.com> parents: 37287 diff changeset	491 stream.streamid)
37063 0a6c5cc09a88 wireproto: define human output side channel frame Gregory Szorc <gregory.szorc@gmail.com> parents: 37061 diff changeset	492
37055 8c3c47362934 wireproto: implement basic frame reading and processing Gregory Szorc <gregory.szorc@gmail.com> parents: 37054 diff changeset	493 class serverreactor(object):
8c3c47362934 wireproto: implement basic frame reading and processing Gregory Szorc <gregory.szorc@gmail.com> parents: 37054 diff changeset	494 """Holds state of a server handling frame-based protocol requests.
8c3c47362934 wireproto: implement basic frame reading and processing Gregory Szorc <gregory.szorc@gmail.com> parents: 37054 diff changeset	495
8c3c47362934 wireproto: implement basic frame reading and processing Gregory Szorc <gregory.szorc@gmail.com> parents: 37054 diff changeset	496 This class is the "brain" of the unified frame-based protocol server
8c3c47362934 wireproto: implement basic frame reading and processing Gregory Szorc <gregory.szorc@gmail.com> parents: 37054 diff changeset	497 component. While the protocol is stateless from the perspective of
8c3c47362934 wireproto: implement basic frame reading and processing Gregory Szorc <gregory.szorc@gmail.com> parents: 37054 diff changeset	498 requests/commands, something needs to track which frames have been
8c3c47362934 wireproto: implement basic frame reading and processing Gregory Szorc <gregory.szorc@gmail.com> parents: 37054 diff changeset	499 received, what frames to expect, etc. This class is that thing.
8c3c47362934 wireproto: implement basic frame reading and processing Gregory Szorc <gregory.szorc@gmail.com> parents: 37054 diff changeset	500
8c3c47362934 wireproto: implement basic frame reading and processing Gregory Szorc <gregory.szorc@gmail.com> parents: 37054 diff changeset	501 Instances are modeled as a state machine of sorts. Instances are also
8c3c47362934 wireproto: implement basic frame reading and processing Gregory Szorc <gregory.szorc@gmail.com> parents: 37054 diff changeset	502 reactionary to external events. The point of this class is to encapsulate
8c3c47362934 wireproto: implement basic frame reading and processing Gregory Szorc <gregory.szorc@gmail.com> parents: 37054 diff changeset	503 the state of the connection and the exchange of frames, not to perform
8c3c47362934 wireproto: implement basic frame reading and processing Gregory Szorc <gregory.szorc@gmail.com> parents: 37054 diff changeset	504 work. Instead, callers tell this class when something occurs, like a
8c3c47362934 wireproto: implement basic frame reading and processing Gregory Szorc <gregory.szorc@gmail.com> parents: 37054 diff changeset	505 frame arriving. If that activity is worthy of a follow-up action (say
8c3c47362934 wireproto: implement basic frame reading and processing Gregory Szorc <gregory.szorc@gmail.com> parents: 37054 diff changeset	506 run a command), the return value of that handler will say so.
8c3c47362934 wireproto: implement basic frame reading and processing Gregory Szorc <gregory.szorc@gmail.com> parents: 37054 diff changeset	507
8c3c47362934 wireproto: implement basic frame reading and processing Gregory Szorc <gregory.szorc@gmail.com> parents: 37054 diff changeset	508 I/O and CPU intensive operations are purposefully delegated outside of
8c3c47362934 wireproto: implement basic frame reading and processing Gregory Szorc <gregory.szorc@gmail.com> parents: 37054 diff changeset	509 this class.
8c3c47362934 wireproto: implement basic frame reading and processing Gregory Szorc <gregory.szorc@gmail.com> parents: 37054 diff changeset	510
8c3c47362934 wireproto: implement basic frame reading and processing Gregory Szorc <gregory.szorc@gmail.com> parents: 37054 diff changeset	511 Consumers are expected to tell instances when events occur. They do so by
8c3c47362934 wireproto: implement basic frame reading and processing Gregory Szorc <gregory.szorc@gmail.com> parents: 37054 diff changeset	512 calling the various ``on*`` methods. These methods return a 2-tuple
8c3c47362934 wireproto: implement basic frame reading and processing Gregory Szorc <gregory.szorc@gmail.com> parents: 37054 diff changeset	513 describing any follow-up action(s) to take. The first element is the
8c3c47362934 wireproto: implement basic frame reading and processing Gregory Szorc <gregory.szorc@gmail.com> parents: 37054 diff changeset	514 name of an action to perform. The second is a data structure (usually
8c3c47362934 wireproto: implement basic frame reading and processing Gregory Szorc <gregory.szorc@gmail.com> parents: 37054 diff changeset	515 a dict) specific to that action that contains more information. e.g.
8c3c47362934 wireproto: implement basic frame reading and processing Gregory Szorc <gregory.szorc@gmail.com> parents: 37054 diff changeset	516 if the server wants to send frames back to the client, the data structure
8c3c47362934 wireproto: implement basic frame reading and processing Gregory Szorc <gregory.szorc@gmail.com> parents: 37054 diff changeset	517 will contain a reference to those frames.
8c3c47362934 wireproto: implement basic frame reading and processing Gregory Szorc <gregory.szorc@gmail.com> parents: 37054 diff changeset	518
8c3c47362934 wireproto: implement basic frame reading and processing Gregory Szorc <gregory.szorc@gmail.com> parents: 37054 diff changeset	519 Valid actions that consumers can be instructed to take are:
8c3c47362934 wireproto: implement basic frame reading and processing Gregory Szorc <gregory.szorc@gmail.com> parents: 37054 diff changeset	520
37058 61393f888dfe wireproto: define and implement responses in framing protocol Gregory Szorc <gregory.szorc@gmail.com> parents: 37055 diff changeset	521 sendframes
61393f888dfe wireproto: define and implement responses in framing protocol Gregory Szorc <gregory.szorc@gmail.com> parents: 37055 diff changeset	522 Indicates that frames should be sent to the client. The ``framegen``
61393f888dfe wireproto: define and implement responses in framing protocol Gregory Szorc <gregory.szorc@gmail.com> parents: 37055 diff changeset	523 key contains a generator of frames that should be sent. The server
61393f888dfe wireproto: define and implement responses in framing protocol Gregory Szorc <gregory.szorc@gmail.com> parents: 37055 diff changeset	524 assumes that all frames are sent to the client.
61393f888dfe wireproto: define and implement responses in framing protocol Gregory Szorc <gregory.szorc@gmail.com> parents: 37055 diff changeset	525
37055 8c3c47362934 wireproto: implement basic frame reading and processing Gregory Szorc <gregory.szorc@gmail.com> parents: 37054 diff changeset	526 error
8c3c47362934 wireproto: implement basic frame reading and processing Gregory Szorc <gregory.szorc@gmail.com> parents: 37054 diff changeset	527 Indicates that an error occurred. Consumer should probably abort.
8c3c47362934 wireproto: implement basic frame reading and processing Gregory Szorc <gregory.szorc@gmail.com> parents: 37054 diff changeset	528
8c3c47362934 wireproto: implement basic frame reading and processing Gregory Szorc <gregory.szorc@gmail.com> parents: 37054 diff changeset	529 runcommand
8c3c47362934 wireproto: implement basic frame reading and processing Gregory Szorc <gregory.szorc@gmail.com> parents: 37054 diff changeset	530 Indicates that the consumer should run a wire protocol command. Details
8c3c47362934 wireproto: implement basic frame reading and processing Gregory Szorc <gregory.szorc@gmail.com> parents: 37054 diff changeset	531 of the command to run are given in the data structure.
8c3c47362934 wireproto: implement basic frame reading and processing Gregory Szorc <gregory.szorc@gmail.com> parents: 37054 diff changeset	532
8c3c47362934 wireproto: implement basic frame reading and processing Gregory Szorc <gregory.szorc@gmail.com> parents: 37054 diff changeset	533 wantframe
8c3c47362934 wireproto: implement basic frame reading and processing Gregory Szorc <gregory.szorc@gmail.com> parents: 37054 diff changeset	534 Indicates that nothing of interest happened and the server is waiting on
8c3c47362934 wireproto: implement basic frame reading and processing Gregory Szorc <gregory.szorc@gmail.com> parents: 37054 diff changeset	535 more frames from the client before anything interesting can be done.
37059 861e9d37e56e wireproto: buffer output frames when in half duplex mode Gregory Szorc <gregory.szorc@gmail.com> parents: 37058 diff changeset	536
861e9d37e56e wireproto: buffer output frames when in half duplex mode Gregory Szorc <gregory.szorc@gmail.com> parents: 37058 diff changeset	537 noop
861e9d37e56e wireproto: buffer output frames when in half duplex mode Gregory Szorc <gregory.szorc@gmail.com> parents: 37058 diff changeset	538 Indicates no additional action is required.
37061 c5e9c3b47366 wireproto: support for receiving multiple requests Gregory Szorc <gregory.szorc@gmail.com> parents: 37060 diff changeset	539
c5e9c3b47366 wireproto: support for receiving multiple requests Gregory Szorc <gregory.szorc@gmail.com> parents: 37060 diff changeset	540 Known Issues
c5e9c3b47366 wireproto: support for receiving multiple requests Gregory Szorc <gregory.szorc@gmail.com> parents: 37060 diff changeset	541 ------------
c5e9c3b47366 wireproto: support for receiving multiple requests Gregory Szorc <gregory.szorc@gmail.com> parents: 37060 diff changeset	542
c5e9c3b47366 wireproto: support for receiving multiple requests Gregory Szorc <gregory.szorc@gmail.com> parents: 37060 diff changeset	543 There are no limits to the number of partially received commands or their
c5e9c3b47366 wireproto: support for receiving multiple requests Gregory Szorc <gregory.szorc@gmail.com> parents: 37060 diff changeset	544 size. A malicious client could stream command request data and exhaust the
c5e9c3b47366 wireproto: support for receiving multiple requests Gregory Szorc <gregory.szorc@gmail.com> parents: 37060 diff changeset	545 server's memory.
c5e9c3b47366 wireproto: support for receiving multiple requests Gregory Szorc <gregory.szorc@gmail.com> parents: 37060 diff changeset	546
c5e9c3b47366 wireproto: support for receiving multiple requests Gregory Szorc <gregory.szorc@gmail.com> parents: 37060 diff changeset	547 Partially received commands are not acted upon when end of input is
c5e9c3b47366 wireproto: support for receiving multiple requests Gregory Szorc <gregory.szorc@gmail.com> parents: 37060 diff changeset	548 reached. Should the server error if it receives a partial request?
c5e9c3b47366 wireproto: support for receiving multiple requests Gregory Szorc <gregory.szorc@gmail.com> parents: 37060 diff changeset	549 Should the client send a message to abort a partially transmitted request
c5e9c3b47366 wireproto: support for receiving multiple requests Gregory Szorc <gregory.szorc@gmail.com> parents: 37060 diff changeset	550 to facilitate graceful shutdown?
c5e9c3b47366 wireproto: support for receiving multiple requests Gregory Szorc <gregory.szorc@gmail.com> parents: 37060 diff changeset	551
c5e9c3b47366 wireproto: support for receiving multiple requests Gregory Szorc <gregory.szorc@gmail.com> parents: 37060 diff changeset	552 Active requests that haven't been responded to aren't tracked. This means
c5e9c3b47366 wireproto: support for receiving multiple requests Gregory Szorc <gregory.szorc@gmail.com> parents: 37060 diff changeset	553 that if we receive a command and instruct its dispatch, another command
c5e9c3b47366 wireproto: support for receiving multiple requests Gregory Szorc <gregory.szorc@gmail.com> parents: 37060 diff changeset	554 with its request ID can come in over the wire and there will be a race
c5e9c3b47366 wireproto: support for receiving multiple requests Gregory Szorc <gregory.szorc@gmail.com> parents: 37060 diff changeset	555 between who responds to what.
37055 8c3c47362934 wireproto: implement basic frame reading and processing Gregory Szorc <gregory.szorc@gmail.com> parents: 37054 diff changeset	556 """
8c3c47362934 wireproto: implement basic frame reading and processing Gregory Szorc <gregory.szorc@gmail.com> parents: 37054 diff changeset	557
37059 861e9d37e56e wireproto: buffer output frames when in half duplex mode Gregory Szorc <gregory.szorc@gmail.com> parents: 37058 diff changeset	558 def __init__(self, deferoutput=False):
861e9d37e56e wireproto: buffer output frames when in half duplex mode Gregory Szorc <gregory.szorc@gmail.com> parents: 37058 diff changeset	559 """Construct a new server reactor.
861e9d37e56e wireproto: buffer output frames when in half duplex mode Gregory Szorc <gregory.szorc@gmail.com> parents: 37058 diff changeset	560
861e9d37e56e wireproto: buffer output frames when in half duplex mode Gregory Szorc <gregory.szorc@gmail.com> parents: 37058 diff changeset	561 ``deferoutput`` can be used to indicate that no output frames should be
861e9d37e56e wireproto: buffer output frames when in half duplex mode Gregory Szorc <gregory.szorc@gmail.com> parents: 37058 diff changeset	562 instructed to be sent until input has been exhausted. In this mode,
861e9d37e56e wireproto: buffer output frames when in half duplex mode Gregory Szorc <gregory.szorc@gmail.com> parents: 37058 diff changeset	563 events that would normally generate output frames (such as a command
861e9d37e56e wireproto: buffer output frames when in half duplex mode Gregory Szorc <gregory.szorc@gmail.com> parents: 37058 diff changeset	564 response being ready) will instead defer instructing the consumer to
861e9d37e56e wireproto: buffer output frames when in half duplex mode Gregory Szorc <gregory.szorc@gmail.com> parents: 37058 diff changeset	565 send those frames. This is useful for half-duplex transports where the
861e9d37e56e wireproto: buffer output frames when in half duplex mode Gregory Szorc <gregory.szorc@gmail.com> parents: 37058 diff changeset	566 sender cannot receive until all data has been transmitted.
861e9d37e56e wireproto: buffer output frames when in half duplex mode Gregory Szorc <gregory.szorc@gmail.com> parents: 37058 diff changeset	567 """
861e9d37e56e wireproto: buffer output frames when in half duplex mode Gregory Szorc <gregory.szorc@gmail.com> parents: 37058 diff changeset	568 self._deferoutput = deferoutput
37055 8c3c47362934 wireproto: implement basic frame reading and processing Gregory Szorc <gregory.szorc@gmail.com> parents: 37054 diff changeset	569 self._state = 'idle'
37289 5fadc63ac99f wireproto: explicit API to create outgoing streams Gregory Szorc <gregory.szorc@gmail.com> parents: 37288 diff changeset	570 self._nextoutgoingstreamid = 2
37059 861e9d37e56e wireproto: buffer output frames when in half duplex mode Gregory Szorc <gregory.szorc@gmail.com> parents: 37058 diff changeset	571 self._bufferedframegens = []
37288 9bfcbe4f4745 wireproto: add streams to frame-based protocol Gregory Szorc <gregory.szorc@gmail.com> parents: 37287 diff changeset	572 # stream id -> stream instance for all active streams from the client.
9bfcbe4f4745 wireproto: add streams to frame-based protocol Gregory Szorc <gregory.szorc@gmail.com> parents: 37287 diff changeset	573 self._incomingstreams = {}
37289 5fadc63ac99f wireproto: explicit API to create outgoing streams Gregory Szorc <gregory.szorc@gmail.com> parents: 37288 diff changeset	574 self._outgoingstreams = {}
37061 c5e9c3b47366 wireproto: support for receiving multiple requests Gregory Szorc <gregory.szorc@gmail.com> parents: 37060 diff changeset	575 # request id -> dict of commands that are actively being received.
c5e9c3b47366 wireproto: support for receiving multiple requests Gregory Szorc <gregory.szorc@gmail.com> parents: 37060 diff changeset	576 self._receivingcommands = {}
37066 39304dd63589 wireproto: explicitly track which requests are active Gregory Szorc <gregory.szorc@gmail.com> parents: 37065 diff changeset	577 # Request IDs that have been received and are actively being processed.
39304dd63589 wireproto: explicitly track which requests are active Gregory Szorc <gregory.szorc@gmail.com> parents: 37065 diff changeset	578 # Once all output for a request has been sent, it is removed from this
39304dd63589 wireproto: explicitly track which requests are active Gregory Szorc <gregory.szorc@gmail.com> parents: 37065 diff changeset	579 # set.
39304dd63589 wireproto: explicitly track which requests are active Gregory Szorc <gregory.szorc@gmail.com> parents: 37065 diff changeset	580 self._activecommands = set()
37055 8c3c47362934 wireproto: implement basic frame reading and processing Gregory Szorc <gregory.szorc@gmail.com> parents: 37054 diff changeset	581
37064 884a0c1604ad wireproto: define attr-based classes for representing frames Gregory Szorc <gregory.szorc@gmail.com> parents: 37063 diff changeset	582 def onframerecv(self, frame):
37055 8c3c47362934 wireproto: implement basic frame reading and processing Gregory Szorc <gregory.szorc@gmail.com> parents: 37054 diff changeset	583 """Process a frame that has been received off the wire.
8c3c47362934 wireproto: implement basic frame reading and processing Gregory Szorc <gregory.szorc@gmail.com> parents: 37054 diff changeset	584
8c3c47362934 wireproto: implement basic frame reading and processing Gregory Szorc <gregory.szorc@gmail.com> parents: 37054 diff changeset	585 Returns a dict with an ``action`` key that details what action,
8c3c47362934 wireproto: implement basic frame reading and processing Gregory Szorc <gregory.szorc@gmail.com> parents: 37054 diff changeset	586 if any, the consumer should take next.
8c3c47362934 wireproto: implement basic frame reading and processing Gregory Szorc <gregory.szorc@gmail.com> parents: 37054 diff changeset	587 """
37288 9bfcbe4f4745 wireproto: add streams to frame-based protocol Gregory Szorc <gregory.szorc@gmail.com> parents: 37287 diff changeset	588 if not frame.streamid % 2:
9bfcbe4f4745 wireproto: add streams to frame-based protocol Gregory Szorc <gregory.szorc@gmail.com> parents: 37287 diff changeset	589 self._state = 'errored'
9bfcbe4f4745 wireproto: add streams to frame-based protocol Gregory Szorc <gregory.szorc@gmail.com> parents: 37287 diff changeset	590 return self._makeerrorresult(
9bfcbe4f4745 wireproto: add streams to frame-based protocol Gregory Szorc <gregory.szorc@gmail.com> parents: 37287 diff changeset	591 _('received frame with even numbered stream ID: %d') %
9bfcbe4f4745 wireproto: add streams to frame-based protocol Gregory Szorc <gregory.szorc@gmail.com> parents: 37287 diff changeset	592 frame.streamid)
9bfcbe4f4745 wireproto: add streams to frame-based protocol Gregory Szorc <gregory.szorc@gmail.com> parents: 37287 diff changeset	593
9bfcbe4f4745 wireproto: add streams to frame-based protocol Gregory Szorc <gregory.szorc@gmail.com> parents: 37287 diff changeset	594 if frame.streamid not in self._incomingstreams:
9bfcbe4f4745 wireproto: add streams to frame-based protocol Gregory Szorc <gregory.szorc@gmail.com> parents: 37287 diff changeset	595 if not frame.streamflags & STREAM_FLAG_BEGIN_STREAM:
9bfcbe4f4745 wireproto: add streams to frame-based protocol Gregory Szorc <gregory.szorc@gmail.com> parents: 37287 diff changeset	596 self._state = 'errored'
9bfcbe4f4745 wireproto: add streams to frame-based protocol Gregory Szorc <gregory.szorc@gmail.com> parents: 37287 diff changeset	597 return self._makeerrorresult(
9bfcbe4f4745 wireproto: add streams to frame-based protocol Gregory Szorc <gregory.szorc@gmail.com> parents: 37287 diff changeset	598 _('received frame on unknown inactive stream without '
9bfcbe4f4745 wireproto: add streams to frame-based protocol Gregory Szorc <gregory.szorc@gmail.com> parents: 37287 diff changeset	599 'beginning of stream flag set'))
9bfcbe4f4745 wireproto: add streams to frame-based protocol Gregory Szorc <gregory.szorc@gmail.com> parents: 37287 diff changeset	600
9bfcbe4f4745 wireproto: add streams to frame-based protocol Gregory Szorc <gregory.szorc@gmail.com> parents: 37287 diff changeset	601 self._incomingstreams[frame.streamid] = stream(frame.streamid)
9bfcbe4f4745 wireproto: add streams to frame-based protocol Gregory Szorc <gregory.szorc@gmail.com> parents: 37287 diff changeset	602
9bfcbe4f4745 wireproto: add streams to frame-based protocol Gregory Szorc <gregory.szorc@gmail.com> parents: 37287 diff changeset	603 if frame.streamflags & STREAM_FLAG_ENCODING_APPLIED:
9bfcbe4f4745 wireproto: add streams to frame-based protocol Gregory Szorc <gregory.szorc@gmail.com> parents: 37287 diff changeset	604 # TODO handle decoding frames
9bfcbe4f4745 wireproto: add streams to frame-based protocol Gregory Szorc <gregory.szorc@gmail.com> parents: 37287 diff changeset	605 self._state = 'errored'
9bfcbe4f4745 wireproto: add streams to frame-based protocol Gregory Szorc <gregory.szorc@gmail.com> parents: 37287 diff changeset	606 raise error.ProgrammingError('support for decoding stream payloads '
9bfcbe4f4745 wireproto: add streams to frame-based protocol Gregory Szorc <gregory.szorc@gmail.com> parents: 37287 diff changeset	607 'not yet implemented')
9bfcbe4f4745 wireproto: add streams to frame-based protocol Gregory Szorc <gregory.szorc@gmail.com> parents: 37287 diff changeset	608
9bfcbe4f4745 wireproto: add streams to frame-based protocol Gregory Szorc <gregory.szorc@gmail.com> parents: 37287 diff changeset	609 if frame.streamflags & STREAM_FLAG_END_STREAM:
9bfcbe4f4745 wireproto: add streams to frame-based protocol Gregory Szorc <gregory.szorc@gmail.com> parents: 37287 diff changeset	610 del self._incomingstreams[frame.streamid]
9bfcbe4f4745 wireproto: add streams to frame-based protocol Gregory Szorc <gregory.szorc@gmail.com> parents: 37287 diff changeset	611
37055 8c3c47362934 wireproto: implement basic frame reading and processing Gregory Szorc <gregory.szorc@gmail.com> parents: 37054 diff changeset	612 handlers = {
8c3c47362934 wireproto: implement basic frame reading and processing Gregory Szorc <gregory.szorc@gmail.com> parents: 37054 diff changeset	613 'idle': self._onframeidle,
37061 c5e9c3b47366 wireproto: support for receiving multiple requests Gregory Szorc <gregory.szorc@gmail.com> parents: 37060 diff changeset	614 'command-receiving': self._onframecommandreceiving,
37055 8c3c47362934 wireproto: implement basic frame reading and processing Gregory Szorc <gregory.szorc@gmail.com> parents: 37054 diff changeset	615 'errored': self._onframeerrored,
8c3c47362934 wireproto: implement basic frame reading and processing Gregory Szorc <gregory.szorc@gmail.com> parents: 37054 diff changeset	616 }
8c3c47362934 wireproto: implement basic frame reading and processing Gregory Szorc <gregory.szorc@gmail.com> parents: 37054 diff changeset	617
8c3c47362934 wireproto: implement basic frame reading and processing Gregory Szorc <gregory.szorc@gmail.com> parents: 37054 diff changeset	618 meth = handlers.get(self._state)
8c3c47362934 wireproto: implement basic frame reading and processing Gregory Szorc <gregory.szorc@gmail.com> parents: 37054 diff changeset	619 if not meth:
8c3c47362934 wireproto: implement basic frame reading and processing Gregory Szorc <gregory.szorc@gmail.com> parents: 37054 diff changeset	620 raise error.ProgrammingError('unhandled state: %s' % self._state)
8c3c47362934 wireproto: implement basic frame reading and processing Gregory Szorc <gregory.szorc@gmail.com> parents: 37054 diff changeset	621
37064 884a0c1604ad wireproto: define attr-based classes for representing frames Gregory Szorc <gregory.szorc@gmail.com> parents: 37063 diff changeset	622 return meth(frame)
37055 8c3c47362934 wireproto: implement basic frame reading and processing Gregory Szorc <gregory.szorc@gmail.com> parents: 37054 diff changeset	623
37287 3ed344546d9e wireproto: start to associate frame generation with a stream Gregory Szorc <gregory.szorc@gmail.com> parents: 37087 diff changeset	624 def onbytesresponseready(self, stream, requestid, data):
37058 61393f888dfe wireproto: define and implement responses in framing protocol Gregory Szorc <gregory.szorc@gmail.com> parents: 37055 diff changeset	625 """Signal that a bytes response is ready to be sent to the client.
61393f888dfe wireproto: define and implement responses in framing protocol Gregory Szorc <gregory.szorc@gmail.com> parents: 37055 diff changeset	626
61393f888dfe wireproto: define and implement responses in framing protocol Gregory Szorc <gregory.szorc@gmail.com> parents: 37055 diff changeset	627 The raw bytes response is passed as an argument.
61393f888dfe wireproto: define and implement responses in framing protocol Gregory Szorc <gregory.szorc@gmail.com> parents: 37055 diff changeset	628 """
37288 9bfcbe4f4745 wireproto: add streams to frame-based protocol Gregory Szorc <gregory.szorc@gmail.com> parents: 37287 diff changeset	629 ensureserverstream(stream)
9bfcbe4f4745 wireproto: add streams to frame-based protocol Gregory Szorc <gregory.szorc@gmail.com> parents: 37287 diff changeset	630
37066 39304dd63589 wireproto: explicitly track which requests are active Gregory Szorc <gregory.szorc@gmail.com> parents: 37065 diff changeset	631 def sendframes():
37287 3ed344546d9e wireproto: start to associate frame generation with a stream Gregory Szorc <gregory.szorc@gmail.com> parents: 37087 diff changeset	632 for frame in createbytesresponseframesfrombytes(stream, requestid,
3ed344546d9e wireproto: start to associate frame generation with a stream Gregory Szorc <gregory.szorc@gmail.com> parents: 37087 diff changeset	633 data):
37066 39304dd63589 wireproto: explicitly track which requests are active Gregory Szorc <gregory.szorc@gmail.com> parents: 37065 diff changeset	634 yield frame
39304dd63589 wireproto: explicitly track which requests are active Gregory Szorc <gregory.szorc@gmail.com> parents: 37065 diff changeset	635
39304dd63589 wireproto: explicitly track which requests are active Gregory Szorc <gregory.szorc@gmail.com> parents: 37065 diff changeset	636 self._activecommands.remove(requestid)
39304dd63589 wireproto: explicitly track which requests are active Gregory Szorc <gregory.szorc@gmail.com> parents: 37065 diff changeset	637
39304dd63589 wireproto: explicitly track which requests are active Gregory Szorc <gregory.szorc@gmail.com> parents: 37065 diff changeset	638 result = sendframes()
37059 861e9d37e56e wireproto: buffer output frames when in half duplex mode Gregory Szorc <gregory.szorc@gmail.com> parents: 37058 diff changeset	639
861e9d37e56e wireproto: buffer output frames when in half duplex mode Gregory Szorc <gregory.szorc@gmail.com> parents: 37058 diff changeset	640 if self._deferoutput:
37066 39304dd63589 wireproto: explicitly track which requests are active Gregory Szorc <gregory.szorc@gmail.com> parents: 37065 diff changeset	641 self._bufferedframegens.append(result)
37059 861e9d37e56e wireproto: buffer output frames when in half duplex mode Gregory Szorc <gregory.szorc@gmail.com> parents: 37058 diff changeset	642 return 'noop', {}
861e9d37e56e wireproto: buffer output frames when in half duplex mode Gregory Szorc <gregory.szorc@gmail.com> parents: 37058 diff changeset	643 else:
861e9d37e56e wireproto: buffer output frames when in half duplex mode Gregory Szorc <gregory.szorc@gmail.com> parents: 37058 diff changeset	644 return 'sendframes', {
37066 39304dd63589 wireproto: explicitly track which requests are active Gregory Szorc <gregory.szorc@gmail.com> parents: 37065 diff changeset	645 'framegen': result,
37059 861e9d37e56e wireproto: buffer output frames when in half duplex mode Gregory Szorc <gregory.szorc@gmail.com> parents: 37058 diff changeset	646 }
861e9d37e56e wireproto: buffer output frames when in half duplex mode Gregory Szorc <gregory.szorc@gmail.com> parents: 37058 diff changeset	647
861e9d37e56e wireproto: buffer output frames when in half duplex mode Gregory Szorc <gregory.szorc@gmail.com> parents: 37058 diff changeset	648 def oninputeof(self):
861e9d37e56e wireproto: buffer output frames when in half duplex mode Gregory Szorc <gregory.szorc@gmail.com> parents: 37058 diff changeset	649 """Signals that end of input has been received.
861e9d37e56e wireproto: buffer output frames when in half duplex mode Gregory Szorc <gregory.szorc@gmail.com> parents: 37058 diff changeset	650
861e9d37e56e wireproto: buffer output frames when in half duplex mode Gregory Szorc <gregory.szorc@gmail.com> parents: 37058 diff changeset	651 No more frames will be received. All pending activity should be
861e9d37e56e wireproto: buffer output frames when in half duplex mode Gregory Szorc <gregory.szorc@gmail.com> parents: 37058 diff changeset	652 completed.
861e9d37e56e wireproto: buffer output frames when in half duplex mode Gregory Szorc <gregory.szorc@gmail.com> parents: 37058 diff changeset	653 """
37061 c5e9c3b47366 wireproto: support for receiving multiple requests Gregory Szorc <gregory.szorc@gmail.com> parents: 37060 diff changeset	654 # TODO should we do anything about in-flight commands?
c5e9c3b47366 wireproto: support for receiving multiple requests Gregory Szorc <gregory.szorc@gmail.com> parents: 37060 diff changeset	655
37059 861e9d37e56e wireproto: buffer output frames when in half duplex mode Gregory Szorc <gregory.szorc@gmail.com> parents: 37058 diff changeset	656 if not self._deferoutput or not self._bufferedframegens:
861e9d37e56e wireproto: buffer output frames when in half duplex mode Gregory Szorc <gregory.szorc@gmail.com> parents: 37058 diff changeset	657 return 'noop', {}
861e9d37e56e wireproto: buffer output frames when in half duplex mode Gregory Szorc <gregory.szorc@gmail.com> parents: 37058 diff changeset	658
861e9d37e56e wireproto: buffer output frames when in half duplex mode Gregory Szorc <gregory.szorc@gmail.com> parents: 37058 diff changeset	659 # If we buffered all our responses, emit those.
861e9d37e56e wireproto: buffer output frames when in half duplex mode Gregory Szorc <gregory.szorc@gmail.com> parents: 37058 diff changeset	660 def makegen():
861e9d37e56e wireproto: buffer output frames when in half duplex mode Gregory Szorc <gregory.szorc@gmail.com> parents: 37058 diff changeset	661 for gen in self._bufferedframegens:
861e9d37e56e wireproto: buffer output frames when in half duplex mode Gregory Szorc <gregory.szorc@gmail.com> parents: 37058 diff changeset	662 for frame in gen:
861e9d37e56e wireproto: buffer output frames when in half duplex mode Gregory Szorc <gregory.szorc@gmail.com> parents: 37058 diff changeset	663 yield frame
861e9d37e56e wireproto: buffer output frames when in half duplex mode Gregory Szorc <gregory.szorc@gmail.com> parents: 37058 diff changeset	664
37058 61393f888dfe wireproto: define and implement responses in framing protocol Gregory Szorc <gregory.szorc@gmail.com> parents: 37055 diff changeset	665 return 'sendframes', {
37059 861e9d37e56e wireproto: buffer output frames when in half duplex mode Gregory Szorc <gregory.szorc@gmail.com> parents: 37058 diff changeset	666 'framegen': makegen(),
37058 61393f888dfe wireproto: define and implement responses in framing protocol Gregory Szorc <gregory.szorc@gmail.com> parents: 37055 diff changeset	667 }
61393f888dfe wireproto: define and implement responses in framing protocol Gregory Szorc <gregory.szorc@gmail.com> parents: 37055 diff changeset	668
37287 3ed344546d9e wireproto: start to associate frame generation with a stream Gregory Szorc <gregory.szorc@gmail.com> parents: 37087 diff changeset	669 def onapplicationerror(self, stream, requestid, msg):
37288 9bfcbe4f4745 wireproto: add streams to frame-based protocol Gregory Szorc <gregory.szorc@gmail.com> parents: 37287 diff changeset	670 ensureserverstream(stream)
9bfcbe4f4745 wireproto: add streams to frame-based protocol Gregory Szorc <gregory.szorc@gmail.com> parents: 37287 diff changeset	671
37058 61393f888dfe wireproto: define and implement responses in framing protocol Gregory Szorc <gregory.szorc@gmail.com> parents: 37055 diff changeset	672 return 'sendframes', {
37287 3ed344546d9e wireproto: start to associate frame generation with a stream Gregory Szorc <gregory.szorc@gmail.com> parents: 37087 diff changeset	673 'framegen': createerrorframe(stream, requestid, msg,
3ed344546d9e wireproto: start to associate frame generation with a stream Gregory Szorc <gregory.szorc@gmail.com> parents: 37087 diff changeset	674 application=True),
37058 61393f888dfe wireproto: define and implement responses in framing protocol Gregory Szorc <gregory.szorc@gmail.com> parents: 37055 diff changeset	675 }
61393f888dfe wireproto: define and implement responses in framing protocol Gregory Szorc <gregory.szorc@gmail.com> parents: 37055 diff changeset	676
37289 5fadc63ac99f wireproto: explicit API to create outgoing streams Gregory Szorc <gregory.szorc@gmail.com> parents: 37288 diff changeset	677 def makeoutputstream(self):
5fadc63ac99f wireproto: explicit API to create outgoing streams Gregory Szorc <gregory.szorc@gmail.com> parents: 37288 diff changeset	678 """Create a stream to be used for sending data to the client."""
5fadc63ac99f wireproto: explicit API to create outgoing streams Gregory Szorc <gregory.szorc@gmail.com> parents: 37288 diff changeset	679 streamid = self._nextoutgoingstreamid
5fadc63ac99f wireproto: explicit API to create outgoing streams Gregory Szorc <gregory.szorc@gmail.com> parents: 37288 diff changeset	680 self._nextoutgoingstreamid += 2
5fadc63ac99f wireproto: explicit API to create outgoing streams Gregory Szorc <gregory.szorc@gmail.com> parents: 37288 diff changeset	681
5fadc63ac99f wireproto: explicit API to create outgoing streams Gregory Szorc <gregory.szorc@gmail.com> parents: 37288 diff changeset	682 s = stream(streamid)
5fadc63ac99f wireproto: explicit API to create outgoing streams Gregory Szorc <gregory.szorc@gmail.com> parents: 37288 diff changeset	683 self._outgoingstreams[streamid] = s
5fadc63ac99f wireproto: explicit API to create outgoing streams Gregory Szorc <gregory.szorc@gmail.com> parents: 37288 diff changeset	684
5fadc63ac99f wireproto: explicit API to create outgoing streams Gregory Szorc <gregory.szorc@gmail.com> parents: 37288 diff changeset	685 return s
5fadc63ac99f wireproto: explicit API to create outgoing streams Gregory Szorc <gregory.szorc@gmail.com> parents: 37288 diff changeset	686
37055 8c3c47362934 wireproto: implement basic frame reading and processing Gregory Szorc <gregory.szorc@gmail.com> parents: 37054 diff changeset	687 def _makeerrorresult(self, msg):
8c3c47362934 wireproto: implement basic frame reading and processing Gregory Szorc <gregory.szorc@gmail.com> parents: 37054 diff changeset	688 return 'error', {
8c3c47362934 wireproto: implement basic frame reading and processing Gregory Szorc <gregory.szorc@gmail.com> parents: 37054 diff changeset	689 'message': msg,
8c3c47362934 wireproto: implement basic frame reading and processing Gregory Szorc <gregory.szorc@gmail.com> parents: 37054 diff changeset	690 }
8c3c47362934 wireproto: implement basic frame reading and processing Gregory Szorc <gregory.szorc@gmail.com> parents: 37054 diff changeset	691
37061 c5e9c3b47366 wireproto: support for receiving multiple requests Gregory Szorc <gregory.szorc@gmail.com> parents: 37060 diff changeset	692 def _makeruncommandresult(self, requestid):
c5e9c3b47366 wireproto: support for receiving multiple requests Gregory Szorc <gregory.szorc@gmail.com> parents: 37060 diff changeset	693 entry = self._receivingcommands[requestid]
37292 3d0e2cd86e05 wireproto: use CBOR for command requests Gregory Szorc <gregory.szorc@gmail.com> parents: 37291 diff changeset	694
3d0e2cd86e05 wireproto: use CBOR for command requests Gregory Szorc <gregory.szorc@gmail.com> parents: 37291 diff changeset	695 if not entry['requestdone']:
3d0e2cd86e05 wireproto: use CBOR for command requests Gregory Szorc <gregory.szorc@gmail.com> parents: 37291 diff changeset	696 self._state = 'errored'
3d0e2cd86e05 wireproto: use CBOR for command requests Gregory Szorc <gregory.szorc@gmail.com> parents: 37291 diff changeset	697 raise error.ProgrammingError('should not be called without '
3d0e2cd86e05 wireproto: use CBOR for command requests Gregory Szorc <gregory.szorc@gmail.com> parents: 37291 diff changeset	698 'requestdone set')
3d0e2cd86e05 wireproto: use CBOR for command requests Gregory Szorc <gregory.szorc@gmail.com> parents: 37291 diff changeset	699
37061 c5e9c3b47366 wireproto: support for receiving multiple requests Gregory Szorc <gregory.szorc@gmail.com> parents: 37060 diff changeset	700 del self._receivingcommands[requestid]
c5e9c3b47366 wireproto: support for receiving multiple requests Gregory Szorc <gregory.szorc@gmail.com> parents: 37060 diff changeset	701
c5e9c3b47366 wireproto: support for receiving multiple requests Gregory Szorc <gregory.szorc@gmail.com> parents: 37060 diff changeset	702 if self._receivingcommands:
c5e9c3b47366 wireproto: support for receiving multiple requests Gregory Szorc <gregory.szorc@gmail.com> parents: 37060 diff changeset	703 self._state = 'command-receiving'
c5e9c3b47366 wireproto: support for receiving multiple requests Gregory Szorc <gregory.szorc@gmail.com> parents: 37060 diff changeset	704 else:
c5e9c3b47366 wireproto: support for receiving multiple requests Gregory Szorc <gregory.szorc@gmail.com> parents: 37060 diff changeset	705 self._state = 'idle'
c5e9c3b47366 wireproto: support for receiving multiple requests Gregory Szorc <gregory.szorc@gmail.com> parents: 37060 diff changeset	706
37292 3d0e2cd86e05 wireproto: use CBOR for command requests Gregory Szorc <gregory.szorc@gmail.com> parents: 37291 diff changeset	707 # Decode the payloads as CBOR.
3d0e2cd86e05 wireproto: use CBOR for command requests Gregory Szorc <gregory.szorc@gmail.com> parents: 37291 diff changeset	708 entry['payload'].seek(0)
3d0e2cd86e05 wireproto: use CBOR for command requests Gregory Szorc <gregory.szorc@gmail.com> parents: 37291 diff changeset	709 request = cbor.load(entry['payload'])
3d0e2cd86e05 wireproto: use CBOR for command requests Gregory Szorc <gregory.szorc@gmail.com> parents: 37291 diff changeset	710
3d0e2cd86e05 wireproto: use CBOR for command requests Gregory Szorc <gregory.szorc@gmail.com> parents: 37291 diff changeset	711 if b'name' not in request:
3d0e2cd86e05 wireproto: use CBOR for command requests Gregory Szorc <gregory.szorc@gmail.com> parents: 37291 diff changeset	712 self._state = 'errored'
3d0e2cd86e05 wireproto: use CBOR for command requests Gregory Szorc <gregory.szorc@gmail.com> parents: 37291 diff changeset	713 return self._makeerrorresult(
3d0e2cd86e05 wireproto: use CBOR for command requests Gregory Szorc <gregory.szorc@gmail.com> parents: 37291 diff changeset	714 _('command request missing "name" field'))
3d0e2cd86e05 wireproto: use CBOR for command requests Gregory Szorc <gregory.szorc@gmail.com> parents: 37291 diff changeset	715
3d0e2cd86e05 wireproto: use CBOR for command requests Gregory Szorc <gregory.szorc@gmail.com> parents: 37291 diff changeset	716 if b'args' not in request:
3d0e2cd86e05 wireproto: use CBOR for command requests Gregory Szorc <gregory.szorc@gmail.com> parents: 37291 diff changeset	717 request[b'args'] = {}
3d0e2cd86e05 wireproto: use CBOR for command requests Gregory Szorc <gregory.szorc@gmail.com> parents: 37291 diff changeset	718
37066 39304dd63589 wireproto: explicitly track which requests are active Gregory Szorc <gregory.szorc@gmail.com> parents: 37065 diff changeset	719 assert requestid not in self._activecommands
39304dd63589 wireproto: explicitly track which requests are active Gregory Szorc <gregory.szorc@gmail.com> parents: 37065 diff changeset	720 self._activecommands.add(requestid)
39304dd63589 wireproto: explicitly track which requests are active Gregory Szorc <gregory.szorc@gmail.com> parents: 37065 diff changeset	721
37055 8c3c47362934 wireproto: implement basic frame reading and processing Gregory Szorc <gregory.szorc@gmail.com> parents: 37054 diff changeset	722 return 'runcommand', {
37061 c5e9c3b47366 wireproto: support for receiving multiple requests Gregory Szorc <gregory.szorc@gmail.com> parents: 37060 diff changeset	723 'requestid': requestid,
37292 3d0e2cd86e05 wireproto: use CBOR for command requests Gregory Szorc <gregory.szorc@gmail.com> parents: 37291 diff changeset	724 'command': request[b'name'],
3d0e2cd86e05 wireproto: use CBOR for command requests Gregory Szorc <gregory.szorc@gmail.com> parents: 37291 diff changeset	725 'args': request[b'args'],
37061 c5e9c3b47366 wireproto: support for receiving multiple requests Gregory Szorc <gregory.szorc@gmail.com> parents: 37060 diff changeset	726 'data': entry['data'].getvalue() if entry['data'] else None,
37055 8c3c47362934 wireproto: implement basic frame reading and processing Gregory Szorc <gregory.szorc@gmail.com> parents: 37054 diff changeset	727 }
8c3c47362934 wireproto: implement basic frame reading and processing Gregory Szorc <gregory.szorc@gmail.com> parents: 37054 diff changeset	728
8c3c47362934 wireproto: implement basic frame reading and processing Gregory Szorc <gregory.szorc@gmail.com> parents: 37054 diff changeset	729 def _makewantframeresult(self):
8c3c47362934 wireproto: implement basic frame reading and processing Gregory Szorc <gregory.szorc@gmail.com> parents: 37054 diff changeset	730 return 'wantframe', {
8c3c47362934 wireproto: implement basic frame reading and processing Gregory Szorc <gregory.szorc@gmail.com> parents: 37054 diff changeset	731 'state': self._state,
8c3c47362934 wireproto: implement basic frame reading and processing Gregory Szorc <gregory.szorc@gmail.com> parents: 37054 diff changeset	732 }
8c3c47362934 wireproto: implement basic frame reading and processing Gregory Szorc <gregory.szorc@gmail.com> parents: 37054 diff changeset	733
37292 3d0e2cd86e05 wireproto: use CBOR for command requests Gregory Szorc <gregory.szorc@gmail.com> parents: 37291 diff changeset	734 def _validatecommandrequestframe(self, frame):
3d0e2cd86e05 wireproto: use CBOR for command requests Gregory Szorc <gregory.szorc@gmail.com> parents: 37291 diff changeset	735 new = frame.flags & FLAG_COMMAND_REQUEST_NEW
3d0e2cd86e05 wireproto: use CBOR for command requests Gregory Szorc <gregory.szorc@gmail.com> parents: 37291 diff changeset	736 continuation = frame.flags & FLAG_COMMAND_REQUEST_CONTINUATION
3d0e2cd86e05 wireproto: use CBOR for command requests Gregory Szorc <gregory.szorc@gmail.com> parents: 37291 diff changeset	737
3d0e2cd86e05 wireproto: use CBOR for command requests Gregory Szorc <gregory.szorc@gmail.com> parents: 37291 diff changeset	738 if new and continuation:
3d0e2cd86e05 wireproto: use CBOR for command requests Gregory Szorc <gregory.szorc@gmail.com> parents: 37291 diff changeset	739 self._state = 'errored'
3d0e2cd86e05 wireproto: use CBOR for command requests Gregory Szorc <gregory.szorc@gmail.com> parents: 37291 diff changeset	740 return self._makeerrorresult(
3d0e2cd86e05 wireproto: use CBOR for command requests Gregory Szorc <gregory.szorc@gmail.com> parents: 37291 diff changeset	741 _('received command request frame with both new and '
3d0e2cd86e05 wireproto: use CBOR for command requests Gregory Szorc <gregory.szorc@gmail.com> parents: 37291 diff changeset	742 'continuation flags set'))
3d0e2cd86e05 wireproto: use CBOR for command requests Gregory Szorc <gregory.szorc@gmail.com> parents: 37291 diff changeset	743
3d0e2cd86e05 wireproto: use CBOR for command requests Gregory Szorc <gregory.szorc@gmail.com> parents: 37291 diff changeset	744 if not new and not continuation:
3d0e2cd86e05 wireproto: use CBOR for command requests Gregory Szorc <gregory.szorc@gmail.com> parents: 37291 diff changeset	745 self._state = 'errored'
3d0e2cd86e05 wireproto: use CBOR for command requests Gregory Szorc <gregory.szorc@gmail.com> parents: 37291 diff changeset	746 return self._makeerrorresult(
3d0e2cd86e05 wireproto: use CBOR for command requests Gregory Szorc <gregory.szorc@gmail.com> parents: 37291 diff changeset	747 _('received command request frame with neither new nor '
3d0e2cd86e05 wireproto: use CBOR for command requests Gregory Szorc <gregory.szorc@gmail.com> parents: 37291 diff changeset	748 'continuation flags set'))
3d0e2cd86e05 wireproto: use CBOR for command requests Gregory Szorc <gregory.szorc@gmail.com> parents: 37291 diff changeset	749
37064 884a0c1604ad wireproto: define attr-based classes for representing frames Gregory Szorc <gregory.szorc@gmail.com> parents: 37063 diff changeset	750 def _onframeidle(self, frame):
37055 8c3c47362934 wireproto: implement basic frame reading and processing Gregory Szorc <gregory.szorc@gmail.com> parents: 37054 diff changeset	751 # The only frame type that should be received in this state is a
8c3c47362934 wireproto: implement basic frame reading and processing Gregory Szorc <gregory.szorc@gmail.com> parents: 37054 diff changeset	752 # command request.
37292 3d0e2cd86e05 wireproto: use CBOR for command requests Gregory Szorc <gregory.szorc@gmail.com> parents: 37291 diff changeset	753 if frame.typeid != FRAME_TYPE_COMMAND_REQUEST:
37055 8c3c47362934 wireproto: implement basic frame reading and processing Gregory Szorc <gregory.szorc@gmail.com> parents: 37054 diff changeset	754 self._state = 'errored'
8c3c47362934 wireproto: implement basic frame reading and processing Gregory Szorc <gregory.szorc@gmail.com> parents: 37054 diff changeset	755 return self._makeerrorresult(
37292 3d0e2cd86e05 wireproto: use CBOR for command requests Gregory Szorc <gregory.szorc@gmail.com> parents: 37291 diff changeset	756 _('expected command request frame; got %d') % frame.typeid)
3d0e2cd86e05 wireproto: use CBOR for command requests Gregory Szorc <gregory.szorc@gmail.com> parents: 37291 diff changeset	757
3d0e2cd86e05 wireproto: use CBOR for command requests Gregory Szorc <gregory.szorc@gmail.com> parents: 37291 diff changeset	758 res = self._validatecommandrequestframe(frame)
3d0e2cd86e05 wireproto: use CBOR for command requests Gregory Szorc <gregory.szorc@gmail.com> parents: 37291 diff changeset	759 if res:
3d0e2cd86e05 wireproto: use CBOR for command requests Gregory Szorc <gregory.szorc@gmail.com> parents: 37291 diff changeset	760 return res
37055 8c3c47362934 wireproto: implement basic frame reading and processing Gregory Szorc <gregory.szorc@gmail.com> parents: 37054 diff changeset	761
37064 884a0c1604ad wireproto: define attr-based classes for representing frames Gregory Szorc <gregory.szorc@gmail.com> parents: 37063 diff changeset	762 if frame.requestid in self._receivingcommands:
37061 c5e9c3b47366 wireproto: support for receiving multiple requests Gregory Szorc <gregory.szorc@gmail.com> parents: 37060 diff changeset	763 self._state = 'errored'
c5e9c3b47366 wireproto: support for receiving multiple requests Gregory Szorc <gregory.szorc@gmail.com> parents: 37060 diff changeset	764 return self._makeerrorresult(
37064 884a0c1604ad wireproto: define attr-based classes for representing frames Gregory Szorc <gregory.szorc@gmail.com> parents: 37063 diff changeset	765 _('request with ID %d already received') % frame.requestid)
37061 c5e9c3b47366 wireproto: support for receiving multiple requests Gregory Szorc <gregory.szorc@gmail.com> parents: 37060 diff changeset	766
37066 39304dd63589 wireproto: explicitly track which requests are active Gregory Szorc <gregory.szorc@gmail.com> parents: 37065 diff changeset	767 if frame.requestid in self._activecommands:
39304dd63589 wireproto: explicitly track which requests are active Gregory Szorc <gregory.szorc@gmail.com> parents: 37065 diff changeset	768 self._state = 'errored'
37292 3d0e2cd86e05 wireproto: use CBOR for command requests Gregory Szorc <gregory.szorc@gmail.com> parents: 37291 diff changeset	769 return self._makeerrorresult(
3d0e2cd86e05 wireproto: use CBOR for command requests Gregory Szorc <gregory.szorc@gmail.com> parents: 37291 diff changeset	770 _('request with ID %d is already active') % frame.requestid)
3d0e2cd86e05 wireproto: use CBOR for command requests Gregory Szorc <gregory.szorc@gmail.com> parents: 37291 diff changeset	771
3d0e2cd86e05 wireproto: use CBOR for command requests Gregory Szorc <gregory.szorc@gmail.com> parents: 37291 diff changeset	772 new = frame.flags & FLAG_COMMAND_REQUEST_NEW
3d0e2cd86e05 wireproto: use CBOR for command requests Gregory Szorc <gregory.szorc@gmail.com> parents: 37291 diff changeset	773 moreframes = frame.flags & FLAG_COMMAND_REQUEST_MORE_FRAMES
3d0e2cd86e05 wireproto: use CBOR for command requests Gregory Szorc <gregory.szorc@gmail.com> parents: 37291 diff changeset	774 expectingdata = frame.flags & FLAG_COMMAND_REQUEST_EXPECT_DATA
37066 39304dd63589 wireproto: explicitly track which requests are active Gregory Szorc <gregory.szorc@gmail.com> parents: 37065 diff changeset	775
37292 3d0e2cd86e05 wireproto: use CBOR for command requests Gregory Szorc <gregory.szorc@gmail.com> parents: 37291 diff changeset	776 if not new:
3d0e2cd86e05 wireproto: use CBOR for command requests Gregory Szorc <gregory.szorc@gmail.com> parents: 37291 diff changeset	777 self._state = 'errored'
3d0e2cd86e05 wireproto: use CBOR for command requests Gregory Szorc <gregory.szorc@gmail.com> parents: 37291 diff changeset	778 return self._makeerrorresult(
3d0e2cd86e05 wireproto: use CBOR for command requests Gregory Szorc <gregory.szorc@gmail.com> parents: 37291 diff changeset	779 _('received command request frame without new flag set'))
3d0e2cd86e05 wireproto: use CBOR for command requests Gregory Szorc <gregory.szorc@gmail.com> parents: 37291 diff changeset	780
3d0e2cd86e05 wireproto: use CBOR for command requests Gregory Szorc <gregory.szorc@gmail.com> parents: 37291 diff changeset	781 payload = util.bytesio()
3d0e2cd86e05 wireproto: use CBOR for command requests Gregory Szorc <gregory.szorc@gmail.com> parents: 37291 diff changeset	782 payload.write(frame.payload)
37061 c5e9c3b47366 wireproto: support for receiving multiple requests Gregory Szorc <gregory.szorc@gmail.com> parents: 37060 diff changeset	783
37064 884a0c1604ad wireproto: define attr-based classes for representing frames Gregory Szorc <gregory.szorc@gmail.com> parents: 37063 diff changeset	784 self._receivingcommands[frame.requestid] = {
37292 3d0e2cd86e05 wireproto: use CBOR for command requests Gregory Szorc <gregory.szorc@gmail.com> parents: 37291 diff changeset	785 'payload': payload,
37061 c5e9c3b47366 wireproto: support for receiving multiple requests Gregory Szorc <gregory.szorc@gmail.com> parents: 37060 diff changeset	786 'data': None,
37292 3d0e2cd86e05 wireproto: use CBOR for command requests Gregory Szorc <gregory.szorc@gmail.com> parents: 37291 diff changeset	787 'requestdone': not moreframes,
3d0e2cd86e05 wireproto: use CBOR for command requests Gregory Szorc <gregory.szorc@gmail.com> parents: 37291 diff changeset	788 'expectingdata': bool(expectingdata),
37061 c5e9c3b47366 wireproto: support for receiving multiple requests Gregory Szorc <gregory.szorc@gmail.com> parents: 37060 diff changeset	789 }
37055 8c3c47362934 wireproto: implement basic frame reading and processing Gregory Szorc <gregory.szorc@gmail.com> parents: 37054 diff changeset	790
37292 3d0e2cd86e05 wireproto: use CBOR for command requests Gregory Szorc <gregory.szorc@gmail.com> parents: 37291 diff changeset	791 # This is the final frame for this request. Dispatch it.
3d0e2cd86e05 wireproto: use CBOR for command requests Gregory Szorc <gregory.szorc@gmail.com> parents: 37291 diff changeset	792 if not moreframes and not expectingdata:
37064 884a0c1604ad wireproto: define attr-based classes for representing frames Gregory Szorc <gregory.szorc@gmail.com> parents: 37063 diff changeset	793 return self._makeruncommandresult(frame.requestid)
37055 8c3c47362934 wireproto: implement basic frame reading and processing Gregory Szorc <gregory.szorc@gmail.com> parents: 37054 diff changeset	794
37292 3d0e2cd86e05 wireproto: use CBOR for command requests Gregory Szorc <gregory.szorc@gmail.com> parents: 37291 diff changeset	795 assert moreframes or expectingdata
3d0e2cd86e05 wireproto: use CBOR for command requests Gregory Szorc <gregory.szorc@gmail.com> parents: 37291 diff changeset	796 self._state = 'command-receiving'
3d0e2cd86e05 wireproto: use CBOR for command requests Gregory Szorc <gregory.szorc@gmail.com> parents: 37291 diff changeset	797 return self._makewantframeresult()
37055 8c3c47362934 wireproto: implement basic frame reading and processing Gregory Szorc <gregory.szorc@gmail.com> parents: 37054 diff changeset	798
37064 884a0c1604ad wireproto: define attr-based classes for representing frames Gregory Szorc <gregory.szorc@gmail.com> parents: 37063 diff changeset	799 def _onframecommandreceiving(self, frame):
37292 3d0e2cd86e05 wireproto: use CBOR for command requests Gregory Szorc <gregory.szorc@gmail.com> parents: 37291 diff changeset	800 if frame.typeid == FRAME_TYPE_COMMAND_REQUEST:
3d0e2cd86e05 wireproto: use CBOR for command requests Gregory Szorc <gregory.szorc@gmail.com> parents: 37291 diff changeset	801 # Process new command requests as such.
3d0e2cd86e05 wireproto: use CBOR for command requests Gregory Szorc <gregory.szorc@gmail.com> parents: 37291 diff changeset	802 if frame.flags & FLAG_COMMAND_REQUEST_NEW:
3d0e2cd86e05 wireproto: use CBOR for command requests Gregory Szorc <gregory.szorc@gmail.com> parents: 37291 diff changeset	803 return self._onframeidle(frame)
3d0e2cd86e05 wireproto: use CBOR for command requests Gregory Szorc <gregory.szorc@gmail.com> parents: 37291 diff changeset	804
3d0e2cd86e05 wireproto: use CBOR for command requests Gregory Szorc <gregory.szorc@gmail.com> parents: 37291 diff changeset	805 res = self._validatecommandrequestframe(frame)
3d0e2cd86e05 wireproto: use CBOR for command requests Gregory Szorc <gregory.szorc@gmail.com> parents: 37291 diff changeset	806 if res:
3d0e2cd86e05 wireproto: use CBOR for command requests Gregory Szorc <gregory.szorc@gmail.com> parents: 37291 diff changeset	807 return res
37061 c5e9c3b47366 wireproto: support for receiving multiple requests Gregory Szorc <gregory.szorc@gmail.com> parents: 37060 diff changeset	808
c5e9c3b47366 wireproto: support for receiving multiple requests Gregory Szorc <gregory.szorc@gmail.com> parents: 37060 diff changeset	809 # All other frames should be related to a command that is currently
37066 39304dd63589 wireproto: explicitly track which requests are active Gregory Szorc <gregory.szorc@gmail.com> parents: 37065 diff changeset	810 # receiving but is not active.
39304dd63589 wireproto: explicitly track which requests are active Gregory Szorc <gregory.szorc@gmail.com> parents: 37065 diff changeset	811 if frame.requestid in self._activecommands:
39304dd63589 wireproto: explicitly track which requests are active Gregory Szorc <gregory.szorc@gmail.com> parents: 37065 diff changeset	812 self._state = 'errored'
39304dd63589 wireproto: explicitly track which requests are active Gregory Szorc <gregory.szorc@gmail.com> parents: 37065 diff changeset	813 return self._makeerrorresult(
39304dd63589 wireproto: explicitly track which requests are active Gregory Szorc <gregory.szorc@gmail.com> parents: 37065 diff changeset	814 _('received frame for request that is still active: %d') %
39304dd63589 wireproto: explicitly track which requests are active Gregory Szorc <gregory.szorc@gmail.com> parents: 37065 diff changeset	815 frame.requestid)
39304dd63589 wireproto: explicitly track which requests are active Gregory Szorc <gregory.szorc@gmail.com> parents: 37065 diff changeset	816
37064 884a0c1604ad wireproto: define attr-based classes for representing frames Gregory Szorc <gregory.szorc@gmail.com> parents: 37063 diff changeset	817 if frame.requestid not in self._receivingcommands:
37055 8c3c47362934 wireproto: implement basic frame reading and processing Gregory Szorc <gregory.szorc@gmail.com> parents: 37054 diff changeset	818 self._state = 'errored'
37061 c5e9c3b47366 wireproto: support for receiving multiple requests Gregory Szorc <gregory.szorc@gmail.com> parents: 37060 diff changeset	819 return self._makeerrorresult(
c5e9c3b47366 wireproto: support for receiving multiple requests Gregory Szorc <gregory.szorc@gmail.com> parents: 37060 diff changeset	820 _('received frame for request that is not receiving: %d') %
37064 884a0c1604ad wireproto: define attr-based classes for representing frames Gregory Szorc <gregory.szorc@gmail.com> parents: 37063 diff changeset	821 frame.requestid)
37061 c5e9c3b47366 wireproto: support for receiving multiple requests Gregory Szorc <gregory.szorc@gmail.com> parents: 37060 diff changeset	822
37064 884a0c1604ad wireproto: define attr-based classes for representing frames Gregory Szorc <gregory.szorc@gmail.com> parents: 37063 diff changeset	823 entry = self._receivingcommands[frame.requestid]
37061 c5e9c3b47366 wireproto: support for receiving multiple requests Gregory Szorc <gregory.szorc@gmail.com> parents: 37060 diff changeset	824
37292 3d0e2cd86e05 wireproto: use CBOR for command requests Gregory Szorc <gregory.szorc@gmail.com> parents: 37291 diff changeset	825 if frame.typeid == FRAME_TYPE_COMMAND_REQUEST:
3d0e2cd86e05 wireproto: use CBOR for command requests Gregory Szorc <gregory.szorc@gmail.com> parents: 37291 diff changeset	826 moreframes = frame.flags & FLAG_COMMAND_REQUEST_MORE_FRAMES
3d0e2cd86e05 wireproto: use CBOR for command requests Gregory Szorc <gregory.szorc@gmail.com> parents: 37291 diff changeset	827 expectingdata = bool(frame.flags & FLAG_COMMAND_REQUEST_EXPECT_DATA)
3d0e2cd86e05 wireproto: use CBOR for command requests Gregory Szorc <gregory.szorc@gmail.com> parents: 37291 diff changeset	828
3d0e2cd86e05 wireproto: use CBOR for command requests Gregory Szorc <gregory.szorc@gmail.com> parents: 37291 diff changeset	829 if entry['requestdone']:
3d0e2cd86e05 wireproto: use CBOR for command requests Gregory Szorc <gregory.szorc@gmail.com> parents: 37291 diff changeset	830 self._state = 'errored'
3d0e2cd86e05 wireproto: use CBOR for command requests Gregory Szorc <gregory.szorc@gmail.com> parents: 37291 diff changeset	831 return self._makeerrorresult(
3d0e2cd86e05 wireproto: use CBOR for command requests Gregory Szorc <gregory.szorc@gmail.com> parents: 37291 diff changeset	832 _('received command request frame when request frames '
3d0e2cd86e05 wireproto: use CBOR for command requests Gregory Szorc <gregory.szorc@gmail.com> parents: 37291 diff changeset	833 'were supposedly done'))
3d0e2cd86e05 wireproto: use CBOR for command requests Gregory Szorc <gregory.szorc@gmail.com> parents: 37291 diff changeset	834
3d0e2cd86e05 wireproto: use CBOR for command requests Gregory Szorc <gregory.szorc@gmail.com> parents: 37291 diff changeset	835 if expectingdata != entry['expectingdata']:
37061 c5e9c3b47366 wireproto: support for receiving multiple requests Gregory Szorc <gregory.szorc@gmail.com> parents: 37060 diff changeset	836 self._state = 'errored'
37292 3d0e2cd86e05 wireproto: use CBOR for command requests Gregory Szorc <gregory.szorc@gmail.com> parents: 37291 diff changeset	837 return self._makeerrorresult(
3d0e2cd86e05 wireproto: use CBOR for command requests Gregory Szorc <gregory.szorc@gmail.com> parents: 37291 diff changeset	838 _('mismatch between expect data flag and previous frame'))
3d0e2cd86e05 wireproto: use CBOR for command requests Gregory Szorc <gregory.szorc@gmail.com> parents: 37291 diff changeset	839
3d0e2cd86e05 wireproto: use CBOR for command requests Gregory Szorc <gregory.szorc@gmail.com> parents: 37291 diff changeset	840 entry['payload'].write(frame.payload)
37061 c5e9c3b47366 wireproto: support for receiving multiple requests Gregory Szorc <gregory.szorc@gmail.com> parents: 37060 diff changeset	841
37292 3d0e2cd86e05 wireproto: use CBOR for command requests Gregory Szorc <gregory.szorc@gmail.com> parents: 37291 diff changeset	842 if not moreframes:
3d0e2cd86e05 wireproto: use CBOR for command requests Gregory Szorc <gregory.szorc@gmail.com> parents: 37291 diff changeset	843 entry['requestdone'] = True
3d0e2cd86e05 wireproto: use CBOR for command requests Gregory Szorc <gregory.szorc@gmail.com> parents: 37291 diff changeset	844
3d0e2cd86e05 wireproto: use CBOR for command requests Gregory Szorc <gregory.szorc@gmail.com> parents: 37291 diff changeset	845 if not moreframes and not expectingdata:
3d0e2cd86e05 wireproto: use CBOR for command requests Gregory Szorc <gregory.szorc@gmail.com> parents: 37291 diff changeset	846 return self._makeruncommandresult(frame.requestid)
3d0e2cd86e05 wireproto: use CBOR for command requests Gregory Szorc <gregory.szorc@gmail.com> parents: 37291 diff changeset	847
3d0e2cd86e05 wireproto: use CBOR for command requests Gregory Szorc <gregory.szorc@gmail.com> parents: 37291 diff changeset	848 return self._makewantframeresult()
37061 c5e9c3b47366 wireproto: support for receiving multiple requests Gregory Szorc <gregory.szorc@gmail.com> parents: 37060 diff changeset	849
37064 884a0c1604ad wireproto: define attr-based classes for representing frames Gregory Szorc <gregory.szorc@gmail.com> parents: 37063 diff changeset	850 elif frame.typeid == FRAME_TYPE_COMMAND_DATA:
37061 c5e9c3b47366 wireproto: support for receiving multiple requests Gregory Szorc <gregory.szorc@gmail.com> parents: 37060 diff changeset	851 if not entry['expectingdata']:
c5e9c3b47366 wireproto: support for receiving multiple requests Gregory Szorc <gregory.szorc@gmail.com> parents: 37060 diff changeset	852 self._state = 'errored'
c5e9c3b47366 wireproto: support for receiving multiple requests Gregory Szorc <gregory.szorc@gmail.com> parents: 37060 diff changeset	853 return self._makeerrorresult(_(
c5e9c3b47366 wireproto: support for receiving multiple requests Gregory Szorc <gregory.szorc@gmail.com> parents: 37060 diff changeset	854 'received command data frame for request that is not '
37064 884a0c1604ad wireproto: define attr-based classes for representing frames Gregory Szorc <gregory.szorc@gmail.com> parents: 37063 diff changeset	855 'expecting data: %d') % frame.requestid)
37061 c5e9c3b47366 wireproto: support for receiving multiple requests Gregory Szorc <gregory.szorc@gmail.com> parents: 37060 diff changeset	856
c5e9c3b47366 wireproto: support for receiving multiple requests Gregory Szorc <gregory.szorc@gmail.com> parents: 37060 diff changeset	857 if entry['data'] is None:
c5e9c3b47366 wireproto: support for receiving multiple requests Gregory Szorc <gregory.szorc@gmail.com> parents: 37060 diff changeset	858 entry['data'] = util.bytesio()
c5e9c3b47366 wireproto: support for receiving multiple requests Gregory Szorc <gregory.szorc@gmail.com> parents: 37060 diff changeset	859
37064 884a0c1604ad wireproto: define attr-based classes for representing frames Gregory Szorc <gregory.szorc@gmail.com> parents: 37063 diff changeset	860 return self._handlecommanddataframe(frame, entry)
37292 3d0e2cd86e05 wireproto: use CBOR for command requests Gregory Szorc <gregory.szorc@gmail.com> parents: 37291 diff changeset	861 else:
37055 8c3c47362934 wireproto: implement basic frame reading and processing Gregory Szorc <gregory.szorc@gmail.com> parents: 37054 diff changeset	862 self._state = 'errored'
37292 3d0e2cd86e05 wireproto: use CBOR for command requests Gregory Szorc <gregory.szorc@gmail.com> parents: 37291 diff changeset	863 return self._makeerrorresult(_(
3d0e2cd86e05 wireproto: use CBOR for command requests Gregory Szorc <gregory.szorc@gmail.com> parents: 37291 diff changeset	864 'received unexpected frame type: %d') % frame.typeid)
37055 8c3c47362934 wireproto: implement basic frame reading and processing Gregory Szorc <gregory.szorc@gmail.com> parents: 37054 diff changeset	865
37064 884a0c1604ad wireproto: define attr-based classes for representing frames Gregory Szorc <gregory.szorc@gmail.com> parents: 37063 diff changeset	866 def _handlecommanddataframe(self, frame, entry):
884a0c1604ad wireproto: define attr-based classes for representing frames Gregory Szorc <gregory.szorc@gmail.com> parents: 37063 diff changeset	867 assert frame.typeid == FRAME_TYPE_COMMAND_DATA
37055 8c3c47362934 wireproto: implement basic frame reading and processing Gregory Szorc <gregory.szorc@gmail.com> parents: 37054 diff changeset	868
8c3c47362934 wireproto: implement basic frame reading and processing Gregory Szorc <gregory.szorc@gmail.com> parents: 37054 diff changeset	869 # TODO support streaming data instead of buffering it.
37064 884a0c1604ad wireproto: define attr-based classes for representing frames Gregory Szorc <gregory.szorc@gmail.com> parents: 37063 diff changeset	870 entry['data'].write(frame.payload)
37055 8c3c47362934 wireproto: implement basic frame reading and processing Gregory Szorc <gregory.szorc@gmail.com> parents: 37054 diff changeset	871
37064 884a0c1604ad wireproto: define attr-based classes for representing frames Gregory Szorc <gregory.szorc@gmail.com> parents: 37063 diff changeset	872 if frame.flags & FLAG_COMMAND_DATA_CONTINUATION:
37055 8c3c47362934 wireproto: implement basic frame reading and processing Gregory Szorc <gregory.szorc@gmail.com> parents: 37054 diff changeset	873 return self._makewantframeresult()
37064 884a0c1604ad wireproto: define attr-based classes for representing frames Gregory Szorc <gregory.szorc@gmail.com> parents: 37063 diff changeset	874 elif frame.flags & FLAG_COMMAND_DATA_EOS:
37061 c5e9c3b47366 wireproto: support for receiving multiple requests Gregory Szorc <gregory.szorc@gmail.com> parents: 37060 diff changeset	875 entry['data'].seek(0)
37064 884a0c1604ad wireproto: define attr-based classes for representing frames Gregory Szorc <gregory.szorc@gmail.com> parents: 37063 diff changeset	876 return self._makeruncommandresult(frame.requestid)
37055 8c3c47362934 wireproto: implement basic frame reading and processing Gregory Szorc <gregory.szorc@gmail.com> parents: 37054 diff changeset	877 else:
8c3c47362934 wireproto: implement basic frame reading and processing Gregory Szorc <gregory.szorc@gmail.com> parents: 37054 diff changeset	878 self._state = 'errored'
8c3c47362934 wireproto: implement basic frame reading and processing Gregory Szorc <gregory.szorc@gmail.com> parents: 37054 diff changeset	879 return self._makeerrorresult(_('command data frame without '
8c3c47362934 wireproto: implement basic frame reading and processing Gregory Szorc <gregory.szorc@gmail.com> parents: 37054 diff changeset	880 'flags'))
8c3c47362934 wireproto: implement basic frame reading and processing Gregory Szorc <gregory.szorc@gmail.com> parents: 37054 diff changeset	881
37064 884a0c1604ad wireproto: define attr-based classes for representing frames Gregory Szorc <gregory.szorc@gmail.com> parents: 37063 diff changeset	882 def _onframeerrored(self, frame):
37055 8c3c47362934 wireproto: implement basic frame reading and processing Gregory Szorc <gregory.szorc@gmail.com> parents: 37054 diff changeset	883 return self._makeerrorresult(_('server already errored'))

Mercurial > public > mercurial-scm > hg-stable

annotate mercurial/wireprotoframing.py @ 37298:5ef2da00e935