mercurial-scm/hg: mercurial/wireprotoframing.py annotate

annotate mercurial/wireprotoframing.py @ 40130:5d44c4d1d516

wireprotov2: establish dedicated classes for input and output streams Streams are unidirectional. As part of implementing encoding/decoding support, it became clear that it didn't make sense for a generic "stream" class to hold functionality related to both encoding and decoding. So we create new classes to represent the flavor of stream. Differential Revision: https://phab.mercurial-scm.org/D4920

author	Gregory Szorc <gregory.szorc@gmail.com>
date	Thu, 04 Oct 2018 17:39:16 -0700
parents	293835e0fff7
children	e67522413ca8

rev	line source
37051 40206e227412 wireproto: define and implement protocol for issuing requests Gregory Szorc <gregory.szorc@gmail.com> parents: diff changeset	1 # wireprotoframing.py - unified framing protocol for wire protocol
40206e227412 wireproto: define and implement protocol for issuing requests Gregory Szorc <gregory.szorc@gmail.com> parents: diff changeset	2 #
40206e227412 wireproto: define and implement protocol for issuing requests Gregory Szorc <gregory.szorc@gmail.com> parents: diff changeset	3 # Copyright 2018 Gregory Szorc <gregory.szorc@gmail.com>
40206e227412 wireproto: define and implement protocol for issuing requests Gregory Szorc <gregory.szorc@gmail.com> parents: diff changeset	4 #
40206e227412 wireproto: define and implement protocol for issuing requests Gregory Szorc <gregory.szorc@gmail.com> parents: diff changeset	5 # This software may be used and distributed according to the terms of the
40206e227412 wireproto: define and implement protocol for issuing requests Gregory Szorc <gregory.szorc@gmail.com> parents: diff changeset	6 # GNU General Public License version 2 or any later version.
40206e227412 wireproto: define and implement protocol for issuing requests Gregory Szorc <gregory.szorc@gmail.com> parents: diff changeset	7
40206e227412 wireproto: define and implement protocol for issuing requests Gregory Szorc <gregory.szorc@gmail.com> parents: diff changeset	8 # This file contains functionality to support the unified frame-based wire
40206e227412 wireproto: define and implement protocol for issuing requests Gregory Szorc <gregory.szorc@gmail.com> parents: diff changeset	9 # protocol. For details about the protocol, see
40206e227412 wireproto: define and implement protocol for issuing requests Gregory Szorc <gregory.szorc@gmail.com> parents: diff changeset	10 # `hg help internals.wireprotocol`.
40206e227412 wireproto: define and implement protocol for issuing requests Gregory Szorc <gregory.szorc@gmail.com> parents: diff changeset	11
40206e227412 wireproto: define and implement protocol for issuing requests Gregory Szorc <gregory.szorc@gmail.com> parents: diff changeset	12 from __future__ import absolute_import
40206e227412 wireproto: define and implement protocol for issuing requests Gregory Szorc <gregory.szorc@gmail.com> parents: diff changeset	13
37543 01361be9e2dc wireproto: introduce a reactor for client-side state Gregory Szorc <gregory.szorc@gmail.com> parents: 37485 diff changeset	14 import collections
37051 40206e227412 wireproto: define and implement protocol for issuing requests Gregory Szorc <gregory.szorc@gmail.com> parents: diff changeset	15 import struct
40206e227412 wireproto: define and implement protocol for issuing requests Gregory Szorc <gregory.szorc@gmail.com> parents: diff changeset	16
37052 8c3c47362934 wireproto: implement basic frame reading and processing Gregory Szorc <gregory.szorc@gmail.com> parents: 37051 diff changeset	17 from .i18n import _
37061 884a0c1604ad wireproto: define attr-based classes for representing frames Gregory Szorc <gregory.szorc@gmail.com> parents: 37060 diff changeset	18 from .thirdparty import (
884a0c1604ad wireproto: define attr-based classes for representing frames Gregory Szorc <gregory.szorc@gmail.com> parents: 37060 diff changeset	19 attr,
884a0c1604ad wireproto: define attr-based classes for representing frames Gregory Szorc <gregory.szorc@gmail.com> parents: 37060 diff changeset	20 )
37051 40206e227412 wireproto: define and implement protocol for issuing requests Gregory Szorc <gregory.szorc@gmail.com> parents: diff changeset	21 from . import (
37474 d33997123ea5 py3: system-stringify repr(frame) Yuya Nishihara <yuya@tcha.org> parents: 37473 diff changeset	22 encoding,
37052 8c3c47362934 wireproto: implement basic frame reading and processing Gregory Szorc <gregory.szorc@gmail.com> parents: 37051 diff changeset	23 error,
40025 b099e6032f38 wireprotov2: server support for sending content redirects Gregory Szorc <gregory.szorc@gmail.com> parents: 40024 diff changeset	24 pycompat,
37051 40206e227412 wireproto: define and implement protocol for issuing requests Gregory Szorc <gregory.szorc@gmail.com> parents: diff changeset	25 util,
40020 ed919b90acda wireprotov2: define type to represent pre-encoded object Gregory Szorc <gregory.szorc@gmail.com> parents: 39838 diff changeset	26 wireprototypes,
37051 40206e227412 wireproto: define and implement protocol for issuing requests Gregory Szorc <gregory.szorc@gmail.com> parents: diff changeset	27 )
37084 f0b6fbea00cf stringutil: bulk-replace call sites to point to new module Yuya Nishihara <yuya@tcha.org> parents: 37063 diff changeset	28 from .utils import (
39446 36f487a332ad wireprotoframing: use our CBOR module Gregory Szorc <gregory.szorc@gmail.com> parents: 37728 diff changeset	29 cborutil,
37084 f0b6fbea00cf stringutil: bulk-replace call sites to point to new module Yuya Nishihara <yuya@tcha.org> parents: 37063 diff changeset	30 stringutil,
f0b6fbea00cf stringutil: bulk-replace call sites to point to new module Yuya Nishihara <yuya@tcha.org> parents: 37063 diff changeset	31 )
37051 40206e227412 wireproto: define and implement protocol for issuing requests Gregory Szorc <gregory.szorc@gmail.com> parents: diff changeset	32
37288 9bfcbe4f4745 wireproto: add streams to frame-based protocol Gregory Szorc <gregory.szorc@gmail.com> parents: 37285 diff changeset	33 FRAME_HEADER_SIZE = 8
37051 40206e227412 wireproto: define and implement protocol for issuing requests Gregory Szorc <gregory.szorc@gmail.com> parents: diff changeset	34 DEFAULT_MAX_FRAME_SIZE = 32768
40206e227412 wireproto: define and implement protocol for issuing requests Gregory Szorc <gregory.szorc@gmail.com> parents: diff changeset	35
37288 9bfcbe4f4745 wireproto: add streams to frame-based protocol Gregory Szorc <gregory.szorc@gmail.com> parents: 37285 diff changeset	36 STREAM_FLAG_BEGIN_STREAM = 0x01
9bfcbe4f4745 wireproto: add streams to frame-based protocol Gregory Szorc <gregory.szorc@gmail.com> parents: 37285 diff changeset	37 STREAM_FLAG_END_STREAM = 0x02
9bfcbe4f4745 wireproto: add streams to frame-based protocol Gregory Szorc <gregory.szorc@gmail.com> parents: 37285 diff changeset	38 STREAM_FLAG_ENCODING_APPLIED = 0x04
9bfcbe4f4745 wireproto: add streams to frame-based protocol Gregory Szorc <gregory.szorc@gmail.com> parents: 37285 diff changeset	39
9bfcbe4f4745 wireproto: add streams to frame-based protocol Gregory Szorc <gregory.szorc@gmail.com> parents: 37285 diff changeset	40 STREAM_FLAGS = {
9bfcbe4f4745 wireproto: add streams to frame-based protocol Gregory Szorc <gregory.szorc@gmail.com> parents: 37285 diff changeset	41 b'stream-begin': STREAM_FLAG_BEGIN_STREAM,
9bfcbe4f4745 wireproto: add streams to frame-based protocol Gregory Szorc <gregory.szorc@gmail.com> parents: 37285 diff changeset	42 b'stream-end': STREAM_FLAG_END_STREAM,
9bfcbe4f4745 wireproto: add streams to frame-based protocol Gregory Szorc <gregory.szorc@gmail.com> parents: 37285 diff changeset	43 b'encoded': STREAM_FLAG_ENCODING_APPLIED,
9bfcbe4f4745 wireproto: add streams to frame-based protocol Gregory Szorc <gregory.szorc@gmail.com> parents: 37285 diff changeset	44 }
9bfcbe4f4745 wireproto: add streams to frame-based protocol Gregory Szorc <gregory.szorc@gmail.com> parents: 37285 diff changeset	45
37292 3d0e2cd86e05 wireproto: use CBOR for command requests Gregory Szorc <gregory.szorc@gmail.com> parents: 37291 diff changeset	46 FRAME_TYPE_COMMAND_REQUEST = 0x01
37723 e8fba6d578f0 wireprotov2: change frame type value for command data Gregory Szorc <gregory.szorc@gmail.com> parents: 37722 diff changeset	47 FRAME_TYPE_COMMAND_DATA = 0x02
37724 deff7cf7eefd wireprotov2: change frame type and name for command response Gregory Szorc <gregory.szorc@gmail.com> parents: 37723 diff changeset	48 FRAME_TYPE_COMMAND_RESPONSE = 0x03
37055 61393f888dfe wireproto: define and implement responses in framing protocol Gregory Szorc <gregory.szorc@gmail.com> parents: 37052 diff changeset	49 FRAME_TYPE_ERROR_RESPONSE = 0x05
37060 0a6c5cc09a88 wireproto: define human output side channel frame Gregory Szorc <gregory.szorc@gmail.com> parents: 37058 diff changeset	50 FRAME_TYPE_TEXT_OUTPUT = 0x06
37291 b0041036214e wireproto: define frame to represent progress updates Gregory Szorc <gregory.szorc@gmail.com> parents: 37290 diff changeset	51 FRAME_TYPE_PROGRESS = 0x07
40125 e2fe1074024c wireprotov2: update stream encoding specification Gregory Szorc <gregory.szorc@gmail.com> parents: 40025 diff changeset	52 FRAME_TYPE_SENDER_PROTOCOL_SETTINGS = 0x08
e2fe1074024c wireprotov2: update stream encoding specification Gregory Szorc <gregory.szorc@gmail.com> parents: 40025 diff changeset	53 FRAME_TYPE_STREAM_SETTINGS = 0x09
37051 40206e227412 wireproto: define and implement protocol for issuing requests Gregory Szorc <gregory.szorc@gmail.com> parents: diff changeset	54
40206e227412 wireproto: define and implement protocol for issuing requests Gregory Szorc <gregory.szorc@gmail.com> parents: diff changeset	55 FRAME_TYPES = {
37292 3d0e2cd86e05 wireproto: use CBOR for command requests Gregory Szorc <gregory.szorc@gmail.com> parents: 37291 diff changeset	56 b'command-request': FRAME_TYPE_COMMAND_REQUEST,
37051 40206e227412 wireproto: define and implement protocol for issuing requests Gregory Szorc <gregory.szorc@gmail.com> parents: diff changeset	57 b'command-data': FRAME_TYPE_COMMAND_DATA,
37724 deff7cf7eefd wireprotov2: change frame type and name for command response Gregory Szorc <gregory.szorc@gmail.com> parents: 37723 diff changeset	58 b'command-response': FRAME_TYPE_COMMAND_RESPONSE,
37055 61393f888dfe wireproto: define and implement responses in framing protocol Gregory Szorc <gregory.szorc@gmail.com> parents: 37052 diff changeset	59 b'error-response': FRAME_TYPE_ERROR_RESPONSE,
37060 0a6c5cc09a88 wireproto: define human output side channel frame Gregory Szorc <gregory.szorc@gmail.com> parents: 37058 diff changeset	60 b'text-output': FRAME_TYPE_TEXT_OUTPUT,
37291 b0041036214e wireproto: define frame to represent progress updates Gregory Szorc <gregory.szorc@gmail.com> parents: 37290 diff changeset	61 b'progress': FRAME_TYPE_PROGRESS,
40125 e2fe1074024c wireprotov2: update stream encoding specification Gregory Szorc <gregory.szorc@gmail.com> parents: 40025 diff changeset	62 b'sender-protocol-settings': FRAME_TYPE_SENDER_PROTOCOL_SETTINGS,
37288 9bfcbe4f4745 wireproto: add streams to frame-based protocol Gregory Szorc <gregory.szorc@gmail.com> parents: 37285 diff changeset	63 b'stream-settings': FRAME_TYPE_STREAM_SETTINGS,
37051 40206e227412 wireproto: define and implement protocol for issuing requests Gregory Szorc <gregory.szorc@gmail.com> parents: diff changeset	64 }
40206e227412 wireproto: define and implement protocol for issuing requests Gregory Szorc <gregory.szorc@gmail.com> parents: diff changeset	65
37292 3d0e2cd86e05 wireproto: use CBOR for command requests Gregory Szorc <gregory.szorc@gmail.com> parents: 37291 diff changeset	66 FLAG_COMMAND_REQUEST_NEW = 0x01
3d0e2cd86e05 wireproto: use CBOR for command requests Gregory Szorc <gregory.szorc@gmail.com> parents: 37291 diff changeset	67 FLAG_COMMAND_REQUEST_CONTINUATION = 0x02
3d0e2cd86e05 wireproto: use CBOR for command requests Gregory Szorc <gregory.szorc@gmail.com> parents: 37291 diff changeset	68 FLAG_COMMAND_REQUEST_MORE_FRAMES = 0x04
3d0e2cd86e05 wireproto: use CBOR for command requests Gregory Szorc <gregory.szorc@gmail.com> parents: 37291 diff changeset	69 FLAG_COMMAND_REQUEST_EXPECT_DATA = 0x08
37051 40206e227412 wireproto: define and implement protocol for issuing requests Gregory Szorc <gregory.szorc@gmail.com> parents: diff changeset	70
37292 3d0e2cd86e05 wireproto: use CBOR for command requests Gregory Szorc <gregory.szorc@gmail.com> parents: 37291 diff changeset	71 FLAGS_COMMAND_REQUEST = {
3d0e2cd86e05 wireproto: use CBOR for command requests Gregory Szorc <gregory.szorc@gmail.com> parents: 37291 diff changeset	72 b'new': FLAG_COMMAND_REQUEST_NEW,
3d0e2cd86e05 wireproto: use CBOR for command requests Gregory Szorc <gregory.szorc@gmail.com> parents: 37291 diff changeset	73 b'continuation': FLAG_COMMAND_REQUEST_CONTINUATION,
3d0e2cd86e05 wireproto: use CBOR for command requests Gregory Szorc <gregory.szorc@gmail.com> parents: 37291 diff changeset	74 b'more': FLAG_COMMAND_REQUEST_MORE_FRAMES,
3d0e2cd86e05 wireproto: use CBOR for command requests Gregory Szorc <gregory.szorc@gmail.com> parents: 37291 diff changeset	75 b'have-data': FLAG_COMMAND_REQUEST_EXPECT_DATA,
37051 40206e227412 wireproto: define and implement protocol for issuing requests Gregory Szorc <gregory.szorc@gmail.com> parents: diff changeset	76 }
40206e227412 wireproto: define and implement protocol for issuing requests Gregory Szorc <gregory.szorc@gmail.com> parents: diff changeset	77
40206e227412 wireproto: define and implement protocol for issuing requests Gregory Szorc <gregory.szorc@gmail.com> parents: diff changeset	78 FLAG_COMMAND_DATA_CONTINUATION = 0x01
40206e227412 wireproto: define and implement protocol for issuing requests Gregory Szorc <gregory.szorc@gmail.com> parents: diff changeset	79 FLAG_COMMAND_DATA_EOS = 0x02
40206e227412 wireproto: define and implement protocol for issuing requests Gregory Szorc <gregory.szorc@gmail.com> parents: diff changeset	80
40206e227412 wireproto: define and implement protocol for issuing requests Gregory Szorc <gregory.szorc@gmail.com> parents: diff changeset	81 FLAGS_COMMAND_DATA = {
40206e227412 wireproto: define and implement protocol for issuing requests Gregory Szorc <gregory.szorc@gmail.com> parents: diff changeset	82 b'continuation': FLAG_COMMAND_DATA_CONTINUATION,
40206e227412 wireproto: define and implement protocol for issuing requests Gregory Szorc <gregory.szorc@gmail.com> parents: diff changeset	83 b'eos': FLAG_COMMAND_DATA_EOS,
40206e227412 wireproto: define and implement protocol for issuing requests Gregory Szorc <gregory.szorc@gmail.com> parents: diff changeset	84 }
40206e227412 wireproto: define and implement protocol for issuing requests Gregory Szorc <gregory.szorc@gmail.com> parents: diff changeset	85
37724 deff7cf7eefd wireprotov2: change frame type and name for command response Gregory Szorc <gregory.szorc@gmail.com> parents: 37723 diff changeset	86 FLAG_COMMAND_RESPONSE_CONTINUATION = 0x01
deff7cf7eefd wireprotov2: change frame type and name for command response Gregory Szorc <gregory.szorc@gmail.com> parents: 37723 diff changeset	87 FLAG_COMMAND_RESPONSE_EOS = 0x02
37055 61393f888dfe wireproto: define and implement responses in framing protocol Gregory Szorc <gregory.szorc@gmail.com> parents: 37052 diff changeset	88
37724 deff7cf7eefd wireprotov2: change frame type and name for command response Gregory Szorc <gregory.szorc@gmail.com> parents: 37723 diff changeset	89 FLAGS_COMMAND_RESPONSE = {
deff7cf7eefd wireprotov2: change frame type and name for command response Gregory Szorc <gregory.szorc@gmail.com> parents: 37723 diff changeset	90 b'continuation': FLAG_COMMAND_RESPONSE_CONTINUATION,
deff7cf7eefd wireprotov2: change frame type and name for command response Gregory Szorc <gregory.szorc@gmail.com> parents: 37723 diff changeset	91 b'eos': FLAG_COMMAND_RESPONSE_EOS,
37055 61393f888dfe wireproto: define and implement responses in framing protocol Gregory Szorc <gregory.szorc@gmail.com> parents: 37052 diff changeset	92 }
61393f888dfe wireproto: define and implement responses in framing protocol Gregory Szorc <gregory.szorc@gmail.com> parents: 37052 diff changeset	93
40125 e2fe1074024c wireprotov2: update stream encoding specification Gregory Szorc <gregory.szorc@gmail.com> parents: 40025 diff changeset	94 FLAG_SENDER_PROTOCOL_SETTINGS_CONTINUATION = 0x01
e2fe1074024c wireprotov2: update stream encoding specification Gregory Szorc <gregory.szorc@gmail.com> parents: 40025 diff changeset	95 FLAG_SENDER_PROTOCOL_SETTINGS_EOS = 0x02
e2fe1074024c wireprotov2: update stream encoding specification Gregory Szorc <gregory.szorc@gmail.com> parents: 40025 diff changeset	96
e2fe1074024c wireprotov2: update stream encoding specification Gregory Szorc <gregory.szorc@gmail.com> parents: 40025 diff changeset	97 FLAGS_SENDER_PROTOCOL_SETTINGS = {
e2fe1074024c wireprotov2: update stream encoding specification Gregory Szorc <gregory.szorc@gmail.com> parents: 40025 diff changeset	98 b'continuation': FLAG_SENDER_PROTOCOL_SETTINGS_CONTINUATION,
e2fe1074024c wireprotov2: update stream encoding specification Gregory Szorc <gregory.szorc@gmail.com> parents: 40025 diff changeset	99 b'eos': FLAG_SENDER_PROTOCOL_SETTINGS_EOS,
e2fe1074024c wireprotov2: update stream encoding specification Gregory Szorc <gregory.szorc@gmail.com> parents: 40025 diff changeset	100 }
e2fe1074024c wireprotov2: update stream encoding specification Gregory Szorc <gregory.szorc@gmail.com> parents: 40025 diff changeset	101
e2fe1074024c wireprotov2: update stream encoding specification Gregory Szorc <gregory.szorc@gmail.com> parents: 40025 diff changeset	102 FLAG_STREAM_ENCODING_SETTINGS_CONTINUATION = 0x01
e2fe1074024c wireprotov2: update stream encoding specification Gregory Szorc <gregory.szorc@gmail.com> parents: 40025 diff changeset	103 FLAG_STREAM_ENCODING_SETTINGS_EOS = 0x02
e2fe1074024c wireprotov2: update stream encoding specification Gregory Szorc <gregory.szorc@gmail.com> parents: 40025 diff changeset	104
e2fe1074024c wireprotov2: update stream encoding specification Gregory Szorc <gregory.szorc@gmail.com> parents: 40025 diff changeset	105 FLAGS_STREAM_ENCODING_SETTINGS = {
e2fe1074024c wireprotov2: update stream encoding specification Gregory Szorc <gregory.szorc@gmail.com> parents: 40025 diff changeset	106 b'continuation': FLAG_STREAM_ENCODING_SETTINGS_CONTINUATION,
e2fe1074024c wireprotov2: update stream encoding specification Gregory Szorc <gregory.szorc@gmail.com> parents: 40025 diff changeset	107 b'eos': FLAG_STREAM_ENCODING_SETTINGS_EOS,
e2fe1074024c wireprotov2: update stream encoding specification Gregory Szorc <gregory.szorc@gmail.com> parents: 40025 diff changeset	108 }
e2fe1074024c wireprotov2: update stream encoding specification Gregory Szorc <gregory.szorc@gmail.com> parents: 40025 diff changeset	109
37051 40206e227412 wireproto: define and implement protocol for issuing requests Gregory Szorc <gregory.szorc@gmail.com> parents: diff changeset	110 # Maps frame types to their available flags.
40206e227412 wireproto: define and implement protocol for issuing requests Gregory Szorc <gregory.szorc@gmail.com> parents: diff changeset	111 FRAME_TYPE_FLAGS = {
37292 3d0e2cd86e05 wireproto: use CBOR for command requests Gregory Szorc <gregory.szorc@gmail.com> parents: 37291 diff changeset	112 FRAME_TYPE_COMMAND_REQUEST: FLAGS_COMMAND_REQUEST,
37051 40206e227412 wireproto: define and implement protocol for issuing requests Gregory Szorc <gregory.szorc@gmail.com> parents: diff changeset	113 FRAME_TYPE_COMMAND_DATA: FLAGS_COMMAND_DATA,
37724 deff7cf7eefd wireprotov2: change frame type and name for command response Gregory Szorc <gregory.szorc@gmail.com> parents: 37723 diff changeset	114 FRAME_TYPE_COMMAND_RESPONSE: FLAGS_COMMAND_RESPONSE,
37726 0c184ca594bb wireprotov2: change behavior of error frame Gregory Szorc <gregory.szorc@gmail.com> parents: 37725 diff changeset	115 FRAME_TYPE_ERROR_RESPONSE: {},
37060 0a6c5cc09a88 wireproto: define human output side channel frame Gregory Szorc <gregory.szorc@gmail.com> parents: 37058 diff changeset	116 FRAME_TYPE_TEXT_OUTPUT: {},
37291 b0041036214e wireproto: define frame to represent progress updates Gregory Szorc <gregory.szorc@gmail.com> parents: 37290 diff changeset	117 FRAME_TYPE_PROGRESS: {},
40125 e2fe1074024c wireprotov2: update stream encoding specification Gregory Szorc <gregory.szorc@gmail.com> parents: 40025 diff changeset	118 FRAME_TYPE_SENDER_PROTOCOL_SETTINGS: FLAGS_SENDER_PROTOCOL_SETTINGS,
e2fe1074024c wireprotov2: update stream encoding specification Gregory Szorc <gregory.szorc@gmail.com> parents: 40025 diff changeset	119 FRAME_TYPE_STREAM_SETTINGS: FLAGS_STREAM_ENCODING_SETTINGS,
37051 40206e227412 wireproto: define and implement protocol for issuing requests Gregory Szorc <gregory.szorc@gmail.com> parents: diff changeset	120 }
40206e227412 wireproto: define and implement protocol for issuing requests Gregory Szorc <gregory.szorc@gmail.com> parents: diff changeset	121
37292 3d0e2cd86e05 wireproto: use CBOR for command requests Gregory Szorc <gregory.szorc@gmail.com> parents: 37291 diff changeset	122 ARGUMENT_RECORD_HEADER = struct.Struct(r'<HH')
37051 40206e227412 wireproto: define and implement protocol for issuing requests Gregory Szorc <gregory.szorc@gmail.com> parents: diff changeset	123
37298 5ef2da00e935 wireproto: implement custom __repr__ for frame Gregory Szorc <gregory.szorc@gmail.com> parents: 37292 diff changeset	124 def humanflags(mapping, value):
5ef2da00e935 wireproto: implement custom __repr__ for frame Gregory Szorc <gregory.szorc@gmail.com> parents: 37292 diff changeset	125 """Convert a numeric flags value to a human value, using a mapping table."""
37473 7c2c7c749411 wireproto: show unknown id and flags in repr(frame) Yuya Nishihara <yuya@tcha.org> parents: 37472 diff changeset	126 namemap = {v: k for k, v in mapping.iteritems()}
37298 5ef2da00e935 wireproto: implement custom __repr__ for frame Gregory Szorc <gregory.szorc@gmail.com> parents: 37292 diff changeset	127 flags = []
37473 7c2c7c749411 wireproto: show unknown id and flags in repr(frame) Yuya Nishihara <yuya@tcha.org> parents: 37472 diff changeset	128 val = 1
7c2c7c749411 wireproto: show unknown id and flags in repr(frame) Yuya Nishihara <yuya@tcha.org> parents: 37472 diff changeset	129 while value >= val:
37298 5ef2da00e935 wireproto: implement custom __repr__ for frame Gregory Szorc <gregory.szorc@gmail.com> parents: 37292 diff changeset	130 if value & val:
37473 7c2c7c749411 wireproto: show unknown id and flags in repr(frame) Yuya Nishihara <yuya@tcha.org> parents: 37472 diff changeset	131 flags.append(namemap.get(val, '<unknown 0x%02x>' % val))
7c2c7c749411 wireproto: show unknown id and flags in repr(frame) Yuya Nishihara <yuya@tcha.org> parents: 37472 diff changeset	132 val <<= 1
37298 5ef2da00e935 wireproto: implement custom __repr__ for frame Gregory Szorc <gregory.szorc@gmail.com> parents: 37292 diff changeset	133
5ef2da00e935 wireproto: implement custom __repr__ for frame Gregory Szorc <gregory.szorc@gmail.com> parents: 37292 diff changeset	134 return b'\|'.join(flags)
5ef2da00e935 wireproto: implement custom __repr__ for frame Gregory Szorc <gregory.szorc@gmail.com> parents: 37292 diff changeset	135
37061 884a0c1604ad wireproto: define attr-based classes for representing frames Gregory Szorc <gregory.szorc@gmail.com> parents: 37060 diff changeset	136 @attr.s(slots=True)
884a0c1604ad wireproto: define attr-based classes for representing frames Gregory Szorc <gregory.szorc@gmail.com> parents: 37060 diff changeset	137 class frameheader(object):
884a0c1604ad wireproto: define attr-based classes for representing frames Gregory Szorc <gregory.szorc@gmail.com> parents: 37060 diff changeset	138 """Represents the data in a frame header."""
884a0c1604ad wireproto: define attr-based classes for representing frames Gregory Szorc <gregory.szorc@gmail.com> parents: 37060 diff changeset	139
884a0c1604ad wireproto: define attr-based classes for representing frames Gregory Szorc <gregory.szorc@gmail.com> parents: 37060 diff changeset	140 length = attr.ib()
884a0c1604ad wireproto: define attr-based classes for representing frames Gregory Szorc <gregory.szorc@gmail.com> parents: 37060 diff changeset	141 requestid = attr.ib()
37288 9bfcbe4f4745 wireproto: add streams to frame-based protocol Gregory Szorc <gregory.szorc@gmail.com> parents: 37285 diff changeset	142 streamid = attr.ib()
9bfcbe4f4745 wireproto: add streams to frame-based protocol Gregory Szorc <gregory.szorc@gmail.com> parents: 37285 diff changeset	143 streamflags = attr.ib()
37061 884a0c1604ad wireproto: define attr-based classes for representing frames Gregory Szorc <gregory.szorc@gmail.com> parents: 37060 diff changeset	144 typeid = attr.ib()
884a0c1604ad wireproto: define attr-based classes for representing frames Gregory Szorc <gregory.szorc@gmail.com> parents: 37060 diff changeset	145 flags = attr.ib()
884a0c1604ad wireproto: define attr-based classes for representing frames Gregory Szorc <gregory.szorc@gmail.com> parents: 37060 diff changeset	146
37298 5ef2da00e935 wireproto: implement custom __repr__ for frame Gregory Szorc <gregory.szorc@gmail.com> parents: 37292 diff changeset	147 @attr.s(slots=True, repr=False)
37061 884a0c1604ad wireproto: define attr-based classes for representing frames Gregory Szorc <gregory.szorc@gmail.com> parents: 37060 diff changeset	148 class frame(object):
884a0c1604ad wireproto: define attr-based classes for representing frames Gregory Szorc <gregory.szorc@gmail.com> parents: 37060 diff changeset	149 """Represents a parsed frame."""
884a0c1604ad wireproto: define attr-based classes for representing frames Gregory Szorc <gregory.szorc@gmail.com> parents: 37060 diff changeset	150
884a0c1604ad wireproto: define attr-based classes for representing frames Gregory Szorc <gregory.szorc@gmail.com> parents: 37060 diff changeset	151 requestid = attr.ib()
37288 9bfcbe4f4745 wireproto: add streams to frame-based protocol Gregory Szorc <gregory.szorc@gmail.com> parents: 37285 diff changeset	152 streamid = attr.ib()
9bfcbe4f4745 wireproto: add streams to frame-based protocol Gregory Szorc <gregory.szorc@gmail.com> parents: 37285 diff changeset	153 streamflags = attr.ib()
37061 884a0c1604ad wireproto: define attr-based classes for representing frames Gregory Szorc <gregory.szorc@gmail.com> parents: 37060 diff changeset	154 typeid = attr.ib()
884a0c1604ad wireproto: define attr-based classes for representing frames Gregory Szorc <gregory.szorc@gmail.com> parents: 37060 diff changeset	155 flags = attr.ib()
884a0c1604ad wireproto: define attr-based classes for representing frames Gregory Szorc <gregory.szorc@gmail.com> parents: 37060 diff changeset	156 payload = attr.ib()
884a0c1604ad wireproto: define attr-based classes for representing frames Gregory Szorc <gregory.szorc@gmail.com> parents: 37060 diff changeset	157
37474 d33997123ea5 py3: system-stringify repr(frame) Yuya Nishihara <yuya@tcha.org> parents: 37473 diff changeset	158 @encoding.strmethod
37298 5ef2da00e935 wireproto: implement custom __repr__ for frame Gregory Szorc <gregory.szorc@gmail.com> parents: 37292 diff changeset	159 def __repr__(self):
37473 7c2c7c749411 wireproto: show unknown id and flags in repr(frame) Yuya Nishihara <yuya@tcha.org> parents: 37472 diff changeset	160 typename = '<unknown 0x%02x>' % self.typeid
37298 5ef2da00e935 wireproto: implement custom __repr__ for frame Gregory Szorc <gregory.szorc@gmail.com> parents: 37292 diff changeset	161 for name, value in FRAME_TYPES.iteritems():
5ef2da00e935 wireproto: implement custom __repr__ for frame Gregory Szorc <gregory.szorc@gmail.com> parents: 37292 diff changeset	162 if value == self.typeid:
5ef2da00e935 wireproto: implement custom __repr__ for frame Gregory Szorc <gregory.szorc@gmail.com> parents: 37292 diff changeset	163 typename = name
5ef2da00e935 wireproto: implement custom __repr__ for frame Gregory Szorc <gregory.szorc@gmail.com> parents: 37292 diff changeset	164 break
5ef2da00e935 wireproto: implement custom __repr__ for frame Gregory Szorc <gregory.szorc@gmail.com> parents: 37292 diff changeset	165
5ef2da00e935 wireproto: implement custom __repr__ for frame Gregory Szorc <gregory.szorc@gmail.com> parents: 37292 diff changeset	166 return ('frame(size=%d; request=%d; stream=%d; streamflags=%s; '
5ef2da00e935 wireproto: implement custom __repr__ for frame Gregory Szorc <gregory.szorc@gmail.com> parents: 37292 diff changeset	167 'type=%s; flags=%s)' % (
5ef2da00e935 wireproto: implement custom __repr__ for frame Gregory Szorc <gregory.szorc@gmail.com> parents: 37292 diff changeset	168 len(self.payload), self.requestid, self.streamid,
5ef2da00e935 wireproto: implement custom __repr__ for frame Gregory Szorc <gregory.szorc@gmail.com> parents: 37292 diff changeset	169 humanflags(STREAM_FLAGS, self.streamflags), typename,
37472 2f81926c7f89 wireproto: fix repr(frame) to not crash by unknown type id Yuya Nishihara <yuya@tcha.org> parents: 37319 diff changeset	170 humanflags(FRAME_TYPE_FLAGS.get(self.typeid, {}), self.flags)))
37298 5ef2da00e935 wireproto: implement custom __repr__ for frame Gregory Szorc <gregory.szorc@gmail.com> parents: 37292 diff changeset	171
37288 9bfcbe4f4745 wireproto: add streams to frame-based protocol Gregory Szorc <gregory.szorc@gmail.com> parents: 37285 diff changeset	172 def makeframe(requestid, streamid, streamflags, typeid, flags, payload):
37051 40206e227412 wireproto: define and implement protocol for issuing requests Gregory Szorc <gregory.szorc@gmail.com> parents: diff changeset	173 """Assemble a frame into a byte array."""
40206e227412 wireproto: define and implement protocol for issuing requests Gregory Szorc <gregory.szorc@gmail.com> parents: diff changeset	174 # TODO assert size of payload.
40206e227412 wireproto: define and implement protocol for issuing requests Gregory Szorc <gregory.szorc@gmail.com> parents: diff changeset	175 frame = bytearray(FRAME_HEADER_SIZE + len(payload))
40206e227412 wireproto: define and implement protocol for issuing requests Gregory Szorc <gregory.szorc@gmail.com> parents: diff changeset	176
37057 2ec1fb9de638 wireproto: add request IDs to frames Gregory Szorc <gregory.szorc@gmail.com> parents: 37056 diff changeset	177 # 24 bits length
2ec1fb9de638 wireproto: add request IDs to frames Gregory Szorc <gregory.szorc@gmail.com> parents: 37056 diff changeset	178 # 16 bits request id
37288 9bfcbe4f4745 wireproto: add streams to frame-based protocol Gregory Szorc <gregory.szorc@gmail.com> parents: 37285 diff changeset	179 # 8 bits stream id
9bfcbe4f4745 wireproto: add streams to frame-based protocol Gregory Szorc <gregory.szorc@gmail.com> parents: 37285 diff changeset	180 # 8 bits stream flags
37057 2ec1fb9de638 wireproto: add request IDs to frames Gregory Szorc <gregory.szorc@gmail.com> parents: 37056 diff changeset	181 # 4 bits type
2ec1fb9de638 wireproto: add request IDs to frames Gregory Szorc <gregory.szorc@gmail.com> parents: 37056 diff changeset	182 # 4 bits flags
2ec1fb9de638 wireproto: add request IDs to frames Gregory Szorc <gregory.szorc@gmail.com> parents: 37056 diff changeset	183
37051 40206e227412 wireproto: define and implement protocol for issuing requests Gregory Szorc <gregory.szorc@gmail.com> parents: diff changeset	184 l = struct.pack(r'<I', len(payload))
40206e227412 wireproto: define and implement protocol for issuing requests Gregory Szorc <gregory.szorc@gmail.com> parents: diff changeset	185 frame[0:3] = l[0:3]
37288 9bfcbe4f4745 wireproto: add streams to frame-based protocol Gregory Szorc <gregory.szorc@gmail.com> parents: 37285 diff changeset	186 struct.pack_into(r'<HBB', frame, 3, requestid, streamid, streamflags)
9bfcbe4f4745 wireproto: add streams to frame-based protocol Gregory Szorc <gregory.szorc@gmail.com> parents: 37285 diff changeset	187 frame[7] = (typeid << 4) \| flags
9bfcbe4f4745 wireproto: add streams to frame-based protocol Gregory Szorc <gregory.szorc@gmail.com> parents: 37285 diff changeset	188 frame[8:] = payload
37051 40206e227412 wireproto: define and implement protocol for issuing requests Gregory Szorc <gregory.szorc@gmail.com> parents: diff changeset	189
40206e227412 wireproto: define and implement protocol for issuing requests Gregory Szorc <gregory.szorc@gmail.com> parents: diff changeset	190 return frame
40206e227412 wireproto: define and implement protocol for issuing requests Gregory Szorc <gregory.szorc@gmail.com> parents: diff changeset	191
40206e227412 wireproto: define and implement protocol for issuing requests Gregory Szorc <gregory.szorc@gmail.com> parents: diff changeset	192 def makeframefromhumanstring(s):
37057 2ec1fb9de638 wireproto: add request IDs to frames Gregory Szorc <gregory.szorc@gmail.com> parents: 37056 diff changeset	193 """Create a frame from a human readable string
2ec1fb9de638 wireproto: add request IDs to frames Gregory Szorc <gregory.szorc@gmail.com> parents: 37056 diff changeset	194
2ec1fb9de638 wireproto: add request IDs to frames Gregory Szorc <gregory.szorc@gmail.com> parents: 37056 diff changeset	195 Strings have the form:
2ec1fb9de638 wireproto: add request IDs to frames Gregory Szorc <gregory.szorc@gmail.com> parents: 37056 diff changeset	196
37288 9bfcbe4f4745 wireproto: add streams to frame-based protocol Gregory Szorc <gregory.szorc@gmail.com> parents: 37285 diff changeset	197 <request-id> <stream-id> <stream-flags> <type> <flags> <payload>
37051 40206e227412 wireproto: define and implement protocol for issuing requests Gregory Szorc <gregory.szorc@gmail.com> parents: diff changeset	198
40206e227412 wireproto: define and implement protocol for issuing requests Gregory Szorc <gregory.szorc@gmail.com> parents: diff changeset	199 This can be used by user-facing applications and tests for creating
40206e227412 wireproto: define and implement protocol for issuing requests Gregory Szorc <gregory.szorc@gmail.com> parents: diff changeset	200 frames easily without having to type out a bunch of constants.
40206e227412 wireproto: define and implement protocol for issuing requests Gregory Szorc <gregory.szorc@gmail.com> parents: diff changeset	201
37288 9bfcbe4f4745 wireproto: add streams to frame-based protocol Gregory Szorc <gregory.szorc@gmail.com> parents: 37285 diff changeset	202 Request ID and stream IDs are integers.
37057 2ec1fb9de638 wireproto: add request IDs to frames Gregory Szorc <gregory.szorc@gmail.com> parents: 37056 diff changeset	203
37288 9bfcbe4f4745 wireproto: add streams to frame-based protocol Gregory Szorc <gregory.szorc@gmail.com> parents: 37285 diff changeset	204 Stream flags, frame type, and flags can be specified by integer or
9bfcbe4f4745 wireproto: add streams to frame-based protocol Gregory Szorc <gregory.szorc@gmail.com> parents: 37285 diff changeset	205 named constant.
37057 2ec1fb9de638 wireproto: add request IDs to frames Gregory Szorc <gregory.szorc@gmail.com> parents: 37056 diff changeset	206
37051 40206e227412 wireproto: define and implement protocol for issuing requests Gregory Szorc <gregory.szorc@gmail.com> parents: diff changeset	207 Flags can be delimited by `\|` to bitwise OR them together.
37290 cc5a040fe150 wireproto: syntax for encoding CBOR into frames Gregory Szorc <gregory.szorc@gmail.com> parents: 37289 diff changeset	208
cc5a040fe150 wireproto: syntax for encoding CBOR into frames Gregory Szorc <gregory.szorc@gmail.com> parents: 37289 diff changeset	209 If the payload begins with ``cbor:``, the following string will be
37476 e9dea82ea1f3 wireproto: convert python literal to object without using unsafe eval() Yuya Nishihara <yuya@tcha.org> parents: 37474 diff changeset	210 evaluated as Python literal and the resulting object will be fed into
37290 cc5a040fe150 wireproto: syntax for encoding CBOR into frames Gregory Szorc <gregory.szorc@gmail.com> parents: 37289 diff changeset	211 a CBOR encoder. Otherwise, the payload is interpreted as a Python
cc5a040fe150 wireproto: syntax for encoding CBOR into frames Gregory Szorc <gregory.szorc@gmail.com> parents: 37289 diff changeset	212 byte string literal.
37051 40206e227412 wireproto: define and implement protocol for issuing requests Gregory Szorc <gregory.szorc@gmail.com> parents: diff changeset	213 """
37288 9bfcbe4f4745 wireproto: add streams to frame-based protocol Gregory Szorc <gregory.szorc@gmail.com> parents: 37285 diff changeset	214 fields = s.split(b' ', 5)
9bfcbe4f4745 wireproto: add streams to frame-based protocol Gregory Szorc <gregory.szorc@gmail.com> parents: 37285 diff changeset	215 requestid, streamid, streamflags, frametype, frameflags, payload = fields
37057 2ec1fb9de638 wireproto: add request IDs to frames Gregory Szorc <gregory.szorc@gmail.com> parents: 37056 diff changeset	216
2ec1fb9de638 wireproto: add request IDs to frames Gregory Szorc <gregory.szorc@gmail.com> parents: 37056 diff changeset	217 requestid = int(requestid)
37288 9bfcbe4f4745 wireproto: add streams to frame-based protocol Gregory Szorc <gregory.szorc@gmail.com> parents: 37285 diff changeset	218 streamid = int(streamid)
9bfcbe4f4745 wireproto: add streams to frame-based protocol Gregory Szorc <gregory.szorc@gmail.com> parents: 37285 diff changeset	219
9bfcbe4f4745 wireproto: add streams to frame-based protocol Gregory Szorc <gregory.szorc@gmail.com> parents: 37285 diff changeset	220 finalstreamflags = 0
9bfcbe4f4745 wireproto: add streams to frame-based protocol Gregory Szorc <gregory.szorc@gmail.com> parents: 37285 diff changeset	221 for flag in streamflags.split(b'\|'):
9bfcbe4f4745 wireproto: add streams to frame-based protocol Gregory Szorc <gregory.szorc@gmail.com> parents: 37285 diff changeset	222 if flag in STREAM_FLAGS:
9bfcbe4f4745 wireproto: add streams to frame-based protocol Gregory Szorc <gregory.szorc@gmail.com> parents: 37285 diff changeset	223 finalstreamflags \|= STREAM_FLAGS[flag]
9bfcbe4f4745 wireproto: add streams to frame-based protocol Gregory Szorc <gregory.szorc@gmail.com> parents: 37285 diff changeset	224 else:
9bfcbe4f4745 wireproto: add streams to frame-based protocol Gregory Szorc <gregory.szorc@gmail.com> parents: 37285 diff changeset	225 finalstreamflags \|= int(flag)
37051 40206e227412 wireproto: define and implement protocol for issuing requests Gregory Szorc <gregory.szorc@gmail.com> parents: diff changeset	226
40206e227412 wireproto: define and implement protocol for issuing requests Gregory Szorc <gregory.szorc@gmail.com> parents: diff changeset	227 if frametype in FRAME_TYPES:
40206e227412 wireproto: define and implement protocol for issuing requests Gregory Szorc <gregory.szorc@gmail.com> parents: diff changeset	228 frametype = FRAME_TYPES[frametype]
40206e227412 wireproto: define and implement protocol for issuing requests Gregory Szorc <gregory.szorc@gmail.com> parents: diff changeset	229 else:
40206e227412 wireproto: define and implement protocol for issuing requests Gregory Szorc <gregory.szorc@gmail.com> parents: diff changeset	230 frametype = int(frametype)
40206e227412 wireproto: define and implement protocol for issuing requests Gregory Szorc <gregory.szorc@gmail.com> parents: diff changeset	231
40206e227412 wireproto: define and implement protocol for issuing requests Gregory Szorc <gregory.szorc@gmail.com> parents: diff changeset	232 finalflags = 0
40206e227412 wireproto: define and implement protocol for issuing requests Gregory Szorc <gregory.szorc@gmail.com> parents: diff changeset	233 validflags = FRAME_TYPE_FLAGS[frametype]
40206e227412 wireproto: define and implement protocol for issuing requests Gregory Szorc <gregory.szorc@gmail.com> parents: diff changeset	234 for flag in frameflags.split(b'\|'):
40206e227412 wireproto: define and implement protocol for issuing requests Gregory Szorc <gregory.szorc@gmail.com> parents: diff changeset	235 if flag in validflags:
40206e227412 wireproto: define and implement protocol for issuing requests Gregory Szorc <gregory.szorc@gmail.com> parents: diff changeset	236 finalflags \|= validflags[flag]
40206e227412 wireproto: define and implement protocol for issuing requests Gregory Szorc <gregory.szorc@gmail.com> parents: diff changeset	237 else:
40206e227412 wireproto: define and implement protocol for issuing requests Gregory Szorc <gregory.szorc@gmail.com> parents: diff changeset	238 finalflags \|= int(flag)
40206e227412 wireproto: define and implement protocol for issuing requests Gregory Szorc <gregory.szorc@gmail.com> parents: diff changeset	239
37290 cc5a040fe150 wireproto: syntax for encoding CBOR into frames Gregory Szorc <gregory.szorc@gmail.com> parents: 37289 diff changeset	240 if payload.startswith(b'cbor:'):
39446 36f487a332ad wireprotoframing: use our CBOR module Gregory Szorc <gregory.szorc@gmail.com> parents: 37728 diff changeset	241 payload = b''.join(cborutil.streamencode(
36f487a332ad wireprotoframing: use our CBOR module Gregory Szorc <gregory.szorc@gmail.com> parents: 37728 diff changeset	242 stringutil.evalpythonliteral(payload[5:])))
37290 cc5a040fe150 wireproto: syntax for encoding CBOR into frames Gregory Szorc <gregory.szorc@gmail.com> parents: 37289 diff changeset	243
cc5a040fe150 wireproto: syntax for encoding CBOR into frames Gregory Szorc <gregory.szorc@gmail.com> parents: 37289 diff changeset	244 else:
cc5a040fe150 wireproto: syntax for encoding CBOR into frames Gregory Szorc <gregory.szorc@gmail.com> parents: 37289 diff changeset	245 payload = stringutil.unescapestr(payload)
37051 40206e227412 wireproto: define and implement protocol for issuing requests Gregory Szorc <gregory.szorc@gmail.com> parents: diff changeset	246
37288 9bfcbe4f4745 wireproto: add streams to frame-based protocol Gregory Szorc <gregory.szorc@gmail.com> parents: 37285 diff changeset	247 return makeframe(requestid=requestid, streamid=streamid,
9bfcbe4f4745 wireproto: add streams to frame-based protocol Gregory Szorc <gregory.szorc@gmail.com> parents: 37285 diff changeset	248 streamflags=finalstreamflags, typeid=frametype,
37062 fe4c944f95bb wireproto: use named arguments when passing around frame data Gregory Szorc <gregory.szorc@gmail.com> parents: 37061 diff changeset	249 flags=finalflags, payload=payload)
37051 40206e227412 wireproto: define and implement protocol for issuing requests Gregory Szorc <gregory.szorc@gmail.com> parents: diff changeset	250
37052 8c3c47362934 wireproto: implement basic frame reading and processing Gregory Szorc <gregory.szorc@gmail.com> parents: 37051 diff changeset	251 def parseheader(data):
8c3c47362934 wireproto: implement basic frame reading and processing Gregory Szorc <gregory.szorc@gmail.com> parents: 37051 diff changeset	252 """Parse a unified framing protocol frame header from a buffer.
8c3c47362934 wireproto: implement basic frame reading and processing Gregory Szorc <gregory.szorc@gmail.com> parents: 37051 diff changeset	253
8c3c47362934 wireproto: implement basic frame reading and processing Gregory Szorc <gregory.szorc@gmail.com> parents: 37051 diff changeset	254 The header is expected to be in the buffer at offset 0 and the
8c3c47362934 wireproto: implement basic frame reading and processing Gregory Szorc <gregory.szorc@gmail.com> parents: 37051 diff changeset	255 buffer is expected to be large enough to hold a full header.
8c3c47362934 wireproto: implement basic frame reading and processing Gregory Szorc <gregory.szorc@gmail.com> parents: 37051 diff changeset	256 """
8c3c47362934 wireproto: implement basic frame reading and processing Gregory Szorc <gregory.szorc@gmail.com> parents: 37051 diff changeset	257 # 24 bits payload length (little endian)
37288 9bfcbe4f4745 wireproto: add streams to frame-based protocol Gregory Szorc <gregory.szorc@gmail.com> parents: 37285 diff changeset	258 # 16 bits request ID
9bfcbe4f4745 wireproto: add streams to frame-based protocol Gregory Szorc <gregory.szorc@gmail.com> parents: 37285 diff changeset	259 # 8 bits stream ID
9bfcbe4f4745 wireproto: add streams to frame-based protocol Gregory Szorc <gregory.szorc@gmail.com> parents: 37285 diff changeset	260 # 8 bits stream flags
37052 8c3c47362934 wireproto: implement basic frame reading and processing Gregory Szorc <gregory.szorc@gmail.com> parents: 37051 diff changeset	261 # 4 bits frame type
8c3c47362934 wireproto: implement basic frame reading and processing Gregory Szorc <gregory.szorc@gmail.com> parents: 37051 diff changeset	262 # 4 bits frame flags
8c3c47362934 wireproto: implement basic frame reading and processing Gregory Szorc <gregory.szorc@gmail.com> parents: 37051 diff changeset	263 # ... payload
8c3c47362934 wireproto: implement basic frame reading and processing Gregory Szorc <gregory.szorc@gmail.com> parents: 37051 diff changeset	264 framelength = data[0] + 256 * data[1] + 16384 * data[2]
37288 9bfcbe4f4745 wireproto: add streams to frame-based protocol Gregory Szorc <gregory.szorc@gmail.com> parents: 37285 diff changeset	265 requestid, streamid, streamflags = struct.unpack_from(r'<HBB', data, 3)
9bfcbe4f4745 wireproto: add streams to frame-based protocol Gregory Szorc <gregory.szorc@gmail.com> parents: 37285 diff changeset	266 typeflags = data[7]
37052 8c3c47362934 wireproto: implement basic frame reading and processing Gregory Szorc <gregory.szorc@gmail.com> parents: 37051 diff changeset	267
8c3c47362934 wireproto: implement basic frame reading and processing Gregory Szorc <gregory.szorc@gmail.com> parents: 37051 diff changeset	268 frametype = (typeflags & 0xf0) >> 4
8c3c47362934 wireproto: implement basic frame reading and processing Gregory Szorc <gregory.szorc@gmail.com> parents: 37051 diff changeset	269 frameflags = typeflags & 0x0f
8c3c47362934 wireproto: implement basic frame reading and processing Gregory Szorc <gregory.szorc@gmail.com> parents: 37051 diff changeset	270
37288 9bfcbe4f4745 wireproto: add streams to frame-based protocol Gregory Szorc <gregory.szorc@gmail.com> parents: 37285 diff changeset	271 return frameheader(framelength, requestid, streamid, streamflags,
9bfcbe4f4745 wireproto: add streams to frame-based protocol Gregory Szorc <gregory.szorc@gmail.com> parents: 37285 diff changeset	272 frametype, frameflags)
37052 8c3c47362934 wireproto: implement basic frame reading and processing Gregory Szorc <gregory.szorc@gmail.com> parents: 37051 diff changeset	273
8c3c47362934 wireproto: implement basic frame reading and processing Gregory Szorc <gregory.szorc@gmail.com> parents: 37051 diff changeset	274 def readframe(fh):
8c3c47362934 wireproto: implement basic frame reading and processing Gregory Szorc <gregory.szorc@gmail.com> parents: 37051 diff changeset	275 """Read a unified framing protocol frame from a file object.
8c3c47362934 wireproto: implement basic frame reading and processing Gregory Szorc <gregory.szorc@gmail.com> parents: 37051 diff changeset	276
8c3c47362934 wireproto: implement basic frame reading and processing Gregory Szorc <gregory.szorc@gmail.com> parents: 37051 diff changeset	277 Returns a 3-tuple of (type, flags, payload) for the decoded frame or
8c3c47362934 wireproto: implement basic frame reading and processing Gregory Szorc <gregory.szorc@gmail.com> parents: 37051 diff changeset	278 None if no frame is available. May raise if a malformed frame is
8c3c47362934 wireproto: implement basic frame reading and processing Gregory Szorc <gregory.szorc@gmail.com> parents: 37051 diff changeset	279 seen.
8c3c47362934 wireproto: implement basic frame reading and processing Gregory Szorc <gregory.szorc@gmail.com> parents: 37051 diff changeset	280 """
8c3c47362934 wireproto: implement basic frame reading and processing Gregory Szorc <gregory.szorc@gmail.com> parents: 37051 diff changeset	281 header = bytearray(FRAME_HEADER_SIZE)
8c3c47362934 wireproto: implement basic frame reading and processing Gregory Szorc <gregory.szorc@gmail.com> parents: 37051 diff changeset	282
8c3c47362934 wireproto: implement basic frame reading and processing Gregory Szorc <gregory.szorc@gmail.com> parents: 37051 diff changeset	283 readcount = fh.readinto(header)
8c3c47362934 wireproto: implement basic frame reading and processing Gregory Szorc <gregory.szorc@gmail.com> parents: 37051 diff changeset	284
8c3c47362934 wireproto: implement basic frame reading and processing Gregory Szorc <gregory.szorc@gmail.com> parents: 37051 diff changeset	285 if readcount == 0:
8c3c47362934 wireproto: implement basic frame reading and processing Gregory Szorc <gregory.szorc@gmail.com> parents: 37051 diff changeset	286 return None
8c3c47362934 wireproto: implement basic frame reading and processing Gregory Szorc <gregory.szorc@gmail.com> parents: 37051 diff changeset	287
8c3c47362934 wireproto: implement basic frame reading and processing Gregory Szorc <gregory.szorc@gmail.com> parents: 37051 diff changeset	288 if readcount != FRAME_HEADER_SIZE:
8c3c47362934 wireproto: implement basic frame reading and processing Gregory Szorc <gregory.szorc@gmail.com> parents: 37051 diff changeset	289 raise error.Abort(_('received incomplete frame: got %d bytes: %s') %
8c3c47362934 wireproto: implement basic frame reading and processing Gregory Szorc <gregory.szorc@gmail.com> parents: 37051 diff changeset	290 (readcount, header))
8c3c47362934 wireproto: implement basic frame reading and processing Gregory Szorc <gregory.szorc@gmail.com> parents: 37051 diff changeset	291
37061 884a0c1604ad wireproto: define attr-based classes for representing frames Gregory Szorc <gregory.szorc@gmail.com> parents: 37060 diff changeset	292 h = parseheader(header)
37052 8c3c47362934 wireproto: implement basic frame reading and processing Gregory Szorc <gregory.szorc@gmail.com> parents: 37051 diff changeset	293
37061 884a0c1604ad wireproto: define attr-based classes for representing frames Gregory Szorc <gregory.szorc@gmail.com> parents: 37060 diff changeset	294 payload = fh.read(h.length)
884a0c1604ad wireproto: define attr-based classes for representing frames Gregory Szorc <gregory.szorc@gmail.com> parents: 37060 diff changeset	295 if len(payload) != h.length:
37052 8c3c47362934 wireproto: implement basic frame reading and processing Gregory Szorc <gregory.szorc@gmail.com> parents: 37051 diff changeset	296 raise error.Abort(_('frame length error: expected %d; got %d') %
37061 884a0c1604ad wireproto: define attr-based classes for representing frames Gregory Szorc <gregory.szorc@gmail.com> parents: 37060 diff changeset	297 (h.length, len(payload)))
37052 8c3c47362934 wireproto: implement basic frame reading and processing Gregory Szorc <gregory.szorc@gmail.com> parents: 37051 diff changeset	298
37288 9bfcbe4f4745 wireproto: add streams to frame-based protocol Gregory Szorc <gregory.szorc@gmail.com> parents: 37285 diff changeset	299 return frame(h.requestid, h.streamid, h.streamflags, h.typeid, h.flags,
9bfcbe4f4745 wireproto: add streams to frame-based protocol Gregory Szorc <gregory.szorc@gmail.com> parents: 37285 diff changeset	300 payload)
37052 8c3c47362934 wireproto: implement basic frame reading and processing Gregory Szorc <gregory.szorc@gmail.com> parents: 37051 diff changeset	301
37292 3d0e2cd86e05 wireproto: use CBOR for command requests Gregory Szorc <gregory.szorc@gmail.com> parents: 37291 diff changeset	302 def createcommandframes(stream, requestid, cmd, args, datafh=None,
40024 86b22a4cfab1 wireprotov2: client support for advertising redirect targets Gregory Szorc <gregory.szorc@gmail.com> parents: 40020 diff changeset	303 maxframesize=DEFAULT_MAX_FRAME_SIZE,
86b22a4cfab1 wireprotov2: client support for advertising redirect targets Gregory Szorc <gregory.szorc@gmail.com> parents: 40020 diff changeset	304 redirect=None):
37051 40206e227412 wireproto: define and implement protocol for issuing requests Gregory Szorc <gregory.szorc@gmail.com> parents: diff changeset	305 """Create frames necessary to transmit a request to run a command.
40206e227412 wireproto: define and implement protocol for issuing requests Gregory Szorc <gregory.szorc@gmail.com> parents: diff changeset	306
40206e227412 wireproto: define and implement protocol for issuing requests Gregory Szorc <gregory.szorc@gmail.com> parents: diff changeset	307 This is a generator of bytearrays. Each item represents a frame
40206e227412 wireproto: define and implement protocol for issuing requests Gregory Szorc <gregory.szorc@gmail.com> parents: diff changeset	308 ready to be sent over the wire to a peer.
40206e227412 wireproto: define and implement protocol for issuing requests Gregory Szorc <gregory.szorc@gmail.com> parents: diff changeset	309 """
37292 3d0e2cd86e05 wireproto: use CBOR for command requests Gregory Szorc <gregory.szorc@gmail.com> parents: 37291 diff changeset	310 data = {b'name': cmd}
37051 40206e227412 wireproto: define and implement protocol for issuing requests Gregory Szorc <gregory.szorc@gmail.com> parents: diff changeset	311 if args:
37292 3d0e2cd86e05 wireproto: use CBOR for command requests Gregory Szorc <gregory.szorc@gmail.com> parents: 37291 diff changeset	312 data[b'args'] = args
37051 40206e227412 wireproto: define and implement protocol for issuing requests Gregory Szorc <gregory.szorc@gmail.com> parents: diff changeset	313
40024 86b22a4cfab1 wireprotov2: client support for advertising redirect targets Gregory Szorc <gregory.szorc@gmail.com> parents: 40020 diff changeset	314 if redirect:
86b22a4cfab1 wireprotov2: client support for advertising redirect targets Gregory Szorc <gregory.szorc@gmail.com> parents: 40020 diff changeset	315 data[b'redirect'] = redirect
86b22a4cfab1 wireprotov2: client support for advertising redirect targets Gregory Szorc <gregory.szorc@gmail.com> parents: 40020 diff changeset	316
39446 36f487a332ad wireprotoframing: use our CBOR module Gregory Szorc <gregory.szorc@gmail.com> parents: 37728 diff changeset	317 data = b''.join(cborutil.streamencode(data))
37051 40206e227412 wireproto: define and implement protocol for issuing requests Gregory Szorc <gregory.szorc@gmail.com> parents: diff changeset	318
37292 3d0e2cd86e05 wireproto: use CBOR for command requests Gregory Szorc <gregory.szorc@gmail.com> parents: 37291 diff changeset	319 offset = 0
3d0e2cd86e05 wireproto: use CBOR for command requests Gregory Szorc <gregory.szorc@gmail.com> parents: 37291 diff changeset	320
3d0e2cd86e05 wireproto: use CBOR for command requests Gregory Szorc <gregory.szorc@gmail.com> parents: 37291 diff changeset	321 while True:
3d0e2cd86e05 wireproto: use CBOR for command requests Gregory Szorc <gregory.szorc@gmail.com> parents: 37291 diff changeset	322 flags = 0
37051 40206e227412 wireproto: define and implement protocol for issuing requests Gregory Szorc <gregory.szorc@gmail.com> parents: diff changeset	323
37292 3d0e2cd86e05 wireproto: use CBOR for command requests Gregory Szorc <gregory.szorc@gmail.com> parents: 37291 diff changeset	324 # Must set new or continuation flag.
3d0e2cd86e05 wireproto: use CBOR for command requests Gregory Szorc <gregory.szorc@gmail.com> parents: 37291 diff changeset	325 if not offset:
3d0e2cd86e05 wireproto: use CBOR for command requests Gregory Szorc <gregory.szorc@gmail.com> parents: 37291 diff changeset	326 flags \|= FLAG_COMMAND_REQUEST_NEW
3d0e2cd86e05 wireproto: use CBOR for command requests Gregory Szorc <gregory.szorc@gmail.com> parents: 37291 diff changeset	327 else:
3d0e2cd86e05 wireproto: use CBOR for command requests Gregory Szorc <gregory.szorc@gmail.com> parents: 37291 diff changeset	328 flags \|= FLAG_COMMAND_REQUEST_CONTINUATION
37051 40206e227412 wireproto: define and implement protocol for issuing requests Gregory Szorc <gregory.szorc@gmail.com> parents: diff changeset	329
37292 3d0e2cd86e05 wireproto: use CBOR for command requests Gregory Szorc <gregory.szorc@gmail.com> parents: 37291 diff changeset	330 # Data frames is set on all frames.
3d0e2cd86e05 wireproto: use CBOR for command requests Gregory Szorc <gregory.szorc@gmail.com> parents: 37291 diff changeset	331 if datafh:
3d0e2cd86e05 wireproto: use CBOR for command requests Gregory Szorc <gregory.szorc@gmail.com> parents: 37291 diff changeset	332 flags \|= FLAG_COMMAND_REQUEST_EXPECT_DATA
37051 40206e227412 wireproto: define and implement protocol for issuing requests Gregory Szorc <gregory.szorc@gmail.com> parents: diff changeset	333
37292 3d0e2cd86e05 wireproto: use CBOR for command requests Gregory Szorc <gregory.szorc@gmail.com> parents: 37291 diff changeset	334 payload = data[offset:offset + maxframesize]
3d0e2cd86e05 wireproto: use CBOR for command requests Gregory Szorc <gregory.szorc@gmail.com> parents: 37291 diff changeset	335 offset += len(payload)
3d0e2cd86e05 wireproto: use CBOR for command requests Gregory Szorc <gregory.szorc@gmail.com> parents: 37291 diff changeset	336
3d0e2cd86e05 wireproto: use CBOR for command requests Gregory Szorc <gregory.szorc@gmail.com> parents: 37291 diff changeset	337 if len(payload) == maxframesize and offset < len(data):
3d0e2cd86e05 wireproto: use CBOR for command requests Gregory Szorc <gregory.szorc@gmail.com> parents: 37291 diff changeset	338 flags \|= FLAG_COMMAND_REQUEST_MORE_FRAMES
3d0e2cd86e05 wireproto: use CBOR for command requests Gregory Szorc <gregory.szorc@gmail.com> parents: 37291 diff changeset	339
37285 3ed344546d9e wireproto: start to associate frame generation with a stream Gregory Szorc <gregory.szorc@gmail.com> parents: 37084 diff changeset	340 yield stream.makeframe(requestid=requestid,
37292 3d0e2cd86e05 wireproto: use CBOR for command requests Gregory Szorc <gregory.szorc@gmail.com> parents: 37291 diff changeset	341 typeid=FRAME_TYPE_COMMAND_REQUEST,
37285 3ed344546d9e wireproto: start to associate frame generation with a stream Gregory Szorc <gregory.szorc@gmail.com> parents: 37084 diff changeset	342 flags=flags,
3ed344546d9e wireproto: start to associate frame generation with a stream Gregory Szorc <gregory.szorc@gmail.com> parents: 37084 diff changeset	343 payload=payload)
37051 40206e227412 wireproto: define and implement protocol for issuing requests Gregory Szorc <gregory.szorc@gmail.com> parents: diff changeset	344
37292 3d0e2cd86e05 wireproto: use CBOR for command requests Gregory Szorc <gregory.szorc@gmail.com> parents: 37291 diff changeset	345 if not (flags & FLAG_COMMAND_REQUEST_MORE_FRAMES):
3d0e2cd86e05 wireproto: use CBOR for command requests Gregory Szorc <gregory.szorc@gmail.com> parents: 37291 diff changeset	346 break
3d0e2cd86e05 wireproto: use CBOR for command requests Gregory Szorc <gregory.szorc@gmail.com> parents: 37291 diff changeset	347
37051 40206e227412 wireproto: define and implement protocol for issuing requests Gregory Szorc <gregory.szorc@gmail.com> parents: diff changeset	348 if datafh:
40206e227412 wireproto: define and implement protocol for issuing requests Gregory Szorc <gregory.szorc@gmail.com> parents: diff changeset	349 while True:
40206e227412 wireproto: define and implement protocol for issuing requests Gregory Szorc <gregory.szorc@gmail.com> parents: diff changeset	350 data = datafh.read(DEFAULT_MAX_FRAME_SIZE)
40206e227412 wireproto: define and implement protocol for issuing requests Gregory Szorc <gregory.szorc@gmail.com> parents: diff changeset	351
40206e227412 wireproto: define and implement protocol for issuing requests Gregory Szorc <gregory.szorc@gmail.com> parents: diff changeset	352 done = False
40206e227412 wireproto: define and implement protocol for issuing requests Gregory Szorc <gregory.szorc@gmail.com> parents: diff changeset	353 if len(data) == DEFAULT_MAX_FRAME_SIZE:
40206e227412 wireproto: define and implement protocol for issuing requests Gregory Szorc <gregory.szorc@gmail.com> parents: diff changeset	354 flags = FLAG_COMMAND_DATA_CONTINUATION
40206e227412 wireproto: define and implement protocol for issuing requests Gregory Szorc <gregory.szorc@gmail.com> parents: diff changeset	355 else:
40206e227412 wireproto: define and implement protocol for issuing requests Gregory Szorc <gregory.szorc@gmail.com> parents: diff changeset	356 flags = FLAG_COMMAND_DATA_EOS
40206e227412 wireproto: define and implement protocol for issuing requests Gregory Szorc <gregory.szorc@gmail.com> parents: diff changeset	357 assert datafh.read(1) == b''
40206e227412 wireproto: define and implement protocol for issuing requests Gregory Szorc <gregory.szorc@gmail.com> parents: diff changeset	358 done = True
40206e227412 wireproto: define and implement protocol for issuing requests Gregory Szorc <gregory.szorc@gmail.com> parents: diff changeset	359
37285 3ed344546d9e wireproto: start to associate frame generation with a stream Gregory Szorc <gregory.szorc@gmail.com> parents: 37084 diff changeset	360 yield stream.makeframe(requestid=requestid,
3ed344546d9e wireproto: start to associate frame generation with a stream Gregory Szorc <gregory.szorc@gmail.com> parents: 37084 diff changeset	361 typeid=FRAME_TYPE_COMMAND_DATA,
3ed344546d9e wireproto: start to associate frame generation with a stream Gregory Szorc <gregory.szorc@gmail.com> parents: 37084 diff changeset	362 flags=flags,
3ed344546d9e wireproto: start to associate frame generation with a stream Gregory Szorc <gregory.szorc@gmail.com> parents: 37084 diff changeset	363 payload=data)
37051 40206e227412 wireproto: define and implement protocol for issuing requests Gregory Szorc <gregory.szorc@gmail.com> parents: diff changeset	364
40206e227412 wireproto: define and implement protocol for issuing requests Gregory Szorc <gregory.szorc@gmail.com> parents: diff changeset	365 if done:
40206e227412 wireproto: define and implement protocol for issuing requests Gregory Szorc <gregory.szorc@gmail.com> parents: diff changeset	366 break
37052 8c3c47362934 wireproto: implement basic frame reading and processing Gregory Szorc <gregory.szorc@gmail.com> parents: 37051 diff changeset	367
37724 deff7cf7eefd wireprotov2: change frame type and name for command response Gregory Szorc <gregory.szorc@gmail.com> parents: 37723 diff changeset	368 def createcommandresponseframesfrombytes(stream, requestid, data,
deff7cf7eefd wireprotov2: change frame type and name for command response Gregory Szorc <gregory.szorc@gmail.com> parents: 37723 diff changeset	369 maxframesize=DEFAULT_MAX_FRAME_SIZE):
37055 61393f888dfe wireproto: define and implement responses in framing protocol Gregory Szorc <gregory.szorc@gmail.com> parents: 37052 diff changeset	370 """Create a raw frame to send a bytes response from static bytes input.
61393f888dfe wireproto: define and implement responses in framing protocol Gregory Szorc <gregory.szorc@gmail.com> parents: 37052 diff changeset	371
61393f888dfe wireproto: define and implement responses in framing protocol Gregory Szorc <gregory.szorc@gmail.com> parents: 37052 diff changeset	372 Returns a generator of bytearrays.
61393f888dfe wireproto: define and implement responses in framing protocol Gregory Szorc <gregory.szorc@gmail.com> parents: 37052 diff changeset	373 """
37725 3ea8323d6f95 wireprotov2: change command response protocol to include a leading map Gregory Szorc <gregory.szorc@gmail.com> parents: 37724 diff changeset	374 # Automatically send the overall CBOR response map.
39446 36f487a332ad wireprotoframing: use our CBOR module Gregory Szorc <gregory.szorc@gmail.com> parents: 37728 diff changeset	375 overall = b''.join(cborutil.streamencode({b'status': b'ok'}))
37725 3ea8323d6f95 wireprotov2: change command response protocol to include a leading map Gregory Szorc <gregory.szorc@gmail.com> parents: 37724 diff changeset	376 if len(overall) > maxframesize:
3ea8323d6f95 wireprotov2: change command response protocol to include a leading map Gregory Szorc <gregory.szorc@gmail.com> parents: 37724 diff changeset	377 raise error.ProgrammingError('not yet implemented')
37055 61393f888dfe wireproto: define and implement responses in framing protocol Gregory Szorc <gregory.szorc@gmail.com> parents: 37052 diff changeset	378
37725 3ea8323d6f95 wireprotov2: change command response protocol to include a leading map Gregory Szorc <gregory.szorc@gmail.com> parents: 37724 diff changeset	379 # Simple case where we can fit the full response in a single frame.
3ea8323d6f95 wireprotov2: change command response protocol to include a leading map Gregory Szorc <gregory.szorc@gmail.com> parents: 37724 diff changeset	380 if len(overall) + len(data) <= maxframesize:
37724 deff7cf7eefd wireprotov2: change frame type and name for command response Gregory Szorc <gregory.szorc@gmail.com> parents: 37723 diff changeset	381 flags = FLAG_COMMAND_RESPONSE_EOS
37285 3ed344546d9e wireproto: start to associate frame generation with a stream Gregory Szorc <gregory.szorc@gmail.com> parents: 37084 diff changeset	382 yield stream.makeframe(requestid=requestid,
37724 deff7cf7eefd wireprotov2: change frame type and name for command response Gregory Szorc <gregory.szorc@gmail.com> parents: 37723 diff changeset	383 typeid=FRAME_TYPE_COMMAND_RESPONSE,
37485 0b7475ea38cf wireproto: port heads command to wire protocol v2 Gregory Szorc <gregory.szorc@gmail.com> parents: 37476 diff changeset	384 flags=flags,
37725 3ea8323d6f95 wireprotov2: change command response protocol to include a leading map Gregory Szorc <gregory.szorc@gmail.com> parents: 37724 diff changeset	385 payload=overall + data)
37055 61393f888dfe wireproto: define and implement responses in framing protocol Gregory Szorc <gregory.szorc@gmail.com> parents: 37052 diff changeset	386 return
61393f888dfe wireproto: define and implement responses in framing protocol Gregory Szorc <gregory.szorc@gmail.com> parents: 37052 diff changeset	387
37725 3ea8323d6f95 wireprotov2: change command response protocol to include a leading map Gregory Szorc <gregory.szorc@gmail.com> parents: 37724 diff changeset	388 # It's easier to send the overall CBOR map in its own frame than to track
3ea8323d6f95 wireprotov2: change command response protocol to include a leading map Gregory Szorc <gregory.szorc@gmail.com> parents: 37724 diff changeset	389 # offsets.
3ea8323d6f95 wireprotov2: change command response protocol to include a leading map Gregory Szorc <gregory.szorc@gmail.com> parents: 37724 diff changeset	390 yield stream.makeframe(requestid=requestid,
3ea8323d6f95 wireprotov2: change command response protocol to include a leading map Gregory Szorc <gregory.szorc@gmail.com> parents: 37724 diff changeset	391 typeid=FRAME_TYPE_COMMAND_RESPONSE,
3ea8323d6f95 wireprotov2: change command response protocol to include a leading map Gregory Szorc <gregory.szorc@gmail.com> parents: 37724 diff changeset	392 flags=FLAG_COMMAND_RESPONSE_CONTINUATION,
3ea8323d6f95 wireprotov2: change command response protocol to include a leading map Gregory Szorc <gregory.szorc@gmail.com> parents: 37724 diff changeset	393 payload=overall)
3ea8323d6f95 wireprotov2: change command response protocol to include a leading map Gregory Szorc <gregory.szorc@gmail.com> parents: 37724 diff changeset	394
37055 61393f888dfe wireproto: define and implement responses in framing protocol Gregory Szorc <gregory.szorc@gmail.com> parents: 37052 diff changeset	395 offset = 0
61393f888dfe wireproto: define and implement responses in framing protocol Gregory Szorc <gregory.szorc@gmail.com> parents: 37052 diff changeset	396 while True:
61393f888dfe wireproto: define and implement responses in framing protocol Gregory Szorc <gregory.szorc@gmail.com> parents: 37052 diff changeset	397 chunk = data[offset:offset + maxframesize]
61393f888dfe wireproto: define and implement responses in framing protocol Gregory Szorc <gregory.szorc@gmail.com> parents: 37052 diff changeset	398 offset += len(chunk)
61393f888dfe wireproto: define and implement responses in framing protocol Gregory Szorc <gregory.szorc@gmail.com> parents: 37052 diff changeset	399 done = offset == len(data)
61393f888dfe wireproto: define and implement responses in framing protocol Gregory Szorc <gregory.szorc@gmail.com> parents: 37052 diff changeset	400
61393f888dfe wireproto: define and implement responses in framing protocol Gregory Szorc <gregory.szorc@gmail.com> parents: 37052 diff changeset	401 if done:
37724 deff7cf7eefd wireprotov2: change frame type and name for command response Gregory Szorc <gregory.szorc@gmail.com> parents: 37723 diff changeset	402 flags = FLAG_COMMAND_RESPONSE_EOS
37055 61393f888dfe wireproto: define and implement responses in framing protocol Gregory Szorc <gregory.szorc@gmail.com> parents: 37052 diff changeset	403 else:
37724 deff7cf7eefd wireprotov2: change frame type and name for command response Gregory Szorc <gregory.szorc@gmail.com> parents: 37723 diff changeset	404 flags = FLAG_COMMAND_RESPONSE_CONTINUATION
37055 61393f888dfe wireproto: define and implement responses in framing protocol Gregory Szorc <gregory.szorc@gmail.com> parents: 37052 diff changeset	405
37285 3ed344546d9e wireproto: start to associate frame generation with a stream Gregory Szorc <gregory.szorc@gmail.com> parents: 37084 diff changeset	406 yield stream.makeframe(requestid=requestid,
37724 deff7cf7eefd wireprotov2: change frame type and name for command response Gregory Szorc <gregory.szorc@gmail.com> parents: 37723 diff changeset	407 typeid=FRAME_TYPE_COMMAND_RESPONSE,
37285 3ed344546d9e wireproto: start to associate frame generation with a stream Gregory Szorc <gregory.szorc@gmail.com> parents: 37084 diff changeset	408 flags=flags,
3ed344546d9e wireproto: start to associate frame generation with a stream Gregory Szorc <gregory.szorc@gmail.com> parents: 37084 diff changeset	409 payload=chunk)
37055 61393f888dfe wireproto: define and implement responses in framing protocol Gregory Szorc <gregory.szorc@gmail.com> parents: 37052 diff changeset	410
61393f888dfe wireproto: define and implement responses in framing protocol Gregory Szorc <gregory.szorc@gmail.com> parents: 37052 diff changeset	411 if done:
61393f888dfe wireproto: define and implement responses in framing protocol Gregory Szorc <gregory.szorc@gmail.com> parents: 37052 diff changeset	412 break
61393f888dfe wireproto: define and implement responses in framing protocol Gregory Szorc <gregory.szorc@gmail.com> parents: 37052 diff changeset	413
37728 564a3eec6e63 wireprotov2: add support for more response types Gregory Szorc <gregory.szorc@gmail.com> parents: 37726 diff changeset	414 def createbytesresponseframesfromgen(stream, requestid, gen,
564a3eec6e63 wireprotov2: add support for more response types Gregory Szorc <gregory.szorc@gmail.com> parents: 37726 diff changeset	415 maxframesize=DEFAULT_MAX_FRAME_SIZE):
39559 07b58266bce3 wireprotov2: implement commands as a generator of objects Gregory Szorc <gregory.szorc@gmail.com> parents: 39486 diff changeset	416 """Generator of frames from a generator of byte chunks.
37728 564a3eec6e63 wireprotov2: add support for more response types Gregory Szorc <gregory.szorc@gmail.com> parents: 37726 diff changeset	417
39559 07b58266bce3 wireprotov2: implement commands as a generator of objects Gregory Szorc <gregory.szorc@gmail.com> parents: 39486 diff changeset	418 This assumes that another frame will follow whatever this emits. i.e.
07b58266bce3 wireprotov2: implement commands as a generator of objects Gregory Szorc <gregory.szorc@gmail.com> parents: 39486 diff changeset	419 this always emits the continuation flag and never emits the end-of-stream
07b58266bce3 wireprotov2: implement commands as a generator of objects Gregory Szorc <gregory.szorc@gmail.com> parents: 39486 diff changeset	420 flag.
07b58266bce3 wireprotov2: implement commands as a generator of objects Gregory Szorc <gregory.szorc@gmail.com> parents: 39486 diff changeset	421 """
37728 564a3eec6e63 wireprotov2: add support for more response types Gregory Szorc <gregory.szorc@gmail.com> parents: 37726 diff changeset	422 cb = util.chunkbuffer(gen)
39559 07b58266bce3 wireprotov2: implement commands as a generator of objects Gregory Szorc <gregory.szorc@gmail.com> parents: 39486 diff changeset	423 flags = FLAG_COMMAND_RESPONSE_CONTINUATION
37728 564a3eec6e63 wireprotov2: add support for more response types Gregory Szorc <gregory.szorc@gmail.com> parents: 37726 diff changeset	424
564a3eec6e63 wireprotov2: add support for more response types Gregory Szorc <gregory.szorc@gmail.com> parents: 37726 diff changeset	425 while True:
564a3eec6e63 wireprotov2: add support for more response types Gregory Szorc <gregory.szorc@gmail.com> parents: 37726 diff changeset	426 chunk = cb.read(maxframesize)
564a3eec6e63 wireprotov2: add support for more response types Gregory Szorc <gregory.szorc@gmail.com> parents: 37726 diff changeset	427 if not chunk:
564a3eec6e63 wireprotov2: add support for more response types Gregory Szorc <gregory.szorc@gmail.com> parents: 37726 diff changeset	428 break
564a3eec6e63 wireprotov2: add support for more response types Gregory Szorc <gregory.szorc@gmail.com> parents: 37726 diff changeset	429
564a3eec6e63 wireprotov2: add support for more response types Gregory Szorc <gregory.szorc@gmail.com> parents: 37726 diff changeset	430 yield stream.makeframe(requestid=requestid,
564a3eec6e63 wireprotov2: add support for more response types Gregory Szorc <gregory.szorc@gmail.com> parents: 37726 diff changeset	431 typeid=FRAME_TYPE_COMMAND_RESPONSE,
564a3eec6e63 wireprotov2: add support for more response types Gregory Szorc <gregory.szorc@gmail.com> parents: 37726 diff changeset	432 flags=flags,
564a3eec6e63 wireprotov2: add support for more response types Gregory Szorc <gregory.szorc@gmail.com> parents: 37726 diff changeset	433 payload=chunk)
564a3eec6e63 wireprotov2: add support for more response types Gregory Szorc <gregory.szorc@gmail.com> parents: 37726 diff changeset	434
564a3eec6e63 wireprotov2: add support for more response types Gregory Szorc <gregory.szorc@gmail.com> parents: 37726 diff changeset	435 flags \|= FLAG_COMMAND_RESPONSE_CONTINUATION
564a3eec6e63 wireprotov2: add support for more response types Gregory Szorc <gregory.szorc@gmail.com> parents: 37726 diff changeset	436
39559 07b58266bce3 wireprotov2: implement commands as a generator of objects Gregory Szorc <gregory.szorc@gmail.com> parents: 39486 diff changeset	437 def createcommandresponseokframe(stream, requestid):
07b58266bce3 wireprotov2: implement commands as a generator of objects Gregory Szorc <gregory.szorc@gmail.com> parents: 39486 diff changeset	438 overall = b''.join(cborutil.streamencode({b'status': b'ok'}))
07b58266bce3 wireprotov2: implement commands as a generator of objects Gregory Szorc <gregory.szorc@gmail.com> parents: 39486 diff changeset	439
07b58266bce3 wireprotov2: implement commands as a generator of objects Gregory Szorc <gregory.szorc@gmail.com> parents: 39486 diff changeset	440 return stream.makeframe(requestid=requestid,
07b58266bce3 wireprotov2: implement commands as a generator of objects Gregory Szorc <gregory.szorc@gmail.com> parents: 39486 diff changeset	441 typeid=FRAME_TYPE_COMMAND_RESPONSE,
07b58266bce3 wireprotov2: implement commands as a generator of objects Gregory Szorc <gregory.szorc@gmail.com> parents: 39486 diff changeset	442 flags=FLAG_COMMAND_RESPONSE_CONTINUATION,
07b58266bce3 wireprotov2: implement commands as a generator of objects Gregory Szorc <gregory.szorc@gmail.com> parents: 39486 diff changeset	443 payload=overall)
07b58266bce3 wireprotov2: implement commands as a generator of objects Gregory Szorc <gregory.szorc@gmail.com> parents: 39486 diff changeset	444
07b58266bce3 wireprotov2: implement commands as a generator of objects Gregory Szorc <gregory.szorc@gmail.com> parents: 39486 diff changeset	445 def createcommandresponseeosframe(stream, requestid):
07b58266bce3 wireprotov2: implement commands as a generator of objects Gregory Szorc <gregory.szorc@gmail.com> parents: 39486 diff changeset	446 """Create an empty payload frame representing command end-of-stream."""
07b58266bce3 wireprotov2: implement commands as a generator of objects Gregory Szorc <gregory.szorc@gmail.com> parents: 39486 diff changeset	447 return stream.makeframe(requestid=requestid,
07b58266bce3 wireprotov2: implement commands as a generator of objects Gregory Szorc <gregory.szorc@gmail.com> parents: 39486 diff changeset	448 typeid=FRAME_TYPE_COMMAND_RESPONSE,
07b58266bce3 wireprotov2: implement commands as a generator of objects Gregory Szorc <gregory.szorc@gmail.com> parents: 39486 diff changeset	449 flags=FLAG_COMMAND_RESPONSE_EOS,
07b58266bce3 wireprotov2: implement commands as a generator of objects Gregory Szorc <gregory.szorc@gmail.com> parents: 39486 diff changeset	450 payload=b'')
37728 564a3eec6e63 wireprotov2: add support for more response types Gregory Szorc <gregory.szorc@gmail.com> parents: 37726 diff changeset	451
40025 b099e6032f38 wireprotov2: server support for sending content redirects Gregory Szorc <gregory.szorc@gmail.com> parents: 40024 diff changeset	452 def createalternatelocationresponseframe(stream, requestid, location):
b099e6032f38 wireprotov2: server support for sending content redirects Gregory Szorc <gregory.szorc@gmail.com> parents: 40024 diff changeset	453 data = {
b099e6032f38 wireprotov2: server support for sending content redirects Gregory Szorc <gregory.szorc@gmail.com> parents: 40024 diff changeset	454 b'status': b'redirect',
b099e6032f38 wireprotov2: server support for sending content redirects Gregory Szorc <gregory.szorc@gmail.com> parents: 40024 diff changeset	455 b'location': {
b099e6032f38 wireprotov2: server support for sending content redirects Gregory Szorc <gregory.szorc@gmail.com> parents: 40024 diff changeset	456 b'url': location.url,
b099e6032f38 wireprotov2: server support for sending content redirects Gregory Szorc <gregory.szorc@gmail.com> parents: 40024 diff changeset	457 b'mediatype': location.mediatype,
b099e6032f38 wireprotov2: server support for sending content redirects Gregory Szorc <gregory.szorc@gmail.com> parents: 40024 diff changeset	458 }
b099e6032f38 wireprotov2: server support for sending content redirects Gregory Szorc <gregory.szorc@gmail.com> parents: 40024 diff changeset	459 }
b099e6032f38 wireprotov2: server support for sending content redirects Gregory Szorc <gregory.szorc@gmail.com> parents: 40024 diff changeset	460
b099e6032f38 wireprotov2: server support for sending content redirects Gregory Szorc <gregory.szorc@gmail.com> parents: 40024 diff changeset	461 for a in (r'size', r'fullhashes', r'fullhashseed', r'serverdercerts',
b099e6032f38 wireprotov2: server support for sending content redirects Gregory Szorc <gregory.szorc@gmail.com> parents: 40024 diff changeset	462 r'servercadercerts'):
b099e6032f38 wireprotov2: server support for sending content redirects Gregory Szorc <gregory.szorc@gmail.com> parents: 40024 diff changeset	463 value = getattr(location, a)
b099e6032f38 wireprotov2: server support for sending content redirects Gregory Szorc <gregory.szorc@gmail.com> parents: 40024 diff changeset	464 if value is not None:
b099e6032f38 wireprotov2: server support for sending content redirects Gregory Szorc <gregory.szorc@gmail.com> parents: 40024 diff changeset	465 data[b'location'][pycompat.bytestr(a)] = value
b099e6032f38 wireprotov2: server support for sending content redirects Gregory Szorc <gregory.szorc@gmail.com> parents: 40024 diff changeset	466
b099e6032f38 wireprotov2: server support for sending content redirects Gregory Szorc <gregory.szorc@gmail.com> parents: 40024 diff changeset	467 return stream.makeframe(requestid=requestid,
b099e6032f38 wireprotov2: server support for sending content redirects Gregory Szorc <gregory.szorc@gmail.com> parents: 40024 diff changeset	468 typeid=FRAME_TYPE_COMMAND_RESPONSE,
b099e6032f38 wireprotov2: server support for sending content redirects Gregory Szorc <gregory.szorc@gmail.com> parents: 40024 diff changeset	469 flags=FLAG_COMMAND_RESPONSE_CONTINUATION,
b099e6032f38 wireprotov2: server support for sending content redirects Gregory Szorc <gregory.szorc@gmail.com> parents: 40024 diff changeset	470 payload=b''.join(cborutil.streamencode(data)))
b099e6032f38 wireprotov2: server support for sending content redirects Gregory Szorc <gregory.szorc@gmail.com> parents: 40024 diff changeset	471
37728 564a3eec6e63 wireprotov2: add support for more response types Gregory Szorc <gregory.szorc@gmail.com> parents: 37726 diff changeset	472 def createcommanderrorresponse(stream, requestid, message, args=None):
39486 43d92d68ac88 wireprotov2peer: properly format errors Gregory Szorc <gregory.szorc@gmail.com> parents: 39446 diff changeset	473 # TODO should this be using a list of {'msg': ..., 'args': {}} so atom
43d92d68ac88 wireprotov2peer: properly format errors Gregory Szorc <gregory.szorc@gmail.com> parents: 39446 diff changeset	474 # formatting works consistently?
37728 564a3eec6e63 wireprotov2: add support for more response types Gregory Szorc <gregory.szorc@gmail.com> parents: 37726 diff changeset	475 m = {
564a3eec6e63 wireprotov2: add support for more response types Gregory Szorc <gregory.szorc@gmail.com> parents: 37726 diff changeset	476 b'status': b'error',
564a3eec6e63 wireprotov2: add support for more response types Gregory Szorc <gregory.szorc@gmail.com> parents: 37726 diff changeset	477 b'error': {
564a3eec6e63 wireprotov2: add support for more response types Gregory Szorc <gregory.szorc@gmail.com> parents: 37726 diff changeset	478 b'message': message,
564a3eec6e63 wireprotov2: add support for more response types Gregory Szorc <gregory.szorc@gmail.com> parents: 37726 diff changeset	479 }
564a3eec6e63 wireprotov2: add support for more response types Gregory Szorc <gregory.szorc@gmail.com> parents: 37726 diff changeset	480 }
564a3eec6e63 wireprotov2: add support for more response types Gregory Szorc <gregory.szorc@gmail.com> parents: 37726 diff changeset	481
564a3eec6e63 wireprotov2: add support for more response types Gregory Szorc <gregory.szorc@gmail.com> parents: 37726 diff changeset	482 if args:
564a3eec6e63 wireprotov2: add support for more response types Gregory Szorc <gregory.szorc@gmail.com> parents: 37726 diff changeset	483 m[b'error'][b'args'] = args
564a3eec6e63 wireprotov2: add support for more response types Gregory Szorc <gregory.szorc@gmail.com> parents: 37726 diff changeset	484
39446 36f487a332ad wireprotoframing: use our CBOR module Gregory Szorc <gregory.szorc@gmail.com> parents: 37728 diff changeset	485 overall = b''.join(cborutil.streamencode(m))
37728 564a3eec6e63 wireprotov2: add support for more response types Gregory Szorc <gregory.szorc@gmail.com> parents: 37726 diff changeset	486
564a3eec6e63 wireprotov2: add support for more response types Gregory Szorc <gregory.szorc@gmail.com> parents: 37726 diff changeset	487 yield stream.makeframe(requestid=requestid,
564a3eec6e63 wireprotov2: add support for more response types Gregory Szorc <gregory.szorc@gmail.com> parents: 37726 diff changeset	488 typeid=FRAME_TYPE_COMMAND_RESPONSE,
564a3eec6e63 wireprotov2: add support for more response types Gregory Szorc <gregory.szorc@gmail.com> parents: 37726 diff changeset	489 flags=FLAG_COMMAND_RESPONSE_EOS,
564a3eec6e63 wireprotov2: add support for more response types Gregory Szorc <gregory.szorc@gmail.com> parents: 37726 diff changeset	490 payload=overall)
564a3eec6e63 wireprotov2: add support for more response types Gregory Szorc <gregory.szorc@gmail.com> parents: 37726 diff changeset	491
37726 0c184ca594bb wireprotov2: change behavior of error frame Gregory Szorc <gregory.szorc@gmail.com> parents: 37725 diff changeset	492 def createerrorframe(stream, requestid, msg, errtype):
37055 61393f888dfe wireproto: define and implement responses in framing protocol Gregory Szorc <gregory.szorc@gmail.com> parents: 37052 diff changeset	493 # TODO properly handle frame size limits.
61393f888dfe wireproto: define and implement responses in framing protocol Gregory Szorc <gregory.szorc@gmail.com> parents: 37052 diff changeset	494 assert len(msg) <= DEFAULT_MAX_FRAME_SIZE
61393f888dfe wireproto: define and implement responses in framing protocol Gregory Szorc <gregory.szorc@gmail.com> parents: 37052 diff changeset	495
39446 36f487a332ad wireprotoframing: use our CBOR module Gregory Szorc <gregory.szorc@gmail.com> parents: 37728 diff changeset	496 payload = b''.join(cborutil.streamencode({
37726 0c184ca594bb wireprotov2: change behavior of error frame Gregory Szorc <gregory.szorc@gmail.com> parents: 37725 diff changeset	497 b'type': errtype,
0c184ca594bb wireprotov2: change behavior of error frame Gregory Szorc <gregory.szorc@gmail.com> parents: 37725 diff changeset	498 b'message': [{b'msg': msg}],
39446 36f487a332ad wireprotoframing: use our CBOR module Gregory Szorc <gregory.szorc@gmail.com> parents: 37728 diff changeset	499 }))
37055 61393f888dfe wireproto: define and implement responses in framing protocol Gregory Szorc <gregory.szorc@gmail.com> parents: 37052 diff changeset	500
37285 3ed344546d9e wireproto: start to associate frame generation with a stream Gregory Szorc <gregory.szorc@gmail.com> parents: 37084 diff changeset	501 yield stream.makeframe(requestid=requestid,
3ed344546d9e wireproto: start to associate frame generation with a stream Gregory Szorc <gregory.szorc@gmail.com> parents: 37084 diff changeset	502 typeid=FRAME_TYPE_ERROR_RESPONSE,
37726 0c184ca594bb wireprotov2: change behavior of error frame Gregory Szorc <gregory.szorc@gmail.com> parents: 37725 diff changeset	503 flags=0,
0c184ca594bb wireprotov2: change behavior of error frame Gregory Szorc <gregory.szorc@gmail.com> parents: 37725 diff changeset	504 payload=payload)
37055 61393f888dfe wireproto: define and implement responses in framing protocol Gregory Szorc <gregory.szorc@gmail.com> parents: 37052 diff changeset	505
37319 36d17f37db91 wireproto: convert human output frames to CBOR Gregory Szorc <gregory.szorc@gmail.com> parents: 37299 diff changeset	506 def createtextoutputframe(stream, requestid, atoms,
36d17f37db91 wireproto: convert human output frames to CBOR Gregory Szorc <gregory.szorc@gmail.com> parents: 37299 diff changeset	507 maxframesize=DEFAULT_MAX_FRAME_SIZE):
37060 0a6c5cc09a88 wireproto: define human output side channel frame Gregory Szorc <gregory.szorc@gmail.com> parents: 37058 diff changeset	508 """Create a text output frame to render text to people.
0a6c5cc09a88 wireproto: define human output side channel frame Gregory Szorc <gregory.szorc@gmail.com> parents: 37058 diff changeset	509
0a6c5cc09a88 wireproto: define human output side channel frame Gregory Szorc <gregory.szorc@gmail.com> parents: 37058 diff changeset	510 ``atoms`` is a 3-tuple of (formatting string, args, labels).
0a6c5cc09a88 wireproto: define human output side channel frame Gregory Szorc <gregory.szorc@gmail.com> parents: 37058 diff changeset	511
0a6c5cc09a88 wireproto: define human output side channel frame Gregory Szorc <gregory.szorc@gmail.com> parents: 37058 diff changeset	512 The formatting string contains ``%s`` tokens to be replaced by the
0a6c5cc09a88 wireproto: define human output side channel frame Gregory Szorc <gregory.szorc@gmail.com> parents: 37058 diff changeset	513 corresponding indexed entry in ``args``. ``labels`` is an iterable of
0a6c5cc09a88 wireproto: define human output side channel frame Gregory Szorc <gregory.szorc@gmail.com> parents: 37058 diff changeset	514 formatters to be applied at rendering time. In terms of the ``ui``
0a6c5cc09a88 wireproto: define human output side channel frame Gregory Szorc <gregory.szorc@gmail.com> parents: 37058 diff changeset	515 class, each atom corresponds to a ``ui.write()``.
0a6c5cc09a88 wireproto: define human output side channel frame Gregory Szorc <gregory.szorc@gmail.com> parents: 37058 diff changeset	516 """
37319 36d17f37db91 wireproto: convert human output frames to CBOR Gregory Szorc <gregory.szorc@gmail.com> parents: 37299 diff changeset	517 atomdicts = []
37060 0a6c5cc09a88 wireproto: define human output side channel frame Gregory Szorc <gregory.szorc@gmail.com> parents: 37058 diff changeset	518
0a6c5cc09a88 wireproto: define human output side channel frame Gregory Szorc <gregory.szorc@gmail.com> parents: 37058 diff changeset	519 for (formatting, args, labels) in atoms:
0a6c5cc09a88 wireproto: define human output side channel frame Gregory Szorc <gregory.szorc@gmail.com> parents: 37058 diff changeset	520 # TODO look for localstr, other types here?
0a6c5cc09a88 wireproto: define human output side channel frame Gregory Szorc <gregory.szorc@gmail.com> parents: 37058 diff changeset	521
0a6c5cc09a88 wireproto: define human output side channel frame Gregory Szorc <gregory.szorc@gmail.com> parents: 37058 diff changeset	522 if not isinstance(formatting, bytes):
0a6c5cc09a88 wireproto: define human output side channel frame Gregory Szorc <gregory.szorc@gmail.com> parents: 37058 diff changeset	523 raise ValueError('must use bytes formatting strings')
0a6c5cc09a88 wireproto: define human output side channel frame Gregory Szorc <gregory.szorc@gmail.com> parents: 37058 diff changeset	524 for arg in args:
0a6c5cc09a88 wireproto: define human output side channel frame Gregory Szorc <gregory.szorc@gmail.com> parents: 37058 diff changeset	525 if not isinstance(arg, bytes):
0a6c5cc09a88 wireproto: define human output side channel frame Gregory Szorc <gregory.szorc@gmail.com> parents: 37058 diff changeset	526 raise ValueError('must use bytes for arguments')
0a6c5cc09a88 wireproto: define human output side channel frame Gregory Szorc <gregory.szorc@gmail.com> parents: 37058 diff changeset	527 for label in labels:
0a6c5cc09a88 wireproto: define human output side channel frame Gregory Szorc <gregory.szorc@gmail.com> parents: 37058 diff changeset	528 if not isinstance(label, bytes):
0a6c5cc09a88 wireproto: define human output side channel frame Gregory Szorc <gregory.szorc@gmail.com> parents: 37058 diff changeset	529 raise ValueError('must use bytes for labels')
0a6c5cc09a88 wireproto: define human output side channel frame Gregory Szorc <gregory.szorc@gmail.com> parents: 37058 diff changeset	530
37319 36d17f37db91 wireproto: convert human output frames to CBOR Gregory Szorc <gregory.szorc@gmail.com> parents: 37299 diff changeset	531 # Formatting string must be ASCII.
36d17f37db91 wireproto: convert human output frames to CBOR Gregory Szorc <gregory.szorc@gmail.com> parents: 37299 diff changeset	532 formatting = formatting.decode(r'ascii', r'replace').encode(r'ascii')
37060 0a6c5cc09a88 wireproto: define human output side channel frame Gregory Szorc <gregory.szorc@gmail.com> parents: 37058 diff changeset	533
0a6c5cc09a88 wireproto: define human output side channel frame Gregory Szorc <gregory.szorc@gmail.com> parents: 37058 diff changeset	534 # Arguments must be UTF-8.
0a6c5cc09a88 wireproto: define human output side channel frame Gregory Szorc <gregory.szorc@gmail.com> parents: 37058 diff changeset	535 args = [a.decode(r'utf-8', r'replace').encode(r'utf-8') for a in args]
0a6c5cc09a88 wireproto: define human output side channel frame Gregory Szorc <gregory.szorc@gmail.com> parents: 37058 diff changeset	536
0a6c5cc09a88 wireproto: define human output side channel frame Gregory Szorc <gregory.szorc@gmail.com> parents: 37058 diff changeset	537 # Labels must be ASCII.
0a6c5cc09a88 wireproto: define human output side channel frame Gregory Szorc <gregory.szorc@gmail.com> parents: 37058 diff changeset	538 labels = [l.decode(r'ascii', r'strict').encode(r'ascii')
0a6c5cc09a88 wireproto: define human output side channel frame Gregory Szorc <gregory.szorc@gmail.com> parents: 37058 diff changeset	539 for l in labels]
0a6c5cc09a88 wireproto: define human output side channel frame Gregory Szorc <gregory.szorc@gmail.com> parents: 37058 diff changeset	540
37319 36d17f37db91 wireproto: convert human output frames to CBOR Gregory Szorc <gregory.szorc@gmail.com> parents: 37299 diff changeset	541 atom = {b'msg': formatting}
36d17f37db91 wireproto: convert human output frames to CBOR Gregory Szorc <gregory.szorc@gmail.com> parents: 37299 diff changeset	542 if args:
36d17f37db91 wireproto: convert human output frames to CBOR Gregory Szorc <gregory.szorc@gmail.com> parents: 37299 diff changeset	543 atom[b'args'] = args
36d17f37db91 wireproto: convert human output frames to CBOR Gregory Szorc <gregory.szorc@gmail.com> parents: 37299 diff changeset	544 if labels:
36d17f37db91 wireproto: convert human output frames to CBOR Gregory Szorc <gregory.szorc@gmail.com> parents: 37299 diff changeset	545 atom[b'labels'] = labels
37060 0a6c5cc09a88 wireproto: define human output side channel frame Gregory Szorc <gregory.szorc@gmail.com> parents: 37058 diff changeset	546
37319 36d17f37db91 wireproto: convert human output frames to CBOR Gregory Szorc <gregory.szorc@gmail.com> parents: 37299 diff changeset	547 atomdicts.append(atom)
37060 0a6c5cc09a88 wireproto: define human output side channel frame Gregory Szorc <gregory.szorc@gmail.com> parents: 37058 diff changeset	548
39446 36f487a332ad wireprotoframing: use our CBOR module Gregory Szorc <gregory.szorc@gmail.com> parents: 37728 diff changeset	549 payload = b''.join(cborutil.streamencode(atomdicts))
37060 0a6c5cc09a88 wireproto: define human output side channel frame Gregory Szorc <gregory.szorc@gmail.com> parents: 37058 diff changeset	550
37319 36d17f37db91 wireproto: convert human output frames to CBOR Gregory Szorc <gregory.szorc@gmail.com> parents: 37299 diff changeset	551 if len(payload) > maxframesize:
37060 0a6c5cc09a88 wireproto: define human output side channel frame Gregory Szorc <gregory.szorc@gmail.com> parents: 37058 diff changeset	552 raise ValueError('cannot encode data in a single frame')
0a6c5cc09a88 wireproto: define human output side channel frame Gregory Szorc <gregory.szorc@gmail.com> parents: 37058 diff changeset	553
37285 3ed344546d9e wireproto: start to associate frame generation with a stream Gregory Szorc <gregory.szorc@gmail.com> parents: 37084 diff changeset	554 yield stream.makeframe(requestid=requestid,
3ed344546d9e wireproto: start to associate frame generation with a stream Gregory Szorc <gregory.szorc@gmail.com> parents: 37084 diff changeset	555 typeid=FRAME_TYPE_TEXT_OUTPUT,
3ed344546d9e wireproto: start to associate frame generation with a stream Gregory Szorc <gregory.szorc@gmail.com> parents: 37084 diff changeset	556 flags=0,
37319 36d17f37db91 wireproto: convert human output frames to CBOR Gregory Szorc <gregory.szorc@gmail.com> parents: 37299 diff changeset	557 payload=payload)
37285 3ed344546d9e wireproto: start to associate frame generation with a stream Gregory Szorc <gregory.szorc@gmail.com> parents: 37084 diff changeset	558
39560 84bf6ded9317 wireprotoframing: buffer emitted data to reduce frame count Gregory Szorc <gregory.szorc@gmail.com> parents: 39559 diff changeset	559 class bufferingcommandresponseemitter(object):
84bf6ded9317 wireprotoframing: buffer emitted data to reduce frame count Gregory Szorc <gregory.szorc@gmail.com> parents: 39559 diff changeset	560 """Helper object to emit command response frames intelligently.
84bf6ded9317 wireprotoframing: buffer emitted data to reduce frame count Gregory Szorc <gregory.szorc@gmail.com> parents: 39559 diff changeset	561
84bf6ded9317 wireprotoframing: buffer emitted data to reduce frame count Gregory Szorc <gregory.szorc@gmail.com> parents: 39559 diff changeset	562 Raw command response data is likely emitted in chunks much smaller
84bf6ded9317 wireprotoframing: buffer emitted data to reduce frame count Gregory Szorc <gregory.szorc@gmail.com> parents: 39559 diff changeset	563 than what can fit in a single frame. This class exists to buffer
84bf6ded9317 wireprotoframing: buffer emitted data to reduce frame count Gregory Szorc <gregory.szorc@gmail.com> parents: 39559 diff changeset	564 chunks until enough data is available to fit in a single frame.
84bf6ded9317 wireprotoframing: buffer emitted data to reduce frame count Gregory Szorc <gregory.szorc@gmail.com> parents: 39559 diff changeset	565
84bf6ded9317 wireprotoframing: buffer emitted data to reduce frame count Gregory Szorc <gregory.szorc@gmail.com> parents: 39559 diff changeset	566 TODO we'll need something like this when compression is supported.
84bf6ded9317 wireprotoframing: buffer emitted data to reduce frame count Gregory Szorc <gregory.szorc@gmail.com> parents: 39559 diff changeset	567 So it might make sense to implement this functionality at the stream
84bf6ded9317 wireprotoframing: buffer emitted data to reduce frame count Gregory Szorc <gregory.szorc@gmail.com> parents: 39559 diff changeset	568 level.
84bf6ded9317 wireprotoframing: buffer emitted data to reduce frame count Gregory Szorc <gregory.szorc@gmail.com> parents: 39559 diff changeset	569 """
84bf6ded9317 wireprotoframing: buffer emitted data to reduce frame count Gregory Szorc <gregory.szorc@gmail.com> parents: 39559 diff changeset	570 def __init__(self, stream, requestid, maxframesize=DEFAULT_MAX_FRAME_SIZE):
84bf6ded9317 wireprotoframing: buffer emitted data to reduce frame count Gregory Szorc <gregory.szorc@gmail.com> parents: 39559 diff changeset	571 self._stream = stream
84bf6ded9317 wireprotoframing: buffer emitted data to reduce frame count Gregory Szorc <gregory.szorc@gmail.com> parents: 39559 diff changeset	572 self._requestid = requestid
84bf6ded9317 wireprotoframing: buffer emitted data to reduce frame count Gregory Szorc <gregory.szorc@gmail.com> parents: 39559 diff changeset	573 self._maxsize = maxframesize
84bf6ded9317 wireprotoframing: buffer emitted data to reduce frame count Gregory Szorc <gregory.szorc@gmail.com> parents: 39559 diff changeset	574 self._chunks = []
84bf6ded9317 wireprotoframing: buffer emitted data to reduce frame count Gregory Szorc <gregory.szorc@gmail.com> parents: 39559 diff changeset	575 self._chunkssize = 0
84bf6ded9317 wireprotoframing: buffer emitted data to reduce frame count Gregory Szorc <gregory.szorc@gmail.com> parents: 39559 diff changeset	576
84bf6ded9317 wireprotoframing: buffer emitted data to reduce frame count Gregory Szorc <gregory.szorc@gmail.com> parents: 39559 diff changeset	577 def send(self, data):
84bf6ded9317 wireprotoframing: buffer emitted data to reduce frame count Gregory Szorc <gregory.szorc@gmail.com> parents: 39559 diff changeset	578 """Send new data for emission.
84bf6ded9317 wireprotoframing: buffer emitted data to reduce frame count Gregory Szorc <gregory.szorc@gmail.com> parents: 39559 diff changeset	579
84bf6ded9317 wireprotoframing: buffer emitted data to reduce frame count Gregory Szorc <gregory.szorc@gmail.com> parents: 39559 diff changeset	580 Is a generator of new frames that were derived from the new input.
84bf6ded9317 wireprotoframing: buffer emitted data to reduce frame count Gregory Szorc <gregory.szorc@gmail.com> parents: 39559 diff changeset	581
84bf6ded9317 wireprotoframing: buffer emitted data to reduce frame count Gregory Szorc <gregory.szorc@gmail.com> parents: 39559 diff changeset	582 If the special input ``None`` is received, flushes all buffered
84bf6ded9317 wireprotoframing: buffer emitted data to reduce frame count Gregory Szorc <gregory.szorc@gmail.com> parents: 39559 diff changeset	583 data to frames.
84bf6ded9317 wireprotoframing: buffer emitted data to reduce frame count Gregory Szorc <gregory.szorc@gmail.com> parents: 39559 diff changeset	584 """
84bf6ded9317 wireprotoframing: buffer emitted data to reduce frame count Gregory Szorc <gregory.szorc@gmail.com> parents: 39559 diff changeset	585
84bf6ded9317 wireprotoframing: buffer emitted data to reduce frame count Gregory Szorc <gregory.szorc@gmail.com> parents: 39559 diff changeset	586 if data is None:
84bf6ded9317 wireprotoframing: buffer emitted data to reduce frame count Gregory Szorc <gregory.szorc@gmail.com> parents: 39559 diff changeset	587 for frame in self._flush():
84bf6ded9317 wireprotoframing: buffer emitted data to reduce frame count Gregory Szorc <gregory.szorc@gmail.com> parents: 39559 diff changeset	588 yield frame
84bf6ded9317 wireprotoframing: buffer emitted data to reduce frame count Gregory Szorc <gregory.szorc@gmail.com> parents: 39559 diff changeset	589 return
84bf6ded9317 wireprotoframing: buffer emitted data to reduce frame count Gregory Szorc <gregory.szorc@gmail.com> parents: 39559 diff changeset	590
84bf6ded9317 wireprotoframing: buffer emitted data to reduce frame count Gregory Szorc <gregory.szorc@gmail.com> parents: 39559 diff changeset	591 # There is a ton of potential to do more complicated things here.
84bf6ded9317 wireprotoframing: buffer emitted data to reduce frame count Gregory Szorc <gregory.szorc@gmail.com> parents: 39559 diff changeset	592 # Our immediate goal is to coalesce small chunks into big frames,
84bf6ded9317 wireprotoframing: buffer emitted data to reduce frame count Gregory Szorc <gregory.szorc@gmail.com> parents: 39559 diff changeset	593 # not achieve the fewest number of frames possible. So we go with
84bf6ded9317 wireprotoframing: buffer emitted data to reduce frame count Gregory Szorc <gregory.szorc@gmail.com> parents: 39559 diff changeset	594 # a simple implementation:
84bf6ded9317 wireprotoframing: buffer emitted data to reduce frame count Gregory Szorc <gregory.szorc@gmail.com> parents: 39559 diff changeset	595 #
84bf6ded9317 wireprotoframing: buffer emitted data to reduce frame count Gregory Szorc <gregory.szorc@gmail.com> parents: 39559 diff changeset	596 # * If a chunk is too large for a frame, we flush and emit frames
84bf6ded9317 wireprotoframing: buffer emitted data to reduce frame count Gregory Szorc <gregory.szorc@gmail.com> parents: 39559 diff changeset	597 # for the new chunk.
84bf6ded9317 wireprotoframing: buffer emitted data to reduce frame count Gregory Szorc <gregory.szorc@gmail.com> parents: 39559 diff changeset	598 # * If a chunk can be buffered without total buffered size limits
84bf6ded9317 wireprotoframing: buffer emitted data to reduce frame count Gregory Szorc <gregory.szorc@gmail.com> parents: 39559 diff changeset	599 # being exceeded, we do that.
84bf6ded9317 wireprotoframing: buffer emitted data to reduce frame count Gregory Szorc <gregory.szorc@gmail.com> parents: 39559 diff changeset	600 # * If a chunk causes us to go over our buffering limit, we flush
84bf6ded9317 wireprotoframing: buffer emitted data to reduce frame count Gregory Szorc <gregory.szorc@gmail.com> parents: 39559 diff changeset	601 # and then buffer the new chunk.
84bf6ded9317 wireprotoframing: buffer emitted data to reduce frame count Gregory Szorc <gregory.szorc@gmail.com> parents: 39559 diff changeset	602
84bf6ded9317 wireprotoframing: buffer emitted data to reduce frame count Gregory Szorc <gregory.szorc@gmail.com> parents: 39559 diff changeset	603 if len(data) > self._maxsize:
84bf6ded9317 wireprotoframing: buffer emitted data to reduce frame count Gregory Szorc <gregory.szorc@gmail.com> parents: 39559 diff changeset	604 for frame in self._flush():
84bf6ded9317 wireprotoframing: buffer emitted data to reduce frame count Gregory Szorc <gregory.szorc@gmail.com> parents: 39559 diff changeset	605 yield frame
84bf6ded9317 wireprotoframing: buffer emitted data to reduce frame count Gregory Szorc <gregory.szorc@gmail.com> parents: 39559 diff changeset	606
84bf6ded9317 wireprotoframing: buffer emitted data to reduce frame count Gregory Szorc <gregory.szorc@gmail.com> parents: 39559 diff changeset	607 # Now emit frames for the big chunk.
84bf6ded9317 wireprotoframing: buffer emitted data to reduce frame count Gregory Szorc <gregory.szorc@gmail.com> parents: 39559 diff changeset	608 offset = 0
84bf6ded9317 wireprotoframing: buffer emitted data to reduce frame count Gregory Szorc <gregory.szorc@gmail.com> parents: 39559 diff changeset	609 while True:
84bf6ded9317 wireprotoframing: buffer emitted data to reduce frame count Gregory Szorc <gregory.szorc@gmail.com> parents: 39559 diff changeset	610 chunk = data[offset:offset + self._maxsize]
84bf6ded9317 wireprotoframing: buffer emitted data to reduce frame count Gregory Szorc <gregory.szorc@gmail.com> parents: 39559 diff changeset	611 offset += len(chunk)
84bf6ded9317 wireprotoframing: buffer emitted data to reduce frame count Gregory Szorc <gregory.szorc@gmail.com> parents: 39559 diff changeset	612
84bf6ded9317 wireprotoframing: buffer emitted data to reduce frame count Gregory Szorc <gregory.szorc@gmail.com> parents: 39559 diff changeset	613 yield self._stream.makeframe(
84bf6ded9317 wireprotoframing: buffer emitted data to reduce frame count Gregory Szorc <gregory.szorc@gmail.com> parents: 39559 diff changeset	614 self._requestid,
84bf6ded9317 wireprotoframing: buffer emitted data to reduce frame count Gregory Szorc <gregory.szorc@gmail.com> parents: 39559 diff changeset	615 typeid=FRAME_TYPE_COMMAND_RESPONSE,
84bf6ded9317 wireprotoframing: buffer emitted data to reduce frame count Gregory Szorc <gregory.szorc@gmail.com> parents: 39559 diff changeset	616 flags=FLAG_COMMAND_RESPONSE_CONTINUATION,
84bf6ded9317 wireprotoframing: buffer emitted data to reduce frame count Gregory Szorc <gregory.szorc@gmail.com> parents: 39559 diff changeset	617 payload=chunk)
84bf6ded9317 wireprotoframing: buffer emitted data to reduce frame count Gregory Szorc <gregory.szorc@gmail.com> parents: 39559 diff changeset	618
84bf6ded9317 wireprotoframing: buffer emitted data to reduce frame count Gregory Szorc <gregory.szorc@gmail.com> parents: 39559 diff changeset	619 if offset == len(data):
84bf6ded9317 wireprotoframing: buffer emitted data to reduce frame count Gregory Szorc <gregory.szorc@gmail.com> parents: 39559 diff changeset	620 return
84bf6ded9317 wireprotoframing: buffer emitted data to reduce frame count Gregory Szorc <gregory.szorc@gmail.com> parents: 39559 diff changeset	621
84bf6ded9317 wireprotoframing: buffer emitted data to reduce frame count Gregory Szorc <gregory.szorc@gmail.com> parents: 39559 diff changeset	622 # If we don't have enough to constitute a full frame, buffer and
84bf6ded9317 wireprotoframing: buffer emitted data to reduce frame count Gregory Szorc <gregory.szorc@gmail.com> parents: 39559 diff changeset	623 # return.
84bf6ded9317 wireprotoframing: buffer emitted data to reduce frame count Gregory Szorc <gregory.szorc@gmail.com> parents: 39559 diff changeset	624 if len(data) + self._chunkssize < self._maxsize:
84bf6ded9317 wireprotoframing: buffer emitted data to reduce frame count Gregory Szorc <gregory.szorc@gmail.com> parents: 39559 diff changeset	625 self._chunks.append(data)
84bf6ded9317 wireprotoframing: buffer emitted data to reduce frame count Gregory Szorc <gregory.szorc@gmail.com> parents: 39559 diff changeset	626 self._chunkssize += len(data)
84bf6ded9317 wireprotoframing: buffer emitted data to reduce frame count Gregory Szorc <gregory.szorc@gmail.com> parents: 39559 diff changeset	627 return
84bf6ded9317 wireprotoframing: buffer emitted data to reduce frame count Gregory Szorc <gregory.szorc@gmail.com> parents: 39559 diff changeset	628
84bf6ded9317 wireprotoframing: buffer emitted data to reduce frame count Gregory Szorc <gregory.szorc@gmail.com> parents: 39559 diff changeset	629 # Else flush what we have and buffer the new chunk. We could do
84bf6ded9317 wireprotoframing: buffer emitted data to reduce frame count Gregory Szorc <gregory.szorc@gmail.com> parents: 39559 diff changeset	630 # something more intelligent here, like break the chunk. Let's
84bf6ded9317 wireprotoframing: buffer emitted data to reduce frame count Gregory Szorc <gregory.szorc@gmail.com> parents: 39559 diff changeset	631 # keep things simple for now.
84bf6ded9317 wireprotoframing: buffer emitted data to reduce frame count Gregory Szorc <gregory.szorc@gmail.com> parents: 39559 diff changeset	632 for frame in self._flush():
84bf6ded9317 wireprotoframing: buffer emitted data to reduce frame count Gregory Szorc <gregory.szorc@gmail.com> parents: 39559 diff changeset	633 yield frame
84bf6ded9317 wireprotoframing: buffer emitted data to reduce frame count Gregory Szorc <gregory.szorc@gmail.com> parents: 39559 diff changeset	634
84bf6ded9317 wireprotoframing: buffer emitted data to reduce frame count Gregory Szorc <gregory.szorc@gmail.com> parents: 39559 diff changeset	635 self._chunks.append(data)
84bf6ded9317 wireprotoframing: buffer emitted data to reduce frame count Gregory Szorc <gregory.szorc@gmail.com> parents: 39559 diff changeset	636 self._chunkssize = len(data)
84bf6ded9317 wireprotoframing: buffer emitted data to reduce frame count Gregory Szorc <gregory.szorc@gmail.com> parents: 39559 diff changeset	637
84bf6ded9317 wireprotoframing: buffer emitted data to reduce frame count Gregory Szorc <gregory.szorc@gmail.com> parents: 39559 diff changeset	638 def _flush(self):
84bf6ded9317 wireprotoframing: buffer emitted data to reduce frame count Gregory Szorc <gregory.szorc@gmail.com> parents: 39559 diff changeset	639 payload = b''.join(self._chunks)
84bf6ded9317 wireprotoframing: buffer emitted data to reduce frame count Gregory Szorc <gregory.szorc@gmail.com> parents: 39559 diff changeset	640 assert len(payload) <= self._maxsize
84bf6ded9317 wireprotoframing: buffer emitted data to reduce frame count Gregory Szorc <gregory.szorc@gmail.com> parents: 39559 diff changeset	641
84bf6ded9317 wireprotoframing: buffer emitted data to reduce frame count Gregory Szorc <gregory.szorc@gmail.com> parents: 39559 diff changeset	642 self._chunks[:] = []
84bf6ded9317 wireprotoframing: buffer emitted data to reduce frame count Gregory Szorc <gregory.szorc@gmail.com> parents: 39559 diff changeset	643 self._chunkssize = 0
84bf6ded9317 wireprotoframing: buffer emitted data to reduce frame count Gregory Szorc <gregory.szorc@gmail.com> parents: 39559 diff changeset	644
84bf6ded9317 wireprotoframing: buffer emitted data to reduce frame count Gregory Szorc <gregory.szorc@gmail.com> parents: 39559 diff changeset	645 yield self._stream.makeframe(
84bf6ded9317 wireprotoframing: buffer emitted data to reduce frame count Gregory Szorc <gregory.szorc@gmail.com> parents: 39559 diff changeset	646 self._requestid,
84bf6ded9317 wireprotoframing: buffer emitted data to reduce frame count Gregory Szorc <gregory.szorc@gmail.com> parents: 39559 diff changeset	647 typeid=FRAME_TYPE_COMMAND_RESPONSE,
84bf6ded9317 wireprotoframing: buffer emitted data to reduce frame count Gregory Szorc <gregory.szorc@gmail.com> parents: 39559 diff changeset	648 flags=FLAG_COMMAND_RESPONSE_CONTINUATION,
84bf6ded9317 wireprotoframing: buffer emitted data to reduce frame count Gregory Szorc <gregory.szorc@gmail.com> parents: 39559 diff changeset	649 payload=payload)
84bf6ded9317 wireprotoframing: buffer emitted data to reduce frame count Gregory Szorc <gregory.szorc@gmail.com> parents: 39559 diff changeset	650
37285 3ed344546d9e wireproto: start to associate frame generation with a stream Gregory Szorc <gregory.szorc@gmail.com> parents: 37084 diff changeset	651 class stream(object):
3ed344546d9e wireproto: start to associate frame generation with a stream Gregory Szorc <gregory.szorc@gmail.com> parents: 37084 diff changeset	652 """Represents a logical unidirectional series of frames."""
3ed344546d9e wireproto: start to associate frame generation with a stream Gregory Szorc <gregory.szorc@gmail.com> parents: 37084 diff changeset	653
37288 9bfcbe4f4745 wireproto: add streams to frame-based protocol Gregory Szorc <gregory.szorc@gmail.com> parents: 37285 diff changeset	654 def __init__(self, streamid, active=False):
9bfcbe4f4745 wireproto: add streams to frame-based protocol Gregory Szorc <gregory.szorc@gmail.com> parents: 37285 diff changeset	655 self.streamid = streamid
37655 b9502b5f2066 wireprotoframing: use value passed into function Gregory Szorc <gregory.szorc@gmail.com> parents: 37544 diff changeset	656 self._active = active
37288 9bfcbe4f4745 wireproto: add streams to frame-based protocol Gregory Szorc <gregory.szorc@gmail.com> parents: 37285 diff changeset	657
37285 3ed344546d9e wireproto: start to associate frame generation with a stream Gregory Szorc <gregory.szorc@gmail.com> parents: 37084 diff changeset	658 def makeframe(self, requestid, typeid, flags, payload):
3ed344546d9e wireproto: start to associate frame generation with a stream Gregory Szorc <gregory.szorc@gmail.com> parents: 37084 diff changeset	659 """Create a frame to be sent out over this stream.
3ed344546d9e wireproto: start to associate frame generation with a stream Gregory Szorc <gregory.szorc@gmail.com> parents: 37084 diff changeset	660
3ed344546d9e wireproto: start to associate frame generation with a stream Gregory Szorc <gregory.szorc@gmail.com> parents: 37084 diff changeset	661 Only returns the frame instance. Does not actually send it.
3ed344546d9e wireproto: start to associate frame generation with a stream Gregory Szorc <gregory.szorc@gmail.com> parents: 37084 diff changeset	662 """
37288 9bfcbe4f4745 wireproto: add streams to frame-based protocol Gregory Szorc <gregory.szorc@gmail.com> parents: 37285 diff changeset	663 streamflags = 0
9bfcbe4f4745 wireproto: add streams to frame-based protocol Gregory Szorc <gregory.szorc@gmail.com> parents: 37285 diff changeset	664 if not self._active:
9bfcbe4f4745 wireproto: add streams to frame-based protocol Gregory Szorc <gregory.szorc@gmail.com> parents: 37285 diff changeset	665 streamflags \|= STREAM_FLAG_BEGIN_STREAM
9bfcbe4f4745 wireproto: add streams to frame-based protocol Gregory Szorc <gregory.szorc@gmail.com> parents: 37285 diff changeset	666 self._active = True
9bfcbe4f4745 wireproto: add streams to frame-based protocol Gregory Szorc <gregory.szorc@gmail.com> parents: 37285 diff changeset	667
9bfcbe4f4745 wireproto: add streams to frame-based protocol Gregory Szorc <gregory.szorc@gmail.com> parents: 37285 diff changeset	668 return makeframe(requestid, self.streamid, streamflags, typeid, flags,
9bfcbe4f4745 wireproto: add streams to frame-based protocol Gregory Szorc <gregory.szorc@gmail.com> parents: 37285 diff changeset	669 payload)
9bfcbe4f4745 wireproto: add streams to frame-based protocol Gregory Szorc <gregory.szorc@gmail.com> parents: 37285 diff changeset	670
40130 5d44c4d1d516 wireprotov2: establish dedicated classes for input and output streams Gregory Szorc <gregory.szorc@gmail.com> parents: 40129 diff changeset	671 class inputstream(stream):
5d44c4d1d516 wireprotov2: establish dedicated classes for input and output streams Gregory Szorc <gregory.szorc@gmail.com> parents: 40129 diff changeset	672 """Represents a stream used for receiving data."""
5d44c4d1d516 wireprotov2: establish dedicated classes for input and output streams Gregory Szorc <gregory.szorc@gmail.com> parents: 40129 diff changeset	673
40128 57782791b7e9 wireprotov2: handle stream encoding settings frames Gregory Szorc <gregory.szorc@gmail.com> parents: 40127 diff changeset	674 def setdecoder(self, name, extraobjs):
57782791b7e9 wireprotov2: handle stream encoding settings frames Gregory Szorc <gregory.szorc@gmail.com> parents: 40127 diff changeset	675 """Set the decoder for this stream.
57782791b7e9 wireprotov2: handle stream encoding settings frames Gregory Szorc <gregory.szorc@gmail.com> parents: 40127 diff changeset	676
57782791b7e9 wireprotov2: handle stream encoding settings frames Gregory Szorc <gregory.szorc@gmail.com> parents: 40127 diff changeset	677 Receives the stream profile name and any additional CBOR objects
57782791b7e9 wireprotov2: handle stream encoding settings frames Gregory Szorc <gregory.szorc@gmail.com> parents: 40127 diff changeset	678 decoded from the stream encoding settings frame payloads.
57782791b7e9 wireprotov2: handle stream encoding settings frames Gregory Szorc <gregory.szorc@gmail.com> parents: 40127 diff changeset	679 """
57782791b7e9 wireprotov2: handle stream encoding settings frames Gregory Szorc <gregory.szorc@gmail.com> parents: 40127 diff changeset	680
40130 5d44c4d1d516 wireprotov2: establish dedicated classes for input and output streams Gregory Szorc <gregory.szorc@gmail.com> parents: 40129 diff changeset	681 class outputstream(stream):
5d44c4d1d516 wireprotov2: establish dedicated classes for input and output streams Gregory Szorc <gregory.szorc@gmail.com> parents: 40129 diff changeset	682 """Represents a stream used for sending data."""
5d44c4d1d516 wireprotov2: establish dedicated classes for input and output streams Gregory Szorc <gregory.szorc@gmail.com> parents: 40129 diff changeset	683
37288 9bfcbe4f4745 wireproto: add streams to frame-based protocol Gregory Szorc <gregory.szorc@gmail.com> parents: 37285 diff changeset	684 def ensureserverstream(stream):
9bfcbe4f4745 wireproto: add streams to frame-based protocol Gregory Szorc <gregory.szorc@gmail.com> parents: 37285 diff changeset	685 if stream.streamid % 2:
9bfcbe4f4745 wireproto: add streams to frame-based protocol Gregory Szorc <gregory.szorc@gmail.com> parents: 37285 diff changeset	686 raise error.ProgrammingError('server should only write to even '
9bfcbe4f4745 wireproto: add streams to frame-based protocol Gregory Szorc <gregory.szorc@gmail.com> parents: 37285 diff changeset	687 'numbered streams; %d is not even' %
9bfcbe4f4745 wireproto: add streams to frame-based protocol Gregory Szorc <gregory.szorc@gmail.com> parents: 37285 diff changeset	688 stream.streamid)
37060 0a6c5cc09a88 wireproto: define human output side channel frame Gregory Szorc <gregory.szorc@gmail.com> parents: 37058 diff changeset	689
40126 327d40b94bed wireprotov2: handle sender protocol settings frames Gregory Szorc <gregory.szorc@gmail.com> parents: 40125 diff changeset	690 DEFAULT_PROTOCOL_SETTINGS = {
327d40b94bed wireprotov2: handle sender protocol settings frames Gregory Szorc <gregory.szorc@gmail.com> parents: 40125 diff changeset	691 'contentencodings': [b'identity'],
327d40b94bed wireprotov2: handle sender protocol settings frames Gregory Szorc <gregory.szorc@gmail.com> parents: 40125 diff changeset	692 }
327d40b94bed wireprotov2: handle sender protocol settings frames Gregory Szorc <gregory.szorc@gmail.com> parents: 40125 diff changeset	693
37052 8c3c47362934 wireproto: implement basic frame reading and processing Gregory Szorc <gregory.szorc@gmail.com> parents: 37051 diff changeset	694 class serverreactor(object):
8c3c47362934 wireproto: implement basic frame reading and processing Gregory Szorc <gregory.szorc@gmail.com> parents: 37051 diff changeset	695 """Holds state of a server handling frame-based protocol requests.
8c3c47362934 wireproto: implement basic frame reading and processing Gregory Szorc <gregory.szorc@gmail.com> parents: 37051 diff changeset	696
8c3c47362934 wireproto: implement basic frame reading and processing Gregory Szorc <gregory.szorc@gmail.com> parents: 37051 diff changeset	697 This class is the "brain" of the unified frame-based protocol server
8c3c47362934 wireproto: implement basic frame reading and processing Gregory Szorc <gregory.szorc@gmail.com> parents: 37051 diff changeset	698 component. While the protocol is stateless from the perspective of
8c3c47362934 wireproto: implement basic frame reading and processing Gregory Szorc <gregory.szorc@gmail.com> parents: 37051 diff changeset	699 requests/commands, something needs to track which frames have been
8c3c47362934 wireproto: implement basic frame reading and processing Gregory Szorc <gregory.szorc@gmail.com> parents: 37051 diff changeset	700 received, what frames to expect, etc. This class is that thing.
8c3c47362934 wireproto: implement basic frame reading and processing Gregory Szorc <gregory.szorc@gmail.com> parents: 37051 diff changeset	701
8c3c47362934 wireproto: implement basic frame reading and processing Gregory Szorc <gregory.szorc@gmail.com> parents: 37051 diff changeset	702 Instances are modeled as a state machine of sorts. Instances are also
8c3c47362934 wireproto: implement basic frame reading and processing Gregory Szorc <gregory.szorc@gmail.com> parents: 37051 diff changeset	703 reactionary to external events. The point of this class is to encapsulate
8c3c47362934 wireproto: implement basic frame reading and processing Gregory Szorc <gregory.szorc@gmail.com> parents: 37051 diff changeset	704 the state of the connection and the exchange of frames, not to perform
8c3c47362934 wireproto: implement basic frame reading and processing Gregory Szorc <gregory.szorc@gmail.com> parents: 37051 diff changeset	705 work. Instead, callers tell this class when something occurs, like a
8c3c47362934 wireproto: implement basic frame reading and processing Gregory Szorc <gregory.szorc@gmail.com> parents: 37051 diff changeset	706 frame arriving. If that activity is worthy of a follow-up action (say
8c3c47362934 wireproto: implement basic frame reading and processing Gregory Szorc <gregory.szorc@gmail.com> parents: 37051 diff changeset	707 run a command), the return value of that handler will say so.
8c3c47362934 wireproto: implement basic frame reading and processing Gregory Szorc <gregory.szorc@gmail.com> parents: 37051 diff changeset	708
8c3c47362934 wireproto: implement basic frame reading and processing Gregory Szorc <gregory.szorc@gmail.com> parents: 37051 diff changeset	709 I/O and CPU intensive operations are purposefully delegated outside of
8c3c47362934 wireproto: implement basic frame reading and processing Gregory Szorc <gregory.szorc@gmail.com> parents: 37051 diff changeset	710 this class.
8c3c47362934 wireproto: implement basic frame reading and processing Gregory Szorc <gregory.szorc@gmail.com> parents: 37051 diff changeset	711
8c3c47362934 wireproto: implement basic frame reading and processing Gregory Szorc <gregory.szorc@gmail.com> parents: 37051 diff changeset	712 Consumers are expected to tell instances when events occur. They do so by
8c3c47362934 wireproto: implement basic frame reading and processing Gregory Szorc <gregory.szorc@gmail.com> parents: 37051 diff changeset	713 calling the various ``on*`` methods. These methods return a 2-tuple
8c3c47362934 wireproto: implement basic frame reading and processing Gregory Szorc <gregory.szorc@gmail.com> parents: 37051 diff changeset	714 describing any follow-up action(s) to take. The first element is the
8c3c47362934 wireproto: implement basic frame reading and processing Gregory Szorc <gregory.szorc@gmail.com> parents: 37051 diff changeset	715 name of an action to perform. The second is a data structure (usually
8c3c47362934 wireproto: implement basic frame reading and processing Gregory Szorc <gregory.szorc@gmail.com> parents: 37051 diff changeset	716 a dict) specific to that action that contains more information. e.g.
8c3c47362934 wireproto: implement basic frame reading and processing Gregory Szorc <gregory.szorc@gmail.com> parents: 37051 diff changeset	717 if the server wants to send frames back to the client, the data structure
8c3c47362934 wireproto: implement basic frame reading and processing Gregory Szorc <gregory.szorc@gmail.com> parents: 37051 diff changeset	718 will contain a reference to those frames.
8c3c47362934 wireproto: implement basic frame reading and processing Gregory Szorc <gregory.szorc@gmail.com> parents: 37051 diff changeset	719
8c3c47362934 wireproto: implement basic frame reading and processing Gregory Szorc <gregory.szorc@gmail.com> parents: 37051 diff changeset	720 Valid actions that consumers can be instructed to take are:
8c3c47362934 wireproto: implement basic frame reading and processing Gregory Szorc <gregory.szorc@gmail.com> parents: 37051 diff changeset	721
37055 61393f888dfe wireproto: define and implement responses in framing protocol Gregory Szorc <gregory.szorc@gmail.com> parents: 37052 diff changeset	722 sendframes
61393f888dfe wireproto: define and implement responses in framing protocol Gregory Szorc <gregory.szorc@gmail.com> parents: 37052 diff changeset	723 Indicates that frames should be sent to the client. The ``framegen``
61393f888dfe wireproto: define and implement responses in framing protocol Gregory Szorc <gregory.szorc@gmail.com> parents: 37052 diff changeset	724 key contains a generator of frames that should be sent. The server
61393f888dfe wireproto: define and implement responses in framing protocol Gregory Szorc <gregory.szorc@gmail.com> parents: 37052 diff changeset	725 assumes that all frames are sent to the client.
61393f888dfe wireproto: define and implement responses in framing protocol Gregory Szorc <gregory.szorc@gmail.com> parents: 37052 diff changeset	726
37052 8c3c47362934 wireproto: implement basic frame reading and processing Gregory Szorc <gregory.szorc@gmail.com> parents: 37051 diff changeset	727 error
8c3c47362934 wireproto: implement basic frame reading and processing Gregory Szorc <gregory.szorc@gmail.com> parents: 37051 diff changeset	728 Indicates that an error occurred. Consumer should probably abort.
8c3c47362934 wireproto: implement basic frame reading and processing Gregory Szorc <gregory.szorc@gmail.com> parents: 37051 diff changeset	729
8c3c47362934 wireproto: implement basic frame reading and processing Gregory Szorc <gregory.szorc@gmail.com> parents: 37051 diff changeset	730 runcommand
8c3c47362934 wireproto: implement basic frame reading and processing Gregory Szorc <gregory.szorc@gmail.com> parents: 37051 diff changeset	731 Indicates that the consumer should run a wire protocol command. Details
8c3c47362934 wireproto: implement basic frame reading and processing Gregory Szorc <gregory.szorc@gmail.com> parents: 37051 diff changeset	732 of the command to run are given in the data structure.
8c3c47362934 wireproto: implement basic frame reading and processing Gregory Szorc <gregory.szorc@gmail.com> parents: 37051 diff changeset	733
8c3c47362934 wireproto: implement basic frame reading and processing Gregory Szorc <gregory.szorc@gmail.com> parents: 37051 diff changeset	734 wantframe
8c3c47362934 wireproto: implement basic frame reading and processing Gregory Szorc <gregory.szorc@gmail.com> parents: 37051 diff changeset	735 Indicates that nothing of interest happened and the server is waiting on
8c3c47362934 wireproto: implement basic frame reading and processing Gregory Szorc <gregory.szorc@gmail.com> parents: 37051 diff changeset	736 more frames from the client before anything interesting can be done.
37056 861e9d37e56e wireproto: buffer output frames when in half duplex mode Gregory Szorc <gregory.szorc@gmail.com> parents: 37055 diff changeset	737
861e9d37e56e wireproto: buffer output frames when in half duplex mode Gregory Szorc <gregory.szorc@gmail.com> parents: 37055 diff changeset	738 noop
861e9d37e56e wireproto: buffer output frames when in half duplex mode Gregory Szorc <gregory.szorc@gmail.com> parents: 37055 diff changeset	739 Indicates no additional action is required.
37058 c5e9c3b47366 wireproto: support for receiving multiple requests Gregory Szorc <gregory.szorc@gmail.com> parents: 37057 diff changeset	740
c5e9c3b47366 wireproto: support for receiving multiple requests Gregory Szorc <gregory.szorc@gmail.com> parents: 37057 diff changeset	741 Known Issues
c5e9c3b47366 wireproto: support for receiving multiple requests Gregory Szorc <gregory.szorc@gmail.com> parents: 37057 diff changeset	742 ------------
c5e9c3b47366 wireproto: support for receiving multiple requests Gregory Szorc <gregory.szorc@gmail.com> parents: 37057 diff changeset	743
c5e9c3b47366 wireproto: support for receiving multiple requests Gregory Szorc <gregory.szorc@gmail.com> parents: 37057 diff changeset	744 There are no limits to the number of partially received commands or their
c5e9c3b47366 wireproto: support for receiving multiple requests Gregory Szorc <gregory.szorc@gmail.com> parents: 37057 diff changeset	745 size. A malicious client could stream command request data and exhaust the
c5e9c3b47366 wireproto: support for receiving multiple requests Gregory Szorc <gregory.szorc@gmail.com> parents: 37057 diff changeset	746 server's memory.
c5e9c3b47366 wireproto: support for receiving multiple requests Gregory Szorc <gregory.szorc@gmail.com> parents: 37057 diff changeset	747
c5e9c3b47366 wireproto: support for receiving multiple requests Gregory Szorc <gregory.szorc@gmail.com> parents: 37057 diff changeset	748 Partially received commands are not acted upon when end of input is
c5e9c3b47366 wireproto: support for receiving multiple requests Gregory Szorc <gregory.szorc@gmail.com> parents: 37057 diff changeset	749 reached. Should the server error if it receives a partial request?
c5e9c3b47366 wireproto: support for receiving multiple requests Gregory Szorc <gregory.szorc@gmail.com> parents: 37057 diff changeset	750 Should the client send a message to abort a partially transmitted request
c5e9c3b47366 wireproto: support for receiving multiple requests Gregory Szorc <gregory.szorc@gmail.com> parents: 37057 diff changeset	751 to facilitate graceful shutdown?
c5e9c3b47366 wireproto: support for receiving multiple requests Gregory Szorc <gregory.szorc@gmail.com> parents: 37057 diff changeset	752
c5e9c3b47366 wireproto: support for receiving multiple requests Gregory Szorc <gregory.szorc@gmail.com> parents: 37057 diff changeset	753 Active requests that haven't been responded to aren't tracked. This means
c5e9c3b47366 wireproto: support for receiving multiple requests Gregory Szorc <gregory.szorc@gmail.com> parents: 37057 diff changeset	754 that if we receive a command and instruct its dispatch, another command
c5e9c3b47366 wireproto: support for receiving multiple requests Gregory Szorc <gregory.szorc@gmail.com> parents: 37057 diff changeset	755 with its request ID can come in over the wire and there will be a race
c5e9c3b47366 wireproto: support for receiving multiple requests Gregory Szorc <gregory.szorc@gmail.com> parents: 37057 diff changeset	756 between who responds to what.
37052 8c3c47362934 wireproto: implement basic frame reading and processing Gregory Szorc <gregory.szorc@gmail.com> parents: 37051 diff changeset	757 """
8c3c47362934 wireproto: implement basic frame reading and processing Gregory Szorc <gregory.szorc@gmail.com> parents: 37051 diff changeset	758
40129 293835e0fff7 wireprotov2: pass ui into clientreactor and serverreactor Gregory Szorc <gregory.szorc@gmail.com> parents: 40128 diff changeset	759 def __init__(self, ui, deferoutput=False):
37056 861e9d37e56e wireproto: buffer output frames when in half duplex mode Gregory Szorc <gregory.szorc@gmail.com> parents: 37055 diff changeset	760 """Construct a new server reactor.
861e9d37e56e wireproto: buffer output frames when in half duplex mode Gregory Szorc <gregory.szorc@gmail.com> parents: 37055 diff changeset	761
861e9d37e56e wireproto: buffer output frames when in half duplex mode Gregory Szorc <gregory.szorc@gmail.com> parents: 37055 diff changeset	762 ``deferoutput`` can be used to indicate that no output frames should be
861e9d37e56e wireproto: buffer output frames when in half duplex mode Gregory Szorc <gregory.szorc@gmail.com> parents: 37055 diff changeset	763 instructed to be sent until input has been exhausted. In this mode,
861e9d37e56e wireproto: buffer output frames when in half duplex mode Gregory Szorc <gregory.szorc@gmail.com> parents: 37055 diff changeset	764 events that would normally generate output frames (such as a command
861e9d37e56e wireproto: buffer output frames when in half duplex mode Gregory Szorc <gregory.szorc@gmail.com> parents: 37055 diff changeset	765 response being ready) will instead defer instructing the consumer to
861e9d37e56e wireproto: buffer output frames when in half duplex mode Gregory Szorc <gregory.szorc@gmail.com> parents: 37055 diff changeset	766 send those frames. This is useful for half-duplex transports where the
861e9d37e56e wireproto: buffer output frames when in half duplex mode Gregory Szorc <gregory.szorc@gmail.com> parents: 37055 diff changeset	767 sender cannot receive until all data has been transmitted.
861e9d37e56e wireproto: buffer output frames when in half duplex mode Gregory Szorc <gregory.szorc@gmail.com> parents: 37055 diff changeset	768 """
40129 293835e0fff7 wireprotov2: pass ui into clientreactor and serverreactor Gregory Szorc <gregory.szorc@gmail.com> parents: 40128 diff changeset	769 self._ui = ui
37056 861e9d37e56e wireproto: buffer output frames when in half duplex mode Gregory Szorc <gregory.szorc@gmail.com> parents: 37055 diff changeset	770 self._deferoutput = deferoutput
40126 327d40b94bed wireprotov2: handle sender protocol settings frames Gregory Szorc <gregory.szorc@gmail.com> parents: 40125 diff changeset	771 self._state = 'initial'
37289 5fadc63ac99f wireproto: explicit API to create outgoing streams Gregory Szorc <gregory.szorc@gmail.com> parents: 37288 diff changeset	772 self._nextoutgoingstreamid = 2
37056 861e9d37e56e wireproto: buffer output frames when in half duplex mode Gregory Szorc <gregory.szorc@gmail.com> parents: 37055 diff changeset	773 self._bufferedframegens = []
37288 9bfcbe4f4745 wireproto: add streams to frame-based protocol Gregory Szorc <gregory.szorc@gmail.com> parents: 37285 diff changeset	774 # stream id -> stream instance for all active streams from the client.
9bfcbe4f4745 wireproto: add streams to frame-based protocol Gregory Szorc <gregory.szorc@gmail.com> parents: 37285 diff changeset	775 self._incomingstreams = {}
37289 5fadc63ac99f wireproto: explicit API to create outgoing streams Gregory Szorc <gregory.szorc@gmail.com> parents: 37288 diff changeset	776 self._outgoingstreams = {}
37058 c5e9c3b47366 wireproto: support for receiving multiple requests Gregory Szorc <gregory.szorc@gmail.com> parents: 37057 diff changeset	777 # request id -> dict of commands that are actively being received.
c5e9c3b47366 wireproto: support for receiving multiple requests Gregory Szorc <gregory.szorc@gmail.com> parents: 37057 diff changeset	778 self._receivingcommands = {}
37063 39304dd63589 wireproto: explicitly track which requests are active Gregory Szorc <gregory.szorc@gmail.com> parents: 37062 diff changeset	779 # Request IDs that have been received and are actively being processed.
39304dd63589 wireproto: explicitly track which requests are active Gregory Szorc <gregory.szorc@gmail.com> parents: 37062 diff changeset	780 # Once all output for a request has been sent, it is removed from this
39304dd63589 wireproto: explicitly track which requests are active Gregory Szorc <gregory.szorc@gmail.com> parents: 37062 diff changeset	781 # set.
39304dd63589 wireproto: explicitly track which requests are active Gregory Szorc <gregory.szorc@gmail.com> parents: 37062 diff changeset	782 self._activecommands = set()
37052 8c3c47362934 wireproto: implement basic frame reading and processing Gregory Szorc <gregory.szorc@gmail.com> parents: 37051 diff changeset	783
40126 327d40b94bed wireprotov2: handle sender protocol settings frames Gregory Szorc <gregory.szorc@gmail.com> parents: 40125 diff changeset	784 self._protocolsettingsdecoder = None
327d40b94bed wireprotov2: handle sender protocol settings frames Gregory Szorc <gregory.szorc@gmail.com> parents: 40125 diff changeset	785
327d40b94bed wireprotov2: handle sender protocol settings frames Gregory Szorc <gregory.szorc@gmail.com> parents: 40125 diff changeset	786 # Sender protocol settings are optional. Set implied default values.
327d40b94bed wireprotov2: handle sender protocol settings frames Gregory Szorc <gregory.szorc@gmail.com> parents: 40125 diff changeset	787 self._sendersettings = dict(DEFAULT_PROTOCOL_SETTINGS)
327d40b94bed wireprotov2: handle sender protocol settings frames Gregory Szorc <gregory.szorc@gmail.com> parents: 40125 diff changeset	788
37061 884a0c1604ad wireproto: define attr-based classes for representing frames Gregory Szorc <gregory.szorc@gmail.com> parents: 37060 diff changeset	789 def onframerecv(self, frame):
37052 8c3c47362934 wireproto: implement basic frame reading and processing Gregory Szorc <gregory.szorc@gmail.com> parents: 37051 diff changeset	790 """Process a frame that has been received off the wire.
8c3c47362934 wireproto: implement basic frame reading and processing Gregory Szorc <gregory.szorc@gmail.com> parents: 37051 diff changeset	791
8c3c47362934 wireproto: implement basic frame reading and processing Gregory Szorc <gregory.szorc@gmail.com> parents: 37051 diff changeset	792 Returns a dict with an ``action`` key that details what action,
8c3c47362934 wireproto: implement basic frame reading and processing Gregory Szorc <gregory.szorc@gmail.com> parents: 37051 diff changeset	793 if any, the consumer should take next.
8c3c47362934 wireproto: implement basic frame reading and processing Gregory Szorc <gregory.szorc@gmail.com> parents: 37051 diff changeset	794 """
37288 9bfcbe4f4745 wireproto: add streams to frame-based protocol Gregory Szorc <gregory.szorc@gmail.com> parents: 37285 diff changeset	795 if not frame.streamid % 2:
9bfcbe4f4745 wireproto: add streams to frame-based protocol Gregory Szorc <gregory.szorc@gmail.com> parents: 37285 diff changeset	796 self._state = 'errored'
9bfcbe4f4745 wireproto: add streams to frame-based protocol Gregory Szorc <gregory.szorc@gmail.com> parents: 37285 diff changeset	797 return self._makeerrorresult(
9bfcbe4f4745 wireproto: add streams to frame-based protocol Gregory Szorc <gregory.szorc@gmail.com> parents: 37285 diff changeset	798 _('received frame with even numbered stream ID: %d') %
9bfcbe4f4745 wireproto: add streams to frame-based protocol Gregory Szorc <gregory.szorc@gmail.com> parents: 37285 diff changeset	799 frame.streamid)
9bfcbe4f4745 wireproto: add streams to frame-based protocol Gregory Szorc <gregory.szorc@gmail.com> parents: 37285 diff changeset	800
9bfcbe4f4745 wireproto: add streams to frame-based protocol Gregory Szorc <gregory.szorc@gmail.com> parents: 37285 diff changeset	801 if frame.streamid not in self._incomingstreams:
9bfcbe4f4745 wireproto: add streams to frame-based protocol Gregory Szorc <gregory.szorc@gmail.com> parents: 37285 diff changeset	802 if not frame.streamflags & STREAM_FLAG_BEGIN_STREAM:
9bfcbe4f4745 wireproto: add streams to frame-based protocol Gregory Szorc <gregory.szorc@gmail.com> parents: 37285 diff changeset	803 self._state = 'errored'
9bfcbe4f4745 wireproto: add streams to frame-based protocol Gregory Szorc <gregory.szorc@gmail.com> parents: 37285 diff changeset	804 return self._makeerrorresult(
9bfcbe4f4745 wireproto: add streams to frame-based protocol Gregory Szorc <gregory.szorc@gmail.com> parents: 37285 diff changeset	805 _('received frame on unknown inactive stream without '
9bfcbe4f4745 wireproto: add streams to frame-based protocol Gregory Szorc <gregory.szorc@gmail.com> parents: 37285 diff changeset	806 'beginning of stream flag set'))
9bfcbe4f4745 wireproto: add streams to frame-based protocol Gregory Szorc <gregory.szorc@gmail.com> parents: 37285 diff changeset	807
40130 5d44c4d1d516 wireprotov2: establish dedicated classes for input and output streams Gregory Szorc <gregory.szorc@gmail.com> parents: 40129 diff changeset	808 self._incomingstreams[frame.streamid] = inputstream(frame.streamid)
37288 9bfcbe4f4745 wireproto: add streams to frame-based protocol Gregory Szorc <gregory.szorc@gmail.com> parents: 37285 diff changeset	809
9bfcbe4f4745 wireproto: add streams to frame-based protocol Gregory Szorc <gregory.szorc@gmail.com> parents: 37285 diff changeset	810 if frame.streamflags & STREAM_FLAG_ENCODING_APPLIED:
9bfcbe4f4745 wireproto: add streams to frame-based protocol Gregory Szorc <gregory.szorc@gmail.com> parents: 37285 diff changeset	811 # TODO handle decoding frames
9bfcbe4f4745 wireproto: add streams to frame-based protocol Gregory Szorc <gregory.szorc@gmail.com> parents: 37285 diff changeset	812 self._state = 'errored'
9bfcbe4f4745 wireproto: add streams to frame-based protocol Gregory Szorc <gregory.szorc@gmail.com> parents: 37285 diff changeset	813 raise error.ProgrammingError('support for decoding stream payloads '
9bfcbe4f4745 wireproto: add streams to frame-based protocol Gregory Szorc <gregory.szorc@gmail.com> parents: 37285 diff changeset	814 'not yet implemented')
9bfcbe4f4745 wireproto: add streams to frame-based protocol Gregory Szorc <gregory.szorc@gmail.com> parents: 37285 diff changeset	815
9bfcbe4f4745 wireproto: add streams to frame-based protocol Gregory Szorc <gregory.szorc@gmail.com> parents: 37285 diff changeset	816 if frame.streamflags & STREAM_FLAG_END_STREAM:
9bfcbe4f4745 wireproto: add streams to frame-based protocol Gregory Szorc <gregory.szorc@gmail.com> parents: 37285 diff changeset	817 del self._incomingstreams[frame.streamid]
9bfcbe4f4745 wireproto: add streams to frame-based protocol Gregory Szorc <gregory.szorc@gmail.com> parents: 37285 diff changeset	818
37052 8c3c47362934 wireproto: implement basic frame reading and processing Gregory Szorc <gregory.szorc@gmail.com> parents: 37051 diff changeset	819 handlers = {
40126 327d40b94bed wireprotov2: handle sender protocol settings frames Gregory Szorc <gregory.szorc@gmail.com> parents: 40125 diff changeset	820 'initial': self._onframeinitial,
327d40b94bed wireprotov2: handle sender protocol settings frames Gregory Szorc <gregory.szorc@gmail.com> parents: 40125 diff changeset	821 'protocol-settings-receiving': self._onframeprotocolsettings,
37052 8c3c47362934 wireproto: implement basic frame reading and processing Gregory Szorc <gregory.szorc@gmail.com> parents: 37051 diff changeset	822 'idle': self._onframeidle,
37058 c5e9c3b47366 wireproto: support for receiving multiple requests Gregory Szorc <gregory.szorc@gmail.com> parents: 37057 diff changeset	823 'command-receiving': self._onframecommandreceiving,
37052 8c3c47362934 wireproto: implement basic frame reading and processing Gregory Szorc <gregory.szorc@gmail.com> parents: 37051 diff changeset	824 'errored': self._onframeerrored,
8c3c47362934 wireproto: implement basic frame reading and processing Gregory Szorc <gregory.szorc@gmail.com> parents: 37051 diff changeset	825 }
8c3c47362934 wireproto: implement basic frame reading and processing Gregory Szorc <gregory.szorc@gmail.com> parents: 37051 diff changeset	826
8c3c47362934 wireproto: implement basic frame reading and processing Gregory Szorc <gregory.szorc@gmail.com> parents: 37051 diff changeset	827 meth = handlers.get(self._state)
8c3c47362934 wireproto: implement basic frame reading and processing Gregory Szorc <gregory.szorc@gmail.com> parents: 37051 diff changeset	828 if not meth:
8c3c47362934 wireproto: implement basic frame reading and processing Gregory Szorc <gregory.szorc@gmail.com> parents: 37051 diff changeset	829 raise error.ProgrammingError('unhandled state: %s' % self._state)
8c3c47362934 wireproto: implement basic frame reading and processing Gregory Szorc <gregory.szorc@gmail.com> parents: 37051 diff changeset	830
37061 884a0c1604ad wireproto: define attr-based classes for representing frames Gregory Szorc <gregory.szorc@gmail.com> parents: 37060 diff changeset	831 return meth(frame)
37052 8c3c47362934 wireproto: implement basic frame reading and processing Gregory Szorc <gregory.szorc@gmail.com> parents: 37051 diff changeset	832
37724 deff7cf7eefd wireprotov2: change frame type and name for command response Gregory Szorc <gregory.szorc@gmail.com> parents: 37723 diff changeset	833 def oncommandresponseready(self, stream, requestid, data):
37055 61393f888dfe wireproto: define and implement responses in framing protocol Gregory Szorc <gregory.szorc@gmail.com> parents: 37052 diff changeset	834 """Signal that a bytes response is ready to be sent to the client.
61393f888dfe wireproto: define and implement responses in framing protocol Gregory Szorc <gregory.szorc@gmail.com> parents: 37052 diff changeset	835
61393f888dfe wireproto: define and implement responses in framing protocol Gregory Szorc <gregory.szorc@gmail.com> parents: 37052 diff changeset	836 The raw bytes response is passed as an argument.
61393f888dfe wireproto: define and implement responses in framing protocol Gregory Szorc <gregory.szorc@gmail.com> parents: 37052 diff changeset	837 """
37288 9bfcbe4f4745 wireproto: add streams to frame-based protocol Gregory Szorc <gregory.szorc@gmail.com> parents: 37285 diff changeset	838 ensureserverstream(stream)
9bfcbe4f4745 wireproto: add streams to frame-based protocol Gregory Szorc <gregory.szorc@gmail.com> parents: 37285 diff changeset	839
37063 39304dd63589 wireproto: explicitly track which requests are active Gregory Szorc <gregory.szorc@gmail.com> parents: 37062 diff changeset	840 def sendframes():
37724 deff7cf7eefd wireprotov2: change frame type and name for command response Gregory Szorc <gregory.szorc@gmail.com> parents: 37723 diff changeset	841 for frame in createcommandresponseframesfrombytes(stream, requestid,
deff7cf7eefd wireprotov2: change frame type and name for command response Gregory Szorc <gregory.szorc@gmail.com> parents: 37723 diff changeset	842 data):
37063 39304dd63589 wireproto: explicitly track which requests are active Gregory Szorc <gregory.szorc@gmail.com> parents: 37062 diff changeset	843 yield frame
39304dd63589 wireproto: explicitly track which requests are active Gregory Szorc <gregory.szorc@gmail.com> parents: 37062 diff changeset	844
39304dd63589 wireproto: explicitly track which requests are active Gregory Szorc <gregory.szorc@gmail.com> parents: 37062 diff changeset	845 self._activecommands.remove(requestid)
39304dd63589 wireproto: explicitly track which requests are active Gregory Szorc <gregory.szorc@gmail.com> parents: 37062 diff changeset	846
39304dd63589 wireproto: explicitly track which requests are active Gregory Szorc <gregory.szorc@gmail.com> parents: 37062 diff changeset	847 result = sendframes()
37056 861e9d37e56e wireproto: buffer output frames when in half duplex mode Gregory Szorc <gregory.szorc@gmail.com> parents: 37055 diff changeset	848
861e9d37e56e wireproto: buffer output frames when in half duplex mode Gregory Szorc <gregory.szorc@gmail.com> parents: 37055 diff changeset	849 if self._deferoutput:
37063 39304dd63589 wireproto: explicitly track which requests are active Gregory Szorc <gregory.szorc@gmail.com> parents: 37062 diff changeset	850 self._bufferedframegens.append(result)
37056 861e9d37e56e wireproto: buffer output frames when in half duplex mode Gregory Szorc <gregory.szorc@gmail.com> parents: 37055 diff changeset	851 return 'noop', {}
861e9d37e56e wireproto: buffer output frames when in half duplex mode Gregory Szorc <gregory.szorc@gmail.com> parents: 37055 diff changeset	852 else:
861e9d37e56e wireproto: buffer output frames when in half duplex mode Gregory Szorc <gregory.szorc@gmail.com> parents: 37055 diff changeset	853 return 'sendframes', {
37063 39304dd63589 wireproto: explicitly track which requests are active Gregory Szorc <gregory.szorc@gmail.com> parents: 37062 diff changeset	854 'framegen': result,
37056 861e9d37e56e wireproto: buffer output frames when in half duplex mode Gregory Szorc <gregory.szorc@gmail.com> parents: 37055 diff changeset	855 }
861e9d37e56e wireproto: buffer output frames when in half duplex mode Gregory Szorc <gregory.szorc@gmail.com> parents: 37055 diff changeset	856
39559 07b58266bce3 wireprotov2: implement commands as a generator of objects Gregory Szorc <gregory.szorc@gmail.com> parents: 39486 diff changeset	857 def oncommandresponsereadyobjects(self, stream, requestid, objs):
07b58266bce3 wireprotov2: implement commands as a generator of objects Gregory Szorc <gregory.szorc@gmail.com> parents: 39486 diff changeset	858 """Signal that objects are ready to be sent to the client.
07b58266bce3 wireprotov2: implement commands as a generator of objects Gregory Szorc <gregory.szorc@gmail.com> parents: 39486 diff changeset	859
07b58266bce3 wireprotov2: implement commands as a generator of objects Gregory Szorc <gregory.szorc@gmail.com> parents: 39486 diff changeset	860 ``objs`` is an iterable of objects (typically a generator) that will
07b58266bce3 wireprotov2: implement commands as a generator of objects Gregory Szorc <gregory.szorc@gmail.com> parents: 39486 diff changeset	861 be encoded via CBOR and added to frames, which will be sent to the
07b58266bce3 wireprotov2: implement commands as a generator of objects Gregory Szorc <gregory.szorc@gmail.com> parents: 39486 diff changeset	862 client.
07b58266bce3 wireprotov2: implement commands as a generator of objects Gregory Szorc <gregory.szorc@gmail.com> parents: 39486 diff changeset	863 """
37728 564a3eec6e63 wireprotov2: add support for more response types Gregory Szorc <gregory.szorc@gmail.com> parents: 37726 diff changeset	864 ensureserverstream(stream)
564a3eec6e63 wireprotov2: add support for more response types Gregory Szorc <gregory.szorc@gmail.com> parents: 37726 diff changeset	865
39559 07b58266bce3 wireprotov2: implement commands as a generator of objects Gregory Szorc <gregory.szorc@gmail.com> parents: 39486 diff changeset	866 # We need to take care over exception handling. Uncaught exceptions
07b58266bce3 wireprotov2: implement commands as a generator of objects Gregory Szorc <gregory.szorc@gmail.com> parents: 39486 diff changeset	867 # when generating frames could lead to premature end of the frame
07b58266bce3 wireprotov2: implement commands as a generator of objects Gregory Szorc <gregory.szorc@gmail.com> parents: 39486 diff changeset	868 # stream and the possibility of the server or client process getting
07b58266bce3 wireprotov2: implement commands as a generator of objects Gregory Szorc <gregory.szorc@gmail.com> parents: 39486 diff changeset	869 # in a bad state.
07b58266bce3 wireprotov2: implement commands as a generator of objects Gregory Szorc <gregory.szorc@gmail.com> parents: 39486 diff changeset	870 #
07b58266bce3 wireprotov2: implement commands as a generator of objects Gregory Szorc <gregory.szorc@gmail.com> parents: 39486 diff changeset	871 # Keep in mind that if ``objs`` is a generator, advancing it could
07b58266bce3 wireprotov2: implement commands as a generator of objects Gregory Szorc <gregory.szorc@gmail.com> parents: 39486 diff changeset	872 # raise exceptions that originated in e.g. wire protocol command
07b58266bce3 wireprotov2: implement commands as a generator of objects Gregory Szorc <gregory.szorc@gmail.com> parents: 39486 diff changeset	873 # functions. That is why we differentiate between exceptions raised
07b58266bce3 wireprotov2: implement commands as a generator of objects Gregory Szorc <gregory.szorc@gmail.com> parents: 39486 diff changeset	874 # when iterating versus other exceptions that occur.
07b58266bce3 wireprotov2: implement commands as a generator of objects Gregory Szorc <gregory.szorc@gmail.com> parents: 39486 diff changeset	875 #
07b58266bce3 wireprotov2: implement commands as a generator of objects Gregory Szorc <gregory.szorc@gmail.com> parents: 39486 diff changeset	876 # In all cases, when the function finishes, the request is fully
07b58266bce3 wireprotov2: implement commands as a generator of objects Gregory Szorc <gregory.szorc@gmail.com> parents: 39486 diff changeset	877 # handled and no new frames for it should be seen.
07b58266bce3 wireprotov2: implement commands as a generator of objects Gregory Szorc <gregory.szorc@gmail.com> parents: 39486 diff changeset	878
37728 564a3eec6e63 wireprotov2: add support for more response types Gregory Szorc <gregory.szorc@gmail.com> parents: 37726 diff changeset	879 def sendframes():
39559 07b58266bce3 wireprotov2: implement commands as a generator of objects Gregory Szorc <gregory.szorc@gmail.com> parents: 39486 diff changeset	880 emitted = False
40025 b099e6032f38 wireprotov2: server support for sending content redirects Gregory Szorc <gregory.szorc@gmail.com> parents: 40024 diff changeset	881 alternatelocationsent = False
39560 84bf6ded9317 wireprotoframing: buffer emitted data to reduce frame count Gregory Szorc <gregory.szorc@gmail.com> parents: 39559 diff changeset	882 emitter = bufferingcommandresponseemitter(stream, requestid)
39559 07b58266bce3 wireprotov2: implement commands as a generator of objects Gregory Szorc <gregory.szorc@gmail.com> parents: 39486 diff changeset	883 while True:
07b58266bce3 wireprotov2: implement commands as a generator of objects Gregory Szorc <gregory.szorc@gmail.com> parents: 39486 diff changeset	884 try:
07b58266bce3 wireprotov2: implement commands as a generator of objects Gregory Szorc <gregory.szorc@gmail.com> parents: 39486 diff changeset	885 o = next(objs)
07b58266bce3 wireprotov2: implement commands as a generator of objects Gregory Szorc <gregory.szorc@gmail.com> parents: 39486 diff changeset	886 except StopIteration:
39560 84bf6ded9317 wireprotoframing: buffer emitted data to reduce frame count Gregory Szorc <gregory.szorc@gmail.com> parents: 39559 diff changeset	887 for frame in emitter.send(None):
84bf6ded9317 wireprotoframing: buffer emitted data to reduce frame count Gregory Szorc <gregory.szorc@gmail.com> parents: 39559 diff changeset	888 yield frame
84bf6ded9317 wireprotoframing: buffer emitted data to reduce frame count Gregory Szorc <gregory.szorc@gmail.com> parents: 39559 diff changeset	889
39559 07b58266bce3 wireprotov2: implement commands as a generator of objects Gregory Szorc <gregory.szorc@gmail.com> parents: 39486 diff changeset	890 if emitted:
07b58266bce3 wireprotov2: implement commands as a generator of objects Gregory Szorc <gregory.szorc@gmail.com> parents: 39486 diff changeset	891 yield createcommandresponseeosframe(stream, requestid)
07b58266bce3 wireprotov2: implement commands as a generator of objects Gregory Szorc <gregory.szorc@gmail.com> parents: 39486 diff changeset	892 break
07b58266bce3 wireprotov2: implement commands as a generator of objects Gregory Szorc <gregory.szorc@gmail.com> parents: 39486 diff changeset	893
07b58266bce3 wireprotov2: implement commands as a generator of objects Gregory Szorc <gregory.szorc@gmail.com> parents: 39486 diff changeset	894 except error.WireprotoCommandError as e:
07b58266bce3 wireprotov2: implement commands as a generator of objects Gregory Szorc <gregory.szorc@gmail.com> parents: 39486 diff changeset	895 for frame in createcommanderrorresponse(
07b58266bce3 wireprotov2: implement commands as a generator of objects Gregory Szorc <gregory.szorc@gmail.com> parents: 39486 diff changeset	896 stream, requestid, e.message, e.messageargs):
07b58266bce3 wireprotov2: implement commands as a generator of objects Gregory Szorc <gregory.szorc@gmail.com> parents: 39486 diff changeset	897 yield frame
07b58266bce3 wireprotov2: implement commands as a generator of objects Gregory Szorc <gregory.szorc@gmail.com> parents: 39486 diff changeset	898 break
07b58266bce3 wireprotov2: implement commands as a generator of objects Gregory Szorc <gregory.szorc@gmail.com> parents: 39486 diff changeset	899
07b58266bce3 wireprotov2: implement commands as a generator of objects Gregory Szorc <gregory.szorc@gmail.com> parents: 39486 diff changeset	900 except Exception as e:
39838 bce1c1af7518 py3: cast exception to bytes Gregory Szorc <gregory.szorc@gmail.com> parents: 39560 diff changeset	901 for frame in createerrorframe(
bce1c1af7518 py3: cast exception to bytes Gregory Szorc <gregory.szorc@gmail.com> parents: 39560 diff changeset	902 stream, requestid, '%s' % stringutil.forcebytestr(e),
bce1c1af7518 py3: cast exception to bytes Gregory Szorc <gregory.szorc@gmail.com> parents: 39560 diff changeset	903 errtype='server'):
bce1c1af7518 py3: cast exception to bytes Gregory Szorc <gregory.szorc@gmail.com> parents: 39560 diff changeset	904
39559 07b58266bce3 wireprotov2: implement commands as a generator of objects Gregory Szorc <gregory.szorc@gmail.com> parents: 39486 diff changeset	905 yield frame
07b58266bce3 wireprotov2: implement commands as a generator of objects Gregory Szorc <gregory.szorc@gmail.com> parents: 39486 diff changeset	906
07b58266bce3 wireprotov2: implement commands as a generator of objects Gregory Szorc <gregory.szorc@gmail.com> parents: 39486 diff changeset	907 break
07b58266bce3 wireprotov2: implement commands as a generator of objects Gregory Szorc <gregory.szorc@gmail.com> parents: 39486 diff changeset	908
07b58266bce3 wireprotov2: implement commands as a generator of objects Gregory Szorc <gregory.szorc@gmail.com> parents: 39486 diff changeset	909 try:
40025 b099e6032f38 wireprotov2: server support for sending content redirects Gregory Szorc <gregory.szorc@gmail.com> parents: 40024 diff changeset	910 # Alternate location responses can only be the first and
b099e6032f38 wireprotov2: server support for sending content redirects Gregory Szorc <gregory.szorc@gmail.com> parents: 40024 diff changeset	911 # only object in the output stream.
b099e6032f38 wireprotov2: server support for sending content redirects Gregory Szorc <gregory.szorc@gmail.com> parents: 40024 diff changeset	912 if isinstance(o, wireprototypes.alternatelocationresponse):
b099e6032f38 wireprotov2: server support for sending content redirects Gregory Szorc <gregory.szorc@gmail.com> parents: 40024 diff changeset	913 if emitted:
b099e6032f38 wireprotov2: server support for sending content redirects Gregory Szorc <gregory.szorc@gmail.com> parents: 40024 diff changeset	914 raise error.ProgrammingError(
b099e6032f38 wireprotov2: server support for sending content redirects Gregory Szorc <gregory.szorc@gmail.com> parents: 40024 diff changeset	915 'alternatelocationresponse seen after initial '
b099e6032f38 wireprotov2: server support for sending content redirects Gregory Szorc <gregory.szorc@gmail.com> parents: 40024 diff changeset	916 'output object')
b099e6032f38 wireprotov2: server support for sending content redirects Gregory Szorc <gregory.szorc@gmail.com> parents: 40024 diff changeset	917
b099e6032f38 wireprotov2: server support for sending content redirects Gregory Szorc <gregory.szorc@gmail.com> parents: 40024 diff changeset	918 yield createalternatelocationresponseframe(
b099e6032f38 wireprotov2: server support for sending content redirects Gregory Szorc <gregory.szorc@gmail.com> parents: 40024 diff changeset	919 stream, requestid, o)
b099e6032f38 wireprotov2: server support for sending content redirects Gregory Szorc <gregory.szorc@gmail.com> parents: 40024 diff changeset	920
b099e6032f38 wireprotov2: server support for sending content redirects Gregory Szorc <gregory.szorc@gmail.com> parents: 40024 diff changeset	921 alternatelocationsent = True
b099e6032f38 wireprotov2: server support for sending content redirects Gregory Szorc <gregory.szorc@gmail.com> parents: 40024 diff changeset	922 emitted = True
b099e6032f38 wireprotov2: server support for sending content redirects Gregory Szorc <gregory.szorc@gmail.com> parents: 40024 diff changeset	923 continue
b099e6032f38 wireprotov2: server support for sending content redirects Gregory Szorc <gregory.szorc@gmail.com> parents: 40024 diff changeset	924
b099e6032f38 wireprotov2: server support for sending content redirects Gregory Szorc <gregory.szorc@gmail.com> parents: 40024 diff changeset	925 if alternatelocationsent:
b099e6032f38 wireprotov2: server support for sending content redirects Gregory Szorc <gregory.szorc@gmail.com> parents: 40024 diff changeset	926 raise error.ProgrammingError(
b099e6032f38 wireprotov2: server support for sending content redirects Gregory Szorc <gregory.szorc@gmail.com> parents: 40024 diff changeset	927 'object follows alternatelocationresponse')
b099e6032f38 wireprotov2: server support for sending content redirects Gregory Szorc <gregory.szorc@gmail.com> parents: 40024 diff changeset	928
39559 07b58266bce3 wireprotov2: implement commands as a generator of objects Gregory Szorc <gregory.szorc@gmail.com> parents: 39486 diff changeset	929 if not emitted:
07b58266bce3 wireprotov2: implement commands as a generator of objects Gregory Szorc <gregory.szorc@gmail.com> parents: 39486 diff changeset	930 yield createcommandresponseokframe(stream, requestid)
07b58266bce3 wireprotov2: implement commands as a generator of objects Gregory Szorc <gregory.szorc@gmail.com> parents: 39486 diff changeset	931 emitted = True
07b58266bce3 wireprotov2: implement commands as a generator of objects Gregory Szorc <gregory.szorc@gmail.com> parents: 39486 diff changeset	932
40020 ed919b90acda wireprotov2: define type to represent pre-encoded object Gregory Szorc <gregory.szorc@gmail.com> parents: 39838 diff changeset	933 # Objects emitted by command functions can be serializable
ed919b90acda wireprotov2: define type to represent pre-encoded object Gregory Szorc <gregory.szorc@gmail.com> parents: 39838 diff changeset	934 # data structures or special types.
ed919b90acda wireprotov2: define type to represent pre-encoded object Gregory Szorc <gregory.szorc@gmail.com> parents: 39838 diff changeset	935 # TODO consider extracting the content normalization to a
ed919b90acda wireprotov2: define type to represent pre-encoded object Gregory Szorc <gregory.szorc@gmail.com> parents: 39838 diff changeset	936 # standalone function, as it may be useful for e.g. cachers.
ed919b90acda wireprotov2: define type to represent pre-encoded object Gregory Szorc <gregory.szorc@gmail.com> parents: 39838 diff changeset	937
ed919b90acda wireprotov2: define type to represent pre-encoded object Gregory Szorc <gregory.szorc@gmail.com> parents: 39838 diff changeset	938 # A pre-encoded object is sent directly to the emitter.
ed919b90acda wireprotov2: define type to represent pre-encoded object Gregory Szorc <gregory.szorc@gmail.com> parents: 39838 diff changeset	939 if isinstance(o, wireprototypes.encodedresponse):
ed919b90acda wireprotov2: define type to represent pre-encoded object Gregory Szorc <gregory.szorc@gmail.com> parents: 39838 diff changeset	940 for frame in emitter.send(o.data):
39560 84bf6ded9317 wireprotoframing: buffer emitted data to reduce frame count Gregory Szorc <gregory.szorc@gmail.com> parents: 39559 diff changeset	941 yield frame
84bf6ded9317 wireprotoframing: buffer emitted data to reduce frame count Gregory Szorc <gregory.szorc@gmail.com> parents: 39559 diff changeset	942
40020 ed919b90acda wireprotov2: define type to represent pre-encoded object Gregory Szorc <gregory.szorc@gmail.com> parents: 39838 diff changeset	943 # A regular object is CBOR encoded.
ed919b90acda wireprotov2: define type to represent pre-encoded object Gregory Szorc <gregory.szorc@gmail.com> parents: 39838 diff changeset	944 else:
ed919b90acda wireprotov2: define type to represent pre-encoded object Gregory Szorc <gregory.szorc@gmail.com> parents: 39838 diff changeset	945 for chunk in cborutil.streamencode(o):
ed919b90acda wireprotov2: define type to represent pre-encoded object Gregory Szorc <gregory.szorc@gmail.com> parents: 39838 diff changeset	946 for frame in emitter.send(chunk):
ed919b90acda wireprotov2: define type to represent pre-encoded object Gregory Szorc <gregory.szorc@gmail.com> parents: 39838 diff changeset	947 yield frame
ed919b90acda wireprotov2: define type to represent pre-encoded object Gregory Szorc <gregory.szorc@gmail.com> parents: 39838 diff changeset	948
39559 07b58266bce3 wireprotov2: implement commands as a generator of objects Gregory Szorc <gregory.szorc@gmail.com> parents: 39486 diff changeset	949 except Exception as e:
07b58266bce3 wireprotov2: implement commands as a generator of objects Gregory Szorc <gregory.szorc@gmail.com> parents: 39486 diff changeset	950 for frame in createerrorframe(stream, requestid,
07b58266bce3 wireprotov2: implement commands as a generator of objects Gregory Szorc <gregory.szorc@gmail.com> parents: 39486 diff changeset	951 '%s' % e,
07b58266bce3 wireprotov2: implement commands as a generator of objects Gregory Szorc <gregory.szorc@gmail.com> parents: 39486 diff changeset	952 errtype='server'):
07b58266bce3 wireprotov2: implement commands as a generator of objects Gregory Szorc <gregory.szorc@gmail.com> parents: 39486 diff changeset	953 yield frame
07b58266bce3 wireprotov2: implement commands as a generator of objects Gregory Szorc <gregory.szorc@gmail.com> parents: 39486 diff changeset	954
07b58266bce3 wireprotov2: implement commands as a generator of objects Gregory Szorc <gregory.szorc@gmail.com> parents: 39486 diff changeset	955 break
37728 564a3eec6e63 wireprotov2: add support for more response types Gregory Szorc <gregory.szorc@gmail.com> parents: 37726 diff changeset	956
564a3eec6e63 wireprotov2: add support for more response types Gregory Szorc <gregory.szorc@gmail.com> parents: 37726 diff changeset	957 self._activecommands.remove(requestid)
564a3eec6e63 wireprotov2: add support for more response types Gregory Szorc <gregory.szorc@gmail.com> parents: 37726 diff changeset	958
564a3eec6e63 wireprotov2: add support for more response types Gregory Szorc <gregory.szorc@gmail.com> parents: 37726 diff changeset	959 return self._handlesendframes(sendframes())
564a3eec6e63 wireprotov2: add support for more response types Gregory Szorc <gregory.szorc@gmail.com> parents: 37726 diff changeset	960
37056 861e9d37e56e wireproto: buffer output frames when in half duplex mode Gregory Szorc <gregory.szorc@gmail.com> parents: 37055 diff changeset	961 def oninputeof(self):
861e9d37e56e wireproto: buffer output frames when in half duplex mode Gregory Szorc <gregory.szorc@gmail.com> parents: 37055 diff changeset	962 """Signals that end of input has been received.
861e9d37e56e wireproto: buffer output frames when in half duplex mode Gregory Szorc <gregory.szorc@gmail.com> parents: 37055 diff changeset	963
861e9d37e56e wireproto: buffer output frames when in half duplex mode Gregory Szorc <gregory.szorc@gmail.com> parents: 37055 diff changeset	964 No more frames will be received. All pending activity should be
861e9d37e56e wireproto: buffer output frames when in half duplex mode Gregory Szorc <gregory.szorc@gmail.com> parents: 37055 diff changeset	965 completed.
861e9d37e56e wireproto: buffer output frames when in half duplex mode Gregory Szorc <gregory.szorc@gmail.com> parents: 37055 diff changeset	966 """
37058 c5e9c3b47366 wireproto: support for receiving multiple requests Gregory Szorc <gregory.szorc@gmail.com> parents: 37057 diff changeset	967 # TODO should we do anything about in-flight commands?
c5e9c3b47366 wireproto: support for receiving multiple requests Gregory Szorc <gregory.szorc@gmail.com> parents: 37057 diff changeset	968
37056 861e9d37e56e wireproto: buffer output frames when in half duplex mode Gregory Szorc <gregory.szorc@gmail.com> parents: 37055 diff changeset	969 if not self._deferoutput or not self._bufferedframegens:
861e9d37e56e wireproto: buffer output frames when in half duplex mode Gregory Szorc <gregory.szorc@gmail.com> parents: 37055 diff changeset	970 return 'noop', {}
861e9d37e56e wireproto: buffer output frames when in half duplex mode Gregory Szorc <gregory.szorc@gmail.com> parents: 37055 diff changeset	971
861e9d37e56e wireproto: buffer output frames when in half duplex mode Gregory Szorc <gregory.szorc@gmail.com> parents: 37055 diff changeset	972 # If we buffered all our responses, emit those.
861e9d37e56e wireproto: buffer output frames when in half duplex mode Gregory Szorc <gregory.szorc@gmail.com> parents: 37055 diff changeset	973 def makegen():
861e9d37e56e wireproto: buffer output frames when in half duplex mode Gregory Szorc <gregory.szorc@gmail.com> parents: 37055 diff changeset	974 for gen in self._bufferedframegens:
861e9d37e56e wireproto: buffer output frames when in half duplex mode Gregory Szorc <gregory.szorc@gmail.com> parents: 37055 diff changeset	975 for frame in gen:
861e9d37e56e wireproto: buffer output frames when in half duplex mode Gregory Szorc <gregory.szorc@gmail.com> parents: 37055 diff changeset	976 yield frame
861e9d37e56e wireproto: buffer output frames when in half duplex mode Gregory Szorc <gregory.szorc@gmail.com> parents: 37055 diff changeset	977
37055 61393f888dfe wireproto: define and implement responses in framing protocol Gregory Szorc <gregory.szorc@gmail.com> parents: 37052 diff changeset	978 return 'sendframes', {
37056 861e9d37e56e wireproto: buffer output frames when in half duplex mode Gregory Szorc <gregory.szorc@gmail.com> parents: 37055 diff changeset	979 'framegen': makegen(),
37055 61393f888dfe wireproto: define and implement responses in framing protocol Gregory Szorc <gregory.szorc@gmail.com> parents: 37052 diff changeset	980 }
61393f888dfe wireproto: define and implement responses in framing protocol Gregory Szorc <gregory.szorc@gmail.com> parents: 37052 diff changeset	981
37728 564a3eec6e63 wireprotov2: add support for more response types Gregory Szorc <gregory.szorc@gmail.com> parents: 37726 diff changeset	982 def _handlesendframes(self, framegen):
564a3eec6e63 wireprotov2: add support for more response types Gregory Szorc <gregory.szorc@gmail.com> parents: 37726 diff changeset	983 if self._deferoutput:
564a3eec6e63 wireprotov2: add support for more response types Gregory Szorc <gregory.szorc@gmail.com> parents: 37726 diff changeset	984 self._bufferedframegens.append(framegen)
564a3eec6e63 wireprotov2: add support for more response types Gregory Szorc <gregory.szorc@gmail.com> parents: 37726 diff changeset	985 return 'noop', {}
564a3eec6e63 wireprotov2: add support for more response types Gregory Szorc <gregory.szorc@gmail.com> parents: 37726 diff changeset	986 else:
564a3eec6e63 wireprotov2: add support for more response types Gregory Szorc <gregory.szorc@gmail.com> parents: 37726 diff changeset	987 return 'sendframes', {
564a3eec6e63 wireprotov2: add support for more response types Gregory Szorc <gregory.szorc@gmail.com> parents: 37726 diff changeset	988 'framegen': framegen,
564a3eec6e63 wireprotov2: add support for more response types Gregory Szorc <gregory.szorc@gmail.com> parents: 37726 diff changeset	989 }
564a3eec6e63 wireprotov2: add support for more response types Gregory Szorc <gregory.szorc@gmail.com> parents: 37726 diff changeset	990
37726 0c184ca594bb wireprotov2: change behavior of error frame Gregory Szorc <gregory.szorc@gmail.com> parents: 37725 diff changeset	991 def onservererror(self, stream, requestid, msg):
37288 9bfcbe4f4745 wireproto: add streams to frame-based protocol Gregory Szorc <gregory.szorc@gmail.com> parents: 37285 diff changeset	992 ensureserverstream(stream)
9bfcbe4f4745 wireproto: add streams to frame-based protocol Gregory Szorc <gregory.szorc@gmail.com> parents: 37285 diff changeset	993
37728 564a3eec6e63 wireprotov2: add support for more response types Gregory Szorc <gregory.szorc@gmail.com> parents: 37726 diff changeset	994 def sendframes():
564a3eec6e63 wireprotov2: add support for more response types Gregory Szorc <gregory.szorc@gmail.com> parents: 37726 diff changeset	995 for frame in createerrorframe(stream, requestid, msg,
564a3eec6e63 wireprotov2: add support for more response types Gregory Szorc <gregory.szorc@gmail.com> parents: 37726 diff changeset	996 errtype='server'):
564a3eec6e63 wireprotov2: add support for more response types Gregory Szorc <gregory.szorc@gmail.com> parents: 37726 diff changeset	997 yield frame
564a3eec6e63 wireprotov2: add support for more response types Gregory Szorc <gregory.szorc@gmail.com> parents: 37726 diff changeset	998
564a3eec6e63 wireprotov2: add support for more response types Gregory Szorc <gregory.szorc@gmail.com> parents: 37726 diff changeset	999 self._activecommands.remove(requestid)
564a3eec6e63 wireprotov2: add support for more response types Gregory Szorc <gregory.szorc@gmail.com> parents: 37726 diff changeset	1000
564a3eec6e63 wireprotov2: add support for more response types Gregory Szorc <gregory.szorc@gmail.com> parents: 37726 diff changeset	1001 return self._handlesendframes(sendframes())
564a3eec6e63 wireprotov2: add support for more response types Gregory Szorc <gregory.szorc@gmail.com> parents: 37726 diff changeset	1002
564a3eec6e63 wireprotov2: add support for more response types Gregory Szorc <gregory.szorc@gmail.com> parents: 37726 diff changeset	1003 def oncommanderror(self, stream, requestid, message, args=None):
564a3eec6e63 wireprotov2: add support for more response types Gregory Szorc <gregory.szorc@gmail.com> parents: 37726 diff changeset	1004 """Called when a command encountered an error before sending output."""
564a3eec6e63 wireprotov2: add support for more response types Gregory Szorc <gregory.szorc@gmail.com> parents: 37726 diff changeset	1005 ensureserverstream(stream)
564a3eec6e63 wireprotov2: add support for more response types Gregory Szorc <gregory.szorc@gmail.com> parents: 37726 diff changeset	1006
564a3eec6e63 wireprotov2: add support for more response types Gregory Szorc <gregory.szorc@gmail.com> parents: 37726 diff changeset	1007 def sendframes():
564a3eec6e63 wireprotov2: add support for more response types Gregory Szorc <gregory.szorc@gmail.com> parents: 37726 diff changeset	1008 for frame in createcommanderrorresponse(stream, requestid, message,
564a3eec6e63 wireprotov2: add support for more response types Gregory Szorc <gregory.szorc@gmail.com> parents: 37726 diff changeset	1009 args):
564a3eec6e63 wireprotov2: add support for more response types Gregory Szorc <gregory.szorc@gmail.com> parents: 37726 diff changeset	1010 yield frame
564a3eec6e63 wireprotov2: add support for more response types Gregory Szorc <gregory.szorc@gmail.com> parents: 37726 diff changeset	1011
564a3eec6e63 wireprotov2: add support for more response types Gregory Szorc <gregory.szorc@gmail.com> parents: 37726 diff changeset	1012 self._activecommands.remove(requestid)
564a3eec6e63 wireprotov2: add support for more response types Gregory Szorc <gregory.szorc@gmail.com> parents: 37726 diff changeset	1013
564a3eec6e63 wireprotov2: add support for more response types Gregory Szorc <gregory.szorc@gmail.com> parents: 37726 diff changeset	1014 return self._handlesendframes(sendframes())
37055 61393f888dfe wireproto: define and implement responses in framing protocol Gregory Szorc <gregory.szorc@gmail.com> parents: 37052 diff changeset	1015
37289 5fadc63ac99f wireproto: explicit API to create outgoing streams Gregory Szorc <gregory.szorc@gmail.com> parents: 37288 diff changeset	1016 def makeoutputstream(self):
5fadc63ac99f wireproto: explicit API to create outgoing streams Gregory Szorc <gregory.szorc@gmail.com> parents: 37288 diff changeset	1017 """Create a stream to be used for sending data to the client."""
5fadc63ac99f wireproto: explicit API to create outgoing streams Gregory Szorc <gregory.szorc@gmail.com> parents: 37288 diff changeset	1018 streamid = self._nextoutgoingstreamid
5fadc63ac99f wireproto: explicit API to create outgoing streams Gregory Szorc <gregory.szorc@gmail.com> parents: 37288 diff changeset	1019 self._nextoutgoingstreamid += 2
5fadc63ac99f wireproto: explicit API to create outgoing streams Gregory Szorc <gregory.szorc@gmail.com> parents: 37288 diff changeset	1020
40130 5d44c4d1d516 wireprotov2: establish dedicated classes for input and output streams Gregory Szorc <gregory.szorc@gmail.com> parents: 40129 diff changeset	1021 s = outputstream(streamid)
37289 5fadc63ac99f wireproto: explicit API to create outgoing streams Gregory Szorc <gregory.szorc@gmail.com> parents: 37288 diff changeset	1022 self._outgoingstreams[streamid] = s
5fadc63ac99f wireproto: explicit API to create outgoing streams Gregory Szorc <gregory.szorc@gmail.com> parents: 37288 diff changeset	1023
5fadc63ac99f wireproto: explicit API to create outgoing streams Gregory Szorc <gregory.szorc@gmail.com> parents: 37288 diff changeset	1024 return s
5fadc63ac99f wireproto: explicit API to create outgoing streams Gregory Szorc <gregory.szorc@gmail.com> parents: 37288 diff changeset	1025
37052 8c3c47362934 wireproto: implement basic frame reading and processing Gregory Szorc <gregory.szorc@gmail.com> parents: 37051 diff changeset	1026 def _makeerrorresult(self, msg):
8c3c47362934 wireproto: implement basic frame reading and processing Gregory Szorc <gregory.szorc@gmail.com> parents: 37051 diff changeset	1027 return 'error', {
8c3c47362934 wireproto: implement basic frame reading and processing Gregory Szorc <gregory.szorc@gmail.com> parents: 37051 diff changeset	1028 'message': msg,
8c3c47362934 wireproto: implement basic frame reading and processing Gregory Szorc <gregory.szorc@gmail.com> parents: 37051 diff changeset	1029 }
8c3c47362934 wireproto: implement basic frame reading and processing Gregory Szorc <gregory.szorc@gmail.com> parents: 37051 diff changeset	1030
37058 c5e9c3b47366 wireproto: support for receiving multiple requests Gregory Szorc <gregory.szorc@gmail.com> parents: 37057 diff changeset	1031 def _makeruncommandresult(self, requestid):
c5e9c3b47366 wireproto: support for receiving multiple requests Gregory Szorc <gregory.szorc@gmail.com> parents: 37057 diff changeset	1032 entry = self._receivingcommands[requestid]
37292 3d0e2cd86e05 wireproto: use CBOR for command requests Gregory Szorc <gregory.szorc@gmail.com> parents: 37291 diff changeset	1033
3d0e2cd86e05 wireproto: use CBOR for command requests Gregory Szorc <gregory.szorc@gmail.com> parents: 37291 diff changeset	1034 if not entry['requestdone']:
3d0e2cd86e05 wireproto: use CBOR for command requests Gregory Szorc <gregory.szorc@gmail.com> parents: 37291 diff changeset	1035 self._state = 'errored'
3d0e2cd86e05 wireproto: use CBOR for command requests Gregory Szorc <gregory.szorc@gmail.com> parents: 37291 diff changeset	1036 raise error.ProgrammingError('should not be called without '
3d0e2cd86e05 wireproto: use CBOR for command requests Gregory Szorc <gregory.szorc@gmail.com> parents: 37291 diff changeset	1037 'requestdone set')
3d0e2cd86e05 wireproto: use CBOR for command requests Gregory Szorc <gregory.szorc@gmail.com> parents: 37291 diff changeset	1038
37058 c5e9c3b47366 wireproto: support for receiving multiple requests Gregory Szorc <gregory.szorc@gmail.com> parents: 37057 diff changeset	1039 del self._receivingcommands[requestid]
c5e9c3b47366 wireproto: support for receiving multiple requests Gregory Szorc <gregory.szorc@gmail.com> parents: 37057 diff changeset	1040
c5e9c3b47366 wireproto: support for receiving multiple requests Gregory Szorc <gregory.szorc@gmail.com> parents: 37057 diff changeset	1041 if self._receivingcommands:
c5e9c3b47366 wireproto: support for receiving multiple requests Gregory Szorc <gregory.szorc@gmail.com> parents: 37057 diff changeset	1042 self._state = 'command-receiving'
c5e9c3b47366 wireproto: support for receiving multiple requests Gregory Szorc <gregory.szorc@gmail.com> parents: 37057 diff changeset	1043 else:
c5e9c3b47366 wireproto: support for receiving multiple requests Gregory Szorc <gregory.szorc@gmail.com> parents: 37057 diff changeset	1044 self._state = 'idle'
c5e9c3b47366 wireproto: support for receiving multiple requests Gregory Szorc <gregory.szorc@gmail.com> parents: 37057 diff changeset	1045
37292 3d0e2cd86e05 wireproto: use CBOR for command requests Gregory Szorc <gregory.szorc@gmail.com> parents: 37291 diff changeset	1046 # Decode the payloads as CBOR.
3d0e2cd86e05 wireproto: use CBOR for command requests Gregory Szorc <gregory.szorc@gmail.com> parents: 37291 diff changeset	1047 entry['payload'].seek(0)
39446 36f487a332ad wireprotoframing: use our CBOR module Gregory Szorc <gregory.szorc@gmail.com> parents: 37728 diff changeset	1048 request = cborutil.decodeall(entry['payload'].getvalue())[0]
37292 3d0e2cd86e05 wireproto: use CBOR for command requests Gregory Szorc <gregory.szorc@gmail.com> parents: 37291 diff changeset	1049
3d0e2cd86e05 wireproto: use CBOR for command requests Gregory Szorc <gregory.szorc@gmail.com> parents: 37291 diff changeset	1050 if b'name' not in request:
3d0e2cd86e05 wireproto: use CBOR for command requests Gregory Szorc <gregory.szorc@gmail.com> parents: 37291 diff changeset	1051 self._state = 'errored'
3d0e2cd86e05 wireproto: use CBOR for command requests Gregory Szorc <gregory.szorc@gmail.com> parents: 37291 diff changeset	1052 return self._makeerrorresult(
3d0e2cd86e05 wireproto: use CBOR for command requests Gregory Szorc <gregory.szorc@gmail.com> parents: 37291 diff changeset	1053 _('command request missing "name" field'))
3d0e2cd86e05 wireproto: use CBOR for command requests Gregory Szorc <gregory.szorc@gmail.com> parents: 37291 diff changeset	1054
3d0e2cd86e05 wireproto: use CBOR for command requests Gregory Szorc <gregory.szorc@gmail.com> parents: 37291 diff changeset	1055 if b'args' not in request:
3d0e2cd86e05 wireproto: use CBOR for command requests Gregory Szorc <gregory.szorc@gmail.com> parents: 37291 diff changeset	1056 request[b'args'] = {}
3d0e2cd86e05 wireproto: use CBOR for command requests Gregory Szorc <gregory.szorc@gmail.com> parents: 37291 diff changeset	1057
37063 39304dd63589 wireproto: explicitly track which requests are active Gregory Szorc <gregory.szorc@gmail.com> parents: 37062 diff changeset	1058 assert requestid not in self._activecommands
39304dd63589 wireproto: explicitly track which requests are active Gregory Szorc <gregory.szorc@gmail.com> parents: 37062 diff changeset	1059 self._activecommands.add(requestid)
39304dd63589 wireproto: explicitly track which requests are active Gregory Szorc <gregory.szorc@gmail.com> parents: 37062 diff changeset	1060
37052 8c3c47362934 wireproto: implement basic frame reading and processing Gregory Szorc <gregory.szorc@gmail.com> parents: 37051 diff changeset	1061 return 'runcommand', {
37058 c5e9c3b47366 wireproto: support for receiving multiple requests Gregory Szorc <gregory.szorc@gmail.com> parents: 37057 diff changeset	1062 'requestid': requestid,
37292 3d0e2cd86e05 wireproto: use CBOR for command requests Gregory Szorc <gregory.szorc@gmail.com> parents: 37291 diff changeset	1063 'command': request[b'name'],
3d0e2cd86e05 wireproto: use CBOR for command requests Gregory Szorc <gregory.szorc@gmail.com> parents: 37291 diff changeset	1064 'args': request[b'args'],
40025 b099e6032f38 wireprotov2: server support for sending content redirects Gregory Szorc <gregory.szorc@gmail.com> parents: 40024 diff changeset	1065 'redirect': request.get(b'redirect'),
37058 c5e9c3b47366 wireproto: support for receiving multiple requests Gregory Szorc <gregory.szorc@gmail.com> parents: 37057 diff changeset	1066 'data': entry['data'].getvalue() if entry['data'] else None,
37052 8c3c47362934 wireproto: implement basic frame reading and processing Gregory Szorc <gregory.szorc@gmail.com> parents: 37051 diff changeset	1067 }
8c3c47362934 wireproto: implement basic frame reading and processing Gregory Szorc <gregory.szorc@gmail.com> parents: 37051 diff changeset	1068
8c3c47362934 wireproto: implement basic frame reading and processing Gregory Szorc <gregory.szorc@gmail.com> parents: 37051 diff changeset	1069 def _makewantframeresult(self):
8c3c47362934 wireproto: implement basic frame reading and processing Gregory Szorc <gregory.szorc@gmail.com> parents: 37051 diff changeset	1070 return 'wantframe', {
8c3c47362934 wireproto: implement basic frame reading and processing Gregory Szorc <gregory.szorc@gmail.com> parents: 37051 diff changeset	1071 'state': self._state,
8c3c47362934 wireproto: implement basic frame reading and processing Gregory Szorc <gregory.szorc@gmail.com> parents: 37051 diff changeset	1072 }
8c3c47362934 wireproto: implement basic frame reading and processing Gregory Szorc <gregory.szorc@gmail.com> parents: 37051 diff changeset	1073
37292 3d0e2cd86e05 wireproto: use CBOR for command requests Gregory Szorc <gregory.szorc@gmail.com> parents: 37291 diff changeset	1074 def _validatecommandrequestframe(self, frame):
3d0e2cd86e05 wireproto: use CBOR for command requests Gregory Szorc <gregory.szorc@gmail.com> parents: 37291 diff changeset	1075 new = frame.flags & FLAG_COMMAND_REQUEST_NEW
3d0e2cd86e05 wireproto: use CBOR for command requests Gregory Szorc <gregory.szorc@gmail.com> parents: 37291 diff changeset	1076 continuation = frame.flags & FLAG_COMMAND_REQUEST_CONTINUATION
3d0e2cd86e05 wireproto: use CBOR for command requests Gregory Szorc <gregory.szorc@gmail.com> parents: 37291 diff changeset	1077
3d0e2cd86e05 wireproto: use CBOR for command requests Gregory Szorc <gregory.szorc@gmail.com> parents: 37291 diff changeset	1078 if new and continuation:
3d0e2cd86e05 wireproto: use CBOR for command requests Gregory Szorc <gregory.szorc@gmail.com> parents: 37291 diff changeset	1079 self._state = 'errored'
3d0e2cd86e05 wireproto: use CBOR for command requests Gregory Szorc <gregory.szorc@gmail.com> parents: 37291 diff changeset	1080 return self._makeerrorresult(
3d0e2cd86e05 wireproto: use CBOR for command requests Gregory Szorc <gregory.szorc@gmail.com> parents: 37291 diff changeset	1081 _('received command request frame with both new and '
3d0e2cd86e05 wireproto: use CBOR for command requests Gregory Szorc <gregory.szorc@gmail.com> parents: 37291 diff changeset	1082 'continuation flags set'))
3d0e2cd86e05 wireproto: use CBOR for command requests Gregory Szorc <gregory.szorc@gmail.com> parents: 37291 diff changeset	1083
3d0e2cd86e05 wireproto: use CBOR for command requests Gregory Szorc <gregory.szorc@gmail.com> parents: 37291 diff changeset	1084 if not new and not continuation:
3d0e2cd86e05 wireproto: use CBOR for command requests Gregory Szorc <gregory.szorc@gmail.com> parents: 37291 diff changeset	1085 self._state = 'errored'
3d0e2cd86e05 wireproto: use CBOR for command requests Gregory Szorc <gregory.szorc@gmail.com> parents: 37291 diff changeset	1086 return self._makeerrorresult(
3d0e2cd86e05 wireproto: use CBOR for command requests Gregory Szorc <gregory.szorc@gmail.com> parents: 37291 diff changeset	1087 _('received command request frame with neither new nor '
3d0e2cd86e05 wireproto: use CBOR for command requests Gregory Szorc <gregory.szorc@gmail.com> parents: 37291 diff changeset	1088 'continuation flags set'))
3d0e2cd86e05 wireproto: use CBOR for command requests Gregory Szorc <gregory.szorc@gmail.com> parents: 37291 diff changeset	1089
40126 327d40b94bed wireprotov2: handle sender protocol settings frames Gregory Szorc <gregory.szorc@gmail.com> parents: 40125 diff changeset	1090 def _onframeinitial(self, frame):
327d40b94bed wireprotov2: handle sender protocol settings frames Gregory Szorc <gregory.szorc@gmail.com> parents: 40125 diff changeset	1091 # Called when we receive a frame when in the "initial" state.
327d40b94bed wireprotov2: handle sender protocol settings frames Gregory Szorc <gregory.szorc@gmail.com> parents: 40125 diff changeset	1092 if frame.typeid == FRAME_TYPE_SENDER_PROTOCOL_SETTINGS:
327d40b94bed wireprotov2: handle sender protocol settings frames Gregory Szorc <gregory.szorc@gmail.com> parents: 40125 diff changeset	1093 self._state = 'protocol-settings-receiving'
327d40b94bed wireprotov2: handle sender protocol settings frames Gregory Szorc <gregory.szorc@gmail.com> parents: 40125 diff changeset	1094 self._protocolsettingsdecoder = cborutil.bufferingdecoder()
327d40b94bed wireprotov2: handle sender protocol settings frames Gregory Szorc <gregory.szorc@gmail.com> parents: 40125 diff changeset	1095 return self._onframeprotocolsettings(frame)
327d40b94bed wireprotov2: handle sender protocol settings frames Gregory Szorc <gregory.szorc@gmail.com> parents: 40125 diff changeset	1096
327d40b94bed wireprotov2: handle sender protocol settings frames Gregory Szorc <gregory.szorc@gmail.com> parents: 40125 diff changeset	1097 elif frame.typeid == FRAME_TYPE_COMMAND_REQUEST:
327d40b94bed wireprotov2: handle sender protocol settings frames Gregory Szorc <gregory.szorc@gmail.com> parents: 40125 diff changeset	1098 self._state = 'idle'
327d40b94bed wireprotov2: handle sender protocol settings frames Gregory Szorc <gregory.szorc@gmail.com> parents: 40125 diff changeset	1099 return self._onframeidle(frame)
327d40b94bed wireprotov2: handle sender protocol settings frames Gregory Szorc <gregory.szorc@gmail.com> parents: 40125 diff changeset	1100
327d40b94bed wireprotov2: handle sender protocol settings frames Gregory Szorc <gregory.szorc@gmail.com> parents: 40125 diff changeset	1101 else:
327d40b94bed wireprotov2: handle sender protocol settings frames Gregory Szorc <gregory.szorc@gmail.com> parents: 40125 diff changeset	1102 self._state = 'errored'
327d40b94bed wireprotov2: handle sender protocol settings frames Gregory Szorc <gregory.szorc@gmail.com> parents: 40125 diff changeset	1103 return self._makeerrorresult(
327d40b94bed wireprotov2: handle sender protocol settings frames Gregory Szorc <gregory.szorc@gmail.com> parents: 40125 diff changeset	1104 _('expected sender protocol settings or command request '
327d40b94bed wireprotov2: handle sender protocol settings frames Gregory Szorc <gregory.szorc@gmail.com> parents: 40125 diff changeset	1105 'frame; got %d') % frame.typeid)
327d40b94bed wireprotov2: handle sender protocol settings frames Gregory Szorc <gregory.szorc@gmail.com> parents: 40125 diff changeset	1106
327d40b94bed wireprotov2: handle sender protocol settings frames Gregory Szorc <gregory.szorc@gmail.com> parents: 40125 diff changeset	1107 def _onframeprotocolsettings(self, frame):
327d40b94bed wireprotov2: handle sender protocol settings frames Gregory Szorc <gregory.szorc@gmail.com> parents: 40125 diff changeset	1108 assert self._state == 'protocol-settings-receiving'
327d40b94bed wireprotov2: handle sender protocol settings frames Gregory Szorc <gregory.szorc@gmail.com> parents: 40125 diff changeset	1109 assert self._protocolsettingsdecoder is not None
327d40b94bed wireprotov2: handle sender protocol settings frames Gregory Szorc <gregory.szorc@gmail.com> parents: 40125 diff changeset	1110
327d40b94bed wireprotov2: handle sender protocol settings frames Gregory Szorc <gregory.szorc@gmail.com> parents: 40125 diff changeset	1111 if frame.typeid != FRAME_TYPE_SENDER_PROTOCOL_SETTINGS:
327d40b94bed wireprotov2: handle sender protocol settings frames Gregory Szorc <gregory.szorc@gmail.com> parents: 40125 diff changeset	1112 self._state = 'errored'
327d40b94bed wireprotov2: handle sender protocol settings frames Gregory Szorc <gregory.szorc@gmail.com> parents: 40125 diff changeset	1113 return self._makeerrorresult(
327d40b94bed wireprotov2: handle sender protocol settings frames Gregory Szorc <gregory.szorc@gmail.com> parents: 40125 diff changeset	1114 _('expected sender protocol settings frame; got %d') %
327d40b94bed wireprotov2: handle sender protocol settings frames Gregory Szorc <gregory.szorc@gmail.com> parents: 40125 diff changeset	1115 frame.typeid)
327d40b94bed wireprotov2: handle sender protocol settings frames Gregory Szorc <gregory.szorc@gmail.com> parents: 40125 diff changeset	1116
327d40b94bed wireprotov2: handle sender protocol settings frames Gregory Szorc <gregory.szorc@gmail.com> parents: 40125 diff changeset	1117 more = frame.flags & FLAG_SENDER_PROTOCOL_SETTINGS_CONTINUATION
327d40b94bed wireprotov2: handle sender protocol settings frames Gregory Szorc <gregory.szorc@gmail.com> parents: 40125 diff changeset	1118 eos = frame.flags & FLAG_SENDER_PROTOCOL_SETTINGS_EOS
327d40b94bed wireprotov2: handle sender protocol settings frames Gregory Szorc <gregory.szorc@gmail.com> parents: 40125 diff changeset	1119
327d40b94bed wireprotov2: handle sender protocol settings frames Gregory Szorc <gregory.szorc@gmail.com> parents: 40125 diff changeset	1120 if more and eos:
327d40b94bed wireprotov2: handle sender protocol settings frames Gregory Szorc <gregory.szorc@gmail.com> parents: 40125 diff changeset	1121 self._state = 'errored'
327d40b94bed wireprotov2: handle sender protocol settings frames Gregory Szorc <gregory.szorc@gmail.com> parents: 40125 diff changeset	1122 return self._makeerrorresult(
327d40b94bed wireprotov2: handle sender protocol settings frames Gregory Szorc <gregory.szorc@gmail.com> parents: 40125 diff changeset	1123 _('sender protocol settings frame cannot have both '
327d40b94bed wireprotov2: handle sender protocol settings frames Gregory Szorc <gregory.szorc@gmail.com> parents: 40125 diff changeset	1124 'continuation and end of stream flags set'))
327d40b94bed wireprotov2: handle sender protocol settings frames Gregory Szorc <gregory.szorc@gmail.com> parents: 40125 diff changeset	1125
327d40b94bed wireprotov2: handle sender protocol settings frames Gregory Szorc <gregory.szorc@gmail.com> parents: 40125 diff changeset	1126 if not more and not eos:
327d40b94bed wireprotov2: handle sender protocol settings frames Gregory Szorc <gregory.szorc@gmail.com> parents: 40125 diff changeset	1127 self._state = 'errored'
327d40b94bed wireprotov2: handle sender protocol settings frames Gregory Szorc <gregory.szorc@gmail.com> parents: 40125 diff changeset	1128 return self._makeerrorresult(
327d40b94bed wireprotov2: handle sender protocol settings frames Gregory Szorc <gregory.szorc@gmail.com> parents: 40125 diff changeset	1129 _('sender protocol settings frame must have continuation or '
327d40b94bed wireprotov2: handle sender protocol settings frames Gregory Szorc <gregory.szorc@gmail.com> parents: 40125 diff changeset	1130 'end of stream flag set'))
327d40b94bed wireprotov2: handle sender protocol settings frames Gregory Szorc <gregory.szorc@gmail.com> parents: 40125 diff changeset	1131
327d40b94bed wireprotov2: handle sender protocol settings frames Gregory Szorc <gregory.szorc@gmail.com> parents: 40125 diff changeset	1132 # TODO establish limits for maximum amount of data that can be
327d40b94bed wireprotov2: handle sender protocol settings frames Gregory Szorc <gregory.szorc@gmail.com> parents: 40125 diff changeset	1133 # buffered.
327d40b94bed wireprotov2: handle sender protocol settings frames Gregory Szorc <gregory.szorc@gmail.com> parents: 40125 diff changeset	1134 try:
327d40b94bed wireprotov2: handle sender protocol settings frames Gregory Szorc <gregory.szorc@gmail.com> parents: 40125 diff changeset	1135 self._protocolsettingsdecoder.decode(frame.payload)
327d40b94bed wireprotov2: handle sender protocol settings frames Gregory Szorc <gregory.szorc@gmail.com> parents: 40125 diff changeset	1136 except Exception as e:
327d40b94bed wireprotov2: handle sender protocol settings frames Gregory Szorc <gregory.szorc@gmail.com> parents: 40125 diff changeset	1137 self._state = 'errored'
327d40b94bed wireprotov2: handle sender protocol settings frames Gregory Szorc <gregory.szorc@gmail.com> parents: 40125 diff changeset	1138 return self._makeerrorresult(
327d40b94bed wireprotov2: handle sender protocol settings frames Gregory Szorc <gregory.szorc@gmail.com> parents: 40125 diff changeset	1139 _('error decoding CBOR from sender protocol settings frame: %s')
327d40b94bed wireprotov2: handle sender protocol settings frames Gregory Szorc <gregory.szorc@gmail.com> parents: 40125 diff changeset	1140 % stringutil.forcebytestr(e))
327d40b94bed wireprotov2: handle sender protocol settings frames Gregory Szorc <gregory.szorc@gmail.com> parents: 40125 diff changeset	1141
327d40b94bed wireprotov2: handle sender protocol settings frames Gregory Szorc <gregory.szorc@gmail.com> parents: 40125 diff changeset	1142 if more:
327d40b94bed wireprotov2: handle sender protocol settings frames Gregory Szorc <gregory.szorc@gmail.com> parents: 40125 diff changeset	1143 return self._makewantframeresult()
327d40b94bed wireprotov2: handle sender protocol settings frames Gregory Szorc <gregory.szorc@gmail.com> parents: 40125 diff changeset	1144
327d40b94bed wireprotov2: handle sender protocol settings frames Gregory Szorc <gregory.szorc@gmail.com> parents: 40125 diff changeset	1145 assert eos
327d40b94bed wireprotov2: handle sender protocol settings frames Gregory Szorc <gregory.szorc@gmail.com> parents: 40125 diff changeset	1146
327d40b94bed wireprotov2: handle sender protocol settings frames Gregory Szorc <gregory.szorc@gmail.com> parents: 40125 diff changeset	1147 decoded = self._protocolsettingsdecoder.getavailable()
327d40b94bed wireprotov2: handle sender protocol settings frames Gregory Szorc <gregory.szorc@gmail.com> parents: 40125 diff changeset	1148 self._protocolsettingsdecoder = None
327d40b94bed wireprotov2: handle sender protocol settings frames Gregory Szorc <gregory.szorc@gmail.com> parents: 40125 diff changeset	1149
327d40b94bed wireprotov2: handle sender protocol settings frames Gregory Szorc <gregory.szorc@gmail.com> parents: 40125 diff changeset	1150 if not decoded:
327d40b94bed wireprotov2: handle sender protocol settings frames Gregory Szorc <gregory.szorc@gmail.com> parents: 40125 diff changeset	1151 self._state = 'errored'
327d40b94bed wireprotov2: handle sender protocol settings frames Gregory Szorc <gregory.szorc@gmail.com> parents: 40125 diff changeset	1152 return self._makeerrorresult(
327d40b94bed wireprotov2: handle sender protocol settings frames Gregory Szorc <gregory.szorc@gmail.com> parents: 40125 diff changeset	1153 _('sender protocol settings frame did not contain CBOR data'))
327d40b94bed wireprotov2: handle sender protocol settings frames Gregory Szorc <gregory.szorc@gmail.com> parents: 40125 diff changeset	1154 elif len(decoded) > 1:
327d40b94bed wireprotov2: handle sender protocol settings frames Gregory Szorc <gregory.szorc@gmail.com> parents: 40125 diff changeset	1155 self._state = 'errored'
327d40b94bed wireprotov2: handle sender protocol settings frames Gregory Szorc <gregory.szorc@gmail.com> parents: 40125 diff changeset	1156 return self._makeerrorresult(
327d40b94bed wireprotov2: handle sender protocol settings frames Gregory Szorc <gregory.szorc@gmail.com> parents: 40125 diff changeset	1157 _('sender protocol settings frame contained multiple CBOR '
327d40b94bed wireprotov2: handle sender protocol settings frames Gregory Szorc <gregory.szorc@gmail.com> parents: 40125 diff changeset	1158 'values'))
327d40b94bed wireprotov2: handle sender protocol settings frames Gregory Szorc <gregory.szorc@gmail.com> parents: 40125 diff changeset	1159
327d40b94bed wireprotov2: handle sender protocol settings frames Gregory Szorc <gregory.szorc@gmail.com> parents: 40125 diff changeset	1160 d = decoded[0]
327d40b94bed wireprotov2: handle sender protocol settings frames Gregory Szorc <gregory.szorc@gmail.com> parents: 40125 diff changeset	1161
327d40b94bed wireprotov2: handle sender protocol settings frames Gregory Szorc <gregory.szorc@gmail.com> parents: 40125 diff changeset	1162 if b'contentencodings' in d:
327d40b94bed wireprotov2: handle sender protocol settings frames Gregory Szorc <gregory.szorc@gmail.com> parents: 40125 diff changeset	1163 self._sendersettings['contentencodings'] = d[b'contentencodings']
327d40b94bed wireprotov2: handle sender protocol settings frames Gregory Szorc <gregory.szorc@gmail.com> parents: 40125 diff changeset	1164
327d40b94bed wireprotov2: handle sender protocol settings frames Gregory Szorc <gregory.szorc@gmail.com> parents: 40125 diff changeset	1165 self._state = 'idle'
327d40b94bed wireprotov2: handle sender protocol settings frames Gregory Szorc <gregory.szorc@gmail.com> parents: 40125 diff changeset	1166
327d40b94bed wireprotov2: handle sender protocol settings frames Gregory Szorc <gregory.szorc@gmail.com> parents: 40125 diff changeset	1167 return self._makewantframeresult()
327d40b94bed wireprotov2: handle sender protocol settings frames Gregory Szorc <gregory.szorc@gmail.com> parents: 40125 diff changeset	1168
37061 884a0c1604ad wireproto: define attr-based classes for representing frames Gregory Szorc <gregory.szorc@gmail.com> parents: 37060 diff changeset	1169 def _onframeidle(self, frame):
37052 8c3c47362934 wireproto: implement basic frame reading and processing Gregory Szorc <gregory.szorc@gmail.com> parents: 37051 diff changeset	1170 # The only frame type that should be received in this state is a
8c3c47362934 wireproto: implement basic frame reading and processing Gregory Szorc <gregory.szorc@gmail.com> parents: 37051 diff changeset	1171 # command request.
37292 3d0e2cd86e05 wireproto: use CBOR for command requests Gregory Szorc <gregory.szorc@gmail.com> parents: 37291 diff changeset	1172 if frame.typeid != FRAME_TYPE_COMMAND_REQUEST:
37052 8c3c47362934 wireproto: implement basic frame reading and processing Gregory Szorc <gregory.szorc@gmail.com> parents: 37051 diff changeset	1173 self._state = 'errored'
8c3c47362934 wireproto: implement basic frame reading and processing Gregory Szorc <gregory.szorc@gmail.com> parents: 37051 diff changeset	1174 return self._makeerrorresult(
37292 3d0e2cd86e05 wireproto: use CBOR for command requests Gregory Szorc <gregory.szorc@gmail.com> parents: 37291 diff changeset	1175 _('expected command request frame; got %d') % frame.typeid)
3d0e2cd86e05 wireproto: use CBOR for command requests Gregory Szorc <gregory.szorc@gmail.com> parents: 37291 diff changeset	1176
3d0e2cd86e05 wireproto: use CBOR for command requests Gregory Szorc <gregory.szorc@gmail.com> parents: 37291 diff changeset	1177 res = self._validatecommandrequestframe(frame)
3d0e2cd86e05 wireproto: use CBOR for command requests Gregory Szorc <gregory.szorc@gmail.com> parents: 37291 diff changeset	1178 if res:
3d0e2cd86e05 wireproto: use CBOR for command requests Gregory Szorc <gregory.szorc@gmail.com> parents: 37291 diff changeset	1179 return res
37052 8c3c47362934 wireproto: implement basic frame reading and processing Gregory Szorc <gregory.szorc@gmail.com> parents: 37051 diff changeset	1180
37061 884a0c1604ad wireproto: define attr-based classes for representing frames Gregory Szorc <gregory.szorc@gmail.com> parents: 37060 diff changeset	1181 if frame.requestid in self._receivingcommands:
37058 c5e9c3b47366 wireproto: support for receiving multiple requests Gregory Szorc <gregory.szorc@gmail.com> parents: 37057 diff changeset	1182 self._state = 'errored'
c5e9c3b47366 wireproto: support for receiving multiple requests Gregory Szorc <gregory.szorc@gmail.com> parents: 37057 diff changeset	1183 return self._makeerrorresult(
37061 884a0c1604ad wireproto: define attr-based classes for representing frames Gregory Szorc <gregory.szorc@gmail.com> parents: 37060 diff changeset	1184 _('request with ID %d already received') % frame.requestid)
37058 c5e9c3b47366 wireproto: support for receiving multiple requests Gregory Szorc <gregory.szorc@gmail.com> parents: 37057 diff changeset	1185
37063 39304dd63589 wireproto: explicitly track which requests are active Gregory Szorc <gregory.szorc@gmail.com> parents: 37062 diff changeset	1186 if frame.requestid in self._activecommands:
39304dd63589 wireproto: explicitly track which requests are active Gregory Szorc <gregory.szorc@gmail.com> parents: 37062 diff changeset	1187 self._state = 'errored'
37292 3d0e2cd86e05 wireproto: use CBOR for command requests Gregory Szorc <gregory.szorc@gmail.com> parents: 37291 diff changeset	1188 return self._makeerrorresult(
3d0e2cd86e05 wireproto: use CBOR for command requests Gregory Szorc <gregory.szorc@gmail.com> parents: 37291 diff changeset	1189 _('request with ID %d is already active') % frame.requestid)
3d0e2cd86e05 wireproto: use CBOR for command requests Gregory Szorc <gregory.szorc@gmail.com> parents: 37291 diff changeset	1190
3d0e2cd86e05 wireproto: use CBOR for command requests Gregory Szorc <gregory.szorc@gmail.com> parents: 37291 diff changeset	1191 new = frame.flags & FLAG_COMMAND_REQUEST_NEW
3d0e2cd86e05 wireproto: use CBOR for command requests Gregory Szorc <gregory.szorc@gmail.com> parents: 37291 diff changeset	1192 moreframes = frame.flags & FLAG_COMMAND_REQUEST_MORE_FRAMES
3d0e2cd86e05 wireproto: use CBOR for command requests Gregory Szorc <gregory.szorc@gmail.com> parents: 37291 diff changeset	1193 expectingdata = frame.flags & FLAG_COMMAND_REQUEST_EXPECT_DATA
37063 39304dd63589 wireproto: explicitly track which requests are active Gregory Szorc <gregory.szorc@gmail.com> parents: 37062 diff changeset	1194
37292 3d0e2cd86e05 wireproto: use CBOR for command requests Gregory Szorc <gregory.szorc@gmail.com> parents: 37291 diff changeset	1195 if not new:
3d0e2cd86e05 wireproto: use CBOR for command requests Gregory Szorc <gregory.szorc@gmail.com> parents: 37291 diff changeset	1196 self._state = 'errored'
3d0e2cd86e05 wireproto: use CBOR for command requests Gregory Szorc <gregory.szorc@gmail.com> parents: 37291 diff changeset	1197 return self._makeerrorresult(
3d0e2cd86e05 wireproto: use CBOR for command requests Gregory Szorc <gregory.szorc@gmail.com> parents: 37291 diff changeset	1198 _('received command request frame without new flag set'))
3d0e2cd86e05 wireproto: use CBOR for command requests Gregory Szorc <gregory.szorc@gmail.com> parents: 37291 diff changeset	1199
3d0e2cd86e05 wireproto: use CBOR for command requests Gregory Szorc <gregory.szorc@gmail.com> parents: 37291 diff changeset	1200 payload = util.bytesio()
3d0e2cd86e05 wireproto: use CBOR for command requests Gregory Szorc <gregory.szorc@gmail.com> parents: 37291 diff changeset	1201 payload.write(frame.payload)
37058 c5e9c3b47366 wireproto: support for receiving multiple requests Gregory Szorc <gregory.szorc@gmail.com> parents: 37057 diff changeset	1202
37061 884a0c1604ad wireproto: define attr-based classes for representing frames Gregory Szorc <gregory.szorc@gmail.com> parents: 37060 diff changeset	1203 self._receivingcommands[frame.requestid] = {
37292 3d0e2cd86e05 wireproto: use CBOR for command requests Gregory Szorc <gregory.szorc@gmail.com> parents: 37291 diff changeset	1204 'payload': payload,
37058 c5e9c3b47366 wireproto: support for receiving multiple requests Gregory Szorc <gregory.szorc@gmail.com> parents: 37057 diff changeset	1205 'data': None,
37292 3d0e2cd86e05 wireproto: use CBOR for command requests Gregory Szorc <gregory.szorc@gmail.com> parents: 37291 diff changeset	1206 'requestdone': not moreframes,
3d0e2cd86e05 wireproto: use CBOR for command requests Gregory Szorc <gregory.szorc@gmail.com> parents: 37291 diff changeset	1207 'expectingdata': bool(expectingdata),
37058 c5e9c3b47366 wireproto: support for receiving multiple requests Gregory Szorc <gregory.szorc@gmail.com> parents: 37057 diff changeset	1208 }
37052 8c3c47362934 wireproto: implement basic frame reading and processing Gregory Szorc <gregory.szorc@gmail.com> parents: 37051 diff changeset	1209
37292 3d0e2cd86e05 wireproto: use CBOR for command requests Gregory Szorc <gregory.szorc@gmail.com> parents: 37291 diff changeset	1210 # This is the final frame for this request. Dispatch it.
3d0e2cd86e05 wireproto: use CBOR for command requests Gregory Szorc <gregory.szorc@gmail.com> parents: 37291 diff changeset	1211 if not moreframes and not expectingdata:
37061 884a0c1604ad wireproto: define attr-based classes for representing frames Gregory Szorc <gregory.szorc@gmail.com> parents: 37060 diff changeset	1212 return self._makeruncommandresult(frame.requestid)
37052 8c3c47362934 wireproto: implement basic frame reading and processing Gregory Szorc <gregory.szorc@gmail.com> parents: 37051 diff changeset	1213
37292 3d0e2cd86e05 wireproto: use CBOR for command requests Gregory Szorc <gregory.szorc@gmail.com> parents: 37291 diff changeset	1214 assert moreframes or expectingdata
3d0e2cd86e05 wireproto: use CBOR for command requests Gregory Szorc <gregory.szorc@gmail.com> parents: 37291 diff changeset	1215 self._state = 'command-receiving'
3d0e2cd86e05 wireproto: use CBOR for command requests Gregory Szorc <gregory.szorc@gmail.com> parents: 37291 diff changeset	1216 return self._makewantframeresult()
37052 8c3c47362934 wireproto: implement basic frame reading and processing Gregory Szorc <gregory.szorc@gmail.com> parents: 37051 diff changeset	1217
37061 884a0c1604ad wireproto: define attr-based classes for representing frames Gregory Szorc <gregory.szorc@gmail.com> parents: 37060 diff changeset	1218 def _onframecommandreceiving(self, frame):
37292 3d0e2cd86e05 wireproto: use CBOR for command requests Gregory Szorc <gregory.szorc@gmail.com> parents: 37291 diff changeset	1219 if frame.typeid == FRAME_TYPE_COMMAND_REQUEST:
3d0e2cd86e05 wireproto: use CBOR for command requests Gregory Szorc <gregory.szorc@gmail.com> parents: 37291 diff changeset	1220 # Process new command requests as such.
3d0e2cd86e05 wireproto: use CBOR for command requests Gregory Szorc <gregory.szorc@gmail.com> parents: 37291 diff changeset	1221 if frame.flags & FLAG_COMMAND_REQUEST_NEW:
3d0e2cd86e05 wireproto: use CBOR for command requests Gregory Szorc <gregory.szorc@gmail.com> parents: 37291 diff changeset	1222 return self._onframeidle(frame)
3d0e2cd86e05 wireproto: use CBOR for command requests Gregory Szorc <gregory.szorc@gmail.com> parents: 37291 diff changeset	1223
3d0e2cd86e05 wireproto: use CBOR for command requests Gregory Szorc <gregory.szorc@gmail.com> parents: 37291 diff changeset	1224 res = self._validatecommandrequestframe(frame)
3d0e2cd86e05 wireproto: use CBOR for command requests Gregory Szorc <gregory.szorc@gmail.com> parents: 37291 diff changeset	1225 if res:
3d0e2cd86e05 wireproto: use CBOR for command requests Gregory Szorc <gregory.szorc@gmail.com> parents: 37291 diff changeset	1226 return res
37058 c5e9c3b47366 wireproto: support for receiving multiple requests Gregory Szorc <gregory.szorc@gmail.com> parents: 37057 diff changeset	1227
c5e9c3b47366 wireproto: support for receiving multiple requests Gregory Szorc <gregory.szorc@gmail.com> parents: 37057 diff changeset	1228 # All other frames should be related to a command that is currently
37063 39304dd63589 wireproto: explicitly track which requests are active Gregory Szorc <gregory.szorc@gmail.com> parents: 37062 diff changeset	1229 # receiving but is not active.
39304dd63589 wireproto: explicitly track which requests are active Gregory Szorc <gregory.szorc@gmail.com> parents: 37062 diff changeset	1230 if frame.requestid in self._activecommands:
39304dd63589 wireproto: explicitly track which requests are active Gregory Szorc <gregory.szorc@gmail.com> parents: 37062 diff changeset	1231 self._state = 'errored'
39304dd63589 wireproto: explicitly track which requests are active Gregory Szorc <gregory.szorc@gmail.com> parents: 37062 diff changeset	1232 return self._makeerrorresult(
39304dd63589 wireproto: explicitly track which requests are active Gregory Szorc <gregory.szorc@gmail.com> parents: 37062 diff changeset	1233 _('received frame for request that is still active: %d') %
39304dd63589 wireproto: explicitly track which requests are active Gregory Szorc <gregory.szorc@gmail.com> parents: 37062 diff changeset	1234 frame.requestid)
39304dd63589 wireproto: explicitly track which requests are active Gregory Szorc <gregory.szorc@gmail.com> parents: 37062 diff changeset	1235
37061 884a0c1604ad wireproto: define attr-based classes for representing frames Gregory Szorc <gregory.szorc@gmail.com> parents: 37060 diff changeset	1236 if frame.requestid not in self._receivingcommands:
37052 8c3c47362934 wireproto: implement basic frame reading and processing Gregory Szorc <gregory.szorc@gmail.com> parents: 37051 diff changeset	1237 self._state = 'errored'
37058 c5e9c3b47366 wireproto: support for receiving multiple requests Gregory Szorc <gregory.szorc@gmail.com> parents: 37057 diff changeset	1238 return self._makeerrorresult(
c5e9c3b47366 wireproto: support for receiving multiple requests Gregory Szorc <gregory.szorc@gmail.com> parents: 37057 diff changeset	1239 _('received frame for request that is not receiving: %d') %
37061 884a0c1604ad wireproto: define attr-based classes for representing frames Gregory Szorc <gregory.szorc@gmail.com> parents: 37060 diff changeset	1240 frame.requestid)
37058 c5e9c3b47366 wireproto: support for receiving multiple requests Gregory Szorc <gregory.szorc@gmail.com> parents: 37057 diff changeset	1241
37061 884a0c1604ad wireproto: define attr-based classes for representing frames Gregory Szorc <gregory.szorc@gmail.com> parents: 37060 diff changeset	1242 entry = self._receivingcommands[frame.requestid]
37058 c5e9c3b47366 wireproto: support for receiving multiple requests Gregory Szorc <gregory.szorc@gmail.com> parents: 37057 diff changeset	1243
37292 3d0e2cd86e05 wireproto: use CBOR for command requests Gregory Szorc <gregory.szorc@gmail.com> parents: 37291 diff changeset	1244 if frame.typeid == FRAME_TYPE_COMMAND_REQUEST:
3d0e2cd86e05 wireproto: use CBOR for command requests Gregory Szorc <gregory.szorc@gmail.com> parents: 37291 diff changeset	1245 moreframes = frame.flags & FLAG_COMMAND_REQUEST_MORE_FRAMES
3d0e2cd86e05 wireproto: use CBOR for command requests Gregory Szorc <gregory.szorc@gmail.com> parents: 37291 diff changeset	1246 expectingdata = bool(frame.flags & FLAG_COMMAND_REQUEST_EXPECT_DATA)
3d0e2cd86e05 wireproto: use CBOR for command requests Gregory Szorc <gregory.szorc@gmail.com> parents: 37291 diff changeset	1247
3d0e2cd86e05 wireproto: use CBOR for command requests Gregory Szorc <gregory.szorc@gmail.com> parents: 37291 diff changeset	1248 if entry['requestdone']:
3d0e2cd86e05 wireproto: use CBOR for command requests Gregory Szorc <gregory.szorc@gmail.com> parents: 37291 diff changeset	1249 self._state = 'errored'
3d0e2cd86e05 wireproto: use CBOR for command requests Gregory Szorc <gregory.szorc@gmail.com> parents: 37291 diff changeset	1250 return self._makeerrorresult(
3d0e2cd86e05 wireproto: use CBOR for command requests Gregory Szorc <gregory.szorc@gmail.com> parents: 37291 diff changeset	1251 _('received command request frame when request frames '
3d0e2cd86e05 wireproto: use CBOR for command requests Gregory Szorc <gregory.szorc@gmail.com> parents: 37291 diff changeset	1252 'were supposedly done'))
3d0e2cd86e05 wireproto: use CBOR for command requests Gregory Szorc <gregory.szorc@gmail.com> parents: 37291 diff changeset	1253
3d0e2cd86e05 wireproto: use CBOR for command requests Gregory Szorc <gregory.szorc@gmail.com> parents: 37291 diff changeset	1254 if expectingdata != entry['expectingdata']:
37058 c5e9c3b47366 wireproto: support for receiving multiple requests Gregory Szorc <gregory.szorc@gmail.com> parents: 37057 diff changeset	1255 self._state = 'errored'
37292 3d0e2cd86e05 wireproto: use CBOR for command requests Gregory Szorc <gregory.szorc@gmail.com> parents: 37291 diff changeset	1256 return self._makeerrorresult(
3d0e2cd86e05 wireproto: use CBOR for command requests Gregory Szorc <gregory.szorc@gmail.com> parents: 37291 diff changeset	1257 _('mismatch between expect data flag and previous frame'))
3d0e2cd86e05 wireproto: use CBOR for command requests Gregory Szorc <gregory.szorc@gmail.com> parents: 37291 diff changeset	1258
3d0e2cd86e05 wireproto: use CBOR for command requests Gregory Szorc <gregory.szorc@gmail.com> parents: 37291 diff changeset	1259 entry['payload'].write(frame.payload)
37058 c5e9c3b47366 wireproto: support for receiving multiple requests Gregory Szorc <gregory.szorc@gmail.com> parents: 37057 diff changeset	1260
37292 3d0e2cd86e05 wireproto: use CBOR for command requests Gregory Szorc <gregory.szorc@gmail.com> parents: 37291 diff changeset	1261 if not moreframes:
3d0e2cd86e05 wireproto: use CBOR for command requests Gregory Szorc <gregory.szorc@gmail.com> parents: 37291 diff changeset	1262 entry['requestdone'] = True
3d0e2cd86e05 wireproto: use CBOR for command requests Gregory Szorc <gregory.szorc@gmail.com> parents: 37291 diff changeset	1263
3d0e2cd86e05 wireproto: use CBOR for command requests Gregory Szorc <gregory.szorc@gmail.com> parents: 37291 diff changeset	1264 if not moreframes and not expectingdata:
3d0e2cd86e05 wireproto: use CBOR for command requests Gregory Szorc <gregory.szorc@gmail.com> parents: 37291 diff changeset	1265 return self._makeruncommandresult(frame.requestid)
3d0e2cd86e05 wireproto: use CBOR for command requests Gregory Szorc <gregory.szorc@gmail.com> parents: 37291 diff changeset	1266
3d0e2cd86e05 wireproto: use CBOR for command requests Gregory Szorc <gregory.szorc@gmail.com> parents: 37291 diff changeset	1267 return self._makewantframeresult()
37058 c5e9c3b47366 wireproto: support for receiving multiple requests Gregory Szorc <gregory.szorc@gmail.com> parents: 37057 diff changeset	1268
37061 884a0c1604ad wireproto: define attr-based classes for representing frames Gregory Szorc <gregory.szorc@gmail.com> parents: 37060 diff changeset	1269 elif frame.typeid == FRAME_TYPE_COMMAND_DATA:
37058 c5e9c3b47366 wireproto: support for receiving multiple requests Gregory Szorc <gregory.szorc@gmail.com> parents: 37057 diff changeset	1270 if not entry['expectingdata']:
c5e9c3b47366 wireproto: support for receiving multiple requests Gregory Szorc <gregory.szorc@gmail.com> parents: 37057 diff changeset	1271 self._state = 'errored'
c5e9c3b47366 wireproto: support for receiving multiple requests Gregory Szorc <gregory.szorc@gmail.com> parents: 37057 diff changeset	1272 return self._makeerrorresult(_(
c5e9c3b47366 wireproto: support for receiving multiple requests Gregory Szorc <gregory.szorc@gmail.com> parents: 37057 diff changeset	1273 'received command data frame for request that is not '
37061 884a0c1604ad wireproto: define attr-based classes for representing frames Gregory Szorc <gregory.szorc@gmail.com> parents: 37060 diff changeset	1274 'expecting data: %d') % frame.requestid)
37058 c5e9c3b47366 wireproto: support for receiving multiple requests Gregory Szorc <gregory.szorc@gmail.com> parents: 37057 diff changeset	1275
c5e9c3b47366 wireproto: support for receiving multiple requests Gregory Szorc <gregory.szorc@gmail.com> parents: 37057 diff changeset	1276 if entry['data'] is None:
c5e9c3b47366 wireproto: support for receiving multiple requests Gregory Szorc <gregory.szorc@gmail.com> parents: 37057 diff changeset	1277 entry['data'] = util.bytesio()
c5e9c3b47366 wireproto: support for receiving multiple requests Gregory Szorc <gregory.szorc@gmail.com> parents: 37057 diff changeset	1278
37061 884a0c1604ad wireproto: define attr-based classes for representing frames Gregory Szorc <gregory.szorc@gmail.com> parents: 37060 diff changeset	1279 return self._handlecommanddataframe(frame, entry)
37292 3d0e2cd86e05 wireproto: use CBOR for command requests Gregory Szorc <gregory.szorc@gmail.com> parents: 37291 diff changeset	1280 else:
37052 8c3c47362934 wireproto: implement basic frame reading and processing Gregory Szorc <gregory.szorc@gmail.com> parents: 37051 diff changeset	1281 self._state = 'errored'
37292 3d0e2cd86e05 wireproto: use CBOR for command requests Gregory Szorc <gregory.szorc@gmail.com> parents: 37291 diff changeset	1282 return self._makeerrorresult(_(
3d0e2cd86e05 wireproto: use CBOR for command requests Gregory Szorc <gregory.szorc@gmail.com> parents: 37291 diff changeset	1283 'received unexpected frame type: %d') % frame.typeid)
37052 8c3c47362934 wireproto: implement basic frame reading and processing Gregory Szorc <gregory.szorc@gmail.com> parents: 37051 diff changeset	1284
37061 884a0c1604ad wireproto: define attr-based classes for representing frames Gregory Szorc <gregory.szorc@gmail.com> parents: 37060 diff changeset	1285 def _handlecommanddataframe(self, frame, entry):
884a0c1604ad wireproto: define attr-based classes for representing frames Gregory Szorc <gregory.szorc@gmail.com> parents: 37060 diff changeset	1286 assert frame.typeid == FRAME_TYPE_COMMAND_DATA
37052 8c3c47362934 wireproto: implement basic frame reading and processing Gregory Szorc <gregory.szorc@gmail.com> parents: 37051 diff changeset	1287
8c3c47362934 wireproto: implement basic frame reading and processing Gregory Szorc <gregory.szorc@gmail.com> parents: 37051 diff changeset	1288 # TODO support streaming data instead of buffering it.
37061 884a0c1604ad wireproto: define attr-based classes for representing frames Gregory Szorc <gregory.szorc@gmail.com> parents: 37060 diff changeset	1289 entry['data'].write(frame.payload)
37052 8c3c47362934 wireproto: implement basic frame reading and processing Gregory Szorc <gregory.szorc@gmail.com> parents: 37051 diff changeset	1290
37061 884a0c1604ad wireproto: define attr-based classes for representing frames Gregory Szorc <gregory.szorc@gmail.com> parents: 37060 diff changeset	1291 if frame.flags & FLAG_COMMAND_DATA_CONTINUATION:
37052 8c3c47362934 wireproto: implement basic frame reading and processing Gregory Szorc <gregory.szorc@gmail.com> parents: 37051 diff changeset	1292 return self._makewantframeresult()
37061 884a0c1604ad wireproto: define attr-based classes for representing frames Gregory Szorc <gregory.szorc@gmail.com> parents: 37060 diff changeset	1293 elif frame.flags & FLAG_COMMAND_DATA_EOS:
37058 c5e9c3b47366 wireproto: support for receiving multiple requests Gregory Szorc <gregory.szorc@gmail.com> parents: 37057 diff changeset	1294 entry['data'].seek(0)
37061 884a0c1604ad wireproto: define attr-based classes for representing frames Gregory Szorc <gregory.szorc@gmail.com> parents: 37060 diff changeset	1295 return self._makeruncommandresult(frame.requestid)
37052 8c3c47362934 wireproto: implement basic frame reading and processing Gregory Szorc <gregory.szorc@gmail.com> parents: 37051 diff changeset	1296 else:
8c3c47362934 wireproto: implement basic frame reading and processing Gregory Szorc <gregory.szorc@gmail.com> parents: 37051 diff changeset	1297 self._state = 'errored'
8c3c47362934 wireproto: implement basic frame reading and processing Gregory Szorc <gregory.szorc@gmail.com> parents: 37051 diff changeset	1298 return self._makeerrorresult(_('command data frame without '
8c3c47362934 wireproto: implement basic frame reading and processing Gregory Szorc <gregory.szorc@gmail.com> parents: 37051 diff changeset	1299 'flags'))
8c3c47362934 wireproto: implement basic frame reading and processing Gregory Szorc <gregory.szorc@gmail.com> parents: 37051 diff changeset	1300
37061 884a0c1604ad wireproto: define attr-based classes for representing frames Gregory Szorc <gregory.szorc@gmail.com> parents: 37060 diff changeset	1301 def _onframeerrored(self, frame):
37052 8c3c47362934 wireproto: implement basic frame reading and processing Gregory Szorc <gregory.szorc@gmail.com> parents: 37051 diff changeset	1302 return self._makeerrorresult(_('server already errored'))
37543 01361be9e2dc wireproto: introduce a reactor for client-side state Gregory Szorc <gregory.szorc@gmail.com> parents: 37485 diff changeset	1303
01361be9e2dc wireproto: introduce a reactor for client-side state Gregory Szorc <gregory.szorc@gmail.com> parents: 37485 diff changeset	1304 class commandrequest(object):
01361be9e2dc wireproto: introduce a reactor for client-side state Gregory Szorc <gregory.szorc@gmail.com> parents: 37485 diff changeset	1305 """Represents a request to run a command."""
01361be9e2dc wireproto: introduce a reactor for client-side state Gregory Szorc <gregory.szorc@gmail.com> parents: 37485 diff changeset	1306
40024 86b22a4cfab1 wireprotov2: client support for advertising redirect targets Gregory Szorc <gregory.szorc@gmail.com> parents: 40020 diff changeset	1307 def __init__(self, requestid, name, args, datafh=None, redirect=None):
37543 01361be9e2dc wireproto: introduce a reactor for client-side state Gregory Szorc <gregory.szorc@gmail.com> parents: 37485 diff changeset	1308 self.requestid = requestid
01361be9e2dc wireproto: introduce a reactor for client-side state Gregory Szorc <gregory.szorc@gmail.com> parents: 37485 diff changeset	1309 self.name = name
01361be9e2dc wireproto: introduce a reactor for client-side state Gregory Szorc <gregory.szorc@gmail.com> parents: 37485 diff changeset	1310 self.args = args
01361be9e2dc wireproto: introduce a reactor for client-side state Gregory Szorc <gregory.szorc@gmail.com> parents: 37485 diff changeset	1311 self.datafh = datafh
40024 86b22a4cfab1 wireprotov2: client support for advertising redirect targets Gregory Szorc <gregory.szorc@gmail.com> parents: 40020 diff changeset	1312 self.redirect = redirect
37543 01361be9e2dc wireproto: introduce a reactor for client-side state Gregory Szorc <gregory.szorc@gmail.com> parents: 37485 diff changeset	1313 self.state = 'pending'
01361be9e2dc wireproto: introduce a reactor for client-side state Gregory Szorc <gregory.szorc@gmail.com> parents: 37485 diff changeset	1314
01361be9e2dc wireproto: introduce a reactor for client-side state Gregory Szorc <gregory.szorc@gmail.com> parents: 37485 diff changeset	1315 class clientreactor(object):
01361be9e2dc wireproto: introduce a reactor for client-side state Gregory Szorc <gregory.szorc@gmail.com> parents: 37485 diff changeset	1316 """Holds state of a client issuing frame-based protocol requests.
01361be9e2dc wireproto: introduce a reactor for client-side state Gregory Szorc <gregory.szorc@gmail.com> parents: 37485 diff changeset	1317
01361be9e2dc wireproto: introduce a reactor for client-side state Gregory Szorc <gregory.szorc@gmail.com> parents: 37485 diff changeset	1318 This is like ``serverreactor`` but for client-side state.
01361be9e2dc wireproto: introduce a reactor for client-side state Gregory Szorc <gregory.szorc@gmail.com> parents: 37485 diff changeset	1319
01361be9e2dc wireproto: introduce a reactor for client-side state Gregory Szorc <gregory.szorc@gmail.com> parents: 37485 diff changeset	1320 Each instance is bound to the lifetime of a connection. For persistent
01361be9e2dc wireproto: introduce a reactor for client-side state Gregory Szorc <gregory.szorc@gmail.com> parents: 37485 diff changeset	1321 connection transports using e.g. TCP sockets and speaking the raw
01361be9e2dc wireproto: introduce a reactor for client-side state Gregory Szorc <gregory.szorc@gmail.com> parents: 37485 diff changeset	1322 framing protocol, there will be a single instance for the lifetime of
01361be9e2dc wireproto: introduce a reactor for client-side state Gregory Szorc <gregory.szorc@gmail.com> parents: 37485 diff changeset	1323 the TCP socket. For transports where there are multiple discrete
01361be9e2dc wireproto: introduce a reactor for client-side state Gregory Szorc <gregory.szorc@gmail.com> parents: 37485 diff changeset	1324 interactions (say tunneled within in HTTP request), there will be a
01361be9e2dc wireproto: introduce a reactor for client-side state Gregory Szorc <gregory.szorc@gmail.com> parents: 37485 diff changeset	1325 separate instance for each distinct interaction.
40127 080419fa4fe4 wireprotov2: document client reactor actions Gregory Szorc <gregory.szorc@gmail.com> parents: 40126 diff changeset	1326
080419fa4fe4 wireprotov2: document client reactor actions Gregory Szorc <gregory.szorc@gmail.com> parents: 40126 diff changeset	1327 Consumers are expected to tell instances when events occur by calling
080419fa4fe4 wireprotov2: document client reactor actions Gregory Szorc <gregory.szorc@gmail.com> parents: 40126 diff changeset	1328 various methods. These methods return a 2-tuple describing any follow-up
080419fa4fe4 wireprotov2: document client reactor actions Gregory Szorc <gregory.szorc@gmail.com> parents: 40126 diff changeset	1329 action(s) to take. The first element is the name of an action to
080419fa4fe4 wireprotov2: document client reactor actions Gregory Szorc <gregory.szorc@gmail.com> parents: 40126 diff changeset	1330 perform. The second is a data structure (usually a dict) specific to
080419fa4fe4 wireprotov2: document client reactor actions Gregory Szorc <gregory.szorc@gmail.com> parents: 40126 diff changeset	1331 that action that contains more information. e.g. if the reactor wants
080419fa4fe4 wireprotov2: document client reactor actions Gregory Szorc <gregory.szorc@gmail.com> parents: 40126 diff changeset	1332 to send frames to the server, the data structure will contain a reference
080419fa4fe4 wireprotov2: document client reactor actions Gregory Szorc <gregory.szorc@gmail.com> parents: 40126 diff changeset	1333 to those frames.
080419fa4fe4 wireprotov2: document client reactor actions Gregory Szorc <gregory.szorc@gmail.com> parents: 40126 diff changeset	1334
080419fa4fe4 wireprotov2: document client reactor actions Gregory Szorc <gregory.szorc@gmail.com> parents: 40126 diff changeset	1335 Valid actions that consumers can be instructed to take are:
080419fa4fe4 wireprotov2: document client reactor actions Gregory Szorc <gregory.szorc@gmail.com> parents: 40126 diff changeset	1336
080419fa4fe4 wireprotov2: document client reactor actions Gregory Szorc <gregory.szorc@gmail.com> parents: 40126 diff changeset	1337 noop
080419fa4fe4 wireprotov2: document client reactor actions Gregory Szorc <gregory.szorc@gmail.com> parents: 40126 diff changeset	1338 Indicates no additional action is required.
080419fa4fe4 wireprotov2: document client reactor actions Gregory Szorc <gregory.szorc@gmail.com> parents: 40126 diff changeset	1339
080419fa4fe4 wireprotov2: document client reactor actions Gregory Szorc <gregory.szorc@gmail.com> parents: 40126 diff changeset	1340 sendframes
080419fa4fe4 wireprotov2: document client reactor actions Gregory Szorc <gregory.szorc@gmail.com> parents: 40126 diff changeset	1341 Indicates that frames should be sent to the server. The ``framegen``
080419fa4fe4 wireprotov2: document client reactor actions Gregory Szorc <gregory.szorc@gmail.com> parents: 40126 diff changeset	1342 key contains a generator of frames that should be sent. The reactor
080419fa4fe4 wireprotov2: document client reactor actions Gregory Szorc <gregory.szorc@gmail.com> parents: 40126 diff changeset	1343 assumes that all frames in this generator are sent to the server.
080419fa4fe4 wireprotov2: document client reactor actions Gregory Szorc <gregory.szorc@gmail.com> parents: 40126 diff changeset	1344
080419fa4fe4 wireprotov2: document client reactor actions Gregory Szorc <gregory.szorc@gmail.com> parents: 40126 diff changeset	1345 error
080419fa4fe4 wireprotov2: document client reactor actions Gregory Szorc <gregory.szorc@gmail.com> parents: 40126 diff changeset	1346 Indicates that an error occurred. The ``message`` key contains an
080419fa4fe4 wireprotov2: document client reactor actions Gregory Szorc <gregory.szorc@gmail.com> parents: 40126 diff changeset	1347 error message describing the failure.
080419fa4fe4 wireprotov2: document client reactor actions Gregory Szorc <gregory.szorc@gmail.com> parents: 40126 diff changeset	1348
080419fa4fe4 wireprotov2: document client reactor actions Gregory Szorc <gregory.szorc@gmail.com> parents: 40126 diff changeset	1349 responsedata
080419fa4fe4 wireprotov2: document client reactor actions Gregory Szorc <gregory.szorc@gmail.com> parents: 40126 diff changeset	1350 Indicates a response to a previously-issued command was received.
080419fa4fe4 wireprotov2: document client reactor actions Gregory Szorc <gregory.szorc@gmail.com> parents: 40126 diff changeset	1351
080419fa4fe4 wireprotov2: document client reactor actions Gregory Szorc <gregory.szorc@gmail.com> parents: 40126 diff changeset	1352 The ``request`` key contains the ``commandrequest`` instance that
080419fa4fe4 wireprotov2: document client reactor actions Gregory Szorc <gregory.szorc@gmail.com> parents: 40126 diff changeset	1353 represents the request this data is for.
080419fa4fe4 wireprotov2: document client reactor actions Gregory Szorc <gregory.szorc@gmail.com> parents: 40126 diff changeset	1354
080419fa4fe4 wireprotov2: document client reactor actions Gregory Szorc <gregory.szorc@gmail.com> parents: 40126 diff changeset	1355 The ``data`` key contains the decoded data from the server.
080419fa4fe4 wireprotov2: document client reactor actions Gregory Szorc <gregory.szorc@gmail.com> parents: 40126 diff changeset	1356
080419fa4fe4 wireprotov2: document client reactor actions Gregory Szorc <gregory.szorc@gmail.com> parents: 40126 diff changeset	1357 ``expectmore`` and ``eos`` evaluate to True when more response data
080419fa4fe4 wireprotov2: document client reactor actions Gregory Szorc <gregory.szorc@gmail.com> parents: 40126 diff changeset	1358 is expected to follow or we're at the end of the response stream,
080419fa4fe4 wireprotov2: document client reactor actions Gregory Szorc <gregory.szorc@gmail.com> parents: 40126 diff changeset	1359 respectively.
37543 01361be9e2dc wireproto: introduce a reactor for client-side state Gregory Szorc <gregory.szorc@gmail.com> parents: 37485 diff changeset	1360 """
40129 293835e0fff7 wireprotov2: pass ui into clientreactor and serverreactor Gregory Szorc <gregory.szorc@gmail.com> parents: 40128 diff changeset	1361 def __init__(self, ui, hasmultiplesend=False, buffersends=True):
37543 01361be9e2dc wireproto: introduce a reactor for client-side state Gregory Szorc <gregory.szorc@gmail.com> parents: 37485 diff changeset	1362 """Create a new instance.
01361be9e2dc wireproto: introduce a reactor for client-side state Gregory Szorc <gregory.szorc@gmail.com> parents: 37485 diff changeset	1363
01361be9e2dc wireproto: introduce a reactor for client-side state Gregory Szorc <gregory.szorc@gmail.com> parents: 37485 diff changeset	1364 ``hasmultiplesend`` indicates whether multiple sends are supported
01361be9e2dc wireproto: introduce a reactor for client-side state Gregory Szorc <gregory.szorc@gmail.com> parents: 37485 diff changeset	1365 by the transport. When True, it is possible to send commands immediately
01361be9e2dc wireproto: introduce a reactor for client-side state Gregory Szorc <gregory.szorc@gmail.com> parents: 37485 diff changeset	1366 instead of buffering until the caller signals an intent to finish a
01361be9e2dc wireproto: introduce a reactor for client-side state Gregory Szorc <gregory.szorc@gmail.com> parents: 37485 diff changeset	1367 send operation.
01361be9e2dc wireproto: introduce a reactor for client-side state Gregory Szorc <gregory.szorc@gmail.com> parents: 37485 diff changeset	1368
01361be9e2dc wireproto: introduce a reactor for client-side state Gregory Szorc <gregory.szorc@gmail.com> parents: 37485 diff changeset	1369 ``buffercommands`` indicates whether sends should be buffered until the
01361be9e2dc wireproto: introduce a reactor for client-side state Gregory Szorc <gregory.szorc@gmail.com> parents: 37485 diff changeset	1370 last request has been issued.
01361be9e2dc wireproto: introduce a reactor for client-side state Gregory Szorc <gregory.szorc@gmail.com> parents: 37485 diff changeset	1371 """
40129 293835e0fff7 wireprotov2: pass ui into clientreactor and serverreactor Gregory Szorc <gregory.szorc@gmail.com> parents: 40128 diff changeset	1372 self._ui = ui
37543 01361be9e2dc wireproto: introduce a reactor for client-side state Gregory Szorc <gregory.szorc@gmail.com> parents: 37485 diff changeset	1373 self._hasmultiplesend = hasmultiplesend
01361be9e2dc wireproto: introduce a reactor for client-side state Gregory Szorc <gregory.szorc@gmail.com> parents: 37485 diff changeset	1374 self._buffersends = buffersends
01361be9e2dc wireproto: introduce a reactor for client-side state Gregory Szorc <gregory.szorc@gmail.com> parents: 37485 diff changeset	1375
01361be9e2dc wireproto: introduce a reactor for client-side state Gregory Szorc <gregory.szorc@gmail.com> parents: 37485 diff changeset	1376 self._canissuecommands = True
01361be9e2dc wireproto: introduce a reactor for client-side state Gregory Szorc <gregory.szorc@gmail.com> parents: 37485 diff changeset	1377 self._cansend = True
01361be9e2dc wireproto: introduce a reactor for client-side state Gregory Szorc <gregory.szorc@gmail.com> parents: 37485 diff changeset	1378
01361be9e2dc wireproto: introduce a reactor for client-side state Gregory Szorc <gregory.szorc@gmail.com> parents: 37485 diff changeset	1379 self._nextrequestid = 1
01361be9e2dc wireproto: introduce a reactor for client-side state Gregory Szorc <gregory.szorc@gmail.com> parents: 37485 diff changeset	1380 # We only support a single outgoing stream for now.
40130 5d44c4d1d516 wireprotov2: establish dedicated classes for input and output streams Gregory Szorc <gregory.szorc@gmail.com> parents: 40129 diff changeset	1381 self._outgoingstream = outputstream(1)
37543 01361be9e2dc wireproto: introduce a reactor for client-side state Gregory Szorc <gregory.szorc@gmail.com> parents: 37485 diff changeset	1382 self._pendingrequests = collections.deque()
01361be9e2dc wireproto: introduce a reactor for client-side state Gregory Szorc <gregory.szorc@gmail.com> parents: 37485 diff changeset	1383 self._activerequests = {}
37544 55b5ba8d4e68 wireproto: client reactor support for receiving frames Gregory Szorc <gregory.szorc@gmail.com> parents: 37543 diff changeset	1384 self._incomingstreams = {}
40128 57782791b7e9 wireprotov2: handle stream encoding settings frames Gregory Szorc <gregory.szorc@gmail.com> parents: 40127 diff changeset	1385 self._streamsettingsdecoders = {}
37543 01361be9e2dc wireproto: introduce a reactor for client-side state Gregory Szorc <gregory.szorc@gmail.com> parents: 37485 diff changeset	1386
40024 86b22a4cfab1 wireprotov2: client support for advertising redirect targets Gregory Szorc <gregory.szorc@gmail.com> parents: 40020 diff changeset	1387 def callcommand(self, name, args, datafh=None, redirect=None):
37543 01361be9e2dc wireproto: introduce a reactor for client-side state Gregory Szorc <gregory.szorc@gmail.com> parents: 37485 diff changeset	1388 """Request that a command be executed.
01361be9e2dc wireproto: introduce a reactor for client-side state Gregory Szorc <gregory.szorc@gmail.com> parents: 37485 diff changeset	1389
01361be9e2dc wireproto: introduce a reactor for client-side state Gregory Szorc <gregory.szorc@gmail.com> parents: 37485 diff changeset	1390 Receives the command name, a dict of arguments to pass to the command,
01361be9e2dc wireproto: introduce a reactor for client-side state Gregory Szorc <gregory.szorc@gmail.com> parents: 37485 diff changeset	1391 and an optional file object containing the raw data for the command.
01361be9e2dc wireproto: introduce a reactor for client-side state Gregory Szorc <gregory.szorc@gmail.com> parents: 37485 diff changeset	1392
01361be9e2dc wireproto: introduce a reactor for client-side state Gregory Szorc <gregory.szorc@gmail.com> parents: 37485 diff changeset	1393 Returns a 3-tuple of (request, action, action data).
01361be9e2dc wireproto: introduce a reactor for client-side state Gregory Szorc <gregory.szorc@gmail.com> parents: 37485 diff changeset	1394 """
01361be9e2dc wireproto: introduce a reactor for client-side state Gregory Szorc <gregory.szorc@gmail.com> parents: 37485 diff changeset	1395 if not self._canissuecommands:
01361be9e2dc wireproto: introduce a reactor for client-side state Gregory Szorc <gregory.szorc@gmail.com> parents: 37485 diff changeset	1396 raise error.ProgrammingError('cannot issue new commands')
01361be9e2dc wireproto: introduce a reactor for client-side state Gregory Szorc <gregory.szorc@gmail.com> parents: 37485 diff changeset	1397
01361be9e2dc wireproto: introduce a reactor for client-side state Gregory Szorc <gregory.szorc@gmail.com> parents: 37485 diff changeset	1398 requestid = self._nextrequestid
01361be9e2dc wireproto: introduce a reactor for client-side state Gregory Szorc <gregory.szorc@gmail.com> parents: 37485 diff changeset	1399 self._nextrequestid += 2
01361be9e2dc wireproto: introduce a reactor for client-side state Gregory Szorc <gregory.szorc@gmail.com> parents: 37485 diff changeset	1400
40024 86b22a4cfab1 wireprotov2: client support for advertising redirect targets Gregory Szorc <gregory.szorc@gmail.com> parents: 40020 diff changeset	1401 request = commandrequest(requestid, name, args, datafh=datafh,
86b22a4cfab1 wireprotov2: client support for advertising redirect targets Gregory Szorc <gregory.szorc@gmail.com> parents: 40020 diff changeset	1402 redirect=redirect)
37543 01361be9e2dc wireproto: introduce a reactor for client-side state Gregory Szorc <gregory.szorc@gmail.com> parents: 37485 diff changeset	1403
01361be9e2dc wireproto: introduce a reactor for client-side state Gregory Szorc <gregory.szorc@gmail.com> parents: 37485 diff changeset	1404 if self._buffersends:
01361be9e2dc wireproto: introduce a reactor for client-side state Gregory Szorc <gregory.szorc@gmail.com> parents: 37485 diff changeset	1405 self._pendingrequests.append(request)
01361be9e2dc wireproto: introduce a reactor for client-side state Gregory Szorc <gregory.szorc@gmail.com> parents: 37485 diff changeset	1406 return request, 'noop', {}
01361be9e2dc wireproto: introduce a reactor for client-side state Gregory Szorc <gregory.szorc@gmail.com> parents: 37485 diff changeset	1407 else:
01361be9e2dc wireproto: introduce a reactor for client-side state Gregory Szorc <gregory.szorc@gmail.com> parents: 37485 diff changeset	1408 if not self._cansend:
01361be9e2dc wireproto: introduce a reactor for client-side state Gregory Szorc <gregory.szorc@gmail.com> parents: 37485 diff changeset	1409 raise error.ProgrammingError('sends cannot be performed on '
01361be9e2dc wireproto: introduce a reactor for client-side state Gregory Szorc <gregory.szorc@gmail.com> parents: 37485 diff changeset	1410 'this instance')
01361be9e2dc wireproto: introduce a reactor for client-side state Gregory Szorc <gregory.szorc@gmail.com> parents: 37485 diff changeset	1411
01361be9e2dc wireproto: introduce a reactor for client-side state Gregory Szorc <gregory.szorc@gmail.com> parents: 37485 diff changeset	1412 if not self._hasmultiplesend:
01361be9e2dc wireproto: introduce a reactor for client-side state Gregory Szorc <gregory.szorc@gmail.com> parents: 37485 diff changeset	1413 self._cansend = False
01361be9e2dc wireproto: introduce a reactor for client-side state Gregory Szorc <gregory.szorc@gmail.com> parents: 37485 diff changeset	1414 self._canissuecommands = False
01361be9e2dc wireproto: introduce a reactor for client-side state Gregory Szorc <gregory.szorc@gmail.com> parents: 37485 diff changeset	1415
01361be9e2dc wireproto: introduce a reactor for client-side state Gregory Szorc <gregory.szorc@gmail.com> parents: 37485 diff changeset	1416 return request, 'sendframes', {
01361be9e2dc wireproto: introduce a reactor for client-side state Gregory Szorc <gregory.szorc@gmail.com> parents: 37485 diff changeset	1417 'framegen': self._makecommandframes(request),
01361be9e2dc wireproto: introduce a reactor for client-side state Gregory Szorc <gregory.szorc@gmail.com> parents: 37485 diff changeset	1418 }
01361be9e2dc wireproto: introduce a reactor for client-side state Gregory Szorc <gregory.szorc@gmail.com> parents: 37485 diff changeset	1419
01361be9e2dc wireproto: introduce a reactor for client-side state Gregory Szorc <gregory.szorc@gmail.com> parents: 37485 diff changeset	1420 def flushcommands(self):
01361be9e2dc wireproto: introduce a reactor for client-side state Gregory Szorc <gregory.szorc@gmail.com> parents: 37485 diff changeset	1421 """Request that all queued commands be sent.
01361be9e2dc wireproto: introduce a reactor for client-side state Gregory Szorc <gregory.szorc@gmail.com> parents: 37485 diff changeset	1422
01361be9e2dc wireproto: introduce a reactor for client-side state Gregory Szorc <gregory.szorc@gmail.com> parents: 37485 diff changeset	1423 If any commands are buffered, this will instruct the caller to send
01361be9e2dc wireproto: introduce a reactor for client-side state Gregory Szorc <gregory.szorc@gmail.com> parents: 37485 diff changeset	1424 them over the wire. If no commands are buffered it instructs the client
01361be9e2dc wireproto: introduce a reactor for client-side state Gregory Szorc <gregory.szorc@gmail.com> parents: 37485 diff changeset	1425 to no-op.
01361be9e2dc wireproto: introduce a reactor for client-side state Gregory Szorc <gregory.szorc@gmail.com> parents: 37485 diff changeset	1426
01361be9e2dc wireproto: introduce a reactor for client-side state Gregory Szorc <gregory.szorc@gmail.com> parents: 37485 diff changeset	1427 If instances aren't configured for multiple sends, no new command
01361be9e2dc wireproto: introduce a reactor for client-side state Gregory Szorc <gregory.szorc@gmail.com> parents: 37485 diff changeset	1428 requests are allowed after this is called.
01361be9e2dc wireproto: introduce a reactor for client-side state Gregory Szorc <gregory.szorc@gmail.com> parents: 37485 diff changeset	1429 """
01361be9e2dc wireproto: introduce a reactor for client-side state Gregory Szorc <gregory.szorc@gmail.com> parents: 37485 diff changeset	1430 if not self._pendingrequests:
01361be9e2dc wireproto: introduce a reactor for client-side state Gregory Szorc <gregory.szorc@gmail.com> parents: 37485 diff changeset	1431 return 'noop', {}
01361be9e2dc wireproto: introduce a reactor for client-side state Gregory Szorc <gregory.szorc@gmail.com> parents: 37485 diff changeset	1432
01361be9e2dc wireproto: introduce a reactor for client-side state Gregory Szorc <gregory.szorc@gmail.com> parents: 37485 diff changeset	1433 if not self._cansend:
01361be9e2dc wireproto: introduce a reactor for client-side state Gregory Szorc <gregory.szorc@gmail.com> parents: 37485 diff changeset	1434 raise error.ProgrammingError('sends cannot be performed on this '
01361be9e2dc wireproto: introduce a reactor for client-side state Gregory Szorc <gregory.szorc@gmail.com> parents: 37485 diff changeset	1435 'instance')
01361be9e2dc wireproto: introduce a reactor for client-side state Gregory Szorc <gregory.szorc@gmail.com> parents: 37485 diff changeset	1436
01361be9e2dc wireproto: introduce a reactor for client-side state Gregory Szorc <gregory.szorc@gmail.com> parents: 37485 diff changeset	1437 # If the instance only allows sending once, mark that we have fired
01361be9e2dc wireproto: introduce a reactor for client-side state Gregory Szorc <gregory.szorc@gmail.com> parents: 37485 diff changeset	1438 # our one shot.
01361be9e2dc wireproto: introduce a reactor for client-side state Gregory Szorc <gregory.szorc@gmail.com> parents: 37485 diff changeset	1439 if not self._hasmultiplesend:
01361be9e2dc wireproto: introduce a reactor for client-side state Gregory Szorc <gregory.szorc@gmail.com> parents: 37485 diff changeset	1440 self._canissuecommands = False
01361be9e2dc wireproto: introduce a reactor for client-side state Gregory Szorc <gregory.szorc@gmail.com> parents: 37485 diff changeset	1441 self._cansend = False
01361be9e2dc wireproto: introduce a reactor for client-side state Gregory Szorc <gregory.szorc@gmail.com> parents: 37485 diff changeset	1442
01361be9e2dc wireproto: introduce a reactor for client-side state Gregory Szorc <gregory.szorc@gmail.com> parents: 37485 diff changeset	1443 def makeframes():
01361be9e2dc wireproto: introduce a reactor for client-side state Gregory Szorc <gregory.szorc@gmail.com> parents: 37485 diff changeset	1444 while self._pendingrequests:
01361be9e2dc wireproto: introduce a reactor for client-side state Gregory Szorc <gregory.szorc@gmail.com> parents: 37485 diff changeset	1445 request = self._pendingrequests.popleft()
01361be9e2dc wireproto: introduce a reactor for client-side state Gregory Szorc <gregory.szorc@gmail.com> parents: 37485 diff changeset	1446 for frame in self._makecommandframes(request):
01361be9e2dc wireproto: introduce a reactor for client-side state Gregory Szorc <gregory.szorc@gmail.com> parents: 37485 diff changeset	1447 yield frame
01361be9e2dc wireproto: introduce a reactor for client-side state Gregory Szorc <gregory.szorc@gmail.com> parents: 37485 diff changeset	1448
01361be9e2dc wireproto: introduce a reactor for client-side state Gregory Szorc <gregory.szorc@gmail.com> parents: 37485 diff changeset	1449 return 'sendframes', {
01361be9e2dc wireproto: introduce a reactor for client-side state Gregory Szorc <gregory.szorc@gmail.com> parents: 37485 diff changeset	1450 'framegen': makeframes(),
01361be9e2dc wireproto: introduce a reactor for client-side state Gregory Szorc <gregory.szorc@gmail.com> parents: 37485 diff changeset	1451 }
01361be9e2dc wireproto: introduce a reactor for client-side state Gregory Szorc <gregory.szorc@gmail.com> parents: 37485 diff changeset	1452
01361be9e2dc wireproto: introduce a reactor for client-side state Gregory Szorc <gregory.szorc@gmail.com> parents: 37485 diff changeset	1453 def _makecommandframes(self, request):
01361be9e2dc wireproto: introduce a reactor for client-side state Gregory Szorc <gregory.szorc@gmail.com> parents: 37485 diff changeset	1454 """Emit frames to issue a command request.
01361be9e2dc wireproto: introduce a reactor for client-side state Gregory Szorc <gregory.szorc@gmail.com> parents: 37485 diff changeset	1455
01361be9e2dc wireproto: introduce a reactor for client-side state Gregory Szorc <gregory.szorc@gmail.com> parents: 37485 diff changeset	1456 As a side-effect, update request accounting to reflect its changed
01361be9e2dc wireproto: introduce a reactor for client-side state Gregory Szorc <gregory.szorc@gmail.com> parents: 37485 diff changeset	1457 state.
01361be9e2dc wireproto: introduce a reactor for client-side state Gregory Szorc <gregory.szorc@gmail.com> parents: 37485 diff changeset	1458 """
01361be9e2dc wireproto: introduce a reactor for client-side state Gregory Szorc <gregory.szorc@gmail.com> parents: 37485 diff changeset	1459 self._activerequests[request.requestid] = request
01361be9e2dc wireproto: introduce a reactor for client-side state Gregory Szorc <gregory.szorc@gmail.com> parents: 37485 diff changeset	1460 request.state = 'sending'
01361be9e2dc wireproto: introduce a reactor for client-side state Gregory Szorc <gregory.szorc@gmail.com> parents: 37485 diff changeset	1461
01361be9e2dc wireproto: introduce a reactor for client-side state Gregory Szorc <gregory.szorc@gmail.com> parents: 37485 diff changeset	1462 res = createcommandframes(self._outgoingstream,
01361be9e2dc wireproto: introduce a reactor for client-side state Gregory Szorc <gregory.szorc@gmail.com> parents: 37485 diff changeset	1463 request.requestid,
01361be9e2dc wireproto: introduce a reactor for client-side state Gregory Szorc <gregory.szorc@gmail.com> parents: 37485 diff changeset	1464 request.name,
01361be9e2dc wireproto: introduce a reactor for client-side state Gregory Szorc <gregory.szorc@gmail.com> parents: 37485 diff changeset	1465 request.args,
40024 86b22a4cfab1 wireprotov2: client support for advertising redirect targets Gregory Szorc <gregory.szorc@gmail.com> parents: 40020 diff changeset	1466 datafh=request.datafh,
86b22a4cfab1 wireprotov2: client support for advertising redirect targets Gregory Szorc <gregory.szorc@gmail.com> parents: 40020 diff changeset	1467 redirect=request.redirect)
37543 01361be9e2dc wireproto: introduce a reactor for client-side state Gregory Szorc <gregory.szorc@gmail.com> parents: 37485 diff changeset	1468
01361be9e2dc wireproto: introduce a reactor for client-side state Gregory Szorc <gregory.szorc@gmail.com> parents: 37485 diff changeset	1469 for frame in res:
01361be9e2dc wireproto: introduce a reactor for client-side state Gregory Szorc <gregory.szorc@gmail.com> parents: 37485 diff changeset	1470 yield frame
01361be9e2dc wireproto: introduce a reactor for client-side state Gregory Szorc <gregory.szorc@gmail.com> parents: 37485 diff changeset	1471
01361be9e2dc wireproto: introduce a reactor for client-side state Gregory Szorc <gregory.szorc@gmail.com> parents: 37485 diff changeset	1472 request.state = 'sent'
37544 55b5ba8d4e68 wireproto: client reactor support for receiving frames Gregory Szorc <gregory.szorc@gmail.com> parents: 37543 diff changeset	1473
55b5ba8d4e68 wireproto: client reactor support for receiving frames Gregory Szorc <gregory.szorc@gmail.com> parents: 37543 diff changeset	1474 def onframerecv(self, frame):
55b5ba8d4e68 wireproto: client reactor support for receiving frames Gregory Szorc <gregory.szorc@gmail.com> parents: 37543 diff changeset	1475 """Process a frame that has been received off the wire.
55b5ba8d4e68 wireproto: client reactor support for receiving frames Gregory Szorc <gregory.szorc@gmail.com> parents: 37543 diff changeset	1476
55b5ba8d4e68 wireproto: client reactor support for receiving frames Gregory Szorc <gregory.szorc@gmail.com> parents: 37543 diff changeset	1477 Returns a 2-tuple of (action, meta) describing further action the
55b5ba8d4e68 wireproto: client reactor support for receiving frames Gregory Szorc <gregory.szorc@gmail.com> parents: 37543 diff changeset	1478 caller needs to take as a result of receiving this frame.
55b5ba8d4e68 wireproto: client reactor support for receiving frames Gregory Szorc <gregory.szorc@gmail.com> parents: 37543 diff changeset	1479 """
55b5ba8d4e68 wireproto: client reactor support for receiving frames Gregory Szorc <gregory.szorc@gmail.com> parents: 37543 diff changeset	1480 if frame.streamid % 2:
55b5ba8d4e68 wireproto: client reactor support for receiving frames Gregory Szorc <gregory.szorc@gmail.com> parents: 37543 diff changeset	1481 return 'error', {
55b5ba8d4e68 wireproto: client reactor support for receiving frames Gregory Szorc <gregory.szorc@gmail.com> parents: 37543 diff changeset	1482 'message': (
55b5ba8d4e68 wireproto: client reactor support for receiving frames Gregory Szorc <gregory.szorc@gmail.com> parents: 37543 diff changeset	1483 _('received frame with odd numbered stream ID: %d') %
55b5ba8d4e68 wireproto: client reactor support for receiving frames Gregory Szorc <gregory.szorc@gmail.com> parents: 37543 diff changeset	1484 frame.streamid),
55b5ba8d4e68 wireproto: client reactor support for receiving frames Gregory Szorc <gregory.szorc@gmail.com> parents: 37543 diff changeset	1485 }
55b5ba8d4e68 wireproto: client reactor support for receiving frames Gregory Szorc <gregory.szorc@gmail.com> parents: 37543 diff changeset	1486
55b5ba8d4e68 wireproto: client reactor support for receiving frames Gregory Szorc <gregory.szorc@gmail.com> parents: 37543 diff changeset	1487 if frame.streamid not in self._incomingstreams:
55b5ba8d4e68 wireproto: client reactor support for receiving frames Gregory Szorc <gregory.szorc@gmail.com> parents: 37543 diff changeset	1488 if not frame.streamflags & STREAM_FLAG_BEGIN_STREAM:
55b5ba8d4e68 wireproto: client reactor support for receiving frames Gregory Szorc <gregory.szorc@gmail.com> parents: 37543 diff changeset	1489 return 'error', {
55b5ba8d4e68 wireproto: client reactor support for receiving frames Gregory Szorc <gregory.szorc@gmail.com> parents: 37543 diff changeset	1490 'message': _('received frame on unknown stream '
55b5ba8d4e68 wireproto: client reactor support for receiving frames Gregory Szorc <gregory.szorc@gmail.com> parents: 37543 diff changeset	1491 'without beginning of stream flag set'),
55b5ba8d4e68 wireproto: client reactor support for receiving frames Gregory Szorc <gregory.szorc@gmail.com> parents: 37543 diff changeset	1492 }
55b5ba8d4e68 wireproto: client reactor support for receiving frames Gregory Szorc <gregory.szorc@gmail.com> parents: 37543 diff changeset	1493
40130 5d44c4d1d516 wireprotov2: establish dedicated classes for input and output streams Gregory Szorc <gregory.szorc@gmail.com> parents: 40129 diff changeset	1494 self._incomingstreams[frame.streamid] = inputstream(
5d44c4d1d516 wireprotov2: establish dedicated classes for input and output streams Gregory Szorc <gregory.szorc@gmail.com> parents: 40129 diff changeset	1495 frame.streamid)
37656 e6870bca1f47 wireprotoframing: record when new stream is encountered Gregory Szorc <gregory.szorc@gmail.com> parents: 37655 diff changeset	1496
37544 55b5ba8d4e68 wireproto: client reactor support for receiving frames Gregory Szorc <gregory.szorc@gmail.com> parents: 37543 diff changeset	1497 if frame.streamflags & STREAM_FLAG_ENCODING_APPLIED:
55b5ba8d4e68 wireproto: client reactor support for receiving frames Gregory Szorc <gregory.szorc@gmail.com> parents: 37543 diff changeset	1498 raise error.ProgrammingError('support for decoding stream '
55b5ba8d4e68 wireproto: client reactor support for receiving frames Gregory Szorc <gregory.szorc@gmail.com> parents: 37543 diff changeset	1499 'payloads not yet implemneted')
55b5ba8d4e68 wireproto: client reactor support for receiving frames Gregory Szorc <gregory.szorc@gmail.com> parents: 37543 diff changeset	1500
55b5ba8d4e68 wireproto: client reactor support for receiving frames Gregory Szorc <gregory.szorc@gmail.com> parents: 37543 diff changeset	1501 if frame.streamflags & STREAM_FLAG_END_STREAM:
55b5ba8d4e68 wireproto: client reactor support for receiving frames Gregory Szorc <gregory.szorc@gmail.com> parents: 37543 diff changeset	1502 del self._incomingstreams[frame.streamid]
55b5ba8d4e68 wireproto: client reactor support for receiving frames Gregory Szorc <gregory.szorc@gmail.com> parents: 37543 diff changeset	1503
40128 57782791b7e9 wireprotov2: handle stream encoding settings frames Gregory Szorc <gregory.szorc@gmail.com> parents: 40127 diff changeset	1504 if frame.typeid == FRAME_TYPE_STREAM_SETTINGS:
57782791b7e9 wireprotov2: handle stream encoding settings frames Gregory Szorc <gregory.szorc@gmail.com> parents: 40127 diff changeset	1505 return self._onstreamsettingsframe(frame)
57782791b7e9 wireprotov2: handle stream encoding settings frames Gregory Szorc <gregory.szorc@gmail.com> parents: 40127 diff changeset	1506
37544 55b5ba8d4e68 wireproto: client reactor support for receiving frames Gregory Szorc <gregory.szorc@gmail.com> parents: 37543 diff changeset	1507 if frame.requestid not in self._activerequests:
55b5ba8d4e68 wireproto: client reactor support for receiving frames Gregory Szorc <gregory.szorc@gmail.com> parents: 37543 diff changeset	1508 return 'error', {
55b5ba8d4e68 wireproto: client reactor support for receiving frames Gregory Szorc <gregory.szorc@gmail.com> parents: 37543 diff changeset	1509 'message': (_('received frame for inactive request ID: %d') %
55b5ba8d4e68 wireproto: client reactor support for receiving frames Gregory Szorc <gregory.szorc@gmail.com> parents: 37543 diff changeset	1510 frame.requestid),
55b5ba8d4e68 wireproto: client reactor support for receiving frames Gregory Szorc <gregory.szorc@gmail.com> parents: 37543 diff changeset	1511 }
55b5ba8d4e68 wireproto: client reactor support for receiving frames Gregory Szorc <gregory.szorc@gmail.com> parents: 37543 diff changeset	1512
55b5ba8d4e68 wireproto: client reactor support for receiving frames Gregory Szorc <gregory.szorc@gmail.com> parents: 37543 diff changeset	1513 request = self._activerequests[frame.requestid]
55b5ba8d4e68 wireproto: client reactor support for receiving frames Gregory Szorc <gregory.szorc@gmail.com> parents: 37543 diff changeset	1514 request.state = 'receiving'
55b5ba8d4e68 wireproto: client reactor support for receiving frames Gregory Szorc <gregory.szorc@gmail.com> parents: 37543 diff changeset	1515
55b5ba8d4e68 wireproto: client reactor support for receiving frames Gregory Szorc <gregory.szorc@gmail.com> parents: 37543 diff changeset	1516 handlers = {
37724 deff7cf7eefd wireprotov2: change frame type and name for command response Gregory Szorc <gregory.szorc@gmail.com> parents: 37723 diff changeset	1517 FRAME_TYPE_COMMAND_RESPONSE: self._oncommandresponseframe,
37726 0c184ca594bb wireprotov2: change behavior of error frame Gregory Szorc <gregory.szorc@gmail.com> parents: 37725 diff changeset	1518 FRAME_TYPE_ERROR_RESPONSE: self._onerrorresponseframe,
37544 55b5ba8d4e68 wireproto: client reactor support for receiving frames Gregory Szorc <gregory.szorc@gmail.com> parents: 37543 diff changeset	1519 }
55b5ba8d4e68 wireproto: client reactor support for receiving frames Gregory Szorc <gregory.szorc@gmail.com> parents: 37543 diff changeset	1520
55b5ba8d4e68 wireproto: client reactor support for receiving frames Gregory Szorc <gregory.szorc@gmail.com> parents: 37543 diff changeset	1521 meth = handlers.get(frame.typeid)
55b5ba8d4e68 wireproto: client reactor support for receiving frames Gregory Szorc <gregory.szorc@gmail.com> parents: 37543 diff changeset	1522 if not meth:
55b5ba8d4e68 wireproto: client reactor support for receiving frames Gregory Szorc <gregory.szorc@gmail.com> parents: 37543 diff changeset	1523 raise error.ProgrammingError('unhandled frame type: %d' %
55b5ba8d4e68 wireproto: client reactor support for receiving frames Gregory Szorc <gregory.szorc@gmail.com> parents: 37543 diff changeset	1524 frame.typeid)
55b5ba8d4e68 wireproto: client reactor support for receiving frames Gregory Szorc <gregory.szorc@gmail.com> parents: 37543 diff changeset	1525
55b5ba8d4e68 wireproto: client reactor support for receiving frames Gregory Szorc <gregory.szorc@gmail.com> parents: 37543 diff changeset	1526 return meth(request, frame)
55b5ba8d4e68 wireproto: client reactor support for receiving frames Gregory Szorc <gregory.szorc@gmail.com> parents: 37543 diff changeset	1527
40128 57782791b7e9 wireprotov2: handle stream encoding settings frames Gregory Szorc <gregory.szorc@gmail.com> parents: 40127 diff changeset	1528 def _onstreamsettingsframe(self, frame):
57782791b7e9 wireprotov2: handle stream encoding settings frames Gregory Szorc <gregory.szorc@gmail.com> parents: 40127 diff changeset	1529 assert frame.typeid == FRAME_TYPE_STREAM_SETTINGS
57782791b7e9 wireprotov2: handle stream encoding settings frames Gregory Szorc <gregory.szorc@gmail.com> parents: 40127 diff changeset	1530
57782791b7e9 wireprotov2: handle stream encoding settings frames Gregory Szorc <gregory.szorc@gmail.com> parents: 40127 diff changeset	1531 more = frame.flags & FLAG_STREAM_ENCODING_SETTINGS_CONTINUATION
57782791b7e9 wireprotov2: handle stream encoding settings frames Gregory Szorc <gregory.szorc@gmail.com> parents: 40127 diff changeset	1532 eos = frame.flags & FLAG_STREAM_ENCODING_SETTINGS_EOS
57782791b7e9 wireprotov2: handle stream encoding settings frames Gregory Szorc <gregory.szorc@gmail.com> parents: 40127 diff changeset	1533
57782791b7e9 wireprotov2: handle stream encoding settings frames Gregory Szorc <gregory.szorc@gmail.com> parents: 40127 diff changeset	1534 if more and eos:
57782791b7e9 wireprotov2: handle stream encoding settings frames Gregory Szorc <gregory.szorc@gmail.com> parents: 40127 diff changeset	1535 return 'error', {
57782791b7e9 wireprotov2: handle stream encoding settings frames Gregory Szorc <gregory.szorc@gmail.com> parents: 40127 diff changeset	1536 'message': (_('stream encoding settings frame cannot have both '
57782791b7e9 wireprotov2: handle stream encoding settings frames Gregory Szorc <gregory.szorc@gmail.com> parents: 40127 diff changeset	1537 'continuation and end of stream flags set')),
57782791b7e9 wireprotov2: handle stream encoding settings frames Gregory Szorc <gregory.szorc@gmail.com> parents: 40127 diff changeset	1538 }
57782791b7e9 wireprotov2: handle stream encoding settings frames Gregory Szorc <gregory.szorc@gmail.com> parents: 40127 diff changeset	1539
57782791b7e9 wireprotov2: handle stream encoding settings frames Gregory Szorc <gregory.szorc@gmail.com> parents: 40127 diff changeset	1540 if not more and not eos:
57782791b7e9 wireprotov2: handle stream encoding settings frames Gregory Szorc <gregory.szorc@gmail.com> parents: 40127 diff changeset	1541 return 'error', {
57782791b7e9 wireprotov2: handle stream encoding settings frames Gregory Szorc <gregory.szorc@gmail.com> parents: 40127 diff changeset	1542 'message': _('stream encoding settings frame must have '
57782791b7e9 wireprotov2: handle stream encoding settings frames Gregory Szorc <gregory.szorc@gmail.com> parents: 40127 diff changeset	1543 'continuation or end of stream flag set'),
57782791b7e9 wireprotov2: handle stream encoding settings frames Gregory Szorc <gregory.szorc@gmail.com> parents: 40127 diff changeset	1544 }
57782791b7e9 wireprotov2: handle stream encoding settings frames Gregory Szorc <gregory.szorc@gmail.com> parents: 40127 diff changeset	1545
57782791b7e9 wireprotov2: handle stream encoding settings frames Gregory Szorc <gregory.szorc@gmail.com> parents: 40127 diff changeset	1546 if frame.streamid not in self._streamsettingsdecoders:
57782791b7e9 wireprotov2: handle stream encoding settings frames Gregory Szorc <gregory.szorc@gmail.com> parents: 40127 diff changeset	1547 decoder = cborutil.bufferingdecoder()
57782791b7e9 wireprotov2: handle stream encoding settings frames Gregory Szorc <gregory.szorc@gmail.com> parents: 40127 diff changeset	1548 self._streamsettingsdecoders[frame.streamid] = decoder
57782791b7e9 wireprotov2: handle stream encoding settings frames Gregory Szorc <gregory.szorc@gmail.com> parents: 40127 diff changeset	1549
57782791b7e9 wireprotov2: handle stream encoding settings frames Gregory Szorc <gregory.szorc@gmail.com> parents: 40127 diff changeset	1550 decoder = self._streamsettingsdecoders[frame.streamid]
57782791b7e9 wireprotov2: handle stream encoding settings frames Gregory Szorc <gregory.szorc@gmail.com> parents: 40127 diff changeset	1551
57782791b7e9 wireprotov2: handle stream encoding settings frames Gregory Szorc <gregory.szorc@gmail.com> parents: 40127 diff changeset	1552 try:
57782791b7e9 wireprotov2: handle stream encoding settings frames Gregory Szorc <gregory.szorc@gmail.com> parents: 40127 diff changeset	1553 decoder.decode(frame.payload)
57782791b7e9 wireprotov2: handle stream encoding settings frames Gregory Szorc <gregory.szorc@gmail.com> parents: 40127 diff changeset	1554 except Exception as e:
57782791b7e9 wireprotov2: handle stream encoding settings frames Gregory Szorc <gregory.szorc@gmail.com> parents: 40127 diff changeset	1555 return 'error', {
57782791b7e9 wireprotov2: handle stream encoding settings frames Gregory Szorc <gregory.szorc@gmail.com> parents: 40127 diff changeset	1556 'message': (_('error decoding CBOR from stream encoding '
57782791b7e9 wireprotov2: handle stream encoding settings frames Gregory Szorc <gregory.szorc@gmail.com> parents: 40127 diff changeset	1557 'settings frame: %s') %
57782791b7e9 wireprotov2: handle stream encoding settings frames Gregory Szorc <gregory.szorc@gmail.com> parents: 40127 diff changeset	1558 stringutil.forcebytestr(e)),
57782791b7e9 wireprotov2: handle stream encoding settings frames Gregory Szorc <gregory.szorc@gmail.com> parents: 40127 diff changeset	1559 }
57782791b7e9 wireprotov2: handle stream encoding settings frames Gregory Szorc <gregory.szorc@gmail.com> parents: 40127 diff changeset	1560
57782791b7e9 wireprotov2: handle stream encoding settings frames Gregory Szorc <gregory.szorc@gmail.com> parents: 40127 diff changeset	1561 if more:
57782791b7e9 wireprotov2: handle stream encoding settings frames Gregory Szorc <gregory.szorc@gmail.com> parents: 40127 diff changeset	1562 return 'noop', {}
57782791b7e9 wireprotov2: handle stream encoding settings frames Gregory Szorc <gregory.szorc@gmail.com> parents: 40127 diff changeset	1563
57782791b7e9 wireprotov2: handle stream encoding settings frames Gregory Szorc <gregory.szorc@gmail.com> parents: 40127 diff changeset	1564 assert eos
57782791b7e9 wireprotov2: handle stream encoding settings frames Gregory Szorc <gregory.szorc@gmail.com> parents: 40127 diff changeset	1565
57782791b7e9 wireprotov2: handle stream encoding settings frames Gregory Szorc <gregory.szorc@gmail.com> parents: 40127 diff changeset	1566 decoded = decoder.getavailable()
57782791b7e9 wireprotov2: handle stream encoding settings frames Gregory Szorc <gregory.szorc@gmail.com> parents: 40127 diff changeset	1567 del self._streamsettingsdecoders[frame.streamid]
57782791b7e9 wireprotov2: handle stream encoding settings frames Gregory Szorc <gregory.szorc@gmail.com> parents: 40127 diff changeset	1568
57782791b7e9 wireprotov2: handle stream encoding settings frames Gregory Szorc <gregory.szorc@gmail.com> parents: 40127 diff changeset	1569 if not decoded:
57782791b7e9 wireprotov2: handle stream encoding settings frames Gregory Szorc <gregory.szorc@gmail.com> parents: 40127 diff changeset	1570 return 'error', {
57782791b7e9 wireprotov2: handle stream encoding settings frames Gregory Szorc <gregory.szorc@gmail.com> parents: 40127 diff changeset	1571 'message': _('stream encoding settings frame did not contain '
57782791b7e9 wireprotov2: handle stream encoding settings frames Gregory Szorc <gregory.szorc@gmail.com> parents: 40127 diff changeset	1572 'CBOR data'),
57782791b7e9 wireprotov2: handle stream encoding settings frames Gregory Szorc <gregory.szorc@gmail.com> parents: 40127 diff changeset	1573 }
57782791b7e9 wireprotov2: handle stream encoding settings frames Gregory Szorc <gregory.szorc@gmail.com> parents: 40127 diff changeset	1574
57782791b7e9 wireprotov2: handle stream encoding settings frames Gregory Szorc <gregory.szorc@gmail.com> parents: 40127 diff changeset	1575 try:
57782791b7e9 wireprotov2: handle stream encoding settings frames Gregory Szorc <gregory.szorc@gmail.com> parents: 40127 diff changeset	1576 self._incomingstreams[frame.streamid].setdecoder(decoded[0],
57782791b7e9 wireprotov2: handle stream encoding settings frames Gregory Szorc <gregory.szorc@gmail.com> parents: 40127 diff changeset	1577 decoded[1:])
57782791b7e9 wireprotov2: handle stream encoding settings frames Gregory Szorc <gregory.szorc@gmail.com> parents: 40127 diff changeset	1578 except Exception as e:
57782791b7e9 wireprotov2: handle stream encoding settings frames Gregory Szorc <gregory.szorc@gmail.com> parents: 40127 diff changeset	1579 return 'error', {
57782791b7e9 wireprotov2: handle stream encoding settings frames Gregory Szorc <gregory.szorc@gmail.com> parents: 40127 diff changeset	1580 'message': (_('error setting stream decoder: %s') %
57782791b7e9 wireprotov2: handle stream encoding settings frames Gregory Szorc <gregory.szorc@gmail.com> parents: 40127 diff changeset	1581 stringutil.forcebytestr(e)),
57782791b7e9 wireprotov2: handle stream encoding settings frames Gregory Szorc <gregory.szorc@gmail.com> parents: 40127 diff changeset	1582 }
57782791b7e9 wireprotov2: handle stream encoding settings frames Gregory Szorc <gregory.szorc@gmail.com> parents: 40127 diff changeset	1583
57782791b7e9 wireprotov2: handle stream encoding settings frames Gregory Szorc <gregory.szorc@gmail.com> parents: 40127 diff changeset	1584 return 'noop', {}
57782791b7e9 wireprotov2: handle stream encoding settings frames Gregory Szorc <gregory.szorc@gmail.com> parents: 40127 diff changeset	1585
37724 deff7cf7eefd wireprotov2: change frame type and name for command response Gregory Szorc <gregory.szorc@gmail.com> parents: 37723 diff changeset	1586 def _oncommandresponseframe(self, request, frame):
deff7cf7eefd wireprotov2: change frame type and name for command response Gregory Szorc <gregory.szorc@gmail.com> parents: 37723 diff changeset	1587 if frame.flags & FLAG_COMMAND_RESPONSE_EOS:
37544 55b5ba8d4e68 wireproto: client reactor support for receiving frames Gregory Szorc <gregory.szorc@gmail.com> parents: 37543 diff changeset	1588 request.state = 'received'
55b5ba8d4e68 wireproto: client reactor support for receiving frames Gregory Szorc <gregory.szorc@gmail.com> parents: 37543 diff changeset	1589 del self._activerequests[request.requestid]
55b5ba8d4e68 wireproto: client reactor support for receiving frames Gregory Szorc <gregory.szorc@gmail.com> parents: 37543 diff changeset	1590
55b5ba8d4e68 wireproto: client reactor support for receiving frames Gregory Szorc <gregory.szorc@gmail.com> parents: 37543 diff changeset	1591 return 'responsedata', {
55b5ba8d4e68 wireproto: client reactor support for receiving frames Gregory Szorc <gregory.szorc@gmail.com> parents: 37543 diff changeset	1592 'request': request,
37724 deff7cf7eefd wireprotov2: change frame type and name for command response Gregory Szorc <gregory.szorc@gmail.com> parents: 37723 diff changeset	1593 'expectmore': frame.flags & FLAG_COMMAND_RESPONSE_CONTINUATION,
deff7cf7eefd wireprotov2: change frame type and name for command response Gregory Szorc <gregory.szorc@gmail.com> parents: 37723 diff changeset	1594 'eos': frame.flags & FLAG_COMMAND_RESPONSE_EOS,
37544 55b5ba8d4e68 wireproto: client reactor support for receiving frames Gregory Szorc <gregory.szorc@gmail.com> parents: 37543 diff changeset	1595 'data': frame.payload,
55b5ba8d4e68 wireproto: client reactor support for receiving frames Gregory Szorc <gregory.szorc@gmail.com> parents: 37543 diff changeset	1596 }
37726 0c184ca594bb wireprotov2: change behavior of error frame Gregory Szorc <gregory.szorc@gmail.com> parents: 37725 diff changeset	1597
0c184ca594bb wireprotov2: change behavior of error frame Gregory Szorc <gregory.szorc@gmail.com> parents: 37725 diff changeset	1598 def _onerrorresponseframe(self, request, frame):
0c184ca594bb wireprotov2: change behavior of error frame Gregory Szorc <gregory.szorc@gmail.com> parents: 37725 diff changeset	1599 request.state = 'errored'
0c184ca594bb wireprotov2: change behavior of error frame Gregory Szorc <gregory.szorc@gmail.com> parents: 37725 diff changeset	1600 del self._activerequests[request.requestid]
0c184ca594bb wireprotov2: change behavior of error frame Gregory Szorc <gregory.szorc@gmail.com> parents: 37725 diff changeset	1601
0c184ca594bb wireprotov2: change behavior of error frame Gregory Szorc <gregory.szorc@gmail.com> parents: 37725 diff changeset	1602 # The payload should be a CBOR map.
39446 36f487a332ad wireprotoframing: use our CBOR module Gregory Szorc <gregory.szorc@gmail.com> parents: 37728 diff changeset	1603 m = cborutil.decodeall(frame.payload)[0]
37726 0c184ca594bb wireprotov2: change behavior of error frame Gregory Szorc <gregory.szorc@gmail.com> parents: 37725 diff changeset	1604
0c184ca594bb wireprotov2: change behavior of error frame Gregory Szorc <gregory.szorc@gmail.com> parents: 37725 diff changeset	1605 return 'error', {
0c184ca594bb wireprotov2: change behavior of error frame Gregory Szorc <gregory.szorc@gmail.com> parents: 37725 diff changeset	1606 'request': request,
0c184ca594bb wireprotov2: change behavior of error frame Gregory Szorc <gregory.szorc@gmail.com> parents: 37725 diff changeset	1607 'type': m['type'],
0c184ca594bb wireprotov2: change behavior of error frame Gregory Szorc <gregory.szorc@gmail.com> parents: 37725 diff changeset	1608 'message': m['message'],
0c184ca594bb wireprotov2: change behavior of error frame Gregory Szorc <gregory.szorc@gmail.com> parents: 37725 diff changeset	1609 }

Mercurial > public > mercurial-scm > hg

annotate mercurial/wireprotoframing.py @ 40130:5d44c4d1d516