mercurial-scm/hg-stable: mercurial/wireprotoframing.py annotate

annotate mercurial/wireprotoframing.py @ 40026:b099e6032f38

wireprotov2: server support for sending content redirects A "content redirect" can be sent in place of inline response content. In terms of code, we model a content redirect as a special type of response object holding the attributes describing that redirect. Sending a content redirect thus becomes as simple as the object emission layer sending an instance of that type. A cacher using externally-addressable content storage could replace the outgoing object stream with an object advertising its location. The bulk of the code in this commit is teaching the output layer which handles the object stream to recognize alternate location objects. The rules are that if an alternate location object is present, it must be the first and only object in the object stream. Otherwise the server emits an error. Differential Revision: https://phab.mercurial-scm.org/D4777

author	Gregory Szorc <gregory.szorc@gmail.com>
date	Wed, 26 Sep 2018 18:07:55 -0700
parents	86b22a4cfab1
children	e2fe1074024c

rev	line source
37054 40206e227412 wireproto: define and implement protocol for issuing requests Gregory Szorc <gregory.szorc@gmail.com> parents: diff changeset	1 # wireprotoframing.py - unified framing protocol for wire protocol
40206e227412 wireproto: define and implement protocol for issuing requests Gregory Szorc <gregory.szorc@gmail.com> parents: diff changeset	2 #
40206e227412 wireproto: define and implement protocol for issuing requests Gregory Szorc <gregory.szorc@gmail.com> parents: diff changeset	3 # Copyright 2018 Gregory Szorc <gregory.szorc@gmail.com>
40206e227412 wireproto: define and implement protocol for issuing requests Gregory Szorc <gregory.szorc@gmail.com> parents: diff changeset	4 #
40206e227412 wireproto: define and implement protocol for issuing requests Gregory Szorc <gregory.szorc@gmail.com> parents: diff changeset	5 # This software may be used and distributed according to the terms of the
40206e227412 wireproto: define and implement protocol for issuing requests Gregory Szorc <gregory.szorc@gmail.com> parents: diff changeset	6 # GNU General Public License version 2 or any later version.
40206e227412 wireproto: define and implement protocol for issuing requests Gregory Szorc <gregory.szorc@gmail.com> parents: diff changeset	7
40206e227412 wireproto: define and implement protocol for issuing requests Gregory Szorc <gregory.szorc@gmail.com> parents: diff changeset	8 # This file contains functionality to support the unified frame-based wire
40206e227412 wireproto: define and implement protocol for issuing requests Gregory Szorc <gregory.szorc@gmail.com> parents: diff changeset	9 # protocol. For details about the protocol, see
40206e227412 wireproto: define and implement protocol for issuing requests Gregory Szorc <gregory.szorc@gmail.com> parents: diff changeset	10 # `hg help internals.wireprotocol`.
40206e227412 wireproto: define and implement protocol for issuing requests Gregory Szorc <gregory.szorc@gmail.com> parents: diff changeset	11
40206e227412 wireproto: define and implement protocol for issuing requests Gregory Szorc <gregory.szorc@gmail.com> parents: diff changeset	12 from __future__ import absolute_import
40206e227412 wireproto: define and implement protocol for issuing requests Gregory Szorc <gregory.szorc@gmail.com> parents: diff changeset	13
37543 01361be9e2dc wireproto: introduce a reactor for client-side state Gregory Szorc <gregory.szorc@gmail.com> parents: 37485 diff changeset	14 import collections
37054 40206e227412 wireproto: define and implement protocol for issuing requests Gregory Szorc <gregory.szorc@gmail.com> parents: diff changeset	15 import struct
40206e227412 wireproto: define and implement protocol for issuing requests Gregory Szorc <gregory.szorc@gmail.com> parents: diff changeset	16
37055 8c3c47362934 wireproto: implement basic frame reading and processing Gregory Szorc <gregory.szorc@gmail.com> parents: 37054 diff changeset	17 from .i18n import _
37064 884a0c1604ad wireproto: define attr-based classes for representing frames Gregory Szorc <gregory.szorc@gmail.com> parents: 37063 diff changeset	18 from .thirdparty import (
884a0c1604ad wireproto: define attr-based classes for representing frames Gregory Szorc <gregory.szorc@gmail.com> parents: 37063 diff changeset	19 attr,
884a0c1604ad wireproto: define attr-based classes for representing frames Gregory Szorc <gregory.szorc@gmail.com> parents: 37063 diff changeset	20 )
37054 40206e227412 wireproto: define and implement protocol for issuing requests Gregory Szorc <gregory.szorc@gmail.com> parents: diff changeset	21 from . import (
37474 d33997123ea5 py3: system-stringify repr(frame) Yuya Nishihara <yuya@tcha.org> parents: 37473 diff changeset	22 encoding,
37055 8c3c47362934 wireproto: implement basic frame reading and processing Gregory Szorc <gregory.szorc@gmail.com> parents: 37054 diff changeset	23 error,
40026 b099e6032f38 wireprotov2: server support for sending content redirects Gregory Szorc <gregory.szorc@gmail.com> parents: 40025 diff changeset	24 pycompat,
37054 40206e227412 wireproto: define and implement protocol for issuing requests Gregory Szorc <gregory.szorc@gmail.com> parents: diff changeset	25 util,
40021 ed919b90acda wireprotov2: define type to represent pre-encoded object Gregory Szorc <gregory.szorc@gmail.com> parents: 39843 diff changeset	26 wireprototypes,
37054 40206e227412 wireproto: define and implement protocol for issuing requests Gregory Szorc <gregory.szorc@gmail.com> parents: diff changeset	27 )
37087 f0b6fbea00cf stringutil: bulk-replace call sites to point to new module Yuya Nishihara <yuya@tcha.org> parents: 37066 diff changeset	28 from .utils import (
39465 36f487a332ad wireprotoframing: use our CBOR module Gregory Szorc <gregory.szorc@gmail.com> parents: 37728 diff changeset	29 cborutil,
37087 f0b6fbea00cf stringutil: bulk-replace call sites to point to new module Yuya Nishihara <yuya@tcha.org> parents: 37066 diff changeset	30 stringutil,
f0b6fbea00cf stringutil: bulk-replace call sites to point to new module Yuya Nishihara <yuya@tcha.org> parents: 37066 diff changeset	31 )
37054 40206e227412 wireproto: define and implement protocol for issuing requests Gregory Szorc <gregory.szorc@gmail.com> parents: diff changeset	32
37288 9bfcbe4f4745 wireproto: add streams to frame-based protocol Gregory Szorc <gregory.szorc@gmail.com> parents: 37287 diff changeset	33 FRAME_HEADER_SIZE = 8
37054 40206e227412 wireproto: define and implement protocol for issuing requests Gregory Szorc <gregory.szorc@gmail.com> parents: diff changeset	34 DEFAULT_MAX_FRAME_SIZE = 32768
40206e227412 wireproto: define and implement protocol for issuing requests Gregory Szorc <gregory.szorc@gmail.com> parents: diff changeset	35
37288 9bfcbe4f4745 wireproto: add streams to frame-based protocol Gregory Szorc <gregory.szorc@gmail.com> parents: 37287 diff changeset	36 STREAM_FLAG_BEGIN_STREAM = 0x01
9bfcbe4f4745 wireproto: add streams to frame-based protocol Gregory Szorc <gregory.szorc@gmail.com> parents: 37287 diff changeset	37 STREAM_FLAG_END_STREAM = 0x02
9bfcbe4f4745 wireproto: add streams to frame-based protocol Gregory Szorc <gregory.szorc@gmail.com> parents: 37287 diff changeset	38 STREAM_FLAG_ENCODING_APPLIED = 0x04
9bfcbe4f4745 wireproto: add streams to frame-based protocol Gregory Szorc <gregory.szorc@gmail.com> parents: 37287 diff changeset	39
9bfcbe4f4745 wireproto: add streams to frame-based protocol Gregory Szorc <gregory.szorc@gmail.com> parents: 37287 diff changeset	40 STREAM_FLAGS = {
9bfcbe4f4745 wireproto: add streams to frame-based protocol Gregory Szorc <gregory.szorc@gmail.com> parents: 37287 diff changeset	41 b'stream-begin': STREAM_FLAG_BEGIN_STREAM,
9bfcbe4f4745 wireproto: add streams to frame-based protocol Gregory Szorc <gregory.szorc@gmail.com> parents: 37287 diff changeset	42 b'stream-end': STREAM_FLAG_END_STREAM,
9bfcbe4f4745 wireproto: add streams to frame-based protocol Gregory Szorc <gregory.szorc@gmail.com> parents: 37287 diff changeset	43 b'encoded': STREAM_FLAG_ENCODING_APPLIED,
9bfcbe4f4745 wireproto: add streams to frame-based protocol Gregory Szorc <gregory.szorc@gmail.com> parents: 37287 diff changeset	44 }
9bfcbe4f4745 wireproto: add streams to frame-based protocol Gregory Szorc <gregory.szorc@gmail.com> parents: 37287 diff changeset	45
37292 3d0e2cd86e05 wireproto: use CBOR for command requests Gregory Szorc <gregory.szorc@gmail.com> parents: 37291 diff changeset	46 FRAME_TYPE_COMMAND_REQUEST = 0x01
37723 e8fba6d578f0 wireprotov2: change frame type value for command data Gregory Szorc <gregory.szorc@gmail.com> parents: 37722 diff changeset	47 FRAME_TYPE_COMMAND_DATA = 0x02
37724 deff7cf7eefd wireprotov2: change frame type and name for command response Gregory Szorc <gregory.szorc@gmail.com> parents: 37723 diff changeset	48 FRAME_TYPE_COMMAND_RESPONSE = 0x03
37058 61393f888dfe wireproto: define and implement responses in framing protocol Gregory Szorc <gregory.szorc@gmail.com> parents: 37055 diff changeset	49 FRAME_TYPE_ERROR_RESPONSE = 0x05
37063 0a6c5cc09a88 wireproto: define human output side channel frame Gregory Szorc <gregory.szorc@gmail.com> parents: 37061 diff changeset	50 FRAME_TYPE_TEXT_OUTPUT = 0x06
37291 b0041036214e wireproto: define frame to represent progress updates Gregory Szorc <gregory.szorc@gmail.com> parents: 37290 diff changeset	51 FRAME_TYPE_PROGRESS = 0x07
37288 9bfcbe4f4745 wireproto: add streams to frame-based protocol Gregory Szorc <gregory.szorc@gmail.com> parents: 37287 diff changeset	52 FRAME_TYPE_STREAM_SETTINGS = 0x08
37054 40206e227412 wireproto: define and implement protocol for issuing requests Gregory Szorc <gregory.szorc@gmail.com> parents: diff changeset	53
40206e227412 wireproto: define and implement protocol for issuing requests Gregory Szorc <gregory.szorc@gmail.com> parents: diff changeset	54 FRAME_TYPES = {
37292 3d0e2cd86e05 wireproto: use CBOR for command requests Gregory Szorc <gregory.szorc@gmail.com> parents: 37291 diff changeset	55 b'command-request': FRAME_TYPE_COMMAND_REQUEST,
37054 40206e227412 wireproto: define and implement protocol for issuing requests Gregory Szorc <gregory.szorc@gmail.com> parents: diff changeset	56 b'command-data': FRAME_TYPE_COMMAND_DATA,
37724 deff7cf7eefd wireprotov2: change frame type and name for command response Gregory Szorc <gregory.szorc@gmail.com> parents: 37723 diff changeset	57 b'command-response': FRAME_TYPE_COMMAND_RESPONSE,
37058 61393f888dfe wireproto: define and implement responses in framing protocol Gregory Szorc <gregory.szorc@gmail.com> parents: 37055 diff changeset	58 b'error-response': FRAME_TYPE_ERROR_RESPONSE,
37063 0a6c5cc09a88 wireproto: define human output side channel frame Gregory Szorc <gregory.szorc@gmail.com> parents: 37061 diff changeset	59 b'text-output': FRAME_TYPE_TEXT_OUTPUT,
37291 b0041036214e wireproto: define frame to represent progress updates Gregory Szorc <gregory.szorc@gmail.com> parents: 37290 diff changeset	60 b'progress': FRAME_TYPE_PROGRESS,
37288 9bfcbe4f4745 wireproto: add streams to frame-based protocol Gregory Szorc <gregory.szorc@gmail.com> parents: 37287 diff changeset	61 b'stream-settings': FRAME_TYPE_STREAM_SETTINGS,
37054 40206e227412 wireproto: define and implement protocol for issuing requests Gregory Szorc <gregory.szorc@gmail.com> parents: diff changeset	62 }
40206e227412 wireproto: define and implement protocol for issuing requests Gregory Szorc <gregory.szorc@gmail.com> parents: diff changeset	63
37292 3d0e2cd86e05 wireproto: use CBOR for command requests Gregory Szorc <gregory.szorc@gmail.com> parents: 37291 diff changeset	64 FLAG_COMMAND_REQUEST_NEW = 0x01
3d0e2cd86e05 wireproto: use CBOR for command requests Gregory Szorc <gregory.szorc@gmail.com> parents: 37291 diff changeset	65 FLAG_COMMAND_REQUEST_CONTINUATION = 0x02
3d0e2cd86e05 wireproto: use CBOR for command requests Gregory Szorc <gregory.szorc@gmail.com> parents: 37291 diff changeset	66 FLAG_COMMAND_REQUEST_MORE_FRAMES = 0x04
3d0e2cd86e05 wireproto: use CBOR for command requests Gregory Szorc <gregory.szorc@gmail.com> parents: 37291 diff changeset	67 FLAG_COMMAND_REQUEST_EXPECT_DATA = 0x08
37054 40206e227412 wireproto: define and implement protocol for issuing requests Gregory Szorc <gregory.szorc@gmail.com> parents: diff changeset	68
37292 3d0e2cd86e05 wireproto: use CBOR for command requests Gregory Szorc <gregory.szorc@gmail.com> parents: 37291 diff changeset	69 FLAGS_COMMAND_REQUEST = {
3d0e2cd86e05 wireproto: use CBOR for command requests Gregory Szorc <gregory.szorc@gmail.com> parents: 37291 diff changeset	70 b'new': FLAG_COMMAND_REQUEST_NEW,
3d0e2cd86e05 wireproto: use CBOR for command requests Gregory Szorc <gregory.szorc@gmail.com> parents: 37291 diff changeset	71 b'continuation': FLAG_COMMAND_REQUEST_CONTINUATION,
3d0e2cd86e05 wireproto: use CBOR for command requests Gregory Szorc <gregory.szorc@gmail.com> parents: 37291 diff changeset	72 b'more': FLAG_COMMAND_REQUEST_MORE_FRAMES,
3d0e2cd86e05 wireproto: use CBOR for command requests Gregory Szorc <gregory.szorc@gmail.com> parents: 37291 diff changeset	73 b'have-data': FLAG_COMMAND_REQUEST_EXPECT_DATA,
37054 40206e227412 wireproto: define and implement protocol for issuing requests Gregory Szorc <gregory.szorc@gmail.com> parents: diff changeset	74 }
40206e227412 wireproto: define and implement protocol for issuing requests Gregory Szorc <gregory.szorc@gmail.com> parents: diff changeset	75
40206e227412 wireproto: define and implement protocol for issuing requests Gregory Szorc <gregory.szorc@gmail.com> parents: diff changeset	76 FLAG_COMMAND_DATA_CONTINUATION = 0x01
40206e227412 wireproto: define and implement protocol for issuing requests Gregory Szorc <gregory.szorc@gmail.com> parents: diff changeset	77 FLAG_COMMAND_DATA_EOS = 0x02
40206e227412 wireproto: define and implement protocol for issuing requests Gregory Szorc <gregory.szorc@gmail.com> parents: diff changeset	78
40206e227412 wireproto: define and implement protocol for issuing requests Gregory Szorc <gregory.szorc@gmail.com> parents: diff changeset	79 FLAGS_COMMAND_DATA = {
40206e227412 wireproto: define and implement protocol for issuing requests Gregory Szorc <gregory.szorc@gmail.com> parents: diff changeset	80 b'continuation': FLAG_COMMAND_DATA_CONTINUATION,
40206e227412 wireproto: define and implement protocol for issuing requests Gregory Szorc <gregory.szorc@gmail.com> parents: diff changeset	81 b'eos': FLAG_COMMAND_DATA_EOS,
40206e227412 wireproto: define and implement protocol for issuing requests Gregory Szorc <gregory.szorc@gmail.com> parents: diff changeset	82 }
40206e227412 wireproto: define and implement protocol for issuing requests Gregory Szorc <gregory.szorc@gmail.com> parents: diff changeset	83
37724 deff7cf7eefd wireprotov2: change frame type and name for command response Gregory Szorc <gregory.szorc@gmail.com> parents: 37723 diff changeset	84 FLAG_COMMAND_RESPONSE_CONTINUATION = 0x01
deff7cf7eefd wireprotov2: change frame type and name for command response Gregory Szorc <gregory.szorc@gmail.com> parents: 37723 diff changeset	85 FLAG_COMMAND_RESPONSE_EOS = 0x02
37058 61393f888dfe wireproto: define and implement responses in framing protocol Gregory Szorc <gregory.szorc@gmail.com> parents: 37055 diff changeset	86
37724 deff7cf7eefd wireprotov2: change frame type and name for command response Gregory Szorc <gregory.szorc@gmail.com> parents: 37723 diff changeset	87 FLAGS_COMMAND_RESPONSE = {
deff7cf7eefd wireprotov2: change frame type and name for command response Gregory Szorc <gregory.szorc@gmail.com> parents: 37723 diff changeset	88 b'continuation': FLAG_COMMAND_RESPONSE_CONTINUATION,
deff7cf7eefd wireprotov2: change frame type and name for command response Gregory Szorc <gregory.szorc@gmail.com> parents: 37723 diff changeset	89 b'eos': FLAG_COMMAND_RESPONSE_EOS,
37058 61393f888dfe wireproto: define and implement responses in framing protocol Gregory Szorc <gregory.szorc@gmail.com> parents: 37055 diff changeset	90 }
61393f888dfe wireproto: define and implement responses in framing protocol Gregory Szorc <gregory.szorc@gmail.com> parents: 37055 diff changeset	91
37054 40206e227412 wireproto: define and implement protocol for issuing requests Gregory Szorc <gregory.szorc@gmail.com> parents: diff changeset	92 # Maps frame types to their available flags.
40206e227412 wireproto: define and implement protocol for issuing requests Gregory Szorc <gregory.szorc@gmail.com> parents: diff changeset	93 FRAME_TYPE_FLAGS = {
37292 3d0e2cd86e05 wireproto: use CBOR for command requests Gregory Szorc <gregory.szorc@gmail.com> parents: 37291 diff changeset	94 FRAME_TYPE_COMMAND_REQUEST: FLAGS_COMMAND_REQUEST,
37054 40206e227412 wireproto: define and implement protocol for issuing requests Gregory Szorc <gregory.szorc@gmail.com> parents: diff changeset	95 FRAME_TYPE_COMMAND_DATA: FLAGS_COMMAND_DATA,
37724 deff7cf7eefd wireprotov2: change frame type and name for command response Gregory Szorc <gregory.szorc@gmail.com> parents: 37723 diff changeset	96 FRAME_TYPE_COMMAND_RESPONSE: FLAGS_COMMAND_RESPONSE,
37726 0c184ca594bb wireprotov2: change behavior of error frame Gregory Szorc <gregory.szorc@gmail.com> parents: 37725 diff changeset	97 FRAME_TYPE_ERROR_RESPONSE: {},
37063 0a6c5cc09a88 wireproto: define human output side channel frame Gregory Szorc <gregory.szorc@gmail.com> parents: 37061 diff changeset	98 FRAME_TYPE_TEXT_OUTPUT: {},
37291 b0041036214e wireproto: define frame to represent progress updates Gregory Szorc <gregory.szorc@gmail.com> parents: 37290 diff changeset	99 FRAME_TYPE_PROGRESS: {},
37288 9bfcbe4f4745 wireproto: add streams to frame-based protocol Gregory Szorc <gregory.szorc@gmail.com> parents: 37287 diff changeset	100 FRAME_TYPE_STREAM_SETTINGS: {},
37054 40206e227412 wireproto: define and implement protocol for issuing requests Gregory Szorc <gregory.szorc@gmail.com> parents: diff changeset	101 }
40206e227412 wireproto: define and implement protocol for issuing requests Gregory Szorc <gregory.szorc@gmail.com> parents: diff changeset	102
37292 3d0e2cd86e05 wireproto: use CBOR for command requests Gregory Szorc <gregory.szorc@gmail.com> parents: 37291 diff changeset	103 ARGUMENT_RECORD_HEADER = struct.Struct(r'<HH')
37054 40206e227412 wireproto: define and implement protocol for issuing requests Gregory Szorc <gregory.szorc@gmail.com> parents: diff changeset	104
37298 5ef2da00e935 wireproto: implement custom __repr__ for frame Gregory Szorc <gregory.szorc@gmail.com> parents: 37292 diff changeset	105 def humanflags(mapping, value):
5ef2da00e935 wireproto: implement custom __repr__ for frame Gregory Szorc <gregory.szorc@gmail.com> parents: 37292 diff changeset	106 """Convert a numeric flags value to a human value, using a mapping table."""
37473 7c2c7c749411 wireproto: show unknown id and flags in repr(frame) Yuya Nishihara <yuya@tcha.org> parents: 37472 diff changeset	107 namemap = {v: k for k, v in mapping.iteritems()}
37298 5ef2da00e935 wireproto: implement custom __repr__ for frame Gregory Szorc <gregory.szorc@gmail.com> parents: 37292 diff changeset	108 flags = []
37473 7c2c7c749411 wireproto: show unknown id and flags in repr(frame) Yuya Nishihara <yuya@tcha.org> parents: 37472 diff changeset	109 val = 1
7c2c7c749411 wireproto: show unknown id and flags in repr(frame) Yuya Nishihara <yuya@tcha.org> parents: 37472 diff changeset	110 while value >= val:
37298 5ef2da00e935 wireproto: implement custom __repr__ for frame Gregory Szorc <gregory.szorc@gmail.com> parents: 37292 diff changeset	111 if value & val:
37473 7c2c7c749411 wireproto: show unknown id and flags in repr(frame) Yuya Nishihara <yuya@tcha.org> parents: 37472 diff changeset	112 flags.append(namemap.get(val, '<unknown 0x%02x>' % val))
7c2c7c749411 wireproto: show unknown id and flags in repr(frame) Yuya Nishihara <yuya@tcha.org> parents: 37472 diff changeset	113 val <<= 1
37298 5ef2da00e935 wireproto: implement custom __repr__ for frame Gregory Szorc <gregory.szorc@gmail.com> parents: 37292 diff changeset	114
5ef2da00e935 wireproto: implement custom __repr__ for frame Gregory Szorc <gregory.szorc@gmail.com> parents: 37292 diff changeset	115 return b'\|'.join(flags)
5ef2da00e935 wireproto: implement custom __repr__ for frame Gregory Szorc <gregory.szorc@gmail.com> parents: 37292 diff changeset	116
37064 884a0c1604ad wireproto: define attr-based classes for representing frames Gregory Szorc <gregory.szorc@gmail.com> parents: 37063 diff changeset	117 @attr.s(slots=True)
884a0c1604ad wireproto: define attr-based classes for representing frames Gregory Szorc <gregory.szorc@gmail.com> parents: 37063 diff changeset	118 class frameheader(object):
884a0c1604ad wireproto: define attr-based classes for representing frames Gregory Szorc <gregory.szorc@gmail.com> parents: 37063 diff changeset	119 """Represents the data in a frame header."""
884a0c1604ad wireproto: define attr-based classes for representing frames Gregory Szorc <gregory.szorc@gmail.com> parents: 37063 diff changeset	120
884a0c1604ad wireproto: define attr-based classes for representing frames Gregory Szorc <gregory.szorc@gmail.com> parents: 37063 diff changeset	121 length = attr.ib()
884a0c1604ad wireproto: define attr-based classes for representing frames Gregory Szorc <gregory.szorc@gmail.com> parents: 37063 diff changeset	122 requestid = attr.ib()
37288 9bfcbe4f4745 wireproto: add streams to frame-based protocol Gregory Szorc <gregory.szorc@gmail.com> parents: 37287 diff changeset	123 streamid = attr.ib()
9bfcbe4f4745 wireproto: add streams to frame-based protocol Gregory Szorc <gregory.szorc@gmail.com> parents: 37287 diff changeset	124 streamflags = attr.ib()
37064 884a0c1604ad wireproto: define attr-based classes for representing frames Gregory Szorc <gregory.szorc@gmail.com> parents: 37063 diff changeset	125 typeid = attr.ib()
884a0c1604ad wireproto: define attr-based classes for representing frames Gregory Szorc <gregory.szorc@gmail.com> parents: 37063 diff changeset	126 flags = attr.ib()
884a0c1604ad wireproto: define attr-based classes for representing frames Gregory Szorc <gregory.szorc@gmail.com> parents: 37063 diff changeset	127
37298 5ef2da00e935 wireproto: implement custom __repr__ for frame Gregory Szorc <gregory.szorc@gmail.com> parents: 37292 diff changeset	128 @attr.s(slots=True, repr=False)
37064 884a0c1604ad wireproto: define attr-based classes for representing frames Gregory Szorc <gregory.szorc@gmail.com> parents: 37063 diff changeset	129 class frame(object):
884a0c1604ad wireproto: define attr-based classes for representing frames Gregory Szorc <gregory.szorc@gmail.com> parents: 37063 diff changeset	130 """Represents a parsed frame."""
884a0c1604ad wireproto: define attr-based classes for representing frames Gregory Szorc <gregory.szorc@gmail.com> parents: 37063 diff changeset	131
884a0c1604ad wireproto: define attr-based classes for representing frames Gregory Szorc <gregory.szorc@gmail.com> parents: 37063 diff changeset	132 requestid = attr.ib()
37288 9bfcbe4f4745 wireproto: add streams to frame-based protocol Gregory Szorc <gregory.szorc@gmail.com> parents: 37287 diff changeset	133 streamid = attr.ib()
9bfcbe4f4745 wireproto: add streams to frame-based protocol Gregory Szorc <gregory.szorc@gmail.com> parents: 37287 diff changeset	134 streamflags = attr.ib()
37064 884a0c1604ad wireproto: define attr-based classes for representing frames Gregory Szorc <gregory.szorc@gmail.com> parents: 37063 diff changeset	135 typeid = attr.ib()
884a0c1604ad wireproto: define attr-based classes for representing frames Gregory Szorc <gregory.szorc@gmail.com> parents: 37063 diff changeset	136 flags = attr.ib()
884a0c1604ad wireproto: define attr-based classes for representing frames Gregory Szorc <gregory.szorc@gmail.com> parents: 37063 diff changeset	137 payload = attr.ib()
884a0c1604ad wireproto: define attr-based classes for representing frames Gregory Szorc <gregory.szorc@gmail.com> parents: 37063 diff changeset	138
37474 d33997123ea5 py3: system-stringify repr(frame) Yuya Nishihara <yuya@tcha.org> parents: 37473 diff changeset	139 @encoding.strmethod
37298 5ef2da00e935 wireproto: implement custom __repr__ for frame Gregory Szorc <gregory.szorc@gmail.com> parents: 37292 diff changeset	140 def __repr__(self):
37473 7c2c7c749411 wireproto: show unknown id and flags in repr(frame) Yuya Nishihara <yuya@tcha.org> parents: 37472 diff changeset	141 typename = '<unknown 0x%02x>' % self.typeid
37298 5ef2da00e935 wireproto: implement custom __repr__ for frame Gregory Szorc <gregory.szorc@gmail.com> parents: 37292 diff changeset	142 for name, value in FRAME_TYPES.iteritems():
5ef2da00e935 wireproto: implement custom __repr__ for frame Gregory Szorc <gregory.szorc@gmail.com> parents: 37292 diff changeset	143 if value == self.typeid:
5ef2da00e935 wireproto: implement custom __repr__ for frame Gregory Szorc <gregory.szorc@gmail.com> parents: 37292 diff changeset	144 typename = name
5ef2da00e935 wireproto: implement custom __repr__ for frame Gregory Szorc <gregory.szorc@gmail.com> parents: 37292 diff changeset	145 break
5ef2da00e935 wireproto: implement custom __repr__ for frame Gregory Szorc <gregory.szorc@gmail.com> parents: 37292 diff changeset	146
5ef2da00e935 wireproto: implement custom __repr__ for frame Gregory Szorc <gregory.szorc@gmail.com> parents: 37292 diff changeset	147 return ('frame(size=%d; request=%d; stream=%d; streamflags=%s; '
5ef2da00e935 wireproto: implement custom __repr__ for frame Gregory Szorc <gregory.szorc@gmail.com> parents: 37292 diff changeset	148 'type=%s; flags=%s)' % (
5ef2da00e935 wireproto: implement custom __repr__ for frame Gregory Szorc <gregory.szorc@gmail.com> parents: 37292 diff changeset	149 len(self.payload), self.requestid, self.streamid,
5ef2da00e935 wireproto: implement custom __repr__ for frame Gregory Szorc <gregory.szorc@gmail.com> parents: 37292 diff changeset	150 humanflags(STREAM_FLAGS, self.streamflags), typename,
37472 2f81926c7f89 wireproto: fix repr(frame) to not crash by unknown type id Yuya Nishihara <yuya@tcha.org> parents: 37319 diff changeset	151 humanflags(FRAME_TYPE_FLAGS.get(self.typeid, {}), self.flags)))
37298 5ef2da00e935 wireproto: implement custom __repr__ for frame Gregory Szorc <gregory.szorc@gmail.com> parents: 37292 diff changeset	152
37288 9bfcbe4f4745 wireproto: add streams to frame-based protocol Gregory Szorc <gregory.szorc@gmail.com> parents: 37287 diff changeset	153 def makeframe(requestid, streamid, streamflags, typeid, flags, payload):
37054 40206e227412 wireproto: define and implement protocol for issuing requests Gregory Szorc <gregory.szorc@gmail.com> parents: diff changeset	154 """Assemble a frame into a byte array."""
40206e227412 wireproto: define and implement protocol for issuing requests Gregory Szorc <gregory.szorc@gmail.com> parents: diff changeset	155 # TODO assert size of payload.
40206e227412 wireproto: define and implement protocol for issuing requests Gregory Szorc <gregory.szorc@gmail.com> parents: diff changeset	156 frame = bytearray(FRAME_HEADER_SIZE + len(payload))
40206e227412 wireproto: define and implement protocol for issuing requests Gregory Szorc <gregory.szorc@gmail.com> parents: diff changeset	157
37060 2ec1fb9de638 wireproto: add request IDs to frames Gregory Szorc <gregory.szorc@gmail.com> parents: 37059 diff changeset	158 # 24 bits length
2ec1fb9de638 wireproto: add request IDs to frames Gregory Szorc <gregory.szorc@gmail.com> parents: 37059 diff changeset	159 # 16 bits request id
37288 9bfcbe4f4745 wireproto: add streams to frame-based protocol Gregory Szorc <gregory.szorc@gmail.com> parents: 37287 diff changeset	160 # 8 bits stream id
9bfcbe4f4745 wireproto: add streams to frame-based protocol Gregory Szorc <gregory.szorc@gmail.com> parents: 37287 diff changeset	161 # 8 bits stream flags
37060 2ec1fb9de638 wireproto: add request IDs to frames Gregory Szorc <gregory.szorc@gmail.com> parents: 37059 diff changeset	162 # 4 bits type
2ec1fb9de638 wireproto: add request IDs to frames Gregory Szorc <gregory.szorc@gmail.com> parents: 37059 diff changeset	163 # 4 bits flags
2ec1fb9de638 wireproto: add request IDs to frames Gregory Szorc <gregory.szorc@gmail.com> parents: 37059 diff changeset	164
37054 40206e227412 wireproto: define and implement protocol for issuing requests Gregory Szorc <gregory.szorc@gmail.com> parents: diff changeset	165 l = struct.pack(r'<I', len(payload))
40206e227412 wireproto: define and implement protocol for issuing requests Gregory Szorc <gregory.szorc@gmail.com> parents: diff changeset	166 frame[0:3] = l[0:3]
37288 9bfcbe4f4745 wireproto: add streams to frame-based protocol Gregory Szorc <gregory.szorc@gmail.com> parents: 37287 diff changeset	167 struct.pack_into(r'<HBB', frame, 3, requestid, streamid, streamflags)
9bfcbe4f4745 wireproto: add streams to frame-based protocol Gregory Szorc <gregory.szorc@gmail.com> parents: 37287 diff changeset	168 frame[7] = (typeid << 4) \| flags
9bfcbe4f4745 wireproto: add streams to frame-based protocol Gregory Szorc <gregory.szorc@gmail.com> parents: 37287 diff changeset	169 frame[8:] = payload
37054 40206e227412 wireproto: define and implement protocol for issuing requests Gregory Szorc <gregory.szorc@gmail.com> parents: diff changeset	170
40206e227412 wireproto: define and implement protocol for issuing requests Gregory Szorc <gregory.szorc@gmail.com> parents: diff changeset	171 return frame
40206e227412 wireproto: define and implement protocol for issuing requests Gregory Szorc <gregory.szorc@gmail.com> parents: diff changeset	172
40206e227412 wireproto: define and implement protocol for issuing requests Gregory Szorc <gregory.szorc@gmail.com> parents: diff changeset	173 def makeframefromhumanstring(s):
37060 2ec1fb9de638 wireproto: add request IDs to frames Gregory Szorc <gregory.szorc@gmail.com> parents: 37059 diff changeset	174 """Create a frame from a human readable string
2ec1fb9de638 wireproto: add request IDs to frames Gregory Szorc <gregory.szorc@gmail.com> parents: 37059 diff changeset	175
2ec1fb9de638 wireproto: add request IDs to frames Gregory Szorc <gregory.szorc@gmail.com> parents: 37059 diff changeset	176 Strings have the form:
2ec1fb9de638 wireproto: add request IDs to frames Gregory Szorc <gregory.szorc@gmail.com> parents: 37059 diff changeset	177
37288 9bfcbe4f4745 wireproto: add streams to frame-based protocol Gregory Szorc <gregory.szorc@gmail.com> parents: 37287 diff changeset	178 <request-id> <stream-id> <stream-flags> <type> <flags> <payload>
37054 40206e227412 wireproto: define and implement protocol for issuing requests Gregory Szorc <gregory.szorc@gmail.com> parents: diff changeset	179
40206e227412 wireproto: define and implement protocol for issuing requests Gregory Szorc <gregory.szorc@gmail.com> parents: diff changeset	180 This can be used by user-facing applications and tests for creating
40206e227412 wireproto: define and implement protocol for issuing requests Gregory Szorc <gregory.szorc@gmail.com> parents: diff changeset	181 frames easily without having to type out a bunch of constants.
40206e227412 wireproto: define and implement protocol for issuing requests Gregory Szorc <gregory.szorc@gmail.com> parents: diff changeset	182
37288 9bfcbe4f4745 wireproto: add streams to frame-based protocol Gregory Szorc <gregory.szorc@gmail.com> parents: 37287 diff changeset	183 Request ID and stream IDs are integers.
37060 2ec1fb9de638 wireproto: add request IDs to frames Gregory Szorc <gregory.szorc@gmail.com> parents: 37059 diff changeset	184
37288 9bfcbe4f4745 wireproto: add streams to frame-based protocol Gregory Szorc <gregory.szorc@gmail.com> parents: 37287 diff changeset	185 Stream flags, frame type, and flags can be specified by integer or
9bfcbe4f4745 wireproto: add streams to frame-based protocol Gregory Szorc <gregory.szorc@gmail.com> parents: 37287 diff changeset	186 named constant.
37060 2ec1fb9de638 wireproto: add request IDs to frames Gregory Szorc <gregory.szorc@gmail.com> parents: 37059 diff changeset	187
37054 40206e227412 wireproto: define and implement protocol for issuing requests Gregory Szorc <gregory.szorc@gmail.com> parents: diff changeset	188 Flags can be delimited by `\|` to bitwise OR them together.
37290 cc5a040fe150 wireproto: syntax for encoding CBOR into frames Gregory Szorc <gregory.szorc@gmail.com> parents: 37289 diff changeset	189
cc5a040fe150 wireproto: syntax for encoding CBOR into frames Gregory Szorc <gregory.szorc@gmail.com> parents: 37289 diff changeset	190 If the payload begins with ``cbor:``, the following string will be
37476 e9dea82ea1f3 wireproto: convert python literal to object without using unsafe eval() Yuya Nishihara <yuya@tcha.org> parents: 37474 diff changeset	191 evaluated as Python literal and the resulting object will be fed into
37290 cc5a040fe150 wireproto: syntax for encoding CBOR into frames Gregory Szorc <gregory.szorc@gmail.com> parents: 37289 diff changeset	192 a CBOR encoder. Otherwise, the payload is interpreted as a Python
cc5a040fe150 wireproto: syntax for encoding CBOR into frames Gregory Szorc <gregory.szorc@gmail.com> parents: 37289 diff changeset	193 byte string literal.
37054 40206e227412 wireproto: define and implement protocol for issuing requests Gregory Szorc <gregory.szorc@gmail.com> parents: diff changeset	194 """
37288 9bfcbe4f4745 wireproto: add streams to frame-based protocol Gregory Szorc <gregory.szorc@gmail.com> parents: 37287 diff changeset	195 fields = s.split(b' ', 5)
9bfcbe4f4745 wireproto: add streams to frame-based protocol Gregory Szorc <gregory.szorc@gmail.com> parents: 37287 diff changeset	196 requestid, streamid, streamflags, frametype, frameflags, payload = fields
37060 2ec1fb9de638 wireproto: add request IDs to frames Gregory Szorc <gregory.szorc@gmail.com> parents: 37059 diff changeset	197
2ec1fb9de638 wireproto: add request IDs to frames Gregory Szorc <gregory.szorc@gmail.com> parents: 37059 diff changeset	198 requestid = int(requestid)
37288 9bfcbe4f4745 wireproto: add streams to frame-based protocol Gregory Szorc <gregory.szorc@gmail.com> parents: 37287 diff changeset	199 streamid = int(streamid)
9bfcbe4f4745 wireproto: add streams to frame-based protocol Gregory Szorc <gregory.szorc@gmail.com> parents: 37287 diff changeset	200
9bfcbe4f4745 wireproto: add streams to frame-based protocol Gregory Szorc <gregory.szorc@gmail.com> parents: 37287 diff changeset	201 finalstreamflags = 0
9bfcbe4f4745 wireproto: add streams to frame-based protocol Gregory Szorc <gregory.szorc@gmail.com> parents: 37287 diff changeset	202 for flag in streamflags.split(b'\|'):
9bfcbe4f4745 wireproto: add streams to frame-based protocol Gregory Szorc <gregory.szorc@gmail.com> parents: 37287 diff changeset	203 if flag in STREAM_FLAGS:
9bfcbe4f4745 wireproto: add streams to frame-based protocol Gregory Szorc <gregory.szorc@gmail.com> parents: 37287 diff changeset	204 finalstreamflags \|= STREAM_FLAGS[flag]
9bfcbe4f4745 wireproto: add streams to frame-based protocol Gregory Szorc <gregory.szorc@gmail.com> parents: 37287 diff changeset	205 else:
9bfcbe4f4745 wireproto: add streams to frame-based protocol Gregory Szorc <gregory.szorc@gmail.com> parents: 37287 diff changeset	206 finalstreamflags \|= int(flag)
37054 40206e227412 wireproto: define and implement protocol for issuing requests Gregory Szorc <gregory.szorc@gmail.com> parents: diff changeset	207
40206e227412 wireproto: define and implement protocol for issuing requests Gregory Szorc <gregory.szorc@gmail.com> parents: diff changeset	208 if frametype in FRAME_TYPES:
40206e227412 wireproto: define and implement protocol for issuing requests Gregory Szorc <gregory.szorc@gmail.com> parents: diff changeset	209 frametype = FRAME_TYPES[frametype]
40206e227412 wireproto: define and implement protocol for issuing requests Gregory Szorc <gregory.szorc@gmail.com> parents: diff changeset	210 else:
40206e227412 wireproto: define and implement protocol for issuing requests Gregory Szorc <gregory.szorc@gmail.com> parents: diff changeset	211 frametype = int(frametype)
40206e227412 wireproto: define and implement protocol for issuing requests Gregory Szorc <gregory.szorc@gmail.com> parents: diff changeset	212
40206e227412 wireproto: define and implement protocol for issuing requests Gregory Szorc <gregory.szorc@gmail.com> parents: diff changeset	213 finalflags = 0
40206e227412 wireproto: define and implement protocol for issuing requests Gregory Szorc <gregory.szorc@gmail.com> parents: diff changeset	214 validflags = FRAME_TYPE_FLAGS[frametype]
40206e227412 wireproto: define and implement protocol for issuing requests Gregory Szorc <gregory.szorc@gmail.com> parents: diff changeset	215 for flag in frameflags.split(b'\|'):
40206e227412 wireproto: define and implement protocol for issuing requests Gregory Szorc <gregory.szorc@gmail.com> parents: diff changeset	216 if flag in validflags:
40206e227412 wireproto: define and implement protocol for issuing requests Gregory Szorc <gregory.szorc@gmail.com> parents: diff changeset	217 finalflags \|= validflags[flag]
40206e227412 wireproto: define and implement protocol for issuing requests Gregory Szorc <gregory.szorc@gmail.com> parents: diff changeset	218 else:
40206e227412 wireproto: define and implement protocol for issuing requests Gregory Szorc <gregory.szorc@gmail.com> parents: diff changeset	219 finalflags \|= int(flag)
40206e227412 wireproto: define and implement protocol for issuing requests Gregory Szorc <gregory.szorc@gmail.com> parents: diff changeset	220
37290 cc5a040fe150 wireproto: syntax for encoding CBOR into frames Gregory Szorc <gregory.szorc@gmail.com> parents: 37289 diff changeset	221 if payload.startswith(b'cbor:'):
39465 36f487a332ad wireprotoframing: use our CBOR module Gregory Szorc <gregory.szorc@gmail.com> parents: 37728 diff changeset	222 payload = b''.join(cborutil.streamencode(
36f487a332ad wireprotoframing: use our CBOR module Gregory Szorc <gregory.szorc@gmail.com> parents: 37728 diff changeset	223 stringutil.evalpythonliteral(payload[5:])))
37290 cc5a040fe150 wireproto: syntax for encoding CBOR into frames Gregory Szorc <gregory.szorc@gmail.com> parents: 37289 diff changeset	224
cc5a040fe150 wireproto: syntax for encoding CBOR into frames Gregory Szorc <gregory.szorc@gmail.com> parents: 37289 diff changeset	225 else:
cc5a040fe150 wireproto: syntax for encoding CBOR into frames Gregory Szorc <gregory.szorc@gmail.com> parents: 37289 diff changeset	226 payload = stringutil.unescapestr(payload)
37054 40206e227412 wireproto: define and implement protocol for issuing requests Gregory Szorc <gregory.szorc@gmail.com> parents: diff changeset	227
37288 9bfcbe4f4745 wireproto: add streams to frame-based protocol Gregory Szorc <gregory.szorc@gmail.com> parents: 37287 diff changeset	228 return makeframe(requestid=requestid, streamid=streamid,
9bfcbe4f4745 wireproto: add streams to frame-based protocol Gregory Szorc <gregory.szorc@gmail.com> parents: 37287 diff changeset	229 streamflags=finalstreamflags, typeid=frametype,
37065 fe4c944f95bb wireproto: use named arguments when passing around frame data Gregory Szorc <gregory.szorc@gmail.com> parents: 37064 diff changeset	230 flags=finalflags, payload=payload)
37054 40206e227412 wireproto: define and implement protocol for issuing requests Gregory Szorc <gregory.szorc@gmail.com> parents: diff changeset	231
37055 8c3c47362934 wireproto: implement basic frame reading and processing Gregory Szorc <gregory.szorc@gmail.com> parents: 37054 diff changeset	232 def parseheader(data):
8c3c47362934 wireproto: implement basic frame reading and processing Gregory Szorc <gregory.szorc@gmail.com> parents: 37054 diff changeset	233 """Parse a unified framing protocol frame header from a buffer.
8c3c47362934 wireproto: implement basic frame reading and processing Gregory Szorc <gregory.szorc@gmail.com> parents: 37054 diff changeset	234
8c3c47362934 wireproto: implement basic frame reading and processing Gregory Szorc <gregory.szorc@gmail.com> parents: 37054 diff changeset	235 The header is expected to be in the buffer at offset 0 and the
8c3c47362934 wireproto: implement basic frame reading and processing Gregory Szorc <gregory.szorc@gmail.com> parents: 37054 diff changeset	236 buffer is expected to be large enough to hold a full header.
8c3c47362934 wireproto: implement basic frame reading and processing Gregory Szorc <gregory.szorc@gmail.com> parents: 37054 diff changeset	237 """
8c3c47362934 wireproto: implement basic frame reading and processing Gregory Szorc <gregory.szorc@gmail.com> parents: 37054 diff changeset	238 # 24 bits payload length (little endian)
37288 9bfcbe4f4745 wireproto: add streams to frame-based protocol Gregory Szorc <gregory.szorc@gmail.com> parents: 37287 diff changeset	239 # 16 bits request ID
9bfcbe4f4745 wireproto: add streams to frame-based protocol Gregory Szorc <gregory.szorc@gmail.com> parents: 37287 diff changeset	240 # 8 bits stream ID
9bfcbe4f4745 wireproto: add streams to frame-based protocol Gregory Szorc <gregory.szorc@gmail.com> parents: 37287 diff changeset	241 # 8 bits stream flags
37055 8c3c47362934 wireproto: implement basic frame reading and processing Gregory Szorc <gregory.szorc@gmail.com> parents: 37054 diff changeset	242 # 4 bits frame type
8c3c47362934 wireproto: implement basic frame reading and processing Gregory Szorc <gregory.szorc@gmail.com> parents: 37054 diff changeset	243 # 4 bits frame flags
8c3c47362934 wireproto: implement basic frame reading and processing Gregory Szorc <gregory.szorc@gmail.com> parents: 37054 diff changeset	244 # ... payload
8c3c47362934 wireproto: implement basic frame reading and processing Gregory Szorc <gregory.szorc@gmail.com> parents: 37054 diff changeset	245 framelength = data[0] + 256 * data[1] + 16384 * data[2]
37288 9bfcbe4f4745 wireproto: add streams to frame-based protocol Gregory Szorc <gregory.szorc@gmail.com> parents: 37287 diff changeset	246 requestid, streamid, streamflags = struct.unpack_from(r'<HBB', data, 3)
9bfcbe4f4745 wireproto: add streams to frame-based protocol Gregory Szorc <gregory.szorc@gmail.com> parents: 37287 diff changeset	247 typeflags = data[7]
37055 8c3c47362934 wireproto: implement basic frame reading and processing Gregory Szorc <gregory.szorc@gmail.com> parents: 37054 diff changeset	248
8c3c47362934 wireproto: implement basic frame reading and processing Gregory Szorc <gregory.szorc@gmail.com> parents: 37054 diff changeset	249 frametype = (typeflags & 0xf0) >> 4
8c3c47362934 wireproto: implement basic frame reading and processing Gregory Szorc <gregory.szorc@gmail.com> parents: 37054 diff changeset	250 frameflags = typeflags & 0x0f
8c3c47362934 wireproto: implement basic frame reading and processing Gregory Szorc <gregory.szorc@gmail.com> parents: 37054 diff changeset	251
37288 9bfcbe4f4745 wireproto: add streams to frame-based protocol Gregory Szorc <gregory.szorc@gmail.com> parents: 37287 diff changeset	252 return frameheader(framelength, requestid, streamid, streamflags,
9bfcbe4f4745 wireproto: add streams to frame-based protocol Gregory Szorc <gregory.szorc@gmail.com> parents: 37287 diff changeset	253 frametype, frameflags)
37055 8c3c47362934 wireproto: implement basic frame reading and processing Gregory Szorc <gregory.szorc@gmail.com> parents: 37054 diff changeset	254
8c3c47362934 wireproto: implement basic frame reading and processing Gregory Szorc <gregory.szorc@gmail.com> parents: 37054 diff changeset	255 def readframe(fh):
8c3c47362934 wireproto: implement basic frame reading and processing Gregory Szorc <gregory.szorc@gmail.com> parents: 37054 diff changeset	256 """Read a unified framing protocol frame from a file object.
8c3c47362934 wireproto: implement basic frame reading and processing Gregory Szorc <gregory.szorc@gmail.com> parents: 37054 diff changeset	257
8c3c47362934 wireproto: implement basic frame reading and processing Gregory Szorc <gregory.szorc@gmail.com> parents: 37054 diff changeset	258 Returns a 3-tuple of (type, flags, payload) for the decoded frame or
8c3c47362934 wireproto: implement basic frame reading and processing Gregory Szorc <gregory.szorc@gmail.com> parents: 37054 diff changeset	259 None if no frame is available. May raise if a malformed frame is
8c3c47362934 wireproto: implement basic frame reading and processing Gregory Szorc <gregory.szorc@gmail.com> parents: 37054 diff changeset	260 seen.
8c3c47362934 wireproto: implement basic frame reading and processing Gregory Szorc <gregory.szorc@gmail.com> parents: 37054 diff changeset	261 """
8c3c47362934 wireproto: implement basic frame reading and processing Gregory Szorc <gregory.szorc@gmail.com> parents: 37054 diff changeset	262 header = bytearray(FRAME_HEADER_SIZE)
8c3c47362934 wireproto: implement basic frame reading and processing Gregory Szorc <gregory.szorc@gmail.com> parents: 37054 diff changeset	263
8c3c47362934 wireproto: implement basic frame reading and processing Gregory Szorc <gregory.szorc@gmail.com> parents: 37054 diff changeset	264 readcount = fh.readinto(header)
8c3c47362934 wireproto: implement basic frame reading and processing Gregory Szorc <gregory.szorc@gmail.com> parents: 37054 diff changeset	265
8c3c47362934 wireproto: implement basic frame reading and processing Gregory Szorc <gregory.szorc@gmail.com> parents: 37054 diff changeset	266 if readcount == 0:
8c3c47362934 wireproto: implement basic frame reading and processing Gregory Szorc <gregory.szorc@gmail.com> parents: 37054 diff changeset	267 return None
8c3c47362934 wireproto: implement basic frame reading and processing Gregory Szorc <gregory.szorc@gmail.com> parents: 37054 diff changeset	268
8c3c47362934 wireproto: implement basic frame reading and processing Gregory Szorc <gregory.szorc@gmail.com> parents: 37054 diff changeset	269 if readcount != FRAME_HEADER_SIZE:
8c3c47362934 wireproto: implement basic frame reading and processing Gregory Szorc <gregory.szorc@gmail.com> parents: 37054 diff changeset	270 raise error.Abort(_('received incomplete frame: got %d bytes: %s') %
8c3c47362934 wireproto: implement basic frame reading and processing Gregory Szorc <gregory.szorc@gmail.com> parents: 37054 diff changeset	271 (readcount, header))
8c3c47362934 wireproto: implement basic frame reading and processing Gregory Szorc <gregory.szorc@gmail.com> parents: 37054 diff changeset	272
37064 884a0c1604ad wireproto: define attr-based classes for representing frames Gregory Szorc <gregory.szorc@gmail.com> parents: 37063 diff changeset	273 h = parseheader(header)
37055 8c3c47362934 wireproto: implement basic frame reading and processing Gregory Szorc <gregory.szorc@gmail.com> parents: 37054 diff changeset	274
37064 884a0c1604ad wireproto: define attr-based classes for representing frames Gregory Szorc <gregory.szorc@gmail.com> parents: 37063 diff changeset	275 payload = fh.read(h.length)
884a0c1604ad wireproto: define attr-based classes for representing frames Gregory Szorc <gregory.szorc@gmail.com> parents: 37063 diff changeset	276 if len(payload) != h.length:
37055 8c3c47362934 wireproto: implement basic frame reading and processing Gregory Szorc <gregory.szorc@gmail.com> parents: 37054 diff changeset	277 raise error.Abort(_('frame length error: expected %d; got %d') %
37064 884a0c1604ad wireproto: define attr-based classes for representing frames Gregory Szorc <gregory.szorc@gmail.com> parents: 37063 diff changeset	278 (h.length, len(payload)))
37055 8c3c47362934 wireproto: implement basic frame reading and processing Gregory Szorc <gregory.szorc@gmail.com> parents: 37054 diff changeset	279
37288 9bfcbe4f4745 wireproto: add streams to frame-based protocol Gregory Szorc <gregory.szorc@gmail.com> parents: 37287 diff changeset	280 return frame(h.requestid, h.streamid, h.streamflags, h.typeid, h.flags,
9bfcbe4f4745 wireproto: add streams to frame-based protocol Gregory Szorc <gregory.szorc@gmail.com> parents: 37287 diff changeset	281 payload)
37055 8c3c47362934 wireproto: implement basic frame reading and processing Gregory Szorc <gregory.szorc@gmail.com> parents: 37054 diff changeset	282
37292 3d0e2cd86e05 wireproto: use CBOR for command requests Gregory Szorc <gregory.szorc@gmail.com> parents: 37291 diff changeset	283 def createcommandframes(stream, requestid, cmd, args, datafh=None,
40025 86b22a4cfab1 wireprotov2: client support for advertising redirect targets Gregory Szorc <gregory.szorc@gmail.com> parents: 40021 diff changeset	284 maxframesize=DEFAULT_MAX_FRAME_SIZE,
86b22a4cfab1 wireprotov2: client support for advertising redirect targets Gregory Szorc <gregory.szorc@gmail.com> parents: 40021 diff changeset	285 redirect=None):
37054 40206e227412 wireproto: define and implement protocol for issuing requests Gregory Szorc <gregory.szorc@gmail.com> parents: diff changeset	286 """Create frames necessary to transmit a request to run a command.
40206e227412 wireproto: define and implement protocol for issuing requests Gregory Szorc <gregory.szorc@gmail.com> parents: diff changeset	287
40206e227412 wireproto: define and implement protocol for issuing requests Gregory Szorc <gregory.szorc@gmail.com> parents: diff changeset	288 This is a generator of bytearrays. Each item represents a frame
40206e227412 wireproto: define and implement protocol for issuing requests Gregory Szorc <gregory.szorc@gmail.com> parents: diff changeset	289 ready to be sent over the wire to a peer.
40206e227412 wireproto: define and implement protocol for issuing requests Gregory Szorc <gregory.szorc@gmail.com> parents: diff changeset	290 """
37292 3d0e2cd86e05 wireproto: use CBOR for command requests Gregory Szorc <gregory.szorc@gmail.com> parents: 37291 diff changeset	291 data = {b'name': cmd}
37054 40206e227412 wireproto: define and implement protocol for issuing requests Gregory Szorc <gregory.szorc@gmail.com> parents: diff changeset	292 if args:
37292 3d0e2cd86e05 wireproto: use CBOR for command requests Gregory Szorc <gregory.szorc@gmail.com> parents: 37291 diff changeset	293 data[b'args'] = args
37054 40206e227412 wireproto: define and implement protocol for issuing requests Gregory Szorc <gregory.szorc@gmail.com> parents: diff changeset	294
40025 86b22a4cfab1 wireprotov2: client support for advertising redirect targets Gregory Szorc <gregory.szorc@gmail.com> parents: 40021 diff changeset	295 if redirect:
86b22a4cfab1 wireprotov2: client support for advertising redirect targets Gregory Szorc <gregory.szorc@gmail.com> parents: 40021 diff changeset	296 data[b'redirect'] = redirect
86b22a4cfab1 wireprotov2: client support for advertising redirect targets Gregory Szorc <gregory.szorc@gmail.com> parents: 40021 diff changeset	297
39465 36f487a332ad wireprotoframing: use our CBOR module Gregory Szorc <gregory.szorc@gmail.com> parents: 37728 diff changeset	298 data = b''.join(cborutil.streamencode(data))
37054 40206e227412 wireproto: define and implement protocol for issuing requests Gregory Szorc <gregory.szorc@gmail.com> parents: diff changeset	299
37292 3d0e2cd86e05 wireproto: use CBOR for command requests Gregory Szorc <gregory.szorc@gmail.com> parents: 37291 diff changeset	300 offset = 0
3d0e2cd86e05 wireproto: use CBOR for command requests Gregory Szorc <gregory.szorc@gmail.com> parents: 37291 diff changeset	301
3d0e2cd86e05 wireproto: use CBOR for command requests Gregory Szorc <gregory.szorc@gmail.com> parents: 37291 diff changeset	302 while True:
3d0e2cd86e05 wireproto: use CBOR for command requests Gregory Szorc <gregory.szorc@gmail.com> parents: 37291 diff changeset	303 flags = 0
37054 40206e227412 wireproto: define and implement protocol for issuing requests Gregory Szorc <gregory.szorc@gmail.com> parents: diff changeset	304
37292 3d0e2cd86e05 wireproto: use CBOR for command requests Gregory Szorc <gregory.szorc@gmail.com> parents: 37291 diff changeset	305 # Must set new or continuation flag.
3d0e2cd86e05 wireproto: use CBOR for command requests Gregory Szorc <gregory.szorc@gmail.com> parents: 37291 diff changeset	306 if not offset:
3d0e2cd86e05 wireproto: use CBOR for command requests Gregory Szorc <gregory.szorc@gmail.com> parents: 37291 diff changeset	307 flags \|= FLAG_COMMAND_REQUEST_NEW
3d0e2cd86e05 wireproto: use CBOR for command requests Gregory Szorc <gregory.szorc@gmail.com> parents: 37291 diff changeset	308 else:
3d0e2cd86e05 wireproto: use CBOR for command requests Gregory Szorc <gregory.szorc@gmail.com> parents: 37291 diff changeset	309 flags \|= FLAG_COMMAND_REQUEST_CONTINUATION
37054 40206e227412 wireproto: define and implement protocol for issuing requests Gregory Szorc <gregory.szorc@gmail.com> parents: diff changeset	310
37292 3d0e2cd86e05 wireproto: use CBOR for command requests Gregory Szorc <gregory.szorc@gmail.com> parents: 37291 diff changeset	311 # Data frames is set on all frames.
3d0e2cd86e05 wireproto: use CBOR for command requests Gregory Szorc <gregory.szorc@gmail.com> parents: 37291 diff changeset	312 if datafh:
3d0e2cd86e05 wireproto: use CBOR for command requests Gregory Szorc <gregory.szorc@gmail.com> parents: 37291 diff changeset	313 flags \|= FLAG_COMMAND_REQUEST_EXPECT_DATA
37054 40206e227412 wireproto: define and implement protocol for issuing requests Gregory Szorc <gregory.szorc@gmail.com> parents: diff changeset	314
37292 3d0e2cd86e05 wireproto: use CBOR for command requests Gregory Szorc <gregory.szorc@gmail.com> parents: 37291 diff changeset	315 payload = data[offset:offset + maxframesize]
3d0e2cd86e05 wireproto: use CBOR for command requests Gregory Szorc <gregory.szorc@gmail.com> parents: 37291 diff changeset	316 offset += len(payload)
3d0e2cd86e05 wireproto: use CBOR for command requests Gregory Szorc <gregory.szorc@gmail.com> parents: 37291 diff changeset	317
3d0e2cd86e05 wireproto: use CBOR for command requests Gregory Szorc <gregory.szorc@gmail.com> parents: 37291 diff changeset	318 if len(payload) == maxframesize and offset < len(data):
3d0e2cd86e05 wireproto: use CBOR for command requests Gregory Szorc <gregory.szorc@gmail.com> parents: 37291 diff changeset	319 flags \|= FLAG_COMMAND_REQUEST_MORE_FRAMES
3d0e2cd86e05 wireproto: use CBOR for command requests Gregory Szorc <gregory.szorc@gmail.com> parents: 37291 diff changeset	320
37287 3ed344546d9e wireproto: start to associate frame generation with a stream Gregory Szorc <gregory.szorc@gmail.com> parents: 37087 diff changeset	321 yield stream.makeframe(requestid=requestid,
37292 3d0e2cd86e05 wireproto: use CBOR for command requests Gregory Szorc <gregory.szorc@gmail.com> parents: 37291 diff changeset	322 typeid=FRAME_TYPE_COMMAND_REQUEST,
37287 3ed344546d9e wireproto: start to associate frame generation with a stream Gregory Szorc <gregory.szorc@gmail.com> parents: 37087 diff changeset	323 flags=flags,
3ed344546d9e wireproto: start to associate frame generation with a stream Gregory Szorc <gregory.szorc@gmail.com> parents: 37087 diff changeset	324 payload=payload)
37054 40206e227412 wireproto: define and implement protocol for issuing requests Gregory Szorc <gregory.szorc@gmail.com> parents: diff changeset	325
37292 3d0e2cd86e05 wireproto: use CBOR for command requests Gregory Szorc <gregory.szorc@gmail.com> parents: 37291 diff changeset	326 if not (flags & FLAG_COMMAND_REQUEST_MORE_FRAMES):
3d0e2cd86e05 wireproto: use CBOR for command requests Gregory Szorc <gregory.szorc@gmail.com> parents: 37291 diff changeset	327 break
3d0e2cd86e05 wireproto: use CBOR for command requests Gregory Szorc <gregory.szorc@gmail.com> parents: 37291 diff changeset	328
37054 40206e227412 wireproto: define and implement protocol for issuing requests Gregory Szorc <gregory.szorc@gmail.com> parents: diff changeset	329 if datafh:
40206e227412 wireproto: define and implement protocol for issuing requests Gregory Szorc <gregory.szorc@gmail.com> parents: diff changeset	330 while True:
40206e227412 wireproto: define and implement protocol for issuing requests Gregory Szorc <gregory.szorc@gmail.com> parents: diff changeset	331 data = datafh.read(DEFAULT_MAX_FRAME_SIZE)
40206e227412 wireproto: define and implement protocol for issuing requests Gregory Szorc <gregory.szorc@gmail.com> parents: diff changeset	332
40206e227412 wireproto: define and implement protocol for issuing requests Gregory Szorc <gregory.szorc@gmail.com> parents: diff changeset	333 done = False
40206e227412 wireproto: define and implement protocol for issuing requests Gregory Szorc <gregory.szorc@gmail.com> parents: diff changeset	334 if len(data) == DEFAULT_MAX_FRAME_SIZE:
40206e227412 wireproto: define and implement protocol for issuing requests Gregory Szorc <gregory.szorc@gmail.com> parents: diff changeset	335 flags = FLAG_COMMAND_DATA_CONTINUATION
40206e227412 wireproto: define and implement protocol for issuing requests Gregory Szorc <gregory.szorc@gmail.com> parents: diff changeset	336 else:
40206e227412 wireproto: define and implement protocol for issuing requests Gregory Szorc <gregory.szorc@gmail.com> parents: diff changeset	337 flags = FLAG_COMMAND_DATA_EOS
40206e227412 wireproto: define and implement protocol for issuing requests Gregory Szorc <gregory.szorc@gmail.com> parents: diff changeset	338 assert datafh.read(1) == b''
40206e227412 wireproto: define and implement protocol for issuing requests Gregory Szorc <gregory.szorc@gmail.com> parents: diff changeset	339 done = True
40206e227412 wireproto: define and implement protocol for issuing requests Gregory Szorc <gregory.szorc@gmail.com> parents: diff changeset	340
37287 3ed344546d9e wireproto: start to associate frame generation with a stream Gregory Szorc <gregory.szorc@gmail.com> parents: 37087 diff changeset	341 yield stream.makeframe(requestid=requestid,
3ed344546d9e wireproto: start to associate frame generation with a stream Gregory Szorc <gregory.szorc@gmail.com> parents: 37087 diff changeset	342 typeid=FRAME_TYPE_COMMAND_DATA,
3ed344546d9e wireproto: start to associate frame generation with a stream Gregory Szorc <gregory.szorc@gmail.com> parents: 37087 diff changeset	343 flags=flags,
3ed344546d9e wireproto: start to associate frame generation with a stream Gregory Szorc <gregory.szorc@gmail.com> parents: 37087 diff changeset	344 payload=data)
37054 40206e227412 wireproto: define and implement protocol for issuing requests Gregory Szorc <gregory.szorc@gmail.com> parents: diff changeset	345
40206e227412 wireproto: define and implement protocol for issuing requests Gregory Szorc <gregory.szorc@gmail.com> parents: diff changeset	346 if done:
40206e227412 wireproto: define and implement protocol for issuing requests Gregory Szorc <gregory.szorc@gmail.com> parents: diff changeset	347 break
37055 8c3c47362934 wireproto: implement basic frame reading and processing Gregory Szorc <gregory.szorc@gmail.com> parents: 37054 diff changeset	348
37724 deff7cf7eefd wireprotov2: change frame type and name for command response Gregory Szorc <gregory.szorc@gmail.com> parents: 37723 diff changeset	349 def createcommandresponseframesfrombytes(stream, requestid, data,
deff7cf7eefd wireprotov2: change frame type and name for command response Gregory Szorc <gregory.szorc@gmail.com> parents: 37723 diff changeset	350 maxframesize=DEFAULT_MAX_FRAME_SIZE):
37058 61393f888dfe wireproto: define and implement responses in framing protocol Gregory Szorc <gregory.szorc@gmail.com> parents: 37055 diff changeset	351 """Create a raw frame to send a bytes response from static bytes input.
61393f888dfe wireproto: define and implement responses in framing protocol Gregory Szorc <gregory.szorc@gmail.com> parents: 37055 diff changeset	352
61393f888dfe wireproto: define and implement responses in framing protocol Gregory Szorc <gregory.szorc@gmail.com> parents: 37055 diff changeset	353 Returns a generator of bytearrays.
61393f888dfe wireproto: define and implement responses in framing protocol Gregory Szorc <gregory.szorc@gmail.com> parents: 37055 diff changeset	354 """
37725 3ea8323d6f95 wireprotov2: change command response protocol to include a leading map Gregory Szorc <gregory.szorc@gmail.com> parents: 37724 diff changeset	355 # Automatically send the overall CBOR response map.
39465 36f487a332ad wireprotoframing: use our CBOR module Gregory Szorc <gregory.szorc@gmail.com> parents: 37728 diff changeset	356 overall = b''.join(cborutil.streamencode({b'status': b'ok'}))
37725 3ea8323d6f95 wireprotov2: change command response protocol to include a leading map Gregory Szorc <gregory.szorc@gmail.com> parents: 37724 diff changeset	357 if len(overall) > maxframesize:
3ea8323d6f95 wireprotov2: change command response protocol to include a leading map Gregory Szorc <gregory.szorc@gmail.com> parents: 37724 diff changeset	358 raise error.ProgrammingError('not yet implemented')
37058 61393f888dfe wireproto: define and implement responses in framing protocol Gregory Szorc <gregory.szorc@gmail.com> parents: 37055 diff changeset	359
37725 3ea8323d6f95 wireprotov2: change command response protocol to include a leading map Gregory Szorc <gregory.szorc@gmail.com> parents: 37724 diff changeset	360 # Simple case where we can fit the full response in a single frame.
3ea8323d6f95 wireprotov2: change command response protocol to include a leading map Gregory Szorc <gregory.szorc@gmail.com> parents: 37724 diff changeset	361 if len(overall) + len(data) <= maxframesize:
37724 deff7cf7eefd wireprotov2: change frame type and name for command response Gregory Szorc <gregory.szorc@gmail.com> parents: 37723 diff changeset	362 flags = FLAG_COMMAND_RESPONSE_EOS
37287 3ed344546d9e wireproto: start to associate frame generation with a stream Gregory Szorc <gregory.szorc@gmail.com> parents: 37087 diff changeset	363 yield stream.makeframe(requestid=requestid,
37724 deff7cf7eefd wireprotov2: change frame type and name for command response Gregory Szorc <gregory.szorc@gmail.com> parents: 37723 diff changeset	364 typeid=FRAME_TYPE_COMMAND_RESPONSE,
37485 0b7475ea38cf wireproto: port heads command to wire protocol v2 Gregory Szorc <gregory.szorc@gmail.com> parents: 37476 diff changeset	365 flags=flags,
37725 3ea8323d6f95 wireprotov2: change command response protocol to include a leading map Gregory Szorc <gregory.szorc@gmail.com> parents: 37724 diff changeset	366 payload=overall + data)
37058 61393f888dfe wireproto: define and implement responses in framing protocol Gregory Szorc <gregory.szorc@gmail.com> parents: 37055 diff changeset	367 return
61393f888dfe wireproto: define and implement responses in framing protocol Gregory Szorc <gregory.szorc@gmail.com> parents: 37055 diff changeset	368
37725 3ea8323d6f95 wireprotov2: change command response protocol to include a leading map Gregory Szorc <gregory.szorc@gmail.com> parents: 37724 diff changeset	369 # It's easier to send the overall CBOR map in its own frame than to track
3ea8323d6f95 wireprotov2: change command response protocol to include a leading map Gregory Szorc <gregory.szorc@gmail.com> parents: 37724 diff changeset	370 # offsets.
3ea8323d6f95 wireprotov2: change command response protocol to include a leading map Gregory Szorc <gregory.szorc@gmail.com> parents: 37724 diff changeset	371 yield stream.makeframe(requestid=requestid,
3ea8323d6f95 wireprotov2: change command response protocol to include a leading map Gregory Szorc <gregory.szorc@gmail.com> parents: 37724 diff changeset	372 typeid=FRAME_TYPE_COMMAND_RESPONSE,
3ea8323d6f95 wireprotov2: change command response protocol to include a leading map Gregory Szorc <gregory.szorc@gmail.com> parents: 37724 diff changeset	373 flags=FLAG_COMMAND_RESPONSE_CONTINUATION,
3ea8323d6f95 wireprotov2: change command response protocol to include a leading map Gregory Szorc <gregory.szorc@gmail.com> parents: 37724 diff changeset	374 payload=overall)
3ea8323d6f95 wireprotov2: change command response protocol to include a leading map Gregory Szorc <gregory.szorc@gmail.com> parents: 37724 diff changeset	375
37058 61393f888dfe wireproto: define and implement responses in framing protocol Gregory Szorc <gregory.szorc@gmail.com> parents: 37055 diff changeset	376 offset = 0
61393f888dfe wireproto: define and implement responses in framing protocol Gregory Szorc <gregory.szorc@gmail.com> parents: 37055 diff changeset	377 while True:
61393f888dfe wireproto: define and implement responses in framing protocol Gregory Szorc <gregory.szorc@gmail.com> parents: 37055 diff changeset	378 chunk = data[offset:offset + maxframesize]
61393f888dfe wireproto: define and implement responses in framing protocol Gregory Szorc <gregory.szorc@gmail.com> parents: 37055 diff changeset	379 offset += len(chunk)
61393f888dfe wireproto: define and implement responses in framing protocol Gregory Szorc <gregory.szorc@gmail.com> parents: 37055 diff changeset	380 done = offset == len(data)
61393f888dfe wireproto: define and implement responses in framing protocol Gregory Szorc <gregory.szorc@gmail.com> parents: 37055 diff changeset	381
61393f888dfe wireproto: define and implement responses in framing protocol Gregory Szorc <gregory.szorc@gmail.com> parents: 37055 diff changeset	382 if done:
37724 deff7cf7eefd wireprotov2: change frame type and name for command response Gregory Szorc <gregory.szorc@gmail.com> parents: 37723 diff changeset	383 flags = FLAG_COMMAND_RESPONSE_EOS
37058 61393f888dfe wireproto: define and implement responses in framing protocol Gregory Szorc <gregory.szorc@gmail.com> parents: 37055 diff changeset	384 else:
37724 deff7cf7eefd wireprotov2: change frame type and name for command response Gregory Szorc <gregory.szorc@gmail.com> parents: 37723 diff changeset	385 flags = FLAG_COMMAND_RESPONSE_CONTINUATION
37058 61393f888dfe wireproto: define and implement responses in framing protocol Gregory Szorc <gregory.szorc@gmail.com> parents: 37055 diff changeset	386
37287 3ed344546d9e wireproto: start to associate frame generation with a stream Gregory Szorc <gregory.szorc@gmail.com> parents: 37087 diff changeset	387 yield stream.makeframe(requestid=requestid,
37724 deff7cf7eefd wireprotov2: change frame type and name for command response Gregory Szorc <gregory.szorc@gmail.com> parents: 37723 diff changeset	388 typeid=FRAME_TYPE_COMMAND_RESPONSE,
37287 3ed344546d9e wireproto: start to associate frame generation with a stream Gregory Szorc <gregory.szorc@gmail.com> parents: 37087 diff changeset	389 flags=flags,
3ed344546d9e wireproto: start to associate frame generation with a stream Gregory Szorc <gregory.szorc@gmail.com> parents: 37087 diff changeset	390 payload=chunk)
37058 61393f888dfe wireproto: define and implement responses in framing protocol Gregory Szorc <gregory.szorc@gmail.com> parents: 37055 diff changeset	391
61393f888dfe wireproto: define and implement responses in framing protocol Gregory Szorc <gregory.szorc@gmail.com> parents: 37055 diff changeset	392 if done:
61393f888dfe wireproto: define and implement responses in framing protocol Gregory Szorc <gregory.szorc@gmail.com> parents: 37055 diff changeset	393 break
61393f888dfe wireproto: define and implement responses in framing protocol Gregory Szorc <gregory.szorc@gmail.com> parents: 37055 diff changeset	394
37728 564a3eec6e63 wireprotov2: add support for more response types Gregory Szorc <gregory.szorc@gmail.com> parents: 37726 diff changeset	395 def createbytesresponseframesfromgen(stream, requestid, gen,
564a3eec6e63 wireprotov2: add support for more response types Gregory Szorc <gregory.szorc@gmail.com> parents: 37726 diff changeset	396 maxframesize=DEFAULT_MAX_FRAME_SIZE):
39575 07b58266bce3 wireprotov2: implement commands as a generator of objects Gregory Szorc <gregory.szorc@gmail.com> parents: 39503 diff changeset	397 """Generator of frames from a generator of byte chunks.
37728 564a3eec6e63 wireprotov2: add support for more response types Gregory Szorc <gregory.szorc@gmail.com> parents: 37726 diff changeset	398
39575 07b58266bce3 wireprotov2: implement commands as a generator of objects Gregory Szorc <gregory.szorc@gmail.com> parents: 39503 diff changeset	399 This assumes that another frame will follow whatever this emits. i.e.
07b58266bce3 wireprotov2: implement commands as a generator of objects Gregory Szorc <gregory.szorc@gmail.com> parents: 39503 diff changeset	400 this always emits the continuation flag and never emits the end-of-stream
07b58266bce3 wireprotov2: implement commands as a generator of objects Gregory Szorc <gregory.szorc@gmail.com> parents: 39503 diff changeset	401 flag.
07b58266bce3 wireprotov2: implement commands as a generator of objects Gregory Szorc <gregory.szorc@gmail.com> parents: 39503 diff changeset	402 """
37728 564a3eec6e63 wireprotov2: add support for more response types Gregory Szorc <gregory.szorc@gmail.com> parents: 37726 diff changeset	403 cb = util.chunkbuffer(gen)
39575 07b58266bce3 wireprotov2: implement commands as a generator of objects Gregory Szorc <gregory.szorc@gmail.com> parents: 39503 diff changeset	404 flags = FLAG_COMMAND_RESPONSE_CONTINUATION
37728 564a3eec6e63 wireprotov2: add support for more response types Gregory Szorc <gregory.szorc@gmail.com> parents: 37726 diff changeset	405
564a3eec6e63 wireprotov2: add support for more response types Gregory Szorc <gregory.szorc@gmail.com> parents: 37726 diff changeset	406 while True:
564a3eec6e63 wireprotov2: add support for more response types Gregory Szorc <gregory.szorc@gmail.com> parents: 37726 diff changeset	407 chunk = cb.read(maxframesize)
564a3eec6e63 wireprotov2: add support for more response types Gregory Szorc <gregory.szorc@gmail.com> parents: 37726 diff changeset	408 if not chunk:
564a3eec6e63 wireprotov2: add support for more response types Gregory Szorc <gregory.szorc@gmail.com> parents: 37726 diff changeset	409 break
564a3eec6e63 wireprotov2: add support for more response types Gregory Szorc <gregory.szorc@gmail.com> parents: 37726 diff changeset	410
564a3eec6e63 wireprotov2: add support for more response types Gregory Szorc <gregory.szorc@gmail.com> parents: 37726 diff changeset	411 yield stream.makeframe(requestid=requestid,
564a3eec6e63 wireprotov2: add support for more response types Gregory Szorc <gregory.szorc@gmail.com> parents: 37726 diff changeset	412 typeid=FRAME_TYPE_COMMAND_RESPONSE,
564a3eec6e63 wireprotov2: add support for more response types Gregory Szorc <gregory.szorc@gmail.com> parents: 37726 diff changeset	413 flags=flags,
564a3eec6e63 wireprotov2: add support for more response types Gregory Szorc <gregory.szorc@gmail.com> parents: 37726 diff changeset	414 payload=chunk)
564a3eec6e63 wireprotov2: add support for more response types Gregory Szorc <gregory.szorc@gmail.com> parents: 37726 diff changeset	415
564a3eec6e63 wireprotov2: add support for more response types Gregory Szorc <gregory.szorc@gmail.com> parents: 37726 diff changeset	416 flags \|= FLAG_COMMAND_RESPONSE_CONTINUATION
564a3eec6e63 wireprotov2: add support for more response types Gregory Szorc <gregory.szorc@gmail.com> parents: 37726 diff changeset	417
39575 07b58266bce3 wireprotov2: implement commands as a generator of objects Gregory Szorc <gregory.szorc@gmail.com> parents: 39503 diff changeset	418 def createcommandresponseokframe(stream, requestid):
07b58266bce3 wireprotov2: implement commands as a generator of objects Gregory Szorc <gregory.szorc@gmail.com> parents: 39503 diff changeset	419 overall = b''.join(cborutil.streamencode({b'status': b'ok'}))
07b58266bce3 wireprotov2: implement commands as a generator of objects Gregory Szorc <gregory.szorc@gmail.com> parents: 39503 diff changeset	420
07b58266bce3 wireprotov2: implement commands as a generator of objects Gregory Szorc <gregory.szorc@gmail.com> parents: 39503 diff changeset	421 return stream.makeframe(requestid=requestid,
07b58266bce3 wireprotov2: implement commands as a generator of objects Gregory Szorc <gregory.szorc@gmail.com> parents: 39503 diff changeset	422 typeid=FRAME_TYPE_COMMAND_RESPONSE,
07b58266bce3 wireprotov2: implement commands as a generator of objects Gregory Szorc <gregory.szorc@gmail.com> parents: 39503 diff changeset	423 flags=FLAG_COMMAND_RESPONSE_CONTINUATION,
07b58266bce3 wireprotov2: implement commands as a generator of objects Gregory Szorc <gregory.szorc@gmail.com> parents: 39503 diff changeset	424 payload=overall)
07b58266bce3 wireprotov2: implement commands as a generator of objects Gregory Szorc <gregory.szorc@gmail.com> parents: 39503 diff changeset	425
07b58266bce3 wireprotov2: implement commands as a generator of objects Gregory Szorc <gregory.szorc@gmail.com> parents: 39503 diff changeset	426 def createcommandresponseeosframe(stream, requestid):
07b58266bce3 wireprotov2: implement commands as a generator of objects Gregory Szorc <gregory.szorc@gmail.com> parents: 39503 diff changeset	427 """Create an empty payload frame representing command end-of-stream."""
07b58266bce3 wireprotov2: implement commands as a generator of objects Gregory Szorc <gregory.szorc@gmail.com> parents: 39503 diff changeset	428 return stream.makeframe(requestid=requestid,
07b58266bce3 wireprotov2: implement commands as a generator of objects Gregory Szorc <gregory.szorc@gmail.com> parents: 39503 diff changeset	429 typeid=FRAME_TYPE_COMMAND_RESPONSE,
07b58266bce3 wireprotov2: implement commands as a generator of objects Gregory Szorc <gregory.szorc@gmail.com> parents: 39503 diff changeset	430 flags=FLAG_COMMAND_RESPONSE_EOS,
07b58266bce3 wireprotov2: implement commands as a generator of objects Gregory Szorc <gregory.szorc@gmail.com> parents: 39503 diff changeset	431 payload=b'')
37728 564a3eec6e63 wireprotov2: add support for more response types Gregory Szorc <gregory.szorc@gmail.com> parents: 37726 diff changeset	432
40026 b099e6032f38 wireprotov2: server support for sending content redirects Gregory Szorc <gregory.szorc@gmail.com> parents: 40025 diff changeset	433 def createalternatelocationresponseframe(stream, requestid, location):
b099e6032f38 wireprotov2: server support for sending content redirects Gregory Szorc <gregory.szorc@gmail.com> parents: 40025 diff changeset	434 data = {
b099e6032f38 wireprotov2: server support for sending content redirects Gregory Szorc <gregory.szorc@gmail.com> parents: 40025 diff changeset	435 b'status': b'redirect',
b099e6032f38 wireprotov2: server support for sending content redirects Gregory Szorc <gregory.szorc@gmail.com> parents: 40025 diff changeset	436 b'location': {
b099e6032f38 wireprotov2: server support for sending content redirects Gregory Szorc <gregory.szorc@gmail.com> parents: 40025 diff changeset	437 b'url': location.url,
b099e6032f38 wireprotov2: server support for sending content redirects Gregory Szorc <gregory.szorc@gmail.com> parents: 40025 diff changeset	438 b'mediatype': location.mediatype,
b099e6032f38 wireprotov2: server support for sending content redirects Gregory Szorc <gregory.szorc@gmail.com> parents: 40025 diff changeset	439 }
b099e6032f38 wireprotov2: server support for sending content redirects Gregory Szorc <gregory.szorc@gmail.com> parents: 40025 diff changeset	440 }
b099e6032f38 wireprotov2: server support for sending content redirects Gregory Szorc <gregory.szorc@gmail.com> parents: 40025 diff changeset	441
b099e6032f38 wireprotov2: server support for sending content redirects Gregory Szorc <gregory.szorc@gmail.com> parents: 40025 diff changeset	442 for a in (r'size', r'fullhashes', r'fullhashseed', r'serverdercerts',
b099e6032f38 wireprotov2: server support for sending content redirects Gregory Szorc <gregory.szorc@gmail.com> parents: 40025 diff changeset	443 r'servercadercerts'):
b099e6032f38 wireprotov2: server support for sending content redirects Gregory Szorc <gregory.szorc@gmail.com> parents: 40025 diff changeset	444 value = getattr(location, a)
b099e6032f38 wireprotov2: server support for sending content redirects Gregory Szorc <gregory.szorc@gmail.com> parents: 40025 diff changeset	445 if value is not None:
b099e6032f38 wireprotov2: server support for sending content redirects Gregory Szorc <gregory.szorc@gmail.com> parents: 40025 diff changeset	446 data[b'location'][pycompat.bytestr(a)] = value
b099e6032f38 wireprotov2: server support for sending content redirects Gregory Szorc <gregory.szorc@gmail.com> parents: 40025 diff changeset	447
b099e6032f38 wireprotov2: server support for sending content redirects Gregory Szorc <gregory.szorc@gmail.com> parents: 40025 diff changeset	448 return stream.makeframe(requestid=requestid,
b099e6032f38 wireprotov2: server support for sending content redirects Gregory Szorc <gregory.szorc@gmail.com> parents: 40025 diff changeset	449 typeid=FRAME_TYPE_COMMAND_RESPONSE,
b099e6032f38 wireprotov2: server support for sending content redirects Gregory Szorc <gregory.szorc@gmail.com> parents: 40025 diff changeset	450 flags=FLAG_COMMAND_RESPONSE_CONTINUATION,
b099e6032f38 wireprotov2: server support for sending content redirects Gregory Szorc <gregory.szorc@gmail.com> parents: 40025 diff changeset	451 payload=b''.join(cborutil.streamencode(data)))
b099e6032f38 wireprotov2: server support for sending content redirects Gregory Szorc <gregory.szorc@gmail.com> parents: 40025 diff changeset	452
37728 564a3eec6e63 wireprotov2: add support for more response types Gregory Szorc <gregory.szorc@gmail.com> parents: 37726 diff changeset	453 def createcommanderrorresponse(stream, requestid, message, args=None):
39503 43d92d68ac88 wireprotov2peer: properly format errors Gregory Szorc <gregory.szorc@gmail.com> parents: 39465 diff changeset	454 # TODO should this be using a list of {'msg': ..., 'args': {}} so atom
43d92d68ac88 wireprotov2peer: properly format errors Gregory Szorc <gregory.szorc@gmail.com> parents: 39465 diff changeset	455 # formatting works consistently?
37728 564a3eec6e63 wireprotov2: add support for more response types Gregory Szorc <gregory.szorc@gmail.com> parents: 37726 diff changeset	456 m = {
564a3eec6e63 wireprotov2: add support for more response types Gregory Szorc <gregory.szorc@gmail.com> parents: 37726 diff changeset	457 b'status': b'error',
564a3eec6e63 wireprotov2: add support for more response types Gregory Szorc <gregory.szorc@gmail.com> parents: 37726 diff changeset	458 b'error': {
564a3eec6e63 wireprotov2: add support for more response types Gregory Szorc <gregory.szorc@gmail.com> parents: 37726 diff changeset	459 b'message': message,
564a3eec6e63 wireprotov2: add support for more response types Gregory Szorc <gregory.szorc@gmail.com> parents: 37726 diff changeset	460 }
564a3eec6e63 wireprotov2: add support for more response types Gregory Szorc <gregory.szorc@gmail.com> parents: 37726 diff changeset	461 }
564a3eec6e63 wireprotov2: add support for more response types Gregory Szorc <gregory.szorc@gmail.com> parents: 37726 diff changeset	462
564a3eec6e63 wireprotov2: add support for more response types Gregory Szorc <gregory.szorc@gmail.com> parents: 37726 diff changeset	463 if args:
564a3eec6e63 wireprotov2: add support for more response types Gregory Szorc <gregory.szorc@gmail.com> parents: 37726 diff changeset	464 m[b'error'][b'args'] = args
564a3eec6e63 wireprotov2: add support for more response types Gregory Szorc <gregory.szorc@gmail.com> parents: 37726 diff changeset	465
39465 36f487a332ad wireprotoframing: use our CBOR module Gregory Szorc <gregory.szorc@gmail.com> parents: 37728 diff changeset	466 overall = b''.join(cborutil.streamencode(m))
37728 564a3eec6e63 wireprotov2: add support for more response types Gregory Szorc <gregory.szorc@gmail.com> parents: 37726 diff changeset	467
564a3eec6e63 wireprotov2: add support for more response types Gregory Szorc <gregory.szorc@gmail.com> parents: 37726 diff changeset	468 yield stream.makeframe(requestid=requestid,
564a3eec6e63 wireprotov2: add support for more response types Gregory Szorc <gregory.szorc@gmail.com> parents: 37726 diff changeset	469 typeid=FRAME_TYPE_COMMAND_RESPONSE,
564a3eec6e63 wireprotov2: add support for more response types Gregory Szorc <gregory.szorc@gmail.com> parents: 37726 diff changeset	470 flags=FLAG_COMMAND_RESPONSE_EOS,
564a3eec6e63 wireprotov2: add support for more response types Gregory Szorc <gregory.szorc@gmail.com> parents: 37726 diff changeset	471 payload=overall)
564a3eec6e63 wireprotov2: add support for more response types Gregory Szorc <gregory.szorc@gmail.com> parents: 37726 diff changeset	472
37726 0c184ca594bb wireprotov2: change behavior of error frame Gregory Szorc <gregory.szorc@gmail.com> parents: 37725 diff changeset	473 def createerrorframe(stream, requestid, msg, errtype):
37058 61393f888dfe wireproto: define and implement responses in framing protocol Gregory Szorc <gregory.szorc@gmail.com> parents: 37055 diff changeset	474 # TODO properly handle frame size limits.
61393f888dfe wireproto: define and implement responses in framing protocol Gregory Szorc <gregory.szorc@gmail.com> parents: 37055 diff changeset	475 assert len(msg) <= DEFAULT_MAX_FRAME_SIZE
61393f888dfe wireproto: define and implement responses in framing protocol Gregory Szorc <gregory.szorc@gmail.com> parents: 37055 diff changeset	476
39465 36f487a332ad wireprotoframing: use our CBOR module Gregory Szorc <gregory.szorc@gmail.com> parents: 37728 diff changeset	477 payload = b''.join(cborutil.streamencode({
37726 0c184ca594bb wireprotov2: change behavior of error frame Gregory Szorc <gregory.szorc@gmail.com> parents: 37725 diff changeset	478 b'type': errtype,
0c184ca594bb wireprotov2: change behavior of error frame Gregory Szorc <gregory.szorc@gmail.com> parents: 37725 diff changeset	479 b'message': [{b'msg': msg}],
39465 36f487a332ad wireprotoframing: use our CBOR module Gregory Szorc <gregory.szorc@gmail.com> parents: 37728 diff changeset	480 }))
37058 61393f888dfe wireproto: define and implement responses in framing protocol Gregory Szorc <gregory.szorc@gmail.com> parents: 37055 diff changeset	481
37287 3ed344546d9e wireproto: start to associate frame generation with a stream Gregory Szorc <gregory.szorc@gmail.com> parents: 37087 diff changeset	482 yield stream.makeframe(requestid=requestid,
3ed344546d9e wireproto: start to associate frame generation with a stream Gregory Szorc <gregory.szorc@gmail.com> parents: 37087 diff changeset	483 typeid=FRAME_TYPE_ERROR_RESPONSE,
37726 0c184ca594bb wireprotov2: change behavior of error frame Gregory Szorc <gregory.szorc@gmail.com> parents: 37725 diff changeset	484 flags=0,
0c184ca594bb wireprotov2: change behavior of error frame Gregory Szorc <gregory.szorc@gmail.com> parents: 37725 diff changeset	485 payload=payload)
37058 61393f888dfe wireproto: define and implement responses in framing protocol Gregory Szorc <gregory.szorc@gmail.com> parents: 37055 diff changeset	486
37319 36d17f37db91 wireproto: convert human output frames to CBOR Gregory Szorc <gregory.szorc@gmail.com> parents: 37299 diff changeset	487 def createtextoutputframe(stream, requestid, atoms,
36d17f37db91 wireproto: convert human output frames to CBOR Gregory Szorc <gregory.szorc@gmail.com> parents: 37299 diff changeset	488 maxframesize=DEFAULT_MAX_FRAME_SIZE):
37063 0a6c5cc09a88 wireproto: define human output side channel frame Gregory Szorc <gregory.szorc@gmail.com> parents: 37061 diff changeset	489 """Create a text output frame to render text to people.
0a6c5cc09a88 wireproto: define human output side channel frame Gregory Szorc <gregory.szorc@gmail.com> parents: 37061 diff changeset	490
0a6c5cc09a88 wireproto: define human output side channel frame Gregory Szorc <gregory.szorc@gmail.com> parents: 37061 diff changeset	491 ``atoms`` is a 3-tuple of (formatting string, args, labels).
0a6c5cc09a88 wireproto: define human output side channel frame Gregory Szorc <gregory.szorc@gmail.com> parents: 37061 diff changeset	492
0a6c5cc09a88 wireproto: define human output side channel frame Gregory Szorc <gregory.szorc@gmail.com> parents: 37061 diff changeset	493 The formatting string contains ``%s`` tokens to be replaced by the
0a6c5cc09a88 wireproto: define human output side channel frame Gregory Szorc <gregory.szorc@gmail.com> parents: 37061 diff changeset	494 corresponding indexed entry in ``args``. ``labels`` is an iterable of
0a6c5cc09a88 wireproto: define human output side channel frame Gregory Szorc <gregory.szorc@gmail.com> parents: 37061 diff changeset	495 formatters to be applied at rendering time. In terms of the ``ui``
0a6c5cc09a88 wireproto: define human output side channel frame Gregory Szorc <gregory.szorc@gmail.com> parents: 37061 diff changeset	496 class, each atom corresponds to a ``ui.write()``.
0a6c5cc09a88 wireproto: define human output side channel frame Gregory Szorc <gregory.szorc@gmail.com> parents: 37061 diff changeset	497 """
37319 36d17f37db91 wireproto: convert human output frames to CBOR Gregory Szorc <gregory.szorc@gmail.com> parents: 37299 diff changeset	498 atomdicts = []
37063 0a6c5cc09a88 wireproto: define human output side channel frame Gregory Szorc <gregory.szorc@gmail.com> parents: 37061 diff changeset	499
0a6c5cc09a88 wireproto: define human output side channel frame Gregory Szorc <gregory.szorc@gmail.com> parents: 37061 diff changeset	500 for (formatting, args, labels) in atoms:
0a6c5cc09a88 wireproto: define human output side channel frame Gregory Szorc <gregory.szorc@gmail.com> parents: 37061 diff changeset	501 # TODO look for localstr, other types here?
0a6c5cc09a88 wireproto: define human output side channel frame Gregory Szorc <gregory.szorc@gmail.com> parents: 37061 diff changeset	502
0a6c5cc09a88 wireproto: define human output side channel frame Gregory Szorc <gregory.szorc@gmail.com> parents: 37061 diff changeset	503 if not isinstance(formatting, bytes):
0a6c5cc09a88 wireproto: define human output side channel frame Gregory Szorc <gregory.szorc@gmail.com> parents: 37061 diff changeset	504 raise ValueError('must use bytes formatting strings')
0a6c5cc09a88 wireproto: define human output side channel frame Gregory Szorc <gregory.szorc@gmail.com> parents: 37061 diff changeset	505 for arg in args:
0a6c5cc09a88 wireproto: define human output side channel frame Gregory Szorc <gregory.szorc@gmail.com> parents: 37061 diff changeset	506 if not isinstance(arg, bytes):
0a6c5cc09a88 wireproto: define human output side channel frame Gregory Szorc <gregory.szorc@gmail.com> parents: 37061 diff changeset	507 raise ValueError('must use bytes for arguments')
0a6c5cc09a88 wireproto: define human output side channel frame Gregory Szorc <gregory.szorc@gmail.com> parents: 37061 diff changeset	508 for label in labels:
0a6c5cc09a88 wireproto: define human output side channel frame Gregory Szorc <gregory.szorc@gmail.com> parents: 37061 diff changeset	509 if not isinstance(label, bytes):
0a6c5cc09a88 wireproto: define human output side channel frame Gregory Szorc <gregory.szorc@gmail.com> parents: 37061 diff changeset	510 raise ValueError('must use bytes for labels')
0a6c5cc09a88 wireproto: define human output side channel frame Gregory Szorc <gregory.szorc@gmail.com> parents: 37061 diff changeset	511
37319 36d17f37db91 wireproto: convert human output frames to CBOR Gregory Szorc <gregory.szorc@gmail.com> parents: 37299 diff changeset	512 # Formatting string must be ASCII.
36d17f37db91 wireproto: convert human output frames to CBOR Gregory Szorc <gregory.szorc@gmail.com> parents: 37299 diff changeset	513 formatting = formatting.decode(r'ascii', r'replace').encode(r'ascii')
37063 0a6c5cc09a88 wireproto: define human output side channel frame Gregory Szorc <gregory.szorc@gmail.com> parents: 37061 diff changeset	514
0a6c5cc09a88 wireproto: define human output side channel frame Gregory Szorc <gregory.szorc@gmail.com> parents: 37061 diff changeset	515 # Arguments must be UTF-8.
0a6c5cc09a88 wireproto: define human output side channel frame Gregory Szorc <gregory.szorc@gmail.com> parents: 37061 diff changeset	516 args = [a.decode(r'utf-8', r'replace').encode(r'utf-8') for a in args]
0a6c5cc09a88 wireproto: define human output side channel frame Gregory Szorc <gregory.szorc@gmail.com> parents: 37061 diff changeset	517
0a6c5cc09a88 wireproto: define human output side channel frame Gregory Szorc <gregory.szorc@gmail.com> parents: 37061 diff changeset	518 # Labels must be ASCII.
0a6c5cc09a88 wireproto: define human output side channel frame Gregory Szorc <gregory.szorc@gmail.com> parents: 37061 diff changeset	519 labels = [l.decode(r'ascii', r'strict').encode(r'ascii')
0a6c5cc09a88 wireproto: define human output side channel frame Gregory Szorc <gregory.szorc@gmail.com> parents: 37061 diff changeset	520 for l in labels]
0a6c5cc09a88 wireproto: define human output side channel frame Gregory Szorc <gregory.szorc@gmail.com> parents: 37061 diff changeset	521
37319 36d17f37db91 wireproto: convert human output frames to CBOR Gregory Szorc <gregory.szorc@gmail.com> parents: 37299 diff changeset	522 atom = {b'msg': formatting}
36d17f37db91 wireproto: convert human output frames to CBOR Gregory Szorc <gregory.szorc@gmail.com> parents: 37299 diff changeset	523 if args:
36d17f37db91 wireproto: convert human output frames to CBOR Gregory Szorc <gregory.szorc@gmail.com> parents: 37299 diff changeset	524 atom[b'args'] = args
36d17f37db91 wireproto: convert human output frames to CBOR Gregory Szorc <gregory.szorc@gmail.com> parents: 37299 diff changeset	525 if labels:
36d17f37db91 wireproto: convert human output frames to CBOR Gregory Szorc <gregory.szorc@gmail.com> parents: 37299 diff changeset	526 atom[b'labels'] = labels
37063 0a6c5cc09a88 wireproto: define human output side channel frame Gregory Szorc <gregory.szorc@gmail.com> parents: 37061 diff changeset	527
37319 36d17f37db91 wireproto: convert human output frames to CBOR Gregory Szorc <gregory.szorc@gmail.com> parents: 37299 diff changeset	528 atomdicts.append(atom)
37063 0a6c5cc09a88 wireproto: define human output side channel frame Gregory Szorc <gregory.szorc@gmail.com> parents: 37061 diff changeset	529
39465 36f487a332ad wireprotoframing: use our CBOR module Gregory Szorc <gregory.szorc@gmail.com> parents: 37728 diff changeset	530 payload = b''.join(cborutil.streamencode(atomdicts))
37063 0a6c5cc09a88 wireproto: define human output side channel frame Gregory Szorc <gregory.szorc@gmail.com> parents: 37061 diff changeset	531
37319 36d17f37db91 wireproto: convert human output frames to CBOR Gregory Szorc <gregory.szorc@gmail.com> parents: 37299 diff changeset	532 if len(payload) > maxframesize:
37063 0a6c5cc09a88 wireproto: define human output side channel frame Gregory Szorc <gregory.szorc@gmail.com> parents: 37061 diff changeset	533 raise ValueError('cannot encode data in a single frame')
0a6c5cc09a88 wireproto: define human output side channel frame Gregory Szorc <gregory.szorc@gmail.com> parents: 37061 diff changeset	534
37287 3ed344546d9e wireproto: start to associate frame generation with a stream Gregory Szorc <gregory.szorc@gmail.com> parents: 37087 diff changeset	535 yield stream.makeframe(requestid=requestid,
3ed344546d9e wireproto: start to associate frame generation with a stream Gregory Szorc <gregory.szorc@gmail.com> parents: 37087 diff changeset	536 typeid=FRAME_TYPE_TEXT_OUTPUT,
3ed344546d9e wireproto: start to associate frame generation with a stream Gregory Szorc <gregory.szorc@gmail.com> parents: 37087 diff changeset	537 flags=0,
37319 36d17f37db91 wireproto: convert human output frames to CBOR Gregory Szorc <gregory.szorc@gmail.com> parents: 37299 diff changeset	538 payload=payload)
37287 3ed344546d9e wireproto: start to associate frame generation with a stream Gregory Szorc <gregory.szorc@gmail.com> parents: 37087 diff changeset	539
39576 84bf6ded9317 wireprotoframing: buffer emitted data to reduce frame count Gregory Szorc <gregory.szorc@gmail.com> parents: 39575 diff changeset	540 class bufferingcommandresponseemitter(object):
84bf6ded9317 wireprotoframing: buffer emitted data to reduce frame count Gregory Szorc <gregory.szorc@gmail.com> parents: 39575 diff changeset	541 """Helper object to emit command response frames intelligently.
84bf6ded9317 wireprotoframing: buffer emitted data to reduce frame count Gregory Szorc <gregory.szorc@gmail.com> parents: 39575 diff changeset	542
84bf6ded9317 wireprotoframing: buffer emitted data to reduce frame count Gregory Szorc <gregory.szorc@gmail.com> parents: 39575 diff changeset	543 Raw command response data is likely emitted in chunks much smaller
84bf6ded9317 wireprotoframing: buffer emitted data to reduce frame count Gregory Szorc <gregory.szorc@gmail.com> parents: 39575 diff changeset	544 than what can fit in a single frame. This class exists to buffer
84bf6ded9317 wireprotoframing: buffer emitted data to reduce frame count Gregory Szorc <gregory.szorc@gmail.com> parents: 39575 diff changeset	545 chunks until enough data is available to fit in a single frame.
84bf6ded9317 wireprotoframing: buffer emitted data to reduce frame count Gregory Szorc <gregory.szorc@gmail.com> parents: 39575 diff changeset	546
84bf6ded9317 wireprotoframing: buffer emitted data to reduce frame count Gregory Szorc <gregory.szorc@gmail.com> parents: 39575 diff changeset	547 TODO we'll need something like this when compression is supported.
84bf6ded9317 wireprotoframing: buffer emitted data to reduce frame count Gregory Szorc <gregory.szorc@gmail.com> parents: 39575 diff changeset	548 So it might make sense to implement this functionality at the stream
84bf6ded9317 wireprotoframing: buffer emitted data to reduce frame count Gregory Szorc <gregory.szorc@gmail.com> parents: 39575 diff changeset	549 level.
84bf6ded9317 wireprotoframing: buffer emitted data to reduce frame count Gregory Szorc <gregory.szorc@gmail.com> parents: 39575 diff changeset	550 """
84bf6ded9317 wireprotoframing: buffer emitted data to reduce frame count Gregory Szorc <gregory.szorc@gmail.com> parents: 39575 diff changeset	551 def __init__(self, stream, requestid, maxframesize=DEFAULT_MAX_FRAME_SIZE):
84bf6ded9317 wireprotoframing: buffer emitted data to reduce frame count Gregory Szorc <gregory.szorc@gmail.com> parents: 39575 diff changeset	552 self._stream = stream
84bf6ded9317 wireprotoframing: buffer emitted data to reduce frame count Gregory Szorc <gregory.szorc@gmail.com> parents: 39575 diff changeset	553 self._requestid = requestid
84bf6ded9317 wireprotoframing: buffer emitted data to reduce frame count Gregory Szorc <gregory.szorc@gmail.com> parents: 39575 diff changeset	554 self._maxsize = maxframesize
84bf6ded9317 wireprotoframing: buffer emitted data to reduce frame count Gregory Szorc <gregory.szorc@gmail.com> parents: 39575 diff changeset	555 self._chunks = []
84bf6ded9317 wireprotoframing: buffer emitted data to reduce frame count Gregory Szorc <gregory.szorc@gmail.com> parents: 39575 diff changeset	556 self._chunkssize = 0
84bf6ded9317 wireprotoframing: buffer emitted data to reduce frame count Gregory Szorc <gregory.szorc@gmail.com> parents: 39575 diff changeset	557
84bf6ded9317 wireprotoframing: buffer emitted data to reduce frame count Gregory Szorc <gregory.szorc@gmail.com> parents: 39575 diff changeset	558 def send(self, data):
84bf6ded9317 wireprotoframing: buffer emitted data to reduce frame count Gregory Szorc <gregory.szorc@gmail.com> parents: 39575 diff changeset	559 """Send new data for emission.
84bf6ded9317 wireprotoframing: buffer emitted data to reduce frame count Gregory Szorc <gregory.szorc@gmail.com> parents: 39575 diff changeset	560
84bf6ded9317 wireprotoframing: buffer emitted data to reduce frame count Gregory Szorc <gregory.szorc@gmail.com> parents: 39575 diff changeset	561 Is a generator of new frames that were derived from the new input.
84bf6ded9317 wireprotoframing: buffer emitted data to reduce frame count Gregory Szorc <gregory.szorc@gmail.com> parents: 39575 diff changeset	562
84bf6ded9317 wireprotoframing: buffer emitted data to reduce frame count Gregory Szorc <gregory.szorc@gmail.com> parents: 39575 diff changeset	563 If the special input ``None`` is received, flushes all buffered
84bf6ded9317 wireprotoframing: buffer emitted data to reduce frame count Gregory Szorc <gregory.szorc@gmail.com> parents: 39575 diff changeset	564 data to frames.
84bf6ded9317 wireprotoframing: buffer emitted data to reduce frame count Gregory Szorc <gregory.szorc@gmail.com> parents: 39575 diff changeset	565 """
84bf6ded9317 wireprotoframing: buffer emitted data to reduce frame count Gregory Szorc <gregory.szorc@gmail.com> parents: 39575 diff changeset	566
84bf6ded9317 wireprotoframing: buffer emitted data to reduce frame count Gregory Szorc <gregory.szorc@gmail.com> parents: 39575 diff changeset	567 if data is None:
84bf6ded9317 wireprotoframing: buffer emitted data to reduce frame count Gregory Szorc <gregory.szorc@gmail.com> parents: 39575 diff changeset	568 for frame in self._flush():
84bf6ded9317 wireprotoframing: buffer emitted data to reduce frame count Gregory Szorc <gregory.szorc@gmail.com> parents: 39575 diff changeset	569 yield frame
84bf6ded9317 wireprotoframing: buffer emitted data to reduce frame count Gregory Szorc <gregory.szorc@gmail.com> parents: 39575 diff changeset	570 return
84bf6ded9317 wireprotoframing: buffer emitted data to reduce frame count Gregory Szorc <gregory.szorc@gmail.com> parents: 39575 diff changeset	571
84bf6ded9317 wireprotoframing: buffer emitted data to reduce frame count Gregory Szorc <gregory.szorc@gmail.com> parents: 39575 diff changeset	572 # There is a ton of potential to do more complicated things here.
84bf6ded9317 wireprotoframing: buffer emitted data to reduce frame count Gregory Szorc <gregory.szorc@gmail.com> parents: 39575 diff changeset	573 # Our immediate goal is to coalesce small chunks into big frames,
84bf6ded9317 wireprotoframing: buffer emitted data to reduce frame count Gregory Szorc <gregory.szorc@gmail.com> parents: 39575 diff changeset	574 # not achieve the fewest number of frames possible. So we go with
84bf6ded9317 wireprotoframing: buffer emitted data to reduce frame count Gregory Szorc <gregory.szorc@gmail.com> parents: 39575 diff changeset	575 # a simple implementation:
84bf6ded9317 wireprotoframing: buffer emitted data to reduce frame count Gregory Szorc <gregory.szorc@gmail.com> parents: 39575 diff changeset	576 #
84bf6ded9317 wireprotoframing: buffer emitted data to reduce frame count Gregory Szorc <gregory.szorc@gmail.com> parents: 39575 diff changeset	577 # * If a chunk is too large for a frame, we flush and emit frames
84bf6ded9317 wireprotoframing: buffer emitted data to reduce frame count Gregory Szorc <gregory.szorc@gmail.com> parents: 39575 diff changeset	578 # for the new chunk.
84bf6ded9317 wireprotoframing: buffer emitted data to reduce frame count Gregory Szorc <gregory.szorc@gmail.com> parents: 39575 diff changeset	579 # * If a chunk can be buffered without total buffered size limits
84bf6ded9317 wireprotoframing: buffer emitted data to reduce frame count Gregory Szorc <gregory.szorc@gmail.com> parents: 39575 diff changeset	580 # being exceeded, we do that.
84bf6ded9317 wireprotoframing: buffer emitted data to reduce frame count Gregory Szorc <gregory.szorc@gmail.com> parents: 39575 diff changeset	581 # * If a chunk causes us to go over our buffering limit, we flush
84bf6ded9317 wireprotoframing: buffer emitted data to reduce frame count Gregory Szorc <gregory.szorc@gmail.com> parents: 39575 diff changeset	582 # and then buffer the new chunk.
84bf6ded9317 wireprotoframing: buffer emitted data to reduce frame count Gregory Szorc <gregory.szorc@gmail.com> parents: 39575 diff changeset	583
84bf6ded9317 wireprotoframing: buffer emitted data to reduce frame count Gregory Szorc <gregory.szorc@gmail.com> parents: 39575 diff changeset	584 if len(data) > self._maxsize:
84bf6ded9317 wireprotoframing: buffer emitted data to reduce frame count Gregory Szorc <gregory.szorc@gmail.com> parents: 39575 diff changeset	585 for frame in self._flush():
84bf6ded9317 wireprotoframing: buffer emitted data to reduce frame count Gregory Szorc <gregory.szorc@gmail.com> parents: 39575 diff changeset	586 yield frame
84bf6ded9317 wireprotoframing: buffer emitted data to reduce frame count Gregory Szorc <gregory.szorc@gmail.com> parents: 39575 diff changeset	587
84bf6ded9317 wireprotoframing: buffer emitted data to reduce frame count Gregory Szorc <gregory.szorc@gmail.com> parents: 39575 diff changeset	588 # Now emit frames for the big chunk.
84bf6ded9317 wireprotoframing: buffer emitted data to reduce frame count Gregory Szorc <gregory.szorc@gmail.com> parents: 39575 diff changeset	589 offset = 0
84bf6ded9317 wireprotoframing: buffer emitted data to reduce frame count Gregory Szorc <gregory.szorc@gmail.com> parents: 39575 diff changeset	590 while True:
84bf6ded9317 wireprotoframing: buffer emitted data to reduce frame count Gregory Szorc <gregory.szorc@gmail.com> parents: 39575 diff changeset	591 chunk = data[offset:offset + self._maxsize]
84bf6ded9317 wireprotoframing: buffer emitted data to reduce frame count Gregory Szorc <gregory.szorc@gmail.com> parents: 39575 diff changeset	592 offset += len(chunk)
84bf6ded9317 wireprotoframing: buffer emitted data to reduce frame count Gregory Szorc <gregory.szorc@gmail.com> parents: 39575 diff changeset	593
84bf6ded9317 wireprotoframing: buffer emitted data to reduce frame count Gregory Szorc <gregory.szorc@gmail.com> parents: 39575 diff changeset	594 yield self._stream.makeframe(
84bf6ded9317 wireprotoframing: buffer emitted data to reduce frame count Gregory Szorc <gregory.szorc@gmail.com> parents: 39575 diff changeset	595 self._requestid,
84bf6ded9317 wireprotoframing: buffer emitted data to reduce frame count Gregory Szorc <gregory.szorc@gmail.com> parents: 39575 diff changeset	596 typeid=FRAME_TYPE_COMMAND_RESPONSE,
84bf6ded9317 wireprotoframing: buffer emitted data to reduce frame count Gregory Szorc <gregory.szorc@gmail.com> parents: 39575 diff changeset	597 flags=FLAG_COMMAND_RESPONSE_CONTINUATION,
84bf6ded9317 wireprotoframing: buffer emitted data to reduce frame count Gregory Szorc <gregory.szorc@gmail.com> parents: 39575 diff changeset	598 payload=chunk)
84bf6ded9317 wireprotoframing: buffer emitted data to reduce frame count Gregory Szorc <gregory.szorc@gmail.com> parents: 39575 diff changeset	599
84bf6ded9317 wireprotoframing: buffer emitted data to reduce frame count Gregory Szorc <gregory.szorc@gmail.com> parents: 39575 diff changeset	600 if offset == len(data):
84bf6ded9317 wireprotoframing: buffer emitted data to reduce frame count Gregory Szorc <gregory.szorc@gmail.com> parents: 39575 diff changeset	601 return
84bf6ded9317 wireprotoframing: buffer emitted data to reduce frame count Gregory Szorc <gregory.szorc@gmail.com> parents: 39575 diff changeset	602
84bf6ded9317 wireprotoframing: buffer emitted data to reduce frame count Gregory Szorc <gregory.szorc@gmail.com> parents: 39575 diff changeset	603 # If we don't have enough to constitute a full frame, buffer and
84bf6ded9317 wireprotoframing: buffer emitted data to reduce frame count Gregory Szorc <gregory.szorc@gmail.com> parents: 39575 diff changeset	604 # return.
84bf6ded9317 wireprotoframing: buffer emitted data to reduce frame count Gregory Szorc <gregory.szorc@gmail.com> parents: 39575 diff changeset	605 if len(data) + self._chunkssize < self._maxsize:
84bf6ded9317 wireprotoframing: buffer emitted data to reduce frame count Gregory Szorc <gregory.szorc@gmail.com> parents: 39575 diff changeset	606 self._chunks.append(data)
84bf6ded9317 wireprotoframing: buffer emitted data to reduce frame count Gregory Szorc <gregory.szorc@gmail.com> parents: 39575 diff changeset	607 self._chunkssize += len(data)
84bf6ded9317 wireprotoframing: buffer emitted data to reduce frame count Gregory Szorc <gregory.szorc@gmail.com> parents: 39575 diff changeset	608 return
84bf6ded9317 wireprotoframing: buffer emitted data to reduce frame count Gregory Szorc <gregory.szorc@gmail.com> parents: 39575 diff changeset	609
84bf6ded9317 wireprotoframing: buffer emitted data to reduce frame count Gregory Szorc <gregory.szorc@gmail.com> parents: 39575 diff changeset	610 # Else flush what we have and buffer the new chunk. We could do
84bf6ded9317 wireprotoframing: buffer emitted data to reduce frame count Gregory Szorc <gregory.szorc@gmail.com> parents: 39575 diff changeset	611 # something more intelligent here, like break the chunk. Let's
84bf6ded9317 wireprotoframing: buffer emitted data to reduce frame count Gregory Szorc <gregory.szorc@gmail.com> parents: 39575 diff changeset	612 # keep things simple for now.
84bf6ded9317 wireprotoframing: buffer emitted data to reduce frame count Gregory Szorc <gregory.szorc@gmail.com> parents: 39575 diff changeset	613 for frame in self._flush():
84bf6ded9317 wireprotoframing: buffer emitted data to reduce frame count Gregory Szorc <gregory.szorc@gmail.com> parents: 39575 diff changeset	614 yield frame
84bf6ded9317 wireprotoframing: buffer emitted data to reduce frame count Gregory Szorc <gregory.szorc@gmail.com> parents: 39575 diff changeset	615
84bf6ded9317 wireprotoframing: buffer emitted data to reduce frame count Gregory Szorc <gregory.szorc@gmail.com> parents: 39575 diff changeset	616 self._chunks.append(data)
84bf6ded9317 wireprotoframing: buffer emitted data to reduce frame count Gregory Szorc <gregory.szorc@gmail.com> parents: 39575 diff changeset	617 self._chunkssize = len(data)
84bf6ded9317 wireprotoframing: buffer emitted data to reduce frame count Gregory Szorc <gregory.szorc@gmail.com> parents: 39575 diff changeset	618
84bf6ded9317 wireprotoframing: buffer emitted data to reduce frame count Gregory Szorc <gregory.szorc@gmail.com> parents: 39575 diff changeset	619 def _flush(self):
84bf6ded9317 wireprotoframing: buffer emitted data to reduce frame count Gregory Szorc <gregory.szorc@gmail.com> parents: 39575 diff changeset	620 payload = b''.join(self._chunks)
84bf6ded9317 wireprotoframing: buffer emitted data to reduce frame count Gregory Szorc <gregory.szorc@gmail.com> parents: 39575 diff changeset	621 assert len(payload) <= self._maxsize
84bf6ded9317 wireprotoframing: buffer emitted data to reduce frame count Gregory Szorc <gregory.szorc@gmail.com> parents: 39575 diff changeset	622
84bf6ded9317 wireprotoframing: buffer emitted data to reduce frame count Gregory Szorc <gregory.szorc@gmail.com> parents: 39575 diff changeset	623 self._chunks[:] = []
84bf6ded9317 wireprotoframing: buffer emitted data to reduce frame count Gregory Szorc <gregory.szorc@gmail.com> parents: 39575 diff changeset	624 self._chunkssize = 0
84bf6ded9317 wireprotoframing: buffer emitted data to reduce frame count Gregory Szorc <gregory.szorc@gmail.com> parents: 39575 diff changeset	625
84bf6ded9317 wireprotoframing: buffer emitted data to reduce frame count Gregory Szorc <gregory.szorc@gmail.com> parents: 39575 diff changeset	626 yield self._stream.makeframe(
84bf6ded9317 wireprotoframing: buffer emitted data to reduce frame count Gregory Szorc <gregory.szorc@gmail.com> parents: 39575 diff changeset	627 self._requestid,
84bf6ded9317 wireprotoframing: buffer emitted data to reduce frame count Gregory Szorc <gregory.szorc@gmail.com> parents: 39575 diff changeset	628 typeid=FRAME_TYPE_COMMAND_RESPONSE,
84bf6ded9317 wireprotoframing: buffer emitted data to reduce frame count Gregory Szorc <gregory.szorc@gmail.com> parents: 39575 diff changeset	629 flags=FLAG_COMMAND_RESPONSE_CONTINUATION,
84bf6ded9317 wireprotoframing: buffer emitted data to reduce frame count Gregory Szorc <gregory.szorc@gmail.com> parents: 39575 diff changeset	630 payload=payload)
84bf6ded9317 wireprotoframing: buffer emitted data to reduce frame count Gregory Szorc <gregory.szorc@gmail.com> parents: 39575 diff changeset	631
37287 3ed344546d9e wireproto: start to associate frame generation with a stream Gregory Szorc <gregory.szorc@gmail.com> parents: 37087 diff changeset	632 class stream(object):
3ed344546d9e wireproto: start to associate frame generation with a stream Gregory Szorc <gregory.szorc@gmail.com> parents: 37087 diff changeset	633 """Represents a logical unidirectional series of frames."""
3ed344546d9e wireproto: start to associate frame generation with a stream Gregory Szorc <gregory.szorc@gmail.com> parents: 37087 diff changeset	634
37288 9bfcbe4f4745 wireproto: add streams to frame-based protocol Gregory Szorc <gregory.szorc@gmail.com> parents: 37287 diff changeset	635 def __init__(self, streamid, active=False):
9bfcbe4f4745 wireproto: add streams to frame-based protocol Gregory Szorc <gregory.szorc@gmail.com> parents: 37287 diff changeset	636 self.streamid = streamid
37655 b9502b5f2066 wireprotoframing: use value passed into function Gregory Szorc <gregory.szorc@gmail.com> parents: 37544 diff changeset	637 self._active = active
37288 9bfcbe4f4745 wireproto: add streams to frame-based protocol Gregory Szorc <gregory.szorc@gmail.com> parents: 37287 diff changeset	638
37287 3ed344546d9e wireproto: start to associate frame generation with a stream Gregory Szorc <gregory.szorc@gmail.com> parents: 37087 diff changeset	639 def makeframe(self, requestid, typeid, flags, payload):
3ed344546d9e wireproto: start to associate frame generation with a stream Gregory Szorc <gregory.szorc@gmail.com> parents: 37087 diff changeset	640 """Create a frame to be sent out over this stream.
3ed344546d9e wireproto: start to associate frame generation with a stream Gregory Szorc <gregory.szorc@gmail.com> parents: 37087 diff changeset	641
3ed344546d9e wireproto: start to associate frame generation with a stream Gregory Szorc <gregory.szorc@gmail.com> parents: 37087 diff changeset	642 Only returns the frame instance. Does not actually send it.
3ed344546d9e wireproto: start to associate frame generation with a stream Gregory Szorc <gregory.szorc@gmail.com> parents: 37087 diff changeset	643 """
37288 9bfcbe4f4745 wireproto: add streams to frame-based protocol Gregory Szorc <gregory.szorc@gmail.com> parents: 37287 diff changeset	644 streamflags = 0
9bfcbe4f4745 wireproto: add streams to frame-based protocol Gregory Szorc <gregory.szorc@gmail.com> parents: 37287 diff changeset	645 if not self._active:
9bfcbe4f4745 wireproto: add streams to frame-based protocol Gregory Szorc <gregory.szorc@gmail.com> parents: 37287 diff changeset	646 streamflags \|= STREAM_FLAG_BEGIN_STREAM
9bfcbe4f4745 wireproto: add streams to frame-based protocol Gregory Szorc <gregory.szorc@gmail.com> parents: 37287 diff changeset	647 self._active = True
9bfcbe4f4745 wireproto: add streams to frame-based protocol Gregory Szorc <gregory.szorc@gmail.com> parents: 37287 diff changeset	648
9bfcbe4f4745 wireproto: add streams to frame-based protocol Gregory Szorc <gregory.szorc@gmail.com> parents: 37287 diff changeset	649 return makeframe(requestid, self.streamid, streamflags, typeid, flags,
9bfcbe4f4745 wireproto: add streams to frame-based protocol Gregory Szorc <gregory.szorc@gmail.com> parents: 37287 diff changeset	650 payload)
9bfcbe4f4745 wireproto: add streams to frame-based protocol Gregory Szorc <gregory.szorc@gmail.com> parents: 37287 diff changeset	651
9bfcbe4f4745 wireproto: add streams to frame-based protocol Gregory Szorc <gregory.szorc@gmail.com> parents: 37287 diff changeset	652 def ensureserverstream(stream):
9bfcbe4f4745 wireproto: add streams to frame-based protocol Gregory Szorc <gregory.szorc@gmail.com> parents: 37287 diff changeset	653 if stream.streamid % 2:
9bfcbe4f4745 wireproto: add streams to frame-based protocol Gregory Szorc <gregory.szorc@gmail.com> parents: 37287 diff changeset	654 raise error.ProgrammingError('server should only write to even '
9bfcbe4f4745 wireproto: add streams to frame-based protocol Gregory Szorc <gregory.szorc@gmail.com> parents: 37287 diff changeset	655 'numbered streams; %d is not even' %
9bfcbe4f4745 wireproto: add streams to frame-based protocol Gregory Szorc <gregory.szorc@gmail.com> parents: 37287 diff changeset	656 stream.streamid)
37063 0a6c5cc09a88 wireproto: define human output side channel frame Gregory Szorc <gregory.szorc@gmail.com> parents: 37061 diff changeset	657
37055 8c3c47362934 wireproto: implement basic frame reading and processing Gregory Szorc <gregory.szorc@gmail.com> parents: 37054 diff changeset	658 class serverreactor(object):
8c3c47362934 wireproto: implement basic frame reading and processing Gregory Szorc <gregory.szorc@gmail.com> parents: 37054 diff changeset	659 """Holds state of a server handling frame-based protocol requests.
8c3c47362934 wireproto: implement basic frame reading and processing Gregory Szorc <gregory.szorc@gmail.com> parents: 37054 diff changeset	660
8c3c47362934 wireproto: implement basic frame reading and processing Gregory Szorc <gregory.szorc@gmail.com> parents: 37054 diff changeset	661 This class is the "brain" of the unified frame-based protocol server
8c3c47362934 wireproto: implement basic frame reading and processing Gregory Szorc <gregory.szorc@gmail.com> parents: 37054 diff changeset	662 component. While the protocol is stateless from the perspective of
8c3c47362934 wireproto: implement basic frame reading and processing Gregory Szorc <gregory.szorc@gmail.com> parents: 37054 diff changeset	663 requests/commands, something needs to track which frames have been
8c3c47362934 wireproto: implement basic frame reading and processing Gregory Szorc <gregory.szorc@gmail.com> parents: 37054 diff changeset	664 received, what frames to expect, etc. This class is that thing.
8c3c47362934 wireproto: implement basic frame reading and processing Gregory Szorc <gregory.szorc@gmail.com> parents: 37054 diff changeset	665
8c3c47362934 wireproto: implement basic frame reading and processing Gregory Szorc <gregory.szorc@gmail.com> parents: 37054 diff changeset	666 Instances are modeled as a state machine of sorts. Instances are also
8c3c47362934 wireproto: implement basic frame reading and processing Gregory Szorc <gregory.szorc@gmail.com> parents: 37054 diff changeset	667 reactionary to external events. The point of this class is to encapsulate
8c3c47362934 wireproto: implement basic frame reading and processing Gregory Szorc <gregory.szorc@gmail.com> parents: 37054 diff changeset	668 the state of the connection and the exchange of frames, not to perform
8c3c47362934 wireproto: implement basic frame reading and processing Gregory Szorc <gregory.szorc@gmail.com> parents: 37054 diff changeset	669 work. Instead, callers tell this class when something occurs, like a
8c3c47362934 wireproto: implement basic frame reading and processing Gregory Szorc <gregory.szorc@gmail.com> parents: 37054 diff changeset	670 frame arriving. If that activity is worthy of a follow-up action (say
8c3c47362934 wireproto: implement basic frame reading and processing Gregory Szorc <gregory.szorc@gmail.com> parents: 37054 diff changeset	671 run a command), the return value of that handler will say so.
8c3c47362934 wireproto: implement basic frame reading and processing Gregory Szorc <gregory.szorc@gmail.com> parents: 37054 diff changeset	672
8c3c47362934 wireproto: implement basic frame reading and processing Gregory Szorc <gregory.szorc@gmail.com> parents: 37054 diff changeset	673 I/O and CPU intensive operations are purposefully delegated outside of
8c3c47362934 wireproto: implement basic frame reading and processing Gregory Szorc <gregory.szorc@gmail.com> parents: 37054 diff changeset	674 this class.
8c3c47362934 wireproto: implement basic frame reading and processing Gregory Szorc <gregory.szorc@gmail.com> parents: 37054 diff changeset	675
8c3c47362934 wireproto: implement basic frame reading and processing Gregory Szorc <gregory.szorc@gmail.com> parents: 37054 diff changeset	676 Consumers are expected to tell instances when events occur. They do so by
8c3c47362934 wireproto: implement basic frame reading and processing Gregory Szorc <gregory.szorc@gmail.com> parents: 37054 diff changeset	677 calling the various ``on*`` methods. These methods return a 2-tuple
8c3c47362934 wireproto: implement basic frame reading and processing Gregory Szorc <gregory.szorc@gmail.com> parents: 37054 diff changeset	678 describing any follow-up action(s) to take. The first element is the
8c3c47362934 wireproto: implement basic frame reading and processing Gregory Szorc <gregory.szorc@gmail.com> parents: 37054 diff changeset	679 name of an action to perform. The second is a data structure (usually
8c3c47362934 wireproto: implement basic frame reading and processing Gregory Szorc <gregory.szorc@gmail.com> parents: 37054 diff changeset	680 a dict) specific to that action that contains more information. e.g.
8c3c47362934 wireproto: implement basic frame reading and processing Gregory Szorc <gregory.szorc@gmail.com> parents: 37054 diff changeset	681 if the server wants to send frames back to the client, the data structure
8c3c47362934 wireproto: implement basic frame reading and processing Gregory Szorc <gregory.szorc@gmail.com> parents: 37054 diff changeset	682 will contain a reference to those frames.
8c3c47362934 wireproto: implement basic frame reading and processing Gregory Szorc <gregory.szorc@gmail.com> parents: 37054 diff changeset	683
8c3c47362934 wireproto: implement basic frame reading and processing Gregory Szorc <gregory.szorc@gmail.com> parents: 37054 diff changeset	684 Valid actions that consumers can be instructed to take are:
8c3c47362934 wireproto: implement basic frame reading and processing Gregory Szorc <gregory.szorc@gmail.com> parents: 37054 diff changeset	685
37058 61393f888dfe wireproto: define and implement responses in framing protocol Gregory Szorc <gregory.szorc@gmail.com> parents: 37055 diff changeset	686 sendframes
61393f888dfe wireproto: define and implement responses in framing protocol Gregory Szorc <gregory.szorc@gmail.com> parents: 37055 diff changeset	687 Indicates that frames should be sent to the client. The ``framegen``
61393f888dfe wireproto: define and implement responses in framing protocol Gregory Szorc <gregory.szorc@gmail.com> parents: 37055 diff changeset	688 key contains a generator of frames that should be sent. The server
61393f888dfe wireproto: define and implement responses in framing protocol Gregory Szorc <gregory.szorc@gmail.com> parents: 37055 diff changeset	689 assumes that all frames are sent to the client.
61393f888dfe wireproto: define and implement responses in framing protocol Gregory Szorc <gregory.szorc@gmail.com> parents: 37055 diff changeset	690
37055 8c3c47362934 wireproto: implement basic frame reading and processing Gregory Szorc <gregory.szorc@gmail.com> parents: 37054 diff changeset	691 error
8c3c47362934 wireproto: implement basic frame reading and processing Gregory Szorc <gregory.szorc@gmail.com> parents: 37054 diff changeset	692 Indicates that an error occurred. Consumer should probably abort.
8c3c47362934 wireproto: implement basic frame reading and processing Gregory Szorc <gregory.szorc@gmail.com> parents: 37054 diff changeset	693
8c3c47362934 wireproto: implement basic frame reading and processing Gregory Szorc <gregory.szorc@gmail.com> parents: 37054 diff changeset	694 runcommand
8c3c47362934 wireproto: implement basic frame reading and processing Gregory Szorc <gregory.szorc@gmail.com> parents: 37054 diff changeset	695 Indicates that the consumer should run a wire protocol command. Details
8c3c47362934 wireproto: implement basic frame reading and processing Gregory Szorc <gregory.szorc@gmail.com> parents: 37054 diff changeset	696 of the command to run are given in the data structure.
8c3c47362934 wireproto: implement basic frame reading and processing Gregory Szorc <gregory.szorc@gmail.com> parents: 37054 diff changeset	697
8c3c47362934 wireproto: implement basic frame reading and processing Gregory Szorc <gregory.szorc@gmail.com> parents: 37054 diff changeset	698 wantframe
8c3c47362934 wireproto: implement basic frame reading and processing Gregory Szorc <gregory.szorc@gmail.com> parents: 37054 diff changeset	699 Indicates that nothing of interest happened and the server is waiting on
8c3c47362934 wireproto: implement basic frame reading and processing Gregory Szorc <gregory.szorc@gmail.com> parents: 37054 diff changeset	700 more frames from the client before anything interesting can be done.
37059 861e9d37e56e wireproto: buffer output frames when in half duplex mode Gregory Szorc <gregory.szorc@gmail.com> parents: 37058 diff changeset	701
861e9d37e56e wireproto: buffer output frames when in half duplex mode Gregory Szorc <gregory.szorc@gmail.com> parents: 37058 diff changeset	702 noop
861e9d37e56e wireproto: buffer output frames when in half duplex mode Gregory Szorc <gregory.szorc@gmail.com> parents: 37058 diff changeset	703 Indicates no additional action is required.
37061 c5e9c3b47366 wireproto: support for receiving multiple requests Gregory Szorc <gregory.szorc@gmail.com> parents: 37060 diff changeset	704
c5e9c3b47366 wireproto: support for receiving multiple requests Gregory Szorc <gregory.szorc@gmail.com> parents: 37060 diff changeset	705 Known Issues
c5e9c3b47366 wireproto: support for receiving multiple requests Gregory Szorc <gregory.szorc@gmail.com> parents: 37060 diff changeset	706 ------------
c5e9c3b47366 wireproto: support for receiving multiple requests Gregory Szorc <gregory.szorc@gmail.com> parents: 37060 diff changeset	707
c5e9c3b47366 wireproto: support for receiving multiple requests Gregory Szorc <gregory.szorc@gmail.com> parents: 37060 diff changeset	708 There are no limits to the number of partially received commands or their
c5e9c3b47366 wireproto: support for receiving multiple requests Gregory Szorc <gregory.szorc@gmail.com> parents: 37060 diff changeset	709 size. A malicious client could stream command request data and exhaust the
c5e9c3b47366 wireproto: support for receiving multiple requests Gregory Szorc <gregory.szorc@gmail.com> parents: 37060 diff changeset	710 server's memory.
c5e9c3b47366 wireproto: support for receiving multiple requests Gregory Szorc <gregory.szorc@gmail.com> parents: 37060 diff changeset	711
c5e9c3b47366 wireproto: support for receiving multiple requests Gregory Szorc <gregory.szorc@gmail.com> parents: 37060 diff changeset	712 Partially received commands are not acted upon when end of input is
c5e9c3b47366 wireproto: support for receiving multiple requests Gregory Szorc <gregory.szorc@gmail.com> parents: 37060 diff changeset	713 reached. Should the server error if it receives a partial request?
c5e9c3b47366 wireproto: support for receiving multiple requests Gregory Szorc <gregory.szorc@gmail.com> parents: 37060 diff changeset	714 Should the client send a message to abort a partially transmitted request
c5e9c3b47366 wireproto: support for receiving multiple requests Gregory Szorc <gregory.szorc@gmail.com> parents: 37060 diff changeset	715 to facilitate graceful shutdown?
c5e9c3b47366 wireproto: support for receiving multiple requests Gregory Szorc <gregory.szorc@gmail.com> parents: 37060 diff changeset	716
c5e9c3b47366 wireproto: support for receiving multiple requests Gregory Szorc <gregory.szorc@gmail.com> parents: 37060 diff changeset	717 Active requests that haven't been responded to aren't tracked. This means
c5e9c3b47366 wireproto: support for receiving multiple requests Gregory Szorc <gregory.szorc@gmail.com> parents: 37060 diff changeset	718 that if we receive a command and instruct its dispatch, another command
c5e9c3b47366 wireproto: support for receiving multiple requests Gregory Szorc <gregory.szorc@gmail.com> parents: 37060 diff changeset	719 with its request ID can come in over the wire and there will be a race
c5e9c3b47366 wireproto: support for receiving multiple requests Gregory Szorc <gregory.szorc@gmail.com> parents: 37060 diff changeset	720 between who responds to what.
37055 8c3c47362934 wireproto: implement basic frame reading and processing Gregory Szorc <gregory.szorc@gmail.com> parents: 37054 diff changeset	721 """
8c3c47362934 wireproto: implement basic frame reading and processing Gregory Szorc <gregory.szorc@gmail.com> parents: 37054 diff changeset	722
37059 861e9d37e56e wireproto: buffer output frames when in half duplex mode Gregory Szorc <gregory.szorc@gmail.com> parents: 37058 diff changeset	723 def __init__(self, deferoutput=False):
861e9d37e56e wireproto: buffer output frames when in half duplex mode Gregory Szorc <gregory.szorc@gmail.com> parents: 37058 diff changeset	724 """Construct a new server reactor.
861e9d37e56e wireproto: buffer output frames when in half duplex mode Gregory Szorc <gregory.szorc@gmail.com> parents: 37058 diff changeset	725
861e9d37e56e wireproto: buffer output frames when in half duplex mode Gregory Szorc <gregory.szorc@gmail.com> parents: 37058 diff changeset	726 ``deferoutput`` can be used to indicate that no output frames should be
861e9d37e56e wireproto: buffer output frames when in half duplex mode Gregory Szorc <gregory.szorc@gmail.com> parents: 37058 diff changeset	727 instructed to be sent until input has been exhausted. In this mode,
861e9d37e56e wireproto: buffer output frames when in half duplex mode Gregory Szorc <gregory.szorc@gmail.com> parents: 37058 diff changeset	728 events that would normally generate output frames (such as a command
861e9d37e56e wireproto: buffer output frames when in half duplex mode Gregory Szorc <gregory.szorc@gmail.com> parents: 37058 diff changeset	729 response being ready) will instead defer instructing the consumer to
861e9d37e56e wireproto: buffer output frames when in half duplex mode Gregory Szorc <gregory.szorc@gmail.com> parents: 37058 diff changeset	730 send those frames. This is useful for half-duplex transports where the
861e9d37e56e wireproto: buffer output frames when in half duplex mode Gregory Szorc <gregory.szorc@gmail.com> parents: 37058 diff changeset	731 sender cannot receive until all data has been transmitted.
861e9d37e56e wireproto: buffer output frames when in half duplex mode Gregory Szorc <gregory.szorc@gmail.com> parents: 37058 diff changeset	732 """
861e9d37e56e wireproto: buffer output frames when in half duplex mode Gregory Szorc <gregory.szorc@gmail.com> parents: 37058 diff changeset	733 self._deferoutput = deferoutput
37055 8c3c47362934 wireproto: implement basic frame reading and processing Gregory Szorc <gregory.szorc@gmail.com> parents: 37054 diff changeset	734 self._state = 'idle'
37289 5fadc63ac99f wireproto: explicit API to create outgoing streams Gregory Szorc <gregory.szorc@gmail.com> parents: 37288 diff changeset	735 self._nextoutgoingstreamid = 2
37059 861e9d37e56e wireproto: buffer output frames when in half duplex mode Gregory Szorc <gregory.szorc@gmail.com> parents: 37058 diff changeset	736 self._bufferedframegens = []
37288 9bfcbe4f4745 wireproto: add streams to frame-based protocol Gregory Szorc <gregory.szorc@gmail.com> parents: 37287 diff changeset	737 # stream id -> stream instance for all active streams from the client.
9bfcbe4f4745 wireproto: add streams to frame-based protocol Gregory Szorc <gregory.szorc@gmail.com> parents: 37287 diff changeset	738 self._incomingstreams = {}
37289 5fadc63ac99f wireproto: explicit API to create outgoing streams Gregory Szorc <gregory.szorc@gmail.com> parents: 37288 diff changeset	739 self._outgoingstreams = {}
37061 c5e9c3b47366 wireproto: support for receiving multiple requests Gregory Szorc <gregory.szorc@gmail.com> parents: 37060 diff changeset	740 # request id -> dict of commands that are actively being received.
c5e9c3b47366 wireproto: support for receiving multiple requests Gregory Szorc <gregory.szorc@gmail.com> parents: 37060 diff changeset	741 self._receivingcommands = {}
37066 39304dd63589 wireproto: explicitly track which requests are active Gregory Szorc <gregory.szorc@gmail.com> parents: 37065 diff changeset	742 # Request IDs that have been received and are actively being processed.
39304dd63589 wireproto: explicitly track which requests are active Gregory Szorc <gregory.szorc@gmail.com> parents: 37065 diff changeset	743 # Once all output for a request has been sent, it is removed from this
39304dd63589 wireproto: explicitly track which requests are active Gregory Szorc <gregory.szorc@gmail.com> parents: 37065 diff changeset	744 # set.
39304dd63589 wireproto: explicitly track which requests are active Gregory Szorc <gregory.szorc@gmail.com> parents: 37065 diff changeset	745 self._activecommands = set()
37055 8c3c47362934 wireproto: implement basic frame reading and processing Gregory Szorc <gregory.szorc@gmail.com> parents: 37054 diff changeset	746
37064 884a0c1604ad wireproto: define attr-based classes for representing frames Gregory Szorc <gregory.szorc@gmail.com> parents: 37063 diff changeset	747 def onframerecv(self, frame):
37055 8c3c47362934 wireproto: implement basic frame reading and processing Gregory Szorc <gregory.szorc@gmail.com> parents: 37054 diff changeset	748 """Process a frame that has been received off the wire.
8c3c47362934 wireproto: implement basic frame reading and processing Gregory Szorc <gregory.szorc@gmail.com> parents: 37054 diff changeset	749
8c3c47362934 wireproto: implement basic frame reading and processing Gregory Szorc <gregory.szorc@gmail.com> parents: 37054 diff changeset	750 Returns a dict with an ``action`` key that details what action,
8c3c47362934 wireproto: implement basic frame reading and processing Gregory Szorc <gregory.szorc@gmail.com> parents: 37054 diff changeset	751 if any, the consumer should take next.
8c3c47362934 wireproto: implement basic frame reading and processing Gregory Szorc <gregory.szorc@gmail.com> parents: 37054 diff changeset	752 """
37288 9bfcbe4f4745 wireproto: add streams to frame-based protocol Gregory Szorc <gregory.szorc@gmail.com> parents: 37287 diff changeset	753 if not frame.streamid % 2:
9bfcbe4f4745 wireproto: add streams to frame-based protocol Gregory Szorc <gregory.szorc@gmail.com> parents: 37287 diff changeset	754 self._state = 'errored'
9bfcbe4f4745 wireproto: add streams to frame-based protocol Gregory Szorc <gregory.szorc@gmail.com> parents: 37287 diff changeset	755 return self._makeerrorresult(
9bfcbe4f4745 wireproto: add streams to frame-based protocol Gregory Szorc <gregory.szorc@gmail.com> parents: 37287 diff changeset	756 _('received frame with even numbered stream ID: %d') %
9bfcbe4f4745 wireproto: add streams to frame-based protocol Gregory Szorc <gregory.szorc@gmail.com> parents: 37287 diff changeset	757 frame.streamid)
9bfcbe4f4745 wireproto: add streams to frame-based protocol Gregory Szorc <gregory.szorc@gmail.com> parents: 37287 diff changeset	758
9bfcbe4f4745 wireproto: add streams to frame-based protocol Gregory Szorc <gregory.szorc@gmail.com> parents: 37287 diff changeset	759 if frame.streamid not in self._incomingstreams:
9bfcbe4f4745 wireproto: add streams to frame-based protocol Gregory Szorc <gregory.szorc@gmail.com> parents: 37287 diff changeset	760 if not frame.streamflags & STREAM_FLAG_BEGIN_STREAM:
9bfcbe4f4745 wireproto: add streams to frame-based protocol Gregory Szorc <gregory.szorc@gmail.com> parents: 37287 diff changeset	761 self._state = 'errored'
9bfcbe4f4745 wireproto: add streams to frame-based protocol Gregory Szorc <gregory.szorc@gmail.com> parents: 37287 diff changeset	762 return self._makeerrorresult(
9bfcbe4f4745 wireproto: add streams to frame-based protocol Gregory Szorc <gregory.szorc@gmail.com> parents: 37287 diff changeset	763 _('received frame on unknown inactive stream without '
9bfcbe4f4745 wireproto: add streams to frame-based protocol Gregory Szorc <gregory.szorc@gmail.com> parents: 37287 diff changeset	764 'beginning of stream flag set'))
9bfcbe4f4745 wireproto: add streams to frame-based protocol Gregory Szorc <gregory.szorc@gmail.com> parents: 37287 diff changeset	765
9bfcbe4f4745 wireproto: add streams to frame-based protocol Gregory Szorc <gregory.szorc@gmail.com> parents: 37287 diff changeset	766 self._incomingstreams[frame.streamid] = stream(frame.streamid)
9bfcbe4f4745 wireproto: add streams to frame-based protocol Gregory Szorc <gregory.szorc@gmail.com> parents: 37287 diff changeset	767
9bfcbe4f4745 wireproto: add streams to frame-based protocol Gregory Szorc <gregory.szorc@gmail.com> parents: 37287 diff changeset	768 if frame.streamflags & STREAM_FLAG_ENCODING_APPLIED:
9bfcbe4f4745 wireproto: add streams to frame-based protocol Gregory Szorc <gregory.szorc@gmail.com> parents: 37287 diff changeset	769 # TODO handle decoding frames
9bfcbe4f4745 wireproto: add streams to frame-based protocol Gregory Szorc <gregory.szorc@gmail.com> parents: 37287 diff changeset	770 self._state = 'errored'
9bfcbe4f4745 wireproto: add streams to frame-based protocol Gregory Szorc <gregory.szorc@gmail.com> parents: 37287 diff changeset	771 raise error.ProgrammingError('support for decoding stream payloads '
9bfcbe4f4745 wireproto: add streams to frame-based protocol Gregory Szorc <gregory.szorc@gmail.com> parents: 37287 diff changeset	772 'not yet implemented')
9bfcbe4f4745 wireproto: add streams to frame-based protocol Gregory Szorc <gregory.szorc@gmail.com> parents: 37287 diff changeset	773
9bfcbe4f4745 wireproto: add streams to frame-based protocol Gregory Szorc <gregory.szorc@gmail.com> parents: 37287 diff changeset	774 if frame.streamflags & STREAM_FLAG_END_STREAM:
9bfcbe4f4745 wireproto: add streams to frame-based protocol Gregory Szorc <gregory.szorc@gmail.com> parents: 37287 diff changeset	775 del self._incomingstreams[frame.streamid]
9bfcbe4f4745 wireproto: add streams to frame-based protocol Gregory Szorc <gregory.szorc@gmail.com> parents: 37287 diff changeset	776
37055 8c3c47362934 wireproto: implement basic frame reading and processing Gregory Szorc <gregory.szorc@gmail.com> parents: 37054 diff changeset	777 handlers = {
8c3c47362934 wireproto: implement basic frame reading and processing Gregory Szorc <gregory.szorc@gmail.com> parents: 37054 diff changeset	778 'idle': self._onframeidle,
37061 c5e9c3b47366 wireproto: support for receiving multiple requests Gregory Szorc <gregory.szorc@gmail.com> parents: 37060 diff changeset	779 'command-receiving': self._onframecommandreceiving,
37055 8c3c47362934 wireproto: implement basic frame reading and processing Gregory Szorc <gregory.szorc@gmail.com> parents: 37054 diff changeset	780 'errored': self._onframeerrored,
8c3c47362934 wireproto: implement basic frame reading and processing Gregory Szorc <gregory.szorc@gmail.com> parents: 37054 diff changeset	781 }
8c3c47362934 wireproto: implement basic frame reading and processing Gregory Szorc <gregory.szorc@gmail.com> parents: 37054 diff changeset	782
8c3c47362934 wireproto: implement basic frame reading and processing Gregory Szorc <gregory.szorc@gmail.com> parents: 37054 diff changeset	783 meth = handlers.get(self._state)
8c3c47362934 wireproto: implement basic frame reading and processing Gregory Szorc <gregory.szorc@gmail.com> parents: 37054 diff changeset	784 if not meth:
8c3c47362934 wireproto: implement basic frame reading and processing Gregory Szorc <gregory.szorc@gmail.com> parents: 37054 diff changeset	785 raise error.ProgrammingError('unhandled state: %s' % self._state)
8c3c47362934 wireproto: implement basic frame reading and processing Gregory Szorc <gregory.szorc@gmail.com> parents: 37054 diff changeset	786
37064 884a0c1604ad wireproto: define attr-based classes for representing frames Gregory Szorc <gregory.szorc@gmail.com> parents: 37063 diff changeset	787 return meth(frame)
37055 8c3c47362934 wireproto: implement basic frame reading and processing Gregory Szorc <gregory.szorc@gmail.com> parents: 37054 diff changeset	788
37724 deff7cf7eefd wireprotov2: change frame type and name for command response Gregory Szorc <gregory.szorc@gmail.com> parents: 37723 diff changeset	789 def oncommandresponseready(self, stream, requestid, data):
37058 61393f888dfe wireproto: define and implement responses in framing protocol Gregory Szorc <gregory.szorc@gmail.com> parents: 37055 diff changeset	790 """Signal that a bytes response is ready to be sent to the client.
61393f888dfe wireproto: define and implement responses in framing protocol Gregory Szorc <gregory.szorc@gmail.com> parents: 37055 diff changeset	791
61393f888dfe wireproto: define and implement responses in framing protocol Gregory Szorc <gregory.szorc@gmail.com> parents: 37055 diff changeset	792 The raw bytes response is passed as an argument.
61393f888dfe wireproto: define and implement responses in framing protocol Gregory Szorc <gregory.szorc@gmail.com> parents: 37055 diff changeset	793 """
37288 9bfcbe4f4745 wireproto: add streams to frame-based protocol Gregory Szorc <gregory.szorc@gmail.com> parents: 37287 diff changeset	794 ensureserverstream(stream)
9bfcbe4f4745 wireproto: add streams to frame-based protocol Gregory Szorc <gregory.szorc@gmail.com> parents: 37287 diff changeset	795
37066 39304dd63589 wireproto: explicitly track which requests are active Gregory Szorc <gregory.szorc@gmail.com> parents: 37065 diff changeset	796 def sendframes():
37724 deff7cf7eefd wireprotov2: change frame type and name for command response Gregory Szorc <gregory.szorc@gmail.com> parents: 37723 diff changeset	797 for frame in createcommandresponseframesfrombytes(stream, requestid,
deff7cf7eefd wireprotov2: change frame type and name for command response Gregory Szorc <gregory.szorc@gmail.com> parents: 37723 diff changeset	798 data):
37066 39304dd63589 wireproto: explicitly track which requests are active Gregory Szorc <gregory.szorc@gmail.com> parents: 37065 diff changeset	799 yield frame
39304dd63589 wireproto: explicitly track which requests are active Gregory Szorc <gregory.szorc@gmail.com> parents: 37065 diff changeset	800
39304dd63589 wireproto: explicitly track which requests are active Gregory Szorc <gregory.szorc@gmail.com> parents: 37065 diff changeset	801 self._activecommands.remove(requestid)
39304dd63589 wireproto: explicitly track which requests are active Gregory Szorc <gregory.szorc@gmail.com> parents: 37065 diff changeset	802
39304dd63589 wireproto: explicitly track which requests are active Gregory Szorc <gregory.szorc@gmail.com> parents: 37065 diff changeset	803 result = sendframes()
37059 861e9d37e56e wireproto: buffer output frames when in half duplex mode Gregory Szorc <gregory.szorc@gmail.com> parents: 37058 diff changeset	804
861e9d37e56e wireproto: buffer output frames when in half duplex mode Gregory Szorc <gregory.szorc@gmail.com> parents: 37058 diff changeset	805 if self._deferoutput:
37066 39304dd63589 wireproto: explicitly track which requests are active Gregory Szorc <gregory.szorc@gmail.com> parents: 37065 diff changeset	806 self._bufferedframegens.append(result)
37059 861e9d37e56e wireproto: buffer output frames when in half duplex mode Gregory Szorc <gregory.szorc@gmail.com> parents: 37058 diff changeset	807 return 'noop', {}
861e9d37e56e wireproto: buffer output frames when in half duplex mode Gregory Szorc <gregory.szorc@gmail.com> parents: 37058 diff changeset	808 else:
861e9d37e56e wireproto: buffer output frames when in half duplex mode Gregory Szorc <gregory.szorc@gmail.com> parents: 37058 diff changeset	809 return 'sendframes', {
37066 39304dd63589 wireproto: explicitly track which requests are active Gregory Szorc <gregory.szorc@gmail.com> parents: 37065 diff changeset	810 'framegen': result,
37059 861e9d37e56e wireproto: buffer output frames when in half duplex mode Gregory Szorc <gregory.szorc@gmail.com> parents: 37058 diff changeset	811 }
861e9d37e56e wireproto: buffer output frames when in half duplex mode Gregory Szorc <gregory.szorc@gmail.com> parents: 37058 diff changeset	812
39575 07b58266bce3 wireprotov2: implement commands as a generator of objects Gregory Szorc <gregory.szorc@gmail.com> parents: 39503 diff changeset	813 def oncommandresponsereadyobjects(self, stream, requestid, objs):
07b58266bce3 wireprotov2: implement commands as a generator of objects Gregory Szorc <gregory.szorc@gmail.com> parents: 39503 diff changeset	814 """Signal that objects are ready to be sent to the client.
07b58266bce3 wireprotov2: implement commands as a generator of objects Gregory Szorc <gregory.szorc@gmail.com> parents: 39503 diff changeset	815
07b58266bce3 wireprotov2: implement commands as a generator of objects Gregory Szorc <gregory.szorc@gmail.com> parents: 39503 diff changeset	816 ``objs`` is an iterable of objects (typically a generator) that will
07b58266bce3 wireprotov2: implement commands as a generator of objects Gregory Szorc <gregory.szorc@gmail.com> parents: 39503 diff changeset	817 be encoded via CBOR and added to frames, which will be sent to the
07b58266bce3 wireprotov2: implement commands as a generator of objects Gregory Szorc <gregory.szorc@gmail.com> parents: 39503 diff changeset	818 client.
07b58266bce3 wireprotov2: implement commands as a generator of objects Gregory Szorc <gregory.szorc@gmail.com> parents: 39503 diff changeset	819 """
37728 564a3eec6e63 wireprotov2: add support for more response types Gregory Szorc <gregory.szorc@gmail.com> parents: 37726 diff changeset	820 ensureserverstream(stream)
564a3eec6e63 wireprotov2: add support for more response types Gregory Szorc <gregory.szorc@gmail.com> parents: 37726 diff changeset	821
39575 07b58266bce3 wireprotov2: implement commands as a generator of objects Gregory Szorc <gregory.szorc@gmail.com> parents: 39503 diff changeset	822 # We need to take care over exception handling. Uncaught exceptions
07b58266bce3 wireprotov2: implement commands as a generator of objects Gregory Szorc <gregory.szorc@gmail.com> parents: 39503 diff changeset	823 # when generating frames could lead to premature end of the frame
07b58266bce3 wireprotov2: implement commands as a generator of objects Gregory Szorc <gregory.szorc@gmail.com> parents: 39503 diff changeset	824 # stream and the possibility of the server or client process getting
07b58266bce3 wireprotov2: implement commands as a generator of objects Gregory Szorc <gregory.szorc@gmail.com> parents: 39503 diff changeset	825 # in a bad state.
07b58266bce3 wireprotov2: implement commands as a generator of objects Gregory Szorc <gregory.szorc@gmail.com> parents: 39503 diff changeset	826 #
07b58266bce3 wireprotov2: implement commands as a generator of objects Gregory Szorc <gregory.szorc@gmail.com> parents: 39503 diff changeset	827 # Keep in mind that if ``objs`` is a generator, advancing it could
07b58266bce3 wireprotov2: implement commands as a generator of objects Gregory Szorc <gregory.szorc@gmail.com> parents: 39503 diff changeset	828 # raise exceptions that originated in e.g. wire protocol command
07b58266bce3 wireprotov2: implement commands as a generator of objects Gregory Szorc <gregory.szorc@gmail.com> parents: 39503 diff changeset	829 # functions. That is why we differentiate between exceptions raised
07b58266bce3 wireprotov2: implement commands as a generator of objects Gregory Szorc <gregory.szorc@gmail.com> parents: 39503 diff changeset	830 # when iterating versus other exceptions that occur.
07b58266bce3 wireprotov2: implement commands as a generator of objects Gregory Szorc <gregory.szorc@gmail.com> parents: 39503 diff changeset	831 #
07b58266bce3 wireprotov2: implement commands as a generator of objects Gregory Szorc <gregory.szorc@gmail.com> parents: 39503 diff changeset	832 # In all cases, when the function finishes, the request is fully
07b58266bce3 wireprotov2: implement commands as a generator of objects Gregory Szorc <gregory.szorc@gmail.com> parents: 39503 diff changeset	833 # handled and no new frames for it should be seen.
07b58266bce3 wireprotov2: implement commands as a generator of objects Gregory Szorc <gregory.szorc@gmail.com> parents: 39503 diff changeset	834
37728 564a3eec6e63 wireprotov2: add support for more response types Gregory Szorc <gregory.szorc@gmail.com> parents: 37726 diff changeset	835 def sendframes():
39575 07b58266bce3 wireprotov2: implement commands as a generator of objects Gregory Szorc <gregory.szorc@gmail.com> parents: 39503 diff changeset	836 emitted = False
40026 b099e6032f38 wireprotov2: server support for sending content redirects Gregory Szorc <gregory.szorc@gmail.com> parents: 40025 diff changeset	837 alternatelocationsent = False
39576 84bf6ded9317 wireprotoframing: buffer emitted data to reduce frame count Gregory Szorc <gregory.szorc@gmail.com> parents: 39575 diff changeset	838 emitter = bufferingcommandresponseemitter(stream, requestid)
39575 07b58266bce3 wireprotov2: implement commands as a generator of objects Gregory Szorc <gregory.szorc@gmail.com> parents: 39503 diff changeset	839 while True:
07b58266bce3 wireprotov2: implement commands as a generator of objects Gregory Szorc <gregory.szorc@gmail.com> parents: 39503 diff changeset	840 try:
07b58266bce3 wireprotov2: implement commands as a generator of objects Gregory Szorc <gregory.szorc@gmail.com> parents: 39503 diff changeset	841 o = next(objs)
07b58266bce3 wireprotov2: implement commands as a generator of objects Gregory Szorc <gregory.szorc@gmail.com> parents: 39503 diff changeset	842 except StopIteration:
39576 84bf6ded9317 wireprotoframing: buffer emitted data to reduce frame count Gregory Szorc <gregory.szorc@gmail.com> parents: 39575 diff changeset	843 for frame in emitter.send(None):
84bf6ded9317 wireprotoframing: buffer emitted data to reduce frame count Gregory Szorc <gregory.szorc@gmail.com> parents: 39575 diff changeset	844 yield frame
84bf6ded9317 wireprotoframing: buffer emitted data to reduce frame count Gregory Szorc <gregory.szorc@gmail.com> parents: 39575 diff changeset	845
39575 07b58266bce3 wireprotov2: implement commands as a generator of objects Gregory Szorc <gregory.szorc@gmail.com> parents: 39503 diff changeset	846 if emitted:
07b58266bce3 wireprotov2: implement commands as a generator of objects Gregory Szorc <gregory.szorc@gmail.com> parents: 39503 diff changeset	847 yield createcommandresponseeosframe(stream, requestid)
07b58266bce3 wireprotov2: implement commands as a generator of objects Gregory Szorc <gregory.szorc@gmail.com> parents: 39503 diff changeset	848 break
07b58266bce3 wireprotov2: implement commands as a generator of objects Gregory Szorc <gregory.szorc@gmail.com> parents: 39503 diff changeset	849
07b58266bce3 wireprotov2: implement commands as a generator of objects Gregory Szorc <gregory.szorc@gmail.com> parents: 39503 diff changeset	850 except error.WireprotoCommandError as e:
07b58266bce3 wireprotov2: implement commands as a generator of objects Gregory Szorc <gregory.szorc@gmail.com> parents: 39503 diff changeset	851 for frame in createcommanderrorresponse(
07b58266bce3 wireprotov2: implement commands as a generator of objects Gregory Szorc <gregory.szorc@gmail.com> parents: 39503 diff changeset	852 stream, requestid, e.message, e.messageargs):
07b58266bce3 wireprotov2: implement commands as a generator of objects Gregory Szorc <gregory.szorc@gmail.com> parents: 39503 diff changeset	853 yield frame
07b58266bce3 wireprotov2: implement commands as a generator of objects Gregory Szorc <gregory.szorc@gmail.com> parents: 39503 diff changeset	854 break
07b58266bce3 wireprotov2: implement commands as a generator of objects Gregory Szorc <gregory.szorc@gmail.com> parents: 39503 diff changeset	855
07b58266bce3 wireprotov2: implement commands as a generator of objects Gregory Szorc <gregory.szorc@gmail.com> parents: 39503 diff changeset	856 except Exception as e:
39843 bce1c1af7518 py3: cast exception to bytes Gregory Szorc <gregory.szorc@gmail.com> parents: 39576 diff changeset	857 for frame in createerrorframe(
bce1c1af7518 py3: cast exception to bytes Gregory Szorc <gregory.szorc@gmail.com> parents: 39576 diff changeset	858 stream, requestid, '%s' % stringutil.forcebytestr(e),
bce1c1af7518 py3: cast exception to bytes Gregory Szorc <gregory.szorc@gmail.com> parents: 39576 diff changeset	859 errtype='server'):
bce1c1af7518 py3: cast exception to bytes Gregory Szorc <gregory.szorc@gmail.com> parents: 39576 diff changeset	860
39575 07b58266bce3 wireprotov2: implement commands as a generator of objects Gregory Szorc <gregory.szorc@gmail.com> parents: 39503 diff changeset	861 yield frame
07b58266bce3 wireprotov2: implement commands as a generator of objects Gregory Szorc <gregory.szorc@gmail.com> parents: 39503 diff changeset	862
07b58266bce3 wireprotov2: implement commands as a generator of objects Gregory Szorc <gregory.szorc@gmail.com> parents: 39503 diff changeset	863 break
07b58266bce3 wireprotov2: implement commands as a generator of objects Gregory Szorc <gregory.szorc@gmail.com> parents: 39503 diff changeset	864
07b58266bce3 wireprotov2: implement commands as a generator of objects Gregory Szorc <gregory.szorc@gmail.com> parents: 39503 diff changeset	865 try:
40026 b099e6032f38 wireprotov2: server support for sending content redirects Gregory Szorc <gregory.szorc@gmail.com> parents: 40025 diff changeset	866 # Alternate location responses can only be the first and
b099e6032f38 wireprotov2: server support for sending content redirects Gregory Szorc <gregory.szorc@gmail.com> parents: 40025 diff changeset	867 # only object in the output stream.
b099e6032f38 wireprotov2: server support for sending content redirects Gregory Szorc <gregory.szorc@gmail.com> parents: 40025 diff changeset	868 if isinstance(o, wireprototypes.alternatelocationresponse):
b099e6032f38 wireprotov2: server support for sending content redirects Gregory Szorc <gregory.szorc@gmail.com> parents: 40025 diff changeset	869 if emitted:
b099e6032f38 wireprotov2: server support for sending content redirects Gregory Szorc <gregory.szorc@gmail.com> parents: 40025 diff changeset	870 raise error.ProgrammingError(
b099e6032f38 wireprotov2: server support for sending content redirects Gregory Szorc <gregory.szorc@gmail.com> parents: 40025 diff changeset	871 'alternatelocationresponse seen after initial '
b099e6032f38 wireprotov2: server support for sending content redirects Gregory Szorc <gregory.szorc@gmail.com> parents: 40025 diff changeset	872 'output object')
b099e6032f38 wireprotov2: server support for sending content redirects Gregory Szorc <gregory.szorc@gmail.com> parents: 40025 diff changeset	873
b099e6032f38 wireprotov2: server support for sending content redirects Gregory Szorc <gregory.szorc@gmail.com> parents: 40025 diff changeset	874 yield createalternatelocationresponseframe(
b099e6032f38 wireprotov2: server support for sending content redirects Gregory Szorc <gregory.szorc@gmail.com> parents: 40025 diff changeset	875 stream, requestid, o)
b099e6032f38 wireprotov2: server support for sending content redirects Gregory Szorc <gregory.szorc@gmail.com> parents: 40025 diff changeset	876
b099e6032f38 wireprotov2: server support for sending content redirects Gregory Szorc <gregory.szorc@gmail.com> parents: 40025 diff changeset	877 alternatelocationsent = True
b099e6032f38 wireprotov2: server support for sending content redirects Gregory Szorc <gregory.szorc@gmail.com> parents: 40025 diff changeset	878 emitted = True
b099e6032f38 wireprotov2: server support for sending content redirects Gregory Szorc <gregory.szorc@gmail.com> parents: 40025 diff changeset	879 continue
b099e6032f38 wireprotov2: server support for sending content redirects Gregory Szorc <gregory.szorc@gmail.com> parents: 40025 diff changeset	880
b099e6032f38 wireprotov2: server support for sending content redirects Gregory Szorc <gregory.szorc@gmail.com> parents: 40025 diff changeset	881 if alternatelocationsent:
b099e6032f38 wireprotov2: server support for sending content redirects Gregory Szorc <gregory.szorc@gmail.com> parents: 40025 diff changeset	882 raise error.ProgrammingError(
b099e6032f38 wireprotov2: server support for sending content redirects Gregory Szorc <gregory.szorc@gmail.com> parents: 40025 diff changeset	883 'object follows alternatelocationresponse')
b099e6032f38 wireprotov2: server support for sending content redirects Gregory Szorc <gregory.szorc@gmail.com> parents: 40025 diff changeset	884
39575 07b58266bce3 wireprotov2: implement commands as a generator of objects Gregory Szorc <gregory.szorc@gmail.com> parents: 39503 diff changeset	885 if not emitted:
07b58266bce3 wireprotov2: implement commands as a generator of objects Gregory Szorc <gregory.szorc@gmail.com> parents: 39503 diff changeset	886 yield createcommandresponseokframe(stream, requestid)
07b58266bce3 wireprotov2: implement commands as a generator of objects Gregory Szorc <gregory.szorc@gmail.com> parents: 39503 diff changeset	887 emitted = True
07b58266bce3 wireprotov2: implement commands as a generator of objects Gregory Szorc <gregory.szorc@gmail.com> parents: 39503 diff changeset	888
40021 ed919b90acda wireprotov2: define type to represent pre-encoded object Gregory Szorc <gregory.szorc@gmail.com> parents: 39843 diff changeset	889 # Objects emitted by command functions can be serializable
ed919b90acda wireprotov2: define type to represent pre-encoded object Gregory Szorc <gregory.szorc@gmail.com> parents: 39843 diff changeset	890 # data structures or special types.
ed919b90acda wireprotov2: define type to represent pre-encoded object Gregory Szorc <gregory.szorc@gmail.com> parents: 39843 diff changeset	891 # TODO consider extracting the content normalization to a
ed919b90acda wireprotov2: define type to represent pre-encoded object Gregory Szorc <gregory.szorc@gmail.com> parents: 39843 diff changeset	892 # standalone function, as it may be useful for e.g. cachers.
ed919b90acda wireprotov2: define type to represent pre-encoded object Gregory Szorc <gregory.szorc@gmail.com> parents: 39843 diff changeset	893
ed919b90acda wireprotov2: define type to represent pre-encoded object Gregory Szorc <gregory.szorc@gmail.com> parents: 39843 diff changeset	894 # A pre-encoded object is sent directly to the emitter.
ed919b90acda wireprotov2: define type to represent pre-encoded object Gregory Szorc <gregory.szorc@gmail.com> parents: 39843 diff changeset	895 if isinstance(o, wireprototypes.encodedresponse):
ed919b90acda wireprotov2: define type to represent pre-encoded object Gregory Szorc <gregory.szorc@gmail.com> parents: 39843 diff changeset	896 for frame in emitter.send(o.data):
39576 84bf6ded9317 wireprotoframing: buffer emitted data to reduce frame count Gregory Szorc <gregory.szorc@gmail.com> parents: 39575 diff changeset	897 yield frame
84bf6ded9317 wireprotoframing: buffer emitted data to reduce frame count Gregory Szorc <gregory.szorc@gmail.com> parents: 39575 diff changeset	898
40021 ed919b90acda wireprotov2: define type to represent pre-encoded object Gregory Szorc <gregory.szorc@gmail.com> parents: 39843 diff changeset	899 # A regular object is CBOR encoded.
ed919b90acda wireprotov2: define type to represent pre-encoded object Gregory Szorc <gregory.szorc@gmail.com> parents: 39843 diff changeset	900 else:
ed919b90acda wireprotov2: define type to represent pre-encoded object Gregory Szorc <gregory.szorc@gmail.com> parents: 39843 diff changeset	901 for chunk in cborutil.streamencode(o):
ed919b90acda wireprotov2: define type to represent pre-encoded object Gregory Szorc <gregory.szorc@gmail.com> parents: 39843 diff changeset	902 for frame in emitter.send(chunk):
ed919b90acda wireprotov2: define type to represent pre-encoded object Gregory Szorc <gregory.szorc@gmail.com> parents: 39843 diff changeset	903 yield frame
ed919b90acda wireprotov2: define type to represent pre-encoded object Gregory Szorc <gregory.szorc@gmail.com> parents: 39843 diff changeset	904
39575 07b58266bce3 wireprotov2: implement commands as a generator of objects Gregory Szorc <gregory.szorc@gmail.com> parents: 39503 diff changeset	905 except Exception as e:
07b58266bce3 wireprotov2: implement commands as a generator of objects Gregory Szorc <gregory.szorc@gmail.com> parents: 39503 diff changeset	906 for frame in createerrorframe(stream, requestid,
07b58266bce3 wireprotov2: implement commands as a generator of objects Gregory Szorc <gregory.szorc@gmail.com> parents: 39503 diff changeset	907 '%s' % e,
07b58266bce3 wireprotov2: implement commands as a generator of objects Gregory Szorc <gregory.szorc@gmail.com> parents: 39503 diff changeset	908 errtype='server'):
07b58266bce3 wireprotov2: implement commands as a generator of objects Gregory Szorc <gregory.szorc@gmail.com> parents: 39503 diff changeset	909 yield frame
07b58266bce3 wireprotov2: implement commands as a generator of objects Gregory Szorc <gregory.szorc@gmail.com> parents: 39503 diff changeset	910
07b58266bce3 wireprotov2: implement commands as a generator of objects Gregory Szorc <gregory.szorc@gmail.com> parents: 39503 diff changeset	911 break
37728 564a3eec6e63 wireprotov2: add support for more response types Gregory Szorc <gregory.szorc@gmail.com> parents: 37726 diff changeset	912
564a3eec6e63 wireprotov2: add support for more response types Gregory Szorc <gregory.szorc@gmail.com> parents: 37726 diff changeset	913 self._activecommands.remove(requestid)
564a3eec6e63 wireprotov2: add support for more response types Gregory Szorc <gregory.szorc@gmail.com> parents: 37726 diff changeset	914
564a3eec6e63 wireprotov2: add support for more response types Gregory Szorc <gregory.szorc@gmail.com> parents: 37726 diff changeset	915 return self._handlesendframes(sendframes())
564a3eec6e63 wireprotov2: add support for more response types Gregory Szorc <gregory.szorc@gmail.com> parents: 37726 diff changeset	916
37059 861e9d37e56e wireproto: buffer output frames when in half duplex mode Gregory Szorc <gregory.szorc@gmail.com> parents: 37058 diff changeset	917 def oninputeof(self):
861e9d37e56e wireproto: buffer output frames when in half duplex mode Gregory Szorc <gregory.szorc@gmail.com> parents: 37058 diff changeset	918 """Signals that end of input has been received.
861e9d37e56e wireproto: buffer output frames when in half duplex mode Gregory Szorc <gregory.szorc@gmail.com> parents: 37058 diff changeset	919
861e9d37e56e wireproto: buffer output frames when in half duplex mode Gregory Szorc <gregory.szorc@gmail.com> parents: 37058 diff changeset	920 No more frames will be received. All pending activity should be
861e9d37e56e wireproto: buffer output frames when in half duplex mode Gregory Szorc <gregory.szorc@gmail.com> parents: 37058 diff changeset	921 completed.
861e9d37e56e wireproto: buffer output frames when in half duplex mode Gregory Szorc <gregory.szorc@gmail.com> parents: 37058 diff changeset	922 """
37061 c5e9c3b47366 wireproto: support for receiving multiple requests Gregory Szorc <gregory.szorc@gmail.com> parents: 37060 diff changeset	923 # TODO should we do anything about in-flight commands?
c5e9c3b47366 wireproto: support for receiving multiple requests Gregory Szorc <gregory.szorc@gmail.com> parents: 37060 diff changeset	924
37059 861e9d37e56e wireproto: buffer output frames when in half duplex mode Gregory Szorc <gregory.szorc@gmail.com> parents: 37058 diff changeset	925 if not self._deferoutput or not self._bufferedframegens:
861e9d37e56e wireproto: buffer output frames when in half duplex mode Gregory Szorc <gregory.szorc@gmail.com> parents: 37058 diff changeset	926 return 'noop', {}
861e9d37e56e wireproto: buffer output frames when in half duplex mode Gregory Szorc <gregory.szorc@gmail.com> parents: 37058 diff changeset	927
861e9d37e56e wireproto: buffer output frames when in half duplex mode Gregory Szorc <gregory.szorc@gmail.com> parents: 37058 diff changeset	928 # If we buffered all our responses, emit those.
861e9d37e56e wireproto: buffer output frames when in half duplex mode Gregory Szorc <gregory.szorc@gmail.com> parents: 37058 diff changeset	929 def makegen():
861e9d37e56e wireproto: buffer output frames when in half duplex mode Gregory Szorc <gregory.szorc@gmail.com> parents: 37058 diff changeset	930 for gen in self._bufferedframegens:
861e9d37e56e wireproto: buffer output frames when in half duplex mode Gregory Szorc <gregory.szorc@gmail.com> parents: 37058 diff changeset	931 for frame in gen:
861e9d37e56e wireproto: buffer output frames when in half duplex mode Gregory Szorc <gregory.szorc@gmail.com> parents: 37058 diff changeset	932 yield frame
861e9d37e56e wireproto: buffer output frames when in half duplex mode Gregory Szorc <gregory.szorc@gmail.com> parents: 37058 diff changeset	933
37058 61393f888dfe wireproto: define and implement responses in framing protocol Gregory Szorc <gregory.szorc@gmail.com> parents: 37055 diff changeset	934 return 'sendframes', {
37059 861e9d37e56e wireproto: buffer output frames when in half duplex mode Gregory Szorc <gregory.szorc@gmail.com> parents: 37058 diff changeset	935 'framegen': makegen(),
37058 61393f888dfe wireproto: define and implement responses in framing protocol Gregory Szorc <gregory.szorc@gmail.com> parents: 37055 diff changeset	936 }
61393f888dfe wireproto: define and implement responses in framing protocol Gregory Szorc <gregory.szorc@gmail.com> parents: 37055 diff changeset	937
37728 564a3eec6e63 wireprotov2: add support for more response types Gregory Szorc <gregory.szorc@gmail.com> parents: 37726 diff changeset	938 def _handlesendframes(self, framegen):
564a3eec6e63 wireprotov2: add support for more response types Gregory Szorc <gregory.szorc@gmail.com> parents: 37726 diff changeset	939 if self._deferoutput:
564a3eec6e63 wireprotov2: add support for more response types Gregory Szorc <gregory.szorc@gmail.com> parents: 37726 diff changeset	940 self._bufferedframegens.append(framegen)
564a3eec6e63 wireprotov2: add support for more response types Gregory Szorc <gregory.szorc@gmail.com> parents: 37726 diff changeset	941 return 'noop', {}
564a3eec6e63 wireprotov2: add support for more response types Gregory Szorc <gregory.szorc@gmail.com> parents: 37726 diff changeset	942 else:
564a3eec6e63 wireprotov2: add support for more response types Gregory Szorc <gregory.szorc@gmail.com> parents: 37726 diff changeset	943 return 'sendframes', {
564a3eec6e63 wireprotov2: add support for more response types Gregory Szorc <gregory.szorc@gmail.com> parents: 37726 diff changeset	944 'framegen': framegen,
564a3eec6e63 wireprotov2: add support for more response types Gregory Szorc <gregory.szorc@gmail.com> parents: 37726 diff changeset	945 }
564a3eec6e63 wireprotov2: add support for more response types Gregory Szorc <gregory.szorc@gmail.com> parents: 37726 diff changeset	946
37726 0c184ca594bb wireprotov2: change behavior of error frame Gregory Szorc <gregory.szorc@gmail.com> parents: 37725 diff changeset	947 def onservererror(self, stream, requestid, msg):
37288 9bfcbe4f4745 wireproto: add streams to frame-based protocol Gregory Szorc <gregory.szorc@gmail.com> parents: 37287 diff changeset	948 ensureserverstream(stream)
9bfcbe4f4745 wireproto: add streams to frame-based protocol Gregory Szorc <gregory.szorc@gmail.com> parents: 37287 diff changeset	949
37728 564a3eec6e63 wireprotov2: add support for more response types Gregory Szorc <gregory.szorc@gmail.com> parents: 37726 diff changeset	950 def sendframes():
564a3eec6e63 wireprotov2: add support for more response types Gregory Szorc <gregory.szorc@gmail.com> parents: 37726 diff changeset	951 for frame in createerrorframe(stream, requestid, msg,
564a3eec6e63 wireprotov2: add support for more response types Gregory Szorc <gregory.szorc@gmail.com> parents: 37726 diff changeset	952 errtype='server'):
564a3eec6e63 wireprotov2: add support for more response types Gregory Szorc <gregory.szorc@gmail.com> parents: 37726 diff changeset	953 yield frame
564a3eec6e63 wireprotov2: add support for more response types Gregory Szorc <gregory.szorc@gmail.com> parents: 37726 diff changeset	954
564a3eec6e63 wireprotov2: add support for more response types Gregory Szorc <gregory.szorc@gmail.com> parents: 37726 diff changeset	955 self._activecommands.remove(requestid)
564a3eec6e63 wireprotov2: add support for more response types Gregory Szorc <gregory.szorc@gmail.com> parents: 37726 diff changeset	956
564a3eec6e63 wireprotov2: add support for more response types Gregory Szorc <gregory.szorc@gmail.com> parents: 37726 diff changeset	957 return self._handlesendframes(sendframes())
564a3eec6e63 wireprotov2: add support for more response types Gregory Szorc <gregory.szorc@gmail.com> parents: 37726 diff changeset	958
564a3eec6e63 wireprotov2: add support for more response types Gregory Szorc <gregory.szorc@gmail.com> parents: 37726 diff changeset	959 def oncommanderror(self, stream, requestid, message, args=None):
564a3eec6e63 wireprotov2: add support for more response types Gregory Szorc <gregory.szorc@gmail.com> parents: 37726 diff changeset	960 """Called when a command encountered an error before sending output."""
564a3eec6e63 wireprotov2: add support for more response types Gregory Szorc <gregory.szorc@gmail.com> parents: 37726 diff changeset	961 ensureserverstream(stream)
564a3eec6e63 wireprotov2: add support for more response types Gregory Szorc <gregory.szorc@gmail.com> parents: 37726 diff changeset	962
564a3eec6e63 wireprotov2: add support for more response types Gregory Szorc <gregory.szorc@gmail.com> parents: 37726 diff changeset	963 def sendframes():
564a3eec6e63 wireprotov2: add support for more response types Gregory Szorc <gregory.szorc@gmail.com> parents: 37726 diff changeset	964 for frame in createcommanderrorresponse(stream, requestid, message,
564a3eec6e63 wireprotov2: add support for more response types Gregory Szorc <gregory.szorc@gmail.com> parents: 37726 diff changeset	965 args):
564a3eec6e63 wireprotov2: add support for more response types Gregory Szorc <gregory.szorc@gmail.com> parents: 37726 diff changeset	966 yield frame
564a3eec6e63 wireprotov2: add support for more response types Gregory Szorc <gregory.szorc@gmail.com> parents: 37726 diff changeset	967
564a3eec6e63 wireprotov2: add support for more response types Gregory Szorc <gregory.szorc@gmail.com> parents: 37726 diff changeset	968 self._activecommands.remove(requestid)
564a3eec6e63 wireprotov2: add support for more response types Gregory Szorc <gregory.szorc@gmail.com> parents: 37726 diff changeset	969
564a3eec6e63 wireprotov2: add support for more response types Gregory Szorc <gregory.szorc@gmail.com> parents: 37726 diff changeset	970 return self._handlesendframes(sendframes())
37058 61393f888dfe wireproto: define and implement responses in framing protocol Gregory Szorc <gregory.szorc@gmail.com> parents: 37055 diff changeset	971
37289 5fadc63ac99f wireproto: explicit API to create outgoing streams Gregory Szorc <gregory.szorc@gmail.com> parents: 37288 diff changeset	972 def makeoutputstream(self):
5fadc63ac99f wireproto: explicit API to create outgoing streams Gregory Szorc <gregory.szorc@gmail.com> parents: 37288 diff changeset	973 """Create a stream to be used for sending data to the client."""
5fadc63ac99f wireproto: explicit API to create outgoing streams Gregory Szorc <gregory.szorc@gmail.com> parents: 37288 diff changeset	974 streamid = self._nextoutgoingstreamid
5fadc63ac99f wireproto: explicit API to create outgoing streams Gregory Szorc <gregory.szorc@gmail.com> parents: 37288 diff changeset	975 self._nextoutgoingstreamid += 2
5fadc63ac99f wireproto: explicit API to create outgoing streams Gregory Szorc <gregory.szorc@gmail.com> parents: 37288 diff changeset	976
5fadc63ac99f wireproto: explicit API to create outgoing streams Gregory Szorc <gregory.szorc@gmail.com> parents: 37288 diff changeset	977 s = stream(streamid)
5fadc63ac99f wireproto: explicit API to create outgoing streams Gregory Szorc <gregory.szorc@gmail.com> parents: 37288 diff changeset	978 self._outgoingstreams[streamid] = s
5fadc63ac99f wireproto: explicit API to create outgoing streams Gregory Szorc <gregory.szorc@gmail.com> parents: 37288 diff changeset	979
5fadc63ac99f wireproto: explicit API to create outgoing streams Gregory Szorc <gregory.szorc@gmail.com> parents: 37288 diff changeset	980 return s
5fadc63ac99f wireproto: explicit API to create outgoing streams Gregory Szorc <gregory.szorc@gmail.com> parents: 37288 diff changeset	981
37055 8c3c47362934 wireproto: implement basic frame reading and processing Gregory Szorc <gregory.szorc@gmail.com> parents: 37054 diff changeset	982 def _makeerrorresult(self, msg):
8c3c47362934 wireproto: implement basic frame reading and processing Gregory Szorc <gregory.szorc@gmail.com> parents: 37054 diff changeset	983 return 'error', {
8c3c47362934 wireproto: implement basic frame reading and processing Gregory Szorc <gregory.szorc@gmail.com> parents: 37054 diff changeset	984 'message': msg,
8c3c47362934 wireproto: implement basic frame reading and processing Gregory Szorc <gregory.szorc@gmail.com> parents: 37054 diff changeset	985 }
8c3c47362934 wireproto: implement basic frame reading and processing Gregory Szorc <gregory.szorc@gmail.com> parents: 37054 diff changeset	986
37061 c5e9c3b47366 wireproto: support for receiving multiple requests Gregory Szorc <gregory.szorc@gmail.com> parents: 37060 diff changeset	987 def _makeruncommandresult(self, requestid):
c5e9c3b47366 wireproto: support for receiving multiple requests Gregory Szorc <gregory.szorc@gmail.com> parents: 37060 diff changeset	988 entry = self._receivingcommands[requestid]
37292 3d0e2cd86e05 wireproto: use CBOR for command requests Gregory Szorc <gregory.szorc@gmail.com> parents: 37291 diff changeset	989
3d0e2cd86e05 wireproto: use CBOR for command requests Gregory Szorc <gregory.szorc@gmail.com> parents: 37291 diff changeset	990 if not entry['requestdone']:
3d0e2cd86e05 wireproto: use CBOR for command requests Gregory Szorc <gregory.szorc@gmail.com> parents: 37291 diff changeset	991 self._state = 'errored'
3d0e2cd86e05 wireproto: use CBOR for command requests Gregory Szorc <gregory.szorc@gmail.com> parents: 37291 diff changeset	992 raise error.ProgrammingError('should not be called without '
3d0e2cd86e05 wireproto: use CBOR for command requests Gregory Szorc <gregory.szorc@gmail.com> parents: 37291 diff changeset	993 'requestdone set')
3d0e2cd86e05 wireproto: use CBOR for command requests Gregory Szorc <gregory.szorc@gmail.com> parents: 37291 diff changeset	994
37061 c5e9c3b47366 wireproto: support for receiving multiple requests Gregory Szorc <gregory.szorc@gmail.com> parents: 37060 diff changeset	995 del self._receivingcommands[requestid]
c5e9c3b47366 wireproto: support for receiving multiple requests Gregory Szorc <gregory.szorc@gmail.com> parents: 37060 diff changeset	996
c5e9c3b47366 wireproto: support for receiving multiple requests Gregory Szorc <gregory.szorc@gmail.com> parents: 37060 diff changeset	997 if self._receivingcommands:
c5e9c3b47366 wireproto: support for receiving multiple requests Gregory Szorc <gregory.szorc@gmail.com> parents: 37060 diff changeset	998 self._state = 'command-receiving'
c5e9c3b47366 wireproto: support for receiving multiple requests Gregory Szorc <gregory.szorc@gmail.com> parents: 37060 diff changeset	999 else:
c5e9c3b47366 wireproto: support for receiving multiple requests Gregory Szorc <gregory.szorc@gmail.com> parents: 37060 diff changeset	1000 self._state = 'idle'
c5e9c3b47366 wireproto: support for receiving multiple requests Gregory Szorc <gregory.szorc@gmail.com> parents: 37060 diff changeset	1001
37292 3d0e2cd86e05 wireproto: use CBOR for command requests Gregory Szorc <gregory.szorc@gmail.com> parents: 37291 diff changeset	1002 # Decode the payloads as CBOR.
3d0e2cd86e05 wireproto: use CBOR for command requests Gregory Szorc <gregory.szorc@gmail.com> parents: 37291 diff changeset	1003 entry['payload'].seek(0)
39465 36f487a332ad wireprotoframing: use our CBOR module Gregory Szorc <gregory.szorc@gmail.com> parents: 37728 diff changeset	1004 request = cborutil.decodeall(entry['payload'].getvalue())[0]
37292 3d0e2cd86e05 wireproto: use CBOR for command requests Gregory Szorc <gregory.szorc@gmail.com> parents: 37291 diff changeset	1005
3d0e2cd86e05 wireproto: use CBOR for command requests Gregory Szorc <gregory.szorc@gmail.com> parents: 37291 diff changeset	1006 if b'name' not in request:
3d0e2cd86e05 wireproto: use CBOR for command requests Gregory Szorc <gregory.szorc@gmail.com> parents: 37291 diff changeset	1007 self._state = 'errored'
3d0e2cd86e05 wireproto: use CBOR for command requests Gregory Szorc <gregory.szorc@gmail.com> parents: 37291 diff changeset	1008 return self._makeerrorresult(
3d0e2cd86e05 wireproto: use CBOR for command requests Gregory Szorc <gregory.szorc@gmail.com> parents: 37291 diff changeset	1009 _('command request missing "name" field'))
3d0e2cd86e05 wireproto: use CBOR for command requests Gregory Szorc <gregory.szorc@gmail.com> parents: 37291 diff changeset	1010
3d0e2cd86e05 wireproto: use CBOR for command requests Gregory Szorc <gregory.szorc@gmail.com> parents: 37291 diff changeset	1011 if b'args' not in request:
3d0e2cd86e05 wireproto: use CBOR for command requests Gregory Szorc <gregory.szorc@gmail.com> parents: 37291 diff changeset	1012 request[b'args'] = {}
3d0e2cd86e05 wireproto: use CBOR for command requests Gregory Szorc <gregory.szorc@gmail.com> parents: 37291 diff changeset	1013
37066 39304dd63589 wireproto: explicitly track which requests are active Gregory Szorc <gregory.szorc@gmail.com> parents: 37065 diff changeset	1014 assert requestid not in self._activecommands
39304dd63589 wireproto: explicitly track which requests are active Gregory Szorc <gregory.szorc@gmail.com> parents: 37065 diff changeset	1015 self._activecommands.add(requestid)
39304dd63589 wireproto: explicitly track which requests are active Gregory Szorc <gregory.szorc@gmail.com> parents: 37065 diff changeset	1016
37055 8c3c47362934 wireproto: implement basic frame reading and processing Gregory Szorc <gregory.szorc@gmail.com> parents: 37054 diff changeset	1017 return 'runcommand', {
37061 c5e9c3b47366 wireproto: support for receiving multiple requests Gregory Szorc <gregory.szorc@gmail.com> parents: 37060 diff changeset	1018 'requestid': requestid,
37292 3d0e2cd86e05 wireproto: use CBOR for command requests Gregory Szorc <gregory.szorc@gmail.com> parents: 37291 diff changeset	1019 'command': request[b'name'],
3d0e2cd86e05 wireproto: use CBOR for command requests Gregory Szorc <gregory.szorc@gmail.com> parents: 37291 diff changeset	1020 'args': request[b'args'],
40026 b099e6032f38 wireprotov2: server support for sending content redirects Gregory Szorc <gregory.szorc@gmail.com> parents: 40025 diff changeset	1021 'redirect': request.get(b'redirect'),
37061 c5e9c3b47366 wireproto: support for receiving multiple requests Gregory Szorc <gregory.szorc@gmail.com> parents: 37060 diff changeset	1022 'data': entry['data'].getvalue() if entry['data'] else None,
37055 8c3c47362934 wireproto: implement basic frame reading and processing Gregory Szorc <gregory.szorc@gmail.com> parents: 37054 diff changeset	1023 }
8c3c47362934 wireproto: implement basic frame reading and processing Gregory Szorc <gregory.szorc@gmail.com> parents: 37054 diff changeset	1024
8c3c47362934 wireproto: implement basic frame reading and processing Gregory Szorc <gregory.szorc@gmail.com> parents: 37054 diff changeset	1025 def _makewantframeresult(self):
8c3c47362934 wireproto: implement basic frame reading and processing Gregory Szorc <gregory.szorc@gmail.com> parents: 37054 diff changeset	1026 return 'wantframe', {
8c3c47362934 wireproto: implement basic frame reading and processing Gregory Szorc <gregory.szorc@gmail.com> parents: 37054 diff changeset	1027 'state': self._state,
8c3c47362934 wireproto: implement basic frame reading and processing Gregory Szorc <gregory.szorc@gmail.com> parents: 37054 diff changeset	1028 }
8c3c47362934 wireproto: implement basic frame reading and processing Gregory Szorc <gregory.szorc@gmail.com> parents: 37054 diff changeset	1029
37292 3d0e2cd86e05 wireproto: use CBOR for command requests Gregory Szorc <gregory.szorc@gmail.com> parents: 37291 diff changeset	1030 def _validatecommandrequestframe(self, frame):
3d0e2cd86e05 wireproto: use CBOR for command requests Gregory Szorc <gregory.szorc@gmail.com> parents: 37291 diff changeset	1031 new = frame.flags & FLAG_COMMAND_REQUEST_NEW
3d0e2cd86e05 wireproto: use CBOR for command requests Gregory Szorc <gregory.szorc@gmail.com> parents: 37291 diff changeset	1032 continuation = frame.flags & FLAG_COMMAND_REQUEST_CONTINUATION
3d0e2cd86e05 wireproto: use CBOR for command requests Gregory Szorc <gregory.szorc@gmail.com> parents: 37291 diff changeset	1033
3d0e2cd86e05 wireproto: use CBOR for command requests Gregory Szorc <gregory.szorc@gmail.com> parents: 37291 diff changeset	1034 if new and continuation:
3d0e2cd86e05 wireproto: use CBOR for command requests Gregory Szorc <gregory.szorc@gmail.com> parents: 37291 diff changeset	1035 self._state = 'errored'
3d0e2cd86e05 wireproto: use CBOR for command requests Gregory Szorc <gregory.szorc@gmail.com> parents: 37291 diff changeset	1036 return self._makeerrorresult(
3d0e2cd86e05 wireproto: use CBOR for command requests Gregory Szorc <gregory.szorc@gmail.com> parents: 37291 diff changeset	1037 _('received command request frame with both new and '
3d0e2cd86e05 wireproto: use CBOR for command requests Gregory Szorc <gregory.szorc@gmail.com> parents: 37291 diff changeset	1038 'continuation flags set'))
3d0e2cd86e05 wireproto: use CBOR for command requests Gregory Szorc <gregory.szorc@gmail.com> parents: 37291 diff changeset	1039
3d0e2cd86e05 wireproto: use CBOR for command requests Gregory Szorc <gregory.szorc@gmail.com> parents: 37291 diff changeset	1040 if not new and not continuation:
3d0e2cd86e05 wireproto: use CBOR for command requests Gregory Szorc <gregory.szorc@gmail.com> parents: 37291 diff changeset	1041 self._state = 'errored'
3d0e2cd86e05 wireproto: use CBOR for command requests Gregory Szorc <gregory.szorc@gmail.com> parents: 37291 diff changeset	1042 return self._makeerrorresult(
3d0e2cd86e05 wireproto: use CBOR for command requests Gregory Szorc <gregory.szorc@gmail.com> parents: 37291 diff changeset	1043 _('received command request frame with neither new nor '
3d0e2cd86e05 wireproto: use CBOR for command requests Gregory Szorc <gregory.szorc@gmail.com> parents: 37291 diff changeset	1044 'continuation flags set'))
3d0e2cd86e05 wireproto: use CBOR for command requests Gregory Szorc <gregory.szorc@gmail.com> parents: 37291 diff changeset	1045
37064 884a0c1604ad wireproto: define attr-based classes for representing frames Gregory Szorc <gregory.szorc@gmail.com> parents: 37063 diff changeset	1046 def _onframeidle(self, frame):
37055 8c3c47362934 wireproto: implement basic frame reading and processing Gregory Szorc <gregory.szorc@gmail.com> parents: 37054 diff changeset	1047 # The only frame type that should be received in this state is a
8c3c47362934 wireproto: implement basic frame reading and processing Gregory Szorc <gregory.szorc@gmail.com> parents: 37054 diff changeset	1048 # command request.
37292 3d0e2cd86e05 wireproto: use CBOR for command requests Gregory Szorc <gregory.szorc@gmail.com> parents: 37291 diff changeset	1049 if frame.typeid != FRAME_TYPE_COMMAND_REQUEST:
37055 8c3c47362934 wireproto: implement basic frame reading and processing Gregory Szorc <gregory.szorc@gmail.com> parents: 37054 diff changeset	1050 self._state = 'errored'
8c3c47362934 wireproto: implement basic frame reading and processing Gregory Szorc <gregory.szorc@gmail.com> parents: 37054 diff changeset	1051 return self._makeerrorresult(
37292 3d0e2cd86e05 wireproto: use CBOR for command requests Gregory Szorc <gregory.szorc@gmail.com> parents: 37291 diff changeset	1052 _('expected command request frame; got %d') % frame.typeid)
3d0e2cd86e05 wireproto: use CBOR for command requests Gregory Szorc <gregory.szorc@gmail.com> parents: 37291 diff changeset	1053
3d0e2cd86e05 wireproto: use CBOR for command requests Gregory Szorc <gregory.szorc@gmail.com> parents: 37291 diff changeset	1054 res = self._validatecommandrequestframe(frame)
3d0e2cd86e05 wireproto: use CBOR for command requests Gregory Szorc <gregory.szorc@gmail.com> parents: 37291 diff changeset	1055 if res:
3d0e2cd86e05 wireproto: use CBOR for command requests Gregory Szorc <gregory.szorc@gmail.com> parents: 37291 diff changeset	1056 return res
37055 8c3c47362934 wireproto: implement basic frame reading and processing Gregory Szorc <gregory.szorc@gmail.com> parents: 37054 diff changeset	1057
37064 884a0c1604ad wireproto: define attr-based classes for representing frames Gregory Szorc <gregory.szorc@gmail.com> parents: 37063 diff changeset	1058 if frame.requestid in self._receivingcommands:
37061 c5e9c3b47366 wireproto: support for receiving multiple requests Gregory Szorc <gregory.szorc@gmail.com> parents: 37060 diff changeset	1059 self._state = 'errored'
c5e9c3b47366 wireproto: support for receiving multiple requests Gregory Szorc <gregory.szorc@gmail.com> parents: 37060 diff changeset	1060 return self._makeerrorresult(
37064 884a0c1604ad wireproto: define attr-based classes for representing frames Gregory Szorc <gregory.szorc@gmail.com> parents: 37063 diff changeset	1061 _('request with ID %d already received') % frame.requestid)
37061 c5e9c3b47366 wireproto: support for receiving multiple requests Gregory Szorc <gregory.szorc@gmail.com> parents: 37060 diff changeset	1062
37066 39304dd63589 wireproto: explicitly track which requests are active Gregory Szorc <gregory.szorc@gmail.com> parents: 37065 diff changeset	1063 if frame.requestid in self._activecommands:
39304dd63589 wireproto: explicitly track which requests are active Gregory Szorc <gregory.szorc@gmail.com> parents: 37065 diff changeset	1064 self._state = 'errored'
37292 3d0e2cd86e05 wireproto: use CBOR for command requests Gregory Szorc <gregory.szorc@gmail.com> parents: 37291 diff changeset	1065 return self._makeerrorresult(
3d0e2cd86e05 wireproto: use CBOR for command requests Gregory Szorc <gregory.szorc@gmail.com> parents: 37291 diff changeset	1066 _('request with ID %d is already active') % frame.requestid)
3d0e2cd86e05 wireproto: use CBOR for command requests Gregory Szorc <gregory.szorc@gmail.com> parents: 37291 diff changeset	1067
3d0e2cd86e05 wireproto: use CBOR for command requests Gregory Szorc <gregory.szorc@gmail.com> parents: 37291 diff changeset	1068 new = frame.flags & FLAG_COMMAND_REQUEST_NEW
3d0e2cd86e05 wireproto: use CBOR for command requests Gregory Szorc <gregory.szorc@gmail.com> parents: 37291 diff changeset	1069 moreframes = frame.flags & FLAG_COMMAND_REQUEST_MORE_FRAMES
3d0e2cd86e05 wireproto: use CBOR for command requests Gregory Szorc <gregory.szorc@gmail.com> parents: 37291 diff changeset	1070 expectingdata = frame.flags & FLAG_COMMAND_REQUEST_EXPECT_DATA
37066 39304dd63589 wireproto: explicitly track which requests are active Gregory Szorc <gregory.szorc@gmail.com> parents: 37065 diff changeset	1071
37292 3d0e2cd86e05 wireproto: use CBOR for command requests Gregory Szorc <gregory.szorc@gmail.com> parents: 37291 diff changeset	1072 if not new:
3d0e2cd86e05 wireproto: use CBOR for command requests Gregory Szorc <gregory.szorc@gmail.com> parents: 37291 diff changeset	1073 self._state = 'errored'
3d0e2cd86e05 wireproto: use CBOR for command requests Gregory Szorc <gregory.szorc@gmail.com> parents: 37291 diff changeset	1074 return self._makeerrorresult(
3d0e2cd86e05 wireproto: use CBOR for command requests Gregory Szorc <gregory.szorc@gmail.com> parents: 37291 diff changeset	1075 _('received command request frame without new flag set'))
3d0e2cd86e05 wireproto: use CBOR for command requests Gregory Szorc <gregory.szorc@gmail.com> parents: 37291 diff changeset	1076
3d0e2cd86e05 wireproto: use CBOR for command requests Gregory Szorc <gregory.szorc@gmail.com> parents: 37291 diff changeset	1077 payload = util.bytesio()
3d0e2cd86e05 wireproto: use CBOR for command requests Gregory Szorc <gregory.szorc@gmail.com> parents: 37291 diff changeset	1078 payload.write(frame.payload)
37061 c5e9c3b47366 wireproto: support for receiving multiple requests Gregory Szorc <gregory.szorc@gmail.com> parents: 37060 diff changeset	1079
37064 884a0c1604ad wireproto: define attr-based classes for representing frames Gregory Szorc <gregory.szorc@gmail.com> parents: 37063 diff changeset	1080 self._receivingcommands[frame.requestid] = {
37292 3d0e2cd86e05 wireproto: use CBOR for command requests Gregory Szorc <gregory.szorc@gmail.com> parents: 37291 diff changeset	1081 'payload': payload,
37061 c5e9c3b47366 wireproto: support for receiving multiple requests Gregory Szorc <gregory.szorc@gmail.com> parents: 37060 diff changeset	1082 'data': None,
37292 3d0e2cd86e05 wireproto: use CBOR for command requests Gregory Szorc <gregory.szorc@gmail.com> parents: 37291 diff changeset	1083 'requestdone': not moreframes,
3d0e2cd86e05 wireproto: use CBOR for command requests Gregory Szorc <gregory.szorc@gmail.com> parents: 37291 diff changeset	1084 'expectingdata': bool(expectingdata),
37061 c5e9c3b47366 wireproto: support for receiving multiple requests Gregory Szorc <gregory.szorc@gmail.com> parents: 37060 diff changeset	1085 }
37055 8c3c47362934 wireproto: implement basic frame reading and processing Gregory Szorc <gregory.szorc@gmail.com> parents: 37054 diff changeset	1086
37292 3d0e2cd86e05 wireproto: use CBOR for command requests Gregory Szorc <gregory.szorc@gmail.com> parents: 37291 diff changeset	1087 # This is the final frame for this request. Dispatch it.
3d0e2cd86e05 wireproto: use CBOR for command requests Gregory Szorc <gregory.szorc@gmail.com> parents: 37291 diff changeset	1088 if not moreframes and not expectingdata:
37064 884a0c1604ad wireproto: define attr-based classes for representing frames Gregory Szorc <gregory.szorc@gmail.com> parents: 37063 diff changeset	1089 return self._makeruncommandresult(frame.requestid)
37055 8c3c47362934 wireproto: implement basic frame reading and processing Gregory Szorc <gregory.szorc@gmail.com> parents: 37054 diff changeset	1090
37292 3d0e2cd86e05 wireproto: use CBOR for command requests Gregory Szorc <gregory.szorc@gmail.com> parents: 37291 diff changeset	1091 assert moreframes or expectingdata
3d0e2cd86e05 wireproto: use CBOR for command requests Gregory Szorc <gregory.szorc@gmail.com> parents: 37291 diff changeset	1092 self._state = 'command-receiving'
3d0e2cd86e05 wireproto: use CBOR for command requests Gregory Szorc <gregory.szorc@gmail.com> parents: 37291 diff changeset	1093 return self._makewantframeresult()
37055 8c3c47362934 wireproto: implement basic frame reading and processing Gregory Szorc <gregory.szorc@gmail.com> parents: 37054 diff changeset	1094
37064 884a0c1604ad wireproto: define attr-based classes for representing frames Gregory Szorc <gregory.szorc@gmail.com> parents: 37063 diff changeset	1095 def _onframecommandreceiving(self, frame):
37292 3d0e2cd86e05 wireproto: use CBOR for command requests Gregory Szorc <gregory.szorc@gmail.com> parents: 37291 diff changeset	1096 if frame.typeid == FRAME_TYPE_COMMAND_REQUEST:
3d0e2cd86e05 wireproto: use CBOR for command requests Gregory Szorc <gregory.szorc@gmail.com> parents: 37291 diff changeset	1097 # Process new command requests as such.
3d0e2cd86e05 wireproto: use CBOR for command requests Gregory Szorc <gregory.szorc@gmail.com> parents: 37291 diff changeset	1098 if frame.flags & FLAG_COMMAND_REQUEST_NEW:
3d0e2cd86e05 wireproto: use CBOR for command requests Gregory Szorc <gregory.szorc@gmail.com> parents: 37291 diff changeset	1099 return self._onframeidle(frame)
3d0e2cd86e05 wireproto: use CBOR for command requests Gregory Szorc <gregory.szorc@gmail.com> parents: 37291 diff changeset	1100
3d0e2cd86e05 wireproto: use CBOR for command requests Gregory Szorc <gregory.szorc@gmail.com> parents: 37291 diff changeset	1101 res = self._validatecommandrequestframe(frame)
3d0e2cd86e05 wireproto: use CBOR for command requests Gregory Szorc <gregory.szorc@gmail.com> parents: 37291 diff changeset	1102 if res:
3d0e2cd86e05 wireproto: use CBOR for command requests Gregory Szorc <gregory.szorc@gmail.com> parents: 37291 diff changeset	1103 return res
37061 c5e9c3b47366 wireproto: support for receiving multiple requests Gregory Szorc <gregory.szorc@gmail.com> parents: 37060 diff changeset	1104
c5e9c3b47366 wireproto: support for receiving multiple requests Gregory Szorc <gregory.szorc@gmail.com> parents: 37060 diff changeset	1105 # All other frames should be related to a command that is currently
37066 39304dd63589 wireproto: explicitly track which requests are active Gregory Szorc <gregory.szorc@gmail.com> parents: 37065 diff changeset	1106 # receiving but is not active.
39304dd63589 wireproto: explicitly track which requests are active Gregory Szorc <gregory.szorc@gmail.com> parents: 37065 diff changeset	1107 if frame.requestid in self._activecommands:
39304dd63589 wireproto: explicitly track which requests are active Gregory Szorc <gregory.szorc@gmail.com> parents: 37065 diff changeset	1108 self._state = 'errored'
39304dd63589 wireproto: explicitly track which requests are active Gregory Szorc <gregory.szorc@gmail.com> parents: 37065 diff changeset	1109 return self._makeerrorresult(
39304dd63589 wireproto: explicitly track which requests are active Gregory Szorc <gregory.szorc@gmail.com> parents: 37065 diff changeset	1110 _('received frame for request that is still active: %d') %
39304dd63589 wireproto: explicitly track which requests are active Gregory Szorc <gregory.szorc@gmail.com> parents: 37065 diff changeset	1111 frame.requestid)
39304dd63589 wireproto: explicitly track which requests are active Gregory Szorc <gregory.szorc@gmail.com> parents: 37065 diff changeset	1112
37064 884a0c1604ad wireproto: define attr-based classes for representing frames Gregory Szorc <gregory.szorc@gmail.com> parents: 37063 diff changeset	1113 if frame.requestid not in self._receivingcommands:
37055 8c3c47362934 wireproto: implement basic frame reading and processing Gregory Szorc <gregory.szorc@gmail.com> parents: 37054 diff changeset	1114 self._state = 'errored'
37061 c5e9c3b47366 wireproto: support for receiving multiple requests Gregory Szorc <gregory.szorc@gmail.com> parents: 37060 diff changeset	1115 return self._makeerrorresult(
c5e9c3b47366 wireproto: support for receiving multiple requests Gregory Szorc <gregory.szorc@gmail.com> parents: 37060 diff changeset	1116 _('received frame for request that is not receiving: %d') %
37064 884a0c1604ad wireproto: define attr-based classes for representing frames Gregory Szorc <gregory.szorc@gmail.com> parents: 37063 diff changeset	1117 frame.requestid)
37061 c5e9c3b47366 wireproto: support for receiving multiple requests Gregory Szorc <gregory.szorc@gmail.com> parents: 37060 diff changeset	1118
37064 884a0c1604ad wireproto: define attr-based classes for representing frames Gregory Szorc <gregory.szorc@gmail.com> parents: 37063 diff changeset	1119 entry = self._receivingcommands[frame.requestid]
37061 c5e9c3b47366 wireproto: support for receiving multiple requests Gregory Szorc <gregory.szorc@gmail.com> parents: 37060 diff changeset	1120
37292 3d0e2cd86e05 wireproto: use CBOR for command requests Gregory Szorc <gregory.szorc@gmail.com> parents: 37291 diff changeset	1121 if frame.typeid == FRAME_TYPE_COMMAND_REQUEST:
3d0e2cd86e05 wireproto: use CBOR for command requests Gregory Szorc <gregory.szorc@gmail.com> parents: 37291 diff changeset	1122 moreframes = frame.flags & FLAG_COMMAND_REQUEST_MORE_FRAMES
3d0e2cd86e05 wireproto: use CBOR for command requests Gregory Szorc <gregory.szorc@gmail.com> parents: 37291 diff changeset	1123 expectingdata = bool(frame.flags & FLAG_COMMAND_REQUEST_EXPECT_DATA)
3d0e2cd86e05 wireproto: use CBOR for command requests Gregory Szorc <gregory.szorc@gmail.com> parents: 37291 diff changeset	1124
3d0e2cd86e05 wireproto: use CBOR for command requests Gregory Szorc <gregory.szorc@gmail.com> parents: 37291 diff changeset	1125 if entry['requestdone']:
3d0e2cd86e05 wireproto: use CBOR for command requests Gregory Szorc <gregory.szorc@gmail.com> parents: 37291 diff changeset	1126 self._state = 'errored'
3d0e2cd86e05 wireproto: use CBOR for command requests Gregory Szorc <gregory.szorc@gmail.com> parents: 37291 diff changeset	1127 return self._makeerrorresult(
3d0e2cd86e05 wireproto: use CBOR for command requests Gregory Szorc <gregory.szorc@gmail.com> parents: 37291 diff changeset	1128 _('received command request frame when request frames '
3d0e2cd86e05 wireproto: use CBOR for command requests Gregory Szorc <gregory.szorc@gmail.com> parents: 37291 diff changeset	1129 'were supposedly done'))
3d0e2cd86e05 wireproto: use CBOR for command requests Gregory Szorc <gregory.szorc@gmail.com> parents: 37291 diff changeset	1130
3d0e2cd86e05 wireproto: use CBOR for command requests Gregory Szorc <gregory.szorc@gmail.com> parents: 37291 diff changeset	1131 if expectingdata != entry['expectingdata']:
37061 c5e9c3b47366 wireproto: support for receiving multiple requests Gregory Szorc <gregory.szorc@gmail.com> parents: 37060 diff changeset	1132 self._state = 'errored'
37292 3d0e2cd86e05 wireproto: use CBOR for command requests Gregory Szorc <gregory.szorc@gmail.com> parents: 37291 diff changeset	1133 return self._makeerrorresult(
3d0e2cd86e05 wireproto: use CBOR for command requests Gregory Szorc <gregory.szorc@gmail.com> parents: 37291 diff changeset	1134 _('mismatch between expect data flag and previous frame'))
3d0e2cd86e05 wireproto: use CBOR for command requests Gregory Szorc <gregory.szorc@gmail.com> parents: 37291 diff changeset	1135
3d0e2cd86e05 wireproto: use CBOR for command requests Gregory Szorc <gregory.szorc@gmail.com> parents: 37291 diff changeset	1136 entry['payload'].write(frame.payload)
37061 c5e9c3b47366 wireproto: support for receiving multiple requests Gregory Szorc <gregory.szorc@gmail.com> parents: 37060 diff changeset	1137
37292 3d0e2cd86e05 wireproto: use CBOR for command requests Gregory Szorc <gregory.szorc@gmail.com> parents: 37291 diff changeset	1138 if not moreframes:
3d0e2cd86e05 wireproto: use CBOR for command requests Gregory Szorc <gregory.szorc@gmail.com> parents: 37291 diff changeset	1139 entry['requestdone'] = True
3d0e2cd86e05 wireproto: use CBOR for command requests Gregory Szorc <gregory.szorc@gmail.com> parents: 37291 diff changeset	1140
3d0e2cd86e05 wireproto: use CBOR for command requests Gregory Szorc <gregory.szorc@gmail.com> parents: 37291 diff changeset	1141 if not moreframes and not expectingdata:
3d0e2cd86e05 wireproto: use CBOR for command requests Gregory Szorc <gregory.szorc@gmail.com> parents: 37291 diff changeset	1142 return self._makeruncommandresult(frame.requestid)
3d0e2cd86e05 wireproto: use CBOR for command requests Gregory Szorc <gregory.szorc@gmail.com> parents: 37291 diff changeset	1143
3d0e2cd86e05 wireproto: use CBOR for command requests Gregory Szorc <gregory.szorc@gmail.com> parents: 37291 diff changeset	1144 return self._makewantframeresult()
37061 c5e9c3b47366 wireproto: support for receiving multiple requests Gregory Szorc <gregory.szorc@gmail.com> parents: 37060 diff changeset	1145
37064 884a0c1604ad wireproto: define attr-based classes for representing frames Gregory Szorc <gregory.szorc@gmail.com> parents: 37063 diff changeset	1146 elif frame.typeid == FRAME_TYPE_COMMAND_DATA:
37061 c5e9c3b47366 wireproto: support for receiving multiple requests Gregory Szorc <gregory.szorc@gmail.com> parents: 37060 diff changeset	1147 if not entry['expectingdata']:
c5e9c3b47366 wireproto: support for receiving multiple requests Gregory Szorc <gregory.szorc@gmail.com> parents: 37060 diff changeset	1148 self._state = 'errored'
c5e9c3b47366 wireproto: support for receiving multiple requests Gregory Szorc <gregory.szorc@gmail.com> parents: 37060 diff changeset	1149 return self._makeerrorresult(_(
c5e9c3b47366 wireproto: support for receiving multiple requests Gregory Szorc <gregory.szorc@gmail.com> parents: 37060 diff changeset	1150 'received command data frame for request that is not '
37064 884a0c1604ad wireproto: define attr-based classes for representing frames Gregory Szorc <gregory.szorc@gmail.com> parents: 37063 diff changeset	1151 'expecting data: %d') % frame.requestid)
37061 c5e9c3b47366 wireproto: support for receiving multiple requests Gregory Szorc <gregory.szorc@gmail.com> parents: 37060 diff changeset	1152
c5e9c3b47366 wireproto: support for receiving multiple requests Gregory Szorc <gregory.szorc@gmail.com> parents: 37060 diff changeset	1153 if entry['data'] is None:
c5e9c3b47366 wireproto: support for receiving multiple requests Gregory Szorc <gregory.szorc@gmail.com> parents: 37060 diff changeset	1154 entry['data'] = util.bytesio()
c5e9c3b47366 wireproto: support for receiving multiple requests Gregory Szorc <gregory.szorc@gmail.com> parents: 37060 diff changeset	1155
37064 884a0c1604ad wireproto: define attr-based classes for representing frames Gregory Szorc <gregory.szorc@gmail.com> parents: 37063 diff changeset	1156 return self._handlecommanddataframe(frame, entry)
37292 3d0e2cd86e05 wireproto: use CBOR for command requests Gregory Szorc <gregory.szorc@gmail.com> parents: 37291 diff changeset	1157 else:
37055 8c3c47362934 wireproto: implement basic frame reading and processing Gregory Szorc <gregory.szorc@gmail.com> parents: 37054 diff changeset	1158 self._state = 'errored'
37292 3d0e2cd86e05 wireproto: use CBOR for command requests Gregory Szorc <gregory.szorc@gmail.com> parents: 37291 diff changeset	1159 return self._makeerrorresult(_(
3d0e2cd86e05 wireproto: use CBOR for command requests Gregory Szorc <gregory.szorc@gmail.com> parents: 37291 diff changeset	1160 'received unexpected frame type: %d') % frame.typeid)
37055 8c3c47362934 wireproto: implement basic frame reading and processing Gregory Szorc <gregory.szorc@gmail.com> parents: 37054 diff changeset	1161
37064 884a0c1604ad wireproto: define attr-based classes for representing frames Gregory Szorc <gregory.szorc@gmail.com> parents: 37063 diff changeset	1162 def _handlecommanddataframe(self, frame, entry):
884a0c1604ad wireproto: define attr-based classes for representing frames Gregory Szorc <gregory.szorc@gmail.com> parents: 37063 diff changeset	1163 assert frame.typeid == FRAME_TYPE_COMMAND_DATA
37055 8c3c47362934 wireproto: implement basic frame reading and processing Gregory Szorc <gregory.szorc@gmail.com> parents: 37054 diff changeset	1164
8c3c47362934 wireproto: implement basic frame reading and processing Gregory Szorc <gregory.szorc@gmail.com> parents: 37054 diff changeset	1165 # TODO support streaming data instead of buffering it.
37064 884a0c1604ad wireproto: define attr-based classes for representing frames Gregory Szorc <gregory.szorc@gmail.com> parents: 37063 diff changeset	1166 entry['data'].write(frame.payload)
37055 8c3c47362934 wireproto: implement basic frame reading and processing Gregory Szorc <gregory.szorc@gmail.com> parents: 37054 diff changeset	1167
37064 884a0c1604ad wireproto: define attr-based classes for representing frames Gregory Szorc <gregory.szorc@gmail.com> parents: 37063 diff changeset	1168 if frame.flags & FLAG_COMMAND_DATA_CONTINUATION:
37055 8c3c47362934 wireproto: implement basic frame reading and processing Gregory Szorc <gregory.szorc@gmail.com> parents: 37054 diff changeset	1169 return self._makewantframeresult()
37064 884a0c1604ad wireproto: define attr-based classes for representing frames Gregory Szorc <gregory.szorc@gmail.com> parents: 37063 diff changeset	1170 elif frame.flags & FLAG_COMMAND_DATA_EOS:
37061 c5e9c3b47366 wireproto: support for receiving multiple requests Gregory Szorc <gregory.szorc@gmail.com> parents: 37060 diff changeset	1171 entry['data'].seek(0)
37064 884a0c1604ad wireproto: define attr-based classes for representing frames Gregory Szorc <gregory.szorc@gmail.com> parents: 37063 diff changeset	1172 return self._makeruncommandresult(frame.requestid)
37055 8c3c47362934 wireproto: implement basic frame reading and processing Gregory Szorc <gregory.szorc@gmail.com> parents: 37054 diff changeset	1173 else:
8c3c47362934 wireproto: implement basic frame reading and processing Gregory Szorc <gregory.szorc@gmail.com> parents: 37054 diff changeset	1174 self._state = 'errored'
8c3c47362934 wireproto: implement basic frame reading and processing Gregory Szorc <gregory.szorc@gmail.com> parents: 37054 diff changeset	1175 return self._makeerrorresult(_('command data frame without '
8c3c47362934 wireproto: implement basic frame reading and processing Gregory Szorc <gregory.szorc@gmail.com> parents: 37054 diff changeset	1176 'flags'))
8c3c47362934 wireproto: implement basic frame reading and processing Gregory Szorc <gregory.szorc@gmail.com> parents: 37054 diff changeset	1177
37064 884a0c1604ad wireproto: define attr-based classes for representing frames Gregory Szorc <gregory.szorc@gmail.com> parents: 37063 diff changeset	1178 def _onframeerrored(self, frame):
37055 8c3c47362934 wireproto: implement basic frame reading and processing Gregory Szorc <gregory.szorc@gmail.com> parents: 37054 diff changeset	1179 return self._makeerrorresult(_('server already errored'))
37543 01361be9e2dc wireproto: introduce a reactor for client-side state Gregory Szorc <gregory.szorc@gmail.com> parents: 37485 diff changeset	1180
01361be9e2dc wireproto: introduce a reactor for client-side state Gregory Szorc <gregory.szorc@gmail.com> parents: 37485 diff changeset	1181 class commandrequest(object):
01361be9e2dc wireproto: introduce a reactor for client-side state Gregory Szorc <gregory.szorc@gmail.com> parents: 37485 diff changeset	1182 """Represents a request to run a command."""
01361be9e2dc wireproto: introduce a reactor for client-side state Gregory Szorc <gregory.szorc@gmail.com> parents: 37485 diff changeset	1183
40025 86b22a4cfab1 wireprotov2: client support for advertising redirect targets Gregory Szorc <gregory.szorc@gmail.com> parents: 40021 diff changeset	1184 def __init__(self, requestid, name, args, datafh=None, redirect=None):
37543 01361be9e2dc wireproto: introduce a reactor for client-side state Gregory Szorc <gregory.szorc@gmail.com> parents: 37485 diff changeset	1185 self.requestid = requestid
01361be9e2dc wireproto: introduce a reactor for client-side state Gregory Szorc <gregory.szorc@gmail.com> parents: 37485 diff changeset	1186 self.name = name
01361be9e2dc wireproto: introduce a reactor for client-side state Gregory Szorc <gregory.szorc@gmail.com> parents: 37485 diff changeset	1187 self.args = args
01361be9e2dc wireproto: introduce a reactor for client-side state Gregory Szorc <gregory.szorc@gmail.com> parents: 37485 diff changeset	1188 self.datafh = datafh
40025 86b22a4cfab1 wireprotov2: client support for advertising redirect targets Gregory Szorc <gregory.szorc@gmail.com> parents: 40021 diff changeset	1189 self.redirect = redirect
37543 01361be9e2dc wireproto: introduce a reactor for client-side state Gregory Szorc <gregory.szorc@gmail.com> parents: 37485 diff changeset	1190 self.state = 'pending'
01361be9e2dc wireproto: introduce a reactor for client-side state Gregory Szorc <gregory.szorc@gmail.com> parents: 37485 diff changeset	1191
01361be9e2dc wireproto: introduce a reactor for client-side state Gregory Szorc <gregory.szorc@gmail.com> parents: 37485 diff changeset	1192 class clientreactor(object):
01361be9e2dc wireproto: introduce a reactor for client-side state Gregory Szorc <gregory.szorc@gmail.com> parents: 37485 diff changeset	1193 """Holds state of a client issuing frame-based protocol requests.
01361be9e2dc wireproto: introduce a reactor for client-side state Gregory Szorc <gregory.szorc@gmail.com> parents: 37485 diff changeset	1194
01361be9e2dc wireproto: introduce a reactor for client-side state Gregory Szorc <gregory.szorc@gmail.com> parents: 37485 diff changeset	1195 This is like ``serverreactor`` but for client-side state.
01361be9e2dc wireproto: introduce a reactor for client-side state Gregory Szorc <gregory.szorc@gmail.com> parents: 37485 diff changeset	1196
01361be9e2dc wireproto: introduce a reactor for client-side state Gregory Szorc <gregory.szorc@gmail.com> parents: 37485 diff changeset	1197 Each instance is bound to the lifetime of a connection. For persistent
01361be9e2dc wireproto: introduce a reactor for client-side state Gregory Szorc <gregory.szorc@gmail.com> parents: 37485 diff changeset	1198 connection transports using e.g. TCP sockets and speaking the raw
01361be9e2dc wireproto: introduce a reactor for client-side state Gregory Szorc <gregory.szorc@gmail.com> parents: 37485 diff changeset	1199 framing protocol, there will be a single instance for the lifetime of
01361be9e2dc wireproto: introduce a reactor for client-side state Gregory Szorc <gregory.szorc@gmail.com> parents: 37485 diff changeset	1200 the TCP socket. For transports where there are multiple discrete
01361be9e2dc wireproto: introduce a reactor for client-side state Gregory Szorc <gregory.szorc@gmail.com> parents: 37485 diff changeset	1201 interactions (say tunneled within in HTTP request), there will be a
01361be9e2dc wireproto: introduce a reactor for client-side state Gregory Szorc <gregory.szorc@gmail.com> parents: 37485 diff changeset	1202 separate instance for each distinct interaction.
01361be9e2dc wireproto: introduce a reactor for client-side state Gregory Szorc <gregory.szorc@gmail.com> parents: 37485 diff changeset	1203 """
01361be9e2dc wireproto: introduce a reactor for client-side state Gregory Szorc <gregory.szorc@gmail.com> parents: 37485 diff changeset	1204 def __init__(self, hasmultiplesend=False, buffersends=True):
01361be9e2dc wireproto: introduce a reactor for client-side state Gregory Szorc <gregory.szorc@gmail.com> parents: 37485 diff changeset	1205 """Create a new instance.
01361be9e2dc wireproto: introduce a reactor for client-side state Gregory Szorc <gregory.szorc@gmail.com> parents: 37485 diff changeset	1206
01361be9e2dc wireproto: introduce a reactor for client-side state Gregory Szorc <gregory.szorc@gmail.com> parents: 37485 diff changeset	1207 ``hasmultiplesend`` indicates whether multiple sends are supported
01361be9e2dc wireproto: introduce a reactor for client-side state Gregory Szorc <gregory.szorc@gmail.com> parents: 37485 diff changeset	1208 by the transport. When True, it is possible to send commands immediately
01361be9e2dc wireproto: introduce a reactor for client-side state Gregory Szorc <gregory.szorc@gmail.com> parents: 37485 diff changeset	1209 instead of buffering until the caller signals an intent to finish a
01361be9e2dc wireproto: introduce a reactor for client-side state Gregory Szorc <gregory.szorc@gmail.com> parents: 37485 diff changeset	1210 send operation.
01361be9e2dc wireproto: introduce a reactor for client-side state Gregory Szorc <gregory.szorc@gmail.com> parents: 37485 diff changeset	1211
01361be9e2dc wireproto: introduce a reactor for client-side state Gregory Szorc <gregory.szorc@gmail.com> parents: 37485 diff changeset	1212 ``buffercommands`` indicates whether sends should be buffered until the
01361be9e2dc wireproto: introduce a reactor for client-side state Gregory Szorc <gregory.szorc@gmail.com> parents: 37485 diff changeset	1213 last request has been issued.
01361be9e2dc wireproto: introduce a reactor for client-side state Gregory Szorc <gregory.szorc@gmail.com> parents: 37485 diff changeset	1214 """
01361be9e2dc wireproto: introduce a reactor for client-side state Gregory Szorc <gregory.szorc@gmail.com> parents: 37485 diff changeset	1215 self._hasmultiplesend = hasmultiplesend
01361be9e2dc wireproto: introduce a reactor for client-side state Gregory Szorc <gregory.szorc@gmail.com> parents: 37485 diff changeset	1216 self._buffersends = buffersends
01361be9e2dc wireproto: introduce a reactor for client-side state Gregory Szorc <gregory.szorc@gmail.com> parents: 37485 diff changeset	1217
01361be9e2dc wireproto: introduce a reactor for client-side state Gregory Szorc <gregory.szorc@gmail.com> parents: 37485 diff changeset	1218 self._canissuecommands = True
01361be9e2dc wireproto: introduce a reactor for client-side state Gregory Szorc <gregory.szorc@gmail.com> parents: 37485 diff changeset	1219 self._cansend = True
01361be9e2dc wireproto: introduce a reactor for client-side state Gregory Szorc <gregory.szorc@gmail.com> parents: 37485 diff changeset	1220
01361be9e2dc wireproto: introduce a reactor for client-side state Gregory Szorc <gregory.szorc@gmail.com> parents: 37485 diff changeset	1221 self._nextrequestid = 1
01361be9e2dc wireproto: introduce a reactor for client-side state Gregory Szorc <gregory.szorc@gmail.com> parents: 37485 diff changeset	1222 # We only support a single outgoing stream for now.
01361be9e2dc wireproto: introduce a reactor for client-side state Gregory Szorc <gregory.szorc@gmail.com> parents: 37485 diff changeset	1223 self._outgoingstream = stream(1)
01361be9e2dc wireproto: introduce a reactor for client-side state Gregory Szorc <gregory.szorc@gmail.com> parents: 37485 diff changeset	1224 self._pendingrequests = collections.deque()
01361be9e2dc wireproto: introduce a reactor for client-side state Gregory Szorc <gregory.szorc@gmail.com> parents: 37485 diff changeset	1225 self._activerequests = {}
37544 55b5ba8d4e68 wireproto: client reactor support for receiving frames Gregory Szorc <gregory.szorc@gmail.com> parents: 37543 diff changeset	1226 self._incomingstreams = {}
37543 01361be9e2dc wireproto: introduce a reactor for client-side state Gregory Szorc <gregory.szorc@gmail.com> parents: 37485 diff changeset	1227
40025 86b22a4cfab1 wireprotov2: client support for advertising redirect targets Gregory Szorc <gregory.szorc@gmail.com> parents: 40021 diff changeset	1228 def callcommand(self, name, args, datafh=None, redirect=None):
37543 01361be9e2dc wireproto: introduce a reactor for client-side state Gregory Szorc <gregory.szorc@gmail.com> parents: 37485 diff changeset	1229 """Request that a command be executed.
01361be9e2dc wireproto: introduce a reactor for client-side state Gregory Szorc <gregory.szorc@gmail.com> parents: 37485 diff changeset	1230
01361be9e2dc wireproto: introduce a reactor for client-side state Gregory Szorc <gregory.szorc@gmail.com> parents: 37485 diff changeset	1231 Receives the command name, a dict of arguments to pass to the command,
01361be9e2dc wireproto: introduce a reactor for client-side state Gregory Szorc <gregory.szorc@gmail.com> parents: 37485 diff changeset	1232 and an optional file object containing the raw data for the command.
01361be9e2dc wireproto: introduce a reactor for client-side state Gregory Szorc <gregory.szorc@gmail.com> parents: 37485 diff changeset	1233
01361be9e2dc wireproto: introduce a reactor for client-side state Gregory Szorc <gregory.szorc@gmail.com> parents: 37485 diff changeset	1234 Returns a 3-tuple of (request, action, action data).
01361be9e2dc wireproto: introduce a reactor for client-side state Gregory Szorc <gregory.szorc@gmail.com> parents: 37485 diff changeset	1235 """
01361be9e2dc wireproto: introduce a reactor for client-side state Gregory Szorc <gregory.szorc@gmail.com> parents: 37485 diff changeset	1236 if not self._canissuecommands:
01361be9e2dc wireproto: introduce a reactor for client-side state Gregory Szorc <gregory.szorc@gmail.com> parents: 37485 diff changeset	1237 raise error.ProgrammingError('cannot issue new commands')
01361be9e2dc wireproto: introduce a reactor for client-side state Gregory Szorc <gregory.szorc@gmail.com> parents: 37485 diff changeset	1238
01361be9e2dc wireproto: introduce a reactor for client-side state Gregory Szorc <gregory.szorc@gmail.com> parents: 37485 diff changeset	1239 requestid = self._nextrequestid
01361be9e2dc wireproto: introduce a reactor for client-side state Gregory Szorc <gregory.szorc@gmail.com> parents: 37485 diff changeset	1240 self._nextrequestid += 2
01361be9e2dc wireproto: introduce a reactor for client-side state Gregory Szorc <gregory.szorc@gmail.com> parents: 37485 diff changeset	1241
40025 86b22a4cfab1 wireprotov2: client support for advertising redirect targets Gregory Szorc <gregory.szorc@gmail.com> parents: 40021 diff changeset	1242 request = commandrequest(requestid, name, args, datafh=datafh,
86b22a4cfab1 wireprotov2: client support for advertising redirect targets Gregory Szorc <gregory.szorc@gmail.com> parents: 40021 diff changeset	1243 redirect=redirect)
37543 01361be9e2dc wireproto: introduce a reactor for client-side state Gregory Szorc <gregory.szorc@gmail.com> parents: 37485 diff changeset	1244
01361be9e2dc wireproto: introduce a reactor for client-side state Gregory Szorc <gregory.szorc@gmail.com> parents: 37485 diff changeset	1245 if self._buffersends:
01361be9e2dc wireproto: introduce a reactor for client-side state Gregory Szorc <gregory.szorc@gmail.com> parents: 37485 diff changeset	1246 self._pendingrequests.append(request)
01361be9e2dc wireproto: introduce a reactor for client-side state Gregory Szorc <gregory.szorc@gmail.com> parents: 37485 diff changeset	1247 return request, 'noop', {}
01361be9e2dc wireproto: introduce a reactor for client-side state Gregory Szorc <gregory.szorc@gmail.com> parents: 37485 diff changeset	1248 else:
01361be9e2dc wireproto: introduce a reactor for client-side state Gregory Szorc <gregory.szorc@gmail.com> parents: 37485 diff changeset	1249 if not self._cansend:
01361be9e2dc wireproto: introduce a reactor for client-side state Gregory Szorc <gregory.szorc@gmail.com> parents: 37485 diff changeset	1250 raise error.ProgrammingError('sends cannot be performed on '
01361be9e2dc wireproto: introduce a reactor for client-side state Gregory Szorc <gregory.szorc@gmail.com> parents: 37485 diff changeset	1251 'this instance')
01361be9e2dc wireproto: introduce a reactor for client-side state Gregory Szorc <gregory.szorc@gmail.com> parents: 37485 diff changeset	1252
01361be9e2dc wireproto: introduce a reactor for client-side state Gregory Szorc <gregory.szorc@gmail.com> parents: 37485 diff changeset	1253 if not self._hasmultiplesend:
01361be9e2dc wireproto: introduce a reactor for client-side state Gregory Szorc <gregory.szorc@gmail.com> parents: 37485 diff changeset	1254 self._cansend = False
01361be9e2dc wireproto: introduce a reactor for client-side state Gregory Szorc <gregory.szorc@gmail.com> parents: 37485 diff changeset	1255 self._canissuecommands = False
01361be9e2dc wireproto: introduce a reactor for client-side state Gregory Szorc <gregory.szorc@gmail.com> parents: 37485 diff changeset	1256
01361be9e2dc wireproto: introduce a reactor for client-side state Gregory Szorc <gregory.szorc@gmail.com> parents: 37485 diff changeset	1257 return request, 'sendframes', {
01361be9e2dc wireproto: introduce a reactor for client-side state Gregory Szorc <gregory.szorc@gmail.com> parents: 37485 diff changeset	1258 'framegen': self._makecommandframes(request),
01361be9e2dc wireproto: introduce a reactor for client-side state Gregory Szorc <gregory.szorc@gmail.com> parents: 37485 diff changeset	1259 }
01361be9e2dc wireproto: introduce a reactor for client-side state Gregory Szorc <gregory.szorc@gmail.com> parents: 37485 diff changeset	1260
01361be9e2dc wireproto: introduce a reactor for client-side state Gregory Szorc <gregory.szorc@gmail.com> parents: 37485 diff changeset	1261 def flushcommands(self):
01361be9e2dc wireproto: introduce a reactor for client-side state Gregory Szorc <gregory.szorc@gmail.com> parents: 37485 diff changeset	1262 """Request that all queued commands be sent.
01361be9e2dc wireproto: introduce a reactor for client-side state Gregory Szorc <gregory.szorc@gmail.com> parents: 37485 diff changeset	1263
01361be9e2dc wireproto: introduce a reactor for client-side state Gregory Szorc <gregory.szorc@gmail.com> parents: 37485 diff changeset	1264 If any commands are buffered, this will instruct the caller to send
01361be9e2dc wireproto: introduce a reactor for client-side state Gregory Szorc <gregory.szorc@gmail.com> parents: 37485 diff changeset	1265 them over the wire. If no commands are buffered it instructs the client
01361be9e2dc wireproto: introduce a reactor for client-side state Gregory Szorc <gregory.szorc@gmail.com> parents: 37485 diff changeset	1266 to no-op.
01361be9e2dc wireproto: introduce a reactor for client-side state Gregory Szorc <gregory.szorc@gmail.com> parents: 37485 diff changeset	1267
01361be9e2dc wireproto: introduce a reactor for client-side state Gregory Szorc <gregory.szorc@gmail.com> parents: 37485 diff changeset	1268 If instances aren't configured for multiple sends, no new command
01361be9e2dc wireproto: introduce a reactor for client-side state Gregory Szorc <gregory.szorc@gmail.com> parents: 37485 diff changeset	1269 requests are allowed after this is called.
01361be9e2dc wireproto: introduce a reactor for client-side state Gregory Szorc <gregory.szorc@gmail.com> parents: 37485 diff changeset	1270 """
01361be9e2dc wireproto: introduce a reactor for client-side state Gregory Szorc <gregory.szorc@gmail.com> parents: 37485 diff changeset	1271 if not self._pendingrequests:
01361be9e2dc wireproto: introduce a reactor for client-side state Gregory Szorc <gregory.szorc@gmail.com> parents: 37485 diff changeset	1272 return 'noop', {}
01361be9e2dc wireproto: introduce a reactor for client-side state Gregory Szorc <gregory.szorc@gmail.com> parents: 37485 diff changeset	1273
01361be9e2dc wireproto: introduce a reactor for client-side state Gregory Szorc <gregory.szorc@gmail.com> parents: 37485 diff changeset	1274 if not self._cansend:
01361be9e2dc wireproto: introduce a reactor for client-side state Gregory Szorc <gregory.szorc@gmail.com> parents: 37485 diff changeset	1275 raise error.ProgrammingError('sends cannot be performed on this '
01361be9e2dc wireproto: introduce a reactor for client-side state Gregory Szorc <gregory.szorc@gmail.com> parents: 37485 diff changeset	1276 'instance')
01361be9e2dc wireproto: introduce a reactor for client-side state Gregory Szorc <gregory.szorc@gmail.com> parents: 37485 diff changeset	1277
01361be9e2dc wireproto: introduce a reactor for client-side state Gregory Szorc <gregory.szorc@gmail.com> parents: 37485 diff changeset	1278 # If the instance only allows sending once, mark that we have fired
01361be9e2dc wireproto: introduce a reactor for client-side state Gregory Szorc <gregory.szorc@gmail.com> parents: 37485 diff changeset	1279 # our one shot.
01361be9e2dc wireproto: introduce a reactor for client-side state Gregory Szorc <gregory.szorc@gmail.com> parents: 37485 diff changeset	1280 if not self._hasmultiplesend:
01361be9e2dc wireproto: introduce a reactor for client-side state Gregory Szorc <gregory.szorc@gmail.com> parents: 37485 diff changeset	1281 self._canissuecommands = False
01361be9e2dc wireproto: introduce a reactor for client-side state Gregory Szorc <gregory.szorc@gmail.com> parents: 37485 diff changeset	1282 self._cansend = False
01361be9e2dc wireproto: introduce a reactor for client-side state Gregory Szorc <gregory.szorc@gmail.com> parents: 37485 diff changeset	1283
01361be9e2dc wireproto: introduce a reactor for client-side state Gregory Szorc <gregory.szorc@gmail.com> parents: 37485 diff changeset	1284 def makeframes():
01361be9e2dc wireproto: introduce a reactor for client-side state Gregory Szorc <gregory.szorc@gmail.com> parents: 37485 diff changeset	1285 while self._pendingrequests:
01361be9e2dc wireproto: introduce a reactor for client-side state Gregory Szorc <gregory.szorc@gmail.com> parents: 37485 diff changeset	1286 request = self._pendingrequests.popleft()
01361be9e2dc wireproto: introduce a reactor for client-side state Gregory Szorc <gregory.szorc@gmail.com> parents: 37485 diff changeset	1287 for frame in self._makecommandframes(request):
01361be9e2dc wireproto: introduce a reactor for client-side state Gregory Szorc <gregory.szorc@gmail.com> parents: 37485 diff changeset	1288 yield frame
01361be9e2dc wireproto: introduce a reactor for client-side state Gregory Szorc <gregory.szorc@gmail.com> parents: 37485 diff changeset	1289
01361be9e2dc wireproto: introduce a reactor for client-side state Gregory Szorc <gregory.szorc@gmail.com> parents: 37485 diff changeset	1290 return 'sendframes', {
01361be9e2dc wireproto: introduce a reactor for client-side state Gregory Szorc <gregory.szorc@gmail.com> parents: 37485 diff changeset	1291 'framegen': makeframes(),
01361be9e2dc wireproto: introduce a reactor for client-side state Gregory Szorc <gregory.szorc@gmail.com> parents: 37485 diff changeset	1292 }
01361be9e2dc wireproto: introduce a reactor for client-side state Gregory Szorc <gregory.szorc@gmail.com> parents: 37485 diff changeset	1293
01361be9e2dc wireproto: introduce a reactor for client-side state Gregory Szorc <gregory.szorc@gmail.com> parents: 37485 diff changeset	1294 def _makecommandframes(self, request):
01361be9e2dc wireproto: introduce a reactor for client-side state Gregory Szorc <gregory.szorc@gmail.com> parents: 37485 diff changeset	1295 """Emit frames to issue a command request.
01361be9e2dc wireproto: introduce a reactor for client-side state Gregory Szorc <gregory.szorc@gmail.com> parents: 37485 diff changeset	1296
01361be9e2dc wireproto: introduce a reactor for client-side state Gregory Szorc <gregory.szorc@gmail.com> parents: 37485 diff changeset	1297 As a side-effect, update request accounting to reflect its changed
01361be9e2dc wireproto: introduce a reactor for client-side state Gregory Szorc <gregory.szorc@gmail.com> parents: 37485 diff changeset	1298 state.
01361be9e2dc wireproto: introduce a reactor for client-side state Gregory Szorc <gregory.szorc@gmail.com> parents: 37485 diff changeset	1299 """
01361be9e2dc wireproto: introduce a reactor for client-side state Gregory Szorc <gregory.szorc@gmail.com> parents: 37485 diff changeset	1300 self._activerequests[request.requestid] = request
01361be9e2dc wireproto: introduce a reactor for client-side state Gregory Szorc <gregory.szorc@gmail.com> parents: 37485 diff changeset	1301 request.state = 'sending'
01361be9e2dc wireproto: introduce a reactor for client-side state Gregory Szorc <gregory.szorc@gmail.com> parents: 37485 diff changeset	1302
01361be9e2dc wireproto: introduce a reactor for client-side state Gregory Szorc <gregory.szorc@gmail.com> parents: 37485 diff changeset	1303 res = createcommandframes(self._outgoingstream,
01361be9e2dc wireproto: introduce a reactor for client-side state Gregory Szorc <gregory.szorc@gmail.com> parents: 37485 diff changeset	1304 request.requestid,
01361be9e2dc wireproto: introduce a reactor for client-side state Gregory Szorc <gregory.szorc@gmail.com> parents: 37485 diff changeset	1305 request.name,
01361be9e2dc wireproto: introduce a reactor for client-side state Gregory Szorc <gregory.szorc@gmail.com> parents: 37485 diff changeset	1306 request.args,
40025 86b22a4cfab1 wireprotov2: client support for advertising redirect targets Gregory Szorc <gregory.szorc@gmail.com> parents: 40021 diff changeset	1307 datafh=request.datafh,
86b22a4cfab1 wireprotov2: client support for advertising redirect targets Gregory Szorc <gregory.szorc@gmail.com> parents: 40021 diff changeset	1308 redirect=request.redirect)
37543 01361be9e2dc wireproto: introduce a reactor for client-side state Gregory Szorc <gregory.szorc@gmail.com> parents: 37485 diff changeset	1309
01361be9e2dc wireproto: introduce a reactor for client-side state Gregory Szorc <gregory.szorc@gmail.com> parents: 37485 diff changeset	1310 for frame in res:
01361be9e2dc wireproto: introduce a reactor for client-side state Gregory Szorc <gregory.szorc@gmail.com> parents: 37485 diff changeset	1311 yield frame
01361be9e2dc wireproto: introduce a reactor for client-side state Gregory Szorc <gregory.szorc@gmail.com> parents: 37485 diff changeset	1312
01361be9e2dc wireproto: introduce a reactor for client-side state Gregory Szorc <gregory.szorc@gmail.com> parents: 37485 diff changeset	1313 request.state = 'sent'
37544 55b5ba8d4e68 wireproto: client reactor support for receiving frames Gregory Szorc <gregory.szorc@gmail.com> parents: 37543 diff changeset	1314
55b5ba8d4e68 wireproto: client reactor support for receiving frames Gregory Szorc <gregory.szorc@gmail.com> parents: 37543 diff changeset	1315 def onframerecv(self, frame):
55b5ba8d4e68 wireproto: client reactor support for receiving frames Gregory Szorc <gregory.szorc@gmail.com> parents: 37543 diff changeset	1316 """Process a frame that has been received off the wire.
55b5ba8d4e68 wireproto: client reactor support for receiving frames Gregory Szorc <gregory.szorc@gmail.com> parents: 37543 diff changeset	1317
55b5ba8d4e68 wireproto: client reactor support for receiving frames Gregory Szorc <gregory.szorc@gmail.com> parents: 37543 diff changeset	1318 Returns a 2-tuple of (action, meta) describing further action the
55b5ba8d4e68 wireproto: client reactor support for receiving frames Gregory Szorc <gregory.szorc@gmail.com> parents: 37543 diff changeset	1319 caller needs to take as a result of receiving this frame.
55b5ba8d4e68 wireproto: client reactor support for receiving frames Gregory Szorc <gregory.szorc@gmail.com> parents: 37543 diff changeset	1320 """
55b5ba8d4e68 wireproto: client reactor support for receiving frames Gregory Szorc <gregory.szorc@gmail.com> parents: 37543 diff changeset	1321 if frame.streamid % 2:
55b5ba8d4e68 wireproto: client reactor support for receiving frames Gregory Szorc <gregory.szorc@gmail.com> parents: 37543 diff changeset	1322 return 'error', {
55b5ba8d4e68 wireproto: client reactor support for receiving frames Gregory Szorc <gregory.szorc@gmail.com> parents: 37543 diff changeset	1323 'message': (
55b5ba8d4e68 wireproto: client reactor support for receiving frames Gregory Szorc <gregory.szorc@gmail.com> parents: 37543 diff changeset	1324 _('received frame with odd numbered stream ID: %d') %
55b5ba8d4e68 wireproto: client reactor support for receiving frames Gregory Szorc <gregory.szorc@gmail.com> parents: 37543 diff changeset	1325 frame.streamid),
55b5ba8d4e68 wireproto: client reactor support for receiving frames Gregory Szorc <gregory.szorc@gmail.com> parents: 37543 diff changeset	1326 }
55b5ba8d4e68 wireproto: client reactor support for receiving frames Gregory Szorc <gregory.szorc@gmail.com> parents: 37543 diff changeset	1327
55b5ba8d4e68 wireproto: client reactor support for receiving frames Gregory Szorc <gregory.szorc@gmail.com> parents: 37543 diff changeset	1328 if frame.streamid not in self._incomingstreams:
55b5ba8d4e68 wireproto: client reactor support for receiving frames Gregory Szorc <gregory.szorc@gmail.com> parents: 37543 diff changeset	1329 if not frame.streamflags & STREAM_FLAG_BEGIN_STREAM:
55b5ba8d4e68 wireproto: client reactor support for receiving frames Gregory Szorc <gregory.szorc@gmail.com> parents: 37543 diff changeset	1330 return 'error', {
55b5ba8d4e68 wireproto: client reactor support for receiving frames Gregory Szorc <gregory.szorc@gmail.com> parents: 37543 diff changeset	1331 'message': _('received frame on unknown stream '
55b5ba8d4e68 wireproto: client reactor support for receiving frames Gregory Szorc <gregory.szorc@gmail.com> parents: 37543 diff changeset	1332 'without beginning of stream flag set'),
55b5ba8d4e68 wireproto: client reactor support for receiving frames Gregory Szorc <gregory.szorc@gmail.com> parents: 37543 diff changeset	1333 }
55b5ba8d4e68 wireproto: client reactor support for receiving frames Gregory Szorc <gregory.szorc@gmail.com> parents: 37543 diff changeset	1334
37656 e6870bca1f47 wireprotoframing: record when new stream is encountered Gregory Szorc <gregory.szorc@gmail.com> parents: 37655 diff changeset	1335 self._incomingstreams[frame.streamid] = stream(frame.streamid)
e6870bca1f47 wireprotoframing: record when new stream is encountered Gregory Szorc <gregory.szorc@gmail.com> parents: 37655 diff changeset	1336
37544 55b5ba8d4e68 wireproto: client reactor support for receiving frames Gregory Szorc <gregory.szorc@gmail.com> parents: 37543 diff changeset	1337 if frame.streamflags & STREAM_FLAG_ENCODING_APPLIED:
55b5ba8d4e68 wireproto: client reactor support for receiving frames Gregory Szorc <gregory.szorc@gmail.com> parents: 37543 diff changeset	1338 raise error.ProgrammingError('support for decoding stream '
55b5ba8d4e68 wireproto: client reactor support for receiving frames Gregory Szorc <gregory.szorc@gmail.com> parents: 37543 diff changeset	1339 'payloads not yet implemneted')
55b5ba8d4e68 wireproto: client reactor support for receiving frames Gregory Szorc <gregory.szorc@gmail.com> parents: 37543 diff changeset	1340
55b5ba8d4e68 wireproto: client reactor support for receiving frames Gregory Szorc <gregory.szorc@gmail.com> parents: 37543 diff changeset	1341 if frame.streamflags & STREAM_FLAG_END_STREAM:
55b5ba8d4e68 wireproto: client reactor support for receiving frames Gregory Szorc <gregory.szorc@gmail.com> parents: 37543 diff changeset	1342 del self._incomingstreams[frame.streamid]
55b5ba8d4e68 wireproto: client reactor support for receiving frames Gregory Szorc <gregory.szorc@gmail.com> parents: 37543 diff changeset	1343
55b5ba8d4e68 wireproto: client reactor support for receiving frames Gregory Szorc <gregory.szorc@gmail.com> parents: 37543 diff changeset	1344 if frame.requestid not in self._activerequests:
55b5ba8d4e68 wireproto: client reactor support for receiving frames Gregory Szorc <gregory.szorc@gmail.com> parents: 37543 diff changeset	1345 return 'error', {
55b5ba8d4e68 wireproto: client reactor support for receiving frames Gregory Szorc <gregory.szorc@gmail.com> parents: 37543 diff changeset	1346 'message': (_('received frame for inactive request ID: %d') %
55b5ba8d4e68 wireproto: client reactor support for receiving frames Gregory Szorc <gregory.szorc@gmail.com> parents: 37543 diff changeset	1347 frame.requestid),
55b5ba8d4e68 wireproto: client reactor support for receiving frames Gregory Szorc <gregory.szorc@gmail.com> parents: 37543 diff changeset	1348 }
55b5ba8d4e68 wireproto: client reactor support for receiving frames Gregory Szorc <gregory.szorc@gmail.com> parents: 37543 diff changeset	1349
55b5ba8d4e68 wireproto: client reactor support for receiving frames Gregory Szorc <gregory.szorc@gmail.com> parents: 37543 diff changeset	1350 request = self._activerequests[frame.requestid]
55b5ba8d4e68 wireproto: client reactor support for receiving frames Gregory Szorc <gregory.szorc@gmail.com> parents: 37543 diff changeset	1351 request.state = 'receiving'
55b5ba8d4e68 wireproto: client reactor support for receiving frames Gregory Szorc <gregory.szorc@gmail.com> parents: 37543 diff changeset	1352
55b5ba8d4e68 wireproto: client reactor support for receiving frames Gregory Szorc <gregory.szorc@gmail.com> parents: 37543 diff changeset	1353 handlers = {
37724 deff7cf7eefd wireprotov2: change frame type and name for command response Gregory Szorc <gregory.szorc@gmail.com> parents: 37723 diff changeset	1354 FRAME_TYPE_COMMAND_RESPONSE: self._oncommandresponseframe,
37726 0c184ca594bb wireprotov2: change behavior of error frame Gregory Szorc <gregory.szorc@gmail.com> parents: 37725 diff changeset	1355 FRAME_TYPE_ERROR_RESPONSE: self._onerrorresponseframe,
37544 55b5ba8d4e68 wireproto: client reactor support for receiving frames Gregory Szorc <gregory.szorc@gmail.com> parents: 37543 diff changeset	1356 }
55b5ba8d4e68 wireproto: client reactor support for receiving frames Gregory Szorc <gregory.szorc@gmail.com> parents: 37543 diff changeset	1357
55b5ba8d4e68 wireproto: client reactor support for receiving frames Gregory Szorc <gregory.szorc@gmail.com> parents: 37543 diff changeset	1358 meth = handlers.get(frame.typeid)
55b5ba8d4e68 wireproto: client reactor support for receiving frames Gregory Szorc <gregory.szorc@gmail.com> parents: 37543 diff changeset	1359 if not meth:
55b5ba8d4e68 wireproto: client reactor support for receiving frames Gregory Szorc <gregory.szorc@gmail.com> parents: 37543 diff changeset	1360 raise error.ProgrammingError('unhandled frame type: %d' %
55b5ba8d4e68 wireproto: client reactor support for receiving frames Gregory Szorc <gregory.szorc@gmail.com> parents: 37543 diff changeset	1361 frame.typeid)
55b5ba8d4e68 wireproto: client reactor support for receiving frames Gregory Szorc <gregory.szorc@gmail.com> parents: 37543 diff changeset	1362
55b5ba8d4e68 wireproto: client reactor support for receiving frames Gregory Szorc <gregory.szorc@gmail.com> parents: 37543 diff changeset	1363 return meth(request, frame)
55b5ba8d4e68 wireproto: client reactor support for receiving frames Gregory Szorc <gregory.szorc@gmail.com> parents: 37543 diff changeset	1364
37724 deff7cf7eefd wireprotov2: change frame type and name for command response Gregory Szorc <gregory.szorc@gmail.com> parents: 37723 diff changeset	1365 def _oncommandresponseframe(self, request, frame):
deff7cf7eefd wireprotov2: change frame type and name for command response Gregory Szorc <gregory.szorc@gmail.com> parents: 37723 diff changeset	1366 if frame.flags & FLAG_COMMAND_RESPONSE_EOS:
37544 55b5ba8d4e68 wireproto: client reactor support for receiving frames Gregory Szorc <gregory.szorc@gmail.com> parents: 37543 diff changeset	1367 request.state = 'received'
55b5ba8d4e68 wireproto: client reactor support for receiving frames Gregory Szorc <gregory.szorc@gmail.com> parents: 37543 diff changeset	1368 del self._activerequests[request.requestid]
55b5ba8d4e68 wireproto: client reactor support for receiving frames Gregory Szorc <gregory.szorc@gmail.com> parents: 37543 diff changeset	1369
55b5ba8d4e68 wireproto: client reactor support for receiving frames Gregory Szorc <gregory.szorc@gmail.com> parents: 37543 diff changeset	1370 return 'responsedata', {
55b5ba8d4e68 wireproto: client reactor support for receiving frames Gregory Szorc <gregory.szorc@gmail.com> parents: 37543 diff changeset	1371 'request': request,
37724 deff7cf7eefd wireprotov2: change frame type and name for command response Gregory Szorc <gregory.szorc@gmail.com> parents: 37723 diff changeset	1372 'expectmore': frame.flags & FLAG_COMMAND_RESPONSE_CONTINUATION,
deff7cf7eefd wireprotov2: change frame type and name for command response Gregory Szorc <gregory.szorc@gmail.com> parents: 37723 diff changeset	1373 'eos': frame.flags & FLAG_COMMAND_RESPONSE_EOS,
37544 55b5ba8d4e68 wireproto: client reactor support for receiving frames Gregory Szorc <gregory.szorc@gmail.com> parents: 37543 diff changeset	1374 'data': frame.payload,
55b5ba8d4e68 wireproto: client reactor support for receiving frames Gregory Szorc <gregory.szorc@gmail.com> parents: 37543 diff changeset	1375 }
37726 0c184ca594bb wireprotov2: change behavior of error frame Gregory Szorc <gregory.szorc@gmail.com> parents: 37725 diff changeset	1376
0c184ca594bb wireprotov2: change behavior of error frame Gregory Szorc <gregory.szorc@gmail.com> parents: 37725 diff changeset	1377 def _onerrorresponseframe(self, request, frame):
0c184ca594bb wireprotov2: change behavior of error frame Gregory Szorc <gregory.szorc@gmail.com> parents: 37725 diff changeset	1378 request.state = 'errored'
0c184ca594bb wireprotov2: change behavior of error frame Gregory Szorc <gregory.szorc@gmail.com> parents: 37725 diff changeset	1379 del self._activerequests[request.requestid]
0c184ca594bb wireprotov2: change behavior of error frame Gregory Szorc <gregory.szorc@gmail.com> parents: 37725 diff changeset	1380
0c184ca594bb wireprotov2: change behavior of error frame Gregory Szorc <gregory.szorc@gmail.com> parents: 37725 diff changeset	1381 # The payload should be a CBOR map.
39465 36f487a332ad wireprotoframing: use our CBOR module Gregory Szorc <gregory.szorc@gmail.com> parents: 37728 diff changeset	1382 m = cborutil.decodeall(frame.payload)[0]
37726 0c184ca594bb wireprotov2: change behavior of error frame Gregory Szorc <gregory.szorc@gmail.com> parents: 37725 diff changeset	1383
0c184ca594bb wireprotov2: change behavior of error frame Gregory Szorc <gregory.szorc@gmail.com> parents: 37725 diff changeset	1384 return 'error', {
0c184ca594bb wireprotov2: change behavior of error frame Gregory Szorc <gregory.szorc@gmail.com> parents: 37725 diff changeset	1385 'request': request,
0c184ca594bb wireprotov2: change behavior of error frame Gregory Szorc <gregory.szorc@gmail.com> parents: 37725 diff changeset	1386 'type': m['type'],
0c184ca594bb wireprotov2: change behavior of error frame Gregory Szorc <gregory.szorc@gmail.com> parents: 37725 diff changeset	1387 'message': m['message'],
0c184ca594bb wireprotov2: change behavior of error frame Gregory Szorc <gregory.szorc@gmail.com> parents: 37725 diff changeset	1388 }

Mercurial > public > mercurial-scm > hg-stable

annotate mercurial/wireprotoframing.py @ 40026:b099e6032f38